Compute attention with less memory traffic

Standard attention writes a gigabyte-sized score matrix to memory just to read it back. FlashAttention never writes it, and still gets exactly the same answer. Here's the trick.

What you’ll explore

• Tiled attention kernels can preserve dense attention mathematics while avoiding a fully stored score matrix; kernel fusion and overlap improve execution without automatically changing asymptotic arithmetic.