Vibeleaderboard
← All Intel
Intel / article

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

Source
arxiv.org
Author
Tri Dao et al.
Date

Context

FlashAttention speeds up by reducing data movement inside a GPU. Attention lets a combine information from different positions in a sequence. The method divides that calculation into tiles, reducing transfers between the GPU's large memory and its smaller on-chip memory while preserving exact attention. The 2022 paper reports faster training across several tested workloads. Its central contribution is showing that the arrangement of memory reads and writes can materially improve execution speed, even when the attention calculation itself remains exact.

Terms in this piece · Glossary
  • attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
  • transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
Recommended reads
Comments

Checking sign-in…

Loading comments…