Tri Dao

Dao’s CV records a mathematics bachelor’s degree and computer-science master’s degree from Stanford, both completed in June 2016, followed by a statistics master’s in January 2019 and computer-science doctorate in June 2023. His dissertation concerns hardware-aware algorithms for efficient machine learning. His research agenda focuses on efficient training and inference, including sequence models that retain information over long inputs. The common question is how to make useful model computation fit the constraints of the machine running it.[2][1]

Dao’s CV dates his Together AI co-founder and chief scientist role to July 2023, and the company’s current leadership page confirms the title. Princeton’s directory now identifies him as a visiting research scholar and explicitly says he was previously an assistant professor. His personal page and CV still list the assistant professorship, which the CV dates from September 2024. The transition date is not documented in these sources.[2][4][3][1]

FlashAttention reduces expensive memory movement

Dao is first author of FlashAttention, which accelerates the attention calculation used in Transformers. Attention lets a model combine information from different positions in an input sequence. The paper’s central systems insight is that counting arithmetic operations alone misses the cost of repeatedly moving data between GPU memory levels. FlashAttention works on tiles to reduce those transfers while computing exact attention. This makes the method useful without requiring the approximation trade-off of replacing attention with a different mathematical operation. Its reported speedups are measurements for particular workloads, not a fixed gain on every model.[5]

Mamba changes how a sequence is represented

Dao and Albert Gu co-authored Mamba, which follows a different route to handling long sequences. It uses a selective state-space model: the input determines what information the running state retains or forgets. The authors combine this selection mechanism with a hardware-aware parallel algorithm. The resulting architecture avoids attention and scales linearly with sequence length. The paper evaluates language, audio, and genomics tasks. Mamba is an architectural alternative, whereas FlashAttention improves the execution of an attention operation.[6]

The hardware target keeps changing

Dao also co-authored FlashAttention-4, which adapts attention computation to Nvidia Blackwell GPUs. The paper explains that matrix-computation throughput grew faster than some other hardware resources, changing which operations became bottlenecks. Its techniques revise execution pipelines and reduce other operations and memory traffic; the implementation uses CuTe-DSL in Python. Start with the original FlashAttention paper to understand memory movement, then read this follow-up to see why a faster GPU can require a different implementation. Familiarity with Transformers and GPU memory makes the technical sections easier.[7][5]