Profiling in PyTorch (Part 3): Attention is all you profile
Source
huggingface.co
Date
Terms in this piece · Glossary
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
Why it matters
Shows what each attentionThe mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.Full definition → implementation actually costs at the kernel level, so you can confirm whether SDPA is dispatching to the backend you assume and where the time really goes.
