embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
Why it matters
Lets practitioners tweak temperature and top-k/top-p against a real running GPT-2 model while watching each transformerThe neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.Full definition → component fire, making next-tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → prediction concrete rather than abstract.
