• LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
• KV cache — The memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.
• quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
• attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.

Gives engineers concrete formulas and serving techniques to diagnose and fix memory and throughput failures in self-hosted LLM deployments.

• videoWhy AI Agents Need Million-Token Context — Thomas Wolf & Olive Song, MiniMax
• videoYour company brain will leak secrets: how we stopped it for big banks — Tanmai Gopal, PromptQL
• videoTethered: Our Agents Are Us — Shu Fang, Two Sigma
• videoAgents' next frontier: agent-to-agent and network effects — Jean-Denis Greze, Town

• videoCompression at the Edge — Chris Alexiuk, NVIDIAAI Engineer
• videoKV Cache-Aware Routing and P/D Disaggregation on Kubernetes — Yuchen Fama & Ashish Kamra, Red HatAI Engineer
• videoTraining an LLM from Scratch, Locally — Angelos Perivolaropoulos, ElevenLabsAI Engineer

Checking sign-in…

Loading comments…