• AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
• token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
• speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.
• context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.

Speculative decoding cuts generation latency without changing outputs, and this is a production account of where acceptance rates pay off for long-context agent responses.

• articleHow We Leveraged Vllm To Power Our Genai Applications
• articleThe Linkedin Generative Ai Application Tech Stack Extending To Build Ai Agents
• articleTurbocharging Linkedins Recommendation Systems With Sglang
• articleHow We Engineered Linkedins Hiring Assistant

• articleHow We Engineered Linkedins Hiring AssistantLinkedIn Engineering
• articleFast Inference from Transformers via Speculative DecodingYaniv Leviathan et al.
• articleFaster Than Light Optimizing Generative Recommender Training Efficiency At LinkedinLinkedIn Engineering

Checking sign-in…

Loading comments…