Stop Guessing Your vLLM Config: Benchmarking LLMs on Runpod - Chris Vela, Runpod
Source
youtube.com
Author
AI Engineer
Date
Why it matters
Shows how to pick vLLM settings from measured concurrency and latency instead of defaults, including how max batch tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → affects inter-token latency. Helps teams self-hosting open models.
Terms in this piece · Glossary
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
videoWho Is Making This Request? Agent Auth 101 - Kamil Potrec, Keycard
videoStop Giving Your Agent Tools. Give It a Sandbox - Mikyo King, Arize AI
videoWe Kept Rebuilding Our AI Agent. Here's What Survived - Fuad Ali, Arize AI
videoExploits Now Beat Patches: Open Source Security in the AI Era - Patrick Smyth, Chainguard