AgentX: an open benchmark for million-token agentic inference

Hardware and serving comparisons built on single-turn 8k/1k traffic mislead for agentic workloads; this one models KV-cache reuse and sub-agent bursts, and the dataset and dashboard are open.

• benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
• inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
• context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
• subagent — A separate agent spawned by another to do one scoped piece of work in its own context, returning only the result.

• postOpen models are eating easy tasks. Do frontier labs outgrow it?
• postSemiAnalysis: a $200 Claude plan can return $12,000 in API-priced tokens
• postWhy a subscription plan has no fixed API-equivalent value
• postAI is flooding ML conferences with papers faster than reviewers can scale

• postAgentX benchmarks inference against real coding-agent tracesSemiAnalysis
• articleNVIDIA Vera Rubin and Blackwell Set a New Standard for Agentic AI Performance per WattElizabeth Goodman
• articleBenchmarking inference at scale: coding agentswww.together.ai

Checking sign-in…

Loading comments…