AMD's inference gap is a software and tooling problem, not a silicon one
- Source
- x.com
- Date
AMD team is grinding hard 🚀🚀 Although, for most SLOs/open models, AMD is behind NVIDIA, we think that AMD's software will do well on agentic workloads in the near future, as AMD engineers are grinding hard using AgentX as the north star benchmark for realistic workloads! In theoretical modelling, AMD hardware should have way better perf per dollar than NVIDIA, especially considering the level of equitable rebates it is giving to Meta & OpenAI, but AMD's software performance is currently the blocker. Really, the main blocker is their leadership not providing the correct tools to their engineers, i.e., internal R&D GPU capacity & stable automated QA GPU clusters.
Context
SemiAnalysis argues that AMD's theoretical hardware economics were not translating into delivered performance. In its August 30 assessment, the publication identifies software performance and engineers' access to dependable development and testing clusters as obstacles.
Its preferred reference is AgentX, which tests extended coding- interactions. These workloads repeatedly reuse earlier and place demands on the surrounding serving software. That explains why attractive chip specifications alone cannot establish the cost of serving an actual workload. The prediction that AMD will improve remains the publication's forecast.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Checking sign-in…
Loading comments…