Vibeleaderboard
← All Intel
Intel / video

Ex-NVIDIA Engineer: Why AI Is About to Get 1000x Cheaper

Source
youtube.com
Author
Invest Like the Best
Date

Context

AI jobs that run in the background could accept slower individual responses in exchange for cheaper processing. The cost argument in this interview centers on batching: grouping many requests so a graphics processor has more parallel work to perform. That can improve overall hardware utilization while making each request take longer. Listen to the explanation.

The speaker proposes designing services around agents that work for hours, with people checking back later. Such jobs create room to prioritize total work completed per dollar over immediate responses. The proposed savings depend on users having tasks that tolerate the delay; the interview's argument does not establish a universal thousandfold cost reduction. See the background-agent discussion.

Terms in this piece · Glossary
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
  • token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
Read the source www.youtube.com
More from Invest Like the Best
Recommended reads
Comments

Checking sign-in…

Loading comments…