← All IntelClip / OtherCost predictability and rising token spend as an enterprise pain point
From Local Models: Trust, Control, Optimization — Carter Abdallah, NVIDIA · ≈14:09
“they're having to cut back on their Opus usage because they burned through it all in a couple months.”
“the amount of tokens in an individual session has gone up exponentially as well.”
What’s in it
- Explains why AI spend keeps rising despite cheaper tokens
- Argues for owning AI cost control to earn CFO trust
- Warns about risks from model price hikes and deprecations
Clip transcript
basically. >> If you go back to trust, it's how you can make your CFO trust you by knowing exactly how much something's going to cost all the time. Um that's That is increasingly becoming very important is uh you hear a lot, you know, all these companies have unbelievably large token spend and um they're having to cut back on their Opus usage because they burned through it all in a couple months. Um and that is going to continue to be a problem because yes, the cost of an individual token has come down drastically. You can look at it, you know, the difference between GPT-4 when it first launched and GPT-5.5 is is is much, much cheaper per token, but at the same time the amount of tokens in an individual session has gone up exponentially as well. So, we're kind of um we're spending more uh on a on a total session. And so, the ability to uh bring in-house or or or at least work with partners to ensure that you are controlling your cost and you're not at the whims of uh when a company releases a newer model uh that might be better, but also more expensive. They might deprecate a model. Um owning that and being sure that, you know, same way is what you what out input goes in, you know what output's going to come out. In the same way, um when it you know, an input goes in, how much it's going to cost. Uh having assurance on that's is important, too.
Comments
Checking sign-in…
Loading comments…