← All IntelClip / OtherCognition's early Devin economics: buying raw compute instead of per-token pricing
From The State of Model Routing — NVIDIA, Cognition, OpenRouter · ≈36:39
“One of the key things that let us build Devin and build these first agents was we actually bought direct compute capacity from these providers, and instead of paying on a per-token basis, we just paid for the underlying compute, knowing that the economics of the compute was that we were actually paying far less for for the cash tokens that we'd send over.”
“This was before cash tokens was a thing that API providers paid for.”
What’s in it
- Explains how Cognition beat pre-caching token costs to build Devin
- Reveals they bought raw compute instead of paying per-token API prices
- Teases idea of backing prompt caches to S3-style storage for longer retention
Clip transcript
for your use, and you'll likely pay much less. >> Yeah. This is like kind of like the level of like, you know, hardware-software frontier that we kind of like think about. When When we started Cognition, we were working on the first agents, I think one reason why no one else worked on agents is they were just extremely expensive. This was before cash tokens was a thing that API providers paid for. Like if you were sending 100,000 tokens and the same 100,000 tokens, you were paying full price for those tokens back in 2024 when we started. One of the key things that let us build Devin and build these first agents was we actually bought direct compute capacity from these providers, and instead of paying on a per-token basis, we just paid for the underlying compute, knowing that the economics of the compute was that we were actually paying far less for for the cash tokens that we'd send over. Um And And then nowadays, you know, that uh you know, there's there's like similar, you know, things I would like about like, you know, having a version of the cache that maybe like you can just back out to like storage in S3 or something and just like hold for much longer. >> Yeah. Now, this is not extremely
Comments
Sign in to comment.
Loading comments…