← All IntelClip / AI Agents
Why frontier agents hold the long-horizon gap
From Claude for Long-Horizon Tasks — Lance Martin, Anthropic · ≈20:00
Argues the 12-plus-hour METR regime isn't purely model capability — memory of your preferences, prompt-injection resistance, and brain/hands decoupling all have to ship together.
What’s in it
- Argues the 12-plus-hour METR regime isn't purely model capability — memory of your preferences, prompt-injection resistance, and brain/hands decoupling all have to ship together.
Clip transcript
then how long the frontier will be able to maintain that gap? >> I see. So, the question was kind of on the the gap between the frontier models kind of on for example, a benchmark like meter on like long horizon tasks. >> Yeah. >> Okay. Um, so, in the latest results that I saw from like for example, Codex like 56, I think I'd also kind of is in that 12 plus hour regime on meter. So, I think you're right that like the frontier models like the, you know, mythos class models, strong models from Open AI kind of are in this like 12 plus hour regime. Um, why is it that non-frontier models are not kind of in that regime? I am actually not necessarily sure. I do think I do think um, in order to build agents that can effectively operate in this regime, it's important to note that it's not just the model capability. Like for example, with a Claude tag product, it's actually a combination of improvement in memory because memory is very important. If you have agents working for for example, 12 hours, you want to make sure that your productivity preferences are well encoded in memory. So, it knows when to reach out to you if it gets stuck for example. So, memory is very important, security is very important, so resistance to prompt injection. Um also like model architecture, like kind of the agent architecture is very important, that decoupling of brain and hand, so it's secure and safe and like resistant to failure. So, actually I think to build real agents that can operate in these long time horizons, a bunch of things need to come together in terms of like architecture, infrastructure, security, memory. And that might be why and and but Frontier Labs invested in all these areas, so that might be why it's become a gap in terms of like the agent products that we've released. And so we spent a lot of time, for example, building managed agents to kind of have these kind of considerations baked in. Yeah.
Recommended reads
Comments
Checking sign-in…
Loading comments…
