Scaling to Long Horizons — Ross Taylor & Chengxi Taylor, General Reasoning
Source
AI Engineer
Author
AI Engineer
Date
Terms in this piece · Glossary
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Why it matters
Long-horizon RL is where AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → reliability is currently being won or lost.
Key quotes
“So again, that also shows you how powerful RL is. You can have a sota base model, but that is not enough.”
Ross Taylor
“Like a 1 billion parameter model with RLHF was outperforming 175 billion models. So two orders of magnitude fewer parameters, but getting better results.”
Ross Taylor
“it was just like the bitter lesson, like the most purest form of bitter lesson possible. Like, better base models, more RL computes, bigger context windows, and that's all you need for this kind of emergent behavior.”
Ross Taylor
“Long horizon task is not just an engineering problem. It is a mindset.”
Chengxi Taylor
“we gave all the frontier models a 100K to start. All of them lost”
