The Future of Meta Superintelligence: A 1 Year Progress Update
Source
Max Kan
Author
Max Kan
Date
Key takeaways · AI-distilled
Authoring a good RL task is hard because difficulty can only be calibrationHow well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.Full definition → by trial: too easy and there is nothing to learn, too hard and the model never earns reward. Mechanize pays software engineers $400k+ and expects one usable task per week.
The knock on benchmarks like GDPval is over-specification. Real knowledge work is hard because context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → is scattered across email, texts, and random sites. Hand the model a 1,000-word spec and you have deleted the part that was difficult.
Rubric graders can be bootstrapped from traces. Collect thousands of people doing roughly the same task and you cover the action space well enough for an LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → to draft the rubric; humans then review it and, crucially, weight the criteria.
RL training data goes stale fast. A realistic coding task today involves orchestrating subagents, a workflow that did not exist seven months ago, so a task set only stays representative with a continuous stream of fresh recordings.
The human-data supply chain became a real industry: Mercor, Surge, and Handshake are each past $1B ARR, with barely year-old entrants like Fleet, Mechanize, and AfterQuery near $100M. Demand for RL environments manufactured the supply.
Terms in this piece · Glossary
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
Why it matters
Offers concrete, evidence-backed tracking of Meta's compute and data strategy — including its RL-environment data moat via employee screen recording — that helps engineers gauge where frontier model capability and compute access may shift next.
