• benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.

Tenfold growth in how many discrete instructions a model can reliably follow changes what's safe to put in a skills file, but new failure modes (refusal, thinking-budget exhaustion) mean verification still matters.

• 0:00The 200 instruction ceiling and where it came from
• 1:49Not knowing whether it followed your rules
• 2:58How the IFScale benchmark works
• 4:19Replicating last year's result
• 6:12Pointing the same test at current models
• 7:07The headline, a tenfold jump in a year
• 9:01Four models, four different ways to fail
• 14:27What this changes in your workflow
• 16:29Caveats, and what the test does not prove
• 19:32Capacity went up, reliability did not
• 21:06Compression solved, verification remains

• videoBuild-Time vs. Run-Time: Why Dev Tools Fail in Production — Averi Kitsch & Prerna Kakkar, Google
• video500 Skills, Zero Fine-Tuning: LinkedIn's Playbook for AI Agents — Ajay Prakash, LinkedIn
• videoDeep dive on LLM Inference at Scale — Harshul Jain, Audible & Tanmay Sah, Independent AI Researcher
• videoWhy AI Agents Need Million-Token Context — Thomas Wolf & Olive Song, MiniMax

• videoHow I deleted 95% of my agent skills and got better results — Nick Nisi, WorkOSAI Engineer
• videoYour Fine-Tuned Model Is Tech Debt: A 50x ROI House of Cards — Dan Bjornn, Lease EndAI Engineer
• videoComputer Use at the Edge of the Statistical Precipice — Pierluca D'Oro, Programma LabsAI Engineer

Checking sign-in…

Loading comments…