Shows how test-time reasoning models perform on tasks that need novel adaptation rather than memorized patterns. That helps engineers calibrate what reasoning models can and cannot generalize to.
Terms in this piece · Glossary
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
