The Best Models Still Reason Like Toddlers — Andrew Dai, Elorian
Source
AI Engineer
Author
AI Engineer
Date
Key takeaways · AI-distilled
Andrew Dai's chessboard example: shown only part of a board, a frontier model answers 32 white squares because it recognizes a chessboard and recalls the full count, rather than counting what is actually visible.
His rule of thumb: if a person can answer a visual question in about one second, the model can too; if it takes a human longer, models fall apart, as with counting Catan roads or tracking a robot arm lifting a lid across a long video.
Dai says benchmarks hide the gap: one popular reasoning suite uses 32 by 32 pixel images, and a multimodal science exam can mostly be answered without looking at the image.
Elorian's approach combines visual reasoning data that does not exist online, a synthetic flywheel of evals, agents, SFT and RL, architectural changes on the transformerThe neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.Full definition →, and native visual chain of thought, such as boxing every hotel before narrowing to the red ones.
Terms in this piece · Glossary
hallucination — When a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.
chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.
transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
Why it matters
Points builders of vision-groundingTying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.Full definition → agents to a concrete gap between model 'understanding' and true visual reasoning, and argues current benchmarks (low-res images, answerable-without-image exams) mask the problem.
