In a Turkish high school trial with about a thousand math students, the ChatGPT group did better homework and felt they were learning more, then scored worse on the exam. The tool supplied answers, and skipping the effort skipped the learning.
A five-month Python course across ten Taipei schools flipped that result: an AI that sequenced personalized problems raised final-exam scores 0.15 standard deviations, roughly six to nine months of extra schooling, with no added teacher time.
Tutor modes exist but are buried. Gemini has Guided Learning under the plus menu, ChatGPT takes /learn in the chatbox, and Claude hides a learning style in the style picker. Pair them with a reasoning modelA model trained to think — generating extended internal reasoning before answering — trading time and tokens for accuracy on hard problems.Full definition → for STEM.
In the BCG experiment with 758 consultants, AI users beat non-users on everything except the one task built to defeat the model. There they did worse, accepting an authoritative wrong answer without checking it.
Anthropic found the antidote is cheap: programmers who asked the AI to explain what it was doing, or who kept part of the work themselves, could still answer questions about the result. Those who just let it run could not.
Terms in this piece · Glossary
agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
reasoning model — A model trained to think — generating extended internal reasoning before answering — trading time and tokens for accuracy on hard problems.
Why it matters
It gives a research-backed framework for using AI on writing and learning tasks in ways that build agent skillA reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.Full definition → instead of eroding it, with specific prompting patterns like activating tutor modes rather than accepting the frictionless default.
Key quotes
“The students with ChatGPT did their homework better and thought they were learning more, but at test time, they underperformed their classmates without ChatGPT.”
“Students who were given a personalized sequence of problems by an AI tutor scored 0.15 standard deviations higher on a final exam taken without AI help.”
“To benefit from AI in learning you need to pivot from using AI to solve problems, to pushing you to solve problems yourself.”
“The AI gave them an authoritative-looking answer that happened to be incorrect, and most of them, the same elite consultants who outperformed on everything else, did not catch it.”
“The point isn't to avoid AI but to be intentional about it by making a conscious choice about AI use, rather than reflexive dependence or reflexive avoidance.”
