The Second Half

Shunyu Yao's The Second Half argues that AI research needs evaluations closer to the conditions in which people actually use assistants. The April 2025 essay identifies two gaps: many tests omit ongoing interaction with a user, and they score tasks independently even when real workers learn from earlier tasks.

An assistant that improves as it becomes familiar with a project could have that benefit missed by a test that resets its experience each time. Yao's proposal is to change what evaluations ask models to do, making collaboration and accumulated experience part of the task. This is a research argument about how to measure usefulness, not a measured result.