A Harvard study prices agent delegation against chat search
- Source
- x.com
- Date
We published new research with Harvard on the shift from chat interfaces to autonomous agents like Computer. Over 3 months, findings show workers using Computer finish tasks in 87% less time at 94% lower cost than Search alone, with higher satisfaction. https://t.co/qmcUqcj8CI
To compare the two products, we built 10,000 matched pairs where the same user asked near-identical queries to both Search and Computer, and Computer ran at least one execution tool. We then priced human time using BLS wages.
Search is cheap to start but costly per step, since the human runs each one. Computer costs more upfront to delegate and verify, but little per step, since the agent runs the steps. It pays off once a task is long enough to clear that fixed cost, expanding the affordable frontier of work worth attempting.
Cumulative Computer queries reached 84x their first-week level by the end of the study. Most of them knowledge work queries. Research and analysis (25.8%) and document and asset creation (18.6%) lead, concentrated in software, finance, and marketing.
Context
Perplexity's reported 87% time saving and 94% cost reduction for Computer are estimates of task economics, not directly timed results. Its research account compares matched pairs of near-identical requests sent to Computer and to Search (the thread cites 10,000 pairs), then estimates how long a person would need to perform by hand the execution steps that Search leaves to the user. Labor is priced using occupational wage data.
The proposed advantage comes from shifting human effort toward specifying the task and checking the output while the performs intermediate steps. This helps explain why delegation could pay off on longer workflows. Because the researchers could not directly observe the human-only completion time, the percentages depend on their time and supervision assumptions.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Checking sign-in…
Loading comments…