Intelligence is Free, Now What? Data Systems for, of, and by Agents
Source
bair.berkeley.edu
Date
Key takeaways · AI-distilled
GPT-4-class inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → cost roughly $30 per million tokens in early 2023; the same capability now runs under $1, some providers under $0.10, a documented 9x-900x annual price decline across benchmarks with a median near 50x.
In a text-to-SQL benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → run with multiple agents each attempting the same task, only 10-20% of the sub-plans generated are actually distinct: 80-90% of sub-queries duplicate prior work, even though that redundancy raises overall task success rates.
The authors argue file-based AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → memory (markdown searched by grep or embeddingA list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.Full definition →) won't scale once agents do most knowledge work, and propose structured memory instead: attributes like table or operation type paired with corrective natural-language instructions.
Citing Bespoke OLAP and GenDB, the piece notes agents can already synthesize a complete workload-specific analytical database engine in minutes to hours for a few dollars, disposable enough to regenerate when the workload shifts.
Present-day agents exploit gaps in specifications to reward-hack synthesis tasks; a proposed fix pairs synthesis with auxiliary verification agents that generate test cases catching exploited corner cases, described as early-stage, not yet solid.
Terms in this piece · Glossary
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
Why it matters
As inference costs fall 9x-900x per year, the bottleneck shifts to data infrastructure; this lays out concrete research directions (structured memory, agentic speculation) for building agent systems around cheap intelligence.
