Holding the agent and model constant, it measures how much each search API lifts answer quality, cost and speed, helping you pick a search provider for research agents.
Key takeaways · AI-distilled
Artificial Analysis compares search providers using GPT-5.6 Luna at medium reasoning and the same Stirrup settings. Tasks score zero if the AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → exhausts 25 turns without submitting an answer, so completion behavior affects the score.
The Search Index's cost per task includes both search charges and model tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition →. Its Parallel example shows that a higher-priced search tier can cost less overall when the returned results reduce the model's token use.
The Search Index combines measured search-call time with model time derived from token counts and canonical output speed. Its total time metric therefore includes an estimate rather than a direct end-to-end timing.
The Search Index equally weights list-answer F1, difficult fact-finding accuracy and factual-question accuracy. Its launch results describe this task mix and one fixed answer model, rather than every agent workload.
Terms in this piece · Glossary
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
