Octen Search debuts third on the Artificial Analysis Search Index with a score of 77, achieving the fastest Time per Task at 16.9 seconds and one of the lowest Costs per Task
We benchmarked Octen Search from @OctenAI using our search agent methodology: a fixed base model running inside our open-source agent harness, Stirrup, with tools for searching and fetching pages from the web. Only the provider behind the search tool changes.
Key results of Octen Search with highlights enabled:
➤ Strong Search Index performance, led by BrowseComp: Octen Search scores 77 on the Artificial Analysis Search Index, placing third behind Perplexity Search (medium, 80) and Perplexity Search (high, 79), and level with Perplexity Search (low, 77). Octen performs particularly well on BrowseComp, scoring 86, just one point behind the leader. It also scores within one point of the leaders on DeepSearchQA at 80, while its AA-Omniscience score of 66 trails Firecrawl Search (73) and Perplexity Search (medium, 72)
➤ Fastest search across providers: Octen Search averages 0.2 seconds per query, 5x faster than the Perplexity Search variants at 1.0 to 1.2 seconds. Its Time per Task is 16.9 seconds, the lowest…
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Why it matters
Octen Search ranks third on accuracy but delivers the fastest per-query latency and among the lowest costs, a concrete tradeoff for teams picking a search backend for AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → pipelines.
Key quotes
“Octen Search scores 77 on the Artificial Analysis Search Index, placing third behind Perplexity Search (medium, 80) and Perplexity Search (high, 79), and level with Perplexity Search (low, 77).”
“Octen Search averages 0.2 seconds per query, 5x faster than the Perplexity Search variants at 1.0 to 1.2 seconds.”
“Octen Search costs $0.058 per Search Index task at a price of $1 per 1k search queries.”
Read the full post on X ↗
postPerplexity's search API takes the top spots on the Artificial Analysis indexArtificial Analysis
postAnnouncing the Artificial Analysis Search Index, benchmarking how search API…Artificial Analysis