Vibeleaderboard
← All Intel
Intel / article

Droid 1.0: Building a Frontier Harness

Source
factory.com
Date

Context

Factory says the biggest change in Droid 1.0, the new version of its coding agent, is that the model now calls tools by writing a program instead of one call at a time. Factory calls Droid a : the software around a model, meaning its prompts, its tools and the loop that feeds tool results back. In an ordinary loop the model asks for a tool call or two, receives the full output in its , and is called again, re-sending its instructions and growing history each time. Droid's new Script tool instead lets the model write JavaScript that makes many tool calls in a sandbox with no file, network or system access of its own, and only what the program prints goes back to the model. According to Factory's announcement, this programmatic is now the default for models including GPT-6.1 Sol, Claude Opus 5.5 and Kimi K3 from Moonshot AI.

Factory's examples from its own sessions show where the savings come from. Moving 151 Linear issues between queues and checking each one took 312 tool calls; the Script did it in 2 model requests and about 61,000 input tokens, against an estimated 36 requests and about 3.1 million tokens with direct calls at 10 calls per request. In a sweep of 5,384 CI jobs, the model saw only a five-number summary of the results. The direct-call figures are Factory's estimates rather than measurements, and most Scripts are small: Factory puts the overall saving on Claude Opus 5.5 at 15%, concentrated in the 12.5% of Scripts that make six or more tool calls.

The headline comparison comes from Factory's own Terminal-Bench 4 runs, October 23 to 28, with each harness on the same model. With GPT-6.1 Sol, Droid scored 57.6% against 54.0% for OpenAI's Codex at 12% lower cost per passed task, though it took 10% longer. With Anthropic's Claude Opus 5.5, it scored 65.2% against 57.6% for Claude Code and took 26% less time per passed task. Factory ran three trials per task, reran trials that failed for reasons it judged outside the harness, reran Opus 5.5 trials stopped by Anthropic's content moderation on Claude Opus 5, counted tasks where Devin stopped to ask a question as passed whenever Codex passed them, and measured cost and time only on tasks both harnesses passed in all three trials. No independent run is reported, so these are the vendor's results under its own setup.

Key takeaways · AI-distilled
  • Factory reports Script mode, where the model writes sandboxed JavaScript that runs many tool calls in one request, cut Droid's cost by 15% on Opus 5.5, with savings concentrated in the 12.5% of Scripts making six or more tool calls.
  • In one Factory session, a Script moved and verified 151 Linear issues with 312 tool calls in 2 model requests and about 61,000 tokens; Factory estimates direct calls would have taken 36 requests and about 3.1 million input tokens.
  • Droid now appends newly discovered MCP and connector tools to the end of each request instead of the top-of-request tool list, so loading a tool no longer invalidates the cached prefix; Factory says the typical request reads 98% of input from cache.
  • Numbering every line of file reads cut Opus's cost 22% on one task but raised cost 8% on GLM and 32% on Kimi, so Factory shipped it as an option the model requests, an example of testing each harness change across model families.
  • Droid's base prompt fell 37%, from 20,887 to 13,055 tokens, mainly by sending 2 tool definitions up front instead of 11 and loading rarely used tools on demand, even as the itself grew by about a third.
Terms in this piece · Glossary
  • agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
  • tool use — A model's ability to call external functions — run code, search the web, edit files — instead of only generating text.
  • system prompt — The standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.
Read the source factory.com
Recommended reads
Comments

Checking sign-in…

Loading comments…