Designing the harness around a small model, not the other way round
- Source
- x.com
- Date
New research: Portable Computer is a local-first agent for private and cost-effective work. With an on-device 27B model, our harness scores 82.6% on real knowledge work, beating open-source harnesses Pi and Hermes. Our post-trained PPLX 27B reaches 85.4%.
The model and harness are designed together because small models fail in harnesses built for frontier models. Portable is a minimal system prompt, skills that load on demand, connectors as compact CLI tools instead of MCP servers, self-verification, and an always-on sandbox.
Web research: on 1,266 BrowseComp tasks, Computer reaches 66.7% accuracy vs 50.2% for Pi and 43.9% for Hermes on their respective search providers. Portable uses the least wall time and the fewest tokens. Inference and private documents stay local; only search touches the web.
Parsing runs entirely on-device, so sensitive documents never leave the machine. On ParseBench-100, Computer scores 65.1% vs 34.6% for Hermes and 13.9% for Pi, in least time with fewest tokens.
Context
Perplexity's Portable Computer research treats the software around a local model as part of the design problem. The system uses short initial instructions, loads skills when needed and presents application connectors as compact command-line tools. Its architecture diagram shows software controlling the task loop while the local model proposes actions, with tools running in a and checks feeding back into the process. The design aims to make the surrounding system manageable for a smaller model. The research thread describes those choices.
- sandbox — An isolated environment where AI-generated code or agent actions run without being able to touch anything real.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
- MCP — The Model Context Protocol — an open standard that lets any AI assistant plug into any tool or data source without custom integration code.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Checking sign-in…
Loading comments…