Markdown is 40× more token-efficient with SQLite

In a benchmark Native ran on its own company records, AI agents used about 43 times fewer tokens per fully correct answer when they queried those records in a SQLite database than when they searched the same content as a folder of linked Markdown files. Native, whose founder Richard Ng posted the result to Hacker News, builds open-source software that keeps a team's messages, documents and projects in one append-only event log, which can be turned into either a SQLite database or a Markdown vault. Its write-up took two months of its own log, about 77,000 events forming about 11,000 records, and built four setups: SQLite exposed to the agent as tools over MCP, a common protocol for connecting agents to tools; Markdown with strongly typed metadata at the top of each file plus a JSON index; Markdown with wikilinks, like a typical Obsidian vault; and plain Markdown. Each setup answered 40 questions three times, using the OpenCode agent with DeepSeek v4.1 Flash.

The gap comes mostly from how agents loop. Measured across all attempts, including failed ones, a fully correct answer cost about 172,000 tokens with SQLite, 653,000 with typed Markdown and 7.5 million with wikilinked Markdown; the headline 40 times rounds the first comparison. An agent resends its whole conversation on every step, and providers bill the part of that history already in their cache at a discount, as cached input. Native says SQL let agents filter and join records in a single query, keeping intermediate results out of the conversation and cutting the number of steps, so cached input fell about 50 times while uncached input fell about 7 times. Because token types are priced differently, Native models the cost saving at 10 to 30 times rather than 40. The typed-Markdown arm recovered much of the benefit on its own: agents there wrote Python scripts to do SQL-like filtering, which Ng writes was unanticipated.

The evidence comes from a vendor testing its own product on private data. Native calls the dataset authentic but not representative, since few teams produce this much agent-generated context, and Ng says the data is private because it is two months of Ng's own work. The test used one model, one agent harness and no direct SQLite setup without MCP, and costs are modelled rather than billed. On at least one question, the prose-based Markdown versions lacked typed fields, and Native says that comparison does not isolate SQL. The write-up itself says the results do not establish a universal advantage for every vault or budget.