Provable Inference

Anthropic's FlashProve lets someone check that a specific AI model produced a response without seeing the model's weights or rerunning it. Today a user cannot tell whether the model they chose actually answered, or whether a routing service substituted a cheaper one or an attacker tampered with the output. FlashProve uses a zkSNARK, a type of cryptographic proof: the provider first publishes a commitment that pins down one exact set of weights while keeping them hidden, then produces, for a given prompt and reply, a proof that those committed weights produce exactly those tokens. Anyone holding the commitment, the prompt and the reply can check the proof, and it reveals nothing about the weights, according to Anthropic's research post.

What the post adds is speed. In Anthropic's measurements on Meta's Llama 4 Maverick, a 400-billion-parameter mixture-of-experts model run with 4-bit weights, a single H100 GPU proves about 110 tokens per second. Each 1,024-token request gets a proof of about 19 MB that a 24-core CPU checks in about 17 seconds. Proving still takes roughly 70 to 95 times the GPU time of serving the same requests with vLLM, a common open-source inference engine. The only earlier proof of this model the post knows of, VerInf, reports 14.3 hours for 1,000 tokens on a DGX Spark, a different machine, with a 93.6 GB proof that took almost a day to verify. Anthropic says Claude produced the improvements to the proof system and that the top-level security properties are stated and proved in Lean, a formal verification language.

The uses the post describes all depend on someone needing assurance they cannot get from the output alone. Outside evaluators could confirm they are testing the model being served, customers of routing services could confirm they got the model they paid for, a company could detect a compromised inference server, and a regulator could check that only approved models are served or that hardware is really running inference. The post is careful about that last case: proving that hardware served inference does not show it was not also training on the side.

FlashProve is a research prototype released under the Apache 2.0 license, and its limits are specific. It proves an exact integer version of the model under greedy decoding, so a reply from ordinary floating-point serving or sampled decoding can depart from what was proved. It handles contexts of only 1,024 tokens, depends on the mixture-of-experts structure, hides the weights but not the architecture, and treats timing side channels as out of scope. The fast C++ verifier meant for daily use carries no formal guarantee; only the slower Lean verifier does. Anthropic also says it met its own September 30, 2026 goal for a provable-inference prototype through a separate project that signs inference traffic and recomputes it on a trusted server, so FlashProve is a second route to the same aim.