Anthropic FlashProve: Proofs for 400B MoE Replies at 110 Tok/s

Verifying which model actually produced a given API response remains impossible for an external party today. A routing layer could substitute a cheaper model, an attacker could tamper with the token stream, and output quality alone provides no reliable signal. Anthropic's Frontier Red Team has released FlashProve, an open-source research prototype that uses zkSNARKs to prove a reply came from a specific committed set of weights without revealing those weights. MarkTechPost reports that on Llama 4 Maverick, a 400-billion-parameter mixture-of-experts model, it proves about 110 tokens per second on a single H100, with 18.78 MB proofs that verify on a CPU in 17 seconds.

The gap FlashProve targets is structural: today's AI deployment stack has no mechanism for identity verification at the inference layer, and FlashProve is a published open-source attempt to close it at production-relevant scale.

How FlashProve works

FlashProve separates a prover, the model provider, from a verifier, the user. In the offline phase the provider publishes a commitment, a short fingerprint binding one exact set of weights; Anthropic reports this takes about 39 minutes and remains valid for roughly 16 million requests. In the online phase, the GPU proves the committed model produced the returned tokens, processing batches after the fact.

The proof system compiles an interactive oracle proof into a cryptographic proof using hash functions. Anthropic states this offers post-quantum security and belongs to the same family used by validity-proof Layer 2 networks such as Starknet. Soundness, completeness and zero-knowledge properties are formalized in Lean. Anthropic reports a cheating provider is caught except with probability about q/2^124 + 2^-101. The release ships two verifiers: a fast C++ verifier for daily use and a slower Lean verifier formally proven to accept only what the soundness theorem accepts.

Three engineering choices drive the throughput. Prover work scales with the approximately 17 billion active parameters per token in the MoE architecture, not the full 400 billion. Requests are batched and proved post-hoc, analogous to prefill. Weights stream from host memory, so a single 80 GB GPU suffices without a multi-GPU cluster.

Performance benchmarks

Anthropic measured the public 4-bit Maverick weights on one H100 with 24 CPU cores and 1,024-token prompts; these are vendor-reported figures from the FlashProve performance guide.

Within Anthropic's own batch measurements, throughput rises from 17.41 tokens per second at batch size 1 to 111.23 tokens per second at batch size 24, taking about 221 to 226 seconds per batch. Batch size 64 reaches 123.31 tokens per second but is flagged unreliable on an 80 GB card. Compared with vLLM running FP8 Maverick on eight H100s, Anthropic reports proving costs about 72 times the GPU time per batch at 24 requests, and about 94 times at vLLM's highest tested load. Clients download about 6.5 MB once per commitment in addition to the 18.78 MB per-request proof.

Current constraints

FlashProve is a research artifact, not a production feature. Context is capped at 1,024 tokens per request, far below the hundreds-of-thousands-token windows common in production. The proof covers integer arithmetic and greedy decoding; sampled or floating-point replies may diverge from the proved computation, which breaks the guarantee for the most common serving configuration. The model architecture must be public even though weights stay hidden. Proof generation is post-hoc and batched, so a batch of 24 produces verified output only after roughly 221 to 226 seconds, ruling out latency-sensitive use cases. The project is under Apache 2.0 and explicitly not accepting contributions.

AI Mastery analysis

FlashProve's most consequential trade-off is its active-parameter scaling. Because prover work tracks the roughly 17 billion active MoE parameters rather than all 400 billion, FlashProve clears throughput that prior dense-model provers cannot match. That advantage is architecture-specific: a dense model of comparable quality would require proving every parameter on every token, which changes the trade-off substantially and limits transfer to dense architectures without re-engineering.

The 1,024-token context cap is the more immediate deployment blocker. As long-context pricing comparisons have shown, production workloads increasingly use windows measured in tens of thousands of tokens. Until the prover handles at least an order of magnitude more context, applicability is confined to short-turn tasks such as classification, structured extraction, or customer-service replies, not document analysis or agent trajectories. The greedy-decoding restriction compounds this: quality-critical deployments usually sample, so the proof would not cover the actual generation path.

The Lean formalization is a meaningful differentiator. Many zkML systems ship with informal security arguments; machine-checked soundness and zero-knowledge properties address the implementation errors that have broken protocols with plausible-looking proofs. The dual verifier design, fast C++ for operations and Lean for audits, acknowledges that formally correct verification is still too slow for routine use.

The 72–94× overhead sets the adoption bar. It is prohibitive for high-volume consumer APIs but may be tolerable where the assurance value exceeds the compute premium: compliance audits, regulated serving, or inter-lab verification. The prior VerInf system proves the same 400B model at about 0.02 tokens per second, so FlashProve's leap comes from batching, weight streaming, and MoE-aware circuit design rather than a better proof system. Extending those ideas to longer contexts and sampled decoding while keeping the Lean proof intact is the engineering problem that will determine whether provable inference becomes a deployable primitive.

Sources

• Primary: https://anthropic.com/document/flashprove-technical-report
• Reported by: MarkTechPost: Anthropic Releases FlashProve: Cryptographic Proofs That a Reply Came From Committed Weights