In a landmark push for AI-driven formal verification, Anthropic and Theorem used Claude to prove memory-safety properties across 123 open-source libraries compiled to RISC-V assembly.

The effort focused on assembly-code level verification using Theorem’s formal semantics for RISC-V and Linux process behavior, including real-world OSS-Fuzz projects.

By translating crash tests into mathematical proofs, the team produced roughly 180 million lines of proof code within weeks, covering 123 libraries.

Costs are high but improving: a single verification can consume more than 100 million tokens, though token use declines as models improve.

Compute costs are substantial, with proofs consuming hundreds of millions of tokens; while total project costs aren’t disclosed, the approach promises potential savings through reduced manual review and reusable proofs.

Token usage per run remains large (over 100 million), but is expected to drop as proofs are reused across similar software and models improve.

Industry-wide view supports formal verification as a complement to testing, with proof value hinging on how well specifications and machine models reflect reality.

Acknowledged limitations include platform-specific bugs and the need to model diverse machine configurations, with optimism that advances in LLM-based verification will address these gaps.

A subset of proofs and related semantics are published on GitHub; full proofs for fuzz tests that required no patching are released, while some proofs remain private to avoid exposing unpatched bugs.

Proofs operate under strict conditions (single-threaded, specific architecture, inputs under 64 MB), so they do not guarantee universal error-free software across all hardware and configurations.

Claude-generated patches were audited for overreach, with human reviewers focusing on specifications, trusted models, and the Rocq system.

Industry voices, including Mitch Ashley, contend AI can accelerate code generation, while verification should shift toward checking specifications, trusted models, and AI-introduced patches.