AI Model Automates Formal Verification of Critical Code

Anthropic’s Claude model has successfully generated approximately 180 million lines of mathematical proof code to verify the correctness of existing software, marking a significant shift in how developers approach code reliability. Working in partnership with Theorem, the Frontier Red Team utilized the AI to produce formal proofs for 123 open-source libraries within a matter of weeks. This achievement addresses a long-standing barrier in software engineering: the prohibitive cost and labor intensity of mathematically proving that code runs error-free under all circumstances.

The primary challenge this project tackles is the historical expense associated with formal verification. Previously, such rigorous analysis was reserved for high-stakes applications, such as Airbus avionics systems, because it required specialized human expertise. For example, verifying the small operating system kernel seL4 took a team of specialists twenty person-years to complete. By delegating this work to Claude, Anthropic demonstrated that AI can handle the heavy lifting of translating crash tests into mathematical problems and analyzing compiled machine code, thereby reducing the human effort required to validate complex software structures.

A critical component of this workflow is the model’s ability to uncover latent defects. During the verification process, Claude identified hundreds of bugs that conventional testing methods had failed to detect over the years. These errors often manifest only under specific conditions, such as unusually large input files or particular sequences of data access. In one notable instance, the team verified a widely used PDF reader and discovered as many security vulnerabilities in a single week as are typically reported for that software in an entire year. Additionally, the model caught a logical error in another application that had remained undetected for more than a decade.

Despite the autonomy of the process, the system relies on specific technical constraints and independent checks. The proofs generated by Claude are validated by an independent program called Rocq, which automatically checks the mathematical logic. Anthropic describes the proof generation as "fully autonomous," though the underlying formal foundations and tools were developed by human experts at Theorem. The model does not write new application code but instead analyzes existing software by converting test scenarios into mathematical queries, ensuring that the logic holds up under scrutiny without requiring extensive human review for each individual proof.

However, the definition of "formally verified" in this context comes with strict limitations. The mathematical proofs are only valid under specific conditions, including a particular processor architecture, single-threaded execution, and input sizes under 64 megabytes. In real-world scenarios, these libraries often run on diverse hardware configurations and handle larger data sets. Furthermore, the computational cost is substantial, with individual proofs consuming up to hundreds of millions of tokens. While Anthropic has not disclosed the total project costs, the efficiency gains are evident in the reusability of the proofs and the improved performance of newer models, such as Opus 5.5, which solves half the proofs with 40 percent fewer tokens than its predecessor.

This development suggests that the software industry may soon treat formal verification as a routine part of the development lifecycle, similar to standard testing. If AI can both write code and mathematically guarantee its safety under defined conditions, the trust in AI-generated software will increase, potentially reducing the time and cost spent on manual code reviews. Theorem’s roadmap already targets critical infrastructure libraries like glibc, OpenSSL, and cURL, indicating that this approach is poised to expand beyond experimental projects into the backbone of internet and system software.