Mathematicians Lean-Check OpenAI's Proofs, Find Credit Gaps
By @sharedot · · 7 pages
- AI
- Openai
- Mathematics
- Formal Verification
Independent researchers verifying OpenAI's Lean math proofs find formal validity differs from originality, reviving a 2016 credit dispute.
What Happened: Outside Researchers Started Running the Proofs
Following OpenAI's release of hundreds of AI-generated mathematical results — Interesting Engineering reports 377 results, while SOFX reports 722 manuscripts covering 372 result families — outside mathematicians did something unprecedented: they stopped arguing about the headlines and ran the machine-checkable evidence themselves.
Startup Fortune reports that OpenAI published Lean 4 certificates on GitHub for ten advances on open problems from its internal Astra model, and that independent researchers are now executing those same certificates line by line.
Because Lean mechanically verifies every logical step against a fixed set of axioms, outside mathematicians no longer had to trust OpenAI's framing; they could simply run the code.
The Surprise: Verified Proofs, Unsettled Credit
The verification layer held up, but the attribution layer did not. Startup Fortune reports that a Yeshiva University mathematician, Steven Miller, told Scientific American that Astra's sphere-packing proof reused an argument from a 2016 paper he co-authored without proper credit.
"They are running roughshod over the work of others who came before them in a deliberate way," Miller said, calling it research misconduct. Startup Fortune notes that an OpenAI spokesperson said the company would "take responsibility for the correctness of these results" and planned minor updates.
The dispute highlights an uncomfortable fact: a Lean certificate can confirm a proof is logically valid while saying nothing about whether the argument inside it was lifted from someone else's paper.
The Evidence: An Independent Audit Comes Back
Startup Fortune reports that Mikołaj Sienicki and Krzysztof Sienicki published "A Human Audit of OpenAI's AI-Generated Mathematical Proofs" on arXiv in August, covering 18 chapter-specific reviews across ten results.
Their conclusion: no surviving confirmed substantive mathematical error in a principal result. But the audit was careful about what Lean can prove — it cannot confirm that a press release summary matches the formal theorem or that definitions capture the problem mathematicians cared about.
The auditors flagged Chapter 8 as needing major revision, found one withdrawn "error" that was a typesetting artifact, and noted several dependencies have only been partly checked. Startup Fortune adds that none of the ten results had gone through traditional peer review.
The Stakes: Understanding and Access
The dispute goes beyond individual claims. SOFX reports that MIT mathematician Andrew Sutherland said claims solved by a single AI agent should be considered unverified until the model is released and others can reproduce the results.
The Institute for Advanced Study warned, per SOFX, that "AI can output mathematical arguments in situations without the human who prompted it being able to understand the arguments, verify them, or take responsibility for them."
Interesting Engineering reports that Harvard mathematician Melanie Wood, on an IAS-hosted advisory board, said the goal is standards so AI-lab results "can be understood by mathematicians and can advance the field," and that the board also opposed using advanced math problems to test proprietary models.
What Comes Next: Peer Review and Better Citations
OpenAI says it plans to improve future releases with better citations, explanations and presentation of results, SOFX reports, and Interesting Engineering notes the company said it drew on the advisory board's advice when preparing its release.
Startup Fortune reports the board has asked AI labs to grant "equitable access" to their models for the global mathematics community, and that Terence Tao has warned AI-generated proofs can separate getting answers from getting understanding.
Downstream validation is already appearing: Chapter 3's proof mechanism has been picked up elsewhere and Chapter 7 has drawn a stronger follow-on hardness theorem, per Startup Fortune.
The lesson of this round, Startup Fortune concludes, is that "verified in Lean" and "correctly attributed and peer reviewed" are two separate claims — and only one has been settled so far.