Elektrine lite

← Feed

@mjd@mathstodon.xyz

Post #3992424

2026-07-21 17:32 UTC

OK, so this article is discussing followup work to the recent work on the Erdős unit distance problem. It says: “The basic structure of the proof is that a profound theorem in number theory due to Golod and Shafarevich from the 1960s could be used to construct a counterexample to the conjecture.” OK so far. Later it continues: “Boris Alexeev announced on the Lean Zulip that he had steered ChatGPT to a complete formalization of the Erdős counterexample, assuming nothing beyond the axioms of mathematics. … [The model] had generated 1.2 million lines of Lean code in the three weeks that it had worked on the project. … One cannot trust AI-generated code so I ran it in a sandbox on my machine … . Wow.” I understand that the Lean code can be easily checked for correctness by the Lean kernel, I think that part is clear. But who is it exactly that is looking at 1.2 million lines of autogenerated Lean code to confirm that the claim that has been formalized and checked actually says what was wanted? For example, won't it start with a huge pile of Lean formalizations of concepts from algebraic number theory? Mathematicians don't always understand one anothers' definitions, isn't it possible that one of the Lean definitions is subtly different from what is really needed, so that the theorem that has been checked isn't saying what we thought it was? In what way is 1.2 million lines of autogenerated and auto-checked Lean code useful for anything? This is a serious question. I don't understand. https://xenaproject.wordpress.com/2026/07/20/human-mathematicians-are-being-outcounterexampled/

Replies (2)

  • @barubary@infosec.exchange 2026-07-21 18:12

    @mjd@mathstodon.xyz Reminds me of https://web.archive.org/web/20041108121656/http://www.research.ibm.com/people/d/dfb/papers/Bacon01Concurrent.pdf, which contains a few proofs of correctness (of the concurrent version of their algorithm), but their basic single-threaded algorithm has a use-after-free bug.

    Open ##3992421

  • @mjd@mathstodon.xyz 2026-07-21 17:36

    Or maybe more briefly: When doing AI-assisted programming, it sometimes happens that people have the AI generate both the code and its tests, and then discover that although the tests pass because code and tests match, both diverge from the specification of what the AI claimed its code would actually do? How do we know this didn't happen with the 1.2 million lines of Lean that purport to be validating the solution to the Erdős unit distance problem? Couldn't they be a 100% correct verification of the wrong theorem?

    Open ##4445801