Disclosure: AI was used to assist with the writing and spell checking of this post. All the ideas are the author’s own, who takes full responsibility for them.
We mathematicians know that published papers and books often contain mistakes: sometimes small, like a missing hypothesis or a wrong constant, and sometimes more serious, like a theorem that is false as stated. We are kind of used to this. However, these mistakes are rarely corrected in public: the paper stays as it is, and the correction lives in the folklore of the community.
Over the last few months, I have used AI to check a few dozen published papers in research areas that I am interested in. In five of them, the AI flagged a mistake. I checked each flag myself, and all five were real.
As an example, a few weeks ago I asked ChatGPT 5.6 Sol to check our own 2023 paper “The relative rank of the endomorphism monoid of a finite G-set”, published in Semigroup Forum. The last section of this paper contains a subtle calculation, and for some time I have wondered whether it is entirely correct. The AI told me that the proof of Theorem 17 contains a mistake: we did not correctly describe the kernels of some G-equivariant transformations. It added that, with the correct description, the theorem still seems to hold.
In fact, my co-author and former PhD student, Ramón H. Ruiz-Medina, had pointed out the same problem to me a few months earlier, so the AI arrived on its own at something we already knew. It is unfortunate that the published proof has a mistake, but still, it gave me some peace of mind that the AI also believes the final calculation is correct.
This was a lucky case, since the mistake can be fixed. Not every case was like this: some of the mistakes the AI pointed out to me were real counterexamples to stated theorems.
For me, the main advantage of this approach is how little time and effort it takes to check a flag. If someone tells me that Lemma 3.2 fails for n = 2 and shows me why, I can verify it in a few minutes, whereas certifying every proof in a paper could take weeks.
The formalization community, including the Lean community, has argued for a long time that mathematical literature needs checking, and I fully agree. If we need to check every step, formalization is invaluable. But formalizing every single paper is slow and expensive, and in most cases a certificate is more than what we need.
Of course, my experiment is small. A few dozen papers is not much, and I do not know how many mistakes the AI missed in the papers where it found nothing. When the AI finds nothing, that does not mean the paper is correct, as it is far from being the same as a Lean certificate.
I am less sure what should happen after we find a mistake. Journals rarely publish corrections, and many mistakes that experts know about are never written down anywhere. It would be great to see a public record linked to each paper where anyone could look up the mistakes that have been reported, whether they have been confirmed, and whether they have been fixed.
The same kind of check could help before a paper is published. David Craven in his post “Where will all the papers go?” (Proofs and Prompts, Sep 19, 2026) argues that authors should run an LLM check before submitting a paper. I would go a step further: journals could offer an AI check at submission, with referees examining any concrete objections it raises. I think this could significantly reduce the number of published papers with mistakes in the future.
I do not plan to run a large project myself, but I think the idea deserves more attention. If you have used AI to check published papers, I would be interesting to hear about it. How many papers did you check, how many flags did you get, and how many were real mistakes?
Received 28 September 2026
Add to the discussion