Use LLMs to check your work

Anonymous

SERIES · AI is making papers cheap to produce but they remain costly to referee. Proofs and Prompts is publishing a series of posts from 26 October on the future of publishing, papers and credit. Questions such as how to disclose AI use, who bears the cost of checking, what authorship means and what papers of the future might look like will be addressed. This post is part of the series. Each is written independently and reflects only its authors’ views. All posts in the series

AI disclosure. This essay was composed on the Notes app of the author’s iPhone. This app provides AI assistance with spelling, grammar, and auto-completion of words being typed.

Most discussions about LLMs in mathematics seem to focus on their (rapidly advancing) proof discovery abilities and the effects these abilities may have on professional mathematics. I write here about a different use of these technologies, to make the following point: we have a professional duty to use LLMs to proofread our work. I use “proofread” to mean both copy editing of no mathematical substance, as well as the detection and correction of mathematical errors and gaps in arguments. By “our work,” I mean our research papers, though the arguments below extend with straightforward modifications to teaching materials.

First, a fundamental community norm is that we should make all reasonable efforts to ensure that our proofs are correct. Modern LLMs are tools that, when used appropriately, can provide this assurance with high confidence and little effort.

Second, incorrect or inadequately justified proofs exact a substantial human cost. I have refereed many papers in my career; the vast majority had at least some minor mathematical issues, and a not insignificant minority had issues that required new contributions by the author(s) to patch. Identifying these issues and writing a detailed referee report to alert the author(s) takes many hours, especially for long and technical papers.

Before LLMs, I executed my referee responsibilities with pride, knowing that authors were (typically) putting forth the best work they could, and that operating on the frontier of mathematical knowledge would inevitably involve some mistakes. Now, however, when I see clear mistakes in papers I am reviewing, I just feel disrespected. The author(s) had a tool they could have used to eliminate the mistakes and did not, wasting my time. (Certainly, LLMs are not perfect, but it is easy to tell when a cursory check has not been done.)

Recent events in my personal life have emphasized for me the value and precariousness of human life, and how death and life-limiting illness can strike unpredictably. Submitting work that has not gone through an LLM check to a journal may cost a conscientious referee many hours of their time. To ask someone to spend their most precious resource correcting errors that could have been caught with the push of a button is both rude and professionally negligent.

Some may argue that abstention from LLM checking is justified on ethical grounds. If this objection rests on the behavior of frontier labs, one can simply use open-weight models. If the objection rests on resource use, a simple calculation shows that the resource use from a few minutes of LLM computation pales in comparison to other activities we regularly undertake to advance mathematics, such as conference travel. I am sure there will be other creative objections, but it is hard for me to see how they can outweigh the cost refereeing erroneous mathematics has on human life (especially in combination with the cost to all other readers who might be interested in the result).

I also believe we have a duty to use LLMs for literature searches, to ensure correct attribution, and to make sure our arguments are as simple and straightforward as possible. The reasons are similar.


Received 30 September 2026.

Filed under
More by →

5 responses to “Use LLMs to check your work”

  1. Grigori Avramidi Avatar
    Grigori Avramidi

    I second this. But, I feel uncomfortable to even suggest this to coauthors who are more skeptical of llm use and want to keep a particular paper completely llm free.

    One worry to keep in mind is that this may lead to disjointed papers that are more correct but harder to read (if there are humans who grudgingly fix gaps pointed out by an llm), just like ordinary refereeing often leads to such papers. So, there is a balance to be struck.

    On a related note, I have been asked to referee ai assisted papers by a journal that forbids me to use ai assistance to referee said paper. This is absurd and needs to change.

  2. Gordon Royle Avatar
    Gordon Royle

    What even is the purpose of refereeing at all now? An LLM can verify correctness and novelty so what else? Formerly a human referee might give an opinion on the significance of the result or quality of the exposition but I am now hopelessly uncalibrated. Many people are rushing out AI-assisted papers proving results that would previously have been acceptable but now are routine. Is a complicated exposition using a wide range of unfamiliar techniques poor writing or is it actually the simplest possible given that only an LLM would have combined so many disparate ideas? Personally I think that sites such as hexagon will rapidly dominate and journals will fade into irrelevance. If you need a result then search hexagon, verify the result with your own LLM and then use it.

    1. Anonymous Avatar
      Anonymous

      A slippery slope.

      What is even the purpose of any humans interacting with mathematics at all now? An LLM can produce novel mathematics now, at a faster pace than humans, and even produce conjectures, so what else?

      Just let the LLM produce, verify, referee, and expand upon everything, and take humans out of the loop entirely. In fact, humans are so inefficient, we should just get rid of them in favor of the machine god.

    2. Joker Avatar
      Joker

      I don’t like this but my horrible experience of editor not sending out my submission for review for over a year and a half and then rejected it after I got a revision request from the referee just prevents me from refuting the argument.

  3. Sash Avatar
    Sash

    I’ll go further, we have an “obligated” duty to first use a billion tokens at least to solve any mathematical problem, if not we should wait for the next reset, and spend a few billion more tokens and only if that doesn’t work and we’re sure no new model release is coming in the next few weeks, should we be professionally obligated to use our minds to do something, anything else is not the most efficient use of resources.

    Actually this is not enough, we are obligated to spend a trillion tokens with every new frontier model to try to solve every existing open problem since it is a mathematicians duty to produce as many true statements as possible.

    Why only focus on open problems. If it is our duty to society to maximize number of true statements, we should actually just ask the LLM to open-endedly search a field for true statements that it can prove. We can categorize each statement by number of tokens it took for the proof, to understand how deep the search was. Depending on our grant money, I think we are obligated to spend at least a billion tokens on this every day (~10$) and then every week we should release a book of our findings so that other LLMs can build on our work. Imagine just how much further our towers of abstraction can go.

    I’m not being sarcastic. I think there should be Mathematicians who do this, there should be complete AI maxing, along with complete AI manning and then like a controlled experiment we should see what is actually more valuable to a society and funnel grant money to those people.

Comments are moderated. Read our comment policy.

Add to the discussion

New posts by email.

Prefer a feed reader? Subscribe by RSS.

Also on Mathstodon.

Latest comments across the site.

  1. I’ll go further, we have an “obligated” duty to first use a billion tokens at least to solve any mathematical...

Discover more from Proofs and Prompts

Subscribe now to keep reading and get access to the full archive.

Continue reading