In the following essay, I would like to give a reasonable defense of what I call the AI dissenter viewpoint. I have argued along broadly similar lines in two previous essays.1 My views are continually evolving and I recognize that a no-AI future for mathematics is unlikely to succeed, but I think it would be intellectually dishonest of me to water down the argument and propose some nominally more pragmatic scheme. So let me push the Overton window a little bit further and offer my honest thoughts.
The thrust of my argument is the following: the introduction of increasingly capable LLMs has a disastrous effect on the social context and professional environment through which human mathematical understanding is produced. Moreover, we are not alone in the crisis we now face, and we should expect LLMs to affect a range of white-collar and scientific professions in the long-term. As citizens of a broader society, we have moral obligations to avoid collaborating with AI companies and improving their models with the aid of our extensive mathematical training; in fact, we should take an explicitly adversarial position to these companies and the future they are creating. When we use LLMs to generate new proofs of mathematical theorems, we are directly benefitting AI companies, who stand to financially and politically benefit. The more important2 the theorem, the worse the impact. For moral reasons alone, we should not use LLMs to generate new proofs, even if increasing scientific progress might otherwise be a good use case for AI tools.
We should not embrace the use of LLMs to maximally increase mathematical progress, but instead safeguard a social context which preserves and increases human mathematical understanding. Human mathematicians must extensively cooperate to establish reasonable professional norms around AI use in this new regime. In the near-term, we should preserve the current model as much as possible until a consensus coalesces; that means an ongoing moratorium on asking LLMs to prove novel mathematical theorems. Do not collaborate with AI companies, do not defect, and do not pollute the commons with LLM-generated mathematics. Since non-defectors are obviously punished by the existence of defectors, cooperation and coordination is the only path forward. In every regime, the current model of mathematics will clearly substantially change. (For instance, I believe it is likely that the “ownership model” of mathematics will collapse, and so we should redesign our incentives to preserve human understanding of mathematics.)
The current regime: Humans competing with LLMs
Right now, in the ongoing ownership model of mathematics, we have created a situation where human mathematicians are now directly competing with LLMs. So we need to redesign our incentives, fast. Humans who are doing mathematics without AI use are spending labour and time; humans prompting LLMs are spending money on tokens.
Concern: LLM capabilities are quite spiky; their capabilities not only vary field-by-field, they vary problem-by-problem, such that an LLM may be capable of proving Hard Theorem A, yet spit out nonsense when asked to prove Easy Theorem B. Moreover, different models have varying capabilities, and some mathematicians have access to frontier models while others do not. As a result, vast inequities result for mathematicians in different research niches and with differing access to models.
Concern: Mathematicians are directly disincentivized to communicate and disseminate their ideas, proof sketches, and new theories with other people, because they can be scooped by LLM users. People are incentivized to either quickly publish lower-quality work in an LLM-accelerated environment (slop mathematics), or to fully flesh out a theory entirely alone and then drop a 100-page mathematical monograph. Long-term, high-quality projects are now significantly more risky.
Concern: Various actors (AI companies, social media users, mathematicians) will pollute the commons in the near future by quickly producing and generating mathematical work using LLMs and releasing it without refining, disseminating, communicating, and understanding that work carefully. In other words, they will produce slop mathematics. This is already evident in the appalling phenomenon of important mathematical counterexamples being dumped on social media by users who clearly have no intention of ever producing an ArXiv preprint or publication; instead, the mathematical community steps in to pick up the pieces.3 In short, we should expect that all the worst aspects of the publish-and-perish model (the production of ‘slop’ mathematics, underverified proofs, etc.) will be exacerbated in the near-term LLM regime.
Concern: Early-career mathematicians will leave en masse in the next 5 years. They will respond to a field-wide sense of uncertainty and fear if reasonable professional norms do not coalesce quickly. Many of these mathematicians already face suboptimal working conditions of severe financial precarity.
Concern: Mathematicians with serious ethical concerns about AI companies are directly penalized in the near term; this fuels discrimination in our field. In other words, a person’s ideological lens or political sensibilities could unacceptably curtail their career opportunities.
Broadly: My sense is that using LLMs contributes to a social ecosystem in which human mathematical understanding is broadly degraded, disincentivized, and punished. The degree to which human mathematical understanding is preserved is largely due to individual actors consciously or pro-socially moderating their LLM use.
Arguments against LLM uptake in the mathematical profession
I would like to present three different arguments for avoiding using LLMs in our profession. I should add that clearly, LLMs are complex enough tools as to create a moral spectrum of responsible use; but I believe the conclusion of the three arguments below is that in our profession, the less we use them, the better.
The social context lens (the preservation of mathematical understanding)
The introduction of highly capable LLMs imperils human understanding of mathematics, because human understanding of mathematics happens in a social context. We should expect a serious loss of understanding even if we only allow LLMs to generate and verify proofs, while we digest these proofs as an active and living community.4 Let me offer some arguments.
Argument 1: When humans generate mathematics, it is useful for mathematical understanding. We understand mathematics more when we do it ourselves. If we increasingly delegate the generation of mathematical proofs to LLMs, that already has a drastically negative impact on our understanding.5 Of course we spend vast amounts of time understanding other people’s work, but frequently we do so with the aim of eventually writing original mathematical work ourselves.6
Argument 2: People need to believe they add value to our field. If LLMs are pervasively used to generate research mathematics and eventually become more capable at generating new proofs than human mathematicians, then many people will not choose to work in mathematics. Long term, many people are not attracted to working on solved problems. Does reading AI-generated mathematics full-time sound like a profession or a hobby?
Argument 3: When we outsource the work of generating research mathematics to LLMs, we degrade the working conditions of our field. The labour of doing original mathematical work is creative, important, and enjoyable. Is refereeing papers your favourite part of your job? What about papers written by LLMs?
These are serious concerns which I cannot quiet even if we wholeheartedly pivot to incentivizing the communication and digestion of proofs (which I believe we should: see below). I think if humans increasingly relinquish the work of generating autonomous proofs, that is an incalculable loss to our understanding and our profession.
The labour lens
AI companies have extracted value from the scientific community’s labour by using our papers, textbooks, and research output as training input for frontier models. Simultaneously, they exert pressures that in the near-term will result in mathematicians losing their jobs and our working conditions deteriorating. How is that fair? To state the obvious: AI companies have not turned to our field because they are benignly interested in increasing the scientific progress of our field. They use their models to prove theorems because when they announce a nonsofic group exists, they increase shareholder value and establish market dominance.
AI companies are attempting to create models which outperform humans at almost all economic tasks.7 If they succeed, almost all of human society stands to be affected. This is so unbelievably inequitable that it is poised to be one of the greatest wealth transfers in human history.
It is incumbent upon us to organize in light of this and to hold the line. We must defend our material interests and our livelihoods, particularly when AI companies used our corpus of work to create these highly capable models. Many of us produced scientific work with the understanding that it would become the collective intellectual property of humanity; we did not produce it anticipating that it would become fodder for private interests and the creation of models which disrupt the production of exactly that scientific work.8
The ethical lens
In mathematics, we have a tendency to treat AI like it is some massive exogenous force which came out of thin air to disrupt our profession. But highly capable LLMs were not produced ex nihilo; they were produced by an industry with intense financial interests, appalling professional norms around safety, and executives with so little message discipline they say things like this in public:
Weirdly enough, if you think that this moment is, I don’t necessarily believe this, but a lot of people would say we’re living through this kind of eclipse of the human intellect where we’re in the final days of humans being the primary actors on this planet, um, and that soon machines will rise. […] It’s a little bit like, it’s, in that sense, it’s a very beautiful time period to live through because in a Dionysian way, there’s a lot of ugliness about it, but there’s a beauty in the ugliness of when a star dies, it grows super big into the red giant, right? And it’s like that, where you, as you watch this final flowering of humanity and the birthing of the machine intelligence, it’s like you see this greatness in human effort.9
Right now, thousands of people are attempting to build artificial general intelligence which outperforms humans at all cognitive tasks. Is that a good idea? Is that democratic? Set aside for a moment empirical claims about whether or not AI companies succeed at their stated goals. Should they even be trying?
Here is my sense of the situation. It is scientifically irresponsible to initiate a scientific project which, if it succeeds, leads to disastrous consequences for human society. Why should we not spend our time doing frontier human genetic engineering or creating super-contagious pathogens? We could make huge scientific progress in those domains. The reason why is that if we succeeded, we would have done something deeply unethical.
In light of that, we have moral obligations to avoid collaborating with AI companies, and in fact to resist them as much as possible. When we use LLMs to generate novel mathematical proofs, AI companies benefit. When we offer our expertise to help mathematically benchmark models, we feed AI hype.10 When we collaborate with AI companies, we are aiding and abetting the Manhattan Project of our time.11 The labour of mathematicians is critical for helping AI capabilities improve; it’s past time for a boycott. I believe that we are desperately in need of a global AI pause. We can either lend our support to the movement for an AI pause, or continue to manufacture consent for AI companies.
Of course, many mathematicians are not directly building LLMs or working for AI companies, and we may console ourselves that we are just downstream beneficiaries of this irresponsible scientific project. Maybe we can just use LLMs to resolve our own scientific curiosities and finally make some progress on our favourite pet conjecture. After all, we tell ourselves, scientific progress is morally neutral. Whether humans or machines generate proofs is morally neutral. But this is clearly not true. The more important the theorem, the more these companies stand to benefit. The more we use LLMs, the more social consent we manufacture for this dangerous and rogue industry.
The fact that increased AI capabilities are broadly bad for mathematicians is a corollary of a more general proposition: increased AI capabilities are bad for everyone. So we can’t use LLMs, even if nominally a proof is a proof, whether it comes from a human or a machine.
Rebuttals to the view above
I now address some rebuttals to the view above.
LLMs are not that good at mathematics yet. LLMs may be good at [y] thing that was never really the essence of math (e.g. problem-solving, finding counterexamples to conjectures, tedious computations of technical lemmas), but it will not replace [x] aspect of our mathematical work which is actually the essence of mathematics (building theory, teaching younger mathematicians, clearly expositing mathematical proofs).
Unconvincing. What happens if this intermediate regime does not last? What happens when [x] aspect of our career can be performed by LLMs too?
Of course, the one kind of labour that LLMs cannot do for us, definitionally, is the human digestion of mathematical proofs. But the human digestion of mathematical proofs occurs in a social context. See the social context argument above.
Human mathematical understanding will just become much higher-level. LLMs will function similarly to calculators and computers.
Low-level details are also key to understanding things deeply. Moreover, LLMs are substantially different from calculators and computers in their capabilities. See the social context argumentabove.
Mathematicians are paid to produce theorems and it is unfair to taxpayers and the public to not maximally accelerate mathematical progress with LLMs.
I think this is a terrible argument. The main way the public benefits from government-funded math research is from the ancillary benefits of having a large, public research mathematics community. I think it is pretty easy to argue that human mathematicians are more useful to the public than LLMs. Mathematicians can explain mathematics to the public (in the classroom, on Youtube, in popular science magazines). They teach undergraduates across scientific disciplines foundational tools like calculus and linear algebra. They educate and train graduate students on problem-solving skills which transfer to industry and academia. The public largely doesn’t care about the production of new mathematical theorems; the main benefits they receive are indirect benefits like those above.
LLMs may prove to be cheaper at producing novel mathematical theorems than human mathematicians, but the indirect benefits the public receives will only decrease if we replace mathematicians with LLMs.
If the theorem economy is artificial12, then there is no harm in simply producing as many new theorems as possible with LLMs in the interest of satisfying our mathematical understanding.
Theorems may have little market value (in plainer terms: the majority of mathematical work is useless). However, that does not entail that it is consequence-free to produce as many theorems as possible with LLMs. See: the social context and ethical anti-AI arguments above.
If we institute some kind of professional ban on using LLMs, that actually decreases and hinders human mathematical understanding (possibly to an enormous degree). It is incoherent to defend human mathematical understanding while asking mathematicians to desist from LLM use, and to inhibit scientific progress, which is nominally the goal of a research community. In other words, scientific progress is one of the positive use cases of artificial intelligence.
I think this is one of the most interesting and compelling rebuttals to the arguments I have outlined above. It is by far the strongest.
One possible rebuttal is to reiterate the social context argument. Already, a vanishingly small fraction of human society is interested in learning the proofs of famous theorems. If LLMs become more capable than humans at generating mathematics, I think many people will lose interest in mathematics. If our profession is hollowed from the inside out and mathematics becomes a hobbyist endeavour, then it’s not really the frontier that matters anyway. We can know; we won’t know.
But I think the more intellectually honest thing is to concede the point. Yes, we won’t make the most mathematical progress possible without the use of LLMs. But we have compelling reasons to refrain from using them anyway. See: the ethical anti-AI argument above.
People can just lie and use LLMs anyway. Mathematics is a competitive environment and actors who don’t use LLMs will be punished. A worldwide total ban on AI use is unlikely to succeed. There are morally grey methods of AI use (literature searches, learning classical mathematics, etc.) which would undermine the efficacy of an outright ban. Even if we control the use of LLMs in some countries, other countries may not abide by such agreements. In short, the AI dissenter view is unrealistic and unlikely to succeed.
I concede all of these points. I expect that the actual course of events does not land us in the AI dissenter’s ideal world, and at best in some intermediate regime.
Nevertheless, I would like to shift the Overton window to a more radical place by being as intellectually honest as possible. From a pragmatic perspective, I think if we don’t have the intellectual courage to articulate ‘naive’ points of view, then we have no chance of them every succeeding. From a moral perspective, I think the AI dissenter viewpoint is just correct. To state a fairly obvious observation in moral philosophy, there is no reason to believe the morality of a given action is at all related to the facility of performing that action. But actually, I think we still have very good options for protecting our field from AI companies; see the radical proposals below.13
If LLMs outcompete humans at research mathematics, then we should just allow LLMs to largely monopolize the work of generating research mathematics (that is, producing proofs). The job of a mathematician could instead be about digesting and disseminating frontier AI-generated mathematics. The work of our profession will be teaching each other math, educating undergraduates, service work, and community.
I actually find this very interesting. Broadly, I think the mathematical community is coalescing around redesigning the incentives in this direction, and I support this initiative. These sorts of labour are valuable. The work of it assembles into the labour of a real profession. I think it is an interesting rebuttal which addresses concerns about both labour and the social context by which human mathematical understanding is produced.
There are two issues with this proposal to my mind. In this theoretical regime, the work of generating original research mathematics now plays a marginal career in our careers. That is an enormous loss.
The other is that this course of action does not address the ethical anti-AI arguments. Mathematicians will still have aided and abetted AI companies in improving AI capabilities. When novel mathematical theorems are proved by us with extensive AI use, then AI companies stand to gain enormously, in terms of both stock value and sociocultural sway. Every time you use AI to prove a new “big” theorem, you are helping AI companies (even if the ownership model is long dead).
Some radical proposals
I now suggest some proposals which can help address the AI crisis in mathematics. Critically, supporting these proposals does not require too much anti-AI buy-in; I believe these proposals could be supported by a coalition of mathematicians.
- Move away from the ownership model and the current theorem economy. Redesign the incentives to encourage human mathematical understanding. Incentivize work like: mathematical communication; education; and textbook writing, instead of just the production of original mathematical work.
- Even those of us who are attached to the human generation of mathematics should support this initiative, because this initiative directly disincentivizes defection. It curbs the creation of slop mathematics. People will be much less motivated to use LLMs to thoughtlessly produce novel mathematical theorems if they aren’t rewarded for it. Already, too many theorems were proven for the community to be able to reasonably digest them.
- Limit the number of papers that an individual is allowed to output per year. This is an immediate way to punish actors who want to flood ArXiv with LLM slop, and to encourage higher-quality work. This essay by Mario Pasquato expands on this proposition at greater length.
- Determine the leverage we have over AI companies and exercise it. What can we do to oppose this industry? Options could include: a field-wide boycott on collaborating with AI companies; deliberately polluting the commons with bad and faulty mathematics to wreck models; restricting frontier mathematics from being accessed as training data by AI companies; demanding direct remuneration from AI companies; building coalitions and organizing politically; writing public essays and speaking to the media.
- Organize at your own university. I reiterate here urgent proposals from mathematician Max Weinreich:
Every college mathematics department needs a committee, formal or not, to regularly discuss and make recommendations for AI use. You can start this committee. Time is too short to coordinate a national project from the top down or to wait for someone else to do it. Departments hold the keys to ensuring that human mathematicians may at least persist as a minority in the mathematical world in many ways. Departments can lead by establishing anti-AI policies for student work, by reserving hire lines for mathematicians who eschew AI, by valuing AI-free papers more highly in tenure promotion, and by increasing resources for talks, seminars, and conference attendance. [emphasis mine]
- At an individual level: avoid using LLMs to prove novel mathematical theorems. Don’t defect and don’t pollute the commons. Every time you do this, you directly benefit AI companies and you marginalize the production of human-generated mathematics. You behave unethically. If you use LLMs, limit their use to applications which don’t directly compete with humans. In all cases, act conscientiously and pro-socially, and don’t engage in mathematical arbitrage.
Thoughts on cooperation and defection
Clearly, it is extremely difficult for humans to cooperate on a large scale even when it is in their best interests to do so (see: all of human history). But any path forward requires extensive cooperation. Everything from here on out is game theory. People can make locally rational decisions to globally disastrous effect. If you don’t make your own views known in a public forum, then sympathetic actors are unable to coordinate with you. Within mathematics, we are obviously deeply divided as a community on our views about how to proceed in an era of increasingly capable LLMs, and every person will have to make concessions.
I would like to push back against this manufactured consensus that we all have to start using AI tools or risk falling behind. I have been told it is exceedingly naive to resist these tools and to ask people to cooperate and not defect. Probably this is true. But I think it’s also exceedingly naive to embrace the use of LLMs in mathematics and expect good long-term outcomes for our field. I do not find the people whose argument is to submit to entropy practically convincing. I do not find even thoughtful and nuanced AI proponents morally convincing. I find AI companies morally despicable and I continue to dissent.
All of us know that LLMs are certainly not better than the mathematical community in aggregate, yet. What happens when LLM capabilities at mathematics dramatically increase? An entire industry exists which will only meet its financial targets if it succeeds at exactly that. Thus, we will be on the back foot if we have not already built a coalition.
It takes a certain kind of person to form a union. It takes a certain kind of community to cooperate en masse. Our working conditions involve relentless competition; thus, from the outset, it seems unlikely that we are that community.14 But I’m not so sure. Only time will tell.
Acknowledgements. I am grateful for conversations with Zachary Glaser and Merrick (Dongming) Hua. All of the views above are solely my own and do not reflect those of anyone else.
Further reading:
- Statement on LLMs, by Jonny Evans.
- The crisis of AI-generated mathematics, by Max Weinreich.
- What will become of us? by Mario Pasquato. This essay outlines a proposal to limit the number of scientific publications a person can output each year.
- With some important differences: after reflection, my ethical objections to LLM use have only increased. For instance, I no longer think the “ethical (collectivized) model for using AI” I outlined in my first essay would actually be ethical. (It goes without saying that it was never practical.)
↩︎ - As determined by social consensus, first chapters of introductory math textbooks, the zeitgeist of the mathematical community, whatever.
↩︎ - We should also consider the dark shadow of this phenomenon, when users publish incorrect LLM-generated proofs on social media (often because they lack the mathematical training to read such proofs) and take no accountability for the errors, even when they’re pointed out. This kind of thing directly erodes scientific discourse and wastes everyone’s time; it’s slop.
↩︎ - Here, I am referencing a framework advanced by Terence Tao, who has argued there are three core components of mathematical problem-solving: proof generation, proof verification, and proof digestion (or understanding). Source.
↩︎ - For similar reasons, sometimes an unsolved problem can be more useful for human mathematical understanding than a solved one (for sociological reasons alone). When problems are unsolved, people try to work on them. Compare this to solved-yet-dead fields.
↩︎ - As a metaphor: One could imagine mathematics as a mysterious, opaque language that we access through unclear means (mathematical sensory perception, physical intuition, formal proofs, mental thought, etc.) Right now, we are a community who all speak this language to various degrees, and many of us spend immense amounts of time listening, understanding, and learning this language from each other. But we learn it so that we can one day speak it (that is, generate useful, not necessarily novel, mathematical proofs), and all of us expect that speech production eventually becomes part of our regular linguistic experience. In other words, mathematics today is a living language.
Now imagine that we were able to learn many new words in this language through artificial means, and discovered that it is vastly more efficient to access this language through such means. Imagine we completely relinquished speaking this language in light of that: clearly it is just more efficient to digest and learn the language collectively than to continue speaking it ourselves. The language becomes a dead language; we study it the way we might study Ancient Greek or Latin, able to cobble some words and phrases together, but primarily acceding the production of words in this language to more authoritative source material.
People don’t learn dead languages very well.
↩︎ - See OpenAI’s charter.
↩︎ - Intellectual property alone is an ethical minefield: AI companies have disrespected intellectual property to an extraordinary degree. I think the scientific community at large is (rightfully) reluctant to lead with the logic of Disney and Nintendo, but again it bears mentioning that AI companies have extracted enormous amounts of wealth from harvesting our labour in unheard-of ways.
↩︎ - This is a verbatim quote from Dean W. Ball, the head of strategic futures at OpenAI. Source.
↩︎ - Even if all we offer is a sober assessment of the limitations of their current capabilities.
↩︎ - A turn of phrase borrowed from Max Weinreich in this essay.
↩︎ - In the sense that most theorems are not useful products for taxpayers or for industry, and our professional norms have created the theorem economy.
↩︎ - On a personal level, I feel there is a sense in which it is fine if I do not succeed as an AI dissenter. If the working conditions of mathematics in the future become
• “Use LLMs as much as possible to pump out theorems and produce new mathematics” in the near-term; and
• “Read interesting LLM-generated frontier mathematics in my spare time while the thrust of my job becomes teaching undergraduates who offload cognition onto LLMs where possible, knowing that I have aided and abetted AI development” in the long-term,
then I’m not very interested in doing mathematics. For me, the answer will be simple: I’ll just leave and pursue another career which is less sensitive to automation. So will many other PhD students, postdocs, and people without job security; they’ll go off to work in the relational sector, finance, the military, and for AI companies.
When I talk to other PhD students my age, the overwhelming consensus is that we will leave if working conditions deteriorate in the LLM-uptake regime. So if we all do nothing and give into entropy, the outcome is simple: early-career researchers will leave. The research mathematics community will be impoverished as a result, but I won’t actually have to suffer the consequences; you will.
↩︎ - A few of my thoughts here being informed by this comment of Mario Pasquato. ↩︎
Leave a Reply