Navigating the Risks of AI in Academia

Sahana Balasubramanya, Assistant Professor at Lafayette college

We are in an age in which artificial intelligence is no longer a technological curiosity or a tool used by specialists. Generative AI can now write, summarize, translate, code, solve mathematical problems, and assist with almost every stage of academic work. The opportunities are enormous. So are the risks. This post focuses on the risks in academia, while others posts (this one penned by Vadim Lebovici and another by Alexis Marchand) explore the environmental risks associated with AI use.

It is not a question whether we will use AI or not. We will. Full disclosure – I used ChatGPT to help me organize my many thoughts into subtopics to help me draft this article! I also used it to proofread the final draft.

The more difficult question to contend with is how we should use it while preserving the values that make education and research meaningful: independent thought, intellectual honesty and copyright, human judgment, authorship, accountability, and fairness. As an academic, I see a particularly urgent need for these topics to be addressed in universities, both on the teaching and research sides. We cannot wait until the technology stops changing before establishing standards. The technology is a moving target; our principles, goals, ethical and educational standards should not be.

Learning and methods of assessment

As most instructors are already aware, our students are using various AI sources in their day-to-day lives. They use it to explain a concept, solve a problem, write an essay, produce computer code, and more. This can be useful when used to provide explanations at different levels, offer alternative approaches, and has the potential to help students overcome barriers to understanding.

But there is a fundamental distinction between having an answer and having learned how to obtain the answer. Learning requires struggle. It requires making mistakes, developing intuition, pattern recognition, understanding why a method fails, and at the highest level, being able to reproduce and extend an idea independently. The risk of utilizing AI lies in the elimination of much of this process, where students become efficient at producing answers without developing the intellectual ability required to generate them or adapt to (even slightly) different problems. This concern is not merely about academic honesty. It is about the value of education itself, whose ultimate goal is to help students develop the mechanisms of how to think and learn. For the interested reader, this lovely post by Amruth Srivathsan delves deeper into the value of the struggle, while Benjamin Jaye and Galyna V.Livshyts postulate assignments to combat the problem in this well-thought out post. Ph.D. students are also considering this issue; Baptiste Serraille pens this thoughts on rethinking math education in this post, while Trevor Cheung makes a case for the value of human teaching in this post.

The other side of this topic is that of assessment. Take-home essays and unsupervised assignments have been used to assess understanding and provide students with the space to think and produce work of sufficient quality. But if an AI system can produce a sophisticated response in seconds, it becomes increasingly difficult to know what the student actually understands. On a personal note, this has caused me to stick to traditional in-class exams as a major method of assessment, especially at the level of Calculus classes. Since these concepts are fundamental to all branches of science, I remain particular about ensuring that students are actually learning some of the skills they will need in the future. I remain more flexible on homework assignments however, allowing students multiple attempts at problems without penalty.

Another avenue that has suffered with the advent of AI is in the dwindling numbers we see in office hours. I am aware that some traditional methods of assessment can sadly be a barrier to learning, and do wonder how to strike a balance between accessibility issues and my job as an instructor on this front. A race between AI generators and AI detectors is unlikely to provide a durable solution to this problem. I instead lean towards designing assessments and setting and communicating standards around the intellectual abilities I actually want students to acquire. The mixed use of oral examinations, presentations, problem-solving sessions, and other forms of direct interaction may become more important in this regard.

Research and confidentiality

As AI is becoming better at attempting high level research problems, the avenue is also undoubtedly affected and must overhaul its own operating procedures. The nature of Ph.D. projects themselves may require an overhaul. That AI can viably produce work that would be acceptable as a Ph.D. thesis is a cause for concern. It creates opportunities to produce papers quickly rather than spend years developing deep expertise and genuinely independent research programs. This throws into serious question as to what meaningfully constitutes a sufficient quantity versus quality of work to be awarded an advanced degree, which is at its core supposed to be a testament to the depth of understanding. There is already the dual issue of the academic system churning out more Ph.D.s than the system can itself absorb and the resulting pressure on Ph.D. students to have multiple publications before graduation is rising. We risk exacerbating these issues and compromising quality if the use of AI remains unregulated in this regard.

One possibility is that doctoral programmes become longer and deeper, with greater emphasis on independent work, oral examinations and foundational knowledge. The goal would be less about producing a certain number of papers and more about demonstrating that the student has become capable of doing research independently. While this may sound like a step backwards, it is a recognition that some forms of intellectual development cannot be compressed. If AI accelerates the production of academic text, the scarce resource will become something else: deep human understanding –\text{\textendash} a sentiment echoed in this post by Alonso Castillo-Ramirez. Manuel Rivera makes a strong statement on the subject in this post, while Daniel Litt proposes a deeper schematic of what it means to produce quality mathematics in a Ph.D. program in another. No matter which model we decide to accept, this also means that institutions will need to revise their scholarship and tenure policies, and REU guidelines.

On the other hand, some scholars lean towards a model when AI can be ethically integrated into the research process. To offer another viewpoint, I encourage readers to see this post by Joshua Siktar.

Many conference websites host talk notes, slides containing proof strategies, and notes from problem sessions. While I support the sharing of information, a non-trivial amount of care must also be taken to protect the interests of students and junior faculty, who remain the most “at-risk” groups in academia. Tasmin Chu provides a strong opinion as a PhD student and AI dissenter in this post on the matter. Her thoughts align with those of Andrew Ng in this post. I also encourage the reader to peruse the (partly counter) reflections of Giorgio Mangioni in this post.

Another issue in this regard is that of access, since AI is not cheap and the disparity in access to resources across faculty is already large. As a faculty member at a liberal arts school, my research time is already constrained by a higher level of teaching and service than that of someone at an R1 institution (not a complaint, but an objective observation). At a time when higher education is already under intense scrutiny and job prospects remain uncertain, it would be unwise to add stressors in the form of unethical users who would seek to complete a project based on another’s ideas on an accelerated timeline using AI.

Equally important is the impact to the publication process, which is already stretched to provide responses in a timely manner. It would be to no one’s advantage to inundate our publication channels with a slew of articles; nor is it fair to ask academic faculty to review articles generated by AI companies. The peer review process is largely a courtesy that we as academics extend to one another, as service to the scientific community. The inherent reciprocity that exists in this process is lost when the article is not backed by a human author. I am also personally not comfortable with the review process being handed over to a machine as a potential quid-pro-quo; the reasons are similar to those I express in the next segment.

Grant proposal assessment

AI may also change how research proposals are evaluated, with some suggesting the decision could be handed to a machine to save time. Grant committees face an extraordinarily difficult task: comparing researchers working in different subfields, at different career stages, using different methodologies and asking fundamentally different questions. If some applicants use AI extensively to polish proposals while others do not, committees may effectively be comparing different kinds of documents. This does not necessarily mean that AI assistance should be prohibited. But it does mean that grant committees need to think carefully about what they are actually evaluating.

Another potential risk is the use of an AI system to sift through grant proposals and make selection decisions. While human judgement is not devoid of bias, I personally feel deeply uneasy about the prospect of outsourcing such decisions to an algorithm. An AI system might summarize applications, identify similarities, or suggest weaknesses. But ranking decisions about intellectual importance, originality, feasibility, and potential value require human judgment and debate. I believe it would be a loss for the community to do away with the process where committee members read proposals and exchange thoughts on the same, before making decisions. Such an exchange is important for opportunities for reflection, reconsideration and bias correction. In my opinion, grant evaluation should therefore remain fundamentally a human responsibility, even if AI tools are used to assist the process.

Most research statements contain unpublished ideas, confidential information, intellectual property, or sensitive data. Grant committees and hiring committees must also be held to the highest standards to protect intellectual property and to avoid conflicts of interest and “scooping” another researcher’s work. Confidentiality agreements or NDAs could be considered in this regard.

Authorship, attribution and intellectual honesty

Perhaps the most important principle in the age of AI is transparency. As Martin Hairer elaborates in this post (and touches upon other aspects in this vein), authors should clearly state when and how AI was used. Honest disclosure can reduce the panic and sensationalism surrounding AI. Researchers have always used tools: calculators, statistical software, databases, reference managers, spell-checkers, programming languages, etc. What matters is what the tool did and what the human researcher did, and that this information was openly and accurately disclosed, so that it could be verified independently.

For mathematics in particular, we must not lose sight of attribution. If an AI system produces a mathematical proof, AI should not receive credit for ideas that originated from human mathematicians. Mathematical ideas have histories, influences, predecessors, and intellectual contexts. Proper citation therefore becomes even more important. There are human beings on the other side of the AI who must be acknowledged.

Standards must be set by institutions

This invariably brings us to the responsibility of universities, colleges, departments, and other research institutions. Students and researchers should not be left to guess what constitutes acceptable AI use. Institutions and the research community in general need to set clear expectations about when and how AI can be used, when it must be disclosed, what constitutes unacceptable assistance and practices, and what information must never be entered into an AI system.

These standards should be realistic. A complete prohibition on AI is unlikely to be workable or enforceable. The appropriate rules will likely differ by discipline and activity. Using AI to improve grammar is different from asking it to write an essay completely. Using an AI tutor to understand a math concept is different from submitting an AI-generated proof as one’s own. Using AI to solve a complicated integral, differential equation or statistical computation is different from using AI to build and justify a model that might require such computations.

We need standards now, not perfect standards later. AI technology is changing so rapidly that any policy written today may look outdated tomorrow. But we do not need to predict exactly what AI will look like five years from now. We need a set of principles that can survive technological change, emphasize our ethics and uphold what we value most: human standards that make knowledge worth producing in the first place. The age of AI does not have to be the end of independent thinking, academic integrity, or human scholarship. But preserving these things will require deliberate choices (at both an individual and macro level) and immediate action.

I am grateful for the opportunity provided by this blog to put down my thoughts on a subject that will affect virtually every part of academic life. I also thank Denis Osin, Ekatrina Rybak, Dan Margalit, Danny Calegari and Priyam Patel for helpful conversations that shaped different aspects of this essay. The hope is also to invite further discussion: principled rather than reactionary, and above all reasonable and implementable. We will need to come together as a global community of scholars to do so, both to set equitable standards and also to retain our integrity.


Received 13 September 2026.

3 responses to “Navigating the Risks of AI in Academia”

  1. lohn_jennon Avatar
    lohn_jennon

    I respect the sentiment behind this post. However, since the advent of AI it is hard for me to imagine how we can survive the fundamental deskilling that we are facing right now.

    I think until we address this question of how to motivate people to continue doing math, discussions of incentive structures are probably not going to help that much. Almost everyone in my undergrad who was talented in math and went to top grad schools, were really interested in having their name attached to a theorem (or equivalently of producing new math).

    Who would the profession be attracting if they are really just going to sell the incentive as understanding LLM slop? I think we are going to lose out a generation of mathematicians right now (think about first year math majors starting right now).

    Curiously the only people so invested in LLMs doing math for them are tenured or near tenured faculty.

    1. Nick Galatos Avatar
      Nick Galatos

      It might help to sent the incentive structures first, instead. If there were prestigious journals that would publish only unassisted research, then people would be interested in it and others would value them more than someone who would be the first in prompting AI for a proof. This would disincentivize prompting.

      The obvious question is how to enforce this. But there are so many loopholes already (of hard to enforce policies) that one wonders how the math community has survived so far: you can pay someone to write a paper for you or your PhD thesis and claim it as your own; you can submit your paper to multiple journals and retract the others once one of them is accepted; you can scour the math literature in other languages (say Russian during the cold war) for unknown theorems to the west and claim them as your own, etc. It has been impossible to enforce all of these policies, but the consequences of dishonesty have been severe (even upper administrators lost their job for academic dishonesty). In any case, people who go to pure math do not do it for the money and are a self-selected group of higher ethical standards. I think we should maybe first try that approach of incentives, before giving up and fully embracing AI in everything.

  2. Darius Bacani Avatar
    Darius Bacani

    Balancing the benefits of AI with academic honesty is becoming a real challenge for students and educators. I think tools like Undetectable.ai’s AI Essay Writer fit into this discussion because using AI responsibly starts with knowing how to make it support your writing without replacing your own thinking.

Comments are moderated. Read our comment policy.

Add to the discussion

New posts by email.

Prefer a feed reader? Subscribe by RSS.

Also on Mathstodon.

Latest comments across the site.

  1. ramius4: Your first sentence is probably true from the viewpoint of mathematical theorists, but as others have said, it’s not...