Evaluating mathematical merit

AI-Reproducible Work, Human Mathematical Contribution, and a Residual Evaluation Framework

by Qi Guo, Associate professor at the School of Mathematics, Renmin University of China

Personal principles. AI is a tool. In evaluating a paper as a human research contribution, every component that standard AI tools can reproduce should be subtracted, even if a human happened to perform that component manually. Credit should be assigned to the genuinely indispensable human input: choosing the problem, making a decisive observation or reformulation, creating concepts, organizing a theory, and assuming mathematical responsibility.

Correctness is judged by the same standard with or without AI assistance. Substantive use of AI tools should be disclosed by function: for example, literature retrieval, counterexample search, symbolic computation, proof repair, formalization, or drafting. Public disclosure should normally describe what the tools did rather than advertise a particular product; exact system metadata may be kept in a separate reproducibility record when technically necessary. Hidden use cannot be inferred reliably from prose style, so evaluation should apply the same capability baseline to every paper rather than rely on “AI detectors.” A clear account of failed routes, repairs, rewriting, and tool use is a modest positive factor, but it does not replace mathematical originality.

Mathematics is not merely the solution of named conjectures. Definitions, examples, classifications, methods, simplifications, structural explanations, and the creation of a field may be more valuable than solving a supplied problem. More fundamentally, mathematics sometimes advances not by solving a problem, but by constructing a framework in which the problem ceases to be a problem.

Current strengths and limits of AI tools

AI tools should not be accepted at face value. They can produce plausible but incorrect arguments and equally plausible but incorrect objections. The appropriate standard is repeated, adversarial, and independent checking rather than automatic trust or automatic dismissal. Recent research-level examples show that these tools now belong inside serious mathematical workflows, but their output still requires human responsibility and external verification [2, 3, 4].

A useful analogy is high-speed rail. AI tools can carry a mathematician much farther with substantially less routine effort, but they do not choose the destination, identify the correct line, or place the researcher at the station. One must still decide where to go, why the destination matters, and how to reach the point from which acceleration becomes useful.

No.StrengthTypical mathematical contribution
1Counterexample constructionSearch structured families, construct witnesses that refute universal statements, and expose hidden assumptions or nonuniform steps.
2Local logical checkingVerify medium-length deductions, algebraic identities, parameter ranges, and compatibility of hypotheses.
3Cross-field retrievalLocate potentially relevant tools in distant areas and compare established frameworks.
4Parallel strategy searchGenerate, rank, combine, and discard many proof routes more systematically than unaided search.
5Symbolic and numerical workUse computer algebra, interval arithmetic, exhaustive enumeration, simulation, and code-assisted estimates.
6Formal verificationTranslate suitable arguments into proof-assistant languages and use kernel checking as an exact local signal inside the formalized statement.
7Expository reorganizationExpand omitted steps, normalize notation, and produce alternative presentations for human checking.
Table 1: Practical strengths of AI tools.

The common feature is an explicit local objective: a candidate can be tested, compared, or rejected. The more completely the mathematical search space is specified, the more effectively AI tools can operate inside it. Recent cases occupy different parts of the table: the reported Jacobian counterexamples mainly use counterexample construction and exact symbolic verification (1, 5); the recent advance on zeros of the Riemann zeta function combines cross-field retrieval, parallel strategy search, symbolic computation, and formal verification (3–6); the reported Crouzeix proof is mainly a branching agentic search with local audits, followed by formal and expert checking (2, 4, 6) [22–27].

A tool, not a mathematical person. Conversational fluency, emotional responsiveness, and static benchmark performance should not be confused with general adaptive understanding. In a recent data snapshot, selected frontier systems scored roughly 90–93% on the static ARC-AGI-2 task, above the reported human average of about 66%, while the same class of systems scored only about 8–30 against the human-normalized baseline 100 on the interactive ARC-AGI-3 task [17, 14, 15, 16]. The exact values depend on configuration and protocol; the structural contrast is the point. Smooth dialogue may be useful, but it is not evidence of broad mathematical judgement.

In a livestream, Go player Ke Jie described an adversarial strategy against Go AI: play apparently foolish moves, induce the engine to misread an unusual global shape, and then execute a precise large-group capture. The case he discussed involved Ryunosuke Shibano, Toramaru Shibano’s elder brother, who is himself a professional Go player [28, 29]. Controlled studies give the stronger evidence: weak, human-comprehensible adversarial policies can defeat superhuman Go engines through persistent cyclic blind spots, and fresh attacks can bypass natural defenses [30, 31]. This does not settle a philosophical definition of understanding, but it shows that superhuman performance on a fixed distribution is not sufficient evidence of robust, transferable understanding.

Announcements by AI companies should also be read partly as business communication. Product names, selected benchmark milestones, and dramatic research claims are designed to attract attention. Their mathematical content should be assessed only after the theorem, proof, provenance, and independent verification are separated from the publicity surrounding them.

Problems favorable to AI tools

The probability of successful AI assistance rises when several of the following features occur together. No single feature is decisive, and a short proof may contain a profound observation. The table is a structural diagnostic, not a theorem about difficulty.

No.FeatureWhy it helpsRepresentative examples
1The proposed statement is falseA counterexample often lies in a large but searchable family, and one valid witness refutes the universal claim, although restricted or lower-dimensional cases may remain.The recently reported Jacobian counterexamples refute the conjecture for n ≥ 3; the case n = 2 remains open [22, 23]; Gaussian Moments counterexamples [5]; Chapters 3, 4, and 10 of Ten Advances [1].
2A computable objective is availableCandidate constructions can be scored automatically, making evolutionary or reinforcement search effective.FunSearch on cap sets and AlphaEvolve on matrix multiplication [6, 7].
3The proof is modularIndependent lemmas can be delegated, checked, and recombined with limited long-range dependence.Several chapters of Ten Advances; formal proof-search systems [1, 8, 9].
4The main tools already existThe main difficulty is to identify and assemble known results rather than invent a new mathematical language.The recent zeta-zero advance combines established analytic inputs with a new rank–trace argument on Weil’s Hermitian form [24, 25].
5The bottleneck is calculation or case controlSymbolic computation, search, and automated consistency checks directly attack the main obstacle.Arithmetic complexity, lattice reductions, cryptanalytic search [1, 4], and exact verification of the Jacobian witness [22, 23].
6The target is precisely formalizableA proof assistant or executable verifier supplies immediate feedback and blocks errors within the formalized kernel; it does not validate the chosen statement or model.AlphaProof-style and Lean/Rocq-assisted proof search [8, 9, 13], including the Lean formalization of the zeta result [24].
7Many routes admit cheap local rejectionParallel search is effective when the route space is large but most branches can be pruned quickly.Generator–verifier agents and computational exploration loops [12, 11]; the reported 16-hour branching and adversarially audited search for Crouzeix’s conjecture [26, 27].
8The compressed proof is short and locally checkableShallow dependencies and explicit identities make clean-room reproduction and adversarial checking easier.Many olympiad, benchmark, and short open-problem solutions; the explicit Jacobian witness [22].
Table 2: Features of problems especially amenable to AI tools.

These examples should not be conflated. The Jacobian result is a directly checkable witness. The zeta result is a substantial synthesis and search result: it proves unconditionally that at least two thirds of the nontrivial zeros, counted with multiplicity, are simple and lie on the critical line; it does not prove the Riemann hypothesis or determine the remaining zeros [24, 25]. The Crouzeix case is agentic proof search under an explicit branching protocol. Its institutional lesson is not that verification has become optional, but that provenance and human responsibility have become more important [26, 27].

Once these strengths are known, the value of a new paper can be assessed more sharply. The theorem itself may remain important, but the human credit for proof execution weakens when the core argument belongs to a well-established AI strength and can be reproduced in a clean-room run. Thus human originality is assessed after subtracting AI-reproducible work. The subtraction concerns attribution, not mathematical truth, and it applies equally to routine work done manually and to routine work done by a model.

AI agents and the reasoning horizon

An AI agent places one or more models inside a persistent research process. Relative to a single response, the main gains are external memory, task decomposition, parallel exploration, and repeated verification. Schematically,

effective reasoning horizonpersistent state+decomposition+parallel search+verification loops.\small{\text{effective reasoning horizon}\approx \text{persistent state}+\text{decomposition}+\text{parallel search}+\text{verification loops}.}

This is not a numerical identity. It explains why an agent can retain definitions and failed routes, divide a long project into stable subproblems, and sustain attention over a longer trajectory. Agentic systems use persistent workspaces, generator–verifier–reviser loops, parallel workstreams, and compiler feedback to extend the effective horizon of a base model [12, 10, 11, 9].

A difficult problem may be compared with a maze in fog. Agents can remember explored corridors, revisit earlier branches, and dispatch searches along several visible directions. In a sufficiently large or partially hidden maze, however, they may not know which corridors exist, whether they are moving toward an exit, or whether a route merely returns to an earlier region. A symmetry, invariant, monotonicity formula, geometric model, or change of variables can reveal a global direction. Mathematical intuition often consists in discovering precisely such a coordinate.

Two panels. Left, labelled Acceleration: a train travelling along a track toward a flag. Right, labelled Foggy maze: a maze of corridors partly obscured by fog, with a dashed arrow doubling back on itself and a star marked global invariant whose arrow points to the exit.

Agentic scaffolding can extend memory, concentration, and effective thinking time. It does not by itself create the invariant, category, or conceptual architecture that makes a large search space mathematically intelligible.

Sub-AI, AI-critical, and super-AI work

The following terminology is relative to the current frontier of standard AI tools and ordinary computational budgets. The boundary will move as models and agents improve.

  • Sub-AI work. The essential mathematical content can be reproduced by standard AI tools with little indispensable human input. Human work is mainly selection, editing, verification, or presentation.
  • AI-critical work. AI tools perform substantial search or proof execution, but human choices remain necessary for direction, significance, repair, or conceptual organization.
  • Super-AI work. The decisive step depends on a concept, reformulation, or theoretical architecture that standard AI tools do not recover under a reasonable research budget.

On this attribution rule, a result produced directly by an AI tool is Sub-AI as proof-production credit for a human who merely supplies the statement and transmits the output, even if the theorem is difficult or historically important. Knuth’s reported Hamiltonian-cycle construction, the reported direct Jacobian counterexample, autonomous cryptanalytic attacks, and direct Gaussian Moments counterexamples belong to this category unless a mathematically indispensable human observation can be identified separately [3, 22, 4, 5]. This classification does not diminish the mathematical truth or importance of the result; it limits the human credit that may be claimed for producing it.

The Crouzeix episode also shows why classification should be applied to contribution units rather than attached mechanically to an entire paper. Problem selection, prompt architecture, verification, exposition, and responsibility may be AI-critical human units, while the decisive proof unit may be Sub-AI if the public provenance record correctly reports autonomous generation [26, 27].

A practical classification test

The classification should not be inferred from page count alone. A one-line observation may carry more human originality than a forty-page derivation. The decisive issue is the location and necessity of the contribution.

No.TestEvidence for Sub-AIEvidence for AI-critical or super-AI
1Capability baselineSeveral frontier tools independently recover the core proof from a precise statement and standard literature access.Repeated clean-room attempts fail unless a human supplies a decisive representation, invariant, or reformulation.
2Counterfactual human necessityRemoving the human changes mainly the exposition, timing, or choice among already visible routes.Removing the human changes the problem, the search space, or the only viable route.
3Location of noveltyNovelty lies mainly in local execution, computation, or combination of established modules.Novelty lies in choosing the right object, changing coordinates, building a theory, or opening a new class of questions.
4Provenance and aftermathThe final proof is essentially machine-produced and generates few new structures beyond the solved statement.Dated notes and failed routes locate the decisive human input; the observation generates further theorems, methods, or a new framework.
Table 3: A practical test for distinguishing the three classes.

Applied to the ten results in the Ten Advances collection, this test classifies the machine-generated core units as Sub-AI for the purpose of human attribution, because the collection presents those units as obtained directly by an internal model [1]. Separate human units, if any, should be evaluated independently. Their compressed core proofs range from concise constructions to long, technically sophisticated architectures. That variation matters for mathematical difficulty, but it does not by itself restore human originality. The classification says nothing about correctness, theorem importance, or technical sophistication.

Human mathematical contribution

A brilliant human observation is identified by what it does to the mathematical state space, not by the number of lines it occupies. The strongest evidence is counterfactual: before the observation, the search remains diffuse or structurally blocked; after it, the problem is reorganized around a narrow mechanism. Such an observation may be a new invariant, a change of variables, an unexpected model, a hidden equivalence, the correct scale, or a definition that unifies previously isolated results.

No.ContributionMathematical effectEvidence in an evaluation
1Problem selection and directionIdentifies a central phenomenon and rejects technically difficult but peripheral questions.The problem choice changes what the field regards as worth understanding; the author can explain the surrounding alternatives.
2Decisive observation or reformulationCollapses a large search space, introduces the right coordinate, or converts the problem into a tractable one.Clean-room tools fail without the observation and advance after receiving it; dated notes, drafts, or discussions locate the step.
3Concept and theory creationIntroduces objects, invariants, definitions, or a general theorem that organizes many results.The idea has several consequences, simplifies earlier arguments, or generates a new research program.
4Judgment, verification, and responsibilityDetects failure modes, chooses sufficient evidence, repairs arguments, and states limitations honestly.Independent checking, oral defense, version history, and a transparent account of AI use and human correction. The author can reconstruct and defend every essential step.
Table 4: Human contributions that cannot be read from proof length alone.

Current standard AI tools do not reliably supply these contributions because the objective and the relevant state space are not fully specified. Problem selection requires comparing counterfactual research programs; decisive reformulation changes the search space rather than optimizing inside it; concept creation proposes a reusable language before its consequences are known; and responsibility requires stable judgement, explanation, and correction across time. AI tools can assist once a representation and a criterion are supplied, but they remain much weaker at deciding which representation and criterion should exist.

This is also why solving a named conjecture should not be assigned automatic priority. A proof may merely solve a supplied statement by a familiar route. Conversely, a new definition, a classification theorem, a canonical example, a simplification, or a conceptual bridge between fields may alter mathematics more deeply. The strongest advance may turn the original question into a corollary, a change of coordinates, or an instance of a general theory: it makes the problem cease to be an independent problem. The relevant question is not only “Was a problem answered?” but “What new structure became visible?”

Human mathematicians retain their strongest advantage where the objective is not fully specified in advance: choosing the problem, reformulating it, creating concepts, building theory, judging significance, mentoring a community, and accepting responsibility. AI tools are increasingly strong inside a mathematical space; the highest-leverage human work often defines or changes that space.

A stable AI-residual evaluation system

The evaluation system should separate mathematical value from attribution. Correctness and fidelity to the stated theorem form a noncompensable gate. A paper that passes this gate is then evaluated through three primary outputs:

V(P): mathematical value,A(P): AI-deductible fraction,SH(P): residual human contribution.\small{V(P):\ \text{mathematical value},\qquad A(P):\ \text{AI-deductible fraction},\qquad S_H(P):\ \text{residual human contribution}.}

The capability subtraction applies to every paper. Even when an entirely human-written paper is not marked as having used AI, any component that the frozen AI capability baseline can reliably reproduce is deducted from the human-contribution score. The system therefore evaluates capabilities, not writing style or unverified claims about tool use.

The basic computation is as follows. First, compress the central theorem families into a small set of indispensable contribution units u. Give each unit a normalized weight ωu according to its type, scope, and criticality: problem formation, decisive observation, method or theory, technical closure, and integration or generativity are distinguished, with decisive observations receiving the largest base weight. Thus

uωu=1.\sum_u\omega_u=1.

Second, estimate the deductible fraction a_u \in [0,1] of each unit under a frozen multi-system baseline. Repeated clean-room runs are conducted at four information levels: the problem alone, the problem plus standard tools, a proof skeleton with the decisive step hidden, and the decisive step supplied with only technical closure remaining. If uh is the robust median success rate at level h, the capability estimate is

aucap=0.50p~u0+0.30p~u1+0.15p~u2+0.05p~u3.a_u^{\mathrm{cap}}=0.50\widetilde p_{u0}+0.30\widetilde p_{u1}+0.15\widetilde p_{u2}+0.05\widetilde p_{u3}.

A verifiable provenance record supplies a lower bound auprov, and

au=max{aucap,auprov},A(P)=uωuau.a_u=\max\{a_u^{\mathrm{cap}},a_u^{\mathrm{prov}}\},\qquad A(P)=\sum_u\omega_u a_u.

This prevents a known machine-generated contribution from being credited to a human merely because a later reproduction attempt failed, while also avoiding the false inference that reproduction failure proves human originality.

Third, use calibrated paper-level scores I, R, E ∈ [0,1] and unit-level scores cu, mu, hu ∈ [0,1], representing conceptual originality, method value, and human indispensability. Set

C0=uωucu,M0=uωumu.C_0=\sum_u\omega_uc_u,\qquad M_0=\sum_u\omega_um_u.

The mathematical value of the result is

V(P)=100(0.25I+0.30C0+0.30M0+0.10R+0.05E).V(P)=100\bigl(0.25I+0.30C_0+0.30M_0+0.10R+0.05E\bigr).

This score does not decrease merely because the result was found by a machine. For attribution, define

IR=(1A)I,CR=uωu(1au)cu,MR=uωu(1au)mu,I_R=(1-A)I,\qquad C_R=\sum_u\omega_u(1-a_u)c_u,\qquad M_R=\sum_u\omega_u(1-a_u)m_u,
HR=uωu(1au)hu.H_R=\sum_u\omega_u(1-a_u)h_u.

The residual human-contribution score is

SH(P)=100(0.15IR+0.20CR+0.20MR+0.35HR+0.05R+0.05E).S_H(P)=100\bigl(0.15I_R+0.20C_R+0.20M_R+0.35H_R+0.05R+0.05E\bigr).

Robustness and exposition remain small, non-deducted terms because human authors still bear responsibility for checking, presentation, and limitations; together they cannot convert a machine-generated core proof into a high human-contribution score. Responsibility is nevertheless a gate rather than a compensatory bonus: if the named author cannot reconstruct and defend the core argument, the paper fails the correctness/provenance gate even when the theorem later appears to be true.

For high-stakes use, expert ratings are anchor-calibrated and aggregated by robust medians, AI estimates are reported with uncertainty intervals, and the central weights are allowed to vary within a prescribed range. A paper is ranked above another only when the ordering survives all admissible weights and input intervals; otherwise the papers are placed in the same robust tier. The final report is

(G,V,A,SH,[SH,SH+],T),\bigl(G,V,A,S_H,[S_H^-,S_H^+],T\bigr),

where G is the correctness status and T is the robust tier. Full rubrics, frozen data formats, calibration rules, interval calculations, and the reference implementation are given in the companion Sare-Math methodology [21].

Education and knowledge networks

The claim that learning mathematics is unnecessary because AI tools exist confuses possession of a tool with the ability to formulate, evaluate, and direct mathematical work. It is a simple conceptual error and should not determine curriculum design. A student who cannot reason independently is poorly equipped to notice when an AI tool has changed a hypothesis, invoked a false lemma, or optimized the wrong quantity.

AI can often explain one definition or one exercise quickly. If that item remains isolated, however, the explanation fades and the learner returns to essentially the same state. A course is genuinely learned when definitions, examples, theorems, counterexamples, and proof mechanisms are connected into a stable knowledge network. Once the network is built, many exercises cease to be separate problems: they become special cases, short paths through the network, or consequences of a general principle. Mathematics often advances not by solving more and more problems individually, but by finding a framework in which the former problems are no longer problems.

Two panels. Left, labelled Fragmented clarification: isolated boxes for definition, theorem, example, exercise, formula and counterexample with weak connections. Right, labelled A connected knowledge network: the same items joined by solid edges with Invariant at the centre, and a highlighted path running from a former problem through the invariant to a theorem.
Figure 2: From fragmented clarification to a connected knowledge network.

A recent qualitative case study of 75 students aged 13–15 in Slovenia, Spain, and Germany found that uncritical reliance on generative AI in poems, logos, and short stories often turned the tool into a shortcut through the demanding stages of idea formation, experimentation, and revision. The study cautions that such overuse can weaken creative confidence, ownership, and long-term creative development. The educational implication is that adolescents should first generate and test their own ideas, while AI should be introduced as a scaffold rather than a substitute for the creative process [32].

Training should therefore become more demanding in a constructive way. Routine exercises remain necessary for fluency, but advanced work should move toward AI-critical tasks: selecting a useful question, testing whether it is meaningful, finding the right formulation, designing counterexamples, and defending the argument under adversarial questioning. Young researchers should be expected not only to solve assigned problems but also to find problems whose solution would organize a genuine phenomenon.

AI tools may reduce the number of researchers needed for routine theorem production. That is not an argument for weakening mathematical education. Broad mathematical training remains valuable, while frontier research positions may become more selective and more conceptually demanding. The appropriate response is educational reform, not intellectual retreat.

No.StageCore trainingAI-era extension
1Foundational studyDefinitions, examples, calculations, proof writing, and independent problem solving.Use AI tools only after an independent attempt; compare proofs and locate hidden assumptions.
2Advanced courseworkStructural theory, canonical examples, counterexamples, and links between fields.Build knowledge networks; audit, repair, simplify, or generalize AI-generated arguments.
3Doctoral trainingDeep command of a field and the ability to complete a long proof.Require problem formulation, literature discrimination, contribution-graph analysis, and work on AI-critical rather than routine Sub-AI tasks.
4Early researchIndependent results and participation in a research community.Evaluate the ability to choose questions, create a conceptual route, coordinate tools, and explain the indispensable human contribution.
Table 5: A compact training program for the AI era.

Journals, attribution, and comparative selection

Even if the number of submissions grows rapidly, a journal remains a mechanism for selecting a distinguished subset of the credible papers available during the same period. Correctness is the entry gate; within the admissible pool, different venues legitimately emphasize different mixtures of conceptual reach, field-specific depth, methodological value, robustness, and general interest. Comparative selection therefore remains meaningful even when proof production becomes abundant.

The residual evaluation framework clarifies what a journal should compare. The mathematical value V asks what the result adds to mathematics; the deductible fraction A records how much of the core lies within the frozen AI capability baseline; the residual score SH asks what indispensable human contribution remains. These quantities should not be collapsed into venue prestige or into a single universal ranking. A mathematically important result may have high V and low SH; a less spectacular theorem may contain a genuinely field-shaping human concept.

A mature publication system should not present a Sub-AI result as ordinary human-original research when the human role is only to polish or transmit a machine-generated proof. Such work may still deserve dissemination as a mathematical or computational result, but its provenance and attribution must be explicit. AI-critical and super-AI papers remain fully publishable under transparent attribution, ordinary mathematical review, and human responsibility. Current policies already require human authors to assume responsibility and disclose substantive AI-tool contributions [18, 19, 20].

The proof of a famous conjecture will still attract attention, but journals should ask whether it creates reusable methods, produces structural understanding, or only solves an inherited statement without changing the surrounding theory. Conversely, papers that build concepts, classify structures, simplify a theory, or open a field may deserve the strongest consideration even without resolving a named problem.

A practical transition

The preceding principles can be implemented without creating a large new bureaucracy. The essential measures are:

  1. Freeze the evaluation object. Record the paper version, literature cutoff, frozen capability baseline, computational configuration, budget, scoring rule, and evaluation hash so that later reruns do not silently change the baseline.
  2. Preserve provenance. Disclose the mathematical tasks performed by AI tools, the agent structure, task specifications, code, failed routes, and decisive human interventions. Public prose should normally describe functions rather than product names; exact system and version metadata, when needed for reproducibility, belongs in a separate provenance record. Transparency earns only modest credit, but it is necessary for attribution.
  3. Maintain a correctness gate. Major claims require adversarial checking and, where possible, independent reproof using a different team or toolchain. Human authors remain responsible for every theorem they sign.
  4. Separate result value from human residual. Report V, A, and SH rather than one opaque score, and apply the same capability subtraction to all papers, including papers with no AI-use label.
  5. Use intervals and robust tiers in high-stakes decisions. Calibrate reviewers with field-specific anchors, aggregate robustly, and refuse false precision when plausible weights or AI estimates can reverse an ordering.
  6. Reform training and infrastructure together. Preserve independent proof writing, counterexample construction, oral defense, and problem formulation; credit decisive observations, error detection, impossibility results, reusable software, and theory building; protect confidential manuscripts and support public mathematical infrastructure so that progress does not depend entirely on proprietary systems or unequal compute access.

These measures refine familiar standards of rigor, attribution, and responsibility for an environment in which search and proof execution are much faster.

Conclusion

AI tools make it easier to travel farther in mathematics, and agents make the journey longer, more organized, and less forgetful. Yet acceleration is not orientation. In a foggy mathematical landscape, local search may loop back to an earlier point, while the decisive contribution may be the observation that reveals a hidden coordinate, the definition that organizes a phenomenon, or the theory that makes an entire family of former problems transparent.

The future mathematical system should therefore separate correctness, mathematical value, AI-reproducible work, residual human contribution, and responsibility. Journals will continue to select outstanding work from the contemporaneous literature, but evaluation should increasingly recognize the ability to choose important problems, make decisive observations, create concepts, build theories, train other mathematicians, and exercise reliable judgment. As routine proof production becomes more abundant, the value of genuine mathematical understanding becomes easier to see.

References

  1. OpenAI, Ten Advances in Mathematics and Theoretical Computer Science, 2026. Link
  2. OpenAI, Our First Proof Submissions, 2026. Link
  3. D. E. Knuth, Claude’s Cycles, 28 February 2026, revised 14 April 2026. Link
  4. Anthropic, Discovering Cryptographic Weaknesses with Claude, 28 July 2026. Link
  5. C. D. Long, Small Counterexamples to the Gaussian Moments Conjecture, 2026. arXiv:2607.18186
  6. B. Romera-Paredes et al., “Mathematical Discoveries from Program Search with Large Language Models,” Nature 625 (2024), 468–475. Link
  7. A. Novikov et al., AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery, 2025. arXiv:2506.13131
  8. T. Achim et al., Aristotle: IMO-Level Automated Theorem Proving, 2025. arXiv:2510.01346
  9. G. Tsoukalas et al., Advancing Mathematics Research with AI-Driven Formal Proof Search, 2026. arXiv:2605.22763
  10. D. Zheng et al., AI Co-Mathematician: Accelerating Mathematicians with Agentic AI, 2026. arXiv:2605.06651
  11. L. Chen, Z. Liu, W. He, and B. Dong, Iteris: Agentic Research Loops for Computational Mathematics, 2026. arXiv:2606.02484
  12. Google DeepMind, Accelerating Mathematical and Scientific Discovery with Gemini Deep Think, 2026. Research note
  13. G. Baudart, M. Lelarge, T. Stérin, and J. Viennot, Putnam 2025 Problems in Rocq Using Opus 4.6 and Rocq-MCP, 2026. arXiv:2603.20405
  14. F. Chollet, M. Knoop, G. Kamradt, B. Landers, and H. Pinkard, ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems, 2025. arXiv:2505.11831
  15. ARC Prize Foundation, ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence, 2026. arXiv:2603.24621
  16. ARC Prize Foundation, ARC-AGI-3 Scoring Methodology, 2026. Link
  17. Q. Guo, Frontier Generative AI Model Evaluation and IQ-Scale Equivalence Analysis, research report, data snapshot 29 July 2026.
  18. Leiden Declaration on Artificial Intelligence and Mathematics, 2026. Link
  19. American Mathematical Society, Use of Artificial Intelligence, journal policy, 2026. Link
  20. Annals of Mathematics, Submission Guidelines: AI and LLM Policy, 2026. Link
  21. Q. Guo, Stable AI-Residual Mathematical Evaluation: AI Subtraction, Robust Aggregation, and Stable Ranking, companion methodology and reference implementation, August 2026. SARE Evaluation System
  22. T. Tao, “A digestion of the Jacobian conjecture counterexample,” 21 July 2026. Link
  23. S. Gao, “Counterexamples to the Jacobian conjecture in dimensions greater than two,” 2026. arXiv:2608.00222
  24. L. Alpöge and R. Furman, “More than two thirds of the zeta zeros are simple and on the critical line,” 2026. arXiv:2608.13637
  25. Anthropic, “Learning more about Claude’s mathematical capabilities,” 10 August 2026, updated 13 August 2026. Link
  26. S. Jin, “The Numerical Range Is a 2-Spectral Set,” version 4, 7 August 2026. Link
  27. A. Townsend and A. Greenbaum, “The Neurosurgery Resident Who Proved Crouzeix’s Conjecture,” 15 August 2026. Essay
  28. Ke Jie, livestream discussion of an adversarial strategy against Go AI, video clip, accessed August 2026. Link
  29. Nihon Ki-in, professional profile of Ryunosuke Shibano. Link
  30. T. T. Wang et al., “Adversarial Policies Beat Superhuman Go AIs,” Proceedings of ICML 2023, PMLR 202, 35655–35739. Link
  31. T. Tseng, E. McLean, K. Pelrine, T. T. Wang, and A. Gleave, “Can Go AIs Be Adversarially Robust?” AAAI 2025, 27662–27670. Link
  32. D. Pipuš, “Student Creativity in the Age of AI,” Proceedings of the World Conference on Education and Teaching 4(2) (2025), 14–30. Link

4 responses to “Evaluating mathematical merit”

  1. Joe Lee Avatar
    Joe Lee

    Please, no more AI generated posts.

    1. SM Avatar
      SM

      Indeed!

  2. AS Avatar

    I see the author woke up and chose violence. 🙂

    I encourage readers to see past the stylistic choices and engage with the substance of what’s being said.

  3. just different Avatar

    When was this submitted?

Add to the discussion

Discover more from Proofs and Prompts

Subscribe now to keep reading and get access to the full archive.

Continue reading