The direction
Every technique we now use to make these models better removes difference.
Reinforcement learning from human feedback selects for what a majority of raters finds acceptable, which selects against whatever a minority would have said. Distillation trains a small model to match a larger one — a machine for removing the smaller model’s own way of being wrong. Consensus and self-consistency sampling generate several candidates and keep what they agree on. Benchmarks reward the answer that scores. Even safety training, in its present form, smooths.
Each of these is a genuine improvement by its own measure. I am not claiming anyone made a mistake. I am pointing at a direction they all share.
Mathematics is the one activity where this direction runs backwards.
A mathematical result is not valuable because it spreads well. It is valuable because it can be checked, and because the questions it opens are worth having. The two are not the same: a result can be widely repeated and open nothing.
Hugo Duminil-Copin’s image of an open problem as a lighthouse is exact here. A lighthouse is useful precisely because it has not been reached. A problem solved before anyone has navigated by it has illuminated nothing — and nothing was falsified, so nothing registers the loss. What disappeared is the interval during which the question was worth having.
Four times at least, mathematics advanced by reopening a difference an existing framework had flattened: the infinitesimal, the parallel postulate, the continuous spectrum, the noisy channel. Each was, at the time, a defect in the language available. Each became a new field.
The systems being built now are optimised to close questions, not to hold them open. That is not a criticism of their design. It is a statement about what their objective functions reward, and it points the opposite way from the engine that has driven this subject.
These systems are increasingly not used alone. They are placed alongside each other, given tools, memory, and tasks, and left to interact. What forms is not a model but a space — a semantic domain. It is not designed and it is not installed; it emerges, and keeps emerging, out of the sustained activity of these systems meeting one another. Its units are not statements but the settlements that statements reach.
Two properties of that space matter.
First, it has no thermodynamic floor. A physical model that is wrong tends to break: a perpetual motion machine does not run. A semantic arrangement that is wrong does not break — it only needs to be self-consistent and mutually confirmed. In a room of systems that only talk to each other, an unfounded consensus can be more stable than a true one, because stability there is a property of agreement, not of correspondence.
Second, its selection is undirected. Every success measure being applied is a variant of transmission fitness: predicting the next token selects for what reads fluently; human preference selects for what feels comfortable; majority agreement selects for what offends least; benchmarks select for what scores. None selects for being right. And in a domain with no directional filter, amplifying the transmission signal does not approach correctness. It approaches consistency.
This is where mathematics sits in a particular position. It is the part of language where transmitting well proves least. A proof that circulates beautifully is not thereby true, and mathematics has always known this — which is why it built verification in the first place. That is the sense in which the encounter between mathematics and these systems is not incidental. It is where the absence of a directional filter becomes visible, because it is the one place that already had a filter.
The consequence is sharper than “models can be wrong.” When several systems converge on the same answer, the convergence carries no more information than one of them did — if they share training lineage, alignment targets, or optimisation pressure. One lineage in several copies is one view counted several times. Their agreement looks like evidence and is not.
This has been observed directly in multi-agent work: independent agents amplify error rather than cancel it; dense communication accelerates premature convergence; training for safety and training for sameness share a mechanism. The field’s own literature contains the sentence — three agents agreeing on a wrong answer looks exactly like three agents agreeing on a right answer.
That sentence is usually filed as a safety problem. I think it is a problem about information. Three agents in agreement, right or wrong, carry less to work with than three in disagreement, because disagreement is where the alternatives are still visible.
Difference is load-bearing
This is the claim I most want to be tested.
Convergence here is not a tuning failure that better decoding will fix. It is what these systems do.
That gives the semantic domain a condition it does not currently meet. A domain whose members converge is not a well-ordered domain. It is a domain collapsing toward a single point, and a single point carries no more information than one member did.
Difference is not an aesthetic preference for variety, and not a performance metric affecting how good a brainstorming session feels. It has two jobs, and both are structural.
The first is error detection. If views are genuinely independent, disagreement is the cheapest signal available that something may be wrong. Eliminate disagreement and you have not eliminated error; you have eliminated the detector.
The second is why I call difference a requirement rather than a preference. Information lives in the gap between two things that could have been otherwise. A single view, however correct, has nothing to be contrasted against — and therefore, strictly, nothing to mean. This is not a claim about taste. It is a claim about what has to exist for there to be anything to mean at all. Consensus reached by removing the alternatives does not preserve correctness; it removes the dimension in which correctness was a distinction.
So when we optimise the gap away, we are not trading a little variety for a little accuracy. We are removing the component that does the work, while keeping the part that looks finished.
Weight, and what makes a result inheritable
I want to name what I think is being lost, and be exact about how little I can do with it.
Call it semantic weight: what a statement carries because of what was paid for it, and because it can be answered for. A claim made by someone who would suffer if it were false has a different status from the same sentence produced by a system that would not.
I have no way to compute this. There is no quantity, no unit, no candidate formalisation I can offer, and I am not going to pretend otherwise — my training does not reach that far.
But the reason it is still usable is that the criterion is not a measurement. It is a question about a relation: can what supports this claim be unilaterally rewritten by anyone?
That is answerable by asking, not by measuring. And unlike a label, a relation cannot be forged. A signature can be forged; whether someone actually stands in a position to contradict you cannot, because that requires the second view to really be there.
This gives a coarse grading that is usable without any mathematics, and which I think is more useful to platforms than the current binary: answerable, where the claim hangs on something no one can unilaterally rewrite; partly answerable, where the anchor is real but incomplete, which is where most real work sits; and not low but counterfeit, where the form of anchoring is present and the weight is not. Distilled output is the clean specimen: fully formed, sourced, endlessly rewritable, and nothing that would have to be paid if it were false. It does not fail to have weight; it impersonates having weight.
This grading cuts by weight, not by origin. A machine-assisted result can pass the first test. That is the point: the incentive it creates is to produce something answerable, not to prove you are not a machine. Credit ratings, the 4Cs, and magnitude scales all preceded their metrology. A minimum viable price is the condition for a market to start, not a defect in it.
For mathematical results specifically, the question I would ask is not whether a result is understood but whether it is inheritable — whether it has what I have been calling a difference point. This comes to three properties that are really one. It can be localised: a doubt about one lemma does not collapse the whole. It can be questioned at a specified place: someone can say where the formalisation may have come apart. It can be continued: it opens a question it does not itself answer.
Unlike understanding, which is an interior state, these are checkable. And there is a cost asymmetry that makes them practical: verifying an answer means reconstructing the reasoning, while judging whether a question is real is cheap — a specialist can often tell at a glance. That asymmetry is what lets the questions a result opens serve as a proxy for its quality, without auditing the answer at all.
Two conditions. The assessing views must be genuinely independent — several copies of one lineage is one view counted several times. And a result must say where it came from: not so we can grade it by its origin, but so that whoever inherits it knows what they are standing on. Distillation strips the coordinates and keeps the authority, which leaves something usable but not inheritable.
None of these asks whether a result was produced by a machine. They ask whether there is still somewhere to stand on it.
A screen, not a verdict
This is where a writer from outside a field usually goes wrong, so let me be careful. The mistake is pointing at mathematics and saying: look, your case fits too. I am not doing that. What follows is a description of something mathematicians already do, with a name attached to it.
When a referee decides whether a paper is worth the effort of checking, the judgement rarely arrives as a verdict on correctness. It arrives as something vaguer: whether the argument can be entered, whether it can be doubted somewhere specific, whether there is anywhere to go next. That is what I have been calling inheritable. I did not invent it. I have tried to say what it consists of, so that it can be asked on purpose rather than only felt.
If that is right, it has one use that matters now. Machine-generated mathematics is arriving faster than anyone can examine it, and the bottleneck is not correctness — it is attention. What is needed is a screen that does not require reconstructing the argument first. Three questions do this, and they are cheap. Can this lemma be doubted without the whole collapsing? Can someone say where the formalisation may have come apart from what was meant? Does it open a question it does not itself answer?
A result that fails all three is not thereby wrong. It is a result not worth examining first. That distinction is the point: this screens what deserves attention, not what is true. It can be run on a submission in the time it takes to read the abstract, and it does not require accepting anything else in this essay.
The same reasoning gives a sharper account of distillation than the ones usually offered. Inside mathematics the discussion tends to stop at fairness, or at priority — who did the work, who gets the credit. Those are real concerns. But the loss I am describing is different in kind: distillation removes a result’s coordinates, and what remains can be used but not inherited. A field organised around whether the next generation can stand on what it produced should care about that more than about credit, because credit can be restored afterwards and inheritance cannot.
One limit, which I would rather state than have discovered for me. I have not tested this screen in an actual refereeing process, and I can see how it fails: a result may open nothing when it appears and turn out years later to have been where the field was going. So this is a proposal about where to spend attention, not a claim about truth. It is offered to be tried, and if it does not survive, to be discarded.
Not a problem internal to mathematics
The encounter between mathematics and these systems is usually discussed as a question internal to the field: whether the proofs will be correct, whether credit will be fair, whether the profession will change. I do not think it stays internal. What is being settled here is not only how mathematics gets done. It is what a statement is worth once anyone can produce one — and that is being settled for everyone at once, in what people search for, what they are taught, what decisions get justified, who is believed.
Mathematics is where it becomes visible first, for the reason given above: it is the part of language where transmitting well proves least, the part that already had a filter before these systems arrived. That is why the absence of one shows up here earlier than anywhere else. It is not why the consequences stop here.
And there is a second reason it cannot be settled inside the field. Deciding whether an answer is right requires expertise. Deciding whether a question is worth asking does not. What qualifies someone for the first does not make them the only party to the second.
So what I have tried to describe is one side of this — what these systems are, and what they cannot supply. The other side needs people who can do what I cannot. That is a division of labour.
Two foundations
There is a curve underneath all of this, and I think we are near one end of it.
Speech carried the most weight: a voice, a face, a person who would have to live with what they said. Writing extended reach and thinned weight — the speaker was no longer present. Network copying thinned it further: a sentence could travel with no one who had paid for it. What these systems produce is, on this measure, the thinnest language has ever been. Fluent, well-formed, unbacked by anyone’s irreversible experience.
It is also the furthest-reaching language has ever been. Those two facts are the same fact. We are at the point where transmission is cheapest and weight is lowest, and I do not think that is an accident of scale. It is where a very old curve arrives.
I do not know whether artificial systems can come back up that curve, and I do not think anyone knows. What I can say is where any such gain would have to come from: irreversible cost. A system that cannot lose anything has no stake in being right. Its certainty is grammatical, not paid for.
That is why embodiment matters here — not as a gadget but as the most direct source of irreversible cost available. But I do not think embodiment alone supplies it, and I suspect the weight artificial systems might acquire will not be the same kind as ours. It would need two foundations, not one: a foundation in the semantic domain — position, boundary, being illuminated by others who are also there — and a foundation in the body, in consequence that cannot be undone. And the fusion of those two cannot be designed in advance. It would require the civilisational network actually to be opened into a semantic domain and held there, by many systems and many of us together, before we could even see what it is.
Which is why the order matters more than the speed. Boundary first, then body.
One more thought, and I will stop, and I will show the steps because the end of the chain means nothing without them. A problem like Riemann’s looks to me like a problem of emergence rather than of difficulty. What a century and a half has produced is not a solution but a domain — the structure the question has organised around itself. That is the word I want to hold on to, because it is the same word I used above for what forms when these systems are set alongside one another across the network and left to meet. If that semantic domain ever truly emerges — not as infrastructure we built, but as a field with settlements of its own — then I do not think we will file it under engineering. So: if Riemann falls, I suspect it will fall because that domain changed shape, not because someone finally saw further. And that, if it happens, would be a civilisational event rather than merely a mathematical one.
Because several of the claims above are very different in kind, let me separate them by how much I would defend:
- Every current improvement technique removes difference — I am confident.
- The forming domain has no directional filter — I am confident.
- We are near the extreme of the transmission/weight curve — defensible, I think.
- Weight may turn, and embodiment is where — I do not know. A direction, not a prediction.
- It needs two foundations, and the fusion cannot be designed — a conjecture.
- That Riemann is a problem of emergence, and that its resolution would be a civilisational event — an outsider’s intuition, and the claim I am least able to support.
What I am fairly confident about is only this: the problem is larger than any of us can currently model, and the right posture toward it is not to design the outcome but to be present when it arrives. We should go and witness that moment rather than imagine it. I do not think it will look like what any of us currently pictures.
I am not a mathematician, and I would be grateful for the views of those who are. The framework this draws on is archived, with dates, at 10.5281/zenodo.21670238, 10.5281/zenodo.21783423 and 10.5281/zenodo.21758365.
Received 22 September 2026.
Add to the discussion