the verification loop for theories can be on the order of decades and centuries, and even then we know today as the better theory can often actually make worse predictions
The decades-long verification loop is the load-bearing argument here. The cleaner version is that RLVR fails at science not because science is slow but because the reward function is what gets argued over in real research. Copernicus losing to Ptolemy on accuracy in 1543 is the same shape as a modern theory-selection problem: better theories often make worse short-run predictions, and any verifier trained on existing data would have penalized the right answer. RL needs a fixed objective; science is the discipline of editing the objective.
One way to sharpen the RLVR/science question is to ask what it would mean for an AI system to regard two scientific theories as “the same theory.”
A naive falsificationist picture — crisp experiments cleanly deciding between theories — would give us the ideal setting for closing the verification loop. But, as the post makes clear, fundamental science does not usually work that way. The same anomaly can be read as a missing planet, an unmodeled perturbation, a bad instrument, or a sign that the underlying theory has to change.
A framing I find useful is this: a scientific theory is an algorithm proposed by the scientist, while nature is treated as implementing an underlying algorithm of its own. An experiment compares the outputs of those two algorithms on suitably matched inputs. This does not require taking the algorithm as the final ontology; ontology can be read, provisionally, from the mechanisms the algorithm makes explicit. But computationally equivalent formulations may suggest different ontological pictures.
This matters because a “scientific revolution” is not just a change in verbal description or even in ontology. It is a change in the algorithmic representation of the world, relative to a computational and experimental budget. Some reformulations — Lagrangian versus Hamiltonian mechanics, for example — are extraordinarily valuable without being revolutions in this sense. Even the Ptolemy-to-Copernicus move looks different under this lens: not simply “the true ontology replacing the false one,” but a reorganization of the search space that made later compression by Kepler and Newton accessible to human minds with limited computational resources.
A sufficiently powerful agent might not need to traverse the historical path Copernicus → Kepler → Newton. Given enough compute and the right selection criteria over coordinate systems and dynamical descriptions, it might infer the Newtonian structure more directly. What appears to us as a sequence of conceptual breakthroughs may partly reflect the path forced on compute-limited humans.
If something like this is right, the RLVR problem in fundamental science is not only that experiments are slow, expensive, or ambiguous. It is that the verifier must decide when two theories count as equivalent. A natural criterion is resource-sensitive empirical equivalence: two theories are equivalent for a given agent if they yield the same observable outputs, within the relevant tolerance, on the relevant class of inputs, and do so with comparable computational and experimental resources.
But the word “comparable” is doing real work. It depends on the agent’s hardware, algorithms, precision requirements, available data, experimental technology, and scientific goals. What counts as a genuine breakthrough for humans may be merely a change of coordinates for a superintelligent agent with a different resource budget. The hard problem may therefore not be only closing the verification loop, but specifying the resource-relative notion of theoretical equivalence that the loop is supposed to reward.
Another example is evolution itself, evolutuion provided a lot of time for experiments with mutations but at last the devastating factors (genetic shift) such as comet decided that time of dinosaurs is over and mammals can progress. It is human ego at last that keeps them going to explore things that others are not agreeing with, in case of AI we have to make sure not all AI have the exact same ego.
AI itself is an example of this. Neural nets were largely ignored for years because they yielded worse results than other algorithms. A few obstinate people kept plugging away at them though, until they showed that scaling them up way past any prior expectations made them work. And then, boom.
But to say maybe science can't be done by AI because verification loops are too long, seems like overdoing it. What kind of science? Some of the most valuable science is biological, where experimental loops can be fairly fast. Especially if we manage to build more powerful molecular dynamics simulations so experiments can happen first in sim at high speed and then be 'replicated' in vitro.
I think maybe the bigger issue than RLVR bottlenecks is the other aspect you identified - a lot of the best scientific breakthroughs seem to come from combining or chasing obscure ideas, even though they aren't supported by the consensus of the era. This is something AI is very weak on. It doesn't passively meditate upon ideas and form unusual connections between them. Maybe it could, but today it doesn't. And all the models have very similar personalities. There isn't such a thing as a model that just gets hung up on some crazy idea and chases it to the bitter end. Again, maybe there could be, but today there isn't.
Certainly, some of my best ideas in computing have come about from combining several apparently unrelated areas of knowledge together.
The decades-long verification loop is the load-bearing argument here. The cleaner version is that RLVR fails at science not because science is slow but because the reward function is what gets argued over in real research. Copernicus losing to Ptolemy on accuracy in 1543 is the same shape as a modern theory-selection problem: better theories often make worse short-run predictions, and any verifier trained on existing data would have penalized the right answer. RL needs a fixed objective; science is the discipline of editing the objective.
Nice to feeeeel your way, huh?
One way to sharpen the RLVR/science question is to ask what it would mean for an AI system to regard two scientific theories as “the same theory.”
A naive falsificationist picture — crisp experiments cleanly deciding between theories — would give us the ideal setting for closing the verification loop. But, as the post makes clear, fundamental science does not usually work that way. The same anomaly can be read as a missing planet, an unmodeled perturbation, a bad instrument, or a sign that the underlying theory has to change.
A framing I find useful is this: a scientific theory is an algorithm proposed by the scientist, while nature is treated as implementing an underlying algorithm of its own. An experiment compares the outputs of those two algorithms on suitably matched inputs. This does not require taking the algorithm as the final ontology; ontology can be read, provisionally, from the mechanisms the algorithm makes explicit. But computationally equivalent formulations may suggest different ontological pictures.
This matters because a “scientific revolution” is not just a change in verbal description or even in ontology. It is a change in the algorithmic representation of the world, relative to a computational and experimental budget. Some reformulations — Lagrangian versus Hamiltonian mechanics, for example — are extraordinarily valuable without being revolutions in this sense. Even the Ptolemy-to-Copernicus move looks different under this lens: not simply “the true ontology replacing the false one,” but a reorganization of the search space that made later compression by Kepler and Newton accessible to human minds with limited computational resources.
A sufficiently powerful agent might not need to traverse the historical path Copernicus → Kepler → Newton. Given enough compute and the right selection criteria over coordinate systems and dynamical descriptions, it might infer the Newtonian structure more directly. What appears to us as a sequence of conceptual breakthroughs may partly reflect the path forced on compute-limited humans.
If something like this is right, the RLVR problem in fundamental science is not only that experiments are slow, expensive, or ambiguous. It is that the verifier must decide when two theories count as equivalent. A natural criterion is resource-sensitive empirical equivalence: two theories are equivalent for a given agent if they yield the same observable outputs, within the relevant tolerance, on the relevant class of inputs, and do so with comparable computational and experimental resources.
But the word “comparable” is doing real work. It depends on the agent’s hardware, algorithms, precision requirements, available data, experimental technology, and scientific goals. What counts as a genuine breakthrough for humans may be merely a change of coordinates for a superintelligent agent with a different resource budget. The hard problem may therefore not be only closing the verification loop, but specifying the resource-relative notion of theoretical equivalence that the loop is supposed to reward.
Is it known whether frontier labs are still relying purely on verifiable rewards without process supervision, or ancillary rewards?
Another example is evolution itself, evolutuion provided a lot of time for experiments with mutations but at last the devastating factors (genetic shift) such as comet decided that time of dinosaurs is over and mammals can progress. It is human ego at last that keeps them going to explore things that others are not agreeing with, in case of AI we have to make sure not all AI have the exact same ego.
AI itself is an example of this. Neural nets were largely ignored for years because they yielded worse results than other algorithms. A few obstinate people kept plugging away at them though, until they showed that scaling them up way past any prior expectations made them work. And then, boom.
But to say maybe science can't be done by AI because verification loops are too long, seems like overdoing it. What kind of science? Some of the most valuable science is biological, where experimental loops can be fairly fast. Especially if we manage to build more powerful molecular dynamics simulations so experiments can happen first in sim at high speed and then be 'replicated' in vitro.
I think maybe the bigger issue than RLVR bottlenecks is the other aspect you identified - a lot of the best scientific breakthroughs seem to come from combining or chasing obscure ideas, even though they aren't supported by the consensus of the era. This is something AI is very weak on. It doesn't passively meditate upon ideas and form unusual connections between them. Maybe it could, but today it doesn't. And all the models have very similar personalities. There isn't such a thing as a model that just gets hung up on some crazy idea and chases it to the bitter end. Again, maybe there could be, but today there isn't.
Certainly, some of my best ideas in computing have come about from combining several apparently unrelated areas of knowledge together.