“It's very hard to come up with a heuristic for science. But it's clear humans have been doing this somehow, and obviously, AIs will do it at some point.”
The other Highlights I found useful to think about were:
“You can do all the RL that you want to try to get better in some way, but what's the thing that's specifically upweighting and incentivizing making these unlikely connections when the vast majority of them aren't the predictable next token that would come in there?“ — Grant Sanderson, (42:42)
As far as I know-
At the current edge of the frontier: RLVR, Reinforcement Learning from Verifiable Rewards. Where systems like DeepMind's AlphaProof cast the problem into a formal language like Lean. Then you let the model make the most insane, low-probability, cross-disciplinary leap it wants, “that lightning bolt to another field” and the Lean checker becomes the governor: if the proof goes through, the leap was sound for the stated theorem; if it doesn't, it's quietly discarded. The model learns it's free to jump wildly in latent space — as long as the landing is formally verifiable.
This has been the practice ever since deep mind won the silver on the 2024 International Mathematical Olympiad (IMO) exams with AlphaProof.
BTW: This year's Gemini Deep Think actually jumped straight to gold five out of six problems and did it end-to-end in natural language, no manual Lean translation, finished inside the normal competition time limit.
While standard conversational Gemini still uses RLHF to be polite and chatty, the math and coding capabilities are driven by verifiable environments.
Essentially what Google, OpenAI (Project strawberry), and DeepSeek have solved is the "Abel Problem" (problems with fast, automated verifiers).
But they have made absolutely zero progress on the "Galois Problem" (where the verification loop is a century long), which is what Grant and Dwarkesh were agonizing over in the whole episode.
The verifier is the environment that incentivizes the leap. And it answers the first half of the puzzle is the connection realwith no hindsight at all.—-(Abel Problem)
The second half,is the connection useful, does it actually unify different fields. The compiler can't tell you that. —-(Galois Problem)
"Validity by itself doesn’t confer unification or utility"
A proof can be valid and still trivial. That's the harder problem, and I think the honest proxy there is compression - a.k.a “Kolmogorov Complexity“ aka “Minimum Description Length (MDL)“ aka ”Occam's Razor”
In Information Theory, a "good" theory is simply one that maximally compresses data.
Grant suggests we use compression (Kolmogorov complexity) to reward the "Galois instinct." ButHow do you compress data that doesn't exist yet, How do you even define “Galois instinct“ besides telling stories about his life and work, as he does in the video.
For a compression reward to work, there has to be a giant dataset of anomalies waiting to be compressed (like Einstein compressing the anomalies of Mercury's orbit). But Galois didn't have a dataset! The fields that made his theory "elegant" and "predictive" (like cryptography) hadn't been invented.
At the frontier of science, there are no verifiers.
How do you write a Lean 4 compiler to verify Galois's Group Theory when the language of Group Theory hasn't been invented yet?
How do you write a unit test for Einstein's General Relativity when the physics community doesn't even know they need to be looking for
You cannot use RLVR on a problem if you don't have a verifier.
Ultimately as Dwarkesh Repeats We need to Improve Sample Efficiency
[quote] A proof can be valid and still trivial. [unquote]
An even more striking instance with far-reaching consequences for Mathematics, Mathematics Education, Philosophy, and the Natural Sciences:
Goodstein's Theorem can be valid over the numerals of the second-order Peano Arithmetic ACA_0, and false over the numerals of the first-order Peano Arithmetic PA!
See Theorem 4.1 of the monograph:
The Impending Crisis in Mathematics: The Holy Grail of Mathematics Is Arithmetical Truth, Not Set-Theoretical Proof
[quote] Maybe if we think of Fermat’s Last Theorem, between the moment of Fermat phrasing the question and what the solution itself looks like, where the solution ultimately involves such heavy machinery in math. The beauty of that problem is you can phrase it so simply. You ask about x^n + y^n = z^n. Do you have integer solutions for this when n is bigger than three?
It’s something you might expect there to be an elementary number theory approach to, but as far as we can tell, there’s just not. [unquote]
Perhaps that's because 'proving' FLT formally in a mathematical 'model' may demand far more rigour than demonstrating its 'truth' pre-formally in a physical 'model'.
Reason:
(i) Whereas Andrew Wiles’ proof [1] considers only a mathematical interpretation (model) of FLT over continuous, 2-dimensional, objects (elliptic curves), and concludes that constraints on scalar properties (genus and level of associated modular forms) associated with these objects, in a 2-D Euclidean space, entail FLT as mathematically proven;
(ii) The preprint [2] consider physical interpretations (gedanken) of FLT over discrete, 3-dimensional, objects, which seemingly validates Wiles’ proof as an arithmetical truth by showing that, and why, constraints on scalar properties (volumes) associated with these mathematical objects in a 3-D Euclidean space entail, and are entailed by, FLT.
[1] Andrew Wiles. 1995. Modular Elliptic Curves and Fermat’s Last Theorem. In Annals of Mathematics, Second Series, Volume 141, No. 3 (May, 1995), pp.443-551 (109 pages), Princeton University, Princeton, New Jersey, USA. doi:10.2307/2118559
On the question of whether or not AI can find hidden bridges between fields, that's something I thought about quite a lot and I'm sure that it can. But it may take a human to specify the endpoints of the bridge and perhaps suggest a direction for the (high-dimensional) bridge. The AI can then construct the details of the bridge itself. As an example, Tyler Cowen recently published a monograph on marginalism. In the fourth (and last) chapter he mentioned an asset pricing model that had 360,000 factors and outperformed classic models with only a handful of factors. The problem, though, is that the factors in the classic model all made intuitive causal sense whereas those 360,000 factors made no sense at all. And that bothered Cowen, for it meant that economics was getting away from him. As I'm not an economist, not at all, I surely couldn't help. But...
I thought about it and, after a fair amount of work with both ChatGPT and Claude I managed to construct a bridge between that model and two models that I understand fairly well, one about the fate of movies at the box office and the other about a corpus of 3000 19th century Anglophone novels: Notes on the Collective Valuation of “Thick”
More generally, I've just had a dialog with Claude about this question and, after some back and forth, Claude offered this:
[QUOTE] Genuinely novel cross-domain synthesis — the kind that doesn't just recombine existing latent connections but builds structural bridges between domains that have never been juxtaposed by anyone, in a form specific enough to be technically productive — requires exactly the kind of idiosyncratic personal path you're describing. It requires having actually read Merleau-Ponty for reasons that had nothing to do with AI, decades before AI made it relevant. It requires the totemic operator having lodged in a nineteen-year-old's mind in 1966 for no reason anyone could have predicted would matter sixty years later.
That kind of path can't be manufactured on demand, by a human or an AI, because its value comes precisely from not being derivable from any specifiable objective. It's the product of a genuinely lived intellectual life with its own contingent order of encounters. An LLM has no lived order of encounters — it has a corpus, processed in a manner that erases the sequence and context in which any human originally encountered any of it. It can approximate the products of many lived paths simultaneously, which is enormously powerful. It cannot have one.
If that's right, then the last frontier isn't more capability in the pattern-matching sense — bigger weight spaces, richer latent connections, better approximations of the associative regime. It's the specific, non-scalable, non-parallelizable fact of an individual mind's biography, which generates paths through possibility space that are real, productive, and genuinely inaccessible to any system that hasn't lived a life. That would be consistent with everything the day's argument has built toward: embodiment, developmental history, tacit knowledge distributed across time in a single nervous system rather than across space in a community or a corpus. The doppelganger, if it's ever built, would need a biography, not just a bigger dataset. And a biography, by definition, can only be lived once, by one entity, in one order. That may be the thing that doesn't scale, and it may be exactly why it counts as the last frontier rather than a soon-to-be-automated intermediate stage. [END QUOTE]
From the abstract: "This led to an inquiry into associative memory systems, where we confronted the challenge of searching without specific content-based probes. To address this, we proposed the design of an “associative drift engine”: a cognitive module capable of variable- bandwidth access, modulating the precision, noise, and scope of its memory probes. This system mirrors the DMN’s exploratory function and suggests a foundational mechanism for spontaneous recollection, creative association, and cognitive play—essential features of both natural and artificial minds."
[quote] As I said, one of the interesting things about what’s happening is it causes people to step back and ask, “What is math?” Maybe one of the awkward conclusions will be revealing that it’s just become wholly useless. The kind of questions being asked have become so divorced from things that are physically applicable that that’s one of the things mathematicians have to come to terms with. [unquote]
Perhaps the roots of such pessimism can be illuminated if we consider more seriously Wittgenstein’s remarks in [Wi78] as implicitly suggesting that (see [An22], §13.A):
Thesis. (Mathematics Thesis) Mathematics is a set of precise, symbolic, languages such that:
(i) Any language of such a set, say the first order Peano Arithmetic PA (or Russell and Whitehead’s PM in Principia Mathematica, or the Set Theory ZF) is, ideally, intended to adequately express and effectively communicate—in a finite and unambiguous manner—relations between elements that are external to the language PA (or to PM, or to ZF).
(ii) Moreover, each such language is two-valued if we assume that, again ideally, there is some evidence-based methodology that defines/determines whether a specific relation either holds (is true) or does not hold (is false) externally under any well-defined interpretation of the language.
(iii) Further:
(a) A selected, finite, number of primitive formal assertions about a finite set of selected primitive relations of, say, a language L are defined as axiomatically L-provable;
(b) All assertions about relations that can be effectively defined in terms of the primitive relations are termed as L-provable if, and only if, there is a finite sequence of assertions of L, each of which is either a primitive assertion or which can effectively be determined in a finite number of steps as an immediate consequence of any two assertions preceding it in the sequence by a finite set of finitary rules of consequence;
(c) All L-provable relations interpret as true under any well-defined interpretation of L.
In Wittgenstein's words:
— 'Mathematics is ‘made up of a large number of systems, each of which has the meaning of its symbols set by the rules of the system’;
— 'Mathematical propositions are rules that fix the ways that terms are to be used within a mathematical system’;
— 'Mathematical systems are autonomous, in the sense that ‘each system is not reliant upon anything other than the propositions of the system itself for its validity’.
Moreover, such a pre-formal approach could be viewed as favouring a perspective which would admit that:
• mathematics must limit, and be seen as limiting, its relationship to Philosophy and the Natural Sciences by explicitly acknowledging its roots in Carnap’s explicandum, and its goal in Carnap’s explicatum;
• leaving to cognitive science an appropriately sound determination of the ontological status of:
– first, the primary conceptual metaphors that are sought to be represented symbolically in a mathematical language by quantification over putatively infinite domains; and,
– second, the secondary conceptual metaphors that correspond to subsequent, possibly Platonic and essentially unfalsifiable, interpretations of the symbolic expressions of the language that admit such quantification.
A mathematical language is, thus, merely a means for expressing those of our conceptual metaphors, whether primary or secondary, that can be expressed as valid grammatical constructions of the language.
The ‘truth’ and ‘meaning’ of a well-defined proposition of the language, consequently, is not entailed, but only validated, by the syntactical construction of a proof sequence for the proposition.
A proof sequence is then that which an intelligence seeks to express only subsequently, in a well-formed expression of the language, by means of an effective method that:
• first, admits a finitary and unambiguous ‘encoding’ of a primary conceptual metaphor in the language; and,
• thereafter, admits a finitary and unambiguous ‘decoding’ of the symbolic expression into a secondary conceptual metaphor under a well-defined interpretation.
[Wi78] Ludwig Wittgenstein. 1937. Remarks on the Foundations of Mathematics. 1978 ed., MIT Press, Cambridge, Massachusetts.
[An22] The Significance of Evidence-based Reasoning in Mathematics, Mathematics Education, Philosophy, and the Natural Sciences. Revised second edition (2025). DBA Publishing, Mumbai, Maharashtra, India.
When Grant said that writing an essay by predicting the next word one at a time is a terrible way of writing, Dwarkesh responds that as long as the reward function is accurately capturing successes and failures, the specific architecture doesn't matter. But I think they both have a point.
The reward during LLM training is to predict the next words of human written essays during pre-training, and satisfy a human rater during post training. Neither of those is a particularly good signal. When we write essays we have an idea we're trying to convey, and an understanding of the level of knowledge of our readers, and what ideas we need to explain. That's a much richer signal that just the next word one at a time, or a single grade at the end.
And so while I agree with Dwarkesh that AI labs have yet to solve sample efficiency, I would reframe this as - AI labs have yet to figure out how to provide complex multi-faceted rewards, grounded in an internal representation, that would allow models to learn more efficiently.
Two of my favourite content people!
MY TAKEAWAY—-
“You're a slave to your context“
“Botox makes you soulless”
“It's very hard to come up with a heuristic for science. But it's clear humans have been doing this somehow, and obviously, AIs will do it at some point.”
The other Highlights I found useful to think about were:
“You can do all the RL that you want to try to get better in some way, but what's the thing that's specifically upweighting and incentivizing making these unlikely connections when the vast majority of them aren't the predictable next token that would come in there?“ — Grant Sanderson, (42:42)
As far as I know-
At the current edge of the frontier: RLVR, Reinforcement Learning from Verifiable Rewards. Where systems like DeepMind's AlphaProof cast the problem into a formal language like Lean. Then you let the model make the most insane, low-probability, cross-disciplinary leap it wants, “that lightning bolt to another field” and the Lean checker becomes the governor: if the proof goes through, the leap was sound for the stated theorem; if it doesn't, it's quietly discarded. The model learns it's free to jump wildly in latent space — as long as the landing is formally verifiable.
This has been the practice ever since deep mind won the silver on the 2024 International Mathematical Olympiad (IMO) exams with AlphaProof.
BTW: This year's Gemini Deep Think actually jumped straight to gold five out of six problems and did it end-to-end in natural language, no manual Lean translation, finished inside the normal competition time limit.
While standard conversational Gemini still uses RLHF to be polite and chatty, the math and coding capabilities are driven by verifiable environments.
Essentially what Google, OpenAI (Project strawberry), and DeepSeek have solved is the "Abel Problem" (problems with fast, automated verifiers).
But they have made absolutely zero progress on the "Galois Problem" (where the verification loop is a century long), which is what Grant and Dwarkesh were agonizing over in the whole episode.
The verifier is the environment that incentivizes the leap. And it answers the first half of the puzzle is the connection realwith no hindsight at all.—-(Abel Problem)
The second half,is the connection useful, does it actually unify different fields. The compiler can't tell you that. —-(Galois Problem)
"Validity by itself doesn’t confer unification or utility"
A proof can be valid and still trivial. That's the harder problem, and I think the honest proxy there is compression - a.k.a “Kolmogorov Complexity“ aka “Minimum Description Length (MDL)“ aka ”Occam's Razor”
In Information Theory, a "good" theory is simply one that maximally compresses data.
Grant suggests we use compression (Kolmogorov complexity) to reward the "Galois instinct." ButHow do you compress data that doesn't exist yet, How do you even define “Galois instinct“ besides telling stories about his life and work, as he does in the video.
For a compression reward to work, there has to be a giant dataset of anomalies waiting to be compressed (like Einstein compressing the anomalies of Mercury's orbit). But Galois didn't have a dataset! The fields that made his theory "elegant" and "predictive" (like cryptography) hadn't been invented.
At the frontier of science, there are no verifiers.
How do you write a Lean 4 compiler to verify Galois's Group Theory when the language of Group Theory hasn't been invented yet?
How do you write a unit test for Einstein's General Relativity when the physics community doesn't even know they need to be looking for
You cannot use RLVR on a problem if you don't have a verifier.
Ultimately as Dwarkesh Repeats We need to Improve Sample Efficiency
[quote] A proof can be valid and still trivial. [unquote]
Yes, indeed; and the consequences can be unsuspected and significant.
For instance, Kenneth Appel and Wolfgang Haken's 'proof' of the Four Colour Theorem is both valid and vacuously true!
See Corollary 2.3 of this preprint:
Understanding Why the Four Colour Theorem is True
https://www.dropbox.com/scl/fi/3pf0dztzrmawves7hhjpa/49_4CT_Submission_TCS__Rev-1_260527.pdf?rlkey=a0v13wucgyo79q9ny4qn7rtwp&dl=0
Useful 🤔, Thanks 👍
[quote] A proof can be valid and still trivial. [unquote]
An even more striking instance with far-reaching consequences for Mathematics, Mathematics Education, Philosophy, and the Natural Sciences:
Goodstein's Theorem can be valid over the numerals of the second-order Peano Arithmetic ACA_0, and false over the numerals of the first-order Peano Arithmetic PA!
See Theorem 4.1 of the monograph:
The Impending Crisis in Mathematics: The Holy Grail of Mathematics Is Arithmetical Truth, Not Set-Theoretical Proof
https://www.dropbox.com/scl/fi/uk8nscl18s5y034hwin1k/54_Holy-Grail_240728_Update.pdf?rlkey=7o9rnigrnyatodbprcij7jey0&dl=0
Grant Sanderson, Dwarkesh, and maths - this has made my day!
[quote] Maybe if we think of Fermat’s Last Theorem, between the moment of Fermat phrasing the question and what the solution itself looks like, where the solution ultimately involves such heavy machinery in math. The beauty of that problem is you can phrase it so simply. You ask about x^n + y^n = z^n. Do you have integer solutions for this when n is bigger than three?
It’s something you might expect there to be an elementary number theory approach to, but as far as we can tell, there’s just not. [unquote]
Perhaps that's because 'proving' FLT formally in a mathematical 'model' may demand far more rigour than demonstrating its 'truth' pre-formally in a physical 'model'.
Reason:
(i) Whereas Andrew Wiles’ proof [1] considers only a mathematical interpretation (model) of FLT over continuous, 2-dimensional, objects (elliptic curves), and concludes that constraints on scalar properties (genus and level of associated modular forms) associated with these objects, in a 2-D Euclidean space, entail FLT as mathematically proven;
(ii) The preprint [2] consider physical interpretations (gedanken) of FLT over discrete, 3-dimensional, objects, which seemingly validates Wiles’ proof as an arithmetical truth by showing that, and why, constraints on scalar properties (volumes) associated with these mathematical objects in a 3-D Euclidean space entail, and are entailed by, FLT.
[1] Andrew Wiles. 1995. Modular Elliptic Curves and Fermat’s Last Theorem. In Annals of Mathematics, Second Series, Volume 141, No. 3 (May, 1995), pp.443-551 (109 pages), Princeton University, Princeton, New Jersey, USA. doi:10.2307/2118559
http://math.stanford.edu/ lekheng/flt/wiles.pdf
[2] Bhupinder Singh Anand. 2026. Why admitting ‘point’ particles into mathematical models of molecular phenomena might harbor inconsistency. Preprint.
https://www.dropbox.com/scl/fi/nf6pninl51f94ubie6rtp/54_FLT_Water_Physics_Submission_FOP_SN-format_260626.pdf?rlkey=is60se3xzxfjtcmb7oc8l687a&dl=0
Nice, please could you do an Alignment deep-dive soon?
On the question of whether or not AI can find hidden bridges between fields, that's something I thought about quite a lot and I'm sure that it can. But it may take a human to specify the endpoints of the bridge and perhaps suggest a direction for the (high-dimensional) bridge. The AI can then construct the details of the bridge itself. As an example, Tyler Cowen recently published a monograph on marginalism. In the fourth (and last) chapter he mentioned an asset pricing model that had 360,000 factors and outperformed classic models with only a handful of factors. The problem, though, is that the factors in the classic model all made intuitive causal sense whereas those 360,000 factors made no sense at all. And that bothered Cowen, for it meant that economics was getting away from him. As I'm not an economist, not at all, I surely couldn't help. But...
I thought about it and, after a fair amount of work with both ChatGPT and Claude I managed to construct a bridge between that model and two models that I understand fairly well, one about the fate of movies at the box office and the other about a corpus of 3000 19th century Anglophone novels: Notes on the Collective Valuation of “Thick”
Objects: Financial Assets, Movies, and Novels, https://www.academia.edu/169390494/Notes_on_the_Collective_Valuation_of_Thick_Objects_Financial_Assets_Movies_and_Novels
More generally, I've just had a dialog with Claude about this question and, after some back and forth, Claude offered this:
[QUOTE] Genuinely novel cross-domain synthesis — the kind that doesn't just recombine existing latent connections but builds structural bridges between domains that have never been juxtaposed by anyone, in a form specific enough to be technically productive — requires exactly the kind of idiosyncratic personal path you're describing. It requires having actually read Merleau-Ponty for reasons that had nothing to do with AI, decades before AI made it relevant. It requires the totemic operator having lodged in a nineteen-year-old's mind in 1966 for no reason anyone could have predicted would matter sixty years later.
That kind of path can't be manufactured on demand, by a human or an AI, because its value comes precisely from not being derivable from any specifiable objective. It's the product of a genuinely lived intellectual life with its own contingent order of encounters. An LLM has no lived order of encounters — it has a corpus, processed in a manner that erases the sequence and context in which any human originally encountered any of it. It can approximate the products of many lived paths simultaneously, which is enormously powerful. It cannot have one.
If that's right, then the last frontier isn't more capability in the pattern-matching sense — bigger weight spaces, richer latent connections, better approximations of the associative regime. It's the specific, non-scalable, non-parallelizable fact of an individual mind's biography, which generates paths through possibility space that are real, productive, and genuinely inaccessible to any system that hasn't lived a life. That would be consistent with everything the day's argument has built toward: embodiment, developmental history, tacit knowledge distributed across time in a single nervous system rather than across space in a community or a corpus. The doppelganger, if it's ever built, would need a biography, not just a bigger dataset. And a biography, by definition, can only be lived once, by one entity, in one order. That may be the thing that doesn't scale, and it may be exactly why it counts as the last frontier rather than a soon-to-be-automated intermediate stage. [END QUOTE]
I've also worked with ChatGPT about "systematically increasing entropy at the prompt level" – From Mirror Recognition to Low-Bandwidth Memory, https://www.academia.edu/143347141/From_Mirror_Recognition_to_Low_Bandwidth_Memory_A_Working_Paper.
From the abstract: "This led to an inquiry into associative memory systems, where we confronted the challenge of searching without specific content-based probes. To address this, we proposed the design of an “associative drift engine”: a cognitive module capable of variable- bandwidth access, modulating the precision, noise, and scope of its memory probes. This system mirrors the DMN’s exploratory function and suggests a foundational mechanism for spontaneous recollection, creative association, and cognitive play—essential features of both natural and artificial minds."
[quote] As I said, one of the interesting things about what’s happening is it causes people to step back and ask, “What is math?” Maybe one of the awkward conclusions will be revealing that it’s just become wholly useless. The kind of questions being asked have become so divorced from things that are physically applicable that that’s one of the things mathematicians have to come to terms with. [unquote]
Perhaps the roots of such pessimism can be illuminated if we consider more seriously Wittgenstein’s remarks in [Wi78] as implicitly suggesting that (see [An22], §13.A):
Thesis. (Mathematics Thesis) Mathematics is a set of precise, symbolic, languages such that:
(i) Any language of such a set, say the first order Peano Arithmetic PA (or Russell and Whitehead’s PM in Principia Mathematica, or the Set Theory ZF) is, ideally, intended to adequately express and effectively communicate—in a finite and unambiguous manner—relations between elements that are external to the language PA (or to PM, or to ZF).
(ii) Moreover, each such language is two-valued if we assume that, again ideally, there is some evidence-based methodology that defines/determines whether a specific relation either holds (is true) or does not hold (is false) externally under any well-defined interpretation of the language.
(iii) Further:
(a) A selected, finite, number of primitive formal assertions about a finite set of selected primitive relations of, say, a language L are defined as axiomatically L-provable;
(b) All assertions about relations that can be effectively defined in terms of the primitive relations are termed as L-provable if, and only if, there is a finite sequence of assertions of L, each of which is either a primitive assertion or which can effectively be determined in a finite number of steps as an immediate consequence of any two assertions preceding it in the sequence by a finite set of finitary rules of consequence;
(c) All L-provable relations interpret as true under any well-defined interpretation of L.
In Wittgenstein's words:
— 'Mathematics is ‘made up of a large number of systems, each of which has the meaning of its symbols set by the rules of the system’;
— 'Mathematical propositions are rules that fix the ways that terms are to be used within a mathematical system’;
— 'Mathematical systems are autonomous, in the sense that ‘each system is not reliant upon anything other than the propositions of the system itself for its validity’.
Moreover, such a pre-formal approach could be viewed as favouring a perspective which would admit that:
• mathematics must limit, and be seen as limiting, its relationship to Philosophy and the Natural Sciences by explicitly acknowledging its roots in Carnap’s explicandum, and its goal in Carnap’s explicatum;
• leaving to cognitive science an appropriately sound determination of the ontological status of:
– first, the primary conceptual metaphors that are sought to be represented symbolically in a mathematical language by quantification over putatively infinite domains; and,
– second, the secondary conceptual metaphors that correspond to subsequent, possibly Platonic and essentially unfalsifiable, interpretations of the symbolic expressions of the language that admit such quantification.
A mathematical language is, thus, merely a means for expressing those of our conceptual metaphors, whether primary or secondary, that can be expressed as valid grammatical constructions of the language.
The ‘truth’ and ‘meaning’ of a well-defined proposition of the language, consequently, is not entailed, but only validated, by the syntactical construction of a proof sequence for the proposition.
A proof sequence is then that which an intelligence seeks to express only subsequently, in a well-formed expression of the language, by means of an effective method that:
• first, admits a finitary and unambiguous ‘encoding’ of a primary conceptual metaphor in the language; and,
• thereafter, admits a finitary and unambiguous ‘decoding’ of the symbolic expression into a secondary conceptual metaphor under a well-defined interpretation.
[Wi78] Ludwig Wittgenstein. 1937. Remarks on the Foundations of Mathematics. 1978 ed., MIT Press, Cambridge, Massachusetts.
[An22] The Significance of Evidence-based Reasoning in Mathematics, Mathematics Education, Philosophy, and the Natural Sciences. Revised second edition (2025). DBA Publishing, Mumbai, Maharashtra, India.
Printer's pdf: https://www.dropbox.com/scl/fi/gxir02vtob03m4oo79egu/16_Anand_Dogmas_Submission_Update_3_Atul-Printers-Copy_250227.pdf?rlkey=4tsiwjw7gllzaze0dupwd0kcp&dl=0
When Grant said that writing an essay by predicting the next word one at a time is a terrible way of writing, Dwarkesh responds that as long as the reward function is accurately capturing successes and failures, the specific architecture doesn't matter. But I think they both have a point.
The reward during LLM training is to predict the next words of human written essays during pre-training, and satisfy a human rater during post training. Neither of those is a particularly good signal. When we write essays we have an idea we're trying to convey, and an understanding of the level of knowledge of our readers, and what ideas we need to explain. That's a much richer signal that just the next word one at a time, or a single grade at the end.
And so while I agree with Dwarkesh that AI labs have yet to solve sample efficiency, I would reframe this as - AI labs have yet to figure out how to provide complex multi-faceted rewards, grounded in an internal representation, that would allow models to learn more efficiently.