This learning/steering model with the framing of lower-level more deterministic parts of the brain as python-code-like sets up some interesting metaphors: Seen through that lens, we have a flexible higher level (learning system = reasoning AI) that has access to tools with predictable functionality (steering system = MCPs, skills, etc) that it accesses via abstractions and APIs.
Interesting not so much for what it says about brains or AGI safety, but even just for tool design. Following the metaphor (very loosely) in one direction, the existing chat memory update tool is maybe like the hippocampus in its task, but in the other maybe the cerebellum and reflexes suggest that for some jobs, an external timing tool might be handy.
Interesting episode. Thank you for an informative discussion.
I come from a background in psychiatry, neuroscience, and psychoanalysis. Many of the ideas discussed, especially those related to what is referenced in Steve Byrnes work, have been worked on for sometime in the field of neuropsychoanalysis, albeit in the service of understanding how minds work, not AI. However given the emphasis on this talk and many others on understanding how to engineer more capable AI, I think there are useful parallels.
Without getting into too much detail, the work of Mark Solms in his book The Hidden Spring, offers a coherent model of how a mind works. He does so through integration of the work of Jaak Panksepp, whose mammalian research demonstrated intrinsic drives shared across all mammalian species based on homologous neural circuitry. These drives are compromised of distinct groups of nuclei and neural circuitry in the brain stem and midbrain e.g. the mesolimbic/mesocortical dopamine system, which he labels the SEEKING system. Panksepp delineates 7 emotional drives in all: SEEKING, RAGE, FEAR, LUST, CARE, PANIC/GRIEF, and PLAY. They are specifically of interest in understanding how human minds work, but also the fundamental drives of mammalian behavior. These 'drives' are the biological corollary to what Byrne describes in his steering system.
Solms integrates Panksepps affective neuroscience with Karl Friston's ideas related to the free energy principle and active inference. The result is one of the more coherent models of the mind that I know of, which I think comes closest to addressing the infamous hard problem of consciousness. As seemed to be implied in this talk, the function of the cortex, albeit critical for the accomplishments of humans, is not sufficient for generating intelligent behavior. There has been a cortical bias in much neuroscience research, and these crucial motivational systems have been neglected.
Don't take my word for it. Please read Solms book. It has greatly influenced my work as a clinician and how I think about minds.
This conversation is very enlightening. I really appreciate your infinite, informed, prepared and engaging curiosity in these interviews @Dwarkesh!
2 things; one procedural for the Dwarkesh team/podcast crew: this transcript (with Adam Marblestone) lacks the timestamp like you have on most of your prior shows outputs (i.e. thinking of the last Ilya interview, where the transcript was formatted like this;
"
Transcript
00:00:00 – Explaining model jaggedness
Ilya Sutskever 00:00:00
You know what’s crazy? That all of this is real.
Dwarkesh Patel 00:00:04
Meaning what?
Ilya Sutskever 00:00:05
Don’t you think so? All this AI stuff and all this Bay Area… that it’s happening. Isn’t it straight out of science fiction?
Dwarkesh Patel 00:00:14
Another thing that’s crazy is how normal the slow takeoff feels. The idea that we’d be investing 1% of GDP in AI, I feel like it would have felt like a bigger deal, whereas right now it just feels...
"
where as this one with Adam only has the timestamps per section. Will that be updated over time as you process that since it was just released yesterday?
2- more topical: I do wish that both Adam & Dwarkesh would have stayed with the Joseph Heinrich Cultural Evolutionary thread once Dwarkesh brought it up around the 00:42:42 – Model-based vs model-free RL in the brain section. That concept is much more ripe with opportunity to add value to the LLM current iteration of AI technology that is in market / mainstream right now and was given a nod...but anyways.
great content as usual. looking forward to the updated fully formatted transcript. I use them for content in another RAG / long form context window that I am accruing to track thought leadership and critical reflections on the current trajectory of the LLM AI era. This content is amongst the best available.
I find these conversations very interesting, as with the Sutton interview it seems like the point of AGI or possibly the pathway to it is to replicate the human brain or at the very least approximate its loss function.
This doesn’t seem to be the point of most of the laps who are instead looking for capabilities as a proxy for economic impact which provides sufficient incentive for mass adoption in lieu of human labour.
A second potential explanation is that there is a potential belief that humans max out near the top of the intelligence ability (think speed of light as the speed limit proxy for physics), this doesn’t seem to have much basis other than us being the smartest species on earth which is only a sample size of 1. It could be true but doesn’t seem to be likely that it would be. Therefor it’s unlikely that there must be asymptotic approaches in types of intelligence approaching human intelligence (i.e. pertaining, post-training, continual learning, etc).
Whilst there is a decline in the exponential of pre training on textual data it doesn’t seem to be reaching asymptotic levels. though that’s not to say scale is sufficient for AGI, just that the premise that there needs to be all this extra stuff which seems predicated on the idea that humans are close to the potential max of intelligence.
Whilst I’m not an AI researcher, and my albeit anecdotal data much of frontier AI seems at the level of an early career started albeit with a better ability to handle abstraction. If its ability to handle abstraction and raw knowledge goes up, the need for context will go down for the same results, such that context may be sufficient sans continual learning for early ai systems
https://swenldn.substack.com/p/nazi-camp-w14-west-kensington-live?utm_campaign=post-expanded-share&utm_medium=web
https://www.facebook.comhttps/?fbclid=IwY2xjawQP9SlleHRuA2FlbQIxMABzcnRjBmFwcF9pZBAyMjIwMzkxNzg4MjAwODkyAAEefb7H0QqYmqS7fHOlzeSA9qUl8O-O-h7b44QJCkW2U61cEbGA5g-4cvl7UsI_aem_503lEKUMuR0_R4zpDVtGkQ
Would be interesting if you could get people from XAI on, or look into that frontier lab as its probably one of the least covered
some Mungu theoretic notes on this podcast: https://open.substack.com/pub/systemics/p/llms?utm_campaign=post-expanded-share&utm_medium=web
This learning/steering model with the framing of lower-level more deterministic parts of the brain as python-code-like sets up some interesting metaphors: Seen through that lens, we have a flexible higher level (learning system = reasoning AI) that has access to tools with predictable functionality (steering system = MCPs, skills, etc) that it accesses via abstractions and APIs.
Interesting not so much for what it says about brains or AGI safety, but even just for tool design. Following the metaphor (very loosely) in one direction, the existing chat memory update tool is maybe like the hippocampus in its task, but in the other maybe the cerebellum and reflexes suggest that for some jobs, an external timing tool might be handy.
Interesting episode. Thank you for an informative discussion.
I come from a background in psychiatry, neuroscience, and psychoanalysis. Many of the ideas discussed, especially those related to what is referenced in Steve Byrnes work, have been worked on for sometime in the field of neuropsychoanalysis, albeit in the service of understanding how minds work, not AI. However given the emphasis on this talk and many others on understanding how to engineer more capable AI, I think there are useful parallels.
Without getting into too much detail, the work of Mark Solms in his book The Hidden Spring, offers a coherent model of how a mind works. He does so through integration of the work of Jaak Panksepp, whose mammalian research demonstrated intrinsic drives shared across all mammalian species based on homologous neural circuitry. These drives are compromised of distinct groups of nuclei and neural circuitry in the brain stem and midbrain e.g. the mesolimbic/mesocortical dopamine system, which he labels the SEEKING system. Panksepp delineates 7 emotional drives in all: SEEKING, RAGE, FEAR, LUST, CARE, PANIC/GRIEF, and PLAY. They are specifically of interest in understanding how human minds work, but also the fundamental drives of mammalian behavior. These 'drives' are the biological corollary to what Byrne describes in his steering system.
Solms integrates Panksepps affective neuroscience with Karl Friston's ideas related to the free energy principle and active inference. The result is one of the more coherent models of the mind that I know of, which I think comes closest to addressing the infamous hard problem of consciousness. As seemed to be implied in this talk, the function of the cortex, albeit critical for the accomplishments of humans, is not sufficient for generating intelligent behavior. There has been a cortical bias in much neuroscience research, and these crucial motivational systems have been neglected.
Don't take my word for it. Please read Solms book. It has greatly influenced my work as a clinician and how I think about minds.
This conversation is very enlightening. I really appreciate your infinite, informed, prepared and engaging curiosity in these interviews @Dwarkesh!
2 things; one procedural for the Dwarkesh team/podcast crew: this transcript (with Adam Marblestone) lacks the timestamp like you have on most of your prior shows outputs (i.e. thinking of the last Ilya interview, where the transcript was formatted like this;
"
Transcript
00:00:00 – Explaining model jaggedness
Ilya Sutskever 00:00:00
You know what’s crazy? That all of this is real.
Dwarkesh Patel 00:00:04
Meaning what?
Ilya Sutskever 00:00:05
Don’t you think so? All this AI stuff and all this Bay Area… that it’s happening. Isn’t it straight out of science fiction?
Dwarkesh Patel 00:00:14
Another thing that’s crazy is how normal the slow takeoff feels. The idea that we’d be investing 1% of GDP in AI, I feel like it would have felt like a bigger deal, whereas right now it just feels...
"
where as this one with Adam only has the timestamps per section. Will that be updated over time as you process that since it was just released yesterday?
2- more topical: I do wish that both Adam & Dwarkesh would have stayed with the Joseph Heinrich Cultural Evolutionary thread once Dwarkesh brought it up around the 00:42:42 – Model-based vs model-free RL in the brain section. That concept is much more ripe with opportunity to add value to the LLM current iteration of AI technology that is in market / mainstream right now and was given a nod...but anyways.
great content as usual. looking forward to the updated fully formatted transcript. I use them for content in another RAG / long form context window that I am accruing to track thought leadership and critical reflections on the current trajectory of the LLM AI era. This content is amongst the best available.
I find these conversations very interesting, as with the Sutton interview it seems like the point of AGI or possibly the pathway to it is to replicate the human brain or at the very least approximate its loss function.
This doesn’t seem to be the point of most of the laps who are instead looking for capabilities as a proxy for economic impact which provides sufficient incentive for mass adoption in lieu of human labour.
A second potential explanation is that there is a potential belief that humans max out near the top of the intelligence ability (think speed of light as the speed limit proxy for physics), this doesn’t seem to have much basis other than us being the smartest species on earth which is only a sample size of 1. It could be true but doesn’t seem to be likely that it would be. Therefor it’s unlikely that there must be asymptotic approaches in types of intelligence approaching human intelligence (i.e. pertaining, post-training, continual learning, etc).
Whilst there is a decline in the exponential of pre training on textual data it doesn’t seem to be reaching asymptotic levels. though that’s not to say scale is sufficient for AGI, just that the premise that there needs to be all this extra stuff which seems predicated on the idea that humans are close to the potential max of intelligence.
Whilst I’m not an AI researcher, and my albeit anecdotal data much of frontier AI seems at the level of an early career started albeit with a better ability to handle abstraction. If its ability to handle abstraction and raw knowledge goes up, the need for context will go down for the same results, such that context may be sufficient sans continual learning for early ai systems