I finally got around to reading "Toy Models of Superposition" (https://transformer-circuits.pub/2022/toy_model/index.html) mentioned previously on your podcast. It made me think about your oft-asked question of "Why can't an intelligence with all the world's knowledge make novel connections." I don't think this is the _whole_ answer, but I wonder whether superposition plays a part. Counterintuitively, the idea of having dense representations of unrelated ideas might make it harder for the model to reason about those overlapping ideas together, versus having cleanly separable ideas that can be distinctly represented. I.e., is it harder to draw connections between ideas A and B when A and B are superposed and downstream layers can't easily determine whether it's A, B, or both being represented, versus when A and B are not superposed and it's clear that _both_ A and B are present?
Very interesting! I've been trying to put my finger on something and I think this is part of it. A very good paper to twin with the superposition paper is this....
Scott & Daniel, absolutely love your trajectories.
I have been aspiring towsrds sn instance of it -- having my SpiceTradeAsia region to continue where we left off on our mindset graph in 1450. When the region was anongst the happiest,richest and most emergent in the world.
I want us to trace, confirm and reverse the factors that fell us.
Game? Game to team with great minds in Yogjakarta, Indonesia to juxtapose this applied on your model and iterate to specific agent and human do's?
When you do AMA, please include the evolution of American Health Care and the vertical integration of every portion of the HC pie, save PhRMA.
Docs, hospitals, pharmacy, insurance, even PBM’s now VS the only non-integrated one standing, and that is PhRMA.
Yea, US mostly does the research for the rest of the world, and it is exorbitant, but worth it. Much like we protect our northern and southern borders.
Our military expenditures are exorbitant, but also-worth it.
I haven’t clicked play faster in my life. Well done for getting Scott on the pod!!
Stoked for this, long-time SSC/ACX reader.
The intelligence explosion is upon us. FOOOOOOOOOM!!!! I've been jamming with Gemini 2.5 all week and I hate coding like no one else hates coding.
Imagine we don’t end factory farming because we’ve given ourselves moral licensing to prioritize digital minds
I finally got around to reading "Toy Models of Superposition" (https://transformer-circuits.pub/2022/toy_model/index.html) mentioned previously on your podcast. It made me think about your oft-asked question of "Why can't an intelligence with all the world's knowledge make novel connections." I don't think this is the _whole_ answer, but I wonder whether superposition plays a part. Counterintuitively, the idea of having dense representations of unrelated ideas might make it harder for the model to reason about those overlapping ideas together, versus having cleanly separable ideas that can be distinctly represented. I.e., is it harder to draw connections between ideas A and B when A and B are superposed and downstream layers can't easily determine whether it's A, B, or both being represented, versus when A and B are not superposed and it's clear that _both_ A and B are present?
Very interesting! I've been trying to put my finger on something and I think this is part of it. A very good paper to twin with the superposition paper is this....
https://arxiv.org/pdf/2505.11581
Re: stealing the weights, spies in companies: https://www.yahoo.com/news/thousands-north-korean-workers-infiltrated-110000417.html
Scott & Daniel, absolutely love your trajectories.
I have been aspiring towsrds sn instance of it -- having my SpiceTradeAsia region to continue where we left off on our mindset graph in 1450. When the region was anongst the happiest,richest and most emergent in the world.
I want us to trace, confirm and reverse the factors that fell us.
Game? Game to team with great minds in Yogjakarta, Indonesia to juxtapose this applied on your model and iterate to specific agent and human do's?
x.com/balapillai
Link to the piece Scott mentions by "L Rudolph L"?
When you do AMA, please include the evolution of American Health Care and the vertical integration of every portion of the HC pie, save PhRMA.
Docs, hospitals, pharmacy, insurance, even PBM’s now VS the only non-integrated one standing, and that is PhRMA.
Yea, US mostly does the research for the rest of the world, and it is exorbitant, but worth it. Much like we protect our northern and southern borders.
Our military expenditures are exorbitant, but also-worth it.