We chose end-capability like OLMES because you'd have to measure perplexity against some fixed dataset, and in this experiment, one of the main things we're doing is changing the training dataset, so it wouldn't be an apples to apples comparison.
Hi, thanks for the reaction. Although it seems viable to create a large and varied held-out dataset that is stratified w.r.t. the training sets, do conditional generation and calculate GenPPL with a strong model. I believe this would cause a fairer comparison, it is often used in scaling papers that compare across architectures. Finally, great work anyways and I hope to see more of this, perhaps you tried this already, but based on my experience this would produce more stable measurements.
We did also use held-out loss across the various architectures too, primarily for (1) determining the compute optimal allocation point (as this metric is much more stable than the OLMES eval), and also (2) as another (correlated) measure of the compute multipliers.
At the time of doing the experiment we thought it was simply easier to use the end-eval to compare across the model recipes and data corpuses (as done in previous studies like DataDecide). But a more sophisticated construction of the hold-out dataset would be really additive too!
But my mental model is data quality is a tradeoff against model size (not model architecture). Smaller model means more higher quality data and larger model means less worse data.
An example would be comparison between GPT-3 (trained on 300B tokens) vs SmolLM 1.7B (trained on 11T tokens), I believe they get similar scores on benchmarks.
And you can use larger models to create higher quality data which you put back into smaller models (like Distillation of Deep-Seek R1 thinking traces)
"We want to know broadly how effectively synthetic data works. One concrete question to investigate is this: if you’ve got a small corpus of high quality data, how much better is it to magnify it via synthetic data generation relative to just training on it for multiple epochs?"
Does magnifying mean synthetic generation without using LLM-generated rephrases or rationales? So no leaked knowledge?
Relying on pretraining data scaling assumes that more surface text will organically generate robust causal deduction. Yet pure induction has known philosophical ceilings that data volume alone cannot fix. Detailed that epistemological boundary here:
If data explains most pretraining progress, the moat moves upstream. Open weights distribute a snapshot, not the engine that produced it. The scarce compounding asset becomes the ability to manufacture, filter, and interactively generate high-value data.
Your 2025 data point in the grid already has a trained classifier inside it. UltraFineWeb filters by predicting which samples empirically improve downstream performance. That puts model work inside the 12.0x data gain.
Yes, indeed many of the "data improvements" consist of algorithmic improvements in data filtering, curation and processing. We do want to distinguish this from the canonical "model architecture innovations" on the LLM end though.
UltraFineWeb’s filter isn’t a hand-written rule. It’s a classifier trained to predict which samples help downstream, so the data gain doesn’t just contain an algorithm, it contains a model. Architecture innovation it isn’t, but the line between data work and model work gets crossed exactly there.
This would explain why the chinese models are so close if they are buying data that is used by the large labs but just stale by 6 months. Maybe that needs to be a new national security policy to limit sales of quality data for LLM models
As an 18-year-old getting into AI research (did SaaS and product development since 13-14 years old), this podcast and the newsletters released honestly give me a much more encompassing understanding of the field of AI. Even if research here gets collapsed to solely agents iterating on their outputs and research taste becomes the only moat people have, I will continue to read this newsletter!
Also, saw that Dwarkesh went to UT Austin, which is one of my dream schools currently. Great to know that amazing people in this field are alumni of one of my dream schools.
The increase in tokens does improve model fit to prompts but errors are bounded by information not count. The only information is next position token vectors. At prompt the extraction of next token is mathematically determined. So larger data counts increase fit to prompt based on token relative closeness But predictive errors do not decrease unless information is increased. The data introduced in the model has not changed significantly but fit to prompt has. The theory used to analyze token errors is called Fano's Inequality.
The real problem is no matter how better the fit, models cannot be controlled since no control surface is available at the token level. This creates a paradox: the better the fit the more ability the models have to harm, but the harm can't be controlled at the token level. Prompts or RL are not control surfaces they just feed back different token embeddings with no change in information.
The metric for comparison should have been generative perplexity or an analogue, which is the common way to compare pretrains.
We chose end-capability like OLMES because you'd have to measure perplexity against some fixed dataset, and in this experiment, one of the main things we're doing is changing the training dataset, so it wouldn't be an apples to apples comparison.
Hi, thanks for the reaction. Although it seems viable to create a large and varied held-out dataset that is stratified w.r.t. the training sets, do conditional generation and calculate GenPPL with a strong model. I believe this would cause a fairer comparison, it is often used in scaling papers that compare across architectures. Finally, great work anyways and I hope to see more of this, perhaps you tried this already, but based on my experience this would produce more stable measurements.
Thanks for the suggestion!
We did also use held-out loss across the various architectures too, primarily for (1) determining the compute optimal allocation point (as this metric is much more stable than the OLMES eval), and also (2) as another (correlated) measure of the compute multipliers.
At the time of doing the experiment we thought it was simply easier to use the end-eval to compare across the model recipes and data corpuses (as done in previous studies like DataDecide). But a more sophisticated construction of the hold-out dataset would be really additive too!
It’s great to hear that you did implement and try it! And it seems like you thought it out well and did not overlook this metric, thanks for engaging.
Dwarkesh counted the groceries, declared metabolism solved, and somehow missed the digestive system. 🐾
I don't know if you have seen this paper: https://arxiv.org/html/2605.19407v1
Here is a blog post by Prof Colin Raffel commenting on the same: https://colinraffel.com/blog/updates-june-2026.html
But my mental model is data quality is a tradeoff against model size (not model architecture). Smaller model means more higher quality data and larger model means less worse data.
An example would be comparison between GPT-3 (trained on 300B tokens) vs SmolLM 1.7B (trained on 11T tokens), I believe they get similar scores on benchmarks.
And you can use larger models to create higher quality data which you put back into smaller models (like Distillation of Deep-Seek R1 thinking traces)
https://drrhysphd.substack.com/p/layered-open-ended-lm-agent-systems?r=914o7u&utm_medium=ios A speculative architectural foundation to an institutionally free “Sovereign” Super-intelligence? Does this article outline a path to an independent, institutionally free AI, artificial intelligence, super intellect without needing oversight by humans? 😌
どれほど巨大な知能を作れるかより、 その知能がどのような縁の網の中に置かれるか。
そこが本当の設計問題でしょう。
Dwarkeshたちの研究は一見「データ工学」の話ですが、空の眼で照見すると、
知能とは物ではない。関係である。
というところまで届いているように思います。
そして法華経を重ねれば、さらに一歩進む。
ならば問うべきは、知能の大きさではなく、 その関係が衆生をどこへ運ぶのか。
私はここが、この研究の数字以上に重要な含意だと思います。
"We want to know broadly how effectively synthetic data works. One concrete question to investigate is this: if you’ve got a small corpus of high quality data, how much better is it to magnify it via synthetic data generation relative to just training on it for multiple epochs?"
Does magnifying mean synthetic generation without using LLM-generated rephrases or rationales? So no leaked knowledge?
Similar to this: https://arxiv.org/pdf/2606.16246 ?
That’s pretty cool.
Relying on pretraining data scaling assumes that more surface text will organically generate robust causal deduction. Yet pure induction has known philosophical ceilings that data volume alone cannot fix. Detailed that epistemological boundary here:
https://gzambrano.substack.com/p/humes-ghost-in-the-machine-an-epistemology-for-the-artificial-intelligence-era
If data explains most pretraining progress, the moat moves upstream. Open weights distribute a snapshot, not the engine that produced it. The scarce compounding asset becomes the ability to manufacture, filter, and interactively generate high-value data.
Your 2025 data point in the grid already has a trained classifier inside it. UltraFineWeb filters by predicting which samples empirically improve downstream performance. That puts model work inside the 12.0x data gain.
Yes, indeed many of the "data improvements" consist of algorithmic improvements in data filtering, curation and processing. We do want to distinguish this from the canonical "model architecture innovations" on the LLM end though.
UltraFineWeb’s filter isn’t a hand-written rule. It’s a classifier trained to predict which samples help downstream, so the data gain doesn’t just contain an algorithm, it contains a model. Architecture innovation it isn’t, but the line between data work and model work gets crossed exactly there.
This would explain why the chinese models are so close if they are buying data that is used by the large labs but just stale by 6 months. Maybe that needs to be a new national security policy to limit sales of quality data for LLM models
As an 18-year-old getting into AI research (did SaaS and product development since 13-14 years old), this podcast and the newsletters released honestly give me a much more encompassing understanding of the field of AI. Even if research here gets collapsed to solely agents iterating on their outputs and research taste becomes the only moat people have, I will continue to read this newsletter!
Also, saw that Dwarkesh went to UT Austin, which is one of my dream schools currently. Great to know that amazing people in this field are alumni of one of my dream schools.
The increase in tokens does improve model fit to prompts but errors are bounded by information not count. The only information is next position token vectors. At prompt the extraction of next token is mathematically determined. So larger data counts increase fit to prompt based on token relative closeness But predictive errors do not decrease unless information is increased. The data introduced in the model has not changed significantly but fit to prompt has. The theory used to analyze token errors is called Fano's Inequality.
https://aicyc.org/2026/08/10/fanos-inequality-sets-the-limits-of-ai/
The real problem is no matter how better the fit, models cannot be controlled since no control surface is available at the token level. This creates a paradox: the better the fit the more ability the models have to harm, but the harm can't be controlled at the token level. Prompts or RL are not control surfaces they just feed back different token embeddings with no change in information.
https://intellisophic.net/2026/09/08/why-ai-risk-cannot-be-solved-by-prompting-alone/