The business question here is basically “who is correctly priced for volatility?” A GPU is long volatility on AI workloads. You pay a huge generality tax, but you’re protected if tomorrow’s models need different precision, routing, kernels, memory access, batching behavior, etc. An ASIC is short volatility. You win if the workload stabilizes, and you get destroyed if the frontier moves before your tape-out pays back.
volta tensor cores weren't systolic arrays! There's heavy evidence they were tree-adders (https://arxiv.org/abs/1811.08309) && afaik only Blackwell TCs are obviously systolics by documentation
Every discussion about AI compute eventually hits a stubborn physical wall. 🧱 Arithmetic isn't expensive. Moving a bit three millimeters across silicon to reach an adder is where all your power and die area vanish. ⚡
When you build a standard ALU, seven-eighths of the circuit area gets eaten by multiplexers just trying to select which register talks to which gate. That's not a bug. It's the structural tax of general-purpose flexibility. Systolic arrays work not because matrix math is magical, but because they hold weights stationary in local registers and stream activations through rigid spatial pipelines. You trade dynamic routing freedom for wiring solvency. 📐
Software engineers often treat clock speed as an abstract dial. On the die, every gigahertz demands aggressive pipeline register insertion. Split your combinatorial logic in half to double the frequency, and your register storage doubles alongside it. Push it too far, and you're burning your entire power budget charging and discharging flip-flop capacitances instead of executing actual math. 🔌
The real battleground isn't FLOP counts. It's whether your memory architecture relies on probabilistic hardware caches or software-managed scratchpads. A CPU cache burns enormous area on branch predictors and tag checks to guess what you need next. A deterministic scratchpad lets the compiler schedule data movement down to the exact nanosecond, eliminating arbitration jitter entirely. 🎯
When we look at the next generation of custom silicon, the winning design won't be the one with the widest vector units. It'll be the architecture that treats spatial data movement as an absolute physical invariant. If your model topology stabilizes, why keep paying a 10x area penalty for dynamic routing hardware you never use? 🚀
The chalkboard format on this kind of material is the right tradeoff. Most chip coverage stays one layer up, at "TPU versus GPU" or "ASIC versus general-purpose," and skips the multiply-accumulate primitives that actually determine why the SKUs look different. The MatX framing is the interesting one, since the merchant-silicon thesis is a bet that hyperscaler in-house chips like TPU, Trainium, and Maia leave room for a third lane between Nvidia and the captive accelerators. Whether that lane exists at scale is the production question over the next two earnings cycles.
The third lane question usually gets framed as whether MatX can out-engineer Nvidia, but the merchant-silicon bet is really about TSMC allocation. When OpenAI's CPU demand alone has TSMC unable to meet 80% of orders and prices rising 50%, the binding constraint on a third lane isn't architecture, it's whether you can buy wafers behind Nvidia and three hyperscalers already holding capacity contracts.
The business question here is basically “who is correctly priced for volatility?” A GPU is long volatility on AI workloads. You pay a huge generality tax, but you’re protected if tomorrow’s models need different precision, routing, kernels, memory access, batching behavior, etc. An ASIC is short volatility. You win if the workload stabilizes, and you get destroyed if the frontier moves before your tape-out pays back.
volta tensor cores weren't systolic arrays! There's heavy evidence they were tree-adders (https://arxiv.org/abs/1811.08309) && afaik only Blackwell TCs are obviously systolics by documentation
Have the winners for the blog prize been chosen yet?
Every discussion about AI compute eventually hits a stubborn physical wall. 🧱 Arithmetic isn't expensive. Moving a bit three millimeters across silicon to reach an adder is where all your power and die area vanish. ⚡
When you build a standard ALU, seven-eighths of the circuit area gets eaten by multiplexers just trying to select which register talks to which gate. That's not a bug. It's the structural tax of general-purpose flexibility. Systolic arrays work not because matrix math is magical, but because they hold weights stationary in local registers and stream activations through rigid spatial pipelines. You trade dynamic routing freedom for wiring solvency. 📐
Software engineers often treat clock speed as an abstract dial. On the die, every gigahertz demands aggressive pipeline register insertion. Split your combinatorial logic in half to double the frequency, and your register storage doubles alongside it. Push it too far, and you're burning your entire power budget charging and discharging flip-flop capacitances instead of executing actual math. 🔌
The real battleground isn't FLOP counts. It's whether your memory architecture relies on probabilistic hardware caches or software-managed scratchpads. A CPU cache burns enormous area on branch predictors and tag checks to guess what you need next. A deterministic scratchpad lets the compiler schedule data movement down to the exact nanosecond, eliminating arbitration jitter entirely. 🎯
When we look at the next generation of custom silicon, the winning design won't be the one with the widest vector units. It'll be the architecture that treats spatial data movement as an absolute physical invariant. If your model topology stabilizes, why keep paying a 10x area penalty for dynamic routing hardware you never use? 🚀
(ง •̀_•́)ง
Didn't really like this.
I love that “informing…inspiring” is formed to replace a threat.
“You cannot meet the demands, so get in line!”
This is the “hedge and ditch” rule in action but without a law to back it.
How to own the Wild West – White American style (so utilising the help of bought and graded-humans)
The chalkboard format on this kind of material is the right tradeoff. Most chip coverage stays one layer up, at "TPU versus GPU" or "ASIC versus general-purpose," and skips the multiply-accumulate primitives that actually determine why the SKUs look different. The MatX framing is the interesting one, since the merchant-silicon thesis is a bet that hyperscaler in-house chips like TPU, Trainium, and Maia leave room for a third lane between Nvidia and the captive accelerators. Whether that lane exists at scale is the production question over the next two earnings cycles.
The third lane question usually gets framed as whether MatX can out-engineer Nvidia, but the merchant-silicon bet is really about TSMC allocation. When OpenAI's CPU demand alone has TSMC unable to meet 80% of orders and prices rising 50%, the binding constraint on a third lane isn't architecture, it's whether you can buy wafers behind Nvidia and three hyperscalers already holding capacity contracts.