@ariadne@social.treehouse.systems
2026-09-14 18:19 UTC
the scaling laws themselves will likely continue to hold just fine, but the bottleneck is increasingly the infrastructure required to realize the next point on the scaling curve.
for a concrete example: a 1-trillion-parameter dense model at 16-bit precision needs about 2 TB of memory just to hold its weights. that's roughly 25 nvidia H100 accelerators just to load one instance of the model.
and once a single inference has to cross dozens of accelerators, memory bandwidth and chip-to-chip communication become part of the latency budget too.
at that point, “make it bigger” is no longer mainly a model-design problem. it is a hardware, networking, power, and datacenter problem.
Replies (1)
-
@ariadne@social.treehouse.systems 2026-09-14 18:23
the growth in infrastructure requirements just to scale models dramatically changes what frontier scaling looks like over time. instead of a relatively smooth progression where each new generation mostly waits on another training run, you get something much more stair-stepped similar to the intel tick-tock model: new accelerator generation (tick) → build out enough capacity → scale the model → optimize around the new bottlenecks (tock) → plateau → wait for the next hardware generation. in other words, the pace of frontier capability starts getting set less by model architecture and more by nvidia, tsmc, hbm supply, networking, power delivery and datacenter construction.