@ariadne@social.treehouse.systems
2026-09-14 18:14 UTC
Replies (1)
-
@ariadne@social.treehouse.systems 2026-09-14 18:19
the scaling laws themselves will likely continue to hold just fine, but the bottleneck is increasingly the infrastructure required to realize the next point on the scaling curve. for a concrete example: a 1-trillion-parameter dense model at 16-bit precision needs about 2 TB of memory just to hold its weights. that's roughly 25 nvidia H100 accelerators just to load one instance of the model. and once a single inference has to cross dozens of accelerators, memory bandwidth and chip-to-chip communication become part of the latency budget too. at that point, “make it bigger” is no longer mainly a model-design problem. it is a hardware, networking, power, and datacenter problem.