Post #4505509

@jcoglan@mastodon.social Yeah, you've fundamentally misunderstood how 'running for hours on end' works. Compaction and other reductions of the context window happen. Things are delegated to sub-agents with their own context windows. This is not unbounded context growth at O^2, this is pretty tightly managed context windows where window^2 is an acceptable number and GPU does go BRR. Nobody is doing billions squared. They're doing, at most, 1M, and usually more like 100k. All of this is made more efficient by attention architectures, too, which reduce that.