Post #3121857

It means that capability growth is going to slow and require more creative ways to improve than just more tokens and more compute. I've seen some research that we can create chips that are 10x as efficient but they can only run a single model and aren't upgradable. If a model is viable for a number of years because genuine improvement is so slow, the math starts to make sense there. It's going to be a good thing when we are forced to start looking at more creative ways of improvement.