Post #4373822

@junesim63@mstdn.social There are a few things I disagree with in this: First, a couple of small nits, it says: The big tech companies have spent $1 trillion on capital investment since the start of the AI boom That's a very low estimate, most estimates are closer to $3 T. First, there’s too much competition among US LLM providers. It’s not just ChatGPT and Claude now – Gemini and Copilot are nipping at their heels There's some conflation of models and end-user products here, which is important because it hides the shape of the supply chain. But then we get to: And the world will be left with hundreds of massive data centres without enough customers. Compute will become extremely cheap – just like accessing the internet did after the telecoms companies spent billions competing to lay fibre optic cables in the 1990s. Accessing the Internet didn't immediately get cheap. Companies laid a lot of fibre. This was very expensive to do and had low returns. These assets became available cheaply. Those assets were useful because it's easy to light up dark fibre. The equipment at the ends that you need to provide (and which can be a newer technology than was available when the fibre was laid) is much cheaper than digging up streets or laying very long runs of fibre between cities. And most of that fibre is still useful 20+ years later, with an expected lifespan of 25-50 years. 'AI' datacentres are not like this, for several reasons: First, they are very expensive to operate. They're measured in GWs: even if all of the capital is a sunk cost that you can write off, the cost of simply turning them on is enormous. That sets an absolute lower bound on the price. And, because they're such massive grid consumers, they're likely to be blocked from that much consumption as soon as the hype wave's marketing shield wears off. Second, the GPUs wear out. The average lifetime for NVIDIA GPUs in these things is estimated at three years. So even if you buy one for pennies on the dollar and can afford the electricity, it won't stat working at maximum capacity for long. And those GPUs are aggressively tailored for specific kinds of ML workload. Even if they're basically free, they are unlikely to be the most cost effective way of running any other kind of workload. Finally, these datacentre designs are very specialised. These kinds of machine-learning workloads are incredibly memory-sensitive. This means that you need to put very fast memory next to very dense floating-point compute to get performance. This, in turn, means that it isn't just enough to have a lot of power, you need a a lot of power per rack. That places a lot of strain on cooling and makes it very expensive to cool. If you're building a datacentre for any other workload, that's unnecessary. Land is not that expensive in the places where they're built, so you build at a much lower power density, which reduces your cooling costs. It's not at all clear that refitting an 'AI' datacentre to be a useful datacentre will be cheaper than building a new datacentre from scratch. If you visit the outside of Taipei, you can see the history of semiconductor fabrication because it's always been cheaper to build a new factory than refurbish an old one. 'AI' datacentres may well end up like this. The boom will get going again after the bust – with valuations returning to more sensible levels. A few of the massive tech companies will buy up the remnants of the others, leaving the market even more concentrated. There are a lot of assumptions here. Currently, we are seeing that the market for these things is around 1% of the cost of building and running them. Training an LLM costs an enormous amount. Companies are basically eating those costs because they expect to be able to make it up by charging for access to the models. And they're also losing money providing the models as a service because they're expensive to operate. After the bubble bursts who will fund training of new models? Old models gradually use utility (people are already complaining about LLMs that don't know about anything that happened after 2024 because that's the end of their training data set). If inference is not bringing in as much money as it costs to operate the systems, who will continue doing it? And, when the 'it will massively improve your economic competitiveness' claims are proven to be lies, will governments keep giving companies a free pass from massive copyright infringement? If you want to claim that companies will keep selling these things after the bubble bursts, you need to answer these questions. The economics of LLMs are terrible. They have high costs to train (capex) and high costs for inference (opex). The most successful things in the market generally have a different mix: High capex and low opex means ahigh barrier to entry for new competitors but then increasing profits the larger you are. Microsoft, Google, and Amazon's core products all look like this. The best products in this case have per-unit opex that drops as you increase the number of units, because even a new company with the same amount of up-front capital as you isn't able to match your price without losing money until they have as many customers as you. Low capex and low opex means things become cheap commodities. They will often become ubiquitous, but generic. It's hard to gain a dominant market position because it's trivial for a new competitor to pop up. A lot of consumer goods look like this. Low capex and high opex often survives as boutique services. Things like Michelin-starred restaurants: easy to create (lease a building), expensive to operate (hire some very expensive staff). But there are very, very few examples of things where the capex and opex are both high becoming major parts of an economy. Private jets, for example, are in this class and they're a tiny niche in the corner of travel. Wealth and power will become even more concentrated, as the largest tech companies buy up their defunct competitors, before leveraging their control over the technological infrastructure that most other businesses require to function I suspect the latter part of this is misdiagnosed. Big tech firms are massively over leveraged at the moment. NVIDIA has loans that amount to more than its pre-bubble valuation and its current valuation is predicated on completely unattainable growth. It makes sense only if you imagine that it's a company that will, over the long term, be able to sell more than 10x as many chips as Intel at its peak. When these companies' stock prices collapse, they're going to find it very hard to raise money for anything and they won't be able to buy companies with their stock (that's partly why they're on a buying spree now). The people who will make money are the people who have other liquid assets. Either cash, or things like real estate that they can borrow against as safe assets when the bubble bursts.