Post #4262176

@faithisleaping@anarres.family @sundogplanets@mastodon.social The DCs powering AI are quite a different beast—modular DCs are still a thing but have different economics, and in the case of a neocloud, those DCs are monetized by having every GPU available working at full throttle. A dead GPU means compute capacity not being sold, which is already a loss, but if you're still consuming power and/or cooling, additional money left on the table. It's also worth noting that some model workloads can span multiple servers, so having "holes" in the rack makes the logistics of that kind of workload more complicated (because you have to deal with latency differentials, etc., going east-west that are faster going north-south.) The economics of a neocloud DC are also why they're so terrible for electricity and water consumption: idle nodes don't generate revenue.