Post #3367576
2026-06-20 11:25 UTC
@Hex@kolektiva.social @orange_lux@eldritch.cafe @saxnot@chaos.social @dibi58@this.mouse.rocks @karl@infosec.exchange I suspect you're underestimating the compute being dedicated to running (not training) the models.
I was recently observing a conversation between small startup founders who are buying €100k EGX Station AI workstations to move their developers from Cursor/Claude to local models. The same person seemed both to be questioning the value of all this LLM use by his dev team, at the same time considering buying €100k in hardware to reduce his LLM costs.
Replies (1)
-
@karl@infosec.exchange 2026-06-20 15:24
@sherbang@chaos.social @Hex@kolektiva.social @orange_lux@eldritch.cafe @saxnot@chaos.social @dibi58@this.mouse.rocks I work at a cloud provider and can confirm that running ai isn't as cheap as it sounds. My take is that most models are so general purpose that they're very inefficient (versatile, yes, but inefficient). Think "sorting an array through a bogo sort" inefficient. LLMs trained for a specific purpose may be more cost-effective to run long term, but that's not the norm.