Post #4375254

2026-08-02 17:21 UTC

@berkes@mastodon.nl you do know that you can run all this at home. I'm only spending $20 a month with any provider at any given point, just to learn about the different platforms. I'm writing tons of software for me and my family using a 3090. I think it's a really accessible technology.

Replies (1)

@berkes@mastodon.nl 2026-08-04 08:05

@Xavier@infosec.exchange I know this. And I disagree about your TCO. I'm helping a friend with a $2500 computer, that uses approx $90 of electricity a month to run models which are inferior to those that I can rent off Mistral for about $40/month. And those are inferior - though not so much in practice - to frontier models like Opus which are impossible to host yourself, both technically and legally. When I runf Devstral2 on rented machines (hourly billed GPU VPSes) I can get comparable window sizes to that $40/month hosted version for about $800/month, provided I carefully engineer provisioning that bring the VPS down when unused. And I haven't then counted the hours tuning, deploying, monitoring etc that. Which could easily be another $1000/month. I then calculated the $/token or $/month for hosted models that in features/capabilities compare to the one in the $2500 computer at $90/month and it would've cost us under $5/month for the same usage. In other words: running models that are practically comparable to what's offered at $/token or $/month online currently costs way more than that. This obv. ignores all the other benefits of running yourself! I only look at the economics here, which, I know and admit, is too narrow.

Open ##4375253