Post #4375253

@Xavier@infosec.exchange I know this. And I disagree about your TCO. I'm helping a friend with a $2500 computer, that uses approx $90 of electricity a month to run models which are inferior to those that I can rent off Mistral for about $40/month. And those are inferior - though not so much in practice - to frontier models like Opus which are impossible to host yourself, both technically and legally. When I runf Devstral2 on rented machines (hourly billed GPU VPSes) I can get comparable window sizes to that $40/month hosted version for about $800/month, provided I carefully engineer provisioning that bring the VPS down when unused. And I haven't then counted the hours tuning, deploying, monitoring etc that. Which could easily be another $1000/month. I then calculated the $/token or $/month for hosted models that in features/capabilities compare to the one in the $2500 computer at $90/month and it would've cost us under $5/month for the same usage. In other words: running models that are practically comparable to what's offered at $/token or $/month online currently costs way more than that. This obv. ignores all the other benefits of running yourself! I only look at the economics here, which, I know and admit, is too narrow.