Post #4317940

So it's economically not realistic to buy the GPU capacity to use opencode and an open source / open weights model in your (or at least in my) home lab. And renting time from a hyperscaler to have the model sit idle most of the time is equally cost prohibitive. Has anyone created a LLMaaS setup where I control the model and all the data, but only pay for the compute when I actually use it? So like FaaS, except for LLMs with the primary use case being coding and code review? No, I'm not interested in cheaper alternatives to the frontier labs where my data still goes to a third party and said third party can turn things off when the powers that be decide to do so...