Post #2245567
2026-04-24 12:58 UTC
Nice article. Thanks for sharing your experience. @alan@lighthouse.co.im
I really like reading real world reports from people using AI independent. Too many success stories leave out which API (or external token) usage was needed to get to the goal. Also many claim that running a solution all your own works because ollama serves the same API.
I think there is a lot possible already with even moderate consumer hardware. It requires some engineering effort to tune such systems right. But its worth it.
Replies (1)
-
@alan@lighthouse.co.im 2026-04-25 06:45
@guesser@sigmoid.social Thanks Roman – you’ve put your finger on something that bothers me too. The “just run Ollama” narrative skips the part where you’ve tuned the context window, wired up the API bridge, and figured out which model actually fits in your VRAM without thrashing. The engineering is real. The MacBook Air M4 with 24GB unified memory is what makes Mistral Small 24B viable locally – that’s not a footnote, that’s the whole story. (1/2)