Post #1333343

2026-04-17 05:32 UTC

Say I have a GPU with 32GB VRAM and I am on Linux, what local LLM would be good for coding? Currently I just have an iGPU ;) but that's always an option, albeit a very expensive one.

Replies (2)

@andrew0@lemmy.dbzer0.com 2026-04-17 06:27

Get llama.cpp and try Qwen3.6-35B-A3B. Just came out and looks good. You'll have to look into optimal settings, as it's a Mixture of Experts (MoE) model with only 3B parameters active. That means that the rest can stay in RAM for quick inference. You could also try the dense model (Qwen3.5-27B), but that will be significantly slower. Put these in a coding harness like Oh-My-Pi, OpenCode, etc. and see how it fares for your tasks. Should be ok for small tasks, but don't expect Opus / Sonnet 4.6 quality, more like better than Haiku.

Open ##1333350
@SuspciousCarrot78@lemmy.world 2026-04-17 05:48

Sadly...none. Well, I mean...it depends what you mean by "coding". If you mean "replace Claude with local?". Then...none. Sorry. If you mean "actually, if I use [ECA ](https://eca.dev/)to call a cloud model from [OpenRouter ](https://openrouter.ai/) for planning, then have it direct a local LLM to do the scutt work", then the Qwen series of models (like Qwen 3 Next) are pretty awesome. The iGPU will make you want to kill yourself though. Get a GPU :) Even a 4-16GB one can make a difference. PS: You said GPU and iGPU, so I'm not sure which one has the 32GB or what rig your running. I have suspicion though you're running on a i5 or i7 with something like a intel 630 igpu inbuilt? In which case, the iGPU is pretty slow and depending on the exact chip, you likely won't be able to use CUDA or Vulkan acceleration. So, the "get a GPU" thing still holds :)

Open ##1333351