Google releases Gemma 4 open models
2026-04-03 07:01 UTC
Replies (4)
-
@madcaesar@lemmy.world 2026-04-03 10:06
What could this be used for?
-
@mrnobody@reddthat.com 2026-04-03 11:41
Why would anyone care about Gemini or other AI here? I mean, i get this is the tech Space, but AI=bad.
-
@brucethemoose@lemmy.world 2026-04-03 13:54
They seem to have held back the "big" locally runnable model. It's also kinda conservative/old, architecture wise: 16-bit weights, sliding window attention interleaved with global attention. No MTP, no QAT (yet), no tightly integrated vision, no hybrid mamba like Qwen/Deepseek, nothing weird like that. It's especially glaring since *we know* Google is using an exotic architecture for Gemini, and has basically infinite resources for experimentation. It also feels kinda "deep fried" like GPT-OSS to me, see: https://github.com/ikawrakow/ik_llama.cpp/issues/1572 > it is acting crazy. it can't do anything without the proper chat template, or it goes crazy. *** IMO it's not very interesting, especially with so many other models that run really well on desktops.
-
@brucethemoose@lemmy.world 2026-04-03 14:14
Also, for any interested, desktop inference and quantization is my autistic interest. Ask my anything. I don't like Gemma 4 much so far, but if you want to try it anyway: - On Nvidia with no CPU offloading, watch this PR and run it with TabbyAPI: https://github.com/turboderp-org/exllamav3/pull/185 - *With* CPU offloading, watch this PR and the mainline llama.cpp issues they link. Once Gemma4 inference isn't busted, run it in IK or mainline llama.cpp: https://github.com/ikawrakow/ik_llama.cpp/issues/1572 - If you're on an AMD APU, like a Mini PC server, look at: https://github.com/lemonade-sdk/lemonade - On an AMD or Intel GPU, either use llama.cpp or kobold.cpp with the vulkan backend. - Avoid ollama like it's the plague. - Learn chat templating and play with it in mikupad before you use a "easy" frontend, so you understand what its doing internally (and know when/how it goes wrong): https://github.com/lmg-anon/mikupad *** But TBH I'd point most people to Qwen 3.5/3.6 or Step 3.5 instead. They seem big, but being sparse MoEs, they can run quite quickly on single-GPU desktops: https://huggingface.co/models?other=ik_llama.cpp&sort=modified