2026-10-15 12:41 UTC
What kind of hardware do you need to run a model like that? And what do you use :-) ?
Replies (1)
-
@Sabata11792@ani.social 2026-10-15 13:16
I run a 5 bit quantized Qwen 3.8 27b at home on a gaming PC with an RX 7900 XTX and RAG memory using Gemma MoE (i forget what one)running on an old 1080 ti on my Valheim server. It takes 3-15 minutes to run depending on how many tool calls the model makes and that’s mostly memory or web scrapes since that runs on the slower hardware. I mostly just use it as a discord bot to look stuff up for me and to fetch the news.