Elektrine lite

← Feed

@Valmond@lemmy.dbzer0.com

2026-10-15 12:41 UTC

What kind of hardware do you need to run a model like that? And what do you use :-) ?

Replies (1)

  • @Sabata11792@ani.social 2026-10-15 13:16

    I run a 5 bit quantized Qwen 3.8 27b at home on a gaming PC with an RX 7900 XTX and RAG memory using Gemma MoE (i forget what one)running on an old 1080 ti on my Valheim server. It takes 3-15 minutes to run depending on how many tool calls the model makes and that’s mostly memory or web scrapes since that runs on the slower hardware. I mostly just use it as a discord bot to look stuff up for me and to fetch the news.

    Open ##5231404