Elektrine lite

← Feed

@ebelo@mastodon.social

2026-10-09 09:09 UTC

Qwen3.6-35B-A3B running locally on my laptop: RTX PRO 1000, 8GB VRAM + 32GB RAM in WSL. 4-bit GGUF, llama.cpp/CUDA, expert weights in pinned RAM, dynamic GPU expert caching, and Q8 KV cache. Connected to Pi for coding and tool use. My Pi session is at \~30% of the 262K context (\~79K tokens). Even with Pi set to medium thinking, it feels fast. Prompt caching helps keep follow-up turns responsive. Also verified retrieval across 261K input tokens. \#LocalAI \#Qwen \#llamacpp

Replies (0)

No replies.