2026-10-09 09:09 UTC
Qwen3.6-35B-A3B running locally on my laptop: RTX PRO 1000, 8GB VRAM + 32GB RAM in WSL.
4-bit GGUF, llama.cpp/CUDA, expert weights in pinned RAM, dynamic GPU expert caching, and Q8 KV cache. Connected to Pi for coding and tool use.
My Pi session is at \~30% of the 262K context (\~79K tokens). Even with Pi set to medium thinking, it feels fast. Prompt caching helps keep follow-up turns responsive.
Also verified retrieval across 261K input tokens.
\#LocalAI \#Qwen \#llamacpp
Replies (0)
No replies.