2026-08-12 18:14 UTC
Shrinking Apertus 1.5 8B for an 8 GB Laptop GPU.
Swiss AI's Apertus 1.5 8B fully local on an RTX PRO 1000, 8 GB VRAM, no cloud, no CPU offloading, 32K context. W4 quantization of embedding and output head. 14-22 tokens/s, up from 5.
Chat: solid. Tool calling: needs improvement. Model behavior, not quantization.
Open source: Linux, vLLM, llm-compressor, Pi coding agent, Local Agent Bench. No data leakage.
https://digitalpathlines.ch/2026/08/12/shrinking-apertus-1-5-8b-for-an-8-gb-laptop-gpu/
#LocalLLM #LocalAI #OpenSource #AI #SwissAI
Replies (0)
No replies.