Elektrine lite

← Feed

@ebelo@mastodon.social

2026-08-12 18:14 UTC

Shrinking Apertus 1.5 8B for an 8 GB Laptop GPU. Swiss AI's Apertus 1.5 8B fully local on an RTX PRO 1000, 8 GB VRAM, no cloud, no CPU offloading, 32K context. W4 quantization of embedding and output head. 14-22 tokens/s, up from 5. Chat: solid. Tool calling: needs improvement. Model behavior, not quantization. Open source: Linux, vLLM, llm-compressor, Pi coding agent, Local Agent Bench. No data leakage. https://digitalpathlines.ch/2026/08/12/shrinking-apertus-1-5-8b-for-an-8-gb-laptop-gpu/ #LocalLLM #LocalAI #OpenSource #AI #SwissAI

Replies (0)

No replies.