Post #3713666
2026-07-10 06:34 UTC
Tiny engine, immense model.
- In a 744B mixture-of-experts GLM, only keep the dense part (attention, shared experts, embeddings) in memory, for 10GB.
- Swap the 20,000 routed experts in and out by need.
> This is not fast. It is a 744B frontier-class model answering correctly on a machine that costs less than one H100 fan.
https://github.com/JustVugg/colibri
Replies (1)
-
@troed@swecyb.com 2026-07-10 09:33
@avandeursen@mastodon.acm.org This is really interesting! A slow SOTA LLM for initial architecture decisions that can then hand over to much smaller models for implementation is very useful indeed.