Post #3232039

Rabbithole of the day: Some guy blogs since 6mon and did more cool shit than i in 30+y > Randomly finding "LLM Neuroanatomy" that archived a new Open LLM Leaderboard record by repeating middle layers and 0 finetuning https://dnhkng.github.io/posts/rys/ > He bought a semi-broken server with 2 nvidia gh200 (80k€ for 1/10th of the price) and fixed it up. The 1tb ram alone costs the 8k https://dnhkng.github.io/posts/hopper/ > He served the perfect chinese SOTA model from it https://dnhkng.github.io/posts/vllm-optimization-gh200/ It even has rss support