Rost Glukhov
@ros@techhub.social
Experienced technical leader with an eclectic range of interests - .net, golang, flutter, aws, reactjs and kubernetes, music, team leadership and astrophotography - love them all
techhub.social
Install llama.cpp, run GGUF models with llama-cli, and serve OpenAI-compatible APIs using llama-server. Key flags, examples, and tuning tips with a short commands cheatsheet
#Cheatsheet #AI #LLM #DevOps #OpenAI #API #SelfHosting #Prometheus #llama.cpp
https://www.glukhov.org/llm-hosting/llama-cpp/