Post #1099314

I wrote about running LLMs locally on the Intel Arc Pro B60 GPU previously, where I used Intel's official software stack (llm-scaler / vLLM). This time I focus on the impactful open-source project llama.cpp: https://marvin.damschen.net/post/intel-arc-llama.cpp/ Taking this opportunity to test federation with lemmy 😊: @localllama #llm #selfhosting #privacy #intel #llamacpp