Post #3006802
2026-02-19 13:24 UTC
Have you tried koboldcpp? I’m curious about the performance metrics between vulkan and opencl mode.
Replies (1)
-
@marvin@mastodon.nu 2026-02-19 19:19
I did not try koboldcpp but gave it a try now. I am not sure how to run the OpenCL backend, the wiki says "Vulkan is a newer option that provides a good balance of speed and utility compared to the OpenCL backend." cli arguments do not mention it. The Vulkan backend seems marginally slower than the one of llama.cpp for GLM-4.7-Flash: 19.8 tps with FlashAttention, 29.8 without. It seems koboldcpp is a fork of llama.cpp, maybe some Vulkan optimisations have not made it there, yet. @localllama