Post #4391839

tested the XDNA NPU and i just can’t find a use case for the thing. in its fastest mode it runs half as fast as the iGPU does with llama.cpp/vulkan but needs more than half as much power (though it IS somewhat comparable to the performance i get with rocm so i guess that’s where the efficiency claims come from). the software for it can only use a very small number of predefined models with subpar quants and also doesn’t use mmap so instead of relying on the page cache it just keeps the entire model in RAM and does a bunch of expensive copies. that’s just embarrassing given that the all systems with supported AMD NPUs use unified memory. literally just use the page cache and have the kernel fault things into GPU/NPU memory when you need them. it’s free and you can’t do it any faster. if i switch to powersave mode, the iGPU consumes about the same amount of power but still runs considerably faster. and because it’s unified memory, the NPU can’t be used to reduce GPU burden either because it’d just be another thing competing for bandwidth. i sure am grateful to microsoft for requiring all this dead silicon that could have been part of the GPU 🙄