Post #2687706
2024-09-08 16:29 UTC
@zrail@hachyderm.io The P40's need two physical x16 PCIe slots but I am running them x8 lanes. The issue is of course the lack of usable PCIe lanes on consumer grade hardware. I am unsure how much it really affects performance - running Llama 3.1 70b model is of course slower than ChatGPT but not unacceptable.
Replies (1)
-
@zrail@hachyderm.io 2024-09-08 16:40
@davidthor@hachyderm.io for sure. I have seen some builds (not for LLM) that go back a gen or two and use used Epyc boards off of eBay specifically for the PCIe lane overkill.