Post #4237149
2026-07-30 10:22 UTC
@jaseg@chaos.social my storage cluster nodes are supermicro x11spa-tf based, one with a xeon 4116 and the other two with 4208s.
The socket has three gen3 x16 pipes off it... one goes to the bifurcation card, one goes to the four m.2 ports on the mobo, and the remaining one goes into a PEX8747 that splits off to the remaining pcie slots which serve the NIC and have a few empty slots for expansion in the future.
I've iterated a few times with what goes in which slot to try to get the most pcie bandwidth but also cram as much NVMe into the nodes as possible without having to upgrade the mobos and processors. I can still add another 2ish SSDs per ndoe pretty comfortably without running into bandwidth limits so I'm confident in my original goal of keeping the cluster running into the 2030s without a chassis/mobo swap.
Replies (2)
-
@azonenberg@ioc.exchange 2026-07-30 10:27
@jaseg@chaos.social there's a reason i stopped buying consumer hardware ~15 years ago lol. well ok mostly it was I wanted ECC RAM (and a lot of it) but tons of PCIe lanes also come in handy for the things I do. I can't imagine having only 24 lanes in a workstation, a 100G NIC and a GPU is 32 before you even add anything else like SSDs
-
@jaseg@chaos.social 2026-07-30 10:26
@azonenberg@ioc.exchange Yeah I really want to avoid buying more hardware, and also my home office situation doesn’t really work with loud server hardware. When I upgrade, I think I’d go for something that I can fit into a desktop case (ATX/EATX with an older server or workstation platform)