Post #1546580

2026-04-14 06:58 UTC

@regehr There are build systems with configurable retry, but avoiding the OOM killer is not my main reason for limiting link jobs to 1. LLD is multithreaded and linking is almost embarrassing parallel and so running two link jobs at a time doesn’t speed things up, but it does make cache usage much worse, and additional time slicing means the linker has to do more work stealing. Even on a system with enough memory for the link jobs to all fit in, running the sequentially is normally faster. Oh, and the amount of anonymous memory that the linker owns tends not to be that big. Less than clang compiling a big C++ file. It’s mostly mmap’d .o files. If the OOM killer’s heuristics are sensible, these won’t count because they can be kicked out of the buffer cache for free (pulling them back in is slow, which is another reason you want to do the links sequentially).

Replies (1)

@edwintorok@discuss.systems 2026-04-14 11:22

@david_chisnall @regehr if you have LTO enabled then linking becomes much slower, so running multiple linking jobs in parallel is still useful if your project consists of lots of executables. I noticed this when Fedora turned on LTO by default in GCC. Perhaps the build manager could look at the number and size of input files to the linker and try to limit parallelism based on that. E.g. you could probably link 2 executables in parallel if each uses 3 small inputs, but not if they use 1000 small inputs, or 3 large inputs. i.e. gave a budget of N linket inputs and M MiB and only launch additional linker jobs if this is not exceeded (with an exception to always launch the first if it is alone). But I don't know how to express this in any existing build system.

Open ##1546581