Post #1546504

2026-04-14 03:51 UTC

@dotstdy @tedmielczarek @regehr but we don't have a way to predict how much memory a linker invocation will consume, do we? in which case that's sort of what we're stuck with, since any solution must necessarily be reactive

Replies (3)

@dotstdy@mastodon.social 2026-04-14 03:56

@whitequark @tedmielczarek @regehr I think tuning it the other way (i.e. this invocation requires 2gb) and having the job server track a budget, delaying launch (without introducing deadlocks) would allow you to scale to arbitrary hardware so long as the annotations are somewhat accurate. But yeah it's fraught with complexity. See also some kind of back-off recovery for extreme pressure since the work is ideally possible to re-launch.

Open ##1546505
@jannem@fosstodon.org 2026-04-14 03:56

@whitequark @dotstdy @tedmielczarek @regehr Exactly. The build system had no clue how much resources each tool invocation is going to need. And say, the linker can only see how much memory is available on the host, not how much would actually be okay for it to use. Perhaps we need a "--mem" option for tools, analogous to the -j option for threads.

Open ##1546551
@tedmielczarek@mastodon.social 2026-04-14 14:22

@whitequark @dotstdy @regehr now I'm wondering if there are heuristics that would be workable… Number of input objects? Total size (in bytes) of all input objects? Things like LTO complicate matters, but maybe they could be accounted for?

Open ##1546552