I cannot remember what I did to make ninja cap itself at 2 processes instead of auto-detecting 6 cores and deciding to run 6 processes. (ninja is invoked by cmake, according to the docs there is no way to control this through an environment variable for ninja but I think there was a way to get cmake to pass -j2 or something. There are too many build systems. At any rate, since the qt6-webengine build has been running for two days, I don’t want to kill it and start from scratch.)
The issue is that too many parallel invocations of the C++ compiler overfill the RAM, and start bleeding into swap, and once the swap starts filling up, everything starts running at a glacial pace. You can kind of manually do it to a running process, though, by just suspending the ninja process (kill -STOP) and then letting the C++ compilers finish, and resuming the ninja process once RAM use is low enough. Since it’s swapping, I figured it’d be easiest to use renice -n -1 to bump the priority of whatever compiler process was using the most RAM, to get it out of the way faster.
Larry Wall said the three virtues of a great programmer are laziness, impatience, and hubris.
Exercising my laziness, the first virtue, I enlisted awk to just stop ninja when load average got over 4 and bring it back when it went back under 3:
while sleep 1; do uptime; done | mawk -Winteractive '
BEGIN{system("sudo killall -CONT ninja");s=0}
(0+$10) > 4 && !s{system("sudo killall -STOP ninja");s=1}
(0+$10) < 3 && s{system("sudo killall -CONT ninja");s=0}
{print $10, s}'
The second virtue being impatience, maybe there’s a good way to speed the process up. Since the bottleneck for the C++ compiler is RAM and I/O rather than CPU, I think I’ll make another attempt at distcc, but this time, instead of running a cross-compiler, I’ll just copy enough of the rootfs to run a minimal chroot with a compiler under qemu-aarch64 and then run distccd under that. (Shouldn’t have the problem with bad binaries if it’s not just the same version of the compiler, but an emulator running an exact copy of the same compiler, right? …Right?) If that works, it should help with the C++ stuff; I don’t think distcc works for Rust (so it might not help when building Firefox), and gcc doesn’t require nearly as much RAM as g++ (so it’s not needed for regular C compiler invocations).
The third virtue is hubris, and I use this for everything I do.