Re: Wislist for Linux from the mold linker's POV

[Florian Weimer <fw@deneb.enyo.de>]

* Rui Ueyama:

> - exit(2) takes a few hundred milliseconds for a large process
>
> I believe this is because mold mmaps all input files and an output
> file, and clearing/flushing memory-mapped data is fairly expensive. I
> wondered if this could be improved. If it is unavoidable, could the
> cleanup process be made asynchronous so that exit(2) takes effect
> immediately?

It's definitely a two-edged sword.  For example, when running parallel
make (or Ninja), it's essential that process exit is only signaled
after all process-related resources have been released.  Otherwise,
it's possible to see spurious failures because make respawns processes
so quickly that some resource limit is exceeded.  This is already a
problem today, and more lazy resource deallocation on exit would make
it more prevalent.

The situation is already bad enough that many developers have resorted
to retry loops around fork/clone/pthread_create if an EAGAIN error is
encountered, assuming  that it's related to this.

  Bug 154011 - Task exit is signaled before task resource
  deallocation, leading to bogus EAGAIN errors
  <https://bugzilla.kernel.org/show_bug.cgi?id=154011>

> - Writing to a fresh file is slower than writing to an existing file
>
> mold can link a 4 GiB LLVM/clang executable in ~1.8 seconds on my
> machine if the linker reuses an existing file and overwrites it.
> However, the speed decreases to ~2.8 seconds if the output file does
> not exist and mold needs to create a fresh file. I tried using
> fallocate(2) to preallocate disk blocks, but it didn't help. While 4
> GiB is not small, should creating a file really take almost a second?

Which file system is that?

> - Lack of a safe system-wide semaphore

Other toolchain components use the make jobserver protocol for that.