Re: pahole -J non-determinism and reproducible builds

[Arnaldo Carvalho de Melo <acme@kernel.org>]

On Thu, May 23, 2024 at 08:25:17PM +0100, Giuliano Procida wrote:
> Hi.
> 
> (edited repost due to unfortunate HTML email).
> 
> Until the reproducible BTF changes land in a new pahole version, and for
> anyone stuck on older versions, pahole -J -j typically produces
> non-deterministic BTF.
> 
> For Android, we are converging to hermetic, reproducible builds. .BTF
> sections are a significant difference we see. I have trivial patches to
> pass -j1 instead of -j for the older versions we are using (1.23 and 1.25).
> 
> Trivial patches:
> https://android-review.googlesource.com/q/Ibd72ac638faa1826f6655b336cc7001591ea70f1

That should be at least configurable thru a CONFIG_REPRODUCIBLE_BUILD
that then would check the pahole version and if less than 1.27 then
would use -j1

So it continues to use -j from the version where it was introduced,
1.22, but for people wanting reproducible builds it plain don't use -j
or if you really want to be explicit, uses -j1 as you did.

For >= 1.27 we will have 'reproducible_build' added to the
--btf_features= line if CONFIG_REPRODUCIBLE_BUILD is set.
 
> Gaining determinism for the older versions means giving up some performance.
> About a factor of 2.1 in our case (or an extra 10s on vmlinux).
 
> Do others want performance or determinism, should I send patches to LKML
> and the stable list?

But that is with older paholes, for 1.27, to be released next week, one
will just use '--btf_features=+reproducible_build and continue to have
parallel DWARF loading with serialized BTF encoding, which should be
close to the non-reproducible build numbers.
 
> Off-topic: pahole -J -j9 is faster than -j on my machine with 36
> cores, by another
> factor of 2. This is v1.25. -j1 18.7s, -j9 4.5s, -j 8.9s.

I'll try to profile that on a 5950X (16 cores, 32 threads) and a 14700K
(8 performance cores (16 threads) + 12 efficiency cores (12 threads),
total 28 threads) to try and see if I can get a better heuristic that
takes into account these factors on the most recent codebase that will
become 1.27 next week.

- Arnaldo