[PATCH] kbuild: disable pahole multithreading for reproducible builds

[PATCH] kbuild: disable pahole multithreading for reproducible builds

[Thomas Weißschuh]

A BTF type_id is a numeric identifier allocated by pahole through
libbpfd. Ids are incremented for each allocation.
Running pahole multithreaded makes the sequence of allocations
non-deterministic which also makes the type_id itself non-deterministic.
As the type_id end up in the binary this breaks reproducibility.

Therefore, if the kernel is built reproducibly as indicated by
KBUILD_BUILD_TIMESTAMP, disable threading in pahole.

Fixes: b4f72786429c ("scripts/pahole-flags.sh: Parse DWARF and generate BTF with multithreading.")
Fixes: 72d091846de9 ("kbuild: avoid too many execution of scripts/pahole-flags.sh")
Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 scripts/Makefile.btf | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf
index 82377e470aed..2356929e62e4 100644
--- a/scripts/Makefile.btf
+++ b/scripts/Makefile.btf
@@ -10,7 +10,10 @@ endif
 
 pahole-flags-$(call test-ge, $(pahole-ver), 121)	+= --btf_gen_floats
 
+# threaded execution randomizes BTF type_id allocation, breaking reproducibility
+ifeq ($(KBUILD_BUILD_TIMESTAMP),)
 pahole-flags-$(call test-ge, $(pahole-ver), 122)	+= -j
+endif
 
 pahole-flags-$(CONFIG_PAHOLE_HAS_LANG_EXCLUDE)		+= --lang_exclude=rust
 

---
base-commit: 8e938e39866920ddc266898e6ae1fffc5c8f51aa
change-id: 20240322-pahole-reprodicible-02e904e45686

Best regards,
-- 
Thomas Weißschuh <linux@weissschuh.net>

Re: [PATCH] kbuild: disable pahole multithreading for reproducible builds

[Alan Maguire]

On 22/03/2024 08:31, Thomas Weißschuh wrote:
> A BTF type_id is a numeric identifier allocated by pahole through
> libbpfd. Ids are incremented for each allocation.
> Running pahole multithreaded makes the sequence of allocations
> non-deterministic which also makes the type_id itself non-deterministic.
> As the type_id end up in the binary this breaks reproducibility.
> 
> Therefore, if the kernel is built reproducibly as indicated by
> KBUILD_BUILD_TIMESTAMP, disable threading in pahole.
> 
> Fixes: b4f72786429c ("scripts/pahole-flags.sh: Parse DWARF and generate BTF with multithreading.")
> Fixes: 72d091846de9 ("kbuild: avoid too many execution of scripts/pahole-flags.sh")
> Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>

Yep, we've talked about sorting final BTF in various ways but until that
is implemented, non-parallel execution of pahole is the best way to get
reliabily-reproducing BTF ids. We should probably update the pahole
manpage to warn about parallel non-deterministic id allocation in the
meantime.

Reviewed-by: Alan Maguire <alan.maguire@oracle.com>

> ---
>  scripts/Makefile.btf | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf
> index 82377e470aed..2356929e62e4 100644
> --- a/scripts/Makefile.btf
> +++ b/scripts/Makefile.btf
> @@ -10,7 +10,10 @@ endif
>  
>  pahole-flags-$(call test-ge, $(pahole-ver), 121)	+= --btf_gen_floats
>  
> +# threaded execution randomizes BTF type_id allocation, breaking reproducibility
> +ifeq ($(KBUILD_BUILD_TIMESTAMP),)
>  pahole-flags-$(call test-ge, $(pahole-ver), 122)	+= -j
> +endif
>  
>  pahole-flags-$(CONFIG_PAHOLE_HAS_LANG_EXCLUDE)		+= --lang_exclude=rust
>  
> 
> ---
> base-commit: 8e938e39866920ddc266898e6ae1fffc5c8f51aa
> change-id: 20240322-pahole-reprodicible-02e904e45686
> 
> Best regards,

Re: [PATCH] kbuild: disable pahole multithreading for reproducible builds

[Arnaldo Carvalho de Melo]

On Fri, Mar 22, 2024 at 10:07:00AM +0000, Alan Maguire wrote:
> On 22/03/2024 08:31, Thomas Weißschuh wrote:
> > A BTF type_id is a numeric identifier allocated by pahole through
> > libbpfd. Ids are incremented for each allocation.
> > Running pahole multithreaded makes the sequence of allocations
> > non-deterministic which also makes the type_id itself non-deterministic.
> > As the type_id end up in the binary this breaks reproducibility.
> > 
> > Therefore, if the kernel is built reproducibly as indicated by
> > KBUILD_BUILD_TIMESTAMP, disable threading in pahole.
> > 
> > Fixes: b4f72786429c ("scripts/pahole-flags.sh: Parse DWARF and generate BTF with multithreading.")
> > Fixes: 72d091846de9 ("kbuild: avoid too many execution of scripts/pahole-flags.sh")
> > Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
> 
> Yep, we've talked about sorting final BTF in various ways but until that
> is implemented, non-parallel execution of pahole is the best way to get
> reliabily-reproducing BTF ids. We should probably update the pahole

So, the problem is about parallel BTF encoding, not so much about
parallel DWARF loading, which speeds up the whole process and could
continue if we make sure that we feed the BTF encoder in the same order
as when not doing anything in parallel.

I.e. a serial BTF encoding done at the same time as a parallel DWARF
loading would produce a reproducible build faster than doing
_everything_ serially.

The important point would be to feed the btf encoder the DWARF CUs in
the same order as when doing it serially.

No need to order anything, just create a linked list/array of CUs and
then start loading them and leave the BTF encoder go on taking CUs from
that linked list/array in the order they are in vmlinux.

- Arnaldo

> manpage to warn about parallel non-deterministic id allocation in the
> meantime.
> 
> Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
> 
> > ---
> >  scripts/Makefile.btf | 3 +++
> >  1 file changed, 3 insertions(+)
> > 
> > diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf
> > index 82377e470aed..2356929e62e4 100644
> > --- a/scripts/Makefile.btf
> > +++ b/scripts/Makefile.btf
> > @@ -10,7 +10,10 @@ endif
> >  
> >  pahole-flags-$(call test-ge, $(pahole-ver), 121)	+= --btf_gen_floats
> >  
> > +# threaded execution randomizes BTF type_id allocation, breaking reproducibility
> > +ifeq ($(KBUILD_BUILD_TIMESTAMP),)
> >  pahole-flags-$(call test-ge, $(pahole-ver), 122)	+= -j
> > +endif
> >  
> >  pahole-flags-$(CONFIG_PAHOLE_HAS_LANG_EXCLUDE)		+= --lang_exclude=rust
> >  
> > 
> > ---
> > base-commit: 8e938e39866920ddc266898e6ae1fffc5c8f51aa
> > change-id: 20240322-pahole-reprodicible-02e904e45686
> > 
> > Best regards,
>