Re: [PATCH dwarves] pahole: enable --reproducible_build when SOURCE_DATE_EPOCH is set

[Arnaldo Carvalho de Melo <acme@kernel.org>]

On Fri, Jun 28, 2024 at 09:31:17AM +0100, Alan Maguire wrote:
> On 27/06/2024 18:25, Alan Maguire wrote:
> > On 27/06/2024 00:01, Dominique Martinet wrote:
> >> Alan Maguire wrote on Wed, Jun 26, 2024 at 03:29:37PM +0100:
> >>> On 26/06/2024 04:22, Dominique Martinet wrote:
> >>>> The SOURCE_DATE_EPOCH environment variable is used in reproducible
> >>>> builds to set build timestamps and is generally synonymous with
> >>>> reproducible builds: instead of making users figure out how to pass
> >>>> pahole flags (e.g. explicitly setting PAHOLE_FLAGS in linux) just assume
> >>>> that the user wants a reproducible build if this variable is set.

> >>> From the kernel perspective it looks like as a result of [1]
> >>> [1] https://patchwork.kernel.org/project/linux-kbuild/patch/1443741332.2730.75.camel@decadent.org.ukOB/
> >>> the setting of SOURCE_DATE_EPOCH will be converted into
> >>> KBUILD_BUILD_TIMESTAMP; I thought the plan was to add the
> >>> BTF feature flag explicitly if KBUILD_BUILD_TIMESTAMP was set?

> >> Hmm, that patch (2015) never got in apparently?...
> >> And I don't see anythiing that'd make pahole output reproducible if
> >> either variables are set in the kernel either right now; we still build
> > 
> > I can send a patch for this to bpf-next if it's needed. But it'd be good
> > to figure out the right approach first, more below...
> > 
> >> plenty of older kernels so even if such a patch gets in I'd rather this
> >> is checked in pahole than kbuild.

> > I'm not sure that's the right answer though - there's a bunch of
> > existing infrastructure in the kernel tree that relies on
> > KBUILD_BUILD_TIMESTAMP, so to integrate properly with the existing
> > kbuild infrastructure around reproducible builds feels like the right
> > direction to me. It's a bit more work, but not too much. Arnaldo may
> > disagree of course, and he's the maintainer so it's his decision - this
> > is just my personal take.
> > 
> > We've tried to explicitly enable flags in pahole Makefile.btf so that
> > it's obvious from pahole invocations which features are wanted, and to
> > get that support on older kernels it would just be a matter of
> > backporting the
> > 
> > if > 1.26
> > 	use --btf_features
> > 
> > ...patch, and a patch for adding the reproducible build option to stable
> > kernels. Similar has been done in the past for other pahole features.
> > 
> > All that's needed here for bpf-next tree is I think is the below 3-line
> > addition:
> > 
> > diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf
> > index b75f09f3f424..40bb72662967 100644
> > --- a/scripts/Makefile.btf
> > +++ b/scripts/Makefile.btf
> > @@ -21,6 +21,10 @@ else
> >  # Switch to using --btf_features for v1.26 and later.
> >  pahole-flags-$(call test-ge, $(pahole-ver), 126)  = -j
> > --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func,decl_tag_kfuncs
> > 
> > +ifneq ($(KBUILD_BUILD_TIMESTAMP),)
> > +pahole-flags-$(call test-ge, $(pahole-ver), 126) +=
> > --btf_features=reproducible_build
> > +endif
> > +
> >  ifneq ($(KBUILD_EXTMOD),)
> >  module-pahole-flags-$(call test-ge, $(pahole-ver), 126) +=
> > --btf_features=distilled_base
> >  endif
> > 
> > 
> > Can you test at your end if that does the right thing? Thanks!
> 
> FWIW I verified that with the above change, and
> 
> export KBUILD_BUILD_TIMESTAMP=$(date -I)
> 
> ...the feature flag is added to pahole and as a result we get
> reproducible BTF in vmlinux; .btf.vmlinux.bin.o is identical across
> multiple kernel builds.

This sounds reasonable, pahole doesn't have to look at something so
specific for a particular use case (the kernel) while using what pahole
would look (KBUILD_BUILD_TIMESTAMP) but in the kernel build scripts,
that in turn uses the prefered, self-documenting way of asking for some
specific BTF feature (--btf_features).

- Arnaldo