ANNOUNCE: pahole v1.27 (reproducible builds, BTF kfuncs)

ANNOUNCE: pahole v1.27 (reproducible builds, BTF kfuncs)

[Arnaldo Carvalho de Melo]

Hi,
 
	The v1.27 release of pahole and its friends is out, supporting
parallel reproducible builds and encoding kernel kfuncs in BTF, allowing
tools such as bpftrace to enumerate the available kfuncs and obtain its
function signatures and return types.

Main git repo:

   https://git.kernel.org/pub/scm/devel/pahole/pahole.git

Mirror git repo:

   https://github.com/acmel/dwarves.git

tarball + gpg signature:

   https://fedorapeople.org/~acme/dwarves/dwarves-1.27.tar.xz
   https://fedorapeople.org/~acme/dwarves/dwarves-1.27.tar.bz2
   https://fedorapeople.org/~acme/dwarves/dwarves-1.27.tar.sign

	Thanks a lot to all the contributors and distro packagers, you're on the
CC list, I appreciate a lot the work you put into these tools,

Best Regards,

- Arnaldo

BTF encoder:

- Inject kfunc decl tags into BTF from the BTF IDs ELF section in the Linux
  kernel vmlinux file.

  This allows tools such as bpftools and pfunct to enumerate the available kfuncs
  and to gets its function signature, the type of its return and of its
  arguments. See the example in the BTF loader changes description, below.

- Support parallel reproducible builds, where it doesn't matter how many
  threads are used, the end BTF encoding result is the same.

- Sanitize unsupported DWARF int type with greater-than-16 byte, as BTF doesn't
  support it.

BTF loader:

- Initial support for BTF_KIND_DECL_TAG:

  $ pfunct --prototypes -F btf vmlinux.btf.decl_tag,decl_tag_kfuncs | grep ^bpf_kfunc | head
  bpf_kfunc void cubictcp_init(struct sock * sk);
  bpf_kfunc void cubictcp_cwnd_event(struct sock * sk, enum tcp_ca_event event);
  bpf_kfunc void cubictcp_cong_avoid(struct sock * sk, u32 ack, u32 acked);
  bpf_kfunc u32 cubictcp_recalc_ssthresh(struct sock * sk);
  bpf_kfunc void cubictcp_state(struct sock * sk, u8 new_state);
  bpf_kfunc void cubictcp_acked(struct sock * sk, const struct ack_sample  * sample);
  bpf_kfunc int bpf_iter_css_new(struct bpf_iter_css * it, struct cgroup_subsys_state * start, unsigned int flags);
  bpf_kfunc struct cgroup_subsys_state * bpf_iter_css_next(struct bpf_iter_css * it);
  bpf_kfunc void bpf_iter_css_destroy(struct bpf_iter_css * it);
  bpf_kfunc s64 bpf_map_sum_elem_count(const struct bpf_map  * map);
  $ pfunct --prototypes -F btf vmlinux.btf.decl_tag,decl_tag_kfuncs | grep ^bpf_kfunc | wc -l
  116
  $

pretty printing:

- Fix hole discovery with inheritance in C++.

Re: ANNOUNCE: pahole v1.27 (reproducible builds, BTF kfuncs)

[Dominique Martinet]

Hi,

Arnaldo Carvalho de Melo wrote on Tue, Jun 11, 2024 at 06:26:53PM -0300:
> 	The v1.27 release of pahole and its friends is out, supporting
> parallel reproducible builds and encoding kernel kfuncs in BTF, allowing
> tools such as bpftrace to enumerate the available kfuncs and obtain its
> function signatures and return types.
> 
> Main git repo:
> 
>    https://git.kernel.org/pub/scm/devel/pahole/pahole.git

It looks like the v1.27 tag has not been pushed to the git repos (either
this or github), we're using git snapshots for nixpkgs, so it'd be great
if a tag could be pushed out.
(I think some release monitoring tools left and right also use tags,
even if that's less important if you Cc other distro maintainers... I
just happened to see the mail on bpf@vger.)

Thanks,
-- 
Dominique Martinet | Asmadeus

Re: ANNOUNCE: pahole v1.27 (reproducible builds, BTF kfuncs)

[Arnaldo Carvalho de Melo]

On Wed, Jun 12, 2024 at 06:38:02AM +0900, Dominique Martinet wrote:
> It looks like the v1.27 tag has not been pushed to the git repos (either
> this or github), we're using git snapshots for nixpkgs, so it'd be great
> if a tag could be pushed out.

Done.

https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tag/?h=v1.27
https://github.com/acmel/dwarves/releases/tag/v1.27

> (I think some release monitoring tools left and right also use tags,
> even if that's less important if you Cc other distro maintainers... I
> just happened to see the mail on bpf@vger.)

May I add your e-mail here:

acme@x1:~/git/pahole$ cat PKG-MAINTAINERS 
# Please let me know if I should remove/update/add more distro package maintainers here
# I'm keeping this so that I CC them when releasing new versions, thanks!

Jan Alexander Steffens <heftig@archlinux.org>
Domenico Andreoli <cavok@debian.org>
Matthias Schwarzott <zzam@gentoo.org>
Dominique Leuenberger <dimstar@opensuse.org>
acme@x1:~/git/pahole$

So that on the next release I CC you?

Thanks for reporting!

- Arnaldo

Re: ANNOUNCE: pahole v1.27 (reproducible builds, BTF kfuncs)

[Dominique Martinet]

Arnaldo Carvalho de Melo wrote on Tue, Jun 11, 2024 at 07:52:12PM -0300:
> On Wed, Jun 12, 2024 at 06:38:02AM +0900, Dominique Martinet wrote:
> > It looks like the v1.27 tag has not been pushed to the git repos (either
> > this or github), we're using git snapshots for nixpkgs, so it'd be great
> > if a tag could be pushed out.
> 
> Done.
> 
> https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tag/?h=v1.27
> https://github.com/acmel/dwarves/releases/tag/v1.27

Thank you!

> > (I think some release monitoring tools left and right also use tags,
> > even if that's less important if you Cc other distro maintainers... I
> > just happened to see the mail on bpf@vger.)
> 
> May I add your e-mail here:
> 
> acme@x1:~/git/pahole$ cat PKG-MAINTAINERS 
[..]
> 
> So that on the next release I CC you?

Sure; in practice this shouldn't change much as package update is
semi-automated within a few days, but getting a heads up cannot hurt.

-- 
Dominique Martinet | Asmadeus

Re: ANNOUNCE: pahole v1.27 (reproducible builds, BTF kfuncs)

[Matthias Schwarzott]

Am 11.06.24 um 23:26 schrieb Arnaldo Carvalho de Melo:
> Hi,
>   
> 	The v1.27 release of pahole and its friends is out, supporting
> parallel reproducible builds and encoding kernel kfuncs in BTF, allowing
> tools such as bpftrace to enumerate the available kfuncs and obtain its
> function signatures and return types.
> 

Regarding packaging of pahole:
What is the state of the contained ostra-cg?
I have no clue what it is and how to use it. Is there still a use-case 
for it?

Starting it without arguments only shows the usage string.
Running it with two dummy arguments:
$ ostra-cg x y
Traceback (most recent call last):
   File "/usr/bin/ostra-cg", line 404, in <module>
     class_def = ostra.class_definition(class_def_file = "%s.fields" % 
traced_class,
 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/share/dwarves/runtime/python/ostra.py", line 154, in __init__
     f = file(class_def_file)
         ^^^^
NameError: name 'file' is not defined. Did you mean: 'field'?

According to 
https://stackoverflow.com/questions/32131230/python-file-function the 
function file() does not exist in python3.

This part could be fixed by replacing it with open() but I wonder if 
this is worth it.

As nobody has complained about it being broken:
Should ostra just be removed?

Regards
Matthias

Re: ANNOUNCE: pahole v1.27 (reproducible builds, BTF kfuncs)

[Arnaldo Carvalho de Melo]

On Wed, Jun 12, 2024 at 12:07:09PM +0200, Matthias Schwarzott wrote:
> Am 11.06.24 um 23:26 schrieb Arnaldo Carvalho de Melo:
> > Hi,
> > 	The v1.27 release of pahole and its friends is out, supporting
> > parallel reproducible builds and encoding kernel kfuncs in BTF, allowing
> > tools such as bpftrace to enumerate the available kfuncs and obtain its
> > function signatures and return types.
> > 
> 
> Regarding packaging of pahole:
> What is the state of the contained ostra-cg?

I need to make a decision on that, it is used to produce things like:

http://vger.kernel.org/~acme/ostra/callgraphs/sock/0xf61bf500/

As documented in:

https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/README.ctracer

But yes, it needs to get retested after all these years to see how
difficult it would be to try and get it back working.

- Arnaldo

> I have no clue what it is and how to use it. Is there still a use-case for
> it?
> 
> Starting it without arguments only shows the usage string.
> Running it with two dummy arguments:
> $ ostra-cg x y
> Traceback (most recent call last):
>   File "/usr/bin/ostra-cg", line 404, in <module>
>     class_def = ostra.class_definition(class_def_file = "%s.fields" %
> traced_class,
> 
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/usr/share/dwarves/runtime/python/ostra.py", line 154, in __init__
>     f = file(class_def_file)
>         ^^^^
> NameError: name 'file' is not defined. Did you mean: 'field'?
> 
> According to
> https://stackoverflow.com/questions/32131230/python-file-function the
> function file() does not exist in python3.
> 
> This part could be fixed by replacing it with open() but I wonder if this is
> worth it.
> 
> As nobody has complained about it being broken:
> Should ostra just be removed?
> 
> Regards
> Matthias

Re: ANNOUNCE: pahole v1.27 (reproducible builds, BTF kfuncs)

[Nathan Chancellor]

Hi Arnaldo,

On Tue, Jun 11, 2024 at 06:26:53PM -0300, Arnaldo Carvalho de Melo wrote:
> 	The v1.27 release of pahole and its friends is out, supporting
> parallel reproducible builds and encoding kernel kfuncs in BTF, allowing
> tools such as bpftrace to enumerate the available kfuncs and obtain its
> function signatures and return types.

After commit f632e75 ("dwarf_loader: Add the cu to the cus list early,
remove on LSK_DELETE"), I (and others[1]) notice a crash when running
pahole on modules built with Clang when CONFIG_LTO_CLANG is enabled:

  $ curl -LSso .config https://gitlab.archlinux.org/archlinux/packaging/packages/linux/-/raw/main/config

  $ scripts/config -d LTO_NONE -e LTO_CLANG_THIN

  $ make -skj"$(nproc)" ARCH=x86_64 LLVM=1 olddefconfig vmlinux crypto/cast_common.ko
  make[3]: *** [scripts/Makefile.modfinal:59: crypto/cast_common.ko] Error 139

I've isolated this to the following commands using the files available
at [2] (these were built with LLVM 18 but I could reproduce it with LLVM
17 and LLVM 19, so it appears to impact a number of versions):

  $ tar -tf clang-lto-pahole-1.27-crash.tar.zst
  clang-lto-pahole-1.27-crash/
  clang-lto-pahole-1.27-crash/cast_common.mod.o
  clang-lto-pahole-1.27-crash/module.lds
  clang-lto-pahole-1.27-crash/cast_common.o
  clang-lto-pahole-1.27-crash/cast_common.ko.bak
  clang-lto-pahole-1.27-crash/vmlinux
  clang-lto-pahole-1.27-crash/cast_common.ko

  $ tar -axf clang-lto-pahole-1.27-crash.tar.zst

  $ cd clang-lto-pahole-1.27-crash

  $ LLVM_OBJCOPY="llvm-objcopy" pahole-1.26 -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func --lang_exclude=rust --btf_base vmlinux cast_common.ko

  $ cp cast_common.ko{.bak,}

  $ LLVM_OBJCOPY="llvm-objcopy" pahole-1.27 -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func --lang_exclude=rust --btf_base vmlinux cast_common.ko
  fish: Job 1, '...' terminated by signal SIGSEGV (Address boundary error)

If there is any more information I can provide or patches I can test, I
am more than happy to do so.

[1]: https://gitlab.archlinux.org/archlinux/packaging/packages/pahole/-/issues/1
[2]: https://1drv.ms/u/s!AsQNYeB-IEbqqC2F28JuLy__Q7Vd?e=KsraMU

Cheers,
Nathan

Re: ANNOUNCE: pahole v1.27 (reproducible builds, BTF kfuncs)

[Arnaldo Carvalho de Melo]

On Thu, Jun 13, 2024 at 02:40:19PM -0700, Nathan Chancellor wrote:
> Hi Arnaldo,
> 
> On Tue, Jun 11, 2024 at 06:26:53PM -0300, Arnaldo Carvalho de Melo wrote:
> > 	The v1.27 release of pahole and its friends is out, supporting
> > parallel reproducible builds and encoding kernel kfuncs in BTF, allowing
> > tools such as bpftrace to enumerate the available kfuncs and obtain its
> > function signatures and return types.
> 
> After commit f632e75 ("dwarf_loader: Add the cu to the cus list early,
> remove on LSK_DELETE"), I (and others[1]) notice a crash when running
> pahole on modules built with Clang when CONFIG_LTO_CLANG is enabled:
> 
>   $ curl -LSso .config https://gitlab.archlinux.org/archlinux/packaging/packages/linux/-/raw/main/config
> 
>   $ scripts/config -d LTO_NONE -e LTO_CLANG_THIN
> 
>   $ make -skj"$(nproc)" ARCH=x86_64 LLVM=1 olddefconfig vmlinux crypto/cast_common.ko
>   make[3]: *** [scripts/Makefile.modfinal:59: crypto/cast_common.ko] Error 139
> 
> I've isolated this to the following commands using the files available
> at [2] (these were built with LLVM 18 but I could reproduce it with LLVM
> 17 and LLVM 19, so it appears to impact a number of versions):
> 
>   $ tar -tf clang-lto-pahole-1.27-crash.tar.zst
>   clang-lto-pahole-1.27-crash/
>   clang-lto-pahole-1.27-crash/cast_common.mod.o
>   clang-lto-pahole-1.27-crash/module.lds
>   clang-lto-pahole-1.27-crash/cast_common.o
>   clang-lto-pahole-1.27-crash/cast_common.ko.bak
>   clang-lto-pahole-1.27-crash/vmlinux
>   clang-lto-pahole-1.27-crash/cast_common.ko
> 
>   $ tar -axf clang-lto-pahole-1.27-crash.tar.zst
> 
>   $ cd clang-lto-pahole-1.27-crash
> 
>   $ LLVM_OBJCOPY="llvm-objcopy" pahole-1.26 -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func --lang_exclude=rust --btf_base vmlinux cast_common.ko
> 
>   $ cp cast_common.ko{.bak,}
> 
>   $ LLVM_OBJCOPY="llvm-objcopy" pahole-1.27 -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func --lang_exclude=rust --btf_base vmlinux cast_common.ko
>   fish: Job 1, '...' terminated by signal SIGSEGV (Address boundary error)
> 
> If there is any more information I can provide or patches I can test, I
> am more than happy to do so.

I reproduced the problem by just running 'pahole cast_common.ko", so
this isn't even related to the BTF parts, its about the DWARF loader,
I'm on it, thanks for the detailed report and for providing the files.

- Arnaldo
 
> [1]: https://gitlab.archlinux.org/archlinux/packaging/packages/pahole/-/issues/1
> [2]: https://1drv.ms/u/s!AsQNYeB-IEbqqC2F28JuLy__Q7Vd?e=KsraMU
> 
> Cheers,
> Nathan
> 

[PATCH/RFT] Re: ANNOUNCE: pahole v1.27 (reproducible builds, BTF kfuncs)

[Arnaldo Carvalho de Melo]

On Mon, Jun 17, 2024 at 04:39:40PM -0300, Arnaldo Carvalho de Melo wrote:
> On Thu, Jun 13, 2024 at 02:40:19PM -0700, Nathan Chancellor wrote:
> > On Tue, Jun 11, 2024 at 06:26:53PM -0300, Arnaldo Carvalho de Melo wrote:
> > > 	The v1.27 release of pahole and its friends is out, supporting
> > > parallel reproducible builds and encoding kernel kfuncs in BTF, allowing
> > > tools such as bpftrace to enumerate the available kfuncs and obtain its
> > > function signatures and return types.
> > 
> > After commit f632e75 ("dwarf_loader: Add the cu to the cus list early,
> > remove on LSK_DELETE"), I (and others[1]) notice a crash when running
> > pahole on modules built with Clang when CONFIG_LTO_CLANG is enabled:
> > 
> >   $ curl -LSso .config https://gitlab.archlinux.org/archlinux/packaging/packages/linux/-/raw/main/config
> > 
> >   $ scripts/config -d LTO_NONE -e LTO_CLANG_THIN
> > 
> >   $ make -skj"$(nproc)" ARCH=x86_64 LLVM=1 olddefconfig vmlinux crypto/cast_common.ko
> >   make[3]: *** [scripts/Makefile.modfinal:59: crypto/cast_common.ko] Error 139
> > 
> > I've isolated this to the following commands using the files available
> > at [2] (these were built with LLVM 18 but I could reproduce it with LLVM
> > 17 and LLVM 19, so it appears to impact a number of versions):
> > 
> >   $ tar -tf clang-lto-pahole-1.27-crash.tar.zst
> >   clang-lto-pahole-1.27-crash/
> >   clang-lto-pahole-1.27-crash/cast_common.mod.o
> >   clang-lto-pahole-1.27-crash/module.lds
> >   clang-lto-pahole-1.27-crash/cast_common.o
> >   clang-lto-pahole-1.27-crash/cast_common.ko.bak
> >   clang-lto-pahole-1.27-crash/vmlinux
> >   clang-lto-pahole-1.27-crash/cast_common.ko
> > 
> >   $ tar -axf clang-lto-pahole-1.27-crash.tar.zst
> > 
> >   $ cd clang-lto-pahole-1.27-crash
> > 
> >   $ LLVM_OBJCOPY="llvm-objcopy" pahole-1.26 -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func --lang_exclude=rust --btf_base vmlinux cast_common.ko
> > 
> >   $ cp cast_common.ko{.bak,}
> > 
> >   $ LLVM_OBJCOPY="llvm-objcopy" pahole-1.27 -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func --lang_exclude=rust --btf_base vmlinux cast_common.ko
> >   fish: Job 1, '...' terminated by signal SIGSEGV (Address boundary error)
> > 
> > If there is any more information I can provide or patches I can test, I
> > am more than happy to do so.
> 
> I reproduced the problem by just running 'pahole cast_common.ko", so
> this isn't even related to the BTF parts, its about the DWARF loader,
> I'm on it, thanks for the detailed report and for providing the files.

One liner + explanation at the end, with it both DWARF and BTF loaders
work:

⬢[acme@toolbox clang-lto-pahole-1.27-crash]$ pahole -F btf -C task_struct cast_common.ko | tail
	/* XXX last struct has 1 hole, 1 bit hole */

	/* size: 11584, cachelines: 181, members: 265 */
	/* sum members: 11483, holes: 22, sum holes: 85 */
	/* sum bitfield members: 82 bits, bit holes: 2, sum bit holes: 46 bits */
	/* member types with holes: 4, total: 6, bit holes: 2, total: 2 */
	/* paddings: 6, sum paddings: 49 */
	/* forced alignments: 1, forced holes: 1, sum forced holes: 8 */
};

⬢[acme@toolbox clang-lto-pahole-1.27-crash]$ pahole -F dwarf -C task_struct cast_common.ko | tail
	/* XXX last struct has 1 hole, 1 bit hole */

	/* size: 11584, cachelines: 181, members: 265 */
	/* sum members: 11483, holes: 22, sum holes: 85 */
	/* sum bitfield members: 82 bits, bit holes: 2, sum bit holes: 46 bits */
	/* member types with holes: 4, total: 6, bit holes: 2, total: 2 */
	/* paddings: 6, sum paddings: 49 */
	/* forced alignments: 8, forced holes: 1, sum forced holes: 8 */
};

⬢[acme@toolbox clang-lto-pahole-1.27-crash]$

Also the BTF encoder (that uses the DWARF loader), etc.

⬢[acme@toolbox clang-lto-pahole-1.27-crash]$ pahole -j --btf_encode cast_common.ko
⬢[acme@toolbox clang-lto-pahole-1.27-crash]$ readelf -SW cast_common.ko | grep BTF
  [30] .BTF              PROGBITS        0000000000000000 0223c9 01172c 00      0   0  1
⬢[acme@toolbox clang-lto-pahole-1.27-crash]$ pahole -F btf cast_common.ko | wc -l
4302
⬢[acme@toolbox clang-lto-pahole-1.27-crash]$ set -o vi
⬢[acme@toolbox clang-lto-pahole-1.27-crash]$ pahole -F btf cast_common.ko | head
struct elf32_note {
	Elf32_Word                 n_namesz;             /*     0     4 */
	Elf32_Word                 n_descsz;             /*     4     4 */
	Elf32_Word                 n_type;               /*     8     4 */

	/* size: 12, cachelines: 1, members: 3 */
	/* last cacheline: 12 bytes */
};
struct module {
	enum module_state          state;                /*     0     4 */
⬢[acme@toolbox clang-lto-pahole-1.27-crash]$ pahole -F btf cast_common.ko | tail
	const char  *              mod_name;             /*    24     8 */
	const char  * *            class_names;          /*    32     8 */
	const int                  length;               /*    40     4 */
	const int                  base;                 /*    44     4 */
	enum class_map_type        map_type;             /*    48     4 */

	/* size: 56, cachelines: 1, members: 7 */
	/* padding: 4 */
	/* last cacheline: 56 bytes */
};
⬢[acme@toolbox clang-lto-pahole-1.27-crash]$

Can you try with the one liner below? We remove it from the cus list
unconditionally, and since we alloc space with zalloc/calloc in
cu__new() and missed initializing that list_head (cu->node) we ended up
hitting list_del with a zeroed 'struct list_head' :-\

I'll try and get this cast_common.ko checked into a test repo for pahole
so that this gets regression tested.

Please test this patch so that we see if this is the only problem and
your kernel build with clang completes successfully.

Thanks,

- Arnaldo

diff --git a/dwarves.c b/dwarves.c
index 1ec259f50dbd3778..823a01524a12bb37 100644
--- a/dwarves.c
+++ b/dwarves.c
@@ -739,6 +739,7 @@ struct cu *cu__new(const char *name, uint8_t addr_size,
 		cu->dfops	= NULL;
 		INIT_LIST_HEAD(&cu->tags);
 		INIT_LIST_HEAD(&cu->tool_list);
+		INIT_LIST_HEAD(&cu->node);
 
 		cu->addr_size = addr_size;
 		cu->extra_dbg_info = 0;

Re: [PATCH/RFT] Re: ANNOUNCE: pahole v1.27 (reproducible builds, BTF kfuncs)

[Nathan Chancellor]

On Mon, Jun 17, 2024 at 05:02:12PM -0300, Arnaldo Carvalho de Melo wrote:
> Can you try with the one liner below? We remove it from the cus list
> unconditionally, and since we alloc space with zalloc/calloc in
> cu__new() and missed initializing that list_head (cu->node) we ended up
> hitting list_del with a zeroed 'struct list_head' :-\
> 
> I'll try and get this cast_common.ko checked into a test repo for pahole
> so that this gets regression tested.
> 
> Please test this patch so that we see if this is the only problem and
> your kernel build with clang completes successfully.

Thanks, I rebuilt pahole with the following diff and both my build and
the other configuration I tested for this regression successfully
complete.

Tested-by: Nathan Chancellor <nathan@kernel.org>

> diff --git a/dwarves.c b/dwarves.c
> index 1ec259f50dbd3778..823a01524a12bb37 100644
> --- a/dwarves.c
> +++ b/dwarves.c
> @@ -739,6 +739,7 @@ struct cu *cu__new(const char *name, uint8_t addr_size,
>  		cu->dfops	= NULL;
>  		INIT_LIST_HEAD(&cu->tags);
>  		INIT_LIST_HEAD(&cu->tool_list);
> +		INIT_LIST_HEAD(&cu->node);
>  
>  		cu->addr_size = addr_size;
>  		cu->extra_dbg_info = 0;

Re: [PATCH/RFT] Re: ANNOUNCE: pahole v1.27 (reproducible builds, BTF kfuncs)

[Arnaldo Carvalho de Melo]

On Mon, Jun 17, 2024 at 02:08:10PM -0700, Nathan Chancellor wrote:
> On Mon, Jun 17, 2024 at 05:02:12PM -0300, Arnaldo Carvalho de Melo wrote:
> > Can you try with the one liner below? We remove it from the cus list
> > unconditionally, and since we alloc space with zalloc/calloc in
> > cu__new() and missed initializing that list_head (cu->node) we ended up
> > hitting list_del with a zeroed 'struct list_head' :-\
> > 
> > I'll try and get this cast_common.ko checked into a test repo for pahole
> > so that this gets regression tested.
> > 
> > Please test this patch so that we see if this is the only problem and
> > your kernel build with clang completes successfully.
> 
> Thanks, I rebuilt pahole with the following diff and both my build and
> the other configuration I tested for this regression successfully
> complete.
> 
> Tested-by: Nathan Chancellor <nathan@kernel.org>

Great, I just added this:

From 6a2b27c0f512619b0e7a769a18a0fb05bb3789a5 Mon Sep 17 00:00:00 2001
From: Arnaldo Carvalho de Melo <acme@redhat.com>
Date: Tue, 18 Jun 2024 10:37:30 -0300
Subject: [PATCH 1/1] core: Initialize cu->node with INIT_LIST_HEAD()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

In cu__new() zalloc() is used defensively, and that helped catch this
problem where we assume that a cu us in the cus list of cu instances,
but that is not the case when we use cus__merge_and_process_cu(), for
instance when loading files created by clang with LTO, as reported by
Peter Jung and narrowed down by Nathan Chancellor.

If we use INIT_LIST_HEAD() in cu__new() to initialize cu->node, which is
what we do with other lists and nodes there, then the unconditional
removal using list_del_init() will be a no-op and removing something not
on the cus list of cu instances will not cause problems, just keep an
unconsistent cus->nr_entries field.

So lets just have this fix in first, keeping Nathan's Tested-by and then
do the a bit more involved fix of either adding that cu to the cus list
or checking at removal time if it is there.

  Program received signal SIGSEGV, Segmentation fault.
  0x00007ffff7f1e13e in __list_del (prev=0x0, next=0x0) at /home/acme/git/pahole/list.h:106
  106		next->prev = prev;
  (gdb) bt
  #0  0x00007ffff7f1e13e in __list_del (prev=0x0, next=0x0) at /home/acme/git/pahole/list.h:106
  #1  0x00007ffff7f1e176 in list_del_init (entry=0x417980) at /home/acme/git/pahole/list.h:165
  #2  0x00007ffff7f1f8f9 in __cus__remove (cus=0x4142a0, cu=0x417980) at /home/acme/git/pahole/dwarves.c:527
  #3  0x00007ffff7f1f92b in cus__remove (cus=0x4142a0, cu=0x417980) at /home/acme/git/pahole/dwarves.c:533
  #4  0x00007ffff7f3d01c in cus__finalize (cus=0x4142a0, cu=0x417980, conf=0x4133c0 <conf_load>, thr_data=0x0)
      at /home/acme/git/pahole/dwarf_loader.c:3040
  #5  0x00007ffff7f3e05c in cus__merge_and_process_cu (cus=0x4142a0, conf=0x4133c0 <conf_load>, mod=0x415cf0, dw=0x416110, elf=0x414380,
      filename=0x7fffffffe3f7 "cast_common.ko", build_id=0x416680 "\265D\371U\213\373u|\037\250\242\032\271\365⒜]y\023", build_id_len=20,
      type_dcu=0x0) at /home/acme/git/pahole/dwarf_loader.c:3482
  #6  0x00007ffff7f3e218 in cus__load_module (cus=0x4142a0, conf=0x4133c0 <conf_load>, mod=0x415cf0, dw=0x416110, elf=0x414380,
      filename=0x7fffffffe3f7 "cast_common.ko") at /home/acme/git/pahole/dwarf_loader.c:3521
  #7  0x00007ffff7f3e396 in cus__process_dwflmod (dwflmod=0x415cf0, userdata=0x415d00, name=0x415ea0 "cast_common.ko", base=65536,
      arg=0x7fffffffde40) at /home/acme/git/pahole/dwarf_loader.c:3581
  #8  0x00007ffff7eb4609 in dwfl_getmodules (dwfl=0x414300, callback=0x7ffff7f3e2ec <cus__process_dwflmod>, arg=0x7fffffffde40, offset=0)
      at ../libdwfl/dwfl_getmodules.c:86
  #9  0x00007ffff7f3e4c5 in cus__process_file (cus=0x4142a0, conf=0x4133c0 <conf_load>, fd=3, filename=0x7fffffffe3f7 "cast_common.ko")
      at /home/acme/git/pahole/dwarf_loader.c:3647
  #10 0x00007ffff7f3e5cd in dwarf__load_file (cus=0x4142a0, conf=0x4133c0 <conf_load>, filename=0x7fffffffe3f7 "cast_common.ko")
      at /home/acme/git/pahole/dwarf_loader.c:3684
  #11 0x00007ffff7f232df in cus__load_file (cus=0x4142a0, conf=0x4133c0 <conf_load>, filename=0x7fffffffe3f7 "cast_common.ko")
      at /home/acme/git/pahole/dwarves.c:2134
  #12 0x00007ffff7f23e8b in cus__load_files (cus=0x4142a0, conf=0x4133c0 <conf_load>, filenames=0x7fffffffe0f0)
      at /home/acme/git/pahole/dwarves.c:2637
  #13 0x000000000040aec0 in main (argc=2, argv=0x7fffffffe0e8) at /home/acme/git/pahole/pahole.c:3805
  (gdb) fr 1
  #1  0x00007ffff7f1e176 in list_del_init (entry=0x417980) at /home/acme/git/pahole/list.h:165
  165		__list_del(entry->prev, entry->next);
  (gdb) p entry
  $1 = (struct list_head *) 0x417980
  (gdb) p entry->next
  $2 = (struct list_head *) 0x0
  (gdb) p entry->prev
  $3 = (struct list_head *) 0x0

Closes: https://github.com/acmel/dwarves/issues/53
Closes: https://gitlab.archlinux.org/archlinux/packaging/packages/pahole/-/issues/1
Tested-by: Nathan Chancellor <nathan@kernel.org>
Link: https://lore.kernel.org/all/20240617210810.GA1877676@thelio-3990X
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 dwarves.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/dwarves.c b/dwarves.c
index 1ec259f50dbd3778..823a01524a12bb37 100644
--- a/dwarves.c
+++ b/dwarves.c
@@ -739,6 +739,7 @@ struct cu *cu__new(const char *name, uint8_t addr_size,
 		cu->dfops	= NULL;
 		INIT_LIST_HEAD(&cu->tags);
 		INIT_LIST_HEAD(&cu->tool_list);
+		INIT_LIST_HEAD(&cu->node);
 
 		cu->addr_size = addr_size;
 		cu->extra_dbg_info = 0;
-- 
2.45.0

[PATCH fyi 1/1] dwarf_loader: Add missing cus__add(cus, cu) to cus__merge_and_process_cu()

[Arnaldo Carvalho de Melo]

Just FYI, I added this on top of the previous one initializing cu->node.

Tested with that cast_common.ko and the vmlinux you made available.

- Arnaldo

---

In cus__finalize() if cu__finalize() returns LSK__DELETE, i.e. if the
tool processing the cu is done with it, we will assume that it is in the
cus list of cu instances, remove it and then delete it.

This was not being done by cus__merge_and_process_cu(), used when
merging all DWARF CUs into a single 'struct cu', such as when processing
binaries generated by clang using LTO. Add the missing cus__add() to
keep cus->nr_entries consistent.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 dwarf_loader.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/dwarf_loader.c b/dwarf_loader.c
index b832c93cc2194eaf..3caf32457f42ec2e 100644
--- a/dwarf_loader.c
+++ b/dwarf_loader.c
@@ -3452,6 +3452,7 @@ static int cus__merge_and_process_cu(struct cus *cus, struct conf_load *conf,
 			cu->priv = dcu;
 			cu->dfops = &dwarf__ops;
 			cu->language = attr_numeric(cu_die, DW_AT_language);
+			cus__add(cus, cu);
 		}
 
 		Dwarf_Die child;
-- 
2.45.0

Re: [PATCH/RFT] Re: ANNOUNCE: pahole v1.27 (reproducible builds, BTF kfuncs)

[Nathan Chancellor]

Hi Arnaldo,

On Tue, Jun 18, 2024 at 10:51:44AM -0300, Arnaldo Carvalho de Melo wrote:
> >From 6a2b27c0f512619b0e7a769a18a0fb05bb3789a5 Mon Sep 17 00:00:00 2001
> From: Arnaldo Carvalho de Melo <acme@redhat.com>
> Date: Tue, 18 Jun 2024 10:37:30 -0300
> Subject: [PATCH 1/1] core: Initialize cu->node with INIT_LIST_HEAD()

Could a new release be cut for this issue? Several people have been bit
by this (including CI systems) and two distributions have talked about
backporting this change on top of 1.27 to resolve it:

https://gitlab.archlinux.org/archlinux/packaging/packages/pahole/-/issues/1
https://src.fedoraproject.org/rpms/dwarves/pull-request/4

Cheers,
Nathan