Re: [RFC PATCH 0/9] pahole: shared ELF and faster reproducible BTF encoding

[Arnaldo Carvalho de Melo <acme@kernel.org>]

On Fri, Dec 06, 2024 at 06:19:24PM +0000, Ihor Solodrai wrote:
> On Monday, December 2nd, 2024 at 5:55 AM, Jiri Olsa <olsajiri@gmail.com> wrote:
> 
> > 
> > 
> > On Thu, Nov 28, 2024 at 01:23:44AM +0000, Ihor Solodrai wrote:
> > 
> > SNIP
> > 
> > > Test results for this patch series:
> > > 
> > > 1: Validation of BTF encoding of functions; this may take some time: Ok
> > > 2: Default BTF on a system without BTF: Ok
> > > 3: Flexible arrays accounting: WARNING: still unsuported BTF_KIND_DECL_TAG(bpf_fastcall) for bpf_cast_to_kern_ctx already with attribute (bpf_kfunc), ignoring
> > > WARNING: still unsuported BTF_KIND_DECL_TAG(bpf_fastcall) for bpf_rdonly_cast already with attribute (bpf_kfunc), ignoring
> > > pahole: type 'nft_pipapo_elem' not found
> > > pahole: type 'ip6t_standard' not found
> > > pahole: type 'ip6t_error' not found
> > > pahole: type 'nft_rbtree_elem' not found
> > > pahole: type 'nft_rule_dp_last' not found
> > > pahole: type 'nft_bitmap_elem' not found
> > > pahole: type 'fuse_direntplus' not found
> > > pahole: type 'ipt_standard' not found
> > > pahole: type 'ipt_error' not found
> > > pahole: type 'tls_rec' not found
> > > pahole: type 'nft_rhash_elem' not found
> > > pahole: type 'nft_hash_elem' not found
> > > Ok
> > > 4: Pretty printing of files using DWARF type information: Ok
> > > 5: Parallel reproducible DWARF Loading/Serial BTF encoding: Ok
> > 
> > 
> > hi,
> > when trying selftests with this change, I'm getting wrong .BTF
> > on bpf selftest bpf_testmod.ko module
> > 
> > $ bpftool btf dump file ./bpf_testmod.ko
> > Error: failed to load BTF from ./bpf_testmod.ko: Invalid argument
> 
> Hi Jiri, thank you for testing.
> 
> I think the reason for this failure is that changes in the last patch
> of the series [1] don't handle correctly a situation when the number
> of CUs is lesser than the number of jobs. I was too focused on trying
> to speed up vmlinux encoding.
> 
> I am going to try implementing a clear queueing interface between
> dwarf_loader and pahole_stealer. Hopefully it will make it harder to
> introduce bugs like this.
> 
> I've started working on the v2 of this series which I hope to submit
> sometime next week.

Thanks for working on this! I'll try to join the reviewers team soon.

- Arnaldo
 
> [1] https://lore.kernel.org/dwarves/20241128012341.4081072-10-ihor.solodrai@pm.me/
> 
> > 
> > jirka
> > 
> > > Alan Maguire (3):
> > > btf_encoder: simplify function encoding
> > > btf_encoder: store,use section-relative addresses in ELF function
> > > representation
> > > btf_encoder: separate elf function, saved function representations
> > > 
> > > Ihor Solodrai (6):
> > > dwarf_loader: introduce pre_load_module hook to conf_load
> > > btf_encoder: introduce elf_functions struct type
> > > btf_encoder: collect elf_functions in btf_encoder__pre_load_module
> > > btf_encoder: switch to shared elf_functions table
> > > btf_encoder: introduce btf_encoding_context
> > > pahole: faster reproducible BTF encoding
> > > 
> > > btf_encoder.c | 661 ++++++++++++++++++++++++++++++-------------------
> > > btf_encoder.h | 6 +
> > > dwarf_loader.c | 18 +-
> > > dwarves.c | 47 ++--
> > > dwarves.h | 16 +-
> > > pahole.c | 265 +++++++++-----------
> > > 6 files changed, 567 insertions(+), 446 deletions(-)
> > > 
> > > --
> > > 2.47.0