From: Jiri Olsa <olsajiri@gmail.com>
To: Ihor Solodrai <ihor.solodrai@pm.me>
Cc: dwarves@vger.kernel.org, bpf@vger.kernel.org, acme@kernel.org,
alan.maguire@oracle.com, eddyz87@gmail.com, andrii@kernel.org,
mykolal@fb.com, olsajiri@gmail.com
Subject: Re: [PATCH dwarves v4 RESEND 00/10] pahole: faster reproducible BTF encoding
Date: Thu, 9 Jan 2025 23:03:31 +0100 [thread overview]
Message-ID: <Z4BHs_rp6Ts2Dj5O@krava> (raw)
In-Reply-To: <20250109185950.653110-1-ihor.solodrai@pm.me>
On Thu, Jan 09, 2025 at 06:59:54PM +0000, Ihor Solodrai wrote:
> Note: a resend due to https://lore.kernel.org/dwarves/Z4AWJBNsGJvBU7ZY@x1/
>
> This is v4 of the series aiming to speed up parallel reproducible BTF
> encoding. This version mostly addresses feedback from Jiri Olsa on v3.
>
> A notable adition is a patch 10/10, which changes func_states in
> btf_encoder from a list to an array.
>
> Testing:
>
> vmlinux=/home/theihor/kernels/bpf-next/kbuild-output/.tmp_vmlinux1 PATH=$(realpath build):$PATH ./tests/tests
> 1: Validation of BTF encoding of functions; this may take some time: Ok
> 2: Default BTF on a system without BTF: Ok
> 3: Flexible arrays accounting: pahole: type 'nft_pipapo_elem' not found
> pahole: type 'tls_rec' not found
> pahole: type 'fuse_direntplus' not found
> pahole: type 'nft_rhash_elem' not found
> pahole: type 'nft_hash_elem' not found
> pahole: type 'nft_bitmap_elem' not found
> pahole: type 'ipt_standard' not found
> pahole: type 'nft_rule_dp_last' not found
> pahole: type 'ip6t_standard' not found
> pahole: type 'ipt_error' not found
> pahole: type 'ip6t_error' not found
> pahole: type 'nft_rbtree_elem' not found
> Ok
> 4: Check that pfunct can print btf_decl_tags read from BTF: Ok
> 5: Pretty printing of files using DWARF type information: Ok
> 6: Parallel reproducible DWARF Loading/Serial BTF encoding: Ok
>
> The warnings about not found types are also present at pahole/next, so
> not related to this patchset.
>
>
> Performance check. This patchset (always reproducible):
>
> Performance counter stats for '/home/theihor/dev/dwarves/build/pahole -J -j24 --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func,decl_tag_kfuncs,reproducible_build --btf_encode_detached=/dev/null --lang_exclude=rust /home/theihor/kernels/bpf-next/kbuild-output/.tmp_vmlinux1' (13 runs):
>
> 5,788.22 msec cpu-clock:u # 3.776 CPUs utilized ( +- 0.17% )
>
> 1.53288 +- 0.00334 seconds time elapsed ( +- 0.22% )
>
>
> pahole/next (d444eb6), parallel non-reproducible:
>
> Performance counter stats for '/home/theihor/dev/dwarves/build/pahole -J -j24 --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func,decl_tag_kfuncs --btf_encode_detached=/dev/null --lang_exclude=rust /home/theihor/kernels/bpf-next/kbuild-output/.tmp_vmlinux1' (13 runs):
>
> 10,462.38 msec cpu-clock:u # 6.678 CPUs utilized ( +- 0.15% )
>
> 1.56670 +- 0.00548 seconds time elapsed ( +- 0.35% )
>
>
> pahole/next (d444eb6), parallel reproducible:
>
> Performance counter stats for '/home/theihor/dev/dwarves/build/pahole -J -j24 --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func,decl_tag_kfuncs,reproducible_build --btf_encode_detached=/dev/null --lang_exclude=rust /home/theihor/kernels/bpf-next/kbuild-output/.tmp_vmlinux1' (13 runs):
>
> 6,399.88 msec cpu-clock:u # 3.164 CPUs utilized ( +- 0.22% )
>
> 2.02269 +- 0.00359 seconds time elapsed ( +- 0.18% )
>
>
> v3: https://lore.kernel.org/dwarves/20241221012245.243845-1-ihor.solodrai@pm.me/
> v2: https://lore.kernel.org/dwarves/20241213223641.564002-1-ihor.solodrai@pm.me/
> v1: https://lore.kernel.org/dwarves/20241128012341.4081072-1-ihor.solodrai@pm.me/
>
> Alan Maguire (2):
> btf_encoder: simplify function encoding
> btf_encoder: separate elf function, saved function representations
>
> Ihor Solodrai (8):
> btf_encoder: free encoder->secinfo in btf_encoder__delete
> btf_encoder: introduce elf_functions struct type
> btf_encoder: introduce elf_functions_list
> btf_encoder: remove skip_encoding_inconsistent_proto
> dwarf_loader: introduce cu->id
> dwarf_loader: multithreading with a job/worker model
> btf_encoder: clean up global encoders list
> btf_encoder: switch func_states from a list to an array
Acked-by: Jiri Olsa <jolsa@kernel.org>
thanks,
jirka
>
> btf_encoder.c | 662 +++++++++++++++++++-----------------
> btf_encoder.h | 7 +-
> btf_loader.c | 2 +-
> ctf_loader.c | 2 +-
> dwarf_loader.c | 335 ++++++++++++------
> dwarves.c | 44 ---
> dwarves.h | 20 +-
> pahole.c | 230 ++-----------
> pdwtags.c | 3 +-
> pfunct.c | 3 +-
> tests/reproducible_build.sh | 5 +-
> 11 files changed, 623 insertions(+), 690 deletions(-)
>
> --
> 2.47.1
>
>
prev parent reply other threads:[~2025-01-09 22:03 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-01-09 18:59 [PATCH dwarves v4 RESEND 00/10] pahole: faster reproducible BTF encoding Ihor Solodrai
2025-01-09 18:59 ` [PATCH dwarves v4 RESEND 01/10] btf_encoder: simplify function encoding Ihor Solodrai
2025-01-09 19:00 ` [PATCH dwarves v4 RESEND 02/10] btf_encoder: free encoder->secinfo in btf_encoder__delete Ihor Solodrai
2025-01-09 19:00 ` [PATCH dwarves v4 RESEND 03/10] btf_encoder: separate elf function, saved function representations Ihor Solodrai
2025-01-09 19:00 ` [PATCH dwarves v4 RESEND 04/10] btf_encoder: introduce elf_functions struct type Ihor Solodrai
2025-01-09 19:00 ` [PATCH dwarves v4 RESEND 05/10] btf_encoder: introduce elf_functions_list Ihor Solodrai
2025-01-09 19:00 ` [PATCH dwarves v4 RESEND 06/10] btf_encoder: remove skip_encoding_inconsistent_proto Ihor Solodrai
2025-01-09 19:00 ` [PATCH dwarves v4 RESEND 07/10] dwarf_loader: introduce cu->id Ihor Solodrai
2025-01-09 19:00 ` [PATCH dwarves v4 RESEND 08/10] dwarf_loader: multithreading with a job/worker model Ihor Solodrai
2025-01-09 19:00 ` [PATCH dwarves v4 RESEND 09/10] btf_encoder: clean up global encoders list Ihor Solodrai
2025-01-09 19:00 ` [PATCH dwarves v4 RESEND 10/10] btf_encoder: switch func_states from a list to an array Ihor Solodrai
2025-01-09 22:03 ` Jiri Olsa [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Z4BHs_rp6Ts2Dj5O@krava \
--to=olsajiri@gmail.com \
--cc=acme@kernel.org \
--cc=alan.maguire@oracle.com \
--cc=andrii@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=dwarves@vger.kernel.org \
--cc=eddyz87@gmail.com \
--cc=ihor.solodrai@pm.me \
--cc=mykolal@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox