From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Andrii Nakryiko <andrii@kernel.org>
Cc: Andrii Nakryiko <andrii@kernel.org>, Hao Luo <haoluo@google.com>,
Oleg Rombakh <olegrom@google.com>,
dwarves@vger.kernel.org, bpf@vger.kernel.org, kernel-team@fb.com,
ast@kernel.org, Tony Ambardar <tony.ambardar@gmail.com>,
Ilya Leoshkevich <iii@linux.ibm.com>,
David Marcinkovic <david.marcinkovic@sartura.hr>,
Luka Perkov <luka.perkov@sartura.hr>,
Borna Cafuk <borna.cafuk@sartura.hr>,
Juraj Vijtiuk <juraj.vijtiuk@sartura.hr>
Subject: Re: [PATCH v2 dwarves 0/8] Switch BTF loading and encoding to libbpf APIs
Date: Fri, 9 Oct 2020 13:22:43 -0300 [thread overview]
Message-ID: <20201009162243.GD322246@kernel.org> (raw)
In-Reply-To: <20201008234000.740660-1-andrii@kernel.org>
Em Thu, Oct 08, 2020 at 04:39:52PM -0700, Andrii Nakryiko escreveu:
> This patch set switches pahole to use libbpf-provided BTF loading and encoding
> APIs. This reduces pahole's own BTF encoding code, speeds up the process,
> reduces amount of RAM needed for DWARF-to-BTF conversion. Also, pahole finally
> gets support to generating BTF for cross-compiled ELF binaries with different
> endianness (patch #8).
>
> Additionally, patch #3 fixes previously missed problem with invalid array
> index type generation.
>
> Patches #4-7 are speeding up DWARF-to-BTF convertion/dedup pretty
> significantly, saving overall about 9 seconds out of current 27 or so.
>
> Patch #5 revamps how per-CPU BTF variables are emitted, eliminating repeated
> and expensive looping over ELF symbols table. The critical detail that took
> few hours of investigation is that when DW_AT_variable has
> DW_AT_specification, variable address (to correlate with symbol's address) has
> to be taken before specification is followed.
>
> More details could be found in respective patches.
>
> v1->v2:
> - rebase on latest dwarves master and fix var->spec's address problem.
Thanks, I applied all of them, tested and reproduced the performance
gains, great work!
I'll do some more testing on encoding a vmlinux for some big endian arch
on my x86_64 workstation and then push things publicly.
If Hao find any issues we can fix in a follow up patch.
I also added the people involved in the discussion about cross builds
failing, please take a look, I'm pushing now to a tmp.libbtf_encoder so
that you can test it from there, ok?
- Arnaldo
> Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
>
> Andrii Nakryiko (8):
> btf_loader: use libbpf to load BTF
> btf_encoder: use libbpf APIs to encode BTF type info
> btf_encoder: fix emitting __ARRAY_SIZE_TYPE__ as index range type
> btf_encoder: discard CUs after BTF encoding
> btf_encoder: revamp how per-CPU variables are encoded
> dwarf_loader: increase the size of lookup hash map
> strings: use BTF's string APIs for strings management
> btf_encoder: support cross-compiled ELF binaries with different
> endianness
>
> btf_encoder.c | 370 +++++++++++++++------------
> btf_loader.c | 244 +++++++-----------
> ctf_encoder.c | 2 +-
> dwarf_loader.c | 2 +-
> libbtf.c | 661 +++++++++++++++++++++----------------------------
> libbtf.h | 41 ++-
> libctf.c | 14 +-
> libctf.h | 4 +-
> pahole.c | 2 +-
> strings.c | 91 +++----
> strings.h | 32 +--
> 11 files changed, 645 insertions(+), 818 deletions(-)
>
> --
> 2.24.1
>
--
- Arnaldo
next prev parent reply other threads:[~2020-10-09 16:22 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-08 23:39 [PATCH v2 dwarves 0/8] Switch BTF loading and encoding to libbpf APIs Andrii Nakryiko
2020-10-08 23:39 ` [PATCH v2 dwarves 1/8] btf_loader: use libbpf to load BTF Andrii Nakryiko
2020-10-09 14:53 ` Arnaldo Carvalho de Melo
2020-10-09 17:49 ` Andrii Nakryiko
2020-10-08 23:39 ` [PATCH v2 dwarves 2/8] btf_encoder: use libbpf APIs to encode BTF type info Andrii Nakryiko
2020-10-08 23:39 ` [PATCH v2 dwarves 3/8] btf_encoder: fix emitting __ARRAY_SIZE_TYPE__ as index range type Andrii Nakryiko
2020-10-08 23:39 ` [PATCH v2 dwarves 4/8] btf_encoder: discard CUs after BTF encoding Andrii Nakryiko
2020-10-09 15:47 ` Arnaldo Carvalho de Melo
2020-10-08 23:39 ` [PATCH v2 dwarves 5/8] btf_encoder: revamp how per-CPU variables are encoded Andrii Nakryiko
2020-10-09 15:53 ` Arnaldo Carvalho de Melo
2020-10-08 23:39 ` [PATCH v2 dwarves 6/8] dwarf_loader: increase the size of lookup hash map Andrii Nakryiko
2020-10-08 23:39 ` [PATCH v2 dwarves 7/8] strings: use BTF's string APIs for strings management Andrii Nakryiko
2020-10-08 23:40 ` [PATCH v2 dwarves 8/8] btf_encoder: support cross-compiled ELF binaries with different endianness Andrii Nakryiko
2020-10-09 16:22 ` Arnaldo Carvalho de Melo [this message]
2020-10-09 17:59 ` [PATCH v2 dwarves 0/8] Switch BTF loading and encoding to libbpf APIs Andrii Nakryiko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201009162243.GD322246@kernel.org \
--to=acme@kernel.org \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=borna.cafuk@sartura.hr \
--cc=bpf@vger.kernel.org \
--cc=david.marcinkovic@sartura.hr \
--cc=dwarves@vger.kernel.org \
--cc=haoluo@google.com \
--cc=iii@linux.ibm.com \
--cc=juraj.vijtiuk@sartura.hr \
--cc=kernel-team@fb.com \
--cc=luka.perkov@sartura.hr \
--cc=olegrom@google.com \
--cc=tony.ambardar@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.