* Re: python3-perf pulling dependencies after build refactoring
2024-11-04 21:21 python3-perf pulling dependencies after build refactoring Michael Petlan
@ 2024-11-04 21:57 ` Ian Rogers
0 siblings, 0 replies; 2+ messages in thread
From: Ian Rogers @ 2024-11-04 21:57 UTC (permalink / raw)
To: Michael Petlan; +Cc: linux-perf-users, acme, vmolnaro, jplesnik
On Mon, Nov 4, 2024 at 1:22 PM Michael Petlan <mpetlan@redhat.com> wrote:
>
> Hello Ian,
>
> Jitka Plesnikova has spotted an interesting fact about he python bindings --
> perf.cpython-*-x86_64-linux-gnu.so library -- the number of shared objects it
> links against has significantly grown between v6.10 and v6.11:
>
> Before:
>
> # ldd python/perf.cpython-39-x86_64-linux-gnu.so
> linux-vdso.so.1 (0x00007ffefd1cb000)
> libm.so.6 => /lib64/libm.so.6 (0x00007f4c9315e000)
> libz.so.1 => /lib64/libz.so.1 (0x00007f4c93144000)
> libelf.so.1 => /lib64/libelf.so.1 (0x00007f4c929e4000)
> libslang.so.2 => /lib64/libslang.so.2 (0x00007f4c92600000)
> libc.so.6 => /lib64/libc.so.6 (0x00007f4c92200000)
> libnuma.so.1 => /lib64/libnuma.so.1 (0x00007f4c93136000)
> /lib64/ld-linux-x86-64.so.2 (0x00007f4c93247000)
> libzstd.so.1 => /lib64/libzstd.so.1 (0x00007f4c9290d000)
>
> After:
>
> # ldd python/perf.cpython-39-x86_64-linux-gnu.so
> linux-vdso.so.1 (0x00007ffd889f3000)
> libm.so.6 => /lib64/libm.so.6 (0x00007f75cd125000)
> libz.so.1 => /lib64/libz.so.1 (0x00007f75cdbc0000)
> libelf.so.1 => /lib64/libelf.so.1 (0x00007f75cdba4000)
> libdw.so.1 => /lib64/libdw.so.1 (0x00007f75cd089000)
> libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f75ccc00000)
> libslang.so.2 => /lib64/libslang.so.2 (0x00007f75cc800000)
> libperl.so.5.32 => /lib64/libperl.so.5.32 (0x00007f75cc400000)
> libc.so.6 => /lib64/libc.so.6 (0x00007f75cc000000)
> libpython3.9.so.1.0 => /lib64/libpython3.9.so.1.0 (0x00007f75cbc00000)
> libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f75cb800000)
> liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f75cdb76000)
> libzstd.so.1 => /lib64/libzstd.so.1 (0x00007f75ccb29000)
> libcap.so.2 => /lib64/libcap.so.2 (0x00007f75cdb6a000)
> libnuma.so.1 => /lib64/libnuma.so.1 (0x00007f75cdb5c000)
> libbabeltrace-ctf.so.1 => /lib64/libbabeltrace-ctf.so.1 (0x00007f75cd03f000)
> libpfm.so.4 => /lib64/libpfm.so.4 (0x00007f75cb400000)
> libtraceevent.so.1 => /lib64/libtraceevent.so.1 (0x00007f75ccb08000)
> /lib64/ld-linux-x86-64.so.2 (0x00007f75cdbe8000)
> libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f75cdb49000)
> libcrypt.so.2 => /lib64/libcrypt.so.2 (0x00007f75cc7c6000)
> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f75ccaed000)
> libbabeltrace.so.1 => /lib64/libbabeltrace.so.1 (0x00007f75ccadd000)
> libpopt.so.0 => /lib64/libpopt.so.0 (0x00007f75cc7b7000)
> libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f75cdb3d000)
> libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007f75cd039000)
> libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007f75cc2c5000)
> libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f75cc24d000)
>
> I have narrowed it down to e467705a9fb3 ("perf util: Make util its own library").
> It has also grown in size, probably because of this linking change. I think we
> need to get rid of this probably unwanted side-effect of the build optimization.
> What do you think?
Hi Michael,
it wasn't a build optimization. The problem was that we have things
like lex and yacc generated C code that things like parse_events
require to, you know, be able to parse events :-) So that we don't
duplicate logic from event parsing for things like
heterogeneous/hybrid/BIG.little machines we've been moving logic in
places like evlist to use parse_events rather than hand craft struct
perf_event_attr. So evlist has a dependence on parse_events but
previously the python code would stub out parse_events meaning the
python use of evlist had regressed.
I tried a few times to restructure things so that python and the build
would work as it was, but in the end the simplest solution was to
break apart the build. That is make util, tests, ui, bench.. their own
library and then link these with the builtin and perf.c code to make
the executable. Then the python code could depend on these
intermediate libraries, not worry about how they were made (lex/yacc
building, etc.), and we could have the full parse_events code in the
python code.
Now the code in util has additional dependencies, and we're linking
against libraries with those dependencies, so that's why the size has
gone up. Can we do better? Yes, we can. In these two patches we remove
the test and bench libraries from the python build:
https://lore.kernel.org/lkml/20241031014252.753588-16-irogers@google.com/
(unmerged)
https://lore.kernel.org/lkml/20241031014252.753588-18-irogers@google.com/
(unmerged)
The removal of the bench library will remove the use of libnuma.
Can we do yet better? Yes, if we can decouple the util and ui code
then there is no reason libslang be part of the python code.
Unfortunately that code is quite interwoven currently.
Can we do even more better? Possibly, in these changes libtraceevent
becomes more optional:
https://lore.kernel.org/lkml/20241102165400.75785-1-irogers@google.com/
(unmerged)
But removing this would also remove libtracevent support from perf, so
this probably isn't something you'd want to enable.
The libcap dependency should be gone after:
https://lore.kernel.org/lkml/20240806220614.831914-1-irogers@google.com/
libpfm is tied to event parsing so we probably want to keep it as a dependence.
libbabeltrace I thought was used for syscall tables when other things
fail. Hopefully we can just have syscall tables and do away with this.
This cleanup makes me excited:
https://lore.kernel.org/linux-perf-users/Zyk9hX8CB_2rbWsi@ghost/T/#t
The compression library stuff is so that compressed events can work,
hard to see how to remove these without disabling compression support.
As perf script can run python and perl code we get those dependencies,
maybe we can get the relevant code only to be part of the
builtin-script.c and lose those dependencies from the util code.
libglib and libmodule I think relate to the not very maintained GTk
code. We probably should delete that code from the build. If we
decouple util and ui then again I think the dependency goes.
libpopt, I don't know where this dependency is coming from but it
looks bogus. perf's command line processing comes from the statically
linked libsubcommand.
libcrypt and lubuuid I think are only needed for genelf which is part
of `perf inject -j`. Possibly this logic can be carved out of util to
avoid bringing in the dependencies.
Thanks,
Ian
^ permalink raw reply [flat|nested] 2+ messages in thread