Re: [BUG] perf test: Regression because of d6e6286a12e7

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Em Wed, Jul 12, 2023 at 07:38:58PM +0530, Ravi Bangoria escreveu:
> On 11-Jul-23 3:06 AM, Andrii Nakryiko wrote:
> > On Sun, Jul 9, 2023 at 9:05 PM Ravi Bangoria <ravi.bangoria@amd.com> wrote:
> >> On 08-Jul-23 4:46 AM, Andrii Nakryiko wrote:
> >>> On Wed, Jul 5, 2023 at 9:39 PM Ravi Bangoria <ravi.bangoria@amd.com> wrote:
> >>>> I'm seeing perf test failure because of commit d6e6286a12e7 ("libbpf:
> >>>> disassociate section handler on explicit bpf_program__set_type() call").

> >>> Yep, this commit would reset catch-all custom handler, which perf is
> >>> setting. I've just sent a fix upstream ([0]). And once it lands, I'll
> >>> cut a v1.2.1 libbpf bugfix release with just this fix on top of v1.2.

> >>> Can you please double-check that this patch indeed fixes the issue for
> >>> you? I tried to do this locally, but for me perf test 42 fails both at
> >>> current bpf-next, with the above commit reverted, and with my fix
> >>> applied on top. So I can't be 100% sure.

> >>>   [0] https://patchwork.kernel.org/project/netdevbpf/patch/20230707231156.1711948-1-andrii@kernel.org/

> >> Thanks. A quick test seems to be working fine.

> > Alright, thanks for confirming! I've just released v1.2.1 bug fix
> > release with just this fix on top of v1.2.

> > Thanks for reporting!

> > But given v1.2 was cut on May 1st, and the offending commit landed
> > some time late March, I wonder how did this slip through the cracks
> > and go unreported for so long? Is there something we can do to catch
> > these perf-only regressions a bit sooner?

> I guess it got slipped because that patch went in via bpf tree. Would
> it be possible to run bpf related perf tests at the time of applying
> libbpf patches? Arnaldo might have better ideas :)

Right, perhaps the libbpf CI could try building perf, preferably with
BUILD_BPF_SKEL=1, to enable these tools:

[acme@nine linux]$ ls -la tools/perf/util/bpf_skel/*.bpf.*
-rw-r--r--. 1 acme acme  5581 Jul  7 12:38 tools/perf/util/bpf_skel/bperf_cgroup.bpf.c
-rw-r--r--. 1 acme acme  1764 Jul  7 12:38 tools/perf/util/bpf_skel/bperf_follower.bpf.c
-rw-r--r--. 1 acme acme  1438 Jul  7 12:38 tools/perf/util/bpf_skel/bperf_leader.bpf.c
-rw-r--r--. 1 acme acme  2290 Jul  7 12:38 tools/perf/util/bpf_skel/bpf_prog_profiler.bpf.c
-rw-r--r--. 1 acme acme  2164 Jul  7 12:38 tools/perf/util/bpf_skel/func_latency.bpf.c
-rw-r--r--. 1 acme acme  9017 Jul  7 12:38 tools/perf/util/bpf_skel/kwork_trace.bpf.c
-rw-r--r--. 1 acme acme 10147 Jul 12 11:49 tools/perf/util/bpf_skel/lock_contention.bpf.c
-rw-r--r--. 1 acme acme  6109 Jul  7 12:38 tools/perf/util/bpf_skel/off_cpu.bpf.c
-rw-r--r--. 1 acme acme  4932 Jul  7 12:38 tools/perf/util/bpf_skel/sample_filter.bpf.c
[acme@nine linux]$

There are 'perf test' entries for the BPF support in perf that is pre
libbpf skel functionality above and as well some for the libbpf based
skel features, such as:

[root@quaco ~]# perf test -vvv contention
 87: kernel lock contention analysis test                            :
--- start ---
test child forked, pid 213314
Testing perf lock record and perf lock contention
Testing perf lock contention --use-bpf
Testing perf lock record and perf lock contention at the same time
Testing perf lock contention --threads
Testing perf lock contention --lock-addr
Testing perf lock contention --type-filter (w/ spinlock)
Testing perf lock contention --lock-filter (w/ tasklist_lock)
Testing perf lock contention --callstack-filter (w/ unix_stream)
Testing perf lock contention --callstack-filter with task aggregation
Testing perf lock contention CSV output
test child finished with 0
---- end ----
kernel lock contention analysis test: Ok
[root@quaco ~]#

We could perhaps add support for some kind of tags to the tests, to
help run just the ones that use bpf and then you could use:

 # perf test --uses bpf

At first we would just add this tag, then we would use it for other
stuff as the need arises to run just a subset of the tests.

But for now you could start perhaps with:

[root@quaco ~]# perf test LLVM ; perf test "BPF " ; perf test "lock contention"
 40: LLVM search and compile                                         :
 40.1: Basic BPF llvm compile                                        : Ok
 40: LLVM search and compile                                         :
 40.1: Basic BPF llvm compile                                        : Ok
 40.3: Compile source for BPF prologue generation                    : Ok
 40.4: Compile source for BPF relocation                             : Ok
 42: BPF filter                                                      :
 42.1: Basic BPF filtering                                           : Ok
 42.2: BPF pinning                                                   : Ok
 42.3: BPF prologue generation                                       : FAILED!
 87: kernel lock contention analysis test                            : Ok
[root@quaco ~]#

- Arnaldo