Re: Flaky tests

[Arnaldo Carvalho de Melo <acme@kernel.org>]

On Fri, Jul 31, 2026 at 04:32:38PM -0700, Namhyung Kim wrote:
> On Sun, Jul 26, 2026 at 08:52:36PM -0300, Arnaldo Carvalho de Melo wrote:
> > On Sun, Jul 26, 2026 at 08:22:08PM -0300, Arnaldo Carvalho de Melo wrote:
> > > ⬢ [acme@toolbx perf-tools-next]$ git log --oneline -1
> > > 98bc7deff4858aa8 (HEAD -> perf/hardening-machine, number/perf/hardening-machine) perf machine: Check snprintf truncation for guest kallsyms path
> > > ⬢ [acme@toolbx perf-tools-next]$ perf -v
> > > perf version 7.2.rc1.g98bc7deff485
> > > ⬢ [acme@toolbx perf-tools-next]$
> > > logout
> > > acme@x1:~/git/perf-tools-next$
> > > acme@x1:~/git/perf-tools-next$ sudo su -
> > > [sudo] password for acme:
> > > root@x1:~# perf -vv
> > > perf version 7.2.rc1.g98bc7deff485
> > >                    aio: [ on  ]  # HAVE_AIO_SUPPORT
> > >                    bpf: [ on  ]  # HAVE_LIBBPF_SUPPORT
> > >          bpf_skeletons: [ on  ]  # HAVE_BPF_SKEL
> > >             debuginfod: [ on  ]  # HAVE_DEBUGINFOD_SUPPORT
> > >                  dwarf: [ on  ]  # HAVE_LIBDW_SUPPORT
> > >     dwarf_getlocations: [ on  ]  # HAVE_LIBDW_SUPPORT
> > >           dwarf-unwind: [ on  ]  # HAVE_DWARF_UNWIND_SUPPORT
> > >                 libbfd: [ OFF ]  # HAVE_LIBBFD_SUPPORT ( tip: Deprecated, license incompatibility, use BUILD_NONDISTRO=1 and install binutils-dev[el] )
> > > babeltrace2-ctf-writer: [ on  ]  # HAVE_BABELTRACE2_CTF_WRITER_SUPPORT
> > >         libbpf-strings: [ on  ]  # HAVE_LIBBPF_STRINGS_SUPPORT
> > >            libcapstone: [ on  ]  # HAVE_LIBCAPSTONE_SUPPORT
> > >     libdw-dwarf-unwind: [ on  ]  # HAVE_LIBDW_SUPPORT
> > >                 libelf: [ on  ]  # HAVE_LIBELF_SUPPORT
> > >                libLLVM: [ on  ]  # HAVE_LIBLLVM_SUPPORT
> > >                libnuma: [ on  ]  # HAVE_LIBNUMA_SUPPORT
> > >             libopencsd: [ on  ]  # HAVE_CSTRACE_SUPPORT
> > >                libperl: [ OFF ]  # HAVE_LIBPERL_SUPPORT ( tip: Deprecated, use LIBPERL=1 and install perl-ExtUtils-Embed/libperl-dev to build with it )
> > >                libpfm4: [ on  ]  # HAVE_LIBPFM
> > >              libpython: [ on  ]  # HAVE_LIBPYTHON_SUPPORT
> > >               libslang: [ on  ]  # HAVE_SLANG_SUPPORT
> > >          libtraceevent: [ on  ]  # HAVE_LIBTRACEEVENT
> > >              libunwind: [ OFF ]  # HAVE_LIBUNWIND_SUPPORT ( tip: Deprecated, use LIBUNWIND=1 and install libunwind-dev[el] to build with it )
> > >                   lzma: [ on  ]  # HAVE_LZMA_SUPPORT
> > > numa_num_possible_cpus: [ on  ]  # HAVE_LIBNUMA_SUPPORT
> > >                   zlib: [ on  ]  # HAVE_ZLIB_SUPPORT
> > >                   zstd: [ on  ]  # HAVE_ZSTD_SUPPORT
> > >                   rust: [ on  ]  # HAVE_RUST_SUPPORT
> > > root@x1:~# perf test
> [SNIP]
> > > === Test Summary ===
> > > Passed main tests : 133
> > > Passed subtests   : 152
> > > Skipped tests     : 24
> > > Failed tests      : 2
> > > List of failed tests:
> > >    23: Object code reading
> > >   110: perf record sample filtering (by BPF) tests
> > > root@x1:~# perf test 23
> > >  23: Object code reading                                                                                       : Ok
> 
> Right, I'm seeing this is flaky.  Sometimes it fails to find a map for
> some kernel address.
> 
> > > 
> > > === Test Summary ===
> > > Passed main tests : 1
> > > Passed subtests   : 0
> > > Skipped tests     : 0
> > > Failed tests      : 0
> > > root@x1:~# perf test 110
> > > 110: perf record sample filtering (by BPF) tests                                                               : Ok
> 
> I never saw this before.  Do you have error message?

So, I just ran 'perf test -vvv', i.e. everything and in parallel, when I
run it alone it works, but together with the other tests:

110: perf record sample filtering (by BPF) tests:
---- start ----
test child forked, pid 195512
Checking BPF-filter privilege
Basic bpf-filter test
Basic bpf-filter test [Success]
Failing bpf-filter test
Failing bpf-filter test [Success]
Group bpf-filter test
Group bpf-filter test [Failed forbidden CPU]
---- end(-1) ----
110: perf record sample filtering (by BPF) tests                                : FAILED!


Alone:

root@x1:~# perf test 110
110: perf record sample filtering (by BPF) tests                                : Ok

=== Test Summary ===
Passed main tests : 1
Passed subtests   : 0
Skipped tests     : 0
Failed tests      : 0
root@x1:~# perf stat --repeat 5 --null perf test 110
110: perf record sample filtering (by BPF) tests                                : Ok

=== Test Summary ===
Passed main tests : 1
Passed subtests   : 0
Skipped tests     : 0
Failed tests      : 0
110: perf record sample filtering (by BPF) tests                                : Ok

=== Test Summary ===
Passed main tests : 1
Passed subtests   : 0
Skipped tests     : 0
Failed tests      : 0
110: perf record sample filtering (by BPF) tests                                : Ok

=== Test Summary ===
Passed main tests : 1
Passed subtests   : 0
Skipped tests     : 0
Failed tests      : 0
110: perf record sample filtering (by BPF) tests                                : Ok

=== Test Summary ===
Passed main tests : 1
Passed subtests   : 0
Skipped tests     : 0
Failed tests      : 0
110: perf record sample filtering (by BPF) tests                                : Ok

=== Test Summary ===
Passed main tests : 1
Passed subtests   : 0
Skipped tests     : 0
Failed tests      : 0

 Performance counter stats for 'perf test 110' (5 runs):

      13.785992473 +- 0.048784821 seconds time elapsed  ( +-  0.35% )

root@x1:~#