Re: [PATCH bpf-next 0/4] selftests/bpf: add a new runner for bpftool tests

Andrii Nakryiko
2026-01-16T22:20:47Z

On Thu, Jan 15, 2026 at 11:57 PM Alexis Lothoré
<alexis.lothore@bootlin.com> wrote:
>
> Hi Andrii,
>
> On Thu Jan 15, 2026 at 6:58 PM CET, Andrii Nakryiko wrote:
> > On Wed, Jan 14, 2026 at 12:59 AM Alexis Lothoré (eBPF Foundation)
> > <alexis.lothore@bootlin.com> wrote:
> >>
> >> Hello,
> >> this series is part of the larger effort aiming to convert all
> >> standalone tests to the CI runners so that they are properly executed on
> >> patches submission.
> >>
> >> Some of those tests are validating bpftool behavior(test_bpftool_map.sh,
> >> test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)
> >> and so they do not integrate well in test_progs. This series proposes to
> >
> > Can you elaborate why they do not integrate well? In my mind,
> > test_progs should be the only runner into which we invest effort
> > (parallel tests, all the different filtering, etc; why would we have
> > to reimplement subsets of this). The fact that we have test_maps and
> > test_verifier is historical and if we had enough time we'd merge all
> > of them into test_progs.
> >
> > What exactly in test_progs would prevent us from implementing bpftool
> > test runner?
>
> I don't think there is any strong technical blocker preventing from
> integrating those tests directly into test_progs. That's rather about
> the fact that test_progs tests depends (almost) exclusively on
> libbpf/skeletons. Those bpftool tests rather need to directly execute

There are actually plenty of test in test_progs that do networking
setups, calling system() to launch various binaries, etc. So it never
was purely for libbpf/skeletons/whatnot.

So yeah, I think bpftool testing should still be implemented as one
(or many) test_progs tests (and maybe subtests), utilizing
test_progs's generic multi-process testing setup, filtering,
reporting, etc infrastructure. No need to add extra runners.

> bpftool and parse its stdout output, so I thought that it made sense to
> have a dedicated runner for this. If I'm wrong and so if those tests
> should rather be moved in the test_progs runner (eg to avoid duplicating
> the runner features), I'm fine with it. Any additional opinion on this
> is welcome.
>
> Thanks,
>
> Alexis
> --
> Alexis Lothoré, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>