Re: [RFC] running bpftool build tests in CI

[Ihor Solodrai <ihor.solodrai@linux.dev>]

On 8/19/26 12:59 PM, Alexis Lothoré wrote:
> Hi,
> as part of the cleanup/automation effort in the BPF selfests directory,
> I am now taking a look at test_bpftool_build.sh, which ensures that the
> different supported ways of building bpftool work correctly. As this
> script does not really exercize anything at runtime but rather at build
> time, I'd like to propose to introduce a dedicated step in the CI
> automation that already builds and run selftests. I have opened two PRs
> in kernel-patches vmtest ([0]) and libbpf/ci ([1]), hoping I am not
> confusing which code should go where between the different repositories
> kept in sync with each other. I have also opened a dummy PR on
> kernel-patches/bpf ([2], not to be merged) that shows how this test
> would look like in CI.
> 
> This really is a RFC, as not all tests from test_bpftool_build.sh are
> being executed: I suspect those based on .config to be currently broken,
> but that can be handled as a second step, depending on the chosen
> direction.
> 
> Any comment welcome !

Hi Alexis, thank you for working on this.

I am a little confused about what are we trying to test here and
why. Maybe you could explain.

Some variants of bpftool build are CI-exercised regularly:
  * selftests/bpf directly depend on bpftool, it's built and is used
    as part of the main "test_progs*" suite
  * bpftool has a standalone github mirror with it's own CI pipeline:
    https://github.com/libbpf/bpftool

Looking at test_bpftool_build.sh (written 6y ago btw), it seems to be
more of a Makefile infra test than bpftool test specifically.

Given how complicated some in-tree tools/selftests/ Makefiles are [1],
running infra tests like this would be nice. But then maybe we should
develop them beyond test_bpftool_build.sh?

I briefly skimmed over the CI PRs you've submitted, and one thing I
would say is that we don't want a "bpftool build" test to block the
kernel build when it fails. It either needs to be a separate job, or a
step that is allowed to fail.

Vineet recently did a similar thing, where the gcc-bpf selftests build
was interleaved with the kernel build and I nacked it with the same
justification [2].

I'm thinking the whole "kernel build" workflow should be refactored
into either separate selftests build jobs, or some post processing
that looks at what artifacts where built successfully and reports
failures. This will provide a little more friendly UI/UX. You're
welcome to look into that if you're interested. But that's not
directly relevant to the changes you're proposing.

[1] https://lore.kernel.org/bpf/20260804170156.1709916-1-nickolay.lysenko@gmail.com/
[2] https://github.com/kernel-patches/vmtest/pull/502#discussion_r3724931742


> 
> Alexis
> 
> [0] https://github.com/kernel-patches/vmtest/pull/517
> [1] https://github.com/libbpf/ci/pull/236
> [2] https://github.com/kernel-patches/bpf/pull/13368
>