[RFC] running bpftool build tests in CI

[RFC] running bpftool build tests in CI

[Alexis Lothoré]

Hi,
as part of the cleanup/automation effort in the BPF selfests directory,
I am now taking a look at test_bpftool_build.sh, which ensures that the
different supported ways of building bpftool work correctly. As this
script does not really exercize anything at runtime but rather at build
time, I'd like to propose to introduce a dedicated step in the CI
automation that already builds and run selftests. I have opened two PRs
in kernel-patches vmtest ([0]) and libbpf/ci ([1]), hoping I am not
confusing which code should go where between the different repositories
kept in sync with each other. I have also opened a dummy PR on
kernel-patches/bpf ([2], not to be merged) that shows how this test
would look like in CI.

This really is a RFC, as not all tests from test_bpftool_build.sh are
being executed: I suspect those based on .config to be currently broken,
but that can be handled as a second step, depending on the chosen
direction.

Any comment welcome !

Alexis

[0] https://github.com/kernel-patches/vmtest/pull/517
[1] https://github.com/libbpf/ci/pull/236
[2] https://github.com/kernel-patches/bpf/pull/13368

-- 
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [RFC] running bpftool build tests in CI

[Ihor Solodrai]

On 8/19/26 12:59 PM, Alexis Lothoré wrote:
> Hi,
> as part of the cleanup/automation effort in the BPF selfests directory,
> I am now taking a look at test_bpftool_build.sh, which ensures that the
> different supported ways of building bpftool work correctly. As this
> script does not really exercize anything at runtime but rather at build
> time, I'd like to propose to introduce a dedicated step in the CI
> automation that already builds and run selftests. I have opened two PRs
> in kernel-patches vmtest ([0]) and libbpf/ci ([1]), hoping I am not
> confusing which code should go where between the different repositories
> kept in sync with each other. I have also opened a dummy PR on
> kernel-patches/bpf ([2], not to be merged) that shows how this test
> would look like in CI.
> 
> This really is a RFC, as not all tests from test_bpftool_build.sh are
> being executed: I suspect those based on .config to be currently broken,
> but that can be handled as a second step, depending on the chosen
> direction.
> 
> Any comment welcome !

Hi Alexis, thank you for working on this.

I am a little confused about what are we trying to test here and
why. Maybe you could explain.

Some variants of bpftool build are CI-exercised regularly:
  * selftests/bpf directly depend on bpftool, it's built and is used
    as part of the main "test_progs*" suite
  * bpftool has a standalone github mirror with it's own CI pipeline:
    https://github.com/libbpf/bpftool

Looking at test_bpftool_build.sh (written 6y ago btw), it seems to be
more of a Makefile infra test than bpftool test specifically.

Given how complicated some in-tree tools/selftests/ Makefiles are [1],
running infra tests like this would be nice. But then maybe we should
develop them beyond test_bpftool_build.sh?

I briefly skimmed over the CI PRs you've submitted, and one thing I
would say is that we don't want a "bpftool build" test to block the
kernel build when it fails. It either needs to be a separate job, or a
step that is allowed to fail.

Vineet recently did a similar thing, where the gcc-bpf selftests build
was interleaved with the kernel build and I nacked it with the same
justification [2].

I'm thinking the whole "kernel build" workflow should be refactored
into either separate selftests build jobs, or some post processing
that looks at what artifacts where built successfully and reports
failures. This will provide a little more friendly UI/UX. You're
welcome to look into that if you're interested. But that's not
directly relevant to the changes you're proposing.

[1] https://lore.kernel.org/bpf/20260804170156.1709916-1-nickolay.lysenko@gmail.com/
[2] https://github.com/kernel-patches/vmtest/pull/502#discussion_r3724931742


> 
> Alexis
> 
> [0] https://github.com/kernel-patches/vmtest/pull/517
> [1] https://github.com/libbpf/ci/pull/236
> [2] https://github.com/kernel-patches/bpf/pull/13368
> 

Re: [RFC] running bpftool build tests in CI

[Alexis Lothoré]

Hi Ihor,
thanks for the feedback

On Wed Aug 19, 2026 at 10:46 PM CEST, Ihor Solodrai wrote:
> On 8/19/26 12:59 PM, Alexis Lothoré wrote:
>> Hi,
>> as part of the cleanup/automation effort in the BPF selfests directory,
>> I am now taking a look at test_bpftool_build.sh, which ensures that the
>> different supported ways of building bpftool work correctly. As this
>> script does not really exercize anything at runtime but rather at build
>> time, I'd like to propose to introduce a dedicated step in the CI
>> automation that already builds and run selftests. I have opened two PRs
>> in kernel-patches vmtest ([0]) and libbpf/ci ([1]), hoping I am not
>> confusing which code should go where between the different repositories
>> kept in sync with each other. I have also opened a dummy PR on
>> kernel-patches/bpf ([2], not to be merged) that shows how this test
>> would look like in CI.
>> 
>> This really is a RFC, as not all tests from test_bpftool_build.sh are
>> being executed: I suspect those based on .config to be currently broken,
>> but that can be handled as a second step, depending on the chosen
>> direction.
>> 
>> Any comment welcome !
>
> Hi Alexis, thank you for working on this.
>
> I am a little confused about what are we trying to test here and
> why. Maybe you could explain.

Sure, I may have been a bit light on this. The test_bpftool_build.sh
located in tools/testing/selftests/bpf focuses on the various ways of
building bpftool:
- through kbuild (eg: make tools/bpf)
- by changing make execution dir (eg: make -C tools/bpf/bpftool)
- by running the tools/ main makefile (eg: cd tools && make bpf)
- by running the bpftool main makefile (eg: cd tools/bpf/bpftool &&
  make)

This listing is also cross-tested with output path configuration,
testing both O=<output_dir> and OUTPUT=<output_dir>.

Will be included in the relevant commits if this RFC goes further in
this direction.

> Some variants of bpftool build are CI-exercised regularly:
>   * selftests/bpf directly depend on bpftool, it's built and is used
>     as part of the main "test_progs*" suite
>   * bpftool has a standalone github mirror with it's own CI pipeline:
>     https://github.com/libbpf/bpftool
>
> Looking at test_bpftool_build.sh (written 6y ago btw), it seems to be
> more of a Makefile infra test than bpftool test specifically.
>
> Given how complicated some in-tree tools/selftests/ Makefiles are [1],
> running infra tests like this would be nice. But then maybe we should
> develop them beyond test_bpftool_build.sh?
>
> I briefly skimmed over the CI PRs you've submitted, and one thing I
> would say is that we don't want a "bpftool build" test to block the
> kernel build when it fails. It either needs to be a separate job, or a
> step that is allowed to fail.
>
> Vineet recently did a similar thing, where the gcc-bpf selftests build
> was interleaved with the kernel build and I nacked it with the same
> justification [2].

Understood, thanks for the pointers. I see that Vineet came up with a
new revision that eventually got merged, moving gcc-bpf as a dedicated
job in kernel-build-test.yml instead of kernel-build.yml (and so, a
failure on gcc-bpf does not mark kernel build as failed) I guess I can
replicate this (also, making sure to add this run_tests toggle, maybe
?). However, I feel like it would not make much sense to have a build
and a test part, with artifacts going from the former to the latter, as
the build step actually _is_ the test. Also, the only needed artifact to
run the build is the kernel source tree (we don't need any vmlinux or
.config, at least for now)

Would it be ok if I try it this way ?

> I'm thinking the whole "kernel build" workflow should be refactored
> into either separate selftests build jobs, or some post processing
> that looks at what artifacts where built successfully and reports
> failures. This will provide a little more friendly UI/UX. You're
> welcome to look into that if you're interested. But that's not
> directly relevant to the changes you're proposing.

Trying to rephrase it to make sure I get your point, the goal would be
to have green/red checks specifically on selftests builds, rather than a
global green/red check on the macro job "build kernel and selftests" ?
Or does it go further than that ?

If I get it correctly, yes I'd be glad to try and help on that, as a
separate task.

Alexis

> [1] https://lore.kernel.org/bpf/20260804170156.1709916-1-nickolay.lysenko@gmail.com/
> [2] https://github.com/kernel-patches/vmtest/pull/502#discussion_r3724931742
>
>
>> 
>> Alexis
>> 
>> [0] https://github.com/kernel-patches/vmtest/pull/517
>> [1] https://github.com/libbpf/ci/pull/236
>> [2] https://github.com/kernel-patches/bpf/pull/13368
>> 




-- 
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com