Re: Announcement: GCC BPF is now being tested on BPF CI

["Jose E. Marchesi" <jose.marchesi@oracle.com>]

> On Thursday, January 16th, 2025 at 12:44 PM, Ihor Solodrai <ihor.solodrai@pm.me> wrote:
>
>> 
>> 
>> Hi everyone.
>> 
>> GCC BPF support in BPF CI has been landed.
>> 
>> The BPF CI dashboard is here:
>> https://github.com/kernel-patches/bpf/actions/workflows/test.yml
>> 
>> A summary of what happens on CI (relevant to GCC BPF):
>> * Linux Kernel is built on a target source revision
>> * Latest snapshots of GCC 15 and binutils are downloaded
>> * GCC BPF compiler is built and cached
>> * selftests/bpf test runners are built with BPF_GCC variable set
>> * BPF_GCC triggers a build of test_progs-bpf_gcc runner
>> * The runner contains BPF binaries produced by GCC BPF
>> * In a separate job, test_progs-bpf_gcc is executed within qemu
>> against the target kernel
>> 
>> GCC BPF is only tested on x86_64.
>> 
>> On x86_64 we test the following toolchains for building the kernel and
>> test runners: gcc-13 (ubuntu 24 default), clang-17, clang-18.
>> 
>> An example of successful test run (you have to login to github to see
>> the logs):
>> https://github.com/kernel-patches/bpf/actions/runs/12816136141/job/35736973856
>> 
>> Currently 2513 of 4340 tests pass for GCC BPF, so a bit more than a half.
>> 
>> Effective BPF selftests denylist for GCC BPF is located here:
>> https://github.com/kernel-patches/vmtest/blob/master/ci/vmtest/configs/DENYLIST.test_progs-bpf_gcc
>
> The announcement triggered an off-list discussion among BPF devs about
> how to handle the test running, given the long denylist.
>
> The problem is that any new test is now a potential subject to
> debugging whether the test needs changes, or GCC doesn't work for it.
>
> As of now, an important missing piece on GCC side is the decl_tags
> support, as they are heavily used by BPF selftests. See a message from
> Yonghong Song:
> https://gcc.gnu.org/pipermail/gcc-patches/2025-January/673841.html
>
> Some discussed suggestions:
>   * Run test_progs-bpf_gcc with "allowed to fail", so that the
>     pipeline is never blocked
>   * Only run GCC BPF *compilation*, and don't execute the tests

I think that this is the best solution for now, and the most useful.

As soon as we achieve passing all the selftests (hopefully soon) then we
can change the CI to flag regressions on test run failures as well.


>   * Flip denylist to allowlist to prevent regressions, but not force
>     new tests to work with GCC
>
> Input from GCC devs will be much appreciated.
>
> Thanks.
>
>> 
>> When a patch is submitted to BPF, normally a corresponding PR for
>> kernel-patches/bpf github repo is automatically created to trigger a
>> BPF CI run for this change. PRs opened manually will do that too, and
>> this can be used to test patches before submission.
>> 
>> Since the CI automatically pulls latest GCC snapshot, a change in GCC
>> can potentially cause CI failures unrelated to Linux changes being
>> tested. This is not the only dependency like that, of course.
>> 
>> In such situations, a change is usually made in CI code to mitigate
>> the failure in order to unblock the pipeline for patches. If that
>> happens with GCC, someone (most likely me) will have to reach out to
>> GCC team. I guess gcc@gcc.gnu.org would be the default point of
>> contact, but if there are specific people who should be notified
>> please let me know.