Announcement: GCC BPF is now being tested on BPF CI

Announcement: GCC BPF is now being tested on BPF CI

[Ihor Solodrai]

Hi everyone.

GCC BPF support in BPF CI has been landed.

The BPF CI dashboard is here:
https://github.com/kernel-patches/bpf/actions/workflows/test.yml

A summary of what happens on CI (relevant to GCC BPF):
  * Linux Kernel is built on a target source revision
  * Latest snapshots of GCC 15 and binutils are downloaded
    * GCC BPF compiler is built and cached
  * selftests/bpf test runners are built with BPF_GCC variable set
    * BPF_GCC triggers a build of test_progs-bpf_gcc runner
    * The runner contains BPF binaries produced by GCC BPF
  * In a separate job, test_progs-bpf_gcc is executed within qemu
    against the target kernel

GCC BPF is only tested on x86_64.

On x86_64 we test the following toolchains for building the kernel and
test runners: gcc-13 (ubuntu 24 default), clang-17, clang-18.

An example of successful test run (you have to login to github to see
the logs):
https://github.com/kernel-patches/bpf/actions/runs/12816136141/job/35736973856

Currently 2513 of 4340 tests pass for GCC BPF, so a bit more than a half.

Effective BPF selftests denylist for GCC BPF is located here:
https://github.com/kernel-patches/vmtest/blob/master/ci/vmtest/configs/DENYLIST.test_progs-bpf_gcc

When a patch is submitted to BPF, normally a corresponding PR for
kernel-patches/bpf github repo is automatically created to trigger a
BPF CI run for this change. PRs opened manually will do that too, and
this can be used to test patches before submission.

Since the CI automatically pulls latest GCC snapshot, a change in GCC
can potentially cause CI failures unrelated to Linux changes being
tested. This is not the only dependency like that, of course.

In such situations, a change is usually made in CI code to mitigate
the failure in order to unblock the pipeline for patches. If that
happens with GCC, someone (most likely me) will have to reach out to
GCC team. I guess gcc@gcc.gnu.org would be the default point of
contact, but if there are specific people who should be notified
please let me know.

Re: Announcement: GCC BPF is now being tested on BPF CI

[Sam James]

Wonderful news! Massive thanks for working on this.

> but if there are specific people who should be notified
> please let me know.

Please feel free to include me on that list too.

Re: Announcement: GCC BPF is now being tested on BPF CI

[Ihor Solodrai]

On Thursday, January 16th, 2025 at 12:44 PM, Ihor Solodrai <ihor.solodrai@pm.me> wrote:

> 
> 
> Hi everyone.
> 
> GCC BPF support in BPF CI has been landed.
> 
> The BPF CI dashboard is here:
> https://github.com/kernel-patches/bpf/actions/workflows/test.yml
> 
> A summary of what happens on CI (relevant to GCC BPF):
> * Linux Kernel is built on a target source revision
> * Latest snapshots of GCC 15 and binutils are downloaded
> * GCC BPF compiler is built and cached
> * selftests/bpf test runners are built with BPF_GCC variable set
> * BPF_GCC triggers a build of test_progs-bpf_gcc runner
> * The runner contains BPF binaries produced by GCC BPF
> * In a separate job, test_progs-bpf_gcc is executed within qemu
> against the target kernel
> 
> GCC BPF is only tested on x86_64.
> 
> On x86_64 we test the following toolchains for building the kernel and
> test runners: gcc-13 (ubuntu 24 default), clang-17, clang-18.
> 
> An example of successful test run (you have to login to github to see
> the logs):
> https://github.com/kernel-patches/bpf/actions/runs/12816136141/job/35736973856
> 
> Currently 2513 of 4340 tests pass for GCC BPF, so a bit more than a half.
> 
> Effective BPF selftests denylist for GCC BPF is located here:
> https://github.com/kernel-patches/vmtest/blob/master/ci/vmtest/configs/DENYLIST.test_progs-bpf_gcc

The announcement triggered an off-list discussion among BPF devs about
how to handle the test running, given the long denylist.

The problem is that any new test is now a potential subject to
debugging whether the test needs changes, or GCC doesn't work for it.

As of now, an important missing piece on GCC side is the decl_tags
support, as they are heavily used by BPF selftests. See a message from
Yonghong Song:
https://gcc.gnu.org/pipermail/gcc-patches/2025-January/673841.html

Some discussed suggestions:
  * Run test_progs-bpf_gcc with "allowed to fail", so that the
    pipeline is never blocked
  * Only run GCC BPF *compilation*, and don't execute the tests
  * Flip denylist to allowlist to prevent regressions, but not force
    new tests to work with GCC

Input from GCC devs will be much appreciated.

Thanks.

> 
> When a patch is submitted to BPF, normally a corresponding PR for
> kernel-patches/bpf github repo is automatically created to trigger a
> BPF CI run for this change. PRs opened manually will do that too, and
> this can be used to test patches before submission.
> 
> Since the CI automatically pulls latest GCC snapshot, a change in GCC
> can potentially cause CI failures unrelated to Linux changes being
> tested. This is not the only dependency like that, of course.
> 
> In such situations, a change is usually made in CI code to mitigate
> the failure in order to unblock the pipeline for patches. If that
> happens with GCC, someone (most likely me) will have to reach out to
> GCC team. I guess gcc@gcc.gnu.org would be the default point of
> contact, but if there are specific people who should be notified
> please let me know.

Re: Announcement: GCC BPF is now being tested on BPF CI

[Jose E. Marchesi]

> On Thursday, January 16th, 2025 at 12:44 PM, Ihor Solodrai <ihor.solodrai@pm.me> wrote:
>
>> 
>> 
>> Hi everyone.
>> 
>> GCC BPF support in BPF CI has been landed.
>> 
>> The BPF CI dashboard is here:
>> https://github.com/kernel-patches/bpf/actions/workflows/test.yml
>> 
>> A summary of what happens on CI (relevant to GCC BPF):
>> * Linux Kernel is built on a target source revision
>> * Latest snapshots of GCC 15 and binutils are downloaded
>> * GCC BPF compiler is built and cached
>> * selftests/bpf test runners are built with BPF_GCC variable set
>> * BPF_GCC triggers a build of test_progs-bpf_gcc runner
>> * The runner contains BPF binaries produced by GCC BPF
>> * In a separate job, test_progs-bpf_gcc is executed within qemu
>> against the target kernel
>> 
>> GCC BPF is only tested on x86_64.
>> 
>> On x86_64 we test the following toolchains for building the kernel and
>> test runners: gcc-13 (ubuntu 24 default), clang-17, clang-18.
>> 
>> An example of successful test run (you have to login to github to see
>> the logs):
>> https://github.com/kernel-patches/bpf/actions/runs/12816136141/job/35736973856
>> 
>> Currently 2513 of 4340 tests pass for GCC BPF, so a bit more than a half.
>> 
>> Effective BPF selftests denylist for GCC BPF is located here:
>> https://github.com/kernel-patches/vmtest/blob/master/ci/vmtest/configs/DENYLIST.test_progs-bpf_gcc
>
> The announcement triggered an off-list discussion among BPF devs about
> how to handle the test running, given the long denylist.
>
> The problem is that any new test is now a potential subject to
> debugging whether the test needs changes, or GCC doesn't work for it.
>
> As of now, an important missing piece on GCC side is the decl_tags
> support, as they are heavily used by BPF selftests. See a message from
> Yonghong Song:
> https://gcc.gnu.org/pipermail/gcc-patches/2025-January/673841.html
>
> Some discussed suggestions:
>   * Run test_progs-bpf_gcc with "allowed to fail", so that the
>     pipeline is never blocked
>   * Only run GCC BPF *compilation*, and don't execute the tests

I think that this is the best solution for now, and the most useful.

As soon as we achieve passing all the selftests (hopefully soon) then we
can change the CI to flag regressions on test run failures as well.


>   * Flip denylist to allowlist to prevent regressions, but not force
>     new tests to work with GCC
>
> Input from GCC devs will be much appreciated.
>
> Thanks.
>
>> 
>> When a patch is submitted to BPF, normally a corresponding PR for
>> kernel-patches/bpf github repo is automatically created to trigger a
>> BPF CI run for this change. PRs opened manually will do that too, and
>> this can be used to test patches before submission.
>> 
>> Since the CI automatically pulls latest GCC snapshot, a change in GCC
>> can potentially cause CI failures unrelated to Linux changes being
>> tested. This is not the only dependency like that, of course.
>> 
>> In such situations, a change is usually made in CI code to mitigate
>> the failure in order to unblock the pipeline for patches. If that
>> happens with GCC, someone (most likely me) will have to reach out to
>> GCC team. I guess gcc@gcc.gnu.org would be the default point of
>> contact, but if there are specific people who should be notified
>> please let me know.

Re: Announcement: GCC BPF is now being tested on BPF CI

[Jose E. Marchesi]

Thank you for getting this up and running!

> Hi everyone.
>
> GCC BPF support in BPF CI has been landed.
>
> The BPF CI dashboard is here:
> https://github.com/kernel-patches/bpf/actions/workflows/test.yml
>
> A summary of what happens on CI (relevant to GCC BPF):
>   * Linux Kernel is built on a target source revision
>   * Latest snapshots of GCC 15 and binutils are downloaded
>     * GCC BPF compiler is built and cached
>   * selftests/bpf test runners are built with BPF_GCC variable set
>     * BPF_GCC triggers a build of test_progs-bpf_gcc runner
>     * The runner contains BPF binaries produced by GCC BPF
>   * In a separate job, test_progs-bpf_gcc is executed within qemu
>     against the target kernel
>
> GCC BPF is only tested on x86_64.
>
> On x86_64 we test the following toolchains for building the kernel and
> test runners: gcc-13 (ubuntu 24 default), clang-17, clang-18.
>
> An example of successful test run (you have to login to github to see
> the logs):
> https://github.com/kernel-patches/bpf/actions/runs/12816136141/job/35736973856
>
> Currently 2513 of 4340 tests pass for GCC BPF, so a bit more than a half.
>
> Effective BPF selftests denylist for GCC BPF is located here:
> https://github.com/kernel-patches/vmtest/blob/master/ci/vmtest/configs/DENYLIST.test_progs-bpf_gcc
>
> When a patch is submitted to BPF, normally a corresponding PR for
> kernel-patches/bpf github repo is automatically created to trigger a
> BPF CI run for this change. PRs opened manually will do that too, and
> this can be used to test patches before submission.
>
> Since the CI automatically pulls latest GCC snapshot, a change in GCC
> can potentially cause CI failures unrelated to Linux changes being
> tested. This is not the only dependency like that, of course.
>
> In such situations, a change is usually made in CI code to mitigate
> the failure in order to unblock the pipeline for patches. If that
> happens with GCC, someone (most likely me) will have to reach out to
> GCC team. I guess gcc@gcc.gnu.org would be the default point of
> contact, but if there are specific people who should be notified
> please let me know.

Re: Announcement: GCC BPF is now being tested on BPF CI

[Jose E. Marchesi]

> As of now, an important missing piece on GCC side is the decl_tags
> support, as they are heavily used by BPF selftests. See a message from
> Yonghong Song:
> https://gcc.gnu.org/pipermail/gcc-patches/2025-January/673841.html

Faust is working on a V2 of that patch series that will be submitted to
gcc-patches soon.

Re: Announcement: GCC BPF is now being tested on BPF CI

[Ihor Solodrai]

On Thursday, January 16th, 2025 at 3:58 PM, Jose E. Marchesi <jose.marchesi@oracle.com> wrote:

> 
> [...]
> > > 
> > > Effective BPF selftests denylist for GCC BPF is located here:
> > > https://github.com/kernel-patches/vmtest/blob/master/ci/vmtest/configs/DENYLIST.test_progs-bpf_gcc
> > 
> > The announcement triggered an off-list discussion among BPF devs about
> > how to handle the test running, given the long denylist.
> > 
> > The problem is that any new test is now a potential subject to
> > debugging whether the test needs changes, or GCC doesn't work for it.
> > 
> > As of now, an important missing piece on GCC side is the decl_tags
> > support, as they are heavily used by BPF selftests. See a message from
> > Yonghong Song:
> > https://gcc.gnu.org/pipermail/gcc-patches/2025-January/673841.html
> > 
> > Some discussed suggestions:
> > * Run test_progs-bpf_gcc with "allowed to fail", so that the
> > pipeline is never blocked
> > * Only run GCC BPF compilation, and don't execute the tests
> 
> 
> I think that this is the best solution for now, and the most useful.
> 
> As soon as we achieve passing all the selftests (hopefully soon) then we
> can change the CI to flag regressions on test run failures as well.

Ok. I disabled the execution of the test_progs-bpf_gcc test runner for now.

I think we should check on the state of the tests again after decl_tags
support is landed.

Thanks.

> 
> > * Flip denylist to allowlist to prevent regressions, but not force
> > new tests to work with GCC
> > 
> > Input from GCC devs will be much appreciated.
> > 
> > Thanks.
> > 
> > > When a patch is submitted to BPF, normally a corresponding PR for
> > > kernel-patches/bpf github repo is automatically created to trigger a
> > > BPF CI run for this change. PRs opened manually will do that too, and
> > > this can be used to test patches before submission.
> > > 
> > > Since the CI automatically pulls latest GCC snapshot, a change in GCC
> > > can potentially cause CI failures unrelated to Linux changes being
> > > tested. This is not the only dependency like that, of course.
> > > 
> > > In such situations, a change is usually made in CI code to mitigate
> > > the failure in order to unblock the pipeline for patches. If that
> > > happens with GCC, someone (most likely me) will have to reach out to
> > > GCC team. I guess gcc@gcc.gnu.org would be the default point of
> > > contact, but if there are specific people who should be notified
> > > please let me know.

Re: Announcement: GCC BPF is now being tested on BPF CI

[Cupertino Miranda]

I remind that just as bad as the decl_tags it also misses a solution to 
the attribute ((preserve_access_index)).
Something like #pragma clang push/pop is not viable in GCC.

Jose proposed the patch in:
  
https://lore.kernel.org/bpf/20240503111836.25275-1-jose.marchesi@oracle.com/

Maybe you could accept his patch in the meanwhile, and work on the 
intended improvements later. It would be passing more tests then roughly 
half.

Thanks

> 
> Thank you for getting this up and running!
> 
>> Hi everyone.
>>
>> GCC BPF support in BPF CI has been landed.
>>
>> The BPF CI dashboard is here:
>> https://github.com/kernel-patches/bpf/actions/workflows/test.yml
>>
>> A summary of what happens on CI (relevant to GCC BPF):
>>    * Linux Kernel is built on a target source revision
>>    * Latest snapshots of GCC 15 and binutils are downloaded
>>      * GCC BPF compiler is built and cached
>>    * selftests/bpf test runners are built with BPF_GCC variable set
>>      * BPF_GCC triggers a build of test_progs-bpf_gcc runner
>>      * The runner contains BPF binaries produced by GCC BPF
>>    * In a separate job, test_progs-bpf_gcc is executed within qemu
>>      against the target kernel
>>
>> GCC BPF is only tested on x86_64.
>>
>> On x86_64 we test the following toolchains for building the kernel and
>> test runners: gcc-13 (ubuntu 24 default), clang-17, clang-18.
>>
>> An example of successful test run (you have to login to github to see
>> the logs):
>> https://github.com/kernel-patches/bpf/actions/runs/12816136141/job/35736973856
>>
>> Currently 2513 of 4340 tests pass for GCC BPF, so a bit more than a half.
>>
>> Effective BPF selftests denylist for GCC BPF is located here:
>> https://github.com/kernel-patches/vmtest/blob/master/ci/vmtest/configs/DENYLIST.test_progs-bpf_gcc
>>
>> When a patch is submitted to BPF, normally a corresponding PR for
>> kernel-patches/bpf github repo is automatically created to trigger a
>> BPF CI run for this change. PRs opened manually will do that too, and
>> this can be used to test patches before submission.
>>
>> Since the CI automatically pulls latest GCC snapshot, a change in GCC
>> can potentially cause CI failures unrelated to Linux changes being
>> tested. This is not the only dependency like that, of course.
>>
>> In such situations, a change is usually made in CI code to mitigate
>> the failure in order to unblock the pipeline for patches. If that
>> happens with GCC, someone (most likely me) will have to reach out to
>> GCC team. I guess gcc@gcc.gnu.org would be the default point of
>> contact, but if there are specific people who should be notified
>> please let me know.

Re: Announcement: GCC BPF is now being tested on BPF CI

[Jose E. Marchesi]

> On Thursday, January 16th, 2025 at 3:58 PM, Jose E. Marchesi <jose.marchesi@oracle.com> wrote:
>
>> 
>> [...]
>> > > 
>> > > Effective BPF selftests denylist for GCC BPF is located here:
>> > > https://github.com/kernel-patches/vmtest/blob/master/ci/vmtest/configs/DENYLIST.test_progs-bpf_gcc
>> > 
>> > The announcement triggered an off-list discussion among BPF devs about
>> > how to handle the test running, given the long denylist.
>> > 
>> > The problem is that any new test is now a potential subject to
>> > debugging whether the test needs changes, or GCC doesn't work for it.
>> > 
>> > As of now, an important missing piece on GCC side is the decl_tags
>> > support, as they are heavily used by BPF selftests. See a message from
>> > Yonghong Song:
>> > https://gcc.gnu.org/pipermail/gcc-patches/2025-January/673841.html
>> > 
>> > Some discussed suggestions:
>> > * Run test_progs-bpf_gcc with "allowed to fail", so that the
>> > pipeline is never blocked
>> > * Only run GCC BPF compilation, and don't execute the tests
>> 
>> 
>> I think that this is the best solution for now, and the most useful.
>> 
>> As soon as we achieve passing all the selftests (hopefully soon) then we
>> can change the CI to flag regressions on test run failures as well.
>
> Ok. I disabled the execution of the test_progs-bpf_gcc test runner for now.
>
> I think we should check on the state of the tests again after decl_tags
> support is landed.

Thank you.  Sounds like a plan :)

Is it possible to configure the CI to send an email to certain
recipients when the build of the selftests with GCC fails?  That would
help us to keep an eye on the patches and either fix GCC or provide
advise on how to fix the selftest in case it contains bad C.

>
> Thanks.
>
>> 
>> > * Flip denylist to allowlist to prevent regressions, but not force
>> > new tests to work with GCC
>> > 
>> > Input from GCC devs will be much appreciated.
>> > 
>> > Thanks.
>> > 
>> > > When a patch is submitted to BPF, normally a corresponding PR for
>> > > kernel-patches/bpf github repo is automatically created to trigger a
>> > > BPF CI run for this change. PRs opened manually will do that too, and
>> > > this can be used to test patches before submission.
>> > > 
>> > > Since the CI automatically pulls latest GCC snapshot, a change in GCC
>> > > can potentially cause CI failures unrelated to Linux changes being
>> > > tested. This is not the only dependency like that, of course.
>> > > 
>> > > In such situations, a change is usually made in CI code to mitigate
>> > > the failure in order to unblock the pipeline for patches. If that
>> > > happens with GCC, someone (most likely me) will have to reach out to
>> > > GCC team. I guess gcc@gcc.gnu.org would be the default point of
>> > > contact, but if there are specific people who should be notified
>> > > please let me know.

Re: Announcement: GCC BPF is now being tested on BPF CI

[Mark Wielaard]

Hi Ihor,

On Thu, Jan 16, 2025 at 08:44:54PM +0000, Ihor Solodrai via Gcc wrote:
> An example of successful test run (you have to login to github to see
> the logs):
> https://github.com/kernel-patches/bpf/actions/runs/12816136141/job/35736973856
> 
> Currently 2513 of 4340 tests pass for GCC BPF, so a bit more than a half.

Nice. Could you make the logs public so people don't have to create a
github account? Or post the results to gcc-testresults@gcc.gnu.org so
others can easily inspect them. You can also submit them to bunsen of
course.

Thanks,

Mark

Re: Announcement: GCC BPF is now being tested on BPF CI

[Ihor Solodrai]

On Friday, January 17th, 2025 at 2:44 AM, Jose E. Marchesi <jose.marchesi@oracle.com> wrote:

> [...]
>
> > Ok. I disabled the execution of the test_progs-bpf_gcc test runner for now.
> > 
> > I think we should check on the state of the tests again after decl_tags
> > support is landed.
> 
> 
> Thank you. Sounds like a plan :)
> 
> Is it possible to configure the CI to send an email to certain
> recipients when the build of the selftests with GCC fails? That would
> help us to keep an eye on the patches and either fix GCC or provide
> advise on how to fix the selftest in case it contains bad C.

In principle, yes. In practice email notifications are not that
straightforward.

Currently a BPF patch submitter gets a notification about the status
of the CI pipeline for their patch. This makes sense, recipient is
obvious in this case.

In case of GCC (or any other CI dependency for that matter), it is
necessary to determine the potential cause before sending
notifications. There are all kinds of things that might have caused a
failure independent of the target being tested: could be a bug in CI
scripts, or github could have changed runner configuration, or a merge
commit from (Linux) upstream broke something, etc.

Point is, dependency maintainers (GCC team in this case) don't want to
get notifications for *all* such failures, because you will have to
ignore most of them, and so they become noise. A boy crying wolf kind
of thing.

The other issue is that maintaining email notifications is an
operational overhead, meaning that the system managing the
notifications needs to be looked after. Currently for BPF CI it's
Kernel Patches Daemon instance maintained by Meta engineers [1].

As it stands, if there is problem with GCC that affects BPF CI, you
can be assured it'll be reported, because it will block the testing of
the BPF patches.

I suggest GCC BPF team to think about setting up your own automated
testing infrastructure, focused on testing the GCC compiler. Maybe you
already have something like that, I don't know. You certainly
shouldn't rely exclusively on BPF CI for testing the BPF backend.

[1] https://github.com/facebookincubator/kernel-patches-daemon

> 
> > Thanks.
> > 
> > [...]

Re: Announcement: GCC BPF is now being tested on BPF CI

[Andrii Nakryiko]

On Fri, Jan 17, 2025 at 12:34 AM Cupertino Miranda
<cupertino.miranda@oracle.com> wrote:
>
> I remind that just as bad as the decl_tags it also misses a solution to
> the attribute ((preserve_access_index)).
> Something like #pragma clang push/pop is not viable in GCC.
>
> Jose proposed the patch in:
>
> https://lore.kernel.org/bpf/20240503111836.25275-1-jose.marchesi@oracle.com/
>

Ihor is working on an alternative, cleaner, and much more generic
solution that would make it unnecessary to add custom options for
specific cases like this. Stay tuned.

> Maybe you could accept his patch in the meanwhile, and work on the
> intended improvements later. It would be passing more tests then roughly
> half.
>
> Thanks
>
> >
> > Thank you for getting this up and running!
> >
> >> Hi everyone.
> >>
> >> GCC BPF support in BPF CI has been landed.
> >>
> >> The BPF CI dashboard is here:
> >> https://github.com/kernel-patches/bpf/actions/workflows/test.yml
> >>
> >> A summary of what happens on CI (relevant to GCC BPF):
> >>    * Linux Kernel is built on a target source revision
> >>    * Latest snapshots of GCC 15 and binutils are downloaded
> >>      * GCC BPF compiler is built and cached
> >>    * selftests/bpf test runners are built with BPF_GCC variable set
> >>      * BPF_GCC triggers a build of test_progs-bpf_gcc runner
> >>      * The runner contains BPF binaries produced by GCC BPF
> >>    * In a separate job, test_progs-bpf_gcc is executed within qemu
> >>      against the target kernel
> >>
> >> GCC BPF is only tested on x86_64.
> >>
> >> On x86_64 we test the following toolchains for building the kernel and
> >> test runners: gcc-13 (ubuntu 24 default), clang-17, clang-18.
> >>
> >> An example of successful test run (you have to login to github to see
> >> the logs):
> >> https://github.com/kernel-patches/bpf/actions/runs/12816136141/job/35736973856
> >>
> >> Currently 2513 of 4340 tests pass for GCC BPF, so a bit more than a half.
> >>
> >> Effective BPF selftests denylist for GCC BPF is located here:
> >> https://github.com/kernel-patches/vmtest/blob/master/ci/vmtest/configs/DENYLIST.test_progs-bpf_gcc
> >>
> >> When a patch is submitted to BPF, normally a corresponding PR for
> >> kernel-patches/bpf github repo is automatically created to trigger a
> >> BPF CI run for this change. PRs opened manually will do that too, and
> >> this can be used to test patches before submission.
> >>
> >> Since the CI automatically pulls latest GCC snapshot, a change in GCC
> >> can potentially cause CI failures unrelated to Linux changes being
> >> tested. This is not the only dependency like that, of course.
> >>
> >> In such situations, a change is usually made in CI code to mitigate
> >> the failure in order to unblock the pipeline for patches. If that
> >> happens with GCC, someone (most likely me) will have to reach out to
> >> GCC team. I guess gcc@gcc.gnu.org would be the default point of
> >> contact, but if there are specific people who should be notified
> >> please let me know.
>

Re: Announcement: GCC BPF is now being tested on BPF CI

[Sam James]

Ihor Solodrai via Gcc <gcc@gcc.gnu.org> writes:

> On Friday, January 17th, 2025 at 2:44 AM, Jose E. Marchesi <jose.marchesi@oracle.com> wrote:
>
>> [...]
>>
>> > Ok. I disabled the execution of the test_progs-bpf_gcc test runner for now.
>> > 
>> > I think we should check on the state of the tests again after decl_tags
>> > support is landed.
>> 
>> 
>> Thank you. Sounds like a plan :)
>> 
>> Is it possible to configure the CI to send an email to certain
>> recipients when the build of the selftests with GCC fails? That would
>> help us to keep an eye on the patches and either fix GCC or provide
>> advise on how to fix the selftest in case it contains bad C.
>
> In principle, yes. In practice email notifications are not that
> straightforward.
>
> Currently a BPF patch submitter gets a notification about the status
> of the CI pipeline for their patch. This makes sense, recipient is
> obvious in this case.
>
> In case of GCC (or any other CI dependency for that matter), it is
> necessary to determine the potential cause before sending
> notifications. There are all kinds of things that might have caused a
> failure independent of the target being tested: could be a bug in CI
> scripts, or github could have changed runner configuration, or a merge
> commit from (Linux) upstream broke something, etc.
>
> Point is, dependency maintainers (GCC team in this case) don't want to
> get notifications for *all* such failures, because you will have to
> ignore most of them, and so they become noise. A boy crying wolf kind
> of thing.
>
> The other issue is that maintaining email notifications is an
> operational overhead, meaning that the system managing the
> notifications needs to be looked after. Currently for BPF CI it's
> Kernel Patches Daemon instance maintained by Meta engineers [1].
>
> As it stands, if there is problem with GCC that affects BPF CI, you
> can be assured it'll be reported, because it will block the testing of
> the BPF patches.
>
> I suggest GCC BPF team to think about setting up your own automated
> testing infrastructure, focused on testing the GCC compiler. Maybe you
> already have something like that, I don't know. You certainly
> shouldn't rely exclusively on BPF CI for testing the BPF backend.

I think Jose is asking from the angle of wanting to make GCC support as
painless as possible for you upstream, not to ask you to provide a
substitute for our own CI. i.e. We don't want you to feel burdened by
providing that and we're happy to look into any problems as soon as they
arsie.

>
> [1] https://github.com/facebookincubator/kernel-patches-daemon
>
>> 
>> > Thanks.
>> > 
>> > [...]

Re: Announcement: GCC BPF is now being tested on BPF CI

[Ihor Solodrai]

On Friday, January 17th, 2025 at 5:44 AM, Mark Wielaard <mark@klomp.org> wrote:

> 
> 
> Hi Ihor,
> 
> On Thu, Jan 16, 2025 at 08:44:54PM +0000, Ihor Solodrai via Gcc wrote:
> 
> > An example of successful test run (you have to login to github to see
> > the logs):
> > https://github.com/kernel-patches/bpf/actions/runs/12816136141/job/35736973856
> > 
> > Currently 2513 of 4340 tests pass for GCC BPF, so a bit more than a half.
> 
> 
> Nice. Could you make the logs public so people don't have to create a
> github account? Or post the results to gcc-testresults@gcc.gnu.org so
> others can easily inspect them. You can also submit them to bunsen of
> course.

Hi Mark,

Well, re-publishing logs somewhere is certainly possible, however
since BPF CI runs on Github Actions there was never a need for
something like this.

When I got the selftests compilation working, I've shared the logs
produced by running the tests individually [1][2] through github. It's
72Mb of logs, and sending them directly to email isn't a good idea.

I don't know what's bunsen. Can you share a pointer?

Thanks.

[1] https://lore.kernel.org/bpf/EYcXjcKDCJY7Yb0GGtAAb7nLKPEvrgWdvWpuNzXm2qi6rYMZDixKv5KwfVVMBq17V55xyC-A1wIjrqG3aw-Imqudo9q9X7D7nLU2gWgbN0w=@pm.me/
[2] https://github.com/kernel-patches/bpf/blob/8f2e62702ee17675464ab00d97d89d599922de20/tools/testing/selftests/bpf/gcc-bpf-selftests-logs.tgz

> 
> Thanks,
> 
> Mark