From: Matthieu Baerts <matttbe@kernel.org>
To: Sheena Mohan <sheenamo@google.com>
Cc: Willem de Bruijn <willemb@google.com>,
Netdev <netdev@vger.kernel.org>, Jakub Kicinski <kuba@kernel.org>
Subject: Re: [TEST] Google's Netdev-CI: a few flaky tests
Date: Mon, 26 Oct 2026 17:41:51 +0100 [thread overview]
Message-ID: <a9e4655a-0a12-4b40-9641-36e7acc2cdcd@kernel.org> (raw)
In-Reply-To: <CADWJPTundpvzkpJrxva=M+6YF4zQ9dRSY78Xypi9B1NuH58Usw@mail.gmail.com>
Hi Sheena,
On 24/10/2026 19:49, Sheena Mohan wrote:
> On Fri, Oct 23, 2026 at 9:02 AM Matthieu Baerts <matttbe@kernel.org> wrote:
>>
>> Hi Sheena,
>>
>> I hope you are doing well.
>>
>> While looking at the results on the Netdev CI for the Google runners, I
>> notice some tests are quite flaky or constantly failing. I marked them
>> as "ignored" on the Netdev CI while this is being investigated.
>>
>> - IDPF: psp-py started to a month ago. I guess you are monitoring the
>> results, but then if it is normal, it might be easier if you block the
>> known bad tests from running on your side. Just in case, a few links:
>>
>> https://netdev.bots.linux.dev/flakes.html?tn-needle=psp-py&br-pfx=net-next-hw&ld-remote=google-idpf
>>
>> Because the previous branch didn't get tested, the regression might be
>> due to the modifications introduced in the previous branch as well:
>>
>> https://netdev.bots.linux.dev/branch_deltas/net-next-hw-2026-09-23--16-00.html
>> https://netdev.bots.linux.dev/branch_deltas/net-next-hw-2026-09-23--08-00.html
>>
>>
>> - GVE: "queues-py" test started to fail recently: I *think* it is a
>> regression due to the following patch that has just been dropped, but
>> we will only know that in a couple of hours:
>>
>> https://lore.kernel.org/20261021155611.68473-1-phx0fer@gmail.com
>>
>> https://netdev.bots.linux.dev/contest.html?remote=google-gve&executor=gve-nipa&test=queues-py
>>
>> https://netdev.bots.linux.dev/branch_deltas/net-next-hw-2026-10-22--00-00.html
>>
>>
>> - GVE: A few tests are unstable for a while and have been ignored: do
>> you mind looking at them, please?
>>
>> https://netdev.bots.linux.dev/flakes.html?min-flip=0&pw-y=0&br-pfx=net-next-hw&ld-remote=google-gve
>>
>>
>> - IDPF: sometimes, it takes ~4h to run the results (+1h to start).
>> That's fine because we have a 6h window, but just wondering: is it
>> normal? (The GVE runner seems to start quicker, but I guess that's
>> because one is on bare metal and the other in a VM)
>>
>> Thank you for your help!
>>
>> Cheers,
>> Matt
>
> Hi Matt,
>
> Thanks for reaching out and flagging these issues. Here is an update
> on each of the items:
>
> IDPF
>
> - psp-py: We agree that blocking known bad tests on our side is
> best. While we resolve these
> failures, we will disable the failing test cases to reduce noise
> on the CI dashboard.
> - Run/Setup Duration: The longer setup and execution times are due
> to running in a bare metal
> environment. We are looking into optimizing this to reduce the duration.
Thank you!
> GVE
>
> - queues-py: We also noticed this failure recently. Since it is
> suspected to be a kernel
> regression, we will continue monitoring the results.
It looks like the issue has been fixed when the bad patch has been removed.
> - Unstable/Ignored Tests:
> - devmem-py: Should be fixed by patch 4 of the series recently
> sent upstream:
> https://lore.kernel.org/netdev/20261023204706.3112215-5-hramamurthy@google.com/T/#m0f075c545d9de1c0ad3bd95da0f009a82e80b6c5
Great! Indeed, it looks good so far.
> - napi-id-py & bond-options-sh: We will investigate both tests.
When fixes are applied or unstable tests are blocked on your side, do
you mind pinging me so I can remove them from this ignore list, please?
https://netdev.bots.linux.dev/contest/filters-hw.json
Cheers,
Matt
next prev parent reply other threads:[~2026-10-26 16:41 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-23 16:02 [TEST] Google's Netdev-CI: a few flaky tests Matthieu Baerts
2026-10-24 17:49 ` Sheena Mohan
2026-10-26 16:41 ` Matthieu Baerts [this message]
2026-11-20 3:12 ` Pin-yen Lin
2026-11-20 9:09 ` Matthieu Baerts
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a9e4655a-0a12-4b40-9641-36e7acc2cdcd@kernel.org \
--to=matttbe@kernel.org \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=sheenamo@google.com \
--cc=willemb@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.