Re: [TEST] Flake report

[Simon Horman <horms@kernel.org>]

+ Aaron

On Thu, May 09, 2024 at 04:09:58PM -0700, Jakub Kicinski wrote:
> Hi!
> 
> Feels like the efforts to get rid of flaky tests have slowed down a bit,
> so I thought I'd poke people..
> 
> Here's the full list:
> https://netdev.bots.linux.dev/flakes.html?min-flip=0&pw-y=0
> click on test name to get the list of runs and links to outputs.
> 
> As a reminder please see these instructions for repro:
> https://github.com/linux-netdev/nipa/wiki/How-to-run-netdev-selftests-CI-style
> 
> I'll try to tag folks who touched the tests most recently, but please
> don't hesitate to chime in.
> 
> 
> net
> ---
> 
> arp-ndisc-untracked-subnets-sh
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> To: Jaehee Park <jhpark1013@gmail.com>
> Cc: Hangbin Liu <liuhangbin@gmail.com>
> 
> Times out on debug kernels, passes on non-debug.
> This is a real timeout, eats full 7200 seconds.
> 
> xfrm-policy-sh
> ~~~~~~~~~~~~~~
> To: Hangbin Liu <liuhangbin@gmail.com>
> 
> Times out on debug kernels, passed on non-debug,
> This is a "inactivity" timeout, test doesn't print anything
> for 900 seconds so the runner kills it. We can bump the timeout
> but not printing for 15min is bad..
> 
> cmsg-time-sh
> ~~~~~~~~~~~~
> To: Jakub Kicinski <kuba@kernel.org> (forgot I wrote this :D)
> 
> Fails randomly.
> 
> pmtu-sh
> ~~~~~~~
> To: Simon Horman <horms@kernel.org>
> 
> Skipped because it wants full OVS tooling.

My understanding is that Aaron (CCed) is working on addressing
this problem by allowing the test to run without full OVS tooling.

> forwarding
> ----------
> 
> sch-tbf-ets-sh, sch-tbf-prio-sh
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> To: Petr Machata <petrm@nvidia.com>
> 
> These fail way too often on non-debug kernels :(
> Perhaps we can extend the lower bound?
> 
> bridge-igmp-sh, bridge-mld-sh
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> To: Nikolay Aleksandrov <razor@blackwall.org>
> Cc: Ido Schimmel <idosch@nvidia.com>
> 
> On debug kernels it always fails with:
> 
> # TEST: IGMPv3 group 239.10.10.10 exclude timeout                     [FAIL]
> # Entry 192.0.2.21 has blocked flag failed
> 
> For MLD:
> 
> # TEST: MLDv2 group ff02::cc exclude timeout                          [FAIL]
> # Entry 2001:db8:1::21 has blocked flag failed
> 
> vxlan-bridge-1d-sh
> ~~~~~~~~~~~~~~~~~~
> To: Ido Schimmel <idosch@nvidia.com>
> Cc: Petr Machata <petrm@nvidia.com>
> 
> Flake fails almost always, with some form of "Expected to capture 0
> packets, got $X"
> 
> mirror-gre-lag-lacp-sh
> ~~~~~~~~~~~~~~~~~~~~~~
> To: Petr Machata <petrm@nvidia.com>
> 
> Often fails on debug with:
> 
> # TEST: mirror to gretap: LAG first slave (skip_hw)                   [FAIL]
> # Expected to capture 10 packets, got 13.
> 
> mirror-gre-vlan-bridge-1q-sh, mirror-gre-bridge-1d-vlan-sh
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> To: Petr Machata <petrm@nvidia.com>
> 
> Same kind of failure as above but less often and both on debug and non-debug.
> 
> tc-actions-sh
> ~~~~~~~~~~~~~
> To: Davide Caratti <dcaratti@redhat.com>
> 
> It triggers a random unhandled interrupt, somehow (look at stderr).
> It's the only test that does that.
> 
> 
> mptcp
> -----
> To: Matthieu Baerts <matttbe@kernel.org>
> 
> simult-flows-sh is still quite flaky :(
> 
> 
> nf
> --
> To: Florian Westphal <fw@strlen.de>
> 
> These are skipped because of some compatibility issues:
> 
>  nft-flowtable-sh, bridge-brouter-sh, nft-audit-sh
> 
> Please LMK if I need to update the CLI tooling. 
> Or is this missing kernel config?
>