[PATCH net-next] selftests: forwarding: Add missing config entries

[PATCH net-next] selftests: forwarding: Add missing config entries

[Petr Machata]

The config file contains a partial kernel configuration to be used by
`virtme-configkernel --custom'. The presumption is that the config file
contains all Kconfig options needed by the selftests from the directory.

In net/forwarding/config, many are missing, which manifests as spurious
failures when running the selftests, with messages about unknown device
types, qdisc kinds or classifier actions. Add the missing configurations.

Tested the resulting configuration using virtme-ng as follows:

 # vng -b -f tools/testing/selftests/net/forwarding/config
 # vng --user root
 (within the VM:)
 # make -C tools/testing/selftests TARGETS=net/forwarding run_tests

Signed-off-by: Petr Machata <petrm@nvidia.com>
---
 tools/testing/selftests/net/forwarding/config | 28 +++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/tools/testing/selftests/net/forwarding/config b/tools/testing/selftests/net/forwarding/config
index 697994a9278b..ba2343514582 100644
--- a/tools/testing/selftests/net/forwarding/config
+++ b/tools/testing/selftests/net/forwarding/config
@@ -6,14 +6,42 @@ CONFIG_IPV6_MULTIPLE_TABLES=y
 CONFIG_NET_VRF=m
 CONFIG_BPF_SYSCALL=y
 CONFIG_CGROUP_BPF=y
+CONFIG_DUMMY=m
+CONFIG_IPV6=y
+CONFIG_IPV6_GRE=m
+CONFIG_MACVLAN=m
 CONFIG_NET_ACT_CT=m
 CONFIG_NET_ACT_MIRRED=m
 CONFIG_NET_ACT_MPLS=m
+CONFIG_NET_ACT_PEDIT=m
+CONFIG_NET_ACT_POLICE=m
+CONFIG_NET_ACT_SAMPLE=m
+CONFIG_NET_ACT_SKBEDIT=m
+CONFIG_NET_ACT_TUNNEL_KEY=m
 CONFIG_NET_ACT_VLAN=m
 CONFIG_NET_CLS_FLOWER=m
 CONFIG_NET_CLS_MATCHALL=m
+CONFIG_NET_CLS_BASIC=m
+CONFIG_NET_EMATCH=y
+CONFIG_NET_EMATCH_META=m
+CONFIG_NET_IPGRE=m
+CONFIG_NET_IPGRE_DEMUX=m
+CONFIG_NET_IPIP=m
+CONFIG_NET_SCH_ETS=m
 CONFIG_NET_SCH_INGRESS=m
 CONFIG_NET_ACT_GACT=m
+CONFIG_NET_SCH_PRIO=m
+CONFIG_NET_SCH_RED=m
+CONFIG_NET_SCH_TBF=m
+CONFIG_NET_TC_SKB_EXT=y
+CONFIG_NET_TEAM=y
+CONFIG_NET_TEAM_MODE_LOADBALANCE=y
+CONFIG_NETFILTER=y
+CONFIG_NF_CONNTRACK=m
+CONFIG_NF_FLOW_TABLE=m
+CONFIG_NF_TABLES=m
 CONFIG_VETH=m
 CONFIG_NAMESPACES=y
 CONFIG_NET_NS=y
+CONFIG_VXLAN=m
+CONFIG_XFRM_USER=m
-- 
2.43.0

Re: [PATCH net-next] selftests: forwarding: Add missing config entries

[Jakub Kicinski]

On Fri, 26 Jan 2024 17:36:16 +0100 Petr Machata wrote:
> The config file contains a partial kernel configuration to be used by
> `virtme-configkernel --custom'. The presumption is that the config file
> contains all Kconfig options needed by the selftests from the directory.
> 
> In net/forwarding/config, many are missing, which manifests as spurious
> failures when running the selftests, with messages about unknown device
> types, qdisc kinds or classifier actions. Add the missing configurations.
> 
> Tested the resulting configuration using virtme-ng as follows:
> 
>  # vng -b -f tools/testing/selftests/net/forwarding/config
>  # vng --user root
>  (within the VM:)
>  # make -C tools/testing/selftests TARGETS=net/forwarding run_tests

Thanks a lot for fixing this stuff! The patch went into the
net-next-2024-01-26--18-00 branch we got: pass 94 / skip 2 / fail 15

https://netdev.bots.linux.dev/contest.html?branch=net-next-2024-01-26--18-00&executor=vmksft-forwarding&pw-y=0

Clicking thru a handful of the failures it looks like it's about a 50/50
split between timeouts and perf mismatch. 

BTW when it says: "WAIT TIMEOUT stdout" that means the runner got angry
because the test didn't print anything for 20 minutes. I'll bump this
timeout to an hour for the next branch, and lets see how many failures
we have left.

Re: [PATCH net-next] selftests: forwarding: Add missing config entries

[Petr Machata]

Jakub Kicinski <kuba@kernel.org> writes:

> On Fri, 26 Jan 2024 17:36:16 +0100 Petr Machata wrote:
>> The config file contains a partial kernel configuration to be used by
>> `virtme-configkernel --custom'. The presumption is that the config file
>> contains all Kconfig options needed by the selftests from the directory.
>> 
>> In net/forwarding/config, many are missing, which manifests as spurious
>> failures when running the selftests, with messages about unknown device
>> types, qdisc kinds or classifier actions. Add the missing configurations.
>> 
>> Tested the resulting configuration using virtme-ng as follows:
>> 
>>  # vng -b -f tools/testing/selftests/net/forwarding/config
>>  # vng --user root
>>  (within the VM:)
>>  # make -C tools/testing/selftests TARGETS=net/forwarding run_tests
>
> Thanks a lot for fixing this stuff! The patch went into the
> net-next-2024-01-26--18-00 branch we got: pass 94 / skip 2 / fail 15
>
> https://netdev.bots.linux.dev/contest.html?branch=net-next-2024-01-26--18-00&executor=vmksft-forwarding&pw-y=0
>
> Clicking thru a handful of the failures it looks like it's about a 50/50
> split between timeouts and perf mismatch. 

Looking at some recent runs. A number of failures are probably due to
the system failing to oversubscribe the interface with the tested
qdiscs. That's sch_ets, sch_tbf_ets, sch_tbf_prio, sch_tbf_root,
tc_police.

Not sure what to do about it. Maybe separate out heavy traffic tests,
and add a make run_lotraf_tests?


tc_actions started getting a passible deadlocking warning between Jan 27
00:37 and Jan 28 18:27:

    https://netdev-2.bots.linux.dev/vmksft-forwarding/results/438201/108-tc-actions-sh/
    https://netdev-2.bots.linux.dev/vmksft-forwarding/results/438566/109-tc-actions-sh/

So either something landed that broke it, or the host kernel now has
more debugging enabled, so it now gives a citation.


ip6gre_inner_v6_multipath is just noisy? It failed the last run, but
passed several before.


router_multicast and router get a complaint about a missing control
socket. I think at first approximation they need:

    # mkdir -p /usr/local/var/run

But even then I'm getting a fail. This and the others seem to all be in
IPv6 multicast.

Re: [PATCH net-next] selftests: forwarding: Add missing config entries

[Jakub Kicinski]

On Mon, 29 Jan 2024 11:45:07 +0100 Petr Machata wrote:
> Jakub Kicinski <kuba@kernel.org> writes:
> > Thanks a lot for fixing this stuff! The patch went into the
> > net-next-2024-01-26--18-00 branch we got: pass 94 / skip 2 / fail 15
> >
> > https://netdev.bots.linux.dev/contest.html?branch=net-next-2024-01-26--18-00&executor=vmksft-forwarding&pw-y=0
> >
> > Clicking thru a handful of the failures it looks like it's about a 50/50
> > split between timeouts and perf mismatch.   
> 
> Looking at some recent runs. A number of failures are probably due to
> the system failing to oversubscribe the interface with the tested
> qdiscs. That's sch_ets, sch_tbf_ets, sch_tbf_prio, sch_tbf_root,
> tc_police.
> 
> Not sure what to do about it. Maybe separate out heavy traffic tests,
> and add a make run_lotraf_tests?

Either that or the other way - express the expectation that the
environment is slow to the test. It came up in the "net-next is OPEN"
thread and also off-list. Perhaps we should discuss tomorrow on the
netdev call?

https://lore.kernel.org/all/20240129112057.26f5fc19@kernel.org/

To make sure I've done my homework I kicked off few more instances of
the tester metal-* and metal-*-dbg. They are running in AWS (AWS Linux,
again) but on bare metal. Former are identical to the previous ones,
just with KVM / HW virt support now, so those should really be all
green. The metal-*-dbg ones have kernel/configs/debug.config and
kernel/configs/x86_debug.config configed in, so "worst case slow".

> tc_actions started getting a passible deadlocking warning between Jan 27
> 00:37 and Jan 28 18:27:
> 
>     https://netdev-2.bots.linux.dev/vmksft-forwarding/results/438201/108-tc-actions-sh/
>     https://netdev-2.bots.linux.dev/vmksft-forwarding/results/438566/109-tc-actions-sh/
> 
> So either something landed that broke it, or the host kernel now has
> more debugging enabled, so it now gives a citation.

Hm. configs are identical.

$ git diff net-next-2024-01-26--18-00..net-next-2024-01-27--00-04 --stat 
 Documentation/devicetree/bindings/net/snps,dwmac.yaml            |  11 +++---
 Documentation/devicetree/bindings/net/starfive,jh7110-dwmac.yaml |  72 +++++++++++++++++++++++++++------------
 drivers/net/ethernet/amd/pds_core/adminq.c                       |  74 +++++++++++++++++++++++++++-------------
 drivers/net/ethernet/amd/pds_core/core.c                         | 130 +++++++++++++++++++++++++++++++++++++++++-----------------------------
 drivers/net/ethernet/amd/pds_core/core.h                         |   3 +-
 drivers/net/ethernet/amd/pds_core/debugfs.c                      |  12 ++++---
 drivers/net/ethernet/amd/pds_core/dev.c                          |  30 ++++++++++++-----
 drivers/net/ethernet/amd/pds_core/devlink.c                      |   3 +-
 drivers/net/ethernet/amd/pds_core/fw.c                           |   3 ++
 drivers/net/ethernet/amd/pds_core/main.c                         |  26 +++++++++++---
 drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.c      |  16 ++++-----
 drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_keys.c         |   9 ++---
 drivers/net/ethernet/mellanox/mlxsw/spectrum.c                   | 160 +++++++++++++++++++++++++++++++++++++++++++++++----------------------------------------
 drivers/net/ethernet/mellanox/mlxsw/spectrum.h                   |  15 ++-------
 drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c               |  11 +++---
 drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_tcam.c          |  17 +++++-----
 drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c            |  15 +++++----
 drivers/net/ethernet/mellanox/mlxsw/spectrum_switchdev.c         |   8 ++---
 drivers/net/ethernet/stmicro/stmmac/Kconfig                      |   6 ++--
 drivers/net/ethernet/stmicro/stmmac/dwmac-starfive.c             |  32 +++++++++++++++---
 net/core/dev.c                                                   |  27 +++++++++------
 net/core/dev.h                                                   |   1 +
 net/ipv4/ip_output.c                                             |   9 +----
 tools/testing/selftests/net/config                               |  14 ++++----
 24 files changed, 430 insertions(+), 274 deletions(-)

No idea.

> ip6gre_inner_v6_multipath is just noisy? It failed the last run, but
> passed several before.

FWIW I made this: https://netdev.bots.linux.dev/flakes.html

> router_multicast and router get a complaint about a missing control
> socket. I think at first approximation they need:
> 
>     # mkdir -p /usr/local/var/run
> 
> But even then I'm getting a fail. This and the others seem to all be in
> IPv6 multicast.

Re: [PATCH net-next] selftests: forwarding: Add missing config entries

[Paolo Abeni]

On Fri, 2024-01-26 at 17:36 +0100, Petr Machata wrote:
> The config file contains a partial kernel configuration to be used by
> `virtme-configkernel --custom'. The presumption is that the config file
> contains all Kconfig options needed by the selftests from the directory.
> 
> In net/forwarding/config, many are missing, which manifests as spurious
> failures when running the selftests, with messages about unknown device
> types, qdisc kinds or classifier actions. Add the missing configurations.
> 
> Tested the resulting configuration using virtme-ng as follows:
> 
>  # vng -b -f tools/testing/selftests/net/forwarding/config
>  # vng --user root
>  (within the VM:)
>  # make -C tools/testing/selftests TARGETS=net/forwarding run_tests

FTR, you should be able to combine the last 2 commands in a single one
with:

vng --user root -- make -C tools/testing/selftests TARGETS=net/forwarding run_tests

/P

Re: [PATCH net-next] selftests: forwarding: Add missing config entries

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to netdev/net-next.git (main)
by Paolo Abeni <pabeni@redhat.com>:

On Fri, 26 Jan 2024 17:36:16 +0100 you wrote:
> The config file contains a partial kernel configuration to be used by
> `virtme-configkernel --custom'. The presumption is that the config file
> contains all Kconfig options needed by the selftests from the directory.
> 
> In net/forwarding/config, many are missing, which manifests as spurious
> failures when running the selftests, with messages about unknown device
> types, qdisc kinds or classifier actions. Add the missing configurations.
> 
> [...]

Here is the summary with links:
  - [net-next] selftests: forwarding: Add missing config entries
    https://git.kernel.org/netdev/net-next/c/4acf4e62cd57

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

Re: [PATCH net-next] selftests: forwarding: Add missing config entries

[Jiri Pirko]

Fri, Jan 26, 2024 at 05:36:16PM CET, petrm@nvidia.com wrote:
>The config file contains a partial kernel configuration to be used by
>`virtme-configkernel --custom'. The presumption is that the config file
>contains all Kconfig options needed by the selftests from the directory.
>
>In net/forwarding/config, many are missing, which manifests as spurious
>failures when running the selftests, with messages about unknown device
>types, qdisc kinds or classifier actions. Add the missing configurations.
>
>Tested the resulting configuration using virtme-ng as follows:
>
> # vng -b -f tools/testing/selftests/net/forwarding/config
> # vng --user root
> (within the VM:)
> # make -C tools/testing/selftests TARGETS=net/forwarding run_tests

For me, all tests end up with:
SKIP: Cannot create interface. Name not specified

Do I miss some config file? If yes, can't we have some default testing
ifnames in case the config is not there?

Re: [PATCH net-next] selftests: forwarding: Add missing config entries

[Jakub Kicinski]

On Tue, 30 Jan 2024 15:30:46 +0100 Jiri Pirko wrote:
> For me, all tests end up with:
> SKIP: Cannot create interface. Name not specified
> 
> Do I miss some config file? If yes, can't we have some default testing
> ifnames in case the config is not there?

Thanks for bringing this up... I forgot to

  cp forwarding.config.sample forwarding.config

an embarrassing number of times. +1 for printing a warning and using
the sample if forwarding.config does not exist.

Re: [PATCH net-next] selftests: forwarding: Add missing config entries

[Petr Machata]

Jakub Kicinski <kuba@kernel.org> writes:

> On Tue, 30 Jan 2024 15:30:46 +0100 Jiri Pirko wrote:
>> For me, all tests end up with:
>> SKIP: Cannot create interface. Name not specified
>> 
>> Do I miss some config file? If yes, can't we have some default testing
>> ifnames in case the config is not there?
>
> Thanks for bringing this up... I forgot to
>
>   cp forwarding.config.sample forwarding.config
>
> an embarrassing number of times. +1 for printing a warning and using
> the sample if forwarding.config does not exist.

I'll roll this into my selftest update patchset.