[PATCH net-next] selftests: drv-net: hide the devlink port_split test

[PATCH net-next] selftests: drv-net: hide the devlink port_split test

[Jakub Kicinski]

The devlink port_split test has limited applicability.
NICs (as opposed to switches) require at least a re-probe
to apply the split configuration.

On top of that the test is not compatible with our driver env,
it just splits all ports on the system, not only what NETIF
points at.

Long term we may want to add some indication in devlink whether
the port splitting is runtime (cmode of sorts), and fix the
test to follow driver env. But since no (known) NIC driver can
support runtime anyway let's just hide the test from the selftest
framework by moving it to extra files.

Having this test randomly break unrelated NICs within the DUT
makes people implement allow-lists for ksft, which then means
their setups don't run new tests.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
CC: danieller@nvidia.com
CC: petrm@nvidia.com
CC: shuah@kernel.org
CC: linux-kselftest@vger.kernel.org
---
 tools/testing/selftests/drivers/net/hw/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/drivers/net/hw/Makefile b/tools/testing/selftests/drivers/net/hw/Makefile
index 234db5c2c90c..ab13f58d703e 100644
--- a/tools/testing/selftests/drivers/net/hw/Makefile
+++ b/tools/testing/selftests/drivers/net/hw/Makefile
@@ -19,7 +19,6 @@ TEST_GEN_FILES := \
 
 TEST_PROGS = \
 	csum.py \
-	devlink_port_split.py \
 	devlink_rate_cross_esw.py \
 	devlink_rate_tc_bw.py \
 	devmem.py \
@@ -55,6 +54,7 @@ TEST_PROGS = \
 	#
 
 TEST_FILES := \
+	devlink_port_split.py \
 	devmem_lib.py \
 	ethtool_lib.sh \
 	#
-- 
2.55.0

Re: [PATCH net-next] selftests: drv-net: hide the devlink port_split test

[Petr Machata]

Jakub Kicinski <kuba@kernel.org> writes:

> The devlink port_split test has limited applicability.
> NICs (as opposed to switches) require at least a re-probe
> to apply the split configuration.

Interesting, apparently some NICs require reboot after devlink port
split. Hmm, should I just move the test to drivers/mlxsw?

> On top of that the test is not compatible with our driver env,
> it just splits all ports on the system, not only what NETIF
> points at.
>
> Long term we may want to add some indication in devlink whether
> the port splitting is runtime (cmode of sorts), and fix the
> test to follow driver env. But since no (known) NIC driver can
> support runtime anyway let's just hide the test from the selftest
> framework by moving it to extra files.

Sure, go for it.

> Having this test randomly break unrelated NICs within the DUT
> makes people implement allow-lists for ksft, which then means
> their setups don't run new tests.

So I'm fine with the patch, but I don't buy this argument. People
presumably understand that allow-listing implies that only the, well,
allowed tests will be run. Presumably there was a history of new stuff
blowing up, or gradual enablement or whatever, otherwise why not just
blacklist the one problematic one? I.e. don't blame the existence of
allow-lists on devlink_port_split.

> Signed-off-by: Jakub Kicinski <kuba@kernel.org>

Reviewed-by: Petr Machata <petrm@nvidia.com>

> ---
> CC: danieller@nvidia.com
> CC: petrm@nvidia.com
> CC: shuah@kernel.org
> CC: linux-kselftest@vger.kernel.org
> ---
>  tools/testing/selftests/drivers/net/hw/Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/drivers/net/hw/Makefile b/tools/testing/selftests/drivers/net/hw/Makefile
> index 234db5c2c90c..ab13f58d703e 100644
> --- a/tools/testing/selftests/drivers/net/hw/Makefile
> +++ b/tools/testing/selftests/drivers/net/hw/Makefile
> @@ -19,7 +19,6 @@ TEST_GEN_FILES := \
>  
>  TEST_PROGS = \
>  	csum.py \
> -	devlink_port_split.py \
>  	devlink_rate_cross_esw.py \
>  	devlink_rate_tc_bw.py \
>  	devmem.py \
> @@ -55,6 +54,7 @@ TEST_PROGS = \
>  	#
>  
>  TEST_FILES := \
> +	devlink_port_split.py \
>  	devmem_lib.py \
>  	ethtool_lib.sh \
>  	#