[PATCH v8 net-next 0/3] Enhance network interface feature testing

[PATCH v8 net-next 0/3] Enhance network interface feature testing

[Abhinav Jain]

This small series includes fixes for creation of veth pairs for
networkless kernels & adds tests for turning the different network
interface features on and off in selftests/net/netdevice.sh script.
Tested using vng and compiles for network as well as networkless kernel.

Changes in v8:
Remove redundant variable initial_state as per feedback in v7.

Changes in v7:
https://lore.kernel.org/all/20240815105924.1389290-1-jain.abhinav177@gmail.com

Create a third patch in the series to do SKIP -> XFAIL replacement.
Add logic to incorporate XFAIL on setting IP address for veth pair.

Changes in v6:
https://lore.kernel.org/all/20240814191517.50466-1-jain.abhinav177@gmail.com

Use XFAIL for ethtool operations that are unsupported instead of SKIP.

Changes in v5:
https://lore.kernel.org/all/20240808122452.25683-1-jain.abhinav177@gmail.com

Rectify the syntax for ip add link.
Fix the veth_created condition check.

Changes in v4:
https://lore.kernel.org/all/20240807175717.7775-1-jain.abhinav177@gmail.com

Move veth creation/removal to the main shell script.
Tested using vng on a networkless kernel and the script works, sample
output below the changes.

Changes in v3:
https://lore.kernel.org/all/20240614113240.41550-1-jain.abhinav177@gmail.com

Add a check for netdev, create veth pair for testing.
Restore feature to its initial state.

Changes in v2:
https://lore.kernel.org/all/20240609132124.51683-1-jain.abhinav177@gmail.com

Remove tail usage; use read to parse the features from temp file.

v1:
https://lore.kernel.org/all/20240606212714.27472-1-jain.abhinav177@gmail.com

```
# selftests: net: netdevice.sh
# No valid network device found, creating veth pair
# PASS: veth0: set interface up
# PASS: veth0: set MAC address
# XFAIL: veth0: set IP address unsupported for veth*
# PASS: veth0: ethtool list features
# PASS: veth0: Turned off feature: rx-checksumming
# PASS: veth0: Turned on feature: rx-checksumming
# PASS: veth0: Restore feature rx-checksumming to initial state on
# Actual changes:
# tx-checksum-ip-generic: off

...

# PASS: veth0: Turned on feature: rx-udp-gro-forwarding
# PASS: veth0: Restore feature rx-udp-gro-forwarding to initial state off
# Cannot get register dump: Operation not supported
# XFAIL: veth0: ethtool dump not supported
# PASS: veth0: ethtool stats
# PASS: veth0: stop interface
```

Abhinav Jain (3):
  selftests: net: Create veth pair for testing in networkless kernel
  selftests: net: Add on/off checks for non-fixed features of interface
  selftests: net: Use XFAIL for operations not supported by the driver

 tools/testing/selftests/net/netdevice.sh | 61 ++++++++++++++++++++++--
 1 file changed, 57 insertions(+), 4 deletions(-)

--
2.34.1

[PATCH v8 net-next 1/3] selftests: net: Create veth pair for testing in networkless kernel

[Abhinav Jain]

Check if the netdev list is empty and create veth pair to be used for
feature on/off testing.
Remove the veth pair after testing is complete.

Signed-off-by: Abhinav Jain <jain.abhinav177@gmail.com>
---
 tools/testing/selftests/net/netdevice.sh | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/tools/testing/selftests/net/netdevice.sh b/tools/testing/selftests/net/netdevice.sh
index e3afcb424710..0c32950fdd17 100755
--- a/tools/testing/selftests/net/netdevice.sh
+++ b/tools/testing/selftests/net/netdevice.sh
@@ -129,6 +129,7 @@ kci_netdev_ethtool()
 
 	kci_netdev_ethtool_test 74 'dump' "ethtool -d $netdev"
 	kci_netdev_ethtool_test 94 'stats' "ethtool -S $netdev"
+
 	return 0
 }
 
@@ -196,10 +197,25 @@ if [ ! -e "$TMP_LIST_NETDEV" ];then
 fi
 
 ip link show |grep '^[0-9]' | grep -oE '[[:space:]].*eth[0-9]*:|[[:space:]].*enp[0-9]s[0-9]:' | cut -d\  -f2 | cut -d: -f1> "$TMP_LIST_NETDEV"
+
+if [ ! -s "$TMP_LIST_NETDEV" ]; then
+	echo "No valid network device found, creating veth pair"
+	ip link add veth0 type veth peer name veth1
+	echo "veth0" > "$TMP_LIST_NETDEV"
+	echo "veth1" >> "$TMP_LIST_NETDEV"
+	veth_created=1
+fi
+
 while read netdev
 do
 	kci_test_netdev "$netdev"
 done < "$TMP_LIST_NETDEV"
 
+#clean up veth interface pair if it was created
+if [ "$veth_created" ]; then
+	ip link delete veth0
+	echo "Removed veth pair"
+fi
+
 rm "$TMP_LIST_NETDEV"
 exit 0
-- 
2.34.1

[PATCH v8 net-next 2/3] selftests: net: Add on/off checks for non-fixed features of interface

[Abhinav Jain]

Implement on/off testing for all non-fixed features via while loop.

Signed-off-by: Abhinav Jain <jain.abhinav177@gmail.com>
---
 tools/testing/selftests/net/netdevice.sh | 35 +++++++++++++++++++++++-
 1 file changed, 34 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/netdevice.sh b/tools/testing/selftests/net/netdevice.sh
index 0c32950fdd17..20febf9246ea 100755
--- a/tools/testing/selftests/net/netdevice.sh
+++ b/tools/testing/selftests/net/netdevice.sh
@@ -124,7 +124,40 @@ kci_netdev_ethtool()
 		return 1
 	fi
 	echo "PASS: $netdev: ethtool list features"
-	#TODO for each non fixed features, try to turn them on/off
+
+	while read -r FEATURE VALUE FIXED; do
+		[ "$FEATURE" != "Features" ] || continue # Skip "Features"
+		[ "$FIXED" != "[fixed]" ] || continue # Skip fixed features
+		feature="${FEATURE%:*}"
+
+		ethtool --offload "$netdev" "$feature" off
+		if [ $? -eq 0 ]; then
+			echo "PASS: $netdev: Turned off feature: $feature"
+		else
+			echo "FAIL: $netdev: Failed to turn off feature:" \
+				"$feature"
+		fi
+
+		ethtool --offload "$netdev" "$feature" on
+		if [ $? -eq 0 ]; then
+			echo "PASS: $netdev: Turned on feature: $feature"
+		else
+			echo "FAIL: $netdev: Failed to turn on feature:" \
+				"$feature"
+		fi
+
+		#restore the feature to its initial state
+		ethtool --offload "$netdev" "$feature" "$VALUE"
+		if [ $? -eq 0 ]; then
+			echo "PASS: $netdev: Restore feature $feature" \
+				"to initial state $VALUE"
+		else
+			echo "FAIL: $netdev: Failed to restore feature" \
+				"$feature to initial state $VALUE"
+		fi
+
+	done < "$TMP_ETHTOOL_FEATURES"
+
 	rm "$TMP_ETHTOOL_FEATURES"
 
 	kci_netdev_ethtool_test 74 'dump' "ethtool -d $netdev"
-- 
2.34.1

[PATCH v8 net-next 3/3] selftests: net: Use XFAIL for operations not supported by the driver

[Abhinav Jain]

Check if veth pair was created and if yes, xfail on setting IP address
logging an informational message.
Use XFAIL instead of SKIP for unsupported ethtool APIs.

Signed-off-by: Abhinav Jain <jain.abhinav177@gmail.com>
---
 tools/testing/selftests/net/netdevice.sh | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/net/netdevice.sh b/tools/testing/selftests/net/netdevice.sh
index 20febf9246ea..a596f0be9ace 100755
--- a/tools/testing/selftests/net/netdevice.sh
+++ b/tools/testing/selftests/net/netdevice.sh
@@ -67,8 +67,12 @@ kci_net_setup()
 		return $ksft_skip
 	fi
 
-	# TODO what ipaddr to set ? DHCP ?
-	echo "SKIP: $netdev: set IP address"
+	if [ "$veth_created" ]; then
+		echo "XFAIL: $netdev: set IP address unsupported for veth*"
+	else
+		# TODO what ipaddr to set ? DHCP ?
+		echo "SKIP: $netdev: set IP address"
+	fi
 	return $ksft_skip
 }
 
@@ -86,7 +90,7 @@ kci_netdev_ethtool_test()
 	ret=$?
 	if [ $ret -ne 0 ];then
 		if [ $ret -eq "$1" ];then
-			echo "SKIP: $netdev: ethtool $2 not supported"
+			echo "XFAIL: $netdev: ethtool $2 not supported"
 			return $ksft_skip
 		else
 			echo "FAIL: $netdev: ethtool $2"
-- 
2.34.1

Re: [PATCH v8 net-next 1/3] selftests: net: Create veth pair for testing in networkless kernel

[Simon Horman]

On Mon, Aug 19, 2024 at 05:42:33PM +0530, Abhinav Jain wrote:
> Check if the netdev list is empty and create veth pair to be used for
> feature on/off testing.
> Remove the veth pair after testing is complete.
> 
> Signed-off-by: Abhinav Jain <jain.abhinav177@gmail.com>

Reviewed-by: Simon Horman <horms@kernel.org>

Re: [PATCH v8 net-next 2/3] selftests: net: Add on/off checks for non-fixed features of interface

[Simon Horman]

On Mon, Aug 19, 2024 at 05:42:34PM +0530, Abhinav Jain wrote:
> Implement on/off testing for all non-fixed features via while loop.
> 
> Signed-off-by: Abhinav Jain <jain.abhinav177@gmail.com>

Reviewed-by: Simon Horman <horms@kernel.org>

Re: [PATCH v8 net-next 3/3] selftests: net: Use XFAIL for operations not supported by the driver

[Simon Horman]

On Mon, Aug 19, 2024 at 05:42:35PM +0530, Abhinav Jain wrote:
> Check if veth pair was created and if yes, xfail on setting IP address
> logging an informational message.
> Use XFAIL instead of SKIP for unsupported ethtool APIs.
> 
> Signed-off-by: Abhinav Jain <jain.abhinav177@gmail.com>

Reviewed-by: Simon Horman <horms@kernel.org>

Re: [PATCH v8 net-next 1/3] selftests: net: Create veth pair for testing in networkless kernel

[Jakub Kicinski]

On Mon, 19 Aug 2024 17:42:33 +0530 Abhinav Jain wrote:
> +	echo "veth0" > "$TMP_LIST_NETDEV"
> +	echo "veth1" >> "$TMP_LIST_NETDEV"

Why test both ends? 
Aren't we going to do the same exact test twice?

Re: [PATCH v8 net-next 1/3] selftests: net: Create veth pair for testing in networkless kernel

[Abhinav Jain]

On Tue, 20 Aug 2024 16:50:06 -0700, Jakub Kicinski wrote:
> On Mon, 19 Aug 2024 17:42:33 +0530 Abhinav Jain wrote:
> > +	echo "veth0" > "$TMP_LIST_NETDEV"
> > +	echo "veth1" >> "$TMP_LIST_NETDEV"
>
> Why test both ends? 
> Aren't we going to do the same exact test twice?

I presumed that we would want to run the interface up/down, setup and ethtool tests on both veth.
If this is not required, should I submit a v9 removing veth1 from the temp list?

Also, while sending v9, do I add the Review tag from Simon or it has to be reviewed again?

Re: [PATCH v8 net-next 1/3] selftests: net: Create veth pair for testing in networkless kernel

[Jakub Kicinski]

On Wed, 21 Aug 2024 18:07:33 +0530 Abhinav Jain wrote:
> I presumed that we would want to run the interface up/down, setup and ethtool tests on both veth.
> If this is not required, should I submit a v9 removing veth1 from the temp list?

Yes, please.

> Also, while sending v9, do I add the Review tag from Simon or it has to be reviewed again?

Yes, you should add the review tags

Re: [PATCH v8 net-next 1/3] selftests: net: Create veth pair for testing in networkless kernel

[Abhinav Jain]

On Wed, 21 Aug 2024 09:00:41 -0700, Jakub Kicinski wrote:
>> I presumed that we would want to run the interface up/down, setup and ethtool tests on both veth.
>> If this is not required, should I submit a v9 removing veth1 from the temp list?
>
> Yes, please.
>
>> Also, while sending v9, do I add the Review tag from Simon or it has to be reviewed again?
> 
> Yes, you should add the review tags

Thanks for confirming, I have sent v9 accordingly:
https://lore.kernel.org/all/20240821171903.118324-1-jain.abhinav177@gmail.com

Please review, thanks again.