[PATCH v7 net-next 0/3] Enhance network interface feature testing

[PATCH v7 net-next 0/3] Enhance network interface feature testing

[Abhinav Jain]

This small series includes fixes for creation of veth pairs for
networkless kernels & adds tests for turning the different network
interface features on and off in selftests/net/netdevice.sh script.

Changes in v7:
Create a third patch in the series to do SKIP -> XFAIL replacement.
Add logic to incorporate XFAIL on setting IP address for veth pair.

Changes in v6:
https://lore.kernel.org/all/20240814191517.50466-1-jain.abhinav177@gmail.com

Use XFAIL for ethtool operations that are unsupported instead of SKIP.

Changes in v5:
https://lore.kernel.org/all/20240808122452.25683-1-jain.abhinav177@gmail.com

Rectify the syntax for ip add link.
Fix the veth_created condition check.

Changes in v4:
https://lore.kernel.org/all/20240807175717.7775-1-jain.abhinav177@gmail.com

Move veth creation/removal to the main shell script.
Tested using vng on a networkless kernel and the script works, sample
output below the changes.

Changes in v3:
https://lore.kernel.org/all/20240614113240.41550-1-jain.abhinav177@gmail.com

Add a check for netdev, create veth pair for testing.
Restore feature to its initial state.

Changes in v2:
https://lore.kernel.org/all/20240609132124.51683-1-jain.abhinav177@gmail.com

Remove tail usage; use read to parse the features from temp file.

v1:
https://lore.kernel.org/all/20240606212714.27472-1-jain.abhinav177@gmail.com

```
# selftests: net: netdevice.sh
# No valid network device found, creating veth pair
# PASS: veth0: set interface up
# PASS: veth0: set MAC address
# XFAIL: veth0: set IP address
# PASS: veth0: ethtool list features
# PASS: veth0: Turned off feature: rx-checksumming
# PASS: veth0: Turned on feature: rx-checksumming
# PASS: veth0: Restore feature rx-checksumming to initial state on
# Actual changes:
# tx-checksum-ip-generic: off
# tx-tcp-segmentation: off [not requested]


# PASS: veth0: Turned off feature: rx-udp-gro-forwarding
# PASS: veth0: Turned on feature: rx-udp-gro-forwarding
# PASS: veth0: Restore feature rx-udp-gro-forwarding to initial state off
# Cannot get register dump: Operation not supported
# XFAIL: veth0: ethtool dump not supported
# PASS: veth0: ethtool stats
# PASS: veth0: stop interface
```

Abhinav Jain (3):
  selftests: net: Create veth pair for testing in networkless kernel
  selftests: net: Add on/off checks for non-fixed features of interface
  selftests: net: Use XFAIL for operations not supported by the driver

 tools/testing/selftests/net/netdevice.sh | 61 ++++++++++++++++++++++--
 1 file changed, 58 insertions(+), 3 deletions(-)

--
2.34.1

[PATCH v7 net-next 1/3] selftests: net: Create veth pair for testing in networkless kernel

[Abhinav Jain]

Check if the netdev list is empty and create veth pair to be used for
feature on/off testing.
Remove the veth pair after testing is complete.

Signed-off-by: Abhinav Jain <jain.abhinav177@gmail.com>
---
 tools/testing/selftests/net/netdevice.sh | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/tools/testing/selftests/net/netdevice.sh b/tools/testing/selftests/net/netdevice.sh
index e3afcb424710..0c32950fdd17 100755
--- a/tools/testing/selftests/net/netdevice.sh
+++ b/tools/testing/selftests/net/netdevice.sh
@@ -129,6 +129,7 @@ kci_netdev_ethtool()
 
 	kci_netdev_ethtool_test 74 'dump' "ethtool -d $netdev"
 	kci_netdev_ethtool_test 94 'stats' "ethtool -S $netdev"
+
 	return 0
 }
 
@@ -196,10 +197,25 @@ if [ ! -e "$TMP_LIST_NETDEV" ];then
 fi
 
 ip link show |grep '^[0-9]' | grep -oE '[[:space:]].*eth[0-9]*:|[[:space:]].*enp[0-9]s[0-9]:' | cut -d\  -f2 | cut -d: -f1> "$TMP_LIST_NETDEV"
+
+if [ ! -s "$TMP_LIST_NETDEV" ]; then
+	echo "No valid network device found, creating veth pair"
+	ip link add veth0 type veth peer name veth1
+	echo "veth0" > "$TMP_LIST_NETDEV"
+	echo "veth1" >> "$TMP_LIST_NETDEV"
+	veth_created=1
+fi
+
 while read netdev
 do
 	kci_test_netdev "$netdev"
 done < "$TMP_LIST_NETDEV"
 
+#clean up veth interface pair if it was created
+if [ "$veth_created" ]; then
+	ip link delete veth0
+	echo "Removed veth pair"
+fi
+
 rm "$TMP_LIST_NETDEV"
 exit 0
-- 
2.34.1

[PATCH v7 net-next 2/3] selftests: net: Add on/off checks for non-fixed features of interface

[Abhinav Jain]

Implement on/off testing for all non-fixed features via while loop.
Save the initial state so that it can be restored after on/off checks.

Signed-off-by: Abhinav Jain <jain.abhinav177@gmail.com>
---
 tools/testing/selftests/net/netdevice.sh | 37 +++++++++++++++++++++++-
 1 file changed, 36 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/netdevice.sh b/tools/testing/selftests/net/netdevice.sh
index 0c32950fdd17..50f7b9d1163d 100755
--- a/tools/testing/selftests/net/netdevice.sh
+++ b/tools/testing/selftests/net/netdevice.sh
@@ -124,7 +124,42 @@ kci_netdev_ethtool()
 		return 1
 	fi
 	echo "PASS: $netdev: ethtool list features"
-	#TODO for each non fixed features, try to turn them on/off
+
+	while read -r FEATURE VALUE FIXED; do
+		[ "$FEATURE" != "Features" ] || continue # Skip "Features"
+		[ "$FIXED" != "[fixed]" ] || continue # Skip fixed features
+		feature="${FEATURE%:*}"
+
+		initial_state=$(ethtool -k "$netdev" | grep "$feature:" \
+			| awk '{print $2}')
+		ethtool --offload "$netdev" "$feature" off
+		if [ $? -eq 0 ]; then
+			echo "PASS: $netdev: Turned off feature: $feature"
+		else
+			echo "FAIL: $netdev: Failed to turn off feature:" \
+				"$feature"
+		fi
+
+		ethtool --offload "$netdev" "$feature" on
+		if [ $? -eq 0 ]; then
+			echo "PASS: $netdev: Turned on feature: $feature"
+		else
+			echo "FAIL: $netdev: Failed to turn on feature:" \
+				"$feature"
+		fi
+
+		#restore the feature to its initial state
+		ethtool --offload "$netdev" "$feature" "$initial_state"
+		if [ $? -eq 0 ]; then
+			echo "PASS: $netdev: Restore feature $feature" \
+				"to initial state $initial_state"
+		else
+			echo "FAIL: $netdev: Failed to restore feature" \
+				"$feature to default $initial_state"
+		fi
+
+	done < "$TMP_ETHTOOL_FEATURES"
+
 	rm "$TMP_ETHTOOL_FEATURES"
 
 	kci_netdev_ethtool_test 74 'dump' "ethtool -d $netdev"
-- 
2.34.1

[PATCH v7 net-next 3/3] selftests: net: Use XFAIL for operations not supported by the driver

[Abhinav Jain]

Check if veth pair was created and if yes, xfail on setting IP address.
Use XFAIL instead of SKIP for unsupported ethtool APIs.

Signed-off-by: Abhinav Jain <jain.abhinav177@gmail.com>
---
 tools/testing/selftests/net/netdevice.sh | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/net/netdevice.sh b/tools/testing/selftests/net/netdevice.sh
index 50f7b9d1163d..15d702adb0ea 100755
--- a/tools/testing/selftests/net/netdevice.sh
+++ b/tools/testing/selftests/net/netdevice.sh
@@ -68,7 +68,11 @@ kci_net_setup()
 	fi
 
 	# TODO what ipaddr to set ? DHCP ?
-	echo "SKIP: $netdev: set IP address"
+	if [ "$veth_created" ]; then
+		echo "XFAIL: $netdev: set IP address"
+	else
+		echo "SKIP: $netdev: set IP address"
+	fi
 	return $ksft_skip
 }
 
@@ -86,7 +90,7 @@ kci_netdev_ethtool_test()
 	ret=$?
 	if [ $ret -ne 0 ];then
 		if [ $ret -eq "$1" ];then
-			echo "SKIP: $netdev: ethtool $2 not supported"
+			echo "XFAIL: $netdev: ethtool $2 not supported"
 			return $ksft_skip
 		else
 			echo "FAIL: $netdev: ethtool $2"
-- 
2.34.1

Re: [PATCH v7 net-next 2/3] selftests: net: Add on/off checks for non-fixed features of interface

[Simon Horman]

On Thu, Aug 15, 2024 at 04:29:23PM +0530, Abhinav Jain wrote:
> Implement on/off testing for all non-fixed features via while loop.
> Save the initial state so that it can be restored after on/off checks.
> 
> Signed-off-by: Abhinav Jain <jain.abhinav177@gmail.com>
> ---
>  tools/testing/selftests/net/netdevice.sh | 37 +++++++++++++++++++++++-
>  1 file changed, 36 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/net/netdevice.sh b/tools/testing/selftests/net/netdevice.sh
> index 0c32950fdd17..50f7b9d1163d 100755
> --- a/tools/testing/selftests/net/netdevice.sh
> +++ b/tools/testing/selftests/net/netdevice.sh
> @@ -124,7 +124,42 @@ kci_netdev_ethtool()
>  		return 1
>  	fi
>  	echo "PASS: $netdev: ethtool list features"
> -	#TODO for each non fixed features, try to turn them on/off
> +
> +	while read -r FEATURE VALUE FIXED; do
> +		[ "$FEATURE" != "Features" ] || continue # Skip "Features"
> +		[ "$FIXED" != "[fixed]" ] || continue # Skip fixed features
> +		feature="${FEATURE%:*}"
> +
> +		initial_state=$(ethtool -k "$netdev" | grep "$feature:" \
> +			| awk '{print $2}')

Hi Abhinav,

Isn't the value being read into $initial_state here already present in $VALUE?

> +		ethtool --offload "$netdev" "$feature" off
> +		if [ $? -eq 0 ]; then
> +			echo "PASS: $netdev: Turned off feature: $feature"
> +		else
> +			echo "FAIL: $netdev: Failed to turn off feature:" \
> +				"$feature"
> +		fi
> +
> +		ethtool --offload "$netdev" "$feature" on
> +		if [ $? -eq 0 ]; then
> +			echo "PASS: $netdev: Turned on feature: $feature"
> +		else
> +			echo "FAIL: $netdev: Failed to turn on feature:" \
> +				"$feature"
> +		fi
> +
> +		#restore the feature to its initial state
> +		ethtool --offload "$netdev" "$feature" "$initial_state"
> +		if [ $? -eq 0 ]; then
> +			echo "PASS: $netdev: Restore feature $feature" \
> +				"to initial state $initial_state"
> +		else
> +			echo "FAIL: $netdev: Failed to restore feature" \
> +				"$feature to default $initial_state"
> +		fi
> +
> +	done < "$TMP_ETHTOOL_FEATURES"
> +
>  	rm "$TMP_ETHTOOL_FEATURES"
>  
>  	kci_netdev_ethtool_test 74 'dump' "ethtool -d $netdev"
> -- 
> 2.34.1
> 
> 

Re: [PATCH v7 net-next 2/3] selftests: net: Add on/off checks for non-fixed features of interface

[Abhinav Jain]

On Thu, 15 Aug 2024 14:03:53 +0100, Simon Horman wrote:

> Hi Abhinav,
> 
> Isn't the value being read into $initial_state here already present in $VALUE?

Yes, that is correct. I will wait for a day and send v8 using $VALUE.
Thanks.
---