Re: [PATCH net v3] selftests: net: local_termination: annotate the expected failures

[Hangbin Liu <liuhangbin@gmail.com>]

On Thu, May 16, 2024 at 08:25:13AM -0700, Jakub Kicinski wrote:
> Vladimir said when adding this test:
> 
>   The bridge driver fares particularly badly [...] mainly because
>   it does not implement IFF_UNICAST_FLT.
> 
> See commit 90b9566aa5cd ("selftests: forwarding: add a test for
> local_termination.sh").
> 
> We don't want to hide the known gaps, but having a test which
> always fails prevents us from catching regressions. Report
> the cases we know may fail as XFAIL.
> 
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
> CC: liuhangbin@gmail.com
> CC: shuah@kernel.org
> CC: linux-kselftest@vger.kernel.org
> CC: Petr Machata <petrm@nvidia.com>
> CC: vladimir.oltean@nxp.com
> 
> v3:
>  - use xfail_on_veth correctly as a "prefix" call
>  - dropping Vladimir's tags since the code is quite different now
> v2: https://lore.kernel.org/r/20240509235553.5740-1-kuba@kernel.org/
>  - remove duplicated log_test_xfail
> v1: https://lore.kernel.org/all/20240509235553.5740-1-kuba@kernel.org/
> ---
>  .../net/forwarding/local_termination.sh       | 30 +++++++++++--------
>  1 file changed, 18 insertions(+), 12 deletions(-)
> 
> diff --git a/tools/testing/selftests/net/forwarding/local_termination.sh b/tools/testing/selftests/net/forwarding/local_termination.sh
> index c5b0cbc85b3e..4b364cdf3ef0 100755
> --- a/tools/testing/selftests/net/forwarding/local_termination.sh
> +++ b/tools/testing/selftests/net/forwarding/local_termination.sh
> @@ -155,25 +155,30 @@ run_test()
>  		"$smac > $MACVLAN_ADDR, ethertype IPv4 (0x0800)" \
>  		true
>  
> -	check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address" \
> -		"$smac > $UNKNOWN_UC_ADDR1, ethertype IPv4 (0x0800)" \
> -		false
> +	xfail_on_veth $h1 \
> +		check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address" \
> +			"$smac > $UNKNOWN_UC_ADDR1, ethertype IPv4 (0x0800)" \
> +			false
>  
>  	check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address, promisc" \
>  		"$smac > $UNKNOWN_UC_ADDR2, ethertype IPv4 (0x0800)" \
>  		true
>  
> -	check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address, allmulti" \
> -		"$smac > $UNKNOWN_UC_ADDR3, ethertype IPv4 (0x0800)" \
> -		false
> +	xfail_on_veth $h1 \
> +		check_rcv $rcv_if_name \
> +			"Unicast IPv4 to unknown MAC address, allmulti" \
> +			"$smac > $UNKNOWN_UC_ADDR3, ethertype IPv4 (0x0800)" \
> +			false
>  
>  	check_rcv $rcv_if_name "Multicast IPv4 to joined group" \
>  		"$smac > $JOINED_MACV4_MC_ADDR, ethertype IPv4 (0x0800)" \
>  		true
>  
> -	check_rcv $rcv_if_name "Multicast IPv4 to unknown group" \
> -		"$smac > $UNKNOWN_MACV4_MC_ADDR1, ethertype IPv4 (0x0800)" \
> -		false
> +	xfail_on_veth $h1 \
> +		check_rcv $rcv_if_name \
> +			"Multicast IPv4 to unknown group" \
> +			"$smac > $UNKNOWN_MACV4_MC_ADDR1, ethertype IPv4 (0x0800)" \
> +			false
>  
>  	check_rcv $rcv_if_name "Multicast IPv4 to unknown group, promisc" \
>  		"$smac > $UNKNOWN_MACV4_MC_ADDR2, ethertype IPv4 (0x0800)" \
> @@ -187,9 +192,10 @@ run_test()
>  		"$smac > $JOINED_MACV6_MC_ADDR, ethertype IPv6 (0x86dd)" \
>  		true
>  
> -	check_rcv $rcv_if_name "Multicast IPv6 to unknown group" \
> -		"$smac > $UNKNOWN_MACV6_MC_ADDR1, ethertype IPv6 (0x86dd)" \
> -		false
> +	xfail_on_veth $h1 \
> +		check_rcv $rcv_if_name "Multicast IPv6 to unknown group" \
> +			"$smac > $UNKNOWN_MACV6_MC_ADDR1, ethertype IPv6 (0x86dd)" \
> +			false
>  
>  	check_rcv $rcv_if_name "Multicast IPv6 to unknown group, promisc" \
>  		"$smac > $UNKNOWN_MACV6_MC_ADDR2, ethertype IPv6 (0x86dd)" \
> -- 
> 2.45.0
> 

Reviewed-by: Hangbin Liu <liuhangbin@gmail.com>