* [PATCH net v3] selftests: net: local_termination: annotate the expected failures
@ 2024-05-16 15:25 Jakub Kicinski
2024-05-17 8:54 ` Hangbin Liu
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Jakub Kicinski @ 2024-05-16 15:25 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, pabeni, Jakub Kicinski, liuhangbin, shuah,
linux-kselftest, Petr Machata, vladimir.oltean
Vladimir said when adding this test:
The bridge driver fares particularly badly [...] mainly because
it does not implement IFF_UNICAST_FLT.
See commit 90b9566aa5cd ("selftests: forwarding: add a test for
local_termination.sh").
We don't want to hide the known gaps, but having a test which
always fails prevents us from catching regressions. Report
the cases we know may fail as XFAIL.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
CC: liuhangbin@gmail.com
CC: shuah@kernel.org
CC: linux-kselftest@vger.kernel.org
CC: Petr Machata <petrm@nvidia.com>
CC: vladimir.oltean@nxp.com
v3:
- use xfail_on_veth correctly as a "prefix" call
- dropping Vladimir's tags since the code is quite different now
v2: https://lore.kernel.org/r/20240509235553.5740-1-kuba@kernel.org/
- remove duplicated log_test_xfail
v1: https://lore.kernel.org/all/20240509235553.5740-1-kuba@kernel.org/
---
.../net/forwarding/local_termination.sh | 30 +++++++++++--------
1 file changed, 18 insertions(+), 12 deletions(-)
diff --git a/tools/testing/selftests/net/forwarding/local_termination.sh b/tools/testing/selftests/net/forwarding/local_termination.sh
index c5b0cbc85b3e..4b364cdf3ef0 100755
--- a/tools/testing/selftests/net/forwarding/local_termination.sh
+++ b/tools/testing/selftests/net/forwarding/local_termination.sh
@@ -155,25 +155,30 @@ run_test()
"$smac > $MACVLAN_ADDR, ethertype IPv4 (0x0800)" \
true
- check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address" \
- "$smac > $UNKNOWN_UC_ADDR1, ethertype IPv4 (0x0800)" \
- false
+ xfail_on_veth $h1 \
+ check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address" \
+ "$smac > $UNKNOWN_UC_ADDR1, ethertype IPv4 (0x0800)" \
+ false
check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address, promisc" \
"$smac > $UNKNOWN_UC_ADDR2, ethertype IPv4 (0x0800)" \
true
- check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address, allmulti" \
- "$smac > $UNKNOWN_UC_ADDR3, ethertype IPv4 (0x0800)" \
- false
+ xfail_on_veth $h1 \
+ check_rcv $rcv_if_name \
+ "Unicast IPv4 to unknown MAC address, allmulti" \
+ "$smac > $UNKNOWN_UC_ADDR3, ethertype IPv4 (0x0800)" \
+ false
check_rcv $rcv_if_name "Multicast IPv4 to joined group" \
"$smac > $JOINED_MACV4_MC_ADDR, ethertype IPv4 (0x0800)" \
true
- check_rcv $rcv_if_name "Multicast IPv4 to unknown group" \
- "$smac > $UNKNOWN_MACV4_MC_ADDR1, ethertype IPv4 (0x0800)" \
- false
+ xfail_on_veth $h1 \
+ check_rcv $rcv_if_name \
+ "Multicast IPv4 to unknown group" \
+ "$smac > $UNKNOWN_MACV4_MC_ADDR1, ethertype IPv4 (0x0800)" \
+ false
check_rcv $rcv_if_name "Multicast IPv4 to unknown group, promisc" \
"$smac > $UNKNOWN_MACV4_MC_ADDR2, ethertype IPv4 (0x0800)" \
@@ -187,9 +192,10 @@ run_test()
"$smac > $JOINED_MACV6_MC_ADDR, ethertype IPv6 (0x86dd)" \
true
- check_rcv $rcv_if_name "Multicast IPv6 to unknown group" \
- "$smac > $UNKNOWN_MACV6_MC_ADDR1, ethertype IPv6 (0x86dd)" \
- false
+ xfail_on_veth $h1 \
+ check_rcv $rcv_if_name "Multicast IPv6 to unknown group" \
+ "$smac > $UNKNOWN_MACV6_MC_ADDR1, ethertype IPv6 (0x86dd)" \
+ false
check_rcv $rcv_if_name "Multicast IPv6 to unknown group, promisc" \
"$smac > $UNKNOWN_MACV6_MC_ADDR2, ethertype IPv6 (0x86dd)" \
--
2.45.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH net v3] selftests: net: local_termination: annotate the expected failures
2024-05-16 15:25 [PATCH net v3] selftests: net: local_termination: annotate the expected failures Jakub Kicinski
@ 2024-05-17 8:54 ` Hangbin Liu
2024-05-17 13:04 ` Petr Machata
2024-05-17 8:54 ` Simon Horman
2024-05-17 20:38 ` patchwork-bot+netdevbpf
2 siblings, 1 reply; 5+ messages in thread
From: Hangbin Liu @ 2024-05-17 8:54 UTC (permalink / raw)
To: Jakub Kicinski
Cc: davem, netdev, edumazet, pabeni, shuah, linux-kselftest,
Petr Machata, vladimir.oltean
On Thu, May 16, 2024 at 08:25:13AM -0700, Jakub Kicinski wrote:
> Vladimir said when adding this test:
>
> The bridge driver fares particularly badly [...] mainly because
> it does not implement IFF_UNICAST_FLT.
>
> See commit 90b9566aa5cd ("selftests: forwarding: add a test for
> local_termination.sh").
>
> We don't want to hide the known gaps, but having a test which
> always fails prevents us from catching regressions. Report
> the cases we know may fail as XFAIL.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
> CC: liuhangbin@gmail.com
> CC: shuah@kernel.org
> CC: linux-kselftest@vger.kernel.org
> CC: Petr Machata <petrm@nvidia.com>
> CC: vladimir.oltean@nxp.com
>
> v3:
> - use xfail_on_veth correctly as a "prefix" call
> - dropping Vladimir's tags since the code is quite different now
> v2: https://lore.kernel.org/r/20240509235553.5740-1-kuba@kernel.org/
> - remove duplicated log_test_xfail
> v1: https://lore.kernel.org/all/20240509235553.5740-1-kuba@kernel.org/
> ---
> .../net/forwarding/local_termination.sh | 30 +++++++++++--------
> 1 file changed, 18 insertions(+), 12 deletions(-)
>
> diff --git a/tools/testing/selftests/net/forwarding/local_termination.sh b/tools/testing/selftests/net/forwarding/local_termination.sh
> index c5b0cbc85b3e..4b364cdf3ef0 100755
> --- a/tools/testing/selftests/net/forwarding/local_termination.sh
> +++ b/tools/testing/selftests/net/forwarding/local_termination.sh
> @@ -155,25 +155,30 @@ run_test()
> "$smac > $MACVLAN_ADDR, ethertype IPv4 (0x0800)" \
> true
>
> - check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address" \
> - "$smac > $UNKNOWN_UC_ADDR1, ethertype IPv4 (0x0800)" \
> - false
> + xfail_on_veth $h1 \
> + check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address" \
> + "$smac > $UNKNOWN_UC_ADDR1, ethertype IPv4 (0x0800)" \
> + false
>
> check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address, promisc" \
> "$smac > $UNKNOWN_UC_ADDR2, ethertype IPv4 (0x0800)" \
> true
>
> - check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address, allmulti" \
> - "$smac > $UNKNOWN_UC_ADDR3, ethertype IPv4 (0x0800)" \
> - false
> + xfail_on_veth $h1 \
> + check_rcv $rcv_if_name \
> + "Unicast IPv4 to unknown MAC address, allmulti" \
> + "$smac > $UNKNOWN_UC_ADDR3, ethertype IPv4 (0x0800)" \
> + false
>
> check_rcv $rcv_if_name "Multicast IPv4 to joined group" \
> "$smac > $JOINED_MACV4_MC_ADDR, ethertype IPv4 (0x0800)" \
> true
>
> - check_rcv $rcv_if_name "Multicast IPv4 to unknown group" \
> - "$smac > $UNKNOWN_MACV4_MC_ADDR1, ethertype IPv4 (0x0800)" \
> - false
> + xfail_on_veth $h1 \
> + check_rcv $rcv_if_name \
> + "Multicast IPv4 to unknown group" \
> + "$smac > $UNKNOWN_MACV4_MC_ADDR1, ethertype IPv4 (0x0800)" \
> + false
>
> check_rcv $rcv_if_name "Multicast IPv4 to unknown group, promisc" \
> "$smac > $UNKNOWN_MACV4_MC_ADDR2, ethertype IPv4 (0x0800)" \
> @@ -187,9 +192,10 @@ run_test()
> "$smac > $JOINED_MACV6_MC_ADDR, ethertype IPv6 (0x86dd)" \
> true
>
> - check_rcv $rcv_if_name "Multicast IPv6 to unknown group" \
> - "$smac > $UNKNOWN_MACV6_MC_ADDR1, ethertype IPv6 (0x86dd)" \
> - false
> + xfail_on_veth $h1 \
> + check_rcv $rcv_if_name "Multicast IPv6 to unknown group" \
> + "$smac > $UNKNOWN_MACV6_MC_ADDR1, ethertype IPv6 (0x86dd)" \
> + false
>
> check_rcv $rcv_if_name "Multicast IPv6 to unknown group, promisc" \
> "$smac > $UNKNOWN_MACV6_MC_ADDR2, ethertype IPv6 (0x86dd)" \
> --
> 2.45.0
>
Reviewed-by: Hangbin Liu <liuhangbin@gmail.com>
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH net v3] selftests: net: local_termination: annotate the expected failures
2024-05-17 8:54 ` Hangbin Liu
@ 2024-05-17 13:04 ` Petr Machata
0 siblings, 0 replies; 5+ messages in thread
From: Petr Machata @ 2024-05-17 13:04 UTC (permalink / raw)
To: Hangbin Liu
Cc: Jakub Kicinski, davem, netdev, edumazet, pabeni, shuah,
linux-kselftest, Petr Machata, vladimir.oltean
Hangbin Liu <liuhangbin@gmail.com> writes:
> On Thu, May 16, 2024 at 08:25:13AM -0700, Jakub Kicinski wrote:
>> Vladimir said when adding this test:
>>
>> The bridge driver fares particularly badly [...] mainly because
>> it does not implement IFF_UNICAST_FLT.
>>
>> See commit 90b9566aa5cd ("selftests: forwarding: add a test for
>> local_termination.sh").
>>
>> We don't want to hide the known gaps, but having a test which
>> always fails prevents us from catching regressions. Report
>> the cases we know may fail as XFAIL.
>>
>> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
I'm still confused that the failure was shown for $rcv_if_name==bridge,
yet we are testing $h1==veth. But mechanically the code is correct.
Reviewed-by: Petr Machata <petrm@nvidia.com>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH net v3] selftests: net: local_termination: annotate the expected failures
2024-05-16 15:25 [PATCH net v3] selftests: net: local_termination: annotate the expected failures Jakub Kicinski
2024-05-17 8:54 ` Hangbin Liu
@ 2024-05-17 8:54 ` Simon Horman
2024-05-17 20:38 ` patchwork-bot+netdevbpf
2 siblings, 0 replies; 5+ messages in thread
From: Simon Horman @ 2024-05-17 8:54 UTC (permalink / raw)
To: Jakub Kicinski
Cc: davem, netdev, edumazet, pabeni, liuhangbin, shuah,
linux-kselftest, Petr Machata, vladimir.oltean
On Thu, May 16, 2024 at 08:25:13AM -0700, Jakub Kicinski wrote:
> Vladimir said when adding this test:
>
> The bridge driver fares particularly badly [...] mainly because
> it does not implement IFF_UNICAST_FLT.
>
> See commit 90b9566aa5cd ("selftests: forwarding: add a test for
> local_termination.sh").
>
> We don't want to hide the known gaps, but having a test which
> always fails prevents us from catching regressions. Report
> the cases we know may fail as XFAIL.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
I agree that XFAIL is appropriate for cases like this - the test
is expected to fail and this can act like a TODO.
I also looked over how xfail_on_veth works and this usage seems
correct to me.
Reviewed-by: Simon Horman <horms@kernel.org>
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH net v3] selftests: net: local_termination: annotate the expected failures
2024-05-16 15:25 [PATCH net v3] selftests: net: local_termination: annotate the expected failures Jakub Kicinski
2024-05-17 8:54 ` Hangbin Liu
2024-05-17 8:54 ` Simon Horman
@ 2024-05-17 20:38 ` patchwork-bot+netdevbpf
2 siblings, 0 replies; 5+ messages in thread
From: patchwork-bot+netdevbpf @ 2024-05-17 20:38 UTC (permalink / raw)
To: Jakub Kicinski
Cc: davem, netdev, edumazet, pabeni, liuhangbin, shuah,
linux-kselftest, petrm, vladimir.oltean
Hello:
This patch was applied to netdev/net.git (main)
by Jakub Kicinski <kuba@kernel.org>:
On Thu, 16 May 2024 08:25:13 -0700 you wrote:
> Vladimir said when adding this test:
>
> The bridge driver fares particularly badly [...] mainly because
> it does not implement IFF_UNICAST_FLT.
>
> See commit 90b9566aa5cd ("selftests: forwarding: add a test for
> local_termination.sh").
>
> [...]
Here is the summary with links:
- [net,v3] selftests: net: local_termination: annotate the expected failures
https://git.kernel.org/netdev/net/c/fe56d6e4a99a
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2024-05-17 20:38 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-05-16 15:25 [PATCH net v3] selftests: net: local_termination: annotate the expected failures Jakub Kicinski
2024-05-17 8:54 ` Hangbin Liu
2024-05-17 13:04 ` Petr Machata
2024-05-17 8:54 ` Simon Horman
2024-05-17 20:38 ` patchwork-bot+netdevbpf
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).