[PATCH net-next 0/4] selftests: fib_tests: simplifications, verbosity and a race

[PATCH net-next 0/4] selftests: fib_tests: simplifications, verbosity and a race

[David Ahern]

Improve efficiency of fib_tests.sh and make the test result more verbose,
from this summary:
$ fib_tests.sh is failing in a VM:
    $ fib_tests.sh
    Running netdev unregister tests
    PASS: unicast route test
    PASS: multipath route test
    Running netdev down tests
    PASS: unicast route test
    PASS: multipath route test
    Running netdev carrier change tests
    PASS: local route carrier test
    FAIL: unicast route carrier test

where a single entry actually corresponds to many checks to a much more
verbse output that clarifies test cases:
$fib_tests.sh
Single path route carrier test
    ....
    Carrier down
        IPv4 fibmatch                                         [ OK ]
        IPv6 fibmatch                                         [ OK ]
        IPv4 linkdown flag set                                [FAIL]
        IPv6 linkdown flag set                                [FAIL]
    Second address added with carrier down
        IPv4 fibmatch                                         [ OK ]
        IPv6 fibmatch                                         [ OK ]
        IPv4 linkdown flag set                                [FAIL]
        IPv6 linkdown flag set                                [ OK ]

And then fix the race in changing carrier down on dummy device to checking
the corresponding routes.

David Ahern (4):
  selftests: fib_tests: simplify ip commands in a namespace
  selftests: fib_tests: Make test results more verbose
  selftests: fib_tests: Move admin of dummy0 to helpers
  selftests: fib_tests: sleep after changing carrier

 tools/testing/selftests/net/fib_tests.sh | 483 +++++++++++++++----------------
 1 file changed, 237 insertions(+), 246 deletions(-)

-- 
2.11.0

[PATCH net-next 1/4] selftests: fib_tests: simplify ip commands in a namespace

[David Ahern]

'ip netns exec testns ip' is more efficiently handled using 'ip -netns';
runs the ip command after switching the namespace and avoids an exec.

Signed-off-by: David Ahern <dsahern@gmail.com>
---
 tools/testing/selftests/net/fib_tests.sh | 210 +++++++++++++++----------------
 1 file changed, 105 insertions(+), 105 deletions(-)

diff --git a/tools/testing/selftests/net/fib_tests.sh b/tools/testing/selftests/net/fib_tests.sh
index a9154eefb2e2..672f263f8f53 100755
--- a/tools/testing/selftests/net/fib_tests.sh
+++ b/tools/testing/selftests/net/fib_tests.sh
@@ -34,23 +34,23 @@ fib_unreg_unicast_test()
 
 	netns_create "testns"
 
-	ip netns exec testns ip link add dummy0 type dummy
-	ip netns exec testns ip link set dev dummy0 up
+	ip -netns testns link add dummy0 type dummy
+	ip -netns testns link set dev dummy0 up
 
-	ip netns exec testns ip address add 198.51.100.1/24 dev dummy0
-	ip netns exec testns ip -6 address add 2001:db8:1::1/64 dev dummy0
+	ip -netns testns address add 198.51.100.1/24 dev dummy0
+	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
 
-	ip netns exec testns ip route get fibmatch 198.51.100.2 &> /dev/null
+	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:1::2 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
 	check_err $?
 
-	ip netns exec testns ip link del dev dummy0
+	ip -netns testns link del dev dummy0
 	check_err $?
 
-	ip netns exec testns ip route get fibmatch 198.51.100.2 &> /dev/null
+	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
 	check_fail $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:1::2 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
 	check_fail $?
 
 	ip netns del testns
@@ -68,40 +68,40 @@ fib_unreg_multipath_test()
 
 	netns_create "testns"
 
-	ip netns exec testns ip link add dummy0 type dummy
-	ip netns exec testns ip link set dev dummy0 up
+	ip -netns testns link add dummy0 type dummy
+	ip -netns testns link set dev dummy0 up
 
-	ip netns exec testns ip link add dummy1 type dummy
-	ip netns exec testns ip link set dev dummy1 up
+	ip -netns testns link add dummy1 type dummy
+	ip -netns testns link set dev dummy1 up
 
-	ip netns exec testns ip address add 198.51.100.1/24 dev dummy0
-	ip netns exec testns ip -6 address add 2001:db8:1::1/64 dev dummy0
+	ip -netns testns address add 198.51.100.1/24 dev dummy0
+	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
 
-	ip netns exec testns ip address add 192.0.2.1/24 dev dummy1
-	ip netns exec testns ip -6 address add 2001:db8:2::1/64 dev dummy1
+	ip -netns testns address add 192.0.2.1/24 dev dummy1
+	ip -netns testns -6 address add 2001:db8:2::1/64 dev dummy1
 
-	ip netns exec testns ip route add 203.0.113.0/24 \
+	ip -netns testns route add 203.0.113.0/24 \
 		nexthop via 198.51.100.2 dev dummy0 \
 		nexthop via 192.0.2.2 dev dummy1
-	ip netns exec testns ip -6 route add 2001:db8:3::/64 \
+	ip -netns testns -6 route add 2001:db8:3::/64 \
 		nexthop via 2001:db8:1::2 dev dummy0 \
 		nexthop via 2001:db8:2::2 dev dummy1
 
-	ip netns exec testns ip route get fibmatch 203.0.113.1 &> /dev/null
+	ip -netns testns route get fibmatch 203.0.113.1 &> /dev/null
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:3::1 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:3::1 &> /dev/null
 	check_err $?
 
-	ip netns exec testns ip link del dev dummy0
+	ip -netns testns link del dev dummy0
 	check_err $?
 
-	ip netns exec testns ip route get fibmatch 203.0.113.1 &> /dev/null
+	ip -netns testns route get fibmatch 203.0.113.1 &> /dev/null
 	check_fail $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:3::1 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:3::1 &> /dev/null
 	# In IPv6 we do not flush the entire multipath route.
 	check_err $?
 
-	ip netns exec testns ip link del dev dummy1
+	ip -netns testns link del dev dummy1
 
 	ip netns del testns
 
@@ -126,26 +126,26 @@ fib_down_unicast_test()
 
 	netns_create "testns"
 
-	ip netns exec testns ip link add dummy0 type dummy
-	ip netns exec testns ip link set dev dummy0 up
+	ip -netns testns link add dummy0 type dummy
+	ip -netns testns link set dev dummy0 up
 
-	ip netns exec testns ip address add 198.51.100.1/24 dev dummy0
-	ip netns exec testns ip -6 address add 2001:db8:1::1/64 dev dummy0
+	ip -netns testns address add 198.51.100.1/24 dev dummy0
+	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
 
-	ip netns exec testns ip route get fibmatch 198.51.100.2 &> /dev/null
+	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:1::2 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
 	check_err $?
 
-	ip netns exec testns ip link set dev dummy0 down
+	ip -netns testns link set dev dummy0 down
 	check_err $?
 
-	ip netns exec testns ip route get fibmatch 198.51.100.2 &> /dev/null
+	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
 	check_fail $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:1::2 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
 	check_fail $?
 
-	ip netns exec testns ip link del dev dummy0
+	ip -netns testns link del dev dummy0
 
 	ip netns del testns
 
@@ -161,31 +161,31 @@ fib_down_multipath_test_do()
 	local down_dev=$1
 	local up_dev=$2
 
-	ip netns exec testns ip route get fibmatch 203.0.113.1 \
+	ip -netns testns route get fibmatch 203.0.113.1 \
 		oif $down_dev &> /dev/null
 	check_fail $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:3::1 \
+	ip -netns testns -6 route get fibmatch 2001:db8:3::1 \
 		oif $down_dev &> /dev/null
 	check_fail $?
 
-	ip netns exec testns ip route get fibmatch 203.0.113.1 \
+	ip -netns testns route get fibmatch 203.0.113.1 \
 		oif $up_dev &> /dev/null
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:3::1 \
+	ip -netns testns -6 route get fibmatch 2001:db8:3::1 \
 		oif $up_dev &> /dev/null
 	check_err $?
 
-	ip netns exec testns ip route get fibmatch 203.0.113.1 | \
+	ip -netns testns route get fibmatch 203.0.113.1 | \
 		grep $down_dev | grep -q "dead linkdown"
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:3::1 | \
+	ip -netns testns -6 route get fibmatch 2001:db8:3::1 | \
 		grep $down_dev | grep -q "dead linkdown"
 	check_err $?
 
-	ip netns exec testns ip route get fibmatch 203.0.113.1 | \
+	ip -netns testns route get fibmatch 203.0.113.1 | \
 		grep $up_dev | grep -q "dead linkdown"
 	check_fail $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:3::1 | \
+	ip -netns testns -6 route get fibmatch 2001:db8:3::1 | \
 		grep $up_dev | grep -q "dead linkdown"
 	check_fail $?
 }
@@ -196,52 +196,52 @@ fib_down_multipath_test()
 
 	netns_create "testns"
 
-	ip netns exec testns ip link add dummy0 type dummy
-	ip netns exec testns ip link set dev dummy0 up
+	ip -netns testns link add dummy0 type dummy
+	ip -netns testns link set dev dummy0 up
 
-	ip netns exec testns ip link add dummy1 type dummy
-	ip netns exec testns ip link set dev dummy1 up
+	ip -netns testns link add dummy1 type dummy
+	ip -netns testns link set dev dummy1 up
 
-	ip netns exec testns ip address add 198.51.100.1/24 dev dummy0
-	ip netns exec testns ip -6 address add 2001:db8:1::1/64 dev dummy0
+	ip -netns testns address add 198.51.100.1/24 dev dummy0
+	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
 
-	ip netns exec testns ip address add 192.0.2.1/24 dev dummy1
-	ip netns exec testns ip -6 address add 2001:db8:2::1/64 dev dummy1
+	ip -netns testns address add 192.0.2.1/24 dev dummy1
+	ip -netns testns -6 address add 2001:db8:2::1/64 dev dummy1
 
-	ip netns exec testns ip route add 203.0.113.0/24 \
+	ip -netns testns route add 203.0.113.0/24 \
 		nexthop via 198.51.100.2 dev dummy0 \
 		nexthop via 192.0.2.2 dev dummy1
-	ip netns exec testns ip -6 route add 2001:db8:3::/64 \
+	ip -netns testns -6 route add 2001:db8:3::/64 \
 		nexthop via 2001:db8:1::2 dev dummy0 \
 		nexthop via 2001:db8:2::2 dev dummy1
 
-	ip netns exec testns ip route get fibmatch 203.0.113.1 &> /dev/null
+	ip -netns testns route get fibmatch 203.0.113.1 &> /dev/null
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:3::1 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:3::1 &> /dev/null
 	check_err $?
 
-	ip netns exec testns ip link set dev dummy0 down
+	ip -netns testns link set dev dummy0 down
 	check_err $?
 
 	fib_down_multipath_test_do "dummy0" "dummy1"
 
-	ip netns exec testns ip link set dev dummy0 up
+	ip -netns testns link set dev dummy0 up
 	check_err $?
-	ip netns exec testns ip link set dev dummy1 down
+	ip -netns testns link set dev dummy1 down
 	check_err $?
 
 	fib_down_multipath_test_do "dummy1" "dummy0"
 
-	ip netns exec testns ip link set dev dummy0 down
+	ip -netns testns link set dev dummy0 down
 	check_err $?
 
-	ip netns exec testns ip route get fibmatch 203.0.113.1 &> /dev/null
+	ip -netns testns route get fibmatch 203.0.113.1 &> /dev/null
 	check_fail $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:3::1 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:3::1 &> /dev/null
 	check_fail $?
 
-	ip netns exec testns ip link del dev dummy1
-	ip netns exec testns ip link del dev dummy0
+	ip -netns testns link del dev dummy1
+	ip -netns testns link del dev dummy0
 
 	ip netns del testns
 
@@ -267,56 +267,56 @@ fib_carrier_local_test()
 	# Local routes should not be affected when carrier changes.
 	netns_create "testns"
 
-	ip netns exec testns ip link add dummy0 type dummy
-	ip netns exec testns ip link set dev dummy0 up
+	ip -netns testns link add dummy0 type dummy
+	ip -netns testns link set dev dummy0 up
 
-	ip netns exec testns ip link set dev dummy0 carrier on
+	ip -netns testns link set dev dummy0 carrier on
 
-	ip netns exec testns ip address add 198.51.100.1/24 dev dummy0
-	ip netns exec testns ip -6 address add 2001:db8:1::1/64 dev dummy0
+	ip -netns testns address add 198.51.100.1/24 dev dummy0
+	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
 
-	ip netns exec testns ip route get fibmatch 198.51.100.1 &> /dev/null
+	ip -netns testns route get fibmatch 198.51.100.1 &> /dev/null
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:1::1 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:1::1 &> /dev/null
 	check_err $?
 
-	ip netns exec testns ip route get fibmatch 198.51.100.1 | \
+	ip -netns testns route get fibmatch 198.51.100.1 | \
 		grep -q "linkdown"
 	check_fail $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:1::1 | \
+	ip -netns testns -6 route get fibmatch 2001:db8:1::1 | \
 		grep -q "linkdown"
 	check_fail $?
 
-	ip netns exec testns ip link set dev dummy0 carrier off
+	ip -netns testns link set dev dummy0 carrier off
 
-	ip netns exec testns ip route get fibmatch 198.51.100.1 &> /dev/null
+	ip -netns testns route get fibmatch 198.51.100.1 &> /dev/null
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:1::1 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:1::1 &> /dev/null
 	check_err $?
 
-	ip netns exec testns ip route get fibmatch 198.51.100.1 | \
+	ip -netns testns route get fibmatch 198.51.100.1 | \
 		grep -q "linkdown"
 	check_fail $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:1::1 | \
+	ip -netns testns -6 route get fibmatch 2001:db8:1::1 | \
 		grep -q "linkdown"
 	check_fail $?
 
-	ip netns exec testns ip address add 192.0.2.1/24 dev dummy0
-	ip netns exec testns ip -6 address add 2001:db8:2::1/64 dev dummy0
+	ip -netns testns address add 192.0.2.1/24 dev dummy0
+	ip -netns testns -6 address add 2001:db8:2::1/64 dev dummy0
 
-	ip netns exec testns ip route get fibmatch 192.0.2.1 &> /dev/null
+	ip -netns testns route get fibmatch 192.0.2.1 &> /dev/null
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:2::1 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:2::1 &> /dev/null
 	check_err $?
 
-	ip netns exec testns ip route get fibmatch 192.0.2.1 | \
+	ip -netns testns route get fibmatch 192.0.2.1 | \
 		grep -q "linkdown"
 	check_fail $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:2::1 | \
+	ip -netns testns -6 route get fibmatch 2001:db8:2::1 | \
 		grep -q "linkdown"
 	check_fail $?
 
-	ip netns exec testns ip link del dev dummy0
+	ip -netns testns link del dev dummy0
 
 	ip netns del testns
 
@@ -333,56 +333,56 @@ fib_carrier_unicast_test()
 
 	netns_create "testns"
 
-	ip netns exec testns ip link add dummy0 type dummy
-	ip netns exec testns ip link set dev dummy0 up
+	ip -netns testns link add dummy0 type dummy
+	ip -netns testns link set dev dummy0 up
 
-	ip netns exec testns ip link set dev dummy0 carrier on
+	ip -netns testns link set dev dummy0 carrier on
 
-	ip netns exec testns ip address add 198.51.100.1/24 dev dummy0
-	ip netns exec testns ip -6 address add 2001:db8:1::1/64 dev dummy0
+	ip -netns testns address add 198.51.100.1/24 dev dummy0
+	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
 
-	ip netns exec testns ip route get fibmatch 198.51.100.2 &> /dev/null
+	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:1::2 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
 	check_err $?
 
-	ip netns exec testns ip route get fibmatch 198.51.100.2 | \
+	ip -netns testns route get fibmatch 198.51.100.2 | \
 		grep -q "linkdown"
 	check_fail $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:1::2 | \
+	ip -netns testns -6 route get fibmatch 2001:db8:1::2 | \
 		grep -q "linkdown"
 	check_fail $?
 
-	ip netns exec testns ip link set dev dummy0 carrier off
+	ip -netns testns link set dev dummy0 carrier off
 
-	ip netns exec testns ip route get fibmatch 198.51.100.2 &> /dev/null
+	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:1::2 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
 	check_err $?
 
-	ip netns exec testns ip route get fibmatch 198.51.100.2 | \
+	ip -netns testns route get fibmatch 198.51.100.2 | \
 		grep -q "linkdown"
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:1::2 | \
+	ip -netns testns -6 route get fibmatch 2001:db8:1::2 | \
 		grep -q "linkdown"
 	check_err $?
 
-	ip netns exec testns ip address add 192.0.2.1/24 dev dummy0
-	ip netns exec testns ip -6 address add 2001:db8:2::1/64 dev dummy0
+	ip -netns testns address add 192.0.2.1/24 dev dummy0
+	ip -netns testns -6 address add 2001:db8:2::1/64 dev dummy0
 
-	ip netns exec testns ip route get fibmatch 192.0.2.2 &> /dev/null
+	ip -netns testns route get fibmatch 192.0.2.2 &> /dev/null
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:2::2 &> /dev/null
+	ip -netns testns -6 route get fibmatch 2001:db8:2::2 &> /dev/null
 	check_err $?
 
-	ip netns exec testns ip route get fibmatch 192.0.2.2 | \
+	ip -netns testns route get fibmatch 192.0.2.2 | \
 		grep -q "linkdown"
 	check_err $?
-	ip netns exec testns ip -6 route get fibmatch 2001:db8:2::2 | \
+	ip -netns testns -6 route get fibmatch 2001:db8:2::2 | \
 		grep -q "linkdown"
 	check_err $?
 
-	ip netns exec testns ip link del dev dummy0
+	ip -netns testns link del dev dummy0
 
 	ip netns del testns
 
-- 
2.11.0

[PATCH net-next 2/4] selftests: fib_tests: Make test results more verbose

[David Ahern]

fib_tests.sh is failing in a VM:
    $ fib_tests.sh
    Running netdev unregister tests
    PASS: unicast route test
    PASS: multipath route test
    Running netdev down tests
    PASS: unicast route test
    PASS: multipath route test
    Running netdev carrier change tests
    PASS: local route carrier test
    FAIL: unicast route carrier test

The last test corresponds to fib_carrier_unicast_test which 12 places
that could be failing. Be more verbose in the output so a failure is
easier to track down and separate test setup failures with set -e and
set +e pairs.

With the verbose logging it is easier to see which checks are failing:
    $fib_tests.sh
    Single path route carrier test
        ....
        Carrier down
            IPv4 fibmatch                                         [ OK ]
            IPv6 fibmatch                                         [ OK ]
            IPv4 linkdown flag set                                [FAIL]
            IPv6 linkdown flag set                                [FAIL]
        Second address added with carrier down
            IPv4 fibmatch                                         [ OK ]
            IPv6 fibmatch                                         [ OK ]
            IPv4 linkdown flag set                                [FAIL]
            IPv6 linkdown flag set                                [ OK ]

Signed-off-by: David Ahern <dsahern@gmail.com>
---
 tools/testing/selftests/net/fib_tests.sh | 248 +++++++++++++++++--------------
 1 file changed, 135 insertions(+), 113 deletions(-)

diff --git a/tools/testing/selftests/net/fib_tests.sh b/tools/testing/selftests/net/fib_tests.sh
index 672f263f8f53..d4e0b5cb4355 100755
--- a/tools/testing/selftests/net/fib_tests.sh
+++ b/tools/testing/selftests/net/fib_tests.sh
@@ -6,17 +6,25 @@
 
 ret=0
 
-check_err()
-{
-	if [ $ret -eq 0 ]; then
-		ret=$1
-	fi
-}
+PAUSE_ON_FAIL=${PAUSE_ON_FAIL:=no}
 
-check_fail()
+log_test()
 {
-	if [ $1 -eq 0 ]; then
+	local rc=$1
+	local expected=$2
+	local msg="$3"
+
+	if [ ${rc} -eq ${expected} ]; then
+		printf "        %-60s  [ OK ]\n" "${msg}"
+	else
 		ret=1
+		printf "        %-60s  [FAIL]\n" "${msg}"
+		if [ "${PAUSE_ON_FAIL}" = "yes" ]; then
+		echo
+			echo "hit enter to continue, 'q' to quit"
+			read a
+			[ "$a" = "q" ] && exit 1
+		fi
 	fi
 }
 
@@ -30,42 +38,44 @@ netns_create()
 
 fib_unreg_unicast_test()
 {
-	ret=0
+	echo
+	echo "Single path route test"
 
+	set -e
 	netns_create "testns"
 
 	ip -netns testns link add dummy0 type dummy
 	ip -netns testns link set dev dummy0 up
-
 	ip -netns testns address add 198.51.100.1/24 dev dummy0
 	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
+	set +e
 
+	echo "    Start point"
 	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv4 fibmatch"
 	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv6 fibmatch"
 
+	set -e
 	ip -netns testns link del dev dummy0
-	check_err $?
+	set +e
 
+	echo "    Nexthop device deleted"
 	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
-	check_fail $?
+	log_test $? 2 "IPv4 fibmatch - no route"
 	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
-	check_fail $?
+	log_test $? 2 "IPv6 fibmatch - no route"
 
 	ip netns del testns
-
-	if [ $ret -ne 0 ]; then
-		echo "FAIL: unicast route test"
-		return 1
-	fi
-	echo "PASS: unicast route test"
 }
 
 fib_unreg_multipath_test()
 {
-	ret=0
 
+	echo
+	echo "Multipath route test"
+
+	set -e
 	netns_create "testns"
 
 	ip -netns testns link add dummy0 type dummy
@@ -86,44 +96,49 @@ fib_unreg_multipath_test()
 	ip -netns testns -6 route add 2001:db8:3::/64 \
 		nexthop via 2001:db8:1::2 dev dummy0 \
 		nexthop via 2001:db8:2::2 dev dummy1
+	set +e
 
+	echo "    Start point"
 	ip -netns testns route get fibmatch 203.0.113.1 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv4 fibmatch"
 	ip -netns testns -6 route get fibmatch 2001:db8:3::1 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv6 fibmatch"
 
+	set -e
 	ip -netns testns link del dev dummy0
-	check_err $?
+	set +e
 
+	echo "    One nexthop device deleted"
 	ip -netns testns route get fibmatch 203.0.113.1 &> /dev/null
-	check_fail $?
+	log_test $? 2 "IPv4 - multipath route removed on delete"
+
 	ip -netns testns -6 route get fibmatch 2001:db8:3::1 &> /dev/null
 	# In IPv6 we do not flush the entire multipath route.
-	check_err $?
+	log_test $? 0 "IPv6 - multipath down to single path"
 
+	set -e
 	ip -netns testns link del dev dummy1
+	set +e
 
-	ip netns del testns
+	echo "    Second nexthop device deleted"
+	ip -netns testns -6 route get fibmatch 2001:db8:3::1 &> /dev/null
+	log_test $? 2 "IPv6 - no route"
 
-	if [ $ret -ne 0 ]; then
-		echo "FAIL: multipath route test"
-		return 1
-	fi
-	echo "PASS: multipath route test"
+	ip netns del testns
 }
 
 fib_unreg_test()
 {
-	echo "Running netdev unregister tests"
-
 	fib_unreg_unicast_test
 	fib_unreg_multipath_test
 }
 
 fib_down_unicast_test()
 {
-	ret=0
+	echo
+	echo "Single path, admin down"
 
+	set -e
 	netns_create "testns"
 
 	ip -netns testns link add dummy0 type dummy
@@ -131,29 +146,27 @@ fib_down_unicast_test()
 
 	ip -netns testns address add 198.51.100.1/24 dev dummy0
 	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
+	set +e
 
+	echo "    Start point"
 	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv4 fibmatch"
 	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv6 fibmatch"
 
+	set -e
 	ip -netns testns link set dev dummy0 down
-	check_err $?
+	set +e
 
+	echo "    Route deleted on down"
 	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
-	check_fail $?
+	log_test $? 2 "IPv4 fibmatch"
 	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
-	check_fail $?
+	log_test $? 2 "IPv6 fibmatch"
 
 	ip -netns testns link del dev dummy0
 
 	ip netns del testns
-
-	if [ $ret -ne 0 ]; then
-		echo "FAIL: unicast route test"
-		return 1
-	fi
-	echo "PASS: unicast route test"
 }
 
 fib_down_multipath_test_do()
@@ -163,37 +176,39 @@ fib_down_multipath_test_do()
 
 	ip -netns testns route get fibmatch 203.0.113.1 \
 		oif $down_dev &> /dev/null
-	check_fail $?
+	log_test $? 2 "IPv4 fibmatch on down device"
 	ip -netns testns -6 route get fibmatch 2001:db8:3::1 \
 		oif $down_dev &> /dev/null
-	check_fail $?
+	log_test $? 2 "IPv6 fibmatch on down device"
 
 	ip -netns testns route get fibmatch 203.0.113.1 \
 		oif $up_dev &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv4 fibmatch on up device"
 	ip -netns testns -6 route get fibmatch 2001:db8:3::1 \
 		oif $up_dev &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv6 fibmatch on up device"
 
 	ip -netns testns route get fibmatch 203.0.113.1 | \
 		grep $down_dev | grep -q "dead linkdown"
-	check_err $?
+	log_test $? 0 "IPv4 flags on down device"
 	ip -netns testns -6 route get fibmatch 2001:db8:3::1 | \
 		grep $down_dev | grep -q "dead linkdown"
-	check_err $?
+	log_test $? 0 "IPv6 flags on down device"
 
 	ip -netns testns route get fibmatch 203.0.113.1 | \
 		grep $up_dev | grep -q "dead linkdown"
-	check_fail $?
+	log_test $? 1 "IPv4 flags on up device"
 	ip -netns testns -6 route get fibmatch 2001:db8:3::1 | \
 		grep $up_dev | grep -q "dead linkdown"
-	check_fail $?
+	log_test $? 1 "IPv6 flags on up device"
 }
 
 fib_down_multipath_test()
 {
-	ret=0
+	echo
+	echo "Admin down multipath"
 
+	set -e
 	netns_create "testns"
 
 	ip -netns testns link add dummy0 type dummy
@@ -214,57 +229,58 @@ fib_down_multipath_test()
 	ip -netns testns -6 route add 2001:db8:3::/64 \
 		nexthop via 2001:db8:1::2 dev dummy0 \
 		nexthop via 2001:db8:2::2 dev dummy1
+	set +e
 
+	echo "    Verify start point"
 	ip -netns testns route get fibmatch 203.0.113.1 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv4 fibmatch"
+
 	ip -netns testns -6 route get fibmatch 2001:db8:3::1 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv6 fibmatch"
 
+	set -e
 	ip -netns testns link set dev dummy0 down
-	check_err $?
+	set +e
 
+	echo "    One device down, one up"
 	fib_down_multipath_test_do "dummy0" "dummy1"
 
+	set -e
 	ip -netns testns link set dev dummy0 up
-	check_err $?
 	ip -netns testns link set dev dummy1 down
-	check_err $?
+	set +e
 
+	echo "    Other device down and up"
 	fib_down_multipath_test_do "dummy1" "dummy0"
 
+	set -e
 	ip -netns testns link set dev dummy0 down
-	check_err $?
+	set +e
 
+	echo "    Both devices down"
 	ip -netns testns route get fibmatch 203.0.113.1 &> /dev/null
-	check_fail $?
+	log_test $? 2 "IPv4 fibmatch"
 	ip -netns testns -6 route get fibmatch 2001:db8:3::1 &> /dev/null
-	check_fail $?
+	log_test $? 2 "IPv6 fibmatch"
 
 	ip -netns testns link del dev dummy1
 	ip -netns testns link del dev dummy0
-
 	ip netns del testns
-
-	if [ $ret -ne 0 ]; then
-		echo "FAIL: multipath route test"
-		return 1
-	fi
-	echo "PASS: multipath route test"
 }
 
 fib_down_test()
 {
-	echo "Running netdev down tests"
-
 	fib_down_unicast_test
 	fib_down_multipath_test
 }
 
+# Local routes should not be affected when carrier changes.
 fib_carrier_local_test()
 {
-	ret=0
+	echo
+	echo "Local carrier tests - single path"
 
-	# Local routes should not be affected when carrier changes.
+	set -e
 	netns_create "testns"
 
 	ip -netns testns link add dummy0 type dummy
@@ -274,65 +290,71 @@ fib_carrier_local_test()
 
 	ip -netns testns address add 198.51.100.1/24 dev dummy0
 	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
+	set +e
 
+	echo "    Start point"
 	ip -netns testns route get fibmatch 198.51.100.1 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv4 fibmatch"
 	ip -netns testns -6 route get fibmatch 2001:db8:1::1 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv6 fibmatch"
 
 	ip -netns testns route get fibmatch 198.51.100.1 | \
 		grep -q "linkdown"
-	check_fail $?
+	log_test $? 1 "IPv4 - no linkdown flag"
 	ip -netns testns -6 route get fibmatch 2001:db8:1::1 | \
 		grep -q "linkdown"
-	check_fail $?
+	log_test $? 1 "IPv6 - no linkdown flag"
 
+	set -e
 	ip -netns testns link set dev dummy0 carrier off
+	set +e
 
+	echo "    Carrier off on nexthop"
 	ip -netns testns route get fibmatch 198.51.100.1 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv4 fibmatch"
 	ip -netns testns -6 route get fibmatch 2001:db8:1::1 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv6 fibmatch"
 
 	ip -netns testns route get fibmatch 198.51.100.1 | \
 		grep -q "linkdown"
-	check_fail $?
+	log_test $? 1 "IPv4 - linkdown flag set"
 	ip -netns testns -6 route get fibmatch 2001:db8:1::1 | \
 		grep -q "linkdown"
-	check_fail $?
+	log_test $? 1 "IPv6 - linkdown flag set"
 
+	set -e
 	ip -netns testns address add 192.0.2.1/24 dev dummy0
 	ip -netns testns -6 address add 2001:db8:2::1/64 dev dummy0
+	set +e
 
+	echo "    Route to local address with carrier down"
 	ip -netns testns route get fibmatch 192.0.2.1 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv4 fibmatch"
 	ip -netns testns -6 route get fibmatch 2001:db8:2::1 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv6 fibmatch"
 
 	ip -netns testns route get fibmatch 192.0.2.1 | \
 		grep -q "linkdown"
-	check_fail $?
+	log_test $? 1 "IPv4 linkdown flag set"
 	ip -netns testns -6 route get fibmatch 2001:db8:2::1 | \
 		grep -q "linkdown"
-	check_fail $?
+	log_test $? 1 "IPv6 linkdown flag set"
 
 	ip -netns testns link del dev dummy0
 
 	ip netns del testns
-
-	if [ $ret -ne 0 ]; then
-		echo "FAIL: local route carrier test"
-		return 1
-	fi
-	echo "PASS: local route carrier test"
 }
 
 fib_carrier_unicast_test()
 {
 	ret=0
 
+	echo
+	echo "Single path route carrier test"
+
 	netns_create "testns"
 
+	set -e
 	ip -netns testns link add dummy0 type dummy
 	ip -netns testns link set dev dummy0 up
 
@@ -340,63 +362,63 @@ fib_carrier_unicast_test()
 
 	ip -netns testns address add 198.51.100.1/24 dev dummy0
 	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
+	set +e
 
+	echo "    Start point"
 	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv4 fibmatch"
 	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv6 fibmatch"
 
 	ip -netns testns route get fibmatch 198.51.100.2 | \
 		grep -q "linkdown"
-	check_fail $?
+	log_test $? 1 "IPv4 no linkdown flag"
 	ip -netns testns -6 route get fibmatch 2001:db8:1::2 | \
 		grep -q "linkdown"
-	check_fail $?
+	log_test $? 1 "IPv6 no linkdown flag"
 
+	set -e
 	ip -netns testns link set dev dummy0 carrier off
+	set +e
 
+	echo "    Carrier down"
 	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv4 fibmatch"
 	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv6 fibmatch"
 
 	ip -netns testns route get fibmatch 198.51.100.2 | \
 		grep -q "linkdown"
-	check_err $?
+	log_test $? 0 "IPv4 linkdown flag set"
 	ip -netns testns -6 route get fibmatch 2001:db8:1::2 | \
 		grep -q "linkdown"
-	check_err $?
+	log_test $? 0 "IPv6 linkdown flag set"
 
+	set -e
 	ip -netns testns address add 192.0.2.1/24 dev dummy0
 	ip -netns testns -6 address add 2001:db8:2::1/64 dev dummy0
+	set +e
 
+	echo "    Second address added with carrier down"
 	ip -netns testns route get fibmatch 192.0.2.2 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv4 fibmatch"
 	ip -netns testns -6 route get fibmatch 2001:db8:2::2 &> /dev/null
-	check_err $?
+	log_test $? 0 "IPv6 fibmatch"
 
 	ip -netns testns route get fibmatch 192.0.2.2 | \
 		grep -q "linkdown"
-	check_err $?
+	log_test $? 0 "IPv4 linkdown flag set"
 	ip -netns testns -6 route get fibmatch 2001:db8:2::2 | \
 		grep -q "linkdown"
-	check_err $?
+	log_test $? 0 "IPv6 linkdown flag set"
 
 	ip -netns testns link del dev dummy0
 
 	ip netns del testns
-
-	if [ $ret -ne 0 ]; then
-		echo "FAIL: unicast route carrier test"
-		return 1
-	fi
-	echo "PASS: unicast route carrier test"
 }
 
 fib_carrier_test()
 {
-	echo "Running netdev carrier change tests"
-
 	fib_carrier_local_test
 	fib_carrier_unicast_test
 }
-- 
2.11.0

[PATCH net-next 3/4] selftests: fib_tests: Move admin of dummy0 to helpers

[David Ahern]

Move setup and teardown of testns and dummy0 to helpers.

Signed-off-by: David Ahern <dsahern@gmail.com>
---
 tools/testing/selftests/net/fib_tests.sh | 100 +++++++++++--------------------
 1 file changed, 34 insertions(+), 66 deletions(-)

diff --git a/tools/testing/selftests/net/fib_tests.sh b/tools/testing/selftests/net/fib_tests.sh
index d4e0b5cb4355..e113cfd659fc 100755
--- a/tools/testing/selftests/net/fib_tests.sh
+++ b/tools/testing/selftests/net/fib_tests.sh
@@ -28,12 +28,24 @@ log_test()
 	fi
 }
 
-netns_create()
+setup()
 {
-	local testns=$1
+	set -e
+	ip netns add testns
+	ip -netns testns link set dev lo up
+
+	ip -netns testns link add dummy0 type dummy
+	ip -netns testns link set dev dummy0 up
+	ip -netns testns address add 198.51.100.1/24 dev dummy0
+	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
+	set +e
+
+}
 
-	ip netns add $testns
-	ip netns exec $testns ip link set dev lo up
+cleanup()
+{
+	ip -netns testns link del dev dummy0 &> /dev/null
+	ip netns del testns
 }
 
 fib_unreg_unicast_test()
@@ -41,14 +53,7 @@ fib_unreg_unicast_test()
 	echo
 	echo "Single path route test"
 
-	set -e
-	netns_create "testns"
-
-	ip -netns testns link add dummy0 type dummy
-	ip -netns testns link set dev dummy0 up
-	ip -netns testns address add 198.51.100.1/24 dev dummy0
-	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
-	set +e
+	setup
 
 	echo "    Start point"
 	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
@@ -66,7 +71,7 @@ fib_unreg_unicast_test()
 	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
 	log_test $? 2 "IPv6 fibmatch - no route"
 
-	ip netns del testns
+	cleanup
 }
 
 fib_unreg_multipath_test()
@@ -75,18 +80,11 @@ fib_unreg_multipath_test()
 	echo
 	echo "Multipath route test"
 
-	set -e
-	netns_create "testns"
-
-	ip -netns testns link add dummy0 type dummy
-	ip -netns testns link set dev dummy0 up
+	setup
 
+	set -e
 	ip -netns testns link add dummy1 type dummy
 	ip -netns testns link set dev dummy1 up
-
-	ip -netns testns address add 198.51.100.1/24 dev dummy0
-	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
-
 	ip -netns testns address add 192.0.2.1/24 dev dummy1
 	ip -netns testns -6 address add 2001:db8:2::1/64 dev dummy1
 
@@ -124,7 +122,7 @@ fib_unreg_multipath_test()
 	ip -netns testns -6 route get fibmatch 2001:db8:3::1 &> /dev/null
 	log_test $? 2 "IPv6 - no route"
 
-	ip netns del testns
+	cleanup
 }
 
 fib_unreg_test()
@@ -138,15 +136,7 @@ fib_down_unicast_test()
 	echo
 	echo "Single path, admin down"
 
-	set -e
-	netns_create "testns"
-
-	ip -netns testns link add dummy0 type dummy
-	ip -netns testns link set dev dummy0 up
-
-	ip -netns testns address add 198.51.100.1/24 dev dummy0
-	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
-	set +e
+	setup
 
 	echo "    Start point"
 	ip -netns testns route get fibmatch 198.51.100.2 &> /dev/null
@@ -164,9 +154,7 @@ fib_down_unicast_test()
 	ip -netns testns -6 route get fibmatch 2001:db8:1::2 &> /dev/null
 	log_test $? 2 "IPv6 fibmatch"
 
-	ip -netns testns link del dev dummy0
-
-	ip netns del testns
+	cleanup
 }
 
 fib_down_multipath_test_do()
@@ -208,18 +196,12 @@ fib_down_multipath_test()
 	echo
 	echo "Admin down multipath"
 
-	set -e
-	netns_create "testns"
-
-	ip -netns testns link add dummy0 type dummy
-	ip -netns testns link set dev dummy0 up
+	setup
 
+	set -e
 	ip -netns testns link add dummy1 type dummy
 	ip -netns testns link set dev dummy1 up
 
-	ip -netns testns address add 198.51.100.1/24 dev dummy0
-	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
-
 	ip -netns testns address add 192.0.2.1/24 dev dummy1
 	ip -netns testns -6 address add 2001:db8:2::1/64 dev dummy1
 
@@ -264,8 +246,7 @@ fib_down_multipath_test()
 	log_test $? 2 "IPv6 fibmatch"
 
 	ip -netns testns link del dev dummy1
-	ip -netns testns link del dev dummy0
-	ip netns del testns
+	cleanup
 }
 
 fib_down_test()
@@ -280,16 +261,10 @@ fib_carrier_local_test()
 	echo
 	echo "Local carrier tests - single path"
 
-	set -e
-	netns_create "testns"
-
-	ip -netns testns link add dummy0 type dummy
-	ip -netns testns link set dev dummy0 up
+	setup
 
+	set -e
 	ip -netns testns link set dev dummy0 carrier on
-
-	ip -netns testns address add 198.51.100.1/24 dev dummy0
-	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
 	set +e
 
 	echo "    Start point"
@@ -340,9 +315,7 @@ fib_carrier_local_test()
 		grep -q "linkdown"
 	log_test $? 1 "IPv6 linkdown flag set"
 
-	ip -netns testns link del dev dummy0
-
-	ip netns del testns
+	cleanup
 }
 
 fib_carrier_unicast_test()
@@ -352,16 +325,10 @@ fib_carrier_unicast_test()
 	echo
 	echo "Single path route carrier test"
 
-	netns_create "testns"
+	setup
 
 	set -e
-	ip -netns testns link add dummy0 type dummy
-	ip -netns testns link set dev dummy0 up
-
 	ip -netns testns link set dev dummy0 carrier on
-
-	ip -netns testns address add 198.51.100.1/24 dev dummy0
-	ip -netns testns -6 address add 2001:db8:1::1/64 dev dummy0
 	set +e
 
 	echo "    Start point"
@@ -412,9 +379,7 @@ fib_carrier_unicast_test()
 		grep -q "linkdown"
 	log_test $? 0 "IPv6 linkdown flag set"
 
-	ip -netns testns link del dev dummy0
-
-	ip netns del testns
+	cleanup
 }
 
 fib_carrier_test()
@@ -446,6 +411,9 @@ if [ $? -ne 0 ]; then
 	exit 0
 fi
 
+# start clean
+cleanup &> /dev/null
+
 fib_test
 
 exit $ret
-- 
2.11.0

[PATCH net-next 4/4] selftests: fib_tests: sleep after changing carrier

[David Ahern]

sleep for a second after setting carrier down to allow linkwatch
to propagate the change to the routing stack via netdev_state_change.
As it stands there is a race setting carrier down on the dummy
device and then checking the linkdown flag in the routes.

Signed-off-by: David Ahern <dsahern@gmail.com>
---
 tools/testing/selftests/net/fib_tests.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/testing/selftests/net/fib_tests.sh b/tools/testing/selftests/net/fib_tests.sh
index e113cfd659fc..b617985ecdc1 100755
--- a/tools/testing/selftests/net/fib_tests.sh
+++ b/tools/testing/selftests/net/fib_tests.sh
@@ -282,6 +282,7 @@ fib_carrier_local_test()
 
 	set -e
 	ip -netns testns link set dev dummy0 carrier off
+	sleep 1
 	set +e
 
 	echo "    Carrier off on nexthop"
-- 
2.11.0

Re: [PATCH net-next 0/4] selftests: fib_tests: simplifications, verbosity and a race

[David Miller]

From: David Ahern <dsahern@gmail.com>
Date: Tue, 13 Feb 2018 08:37:32 -0800

> Improve efficiency of fib_tests.sh and make the test result more verbose,
> from this summary:
> $ fib_tests.sh is failing in a VM:
>     $ fib_tests.sh
>     Running netdev unregister tests
>     PASS: unicast route test
>     PASS: multipath route test
>     Running netdev down tests
>     PASS: unicast route test
>     PASS: multipath route test
>     Running netdev carrier change tests
>     PASS: local route carrier test
>     FAIL: unicast route carrier test
> 
> where a single entry actually corresponds to many checks to a much more
> verbse output that clarifies test cases:
> $fib_tests.sh
> Single path route carrier test
>     ....
>     Carrier down
>         IPv4 fibmatch                                         [ OK ]
>         IPv6 fibmatch                                         [ OK ]
>         IPv4 linkdown flag set                                [FAIL]
>         IPv6 linkdown flag set                                [FAIL]
>     Second address added with carrier down
>         IPv4 fibmatch                                         [ OK ]
>         IPv6 fibmatch                                         [ OK ]
>         IPv4 linkdown flag set                                [FAIL]
>         IPv6 linkdown flag set                                [ OK ]
> 
> And then fix the race in changing carrier down on dummy device to checking
> the corresponding routes.

Looks great, series applied, thanks David.