[nft PATCH] tests: shell: Fix for unstable sets/0043concatenated_ranges_0

[nft PATCH] tests: shell: Fix for unstable sets/0043concatenated_ranges_0

[Phil Sutter]

On my (slow?) testing VM, The test tends to fail when doing a full run
(i.e., calling run-test.sh without arguments) and tends to pass when run
individually.

The problem seems to be the 1s element timeout which in some cases may
pass before element deletion occurs. Simply fix this by doubling the
timeout. It has to pass just once, so shouldn't hurt too much.

Fixes: 618393c6b3f25 ("tests: Introduce test for set with concatenated ranges")
Cc: Stefano Brivio <sbrivio@redhat.com>
Signed-off-by: Phil Sutter <phil@nwl.cc>
---
 tests/shell/testcases/sets/0043concatenated_ranges_0 | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/shell/testcases/sets/0043concatenated_ranges_0 b/tests/shell/testcases/sets/0043concatenated_ranges_0
index 11767373bcd22..6e8f4000aa4ef 100755
--- a/tests/shell/testcases/sets/0043concatenated_ranges_0
+++ b/tests/shell/testcases/sets/0043concatenated_ranges_0
@@ -147,7 +147,7 @@ for ta in ${TYPES}; do
 			eval add_b=\$ADD_${tb}
 			eval add_c=\$ADD_${tc}
 			${NFT} add element inet filter test \
-				"{ ${add_a} . ${add_b} . ${add_c} timeout 1s${mapv}}"
+				"{ ${add_a} . ${add_b} . ${add_c} timeout 2s${mapv}}"
 			[ $(${NFT} list ${setmap} inet filter test |	\
 			   grep -c "${add_a} . ${add_b} . ${add_c}") -eq 1 ]
 
@@ -180,7 +180,7 @@ for ta in ${TYPES}; do
 				continue
 			fi
 
-			sleep 1
+			sleep 2
 			[ $(${NFT} list ${setmap} inet filter test |		\
 			   grep -c "${add_a} . ${add_b} . ${add_c} ${mapv}") -eq 0 ]
 			timeout_tested=1
-- 
2.40.0

Re: [nft PATCH] tests: shell: Fix for unstable sets/0043concatenated_ranges_0

[Stefano Brivio]

On Thu, 20 Apr 2023 17:47:23 +0200
Phil Sutter <phil@nwl.cc> wrote:

> On my (slow?) testing VM, The test tends to fail when doing a full run
> (i.e., calling run-test.sh without arguments) and tends to pass when run
> individually.
> 
> The problem seems to be the 1s element timeout which in some cases may
> pass before element deletion occurs.

Whoops. Yes I think so too.

> Simply fix this by doubling the
> timeout. It has to pass just once, so shouldn't hurt too much.
> 
> Fixes: 618393c6b3f25 ("tests: Introduce test for set with concatenated ranges")
> Cc: Stefano Brivio <sbrivio@redhat.com>
> Signed-off-by: Phil Sutter <phil@nwl.cc>

Reviewed-by: Stefano Brivio <sbrivio@redhat.com>

-- 
Stefano

Re: [nft PATCH] tests: shell: Fix for unstable sets/0043concatenated_ranges_0

[Pablo Neira Ayuso]

On Thu, Apr 20, 2023 at 05:47:23PM +0200, Phil Sutter wrote:
> On my (slow?) testing VM, The test tends to fail when doing a full run
> (i.e., calling run-test.sh without arguments) and tends to pass when run
> individually.
> 
> The problem seems to be the 1s element timeout which in some cases may
> pass before element deletion occurs. Simply fix this by doubling the
> timeout. It has to pass just once, so shouldn't hurt too much.

I have seen this with CONFIG_KASAN too, thanks Phil.

> Fixes: 618393c6b3f25 ("tests: Introduce test for set with concatenated ranges")
> Cc: Stefano Brivio <sbrivio@redhat.com>
> Signed-off-by: Phil Sutter <phil@nwl.cc>
> ---
>  tests/shell/testcases/sets/0043concatenated_ranges_0 | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/shell/testcases/sets/0043concatenated_ranges_0 b/tests/shell/testcases/sets/0043concatenated_ranges_0
> index 11767373bcd22..6e8f4000aa4ef 100755
> --- a/tests/shell/testcases/sets/0043concatenated_ranges_0
> +++ b/tests/shell/testcases/sets/0043concatenated_ranges_0
> @@ -147,7 +147,7 @@ for ta in ${TYPES}; do
>  			eval add_b=\$ADD_${tb}
>  			eval add_c=\$ADD_${tc}
>  			${NFT} add element inet filter test \
> -				"{ ${add_a} . ${add_b} . ${add_c} timeout 1s${mapv}}"
> +				"{ ${add_a} . ${add_b} . ${add_c} timeout 2s${mapv}}"
>  			[ $(${NFT} list ${setmap} inet filter test |	\
>  			   grep -c "${add_a} . ${add_b} . ${add_c}") -eq 1 ]
>  
> @@ -180,7 +180,7 @@ for ta in ${TYPES}; do
>  				continue
>  			fi
>  
> -			sleep 1
> +			sleep 2
>  			[ $(${NFT} list ${setmap} inet filter test |		\
>  			   grep -c "${add_a} . ${add_b} . ${add_c} ${mapv}") -eq 0 ]
>  			timeout_tested=1
> -- 
> 2.40.0
> 

Re: [nft PATCH] tests: shell: Fix for unstable sets/0043concatenated_ranges_0

[Phil Sutter]

On Thu, Apr 20, 2023 at 05:47:23PM +0200, Phil Sutter wrote:
> On my (slow?) testing VM, The test tends to fail when doing a full run
> (i.e., calling run-test.sh without arguments) and tends to pass when run
> individually.
> 
> The problem seems to be the 1s element timeout which in some cases may
> pass before element deletion occurs. Simply fix this by doubling the
> timeout. It has to pass just once, so shouldn't hurt too much.
> 
> Fixes: 618393c6b3f25 ("tests: Introduce test for set with concatenated ranges")
> Cc: Stefano Brivio <sbrivio@redhat.com>
> Signed-off-by: Phil Sutter <phil@nwl.cc>

Patch applied.