[PATCH] test: shell: run-test.sh: introduce NFT_TEST_EXCLUDES

[PATCH] test: shell: run-test.sh: introduce NFT_TEST_EXCLUDES

[Yi Chen]

Introduce the NFT_TEST_EXCLUDES environment variable to allow excluding
one or more specific test cases.
This is useful for some releases where certain tests are not yet supported.
allowing them to be skipped directly in the test script
without modifying the run-test.sh itself.

Signed-off-by: Yi Chen <yiche@redhat.com>
---
 tests/shell/run-tests.sh | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/tests/shell/run-tests.sh b/tests/shell/run-tests.sh
index 46f523b9..6273c9a2 100755
--- a/tests/shell/run-tests.sh
+++ b/tests/shell/run-tests.sh
@@ -255,6 +255,7 @@ usage() {
 	echo "                 Supported <FEATURE>s are: ${_HAVE_OPTS[@]}."
 	echo " NFT_TEST_SKIP_<OPTION>=*|y: if set, certain tests are skipped."
 	echo "                 Supported <OPTION>s are: ${_SKIP_OPTS[@]}."
+	echo " NFT_TEST_EXCLUDES=\"TEST1 TEST2 ...\": Specify tests to be excluded from running"
 }
 
 NFT_TEST_BASEDIR="$(dirname "$0")"
@@ -455,6 +456,11 @@ done
 
 NFT_TEST_SHUFFLE_TESTS="$(bool_y "$NFT_TEST_SHUFFLE_TESTS")"
 
+if [ -n "$NFT_TEST_EXCLUDES" ]; then
+	TESTS=( $(printf '%s\n' "${TESTS[@]}" |
+            grep -vE "$(printf '%s|' $NFT_TEST_EXCLUDES | sed 's/|$//')") )
+fi
+
 if [ "$DO_LIST_TESTS" = y ] ; then
 	printf '%s\n' "${TESTS[@]}"
 	exit 0
-- 
2.52.0

Re: [PATCH] test: shell: run-test.sh: introduce NFT_TEST_EXCLUDES

[Florian Westphal]

Yi Chen <yiche@redhat.com> wrote:
> Introduce the NFT_TEST_EXCLUDES environment variable to allow excluding
> one or more specific test cases.
> This is useful for some releases where certain tests are not yet supported.
> allowing them to be skipped directly in the test script
> without modifying the run-test.sh itself.

Wouldn't it make more sense to add a feature test for those?
Or is this about test cases that fail because of a real bug that
where the fix wasn't backported yet?

Re: [PATCH] test: shell: run-test.sh: introduce NFT_TEST_EXCLUDES

[Yi Chen]

Some patches may be considered too aggressive to backport to
downstream releases.

For example:
netfilter: nf_reject: don't reply to ICMP error messages

When this patch is missing,
tests/shell/testcases/packetpath/reject_loopback reports a failure.

In addition, introducing this exclude feature makes it easier for
downstream streams to run this test suite while excluding known SKIP
items.
This also helps to quickly detect newly introduced SKIP items, which
may indicate new bugs.
Therefore, I suggest adding this feature.

Apologies that one line in the patch uses spaces instead of tabs for
indentation.

On Wed, Feb 4, 2026 at 11:06 PM Florian Westphal <fw@strlen.de> wrote:
>
> Yi Chen <yiche@redhat.com> wrote:
> > Introduce the NFT_TEST_EXCLUDES environment variable to allow excluding
> > one or more specific test cases.
> > This is useful for some releases where certain tests are not yet supported.
> > allowing them to be skipped directly in the test script
> > without modifying the run-test.sh itself.
>
> Wouldn't it make more sense to add a feature test for those?
> Or is this about test cases that fail because of a real bug that
> where the fix wasn't backported yet?
>

Re: [PATCH] test: shell: run-test.sh: introduce NFT_TEST_EXCLUDES

[Florian Westphal]

Yi Chen <yiche@redhat.com> wrote:
> Some patches may be considered too aggressive to backport to
> downstream releases.
> 
> For example:
> netfilter: nf_reject: don't reply to ICMP error messages
>
> When this patch is missing,
> tests/shell/testcases/packetpath/reject_loopback reports a failure.

Right, that makes sense.

> In addition, introducing this exclude feature makes it easier for
> downstream streams to run this test suite while excluding known SKIP
> items.
> This also helps to quickly detect newly introduced SKIP items, which
> may indicate new bugs.

All good reasons, I'll place this in the commit message before applying.

Thanks!