Re: [PATCH nft 1/5] tests: add feature probing

[Thomas Haller <thaller@redhat.com>]

On Mon, 2023-09-04 at 11:06 +0200, Florian Westphal wrote:
> Running selftests on older kernels makes some of them fail very early
> because some tests use features that are not available on older
> kernels, e.g. -stable releases.
> 
> Known examples:
> - inner header matching
> - anonymous chains
> - elem delete from packet path
> 
> Also, some test cases might fail because a feature isn't
> compiled in, such as netdev chains for example.
> 
> This adds a feature-probing to the shell tests.

On my Fedora 38, I have currently 7 tests failing.
With this patchset, tests/shell/testcases/maps/typeof_maps_add_delete
is now skipped (6 failing left).

Nice!


> +check_features()
> +{
> +       for ffilename in $TESTDIR/../features/*.nft; do
> +               feature=$(basename $ffilename)
> +
> +               feature=${feature#*/}
> +               feature=${feature%*.nft}
> +
> +               eval NFT_HAVE_${feature}=0
> +               $NFT --check -f "$ffilename" 2>/dev/null
> +               if [ $? -eq 0 ]; then
> +                       eval NFT_HAVE_${feature}=1

the existing variables like VERBOSE,VALGRIND use "y" for true and
everything else is false.

I think 0|1 looks better. But it should be consistent. If 0|1 is used,
the other variables should be adjusted.

Note that on the other branch I added normalization functions
bool_y()/bool_n() to accept values like true/1/y/yes from the user, and
normalize to y|n. This could be changed internally to 1|0 without
breaking user setups.


>  for testfile in $(find_tests)
> @@ -277,5 +296,10 @@ check_kmemleak_force
>  
>  msg_info "results: [OK] $ok [FAILED] $failed [TOTAL] $((ok+failed))"
>  
> +if [ "$VERBOSE" == "y" ] ; then
> +       echo "Probed Features:"
> +       env | grep NFT_HAVE_
> +fi


  xxx=$'\nNFT_HAVE_XXXXX=bogus' ./tests/shell/run-tests.sh /bin/true -v

gives the wrong output. Could be instead:

  for v in $(compgen -v | grep '^NFT_HAVE_') ; do
      echo "$v=${!v}";
  done