Re: [PATCH nft v2 3/6] tests/shell: add JSON dump files

[Pablo Neira Ayuso <pablo@netfilter.org>]

On Mon, Nov 06, 2023 at 02:56:18PM +0100, Thomas Haller wrote:
> On Fri, 2023-11-03 at 19:26 +0100, Thomas Haller wrote:
> > Generate and add ".json-nft" files. These files contain the output of
> > `nft -j list ruleset` after the test. Also, "test-wrapper.sh" will
> > compare the current ruleset against the ".json-nft" files and test
> > them
> > with "nft -j --check -f $FILE`. These are useful extra tests, that we
> > almost get for free.
> > 
> > Note that for some JSON dumps, `nft -f --check` fails (or prints
> > something). For those tests no *.json-nft file is added. The bugs
> > needs
> > to be fixed first.
> > 
> > An example of such an issue is:
> > 
> >     $ DUMPGEN=all ./tests/shell/run-tests.sh
> > tests/shell/testcases/maps/nat_addr_port
> > 
> > which gives a file "rc-failed-chkdump" with
> > 
> >     Command `./tests/shell/../../src/nft -j --check -f
> > "tests/shell/testcases/maps/dumps/nat_addr_port.json-nft"` failed
> >     >>>>
> >     internal:0:0-0: Error: Invalid map type 'ipv4_addr .
> > inet_service'.
> > 
> >     internal:0:0-0: Error: Parsing command array at index 3 failed.
> > 
> >     internal:0:0-0: Error: unqualified type integer specified in map
> > definition. Try "typeof expression" instead of "type datatype".
> > 
> >     <<<<
> > 
> > Tests like "tests/shell/testcases/nft-f/0012different_defines_0" and
> > "tests/shell/testcases/nft-f/0024priority_0" also don't get a .json-
> > nft
> > dump yet, because their output is not stable. That needs fixing too.
> > 
> > Cc: Pablo Neira Ayuso <pablo@netfilter.org>
> > Cc: Florian Westphal <fw@strlen.de>
> > Signed-off-by: Thomas Haller <thaller@redhat.com>
> > ---
> ...
> >  tests/shell/testcases/sets/dumps/0062set_connlimit_0.json-nft    | 
> ...
> > tests/shell/testcases/sets/dumps/0062set_connlimit_0.json-nft
> >  create mode 100644
> 
> 
> "tests/shell/testcases/sets/dumps/0062set_connlimit_0.json-nft" need to
> be dropped from this patch.
> 
> Otherwise,
> 
>   make && ./tests/shell/run-tests.sh tests/shell/testcases/sets/0062set_connlimit_0 -V
> 
> fails (in valgrind mode).

I have to fix 0062set_connlimit_0, the listing fails because the GC is
fast enough to remove the entry that just got added, because it has no
conntrack entries.

Maybe valgrind is just getting things slowier there to trigger what
I can already reproduce here on a VM?