Re: [PATCH nft v3 1/1] tests/shell: sanitize "handle" in JSON output

[Thomas Haller <thaller@redhat.com>]

On Wed, 2023-11-22 at 12:22 +0100, Pablo Neira Ayuso wrote:
> On Wed, Nov 22, 2023 at 11:44:54AM +0100, Thomas Haller wrote:
> > On Wed, 2023-11-22 at 11:36 +0100, Pablo Neira Ayuso wrote:
> > > On Tue, Nov 21, 2023 at 02:22:54PM +0100, Thomas Haller wrote:
> > > > The "handle" in JSON output is not stable. Sanitize/normalize
> > > > to
> > > > zero.
> > > > 
> > > > Adjust the sanitize code, and regenerate the .json-nft files.
> > > 
> > > Applied, thanks.
> > > 
> > > I had to adjust a json dump, this diff is not so difficult:
> > 
> > Hi,
> > 
> > Hm. The json dump of the patch was generated.
> > 
> > If you had to "adjust" a dump, does that mean that the output is
> > not
> > stable?
> 
> I had to adjust json output after my recent series for 4.19 -stable
> kernels:
> 
> https://patchwork.ozlabs.org/project/netfilter-devel/list/?series=383354
> 
> (note I splitted a few tests there)
> 
> with your patch output looks stable now here after this patch with
> different kernel versions, so all good, thanks!
> 
> > In that case, the .json-nft file should be removed instead (and the
> > cause for the difference investigated, fixed, and the dump-re-
> > added).
> 
> Agreed, but this different case as explained above.

ACK. Thanks.



> BTW, I am intentionally missing .json-nft files in my series because
> I
> am focusing on making progress on 4.19 backports, if you can help me
> with with missing .json-nft that I am living on my way, I'd
> appreciate.
> I promise to make a more careful look on missing .json-nft in the
> next
> series.

Note that on `master` there are

  - 386 shell tests
  - 14 tests with .nodump files
  - 372 tests with .nft dump files (386-14)
  - 339 tests with .json-nft files
  - 33 tests that lack .json-nft files (372-339)

Meaning, that they are IMO useful already, and there is no immediate
hurry to address the missing 33 files.


Thomas




PS: Reminder: you can identify missing .json-nft files by running
`./tools/check-tree.sh`.

You can generate missing files either via

  $ DUMPGEN=all ./tests/shell/run-tests.sh tests/shell/testcases/cache/0010_implicit_chain_0

or just

  $ touch tests/shell/testcases/cache/dumps/0010_implicit_chain_0.json-nft
  $ ./tests/shell/run-tests.sh tests/shell/testcases/cache/0010_implicit_chain_0 -g