Re: [PATCH v2 nft] parser: tcpopt: fix tcp option parsing with NUM + length field

[Thomas Haller <thaller@redhat.com>]

On Wed, 2023-12-06 at 15:19 +0100, Phil Sutter wrote:
> On Wed, Dec 06, 2023 at 01:04:47PM +0100, Florian Westphal wrote:
> > Florian Westphal <fw@strlen.de> wrote:
> > > Thomas Haller <thaller@redhat.com> wrote:
> > > > On Wed, 2023-12-06 at 12:38 +0100, Florian Westphal wrote:
> > > > > Thomas Haller <thaller@redhat.com> wrote:
> > > > > > Hi Florian,
> > > > > > 
> > > > > > On Tue, 2023-12-05 at 12:56 +0100, Florian Westphal wrote:
> > > > > > >  .../packetpath/dumps/tcp_options.nft          | 14
> > > > > > > +++++++
> > > > > > 
> > > > > > is there a reason not to also generate a .json-nft file?
> > > > > 
> > > > > Yes, I am not adding more one-line monsters.
> > > > > 
> > > > > I'll add one once there is a solution in place that has human
> > > > > readable
> > > > > json dumps that don't fail validation because of identical
> > > > > but
> > > > > differently formatted output.
> > > > > 
> > > > 
> > > > What about the "[PATCH nft 0/2] pretty print .json-nft files"
> > > > patches?
> > > 
> > > I'm fine with that. Phil? Pablo? This is re:
> > > 
> > > https://patchwork.ozlabs.org/project/netfilter-devel/patch/20231124124759.3269219-3-thaller@redhat.com/
> 
> What I don't like is that we'll still get these huge patches/mails if
> the dumps are converted. Those that remain are still hard to handle
> in
> case of errors.

We can of course do a one-time commit to convert all .json-nft to
multi-line. The patch makes an effort to not require re-generating the
existing files unless necessary.

The only question is which one is preferable.

Regenerating all .json-nft files once, also makes the second patch
slightly simpler (but not so much, that it would be an argument for
doing that).

> 
> > What about making it so we NEVER compare json-nft at all?
> > 
> > Instead, feed the json-nft file to nft, then do a normal list-
> > ruleset,
> > then compare that vs. normal .nft file.
> > 
> > This avoids any and all formatting issues and also avoids breakage
> > when
> > the json-nft file is formatted differently.
> 
> We may hide problems because nft might inadvertently sanitize the
> input.
> Also, conversion from standard syntax to JSON may be symmetrically
> broken, so standard->JSON->standard won't detect the problem.
> 
> > Eg. postprocessing via json_pp won't match what this patch above
> > expects.
> 
> Python natively supports JSON. Converting stuff into comparable
> strings
> (which also look pretty when printed) is a simple matter of:
> 
> > import json
> > 
> > json.dumps(json.loads(<dump as string>), \
> > 	   sort_keys = True, indent = 4, \
> > 	   separators = (',', ': '))
> 
> We rely upon Python for the testsuite already, so I don't see why
> there's all the fuss. JSON dump create, load and compare have not
> been a
> problem in the 5 years tests/py does it.

The current patches intentionally don't try to sort keys. I claim, it's
also not necessary to sort them. I can easily be convinced otherwise,
by showing a counter-example/reproducer.

There is no fuzz, aside you and Florian bringing this topic up.


Thomas