Re: [PATCH nft 1/1] tests/shell: sanitize "handle" in JSON output

[Phil Sutter <phil@nwl.cc>]

On Tue, Nov 21, 2023 at 01:10:11PM +0100, Thomas Haller wrote:
> On Sat, 2023-11-18 at 03:36 +0100, Phil Sutter wrote:
[...]
> > Also, scoping these replacements to line 1 is funny with single line
> > input. Worse is identifying the change in the resulting diff. Maybe
> > write a helper in python which lets you more comfortably sanitize
> > input,
> > sort attributes by key and output pretty-printed?
> 
> You mean, to parse and re-encode the JSON? That introduces additional
> changes, which seems undesirable. That's why the regex is limited to
> the first line (even if we only expect to ever see one line there).
> 
> Also, normalization via 2 regex seems simpler than writing some python.
> 
> Well, pretty-printing the output with `jq` would have the advantage,
> that future diffs might be smaller (changing individual lines, vs.
> replace one large line). Still, I think it's better to keep the amount
> of post-processing minimal.

The testsuite relies upon Python and respective modules already, using
jq introduces a new dependency. Hence why I suggested to write a script.

JSON object attributes are not bound to any ordering, the code may
change it.

When analyzing testsuite failures, a diff of two overlong lines is
inconvenient to the point that one may pipe both through json_pp and
then diff again. The testsuite may do just that in case of offending
output, but the problem of reordered attributes remains.

I'd really appreciate if testsuite changes prioritized usability. I
rather focus on fixing bugs instead of parsing the testsuite results.

Cheers, Phil