* [PATCH nft 1/1] tests: prettify JSON in test output and add helper @ 2023-11-22 11:19 Thomas Haller 2023-11-22 12:33 ` Pablo Neira Ayuso 0 siblings, 1 reply; 5+ messages in thread From: Thomas Haller @ 2023-11-22 11:19 UTC (permalink / raw) To: NetFilter; +Cc: Thomas Haller - add helper script "json-pretty.sh" for prettify/format JSON. It uses either `jq` or a `python` fallback. In my tests, they produce the same output, but the output is not guaranteed to be stable. This is mainly for informational purpose. - add a "json-diff-pretty.sh" which prettifies two JSON inputs and shows a diff of them. - in "test-wrapper.sh", after the check for a .json-nft dump fails, also call "json-diff-pretty.sh" and write the output to "ruleset-diff.json.pretty". This is beside "ruleset-diff.json", which contains the original diff. Signed-off-by: Thomas Haller <thaller@redhat.com> --- tests/shell/helpers/json-diff-pretty.sh | 17 +++++++++++++++++ tests/shell/helpers/json-pretty.sh | 17 +++++++++++++++++ tests/shell/helpers/test-wrapper.sh | 4 ++++ 3 files changed, 38 insertions(+) create mode 100755 tests/shell/helpers/json-diff-pretty.sh create mode 100755 tests/shell/helpers/json-pretty.sh diff --git a/tests/shell/helpers/json-diff-pretty.sh b/tests/shell/helpers/json-diff-pretty.sh new file mode 100755 index 000000000000..bebb7e8ed006 --- /dev/null +++ b/tests/shell/helpers/json-diff-pretty.sh @@ -0,0 +1,17 @@ +#!/bin/bash -e + +BASEDIR="$(dirname "$0")" + +[ $# -eq 2 ] || (echo "$0: expects two JSON files as arguments" ; exit 1) + +FILE1="$1" +FILE2="$2" + +pretty() +{ + "$BASEDIR/json-pretty.sh" < "$1" 2>&1 || : +} + +echo "Cmd: \"$0\" \"$FILE1\" \"$FILE2\"" +diff -u "$FILE1" "$FILE2" 2>&1 || : +diff -u <(pretty "$FILE1") <(pretty "$FILE2") 2>&1 || : diff --git a/tests/shell/helpers/json-pretty.sh b/tests/shell/helpers/json-pretty.sh new file mode 100755 index 000000000000..0d6972b81e2f --- /dev/null +++ b/tests/shell/helpers/json-pretty.sh @@ -0,0 +1,17 @@ +#!/bin/bash -e + +# WARNING: the output is not guaranteed to be stable. + +if command -v jq &>/dev/null ; then + # If we have, use `jq` + exec jq +fi + +# Fallback to python. +exec python -c ' +import json +import sys + +parsed = json.load(sys.stdin) +print(json.dumps(parsed, indent=2)) +' diff --git a/tests/shell/helpers/test-wrapper.sh b/tests/shell/helpers/test-wrapper.sh index 9e8e60581890..4ffc48184dd7 100755 --- a/tests/shell/helpers/test-wrapper.sh +++ b/tests/shell/helpers/test-wrapper.sh @@ -202,6 +202,10 @@ if [ "$rc_test" -ne 77 -a "$dump_written" != y ] ; then fi if [ "$NFT_TEST_HAVE_json" != n -a -f "$JDUMPFILE" ] ; then if ! $DIFF -u "$JDUMPFILE" "$NFT_TEST_TESTTMPDIR/ruleset-after.json" &> "$NFT_TEST_TESTTMPDIR/ruleset-diff.json" ; then + "$NFT_TEST_BASEDIR/helpers/json-diff-pretty.sh" \ + "$JDUMPFILE" \ + "$NFT_TEST_TESTTMPDIR/ruleset-after.json" \ + 2>&1 > "$NFT_TEST_TESTTMPDIR/ruleset-diff.json.pretty" show_file "$NFT_TEST_TESTTMPDIR/ruleset-diff.json" "Failed \`$DIFF -u \"$JDUMPFILE\" \"$NFT_TEST_TESTTMPDIR/ruleset-after.json\"\`" >> "$NFT_TEST_TESTTMPDIR/rc-failed-dump" rc_dump=1 else -- 2.42.0 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH nft 1/1] tests: prettify JSON in test output and add helper 2023-11-22 11:19 [PATCH nft 1/1] tests: prettify JSON in test output and add helper Thomas Haller @ 2023-11-22 12:33 ` Pablo Neira Ayuso 2023-11-22 12:55 ` Thomas Haller 0 siblings, 1 reply; 5+ messages in thread From: Pablo Neira Ayuso @ 2023-11-22 12:33 UTC (permalink / raw) To: Thomas Haller; +Cc: NetFilter Hi, On Wed, Nov 22, 2023 at 12:19:40PM +0100, Thomas Haller wrote: > - add helper script "json-pretty.sh" for prettify/format JSON. > It uses either `jq` or a `python` fallback. In my tests, they > produce the same output, but the output is not guaranteed to be > stable. This is mainly for informational purpose. > > - add a "json-diff-pretty.sh" which prettifies two JSON inputs and > shows a diff of them. > > - in "test-wrapper.sh", after the check for a .json-nft dump fails, also > call "json-diff-pretty.sh" and write the output to "ruleset-diff.json.pretty". > This is beside "ruleset-diff.json", which contains the original diff. One silly question: Does the prettify hightlights the difference? tests/py clearly shows what is the difference in the JSON diff that quickly helps you identify what is missing. Thanks! > Signed-off-by: Thomas Haller <thaller@redhat.com> > --- > tests/shell/helpers/json-diff-pretty.sh | 17 +++++++++++++++++ > tests/shell/helpers/json-pretty.sh | 17 +++++++++++++++++ > tests/shell/helpers/test-wrapper.sh | 4 ++++ > 3 files changed, 38 insertions(+) > create mode 100755 tests/shell/helpers/json-diff-pretty.sh > create mode 100755 tests/shell/helpers/json-pretty.sh > > diff --git a/tests/shell/helpers/json-diff-pretty.sh b/tests/shell/helpers/json-diff-pretty.sh > new file mode 100755 > index 000000000000..bebb7e8ed006 > --- /dev/null > +++ b/tests/shell/helpers/json-diff-pretty.sh > @@ -0,0 +1,17 @@ > +#!/bin/bash -e > + > +BASEDIR="$(dirname "$0")" > + > +[ $# -eq 2 ] || (echo "$0: expects two JSON files as arguments" ; exit 1) > + > +FILE1="$1" > +FILE2="$2" > + > +pretty() > +{ > + "$BASEDIR/json-pretty.sh" < "$1" 2>&1 || : > +} > + > +echo "Cmd: \"$0\" \"$FILE1\" \"$FILE2\"" > +diff -u "$FILE1" "$FILE2" 2>&1 || : > +diff -u <(pretty "$FILE1") <(pretty "$FILE2") 2>&1 || : > diff --git a/tests/shell/helpers/json-pretty.sh b/tests/shell/helpers/json-pretty.sh > new file mode 100755 > index 000000000000..0d6972b81e2f > --- /dev/null > +++ b/tests/shell/helpers/json-pretty.sh > @@ -0,0 +1,17 @@ > +#!/bin/bash -e > + > +# WARNING: the output is not guaranteed to be stable. > + > +if command -v jq &>/dev/null ; then > + # If we have, use `jq` > + exec jq > +fi > + > +# Fallback to python. > +exec python -c ' > +import json > +import sys > + > +parsed = json.load(sys.stdin) > +print(json.dumps(parsed, indent=2)) > +' > diff --git a/tests/shell/helpers/test-wrapper.sh b/tests/shell/helpers/test-wrapper.sh > index 9e8e60581890..4ffc48184dd7 100755 > --- a/tests/shell/helpers/test-wrapper.sh > +++ b/tests/shell/helpers/test-wrapper.sh > @@ -202,6 +202,10 @@ if [ "$rc_test" -ne 77 -a "$dump_written" != y ] ; then > fi > if [ "$NFT_TEST_HAVE_json" != n -a -f "$JDUMPFILE" ] ; then > if ! $DIFF -u "$JDUMPFILE" "$NFT_TEST_TESTTMPDIR/ruleset-after.json" &> "$NFT_TEST_TESTTMPDIR/ruleset-diff.json" ; then > + "$NFT_TEST_BASEDIR/helpers/json-diff-pretty.sh" \ > + "$JDUMPFILE" \ > + "$NFT_TEST_TESTTMPDIR/ruleset-after.json" \ > + 2>&1 > "$NFT_TEST_TESTTMPDIR/ruleset-diff.json.pretty" > show_file "$NFT_TEST_TESTTMPDIR/ruleset-diff.json" "Failed \`$DIFF -u \"$JDUMPFILE\" \"$NFT_TEST_TESTTMPDIR/ruleset-after.json\"\`" >> "$NFT_TEST_TESTTMPDIR/rc-failed-dump" > rc_dump=1 > else > -- > 2.42.0 > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH nft 1/1] tests: prettify JSON in test output and add helper 2023-11-22 12:33 ` Pablo Neira Ayuso @ 2023-11-22 12:55 ` Thomas Haller 2023-11-22 17:36 ` Pablo Neira Ayuso 0 siblings, 1 reply; 5+ messages in thread From: Thomas Haller @ 2023-11-22 12:55 UTC (permalink / raw) To: Pablo Neira Ayuso; +Cc: NetFilter On Wed, 2023-11-22 at 13:33 +0100, Pablo Neira Ayuso wrote: > Hi, > > On Wed, Nov 22, 2023 at 12:19:40PM +0100, Thomas Haller wrote: > > - add helper script "json-pretty.sh" for prettify/format JSON. > > It uses either `jq` or a `python` fallback. In my tests, they > > produce the same output, but the output is not guaranteed to be > > stable. This is mainly for informational purpose. > > > > - add a "json-diff-pretty.sh" which prettifies two JSON inputs and > > shows a diff of them. > > > > - in "test-wrapper.sh", after the check for a .json-nft dump fails, > > also > > call "json-diff-pretty.sh" and write the output to "ruleset- > > diff.json.pretty". > > This is beside "ruleset-diff.json", which contains the original > > diff. > > One silly question: Does the prettify hightlights the difference? Yes. That that is the idea. > > tests/py clearly shows what is the difference in the JSON diff that > quickly helps you identify what is missing. As always, you will find some result files in /tmp/nft- test.latest.$USER/, which I usually read with $ grep --color=always ^ -aR /tmp/nft-test.latest.*/ | less -R there will be a new file there, named "ruleset-diff.json.pretty". It contains the content of "./tests/shell/helpers/json-diff-pretty.sh" output. For example: $ cp tests/shell/testcases/bitwise/dumps/0040mark_binop_2.json-nft tests/shell/testcases/bitwise/dumps/0040mark_binop_3.json-nft $ ./tests/shell/run-tests.sh tests/shell/testcases/bitwise/0040mark_binop_3 leaves a file /tmp/nft-test.latest.*/test-tests-shell-testcases-bitwise-0040mark_binop_3.1/ruleset-diff.json.pretty with the following content: Cmd: "./tests/shell/helpers/json-diff-pretty.sh" "tests/shell/testcases/bitwise/dumps/0040mark_binop_3.json-nft" "/tmp/nft-test.20231122-135045.550.28uKHu/test-tests-shell-testcases-bitwise-0040mark_binop_3.1/ruleset-after.json" --- tests/shell/testcases/bitwise/dumps/0040mark_binop_3.json-nft 2023-11-22 13:50:12.114098356 +0100 +++ /tmp/nft-test.20231122-135045.550.28uKHu/test-tests-shell-testcases-bitwise-0040mark_binop_3.1/ruleset-after.json 2023-11-22 13:50:45.622065923 +0100 @@ -1 +1 @@ -{"nftables": [{"metainfo": {"version": "VERSION", "release_name": "RELEASE_NAME", "json_schema_version": 1}}, {"table": {"family": "ip", "name": "t", "handle": 0}}, {"chain": {"family": "ip", "table": "t", "name": "c", "handle": 0, "type": "filter", "hook": "output", "prio": 0, "policy": "accept"}}, {"rule": {"family": "ip", "table": "t", "chain": "c", "handle": 0, "expr": [{"mangle": {"key": {"ct": {"key": "mark"}}, "value": {"|": [{"<<": [{"payload": {"protocol": "ip", "field": "dscp"}}, 2]}, 16]}}}]}}]} +{"nftables": [{"metainfo": {"version": "VERSION", "release_name": "RELEASE_NAME", "json_schema_version": 1}}, {"table": {"family": "ip", "name": "t", "handle": 0}}, {"chain": {"family": "ip", "table": "t", "name": "c", "handle": 0, "type": "filter", "hook": "input", "prio": 0, "policy": "accept"}}, {"rule": {"family": "ip", "table": "t", "chain": "c", "handle": 0, "expr": [{"mangle": {"key": {"meta": {"key": "mark"}}, "value": {"|": [{"<<": [{"payload": {"protocol": "ip", "field": "dscp"}}, 2]}, 16]}}}]}}]} --- /dev/fd/63 2023-11-22 13:50:45.627065918 +0100 +++ /dev/fd/62 2023-11-22 13:50:45.627065918 +0100 @@ -21,7 +21,7 @@ "name": "c", "handle": 0, "type": "filter", - "hook": "output", + "hook": "input", "prio": 0, "policy": "accept" } @@ -36,7 +36,7 @@ { "mangle": { "key": { - "ct": { + "meta": { "key": "mark" } }, ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH nft 1/1] tests: prettify JSON in test output and add helper 2023-11-22 12:55 ` Thomas Haller @ 2023-11-22 17:36 ` Pablo Neira Ayuso 2023-11-22 18:23 ` Thomas Haller 0 siblings, 1 reply; 5+ messages in thread From: Pablo Neira Ayuso @ 2023-11-22 17:36 UTC (permalink / raw) To: Thomas Haller; +Cc: NetFilter On Wed, Nov 22, 2023 at 01:55:57PM +0100, Thomas Haller wrote: [...] > As always, you will find some result files in /tmp/nft- > test.latest.$USER/, which I usually read with > > $ grep --color=always ^ -aR /tmp/nft-test.latest.*/ | less -R > > there will be a new file there, named "ruleset-diff.json.pretty". > > It contains the content of "./tests/shell/helpers/json-diff-pretty.sh" > output. > > > For example: > > $ cp tests/shell/testcases/bitwise/dumps/0040mark_binop_2.json-nft tests/shell/testcases/bitwise/dumps/0040mark_binop_3.json-nft > $ ./tests/shell/run-tests.sh tests/shell/testcases/bitwise/0040mark_binop_3 > > > leaves a file > > /tmp/nft-test.latest.*/test-tests-shell-testcases-bitwise-0040mark_binop_3.1/ruleset-diff.json.pretty Thanks for explaining, this is now applied. And please, add copy and paste this info to the README file (unless I am overlooking anything, it is not yet there). ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH nft 1/1] tests: prettify JSON in test output and add helper 2023-11-22 17:36 ` Pablo Neira Ayuso @ 2023-11-22 18:23 ` Thomas Haller 0 siblings, 0 replies; 5+ messages in thread From: Thomas Haller @ 2023-11-22 18:23 UTC (permalink / raw) To: Pablo Neira Ayuso; +Cc: NetFilter On Wed, 2023-11-22 at 18:36 +0100, Pablo Neira Ayuso wrote: > Thanks for explaining, this is now applied. Thank you. > > And please, add copy and paste this info to the README file (unless I > am overlooking anything, it is not yet there). I hear you. It will still happen!! Thomas ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-11-22 18:23 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2023-11-22 11:19 [PATCH nft 1/1] tests: prettify JSON in test output and add helper Thomas Haller 2023-11-22 12:33 ` Pablo Neira Ayuso 2023-11-22 12:55 ` Thomas Haller 2023-11-22 17:36 ` Pablo Neira Ayuso 2023-11-22 18:23 ` Thomas Haller
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).