netfilter-devel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [nft PATCH 00/28] Fix netlink debug output on Big Endian
@ 2025-10-23 16:13 Phil Sutter
  2025-10-23 16:13 ` [nft PATCH 01/28] datatype: Fix boolean type " Phil Sutter
                   ` (28 more replies)
  0 siblings, 29 replies; 53+ messages in thread
From: Phil Sutter @ 2025-10-23 16:13 UTC (permalink / raw)
  To: Pablo Neira Ayuso; +Cc: netfilter-devel

Make use of recent changes to libnftnl and make tests/py testsuite pass
on Big Endian systems.

Patches 1-7 fix existing code, are valid without the remaining ones but
required for the target at hand.

Patches 8-12 are a mixture of fixes and preparation for the remaining
ones.

Patches 13-15 change the defined byteorder for all string-based
expressions to BYTEORDER_BIG_ENDIAN. This avoids special casing when
communicating data reg byteorder to libnftnl as they should be printed
left to right, just like with Big Endian values.

Patches 16-21 contain the required changes correcting netlink debug
output.

Patches 22 and 23 change how concatenations are linearized before
comparison for set element sorting. Aside from unifying results between
Little and Big Endian, sorting order now seems "correct" to a human
reader.

Patches 24 and 25 Introduce a tool to refresh stored payload records in
py test suite and record the tool's results.

Patch 26 adds expr_print_debug() macro easing "printf debugging" around
expression use.

Patches 27 and 28 finally fix for BUG() messages missing a terminating
newline at times.

Phil Sutter (28):
  datatype: Fix boolean type on Big Endian
  optimize: Fix verdict expression comparison
  tests: py: any/tcpopt.t.json: Fix JSON equivalent
  tests: py: any/ct.t.json.output: Drop leftover entry
  tests: py: inet/osf.t: Fix element ordering in JSON equivalents
  tests: py: Fix for using wrong payload path
  tests: py: Implement payload_record()
  tests: py: Do not rely upon '[end]' marker
  netlink: No need to reference array when passing as pointer
  datatype: Increase symbolic constant printer robustness
  tests: py: ip6/vmap.t: Drop double whitespace in rule
  netlink: Zero nft_data_linearize objects when populating
  Define string-based data types as Big Endian
  segtree: No byteorder conversion for string prefix len calculation
  Fix byteorder conversion of concatenated value expressions and ranges
  expression: Set range expression 'len' field
  segtree: Export complete data before editing
  segtree: Drop problematic constant expr len adjustment
  netlink: Introduce struct nft_data_linearize::byteorder
  netlink: Introduce struct nft_data_linearize::sizes
  netlink: Make use of nftnl_{expr,set_elem}_set_imm()
  mergesort: Linearize concatentations in network byte order
  tests: Adjust JSON records to improved element sorting
  tests: py: tools: Add regen_payloads.sh
  tests: py: Update payload records
  utils: Introduce expr_print_debug()
  utils: Cover for missing newline after BUG() messages
  Drop no longer needed newline in BUG() messages

 include/netlink.h                             |   2 +
 include/utils.h                               |  11 +-
 src/ct.c                                      |   2 +-
 src/datatype.c                                |  28 +-
 src/erec.c                                    |   2 +-
 src/evaluate.c                                |  65 +--
 src/expression.c                              |   7 +-
 src/fib.c                                     |   7 +-
 src/intervals.c                               |  23 +-
 src/json.c                                    |   6 +-
 src/mergesort.c                               |   6 +-
 src/meta.c                                    |  16 +-
 src/mnl.c                                     |  10 +-
 src/netlink.c                                 |  84 +++-
 src/netlink_delinearize.c                     |  25 +-
 src/netlink_linearize.c                       |  62 +--
 src/optimize.c                                |  13 +-
 src/osf.c                                     |   3 +-
 src/parser_bison.y                            |  10 +-
 src/parser_json.c                             |   4 +-
 src/rule.c                                    |  20 +-
 src/segtree.c                                 |  25 +-
 src/statement.c                               |   2 +-
 tests/py/any/ct.t.json.output                 |  51 +-
 tests/py/any/ct.t.payload                     |  89 ++--
 tests/py/any/meta.t.json.output               |  54 ---
 tests/py/any/meta.t.payload                   | 232 ++++-----
 tests/py/any/meta.t.payload.bridge            |  10 +-
 tests/py/any/queue.t.json.output              |   4 +-
 tests/py/any/queue.t.payload                  |   2 +-
 tests/py/any/rawpayload.t.payload             |  64 +--
 tests/py/any/rt.t.payload                     |   5 +-
 tests/py/any/tcpopt.t.json                    |  24 +-
 tests/py/any/tcpopt.t.json.output             |   1 -
 tests/py/any/tcpopt.t.payload                 |  88 ++--
 tests/py/arp/arp.t.payload                    |  99 ++--
 tests/py/arp/arp.t.payload.netdev             | 189 ++++----
 tests/py/bridge/ether.t.payload               |  43 +-
 tests/py/bridge/icmpX.t.payload               |  25 +-
 tests/py/bridge/meta.t.payload                |  20 +-
 tests/py/bridge/redirect.t.payload            |   2 +-
 tests/py/bridge/reject.t.payload              |  43 +-
 tests/py/bridge/vlan.t.payload                | 262 +++++------
 tests/py/bridge/vlan.t.payload.netdev         | 316 ++++++-------
 tests/py/inet/ah.t.payload                    |  89 ++--
 tests/py/inet/comp.t.payload                  |  53 ++-
 tests/py/inet/ct.t.payload                    |  14 +-
 tests/py/inet/dccp.t.payload                  |  53 ++-
 tests/py/inet/dnat.t.payload                  |  47 +-
 tests/py/inet/esp.t.payload                   |  45 +-
 tests/py/inet/ether-ip.t.payload              |  22 +-
 tests/py/inet/ether-ip.t.payload.netdev       |  23 +-
 tests/py/inet/ether.t.payload                 |  37 +-
 tests/py/inet/ether.t.payload.bridge          |  29 +-
 tests/py/inet/ether.t.payload.ip              |  37 +-
 tests/py/inet/fib.t.payload                   |  12 +-
 tests/py/inet/geneve.t.payload                |  83 ++--
 tests/py/inet/gre.t.payload                   |  53 ++-
 tests/py/inet/gretap.t.payload                |  59 ++-
 tests/py/inet/icmp.t.payload                  |  37 +-
 tests/py/inet/icmpX.t.payload                 |  32 +-
 tests/py/inet/ip.t.payload                    |   4 +-
 tests/py/inet/ip.t.payload.bridge             |   4 +-
 tests/py/inet/ip.t.payload.inet               |   7 +-
 tests/py/inet/ip.t.payload.netdev             |   7 +-
 tests/py/inet/ip_tcp.t.payload                |  36 +-
 tests/py/inet/ip_tcp.t.payload.bridge         |  35 +-
 tests/py/inet/ip_tcp.t.payload.netdev         |  37 +-
 tests/py/inet/ipsec.t.payload                 |  11 +-
 tests/py/inet/map.t.payload                   |  11 +-
 tests/py/inet/map.t.payload.ip                |   7 +-
 tests/py/inet/map.t.payload.netdev            |  11 +-
 tests/py/inet/meta.t.payload                  |  83 ++--
 tests/py/inet/osf.t.json                      |  12 +-
 tests/py/inet/osf.t.json.output               | 107 +++++
 tests/py/inet/osf.t.payload                   |  16 +-
 tests/py/inet/payloadmerge.t.payload          |  43 +-
 tests/py/inet/reject.t.payload.inet           |  43 +-
 tests/py/inet/rt.t.payload                    |   7 +-
 tests/py/inet/sctp.t.payload                  | 159 ++++---
 tests/py/inet/sets.t.payload.bridge           |  15 +-
 tests/py/inet/sets.t.payload.inet             |  14 +-
 tests/py/inet/sets.t.payload.netdev           |  14 +-
 tests/py/inet/snat.t.payload                  |  30 +-
 tests/py/inet/socket.t.payload                |   8 +-
 tests/py/inet/tcp.t.payload                   | 360 +++++++-------
 tests/py/inet/tproxy.t.payload                |  45 +-
 tests/py/inet/udp.t.payload                   | 112 ++---
 tests/py/inet/udplite.t.payload               |  81 ++--
 tests/py/inet/vmap.t.payload                  |  13 +-
 tests/py/inet/vmap.t.payload.netdev           |  13 +-
 tests/py/inet/vxlan.t.payload                 |  83 ++--
 tests/py/ip/ct.t.payload                      |  46 +-
 tests/py/ip/dnat.t.payload.ip                 | 117 +++--
 tests/py/ip/dup.t.payload                     |   7 +-
 tests/py/ip/ether.t.payload                   |  37 +-
 tests/py/ip/hash.t.payload                    |   2 +-
 tests/py/ip/icmp.t.payload.ip                 | 312 ++++++-------
 tests/py/ip/igmp.t.payload                    |  61 ++-
 tests/py/ip/ip.t.payload                      | 246 +++++-----
 tests/py/ip/ip.t.payload.bridge               | 440 +++++++++---------
 tests/py/ip/ip.t.payload.inet                 | 440 +++++++++---------
 tests/py/ip/ip.t.payload.netdev               | 434 ++++++++---------
 tests/py/ip/ip_tcp.t.payload                  |   9 +-
 tests/py/ip/masquerade.t.payload              |  71 ++-
 tests/py/ip/meta.t.payload                    |  32 +-
 tests/py/ip/numgen.t.payload                  |   4 +-
 tests/py/ip/objects.t.payload                 |  26 +-
 tests/py/ip/redirect.t.payload                | 121 +++--
 tests/py/ip/reject.t.payload                  |   3 +-
 tests/py/ip/rt.t.payload                      |   3 +-
 tests/py/ip/sets.t.payload.inet               |  38 +-
 tests/py/ip/sets.t.payload.ip                 |  12 +-
 tests/py/ip/sets.t.payload.netdev             |  38 +-
 tests/py/ip/snat.t.payload                    |  74 +--
 tests/py/ip/tcp.t.payload                     |  11 +-
 tests/py/ip/tproxy.t.payload                  |  29 +-
 tests/py/ip6/ct.t.payload                     |  12 +-
 tests/py/ip6/dnat.t.payload.ip6               |  38 +-
 tests/py/ip6/dst.t.payload.inet               |  60 +--
 tests/py/ip6/dst.t.payload.ip6                |  30 +-
 tests/py/ip6/dup.t.payload                    |   7 +-
 tests/py/ip6/ether.t.payload                  |  36 +-
 tests/py/ip6/exthdr.t.payload.ip6             |  25 +-
 tests/py/ip6/frag.t.payload.inet              | 131 +++---
 tests/py/ip6/frag.t.payload.ip6               |  75 ++-
 tests/py/ip6/frag.t.payload.netdev            | 131 +++---
 tests/py/ip6/hbh.t.payload.inet               |  65 ++-
 tests/py/ip6/hbh.t.payload.ip6                |  33 +-
 tests/py/ip6/icmpv6.t.payload.ip6             | 322 ++++++-------
 tests/py/ip6/ip6.t.payload.inet               | 359 +++++++-------
 tests/py/ip6/ip6.t.payload.ip6                | 191 ++++----
 tests/py/ip6/map.t.payload                    |   5 +-
 tests/py/ip6/masquerade.t.payload.ip6         |  71 ++-
 tests/py/ip6/meta.t.payload                   |  36 +-
 tests/py/ip6/mh.t.payload.inet                | 132 +++---
 tests/py/ip6/mh.t.payload.ip6                 |  66 +--
 tests/py/ip6/redirect.t.payload.ip6           | 105 +++--
 tests/py/ip6/reject.t.payload.ip6             |   3 +-
 tests/py/ip6/rt.t.payload.inet                | 121 +++--
 tests/py/ip6/rt.t.payload.ip6                 |  61 ++-
 tests/py/ip6/rt0.t.payload                    |   3 +-
 tests/py/ip6/sets.t.payload.inet              |  18 +-
 tests/py/ip6/sets.t.payload.ip6               |   4 +-
 tests/py/ip6/sets.t.payload.netdev            |  18 +-
 tests/py/ip6/snat.t.payload.ip6               |  23 +-
 tests/py/ip6/srh.t.payload                    |  23 +-
 tests/py/ip6/tproxy.t.payload                 |  29 +-
 tests/py/ip6/vmap.t                           |   2 +-
 tests/py/ip6/vmap.t.json                      |   2 +-
 tests/py/ip6/vmap.t.payload.inet              | 171 ++++---
 tests/py/ip6/vmap.t.payload.ip6               |  87 ++--
 tests/py/ip6/vmap.t.payload.netdev            | 171 ++++---
 tests/py/netdev/dup.t.payload                 |   3 +-
 tests/py/netdev/fwd.t.payload                 |   5 +-
 tests/py/netdev/reject.t.payload              |  41 +-
 tests/py/netdev/tunnel.t.payload              |   5 +-
 tests/py/nft-test.py                          | 138 +++---
 tests/py/tools/regen_payloads.sh              |  74 +++
 .../testcases/maps/dumps/0012map_0.json-nft   |  20 +-
 .../shell/testcases/maps/dumps/0012map_0.nft  |   8 +-
 .../maps/dumps/named_ct_objects.json-nft      |  12 +-
 .../testcases/maps/dumps/named_ct_objects.nft |   6 +-
 .../maps/dumps/typeof_integer_0.json-nft      |  12 +-
 .../testcases/maps/dumps/typeof_integer_0.nft |   4 +-
 .../dumps/0012different_defines_0.json-nft    |   8 +-
 .../nft-f/dumps/0012different_defines_0.nft   |   2 +-
 .../dumps/merge_nat_inet.json-nft             |  12 +-
 .../optimizations/dumps/merge_nat_inet.nft    |   2 +-
 .../optimizations/dumps/merge_reject.json-nft |  16 +-
 .../optimizations/dumps/merge_reject.nft      |   4 +-
 .../dumps/merge_stmts_concat.json-nft         |  42 +-
 .../dumps/merge_stmts_concat.nft              |   8 +-
 .../dumps/merge_stmts_concat_vmap.json-nft    |   4 +-
 .../dumps/merge_stmts_concat_vmap.nft         |   2 +-
 .../sets/dumps/0029named_ifname_dtype_0.nft   |   4 +-
 .../0037_set_with_inet_service_0.json-nft     |  12 +-
 .../dumps/0037_set_with_inet_service_0.nft    |   8 +-
 .../sets/dumps/sets_with_ifnames.json-nft     |   4 +-
 .../sets/dumps/sets_with_ifnames.nft          |   2 +-
 .../testcases/sets/dumps/typeof_sets_0.nft    |   4 +-
 tests/shell/testcases/sets/typeof_sets_0      |   4 +-
 182 files changed, 5023 insertions(+), 4931 deletions(-)
 create mode 100644 tests/py/inet/osf.t.json.output
 create mode 100755 tests/py/tools/regen_payloads.sh

-- 
2.51.0


^ permalink raw reply	[flat|nested] 53+ messages in thread

end of thread, other threads:[~2025-10-30 23:22 UTC | newest]

Thread overview: 53+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-10-23 16:13 [nft PATCH 00/28] Fix netlink debug output on Big Endian Phil Sutter
2025-10-23 16:13 ` [nft PATCH 01/28] datatype: Fix boolean type " Phil Sutter
2025-10-23 16:13 ` [nft PATCH 02/28] optimize: Fix verdict expression comparison Phil Sutter
2025-10-23 16:13 ` [nft PATCH 03/28] tests: py: any/tcpopt.t.json: Fix JSON equivalent Phil Sutter
2025-10-23 16:13 ` [nft PATCH 04/28] tests: py: any/ct.t.json.output: Drop leftover entry Phil Sutter
2025-10-23 16:13 ` [nft PATCH 05/28] tests: py: inet/osf.t: Fix element ordering in JSON equivalents Phil Sutter
2025-10-23 16:13 ` [nft PATCH 06/28] tests: py: Fix for using wrong payload path Phil Sutter
2025-10-23 16:13 ` [nft PATCH 07/28] tests: py: Implement payload_record() Phil Sutter
2025-10-23 16:13 ` [nft PATCH 08/28] tests: py: Do not rely upon '[end]' marker Phil Sutter
2025-10-23 16:13 ` [nft PATCH 09/28] netlink: No need to reference array when passing as pointer Phil Sutter
2025-10-23 16:13 ` [nft PATCH 10/28] datatype: Increase symbolic constant printer robustness Phil Sutter
2025-10-29 18:36   ` Pablo Neira Ayuso
2025-10-30 11:00     ` Phil Sutter
2025-10-30 21:56       ` Pablo Neira Ayuso
2025-10-30 22:35         ` Phil Sutter
2025-10-30 23:22           ` Pablo Neira Ayuso
2025-10-23 16:14 ` [nft PATCH 11/28] tests: py: ip6/vmap.t: Drop double whitespace in rule Phil Sutter
2025-10-23 16:14 ` [nft PATCH 12/28] netlink: Zero nft_data_linearize objects when populating Phil Sutter
2025-10-29 18:37   ` Pablo Neira Ayuso
2025-10-30 11:08     ` Phil Sutter
2025-10-30 22:02       ` Pablo Neira Ayuso
2025-10-30 22:54         ` Phil Sutter
2025-10-23 16:14 ` [nft PATCH 13/28] Define string-based data types as Big Endian Phil Sutter
2025-10-29 18:22   ` Pablo Neira Ayuso
2025-10-30 10:20     ` Phil Sutter
2025-10-23 16:14 ` [nft PATCH 14/28] segtree: No byteorder conversion for string prefix len calculation Phil Sutter
2025-10-23 16:14 ` [nft PATCH 15/28] Fix byteorder conversion of concatenated value expressions and ranges Phil Sutter
2025-10-23 16:14 ` [nft PATCH 16/28] expression: Set range expression 'len' field Phil Sutter
2025-10-23 16:14 ` [nft PATCH 17/28] segtree: Export complete data before editing Phil Sutter
2025-10-23 16:14 ` [nft PATCH 18/28] segtree: Drop problematic constant expr len adjustment Phil Sutter
2025-10-23 16:14 ` [nft PATCH 19/28] netlink: Introduce struct nft_data_linearize::byteorder Phil Sutter
2025-10-23 16:14 ` [nft PATCH 20/28] netlink: Introduce struct nft_data_linearize::sizes Phil Sutter
2025-10-29 18:34   ` Pablo Neira Ayuso
2025-10-30 10:53     ` Phil Sutter
2025-10-23 16:14 ` [nft PATCH 21/28] netlink: Make use of nftnl_{expr,set_elem}_set_imm() Phil Sutter
2025-10-23 16:14 ` [nft PATCH 22/28] mergesort: Linearize concatentations in network byte order Phil Sutter
2025-10-29 18:27   ` Pablo Neira Ayuso
2025-10-30 10:47     ` Phil Sutter
2025-10-23 16:14 ` [nft PATCH 23/28] tests: Adjust JSON records to improved element sorting Phil Sutter
2025-10-23 16:14 ` [nft PATCH 24/28] tests: py: tools: Add regen_payloads.sh Phil Sutter
2025-10-23 16:14 ` [nft PATCH 25/28] tests: py: Update payload records Phil Sutter
2025-10-23 16:14 ` [nft PATCH 26/28] utils: Introduce expr_print_debug() Phil Sutter
2025-10-29 12:46   ` Florian Westphal
2025-10-29 18:31   ` Pablo Neira Ayuso
2025-10-30 10:16     ` Phil Sutter
2025-10-23 16:14 ` [nft PATCH 27/28] utils: Cover for missing newline after BUG() messages Phil Sutter
2025-10-29 12:47   ` Florian Westphal
2025-10-30 10:17     ` Phil Sutter
2025-10-23 16:14 ` [nft PATCH 28/28] Drop no longer needed newline in " Phil Sutter
2025-10-29 18:33   ` Pablo Neira Ayuso
2025-10-30 10:51     ` Phil Sutter
2025-10-23 20:45 ` [nft PATCH 00/28] Fix netlink debug output on Big Endian Florian Westphal
2025-10-23 20:47   ` Phil Sutter

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).