[PATCH net-next v3 00/13] tools: ynl: prepare for wireguard

[PATCH net-next v3 00/13] tools: ynl: prepare for wireguard

[Asbjørn Sloth Tønnesen]

This series contains the last batch of YNL changes to support
the wireguard YNL conversion.

The wireguard changes, to be applied on top of this series,
has been posted as an RFC series here:
  https://lore.kernel.org/netdev/20250904-wg-ynl-rfc@fiberby.net/

---
v3:
- Rebased on top of new net-next, after Matthieu's cleanup.
- Added a Reviewed-by (thanks Donald).
- Added the parsing local vars cleanup as patch 7
- In patch 4, change to use set() for deduplication.
- In patch 8, declare __ynl_attr_validate() as static.
v2: https://lore.kernel.org/netdev/20250910230841.384545-1-ast@fiberby.net/
- Added Reviewed-by's to unchanged patches. Thanks to all reviewers.
- Patch 4, refactors local variables for .attr_put() callers, and
  replaces the old patch 4 and 5.
- Patch 5 and 6 are new, and reduces the differences between the 3
  .attr_put() callers, so it might be easier to keep them in sync.
- Patch 7, now validates the nested payload (thanks Jakub).
- Patch 8, now renames more variables (thanks Jakub),
- Patch 10, got a dead line removed (thanks Donald).
- Patch 11, revised hex input to support macsec (suggested by Sabrina).
v1: https://lore.kernel.org/netdev/20250904-wg-ynl-prep@fiberby.net/

Asbjørn Sloth Tønnesen (13):
  tools: ynl-gen: allow overriding name-prefix for constants
  tools: ynl-gen: generate nested array policies
  tools: ynl-gen: add sub-type check
  tools: ynl-gen: refactor local vars for .attr_put() callers
  tools: ynl-gen: add CodeWriter.p_lines() helper
  tools: ynl-gen: deduplicate fixed_header handling
  tools: ynl-gen: avoid repetitive variables definitions
  tools: ynl-gen: only validate nested array payload
  tools: ynl-gen: rename TypeArrayNest to TypeIndexedArray
  tools: ynl: move nest packing to a helper function
  tools: ynl: encode indexed-arrays
  tools: ynl: decode hex input
  tools: ynl: add ipv4-or-v6 display hint

 Documentation/netlink/genetlink-legacy.yaml |   2 +-
 tools/net/ynl/lib/ynl-priv.h                |   2 +
 tools/net/ynl/lib/ynl.c                     |  17 +-
 tools/net/ynl/pyynl/lib/ynl.py              |  38 ++++-
 tools/net/ynl/pyynl/ynl_gen_c.py            | 164 +++++++++++---------
 5 files changed, 140 insertions(+), 83 deletions(-)

-- 
2.51.0

[PATCH net-next v3 01/13] tools: ynl-gen: allow overriding name-prefix for constants

[Asbjørn Sloth Tønnesen]

Allow using custom name-prefix with constants,
just like it is for enum and flags declarations.

This is needed for generating WG_KEY_LEN in
include/uapi/linux/wireguard.h from a spec.

Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
---
 tools/net/ynl/pyynl/ynl_gen_c.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/net/ynl/pyynl/ynl_gen_c.py b/tools/net/ynl/pyynl/ynl_gen_c.py
index 101d8ba9626f..c8b15569ecc1 100755
--- a/tools/net/ynl/pyynl/ynl_gen_c.py
+++ b/tools/net/ynl/pyynl/ynl_gen_c.py
@@ -3208,8 +3208,9 @@ def render_uapi(family, cw):
             cw.block_end(line=';')
             cw.nl()
         elif const['type'] == 'const':
+            name_pfx = const.get('name-prefix', f"{family.ident_name}-")
             defines.append([c_upper(family.get('c-define-name',
-                                               f"{family.ident_name}-{const['name']}")),
+                                               f"{name_pfx}{const['name']}")),
                             const['value']])
 
     if defines:
-- 
2.51.0

[PATCH net-next v3 02/13] tools: ynl-gen: generate nested array policies

[Asbjørn Sloth Tønnesen]

This patch adds support for NLA_POLICY_NESTED_ARRAY() policies.

Example spec (from future wireguard.yaml):
-
  name: wgpeer
  attributes:
    -
      name: allowedips
      type: indexed-array
      sub-type: nest
      nested-attributes: wgallowedip

yields NLA_POLICY_NESTED_ARRAY(wireguard_wgallowedip_nl_policy).

This doesn't change any currently generated code, as it isn't
used in any specs currently used for generating code.

Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
---
 tools/net/ynl/pyynl/ynl_gen_c.py | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tools/net/ynl/pyynl/ynl_gen_c.py b/tools/net/ynl/pyynl/ynl_gen_c.py
index c8b15569ecc1..95a60fdaf14e 100755
--- a/tools/net/ynl/pyynl/ynl_gen_c.py
+++ b/tools/net/ynl/pyynl/ynl_gen_c.py
@@ -815,6 +815,11 @@ class TypeArrayNest(Type):
                     f'unsigned int n_{self.c_name}']
         return super().arg_member(ri)
 
+    def _attr_policy(self, policy):
+        if self.attr['sub-type'] == 'nest':
+            return f'NLA_POLICY_NESTED_ARRAY({self.nested_render_name}_nl_policy)'
+        return super()._attr_policy(policy)
+
     def _attr_typol(self):
         if self.attr['sub-type'] in scalars:
             return f'.type = YNL_PT_U{c_upper(self.sub_type[1:])}, '
-- 
2.51.0

[PATCH net-next v3 03/13] tools: ynl-gen: add sub-type check

[Asbjørn Sloth Tønnesen]

Add a check to verify that the sub-type is "nest", and throw an
exception if no policy could be generated, as a guard to prevent
against generating a bad policy.

This is a trivial patch with no behavioural changes intended.

Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
---
 tools/net/ynl/pyynl/ynl_gen_c.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tools/net/ynl/pyynl/ynl_gen_c.py b/tools/net/ynl/pyynl/ynl_gen_c.py
index 95a60fdaf14e..3266af19edcd 100755
--- a/tools/net/ynl/pyynl/ynl_gen_c.py
+++ b/tools/net/ynl/pyynl/ynl_gen_c.py
@@ -825,8 +825,10 @@ class TypeArrayNest(Type):
             return f'.type = YNL_PT_U{c_upper(self.sub_type[1:])}, '
         elif self.attr['sub-type'] == 'binary' and 'exact-len' in self.checks:
             return f'.type = YNL_PT_BINARY, .len = {self.checks["exact-len"]}, '
-        else:
+        elif self.attr['sub-type'] == 'nest':
             return f'.type = YNL_PT_NEST, .nest = &{self.nested_render_name}_nest, '
+        else:
+            raise Exception(f"Typol for ArrayNest sub-type {self.attr['sub-type']} not supported, yet")
 
     def _attr_get(self, ri, var):
         local_vars = ['const struct nlattr *attr2;']
-- 
2.51.0

[PATCH net-next v3 04/13] tools: ynl-gen: refactor local vars for .attr_put() callers

[Asbjørn Sloth Tønnesen]

Refactor the generation of local variables needed when building
requests, by moving the logic into the Type classes, and use the
same helper in all places where .attr_put() is called.

If any attributes requests identical local_vars, then they will
be deduplicated, attributes are assumed to only use their local
variables transiently.

This patch fixes the build errors below:
$ make -C tools/net/ynl/generated/
[...]
-e      GEN wireguard-user.c
-e      GEN wireguard-user.h
-e      CC wireguard-user.o
wireguard-user.c: In function ‘wireguard_get_device_dump’:
wireguard-user.c:480:9: error: ‘array’ undeclared (first use in func)
  480 |         array = ynl_attr_nest_start(nlh, WGDEVICE_A_PEERS);
      |         ^~~~~
wireguard-user.c:480:9: note: each undeclared identifier is reported
                        only once for each function it appears in
wireguard-user.c:481:14: error: ‘i’ undeclared (first use in func)
  481 |         for (i = 0; i < req->_count.peers; i++)
      |              ^
wireguard-user.c: In function ‘wireguard_set_device’:
wireguard-user.c:533:9: error: ‘array’ undeclared (first use in func)
  533 |         array = ynl_attr_nest_start(nlh, WGDEVICE_A_PEERS);
      |         ^~~~~
make: *** [Makefile:52: wireguard-user.o] Error 1
make: Leaving directory '/usr/src/linux/tools/net/ynl/generated'

Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
---
 tools/net/ynl/pyynl/ynl_gen_c.py | 35 ++++++++++++++++++++------------
 1 file changed, 22 insertions(+), 13 deletions(-)

diff --git a/tools/net/ynl/pyynl/ynl_gen_c.py b/tools/net/ynl/pyynl/ynl_gen_c.py
index 3266af19edcd..e4cb8c95632c 100755
--- a/tools/net/ynl/pyynl/ynl_gen_c.py
+++ b/tools/net/ynl/pyynl/ynl_gen_c.py
@@ -235,6 +235,12 @@ class Type(SpecAttr):
         line = f"ynl_attr_put_{put_type}(nlh, {self.enum_name}, {var}->{self.c_name})"
         self._attr_put_line(ri, var, line)
 
+    def attr_put_local_vars(self):
+        local_vars = []
+        if self.presence_type() == 'count':
+            local_vars.append('unsigned int i;')
+        return local_vars
+
     def attr_put(self, ri, var):
         raise Exception(f"Put not implemented for class type {self.type}")
 
@@ -840,6 +846,10 @@ class TypeArrayNest(Type):
                      '}']
         return get_lines, None, local_vars
 
+    def attr_put_local_vars(self):
+        local_vars = ['struct nlattr *array;']
+        return local_vars + super().attr_put_local_vars()
+
     def attr_put(self, ri, var):
         ri.cw.p(f'array = ynl_attr_nest_start(nlh, {self.enum_name});')
         if self.sub_type in scalars:
@@ -2040,6 +2050,13 @@ def put_enum_to_str(family, cw, enum):
     _put_enum_to_str_helper(cw, enum.render_name, map_name, 'value', enum=enum)
 
 
+def put_local_vars(struct):
+    local_vars = set()
+    for _, attr in struct.member_list():
+        local_vars |= set(attr.attr_put_local_vars())
+    return list(local_vars)
+
+
 def put_req_nested_prototype(ri, struct, suffix=';'):
     func_args = ['struct nlmsghdr *nlh',
                  'unsigned int attr_type',
@@ -2062,15 +2079,7 @@ def put_req_nested(ri, struct):
         init_lines.append(f"hdr = ynl_nlmsg_put_extra_header(nlh, {struct_sz});")
         init_lines.append(f"memcpy(hdr, &obj->_hdr, {struct_sz});")
 
-    has_anest = False
-    has_count = False
-    for _, arg in struct.member_list():
-        has_anest |= arg.type == 'indexed-array'
-        has_count |= arg.presence_type() == 'count'
-    if has_anest:
-        local_vars.append('struct nlattr *array;')
-    if has_count:
-        local_vars.append('unsigned int i;')
+    local_vars += put_local_vars(struct)
 
     put_req_nested_prototype(ri, struct, suffix='')
     ri.cw.block_start()
@@ -2354,10 +2363,7 @@ def print_req(ri):
         local_vars += ['size_t hdr_len;',
                        'void *hdr;']
 
-    for _, attr in ri.struct["request"].member_list():
-        if attr.presence_type() == 'count':
-            local_vars += ['unsigned int i;']
-            break
+    local_vars += put_local_vars(ri.struct["request"])
 
     print_prototype(ri, direction, terminate=False)
     ri.cw.block_start()
@@ -2424,6 +2430,9 @@ def print_dump(ri):
         local_vars += ['size_t hdr_len;',
                        'void *hdr;']
 
+    if "request" in ri.op[ri.op_mode]:
+        local_vars += put_local_vars(ri.struct["request"])
+
     ri.cw.write_func_lvar(local_vars)
 
     ri.cw.p('yds.yarg.ys = ys;')
-- 
2.51.0

Re: [PATCH net-next v3 04/13] tools: ynl-gen: refactor local vars for .attr_put() callers

[Donald Hunter]

Asbjørn Sloth Tønnesen <ast@fiberby.net> writes:

> Refactor the generation of local variables needed when building
> requests, by moving the logic into the Type classes, and use the
> same helper in all places where .attr_put() is called.
>
> If any attributes requests identical local_vars, then they will
> be deduplicated, attributes are assumed to only use their local
> variables transiently.
>
> This patch fixes the build errors below:
> $ make -C tools/net/ynl/generated/
> [...]
> -e      GEN wireguard-user.c
> -e      GEN wireguard-user.h
> -e      CC wireguard-user.o
> wireguard-user.c: In function ‘wireguard_get_device_dump’:
> wireguard-user.c:480:9: error: ‘array’ undeclared (first use in func)
>   480 |         array = ynl_attr_nest_start(nlh, WGDEVICE_A_PEERS);
>       |         ^~~~~
> wireguard-user.c:480:9: note: each undeclared identifier is reported
>                         only once for each function it appears in
> wireguard-user.c:481:14: error: ‘i’ undeclared (first use in func)
>   481 |         for (i = 0; i < req->_count.peers; i++)
>       |              ^
> wireguard-user.c: In function ‘wireguard_set_device’:
> wireguard-user.c:533:9: error: ‘array’ undeclared (first use in func)
>   533 |         array = ynl_attr_nest_start(nlh, WGDEVICE_A_PEERS);
>       |         ^~~~~
> make: *** [Makefile:52: wireguard-user.o] Error 1
> make: Leaving directory '/usr/src/linux/tools/net/ynl/generated'
>
> Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>

Reviewed-by: Donald Hunter <donald.hunter@gmail.com>

Re: [PATCH net-next v3 04/13] tools: ynl-gen: refactor local vars for .attr_put() callers

[Jakub Kicinski]

On Thu, 11 Sep 2025 20:04:57 +0000 Asbjørn Sloth Tønnesen wrote:
> +    def attr_put_local_vars(self):
> +        local_vars = []
> +        if self.presence_type() == 'count':
> +            local_vars.append('unsigned int i;')
> +        return local_vars
> +
>      def attr_put(self, ri, var):
>          raise Exception(f"Put not implemented for class type {self.type}")
>  
> @@ -840,6 +846,10 @@ class TypeArrayNest(Type):
>                       '}']
>          return get_lines, None, local_vars
>  
> +    def attr_put_local_vars(self):
> +        local_vars = ['struct nlattr *array;']
> +        return local_vars + super().attr_put_local_vars()

Doesn't feel right. The Type method is a helper which is compatible
with the specific types by checking presence, then you override it,
and on top of that combine the output with super(). I don't like.

Re: [PATCH net-next v3 04/13] tools: ynl-gen: refactor local vars for .attr_put() callers

[Asbjørn Sloth Tønnesen]

On 9/13/25 12:19 AM, Jakub Kicinski wrote:
> On Thu, 11 Sep 2025 20:04:57 +0000 Asbjørn Sloth Tønnesen wrote:
>> +    def attr_put_local_vars(self):
>> +        local_vars = []
>> +        if self.presence_type() == 'count':
>> +            local_vars.append('unsigned int i;')
>> +        return local_vars
>> +
>>       def attr_put(self, ri, var):
>>           raise Exception(f"Put not implemented for class type {self.type}")
>>   
>> @@ -840,6 +846,10 @@ class TypeArrayNest(Type):
>>                        '}']
>>           return get_lines, None, local_vars
>>   
>> +    def attr_put_local_vars(self):
>> +        local_vars = ['struct nlattr *array;']
>> +        return local_vars + super().attr_put_local_vars()
> 
> Doesn't feel right. The Type method is a helper which is compatible
> with the specific types by checking presence, then you override it,
> and on top of that combine the output with super(). I don't like.

I prefer to keep the array variable as a detail isolated to TypeArrayNest,
so I have given it another spin in v4.

[PATCH net-next v3 05/13] tools: ynl-gen: add CodeWriter.p_lines() helper

[Asbjørn Sloth Tønnesen]

Add a helper for writing an array of lines, and convert
all the existing loops doing that, to use the new helper.

This is a trivial patch with no behavioural changes intended,
there are no changes to the generated code.

Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
---
 tools/net/ynl/pyynl/ynl_gen_c.py | 32 ++++++++++++++------------------
 1 file changed, 14 insertions(+), 18 deletions(-)

diff --git a/tools/net/ynl/pyynl/ynl_gen_c.py b/tools/net/ynl/pyynl/ynl_gen_c.py
index e4cb8c95632c..7a2e49a84735 100755
--- a/tools/net/ynl/pyynl/ynl_gen_c.py
+++ b/tools/net/ynl/pyynl/ynl_gen_c.py
@@ -180,8 +180,7 @@ class Type(SpecAttr):
 
     def free(self, ri, var, ref):
         lines = self._free_lines(ri, var, ref)
-        for line in lines:
-            ri.cw.p(line)
+        ri.cw.p_lines(lines)
 
     def arg_member(self, ri):
         member = self._complex_member_type(ri)
@@ -267,13 +266,9 @@ class Type(SpecAttr):
             if self.presence_type() == 'present':
                 ri.cw.p(f"{var}->_present.{self.c_name} = 1;")
 
-        if init_lines:
-            ri.cw.nl()
-            for line in init_lines:
-                ri.cw.p(line)
+        ri.cw.p_lines(init_lines, nl_before=True)
 
-        for line in lines:
-            ri.cw.p(line)
+        ri.cw.p_lines(lines)
         ri.cw.block_end()
         return True
 
@@ -1788,8 +1783,7 @@ class CodeWriter:
         self.block_start()
         self.write_func_lvar(local_vars=local_vars)
 
-        for line in body:
-            self.p(line)
+        self.p_lines(body)
         self.block_end()
 
     def writes_defines(self, defines):
@@ -1830,6 +1824,12 @@ class CodeWriter:
             self.p('#ifdef ' + config_option)
         self._ifdef_block = config_option
 
+    def p_lines(self, lines, nl_before=False):
+        if lines and nl_before:
+            self.nl()
+        for line in lines or []:
+            self.p(line)
+
 
 scalars = {'u8', 'u16', 'u32', 'u64', 's8', 's16', 's32', 's64', 'uint', 'sint'}
 
@@ -2085,8 +2085,7 @@ def put_req_nested(ri, struct):
     ri.cw.block_start()
     ri.cw.write_func_lvar(local_vars)
 
-    for line in init_lines:
-        ri.cw.p(line)
+    ri.cw.p_lines(init_lines)
 
     for _, arg in struct.member_list():
         arg.attr_put(ri, "obj")
@@ -2147,8 +2146,7 @@ def _multi_parse(ri, struct, init_lines, local_vars):
     ri.cw.block_start()
     ri.cw.write_func_lvar(local_vars)
 
-    for line in init_lines:
-        ri.cw.p(line)
+    ri.cw.p_lines(init_lines)
     ri.cw.nl()
 
     for arg in struct.inherited:
@@ -2277,10 +2275,8 @@ def parse_rsp_submsg(ri, struct):
 
         ri.cw.block_start(line=f'{kw} (!strcmp(sel, "{name}"))')
         get_lines, init_lines, _ = arg._attr_get(ri, var)
-        for line in init_lines or []:
-            ri.cw.p(line)
-        for line in get_lines:
-            ri.cw.p(line)
+        ri.cw.p_lines(init_lines)
+        ri.cw.p_lines(get_lines)
         if arg.presence_type() == 'present':
             ri.cw.p(f"{var}->_present.{arg.c_name} = 1;")
         ri.cw.block_end()
-- 
2.51.0

Re: [PATCH net-next v3 05/13] tools: ynl-gen: add CodeWriter.p_lines() helper

[Jakub Kicinski]

On Thu, 11 Sep 2025 20:04:58 +0000 Asbjørn Sloth Tønnesen wrote:
> Add a helper for writing an array of lines, and convert
> all the existing loops doing that, to use the new helper.
> 
> This is a trivial patch with no behavioural changes intended,
> there are no changes to the generated code.

I don't see the need for this.

Re: [PATCH net-next v3 05/13] tools: ynl-gen: add CodeWriter.p_lines() helper

[Asbjørn Sloth Tønnesen]

On 9/13/25 12:21 AM, Jakub Kicinski wrote:
> On Thu, 11 Sep 2025 20:04:58 +0000 Asbjørn Sloth Tønnesen wrote:
>> Add a helper for writing an array of lines, and convert
>> all the existing loops doing that, to use the new helper.
>>
>> This is a trivial patch with no behavioural changes intended,
>> there are no changes to the generated code.
> 
> I don't see the need for this.

Ok, loop macros are only for C, not Python. :)

I have dropped this patch in v4, as it's not important for this
series.

[PATCH net-next v3 06/13] tools: ynl-gen: deduplicate fixed_header handling

[Asbjørn Sloth Tønnesen]

Fixed headers are handled nearly identical in print_dump(),
print_req() and put_req_nested(), generalize them and use a
common function to generate them.

This only causes cosmetic changes to tc_netem_attrs_put() in
tc-user.c.

Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
---
 tools/net/ynl/pyynl/ynl_gen_c.py | 39 ++++++++++++++++----------------
 1 file changed, 20 insertions(+), 19 deletions(-)

diff --git a/tools/net/ynl/pyynl/ynl_gen_c.py b/tools/net/ynl/pyynl/ynl_gen_c.py
index 7a2e49a84735..b3ce0901a19b 100755
--- a/tools/net/ynl/pyynl/ynl_gen_c.py
+++ b/tools/net/ynl/pyynl/ynl_gen_c.py
@@ -1828,7 +1828,10 @@ class CodeWriter:
         if lines and nl_before:
             self.nl()
         for line in lines or []:
-            self.p(line)
+            if line == '':
+                self.nl()
+            else:
+                self.p(line)
 
 
 scalars = {'u8', 'u16', 'u32', 'u64', 's8', 's16', 's32', 's64', 'uint', 'sint'}
@@ -1921,6 +1924,15 @@ def type_name(ri, direction, deref=False):
     return f"struct {op_prefix(ri, direction, deref=deref)}"
 
 
+def prepare_fixed_header(var, local_vars, init_lines):
+    local_vars += ['size_t hdr_len;',
+                   'void *hdr;']
+    init_lines += [f'hdr_len = sizeof({var}->_hdr);',
+                   'hdr = ynl_nlmsg_put_extra_header(nlh, hdr_len);',
+                   f'memcpy(hdr, &{var}->_hdr, hdr_len);',
+                   '']
+
+
 def print_prototype(ri, direction, terminate=True, doc=None):
     suffix = ';' if terminate else ''
 
@@ -2074,10 +2086,7 @@ def put_req_nested(ri, struct):
         local_vars.append('struct nlattr *nest;')
         init_lines.append("nest = ynl_attr_nest_start(nlh, attr_type);")
     if struct.fixed_header:
-        local_vars.append('void *hdr;')
-        struct_sz = f'sizeof({struct.fixed_header})'
-        init_lines.append(f"hdr = ynl_nlmsg_put_extra_header(nlh, {struct_sz});")
-        init_lines.append(f"memcpy(hdr, &obj->_hdr, {struct_sz});")
+        prepare_fixed_header('obj', local_vars, init_lines)
 
     local_vars += put_local_vars(struct)
 
@@ -2346,6 +2355,7 @@ def print_req(ri):
     ret_ok = '0'
     ret_err = '-1'
     direction = "request"
+    init_lines = []
     local_vars = ['struct ynl_req_state yrs = { .yarg = { .ys = ys, }, };',
                   'struct nlmsghdr *nlh;',
                   'int err;']
@@ -2356,8 +2366,7 @@ def print_req(ri):
         local_vars += [f'{type_name(ri, rdir(direction))} *rsp;']
 
     if ri.struct["request"].fixed_header:
-        local_vars += ['size_t hdr_len;',
-                       'void *hdr;']
+        prepare_fixed_header('req', local_vars, init_lines)
 
     local_vars += put_local_vars(ri.struct["request"])
 
@@ -2376,11 +2385,7 @@ def print_req(ri):
         ri.cw.p(f"yrs.yarg.rsp_policy = &{ri.struct['reply'].render_name}_nest;")
     ri.cw.nl()
 
-    if ri.struct['request'].fixed_header:
-        ri.cw.p("hdr_len = sizeof(req->_hdr);")
-        ri.cw.p("hdr = ynl_nlmsg_put_extra_header(nlh, hdr_len);")
-        ri.cw.p("memcpy(hdr, &req->_hdr, hdr_len);")
-        ri.cw.nl()
+    ri.cw.p_lines(init_lines)
 
     for _, attr in ri.struct["request"].member_list():
         attr.attr_put(ri, "req")
@@ -2418,13 +2423,13 @@ def print_dump(ri):
     direction = "request"
     print_prototype(ri, direction, terminate=False)
     ri.cw.block_start()
+    init_lines = []
     local_vars = ['struct ynl_dump_state yds = {};',
                   'struct nlmsghdr *nlh;',
                   'int err;']
 
     if ri.struct['request'].fixed_header:
-        local_vars += ['size_t hdr_len;',
-                       'void *hdr;']
+        prepare_fixed_header('req', local_vars, init_lines)
 
     if "request" in ri.op[ri.op_mode]:
         local_vars += put_local_vars(ri.struct["request"])
@@ -2446,11 +2451,7 @@ def print_dump(ri):
     else:
         ri.cw.p(f"nlh = ynl_gemsg_start_dump(ys, {ri.nl.get_family_id()}, {ri.op.enum_name}, 1);")
 
-    if ri.struct['request'].fixed_header:
-        ri.cw.p("hdr_len = sizeof(req->_hdr);")
-        ri.cw.p("hdr = ynl_nlmsg_put_extra_header(nlh, hdr_len);")
-        ri.cw.p("memcpy(hdr, &req->_hdr, hdr_len);")
-        ri.cw.nl()
+    ri.cw.p_lines(init_lines)
 
     if "request" in ri.op[ri.op_mode]:
         ri.cw.p(f"ys->req_policy = &{ri.struct['request'].render_name}_nest;")
-- 
2.51.0

Re: [PATCH net-next v3 06/13] tools: ynl-gen: deduplicate fixed_header handling

[Donald Hunter]

Asbjørn Sloth Tønnesen <ast@fiberby.net> writes:

> Fixed headers are handled nearly identical in print_dump(),
> print_req() and put_req_nested(), generalize them and use a
> common function to generate them.
>
> This only causes cosmetic changes to tc_netem_attrs_put() in
> tc-user.c.
>
> Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>

Reviewed-by: Donald Hunter <donald.hunter@gmail.com>

Re: [PATCH net-next v3 06/13] tools: ynl-gen: deduplicate fixed_header handling

[Jakub Kicinski]

On Thu, 11 Sep 2025 20:04:59 +0000 Asbjørn Sloth Tønnesen wrote:
> Fixed headers are handled nearly identical in print_dump(),
> print_req() and put_req_nested(), generalize them and use a
> common function to generate them.
> 
> This only causes cosmetic changes to tc_netem_attrs_put() in
> tc-user.c.
> 
> Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
> ---
>  tools/net/ynl/pyynl/ynl_gen_c.py | 39 ++++++++++++++++----------------
>  1 file changed, 20 insertions(+), 19 deletions(-)

This only makes the code longer and harder to follow.

Re: [PATCH net-next v3 06/13] tools: ynl-gen: deduplicate fixed_header handling

[Asbjørn Sloth Tønnesen]

On 9/13/25 12:24 AM, Jakub Kicinski wrote:
> On Thu, 11 Sep 2025 20:04:59 +0000 Asbjørn Sloth Tønnesen wrote:
>> Fixed headers are handled nearly identical in print_dump(),
>> print_req() and put_req_nested(), generalize them and use a
>> common function to generate them.
>>
>> This only causes cosmetic changes to tc_netem_attrs_put() in
>> tc-user.c.
>>
>> Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
>> ---
>>   tools/net/ynl/pyynl/ynl_gen_c.py | 39 ++++++++++++++++----------------
>>   1 file changed, 20 insertions(+), 19 deletions(-)
> 
> This only makes the code longer and harder to follow.

We have 3 functions using put_attr():
* put_req_nested()
* print_req()
* print_dump()

I was just trying to align them a bit more so that they don't
do the same thing in three different ways. I would prefer to
make these functions more aligned, as it will hopefully make
it easier avoid issue like the missing local variables for
.attr_put().

I agree these clean up patches would also fit better in a
dedicated cleanup series, the only reason that I added this to
this series, was because you pushed back, then I said that the
`len` dedup, and other "make code look natural" cleanups might
not be for this series.

I have dropped this patch in v4, it's not important for this
series.

[PATCH net-next v3 07/13] tools: ynl-gen: avoid repetitive variables definitions

[Asbjørn Sloth Tønnesen]

In the generated attribute parsing code, avoid repetitively
defining the same variables over and over again, local to
the conditional block for each attribute.

This patch consolidates the definitions of local variables
for attribute parsing, so that they are defined at the
function level, and re-used across attributes, thus making
the generated code read more natural.

If attributes defines identical local_vars, then they will
be deduplicated, attributes are assumed to only use their
local variables transiently.

The example below shows how `len` was defined repeatedly in
tools/net/ynl/generated/nl80211-user.c:

nl80211_iftype_data_attrs_parse(..) {
   [..]
   ynl_attr_for_each_nested(attr, nested) {
     unsigned int type = ynl_attr_type(attr);

     if (type == NL80211_BAND_IFTYPE_ATTR_IFTYPES) {
       unsigned int len;
       [..]
     } else if (type == NL80211_BAND_IFTYPE_ATTR_HE_CAP_MAC) {
       unsigned int len;
       [..]
     [same pattern 8 times, so 11 times in total]
     } else if (type == NL80211_BAND_IFTYPE_ATTR_EHT_CAP_PPE) {
       unsigned int len;
       [..]
     }
   }
   return 0;
}

This patch results in this diffstat for the generated code:

$ diff -Naur pre/ post/ | diffstat
  devlink-user.c      |  187 +++----------------
  dpll-user.c         |   10 -
  ethtool-user.c      |   49 +----
  fou-user.c          |    5
  handshake-user.c    |    3
  mptcp_pm-user.c     |    3
  nfsd-user.c         |   16 -
  nl80211-user.c      |  159 +---------------
  nlctrl-user.c       |   21 --
  ovpn-user.c         |    7
  ovs_datapath-user.c |    9
  ovs_flow-user.c     |   89 ---------
  ovs_vport-user.c    |    7
  rt-addr-user.c      |   14 -
  rt-link-user.c      |  183 ++----------------
  rt-neigh-user.c     |   14 -
  rt-route-user.c     |   26 --
  rt-rule-user.c      |   11 -
  tc-user.c           |  380 +++++----------------------------------
  tcp_metrics-user.c  |    7
  team-user.c         |    5
  21 files changed, 175 insertions(+), 1030 deletions(-)

The changed lines are mostly `unsigned int len;` definitions:

$ diff -Naur pre/ post/ | grep ^[-+] | grep -v '^[-+]\{3\}' |
  grep -v '^.$' | sed -e 's/\t\+/ /g' | sort | uniq -c | sort -nr
    488 - unsigned int len;
    153 + unsigned int len;
     24 - const struct nlattr *attr2;
     18 + const struct nlattr *attr2;
      1 - __u32 policy_id, attr_id;
      1 + __u32 policy_id, attr_id;
      1 - __u32 op_id;
      1 + __u32 op_id;
      1 - const struct nlattr *attr_policy_id, *attr_attr_id;
      1 + const struct nlattr *attr_policy_id, *attr_attr_id;
      1 - const struct nlattr *attr_op_id;
      1 + const struct nlattr *attr_op_id;

Suggested-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
---
 tools/net/ynl/pyynl/ynl_gen_c.py | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/tools/net/ynl/pyynl/ynl_gen_c.py b/tools/net/ynl/pyynl/ynl_gen_c.py
index b3ce0901a19b..d63b63ac0b8e 100755
--- a/tools/net/ynl/pyynl/ynl_gen_c.py
+++ b/tools/net/ynl/pyynl/ynl_gen_c.py
@@ -247,7 +247,7 @@ class Type(SpecAttr):
         raise Exception(f"Attr get not implemented for class type {self.type}")
 
     def attr_get(self, ri, var, first):
-        lines, init_lines, local_vars = self._attr_get(ri, var)
+        lines, init_lines, _ = self._attr_get(ri, var)
         if type(lines) is str:
             lines = [lines]
         if type(init_lines) is str:
@@ -255,10 +255,6 @@ class Type(SpecAttr):
 
         kw = 'if' if first else 'else if'
         ri.cw.block_start(line=f"{kw} (type == {self.enum_name})")
-        if local_vars:
-            for local in local_vars:
-                ri.cw.p(local)
-            ri.cw.nl()
 
         if not self.is_multi_val():
             ri.cw.p("if (ynl_attr_validate(yarg, attr))")
@@ -2124,6 +2120,7 @@ def _multi_parse(ri, struct, init_lines, local_vars):
             else:
                 raise Exception("Per-op fixed header not supported, yet")
 
+    var_set = set()
     array_nests = set()
     multi_attrs = set()
     needs_parg = False
@@ -2141,6 +2138,13 @@ def _multi_parse(ri, struct, init_lines, local_vars):
             multi_attrs.add(arg)
         needs_parg |= 'nested-attributes' in aspec
         needs_parg |= 'sub-message' in aspec
+
+        try:
+            _, _, l_vars = aspec._attr_get(ri, '')
+            var_set |= set(l_vars) if l_vars else set()
+        except Exception:
+            pass  # _attr_get() not implemented by simple types, ignore
+    local_vars += list(var_set)
     if array_nests or multi_attrs:
         local_vars.append('int i;')
     if needs_parg:
-- 
2.51.0

Re: [PATCH net-next v3 07/13] tools: ynl-gen: avoid repetitive variables definitions

[Donald Hunter]

Asbjørn Sloth Tønnesen <ast@fiberby.net> writes:

> In the generated attribute parsing code, avoid repetitively
> defining the same variables over and over again, local to
> the conditional block for each attribute.
>
> This patch consolidates the definitions of local variables
> for attribute parsing, so that they are defined at the
> function level, and re-used across attributes, thus making
> the generated code read more natural.
>
> If attributes defines identical local_vars, then they will
> be deduplicated, attributes are assumed to only use their
> local variables transiently.
>
> ...
>
> Suggested-by: Jakub Kicinski <kuba@kernel.org>
> Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>

Reviewed-by: Donald Hunter <donald.hunter@gmail.com>

[PATCH net-next v3 08/13] tools: ynl-gen: only validate nested array payload

[Asbjørn Sloth Tønnesen]

In nested arrays don't require that the intermediate attribute
type should be a valid attribute type, it might just be zero
or an incrementing index, it is often not even used.

See include/net/netlink.h about NLA_NESTED_ARRAY:
> The difference to NLA_NESTED is the structure:
> NLA_NESTED has the nested attributes directly inside
> while an array has the nested attributes at another
> level down and the attribute types directly in the
> nesting don't matter.

Example based on include/uapi/linux/wireguard.h:
 > WGDEVICE_A_PEERS: NLA_NESTED
 >   0: NLA_NESTED
 >     WGPEER_A_PUBLIC_KEY: NLA_EXACT_LEN, len WG_KEY_LEN
 >     [..]
 >   0: NLA_NESTED
 >     ...
 >   ...

Previous the check required that the nested type was valid in
the parent attribute set, which in this case resolves to
WGDEVICE_A_UNSPEC, which is YNL_PT_REJECT, and it took the
early exit and returned YNL_PARSE_CB_ERROR.

This patch adds a new helper, ynl_attr_validate_payload(),
which we can use to validate the payload of the nested
attribute, in the context of the parents attribute type,
and it's policy, which in the above case is generated as:
[WGDEVICE_A_PEERS] = {
  .name = "peers",
  .type = YNL_PT_NEST,
  .nest = &wireguard_wgpeer_nest,
},

Some other examples are NL80211_BAND_ATTR_FREQS (nest) and
NL80211_ATTR_SUPPORTED_COMMANDS (u32).

Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
---
 tools/net/ynl/lib/ynl-priv.h     |  2 ++
 tools/net/ynl/lib/ynl.c          | 17 ++++++++++++++---
 tools/net/ynl/pyynl/ynl_gen_c.py |  2 +-
 3 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/tools/net/ynl/lib/ynl-priv.h b/tools/net/ynl/lib/ynl-priv.h
index 824777d7e05e..70ea14c0a0e9 100644
--- a/tools/net/ynl/lib/ynl-priv.h
+++ b/tools/net/ynl/lib/ynl-priv.h
@@ -107,6 +107,8 @@ struct nlmsghdr *
 ynl_gemsg_start_dump(struct ynl_sock *ys, __u32 id, __u8 cmd, __u8 version);
 
 int ynl_attr_validate(struct ynl_parse_arg *yarg, const struct nlattr *attr);
+int ynl_attr_validate_payload(struct ynl_parse_arg *yarg,
+			      const struct nlattr *attr, unsigned int type);
 int ynl_submsg_failed(struct ynl_parse_arg *yarg, const char *field_name,
 		      const char *sel_name);
 
diff --git a/tools/net/ynl/lib/ynl.c b/tools/net/ynl/lib/ynl.c
index 2a169c3c0797..0daf39229587 100644
--- a/tools/net/ynl/lib/ynl.c
+++ b/tools/net/ynl/lib/ynl.c
@@ -360,15 +360,15 @@ static int ynl_cb_done(const struct nlmsghdr *nlh, struct ynl_parse_arg *yarg)
 
 /* Attribute validation */
 
-int ynl_attr_validate(struct ynl_parse_arg *yarg, const struct nlattr *attr)
+static int __ynl_attr_validate(struct ynl_parse_arg *yarg,
+			       const struct nlattr *attr, unsigned int type)
 {
 	const struct ynl_policy_attr *policy;
-	unsigned int type, len;
 	unsigned char *data;
+	unsigned int len;
 
 	data = ynl_attr_data(attr);
 	len = ynl_attr_data_len(attr);
-	type = ynl_attr_type(attr);
 	if (type > yarg->rsp_policy->max_attr) {
 		yerr(yarg->ys, YNL_ERROR_INTERNAL,
 		     "Internal error, validating unknown attribute");
@@ -450,6 +450,17 @@ int ynl_attr_validate(struct ynl_parse_arg *yarg, const struct nlattr *attr)
 	return 0;
 }
 
+int ynl_attr_validate(struct ynl_parse_arg *yarg, const struct nlattr *attr)
+{
+	return __ynl_attr_validate(yarg, attr, ynl_attr_type(attr));
+}
+
+int ynl_attr_validate_payload(struct ynl_parse_arg *yarg,
+			      const struct nlattr *attr, unsigned int type)
+{
+	return __ynl_attr_validate(yarg, attr, type);
+}
+
 int ynl_submsg_failed(struct ynl_parse_arg *yarg, const char *field_name,
 		      const char *sel_name)
 {
diff --git a/tools/net/ynl/pyynl/ynl_gen_c.py b/tools/net/ynl/pyynl/ynl_gen_c.py
index d63b63ac0b8e..ab5b8d98cbda 100755
--- a/tools/net/ynl/pyynl/ynl_gen_c.py
+++ b/tools/net/ynl/pyynl/ynl_gen_c.py
@@ -831,7 +831,7 @@ class TypeArrayNest(Type):
         local_vars = ['const struct nlattr *attr2;']
         get_lines = [f'attr_{self.c_name} = attr;',
                      'ynl_attr_for_each_nested(attr2, attr) {',
-                     '\tif (ynl_attr_validate(yarg, attr2))',
+                     '\tif (ynl_attr_validate_payload(yarg, attr2, type))',
                      '\t\treturn YNL_PARSE_CB_ERROR;',
                      f'\tn_{self.c_name}++;',
                      '}']
-- 
2.51.0

Re: [PATCH net-next v3 08/13] tools: ynl-gen: only validate nested array payload

[Jakub Kicinski]

On Thu, 11 Sep 2025 20:05:01 +0000 Asbjørn Sloth Tønnesen wrote:
> +int ynl_attr_validate_payload(struct ynl_parse_arg *yarg,
> +			      const struct nlattr *attr, unsigned int type)
> +{
> +	return __ynl_attr_validate(yarg, attr, type);
> +}

Why not expose __ynl_attr_validate() to the callers?
I don't think the _payload() suffix is crystal clear, we're still
validating attr, _payload() makes it sound like we're validating
what's inside attr?

Re: [PATCH net-next v3 08/13] tools: ynl-gen: only validate nested array payload

[Asbjørn Sloth Tønnesen]

On 9/13/25 12:27 AM, Jakub Kicinski wrote:
> On Thu, 11 Sep 2025 20:05:01 +0000 Asbjørn Sloth Tønnesen wrote:
>> +int ynl_attr_validate_payload(struct ynl_parse_arg *yarg,
>> +			      const struct nlattr *attr, unsigned int type)
>> +{
>> +	return __ynl_attr_validate(yarg, attr, type);
>> +}
> 
> Why not expose __ynl_attr_validate() to the callers?
> I don't think the _payload() suffix is crystal clear, we're still
> validating attr, _payload() makes it sound like we're validating
> what's inside attr?

I didn't wanna call __ynl_attr_validate() directly, as the only __ynl_*
function in ynl-priv.h is __ynl_attr_put_overflow(), and that is only
used in other static functions within that file. I agree, that _payload()
might not be the best given that we currently don't look deeper than
validating that the length a bit, so maybe _length() would have been
better.

In v4, I have changed it to just expose __ynl_attr_validate() in
ynl-priv.h, and changed ynl_attr_validate() to an inline function.

[PATCH net-next v3 09/13] tools: ynl-gen: rename TypeArrayNest to TypeIndexedArray

[Asbjørn Sloth Tønnesen]

Since TypeArrayNest can now be used with many other sub-types
than nest, then rename it to TypeIndexedArray, to reduce
confusion.

This patch continues the rename, that was started in commit
aa6485d813ad ("ynl: rename array-nest to indexed-array"),
when the YNL type was renamed.

In order to get rid of all references to the old naming,
within ynl, then renaming some variables in _multi_parse().

This is a trivial patch with no behavioural changes intended.

Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
---
 tools/net/ynl/pyynl/ynl_gen_c.py | 36 ++++++++++++++++----------------
 1 file changed, 18 insertions(+), 18 deletions(-)

diff --git a/tools/net/ynl/pyynl/ynl_gen_c.py b/tools/net/ynl/pyynl/ynl_gen_c.py
index ab5b8d98cbda..2c5787b518f0 100755
--- a/tools/net/ynl/pyynl/ynl_gen_c.py
+++ b/tools/net/ynl/pyynl/ynl_gen_c.py
@@ -788,7 +788,7 @@ class TypeMultiAttr(Type):
                 f"{presence} = n_{self.c_name};"]
 
 
-class TypeArrayNest(Type):
+class TypeIndexedArray(Type):
     def is_multi_val(self):
         return True
 
@@ -825,7 +825,7 @@ class TypeArrayNest(Type):
         elif self.attr['sub-type'] == 'nest':
             return f'.type = YNL_PT_NEST, .nest = &{self.nested_render_name}_nest, '
         else:
-            raise Exception(f"Typol for ArrayNest sub-type {self.attr['sub-type']} not supported, yet")
+            raise Exception(f"Typol for IndexedArray sub-type {self.attr['sub-type']} not supported, yet")
 
     def _attr_get(self, ri, var):
         local_vars = ['const struct nlattr *attr2;']
@@ -855,7 +855,7 @@ class TypeArrayNest(Type):
             ri.cw.p(f'for (i = 0; i < {var}->_count.{self.c_name}; i++)')
             ri.cw.p(f"{self.nested_render_name}_put(nlh, i, &{var}->{self.c_name}[i]);")
         else:
-            raise Exception(f"Put for ArrayNest sub-type {self.attr['sub-type']} not supported, yet")
+            raise Exception(f"Put for IndexedArray sub-type {self.attr['sub-type']} not supported, yet")
         ri.cw.p('ynl_attr_nest_end(nlh, array);')
 
     def _setter_lines(self, ri, member, presence):
@@ -1132,7 +1132,7 @@ class AttrSet(SpecAttrSet):
             t = TypeNest(self.family, self, elem, value)
         elif elem['type'] == 'indexed-array' and 'sub-type' in elem:
             if elem["sub-type"] in ['binary', 'nest', 'u32']:
-                t = TypeArrayNest(self.family, self, elem, value)
+                t = TypeIndexedArray(self.family, self, elem, value)
             else:
                 raise Exception(f'new_attr: unsupported sub-type {elem["sub-type"]}')
         elif elem['type'] == 'nest-type-value':
@@ -2120,18 +2120,18 @@ def _multi_parse(ri, struct, init_lines, local_vars):
             else:
                 raise Exception("Per-op fixed header not supported, yet")
 
-    var_set = set()
-    array_nests = set()
+    indexed_arrays = set()
     multi_attrs = set()
     needs_parg = False
+    var_set = set()
     for arg, aspec in struct.member_list():
         if aspec['type'] == 'indexed-array' and 'sub-type' in aspec:
             if aspec["sub-type"] in {'binary', 'nest'}:
                 local_vars.append(f'const struct nlattr *attr_{aspec.c_name};')
-                array_nests.add(arg)
+                indexed_arrays.add(arg)
             elif aspec['sub-type'] in scalars:
                 local_vars.append(f'const struct nlattr *attr_{aspec.c_name};')
-                array_nests.add(arg)
+                indexed_arrays.add(arg)
             else:
                 raise Exception(f'Not supported sub-type {aspec["sub-type"]}')
         if 'multi-attr' in aspec:
@@ -2145,16 +2145,16 @@ def _multi_parse(ri, struct, init_lines, local_vars):
         except Exception:
             pass  # _attr_get() not implemented by simple types, ignore
     local_vars += list(var_set)
-    if array_nests or multi_attrs:
+    if indexed_arrays or multi_attrs:
         local_vars.append('int i;')
     if needs_parg:
         local_vars.append('struct ynl_parse_arg parg;')
         init_lines.append('parg.ys = yarg->ys;')
 
-    all_multi = array_nests | multi_attrs
+    all_multi = indexed_arrays | multi_attrs
 
-    for anest in sorted(all_multi):
-        local_vars.append(f"unsigned int n_{struct[anest].c_name} = 0;")
+    for arg in sorted(all_multi):
+        local_vars.append(f"unsigned int n_{struct[arg].c_name} = 0;")
 
     ri.cw.block_start()
     ri.cw.write_func_lvar(local_vars)
@@ -2173,8 +2173,8 @@ def _multi_parse(ri, struct, init_lines, local_vars):
         else:
             ri.cw.p('hdr = ynl_nlmsg_data_offset(nlh, sizeof(struct genlmsghdr));')
         ri.cw.p(f"memcpy(&dst->_hdr, hdr, sizeof({struct.fixed_header}));")
-    for anest in sorted(all_multi):
-        aspec = struct[anest]
+    for arg in sorted(all_multi):
+        aspec = struct[arg]
         ri.cw.p(f"if (dst->{aspec.c_name})")
         ri.cw.p(f'return ynl_error_parse(yarg, "attribute already present ({struct.attr_set.name}.{aspec.name})");')
 
@@ -2192,8 +2192,8 @@ def _multi_parse(ri, struct, init_lines, local_vars):
     ri.cw.block_end()
     ri.cw.nl()
 
-    for anest in sorted(array_nests):
-        aspec = struct[anest]
+    for arg in sorted(indexed_arrays):
+        aspec = struct[arg]
 
         ri.cw.block_start(line=f"if (n_{aspec.c_name})")
         ri.cw.p(f"dst->{aspec.c_name} = calloc(n_{aspec.c_name}, sizeof(*dst->{aspec.c_name}));")
@@ -2218,8 +2218,8 @@ def _multi_parse(ri, struct, init_lines, local_vars):
         ri.cw.block_end()
     ri.cw.nl()
 
-    for anest in sorted(multi_attrs):
-        aspec = struct[anest]
+    for arg in sorted(multi_attrs):
+        aspec = struct[arg]
         ri.cw.block_start(line=f"if (n_{aspec.c_name})")
         ri.cw.p(f"dst->{aspec.c_name} = calloc(n_{aspec.c_name}, sizeof(*dst->{aspec.c_name}));")
         ri.cw.p(f"dst->_count.{aspec.c_name} = n_{aspec.c_name};")
-- 
2.51.0

Re: [PATCH net-next v3 09/13] tools: ynl-gen: rename TypeArrayNest to TypeIndexedArray

[Donald Hunter]

Asbjørn Sloth Tønnesen <ast@fiberby.net> writes:

> Since TypeArrayNest can now be used with many other sub-types
> than nest, then rename it to TypeIndexedArray, to reduce
> confusion.
>
> This patch continues the rename, that was started in commit
> aa6485d813ad ("ynl: rename array-nest to indexed-array"),
> when the YNL type was renamed.
>
> In order to get rid of all references to the old naming,
> within ynl, then renaming some variables in _multi_parse().
>
> This is a trivial patch with no behavioural changes intended.
>
> Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>

Reviewed-by: Donald Hunter <donald.hunter@gmail.com>

[PATCH net-next v3 10/13] tools: ynl: move nest packing to a helper function

[Asbjørn Sloth Tønnesen]

This patch moves nest packing into a helper function,
that can also be used for packing indexed arrays.

No behavioural changes intended.

Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
---
 tools/net/ynl/pyynl/lib/ynl.py | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/tools/net/ynl/pyynl/lib/ynl.py b/tools/net/ynl/pyynl/lib/ynl.py
index 50805e05020a..92ff26f34f4d 100644
--- a/tools/net/ynl/pyynl/lib/ynl.py
+++ b/tools/net/ynl/pyynl/lib/ynl.py
@@ -561,11 +561,8 @@ class YnlFamily(SpecFamily):
 
         if attr["type"] == 'nest':
             nl_type |= Netlink.NLA_F_NESTED
-            attr_payload = b''
             sub_space = attr['nested-attributes']
-            sub_attrs = SpaceAttrs(self.attr_sets[sub_space], value, search_attrs)
-            for subname, subvalue in value.items():
-                attr_payload += self._add_attr(sub_space, subname, subvalue, sub_attrs)
+            attr_payload = self._add_nest_attrs(value, sub_space, search_attrs)
         elif attr["type"] == 'flag':
             if not value:
                 # If value is absent or false then skip attribute creation.
@@ -622,6 +619,14 @@ class YnlFamily(SpecFamily):
         pad = b'\x00' * ((4 - len(attr_payload) % 4) % 4)
         return struct.pack('HH', len(attr_payload) + 4, nl_type) + attr_payload + pad
 
+    def _add_nest_attrs(self, value, sub_space, search_attrs):
+        sub_attrs = SpaceAttrs(self.attr_sets[sub_space], value, search_attrs)
+        attr_payload = b''
+        for subname, subvalue in value.items():
+            attr_payload += self._add_attr(sub_space, subname, subvalue,
+                                           sub_attrs)
+        return attr_payload
+
     def _get_enum_or_unknown(self, enum, raw):
         try:
             name = enum.entries_by_val[raw].name
-- 
2.51.0

[PATCH net-next v3 11/13] tools: ynl: encode indexed-arrays

[Asbjørn Sloth Tønnesen]

This patch adds support for encoding indexed-array
attributes with sub-type nest in pyynl.

Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
---
 tools/net/ynl/pyynl/lib/ynl.py | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/tools/net/ynl/pyynl/lib/ynl.py b/tools/net/ynl/pyynl/lib/ynl.py
index 92ff26f34f4d..9fd83f8b091f 100644
--- a/tools/net/ynl/pyynl/lib/ynl.py
+++ b/tools/net/ynl/pyynl/lib/ynl.py
@@ -563,6 +563,11 @@ class YnlFamily(SpecFamily):
             nl_type |= Netlink.NLA_F_NESTED
             sub_space = attr['nested-attributes']
             attr_payload = self._add_nest_attrs(value, sub_space, search_attrs)
+        elif attr['type'] == 'indexed-array' and attr['sub-type'] == 'nest':
+            nl_type |= Netlink.NLA_F_NESTED
+            sub_space = attr['nested-attributes']
+            attr_payload = self._encode_indexed_array(value, sub_space,
+                                                      search_attrs)
         elif attr["type"] == 'flag':
             if not value:
                 # If value is absent or false then skip attribute creation.
@@ -616,6 +621,9 @@ class YnlFamily(SpecFamily):
         else:
             raise Exception(f'Unknown type at {space} {name} {value} {attr["type"]}')
 
+        return self._add_attr_raw(nl_type, attr_payload)
+
+    def _add_attr_raw(self, nl_type, attr_payload):
         pad = b'\x00' * ((4 - len(attr_payload) % 4) % 4)
         return struct.pack('HH', len(attr_payload) + 4, nl_type) + attr_payload + pad
 
@@ -627,6 +635,14 @@ class YnlFamily(SpecFamily):
                                            sub_attrs)
         return attr_payload
 
+    def _encode_indexed_array(self, vals, sub_space, search_attrs):
+        attr_payload = b''
+        for i, val in enumerate(vals):
+            idx = i | Netlink.NLA_F_NESTED
+            val_payload = self._add_nest_attrs(val, sub_space, search_attrs)
+            attr_payload += self._add_attr_raw(idx, val_payload)
+        return attr_payload
+
     def _get_enum_or_unknown(self, enum, raw):
         try:
             name = enum.entries_by_val[raw].name
-- 
2.51.0

Re: [PATCH net-next v3 11/13] tools: ynl: encode indexed-arrays

[Donald Hunter]

Asbjørn Sloth Tønnesen <ast@fiberby.net> writes:

> This patch adds support for encoding indexed-array
> attributes with sub-type nest in pyynl.
>
> Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>

Reviewed-by: Donald Hunter <donald.hunter@gmail.com>

[PATCH net-next v3 12/13] tools: ynl: decode hex input

[Asbjørn Sloth Tønnesen]

This patch adds support for decoding hex input, so
that binary attributes can be read through --json.

Example (using future wireguard.yaml):
 $ sudo ./tools/net/ynl/pyynl/cli.py --family wireguard \
   --do set-device --json '{"ifindex":3,
     "private-key":"2a ae 6c 35 c9 4f cf <... to 32 bytes>"}'

In order to somewhat mirror what is done in _formatted_string(),
then for non-binary attributes attempt to convert it to an int.

Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
---
 tools/net/ynl/pyynl/lib/ynl.py | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tools/net/ynl/pyynl/lib/ynl.py b/tools/net/ynl/pyynl/lib/ynl.py
index 9fd83f8b091f..707753e371e2 100644
--- a/tools/net/ynl/pyynl/lib/ynl.py
+++ b/tools/net/ynl/pyynl/lib/ynl.py
@@ -971,6 +971,11 @@ class YnlFamily(SpecFamily):
                 raw = ip.packed
             else:
                 raw = int(ip)
+        elif attr_spec.display_hint == 'hex':
+            if attr_spec['type'] == 'binary':
+                raw = bytes.fromhex(string)
+            else:
+                raw = int(string, 16)
         else:
             raise Exception(f"Display hint '{attr_spec.display_hint}' not implemented"
                             f" when parsing '{attr_spec['name']}'")
-- 
2.51.0

Re: [PATCH net-next v3 12/13] tools: ynl: decode hex input

[Donald Hunter]

Asbjørn Sloth Tønnesen <ast@fiberby.net> writes:

> This patch adds support for decoding hex input, so
> that binary attributes can be read through --json.
>
> Example (using future wireguard.yaml):
>  $ sudo ./tools/net/ynl/pyynl/cli.py --family wireguard \
>    --do set-device --json '{"ifindex":3,
>      "private-key":"2a ae 6c 35 c9 4f cf <... to 32 bytes>"}'
>
> In order to somewhat mirror what is done in _formatted_string(),
> then for non-binary attributes attempt to convert it to an int.
>
> Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>

Reviewed-by: Donald Hunter <donald.hunter@gmail.com>

[PATCH net-next v3 13/13] tools: ynl: add ipv4-or-v6 display hint

[Asbjørn Sloth Tønnesen]

The attribute WGALLOWEDIP_A_IPADDR can contain either an IPv4
or an IPv6 address depending on WGALLOWEDIP_A_FAMILY, however
in practice it is enough to look at the attribute length.

This patch implements an ipv4-or-v6 display hint, that can
deal with this kind of attribute.

It only implements this display hint for genetlink-legacy, it
can be added to other protocol variants if needed, but we don't
want to encourage it's use.

Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
---
 Documentation/netlink/genetlink-legacy.yaml | 2 +-
 tools/net/ynl/pyynl/lib/ynl.py              | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/Documentation/netlink/genetlink-legacy.yaml b/Documentation/netlink/genetlink-legacy.yaml
index b29d62eefa16..66fb8653a344 100644
--- a/Documentation/netlink/genetlink-legacy.yaml
+++ b/Documentation/netlink/genetlink-legacy.yaml
@@ -154,7 +154,7 @@ properties:
                   Optional format indicator that is intended only for choosing
                   the right formatting mechanism when displaying values of this
                   type.
-                enum: [ hex, mac, fddi, ipv4, ipv6, uuid ]
+                enum: [ hex, mac, fddi, ipv4, ipv6, ipv4-or-v6, uuid ]
               struct:
                 description: Name of the nested struct type.
                 type: string
diff --git a/tools/net/ynl/pyynl/lib/ynl.py b/tools/net/ynl/pyynl/lib/ynl.py
index 707753e371e2..62383c70ebb9 100644
--- a/tools/net/ynl/pyynl/lib/ynl.py
+++ b/tools/net/ynl/pyynl/lib/ynl.py
@@ -956,7 +956,7 @@ class YnlFamily(SpecFamily):
                 formatted = hex(raw)
             else:
                 formatted = bytes.hex(raw, ' ')
-        elif display_hint in [ 'ipv4', 'ipv6' ]:
+        elif display_hint in [ 'ipv4', 'ipv6', 'ipv4-or-v6' ]:
             formatted = format(ipaddress.ip_address(raw))
         elif display_hint == 'uuid':
             formatted = str(uuid.UUID(bytes=raw))
@@ -965,7 +965,7 @@ class YnlFamily(SpecFamily):
         return formatted
 
     def _from_string(self, string, attr_spec):
-        if attr_spec.display_hint in ['ipv4', 'ipv6']:
+        if attr_spec.display_hint in ['ipv4', 'ipv6', 'ipv4-or-v6']:
             ip = ipaddress.ip_address(string)
             if attr_spec['type'] == 'binary':
                 raw = ip.packed
-- 
2.51.0