[PATCH net-next v1 0/6] tools/net/ynl: Add 'sub-message' support to ynl

[PATCH net-next v1 0/6] tools/net/ynl: Add 'sub-message' support to ynl

[Donald Hunter]

This patchset adds a 'sub-message' attribute type to the netlink-raw
schema and implements it in ynl. This provides support for kind-specific
options attributes as used in rt_link and tc raw netlink families.

A description of the new 'sub-message' attribute type and the
corresponding sub-message definitions is provided in patch 2.

The patchset includes updates to the rt_link spec and a new tc spec that
make use of the new 'sub-message' attribute type.

Patch 1 adds missing scalar types to the netlink-raw schema
Patch 2 and 3 add sub-message support to the schema and ynl
Patch 4 adds binary and pad support to structs in netlink-raw
Patches 5 and 6 contain specs that use the sub-message attribute type

Donald Hunter (6):
  doc/netlink: Add bitfield32, s8, s16 to the netlink-raw schema
  doc/netlink: Add sub-message support to netlink-raw
  tools/net/ynl: Add 'sub-message' attribute decoding to ynl
  tools/net/ynl: Add binary and pad support to structs for tc
  doc/netlink/specs: add sub-message type to rt_link family
  doc/netlink/specs: Add a spec for tc

 Documentation/netlink/netlink-raw.yaml   |   56 +-
 Documentation/netlink/specs/rt_link.yaml |  273 ++-
 Documentation/netlink/specs/tc.yaml      | 2008 ++++++++++++++++++++++
 tools/net/ynl/lib/nlspec.py              |   57 +-
 tools/net/ynl/lib/ynl.py                 |   84 +-
 5 files changed, 2452 insertions(+), 26 deletions(-)
 create mode 100644 Documentation/netlink/specs/tc.yaml

-- 
2.42.0

[PATCH net-next v1 1/6] doc/netlink: Add bitfield32, s8, s16 to the netlink-raw schema

[Donald Hunter]

The netlink-raw schema was not updated when bitfield32 was added
to the genetlink-legacy schema. It is needed for rtnetlink families.

s8 and s16 were also missing.

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
---
 Documentation/netlink/netlink-raw.yaml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/netlink/netlink-raw.yaml b/Documentation/netlink/netlink-raw.yaml
index 775cce8c548a..ad5395040765 100644
--- a/Documentation/netlink/netlink-raw.yaml
+++ b/Documentation/netlink/netlink-raw.yaml
@@ -200,7 +200,8 @@ properties:
                 type: string
               type: &attr-type
                 description: The netlink attribute type
-                enum: [ unused, pad, flag, binary, u8, u16, u32, u64, s32, s64,
+                enum: [ unused, pad, flag, binary, bitfield32,
+                        u8, u16, u32, u64, s8, s16, s32, s64,
                         string, nest, array-nest, nest-type-value ]
               doc:
                 description: Documentation of the attribute.
-- 
2.42.0

[PATCH net-next v1 2/6] doc/netlink: Add sub-message support to netlink-raw

[Donald Hunter]

Add a 'sub-message' attribute type with a selector that supports
polymorphic attribute formats for raw netlink families like tc.

A sub-message attribute uses the value of another attribute as a
selector key to choose the right sub-message format. For example if the
following attribute has already been decoded:

  { "kind": "gre" }

and we encounter the following attribute spec:

  -
    name: data
    type: sub-message
    sub-message: linkinfo-data-msg
    selector: kind

Then we look for a sub-message definition called 'linkinfo-data-msg' and
use the value of the 'kind' attribute i.e. 'gre' as the key to choose
the correct format for the sub-message:

  sub-messages:
    name: linkinfo-data-msg
    formats:
      -
        value: bridge
        attribute-set: linkinfo-bridge-attrs
      -
        value: gre
        attribute-set: linkinfo-gre-attrs
      -
        value: geneve
        attribute-set: linkinfo-geneve-attrs

This would decode the attribute value as a sub-message with the
attribute-set called 'linkinfo-gre-attrs' as the attribute space.

A sub-message can have an optional 'fixed-header' followed by zero or
more attributes from an attribute-set. For example the following
'tc-options-msg' sub-message defines message formats that use a mixture
of fixed-header, attribute-set or both together:

  sub-messages:
    -
      name: tc-options-msg
      formats:
        -
          value: bfifo
          fixed-header: tc-fifo-qopt
        -
          value: cake
          attribute-set: tc-cake-attrs
        -
          value: netem
          fixed-header: tc-netem-qopt
          attribute-set: tc-netem-attrs

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
---
 Documentation/netlink/netlink-raw.yaml | 51 +++++++++++++++++++++++++-
 1 file changed, 50 insertions(+), 1 deletion(-)

diff --git a/Documentation/netlink/netlink-raw.yaml b/Documentation/netlink/netlink-raw.yaml
index ad5395040765..26203282422f 100644
--- a/Documentation/netlink/netlink-raw.yaml
+++ b/Documentation/netlink/netlink-raw.yaml
@@ -202,7 +202,8 @@ properties:
                 description: The netlink attribute type
                 enum: [ unused, pad, flag, binary, bitfield32,
                         u8, u16, u32, u64, s8, s16, s32, s64,
-                        string, nest, array-nest, nest-type-value ]
+                        string, nest, array-nest, nest-type-value,
+                        sub-message ]
               doc:
                 description: Documentation of the attribute.
                 type: string
@@ -261,6 +262,17 @@ properties:
                 description: Name of the struct type used for the attribute.
                 type: string
               # End genetlink-legacy
+              # Start netlink-raw
+              sub-message:
+                description:
+                  Name of the sub-message definition to use for the attribute.
+                type: string
+              selector:
+                description:
+                  Name of the attribute to use for dynamic selection of sub-message
+                  format specifier.
+                type: string
+              # End netlink-raw
 
       # Make sure name-prefix does not appear in subsets (subsets inherit naming)
       dependencies:
@@ -283,6 +295,43 @@ properties:
             items:
               required: [ type ]
 
+  # Start netlink-raw
+  sub-messages:
+    description: Definition of sub message attributes
+    type: array
+    items:
+      type: object
+      additionalProperties: False
+      required: [ name, formats ]
+      properties:
+        name:
+          description: Name of the sub-message definition
+          type: string
+        formats:
+          description: Dynamically selected format specifiers
+          type: array
+          items:
+            type: object
+            additionalProperties: False
+            required: [ value ]
+            properties:
+              value:
+                description:
+                  Value to match for dynamic selection of sub-message format
+                  specifier.
+                type: string
+              fixed-header:
+                description:
+                  Name of the struct definition to use as the fixed header
+                  for the sub message.
+                type: string
+              attribute-set:
+                description:
+                  Name of the attribute space from which to resolve attributes
+                  in the sub message.
+                type: string
+  # End netlink-raw
+
   operations:
     description: Operations supported by the protocol.
     type: object
-- 
2.42.0

Re: [PATCH net-next v1 0/6] tools/net/ynl: Add 'sub-message' support to ynl

[Donald Hunter]

Looks like I'll need to resend because I got send failures.

[PATCH net-next v1 0/6] tools/net/ynl: Add 'sub-message' support to ynl

[Donald Hunter]

This patchset adds a 'sub-message' attribute type to the netlink-raw
schema and implements it in ynl. This provides support for kind-specific
options attributes as used in rt_link and tc raw netlink families.

A description of the new 'sub-message' attribute type and the
corresponding sub-message definitions is provided in patch 2.

The patchset includes updates to the rt_link spec and a new tc spec that
make use of the new 'sub-message' attribute type.

Patch 1 adds missing scalar types to the netlink-raw schema
Patch 2 and 3 add sub-message support to the schema and ynl
Patch 4 adds binary and pad support to structs in netlink-raw
Patches 5 and 6 contain specs that use the sub-message attribute type

Donald Hunter (6):
  doc/netlink: Add bitfield32, s8, s16 to the netlink-raw schema
  doc/netlink: Add sub-message support to netlink-raw
  tools/net/ynl: Add 'sub-message' attribute decoding to ynl
  tools/net/ynl: Add binary and pad support to structs for tc
  doc/netlink/specs: add sub-message type to rt_link family
  doc/netlink/specs: Add a spec for tc

 Documentation/netlink/netlink-raw.yaml   |   56 +-
 Documentation/netlink/specs/rt_link.yaml |  273 ++-
 Documentation/netlink/specs/tc.yaml      | 2008 ++++++++++++++++++++++
 tools/net/ynl/lib/nlspec.py              |   57 +-
 tools/net/ynl/lib/ynl.py                 |   84 +-
 5 files changed, 2452 insertions(+), 26 deletions(-)
 create mode 100644 Documentation/netlink/specs/tc.yaml

-- 
2.42.0

Re: [PATCH net-next v1 0/6] tools/net/ynl: Add 'sub-message' support to ynl

[patchwork-bot+netdevbpf]

Hello:

This series was applied to netdev/net-next.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Thu, 30 Nov 2023 21:49:52 +0000 you wrote:
> This patchset adds a 'sub-message' attribute type to the netlink-raw
> schema and implements it in ynl. This provides support for kind-specific
> options attributes as used in rt_link and tc raw netlink families.
> 
> A description of the new 'sub-message' attribute type and the
> corresponding sub-message definitions is provided in patch 2.
> 
> [...]

Here is the summary with links:
  - [net-next,v1,1/6] doc/netlink: Add bitfield32, s8, s16 to the netlink-raw schema
    https://git.kernel.org/netdev/net-next/c/527d2cd8b852
  - [net-next,v1,2/6] doc/netlink: Add sub-message support to netlink-raw
    (no matching commit)
  - [net-next,v1,3/6] tools/net/ynl: Add 'sub-message' attribute decoding to ynl
    (no matching commit)
  - [net-next,v1,4/6] tools/net/ynl: Add binary and pad support to structs for tc
    (no matching commit)
  - [net-next,v1,5/6] doc/netlink/specs: add sub-message type to rt_link family
    (no matching commit)
  - [net-next,v1,6/6] doc/netlink/specs: Add a spec for tc
    (no matching commit)

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

Re: [PATCH net-next v1 0/6] tools/net/ynl: Add 'sub-message' support to ynl

[Jakub Kicinski]

On Thu, 30 Nov 2023 21:49:52 +0000 Donald Hunter wrote:
>  Documentation/netlink/specs/rt_link.yaml |  273 ++-
>  Documentation/netlink/specs/tc.yaml      | 2008 ++++++++++++++++++++++

Should we add sub-messages to tools/net/ynl/ynl-gen-rst.py ?
Does the output look sane with the new attributes?

Re: [PATCH net-next v1 0/6] tools/net/ynl: Add 'sub-message' support to ynl

[Donald Hunter]

Jakub Kicinski <kuba@kernel.org> writes:

> On Thu, 30 Nov 2023 21:49:52 +0000 Donald Hunter wrote:
>>  Documentation/netlink/specs/rt_link.yaml |  273 ++-
>>  Documentation/netlink/specs/tc.yaml      | 2008 ++++++++++++++++++++++
>
> Should we add sub-messages to tools/net/ynl/ynl-gen-rst.py ?
> Does the output look sane with the new attributes?

Ah, yes we should. Okay if I look at this as a followup patch?

Re: [PATCH net-next v1 0/6] tools/net/ynl: Add 'sub-message' support to ynl

[Jakub Kicinski]

On Mon, 04 Dec 2023 15:54:13 +0000 Donald Hunter wrote:
> > Should we add sub-messages to tools/net/ynl/ynl-gen-rst.py ?
> > Does the output look sane with the new attributes?  
> 
> Ah, yes we should. Okay if I look at this as a followup patch?

Improving looks is 100% fine as a follow up.
But do check we don't break rendering completely for stuff we should
already support when sub-messages appear.