[PATCH net-next v1 0/4] doc: netlink: Fixes for ynl doc generator

[PATCH net-next v1 0/4] doc: netlink: Fixes for ynl doc generator

[Donald Hunter]

Several fixes to ynl-gen-rst to resolve issues that can be seen
in:

https://docs.kernel.org/6.10-rc1/networking/netlink_spec/dpll.html#device-id-get
https://docs.kernel.org/6.10-rc1/networking/netlink_spec/dpll.html#lock-status

In patch 2, by not using 'sanitize' for op docs, any formatting in the
.yaml gets passed straight through to the generated .rst which means
that basic rst (also markdown compatible) list formatting can be used in
the .yaml

Donald Hunter (4):
  doc: netlink: Fix generated .rst for multi-line docs
  doc: netlink: Don't 'sanitize' op docstrings in generated .rst
  doc: netlink: Fix formatting of op flags in generated .rst
  doc: netlink: Fix op pre and post fields in generated .rst

 Documentation/netlink/specs/dpll.yaml |  1 +
 tools/net/ynl/ynl-gen-rst.py          | 13 +++++++++----
 2 files changed, 10 insertions(+), 4 deletions(-)

-- 
2.44.0

[PATCH net-next v1 1/4] doc: netlink: Fix generated .rst for multi-line docs

[Donald Hunter]

Fix the newline replacement in ynl-gen-rst.py to put spaces between
concatenated lines. This fixes the broken doc string formatting.

See the dpll docs for an example of broken concatenation:

https://docs.kernel.org/6.9/networking/netlink_spec/dpll.html#lock-status

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
---
 tools/net/ynl/ynl-gen-rst.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/net/ynl/ynl-gen-rst.py b/tools/net/ynl/ynl-gen-rst.py
index 657e881d2ea4..5c7465d6befa 100755
--- a/tools/net/ynl/ynl-gen-rst.py
+++ b/tools/net/ynl/ynl-gen-rst.py
@@ -49,7 +49,7 @@ def inline(text: str) -> str:
 def sanitize(text: str) -> str:
     """Remove newlines and multiple spaces"""
     # This is useful for some fields that are spread across multiple lines
-    return str(text).replace("\n", "").strip()
+    return str(text).replace("\n", " ").strip()
 
 
 def rst_fields(key: str, value: str, level: int = 0) -> str:
-- 
2.44.0

[PATCH net-next v1 2/4] doc: netlink: Don't 'sanitize' op docstrings in generated .rst

[Donald Hunter]

The doc strings for do/dump ops are emitted as toplevel .rst constructs
so they can be multi-line. Pass multi-line text straight through to the
.rst to retain any simple formatting from the .yaml

This fixes e.g. list formatting for the pin-get docs in dpll.yaml:

https://docs.kernel.org/6.9/networking/netlink_spec/dpll.html#pin-get

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
---
 Documentation/netlink/specs/dpll.yaml | 1 +
 tools/net/ynl/ynl-gen-rst.py          | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/netlink/specs/dpll.yaml b/Documentation/netlink/specs/dpll.yaml
index 95b0eb1486bf..94132d30e0e0 100644
--- a/Documentation/netlink/specs/dpll.yaml
+++ b/Documentation/netlink/specs/dpll.yaml
@@ -479,6 +479,7 @@ operations:
       name: pin-get
       doc: |
         Get list of pins and its attributes.
+
         - dump request without any attributes given - list all the pins in the
           system
         - dump request with target dpll - list all the pins registered with
diff --git a/tools/net/ynl/ynl-gen-rst.py b/tools/net/ynl/ynl-gen-rst.py
index 5c7465d6befa..1096a71d7867 100755
--- a/tools/net/ynl/ynl-gen-rst.py
+++ b/tools/net/ynl/ynl-gen-rst.py
@@ -178,7 +178,7 @@ def parse_operations(operations: List[Dict[str, Any]], namespace: str) -> str:
 
     for operation in operations:
         lines.append(rst_section(namespace, 'operation', operation["name"]))
-        lines.append(rst_paragraph(sanitize(operation["doc"])) + "\n")
+        lines.append(rst_paragraph(operation["doc"]) + "\n")
 
         for key in operation.keys():
             if key in preprocessed:
-- 
2.44.0

[PATCH net-next v1 3/4] doc: netlink: Fix formatting of op flags in generated .rst

[Donald Hunter]

Generate op flags as an inline list instead of a stringified python
value.

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
---
 tools/net/ynl/ynl-gen-rst.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tools/net/ynl/ynl-gen-rst.py b/tools/net/ynl/ynl-gen-rst.py
index 1096a71d7867..a957725b20dc 100755
--- a/tools/net/ynl/ynl-gen-rst.py
+++ b/tools/net/ynl/ynl-gen-rst.py
@@ -172,7 +172,7 @@ def parse_do_attributes(attrs: Dict[str, Any], level: int = 0) -> str:
 
 def parse_operations(operations: List[Dict[str, Any]], namespace: str) -> str:
     """Parse operations block"""
-    preprocessed = ["name", "doc", "title", "do", "dump"]
+    preprocessed = ["name", "doc", "title", "do", "dump", "flags"]
     linkable = ["fixed-header", "attribute-set"]
     lines = []
 
@@ -188,6 +188,8 @@ def parse_operations(operations: List[Dict[str, Any]], namespace: str) -> str:
             if key in linkable:
                 value = rst_ref(namespace, key, value)
             lines.append(rst_fields(key, value, 0))
+        if 'flags' in operation:
+            lines.append(rst_fields('flags', rst_list_inline(operation['flags'])))
 
         if "do" in operation:
             lines.append(rst_paragraph(":do:", 0))
-- 
2.44.0

[PATCH net-next v1 4/4] doc: netlink: Fix op pre and post fields in generated .rst

[Donald Hunter]

The generated .rst has pre and post headings without any values, e.g.
here:

https://docs.kernel.org/6.9/networking/netlink_spec/dpll.html#device-id-get

Emit keys and values in the generated .rst

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
---
 tools/net/ynl/ynl-gen-rst.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tools/net/ynl/ynl-gen-rst.py b/tools/net/ynl/ynl-gen-rst.py
index a957725b20dc..6c56d0d726b4 100755
--- a/tools/net/ynl/ynl-gen-rst.py
+++ b/tools/net/ynl/ynl-gen-rst.py
@@ -156,7 +156,10 @@ def parse_do(do_dict: Dict[str, Any], level: int = 0) -> str:
     lines = []
     for key in do_dict.keys():
         lines.append(rst_paragraph(bold(key), level + 1))
-        lines.append(parse_do_attributes(do_dict[key], level + 1) + "\n")
+        if key in ['request', 'reply']:
+            lines.append(parse_do_attributes(do_dict[key], level + 1) + "\n")
+        else:
+            lines.append(headroom(level + 2) + do_dict[key] + "\n")
 
     return "\n".join(lines)
 
-- 
2.44.0

Re: [PATCH net-next v1 4/4] doc: netlink: Fix op pre and post fields in generated .rst

[Breno Leitao]

On Tue, May 28, 2024 at 03:06:52PM +0100, Donald Hunter wrote:
> The generated .rst has pre and post headings without any values, e.g.
> here:
> 
> https://docs.kernel.org/6.9/networking/netlink_spec/dpll.html#device-id-get
> 
> Emit keys and values in the generated .rst
> 
> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>

Reviwed-by: Breno Leitao <leitao@debian.org>

Re: [PATCH net-next v1 3/4] doc: netlink: Fix formatting of op flags in generated .rst

[Breno Leitao]

On Tue, May 28, 2024 at 03:06:51PM +0100, Donald Hunter wrote:
> Generate op flags as an inline list instead of a stringified python
> value.
> 
> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
> ---
>  tools/net/ynl/ynl-gen-rst.py | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/net/ynl/ynl-gen-rst.py b/tools/net/ynl/ynl-gen-rst.py
> index 1096a71d7867..a957725b20dc 100755
> --- a/tools/net/ynl/ynl-gen-rst.py
> +++ b/tools/net/ynl/ynl-gen-rst.py
> @@ -172,7 +172,7 @@ def parse_do_attributes(attrs: Dict[str, Any], level: int = 0) -> str:
>  
>  def parse_operations(operations: List[Dict[str, Any]], namespace: str) -> str:
>      """Parse operations block"""
> -    preprocessed = ["name", "doc", "title", "do", "dump"]
> +    preprocessed = ["name", "doc", "title", "do", "dump", "flags"]
>      linkable = ["fixed-header", "attribute-set"]
>      lines = []
>  
> @@ -188,6 +188,8 @@ def parse_operations(operations: List[Dict[str, Any]], namespace: str) -> str:
>              if key in linkable:
>                  value = rst_ref(namespace, key, value)
>              lines.append(rst_fields(key, value, 0))
> +        if 'flags' in operation:

You probably want to use double quotes (") as the other cases ("do" and
"dump"). Other than that:

Reviwed-by: Breno Leitao <leitao@debian.org>

Re: [PATCH net-next v1 1/4] doc: netlink: Fix generated .rst for multi-line docs

[Breno Leitao]

On Tue, May 28, 2024 at 03:06:49PM +0100, Donald Hunter wrote:
> Fix the newline replacement in ynl-gen-rst.py to put spaces between
> concatenated lines. This fixes the broken doc string formatting.
> 
> See the dpll docs for an example of broken concatenation:
> 
> https://docs.kernel.org/6.9/networking/netlink_spec/dpll.html#lock-status
> 
> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>

Reviwed-by: Breno Leitao <leitao@debian.org>

Re: [PATCH net-next v1 2/4] doc: netlink: Don't 'sanitize' op docstrings in generated .rst

[Jakub Kicinski]

On Tue, 28 May 2024 15:06:50 +0100 Donald Hunter wrote:
>        name: pin-get
>        doc: |
>          Get list of pins and its attributes.
> +
>          - dump request without any attributes given - list all the pins in the
>            system
>          - dump request with target dpll - list all the pins registered with

This actually gets rendered as a list now, nice!

Re: [PATCH net-next v1 4/4] doc: netlink: Fix op pre and post fields in generated .rst

[Jakub Kicinski]

On Tue, 28 May 2024 15:06:52 +0100 Donald Hunter wrote:
> The generated .rst has pre and post headings without any values, e.g.
> here:
> 
> https://docs.kernel.org/6.9/networking/netlink_spec/dpll.html#device-id-get
> 
> Emit keys and values in the generated .rst

I think your patch still stands (in case there is more such attrs) but
for pre and post in particular - we can hide them completely. They are
annotations used only by the kernel code gen, there's no need to
display them in the docs.

Re: [PATCH net-next v1 0/4] doc: netlink: Fixes for ynl doc generator

[patchwork-bot+netdevbpf]

Hello:

This series was applied to netdev/net-next.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Tue, 28 May 2024 15:06:48 +0100 you wrote:
> Several fixes to ynl-gen-rst to resolve issues that can be seen
> in:
> 
> https://docs.kernel.org/6.10-rc1/networking/netlink_spec/dpll.html#device-id-get
> https://docs.kernel.org/6.10-rc1/networking/netlink_spec/dpll.html#lock-status
> 
> In patch 2, by not using 'sanitize' for op docs, any formatting in the
> .yaml gets passed straight through to the generated .rst which means
> that basic rst (also markdown compatible) list formatting can be used in
> the .yaml
> 
> [...]

Here is the summary with links:
  - [net-next,v1,1/4] doc: netlink: Fix generated .rst for multi-line docs
    https://git.kernel.org/netdev/net-next/c/c697f515b639
  - [net-next,v1,2/4] doc: netlink: Don't 'sanitize' op docstrings in generated .rst
    https://git.kernel.org/netdev/net-next/c/ebf9004136c7
  - [net-next,v1,3/4] doc: netlink: Fix formatting of op flags in generated .rst
    https://git.kernel.org/netdev/net-next/c/cb7351ac1786
  - [net-next,v1,4/4] doc: netlink: Fix op pre and post fields in generated .rst
    https://git.kernel.org/netdev/net-next/c/9104feed4c64

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

Re: [PATCH net-next v1 4/4] doc: netlink: Fix op pre and post fields in generated .rst

[Donald Hunter]

Jakub Kicinski <kuba@kernel.org> writes:

> On Tue, 28 May 2024 15:06:52 +0100 Donald Hunter wrote:
>> The generated .rst has pre and post headings without any values, e.g.
>> here:
>> 
>> https://docs.kernel.org/6.9/networking/netlink_spec/dpll.html#device-id-get
>> 
>> Emit keys and values in the generated .rst
>
> I think your patch still stands (in case there is more such attrs) but
> for pre and post in particular - we can hide them completely. They are
> annotations used only by the kernel code gen, there's no need to
> display them in the docs.

Yep, I wondered about that. I'll look at suppressing pre and post in a
followup.