[GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

[GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

[Mauro Carvalho Chehab]

Hi Jon/Jakub,

In case you both prefer to merge from a stable tag, please pull from:

	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1

For:

- An YAML parser Sphinx plugin, integrated with Netlink YAML doc
  parser.

The patch content is identical to my v10 submission:

	https://lore.kernel.org/linux-doc/cover.1753718185.git.mchehab+huawei@kernel.org/

The tag was rebased on the top of v6.17-rc1 to make easier for it to be
merged on both docs and netlink trees. 

No code changes since v10.

Regards,
Mauro

---

The following changes since commit 8f5ae30d69d7543eee0d70083daf4de8fe15d585:

  Linux 6.17-rc1 (2025-08-10 19:41:16 +0300)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git tags/docs/v6.17-1

for you to fetch changes up to 47459937be8031aae6aaa17ac5f60985f7c9e1bd:

  sphinx: parser_yaml.py: fix line numbers information (2025-08-12 07:47:31 +0200)

----------------------------------------------------------------
[GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

----------------------------------------------------------------
Mauro Carvalho Chehab (14):
      docs: netlink: netlink-raw.rst: use :ref: instead of :doc:
      tools: ynl_gen_rst.py: Split library from command line tool
      docs: netlink: index.rst: add a netlink index file
      tools: ynl_gen_rst.py: cleanup coding style
      docs: sphinx: add a parser for yaml files for Netlink specs
      docs: use parser_yaml extension to handle Netlink specs
      docs: uapi: netlink: update netlink specs link
      tools: ynl_gen_rst.py: drop support for generating index files
      docs: netlink: remove obsolete .gitignore from unused directory
      MAINTAINERS: add netlink_yml_parser.py to linux-doc
      tools: netlink_yml_parser.py: add line numbers to parsed data
      docs: parser_yaml.py: add support for line numbers from the parser
      docs: parser_yaml.py: fix backward compatibility with old docutils
      sphinx: parser_yaml.py: fix line numbers information

 Documentation/Makefile                             |  17 -
 Documentation/conf.py                              |  20 +-
 Documentation/netlink/specs/index.rst              |  13 +
 Documentation/networking/index.rst                 |   2 +-
 Documentation/networking/netlink_spec/.gitignore   |   1 -
 Documentation/networking/netlink_spec/readme.txt   |   4 -
 Documentation/sphinx/parser_yaml.py                | 123 +++++++
 Documentation/userspace-api/netlink/index.rst      |   2 +-
 .../userspace-api/netlink/netlink-raw.rst          |   6 +-
 Documentation/userspace-api/netlink/specs.rst      |   2 +-
 MAINTAINERS                                        |   1 +
 tools/net/ynl/pyynl/lib/__init__.py                |   2 +
 tools/net/ynl/pyynl/lib/doc_generator.py           | 398 +++++++++++++++++++++
 tools/net/ynl/pyynl/ynl_gen_rst.py                 | 384 +-------------------
 14 files changed, 565 insertions(+), 410 deletions(-)
 create mode 100644 Documentation/netlink/specs/index.rst
 delete mode 100644 Documentation/networking/netlink_spec/.gitignore
 delete mode 100644 Documentation/networking/netlink_spec/readme.txt
 create mode 100755 Documentation/sphinx/parser_yaml.py
 create mode 100644 tools/net/ynl/pyynl/lib/doc_generator.py

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

[Jakub Kicinski]

On Tue, 12 Aug 2025 11:33:29 +0200 Mauro Carvalho Chehab wrote:
> The patch content is identical to my v10 submission:
> 
> 	https://lore.kernel.org/linux-doc/cover.1753718185.git.mchehab+huawei@kernel.org/

Thank you! We'll merge on Thu, FWIW, after we fast forward to Linus.

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

[Jonathan Corbet]

Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:

> Hi Jon/Jakub,
>
> In case you both prefer to merge from a stable tag, please pull from:
>
> 	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1
>
> For:
>
> - An YAML parser Sphinx plugin, integrated with Netlink YAML doc
>   parser.

OK, I have done that.  I will note that it adds a warning:

> Documentation/networking/netlink_spec/index.rst: WARNING: document isn't included in any toctree

...it might be nice to get that straightened out.

Thanks,

jon

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

[Randy Dunlap]

On 8/12/25 11:31 AM, Jonathan Corbet wrote:
> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
>> Hi Jon/Jakub,
>>
>> In case you both prefer to merge from a stable tag, please pull from:
>>
>> 	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1
>>
>> For:
>>
>> - An YAML parser Sphinx plugin, integrated with Netlink YAML doc
>>   parser.
> 
> OK, I have done that.  I will note that it adds a warning:
> 
>> Documentation/networking/netlink_spec/index.rst: WARNING: document isn't included in any toctree
> 
> ...it might be nice to get that straightened out.

I see it, at least in linux-next. However, its format is
"different," so that may have confused whatever printed
that message:

from Documentation/networking/index.rst:

   filter
   generic-hdlc
   generic_netlink
   netlink_spec/index
   gen_stats
   gtp
   ila


-- 
~Randy

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

[Mauro Carvalho Chehab]

On Tue, 12 Aug 2025 11:41:58 -0700
Randy Dunlap <rdunlap@infradead.org> wrote:

> On 8/12/25 11:31 AM, Jonathan Corbet wrote:
> > Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> >   
> >> Hi Jon/Jakub,
> >>
> >> In case you both prefer to merge from a stable tag, please pull from:
> >>
> >> 	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1
> >>
> >> For:
> >>
> >> - An YAML parser Sphinx plugin, integrated with Netlink YAML doc
> >>   parser.  
> > 
> > OK, I have done that.  I will note that it adds a warning:
> >   
> >> Documentation/networking/netlink_spec/index.rst: WARNING: document isn't included in any toctree  
> > ...it might be nice to get that straightened out.  
> 
> I see it, at least in linux-next. However, its format is
> "different," so that may have confused whatever printed
> that message:
> 
> from Documentation/networking/index.rst:
> 
>    filter
>    generic-hdlc
>    generic_netlink
>    netlink_spec/index
>    gen_stats
>    gtp
>    ila

Maybe some merge conflict/merge issue, as this was renamed:

$ git show 1ce4da3dd99e98bd4a8b396c291041080e0fe85e Documentation/networking/index.rst
commit 1ce4da3dd99e98bd4a8b396c291041080e0fe85e
Author: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Date:   Thu Jun 12 10:34:30 2025 +0200

    docs: use parser_yaml extension to handle Netlink specs
    
    Instead of manually calling ynl_gen_rst.py, use a Sphinx extension.
    This way, no .rst files would be written to the Kernel source
    directories.
    
    We are using here a toctree with :glob: property. This way, there
    is no need to touch the netlink/specs/index.rst file every time
    a new Netlink spec is added/renamed/removed.
    
    Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
    Reviewed-by: Donald Hunter <donald.hunter@gmail.com>

diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst
index ac90b82f3ce9..b7a4969e9bc9 100644
--- a/Documentation/networking/index.rst
+++ b/Documentation/networking/index.rst
@@ -57,7 +57,7 @@ Contents:
    filter
    generic-hdlc
    generic_netlink
-   netlink_spec/index
+   ../netlink/specs/index
    gen_stats
    gtp
    ila

Thanks,
Mauro

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

[Akira Yokosawa]

[-CC: LKMM folks and list; this has nothing to do with the memory model]

Hi Jon,

On Tue, 12 Aug 2025 12:31:03 -0600, Jonathan Corbet wrote:
> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
>> Hi Jon/Jakub,
>>
>> In case you both prefer to merge from a stable tag, please pull from:
>>
>> 	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1
>>
>> For:
>>
>> - An YAML parser Sphinx plugin, integrated with Netlink YAML doc
>>   parser.
> 
> OK, I have done that.  I will note that it adds a warning:
> 
>> Documentation/networking/netlink_spec/index.rst: WARNING: document isn't included in any toctree
> 
> ...it might be nice to get that straightened out.

After the merge, "git status" complains:

    Untracked files:
      (use "git add <file>..." to include in what will be committed)
	Documentation/networking/netlink_spec/

So, I don't think there is anything you can do in the Git repo side ...

We need to remember to "rm -rf" the directory after crossing this merge
point.

In theory, such "rm -rf" could be added somewhere in Documentation/Makefile,
but that would not work well with write-protected shared kernel repos.

        Thanks, Akira

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

[Mauro Carvalho Chehab]

On Wed, 13 Aug 2025 11:49:50 +0900
Akira Yokosawa <akiyks@gmail.com> wrote:

> [-CC: LKMM folks and list; this has nothing to do with the memory model]
> 
> Hi Jon,
> 
> On Tue, 12 Aug 2025 12:31:03 -0600, Jonathan Corbet wrote:
> > Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> >   
> >> Hi Jon/Jakub,
> >>
> >> In case you both prefer to merge from a stable tag, please pull from:
> >>
> >> 	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1
> >>
> >> For:
> >>
> >> - An YAML parser Sphinx plugin, integrated with Netlink YAML doc
> >>   parser.  
> > 
> > OK, I have done that.  I will note that it adds a warning:
> >   
> >> Documentation/networking/netlink_spec/index.rst: WARNING: document isn't included in any toctree  
> > 
> > ...it might be nice to get that straightened out.  
> 
> After the merge, "git status" complains:
> 
>     Untracked files:
>       (use "git add <file>..." to include in what will be committed)
> 	Documentation/networking/netlink_spec/
> 
> So, I don't think there is anything you can do in the Git repo side ...
> 
> We need to remember to "rm -rf" the directory after crossing this merge
> point.
> 
> In theory, such "rm -rf" could be added somewhere in Documentation/Makefile,
> but that would not work well with write-protected shared kernel repos.

And this is actually what started this patch series: the original
approach of auto-generating and writing files under
Documentation/networking/netlink_spec/ is problematic: a patch
that renamed some files caused the past version of the generated
files to generate warnings.

So, yeah, with the old approach one needs to manually clean up
Documentation/networking/netlink_spec/ if this was not the first
time doc were built on a git clone instance.

Btw, before my patch series, there was the cleandocs target
was:

	YNL_INDEX:=$(srctree)/Documentation/networking/netlink_spec/index.rst
	YNL_RST_DIR:=$(srctree)/Documentation/networking/netlink_spec
	YNL_YAML_DIR:=$(srctree)/Documentation/netlink/specs
	YNL_RST_FILES_TMP := $(patsubst %.yaml,%.rst,$(wildcard $(YNL_YAML_DIR)/*.yaml))
	YNL_RST_FILES := $(patsubst $(YNL_YAML_DIR)%,$(YNL_RST_DIR)%, $(YNL_RST_FILES_TMP))

	cleandocs:
		$(Q)rm -f $(YNL_INDEX) $(YNL_RST_FILES)
		$(Q)rm -rf $(BUILDDIR)
		$(Q)$(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/userspace-api/media clean

It means that, with the old approach, the safe way to apply
patches that touch YAML Netlink (YNL) is to do:

	make cleandocs			# remove old auto-generated patches from /Documentation/networking/netlink_spec
	<apply new series>
	make htmldocs

to ensure that the previous auto-generated files were removed.

Thanks,
Mauro

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

[patchwork-bot+netdevbpf]

Hello:

This pull request was applied to netdev/net-next.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Tue, 12 Aug 2025 11:33:29 +0200 you wrote:
> Hi Jon/Jakub,
> 
> In case you both prefer to merge from a stable tag, please pull from:
> 
> 	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1
> 
> For:
> 
> [...]

Here is the summary with links:
  - [GIT,PULL,for,v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation
    https://git.kernel.org/netdev/net-next/c/c4f72d3747ac

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html