Re: [PATCH v8 05/13] docs: sphinx: add a parser for yaml files for Netlink specs

Donald Hunter
2025-06-27T11:04:55Z

Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:

> Add a simple sphinx.Parser to handle yaml files and add the
> the code to handle Netlink specs. All other yaml files are
> ignored.
>
> The code was written in a way that parsing yaml for different
> subsystems and even for different parts of Netlink are easy.
>
> All it takes to have a different parser is to add an
> import line similar to:
>
> from netlink_yml_parser import YnlDocGenerator

This should be: from doc_generator import YnlDocGenerator

> adding the corresponding parser somewhere at the extension:
>
> netlink_parser = YnlDocGenerator()
>
> And then add a logic inside parse() to handle different
> doc outputs, depending on the file location, similar to:
>
> if "/netlink/specs/" in fname:
> msg = self.netlink_parser.parse_yaml_file(fname)
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
> Documentation/sphinx/parser_yaml.py | 104 ++++++++++++++++++++++++++++
> 1 file changed, 104 insertions(+)
> create mode 100755 Documentation/sphinx/parser_yaml.py
>
> diff --git a/Documentation/sphinx/parser_yaml.py b/Documentation/sphinx/parser_yaml.py
> new file mode 100755
> index 000000000000..585a7ec81ba0
> --- /dev/null
> +++ b/Documentation/sphinx/parser_yaml.py
> @@ -0,0 +1,104 @@
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright 2025 Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> +
> +"""
> +Sphinx extension for processing YAML files
> +"""
> +
> +import os
> +import re
> +import sys
> +
> +from pprint import pformat
> +
> +from docutils.parsers.rst import Parser as RSTParser
> +from docutils.statemachine import ViewList
> +
> +from sphinx.util import logging
> +from sphinx.parsers import Parser
> +
> +srctree = os.path.abspath(os.environ["srctree"])
> +sys.path.insert(0, os.path.join(srctree, "tools/net/ynl/pyynl"))

So that it doesn't need to be changed in a later patch, this should be:

... "tools/net/ynl/pyynl/lib"

> +
> +from netlink_yml_parser import YnlDocGenerator # pylint: disable=C0413

So that it doesn't need to be changed in a later patch, this should be:

from doc_generator ...

> +logger = logging.getLogger(__name__)
> +
> +class YamlParser(Parser):
> + """
> + Kernel parser for YAML files.
> +
> + This is a simple sphinx.Parser to handle yaml files inside the
> + Kernel tree that will be part of the built documentation.
> +
> + The actual parser function is not contained here: the code was
> + written in a way that parsing yaml for different subsystems
> + can be done from a single dispatcher.
> +
> + All it takes to have parse YAML patches is to have an import line:
> +
> + from some_parser_code import NewYamlGenerator
> +
> + To this module. Then add an instance of the parser with:
> +
> + new_parser = NewYamlGenerator()
> +
> + and add a logic inside parse() to handle it based on the path,
> + like this:
> +
> + if "/foo" in fname:
> + msg = self.new_parser.parse_yaml_file(fname)
> + """
> +
> + supported = ('yaml', )
> +
> + netlink_parser = YnlDocGenerator()
> +
> + def rst_parse(self, inputstring, document, msg):
> + """
> + Receives a ReST content that was previously converted by the
> + YAML parser, adding it to the document tree.
> + """
> +
> + self.setup_parse(inputstring, document)
> +
> + result = ViewList()
> +
> + try:
> + # Parse message with RSTParser
> + for i, line in enumerate(msg.split('\n')):
> + result.append(line, document.current_source, i)
> +
> + rst_parser = RSTParser()
> + rst_parser.parse('\n'.join(result), document)
> +
> + except Exception as e:
> + document.reporter.error("YAML parsing error: %s" % pformat(e))
> +
> + self.finish_parse()
> +
> + # Overrides docutils.parsers.Parser. See sphinx.parsers.RSTParser
> + def parse(self, inputstring, document):
> + """Check if a YAML is meant to be parsed."""
> +
> + fname = document.current_source
> +
> + # Handle netlink yaml specs
> + if "/netlink/specs/" in fname:
> + msg = self.netlink_parser.parse_yaml_file(fname)
> + self.rst_parse(inputstring, document, msg)
> +
> + # All other yaml files are ignored
> +
> +def setup(app):
> + """Setup function for the Sphinx extension."""
> +
> + # Add YAML parser
> + app.add_source_parser(YamlParser)
> + app.add_source_suffix('.yaml', 'yaml')
> +
> + return {
> + 'version': '1.0',
> + 'parallel_read_safe': True,
> + 'parallel_write_safe': True,
> + }