[PATCH 00/13] Add kernel-doc modules to Documentation/tools

[PATCH 00/13] Add kernel-doc modules to Documentation/tools

[Mauro Carvalho Chehab]

Hi Jon,

This is the splitted version of my RFC to use sphinx.ext.autodoc
to document kernel-doc modules. It documents only kernel-doc
modules, but, once we merge it, we can document other things.

This series comes after:
    https://lore.kernel.org/linux-doc/cover.1768395332.git.mchehab+huawei@kernel.org/T/#t

patch 1 is actually not directly related: if fixes a bug I noticed
on Sphinx indexes with a large maxdepth;

patch 2 adds sphinx.ext.autodoc and place the directories where
python modules can be found. It doesn't use the extension, though.
This will happen only at the final patch;

patches 3 to 12 are basically documentation cleanups and fixes that
ensure that the documentation will be complete.

patch 13 creates new *.rst files that points to python.lib.kdoc
files.

With this version, python files inside tools/ or scripts/
can be documented, but there is a catch (probably due to PEP8):

    - files must end with ".py"
    - file names can't have "-".

So, unfortunately, we can't document kernel-doc.py, except if
we rename it to kernel_doc.py.

As I mentioned at RFC, we could use:

    $ sphinx-apidoc scripts tools -o foobar
    $ sphinx-apidoc tools/docs -o foobar
    ...

to generate .rst skeletons with the modules to be added, but
manual work is needed afterwards.

Mauro Carvalho Chehab (13):
  docs: custom.css: prevent li marker to override text
  docs: enable Sphinx autodoc extension to allow documenting python
  docs: custom.css: add CSS for python
  docs: kdoc: latex_fonts: Improve docstrings and comments
  docs: kdoc_files: Improve docstrings and comments
  docs: kdoc_item: Improve docstrings and comments
  docs: kdoc_parser: Improve docstrings and comments
  docs: kdoc_output: Improve docstrings and comments
  docs: kdoc_re: Improve docstrings and comments
  docs: kdoc: parse_data_structs: Improve docstrings and comments
  docs: kdoc: enrich_formatter: Improve docstrings and comments
  docs: kdoc: python_version: Improve docstrings and comments
  docs: add kernel-doc modules documentation

 Documentation/conf.py                       |  11 +-
 Documentation/sphinx-static/custom.css      |  12 ++
 Documentation/tools/index.rst               |   1 +
 Documentation/tools/kdoc.rst                |  12 ++
 Documentation/tools/kdoc_ancillary.rst      |  46 +++++
 Documentation/tools/kdoc_output.rst         |  14 ++
 Documentation/tools/kdoc_parser.rst         |  29 ++++
 Documentation/tools/python.rst              |  10 ++
 tools/lib/python/kdoc/enrich_formatter.py   |  20 ++-
 tools/lib/python/kdoc/kdoc_files.py         |  23 +--
 tools/lib/python/kdoc/kdoc_item.py          |  18 ++
 tools/lib/python/kdoc/kdoc_output.py        |  60 ++++---
 tools/lib/python/kdoc/kdoc_parser.py        | 175 +++++++++++---------
 tools/lib/python/kdoc/kdoc_re.py            |  18 +-
 tools/lib/python/kdoc/latex_fonts.py        |  95 ++++++-----
 tools/lib/python/kdoc/parse_data_structs.py |  62 ++++---
 tools/lib/python/kdoc/python_version.py     |  20 ++-
 17 files changed, 430 insertions(+), 196 deletions(-)
 create mode 100644 Documentation/tools/kdoc.rst
 create mode 100644 Documentation/tools/kdoc_ancillary.rst
 create mode 100644 Documentation/tools/kdoc_output.rst
 create mode 100644 Documentation/tools/kdoc_parser.rst
 create mode 100644 Documentation/tools/python.rst

-- 
2.52.0

[PATCH 01/13] docs: custom.css: prevent li marker to override text

[Mauro Carvalho Chehab]

There's currently an issue with li marker: it is set to use
-1em, which actually makes it override the text. This is visible
on indexes that are deep enough.

Fix it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/sphinx-static/custom.css | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Documentation/sphinx-static/custom.css b/Documentation/sphinx-static/custom.css
index 06cedbae095c..b6a7a5f6b6d4 100644
--- a/Documentation/sphinx-static/custom.css
+++ b/Documentation/sphinx-static/custom.css
@@ -30,6 +30,9 @@ img.logo {
     margin-bottom: 20px;
 }
 
+/* The default is to use -1em, wich makes it override text */
+li { text-indent: 0em; }
+
 /*
  * Parameters for the display of function prototypes and such included
  * from C source files.
-- 
2.52.0

[PATCH 02/13] docs: enable Sphinx autodoc extension to allow documenting python

[Mauro Carvalho Chehab]

Adding python documentation is simple with Sphinx: all we need
is to include the ext.autodoc extension and add the directories
where the Python code sits at the sys.path.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/conf.py | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/Documentation/conf.py b/Documentation/conf.py
index 1ea2ae5c6276..429fcc9fd7f7 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -13,11 +13,18 @@ from  textwrap import dedent
 
 import sphinx
 
+# Location of Documentation/ directory
+doctree = os.path.abspath(".")
+
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 sys.path.insert(0, os.path.abspath("sphinx"))
 
+# Allow sphinx.ext.autodoc to document from tools and scripts
+sys.path.append(f"{doctree}/../tools")
+sys.path.append(f"{doctree}/../scripts")
+
 # Minimal supported version
 needs_sphinx = "3.4.3"
 
@@ -32,9 +39,6 @@ else:
     # Include patterns that don't contain directory names, in glob format
     include_patterns = ["**.rst"]
 
-# Location of Documentation/ directory
-doctree = os.path.abspath(".")
-
 # Exclude of patterns that don't contain directory names, in glob format.
 exclude_patterns = []
 
@@ -151,6 +155,7 @@ extensions = [
     "maintainers_include",
     "parser_yaml",
     "rstFlatTable",
+    "sphinx.ext.autodoc",
     "sphinx.ext.autosectionlabel",
     "sphinx.ext.ifconfig",
     "translations",
-- 
2.52.0

[PATCH 03/13] docs: custom.css: add CSS for python

[Mauro Carvalho Chehab]

As we'll start adding python to documentation, add some CSS
templates to better display python code.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/sphinx-static/custom.css | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/Documentation/sphinx-static/custom.css b/Documentation/sphinx-static/custom.css
index b6a7a5f6b6d4..794ed2ac8a48 100644
--- a/Documentation/sphinx-static/custom.css
+++ b/Documentation/sphinx-static/custom.css
@@ -43,6 +43,15 @@ dl.function dt { margin-left: 10em; text-indent: -10em; }
 dt.sig-object { font-size: larger; }
 div.kernelindent { margin-left: 2em; margin-right: 4em; }
 
+/*
+ * Parameters for the display of function prototypes and such included
+ * from Python source files.
+ */
+dl.py { margin-top: 2em; background-color: #ecf0f3; }
+dl.py.class { margin-left: 2em; text-indent: -2em; padding-left: 2em; }
+dl.py.method, dl.py.attribute { margin-left: 2em; text-indent: -2em; }
+dl.py li, pre { text-indent: 0em; padding-left: 0 !important; }
+
 /*
  * Tweaks for our local TOC
  */
-- 
2.52.0

[PATCH 04/13] docs: kdoc: latex_fonts: Improve docstrings and comments

[Mauro Carvalho Chehab]

In preparation to document kernel-doc module, improve its
documentation.

Among the changes, it had to place the xml template inside
a code block, as otherwise doc build would break.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/lib/python/kdoc/latex_fonts.py | 95 ++++++++++++++++------------
 1 file changed, 56 insertions(+), 39 deletions(-)

diff --git a/tools/lib/python/kdoc/latex_fonts.py b/tools/lib/python/kdoc/latex_fonts.py
index 29317f8006ea..1d04cbda169f 100755
--- a/tools/lib/python/kdoc/latex_fonts.py
+++ b/tools/lib/python/kdoc/latex_fonts.py
@@ -5,12 +5,13 @@
 # Ported to Python by (c) Mauro Carvalho Chehab, 2025
 
 """
-Detect problematic Noto CJK variable fonts.
+Detect problematic Noto CJK variable fonts
+==========================================
 
-For "make pdfdocs", reports of build errors of translations.pdf started
-arriving early 2024 [1, 2].  It turned out that Fedora and openSUSE
-tumbleweed have started deploying variable-font [3] format of "Noto CJK"
-fonts [4, 5].  For PDF, a LaTeX package named xeCJK is used for CJK
+For ``make pdfdocs``, reports of build errors of translations.pdf started
+arriving early 2024 [1]_ [2]_.  It turned out that Fedora and openSUSE
+tumbleweed have started deploying variable-font [3]_ format of "Noto CJK"
+fonts [4]_ [5]_.  For PDF, a LaTeX package named xeCJK is used for CJK
 (Chinese, Japanese, Korean) pages.  xeCJK requires XeLaTeX/XeTeX, which
 does not (and likely never will) understand variable fonts for historical
 reasons.
@@ -25,68 +26,77 @@ This script is invoked from the error path of "make pdfdocs" and emits
 suggestions if variable-font files of "Noto CJK" fonts are in the list of
 fonts accessible from XeTeX.
 
-References:
-[1]: https://lore.kernel.org/r/8734tqsrt7.fsf@meer.lwn.net/
-[2]: https://lore.kernel.org/r/1708585803.600323099@f111.i.mail.ru/
-[3]: https://en.wikipedia.org/wiki/Variable_font
-[4]: https://fedoraproject.org/wiki/Changes/Noto_CJK_Variable_Fonts
-[5]: https://build.opensuse.org/request/show/1157217
+.. [1] https://lore.kernel.org/r/8734tqsrt7.fsf@meer.lwn.net/
+.. [2] https://lore.kernel.org/r/1708585803.600323099@f111.i.mail.ru/
+.. [3] https://en.wikipedia.org/wiki/Variable_font
+.. [4] https://fedoraproject.org/wiki/Changes/Noto_CJK_Variable_Fonts
+.. [5] https://build.opensuse.org/request/show/1157217
 
-#===========================================================================
 Workarounds for building translations.pdf
-#===========================================================================
+-----------------------------------------
 
 * Denylist "variable font" Noto CJK fonts.
+
   - Create $HOME/deny-vf/fontconfig/fonts.conf from template below, with
     tweaks if necessary.  Remove leading "".
+
   - Path of fontconfig/fonts.conf can be overridden by setting an env
     variable FONTS_CONF_DENY_VF.
 
-    * Template:
------------------------------------------------------------------
-<?xml version="1.0"?>
-<!DOCTYPE fontconfig SYSTEM "urn:fontconfig:fonts.dtd">
-<fontconfig>
-<!--
-  Ignore variable-font glob (not to break xetex)
--->
-    <selectfont>
-        <rejectfont>
-            <!--
-                for Fedora
-            -->
-            <glob>/usr/share/fonts/google-noto-*-cjk-vf-fonts</glob>
-            <!--
-                for openSUSE tumbleweed
-            -->
-            <glob>/usr/share/fonts/truetype/Noto*CJK*-VF.otf</glob>
-        </rejectfont>
-    </selectfont>
-</fontconfig>
------------------------------------------------------------------
+    * Template::
+
+        <?xml version="1.0"?>
+        <!DOCTYPE fontconfig SYSTEM "urn:fontconfig:fonts.dtd">
+        <fontconfig>
+        <!--
+        Ignore variable-font glob (not to break xetex)
+        -->
+            <selectfont>
+                <rejectfont>
+                    <!--
+                        for Fedora
+                    -->
+                    <glob>/usr/share/fonts/google-noto-*-cjk-vf-fonts</glob>
+                    <!--
+                        for openSUSE tumbleweed
+                    -->
+                    <glob>/usr/share/fonts/truetype/Noto*CJK*-VF.otf</glob>
+                </rejectfont>
+            </selectfont>
+        </fontconfig>
 
     The denylisting is activated for "make pdfdocs".
 
 * For skipping CJK pages in PDF
+
   - Uninstall texlive-xecjk.
     Denylisting is not needed in this case.
 
 * For printing CJK pages in PDF
+
   - Need non-variable "Noto CJK" fonts.
+
     * Fedora
+
       - google-noto-sans-cjk-fonts
       - google-noto-serif-cjk-fonts
+
     * openSUSE tumbleweed
+
       - Non-variable "Noto CJK" fonts are not available as distro packages
         as of April, 2024.  Fetch a set of font files from upstream Noto
         CJK Font released at:
+
           https://github.com/notofonts/noto-cjk/tree/main/Sans#super-otc
+
         and at:
+
           https://github.com/notofonts/noto-cjk/tree/main/Serif#super-otc
-        , then uncompress and deploy them.
+
+        then uncompress and deploy them.
       - Remember to update fontconfig cache by running fc-cache.
 
-!!! Caution !!!
+.. caution::
     Uninstalling "variable font" packages can be dangerous.
     They might be depended upon by other packages important for your work.
     Denylisting should be less invasive, as it is effective only while
@@ -115,10 +125,15 @@ class LatexFontChecker:
         self.re_cjk = re.compile(r"([^:]+):\s*Noto\s+(Sans|Sans Mono|Serif) CJK")
 
     def description(self):
+        """
+        Returns module description.
+        """
         return __doc__
 
     def get_noto_cjk_vf_fonts(self):
-        """Get Noto CJK fonts"""
+        """
+        Get Noto CJK fonts.
+        """
 
         cjk_fonts = set()
         cmd = ["fc-list", ":", "file", "family", "variable"]
@@ -143,7 +158,9 @@ class LatexFontChecker:
         return sorted(cjk_fonts)
 
     def check(self):
-        """Check for problems with CJK fonts"""
+        """
+        Check for problems with CJK fonts.
+        """
 
         fonts = textwrap.indent("\n".join(self.get_noto_cjk_vf_fonts()), "    ")
         if not fonts:
-- 
2.52.0

[PATCH 05/13] docs: kdoc_files: Improve docstrings and comments

[Mauro Carvalho Chehab]

In preparation to document kernel-doc module, improve its
documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/lib/python/kdoc/kdoc_files.py | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)

diff --git a/tools/lib/python/kdoc/kdoc_files.py b/tools/lib/python/kdoc/kdoc_files.py
index bfe02baf1606..022487ea2cc6 100644
--- a/tools/lib/python/kdoc/kdoc_files.py
+++ b/tools/lib/python/kdoc/kdoc_files.py
@@ -5,7 +5,8 @@
 # pylint: disable=R0903,R0913,R0914,R0917
 
 """
-Parse lernel-doc tags on multiple kernel source files.
+Classes for navigating through the files that kernel-doc needs to handle
+to generate documentation.
 """
 
 import argparse
@@ -43,7 +44,7 @@ class GlobSourceFiles:
         self.srctree = srctree
 
     def _parse_dir(self, dirname):
-        """Internal function to parse files recursively"""
+        """Internal function to parse files recursively."""
 
         with os.scandir(dirname) as obj:
             for entry in obj:
@@ -65,7 +66,7 @@ class GlobSourceFiles:
     def parse_files(self, file_list, file_not_found_cb):
         """
         Define an iterator to parse all source files from file_list,
-        handling directories if any
+        handling directories if any.
         """
 
         if not file_list:
@@ -91,18 +92,18 @@ class KernelFiles():
 
     There are two type of parsers defined here:
         - self.parse_file(): parses both kernel-doc markups and
-          EXPORT_SYMBOL* macros;
-        - self.process_export_file(): parses only EXPORT_SYMBOL* macros.
+          ``EXPORT_SYMBOL*`` macros;
+        - self.process_export_file(): parses only ``EXPORT_SYMBOL*`` macros.
     """
 
     def warning(self, msg):
-        """Ancillary routine to output a warning and increment error count"""
+        """Ancillary routine to output a warning and increment error count."""
 
         self.config.log.warning(msg)
         self.errors += 1
 
     def error(self, msg):
-        """Ancillary routine to output an error and increment error count"""
+        """Ancillary routine to output an error and increment error count."""
 
         self.config.log.error(msg)
         self.errors += 1
@@ -128,7 +129,7 @@ class KernelFiles():
 
     def process_export_file(self, fname):
         """
-        Parses EXPORT_SYMBOL* macros from a single Kernel source file.
+        Parses ``EXPORT_SYMBOL*`` macros from a single Kernel source file.
         """
 
         # Prevent parsing the same file twice if results are cached
@@ -157,7 +158,7 @@ class KernelFiles():
                  wcontents_before_sections=False,
                  logger=None):
         """
-        Initialize startup variables and parse all files
+        Initialize startup variables and parse all files.
         """
 
         if not verbose:
@@ -213,7 +214,7 @@ class KernelFiles():
 
     def parse(self, file_list, export_file=None):
         """
-        Parse all files
+        Parse all files.
         """
 
         glob = GlobSourceFiles(srctree=self.config.src_tree)
@@ -242,7 +243,7 @@ class KernelFiles():
             filenames=None, export_file=None):
         """
         Interacts over the kernel-doc results and output messages,
-        returning kernel-doc markups on each interaction
+        returning kernel-doc markups on each interaction.
         """
 
         self.out_style.set_config(self.config)
-- 
2.52.0

[PATCH 06/13] docs: kdoc_item: Improve docstrings and comments

[Mauro Carvalho Chehab]

In preparation to document kernel-doc module, improve its
documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/lib/python/kdoc/kdoc_item.py | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/tools/lib/python/kdoc/kdoc_item.py b/tools/lib/python/kdoc/kdoc_item.py
index 19805301cb2c..2b8a93f79716 100644
--- a/tools/lib/python/kdoc/kdoc_item.py
+++ b/tools/lib/python/kdoc/kdoc_item.py
@@ -4,7 +4,16 @@
 # then pass into the output modules.
 #
 
+"""
+Data class to store a kernel-doc Item.
+"""
+
 class KdocItem:
+    """
+    A class that will, eventually, encapsulate all of the parsed data that we
+    then pass into the output modules.
+    """
+
     def __init__(self, name, fname, type, start_line, **other_stuff):
         self.name = name
         self.fname = fname
@@ -24,6 +33,9 @@ class KdocItem:
         self.other_stuff = other_stuff
 
     def get(self, key, default = None):
+        """
+        Get a value from optional keys.
+        """
         return self.other_stuff.get(key, default)
 
     def __getitem__(self, key):
@@ -33,10 +45,16 @@ class KdocItem:
     # Tracking of section and parameter information.
     #
     def set_sections(self, sections, start_lines):
+        """
+        Set sections and start lines.
+        """
         self.sections = sections
         self.section_start_lines = start_lines
 
     def set_params(self, names, descs, types, starts):
+        """
+        Set parameter list: names, descriptions, types and start lines.
+        """
         self.parameterlist = names
         self.parameterdescs = descs
         self.parametertypes = types
-- 
2.52.0

[PATCH 07/13] docs: kdoc_parser: Improve docstrings and comments

[Mauro Carvalho Chehab]

In preparation to document kernel-doc module, improve its
documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/lib/python/kdoc/kdoc_parser.py | 175 +++++++++++++++------------
 1 file changed, 96 insertions(+), 79 deletions(-)

diff --git a/tools/lib/python/kdoc/kdoc_parser.py b/tools/lib/python/kdoc/kdoc_parser.py
index 4ad7ce0b243e..fd57944ae907 100644
--- a/tools/lib/python/kdoc/kdoc_parser.py
+++ b/tools/lib/python/kdoc/kdoc_parser.py
@@ -5,11 +5,8 @@
 # pylint: disable=C0301,C0302,R0904,R0912,R0913,R0914,R0915,R0917,R1702
 
 """
-kdoc_parser
-===========
-
-Read a C language source or header FILE and extract embedded
-documentation comments
+Classes and functions related to reading a C language source or header FILE
+and extract embedded documentation comments from it.
 """
 
 import sys
@@ -195,25 +192,28 @@ function_xforms  = [
 ]
 
 #
-# Apply a set of transforms to a block of text.
+# Ancillary functions
 #
+
 def apply_transforms(xforms, text):
+    """
+    Apply a set of transforms to a block of text.
+    """
     for search, subst in xforms:
         text = search.sub(subst, text)
     return text
 
-#
-# A little helper to get rid of excess white space
-#
 multi_space = KernRe(r'\s\s+')
 def trim_whitespace(s):
+    """
+    A little helper to get rid of excess white space.
+    """
     return multi_space.sub(' ', s.strip())
 
-#
-# Remove struct/enum members that have been marked "private".
-#
 def trim_private_members(text):
-    #
+    """
+    Remove ``struct``/``enum`` members that have been marked "private".
+    """
     # First look for a "public:" block that ends a private region, then
     # handle the "private until the end" case.
     #
@@ -226,20 +226,21 @@ def trim_private_members(text):
 
 class state:
     """
-    State machine enums
+    States used by the parser's state machine.
     """
 
     # Parser states
-    NORMAL        = 0        # normal code
-    NAME          = 1        # looking for function name
-    DECLARATION   = 2        # We have seen a declaration which might not be done
-    BODY          = 3        # the body of the comment
-    SPECIAL_SECTION = 4      # doc section ending with a blank line
-    PROTO         = 5        # scanning prototype
-    DOCBLOCK      = 6        # documentation block
-    INLINE_NAME   = 7        # gathering doc outside main block
-    INLINE_TEXT   = 8	     # reading the body of inline docs
+    NORMAL        = 0        #: Normal code.
+    NAME          = 1        #: Looking for function name.
+    DECLARATION   = 2        #: We have seen a declaration which might not be done.
+    BODY          = 3        #: The body of the comment.
+    SPECIAL_SECTION = 4      #: Doc section ending with a blank line.
+    PROTO         = 5        #: Scanning prototype.
+    DOCBLOCK      = 6        #: Documentation block.
+    INLINE_NAME   = 7        #: Gathering doc outside main block.
+    INLINE_TEXT   = 8	     #: Reading the body of inline docs.
 
+    #: Names for each parser state.
     name = [
         "NORMAL",
         "NAME",
@@ -253,9 +254,12 @@ class state:
     ]
 
 
-SECTION_DEFAULT = "Description"  # default section
+SECTION_DEFAULT = "Description"  #: Default section.
 
 class KernelEntry:
+    """
+    Encapsulates a Kernel documentation entry.
+    """
 
     def __init__(self, config, fname, ln):
         self.config = config
@@ -288,14 +292,16 @@ class KernelEntry:
     # Management of section contents
     #
     def add_text(self, text):
+        """Add a new text to the entry contents list."""
         self._contents.append(text)
 
     def contents(self):
+        """Returns a string with all content texts that were added."""
         return '\n'.join(self._contents) + '\n'
 
     # TODO: rename to emit_message after removal of kernel-doc.pl
     def emit_msg(self, ln, msg, *, warning=True):
-        """Emit a message"""
+        """Emit a message."""
 
         log_msg = f"{self.fname}:{ln} {msg}"
 
@@ -309,10 +315,10 @@ class KernelEntry:
         self.warnings.append(log_msg)
         return
 
-    #
-    # Begin a new section.
-    #
     def begin_section(self, line_no, title = SECTION_DEFAULT, dump = False):
+        """
+        Begin a new section.
+        """
         if dump:
             self.dump_section(start_new = True)
         self.section = title
@@ -366,11 +372,13 @@ class KernelDoc:
     documentation comments.
     """
 
-    # Section names
-
+    #: Name of context section.
     section_context = "Context"
+
+    #: Name of return section.
     section_return = "Return"
 
+    #: String to write when a parameter is not described.
     undescribed = "-- undescribed --"
 
     def __init__(self, config, fname):
@@ -416,7 +424,7 @@ class KernelDoc:
 
     def dump_section(self, start_new=True):
         """
-        Dumps section contents to arrays/hashes intended for that purpose.
+        Dump section contents to arrays/hashes intended for that purpose.
         """
 
         if self.entry:
@@ -425,9 +433,9 @@ class KernelDoc:
     # TODO: rename it to store_declaration after removal of kernel-doc.pl
     def output_declaration(self, dtype, name, **args):
         """
-        Stores the entry into an entry array.
+        Store the entry into an entry array.
 
-        The actual output and output filters will be handled elsewhere
+        The actual output and output filters will be handled elsewhere.
         """
 
         item = KdocItem(name, self.fname, dtype,
@@ -456,7 +464,9 @@ class KernelDoc:
 
         Ensure that those warnings are not lost.
 
-        NOTE: Because we are calling `config.warning()` here, those
+        .. note::
+
+              Because we are calling `config.warning()` here, those
               warnings are not filtered by the `-W` parameters: they will all
               be produced even when `-Wreturn`, `-Wshort-desc`, and/or
               `-Wcontents-before-sections` are used.
@@ -680,10 +690,12 @@ class KernelDoc:
             self.emit_msg(ln,
                           f"No description found for return value of '{declaration_name}'")
 
-    #
-    # Split apart a structure prototype; returns (struct|union, name, members) or None
-    #
     def split_struct_proto(self, proto):
+        """
+        Split apart a structure prototype; returns (struct|union, name,
+        members) or ``None``.
+        """
+
         type_pattern = r'(struct|union)'
         qualifiers = [
             "__attribute__",
@@ -702,21 +714,26 @@ class KernelDoc:
             if r.search(proto):
                 return (r.group(1), r.group(3), r.group(2))
         return None
-    #
-    # Rewrite the members of a structure or union for easier formatting later on.
-    # Among other things, this function will turn a member like:
-    #
-    #  struct { inner_members; } foo;
-    #
-    # into:
-    #
-    #  struct foo; inner_members;
-    #
+
     def rewrite_struct_members(self, members):
+        """
+        Process ``struct``/``union`` members from the most deeply nested
+        outward.
+
+        Rewrite the members of a ``struct`` or ``union`` for easier formatting
+        later on. Among other things, this function will turn a member like::
+
+          struct { inner_members; } foo;
+
+        into::
+
+          struct foo; inner_members;
+        """
+
         #
-        # Process struct/union members from the most deeply nested outward.  The
-        # trick is in the ^{ below - it prevents a match of an outer struct/union
-        # until the inner one has been munged (removing the "{" in the process).
+        # The trick is in the ``^{`` below - it prevents a match of an outer
+        # ``struct``/``union`` until the inner one has been munged
+        # (removing the ``{`` in the process).
         #
         struct_members = KernRe(r'(struct|union)'   # 0: declaration type
                                 r'([^\{\};]+)' 	    # 1: possible name
@@ -794,11 +811,12 @@ class KernelDoc:
             tuples = struct_members.findall(members)
         return members
 
-    #
-    # Format the struct declaration into a standard form for inclusion in the
-    # resulting docs.
-    #
     def format_struct_decl(self, declaration):
+        """
+        Format the ``struct`` declaration into a standard form for inclusion
+        in the resulting docs.
+        """
+
         #
         # Insert newlines, get rid of extra spaces.
         #
@@ -832,7 +850,7 @@ class KernelDoc:
 
     def dump_struct(self, ln, proto):
         """
-        Store an entry for a struct or union
+        Store an entry for a ``struct`` or ``union``
         """
         #
         # Do the basic parse to get the pieces of the declaration.
@@ -874,7 +892,7 @@ class KernelDoc:
 
     def dump_enum(self, ln, proto):
         """
-        Stores an enum inside self.entries array.
+        Store an ``enum`` inside self.entries array.
         """
         #
         # Strip preprocessor directives.  Note that this depends on the
@@ -1021,7 +1039,7 @@ class KernelDoc:
 
     def dump_declaration(self, ln, prototype):
         """
-        Stores a data declaration inside self.entries array.
+        Store a data declaration inside self.entries array.
         """
 
         if self.entry.decl_type == "enum":
@@ -1038,7 +1056,7 @@ class KernelDoc:
 
     def dump_function(self, ln, prototype):
         """
-        Stores a function or function macro inside self.entries array.
+        Store a function or function macro inside self.entries array.
         """
 
         found = func_macro = False
@@ -1139,7 +1157,7 @@ class KernelDoc:
 
     def dump_typedef(self, ln, proto):
         """
-        Stores a typedef inside self.entries array.
+        Store a ``typedef`` inside self.entries array.
         """
         #
         # We start by looking for function typedefs.
@@ -1193,7 +1211,7 @@ class KernelDoc:
     @staticmethod
     def process_export(function_set, line):
         """
-        process EXPORT_SYMBOL* tags
+        process ``EXPORT_SYMBOL*`` tags
 
         This method doesn't use any variable from the class, so declare it
         with a staticmethod decorator.
@@ -1224,7 +1242,7 @@ class KernelDoc:
 
     def process_normal(self, ln, line):
         """
-        STATE_NORMAL: looking for the /** to begin everything.
+        STATE_NORMAL: looking for the ``/**`` to begin everything.
         """
 
         if not doc_start.match(line):
@@ -1314,10 +1332,10 @@ class KernelDoc:
         else:
             self.emit_msg(ln, f"Cannot find identifier on line:\n{line}")
 
-    #
-    # Helper function to determine if a new section is being started.
-    #
     def is_new_section(self, ln, line):
+        """
+        Helper function to determine if a new section is being started.
+        """
         if doc_sect.search(line):
             self.state = state.BODY
             #
@@ -1349,10 +1367,10 @@ class KernelDoc:
             return True
         return False
 
-    #
-    # Helper function to detect (and effect) the end of a kerneldoc comment.
-    #
     def is_comment_end(self, ln, line):
+        """
+        Helper function to detect (and effect) the end of a kerneldoc comment.
+        """
         if doc_end.search(line):
             self.dump_section()
 
@@ -1371,7 +1389,7 @@ class KernelDoc:
 
     def process_decl(self, ln, line):
         """
-        STATE_DECLARATION: We've seen the beginning of a declaration
+        STATE_DECLARATION: We've seen the beginning of a declaration.
         """
         if self.is_new_section(ln, line) or self.is_comment_end(ln, line):
             return
@@ -1400,7 +1418,7 @@ class KernelDoc:
 
     def process_special(self, ln, line):
         """
-        STATE_SPECIAL_SECTION: a section ending with a blank line
+        STATE_SPECIAL_SECTION: a section ending with a blank line.
         """
         #
         # If we have hit a blank line (only the " * " marker), then this
@@ -1490,7 +1508,7 @@ class KernelDoc:
 
     def syscall_munge(self, ln, proto):         # pylint: disable=W0613
         """
-        Handle syscall definitions
+        Handle syscall definitions.
         """
 
         is_void = False
@@ -1529,7 +1547,7 @@ class KernelDoc:
 
     def tracepoint_munge(self, ln, proto):
         """
-        Handle tracepoint definitions
+        Handle tracepoint definitions.
         """
 
         tracepointname = None
@@ -1565,7 +1583,7 @@ class KernelDoc:
         return proto
 
     def process_proto_function(self, ln, line):
-        """Ancillary routine to process a function prototype"""
+        """Ancillary routine to process a function prototype."""
 
         # strip C99-style comments to end of line
         line = KernRe(r"//.*$", re.S).sub('', line)
@@ -1610,7 +1628,9 @@ class KernelDoc:
             self.reset_state(ln)
 
     def process_proto_type(self, ln, line):
-        """Ancillary routine to process a type"""
+        """
+        Ancillary routine to process a type.
+        """
 
         # Strip C99-style comments and surrounding whitespace
         line = KernRe(r"//.*$", re.S).sub('', line).strip()
@@ -1664,7 +1684,7 @@ class KernelDoc:
             self.process_proto_type(ln, line)
 
     def process_docblock(self, ln, line):
-        """STATE_DOCBLOCK: within a DOC: block."""
+        """STATE_DOCBLOCK: within a ``DOC:`` block."""
 
         if doc_end.search(line):
             self.dump_section()
@@ -1676,7 +1696,7 @@ class KernelDoc:
 
     def parse_export(self):
         """
-        Parses EXPORT_SYMBOL* macros from a single Kernel source file.
+        Parses ``EXPORT_SYMBOL*`` macros from a single Kernel source file.
         """
 
         export_table = set()
@@ -1693,10 +1713,7 @@ class KernelDoc:
 
         return export_table
 
-    #
-    # The state/action table telling us which function to invoke in
-    # each state.
-    #
+    #: The state/action table telling us which function to invoke in each state.
     state_actions = {
         state.NORMAL:			process_normal,
         state.NAME:			process_name,
-- 
2.52.0

[PATCH 08/13] docs: kdoc_output: Improve docstrings and comments

[Mauro Carvalho Chehab]

In preparation to document kernel-doc module, improve its
documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/lib/python/kdoc/kdoc_output.py | 60 ++++++++++++++++------------
 1 file changed, 35 insertions(+), 25 deletions(-)

diff --git a/tools/lib/python/kdoc/kdoc_output.py b/tools/lib/python/kdoc/kdoc_output.py
index d2bf94275d65..4210b91dde5f 100644
--- a/tools/lib/python/kdoc/kdoc_output.py
+++ b/tools/lib/python/kdoc/kdoc_output.py
@@ -5,14 +5,16 @@
 # pylint: disable=C0301,R0902,R0911,R0912,R0913,R0914,R0915,R0917
 
 """
-Implement output filters to print kernel-doc documentation.
+Classes to implement output filters to print kernel-doc documentation.
 
-The implementation uses a virtual base class (OutputFormat) which
+The implementation uses a virtual base class ``OutputFormat``. It
 contains dispatches to virtual methods, and some code to filter
 out output messages.
 
 The actual implementation is done on one separate class per each type
-of output. Currently, there are output classes for ReST and man/troff.
+of output, e.g. ``RestFormat`` and ``ManFormat`` classes.
+
+Currently, there are output classes for ReST and man/troff.
 """
 
 import os
@@ -54,16 +56,19 @@ class OutputFormat:
     """
 
     # output mode.
-    OUTPUT_ALL          = 0 # output all symbols and doc sections
-    OUTPUT_INCLUDE      = 1 # output only specified symbols
-    OUTPUT_EXPORTED     = 2 # output exported symbols
-    OUTPUT_INTERNAL     = 3 # output non-exported symbols
+    OUTPUT_ALL          = 0 #: Output all symbols and doc sections.
+    OUTPUT_INCLUDE      = 1 #: Output only specified symbols.
+    OUTPUT_EXPORTED     = 2 #: Output exported symbols.
+    OUTPUT_INTERNAL     = 3 #: Output non-exported symbols.
 
-    # Virtual member to be overridden at the inherited classes
+    #: Highlights to be used in ReST format.
     highlights = []
 
+    #: Blank line character.
+    blankline = ""
+
     def __init__(self):
-        """Declare internal vars and set mode to OUTPUT_ALL"""
+        """Declare internal vars and set mode to ``OUTPUT_ALL``."""
 
         self.out_mode = self.OUTPUT_ALL
         self.enable_lineno = None
@@ -128,7 +133,7 @@ class OutputFormat:
             self.config.warning(log_msg)
 
     def check_doc(self, name, args):
-        """Check if DOC should be output"""
+        """Check if DOC should be output."""
 
         if self.no_doc_sections:
             return False
@@ -177,7 +182,7 @@ class OutputFormat:
 
     def msg(self, fname, name, args):
         """
-        Handles a single entry from kernel-doc parser
+        Handles a single entry from kernel-doc parser.
         """
 
         self.data = ""
@@ -220,30 +225,31 @@ class OutputFormat:
     # Virtual methods to be overridden by inherited classes
     # At the base class, those do nothing.
     def set_symbols(self, symbols):
-        """Get a list of all symbols from kernel_doc"""
+        """Get a list of all symbols from kernel_doc."""
 
     def out_doc(self, fname, name, args):
-        """Outputs a DOC block"""
+        """Outputs a DOC block."""
 
     def out_function(self, fname, name, args):
-        """Outputs a function"""
+        """Outputs a function."""
 
     def out_enum(self, fname, name, args):
-        """Outputs an enum"""
+        """Outputs an enum."""
 
     def out_var(self, fname, name, args):
-        """Outputs a variable"""
+        """Outputs a variable."""
 
     def out_typedef(self, fname, name, args):
-        """Outputs a typedef"""
+        """Outputs a typedef."""
 
     def out_struct(self, fname, name, args):
-        """Outputs a struct"""
+        """Outputs a struct."""
 
 
 class RestFormat(OutputFormat):
-    """Consts and functions used by ReST output"""
+    """Consts and functions used by ReST output."""
 
+    #: Highlights to be used in ReST format
     highlights = [
         (type_constant, r"``\1``"),
         (type_constant2, r"``\1``"),
@@ -263,9 +269,13 @@ class RestFormat(OutputFormat):
         (type_fallback, r":c:type:`\1`"),
         (type_param_ref, r"**\1\2**")
     ]
+
     blankline = "\n"
 
+    #: Sphinx literal block regex.
     sphinx_literal = KernRe(r'^[^.].*::$', cache=False)
+
+    #: Sphinx code block regex.
     sphinx_cblock = KernRe(r'^\.\.\ +code-block::', cache=False)
 
     def __init__(self):
@@ -280,7 +290,7 @@ class RestFormat(OutputFormat):
         self.lineprefix = ""
 
     def print_lineno(self, ln):
-        """Outputs a line number"""
+        """Outputs a line number."""
 
         if self.enable_lineno and ln is not None:
             ln += 1
@@ -289,7 +299,7 @@ class RestFormat(OutputFormat):
     def output_highlight(self, args):
         """
         Outputs a C symbol that may require being converted to ReST using
-        the self.highlights variable
+        the self.highlights variable.
         """
 
         input_text = args
@@ -570,7 +580,7 @@ class RestFormat(OutputFormat):
 
 
 class ManFormat(OutputFormat):
-    """Consts and functions used by man pages output"""
+    """Consts and functions used by man pages output."""
 
     highlights = (
         (type_constant, r"\1"),
@@ -587,6 +597,7 @@ class ManFormat(OutputFormat):
     )
     blankline = ""
 
+    #: Allowed timestamp formats.
     date_formats = [
         "%a %b %d %H:%M:%S %Z %Y",
         "%a %b %d %H:%M:%S %Y",
@@ -653,7 +664,7 @@ class ManFormat(OutputFormat):
         self.symbols = symbols
 
     def out_tail(self, fname, name, args):
-        """Adds a tail for all man pages"""
+        """Adds a tail for all man pages."""
 
         # SEE ALSO section
         self.data += f'.SH "SEE ALSO"' + "\n.PP\n"
@@ -689,7 +700,7 @@ class ManFormat(OutputFormat):
     def output_highlight(self, block):
         """
         Outputs a C symbol that may require being highlighted with
-        self.highlights variable using troff syntax
+        self.highlights variable using troff syntax.
         """
 
         contents = self.highlight_block(block)
@@ -720,7 +731,6 @@ class ManFormat(OutputFormat):
             self.output_highlight(text)
 
     def out_function(self, fname, name, args):
-        """output function in man"""
 
         out_name = self.arg_name(args, name)
 
-- 
2.52.0

[PATCH 09/13] docs: kdoc_re: Improve docstrings and comments

[Mauro Carvalho Chehab]

In preparation to document kernel-doc module, improve its
documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/lib/python/kdoc/kdoc_re.py | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/tools/lib/python/kdoc/kdoc_re.py b/tools/lib/python/kdoc/kdoc_re.py
index 2dfa1bf83d64..2816bd9f90f8 100644
--- a/tools/lib/python/kdoc/kdoc_re.py
+++ b/tools/lib/python/kdoc/kdoc_re.py
@@ -51,6 +51,9 @@ class KernRe:
         """
         return self.regex.pattern
 
+    def __repr__(self):
+        return f're.compile("{self.regex.pattern}")'
+
     def __add__(self, other):
         """
         Allows adding two regular expressions into one.
@@ -61,7 +64,7 @@ class KernRe:
 
     def match(self, string):
         """
-        Handles a re.match storing its results
+        Handles a re.match storing its results.
         """
 
         self.last_match = self.regex.match(string)
@@ -69,7 +72,7 @@ class KernRe:
 
     def search(self, string):
         """
-        Handles a re.search storing its results
+        Handles a re.search storing its results.
         """
 
         self.last_match = self.regex.search(string)
@@ -77,28 +80,28 @@ class KernRe:
 
     def findall(self, string):
         """
-        Alias to re.findall
+        Alias to re.findall.
         """
 
         return self.regex.findall(string)
 
     def split(self, string):
         """
-        Alias to re.split
+        Alias to re.split.
         """
 
         return self.regex.split(string)
 
     def sub(self, sub, string, count=0):
         """
-        Alias to re.sub
+        Alias to re.sub.
         """
 
         return self.regex.sub(sub, string, count=count)
 
     def group(self, num):
         """
-        Returns the group results of the last match
+        Returns the group results of the last match.
         """
 
         return self.last_match.group(num)
@@ -110,7 +113,7 @@ class NestedMatch:
     even harder on Python with its normal re module, as there are several
     advanced regular expressions that are missing.
 
-    This is the case of this pattern:
+    This is the case of this pattern::
 
             '\\bSTRUCT_GROUP(\\(((?:(?>[^)(]+)|(?1))*)\\))[^;]*;'
 
@@ -121,6 +124,7 @@ class NestedMatch:
     replace nested expressions.
 
     The original approach was suggested by:
+
         https://stackoverflow.com/questions/5454322/python-how-to-match-nested-parentheses-with-regex
 
     Although I re-implemented it to make it more generic and match 3 types
-- 
2.52.0

[PATCH 10/13] docs: kdoc: parse_data_structs: Improve docstrings and comments

[Mauro Carvalho Chehab]

In preparation to document kernel-doc module, improve its
documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/lib/python/kdoc/parse_data_structs.py | 62 +++++++++++++--------
 1 file changed, 39 insertions(+), 23 deletions(-)

diff --git a/tools/lib/python/kdoc/parse_data_structs.py b/tools/lib/python/kdoc/parse_data_structs.py
index 25361996cd20..9941cd19032e 100755
--- a/tools/lib/python/kdoc/parse_data_structs.py
+++ b/tools/lib/python/kdoc/parse_data_structs.py
@@ -9,12 +9,12 @@ Parse a source file or header, creating ReStructured Text cross references.
 It accepts an optional file to change the default symbol reference or to
 suppress symbols from the output.
 
-It is capable of identifying defines, functions, structs, typedefs,
-enums and enum symbols and create cross-references for all of them.
+It is capable of identifying ``define``, function, ``struct``, ``typedef``,
+``enum`` and ``enum`` symbols and create cross-references for all of them.
 It is also capable of distinguish #define used for specifying a Linux
 ioctl.
 
-The optional rules file contains a set of rules like:
+The optional rules file contains a set of rules like::
 
     ignore ioctl VIDIOC_ENUM_FMT
     replace ioctl VIDIOC_DQBUF vidioc_qbuf
@@ -34,8 +34,8 @@ class ParseDataStructs:
     It is meant to allow having a more comprehensive documentation, where
     uAPI headers will create cross-reference links to the code.
 
-    It is capable of identifying defines, functions, structs, typedefs,
-    enums and enum symbols and create cross-references for all of them.
+    It is capable of identifying ``define``, function, ``struct``, ``typedef``,
+    ``enum`` and ``enum`` symbols and create cross-references for all of them.
     It is also capable of distinguish #define used for specifying a Linux
     ioctl.
 
@@ -43,13 +43,13 @@ class ParseDataStructs:
     allows parsing an exception file. Such file contains a set of rules
     using the syntax below:
 
-    1. Ignore rules:
+    1. Ignore rules::
 
         ignore <type> <symbol>`
 
     Removes the symbol from reference generation.
 
-    2. Replace rules:
+    2. Replace rules::
 
         replace <type> <old_symbol> <new_reference>
 
@@ -58,22 +58,22 @@ class ParseDataStructs:
         - A simple symbol name;
         - A full Sphinx reference.
 
-    3. Namespace rules
+    3. Namespace rules::
 
         namespace <namespace>
 
        Sets C namespace to be used during cross-reference generation. Can
        be overridden by replace rules.
 
-    On ignore and replace rules, <type> can be:
-        - ioctl: for defines that end with _IO*, e.g. ioctl definitions
-        - define: for other defines
-        - symbol: for symbols defined within enums;
-        - typedef: for typedefs;
-        - enum: for the name of a non-anonymous enum;
-        - struct: for structs.
+    On ignore and replace rules, ``<type>`` can be:
+        - ``ioctl``: for defines that end with ``_IO*``, e.g. ioctl definitions
+        - ``define``: for other defines
+        - ``symbol``: for symbols defined within enums;
+        - ``typedef``: for typedefs;
+        - ``enum``: for the name of a non-anonymous enum;
+        - ``struct``: for structs.
 
-    Examples:
+    Examples::
 
         ignore define __LINUX_MEDIA_H
         ignore ioctl VIDIOC_ENUM_FMT
@@ -83,13 +83,15 @@ class ParseDataStructs:
         namespace MC
     """
 
-    # Parser regexes with multiple ways to capture enums and structs
+    #: Parser regex with multiple ways to capture enums.
     RE_ENUMS = [
         re.compile(r"^\s*enum\s+([\w_]+)\s*\{"),
         re.compile(r"^\s*enum\s+([\w_]+)\s*$"),
         re.compile(r"^\s*typedef\s*enum\s+([\w_]+)\s*\{"),
         re.compile(r"^\s*typedef\s*enum\s+([\w_]+)\s*$"),
     ]
+
+    #: Parser regex with multiple ways to capture structs.
     RE_STRUCTS = [
         re.compile(r"^\s*struct\s+([_\w][\w\d_]+)\s*\{"),
         re.compile(r"^\s*struct\s+([_\w][\w\d_]+)$"),
@@ -97,11 +99,13 @@ class ParseDataStructs:
         re.compile(r"^\s*typedef\s*struct\s+([_\w][\w\d_]+)$"),
     ]
 
-    # FIXME: the original code was written a long time before Sphinx C
+    # NOTE: the original code was written a long time before Sphinx C
     # domain to have multiple namespaces. To avoid to much turn at the
     # existing hyperlinks, the code kept using "c:type" instead of the
     # right types. To change that, we need to change the types not only
     # here, but also at the uAPI media documentation.
+
+    #: Dictionary containing C type identifiers to be transformed.
     DEF_SYMBOL_TYPES = {
         "ioctl": {
             "prefix": "\\ ",
@@ -158,6 +162,10 @@ class ParseDataStructs:
             self.symbols[symbol_type] = {}
 
     def read_exceptions(self, fname: str):
+        """
+        Read an optional exceptions file, used to override defaults.
+        """
+
         if not fname:
             return
 
@@ -242,9 +250,9 @@ class ParseDataStructs:
     def store_type(self, ln, symbol_type: str, symbol: str,
                    ref_name: str = None, replace_underscores: bool = True):
         """
-        Stores a new symbol at self.symbols under symbol_type.
+        Store a new symbol at self.symbols under symbol_type.
 
-        By default, underscores are replaced by "-"
+        By default, underscores are replaced by ``-``.
         """
         defs = self.DEF_SYMBOL_TYPES[symbol_type]
 
@@ -276,12 +284,16 @@ class ParseDataStructs:
         self.symbols[symbol_type][symbol] = (f"{prefix}{ref_link}{suffix}", ln)
 
     def store_line(self, line):
-        """Stores a line at self.data, properly indented"""
+        """
+        Store a line at self.data, properly indented.
+        """
         line = "    " + line.expandtabs()
         self.data += line.rstrip(" ")
 
     def parse_file(self, file_in: str, exceptions: str = None):
-        """Reads a C source file and get identifiers"""
+        """
+        Read a C source file and get identifiers.
+        """
         self.data = ""
         is_enum = False
         is_comment = False
@@ -433,7 +445,7 @@ class ParseDataStructs:
 
     def gen_toc(self):
         """
-        Create a list of symbols to be part of a TOC contents table
+        Create a list of symbols to be part of a TOC contents table.
         """
         text = []
 
@@ -464,6 +476,10 @@ class ParseDataStructs:
         return "\n".join(text)
 
     def write_output(self, file_in: str, file_out: str, toc: bool):
+        """
+        Write a ReST output file.
+        """
+
         title = os.path.basename(file_in)
 
         if toc:
-- 
2.52.0

[PATCH 11/13] docs: kdoc: enrich_formatter: Improve docstrings and comments

[Mauro Carvalho Chehab]

In preparation to document kernel-doc module, improve its
documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/lib/python/kdoc/enrich_formatter.py | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/tools/lib/python/kdoc/enrich_formatter.py b/tools/lib/python/kdoc/enrich_formatter.py
index bb171567a4ca..d1be4e5e1962 100644
--- a/tools/lib/python/kdoc/enrich_formatter.py
+++ b/tools/lib/python/kdoc/enrich_formatter.py
@@ -26,12 +26,16 @@ class EnrichFormatter(argparse.HelpFormatter):
     and how they're used at the __doc__ description.
     """
     def __init__(self, *args, **kwargs):
-        """Initialize class and check if is TTY"""
+        """
+        Initialize class and check if is TTY.
+        """
         super().__init__(*args, **kwargs)
         self._tty = sys.stdout.isatty()
 
     def enrich_text(self, text):
-        """Handle ReST markups (currently, only ``foo``)"""
+        r"""
+        Handle ReST markups (currently, only \`\`text\`\` markups).
+        """
         if self._tty and text:
             # Replace ``text`` with ANSI SGR (bold)
             return re.sub(r'\`\`(.+?)\`\`',
@@ -39,12 +43,16 @@ class EnrichFormatter(argparse.HelpFormatter):
         return text
 
     def _fill_text(self, text, width, indent):
-        """Enrich descriptions with markups on it"""
+        """
+        Enrich descriptions with markups on it.
+        """
         enriched = self.enrich_text(text)
         return "\n".join(indent + line for line in enriched.splitlines())
 
     def _format_usage(self, usage, actions, groups, prefix):
-        """Enrich positional arguments at usage: line"""
+        """
+        Enrich positional arguments at usage: line.
+        """
 
         prog = self._prog
         parts = []
@@ -63,7 +71,9 @@ class EnrichFormatter(argparse.HelpFormatter):
         return usage_text
 
     def _format_action_invocation(self, action):
-        """Enrich argument names"""
+        """
+        Enrich argument names.
+        """
         if not action.option_strings:
             return self.enrich_text(f"``{action.dest.upper()}``")
 
-- 
2.52.0

[PATCH 12/13] docs: kdoc: python_version: Improve docstrings and comments

[Mauro Carvalho Chehab]

In preparation to document kernel-doc module, improve its
documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/lib/python/kdoc/python_version.py | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/tools/lib/python/kdoc/python_version.py b/tools/lib/python/kdoc/python_version.py
index e83088013db2..4ddb7ead5f56 100644
--- a/tools/lib/python/kdoc/python_version.py
+++ b/tools/lib/python/kdoc/python_version.py
@@ -33,21 +33,31 @@ class PythonVersion:
     """
 
     def __init__(self, version):
-        """Ïnitialize self.version tuple from a version string"""
+        """
+        Ïnitialize self.version tuple from a version string.
+        """
         self.version = self.parse_version(version)
 
     @staticmethod
     def parse_version(version):
-        """Convert a major.minor.patch version into a tuple"""
+        """
+        Convert a major.minor.patch version into a tuple.
+        """
         return tuple(int(x) for x in version.split("."))
 
     @staticmethod
     def ver_str(version):
-        """Returns a version tuple as major.minor.patch"""
+        """
+        Returns a version tuple as major.minor.patch.
+        """
         return ".".join([str(x) for x in version])
 
     @staticmethod
     def cmd_print(cmd, max_len=80):
+        """
+        Outputs a command line, repecting maximum width.
+        """
+
         cmd_line = []
 
         for w in cmd:
@@ -66,7 +76,9 @@ class PythonVersion:
         return "\n  ".join(cmd_line)
 
     def __str__(self):
-        """Returns a version tuple as major.minor.patch from self.version"""
+        """
+        Return a version tuple as major.minor.patch from self.version.
+        """
         return self.ver_str(self.version)
 
     @staticmethod
-- 
2.52.0

[PATCH 13/13] docs: add kernel-doc modules documentation

[Mauro Carvalho Chehab]

Place kernel-doc modules documentation at Linux Kernel docs.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/tools/index.rst          |  1 +
 Documentation/tools/kdoc.rst           | 12 +++++++
 Documentation/tools/kdoc_ancillary.rst | 46 ++++++++++++++++++++++++++
 Documentation/tools/kdoc_output.rst    | 14 ++++++++
 Documentation/tools/kdoc_parser.rst    | 29 ++++++++++++++++
 Documentation/tools/python.rst         | 10 ++++++
 6 files changed, 112 insertions(+)
 create mode 100644 Documentation/tools/kdoc.rst
 create mode 100644 Documentation/tools/kdoc_ancillary.rst
 create mode 100644 Documentation/tools/kdoc_output.rst
 create mode 100644 Documentation/tools/kdoc_parser.rst
 create mode 100644 Documentation/tools/python.rst

diff --git a/Documentation/tools/index.rst b/Documentation/tools/index.rst
index 80488e290e10..89b81a13c6a1 100644
--- a/Documentation/tools/index.rst
+++ b/Documentation/tools/index.rst
@@ -12,6 +12,7 @@ more additions are needed here:
 
    rtla/index
    rv/index
+   python
 
 .. only::  subproject and html
 
diff --git a/Documentation/tools/kdoc.rst b/Documentation/tools/kdoc.rst
new file mode 100644
index 000000000000..e51ba159d8c4
--- /dev/null
+++ b/Documentation/tools/kdoc.rst
@@ -0,0 +1,12 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==================
+Kernel-doc modules
+==================
+
+.. toctree::
+   :maxdepth: 2
+
+   kdoc_parser
+   kdoc_output
+   kdoc_ancillary
diff --git a/Documentation/tools/kdoc_ancillary.rst b/Documentation/tools/kdoc_ancillary.rst
new file mode 100644
index 000000000000..3950d0a3f104
--- /dev/null
+++ b/Documentation/tools/kdoc_ancillary.rst
@@ -0,0 +1,46 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=================
+Ancillary classes
+=================
+
+Argparse formatter class
+========================
+
+.. automodule:: lib.python.kdoc.enrich_formatter
+   :members:
+   :show-inheritance:
+   :undoc-members:
+
+Regular expression class handler
+================================
+
+.. automodule:: lib.python.kdoc.kdoc_re
+   :members:
+   :show-inheritance:
+   :undoc-members:
+
+
+Chinese, Japanese and Korean variable fonts handler
+===================================================
+
+.. automodule:: lib.python.kdoc.latex_fonts
+   :members:
+   :show-inheritance:
+   :undoc-members:
+
+Kernel C file include logic
+===========================
+
+.. automodule:: lib.python.kdoc.parse_data_structs
+   :members:
+   :show-inheritance:
+   :undoc-members:
+
+Python version ancillary methods
+================================
+
+.. automodule:: lib.python.kdoc.python_version
+   :members:
+   :show-inheritance:
+   :undoc-members:
diff --git a/Documentation/tools/kdoc_output.rst b/Documentation/tools/kdoc_output.rst
new file mode 100644
index 000000000000..08fd271ec556
--- /dev/null
+++ b/Documentation/tools/kdoc_output.rst
@@ -0,0 +1,14 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=======================
+Kernel-doc output stage
+=======================
+
+Output handler for man pages and ReST
+=====================================
+
+.. automodule:: lib.python.kdoc.kdoc_output
+   :members:
+   :show-inheritance:
+   :undoc-members:
+
diff --git a/Documentation/tools/kdoc_parser.rst b/Documentation/tools/kdoc_parser.rst
new file mode 100644
index 000000000000..03ee54a1b1cc
--- /dev/null
+++ b/Documentation/tools/kdoc_parser.rst
@@ -0,0 +1,29 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=======================
+Kernel-doc parser stage
+=======================
+
+File handler classes
+====================
+
+.. automodule:: lib.python.kdoc.kdoc_files
+   :members:
+   :show-inheritance:
+   :undoc-members:
+
+Parsed item data class
+======================
+
+.. automodule:: lib.python.kdoc.kdoc_item
+   :members:
+   :show-inheritance:
+   :undoc-members:
+
+Parser classes and methods
+==========================
+
+.. automodule:: lib.python.kdoc.kdoc_parser
+   :members:
+   :show-inheritance:
+   :undoc-members:
diff --git a/Documentation/tools/python.rst b/Documentation/tools/python.rst
new file mode 100644
index 000000000000..e826787ce9dd
--- /dev/null
+++ b/Documentation/tools/python.rst
@@ -0,0 +1,10 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+================
+Python libraries
+================
+
+.. toctree::
+   :maxdepth: 4
+
+   kdoc
-- 
2.52.0

Re: [PATCH 00/13] Add kernel-doc modules to Documentation/tools

[Jonathan Corbet]

Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:

> With this version, python files inside tools/ or scripts/
> can be documented, but there is a catch (probably due to PEP8):
>
>     - files must end with ".py"
>     - file names can't have "-".
>
> So, unfortunately, we can't document kernel-doc.py, except if
> we rename it to kernel_doc.py.

That is ... irritating ...

You've probably seen my other series to finally ... rename
kernel-doc.py.  We could certainly rename it to something different.
But I really dislike having language extensions on files meant to be
executed as commands; you shouldn't care what language it's written in
when you run it.

In the end, I'd say let's not worry about running autodoc on that file.
All there is to document is its command line, and we can certainly
maintain that separately.

jon

Re: [PATCH 00/13] Add kernel-doc modules to Documentation/tools

[Mauro Carvalho Chehab]

Em Wed, 14 Jan 2026 11:01:45 -0700
Jonathan Corbet <corbet@lwn.net> escreveu:

> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
> > With this version, python files inside tools/ or scripts/
> > can be documented, but there is a catch (probably due to PEP8):
> >
> >     - files must end with ".py"
> >     - file names can't have "-".
> >
> > So, unfortunately, we can't document kernel-doc.py, except if
> > we rename it to kernel_doc.py.  
> 
> That is ... irritating ...

Agreed, but this is not really a problem with Sphinx but with
Python itself: you can't include any module that has "-" on it, or
doesn't end with ".py"[1].

[1] I guess it is actually possible, but only using low-level APIs,
    e.g. using importlib.util, importlib.util.module_from_spec(spec)
    and other dirty tricks, but this is certainly something we should
    avoid to do.

> You've probably seen my other series to finally ... rename
> kernel-doc.py. 

Just saw it ;-)

> We could certainly rename it to something different.
> But I really dislike having language extensions on files meant to be
> executed as commands; you shouldn't care what language it's written in
> when you run it.

I don't like it either, but Python is really picky on some things.

The problem here is that this is a Python policy violation. To change
that, one needs to write a PEP and convince Python maintainers to merge
it, together with changes on python "import" directive.

Alternatively, assuming that some magic words would be enough to
convince importlib to load a name without ".py" and with "-", it could be
easier to convince Sphinx autodoc maintainers to take a patch, as they're 
probably using importlib somewhere to dynamically import a file based 
at the string inside "automodule" directive. On a quick grep,
this seems to be the case, and such logic is inside:

	sphinx/ext/autodoc/importer.py


> In the end, I'd say let's not worry about running autodoc on that file.
> All there is to document is its command line, and we can certainly
> maintain that separately.

It may be true for kernel-doc, but:

- there are other scripts that we may want to document; and:
- we need to define a naming convention for python code

Also,

- keeping it in separate makes harder to maintain.

So, even if we don't actually add kernel-doc docstrings and
functions via autodoc, I think it is still worth having a
name convention that would allow that.

Thanks,
Mauro

Re: [PATCH 00/13] Add kernel-doc modules to Documentation/tools

[Jonathan Corbet]

Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:

>> We could certainly rename it to something different.
>> But I really dislike having language extensions on files meant to be
>> executed as commands; you shouldn't care what language it's written in
>> when you run it.
>
> I don't like it either, but Python is really picky on some things.
>
> The problem here is that this is a Python policy violation. To change
> that, one needs to write a PEP and convince Python maintainers to merge
> it, together with changes on python "import" directive.

...or just ignore it.  There is a reason that "pip" is called "pip"
rather than "pip.py" - the Python folks don't keep those extensions on
commands either.

> Alternatively, assuming that some magic words would be enough to
> convince importlib to load a name without ".py" and with "-", it could be
> easier to convince Sphinx autodoc maintainers to take a patch, as they're 
> probably using importlib somewhere to dynamically import a file based 
> at the string inside "automodule" directive. On a quick grep,
> this seems to be the case, and such logic is inside:
>
> 	sphinx/ext/autodoc/importer.py

No doubt we could do that.  But is it really worth the trouble?  There
is not much in kernel-doc that needs documenting, especially since you
did the work to move the actual functionality into separate modules. 

> So, even if we don't actually add kernel-doc docstrings and
> functions via autodoc, I think it is still worth having a
> name convention that would allow that.

Instead, I think you're trying to take a functionality meant to describe
APIs and use it to document command-line stuff.  I'm happy to live by
the import rules for stuff that is actually imported; I think it makes
less sense to let them drive the naming of files that are outside of
their intended scope.

jon

Re: [PATCH 00/13] Add kernel-doc modules to Documentation/tools

[Mauro Carvalho Chehab]

Em Wed, 14 Jan 2026 13:46:31 -0700
Jonathan Corbet <corbet@lwn.net> escreveu:

> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
> >> We could certainly rename it to something different.
> >> But I really dislike having language extensions on files meant to be
> >> executed as commands; you shouldn't care what language it's written in
> >> when you run it.  
> >
> > I don't like it either, but Python is really picky on some things.
> >
> > The problem here is that this is a Python policy violation. To change
> > that, one needs to write a PEP and convince Python maintainers to merge
> > it, together with changes on python "import" directive.  
> 
> ...or just ignore it. 

Indeed this is an option.

> There is a reason that "pip" is called "pip"
> rather than "pip.py" - the Python folks don't keep those extensions on
> commands either.

True, but see what pip has:

	$ more /usr/bin/pip
	#! /usr/bin/python3 -P
	# -*- coding: utf-8 -*-
	import re
	import sys
	from pip._internal.cli.main import main
	if __name__ == '__main__':
	    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
	    sys.exit(main())
	/usr/bin/pip (END)

Everything, including main are outside it. Btw, this code is almost
identical to sphinx-build:

	$ more /usr/bin/sphinx-build
	#! /usr/bin/python3 -P
	# -*- coding: utf-8 -*-
	import re
	import sys
	from sphinx.cmd.build import main
	if __name__ == '__main__':
	    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
	    sys.exit(main())

This would be equivalent of having a tools/docs/kernel-doc like this
(untested):

	#!/usr/bin/env python3
	from .kdoc.main import main
	if __name__ == '__main__':
	    sys.exit(main())

where the actual argparse code would be inside tools/docs/kdoc/main.py

> > Alternatively, assuming that some magic words would be enough to
> > convince importlib to load a name without ".py" and with "-", it could be
> > easier to convince Sphinx autodoc maintainers to take a patch, as they're 
> > probably using importlib somewhere to dynamically import a file based 
> > at the string inside "automodule" directive. On a quick grep,
> > this seems to be the case, and such logic is inside:
> >
> > 	sphinx/ext/autodoc/importer.py  
> 
> No doubt we could do that.  But is it really worth the trouble?  There
> is not much in kernel-doc that needs documenting, especially since you
> did the work to move the actual functionality into separate modules. 

I'm not particularly concerned about kernel-doc here. I'm more
concerned on defining how things like that are expected to be
documented.

Yet, if you add this:

	.. automodule:: docs.kernel_doc
	   :members:
	   :show-inheritance:
	   :undoc-members:

The generated documentation sounds somewhat relevant to me - specially
if placed together with the kernel-doc module API documentation:

	kernel-doc module documentation
	===============================

	kernel_doc

	Print formatted kernel documentation to stdout

	Read C language source or header FILEs, extract embedded documentation comments,
	and print formatted documentation to standard output.

	The documentation comments are identified by the /** opening comment mark.

	See Documentation/doc-guide/kernel-doc.rst for the documentation comment syntax.

	class docs.kernel_doc.MsgFormatter(fmt=None, datefmt=None, style='%', validate=True, *, defaults=None)

	    Bases: Formatter

	    Helper class to format warnings in a similar way to kernel-doc.pl.

	    format(record)

	        Format the specified record as text.

	        The record’s attribute dictionary is used as the operand to a 
		string formatting operation which yields the returned string. 
		Before formatting the dictionary, a couple of preparatory
		steps are carried out. The message attribute of the record
		is computed using LogRecord.getMessage(). If the formatting
		string uses the time (as determined by a call to usesTime(),
		formatTime() is called to format the event time. If there is 
		exception information, it is formatted using formatException()
		and appended to the message.

	docs.kernel_doc.main()

	    Main program By default, the return value is:

	        0: success or Python version is not compatible with
		   kernel-doc. If -Werror is not used, it will also
        	   return 0 if there are issues at kernel-doc markups;

	        1: an abnormal condition happened;

	        2: argparse issued an error;

	        3: -Werror is used, and one or more unfiltered parse warnings
	           happened.

> > So, even if we don't actually add kernel-doc docstrings and
> > functions via autodoc, I think it is still worth having a
> > name convention that would allow that.  
> 
> Instead, I think you're trying to take a functionality meant to describe
> APIs and use it to document command-line stuff.  I'm happy to live by
> the import rules for stuff that is actually imported; I think it makes
> less sense to let them drive the naming of files that are outside of
> their intended scope.

Yeah, `MsgFormatter` doesn't belong to doc-guide. main return values
does, however. IMO it is important to keep it documented together with
the code.

It should be noticed that autodoc has support for selecting symbols.
So:

	.. automodule:: docs.kernel_doc
	   :members: main

would pick only module description + main

and if we add, instead:

	.. autofunction:: docs.kernel_doc.main

it would just pick docstrings for main, e.g. it would place just
this:

	docs.kernel_doc.main()

	    Main program By default, the return value is:

	        0: success or Python version is not compatible with
		   kernel-doc. If -Werror is not used, it will also
        	   return 0 if there are issues at kernel-doc markups;

	        1: an abnormal condition happened;

	        2: argparse issued an error;

	        3: -Werror is used, and one or more unfiltered parse warnings
	           happened.

---

In summary, all I'm saying is that, if we stick to PEP8 names, we
can opt to import a documentation directly from the script instead
of writing it twice: at the code and on a rst file.

Btw, if you want to test it, you need just one patch to enable
it:

	https://lore.kernel.org/linux-doc/6aa5a5b4a686f07c8f3e6cb04fe4c07ed9c1d071.1768396023.git.mchehab+huawei@kernel.org/T/#u

this basically allows using tools/ and scripts/ as the base for
documentation.

You may also want this css patch, as default format with alabaster
is very ugly:

	https://lore.kernel.org/linux-doc/8d66988f05d06d10938e062ed4465bf303c51441.1768396023.git.mchehab+huawei@kernel.org/T/#u
	
with that, you can experiment inserting autodoc stuff using:	

	.. automodule:: docs.name
	   :modules:

to document all public methods from "docs/name.py" file, or:

	.. autofunction:: docs.name.function

and/or:

	.. autoclass:: docs.name.class

for a single "function" (or "class") inside "docs/name.py".

Again, the limitation is that "name" ends with ".py" and only
have (lower case?) letters, numbers and underscores - e.g. it
shall be something that "import" and "from ... import" supports.


Thanks,
Mauro

Re: [PATCH 00/13] Add kernel-doc modules to Documentation/tools

[Mauro Carvalho Chehab]

On Thu, Jan 15, 2026 at 02:29:21AM +0100, Mauro Carvalho Chehab wrote:
> Em Wed, 14 Jan 2026 13:46:31 -0700
> Jonathan Corbet <corbet@lwn.net> escreveu:
> 
> > Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> > 
> > >> We could certainly rename it to something different.
> > >> But I really dislike having language extensions on files meant to be
> > >> executed as commands; you shouldn't care what language it's written in
> > >> when you run it.  
> > >
> > > I don't like it either, but Python is really picky on some things.
> > >
> > > The problem here is that this is a Python policy violation. To change
> > > that, one needs to write a PEP and convince Python maintainers to merge
> > > it, together with changes on python "import" directive.  

> > There is a reason that "pip" is called "pip"
> > rather than "pip.py" - the Python folks don't keep those extensions on
> > commands either.
> 
> This would be equivalent of having a tools/docs/kernel-doc like this
> (untested):
> 
> 	#!/usr/bin/env python3
> 	from .kdoc.main import main
> 	if __name__ == '__main__':
> 	    sys.exit(main())
> 
> where the actual argparse code would be inside tools/docs/kdoc/main.py

In time, I'm not suggesting doing that at all.

I don't like the idea of splitting main() into a separate file: at least 
in my head, the absolute minimum size of a script should be to be able 
to handle command line arguments by itself, without including a module 
for it.

> 	class docs.kernel_doc.MsgFormatter(fmt=None, datefmt=None, style='%', validate=True, *, defaults=None)
> 
> 	    Bases: Formatter
> 
> 	    Helper class to format warnings in a similar way to kernel-doc.pl.
> 
> 	    format(record)
> 
> 	        Format the specified record as text.
> 
> 	        The record’s attribute dictionary is used as the operand to a 
> 		string formatting operation which yields the returned string. 
> 		Before formatting the dictionary, a couple of preparatory
> 		steps are carried out. The message attribute of the record
> 		is computed using LogRecord.getMessage(). If the formatting
> 		string uses the time (as determined by a call to usesTime(),
> 		formatTime() is called to format the event time. If there is 
> 		exception information, it is formatted using formatException()
> 		and appended to the message.

I also think that we should move MsgFormatter to a separate file. What 
this class does is that it bolds arguments when argparse displays help. 
It has nothing to do with kernel-doc itself, and can be reused on all 
other places where we use argparse. So, placing it under 
tools/lib/python, outside kdoc dir, IMO makes sense.

The same applies to some other stuff currently under kdoc, like 
enrich_formatter.py. Once the series moving kernel-doc to tools/docs 
gets merged, IMHO the next step would be to do some cleanup at 
tools/lib/python, moving things that aren't independent on the actual 
parser to be outside kdoc directory.

Thanks,
Mauro

Re: [PATCH 02/13] docs: enable Sphinx autodoc extension to allow documenting python

[Jani Nikula]

On Wed, 14 Jan 2026, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> Adding python documentation is simple with Sphinx: all we need
> is to include the ext.autodoc extension and add the directories
> where the Python code sits at the sys.path.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
>  Documentation/conf.py | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index 1ea2ae5c6276..429fcc9fd7f7 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -13,11 +13,18 @@ from  textwrap import dedent
>  
>  import sphinx
>  
> +# Location of Documentation/ directory
> +doctree = os.path.abspath(".")

Looking this up based on __file__ would be more robust than cwd.

Calling this doctree is misleading because doctree is a specific Sphinx
term that means something else. The doctree directory is where the
parsed and pickled documents are cached.

Oh, I see that you're just moving this, but this is something that
should be fixed first.

> +
>  # If extensions (or modules to document with autodoc) are in another directory,
>  # add these directories to sys.path here. If the directory is relative to the
>  # documentation root, use os.path.abspath to make it absolute, like shown here.
>  sys.path.insert(0, os.path.abspath("sphinx"))
>  
> +# Allow sphinx.ext.autodoc to document from tools and scripts
> +sys.path.append(f"{doctree}/../tools")
> +sys.path.append(f"{doctree}/../scripts")

These would be much nicer with pathlib.Path.

> +
>  # Minimal supported version
>  needs_sphinx = "3.4.3"
>  
> @@ -32,9 +39,6 @@ else:
>      # Include patterns that don't contain directory names, in glob format
>      include_patterns = ["**.rst"]
>  
> -# Location of Documentation/ directory
> -doctree = os.path.abspath(".")
> -
>  # Exclude of patterns that don't contain directory names, in glob format.
>  exclude_patterns = []
>  
> @@ -151,6 +155,7 @@ extensions = [
>      "maintainers_include",
>      "parser_yaml",
>      "rstFlatTable",
> +    "sphinx.ext.autodoc",
>      "sphinx.ext.autosectionlabel",
>      "sphinx.ext.ifconfig",
>      "translations",

-- 
Jani Nikula, Intel

Re: [PATCH 02/13] docs: enable Sphinx autodoc extension to allow documenting python

[Mauro Carvalho Chehab]

On Thu, Jan 15, 2026 at 12:19:48PM +0200, Jani Nikula wrote:
> On Wed, 14 Jan 2026, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> > Adding python documentation is simple with Sphinx: all we need
> > is to include the ext.autodoc extension and add the directories
> > where the Python code sits at the sys.path.
> >
> > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> > ---
> >  Documentation/conf.py | 11 ++++++++---
> >  1 file changed, 8 insertions(+), 3 deletions(-)
> >
> > diff --git a/Documentation/conf.py b/Documentation/conf.py
> > index 1ea2ae5c6276..429fcc9fd7f7 100644
> > --- a/Documentation/conf.py
> > +++ b/Documentation/conf.py
> > @@ -13,11 +13,18 @@ from  textwrap import dedent
> >  
> >  import sphinx
> >  
> > +# Location of Documentation/ directory
> > +doctree = os.path.abspath(".")
> 
> Looking this up based on __file__ would be more robust than cwd.

Agreed.

> Calling this doctree is misleading because doctree is a specific Sphinx
> term that means something else. The doctree directory is where the
> parsed and pickled documents are cached.

Yeah, you're right: better use a different name.

> 
> Oh, I see that you're just moving this, but this is something that
> should be fixed first.

It can also be changed afterwards. Anyway, this should be on another
series, as such changes don't have anything to do with sphinx.ext.autodoc.

> 
> > +
> >  # If extensions (or modules to document with autodoc) are in another directory,
> >  # add these directories to sys.path here. If the directory is relative to the
> >  # documentation root, use os.path.abspath to make it absolute, like shown here.
> >  sys.path.insert(0, os.path.abspath("sphinx"))
> >  
> > +# Allow sphinx.ext.autodoc to document from tools and scripts
> > +sys.path.append(f"{doctree}/../tools")
> > +sys.path.append(f"{doctree}/../scripts")
> 
> These would be much nicer with pathlib.Path.

I guess we agree to disagree here: patchlib basically overrides math divison 
operator to work on patches like[1]

    p = Path('/etc')
    q = p / 'init.d' / 'reboot'

This looks really weird on my eyes. I can't see why this would be better
than:

    q = "/etc/init.d/reboot"

And yeah, I've seen examples in c++ that does similar things overriding 
math operators to do something else. Never liked this kind of math operator
abuse.

[1] got from textbook example at https://docs.python.org/3/library/pathlib.html

-- 
Thanks,
Mauro

Re: [PATCH 02/13] docs: enable Sphinx autodoc extension to allow documenting python

[Jani Nikula]

On Thu, 15 Jan 2026, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> On Thu, Jan 15, 2026 at 12:19:48PM +0200, Jani Nikula wrote:
>> On Wed, 14 Jan 2026, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
>> > Adding python documentation is simple with Sphinx: all we need
>> > is to include the ext.autodoc extension and add the directories
>> > where the Python code sits at the sys.path.
>> >
>> > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
>> > ---
>> >  Documentation/conf.py | 11 ++++++++---
>> >  1 file changed, 8 insertions(+), 3 deletions(-)
>> >
>> > diff --git a/Documentation/conf.py b/Documentation/conf.py
>> > index 1ea2ae5c6276..429fcc9fd7f7 100644
>> > --- a/Documentation/conf.py
>> > +++ b/Documentation/conf.py
>> > @@ -13,11 +13,18 @@ from  textwrap import dedent
>> >  
>> >  import sphinx
>> >  
>> > +# Location of Documentation/ directory
>> > +doctree = os.path.abspath(".")
>> 
>> Looking this up based on __file__ would be more robust than cwd.
>
> Agreed.
>
>> Calling this doctree is misleading because doctree is a specific Sphinx
>> term that means something else. The doctree directory is where the
>> parsed and pickled documents are cached.
>
> Yeah, you're right: better use a different name.
>
>> 
>> Oh, I see that you're just moving this, but this is something that
>> should be fixed first.
>
> It can also be changed afterwards. Anyway, this should be on another
> series, as such changes don't have anything to do with sphinx.ext.autodoc.
>
>> 
>> > +
>> >  # If extensions (or modules to document with autodoc) are in another directory,
>> >  # add these directories to sys.path here. If the directory is relative to the
>> >  # documentation root, use os.path.abspath to make it absolute, like shown here.
>> >  sys.path.insert(0, os.path.abspath("sphinx"))
>> >  
>> > +# Allow sphinx.ext.autodoc to document from tools and scripts
>> > +sys.path.append(f"{doctree}/../tools")
>> > +sys.path.append(f"{doctree}/../scripts")
>> 
>> These would be much nicer with pathlib.Path.
>
> I guess we agree to disagree here: patchlib basically overrides math divison 
> operator to work on patches like[1]
>
>     p = Path('/etc')
>     q = p / 'init.d' / 'reboot'
>
> This looks really weird on my eyes. I can't see why this would be better
> than:
>
>     q = "/etc/init.d/reboot"
>
> And yeah, I've seen examples in c++ that does similar things overriding 
> math operators to do something else. Never liked this kind of math operator
> abuse.

Resistance is futile, you will be assimilated. ;)

The upside is everything's a Path object rather than a string, giving
you methods that you'd expect paths but not strings to have, avoiding
the tedious string manipulation all over the place.

BR,
Jani.


>
> [1] got from textbook example at https://docs.python.org/3/library/pathlib.html

-- 
Jani Nikula, Intel

Re: [PATCH 02/13] docs: enable Sphinx autodoc extension to allow documenting python

[Mauro Carvalho Chehab]

Em Thu, 15 Jan 2026 14:18:45 +0200
Jani Nikula <jani.nikula@linux.intel.com> escreveu:

> On Thu, 15 Jan 2026, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> > On Thu, Jan 15, 2026 at 12:19:48PM +0200, Jani Nikula wrote:  
> >> On Wed, 14 Jan 2026, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:  
> >> > Adding python documentation is simple with Sphinx: all we need
> >> > is to include the ext.autodoc extension and add the directories
> >> > where the Python code sits at the sys.path.
> >> >
> >> > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> >> > ---
> >> >  Documentation/conf.py | 11 ++++++++---
> >> >  1 file changed, 8 insertions(+), 3 deletions(-)
> >> >
> >> > diff --git a/Documentation/conf.py b/Documentation/conf.py
> >> > index 1ea2ae5c6276..429fcc9fd7f7 100644
> >> > --- a/Documentation/conf.py
> >> > +++ b/Documentation/conf.py
> >> > @@ -13,11 +13,18 @@ from  textwrap import dedent
> >> >  
> >> >  import sphinx
> >> >  
> >> > +# Location of Documentation/ directory
> >> > +doctree = os.path.abspath(".")  
> >> 
> >> Looking this up based on __file__ would be more robust than cwd.  
> >
> > Agreed.
> >  
> >> Calling this doctree is misleading because doctree is a specific Sphinx
> >> term that means something else. The doctree directory is where the
> >> parsed and pickled documents are cached.  
> >
> > Yeah, you're right: better use a different name.
> >  
> >> 
> >> Oh, I see that you're just moving this, but this is something that
> >> should be fixed first.  
> >
> > It can also be changed afterwards. Anyway, this should be on another
> > series, as such changes don't have anything to do with sphinx.ext.autodoc.
> >  
> >>   
> >> > +
> >> >  # If extensions (or modules to document with autodoc) are in another directory,
> >> >  # add these directories to sys.path here. If the directory is relative to the
> >> >  # documentation root, use os.path.abspath to make it absolute, like shown here.
> >> >  sys.path.insert(0, os.path.abspath("sphinx"))
> >> >  
> >> > +# Allow sphinx.ext.autodoc to document from tools and scripts
> >> > +sys.path.append(f"{doctree}/../tools")
> >> > +sys.path.append(f"{doctree}/../scripts")  
> >> 
> >> These would be much nicer with pathlib.Path.  
> >
> > I guess we agree to disagree here: patchlib basically overrides math divison 
> > operator to work on patches like[1]
> >
> >     p = Path('/etc')
> >     q = p / 'init.d' / 'reboot'
> >
> > This looks really weird on my eyes. I can't see why this would be better
> > than:
> >
> >     q = "/etc/init.d/reboot"
> >
> > And yeah, I've seen examples in c++ that does similar things overriding 
> > math operators to do something else. Never liked this kind of math operator
> > abuse.  
> 
> Resistance is futile, you will be assimilated. ;)

Yeah, one day I might be forced to move to it. There were much more 
assimilation attempts with regards to IPv6... So, far, I've been
resisting  ;-)

> The upside is everything's a Path object rather than a string, giving
> you methods that you'd expect paths but not strings to have, avoiding
> the tedious string manipulation all over the place.

This could make sense if this were supposed to work outside Linux 
(or Unix). As this is not the case, we don't need class abstraction
to handle directory deliminator, which is mainly the reason why 
someone writes methods and classes like os.path and pathlib.

Also, os.path has methods already to handle them in a portable way.

In this specific case, it could be using it like:

	sys.path.append(os.path.join(doctree, "/../scripts"))

to do patch contactenation, or, to be portable on other OSes:

	sys.path.append(os.path.join(doctree, "..", "scripts"))

IMO, both are easier to read, and IMHO more direct than:

	from pathlib import Path

	script_path = Path(doctree) / ".." / "scripts"
	sys.path.append(str(script_path))

Also, pathlib is more error prone, as if one does:

	>>> doctree="/tmp"
	>>> import sys
	>>> from pathlib import Path
	>>> script_path = Path(doctree) / ".." / "scripts"
	>>> sys.path.append(script_path)

It ends placing PosixPath() at the list:

	>>> sys.path
	['', '/usr/lib64/python314.zip', '/usr/lib64/python3.14', '/usr/lib64/python3.14/lib-dynload', '/usr/lib/python3.14/site-packages', PosixPath('/tmp/../scripts')]

which I don't thing it would do the right thing to do.

Regards,
Mauro