[PATCH v3 00/15] Some improvements and fixes for the doc build system

[PATCH v3 00/15] Some improvements and fixes for the doc build system

[Mauro Carvalho Chehab]

Hi Jon,

On this version, only patch 1 changed to avoid full builds every time.

This series contain patches I made while working at the parser-yaml.
They aren't directly related to it. Instead, they address some issues
at the build system and provide test tools for building docs.
 
Most of the series is related to the new  test_doc_build.py.
This is a tool I wrote from scratch to help identifying regressions
for changes affecting the build system.

As described on its help page, it allows creating Python virtual
environments for different Sphinx versions that are supported 
by the Linux Kernel build system.

Besides creating the virtual environment, it can also test building
the documentation using "make htmldocs" (and/or other doc targets).

If called without "--versions" argument, it covers the versions shipped
on major distros, plus the lowest supported version.

A typical usage is to run:

$ time ./scripts/test_doc_build.py -m -a "SPHINXOPTS=-j8" -l distros.log
...
Summary:
        Sphinx 3.4.3 elapsed time: 00:07:22
        Sphinx 5.3.0 elapsed time: 00:07:30
        Sphinx 6.1.1 elapsed time: 00:23:43
        Sphinx 7.2.1 elapsed time: 00:07:34
        Sphinx 7.2.6 elapsed time: 00:07:43
        Sphinx 7.3.0 elapsed time: 00:07:54
        Sphinx 7.4.7 elapsed time: 00:04:04
        Sphinx 8.1.3 elapsed time: 00:03:14
        Sphinx 8.2.3 elapsed time: 00:03:12

real    72m30.037s
user    116m45.360s
sys     7m44.09

This should check the main backward-compatibility issues.

A more comprehensive test can be done with:

$ time ./scripts/test_doc_build.py -b -a "SPHINXOPTS=-j8" -l full.log --full

Summary:
        Sphinx 3.4.3 elapsed time: 00:07:15
        Sphinx 3.5.0 elapsed time: 00:07:05
        Sphinx 4.0.0 elapsed time: 00:07:10
        Sphinx 4.1.0 elapsed time: 00:07:20
        Sphinx 4.3.0 elapsed time: 00:07:22
        Sphinx 4.4.0 elapsed time: 00:07:24
        Sphinx 4.5.0 elapsed time: 00:07:13
        Sphinx 5.0.0 elapsed time: 00:07:34
        Sphinx 5.1.0 elapsed time: 00:07:32
        Sphinx 5.2.0 elapsed time: 00:07:29
        Sphinx 5.3.0 elapsed time: 00:07:35
        Sphinx 6.0.0 elapsed time: 00:22:34
        Sphinx 6.1.0 elapsed time: 00:23:57
        Sphinx 6.1.1 elapsed time: 00:23:41
        Sphinx 6.2.0 elapsed time: 00:07:26
        Sphinx 7.0.0 elapsed time: 00:07:29
        Sphinx 7.1.0 elapsed time: 00:07:22
        Sphinx 7.2.0 elapsed time: 00:07:24
        Sphinx 7.2.1 elapsed time: 00:07:31
        Sphinx 7.2.6 elapsed time: 00:07:47
        Sphinx 7.3.0 elapsed time: 00:07:44
        Sphinx 7.4.0 elapsed time: 00:04:16
        Sphinx 7.4.7 elapsed time: 00:04:12
        Sphinx 8.0.0 elapsed time: 00:03:11
        Sphinx 8.1.0 elapsed time: 00:03:17
        Sphinx 8.1.3 elapsed time: 00:03:17
        Sphinx 8.2.0 elapsed time: 00:03:12
        Sphinx 8.2.3 elapsed time: 00:03:14

real    229m13.749s
user    377m26.666s
sys     24m32.544s

Some notes:

1) on my machine, "-j8" is usually faster than "-jauto";
2) 6.x.x is problematic: Sphinx uses a lot of memory, being a friend
   of systemd-oomd killer, specially with -jauto. -j8 is a little better,
   but it still caused crashes on other apps, probably due to memory
   consumption.

Regards,
Mauro

---

- v3: initialize config patterns earlier to avoid full builds.

- v2: Added patches 7 to 15.

Mauro Carvalho Chehab (15):
  docs: conf.py: properly handle include and exclude patterns
  docs: Makefile: disable check rules on make cleandocs
  scripts: scripts/test_doc_build.py: add script to test doc build
  scripts: test_doc_build.py: make capture assynchronous
  scripts: test_doc_build.py: better control its output
  scripts: test_doc_build.py: better adjust to python version
  scripts: test_doc_build.py: improve dependency list
  scripts: test_doc_build.py: improve cmd.log logic
  scripts: test_doc_build.py: make the script smarter
  scripts: sphinx-pre-install: properly handle SPHINXBUILD
  scripts: sphinx-pre-install: fix release detection for Fedora
  scripts: test_doc_build.py: regroup and rename arguments
  docs: sphinx: add a file with the requirements for lowest version
  docs: conf.py: several coding style fixes
  docs: conf.py: Check Sphinx and docutils version

 Documentation/Makefile                    |   2 +
 Documentation/conf.py                     | 462 +++++++++++--------
 Documentation/doc-guide/sphinx.rst        |  23 +
 Documentation/sphinx/min_requirements.txt |  10 +
 scripts/sphinx-pre-install                |   6 +-
 scripts/test_doc_build.py                 | 513 ++++++++++++++++++++++
 6 files changed, 842 insertions(+), 174 deletions(-)
 create mode 100644 Documentation/sphinx/min_requirements.txt
 create mode 100755 scripts/test_doc_build.py

-- 
2.49.0

[PATCH v3 01/15] docs: conf.py: properly handle include and exclude patterns

[Mauro Carvalho Chehab]

When one does:
	make SPHINXDIRS="foo" htmldocs

All patterns would be relative to Documentation/foo, which
causes the include/exclude patterns like:

	include_patterns = [
		...
		f'foo/*.{ext}',
	]

to break. This is not what it is expected. Address it by
adding a logic to dynamically adjust the pattern when
SPHINXDIRS is used.

That allows adding parsers for other file types.

It should be noticed that include_patterns was added on
Sphinx 5.1:
	https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-include_patterns

So, a backward-compatible code is needed when we start
using it for real.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
---
 Documentation/conf.py | 67 ++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 63 insertions(+), 4 deletions(-)

diff --git a/Documentation/conf.py b/Documentation/conf.py
index 12de52a2b17e..91ce2b1c33cc 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -17,6 +17,66 @@ import os
 import sphinx
 import shutil
 
+# Get Sphinx version
+major, minor, patch = sphinx.version_info[:3]
+
+# Include_patterns were added on Sphinx 5.1
+if (major < 5) or (major == 5 and minor < 1):
+    has_include_patterns = False
+else:
+    has_include_patterns = True
+    # Include patterns that don't contain directory names, in glob format
+    include_patterns = ['**.rst']
+
+# Location of Documentation/ directory
+doctree = os.path.abspath('.')
+
+# Exclude of patterns that don't contain directory names, in glob format.
+exclude_patterns = []
+
+# List of patterns that contain directory names in glob format.
+dyn_include_patterns = []
+dyn_exclude_patterns = ['output']
+
+# Properly handle include/exclude patterns
+# ----------------------------------------
+
+def update_patterns(app, config):
+
+    """
+    On Sphinx, all directories are relative to what it is passed as
+    SOURCEDIR parameter for sphinx-build. Due to that, all patterns
+    that have directory names on it need to be dynamically set, after
+    converting them to a relative patch.
+
+    As Sphinx doesn't include any patterns outside SOURCEDIR, we should
+    exclude relative patterns that start with "../".
+    """
+
+    sourcedir = app.srcdir  # full path to the source directory
+    builddir = os.environ.get("BUILDDIR")
+
+    # setup include_patterns dynamically
+    if has_include_patterns:
+        for p in dyn_include_patterns:
+            full = os.path.join(doctree, p)
+
+            rel_path = os.path.relpath(full, start = app.srcdir)
+            if rel_path.startswith("../"):
+                continue
+
+            config.include_patterns.append(rel_path)
+
+    # setup exclude_patterns dynamically
+    for p in dyn_exclude_patterns:
+        full = os.path.join(doctree, p)
+
+        rel_path = os.path.relpath(full, start = app.srcdir)
+        if rel_path.startswith("../"):
+            continue
+
+        config.exclude_patterns.append(rel_path)
+
 # helper
 # ------
 
@@ -219,10 +279,6 @@ language = 'en'
 # Else, today_fmt is used as the format for a strftime call.
 #today_fmt = '%B %d, %Y'
 
-# List of patterns, relative to source directory, that match files and
-# directories to ignore when looking for source files.
-exclude_patterns = ['output']
-
 # The reST default role (used for this markup: `text`) to use for all
 # documents.
 #default_role = None
@@ -516,3 +572,6 @@ kerneldoc_srctree = '..'
 # the last statement in the conf.py file
 # ------------------------------------------------------------------------------
 loadConfig(globals())
+
+def setup(app):
+    app.connect('config-inited', update_patterns)
-- 
2.49.0

[PATCH v3 02/15] docs: Makefile: disable check rules on make cleandocs

[Mauro Carvalho Chehab]

It doesn't make sense to check for missing ABI and documents
when cleaning the tree.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
---
 Documentation/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index d30d66ddf1ad..b98477df5ddf 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -5,6 +5,7 @@
 # for cleaning
 subdir- := devicetree/bindings
 
+ifneq ($(MAKECMDGOALS),cleandocs)
 # Check for broken documentation file references
 ifeq ($(CONFIG_WARN_MISSING_DOCUMENTS),y)
 $(shell $(srctree)/scripts/documentation-file-ref-check --warn)
@@ -14,6 +15,7 @@ endif
 ifeq ($(CONFIG_WARN_ABI_ERRORS),y)
 $(shell $(srctree)/scripts/get_abi.py --dir $(srctree)/Documentation/ABI validate)
 endif
+endif
 
 # You can set these variables from the command line.
 SPHINXBUILD   = sphinx-build
-- 
2.49.0

[PATCH v3 03/15] scripts: scripts/test_doc_build.py: add script to test doc build

[Mauro Carvalho Chehab]

Testing Sphinx backward-compatibility is hard, as per version
minimal Python dependency requirements can be a nightmare.

Add a script to help automate such checks.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/test_doc_build.py | 241 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 241 insertions(+)
 create mode 100755 scripts/test_doc_build.py

diff --git a/scripts/test_doc_build.py b/scripts/test_doc_build.py
new file mode 100755
index 000000000000..482716fbe91d
--- /dev/null
+++ b/scripts/test_doc_build.py
@@ -0,0 +1,241 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
+#
+# pylint: disable=C0103,R1715
+
+"""
+Install minimal supported requirements for different Sphinx versions
+and optionally test the build.
+"""
+
+import argparse
+import os.path
+import sys
+import time
+
+from subprocess import run
+
+# Minimal python version supported by the building system
+python_bin = "python3.9"
+
+# Starting from 8.0.2, Python 3.9 becomes too old
+python_changes = {(8, 0, 2): "python3"}
+
+# Sphinx versions to be installed and their incremental requirements
+sphinx_requirements = {
+    (3, 4, 3): {
+        "alabaster": "0.7.13",
+        "babel": "2.17.0",
+        "certifi": "2025.6.15",
+        "charset-normalizer": "3.4.2",
+        "docutils": "0.15",
+        "idna": "3.10",
+        "imagesize": "1.4.1",
+        "Jinja2": "3.0.3",
+        "MarkupSafe": "2.0",
+        "packaging": "25.0",
+        "Pygments": "2.19.1",
+        "PyYAML": "5.1",
+        "requests": "2.32.4",
+        "snowballstemmer": "3.0.1",
+        "sphinxcontrib-applehelp": "1.0.4",
+        "sphinxcontrib-devhelp": "1.0.2",
+        "sphinxcontrib-htmlhelp": "2.0.1",
+        "sphinxcontrib-jsmath": "1.0.1",
+        "sphinxcontrib-qthelp": "1.0.3",
+        "sphinxcontrib-serializinghtml": "1.1.5",
+        "urllib3": "2.4.0",
+    },
+    (3, 5, 4): {},
+    (4, 0, 3): {
+        "docutils": "0.17.1",
+        "PyYAML": "5.1",
+    },
+    (4, 1, 2): {},
+    (4, 3, 2): {},
+    (4, 4, 0): {},
+    (4, 5, 0): {},
+    (5, 0, 2): {},
+    (5, 1, 1): {},
+    (5, 2, 3): {
+        "Jinja2": "3.1.2",
+        "MarkupSafe": "2.0",
+        "PyYAML": "5.3.1",
+    },
+    (5, 3, 0): {
+        "docutils": "0.18.1",
+        "PyYAML": "5.3.1",
+    },
+    (6, 0, 1): {},
+    (6, 1, 3): {},
+    (6, 2, 1): {
+        "PyYAML": "5.4.1",
+    },
+    (7, 0, 1): {},
+    (7, 1, 2): {},
+    (7, 2, 3): {
+        "PyYAML": "6.0.1",
+        "sphinxcontrib-serializinghtml": "1.1.9",
+    },
+    (7, 3, 7): {
+        "alabaster": "0.7.14",
+        "PyYAML": "6.0.1",
+    },
+    (7, 4, 7): {
+        "docutils": "0.20",
+        "PyYAML": "6.0.1",
+    },
+    (8, 0, 2): {},
+    (8, 1, 3): {
+        "PyYAML": "6.0.1",
+        "sphinxcontrib-applehelp": "1.0.7",
+        "sphinxcontrib-devhelp": "1.0.6",
+        "sphinxcontrib-htmlhelp": "2.0.6",
+        "sphinxcontrib-qthelp": "1.0.6",
+    },
+    (8, 2, 3): {
+        "PyYAML": "6.0.1",
+        "sphinxcontrib-serializinghtml": "1.1.9",
+    },
+}
+
+
+def parse_version(ver_str):
+    """Convert a version string into a tuple."""
+
+    return tuple(map(int, ver_str.split(".")))
+
+
+parser = argparse.ArgumentParser(description="Build docs for different sphinx_versions.")
+
+parser.add_argument('-v', '--version', help='Sphinx single version',
+                    type=parse_version)
+parser.add_argument('--min-version', "--min", help='Sphinx minimal version',
+                    type=parse_version)
+parser.add_argument('--max-version', "--max", help='Sphinx maximum version',
+                    type=parse_version)
+parser.add_argument('-a', '--make_args',
+                    help='extra arguments for make htmldocs, like SPHINXDIRS=netlink/specs',
+                    nargs="*")
+parser.add_argument('-w', '--write', help='write a requirements.txt file',
+                    action='store_true')
+parser.add_argument('-m', '--make',
+                    help='Make documentation',
+                    action='store_true')
+parser.add_argument('-i', '--wait-input',
+                    help='Wait for an enter before going to the next version',
+                    action='store_true')
+
+args = parser.parse_args()
+
+if not args.make_args:
+    args.make_args = []
+
+if args.version:
+    if args.min_version or args.max_version:
+        sys.exit("Use either --version or --min-version/--max-version")
+    else:
+        args.min_version = args.version
+        args.max_version = args.version
+
+sphinx_versions = sorted(list(sphinx_requirements.keys()))
+
+if not args.min_version:
+    args.min_version = sphinx_versions[0]
+
+if not args.max_version:
+    args.max_version = sphinx_versions[-1]
+
+first_run = True
+cur_requirements = {}
+built_time = {}
+
+for cur_ver, new_reqs in sphinx_requirements.items():
+    cur_requirements.update(new_reqs)
+
+    if cur_ver in python_changes:
+        python_bin = python_changes[cur_ver]
+
+    ver = ".".join(map(str, cur_ver))
+
+    if args.min_version:
+        if cur_ver < args.min_version:
+            continue
+
+    if args.max_version:
+        if cur_ver > args.max_version:
+            break
+
+    if not first_run and args.wait_input and args.make:
+        ret = input("Press Enter to continue or 'a' to abort: ").strip().lower()
+        if ret == "a":
+            print("Aborted.")
+            sys.exit()
+    else:
+        first_run = False
+
+    venv_dir = f"Sphinx_{ver}"
+    req_file = f"requirements_{ver}.txt"
+
+    print(f"\nSphinx {ver} with {python_bin}")
+
+    # Create venv
+    run([python_bin, "-m", "venv", venv_dir], check=True)
+    pip = os.path.join(venv_dir, "bin/pip")
+
+    # Create install list
+    reqs = []
+    for pkg, verstr in cur_requirements.items():
+        reqs.append(f"{pkg}=={verstr}")
+
+    reqs.append(f"Sphinx=={ver}")
+
+    run([pip, "install"] + reqs, check=True)
+
+    # Freeze environment
+    result = run([pip, "freeze"], capture_output=True, text=True, check=True)
+
+    # Pip install succeeded. Write requirements file
+    if args.write:
+        with open(req_file, "w", encoding="utf-8") as fp:
+            fp.write(result.stdout)
+
+    if args.make:
+        start_time = time.time()
+
+        # Prepare a venv environment
+        env = os.environ.copy()
+        bin_dir = os.path.join(venv_dir, "bin")
+        env["PATH"] = bin_dir + ":" + env["PATH"]
+        env["VIRTUAL_ENV"] = venv_dir
+        if "PYTHONHOME" in env:
+            del env["PYTHONHOME"]
+
+        # Test doc build
+        run(["make", "cleandocs"], env=env, check=True)
+        make = ["make"] + args.make_args + ["htmldocs"]
+
+        print(f". {bin_dir}/activate")
+        print(" ".join(make))
+        print("deactivate")
+        run(make, env=env, check=True)
+
+        end_time = time.time()
+        elapsed_time = end_time - start_time
+        hours, minutes = divmod(elapsed_time, 3600)
+        minutes, seconds = divmod(minutes, 60)
+
+        hours = int(hours)
+        minutes = int(minutes)
+        seconds = int(seconds)
+
+        built_time[ver] = f"{hours:02d}:{minutes:02d}:{seconds:02d}"
+
+        print(f"Finished doc build for Sphinx {ver}. Elapsed time: {built_time[ver]}")
+
+if args.make:
+    print()
+    print("Summary:")
+    for ver, elapsed_time in sorted(built_time.items()):
+        print(f"\tSphinx {ver} elapsed time: {elapsed_time}")
-- 
2.49.0

[PATCH v3 04/15] scripts: test_doc_build.py: make capture assynchronous

[Mauro Carvalho Chehab]

Prepare the tool to allow writing the output into log files.
For such purpose, receive stdin/stdout messages asynchronously.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/test_doc_build.py | 393 +++++++++++++++++++++++++-------------
 1 file changed, 255 insertions(+), 138 deletions(-)

diff --git a/scripts/test_doc_build.py b/scripts/test_doc_build.py
index 482716fbe91d..94f2f2d8c3b7 100755
--- a/scripts/test_doc_build.py
+++ b/scripts/test_doc_build.py
@@ -2,7 +2,7 @@
 # SPDX-License-Identifier: GPL-2.0
 # Copyright(c) 2025: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
 #
-# pylint: disable=C0103,R1715
+# pylint: disable=R0903,R0913,R0914,R0917
 
 """
 Install minimal supported requirements for different Sphinx versions
@@ -10,20 +10,20 @@ and optionally test the build.
 """
 
 import argparse
+import asyncio
 import os.path
 import sys
 import time
-
-from subprocess import run
+import subprocess
 
 # Minimal python version supported by the building system
-python_bin = "python3.9"
+MINIMAL_PYTHON_VERSION = "python3.9"
 
 # Starting from 8.0.2, Python 3.9 becomes too old
-python_changes = {(8, 0, 2): "python3"}
+PYTHON_VER_CHANGES = {(8, 0, 2): "python3"}
 
 # Sphinx versions to be installed and their incremental requirements
-sphinx_requirements = {
+SPHINX_REQUIREMENTS = {
     (3, 4, 3): {
         "alabaster": "0.7.13",
         "babel": "2.17.0",
@@ -101,141 +101,258 @@ sphinx_requirements = {
 }
 
 
+class AsyncCommands:
+    """Excecute command synchronously"""
+
+    stdout = None
+    stderr = None
+    output = None
+
+    async def _read(self, stream, verbose, is_info):
+        """Ancillary routine to capture while displaying"""
+
+        while stream is not None:
+            line = await stream.readline()
+            if line:
+                out = line.decode("utf-8", errors="backslashreplace")
+                self.output += out
+                if is_info:
+                    if verbose:
+                        print(out.rstrip("\n"))
+
+                    self.stdout += out
+                else:
+                    if verbose:
+                        print(out.rstrip("\n"), file=sys.stderr)
+
+                    self.stderr += out
+            else:
+                break
+
+    async def run(self, cmd, capture_output=False, check=False,
+                  env=None, verbose=True):
+
+        """
+        Execute an arbitrary command, handling errors.
+
+        Please notice that this class is not thread safe
+        """
+
+        self.stdout = ""
+        self.stderr = ""
+        self.output = ""
+
+        if verbose:
+            print("$ ", " ".join(cmd))
+
+        proc = await asyncio.create_subprocess_exec(cmd[0],
+                                                    *cmd[1:],
+                                                    env=env,
+                                                    stdout=asyncio.subprocess.PIPE,
+                                                    stderr=asyncio.subprocess.PIPE)
+
+        # Handle input and output in realtime
+        await asyncio.gather(
+            self._read(proc.stdout, verbose, True),
+            self._read(proc.stderr, verbose, False),
+        )
+
+        await proc.wait()
+
+        if check and proc.returncode > 0:
+            raise subprocess.CalledProcessError(returncode=proc.returncode,
+                                                cmd=" ".join(cmd),
+                                                output=self.stdout,
+                                                stderr=self.stderr)
+
+        if capture_output:
+            if proc.returncode > 0:
+                print("Error {proc.returncode}", file=sys.stderr)
+                return ""
+
+            return self.output
+
+        ret = subprocess.CompletedProcess(args=cmd,
+                                          returncode=proc.returncode,
+                                          stdout=self.stdout,
+                                          stderr=self.stderr)
+
+        return ret
+
+
+class SphinxVenv:
+    """
+    Installs Sphinx on one virtual env per Sphinx version with a minimal
+    set of dependencies, adjusting them to each specific version.
+    """
+
+    def __init__(self):
+        """Initialize instance variables"""
+
+        self.built_time = {}
+        self.first_run = True
+
+    async def _handle_version(self, args, cur_ver, cur_requirements, python_bin):
+        """Handle a single Sphinx version"""
+
+        cmd = AsyncCommands()
+
+        ver = ".".join(map(str, cur_ver))
+
+        if not self.first_run and args.wait_input and args.make:
+            ret = input("Press Enter to continue or 'a' to abort: ").strip().lower()
+            if ret == "a":
+                print("Aborted.")
+                sys.exit()
+        else:
+            self.first_run = False
+
+        venv_dir = f"Sphinx_{ver}"
+        req_file = f"requirements_{ver}.txt"
+
+        print(f"\nSphinx {ver} with {python_bin}")
+
+        # Create venv
+        await cmd.run([python_bin, "-m", "venv", venv_dir], check=True)
+        pip = os.path.join(venv_dir, "bin/pip")
+
+        # Create install list
+        reqs = []
+        for pkg, verstr in cur_requirements.items():
+            reqs.append(f"{pkg}=={verstr}")
+
+        reqs.append(f"Sphinx=={ver}")
+
+        await cmd.run([pip, "install"] + reqs, check=True, verbose=True)
+
+        # Freeze environment
+        result = await cmd.run([pip, "freeze"], verbose=False, check=True)
+
+        # Pip install succeeded. Write requirements file
+        if args.write:
+            with open(req_file, "w", encoding="utf-8") as fp:
+                fp.write(result.stdout)
+
+        if args.make:
+            start_time = time.time()
+
+            # Prepare a venv environment
+            env = os.environ.copy()
+            bin_dir = os.path.join(venv_dir, "bin")
+            env["PATH"] = bin_dir + ":" + env["PATH"]
+            env["VIRTUAL_ENV"] = venv_dir
+            if "PYTHONHOME" in env:
+                del env["PYTHONHOME"]
+
+            # Test doc build
+            await cmd.run(["make", "cleandocs"], env=env, check=True)
+            make = ["make"] + args.make_args + ["htmldocs"]
+
+            print(f". {bin_dir}/activate")
+            print(" ".join(make))
+            print("deactivate")
+            await cmd.run(make, env=env, check=True)
+
+            end_time = time.time()
+            elapsed_time = end_time - start_time
+            hours, minutes = divmod(elapsed_time, 3600)
+            minutes, seconds = divmod(minutes, 60)
+
+            hours = int(hours)
+            minutes = int(minutes)
+            seconds = int(seconds)
+
+            self.built_time[ver] = f"{hours:02d}:{minutes:02d}:{seconds:02d}"
+
+            print(f"Finished doc build for Sphinx {ver}. Elapsed time: {self.built_time[ver]}")
+
+    async def run(self, args):
+        """
+        Navigate though multiple Sphinx versions, handling each of them
+        on a loop.
+        """
+
+        cur_requirements = {}
+        python_bin = MINIMAL_PYTHON_VERSION
+
+        for cur_ver, new_reqs in SPHINX_REQUIREMENTS.items():
+            cur_requirements.update(new_reqs)
+
+            if cur_ver in PYTHON_VER_CHANGES:          # pylint: disable=R1715
+
+                python_bin = PYTHON_VER_CHANGES[cur_ver]
+
+            if args.min_version:
+                if cur_ver < args.min_version:
+                    continue
+
+            if args.max_version:
+                if cur_ver > args.max_version:
+                    break
+
+            await self._handle_version(args, cur_ver, cur_requirements,
+                                       python_bin)
+
+        if args.make:
+            print()
+            print("Summary:")
+            for ver, elapsed_time in sorted(self.built_time.items()):
+                print(f"\tSphinx {ver} elapsed time: {elapsed_time}")
+
+
 def parse_version(ver_str):
     """Convert a version string into a tuple."""
 
     return tuple(map(int, ver_str.split(".")))
 
 
-parser = argparse.ArgumentParser(description="Build docs for different sphinx_versions.")
-
-parser.add_argument('-v', '--version', help='Sphinx single version',
-                    type=parse_version)
-parser.add_argument('--min-version', "--min", help='Sphinx minimal version',
-                    type=parse_version)
-parser.add_argument('--max-version', "--max", help='Sphinx maximum version',
-                    type=parse_version)
-parser.add_argument('-a', '--make_args',
-                    help='extra arguments for make htmldocs, like SPHINXDIRS=netlink/specs',
-                    nargs="*")
-parser.add_argument('-w', '--write', help='write a requirements.txt file',
-                    action='store_true')
-parser.add_argument('-m', '--make',
-                    help='Make documentation',
-                    action='store_true')
-parser.add_argument('-i', '--wait-input',
-                    help='Wait for an enter before going to the next version',
-                    action='store_true')
-
-args = parser.parse_args()
-
-if not args.make_args:
-    args.make_args = []
-
-if args.version:
-    if args.min_version or args.max_version:
-        sys.exit("Use either --version or --min-version/--max-version")
-    else:
-        args.min_version = args.version
-        args.max_version = args.version
-
-sphinx_versions = sorted(list(sphinx_requirements.keys()))
-
-if not args.min_version:
-    args.min_version = sphinx_versions[0]
-
-if not args.max_version:
-    args.max_version = sphinx_versions[-1]
-
-first_run = True
-cur_requirements = {}
-built_time = {}
-
-for cur_ver, new_reqs in sphinx_requirements.items():
-    cur_requirements.update(new_reqs)
-
-    if cur_ver in python_changes:
-        python_bin = python_changes[cur_ver]
-
-    ver = ".".join(map(str, cur_ver))
-
-    if args.min_version:
-        if cur_ver < args.min_version:
-            continue
-
-    if args.max_version:
-        if cur_ver > args.max_version:
-            break
-
-    if not first_run and args.wait_input and args.make:
-        ret = input("Press Enter to continue or 'a' to abort: ").strip().lower()
-        if ret == "a":
-            print("Aborted.")
-            sys.exit()
-    else:
-        first_run = False
-
-    venv_dir = f"Sphinx_{ver}"
-    req_file = f"requirements_{ver}.txt"
-
-    print(f"\nSphinx {ver} with {python_bin}")
-
-    # Create venv
-    run([python_bin, "-m", "venv", venv_dir], check=True)
-    pip = os.path.join(venv_dir, "bin/pip")
-
-    # Create install list
-    reqs = []
-    for pkg, verstr in cur_requirements.items():
-        reqs.append(f"{pkg}=={verstr}")
-
-    reqs.append(f"Sphinx=={ver}")
-
-    run([pip, "install"] + reqs, check=True)
-
-    # Freeze environment
-    result = run([pip, "freeze"], capture_output=True, text=True, check=True)
-
-    # Pip install succeeded. Write requirements file
-    if args.write:
-        with open(req_file, "w", encoding="utf-8") as fp:
-            fp.write(result.stdout)
-
-    if args.make:
-        start_time = time.time()
-
-        # Prepare a venv environment
-        env = os.environ.copy()
-        bin_dir = os.path.join(venv_dir, "bin")
-        env["PATH"] = bin_dir + ":" + env["PATH"]
-        env["VIRTUAL_ENV"] = venv_dir
-        if "PYTHONHOME" in env:
-            del env["PYTHONHOME"]
-
-        # Test doc build
-        run(["make", "cleandocs"], env=env, check=True)
-        make = ["make"] + args.make_args + ["htmldocs"]
-
-        print(f". {bin_dir}/activate")
-        print(" ".join(make))
-        print("deactivate")
-        run(make, env=env, check=True)
-
-        end_time = time.time()
-        elapsed_time = end_time - start_time
-        hours, minutes = divmod(elapsed_time, 3600)
-        minutes, seconds = divmod(minutes, 60)
-
-        hours = int(hours)
-        minutes = int(minutes)
-        seconds = int(seconds)
-
-        built_time[ver] = f"{hours:02d}:{minutes:02d}:{seconds:02d}"
-
-        print(f"Finished doc build for Sphinx {ver}. Elapsed time: {built_time[ver]}")
-
-if args.make:
-    print()
-    print("Summary:")
-    for ver, elapsed_time in sorted(built_time.items()):
-        print(f"\tSphinx {ver} elapsed time: {elapsed_time}")
+async def main():
+    """Main program"""
+
+    parser = argparse.ArgumentParser(description="Build docs for different sphinx_versions.")
+
+    parser.add_argument('-v', '--version', help='Sphinx single version',
+                        type=parse_version)
+    parser.add_argument('--min-version', "--min", help='Sphinx minimal version',
+                        type=parse_version)
+    parser.add_argument('--max-version', "--max", help='Sphinx maximum version',
+                        type=parse_version)
+    parser.add_argument('-a', '--make_args',
+                        help='extra arguments for make htmldocs, like SPHINXDIRS=netlink/specs',
+                        nargs="*")
+    parser.add_argument('-w', '--write', help='write a requirements.txt file',
+                        action='store_true')
+    parser.add_argument('-m', '--make',
+                        help='Make documentation',
+                        action='store_true')
+    parser.add_argument('-i', '--wait-input',
+                        help='Wait for an enter before going to the next version',
+                        action='store_true')
+
+    args = parser.parse_args()
+
+    if not args.make_args:
+        args.make_args = []
+
+    if args.version:
+        if args.min_version or args.max_version:
+            sys.exit("Use either --version or --min-version/--max-version")
+        else:
+            args.min_version = args.version
+            args.max_version = args.version
+
+    sphinx_versions = sorted(list(SPHINX_REQUIREMENTS.keys()))
+
+    if not args.min_version:
+        args.min_version = sphinx_versions[0]
+
+    if not args.max_version:
+        args.max_version = sphinx_versions[-1]
+
+    venv = SphinxVenv()
+    await venv.run(args)
+
+
+# Call main method
+if __name__ == "__main__":
+    asyncio.run(main())
-- 
2.49.0

[PATCH v3 05/15] scripts: test_doc_build.py: better control its output

[Mauro Carvalho Chehab]

Now that asyncio is supported, allow userspace to adjust
verbosity level and direct the script output to a file.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/test_doc_build.py | 78 +++++++++++++++++++++++++--------------
 1 file changed, 51 insertions(+), 27 deletions(-)

diff --git a/scripts/test_doc_build.py b/scripts/test_doc_build.py
index 94f2f2d8c3b7..5b9eb2c0bf01 100755
--- a/scripts/test_doc_build.py
+++ b/scripts/test_doc_build.py
@@ -2,7 +2,7 @@
 # SPDX-License-Identifier: GPL-2.0
 # Copyright(c) 2025: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
 #
-# pylint: disable=R0903,R0913,R0914,R0917
+# pylint: disable=R0903,R0912,R0913,R0914,R0917,C0301
 
 """
 Install minimal supported requirements for different Sphinx versions
@@ -104,9 +104,22 @@ SPHINX_REQUIREMENTS = {
 class AsyncCommands:
     """Excecute command synchronously"""
 
-    stdout = None
-    stderr = None
-    output = None
+    def __init__(self, fp=None):
+
+        self.stdout = None
+        self.stderr = None
+        self.output = None
+        self.fp = fp
+
+    def log(self, out, verbose, is_info=True):
+        if verbose:
+            if is_info:
+                print(out.rstrip("\n"))
+            else:
+                print(out.rstrip("\n"), file=sys.stderr)
+
+        if self.fp:
+            self.fp.write(out.rstrip("\n") + "\n")
 
     async def _read(self, stream, verbose, is_info):
         """Ancillary routine to capture while displaying"""
@@ -115,16 +128,10 @@ class AsyncCommands:
             line = await stream.readline()
             if line:
                 out = line.decode("utf-8", errors="backslashreplace")
-                self.output += out
+                self.log(out, verbose, is_info)
                 if is_info:
-                    if verbose:
-                        print(out.rstrip("\n"))
-
                     self.stdout += out
                 else:
-                    if verbose:
-                        print(out.rstrip("\n"), file=sys.stderr)
-
                     self.stderr += out
             else:
                 break
@@ -140,10 +147,8 @@ class AsyncCommands:
 
         self.stdout = ""
         self.stderr = ""
-        self.output = ""
 
-        if verbose:
-            print("$ ", " ".join(cmd))
+        self.log("$ " + " ".join(cmd), verbose)
 
         proc = await asyncio.create_subprocess_exec(cmd[0],
                                                     *cmd[1:],
@@ -167,7 +172,7 @@ class AsyncCommands:
 
         if capture_output:
             if proc.returncode > 0:
-                print("Error {proc.returncode}", file=sys.stderr)
+                self.log(f"Error {proc.returncode}", verbose=True, is_info=False)
                 return ""
 
             return self.output
@@ -192,10 +197,11 @@ class SphinxVenv:
         self.built_time = {}
         self.first_run = True
 
-    async def _handle_version(self, args, cur_ver, cur_requirements, python_bin):
+    async def _handle_version(self, args, fp,
+                              cur_ver, cur_requirements, python_bin):
         """Handle a single Sphinx version"""
 
-        cmd = AsyncCommands()
+        cmd = AsyncCommands(fp)
 
         ver = ".".join(map(str, cur_ver))
 
@@ -210,10 +216,11 @@ class SphinxVenv:
         venv_dir = f"Sphinx_{ver}"
         req_file = f"requirements_{ver}.txt"
 
-        print(f"\nSphinx {ver} with {python_bin}")
+        cmd.log(f"\nSphinx {ver} with {python_bin}", verbose=True)
 
         # Create venv
-        await cmd.run([python_bin, "-m", "venv", venv_dir], check=True)
+        await cmd.run([python_bin, "-m", "venv", venv_dir],
+                      verbose=args.verbose, check=True)
         pip = os.path.join(venv_dir, "bin/pip")
 
         # Create install list
@@ -223,7 +230,7 @@ class SphinxVenv:
 
         reqs.append(f"Sphinx=={ver}")
 
-        await cmd.run([pip, "install"] + reqs, check=True, verbose=True)
+        await cmd.run([pip, "install"] + reqs, check=True, verbose=args.verbose)
 
         # Freeze environment
         result = await cmd.run([pip, "freeze"], verbose=False, check=True)
@@ -248,10 +255,11 @@ class SphinxVenv:
             await cmd.run(["make", "cleandocs"], env=env, check=True)
             make = ["make"] + args.make_args + ["htmldocs"]
 
-            print(f". {bin_dir}/activate")
-            print(" ".join(make))
-            print("deactivate")
-            await cmd.run(make, env=env, check=True)
+            if args.verbose:
+                print(f". {bin_dir}/activate")
+            await cmd.run(make, env=env, check=True, verbose=True)
+            if args.verbose:
+                print("deactivate")
 
             end_time = time.time()
             elapsed_time = end_time - start_time
@@ -264,7 +272,7 @@ class SphinxVenv:
 
             self.built_time[ver] = f"{hours:02d}:{minutes:02d}:{seconds:02d}"
 
-            print(f"Finished doc build for Sphinx {ver}. Elapsed time: {self.built_time[ver]}")
+            cmd.log(f"Finished doc build for Sphinx {ver}. Elapsed time: {self.built_time[ver]}", verbose=True)
 
     async def run(self, args):
         """
@@ -272,6 +280,15 @@ class SphinxVenv:
         on a loop.
         """
 
+        if args.log:
+            fp = open(args.log, "w", encoding="utf-8")
+            if not args.verbose:
+                args.verbose = False
+        else:
+            fp = None
+            if not args.verbose:
+                args.verbose = True
+
         cur_requirements = {}
         python_bin = MINIMAL_PYTHON_VERSION
 
@@ -290,7 +307,7 @@ class SphinxVenv:
                 if cur_ver > args.max_version:
                     break
 
-            await self._handle_version(args, cur_ver, cur_requirements,
+            await self._handle_version(args, fp, cur_ver, cur_requirements,
                                        python_bin)
 
         if args.make:
@@ -299,6 +316,8 @@ class SphinxVenv:
             for ver, elapsed_time in sorted(self.built_time.items()):
                 print(f"\tSphinx {ver} elapsed time: {elapsed_time}")
 
+        if fp:
+            fp.close()
 
 def parse_version(ver_str):
     """Convert a version string into a tuple."""
@@ -311,7 +330,7 @@ async def main():
 
     parser = argparse.ArgumentParser(description="Build docs for different sphinx_versions.")
 
-    parser.add_argument('-v', '--version', help='Sphinx single version',
+    parser.add_argument('-V', '--version', help='Sphinx single version',
                         type=parse_version)
     parser.add_argument('--min-version', "--min", help='Sphinx minimal version',
                         type=parse_version)
@@ -328,6 +347,11 @@ async def main():
     parser.add_argument('-i', '--wait-input',
                         help='Wait for an enter before going to the next version',
                         action='store_true')
+    parser.add_argument('-v', '--verbose',
+                        help='Verbose all commands',
+                        action='store_true')
+    parser.add_argument('-l', '--log',
+                        help='Log command output on a file')
 
     args = parser.parse_args()
 
-- 
2.49.0

[PATCH v3 06/15] scripts: test_doc_build.py: better adjust to python version

[Mauro Carvalho Chehab]

Very old versions of Sphinx require older versions of python.
The original script assumes that a python3.9 exec exists,
but this may not be the case.

Relax python requirements.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/test_doc_build.py | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/scripts/test_doc_build.py b/scripts/test_doc_build.py
index 5b9eb2c0bf01..129c2862065c 100755
--- a/scripts/test_doc_build.py
+++ b/scripts/test_doc_build.py
@@ -12,15 +12,28 @@ and optionally test the build.
 import argparse
 import asyncio
 import os.path
+import shutil
 import sys
 import time
 import subprocess
 
-# Minimal python version supported by the building system
-MINIMAL_PYTHON_VERSION = "python3.9"
+# Minimal python version supported by the building system.
 
-# Starting from 8.0.2, Python 3.9 becomes too old
-PYTHON_VER_CHANGES = {(8, 0, 2): "python3"}
+PYTHON = os.path.basename(sys.executable)
+
+min_python_bin = None
+
+for i in range(9, 13):
+    p = f"python3.{i}"
+    if shutil.which(p):
+        min_python_bin = p
+        break
+
+if not min_python_bin:
+    min_python_bin = PYTHON
+
+# Starting from 8.0, Python 3.9 is not supported anymore.
+PYTHON_VER_CHANGES = {(8, 0, 2): PYTHON}
 
 # Sphinx versions to be installed and their incremental requirements
 SPHINX_REQUIREMENTS = {
@@ -290,7 +303,7 @@ class SphinxVenv:
                 args.verbose = True
 
         cur_requirements = {}
-        python_bin = MINIMAL_PYTHON_VERSION
+        python_bin = min_python_bin
 
         for cur_ver, new_reqs in SPHINX_REQUIREMENTS.items():
             cur_requirements.update(new_reqs)
-- 
2.49.0

[PATCH v3 07/15] scripts: test_doc_build.py: improve dependency list

[Mauro Carvalho Chehab]

Change the dependency list to ensure that:
- all docutils versions are covered;
- provide an explanation about the dependencies;
- set a better minimal requirement for 3.4.3.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/test_doc_build.py | 68 ++++++++++++++++++++++++++++-----------
 1 file changed, 50 insertions(+), 18 deletions(-)

diff --git a/scripts/test_doc_build.py b/scripts/test_doc_build.py
index 129c2862065c..087b8c476f05 100755
--- a/scripts/test_doc_build.py
+++ b/scripts/test_doc_build.py
@@ -37,70 +37,102 @@ PYTHON_VER_CHANGES = {(8, 0, 2): PYTHON}
 
 # Sphinx versions to be installed and their incremental requirements
 SPHINX_REQUIREMENTS = {
+    # Oldest versions we support for each package required by Sphinx 3.4.3
     (3, 4, 3): {
+        "docutils": "0.16",
+        "alabaster": "0.7.12",
+        "babel": "2.8.0",
+        "certifi": "2020.6.20",
+        "docutils": "0.16",
+        "idna": "2.10",
+        "imagesize": "1.2.0",
+        "Jinja2": "2.11.2",
+        "MarkupSafe": "1.1.1",
+        "packaging": "20.4",
+        "Pygments": "2.6.1",
+        "PyYAML": "5.1",
+        "requests": "2.24.0",
+        "snowballstemmer": "2.0.0",
+        "sphinxcontrib-applehelp": "1.0.2",
+        "sphinxcontrib-devhelp": "1.0.2",
+        "sphinxcontrib-htmlhelp": "1.0.3",
+        "sphinxcontrib-jsmath": "1.0.1",
+        "sphinxcontrib-qthelp": "1.0.3",
+        "sphinxcontrib-serializinghtml": "1.1.4",
+        "urllib3": "1.25.9",
+    },
+
+    # Update package dependencies to a more modern base. The goal here
+    # is to avoid to many incremental changes for the next entries
+    (3, 5, 4): {
         "alabaster": "0.7.13",
         "babel": "2.17.0",
         "certifi": "2025.6.15",
-        "charset-normalizer": "3.4.2",
-        "docutils": "0.15",
         "idna": "3.10",
         "imagesize": "1.4.1",
         "Jinja2": "3.0.3",
         "MarkupSafe": "2.0",
         "packaging": "25.0",
         "Pygments": "2.19.1",
-        "PyYAML": "5.1",
         "requests": "2.32.4",
         "snowballstemmer": "3.0.1",
         "sphinxcontrib-applehelp": "1.0.4",
-        "sphinxcontrib-devhelp": "1.0.2",
         "sphinxcontrib-htmlhelp": "2.0.1",
-        "sphinxcontrib-jsmath": "1.0.1",
-        "sphinxcontrib-qthelp": "1.0.3",
         "sphinxcontrib-serializinghtml": "1.1.5",
-        "urllib3": "2.4.0",
     },
-    (3, 5, 4): {},
+
+    # Starting from here, ensure all docutils versions are covered with
+    # supported Sphinx versions. Other packages are upgraded only when
+    # required by pip
     (4, 0, 3): {
-        "docutils": "0.17.1",
+        "docutils": "0.17",
         "PyYAML": "5.1",
     },
-    (4, 1, 2): {},
-    (4, 3, 2): {},
+    (4, 1, 2): {
+    },
+    (4, 3, 2): {
+    },
     (4, 4, 0): {},
     (4, 5, 0): {},
     (5, 0, 2): {},
     (5, 1, 1): {},
     (5, 2, 3): {
+        "docutils": "0.17.1",
         "Jinja2": "3.1.2",
         "MarkupSafe": "2.0",
         "PyYAML": "5.3.1",
     },
-    (5, 3, 0): {
-        "docutils": "0.18.1",
-        "PyYAML": "5.3.1",
+    (5, 3, 0): {},
+    (6, 0, 1): {
+        "docutils": "0.18",
     },
-    (6, 0, 1): {},
     (6, 1, 3): {},
     (6, 2, 1): {
+        "docutils": "0.18.1",
         "PyYAML": "5.4.1",
     },
-    (7, 0, 1): {},
+    (7, 0, 1): {
+    },
     (7, 1, 2): {},
     (7, 2, 3): {
+        "docutils": "0.19",
         "PyYAML": "6.0.1",
         "sphinxcontrib-serializinghtml": "1.1.9",
     },
     (7, 3, 7): {
+        "docutils": "0.20",
         "alabaster": "0.7.14",
         "PyYAML": "6.0.1",
     },
     (7, 4, 7): {
-        "docutils": "0.20",
+        "docutils": "0.21",
         "PyYAML": "6.0.1",
     },
-    (8, 0, 2): {},
+    (8, 0, 2): {
+        "docutils": "0.21.1",
+    },
     (8, 1, 3): {
+        "docutils": "0.21.2",
         "PyYAML": "6.0.1",
         "sphinxcontrib-applehelp": "1.0.7",
         "sphinxcontrib-devhelp": "1.0.6",
-- 
2.49.0

[PATCH v3 08/15] scripts: test_doc_build.py: improve cmd.log logic

[Mauro Carvalho Chehab]

Simplify the logic which handles with new lines.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/test_doc_build.py | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/scripts/test_doc_build.py b/scripts/test_doc_build.py
index 087b8c476f05..7ea6add48f9a 100755
--- a/scripts/test_doc_build.py
+++ b/scripts/test_doc_build.py
@@ -157,14 +157,16 @@ class AsyncCommands:
         self.fp = fp
 
     def log(self, out, verbose, is_info=True):
+        out = out.removesuffix('\n')
+
         if verbose:
             if is_info:
-                print(out.rstrip("\n"))
+                print(out)
             else:
-                print(out.rstrip("\n"), file=sys.stderr)
+                print(out, file=sys.stderr)
 
         if self.fp:
-            self.fp.write(out.rstrip("\n") + "\n")
+            self.fp.write(out + "\n")
 
     async def _read(self, stream, verbose, is_info):
         """Ancillary routine to capture while displaying"""
@@ -301,10 +303,10 @@ class SphinxVenv:
             make = ["make"] + args.make_args + ["htmldocs"]
 
             if args.verbose:
-                print(f". {bin_dir}/activate")
+                cmd.log(f". {bin_dir}/activate", verbose=True)
             await cmd.run(make, env=env, check=True, verbose=True)
             if args.verbose:
-                print("deactivate")
+                cmd.log("deactivate", verbose=True)
 
             end_time = time.time()
             elapsed_time = end_time - start_time
@@ -356,10 +358,11 @@ class SphinxVenv:
                                        python_bin)
 
         if args.make:
-            print()
-            print("Summary:")
+            cmd = AsyncCommands(fp)
+            cmd.log("\nSummary:", verbose=True)
             for ver, elapsed_time in sorted(self.built_time.items()):
-                print(f"\tSphinx {ver} elapsed time: {elapsed_time}")
+                cmd.log(f"\tSphinx {ver} elapsed time: {elapsed_time}",
+                        verbose=True)
 
         if fp:
             fp.close()
-- 
2.49.0

[PATCH v3 09/15] scripts: test_doc_build.py: make the script smarter

[Mauro Carvalho Chehab]

Most of the time, testing the full range of supported Sphinx
version is a waste of time and resources. Instead, the best is
to focus at the versions that are actually shipped by major
distros.

For it to work properly, we need to adjust the requirements for
them to start from first patch for each distro after the
minimal supported one. The requirements were re-adjusted to
avoid build breakages related to version incompatibilities.
Such builds were tested with:

	./scripts/test_doc_build.py -m -a "SPHINXOPTS=-j8" "SPHINXDIRS=networking netlink/specs" --full

Change the logic to pick by default only such versions, adding
another parameter to do a comprehensive test.

While here, improve the script documentation to make it easier
to be used.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/test_doc_build.py | 156 ++++++++++++++++++++++++++------------
 1 file changed, 106 insertions(+), 50 deletions(-)

diff --git a/scripts/test_doc_build.py b/scripts/test_doc_build.py
index 7ea6add48f9a..5e905a350bd0 100755
--- a/scripts/test_doc_build.py
+++ b/scripts/test_doc_build.py
@@ -33,7 +33,19 @@ if not min_python_bin:
     min_python_bin = PYTHON
 
 # Starting from 8.0, Python 3.9 is not supported anymore.
-PYTHON_VER_CHANGES = {(8, 0, 2): PYTHON}
+PYTHON_VER_CHANGES = {(8, 0, 0): PYTHON}
+
+DEFAULT_VERSIONS_TO_TEST = [
+    (3, 4, 3),   # Minimal supported version
+    (5, 3, 0),   # CentOS Stream 9 / AlmaLinux 9
+    (6, 1, 1),   # Debian 12
+    (7, 2, 1),   # openSUSE Leap 15.6
+    (7, 2, 6),   # Ubuntu 24.04 LTS
+    (7, 4, 7),   # Ubuntu 24.10
+    (7, 3, 0),   # openSUSE Tumbleweed
+    (8, 1, 3),   # Fedora 42
+    (8, 2, 3)    # Latest version - covers rolling distros
+]
 
 # Sphinx versions to be installed and their incremental requirements
 SPHINX_REQUIREMENTS = {
@@ -64,82 +76,87 @@ SPHINX_REQUIREMENTS = {
 
     # Update package dependencies to a more modern base. The goal here
     # is to avoid to many incremental changes for the next entries
-    (3, 5, 4): {
+    (3, 5, 0): {
         "alabaster": "0.7.13",
         "babel": "2.17.0",
         "certifi": "2025.6.15",
         "idna": "3.10",
         "imagesize": "1.4.1",
-        "Jinja2": "3.0.3",
-        "MarkupSafe": "2.0",
         "packaging": "25.0",
-        "Pygments": "2.19.1",
+        "Pygments": "2.8.1",
         "requests": "2.32.4",
         "snowballstemmer": "3.0.1",
         "sphinxcontrib-applehelp": "1.0.4",
         "sphinxcontrib-htmlhelp": "2.0.1",
         "sphinxcontrib-serializinghtml": "1.1.5",
+        "urllib3": "2.0.0",
     },
 
     # Starting from here, ensure all docutils versions are covered with
     # supported Sphinx versions. Other packages are upgraded only when
     # required by pip
-    (4, 0, 3): {
+    (4, 0, 0): {
+        "PyYAML": "5.1",
+    },
+    (4, 1, 0): {
         "docutils": "0.17",
-        "PyYAML": "5.1",
-    },
-    (4, 1, 2): {
-    },
-    (4, 3, 2): {
+        "Pygments": "2.19.1",
+        "Jinja2": "3.0.3",
+        "MarkupSafe": "2.0",
     },
+    (4, 3, 0): {},
     (4, 4, 0): {},
-    (4, 5, 0): {},
-    (5, 0, 2): {},
-    (5, 1, 1): {},
-    (5, 2, 3): {
+    (4, 5, 0): {
         "docutils": "0.17.1",
+    },
+    (5, 0, 0): {},
+    (5, 1, 0): {},
+    (5, 2, 0): {
+        "docutils": "0.18",
         "Jinja2": "3.1.2",
         "MarkupSafe": "2.0",
         "PyYAML": "5.3.1",
     },
-    (5, 3, 0): {},
-    (6, 0, 1): {
-        "docutils": "0.18",
-    },
-    (6, 1, 3): {},
-    (6, 2, 1): {
+    (5, 3, 0): {
         "docutils": "0.18.1",
+    },
+    (6, 0, 0): {},
+    (6, 1, 0): {},
+    (6, 2, 0): {
         "PyYAML": "5.4.1",
     },
-    (7, 0, 1): {
-    },
-    (7, 1, 2): {},
-    (7, 2, 3): {
+    (7, 0, 0): {},
+    (7, 1, 0): {},
+    (7, 2, 0): {
         "docutils": "0.19",
         "PyYAML": "6.0.1",
         "sphinxcontrib-serializinghtml": "1.1.9",
     },
-    (7, 3, 7): {
+    (7, 2, 6): {
         "docutils": "0.20",
+    },
+    (7, 3, 0): {
         "alabaster": "0.7.14",
         "PyYAML": "6.0.1",
+        "tomli": "2.0.1",
     },
-    (7, 4, 7): {
+    (7, 4, 0): {
+        "docutils": "0.20.1",
+        "PyYAML": "6.0.1",
+    },
+    (8, 0, 0): {
         "docutils": "0.21",
-        "PyYAML": "6.0.1",
     },
-    (8, 0, 2): {
+    (8, 1, 0): {
         "docutils": "0.21.1",
-    },
-    (8, 1, 3): {
-        "docutils": "0.21.2",
         "PyYAML": "6.0.1",
         "sphinxcontrib-applehelp": "1.0.7",
         "sphinxcontrib-devhelp": "1.0.6",
         "sphinxcontrib-htmlhelp": "2.0.6",
         "sphinxcontrib-qthelp": "1.0.6",
     },
-    (8, 2, 3): {
+    (8, 2, 0): {
+        "docutils": "0.21.2",
         "PyYAML": "6.0.1",
         "sphinxcontrib-serializinghtml": "1.1.9",
     },
@@ -339,13 +356,19 @@ class SphinxVenv:
         cur_requirements = {}
         python_bin = min_python_bin
 
-        for cur_ver, new_reqs in SPHINX_REQUIREMENTS.items():
-            cur_requirements.update(new_reqs)
+        vers = set(SPHINX_REQUIREMENTS.keys()) | set(args.versions)
+
+        for cur_ver in sorted(vers):
+            if cur_ver in SPHINX_REQUIREMENTS:
+                new_reqs = SPHINX_REQUIREMENTS[cur_ver]
+                cur_requirements.update(new_reqs)
 
             if cur_ver in PYTHON_VER_CHANGES:          # pylint: disable=R1715
-
                 python_bin = PYTHON_VER_CHANGES[cur_ver]
 
+            if cur_ver not in args.versions:
+                continue
+
             if args.min_version:
                 if cur_ver < args.min_version:
                     continue
@@ -373,12 +396,52 @@ def parse_version(ver_str):
     return tuple(map(int, ver_str.split(".")))
 
 
+DEFAULT_VERS = "    - "
+DEFAULT_VERS += "\n    - ".join(map(lambda v: f"{v[0]}.{v[1]}.{v[2]}",
+                                    DEFAULT_VERSIONS_TO_TEST))
+
+SCRIPT = os.path.relpath(__file__)
+
+DESCRIPTION = f"""
+This tool allows creating Python virtual environments for different
+Sphinx versions that are supported by the Linux Kernel build system.
+
+Besides creating the virtual environment, it can also test building
+the documentation using "make htmldocs".
+
+If called without "--versions" argument, it covers the versions shipped
+on major distros, plus the lowest supported version:
+
+{DEFAULT_VERS}
+
+A typical usage is to run:
+
+   {SCRIPT} -m -l sphinx_builds.log
+
+This will create one virtual env for the default version set and do a
+full htmldocs build for each version, creating a log file with the
+excecuted commands on it.
+
+NOTE: The build time can be very long, specially on old versions. Also, there
+is a known bug with Sphinx version 6.0.x: each subprocess uses a lot of
+memory. That, together with "-jauto" may cause OOM killer to cause
+failures at the doc generation. To minimize the risk, you may use the
+"-a" command line parameter to constrain the built directories and/or
+reduce the number of threads from "-jauto" to, for instance, "-j4":
+
+    {SCRIPT} -m -V 6.0.1 -a "SPHINXDIRS=process" "SPHINXOPTS='-j4'"
+
+"""
+
+
 async def main():
     """Main program"""
 
-    parser = argparse.ArgumentParser(description="Build docs for different sphinx_versions.")
+    parser = argparse.ArgumentParser(description=DESCRIPTION,
+                                     formatter_class=argparse.RawDescriptionHelpFormatter)
 
-    parser.add_argument('-V', '--version', help='Sphinx single version',
+    parser.add_argument('-V', '--versions', help='Sphinx versions to test',
+                        nargs="*", default=DEFAULT_VERSIONS_TO_TEST,
                         type=parse_version)
     parser.add_argument('--min-version', "--min", help='Sphinx minimal version',
                         type=parse_version)
@@ -392,6 +455,9 @@ async def main():
     parser.add_argument('-m', '--make',
                         help='Make documentation',
                         action='store_true')
+    parser.add_argument('-f', '--full',
+                        help='Add all (major,minor,latest_patch) version to the version list',
+                        action='store_true')
     parser.add_argument('-i', '--wait-input',
                         help='Wait for an enter before going to the next version',
                         action='store_true')
@@ -406,20 +472,10 @@ async def main():
     if not args.make_args:
         args.make_args = []
 
-    if args.version:
-        if args.min_version or args.max_version:
-            sys.exit("Use either --version or --min-version/--max-version")
-        else:
-            args.min_version = args.version
-            args.max_version = args.version
-
     sphinx_versions = sorted(list(SPHINX_REQUIREMENTS.keys()))
 
-    if not args.min_version:
-        args.min_version = sphinx_versions[0]
-
-    if not args.max_version:
-        args.max_version = sphinx_versions[-1]
+    if args.full:
+        args.versions += list(SPHINX_REQUIREMENTS.keys())
 
     venv = SphinxVenv()
     await venv.run(args)
-- 
2.49.0

[PATCH v3 10/15] scripts: sphinx-pre-install: properly handle SPHINXBUILD

[Mauro Carvalho Chehab]

Currently, the script ignores SPHINXBUILD, making it useless.
As we're about to use on another script, fix support for it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/sphinx-pre-install | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index ad9945ccb0cf..2a311ed00179 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -245,6 +245,10 @@ sub check_missing_tex($)
 
 sub get_sphinx_fname()
 {
+	if ($ENV{'SPHINXBUILD'}) {
+	    return $ENV{'SPHINXBUILD'};
+	}
+
 	my $fname = "sphinx-build";
 	return $fname if findprog($fname);
 
-- 
2.49.0

[PATCH v3 11/15] scripts: sphinx-pre-install: fix release detection for Fedora

[Mauro Carvalho Chehab]

Fedora distros are now identified as:

	Fedora Linux 42

Fix the way script detects it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/sphinx-pre-install | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 2a311ed00179..3f8d6925e896 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -413,7 +413,7 @@ sub give_redhat_hints()
 	my $old = 0;
 	my $rel;
 	my $noto_sans_redhat = "google-noto-sans-cjk-ttc-fonts";
-	$rel = $1 if ($system_release =~ /release\s+(\d+)/);
+	$rel = $1 if ($system_release =~ /(release|Linux)\s+(\d+)/);
 
 	if (!($system_release =~ /Fedora/)) {
 		$map{"virtualenv"} = "python-virtualenv";
-- 
2.49.0

[PATCH v3 12/15] scripts: test_doc_build.py: regroup and rename arguments

[Mauro Carvalho Chehab]

The script now have lots or arguments. Better organize and
name them, for it to be a little bit more intuitive.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/test_doc_build.py | 95 +++++++++++++++++++++++++--------------
 1 file changed, 61 insertions(+), 34 deletions(-)

diff --git a/scripts/test_doc_build.py b/scripts/test_doc_build.py
index 5e905a350bd0..47b4606569f9 100755
--- a/scripts/test_doc_build.py
+++ b/scripts/test_doc_build.py
@@ -269,7 +269,7 @@ class SphinxVenv:
 
         ver = ".".join(map(str, cur_ver))
 
-        if not self.first_run and args.wait_input and args.make:
+        if not self.first_run and args.wait_input and args.build:
             ret = input("Press Enter to continue or 'a' to abort: ").strip().lower()
             if ret == "a":
                 print("Aborted.")
@@ -300,11 +300,11 @@ class SphinxVenv:
         result = await cmd.run([pip, "freeze"], verbose=False, check=True)
 
         # Pip install succeeded. Write requirements file
-        if args.write:
+        if args.req_file:
             with open(req_file, "w", encoding="utf-8") as fp:
                 fp.write(result.stdout)
 
-        if args.make:
+        if args.build:
             start_time = time.time()
 
             # Prepare a venv environment
@@ -317,7 +317,16 @@ class SphinxVenv:
 
             # Test doc build
             await cmd.run(["make", "cleandocs"], env=env, check=True)
-            make = ["make"] + args.make_args + ["htmldocs"]
+            make = ["make"]
+
+            if args.output:
+                sphinx_build = os.path.realpath(f"{bin_dir}/sphinx-build")
+                make += [f"O={args.output}", f"SPHINXBUILD={sphinx_build}"]
+
+            if args.make_args:
+                make += args.make_args
+
+            make += args.targets
 
             if args.verbose:
                 cmd.log(f". {bin_dir}/activate", verbose=True)
@@ -380,7 +389,7 @@ class SphinxVenv:
             await self._handle_version(args, fp, cur_ver, cur_requirements,
                                        python_bin)
 
-        if args.make:
+        if args.build:
             cmd = AsyncCommands(fp)
             cmd.log("\nSummary:", verbose=True)
             for ver, elapsed_time in sorted(self.built_time.items()):
@@ -407,7 +416,7 @@ This tool allows creating Python virtual environments for different
 Sphinx versions that are supported by the Linux Kernel build system.
 
 Besides creating the virtual environment, it can also test building
-the documentation using "make htmldocs".
+the documentation using "make htmldocs" (and/or other doc targets).
 
 If called without "--versions" argument, it covers the versions shipped
 on major distros, plus the lowest supported version:
@@ -418,8 +427,8 @@ A typical usage is to run:
 
    {SCRIPT} -m -l sphinx_builds.log
 
-This will create one virtual env for the default version set and do a
-full htmldocs build for each version, creating a log file with the
+This will create one virtual env for the default version set and run
+"make htmldocs" for each version, creating a log file with the
 excecuted commands on it.
 
 NOTE: The build time can be very long, specially on old versions. Also, there
@@ -433,6 +442,15 @@ reduce the number of threads from "-jauto" to, for instance, "-j4":
 
 """
 
+MAKE_TARGETS = [
+    "htmldocs",
+    "texinfodocs",
+    "infodocs",
+    "latexdocs",
+    "pdfdocs",
+    "epubdocs",
+    "xmldocs",
+]
 
 async def main():
     """Main program"""
@@ -440,32 +458,41 @@ async def main():
     parser = argparse.ArgumentParser(description=DESCRIPTION,
                                      formatter_class=argparse.RawDescriptionHelpFormatter)
 
-    parser.add_argument('-V', '--versions', help='Sphinx versions to test',
-                        nargs="*", default=DEFAULT_VERSIONS_TO_TEST,
-                        type=parse_version)
-    parser.add_argument('--min-version', "--min", help='Sphinx minimal version',
-                        type=parse_version)
-    parser.add_argument('--max-version', "--max", help='Sphinx maximum version',
-                        type=parse_version)
-    parser.add_argument('-a', '--make_args',
-                        help='extra arguments for make htmldocs, like SPHINXDIRS=netlink/specs',
-                        nargs="*")
-    parser.add_argument('-w', '--write', help='write a requirements.txt file',
-                        action='store_true')
-    parser.add_argument('-m', '--make',
-                        help='Make documentation',
-                        action='store_true')
-    parser.add_argument('-f', '--full',
-                        help='Add all (major,minor,latest_patch) version to the version list',
-                        action='store_true')
-    parser.add_argument('-i', '--wait-input',
-                        help='Wait for an enter before going to the next version',
-                        action='store_true')
-    parser.add_argument('-v', '--verbose',
-                        help='Verbose all commands',
-                        action='store_true')
-    parser.add_argument('-l', '--log',
-                        help='Log command output on a file')
+    ver_group = parser.add_argument_group("Version range options")
+
+    ver_group.add_argument('-V', '--versions', nargs="*",
+                           default=DEFAULT_VERSIONS_TO_TEST,type=parse_version,
+                           help='Sphinx versions to test')
+    ver_group.add_argument('--min-version', "--min", type=parse_version,
+                           help='Sphinx minimal version')
+    ver_group.add_argument('--max-version', "--max", type=parse_version,
+                           help='Sphinx maximum version')
+    ver_group.add_argument('-f', '--full', action='store_true',
+                           help='Add all Sphinx (major,minor) supported versions to the version range')
+
+    build_group = parser.add_argument_group("Build options")
+
+    build_group.add_argument('-b', '--build', action='store_true',
+                             help='Build documentation')
+    build_group.add_argument('-a', '--make-args', nargs="*",
+                             help='extra arguments for make, like SPHINXDIRS=netlink/specs',
+                        )
+    build_group.add_argument('-t', '--targets', nargs="+", choices=MAKE_TARGETS,
+                             default=[MAKE_TARGETS[0]],
+                             help="make build targets. Default: htmldocs.")
+    build_group.add_argument("-o", '--output',
+                             help="output directory for the make O=OUTPUT")
+
+    other_group = parser.add_argument_group("Other options")
+
+    other_group.add_argument('-r', '--req-file', action='store_true',
+                             help='write a requirements.txt file')
+    other_group.add_argument('-l', '--log',
+                             help='Log command output on a file')
+    other_group.add_argument('-v', '--verbose', action='store_true',
+                             help='Verbose all commands')
+    other_group.add_argument('-i', '--wait-input', action='store_true',
+                        help='Wait for an enter before going to the next version')
 
     args = parser.parse_args()
 
-- 
2.49.0

[PATCH v3 13/15] docs: sphinx: add a file with the requirements for lowest version

[Mauro Carvalho Chehab]

Those days, it is hard to install a virtual env that would
build docs with Sphinx 3.4.3, as even python 3.13 is not
compatible anymore with it.

	/usr/bin/python3.9 -m venv sphinx_3.4.3
	. sphinx_3.4.3/bin/activate
	pip install -r Documentation/sphinx/min_requirements.txt

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/doc-guide/sphinx.rst        | 23 +++++++++++++++++++++++
 Documentation/sphinx/min_requirements.txt | 10 ++++++++++
 2 files changed, 33 insertions(+)
 create mode 100644 Documentation/sphinx/min_requirements.txt

diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index 5a91df105141..607589592bfb 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -131,6 +131,29 @@ It supports two optional parameters:
 ``--no-virtualenv``
 	Use OS packaging for Sphinx instead of Python virtual environment.
 
+Installing Sphinx Minimal Version
+---------------------------------
+
+When changing Sphinx build system, it is important to ensure that
+the minimal version will still be supported. Nowadays, it is
+becoming harder to do that on modern distributions, as it is not
+possible to install with Python 3.13 and above.
+
+Testing with the lowest supported Python version as defined at
+Documentation/process/changes.rst can be done by creating
+a venv with it with, and install minimal requirements with::
+
+	/usr/bin/python3.9 -m venv sphinx_min
+	. sphinx_min/bin/activate
+	pip install -r Documentation/sphinx/min_requirements.txt
+
+A more comprehensive test can be done by using:
+
+	scripts/test_doc_build.py
+
+Such script create one Python venv per supported version,
+optionally building documentation for a range of Sphinx versions.
+
 
 Sphinx Build
 ============
diff --git a/Documentation/sphinx/min_requirements.txt b/Documentation/sphinx/min_requirements.txt
new file mode 100644
index 000000000000..52d9f27010e8
--- /dev/null
+++ b/Documentation/sphinx/min_requirements.txt
@@ -0,0 +1,10 @@
+alabaster >=0.7,<0.8
+docutils>=0.15,<0.18
+jinja2>=2.3,<3.1
+PyYAML>=5.1,<6.1
+Sphinx==3.4.3
+sphinxcontrib-applehelp==1.0.2
+sphinxcontrib-devhelp==1.0.1
+sphinxcontrib-htmlhelp==1.0.3
+sphinxcontrib-qthelp==1.0.2
+sphinxcontrib-serializinghtml==1.1.4
-- 
2.49.0

[PATCH v3 14/15] docs: conf.py: several coding style fixes

[Mauro Carvalho Chehab]

conf.py is missing a SPDX header and doesn't really have
a proper python coding style. It also has an obsolete
commented LaTeX syntax that doesn't work anymore.

Clean it up a little bit with some help from autolints
and manual adjustments.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/conf.py | 351 +++++++++++++++++++++---------------------
 1 file changed, 172 insertions(+), 179 deletions(-)

diff --git a/Documentation/conf.py b/Documentation/conf.py
index 91ce2b1c33cc..700516238d3f 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -1,24 +1,28 @@
-# -*- coding: utf-8 -*-
-#
-# The Linux Kernel documentation build configuration file, created by
-# sphinx-quickstart on Fri Feb 12 13:51:46 2016.
-#
-# This file is execfile()d with the current directory set to its
-# containing dir.
-#
-# Note that not all possible configuration values are present in this
-# autogenerated file.
-#
-# All configuration values have a default; values that are commented out
-# serve to show the default.
+# SPDX-License-Identifier: GPL-2.0-only
+# pylint: disable=C0103,C0209
+
+"""
+The Linux Kernel documentation build configuration file.
+"""
 
-import sys
 import os
-import sphinx
 import shutil
+import sys
+
+import sphinx
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+sys.path.insert(0, os.path.abspath("sphinx"))
+
+from load_config import loadConfig               # pylint: disable=C0413,E0401
+
+# Minimal supported version
+needs_sphinx = "3.4.3"
 
 # Get Sphinx version
-major, minor, patch = sphinx.version_info[:3]
+major, minor, patch = sphinx.version_info[:3]          # pylint: disable=I1101
 
 # Include_patterns were added on Sphinx 5.1
 if (major < 5) or (major == 5 and minor < 1):
@@ -26,23 +30,22 @@ if (major < 5) or (major == 5 and minor < 1):
 else:
     has_include_patterns = True
     # Include patterns that don't contain directory names, in glob format
-    include_patterns = ['**.rst']
+    include_patterns = ["**.rst"]
 
 # Location of Documentation/ directory
-doctree = os.path.abspath('.')
+doctree = os.path.abspath(".")
 
 # Exclude of patterns that don't contain directory names, in glob format.
 exclude_patterns = []
 
 # List of patterns that contain directory names in glob format.
 dyn_include_patterns = []
-dyn_exclude_patterns = ['output']
+dyn_exclude_patterns = ["output"]
 
 # Properly handle include/exclude patterns
 # ----------------------------------------
 
 def update_patterns(app, config):
-
     """
     On Sphinx, all directories are relative to what it is passed as
     SOURCEDIR parameter for sphinx-build. Due to that, all patterns
@@ -53,15 +56,12 @@ def update_patterns(app, config):
     exclude relative patterns that start with "../".
     """
 
-    sourcedir = app.srcdir  # full path to the source directory
-    builddir = os.environ.get("BUILDDIR")
-
     # setup include_patterns dynamically
     if has_include_patterns:
         for p in dyn_include_patterns:
             full = os.path.join(doctree, p)
 
-            rel_path = os.path.relpath(full, start = app.srcdir)
+            rel_path = os.path.relpath(full, start=app.srcdir)
             if rel_path.startswith("../"):
                 continue
 
@@ -71,15 +71,17 @@ def update_patterns(app, config):
     for p in dyn_exclude_patterns:
         full = os.path.join(doctree, p)
 
-        rel_path = os.path.relpath(full, start = app.srcdir)
+        rel_path = os.path.relpath(full, start=app.srcdir)
         if rel_path.startswith("../"):
             continue
 
         config.exclude_patterns.append(rel_path)
 
+
 # helper
 # ------
 
+
 def have_command(cmd):
     """Search ``cmd`` in the ``PATH`` environment.
 
@@ -88,24 +90,23 @@ def have_command(cmd):
     """
     return shutil.which(cmd) is not None
 
-# If extensions (or modules to document with autodoc) are in another directory,
-# add these directories to sys.path here. If the directory is relative to the
-# documentation root, use os.path.abspath to make it absolute, like shown here.
-sys.path.insert(0, os.path.abspath('sphinx'))
-from load_config import loadConfig
 
 # -- General configuration ------------------------------------------------
 
-# If your documentation needs a minimal Sphinx version, state it here.
-needs_sphinx = '3.4.3'
-
-# Add any Sphinx extension module names here, as strings. They can be
-# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
-# ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include',
-              'kfigure', 'sphinx.ext.ifconfig', 'automarkup',
-              'maintainers_include', 'sphinx.ext.autosectionlabel',
-              'kernel_abi', 'kernel_feat', 'translations']
+# Add any Sphinx extensions in alphabetic order
+extensions = [
+    "automarkup",
+    "kernel_abi",
+    "kerneldoc",
+    "kernel_feat",
+    "kernel_include",
+    "kfigure",
+    "maintainers_include",
+    "rstFlatTable",
+    "sphinx.ext.autosectionlabel",
+    "sphinx.ext.ifconfig",
+    "translations",
+]
 
 # Since Sphinx version 3, the C function parser is more pedantic with regards
 # to type checking. Due to that, having macros at c:function cause problems.
@@ -180,28 +181,28 @@ autosectionlabel_maxdepth = 2
 # Load math renderer:
 # For html builder, load imgmath only when its dependencies are met.
 # mathjax is the default math renderer since Sphinx 1.8.
-have_latex =  have_command('latex')
-have_dvipng = have_command('dvipng')
+have_latex = have_command("latex")
+have_dvipng = have_command("dvipng")
 load_imgmath = have_latex and have_dvipng
 
 # Respect SPHINX_IMGMATH (for html docs only)
-if 'SPHINX_IMGMATH' in os.environ:
-    env_sphinx_imgmath = os.environ['SPHINX_IMGMATH']
-    if 'yes' in env_sphinx_imgmath:
+if "SPHINX_IMGMATH" in os.environ:
+    env_sphinx_imgmath = os.environ["SPHINX_IMGMATH"]
+    if "yes" in env_sphinx_imgmath:
         load_imgmath = True
-    elif 'no' in env_sphinx_imgmath:
+    elif "no" in env_sphinx_imgmath:
         load_imgmath = False
     else:
         sys.stderr.write("Unknown env SPHINX_IMGMATH=%s ignored.\n" % env_sphinx_imgmath)
 
 if load_imgmath:
     extensions.append("sphinx.ext.imgmath")
-    math_renderer = 'imgmath'
+    math_renderer = "imgmath"
 else:
-    math_renderer = 'mathjax'
+    math_renderer = "mathjax"
 
 # Add any paths that contain templates here, relative to this directory.
-templates_path = ['sphinx/templates']
+templates_path = ["sphinx/templates"]
 
 # The suffix(es) of source filenames.
 # You can specify multiple suffix as a list of string:
@@ -209,15 +210,15 @@ templates_path = ['sphinx/templates']
 source_suffix = '.rst'
 
 # The encoding of source files.
-#source_encoding = 'utf-8-sig'
+# source_encoding = 'utf-8-sig'
 
 # The master toctree document.
-master_doc = 'index'
+master_doc = "index"
 
 # General information about the project.
-project = 'The Linux Kernel'
-copyright = 'The kernel development community'
-author = 'The kernel development community'
+project = "The Linux Kernel"
+copyright = "The kernel development community"         # pylint: disable=W0622
+author = "The kernel development community"
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
@@ -232,82 +233,86 @@ author = 'The kernel development community'
 try:
     makefile_version = None
     makefile_patchlevel = None
-    for line in open('../Makefile'):
-        key, val = [x.strip() for x in line.split('=', 2)]
-        if key == 'VERSION':
-            makefile_version = val
-        elif key == 'PATCHLEVEL':
-            makefile_patchlevel = val
-        if makefile_version and makefile_patchlevel:
-            break
-except:
+    with open("../Makefile", encoding="utf=8") as fp:
+        for line in fp:
+            key, val = [x.strip() for x in line.split("=", 2)]
+            if key == "VERSION":
+                makefile_version = val
+            elif key == "PATCHLEVEL":
+                makefile_patchlevel = val
+            if makefile_version and makefile_patchlevel:
+                break
+except Exception:
     pass
 finally:
     if makefile_version and makefile_patchlevel:
-        version = release = makefile_version + '.' + makefile_patchlevel
+        version = release = makefile_version + "." + makefile_patchlevel
     else:
         version = release = "unknown version"
 
-#
-# HACK: there seems to be no easy way for us to get at the version and
-# release information passed in from the makefile...so go pawing through the
-# command-line options and find it for ourselves.
-#
+
 def get_cline_version():
-    c_version = c_release = ''
+    """
+    HACK: There seems to be no easy way for us to get at the version and
+    release information passed in from the makefile...so go pawing through the
+    command-line options and find it for ourselves.
+    """
+
+    c_version = c_release = ""
     for arg in sys.argv:
-        if arg.startswith('version='):
+        if arg.startswith("version="):
             c_version = arg[8:]
-        elif arg.startswith('release='):
+        elif arg.startswith("release="):
             c_release = arg[8:]
     if c_version:
         if c_release:
-            return c_version + '-' + c_release
+            return c_version + "-" + c_release
         return c_version
-    return version # Whatever we came up with before
+    return version  # Whatever we came up with before
+
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
 #
 # This is also used if you do content translation via gettext catalogs.
 # Usually you set "language" from the command line for these cases.
-language = 'en'
+language = "en"
 
 # There are two options for replacing |today|: either, you set today to some
 # non-false value, then it is used:
-#today = ''
+# today = ''
 # Else, today_fmt is used as the format for a strftime call.
-#today_fmt = '%B %d, %Y'
+# today_fmt = '%B %d, %Y'
 
 # The reST default role (used for this markup: `text`) to use for all
 # documents.
-#default_role = None
+# default_role = None
 
 # If true, '()' will be appended to :func: etc. cross-reference text.
-#add_function_parentheses = True
+# add_function_parentheses = True
 
 # If true, the current module name will be prepended to all description
 # unit titles (such as .. function::).
-#add_module_names = True
+# add_module_names = True
 
 # If true, sectionauthor and moduleauthor directives will be shown in the
 # output. They are ignored by default.
-#show_authors = False
+# show_authors = False
 
 # The name of the Pygments (syntax highlighting) style to use.
-pygments_style = 'sphinx'
+pygments_style = "sphinx"
 
 # A list of ignored prefixes for module index sorting.
-#modindex_common_prefix = []
+# modindex_common_prefix = []
 
 # If true, keep warnings as "system message" paragraphs in the built documents.
-#keep_warnings = False
+# keep_warnings = False
 
 # If true, `todo` and `todoList` produce output, else they produce nothing.
 todo_include_todos = False
 
-primary_domain = 'c'
-highlight_language = 'none'
+primary_domain = "c"
+highlight_language = "none"
 
 # -- Options for HTML output ----------------------------------------------
 
@@ -315,43 +320,45 @@ highlight_language = 'none'
 # a list of builtin themes.
 
 # Default theme
-html_theme = 'alabaster'
+html_theme = "alabaster"
 html_css_files = []
 
 if "DOCS_THEME" in os.environ:
     html_theme = os.environ["DOCS_THEME"]
 
-if html_theme == 'sphinx_rtd_theme' or html_theme == 'sphinx_rtd_dark_mode':
+if html_theme in ["sphinx_rtd_theme", "sphinx_rtd_dark_mode"]:
     # Read the Docs theme
     try:
         import sphinx_rtd_theme
+
         html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
 
         # Add any paths that contain custom static files (such as style sheets) here,
         # relative to this directory. They are copied after the builtin static files,
         # so a file named "default.css" will overwrite the builtin "default.css".
         html_css_files = [
-            'theme_overrides.css',
+            "theme_overrides.css",
         ]
 
         # Read the Docs dark mode override theme
-        if html_theme == 'sphinx_rtd_dark_mode':
+        if html_theme == "sphinx_rtd_dark_mode":
             try:
-                import sphinx_rtd_dark_mode
-                extensions.append('sphinx_rtd_dark_mode')
+                import sphinx_rtd_dark_mode            # pylint: disable=W0611
+
+                extensions.append("sphinx_rtd_dark_mode")
             except ImportError:
-                html_theme == 'sphinx_rtd_theme'
+                html_theme = "sphinx_rtd_theme"
 
-        if html_theme == 'sphinx_rtd_theme':
-                # Add color-specific RTD normal mode
-                html_css_files.append('theme_rtd_colors.css')
+        if html_theme == "sphinx_rtd_theme":
+            # Add color-specific RTD normal mode
+            html_css_files.append("theme_rtd_colors.css")
 
         html_theme_options = {
-            'navigation_depth': -1,
+            "navigation_depth": -1,
         }
 
     except ImportError:
-        html_theme = 'alabaster'
+        html_theme = "alabaster"
 
 if "DOCS_CSS" in os.environ:
     css = os.environ["DOCS_CSS"].split(" ")
@@ -359,14 +366,14 @@ if "DOCS_CSS" in os.environ:
     for l in css:
         html_css_files.append(l)
 
-if  html_theme == 'alabaster':
+if html_theme == "alabaster":
     html_theme_options = {
-        'description': get_cline_version(),
-        'page_width': '65em',
-        'sidebar_width': '15em',
-        'fixed_sidebar': 'true',
-        'font_size': 'inherit',
-        'font_family': 'serif',
+        "description": get_cline_version(),
+        "page_width": "65em",
+        "sidebar_width": "15em",
+        "fixed_sidebar": "true",
+        "font_size": "inherit",
+        "font_family": "serif",
     }
 
 sys.stderr.write("Using %s theme\n" % html_theme)
@@ -374,104 +381,79 @@ sys.stderr.write("Using %s theme\n" % html_theme)
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
-html_static_path = ['sphinx-static']
+html_static_path = ["sphinx-static"]
 
 # If true, Docutils "smart quotes" will be used to convert quotes and dashes
 # to typographically correct entities.  However, conversion of "--" to "—"
 # is not always what we want, so enable only quotes.
-smartquotes_action = 'q'
+smartquotes_action = "q"
 
 # Custom sidebar templates, maps document names to template names.
 # Note that the RTD theme ignores this
-html_sidebars = { '**': ['searchbox.html', 'kernel-toc.html', 'sourcelink.html']}
+html_sidebars = {"**": ["searchbox.html",
+                        "kernel-toc.html",
+                        "sourcelink.html"]}
 
 # about.html is available for alabaster theme. Add it at the front.
-if html_theme == 'alabaster':
-    html_sidebars['**'].insert(0, 'about.html')
+if html_theme == "alabaster":
+    html_sidebars["**"].insert(0, "about.html")
 
 # The name of an image file (relative to this directory) to place at the top
 # of the sidebar.
-html_logo = 'images/logo.svg'
+html_logo = "images/logo.svg"
 
 # Output file base name for HTML help builder.
-htmlhelp_basename = 'TheLinuxKerneldoc'
+htmlhelp_basename = "TheLinuxKerneldoc"
 
 # -- Options for LaTeX output ---------------------------------------------
 
 latex_elements = {
     # The paper size ('letterpaper' or 'a4paper').
-    'papersize': 'a4paper',
-
+    "papersize": "a4paper",
     # The font size ('10pt', '11pt' or '12pt').
-    'pointsize': '11pt',
-
+    "pointsize": "11pt",
     # Latex figure (float) alignment
-    #'figure_align': 'htbp',
-
+    # 'figure_align': 'htbp',
     # Don't mangle with UTF-8 chars
-    'inputenc': '',
-    'utf8extra': '',
-
+    "inputenc": "",
+    "utf8extra": "",
     # Set document margins
-    'sphinxsetup': '''
+    "sphinxsetup": """
         hmargin=0.5in, vmargin=1in,
         parsedliteralwraps=true,
         verbatimhintsturnover=false,
-    ''',
-
+    """,
     #
     # Some of our authors are fond of deep nesting; tell latex to
     # cope.
     #
-    'maxlistdepth': '10',
-
+    "maxlistdepth": "10",
     # For CJK One-half spacing, need to be in front of hyperref
-    'extrapackages': r'\usepackage{setspace}',
-
+    "extrapackages": r"\usepackage{setspace}",
     # Additional stuff for the LaTeX preamble.
-    'preamble': '''
+    "preamble": """
         % Use some font with UTF-8 support with XeLaTeX
         \\usepackage{fontspec}
         \\setsansfont{DejaVu Sans}
         \\setromanfont{DejaVu Serif}
         \\setmonofont{DejaVu Sans Mono}
-    ''',
+    """,
 }
 
 # Load kerneldoc specific LaTeX settings
-latex_elements['preamble'] += '''
+latex_elements["preamble"] += """
         % Load kerneldoc specific LaTeX settings
-	\\input{kerneldoc-preamble.sty}
-'''
-
-# With Sphinx 1.6, it is possible to change the Bg color directly
-# by using:
-#	\definecolor{sphinxnoteBgColor}{RGB}{204,255,255}
-#	\definecolor{sphinxwarningBgColor}{RGB}{255,204,204}
-#	\definecolor{sphinxattentionBgColor}{RGB}{255,255,204}
-#	\definecolor{sphinximportantBgColor}{RGB}{192,255,204}
-#
-# However, it require to use sphinx heavy box with:
-#
-#	\renewenvironment{sphinxlightbox} {%
-#		\\begin{sphinxheavybox}
-#	}
-#		\\end{sphinxheavybox}
-#	}
-#
-# Unfortunately, the implementation is buggy: if a note is inside a
-# table, it isn't displayed well. So, for now, let's use boring
-# black and white notes.
+        \\input{kerneldoc-preamble.sty}
+"""
 
 # Grouping the document tree into LaTeX files. List of tuples
 # (source start file, target name, title,
 #  author, documentclass [howto, manual, or own class]).
 # Sorted in alphabetical order
-latex_documents = [
-]
+latex_documents = []
 
 # Add all other index files from Documentation/ subdirectories
-for fn in os.listdir('.'):
+for fn in os.listdir("."):
     doc = os.path.join(fn, "index")
     if os.path.exists(doc + ".rst"):
         has = False
@@ -480,34 +462,39 @@ for fn in os.listdir('.'):
                 has = True
                 break
         if not has:
-            latex_documents.append((doc, fn + '.tex',
-                                    'Linux %s Documentation' % fn.capitalize(),
-                                    'The kernel development community',
-                                    'manual'))
+            latex_documents.append(
+                (
+                    doc,
+                    fn + ".tex",
+                    "Linux %s Documentation" % fn.capitalize(),
+                    "The kernel development community",
+                    "manual",
+                )
+            )
 
 # The name of an image file (relative to this directory) to place at the top of
 # the title page.
-#latex_logo = None
+# latex_logo = None
 
 # For "manual" documents, if this is true, then toplevel headings are parts,
 # not chapters.
-#latex_use_parts = False
+# latex_use_parts = False
 
 # If true, show page references after internal links.
-#latex_show_pagerefs = False
+# latex_show_pagerefs = False
 
 # If true, show URL addresses after external links.
-#latex_show_urls = False
+# latex_show_urls = False
 
 # Documents to append as an appendix to all manuals.
-#latex_appendices = []
+# latex_appendices = []
 
 # If false, no module index is generated.
-#latex_domain_indices = True
+# latex_domain_indices = True
 
 # Additional LaTeX stuff to be copied to build directory
 latex_additional_files = [
-    'sphinx/kerneldoc-preamble.sty',
+    "sphinx/kerneldoc-preamble.sty",
 ]
 
 
@@ -516,12 +503,11 @@ latex_additional_files = [
 # One entry per manual page. List of tuples
 # (source start file, name, description, authors, manual section).
 man_pages = [
-    (master_doc, 'thelinuxkernel', 'The Linux Kernel Documentation',
-     [author], 1)
+    (master_doc, "thelinuxkernel", "The Linux Kernel Documentation", [author], 1)
 ]
 
 # If true, show URL addresses after external links.
-#man_show_urls = False
+# man_show_urls = False
 
 
 # -- Options for Texinfo output -------------------------------------------
@@ -529,11 +515,15 @@ man_pages = [
 # Grouping the document tree into Texinfo files. List of tuples
 # (source start file, target name, title, author,
 #  dir menu entry, description, category)
-texinfo_documents = [
-    (master_doc, 'TheLinuxKernel', 'The Linux Kernel Documentation',
-     author, 'TheLinuxKernel', 'One line description of project.',
-     'Miscellaneous'),
-]
+texinfo_documents = [(
+        master_doc,
+        "TheLinuxKernel",
+        "The Linux Kernel Documentation",
+        author,
+        "TheLinuxKernel",
+        "One line description of project.",
+        "Miscellaneous",
+    ),]
 
 # -- Options for Epub output ----------------------------------------------
 
@@ -544,9 +534,9 @@ epub_publisher = author
 epub_copyright = copyright
 
 # A list of files that should not be packed into the epub file.
-epub_exclude_files = ['search.html']
+epub_exclude_files = ["search.html"]
 
-#=======
+# =======
 # rst2pdf
 #
 # Grouping the document tree into PDF files. List of tuples
@@ -558,14 +548,14 @@ epub_exclude_files = ['search.html']
 # multiple PDF files here actually tries to get the cross-referencing right
 # *between* PDF files.
 pdf_documents = [
-    ('kernel-documentation', u'Kernel', u'Kernel', u'J. Random Bozo'),
+    ("kernel-documentation", "Kernel", "Kernel", "J. Random Bozo"),
 ]
 
 # kernel-doc extension configuration for running Sphinx directly (e.g. by Read
 # the Docs). In a normal build, these are supplied from the Makefile via command
 # line arguments.
-kerneldoc_bin = '../scripts/kernel-doc.py'
-kerneldoc_srctree = '..'
+kerneldoc_bin = "../scripts/kernel-doc.py"
+kerneldoc_srctree = ".."
 
 # ------------------------------------------------------------------------------
 # Since loadConfig overwrites settings from the global namespace, it has to be
@@ -573,5 +563,8 @@ kerneldoc_srctree = '..'
 # ------------------------------------------------------------------------------
 loadConfig(globals())
 
+
 def setup(app):
+    """Patterns need to be updated at init time on older Sphinx versions"""
+
     app.connect('config-inited', update_patterns)
-- 
2.49.0

[PATCH v3 15/15] docs: conf.py: Check Sphinx and docutils version

[Mauro Carvalho Chehab]

As reported by Akira, there were incompatibility issues with
Sphinx and docutils with docutils 0.19. There's already
a fix for it, but, as there are incompatibility issues with
different versions, better to add a check to verify if the
combination is supported/tested.

After check Sphinx release notes, it seems that the
version compatibility is given by:

        =======  ============   ============
        Sphinx   Min Docutils   Max Docutils
        Version  Version        Version
        -------  ------------   ------------
        3.4.3    >= 0.12.0      < 0.18.0
        4.0.0    >= 0.12.0      < 0.19.0
        6.0.0    >= 0.18.0      < 0.20.0
        7.0.0    >= 0.18.1      < 0.21.0
        7.2.0    >= 0.18.1      < 0.20.0
        7.4.0    >= 0.18.1      < 0.21.0
        8.0.0    >= 0.20.0      < 0.22.0
        8.2.3    >= 0.20.0      < 0.22.0
        =======  ============   ============

For now, add a logic inside conf.py to check the above
compatibility list, emitting warnings if the docutils
version doesn't match it.

This way, when we discover incompatibility issues, we
can easily adjust the table.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/conf.py | 70 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 67 insertions(+), 3 deletions(-)

diff --git a/Documentation/conf.py b/Documentation/conf.py
index 700516238d3f..df99a4d96b58 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -9,7 +9,11 @@ import os
 import shutil
 import sys
 
+import docutils
 import sphinx
+from sphinx.util import logging
+
+logger = logging.getLogger(__name__)
 
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
@@ -21,11 +25,71 @@ from load_config import loadConfig               # pylint: disable=C0413,E0401
 # Minimal supported version
 needs_sphinx = "3.4.3"
 
-# Get Sphinx version
-major, minor, patch = sphinx.version_info[:3]          # pylint: disable=I1101
+# Get Sphinx and docutils versions
+sphinx_ver = sphinx.version_info[:3]          # pylint: disable=I1101
+docutils_ver = docutils.__version_info__[:3]
+
+sphinx_ver_str = ".".join([str(x) for x in sphinx_ver])
+docutils_ver_str = ".".join([str(x) for x in docutils_ver])
+
+# Docutils min/max versions.
+# The dockutils version check logic is done with:
+#     ver >= min and ver < max
+SPHINX_DOCUTILS_VERS = {
+    (3, 4, 3): {
+        "min": (0, 12, 0),
+        "max": (0, 18, 0)
+    },
+    (4, 0, 0): {
+        "min": (0, 12, 0),
+        "max": (0, 19, 0)
+    },
+    (6, 0, 0): {
+        "min": (0, 18, 0),
+        "max": (0, 20, 0)
+    },
+    (7, 0, 0): {
+        "min": (0, 18, 1),
+        "max": (0, 21, 0)
+    },
+    (7, 2, 0): {
+        "min": (0, 18, 1),
+        "max": (0, 20, 0)
+    },
+    (7, 4, 0): {
+        "min": (0, 18, 1),
+        "max": (0, 21, 0)
+    },
+    (8, 0, 0): {
+        "min": (0, 20, 0),
+        "max": (0, 22, 0)
+    },
+    (8, 2, 3): {
+        "min": (0, 20, 0),
+        "max": (0, 22, 0)
+    },
+}
+
+du_min = None
+du_max = None
+for sp_ver, doc_vers in SPHINX_DOCUTILS_VERS.items():
+    if sp_ver > sphinx_ver:
+        break
+
+    du_min = doc_vers.get("min")
+    du_max = doc_vers.get("max")
+
+if sphinx_ver > sorted(SPHINX_DOCUTILS_VERS.keys())[-1]:
+    logger.warning(f"Sphinx version {sphinx_ver_str} is too new and not tested. Doc generation may fail")
+elif not du_min or not du_max:
+    logger.warning(f"Sphinx version {sphinx_ver_str} is not tested. Doc generation may fail")
+elif docutils_ver < du_min:
+    logger.warning(f"Docutils {docutils_ver_str} is too old for Sphinx {sphinx_ver_str}. Doc generation may fail")
+elif docutils_ver >= du_max:
+    logger.warning(f"Docutils {docutils_ver_str} could be too new for Sphinx {sphinx_ver_str}. Doc generation may fail")
 
 # Include_patterns were added on Sphinx 5.1
-if (major < 5) or (major == 5 and minor < 1):
+if sphinx_ver < (5, 1, 0):
     has_include_patterns = False
 else:
     has_include_patterns = True
-- 
2.49.0

Re: [PATCH v3 15/15] docs: conf.py: Check Sphinx and docutils version

[Akira Yokosawa]

On Sun, 22 Jun 2025 08:02:44 +0200, Mauro Carvalho Chehab wrote:
> As reported by Akira, there were incompatibility issues with
> Sphinx and docutils with docutils 0.19. There's already
> a fix for it, but, as there are incompatibility issues with
> different versions, better to add a check to verify if the
> combination is supported/tested.
> 

I've been skeptical of adding such checks in conf.py.

What happened the other day was that Jon used a deprecated (and rarely
used) method of docutils which failed to work properly only in
docutils 0.19 (there is no mention of related issues in its and 
nearby release notes).

Your integration of parser_yaml extension will raise the minimum
version of docutils to 0.17.1.  I think all you will need is just
to check:

    docutils < 0.17.1

, and to make a warning regardless of Sphinx versions.

> After check Sphinx release notes, it seems that the
> version compatibility is given by:
> 
>         =======  ============   ============
>         Sphinx   Min Docutils   Max Docutils
>         Version  Version        Version
>         -------  ------------   ------------
>         3.4.3    >= 0.12.0      < 0.18.0
>         4.0.0    >= 0.12.0      < 0.19.0
>         6.0.0    >= 0.18.0      < 0.20.0
>         7.0.0    >= 0.18.1      < 0.21.0
>         7.2.0    >= 0.18.1      < 0.20.0
>         7.4.0    >= 0.18.1      < 0.21.0
>         8.0.0    >= 0.20.0      < 0.22.0
>         8.2.3    >= 0.20.0      < 0.22.0
>         =======  ============   ============
> 

<nitpick>

So this is what I see in the changelog (not looked into < 6.0.0):

         ================= ====================
          Sphinx version     docutils version
          (since)              min       max
         ----------------- ------------ -------
         6.0.0             >= 0.18      < 0.20
         6.2.0             >= 0.18.1    < 0.20
	 7.0.1             >= 0.18.1    < 0.21
         7.3.0             >= 0.18.1    < 0.22
	 7.4.0             >= 0.20      < 0.22
	 (up to 8.2.3)
         ================= ====================

This Looks quite different from yours ...

</nitpick>

That said, these dependencies should be recognized and taken care of by
pip/PyPI (or whatever distro package management system), and already met;
unless you have done something strange/dangerous and screwed up your Sphinx
installation.

My limited imagination prevents me from coming up with any plausible
scenario where these checks at every sphinx-build invocation can be helpful.

> For now, add a logic inside conf.py to check the above
> compatibility list, emitting warnings if the docutils
> version doesn't match it.
> 
> This way, when we discover incompatibility issues, we
> can easily adjust the table.

Does not sound convincing enough to me.  Sorry.

        Thanks, Akira

>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
>  Documentation/conf.py | 70 +++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 67 insertions(+), 3 deletions(-)
[...]

Re: [PATCH v3 15/15] docs: conf.py: Check Sphinx and docutils version

[Mauro Carvalho Chehab]

Hi Akira,

Em Sun, 22 Jun 2025 20:19:52 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:

> On Sun, 22 Jun 2025 08:02:44 +0200, Mauro Carvalho Chehab wrote:
> > As reported by Akira, there were incompatibility issues with
> > Sphinx and docutils with docutils 0.19. There's already
> > a fix for it, but, as there are incompatibility issues with
> > different versions, better to add a check to verify if the
> > combination is supported/tested.
> >   
> 
> I've been skeptical of adding such checks in conf.py.
> 
> What happened the other day was that Jon used a deprecated (and rarely
> used) method of docutils which failed to work properly only in
> docutils 0.19 (there is no mention of related issues in its and 
> nearby release notes).

True, but the same also happened with me at the parser_yaml code.

> 
> Your integration of parser_yaml extension will raise the minimum
> version of docutils to 0.17.1. 
>
> I think all you will need is just
> to check:
> 
>     docutils < 0.17.1
> , and to make a warning regardless of Sphinx versions.

That's not the best solution. 

See, Docutils 0.17 was released in Jun, 2021. and 0.17.1 on
October, 2021. If we look at the Sphinx releases, we have:

======  ===================
Sphinx  Release Date
------  -------------------
4.1.0	June 10, 2021
4.1.1	July 25, 2021
4.1.2	August 1, 2021
4.1.3	August 15, 2021
4.2.0	September 28, 2021
======  ===================

As Sphinx policy is to not even fix bugs when a new release
happens, it means that, at the best, the minimal version made to be
compatible with 0.17 is 4.1.x, and the minimal compatible with
0.17.1 is 4.2.x.

So, if we want not to raise the minimal version to 4.2.x (*), the right 
fix is this (I'll be sending at the upcoming YAML patch series I'll be
sending next week):

diff --git a/Documentation/sphinx/parser_yaml.py b/Documentation/sphinx/parser_yaml.py
index 8288e2ff7c7c..1602b31f448e 100755
--- a/Documentation/sphinx/parser_yaml.py
+++ b/Documentation/sphinx/parser_yaml.py
@@ -77,6 +77,10 @@ class YamlParser(Parser):
 
                 result.append(line, document.current_source, lineoffset)
 
+            # Fix backward compatibility with docutils < 0.17.1
+            if "tab_width" not in vars(document.settings):
+                document.settings.tab_width = 8
+
             rst_parser = RSTParser()
             rst_parser.parse('\n'.join(result), document)

What happens is that Sphinx is a shell case built on the top of
docutils: the actual ReST parser is inside docutils. 

(*) the exact patch where 0.17.1 support was added/tested on
    Sphinx needs to be checked

Now, there is another alternative: raise the bar, increasing minimal
versions for Python and Sphinx. Yet, I would do it only once per year.

> That said, these dependencies should be recognized and taken care of by
> pip/PyPI (or whatever distro package management system), and already met;
> unless you have done something strange/dangerous and screwed up your Sphinx
> installation.
>
> My limited imagination prevents me from coming up with any plausible
> scenario where these checks at every sphinx-build invocation can be helpful.
> 

Heh, playing with scripts/test_doc_build.py, pip allowed to create
several environments that are valid for the package management software
but either:

- cause sphinx-build to crash even before returning Sphinx version;
- crash in the middle of the compilation.

As I said, Sphinx is a shell on the top of docutils. Some of the code
there seem to depend on subtile behaviors, like the one with
the tab_width affecting parser_yaml.py on certain Sphinx/docutils
combinations or the one fixed by Jon with regards to setup a class
for the broken references.

The idea of this patch is to warn the one building the docs
that he might be navigating turbulent waters when it is unknown
if the combination is supported. It also better defines the
scope of backward compatibility we want to provide.

Thanks,
Mauro

Re: [PATCH v3 15/15] docs: conf.py: Check Sphinx and docutils version

[Mauro Carvalho Chehab]

Em Sun, 22 Jun 2025 20:19:52 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:

> > After check Sphinx release notes, it seems that the
> > version compatibility is given by:
> > 
> >         =======  ============   ============
> >         Sphinx   Min Docutils   Max Docutils
> >         Version  Version        Version
> >         -------  ------------   ------------
> >         3.4.3    >= 0.12.0      < 0.18.0
> >         4.0.0    >= 0.12.0      < 0.19.0
> >         6.0.0    >= 0.18.0      < 0.20.0
> >         7.0.0    >= 0.18.1      < 0.21.0
> >         7.2.0    >= 0.18.1      < 0.20.0
> >         7.4.0    >= 0.18.1      < 0.21.0
> >         8.0.0    >= 0.20.0      < 0.22.0
> >         8.2.3    >= 0.20.0      < 0.22.0
> >         =======  ============   ============
> >   
> 
> <nitpick>
> So this is what I see in the changelog (not looked into < 6.0.0):
> 
>          ================= ====================
>           Sphinx version     docutils version
>           (since)              min       max
>          ----------------- ------------ -------
>          6.0.0             >= 0.18      < 0.20
>          6.2.0             >= 0.18.1    < 0.20
> 	 7.0.1             >= 0.18.1    < 0.21
>          7.3.0             >= 0.18.1    < 0.22
> 	 7.4.0             >= 0.20      < 0.22
> 	 (up to 8.2.3)
>          ================= ====================
> 
> This Looks quite different from yours ...
> </nitpick>

Thanks for your feedback!

Getting this right is not too easy, specially since not all
changelog mentions docutil version. Btw, one of the factors I 
used when building my table were to check the release dates 
of Sphinx and docutils. So, I manually adjusted some entries,
as changelog usually say something like:

	Release 5.0.0 (released May 30, 2022)¶
	Dependencies

	5.0.0 b1

	    #10164: Support Docutils 0.18. Patch by Adam Turner.

But doesn't mention minimal supported version.

I'll re-check and adjust it as needed, if Jon decides to pick it.

Thanks,
Mauro

Re: [PATCH v3 14/15] docs: conf.py: several coding style fixes

[Jonathan Corbet]

Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:

> conf.py is missing a SPDX header and doesn't really have
> a proper python coding style. It also has an obsolete
> commented LaTeX syntax that doesn't work anymore.
>
> Clean it up a little bit with some help from autolints
> and manual adjustments.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
>  Documentation/conf.py | 351 +++++++++++++++++++++---------------------
>  1 file changed, 172 insertions(+), 179 deletions(-)

This file is definitely getting out of hand...:)

I notice a lot of the changes are just from 'single quotes' to "double
quotes".  That adds a lot of churn; is there a reason for it?

Thanks,

jon

Re: [PATCH v3 15/15] docs: conf.py: Check Sphinx and docutils version

[Jonathan Corbet]

Akira Yokosawa <akiyks@gmail.com> writes:

> On Sun, 22 Jun 2025 08:02:44 +0200, Mauro Carvalho Chehab wrote:
>> As reported by Akira, there were incompatibility issues with
>> Sphinx and docutils with docutils 0.19. There's already
>> a fix for it, but, as there are incompatibility issues with
>> different versions, better to add a check to verify if the
>> combination is supported/tested.
>> 
>
> I've been skeptical of adding such checks in conf.py.

I have to kind of agree with this concern.  We have managed without this
complexity so far.  It looks like we could always be behind on
maintaining it going forward.  Do we *really* need this one?

Thanks,

jon

Re: [PATCH v3 14/15] docs: conf.py: several coding style fixes

[Mauro Carvalho Chehab]

Em Sun, 22 Jun 2025 14:55:26 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
> > conf.py is missing a SPDX header and doesn't really have
> > a proper python coding style. It also has an obsolete
> > commented LaTeX syntax that doesn't work anymore.
> >
> > Clean it up a little bit with some help from autolints
> > and manual adjustments.
> >
> > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> > ---
> >  Documentation/conf.py | 351 +++++++++++++++++++++---------------------
> >  1 file changed, 172 insertions(+), 179 deletions(-)  
> 
> This file is definitely getting out of hand...:)

Yes. I almost removed all commented code to make it a little more
palatable.

> I notice a lot of the changes are just from 'single quotes' to "double
> quotes".  That adds a lot of churn; is there a reason for it?

Those churns are from black/autopep8, which also addressed other 
coding style issues(*).

I don't mind much about using either single or double quote, but
having a standard is a good thing. At the plus side, as C uses 
double quotes for strings this is closer to the kernel coding style.

So, I opted to merge such changes from the lint tools.

(*) I didn't blidnly accept all coding style changes from linters,
    dropping the ones that would make the final code worse.

Thanks,
Mauro

Re: [PATCH v3 15/15] docs: conf.py: Check Sphinx and docutils version

[Mauro Carvalho Chehab]

Em Sun, 22 Jun 2025 14:58:04 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Akira Yokosawa <akiyks@gmail.com> writes:
> 
> > On Sun, 22 Jun 2025 08:02:44 +0200, Mauro Carvalho Chehab wrote:  
> >> As reported by Akira, there were incompatibility issues with
> >> Sphinx and docutils with docutils 0.19. There's already
> >> a fix for it, but, as there are incompatibility issues with
> >> different versions, better to add a check to verify if the
> >> combination is supported/tested.
> >>   
> >
> > I've been skeptical of adding such checks in conf.py.  
> 
> I have to kind of agree with this concern.  We have managed without this
> complexity so far.  It looks like we could always be behind on
> maintaining it going forward.  Do we *really* need this one?

IMO having a check is interesting, as the dependency between
Sphinx and docutils is high. Yet, with the testing script, this may
not be needed, provided that we run it to check if changes at Sphinx
extensions won't cause regressions. Still, the dependency check
at test_doc_build.py is not complete.

Anyway, if you prefer, don't pick this one. We can revisit it later
when needed.

Thanks,
Mauro

Re: [PATCH v3 15/15] docs: conf.py: Check Sphinx and docutils version

[Jonathan Corbet]

Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:

> Em Sun, 22 Jun 2025 14:58:04 -0600
> Jonathan Corbet <corbet@lwn.net> escreveu:
>
>> Akira Yokosawa <akiyks@gmail.com> writes:
>> 
>> > On Sun, 22 Jun 2025 08:02:44 +0200, Mauro Carvalho Chehab wrote:  
>> >> As reported by Akira, there were incompatibility issues with
>> >> Sphinx and docutils with docutils 0.19. There's already
>> >> a fix for it, but, as there are incompatibility issues with
>> >> different versions, better to add a check to verify if the
>> >> combination is supported/tested.
>> >>   
>> >
>> > I've been skeptical of adding such checks in conf.py.  
>> 
>> I have to kind of agree with this concern.  We have managed without this
>> complexity so far.  It looks like we could always be behind on
>> maintaining it going forward.  Do we *really* need this one?
>
> IMO having a check is interesting, as the dependency between
> Sphinx and docutils is high. Yet, with the testing script, this may
> not be needed, provided that we run it to check if changes at Sphinx
> extensions won't cause regressions. Still, the dependency check
> at test_doc_build.py is not complete.
>
> Anyway, if you prefer, don't pick this one. We can revisit it later
> when needed.

I've left it out for now, but applied the rest of the series.  Keep it
around, we may yet decide we need it...

Thanks,

jon

Re: [PATCH v3 15/15] docs: conf.py: Check Sphinx and docutils version

[Mauro Carvalho Chehab]

Em Wed, 25 Jun 2025 12:37:37 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
> > Em Sun, 22 Jun 2025 14:58:04 -0600
> > Jonathan Corbet <corbet@lwn.net> escreveu:
> >  
> >> Akira Yokosawa <akiyks@gmail.com> writes:
> >>   
> >> > On Sun, 22 Jun 2025 08:02:44 +0200, Mauro Carvalho Chehab wrote:    
> >> >> As reported by Akira, there were incompatibility issues with
> >> >> Sphinx and docutils with docutils 0.19. There's already
> >> >> a fix for it, but, as there are incompatibility issues with
> >> >> different versions, better to add a check to verify if the
> >> >> combination is supported/tested.
> >> >>     
> >> >
> >> > I've been skeptical of adding such checks in conf.py.    
> >> 
> >> I have to kind of agree with this concern.  We have managed without this
> >> complexity so far.  It looks like we could always be behind on
> >> maintaining it going forward.  Do we *really* need this one?  
> >
> > IMO having a check is interesting, as the dependency between
> > Sphinx and docutils is high. Yet, with the testing script, this may
> > not be needed, provided that we run it to check if changes at Sphinx
> > extensions won't cause regressions. Still, the dependency check
> > at test_doc_build.py is not complete.
> >
> > Anyway, if you prefer, don't pick this one. We can revisit it later
> > when needed.  
> 
> I've left it out for now, but applied the rest of the series.  Keep it
> around, we may yet decide we need it...

Ok, I placed on my scratch tree on github, on a separate branch:
	https://github.com/mchehab/linux/commits/check_sphinx_at_conf_py/

The patch is here:
	https://github.com/mchehab/linux/commit/178f37fce4aa16592b0c0b567ea0ffca744c3af5

(I'm documenting here just in case we forget and need it again ;-) )

Regards,
Mauro



> 
> Thanks,
> 
> jon



Thanks,
Mauro