[PATCH v2 0/2] docs/python: bump minimum Sphinx version

[PATCH v2 0/2] docs/python: bump minimum Sphinx version

[John Snow]

With recent deprecations, we can advance our minimum sphinx version
safely. This is heavily motivated by new qapidoc work which is much
easier to maintain cross-version compatibility for - see difficulties in
our dbus documentation which only works on sphinx >= 4.

We can only guarantee >= 3.4.3 now, but that's still vastly easier than
maintaining compatibility all the way back to 1.x.

https://gitlab.com/jsnow/qemu/-/pipelines/1359549621

(failures appear to be unrelated to the series.)

V2: refactor all code deletions into patch 2

John Snow (2):
  Python: bump minimum sphinx version to 3.4.3
  docs: remove Sphinx 1.x compatibility code

 docs/conf.py             |  7 +++----
 docs/sphinx/hxtool.py    | 21 ++++-----------------
 docs/sphinx/kerneldoc.py | 38 ++++++++++++--------------------------
 docs/sphinx/kernellog.py | 28 ----------------------------
 docs/sphinx/qapidoc.py   | 29 +++--------------------------
 pythondeps.toml          |  2 +-
 6 files changed, 23 insertions(+), 102 deletions(-)
 delete mode 100644 docs/sphinx/kernellog.py

-- 
2.45.0

[PATCH v2 1/2] Python: bump minimum sphinx version to 3.4.3

[John Snow]

With RHEL 8 support retired (It's been two years since RHEL9 released),
our very oldest build platform version of Sphinx is now 3.4.3; and
keeping backwards compatibility for versions as old as v1.6 when using
domain extensions is a lot of work we don't need to do.

This patch is motivated by my work creating a new QAPI domain, which
unlike the dbus documentation, cannot be allowed to regress by creating
a "dummy" doc when operating under older sphinx versions. Easier is to
raise our minimum version as far as we can push it forwards, reducing my
burden in creating cross-compatibility hacks and patches.

A sampling of sphinx versions from various distributions, courtesy
https://repology.org/project/python:sphinx/versions

Alpine 3.16: v4.3.0 (QEMU support ended 2024-05-23)
Alpine 3.17: v5.3.0
Alpine 3.18: v6.1.3
Alpine 3.19: v6.2.1
Ubuntu 20.04 LTS: EOL
Ubuntu 22.04 LTS: v4.3.2
Ubuntu 22.10: EOL
Ubuntu 23.04: EOL
Ubuntu 23.10: v5.3.0
Ubuntu 24.04 LTS: v7.2.6
Debian 11: v3.4.3 (QEMU support ends 2024-07-xx)
Debian 12: v5.3.0
Fedora 38: EOL
Fedora 39: v6.2.1
Fedora 40: v7.2.6
CentOS Stream 8: v1.7.6 (QEMU support ended 2024-05-17)
CentOS Stream 9: v3.4.3
OpenSUSE Leap 15.4: EOL
OpenSUSE Leap 15.5: 2.3.1, 4.2.0 and 7.2.6

RHEL9 / CentOS Stream 9 becomes the new defining factor in staying at
Sphinx 3.4.3 due to downstream offline build requirements that force us
to use platform Sphinx instead of newer packages from PyPI.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
---
 docs/conf.py    | 7 +++----
 pythondeps.toml | 2 +-
 2 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/docs/conf.py b/docs/conf.py
index aae0304ac6e..876f6768815 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -53,10 +53,9 @@
 
 # If your documentation needs a minimal Sphinx version, state it here.
 #
-# Sphinx 1.5 and earlier can't build our docs because they are too
-# picky about the syntax of the argument to the option:: directive
-# (see Sphinx bugs #646, #3366).
-needs_sphinx = '1.6'
+# 3.4.3 is the oldest version of Sphinx that ships on a platform we
+# pledge build support for.
+needs_sphinx = '3.4.3'
 
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
diff --git a/pythondeps.toml b/pythondeps.toml
index 9c16602d303..bc656376caa 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -23,7 +23,7 @@ meson = { accepted = ">=0.63.0", installed = "1.2.3", canary = "meson" }
 
 [docs]
 # Please keep the installed versions in sync with docs/requirements.txt
-sphinx = { accepted = ">=1.6", installed = "5.3.0", canary = "sphinx-build" }
+sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" }
 sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" }
 
 [avocado]
-- 
2.45.0

[PATCH v2 2/2] docs: remove Sphinx 1.x compatibility code

[John Snow]

In general, the Use_SSI workaround is no longer needed, and neither is
the pre-1.6 logging shim for kerneldoc.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
---
 docs/sphinx/hxtool.py    | 21 ++++-----------------
 docs/sphinx/kerneldoc.py | 38 ++++++++++++--------------------------
 docs/sphinx/kernellog.py | 28 ----------------------------
 docs/sphinx/qapidoc.py   | 29 +++--------------------------
 4 files changed, 19 insertions(+), 97 deletions(-)
 delete mode 100644 docs/sphinx/kernellog.py

diff --git a/docs/sphinx/hxtool.py b/docs/sphinx/hxtool.py
index 3729084a36c..a84723be19e 100644
--- a/docs/sphinx/hxtool.py
+++ b/docs/sphinx/hxtool.py
@@ -24,16 +24,10 @@
 from docutils.statemachine import ViewList
 from docutils.parsers.rst import directives, Directive
 from sphinx.errors import ExtensionError
+from sphinx.util.docutils import switch_source_input
 from sphinx.util.nodes import nested_parse_with_titles
 import sphinx
 
-# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later
-# use switch_source_input. Check borrowed from kerneldoc.py.
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-    from sphinx.util.docutils import switch_source_input
-else:
-    from sphinx.ext.autodoc import AutodocReporter
 
 __version__ = '1.0'
 
@@ -185,16 +179,9 @@ def run(self):
     # of title_styles and section_level that kerneldoc.py does,
     # because nested_parse_with_titles() does that for us.
     def do_parse(self, result, node):
-        if Use_SSI:
-            with switch_source_input(self.state, result):
-                nested_parse_with_titles(self.state, result, node)
-        else:
-            save = self.state.memo.reporter
-            self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter)
-            try:
-                nested_parse_with_titles(self.state, result, node)
-            finally:
-                self.state.memo.reporter = save
+        with switch_source_input(self.state, result):
+            nested_parse_with_titles(self.state, result, node)
+
 
 def setup(app):
     """ Register hxtool-doc directive with Sphinx"""
diff --git a/docs/sphinx/kerneldoc.py b/docs/sphinx/kerneldoc.py
index 72c403a7379..3aa972f2e89 100644
--- a/docs/sphinx/kerneldoc.py
+++ b/docs/sphinx/kerneldoc.py
@@ -38,20 +38,14 @@
 from docutils.statemachine import ViewList
 from docutils.parsers.rst import directives, Directive
 
-#
-# AutodocReporter is only good up to Sphinx 1.7
-#
 import sphinx
+from sphinx.util import logging
+from sphinx.util.docutils import switch_source_input
 
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-    from sphinx.util.docutils import switch_source_input
-else:
-    from sphinx.ext.autodoc import AutodocReporter
-
-import kernellog
 
 __version__  = '1.0'
+logger = logging.getLogger('kerneldoc')
+
 
 class KernelDocDirective(Directive):
     """Extract kernel-doc comments from the specified file"""
@@ -111,8 +105,7 @@ def run(self):
         cmd += [filename]
 
         try:
-            kernellog.verbose(env.app,
-                              'calling kernel-doc \'%s\'' % (" ".join(cmd)))
+            logger.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd)))
 
             p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
             out, err = p.communicate()
@@ -122,8 +115,10 @@ def run(self):
             if p.returncode != 0:
                 sys.stderr.write(err)
 
-                kernellog.warn(env.app,
-                               'kernel-doc \'%s\' failed with return code %d' % (" ".join(cmd), p.returncode))
+                logger.warning(
+                    'kernel-doc \'%s\' failed with return code %d' %
+                    (" ".join(cmd), p.returncode)
+                )
                 return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
             elif env.config.kerneldoc_verbosity > 0:
                 sys.stderr.write(err)
@@ -149,22 +144,13 @@ def run(self):
             return node.children
 
         except Exception as e:  # pylint: disable=W0703
-            kernellog.warn(env.app, 'kernel-doc \'%s\' processing failed with: %s' %
+            logger.warning('kernel-doc \'%s\' processing failed with: %s' %
                            (" ".join(cmd), str(e)))
             return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
 
     def do_parse(self, result, node):
-        if Use_SSI:
-            with switch_source_input(self.state, result):
-                self.state.nested_parse(result, 0, node, match_titles=1)
-        else:
-            save = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
-            self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter)
-            self.state.memo.title_styles, self.state.memo.section_level = [], 0
-            try:
-                self.state.nested_parse(result, 0, node, match_titles=1)
-            finally:
-                self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = save
+        with switch_source_input(self.state, result):
+            self.state.nested_parse(result, 0, node, match_titles=1)
 
 
 def setup(app):
diff --git a/docs/sphinx/kernellog.py b/docs/sphinx/kernellog.py
deleted file mode 100644
index af924f51a7d..00000000000
--- a/docs/sphinx/kernellog.py
+++ /dev/null
@@ -1,28 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-#
-# Sphinx has deprecated its older logging interface, but the replacement
-# only goes back to 1.6.  So here's a wrapper layer to keep around for
-# as long as we support 1.4.
-#
-import sphinx
-
-if sphinx.__version__[:3] >= '1.6':
-    UseLogging = True
-    from sphinx.util import logging
-    logger = logging.getLogger('kerneldoc')
-else:
-    UseLogging = False
-
-def warn(app, message):
-    if UseLogging:
-        logger.warning(message)
-    else:
-        app.warn(message)
-
-def verbose(app, message):
-    if UseLogging:
-        logger.verbose(message)
-    else:
-        app.verbose(message)
-
-
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index f270b494f01..9a0cfcbce75 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -31,6 +31,7 @@
 from docutils.statemachine import ViewList
 from docutils.parsers.rst import directives, Directive
 from sphinx.errors import ExtensionError
+from sphinx.util.docutils import switch_source_input
 from sphinx.util.nodes import nested_parse_with_titles
 import sphinx
 from qapi.gen import QAPISchemaVisitor
@@ -38,15 +39,6 @@
 from qapi.schema import QAPISchema
 
 
-# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later
-# use switch_source_input. Check borrowed from kerneldoc.py.
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-    from sphinx.util.docutils import switch_source_input
-else:
-    from sphinx.ext.autodoc import AutodocReporter
-
-
 __version__ = '1.0'
 
 
@@ -517,23 +509,8 @@ def do_parse(self, rstlist, node):
         subheadings (titles) without confusing the rendering of
         anything else.
         """
-        # This is from kerneldoc.py -- it works around an API change in
-        # Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use
-        # sphinx.util.nodes.nested_parse_with_titles() rather than the
-        # plain self.state.nested_parse(), and so we can drop the saving
-        # of title_styles and section_level that kerneldoc.py does,
-        # because nested_parse_with_titles() does that for us.
-        if Use_SSI:
-            with switch_source_input(self.state, rstlist):
-                nested_parse_with_titles(self.state, rstlist, node)
-        else:
-            save = self.state.memo.reporter
-            self.state.memo.reporter = AutodocReporter(
-                rstlist, self.state.memo.reporter)
-            try:
-                nested_parse_with_titles(self.state, rstlist, node)
-            finally:
-                self.state.memo.reporter = save
+        with switch_source_input(self.state, rstlist):
+            nested_parse_with_titles(self.state, rstlist, node)
 
 
 def setup(app):
-- 
2.45.0

Re: [PATCH v2 1/2] Python: bump minimum sphinx version to 3.4.3

[Thomas Huth]

On 03/07/2024 19.52, John Snow wrote:
> With RHEL 8 support retired (It's been two years since RHEL9 released),
> our very oldest build platform version of Sphinx is now 3.4.3; and
> keeping backwards compatibility for versions as old as v1.6 when using
> domain extensions is a lot of work we don't need to do.
> 
> This patch is motivated by my work creating a new QAPI domain, which
> unlike the dbus documentation, cannot be allowed to regress by creating
> a "dummy" doc when operating under older sphinx versions. Easier is to
> raise our minimum version as far as we can push it forwards, reducing my
> burden in creating cross-compatibility hacks and patches.
> 
> A sampling of sphinx versions from various distributions, courtesy
> https://repology.org/project/python:sphinx/versions
> 
> Alpine 3.16: v4.3.0 (QEMU support ended 2024-05-23)
> Alpine 3.17: v5.3.0
> Alpine 3.18: v6.1.3
> Alpine 3.19: v6.2.1
> Ubuntu 20.04 LTS: EOL
> Ubuntu 22.04 LTS: v4.3.2
> Ubuntu 22.10: EOL
> Ubuntu 23.04: EOL
> Ubuntu 23.10: v5.3.0
> Ubuntu 24.04 LTS: v7.2.6
> Debian 11: v3.4.3 (QEMU support ends 2024-07-xx)
> Debian 12: v5.3.0
> Fedora 38: EOL
> Fedora 39: v6.2.1
> Fedora 40: v7.2.6
> CentOS Stream 8: v1.7.6 (QEMU support ended 2024-05-17)
> CentOS Stream 9: v3.4.3
> OpenSUSE Leap 15.4: EOL
> OpenSUSE Leap 15.5: 2.3.1, 4.2.0 and 7.2.6
> 
> RHEL9 / CentOS Stream 9 becomes the new defining factor in staying at
> Sphinx 3.4.3 due to downstream offline build requirements that force us
> to use platform Sphinx instead of newer packages from PyPI.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> Acked-by: Markus Armbruster <armbru@redhat.com>
> ---
>   docs/conf.py    | 7 +++----
>   pythondeps.toml | 2 +-
>   2 files changed, 4 insertions(+), 5 deletions(-)

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH v2 2/2] docs: remove Sphinx 1.x compatibility code

[Thomas Huth]

On 03/07/2024 19.52, John Snow wrote:
> In general, the Use_SSI workaround is no longer needed, and neither is
> the pre-1.6 logging shim for kerneldoc.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> Acked-by: Markus Armbruster <armbru@redhat.com>
> ---
>   docs/sphinx/hxtool.py    | 21 ++++-----------------
>   docs/sphinx/kerneldoc.py | 38 ++++++++++++--------------------------
>   docs/sphinx/kernellog.py | 28 ----------------------------
>   docs/sphinx/qapidoc.py   | 29 +++--------------------------
>   4 files changed, 19 insertions(+), 97 deletions(-)
>   delete mode 100644 docs/sphinx/kernellog.py

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH v2 0/2] docs/python: bump minimum Sphinx version

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 1439 bytes --]

On Wed, Jul 3, 2024, 1:52 PM John Snow <jsnow@redhat.com> wrote:

> With recent deprecations, we can advance our minimum sphinx version
> safely. This is heavily motivated by new qapidoc work which is much
> easier to maintain cross-version compatibility for - see difficulties in
> our dbus documentation which only works on sphinx >= 4.
>
> We can only guarantee >= 3.4.3 now, but that's still vastly easier than
> maintaining compatibility all the way back to 1.x.
>
> https://gitlab.com/jsnow/qemu/-/pipelines/1359549621
>
> (failures appear to be unrelated to the series.)
>
> V2: refactor all code deletions into patch 2
>
> John Snow (2):
>   Python: bump minimum sphinx version to 3.4.3
>   docs: remove Sphinx 1.x compatibility code
>
>  docs/conf.py             |  7 +++----
>  docs/sphinx/hxtool.py    | 21 ++++-----------------
>  docs/sphinx/kerneldoc.py | 38 ++++++++++++--------------------------
>  docs/sphinx/kernellog.py | 28 ----------------------------
>  docs/sphinx/qapidoc.py   | 29 +++--------------------------
>  pythondeps.toml          |  2 +-
>  6 files changed, 23 insertions(+), 102 deletions(-)
>  delete mode 100644 docs/sphinx/kernellog.py
>
> --
> 2.45.0
>

Going to assume It's okay for me to stage this one under my Python tree.

Markus: This will create a minor conflict with the notes patches (the
delint patch), but the resolution is straightforward.

--js

[-- Attachment #2: Type: text/html, Size: 2083 bytes --]