[PATCH 0/3] docs: Bump sphinx to 6.2.1

[PATCH 0/3] docs: Bump sphinx to 6.2.1

[Akihiko Odaki]

Supersedes: <20250505-sphinx82-v1-0-85f2418b33b1@daynix.com>
("[PATCH 0/2] docs: Bump sphinx to 8.2.3")

sphinx 5.3.0 fails with Python 3.13.1:

../docs/meson.build:37: WARNING: /home/me/qemu/build/pyvenv/bin/sphinx-build:
Extension error:
Could not import extension sphinx.builders.epub3 (exception: No module named 'imghdr')

../docs/meson.build:39:6: ERROR: Problem encountered: Install a Python 3 version of python-sphinx and the readthedoc theme

Bump sphinx to 6.2.1 and also sphinx_rtd_theme as required for the new
sphinx version.

Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com>
---
Akihiko Odaki (3):
      docs: Bump sphinx to 6.2.1
      docs: Require sphinx>=6.2
      MAINTAINERS: Add docs/requirements.txt

 MAINTAINERS                |   1 +
 docs/requirements.txt      |   4 +-
 docs/sphinx/compat.py      | 230 ---------------------------------------------
 docs/sphinx/qapi_domain.py |  33 +++----
 pythondeps.toml            |   4 +-
 5 files changed, 20 insertions(+), 252 deletions(-)
---
base-commit: 1da8f3a3c53b604edfe0d55e475102640490549e
change-id: 20250508-sphinx72-400c521af710

Best regards,
-- 
Akihiko Odaki <akihiko.odaki@daynix.com>

[PATCH 1/3] docs: Bump sphinx to 6.2.1

[Akihiko Odaki]

sphinx 5.3.0 fails with Python 3.13.1:

../docs/meson.build:37: WARNING: /home/me/qemu/build/pyvenv/bin/sphinx-build:
Extension error:
Could not import extension sphinx.builders.epub3 (exception: No module named 'imghdr')

../docs/meson.build:39:6: ERROR: Problem encountered: Install a Python 3 version of python-sphinx and the readthedoc theme

Bump sphinx to 6.2.1 and also sphinx_rtd_theme as required for the new
sphinx version.

Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com>
---
 docs/requirements.txt | 4 ++--
 pythondeps.toml       | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/docs/requirements.txt b/docs/requirements.txt
index 02583f209aa6..87f7afcb2e79 100644
--- a/docs/requirements.txt
+++ b/docs/requirements.txt
@@ -1,5 +1,5 @@
 # Used by readthedocs.io
 # Should be in sync with the "installed" key of pythondeps.toml
 
-sphinx==5.3.0
-sphinx_rtd_theme==1.1.1
+sphinx==6.2.1
+sphinx_rtd_theme==1.2.2
diff --git a/pythondeps.toml b/pythondeps.toml
index c03c9df81b5c..4f6b6f01e0d0 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -24,8 +24,8 @@ pycotap = { accepted = ">=1.1.0", installed = "1.3.1" }
 
 [docs]
 # Please keep the installed versions in sync with docs/requirements.txt
-sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" }
-sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" }
+sphinx = { accepted = ">=3.4.3", installed = "6.2.1", canary = "sphinx-build" }
+sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" }
 
 [avocado]
 # Note that qemu.git/python/ is always implicitly installed.

-- 
2.49.0

[PATCH 2/3] docs: Require sphinx>=6.2

[Akihiko Odaki]

This makes docs/sphinx/compat.py unnecessary. The accepted range of the
sphinx_rtd_theme version was also changed to >=1.2 for compatibility
with sphinx>=6.2.

Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com>
---
 docs/sphinx/compat.py      | 230 ---------------------------------------------
 docs/sphinx/qapi_domain.py |  33 +++----
 pythondeps.toml            |   4 +-
 3 files changed, 17 insertions(+), 250 deletions(-)

diff --git a/docs/sphinx/compat.py b/docs/sphinx/compat.py
deleted file mode 100644
index 9cf7fe006e44..000000000000
--- a/docs/sphinx/compat.py
+++ /dev/null
@@ -1,230 +0,0 @@
-"""
-Sphinx cross-version compatibility goop
-"""
-
-import re
-from typing import (
-    TYPE_CHECKING,
-    Any,
-    Callable,
-    Optional,
-    Type,
-)
-
-from docutils import nodes
-from docutils.nodes import Element, Node, Text
-from docutils.statemachine import StringList
-
-import sphinx
-from sphinx import addnodes, util
-from sphinx.directives import ObjectDescription
-from sphinx.environment import BuildEnvironment
-from sphinx.roles import XRefRole
-from sphinx.util import docfields
-from sphinx.util.docutils import (
-    ReferenceRole,
-    SphinxDirective,
-    switch_source_input,
-)
-from sphinx.util.typing import TextlikeNode
-
-
-MAKE_XREF_WORKAROUND = sphinx.version_info[:3] < (4, 1, 0)
-
-
-SpaceNode: Callable[[str], Node]
-KeywordNode: Callable[[str, str], Node]
-
-if sphinx.version_info[:3] >= (4, 0, 0):
-    SpaceNode = addnodes.desc_sig_space
-    KeywordNode = addnodes.desc_sig_keyword
-else:
-    SpaceNode = Text
-    KeywordNode = addnodes.desc_annotation
-
-
-def nested_parse_with_titles(
-    directive: SphinxDirective, content_node: Element
-) -> None:
-    """
-    This helper preserves error parsing context across sphinx versions.
-    """
-
-    # necessary so that the child nodes get the right source/line set
-    content_node.document = directive.state.document
-
-    try:
-        # Modern sphinx (6.2.0+) supports proper offsetting for
-        # nested parse error context management
-        util.nodes.nested_parse_with_titles(
-            directive.state,
-            directive.content,
-            content_node,
-            content_offset=directive.content_offset,
-        )
-    except TypeError:
-        # No content_offset argument. Fall back to SSI method.
-        with switch_source_input(directive.state, directive.content):
-            util.nodes.nested_parse_with_titles(
-                directive.state, directive.content, content_node
-            )
-
-
-# ###########################################
-# xref compatibility hacks for Sphinx < 4.1 #
-# ###########################################
-
-# When we require >= Sphinx 4.1, the following function and the
-# subsequent 3 compatibility classes can be removed. Anywhere in
-# qapi_domain that uses one of these Compat* types can be switched to
-# using the garden-variety lib-provided classes with no trickery.
-
-
-def _compat_make_xref(  # pylint: disable=unused-argument
-    self: sphinx.util.docfields.Field,
-    rolename: str,
-    domain: str,
-    target: str,
-    innernode: Type[TextlikeNode] = addnodes.literal_emphasis,
-    contnode: Optional[Node] = None,
-    env: Optional[BuildEnvironment] = None,
-    inliner: Any = None,
-    location: Any = None,
-) -> Node:
-    """
-    Compatibility workaround for Sphinx versions prior to 4.1.0.
-
-    Older sphinx versions do not use the domain's XRefRole for parsing
-    and formatting cross-references, so we need to perform this magick
-    ourselves to avoid needing to write the parser/formatter in two
-    separate places.
-
-    This workaround isn't brick-for-brick compatible with modern Sphinx
-    versions, because we do not have access to the parent directive's
-    state during this parsing like we do in more modern versions.
-
-    It's no worse than what pre-Sphinx 4.1.0 does, so... oh well!
-    """
-
-    # Yes, this function is gross. Pre-4.1 support is a miracle.
-    # pylint: disable=too-many-locals
-
-    assert env
-    # Note: Sphinx's own code ignores the type warning here, too.
-    if not rolename:
-        return contnode or innernode(target, target)  # type: ignore[call-arg]
-
-    # Get the role instance, but don't *execute it* - we lack the
-    # correct state to do so. Instead, we'll just use its public
-    # methods to do our reference formatting, and emulate the rest.
-    role = env.get_domain(domain).roles[rolename]
-    assert isinstance(role, XRefRole)
-
-    # XRefRole features not supported by this compatibility shim;
-    # these were not supported in Sphinx 3.x either, so nothing of
-    # value is really lost.
-    assert not target.startswith("!")
-    assert not re.match(ReferenceRole.explicit_title_re, target)
-    assert not role.lowercase
-    assert not role.fix_parens
-
-    # Code below based mostly on sphinx.roles.XRefRole; run() and
-    # create_xref_node()
-    options = {
-        "refdoc": env.docname,
-        "refdomain": domain,
-        "reftype": rolename,
-        "refexplicit": False,
-        "refwarn": role.warn_dangling,
-    }
-    refnode = role.nodeclass(target, **options)
-    title, target = role.process_link(env, refnode, False, target, target)
-    refnode["reftarget"] = target
-    classes = ["xref", domain, f"{domain}-{rolename}"]
-    refnode += role.innernodeclass(target, title, classes=classes)
-
-    # This is the very gross part of the hack. Normally,
-    # result_nodes takes a document object to which we would pass
-    # self.inliner.document. Prior to Sphinx 4.1, we don't *have* an
-    # inliner to pass, so we have nothing to pass here. However, the
-    # actual implementation of role.result_nodes in this case
-    # doesn't actually use that argument, so this winds up being
-    # ... fine. Rest easy at night knowing this code only runs under
-    # old versions of Sphinx, so at least it won't change in the
-    # future on us and lead to surprising new failures.
-    # Gross, I know.
-    result_nodes, _messages = role.result_nodes(
-        None,  # type: ignore
-        env,
-        refnode,
-        is_ref=True,
-    )
-    return nodes.inline(target, "", *result_nodes)
-
-
-class CompatField(docfields.Field):
-    if MAKE_XREF_WORKAROUND:
-        make_xref = _compat_make_xref
-
-
-class CompatGroupedField(docfields.GroupedField):
-    if MAKE_XREF_WORKAROUND:
-        make_xref = _compat_make_xref
-
-
-class CompatTypedField(docfields.TypedField):
-    if MAKE_XREF_WORKAROUND:
-        make_xref = _compat_make_xref
-
-
-# ################################################################
-# Nested parsing error location fix for Sphinx 5.3.0 < x < 6.2.0 #
-# ################################################################
-
-# When we require Sphinx 4.x, the TYPE_CHECKING hack where we avoid
-# subscripting ObjectDescription at runtime can be removed in favor of
-# just always subscripting the class.
-
-# When we require Sphinx > 6.2.0, the rest of this compatibility hack
-# can be dropped and QAPIObject can just inherit directly from
-# ObjectDescription[Signature].
-
-SOURCE_LOCATION_FIX = (5, 3, 0) <= sphinx.version_info[:3] < (6, 2, 0)
-
-Signature = str
-
-
-if TYPE_CHECKING:
-    _BaseClass = ObjectDescription[Signature]
-else:
-    _BaseClass = ObjectDescription
-
-
-class ParserFix(_BaseClass):
-
-    _temp_content: StringList
-    _temp_offset: int
-    _temp_node: Optional[addnodes.desc_content]
-
-    def before_content(self) -> None:
-        # Work around a sphinx bug and parse the content ourselves.
-        self._temp_content = self.content
-        self._temp_offset = self.content_offset
-        self._temp_node = None
-
-        if SOURCE_LOCATION_FIX:
-            self._temp_node = addnodes.desc_content()
-            self.state.nested_parse(
-                self.content, self.content_offset, self._temp_node
-            )
-            # Sphinx will try to parse the content block itself,
-            # Give it nothingness to parse instead.
-            self.content = StringList()
-            self.content_offset = 0
-
-    def transform_content(self, content_node: addnodes.desc_content) -> None:
-        # Sphinx workaround: Inject our parsed content and restore state.
-        if self._temp_node:
-            content_node += self._temp_node.children
-            self.content = self._temp_content
-            self.content_offset = self._temp_offset
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index c94af5719ca0..29e006c9cf0f 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -21,14 +21,10 @@
 from docutils import nodes
 from docutils.parsers.rst import directives
 
-from compat import (
-    CompatField,
-    CompatGroupedField,
-    CompatTypedField,
-    KeywordNode,
-    ParserFix,
-    Signature,
-    SpaceNode,
+from sphinx.util.docfields import (
+    Field,
+    GroupedField,
+    TypedField,
 )
 from sphinx import addnodes
 from sphinx.directives import ObjectDescription
@@ -64,6 +60,7 @@
     from sphinx.util.typing import OptionSpec
 
 
+Signature = str
 logger = logging.getLogger(__name__)
 
 
@@ -158,7 +155,7 @@ def result_nodes(
         return results, []
 
 
-class QAPIDescription(ParserFix):
+class QAPIDescription(ObjectDescription[Signature]):
     """
     Generic QAPI description.
 
@@ -312,7 +309,7 @@ class QAPIObject(QAPIDescription):
 
     doc_field_types = [
         # :feat name: descr
-        CompatGroupedField(
+        GroupedField(
             "feature",
             label=_("Features"),
             names=("feat",),
@@ -324,8 +321,8 @@ def get_signature_prefix(self) -> List[nodes.Node]:
         """Return a prefix to put before the object name in the signature."""
         assert self.objtype
         return [
-            KeywordNode("", self.objtype.title()),
-            SpaceNode(" "),
+            addnodes.desc_sig_keyword("", self.objtype.title()),
+            addnodes.desc_sig_space(" "),
         ]
 
     def get_signature_suffix(self) -> List[nodes.Node]:
@@ -334,7 +331,7 @@ def get_signature_suffix(self) -> List[nodes.Node]:
 
         if "since" in self.options:
             ret += [
-                SpaceNode(" "),
+                addnodes.desc_sig_space(" "),
                 addnodes.desc_sig_element(
                     "", f"(Since: {self.options['since']})"
                 ),
@@ -482,7 +479,7 @@ def transform_content(self, content_node: addnodes.desc_content) -> None:
                     self._validate_field(field)
 
 
-class SpecialTypedField(CompatTypedField):
+class SpecialTypedField(TypedField):
     def make_field(self, *args: Any, **kwargs: Any) -> nodes.field:
         ret = super().make_field(*args, **kwargs)
 
@@ -515,14 +512,14 @@ class QAPICommand(QAPIObject):
                 can_collapse=False,
             ),
             # :error: descr
-            CompatField(
+            Field(
                 "error",
                 label=_("Errors"),
                 names=("error", "errors"),
                 has_arg=False,
             ),
             # :return TypeName: descr
-            CompatGroupedField(
+            GroupedField(
                 "returnvalue",
                 label=_("Return"),
                 rolename="type",
@@ -540,7 +537,7 @@ class QAPIEnum(QAPIObject):
     doc_field_types.extend(
         [
             # :value name: descr
-            CompatGroupedField(
+            GroupedField(
                 "value",
                 label=_("Values"),
                 names=("value",),
@@ -557,7 +554,7 @@ class QAPIAlternate(QAPIObject):
     doc_field_types.extend(
         [
             # :alt type name: descr
-            CompatTypedField(
+            TypedField(
                 "alternative",
                 label=_("Alternatives"),
                 names=("alt",),
diff --git a/pythondeps.toml b/pythondeps.toml
index 4f6b6f01e0d0..0d8f5ae24a24 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -24,8 +24,8 @@ pycotap = { accepted = ">=1.1.0", installed = "1.3.1" }
 
 [docs]
 # Please keep the installed versions in sync with docs/requirements.txt
-sphinx = { accepted = ">=3.4.3", installed = "6.2.1", canary = "sphinx-build" }
-sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" }
+sphinx = { accepted = ">=6.2", installed = "6.2.1", canary = "sphinx-build" }
+sphinx_rtd_theme = { accepted = ">=1.2", installed = "1.2.2" }
 
 [avocado]
 # Note that qemu.git/python/ is always implicitly installed.

-- 
2.49.0

[PATCH 3/3] MAINTAINERS: Add docs/requirements.txt

[Akihiko Odaki]

Add docs/requirements.txt to
"Sphinx documentation configuration and build machinery".

Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com>
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index d54b5578f883..6ae5d2665e16 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4328,6 +4328,7 @@ M: Peter Maydell <peter.maydell@linaro.org>
 S: Maintained
 F: docs/conf.py
 F: docs/*/conf.py
+F: docs/requirements.txt
 F: docs/sphinx/
 F: docs/_templates/
 F: docs/devel/docs.rst

-- 
2.49.0

Re: [PATCH 0/3] docs: Bump sphinx to 6.2.1

[Markus Armbruster]

Akihiko Odaki <akihiko.odaki@daynix.com> writes:

> Supersedes: <20250505-sphinx82-v1-0-85f2418b33b1@daynix.com>
> ("[PATCH 0/2] docs: Bump sphinx to 8.2.3")
>
> sphinx 5.3.0 fails with Python 3.13.1:
>
> ../docs/meson.build:37: WARNING: /home/me/qemu/build/pyvenv/bin/sphinx-build:
> Extension error:
> Could not import extension sphinx.builders.epub3 (exception: No module named 'imghdr')
>
> ../docs/meson.build:39:6: ERROR: Problem encountered: Install a Python 3 version of python-sphinx and the readthedoc theme
>
> Bump sphinx to 6.2.1 and also sphinx_rtd_theme as required for the new
> sphinx version.
>
> Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com>

Works fine for me on a Fedora 41 box (Sphinx 7.3.8)

Tested-by: Markus Armbruster <armbru@redhat.com>

Re: [PATCH 0/3] docs: Bump sphinx to 6.2.1

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 1808 bytes --]

On Sat, May 10, 2025 at 2:17 AM Akihiko Odaki <akihiko.odaki@daynix.com>
wrote:

> Supersedes: <20250505-sphinx82-v1-0-85f2418b33b1@daynix.com>
> ("[PATCH 0/2] docs: Bump sphinx to 8.2.3")
>
> sphinx 5.3.0 fails with Python 3.13.1:
>
> ../docs/meson.build:37: WARNING:
> /home/me/qemu/build/pyvenv/bin/sphinx-build:
> Extension error:
> Could not import extension sphinx.builders.epub3 (exception: No module
> named 'imghdr')
>
> ../docs/meson.build:39:6: ERROR: Problem encountered: Install a Python 3
> version of python-sphinx and the readthedoc theme
>
> Bump sphinx to 6.2.1 and also sphinx_rtd_theme as required for the new
> sphinx version.
>
> Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com>
>

Reviewed-by: John Snow <jsnow@redhat.com>

I figure if this causes issues, this early in the development cycle we can
roll it back or pursue alternate solutions if necessary. I figure because
this touches so much stuff I wrote, it probably ought to go through my
tree, so I'll stage this tentatively.

(Paolo, Peter: But if you want to stage it instead, be my guest.)

---
> Akihiko Odaki (3):
>       docs: Bump sphinx to 6.2.1
>       docs: Require sphinx>=6.2
>       MAINTAINERS: Add docs/requirements.txt
>
>  MAINTAINERS                |   1 +
>  docs/requirements.txt      |   4 +-
>  docs/sphinx/compat.py      | 230
> ---------------------------------------------
>

Oh, thank you so much O:-) how beautiful...


>  docs/sphinx/qapi_domain.py |  33 +++----
>  pythondeps.toml            |   4 +-
>  5 files changed, 20 insertions(+), 252 deletions(-)
> ---
> base-commit: 1da8f3a3c53b604edfe0d55e475102640490549e
> change-id: 20250508-sphinx72-400c521af710
>
> Best regards,
> --
> Akihiko Odaki <akihiko.odaki@daynix.com>
>
>

[-- Attachment #2: Type: text/html, Size: 2910 bytes --]

Re: [PATCH 0/3] docs: Bump sphinx to 6.2.1

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 2473 bytes --]

On Mon, May 19, 2025 at 1:47 PM John Snow <jsnow@redhat.com> wrote:

>
>
> On Sat, May 10, 2025 at 2:17 AM Akihiko Odaki <akihiko.odaki@daynix.com>
> wrote:
>
>> Supersedes: <20250505-sphinx82-v1-0-85f2418b33b1@daynix.com>
>> ("[PATCH 0/2] docs: Bump sphinx to 8.2.3")
>>
>> sphinx 5.3.0 fails with Python 3.13.1:
>>
>> ../docs/meson.build:37: WARNING:
>> /home/me/qemu/build/pyvenv/bin/sphinx-build:
>> Extension error:
>> Could not import extension sphinx.builders.epub3 (exception: No module
>> named 'imghdr')
>>
>> ../docs/meson.build:39:6: ERROR: Problem encountered: Install a Python 3
>> version of python-sphinx and the readthedoc theme
>>
>> Bump sphinx to 6.2.1 and also sphinx_rtd_theme as required for the new
>> sphinx version.
>>
>> Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com>
>>
>
> Reviewed-by: John Snow <jsnow@redhat.com>
>
> I figure if this causes issues, this early in the development cycle we can
> roll it back or pursue alternate solutions if necessary. I figure because
> this touches so much stuff I wrote, it probably ought to go through my
> tree, so I'll stage this tentatively.
>
> (Paolo, Peter: But if you want to stage it instead, be my guest.)
>
> ---
>> Akihiko Odaki (3):
>>       docs: Bump sphinx to 6.2.1
>>       docs: Require sphinx>=6.2
>>       MAINTAINERS: Add docs/requirements.txt
>>
>>  MAINTAINERS                |   1 +
>>  docs/requirements.txt      |   4 +-
>>  docs/sphinx/compat.py      | 230
>> ---------------------------------------------
>>
>
> Oh, thank you so much O:-) how beautiful...
>
>
>>  docs/sphinx/qapi_domain.py |  33 +++----
>>  pythondeps.toml            |   4 +-
>>  5 files changed, 20 insertions(+), 252 deletions(-)
>> ---
>> base-commit: 1da8f3a3c53b604edfe0d55e475102640490549e
>> change-id: 20250508-sphinx72-400c521af710
>>
>> Best regards,
>> --
>> Akihiko Odaki <akihiko.odaki@daynix.com>
>
>
Just a note to say that I started staging and testing this and found a few
problems -- many are minor and I have patched them up accordingly, but I am
currently having an issue with debian claiming it can't find the
sphinxcontrib-jquery extension:

https://gitlab.com/jsnow/qemu/-/jobs/10257469021

It might be the case that we need to install the dependencies too, but that
might pose problems for offline source distributions. I haven't fixed this
yet, but I will continue looking into it.

Thanks,
--js

[-- Attachment #2: Type: text/html, Size: 3991 bytes --]