[PATCH v3 0/5] docs: remove legacy qapidoc

[PATCH v3 0/5] docs: remove legacy qapidoc

[John Snow]

Remove docs/sphinx/qapidoc_legacy.py, and remove special parsing of
freeform QAPI documentation block sections in favor of using standard
rST syntax that is included in the final document with no special
parsing or post-processing.

v3:
 - Fixed indentation for :error: in qapidoc
 - Removed parser restrictions on QAPI-style headers
 - Updated test output so make check passes ...

v2:
 - rebased on origin/master (2025-06-12)
 - Revised commit messages with increased detail

John Snow (5):
  docs/sphinx: adjust qapidoc to cope with same-line error sections
  docs/sphinx: parse @references in freeform text
  docs/sphinx: remove legacy QAPI manual generator
  docs/sphinx: remove special parsing for freeform sections
  qapi: lift restriction on using '=' in doc blocks

 docs/devel/qapi-code-gen.rst                 |  28 +-
 docs/interop/firmware.json                   |   4 +-
 docs/interop/qemu-ga-ref.rst                 |   1 -
 docs/interop/qemu-qmp-ref.rst                |   1 -
 docs/interop/qemu-storage-daemon-qmp-ref.rst |   1 -
 docs/interop/vhost-user.json                 |   4 +-
 docs/sphinx/qapidoc.py                       |  82 +---
 docs/sphinx/qapidoc_legacy.py                | 440 -------------------
 qapi/acpi.json                               |   4 +-
 qapi/audio.json                              |   4 +-
 qapi/authz.json                              |   4 +-
 qapi/block-core.json                         |   3 +-
 qapi/block-export.json                       |   3 +-
 qapi/block.json                              |   7 +-
 qapi/char.json                               |   4 +-
 qapi/common.json                             |   4 +-
 qapi/compat.json                             |   4 +-
 qapi/control.json                            |   4 +-
 qapi/crypto.json                             |   4 +-
 qapi/cryptodev.json                          |   4 +-
 qapi/cxl.json                                |   4 +-
 qapi/dump.json                               |   4 +-
 qapi/ebpf.json                               |   4 +-
 qapi/error.json                              |   4 +-
 qapi/introspect.json                         |   4 +-
 qapi/job.json                                |   4 +-
 qapi/machine-common.json                     |   4 +-
 qapi/machine.json                            |   4 +-
 qapi/migration.json                          |   4 +-
 qapi/misc.json                               |   4 +-
 qapi/net.json                                |   4 +-
 qapi/pci.json                                |   4 +-
 qapi/qapi-schema.json                        |   4 +-
 qapi/qdev.json                               |   4 +-
 qapi/qom.json                                |   4 +-
 qapi/replay.json                             |   4 +-
 qapi/rocker.json                             |   4 +-
 qapi/run-state.json                          |   4 +-
 qapi/sockets.json                            |   4 +-
 qapi/stats.json                              |   4 +-
 qapi/tpm.json                                |   4 +-
 qapi/trace.json                              |   4 +-
 qapi/transaction.json                        |   4 +-
 qapi/uefi.json                               |   4 +-
 qapi/ui.json                                 |  14 +-
 qapi/vfio.json                               |   4 +-
 qapi/virtio.json                             |   4 +-
 qapi/yank.json                               |   4 +-
 python/tests/qapi-isort.sh                   |   2 +-
 scripts/qapi/parser.py                       |  11 -
 storage-daemon/qapi/qapi-schema.json         |   8 +-
 tests/qapi-schema/doc-bad-section.err        |   1 -
 tests/qapi-schema/doc-bad-section.json       |  10 -
 tests/qapi-schema/doc-bad-section.out        |   0
 tests/qapi-schema/doc-good.json              |  10 +-
 tests/qapi-schema/doc-good.out               |  10 +-
 tests/qapi-schema/doc-good.txt               | 274 ++++--------
 tests/qapi-schema/meson.build                |   1 -
 58 files changed, 275 insertions(+), 784 deletions(-)
 delete mode 100644 docs/sphinx/qapidoc_legacy.py
 delete mode 100644 tests/qapi-schema/doc-bad-section.err
 delete mode 100644 tests/qapi-schema/doc-bad-section.json
 delete mode 100644 tests/qapi-schema/doc-bad-section.out

-- 
2.48.1

[PATCH v3 1/5] docs/sphinx: adjust qapidoc to cope with same-line error sections

[John Snow]

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/sphinx/qapidoc.py | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 8011ac9efaf..5374dee8fad 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -267,10 +267,14 @@ def visit_returns(self, section: QAPIDoc.Section) -> None:
         self.add_field("return", typ, section.text, section.info)
 
     def visit_errors(self, section: QAPIDoc.Section) -> None:
-        # FIXME: the formatting for errors may be inconsistent and may
-        # or may not require different newline placement to ensure
-        # proper rendering as a nested list.
-        self.add_lines(f":error:\n{section.text}", section.info)
+        # If the section text does not start with a space, it means text
+        # began on the same line as the "Error:" string and we should
+        # not insert a newline in this case.
+        if section.text[0].isspace():
+            text = f":error:\n{section.text}"
+        else:
+            text = f":error: {section.text}"
+        self.add_lines(text, section.info)
 
     def preamble(self, ent: QAPISchemaDefinition) -> None:
         """
-- 
2.48.1

[PATCH v3 2/5] docs/sphinx: parse @references in freeform text

[John Snow]

Oversight in the new qapidoc transmogrifier: @references in freeform
documentation blocks were not being transformed to literals. This fixes
that, and the next patch ensures that we're testing for this O:-)

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/sphinx/qapidoc.py | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 5374dee8fad..adc14ade456 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -218,6 +218,11 @@ def generate_field(
         typ = self.format_type(member)
         self.add_field(kind, member.name, body, info, typ)
 
+    @staticmethod
+    def reformat_arobase(text: str) -> str:
+        """ reformats @var to ``var`` """
+        return re.sub(r"@([\w-]+)", r"``\1``", text)
+
     # Transmogrification helpers
 
     def visit_paragraph(self, section: QAPIDoc.Section) -> None:
@@ -361,8 +366,7 @@ def visit_sections(self, ent: QAPISchemaDefinition) -> None:
 
         # Add sections in source order:
         for i, section in enumerate(sections):
-            # @var is translated to ``var``:
-            section.text = re.sub(r"@([\w-]+)", r"``\1``", section.text)
+            section.text = self.reformat_arobase(section.text)
 
             if section.kind == QAPIDoc.Kind.PLAIN:
                 self.visit_paragraph(section)
@@ -405,7 +409,7 @@ def visit_freeform(self, doc: QAPIDoc) -> None:
 
         assert len(doc.all_sections) == 1, doc.all_sections
         body = doc.all_sections[0]
-        text = body.text
+        text = self.reformat_arobase(body.text)
         info = doc.info
 
         if re.match(r"=+ ", text):
-- 
2.48.1

[PATCH v3 3/5] docs/sphinx: remove legacy QAPI manual generator

[John Snow]

Thanks for your service!

Remove the old qapidoc and the option to enable the transmogrifier,
leaving the "transmogrifier" as the ONLY qapi doc generator. This in
effect also converts the QAPI test to use the new documentation
generator, too.

Update doc-good.txt output to match the new doc generator, which I
should've done exactly when we switched over to the transmogrifier, but,
uhh, oops!

Notes on the new format:
 1. per-member IFCOND documentation is missing. Known issue.
 2. Freeform documentation without a header is now copied through into
    the output. This is an intentional change.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/interop/qemu-ga-ref.rst                 |   1 -
 docs/interop/qemu-qmp-ref.rst                |   1 -
 docs/interop/qemu-storage-daemon-qmp-ref.rst |   1 -
 docs/sphinx/qapidoc.py                       |  25 +-
 docs/sphinx/qapidoc_legacy.py                | 440 -------------------
 python/tests/qapi-isort.sh                   |   2 +-
 tests/qapi-schema/doc-good.txt               | 274 ++++--------
 7 files changed, 88 insertions(+), 656 deletions(-)
 delete mode 100644 docs/sphinx/qapidoc_legacy.py

diff --git a/docs/interop/qemu-ga-ref.rst b/docs/interop/qemu-ga-ref.rst
index 25f6e24b03c..ea6652ae43e 100644
--- a/docs/interop/qemu-ga-ref.rst
+++ b/docs/interop/qemu-ga-ref.rst
@@ -2,5 +2,4 @@ QEMU Guest Agent Protocol Reference
 ===================================
 
 .. qapi-doc:: qga/qapi-schema.json
-   :transmogrify:
    :namespace: QGA
diff --git a/docs/interop/qemu-qmp-ref.rst b/docs/interop/qemu-qmp-ref.rst
index 3bc1ca12b16..f0ce39ad67d 100644
--- a/docs/interop/qemu-qmp-ref.rst
+++ b/docs/interop/qemu-qmp-ref.rst
@@ -7,5 +7,4 @@ QEMU QMP Reference Manual
    :local:
 
 .. qapi-doc:: qapi/qapi-schema.json
-   :transmogrify:
    :namespace: QMP
diff --git a/docs/interop/qemu-storage-daemon-qmp-ref.rst b/docs/interop/qemu-storage-daemon-qmp-ref.rst
index dc7bde262ae..4dbb6a2cc83 100644
--- a/docs/interop/qemu-storage-daemon-qmp-ref.rst
+++ b/docs/interop/qemu-storage-daemon-qmp-ref.rst
@@ -5,5 +5,4 @@ QEMU Storage Daemon QMP Reference Manual
    :local:
 
 .. qapi-doc:: storage-daemon/qapi/qapi-schema.json
-   :transmogrify:
    :namespace: QSD
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index adc14ade456..d5d2b5eeb50 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -64,8 +64,6 @@
 from sphinx.util.docutils import SphinxDirective, switch_source_input
 from sphinx.util.nodes import nested_parse_with_titles
 
-from qapidoc_legacy import QAPISchemaGenRSTVisitor  # type: ignore
-
 
 if TYPE_CHECKING:
     from typing import (
@@ -512,15 +510,9 @@ class QAPIDocDirective(NestedDirective):
     option_spec = {
         "qapifile": directives.unchanged_required,
         "namespace": directives.unchanged,
-        "transmogrify": directives.flag,
     }
     has_content = False
 
-    def new_serialno(self) -> str:
-        """Return a unique new ID string suitable for use as a node's ID"""
-        env = self.state.document.settings.env
-        return "qapidoc-%d" % env.new_serialno("qapidoc")
-
     def transmogrify(self, schema: QAPISchema) -> nodes.Element:
         logger.info("Transmogrifying QAPI to rST ...")
         vis = Transmogrifier()
@@ -598,21 +590,10 @@ def write_intermediate(self, content: StringList, filename: str) -> None:
                     outfile.write(f" {rcol}")
                 outfile.write("\n")
 
-    def legacy(self, schema: QAPISchema) -> nodes.Element:
-        vis = QAPISchemaGenRSTVisitor(self)
-        vis.visit_begin(schema)
-        for doc in schema.docs:
-            if doc.symbol:
-                vis.symbol(doc, schema.lookup_entity(doc.symbol))
-            else:
-                vis.freeform(doc)
-        return vis.get_document_node()  # type: ignore
-
     def run(self) -> Sequence[nodes.Node]:
         env = self.state.document.settings.env
         qapifile = env.config.qapidoc_srctree + "/" + self.arguments[0]
         qapidir = os.path.dirname(qapifile)
-        transmogrify = "transmogrify" in self.options
 
         try:
             schema = QAPISchema(qapifile)
@@ -625,11 +606,7 @@ def run(self) -> Sequence[nodes.Node]:
             # so they are displayed nicely to the user
             raise ExtensionError(str(err)) from err
 
-        if transmogrify:
-            contentnode = self.transmogrify(schema)
-        else:
-            contentnode = self.legacy(schema)
-
+        contentnode = self.transmogrify(schema)
         return contentnode.children
 
 
diff --git a/docs/sphinx/qapidoc_legacy.py b/docs/sphinx/qapidoc_legacy.py
deleted file mode 100644
index 13520f4c26b..00000000000
--- a/docs/sphinx/qapidoc_legacy.py
+++ /dev/null
@@ -1,440 +0,0 @@
-# coding=utf-8
-# type: ignore
-#
-# QEMU qapidoc QAPI file parsing extension
-#
-# Copyright (c) 2020 Linaro
-#
-# This work is licensed under the terms of the GNU GPLv2 or later.
-# See the COPYING file in the top-level directory.
-
-"""
-qapidoc is a Sphinx extension that implements the qapi-doc directive
-
-The purpose of this extension is to read the documentation comments
-in QAPI schema files, and insert them all into the current document.
-
-It implements one new rST directive, "qapi-doc::".
-Each qapi-doc:: directive takes one argument, which is the
-pathname of the schema file to process, relative to the source tree.
-
-The docs/conf.py file must set the qapidoc_srctree config value to
-the root of the QEMU source tree.
-
-The Sphinx documentation on writing extensions is at:
-https://www.sphinx-doc.org/en/master/development/index.html
-"""
-
-import re
-import textwrap
-
-from docutils import nodes
-from docutils.statemachine import ViewList
-from qapi.error import QAPISemError
-from qapi.gen import QAPISchemaVisitor
-from qapi.parser import QAPIDoc
-
-
-def dedent(text: str) -> str:
-    # Adjust indentation to make description text parse as paragraph.
-
-    lines = text.splitlines(True)
-    if re.match(r"\s+", lines[0]):
-        # First line is indented; description started on the line after
-        # the name. dedent the whole block.
-        return textwrap.dedent(text)
-
-    # Descr started on same line. Dedent line 2+.
-    return lines[0] + textwrap.dedent("".join(lines[1:]))
-
-
-class QAPISchemaGenRSTVisitor(QAPISchemaVisitor):
-    """A QAPI schema visitor which generates docutils/Sphinx nodes
-
-    This class builds up a tree of docutils/Sphinx nodes corresponding
-    to documentation for the various QAPI objects. To use it, first
-    create a QAPISchemaGenRSTVisitor object, and call its
-    visit_begin() method.  Then you can call one of the two methods
-    'freeform' (to add documentation for a freeform documentation
-    chunk) or 'symbol' (to add documentation for a QAPI symbol). These
-    will cause the visitor to build up the tree of document
-    nodes. Once you've added all the documentation via 'freeform' and
-    'symbol' method calls, you can call 'get_document_nodes' to get
-    the final list of document nodes (in a form suitable for returning
-    from a Sphinx directive's 'run' method).
-    """
-    def __init__(self, sphinx_directive):
-        self._cur_doc = None
-        self._sphinx_directive = sphinx_directive
-        self._top_node = nodes.section()
-        self._active_headings = [self._top_node]
-
-    def _make_dlitem(self, term, defn):
-        """Return a dlitem node with the specified term and definition.
-
-        term should be a list of Text and literal nodes.
-        defn should be one of:
-        - a string, which will be handed to _parse_text_into_node
-        - a list of Text and literal nodes, which will be put into
-          a paragraph node
-        """
-        dlitem = nodes.definition_list_item()
-        dlterm = nodes.term('', '', *term)
-        dlitem += dlterm
-        if defn:
-            dldef = nodes.definition()
-            if isinstance(defn, list):
-                dldef += nodes.paragraph('', '', *defn)
-            else:
-                self._parse_text_into_node(defn, dldef)
-            dlitem += dldef
-        return dlitem
-
-    def _make_section(self, title):
-        """Return a section node with optional title"""
-        section = nodes.section(ids=[self._sphinx_directive.new_serialno()])
-        if title:
-            section += nodes.title(title, title)
-        return section
-
-    def _nodes_for_ifcond(self, ifcond, with_if=True):
-        """Return list of Text, literal nodes for the ifcond
-
-        Return a list which gives text like ' (If: condition)'.
-        If with_if is False, we don't return the "(If: " and ")".
-        """
-
-        doc = ifcond.docgen()
-        if not doc:
-            return []
-        doc = nodes.literal('', doc)
-        if not with_if:
-            return [doc]
-
-        nodelist = [nodes.Text(' ('), nodes.strong('', 'If: ')]
-        nodelist.append(doc)
-        nodelist.append(nodes.Text(')'))
-        return nodelist
-
-    def _nodes_for_one_member(self, member):
-        """Return list of Text, literal nodes for this member
-
-        Return a list of doctree nodes which give text like
-        'name: type (optional) (If: ...)' suitable for use as the
-        'term' part of a definition list item.
-        """
-        term = [nodes.literal('', member.name)]
-        if member.type.doc_type():
-            term.append(nodes.Text(': '))
-            term.append(nodes.literal('', member.type.doc_type()))
-        if member.optional:
-            term.append(nodes.Text(' (optional)'))
-        if member.ifcond.is_present():
-            term.extend(self._nodes_for_ifcond(member.ifcond))
-        return term
-
-    def _nodes_for_variant_when(self, branches, variant):
-        """Return list of Text, literal nodes for variant 'when' clause
-
-        Return a list of doctree nodes which give text like
-        'when tagname is variant (If: ...)' suitable for use in
-        the 'branches' part of a definition list.
-        """
-        term = [nodes.Text(' when '),
-                nodes.literal('', branches.tag_member.name),
-                nodes.Text(' is '),
-                nodes.literal('', '"%s"' % variant.name)]
-        if variant.ifcond.is_present():
-            term.extend(self._nodes_for_ifcond(variant.ifcond))
-        return term
-
-    def _nodes_for_members(self, doc, what, base=None, branches=None):
-        """Return list of doctree nodes for the table of members"""
-        dlnode = nodes.definition_list()
-        for section in doc.args.values():
-            term = self._nodes_for_one_member(section.member)
-            # TODO drop fallbacks when undocumented members are outlawed
-            if section.text:
-                defn = dedent(section.text)
-            else:
-                defn = [nodes.Text('Not documented')]
-
-            dlnode += self._make_dlitem(term, defn)
-
-        if base:
-            dlnode += self._make_dlitem([nodes.Text('The members of '),
-                                         nodes.literal('', base.doc_type())],
-                                        None)
-
-        if branches:
-            for v in branches.variants:
-                if v.type.name == 'q_empty':
-                    continue
-                assert not v.type.is_implicit()
-                term = [nodes.Text('The members of '),
-                        nodes.literal('', v.type.doc_type())]
-                term.extend(self._nodes_for_variant_when(branches, v))
-                dlnode += self._make_dlitem(term, None)
-
-        if not dlnode.children:
-            return []
-
-        section = self._make_section(what)
-        section += dlnode
-        return [section]
-
-    def _nodes_for_enum_values(self, doc):
-        """Return list of doctree nodes for the table of enum values"""
-        seen_item = False
-        dlnode = nodes.definition_list()
-        for section in doc.args.values():
-            termtext = [nodes.literal('', section.member.name)]
-            if section.member.ifcond.is_present():
-                termtext.extend(self._nodes_for_ifcond(section.member.ifcond))
-            # TODO drop fallbacks when undocumented members are outlawed
-            if section.text:
-                defn = dedent(section.text)
-            else:
-                defn = [nodes.Text('Not documented')]
-
-            dlnode += self._make_dlitem(termtext, defn)
-            seen_item = True
-
-        if not seen_item:
-            return []
-
-        section = self._make_section('Values')
-        section += dlnode
-        return [section]
-
-    def _nodes_for_arguments(self, doc, arg_type):
-        """Return list of doctree nodes for the arguments section"""
-        if arg_type and not arg_type.is_implicit():
-            assert not doc.args
-            section = self._make_section('Arguments')
-            dlnode = nodes.definition_list()
-            dlnode += self._make_dlitem(
-                [nodes.Text('The members of '),
-                 nodes.literal('', arg_type.name)],
-                None)
-            section += dlnode
-            return [section]
-
-        return self._nodes_for_members(doc, 'Arguments')
-
-    def _nodes_for_features(self, doc):
-        """Return list of doctree nodes for the table of features"""
-        seen_item = False
-        dlnode = nodes.definition_list()
-        for section in doc.features.values():
-            dlnode += self._make_dlitem(
-                [nodes.literal('', section.member.name)], dedent(section.text))
-            seen_item = True
-
-        if not seen_item:
-            return []
-
-        section = self._make_section('Features')
-        section += dlnode
-        return [section]
-
-    def _nodes_for_sections(self, doc):
-        """Return list of doctree nodes for additional sections"""
-        nodelist = []
-        for section in doc.sections:
-            if section.kind == QAPIDoc.Kind.TODO:
-                # Hide TODO: sections
-                continue
-
-            if section.kind == QAPIDoc.Kind.PLAIN:
-                # Sphinx cannot handle sectionless titles;
-                # Instead, just append the results to the prior section.
-                container = nodes.container()
-                self._parse_text_into_node(section.text, container)
-                nodelist += container.children
-                continue
-
-            snode = self._make_section(section.kind.name.title())
-            self._parse_text_into_node(dedent(section.text), snode)
-            nodelist.append(snode)
-        return nodelist
-
-    def _nodes_for_if_section(self, ifcond):
-        """Return list of doctree nodes for the "If" section"""
-        nodelist = []
-        if ifcond.is_present():
-            snode = self._make_section('If')
-            snode += nodes.paragraph(
-                '', '', *self._nodes_for_ifcond(ifcond, with_if=False)
-            )
-            nodelist.append(snode)
-        return nodelist
-
-    def _add_doc(self, typ, sections):
-        """Add documentation for a command/object/enum...
-
-        We assume we're documenting the thing defined in self._cur_doc.
-        typ is the type of thing being added ("Command", "Object", etc)
-
-        sections is a list of nodes for sections to add to the definition.
-        """
-
-        doc = self._cur_doc
-        snode = nodes.section(ids=[self._sphinx_directive.new_serialno()])
-        snode += nodes.title('', '', *[nodes.literal(doc.symbol, doc.symbol),
-                                       nodes.Text(' (' + typ + ')')])
-        self._parse_text_into_node(doc.body.text, snode)
-        for s in sections:
-            if s is not None:
-                snode += s
-        self._add_node_to_current_heading(snode)
-
-    def visit_enum_type(self, name, info, ifcond, features, members, prefix):
-        doc = self._cur_doc
-        self._add_doc('Enum',
-                      self._nodes_for_enum_values(doc)
-                      + self._nodes_for_features(doc)
-                      + self._nodes_for_sections(doc)
-                      + self._nodes_for_if_section(ifcond))
-
-    def visit_object_type(self, name, info, ifcond, features,
-                          base, members, branches):
-        doc = self._cur_doc
-        if base and base.is_implicit():
-            base = None
-        self._add_doc('Object',
-                      self._nodes_for_members(doc, 'Members', base, branches)
-                      + self._nodes_for_features(doc)
-                      + self._nodes_for_sections(doc)
-                      + self._nodes_for_if_section(ifcond))
-
-    def visit_alternate_type(self, name, info, ifcond, features,
-                             alternatives):
-        doc = self._cur_doc
-        self._add_doc('Alternate',
-                      self._nodes_for_members(doc, 'Members')
-                      + self._nodes_for_features(doc)
-                      + self._nodes_for_sections(doc)
-                      + self._nodes_for_if_section(ifcond))
-
-    def visit_command(self, name, info, ifcond, features, arg_type,
-                      ret_type, gen, success_response, boxed, allow_oob,
-                      allow_preconfig, coroutine):
-        doc = self._cur_doc
-        self._add_doc('Command',
-                      self._nodes_for_arguments(doc, arg_type)
-                      + self._nodes_for_features(doc)
-                      + self._nodes_for_sections(doc)
-                      + self._nodes_for_if_section(ifcond))
-
-    def visit_event(self, name, info, ifcond, features, arg_type, boxed):
-        doc = self._cur_doc
-        self._add_doc('Event',
-                      self._nodes_for_arguments(doc, arg_type)
-                      + self._nodes_for_features(doc)
-                      + self._nodes_for_sections(doc)
-                      + self._nodes_for_if_section(ifcond))
-
-    def symbol(self, doc, entity):
-        """Add documentation for one symbol to the document tree
-
-        This is the main entry point which causes us to add documentation
-        nodes for a symbol (which could be a 'command', 'object', 'event',
-        etc). We do this by calling 'visit' on the schema entity, which
-        will then call back into one of our visit_* methods, depending
-        on what kind of thing this symbol is.
-        """
-        self._cur_doc = doc
-        entity.visit(self)
-        self._cur_doc = None
-
-    def _start_new_heading(self, heading, level):
-        """Start a new heading at the specified heading level
-
-        Create a new section whose title is 'heading' and which is placed
-        in the docutils node tree as a child of the most recent level-1
-        heading. Subsequent document sections (commands, freeform doc chunks,
-        etc) will be placed as children of this new heading section.
-        """
-        if len(self._active_headings) < level:
-            raise QAPISemError(self._cur_doc.info,
-                               'Level %d subheading found outside a '
-                               'level %d heading'
-                               % (level, level - 1))
-        snode = self._make_section(heading)
-        self._active_headings[level - 1] += snode
-        self._active_headings = self._active_headings[:level]
-        self._active_headings.append(snode)
-        return snode
-
-    def _add_node_to_current_heading(self, node):
-        """Add the node to whatever the current active heading is"""
-        self._active_headings[-1] += node
-
-    def freeform(self, doc):
-        """Add a piece of 'freeform' documentation to the document tree
-
-        A 'freeform' document chunk doesn't relate to any particular
-        symbol (for instance, it could be an introduction).
-
-        If the freeform document starts with a line of the form
-        '= Heading text', this is a section or subsection heading, with
-        the heading level indicated by the number of '=' signs.
-        """
-
-        # QAPIDoc documentation says free-form documentation blocks
-        # must have only a body section, nothing else.
-        assert not doc.sections
-        assert not doc.args
-        assert not doc.features
-        self._cur_doc = doc
-
-        text = doc.body.text
-        if re.match(r'=+ ', text):
-            # Section/subsection heading (if present, will always be
-            # the first line of the block)
-            (heading, _, text) = text.partition('\n')
-            (leader, _, heading) = heading.partition(' ')
-            node = self._start_new_heading(heading, len(leader))
-            if text == '':
-                return
-        else:
-            node = nodes.container()
-
-        self._parse_text_into_node(text, node)
-        self._cur_doc = None
-
-    def _parse_text_into_node(self, doctext, node):
-        """Parse a chunk of QAPI-doc-format text into the node
-
-        The doc comment can contain most inline rST markup, including
-        bulleted and enumerated lists.
-        As an extra permitted piece of markup, @var will be turned
-        into ``var``.
-        """
-
-        # Handle the "@var means ``var`` case
-        doctext = re.sub(r'@([\w-]+)', r'``\1``', doctext)
-
-        rstlist = ViewList()
-        for line in doctext.splitlines():
-            # The reported line number will always be that of the start line
-            # of the doc comment, rather than the actual location of the error.
-            # Being more precise would require overhaul of the QAPIDoc class
-            # to track lines more exactly within all the sub-parts of the doc
-            # comment, as well as counting lines here.
-            rstlist.append(line, self._cur_doc.info.fname,
-                           self._cur_doc.info.line)
-        # Append a blank line -- in some cases rST syntax errors get
-        # attributed to the line after one with actual text, and if there
-        # isn't anything in the ViewList corresponding to that then Sphinx
-        # 1.6's AutodocReporter will then misidentify the source/line location
-        # in the error message (usually attributing it to the top-level
-        # .rst file rather than the offending .json file). The extra blank
-        # line won't affect the rendered output.
-        rstlist.append("", self._cur_doc.info.fname, self._cur_doc.info.line)
-        self._sphinx_directive.do_parse(rstlist, node)
-
-    def get_document_node(self):
-        """Return the root docutils node which makes up the document"""
-        return self._top_node
diff --git a/python/tests/qapi-isort.sh b/python/tests/qapi-isort.sh
index 78dd947f68c..067c16d5d94 100755
--- a/python/tests/qapi-isort.sh
+++ b/python/tests/qapi-isort.sh
@@ -3,6 +3,6 @@
 
 python3 -m isort --sp . -c ../scripts/qapi/
 # Force isort to recognize "compat" as a local module and not third-party
-python3 -m isort --sp . -c -p compat -p qapidoc_legacy \
+python3 -m isort --sp . -c -p compat \
         ../docs/sphinx/qapi_domain.py \
         ../docs/sphinx/qapidoc.py
diff --git a/tests/qapi-schema/doc-good.txt b/tests/qapi-schema/doc-good.txt
index 17a1d56ef1a..74b73681d32 100644
--- a/tests/qapi-schema/doc-good.txt
+++ b/tests/qapi-schema/doc-good.txt
@@ -1,6 +1,8 @@
 Section
 *******
 
+Just text, no heading.
+
 
 Subsection
 ==========
@@ -35,249 +37,145 @@ Example:
 
 -> in <- out Examples: - *verbatim* - {braces}
 
+Enum Enum
+    *Availability*: "IFCOND"
 
-"Enum" (Enum)
--------------
+   Values:
+      * **one** -- The _one_ {and only}, description on the same line
 
+      * **two** -- Not documented
 
-Values
-~~~~~~
+   Features:
+      * **enum-feat** -- Also _one_ {and only}
 
-"one" (**If: **"IFONE")
-   The _one_ {and only}, description on the same line
+      * **enum-member-feat** -- a member feature
 
-"two"
-   Not documented
+   "two" is undocumented
 
+Object Base
+    *Availability*: "IFALL1 and IFALL2"
 
-Features
-~~~~~~~~
+   Members:
+      * **base1** ("Enum") -- description starts on a new line,
+        minimally indented
 
-"enum-feat"
-   Also _one_ {and only}
+Object Variant1
 
-"enum-member-feat"
-   a member feature
+   A paragraph
 
-"two" is undocumented
+   Another paragraph
 
+   "var1" is undocumented
 
-If
-~~
+   Members:
+      * **var1** ("string") -- Not documented
 
-"IFCOND"
+   Features:
+      * **variant1-feat** -- a feature
 
+      * **member-feat** -- a member feature
 
-"Base" (Object)
----------------
+Object Variant2
 
+Object Object
 
-Members
-~~~~~~~
+   Members:
+      * The members of "Base".
 
-"base1": "Enum"
-   description starts on a new line, minimally indented
+      * When "base1" is "one": The members of "Variant1".
 
+      * When "base1" is "two": The members of "Variant2".
 
-If
-~~
+   Features:
+      * **union-feat1** -- a feature
 
-"IFALL1 and IFALL2"
+Alternate Alternate
+    *Availability*: "not (IFONE or IFTWO)"
 
+   Alternatives:
+      * **i** ("int") -- description starts on the same line remainder
+        indented the same "b" is undocumented
 
-"Variant1" (Object)
--------------------
+      * **b** ("boolean") -- Not documented
 
-A paragraph
-
-Another paragraph
-
-"var1" is undocumented
-
-
-Members
-~~~~~~~
-
-"var1": "string" (**If: **"IFSTR")
-   Not documented
-
-
-Features
-~~~~~~~~
-
-"variant1-feat"
-   a feature
-
-"member-feat"
-   a member feature
-
-
-"Variant2" (Object)
--------------------
-
-
-"Object" (Object)
------------------
-
-
-Members
-~~~~~~~
-
-The members of "Base"
-The members of "Variant1" when "base1" is ""one""
-The members of "Variant2" when "base1" is ""two"" (**If: **"IFONE or
-IFTWO")
-
-Features
-~~~~~~~~
-
-"union-feat1"
-   a feature
-
-
-"Alternate" (Alternate)
------------------------
-
-
-Members
-~~~~~~~
-
-"i": "int"
-   description starts on the same line remainder indented the same "b"
-   is undocumented
-
-"b": "boolean"
-   Not documented
-
-
-Features
-~~~~~~~~
-
-"alt-feat"
-   a feature
-
-
-If
-~~
-
-"not (IFONE or IFTWO)"
+   Features:
+      * **alt-feat** -- a feature
 
 
 Another subsection
 ==================
 
+Command cmd (Since: 2.10)
 
-"cmd" (Command)
----------------
+   Arguments:
+      * **arg1** ("int") -- description starts on a new line, indented
 
+      * **arg2** ("string", *optional*) -- description starts on the
+        same line remainder indented differently
 
-Arguments
-~~~~~~~~~
+      * **arg3** ("boolean") -- Not documented
 
-"arg1": "int"
-   description starts on a new line, indented
+   Features:
+      * **cmd-feat1** -- a feature
 
-"arg2": "string" (optional)
-   description starts on the same line remainder indented differently
+      * **cmd-feat2** -- another feature
 
-"arg3": "boolean"
-   Not documented
+   Note:
 
+     "arg3" is undocumented
 
-Features
-~~~~~~~~
+   Return:
+      "Object" -- "Object"
 
-"cmd-feat1"
-   a feature
+   Errors:
+      some
 
-"cmd-feat2"
-   another feature
+   Notes:
 
-Note:
+   * Lorem ipsum dolor sit amet
 
-  "arg3" is undocumented
+   * Ut enim ad minim veniam
 
+   Duis aute irure dolor
 
-Returns
-~~~~~~~
+   Example: Ideal fast-food burger situation:
 
-"Object"
+      -> "in"
+      <- "out"
 
+   Examples:
 
-Errors
-~~~~~~
+      - Not a QMP code block
+      - Merely a preformatted code block literal
+      It isn't even an rST list.
+      - *verbatim*
+      - {braces}
 
-some
+   Note::
+      Ceci n'est pas une note
 
-Notes:
+Command cmd-boxed
 
-* Lorem ipsum dolor sit amet
+   If you're bored enough to read this, go see a video of boxed cats
 
-* Ut enim ad minim veniam
+   Arguments:
+      * The members of "Object".
 
-Duis aute irure dolor
+   Features:
+      * **cmd-feat1** -- a feature
 
-Example: Ideal fast-food burger situation:
+      * **cmd-feat2** -- another feature
 
-   -> "in"
-   <- "out"
+   Example::
 
-Examples:
+      -> "this example"
 
-   - Not a QMP code block
-   - Merely a preformatted code block literal
-   It isn't even an rST list.
-   - *verbatim*
-   - {braces}
+      <- ... has no title ...
 
-Note::
-   Ceci n'est pas une note
+Event EVT_BOXED
 
+   Members:
+      * The members of "Object".
 
-Since
-~~~~~
-
-2.10
-
-
-"cmd-boxed" (Command)
----------------------
-
-If you're bored enough to read this, go see a video of boxed cats
-
-
-Arguments
-~~~~~~~~~
-
-The members of "Object"
-
-Features
-~~~~~~~~
-
-"cmd-feat1"
-   a feature
-
-"cmd-feat2"
-   another feature
-
-Example::
-
-   -> "this example"
-
-   <- ... has no title ...
-
-
-"EVT_BOXED" (Event)
--------------------
-
-
-Arguments
-~~~~~~~~~
-
-The members of "Object"
-
-Features
-~~~~~~~~
-
-"feat3"
-   a feature
+   Features:
+      * **feat3** -- a feature
-- 
2.48.1

[PATCH v3 4/5] docs/sphinx: remove special parsing for freeform sections

[John Snow]

This change removes special parsing for freeform sections and allows
them to simply be unmodified rST syntax. The existing headings in the
QAPI schema are adjusted to reflect the new paradigm.

Tests and documentation are updated to match.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/devel/qapi-code-gen.rst         | 28 ++++++++++++++-------
 docs/interop/firmware.json           |  4 ++-
 docs/interop/vhost-user.json         |  4 ++-
 docs/sphinx/qapidoc.py               | 37 +---------------------------
 qapi/acpi.json                       |  4 ++-
 qapi/audio.json                      |  4 ++-
 qapi/authz.json                      |  4 ++-
 qapi/block-core.json                 |  3 ++-
 qapi/block-export.json               |  3 ++-
 qapi/block.json                      |  7 ++++--
 qapi/char.json                       |  4 ++-
 qapi/common.json                     |  4 ++-
 qapi/compat.json                     |  4 ++-
 qapi/control.json                    |  4 ++-
 qapi/crypto.json                     |  4 ++-
 qapi/cryptodev.json                  |  4 ++-
 qapi/cxl.json                        |  4 ++-
 qapi/dump.json                       |  4 ++-
 qapi/ebpf.json                       |  4 ++-
 qapi/error.json                      |  4 ++-
 qapi/introspect.json                 |  4 ++-
 qapi/job.json                        |  4 ++-
 qapi/machine-common.json             |  4 ++-
 qapi/machine.json                    |  4 ++-
 qapi/migration.json                  |  4 ++-
 qapi/misc.json                       |  4 ++-
 qapi/net.json                        |  4 ++-
 qapi/pci.json                        |  4 ++-
 qapi/qapi-schema.json                |  4 ++-
 qapi/qdev.json                       |  4 ++-
 qapi/qom.json                        |  4 ++-
 qapi/replay.json                     |  4 ++-
 qapi/rocker.json                     |  4 ++-
 qapi/run-state.json                  |  4 ++-
 qapi/sockets.json                    |  4 ++-
 qapi/stats.json                      |  4 ++-
 qapi/tpm.json                        |  4 ++-
 qapi/trace.json                      |  4 ++-
 qapi/transaction.json                |  4 ++-
 qapi/uefi.json                       |  4 ++-
 qapi/ui.json                         | 14 ++++++++---
 qapi/vfio.json                       |  4 ++-
 qapi/virtio.json                     |  4 ++-
 qapi/yank.json                       |  4 ++-
 scripts/qapi/parser.py               |  7 ------
 storage-daemon/qapi/qapi-schema.json |  8 ++++--
 tests/qapi-schema/doc-good.json      | 10 +++++---
 tests/qapi-schema/doc-good.out       | 10 +++++---
 48 files changed, 173 insertions(+), 106 deletions(-)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index 231cc0fecf7..dfdbeac5a5a 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -876,25 +876,35 @@ structuring content.
 Headings and subheadings
 ~~~~~~~~~~~~~~~~~~~~~~~~
 
-A free-form documentation comment containing a line which starts with
-some ``=`` symbols and then a space defines a section heading::
+Free-form documentation does not start with ``@SYMBOL`` and can contain
+arbitrary rST markup. Headings can be marked up using the standard rST
+syntax::
 
     ##
-    # = This is a top level heading
+    # *************************
+    # This is a level 2 heading
+    # *************************
     #
     # This is a free-form comment which will go under the
     # top level heading.
     ##
 
     ##
-    # == This is a second level heading
+    # This is a third level heading
+    # ==============================
+    #
+    # Level 4
+    # _______
+    #
+    # Level 5
+    # ^^^^^^^
+    #
+    # Level 6
+    # """""""
     ##
 
-A heading line must be the first line of the documentation
-comment block.
-
-Section headings must always be correctly nested, so you can only
-define a third-level heading inside a second-level heading, and so on.
+Level 1 headings are reserved for use by the generated documentation
+page itself, leaving level 2 as the highest level that should be used.
 
 
 Documentation markup
diff --git a/docs/interop/firmware.json b/docs/interop/firmware.json
index 745d21d8223..f46fdedfa89 100644
--- a/docs/interop/firmware.json
+++ b/docs/interop/firmware.json
@@ -11,7 +11,9 @@
 # later. See the COPYING file in the top-level directory.
 
 ##
-# = Firmware
+# ********
+# Firmware
+# ********
 ##
 
 { 'pragma': {
diff --git a/docs/interop/vhost-user.json b/docs/interop/vhost-user.json
index b6ade9e4931..095eb99cf79 100644
--- a/docs/interop/vhost-user.json
+++ b/docs/interop/vhost-user.json
@@ -10,7 +10,9 @@
 # later. See the COPYING file in the top-level directory.
 
 ##
-# = vhost user backend discovery & capabilities
+# *******************************************
+# vhost user backend discovery & capabilities
+# *******************************************
 ##
 
 ##
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index d5d2b5eeb50..b794cde6971 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -399,44 +399,9 @@ def visit_module(self, path: str) -> None:
         self.ensure_blank_line()
 
     def visit_freeform(self, doc: QAPIDoc) -> None:
-        # TODO: Once the old qapidoc transformer is deprecated, freeform
-        # sections can be updated to pure rST, and this transformed removed.
-        #
-        # For now, translate our micro-format into rST. Code adapted
-        # from Peter Maydell's freeform().
-
         assert len(doc.all_sections) == 1, doc.all_sections
         body = doc.all_sections[0]
-        text = self.reformat_arobase(body.text)
-        info = doc.info
-
-        if re.match(r"=+ ", text):
-            # Section/subsection heading (if present, will always be the
-            # first line of the block)
-            (heading, _, text) = text.partition("\n")
-            (leader, _, heading) = heading.partition(" ")
-            # Implicit +1 for heading in the containing .rst doc
-            level = len(leader) + 1
-
-            # https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#sections
-            markers = ' #*=_^"'
-            overline = level <= 2
-            marker = markers[level]
-
-            self.ensure_blank_line()
-            # This credits all 2 or 3 lines to the single source line.
-            if overline:
-                self.add_line(marker * len(heading), info)
-            self.add_line(heading, info)
-            self.add_line(marker * len(heading), info)
-            self.ensure_blank_line()
-
-            # Eat blank line(s) and advance info
-            trimmed = text.lstrip("\n")
-            text = trimmed
-            info = info.next_line(len(text) - len(trimmed) + 1)
-
-        self.add_lines(text, info)
+        self.add_lines(self.reformat_arobase(body.text), doc.info)
         self.ensure_blank_line()
 
     def visit_entity(self, ent: QAPISchemaDefinition) -> None:
diff --git a/qapi/acpi.json b/qapi/acpi.json
index 2d53b823656..90f8f564fda 100644
--- a/qapi/acpi.json
+++ b/qapi/acpi.json
@@ -6,7 +6,9 @@
 # SPDX-License-Identifier: GPL-2.0-or-later
 
 ##
-# = ACPI
+# ****
+# ACPI
+# ****
 ##
 
 ##
diff --git a/qapi/audio.json b/qapi/audio.json
index 16de231f6d8..6a22ca384aa 100644
--- a/qapi/audio.json
+++ b/qapi/audio.json
@@ -7,7 +7,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Audio
+# *****
+# Audio
+# *****
 ##
 
 ##
diff --git a/qapi/authz.json b/qapi/authz.json
index 7fc6e3032ea..ad1b4b3af0c 100644
--- a/qapi/authz.json
+++ b/qapi/authz.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = User authorization
+# ******************
+# User authorization
+# ******************
 ##
 
 ##
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 1df6644f0de..8b413946ff8 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2,7 +2,8 @@
 # vim: filetype=python
 
 ##
-# == Block core (VM unrelated)
+# Block core (VM unrelated)
+# =========================
 ##
 
 { 'include': 'common.json' }
diff --git a/qapi/block-export.json b/qapi/block-export.json
index ed4deb54db2..2241bfecf25 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -2,7 +2,8 @@
 # vim: filetype=python
 
 ##
-# == Block device exports
+# Block device exports
+# ====================
 ##
 
 { 'include': 'sockets.json' }
diff --git a/qapi/block.json b/qapi/block.json
index 1490a1a17f8..2d54a81c366 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -2,13 +2,16 @@
 # vim: filetype=python
 
 ##
-# = Block devices
+# *************
+# Block devices
+# *************
 ##
 
 { 'include': 'block-core.json' }
 
 ##
-# == Additional block stuff (VM related)
+# Additional block stuff (VM related)
+# ===================================
 ##
 
 ##
diff --git a/qapi/char.json b/qapi/char.json
index df6e325e2e1..f38d04986dd 100644
--- a/qapi/char.json
+++ b/qapi/char.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Character devices
+# *****************
+# Character devices
+# *****************
 ##
 
 { 'include': 'sockets.json' }
diff --git a/qapi/common.json b/qapi/common.json
index 0e3a0bbbfb0..af7e3d618a7 100644
--- a/qapi/common.json
+++ b/qapi/common.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = Common data types
+# *****************
+# Common data types
+# *****************
 ##
 
 ##
diff --git a/qapi/compat.json b/qapi/compat.json
index 42034d9368c..90b8d51cf27 100644
--- a/qapi/compat.json
+++ b/qapi/compat.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = Compatibility policy
+# ********************
+# Compatibility policy
+# ********************
 ##
 
 ##
diff --git a/qapi/control.json b/qapi/control.json
index 34b733f63b6..ab0b3a3bbe5 100644
--- a/qapi/control.json
+++ b/qapi/control.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = QMP monitor control
+# *******************
+# QMP monitor control
+# *******************
 ##
 
 ##
diff --git a/qapi/crypto.json b/qapi/crypto.json
index 9ec6301e188..21006de36c4 100644
--- a/qapi/crypto.json
+++ b/qapi/crypto.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Cryptography
+# ************
+# Cryptography
+# ************
 ##
 
 ##
diff --git a/qapi/cryptodev.json b/qapi/cryptodev.json
index b13db264034..1f49e1822c0 100644
--- a/qapi/cryptodev.json
+++ b/qapi/cryptodev.json
@@ -5,7 +5,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Cryptography devices
+# ********************
+# Cryptography devices
+# ********************
 ##
 
 ##
diff --git a/qapi/cxl.json b/qapi/cxl.json
index 8f2e9237b19..52cc5d4f336 100644
--- a/qapi/cxl.json
+++ b/qapi/cxl.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = CXL devices
+# ***********
+# CXL devices
+# ***********
 ##
 
 ##
diff --git a/qapi/dump.json b/qapi/dump.json
index d0ba1f0596f..0642ca157b8 100644
--- a/qapi/dump.json
+++ b/qapi/dump.json
@@ -5,7 +5,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Dump guest memory
+# *****************
+# Dump guest memory
+# *****************
 ##
 
 ##
diff --git a/qapi/ebpf.json b/qapi/ebpf.json
index db19ae850fc..d45054e6663 100644
--- a/qapi/ebpf.json
+++ b/qapi/ebpf.json
@@ -5,7 +5,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = eBPF Objects
+# ************
+# eBPF Objects
+# ************
 #
 # eBPF object is an ELF binary that contains the eBPF program and eBPF
 # map description(BTF).  Overall, eBPF object should contain the
diff --git a/qapi/error.json b/qapi/error.json
index 135c1e82319..54cb02fb880 100644
--- a/qapi/error.json
+++ b/qapi/error.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = QMP errors
+# **********
+# QMP errors
+# **********
 ##
 
 ##
diff --git a/qapi/introspect.json b/qapi/introspect.json
index e9e02972821..26d8839f19c 100644
--- a/qapi/introspect.json
+++ b/qapi/introspect.json
@@ -10,7 +10,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = QMP introspection
+# *****************
+# QMP introspection
+# *****************
 ##
 
 ##
diff --git a/qapi/job.json b/qapi/job.json
index 126fa5ce602..16b280f52f8 100644
--- a/qapi/job.json
+++ b/qapi/job.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = Background jobs
+# ***************
+# Background jobs
+# ***************
 ##
 
 ##
diff --git a/qapi/machine-common.json b/qapi/machine-common.json
index 298e51f373a..0f01599130c 100644
--- a/qapi/machine-common.json
+++ b/qapi/machine-common.json
@@ -5,7 +5,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Common machine types
+# ********************
+# Common machine types
+# ********************
 ##
 
 ##
diff --git a/qapi/machine.json b/qapi/machine.json
index 0650b8de71a..6ebb99dfabe 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -5,7 +5,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Machines
+# ********
+# Machines
+# ********
 ##
 
 { 'include': 'common.json' }
diff --git a/qapi/migration.json b/qapi/migration.json
index 4963f6ca127..84f4c800f7b 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Migration
+# *********
+# Migration
+# *********
 ##
 
 { 'include': 'common.json' }
diff --git a/qapi/misc.json b/qapi/misc.json
index 4b9e601cfa5..a180c16db25 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Miscellanea
+# ***********
+# Miscellanea
+# ***********
 ##
 
 { 'include': 'common.json' }
diff --git a/qapi/net.json b/qapi/net.json
index 97ea1839813..3b03843c165 100644
--- a/qapi/net.json
+++ b/qapi/net.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Net devices
+# ***********
+# Net devices
+# ***********
 ##
 
 { 'include': 'sockets.json' }
diff --git a/qapi/pci.json b/qapi/pci.json
index dc85a41d28b..a8671cd9ac3 100644
--- a/qapi/pci.json
+++ b/qapi/pci.json
@@ -6,7 +6,9 @@
 # SPDX-License-Identifier: GPL-2.0-or-later
 
 ##
-# = PCI
+# ***
+# PCI
+# ***
 ##
 
 ##
diff --git a/qapi/qapi-schema.json b/qapi/qapi-schema.json
index a8f66163cb7..49b9a0267d3 100644
--- a/qapi/qapi-schema.json
+++ b/qapi/qapi-schema.json
@@ -1,7 +1,9 @@
 # -*- Mode: Python -*-
 # vim: filetype=python
 ##
-# = Introduction
+# ************
+# Introduction
+# ************
 #
 # This manual describes the commands and events supported by the QEMU
 # Monitor Protocol (QMP).
diff --git a/qapi/qdev.json b/qapi/qdev.json
index 32c7d100463..441ed518b87 100644
--- a/qapi/qdev.json
+++ b/qapi/qdev.json
@@ -5,7 +5,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Device infrastructure (qdev)
+# ****************************
+# Device infrastructure (qdev)
+# ****************************
 ##
 
 { 'include': 'qom.json' }
diff --git a/qapi/qom.json b/qapi/qom.json
index 3e8debf78c2..c8fe0258a5f 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -10,7 +10,9 @@
 { 'include': 'crypto.json' }
 
 ##
-# = QEMU Object Model (QOM)
+# ***********************
+# QEMU Object Model (QOM)
+# ***********************
 ##
 
 ##
diff --git a/qapi/replay.json b/qapi/replay.json
index 35e0c4a6926..e46c5c1d3f3 100644
--- a/qapi/replay.json
+++ b/qapi/replay.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Record/replay
+# *************
+# Record/replay
+# *************
 ##
 
 { 'include': 'common.json' }
diff --git a/qapi/rocker.json b/qapi/rocker.json
index 0c7ef1f77c8..e4949649526 100644
--- a/qapi/rocker.json
+++ b/qapi/rocker.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = Rocker switch device
+# ********************
+# Rocker switch device
+# ********************
 ##
 
 ##
diff --git a/qapi/run-state.json b/qapi/run-state.json
index fd09beb35cb..083a3c5eb30 100644
--- a/qapi/run-state.json
+++ b/qapi/run-state.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = VM run state
+# ************
+# VM run state
+# ************
 ##
 
 ##
diff --git a/qapi/sockets.json b/qapi/sockets.json
index f9f559dabae..b5f4a8fecda 100644
--- a/qapi/sockets.json
+++ b/qapi/sockets.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = Socket data types
+# *****************
+# Socket data types
+# *****************
 ##
 
 ##
diff --git a/qapi/stats.json b/qapi/stats.json
index 8902ef94e08..78b88881eab 100644
--- a/qapi/stats.json
+++ b/qapi/stats.json
@@ -9,7 +9,9 @@
 # SPDX-License-Identifier: GPL-2.0-or-later
 
 ##
-# = Statistics
+# **********
+# Statistics
+# **********
 ##
 
 ##
diff --git a/qapi/tpm.json b/qapi/tpm.json
index a16a72edb98..e66b107f1d0 100644
--- a/qapi/tpm.json
+++ b/qapi/tpm.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = TPM (trusted platform module) devices
+# *************************************
+# TPM (trusted platform module) devices
+# *************************************
 ##
 
 ##
diff --git a/qapi/trace.json b/qapi/trace.json
index eb5f63f5135..d08c9c6a889 100644
--- a/qapi/trace.json
+++ b/qapi/trace.json
@@ -7,7 +7,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Tracing
+# *******
+# Tracing
+# *******
 ##
 
 ##
diff --git a/qapi/transaction.json b/qapi/transaction.json
index 9d9e7af26cb..927035f5a7e 100644
--- a/qapi/transaction.json
+++ b/qapi/transaction.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Transactions
+# ************
+# Transactions
+# ************
 ##
 
 { 'include': 'block-core.json' }
diff --git a/qapi/uefi.json b/qapi/uefi.json
index 6592183d6cf..a206c2e9539 100644
--- a/qapi/uefi.json
+++ b/qapi/uefi.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = UEFI Variable Store
+# *******************
+# UEFI Variable Store
+# *******************
 #
 # The QEMU efi variable store implementation (hw/uefi/) uses this to
 # store non-volatile variables in json format on disk.
diff --git a/qapi/ui.json b/qapi/ui.json
index 514fa159b10..f5e77ae19d7 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Remote desktop
+# **************
+# Remote desktop
+# **************
 ##
 
 { 'include': 'common.json' }
@@ -200,7 +202,8 @@
   'if': 'CONFIG_PIXMAN' }
 
 ##
-# == Spice
+# Spice
+# =====
 ##
 
 ##
@@ -461,7 +464,8 @@
   'if': 'CONFIG_SPICE' }
 
 ##
-# == VNC
+# VNC
+# ===
 ##
 
 ##
@@ -794,7 +798,9 @@
   'if': 'CONFIG_VNC' }
 
 ##
-# = Input
+# *****
+# Input
+# *****
 ##
 
 ##
diff --git a/qapi/vfio.json b/qapi/vfio.json
index b53b7caecd7..a1a9c5b673d 100644
--- a/qapi/vfio.json
+++ b/qapi/vfio.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = VFIO devices
+# ************
+# VFIO devices
+# ************
 ##
 
 ##
diff --git a/qapi/virtio.json b/qapi/virtio.json
index 73df718a261..3cac0774f7a 100644
--- a/qapi/virtio.json
+++ b/qapi/virtio.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Virtio devices
+# **************
+# Virtio devices
+# **************
 ##
 
 ##
diff --git a/qapi/yank.json b/qapi/yank.json
index 30f46c97c98..d13a8e91171 100644
--- a/qapi/yank.json
+++ b/qapi/yank.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Yank feature
+# ************
+# Yank feature
+# ************
 ##
 
 ##
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 949d9e8bff7..aad7e249f86 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -597,22 +597,15 @@ def get_doc(self) -> 'QAPIDoc':
             # Free-form documentation
             doc = QAPIDoc(info)
             doc.ensure_untagged_section(self.info)
-            first = True
             while line is not None:
                 if match := self._match_at_name_colon(line):
                     raise QAPIParseError(
                         self,
                         "'@%s:' not allowed in free-form documentation"
                         % match.group(1))
-                if line.startswith('='):
-                    if not first:
-                        raise QAPIParseError(
-                            self,
-                            "'=' heading must come first in a comment block")
                 doc.append_line(line)
                 self.accept(False)
                 line = self.get_doc_line()
-                first = False
 
         self.accept()
         doc.end()
diff --git a/storage-daemon/qapi/qapi-schema.json b/storage-daemon/qapi/qapi-schema.json
index 0427594c984..478e7a92b21 100644
--- a/storage-daemon/qapi/qapi-schema.json
+++ b/storage-daemon/qapi/qapi-schema.json
@@ -14,7 +14,9 @@
 # storage daemon.
 
 ##
-# = Introduction
+# ************
+# Introduction
+# ************
 #
 # This manual describes the commands and events supported by the QEMU
 # storage daemon QMP.
@@ -51,7 +53,9 @@
 { 'include': '../../qapi/job.json' }
 
 ##
-# = Block devices
+# *************
+# Block devices
+# *************
 ##
 { 'include': '../../qapi/block-core.json' }
 { 'include': '../../qapi/block-export.json' }
diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index 14b808f9090..fac13425b72 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -8,7 +8,9 @@
     'documentation-exceptions': [ 'Enum', 'Variant1', 'Alternate', 'cmd' ] } }
 
 ##
-# = Section
+# *******
+# Section
+# *******
 ##
 
 ##
@@ -16,7 +18,8 @@
 ##
 
 ##
-# == Subsection
+# Subsection
+# ==========
 #
 # *with emphasis*
 # @var {in braces}
@@ -144,7 +147,8 @@
   'if': { 'not': { 'any': [ 'IFONE', 'IFTWO' ] } } }
 
 ##
-# == Another subsection
+# Another subsection
+# ==================
 ##
 
 ##
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index dc8352eed4f..04a55072646 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -55,13 +55,16 @@ event EVT_BOXED Object
     feature feat3
 doc freeform
     body=
-= Section
+*******
+Section
+*******
 doc freeform
     body=
 Just text, no heading.
 doc freeform
     body=
-== Subsection
+Subsection
+==========
 
 *with emphasis*
 @var {in braces}
@@ -155,7 +158,8 @@ description starts on the same line
 a feature
 doc freeform
     body=
-== Another subsection
+Another subsection
+==================
 doc symbol=cmd
     body=
 
-- 
2.48.1

[PATCH v3 5/5] qapi: lift restriction on using '=' in doc blocks

[John Snow]

Signed-off-by: John Snow <jsnow@redhat.com>
---
 scripts/qapi/parser.py                 |  4 ----
 tests/qapi-schema/doc-bad-section.err  |  1 -
 tests/qapi-schema/doc-bad-section.json | 10 ----------
 tests/qapi-schema/doc-bad-section.out  |  0
 tests/qapi-schema/meson.build          |  1 -
 5 files changed, 16 deletions(-)
 delete mode 100644 tests/qapi-schema/doc-bad-section.err
 delete mode 100644 tests/qapi-schema/doc-bad-section.json
 delete mode 100644 tests/qapi-schema/doc-bad-section.out

diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index aad7e249f86..d43a123cd74 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -584,10 +584,6 @@ def get_doc(self) -> 'QAPIDoc':
                         doc.append_line(text)
                     line = self.get_doc_indented(doc)
                     no_more_args = True
-                elif line.startswith('='):
-                    raise QAPIParseError(
-                        self,
-                        "unexpected '=' markup in definition documentation")
                 else:
                     # plain paragraph
                     doc.ensure_untagged_section(self.info)
diff --git a/tests/qapi-schema/doc-bad-section.err b/tests/qapi-schema/doc-bad-section.err
deleted file mode 100644
index 785cacc08c4..00000000000
--- a/tests/qapi-schema/doc-bad-section.err
+++ /dev/null
@@ -1 +0,0 @@
-doc-bad-section.json:5:1: unexpected '=' markup in definition documentation
diff --git a/tests/qapi-schema/doc-bad-section.json b/tests/qapi-schema/doc-bad-section.json
deleted file mode 100644
index 8175d95867e..00000000000
--- a/tests/qapi-schema/doc-bad-section.json
+++ /dev/null
@@ -1,10 +0,0 @@
-# = section within an expression comment
-
-##
-# @Enum:
-# == No good here
-# @one: The _one_ {and only}
-#
-# @two is undocumented
-##
-{ 'enum': 'Enum', 'data': [ 'one', 'two' ] }
diff --git a/tests/qapi-schema/doc-bad-section.out b/tests/qapi-schema/doc-bad-section.out
deleted file mode 100644
index e69de29bb2d..00000000000
diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build
index 9577178b6ff..c47025d16d8 100644
--- a/tests/qapi-schema/meson.build
+++ b/tests/qapi-schema/meson.build
@@ -61,7 +61,6 @@ schemas = [
   'doc-bad-event-arg.json',
   'doc-bad-feature.json',
   'doc-bad-indent.json',
-  'doc-bad-section.json',
   'doc-bad-symbol.json',
   'doc-bad-union-member.json',
   'doc-before-include.json',
-- 
2.48.1

Re: [PATCH v3 5/5] qapi: lift restriction on using '=' in doc blocks

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

Let's explain why.  Here's my attempt:

  We reject lines starting with '=' in definition documentation.  This
  made sense when such lines were headings in free-form documentation,
  but not in definition documentation.

  Before the previous commit, lines starting with '=' were headings in
  free-form documentation, and rejected in definition documentation,
  where such headings could not work.

  The previous commit dropped the headings feature from free-form
  documentation, because we can simply use plain rST headings.
  Rejecting them in definition documentation no longer makes sense, so
  drop that, too.

> Signed-off-by: John Snow <jsnow@redhat.com>

Re: [PATCH v3 4/5] docs/sphinx: remove special parsing for freeform sections

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> This change removes special parsing for freeform sections and allows
> them to simply be unmodified rST syntax. The existing headings in the
> QAPI schema are adjusted to reflect the new paradigm.

"Allows them to" suggests the patch enables use of rST headings.  Is
this the case?  Or do they just work, and this patch just switches
schema code to use them, and drops now unnecessary generator code?
>
> Tests and documentation are updated to match.
>
> Signed-off-by: John Snow <jsnow@redhat.com>

[...]

> diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
> index 231cc0fecf7..dfdbeac5a5a 100644
> --- a/docs/devel/qapi-code-gen.rst
> +++ b/docs/devel/qapi-code-gen.rst
> @@ -876,25 +876,35 @@ structuring content.
   Documentation comments
   ----------------------

   A multi-line comment that starts and ends with a ``##`` line is a
   documentation comment.

   If the documentation comment starts like ::

       ##
       # @SYMBOL:

   it documents the definition of SYMBOL, else it's free-form
   documentation.

   See below for more on `Definition documentation`_.

   Free-form documentation may be used to provide additional text and
   structuring content.


>  Headings and subheadings
>  ~~~~~~~~~~~~~~~~~~~~~~~~
>  
> -A free-form documentation comment containing a line which starts with
> -some ``=`` symbols and then a space defines a section heading::
> +Free-form documentation does not start with ``@SYMBOL`` and can contain
> +arbitrary rST markup. Headings can be marked up using the standard rST
> +syntax::

Nothing stops you from using such markup in definition documentation.
It's probably a bad idea, though.

I think it's easiest not to talk about the two kinds of doc blocks here
at all.  Scratch the first sentence?

>  
>      ##
> -    # = This is a top level heading
> +    # *************************
> +    # This is a level 2 heading
> +    # *************************
>      #
>      # This is a free-form comment which will go under the
>      # top level heading.
>      ##
>  
>      ##
> -    # == This is a second level heading
> +    # This is a third level heading
> +    # ==============================
> +    #
> +    # Level 4
> +    # _______
> +    #
> +    # Level 5
> +    # ^^^^^^^
> +    #
> +    # Level 6
> +    # """""""
>      ##
>  
> -A heading line must be the first line of the documentation
> -comment block.
> -
> -Section headings must always be correctly nested, so you can only
> -define a third-level heading inside a second-level heading, and so on.
> +Level 1 headings are reserved for use by the generated documentation
> +page itself, leaving level 2 as the highest level that should be used.
>  
>  
>  Documentation markup

[...]

Re: [PATCH v3 3/5] docs/sphinx: remove legacy QAPI manual generator

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> Thanks for your service!
>
> Remove the old qapidoc and the option to enable the transmogrifier,
> leaving the "transmogrifier" as the ONLY qapi doc generator. This in
> effect also converts the QAPI test to use the new documentation
> generator, too.
>
> Update doc-good.txt output to match the new doc generator, which I
> should've done exactly when we switched over to the transmogrifier, but,
> uhh, oops!
>
> Notes on the new format:
>  1. per-member IFCOND documentation is missing. Known issue.
>  2. Freeform documentation without a header is now copied through into
>     the output. This is an intentional change.
>

Make that "This is a bug fix", and add

Fixes: b61a4eb3f32 (docs/qapidoc: support header-less freeform sections)

> Signed-off-by: John Snow <jsnow@redhat.com>

Re: [PATCH v3 2/5] docs/sphinx: parse @references in freeform text

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> Oversight in the new qapidoc transmogrifier: @references in freeform
> documentation blocks were not being transformed to literals. This fixes
> that, and the next patch ensures that we're testing for this O:-)
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  docs/sphinx/qapidoc.py | 10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index 5374dee8fad..adc14ade456 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -218,6 +218,11 @@ def generate_field(
>          typ = self.format_type(member)
>          self.add_field(kind, member.name, body, info, typ)
>  
> +    @staticmethod
> +    def reformat_arobase(text: str) -> str:

What's an "arobase"?  Inquiring mind wants to know!

> +        """ reformats @var to ``var`` """
> +        return re.sub(r"@([\w-]+)", r"``\1``", text)
> +
>      # Transmogrification helpers
>  
>      def visit_paragraph(self, section: QAPIDoc.Section) -> None:
> @@ -361,8 +366,7 @@ def visit_sections(self, ent: QAPISchemaDefinition) -> None:
>  
>          # Add sections in source order:
>          for i, section in enumerate(sections):
> -            # @var is translated to ``var``:
> -            section.text = re.sub(r"@([\w-]+)", r"``\1``", section.text)
> +            section.text = self.reformat_arobase(section.text)
>  
>              if section.kind == QAPIDoc.Kind.PLAIN:
>                  self.visit_paragraph(section)
> @@ -405,7 +409,7 @@ def visit_freeform(self, doc: QAPIDoc) -> None:
>  
>          assert len(doc.all_sections) == 1, doc.all_sections
>          body = doc.all_sections[0]
> -        text = body.text
> +        text = self.reformat_arobase(body.text)
>          info = doc.info
>  
>          if re.match(r"=+ ", text):

Re: [PATCH v3 2/5] docs/sphinx: parse @references in freeform text

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 2176 bytes --]

On Fri, Jun 27, 2025, 5:54 AM Markus Armbruster <armbru@redhat.com> wrote:

> John Snow <jsnow@redhat.com> writes:
>
> > Oversight in the new qapidoc transmogrifier: @references in freeform
> > documentation blocks were not being transformed to literals. This fixes
> > that, and the next patch ensures that we're testing for this O:-)
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> >  docs/sphinx/qapidoc.py | 10 +++++++---
> >  1 file changed, 7 insertions(+), 3 deletions(-)
> >
> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> > index 5374dee8fad..adc14ade456 100644
> > --- a/docs/sphinx/qapidoc.py
> > +++ b/docs/sphinx/qapidoc.py
> > @@ -218,6 +218,11 @@ def generate_field(
> >          typ = self.format_type(member)
> >          self.add_field(kind, member.name, body, info, typ)
> >
> > +    @staticmethod
> > +    def reformat_arobase(text: str) -> str:
>
> What's an "arobase"?  Inquiring mind wants to know!
>

French for "at symbol thingie" :)

(Intentionally obtuse as an oblique joke, forgive my humor)


> > +        """ reformats @var to ``var`` """
> > +        return re.sub(r"@([\w-]+)", r"``\1``", text)
> > +
> >      # Transmogrification helpers
> >
> >      def visit_paragraph(self, section: QAPIDoc.Section) -> None:
> > @@ -361,8 +366,7 @@ def visit_sections(self, ent: QAPISchemaDefinition)
> -> None:
> >
> >          # Add sections in source order:
> >          for i, section in enumerate(sections):
> > -            # @var is translated to ``var``:
> > -            section.text = re.sub(r"@([\w-]+)", r"``\1``", section.text)
> > +            section.text = self.reformat_arobase(section.text)
> >
> >              if section.kind == QAPIDoc.Kind.PLAIN:
> >                  self.visit_paragraph(section)
> > @@ -405,7 +409,7 @@ def visit_freeform(self, doc: QAPIDoc) -> None:
> >
> >          assert len(doc.all_sections) == 1, doc.all_sections
> >          body = doc.all_sections[0]
> > -        text = body.text
> > +        text = self.reformat_arobase(body.text)
> >          info = doc.info
> >
> >          if re.match(r"=+ ", text):
>
>

[-- Attachment #2: Type: text/html, Size: 3548 bytes --]

Re: [PATCH v3 4/5] docs/sphinx: remove special parsing for freeform sections

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 3126 bytes --]

On Fri, Jun 27, 2025, 5:52 AM Markus Armbruster <armbru@redhat.com> wrote:

> John Snow <jsnow@redhat.com> writes:
>
> > This change removes special parsing for freeform sections and allows
> > them to simply be unmodified rST syntax. The existing headings in the
> > QAPI schema are adjusted to reflect the new paradigm.
>
> "Allows them to" suggests the patch enables use of rST headings.  Is
> this the case?  Or do they just work, and this patch just switches
> schema code to use them, and drops now unnecessary generator code?
> >
> > Tests and documentation are updated to match.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
> [...]
>
> > diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
> > index 231cc0fecf7..dfdbeac5a5a 100644
> > --- a/docs/devel/qapi-code-gen.rst
> > +++ b/docs/devel/qapi-code-gen.rst
> > @@ -876,25 +876,35 @@ structuring content.
>    Documentation comments
>    ----------------------
>
>    A multi-line comment that starts and ends with a ``##`` line is a
>    documentation comment.
>
>    If the documentation comment starts like ::
>
>        ##
>        # @SYMBOL:
>
>    it documents the definition of SYMBOL, else it's free-form
>    documentation.
>
>    See below for more on `Definition documentation`_.
>
>    Free-form documentation may be used to provide additional text and
>    structuring content.
>
>
> >  Headings and subheadings
> >  ~~~~~~~~~~~~~~~~~~~~~~~~
> >
> > -A free-form documentation comment containing a line which starts with
> > -some ``=`` symbols and then a space defines a section heading::
> > +Free-form documentation does not start with ``@SYMBOL`` and can contain
> > +arbitrary rST markup. Headings can be marked up using the standard rST
> > +syntax::
>
> Nothing stops you from using such markup in definition documentation.
> It's probably a bad idea, though.
>
> I think it's easiest not to talk about the two kinds of doc blocks here
> at all.  Scratch the first sentence?
>

Sure.


> >
> >      ##
> > -    # = This is a top level heading
> > +    # *************************
> > +    # This is a level 2 heading
> > +    # *************************
> >      #
> >      # This is a free-form comment which will go under the
> >      # top level heading.
> >      ##
> >
> >      ##
> > -    # == This is a second level heading
> > +    # This is a third level heading
> > +    # ==============================
> > +    #
> > +    # Level 4
> > +    # _______
> > +    #
> > +    # Level 5
> > +    # ^^^^^^^
> > +    #
> > +    # Level 6
> > +    # """""""
> >      ##
> >
> > -A heading line must be the first line of the documentation
> > -comment block.
> > -
> > -Section headings must always be correctly nested, so you can only
> > -define a third-level heading inside a second-level heading, and so on.
> > +Level 1 headings are reserved for use by the generated documentation
> > +page itself, leaving level 2 as the highest level that should be used.
> >
> >
> >  Documentation markup
>
> [...]
>
>

[-- Attachment #2: Type: text/html, Size: 4366 bytes --]

Re: [PATCH v3 4/5] docs/sphinx: remove special parsing for freeform sections

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 3458 bytes --]

On Fri, Jun 27, 2025, 5:52 AM Markus Armbruster <armbru@redhat.com> wrote:

> John Snow <jsnow@redhat.com> writes:
>
> > This change removes special parsing for freeform sections and allows
> > them to simply be unmodified rST syntax. The existing headings in the
> > QAPI schema are adjusted to reflect the new paradigm.
>
> "Allows them to" suggests the patch enables use of rST headings.  Is
> this the case?  Or do they just work, and this patch just switches
> schema code to use them, and drops now unnecessary generator code?
>

Ehm... Kind of both? I guess I hadn't considered that rST headings might
already work without the switch. I guess they didn't because of the
headerless freeform section bug I fix in this series.

I guess you're technically right, I just never thought of it in that way.

I'll update the message so I don't confuse you in the future.

>
> > Tests and documentation are updated to match.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
> [...]
>
> > diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
> > index 231cc0fecf7..dfdbeac5a5a 100644
> > --- a/docs/devel/qapi-code-gen.rst
> > +++ b/docs/devel/qapi-code-gen.rst
> > @@ -876,25 +876,35 @@ structuring content.
>    Documentation comments
>    ----------------------
>
>    A multi-line comment that starts and ends with a ``##`` line is a
>    documentation comment.
>
>    If the documentation comment starts like ::
>
>        ##
>        # @SYMBOL:
>
>    it documents the definition of SYMBOL, else it's free-form
>    documentation.
>
>    See below for more on `Definition documentation`_.
>
>    Free-form documentation may be used to provide additional text and
>    structuring content.
>
>
> >  Headings and subheadings
> >  ~~~~~~~~~~~~~~~~~~~~~~~~
> >
> > -A free-form documentation comment containing a line which starts with
> > -some ``=`` symbols and then a space defines a section heading::
> > +Free-form documentation does not start with ``@SYMBOL`` and can contain
> > +arbitrary rST markup. Headings can be marked up using the standard rST
> > +syntax::
>
> Nothing stops you from using such markup in definition documentation.
> It's probably a bad idea, though.
>
> I think it's easiest not to talk about the two kinds of doc blocks here
> at all.  Scratch the first sentence?
>
> >
> >      ##
> > -    # = This is a top level heading
> > +    # *************************
> > +    # This is a level 2 heading
> > +    # *************************
> >      #
> >      # This is a free-form comment which will go under the
> >      # top level heading.
> >      ##
> >
> >      ##
> > -    # == This is a second level heading
> > +    # This is a third level heading
> > +    # ==============================
> > +    #
> > +    # Level 4
> > +    # _______
> > +    #
> > +    # Level 5
> > +    # ^^^^^^^
> > +    #
> > +    # Level 6
> > +    # """""""
> >      ##
> >
> > -A heading line must be the first line of the documentation
> > -comment block.
> > -
> > -Section headings must always be correctly nested, so you can only
> > -define a third-level heading inside a second-level heading, and so on.
> > +Level 1 headings are reserved for use by the generated documentation
> > +page itself, leaving level 2 as the highest level that should be used.
> >
> >
> >  Documentation markup
>
> [...]
>
>

[-- Attachment #2: Type: text/html, Size: 4805 bytes --]

Re: [PATCH v3 1/5] docs/sphinx: adjust qapidoc to cope with same-line error sections

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> Signed-off-by: John Snow <jsnow@redhat.com>

Let's mention the reproducer: "# Errors: some" in doc-good.json with
:transmogrify:.

> ---
>  docs/sphinx/qapidoc.py | 12 ++++++++----
>  1 file changed, 8 insertions(+), 4 deletions(-)
>
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index 8011ac9efaf..5374dee8fad 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -267,10 +267,14 @@ def visit_returns(self, section: QAPIDoc.Section) -> None:
>          self.add_field("return", typ, section.text, section.info)
>  
>      def visit_errors(self, section: QAPIDoc.Section) -> None:
> -        # FIXME: the formatting for errors may be inconsistent and may
> -        # or may not require different newline placement to ensure
> -        # proper rendering as a nested list.
> -        self.add_lines(f":error:\n{section.text}", section.info)
> +        # If the section text does not start with a space, it means text
> +        # began on the same line as the "Error:" string and we should
> +        # not insert a newline in this case.
> +        if section.text[0].isspace():
> +            text = f":error:\n{section.text}"
> +        else:
> +            text = f":error: {section.text}"
> +        self.add_lines(text, section.info)
>  
>      def preamble(self, ent: QAPISchemaDefinition) -> None:
>          """

Re: [PATCH v3 4/5] docs/sphinx: remove special parsing for freeform sections

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> On Fri, Jun 27, 2025, 5:52 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > This change removes special parsing for freeform sections and allows
>> > them to simply be unmodified rST syntax. The existing headings in the
>> > QAPI schema are adjusted to reflect the new paradigm.
>>
>> "Allows them to" suggests the patch enables use of rST headings.  Is
>> this the case?  Or do they just work, and this patch just switches
>> schema code to use them, and drops now unnecessary generator code?
>>
>
> Ehm... Kind of both? I guess I hadn't considered that rST headings might
> already work without the switch. I guess they didn't because of the
> headerless freeform section bug I fix in this series.

To double-check master, I use

diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index 14b808f909..201462688f 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -8,7 +8,9 @@
     'documentation-exceptions': [ 'Enum', 'Variant1', 'Alternate', 'cmd' ] } }
 
 ##
-# = Section
+# *******
+# Section
+# *******
 ##
 
 ##

Doesn't work with the legacy doc generator:

    Exception occurred:
      File "/work/armbru/qemu/docs/sphinx/qapidoc_legacy.py", line 360, in _start_new_heading
        raise QAPISemError(self._cur_doc.info,
        ...<2 lines>...
                           % (level, level - 1))
    qapi.error.QAPISemError: /work/armbru/qemu/docs/../tests/qapi-schema/doc-good.json:20: Level 2 subheading found outside a level 1 heading

So switch to the new one:

diff --git a/tests/qapi-schema/doc-good.rst b/tests/qapi-schema/doc-good.rst
index 1e4c23305a..9978ac2e9c 100644
--- a/tests/qapi-schema/doc-good.rst
+++ b/tests/qapi-schema/doc-good.rst
@@ -3,3 +3,4 @@
    a plain-text output file and compare it against a reference.
 
 .. qapi-doc:: tests/qapi-schema/doc-good.json
+   :transmogrify:

Chokes on "# Errors: some", you fix this (unrelated) bug in PATCH 1.  I
could apply that fix, but instead I simply work around the bug:

diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index 14b808f909..201462688f 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -165,7 +167,8 @@
 #
 # Returns: @Object
 #
-# Errors: some
+# Errors:
+#     some
 #
 # TODO: frobnicate
 #

The rST heading seems to work fine then.

The next-level heading

diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index 14b808f9090..fac13425b72 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -16,7 +18,8 @@
 ##
 
 ##
-# == Subsection
+# Subsection
+# ==========
 #
 # *with emphasis*
 # @var {in braces}

doesn't:

    Extension error:
    /work/armbru/qemu/docs/../tests/qapi-schema/doc-good.json:22:1: '=' heading must come first in a comment block

Removed in this patch.

So you're right, "kind of both": they work just fine with the new doc
generator, except for the "====" rST headings, because the parser still
recognizes them as old-style schema doc headings.

> I guess you're technically right, I just never thought of it in that way.
>
> I'll update the message so I don't confuse you in the future.

Yes, please.

[...]

Re: [PATCH v3 4/5] docs/sphinx: remove special parsing for freeform sections

[Markus Armbruster]

Markus Armbruster <armbru@redhat.com> writes:

> John Snow <jsnow@redhat.com> writes:
>
>> On Fri, Jun 27, 2025, 5:52 AM Markus Armbruster <armbru@redhat.com> wrote:
>>
>>> John Snow <jsnow@redhat.com> writes:
>>>
>>> > This change removes special parsing for freeform sections and allows
>>> > them to simply be unmodified rST syntax. The existing headings in the
>>> > QAPI schema are adjusted to reflect the new paradigm.
>>>
>>> "Allows them to" suggests the patch enables use of rST headings.  Is
>>> this the case?  Or do they just work, and this patch just switches
>>> schema code to use them, and drops now unnecessary generator code?
>>>
>>
>> Ehm... Kind of both? I guess I hadn't considered that rST headings might
>> already work without the switch. I guess they didn't because of the
>> headerless freeform section bug I fix in this series.

[...]

> So you're right, "kind of both": they work just fine with the new doc
> generator, except for the "====" rST headings, because the parser still
> recognizes them as old-style schema doc headings.
>
>> I guess you're technically right, I just never thought of it in that way.
>>
>> I'll update the message so I don't confuse you in the future.
>
> Yes, please.
>
> [...]

I'm going with

    docs/sphinx: remove special parsing for freeform sections

    Remove the QAPI doc section heading syntax, use plain rST section
    headings instead.

    Tests and documentation are updated to match.

    Interestingly, Plain rST headings work fine before this patch, except
    for over- and underlining with '=', which the doc parser rejected as
    invalid QAPI doc section heading in free-form comments.

Re: [PATCH v3 1/5] docs/sphinx: adjust qapidoc to cope with same-line error sections

[Markus Armbruster]

Markus Armbruster <armbru@redhat.com> writes:

> John Snow <jsnow@redhat.com> writes:
>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>
> Let's mention the reproducer: "# Errors: some" in doc-good.json with
> :transmogrify:.

I'm adding

    Without this, the line the new QAPI doc generator chokes on
    
        # Errors: some
    
    in doc-good.json.  We still use the old doc generator for the tests,
    but we're about to correct that.

and

    Fixes: e9fbf1a0c6c2 (docs/qapidoc: add visit_errors() method)

Re: [PATCH v3 0/5] docs: remove legacy qapidoc

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> Remove docs/sphinx/qapidoc_legacy.py, and remove special parsing of
> freeform QAPI documentation block sections in favor of using standard
> rST syntax that is included in the final document with no special
> parsing or post-processing.

PATCH 1-3
Acked-by: Markus Armbruster <armbru@redhat.com>

PATCH 4+5
Reviewed-by: Markus Armbruster <armbru@redhat.com>

Queued, thanks!