[PATCH v2 0/3] docs: remove legacy qapidoc

[PATCH v2 0/3] docs: remove legacy qapidoc

[John Snow]

Remove docs/sphinx/qapidoc_legacy.py, and remove special parsing of
freeform QAPI documentation block sections in favor of using standard
rST syntax that is included in the final document with no special
parsing or post-processing.

v2:
 - rebased on origin/master (2025-06-12)
 - Revised commit messages with increased detail

Markus: this one should be pretty short and sweet with good
payoff. Makes a good candidate for your first post-vacation review ;)

John Snow (3):
  docs: fix errors formatting in tests/qapi-schema/doc-good
  docs: remove legacy QAPI manual generator
  docs: remove special parsing for freeform sections

 docs/devel/qapi-code-gen.rst                 |  28 +-
 docs/interop/firmware.json                   |   4 +-
 docs/interop/qemu-ga-ref.rst                 |   1 -
 docs/interop/qemu-qmp-ref.rst                |   1 -
 docs/interop/qemu-storage-daemon-qmp-ref.rst |   1 -
 docs/interop/vhost-user.json                 |   4 +-
 docs/sphinx/qapidoc.py                       |  62 +--
 docs/sphinx/qapidoc_legacy.py                | 440 -------------------
 qapi/acpi.json                               |   4 +-
 qapi/audio.json                              |   4 +-
 qapi/authz.json                              |   4 +-
 qapi/block-core.json                         |   3 +-
 qapi/block-export.json                       |   3 +-
 qapi/block.json                              |   7 +-
 qapi/char.json                               |   4 +-
 qapi/common.json                             |   4 +-
 qapi/compat.json                             |   4 +-
 qapi/control.json                            |   4 +-
 qapi/crypto.json                             |   4 +-
 qapi/cryptodev.json                          |   4 +-
 qapi/cxl.json                                |   4 +-
 qapi/dump.json                               |   4 +-
 qapi/ebpf.json                               |   4 +-
 qapi/error.json                              |   4 +-
 qapi/introspect.json                         |   4 +-
 qapi/job.json                                |   4 +-
 qapi/machine-common.json                     |   4 +-
 qapi/machine.json                            |   4 +-
 qapi/migration.json                          |   4 +-
 qapi/misc.json                               |   4 +-
 qapi/net.json                                |   4 +-
 qapi/pci.json                                |   4 +-
 qapi/qapi-schema.json                        |   4 +-
 qapi/qdev.json                               |   4 +-
 qapi/qom.json                                |   4 +-
 qapi/replay.json                             |   4 +-
 qapi/rocker.json                             |   4 +-
 qapi/run-state.json                          |   4 +-
 qapi/sockets.json                            |   4 +-
 qapi/stats.json                              |   4 +-
 qapi/tpm.json                                |   4 +-
 qapi/trace.json                              |   4 +-
 qapi/transaction.json                        |   4 +-
 qapi/uefi.json                               |   4 +-
 qapi/ui.json                                 |  14 +-
 qapi/vfio.json                               |   4 +-
 qapi/virtio.json                             |   4 +-
 qapi/yank.json                               |   4 +-
 python/tests/qapi-isort.sh                   |   2 +-
 scripts/qapi/parser.py                       |   7 -
 storage-daemon/qapi/qapi-schema.json         |   8 +-
 tests/qapi-schema/doc-good.json              |  13 +-
 52 files changed, 170 insertions(+), 572 deletions(-)
 delete mode 100644 docs/sphinx/qapidoc_legacy.py

-- 
2.48.1

[PATCH v2 1/3] docs: fix errors formatting in tests/qapi-schema/doc-good

[John Snow]

If we remove the legacy parser, the doc-good.json formatting begins to
fail because the body text is appended directly after the field list
entry, which is invalid rST syntax.

Without this change, we see this error:

/home/jsnow/src/qemu/docs/../tests/qapi-schema/doc-good.json:169:
WARNING: Field list ends without a blank line; unexpected
unindent. [docutils]

And this intermediate rST source:

tests/qapi-schema/doc-good.json:0167 |    :error:
tests/qapi-schema/doc-good.json:0168 |    some

With this patch applied, we instead generate this source:

tests/qapi-schema/doc-good.json:0167 |    :error:
tests/qapi-schema/doc-good.json:0168 |        - some

which compiles successfully.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 tests/qapi-schema/doc-good.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index 14b808f9090..6dcde8fd7e8 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -165,7 +165,8 @@
 #
 # Returns: @Object
 #
-# Errors: some
+# Errors:
+#     - some
 #
 # TODO: frobnicate
 #
-- 
2.48.1

Re: [PATCH v2 1/3] docs: fix errors formatting in tests/qapi-schema/doc-good

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> If we remove the legacy parser, the doc-good.json formatting begins to

"parser"?  You mean docs/sphinx/qapidoc_legacy.py, don't you?

> fail because the body text is appended directly after the field list
> entry, which is invalid rST syntax.

We've been running the test suite with the legacy doc generator.
Unwise; we should've switched to the new one right away.

> Without this change, we see this error:
>
> /home/jsnow/src/qemu/docs/../tests/qapi-schema/doc-good.json:169:
> WARNING: Field list ends without a blank line; unexpected
> unindent. [docutils]

The reporting is less than helpful.

> And this intermediate rST source:
>
> tests/qapi-schema/doc-good.json:0167 |    :error:
> tests/qapi-schema/doc-good.json:0168 |    some
>
> With this patch applied, we instead generate this source:
>
> tests/qapi-schema/doc-good.json:0167 |    :error:
> tests/qapi-schema/doc-good.json:0168 |        - some
>
> which compiles successfully.

Hmm.

As far as I can tell, the problem is lack of indentation[*].

By convention, the contents of an Errors: section is a list.
docs/devel/qapi-code-gen.rst:

    "Errors" sections should be formatted as an rST list, each entry
    detailing a relevant error condition.  For example::

     # Errors:
     #     - If @device does not exist, DeviceNotFound
     #     - Any other error returns a GenericError.

This test case is the only instance of something else.

It's just a convention, though.

Your change to the positive test case makes some sense all the same; it
should cover how we want the thing to be used.

What I don't like is how the new doc generator fails when we fail to
adhere to the convention.

Here's docs/devel/qapi-code-gen.rst on tagged sections:

    A tagged section begins with a paragraph that starts with one of the
    following words: "Since:", "Returns:", "Errors:", "TODO:".  It ends with
    the start of a new section.

    The second and subsequent lines of tagged sections must be indented
    like this::

     # TODO: Ut enim ad minim veniam, quis nostrud exercitation ullamco
     #     laboris nisi ut aliquip ex ea commodo consequat.
     #
     #     Duis aute irure dolor in reprehenderit in voluptate velit esse
     #     cillum dolore eu fugiat nulla pariatur.

This tells us that

    # Errors: some

and

    # Errors:
    #     some

and

    # Errors: some
    #     more

should all work, just like for any other tag.  However, only the second
one works in my testing.  With qapidoc_legacy.py, all three work.

We can make Errors: unlike the other tags.  But it needs to be done
properly, i.e. in scripts/qapi/parser.py (for decent error reporting),
and documented in docs/devel/qapi-code-gen.rst.

Keeping the QAPI domain accept what the generator generates might be
easier.

Thoughts?

>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  tests/qapi-schema/doc-good.json | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
> index 14b808f9090..6dcde8fd7e8 100644
> --- a/tests/qapi-schema/doc-good.json
> +++ b/tests/qapi-schema/doc-good.json
> @@ -165,7 +165,8 @@
>  #
>  # Returns: @Object
>  #
> -# Errors: some
> +# Errors:
> +#     - some
>  #
>  # TODO: frobnicate
>  #

Fails "make check".  Fixup appended.



[*] Evidence:

    # Errors:
    #     - some

which expands into

    :error:
        - some

and

    # Errors:
    #     some

which expands into

    :error:
        some

both work.

docs/devel/qapi-domain.rst:

    ``:error:``
    -----------

    Document the error condition(s) of a QAPI command.

    :availability: This field list is only available in the body of the
                   Command directive.
--> :syntax: ``:error: Lorem ipsum dolor sit amet ...``
    :type: `sphinx.util.docfields.Field
           <https://pydoc.dev/sphinx/latest/sphinx.util.docfields.Field.html?private=1>`_

    The format of the :errors: field list description is free-form rST. The
    alternative spelling ":errors:" is also permitted, but strictly
    analogous.

    Example::

       .. qapi:command:: block-job-set-speed
          :since: 1.1

          Set maximum speed for a background block operation.

          This command can only be issued when there is an active block job.

          Throttling can be disabled by setting the speed to 0.

          :arg string device: The job identifier.  This used to be a device
              name (hence the name of the parameter), but since QEMU 2.7 it
              can have other values.
          :arg int speed: the maximum speed, in bytes per second, or 0 for
              unlimited.  Defaults to 0.
-->       :error:
-->           - If no background operation is active on this device,
-->             DeviceNotActive

This makes me expect

    :error: some

also works.  However, the obvious

    # Errors: some

produces

    :error:
    some

which doesn't work.


diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index dc8352eed4..3711cf5480 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -176,7 +176,7 @@ another feature
     section=Returns
 @Object
     section=Errors
-some
+    - some
     section=Todo
 frobnicate
     section=Plain
diff --git a/tests/qapi-schema/doc-good.txt b/tests/qapi-schema/doc-good.txt
index 17a1d56ef1..e54cc95f4a 100644
--- a/tests/qapi-schema/doc-good.txt
+++ b/tests/qapi-schema/doc-good.txt
@@ -207,7 +207,7 @@ Returns
 Errors
 ~~~~~~
 
-some
+* some
 
 Notes:
 

Re: [PATCH v2 1/3] docs: fix errors formatting in tests/qapi-schema/doc-good

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 7703 bytes --]

On Mon, Jun 16, 2025 at 7:36 AM Markus Armbruster <armbru@redhat.com> wrote:

> John Snow <jsnow@redhat.com> writes:
>
> > If we remove the legacy parser, the doc-good.json formatting begins to
>
> "parser"?  You mean docs/sphinx/qapidoc_legacy.py, don't you?
>

Mmm... yes, I'm conflating the purpose of the series (removing the legacy
freeform doc parser) with what necessitated this change (switching to the
new doc *generator*)

Wiggly-brained, wiggly-mouthed.


>
> > fail because the body text is appended directly after the field list
> > entry, which is invalid rST syntax.
>
> We've been running the test suite with the legacy doc generator.
> Unwise; we should've switched to the new one right away.
>

Oops O:-)


>
> > Without this change, we see this error:
> >
> > /home/jsnow/src/qemu/docs/../tests/qapi-schema/doc-good.json:169:
> > WARNING: Field list ends without a blank line; unexpected
> > unindent. [docutils]
>
> The reporting is less than helpful.
>
> > And this intermediate rST source:
> >
> > tests/qapi-schema/doc-good.json:0167 |    :error:
> > tests/qapi-schema/doc-good.json:0168 |    some
> >
> > With this patch applied, we instead generate this source:
> >
> > tests/qapi-schema/doc-good.json:0167 |    :error:
> > tests/qapi-schema/doc-good.json:0168 |        - some
> >
> > which compiles successfully.
>
> Hmm.
>
> As far as I can tell, the problem is lack of indentation[*].
>
> By convention, the contents of an Errors: section is a list.
> docs/devel/qapi-code-gen.rst:
>
>     "Errors" sections should be formatted as an rST list, each entry
>     detailing a relevant error condition.  For example::
>
>      # Errors:
>      #     - If @device does not exist, DeviceNotFound
>      #     - Any other error returns a GenericError.
>
> This test case is the only instance of something else.
>
> It's just a convention, though.
>
> Your change to the positive test case makes some sense all the same; it
> should cover how we want the thing to be used.
>
> What I don't like is how the new doc generator fails when we fail to
> adhere to the convention.
>

I agree... it should probably generate ":error: some" instead, without the
newline. Bad, but valid.


>
> Here's docs/devel/qapi-code-gen.rst on tagged sections:
>
>     A tagged section begins with a paragraph that starts with one of the
>     following words: "Since:", "Returns:", "Errors:", "TODO:".  It ends
> with
>     the start of a new section.
>
>     The second and subsequent lines of tagged sections must be indented
>     like this::
>
>      # TODO: Ut enim ad minim veniam, quis nostrud exercitation ullamco
>      #     laboris nisi ut aliquip ex ea commodo consequat.
>      #
>      #     Duis aute irure dolor in reprehenderit in voluptate velit esse
>      #     cillum dolore eu fugiat nulla pariatur.
>
> This tells us that
>
>     # Errors: some
>
> and
>
>     # Errors:
>     #     some
>
> and
>
>     # Errors: some
>     #     more
>
> should all work, just like for any other tag.  However, only the second
> one works in my testing.  With qapidoc_legacy.py, all three work.
>
> We can make Errors: unlike the other tags.  But it needs to be done
> properly, i.e. in scripts/qapi/parser.py (for decent error reporting),
> and documented in docs/devel/qapi-code-gen.rst.
>
> Keeping the QAPI domain accept what the generator generates might be
> easier.
>
> Thoughts?
>

I try not to have any as often as I can.

In seriousness, I need a good few minutes with the generator to understand
why this behaves strangely. Might need to switch to a different parser
routine in parser.py, or I might need to adjust the qapidoc generator. Not
entirely sure what's precisely wrong... If I can find something that's both
easy and clean I'll do that instead.

Probably a newline needs to be preserved in the source somewhere, and then
a newline needs to be *not* added to the generator. Something like that.

We can discuss turning our de-facto standard into an actual syntactical
requirement later; I think there might be some benefit to allowing multiple
errors sections that each become their own ":error:" info field list entry,
but I am not confident on that and am not ready to dive into it just yet. I
think it's definitely easier on the sphinx side but it might not be easier
on the QAPI side. We'll see...


>
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> >  tests/qapi-schema/doc-good.json | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/tests/qapi-schema/doc-good.json
> b/tests/qapi-schema/doc-good.json
> > index 14b808f9090..6dcde8fd7e8 100644
> > --- a/tests/qapi-schema/doc-good.json
> > +++ b/tests/qapi-schema/doc-good.json
> > @@ -165,7 +165,8 @@
> >  #
> >  # Returns: @Object
> >  #
> > -# Errors: some
> > +# Errors:
> > +#     - some
> >  #
> >  # TODO: frobnicate
> >  #
>
> Fails "make check".  Fixup appended.
>

Strange. How'd I find this issue and fix it if I wasn't running the tests?
Uhm, sorry. Sloppy of me.


>
>
>
> [*] Evidence:
>
>     # Errors:
>     #     - some
>
> which expands into
>
>     :error:
>         - some
>
> and
>
>     # Errors:
>     #     some
>
> which expands into
>
>     :error:
>         some
>
> both work.
>
> docs/devel/qapi-domain.rst:
>
>     ``:error:``
>     -----------
>
>     Document the error condition(s) of a QAPI command.
>
>     :availability: This field list is only available in the body of the
>                    Command directive.
> --> :syntax: ``:error: Lorem ipsum dolor sit amet ...``
>     :type: `sphinx.util.docfields.Field
>            <
> https://pydoc.dev/sphinx/latest/sphinx.util.docfields.Field.html?private=1
> >`_
>
>     The format of the :errors: field list description is free-form rST. The
>     alternative spelling ":errors:" is also permitted, but strictly
>     analogous.
>
>     Example::
>
>        .. qapi:command:: block-job-set-speed
>           :since: 1.1
>
>           Set maximum speed for a background block operation.
>
>           This command can only be issued when there is an active block
> job.
>
>           Throttling can be disabled by setting the speed to 0.
>
>           :arg string device: The job identifier.  This used to be a device
>               name (hence the name of the parameter), but since QEMU 2.7 it
>               can have other values.
>           :arg int speed: the maximum speed, in bytes per second, or 0 for
>               unlimited.  Defaults to 0.
> -->       :error:
> -->           - If no background operation is active on this device,
> -->             DeviceNotActive
>
> This makes me expect
>
>     :error: some
>
> also works.  However, the obvious
>
>     # Errors: some
>
> produces
>
>     :error:
>     some
>
> which doesn't work.
>
>
> diff --git a/tests/qapi-schema/doc-good.out
> b/tests/qapi-schema/doc-good.out
> index dc8352eed4..3711cf5480 100644
> --- a/tests/qapi-schema/doc-good.out
> +++ b/tests/qapi-schema/doc-good.out
> @@ -176,7 +176,7 @@ another feature
>      section=Returns
>  @Object
>      section=Errors
> -some
> +    - some
>      section=Todo
>  frobnicate
>      section=Plain
> diff --git a/tests/qapi-schema/doc-good.txt
> b/tests/qapi-schema/doc-good.txt
> index 17a1d56ef1..e54cc95f4a 100644
> --- a/tests/qapi-schema/doc-good.txt
> +++ b/tests/qapi-schema/doc-good.txt
> @@ -207,7 +207,7 @@ Returns
>  Errors
>  ~~~~~~
>
> -some
> +* some
>
>  Notes:
>
>
>

[-- Attachment #2: Type: text/html, Size: 10200 bytes --]

[PATCH v2 2/3] docs: remove legacy QAPI manual generator

[John Snow]

Thanks for your service!

Remove the old qapidoc and the option to enable the transmogrifier,
leaving the "transmogrifier" as the ONLY qapi doc generator. This in
effect also converts the QAPI test to use the new documentation
generator, too.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/interop/qemu-ga-ref.rst                 |   1 -
 docs/interop/qemu-qmp-ref.rst                |   1 -
 docs/interop/qemu-storage-daemon-qmp-ref.rst |   1 -
 docs/sphinx/qapidoc.py                       |  25 +-
 docs/sphinx/qapidoc_legacy.py                | 440 -------------------
 python/tests/qapi-isort.sh                   |   2 +-
 6 files changed, 2 insertions(+), 468 deletions(-)
 delete mode 100644 docs/sphinx/qapidoc_legacy.py

diff --git a/docs/interop/qemu-ga-ref.rst b/docs/interop/qemu-ga-ref.rst
index 25f6e24b03c..ea6652ae43e 100644
--- a/docs/interop/qemu-ga-ref.rst
+++ b/docs/interop/qemu-ga-ref.rst
@@ -2,5 +2,4 @@ QEMU Guest Agent Protocol Reference
 ===================================
 
 .. qapi-doc:: qga/qapi-schema.json
-   :transmogrify:
    :namespace: QGA
diff --git a/docs/interop/qemu-qmp-ref.rst b/docs/interop/qemu-qmp-ref.rst
index 3bc1ca12b16..f0ce39ad67d 100644
--- a/docs/interop/qemu-qmp-ref.rst
+++ b/docs/interop/qemu-qmp-ref.rst
@@ -7,5 +7,4 @@ QEMU QMP Reference Manual
    :local:
 
 .. qapi-doc:: qapi/qapi-schema.json
-   :transmogrify:
    :namespace: QMP
diff --git a/docs/interop/qemu-storage-daemon-qmp-ref.rst b/docs/interop/qemu-storage-daemon-qmp-ref.rst
index dc7bde262ae..4dbb6a2cc83 100644
--- a/docs/interop/qemu-storage-daemon-qmp-ref.rst
+++ b/docs/interop/qemu-storage-daemon-qmp-ref.rst
@@ -5,5 +5,4 @@ QEMU Storage Daemon QMP Reference Manual
    :local:
 
 .. qapi-doc:: storage-daemon/qapi/qapi-schema.json
-   :transmogrify:
    :namespace: QSD
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 8011ac9efaf..c9c477378f5 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -64,8 +64,6 @@
 from sphinx.util.docutils import SphinxDirective, switch_source_input
 from sphinx.util.nodes import nested_parse_with_titles
 
-from qapidoc_legacy import QAPISchemaGenRSTVisitor  # type: ignore
-
 
 if TYPE_CHECKING:
     from typing import (
@@ -504,15 +502,9 @@ class QAPIDocDirective(NestedDirective):
     option_spec = {
         "qapifile": directives.unchanged_required,
         "namespace": directives.unchanged,
-        "transmogrify": directives.flag,
     }
     has_content = False
 
-    def new_serialno(self) -> str:
-        """Return a unique new ID string suitable for use as a node's ID"""
-        env = self.state.document.settings.env
-        return "qapidoc-%d" % env.new_serialno("qapidoc")
-
     def transmogrify(self, schema: QAPISchema) -> nodes.Element:
         logger.info("Transmogrifying QAPI to rST ...")
         vis = Transmogrifier()
@@ -590,21 +582,10 @@ def write_intermediate(self, content: StringList, filename: str) -> None:
                     outfile.write(f" {rcol}")
                 outfile.write("\n")
 
-    def legacy(self, schema: QAPISchema) -> nodes.Element:
-        vis = QAPISchemaGenRSTVisitor(self)
-        vis.visit_begin(schema)
-        for doc in schema.docs:
-            if doc.symbol:
-                vis.symbol(doc, schema.lookup_entity(doc.symbol))
-            else:
-                vis.freeform(doc)
-        return vis.get_document_node()  # type: ignore
-
     def run(self) -> Sequence[nodes.Node]:
         env = self.state.document.settings.env
         qapifile = env.config.qapidoc_srctree + "/" + self.arguments[0]
         qapidir = os.path.dirname(qapifile)
-        transmogrify = "transmogrify" in self.options
 
         try:
             schema = QAPISchema(qapifile)
@@ -617,11 +598,7 @@ def run(self) -> Sequence[nodes.Node]:
             # so they are displayed nicely to the user
             raise ExtensionError(str(err)) from err
 
-        if transmogrify:
-            contentnode = self.transmogrify(schema)
-        else:
-            contentnode = self.legacy(schema)
-
+        contentnode = self.transmogrify(schema)
         return contentnode.children
 
 
diff --git a/docs/sphinx/qapidoc_legacy.py b/docs/sphinx/qapidoc_legacy.py
deleted file mode 100644
index 13520f4c26b..00000000000
--- a/docs/sphinx/qapidoc_legacy.py
+++ /dev/null
@@ -1,440 +0,0 @@
-# coding=utf-8
-# type: ignore
-#
-# QEMU qapidoc QAPI file parsing extension
-#
-# Copyright (c) 2020 Linaro
-#
-# This work is licensed under the terms of the GNU GPLv2 or later.
-# See the COPYING file in the top-level directory.
-
-"""
-qapidoc is a Sphinx extension that implements the qapi-doc directive
-
-The purpose of this extension is to read the documentation comments
-in QAPI schema files, and insert them all into the current document.
-
-It implements one new rST directive, "qapi-doc::".
-Each qapi-doc:: directive takes one argument, which is the
-pathname of the schema file to process, relative to the source tree.
-
-The docs/conf.py file must set the qapidoc_srctree config value to
-the root of the QEMU source tree.
-
-The Sphinx documentation on writing extensions is at:
-https://www.sphinx-doc.org/en/master/development/index.html
-"""
-
-import re
-import textwrap
-
-from docutils import nodes
-from docutils.statemachine import ViewList
-from qapi.error import QAPISemError
-from qapi.gen import QAPISchemaVisitor
-from qapi.parser import QAPIDoc
-
-
-def dedent(text: str) -> str:
-    # Adjust indentation to make description text parse as paragraph.
-
-    lines = text.splitlines(True)
-    if re.match(r"\s+", lines[0]):
-        # First line is indented; description started on the line after
-        # the name. dedent the whole block.
-        return textwrap.dedent(text)
-
-    # Descr started on same line. Dedent line 2+.
-    return lines[0] + textwrap.dedent("".join(lines[1:]))
-
-
-class QAPISchemaGenRSTVisitor(QAPISchemaVisitor):
-    """A QAPI schema visitor which generates docutils/Sphinx nodes
-
-    This class builds up a tree of docutils/Sphinx nodes corresponding
-    to documentation for the various QAPI objects. To use it, first
-    create a QAPISchemaGenRSTVisitor object, and call its
-    visit_begin() method.  Then you can call one of the two methods
-    'freeform' (to add documentation for a freeform documentation
-    chunk) or 'symbol' (to add documentation for a QAPI symbol). These
-    will cause the visitor to build up the tree of document
-    nodes. Once you've added all the documentation via 'freeform' and
-    'symbol' method calls, you can call 'get_document_nodes' to get
-    the final list of document nodes (in a form suitable for returning
-    from a Sphinx directive's 'run' method).
-    """
-    def __init__(self, sphinx_directive):
-        self._cur_doc = None
-        self._sphinx_directive = sphinx_directive
-        self._top_node = nodes.section()
-        self._active_headings = [self._top_node]
-
-    def _make_dlitem(self, term, defn):
-        """Return a dlitem node with the specified term and definition.
-
-        term should be a list of Text and literal nodes.
-        defn should be one of:
-        - a string, which will be handed to _parse_text_into_node
-        - a list of Text and literal nodes, which will be put into
-          a paragraph node
-        """
-        dlitem = nodes.definition_list_item()
-        dlterm = nodes.term('', '', *term)
-        dlitem += dlterm
-        if defn:
-            dldef = nodes.definition()
-            if isinstance(defn, list):
-                dldef += nodes.paragraph('', '', *defn)
-            else:
-                self._parse_text_into_node(defn, dldef)
-            dlitem += dldef
-        return dlitem
-
-    def _make_section(self, title):
-        """Return a section node with optional title"""
-        section = nodes.section(ids=[self._sphinx_directive.new_serialno()])
-        if title:
-            section += nodes.title(title, title)
-        return section
-
-    def _nodes_for_ifcond(self, ifcond, with_if=True):
-        """Return list of Text, literal nodes for the ifcond
-
-        Return a list which gives text like ' (If: condition)'.
-        If with_if is False, we don't return the "(If: " and ")".
-        """
-
-        doc = ifcond.docgen()
-        if not doc:
-            return []
-        doc = nodes.literal('', doc)
-        if not with_if:
-            return [doc]
-
-        nodelist = [nodes.Text(' ('), nodes.strong('', 'If: ')]
-        nodelist.append(doc)
-        nodelist.append(nodes.Text(')'))
-        return nodelist
-
-    def _nodes_for_one_member(self, member):
-        """Return list of Text, literal nodes for this member
-
-        Return a list of doctree nodes which give text like
-        'name: type (optional) (If: ...)' suitable for use as the
-        'term' part of a definition list item.
-        """
-        term = [nodes.literal('', member.name)]
-        if member.type.doc_type():
-            term.append(nodes.Text(': '))
-            term.append(nodes.literal('', member.type.doc_type()))
-        if member.optional:
-            term.append(nodes.Text(' (optional)'))
-        if member.ifcond.is_present():
-            term.extend(self._nodes_for_ifcond(member.ifcond))
-        return term
-
-    def _nodes_for_variant_when(self, branches, variant):
-        """Return list of Text, literal nodes for variant 'when' clause
-
-        Return a list of doctree nodes which give text like
-        'when tagname is variant (If: ...)' suitable for use in
-        the 'branches' part of a definition list.
-        """
-        term = [nodes.Text(' when '),
-                nodes.literal('', branches.tag_member.name),
-                nodes.Text(' is '),
-                nodes.literal('', '"%s"' % variant.name)]
-        if variant.ifcond.is_present():
-            term.extend(self._nodes_for_ifcond(variant.ifcond))
-        return term
-
-    def _nodes_for_members(self, doc, what, base=None, branches=None):
-        """Return list of doctree nodes for the table of members"""
-        dlnode = nodes.definition_list()
-        for section in doc.args.values():
-            term = self._nodes_for_one_member(section.member)
-            # TODO drop fallbacks when undocumented members are outlawed
-            if section.text:
-                defn = dedent(section.text)
-            else:
-                defn = [nodes.Text('Not documented')]
-
-            dlnode += self._make_dlitem(term, defn)
-
-        if base:
-            dlnode += self._make_dlitem([nodes.Text('The members of '),
-                                         nodes.literal('', base.doc_type())],
-                                        None)
-
-        if branches:
-            for v in branches.variants:
-                if v.type.name == 'q_empty':
-                    continue
-                assert not v.type.is_implicit()
-                term = [nodes.Text('The members of '),
-                        nodes.literal('', v.type.doc_type())]
-                term.extend(self._nodes_for_variant_when(branches, v))
-                dlnode += self._make_dlitem(term, None)
-
-        if not dlnode.children:
-            return []
-
-        section = self._make_section(what)
-        section += dlnode
-        return [section]
-
-    def _nodes_for_enum_values(self, doc):
-        """Return list of doctree nodes for the table of enum values"""
-        seen_item = False
-        dlnode = nodes.definition_list()
-        for section in doc.args.values():
-            termtext = [nodes.literal('', section.member.name)]
-            if section.member.ifcond.is_present():
-                termtext.extend(self._nodes_for_ifcond(section.member.ifcond))
-            # TODO drop fallbacks when undocumented members are outlawed
-            if section.text:
-                defn = dedent(section.text)
-            else:
-                defn = [nodes.Text('Not documented')]
-
-            dlnode += self._make_dlitem(termtext, defn)
-            seen_item = True
-
-        if not seen_item:
-            return []
-
-        section = self._make_section('Values')
-        section += dlnode
-        return [section]
-
-    def _nodes_for_arguments(self, doc, arg_type):
-        """Return list of doctree nodes for the arguments section"""
-        if arg_type and not arg_type.is_implicit():
-            assert not doc.args
-            section = self._make_section('Arguments')
-            dlnode = nodes.definition_list()
-            dlnode += self._make_dlitem(
-                [nodes.Text('The members of '),
-                 nodes.literal('', arg_type.name)],
-                None)
-            section += dlnode
-            return [section]
-
-        return self._nodes_for_members(doc, 'Arguments')
-
-    def _nodes_for_features(self, doc):
-        """Return list of doctree nodes for the table of features"""
-        seen_item = False
-        dlnode = nodes.definition_list()
-        for section in doc.features.values():
-            dlnode += self._make_dlitem(
-                [nodes.literal('', section.member.name)], dedent(section.text))
-            seen_item = True
-
-        if not seen_item:
-            return []
-
-        section = self._make_section('Features')
-        section += dlnode
-        return [section]
-
-    def _nodes_for_sections(self, doc):
-        """Return list of doctree nodes for additional sections"""
-        nodelist = []
-        for section in doc.sections:
-            if section.kind == QAPIDoc.Kind.TODO:
-                # Hide TODO: sections
-                continue
-
-            if section.kind == QAPIDoc.Kind.PLAIN:
-                # Sphinx cannot handle sectionless titles;
-                # Instead, just append the results to the prior section.
-                container = nodes.container()
-                self._parse_text_into_node(section.text, container)
-                nodelist += container.children
-                continue
-
-            snode = self._make_section(section.kind.name.title())
-            self._parse_text_into_node(dedent(section.text), snode)
-            nodelist.append(snode)
-        return nodelist
-
-    def _nodes_for_if_section(self, ifcond):
-        """Return list of doctree nodes for the "If" section"""
-        nodelist = []
-        if ifcond.is_present():
-            snode = self._make_section('If')
-            snode += nodes.paragraph(
-                '', '', *self._nodes_for_ifcond(ifcond, with_if=False)
-            )
-            nodelist.append(snode)
-        return nodelist
-
-    def _add_doc(self, typ, sections):
-        """Add documentation for a command/object/enum...
-
-        We assume we're documenting the thing defined in self._cur_doc.
-        typ is the type of thing being added ("Command", "Object", etc)
-
-        sections is a list of nodes for sections to add to the definition.
-        """
-
-        doc = self._cur_doc
-        snode = nodes.section(ids=[self._sphinx_directive.new_serialno()])
-        snode += nodes.title('', '', *[nodes.literal(doc.symbol, doc.symbol),
-                                       nodes.Text(' (' + typ + ')')])
-        self._parse_text_into_node(doc.body.text, snode)
-        for s in sections:
-            if s is not None:
-                snode += s
-        self._add_node_to_current_heading(snode)
-
-    def visit_enum_type(self, name, info, ifcond, features, members, prefix):
-        doc = self._cur_doc
-        self._add_doc('Enum',
-                      self._nodes_for_enum_values(doc)
-                      + self._nodes_for_features(doc)
-                      + self._nodes_for_sections(doc)
-                      + self._nodes_for_if_section(ifcond))
-
-    def visit_object_type(self, name, info, ifcond, features,
-                          base, members, branches):
-        doc = self._cur_doc
-        if base and base.is_implicit():
-            base = None
-        self._add_doc('Object',
-                      self._nodes_for_members(doc, 'Members', base, branches)
-                      + self._nodes_for_features(doc)
-                      + self._nodes_for_sections(doc)
-                      + self._nodes_for_if_section(ifcond))
-
-    def visit_alternate_type(self, name, info, ifcond, features,
-                             alternatives):
-        doc = self._cur_doc
-        self._add_doc('Alternate',
-                      self._nodes_for_members(doc, 'Members')
-                      + self._nodes_for_features(doc)
-                      + self._nodes_for_sections(doc)
-                      + self._nodes_for_if_section(ifcond))
-
-    def visit_command(self, name, info, ifcond, features, arg_type,
-                      ret_type, gen, success_response, boxed, allow_oob,
-                      allow_preconfig, coroutine):
-        doc = self._cur_doc
-        self._add_doc('Command',
-                      self._nodes_for_arguments(doc, arg_type)
-                      + self._nodes_for_features(doc)
-                      + self._nodes_for_sections(doc)
-                      + self._nodes_for_if_section(ifcond))
-
-    def visit_event(self, name, info, ifcond, features, arg_type, boxed):
-        doc = self._cur_doc
-        self._add_doc('Event',
-                      self._nodes_for_arguments(doc, arg_type)
-                      + self._nodes_for_features(doc)
-                      + self._nodes_for_sections(doc)
-                      + self._nodes_for_if_section(ifcond))
-
-    def symbol(self, doc, entity):
-        """Add documentation for one symbol to the document tree
-
-        This is the main entry point which causes us to add documentation
-        nodes for a symbol (which could be a 'command', 'object', 'event',
-        etc). We do this by calling 'visit' on the schema entity, which
-        will then call back into one of our visit_* methods, depending
-        on what kind of thing this symbol is.
-        """
-        self._cur_doc = doc
-        entity.visit(self)
-        self._cur_doc = None
-
-    def _start_new_heading(self, heading, level):
-        """Start a new heading at the specified heading level
-
-        Create a new section whose title is 'heading' and which is placed
-        in the docutils node tree as a child of the most recent level-1
-        heading. Subsequent document sections (commands, freeform doc chunks,
-        etc) will be placed as children of this new heading section.
-        """
-        if len(self._active_headings) < level:
-            raise QAPISemError(self._cur_doc.info,
-                               'Level %d subheading found outside a '
-                               'level %d heading'
-                               % (level, level - 1))
-        snode = self._make_section(heading)
-        self._active_headings[level - 1] += snode
-        self._active_headings = self._active_headings[:level]
-        self._active_headings.append(snode)
-        return snode
-
-    def _add_node_to_current_heading(self, node):
-        """Add the node to whatever the current active heading is"""
-        self._active_headings[-1] += node
-
-    def freeform(self, doc):
-        """Add a piece of 'freeform' documentation to the document tree
-
-        A 'freeform' document chunk doesn't relate to any particular
-        symbol (for instance, it could be an introduction).
-
-        If the freeform document starts with a line of the form
-        '= Heading text', this is a section or subsection heading, with
-        the heading level indicated by the number of '=' signs.
-        """
-
-        # QAPIDoc documentation says free-form documentation blocks
-        # must have only a body section, nothing else.
-        assert not doc.sections
-        assert not doc.args
-        assert not doc.features
-        self._cur_doc = doc
-
-        text = doc.body.text
-        if re.match(r'=+ ', text):
-            # Section/subsection heading (if present, will always be
-            # the first line of the block)
-            (heading, _, text) = text.partition('\n')
-            (leader, _, heading) = heading.partition(' ')
-            node = self._start_new_heading(heading, len(leader))
-            if text == '':
-                return
-        else:
-            node = nodes.container()
-
-        self._parse_text_into_node(text, node)
-        self._cur_doc = None
-
-    def _parse_text_into_node(self, doctext, node):
-        """Parse a chunk of QAPI-doc-format text into the node
-
-        The doc comment can contain most inline rST markup, including
-        bulleted and enumerated lists.
-        As an extra permitted piece of markup, @var will be turned
-        into ``var``.
-        """
-
-        # Handle the "@var means ``var`` case
-        doctext = re.sub(r'@([\w-]+)', r'``\1``', doctext)
-
-        rstlist = ViewList()
-        for line in doctext.splitlines():
-            # The reported line number will always be that of the start line
-            # of the doc comment, rather than the actual location of the error.
-            # Being more precise would require overhaul of the QAPIDoc class
-            # to track lines more exactly within all the sub-parts of the doc
-            # comment, as well as counting lines here.
-            rstlist.append(line, self._cur_doc.info.fname,
-                           self._cur_doc.info.line)
-        # Append a blank line -- in some cases rST syntax errors get
-        # attributed to the line after one with actual text, and if there
-        # isn't anything in the ViewList corresponding to that then Sphinx
-        # 1.6's AutodocReporter will then misidentify the source/line location
-        # in the error message (usually attributing it to the top-level
-        # .rst file rather than the offending .json file). The extra blank
-        # line won't affect the rendered output.
-        rstlist.append("", self._cur_doc.info.fname, self._cur_doc.info.line)
-        self._sphinx_directive.do_parse(rstlist, node)
-
-    def get_document_node(self):
-        """Return the root docutils node which makes up the document"""
-        return self._top_node
diff --git a/python/tests/qapi-isort.sh b/python/tests/qapi-isort.sh
index 78dd947f68c..067c16d5d94 100755
--- a/python/tests/qapi-isort.sh
+++ b/python/tests/qapi-isort.sh
@@ -3,6 +3,6 @@
 
 python3 -m isort --sp . -c ../scripts/qapi/
 # Force isort to recognize "compat" as a local module and not third-party
-python3 -m isort --sp . -c -p compat -p qapidoc_legacy \
+python3 -m isort --sp . -c -p compat \
         ../docs/sphinx/qapi_domain.py \
         ../docs/sphinx/qapidoc.py
-- 
2.48.1

Re: [PATCH v2 2/3] docs: remove legacy QAPI manual generator

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> Thanks for your service!
>
> Remove the old qapidoc and the option to enable the transmogrifier,
> leaving the "transmogrifier" as the ONLY qapi doc generator. This in
> effect also converts the QAPI test to use the new documentation
> generator, too.
>
> Signed-off-by: John Snow <jsnow@redhat.com>

Fails "make check", because tests/qapi-schema/doc-good.txt needs an
update.

Unfortunately, the diff of the update is less than useful.  To make
sense of what changes, I split doc-good.txt into parts before and after,
and diffed those.


  diff -rupw o/01 n/01
  --- o/01	2025-06-16 13:53:05.036940854 +0200
  +++ n/01	2025-06-16 13:49:07.167435996 +0200
  @@ -1,11 +1,13 @@
   Section
   *******

  +Just text, no heading.
  +

Looks like a bug fix.  Needs a mention in the commit message then.

   Subsection
   ==========

  -*with emphasis* "var" {in braces}
  +*with emphasis* @var {in braces}

Rendering of @references changes.  Okay.

   * List item one

  @@ -35,4 +37,3 @@ Example:

   -> in <- out Examples: - *verbatim* - {braces}

  -
  diff -rupw o/02 n/02
  --- o/02	2025-06-16 13:53:17.133712123 +0200
  +++ n/02	2025-06-16 13:49:21.024174424 +0200
  @@ -1,32 +1,15 @@
  -"Enum" (Enum)
  --------------
  +Enum Enum
  +    *Availability*: "IFCOND"

  +   Values:
  +      * **one** -- The _one_ {and only}, description on the same line

  -Values
  -~~~~~~
  +      * **two** -- Not documented

  -"one" (**If: **"IFONE")

Member conditional is lost.  Known issue; see commit dbf51d15fdb.

  -   The _one_ {and only}, description on the same line
  +   Features:
  +      * **enum-feat** -- Also _one_ {and only}

  -"two"
  -   Not documented
  -
  -
  -Features
  -~~~~~~~~
  -
  -"enum-feat"
  -   Also _one_ {and only}
  -
  -"enum-member-feat"
  -   a member feature
  +      * **enum-member-feat** -- a member feature

   "two" is undocumented

  -
  -If
  -~~
  -
  -"IFCOND"
  -
  -
  diff -rupw o/03 n/03
  --- o/03	2025-06-16 13:53:27.556514971 +0200
  +++ n/03	2025-06-16 13:50:44.582595942 +0200
  @@ -1,17 +1,7 @@
  -"Base" (Object)
  ----------------
  -
  -
  -Members
  -~~~~~~~
  -
  -"base1": "Enum"
  -   description starts on a new line, minimally indented
  -
  -
  -If
  -~~
  -
  -"IFALL1 and IFALL2"
  +Object Base
  +    *Availability*: "IFALL1 and IFALL2"

  +   Members:
  +      * **base1** ("Enum") -- description starts on a new line,
  +        minimally indented

  diff -rupw o/04 n/04
  --- o/04	2025-06-16 13:53:39.356291772 +0200
  +++ n/04	2025-06-16 13:50:54.486408751 +0200
  @@ -1,5 +1,4 @@
  -"Variant1" (Object)
  --------------------
  +Object Variant1

   A paragraph

  @@ -7,21 +6,11 @@ Another paragraph

   "var1" is undocumented

  +   Members:
  +      * **var1** ("string") -- Not documented

  -Members
  -~~~~~~~
  -
  -"var1": "string" (**If: **"IFSTR")

Likewise.

  -   Not documented
  -
  -
  -Features
  -~~~~~~~~
  -
  -"variant1-feat"
  -   a feature
  -
  -"member-feat"
  -   a member feature
  +   Features:
  +      * **variant1-feat** -- a feature

  +      * **member-feat** -- a member feature

  diff -rupw o/05 n/05
  --- o/05	2025-06-16 13:53:43.429214731 +0200
  +++ n/05	2025-06-16 13:51:05.247205330 +0200
  @@ -1,4 +1,2 @@
  -"Variant2" (Object)
  --------------------
  -
  +Object Variant2

  diff -rupw o/06 n/06
  --- o/06	2025-06-16 13:53:48.461119551 +0200
  +++ n/06	2025-06-16 13:51:10.990096739 +0200
  @@ -1,19 +1,12 @@
  -"Object" (Object)
  ------------------
  +Object Object

  +   Members:
  +      * The members of "Base".

  -Members
  -~~~~~~~
  +      * When "base1" is "one": The members of "Variant1".

  -The members of "Base"
  -The members of "Variant1" when "base1" is ""one""
  -The members of "Variant2" when "base1" is ""two"" (**If: **"IFONE or
  -IFTWO")

Likewise.

  -
  -Features
  -~~~~~~~~
  -
  -"union-feat1"
  -   a feature
  +      * When "base1" is "two": The members of "Variant2".

  +   Features:
  +      * **union-feat1** -- a feature

  diff -rupw o/07 n/07
  --- o/07	2025-06-16 13:53:55.988977158 +0200
  +++ n/07	2025-06-16 13:51:16.869985559 +0200
  @@ -1,28 +1,13 @@
  -"Alternate" (Alternate)
  ------------------------
  +Alternate Alternate
  +    *Availability*: "not (IFONE or IFTWO)"

  +   Alternatives:
  +      * **i** ("int") -- description starts on the same line remainder
  +        indented the same "b" is undocumented

  -Members
  -~~~~~~~
  +      * **b** ("boolean") -- Not documented

  -"i": "int"
  -   description starts on the same line remainder indented the same "b"
  -   is undocumented
  -
  -"b": "boolean"
  -   Not documented
  -
  -
  -Features
  -~~~~~~~~
  -
  -"alt-feat"
  -   a feature
  -
  -
  -If
  -~~
  -
  -"not (IFONE or IFTWO)"
  +   Features:
  +      * **alt-feat** -- a feature


  diff -rupw o/08 n/08
  --- o/08	2025-06-16 13:54:11.692680115 +0200
  +++ n/08	2025-06-16 13:51:35.957624638 +0200
  @@ -1,47 +1,29 @@
   Another subsection
   ==================

  +Command cmd (Since: 2.10)

  -"cmd" (Command)
  ----------------
  +   Arguments:
  +      * **arg1** ("int") -- description starts on a new line, indented

  +      * **arg2** ("string", *optional*) -- description starts on the
  +        same line remainder indented differently

  -Arguments
  -~~~~~~~~~
  +      * **arg3** ("boolean") -- Not documented

  -"arg1": "int"
  -   description starts on a new line, indented
  +   Features:
  +      * **cmd-feat1** -- a feature

  -"arg2": "string" (optional)
  -   description starts on the same line remainder indented differently
  -
  -"arg3": "boolean"
  -   Not documented
  -
  -
  -Features
  -~~~~~~~~
  -
  -"cmd-feat1"
  -   a feature
  -
  -"cmd-feat2"
  -   another feature
  +      * **cmd-feat2** -- another feature

   Note:

     "arg3" is undocumented

  +   Return:
  +      "Object" -- "Object"

  -Returns
  -~~~~~~~
  -
  -"Object"
  -
  -
  -Errors
  -~~~~~~
  -
  +   Errors:
   * some

   Notes:
  @@ -68,10 +50,3 @@ Examples:
   Note::
      Ceci n'est pas une note

  -
  -Since
  -~~~~~
  -
  -2.10
  -
  -
  diff -rupw o/09 n/09
  --- o/09	2025-06-16 14:11:32.819939859 +0200
  +++ n/09	2025-06-16 13:51:43.469482599 +0200
  @@ -1,22 +1,14 @@
  -"cmd-boxed" (Command)
  ----------------------
  +Command cmd-boxed

   If you're bored enough to read this, go see a video of boxed cats

  +   Arguments:
  +      * The members of "Object".

  -Arguments
  -~~~~~~~~~
  +   Features:
  +      * **cmd-feat1** -- a feature

  -The members of "Object"
  -
  -Features
  -~~~~~~~~
  -
  -"cmd-feat1"
  -   a feature
  -
  -"cmd-feat2"
  -   another feature
  +      * **cmd-feat2** -- another feature

   Example::

  @@ -24,4 +16,3 @@ Example::

      <- ... has no title ...

  -
  diff -rupw o/10 n/10
  --- o/10	2025-06-16 14:11:35.771883514 +0200
  +++ n/10	2025-06-16 13:51:46.653422395 +0200
  @@ -1,14 +1,7 @@
  -"EVT_BOXED" (Event)
  --------------------
  +Event EVT_BOXED

  +   Members:
  +      * The members of "Object".

  -Arguments
  -~~~~~~~~~
  -
  -The members of "Object"
  -
  -Features
  -~~~~~~~~
  -
  -"feat3"
  -   a feature
  +   Features:
  +      * **feat3** -- a feature

The only unexpected change is the first hunk.

Re: [PATCH v2 2/3] docs: remove legacy QAPI manual generator

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 9321 bytes --]

On Mon, Jun 16, 2025 at 8:20 AM Markus Armbruster <armbru@redhat.com> wrote:

> John Snow <jsnow@redhat.com> writes:
>
> > Thanks for your service!
> >
> > Remove the old qapidoc and the option to enable the transmogrifier,
> > leaving the "transmogrifier" as the ONLY qapi doc generator. This in
> > effect also converts the QAPI test to use the new documentation
> > generator, too.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
> Fails "make check", because tests/qapi-schema/doc-good.txt needs an
> update.
>
> Unfortunately, the diff of the update is less than useful.  To make
> sense of what changes, I split doc-good.txt into parts before and after,
> and diffed those.
>
>
>   diff -rupw o/01 n/01
>   --- o/01      2025-06-16 13:53:05.036940854 +0200
>   +++ n/01      2025-06-16 13:49:07.167435996 +0200
>   @@ -1,11 +1,13 @@
>    Section
>    *******
>
>   +Just text, no heading.
>   +
>
> Looks like a bug fix.  Needs a mention in the commit message then.
>

I think before, these sections just got ... discarded? but with no special
formatting, they just get copied through. You could call it a bugfix, you
could call it an unintentional side effect.

For my money, I think it's fine to allow free-form docs to have any
arbitrary content. If it produces bad results, don't do it, then. Fewer
constraints = less code.


>
>    Subsection
>    ==========
>
>   -*with emphasis* "var" {in braces}
>   +*with emphasis* @var {in braces}
>
> Rendering of @references changes.  Okay.
>

Unintentional. Freeform sections are handled outside of "visit_sections",
because they are not attached to an entity, and so the @references
rendering is missed. Bug, I'll fix it.


>
>    * List item one
>
>   @@ -35,4 +37,3 @@ Example:
>
>    -> in <- out Examples: - *verbatim* - {braces}
>
>   -
>   diff -rupw o/02 n/02
>   --- o/02      2025-06-16 13:53:17.133712123 +0200
>   +++ n/02      2025-06-16 13:49:21.024174424 +0200
>   @@ -1,32 +1,15 @@
>   -"Enum" (Enum)
>   --------------
>   +Enum Enum
>   +    *Availability*: "IFCOND"
>
>   +   Values:
>   +      * **one** -- The _one_ {and only}, description on the same line
>
>   -Values
>   -~~~~~~
>   +      * **two** -- Not documented
>
>   -"one" (**If: **"IFONE")
>
> Member conditional is lost.  Known issue; see commit dbf51d15fdb.
>
>   -   The _one_ {and only}, description on the same line
>   +   Features:
>   +      * **enum-feat** -- Also _one_ {and only}
>
>   -"two"
>   -   Not documented
>   -
>   -
>   -Features
>   -~~~~~~~~
>   -
>   -"enum-feat"
>   -   Also _one_ {and only}
>   -
>   -"enum-member-feat"
>   -   a member feature
>   +      * **enum-member-feat** -- a member feature
>
>    "two" is undocumented
>
>   -
>   -If
>   -~~
>   -
>   -"IFCOND"
>   -
>   -
>   diff -rupw o/03 n/03
>   --- o/03      2025-06-16 13:53:27.556514971 +0200
>   +++ n/03      2025-06-16 13:50:44.582595942 +0200
>   @@ -1,17 +1,7 @@
>   -"Base" (Object)
>   ----------------
>   -
>   -
>   -Members
>   -~~~~~~~
>   -
>   -"base1": "Enum"
>   -   description starts on a new line, minimally indented
>   -
>   -
>   -If
>   -~~
>   -
>   -"IFALL1 and IFALL2"
>   +Object Base
>   +    *Availability*: "IFALL1 and IFALL2"
>
>   +   Members:
>   +      * **base1** ("Enum") -- description starts on a new line,
>   +        minimally indented
>
>   diff -rupw o/04 n/04
>   --- o/04      2025-06-16 13:53:39.356291772 +0200
>   +++ n/04      2025-06-16 13:50:54.486408751 +0200
>   @@ -1,5 +1,4 @@
>   -"Variant1" (Object)
>   --------------------
>   +Object Variant1
>
>    A paragraph
>
>   @@ -7,21 +6,11 @@ Another paragraph
>
>    "var1" is undocumented
>
>   +   Members:
>   +      * **var1** ("string") -- Not documented
>
>   -Members
>   -~~~~~~~
>   -
>   -"var1": "string" (**If: **"IFSTR")
>
> Likewise.
>
>   -   Not documented
>   -
>   -
>   -Features
>   -~~~~~~~~
>   -
>   -"variant1-feat"
>   -   a feature
>   -
>   -"member-feat"
>   -   a member feature
>   +   Features:
>   +      * **variant1-feat** -- a feature
>
>   +      * **member-feat** -- a member feature
>
>   diff -rupw o/05 n/05
>   --- o/05      2025-06-16 13:53:43.429214731 +0200
>   +++ n/05      2025-06-16 13:51:05.247205330 +0200
>   @@ -1,4 +1,2 @@
>   -"Variant2" (Object)
>   --------------------
>   -
>   +Object Variant2
>
>   diff -rupw o/06 n/06
>   --- o/06      2025-06-16 13:53:48.461119551 +0200
>   +++ n/06      2025-06-16 13:51:10.990096739 +0200
>   @@ -1,19 +1,12 @@
>   -"Object" (Object)
>   ------------------
>   +Object Object
>
>   +   Members:
>   +      * The members of "Base".
>
>   -Members
>   -~~~~~~~
>   +      * When "base1" is "one": The members of "Variant1".
>
>   -The members of "Base"
>   -The members of "Variant1" when "base1" is ""one""
>   -The members of "Variant2" when "base1" is ""two"" (**If: **"IFONE or
>   -IFTWO")
>
> Likewise.
>
>   -
>   -Features
>   -~~~~~~~~
>   -
>   -"union-feat1"
>   -   a feature
>   +      * When "base1" is "two": The members of "Variant2".
>
>   +   Features:
>   +      * **union-feat1** -- a feature
>
>   diff -rupw o/07 n/07
>   --- o/07      2025-06-16 13:53:55.988977158 +0200
>   +++ n/07      2025-06-16 13:51:16.869985559 +0200
>   @@ -1,28 +1,13 @@
>   -"Alternate" (Alternate)
>   ------------------------
>   +Alternate Alternate
>   +    *Availability*: "not (IFONE or IFTWO)"
>
>   +   Alternatives:
>   +      * **i** ("int") -- description starts on the same line remainder
>   +        indented the same "b" is undocumented
>
>   -Members
>   -~~~~~~~
>   +      * **b** ("boolean") -- Not documented
>
>   -"i": "int"
>   -   description starts on the same line remainder indented the same "b"
>   -   is undocumented
>   -
>   -"b": "boolean"
>   -   Not documented
>   -
>   -
>   -Features
>   -~~~~~~~~
>   -
>   -"alt-feat"
>   -   a feature
>   -
>   -
>   -If
>   -~~
>   -
>   -"not (IFONE or IFTWO)"
>   +   Features:
>   +      * **alt-feat** -- a feature
>
>
>   diff -rupw o/08 n/08
>   --- o/08      2025-06-16 13:54:11.692680115 +0200
>   +++ n/08      2025-06-16 13:51:35.957624638 +0200
>   @@ -1,47 +1,29 @@
>    Another subsection
>    ==================
>
>   +Command cmd (Since: 2.10)
>
>   -"cmd" (Command)
>   ----------------
>   +   Arguments:
>   +      * **arg1** ("int") -- description starts on a new line, indented
>
>   +      * **arg2** ("string", *optional*) -- description starts on the
>   +        same line remainder indented differently
>
>   -Arguments
>   -~~~~~~~~~
>   +      * **arg3** ("boolean") -- Not documented
>
>   -"arg1": "int"
>   -   description starts on a new line, indented
>   +   Features:
>   +      * **cmd-feat1** -- a feature
>
>   -"arg2": "string" (optional)
>   -   description starts on the same line remainder indented differently
>   -
>   -"arg3": "boolean"
>   -   Not documented
>   -
>   -
>   -Features
>   -~~~~~~~~
>   -
>   -"cmd-feat1"
>   -   a feature
>   -
>   -"cmd-feat2"
>   -   another feature
>   +      * **cmd-feat2** -- another feature
>
>    Note:
>
>      "arg3" is undocumented
>
>   +   Return:
>   +      "Object" -- "Object"
>
>   -Returns
>   -~~~~~~~
>   -
>   -"Object"
>   -
>   -
>   -Errors
>   -~~~~~~
>   -
>   +   Errors:
>    * some
>
>    Notes:
>   @@ -68,10 +50,3 @@ Examples:
>    Note::
>       Ceci n'est pas une note
>
>   -
>   -Since
>   -~~~~~
>   -
>   -2.10
>   -
>   -
>   diff -rupw o/09 n/09
>   --- o/09      2025-06-16 14:11:32.819939859 +0200
>   +++ n/09      2025-06-16 13:51:43.469482599 +0200
>   @@ -1,22 +1,14 @@
>   -"cmd-boxed" (Command)
>   ----------------------
>   +Command cmd-boxed
>
>    If you're bored enough to read this, go see a video of boxed cats
>
>   +   Arguments:
>   +      * The members of "Object".
>
>   -Arguments
>   -~~~~~~~~~
>   +   Features:
>   +      * **cmd-feat1** -- a feature
>
>   -The members of "Object"
>   -
>   -Features
>   -~~~~~~~~
>   -
>   -"cmd-feat1"
>   -   a feature
>   -
>   -"cmd-feat2"
>   -   another feature
>   +      * **cmd-feat2** -- another feature
>
>    Example::
>
>   @@ -24,4 +16,3 @@ Example::
>
>       <- ... has no title ...
>
>   -
>   diff -rupw o/10 n/10
>   --- o/10      2025-06-16 14:11:35.771883514 +0200
>   +++ n/10      2025-06-16 13:51:46.653422395 +0200
>   @@ -1,14 +1,7 @@
>   -"EVT_BOXED" (Event)
>   --------------------
>   +Event EVT_BOXED
>
>   +   Members:
>   +      * The members of "Object".
>
>   -Arguments
>   -~~~~~~~~~
>   -
>   -The members of "Object"
>   -
>   -Features
>   -~~~~~~~~
>   -
>   -"feat3"
>   -   a feature
>   +   Features:
>   +      * **feat3** -- a feature
>
> The only unexpected change is the first hunk.


Some of these look a little silly, let me see what I can do ... The IFCOND
stuff is a known issue, and I believe is best tackled in conjunction with
your project to add conditional documentation entities.

Everything else, I'll review and see what can be improved upon if anything.

[-- Attachment #2: Type: text/html, Size: 12411 bytes --]

Re: [PATCH v2 2/3] docs: remove legacy QAPI manual generator

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> On Mon, Jun 16, 2025 at 8:20 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > Thanks for your service!
>> >
>> > Remove the old qapidoc and the option to enable the transmogrifier,
>> > leaving the "transmogrifier" as the ONLY qapi doc generator. This in
>> > effect also converts the QAPI test to use the new documentation
>> > generator, too.
>> >
>> > Signed-off-by: John Snow <jsnow@redhat.com>
>>
>> Fails "make check", because tests/qapi-schema/doc-good.txt needs an
>> update.
>>
>> Unfortunately, the diff of the update is less than useful.  To make
>> sense of what changes, I split doc-good.txt into parts before and after,
>> and diffed those.
>>
>>
>>   diff -rupw o/01 n/01
>>   --- o/01      2025-06-16 13:53:05.036940854 +0200
>>   +++ n/01      2025-06-16 13:49:07.167435996 +0200
>>   @@ -1,11 +1,13 @@
>>    Section
>>    *******
>>
>>   +Just text, no heading.
>>   +
>>
>> Looks like a bug fix.  Needs a mention in the commit message then.
>>
>
> I think before, these sections just got ... discarded?

Looks like it.  Definitely a bug.

>                                                        but with no special
> formatting, they just get copied through. You could call it a bugfix, you
> could call it an unintentional side effect.

I'm calling it a bug fix.  Where was it broken?  Dig, dig, dig, ...

commit b61a4eb3f32ce74c5ffe001806f9e786788a546f
Author: John Snow <jsnow@redhat.com>
Date:   Sun Feb 23 22:37:35 2025 -0500

    docs/qapidoc: support header-less freeform sections
    
    The code as written crashes when a free-form documentation block doesn't
    start with a heading or subheading, for example:
    
    | ##
    | # Just text, no heading.
    | ##
    
    The code will attempt to use the `node` variable uninitialized. To fix,
    create a generic block to insert the doc text into.
    
    (This patch also removes a lingering pylint warning in the QAPIDoc
    implementation that prevents getting a clean baseline to use for
    forthcoming additions.)
    
    Fixes: 43e0d14ee09a (docs/sphinx: fix extra stuff in TOC after freeform QMP sections)
    Signed-off-by: John Snow <jsnow@redhat.com>
    Message-ID: <20250224033741.222749-5-jsnow@redhat.com>
    Reviewed-by: Markus Armbruster <armbru@redhat.com>
    [Test updated to cover this]
    Signed-off-by: Markus Armbruster <armbru@redhat.com>

The patch adds such a block to doc-good.json.  It shows up in expected
output doc-good.out, but not in doc-good.txt.  Review fail.

Please mention the fix in the commit message, and add the Fixes: tag.

[...]

[PATCH v2 3/3] docs: remove special parsing for freeform sections

[John Snow]

This change removes special parsing for freeform sections and allows
them to simply be unmodified rST syntax. The existing headings in the
QAPI schema are adjusted to reflect the new paradigm.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/devel/qapi-code-gen.rst         | 28 ++++++++++++++-------
 docs/interop/firmware.json           |  4 ++-
 docs/interop/vhost-user.json         |  4 ++-
 docs/sphinx/qapidoc.py               | 37 +---------------------------
 qapi/acpi.json                       |  4 ++-
 qapi/audio.json                      |  4 ++-
 qapi/authz.json                      |  4 ++-
 qapi/block-core.json                 |  3 ++-
 qapi/block-export.json               |  3 ++-
 qapi/block.json                      |  7 ++++--
 qapi/char.json                       |  4 ++-
 qapi/common.json                     |  4 ++-
 qapi/compat.json                     |  4 ++-
 qapi/control.json                    |  4 ++-
 qapi/crypto.json                     |  4 ++-
 qapi/cryptodev.json                  |  4 ++-
 qapi/cxl.json                        |  4 ++-
 qapi/dump.json                       |  4 ++-
 qapi/ebpf.json                       |  4 ++-
 qapi/error.json                      |  4 ++-
 qapi/introspect.json                 |  4 ++-
 qapi/job.json                        |  4 ++-
 qapi/machine-common.json             |  4 ++-
 qapi/machine.json                    |  4 ++-
 qapi/migration.json                  |  4 ++-
 qapi/misc.json                       |  4 ++-
 qapi/net.json                        |  4 ++-
 qapi/pci.json                        |  4 ++-
 qapi/qapi-schema.json                |  4 ++-
 qapi/qdev.json                       |  4 ++-
 qapi/qom.json                        |  4 ++-
 qapi/replay.json                     |  4 ++-
 qapi/rocker.json                     |  4 ++-
 qapi/run-state.json                  |  4 ++-
 qapi/sockets.json                    |  4 ++-
 qapi/stats.json                      |  4 ++-
 qapi/tpm.json                        |  4 ++-
 qapi/trace.json                      |  4 ++-
 qapi/transaction.json                |  4 ++-
 qapi/uefi.json                       |  4 ++-
 qapi/ui.json                         | 14 ++++++++---
 qapi/vfio.json                       |  4 ++-
 qapi/virtio.json                     |  4 ++-
 qapi/yank.json                       |  4 ++-
 scripts/qapi/parser.py               |  7 ------
 storage-daemon/qapi/qapi-schema.json |  8 ++++--
 tests/qapi-schema/doc-good.json      | 10 +++++---
 47 files changed, 166 insertions(+), 103 deletions(-)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index 231cc0fecf7..dfdbeac5a5a 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -876,25 +876,35 @@ structuring content.
 Headings and subheadings
 ~~~~~~~~~~~~~~~~~~~~~~~~
 
-A free-form documentation comment containing a line which starts with
-some ``=`` symbols and then a space defines a section heading::
+Free-form documentation does not start with ``@SYMBOL`` and can contain
+arbitrary rST markup. Headings can be marked up using the standard rST
+syntax::
 
     ##
-    # = This is a top level heading
+    # *************************
+    # This is a level 2 heading
+    # *************************
     #
     # This is a free-form comment which will go under the
     # top level heading.
     ##
 
     ##
-    # == This is a second level heading
+    # This is a third level heading
+    # ==============================
+    #
+    # Level 4
+    # _______
+    #
+    # Level 5
+    # ^^^^^^^
+    #
+    # Level 6
+    # """""""
     ##
 
-A heading line must be the first line of the documentation
-comment block.
-
-Section headings must always be correctly nested, so you can only
-define a third-level heading inside a second-level heading, and so on.
+Level 1 headings are reserved for use by the generated documentation
+page itself, leaving level 2 as the highest level that should be used.
 
 
 Documentation markup
diff --git a/docs/interop/firmware.json b/docs/interop/firmware.json
index 745d21d8223..f46fdedfa89 100644
--- a/docs/interop/firmware.json
+++ b/docs/interop/firmware.json
@@ -11,7 +11,9 @@
 # later. See the COPYING file in the top-level directory.
 
 ##
-# = Firmware
+# ********
+# Firmware
+# ********
 ##
 
 { 'pragma': {
diff --git a/docs/interop/vhost-user.json b/docs/interop/vhost-user.json
index b6ade9e4931..095eb99cf79 100644
--- a/docs/interop/vhost-user.json
+++ b/docs/interop/vhost-user.json
@@ -10,7 +10,9 @@
 # later. See the COPYING file in the top-level directory.
 
 ##
-# = vhost user backend discovery & capabilities
+# *******************************************
+# vhost user backend discovery & capabilities
+# *******************************************
 ##
 
 ##
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index c9c477378f5..cdf556c2a3c 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -391,44 +391,9 @@ def visit_module(self, path: str) -> None:
         self.ensure_blank_line()
 
     def visit_freeform(self, doc: QAPIDoc) -> None:
-        # TODO: Once the old qapidoc transformer is deprecated, freeform
-        # sections can be updated to pure rST, and this transformed removed.
-        #
-        # For now, translate our micro-format into rST. Code adapted
-        # from Peter Maydell's freeform().
-
         assert len(doc.all_sections) == 1, doc.all_sections
         body = doc.all_sections[0]
-        text = body.text
-        info = doc.info
-
-        if re.match(r"=+ ", text):
-            # Section/subsection heading (if present, will always be the
-            # first line of the block)
-            (heading, _, text) = text.partition("\n")
-            (leader, _, heading) = heading.partition(" ")
-            # Implicit +1 for heading in the containing .rst doc
-            level = len(leader) + 1
-
-            # https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#sections
-            markers = ' #*=_^"'
-            overline = level <= 2
-            marker = markers[level]
-
-            self.ensure_blank_line()
-            # This credits all 2 or 3 lines to the single source line.
-            if overline:
-                self.add_line(marker * len(heading), info)
-            self.add_line(heading, info)
-            self.add_line(marker * len(heading), info)
-            self.ensure_blank_line()
-
-            # Eat blank line(s) and advance info
-            trimmed = text.lstrip("\n")
-            text = trimmed
-            info = info.next_line(len(text) - len(trimmed) + 1)
-
-        self.add_lines(text, info)
+        self.add_lines(body.text, doc.info)
         self.ensure_blank_line()
 
     def visit_entity(self, ent: QAPISchemaDefinition) -> None:
diff --git a/qapi/acpi.json b/qapi/acpi.json
index 2d53b823656..90f8f564fda 100644
--- a/qapi/acpi.json
+++ b/qapi/acpi.json
@@ -6,7 +6,9 @@
 # SPDX-License-Identifier: GPL-2.0-or-later
 
 ##
-# = ACPI
+# ****
+# ACPI
+# ****
 ##
 
 ##
diff --git a/qapi/audio.json b/qapi/audio.json
index 16de231f6d8..6a22ca384aa 100644
--- a/qapi/audio.json
+++ b/qapi/audio.json
@@ -7,7 +7,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Audio
+# *****
+# Audio
+# *****
 ##
 
 ##
diff --git a/qapi/authz.json b/qapi/authz.json
index 7fc6e3032ea..ad1b4b3af0c 100644
--- a/qapi/authz.json
+++ b/qapi/authz.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = User authorization
+# ******************
+# User authorization
+# ******************
 ##
 
 ##
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 1df6644f0de..8b413946ff8 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2,7 +2,8 @@
 # vim: filetype=python
 
 ##
-# == Block core (VM unrelated)
+# Block core (VM unrelated)
+# =========================
 ##
 
 { 'include': 'common.json' }
diff --git a/qapi/block-export.json b/qapi/block-export.json
index ed4deb54db2..2241bfecf25 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -2,7 +2,8 @@
 # vim: filetype=python
 
 ##
-# == Block device exports
+# Block device exports
+# ====================
 ##
 
 { 'include': 'sockets.json' }
diff --git a/qapi/block.json b/qapi/block.json
index 1490a1a17f8..2d54a81c366 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -2,13 +2,16 @@
 # vim: filetype=python
 
 ##
-# = Block devices
+# *************
+# Block devices
+# *************
 ##
 
 { 'include': 'block-core.json' }
 
 ##
-# == Additional block stuff (VM related)
+# Additional block stuff (VM related)
+# ===================================
 ##
 
 ##
diff --git a/qapi/char.json b/qapi/char.json
index df6e325e2e1..f38d04986dd 100644
--- a/qapi/char.json
+++ b/qapi/char.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Character devices
+# *****************
+# Character devices
+# *****************
 ##
 
 { 'include': 'sockets.json' }
diff --git a/qapi/common.json b/qapi/common.json
index 0e3a0bbbfb0..af7e3d618a7 100644
--- a/qapi/common.json
+++ b/qapi/common.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = Common data types
+# *****************
+# Common data types
+# *****************
 ##
 
 ##
diff --git a/qapi/compat.json b/qapi/compat.json
index 42034d9368c..90b8d51cf27 100644
--- a/qapi/compat.json
+++ b/qapi/compat.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = Compatibility policy
+# ********************
+# Compatibility policy
+# ********************
 ##
 
 ##
diff --git a/qapi/control.json b/qapi/control.json
index 34b733f63b6..ab0b3a3bbe5 100644
--- a/qapi/control.json
+++ b/qapi/control.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = QMP monitor control
+# *******************
+# QMP monitor control
+# *******************
 ##
 
 ##
diff --git a/qapi/crypto.json b/qapi/crypto.json
index 9ec6301e188..21006de36c4 100644
--- a/qapi/crypto.json
+++ b/qapi/crypto.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Cryptography
+# ************
+# Cryptography
+# ************
 ##
 
 ##
diff --git a/qapi/cryptodev.json b/qapi/cryptodev.json
index b13db264034..1f49e1822c0 100644
--- a/qapi/cryptodev.json
+++ b/qapi/cryptodev.json
@@ -5,7 +5,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Cryptography devices
+# ********************
+# Cryptography devices
+# ********************
 ##
 
 ##
diff --git a/qapi/cxl.json b/qapi/cxl.json
index 8f2e9237b19..52cc5d4f336 100644
--- a/qapi/cxl.json
+++ b/qapi/cxl.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = CXL devices
+# ***********
+# CXL devices
+# ***********
 ##
 
 ##
diff --git a/qapi/dump.json b/qapi/dump.json
index d0ba1f0596f..0642ca157b8 100644
--- a/qapi/dump.json
+++ b/qapi/dump.json
@@ -5,7 +5,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Dump guest memory
+# *****************
+# Dump guest memory
+# *****************
 ##
 
 ##
diff --git a/qapi/ebpf.json b/qapi/ebpf.json
index db19ae850fc..d45054e6663 100644
--- a/qapi/ebpf.json
+++ b/qapi/ebpf.json
@@ -5,7 +5,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = eBPF Objects
+# ************
+# eBPF Objects
+# ************
 #
 # eBPF object is an ELF binary that contains the eBPF program and eBPF
 # map description(BTF).  Overall, eBPF object should contain the
diff --git a/qapi/error.json b/qapi/error.json
index 135c1e82319..54cb02fb880 100644
--- a/qapi/error.json
+++ b/qapi/error.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = QMP errors
+# **********
+# QMP errors
+# **********
 ##
 
 ##
diff --git a/qapi/introspect.json b/qapi/introspect.json
index e9e02972821..26d8839f19c 100644
--- a/qapi/introspect.json
+++ b/qapi/introspect.json
@@ -10,7 +10,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = QMP introspection
+# *****************
+# QMP introspection
+# *****************
 ##
 
 ##
diff --git a/qapi/job.json b/qapi/job.json
index 126fa5ce602..16b280f52f8 100644
--- a/qapi/job.json
+++ b/qapi/job.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = Background jobs
+# ***************
+# Background jobs
+# ***************
 ##
 
 ##
diff --git a/qapi/machine-common.json b/qapi/machine-common.json
index 298e51f373a..0f01599130c 100644
--- a/qapi/machine-common.json
+++ b/qapi/machine-common.json
@@ -5,7 +5,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Common machine types
+# ********************
+# Common machine types
+# ********************
 ##
 
 ##
diff --git a/qapi/machine.json b/qapi/machine.json
index 0650b8de71a..6ebb99dfabe 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -5,7 +5,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Machines
+# ********
+# Machines
+# ********
 ##
 
 { 'include': 'common.json' }
diff --git a/qapi/migration.json b/qapi/migration.json
index 4963f6ca127..84f4c800f7b 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Migration
+# *********
+# Migration
+# *********
 ##
 
 { 'include': 'common.json' }
diff --git a/qapi/misc.json b/qapi/misc.json
index 4b9e601cfa5..a180c16db25 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Miscellanea
+# ***********
+# Miscellanea
+# ***********
 ##
 
 { 'include': 'common.json' }
diff --git a/qapi/net.json b/qapi/net.json
index 97ea1839813..3b03843c165 100644
--- a/qapi/net.json
+++ b/qapi/net.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Net devices
+# ***********
+# Net devices
+# ***********
 ##
 
 { 'include': 'sockets.json' }
diff --git a/qapi/pci.json b/qapi/pci.json
index dc85a41d28b..a8671cd9ac3 100644
--- a/qapi/pci.json
+++ b/qapi/pci.json
@@ -6,7 +6,9 @@
 # SPDX-License-Identifier: GPL-2.0-or-later
 
 ##
-# = PCI
+# ***
+# PCI
+# ***
 ##
 
 ##
diff --git a/qapi/qapi-schema.json b/qapi/qapi-schema.json
index a8f66163cb7..49b9a0267d3 100644
--- a/qapi/qapi-schema.json
+++ b/qapi/qapi-schema.json
@@ -1,7 +1,9 @@
 # -*- Mode: Python -*-
 # vim: filetype=python
 ##
-# = Introduction
+# ************
+# Introduction
+# ************
 #
 # This manual describes the commands and events supported by the QEMU
 # Monitor Protocol (QMP).
diff --git a/qapi/qdev.json b/qapi/qdev.json
index 32c7d100463..441ed518b87 100644
--- a/qapi/qdev.json
+++ b/qapi/qdev.json
@@ -5,7 +5,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Device infrastructure (qdev)
+# ****************************
+# Device infrastructure (qdev)
+# ****************************
 ##
 
 { 'include': 'qom.json' }
diff --git a/qapi/qom.json b/qapi/qom.json
index 3e8debf78c2..c8fe0258a5f 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -10,7 +10,9 @@
 { 'include': 'crypto.json' }
 
 ##
-# = QEMU Object Model (QOM)
+# ***********************
+# QEMU Object Model (QOM)
+# ***********************
 ##
 
 ##
diff --git a/qapi/replay.json b/qapi/replay.json
index 35e0c4a6926..e46c5c1d3f3 100644
--- a/qapi/replay.json
+++ b/qapi/replay.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Record/replay
+# *************
+# Record/replay
+# *************
 ##
 
 { 'include': 'common.json' }
diff --git a/qapi/rocker.json b/qapi/rocker.json
index 0c7ef1f77c8..e4949649526 100644
--- a/qapi/rocker.json
+++ b/qapi/rocker.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = Rocker switch device
+# ********************
+# Rocker switch device
+# ********************
 ##
 
 ##
diff --git a/qapi/run-state.json b/qapi/run-state.json
index fd09beb35cb..083a3c5eb30 100644
--- a/qapi/run-state.json
+++ b/qapi/run-state.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = VM run state
+# ************
+# VM run state
+# ************
 ##
 
 ##
diff --git a/qapi/sockets.json b/qapi/sockets.json
index f9f559dabae..b5f4a8fecda 100644
--- a/qapi/sockets.json
+++ b/qapi/sockets.json
@@ -2,7 +2,9 @@
 # vim: filetype=python
 
 ##
-# = Socket data types
+# *****************
+# Socket data types
+# *****************
 ##
 
 ##
diff --git a/qapi/stats.json b/qapi/stats.json
index 8902ef94e08..78b88881eab 100644
--- a/qapi/stats.json
+++ b/qapi/stats.json
@@ -9,7 +9,9 @@
 # SPDX-License-Identifier: GPL-2.0-or-later
 
 ##
-# = Statistics
+# **********
+# Statistics
+# **********
 ##
 
 ##
diff --git a/qapi/tpm.json b/qapi/tpm.json
index a16a72edb98..e66b107f1d0 100644
--- a/qapi/tpm.json
+++ b/qapi/tpm.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = TPM (trusted platform module) devices
+# *************************************
+# TPM (trusted platform module) devices
+# *************************************
 ##
 
 ##
diff --git a/qapi/trace.json b/qapi/trace.json
index eb5f63f5135..d08c9c6a889 100644
--- a/qapi/trace.json
+++ b/qapi/trace.json
@@ -7,7 +7,9 @@
 # See the COPYING file in the top-level directory.
 
 ##
-# = Tracing
+# *******
+# Tracing
+# *******
 ##
 
 ##
diff --git a/qapi/transaction.json b/qapi/transaction.json
index 9d9e7af26cb..927035f5a7e 100644
--- a/qapi/transaction.json
+++ b/qapi/transaction.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Transactions
+# ************
+# Transactions
+# ************
 ##
 
 { 'include': 'block-core.json' }
diff --git a/qapi/uefi.json b/qapi/uefi.json
index 6592183d6cf..a206c2e9539 100644
--- a/qapi/uefi.json
+++ b/qapi/uefi.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = UEFI Variable Store
+# *******************
+# UEFI Variable Store
+# *******************
 #
 # The QEMU efi variable store implementation (hw/uefi/) uses this to
 # store non-volatile variables in json format on disk.
diff --git a/qapi/ui.json b/qapi/ui.json
index 514fa159b10..f5e77ae19d7 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Remote desktop
+# **************
+# Remote desktop
+# **************
 ##
 
 { 'include': 'common.json' }
@@ -200,7 +202,8 @@
   'if': 'CONFIG_PIXMAN' }
 
 ##
-# == Spice
+# Spice
+# =====
 ##
 
 ##
@@ -461,7 +464,8 @@
   'if': 'CONFIG_SPICE' }
 
 ##
-# == VNC
+# VNC
+# ===
 ##
 
 ##
@@ -794,7 +798,9 @@
   'if': 'CONFIG_VNC' }
 
 ##
-# = Input
+# *****
+# Input
+# *****
 ##
 
 ##
diff --git a/qapi/vfio.json b/qapi/vfio.json
index b53b7caecd7..a1a9c5b673d 100644
--- a/qapi/vfio.json
+++ b/qapi/vfio.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = VFIO devices
+# ************
+# VFIO devices
+# ************
 ##
 
 ##
diff --git a/qapi/virtio.json b/qapi/virtio.json
index 73df718a261..3cac0774f7a 100644
--- a/qapi/virtio.json
+++ b/qapi/virtio.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Virtio devices
+# **************
+# Virtio devices
+# **************
 ##
 
 ##
diff --git a/qapi/yank.json b/qapi/yank.json
index 30f46c97c98..d13a8e91171 100644
--- a/qapi/yank.json
+++ b/qapi/yank.json
@@ -3,7 +3,9 @@
 #
 
 ##
-# = Yank feature
+# ************
+# Yank feature
+# ************
 ##
 
 ##
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 949d9e8bff7..aad7e249f86 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -597,22 +597,15 @@ def get_doc(self) -> 'QAPIDoc':
             # Free-form documentation
             doc = QAPIDoc(info)
             doc.ensure_untagged_section(self.info)
-            first = True
             while line is not None:
                 if match := self._match_at_name_colon(line):
                     raise QAPIParseError(
                         self,
                         "'@%s:' not allowed in free-form documentation"
                         % match.group(1))
-                if line.startswith('='):
-                    if not first:
-                        raise QAPIParseError(
-                            self,
-                            "'=' heading must come first in a comment block")
                 doc.append_line(line)
                 self.accept(False)
                 line = self.get_doc_line()
-                first = False
 
         self.accept()
         doc.end()
diff --git a/storage-daemon/qapi/qapi-schema.json b/storage-daemon/qapi/qapi-schema.json
index 0427594c984..478e7a92b21 100644
--- a/storage-daemon/qapi/qapi-schema.json
+++ b/storage-daemon/qapi/qapi-schema.json
@@ -14,7 +14,9 @@
 # storage daemon.
 
 ##
-# = Introduction
+# ************
+# Introduction
+# ************
 #
 # This manual describes the commands and events supported by the QEMU
 # storage daemon QMP.
@@ -51,7 +53,9 @@
 { 'include': '../../qapi/job.json' }
 
 ##
-# = Block devices
+# *************
+# Block devices
+# *************
 ##
 { 'include': '../../qapi/block-core.json' }
 { 'include': '../../qapi/block-export.json' }
diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index 6dcde8fd7e8..183da7df724 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -8,7 +8,9 @@
     'documentation-exceptions': [ 'Enum', 'Variant1', 'Alternate', 'cmd' ] } }
 
 ##
-# = Section
+# *******
+# Section
+# *******
 ##
 
 ##
@@ -16,7 +18,8 @@
 ##
 
 ##
-# == Subsection
+# Subsection
+# ==========
 #
 # *with emphasis*
 # @var {in braces}
@@ -144,7 +147,8 @@
   'if': { 'not': { 'any': [ 'IFONE', 'IFTWO' ] } } }
 
 ##
-# == Another subsection
+# Another subsection
+# ==================
 ##
 
 ##
-- 
2.48.1

Re: [PATCH v2 3/3] docs: remove special parsing for freeform sections

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> This change removes special parsing for freeform sections and allows
> them to simply be unmodified rST syntax. The existing headings in the
> QAPI schema are adjusted to reflect the new paradigm.
>
> Signed-off-by: John Snow <jsnow@redhat.com>

[...]

> diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
> index 231cc0fecf7..dfdbeac5a5a 100644
> --- a/docs/devel/qapi-code-gen.rst
> +++ b/docs/devel/qapi-code-gen.rst
> @@ -876,25 +876,35 @@ structuring content.
>  Headings and subheadings
>  ~~~~~~~~~~~~~~~~~~~~~~~~
>  
> -A free-form documentation comment containing a line which starts with
> -some ``=`` symbols and then a space defines a section heading::
> +Free-form documentation does not start with ``@SYMBOL`` and can contain
> +arbitrary rST markup. Headings can be marked up using the standard rST

Two spaces between sentences for consistency, please.

> +syntax::
>  
>      ##
> -    # = This is a top level heading
> +    # *************************
> +    # This is a level 2 heading
> +    # *************************
>      #
>      # This is a free-form comment which will go under the
>      # top level heading.
>      ##
>  
>      ##
> -    # == This is a second level heading
> +    # This is a third level heading
> +    # ==============================
> +    #
> +    # Level 4
> +    # _______
> +    #
> +    # Level 5
> +    # ^^^^^^^
> +    #
> +    # Level 6
> +    # """""""
>      ##
>  
> -A heading line must be the first line of the documentation
> -comment block.
> -
> -Section headings must always be correctly nested, so you can only
> -define a third-level heading inside a second-level heading, and so on.
> +Level 1 headings are reserved for use by the generated documentation
> +page itself, leaving level 2 as the highest level that should be used.
>  
>  
>  Documentation markup
> diff --git a/docs/interop/firmware.json b/docs/interop/firmware.json
> index 745d21d8223..f46fdedfa89 100644
> --- a/docs/interop/firmware.json
> +++ b/docs/interop/firmware.json
> @@ -11,7 +11,9 @@
>  # later. See the COPYING file in the top-level directory.
>  
>  ##
> -# = Firmware
> +# ********
> +# Firmware
> +# ********
>  ##
>  
>  { 'pragma': {
> diff --git a/docs/interop/vhost-user.json b/docs/interop/vhost-user.json
> index b6ade9e4931..095eb99cf79 100644
> --- a/docs/interop/vhost-user.json
> +++ b/docs/interop/vhost-user.json
> @@ -10,7 +10,9 @@
>  # later. See the COPYING file in the top-level directory.
>  
>  ##
> -# = vhost user backend discovery & capabilities
> +# *******************************************
> +# vhost user backend discovery & capabilities
> +# *******************************************
>  ##
>  
>  ##

I dislike reST headings.  Sticking to plain reST makes sense regardless.

> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index c9c477378f5..cdf556c2a3c 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -391,44 +391,9 @@ def visit_module(self, path: str) -> None:
>          self.ensure_blank_line()
>  
>      def visit_freeform(self, doc: QAPIDoc) -> None:
> -        # TODO: Once the old qapidoc transformer is deprecated, freeform
> -        # sections can be updated to pure rST, and this transformed removed.
> -        #
> -        # For now, translate our micro-format into rST. Code adapted
> -        # from Peter Maydell's freeform().
> -
>          assert len(doc.all_sections) == 1, doc.all_sections
>          body = doc.all_sections[0]
> -        text = body.text
> -        info = doc.info
> -
> -        if re.match(r"=+ ", text):
> -            # Section/subsection heading (if present, will always be the
> -            # first line of the block)
> -            (heading, _, text) = text.partition("\n")
> -            (leader, _, heading) = heading.partition(" ")
> -            # Implicit +1 for heading in the containing .rst doc
> -            level = len(leader) + 1
> -
> -            # https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#sections
> -            markers = ' #*=_^"'
> -            overline = level <= 2
> -            marker = markers[level]
> -
> -            self.ensure_blank_line()
> -            # This credits all 2 or 3 lines to the single source line.
> -            if overline:
> -                self.add_line(marker * len(heading), info)
> -            self.add_line(heading, info)
> -            self.add_line(marker * len(heading), info)
> -            self.ensure_blank_line()
> -
> -            # Eat blank line(s) and advance info
> -            trimmed = text.lstrip("\n")
> -            text = trimmed
> -            info = info.next_line(len(text) - len(trimmed) + 1)
> -
> -        self.add_lines(text, info)
> +        self.add_lines(body.text, doc.info)
>          self.ensure_blank_line()
>  
>      def visit_entity(self, ent: QAPISchemaDefinition) -> None:

Lovely.

Left in scripts/qapi/parser.py:

                elif line.startswith('='):
                    raise QAPIParseError(
                        self,
                        "unexpected '=' markup in definition documentation")

with test case tests/qapi-schema/doc-bad-section.json.

We might want to reject '=' markup anywhere for a while, to catch
outmoded headings.

[...]

Fails "make check".  Fixup:

diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index 3711cf5480..f60a24d1b7 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -55,13 +55,16 @@ event EVT_BOXED Object
     feature feat3
 doc freeform
     body=
-= Section
+*******
+Section
+*******
 doc freeform
     body=
 Just text, no heading.
 doc freeform
     body=
-== Subsection
+Subsection
+==========
 
 *with emphasis*
 @var {in braces}
@@ -155,7 +158,8 @@ description starts on the same line
 a feature
 doc freeform
     body=
-== Another subsection
+Another subsection
+==================
 doc symbol=cmd
     body=