[PATCH 00/11] docs/qapi: revise and expand QAPI documentation spec

[PATCH 00/11] docs/qapi: revise and expand QAPI documentation spec

[John Snow]

This patchset looks at expanding and clarifying the documentation for
... writing documentation. Primarily, the doc is being organized into
subsections - one for each QAPI doc section. These sections are
re-ordered to be documented in the same order as they are expected to
appear in-usage.

John Snow (11):
  docs/qapi-code-gen: change "x.y.z" to "x.y" for Since: documentation
  docs/qapi-code-gen: add "Putting it all together" subsection
  docs/qapi-code-gen: add "TODO" subsection
  docs/qapi-code-gen: add "Since" subsection
  docs/qapi-code-gen: add "Details" docs subsection
  docs/qapi-code-gen: add "Errors" and "Returns" subsections
  docs/qapi-code-gen: add "Tagged sections" subsection
  docs/qapi-code-gen: add "Descriptions" subsection
  docs/qapi-code-gen: add "Feature descriptions" subsection
  docs/qapi-code-gen: add "Intro" subsection
  docs/qapi-code-gen: add "Tagged sections" cross-references

 docs/devel/qapi-code-gen.rst | 136 +++++++++++++++++++++++++++++------
 1 file changed, 116 insertions(+), 20 deletions(-)

-- 
2.55.0

[PATCH 01/11] docs/qapi-code-gen: change "x.y.z" to "x.y" for Since: documentation

[John Snow]

We prefer using only x.y for Since: tags, not x.y.z. Adjust docs
accordingly.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/devel/qapi-code-gen.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index b1cc5b5f0db..f276a2bc9bc 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -1001,7 +1001,7 @@ indented like this::
 .. FIXME union branches should be described, too.
 
 Extensions added after the definition was first released carry a
-"(since x.y.z)" comment.
+"(since x.y)" comment.
 
 The feature descriptions must be preceded by a blank line and then a
 line "Features:", like this::
@@ -1034,7 +1034,7 @@ detailing a relevant error condition.  For example::
  #     - If @device does not exist, DeviceNotFound
  #     - Any other error returns a GenericError.
 
-A "Since: x.y.z" tagged section lists the release that introduced the
+A "Since: x.y" tagged section lists the release that introduced the
 definition.
 
 "TODO" sections are not rendered (they are for developers, not users of
-- 
2.55.0

Re: [PATCH 01/11] docs/qapi-code-gen: change "x.y.z" to "x.y" for Since: documentation

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> We prefer using only x.y for Since: tags, not x.y.z.

True:

    $ git-grep -Ei '\(since [0-9]+\.[0-9]+\.' qapi
    qapi/block-core.json:#     Not set if the job completed successfully.  (since 2.12.1)
    qapi/crypto.json:# @sm3: SM3.  (since 9.2.0)

The latter could be shortened to 9.2 for consistency.

The former was merged into master as commit ab9ba614556 on 2018-05-15,
released in v3.0.0 on 2018-08-14.  It was picked into stable as commit
0779afdc897 on 2018-07-18, released in 2.12.1 on 2018-08-02.  We do such
things all the time, but this is the only time we recorded a stable
release.  I think we could replace it by 3.0 for consistency.

>                                                      Adjust docs
> accordingly.
>
> Signed-off-by: John Snow <jsnow@redhat.com>

Reviewed-by: Markus Armbruster <armbru@redhat.com>

[PATCH 02/11] docs/qapi-code-gen: add "Putting it all together" subsection

[John Snow]

Separate the specification portion of the documentation section from
the holistic examples. Avoid naming the section "Examples" because it
might be confused with the literal case of documenting how code
examples should themselves be documented.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/devel/qapi-code-gen.rst | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index f276a2bc9bc..0f66f012be1 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -1089,6 +1089,9 @@ Highlighting in non-QMP languages can be accomplished by using the
 achieved by omitting the language argument.
 
 
+Putting it all together
+^^^^^^^^^^^^^^^^^^^^^^^
+
 Examples of complete definition documentation::
 
  ##
-- 
2.55.0

Re: [PATCH 02/11] docs/qapi-code-gen: add "Putting it all together" subsection

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> Separate the specification portion of the documentation section from
> the holistic examples. Avoid naming the section "Examples" because it
> might be confused with the literal case of documenting how code
> examples should themselves be documented.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  docs/devel/qapi-code-gen.rst | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
> index f276a2bc9bc..0f66f012be1 100644
> --- a/docs/devel/qapi-code-gen.rst
> +++ b/docs/devel/qapi-code-gen.rst
> @@ -1089,6 +1089,9 @@ Highlighting in non-QMP languages can be accomplished by using the
>  achieved by omitting the language argument.
>  
>  
> +Putting it all together
> +^^^^^^^^^^^^^^^^^^^^^^^
> +
>  Examples of complete definition documentation::
>  
>   ##

This adds another level of headings, level seven:

    Developer Information
        QEMU Build System
            How to use the QAPI code generator
                The QAPI schema language
                    Documentation comments
                        Definition documentation
                            Putting it all together

I'm not thrilled by this depth, to be honest.

Adding the heading makes next to no sense by itself.  However, you add
more headings next to it later in the series, and I believe that's why
you add this one now.  Recommend to mention this in your commit message.

The series' total change to the section structure:

    Introduction
    ============
    The QAPI schema language
    ========================
        Schema syntax
        -------------
        Built-in Types
        --------------
        Include directives
        ------------------
        Pragma directives
        -----------------
        Enumeration types
        -----------------
        Type references and array types
        -------------------------------
        Struct types
        ------------
        Union types
        -----------
        Alternate types
        ---------------
        Commands
        --------
        Events
        ------
        Features
        --------
            Special features
            ~~~~~~~~~~~~~~~~
        Naming rules and reserved names
        -------------------------------
        Downstream extensions
        ---------------------
        Configuring the schema
        ----------------------
        Documentation comments
        ----------------------
            Headings and subheadings
            ~~~~~~~~~~~~~~~~~~~~~~~~
            Documentation markup
            ~~~~~~~~~~~~~~~~~~~~
            Definition documentation
            ~~~~~~~~~~~~~~~~~~~~~~~~
   +            Intro
   +            ^^^^^
   +            Descriptions
   +            ^^^^^^^^^^^^
   +            Tagged sections
   +            ^^^^^^^^^^^^^^^
   +            Returns
   +            ^^^^^^^
   +            Errors
   +            ^^^^^^
   +            Feature Descriptions
   +            ^^^^^^^^^^^^^^^^^^^^
   +            Details
   +            ^^^^^^^
   +            Since
   +            ^^^^^
   +            TODO
   +            ^^^^
   +            Putting it all together
   +            ^^^^^^^^^^^^^^^^^^^^^^^
            Markup pitfalls
            ~~~~~~~~~~~~~~~
    Client JSON Protocol introspection
    ==================================
    Compatibility considerations
    ============================
    Code generation
    ===============
        Code generated for QAPI types
        -----------------------------
        Code generated for visiting QAPI types
        --------------------------------------
        Code generated for commands
        ---------------------------
        Code generated for events
        -------------------------
        Code generated for introspection
        --------------------------------

It's more deeply nested than I'd like, but to avoid that, we'd need a
more invasive restructuring.  Let's not go there now.

[PATCH 03/11] docs/qapi-code-gen: add "TODO" subsection

[John Snow]

Split the "TODO" section description into its own little section, and
add a dedicated example for it.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/devel/qapi-code-gen.rst | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index 0f66f012be1..977ed1aa3bd 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -1037,8 +1037,7 @@ detailing a relevant error condition.  For example::
 A "Since: x.y" tagged section lists the release that introduced the
 definition.
 
-"TODO" sections are not rendered (they are for developers, not users of
-QMP).  In other sections, the text is formatted, and rST markup can be
+In other sections, the text is formatted, and rST markup can be
 used.
 
 QMP Examples can be added by using the ``.. qmp-example::`` directive.
@@ -1089,6 +1088,17 @@ Highlighting in non-QMP languages can be accomplished by using the
 achieved by omitting the language argument.
 
 
+TODO
+^^^^
+
+"TODO" tagged sections are not rendered (they are for developers, not users of
+QMP). They can appear anywhere after the introductory section::
+
+  #
+  ## TODO: This is not rendered in the documentation.
+  #
+
+
 Putting it all together
 ^^^^^^^^^^^^^^^^^^^^^^^
 
-- 
2.55.0

Re: [PATCH 03/11] docs/qapi-code-gen: add "TODO" subsection

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> Split the "TODO" section description into its own little section, and
> add a dedicated example for it.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  docs/devel/qapi-code-gen.rst | 14 ++++++++++++--
>  1 file changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
> index 0f66f012be1..977ed1aa3bd 100644
> --- a/docs/devel/qapi-code-gen.rst
> +++ b/docs/devel/qapi-code-gen.rst
> @@ -1037,8 +1037,7 @@ detailing a relevant error condition.  For example::
>  A "Since: x.y" tagged section lists the release that introduced the
>  definition.
>  
> -"TODO" sections are not rendered (they are for developers, not users of
> -QMP).  In other sections, the text is formatted, and rST markup can be
> +In other sections, the text is formatted, and rST markup can be
>  used.
>  
>  QMP Examples can be added by using the ``.. qmp-example::`` directive.
> @@ -1089,6 +1088,17 @@ Highlighting in non-QMP languages can be accomplished by using the
>  achieved by omitting the language argument.
>  
>  
> +TODO
> +^^^^
> +
> +"TODO" tagged sections are not rendered (they are for developers, not users of

Wrap lines at colum 70, please.

> +QMP). They can appear anywhere after the introductory section::

Two spaces between sentences for consistency, please.

> +
> +  #
> +  ## TODO: This is not rendered in the documentation.

Why the second '#'?

> +  #
> +
> +
>  Putting it all together
>  ^^^^^^^^^^^^^^^^^^^^^^^

[PATCH 04/11] docs/qapi-code-gen: add "Since" subsection

[John Snow]

Add a standalone "Since:" documentation section, with usage example.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/devel/qapi-code-gen.rst | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index 977ed1aa3bd..fcb52ce157e 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -1034,8 +1034,6 @@ detailing a relevant error condition.  For example::
  #     - If @device does not exist, DeviceNotFound
  #     - Any other error returns a GenericError.
 
-A "Since: x.y" tagged section lists the release that introduced the
-definition.
 
 In other sections, the text is formatted, and rST markup can be
 used.
@@ -1088,6 +1086,17 @@ Highlighting in non-QMP languages can be accomplished by using the
 achieved by omitting the language argument.
 
 
+Since
+^^^^^
+
+A "Since: x.y" section lists the release that introduced the
+definition::
+
+  #
+  # Since: 11.2
+  #
+
+
 TODO
 ^^^^
 
-- 
2.55.0

[PATCH 05/11] docs/qapi-code-gen: add "Details" docs subsection

[John Snow]

Mostly, this is just adding a heading and explaining that arbitrary
rST formatting is allowed in this section.

Note: The QAPI parser still refers to these sections internally as
"PLAIN", but I wanted to name this documentation section after the
eventual, true name for the section. It is effectively only an
implementation detail at this point.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/devel/qapi-code-gen.rst | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index fcb52ce157e..8fc75488019 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -1035,8 +1035,15 @@ detailing a relevant error condition.  For example::
  #     - Any other error returns a GenericError.
 
 
-In other sections, the text is formatted, and rST markup can be
-used.
+Details
+^^^^^^^
+
+Details sections are comprised of one or more paragraphs of rST
+formatted text, but otherwise are not marked up with any QAPI-specific
+syntax. In general, this text contains in-depth explanations of the
+definition being documented and may include various rST admonitions
+like ``.. seealso::``, ``.. note::``, etc. This is also the section
+where QMP usage examples should be placed.
 
 QMP Examples can be added by using the ``.. qmp-example::`` directive.
 In its simplest form, this can be used to contain a single QMP code
-- 
2.55.0

Re: [PATCH 05/11] docs/qapi-code-gen: add "Details" docs subsection

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> Mostly, this is just adding a heading and explaining that arbitrary
> rST formatting is allowed in this section.
>
> Note: The QAPI parser still refers to these sections internally as
> "PLAIN",

Really?

>          but I wanted to name this documentation section after the
> eventual, true name for the section. It is effectively only an
> implementation detail at this point.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  docs/devel/qapi-code-gen.rst | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)
>
> diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
> index fcb52ce157e..8fc75488019 100644
> --- a/docs/devel/qapi-code-gen.rst
> +++ b/docs/devel/qapi-code-gen.rst
> @@ -1035,8 +1035,15 @@ detailing a relevant error condition.  For example::
>   #     - Any other error returns a GenericError.
>  
>  
> -In other sections, the text is formatted, and rST markup can be
> -used.
> +Details
> +^^^^^^^
> +
> +Details sections are comprised of one or more paragraphs of rST
> +formatted text, but otherwise are not marked up with any QAPI-specific
> +syntax.

Feels awkward.  You're trying to define you're talking about, but what
you're talking about is distinguished by absence of syntax, which makes
it harder to nail down.

I can't see how to do this better right now.  I'll come back to it.

>          In general, this text contains in-depth explanations of the
> +definition being documented and may include various rST admonitions
> +like ``.. seealso::``, ``.. note::``, etc. This is also the section
> +where QMP usage examples should be placed.
>  
>  QMP Examples can be added by using the ``.. qmp-example::`` directive.
>  In its simplest form, this can be used to contain a single QMP code

[PATCH 06/11] docs/qapi-code-gen: add "Errors" and "Returns" subsections

[John Snow]

This is a two-for-one as they are described together in one paragraph,
which I split apart into two sections.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/devel/qapi-code-gen.rst | 27 +++++++++++++++++++++++++--
 1 file changed, 25 insertions(+), 2 deletions(-)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index 8fc75488019..90488da7ffd 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -1024,8 +1024,31 @@ like this::
  #     Duis aute irure dolor in reprehenderit in voluptate velit esse
  #     cillum dolore eu fugiat nulla pariatur.
 
-"Returns" and "Errors" sections are only valid for commands.  They
-document the success and the error response, respectively.
+
+Returns
+^^^^^^^
+
+"Returns" sections are only valid for commands.  They document the
+success response of the command. If there is some nuance to capture
+with the return value, it may be documented as::
+
+  #
+  # Returns: a list describing each virtual block device.  Filter nodes
+  #     that were created implicitly are skipped over.
+  #
+
+Long descriptions are written as a block, with additional lines indented.
+
+The type of the return value will be added to the documentation
+automatically. If there is nothing relevant to add apart from type,
+this section may be omitted.
+
+
+Errors
+^^^^^^
+
+"Errors" sections are only valid for commands.  They document
+the error response(s) for a command.
 
 "Errors" sections should be formatted as an rST list, each entry
 detailing a relevant error condition.  For example::
-- 
2.55.0

[PATCH 07/11] docs/qapi-code-gen: add "Tagged sections" subsection

[John Snow]

The usefulness of naming and defining a "tagged section" is waning;
its only remaining use is to describe a shared syntax common to all
sections.

I think it's probably likely we want to omit this section entirely,
but here's our chance to discuss that.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/devel/qapi-code-gen.rst | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index 90488da7ffd..1897cde39cf 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -1011,6 +1011,12 @@ line "Features:", like this::
   #
   # @feature: Description text
 
+
+.. _doc-tagged:
+
+Tagged sections
+^^^^^^^^^^^^^^^
+
 A tagged section begins with a paragraph that starts with one of the
 following words: "Since:", "Returns:", "Errors:", "TODO:".  It ends with
 the start of a new section.
-- 
2.55.0

[PATCH 08/11] docs/qapi-code-gen: add "Descriptions" subsection

[John Snow]

Section off the part of the spec that describes descriptions.

Similarly to "Tagged sections", the concept of naming "Descriptions"
as a proper noun might also be something we'd like to sunset, but for
now I have retained the section. I believe the documentation itself is
entirely valid and useful, but we may want to come up with a term for
"arguments, members, branches, or values" as a group of concepts
instead - excising only "Features" as something that is different.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/devel/qapi-code-gen.rst | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index 1897cde39cf..7e7b11d01df 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -985,13 +985,23 @@ When documentation is required (see pragma_ 'doc-required'), every
 definition must have documentation.
 
 Definition documentation starts with a description naming the
-definition with an optional indented overview, a description of each
-argument (for commands and events), member (for structs and unions),
-branch (for alternates), or value (for enums), a description of each
-feature (if any), and finally optional tagged sections.
+definition with an optional indented overview, :ref:`descriptions
+<doc-descrip>` of each argument (for commands and events), member (for
+structs and unions), branch (for alternates), or value (for enums), a
+description of each feature (if any), and finally optional tagged
+sections.
 
-Descriptions start with '\@name:'.  The description text must be
-indented like this::
+
+.. _doc-descrip:
+
+Descriptions
+^^^^^^^^^^^^
+
+Descriptions start with '\@name:'.  These are used to document
+arguments (for commands and events), members (for structs and unions),
+branches (for alternates), or values (for enums).
+
+The description text must be indented like this::
 
  # @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed
  #     do eiusmod tempor incididunt ut labore et dolore magna aliqua.
-- 
2.55.0

Re: [PATCH 08/11] docs/qapi-code-gen: add "Descriptions" subsection

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> Section off the part of the spec that describes descriptions.
>
> Similarly to "Tagged sections", the concept of naming "Descriptions"
> as a proper noun might also be something we'd like to sunset, but for
> now I have retained the section. I believe the documentation itself is
> entirely valid and useful, but we may want to come up with a term for
> "arguments, members, branches, or values" as a group of concepts
> instead - excising only "Features" as something that is different.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  docs/devel/qapi-code-gen.rst | 22 ++++++++++++++++------
>  1 file changed, 16 insertions(+), 6 deletions(-)
>
> diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
> index 1897cde39cf..7e7b11d01df 100644
> --- a/docs/devel/qapi-code-gen.rst
> +++ b/docs/devel/qapi-code-gen.rst
> @@ -985,13 +985,23 @@ When documentation is required (see pragma_ 'doc-required'), every
>  definition must have documentation.
>  
>  Definition documentation starts with a description naming the
> -definition with an optional indented overview, a description of each
> -argument (for commands and events), member (for structs and unions),
> -branch (for alternates), or value (for enums), a description of each
> -feature (if any), and finally optional tagged sections.
> +definition with an optional indented overview, :ref:`descriptions
> +<doc-descrip>` of each argument (for commands and events), member (for
> +structs and unions), branch (for alternates), or value (for enums), a
> +description of each feature (if any), and finally optional tagged
> +sections.

Not this patch's fault: this isn't quite right.  The order of tagged
sections is left unspecified, which is fine now, but becomes a gap when
your "qapi: enforce section ordering" series lands.  "Details" sections
aren't mentioned at all.

>  
> -Descriptions start with '\@name:'.  The description text must be
> -indented like this::
> +
> +.. _doc-descrip:
> +
> +Descriptions
> +^^^^^^^^^^^^
> +
> +Descriptions start with '\@name:'.  These are used to document
> +arguments (for commands and events), members (for structs and unions),
> +branches (for alternates), or values (for enums).
> +
> +The description text must be indented like this::
>  
>   # @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed
>   #     do eiusmod tempor incididunt ut labore et dolore magna aliqua.

[PATCH 09/11] docs/qapi-code-gen: add "Feature descriptions" subsection

[John Snow]

Move the features text down to where it would appear in-order in a
QAPI documentation block, and expand it into its own subsection
separate from member/arg/etc descriptions.

This draft refers back to the "Descriptions" section to make it clear
that they are syntactically related, but has otherwise broken the
section out into its own discrete thing to make it clear that it is an
entirely separate section (group of sections).

The key element here is that Descriptions and Feature Descriptions
have been split apart so that they can each occupy their correct
relative position in the docs that mirrors their appearance in actual
documentation.

(i.e. members - returns - errors - features, in that order.)

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/devel/qapi-code-gen.rst | 32 +++++++++++++++++++++-----------
 1 file changed, 21 insertions(+), 11 deletions(-)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index 7e7b11d01df..febe8af49f9 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -987,9 +987,10 @@ definition must have documentation.
 Definition documentation starts with a description naming the
 definition with an optional indented overview, :ref:`descriptions
 <doc-descrip>` of each argument (for commands and events), member (for
-structs and unions), branch (for alternates), or value (for enums), a
-description of each feature (if any), and finally optional tagged
-sections.
+structs and unions), branch (for alternates), or value (for enums),
+:ref:`feature descriptions <doc-feat>` of each feature (if any), and
+finally optional tagged sections.
+
 
 
 .. _doc-descrip:
@@ -1013,14 +1014,6 @@ The description text must be indented like this::
 Extensions added after the definition was first released carry a
 "(since x.y)" comment.
 
-The feature descriptions must be preceded by a blank line and then a
-line "Features:", like this::
-
-  #
-  # Features:
-  #
-  # @feature: Description text
-
 
 .. _doc-tagged:
 
@@ -1074,6 +1067,23 @@ detailing a relevant error condition.  For example::
  #     - Any other error returns a GenericError.
 
 
+.. _doc-feat:
+
+Feature Descriptions
+^^^^^^^^^^^^^^^^^^^^
+
+Feature descriptions are a :ref:`doc-descrip` section that document features
+that apply to this definition.
+
+The feature descriptions must be preceded by a blank line and then a
+line "Features:", like this::
+
+  #
+  # Features:
+  #
+  # @feature: Description text
+
+
 Details
 ^^^^^^^
 
-- 
2.55.0

[PATCH 10/11] docs/qapi-code-gen: add "Intro" subsection

[John Snow]

Give the introductory section its own small treatment.

Without going into details about the as-yet-unmerged inliner, I
struggle to describe precisely how the intro should be used and what
types of documentation are better suited for Details. This is my
attempt.

I find PEP 257 https://peps.python.org/pep-0257/#one-line-docstrings
to have very good advice on what should appear in a doc summary line,
but did not want to reference it directly in our specification.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/devel/qapi-code-gen.rst | 27 ++++++++++++++++++++++-----
 1 file changed, 22 insertions(+), 5 deletions(-)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index febe8af49f9..4932dd35b64 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -985,13 +985,30 @@ When documentation is required (see pragma_ 'doc-required'), every
 definition must have documentation.
 
 Definition documentation starts with a description naming the
-definition with an optional indented overview, :ref:`descriptions
-<doc-descrip>` of each argument (for commands and events), member (for
-structs and unions), branch (for alternates), or value (for enums),
-:ref:`feature descriptions <doc-feat>` of each feature (if any), and
-finally optional tagged sections.
+definition with an optional indented overview (the :ref:`doc-intro`),
+:ref:`descriptions <doc-descrip>` of each argument (for commands and
+events), member (for structs and unions), branch (for alternates), or
+value (for enums), :ref:`feature descriptions <doc-feat>` of each
+feature (if any), and finally optional tagged sections.
 
 
+.. _doc-intro:
+
+Intro
+^^^^^
+
+The Intro section names the symbol being defined and is optionally
+followed by an indented brief summary or overview of the definition.
+
+The intro section *may* consist of multiple indented paragraphs, but
+is ideally a single summary line that does not repeat type
+information or reference individual arguments/members/et al::
+
+  ##
+  # @query-block-jobs:
+  #     Return information about long-running block device operations.
+  #
+
 
 .. _doc-descrip:
 
-- 
2.55.0

[PATCH 11/11] docs/qapi-code-gen: add "Tagged sections" cross-references

[John Snow]

Add some XRef markup that links to the "tagged section" subsection. I
don't know if we want to keep that section, so these cross-reference
additions are isolated here.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/devel/qapi-code-gen.rst | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index 4932dd35b64..ff7663a2666 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -989,7 +989,8 @@ definition with an optional indented overview (the :ref:`doc-intro`),
 :ref:`descriptions <doc-descrip>` of each argument (for commands and
 events), member (for structs and unions), branch (for alternates), or
 value (for enums), :ref:`feature descriptions <doc-feat>` of each
-feature (if any), and finally optional tagged sections.
+feature (if any), and finally optional :ref:`tagged sections
+<doc-tagged>`.
 
 
 .. _doc-intro:
@@ -1054,7 +1055,7 @@ like this::
 Returns
 ^^^^^^^
 
-"Returns" sections are only valid for commands.  They document the
+"Returns" :ref:`doc-tagged` are only valid for commands.  They document the
 success response of the command. If there is some nuance to capture
 with the return value, it may be documented as::
 
@@ -1073,8 +1074,8 @@ this section may be omitted.
 Errors
 ^^^^^^
 
-"Errors" sections are only valid for commands.  They document
-the error response(s) for a command.
+"Errors" :ref:`doc-tagged` are only valid for commands.  They document
+the error responses.
 
 "Errors" sections should be formatted as an rST list, each entry
 detailing a relevant error condition.  For example::
@@ -1162,7 +1163,7 @@ achieved by omitting the language argument.
 Since
 ^^^^^
 
-A "Since: x.y" section lists the release that introduced the
+A "Since: x.y" :ref:`doc-tagged` lists the release that introduced the
 definition::
 
   #
-- 
2.55.0

Re: [PATCH 00/11] docs/qapi: revise and expand QAPI documentation spec

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> This patchset looks at expanding and clarifying the documentation for
> ... writing documentation. Primarily, the doc is being organized into
> subsections - one for each QAPI doc section. These sections are
> re-ordered to be documented in the same order as they are expected to
> appear in-usage.

While I appreciate the split into single-purpose patches, I still need
to review the entire doc update as a whole.

   Definition documentation
   ~~~~~~~~~~~~~~~~~~~~~~~~

   Definition documentation, if present, must immediately precede the
   definition it documents.

   When documentation is required (see pragma_ 'doc-required'), every
   definition must have documentation.

   Definition documentation starts with a description naming the
  -definition with an optional indented overview, a description of each
  -argument (for commands and events), member (for structs and unions),
  -branch (for alternates), or value (for enums), a description of each
  -feature (if any), and finally optional tagged sections.
  +definition with an optional indented overview (the :ref:`doc-intro`),
  +:ref:`descriptions <doc-descrip>` of each argument (for commands and
  +events), member (for structs and unions), branch (for alternates), or
  +value (for enums), :ref:`feature descriptions <doc-feat>` of each
  +feature (if any), and finally optional :ref:`tagged sections
  +<doc-tagged>`.

This needs work.  Copying from my review of PATCH 07...  The order of
tagged sections is left unspecified, which is fine now, but becomes a
gap when your "qapi: enforce section ordering" series lands.  "Details"
sections aren't mentioned at all.

Try a list?  Or do away with the overview entirely?  You do describe the
definition documentation contents in the required order below, which
could be enough.

  -Descriptions start with '\@name:'.  The description text must be
  -indented like this::
  +
  +.. _doc-intro:
  +
  +Intro
  +^^^^^
  +
  +The Intro section names the symbol being defined and is optionally
  +followed by an indented brief summary or overview of the definition.
  +
  +The intro section *may* consist of multiple indented paragraphs, but
  +is ideally a single summary line that does not repeat type
  +information or reference individual arguments/members/et al::

et al.

Perhaps "does not repeat type information and does not contain forward
references, say to command arguments".

  +
  +  ##
  +  # @query-block-jobs:
  +  #     Return information about long-running block device operations.
  +  #
  +
  +
  +.. _doc-descrip:
  +
  +Descriptions
  +^^^^^^^^^^^^

Would be nice to have a better term than "descriptions".  "Argument /
member / branch / value descriptions" is rather long.

  +
  +Descriptions start with '\@name:'.  These are used to document
  +arguments (for commands and events), members (for structs and unions),
  +branches (for alternates), or values (for enums).
  +
  +The description text must be indented like this::

    # @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed
    #     do eiusmod tempor incididunt ut labore et dolore magna aliqua.

   .. FIXME The parser accepts these things in almost any order.

   .. FIXME union branches should be described, too.

   Extensions added after the definition was first released carry a
  -"(since x.y.z)" comment.
  +"(since x.y)" comment.

  -The feature descriptions must be preceded by a blank line and then a
  -line "Features:", like this::

  -  #
  -  # Features:
  -  #
  -  # @feature: Description text
  +.. _doc-tagged:
  +
  +Tagged sections
  +^^^^^^^^^^^^^^^

   A tagged section begins with a paragraph that starts with one of the
   following words: "Since:", "Returns:", "Errors:", "TODO:".  It ends with
   the start of a new section.

   The second and subsequent lines of tagged sections must be indented
   like this::

    # TODO: Ut enim ad minim veniam, quis nostrud exercitation ullamco
    #     laboris nisi ut aliquip ex ea commodo consequat.
    #
    #     Duis aute irure dolor in reprehenderit in voluptate velit esse
    #     cillum dolore eu fugiat nulla pariatur.

This factors out the common syntax (good) at the price of breaking the
linear sequence of sections (not so good).  Hmm.

  -"Returns" and "Errors" sections are only valid for commands.  They
  -document the success and the error response, respectively.
  +
  +Returns
  +^^^^^^^
  +
  +"Returns" :ref:`doc-tagged` are only valid for commands.  They document the

This comes out as

    “Returns” Tagged sections are only valid for commands. They document the

"Tagged" should not be capitalized.

  +success response of the command. If there is some nuance to capture
  +with the return value, it may be documented as::
  +
  +  #
  +  # Returns: a list describing each virtual block device.  Filter nodes
  +  #     that were created implicitly are skipped over.
  +  #
  +
  +Long descriptions are written as a block, with additional lines indented.

Redundant with the common syntax under "Tagged sections".  We need to
make up our mind whether to factor out syntax or not.

  +
  +The type of the return value will be added to the documentation
  +automatically. If there is nothing relevant to add apart from type,
  +this section may be omitted.

Maybe do this first, and only then a suitable rephrasing of "If there is
some nuance ..."

  +
  +
  +Errors
  +^^^^^^
  +
  +"Errors" :ref:`doc-tagged` are only valid for commands.  They document
  +the error responses.

   "Errors" sections should be formatted as an rST list, each entry
   detailing a relevant error condition.  For example::

    # Errors:
    #     - If @device does not exist, DeviceNotFound
    #     - Any other error returns a GenericError.

Problematic example copied from block-commit.  Tidied up there in commit
b3bd7617dba.  Better steal something else.

Hmm, many existing Errors: sections don't mention the error class.  E.g.
nbd-server-add:

    # Errors:
    #     - if the server is not running
    #     - if an export with the same name already exists

Should we document this as acceptable?

  -A "Since: x.y.z" tagged section lists the release that introduced the
  -definition.

  -"TODO" sections are not rendered (they are for developers, not users of
  -QMP).  In other sections, the text is formatted, and rST markup can be
  -used.
  +.. _doc-feat:
  +
  +Feature Descriptions
  +^^^^^^^^^^^^^^^^^^^^
  +
  +Feature descriptions are a :ref:`doc-descrip` section that document features
  +that apply to this definition.
  +
  +The feature descriptions must be preceded by a blank line and then a
  +line "Features:", like this::
  +
  +  #
  +  # Features:
  +  #
  +  # @feature: Description text

Perhaps something like

   Feature descriptions document the features that apply to this
   definition.  They start with a line "Features:" followed by one
   description per feature in the same syntax as argument / member /
   branch / value descriptions, like this::

  +
  +
  +Details
  +^^^^^^^
  +
  +Details sections are comprised of one or more paragraphs of rST
  +formatted text, but otherwise are not marked up with any QAPI-specific
  +syntax. In general, this text contains in-depth explanations of the
  +definition being documented and may include various rST admonitions
  +like ``.. seealso::``, ``.. note::``, etc. This is also the section
  +where QMP usage examples should be placed.

Is the section concept helpful here?

Internally, it's a single section unless split by TODO sections.
Whether and how it's split shouldn't affect the output at all.

Users don't care for all that.  For them, it's just paragraphs.

   QMP Examples can be added by using the ``.. qmp-example::`` directive.
   In its simplest form, this can be used to contain a single QMP code
   block which accepts standard JSON syntax with additional server
   directionality indicators (``->`` and ``<-``), and elisions.  An
   elision is commonly ``...``, but it can also be or a pair of ``...``
   with text in between.

   Optionally, a plaintext title may be provided by using the ``:title:``
   directive option.  If the title is omitted, the example title will
   default to "Example:".

   A simple QMP example::

     # .. qmp-example::
     #
     #     -> { "execute": "query-name" }
     #     <- { "return": { "name": "Fred" } }

   More complex or multi-step examples where exposition is needed before
   or between QMP code blocks can be created by using the ``:annotated:``
   directive option.  When using this option, nested QMP code blocks must
   be entered explicitly with rST's ``::`` syntax.

   For example::

     # .. qmp-example::
     #    :annotated:
     #    :title: A more complex demonstration
     #
     #    This is a more complex example that can use
     #    ``arbitrary rST syntax`` in its exposition::
     #
     #     -> { "execute": "query-block" }
     #     <- { "return": [
     #             {
     #               "device": "ide0-hd0",
     #               ...
     #             }
     #             ... more ...
     #          ] }
     #
     #    Above, lengthy output has been omitted for brevity.

   Highlighting in non-QMP languages can be accomplished by using the
   ``.. code-block:: lang`` directive, and non-highlighted text can be
   achieved by omitting the language argument.


  +Since
  +^^^^^
  +
  +A "Since: x.y" :ref:`doc-tagged` lists the release that introduced the
  +definition::
  +
  +  #
  +  # Since: 11.2
  +  #
  +
  +
  +TODO
  +^^^^
  +
  +"TODO" tagged sections are not rendered (they are for developers, not users of
  +QMP). They can appear anywhere after the introductory section::
  +
  +  #
  +  ## TODO: This is not rendered in the documentation.
  +  #
  +
  +
  +Putting it all together
  +^^^^^^^^^^^^^^^^^^^^^^^
  +
   Examples of complete definition documentation::

    ##
    # @BlockStats:
    #     Statistics of a virtual block device or a block backing device.
    #
    # @device: If the stats are for a virtual block device, the name
    #     corresponding to the virtual block device.
    #
    # @node-name: The node name of the device.  (Since 2.3)
    #
    # ... more members ...
    #
    # Since: 0.14
    ##
    { 'struct': 'BlockStats',
      'data': {'*device': 'str', '*node-name': 'str',
               ... more members ... } }

    ##
    # @query-blockstats:
    #     Query the @BlockStats for all virtual block devices.
    #
    # @query-nodes: If true, the command will query all the block nodes
    #     ... explain, explain ...
    #     (Since 2.3)
    #
    # Returns: A list of @BlockStats for each virtual block devices.
    #
    # Since: 0.14
    #
    # .. qmp-example::
    #
    #     -> { "execute": "query-blockstats" }
    #     <- {
    #          ...
    #        }
    ##
    { 'command': 'query-blockstats',
      'data': { '*query-nodes': 'bool' },
      'returns': ['BlockStats'] }


   Markup pitfalls
   ~~~~~~~~~~~~~~~