[PULL 0/5] QAPI patches patches for 2023-05-17

[PULL 0/5] QAPI patches patches for 2023-05-17

[Markus Armbruster]

The following changes since commit f9d58e0ca53b3f470b84725a7b5e47fcf446a2ea:

  Merge tag 'pull-9p-20230516' of https://github.com/cschoenebeck/qemu into staging (2023-05-16 10:21:44 -0700)

are available in the Git repository at:

  https://repo.or.cz/qemu/armbru.git tags/pull-qapi-2023-05-17

for you to fetch changes up to 28d2a4f620db2bc327dbd3443b777890d39a0bf6:

  qapi/parser: Drop two bad type hints for now (2023-05-17 15:49:22 +0200)

----------------------------------------------------------------
QAPI patches patches for 2023-05-17

----------------------------------------------------------------
Markus Armbruster (2):
      qapi: Improve error message for description following section
      qapi/parser: Drop two bad type hints for now

Peter Maydell (3):
      docs/interop: Convert qmp-spec.txt to rST
      docs/interop/qmp-spec: Update error description for parsing errors
      docs/interop: Delete qmp-intro.txt

 docs/devel/qapi-code-gen.rst                  |   3 +-
 docs/interop/index.rst                        |   1 +
 docs/interop/qmp-intro.txt                    |  88 -------
 docs/interop/{qmp-spec.txt => qmp-spec.rst}   | 337 ++++++++++++++------------
 qapi/control.json                             |   4 +-
 qapi/qapi-schema.json                         |   3 +-
 qobject/json-lexer.c                          |   2 +-
 python/qemu/qmp/models.py                     |   8 +-
 python/qemu/qmp/qmp_client.py                 |   4 +-
 qemu-options.hx                               |  28 ++-
 scripts/qapi/parser.py                        |   8 +-
 tests/qapi-schema/doc-interleaved-section.err |   2 +-
 12 files changed, 226 insertions(+), 262 deletions(-)
 delete mode 100644 docs/interop/qmp-intro.txt
 rename docs/interop/{qmp-spec.txt => qmp-spec.rst} (55%)

-- 
2.39.2

[PULL 1/5] qapi: Improve error message for description following section

[Markus Armbruster]

The error message is bad when the section is untagged.  For instance,
test case doc-interleaved-section produces "'@foobar:' can't follow
'Note' section", which is okay, but if we drop the "Note:" tag, we get
"'@foobar:' can't follow 'None' section, which is bad.

Change the error message to "description of '@foobar:' follows a
section".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230510141637.3685080-1-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
[Conflict with commit 3e32dca3f0d resolved]
---
 scripts/qapi/parser.py                        | 4 ++--
 tests/qapi-schema/doc-interleaved-section.err | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 4923a59d60..acd43cd7e1 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -675,8 +675,8 @@ def _append_various_line(self, line: str) -> None:
         match = self._match_at_name_colon(line)
         if match:
             raise QAPIParseError(self._parser,
-                                 "'@%s:' can't follow '%s' section"
-                                 % (match.group(1), self.sections[0].name))
+                                 "description of '@%s:' follows a section"
+                                 % match.group(1))
         match = self._match_section_tag(line)
         if match:
             line = line[match.end():]
diff --git a/tests/qapi-schema/doc-interleaved-section.err b/tests/qapi-schema/doc-interleaved-section.err
index 715d58cd31..e5d1ef54c1 100644
--- a/tests/qapi-schema/doc-interleaved-section.err
+++ b/tests/qapi-schema/doc-interleaved-section.err
@@ -1 +1 @@
-doc-interleaved-section.json:15:1: '@foobar:' can't follow 'Note' section
+doc-interleaved-section.json:15:1: description of '@foobar:' follows a section
-- 
2.39.2

[PATCH] qapi/parser: Drop two bad type hints for now

[Markus Armbruster]

Two type hints fail centos-stream-8-x86_64 CI.  They are actually
broken.  Changing them to Optional[re.Match[str]] fixes them locally
for me, but then CI fails differently.  Drop them for now.

Fixes: 3e32dca3f0d1 (qapi: Rewrite parsing of doc comment section symbols and tags)
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 scripts/qapi/parser.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 4923a59d60..1ff334e6a8 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -563,11 +563,11 @@ def end_comment(self) -> None:
         self._switch_section(QAPIDoc.NullSection(self._parser))
 
     @staticmethod
-    def _match_at_name_colon(string: str) -> re.Match:
+    def _match_at_name_colon(string: str):
         return re.match(r'@([^:]*): *', string)
 
     @staticmethod
-    def _match_section_tag(string: str) -> re.Match:
+    def _match_section_tag(string: str):
         return re.match(r'(Returns|Since|Notes?|Examples?|TODO): *', string)
 
     def _append_body_line(self, line: str) -> None:
-- 
2.39.2

[PULL 2/5] docs/interop: Convert qmp-spec.txt to rST

[Markus Armbruster]

From: Peter Maydell <peter.maydell@linaro.org>

Convert the qmp-spec.txt document to restructuredText.
Notable points about the conversion:
 * numbers at the start of section headings are removed, to match
   the style of the rest of the manual
 * cross-references to other sections or documents are hyperlinked
 * various formatting tweaks (notably the examples, which need the
   -> and <- prefixed so the QMP code-block lexer will accept them)
 * English prose fixed in a few places

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20230515162245.3964307-2-peter.maydell@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 docs/devel/qapi-code-gen.rst                |   3 +-
 docs/interop/index.rst                      |   1 +
 docs/interop/{qmp-spec.txt => qmp-spec.rst} | 337 +++++++++++---------
 qapi/control.json                           |   4 +-
 qapi/qapi-schema.json                       |   3 +-
 qobject/json-lexer.c                        |   2 +-
 python/qemu/qmp/models.py                   |   8 +-
 python/qemu/qmp/qmp_client.py               |   4 +-
 8 files changed, 199 insertions(+), 163 deletions(-)
 rename docs/interop/{qmp-spec.txt => qmp-spec.rst} (55%)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index 5618a80378..7f78183cd4 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -545,7 +545,8 @@ Member 'allow-oob' declares whether the command supports out-of-band
  { 'command': 'migrate_recover',
    'data': { 'uri': 'str' }, 'allow-oob': true }
 
-See qmp-spec.txt for out-of-band execution syntax and semantics.
+See the :doc:`/interop/qmp-spec` for out-of-band execution syntax
+and semantics.
 
 Commands supporting out-of-band execution can still be executed
 in-band.
diff --git a/docs/interop/index.rst b/docs/interop/index.rst
index 6351ff9ba6..ed65395bfb 100644
--- a/docs/interop/index.rst
+++ b/docs/interop/index.rst
@@ -15,6 +15,7 @@ are useful for making QEMU interoperate with other software.
    dbus-display
    live-block-operations
    pr-helper
+   qmp-spec
    qemu-ga
    qemu-ga-ref
    qemu-qmp-ref
diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.rst
similarity index 55%
rename from docs/interop/qmp-spec.txt
rename to docs/interop/qmp-spec.rst
index b0e8351d5b..bfad570a16 100644
--- a/docs/interop/qmp-spec.txt
+++ b/docs/interop/qmp-spec.rst
@@ -1,24 +1,26 @@
-                      QEMU Machine Protocol Specification
+..
+    Copyright (C) 2009-2016 Red Hat, Inc.
 
-0. About This Document
-======================
+    This work is licensed under the terms of the GNU GPL, version 2 or
+    later. See the COPYING file in the top-level directory.
 
-Copyright (C) 2009-2016 Red Hat, Inc.
 
-This work is licensed under the terms of the GNU GPL, version 2 or
-later. See the COPYING file in the top-level directory.
+===================================
+QEMU Machine Protocol Specification
+===================================
 
-1. Introduction
-===============
-
-This document specifies the QEMU Machine Protocol (QMP), a JSON-based
+The QEMU Machine Protocol (QMP) is a JSON-based
 protocol which is available for applications to operate QEMU at the
 machine-level.  It is also in use by the QEMU Guest Agent (QGA), which
 is available for host applications to interact with the guest
-operating system.
+operating system. This page specifies the general format of
+the protocol; details of the commands and data structures can
+be found in the :doc:`qemu-qmp-ref` and the :doc:`qemu-ga-ref`.
 
-2. Protocol Specification
-=========================
+.. contents::
+
+Protocol Specification
+======================
 
 This section details the protocol format. For the purpose of this
 document, "Server" is either QEMU or the QEMU Guest Agent, and
@@ -30,9 +32,7 @@ following format:
     json-DATA-STRUCTURE-NAME
 
 Where DATA-STRUCTURE-NAME is any valid JSON data structure, as defined
-by the JSON standard:
-
-http://www.ietf.org/rfc/rfc8259.txt
+by the `JSON standard <http://www.ietf.org/rfc/rfc8259.txt>`_.
 
 The server expects its input to be encoded in UTF-8, and sends its
 output encoded in ASCII.
@@ -45,83 +45,89 @@ important unless specifically documented otherwise.  Repeating a key
 within a json-object gives unpredictable results.
 
 Also for convenience, the server will accept an extension of
-'single-quoted' strings in place of the usual "double-quoted"
+``'single-quoted'`` strings in place of the usual ``"double-quoted"``
 json-string, and both input forms of strings understand an additional
-escape sequence of "\'" for a single quote. The server will only use
+escape sequence of ``\'`` for a single quote. The server will only use
 double quoting on output.
 
-2.1 General Definitions
------------------------
-
-2.1.1 All interactions transmitted by the Server are json-objects, always
-      terminating with CRLF
-
-2.1.2 All json-objects members are mandatory when not specified otherwise
-
-2.2 Server Greeting
+General Definitions
 -------------------
 
+All interactions transmitted by the Server are json-objects, always
+terminating with CRLF.
+
+All json-objects members are mandatory when not specified otherwise.
+
+Server Greeting
+---------------
+
 Right when connected the Server will issue a greeting message, which signals
 that the connection has been successfully established and that the Server is
 ready for capabilities negotiation (for more information refer to section
-'4. Capabilities Negotiation').
+`Capabilities Negotiation`_).
 
 The greeting message format is:
 
-{ "QMP": { "version": json-object, "capabilities": json-array } }
+.. code-block::
 
- Where,
+  { "QMP": { "version": json-object, "capabilities": json-array } }
 
-- The "version" member contains the Server's version information (the format
-  is the same of the query-version command)
-- The "capabilities" member specify the availability of features beyond the
+Where:
+
+- The ``version`` member contains the Server's version information (the format
+  is the same as for the query-version command).
+- The ``capabilities`` member specifies the availability of features beyond the
   baseline specification; the order of elements in this array has no
   particular significance.
 
-2.2.1 Capabilities
-------------------
+Capabilities
+------------
 
 Currently supported capabilities are:
 
-- "oob": the QMP server supports "out-of-band" (OOB) command
-  execution, as described in section "2.3.1 Out-of-band execution".
+``oob``
+  the QMP server supports "out-of-band" (OOB) command
+  execution, as described in section `Out-of-band execution`_.
 
-2.3 Issuing Commands
---------------------
+Issuing Commands
+----------------
 
 The format for command execution is:
 
-{ "execute": json-string, "arguments": json-object, "id": json-value }
+.. code-block::
+
+  { "execute": json-string, "arguments": json-object, "id": json-value }
 
 or
 
-{ "exec-oob": json-string, "arguments": json-object, "id": json-value }
+.. code-block::
 
- Where,
+  { "exec-oob": json-string, "arguments": json-object, "id": json-value }
 
-- The "execute" or "exec-oob" member identifies the command to be
+Where:
+
+- The ``execute`` or ``exec-oob`` member identifies the command to be
   executed by the server.  The latter requests out-of-band execution.
-- The "arguments" member is used to pass any arguments required for the
+- The ``arguments`` member is used to pass any arguments required for the
   execution of the command, it is optional when no arguments are
   required. Each command documents what contents will be considered
-  valid when handling the json-argument
-- The "id" member is a transaction identification associated with the
+  valid when handling the json-argument.
+- The ``id`` member is a transaction identification associated with the
   command execution, it is optional and will be part of the response
-  if provided.  The "id" member can be any json-value.  A json-number
+  if provided.  The ``id`` member can be any json-value.  A json-number
   incremented for each successive command works fine.
 
-The actual commands are documented in the QEMU QMP reference manual
-docs/interop/qemu-qmp-ref.{7,html,info,pdf,txt}.
+The actual commands are documented in the :doc:`qemu-qmp-ref`.
 
-2.3.1 Out-of-band execution
----------------------------
+Out-of-band execution
+---------------------
 
 The server normally reads, executes and responds to one command after
 the other.  The client therefore receives command responses in issue
 order.
 
-With out-of-band execution enabled via capability negotiation (section
-4.), the server reads and queues commands as they arrive.  It executes
+With out-of-band execution enabled via `capabilities negotiation`_,
+the server reads and queues commands as they arrive.  It executes
 commands from the queue one after the other.  Commands executed
 out-of-band jump the queue: the command get executed right away,
 possibly overtaking prior in-band commands.  The client may therefore
@@ -129,8 +135,8 @@ receive such a command's response before responses from prior in-band
 commands.
 
 To be able to match responses back to their commands, the client needs
-to pass "id" with out-of-band commands.  Passing it with all commands
-is recommended for clients that accept capability "oob".
+to pass ``id`` with out-of-band commands.  Passing it with all commands
+is recommended for clients that accept capability ``oob``.
 
 If the client sends in-band commands faster than the server can
 execute them, the server will stop reading requests until the request
@@ -140,57 +146,61 @@ To ensure commands to be executed out-of-band get read and executed,
 the client should have at most eight in-band commands in flight.
 
 Only a few commands support out-of-band execution.  The ones that do
-have "allow-oob": true in output of query-qmp-schema.
+have ``"allow-oob": true`` in the output of ``query-qmp-schema``.
 
-2.4 Commands Responses
-----------------------
+Commands Responses
+------------------
 
 There are two possible responses which the Server will issue as the result
 of a command execution: success or error.
 
-As long as the commands were issued with a proper "id" field, then the
-same "id" field will be attached in the corresponding response message
+As long as the commands were issued with a proper ``id`` field, then the
+same ``id`` field will be attached in the corresponding response message
 so that requests and responses can match.  Clients should drop all the
-responses that have an unknown "id" field.
+responses that have an unknown ``id`` field.
 
-2.4.1 success
--------------
+Success
+-------
 
 The format of a success response is:
 
-{ "return": json-value, "id": json-value }
+.. code-block::
 
- Where,
+  { "return": json-value, "id": json-value }
 
-- The "return" member contains the data returned by the command, which
+Where:
+
+- The ``return`` member contains the data returned by the command, which
   is defined on a per-command basis (usually a json-object or
   json-array of json-objects, but sometimes a json-number, json-string,
   or json-array of json-strings); it is an empty json-object if the
-  command does not return data
-- The "id" member contains the transaction identification associated
-  with the command execution if issued by the Client
+  command does not return data.
+- The ``id`` member contains the transaction identification associated
+  with the command execution if issued by the Client.
 
-2.4.2 error
------------
+Error
+-----
 
 The format of an error response is:
 
-{ "error": { "class": json-string, "desc": json-string }, "id": json-value }
+.. code-block::
 
- Where,
+  { "error": { "class": json-string, "desc": json-string }, "id": json-value }
 
-- The "class" member contains the error class name (eg. "GenericError")
-- The "desc" member is a human-readable error message. Clients should
+Where:
+
+- The ``class`` member contains the error class name (eg. ``"GenericError"``).
+- The ``desc`` member is a human-readable error message. Clients should
   not attempt to parse this message.
-- The "id" member contains the transaction identification associated with
-  the command execution if issued by the Client
+- The ``id`` member contains the transaction identification associated with
+  the command execution if issued by the Client.
 
-NOTE: Some errors can occur before the Server is able to read the "id" member,
-in these cases the "id" member will not be part of the error response, even
+NOTE: Some errors can occur before the Server is able to read the ``id`` member;
+in these cases the ``id`` member will not be part of the error response, even
 if provided by the client.
 
-2.5 Asynchronous events
------------------------
+Asynchronous events
+-------------------
 
 As a result of state changes, the Server may send messages unilaterally
 to the Client at any time, when not in the middle of any other
@@ -198,44 +208,45 @@ response. They are called "asynchronous events".
 
 The format of asynchronous events is:
 
-{ "event": json-string, "data": json-object,
-  "timestamp": { "seconds": json-number, "microseconds": json-number } }
+.. code-block::
 
- Where,
+  { "event": json-string, "data": json-object,
+    "timestamp": { "seconds": json-number, "microseconds": json-number } }
 
-- The "event" member contains the event's name
-- The "data" member contains event specific data, which is defined in a
-  per-event basis, it is optional
-- The "timestamp" member contains the exact time of when the event
+Where:
+
+- The ``event`` member contains the event's name.
+- The ``data`` member contains event specific data, which is defined in a
+  per-event basis. It is optional.
+- The ``timestamp`` member contains the exact time of when the event
   occurred in the Server. It is a fixed json-object with time in
   seconds and microseconds relative to the Unix Epoch (1 Jan 1970); if
   there is a failure to retrieve host time, both members of the
   timestamp will be set to -1.
 
-The actual asynchronous events are documented in the QEMU QMP
-reference manual docs/interop/qemu-qmp-ref.{7,html,info,pdf,txt}.
+The actual asynchronous events are documented in the :doc:`qemu-qmp-ref`.
 
 Some events are rate-limited to at most one per second.  If additional
 "similar" events arrive within one second, all but the last one are
 dropped, and the last one is delayed.  "Similar" normally means same
 event type.
 
-2.6 Forcing the JSON parser into known-good state
--------------------------------------------------
+Forcing the JSON parser into known-good state
+---------------------------------------------
 
 Incomplete or invalid input can leave the server's JSON parser in a
 state where it can't parse additional commands.  To get it back into
 known-good state, the client should provoke a lexical error.
 
 The cleanest way to do that is sending an ASCII control character
-other than '\t' (horizontal tab), '\r' (carriage return), or '\n' (new
-line).
+other than ``\t`` (horizontal tab), ``\r`` (carriage return), or
+``\n`` (new line).
 
 Sadly, older versions of QEMU can fail to flag this as an error.  If a
 client needs to deal with them, it should send a 0xFF byte.
 
-2.7 QGA Synchronization
------------------------
+QGA Synchronization
+-------------------
 
 When a client connects to QGA over a transport lacking proper
 connection semantics such as virtio-serial, QGA may have read partial
@@ -243,86 +254,106 @@ input from a previous client.  The client needs to force QGA's parser
 into known-good state using the previous section's technique.
 Moreover, the client may receive output a previous client didn't read.
 To help with skipping that output, QGA provides the
-'guest-sync-delimited' command.  Refer to its documentation for
+``guest-sync-delimited`` command.  Refer to its documentation for
 details.
 
 
-3. QMP Examples
-===============
+QMP Examples
+============
 
 This section provides some examples of real QMP usage, in all of them
-"C" stands for "Client" and "S" stands for "Server".
+``->`` marks text sent by the Client and ``<-`` marks replies by the Server.
 
-3.1 Server greeting
--------------------
+.. admonition:: Example
 
-S: { "QMP": {"version": {"qemu": {"micro": 0, "minor": 0, "major": 3},
-     "package": "v3.0.0"}, "capabilities": ["oob"] } }
+  Server greeting
 
-3.2 Capabilities negotiation
-----------------------------
+  .. code-block:: QMP
 
-C: { "execute": "qmp_capabilities", "arguments": { "enable": ["oob"] } }
-S: { "return": {}}
+    <- { "QMP": {"version": {"qemu": {"micro": 0, "minor": 0, "major": 3},
+         "package": "v3.0.0"}, "capabilities": ["oob"] } }
 
-3.3 Simple 'stop' execution
----------------------------
+.. admonition:: Example
 
-C: { "execute": "stop" }
-S: { "return": {} }
+  Capabilities negotiation
 
-3.4 KVM information
--------------------
+  .. code-block:: QMP
 
-C: { "execute": "query-kvm", "id": "example" }
-S: { "return": { "enabled": true, "present": true }, "id": "example"}
+    -> { "execute": "qmp_capabilities", "arguments": { "enable": ["oob"] } }
+    <- { "return": {}}
 
-3.5 Parsing error
-------------------
+.. admonition:: Example
 
-C: { "execute": }
-S: { "error": { "class": "GenericError", "desc": "Invalid JSON syntax" } }
+  Simple 'stop' execution
 
-3.6 Powerdown event
--------------------
+  .. code-block:: QMP
 
-S: { "timestamp": { "seconds": 1258551470, "microseconds": 802384 },
-    "event": "POWERDOWN" }
+    -> { "execute": "stop" }
+    <- { "return": {} }
 
-3.7 Out-of-band execution
--------------------------
+.. admonition:: Example
 
-C: { "exec-oob": "migrate-pause", "id": 42 }
-S: { "id": 42,
-     "error": { "class": "GenericError",
-      "desc": "migrate-pause is currently only supported during postcopy-active state" } }
+  KVM information
+
+  .. code-block:: QMP
+
+    -> { "execute": "query-kvm", "id": "example" }
+    <- { "return": { "enabled": true, "present": true }, "id": "example"}
+
+.. admonition:: Example
+
+  Parsing error
+
+  .. code-block:: QMP
+
+    -> { "execute": }
+    <- { "error": { "class": "GenericError", "desc": "Invalid JSON syntax" } }
+
+.. admonition:: Example
+
+  Powerdown event
+
+  .. code-block:: QMP
+
+    <- { "timestamp": { "seconds": 1258551470, "microseconds": 802384 },
+        "event": "POWERDOWN" }
+
+.. admonition:: Example
+
+  Out-of-band execution
+
+  .. code-block:: QMP
+
+    -> { "exec-oob": "migrate-pause", "id": 42 }
+    <- { "id": 42,
+         "error": { "class": "GenericError",
+          "desc": "migrate-pause is currently only supported during postcopy-active state" } }
 
 
-4. Capabilities Negotiation
-===========================
+Capabilities Negotiation
+========================
 
 When a Client successfully establishes a connection, the Server is in
 Capabilities Negotiation mode.
 
-In this mode only the qmp_capabilities command is allowed to run, all
-other commands will return the CommandNotFound error. Asynchronous
+In this mode only the ``qmp_capabilities`` command is allowed to run; all
+other commands will return the ``CommandNotFound`` error. Asynchronous
 messages are not delivered either.
 
-Clients should use the qmp_capabilities command to enable capabilities
-advertised in the Server's greeting (section '2.2 Server Greeting') they
-support.
+Clients should use the ``qmp_capabilities`` command to enable capabilities
+advertised in the `Server Greeting`_ which they support.
 
-When the qmp_capabilities command is issued, and if it does not return an
-error, the Server enters in Command mode where capabilities changes take
-effect, all commands (except qmp_capabilities) are allowed and asynchronous
+When the ``qmp_capabilities`` command is issued, and if it does not return an
+error, the Server enters Command mode where capabilities changes take
+effect, all commands (except ``qmp_capabilities``) are allowed and asynchronous
 messages are delivered.
 
-5 Compatibility Considerations
-==============================
+Compatibility Considerations
+============================
 
 All protocol changes or new features which modify the protocol format in an
 incompatible way are disabled by default and will be advertised by the
-capabilities array (section '2.2 Server Greeting'). Thus, Clients can check
+capabilities array (in the `Server Greeting`_). Thus, Clients can check
 that array and enable the capabilities they support.
 
 The QMP Server performs a type check on the arguments to a command.  It
@@ -337,12 +368,12 @@ However, Clients must not assume any particular:
 
 - Length of json-arrays
 - Size of json-objects; in particular, future versions of QEMU may add
-  new keys and Clients should be able to ignore them.
+  new keys and Clients should be able to ignore them
 - Order of json-object members or json-array elements
 - Amount of errors generated by a command, that is, new errors can be added
   to any existing command in newer versions of the Server
 
-Any command or member name beginning with "x-" is deemed experimental,
+Any command or member name beginning with ``x-`` is deemed experimental,
 and may be withdrawn or changed in an incompatible manner in a future
 release.
 
@@ -350,8 +381,8 @@ Of course, the Server does guarantee to send valid JSON.  But apart from
 this, a Client should be "conservative in what they send, and liberal in
 what they accept".
 
-6. Downstream extension of QMP
-==============================
+Downstream extension of QMP
+===========================
 
 We recommend that downstream consumers of QEMU do *not* modify QMP.
 Management tools should be able to support both upstream and downstream
@@ -363,23 +394,25 @@ avoid modifying QMP.  Both upstream and downstream need to take care to
 preserve long-term compatibility and interoperability.
 
 To help with that, QMP reserves JSON object member names beginning with
-'__' (double underscore) for downstream use ("downstream names").  This
+``__`` (double underscore) for downstream use ("downstream names").  This
 means upstream will never use any downstream names for its commands,
 arguments, errors, asynchronous events, and so forth.
 
-Any new names downstream wishes to add must begin with '__'.  To
+Any new names downstream wishes to add must begin with ``__``.  To
 ensure compatibility with other downstreams, it is strongly
-recommended that you prefix your downstream names with '__RFQDN_' where
+recommended that you prefix your downstream names with ``__RFQDN_`` where
 RFQDN is a valid, reverse fully qualified domain name which you
 control.  For example, a qemu-kvm specific monitor command would be:
 
+.. code-block::
+
     (qemu) __org.linux-kvm_enable_irqchip
 
-Downstream must not change the server greeting (section 2.2) other than
+Downstream must not change the `server greeting`_ other than
 to offer additional capabilities.  But see below for why even that is
 discouraged.
 
-Section '5 Compatibility Considerations' applies to downstream as well
+The section `Compatibility Considerations`_ applies to downstream as well
 as to upstream, obviously.  It follows that downstream must behave
 exactly like upstream for any input not containing members with
 downstream names ("downstream members"), except it may add members
diff --git a/qapi/control.json b/qapi/control.json
index 6a7c5af882..a91fa33407 100644
--- a/qapi/control.json
+++ b/qapi/control.json
@@ -27,7 +27,7 @@
 # Notes: This command is valid exactly when first connecting: it must
 #     be issued before any other command will be accepted, and will
 #     fail once the monitor is accepting other commands.  (see qemu
-#     docs/interop/qmp-spec.txt)
+#     docs/interop/qmp-spec.rst)
 #
 #     The QMP client needs to explicitly enable QMP capabilities,
 #     otherwise all the QMP capabilities will be turned off by
@@ -46,7 +46,7 @@
 # connection, used for agreeing on particular QMP extension behaviors.
 #
 # @oob: QMP ability to support out-of-band requests.  (Please refer to
-#     qmp-spec.txt for more information on OOB)
+#     qmp-spec.rst for more information on OOB)
 #
 # Since: 2.12
 ##
diff --git a/qapi/qapi-schema.json b/qapi/qapi-schema.json
index 31e0b36454..6594afba31 100644
--- a/qapi/qapi-schema.json
+++ b/qapi/qapi-schema.json
@@ -29,7 +29,8 @@
 #   -> data issued by the Client
 #   <- Server data response
 #
-# Please, refer to the QMP specification (docs/interop/qmp-spec.txt)
+# Please refer to the
+# :doc:`QEMU Machine Protocol Specification </interop/qmp-spec>`
 # for detailed information on the Server command and response formats.
 ##
 
diff --git a/qobject/json-lexer.c b/qobject/json-lexer.c
index 632320d72d..51341d96e4 100644
--- a/qobject/json-lexer.c
+++ b/qobject/json-lexer.c
@@ -139,7 +139,7 @@ static const uint8_t json_lexer[][256] =  {
          * bytes '\xFE', '\xFF'.  Structural characters and line
          * endings are promising resynchronization points.  Clients
          * may use the others to force the JSON parser into known-good
-         * state; see docs/interop/qmp-spec.txt.
+         * state; see docs/interop/qmp-spec.rst.
          */
         [0 ... 0x1F] = IN_START | LOOKAHEAD,
         [0x20 ... 0xFD] = IN_RECOVERY,
diff --git a/python/qemu/qmp/models.py b/python/qemu/qmp/models.py
index de87f87804..da52848d5a 100644
--- a/python/qemu/qmp/models.py
+++ b/python/qemu/qmp/models.py
@@ -54,7 +54,7 @@ def __repr__(self) -> str:
 
 class Greeting(Model):
     """
-    Defined in qmp-spec.txt, section 2.2, "Server Greeting".
+    Defined in qmp-spec.rst, section "Server Greeting".
 
     :param raw: The raw Greeting object.
     :raise KeyError: If any required fields are absent.
@@ -82,7 +82,7 @@ def _asdict(self) -> Dict[str, object]:
 
 class QMPGreeting(Model):
     """
-    Defined in qmp-spec.txt, section 2.2, "Server Greeting".
+    Defined in qmp-spec.rst, section "Server Greeting".
 
     :param raw: The raw QMPGreeting object.
     :raise KeyError: If any required fields are absent.
@@ -104,7 +104,7 @@ def __init__(self, raw: Mapping[str, Any]):
 
 class ErrorResponse(Model):
     """
-    Defined in qmp-spec.txt, section 2.4.2, "error".
+    Defined in qmp-spec.rst, section "Error".
 
     :param raw: The raw ErrorResponse object.
     :raise KeyError: If any required fields are absent.
@@ -126,7 +126,7 @@ def __init__(self, raw: Mapping[str, Any]):
 
 class ErrorInfo(Model):
     """
-    Defined in qmp-spec.txt, section 2.4.2, "error".
+    Defined in qmp-spec.rst, section "Error".
 
     :param raw: The raw ErrorInfo object.
     :raise KeyError: If any required fields are absent.
diff --git a/python/qemu/qmp/qmp_client.py b/python/qemu/qmp/qmp_client.py
index 9d73ae6e7a..2a817f9db3 100644
--- a/python/qemu/qmp/qmp_client.py
+++ b/python/qemu/qmp/qmp_client.py
@@ -369,7 +369,7 @@ async def _on_message(self, msg: Message) -> None:
             # This is very likely a server parsing error.
             # It doesn't inherently belong to any pending execution.
             # Instead of performing clever recovery, just terminate.
-            # See "NOTE" in qmp-spec.txt, section 2.4.2
+            # See "NOTE" in qmp-spec.rst, section "Error".
             raise ServerParseError(
                 ("Server sent an error response without an ID, "
                  "but there are no ID-less executions pending. "
@@ -377,7 +377,7 @@ async def _on_message(self, msg: Message) -> None:
                 msg
             )
 
-        # qmp-spec.txt, section 2.4:
+        # qmp-spec.rst, section "Commands Responses":
         # 'Clients should drop all the responses
         # that have an unknown "id" field.'
         self.logger.log(
-- 
2.39.2

[PULL 3/5] docs/interop/qmp-spec: Update error description for parsing errors

[Markus Armbruster]

From: Peter Maydell <peter.maydell@linaro.org>

The description text for a parsing error has changed since the
spec doc was first written; update the example in the docs.

Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20230515162245.3964307-3-peter.maydell@linaro.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 docs/interop/qmp-spec.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/interop/qmp-spec.rst b/docs/interop/qmp-spec.rst
index bfad570a16..2609b3ff9b 100644
--- a/docs/interop/qmp-spec.rst
+++ b/docs/interop/qmp-spec.rst
@@ -307,7 +307,7 @@ This section provides some examples of real QMP usage, in all of them
   .. code-block:: QMP
 
     -> { "execute": }
-    <- { "error": { "class": "GenericError", "desc": "Invalid JSON syntax" } }
+    <- { "error": { "class": "GenericError", "desc": "JSON parse error, expecting value" } }
 
 .. admonition:: Example
 
-- 
2.39.2

[PULL 4/5] docs/interop: Delete qmp-intro.txt

[Markus Armbruster]

From: Peter Maydell <peter.maydell@linaro.org>

qmp-intro.txt is quite small and provides very little information
that isn't already in the documentation elsewhere.  Fold the example
command lines into qemu-options.hx, and delete the now-unneeded plain
text document.

While we're touching the qemu-options.hx documentation text,
wordsmith it a little bit and improve the rST formatting.

Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20230515162245.3964307-4-peter.maydell@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 docs/interop/qmp-intro.txt | 88 --------------------------------------
 qemu-options.hx            | 28 +++++++++---
 2 files changed, 22 insertions(+), 94 deletions(-)
 delete mode 100644 docs/interop/qmp-intro.txt

diff --git a/docs/interop/qmp-intro.txt b/docs/interop/qmp-intro.txt
deleted file mode 100644
index 1c745a7af0..0000000000
--- a/docs/interop/qmp-intro.txt
+++ /dev/null
@@ -1,88 +0,0 @@
-                          QEMU Machine Protocol
-                          =====================
-
-Introduction
-------------
-
-The QEMU Machine Protocol (QMP) allows applications to operate a
-QEMU instance.
-
-QMP is JSON[1] based and features the following:
-
-- Lightweight, text-based, easy to parse data format
-- Asynchronous messages support (ie. events)
-- Capabilities Negotiation
-
-For detailed information on QMP's usage, please, refer to the following files:
-
-o qmp-spec.txt      QEMU Machine Protocol current specification
-o qemu-qmp-ref.html QEMU QMP commands and events (auto-generated at build-time)
-
-[1] https://www.json.org
-
-Usage
------
-
-You can use the -qmp option to enable QMP. For example, the following
-makes QMP available on localhost port 4444:
-
-$ qemu [...] -qmp tcp:localhost:4444,server=on,wait=off
-
-However, for more flexibility and to make use of more options, the -mon
-command-line option should be used. For instance, the following example
-creates one HMP instance (human monitor) on stdio and one QMP instance
-on localhost port 4444:
-
-$ qemu [...] -chardev stdio,id=mon0 -mon chardev=mon0,mode=readline \
-             -chardev socket,id=mon1,host=localhost,port=4444,server=on,wait=off \
-             -mon chardev=mon1,mode=control,pretty=on
-
-Please, refer to QEMU's manpage for more information.
-
-Simple Testing
---------------
-
-To manually test QMP one can connect with telnet and issue commands by hand:
-
-$ telnet localhost 4444
-Trying 127.0.0.1...
-Connected to localhost.
-Escape character is '^]'.
-{
-    "QMP": {
-        "version": {
-            "qemu": {
-                "micro": 0,
-                "minor": 0,
-                "major": 3
-            },
-            "package": "v3.0.0"
-        },
-        "capabilities": [
-            "oob"
-        ]
-    }
-}
-
-{ "execute": "qmp_capabilities" }
-{
-    "return": {
-    }
-}
-
-{ "execute": "query-status" }
-{
-    "return": {
-        "status": "prelaunch", 
-        "singlestep": false, 
-        "running": false
-    }
-}
-
-Please refer to docs/interop/qemu-qmp-ref.* for a complete command
-reference, generated from qapi/qapi-schema.json.
-
-QMP wiki page
--------------
-
-https://wiki.qemu.org/QMP
diff --git a/qemu-options.hx b/qemu-options.hx
index 30690d9c3f..e4566149ee 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -4170,26 +4170,42 @@ DEF("qmp", HAS_ARG, QEMU_OPTION_qmp, \
     QEMU_ARCH_ALL)
 SRST
 ``-qmp dev``
-    Like -monitor but opens in 'control' mode.
+    Like ``-monitor`` but opens in 'control' mode. For example, to make
+    QMP available on localhost port 4444::
+
+        -qmp tcp:localhost:4444,server=on,wait=off
+
+    Not all options are configurable via this syntax; for maximum
+    flexibility use the ``-mon`` option and an accompanying ``-chardev``.
+
 ERST
 DEF("qmp-pretty", HAS_ARG, QEMU_OPTION_qmp_pretty, \
     "-qmp-pretty dev like -qmp but uses pretty JSON formatting\n",
     QEMU_ARCH_ALL)
 SRST
 ``-qmp-pretty dev``
-    Like -qmp but uses pretty JSON formatting.
+    Like ``-qmp`` but uses pretty JSON formatting.
 ERST
 
 DEF("mon", HAS_ARG, QEMU_OPTION_mon, \
     "-mon [chardev=]name[,mode=readline|control][,pretty[=on|off]]\n", QEMU_ARCH_ALL)
 SRST
 ``-mon [chardev=]name[,mode=readline|control][,pretty[=on|off]]``
-    Setup monitor on chardev name. ``mode=control`` configures 
-    a QMP monitor (a JSON RPC-style protocol) and it is not the
-    same as HMP, the human monitor that has a "(qemu)" prompt.
-    ``pretty`` is only valid when ``mode=control``, 
+    Set up a monitor connected to the chardev ``name``.
+    QEMU supports two monitors: the Human Monitor Protocol
+    (HMP; for human interaction), and the QEMU Monitor Protocol
+    (QMP; a JSON RPC-style protocol).
+    The default is HMP; ``mode=control`` selects QMP instead.
+    ``pretty`` is only valid when ``mode=control``,
     turning on JSON pretty printing to ease
     human reading and debugging.
+
+    For example::
+
+      -chardev socket,id=mon1,host=localhost,port=4444,server=on,wait=off \
+      -mon chardev=mon1,mode=control,pretty=on
+
+    enables the QMP monitor on localhost port 4444 with pretty-printing.
 ERST
 
 DEF("debugcon", HAS_ARG, QEMU_OPTION_debugcon, \
-- 
2.39.2

[PULL 5/5] qapi/parser: Drop two bad type hints for now

[Markus Armbruster]

Two type hints fail centos-stream-8-x86_64 CI.  They are actually
broken.  Changing them to Optional[re.Match[str]] fixes them locally
for me, but then CI fails differently.  Drop them for now.

Fixes: 3e32dca3f0d1 (qapi: Rewrite parsing of doc comment section symbols and tags)
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230517061600.1782455-1-armbru@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Tested-by: Igor Mammedov <imammedo@redhat.com>
---
 scripts/qapi/parser.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index acd43cd7e1..22e7bcc4b1 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -563,11 +563,11 @@ def end_comment(self) -> None:
         self._switch_section(QAPIDoc.NullSection(self._parser))
 
     @staticmethod
-    def _match_at_name_colon(string: str) -> re.Match:
+    def _match_at_name_colon(string: str):
         return re.match(r'@([^:]*): *', string)
 
     @staticmethod
-    def _match_section_tag(string: str) -> re.Match:
+    def _match_section_tag(string: str):
         return re.match(r'(Returns|Since|Notes?|Examples?|TODO): *', string)
 
     def _append_body_line(self, line: str) -> None:
-- 
2.39.2

Re: [PULL 0/5] QAPI patches patches for 2023-05-17

[Richard Henderson]

On 5/17/23 08:25, Markus Armbruster wrote:
> The following changes since commit f9d58e0ca53b3f470b84725a7b5e47fcf446a2ea:
> 
>    Merge tag 'pull-9p-20230516' of https://github.com/cschoenebeck/qemu into staging (2023-05-16 10:21:44 -0700)
> 
> are available in the Git repository at:
> 
>    https://repo.or.cz/qemu/armbru.git tags/pull-qapi-2023-05-17
> 
> for you to fetch changes up to 28d2a4f620db2bc327dbd3443b777890d39a0bf6:
> 
>    qapi/parser: Drop two bad type hints for now (2023-05-17 15:49:22 +0200)
> 
> ----------------------------------------------------------------
> QAPI patches patches for 2023-05-17

Multiple failures:

https://gitlab.com/qemu-project/qemu/-/jobs/4303506112#L4674

▶  2/61 /x86_64/migration/multifd/tcp/tls/x509/default-host - 
ERROR:../tests/qtest/migration-helpers.c:180:check_migration_status: assertion failed 
(current_status != "failed"): ("failed" != "failed") FAIL


https://gitlab.com/qemu-project/qemu/-/jobs/4303506102#L3766
https://gitlab.com/qemu-project/qemu/-/jobs/4303506050#L5856
https://gitlab.com/qemu-project/qemu/-/jobs/4303506047#L3654

/home/gitlab-runner/builds/Jpwtyaz7/0/qemu-project/qemu/docs/interop/qmp-spec.rst:71:Error 
in "code-block" directive:
1 argument(s) required, 0 supplied.
.. code-block::
   { "QMP": { "version": json-object, "capabilities": json-array } }


r~

Re: [PULL 5/5] qapi/parser: Drop two bad type hints for now

[Richard Henderson]

On 5/17/23 08:25, Markus Armbruster wrote:
> Two type hints fail centos-stream-8-x86_64 CI.  They are actually
> broken.  Changing them to Optional[re.Match[str]] fixes them locally
> for me, but then CI fails differently.  Drop them for now.
> 
> Fixes: 3e32dca3f0d1 (qapi: Rewrite parsing of doc comment section symbols and tags)
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> Message-Id: <20230517061600.1782455-1-armbru@redhat.com>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Tested-by: Igor Mammedov <imammedo@redhat.com>
> ---
>   scripts/qapi/parser.py | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
> index acd43cd7e1..22e7bcc4b1 100644
> --- a/scripts/qapi/parser.py
> +++ b/scripts/qapi/parser.py
> @@ -563,11 +563,11 @@ def end_comment(self) -> None:
>           self._switch_section(QAPIDoc.NullSection(self._parser))
>   
>       @staticmethod
> -    def _match_at_name_colon(string: str) -> re.Match:
> +    def _match_at_name_colon(string: str):
>           return re.match(r'@([^:]*): *', string)
>   
>       @staticmethod
> -    def _match_section_tag(string: str) -> re.Match:
> +    def _match_section_tag(string: str):
>           return re.match(r'(Returns|Since|Notes?|Examples?|TODO): *', string)
>   
>       def _append_body_line(self, line: str) -> None:

I am going to cherry-pick this patch into master, since the build
did succeed sufficiently to show that it fixes the centos-8-stream job.


r~

Re: [PULL 0/5] QAPI patches patches for 2023-05-17

[Peter Maydell]

On Wed, 17 May 2023 at 18:16, Richard Henderson
<richard.henderson@linaro.org> wrote:
> https://gitlab.com/qemu-project/qemu/-/jobs/4303506102#L3766
> https://gitlab.com/qemu-project/qemu/-/jobs/4303506050#L5856
> https://gitlab.com/qemu-project/qemu/-/jobs/4303506047#L3654
>
> /home/gitlab-runner/builds/Jpwtyaz7/0/qemu-project/qemu/docs/interop/qmp-spec.rst:71:Error
> in "code-block" directive:
> 1 argument(s) required, 0 supplied.
> .. code-block::
>    { "QMP": { "version": json-object, "capabilities": json-array } }

Looks like whatever version of Sphinx this is enforces that
code-block comments specify a language. The system I tested on
didn't care.

In theory these blocks should be ".. code-block:: json", you
might think, but for me that results in

Warning, treated as error:
/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/docs/interop/qmp-spec.rst:71:Could
not lex literal_block as "json". Highlighting skipped.

(which I think means my system doesn't have the json lexer,
but who knows...)

So the simplest fix is to turn all the no-language code-blocks into
plain old literal blocks (the ones which are "QMP" format are fine,
but specifying QMP doesn't work for these blocks, because the QMP lexer
insists on seeing the -> and <- that indicate a command/response
direction). This should work as a fixup to the patch.

Markus, could I ask you to squash it in for the next round of this pullreq ?

diff --git a/docs/interop/qmp-spec.rst b/docs/interop/qmp-spec.rst
index 2609b3ff9b2..23ddcc853e5 100644
--- a/docs/interop/qmp-spec.rst
+++ b/docs/interop/qmp-spec.rst
@@ -68,7 +68,7 @@ ready for capabilities negotiation (for more
information refer to section

 The greeting message format is:

-.. code-block::
+::

   { "QMP": { "version": json-object, "capabilities": json-array } }

@@ -94,13 +94,13 @@ Issuing Commands

 The format for command execution is:

-.. code-block::
+::

   { "execute": json-string, "arguments": json-object, "id": json-value }

 or

-.. code-block::
+::

   { "exec-oob": json-string, "arguments": json-object, "id": json-value }

@@ -164,7 +164,7 @@ Success

 The format of a success response is:

-.. code-block::
+::

   { "return": json-value, "id": json-value }

@@ -183,7 +183,7 @@ Error

 The format of an error response is:

-.. code-block::
+::

   { "error": { "class": json-string, "desc": json-string }, "id": json-value }

@@ -208,7 +208,7 @@ response. They are called "asynchronous events".

 The format of asynchronous events is:

-.. code-block::
+::

   { "event": json-string, "data": json-object,
     "timestamp": { "seconds": json-number, "microseconds": json-number } }

thanks
-- PMM