[Qemu-devel] [PATCH 00/32] qmp: Fixes and cleanups around OOB commands

[Qemu-devel] [PATCH 00/32] qmp: Fixes and cleanups around OOB commands

[Markus Armbruster]

We're trying to get the out-of-band execution feature ready.  This
series fixes a number of issues, and marks a design flaw FIXME.  More
work is needed.

Markus Armbruster (32):
  qmp: Say "out-of-band" instead of "Out-Of-Band"
  monitor: Spell "I/O thread" consistently in comments
  docs/interop/qmp: Improve OOB documentation
  qmp: Document COMMAND_DROPPED design flaw
  qmp: Get rid of x-oob-test command
  tests/qmp-test: Test in-band command doesn't overtake
  qmp: Make "id" optional again even in "oob" monitors
  tests/test-qga: Demonstrate the guest-agent ignores "id"
  qmp qemu-ga: Revert change that accidentally made qemu-ga accept "id"
  tests/test-qga: Demonstrate the guest-agent ignores "control"
  qmp qemu-ga: Fix qemu-ga not to accept "control"
  qmp: Redo how the client requests out-of-band execution
  qmp: Revert change to handle_qmp_command tracepoint
  qmp: Always free QMPRequest with qmp_request_free()
  qmp: Simplify code around monitor_qmp_dispatch_one()
  tests/qmp-test: Demonstrate QMP errors jumping the queue
  qmp: Don't let malformed in-band commands jump the queue
  qmp: Don't let JSON errors jump the queue
  monitor: Rename use_io_thr to use_io_thread
  monitor: Peel off @mon_global wrapper
  qobject: New qdict_from_jsonf_nofail()
  qmp: De-duplicate error response building
  qmp: Use QDict * instead of QObject * for response objects
  qmp: Replace monitor_json_emitter{,raw}() by
    qmp_{queue,send}_response()
  qmp: Replace get_qmp_greeting() by qmp_greeting()
  qmp: Simplify monitor_qmp_respond()
  qmp: Add some comments around null responses
  qmp: Switch timestamp_put() to qdict_from_jsonf_nofail()
  qobject: Let qobject_from_jsonf() fail instead of abort
  qmp: Clean up capability negotiation after commit 02130314d8c
  monitor: Improve some comments
  qapi: Polish command flags documentation in qapi-code-gen.txt

 docs/devel/qapi-code-gen.txt            |  63 ++-
 docs/interop/qmp-intro.txt              |  13 +-
 docs/interop/qmp-spec.txt               |  81 ++--
 include/qapi/qmp/dispatch.h             |   6 +-
 include/qapi/qmp/qjson.h                |   2 +
 monitor.c                               | 507 ++++++++++--------------
 qapi/misc.json                          |  23 +-
 qapi/qmp-dispatch.c                     |  97 ++---
 qapi/qmp-event.c                        |  10 +-
 qga/main.c                              |  18 +-
 qmp.c                                   |  16 -
 qobject/qjson.c                         |  23 +-
 tests/qapi-schema/qapi-schema-test.json |   2 +-
 tests/qmp-test.c                        | 103 +++--
 tests/test-qga.c                        |  34 ++
 tests/test-qmp-cmds.c                   |  23 +-
 16 files changed, 499 insertions(+), 522 deletions(-)

-- 
2.17.1

[Qemu-devel] [PATCH 01/32] qmp: Say "out-of-band" instead of "Out-Of-Band"

[Markus Armbruster]

Affects documentation and a few error messages.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 docs/devel/qapi-code-gen.txt            |  2 +-
 docs/interop/qmp-spec.txt               |  2 +-
 monitor.c                               | 14 +++++++-------
 qapi/misc.json                          |  2 +-
 tests/qapi-schema/qapi-schema-test.json |  2 +-
 tests/qmp-test.c                        |  2 +-
 6 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index 94a7e8f4d0..9625798d16 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -641,7 +641,7 @@ possible, the command expression should include the optional key
 'success-response' with boolean value false.  So far, only QGA makes
 use of this member.
 
-A command can be declared to support Out-Of-Band (OOB) execution.  By
+A command can be declared to support out-of-band (OOB) execution.  By
 default, commands do not support OOB.  To declare a command that
 supports it, the schema includes an extra 'allow-oob' field.  For
 example:
diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.txt
index 6fa193a80b..2bb492d1ea 100644
--- a/docs/interop/qmp-spec.txt
+++ b/docs/interop/qmp-spec.txt
@@ -85,7 +85,7 @@ The greeting message format is:
 
 Currently supported capabilities are:
 
-- "oob": the QMP server supports "Out-Of-Band" (OOB) command
+- "oob": the QMP server supports "out-of-band" (OOB) command
   execution.  For more details, please see the "run-oob" parameter in
   the "Issuing Commands" section below.  Not all commands allow this
   "oob" execution.  The "query-qmp-schema" command can be used to
diff --git a/monitor.c b/monitor.c
index 567668a0e7..f5f20e36e3 100644
--- a/monitor.c
+++ b/monitor.c
@@ -1267,11 +1267,11 @@ static void qmp_caps_check(Monitor *mon, QMPCapabilityList *list,
         case QMP_CAPABILITY_OOB:
             if (!mon->use_io_thr) {
                 /*
-                 * Out-Of-Band only works with monitors that are
+                 * Out-of-band only works with monitors that are
                  * running on dedicated IOThread.
                  */
                 error_setg(errp, "This monitor does not support "
-                           "Out-Of-Band (OOB)");
+                           "out-of-band (OOB)");
                 return;
             }
             break;
@@ -1319,7 +1319,7 @@ static bool qmp_cmd_oob_check(Monitor *mon, QDict *req, Error **errp)
 
     if (qmp_is_oob(req)) {
         if (!qmp_oob_enabled(mon)) {
-            error_setg(errp, "Please enable Out-Of-Band first "
+            error_setg(errp, "Please enable out-of-band first "
                        "for the session during capabilities negotiation");
             return false;
         }
@@ -4293,7 +4293,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
 
     /* When OOB is enabled, the "id" field is mandatory. */
     if (qmp_oob_enabled(mon) && !id) {
-        error_setg(&err, "Out-Of-Band capability requires that "
+        error_setg(&err, "Out-of-band capability requires that "
                    "every command contains an 'id' field");
         goto err;
     }
@@ -4307,7 +4307,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     qdict_del(qdict, "id");
 
     if (qmp_is_oob(qdict)) {
-        /* Out-Of-Band (OOB) requests are executed directly in parser. */
+        /* Out-of-band (OOB) requests are executed directly in parser. */
         trace_monitor_qmp_cmd_out_of_band(qobject_get_try_str(req_obj->id)
                                           ?: "");
         monitor_qmp_dispatch_one(req_obj);
@@ -4683,12 +4683,12 @@ void monitor_init(Chardev *chr, int flags)
 
     if (use_oob) {
         if (CHARDEV_IS_MUX(chr)) {
-            error_report("Monitor Out-Of-Band is not supported with "
+            error_report("Monitor out-of-band is not supported with "
                          "MUX typed chardev backend");
             exit(1);
         }
         if (use_readline) {
-            error_report("Monitor Out-Of-band is only supported by QMP");
+            error_report("Monitor out-of-band is only supported by QMP");
             exit(1);
         }
     }
diff --git a/qapi/misc.json b/qapi/misc.json
index 29da7856e3..0446c3e48e 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -46,7 +46,7 @@
 # Enumeration of capabilities to be advertised during initial client
 # connection, used for agreeing on particular QMP extension behaviors.
 #
-# @oob:   QMP ability to support Out-Of-Band requests.
+# @oob:   QMP ability to support out-of-band requests.
 #         (Please refer to qmp-spec.txt for more information on OOB)
 #
 # Since: 2.12
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 7b59817f04..e9e401c01f 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -141,7 +141,7 @@
 { 'command': 'boxed-struct', 'boxed': true, 'data': 'UserDefZero' }
 { 'command': 'boxed-union', 'data': 'UserDefNativeListUnion', 'boxed': true }
 
-# Smoke test on Out-Of-Band and allow-preconfig-test
+# Smoke test on out-of-band and allow-preconfig-test
 { 'command': 'test-flags-command', 'allow-oob': true, 'allow-preconfig': true }
 
 # For testing integer range flattening in opts-visitor. The following schema
diff --git a/tests/qmp-test.c b/tests/qmp-test.c
index a49cbc6fde..5206b14ca6 100644
--- a/tests/qmp-test.c
+++ b/tests/qmp-test.c
@@ -135,7 +135,7 @@ static void test_qmp_protocol(void)
     qtest_quit(qts);
 }
 
-/* Tests for Out-Of-Band support. */
+/* Tests for out-of-band support. */
 static void test_qmp_oob(void)
 {
     QTestState *qts;
-- 
2.17.1

Re: [Qemu-devel] [PATCH 01/32] qmp: Say "out-of-band" instead of "Out-Of-Band"

[Eric Blake]

On 07/02/2018 11:21 AM, Markus Armbruster wrote:
> Affects documentation and a few error messages.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   docs/devel/qapi-code-gen.txt            |  2 +-
>   docs/interop/qmp-spec.txt               |  2 +-
>   monitor.c                               | 14 +++++++-------
>   qapi/misc.json                          |  2 +-
>   tests/qapi-schema/qapi-schema-test.json |  2 +-
>   tests/qmp-test.c                        |  2 +-
>   6 files changed, 12 insertions(+), 12 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 02/32] monitor: Spell "I/O thread" consistently in comments

[Markus Armbruster]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c | 34 +++++++++++++++++-----------------
 1 file changed, 17 insertions(+), 17 deletions(-)

diff --git a/monitor.c b/monitor.c
index f5f20e36e3..3fb05d50aa 100644
--- a/monitor.c
+++ b/monitor.c
@@ -242,7 +242,7 @@ struct Monitor {
 /* Let's add monitor global variables to this struct. */
 static struct {
     IOThread *mon_iothread;
-    /* Bottom half to dispatch the requests received from IO thread */
+    /* Bottom half to dispatch the requests received from I/O thread */
     QEMUBH *qmp_dispatcher_bh;
     /* Bottom half to deliver the responses back to clients */
     QEMUBH *qmp_respond_bh;
@@ -517,7 +517,7 @@ static void monitor_json_emitter(Monitor *mon, QObject *data)
 {
     if (mon->use_io_thr) {
         /*
-         * If using IO thread, we need to queue the item so that IO
+         * If using I/O thread, we need to queue the item so that I/O
          * thread will do the rest for us.  Take refcount so that
          * caller won't free the data (which will be finally freed in
          * responder thread).
@@ -528,7 +528,7 @@ static void monitor_json_emitter(Monitor *mon, QObject *data)
         qemu_bh_schedule(mon_global.qmp_respond_bh);
     } else {
         /*
-         * If not using monitor IO thread, then we are in main thread.
+         * If not using monitor I/O thread, then we are in main thread.
          * Do the emission right away.
          */
         monitor_json_emitter_raw(mon, data);
@@ -1268,7 +1268,7 @@ static void qmp_caps_check(Monitor *mon, QMPCapabilityList *list,
             if (!mon->use_io_thr) {
                 /*
                  * Out-of-band only works with monitors that are
-                 * running on dedicated IOThread.
+                 * running on dedicated I/O thread.
                  */
                 error_setg(errp, "This monitor does not support "
                            "out-of-band (OOB)");
@@ -4402,7 +4402,7 @@ int monitor_suspend(Monitor *mon)
 
     if (monitor_is_qmp(mon)) {
         /*
-         * Kick iothread to make sure this takes effect.  It'll be
+         * Kick I/O thread to make sure this takes effect.  It'll be
          * evaluated again in prepare() of the watch object.
          */
         aio_notify(iothread_get_aio_context(mon_global.mon_iothread));
@@ -4421,7 +4421,7 @@ void monitor_resume(Monitor *mon)
     if (atomic_dec_fetch(&mon->suspend_cnt) == 0) {
         if (monitor_is_qmp(mon)) {
             /*
-             * For QMP monitors that are running in IOThread, let's
+             * For QMP monitors that are running in I/O thread, let's
              * kick the thread in case it's sleeping.
              */
             if (mon->use_io_thr) {
@@ -4445,7 +4445,7 @@ static QObject *get_qmp_greeting(Monitor *mon)
 
     for (cap = 0; cap < QMP_CAPABILITY__MAX; cap++) {
         if (!mon->use_io_thr && cap == QMP_CAPABILITY_OOB) {
-            /* Monitors that are not using IOThread won't support OOB */
+            /* Monitors that are not using I/O thread won't support OOB */
             continue;
         }
         qlist_append_str(cap_list, QMPCapability_str(cap));
@@ -4586,9 +4586,9 @@ static void monitor_iothread_init(void)
                                               NULL);
 
     /*
-     * Unlike the dispatcher BH, this must be run on the monitor IO
-     * thread, so that monitors that are using IO thread will make
-     * sure read/write operations are all done on the IO thread.
+     * Unlike the dispatcher BH, this must be run on the monitor I/O
+     * thread, so that monitors that are using I/O thread will make
+     * sure read/write operations are all done on the I/O thread.
      */
     mon_global.qmp_respond_bh = aio_bh_new(monitor_get_aio_context(),
                                            monitor_qmp_bh_responder,
@@ -4660,7 +4660,7 @@ static void monitor_qmp_setup_handlers_bh(void *opaque)
     if (mon->use_io_thr) {
         /*
          * When use_io_thr is set, we use the global shared dedicated
-         * IO thread for this monitor to handle input/output.
+         * I/O thread for this monitor to handle input/output.
          */
         context = monitor_get_io_context();
         /* We should have inited globals before reaching here. */
@@ -4717,7 +4717,7 @@ void monitor_init(Chardev *chr, int flags)
             /*
              * We can't call qemu_chr_fe_set_handlers() directly here
              * since during the procedure the chardev will be active
-             * and running in monitor iothread, while we'll still do
+             * and running in monitor I/O thread, while we'll still do
              * something before returning from it, which is a possible
              * race too.  To avoid that, we just create a BH to setup
              * the handlers.
@@ -4744,16 +4744,16 @@ void monitor_cleanup(void)
     Monitor *mon, *next;
 
     /*
-     * We need to explicitly stop the iothread (but not destroy it),
-     * cleanup the monitor resources, then destroy the iothread since
+     * We need to explicitly stop the I/O thread (but not destroy it),
+     * cleanup the monitor resources, then destroy the I/O thread since
      * we need to unregister from chardev below in
      * monitor_data_destroy(), and chardev is not thread-safe yet
      */
     iothread_stop(mon_global.mon_iothread);
 
     /*
-     * After we have IOThread to send responses, it's possible that
-     * when we stop the IOThread there are still replies queued in the
+     * After we have I/O thread to send responses, it's possible that
+     * when we stop the I/O thread there are still replies queued in the
      * responder queue.  Flush all of them.  Note that even after this
      * flush it's still possible that out buffer is not flushed.
      * It'll be done in below monitor_flush() as the last resort.
@@ -4769,7 +4769,7 @@ void monitor_cleanup(void)
     }
     qemu_mutex_unlock(&monitor_lock);
 
-    /* QEMUBHs needs to be deleted before destroying the IOThread. */
+    /* QEMUBHs needs to be deleted before destroying the I/O thread */
     qemu_bh_delete(mon_global.qmp_dispatcher_bh);
     mon_global.qmp_dispatcher_bh = NULL;
     qemu_bh_delete(mon_global.qmp_respond_bh);
-- 
2.17.1

Re: [Qemu-devel] [PATCH 02/32] monitor: Spell "I/O thread" consistently in comments

[Eric Blake]

On 07/02/2018 11:21 AM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c | 34 +++++++++++++++++-----------------
>   1 file changed, 17 insertions(+), 17 deletions(-)
> 

Well, we DO have an IOThread C struct (in iothread.h), but I do find 
your use of "I/O thread" more legible.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 03/32] docs/interop/qmp: Improve OOB documentation

[Markus Armbruster]

OOB documentation is spread over qmp-spec.txt sections 2.2.1
Capabilities and 2.3 Issuing Commands.  The amount of detail is a bit
distracting there.  Move the meat of the matter to new section 2.3.1
Out of band execution.

Throw in a few other improvements while there:

* 2.2 Server Greetung: Drop advice to search entire capabilities
  array; should be obvious.

* 3. QMP Examples

  - 3.1 Server Greeting: Update greeting to current one.  Now shows
    capability "oob".  Update qmp-intro.txt likewise.

  - 3.2 Capabilities negotiation: Show client accepting capability
    "oob".

  - 3.7 Out-of-band execution: New.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 docs/interop/qmp-intro.txt | 13 +++----
 docs/interop/qmp-spec.txt  | 74 +++++++++++++++++++++++++-------------
 2 files changed, 56 insertions(+), 31 deletions(-)

diff --git a/docs/interop/qmp-intro.txt b/docs/interop/qmp-intro.txt
index 900d69d612..3dafa35ea6 100644
--- a/docs/interop/qmp-intro.txt
+++ b/docs/interop/qmp-intro.txt
@@ -52,13 +52,14 @@ Escape character is '^]'.
     "QMP": {
         "version": {
             "qemu": {
-                "micro": 50, 
-                "minor": 6, 
-                "major": 1
-            }, 
-            "package": ""
-        }, 
+                "micro": 50,
+                "minor": 12,
+                "major": 2
+            },
+            "package": "v2.12.0-1763-g4e022f5ccd"
+        },
         "capabilities": [
+            "oob"
         ]
     }
 }
diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.txt
index 2bb492d1ea..e5f8116c54 100644
--- a/docs/interop/qmp-spec.txt
+++ b/docs/interop/qmp-spec.txt
@@ -77,8 +77,7 @@ The greeting message format is:
   is the same of the query-version command)
 - The "capabilities" member specify the availability of features beyond the
   baseline specification; the order of elements in this array has no
-  particular significance, so a client must search the entire array
-  when looking for a particular capability
+  particular significance.
 
 2.2.1 Capabilities
 ------------------
@@ -86,16 +85,7 @@ The greeting message format is:
 Currently supported capabilities are:
 
 - "oob": the QMP server supports "out-of-band" (OOB) command
-  execution.  For more details, please see the "run-oob" parameter in
-  the "Issuing Commands" section below.  Not all commands allow this
-  "oob" execution.  The "query-qmp-schema" command can be used to
-  inspect which commands support "oob" execution.
-
-QMP clients can get a list of supported QMP capabilities of the QMP
-server in the greeting message mentioned above.  By default, all the
-capabilities are off.  To enable any QMP capabilities, the QMP client
-needs to send the "qmp_capabilities" command with an extra parameter
-for the requested capabilities.
+  execution, as described in section "2.3.1 Out-of-band execution".
 
 2.3 Issuing Commands
 --------------------
@@ -115,14 +105,38 @@ The format for command execution is:
 - The "id" member is a transaction identification associated with the
   command execution.  It is required for all commands if the OOB -
   capability was enabled at startup, and optional otherwise.  The same
-  "id" field will be part of the response if provided. The "id" member
-  can be any json-value, although most clients merely use a
-  json-number incremented for each successive command
-- The "control" member is optional, and currently only used for
-  out-of-band execution. The handling or response of an "oob" command
-  can overtake prior in-band commands.  To enable "oob" handling of a
-  particular command, just provide a control field with: { "control":
-  { "run-oob": true } }
+  "id" field will be part of the response if provided.  The "id"
+  member can be any json-value.  A json-number incremented for each
+  successive command works fine.
+- The optional "control" member further specifies how the command is
+  to be executed.  Currently, its only member is optional "run-oob".
+  See section "2.3.1 Out-of-band execution" for details.
+
+
+2.3.1 Out-of-band execution
+---------------------------
+
+The server normally reads, executes and responds to one command after
+the other.  The client therefore receives command respones in issue
+order.
+
+With out-of-band execution enabled via capability negotiation (section
+4.), the server reads and queues commands as they arrive.  It executes
+commands from the queue one after the other.  Commands executed
+out-of-band jump the queue: they command get executed right away,
+possibly overtaking prior in-band commands.  The client may therefore
+receive such a command's response before responses from prior in-band
+commands.
+
+To execute a command out-of-band, the client puts "run-oob": true into
+execute's member "control".
+
+If the client sends in-band commands faster than the server can
+execute them, the server will eventually drop commands to limit the
+queue length.  The sever sends event COMMAND_DROPPED then.
+
+Only a few commands support out-of-band execution.  The ones that do
+have "allow-oob": true in output of query-qmp-schema.
 
 2.4 Commands Responses
 ----------------------
@@ -223,12 +237,13 @@ This section provides some examples of real QMP usage, in all of them
 3.1 Server greeting
 -------------------
 
-S: { "QMP": { "version": { "qemu": { "micro": 50, "minor": 6, "major": 1 },
-     "package": ""}, "capabilities": []}}
+S: { "QMP": {"version": {"qemu": {"micro": 50, "minor": 12, "major": 2},
+     "package": "v2.12.0-1763-g4e022f5ccd"}, "capabilities": ["oob"] } }
 
-3.2 Client QMP negotiation
---------------------------
-C: { "execute": "qmp_capabilities" }
+3.2 Capabilities negotiation
+----------------------------
+
+C: { "execute": "qmp_capabilities", "arguments": { "enable": ["oob"] } }
 S: { "return": {}}
 
 3.3 Simple 'stop' execution
@@ -255,6 +270,15 @@ S: { "error": { "class": "GenericError", "desc": "Invalid JSON syntax" } }
 S: { "timestamp": { "seconds": 1258551470, "microseconds": 802384 },
     "event": "POWERDOWN" }
 
+3.7 Out-of-band execution
+-------------------------
+
+C: { "execute": "migrate-pause", "id": 42, "control": { "run-oob": true } }
+S: { "id": 42,
+     "error": { "class": "GenericError",
+      "desc": "migrate-pause is currently only supported during postcopy-active state" } }
+
+
 4. Capabilities Negotiation
 ===========================
 
-- 
2.17.1

Re: [Qemu-devel] [PATCH 03/32] docs/interop/qmp: Improve OOB documentation

[Eric Blake]

On 07/02/2018 11:21 AM, Markus Armbruster wrote:
> OOB documentation is spread over qmp-spec.txt sections 2.2.1
> Capabilities and 2.3 Issuing Commands.  The amount of detail is a bit
> distracting there.  Move the meat of the matter to new section 2.3.1
> Out of band execution.
> 
> Throw in a few other improvements while there:
> 
> * 2.2 Server Greetung: Drop advice to search entire capabilities

s/Greetung/Greeting/

>    array; should be obvious.
> 
> * 3. QMP Examples
> 
>    - 3.1 Server Greeting: Update greeting to current one.  Now shows
>      capability "oob".  Update qmp-intro.txt likewise.
> 
>    - 3.2 Capabilities negotiation: Show client accepting capability
>      "oob".
> 
>    - 3.7 Out-of-band execution: New.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   docs/interop/qmp-intro.txt | 13 +++----
>   docs/interop/qmp-spec.txt  | 74 +++++++++++++++++++++++++-------------
>   2 files changed, 56 insertions(+), 31 deletions(-)
> 
> diff --git a/docs/interop/qmp-intro.txt b/docs/interop/qmp-intro.txt
> index 900d69d612..3dafa35ea6 100644
> --- a/docs/interop/qmp-intro.txt
> +++ b/docs/interop/qmp-intro.txt
> @@ -52,13 +52,14 @@ Escape character is '^]'.
>       "QMP": {
>           "version": {
>               "qemu": {
> -                "micro": 50,
> -                "minor": 6,
> -                "major": 1
> -            },
> -            "package": ""
> -        },
> +                "micro": 50,
> +                "minor": 12,
> +                "major": 2
> +            },
> +            "package": "v2.12.0-1763-g4e022f5ccd"

Obviously tested during development rather than on a released version, 
but that's reasonable.


> +2.3.1 Out-of-band execution
> +---------------------------
> +
> +The server normally reads, executes and responds to one command after
> +the other.  The client therefore receives command respones in issue

s/respones/responses/

> +order.
> +
> +With out-of-band execution enabled via capability negotiation (section
> +4.), the server reads and queues commands as they arrive.  It executes
> +commands from the queue one after the other.  Commands executed
> +out-of-band jump the queue: they command get executed right away,

s/they command get/the command gets/

> +possibly overtaking prior in-band commands.  The client may therefore
> +receive such a command's response before responses from prior in-band
> +commands.
> +
> +To execute a command out-of-band, the client puts "run-oob": true into
> +execute's member "control".

Are we still hoping for a later patch to ditch "control" and add 
"exec-oob"?  But if we do that, building it on top of this patch (with 
appropriate doc tweaks at that time) makes sense.

> +3.7 Out-of-band execution
> +-------------------------
> +
> +C: { "execute": "migrate-pause", "id": 42, "control": { "run-oob": true } }
> +S: { "id": 42,
> +     "error": { "class": "GenericError",
> +      "desc": "migrate-pause is currently only supported during postcopy-active state" } }

Different from a successful command, but works for me.

With the typo fixes,
Reviewed-by: Eric Blake <eblake@redhat.com>

I don't know if this patch will make it in time for 3.0 soft freeze, but 
resolving missing documentation feels like enough of a bug fix to be 
included during freeze as needed.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH 03/32] docs/interop/qmp: Improve OOB documentation

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 07/02/2018 11:21 AM, Markus Armbruster wrote:
>> OOB documentation is spread over qmp-spec.txt sections 2.2.1
>> Capabilities and 2.3 Issuing Commands.  The amount of detail is a bit
>> distracting there.  Move the meat of the matter to new section 2.3.1
>> Out of band execution.
>>
>> Throw in a few other improvements while there:
>>
>> * 2.2 Server Greetung: Drop advice to search entire capabilities
>
> s/Greetung/Greeting/
>
>>    array; should be obvious.
>>
>> * 3. QMP Examples
>>
>>    - 3.1 Server Greeting: Update greeting to current one.  Now shows
>>      capability "oob".  Update qmp-intro.txt likewise.
>>
>>    - 3.2 Capabilities negotiation: Show client accepting capability
>>      "oob".
>>
>>    - 3.7 Out-of-band execution: New.
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>   docs/interop/qmp-intro.txt | 13 +++----
>>   docs/interop/qmp-spec.txt  | 74 +++++++++++++++++++++++++-------------
>>   2 files changed, 56 insertions(+), 31 deletions(-)
>>
>> diff --git a/docs/interop/qmp-intro.txt b/docs/interop/qmp-intro.txt
>> index 900d69d612..3dafa35ea6 100644
>> --- a/docs/interop/qmp-intro.txt
>> +++ b/docs/interop/qmp-intro.txt
>> @@ -52,13 +52,14 @@ Escape character is '^]'.
>>       "QMP": {
>>           "version": {
>>               "qemu": {
>> -                "micro": 50,
>> -                "minor": 6,
>> -                "major": 1
>> -            },
>> -            "package": ""
>> -        },
>> +                "micro": 50,
>> +                "minor": 12,
>> +                "major": 2
>> +            },
>> +            "package": "v2.12.0-1763-g4e022f5ccd"
>
> Obviously tested during development rather than on a released version,
> but that's reasonable.

I could change it to whatever we expect for 3.0.  Tidier, I guess.

>> +2.3.1 Out-of-band execution
>> +---------------------------
>> +
>> +The server normally reads, executes and responds to one command after
>> +the other.  The client therefore receives command respones in issue
>
> s/respones/responses/
>
>> +order.
>> +
>> +With out-of-band execution enabled via capability negotiation (section
>> +4.), the server reads and queues commands as they arrive.  It executes
>> +commands from the queue one after the other.  Commands executed
>> +out-of-band jump the queue: they command get executed right away,
>
> s/they command get/the command gets/

My typing deteriorates as the soft freeze closes in...

>> +possibly overtaking prior in-band commands.  The client may therefore
>> +receive such a command's response before responses from prior in-band
>> +commands.
>> +
>> +To execute a command out-of-band, the client puts "run-oob": true into
>> +execute's member "control".
>
> Are we still hoping for a later patch to ditch "control" and add
> "exec-oob"?  But if we do that, building it on top of this patch (with
> appropriate doc tweaks at that time) makes sense.

[PATCH 12/32] qmp: Redo how the client requests out-of-band execution

>> +3.7 Out-of-band execution
>> +-------------------------
>> +
>> +C: { "execute": "migrate-pause", "id": 42, "control": { "run-oob": true } }
>> +S: { "id": 42,
>> +     "error": { "class": "GenericError",
>> +      "desc": "migrate-pause is currently only supported during postcopy-active state" } }
>
> Different from a successful command, but works for me.
>
> With the typo fixes,
> Reviewed-by: Eric Blake <eblake@redhat.com>
>
> I don't know if this patch will make it in time for 3.0 soft freeze,
> but resolving missing documentation feels like enough of a bug fix to
> be included during freeze as needed.

The OOB feature has been merged already.  We had to downgrade it to
"experimental" when we ran into regressions.  Lucky us; we found more
issues since then.  This series works towards getting it ready.  I think
that's something we can do even after soft feature freeze, as long as we
deem the patches sufficiently unintrusive.  Judgement call, soliciting
yours.

Thanks!

[Qemu-devel] [PATCH 04/32] qmp: Document COMMAND_DROPPED design flaw

[Markus Armbruster]

Events are broadcast to all monitors.  If another monitor's client has
a command with the same ID in flight, the event will incorrectly claim
that command was dropped.  This must be fixed before out-of-band
execution can graduate from "experimental".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c      | 6 ++++++
 qapi/misc.json | 3 +++
 2 files changed, 9 insertions(+)

diff --git a/monitor.c b/monitor.c
index 3fb05d50aa..96e87d8664 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4330,6 +4330,12 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         /* Drop the request if queue is full. */
         if (mon->qmp.qmp_requests->length >= QMP_REQ_QUEUE_LEN_MAX) {
             qemu_mutex_unlock(&mon->qmp.qmp_queue_lock);
+            /*
+             * FIXME @id's scope is just @mon, and broadcasting it is
+             * wrong.  If another monitor's client has a command with
+             * the same ID in flight, the event will incorrectly claim
+             * that command was dropped.
+             */
             qapi_event_send_command_dropped(id,
                                             COMMAND_DROP_REASON_QUEUE_FULL,
                                             &error_abort);
diff --git a/qapi/misc.json b/qapi/misc.json
index 0446c3e48e..74cd97f237 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -3454,6 +3454,9 @@
 # only be dropped when the oob capability is enabled.
 #
 # @id: The dropped command's "id" field.
+# FIXME Broken by design.  Events are broadcast to all monitors.  If
+# another monitor's client has a command with the same ID in flight,
+# the event will incorrectly claim that command was dropped.
 #
 # @reason: The reason why the command is dropped.
 #
-- 
2.17.1

Re: [Qemu-devel] [PATCH 04/32] qmp: Document COMMAND_DROPPED design flaw

[Eric Blake]

On 07/02/2018 11:21 AM, Markus Armbruster wrote:
> Events are broadcast to all monitors.  If another monitor's client has
> a command with the same ID in flight, the event will incorrectly claim
> that command was dropped.  This must be fixed before out-of-band
> execution can graduate from "experimental".
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c      | 6 ++++++
>   qapi/misc.json | 3 +++
>   2 files changed, 9 insertions(+)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

(Seems like we'll get this bug fixed in time for 3.0, even if it misses 
soft freeze)

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 05/32] qmp: Get rid of x-oob-test command

[Markus Armbruster]

tests/qmp-test tests an out-of-band command overtaking a slow in-band
command.  To do that, it needs:

1. An in-band command that *reliably* takes long enough to be
   overtaken.

2. An out-of-band command to do the overtaking.

3. To avoid delays, a way to make the in-band command complete quickly
   after it was overtaken.

To satisfy these needs, commit 469638f9cb3 provides the rather
peculiar oob-capable QMP command x-oob-test:

* With "lock": true, it waits for a global semaphore.

* With "lock": false, it signals the global semaphore.

To satisfy 1., the test runs x-oob-test in-band with "lock": true.
To satisfy 2. and 3., it runs x-oob-test out-of-band with "lock": false.

Note that waiting for a semaphore violates the rules for oob-capable
commands.  Running x-oob-test with "lock": true hangs the monitor
until you run x-oob-test with "lock": false on another monitor (which
you might not have set up).

Having an externally visible QMP command that may hang the monitor is
not nice.  Let's apply a little more ingenuity to the problem.  Idea:
have an existing command block on reading a FIFO special file, unblock
it by opening the FIFO for writing.

For 1., use

    {"execute": "blockdev-add",  "id": ID1,
     "arguments": {
        "driver": "blkdebug", "node-name": ID1, "config": FIFO,
        "image": { "driver": "null-co"}}}

where ID1 is an arbitrary string, and FIFO is the name of the FIFO.

For 2., use

    {"execute": "migrate-pause", "id": ID2, "control": {"run-oob": true}}

where ID2 is a different arbitrary string.  Since there's no migration
to pause, the command will fail, but that's fine.

For 3., open FIFO for writing.

Drop QMP command x-oob-test.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/misc.json   | 18 ----------
 qmp.c            | 16 ---------
 tests/qmp-test.c | 93 ++++++++++++++++++++++++++++++++----------------
 3 files changed, 63 insertions(+), 64 deletions(-)

diff --git a/qapi/misc.json b/qapi/misc.json
index 74cd97f237..f1860418e8 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -3472,24 +3472,6 @@
 { 'event': 'COMMAND_DROPPED' ,
   'data': { 'id': 'any', 'reason': 'CommandDropReason' } }
 
-##
-# @x-oob-test:
-#
-# Test OOB functionality.  When sending this command with lock=true,
-# it'll try to hang the dispatcher.  When sending it with lock=false,
-# it'll try to notify the locked thread to continue.  Note: it should
-# only be used by QMP test program rather than anything else.
-#
-# Since: 2.12
-#
-# Example:
-#
-# { "execute": "x-oob-test",
-#   "arguments": { "lock": true } }
-##
-{ 'command': 'x-oob-test', 'data' : { 'lock': 'bool' },
-  'allow-oob': true }
-
 ##
 # @set-numa-node:
 #
diff --git a/qmp.c b/qmp.c
index 73e46d795f..62325ac6aa 100644
--- a/qmp.c
+++ b/qmp.c
@@ -775,19 +775,3 @@ MemoryInfo *qmp_query_memory_size_summary(Error **errp)
 
     return mem_info;
 }
-
-static QemuSemaphore x_oob_test_sem;
-
-static void __attribute__((constructor)) x_oob_test_init(void)
-{
-    qemu_sem_init(&x_oob_test_sem, 0);
-}
-
-void qmp_x_oob_test(bool lock, Error **errp)
-{
-    if (lock) {
-        qemu_sem_wait(&x_oob_test_sem);
-    } else {
-        qemu_sem_post(&x_oob_test_sem);
-    }
-}
diff --git a/tests/qmp-test.c b/tests/qmp-test.c
index 5206b14ca6..b4e54e8970 100644
--- a/tests/qmp-test.c
+++ b/tests/qmp-test.c
@@ -135,16 +135,64 @@ static void test_qmp_protocol(void)
     qtest_quit(qts);
 }
 
-/* Tests for out-of-band support. */
+/* Out-of-band tests */
+
+char tmpdir[] = "/tmp/qmp-test-XXXXXX";
+char *fifo_name;
+
+static void setup_blocking_cmd(void)
+{
+    if (!mkdtemp(tmpdir)) {
+        g_error("mkdtemp: %s", strerror(errno));
+    }
+    fifo_name = g_strdup_printf("%s/fifo", tmpdir);
+    g_assert(!mkfifo(fifo_name, 0666));
+}
+
+static void cleanup_blocking_cmd(void)
+{
+    unlink(fifo_name);
+    rmdir(tmpdir);
+}
+
+static void send_cmd_that_blocks(QTestState *s, const char *id)
+{
+    qtest_async_qmp(s, "{ 'execute': 'blockdev-add',  'id': %s,"
+                    " 'arguments': {"
+                    " 'driver': 'blkdebug', 'node-name': %s,"
+                    " 'config': %s,"
+                    " 'image': { 'driver': 'null-co' } } }",
+                    id, id, fifo_name);
+}
+
+static void unblock_blocked_cmd(void)
+{
+    int fd = open(fifo_name, O_WRONLY);
+    g_assert(fd >= 0);
+    close(fd);
+}
+
+static void send_oob_cmd(QTestState *s, const char *id)
+{
+    qtest_async_qmp(s, "{ 'execute': 'migrate-pause', 'id': %s,"
+                    "  'control': { 'run-oob': true } }", id);
+}
+
+static void recv_cmd_id(QTestState *s, const char *id)
+{
+    QDict *resp = qtest_qmp_receive(s);
+
+    g_assert_cmpstr(qdict_get_try_str(resp, "id"), ==, id);
+    qobject_unref(resp);
+}
+
 static void test_qmp_oob(void)
 {
     QTestState *qts;
     QDict *resp, *q;
-    int acks = 0;
     const QListEntry *entry;
     QList *capabilities;
     QString *qstr;
-    const char *cmd_id;
 
     qts = qtest_init_without_qmp_handshake(true, common_args);
 
@@ -185,37 +233,20 @@ static void test_qmp_oob(void)
     g_assert(qdict_haskey(resp, "error"));
     qobject_unref(resp);
 
-    /*
-     * First send the "x-oob-test" command with lock=true and
-     * oob=false, it should hang the dispatcher and main thread;
-     * later, we send another lock=false with oob=true to continue
-     * that thread processing.  Finally we should receive replies from
-     * both commands.
-     */
-    qtest_async_qmp(qts,
-                    "{ 'execute': 'x-oob-test',"
-                    "  'arguments': { 'lock': true }, "
-                    "  'id': 'lock-cmd'}");
-    qtest_async_qmp(qts,
-                    "{ 'execute': 'x-oob-test', "
-                    "  'arguments': { 'lock': false }, "
-                    "  'control': { 'run-oob': true }, "
-                    "  'id': 'unlock-cmd' }");
-
-    /* Ignore all events.  Wait for 2 acks */
-    while (acks < 2) {
-        resp = qtest_qmp_receive(qts);
-        cmd_id = qdict_get_str(resp, "id");
-        if (!g_strcmp0(cmd_id, "lock-cmd") ||
-            !g_strcmp0(cmd_id, "unlock-cmd")) {
-            acks++;
-        }
-        qobject_unref(resp);
-    }
+    /* OOB command overtakes slow in-band command */
+    setup_blocking_cmd();
+    send_cmd_that_blocks(qts, "ib-blocks-1");
+    send_oob_cmd(qts, "oob-1");
+    recv_cmd_id(qts, "oob-1");
+    unblock_blocked_cmd();
+    recv_cmd_id(qts, "ib-blocks-1");
+    cleanup_blocking_cmd();
 
     qtest_quit(qts);
 }
 
+/* Query smoke tests */
+
 static int query_error_class(const char *cmd)
 {
     static struct {
@@ -392,6 +423,8 @@ static void add_query_tests(QmpSchema *schema)
     }
 }
 
+/* Preconfig tests */
+
 static void test_qmp_preconfig(void)
 {
     QDict *rsp, *ret;
-- 
2.17.1

Re: [Qemu-devel] [PATCH 05/32] qmp: Get rid of x-oob-test command

[Eric Blake]

On 07/02/2018 11:21 AM, Markus Armbruster wrote:
> tests/qmp-test tests an out-of-band command overtaking a slow in-band
> command.  To do that, it needs:
> 
> 1. An in-band command that *reliably* takes long enough to be
>     overtaken.
> 
> 2. An out-of-band command to do the overtaking.
> 
> 3. To avoid delays, a way to make the in-band command complete quickly
>     after it was overtaken.
> 
> To satisfy these needs, commit 469638f9cb3 provides the rather
> peculiar oob-capable QMP command x-oob-test:
> 
> * With "lock": true, it waits for a global semaphore.
> 
> * With "lock": false, it signals the global semaphore.
> 
> To satisfy 1., the test runs x-oob-test in-band with "lock": true.
> To satisfy 2. and 3., it runs x-oob-test out-of-band with "lock": false.
> 
> Note that waiting for a semaphore violates the rules for oob-capable
> commands.  Running x-oob-test with "lock": true hangs the monitor
> until you run x-oob-test with "lock": false on another monitor (which
> you might not have set up).
> 
> Having an externally visible QMP command that may hang the monitor is
> not nice.  Let's apply a little more ingenuity to the problem.  Idea:
> have an existing command block on reading a FIFO special file, unblock
> it by opening the FIFO for writing.
> 
> For 1., use
> 
>      {"execute": "blockdev-add",  "id": ID1,
>       "arguments": {
>          "driver": "blkdebug", "node-name": ID1, "config": FIFO,
>          "image": { "driver": "null-co"}}}

I like it!

> 
> where ID1 is an arbitrary string, and FIFO is the name of the FIFO.
> 
> For 2., use
> 
>      {"execute": "migrate-pause", "id": ID2, "control": {"run-oob": true}}
> 
> where ID2 is a different arbitrary string.  Since there's no migration
> to pause, the command will fail, but that's fine.

Maybe add:

that's fine, since instant failure is still a test of out-of-band 
responses overtaking in-band commands.

> 
> For 3., open FIFO for writing.
> 
> Drop QMP command x-oob-test.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   qapi/misc.json   | 18 ----------
>   qmp.c            | 16 ---------
>   tests/qmp-test.c | 93 ++++++++++++++++++++++++++++++++----------------
>   3 files changed, 63 insertions(+), 64 deletions(-)

And in about the same lines of code.

Reviewed-by: Eric Blake <eblake@redhat.com>

> +++ b/tests/qmp-test.c
> @@ -135,16 +135,64 @@ static void test_qmp_protocol(void)
>       qtest_quit(qts);
>   }
>   
> -/* Tests for out-of-band support. */
> +/* Out-of-band tests */
> +
> +char tmpdir[] = "/tmp/qmp-test-XXXXXX";
> +char *fifo_name;
> +
> +static void setup_blocking_cmd(void)
> +{
> +    if (!mkdtemp(tmpdir)) {
> +        g_error("mkdtemp: %s", strerror(errno));
> +    }
> +    fifo_name = g_strdup_printf("%s/fifo", tmpdir);
> +    g_assert(!mkfifo(fifo_name, 0666));

g_assert() can be compiled out; should the side-effect mkfifo() call be 
done separately from asserting that its return code is expected?


> +
> +static void send_oob_cmd(QTestState *s, const char *id)
> +{
> +    qtest_async_qmp(s, "{ 'execute': 'migrate-pause', 'id': %s,"
> +                    "  'control': { 'run-oob': true } }", id);
> +}

Worth a comment here that we expect this to fail, and really only care 
that the response overtakes the in-band message?

With those nits addressed,
Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH 05/32] qmp: Get rid of x-oob-test command

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 07/02/2018 11:21 AM, Markus Armbruster wrote:
>> tests/qmp-test tests an out-of-band command overtaking a slow in-band
>> command.  To do that, it needs:
>>
>> 1. An in-band command that *reliably* takes long enough to be
>>     overtaken.
>>
>> 2. An out-of-band command to do the overtaking.
>>
>> 3. To avoid delays, a way to make the in-band command complete quickly
>>     after it was overtaken.
>>
>> To satisfy these needs, commit 469638f9cb3 provides the rather
>> peculiar oob-capable QMP command x-oob-test:
>>
>> * With "lock": true, it waits for a global semaphore.
>>
>> * With "lock": false, it signals the global semaphore.
>>
>> To satisfy 1., the test runs x-oob-test in-band with "lock": true.
>> To satisfy 2. and 3., it runs x-oob-test out-of-band with "lock": false.
>>
>> Note that waiting for a semaphore violates the rules for oob-capable
>> commands.  Running x-oob-test with "lock": true hangs the monitor
>> until you run x-oob-test with "lock": false on another monitor (which
>> you might not have set up).
>>
>> Having an externally visible QMP command that may hang the monitor is
>> not nice.  Let's apply a little more ingenuity to the problem.  Idea:
>> have an existing command block on reading a FIFO special file, unblock
>> it by opening the FIFO for writing.
>>
>> For 1., use
>>
>>      {"execute": "blockdev-add",  "id": ID1,
>>       "arguments": {
>>          "driver": "blkdebug", "node-name": ID1, "config": FIFO,
>>          "image": { "driver": "null-co"}}}
>
> I like it!
>
>>
>> where ID1 is an arbitrary string, and FIFO is the name of the FIFO.
>>
>> For 2., use
>>
>>      {"execute": "migrate-pause", "id": ID2, "control": {"run-oob": true}}
>>
>> where ID2 is a different arbitrary string.  Since there's no migration
>> to pause, the command will fail, but that's fine.
>
> Maybe add:
>
> that's fine, since instant failure is still a test of out-of-band
> responses overtaking in-band commands.

Sold.

>>
>> For 3., open FIFO for writing.
>>
>> Drop QMP command x-oob-test.
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>   qapi/misc.json   | 18 ----------
>>   qmp.c            | 16 ---------
>>   tests/qmp-test.c | 93 ++++++++++++++++++++++++++++++++----------------
>>   3 files changed, 63 insertions(+), 64 deletions(-)
>
> And in about the same lines of code.

/me bows :)

> Reviewed-by: Eric Blake <eblake@redhat.com>
>
>> +++ b/tests/qmp-test.c
>> @@ -135,16 +135,64 @@ static void test_qmp_protocol(void)
>>       qtest_quit(qts);
>>   }
>>   -/* Tests for out-of-band support. */
>> +/* Out-of-band tests */
>> +
>> +char tmpdir[] = "/tmp/qmp-test-XXXXXX";
>> +char *fifo_name;
>> +
>> +static void setup_blocking_cmd(void)
>> +{
>> +    if (!mkdtemp(tmpdir)) {
>> +        g_error("mkdtemp: %s", strerror(errno));
>> +    }
>> +    fifo_name = g_strdup_printf("%s/fifo", tmpdir);
>> +    g_assert(!mkfifo(fifo_name, 0666));
>
> g_assert() can be compiled out; should the side-effect mkfifo() call
> be done separately from asserting that its return code is expected?

We don't support compiling them out (see the note in osdep.h), but
you're right, we should do this properly.

>> +
>> +static void send_oob_cmd(QTestState *s, const char *id)
>> +{
>> +    qtest_async_qmp(s, "{ 'execute': 'migrate-pause', 'id': %s,"
>> +                    "  'control': { 'run-oob': true } }", id);
>> +}
>
> Worth a comment here that we expect this to fail, and really only care
> that the response overtakes the in-band message?

Yes.

> With those nits addressed,
> Reviewed-by: Eric Blake <eblake@redhat.com>

Thanks!

[Qemu-devel] [PATCH 06/32] tests/qmp-test: Test in-band command doesn't overtake

[Markus Armbruster]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 tests/qmp-test.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tests/qmp-test.c b/tests/qmp-test.c
index b4e54e8970..3932901146 100644
--- a/tests/qmp-test.c
+++ b/tests/qmp-test.c
@@ -236,10 +236,12 @@ static void test_qmp_oob(void)
     /* OOB command overtakes slow in-band command */
     setup_blocking_cmd();
     send_cmd_that_blocks(qts, "ib-blocks-1");
+    qtest_async_qmp(qts, "{ 'execute': 'query-name', 'id': 'ib-quick-1' }");
     send_oob_cmd(qts, "oob-1");
     recv_cmd_id(qts, "oob-1");
     unblock_blocked_cmd();
     recv_cmd_id(qts, "ib-blocks-1");
+    recv_cmd_id(qts, "ib-quick-1");
     cleanup_blocking_cmd();
 
     qtest_quit(qts);
-- 
2.17.1

Re: [Qemu-devel] [PATCH 06/32] tests/qmp-test: Test in-band command doesn't overtake

[Eric Blake]

On 07/02/2018 11:21 AM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   tests/qmp-test.c | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/tests/qmp-test.c b/tests/qmp-test.c
> index b4e54e8970..3932901146 100644
> --- a/tests/qmp-test.c
> +++ b/tests/qmp-test.c
> @@ -236,10 +236,12 @@ static void test_qmp_oob(void)
>       /* OOB command overtakes slow in-band command */
>       setup_blocking_cmd();
>       send_cmd_that_blocks(qts, "ib-blocks-1");
> +    qtest_async_qmp(qts, "{ 'execute': 'query-name', 'id': 'ib-quick-1' }");
>       send_oob_cmd(qts, "oob-1");
>       recv_cmd_id(qts, "oob-1");
>       unblock_blocked_cmd();
>       recv_cmd_id(qts, "ib-blocks-1");
> +    recv_cmd_id(qts, "ib-quick-1");
>       cleanup_blocking_cmd();

Nice improvement.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 07/32] qmp: Make "id" optional again even in "oob" monitors

[Markus Armbruster]

Commit cf869d53172 "qmp: support out-of-band (oob) execution" made
"id" mandatory for all commands when the client accepted capability
"oob".  This is rather onerous when you play with QMP by hand, and
unnecessarily so: only out-of-band commands need an ID for reliable
matching of response to command.

Revert that part of commit cf869d53172 for now.  We may still make
"id" mandatory for out-of-band commands.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 docs/interop/qmp-spec.txt | 9 +++------
 monitor.c                 | 7 -------
 2 files changed, 3 insertions(+), 13 deletions(-)

diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.txt
index e5f8116c54..794c95838f 100644
--- a/docs/interop/qmp-spec.txt
+++ b/docs/interop/qmp-spec.txt
@@ -103,16 +103,13 @@ The format for command execution is:
   required. Each command documents what contents will be considered
   valid when handling the json-argument
 - The "id" member is a transaction identification associated with the
-  command execution.  It is required for all commands if the OOB -
-  capability was enabled at startup, and optional otherwise.  The same
-  "id" field will be part of the response if provided.  The "id"
-  member can be any json-value.  A json-number incremented for each
-  successive command works fine.
+  command execution, it is optional and will be part of the response
+  if provided.  The "id" member can be any json-value.  A json-number
+  incremented for each successive command works fine.
 - The optional "control" member further specifies how the command is
   to be executed.  Currently, its only member is optional "run-oob".
   See section "2.3.1 Out-of-band execution" for details.
 
-
 2.3.1 Out-of-band execution
 ---------------------------
 
diff --git a/monitor.c b/monitor.c
index 96e87d8664..b7d74b01b4 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4291,13 +4291,6 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
 
     id = qdict_get(qdict, "id");
 
-    /* When OOB is enabled, the "id" field is mandatory. */
-    if (qmp_oob_enabled(mon) && !id) {
-        error_setg(&err, "Out-of-band capability requires that "
-                   "every command contains an 'id' field");
-        goto err;
-    }
-
     req_obj = g_new0(QMPRequest, 1);
     req_obj->mon = mon;
     req_obj->id = qobject_ref(id);
-- 
2.17.1

Re: [Qemu-devel] [PATCH 07/32] qmp: Make "id" optional again even in "oob" monitors

[Eric Blake]

On 07/02/2018 11:21 AM, Markus Armbruster wrote:
> Commit cf869d53172 "qmp: support out-of-band (oob) execution" made
> "id" mandatory for all commands when the client accepted capability
> "oob".  This is rather onerous when you play with QMP by hand, and
> unnecessarily so: only out-of-band commands need an ID for reliable
> matching of response to command.
> 
> Revert that part of commit cf869d53172 for now.  We may still make
> "id" mandatory for out-of-band commands.

Fair enough for now (I still think mandatory "id" for oob commands is 
worthwhile, but I also think "exec-oob" is worthwhile, and it's probably 
easier to write the logic for mandating "id" after that tweak).

> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   docs/interop/qmp-spec.txt | 9 +++------
>   monitor.c                 | 7 -------
>   2 files changed, 3 insertions(+), 13 deletions(-)
> 

> +++ b/docs/interop/qmp-spec.txt
> @@ -103,16 +103,13 @@ The format for command execution is:
>     required. Each command documents what contents will be considered
>     valid when handling the json-argument
>   - The "id" member is a transaction identification associated with the
> -  command execution.  It is required for all commands if the OOB -
> -  capability was enabled at startup, and optional otherwise.  The same
> -  "id" field will be part of the response if provided.  The "id"
> -  member can be any json-value.  A json-number incremented for each
> -  successive command works fine.
> +  command execution, it is optional and will be part of the response
> +  if provided.  The "id" member can be any json-value.  A json-number
> +  incremented for each successive command works fine.
>   - The optional "control" member further specifies how the command is
>     to be executed.  Currently, its only member is optional "run-oob".
>     See section "2.3.1 Out-of-band execution" for details.

We should STILL recommend that a client should use "id" when using oob, 
whether or not we mandate it later.

>   
> -
>   2.3.1 Out-of-band execution
>   ---------------------------
>   
> diff --git a/monitor.c b/monitor.c
> index 96e87d8664..b7d74b01b4 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -4291,13 +4291,6 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
>   
>       id = qdict_get(qdict, "id");
>   
> -    /* When OOB is enabled, the "id" field is mandatory. */
> -    if (qmp_oob_enabled(mon) && !id) {
> -        error_setg(&err, "Out-of-band capability requires that "
> -                   "every command contains an 'id' field");
> -        goto err;
> -    }

I can live with this hunk right away, but the documentation feels weak 
enough that I'm reluctant to give R-b to the patch as-is

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH 07/32] qmp: Make "id" optional again even in "oob" monitors

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 07/02/2018 11:21 AM, Markus Armbruster wrote:
>> Commit cf869d53172 "qmp: support out-of-band (oob) execution" made
>> "id" mandatory for all commands when the client accepted capability
>> "oob".  This is rather onerous when you play with QMP by hand, and
>> unnecessarily so: only out-of-band commands need an ID for reliable
>> matching of response to command.
>>
>> Revert that part of commit cf869d53172 for now.  We may still make
>> "id" mandatory for out-of-band commands.
>
> Fair enough for now (I still think mandatory "id" for oob commands is
> worthwhile, but I also think "exec-oob" is worthwhile, and it's
> probably easier to write the logic for mandating "id" after that
> tweak).
>
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>   docs/interop/qmp-spec.txt | 9 +++------
>>   monitor.c                 | 7 -------
>>   2 files changed, 3 insertions(+), 13 deletions(-)
>>
>
>> +++ b/docs/interop/qmp-spec.txt
>> @@ -103,16 +103,13 @@ The format for command execution is:
>>     required. Each command documents what contents will be considered
>>     valid when handling the json-argument
>>   - The "id" member is a transaction identification associated with the
>> -  command execution.  It is required for all commands if the OOB -
>> -  capability was enabled at startup, and optional otherwise.  The same
>> -  "id" field will be part of the response if provided.  The "id"
>> -  member can be any json-value.  A json-number incremented for each
>> -  successive command works fine.
>> +  command execution, it is optional and will be part of the response
>> +  if provided.  The "id" member can be any json-value.  A json-number
>> +  incremented for each successive command works fine.
>>   - The optional "control" member further specifies how the command is
>>     to be executed.  Currently, its only member is optional "run-oob".
>>     See section "2.3.1 Out-of-band execution" for details.
>
> We should STILL recommend that a client should use "id" when using
> oob, whether or not we mandate it later.

Yes.  I'll take care of it in v2.

>
>>   -
>>   2.3.1 Out-of-band execution
>>   ---------------------------
>>   diff --git a/monitor.c b/monitor.c
>> index 96e87d8664..b7d74b01b4 100644
>> --- a/monitor.c
>> +++ b/monitor.c
>> @@ -4291,13 +4291,6 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
>>         id = qdict_get(qdict, "id");
>>   -    /* When OOB is enabled, the "id" field is mandatory. */
>> -    if (qmp_oob_enabled(mon) && !id) {
>> -        error_setg(&err, "Out-of-band capability requires that "
>> -                   "every command contains an 'id' field");
>> -        goto err;
>> -    }
>
> I can live with this hunk right away, but the documentation feels weak
> enough that I'm reluctant to give R-b to the patch as-is

Re: [Qemu-devel] [PATCH 07/32] qmp: Make "id" optional again even in "oob" monitors

[Peter Xu]

On Mon, Jul 02, 2018 at 06:21:53PM +0200, Markus Armbruster wrote:
> Commit cf869d53172 "qmp: support out-of-band (oob) execution" made
> "id" mandatory for all commands when the client accepted capability
> "oob".  This is rather onerous when you play with QMP by hand, and
> unnecessarily so: only out-of-band commands need an ID for reliable
> matching of response to command.
> 
> Revert that part of commit cf869d53172 for now.  We may still make
> "id" mandatory for out-of-band commands.

This change should be okay with current implementation when
out-of-band commands are still in order themselves, though I'm still
not that confident on whether we really want this change if only for
the sake of easier usage for human beings.

If we see Libvirt, the real player for QMP - it has the "id" field
even for in-band commands always.  I'd say the "id" field is really
helpful for machines, though not that friendly to us.

Basically I'll read it as: machines like "id"s, humans hate "id"s.
And QMP is Qemu Machine Protocol after all... so not sure whether
it'll be good we change that for us humans.

Regards,

-- 
Peter Xu

Re: [Qemu-devel] [PATCH 07/32] qmp: Make "id" optional again even in "oob" monitors

[Markus Armbruster]

Peter Xu <peterx@redhat.com> writes:

> On Mon, Jul 02, 2018 at 06:21:53PM +0200, Markus Armbruster wrote:
>> Commit cf869d53172 "qmp: support out-of-band (oob) execution" made
>> "id" mandatory for all commands when the client accepted capability
>> "oob".  This is rather onerous when you play with QMP by hand, and
>> unnecessarily so: only out-of-band commands need an ID for reliable
>> matching of response to command.
>> 
>> Revert that part of commit cf869d53172 for now.  We may still make
>> "id" mandatory for out-of-band commands.
>
> This change should be okay with current implementation when
> out-of-band commands are still in order themselves, though I'm still
> not that confident on whether we really want this change if only for
> the sake of easier usage for human beings.
>
> If we see Libvirt, the real player for QMP - it has the "id" field
> even for in-band commands always.  I'd say the "id" field is really
> helpful for machines, though not that friendly to us.
>
> Basically I'll read it as: machines like "id"s, humans hate "id"s.
> And QMP is Qemu Machine Protocol after all... so not sure whether
> it'll be good we change that for us humans.

"id" being optional doesn't hurt libvirt in any way.  Thus, I see no
need to inconvenience humans.

Daniel has argued[*] for making "id" mandatory with OOB commands.  I'm
not rejecting that argument.  But I needed to get this out in a hurry,
and simply reverting something is quicker than debating and implementing
an improvement.  There's still time to tweak this before the release.


[*] Message-ID: <20180628120044.GF3513@redhat.com>
https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg08322.html

Re: [Qemu-devel] [PATCH 07/32] qmp: Make "id" optional again even in "oob" monitors

[Peter Xu]

On Tue, Jul 03, 2018 at 08:14:35AM +0200, Markus Armbruster wrote:
> Peter Xu <peterx@redhat.com> writes:
> 
> > On Mon, Jul 02, 2018 at 06:21:53PM +0200, Markus Armbruster wrote:
> >> Commit cf869d53172 "qmp: support out-of-band (oob) execution" made
> >> "id" mandatory for all commands when the client accepted capability
> >> "oob".  This is rather onerous when you play with QMP by hand, and
> >> unnecessarily so: only out-of-band commands need an ID for reliable
> >> matching of response to command.
> >> 
> >> Revert that part of commit cf869d53172 for now.  We may still make
> >> "id" mandatory for out-of-band commands.
> >
> > This change should be okay with current implementation when
> > out-of-band commands are still in order themselves, though I'm still
> > not that confident on whether we really want this change if only for
> > the sake of easier usage for human beings.
> >
> > If we see Libvirt, the real player for QMP - it has the "id" field
> > even for in-band commands always.  I'd say the "id" field is really
> > helpful for machines, though not that friendly to us.
> >
> > Basically I'll read it as: machines like "id"s, humans hate "id"s.
> > And QMP is Qemu Machine Protocol after all... so not sure whether
> > it'll be good we change that for us humans.
> 
> "id" being optional doesn't hurt libvirt in any way.  Thus, I see no
> need to inconvenience humans.
> 
> Daniel has argued[*] for making "id" mandatory with OOB commands.  I'm
> not rejecting that argument.  But I needed to get this out in a hurry,
> and simply reverting something is quicker than debating and implementing
> an improvement.  There's still time to tweak this before the release.
> 
> 
> [*] Message-ID: <20180628120044.GF3513@redhat.com>
> https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg08322.html

Okay, it's fine to me then, especially if Daniel won't have other
opinions.

Regards,

-- 
Peter Xu

Re: [Qemu-devel] [PATCH 07/32] qmp: Make "id" optional again even in "oob" monitors

[Daniel P. Berrangé]

On Tue, Jul 03, 2018 at 08:14:35AM +0200, Markus Armbruster wrote:
> Peter Xu <peterx@redhat.com> writes:
> 
> > On Mon, Jul 02, 2018 at 06:21:53PM +0200, Markus Armbruster wrote:
> >> Commit cf869d53172 "qmp: support out-of-band (oob) execution" made
> >> "id" mandatory for all commands when the client accepted capability
> >> "oob".  This is rather onerous when you play with QMP by hand, and
> >> unnecessarily so: only out-of-band commands need an ID for reliable
> >> matching of response to command.
> >> 
> >> Revert that part of commit cf869d53172 for now.  We may still make
> >> "id" mandatory for out-of-band commands.
> >
> > This change should be okay with current implementation when
> > out-of-band commands are still in order themselves, though I'm still
> > not that confident on whether we really want this change if only for
> > the sake of easier usage for human beings.
> >
> > If we see Libvirt, the real player for QMP - it has the "id" field
> > even for in-band commands always.  I'd say the "id" field is really
> > helpful for machines, though not that friendly to us.
> >
> > Basically I'll read it as: machines like "id"s, humans hate "id"s.
> > And QMP is Qemu Machine Protocol after all... so not sure whether
> > it'll be good we change that for us humans.
> 
> "id" being optional doesn't hurt libvirt in any way.  Thus, I see no
> need to inconvenience humans.
> 
> Daniel has argued[*] for making "id" mandatory with OOB commands.  I'm
> not rejecting that argument.  But I needed to get this out in a hurry,
> and simply reverting something is quicker than debating and implementing
> an improvement.  There's still time to tweak this before the release.

I'm fine with this - I agree that it is more important to avoid creating
a regression, than to enforce "id" for OOB. We've still documented that
OOB will require "id", so it would not be a surprise once we enforce it
later. 

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Qemu-devel] [PATCH 08/32] tests/test-qga: Demonstrate the guest-agent ignores "id"

[Markus Armbruster]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 tests/test-qga.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/tests/test-qga.c b/tests/test-qga.c
index 30c9643257..4edd35535d 100644
--- a/tests/test-qga.c
+++ b/tests/test-qga.c
@@ -227,6 +227,22 @@ static void test_qga_ping(gconstpointer fix)
     qobject_unref(ret);
 }
 
+static void test_qga_invalid_id(gconstpointer fix)
+{
+    /* FIXME "id" is ignored; it should be rejected */
+    const TestFixture *fixture = fix;
+    QDict *ret, *val;
+
+    ret = qmp_fd(fixture->fd, "{'execute': 'guest-ping', 'id': 1}");
+    g_assert_nonnull(ret);
+    qmp_assert_no_error(ret);
+
+    val = qdict_get_qdict(ret, "return");
+    g_assert(!qdict_haskey(val, "id"));
+
+    qobject_unref(ret);
+}
+
 static void test_qga_invalid_args(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
@@ -934,6 +950,7 @@ int main(int argc, char **argv)
     g_test_add_data_func("/qga/file-ops", &fix, test_qga_file_ops);
     g_test_add_data_func("/qga/file-write-read", &fix, test_qga_file_write_read);
     g_test_add_data_func("/qga/get-time", &fix, test_qga_get_time);
+    g_test_add_data_func("/qga/invalid-id", &fix, test_qga_invalid_id);
     g_test_add_data_func("/qga/invalid-cmd", &fix, test_qga_invalid_cmd);
     g_test_add_data_func("/qga/invalid-args", &fix, test_qga_invalid_args);
     g_test_add_data_func("/qga/fsfreeze-status", &fix,
-- 
2.17.1

Re: [Qemu-devel] [PATCH 08/32] tests/test-qga: Demonstrate the guest-agent ignores "id"

[Eric Blake]

On 07/02/2018 11:21 AM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   tests/test-qga.c | 17 +++++++++++++++++
>   1 file changed, 17 insertions(+)
> 
> diff --git a/tests/test-qga.c b/tests/test-qga.c
> index 30c9643257..4edd35535d 100644
> --- a/tests/test-qga.c
> +++ b/tests/test-qga.c
> @@ -227,6 +227,22 @@ static void test_qga_ping(gconstpointer fix)
>       qobject_unref(ret);
>   }
>   
> +static void test_qga_invalid_id(gconstpointer fix)
> +{
> +    /* FIXME "id" is ignored; it should be rejected */

I might have written:

/* FIXME "id" is ignored; it should either be repeated in the reply, or 
rejected on input */

> +    const TestFixture *fixture = fix;
> +    QDict *ret, *val;
> +
> +    ret = qmp_fd(fixture->fd, "{'execute': 'guest-ping', 'id': 1}");
> +    g_assert_nonnull(ret);
> +    qmp_assert_no_error(ret);
> +
> +    val = qdict_get_qdict(ret, "return");
> +    g_assert(!qdict_haskey(val, "id"));

This is documenting what currently happens, along with the FIXME showing 
that it is not ideal, so whether or not you improve the comment:

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH 08/32] tests/test-qga: Demonstrate the guest-agent ignores "id"

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 07/02/2018 11:21 AM, Markus Armbruster wrote:
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>   tests/test-qga.c | 17 +++++++++++++++++
>>   1 file changed, 17 insertions(+)
>>
>> diff --git a/tests/test-qga.c b/tests/test-qga.c
>> index 30c9643257..4edd35535d 100644
>> --- a/tests/test-qga.c
>> +++ b/tests/test-qga.c
>> @@ -227,6 +227,22 @@ static void test_qga_ping(gconstpointer fix)
>>       qobject_unref(ret);
>>   }
>>   +static void test_qga_invalid_id(gconstpointer fix)
>> +{
>> +    /* FIXME "id" is ignored; it should be rejected */
>
> I might have written:
>
> /* FIXME "id" is ignored; it should either be repeated in the reply,
> or rejected on input */

Sold.

>> +    const TestFixture *fixture = fix;
>> +    QDict *ret, *val;
>> +
>> +    ret = qmp_fd(fixture->fd, "{'execute': 'guest-ping', 'id': 1}");
>> +    g_assert_nonnull(ret);
>> +    qmp_assert_no_error(ret);
>> +
>> +    val = qdict_get_qdict(ret, "return");
>> +    g_assert(!qdict_haskey(val, "id"));
>
> This is documenting what currently happens, along with the FIXME
> showing that it is not ideal, so whether or not you improve the
> comment:
>
> Reviewed-by: Eric Blake <eblake@redhat.com>

Thanks!

[Qemu-devel] [PATCH 09/32] qmp qemu-ga: Revert change that accidentally made qemu-ga accept "id"

[Markus Armbruster]

Commit cf869d53172 "qmp: support out-of-band (oob) execution" changed
how we check "id":

    Note that in the patch I exported qmp_dispatch_check_obj() to be
    used to check the request earlier, and at the same time allowed
    "id" field to be there since actually we always allow that.

The part after "and" is ill-advised: it makes qemu-ga accept and
ignore "id".  Revert.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c           | 14 ++++++++------
 qapi/qmp-dispatch.c |  2 --
 tests/test-qga.c    | 10 +++++-----
 3 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/monitor.c b/monitor.c
index b7d74b01b4..4659399b55 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4263,7 +4263,7 @@ static void monitor_qmp_bh_dispatcher(void *data)
 static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
 {
     QObject *req, *id = NULL;
-    QDict *qdict = NULL;
+    QDict *qdict;
     MonitorQMP *mon_qmp = container_of(parser, MonitorQMP, parser);
     Monitor *mon = container_of(mon_qmp, Monitor, qmp);
     Error *err = NULL;
@@ -4278,6 +4278,12 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         goto err;
     }
 
+    qdict = qobject_to(QDict, req);
+    if (qdict) {
+        id = qobject_ref(qdict_get(qdict, "id"));
+        qdict_del(qdict, "id");
+    } /* else will fail qmp_dispatch() */
+
     /* Check against the request in general layout */
     qdict = qmp_dispatch_check_obj(req, &err);
     if (!qdict) {
@@ -4289,16 +4295,12 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         goto err;
     }
 
-    id = qdict_get(qdict, "id");
-
     req_obj = g_new0(QMPRequest, 1);
     req_obj->mon = mon;
-    req_obj->id = qobject_ref(id);
+    req_obj->id = id;
     req_obj->req = req;
     req_obj->need_resume = false;
 
-    qdict_del(qdict, "id");
-
     if (qmp_is_oob(qdict)) {
         /* Out-of-band (OOB) requests are executed directly in parser. */
         trace_monitor_qmp_cmd_out_of_band(qobject_get_try_str(req_obj->id)
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 935f9e159c..3d5d5e110f 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -52,8 +52,6 @@ QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
                            "QMP input member 'arguments' must be an object");
                 return NULL;
             }
-        } else if (!strcmp(arg_name, "id")) {
-            continue;
         } else if (!strcmp(arg_name, "control")) {
             if (qobject_type(arg_obj) != QTYPE_QDICT) {
                 error_setg(errp,
diff --git a/tests/test-qga.c b/tests/test-qga.c
index 4edd35535d..564a4594b5 100644
--- a/tests/test-qga.c
+++ b/tests/test-qga.c
@@ -229,16 +229,16 @@ static void test_qga_ping(gconstpointer fix)
 
 static void test_qga_invalid_id(gconstpointer fix)
 {
-    /* FIXME "id" is ignored; it should be rejected */
     const TestFixture *fixture = fix;
-    QDict *ret, *val;
+    QDict *ret, *error;
+    const char *class;
 
     ret = qmp_fd(fixture->fd, "{'execute': 'guest-ping', 'id': 1}");
     g_assert_nonnull(ret);
-    qmp_assert_no_error(ret);
 
-    val = qdict_get_qdict(ret, "return");
-    g_assert(!qdict_haskey(val, "id"));
+    error = qdict_get_qdict(ret, "error");
+    class = qdict_get_try_str(error, "class");
+    g_assert_cmpstr(class, ==, "GenericError");
 
     qobject_unref(ret);
 }
-- 
2.17.1

Re: [Qemu-devel] [PATCH 09/32] qmp qemu-ga: Revert change that accidentally made qemu-ga accept "id"

[Eric Blake]

On 07/02/2018 11:21 AM, Markus Armbruster wrote:
> Commit cf869d53172 "qmp: support out-of-band (oob) execution" changed
> how we check "id":
> 
>      Note that in the patch I exported qmp_dispatch_check_obj() to be
>      used to check the request earlier, and at the same time allowed
>      "id" field to be there since actually we always allow that.
> 
> The part after "and" is ill-advised: it makes qemu-ga accept and
> ignore "id".  Revert.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c           | 14 ++++++++------
>   qapi/qmp-dispatch.c |  2 --
>   tests/test-qga.c    | 10 +++++-----
>   3 files changed, 13 insertions(+), 13 deletions(-)
> 

It might be nicer to teach qga to reply with id. But for the purposes of 
fixing regressions, this is reasonable.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 10/32] tests/test-qga: Demonstrate the guest-agent ignores "control"

[Markus Armbruster]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 tests/test-qga.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/tests/test-qga.c b/tests/test-qga.c
index 564a4594b5..2e9e0f73bb 100644
--- a/tests/test-qga.c
+++ b/tests/test-qga.c
@@ -243,6 +243,22 @@ static void test_qga_invalid_id(gconstpointer fix)
     qobject_unref(ret);
 }
 
+static void test_qga_invalid_oob(gconstpointer fix)
+{
+    /* FIXME "control" is ignored; it should be rejected */
+    const TestFixture *fixture = fix;
+    QDict *ret;
+
+    ret = qmp_fd(fixture->fd, "{'execute': 'guest-ping',"
+                 " 'control': {'run-oob': true}}");
+    g_assert_nonnull(ret);
+    qmp_assert_no_error(ret);
+
+    qdict_get_qdict(ret, "return");
+
+    qobject_unref(ret);
+}
+
 static void test_qga_invalid_args(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
@@ -951,6 +967,7 @@ int main(int argc, char **argv)
     g_test_add_data_func("/qga/file-write-read", &fix, test_qga_file_write_read);
     g_test_add_data_func("/qga/get-time", &fix, test_qga_get_time);
     g_test_add_data_func("/qga/invalid-id", &fix, test_qga_invalid_id);
+    g_test_add_data_func("/qga/invalid-oob", &fix, test_qga_invalid_oob);
     g_test_add_data_func("/qga/invalid-cmd", &fix, test_qga_invalid_cmd);
     g_test_add_data_func("/qga/invalid-args", &fix, test_qga_invalid_args);
     g_test_add_data_func("/qga/fsfreeze-status", &fix,
-- 
2.17.1

Re: [Qemu-devel] [PATCH 10/32] tests/test-qga: Demonstrate the guest-agent ignores "control"

[Eric Blake]

On 07/02/2018 11:21 AM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   tests/test-qga.c | 17 +++++++++++++++++
>   1 file changed, 17 insertions(+)

Well, if we get rid of 'control' in favor of 'exec-oob', this test would 
still have to test that QGA doesn't support oob.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH 10/32] tests/test-qga: Demonstrate the guest-agent ignores "control"

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 07/02/2018 11:21 AM, Markus Armbruster wrote:
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>   tests/test-qga.c | 17 +++++++++++++++++
>>   1 file changed, 17 insertions(+)
>
> Well, if we get rid of 'control' in favor of 'exec-oob', this test
> would still have to test that QGA doesn't support oob.
>
> Reviewed-by: Eric Blake <eblake@redhat.com>

PATCH 12 gets rid of it, but the test still serves to cover "random crap
gets rejected" then.

[Qemu-devel] [PATCH 11/32] qmp qemu-ga: Fix qemu-ga not to accept "control"

[Markus Armbruster]

Commit cf869d53172 "qmp: support out-of-band (oob) execution"
accidentally made qemu-ga accept and ignore "control".  Fix that.

Out-of-band execution in a monitor that doesn't support it now fails
with

    {"error": {"class": "GenericError", "desc": "QMP input member 'control' is unexpected"}}

instead of

    {"error": {"class": "GenericError", "desc": "Please enable out-of-band first for the session during capabilities negotiation"}}

The old description is suboptimal when out-of-band cannot not be
enabled, or the command doesn't support out-of-band execution.

The new description is a bit unspecific, but it'll do.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 include/qapi/qmp/dispatch.h |  6 ++++--
 monitor.c                   |  9 ++-------
 qapi/qmp-dispatch.c         | 14 ++++++++------
 qga/main.c                  |  2 +-
 tests/test-qga.c            |  9 +++++----
 tests/test-qmp-cmds.c       |  8 ++++----
 6 files changed, 24 insertions(+), 24 deletions(-)

diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
index b366bb48bd..303a15ba84 100644
--- a/include/qapi/qmp/dispatch.h
+++ b/include/qapi/qmp/dispatch.h
@@ -41,7 +41,6 @@ void qmp_register_command(QmpCommandList *cmds, const char *name,
                           QmpCommandFunc *fn, QmpCommandOptions options);
 void qmp_unregister_command(QmpCommandList *cmds, const char *name);
 QmpCommand *qmp_find_command(QmpCommandList *cmds, const char *name);
-QObject *qmp_dispatch(QmpCommandList *cmds, QObject *request);
 void qmp_disable_command(QmpCommandList *cmds, const char *name);
 void qmp_enable_command(QmpCommandList *cmds, const char *name);
 
@@ -49,7 +48,10 @@ bool qmp_command_is_enabled(const QmpCommand *cmd);
 const char *qmp_command_name(const QmpCommand *cmd);
 bool qmp_has_success_response(const QmpCommand *cmd);
 QObject *qmp_build_error_object(Error *err);
-QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp);
+QDict *qmp_dispatch_check_obj(const QObject *request, bool allow_oob,
+                              Error **errp);
+QObject *qmp_dispatch(QmpCommandList *cmds, QObject *request,
+                      bool allow_oob);
 bool qmp_is_oob(QDict *dict);
 
 typedef void (*qmp_cmd_callback_fn)(QmpCommand *cmd, void *opaque);
diff --git a/monitor.c b/monitor.c
index 4659399b55..2c28b05ecb 100644
--- a/monitor.c
+++ b/monitor.c
@@ -1318,11 +1318,6 @@ static bool qmp_cmd_oob_check(Monitor *mon, QDict *req, Error **errp)
     }
 
     if (qmp_is_oob(req)) {
-        if (!qmp_oob_enabled(mon)) {
-            error_setg(errp, "Please enable out-of-band first "
-                       "for the session during capabilities negotiation");
-            return false;
-        }
         if (!(cmd->options & QCO_ALLOW_OOB)) {
             error_setg(errp, "The command %s does not support OOB",
                        command);
@@ -4194,7 +4189,7 @@ static void monitor_qmp_dispatch_one(QMPRequest *req_obj)
     old_mon = cur_mon;
     cur_mon = mon;
 
-    rsp = qmp_dispatch(mon->qmp.commands, req);
+    rsp = qmp_dispatch(mon->qmp.commands, req, qmp_oob_enabled(mon));
 
     cur_mon = old_mon;
 
@@ -4285,7 +4280,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     } /* else will fail qmp_dispatch() */
 
     /* Check against the request in general layout */
-    qdict = qmp_dispatch_check_obj(req, &err);
+    qdict = qmp_dispatch_check_obj(req, qmp_oob_enabled(mon), &err);
     if (!qdict) {
         goto err;
     }
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 3d5d5e110f..0ad0fab8ed 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -20,7 +20,8 @@
 #include "qapi/qmp/qbool.h"
 #include "sysemu/sysemu.h"
 
-QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
+QDict *qmp_dispatch_check_obj(const QObject *request, bool allow_oob,
+                              Error **errp)
 {
     const QDictEntry *ent;
     const char *arg_name;
@@ -52,7 +53,7 @@ QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
                            "QMP input member 'arguments' must be an object");
                 return NULL;
             }
-        } else if (!strcmp(arg_name, "control")) {
+        } else if (!strcmp(arg_name, "control") && allow_oob) {
             if (qobject_type(arg_obj) != QTYPE_QDICT) {
                 error_setg(errp,
                            "QMP input member 'control' must be a dict");
@@ -74,7 +75,7 @@ QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
 }
 
 static QObject *do_qmp_dispatch(QmpCommandList *cmds, QObject *request,
-                                Error **errp)
+                                bool allow_oob, Error **errp)
 {
     Error *local_err = NULL;
     const char *command;
@@ -82,7 +83,7 @@ static QObject *do_qmp_dispatch(QmpCommandList *cmds, QObject *request,
     QmpCommand *cmd;
     QObject *ret = NULL;
 
-    dict = qmp_dispatch_check_obj(request, errp);
+    dict = qmp_dispatch_check_obj(request, allow_oob, errp);
     if (!dict) {
         return NULL;
     }
@@ -157,13 +158,14 @@ bool qmp_is_oob(QDict *dict)
     return qbool_get_bool(bool_obj);
 }
 
-QObject *qmp_dispatch(QmpCommandList *cmds, QObject *request)
+QObject *qmp_dispatch(QmpCommandList *cmds, QObject *request,
+                      bool allow_oob)
 {
     Error *err = NULL;
     QObject *ret;
     QDict *rsp;
 
-    ret = do_qmp_dispatch(cmds, request, &err);
+    ret = do_qmp_dispatch(cmds, request, allow_oob, &err);
 
     rsp = qdict_new();
     if (err) {
diff --git a/qga/main.c b/qga/main.c
index ea7540edcc..d332bacce5 100644
--- a/qga/main.c
+++ b/qga/main.c
@@ -586,7 +586,7 @@ static void process_command(GAState *s, QDict *req)
 
     g_assert(req);
     g_debug("processing command");
-    rsp = qmp_dispatch(&ga_commands, QOBJECT(req));
+    rsp = qmp_dispatch(&ga_commands, QOBJECT(req), false);
     if (rsp) {
         ret = send_response(s, rsp);
         if (ret < 0) {
diff --git a/tests/test-qga.c b/tests/test-qga.c
index 2e9e0f73bb..febabc7ad5 100644
--- a/tests/test-qga.c
+++ b/tests/test-qga.c
@@ -245,16 +245,17 @@ static void test_qga_invalid_id(gconstpointer fix)
 
 static void test_qga_invalid_oob(gconstpointer fix)
 {
-    /* FIXME "control" is ignored; it should be rejected */
     const TestFixture *fixture = fix;
-    QDict *ret;
+    QDict *ret, *error;
+    const char *class;
 
     ret = qmp_fd(fixture->fd, "{'execute': 'guest-ping',"
                  " 'control': {'run-oob': true}}");
     g_assert_nonnull(ret);
-    qmp_assert_no_error(ret);
 
-    qdict_get_qdict(ret, "return");
+    error = qdict_get_qdict(ret, "error");
+    class = qdict_get_try_str(error, "class");
+    g_assert_cmpstr(class, ==, "GenericError");
 
     qobject_unref(ret);
 }
diff --git a/tests/test-qmp-cmds.c b/tests/test-qmp-cmds.c
index 491b0c4a44..10c7ba40c1 100644
--- a/tests/test-qmp-cmds.c
+++ b/tests/test-qmp-cmds.c
@@ -102,7 +102,7 @@ static void test_dispatch_cmd(void)
 
     qdict_put_str(req, "execute", "user_def_cmd");
 
-    resp = qmp_dispatch(&qmp_commands, QOBJECT(req));
+    resp = qmp_dispatch(&qmp_commands, QOBJECT(req), false);
     assert(resp != NULL);
     assert(!qdict_haskey(qobject_to(QDict, resp), "error"));
 
@@ -119,7 +119,7 @@ static void test_dispatch_cmd_failure(void)
 
     qdict_put_str(req, "execute", "user_def_cmd2");
 
-    resp = qmp_dispatch(&qmp_commands, QOBJECT(req));
+    resp = qmp_dispatch(&qmp_commands, QOBJECT(req), false);
     assert(resp != NULL);
     assert(qdict_haskey(qobject_to(QDict, resp), "error"));
 
@@ -133,7 +133,7 @@ static void test_dispatch_cmd_failure(void)
 
     qdict_put_str(req, "execute", "user_def_cmd");
 
-    resp = qmp_dispatch(&qmp_commands, QOBJECT(req));
+    resp = qmp_dispatch(&qmp_commands, QOBJECT(req), false);
     assert(resp != NULL);
     assert(qdict_haskey(qobject_to(QDict, resp), "error"));
 
@@ -147,7 +147,7 @@ static QObject *test_qmp_dispatch(QDict *req)
     QDict *resp;
     QObject *ret;
 
-    resp_obj = qmp_dispatch(&qmp_commands, QOBJECT(req));
+    resp_obj = qmp_dispatch(&qmp_commands, QOBJECT(req), false);
     assert(resp_obj);
     resp = qobject_to(QDict, resp_obj);
     assert(resp && !qdict_haskey(resp, "error"));
-- 
2.17.1

Re: [Qemu-devel] [PATCH 11/32] qmp qemu-ga: Fix qemu-ga not to accept "control"

[Eric Blake]

On 07/02/2018 11:21 AM, Markus Armbruster wrote:
> Commit cf869d53172 "qmp: support out-of-band (oob) execution"
> accidentally made qemu-ga accept and ignore "control".  Fix that.
> 
> Out-of-band execution in a monitor that doesn't support it now fails
> with
> 
>      {"error": {"class": "GenericError", "desc": "QMP input member 'control' is unexpected"}}
> 
> instead of
> 
>      {"error": {"class": "GenericError", "desc": "Please enable out-of-band first for the session during capabilities negotiation"}}
> 
> The old description is suboptimal when out-of-band cannot not be
> enabled, or the command doesn't support out-of-band execution.
> 
> The new description is a bit unspecific, but it'll do.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 12/32] qmp: Redo how the client requests out-of-band execution

[Markus Armbruster]

Commit cf869d53172 "qmp: support out-of-band (oob) execution" added a
general mechanism for command-independent arguments just for an
out-of-band flag:

    The "control" key is introduced to store this extra flag.  "control"
    field is used to store arguments that are shared by all the commands,
    rather than command specific arguments.  Let "run-oob" be the first.

However, it failed to reject unknown members of "control".  For
instance, in QMP command

    {"execute": "query-name", "id": 42, "control": {"crap": true}}

"crap" gets silently ignored.

Instead of fixing this, revert the general "control" mechanism
(because YAGNI), and do it the way I initially proposed, with key
"exec-oob".  Simpler code, simper interface.

An out-of-band command

    {"execute": "migrate-pause", "id": 42, "control": {"run-oob": true}}

becomes

    {"exec-oob": "migrate-pause", "id": 42}

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 docs/devel/qapi-code-gen.txt | 10 +++----
 docs/interop/qmp-spec.txt    | 18 ++++++-------
 monitor.c                    |  3 +++
 qapi/qmp-dispatch.c          | 51 +++++++++++++++---------------------
 tests/qmp-test.c             |  7 ++---
 tests/test-qga.c             |  3 +--
 6 files changed, 39 insertions(+), 53 deletions(-)

diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index 9625798d16..f020f6bab2 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -649,13 +649,11 @@ example:
  { 'command': 'migrate_recover',
    'data': { 'uri': 'str' }, 'allow-oob': true }
 
-To execute a command with out-of-band priority, the client specifies
-the "control" field in the request, with "run-oob" set to
-true. Example:
+To execute a command with out-of-band priority, the client uses key
+"exec-oob" instead of "execute".  Example:
 
- => { "execute": "command-support-oob",
-      "arguments": { ... },
-      "control": { "run-oob": true } }
+ => { "exec-oob": "migrate-recover",
+      "arguments": { "uri": "tcp:192.168.1.200:12345" } }
  <= { "return": { } }
 
 Without it, even the commands that support out-of-band execution will
diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.txt
index 794c95838f..113c9dcf51 100644
--- a/docs/interop/qmp-spec.txt
+++ b/docs/interop/qmp-spec.txt
@@ -92,12 +92,16 @@ Currently supported capabilities are:
 
 The format for command execution is:
 
-{ "execute": json-string, "arguments": json-object, "id": json-value,
-  "control": json-object }
+{ "execute": json-string, "arguments": json-object, "id": json-value }
+
+or
+
+{ "exec-oob": json-string, "arguments": json-object, "id": json-value }
 
  Where,
 
-- The "execute" member identifies the command to be executed by the Server
+- The "execute" or "exec-oob" member identifies the command to be
+  executed by the server.  The latter requests out-of-band execution.
 - The "arguments" member is used to pass any arguments required for the
   execution of the command, it is optional when no arguments are
   required. Each command documents what contents will be considered
@@ -106,9 +110,6 @@ The format for command execution is:
   command execution, it is optional and will be part of the response
   if provided.  The "id" member can be any json-value.  A json-number
   incremented for each successive command works fine.
-- The optional "control" member further specifies how the command is
-  to be executed.  Currently, its only member is optional "run-oob".
-  See section "2.3.1 Out-of-band execution" for details.
 
 2.3.1 Out-of-band execution
 ---------------------------
@@ -125,9 +126,6 @@ possibly overtaking prior in-band commands.  The client may therefore
 receive such a command's response before responses from prior in-band
 commands.
 
-To execute a command out-of-band, the client puts "run-oob": true into
-execute's member "control".
-
 If the client sends in-band commands faster than the server can
 execute them, the server will eventually drop commands to limit the
 queue length.  The sever sends event COMMAND_DROPPED then.
@@ -270,7 +268,7 @@ S: { "timestamp": { "seconds": 1258551470, "microseconds": 802384 },
 3.7 Out-of-band execution
 -------------------------
 
-C: { "execute": "migrate-pause", "id": 42, "control": { "run-oob": true } }
+C: { "exec-oob": "migrate-pause", "id": 42 }
 S: { "id": 42,
      "error": { "class": "GenericError",
       "desc": "migrate-pause is currently only supported during postcopy-active state" } }
diff --git a/monitor.c b/monitor.c
index 2c28b05ecb..8b0c29ce06 100644
--- a/monitor.c
+++ b/monitor.c
@@ -1299,6 +1299,9 @@ static bool qmp_cmd_oob_check(Monitor *mon, QDict *req, Error **errp)
     QmpCommand *cmd;
 
     command = qdict_get_try_str(req, "execute");
+    if (!command) {
+        command = qdict_get_try_str(req, "exec-oob");
+    }
     if (!command) {
         error_setg(errp, "Command field 'execute' missing");
         return false;
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 0ad0fab8ed..12be120fe7 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -23,11 +23,11 @@
 QDict *qmp_dispatch_check_obj(const QObject *request, bool allow_oob,
                               Error **errp)
 {
+    const char *exec_key = NULL;
     const QDictEntry *ent;
     const char *arg_name;
     const QObject *arg_obj;
-    bool has_exec_key = false;
-    QDict *dict = NULL;
+    QDict *dict;
 
     dict = qobject_to(QDict, request);
     if (!dict) {
@@ -40,25 +40,25 @@ QDict *qmp_dispatch_check_obj(const QObject *request, bool allow_oob,
         arg_name = qdict_entry_key(ent);
         arg_obj = qdict_entry_value(ent);
 
-        if (!strcmp(arg_name, "execute")) {
+        if (!strcmp(arg_name, "execute")
+            || (!strcmp(arg_name, "exec-oob") && allow_oob)) {
             if (qobject_type(arg_obj) != QTYPE_QSTRING) {
-                error_setg(errp,
-                           "QMP input member 'execute' must be a string");
+                error_setg(errp, "QMP input member '%s' must be a string",
+                           arg_name);
                 return NULL;
             }
-            has_exec_key = true;
+            if (exec_key) {
+                error_setg(errp, "QMP input member '%s' clashes with '%s'",
+                           arg_name, exec_key);
+                return NULL;
+            }
+            exec_key = arg_name;
         } else if (!strcmp(arg_name, "arguments")) {
             if (qobject_type(arg_obj) != QTYPE_QDICT) {
                 error_setg(errp,
                            "QMP input member 'arguments' must be an object");
                 return NULL;
             }
-        } else if (!strcmp(arg_name, "control") && allow_oob) {
-            if (qobject_type(arg_obj) != QTYPE_QDICT) {
-                error_setg(errp,
-                           "QMP input member 'control' must be a dict");
-                return NULL;
-            }
         } else {
             error_setg(errp, "QMP input member '%s' is unexpected",
                        arg_name);
@@ -66,7 +66,7 @@ QDict *qmp_dispatch_check_obj(const QObject *request, bool allow_oob,
         }
     }
 
-    if (!has_exec_key) {
+    if (!exec_key) {
         error_setg(errp, "QMP input lacks member 'execute'");
         return NULL;
     }
@@ -88,7 +88,11 @@ static QObject *do_qmp_dispatch(QmpCommandList *cmds, QObject *request,
         return NULL;
     }
 
-    command = qdict_get_str(dict, "execute");
+    command = qdict_get_try_str(dict, "execute");
+    if (!command) {
+        assert(allow_oob);
+        command = qdict_get_str(dict, "exec-oob");
+    }
     cmd = qmp_find_command(cmds, command);
     if (cmd == NULL) {
         error_set(errp, ERROR_CLASS_COMMAND_NOT_FOUND,
@@ -137,25 +141,12 @@ QObject *qmp_build_error_object(Error *err)
 }
 
 /*
- * Detect whether a request should be run out-of-band, by quickly
- * peeking at whether we have: { "control": { "run-oob": true } }. By
- * default commands are run in-band.
+ * Does @qdict look like a command to be run out-of-band?
  */
 bool qmp_is_oob(QDict *dict)
 {
-    QBool *bool_obj;
-
-    dict = qdict_get_qdict(dict, "control");
-    if (!dict) {
-        return false;
-    }
-
-    bool_obj = qobject_to(QBool, qdict_get(dict, "run-oob"));
-    if (!bool_obj) {
-        return false;
-    }
-
-    return qbool_get_bool(bool_obj);
+    return qdict_haskey(dict, "exec-oob")
+        && !qdict_haskey(dict, "execute");
 }
 
 QObject *qmp_dispatch(QmpCommandList *cmds, QObject *request,
diff --git a/tests/qmp-test.c b/tests/qmp-test.c
index 3932901146..dc30930201 100644
--- a/tests/qmp-test.c
+++ b/tests/qmp-test.c
@@ -174,8 +174,7 @@ static void unblock_blocked_cmd(void)
 
 static void send_oob_cmd(QTestState *s, const char *id)
 {
-    qtest_async_qmp(s, "{ 'execute': 'migrate-pause', 'id': %s,"
-                    "  'control': { 'run-oob': true } }", id);
+    qtest_async_qmp(s, "{ 'exec-oob': 'migrate-pause', 'id': %s }", id);
 }
 
 static void recv_cmd_id(QTestState *s, const char *id)
@@ -227,9 +226,7 @@ static void test_qmp_oob(void)
      * Try any command that does not support OOB but with OOB flag. We
      * should get failure.
      */
-    resp = qtest_qmp(qts,
-                     "{ 'execute': 'query-cpus',"
-                     "  'control': { 'run-oob': true } }");
+    resp = qtest_qmp(qts, "{ 'exec-oob': 'query-cpus' }");
     g_assert(qdict_haskey(resp, "error"));
     qobject_unref(resp);
 
diff --git a/tests/test-qga.c b/tests/test-qga.c
index febabc7ad5..daadf22ea3 100644
--- a/tests/test-qga.c
+++ b/tests/test-qga.c
@@ -249,8 +249,7 @@ static void test_qga_invalid_oob(gconstpointer fix)
     QDict *ret, *error;
     const char *class;
 
-    ret = qmp_fd(fixture->fd, "{'execute': 'guest-ping',"
-                 " 'control': {'run-oob': true}}");
+    ret = qmp_fd(fixture->fd, "{'exec-oob': 'guest-ping'}");
     g_assert_nonnull(ret);
 
     error = qdict_get_qdict(ret, "error");
-- 
2.17.1

Re: [Qemu-devel] [PATCH 12/32] qmp: Redo how the client requests out-of-band execution

[Eric Blake]

On 07/02/2018 11:21 AM, Markus Armbruster wrote:
> Commit cf869d53172 "qmp: support out-of-band (oob) execution" added a
> general mechanism for command-independent arguments just for an
> out-of-band flag:
> 
>      The "control" key is introduced to store this extra flag.  "control"
>      field is used to store arguments that are shared by all the commands,
>      rather than command specific arguments.  Let "run-oob" be the first.
> 
> However, it failed to reject unknown members of "control".  For
> instance, in QMP command
> 
>      {"execute": "query-name", "id": 42, "control": {"crap": true}}
> 
> "crap" gets silently ignored.
> 
> Instead of fixing this, revert the general "control" mechanism
> (because YAGNI), and do it the way I initially proposed, with key
> "exec-oob".  Simpler code, simper interface.
> 
> An out-of-band command
> 
>      {"execute": "migrate-pause", "id": 42, "control": {"run-oob": true}}
> 
> becomes
> 
>      {"exec-oob": "migrate-pause", "id": 42}
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---

Yay!

> @@ -106,9 +110,6 @@ The format for command execution is:
>     command execution, it is optional and will be part of the response
>     if provided.  The "id" member can be any json-value.  A json-number
>     incremented for each successive command works fine.
> -- The optional "control" member further specifies how the command is
> -  to be executed.  Currently, its only member is optional "run-oob".
> -  See section "2.3.1 Out-of-band execution" for details.

Still could use a mention that 'id' is a VERY good idea when using 
'exec-oob' - but as 7/32 made it optional, you could tweak that part of 
the doc in a later patch when making 'id' required for 'exec-oob'.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 13/32] qmp: Revert change to handle_qmp_command tracepoint

[Markus Armbruster]

Commit 71da4667db6 "monitor: separate QMP parser and dispatcher" moved
the handle_qmp_command tracepoint from handle_qmp_command() to
monitor_qmp_dispatch_one().  This delays tracing from enqueue time to
dequeue time.  Revert that.  Dequeue remains adequately visible via
tracepoint monitor_qmp_cmd_in_band.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/monitor.c b/monitor.c
index 8b0c29ce06..10991757f6 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4183,12 +4183,6 @@ static void monitor_qmp_dispatch_one(QMPRequest *req_obj)
 
     g_free(req_obj);
 
-    if (trace_event_get_state_backends(TRACE_HANDLE_QMP_COMMAND)) {
-        QString *req_json = qobject_to_json(req);
-        trace_handle_qmp_command(mon, qstring_get_str(req_json));
-        qobject_unref(req_json);
-    }
-
     old_mon = cur_mon;
     cur_mon = mon;
 
@@ -4282,6 +4276,12 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         qdict_del(qdict, "id");
     } /* else will fail qmp_dispatch() */
 
+    if (trace_event_get_state_backends(TRACE_HANDLE_QMP_COMMAND)) {
+        QString *req_json = qobject_to_json(req);
+        trace_handle_qmp_command(mon, qstring_get_str(req_json));
+        qobject_unref(req_json);
+    }
+
     /* Check against the request in general layout */
     qdict = qmp_dispatch_check_obj(req, qmp_oob_enabled(mon), &err);
     if (!qdict) {
-- 
2.17.1

Re: [Qemu-devel] [PATCH 13/32] qmp: Revert change to handle_qmp_command tracepoint

[Eric Blake]

On 07/02/2018 11:21 AM, Markus Armbruster wrote:
> Commit 71da4667db6 "monitor: separate QMP parser and dispatcher" moved
> the handle_qmp_command tracepoint from handle_qmp_command() to
> monitor_qmp_dispatch_one().  This delays tracing from enqueue time to
> dequeue time.  Revert that.  Dequeue remains adequately visible via
> tracepoint monitor_qmp_cmd_in_band.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c | 12 ++++++------
>   1 file changed, 6 insertions(+), 6 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 14/32] qmp: Always free QMPRequest with qmp_request_free()

[Markus Armbruster]

monitor_qmp_dispatch_one() frees a QMPRequest manually, because it
needs to keep a reference to ->id.  Premature optimization.  Take an
additional reference so we can use qmp_request_free().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/monitor.c b/monitor.c
index 10991757f6..94f5660c3c 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4181,8 +4181,6 @@ static void monitor_qmp_dispatch_one(QMPRequest *req_obj)
     id = req_obj->id;
     need_resume = req_obj->need_resume;
 
-    g_free(req_obj);
-
     old_mon = cur_mon;
     cur_mon = mon;
 
@@ -4191,14 +4189,14 @@ static void monitor_qmp_dispatch_one(QMPRequest *req_obj)
     cur_mon = old_mon;
 
     /* Respond if necessary */
-    monitor_qmp_respond(mon, rsp, NULL, id);
+    monitor_qmp_respond(mon, rsp, NULL, qobject_ref(id));
 
     /* This pairs with the monitor_suspend() in handle_qmp_command(). */
     if (need_resume) {
         monitor_resume(mon);
     }
 
-    qobject_unref(req);
+    qmp_request_free(req_obj);
 }
 
 /*
-- 
2.17.1

Re: [Qemu-devel] [PATCH 14/32] qmp: Always free QMPRequest with qmp_request_free()

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> monitor_qmp_dispatch_one() frees a QMPRequest manually, because it
> needs to keep a reference to ->id.  Premature optimization.  Take an
> additional reference so we can use qmp_request_free().
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c | 6 ++----
>   1 file changed, 2 insertions(+), 4 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

> diff --git a/monitor.c b/monitor.c
> index 10991757f6..94f5660c3c 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -4181,8 +4181,6 @@ static void monitor_qmp_dispatch_one(QMPRequest *req_obj)
>       id = req_obj->id;
>       need_resume = req_obj->need_resume;
>   
> -    g_free(req_obj);
> -
>       old_mon = cur_mon;
>       cur_mon = mon;
>   
> @@ -4191,14 +4189,14 @@ static void monitor_qmp_dispatch_one(QMPRequest *req_obj)
>       cur_mon = old_mon;
>   
>       /* Respond if necessary */
> -    monitor_qmp_respond(mon, rsp, NULL, id);
> +    monitor_qmp_respond(mon, rsp, NULL, qobject_ref(id));

This requires qobject_ref(NULL) to work (I thought at one point we were 
considering requiring it to be a non-NULL argument, but right now it 
looks like we are safe).

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH 14/32] qmp: Always free QMPRequest with qmp_request_free()

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 07/02/2018 11:22 AM, Markus Armbruster wrote:
>> monitor_qmp_dispatch_one() frees a QMPRequest manually, because it
>> needs to keep a reference to ->id.  Premature optimization.  Take an
>> additional reference so we can use qmp_request_free().
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>   monitor.c | 6 ++----
>>   1 file changed, 2 insertions(+), 4 deletions(-)
>>
>
> Reviewed-by: Eric Blake <eblake@redhat.com>
>
>> diff --git a/monitor.c b/monitor.c
>> index 10991757f6..94f5660c3c 100644
>> --- a/monitor.c
>> +++ b/monitor.c
>> @@ -4181,8 +4181,6 @@ static void monitor_qmp_dispatch_one(QMPRequest *req_obj)
>>       id = req_obj->id;
>>       need_resume = req_obj->need_resume;
>>   -    g_free(req_obj);
>> -
>>       old_mon = cur_mon;
>>       cur_mon = mon;
>>   @@ -4191,14 +4189,14 @@ static void
>> monitor_qmp_dispatch_one(QMPRequest *req_obj)
>>       cur_mon = old_mon;
>>         /* Respond if necessary */
>> -    monitor_qmp_respond(mon, rsp, NULL, id);
>> +    monitor_qmp_respond(mon, rsp, NULL, qobject_ref(id));
>
> This requires qobject_ref(NULL) to work (I thought at one point we
> were considering requiring it to be a non-NULL argument, but right now
> it looks like we are safe).

[PATCH v6 5/5] qobject: modify qobject_ref() to assert on NULL
Message-Id: <20180419150145.24795-6-marcandre.lureau@redhat.com>

I didn't merge it back then because I lacked the time to convince myself
all users of qobject_ref() the patch doesn't guard cannot pass null, and
also because I wasn't sure requiring qobject_ref()'s argument to be
non-null improves legibility of the code.  Right now we got bigger fish
to fry.

[Qemu-devel] [PATCH 15/32] qmp: Simplify code around monitor_qmp_dispatch_one()

[Markus Armbruster]

Change monitor_qmp_dispatch_one() to take its parameters unwrapped,
move monitor_resume() to the one caller that needs it, rename the
function to monitor_qmp_dispatch().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c | 58 +++++++++++++++++++++++--------------------------------
 1 file changed, 24 insertions(+), 34 deletions(-)

diff --git a/monitor.c b/monitor.c
index 94f5660c3c..51ba1485ad 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4166,20 +4166,10 @@ static void monitor_qmp_respond(Monitor *mon, QObject *rsp,
     qobject_unref(rsp);
 }
 
-/*
- * Dispatch one single QMP request. The function will free the req_obj
- * and objects inside it before return.
- */
-static void monitor_qmp_dispatch_one(QMPRequest *req_obj)
+static void monitor_qmp_dispatch(Monitor *mon, QObject *req, QObject *id)
 {
-    Monitor *mon, *old_mon;
-    QObject *req, *rsp = NULL, *id;
-    bool need_resume;
-
-    req = req_obj->req;
-    mon = req_obj->mon;
-    id = req_obj->id;
-    need_resume = req_obj->need_resume;
+    Monitor *old_mon;
+    QObject *rsp;
 
     old_mon = cur_mon;
     cur_mon = mon;
@@ -4188,15 +4178,7 @@ static void monitor_qmp_dispatch_one(QMPRequest *req_obj)
 
     cur_mon = old_mon;
 
-    /* Respond if necessary */
     monitor_qmp_respond(mon, rsp, NULL, qobject_ref(id));
-
-    /* This pairs with the monitor_suspend() in handle_qmp_command(). */
-    if (need_resume) {
-        monitor_resume(mon);
-    }
-
-    qmp_request_free(req_obj);
 }
 
 /*
@@ -4240,12 +4222,20 @@ static void monitor_qmp_bh_dispatcher(void *data)
 {
     QMPRequest *req_obj = monitor_qmp_requests_pop_any();
 
-    if (req_obj) {
-        trace_monitor_qmp_cmd_in_band(qobject_get_try_str(req_obj->id) ?: "");
-        monitor_qmp_dispatch_one(req_obj);
-        /* Reschedule instead of looping so the main loop stays responsive */
-        qemu_bh_schedule(mon_global.qmp_dispatcher_bh);
+    if (!req_obj) {
+        return;
     }
+
+    trace_monitor_qmp_cmd_in_band(qobject_get_try_str(req_obj->id) ?: "");
+    monitor_qmp_dispatch(req_obj->mon, req_obj->req, req_obj->id);
+    if (req_obj->need_resume) {
+        /* Pairs with the monitor_suspend() in handle_qmp_command() */
+        monitor_resume(req_obj->mon);
+    }
+    qmp_request_free(req_obj);
+
+    /* Reschedule instead of looping so the main loop stays responsive */
+    qemu_bh_schedule(mon_global.qmp_dispatcher_bh);
 }
 
 #define  QMP_REQ_QUEUE_LEN_MAX  (8)
@@ -4291,20 +4281,20 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         goto err;
     }
 
+    if (qmp_is_oob(qdict)) {
+        /* Out-of-band (OOB) requests are executed directly in parser. */
+        trace_monitor_qmp_cmd_out_of_band(qobject_get_try_str(id)
+                                          ?: "");
+        monitor_qmp_dispatch(mon, req, id);
+        return;
+    }
+
     req_obj = g_new0(QMPRequest, 1);
     req_obj->mon = mon;
     req_obj->id = id;
     req_obj->req = req;
     req_obj->need_resume = false;
 
-    if (qmp_is_oob(qdict)) {
-        /* Out-of-band (OOB) requests are executed directly in parser. */
-        trace_monitor_qmp_cmd_out_of_band(qobject_get_try_str(req_obj->id)
-                                          ?: "");
-        monitor_qmp_dispatch_one(req_obj);
-        return;
-    }
-
     /* Protect qmp_requests and fetching its length. */
     qemu_mutex_lock(&mon->qmp.qmp_queue_lock);
 
-- 
2.17.1

Re: [Qemu-devel] [PATCH 15/32] qmp: Simplify code around monitor_qmp_dispatch_one()

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> Change monitor_qmp_dispatch_one() to take its parameters unwrapped,
> move monitor_resume() to the one caller that needs it, rename the
> function to monitor_qmp_dispatch().
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c | 58 +++++++++++++++++++++++--------------------------------
>   1 file changed, 24 insertions(+), 34 deletions(-)
> 
Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 16/32] tests/qmp-test: Demonstrate QMP errors jumping the queue

[Markus Armbruster]

When OOB is enabled, out-of-band commands are executed right away,
everything else is queued.  This lets out-of-band commands "jump the
queue".

However, certain errors are always reported right away, and therefore
can jump the queue even when the erroneous input does not request
out-of-band execution.  These errors are pretty unlikely to occur in
production, but it's wrong all the same.  Mark FIXME.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c        | 1 +
 tests/qmp-test.c | 7 +++++++
 2 files changed, 8 insertions(+)

diff --git a/monitor.c b/monitor.c
index 51ba1485ad..28fa9b8d44 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4338,6 +4338,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     return;
 
 err:
+    /* FIXME overtakes queued in-band commands, wrong when !qmp_is_oob() */
     monitor_qmp_respond(mon, NULL, err, NULL);
     qobject_unref(req);
 }
diff --git a/tests/qmp-test.c b/tests/qmp-test.c
index dc30930201..fe5e5b548a 100644
--- a/tests/qmp-test.c
+++ b/tests/qmp-test.c
@@ -239,6 +239,13 @@ static void test_qmp_oob(void)
     unblock_blocked_cmd();
     recv_cmd_id(qts, "ib-blocks-1");
     recv_cmd_id(qts, "ib-quick-1");
+
+    /* FIXME certain in-band errors overtake slow in-band command */
+    send_cmd_that_blocks(qts, "blocks-2");
+    qtest_async_qmp(qts, "{ 'id': 'err-2' }");
+    recv_cmd_id(qts, NULL);
+    unblock_blocked_cmd();
+    recv_cmd_id(qts, "blocks-2");
     cleanup_blocking_cmd();
 
     qtest_quit(qts);
-- 
2.17.1

Re: [Qemu-devel] [PATCH 16/32] tests/qmp-test: Demonstrate QMP errors jumping the queue

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> When OOB is enabled, out-of-band commands are executed right away,
> everything else is queued.  This lets out-of-band commands "jump the
> queue".
> 
> However, certain errors are always reported right away, and therefore
> can jump the queue even when the erroneous input does not request
> out-of-band execution.  These errors are pretty unlikely to occur in
> production, but it's wrong all the same.  Mark FIXME.

Ouch.

> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c        | 1 +
>   tests/qmp-test.c | 7 +++++++
>   2 files changed, 8 insertions(+)
> 

> +++ b/tests/qmp-test.c
> @@ -239,6 +239,13 @@ static void test_qmp_oob(void)
>       unblock_blocked_cmd();
>       recv_cmd_id(qts, "ib-blocks-1");
>       recv_cmd_id(qts, "ib-quick-1");
> +
> +    /* FIXME certain in-band errors overtake slow in-band command */
> +    send_cmd_that_blocks(qts, "blocks-2");
> +    qtest_async_qmp(qts, "{ 'id': 'err-2' }");

Since this has neither 'execute' nor 'exec-oob', we can't state whether 
it is in-band or out-of-band; back-compatibility says it should be 
treated as in-band.

> +    recv_cmd_id(qts, NULL);
> +    unblock_blocked_cmd();
> +    recv_cmd_id(qts, "blocks-2");

Thus, it should have been queued until after blocks-2 completed.

Useful test.
Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH 16/32] tests/qmp-test: Demonstrate QMP errors jumping the queue

[Peter Xu]

On Mon, Jul 02, 2018 at 06:22:02PM +0200, Markus Armbruster wrote:
> When OOB is enabled, out-of-band commands are executed right away,
> everything else is queued.  This lets out-of-band commands "jump the
> queue".
> 
> However, certain errors are always reported right away, and therefore
> can jump the queue even when the erroneous input does not request
> out-of-band execution.  These errors are pretty unlikely to occur in
> production, but it's wrong all the same.  Mark FIXME.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  monitor.c        | 1 +
>  tests/qmp-test.c | 7 +++++++
>  2 files changed, 8 insertions(+)
> 
> diff --git a/monitor.c b/monitor.c
> index 51ba1485ad..28fa9b8d44 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -4338,6 +4338,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
>      return;
>  
>  err:
> +    /* FIXME overtakes queued in-band commands, wrong when !qmp_is_oob() */
>      monitor_qmp_respond(mon, NULL, err, NULL);
>      qobject_unref(req);
>  }
> diff --git a/tests/qmp-test.c b/tests/qmp-test.c
> index dc30930201..fe5e5b548a 100644
> --- a/tests/qmp-test.c
> +++ b/tests/qmp-test.c
> @@ -239,6 +239,13 @@ static void test_qmp_oob(void)
>      unblock_blocked_cmd();
>      recv_cmd_id(qts, "ib-blocks-1");
>      recv_cmd_id(qts, "ib-quick-1");
> +
> +    /* FIXME certain in-band errors overtake slow in-band command */
> +    send_cmd_that_blocks(qts, "blocks-2");
> +    qtest_async_qmp(qts, "{ 'id': 'err-2' }");
> +    recv_cmd_id(qts, NULL);

(I thought the "id" will be passed in when reply with the error,
 though in fact it's not)

Thanks for the test case.  I'll work upon it when fixing that up.

Reviewed-by: Peter Xu <peterx@redhat.com>

> +    unblock_blocked_cmd();
> +    recv_cmd_id(qts, "blocks-2");
>      cleanup_blocking_cmd();
>  
>      qtest_quit(qts);
> -- 
> 2.17.1
> 

-- 
Peter Xu

Re: [Qemu-devel] [PATCH 16/32] tests/qmp-test: Demonstrate QMP errors jumping the queue

[Markus Armbruster]

Peter Xu <peterx@redhat.com> writes:

> On Mon, Jul 02, 2018 at 06:22:02PM +0200, Markus Armbruster wrote:
>> When OOB is enabled, out-of-band commands are executed right away,
>> everything else is queued.  This lets out-of-band commands "jump the
>> queue".
>> 
>> However, certain errors are always reported right away, and therefore
>> can jump the queue even when the erroneous input does not request
>> out-of-band execution.  These errors are pretty unlikely to occur in
>> production, but it's wrong all the same.  Mark FIXME.
>> 
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>  monitor.c        | 1 +
>>  tests/qmp-test.c | 7 +++++++
>>  2 files changed, 8 insertions(+)
>> 
>> diff --git a/monitor.c b/monitor.c
>> index 51ba1485ad..28fa9b8d44 100644
>> --- a/monitor.c
>> +++ b/monitor.c
>> @@ -4338,6 +4338,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
>>      return;
>>  
>>  err:
>> +    /* FIXME overtakes queued in-band commands, wrong when !qmp_is_oob() */
>>      monitor_qmp_respond(mon, NULL, err, NULL);
>>      qobject_unref(req);
>>  }
>> diff --git a/tests/qmp-test.c b/tests/qmp-test.c
>> index dc30930201..fe5e5b548a 100644
>> --- a/tests/qmp-test.c
>> +++ b/tests/qmp-test.c
>> @@ -239,6 +239,13 @@ static void test_qmp_oob(void)
>>      unblock_blocked_cmd();
>>      recv_cmd_id(qts, "ib-blocks-1");
>>      recv_cmd_id(qts, "ib-quick-1");
>> +
>> +    /* FIXME certain in-band errors overtake slow in-band command */
>> +    send_cmd_that_blocks(qts, "blocks-2");
>> +    qtest_async_qmp(qts, "{ 'id': 'err-2' }");
>> +    recv_cmd_id(qts, NULL);
>
> (I thought the "id" will be passed in when reply with the error,
>  though in fact it's not)

It will be after the next patch :)

> Thanks for the test case.  I'll work upon it when fixing that up.
>
> Reviewed-by: Peter Xu <peterx@redhat.com>
>
>> +    unblock_blocked_cmd();
>> +    recv_cmd_id(qts, "blocks-2");
>>      cleanup_blocking_cmd();
>>  
>>      qtest_quit(qts);
>> -- 
>> 2.17.1
>> 

[Qemu-devel] [PATCH 17/32] qmp: Don't let malformed in-band commands jump the queue

[Markus Armbruster]

handle_qmp_command() reports certain errors right away.  This is wrong
when OOB is enabled, because the errors can "jump the queue" then, as
the previous commit demonstrates.

To fix, we need to delay errors until dispatch.  Do that for semantic
errors, mostly by reverting ill-advised parts of commit cf869d53172
"qmp: support out-of-band (oob) execution".  Bonus: doesn't run
qmp_dispatch_check_obj() twice, once in handle_qmp_command(), and
again in do_qmp_dispatch().  That's also due to commit cf869d53172.

The next commit will fix queue jumping for syntax errors.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 include/qapi/qmp/dispatch.h |  2 -
 monitor.c                   | 79 +++++++++----------------------------
 qapi/qmp-dispatch.c         | 12 +++++-
 tests/qmp-test.c            |  4 +-
 4 files changed, 30 insertions(+), 67 deletions(-)

diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
index 303a15ba84..514bfc45b0 100644
--- a/include/qapi/qmp/dispatch.h
+++ b/include/qapi/qmp/dispatch.h
@@ -48,8 +48,6 @@ bool qmp_command_is_enabled(const QmpCommand *cmd);
 const char *qmp_command_name(const QmpCommand *cmd);
 bool qmp_has_success_response(const QmpCommand *cmd);
 QObject *qmp_build_error_object(Error *err);
-QDict *qmp_dispatch_check_obj(const QObject *request, bool allow_oob,
-                              Error **errp);
 QObject *qmp_dispatch(QmpCommandList *cmds, QObject *request,
                       bool allow_oob);
 bool qmp_is_oob(QDict *dict);
diff --git a/monitor.c b/monitor.c
index 28fa9b8d44..7a80e58bf2 100644
--- a/monitor.c
+++ b/monitor.c
@@ -1289,48 +1289,6 @@ static void qmp_caps_apply(Monitor *mon, QMPCapabilityList *list)
     }
 }
 
-/*
- * Return true if check successful, or false otherwise.  When false is
- * returned, detailed error will be in errp if provided.
- */
-static bool qmp_cmd_oob_check(Monitor *mon, QDict *req, Error **errp)
-{
-    const char *command;
-    QmpCommand *cmd;
-
-    command = qdict_get_try_str(req, "execute");
-    if (!command) {
-        command = qdict_get_try_str(req, "exec-oob");
-    }
-    if (!command) {
-        error_setg(errp, "Command field 'execute' missing");
-        return false;
-    }
-
-    cmd = qmp_find_command(mon->qmp.commands, command);
-    if (!cmd) {
-        if (mon->qmp.commands == &qmp_cap_negotiation_commands) {
-            error_set(errp, ERROR_CLASS_COMMAND_NOT_FOUND,
-                      "Expecting capabilities negotiation "
-                      "with 'qmp_capabilities'");
-        } else {
-            error_set(errp, ERROR_CLASS_COMMAND_NOT_FOUND,
-                      "The command %s has not been found", command);
-        }
-        return false;
-    }
-
-    if (qmp_is_oob(req)) {
-        if (!(cmd->options & QCO_ALLOW_OOB)) {
-            error_setg(errp, "The command %s does not support OOB",
-                       command);
-            return false;
-        }
-    }
-
-    return true;
-}
-
 void qmp_qmp_capabilities(bool has_enable, QMPCapabilityList *enable,
                           Error **errp)
 {
@@ -4170,6 +4128,7 @@ static void monitor_qmp_dispatch(Monitor *mon, QObject *req, QObject *id)
 {
     Monitor *old_mon;
     QObject *rsp;
+    QDict *error;
 
     old_mon = cur_mon;
     cur_mon = mon;
@@ -4178,6 +4137,19 @@ static void monitor_qmp_dispatch(Monitor *mon, QObject *req, QObject *id)
 
     cur_mon = old_mon;
 
+    if (mon->qmp.commands == &qmp_cap_negotiation_commands) {
+        error = qdict_get_qdict(qobject_to(QDict, rsp), "error");
+        if (error
+            && !g_strcmp0(qdict_get_try_str(error, "class"),
+                    QapiErrorClass_str(ERROR_CLASS_COMMAND_NOT_FOUND))) {
+            /* Provide a more useful error message */
+            qdict_del(error, "desc");
+            qdict_put_str(error, "desc", "Expecting capabilities negotiation"
+                          " with 'qmp_capabilities'");
+        }
+    }
+
+    /* Respond if necessary */
     monitor_qmp_respond(mon, rsp, NULL, qobject_ref(id));
 }
 
@@ -4255,7 +4227,9 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         error_setg(&err, QERR_JSON_PARSING);
     }
     if (err) {
-        goto err;
+        assert(!req);
+        monitor_qmp_respond(mon, NULL, err, NULL);
+        return;
     }
 
     qdict = qobject_to(QDict, req);
@@ -4270,18 +4244,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         qobject_unref(req_json);
     }
 
-    /* Check against the request in general layout */
-    qdict = qmp_dispatch_check_obj(req, qmp_oob_enabled(mon), &err);
-    if (!qdict) {
-        goto err;
-    }
-
-    /* Check against OOB specific */
-    if (!qmp_cmd_oob_check(mon, qdict, &err)) {
-        goto err;
-    }
-
-    if (qmp_is_oob(qdict)) {
+    if (qdict && qmp_is_oob(qdict)) {
         /* Out-of-band (OOB) requests are executed directly in parser. */
         trace_monitor_qmp_cmd_out_of_band(qobject_get_try_str(id)
                                           ?: "");
@@ -4335,12 +4298,6 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
 
     /* Kick the dispatcher routine */
     qemu_bh_schedule(mon_global.qmp_dispatcher_bh);
-    return;
-
-err:
-    /* FIXME overtakes queued in-band commands, wrong when !qmp_is_oob() */
-    monitor_qmp_respond(mon, NULL, err, NULL);
-    qobject_unref(req);
 }
 
 static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size)
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 12be120fe7..6d78f3e9f6 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -20,8 +20,8 @@
 #include "qapi/qmp/qbool.h"
 #include "sysemu/sysemu.h"
 
-QDict *qmp_dispatch_check_obj(const QObject *request, bool allow_oob,
-                              Error **errp)
+static QDict *qmp_dispatch_check_obj(const QObject *request, bool allow_oob,
+                                     Error **errp)
 {
     const char *exec_key = NULL;
     const QDictEntry *ent;
@@ -78,6 +78,7 @@ static QObject *do_qmp_dispatch(QmpCommandList *cmds, QObject *request,
                                 bool allow_oob, Error **errp)
 {
     Error *local_err = NULL;
+    bool oob;
     const char *command;
     QDict *args, *dict;
     QmpCommand *cmd;
@@ -89,9 +90,11 @@ static QObject *do_qmp_dispatch(QmpCommandList *cmds, QObject *request,
     }
 
     command = qdict_get_try_str(dict, "execute");
+    oob = false;
     if (!command) {
         assert(allow_oob);
         command = qdict_get_str(dict, "exec-oob");
+        oob = true;
     }
     cmd = qmp_find_command(cmds, command);
     if (cmd == NULL) {
@@ -104,6 +107,11 @@ static QObject *do_qmp_dispatch(QmpCommandList *cmds, QObject *request,
                    command);
         return NULL;
     }
+    if (oob && !(cmd->options & QCO_ALLOW_OOB)) {
+        error_setg(errp, "The command %s does not support OOB",
+                   command);
+        return false;
+    }
 
     if (runstate_check(RUN_STATE_PRECONFIG) &&
         !(cmd->options & QCO_ALLOW_PRECONFIG)) {
diff --git a/tests/qmp-test.c b/tests/qmp-test.c
index fe5e5b548a..3f29c6c305 100644
--- a/tests/qmp-test.c
+++ b/tests/qmp-test.c
@@ -240,12 +240,12 @@ static void test_qmp_oob(void)
     recv_cmd_id(qts, "ib-blocks-1");
     recv_cmd_id(qts, "ib-quick-1");
 
-    /* FIXME certain in-band errors overtake slow in-band command */
+    /* Even malformed in-band command fails in-band */
     send_cmd_that_blocks(qts, "blocks-2");
     qtest_async_qmp(qts, "{ 'id': 'err-2' }");
-    recv_cmd_id(qts, NULL);
     unblock_blocked_cmd();
     recv_cmd_id(qts, "blocks-2");
+    recv_cmd_id(qts, "err-2");
     cleanup_blocking_cmd();
 
     qtest_quit(qts);
-- 
2.17.1

Re: [Qemu-devel] [PATCH 17/32] qmp: Don't let malformed in-band commands jump the queue

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> handle_qmp_command() reports certain errors right away.  This is wrong
> when OOB is enabled, because the errors can "jump the queue" then, as
> the previous commit demonstrates.
> 
> To fix, we need to delay errors until dispatch.  Do that for semantic
> errors, mostly by reverting ill-advised parts of commit cf869d53172
> "qmp: support out-of-band (oob) execution".  Bonus: doesn't run
> qmp_dispatch_check_obj() twice, once in handle_qmp_command(), and
> again in do_qmp_dispatch().  That's also due to commit cf869d53172.
> 
> The next commit will fix queue jumping for syntax errors.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   include/qapi/qmp/dispatch.h |  2 -
>   monitor.c                   | 79 +++++++++----------------------------
>   qapi/qmp-dispatch.c         | 12 +++++-
>   tests/qmp-test.c            |  4 +-
>   4 files changed, 30 insertions(+), 67 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com

Fixing bug and reducing code size. I'm glad I made oob experimental in 
2.12, because I obviously didn't review it as closely in your absence 
for that release as you have done now (and changing it to be 
non-experimental early in the release cycle has also been good for 
letting us chase down these bugs in the original implementation).

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH 17/32] qmp: Don't let malformed in-band commands jump the queue

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 07/02/2018 11:22 AM, Markus Armbruster wrote:
>> handle_qmp_command() reports certain errors right away.  This is wrong
>> when OOB is enabled, because the errors can "jump the queue" then, as
>> the previous commit demonstrates.
>>
>> To fix, we need to delay errors until dispatch.  Do that for semantic
>> errors, mostly by reverting ill-advised parts of commit cf869d53172
>> "qmp: support out-of-band (oob) execution".  Bonus: doesn't run
>> qmp_dispatch_check_obj() twice, once in handle_qmp_command(), and
>> again in do_qmp_dispatch().  That's also due to commit cf869d53172.
>>
>> The next commit will fix queue jumping for syntax errors.
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>   include/qapi/qmp/dispatch.h |  2 -
>>   monitor.c                   | 79 +++++++++----------------------------
>>   qapi/qmp-dispatch.c         | 12 +++++-
>>   tests/qmp-test.c            |  4 +-
>>   4 files changed, 30 insertions(+), 67 deletions(-)
>>
>
> Reviewed-by: Eric Blake <eblake@redhat.com
>
> Fixing bug and reducing code size. I'm glad I made oob experimental in
> 2.12, because I obviously didn't review it as closely in your absence
> for that release as you have done now (and changing it to be

In all fairness, it took me a while to see this.  The first clue was
"hmm, why does qmp_dispatch_check_obj() gets called in two places?"
Pulling that thread got me to this bug and more.

> non-experimental early in the release cycle has also been good for
> letting us chase down these bugs in the original implementation).

Running into that regression just in time was lucky :)

[Qemu-devel] [PATCH 18/32] qmp: Don't let JSON errors jump the queue

[Markus Armbruster]

handle_qmp_command() reports JSON syntax errors right away.  This is
wrong when OOB is enabled, because the errors can "jump the queue"
then.

The previous commit fixed the same bug for semantic errors, by
delaying the checking until dispatch.  We can't delay the checking, so
delay the reporting.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/monitor.c b/monitor.c
index 7a80e58bf2..fd83a57dda 100644
--- a/monitor.c
+++ b/monitor.c
@@ -253,8 +253,12 @@ struct QMPRequest {
     Monitor *mon;
     /* "id" field of the request */
     QObject *id;
-    /* Request object to be handled */
+    /*
+     * Request object to be handled or Error to be reported
+     * (exactly one of them is non-null)
+     */
     QObject *req;
+    Error *err;
     /*
      * Whether we need to resume the monitor afterward.  This flag is
      * used to emulate the old QMP server behavior that the current
@@ -359,6 +363,7 @@ static void qmp_request_free(QMPRequest *req)
 {
     qobject_unref(req->id);
     qobject_unref(req->req);
+    error_free(req->err);
     g_free(req);
 }
 
@@ -4198,8 +4203,14 @@ static void monitor_qmp_bh_dispatcher(void *data)
         return;
     }
 
-    trace_monitor_qmp_cmd_in_band(qobject_get_try_str(req_obj->id) ?: "");
-    monitor_qmp_dispatch(req_obj->mon, req_obj->req, req_obj->id);
+    if (req_obj->req) {
+        trace_monitor_qmp_cmd_in_band(qobject_get_try_str(req_obj->id) ?: "");
+        monitor_qmp_dispatch(req_obj->mon, req_obj->req, req_obj->id);
+    } else {
+        assert(req_obj->err);
+        monitor_qmp_respond(req_obj->mon, NULL, req_obj->err, NULL);
+    }
+
     if (req_obj->need_resume) {
         /* Pairs with the monitor_suspend() in handle_qmp_command() */
         monitor_resume(req_obj->mon);
@@ -4226,11 +4237,6 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         /* json_parser_parse_err() sucks: can fail without setting @err */
         error_setg(&err, QERR_JSON_PARSING);
     }
-    if (err) {
-        assert(!req);
-        monitor_qmp_respond(mon, NULL, err, NULL);
-        return;
-    }
 
     qdict = qobject_to(QDict, req);
     if (qdict) {
@@ -4256,6 +4262,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     req_obj->mon = mon;
     req_obj->id = id;
     req_obj->req = req;
+    req_obj->err = err;
     req_obj->need_resume = false;
 
     /* Protect qmp_requests and fetching its length. */
-- 
2.17.1

Re: [Qemu-devel] [PATCH 18/32] qmp: Don't let JSON errors jump the queue

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> handle_qmp_command() reports JSON syntax errors right away.  This is
> wrong when OOB is enabled, because the errors can "jump the queue"
> then.
> 
> The previous commit fixed the same bug for semantic errors, by
> delaying the checking until dispatch.  We can't delay the checking, so
> delay the reporting.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c | 23 +++++++++++++++--------
>   1 file changed, 15 insertions(+), 8 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 19/32] monitor: Rename use_io_thr to use_io_thread

[Markus Armbruster]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/monitor.c b/monitor.c
index fd83a57dda..10b6634d3e 100644
--- a/monitor.c
+++ b/monitor.c
@@ -206,11 +206,11 @@ struct Monitor {
     int flags;
     int suspend_cnt;            /* Needs to be accessed atomically */
     bool skip_flush;
-    bool use_io_thr;
+    bool use_io_thread;
 
     /*
      * State used only in the thread "owning" the monitor.
-     * If @use_io_thr, this is mon_global.mon_iothread.
+     * If @use_io_thread, this is mon_global.mon_iothread.
      * Else, it's the main thread.
      * These members can be safely accessed without locks.
      */
@@ -520,7 +520,7 @@ static void monitor_json_emitter_raw(Monitor *mon,
 
 static void monitor_json_emitter(Monitor *mon, QObject *data)
 {
-    if (mon->use_io_thr) {
+    if (mon->use_io_thread) {
         /*
          * If using I/O thread, we need to queue the item so that I/O
          * thread will do the rest for us.  Take refcount so that
@@ -766,7 +766,7 @@ static void monitor_qapi_event_init(void)
 static void handle_hmp_command(Monitor *mon, const char *cmdline);
 
 static void monitor_data_init(Monitor *mon, bool skip_flush,
-                              bool use_io_thr)
+                              bool use_io_thread)
 {
     memset(mon, 0, sizeof(Monitor));
     qemu_mutex_init(&mon->mon_lock);
@@ -775,7 +775,7 @@ static void monitor_data_init(Monitor *mon, bool skip_flush,
     /* Use *mon_cmds by default. */
     mon->cmd_table = mon_cmds;
     mon->skip_flush = skip_flush;
-    mon->use_io_thr = use_io_thr;
+    mon->use_io_thread = use_io_thread;
     mon->qmp.qmp_requests = g_queue_new();
     mon->qmp.qmp_responses = g_queue_new();
 }
@@ -1270,7 +1270,7 @@ static void qmp_caps_check(Monitor *mon, QMPCapabilityList *list,
         assert(list->value < QMP_CAPABILITY__MAX);
         switch (list->value) {
         case QMP_CAPABILITY_OOB:
-            if (!mon->use_io_thr) {
+            if (!mon->use_io_thread) {
                 /*
                  * Out-of-band only works with monitors that are
                  * running on dedicated I/O thread.
@@ -4376,7 +4376,7 @@ void monitor_resume(Monitor *mon)
              * For QMP monitors that are running in I/O thread, let's
              * kick the thread in case it's sleeping.
              */
-            if (mon->use_io_thr) {
+            if (mon->use_io_thread) {
                 aio_notify(iothread_get_aio_context(mon_global.mon_iothread));
             }
         } else {
@@ -4396,7 +4396,7 @@ static QObject *get_qmp_greeting(Monitor *mon)
     qmp_marshal_query_version(NULL, &ver, NULL);
 
     for (cap = 0; cap < QMP_CAPABILITY__MAX; cap++) {
-        if (!mon->use_io_thr && cap == QMP_CAPABILITY_OOB) {
+        if (!mon->use_io_thread && cap == QMP_CAPABILITY_OOB) {
             /* Monitors that are not using I/O thread won't support OOB */
             continue;
         }
@@ -4609,9 +4609,9 @@ static void monitor_qmp_setup_handlers_bh(void *opaque)
     Monitor *mon = opaque;
     GMainContext *context;
 
-    if (mon->use_io_thr) {
+    if (mon->use_io_thread) {
         /*
-         * When use_io_thr is set, we use the global shared dedicated
+         * When @use_io_thread is set, we use the global shared dedicated
          * I/O thread for this monitor to handle input/output.
          */
         context = monitor_get_io_context();
@@ -4660,7 +4660,7 @@ void monitor_init(Chardev *chr, int flags)
     if (monitor_is_qmp(mon)) {
         qemu_chr_fe_set_echo(&mon->chr, true);
         json_message_parser_init(&mon->qmp.parser, handle_qmp_command);
-        if (mon->use_io_thr) {
+        if (mon->use_io_thread) {
             /*
              * Make sure the old iowatch is gone.  It's possible when
              * e.g. the chardev is in client mode, with wait=on.
-- 
2.17.1

Re: [Qemu-devel] [PATCH 19/32] monitor: Rename use_io_thr to use_io_thread

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c | 22 +++++++++++-----------
>   1 file changed, 11 insertions(+), 11 deletions(-)
> 
> diff --git a/monitor.c b/monitor.c
> index fd83a57dda..10b6634d3e 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -206,11 +206,11 @@ struct Monitor {
>       int flags;
>       int suspend_cnt;            /* Needs to be accessed atomically */
>       bool skip_flush;
> -    bool use_io_thr;
> +    bool use_io_thread;

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 20/32] monitor: Peel off @mon_global wrapper

[Markus Armbruster]

Wrapping global variables in a struct without a use for the wrapper
struct buys us nothing but longer lines.  Unwrap them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c | 60 +++++++++++++++++++++++++++----------------------------
 1 file changed, 29 insertions(+), 31 deletions(-)

diff --git a/monitor.c b/monitor.c
index 10b6634d3e..ef19562f65 100644
--- a/monitor.c
+++ b/monitor.c
@@ -210,7 +210,7 @@ struct Monitor {
 
     /*
      * State used only in the thread "owning" the monitor.
-     * If @use_io_thread, this is mon_global.mon_iothread.
+     * If @use_io_thread, this is @mon_iothread.
      * Else, it's the main thread.
      * These members can be safely accessed without locks.
      */
@@ -239,14 +239,13 @@ struct Monitor {
     int mux_out;
 };
 
-/* Let's add monitor global variables to this struct. */
-static struct {
-    IOThread *mon_iothread;
-    /* Bottom half to dispatch the requests received from I/O thread */
-    QEMUBH *qmp_dispatcher_bh;
-    /* Bottom half to deliver the responses back to clients */
-    QEMUBH *qmp_respond_bh;
-} mon_global;
+IOThread *mon_iothread;
+
+/* Bottom half to dispatch the requests received from I/O thread */
+QEMUBH *qmp_dispatcher_bh;
+
+/* Bottom half to deliver the responses back to clients */
+QEMUBH *qmp_respond_bh;
 
 struct QMPRequest {
     /* Owner of the request */
@@ -530,7 +529,7 @@ static void monitor_json_emitter(Monitor *mon, QObject *data)
         qemu_mutex_lock(&mon->qmp.qmp_queue_lock);
         g_queue_push_tail(mon->qmp.qmp_responses, qobject_ref(data));
         qemu_mutex_unlock(&mon->qmp.qmp_queue_lock);
-        qemu_bh_schedule(mon_global.qmp_respond_bh);
+        qemu_bh_schedule(qmp_respond_bh);
     } else {
         /*
          * If not using monitor I/O thread, then we are in main thread.
@@ -4218,7 +4217,7 @@ static void monitor_qmp_bh_dispatcher(void *data)
     qmp_request_free(req_obj);
 
     /* Reschedule instead of looping so the main loop stays responsive */
-    qemu_bh_schedule(mon_global.qmp_dispatcher_bh);
+    qemu_bh_schedule(qmp_dispatcher_bh);
 }
 
 #define  QMP_REQ_QUEUE_LEN_MAX  (8)
@@ -4304,7 +4303,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     qemu_mutex_unlock(&mon->qmp.qmp_queue_lock);
 
     /* Kick the dispatcher routine */
-    qemu_bh_schedule(mon_global.qmp_dispatcher_bh);
+    qemu_bh_schedule(qmp_dispatcher_bh);
 }
 
 static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size)
@@ -4357,7 +4356,7 @@ int monitor_suspend(Monitor *mon)
          * Kick I/O thread to make sure this takes effect.  It'll be
          * evaluated again in prepare() of the watch object.
          */
-        aio_notify(iothread_get_aio_context(mon_global.mon_iothread));
+        aio_notify(iothread_get_aio_context(mon_iothread));
     }
 
     trace_monitor_suspend(mon, 1);
@@ -4377,7 +4376,7 @@ void monitor_resume(Monitor *mon)
              * kick the thread in case it's sleeping.
              */
             if (mon->use_io_thread) {
-                aio_notify(iothread_get_aio_context(mon_global.mon_iothread));
+                aio_notify(iothread_get_aio_context(mon_iothread));
             }
         } else {
             assert(mon->rs);
@@ -4515,36 +4514,35 @@ static void sortcmdlist(void)
 
 static GMainContext *monitor_get_io_context(void)
 {
-    return iothread_get_g_main_context(mon_global.mon_iothread);
+    return iothread_get_g_main_context(mon_iothread);
 }
 
 static AioContext *monitor_get_aio_context(void)
 {
-    return iothread_get_aio_context(mon_global.mon_iothread);
+    return iothread_get_aio_context(mon_iothread);
 }
 
 static void monitor_iothread_init(void)
 {
-    mon_global.mon_iothread = iothread_create("mon_iothread",
-                                              &error_abort);
+    mon_iothread = iothread_create("mon_iothread", &error_abort);
 
     /*
      * This MUST be on main loop thread since we have commands that
      * have assumption to be run on main loop thread.  It would be
      * nice that one day we can remove this assumption in the future.
      */
-    mon_global.qmp_dispatcher_bh = aio_bh_new(iohandler_get_aio_context(),
-                                              monitor_qmp_bh_dispatcher,
-                                              NULL);
+    qmp_dispatcher_bh = aio_bh_new(iohandler_get_aio_context(),
+                                   monitor_qmp_bh_dispatcher,
+                                   NULL);
 
     /*
      * Unlike the dispatcher BH, this must be run on the monitor I/O
      * thread, so that monitors that are using I/O thread will make
      * sure read/write operations are all done on the I/O thread.
      */
-    mon_global.qmp_respond_bh = aio_bh_new(monitor_get_aio_context(),
-                                           monitor_qmp_bh_responder,
-                                           NULL);
+    qmp_respond_bh = aio_bh_new(monitor_get_aio_context(),
+                                monitor_qmp_bh_responder,
+                                NULL);
 }
 
 void monitor_init_globals(void)
@@ -4701,7 +4699,7 @@ void monitor_cleanup(void)
      * we need to unregister from chardev below in
      * monitor_data_destroy(), and chardev is not thread-safe yet
      */
-    iothread_stop(mon_global.mon_iothread);
+    iothread_stop(mon_iothread);
 
     /*
      * After we have I/O thread to send responses, it's possible that
@@ -4722,13 +4720,13 @@ void monitor_cleanup(void)
     qemu_mutex_unlock(&monitor_lock);
 
     /* QEMUBHs needs to be deleted before destroying the I/O thread */
-    qemu_bh_delete(mon_global.qmp_dispatcher_bh);
-    mon_global.qmp_dispatcher_bh = NULL;
-    qemu_bh_delete(mon_global.qmp_respond_bh);
-    mon_global.qmp_respond_bh = NULL;
+    qemu_bh_delete(qmp_dispatcher_bh);
+    qmp_dispatcher_bh = NULL;
+    qemu_bh_delete(qmp_respond_bh);
+    qmp_respond_bh = NULL;
 
-    iothread_destroy(mon_global.mon_iothread);
-    mon_global.mon_iothread = NULL;
+    iothread_destroy(mon_iothread);
+    mon_iothread = NULL;
 }
 
 QemuOptsList qemu_mon_opts = {
-- 
2.17.1

Re: [Qemu-devel] [PATCH 20/32] monitor: Peel off @mon_global wrapper

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> Wrapping global variables in a struct without a use for the wrapper
> struct buys us nothing but longer lines.  Unwrap them.

Well, it also makes it slightly more obvious where you are using a 
global variable, and thus need to be careful of reentrancy.  But I'm not 
opposed to this reduction in size.

> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c | 60 +++++++++++++++++++++++++++----------------------------
>   1 file changed, 29 insertions(+), 31 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 21/32] qobject: New qdict_from_jsonf_nofail()

[Markus Armbruster]

Many uses of qobject_from_jsonf() convert JSON objects.  Create new
convenience function qdict_from_jsonf_nofail() that includes the
conversion to QDict.  The next few commits will put it to use.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 include/qapi/qmp/qjson.h |  2 ++
 qobject/qjson.c          | 18 ++++++++++++++++++
 2 files changed, 20 insertions(+)

diff --git a/include/qapi/qmp/qjson.h b/include/qapi/qmp/qjson.h
index b274ac3a86..43b2ce2f33 100644
--- a/include/qapi/qmp/qjson.h
+++ b/include/qapi/qmp/qjson.h
@@ -19,6 +19,8 @@ QObject *qobject_from_jsonf(const char *string, ...) GCC_FMT_ATTR(1, 2);
 QObject *qobject_from_jsonv(const char *string, va_list *ap, Error **errp)
     GCC_FMT_ATTR(1, 0);
 
+QDict *qdict_from_jsonf_nofail(const char *string, ...) GCC_FMT_ATTR(1, 2);
+
 QString *qobject_to_json(const QObject *obj);
 QString *qobject_to_json_pretty(const QObject *obj);
 
diff --git a/qobject/qjson.c b/qobject/qjson.c
index 9816a65c7d..0df3120202 100644
--- a/qobject/qjson.c
+++ b/qobject/qjson.c
@@ -76,6 +76,24 @@ QObject *qobject_from_jsonf(const char *string, ...)
     return obj;
 }
 
+/*
+ * Parse @string as JSON object with %-escapes interpolated.
+ * Abort on error.  Do not use with untrusted @string.
+ * Return the resulting QDict.  It is never null.
+ */
+QDict *qdict_from_jsonf_nofail(const char *string, ...)
+{
+    QDict *obj;
+    va_list ap;
+
+    va_start(ap, string);
+    obj = qobject_to(QDict, qobject_from_jsonv(string, &ap, &error_abort));
+    va_end(ap);
+
+    assert(obj);
+    return obj;
+}
+
 typedef struct ToJsonIterState
 {
     int indent;
-- 
2.17.1

Re: [Qemu-devel] [PATCH 21/32] qobject: New qdict_from_jsonf_nofail()

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> Many uses of qobject_from_jsonf() convert JSON objects.  Create new
> convenience function qdict_from_jsonf_nofail() that includes the
> conversion to QDict.  The next few commits will put it to use.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   include/qapi/qmp/qjson.h |  2 ++
>   qobject/qjson.c          | 18 ++++++++++++++++++
>   2 files changed, 20 insertions(+)

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 22/32] qmp: De-duplicate error response building

[Markus Armbruster]

All callers of qmp_build_error_object() duplicate the code to wrap it
in a response object.  Replace it by qmp_error_response() that
captures the duplicated code, including error_free().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 include/qapi/qmp/dispatch.h |  2 +-
 monitor.c                   |  7 +------
 qapi/qmp-dispatch.c         | 20 +++++++++++---------
 qga/main.c                  |  8 ++------
 4 files changed, 15 insertions(+), 22 deletions(-)

diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
index 514bfc45b0..a53e11c9b1 100644
--- a/include/qapi/qmp/dispatch.h
+++ b/include/qapi/qmp/dispatch.h
@@ -47,7 +47,7 @@ void qmp_enable_command(QmpCommandList *cmds, const char *name);
 bool qmp_command_is_enabled(const QmpCommand *cmd);
 const char *qmp_command_name(const QmpCommand *cmd);
 bool qmp_has_success_response(const QmpCommand *cmd);
-QObject *qmp_build_error_object(Error *err);
+QDict *qmp_error_response(Error *err);
 QObject *qmp_dispatch(QmpCommandList *cmds, QObject *request,
                       bool allow_oob);
 bool qmp_is_oob(QDict *dict);
diff --git a/monitor.c b/monitor.c
index ef19562f65..db85cd0a57 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4106,14 +4106,9 @@ static int monitor_can_read(void *opaque)
 static void monitor_qmp_respond(Monitor *mon, QObject *rsp,
                                 Error *err, QObject *id)
 {
-    QDict *qdict = NULL;
-
     if (err) {
         assert(!rsp);
-        qdict = qdict_new();
-        qdict_put_obj(qdict, "error", qmp_build_error_object(err));
-        error_free(err);
-        rsp = QOBJECT(qdict);
+        rsp = QOBJECT(qmp_error_response(err));
     }
 
     if (rsp) {
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 6d78f3e9f6..c85748a33f 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -141,11 +141,15 @@ static QObject *do_qmp_dispatch(QmpCommandList *cmds, QObject *request,
     return ret;
 }
 
-QObject *qmp_build_error_object(Error *err)
+QDict *qmp_error_response(Error *err)
 {
-    return qobject_from_jsonf("{ 'class': %s, 'desc': %s }",
-                              QapiErrorClass_str(error_get_class(err)),
-                              error_get_pretty(err));
+    QDict *rsp;
+
+    rsp = qdict_from_jsonf_nofail("{ 'error': { 'class': %s, 'desc': %s } }",
+                                  QapiErrorClass_str(error_get_class(err)),
+                                  error_get_pretty(err));
+    error_free(err);
+    return rsp;
 }
 
 /*
@@ -166,15 +170,13 @@ QObject *qmp_dispatch(QmpCommandList *cmds, QObject *request,
 
     ret = do_qmp_dispatch(cmds, request, allow_oob, &err);
 
-    rsp = qdict_new();
     if (err) {
-        qdict_put_obj(rsp, "error", qmp_build_error_object(err));
-        error_free(err);
+        rsp = qmp_error_response(err);
     } else if (ret) {
+        rsp = qdict_new();
         qdict_put_obj(rsp, "return", ret);
     } else {
-        qobject_unref(rsp);
-        return NULL;
+        rsp = NULL;
     }
 
     return QOBJECT(rsp);
diff --git a/qga/main.c b/qga/main.c
index d332bacce5..0e30e30248 100644
--- a/qga/main.c
+++ b/qga/main.c
@@ -610,15 +610,13 @@ static void process_event(JSONMessageParser *parser, GQueue *tokens)
     qdict = qobject_to(QDict, json_parser_parse_err(tokens, NULL, &err));
     if (err || !qdict) {
         qobject_unref(qdict);
-        qdict = qdict_new();
         if (!err) {
             g_warning("failed to parse event: unknown error");
             error_setg(&err, QERR_JSON_PARSING);
         } else {
             g_warning("failed to parse event: %s", error_get_pretty(err));
         }
-        qdict_put_obj(qdict, "error", qmp_build_error_object(err));
-        error_free(err);
+        qdict = qmp_error_response(err);
     }
 
     /* handle host->guest commands */
@@ -627,11 +625,9 @@ static void process_event(JSONMessageParser *parser, GQueue *tokens)
     } else {
         if (!qdict_haskey(qdict, "error")) {
             qobject_unref(qdict);
-            qdict = qdict_new();
             g_warning("unrecognized payload format");
             error_setg(&err, QERR_UNSUPPORTED);
-            qdict_put_obj(qdict, "error", qmp_build_error_object(err));
-            error_free(err);
+            qdict = qmp_error_response(err);
         }
         ret = send_response(s, QOBJECT(qdict));
         if (ret < 0) {
-- 
2.17.1

Re: [Qemu-devel] [PATCH 22/32] qmp: De-duplicate error response building

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> All callers of qmp_build_error_object() duplicate the code to wrap it
> in a response object.  Replace it by qmp_error_response() that
> captures the duplicated code, including error_free().
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   include/qapi/qmp/dispatch.h |  2 +-
>   monitor.c                   |  7 +------
>   qapi/qmp-dispatch.c         | 20 +++++++++++---------
>   qga/main.c                  |  8 ++------
>   4 files changed, 15 insertions(+), 22 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 23/32] qmp: Use QDict * instead of QObject * for response objects

[Markus Armbruster]

By using the more specific type, we get fewer downcasts.  The
downcasts are safe, but not obviously so, at least not locally.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 include/qapi/qmp/dispatch.h |  4 ++--
 monitor.c                   | 31 ++++++++++++++++---------------
 qapi/qmp-dispatch.c         |  6 +++---
 qga/main.c                  |  8 ++++----
 tests/test-qmp-cmds.c       | 17 +++++++----------
 5 files changed, 32 insertions(+), 34 deletions(-)

diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
index a53e11c9b1..4e2e749faf 100644
--- a/include/qapi/qmp/dispatch.h
+++ b/include/qapi/qmp/dispatch.h
@@ -48,8 +48,8 @@ bool qmp_command_is_enabled(const QmpCommand *cmd);
 const char *qmp_command_name(const QmpCommand *cmd);
 bool qmp_has_success_response(const QmpCommand *cmd);
 QDict *qmp_error_response(Error *err);
-QObject *qmp_dispatch(QmpCommandList *cmds, QObject *request,
-                      bool allow_oob);
+QDict *qmp_dispatch(QmpCommandList *cmds, QObject *request,
+                    bool allow_oob);
 bool qmp_is_oob(QDict *dict);
 
 typedef void (*qmp_cmd_callback_fn)(QmpCommand *cmd, void *opaque);
diff --git a/monitor.c b/monitor.c
index db85cd0a57..0f7a96213f 100644
--- a/monitor.c
+++ b/monitor.c
@@ -378,7 +378,7 @@ static void monitor_qmp_cleanup_req_queue_locked(Monitor *mon)
 static void monitor_qmp_cleanup_resp_queue_locked(Monitor *mon)
 {
     while (!g_queue_is_empty(mon->qmp.qmp_responses)) {
-        qobject_unref((QObject *)g_queue_pop_head(mon->qmp.qmp_responses));
+        qobject_unref((QDict *)g_queue_pop_head(mon->qmp.qmp_responses));
     }
 }
 
@@ -527,7 +527,8 @@ static void monitor_json_emitter(Monitor *mon, QObject *data)
          * responder thread).
          */
         qemu_mutex_lock(&mon->qmp.qmp_queue_lock);
-        g_queue_push_tail(mon->qmp.qmp_responses, qobject_ref(data));
+        g_queue_push_tail(mon->qmp.qmp_responses,
+                          qobject_ref(qobject_to(QDict, data)));
         qemu_mutex_unlock(&mon->qmp.qmp_queue_lock);
         qemu_bh_schedule(qmp_respond_bh);
     } else {
@@ -541,13 +542,13 @@ static void monitor_json_emitter(Monitor *mon, QObject *data)
 
 struct QMPResponse {
     Monitor *mon;
-    QObject *data;
+    QDict *data;
 };
 typedef struct QMPResponse QMPResponse;
 
-static QObject *monitor_qmp_response_pop_one(Monitor *mon)
+static QDict *monitor_qmp_response_pop_one(Monitor *mon)
 {
-    QObject *data;
+    QDict *data;
 
     qemu_mutex_lock(&mon->qmp.qmp_queue_lock);
     data = g_queue_pop_head(mon->qmp.qmp_responses);
@@ -558,10 +559,10 @@ static QObject *monitor_qmp_response_pop_one(Monitor *mon)
 
 static void monitor_qmp_response_flush(Monitor *mon)
 {
-    QObject *data;
+    QDict *data;
 
     while ((data = monitor_qmp_response_pop_one(mon))) {
-        monitor_json_emitter_raw(mon, data);
+        monitor_json_emitter_raw(mon, QOBJECT(data));
         qobject_unref(data);
     }
 }
@@ -573,7 +574,7 @@ static void monitor_qmp_response_flush(Monitor *mon)
 static bool monitor_qmp_response_pop_any(QMPResponse *response)
 {
     Monitor *mon;
-    QObject *data = NULL;
+    QDict *data = NULL;
 
     qemu_mutex_lock(&monitor_lock);
     QTAILQ_FOREACH(mon, &mon_list, entry) {
@@ -593,7 +594,7 @@ static void monitor_qmp_bh_responder(void *opaque)
     QMPResponse response;
 
     while (monitor_qmp_response_pop_any(&response)) {
-        monitor_json_emitter_raw(response.mon, response.data);
+        monitor_json_emitter_raw(response.mon, QOBJECT(response.data));
         qobject_unref(response.data);
     }
 }
@@ -4103,20 +4104,20 @@ static int monitor_can_read(void *opaque)
  * 2. rsp, err, and id may be NULL.
  * 3. If err != NULL then rsp must be NULL.
  */
-static void monitor_qmp_respond(Monitor *mon, QObject *rsp,
+static void monitor_qmp_respond(Monitor *mon, QDict *rsp,
                                 Error *err, QObject *id)
 {
     if (err) {
         assert(!rsp);
-        rsp = QOBJECT(qmp_error_response(err));
+        rsp = qmp_error_response(err);
     }
 
     if (rsp) {
         if (id) {
-            qdict_put_obj(qobject_to(QDict, rsp), "id", qobject_ref(id));
+            qdict_put_obj(rsp, "id", qobject_ref(id));
         }
 
-        monitor_json_emitter(mon, rsp);
+        monitor_json_emitter(mon, QOBJECT(rsp));
     }
 
     qobject_unref(id);
@@ -4126,7 +4127,7 @@ static void monitor_qmp_respond(Monitor *mon, QObject *rsp,
 static void monitor_qmp_dispatch(Monitor *mon, QObject *req, QObject *id)
 {
     Monitor *old_mon;
-    QObject *rsp;
+    QDict *rsp;
     QDict *error;
 
     old_mon = cur_mon;
@@ -4137,7 +4138,7 @@ static void monitor_qmp_dispatch(Monitor *mon, QObject *req, QObject *id)
     cur_mon = old_mon;
 
     if (mon->qmp.commands == &qmp_cap_negotiation_commands) {
-        error = qdict_get_qdict(qobject_to(QDict, rsp), "error");
+        error = qdict_get_qdict(rsp, "error");
         if (error
             && !g_strcmp0(qdict_get_try_str(error, "class"),
                     QapiErrorClass_str(ERROR_CLASS_COMMAND_NOT_FOUND))) {
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index c85748a33f..761812e924 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -161,8 +161,8 @@ bool qmp_is_oob(QDict *dict)
         && !qdict_haskey(dict, "execute");
 }
 
-QObject *qmp_dispatch(QmpCommandList *cmds, QObject *request,
-                      bool allow_oob)
+QDict *qmp_dispatch(QmpCommandList *cmds, QObject *request,
+                    bool allow_oob)
 {
     Error *err = NULL;
     QObject *ret;
@@ -179,5 +179,5 @@ QObject *qmp_dispatch(QmpCommandList *cmds, QObject *request,
         rsp = NULL;
     }
 
-    return QOBJECT(rsp);
+    return rsp;
 }
diff --git a/qga/main.c b/qga/main.c
index 0e30e30248..537cc0e162 100644
--- a/qga/main.c
+++ b/qga/main.c
@@ -545,7 +545,7 @@ fail:
 #endif
 }
 
-static int send_response(GAState *s, QObject *payload)
+static int send_response(GAState *s, QDict *payload)
 {
     const char *buf;
     QString *payload_qstr, *response_qstr;
@@ -553,7 +553,7 @@ static int send_response(GAState *s, QObject *payload)
 
     g_assert(payload && s->channel);
 
-    payload_qstr = qobject_to_json(payload);
+    payload_qstr = qobject_to_json(QOBJECT(payload));
     if (!payload_qstr) {
         return -EINVAL;
     }
@@ -581,7 +581,7 @@ static int send_response(GAState *s, QObject *payload)
 
 static void process_command(GAState *s, QDict *req)
 {
-    QObject *rsp = NULL;
+    QDict *rsp;
     int ret;
 
     g_assert(req);
@@ -629,7 +629,7 @@ static void process_event(JSONMessageParser *parser, GQueue *tokens)
             error_setg(&err, QERR_UNSUPPORTED);
             qdict = qmp_error_response(err);
         }
-        ret = send_response(s, QOBJECT(qdict));
+        ret = send_response(s, qdict);
         if (ret < 0) {
             g_warning("error sending error response: %s", strerror(-ret));
         }
diff --git a/tests/test-qmp-cmds.c b/tests/test-qmp-cmds.c
index 10c7ba40c1..afb338a61e 100644
--- a/tests/test-qmp-cmds.c
+++ b/tests/test-qmp-cmds.c
@@ -98,13 +98,13 @@ __org_qemu_x_Union1 *qmp___org_qemu_x_command(__org_qemu_x_EnumList *a,
 static void test_dispatch_cmd(void)
 {
     QDict *req = qdict_new();
-    QObject *resp;
+    QDict *resp;
 
     qdict_put_str(req, "execute", "user_def_cmd");
 
     resp = qmp_dispatch(&qmp_commands, QOBJECT(req), false);
     assert(resp != NULL);
-    assert(!qdict_haskey(qobject_to(QDict, resp), "error"));
+    assert(!qdict_haskey(resp, "error"));
 
     qobject_unref(resp);
     qobject_unref(req);
@@ -115,13 +115,13 @@ static void test_dispatch_cmd_failure(void)
 {
     QDict *req = qdict_new();
     QDict *args = qdict_new();
-    QObject *resp;
+    QDict *resp;
 
     qdict_put_str(req, "execute", "user_def_cmd2");
 
     resp = qmp_dispatch(&qmp_commands, QOBJECT(req), false);
     assert(resp != NULL);
-    assert(qdict_haskey(qobject_to(QDict, resp), "error"));
+    assert(qdict_haskey(resp, "error"));
 
     qobject_unref(resp);
     qobject_unref(req);
@@ -135,7 +135,7 @@ static void test_dispatch_cmd_failure(void)
 
     resp = qmp_dispatch(&qmp_commands, QOBJECT(req), false);
     assert(resp != NULL);
-    assert(qdict_haskey(qobject_to(QDict, resp), "error"));
+    assert(qdict_haskey(resp, "error"));
 
     qobject_unref(resp);
     qobject_unref(req);
@@ -143,18 +143,15 @@ static void test_dispatch_cmd_failure(void)
 
 static QObject *test_qmp_dispatch(QDict *req)
 {
-    QObject *resp_obj;
     QDict *resp;
     QObject *ret;
 
-    resp_obj = qmp_dispatch(&qmp_commands, QOBJECT(req), false);
-    assert(resp_obj);
-    resp = qobject_to(QDict, resp_obj);
+    resp = qmp_dispatch(&qmp_commands, QOBJECT(req), false);
     assert(resp && !qdict_haskey(resp, "error"));
     ret = qdict_get(resp, "return");
     assert(ret);
     qobject_ref(ret);
-    qobject_unref(resp_obj);
+    qobject_unref(resp);
     return ret;
 }
 
-- 
2.17.1

Re: [Qemu-devel] [PATCH 23/32] qmp: Use QDict * instead of QObject * for response objects

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> By using the more specific type, we get fewer downcasts.  The
> downcasts are safe, but not obviously so, at least not locally.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 24/32] qmp: Replace monitor_json_emitter{, raw}() by qmp_{queue, send}_response()

[Markus Armbruster]

monitor_json_emitter() and monitor_json_emitter_raw() are
unnecessarily general: they can send arbitrary JSON values, even
though we only ever use them for QMP, which may send only JSON
objects.

Specialize the argument from QObject * to QDict *, and rename to
qmp_queue_response(), qmp_send_response().

All callers but one lose an upcast.  The lone exception gains a
downcast; the next commit will get rid of it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c | 21 ++++++++++-----------
 1 file changed, 10 insertions(+), 11 deletions(-)

diff --git a/monitor.c b/monitor.c
index 0f7a96213f..399ad6b4e0 100644
--- a/monitor.c
+++ b/monitor.c
@@ -502,9 +502,9 @@ int monitor_fprintf(FILE *stream, const char *fmt, ...)
     return 0;
 }
 
-static void monitor_json_emitter_raw(Monitor *mon,
-                                     QObject *data)
+static void qmp_send_response(Monitor *mon, QDict *rsp)
 {
+    QObject *data = QOBJECT(rsp);
     QString *json;
 
     json = mon->flags & MONITOR_USE_PRETTY ? qobject_to_json_pretty(data) :
@@ -517,7 +517,7 @@ static void monitor_json_emitter_raw(Monitor *mon,
     qobject_unref(json);
 }
 
-static void monitor_json_emitter(Monitor *mon, QObject *data)
+static void qmp_queue_response(Monitor *mon, QDict *rsp)
 {
     if (mon->use_io_thread) {
         /*
@@ -527,8 +527,7 @@ static void monitor_json_emitter(Monitor *mon, QObject *data)
          * responder thread).
          */
         qemu_mutex_lock(&mon->qmp.qmp_queue_lock);
-        g_queue_push_tail(mon->qmp.qmp_responses,
-                          qobject_ref(qobject_to(QDict, data)));
+        g_queue_push_tail(mon->qmp.qmp_responses, qobject_ref(rsp));
         qemu_mutex_unlock(&mon->qmp.qmp_queue_lock);
         qemu_bh_schedule(qmp_respond_bh);
     } else {
@@ -536,7 +535,7 @@ static void monitor_json_emitter(Monitor *mon, QObject *data)
          * If not using monitor I/O thread, then we are in main thread.
          * Do the emission right away.
          */
-        monitor_json_emitter_raw(mon, data);
+        qmp_send_response(mon, rsp);
     }
 }
 
@@ -562,7 +561,7 @@ static void monitor_qmp_response_flush(Monitor *mon)
     QDict *data;
 
     while ((data = monitor_qmp_response_pop_one(mon))) {
-        monitor_json_emitter_raw(mon, QOBJECT(data));
+        qmp_send_response(mon, data);
         qobject_unref(data);
     }
 }
@@ -594,7 +593,7 @@ static void monitor_qmp_bh_responder(void *opaque)
     QMPResponse response;
 
     while (monitor_qmp_response_pop_any(&response)) {
-        monitor_json_emitter_raw(response.mon, QOBJECT(response.data));
+        qmp_send_response(response.mon, response.data);
         qobject_unref(response.data);
     }
 }
@@ -621,7 +620,7 @@ static void monitor_qapi_event_emit(QAPIEvent event, QDict *qdict)
     QTAILQ_FOREACH(mon, &mon_list, entry) {
         if (monitor_is_qmp(mon)
             && mon->qmp.commands != &qmp_cap_negotiation_commands) {
-            monitor_json_emitter(mon, QOBJECT(qdict));
+            qmp_queue_response(mon, qdict);
         }
     }
 }
@@ -4117,7 +4116,7 @@ static void monitor_qmp_respond(Monitor *mon, QDict *rsp,
             qdict_put_obj(rsp, "id", qobject_ref(id));
         }
 
-        monitor_json_emitter(mon, QOBJECT(rsp));
+        qmp_queue_response(mon, rsp);
     }
 
     qobject_unref(id);
@@ -4417,7 +4416,7 @@ static void monitor_qmp_event(void *opaque, int event)
         mon->qmp.commands = &qmp_cap_negotiation_commands;
         monitor_qmp_caps_reset(mon);
         data = get_qmp_greeting(mon);
-        monitor_json_emitter(mon, data);
+        qmp_queue_response(mon, qobject_to(QDict, data));
         qobject_unref(data);
         mon_refcount++;
         break;
-- 
2.17.1

Re: [Qemu-devel] [PATCH 24/32] qmp: Replace monitor_json_emitter{, raw}() by qmp_{queue, send}_response()

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> monitor_json_emitter() and monitor_json_emitter_raw() are
> unnecessarily general: they can send arbitrary JSON values, even
> though we only ever use them for QMP, which may send only JSON
> objects.
> 
> Specialize the argument from QObject * to QDict *, and rename to
> qmp_queue_response(), qmp_send_response().
> 
> All callers but one lose an upcast.  The lone exception gains a
> downcast; the next commit will get rid of it.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c | 21 ++++++++++-----------
>   1 file changed, 10 insertions(+), 11 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

And the new names are easier to tell apart.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 25/32] qmp: Replace get_qmp_greeting() by qmp_greeting()

[Markus Armbruster]

get_qmp_greeting() returns a QDict * as QObject *.  It's caller
converts it right back.

Return QDict * instead.  While there, rename to qmp_greeting().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/monitor.c b/monitor.c
index 399ad6b4e0..ba3124514c 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4381,7 +4381,7 @@ void monitor_resume(Monitor *mon)
     trace_monitor_suspend(mon, -1);
 }
 
-static QObject *get_qmp_greeting(Monitor *mon)
+static QDict *qmp_greeting(Monitor *mon)
 {
     QList *cap_list = qlist_new();
     QObject *ver = NULL;
@@ -4397,8 +4397,9 @@ static QObject *get_qmp_greeting(Monitor *mon)
         qlist_append_str(cap_list, QMPCapability_str(cap));
     }
 
-    return qobject_from_jsonf("{'QMP': {'version': %p, 'capabilities': %p}}",
-                              ver, cap_list);
+    return qdict_from_jsonf_nofail(
+        "{'QMP': {'version': %p, 'capabilities': %p}}",
+        ver, cap_list);
 }
 
 static void monitor_qmp_caps_reset(Monitor *mon)
@@ -4408,15 +4409,15 @@ static void monitor_qmp_caps_reset(Monitor *mon)
 
 static void monitor_qmp_event(void *opaque, int event)
 {
-    QObject *data;
+    QDict *data;
     Monitor *mon = opaque;
 
     switch (event) {
     case CHR_EVENT_OPENED:
         mon->qmp.commands = &qmp_cap_negotiation_commands;
         monitor_qmp_caps_reset(mon);
-        data = get_qmp_greeting(mon);
-        qmp_queue_response(mon, qobject_to(QDict, data));
+        data = qmp_greeting(mon);
+        qmp_queue_response(mon, data);
         qobject_unref(data);
         mon_refcount++;
         break;
-- 
2.17.1

Re: [Qemu-devel] [PATCH 25/32] qmp: Replace get_qmp_greeting() by qmp_greeting()

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> get_qmp_greeting() returns a QDict * as QObject *.  It's caller
> converts it right back.
> 
> Return QDict * instead.  While there, rename to qmp_greeting().
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c | 13 +++++++------
>   1 file changed, 7 insertions(+), 6 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 26/32] qmp: Simplify monitor_qmp_respond()

[Markus Armbruster]

monitor_qmp_respond() takes both a response object and an error
object.  If an error object is non-null, the response object must be
null, and the response is built from the error object.

Of the two callers, one always passes a null response object, and one
a null error object.  Move building the response object from the error
object to the latter, and drop the error object parameter.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c | 26 ++++++++++----------------
 1 file changed, 10 insertions(+), 16 deletions(-)

diff --git a/monitor.c b/monitor.c
index ba3124514c..876a3a23a7 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4099,18 +4099,12 @@ static int monitor_can_read(void *opaque)
 }
 
 /*
- * 1. This function takes ownership of rsp, err, and id.
- * 2. rsp, err, and id may be NULL.
- * 3. If err != NULL then rsp must be NULL.
+ * Emit QMP response @rsp with ID @id to @mon.
+ * Null @rsp can only happen for commands with QCO_NO_SUCCESS_RESP.
+ * Nothing is emitted then.
  */
-static void monitor_qmp_respond(Monitor *mon, QDict *rsp,
-                                Error *err, QObject *id)
+static void monitor_qmp_respond(Monitor *mon, QDict *rsp, QObject *id)
 {
-    if (err) {
-        assert(!rsp);
-        rsp = qmp_error_response(err);
-    }
-
     if (rsp) {
         if (id) {
             qdict_put_obj(rsp, "id", qobject_ref(id));
@@ -4118,9 +4112,6 @@ static void monitor_qmp_respond(Monitor *mon, QDict *rsp,
 
         qmp_queue_response(mon, rsp);
     }
-
-    qobject_unref(id);
-    qobject_unref(rsp);
 }
 
 static void monitor_qmp_dispatch(Monitor *mon, QObject *req, QObject *id)
@@ -4148,8 +4139,8 @@ static void monitor_qmp_dispatch(Monitor *mon, QObject *req, QObject *id)
         }
     }
 
-    /* Respond if necessary */
-    monitor_qmp_respond(mon, rsp, NULL, qobject_ref(id));
+    monitor_qmp_respond(mon, rsp, id);
+    qobject_unref(rsp);
 }
 
 /*
@@ -4192,6 +4183,7 @@ static QMPRequest *monitor_qmp_requests_pop_any(void)
 static void monitor_qmp_bh_dispatcher(void *data)
 {
     QMPRequest *req_obj = monitor_qmp_requests_pop_any();
+    QDict *rsp;
 
     if (!req_obj) {
         return;
@@ -4202,7 +4194,9 @@ static void monitor_qmp_bh_dispatcher(void *data)
         monitor_qmp_dispatch(req_obj->mon, req_obj->req, req_obj->id);
     } else {
         assert(req_obj->err);
-        monitor_qmp_respond(req_obj->mon, NULL, req_obj->err, NULL);
+        rsp = qmp_error_response(req_obj->err);
+        monitor_qmp_respond(req_obj->mon, rsp, NULL);
+        qobject_unref(rsp);
     }
 
     if (req_obj->need_resume) {
-- 
2.17.1

Re: [Qemu-devel] [PATCH 26/32] qmp: Simplify monitor_qmp_respond()

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> monitor_qmp_respond() takes both a response object and an error
> object.  If an error object is non-null, the response object must be
> null, and the response is built from the error object.
> 
> Of the two callers, one always passes a null response object, and one
> a null error object.  Move building the response object from the error
> object to the latter, and drop the error object parameter.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c | 26 ++++++++++----------------
>   1 file changed, 10 insertions(+), 16 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 27/32] qmp: Add some comments around null responses

[Markus Armbruster]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/qmp-dispatch.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 761812e924..6f2d466596 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -133,6 +133,7 @@ static QObject *do_qmp_dispatch(QmpCommandList *cmds, QObject *request,
     } else if (cmd->options & QCO_NO_SUCCESS_RESP) {
         g_assert(!ret);
     } else if (!ret) {
+        /* TODO turn into assertion */
         ret = QOBJECT(qdict_new());
     }
 
@@ -176,6 +177,7 @@ QDict *qmp_dispatch(QmpCommandList *cmds, QObject *request,
         rsp = qdict_new();
         qdict_put_obj(rsp, "return", ret);
     } else {
+        /* Can only happen for commands with QCO_NO_SUCCESS_RESP */
         rsp = NULL;
     }
 
-- 
2.17.1

Re: [Qemu-devel] [PATCH 27/32] qmp: Add some comments around null responses

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   qapi/qmp-dispatch.c | 2 ++
>   1 file changed, 2 insertions(+)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

> diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
> index 761812e924..6f2d466596 100644
> --- a/qapi/qmp-dispatch.c
> +++ b/qapi/qmp-dispatch.c
> @@ -133,6 +133,7 @@ static QObject *do_qmp_dispatch(QmpCommandList *cmds, QObject *request,
>       } else if (cmd->options & QCO_NO_SUCCESS_RESP) {
>           g_assert(!ret);
>       } else if (!ret) {
> +        /* TODO turn into assertion */
>           ret = QOBJECT(qdict_new());
>       }
>   
> @@ -176,6 +177,7 @@ QDict *qmp_dispatch(QmpCommandList *cmds, QObject *request,
>           rsp = qdict_new();
>           qdict_put_obj(rsp, "return", ret);
>       } else {
> +        /* Can only happen for commands with QCO_NO_SUCCESS_RESP */
>           rsp = NULL;
>       }
>   
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 28/32] qmp: Switch timestamp_put() to qdict_from_jsonf_nofail()

[Markus Armbruster]

There's just one use of qobject_from_jsonf() to parse a JSON object
left: timestamp_put().  Switch it to qdict_from_jsonf_nofail().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/qmp-event.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/qapi/qmp-event.c b/qapi/qmp-event.c
index 9d7e88e84a..5b8854043e 100644
--- a/qapi/qmp-event.c
+++ b/qapi/qmp-event.c
@@ -34,15 +34,15 @@ QMPEventFuncEmit qmp_event_get_func_emit(void)
 static void timestamp_put(QDict *qdict)
 {
     int err;
-    QObject *obj;
+    QDict *ts;
     qemu_timeval tv;
 
     err = qemu_gettimeofday(&tv);
     /* Put -1 to indicate failure of getting host time */
-    obj = qobject_from_jsonf("{ 'seconds': %lld, 'microseconds': %lld }",
-                             err < 0 ? -1LL : (long long)tv.tv_sec,
-                             err < 0 ? -1LL : (long long)tv.tv_usec);
-    qdict_put_obj(qdict, "timestamp", obj);
+    ts = qdict_from_jsonf_nofail("{ 'seconds': %lld, 'microseconds': %lld }",
+                                 err < 0 ? -1LL : (long long)tv.tv_sec,
+                                 err < 0 ? -1LL : (long long)tv.tv_usec);
+    qdict_put(qdict, "timestamp", ts);
 }
 
 /*
-- 
2.17.1

Re: [Qemu-devel] [PATCH 28/32] qmp: Switch timestamp_put() to qdict_from_jsonf_nofail()

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> There's just one use of qobject_from_jsonf() to parse a JSON object
> left: timestamp_put().  Switch it to qdict_from_jsonf_nofail().
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   qapi/qmp-event.c | 10 +++++-----
>   1 file changed, 5 insertions(+), 5 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 29/32] qobject: Let qobject_from_jsonf() fail instead of abort

[Markus Armbruster]

qobject_from_jsonf() aborts on error, unlike qobject_from_jsonv(),
which returns null.  Since all remaining users of qobject_from_jsonf()
cope fine with null, change it to return null.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qobject/qjson.c | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/qobject/qjson.c b/qobject/qjson.c
index 0df3120202..2f6a590e44 100644
--- a/qobject/qjson.c
+++ b/qobject/qjson.c
@@ -59,10 +59,6 @@ QObject *qobject_from_json(const char *string, Error **errp)
     return qobject_from_jsonv(string, NULL, errp);
 }
 
-/*
- * IMPORTANT: This function aborts on error, thus it must not
- * be used with untrusted arguments.
- */
 QObject *qobject_from_jsonf(const char *string, ...)
 {
     QObject *obj;
@@ -72,7 +68,6 @@ QObject *qobject_from_jsonf(const char *string, ...)
     obj = qobject_from_jsonv(string, &ap, &error_abort);
     va_end(ap);
 
-    assert(obj != NULL);
     return obj;
 }
 
-- 
2.17.1

Re: [Qemu-devel] [PATCH 29/32] qobject: Let qobject_from_jsonf() fail instead of abort

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> qobject_from_jsonf() aborts on error, unlike qobject_from_jsonv(),
> which returns null.  Since all remaining users of qobject_from_jsonf()
> cope fine with null, change it to return null.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   qobject/qjson.c | 5 -----
>   1 file changed, 5 deletions(-)
> 

Nice that we're finally gaining some symmetry between the ... and 
va_list forms.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 30/32] qmp: Clean up capability negotiation after commit 02130314d8c

[Markus Armbruster]

qmp_greeting() offers capabilities to the client, and
qmp_qmp_capabilities() accepts or denies capabilities requested by the
client.  The two compute the set of available capabilities
independently.  Not nice.

Clean this up as follows.  Compute available capabilities just once in
monitor_qmp_caps_reset(), and store them in Monitor member
qmp.capab_offered[].  Have qmp_greeting() and qmp_qmp_capabilities()
use that.  Both are now oblivious of capability details.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c | 93 ++++++++++++++++++++++++-------------------------------
 1 file changed, 40 insertions(+), 53 deletions(-)

diff --git a/monitor.c b/monitor.c
index 876a3a23a7..590e5b5b04 100644
--- a/monitor.c
+++ b/monitor.c
@@ -172,7 +172,8 @@ typedef struct {
      * mode.
      */
     QmpCommandList *commands;
-    bool qmp_caps[QMP_CAPABILITY__MAX];
+    bool capab_offered[QMP_CAPABILITY__MAX]; /* capabilities offered */
+    bool capab[QMP_CAPABILITY__MAX];         /* offered and accepted */
     /*
      * Protects qmp request/response queue.  Please take monitor_lock
      * first when used together.
@@ -1252,52 +1253,56 @@ static void monitor_init_qmp_commands(void)
                          qmp_marshal_qmp_capabilities, QCO_ALLOW_PRECONFIG);
 }
 
-static bool qmp_cap_enabled(Monitor *mon, QMPCapability cap)
-{
-    return mon->qmp.qmp_caps[cap];
-}
-
 static bool qmp_oob_enabled(Monitor *mon)
 {
-    return qmp_cap_enabled(mon, QMP_CAPABILITY_OOB);
+    return mon->qmp.capab[QMP_CAPABILITY_OOB];
 }
 
-static void qmp_caps_check(Monitor *mon, QMPCapabilityList *list,
-                           Error **errp)
+static void monitor_qmp_caps_reset(Monitor *mon)
 {
+    memset(mon->qmp.capab_offered, 0, sizeof(mon->qmp.capab_offered));
+    memset(mon->qmp.capab, 0, sizeof(mon->qmp.capab));
+    mon->qmp.capab_offered[QMP_CAPABILITY_OOB] = mon->use_io_thread;
+}
+
+/*
+ * Accept QMP capabilities in @list for @mon.
+ * On success, set mon->qmp.capab[], and return true.
+ * On error, set @errp, and return false.
+ */
+static bool qmp_caps_accept(Monitor *mon, QMPCapabilityList *list,
+                            Error **errp)
+{
+    GString *unavailable = NULL;
+    bool capab[QMP_CAPABILITY__MAX];
+
+    memset(capab, 0, sizeof(capab));
+
     for (; list; list = list->next) {
-        assert(list->value < QMP_CAPABILITY__MAX);
-        switch (list->value) {
-        case QMP_CAPABILITY_OOB:
-            if (!mon->use_io_thread) {
-                /*
-                 * Out-of-band only works with monitors that are
-                 * running on dedicated I/O thread.
-                 */
-                error_setg(errp, "This monitor does not support "
-                           "out-of-band (OOB)");
-                return;
+        if (!mon->qmp.capab_offered[list->value]) {
+            if (!unavailable) {
+                unavailable = g_string_new(QMPCapability_str(list->value));
+            } else {
+                g_string_append_printf(unavailable, ", %s",
+                                      QMPCapability_str(list->value));
             }
-            break;
-        default:
-            break;
         }
+        capab[list->value] = true;
     }
-}
 
-/* This function should only be called after capabilities are checked. */
-static void qmp_caps_apply(Monitor *mon, QMPCapabilityList *list)
-{
-    for (; list; list = list->next) {
-        mon->qmp.qmp_caps[list->value] = true;
+    if (unavailable) {
+        error_setg(errp, "Capability %s not available", unavailable->str);
+        g_string_free(unavailable, true);
+        return false;
     }
+
+    memcpy(mon->qmp.capab, capab, sizeof(capab));
+    return true;
 }
 
 void qmp_qmp_capabilities(bool has_enable, QMPCapabilityList *enable,
                           Error **errp)
 {
-    Error *local_err = NULL;
-
     if (cur_mon->qmp.commands == &qmp_commands) {
         error_set(errp, ERROR_CLASS_COMMAND_NOT_FOUND,
                   "Capabilities negotiation is already complete, command "
@@ -1305,19 +1310,8 @@ void qmp_qmp_capabilities(bool has_enable, QMPCapabilityList *enable,
         return;
     }
 
-    /* Enable QMP capabilities provided by the client if applicable. */
-    if (has_enable) {
-        qmp_caps_check(cur_mon, enable, &local_err);
-        if (local_err) {
-            /*
-             * Failed check on any of the capabilities will fail the
-             * entire command (and thus not apply any of the other
-             * capabilities that were also requested).
-             */
-            error_propagate(errp, local_err);
-            return;
-        }
-        qmp_caps_apply(cur_mon, enable);
+    if (!qmp_caps_accept(cur_mon, enable, errp)) {
+        return;
     }
 
     cur_mon->qmp.commands = &qmp_commands;
@@ -4384,11 +4378,9 @@ static QDict *qmp_greeting(Monitor *mon)
     qmp_marshal_query_version(NULL, &ver, NULL);
 
     for (cap = 0; cap < QMP_CAPABILITY__MAX; cap++) {
-        if (!mon->use_io_thread && cap == QMP_CAPABILITY_OOB) {
-            /* Monitors that are not using I/O thread won't support OOB */
-            continue;
+        if (mon->qmp.capab_offered[cap]) {
+            qlist_append_str(cap_list, QMPCapability_str(cap));
         }
-        qlist_append_str(cap_list, QMPCapability_str(cap));
     }
 
     return qdict_from_jsonf_nofail(
@@ -4396,11 +4388,6 @@ static QDict *qmp_greeting(Monitor *mon)
         ver, cap_list);
 }
 
-static void monitor_qmp_caps_reset(Monitor *mon)
-{
-    memset(mon->qmp.qmp_caps, 0, sizeof(mon->qmp.qmp_caps));
-}
-
 static void monitor_qmp_event(void *opaque, int event)
 {
     QDict *data;
-- 
2.17.1

Re: [Qemu-devel] [PATCH 30/32] qmp: Clean up capability negotiation after commit 02130314d8c

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> qmp_greeting() offers capabilities to the client, and
> qmp_qmp_capabilities() accepts or denies capabilities requested by the
> client.  The two compute the set of available capabilities
> independently.  Not nice.
> 
> Clean this up as follows.  Compute available capabilities just once in
> monitor_qmp_caps_reset(), and store them in Monitor member
> qmp.capab_offered[].  Have qmp_greeting() and qmp_qmp_capabilities()
> use that.  Both are now oblivious of capability details.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c | 93 ++++++++++++++++++++++++-------------------------------
>   1 file changed, 40 insertions(+), 53 deletions(-)
> 
> diff --git a/monitor.c b/monitor.c
> index 876a3a23a7..590e5b5b04 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -172,7 +172,8 @@ typedef struct {
>        * mode.
>        */
>       QmpCommandList *commands;
> -    bool qmp_caps[QMP_CAPABILITY__MAX];
> +    bool capab_offered[QMP_CAPABILITY__MAX]; /* capabilities offered */
> +    bool capab[QMP_CAPABILITY__MAX];         /* offered and accepted */

Could also store as uint32_t, using '1U << bit' to compute various 
capabilities as a bitmask (up to 32 capabilities).  But it's not 
frequently executed, so this works.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH 30/32] qmp: Clean up capability negotiation after commit 02130314d8c

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 07/02/2018 11:22 AM, Markus Armbruster wrote:
>> qmp_greeting() offers capabilities to the client, and
>> qmp_qmp_capabilities() accepts or denies capabilities requested by the
>> client.  The two compute the set of available capabilities
>> independently.  Not nice.
>>
>> Clean this up as follows.  Compute available capabilities just once in
>> monitor_qmp_caps_reset(), and store them in Monitor member
>> qmp.capab_offered[].  Have qmp_greeting() and qmp_qmp_capabilities()
>> use that.  Both are now oblivious of capability details.
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>   monitor.c | 93 ++++++++++++++++++++++++-------------------------------
>>   1 file changed, 40 insertions(+), 53 deletions(-)
>>
>> diff --git a/monitor.c b/monitor.c
>> index 876a3a23a7..590e5b5b04 100644
>> --- a/monitor.c
>> +++ b/monitor.c
>> @@ -172,7 +172,8 @@ typedef struct {
>>        * mode.
>>        */
>>       QmpCommandList *commands;
>> -    bool qmp_caps[QMP_CAPABILITY__MAX];
>> +    bool capab_offered[QMP_CAPABILITY__MAX]; /* capabilities offered */
>> +    bool capab[QMP_CAPABILITY__MAX];         /* offered and accepted */
>
> Could also store as uint32_t, using '1U << bit' to compute various
> capabilities as a bitmask (up to 32 capabilities).  But it's not
> frequently executed, so this works.

Yes.  I chose to lessen the churn.  On a green field, I guess I'd go
with bits out of habit.  Might not even be a good habit :)

> Reviewed-by: Eric Blake <eblake@redhat.com>

Thanks!

[Qemu-devel] [PATCH 31/32] monitor: Improve some comments

[Markus Armbruster]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c | 100 ++++++++++++++++++++++++------------------------------
 1 file changed, 45 insertions(+), 55 deletions(-)

diff --git a/monitor.c b/monitor.c
index 590e5b5b04..14af7b7ea6 100644
--- a/monitor.c
+++ b/monitor.c
@@ -168,15 +168,16 @@ typedef struct {
     JSONMessageParser parser;
     /*
      * When a client connects, we're in capabilities negotiation mode.
-     * When command qmp_capabilities succeeds, we go into command
-     * mode.
+     * @commands is &qmp_cap_negotiation_commands then.  When command
+     * qmp_capabilities succeeds, we go into command mode, and
+     * @command becomes &qmp_commands.
      */
     QmpCommandList *commands;
     bool capab_offered[QMP_CAPABILITY__MAX]; /* capabilities offered */
     bool capab[QMP_CAPABILITY__MAX];         /* offered and accepted */
     /*
-     * Protects qmp request/response queue.  Please take monitor_lock
-     * first when used together.
+     * Protects qmp request/response queue.
+     * Take monitor_lock first when you need both.
      */
     QemuMutex qmp_queue_lock;
     /* Input queue that holds all the parsed QMP requests */
@@ -231,7 +232,7 @@ struct Monitor {
     QemuMutex mon_lock;
 
     /*
-     * Fields that are protected by the per-monitor lock.
+     * Members that are protected by the per-monitor lock
      */
     QLIST_HEAD(, mon_fd_t) fds;
     QString *outbuf;
@@ -240,6 +241,7 @@ struct Monitor {
     int mux_out;
 };
 
+/* Shared monitor I/O thread */
 IOThread *mon_iothread;
 
 /* Bottom half to dispatch the requests received from I/O thread */
@@ -301,9 +303,9 @@ static inline bool monitor_is_qmp(const Monitor *mon)
 }
 
 /**
- * Whether @mon is using readline?  Note: not all HMP monitors use
- * readline, e.g., gdbserver has a non-interactive HMP monitor, so
- * readline is not used there.
+ * Is @mon is using readline?
+ * Note: not all HMP monitors use readline, e.g., gdbserver has a
+ * non-interactive HMP monitor, so readline is not used there.
  */
 static inline bool monitor_uses_readline(const Monitor *mon)
 {
@@ -317,14 +319,12 @@ static inline bool monitor_is_hmp_non_interactive(const Monitor *mon)
 
 /*
  * Return the clock to use for recording an event's time.
+ * It's QEMU_CLOCK_REALTIME, except for qtests it's
+ * QEMU_CLOCK_VIRTUAL, to support testing rate limits.
  * Beware: result is invalid before configure_accelerator().
  */
 static inline QEMUClockType monitor_get_event_clock(void)
 {
-    /*
-     * This allows us to perform tests on the monitor queues to verify
-     * that the rate limits are enforced.
-     */
     return qtest_enabled() ? QEMU_CLOCK_VIRTUAL : QEMU_CLOCK_REALTIME;
 }
 
@@ -367,7 +367,7 @@ static void qmp_request_free(QMPRequest *req)
     g_free(req);
 }
 
-/* Must with the mon->qmp.qmp_queue_lock held */
+/* Caller must hold mon->qmp.qmp_queue_lock */
 static void monitor_qmp_cleanup_req_queue_locked(Monitor *mon)
 {
     while (!g_queue_is_empty(mon->qmp.qmp_requests)) {
@@ -375,7 +375,7 @@ static void monitor_qmp_cleanup_req_queue_locked(Monitor *mon)
     }
 }
 
-/* Must with the mon->qmp.qmp_queue_lock held */
+/* Caller must hold the mon->qmp.qmp_queue_lock */
 static void monitor_qmp_cleanup_resp_queue_locked(Monitor *mon)
 {
     while (!g_queue_is_empty(mon->qmp.qmp_responses)) {
@@ -406,7 +406,7 @@ static gboolean monitor_unblocked(GIOChannel *chan, GIOCondition cond,
     return FALSE;
 }
 
-/* Called with mon->mon_lock held.  */
+/* Caller must hold mon->mon_lock */
 static void monitor_flush_locked(Monitor *mon)
 {
     int rc;
@@ -522,10 +522,8 @@ static void qmp_queue_response(Monitor *mon, QDict *rsp)
 {
     if (mon->use_io_thread) {
         /*
-         * If using I/O thread, we need to queue the item so that I/O
-         * thread will do the rest for us.  Take refcount so that
-         * caller won't free the data (which will be finally freed in
-         * responder thread).
+         * Push a reference to the response queue.  The I/O thread
+         * drains that queue and emits.
          */
         qemu_mutex_lock(&mon->qmp.qmp_queue_lock);
         g_queue_push_tail(mon->qmp.qmp_responses, qobject_ref(rsp));
@@ -533,8 +531,8 @@ static void qmp_queue_response(Monitor *mon, QDict *rsp)
         qemu_bh_schedule(qmp_respond_bh);
     } else {
         /*
-         * If not using monitor I/O thread, then we are in main thread.
-         * Do the emission right away.
+         * Not using monitor I/O thread, i.e. we are in the main thread.
+         * Emit right away.
          */
         qmp_send_response(mon, rsp);
     }
@@ -610,8 +608,9 @@ static MonitorQAPIEventConf monitor_qapi_event_conf[QAPI_EVENT__MAX] = {
 };
 
 /*
- * Emits the event to every monitor instance, @event is only used for trace
- * Called with monitor_lock held.
+ * Broadcast an event to all monitors.
+ * @qdict is the event object.  Its member "event" must match @event.
+ * Caller must hold monitor_lock.
  */
 static void monitor_qapi_event_emit(QAPIEvent event, QDict *qdict)
 {
@@ -980,8 +979,7 @@ static int parse_cmdline(const char *cmdline,
 }
 
 /*
- * Returns true if the command can be executed in preconfig mode
- * i.e. it has the 'p' flag.
+ * Can command @cmd be executed in preconfig state?
  */
 static bool cmd_can_preconfig(const mon_cmd_t *cmd)
 {
@@ -2220,7 +2218,7 @@ void qmp_getfd(const char *fdname, Error **errp)
         tmp_fd = monfd->fd;
         monfd->fd = fd;
         qemu_mutex_unlock(&cur_mon->mon_lock);
-        /* Make sure close() is out of critical section */
+        /* Make sure close() is outside critical section */
         close(tmp_fd);
         return;
     }
@@ -2249,7 +2247,7 @@ void qmp_closefd(const char *fdname, Error **errp)
         g_free(monfd->name);
         g_free(monfd);
         qemu_mutex_unlock(&cur_mon->mon_lock);
-        /* Make sure close() is out of critical section */
+        /* Make sure close() is outside critical section */
         close(tmp_fd);
         return;
     }
@@ -4138,7 +4136,8 @@ static void monitor_qmp_dispatch(Monitor *mon, QObject *req, QObject *id)
 }
 
 /*
- * Pop one QMP request from monitor queues, return NULL if not found.
+ * Pop a QMP request from a monitor request queue.
+ * Return the request, or NULL all request queues are empty.
  * We are using round-robin fashion to pop the request, to avoid
  * processing commands only on a very busy monitor.  To achieve that,
  * when we process one request on a specific monitor, we put that
@@ -4233,7 +4232,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     }
 
     if (qdict && qmp_is_oob(qdict)) {
-        /* Out-of-band (OOB) requests are executed directly in parser. */
+        /* OOB commands are executed immediately */
         trace_monitor_qmp_cmd_out_of_band(qobject_get_try_str(id)
                                           ?: "");
         monitor_qmp_dispatch(mon, req, id);
@@ -4355,8 +4354,8 @@ void monitor_resume(Monitor *mon)
     if (atomic_dec_fetch(&mon->suspend_cnt) == 0) {
         if (monitor_is_qmp(mon)) {
             /*
-             * For QMP monitors that are running in I/O thread, let's
-             * kick the thread in case it's sleeping.
+             * For QMP monitors that are running in the I/O thread,
+             * let's kick the thread in case it's sleeping.
              */
             if (mon->use_io_thread) {
                 aio_notify(iothread_get_aio_context(mon_iothread));
@@ -4504,18 +4503,18 @@ static void monitor_iothread_init(void)
     mon_iothread = iothread_create("mon_iothread", &error_abort);
 
     /*
-     * This MUST be on main loop thread since we have commands that
-     * have assumption to be run on main loop thread.  It would be
-     * nice that one day we can remove this assumption in the future.
+     * The dispatcher BH must run in the main loop thread, since we
+     * have commands assuming that context.  It would be nice to get
+     * rid of those assumptions.
      */
     qmp_dispatcher_bh = aio_bh_new(iohandler_get_aio_context(),
                                    monitor_qmp_bh_dispatcher,
                                    NULL);
 
     /*
-     * Unlike the dispatcher BH, this must be run on the monitor I/O
-     * thread, so that monitors that are using I/O thread will make
-     * sure read/write operations are all done on the I/O thread.
+     * The responder BH must be run in the monitor I/O thread, so that
+     * monitors that are using the I/O thread have their output
+     * written by the I/O thread.
      */
     qmp_respond_bh = aio_bh_new(monitor_get_aio_context(),
                                 monitor_qmp_bh_responder,
@@ -4585,15 +4584,11 @@ static void monitor_qmp_setup_handlers_bh(void *opaque)
     GMainContext *context;
 
     if (mon->use_io_thread) {
-        /*
-         * When @use_io_thread is set, we use the global shared dedicated
-         * I/O thread for this monitor to handle input/output.
-         */
+        /* Use @mon_iothread context */
         context = monitor_get_io_context();
-        /* We should have inited globals before reaching here. */
         assert(context);
     } else {
-        /* The default main loop, which is the main thread */
+        /* Use default main loop context */
         context = NULL;
     }
 
@@ -4643,15 +4638,12 @@ void monitor_init(Chardev *chr, int flags)
             remove_fd_in_watch(chr);
             /*
              * We can't call qemu_chr_fe_set_handlers() directly here
-             * since during the procedure the chardev will be active
-             * and running in monitor I/O thread, while we'll still do
-             * something before returning from it, which is a possible
-             * race too.  To avoid that, we just create a BH to setup
-             * the handlers.
+             * since chardev might be running in the monitor I/O
+             * thread.  Schedule a bottom half.
              */
             aio_bh_schedule_oneshot(monitor_get_aio_context(),
                                     monitor_qmp_setup_handlers_bh, mon);
-            /* We'll add this to mon_list in the BH when setup done */
+            /* The bottom half will add @mon to @mon_list */
             return;
         } else {
             qemu_chr_fe_set_handlers(&mon->chr, monitor_can_read,
@@ -4672,21 +4664,19 @@ void monitor_cleanup(void)
 
     /*
      * We need to explicitly stop the I/O thread (but not destroy it),
-     * cleanup the monitor resources, then destroy the I/O thread since
+     * clean up the monitor resources, then destroy the I/O thread since
      * we need to unregister from chardev below in
      * monitor_data_destroy(), and chardev is not thread-safe yet
      */
     iothread_stop(mon_iothread);
 
     /*
-     * After we have I/O thread to send responses, it's possible that
-     * when we stop the I/O thread there are still replies queued in the
-     * responder queue.  Flush all of them.  Note that even after this
-     * flush it's still possible that out buffer is not flushed.
-     * It'll be done in below monitor_flush() as the last resort.
+     * Flush all response queues.  Note that even after this flush,
+     * data may remain in output buffers.
      */
     monitor_qmp_bh_responder(NULL);
 
+    /* Flush output buffers and destroy monitors */
     qemu_mutex_lock(&monitor_lock);
     QTAILQ_FOREACH_SAFE(mon, &mon_list, entry, next) {
         QTAILQ_REMOVE(&mon_list, mon, entry);
-- 
2.17.1

Re: [Qemu-devel] [PATCH 31/32] monitor: Improve some comments

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   monitor.c | 100 ++++++++++++++++++++++++------------------------------
>   1 file changed, 45 insertions(+), 55 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH 32/32] qapi: Polish command flags documentation in qapi-code-gen.txt

[Markus Armbruster]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 docs/devel/qapi-code-gen.txt | 61 +++++++++++++++---------------------
 1 file changed, 25 insertions(+), 36 deletions(-)

diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index f020f6bab2..8decd6f17d 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -624,60 +624,48 @@ its return value.
 In rare cases, QAPI cannot express a type-safe representation of a
 corresponding Client JSON Protocol command.  You then have to suppress
 generation of a marshalling function by including a key 'gen' with
-boolean value false, and instead write your own function.  Please try
-to avoid adding new commands that rely on this, and instead use
-type-safe unions.  For an example of this usage:
+boolean value false, and instead write your own function.  For
+example:
 
  { 'command': 'netdev_add',
    'data': {'type': 'str', 'id': 'str'},
    'gen': false }
 
+Please try to avoid adding new commands that rely on this, and instead
+use type-safe unions.
+
 Normally, the QAPI schema is used to describe synchronous exchanges,
 where a response is expected.  But in some cases, the action of a
 command is expected to change state in a way that a successful
 response is not possible (although the command will still return a
 normal dictionary error on failure).  When a successful reply is not
-possible, the command expression should include the optional key
+possible, the command expression includes the optional key
 'success-response' with boolean value false.  So far, only QGA makes
 use of this member.
 
-A command can be declared to support out-of-band (OOB) execution.  By
-default, commands do not support OOB.  To declare a command that
-supports it, the schema includes an extra 'allow-oob' field.  For
-example:
+Key 'allow-oob' declares whether the command supports out-of-band
+(OOB) execution.  It defaults to false.  For example:
 
  { 'command': 'migrate_recover',
    'data': { 'uri': 'str' }, 'allow-oob': true }
 
-To execute a command with out-of-band priority, the client uses key
-"exec-oob" instead of "execute".  Example:
+See qmp-spec.txt for out-of-band execution syntax and semantics.
 
- => { "exec-oob": "migrate-recover",
-      "arguments": { "uri": "tcp:192.168.1.200:12345" } }
- <= { "return": { } }
+Commands supporting out-of-band execution can still be executed
+in-band.
 
-Without it, even the commands that support out-of-band execution will
-still be run in-band.
+When a command is executed in-band, its handler runs in the main
+thread with the BQL held.
 
-Under normal QMP command execution, the following apply to each
-command:
+When a command is executed out-of-band, its handler runs in a
+dedicated monitor I/O thread with the BQL *not* held.
 
-- They are executed in order,
-- They run only in main thread of QEMU,
-- They run with the BQL held.
+An OOB-capable command handler must satisfy the following conditions:
 
-When a command is executed with OOB, the following changes occur:
-
-- They can be completed before a pending in-band command,
-- They run in a dedicated monitor thread,
-- They run with the BQL not held.
-
-OOB command handlers must satisfy the following conditions:
-
-- It terminates quickly,
-- It does not invoke system calls that may block,
+- It terminates quickly.
+- It does not invoke system calls that may block.
 - It does not access guest RAM that may block when userfaultfd is
-  enabled for postcopy live migration,
+  enabled for postcopy live migration.
 - It takes only "fast" locks, i.e. all critical sections protected by
   any lock it takes also satisfy the conditions for OOB command
   handler code.
@@ -686,17 +674,18 @@ The restrictions on locking limit access to shared state.  Such access
 requires synchronization, but OOB commands can't take the BQL or any
 other "slow" lock.
 
-If in doubt, do not implement OOB execution support.
+When in doubt, do not implement OOB execution support.
 
-A command may use the optional 'allow-preconfig' key to permit its execution
-at early runtime configuration stage (preconfig runstate).
-If not specified then a command defaults to 'allow-preconfig': false.
+Key 'allow-preconfig' declares whether the command is available before
+the machine is built.  It defaults to false.  For example:
 
-An example of declaring a command that is enabled during preconfig:
  { 'command': 'qmp_capabilities',
    'data': { '*enable': [ 'QMPCapability' ] },
    'allow-preconfig': true }
 
+QMP is available before the machine is built only when QEMU was
+started with --preconfig.
+
 === Events ===
 
 Usage: { 'event': STRING, '*data': COMPLEX-TYPE-NAME-OR-DICT,
-- 
2.17.1

Re: [Qemu-devel] [PATCH 32/32] qapi: Polish command flags documentation in qapi-code-gen.txt

[Eric Blake]

On 07/02/2018 11:22 AM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>   docs/devel/qapi-code-gen.txt | 61 +++++++++++++++---------------------
>   1 file changed, 25 insertions(+), 36 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH 00/32] qmp: Fixes and cleanups around OOB commands

[Markus Armbruster]

Copying Michael Roth for the qemu-ga parts, belatedly.

Re: [Qemu-devel] [PATCH 00/32] qmp: Fixes and cleanups around OOB commands

[Peter Xu]

On Mon, Jul 02, 2018 at 06:21:46PM +0200, Markus Armbruster wrote:
> We're trying to get the out-of-band execution feature ready.  This
> series fixes a number of issues, and marks a design flaw FIXME.  More
> work is needed.

Hi, Markus,

I roughly went over the series (without going into every details), I
left a few comments but it looks good to me in general (and I noticed
that Eric has already provided r-bs for most of the patches).

I feel like you want this series to settle fast, so I think I'll just
hold the oob fixes until your series is merged, then I can work upon.

Thanks,

-- 
Peter Xu

Re: [Qemu-devel] [PATCH 00/32] qmp: Fixes and cleanups around OOB commands

[Markus Armbruster]

Peter Xu <peterx@redhat.com> writes:

> On Mon, Jul 02, 2018 at 06:21:46PM +0200, Markus Armbruster wrote:
>> We're trying to get the out-of-band execution feature ready.  This
>> series fixes a number of issues, and marks a design flaw FIXME.  More
>> work is needed.
>
> Hi, Markus,
>
> I roughly went over the series (without going into every details), I
> left a few comments but it looks good to me in general (and I noticed
> that Eric has already provided r-bs for most of the patches).
>
> I feel like you want this series to settle fast, so I think I'll just
> hold the oob fixes until your series is merged, then I can work upon.

I'll push the respin to my public repository, so you can more easily
base your work on it while it's being reviewed and merged.

Thanks!

Re: [Qemu-devel] [PATCH 00/32] qmp: Fixes and cleanups around OOB commands

[Marc-André Lureau]

Hi

On Mon, Jul 2, 2018 at 6:21 PM, Markus Armbruster <armbru@redhat.com> wrote:
> We're trying to get the out-of-band execution feature ready.  This
> series fixes a number of issues, and marks a design flaw FIXME.  More
> work is needed.

They are several related worthy cleanups in the qapi-async series.

They will now conflict with your changes though.

Please take a look roughly from 4/38 to 17/38, and let me know if you
want me to rebase it for 3.0.

>
> Markus Armbruster (32):
>   qmp: Say "out-of-band" instead of "Out-Of-Band"
>   monitor: Spell "I/O thread" consistently in comments
>   docs/interop/qmp: Improve OOB documentation
>   qmp: Document COMMAND_DROPPED design flaw
>   qmp: Get rid of x-oob-test command
>   tests/qmp-test: Test in-band command doesn't overtake
>   qmp: Make "id" optional again even in "oob" monitors
>   tests/test-qga: Demonstrate the guest-agent ignores "id"
>   qmp qemu-ga: Revert change that accidentally made qemu-ga accept "id"
>   tests/test-qga: Demonstrate the guest-agent ignores "control"
>   qmp qemu-ga: Fix qemu-ga not to accept "control"
>   qmp: Redo how the client requests out-of-band execution
>   qmp: Revert change to handle_qmp_command tracepoint
>   qmp: Always free QMPRequest with qmp_request_free()
>   qmp: Simplify code around monitor_qmp_dispatch_one()
>   tests/qmp-test: Demonstrate QMP errors jumping the queue
>   qmp: Don't let malformed in-band commands jump the queue
>   qmp: Don't let JSON errors jump the queue
>   monitor: Rename use_io_thr to use_io_thread
>   monitor: Peel off @mon_global wrapper
>   qobject: New qdict_from_jsonf_nofail()
>   qmp: De-duplicate error response building
>   qmp: Use QDict * instead of QObject * for response objects
>   qmp: Replace monitor_json_emitter{,raw}() by
>     qmp_{queue,send}_response()
>   qmp: Replace get_qmp_greeting() by qmp_greeting()
>   qmp: Simplify monitor_qmp_respond()
>   qmp: Add some comments around null responses
>   qmp: Switch timestamp_put() to qdict_from_jsonf_nofail()
>   qobject: Let qobject_from_jsonf() fail instead of abort
>   qmp: Clean up capability negotiation after commit 02130314d8c
>   monitor: Improve some comments
>   qapi: Polish command flags documentation in qapi-code-gen.txt
>
>  docs/devel/qapi-code-gen.txt            |  63 ++-
>  docs/interop/qmp-intro.txt              |  13 +-
>  docs/interop/qmp-spec.txt               |  81 ++--
>  include/qapi/qmp/dispatch.h             |   6 +-
>  include/qapi/qmp/qjson.h                |   2 +
>  monitor.c                               | 507 ++++++++++--------------
>  qapi/misc.json                          |  23 +-
>  qapi/qmp-dispatch.c                     |  97 ++---
>  qapi/qmp-event.c                        |  10 +-
>  qga/main.c                              |  18 +-
>  qmp.c                                   |  16 -
>  qobject/qjson.c                         |  23 +-
>  tests/qapi-schema/qapi-schema-test.json |   2 +-
>  tests/qmp-test.c                        | 103 +++--
>  tests/test-qga.c                        |  34 ++
>  tests/test-qmp-cmds.c                   |  23 +-
>  16 files changed, 499 insertions(+), 522 deletions(-)
>
> --
> 2.17.1
>
>



-- 
Marc-André Lureau

Re: [Qemu-devel] [PATCH 00/32] qmp: Fixes and cleanups around OOB commands

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@gmail.com> writes:

> Hi
>
> On Mon, Jul 2, 2018 at 6:21 PM, Markus Armbruster <armbru@redhat.com> wrote:
>> We're trying to get the out-of-band execution feature ready.  This
>> series fixes a number of issues, and marks a design flaw FIXME.  More
>> work is needed.
>
> They are several related worthy cleanups in the qapi-async series.
>
> They will now conflict with your changes though.

Puh!

I haven't looked closely at the qapi-async series; the qapi-if series
took priority.

> Please take a look roughly from 4/38 to 17/38, and let me know if you
> want me to rebase it for 3.0.

Extracting its preliminary cleanups into a separate series that could be
merged more quickly makes could make sense.  I'll try to have a look.

Re: [Qemu-devel] [PATCH 00/32] qmp: Fixes and cleanups around OOB commands

[Markus Armbruster]

Markus Armbruster <armbru@redhat.com> writes:

> Marc-André Lureau <marcandre.lureau@gmail.com> writes:
>
>> Hi
>>
>> On Mon, Jul 2, 2018 at 6:21 PM, Markus Armbruster <armbru@redhat.com> wrote:
>>> We're trying to get the out-of-band execution feature ready.  This
>>> series fixes a number of issues, and marks a design flaw FIXME.  More
>>> work is needed.
>>
>> They are several related worthy cleanups in the qapi-async series.
>>
>> They will now conflict with your changes though.
>
> Puh!
>
> I haven't looked closely at the qapi-async series; the qapi-if series
> took priority.
>
>> Please take a look roughly from 4/38 to 17/38, and let me know if you
>> want me to rebase it for 3.0.
>
> Extracting its preliminary cleanups into a separate series that could be
> merged more quickly makes could make sense.  I'll try to have a look.

PATCH 02-17 look like they could stand on their own.  Too bad I missed
them, and thus created a whole bunch of conflicts (-EQUEUE2LONG).  Would
be really nice if you could salvage the parts that still make sense on
top of this series.  Even if they should miss 3.0.