qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH 00/36] post-Eric's fixes, QAPI improvements
@ 2015-09-25 14:03 marcandre.lureau
  2015-09-25 14:03 ` [Qemu-devel] [PATCH 01/36] qapi: add comment block before ChardevDummy marcandre.lureau
                   ` (37 more replies)
  0 siblings, 38 replies; 64+ messages in thread
From: marcandre.lureau @ 2015-09-25 14:03 UTC (permalink / raw)
  To: qemu-devel; +Cc: Marc-André Lureau, armbru, mdroth

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Hi,

I have grown a qapi branch during the past 2 months that was
post-poned for review until the introspection and other fixes got
merged or ready.

It could be splitted roughly in 3 parts, but since they depend on each
other, it make sense to send as one:
 1. generate QAPI documentation from schema (1->12)
 2. remove use generated qmp dispatch in monitor (13->20)
 3. add some support for async commands (22->36)

1 and 2 are hopefully not controversial, however 3 was discussed
previously in 'async commands with QMP' thread:

QMP was initially designed with async support, but the implementation
was buggy, since the async context didn't hold the request 'id', and
thus it wasn't really used, so it was removed in 65207c59d a few
months ago.

However, there a valid reasons to want better async support. Many qmp
operations take time, and it's not a good idea to block the monitor
during this time. Also, some operations, such as screendump with QXL,
have to reenter the loop in order to complete. Furthermore, the
current protocol is not purely synchronous, since the client have to
handle unexpected in the middle replies, such as events.

The current status-quo for async commands is to return immediately,
and send loosely related events later. This is quite poor from an API
point of view, and the events are not related to a previous command
(they don't hold the request 'id'). They are also broadcasted, which
may not make sense to other clients for various commands (all the dump
& query for example) and this prevent from adding meaningful client
'id' field (since it may conflict with other clients 'id').

I propose to re-introduce a better 'async' support, where command may
return with the request 'id' only to the caller, when they are
finished, and not block the monitor. This will be opt-out, so existing
code do not have to change.  Furthermore, since reply order is not
guaranteed, clients will have to have the new 'async' capability to
use those async commands. This scheme is not incompatible with having
additional commands to query the command status, or broadcast related
events to all clients.

Marc-André Lureau (36):
  qapi: add comment block before ChardevDummy
  qapi: add missing @
  qapi: add some line spacing to help doc parser
  monitor: use qapi for qmp_capabilities command
  qapi: move examples to json schema
  qapi: move documentation bits in schema files
  qapi: add some headings in docs
  qapi: add qapi2texi script
  qapi: remove qmp-events.txt
  texi2pod: learn quotation, deftp and deftypefn
  build-sys: generate QAPI doc based on json
  build-sys: generate qmp-commands.txt
  build-sys: do not generate qmp-commands-old.h
  monitor: remove usage of generated marshal functions
  monitor: register gen:false commands manually
  qmp: register qapi commands (no middle mode)
  qmp: use qmp_dispatch()
  qapi: remove "middle" mode
  qmp: implement qmp_query_commands without qmp_cmds
  qmp: remove old qmp-commands table
  misc: spelling
  qmp: teach qmp_dispatch() to take a pre-filled QDict
  qmp: use a return callback for the command reply
  qmp: add QmpClient
  qmp: introduce async command type
  qmp: check that async command have an 'id'
  scripts: learn 'async' qapi commands
  scripts: ensure -async commands are declared async
  qapi: take 'id' from request
  tests: change /0.15/* tests to /qmp/*
  tests: add /qmp/dispatch_cmd_async test
  qmp: update qmp-spec about async capability
  monitor: add 'async' capability
  console: graphic_hw_update return true if async
  console: add graphic_hw_update_done()
  console: add screendump-async

 Makefile                                |   43 +-
 Makefile.target                         |    7 +-
 docs/qmp/qmp-events.txt                 |  664 -----
 docs/qmp/qmp-spec.txt                   |   17 +-
 hw/display/qxl-render.c                 |   14 +-
 hw/display/qxl.c                        |    8 +-
 hw/display/qxl.h                        |    2 +-
 include/qapi/qmp/dispatch.h             |   29 +-
 include/ui/console.h                    |    4 +-
 monitor.c                               |  409 +--
 qapi-schema.json                        | 1258 ++++++++-
 qapi/block-core.json                    |  456 +++-
 qapi/block.json                         |   48 +-
 qapi/common.json                        |   37 +-
 qapi/event.json                         |  211 ++
 qapi/introspect.json                    |    2 +-
 qapi/qmp-dispatch.c                     |   92 +-
 qapi/qmp-registry.c                     |   26 +-
 qapi/rocker.json                        |   46 +
 qapi/trace.json                         |   14 +
 qga/main.c                              |   26 +-
 qga/qapi-schema.json                    |    2 +
 qmp-commands.hx                         | 4324 -------------------------------
 scripts/qapi-commands.py                |  156 +-
 scripts/qapi-introspect.py              |    7 +-
 scripts/qapi.py                         |  103 +-
 scripts/qapi2texi.py                    |  293 +++
 scripts/texi2pod.pl                     |   44 +-
 tests/Makefile                          |    1 +
 tests/qapi-schema/async.err             |    0
 tests/qapi-schema/async.exit            |    1 +
 tests/qapi-schema/async.json            |    1 +
 tests/qapi-schema/async.out             |    5 +
 tests/qapi-schema/qapi-schema-test.json |    2 +
 tests/qapi-schema/qapi-schema-test.out  |    5 +
 tests/qapi-schema/test-qapi.py          |    6 +-
 tests/test-qmp-commands.c               |  132 +-
 ui/console.c                            |   77 +-
 vl.c                                    |    1 +
 39 files changed, 3108 insertions(+), 5465 deletions(-)
 delete mode 100644 docs/qmp/qmp-events.txt
 delete mode 100644 qmp-commands.hx
 create mode 100755 scripts/qapi2texi.py
 create mode 100644 tests/qapi-schema/async.err
 create mode 100644 tests/qapi-schema/async.exit
 create mode 100644 tests/qapi-schema/async.json
 create mode 100644 tests/qapi-schema/async.out

-- 
2.4.3

^ permalink raw reply	[flat|nested] 64+ messages in thread

end of thread, other threads:[~2015-10-05 17:14 UTC | newest]

Thread overview: 64+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-09-25 14:03 [Qemu-devel] [PATCH 00/36] post-Eric's fixes, QAPI improvements marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 01/36] qapi: add comment block before ChardevDummy marcandre.lureau
2015-09-25 15:03   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 02/36] qapi: add missing @ marcandre.lureau
2015-09-25 15:03   ` Eric Blake
2015-10-03 16:57   ` Michael Tokarev
2015-09-25 14:03 ` [Qemu-devel] [PATCH 03/36] qapi: add some line spacing to help doc parser marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 04/36] monitor: use qapi for qmp_capabilities command marcandre.lureau
2015-09-25 15:09   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 05/36] qapi: move examples to json schema marcandre.lureau
2015-09-25 15:16   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 06/36] qapi: move documentation bits in schema files marcandre.lureau
2015-09-25 15:23   ` Eric Blake
2015-09-25 15:29     ` Marc-André Lureau
2015-09-25 15:35       ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 07/36] qapi: add some headings in docs marcandre.lureau
2015-09-25 15:26   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 08/36] qapi: add qapi2texi script marcandre.lureau
2015-09-25 15:34   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 09/36] qapi: remove qmp-events.txt marcandre.lureau
2015-09-25 15:40   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 10/36] texi2pod: learn quotation, deftp and deftypefn marcandre.lureau
2015-09-25 15:44   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 11/36] build-sys: generate QAPI doc based on json marcandre.lureau
2015-09-25 15:49   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 12/36] build-sys: generate qmp-commands.txt marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 13/36] build-sys: do not generate qmp-commands-old.h marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 14/36] monitor: remove usage of generated marshal functions marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 15/36] monitor: register gen:false commands manually marcandre.lureau
2015-09-25 15:56   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 16/36] qmp: register qapi commands (no middle mode) marcandre.lureau
2015-09-25 16:01   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 17/36] qmp: use qmp_dispatch() marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 18/36] qapi: remove "middle" mode marcandre.lureau
2015-09-25 16:03   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 19/36] qmp: implement qmp_query_commands without qmp_cmds marcandre.lureau
2015-09-25 16:06   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 20/36] qmp: remove old qmp-commands table marcandre.lureau
2015-09-25 16:08   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 21/36] misc: spelling marcandre.lureau
2015-09-25 16:08   ` Eric Blake
2015-10-03 16:55     ` Michael Tokarev
2015-10-05  5:09       ` Markus Armbruster
2015-10-05  7:32         ` Michael Tokarev
2015-10-05 17:05           ` Markus Armbruster
2015-09-25 14:03 ` [Qemu-devel] [PATCH 22/36] qmp: teach qmp_dispatch() to take a pre-filled QDict marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 23/36] qmp: use a return callback for the command reply marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 24/36] qmp: add QmpClient marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 25/36] qmp: introduce async command type marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 26/36] qmp: check that async command have an 'id' marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 27/36] scripts: learn 'async' qapi commands marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 28/36] scripts: ensure -async commands are declared async marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 29/36] qapi: take 'id' from request marcandre.lureau
2015-09-25 14:03 ` [Qemu-devel] [PATCH 30/36] tests: change /0.15/* tests to /qmp/* marcandre.lureau
2015-09-25 16:10   ` Eric Blake
2015-09-25 14:03 ` [Qemu-devel] [PATCH 31/36] tests: add /qmp/dispatch_cmd_async test marcandre.lureau
2015-09-25 14:04 ` [Qemu-devel] [PATCH 32/36] qmp: update qmp-spec about async capability marcandre.lureau
2015-09-25 14:04 ` [Qemu-devel] [PATCH 33/36] monitor: add 'async' capability marcandre.lureau
2015-09-25 14:04 ` [Qemu-devel] [PATCH 34/36] console: graphic_hw_update return true if async marcandre.lureau
2015-09-25 14:04 ` [Qemu-devel] [PATCH 35/36] console: add graphic_hw_update_done() marcandre.lureau
2015-09-25 14:04 ` [Qemu-devel] [PATCH 36/36] console: add screendump-async marcandre.lureau
2015-09-25 15:58 ` [Qemu-devel] [PATCH 00/36] post-Eric's fixes, QAPI improvements Eric Blake
2015-09-25 16:01   ` Marc-André Lureau
2015-09-25 16:21 ` Markus Armbruster

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).