[Qemu-devel] [PATCH v12 00/18] qapi visitor cleanups (post-introspection cleanups subset E)

[Qemu-devel] [PATCH v12 00/18] qapi visitor cleanups (post-introspection cleanups subset E)

[Eric Blake]

Requires these patches first:
https://lists.gnu.org/archive/html/qemu-devel/2016-02/msg06079.html

v12: This series (finally) finishes the review comments against the
tail end of my v9 subset E series:
https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg03504.html

v10 and v11 of the series dealt with the first half, and are now
mostly applied (or part of the pre-requisite series).

Also available as a tag at this location:
git fetch git://repo.or.cz/qemu/ericb.git qapi-cleanupv12e

and will soon be part of my branch with the rest of the v5 series, at:
http://repo.or.cz/qemu/ericb.git/shortlog/refs/heads/qapi

There are a number of new patches listed here, the biggest reason
is splitting several large patches into smaller chunks.

001/18:[down] 'qapi-visit: Add visitor.type classification'
002/18:[down] 'qapi: Guarantee NULL obj on input visitor callback error'
003/18:[down] 'qmp: Drop dead command->type'
004/18:[down] 'qmp-input: Clean up stack handling'
005/18:[down] 'qmp-input: Don't consume input when checking has_member'
006/18:[down] 'qmp-input: Refactor when list is advanced'
007/18:[0655] [FC] 'qapi: Document visitor interfaces, add assertions'
008/18:[down] 'tests: Add check-qnull'
009/18:[0024] [FC] 'qapi: Add visit_type_null() visitor'
010/18:[down] 'qmp: Support explicit null during visits'
011/18:[----] [--] 'spapr_drc: Expose 'null' in qom-get when there is no fdt'
012/18:[0047] [FC] 'qmp: Tighten output visitor rules'
013/18:[0088] [FC] 'qapi: Split visit_end_struct() into pieces'
014/18:[down] 'qapi-commands: Wrap argument visit in visit_start_struct'
015/18:[down] 'qom: Wrap prop visit in visit_start_struct'
016/18:[down] 'qmp-input: Require struct push to visit members of top dict'
017/18:[0273] [FC] 'qapi: Simplify semantics of visit_next_list()'
018/18:[0255] [FC] 'qapi: Change visit_type_FOO() to no longer return partial objects'

Eric Blake (18):
  qapi-visit: Add visitor.type classification
  qapi: Guarantee NULL obj on input visitor callback error
  qmp: Drop dead command->type
  qmp-input: Clean up stack handling
  qmp-input: Don't consume input when checking has_member
  qmp-input: Refactor when list is advanced
  qapi: Document visitor interfaces, add assertions
  tests: Add check-qnull
  qapi: Add visit_type_null() visitor
  qmp: Support explicit null during visits
  spapr_drc: Expose 'null' in qom-get when there is no fdt
  qmp: Tighten output visitor rules
  qapi: Split visit_end_struct() into pieces
  qapi-commands: Wrap argument visit in visit_start_struct
  qom: Wrap prop visit in visit_start_struct
  qmp-input: Require struct push to visit members of top dict
  qapi: Simplify semantics of visit_next_list()
  qapi: Change visit_type_FOO() to no longer return partial objects

 include/qapi/visitor.h               | 482 +++++++++++++++++++++++++++++++++--
 include/qapi/visitor-impl.h          |  80 ++++--
 scripts/qapi-commands.py             |  16 +-
 scripts/qapi-event.py                |   3 +-
 scripts/qapi-visit.py                |  56 ++--
 include/qapi/dealloc-visitor.h       |   4 +
 include/qapi/opts-visitor.h          |   4 +
 include/qapi/qmp-output-visitor.h    |   1 +
 include/qapi/qmp/dispatch.h          |   6 -
 include/qapi/string-input-visitor.h  |   5 +
 include/qapi/string-output-visitor.h |   5 +
 qapi/qapi-visit-core.c               | 112 ++++++--
 hw/ppc/spapr_drc.c                   |  11 +-
 hw/virtio/virtio-balloon.c           |  15 +-
 qapi/opts-visitor.c                  |  65 ++---
 qapi/qapi-dealloc-visitor.c          |  43 +---
 qapi/qmp-dispatch.c                  |  18 +-
 qapi/qmp-input-visitor.c             | 168 +++++++-----
 qapi/qmp-output-visitor.c            |  72 +++---
 qapi/qmp-registry.c                  |   1 -
 qapi/string-input-visitor.c          |  39 +--
 qapi/string-output-visitor.c         |  43 ++--
 qom/object.c                         |   5 +-
 qom/object_interfaces.c              |  29 ++-
 tests/check-qnull.c                  |  68 +++++
 tests/test-qmp-commands.c            |  13 +-
 tests/test-qmp-input-strict.c        |  19 +-
 tests/test-qmp-input-visitor.c       |  27 +-
 tests/test-qmp-output-visitor.c      |  17 +-
 docs/qapi-code-gen.txt               |  33 ++-
 tests/.gitignore                     |   1 +
 tests/Makefile                       |   6 +-
 32 files changed, 1079 insertions(+), 388 deletions(-)
 create mode 100644 tests/check-qnull.c

-- 
2.5.0

[Qemu-devel] [PATCH v12 01/18] qapi-visit: Add visitor.type classification

[Eric Blake]

We have three classes of QAPI visitors: input, output, and dealloc.
Currently, all implementations of these visitors have one thing in
common based on their visitor type: the implementation used for the
visit_type_enum() callback.  But since we plan to add more such
common behavior, in relation to documenting and further refining
the semantics, it makes more sense to have the visitor
implementations advertise which class they belong to, so the common
qapi-visit-core code can use that information in multiple places.

For this patch, knowing the class of a visitor implementation lets
us make input_type_enum() and output_type_enum() become static
functions, by replacing the callback function Visitor.type_enum()
with the simpler enum field Visitor.type.  Share a common assertion
in qapi-visit-core as part of the refactoring.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: new patch
---
 include/qapi/visitor-impl.h  | 21 ++++++++++++---------
 qapi/qapi-visit-core.c       | 28 +++++++++++++++-------------
 qapi/opts-visitor.c          | 12 ++----------
 qapi/qapi-dealloc-visitor.c  |  7 +------
 qapi/qmp-input-visitor.c     |  2 +-
 qapi/qmp-output-visitor.c    |  2 +-
 qapi/string-input-visitor.c  |  2 +-
 qapi/string-output-visitor.c |  2 +-
 8 files changed, 34 insertions(+), 42 deletions(-)

diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 2bd8f29..228a2a6 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -14,6 +14,15 @@

 #include "qapi/visitor.h"

+/* There are three classes of visitors; setting the class determines
+ * how QAPI enums are visited, as well as what additional restrictions
+ * can be asserted.  */
+typedef enum VisitorType {
+    VISITOR_INPUT,
+    VISITOR_OUTPUT,
+    VISITOR_DEALLOC,
+} VisitorType;
+
 struct Visitor
 {
     /* Must be set */
@@ -36,10 +45,6 @@ struct Visitor
     void (*end_alternate)(Visitor *v);

     /* Must be set. */
-    void (*type_enum)(Visitor *v, const char *name, int *obj,
-                      const char *const strings[], Error **errp);
-
-    /* Must be set. */
     void (*type_int64)(Visitor *v, const char *name, int64_t *obj,
                        Error **errp);
     /* Must be set. */
@@ -58,11 +63,9 @@ struct Visitor

     /* May be NULL; most useful for input visitors. */
     void (*optional)(Visitor *v, const char *name, bool *present);
+
+    /* Must be set.  */
+    VisitorType type;
 };

-void input_type_enum(Visitor *v, const char *name, int *obj,
-                     const char *const strings[], Error **errp);
-void output_type_enum(Visitor *v, const char *name, int *obj,
-                      const char *const strings[], Error **errp);
-
 #endif
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 856606b..a08d073 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -71,12 +71,6 @@ bool visit_optional(Visitor *v, const char *name, bool *present)
     return *present;
 }

-void visit_type_enum(Visitor *v, const char *name, int *obj,
-                     const char *const strings[], Error **errp)
-{
-    v->type_enum(v, name, obj, strings, errp);
-}
-
 void visit_type_int(Visitor *v, const char *name, int64_t *obj, Error **errp)
 {
     v->type_int64(v, name, obj, errp);
@@ -207,14 +201,13 @@ void visit_type_any(Visitor *v, const char *name, QObject **obj, Error **errp)
     v->type_any(v, name, obj, errp);
 }

-void output_type_enum(Visitor *v, const char *name, int *obj,
-                      const char *const strings[], Error **errp)
+static void output_type_enum(Visitor *v, const char *name, int *obj,
+                             const char *const strings[], Error **errp)
 {
     int i = 0;
     int value = *obj;
     char *enum_str;

-    assert(strings);
     while (strings[i++] != NULL);
     if (value < 0 || value >= i - 1) {
         error_setg(errp, QERR_INVALID_PARAMETER, name ? name : "null");
@@ -225,15 +218,13 @@ void output_type_enum(Visitor *v, const char *name, int *obj,
     visit_type_str(v, name, &enum_str, errp);
 }

-void input_type_enum(Visitor *v, const char *name, int *obj,
-                     const char *const strings[], Error **errp)
+static void input_type_enum(Visitor *v, const char *name, int *obj,
+                            const char *const strings[], Error **errp)
 {
     Error *local_err = NULL;
     int64_t value = 0;
     char *enum_str;

-    assert(strings);
-
     visit_type_str(v, name, &enum_str, &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
@@ -256,3 +247,14 @@ void input_type_enum(Visitor *v, const char *name, int *obj,
     g_free(enum_str);
     *obj = value;
 }
+
+void visit_type_enum(Visitor *v, const char *name, int *obj,
+                     const char *const strings[], Error **errp)
+{
+    assert(strings);
+    if (v->type == VISITOR_INPUT) {
+        input_type_enum(v, name, obj, strings, errp);
+    } else if (v->type == VISITOR_OUTPUT) {
+        output_type_enum(v, name, obj, strings, errp);
+    }
+}
diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
index 73e4ace..f9a2346 100644
--- a/qapi/opts-visitor.c
+++ b/qapi/opts-visitor.c
@@ -506,6 +506,8 @@ opts_visitor_new(const QemuOpts *opts)

     ov = g_malloc0(sizeof *ov);

+    ov->visitor.type = VISITOR_INPUT;
+
     ov->visitor.start_struct = &opts_start_struct;
     ov->visitor.end_struct   = &opts_end_struct;

@@ -513,16 +515,6 @@ opts_visitor_new(const QemuOpts *opts)
     ov->visitor.next_list  = &opts_next_list;
     ov->visitor.end_list   = &opts_end_list;

-    /* input_type_enum() covers both "normal" enums and union discriminators.
-     * The union discriminator field is always generated as "type"; it should
-     * match the "type" QemuOpt child of any QemuOpts.
-     *
-     * input_type_enum() will remove the looked-up key from the
-     * "unprocessed_opts" hash even if the lookup fails, because the removal is
-     * done earlier in opts_type_str(). This should be harmless.
-     */
-    ov->visitor.type_enum = &input_type_enum;
-
     ov->visitor.type_int64  = &opts_type_int64;
     ov->visitor.type_uint64 = &opts_type_uint64;
     ov->visitor.type_size   = &opts_type_size;
diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c
index 6922179..c19a459 100644
--- a/qapi/qapi-dealloc-visitor.c
+++ b/qapi/qapi-dealloc-visitor.c
@@ -163,11 +163,6 @@ static void qapi_dealloc_type_anything(Visitor *v, const char *name,
     }
 }

-static void qapi_dealloc_type_enum(Visitor *v, const char *name, int *obj,
-                                   const char * const strings[], Error **errp)
-{
-}
-
 Visitor *qapi_dealloc_get_visitor(QapiDeallocVisitor *v)
 {
     return &v->visitor;
@@ -184,6 +179,7 @@ QapiDeallocVisitor *qapi_dealloc_visitor_new(void)

     v = g_malloc0(sizeof(*v));

+    v->visitor.type = VISITOR_DEALLOC;
     v->visitor.start_struct = qapi_dealloc_start_struct;
     v->visitor.end_struct = qapi_dealloc_end_struct;
     v->visitor.start_alternate = qapi_dealloc_start_alternate;
@@ -191,7 +187,6 @@ QapiDeallocVisitor *qapi_dealloc_visitor_new(void)
     v->visitor.start_list = qapi_dealloc_start_list;
     v->visitor.next_list = qapi_dealloc_next_list;
     v->visitor.end_list = qapi_dealloc_end_list;
-    v->visitor.type_enum = qapi_dealloc_type_enum;
     v->visitor.type_int64 = qapi_dealloc_type_int64;
     v->visitor.type_uint64 = qapi_dealloc_type_uint64;
     v->visitor.type_bool = qapi_dealloc_type_bool;
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index e659832..03dcb65 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -338,13 +338,13 @@ QmpInputVisitor *qmp_input_visitor_new(QObject *obj)

     v = g_malloc0(sizeof(*v));

+    v->visitor.type = VISITOR_INPUT;
     v->visitor.start_struct = qmp_input_start_struct;
     v->visitor.end_struct = qmp_input_end_struct;
     v->visitor.start_list = qmp_input_start_list;
     v->visitor.next_list = qmp_input_next_list;
     v->visitor.end_list = qmp_input_end_list;
     v->visitor.start_alternate = qmp_input_start_alternate;
-    v->visitor.type_enum = input_type_enum;
     v->visitor.type_int64 = qmp_input_type_int64;
     v->visitor.type_uint64 = qmp_input_type_uint64;
     v->visitor.type_bool = qmp_input_type_bool;
diff --git a/qapi/qmp-output-visitor.c b/qapi/qmp-output-visitor.c
index d44c676..1f2a7ba 100644
--- a/qapi/qmp-output-visitor.c
+++ b/qapi/qmp-output-visitor.c
@@ -234,12 +234,12 @@ QmpOutputVisitor *qmp_output_visitor_new(void)

     v = g_malloc0(sizeof(*v));

+    v->visitor.type = VISITOR_OUTPUT;
     v->visitor.start_struct = qmp_output_start_struct;
     v->visitor.end_struct = qmp_output_end_struct;
     v->visitor.start_list = qmp_output_start_list;
     v->visitor.next_list = qmp_output_next_list;
     v->visitor.end_list = qmp_output_end_list;
-    v->visitor.type_enum = output_type_enum;
     v->visitor.type_int64 = qmp_output_type_int64;
     v->visitor.type_uint64 = qmp_output_type_uint64;
     v->visitor.type_bool = qmp_output_type_bool;
diff --git a/qapi/string-input-visitor.c b/qapi/string-input-visitor.c
index 59eb5dc..d591e67 100644
--- a/qapi/string-input-visitor.c
+++ b/qapi/string-input-visitor.c
@@ -347,7 +347,7 @@ StringInputVisitor *string_input_visitor_new(const char *str)

     v = g_malloc0(sizeof(*v));

-    v->visitor.type_enum = input_type_enum;
+    v->visitor.type = VISITOR_INPUT;
     v->visitor.type_int64 = parse_type_int64;
     v->visitor.type_uint64 = parse_type_uint64;
     v->visitor.type_size = parse_type_size;
diff --git a/qapi/string-output-visitor.c b/qapi/string-output-visitor.c
index c2e5c5b..0d44d7e 100644
--- a/qapi/string-output-visitor.c
+++ b/qapi/string-output-visitor.c
@@ -351,7 +351,7 @@ StringOutputVisitor *string_output_visitor_new(bool human)

     v->string = g_string_new(NULL);
     v->human = human;
-    v->visitor.type_enum = output_type_enum;
+    v->visitor.type = VISITOR_OUTPUT;
     v->visitor.type_int64 = print_type_int64;
     v->visitor.type_uint64 = print_type_uint64;
     v->visitor.type_size = print_type_size;
-- 
2.5.0

[Qemu-devel] [PATCH v12 02/18] qapi: Guarantee NULL obj on input visitor callback error

[Eric Blake]

Our existing input visitors were not very consistent on errors
in a function taking 'TYPE **obj'. While all of them set '*obj'
to allocated storage on success, it was not obvious whether
'*obj' was guaranteed safe on failure, or whether it was left
uninitialized.  But a future patch wants to guarantee that
visit_type_FOO() does not leak a partially-constructed obj back
to the caller; it is easier to implement this if we can reliably
state that '*obj' is assigned on exit, even on failures.

The opts-visitor start_struct() doesn't set an error, but it
also was doing a weird check for 0 size; all callers pass in
non-zero size if obj is non-NULL.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: new patch
---
 qapi/opts-visitor.c         | 3 ++-
 qapi/qmp-input-visitor.c    | 4 ++++
 qapi/string-input-visitor.c | 1 +
 3 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
index f9a2346..fd90608 100644
--- a/qapi/opts-visitor.c
+++ b/qapi/opts-visitor.c
@@ -132,7 +132,7 @@ opts_start_struct(Visitor *v, const char *name, void **obj,
     const QemuOpt *opt;

     if (obj) {
-        *obj = g_malloc0(size > 0 ? size : 1);
+        *obj = g_malloc0(size);
     }
     if (ov->depth++ > 0) {
         return;
@@ -313,6 +313,7 @@ opts_type_str(Visitor *v, const char *name, char **obj, Error **errp)

     opt = lookup_scalar(ov, name, errp);
     if (!opt) {
+        *obj = NULL;
         return;
     }
     *obj = g_strdup(opt->str ? opt->str : "");
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index 03dcb65..550aed6 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -119,6 +119,9 @@ static void qmp_input_start_struct(Visitor *v, const char *name, void **obj,
     QObject *qobj = qmp_input_get_object(qiv, name, true);
     Error *err = NULL;

+    if (obj) {
+        *obj = NULL;
+    }
     if (!qobj || qobject_type(qobj) != QTYPE_QDICT) {
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
                    "QDict");
@@ -266,6 +269,7 @@ static void qmp_input_type_str(Visitor *v, const char *name, char **obj,
     QString *qstr = qobject_to_qstring(qmp_input_get_object(qiv, name, true));

     if (!qstr) {
+        *obj = NULL;
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
                    "string");
         return;
diff --git a/qapi/string-input-visitor.c b/qapi/string-input-visitor.c
index d591e67..4087bf9 100644
--- a/qapi/string-input-visitor.c
+++ b/qapi/string-input-visitor.c
@@ -292,6 +292,7 @@ static void parse_type_str(Visitor *v, const char *name, char **obj,
     if (siv->string) {
         *obj = g_strdup(siv->string);
     } else {
+        *obj = NULL;
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
                    "string");
     }
-- 
2.5.0

[Qemu-devel] [PATCH v12 03/18] qmp: Drop dead command->type

[Eric Blake]

Ever since QMP was first added back in commit 43c20a43, we have
never had any QmpCommandType other than QCT_NORMAL.  It's
pointless to carry around the cruft.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: new patch
---
 include/qapi/qmp/dispatch.h |  6 ------
 qapi/qmp-dispatch.c         | 18 +++++++-----------
 qapi/qmp-registry.c         |  1 -
 3 files changed, 7 insertions(+), 18 deletions(-)

diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
index 4955209..5609946 100644
--- a/include/qapi/qmp/dispatch.h
+++ b/include/qapi/qmp/dispatch.h
@@ -19,11 +19,6 @@

 typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);

-typedef enum QmpCommandType
-{
-    QCT_NORMAL,
-} QmpCommandType;
-
 typedef enum QmpCommandOptions
 {
     QCO_NO_OPTIONS = 0x0,
@@ -33,7 +28,6 @@ typedef enum QmpCommandOptions
 typedef struct QmpCommand
 {
     const char *name;
-    QmpCommandType type;
     QmpCommandFunc *fn;
     QmpCommandOptions options;
     QTAILQ_ENTRY(QmpCommand) node;
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 8f27c34..072e092 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -93,17 +93,13 @@ static QObject *do_qmp_dispatch(QObject *request, Error **errp)
         QINCREF(args);
     }

-    switch (cmd->type) {
-    case QCT_NORMAL:
-        cmd->fn(args, &ret, &local_err);
-        if (local_err) {
-            error_propagate(errp, local_err);
-        } else if (cmd->options & QCO_NO_SUCCESS_RESP) {
-            g_assert(!ret);
-        } else if (!ret) {
-            ret = QOBJECT(qdict_new());
-        }
-        break;
+    cmd->fn(args, &ret, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+    } else if (cmd->options & QCO_NO_SUCCESS_RESP) {
+        g_assert(!ret);
+    } else if (!ret) {
+        ret = QOBJECT(qdict_new());
     }

     QDECREF(args);
diff --git a/qapi/qmp-registry.c b/qapi/qmp-registry.c
index 4ebfbcc..4332a68 100644
--- a/qapi/qmp-registry.c
+++ b/qapi/qmp-registry.c
@@ -25,7 +25,6 @@ void qmp_register_command(const char *name, QmpCommandFunc *fn,
     QmpCommand *cmd = g_malloc0(sizeof(*cmd));

     cmd->name = name;
-    cmd->type = QCT_NORMAL;
     cmd->fn = fn;
     cmd->enabled = true;
     cmd->options = options;
-- 
2.5.0

[Qemu-devel] [PATCH v12 04/18] qmp-input: Clean up stack handling

[Eric Blake]

Management of the top of stack was a bit verbose; creating a
temporary variable and adding some comments makes the existing
code more legible before the next few patches improve things.
No semantic changes other than asserting that we are always
visiting a QObject, and not a NULL value.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: new patch
---
 qapi/qmp-input-visitor.c | 52 ++++++++++++++++++++++++++++++++++--------------
 1 file changed, 37 insertions(+), 15 deletions(-)

diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index 550aed6..7428d15 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -24,16 +24,26 @@

 typedef struct StackObject
 {
-    QObject *obj;
+    QObject *obj; /* Object being visited */
+
+    /* If obj is list: NULL if list is at head, otherwise tail of list
+     * still needing visits */
     const QListEntry *entry;
-    GHashTable *h;
+
+    GHashTable *h; /* If obj is dict: remaining keys needing visits */
 } StackObject;

 struct QmpInputVisitor
 {
     Visitor visitor;
+
+    /* Stack of objects being visited.  stack[0] is root of visit,
+     * stack[1] and below correspond to visit_start_struct (nested
+     * QDict) and visit_start_list (nested QList).  */
     StackObject stack[QIV_STACK_SIZE];
     int nb_stack;
+
+    /* True to track whether all keys in QDict have been parsed.  */
     bool strict;
 };

@@ -46,19 +56,29 @@ static QObject *qmp_input_get_object(QmpInputVisitor *qiv,
                                      const char *name,
                                      bool consume)
 {
-    QObject *qobj = qiv->stack[qiv->nb_stack - 1].obj;
+    StackObject *tos = &qiv->stack[qiv->nb_stack - 1];
+    QObject *qobj = tos->obj;

-    if (qobj) {
-        if (name && qobject_type(qobj) == QTYPE_QDICT) {
-            if (qiv->stack[qiv->nb_stack - 1].h && consume) {
-                g_hash_table_remove(qiv->stack[qiv->nb_stack - 1].h, name);
-            }
-            return qdict_get(qobject_to_qdict(qobj), name);
-        } else if (qiv->stack[qiv->nb_stack - 1].entry) {
-            return qlist_entry_obj(qiv->stack[qiv->nb_stack - 1].entry);
+    assert(qobj);
+
+    /* If we have a name, and we're in a dictionary, then return that
+     * value. */
+    if (name && qobject_type(qobj) == QTYPE_QDICT) {
+        if (tos->h && consume) {
+            g_hash_table_remove(tos->h, name);
         }
+        return qdict_get(qobject_to_qdict(qobj), name);
     }

+    /* If we are in the middle of a list, then return the next element
+     * of the list.  */
+    if (tos->entry) {
+        assert(qobject_type(qobj) == QTYPE_QLIST);
+        return qlist_entry_obj(tos->entry);
+    }
+
+    /* Otherwise, we are at the root of the visit or the start of a
+     * list, and return the object as-is.  */
     return qobj;
 }

@@ -71,20 +91,22 @@ static void qdict_add_key(const char *key, QObject *obj, void *opaque)
 static void qmp_input_push(QmpInputVisitor *qiv, QObject *obj, Error **errp)
 {
     GHashTable *h;
+    StackObject *tos = &qiv->stack[qiv->nb_stack];

+    assert(obj);
     if (qiv->nb_stack >= QIV_STACK_SIZE) {
         error_setg(errp, "An internal buffer overran");
         return;
     }

-    qiv->stack[qiv->nb_stack].obj = obj;
-    qiv->stack[qiv->nb_stack].entry = NULL;
-    qiv->stack[qiv->nb_stack].h = NULL;
+    tos->obj = obj;
+    tos->entry = NULL;
+    tos->h = NULL;

     if (qiv->strict && qobject_type(obj) == QTYPE_QDICT) {
         h = g_hash_table_new(g_str_hash, g_str_equal);
         qdict_iter(qobject_to_qdict(obj), qdict_add_key, h);
-        qiv->stack[qiv->nb_stack].h = h;
+        tos->h = h;
     }

     qiv->nb_stack++;
-- 
2.5.0

[Qemu-devel] [PATCH v12 05/18] qmp-input: Don't consume input when checking has_member

[Eric Blake]

Commit e8316d7 mistakenly passed consume=true when checking if
an optional member was present, but the mistake was silently
ignored since the code happily let us extract a member more than
once.  Tighten up the input visitor to ensure that a member is
consumed exactly once.  To keep the testsuite happy in the case
of incomplete input, we have to check whether a member exists
in the dictionary before trying to remove it.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: new patch
---
 qapi/qmp-input-visitor.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index 7428d15..4bb7b21 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -64,10 +64,12 @@ static QObject *qmp_input_get_object(QmpInputVisitor *qiv,
     /* If we have a name, and we're in a dictionary, then return that
      * value. */
     if (name && qobject_type(qobj) == QTYPE_QDICT) {
-        if (tos->h && consume) {
-            g_hash_table_remove(tos->h, name);
+        qobj = qdict_get(qobject_to_qdict(qobj), name);
+        if (tos->h && consume && qobj) {
+            bool removed = g_hash_table_remove(tos->h, name);
+            assert(removed);
         }
-        return qdict_get(qobject_to_qdict(qobj), name);
+        return qobj;
     }

     /* If we are in the middle of a list, then return the next element
@@ -337,7 +339,7 @@ static void qmp_input_type_any(Visitor *v, const char *name, QObject **obj,
 static void qmp_input_optional(Visitor *v, const char *name, bool *present)
 {
     QmpInputVisitor *qiv = to_qiv(v);
-    QObject *qobj = qmp_input_get_object(qiv, name, true);
+    QObject *qobj = qmp_input_get_object(qiv, name, false);

     if (!qobj) {
         *present = false;
-- 
2.5.0

[Qemu-devel] [PATCH v12 06/18] qmp-input: Refactor when list is advanced

[Eric Blake]

Refactor the code to advance the QListEntry pointer at the point
where visit_type_FOO() is called, rather than visit_next_list().
This will allow a future patch to move the visit of the list head
into visit_start_list(), and get rid of the 'first' flag.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: new patch
---
 qapi/qmp-input-visitor.c | 40 +++++++++++++++++++++-------------------
 1 file changed, 21 insertions(+), 19 deletions(-)

diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index 4bb7b21..ed54d4c 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -26,9 +26,10 @@ typedef struct StackObject
 {
     QObject *obj; /* Object being visited */

-    /* If obj is list: NULL if list is at head, otherwise tail of list
-     * still needing visits */
+    /* If obj is list: tail of list still needing visits */
     const QListEntry *entry;
+    /* If obj is list: true if head is not visited yet */
+    bool first;

     GHashTable *h; /* If obj is dict: remaining keys needing visits */
 } StackObject;
@@ -76,7 +77,12 @@ static QObject *qmp_input_get_object(QmpInputVisitor *qiv,
      * of the list.  */
     if (tos->entry) {
         assert(qobject_type(qobj) == QTYPE_QLIST);
-        return qlist_entry_obj(tos->entry);
+        assert(!tos->first);
+        qobj = qlist_entry_obj(tos->entry);
+        if (consume) {
+            tos->entry = qlist_next(tos->entry);
+        }
+        return qobj;
     }

     /* Otherwise, we are at the root of the visit or the start of a
@@ -90,7 +96,8 @@ static void qdict_add_key(const char *key, QObject *obj, void *opaque)
     g_hash_table_insert(h, (gpointer) key, NULL);
 }

-static void qmp_input_push(QmpInputVisitor *qiv, QObject *obj, Error **errp)
+static void qmp_input_push(QmpInputVisitor *qiv, QObject *obj,
+                           const QListEntry *entry, Error **errp)
 {
     GHashTable *h;
     StackObject *tos = &qiv->stack[qiv->nb_stack];
@@ -102,7 +109,8 @@ static void qmp_input_push(QmpInputVisitor *qiv, QObject *obj, Error **errp)
     }

     tos->obj = obj;
-    tos->entry = NULL;
+    tos->entry = entry;
+    tos->first = true;
     tos->h = NULL;

     if (qiv->strict && qobject_type(obj) == QTYPE_QDICT) {
@@ -152,7 +160,7 @@ static void qmp_input_start_struct(Visitor *v, const char *name, void **obj,
         return;
     }

-    qmp_input_push(qiv, qobj, &err);
+    qmp_input_push(qiv, qobj, NULL, &err);
     if (err) {
         error_propagate(errp, err);
         return;
@@ -174,6 +182,7 @@ static void qmp_input_start_list(Visitor *v, const char *name, Error **errp)
 {
     QmpInputVisitor *qiv = to_qiv(v);
     QObject *qobj = qmp_input_get_object(qiv, name, true);
+    const QListEntry *entry;

     if (!qobj || qobject_type(qobj) != QTYPE_QLIST) {
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
@@ -181,7 +190,8 @@ static void qmp_input_start_list(Visitor *v, const char *name, Error **errp)
         return;
     }

-    qmp_input_push(qiv, qobj, errp);
+    entry = qlist_first(qobject_to_qlist(qobj));
+    qmp_input_push(qiv, qobj, entry, errp);
 }

 static GenericList *qmp_input_next_list(Visitor *v, GenericList **list,
@@ -190,23 +200,15 @@ static GenericList *qmp_input_next_list(Visitor *v, GenericList **list,
     QmpInputVisitor *qiv = to_qiv(v);
     GenericList *entry;
     StackObject *so = &qiv->stack[qiv->nb_stack - 1];
-    bool first;

-    if (so->entry == NULL) {
-        so->entry = qlist_first(qobject_to_qlist(so->obj));
-        first = true;
-    } else {
-        so->entry = qlist_next(so->entry);
-        first = false;
-    }
-
-    if (so->entry == NULL) {
+    if (!so->entry) {
         return NULL;
     }

     entry = g_malloc0(size);
-    if (first) {
+    if (so->first) {
         *list = entry;
+        so->first = false;
     } else {
         (*list)->next = entry;
     }
@@ -381,7 +383,7 @@ QmpInputVisitor *qmp_input_visitor_new(QObject *obj)
     v->visitor.type_any = qmp_input_type_any;
     v->visitor.optional = qmp_input_optional;

-    qmp_input_push(v, obj, NULL);
+    qmp_input_push(v, obj, NULL, NULL);
     qobject_incref(obj);

     return v;
-- 
2.5.0

[Qemu-devel] [PATCH v12 07/18] qapi: Document visitor interfaces, add assertions

[Eric Blake]

The visitor interface for mapping between QObject/QemuOpts/string
and QAPI is scandalously under-documented, making changes to visitor
core, individual visitors, and users of visitors difficult to
coordinate.  Among other questions: when is it safe to pass NULL,
vs. when a string must be provided; which visitors implement which
callbacks; the difference between concrete and virtual visits.

Correct this by retrofitting proper contracts, and document where some
of the interface warts remain (for example, we may want to modify
visit_end_* to require the same 'obj' as the visit_start counterpart,
so the dealloc visitor can be simplified).  Later patches in this
series will tackle some, but not all, of these warts.

Add assertions to (partially) enforce the contract.  Some of these
were only made possible by recent cleanup commits.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: major rework based on Markus' comments, drop R-b
[no v10, v11]
v9: no change
v8: rebase to 'name' motion
v7: retitle; more wording changes, add asserts to enforce the
wording, place later in series to rebase on fixes that would
otherwise trip the new assertions
v6: mention that input visitors blindly assign over *obj; wording
improvements
---
 include/qapi/visitor.h               | 428 +++++++++++++++++++++++++++++++++--
 include/qapi/visitor-impl.h          |  42 +++-
 include/qapi/dealloc-visitor.h       |   4 +
 include/qapi/opts-visitor.h          |   4 +
 include/qapi/qmp-input-visitor.h     |   8 +
 include/qapi/string-input-visitor.h  |   4 +
 include/qapi/string-output-visitor.h |   4 +
 qapi/qapi-visit-core.c               |  19 +-
 8 files changed, 491 insertions(+), 22 deletions(-)

diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 1000da2..33ae28a 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -17,8 +17,185 @@
 #include "qemu/typedefs.h"
 #include "qapi/qmp/qobject.h"

+/*
+ * The QAPI schema defines both a set of C data types, and a QMP wire
+ * format.  A QAPI object is formed as a directed acyclic graph of
+ * QAPI values.  QAPI also generates visitor functions to walk these
+ * graphs.  This file represents the interface for doing work at each
+ * point of a QAPI graph; it can also be used for a virtual walk,
+ * where there is no actual QAPI C struct.
+ *
+ * There are three kinds of visitor classes: input visitors parse an
+ * external representation and allocate the corresponding QAPI graph
+ * (QMP, string, and QemuOpts), output visitors take a completed QAPI
+ * graph and generate an external representation (QMP and string), and
+ * the dealloc visitor can take a (partial) QAPI graph and recursively
+ * free its resources.  While the dealloc and QMP input/output
+ * visitors are general, the string and QemuOpts visitors have some
+ * implementation limitations; see the documentation for each visitor
+ * for more details on what it supports.  Also, see visitor-impl.h for
+ * the callback contracts implemented by each visitor, and
+ * docs/qapi-code-gen.txt for more about the QAPI code generator.
+ *
+ * All QAPI types have a corresponding function with a signature
+ * roughly compatible with this:
+ *
+ * void visit_type_FOO(Visitor *v, const char *name, void *obj, Error **errp);
+ *
+ * except that *@obj is typed correctly as a pointer or scalar,
+ * depending on the type of FOO.  The scalar visitors are declared
+ * here; the remaining visitors are generated in qapi-visit.h.
+ *
+ * The @name parameter of visit_type_FOO() describes the relation
+ * between this QAPI value and its parent container.  When visiting
+ * the root of a tree, @name is usually ignored (although some
+ * visitors require it to be NULL); when visiting a member of an
+ * object, @name is the key associated with the value; and when
+ * visiting a member of a list, @name is NULL.
+ *
+ * The visit_type_FOO() functions expect a non-NULL @obj argument;
+ * they allocate *@obj during input visits, leave it unchanged on
+ * output visits, and recursively free any resources during a dealloc
+ * visit.  Each function also has an @errp argument which may be NULL
+ * to ignore errors, or point to a NULL Error object on entry for
+ * reporting any errors (such as if a member @name is not present, or
+ * is present but not the specified type).
+ *
+ * FIXME: Clients must pass NULL for @name when visiting a member of a
+ * list, but this leads to poor error messages; it might be nicer to
+ * require a non-NULL name such as "key.0" for '{ "key": [ "value" ]
+ * }' if an error is encountered on "value" (or to have the visitor
+ * core auto-generate the nicer name).
+ *
+ * FIXME: At present, input visitors may allocate an incomplete *@obj
+ * even when visit_type_FOO() reports an error.  Using an output
+ * visitor with an incomplete object has undefined behavior; callers
+ * must call qapi_free_FOO() (which uses the dealloc visitor, and
+ * safely handles an incomplete object) to avoid a memory leak.
+ *
+ * Likewise, the QAPI object types (structs, unions, and alternates)
+ * have a generated function in qapi-visit.h compatible with:
+ *
+ * void visit_type_FOO_members(Visitor *v, FOO *obj, Error **errp);
+ *
+ * for visiting the members of a type without also allocating the QAPI
+ * struct.
+ *
+ * Additionally, in qapi-types.h, all QAPI pointer types (structs,
+ * unions, alternates, and lists) have a generated function compatible
+ * with:
+ *
+ * void qapi_free_FOO(FOO *obj);
+ *
+ * which behaves like free() (@obj can be NULL); because of these
+ * functions, the dealloc visitor is seldom used directly outside of
+ * generated code.  QAPI types can also inherit from a base class;
+ * when this happens, a function is generated for easily going from
+ * the derived type to the base type:
+ *
+ * BASE *qapi_CHILD_base(CHILD *obj);
+ *
+ * For a real QAPI struct, a typical input life cycle is thus:
+ *
+ * <example>
+ *  Foo *f;
+ *  FooList *l;
+ *  Error *err = NULL;
+ *  Visitor *v;
+ *
+ *  v = ...obtain input visitor...
+ *  visit_type_Foo(v, NULL, &f, &err);
+ *  if (err) {
+ *      qapi_free_Foo(f);
+ *      ...handle error...
+ *  } else {
+ *      ...use f...
+ *  }
+ *  v = ...obtain input visitor...
+ *  visit_type_FooList(v, NULL, &l, &err);
+ *  if (err) {
+ *      qapi_free_FooList(l);
+ *      ...handle error...
+ *  } else {
+ *      while (l) {
+ *          ...use l->value...
+ *          l = l->next;
+ *      }
+ *  }
+ *  ...clean up v...
+ * </example>
+ *
+ * Similarly, a typical output life cycle is:
+ *
+ * <example>
+ *  Foo *f = g_new0(Foo, 1);
+ *  Error *err = NULL;
+ *  Visitor *v;
+ *
+ *  ...populate f...
+ *  v = ...obtain output visitor...
+ *  visit_type_Foo(v, NULL, &f, &err);
+ *  if (err) {
+ *      ...handle error...
+ *  }
+ *  qapi_free_Foo(f);
+ * </example>
+ *
+ * When visiting a real QAPI struct, this file provides several
+ * helpers that rely on in-tree information to control the walk:
+ * visit_optional() for the 'has_member' field associated with
+ * optional 'member' in the C struct; and visit_next_list() for
+ * advancing through a FooList linked list.  Only the generated
+ * visit_type functions need to use these helpers.
+ *
+ * It is also possible to use the visitors to do a virtual walk, where
+ * no actual QAPI struct is present.  In this situation, decisions
+ * about what needs to be walked are made by the calling code, and
+ * structured visits are split between pairs of start and end methods
+ * (where the end method must be called if the start function
+ * succeeded, even if an intermediate visit encounters an error).
+ * Thus, a virtual walk corresponding to '{ "list": [1, 2] }' looks
+ * like:
+ *
+ * <example>
+ *  Visitor *v;
+ *  Error *err = NULL;
+ *  int value;
+ *
+ *  v = ...obtain visitor...
+ *  visit_start_struct(v, NULL, NULL, 0, &err);
+ *  if (err) {
+ *      goto outobj;
+ *  }
+ *  visit_start_list(v, "list", &err);
+ *  if (err) {
+ *      goto outobj;
+ *  }
+ *  value = 1;
+ *  visit_type_int(v, NULL, &value, &err);
+ *  if (err) {
+ *      goto outlist;
+ *  }
+ *  value = 2;
+ *  visit_type_int(v, NULL, &value, &err);
+ *  if (err) {
+ *      goto outlist;
+ *  }
+ * outlist:
+ *  visit_end_list(v);
+ * outobj:
+ *  error_propagate(errp, err);
+ *  err = NULL;
+ *  visit_end_struct(v, &err);
+ *  error_propagate(errp, err);
+ *  ...clean up v...
+ * </example>
+ */
+
+/* === Useful types */
+
 /* This struct is layout-compatible with all other *List structs
- * created by the qapi generator.  It is used as a typical
+ * created by the QAPI generator.  It is used as a typical
  * singly-linked list. */
 typedef struct GenericList {
     struct GenericList *next;
@@ -26,26 +203,116 @@ typedef struct GenericList {
 } GenericList;

 /* This struct is layout-compatible with all Alternate types
- * created by the qapi generator. */
+ * created by the QAPI generator. */
 typedef struct GenericAlternate {
     QType type;
     char padding[];
 } GenericAlternate;

+/* === Visiting structures */
+
+/*
+ * Start visiting an object value @obj (struct or union).
+ *
+ * @name expresses the relationship of this object to its parent
+ * container; see the general description of @name above.
+ *
+ * @obj must be non-NULL for a real walk, in which case @size
+ * determines how much memory an input visitor will allocate into
+ * *@obj.  @obj may also be NULL for a virtual walk, in which case
+ * @size is ignored.
+ *
+ * @errp must be NULL-initialized, and is set if an error is detected
+ * (such as if a member @name is not present, or is present but not an
+ * object).  On error, input visitors set *@obj to NULL.
+ *
+ * After visit_start_struct() succeeds, the caller may visit its
+ * members one after the other, passing the member's name and address
+ * within the struct.  Finally, visit_end_struct() needs to be called
+ * to clean up, even if intermediate visits fail.  See the examples
+ * above.
+ *
+ * FIXME Should this be named visit_start_object, since it is also
+ * used for QAPI unions, and maps to JSON objects?
+ */
 void visit_start_struct(Visitor *v, const char *name, void **obj,
                         size_t size, Error **errp);
+
+/*
+ * Complete an object visit started earlier.
+ *
+ * @errp must be NULL-initialized, and is set if an error is detected
+ * (such as unparsed keys remaining in the input stream).
+ *
+ * Must be called after any successful use of visit_start_struct(),
+ * even if intermediate processing was skipped due to errors, to allow
+ * the backend to release any resources.  Destroying the visitor may
+ * behave as if this was implicitly called.
+ */
 void visit_end_struct(Visitor *v, Error **errp);

+
+/* === Visiting lists */
+
+/*
+ * Start visiting a list.
+ *
+ * @name expresses the relationship of this list to its parent
+ * container; see the general description of @name above.
+ *
+ * @errp must be NULL-initialized, and is set if an error is detected
+ * (such as if a member @name is not present, or is present but not a
+ * list).
+ *
+ * After visit_start_list() succeeds, the caller may visit its members
+ * one after the other.  A real visit uses visit_next_list() for
+ * traversing the linked list, while a virtual visit uses other means.
+ * For each list element, call the appropriate visit_type_FOO() with
+ * name set to NULL and obj set to the address of the list element.
+ * Finally, visit_end_list() needs to be called to clean up.  See the
+ * examples above.
+ */
 void visit_start_list(Visitor *v, const char *name, Error **errp);
+
+/*
+ * Iterate over a GenericList during a list visit.
+ *
+ * @size represents the size of a linked list node.
+ *
+ * @list must not be NULL; on the first call, @list contains the
+ * address of the list head, and on subsequent calls *@list must be
+ * the previously returned value.  Must be called in a loop until a
+ * NULL return or error occurs; for each non-NULL return, the caller
+ * must then call the appropriate visit_type_*() for the element type
+ * of the list, with that function's name parameter set to NULL and
+ * obj set to the address of (*@list)->value.
+ *
+ * FIXME: This interface is awkward; it requires all callbacks to
+ * track whether it is the first or a subsequent call.  A better
+ * interface would pass the head of the list through
+ * visit_start_list().
+ */
 GenericList *visit_next_list(Visitor *v, GenericList **list, size_t size);
+
+/*
+ * Complete a list visit started earlier.
+ *
+ * Must be called after any successful use of visit_start_list(),
+ * even if intermediate processing was skipped due to errors, to allow
+ * the backend to release any resources.  Destroying the visitor may
+ * behave as if this was implicitly called.
+ */
 void visit_end_list(Visitor *v);

+
+/* === Visiting alternates */
+
 /*
- * Start the visit of an alternate @obj with the given @size.
+ * Start the visit of an alternate @obj with the given @size (at least
+ * sizeof(GenericAlternate)).
  *
- * @name specifies the relationship to the containing struct (ignored
- * for a top level visit, the name of the key if this alternate is
- * part of an object, or NULL if this alternate is part of a list).
+ * @name expresses the relationship of this alternate to its parent
+ * container; see the general description of @name above.
  *
  * @obj must not be NULL. Input visitors will allocate @obj and
  * determine the qtype of the next thing to be visited, stored in
@@ -63,46 +330,181 @@ void visit_start_alternate(Visitor *v, const char *name,
 /*
  * Finish visiting an alternate type.
  *
- * Must be called after a successful visit_start_alternate(), even if
- * an error occurred in the meantime.
+ * Must be called after any successful use of visit_start_alternate(),
+ * even if intermediate processing was skipped due to errors, to allow
+ * the backend to release any resources.  Destroying the visitor may
+ * behave as if this was implicitly called.
  *
  * TODO: Should all the visit_end_* interfaces take obj parameter, so
  * that dealloc visitor need not track what was passed in visit_start?
  */
 void visit_end_alternate(Visitor *v);

-/**
- * Check if an optional member @name of an object needs visiting.
- * For input visitors, set *@present according to whether the
- * corresponding visit_type_*() needs calling; for other visitors,
- * leave *@present unchanged.  Return *@present for convenience.
+
+/* === Other helpers */
+
+/*
+ * Does optional struct member @name need visiting?
+ *
+ * @name must not be NULL.  This function is only useful between
+ * visit_start_struct() and visit_end_struct(), since only objects
+ * have optional keys.
+ *
+ * @present points to the address of the optional member's has_ flag.
+ *
+ * Input visitors set *@present according to input; other visitors
+ * leave it unchanged.  In either case, return *@present for
+ * convenience.
  */
 bool visit_optional(Visitor *v, const char *name, bool *present);

+/*
+ * Visit an enum value.
+ *
+ * @name expresses the relationship of this enum to its parent
+ * container; see the general description of @name above.
+ *
+ * @strings expresses the mapping between C enum values and QAPI enum
+ * names; it should be the ENUM_lookup array from visit-types.h.
+ *
+ * @obj must be non-NULL.  For input visitors, parse a string and set
+ * *@obj to the numeric value of the enum type using @strings as the
+ * mapping, leaving @obj unchanged on error; other visitors leave
+ * *@obj unchanged.  Output visitors use *@obj to reverse the mapping
+ * and visit the appropriate output string.
+ */
 void visit_type_enum(Visitor *v, const char *name, int *obj,
                      const char *const strings[], Error **errp);
+
+
+/* === Visiting built-in types */
+
+/*
+ * Visit an integer value.
+ *
+ * @name expresses the relationship of this integer to its parent
+ * container; see the general description of @name above.
+ *
+ * @obj must be non-NULL.  Input visitors set *@obj to the value;
+ * other visitors will leave *@obj unchanged.
+ */
 void visit_type_int(Visitor *v, const char *name, int64_t *obj, Error **errp);
+
+/*
+ * Visit a uint8_t value.
+ * Like visit_type_int(), except clamps the value to uint8_t range.
+ */
 void visit_type_uint8(Visitor *v, const char *name, uint8_t *obj,
                       Error **errp);
+
+/*
+ * Visit a uint16_t value.
+ * Like visit_type_int(), except clamps the value to uint16_t range.
+ */
 void visit_type_uint16(Visitor *v, const char *name, uint16_t *obj,
                        Error **errp);
+
+/*
+ * Visit a uint32_t value.
+ * Like visit_type_int(), except clamps the value to uint32_t range.
+ */
 void visit_type_uint32(Visitor *v, const char *name, uint32_t *obj,
                        Error **errp);
+
+/*
+ * Visit a uint64_t value.
+ * Like visit_type_int(), except clamps the value to uint64_t range,
+ * that is, ensures it is unsigned.
+ */
 void visit_type_uint64(Visitor *v, const char *name, uint64_t *obj,
                        Error **errp);
+
+/*
+ * Visit an int8_t value.
+ * Like visit_type_int(), except clamps the value to int8_t range.
+ */
 void visit_type_int8(Visitor *v, const char *name, int8_t *obj, Error **errp);
+
+/*
+ * Visit an int16_t value.
+ * Like visit_type_int(), except clamps the value to int16_t range.
+ */
 void visit_type_int16(Visitor *v, const char *name, int16_t *obj,
                       Error **errp);
+
+/*
+ * Visit an int32_t value.
+ * Like visit_type_int(), except clamps the value to int32_t range.
+ */
 void visit_type_int32(Visitor *v, const char *name, int32_t *obj,
                       Error **errp);
+
+/*
+ * Visit an int64_t value.
+ * Identical to visit_type_int().
+ */
 void visit_type_int64(Visitor *v, const char *name, int64_t *obj,
                       Error **errp);
+
+/*
+ * Visit a uint64_t value.
+ * Like visit_type_uint64(), except that some visitors may choose to
+ * recognize additional syntax, such as suffixes for easily scaling
+ * values.
+ */
 void visit_type_size(Visitor *v, const char *name, uint64_t *obj,
                      Error **errp);
+
+/*
+ * Visit a boolean value.
+ *
+ * @name expresses the relationship of this boolean to its parent
+ * container; see the general description of @name above.
+ *
+ * @obj must be non-NULL.  Input visitors set *@obj to the value;
+ * other visitors will leave *@obj unchanged.
+ */
 void visit_type_bool(Visitor *v, const char *name, bool *obj, Error **errp);
+
+/*
+ * Visit a string value.
+ *
+ * @name expresses the relationship of this string to its parent
+ * container; see the general description of @name above.
+ *
+ * @obj must be non-NULL.  Input visitors set *@obj to the value
+ * (never NULL).  Other visitors leave *@obj unchanged, and commonly
+ * treat NULL like "".
+ *
+ * Note that using an output visitor along with a (const char *) value
+ * requires casting away const when computing @obj.
+ *
+ * FIXME: Callers that try to output NULL *obj should not be allowed.
+ */
 void visit_type_str(Visitor *v, const char *name, char **obj, Error **errp);
+
+/*
+ * Visit a number value.
+ *
+ * @name expresses the relationship of this number to its parent
+ * container; see the general description of @name above.
+ *
+ * @obj must be non-NULL.  Input visitors set *@obj to the value;
+ * other visitors will leave *@obj unchanged.
+ */
 void visit_type_number(Visitor *v, const char *name, double *obj,
                        Error **errp);
+
+/*
+ * Visit an arbitrary value.
+ *
+ * @name expresses the relationship of this value to its parent
+ * container; see the general description of @name above.
+ *
+ * @obj must be non-NULL.  Input visitors set *@obj to the value;
+ * other visitors will leave *@obj unchanged.  *@obj must be non-NULL
+ * for output visitors.
+ */
 void visit_type_any(Visitor *v, const char *name, QObject **obj, Error **errp);

 #endif
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 228a2a6..d967b18 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -14,6 +14,16 @@

 #include "qapi/visitor.h"

+/* This file describes the callback interface for implementing a QAPI
+ * visitor.  For the client interface, see visitor.h.  When
+ * implementing the callbacks, it is easiest to declare a struct with
+ * 'Visitor visitor;' as the first member.  A callback's contract
+ * matches the corresponding public functions' contract unless stated
+ * otherwise.  In the comments below, some callbacks are marked "must
+ * be set for $TYPE visits to work"; if a visitor implementation omits
+ * that callback, it should also document that it is only useful for a
+ * subset of QAPI.  */
+
 /* There are three classes of visitors; setting the class determines
  * how QAPI enums are visited, as well as what additional restrictions
  * can be asserted.  */
@@ -25,18 +35,24 @@ typedef enum VisitorType {

 struct Visitor
 {
-    /* Must be set */
+    /* Must be set to visit structs.  */
     void (*start_struct)(Visitor *v, const char *name, void **obj,
                          size_t size, Error **errp);
+
+    /* Must be set to visit structs.  */
     void (*end_struct)(Visitor *v, Error **errp);

+    /* Must be set.  */
     void (*start_list)(Visitor *v, const char *name, Error **errp);
-    /* Must be set */
+
+    /* Must be set.  */
     GenericList *(*next_list)(Visitor *v, GenericList **list, size_t size);
-    /* Must be set */
+
+    /* Must be set.  */
     void (*end_list)(Visitor *v);

-    /* Optional, needed for input and dealloc visitors.  */
+    /* Must be set by input and dealloc visitors to visit alternates;
+     * optional for output visitors.  */
     void (*start_alternate)(Visitor *v, const char *name,
                             GenericAlternate **obj, size_t size,
                             bool promote_int, Error **errp);
@@ -44,24 +60,34 @@ struct Visitor
     /* Optional, needed for dealloc visitor.  */
     void (*end_alternate)(Visitor *v);

-    /* Must be set. */
+    /* Must be set.  */
     void (*type_int64)(Visitor *v, const char *name, int64_t *obj,
                        Error **errp);
-    /* Must be set. */
+
+    /* Must be set.  */
     void (*type_uint64)(Visitor *v, const char *name, uint64_t *obj,
                         Error **errp);
+
     /* Optional; fallback is type_uint64().  */
     void (*type_size)(Visitor *v, const char *name, uint64_t *obj,
                       Error **errp);
-    /* Must be set. */
+
+    /* Must be set.  */
     void (*type_bool)(Visitor *v, const char *name, bool *obj, Error **errp);
+
+    /* Must be set.  */
     void (*type_str)(Visitor *v, const char *name, char **obj, Error **errp);
+
+    /* Must be set to visit numbers.  */
     void (*type_number)(Visitor *v, const char *name, double *obj,
                         Error **errp);
+
+    /* Must be set to visit arbitrary QTypes.  */
     void (*type_any)(Visitor *v, const char *name, QObject **obj,
                      Error **errp);

-    /* May be NULL; most useful for input visitors. */
+    /* Must be set for input visitors, optional otherwise.  The core
+     * takes care of the return type in the public interface.  */
     void (*optional)(Visitor *v, const char *name, bool *present);

     /* Must be set.  */
diff --git a/include/qapi/dealloc-visitor.h b/include/qapi/dealloc-visitor.h
index cf4c36d..7aa8293 100644
--- a/include/qapi/dealloc-visitor.h
+++ b/include/qapi/dealloc-visitor.h
@@ -18,6 +18,10 @@

 typedef struct QapiDeallocVisitor QapiDeallocVisitor;

+/* The dealloc visitor is primarly used only by generated
+ * qapi_free_FOO() functions, and is the only visitor designed to work
+ * correctly in the face of a partially-constructed QAPI tree.
+ */
 QapiDeallocVisitor *qapi_dealloc_visitor_new(void);
 void qapi_dealloc_visitor_cleanup(QapiDeallocVisitor *d);

diff --git a/include/qapi/opts-visitor.h b/include/qapi/opts-visitor.h
index fd48c14..2002e37 100644
--- a/include/qapi/opts-visitor.h
+++ b/include/qapi/opts-visitor.h
@@ -29,6 +29,10 @@ typedef struct OptsVisitor OptsVisitor;
  * - string representations of negative numbers yield negative values,
  * - values below INT64_MIN or LLONG_MIN are rejected,
  * - values above INT64_MAX or LLONG_MAX are rejected.
+ *
+ * The Opts input visitor does not yet implement support for visiting
+ * QAPI alternates, numbers (other than integers), or arbitrary
+ * QTypes.
  */
 OptsVisitor *opts_visitor_new(const QemuOpts *opts);
 void opts_visitor_cleanup(OptsVisitor *nv);
diff --git a/include/qapi/qmp-input-visitor.h b/include/qapi/qmp-input-visitor.h
index 3ed499c..d75ff98 100644
--- a/include/qapi/qmp-input-visitor.h
+++ b/include/qapi/qmp-input-visitor.h
@@ -19,6 +19,14 @@

 typedef struct QmpInputVisitor QmpInputVisitor;

+/*
+ * FIXME: When visiting a QDict, passing a non-NULL @name for the
+ * first visit_type_FOO() when the root is a QDict will find that
+ * particular key within the QDict.  In the future, the contract may
+ * be tightened to require visit_start_struct() with ignored @name as
+ * the first visit; in the meantime, the first visit is safest when
+ * using NULL for @name.
+ */
 QmpInputVisitor *qmp_input_visitor_new(QObject *obj);
 QmpInputVisitor *qmp_input_visitor_new_strict(QObject *obj);

diff --git a/include/qapi/string-input-visitor.h b/include/qapi/string-input-visitor.h
index 089243c..4c8d1ea 100644
--- a/include/qapi/string-input-visitor.h
+++ b/include/qapi/string-input-visitor.h
@@ -17,6 +17,10 @@

 typedef struct StringInputVisitor StringInputVisitor;

+/*
+ * The string input visitor does not yet implement support for
+ * visiting QAPI structs, alternates, or arbitrary QTypes.
+ */
 StringInputVisitor *string_input_visitor_new(const char *str);
 void string_input_visitor_cleanup(StringInputVisitor *v);

diff --git a/include/qapi/string-output-visitor.h b/include/qapi/string-output-visitor.h
index d99717f..094a11e 100644
--- a/include/qapi/string-output-visitor.h
+++ b/include/qapi/string-output-visitor.h
@@ -17,6 +17,10 @@

 typedef struct StringOutputVisitor StringOutputVisitor;

+/*
+ * The string output visitor does not yet implement support for
+ * visiting QAPI structs, alternates, or arbitrary QTypes.
+ */
 StringOutputVisitor *string_output_visitor_new(bool human);
 void string_output_visitor_cleanup(StringOutputVisitor *v);

diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index a08d073..1388462 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -22,6 +22,10 @@
 void visit_start_struct(Visitor *v, const char *name, void **obj,
                         size_t size, Error **errp)
 {
+    if (obj) {
+        assert(size);
+        assert(v->type != VISITOR_OUTPUT || *obj);
+    }
     v->start_struct(v, name, obj, size, errp);
 }

@@ -73,6 +77,7 @@ bool visit_optional(Visitor *v, const char *name, bool *present)

 void visit_type_int(Visitor *v, const char *name, int64_t *obj, Error **errp)
 {
+    assert(obj);
     v->type_int64(v, name, obj, errp);
 }

@@ -120,6 +125,7 @@ void visit_type_uint32(Visitor *v, const char *name, uint32_t *obj,
 void visit_type_uint64(Visitor *v, const char *name, uint64_t *obj,
                        Error **errp)
 {
+    assert(obj);
     v->type_uint64(v, name, obj, errp);
 }

@@ -167,12 +173,14 @@ void visit_type_int32(Visitor *v, const char *name, int32_t *obj,
 void visit_type_int64(Visitor *v, const char *name, int64_t *obj,
                       Error **errp)
 {
+    assert(obj);
     v->type_int64(v, name, obj, errp);
 }

 void visit_type_size(Visitor *v, const char *name, uint64_t *obj,
                      Error **errp)
 {
+    assert(obj);
     if (v->type_size) {
         v->type_size(v, name, obj, errp);
     } else {
@@ -182,22 +190,31 @@ void visit_type_size(Visitor *v, const char *name, uint64_t *obj,

 void visit_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
 {
+    assert(obj);
     v->type_bool(v, name, obj, errp);
 }

 void visit_type_str(Visitor *v, const char *name, char **obj, Error **errp)
 {
+    assert(obj);
+    /* TODO: Fix callers to not pass NULL when they mean "", so that we
+     * can enable:
+    assert(v->type != VISITOR_OUTPUT || *obj);
+     */
     v->type_str(v, name, obj, errp);
 }

 void visit_type_number(Visitor *v, const char *name, double *obj,
                        Error **errp)
 {
+    assert(obj);
     v->type_number(v, name, obj, errp);
 }

 void visit_type_any(Visitor *v, const char *name, QObject **obj, Error **errp)
 {
+    assert(obj);
+    assert(v->type != VISITOR_OUTPUT || *obj);
     v->type_any(v, name, obj, errp);
 }

@@ -251,7 +268,7 @@ static void input_type_enum(Visitor *v, const char *name, int *obj,
 void visit_type_enum(Visitor *v, const char *name, int *obj,
                      const char *const strings[], Error **errp)
 {
-    assert(strings);
+    assert(obj && strings);
     if (v->type == VISITOR_INPUT) {
         input_type_enum(v, name, obj, strings, errp);
     } else if (v->type == VISITOR_OUTPUT) {
-- 
2.5.0

[Qemu-devel] [PATCH v12 08/18] tests: Add check-qnull

[Eric Blake]

Add a new test, for checking reference counting of qnull(). As
part of the new file, move a previous reference counting change
added in commit a861564 into a more logical place.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: new patch
---
 tests/check-qnull.c             | 60 +++++++++++++++++++++++++++++++++++++++++
 tests/test-qmp-output-visitor.c |  2 --
 tests/.gitignore                |  1 +
 tests/Makefile                  |  6 ++++-
 4 files changed, 66 insertions(+), 3 deletions(-)
 create mode 100644 tests/check-qnull.c

diff --git a/tests/check-qnull.c b/tests/check-qnull.c
new file mode 100644
index 0000000..b0fb1e6
--- /dev/null
+++ b/tests/check-qnull.c
@@ -0,0 +1,60 @@
+/*
+ * QNull unit-tests.
+ *
+ * Copyright (C) 2016 Red Hat Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ */
+#include "qemu/osdep.h"
+#include <glib.h>
+
+#include "qapi/qmp/qobject.h"
+#include "qemu-common.h"
+#include "qapi/qmp-output-visitor.h"
+
+/*
+ * Public Interface test-cases
+ *
+ * (with some violations to access 'private' data)
+ */
+
+static void qnull_ref_test(void)
+{
+    QObject *obj;
+
+    g_assert(qnull_.refcnt == 1);
+    obj = qnull();
+    g_assert(obj);
+    g_assert(obj == &qnull_);
+    g_assert(qnull_.refcnt == 2);
+    g_assert(qobject_type(obj) == QTYPE_QNULL);
+    qobject_decref(obj);
+    g_assert(qnull_.refcnt == 1);
+}
+
+static void qnull_visit_test(void)
+{
+    QObject *obj;
+    QmpOutputVisitor *qov;
+
+    g_assert(qnull_.refcnt == 1);
+    qov = qmp_output_visitor_new();
+    /* FIXME: Empty visits are ugly, we should have a visit_type_null(). */
+    obj = qmp_output_get_qobject(qov);
+    g_assert(obj == &qnull_);
+    qobject_decref(obj);
+
+    qmp_output_visitor_cleanup(qov);
+    g_assert(qnull_.refcnt == 1);
+}
+
+int main(int argc, char **argv)
+{
+    g_test_init(&argc, &argv, NULL);
+
+    g_test_add_func("/public/qnull_ref", qnull_ref_test);
+    g_test_add_func("/public/qnull_visit", qnull_visit_test);
+
+    return g_test_run();
+}
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index dc35752..81c90bc 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -483,8 +483,6 @@ static void test_visitor_out_empty(TestOutputVisitorData *data,

     arg = qmp_output_get_qobject(data->qov);
     g_assert(qobject_type(arg) == QTYPE_QNULL);
-    /* Check that qnull reference counting is sane */
-    g_assert(arg->refcnt == 2);
     qobject_decref(arg);
 }

diff --git a/tests/.gitignore b/tests/.gitignore
index 787c95c..615f96d 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -3,6 +3,7 @@ check-qfloat
 check-qint
 check-qjson
 check-qlist
+check-qnull
 check-qstring
 check-qom-interface
 check-qom-proplist
diff --git a/tests/Makefile b/tests/Makefile
index cd4bbd4..b0a4f8e 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -16,6 +16,8 @@ check-unit-y += tests/check-qstring$(EXESUF)
 gcov-files-check-qstring-y = qobject/qstring.c
 check-unit-y += tests/check-qlist$(EXESUF)
 gcov-files-check-qlist-y = qobject/qlist.c
+check-unit-y += tests/check-qnull$(EXESUF)
+gcov-files-check-qnull-y = qobject/qnull.c
 check-unit-y += tests/check-qjson$(EXESUF)
 gcov-files-check-qjson-y = qobject/qjson.c
 check-unit-y += tests/test-qmp-output-visitor$(EXESUF)
@@ -371,7 +373,8 @@ GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h \
 	tests/test-qmp-introspect.h

 test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
-	tests/check-qlist.o tests/check-qfloat.o tests/check-qjson.o \
+	tests/check-qlist.o tests/check-qfloat.o tests/check-qnull.o \
+	tests/check-qjson.o \
 	tests/test-coroutine.o tests/test-string-output-visitor.o \
 	tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \
 	tests/test-qmp-input-visitor.o tests/test-qmp-input-strict.o \
@@ -399,6 +402,7 @@ tests/check-qstring$(EXESUF): tests/check-qstring.o $(test-util-obj-y)
 tests/check-qdict$(EXESUF): tests/check-qdict.o $(test-util-obj-y)
 tests/check-qlist$(EXESUF): tests/check-qlist.o $(test-util-obj-y)
 tests/check-qfloat$(EXESUF): tests/check-qfloat.o $(test-util-obj-y)
+tests/check-qnull$(EXESUF): tests/check-qnull.o $(test-util-obj-y)
 tests/check-qjson$(EXESUF): tests/check-qjson.o $(test-util-obj-y)
 tests/check-qom-interface$(EXESUF): tests/check-qom-interface.o $(test-qom-obj-y)
 tests/check-qom-proplist$(EXESUF): tests/check-qom-proplist.o $(test-qom-obj-y)
-- 
2.5.0

[Qemu-devel] [PATCH v12 09/18] qapi: Add visit_type_null() visitor

[Eric Blake]

Right now, qmp-output-visitor happens to produce a QNull result
if nothing is actually visited between the creation of the visitor
and the request for the resulting QObject.  A stronger protocol
would require that a QMP output visit MUST visit something.  But
to still be able to produce a JSON 'null' output, we need a new
visitor function that states our intentions.  Yes, we could say
that such a visit must go through visit_type_any(), but that
feels clunky.

So this patch introduces the new visit_type_null() interface and
its no-op interface in the dealloc visitor, and the next patch
will then wire it up into the qmp visitors.  For the visitors
that will not implement the callback, document the situation.
The code in qapi-visit-core unconditionally dereferences the
callback pointer, so that a segfault will inform a developer if
they need to implement the callback for their choice of visitor.

If QAPI had a 'null' type, we'd also have to use visit_type_null()
in the generated visitor functions (most likely, we'd use it via
an alternate type that permits 'null' or an object); we'll create
that usage when we need it.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: rebase to earlier changes, drop R-b due to better documentation
[no v10, v11]
v9: no change
v8: rebase to 'name' motion
v7: new patch, based on discussion about spapr_drc.c
---
 include/qapi/visitor.h               | 12 ++++++++++++
 include/qapi/visitor-impl.h          |  3 +++
 include/qapi/opts-visitor.h          |  2 +-
 include/qapi/string-input-visitor.h  |  2 +-
 include/qapi/string-output-visitor.h |  2 +-
 qapi/qapi-visit-core.c               |  5 +++++
 qapi/qapi-dealloc-visitor.c          |  5 +++++
 7 files changed, 28 insertions(+), 3 deletions(-)

diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 33ae28a..2b85c6f 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -507,4 +507,16 @@ void visit_type_number(Visitor *v, const char *name, double *obj,
  */
 void visit_type_any(Visitor *v, const char *name, QObject **obj, Error **errp);

+/*
+ * Visit a JSON null value.
+ *
+ * @name expresses the relationship of the null value to its parent
+ * container; see the general description of @name above.
+ *
+ * Unlike all other visit_type_* functions, no obj parameter is
+ * needed; rather, this is a witness that an explicit null value is
+ * expected rather than any other type.
+ */
+void visit_type_null(Visitor *v, const char *name, Error **errp);
+
 #endif
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index d967b18..90bcaec 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -86,6 +86,9 @@ struct Visitor
     void (*type_any)(Visitor *v, const char *name, QObject **obj,
                      Error **errp);

+    /* Must be set to visit explicit null values.  */
+    void (*type_null)(Visitor *v, const char *name, Error **errp);
+
     /* Must be set for input visitors, optional otherwise.  The core
      * takes care of the return type in the public interface.  */
     void (*optional)(Visitor *v, const char *name, bool *present);
diff --git a/include/qapi/opts-visitor.h b/include/qapi/opts-visitor.h
index 2002e37..3fcd327 100644
--- a/include/qapi/opts-visitor.h
+++ b/include/qapi/opts-visitor.h
@@ -31,7 +31,7 @@ typedef struct OptsVisitor OptsVisitor;
  * - values above INT64_MAX or LLONG_MAX are rejected.
  *
  * The Opts input visitor does not yet implement support for visiting
- * QAPI alternates, numbers (other than integers), or arbitrary
+ * QAPI alternates, numbers (other than integers), null, or arbitrary
  * QTypes.
  */
 OptsVisitor *opts_visitor_new(const QemuOpts *opts);
diff --git a/include/qapi/string-input-visitor.h b/include/qapi/string-input-visitor.h
index 4c8d1ea..1a34c52 100644
--- a/include/qapi/string-input-visitor.h
+++ b/include/qapi/string-input-visitor.h
@@ -19,7 +19,7 @@ typedef struct StringInputVisitor StringInputVisitor;

 /*
  * The string input visitor does not yet implement support for
- * visiting QAPI structs, alternates, or arbitrary QTypes.
+ * visiting QAPI structs, alternates, null, or arbitrary QTypes.
  */
 StringInputVisitor *string_input_visitor_new(const char *str);
 void string_input_visitor_cleanup(StringInputVisitor *v);
diff --git a/include/qapi/string-output-visitor.h b/include/qapi/string-output-visitor.h
index 094a11e..2564833 100644
--- a/include/qapi/string-output-visitor.h
+++ b/include/qapi/string-output-visitor.h
@@ -19,7 +19,7 @@ typedef struct StringOutputVisitor StringOutputVisitor;

 /*
  * The string output visitor does not yet implement support for
- * visiting QAPI structs, alternates, or arbitrary QTypes.
+ * visiting QAPI structs, alternates, null, or arbitrary QTypes.
  */
 StringOutputVisitor *string_output_visitor_new(bool human);
 void string_output_visitor_cleanup(StringOutputVisitor *v);
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 1388462..116c627 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -218,6 +218,11 @@ void visit_type_any(Visitor *v, const char *name, QObject **obj, Error **errp)
     v->type_any(v, name, obj, errp);
 }

+void visit_type_null(Visitor *v, const char *name, Error **errp)
+{
+    v->type_null(v, name, errp);
+}
+
 static void output_type_enum(Visitor *v, const char *name, int *obj,
                              const char *const strings[], Error **errp)
 {
diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c
index c19a459..413d525 100644
--- a/qapi/qapi-dealloc-visitor.c
+++ b/qapi/qapi-dealloc-visitor.c
@@ -163,6 +163,10 @@ static void qapi_dealloc_type_anything(Visitor *v, const char *name,
     }
 }

+static void qapi_dealloc_type_null(Visitor *v, const char *name, Error **errp)
+{
+}
+
 Visitor *qapi_dealloc_get_visitor(QapiDeallocVisitor *v)
 {
     return &v->visitor;
@@ -193,6 +197,7 @@ QapiDeallocVisitor *qapi_dealloc_visitor_new(void)
     v->visitor.type_str = qapi_dealloc_type_str;
     v->visitor.type_number = qapi_dealloc_type_number;
     v->visitor.type_any = qapi_dealloc_type_anything;
+    v->visitor.type_null = qapi_dealloc_type_null;

     QTAILQ_INIT(&v->stack);

-- 
2.5.0

[Qemu-devel] [PATCH v12 10/18] qmp: Support explicit null during visits

[Eric Blake]

Implement the new type_null() callback for the qmp input and
output visitors. While we don't yet have a use for this in QAPI
input (the generator will need some tweaks first), one usage
is already envisioned: when changing blockdev parameters, it
would be nice to have a difference between leaving a tuning
parameter unchanged (omit that parameter from the struct) and
to explicitly reset the parameter to its default without having
to know what the default value is (specify the parameter with
an explicit null value, which will require us to allow a QAPI
alternate that chooses between the normal value and an explicit
null).  Meanwhile, the output visitor could already output
explicit null via type_any, but this gives us finer control.

At any rate, it's easy to test that we can round-trip an explicit
null through manual use of visit_type_null().  Repurpose the
test_visitor_out_empty test, particularly since a future patch
will tighten semantics to forbid immediate use of
qmp_output_get_qobject() without at least one visit_type_*.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: retitle and merge in output visitor support, move refcount
tests to separate file
[no v10, v11]
v9: new patch
---
 qapi/qmp-input-visitor.c        | 12 ++++++++++++
 qapi/qmp-output-visitor.c       |  7 +++++++
 tests/check-qnull.c             | 10 +++++++++-
 tests/test-qmp-input-visitor.c  | 16 ++++++++++++++++
 tests/test-qmp-output-visitor.c |  9 +++++----
 5 files changed, 49 insertions(+), 5 deletions(-)

diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index ed54d4c..faa40e7 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -338,6 +338,17 @@ static void qmp_input_type_any(Visitor *v, const char *name, QObject **obj,
     *obj = qobj;
 }

+static void qmp_input_type_null(Visitor *v, const char *name, Error **errp)
+{
+    QmpInputVisitor *qiv = to_qiv(v);
+    QObject *qobj = qmp_input_get_object(qiv, name, true);
+
+    if (qobject_type(qobj) != QTYPE_QNULL) {
+        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+                   "null");
+    }
+}
+
 static void qmp_input_optional(Visitor *v, const char *name, bool *present)
 {
     QmpInputVisitor *qiv = to_qiv(v);
@@ -381,6 +392,7 @@ QmpInputVisitor *qmp_input_visitor_new(QObject *obj)
     v->visitor.type_str = qmp_input_type_str;
     v->visitor.type_number = qmp_input_type_number;
     v->visitor.type_any = qmp_input_type_any;
+    v->visitor.type_null = qmp_input_type_null;
     v->visitor.optional = qmp_input_optional;

     qmp_input_push(v, obj, NULL, NULL);
diff --git a/qapi/qmp-output-visitor.c b/qapi/qmp-output-visitor.c
index 1f2a7ba..5681ad3 100644
--- a/qapi/qmp-output-visitor.c
+++ b/qapi/qmp-output-visitor.c
@@ -196,6 +196,12 @@ static void qmp_output_type_any(Visitor *v, const char *name, QObject **obj,
     qmp_output_add_obj(qov, name, *obj);
 }

+static void qmp_output_type_null(Visitor *v, const char *name, Error **errp)
+{
+    QmpOutputVisitor *qov = to_qov(v);
+    qmp_output_add_obj(qov, name, qnull());
+}
+
 /* Finish building, and return the root object. Will not be NULL. */
 QObject *qmp_output_get_qobject(QmpOutputVisitor *qov)
 {
@@ -246,6 +252,7 @@ QmpOutputVisitor *qmp_output_visitor_new(void)
     v->visitor.type_str = qmp_output_type_str;
     v->visitor.type_number = qmp_output_type_number;
     v->visitor.type_any = qmp_output_type_any;
+    v->visitor.type_null = qmp_output_type_null;

     QTAILQ_INIT(&v->stack);

diff --git a/tests/check-qnull.c b/tests/check-qnull.c
index b0fb1e6..e16c783 100644
--- a/tests/check-qnull.c
+++ b/tests/check-qnull.c
@@ -11,6 +11,7 @@

 #include "qapi/qmp/qobject.h"
 #include "qemu-common.h"
+#include "qapi/qmp-input-visitor.h"
 #include "qapi/qmp-output-visitor.h"

 /*
@@ -37,15 +38,22 @@ static void qnull_visit_test(void)
 {
     QObject *obj;
     QmpOutputVisitor *qov;
+    QmpInputVisitor *qiv;

     g_assert(qnull_.refcnt == 1);
+    obj = qnull();
+    qiv = qmp_input_visitor_new(obj);
+    qobject_decref(obj);
+    visit_type_null(qmp_input_get_visitor(qiv), NULL, &error_abort);
+
     qov = qmp_output_visitor_new();
-    /* FIXME: Empty visits are ugly, we should have a visit_type_null(). */
+    visit_type_null(qmp_output_get_visitor(qov), NULL, &error_abort);
     obj = qmp_output_get_qobject(qov);
     g_assert(obj == &qnull_);
     qobject_decref(obj);

     qmp_output_visitor_cleanup(qov);
+    qmp_input_visitor_cleanup(qiv);
     g_assert(qnull_.refcnt == 1);
 }

diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 5941e90..46fa1b9 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -278,6 +278,20 @@ static void test_visitor_in_any(TestInputVisitorData *data,
     qobject_decref(res);
 }

+static void test_visitor_in_null(TestInputVisitorData *data,
+                                 const void *unused)
+{
+    Visitor *v;
+
+    v = visitor_input_test_init(data, "null");
+    visit_type_null(v, NULL, &error_abort);
+
+    v = visitor_input_test_init(data, "{ 'a': null }");
+    visit_start_struct(v, NULL, NULL, 0, &error_abort);
+    visit_type_null(v, "a", &error_abort);
+    visit_end_struct(v, &error_abort);
+}
+
 static void test_visitor_in_union_flat(TestInputVisitorData *data,
                                        const void *unused)
 {
@@ -828,6 +842,8 @@ int main(int argc, char **argv)
                            &in_visitor_data, test_visitor_in_list);
     input_visitor_test_add("/visitor/input/any",
                            &in_visitor_data, test_visitor_in_any);
+    input_visitor_test_add("/visitor/input/null",
+                           &in_visitor_data, test_visitor_in_null);
     input_visitor_test_add("/visitor/input/union-flat",
                            &in_visitor_data, test_visitor_in_union_flat);
     input_visitor_test_add("/visitor/input/alternate",
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 81c90bc..4f7692b 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -476,11 +476,12 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
     qobject_decref(arg);
 }

-static void test_visitor_out_empty(TestOutputVisitorData *data,
-                                   const void *unused)
+static void test_visitor_out_null(TestOutputVisitorData *data,
+                                  const void *unused)
 {
     QObject *arg;

+    visit_type_null(data->ov, NULL, &error_abort);
     arg = qmp_output_get_qobject(data->qov);
     g_assert(qobject_type(arg) == QTYPE_QNULL);
     qobject_decref(arg);
@@ -836,8 +837,8 @@ int main(int argc, char **argv)
                             &out_visitor_data, test_visitor_out_union_flat);
     output_visitor_test_add("/visitor/output/alternate",
                             &out_visitor_data, test_visitor_out_alternate);
-    output_visitor_test_add("/visitor/output/empty",
-                            &out_visitor_data, test_visitor_out_empty);
+    output_visitor_test_add("/visitor/output/null",
+                            &out_visitor_data, test_visitor_out_null);
     output_visitor_test_add("/visitor/output/native_list/int",
                             &out_visitor_data,
                             test_visitor_out_native_list_int);
-- 
2.5.0

[Qemu-devel] [PATCH v12 11/18] spapr_drc: Expose 'null' in qom-get when there is no fdt

[Eric Blake]

Now that the QMP output visitor supports an explicit null
output, we should utilize it to make it easier to diagnose
the difference between a missing fdt ('null') vs. a
present-but-empty one ('{}').

(Note that this reverts the behavior of commit ab8bf1d, taking
us back to the behavior of commit 6c2f9a1 [which in turn
stemmed from a crash fix in 1d10b44]; but that this time,
the change is intentional and not an accidental side-effect.)

Signed-off-by: Eric Blake <eblake@redhat.com>
Acked-by: David Gibson <david@gibson.dropbear.id.au>

---
v12: tweak commit message
[no v10, v11]
v9: improved commit message
v8: rebase to 'name' motion
v7: new patch, based on discussion about spapr_drc.c
---
 hw/ppc/spapr_drc.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index ef063c0..b1bf193 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -260,11 +260,7 @@ static void prop_get_fdt(Object *obj, Visitor *v, const char *name,
     void *fdt;

     if (!drc->fdt) {
-        visit_start_struct(v, name, NULL, 0, &err);
-        if (!err) {
-            visit_end_struct(v, &err);
-        }
-        error_propagate(errp, err);
+        visit_type_null(v, NULL, errp);
         return;
     }

-- 
2.5.0

[Qemu-devel] [PATCH v12 12/18] qmp: Tighten output visitor rules

[Eric Blake]

Add a new qmp_output_visitor_reset(), which must be called before
reusing an exising QmpOutputVisitor on a new root object.  Tighten
assertions to require that qmp_output_get_qobject() can only be
called after pairing a visit_end_* for every visit_start_* (rather
than allowing it to return a partially built object), and that it
must not be called unless at least one visit_type_* or visit_start/
visit_end pair has occurred since creation/reset (the accidental
return of NULL fixed by commit ab8bf1d7 would have been much
easier to diagnose).

Also, check that we are encountering the expected object or list
type (both during visit_end*, and also by validating whether 'name'
was NULL - although the latter may need to change later if we
improve error messages by always passing in a sensible name).
This provides protection against mismatched push(struct)/pop(list)
or push(list)/pop(struct), similar to the qmp-input protection
added in commit bdd8e6b5.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: rebase to latest, move type_null() into earlier patches,
don't change signature of pop, don't auto-reset after a single
get_qobject
[no v10, v11]
v9: rebase to added patch, squash in more sanity checks, drop
Marc-Andre's R-b
v8: rename qmp_output_reset to qmp_output_visitor_reset
v7: new patch, based on discussion about spapr_drc.c
---
 include/qapi/qmp-output-visitor.h |  1 +
 qapi/qmp-output-visitor.c         | 39 +++++++++++++++++++++++----------------
 tests/test-qmp-output-visitor.c   |  6 ++++++
 3 files changed, 30 insertions(+), 16 deletions(-)

diff --git a/include/qapi/qmp-output-visitor.h b/include/qapi/qmp-output-visitor.h
index 2266770..5093f0d 100644
--- a/include/qapi/qmp-output-visitor.h
+++ b/include/qapi/qmp-output-visitor.h
@@ -21,6 +21,7 @@ typedef struct QmpOutputVisitor QmpOutputVisitor;

 QmpOutputVisitor *qmp_output_visitor_new(void);
 void qmp_output_visitor_cleanup(QmpOutputVisitor *v);
+void qmp_output_visitor_reset(QmpOutputVisitor *v);

 QObject *qmp_output_get_qobject(QmpOutputVisitor *v);
 Visitor *qmp_output_get_visitor(QmpOutputVisitor *v);
diff --git a/qapi/qmp-output-visitor.c b/qapi/qmp-output-visitor.c
index 5681ad3..7c48dfb 100644
--- a/qapi/qmp-output-visitor.c
+++ b/qapi/qmp-output-visitor.c
@@ -82,9 +82,8 @@ static void qmp_output_add_obj(QmpOutputVisitor *qov, const char *name,
     QObject *cur = e ? e->value : NULL;

     if (!cur) {
-        /* FIXME we should require the user to reset the visitor, rather
-         * than throwing away the previous root */
-        qobject_decref(qov->root);
+        /* Don't allow reuse of visitor on more than one root */
+        assert(!qov->root);
         qov->root = value;
     } else {
         switch (qobject_type(cur)) {
@@ -93,6 +92,9 @@ static void qmp_output_add_obj(QmpOutputVisitor *qov, const char *name,
             qdict_put_obj(qobject_to_qdict(cur), name, value);
             break;
         case QTYPE_QLIST:
+            /* FIXME: assertion needs adjustment if we fix visit-core
+             * to pass "name.0" style name during lists.  */
+            assert(!name);
             qlist_append_obj(qobject_to_qlist(cur), value);
             break;
         default:
@@ -114,7 +116,8 @@ static void qmp_output_start_struct(Visitor *v, const char *name, void **obj,
 static void qmp_output_end_struct(Visitor *v, Error **errp)
 {
     QmpOutputVisitor *qov = to_qov(v);
-    qmp_output_pop(qov);
+    QObject *value = qmp_output_pop(qov);
+    assert(qobject_type(value) == QTYPE_QDICT);
 }

 static void qmp_output_start_list(Visitor *v, const char *name, Error **errp)
@@ -145,7 +148,8 @@ static GenericList *qmp_output_next_list(Visitor *v, GenericList **listp,
 static void qmp_output_end_list(Visitor *v)
 {
     QmpOutputVisitor *qov = to_qov(v);
-    qmp_output_pop(qov);
+    QObject *value = qmp_output_pop(qov);
+    assert(qobject_type(value) == QTYPE_QLIST);
 }

 static void qmp_output_type_int64(Visitor *v, const char *name, int64_t *obj,
@@ -202,18 +206,15 @@ static void qmp_output_type_null(Visitor *v, const char *name, Error **errp)
     qmp_output_add_obj(qov, name, qnull());
 }

-/* Finish building, and return the root object. Will not be NULL. */
+/* Finish building, and return the root object. Will not be NULL.
+ * Caller must use qobject_decref() on the result.  */
 QObject *qmp_output_get_qobject(QmpOutputVisitor *qov)
 {
-    /* FIXME: we should require that a visit occurred, and that it is
-     * complete (no starts without a matching end) */
-    QObject *obj = qov->root;
-    if (obj) {
-        qobject_incref(obj);
-    } else {
-        obj = qnull();
-    }
-    return obj;
+    /* A visit must have occurred, with each start paired with end.  */
+    assert(qov->root && !QTAILQ_FIRST(&qov->stack));
+
+    qobject_incref(qov->root);
+    return qov->root;
 }

 Visitor *qmp_output_get_visitor(QmpOutputVisitor *v)
@@ -221,7 +222,7 @@ Visitor *qmp_output_get_visitor(QmpOutputVisitor *v)
     return &v->visitor;
 }

-void qmp_output_visitor_cleanup(QmpOutputVisitor *v)
+void qmp_output_visitor_reset(QmpOutputVisitor *v)
 {
     QStackEntry *e, *tmp;

@@ -231,6 +232,12 @@ void qmp_output_visitor_cleanup(QmpOutputVisitor *v)
     }

     qobject_decref(v->root);
+    v->root = NULL;
+}
+
+void qmp_output_visitor_cleanup(QmpOutputVisitor *v)
+{
+    qmp_output_visitor_reset(v);
     g_free(v);
 }

diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 4f7692b..e9ebce7 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -138,6 +138,7 @@ static void test_visitor_out_enum(TestOutputVisitorData *data,
         g_assert_cmpstr(qstring_get_str(qobject_to_qstring(obj)), ==,
                         EnumOne_lookup[i]);
         qobject_decref(obj);
+        qmp_output_visitor_reset(data->qov);
     }
 }

@@ -152,6 +153,7 @@ static void test_visitor_out_enum_errors(TestOutputVisitorData *data,
         visit_type_EnumOne(data->ov, "unused", &bad_values[i], &err);
         g_assert(err);
         error_free(err);
+        qmp_output_visitor_reset(data->qov);
     }
 }

@@ -261,6 +263,7 @@ static void test_visitor_out_struct_errors(TestOutputVisitorData *data,
         visit_type_UserDefOne(data->ov, "unused", &pu, &err);
         g_assert(err);
         error_free(err);
+        qmp_output_visitor_reset(data->qov);
     }
 }

@@ -365,6 +368,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
     qobject_decref(obj);
     qobject_decref(qobj);

+    qmp_output_visitor_reset(data->qov);
     qdict = qdict_new();
     qdict_put(qdict, "integer", qint_from_int(-42));
     qdict_put(qdict, "boolean", qbool_from_bool(true));
@@ -441,6 +445,7 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
     qapi_free_UserDefAlternate(tmp);
     qobject_decref(arg);

+    qmp_output_visitor_reset(data->qov);
     tmp = g_new0(UserDefAlternate, 1);
     tmp->type = QTYPE_QSTRING;
     tmp->u.s = g_strdup("hello");
@@ -454,6 +459,7 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
     qapi_free_UserDefAlternate(tmp);
     qobject_decref(arg);

+    qmp_output_visitor_reset(data->qov);
     tmp = g_new0(UserDefAlternate, 1);
     tmp->type = QTYPE_QDICT;
     tmp->u.udfu.integer = 1;
-- 
2.5.0

[Qemu-devel] [PATCH v12 13/18] qapi: Split visit_end_struct() into pieces

[Eric Blake]

As mentioned in previous patches, we want to call visit_end_struct()
functions unconditionally, so that visitors can release resources
tied up since the matching visit_start_struct() without also having
to worry about error priority if more than one error occurs.

Even though error_propagate() can be safely used to ignore a second
error during cleanup caused by a first error, it is simpler if the
cleanup cannot set an error.  So, split out the error checking
portion (basically, input visitors checking for unvisited keys) into
a new function visit_check_struct(), which can be safely skipped if
any earlier errors are encountered, and leave the cleanup portion
(which never fails, but must be called unconditionally if
visit_start_struct() succeeded) in visit_end_struct().

Generated code has diffs resembling:

|@@ -59,10 +59,12 @@ void visit_type_ACPIOSTInfo(Visitor *v,
|         goto out_obj;
|     }
|     visit_type_ACPIOSTInfo_members(v, obj, &err);
|-    error_propagate(errp, err);
|-    err = NULL;
|+    if (err) {
|+        goto out_obj;
|+    }
|+    visit_check_struct(v, &err);
| out_obj:
|-    visit_end_struct(v, &err);
|+    visit_end_struct(v);
| out:

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: rebase to earlier changes, fix bug in spapr_drc not calling
visit_end_struct, fold in docs, fix stray DO_UPCAST from sneaking
back in
[no v10, v11]
v9: rebase to earlier changes, drop Marc-Andre's R-b
v8: rebase to 'name' motion
v7: rebase to earlier changes
v6: new patch, revised version of RFC based on discussion of v5 7/46
---
 include/qapi/visitor.h         | 13 +++++++++++--
 include/qapi/visitor-impl.h    |  5 ++++-
 scripts/qapi-event.py          |  3 ++-
 scripts/qapi-visit.py          | 15 +++++++++------
 qapi/qapi-visit-core.c         | 11 +++++++++--
 hw/ppc/spapr_drc.c             |  3 ++-
 hw/virtio/virtio-balloon.c     | 15 ++++++++-------
 qapi/opts-visitor.c            | 17 +++++++++++++++--
 qapi/qapi-dealloc-visitor.c    |  2 +-
 qapi/qmp-input-visitor.c       | 34 +++++++++++++++++++---------------
 qapi/qmp-output-visitor.c      |  2 +-
 qom/object.c                   |  5 ++---
 qom/object_interfaces.c        | 16 +++++++---------
 tests/test-qmp-input-visitor.c |  3 ++-
 docs/qapi-code-gen.txt         |  8 +++++---
 15 files changed, 97 insertions(+), 55 deletions(-)

diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 2b85c6f..b3ca93c 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -239,17 +239,26 @@ void visit_start_struct(Visitor *v, const char *name, void **obj,
                         size_t size, Error **errp);

 /*
- * Complete an object visit started earlier.
+ * Prepare for completing an object visit.
  *
  * @errp must be NULL-initialized, and is set if an error is detected
  * (such as unparsed keys remaining in the input stream).
  *
+ * Should be called prior to visit_end_struct() if all other
+ * intermediate visit steps were successful, to allow the visitor one
+ * last chance to report errors.
+ */
+void visit_check_struct(Visitor *v, Error **errp);
+
+/*
+ * Complete an object visit started earlier.
+ *
  * Must be called after any successful use of visit_start_struct(),
  * even if intermediate processing was skipped due to errors, to allow
  * the backend to release any resources.  Destroying the visitor may
  * behave as if this was implicitly called.
  */
-void visit_end_struct(Visitor *v, Error **errp);
+void visit_end_struct(Visitor *v);


 /* === Visiting lists */
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 90bcaec..d44fcd1 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -39,8 +39,11 @@ struct Visitor
     void (*start_struct)(Visitor *v, const char *name, void **obj,
                          size_t size, Error **errp);

+    /* Optional; intended for input visitors.  */
+    void (*check_struct)(Visitor *v, Error **errp);
+
     /* Must be set to visit structs.  */
-    void (*end_struct)(Visitor *v, Error **errp);
+    void (*end_struct)(Visitor *v);

     /* Must be set.  */
     void (*start_list)(Visitor *v, const char *name, Error **errp);
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 808ed80..2604c3d 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -71,8 +71,9 @@ def gen_event_send(name, arg_type):
         ret += gen_visit_members(arg_type.members, need_cast=True,
                                  label='out_obj')
         ret += mcgen('''
+    visit_check_struct(v, &err);
 out_obj:
-    visit_end_struct(v, err ? NULL : &err);
+    visit_end_struct(v);
     if (err) {
         goto out;
     }
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index aa244cd..db756a0 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -188,9 +188,10 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
             break;
         }
         visit_type_%(c_type)s_members(v, &(*obj)->u.%(c_name)s, &err);
-        error_propagate(errp, err);
-        err = NULL;
-        visit_end_struct(v, &err);
+        if (!err) {
+            visit_check_struct(v, &err);
+        }
+        visit_end_struct(v);
 ''',
                          c_type=var.type.c_name(),
                          c_name=c_name(var.name))
@@ -240,10 +241,12 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
         goto out_obj;
     }
     visit_type_%(c_name)s_members(v, *obj, &err);
-    error_propagate(errp, err);
-    err = NULL;
+    if (err) {
+        goto out_obj;
+    }
+    visit_check_struct(v, &err);
 out_obj:
-    visit_end_struct(v, &err);
+    visit_end_struct(v);
 out:
     error_propagate(errp, err);
 }
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 116c627..c00a9e1 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -29,9 +29,16 @@ void visit_start_struct(Visitor *v, const char *name, void **obj,
     v->start_struct(v, name, obj, size, errp);
 }

-void visit_end_struct(Visitor *v, Error **errp)
+void visit_check_struct(Visitor *v, Error **errp)
 {
-    v->end_struct(v, errp);
+    if (v->check_struct) {
+        v->check_struct(v, errp);
+    }
+}
+
+void visit_end_struct(Visitor *v)
+{
+    v->end_struct(v);
 }

 void visit_start_list(Visitor *v, const char *name, Error **errp)
diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index b1bf193..77fe01a 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -288,7 +288,8 @@ static void prop_get_fdt(Object *obj, Visitor *v, const char *name,
         case FDT_END_NODE:
             /* shouldn't ever see an FDT_END_NODE before FDT_BEGIN_NODE */
             g_assert(fdt_depth > 0);
-            visit_end_struct(v, &err);
+            visit_check_struct(v, &err);
+            visit_end_struct(v);
             if (err) {
                 error_propagate(errp, err);
                 return;
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index e9c30e9..95cd95c 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -136,17 +136,18 @@ static void balloon_stats_get_all(Object *obj, Visitor *v, const char *name,
     for (i = 0; i < VIRTIO_BALLOON_S_NR; i++) {
         visit_type_uint64(v, balloon_stat_names[i], &s->stats[i], &err);
         if (err) {
-            break;
+            goto out_nested;
         }
     }
-    error_propagate(errp, err);
-    err = NULL;
-    visit_end_struct(v, &err);
+    visit_check_struct(v, &err);
+out_nested:
+    visit_end_struct(v);

+    if (!err) {
+        visit_check_struct(v, &err);
+    }
 out_end:
-    error_propagate(errp, err);
-    err = NULL;
-    visit_end_struct(v, &err);
+    visit_end_struct(v);
 out:
     error_propagate(errp, err);
 }
diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
index fd90608..f2fcfc3 100644
--- a/qapi/opts-visitor.c
+++ b/qapi/opts-visitor.c
@@ -158,13 +158,13 @@ opts_start_struct(Visitor *v, const char *name, void **obj,


 static void
-opts_end_struct(Visitor *v, Error **errp)
+opts_check_struct(Visitor *v, Error **errp)
 {
     OptsVisitor *ov = to_ov(v);
     GHashTableIter iter;
     GQueue *any;

-    if (--ov->depth > 0) {
+    if (ov->depth > 0) {
         return;
     }

@@ -176,6 +176,18 @@ opts_end_struct(Visitor *v, Error **errp)
         first = g_queue_peek_head(any);
         error_setg(errp, QERR_INVALID_PARAMETER, first->name);
     }
+}
+
+
+static void
+opts_end_struct(Visitor *v)
+{
+    OptsVisitor *ov = to_ov(v);
+
+    if (--ov->depth > 0) {
+        return;
+    }
+
     g_hash_table_destroy(ov->unprocessed_opts);
     ov->unprocessed_opts = NULL;
     if (ov->fake_id_opt) {
@@ -510,6 +522,7 @@ opts_visitor_new(const QemuOpts *opts)
     ov->visitor.type = VISITOR_INPUT;

     ov->visitor.start_struct = &opts_start_struct;
+    ov->visitor.check_struct = &opts_check_struct;
     ov->visitor.end_struct   = &opts_end_struct;

     ov->visitor.start_list = &opts_start_list;
diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c
index 413d525..9005bad 100644
--- a/qapi/qapi-dealloc-visitor.c
+++ b/qapi/qapi-dealloc-visitor.c
@@ -67,7 +67,7 @@ static void qapi_dealloc_start_struct(Visitor *v, const char *name, void **obj,
     qapi_dealloc_push(qov, obj);
 }

-static void qapi_dealloc_end_struct(Visitor *v, Error **errp)
+static void qapi_dealloc_end_struct(Visitor *v)
 {
     QapiDeallocVisitor *qov = to_qov(v);
     void **obj = qapi_dealloc_pop(qov);
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index faa40e7..6b69e9a 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -123,8 +123,10 @@ static void qmp_input_push(QmpInputVisitor *qiv, QObject *obj,
 }


-static void qmp_input_pop(QmpInputVisitor *qiv, Error **errp)
+static void qmp_input_check_struct(Visitor *v, Error **errp)
 {
+    QmpInputVisitor *qiv = to_qiv(v);
+
     assert(qiv->nb_stack > 0);

     if (qiv->strict) {
@@ -137,6 +139,19 @@ static void qmp_input_pop(QmpInputVisitor *qiv, Error **errp)
             if (g_hash_table_iter_next(&iter, (void **)&key, NULL)) {
                 error_setg(errp, QERR_QMP_EXTRA_MEMBER, key);
             }
+        }
+    }
+}
+
+static void qmp_input_pop(Visitor *v)
+{
+    QmpInputVisitor *qiv = to_qiv(v);
+
+    assert(qiv->nb_stack > 0);
+
+    if (qiv->strict) {
+        GHashTable * const top_ht = qiv->stack[qiv->nb_stack - 1].h;
+        if (top_ht) {
             g_hash_table_unref(top_ht);
         }
     }
@@ -171,12 +186,6 @@ static void qmp_input_start_struct(Visitor *v, const char *name, void **obj,
     }
 }

-static void qmp_input_end_struct(Visitor *v, Error **errp)
-{
-    QmpInputVisitor *qiv = to_qiv(v);
-
-    qmp_input_pop(qiv, errp);
-}

 static void qmp_input_start_list(Visitor *v, const char *name, Error **errp)
 {
@@ -216,12 +225,6 @@ static GenericList *qmp_input_next_list(Visitor *v, GenericList **list,
     return entry;
 }

-static void qmp_input_end_list(Visitor *v)
-{
-    QmpInputVisitor *qiv = to_qiv(v);
-
-    qmp_input_pop(qiv, &error_abort);
-}

 static void qmp_input_start_alternate(Visitor *v, const char *name,
                                       GenericAlternate **obj, size_t size,
@@ -381,10 +384,11 @@ QmpInputVisitor *qmp_input_visitor_new(QObject *obj)

     v->visitor.type = VISITOR_INPUT;
     v->visitor.start_struct = qmp_input_start_struct;
-    v->visitor.end_struct = qmp_input_end_struct;
+    v->visitor.check_struct = qmp_input_check_struct;
+    v->visitor.end_struct = qmp_input_pop;
     v->visitor.start_list = qmp_input_start_list;
     v->visitor.next_list = qmp_input_next_list;
-    v->visitor.end_list = qmp_input_end_list;
+    v->visitor.end_list = qmp_input_pop;
     v->visitor.start_alternate = qmp_input_start_alternate;
     v->visitor.type_int64 = qmp_input_type_int64;
     v->visitor.type_uint64 = qmp_input_type_uint64;
diff --git a/qapi/qmp-output-visitor.c b/qapi/qmp-output-visitor.c
index 7c48dfb..ecb2005 100644
--- a/qapi/qmp-output-visitor.c
+++ b/qapi/qmp-output-visitor.c
@@ -113,7 +113,7 @@ static void qmp_output_start_struct(Visitor *v, const char *name, void **obj,
     qmp_output_push(qov, dict);
 }

-static void qmp_output_end_struct(Visitor *v, Error **errp)
+static void qmp_output_end_struct(Visitor *v)
 {
     QmpOutputVisitor *qov = to_qov(v);
     QObject *value = qmp_output_pop(qov);
diff --git a/qom/object.c b/qom/object.c
index 844ae7a..858089c 100644
--- a/qom/object.c
+++ b/qom/object.c
@@ -2035,10 +2035,9 @@ static void property_get_tm(Object *obj, Visitor *v, const char *name,
     if (err) {
         goto out_end;
     }
+    visit_check_struct(v, &err);
 out_end:
-    error_propagate(errp, err);
-    err = NULL;
-    visit_end_struct(v, errp);
+    visit_end_struct(v);
 out:
     error_propagate(errp, err);

diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
index c2f6e29..f075b3a 100644
--- a/qom/object_interfaces.c
+++ b/qom/object_interfaces.c
@@ -41,7 +41,7 @@ Object *user_creatable_add(const QDict *qdict,
     char *type = NULL;
     char *id = NULL;
     Object *obj = NULL;
-    Error *local_err = NULL, *end_err = NULL;
+    Error *local_err = NULL;
     QDict *pdict;

     pdict = qdict_clone_shallow(qdict);
@@ -68,16 +68,14 @@ Object *user_creatable_add(const QDict *qdict,
         goto out_visit;
     }

- out_visit:
-    visit_end_struct(v, &end_err);
-    if (end_err) {
-        error_propagate(&local_err, end_err);
-        if (obj) {
-            user_creatable_del(id, NULL);
-        }
-        goto out;
+    visit_check_struct(v, &local_err);
+    if (local_err) {
+        user_creatable_del(id, NULL);
     }

+out_visit:
+    visit_end_struct(v);
+
 out:
     QDECREF(pdict);
     g_free(id);
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 46fa1b9..a62c2b1 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -289,7 +289,8 @@ static void test_visitor_in_null(TestInputVisitorData *data,
     v = visitor_input_test_init(data, "{ 'a': null }");
     visit_start_struct(v, NULL, NULL, 0, &error_abort);
     visit_type_null(v, "a", &error_abort);
-    visit_end_struct(v, &error_abort);
+    visit_check_struct(v, &error_abort);
+    visit_end_struct(v);
 }

 static void test_visitor_in_union_flat(TestInputVisitorData *data,
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 1c8f113..41d856c 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -898,10 +898,12 @@ Example:
             goto out_obj;
         }
         visit_type_UserDefOne_members(v, *obj, &err);
-        error_propagate(errp, err);
-        err = NULL;
+        if (err) {
+            goto out_obj;
+        }
+        visit_check_struct(v, &err);
     out_obj:
-        visit_end_struct(v, &err);
+        visit_end_struct(v);
     out:
         error_propagate(errp, err);
     }
-- 
2.5.0

[Qemu-devel] [PATCH v12 14/18] qapi-commands: Wrap argument visit in visit_start_struct

[Eric Blake]

The qmp-input visitor was playing rather fast and loose: when
visiting a QDict, you could grab members of the root dictionary
without first pushing into the dict.  But we are about to tighten
the input visitor, at which point the generated marshal code
MUST follow the same paradigms as everyone else, of pushing into
the struct before grabbing its keys, because the value of 'name'
should be ignored on the top-level visit.

It will also make things easier to follow when we introduce boxed
command parameters, where we call visit_type_FOO() instead of
individual members of FOO, since visit_type_FOO() also uses a
level of visit_start_struct() for handling the same QDict input.

Generated code grows as follows:

|@@ -63,18 +63,28 @@ void qmp_marshal_add_fd(QDict *args, QOb
|     char *opaque = NULL;
|
|     v = qmp_input_get_visitor(qiv);
|+    visit_start_struct(v, NULL, NULL, 0, &err);
|+    if (err) {
|+        goto out;
|+    }
|     if (visit_optional(v, "fdset-id", &has_fdset_id)) {
|         visit_type_int(v, "fdset-id", &fdset_id, &err);
|         if (err) {
|-            goto out;
|+            goto out_obj;
|         }
|     }
|     if (visit_optional(v, "opaque", &has_opaque)) {
|         visit_type_str(v, "opaque", &opaque, &err);
|         if (err) {
|-            goto out;
|+            goto out_obj;
|         }
|     }
|+    visit_check_struct(v, &err);
|+out_obj:
|+    visit_end_struct(v);
|+    if (err) {
|+        goto out;
|+    }
|
|     retval = qmp_add_fd(has_fdset_id, fdset_id, has_opaque, opaque, &err);

Note that this change could also make it possible for the
marshalling code to automatically detect excess input at the top
level, and not just in nested dictionaries.  However, that checking
is not currently useful (and we rely on the manual checking in
monitor.c:qmp_check_client_args() instead) as long as qmp-commands.hx
uses .args_type, and as long as we have 'name:O' as an arg-type that
explicitly allows unknown top-level keys because we haven't yet
converted 'device_add' and 'netdev_add' to introspectible use of
'any'.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: new patch
---
 scripts/qapi-commands.py | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 7c9cfbf..a586676 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -109,14 +109,28 @@ def gen_marshal_input_visit(arg_type, dealloc=False):
     else:
         ret += mcgen('''
     v = qmp_input_get_visitor(qiv);
+    visit_start_struct(v, NULL, NULL, 0, &err);
+    if (err) {
+        goto out;
+    }
 ''')

-    ret += gen_visit_members(arg_type.members, skiperr=dealloc)
+    ret += gen_visit_members(arg_type.members, skiperr=dealloc,
+                             label='out_obj')

     if dealloc:
         ret += mcgen('''
     qapi_dealloc_visitor_cleanup(qdv);
 ''')
+    else:
+        ret += mcgen('''
+    visit_check_struct(v, &err);
+out_obj:
+    visit_end_struct(v);
+    if (err) {
+        goto out;
+    }
+''')
     return ret


-- 
2.5.0

[Qemu-devel] [PATCH v12 15/18] qom: Wrap prop visit in visit_start_struct

[Eric Blake]

The qmp-input visitor was playing rather fast and loose: when
visiting a QDict, you could grab members of the root dictionary
without first pushing into the dict.  But we are about to tighten
the input visitor, at which point user_creatable_add_type() MUST
follow the same paradigms as everyone else, of pushing into the
struct before grabbing its keys, because the value of 'name'
should be ignored on the top-level visit.

The change has no impact to the testsuite now, but is required to
avoid a failure in tests/test-netfilter once qmp-input is made
stricter.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: new patch
---
 qom/object_interfaces.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
index f075b3a..8efb8d1 100644
--- a/qom/object_interfaces.c
+++ b/qom/object_interfaces.c
@@ -117,12 +117,23 @@ Object *user_creatable_add_type(const char *type, const char *id,

     obj = object_new(type);
     if (qdict) {
+        visit_start_struct(v, NULL, NULL, 0, &local_err);
+        if (local_err) {
+            goto out;
+        }
         for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
             object_property_set(obj, v, e->key, &local_err);
             if (local_err) {
-                goto out;
+                break;
             }
         }
+        if (!local_err) {
+            visit_check_struct(v, &local_err);
+        }
+        visit_end_struct(v);
+        if (local_err) {
+            goto out;
+        }
     }

     object_property_add_child(object_get_objects_root(),
-- 
2.5.0

[Qemu-devel] [PATCH v12 16/18] qmp-input: Require struct push to visit members of top dict

[Eric Blake]

Don't embed the root of the visit into the stack of current
containers being visited.  That way, we no longer get confused
on whether the first visit of a dictionary is to the dictionary
itself or to one of the members of the dictionary, and we no
longer have to require the root visit to pass name=NULL.

An audit of all qmp_input_visitor_new* call sites shows that
the only places where the visit starts directly on a QDict,
but where the first visit_type_* within the visit had passed
a non-NULL name, were fixed in the previous two places to
properly push into the object with visit_start_struct().

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: new patch
---
 include/qapi/visitor.h           |  3 +--
 include/qapi/qmp-input-visitor.h |  8 ------
 qapi/qmp-input-visitor.c         | 54 ++++++++++++++++++++++------------------
 3 files changed, 31 insertions(+), 34 deletions(-)

diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index b3ca93c..fb4582a 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -48,8 +48,7 @@
  *
  * The @name parameter of visit_type_FOO() describes the relation
  * between this QAPI value and its parent container.  When visiting
- * the root of a tree, @name is usually ignored (although some
- * visitors require it to be NULL); when visiting a member of an
+ * the root of a tree, @name is ignored; when visiting a member of an
  * object, @name is the key associated with the value; and when
  * visiting a member of a list, @name is NULL.
  *
diff --git a/include/qapi/qmp-input-visitor.h b/include/qapi/qmp-input-visitor.h
index d75ff98..3ed499c 100644
--- a/include/qapi/qmp-input-visitor.h
+++ b/include/qapi/qmp-input-visitor.h
@@ -19,14 +19,6 @@

 typedef struct QmpInputVisitor QmpInputVisitor;

-/*
- * FIXME: When visiting a QDict, passing a non-NULL @name for the
- * first visit_type_FOO() when the root is a QDict will find that
- * particular key within the QDict.  In the future, the contract may
- * be tightened to require visit_start_struct() with ignored @name as
- * the first visit; in the meantime, the first visit is safest when
- * using NULL for @name.
- */
 QmpInputVisitor *qmp_input_visitor_new(QObject *obj);
 QmpInputVisitor *qmp_input_visitor_new_strict(QObject *obj);

diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index 6b69e9a..2daf83f 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -38,9 +38,11 @@ struct QmpInputVisitor
 {
     Visitor visitor;

-    /* Stack of objects being visited.  stack[0] is root of visit,
-     * stack[1] and below correspond to visit_start_struct (nested
-     * QDict) and visit_start_list (nested QList).  */
+    /* Root of visit at visitor creation.  */
+    QObject *root;
+
+    /* Stack of objects being visited (all entries will be either
+     * QDict or QList).  */
     StackObject stack[QIV_STACK_SIZE];
     int nb_stack;

@@ -57,36 +59,40 @@ static QObject *qmp_input_get_object(QmpInputVisitor *qiv,
                                      const char *name,
                                      bool consume)
 {
-    StackObject *tos = &qiv->stack[qiv->nb_stack - 1];
-    QObject *qobj = tos->obj;
+    StackObject *tos;
+    QObject *qobj;

+    if (!qiv->nb_stack) {
+        /* Starting at root, name is ignored.  */
+        return qiv->root;
+    }
+
+    /* We are in a container; find the next element */
+    tos = &qiv->stack[qiv->nb_stack - 1];
+    qobj = tos->obj;
     assert(qobj);

-    /* If we have a name, and we're in a dictionary, then return that
-     * value. */
-    if (name && qobject_type(qobj) == QTYPE_QDICT) {
+    if (qobject_type(qobj) == QTYPE_QDICT) {
+        assert(name);
         qobj = qdict_get(qobject_to_qdict(qobj), name);
         if (tos->h && consume && qobj) {
             bool removed = g_hash_table_remove(tos->h, name);
             assert(removed);
         }
-        return qobj;
-    }
-
-    /* If we are in the middle of a list, then return the next element
-     * of the list.  */
-    if (tos->entry) {
+    } else {
         assert(qobject_type(qobj) == QTYPE_QLIST);
-        assert(!tos->first);
-        qobj = qlist_entry_obj(tos->entry);
-        if (consume) {
-            tos->entry = qlist_next(tos->entry);
+        /* FIXME: assertion needs adjustment if we fix visit-core
+         * to pass "name.0" style name during lists.  */
+        assert(!name);
+
+        if (tos->entry) {
+            assert(!tos->first);
+            qobj = qlist_entry_obj(tos->entry);
+            if (consume) {
+                tos->entry = qlist_next(tos->entry);
+            }
         }
-        return qobj;
     }
-
-    /* Otherwise, we are at the root of the visit or the start of a
-     * list, and return the object as-is.  */
     return qobj;
 }

@@ -372,7 +378,7 @@ Visitor *qmp_input_get_visitor(QmpInputVisitor *v)

 void qmp_input_visitor_cleanup(QmpInputVisitor *v)
 {
-    qobject_decref(v->stack[0].obj);
+    qobject_decref(v->root);
     g_free(v);
 }

@@ -399,7 +405,7 @@ QmpInputVisitor *qmp_input_visitor_new(QObject *obj)
     v->visitor.type_null = qmp_input_type_null;
     v->visitor.optional = qmp_input_optional;

-    qmp_input_push(v, obj, NULL, NULL);
+    v->root = obj;
     qobject_incref(obj);

     return v;
-- 
2.5.0

[Qemu-devel] [PATCH v12 17/18] qapi: Simplify semantics of visit_next_list()

[Eric Blake]

We have two uses of list visits in the entire code base: one in
spapr_drc (which completely avoids visit_next_list(), feeding in
integers from a different source than uint8List), and one in
qapi-visit.py (that is, all other list visitors are generated
in qapi-visit.c, and share the same paradigm based on a qapi
FooList type).  What's more, the semantics of the list visit are
somewhat baroque, with the following pseudocode when FooList is
used:

start()
for (prev = head; cur = next(prev); prev = &cur) {
    visit(&cur->value)
}

Note that these semantics (advance before visit) requires that
the first call to next() return the list head, while all other
calls return the next element of the list; that is, every visitor
implementation is required to track extra state to decide whether
to return the input as-is, or to advance.  It also requires an
argument of 'GenericList **' to next(), solely because the first
iteration might need to modify the caller's GenericList head, so
that all other calls have to do a layer of dereferencing.

We can greatly simplify things by hoisting the special case
into the start() routine, and flipping the order in the loop
to visit before advance:

start(head)
for (tail = *head; tail; tail = next(tail)) {
    visit(&tail->value)
}

With the simpler semantics, visitors have less state to track,
the argument to next() is reduced to 'GenericList *', and it
also becomes obvious whether an input visitor is allocating a
FooList during visit_start_list() (rather than the old way of
not knowing if an allocation happened until the first
visit_next_list()).

The signature of visit_start_list() is chosen to match
visit_start_struct(), with the new parameters after 'name'.

The spapr_drc case is a virtual visit, done by passing NULL for
list, similarly to how NULL is passed to visit_start_struct()
when a qapi type is not used in those visits.  It was easy to
provide these semantics for qmp-output and dealloc visitors,
and a bit harder for qmp-input (several prerequisite patches
refactored things to make this patch straightforward).  But it
turned out that the string and opts visitors munge enough other
state during visit_next_list() to make it easier to just
document and require a GenericList visit for now; an assertion
will remind us to adjust things if we need the semantics in the
future.

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: rebase to earlier changes, drop R-b, improve documentation,
split out qmp-input cleanups into prereq patches
[no v10, v11]
v9: no change
v8: consistent parameter order, fix qmp_input_get_next_type() to not
skip list entries
v7: new patch
---
 include/qapi/visitor.h               | 46 ++++++++++++++++++++----------------
 include/qapi/visitor-impl.h          |  7 +++---
 scripts/qapi-visit.py                | 18 +++++++-------
 include/qapi/opts-visitor.h          |  2 +-
 include/qapi/string-input-visitor.h  |  3 ++-
 include/qapi/string-output-visitor.h |  3 ++-
 qapi/qapi-visit-core.c               | 12 ++++++----
 hw/ppc/spapr_drc.c                   |  2 +-
 qapi/opts-visitor.c                  | 33 +++++++++++---------------
 qapi/qapi-dealloc-visitor.c          | 35 ++++++---------------------
 qapi/qmp-input-visitor.c             | 32 ++++++++++++-------------
 qapi/qmp-output-visitor.c            | 22 ++++-------------
 qapi/string-input-visitor.c          | 36 ++++++++++++++--------------
 qapi/string-output-visitor.c         | 41 +++++++++++---------------------
 docs/qapi-code-gen.txt               | 17 ++++++-------
 15 files changed, 133 insertions(+), 176 deletions(-)

diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index fb4582a..a81878d 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -166,7 +166,7 @@
  *  if (err) {
  *      goto outobj;
  *  }
- *  visit_start_list(v, "list", &err);
+ *  visit_start_list(v, "list", NULL, 0, &err);
  *  if (err) {
  *      goto outobj;
  *  }
@@ -268,39 +268,43 @@ void visit_end_struct(Visitor *v);
  * @name expresses the relationship of this list to its parent
  * container; see the general description of @name above.
  *
+ * @list must be non-NULL for a real walk, in which case @size
+ * determines how much memory an input visitor will allocate into
+ * *@list (at least sizeof(GenericList)).  Some visitors also allow
+ * @list to be NULL for a virtual walk, in which case @size is
+ * ignored.
+ *
  * @errp must be NULL-initialized, and is set if an error is detected
  * (such as if a member @name is not present, or is present but not a
- * list).
+ * list).  On error, input visitors set *@obj to NULL.
  *
  * After visit_start_list() succeeds, the caller may visit its members
- * one after the other.  A real visit uses visit_next_list() for
- * traversing the linked list, while a virtual visit uses other means.
- * For each list element, call the appropriate visit_type_FOO() with
- * name set to NULL and obj set to the address of the list element.
- * Finally, visit_end_list() needs to be called to clean up.  See the
- * examples above.
+ * one after the other.  A real visit (where @obj was non-NULL) uses
+ * visit_next_list() for traversing the linked list, while a virtual
+ * visit (where @obj was NULL) uses other means.  For each list
+ * element, call the appropriate visit_type_FOO() with name set to
+ * NULL and obj set to the address of the list element.  Finally,
+ * visit_end_list() needs to be called to clean up.  See the examples
+ * above.
  */
-void visit_start_list(Visitor *v, const char *name, Error **errp);
+void visit_start_list(Visitor *v, const char *name, GenericList **list,
+                      size_t size, Error **errp);

 /*
- * Iterate over a GenericList during a list visit.
+ * Iterate over a GenericList during a non-virtual list visit.
  *
- * @size represents the size of a linked list node.
+ * @size represents the size of a linked list node (at least
+ * sizeof(GenericList)).
  *
- * @list must not be NULL; on the first call, @list contains the
- * address of the list head, and on subsequent calls *@list must be
- * the previously returned value.  Must be called in a loop until a
+ * @tail must not be NULL; on the first call, @tail is the value of
+ * *list after visit_start_list(), and on subsequent calls @tail must
+ * be the previously returned value.  Must be called in a loop until a
  * NULL return or error occurs; for each non-NULL return, the caller
  * must then call the appropriate visit_type_*() for the element type
  * of the list, with that function's name parameter set to NULL and
- * obj set to the address of (*@list)->value.
- *
- * FIXME: This interface is awkward; it requires all callbacks to
- * track whether it is the first or a subsequent call.  A better
- * interface would pass the head of the list through
- * visit_start_list().
+ * obj set to the address of @tail->value.
  */
-GenericList *visit_next_list(Visitor *v, GenericList **list, size_t size);
+GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size);

 /*
  * Complete a list visit started earlier.
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index d44fcd1..0471465 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -45,11 +45,12 @@ struct Visitor
     /* Must be set to visit structs.  */
     void (*end_struct)(Visitor *v);

-    /* Must be set.  */
-    void (*start_list)(Visitor *v, const char *name, Error **errp);
+    /* Must be set; document if @list may not be NULL.  */
+    void (*start_list)(Visitor *v, const char *name, GenericList **list,
+                       size_t size, Error **errp);

     /* Must be set.  */
-    GenericList *(*next_list)(Visitor *v, GenericList **list, size_t size);
+    GenericList *(*next_list)(Visitor *v, GenericList *tail, size_t size);

     /* Must be set.  */
     void (*end_list)(Visitor *v);
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index db756a0..5921bad 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -119,20 +119,20 @@ def gen_visit_list(name, element_type):
 void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error **errp)
 {
     Error *err = NULL;
-    GenericList *i, **prev;
+    %(c_name)s *tail;
+    size_t size = sizeof(**obj);

-    visit_start_list(v, name, &err);
+    visit_start_list(v, name, (GenericList **)obj, size, &err);
     if (err) {
         goto out;
     }
-
-    for (prev = (GenericList **)obj;
-         !err && (i = visit_next_list(v, prev, sizeof(**obj))) != NULL;
-         prev = &i) {
-        %(c_name)s *native_i = (%(c_name)s *)i;
-        visit_type_%(c_elt_type)s(v, NULL, &native_i->value, &err);
+    for (tail = *obj; tail;
+         tail = (%(c_name)s *)visit_next_list(v, (GenericList *)tail, size)) {
+        visit_type_%(c_elt_type)s(v, NULL, &tail->value, &err);
+        if (err) {
+            break;
+        }
     }
-
     visit_end_list(v);
 out:
     error_propagate(errp, err);
diff --git a/include/qapi/opts-visitor.h b/include/qapi/opts-visitor.h
index 3fcd327..7c9a7e5 100644
--- a/include/qapi/opts-visitor.h
+++ b/include/qapi/opts-visitor.h
@@ -32,7 +32,7 @@ typedef struct OptsVisitor OptsVisitor;
  *
  * The Opts input visitor does not yet implement support for visiting
  * QAPI alternates, numbers (other than integers), null, or arbitrary
- * QTypes.
+ * QTypes.  It also requires non-NULL list to visit_start_list().
  */
 OptsVisitor *opts_visitor_new(const QemuOpts *opts);
 void opts_visitor_cleanup(OptsVisitor *nv);
diff --git a/include/qapi/string-input-visitor.h b/include/qapi/string-input-visitor.h
index 1a34c52..d26a845 100644
--- a/include/qapi/string-input-visitor.h
+++ b/include/qapi/string-input-visitor.h
@@ -19,7 +19,8 @@ typedef struct StringInputVisitor StringInputVisitor;

 /*
  * The string input visitor does not yet implement support for
- * visiting QAPI structs, alternates, null, or arbitrary QTypes.
+ * visiting QAPI structs, alternates, null, or arbitrary QTypes.  It
+ * also requires non-NULL list to visit_start_list().
  */
 StringInputVisitor *string_input_visitor_new(const char *str);
 void string_input_visitor_cleanup(StringInputVisitor *v);
diff --git a/include/qapi/string-output-visitor.h b/include/qapi/string-output-visitor.h
index 2564833..0a9354c 100644
--- a/include/qapi/string-output-visitor.h
+++ b/include/qapi/string-output-visitor.h
@@ -19,7 +19,8 @@ typedef struct StringOutputVisitor StringOutputVisitor;

 /*
  * The string output visitor does not yet implement support for
- * visiting QAPI structs, alternates, null, or arbitrary QTypes.
+ * visiting QAPI structs, alternates, null, or arbitrary QTypes.  It
+ * also requires non-NULL list to visit_start_list().
  */
 StringOutputVisitor *string_output_visitor_new(bool human);
 void string_output_visitor_cleanup(StringOutputVisitor *v);
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index c00a9e1..bbcedb1 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -41,15 +41,17 @@ void visit_end_struct(Visitor *v)
     v->end_struct(v);
 }

-void visit_start_list(Visitor *v, const char *name, Error **errp)
+void visit_start_list(Visitor *v, const char *name, GenericList **list,
+                      size_t size, Error **errp)
 {
-    v->start_list(v, name, errp);
+    assert(!list || size >= sizeof(GenericList));
+    v->start_list(v, name, list, size, errp);
 }

-GenericList *visit_next_list(Visitor *v, GenericList **list, size_t size)
+GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size)
 {
-    assert(list && size >= sizeof(GenericList));
-    return v->next_list(v, list, size);
+    assert(tail && size >= sizeof(GenericList));
+    return v->next_list(v, tail, size);
 }

 void visit_end_list(Visitor *v)
diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index 77fe01a..4f42449 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -300,7 +300,7 @@ static void prop_get_fdt(Object *obj, Visitor *v, const char *name,
             int i;
             prop = fdt_get_property_by_offset(fdt, fdt_offset, &prop_len);
             name = fdt_string(fdt, fdt32_to_cpu(prop->nameoff));
-            visit_start_list(v, name, &err);
+            visit_start_list(v, name, NULL, 0, &err);
             if (err) {
                 error_propagate(errp, err);
                 return;
diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
index f2fcfc3..525805e 100644
--- a/qapi/opts-visitor.c
+++ b/qapi/opts-visitor.c
@@ -22,9 +22,8 @@
 enum ListMode
 {
     LM_NONE,             /* not traversing a list of repeated options */
-    LM_STARTED,          /* opts_start_list() succeeded */

-    LM_IN_PROGRESS,      /* opts_next_list() has been called.
+    LM_IN_PROGRESS,      /* opts_next_list() ready to be called.
                           *
                           * Generating the next list link will consume the most
                           * recently parsed QemuOpt instance of the repeated
@@ -213,35 +212,33 @@ lookup_distinct(const OptsVisitor *ov, const char *name, Error **errp)


 static void
-opts_start_list(Visitor *v, const char *name, Error **errp)
+opts_start_list(Visitor *v, const char *name, GenericList **list, size_t size,
+                Error **errp)
 {
     OptsVisitor *ov = to_ov(v);

     /* we can't traverse a list in a list */
     assert(ov->list_mode == LM_NONE);
+    /* we don't support visits without GenericList, yet */
+    assert(list);
     ov->repeated_opts = lookup_distinct(ov, name, errp);
-    if (ov->repeated_opts != NULL) {
-        ov->list_mode = LM_STARTED;
+    if (ov->repeated_opts) {
+        ov->list_mode = LM_IN_PROGRESS;
+        *list = g_malloc0(size);
+    } else {
+        *list = NULL;
     }
 }


 static GenericList *
-opts_next_list(Visitor *v, GenericList **list, size_t size)
+opts_next_list(Visitor *v, GenericList *tail, size_t size)
 {
     OptsVisitor *ov = to_ov(v);
-    GenericList **link;

     switch (ov->list_mode) {
-    case LM_STARTED:
-        ov->list_mode = LM_IN_PROGRESS;
-        link = list;
-        break;
-
     case LM_SIGNED_INTERVAL:
     case LM_UNSIGNED_INTERVAL:
-        link = &(*list)->next;
-
         if (ov->list_mode == LM_SIGNED_INTERVAL) {
             if (ov->range_next.s < ov->range_limit.s) {
                 ++ov->range_next.s;
@@ -262,7 +259,6 @@ opts_next_list(Visitor *v, GenericList **list, size_t size)
             g_hash_table_remove(ov->unprocessed_opts, opt->name);
             return NULL;
         }
-        link = &(*list)->next;
         break;
     }

@@ -270,8 +266,8 @@ opts_next_list(Visitor *v, GenericList **list, size_t size)
         abort();
     }

-    *link = g_malloc0(size);
-    return *link;
+    tail->next = g_malloc0(size);
+    return tail->next;
 }


@@ -280,8 +276,7 @@ opts_end_list(Visitor *v)
 {
     OptsVisitor *ov = to_ov(v);

-    assert(ov->list_mode == LM_STARTED ||
-           ov->list_mode == LM_IN_PROGRESS ||
+    assert(ov->list_mode == LM_IN_PROGRESS ||
            ov->list_mode == LM_SIGNED_INTERVAL ||
            ov->list_mode == LM_UNSIGNED_INTERVAL);
     ov->repeated_opts = NULL;
diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c
index 9005bad..cd68b55 100644
--- a/qapi/qapi-dealloc-visitor.c
+++ b/qapi/qapi-dealloc-visitor.c
@@ -22,7 +22,6 @@
 typedef struct StackEntry
 {
     void *value;
-    bool is_list_head;
     QTAILQ_ENTRY(StackEntry) node;
 } StackEntry;

@@ -43,10 +42,6 @@ static void qapi_dealloc_push(QapiDeallocVisitor *qov, void *value)

     e->value = value;

-    /* see if we're just pushing a list head tracker */
-    if (value == NULL) {
-        e->is_list_head = true;
-    }
     QTAILQ_INSERT_HEAD(&qov->stack, e, node);
 }

@@ -93,38 +88,22 @@ static void qapi_dealloc_end_alternate(Visitor *v)
     }
 }

-static void qapi_dealloc_start_list(Visitor *v, const char *name, Error **errp)
+static void qapi_dealloc_start_list(Visitor *v, const char *name,
+                                    GenericList **list, size_t size,
+                                    Error **errp)
 {
-    QapiDeallocVisitor *qov = to_qov(v);
-    qapi_dealloc_push(qov, NULL);
 }

-static GenericList *qapi_dealloc_next_list(Visitor *v, GenericList **listp,
+static GenericList *qapi_dealloc_next_list(Visitor *v, GenericList *tail,
                                            size_t size)
 {
-    GenericList *list = *listp;
-    QapiDeallocVisitor *qov = to_qov(v);
-    StackEntry *e = QTAILQ_FIRST(&qov->stack);
-
-    if (e && e->is_list_head) {
-        e->is_list_head = false;
-        return list;
-    }
-
-    if (list) {
-        list = list->next;
-        g_free(*listp);
-        return list;
-    }
-
-    return NULL;
+    GenericList *next = tail->next;
+    g_free(tail);
+    return next;
 }

 static void qapi_dealloc_end_list(Visitor *v)
 {
-    QapiDeallocVisitor *qov = to_qov(v);
-    void *obj = qapi_dealloc_pop(qov);
-    assert(obj == NULL); /* should've been list head tracker with no payload */
 }

 static void qapi_dealloc_type_str(Visitor *v, const char *name, char **obj,
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index 2daf83f..e1f4085 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -28,8 +28,6 @@ typedef struct StackObject

     /* If obj is list: tail of list still needing visits */
     const QListEntry *entry;
-    /* If obj is list: true if head is not visited yet */
-    bool first;

     GHashTable *h; /* If obj is dict: remaining keys needing visits */
 } StackObject;
@@ -86,7 +84,6 @@ static QObject *qmp_input_get_object(QmpInputVisitor *qiv,
         assert(!name);

         if (tos->entry) {
-            assert(!tos->first);
             qobj = qlist_entry_obj(tos->entry);
             if (consume) {
                 tos->entry = qlist_next(tos->entry);
@@ -116,7 +113,6 @@ static void qmp_input_push(QmpInputVisitor *qiv, QObject *obj,

     tos->obj = obj;
     tos->entry = entry;
-    tos->first = true;
     tos->h = NULL;

     if (qiv->strict && qobject_type(obj) == QTYPE_QDICT) {
@@ -193,13 +189,17 @@ static void qmp_input_start_struct(Visitor *v, const char *name, void **obj,
 }


-static void qmp_input_start_list(Visitor *v, const char *name, Error **errp)
+static void qmp_input_start_list(Visitor *v, const char *name,
+                                 GenericList **list, size_t size, Error **errp)
 {
     QmpInputVisitor *qiv = to_qiv(v);
     QObject *qobj = qmp_input_get_object(qiv, name, true);
     const QListEntry *entry;

     if (!qobj || qobject_type(qobj) != QTYPE_QLIST) {
+        if (list) {
+            *list = NULL;
+        }
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
                    "list");
         return;
@@ -207,28 +207,26 @@ static void qmp_input_start_list(Visitor *v, const char *name, Error **errp)

     entry = qlist_first(qobject_to_qlist(qobj));
     qmp_input_push(qiv, qobj, entry, errp);
+    if (list) {
+        if (entry) {
+            *list = g_malloc0(size);
+        } else {
+            *list = NULL;
+        }
+    }
 }

-static GenericList *qmp_input_next_list(Visitor *v, GenericList **list,
+static GenericList *qmp_input_next_list(Visitor *v, GenericList *tail,
                                         size_t size)
 {
     QmpInputVisitor *qiv = to_qiv(v);
-    GenericList *entry;
     StackObject *so = &qiv->stack[qiv->nb_stack - 1];

     if (!so->entry) {
         return NULL;
     }
-
-    entry = g_malloc0(size);
-    if (so->first) {
-        *list = entry;
-        so->first = false;
-    } else {
-        (*list)->next = entry;
-    }
-
-    return entry;
+    tail->next = g_malloc0(size);
+    return tail->next;
 }


diff --git a/qapi/qmp-output-visitor.c b/qapi/qmp-output-visitor.c
index ecb2005..40657be 100644
--- a/qapi/qmp-output-visitor.c
+++ b/qapi/qmp-output-visitor.c
@@ -22,7 +22,6 @@
 typedef struct QStackEntry
 {
     QObject *value;
-    bool is_list_head;
     QTAILQ_ENTRY(QStackEntry) node;
 } QStackEntry;

@@ -52,9 +51,6 @@ static void qmp_output_push_obj(QmpOutputVisitor *qov, QObject *value)
     assert(qov->root);
     assert(value);
     e->value = value;
-    if (qobject_type(e->value) == QTYPE_QLIST) {
-        e->is_list_head = true;
-    }
     QTAILQ_INSERT_HEAD(&qov->stack, e, node);
 }

@@ -120,7 +116,9 @@ static void qmp_output_end_struct(Visitor *v)
     assert(qobject_type(value) == QTYPE_QDICT);
 }

-static void qmp_output_start_list(Visitor *v, const char *name, Error **errp)
+static void qmp_output_start_list(Visitor *v, const char *name,
+                                  GenericList **listp, size_t size,
+                                  Error **errp)
 {
     QmpOutputVisitor *qov = to_qov(v);
     QList *list = qlist_new();
@@ -129,20 +127,10 @@ static void qmp_output_start_list(Visitor *v, const char *name, Error **errp)
     qmp_output_push(qov, list);
 }

-static GenericList *qmp_output_next_list(Visitor *v, GenericList **listp,
+static GenericList *qmp_output_next_list(Visitor *v, GenericList *tail,
                                          size_t size)
 {
-    GenericList *list = *listp;
-    QmpOutputVisitor *qov = to_qov(v);
-    QStackEntry *e = QTAILQ_FIRST(&qov->stack);
-
-    assert(e);
-    if (e->is_list_head) {
-        e->is_list_head = false;
-        return list;
-    }
-
-    return list ? list->next : NULL;
+    return tail->next;
 }

 static void qmp_output_end_list(Visitor *v)
diff --git a/qapi/string-input-visitor.c b/qapi/string-input-visitor.c
index 4087bf9..a514526 100644
--- a/qapi/string-input-visitor.c
+++ b/qapi/string-input-visitor.c
@@ -24,8 +24,6 @@ struct StringInputVisitor
 {
     Visitor visitor;

-    bool head;
-
     GList *ranges;
     GList *cur_range;
     int64_t cur;
@@ -124,11 +122,21 @@ error:
 }

 static void
-start_list(Visitor *v, const char *name, Error **errp)
+start_list(Visitor *v, const char *name, GenericList **list, size_t size,
+           Error **errp)
 {
     StringInputVisitor *siv = to_siv(v);
+    Error *err = NULL;

-    parse_str(siv, errp);
+    /* We don't support visits without a GenericList, yet */
+    assert(list);
+
+    parse_str(siv, &err);
+    if (err) {
+        *list = NULL;
+        error_propagate(errp, err);
+        return;
+    }

     siv->cur_range = g_list_first(siv->ranges);
     if (siv->cur_range) {
@@ -136,13 +144,15 @@ start_list(Visitor *v, const char *name, Error **errp)
         if (r) {
             siv->cur = r->begin;
         }
+        *list = g_malloc0(size);
+    } else {
+        *list = NULL;
     }
 }

-static GenericList *next_list(Visitor *v, GenericList **list, size_t size)
+static GenericList *next_list(Visitor *v, GenericList *tail, size_t size)
 {
     StringInputVisitor *siv = to_siv(v);
-    GenericList **link;
     Range *r;

     if (!siv->ranges || !siv->cur_range) {
@@ -166,21 +176,12 @@ static GenericList *next_list(Visitor *v, GenericList **list, size_t size)
         siv->cur = r->begin;
     }

-    if (siv->head) {
-        link = list;
-        siv->head = false;
-    } else {
-        link = &(*list)->next;
-    }
-
-    *link = g_malloc0(size);
-    return *link;
+    tail->next = g_malloc0(size);
+    return tail->next;
 }

 static void end_list(Visitor *v)
 {
-    StringInputVisitor *siv = to_siv(v);
-    siv->head = true;
 }

 static void parse_type_int64(Visitor *v, const char *name, int64_t *obj,
@@ -361,6 +362,5 @@ StringInputVisitor *string_input_visitor_new(const char *str)
     v->visitor.optional = parse_optional;

     v->string = str;
-    v->head = true;
     return v;
 }
diff --git a/qapi/string-output-visitor.c b/qapi/string-output-visitor.c
index 0d44d7e..e27e2df 100644
--- a/qapi/string-output-visitor.c
+++ b/qapi/string-output-visitor.c
@@ -20,7 +20,7 @@

 enum ListMode {
     LM_NONE,             /* not traversing a list of repeated options */
-    LM_STARTED,          /* start_list() succeeded */
+    LM_STARTED,          /* start_list() succeeded with multiple elements */

     LM_IN_PROGRESS,      /* next_list() has been called.
                           *
@@ -48,7 +48,7 @@ enum ListMode {

     LM_UNSIGNED_INTERVAL,/* Same as above, only for an unsigned interval. */

-    LM_END
+    LM_END,              /* next_list() called, about to see last element. */
 };

 typedef enum ListMode ListMode;
@@ -58,7 +58,6 @@ struct StringOutputVisitor
     Visitor visitor;
     bool human;
     GString *string;
-    bool head;
     ListMode list_mode;
     union {
         int64_t s;
@@ -266,39 +265,29 @@ static void print_type_number(Visitor *v, const char *name, double *obj,
 }

 static void
-start_list(Visitor *v, const char *name, Error **errp)
+start_list(Visitor *v, const char *name, GenericList **list, size_t size,
+           Error **errp)
 {
     StringOutputVisitor *sov = to_sov(v);

     /* we can't traverse a list in a list */
     assert(sov->list_mode == LM_NONE);
-    sov->list_mode = LM_STARTED;
-    sov->head = true;
+    /* We don't support visits without a GenericList, yet */
+    assert(list);
+    /* List handling is only needed if there are at least two elements */
+    if (*list && (*list)->next) {
+        sov->list_mode = LM_STARTED;
+    }
 }

-static GenericList *next_list(Visitor *v, GenericList **list, size_t size)
+static GenericList *next_list(Visitor *v, GenericList *tail, size_t size)
 {
     StringOutputVisitor *sov = to_sov(v);
-    GenericList *ret = NULL;
-    if (*list) {
-        if (sov->head) {
-            ret = *list;
-        } else {
-            ret = (*list)->next;
-        }
+    GenericList *ret = tail->next;

-        if (sov->head) {
-            if (ret && ret->next == NULL) {
-                sov->list_mode = LM_NONE;
-            }
-            sov->head = false;
-        } else {
-            if (ret && ret->next == NULL) {
-                sov->list_mode = LM_END;
-            }
-        }
+    if (ret && !ret->next) {
+        sov->list_mode = LM_END;
     }
-
     return ret;
 }

@@ -311,8 +300,6 @@ static void end_list(Visitor *v)
            sov->list_mode == LM_NONE ||
            sov->list_mode == LM_IN_PROGRESS);
     sov->list_mode = LM_NONE;
-    sov->head = true;
-
 }

 char *string_output_get_string(StringOutputVisitor *sov)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 41d856c..06f22b7 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -911,18 +911,19 @@ Example:
     void visit_type_UserDefOneList(Visitor *v, const char *name, UserDefOneList **obj, Error **errp)
     {
         Error *err = NULL;
-        GenericList *i, **prev;
+        UserDefOneList *tail;
+        size_t size = sizeof(**obj);

-        visit_start_list(v, name, &err);
+        visit_start_list(v, name, (GenericList **)obj, size, &err);
         if (err) {
             goto out;
         }
-
-        for (prev = (GenericList **)obj;
-             !err && (i = visit_next_list(v, prev, sizeof(**obj))) != NULL;
-             prev = &i) {
-            UserDefOneList *native_i = (UserDefOneList *)i;
-            visit_type_UserDefOne(v, NULL, &native_i->value, &err);
+        for (tail = *obj; tail;
+             tail = (UserDefOneList *)visit_next_list(v, (GenericList *)tail, size)) {
+            visit_type_UserDefOne(v, NULL, &tail->value, &err);
+            if (err) {
+                break;
+            }
         }

         visit_end_list(v);
-- 
2.5.0

[Qemu-devel] [PATCH v12 18/18] qapi: Change visit_type_FOO() to no longer return partial objects

[Eric Blake]

Returning a partial object on error is an invitation for a careless
caller to leak memory.  As no one outside the testsuite was actually
relying on these semantics, it is cleaner to just document and
guarantee that ALL pointer-based visit_type_FOO() functions always
leave a safe value in *obj during an input visitor (either the new
object on success, or NULL if an error is encountered), so callers
can now unconditionally use qapi_free_FOO() to clean up regardless
of whether an error occurred.

The decision is done by enhancing qapi-visit-core to return true
for input visitors (the callbacks themselves do not need
modification); since we've documented that visit_end_* must be
called after any successful visit_start_*, that is a sufficient
point for knowing that something was allocated during start.

Note that we still leave *obj unchanged after a scalar-based
visit_type_FOO(); I did not feel like auditing all uses of
visit_type_Enum() to see if the callers would tolerate a specific
sentinel value (not to mention having to decide whether it would
be better to use 0 or ENUM__MAX as that sentinel).

Signed-off-by: Eric Blake <eblake@redhat.com>

---
v12: rebase to latest, don't modify callback signatures, use newer
approach for detecting input visitors, avoid 'allocated' boolean
[no v10, v11]
v9: fix bug in use of errp
v8: rebase to earlier changes
v7: rebase to earlier changes, enhance commit message, also fix
visit_type_str() and visit_type_any()
v6: rebase on top of earlier doc and formatting improvements, mention
that *obj can be uninitialized on entry to an input visitor, rework
semantics to keep valgrind happy on uninitialized input, break some
long lines
---
 include/qapi/visitor.h         | 40 ++++++++++++++++++++++++++++++----------
 include/qapi/visitor-impl.h    |  8 +++++---
 scripts/qapi-visit.py          | 25 +++++++++++++------------
 qapi/qapi-visit-core.c         | 41 ++++++++++++++++++++++++++++++++++-------
 tests/test-qmp-commands.c      | 13 ++++++-------
 tests/test-qmp-input-strict.c  | 19 ++++++++-----------
 tests/test-qmp-input-visitor.c | 10 ++--------
 docs/qapi-code-gen.txt         | 10 ++++++++--
 8 files changed, 106 insertions(+), 60 deletions(-)

diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index a81878d..f676991 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -66,14 +66,16 @@
  * }' if an error is encountered on "value" (or to have the visitor
  * core auto-generate the nicer name).
  *
- * FIXME: At present, input visitors may allocate an incomplete *@obj
- * even when visit_type_FOO() reports an error.  Using an output
- * visitor with an incomplete object has undefined behavior; callers
- * must call qapi_free_FOO() (which uses the dealloc visitor, and
- * safely handles an incomplete object) to avoid a memory leak.
+ * If an error is detected during visit_type_FOO() with an input
+ * visitor, then *@obj will be NULL for pointer types, and left
+ * unchanged for scalar types.  Using an output visitor with an
+ * incomplete object has undefined behavior (other than a special case
+ * for visit_type_str() treating NULL like ""), while the dealloc
+ * visitor safely handles incomplete objects.
  *
- * Likewise, the QAPI object types (structs, unions, and alternates)
- * have a generated function in qapi-visit.h compatible with:
+ * Additionally, the QAPI object types (structs, unions, and
+ * alternates) have a generated function in qapi-visit.h compatible
+ * with:
  *
  * void visit_type_FOO_members(Visitor *v, FOO *obj, Error **errp);
  *
@@ -256,8 +258,14 @@ void visit_check_struct(Visitor *v, Error **errp);
  * even if intermediate processing was skipped due to errors, to allow
  * the backend to release any resources.  Destroying the visitor may
  * behave as if this was implicitly called.
+ *
+ * Returns true if this is an input visitor (that is, an allocation
+ * occurred during visit_start_struct() if obj was non-NULL).  The
+ * caller can use this, along with tracking whether a local error
+ * occurred in the meantime, to decide when to undo allocation before
+ * returning control from a visit_type_FOO() function.
  */
-void visit_end_struct(Visitor *v);
+bool visit_end_struct(Visitor *v);


 /* === Visiting lists */
@@ -313,8 +321,14 @@ GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size);
  * even if intermediate processing was skipped due to errors, to allow
  * the backend to release any resources.  Destroying the visitor may
  * behave as if this was implicitly called.
+ *
+ * Returns true if this is an input visitor (that is, an allocation
+ * occurred during visit_start_list() if list was non-NULL).  The
+ * caller can use this, along with tracking whether a local error
+ * occurred in the meantime, to decide when to undo allocation before
+ * returning control from a visit_type_FOO() function.
  */
-void visit_end_list(Visitor *v);
+bool visit_end_list(Visitor *v);


 /* === Visiting alternates */
@@ -347,10 +361,16 @@ void visit_start_alternate(Visitor *v, const char *name,
  * the backend to release any resources.  Destroying the visitor may
  * behave as if this was implicitly called.
  *
+ * Returns true if this is an input visitor (that is, an allocation
+ * occurred during visit_start_alternate() if obj was non-NULL).  The
+ * caller can use this, along with tracking whether a local error
+ * occurred in the meantime, to decide when to undo allocation before
+ * returning control from a visit_type_FOO() function.
+ *
  * TODO: Should all the visit_end_* interfaces take obj parameter, so
  * that dealloc visitor need not track what was passed in visit_start?
  */
-void visit_end_alternate(Visitor *v);
+bool visit_end_alternate(Visitor *v);


 /* === Other helpers */
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 0471465..f113869 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -42,7 +42,8 @@ struct Visitor
     /* Optional; intended for input visitors.  */
     void (*check_struct)(Visitor *v, Error **errp);

-    /* Must be set to visit structs.  */
+    /* Must be set to visit structs.  The core takes care of the
+     * return value.  */
     void (*end_struct)(Visitor *v);

     /* Must be set; document if @list may not be NULL.  */
@@ -52,7 +53,7 @@ struct Visitor
     /* Must be set.  */
     GenericList *(*next_list)(Visitor *v, GenericList *tail, size_t size);

-    /* Must be set.  */
+    /* Must be set.  The core takes care of the return value.  */
     void (*end_list)(Visitor *v);

     /* Must be set by input and dealloc visitors to visit alternates;
@@ -61,7 +62,8 @@ struct Visitor
                             GenericAlternate **obj, size_t size,
                             bool promote_int, Error **errp);

-    /* Optional, needed for dealloc visitor.  */
+    /* Optional, needed for dealloc visitor.  The core takes care of
+     * the return value.  */
     void (*end_alternate)(Visitor *v);

     /* Must be set.  */
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 5921bad..fab00b9 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -110,10 +110,6 @@ out:


 def gen_visit_list(name, element_type):
-    # FIXME: if *obj is NULL on entry, and the first visit_next_list()
-    # assigns to *obj, while a later one fails, we should clean up *obj
-    # rather than leaving it non-NULL. As currently written, the caller must
-    # call qapi_free_FOOList() to avoid a memory leak of the partial FOOList.
     return mcgen('''

 void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error **errp)
@@ -133,7 +129,10 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
             break;
         }
     }
-    visit_end_list(v);
+    if (visit_end_list(v) && err) {
+        qapi_free_%(c_name)s(*obj);
+        *obj = NULL;
+    }
 out:
     error_propagate(errp, err);
 }
@@ -210,12 +209,15 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
         error_setg(&err, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
                    "%(name)s");
     }
-    visit_end_alternate(v);
+    if (visit_end_alternate(v) && err) {
+        qapi_free_%(c_name)s(*obj);
+        *obj = NULL;
+    }
 out:
     error_propagate(errp, err);
 }
 ''',
-                 name=name)
+                 name=name, c_name=c_name(name))

     return ret

@@ -223,10 +225,6 @@ out:
 def gen_visit_object(name, base, members, variants):
     ret = gen_visit_object_members(name, base, members, variants)

-    # FIXME: if *obj is NULL on entry, and visit_start_struct() assigns to
-    # *obj, but then visit_type_FOO_members() fails, we should clean up *obj
-    # rather than leaving it non-NULL. As currently written, the caller must
-    # call qapi_free_FOO() to avoid a memory leak of the partial FOO.
     ret += mcgen('''

 void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error **errp)
@@ -246,7 +244,10 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
     }
     visit_check_struct(v, &err);
 out_obj:
-    visit_end_struct(v);
+    if (visit_end_struct(v) && err) {
+        qapi_free_%(c_name)s(*obj);
+        *obj = NULL;
+    }
 out:
     error_propagate(errp, err);
 }
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index bbcedb1..7c36b24 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -22,11 +22,17 @@
 void visit_start_struct(Visitor *v, const char *name, void **obj,
                         size_t size, Error **errp)
 {
+    Error *err = NULL;
+
     if (obj) {
         assert(size);
         assert(v->type != VISITOR_OUTPUT || *obj);
     }
-    v->start_struct(v, name, obj, size, errp);
+    v->start_struct(v, name, obj, size, &err);
+    if (obj && v->type == VISITOR_INPUT) {
+        assert(err || *obj);
+    }
+    error_propagate(errp, err);
 }

 void visit_check_struct(Visitor *v, Error **errp)
@@ -36,11 +42,13 @@ void visit_check_struct(Visitor *v, Error **errp)
     }
 }

-void visit_end_struct(Visitor *v)
+bool visit_end_struct(Visitor *v)
 {
     v->end_struct(v);
+    return v->type == VISITOR_INPUT;
 }

+
 void visit_start_list(Visitor *v, const char *name, GenericList **list,
                       size_t size, Error **errp)
 {
@@ -54,26 +62,34 @@ GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size)
     return v->next_list(v, tail, size);
 }

-void visit_end_list(Visitor *v)
+bool visit_end_list(Visitor *v)
 {
     v->end_list(v);
+    return v->type == VISITOR_INPUT;
 }

 void visit_start_alternate(Visitor *v, const char *name,
                            GenericAlternate **obj, size_t size,
                            bool promote_int, Error **errp)
 {
+    Error *err = NULL;
+
     assert(obj && size >= sizeof(GenericAlternate));
     if (v->start_alternate) {
-        v->start_alternate(v, name, obj, size, promote_int, errp);
+        v->start_alternate(v, name, obj, size, promote_int, &err);
+        if (v->type == VISITOR_INPUT) {
+            assert(err || *obj);
+        }
+        error_propagate(errp, err);
     }
 }

-void visit_end_alternate(Visitor *v)
+bool visit_end_alternate(Visitor *v)
 {
     if (v->end_alternate) {
         v->end_alternate(v);
     }
+    return v->type == VISITOR_INPUT;
 }

 bool visit_optional(Visitor *v, const char *name, bool *present)
@@ -205,12 +221,17 @@ void visit_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)

 void visit_type_str(Visitor *v, const char *name, char **obj, Error **errp)
 {
+    Error *err = NULL;
     assert(obj);
     /* TODO: Fix callers to not pass NULL when they mean "", so that we
      * can enable:
     assert(v->type != VISITOR_OUTPUT || *obj);
      */
-    v->type_str(v, name, obj, errp);
+    v->type_str(v, name, obj, &err);
+    if (v->type == VISITOR_INPUT) {
+        assert(err || *obj);
+    }
+    error_propagate(errp, err);
 }

 void visit_type_number(Visitor *v, const char *name, double *obj,
@@ -222,9 +243,15 @@ void visit_type_number(Visitor *v, const char *name, double *obj,

 void visit_type_any(Visitor *v, const char *name, QObject **obj, Error **errp)
 {
+    Error *err = NULL;
+
     assert(obj);
     assert(v->type != VISITOR_OUTPUT || *obj);
-    v->type_any(v, name, obj, errp);
+    v->type_any(v, name, obj, &err);
+    if (v->type == VISITOR_INPUT) {
+        assert(err || *obj);
+    }
+    error_propagate(errp, err);
 }

 void visit_type_null(Visitor *v, const char *name, Error **errp)
diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
index 14a9ebb..d6c494d 100644
--- a/tests/test-qmp-commands.c
+++ b/tests/test-qmp-commands.c
@@ -228,14 +228,13 @@ static void test_dealloc_partial(void)
         QDECREF(ud2_dict);
     }

-    /* verify partial success */
-    assert(ud2 != NULL);
-    assert(ud2->string0 != NULL);
-    assert(strcmp(ud2->string0, text) == 0);
-    assert(ud2->dict1 == NULL);
-
-    /* confirm & release construction error */
+    /* verify that visit_type_XXX() cleans up properly on error */
     error_free_or_abort(&err);
+    assert(!ud2);
+
+    /* Manually create a partial object, leaving ud2->dict1 at NULL */
+    ud2 = g_new0(UserDefTwo, 1);
+    ud2->string0 = g_strdup(text);

     /* tear down partial object */
     qapi_free_UserDefTwo(ud2);
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 6a33aa4..9587b14 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -181,10 +181,7 @@ static void test_validate_fail_struct(TestInputVisitorData *data,

     visit_type_TestStruct(v, NULL, &p, &err);
     error_free_or_abort(&err);
-    if (p) {
-        g_free(p->string);
-    }
-    g_free(p);
+    g_assert(!p);
 }

 static void test_validate_fail_struct_nested(TestInputVisitorData *data,
@@ -198,7 +195,7 @@ static void test_validate_fail_struct_nested(TestInputVisitorData *data,

     visit_type_UserDefTwo(v, NULL, &udp, &err);
     error_free_or_abort(&err);
-    qapi_free_UserDefTwo(udp);
+    g_assert(!udp);
 }

 static void test_validate_fail_list(TestInputVisitorData *data,
@@ -212,7 +209,7 @@ static void test_validate_fail_list(TestInputVisitorData *data,

     visit_type_UserDefOneList(v, NULL, &head, &err);
     error_free_or_abort(&err);
-    qapi_free_UserDefOneList(head);
+    g_assert(!head);
 }

 static void test_validate_fail_union_native_list(TestInputVisitorData *data,
@@ -227,7 +224,7 @@ static void test_validate_fail_union_native_list(TestInputVisitorData *data,

     visit_type_UserDefNativeListUnion(v, NULL, &tmp, &err);
     error_free_or_abort(&err);
-    qapi_free_UserDefNativeListUnion(tmp);
+    g_assert(!tmp);
 }

 static void test_validate_fail_union_flat(TestInputVisitorData *data,
@@ -241,7 +238,7 @@ static void test_validate_fail_union_flat(TestInputVisitorData *data,

     visit_type_UserDefFlatUnion(v, NULL, &tmp, &err);
     error_free_or_abort(&err);
-    qapi_free_UserDefFlatUnion(tmp);
+    g_assert(!tmp);
 }

 static void test_validate_fail_union_flat_no_discrim(TestInputVisitorData *data,
@@ -256,13 +253,13 @@ static void test_validate_fail_union_flat_no_discrim(TestInputVisitorData *data,

     visit_type_UserDefFlatUnion2(v, NULL, &tmp, &err);
     error_free_or_abort(&err);
-    qapi_free_UserDefFlatUnion2(tmp);
+    g_assert(!tmp);
 }

 static void test_validate_fail_alternate(TestInputVisitorData *data,
                                          const void *unused)
 {
-    UserDefAlternate *tmp = NULL;
+    UserDefAlternate *tmp;
     Visitor *v;
     Error *err = NULL;

@@ -270,7 +267,7 @@ static void test_validate_fail_alternate(TestInputVisitorData *data,

     visit_type_UserDefAlternate(v, NULL, &tmp, &err);
     error_free_or_abort(&err);
-    qapi_free_UserDefAlternate(tmp);
+    g_assert(!tmp);
 }

 static void do_test_validate_qmp_introspect(TestInputVisitorData *data,
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index a62c2b1..19bed0a 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -759,18 +759,12 @@ static void test_visitor_in_errors(TestInputVisitorData *data,

     visit_type_TestStruct(v, NULL, &p, &err);
     error_free_or_abort(&err);
-    /* FIXME - a failed parse should not leave a partially-allocated p
-     * for us to clean up; this could cause callers to leak memory. */
-    g_assert(p->string == NULL);
-
-    g_free(p->string);
-    g_free(p);
+    g_assert(!p);

     v = visitor_input_test_init(data, "[ '1', '2', false, '3' ]");
     visit_type_strList(v, NULL, &q, &err);
     error_free_or_abort(&err);
-    assert(q);
-    qapi_free_strList(q);
+    assert(!q);
 }

 static void test_visitor_in_wrong_type(TestInputVisitorData *data,
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 06f22b7..f7956b8 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -903,7 +903,10 @@ Example:
         }
         visit_check_struct(v, &err);
     out_obj:
-        visit_end_struct(v);
+        if (visit_end_struct(v) && err) {
+            qapi_free_UserDefOne(*obj);
+            *obj = NULL;
+        }
     out:
         error_propagate(errp, err);
     }
@@ -926,7 +929,10 @@ Example:
             }
         }

-        visit_end_list(v);
+        if (visit_end_list(v) && err) {
+            qapi_free_UserDefOneList(*obj);
+            *obj = NULL;
+        }
     out:
         error_propagate(errp, err);
     }
-- 
2.5.0