[Qemu-devel] [PATCH v4 00/18] migration: add static analysis tool to check vmstate compat

[Qemu-devel] [PATCH v4 00/18] migration: add static analysis tool to check vmstate compat

[Amit Shah]

Hello,

v4:
 - Return value capped to 255 to prevent wrap-around (Eric Blake)

v3:
 - Python script returns an error code: 0 if no errors, positive for
   the number of errors identified.

v2:
 - Tabs->spaces (Dave Gilbert)
 - Several changes to the python script to make it more python-like
   (Vitaly Kuznetsov)
 - Don't store the empty fields created by VMSTATE_VALIDATE in the
   json output

This series adds a static vmstate checker to check for breakage of
live migration by analyzing the vmstate information between different
QEMU versions.

In patch 1, QEMU is modified to add a -dump-vmstate commandline
option, which takes a filename as the argument.  When invoked, QEMU
dumps the vmstate info in JSON format for the current machine type to
the file.  This patch is loosely based on a version from Andreas
Färber.

This JSON file is then fed into the Python script introduced in patch
2.  The script takes 'src' and 'dest' arguments, indicating the
direction of migration.  The script then performs a series of checks
and spews out information on inconsistencies it finds in the data.

Two stripped-down versions of JSON dumps are included in this
patchset (patch 3).

Patches 4 - 18 contain modifications to those dumps, to show the
checks that are performed by the script.  The result of running the
script against the final version of those files is appended below.

The checks are to be performed for a particular machine type, and
comparing different machine types is bound to turn up false-positives:
e.g.

(in a qemu 2.0 tree):
./x86_64-softmmu/qemu-system-x86_64 -dump-vmstate qemu-2.0.json

(in a qemu 2.2 tree:)
./x86_64-softmmu/qemu-system-x86_64 -dump-vmstate -M pc-i440fx-2.0 \
   qemu-2.2-m2.0.json

./scripts/vmstate-static-checker.py -s qemu-2.0.json -d qemu-2.2-m2.0.json

should not show any output.

The idea is to include this script in 'make check', ensuring new
commits don't break migration.

Later, this script can also be baked into the release process: vmstate
dumps for released versions of qemu for various machine types can be
stored in a directory in the tree.  At the time of freezing the tree
for releases, (like -rc2), the dump for the current release can be
updated.  A policy that says "No more live migration-breaking changes
can be accepted post -rc2" can be put in place.  Also, for checks for
older machine types on the current tree with the saved dumps should
not indicate any regressions.

I expect there to be a few false positives at the start (though there
aren't any right now).  The script will keep evolving, though, to
include new scenarios, to make it more helpful.

A later project would be to also store other guest-visible information
and use that output for more checks (including lspci output from
inside guests).

As mentioned earlier, this is the result of running the script against
the sample json data included in this patchset:

$ ./scripts/vmstate-static-checker.py --src ./tests/vmstate-static-checker-data/dump1.json  --dest ./tests/vmstate-static-checker-data/dump2.json 
Section "usb-kbd" Description "usb-kbd" Field "kbd.keycodes" size mismatch: 4 , 2
Section "PIIX3-xen" Description "PIIX3": minimum version error: 1 < 2
Section "PIIX3-xen" Description "PIIX3": Entry "Subsections" missing
Section "tpci200": Description "tpci200" missing, got "tpci2002" instead; skipping
Section "megasas", Description "PCIDevice": expected field "irq_state", while dest has no further fields
Section "SUNW,fdtwo" Description "fdc": version error: 2 > 1
Section "SUNW,fdtwo", Description "fdrive": Subsection "fdrive/media_rate" not found
Section "fusbh200-ehci-usb" version error: 2 > 1
Section "fusbh200-ehci-usb", Description "ehci-core": expected field "usbsts", got "usbsts_pending"; skipping rest
Section "intel-hda-generic", Description "intel-hda", Field "pci": missing description
Section "cfi.pflash01": Entry "Description" missing
Section "pci-serial-4x" Description "pci-serial-multi": Entry "Fields" missing
Warning: checking incompatible machine types: "pc-i440fx-2.1", "pc-i440fx-2.2"
Section "fw_cfg" does not exist in dest

$ echo $?
13

Amit Shah (18):
  migration: dump vmstate info as a json file for static analysis
  vmstate-static-checker: script to validate vmstate changes
  tests: vmstate static checker: add dump1 and dump2 files
  tests: vmstate static checker: incompat machine types
  tests: vmstate static checker: add version error in main section
  tests: vmstate static checker: version mismatch inside a Description
  tests: vmstate static checker: minimum_version_id check
  tests: vmstate static checker: remove a section
  tests: vmstate static checker: remove a field
  tests: vmstate static checker: remove last field in a struct
  tests: vmstate static checker: change description name
  tests: vmstate static checker: remove Fields
  tests: vmstate static checker: remove Description
  tests: vmstate static checker: remove Description inside Fields
  tests: vmstate static checker: remove a subsection
  tests: vmstate static checker: remove Subsections
  tests: vmstate static checker: add substructure for usb-kbd for hid
    section
  tests: vmstate static checker: add size mismatch inside substructure

 include/migration/vmstate.h                  |    2 +
 qemu-options.hx                              |    9 +
 savevm.c                                     |  139 +++
 scripts/vmstate-static-checker.py            |  329 ++++++++
 tests/vmstate-static-checker-data/dump1.json | 1163 ++++++++++++++++++++++++++
 tests/vmstate-static-checker-data/dump2.json |  968 +++++++++++++++++++++
 vl.c                                         |   13 +
 7 files changed, 2623 insertions(+)
 create mode 100755 scripts/vmstate-static-checker.py
 create mode 100644 tests/vmstate-static-checker-data/dump1.json
 create mode 100644 tests/vmstate-static-checker-data/dump2.json

-- 
1.9.3

[Qemu-devel] [PATCH v4 01/18] migration: dump vmstate info as a json file for static analysis

[Amit Shah]

This commit adds a new command, '-dump-vmstate', that takes a filename
as a parameter.  When executed, QEMU will dump the vmstate information
for the machine type it's invoked with to the file, and quit.

The JSON-format output can then be used to compare the vmstate info for
different QEMU versions, specifically to test whether live migration
would break due to changes in the vmstate data.

A Python script that compares the output of such JSON dumps is included
in the following commit.

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 include/migration/vmstate.h |   2 +
 qemu-options.hx             |   9 +++
 savevm.c                    | 139 ++++++++++++++++++++++++++++++++++++++++++++
 vl.c                        |  13 +++++
 4 files changed, 163 insertions(+)

diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index 7e45048..9829c0e 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -778,4 +778,6 @@ void vmstate_register_ram(struct MemoryRegion *memory, DeviceState *dev);
 void vmstate_unregister_ram(struct MemoryRegion *memory, DeviceState *dev);
 void vmstate_register_ram_global(struct MemoryRegion *memory);
 
+void dump_vmstate_json_to_file(FILE *out_fp);
+
 #endif
diff --git a/qemu-options.hx b/qemu-options.hx
index d0714c4..48d493b 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -3221,6 +3221,15 @@ STEXI
 prepend a timestamp to each log message.(default:on)
 ETEXI
 
+DEF("dump-vmstate", HAS_ARG, QEMU_OPTION_dump_vmstate,
+    "-dump-vmstate <file>\n" "", QEMU_ARCH_ALL)
+STEXI
+@item -dump-vmstate @var{file}
+@findex -dump-vmstate
+Dump json-encoded vmstate information for current machine type to file
+in @var{file}
+ETEXI
+
 HXCOMM This is the last statement. Insert new options before this line!
 STEXI
 @end table
diff --git a/savevm.c b/savevm.c
index da8aa24..0c8ad45 100644
--- a/savevm.c
+++ b/savevm.c
@@ -24,6 +24,7 @@
 
 #include "config-host.h"
 #include "qemu-common.h"
+#include "hw/boards.h"
 #include "hw/hw.h"
 #include "hw/qdev.h"
 #include "net/net.h"
@@ -241,6 +242,144 @@ static QTAILQ_HEAD(savevm_handlers, SaveStateEntry) savevm_handlers =
     QTAILQ_HEAD_INITIALIZER(savevm_handlers);
 static int global_section_id;
 
+static void dump_vmstate_vmsd(FILE *out_file,
+                              const VMStateDescription *vmsd, int indent,
+                              bool is_subsection);
+
+static void dump_vmstate_vmsf(FILE *out_file, const VMStateField *field,
+                              int indent)
+{
+    fprintf(out_file, "%*s{\n", indent, "");
+    indent += 2;
+    fprintf(out_file, "%*s\"field\": \"%s\",\n", indent, "", field->name);
+    fprintf(out_file, "%*s\"version_id\": %d,\n", indent, "",
+            field->version_id);
+    fprintf(out_file, "%*s\"field_exists\": %s,\n", indent, "",
+            field->field_exists ? "true" : "false");
+    fprintf(out_file, "%*s\"size\": %zu", indent, "", field->size);
+    if (field->vmsd != NULL) {
+        fprintf(out_file, ",\n");
+        dump_vmstate_vmsd(out_file, field->vmsd, indent, false);
+    }
+    fprintf(out_file, "\n%*s}", indent - 2, "");
+}
+
+static void dump_vmstate_vmss(FILE *out_file,
+                              const VMStateSubsection *subsection,
+                              int indent)
+{
+    if (subsection->vmsd != NULL) {
+        dump_vmstate_vmsd(out_file, subsection->vmsd, indent, true);
+    }
+}
+
+static void dump_vmstate_vmsd(FILE *out_file,
+                              const VMStateDescription *vmsd, int indent,
+                              bool is_subsection)
+{
+    if (is_subsection) {
+        fprintf(out_file, "%*s{\n", indent, "");
+    } else {
+        fprintf(out_file, "%*s\"%s\": {\n", indent, "", "Description");
+    }
+    indent += 2;
+    fprintf(out_file, "%*s\"name\": \"%s\",\n", indent, "", vmsd->name);
+    fprintf(out_file, "%*s\"version_id\": %d,\n", indent, "",
+            vmsd->version_id);
+    fprintf(out_file, "%*s\"minimum_version_id\": %d", indent, "",
+            vmsd->minimum_version_id);
+    if (vmsd->fields != NULL) {
+        const VMStateField *field = vmsd->fields;
+        bool first;
+
+        fprintf(out_file, ",\n%*s\"Fields\": [\n", indent, "");
+        first = true;
+        while (field->name != NULL) {
+            if (field->flags & VMS_MUST_EXIST) {
+                /* Ignore VMSTATE_VALIDATE bits; these don't get migrated */
+                field++;
+                continue;
+            }
+            if (!first) {
+                fprintf(out_file, ",\n");
+            }
+            dump_vmstate_vmsf(out_file, field, indent + 2);
+            field++;
+            first = false;
+        }
+        fprintf(out_file, "\n%*s]", indent, "");
+    }
+    if (vmsd->subsections != NULL) {
+        const VMStateSubsection *subsection = vmsd->subsections;
+        bool first;
+
+        fprintf(out_file, ",\n%*s\"Subsections\": [\n", indent, "");
+        first = true;
+        while (subsection->vmsd != NULL) {
+            if (!first) {
+                fprintf(out_file, ",\n");
+            }
+            dump_vmstate_vmss(out_file, subsection, indent + 2);
+            subsection++;
+            first = false;
+        }
+        fprintf(out_file, "\n%*s]", indent, "");
+    }
+    fprintf(out_file, "\n%*s}", indent - 2, "");
+}
+
+static void dump_machine_type(FILE *out_file)
+{
+    MachineClass *mc;
+
+    mc = MACHINE_GET_CLASS(current_machine);
+
+    fprintf(out_file, "  \"vmschkmachine\": {\n");
+    fprintf(out_file, "    \"Name\": \"%s\"\n", mc->name);
+    fprintf(out_file, "  },\n");
+}
+
+void dump_vmstate_json_to_file(FILE *out_file)
+{
+    GSList *list, *elt;
+    bool first;
+
+    fprintf(out_file, "{\n");
+    dump_machine_type(out_file);
+
+    first = true;
+    list = object_class_get_list(TYPE_DEVICE, true);
+    for (elt = list; elt; elt = elt->next) {
+        DeviceClass *dc = OBJECT_CLASS_CHECK(DeviceClass, elt->data,
+                                             TYPE_DEVICE);
+        const char *name;
+        int indent = 2;
+
+        if (!dc->vmsd) {
+            continue;
+        }
+
+        if (!first) {
+            fprintf(out_file, ",\n");
+        }
+        name = object_class_get_name(OBJECT_CLASS(dc));
+        fprintf(out_file, "%*s\"%s\": {\n", indent, "", name);
+        indent += 2;
+        fprintf(out_file, "%*s\"Name\": \"%s\",\n", indent, "", name);
+        fprintf(out_file, "%*s\"version_id\": %d,\n", indent, "",
+                dc->vmsd->version_id);
+        fprintf(out_file, "%*s\"minimum_version_id\": %d,\n", indent, "",
+                dc->vmsd->minimum_version_id);
+
+        dump_vmstate_vmsd(out_file, dc->vmsd, indent, false);
+
+        fprintf(out_file, "\n%*s}", indent - 2, "");
+        first = false;
+    }
+    fprintf(out_file, "\n}\n");
+    fclose(out_file);
+}
+
 static int calculate_new_instance_id(const char *idstr)
 {
     SaveStateEntry *se;
diff --git a/vl.c b/vl.c
index be69c7f..860541f 100644
--- a/vl.c
+++ b/vl.c
@@ -2991,6 +2991,7 @@ int main(int argc, char **argv, char **envp)
     const char *trace_file = NULL;
     const ram_addr_t default_ram_size = (ram_addr_t)DEFAULT_RAM_SIZE *
                                         1024 * 1024;
+    FILE *vmstate_dump_file = NULL;
 
     atexit(qemu_run_exit_notifiers);
     error_set_progname(argv[0]);
@@ -3957,6 +3958,13 @@ int main(int argc, char **argv, char **envp)
                 }
                 configure_msg(opts);
                 break;
+            case QEMU_OPTION_dump_vmstate:
+                vmstate_dump_file = fopen(optarg, "w");
+                if (vmstate_dump_file == NULL) {
+                    fprintf(stderr, "open %s: %s\n", optarg, strerror(errno));
+                    exit(1);
+                }
+                break;
             default:
                 os_parse_cmd_args(popt->index, optarg);
             }
@@ -4542,6 +4550,11 @@ int main(int argc, char **argv, char **envp)
     }
 
     qdev_prop_check_global();
+    if (vmstate_dump_file) {
+        /* dump and exit */
+        dump_vmstate_json_to_file(vmstate_dump_file);
+        return 0;
+    }
 
     if (incoming) {
         Error *local_err = NULL;
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 01/18] migration: dump vmstate info as a json file for static analysis

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> This commit adds a new command, '-dump-vmstate', that takes a filename
> as a parameter.  When executed, QEMU will dump the vmstate information
> for the machine type it's invoked with to the file, and quit.
>
> The JSON-format output can then be used to compare the vmstate info for
> different QEMU versions, specifically to test whether live migration
> would break due to changes in the vmstate data.
>
> A Python script that compares the output of such JSON dumps is included
> in the following commit.
>
> Signed-off-by: Amit Shah <amit.shah@redhat.com>



> +static void dump_vmstate_vmsd(FILE *out_file,
> +                              const VMStateDescription *vmsd, int indent,
> +                              bool is_subsection)
> +{
> +    if (is_subsection) {
> +        fprintf(out_file, "%*s{\n", indent, "");
> +    } else {
> +        fprintf(out_file, "%*s\"%s\": {\n", indent, "", "Description");
> +    }
> +    indent += 2;
> +    fprintf(out_file, "%*s\"name\": \"%s\",\n", indent, "", vmsd->name);
> +    fprintf(out_file, "%*s\"version_id\": %d,\n", indent, "",
> +            vmsd->version_id);
> +    fprintf(out_file, "%*s\"minimum_version_id\": %d", indent, "",
> +            vmsd->minimum_version_id);
> +    if (vmsd->fields != NULL) {
> +        const VMStateField *field = vmsd->fields;
> +        bool first;
> +
> +        fprintf(out_file, ",\n%*s\"Fields\": [\n", indent, "");

Remove last "\n" (*)


> +        first = true;

first can go now

> +        while (field->name != NULL) {
> +            if (field->flags & VMS_MUST_EXIST) {
> +                /* Ignore VMSTATE_VALIDATE bits; these don't get migrated */
> +                field++;
> +                continue;
> +            }
> +            if (!first) {
> +                fprintf(out_file, ",\n");

You can print always \n now, right?

Same for the other places?  Or I am missing something.

I will even go that itwould be better to just left the \n on the (*),
and just add this \n at the end of writing a subsection.


> +    fprintf(out_file, "\n%*s}", indent - 2, "");

And you remove it from here.

<full disclosure>
Yes, I have always hated pretty-printers
</full-disclosure>

Rest of this, I fully agree.

Later, Juan.

Re: [Qemu-devel] [PATCH v4 01/18] migration: dump vmstate info as a json file for static analysis

[Amit Shah]

On (Wed) 18 Jun 2014 [12:24:25], Juan Quintela wrote:
> Amit Shah <amit.shah@redhat.com> wrote:
> > This commit adds a new command, '-dump-vmstate', that takes a filename
> > as a parameter.  When executed, QEMU will dump the vmstate information
> > for the machine type it's invoked with to the file, and quit.
> >
> > The JSON-format output can then be used to compare the vmstate info for
> > different QEMU versions, specifically to test whether live migration
> > would break due to changes in the vmstate data.
> >
> > A Python script that compares the output of such JSON dumps is included
> > in the following commit.
> >
> > Signed-off-by: Amit Shah <amit.shah@redhat.com>
> 
> 
> 
> > +static void dump_vmstate_vmsd(FILE *out_file,
> > +                              const VMStateDescription *vmsd, int indent,
> > +                              bool is_subsection)
> > +{
> > +    if (is_subsection) {
> > +        fprintf(out_file, "%*s{\n", indent, "");
> > +    } else {
> > +        fprintf(out_file, "%*s\"%s\": {\n", indent, "", "Description");
> > +    }
> > +    indent += 2;
> > +    fprintf(out_file, "%*s\"name\": \"%s\",\n", indent, "", vmsd->name);
> > +    fprintf(out_file, "%*s\"version_id\": %d,\n", indent, "",
> > +            vmsd->version_id);
> > +    fprintf(out_file, "%*s\"minimum_version_id\": %d", indent, "",
> > +            vmsd->minimum_version_id);
> > +    if (vmsd->fields != NULL) {
> > +        const VMStateField *field = vmsd->fields;
> > +        bool first;
> > +
> > +        fprintf(out_file, ",\n%*s\"Fields\": [\n", indent, "");
> 
> Remove last "\n" (*)
> 
> 
> > +        first = true;
> 
> first can go now
> 
> > +        while (field->name != NULL) {
> > +            if (field->flags & VMS_MUST_EXIST) {
> > +                /* Ignore VMSTATE_VALIDATE bits; these don't get migrated */
> > +                field++;
> > +                continue;
> > +            }
> > +            if (!first) {
> > +                fprintf(out_file, ",\n");
> 
> You can print always \n now, right?

There's also a , there...

This sequence was added recently (v2 onwards) for the ignoring of the
VMS_MUST_EXIST stuff.

> Same for the other places?  Or I am missing something.
> 
> I will even go that itwould be better to just left the \n on the (*),
> and just add this \n at the end of writing a subsection.
> 
> 
> > +    fprintf(out_file, "\n%*s}", indent - 2, "");
> 
> And you remove it from here.

I tried several things with the \n; the current setting is the best I
found.

Of course, this is just pretty-printing, so I don't actually remember
all the details but I can look it up my git tree...

> <full disclosure>
> Yes, I have always hated pretty-printers
> </full-disclosure>

Oh, me too!

> Rest of this, I fully agree.
> 
> Later, Juan.

		Amit

Re: [Qemu-devel] [PATCH v4 01/18] migration: dump vmstate info as a json file for static analysis

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
>> 
>> You can print always \n now, right?
>
> There's also a , there...
>
> This sequence was added recently (v2 onwards) for the ignoring of the
> VMS_MUST_EXIST stuff.

I knew it needed to be some reason for the ugliness :-()

>> Same for the other places?  Or I am missing something.
>> 
>> I will even go that itwould be better to just left the \n on the (*),
>> and just add this \n at the end of writing a subsection.
>> 
>> 
>> > +    fprintf(out_file, "\n%*s}", indent - 2, "");
>> 
>> And you remove it from here.
>
> I tried several things with the \n; the current setting is the best I
> found.
>
> Of course, this is just pretty-printing, so I don't actually remember
> all the details but I can look it up my git tree...

ok, I retire the comment, and anyways, it is trivial to fix
incrementally if you/anybody came with a clever idea.

Reviewed-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 02/18] vmstate-static-checker: script to validate vmstate changes

[Amit Shah]

This script compares the vmstate dumps in JSON format as output by QEMU
with the -dump-vmstate option.

It flags various errors, like version mismatch, sections going away,
size mismatches, etc.

This script is tolerant of a few changes that do not change the on-wire
format, like embedding a few fields within substructs.

The script takes -s/--src and -d/--dest parameters, to which filenames
are given as arguments.

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 scripts/vmstate-static-checker.py | 329 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 329 insertions(+)
 create mode 100755 scripts/vmstate-static-checker.py

diff --git a/scripts/vmstate-static-checker.py b/scripts/vmstate-static-checker.py
new file mode 100755
index 0000000..cbb4c6f
--- /dev/null
+++ b/scripts/vmstate-static-checker.py
@@ -0,0 +1,329 @@
+#!/usr/bin/python
+#
+# Compares vmstate information stored in JSON format, obtained from
+# the -dump-vmstate QEMU command.
+#
+# Copyright 2014 Amit Shah <amit.shah@redhat.com>
+# Copyright 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, see <http://www.gnu.org/licenses/>.
+
+import argparse
+import json
+import sys
+
+taint = 0
+
+def bump_taint():
+    global taint
+
+    # Ensure we don't wrap around or reset to 0 -- the shell only has
+    # an 8-bit return value.
+    if taint < 255:
+        taint = taint + 1
+
+
+def check_fields_match(s_field, d_field):
+    if s_field == d_field:
+        return True
+
+    # Some fields changed names between qemu versions.  This list
+    # is used to whitelist such changes.
+    changed_names = [
+                     ['d', 'dev', 'pcidev', 'pci_dev', 'parent_obj'],
+                     ['card', 'parent_obj'],
+                     ['bridge.dev', 'parent_obj'],
+                     ['br.dev', 'parent_obj.parent_obj'],
+                     ['port.br.dev', 'parent_obj.parent_obj.parent_obj'],
+                     ['port.br.dev.exp.aer_log',
+                      'parent_obj.parent_obj.parent_obj.exp.aer_log'],
+                     ['br.dev.exp.aer_log',
+                      'parent_obj.parent_obj.exp.aer_log'],
+                     ['shpc', 'bridge.dev.shpc'],
+                     ['pci0_status',
+                      'acpi_pci_hotplug.acpi_pcihp_pci_status[0x0]'],
+                     ['pci_irq_levels', 'pci_irq_levels_vmstate'],
+                     ['usb-ptr-queue', 'HIDPointerEventQueue'],
+                     ['num_surfaces', 'ssd.num_surfaces'],
+                     ['timer', 'timer_expiry'],
+                    ]
+
+    for grp in changed_names:
+        if s_field in grp and d_field in grp:
+            return True
+
+    return False
+
+
+def exists_in_substruct(fields, item):
+    # Some QEMU versions moved a few fields inside a substruct.  This
+    # kept the on-wire format the same.  This function checks if
+    # something got shifted inside a substruct.  For example, the
+    # change in commit 1f42d22233b4f3d1a2933ff30e8d6a6d9ee2d08f
+
+    if not "Description" in fields:
+        return False
+
+    if not "Fields" in fields["Description"]:
+        return False
+
+    substruct_fields = fields["Description"]["Fields"]
+
+    if substruct_fields == []:
+        return False
+
+    return check_fields_match(substruct_fields[0]["field"], item)
+
+
+def check_fields(src_fields, dest_fields, desc, sec):
+    # This function checks for all the fields in a section.  If some
+    # fields got embedded into a substruct, this function will also
+    # attempt to check inside the substruct.
+
+    d_iter = iter(dest_fields)
+    s_iter = iter(src_fields)
+
+    # Using these lists as stacks to store previous value of s_iter
+    # and d_iter, so that when time comes to exit out of a substruct,
+    # we can go back one level up and continue from where we left off.
+
+    s_iter_list = []
+    d_iter_list = []
+
+    advance_src = True
+    advance_dest = True
+
+    while True:
+        if advance_src:
+            try:
+                s_item = s_iter.next()
+            except StopIteration:
+                if s_iter_list == []:
+                    break
+
+                s_iter = s_iter_list.pop()
+                continue
+        else:
+            # We want to avoid advancing just once -- when entering a
+            # substruct, or when exiting one.
+            advance_src = True
+
+        if advance_dest:
+            try:
+                d_item = d_iter.next()
+            except StopIteration:
+                if d_iter_list == []:
+                    # We were not in a substruct
+                    print "Section \"" + sec + "\",",
+                    print "Description " + "\"" + desc + "\":",
+                    print "expected field \"" + s_item["field"] + "\",",
+                    print "while dest has no further fields"
+                    bump_taint()
+                    break
+
+                d_iter = d_iter_list.pop()
+                advance_src = False
+                continue
+        else:
+            advance_dest = True
+
+        if not check_fields_match(s_item["field"], d_item["field"]):
+            # Some fields were put in substructs, keeping the
+            # on-wire format the same, but breaking static tools
+            # like this one.
+
+            # First, check if dest has a new substruct.
+            if exists_in_substruct(d_item, s_item["field"]):
+                # listiterators don't have a prev() function, so we
+                # have to store our current location, descend into the
+                # substruct, and ensure we come out as if nothing
+                # happened when the substruct is over.
+                #
+                # Essentially we're opening the substructs that got
+                # added which didn't change the wire format.
+                d_iter_list.append(d_iter)
+                substruct_fields = d_item["Description"]["Fields"]
+                d_iter = iter(substruct_fields)
+                advance_src = False
+                continue
+            else:
+                # Next, check if src has substruct that dest removed
+                # (can happen in backward migration: 2.0 -> 1.5)
+                if exists_in_substruct(s_item, d_item["field"]):
+                    s_iter_list.append(s_iter)
+                    substruct_fields = s_item["Description"]["Fields"]
+                    s_iter = iter(substruct_fields)
+                    advance_dest = False
+                    continue
+                else:
+                    print "Section \"" + sec + "\",",
+                    print "Description \"" + desc + "\":",
+                    print "expected field \"" + s_item["field"] + "\",",
+                    print "got \"" + d_item["field"] + "\"; skipping rest"
+                    bump_taint()
+                    break
+
+        check_version(s_item, d_item, sec, desc)
+
+        if not "Description" in s_item:
+            # Check size of this field only if it's not a VMSTRUCT entry
+            check_size(s_item, d_item, sec, desc, s_item["field"])
+
+        check_description_in_list(s_item, d_item, sec, desc)
+
+
+def check_subsections(src_sub, dest_sub, desc, sec):
+    for s_item in src_sub:
+        found = False
+        for d_item in dest_sub:
+            if s_item["name"] != d_item["name"]:
+                continue
+
+            found = True
+            check_descriptions(s_item, d_item, sec)
+
+        if not found:
+            print "Section \"" + sec + "\", Description \"" + desc + "\":",
+            print "Subsection \"" + s_item["name"] + "\" not found"
+            bump_taint()
+
+
+def check_description_in_list(s_item, d_item, sec, desc):
+    if not "Description" in s_item:
+        return
+
+    if not "Description" in d_item:
+        print "Section \"" + sec + "\", Description \"" + desc + "\",",
+        print "Field \"" + s_item["field"] + "\": missing description"
+        bump_taint()
+        return
+
+    check_descriptions(s_item["Description"], d_item["Description"], sec)
+
+
+def check_descriptions(src_desc, dest_desc, sec):
+    check_version(src_desc, dest_desc, sec, src_desc["name"])
+
+    if not check_fields_match(src_desc["name"], dest_desc["name"]):
+        print "Section \"" + sec + "\":",
+        print "Description \"" + src_desc["name"] + "\"",
+        print "missing, got \"" + dest_desc["name"] + "\" instead; skipping"
+        bump_taint()
+        return
+
+    for f in src_desc:
+        if not f in dest_desc:
+            print "Section \"" + sec + "\"",
+            print "Description \"" + src_desc["name"] + "\":",
+            print "Entry \"" + f + "\" missing"
+            bump_taint()
+            continue
+
+        if f == 'Fields':
+            check_fields(src_desc[f], dest_desc[f], src_desc["name"], sec)
+
+        if f == 'Subsections':
+            check_subsections(src_desc[f], dest_desc[f], src_desc["name"], sec)
+
+
+def check_version(s, d, sec, desc=None):
+    if s["version_id"] > d["version_id"]:
+        print "Section \"" + sec + "\"",
+        if desc:
+            print "Description \"" + desc + "\":",
+        print "version error:", s["version_id"], ">", d["version_id"]
+        bump_taint()
+
+    if not "minimum_version_id" in d:
+        return
+
+    if s["version_id"] < d["minimum_version_id"]:
+        print "Section \"" + sec + "\"",
+        if desc:
+            print "Description \"" + desc + "\":",
+            print "minimum version error:", s["version_id"], "<",
+            print d["minimum_version_id"]
+            bump_taint()
+
+
+def check_size(s, d, sec, desc=None, field=None):
+    if s["size"] != d["size"]:
+        print "Section \"" + sec + "\"",
+        if desc:
+            print "Description \"" + desc + "\"",
+        if field:
+            print "Field \"" + field + "\"",
+        print "size mismatch:", s["size"], ",", d["size"]
+        bump_taint()
+
+
+def check_machine_type(s, d):
+    if s["Name"] != d["Name"]:
+        print "Warning: checking incompatible machine types:",
+        print "\"" + s["Name"] + "\", \"" + d["Name"] + "\""
+    return
+
+
+def main():
+    parser = argparse.ArgumentParser(description=
+                                     'Parse vmstate dumps from QEMU.')
+    parser.add_argument('-s', '--src', type=file, required=True,
+                        help='json dump from src qemu')
+    parser.add_argument('-d', '--dest', type=file, required=True,
+                        help='json dump from dest qemu')
+    parser.add_argument('--reverse', required=False, default=False,
+                        action='store_true',
+                        help='reverse the direction')
+    args = parser.parse_args()
+
+    src_data = json.load(args.src)
+    dest_data = json.load(args.dest)
+    args.src.close()
+    args.dest.close()
+
+    if args.reverse:
+        temp = src_data
+        src_data = dest_data
+        dest_data = temp
+
+    for sec in src_data:
+        if not sec in dest_data:
+            print "Section \"" + sec + "\" does not exist in dest"
+            bump_taint()
+            continue
+
+        s = src_data[sec]
+        d = dest_data[sec]
+
+        if sec == "vmschkmachine":
+            check_machine_type(s, d)
+            continue
+
+        check_version(s, d, sec)
+
+        for entry in s:
+            if not entry in d:
+                print "Section \"" + sec + "\": Entry \"" + entry + "\"",
+                print "missing"
+                bump_taint()
+                continue
+
+            if entry == "Description":
+                check_descriptions(s[entry], d[entry], sec)
+
+    return taint
+
+
+if __name__ == '__main__':
+    sys.exit(main())
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 02/18] vmstate-static-checker: script to validate vmstate changes

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> This script compares the vmstate dumps in JSON format as output by QEMU
> with the -dump-vmstate option.
>
> It flags various errors, like version mismatch, sections going away,
> size mismatches, etc.
>
> This script is tolerant of a few changes that do not change the on-wire
> format, like embedding a few fields within substructs.
>
> The script takes -s/--src and -d/--dest parameters, to which filenames
> are given as arguments.
>
> Signed-off-by: Amit Shah <amit.shah@redhat.com>



> +def check_fields_match(s_field, d_field):
> +    if s_field == d_field:
> +        return True
> +
> +    # Some fields changed names between qemu versions.  This list
> +    # is used to whitelist such changes.
> +    changed_names = [
> +                     ['d', 'dev', 'pcidev', 'pci_dev', 'parent_obj'],
> +                     ['card', 'parent_obj'],
> +                     ['bridge.dev', 'parent_obj'],
> +                     ['br.dev', 'parent_obj.parent_obj'],
> +                     ['port.br.dev', 'parent_obj.parent_obj.parent_obj'],
> +                     ['port.br.dev.exp.aer_log',
> +                      'parent_obj.parent_obj.parent_obj.exp.aer_log'],
> +                     ['br.dev.exp.aer_log',
> +                      'parent_obj.parent_obj.exp.aer_log'],
> +                     ['shpc', 'bridge.dev.shpc'],
> +                     ['pci0_status',
> +                      'acpi_pci_hotplug.acpi_pcihp_pci_status[0x0]'],
> +                     ['pci_irq_levels', 'pci_irq_levels_vmstate'],
> +                     ['usb-ptr-queue', 'HIDPointerEventQueue'],
> +                     ['num_surfaces', 'ssd.num_surfaces'],
> +                     ['timer', 'timer_expiry'],
> +                    ]

I expect this not to be very big, so ... could we prepend the VMState
description name on this?

"usb-ptr-queue" and "HIDPointerEventQueue" are quite descriptive, but

"d" "dev" "pci_dev" ...

I am not sure that we want to do this translation for any
VMSTateDescription.  In particular, we are ending that parent_obj can
match with anything :-()

My python is rusty (to be generous with me), so I am assuming rest of
things are idiomatic Python O:-)

Later, Juan.

Re: [Qemu-devel] [PATCH v4 02/18] vmstate-static-checker: script to validate vmstate changes

[Amit Shah]

On (Wed) 18 Jun 2014 [12:44:12], Juan Quintela wrote:
> Amit Shah <amit.shah@redhat.com> wrote:
> > This script compares the vmstate dumps in JSON format as output by QEMU
> > with the -dump-vmstate option.
> >
> > It flags various errors, like version mismatch, sections going away,
> > size mismatches, etc.
> >
> > This script is tolerant of a few changes that do not change the on-wire
> > format, like embedding a few fields within substructs.
> >
> > The script takes -s/--src and -d/--dest parameters, to which filenames
> > are given as arguments.
> >
> > Signed-off-by: Amit Shah <amit.shah@redhat.com>
> 
> 
> 
> > +def check_fields_match(s_field, d_field):
> > +    if s_field == d_field:
> > +        return True
> > +
> > +    # Some fields changed names between qemu versions.  This list
> > +    # is used to whitelist such changes.
> > +    changed_names = [
> > +                     ['d', 'dev', 'pcidev', 'pci_dev', 'parent_obj'],
> > +                     ['card', 'parent_obj'],
> > +                     ['bridge.dev', 'parent_obj'],
> > +                     ['br.dev', 'parent_obj.parent_obj'],
> > +                     ['port.br.dev', 'parent_obj.parent_obj.parent_obj'],
> > +                     ['port.br.dev.exp.aer_log',
> > +                      'parent_obj.parent_obj.parent_obj.exp.aer_log'],
> > +                     ['br.dev.exp.aer_log',
> > +                      'parent_obj.parent_obj.exp.aer_log'],
> > +                     ['shpc', 'bridge.dev.shpc'],
> > +                     ['pci0_status',
> > +                      'acpi_pci_hotplug.acpi_pcihp_pci_status[0x0]'],
> > +                     ['pci_irq_levels', 'pci_irq_levels_vmstate'],
> > +                     ['usb-ptr-queue', 'HIDPointerEventQueue'],
> > +                     ['num_surfaces', 'ssd.num_surfaces'],
> > +                     ['timer', 'timer_expiry'],
> > +                    ]
> 
> I expect this not to be very big, so ... could we prepend the VMState
> description name on this?
> 
> "usb-ptr-queue" and "HIDPointerEventQueue" are quite descriptive, but
> 
> "d" "dev" "pci_dev" ...
> 
> I am not sure that we want to do this translation for any
> VMSTateDescription.  In particular, we are ending that parent_obj can
> match with anything :-()

Hm; so a 1-1 mapping of stuff that changed.  A similar mapping will
have to be created for all vmsds.

I'll take a look at it.

Do you want to hold off this series till that, or should I just send a
follow-up patch (series) once I get something?

> My python is rusty (to be generous with me), so I am assuming rest of
> things are idiomatic Python O:-)

Heh, I'm hoping so too -- I'm learning python on-the-go with this
script.


		Amit

Re: [Qemu-devel] [PATCH v4 02/18] vmstate-static-checker: script to validate vmstate changes

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> On (Wed) 18 Jun 2014 [12:44:12], Juan Quintela wrote:
>> Amit Shah <amit.shah@redhat.com> wrote:
>> > This script compares the vmstate dumps in JSON format as output by QEMU
>> > with the -dump-vmstate option.
>> >
>> > It flags various errors, like version mismatch, sections going away,
>> > size mismatches, etc.
>> >
>> > This script is tolerant of a few changes that do not change the on-wire
>> > format, like embedding a few fields within substructs.
>> >
>> > The script takes -s/--src and -d/--dest parameters, to which filenames
>> > are given as arguments.
>> >
>> > Signed-off-by: Amit Shah <amit.shah@redhat.com>
>> 
>> 
>> 
>> > +def check_fields_match(s_field, d_field):
>> > +    if s_field == d_field:
>> > +        return True
>> > +
>> > +    # Some fields changed names between qemu versions.  This list
>> > +    # is used to whitelist such changes.
>> > +    changed_names = [
>> > +                     ['d', 'dev', 'pcidev', 'pci_dev', 'parent_obj'],
>> > +                     ['card', 'parent_obj'],
>> > +                     ['bridge.dev', 'parent_obj'],
>> > +                     ['br.dev', 'parent_obj.parent_obj'],
>> > +                     ['port.br.dev', 'parent_obj.parent_obj.parent_obj'],
>> > +                     ['port.br.dev.exp.aer_log',
>> > +                      'parent_obj.parent_obj.parent_obj.exp.aer_log'],
>> > +                     ['br.dev.exp.aer_log',
>> > +                      'parent_obj.parent_obj.exp.aer_log'],
>> > +                     ['shpc', 'bridge.dev.shpc'],
>> > +                     ['pci0_status',
>> > +                      'acpi_pci_hotplug.acpi_pcihp_pci_status[0x0]'],
>> > +                     ['pci_irq_levels', 'pci_irq_levels_vmstate'],
>> > +                     ['usb-ptr-queue', 'HIDPointerEventQueue'],
>> > +                     ['num_surfaces', 'ssd.num_surfaces'],
>> > +                     ['timer', 'timer_expiry'],
>> > +                    ]
>> 
>> I expect this not to be very big, so ... could we prepend the VMState
>> description name on this?
>> 
>> "usb-ptr-queue" and "HIDPointerEventQueue" are quite descriptive, but
>> 
>> "d" "dev" "pci_dev" ...
>> 
>> I am not sure that we want to do this translation for any
>> VMSTateDescription.  In particular, we are ending that parent_obj can
>> match with anything :-()
>
> Hm; so a 1-1 mapping of stuff that changed.  A similar mapping will
> have to be created for all vmsds.
>
> I'll take a look at it.
>
> Do you want to hold off this series till that, or should I just send a
> follow-up patch (series) once I get something?

No, we can change that later.

Just asked Peter on irc, if nobody complains, as this is new code, I
would integrate on next PULL request.

Later, Juan.

Re: [Qemu-devel] [PATCH v4 02/18] vmstate-static-checker: script to validate vmstate changes

[Amit Shah]

On (Wed) 18 Jun 2014 [13:25:50], Juan Quintela wrote:

> > Do you want to hold off this series till that, or should I just send a
> > follow-up patch (series) once I get something?
> 
> No, we can change that later.
> 
> Just asked Peter on irc, if nobody complains, as this is new code, I
> would integrate on next PULL request.

Thanks!

		Amit

[Qemu-devel] [PATCH v4 03/18] tests: vmstate static checker: add dump1 and dump2 files

[Amit Shah]

These are stripped-down JSON data as obtained from the -dump-vmstate
option.  The two files are identical in this commit, and will be
modified in the later commits to show what the script does with the
data.

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump1.json | 1163 ++++++++++++++++++++++++++
 tests/vmstate-static-checker-data/dump2.json | 1163 ++++++++++++++++++++++++++
 2 files changed, 2326 insertions(+)
 create mode 100644 tests/vmstate-static-checker-data/dump1.json
 create mode 100644 tests/vmstate-static-checker-data/dump2.json

diff --git a/tests/vmstate-static-checker-data/dump1.json b/tests/vmstate-static-checker-data/dump1.json
new file mode 100644
index 0000000..44200fb
--- /dev/null
+++ b/tests/vmstate-static-checker-data/dump1.json
@@ -0,0 +1,1163 @@
+{
+  "vmschkmachine": {
+    "Name": "pc-i440fx-2.1"
+  },
+  "fw_cfg": {
+    "Name": "fw_cfg",
+    "version_id": 2,
+    "minimum_version_id": 1,
+    "Description": {
+      "name": "fw_cfg",
+      "version_id": 2,
+      "minimum_version_id": 1,
+      "Fields": [
+        {
+          "field": "cur_entry",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 2
+        },
+        {
+          "field": "cur_offset",
+          "version_id": 0,
+          "field_exists": true,
+          "size": 4
+        },
+        {
+          "field": "cur_offset",
+          "version_id": 2,
+          "field_exists": false,
+          "size": 4
+        }
+      ]
+    }
+  },
+  "fusbh200-ehci-usb": {
+    "Name": "fusbh200-ehci-usb",
+    "version_id": 2,
+    "minimum_version_id": 1,
+    "Description": {
+      "name": "ehci-sysbus",
+      "version_id": 2,
+      "minimum_version_id": 1,
+      "Fields": [
+        {
+          "field": "ehci",
+          "version_id": 2,
+          "field_exists": false,
+          "size": 1880,
+          "Description": {
+            "name": "ehci-core",
+            "version_id": 2,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "usbcmd",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "usbsts",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "usbsts_pending",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "usbsts_frindex",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "usbintr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "frindex",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "ctrldssegment",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "periodiclistbase",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "asynclistaddr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "configflag",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "portsc[0]",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "portsc[1]",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "portsc[2]",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "portsc[3]",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "portsc[4]",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "portsc[5]",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "frame_timer",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 8
+              },
+              {
+                "field": "last_run_ns",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 8
+              },
+              {
+                "field": "async_stepdown",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "astate",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "pstate",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "a_fetch_addr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "p_fetch_addr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              }
+            ]
+          }
+        }
+      ]
+    }
+  },
+  "pci-serial-4x": {
+    "Name": "pci-serial-4x",
+    "version_id": 1,
+    "minimum_version_id": 1,
+    "Description": {
+      "name": "pci-serial-multi",
+      "version_id": 1,
+      "minimum_version_id": 1,
+      "Fields": [
+        {
+          "field": "dev",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1944,
+          "Description": {
+            "name": "PCIDevice",
+            "version_id": 2,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "version_id",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "config",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 256
+              },
+              {
+                "field": "irq_state",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 16
+              }
+            ]
+          }
+        },
+        {
+          "field": "state",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 368,
+          "Description": {
+            "name": "serial",
+            "version_id": 3,
+            "minimum_version_id": 2,
+            "Fields": [
+              {
+                "field": "divider",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 2
+              },
+              {
+                "field": "rbr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "ier",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "iir",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "lcr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "mcr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "lsr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "msr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "scr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "fcr_vmstate",
+                "version_id": 3,
+                "field_exists": false,
+                "size": 1
+              }
+            ]
+          }
+        },
+        {
+          "field": "level",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        }
+      ]
+    }
+  },
+  "intel-hda-generic": {
+    "Name": "intel-hda-generic",
+    "version_id": 1,
+    "minimum_version_id": 0,
+    "Description": {
+      "name": "intel-hda",
+      "version_id": 1,
+      "minimum_version_id": 0,
+      "Fields": [
+        {
+          "field": "pci",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1944,
+          "Description": {
+            "name": "PCIDevice",
+            "version_id": 2,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "version_id",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "config",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 256
+              },
+              {
+                "field": "irq_state",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 16
+              }
+            ]
+          }
+        },
+        {
+          "field": "g_ctl",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "wake_en",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "state_sts",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "int_ctl",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "int_sts",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "wall_clk",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_lbase",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_ubase",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_rp",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_wp",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_ctl",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_sts",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_size",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_lbase",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_ubase",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_wp",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_cnt",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_ctl",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_sts",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_size",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "dp_lbase",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "dp_ubase",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "icw",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "irr",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "ics",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "st",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 56,
+          "Description": {
+            "name": "intel-hda-stream",
+            "version_id": 1,
+            "minimum_version_id": 0,
+            "Fields": [
+              {
+                "field": "ctl",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "lpib",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "cbl",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "lvi",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "fmt",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "bdlp_lbase",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "bdlp_ubase",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              }
+            ]
+          }
+        },
+        {
+          "field": "rirb_count",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "wall_base_ns",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 8
+        }
+      ]
+    }
+  },
+  "cfi.pflash01": {
+    "Name": "cfi.pflash01",
+    "version_id": 1,
+    "minimum_version_id": 1,
+    "Description": {
+      "name": "pflash_cfi01",
+      "version_id": 1,
+      "minimum_version_id": 1,
+      "Fields": [
+        {
+          "field": "wcycle",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "cmd",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "status",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "counter",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 8
+        }
+      ]
+    }
+  },
+  "megasas": {
+    "Name": "megasas",
+    "version_id": 0,
+    "minimum_version_id": 0,
+    "Description": {
+      "name": "megasas",
+      "version_id": 0,
+      "minimum_version_id": 0,
+      "Fields": [
+        {
+          "field": "parent_obj",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1944,
+          "Description": {
+            "name": "PCIDevice",
+            "version_id": 2,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "version_id",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "config",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 256
+              },
+              {
+                "field": "irq_state",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 16
+              }
+            ]
+          }
+        },
+        {
+          "field": "fw_state",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "intr_mask",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "doorbell",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "reply_queue_pa",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 8
+        },
+        {
+          "field": "consumer_pa",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 8
+        },
+        {
+          "field": "producer_pa",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 8
+        }
+      ]
+    }
+  },
+  "PIIX3-xen": {
+    "Name": "PIIX3-xen",
+    "version_id": 3,
+    "minimum_version_id": 2,
+    "Description": {
+      "name": "PIIX3",
+      "version_id": 3,
+      "minimum_version_id": 2,
+      "Fields": [
+        {
+          "field": "dev",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1944,
+          "Description": {
+            "name": "PCIDevice",
+            "version_id": 2,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "version_id",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "config",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 256
+              },
+              {
+                "field": "irq_state",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 16
+              }
+            ]
+          }
+        },
+        {
+          "field": "pci_irq_levels_vmstate",
+          "version_id": 3,
+          "field_exists": false,
+          "size": 4
+        }
+      ],
+      "Subsections": [
+        {
+          "name": "PIIX3/rcr",
+          "version_id": 1,
+          "minimum_version_id": 1,
+          "Fields": [
+            {
+              "field": "rcr",
+              "version_id": 0,
+              "field_exists": false,
+              "size": 1
+            }
+          ]
+        }
+      ]
+    }
+  },
+  "tpci200": {
+    "Name": "tpci200",
+    "version_id": 1,
+    "minimum_version_id": 1,
+    "Description": {
+      "name": "tpci200",
+      "version_id": 1,
+      "minimum_version_id": 1,
+      "Fields": [
+        {
+          "field": "dev",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1944,
+          "Description": {
+            "name": "PCIDevice",
+            "version_id": 2,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "version_id",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "config",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 256
+              },
+              {
+                "field": "irq_state",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 16
+              }
+            ]
+          }
+        },
+        {
+          "field": "big_endian",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "ctrl",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "status",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 2
+        },
+        {
+          "field": "int_set",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        }
+      ]
+    }
+  },
+  "SUNW,fdtwo": {
+    "Name": "SUNW,fdtwo",
+    "version_id": 2,
+    "minimum_version_id": 2,
+    "Description": {
+      "name": "fdc",
+      "version_id": 2,
+      "minimum_version_id": 2,
+      "Fields": [
+        {
+          "field": "state",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 360,
+          "Description": {
+            "name": "fdc",
+            "version_id": 2,
+            "minimum_version_id": 2,
+            "Fields": [
+              {
+                "field": "sra",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "srb",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "dor_vmstate",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "tdr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "dsr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "msr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "status0",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "status1",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "status2",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "fifo",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "data_pos",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "data_len",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "data_state",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "data_dir",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "eot",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "timer0",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "timer1",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "precomp_trk",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "config",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "lock",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "pwrd",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "num_floppies",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "drives",
+                "version_id": 1,
+                "field_exists": false,
+                "size": 40,
+                "Description": {
+                  "name": "fdrive",
+                  "version_id": 1,
+                  "minimum_version_id": 1,
+                  "Fields": [
+                    {
+                      "field": "head",
+                      "version_id": 0,
+                      "field_exists": false,
+                      "size": 1
+                    },
+                    {
+                      "field": "track",
+                      "version_id": 0,
+                      "field_exists": false,
+                      "size": 1
+                    },
+                    {
+                      "field": "sect",
+                      "version_id": 0,
+                      "field_exists": false,
+                      "size": 1
+                    }
+                  ],
+                  "Subsections": [
+                    {
+                      "name": "fdrive/media_changed",
+                      "version_id": 1,
+                      "minimum_version_id": 1,
+                      "Fields": [
+                        {
+                          "field": "media_changed",
+                          "version_id": 0,
+                          "field_exists": false,
+                          "size": 1
+                        }
+                      ]
+                    },
+                    {
+                      "name": "fdrive/media_rate",
+                      "version_id": 1,
+                      "minimum_version_id": 1,
+                      "Fields": [
+                        {
+                          "field": "media_rate",
+                          "version_id": 0,
+                          "field_exists": false,
+                          "size": 1
+                        }
+                      ]
+                    }
+                  ]
+                }
+              }
+            ]
+          }
+        }
+      ]
+    }
+  },
+  "usb-kbd": {
+    "Name": "usb-kbd",
+    "version_id": 1,
+    "minimum_version_id": 1,
+    "Description": {
+      "name": "usb-kbd",
+      "version_id": 1,
+      "minimum_version_id": 1,
+      "Fields": [
+        {
+          "field": "dev",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4352,
+          "Description": {
+            "name": "USBDevice",
+            "version_id": 1,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "addr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "state",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "remote_wakeup",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "setup_state",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "setup_len",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "setup_index",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "setup_buf",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              }
+            ]
+          }
+        },
+        {
+          "field": "kbd.keycodes",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "head",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "n",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "kbd.modifiers",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 2
+        },
+        {
+          "field": "kbd.leds",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "kbd.key",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "kbd.keys",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "protocol",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "idle",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        }
+      ]
+    }
+  }
+}
diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
new file mode 100644
index 0000000..44200fb
--- /dev/null
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -0,0 +1,1163 @@
+{
+  "vmschkmachine": {
+    "Name": "pc-i440fx-2.1"
+  },
+  "fw_cfg": {
+    "Name": "fw_cfg",
+    "version_id": 2,
+    "minimum_version_id": 1,
+    "Description": {
+      "name": "fw_cfg",
+      "version_id": 2,
+      "minimum_version_id": 1,
+      "Fields": [
+        {
+          "field": "cur_entry",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 2
+        },
+        {
+          "field": "cur_offset",
+          "version_id": 0,
+          "field_exists": true,
+          "size": 4
+        },
+        {
+          "field": "cur_offset",
+          "version_id": 2,
+          "field_exists": false,
+          "size": 4
+        }
+      ]
+    }
+  },
+  "fusbh200-ehci-usb": {
+    "Name": "fusbh200-ehci-usb",
+    "version_id": 2,
+    "minimum_version_id": 1,
+    "Description": {
+      "name": "ehci-sysbus",
+      "version_id": 2,
+      "minimum_version_id": 1,
+      "Fields": [
+        {
+          "field": "ehci",
+          "version_id": 2,
+          "field_exists": false,
+          "size": 1880,
+          "Description": {
+            "name": "ehci-core",
+            "version_id": 2,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "usbcmd",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "usbsts",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "usbsts_pending",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "usbsts_frindex",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "usbintr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "frindex",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "ctrldssegment",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "periodiclistbase",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "asynclistaddr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "configflag",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "portsc[0]",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "portsc[1]",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "portsc[2]",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "portsc[3]",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "portsc[4]",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "portsc[5]",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "frame_timer",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 8
+              },
+              {
+                "field": "last_run_ns",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 8
+              },
+              {
+                "field": "async_stepdown",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "astate",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "pstate",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "a_fetch_addr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "p_fetch_addr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              }
+            ]
+          }
+        }
+      ]
+    }
+  },
+  "pci-serial-4x": {
+    "Name": "pci-serial-4x",
+    "version_id": 1,
+    "minimum_version_id": 1,
+    "Description": {
+      "name": "pci-serial-multi",
+      "version_id": 1,
+      "minimum_version_id": 1,
+      "Fields": [
+        {
+          "field": "dev",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1944,
+          "Description": {
+            "name": "PCIDevice",
+            "version_id": 2,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "version_id",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "config",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 256
+              },
+              {
+                "field": "irq_state",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 16
+              }
+            ]
+          }
+        },
+        {
+          "field": "state",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 368,
+          "Description": {
+            "name": "serial",
+            "version_id": 3,
+            "minimum_version_id": 2,
+            "Fields": [
+              {
+                "field": "divider",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 2
+              },
+              {
+                "field": "rbr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "ier",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "iir",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "lcr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "mcr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "lsr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "msr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "scr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "fcr_vmstate",
+                "version_id": 3,
+                "field_exists": false,
+                "size": 1
+              }
+            ]
+          }
+        },
+        {
+          "field": "level",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        }
+      ]
+    }
+  },
+  "intel-hda-generic": {
+    "Name": "intel-hda-generic",
+    "version_id": 1,
+    "minimum_version_id": 0,
+    "Description": {
+      "name": "intel-hda",
+      "version_id": 1,
+      "minimum_version_id": 0,
+      "Fields": [
+        {
+          "field": "pci",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1944,
+          "Description": {
+            "name": "PCIDevice",
+            "version_id": 2,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "version_id",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "config",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 256
+              },
+              {
+                "field": "irq_state",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 16
+              }
+            ]
+          }
+        },
+        {
+          "field": "g_ctl",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "wake_en",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "state_sts",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "int_ctl",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "int_sts",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "wall_clk",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_lbase",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_ubase",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_rp",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_wp",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_ctl",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_sts",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "corb_size",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_lbase",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_ubase",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_wp",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_cnt",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_ctl",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_sts",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "rirb_size",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "dp_lbase",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "dp_ubase",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "icw",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "irr",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "ics",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "st",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 56,
+          "Description": {
+            "name": "intel-hda-stream",
+            "version_id": 1,
+            "minimum_version_id": 0,
+            "Fields": [
+              {
+                "field": "ctl",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "lpib",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "cbl",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "lvi",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "fmt",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "bdlp_lbase",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "bdlp_ubase",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              }
+            ]
+          }
+        },
+        {
+          "field": "rirb_count",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "wall_base_ns",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 8
+        }
+      ]
+    }
+  },
+  "cfi.pflash01": {
+    "Name": "cfi.pflash01",
+    "version_id": 1,
+    "minimum_version_id": 1,
+    "Description": {
+      "name": "pflash_cfi01",
+      "version_id": 1,
+      "minimum_version_id": 1,
+      "Fields": [
+        {
+          "field": "wcycle",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "cmd",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "status",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "counter",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 8
+        }
+      ]
+    }
+  },
+  "megasas": {
+    "Name": "megasas",
+    "version_id": 0,
+    "minimum_version_id": 0,
+    "Description": {
+      "name": "megasas",
+      "version_id": 0,
+      "minimum_version_id": 0,
+      "Fields": [
+        {
+          "field": "parent_obj",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1944,
+          "Description": {
+            "name": "PCIDevice",
+            "version_id": 2,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "version_id",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "config",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 256
+              },
+              {
+                "field": "irq_state",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 16
+              }
+            ]
+          }
+        },
+        {
+          "field": "fw_state",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "intr_mask",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "doorbell",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "reply_queue_pa",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 8
+        },
+        {
+          "field": "consumer_pa",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 8
+        },
+        {
+          "field": "producer_pa",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 8
+        }
+      ]
+    }
+  },
+  "PIIX3-xen": {
+    "Name": "PIIX3-xen",
+    "version_id": 3,
+    "minimum_version_id": 2,
+    "Description": {
+      "name": "PIIX3",
+      "version_id": 3,
+      "minimum_version_id": 2,
+      "Fields": [
+        {
+          "field": "dev",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1944,
+          "Description": {
+            "name": "PCIDevice",
+            "version_id": 2,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "version_id",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "config",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 256
+              },
+              {
+                "field": "irq_state",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 16
+              }
+            ]
+          }
+        },
+        {
+          "field": "pci_irq_levels_vmstate",
+          "version_id": 3,
+          "field_exists": false,
+          "size": 4
+        }
+      ],
+      "Subsections": [
+        {
+          "name": "PIIX3/rcr",
+          "version_id": 1,
+          "minimum_version_id": 1,
+          "Fields": [
+            {
+              "field": "rcr",
+              "version_id": 0,
+              "field_exists": false,
+              "size": 1
+            }
+          ]
+        }
+      ]
+    }
+  },
+  "tpci200": {
+    "Name": "tpci200",
+    "version_id": 1,
+    "minimum_version_id": 1,
+    "Description": {
+      "name": "tpci200",
+      "version_id": 1,
+      "minimum_version_id": 1,
+      "Fields": [
+        {
+          "field": "dev",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1944,
+          "Description": {
+            "name": "PCIDevice",
+            "version_id": 2,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "version_id",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "config",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 256
+              },
+              {
+                "field": "irq_state",
+                "version_id": 2,
+                "field_exists": false,
+                "size": 16
+              }
+            ]
+          }
+        },
+        {
+          "field": "big_endian",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "ctrl",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "status",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 2
+        },
+        {
+          "field": "int_set",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        }
+      ]
+    }
+  },
+  "SUNW,fdtwo": {
+    "Name": "SUNW,fdtwo",
+    "version_id": 2,
+    "minimum_version_id": 2,
+    "Description": {
+      "name": "fdc",
+      "version_id": 2,
+      "minimum_version_id": 2,
+      "Fields": [
+        {
+          "field": "state",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 360,
+          "Description": {
+            "name": "fdc",
+            "version_id": 2,
+            "minimum_version_id": 2,
+            "Fields": [
+              {
+                "field": "sra",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "srb",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "dor_vmstate",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "tdr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "dsr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "msr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "status0",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "status1",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "status2",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "fifo",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "data_pos",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "data_len",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "data_state",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "data_dir",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "eot",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "timer0",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "timer1",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "precomp_trk",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "config",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "lock",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "pwrd",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "num_floppies",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "drives",
+                "version_id": 1,
+                "field_exists": false,
+                "size": 40,
+                "Description": {
+                  "name": "fdrive",
+                  "version_id": 1,
+                  "minimum_version_id": 1,
+                  "Fields": [
+                    {
+                      "field": "head",
+                      "version_id": 0,
+                      "field_exists": false,
+                      "size": 1
+                    },
+                    {
+                      "field": "track",
+                      "version_id": 0,
+                      "field_exists": false,
+                      "size": 1
+                    },
+                    {
+                      "field": "sect",
+                      "version_id": 0,
+                      "field_exists": false,
+                      "size": 1
+                    }
+                  ],
+                  "Subsections": [
+                    {
+                      "name": "fdrive/media_changed",
+                      "version_id": 1,
+                      "minimum_version_id": 1,
+                      "Fields": [
+                        {
+                          "field": "media_changed",
+                          "version_id": 0,
+                          "field_exists": false,
+                          "size": 1
+                        }
+                      ]
+                    },
+                    {
+                      "name": "fdrive/media_rate",
+                      "version_id": 1,
+                      "minimum_version_id": 1,
+                      "Fields": [
+                        {
+                          "field": "media_rate",
+                          "version_id": 0,
+                          "field_exists": false,
+                          "size": 1
+                        }
+                      ]
+                    }
+                  ]
+                }
+              }
+            ]
+          }
+        }
+      ]
+    }
+  },
+  "usb-kbd": {
+    "Name": "usb-kbd",
+    "version_id": 1,
+    "minimum_version_id": 1,
+    "Description": {
+      "name": "usb-kbd",
+      "version_id": 1,
+      "minimum_version_id": 1,
+      "Fields": [
+        {
+          "field": "dev",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4352,
+          "Description": {
+            "name": "USBDevice",
+            "version_id": 1,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "addr",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "state",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "remote_wakeup",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "setup_state",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "setup_len",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "setup_index",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "setup_buf",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              }
+            ]
+          }
+        },
+        {
+          "field": "kbd.keycodes",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "head",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "n",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "kbd.modifiers",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 2
+        },
+        {
+          "field": "kbd.leds",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "kbd.key",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        },
+        {
+          "field": "kbd.keys",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "protocol",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 4
+        },
+        {
+          "field": "idle",
+          "version_id": 0,
+          "field_exists": false,
+          "size": 1
+        }
+      ]
+    }
+  }
+}
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 03/18] tests: vmstate static checker: add dump1 and dump2 files

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> These are stripped-down JSON data as obtained from the -dump-vmstate
> option.  The two files are identical in this commit, and will be
> modified in the later commits to show what the script does with the
> data.
>
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Acked-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 04/18] tests: vmstate static checker: incompat machine types

[Amit Shah]

This commit modifies the dump2 data to flag incompatibilities in the
machine types being compared.

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index 44200fb..0a8b81d 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -1,6 +1,6 @@
 {
   "vmschkmachine": {
-    "Name": "pc-i440fx-2.1"
+    "Name": "pc-i440fx-2.2"
   },
   "fw_cfg": {
     "Name": "fw_cfg",
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 04/18] tests: vmstate static checker: incompat machine types

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> This commit modifies the dump2 data to flag incompatibilities in the
> machine types being compared.
>
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Acked-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 05/18] tests: vmstate static checker: add version error in main section

[Amit Shah]

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index 0a8b81d..f405534 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -34,7 +34,7 @@
   },
   "fusbh200-ehci-usb": {
     "Name": "fusbh200-ehci-usb",
-    "version_id": 2,
+    "version_id": 1,
     "minimum_version_id": 1,
     "Description": {
       "name": "ehci-sysbus",
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 05/18] tests: vmstate static checker: add version error in main section

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Acked-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 06/18] tests: vmstate static checker: version mismatch inside a Description

[Amit Shah]

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index f405534..36a9b4b 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -829,7 +829,7 @@
     "minimum_version_id": 2,
     "Description": {
       "name": "fdc",
-      "version_id": 2,
+      "version_id": 1,
       "minimum_version_id": 2,
       "Fields": [
         {
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 06/18] tests: vmstate static checker: version mismatch inside a Description

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Signed-off-by: Amit Shah <amit.shah@redhat.com>
Acked-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 07/18] tests: vmstate static checker: minimum_version_id check

[Amit Shah]

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump1.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/vmstate-static-checker-data/dump1.json b/tests/vmstate-static-checker-data/dump1.json
index 44200fb..786ca0b 100644
--- a/tests/vmstate-static-checker-data/dump1.json
+++ b/tests/vmstate-static-checker-data/dump1.json
@@ -698,7 +698,7 @@
     "minimum_version_id": 2,
     "Description": {
       "name": "PIIX3",
-      "version_id": 3,
+      "version_id": 1,
       "minimum_version_id": 2,
       "Fields": [
         {
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 07/18] tests: vmstate static checker: minimum_version_id check

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 08/18] tests: vmstate static checker: remove a section

[Amit Shah]

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index 36a9b4b..4ccfd67 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -2,7 +2,7 @@
   "vmschkmachine": {
     "Name": "pc-i440fx-2.2"
   },
-  "fw_cfg": {
+  "fw_cfg2": {
     "Name": "fw_cfg",
     "version_id": 2,
     "minimum_version_id": 1,
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 08/18] tests: vmstate static checker: remove a section

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 09/18] tests: vmstate static checker: remove a field

[Amit Shah]

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index 4ccfd67..f6b52d0 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -58,12 +58,6 @@
                 "size": 4
               },
               {
-                "field": "usbsts",
-                "version_id": 0,
-                "field_exists": false,
-                "size": 4
-              },
-              {
                 "field": "usbsts_pending",
                 "version_id": 2,
                 "field_exists": false,
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 09/18] tests: vmstate static checker: remove a field

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 10/18] tests: vmstate static checker: remove last field in a struct

[Amit Shah]

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index f6b52d0..34bfbf6 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -637,12 +637,6 @@
                 "version_id": 0,
                 "field_exists": false,
                 "size": 256
-              },
-              {
-                "field": "irq_state",
-                "version_id": 2,
-                "field_exists": false,
-                "size": 16
               }
             ]
           }
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 10/18] tests: vmstate static checker: remove last field in a struct

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 11/18] tests: vmstate static checker: change description name

[Amit Shah]

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index 34bfbf6..f69966d 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -749,7 +749,7 @@
     "version_id": 1,
     "minimum_version_id": 1,
     "Description": {
-      "name": "tpci200",
+      "name": "tpci2002",
       "version_id": 1,
       "minimum_version_id": 1,
       "Fields": [
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 11/18] tests: vmstate static checker: change description name

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:

Reviewed-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 12/18] tests: vmstate static checker: remove Fields

[Amit Shah]

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 114 +--------------------------
 1 file changed, 1 insertion(+), 113 deletions(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index f69966d..cc0aae3 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -196,119 +196,7 @@
     "Description": {
       "name": "pci-serial-multi",
       "version_id": 1,
-      "minimum_version_id": 1,
-      "Fields": [
-        {
-          "field": "dev",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 1944,
-          "Description": {
-            "name": "PCIDevice",
-            "version_id": 2,
-            "minimum_version_id": 1,
-            "Fields": [
-              {
-                "field": "version_id",
-                "version_id": 0,
-                "field_exists": false,
-                "size": 4
-              },
-              {
-                "field": "config",
-                "version_id": 0,
-                "field_exists": false,
-                "size": 256
-              },
-              {
-                "field": "irq_state",
-                "version_id": 2,
-                "field_exists": false,
-                "size": 16
-              }
-            ]
-          }
-        },
-        {
-          "field": "state",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 368,
-          "Description": {
-            "name": "serial",
-            "version_id": 3,
-            "minimum_version_id": 2,
-            "Fields": [
-              {
-                "field": "divider",
-                "version_id": 2,
-                "field_exists": false,
-                "size": 2
-              },
-              {
-                "field": "rbr",
-                "version_id": 0,
-                "field_exists": false,
-                "size": 1
-              },
-              {
-                "field": "ier",
-                "version_id": 0,
-                "field_exists": false,
-                "size": 1
-              },
-              {
-                "field": "iir",
-                "version_id": 0,
-                "field_exists": false,
-                "size": 1
-              },
-              {
-                "field": "lcr",
-                "version_id": 0,
-                "field_exists": false,
-                "size": 1
-              },
-              {
-                "field": "mcr",
-                "version_id": 0,
-                "field_exists": false,
-                "size": 1
-              },
-              {
-                "field": "lsr",
-                "version_id": 0,
-                "field_exists": false,
-                "size": 1
-              },
-              {
-                "field": "msr",
-                "version_id": 0,
-                "field_exists": false,
-                "size": 1
-              },
-              {
-                "field": "scr",
-                "version_id": 0,
-                "field_exists": false,
-                "size": 1
-              },
-              {
-                "field": "fcr_vmstate",
-                "version_id": 3,
-                "field_exists": false,
-                "size": 1
-              }
-            ]
-          }
-        },
-        {
-          "field": "level",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 4
-        }
-      ]
+      "minimum_version_id": 1
     }
   },
   "intel-hda-generic": {
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 12/18] tests: vmstate static checker: remove Fields

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 13/18] tests: vmstate static checker: remove Description

[Amit Shah]

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 33 +---------------------------
 1 file changed, 1 insertion(+), 32 deletions(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index cc0aae3..66ac3bd 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -462,38 +462,7 @@
   "cfi.pflash01": {
     "Name": "cfi.pflash01",
     "version_id": 1,
-    "minimum_version_id": 1,
-    "Description": {
-      "name": "pflash_cfi01",
-      "version_id": 1,
-      "minimum_version_id": 1,
-      "Fields": [
-        {
-          "field": "wcycle",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 1
-        },
-        {
-          "field": "cmd",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 1
-        },
-        {
-          "field": "status",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 1
-        },
-        {
-          "field": "counter",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 8
-        }
-      ]
-    }
+    "minimum_version_id": 1
   },
   "megasas": {
     "Name": "megasas",
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 13/18] tests: vmstate static checker: remove Description

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 14/18] tests: vmstate static checker: remove Description inside Fields

[Amit Shah]

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 27 +--------------------------
 1 file changed, 1 insertion(+), 26 deletions(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index 66ac3bd..05b8fce 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -212,32 +212,7 @@
           "field": "pci",
           "version_id": 0,
           "field_exists": false,
-          "size": 1944,
-          "Description": {
-            "name": "PCIDevice",
-            "version_id": 2,
-            "minimum_version_id": 1,
-            "Fields": [
-              {
-                "field": "version_id",
-                "version_id": 0,
-                "field_exists": false,
-                "size": 4
-              },
-              {
-                "field": "config",
-                "version_id": 0,
-                "field_exists": false,
-                "size": 256
-              },
-              {
-                "field": "irq_state",
-                "version_id": 2,
-                "field_exists": false,
-                "size": 16
-              }
-            ]
-          }
+          "size": 1944
         },
         {
           "field": "g_ctl",
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 14/18] tests: vmstate static checker: remove Description inside Fields

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 15/18] tests: vmstate static checker: remove a subsection

[Amit Shah]

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 13 -------------
 1 file changed, 13 deletions(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index 05b8fce..6f8a617 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -836,19 +836,6 @@
                           "size": 1
                         }
                       ]
-                    },
-                    {
-                      "name": "fdrive/media_rate",
-                      "version_id": 1,
-                      "minimum_version_id": 1,
-                      "Fields": [
-                        {
-                          "field": "media_rate",
-                          "version_id": 0,
-                          "field_exists": false,
-                          "size": 1
-                        }
-                      ]
                     }
                   ]
                 }
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 15/18] tests: vmstate static checker: remove a subsection

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 16/18] tests: vmstate static checker: remove Subsections

[Amit Shah]

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 15 ---------------
 1 file changed, 15 deletions(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index 6f8a617..b5cb1aa 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -558,21 +558,6 @@
           "field_exists": false,
           "size": 4
         }
-      ],
-      "Subsections": [
-        {
-          "name": "PIIX3/rcr",
-          "version_id": 1,
-          "minimum_version_id": 1,
-          "Fields": [
-            {
-              "field": "rcr",
-              "version_id": 0,
-              "field_exists": false,
-              "size": 1
-            }
-          ]
-        }
       ]
     }
   },
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 16/18] tests: vmstate static checker: remove Subsections

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com> 

[Qemu-devel] [PATCH v4 17/18] tests: vmstate static checker: add substructure for usb-kbd for hid section

[Amit Shah]

This shows how the script deals with substructures added to vmstate
descriptions that don't change the on-wire format.

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 115 +++++++++++++++------------
 1 file changed, 64 insertions(+), 51 deletions(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index b5cb1aa..45cccaf 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -844,7 +844,7 @@
           "field": "dev",
           "version_id": 0,
           "field_exists": false,
-          "size": 4352,
+          "size": 5832,
           "Description": {
             "name": "USBDevice",
             "version_id": 1,
@@ -896,58 +896,71 @@
           }
         },
         {
-          "field": "kbd.keycodes",
+          "field": "hid",
           "version_id": 0,
           "field_exists": false,
-          "size": 4
-        },
-        {
-          "field": "head",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 4
-        },
-        {
-          "field": "n",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 4
-        },
-        {
-          "field": "kbd.modifiers",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 2
-        },
-        {
-          "field": "kbd.leds",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 1
-        },
-        {
-          "field": "kbd.key",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 1
-        },
-        {
-          "field": "kbd.keys",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 4
-        },
-        {
-          "field": "protocol",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 4
-        },
-        {
-          "field": "idle",
-          "version_id": 0,
-          "field_exists": false,
-          "size": 1
+          "size": 312,
+          "Description": {
+            "name": "HIDKeyboardDevice",
+            "version_id": 1,
+            "minimum_version_id": 1,
+            "Fields": [
+              {
+                "field": "kbd.keycodes",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "head",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "n",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "kbd.modifiers",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 2
+              },
+              {
+                "field": "kbd.leds",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "kbd.key",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              },
+              {
+                "field": "kbd.keys",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "protocol",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 4
+              },
+              {
+                "field": "idle",
+                "version_id": 0,
+                "field_exists": false,
+                "size": 1
+              }
+            ]
+          }
         }
       ]
     }
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 17/18] tests: vmstate static checker: add substructure for usb-kbd for hid section

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> This shows how the script deals with substructures added to vmstate
> descriptions that don't change the on-wire format.
>
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

[Qemu-devel] [PATCH v4 18/18] tests: vmstate static checker: add size mismatch inside substructure

[Amit Shah]

Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 tests/vmstate-static-checker-data/dump2.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/vmstate-static-checker-data/dump2.json b/tests/vmstate-static-checker-data/dump2.json
index 45cccaf..75719f5 100644
--- a/tests/vmstate-static-checker-data/dump2.json
+++ b/tests/vmstate-static-checker-data/dump2.json
@@ -909,7 +909,7 @@
                 "field": "kbd.keycodes",
                 "version_id": 0,
                 "field_exists": false,
-                "size": 4
+                "size": 2
               },
               {
                 "field": "head",
-- 
1.9.3

Re: [Qemu-devel] [PATCH v4 18/18] tests: vmstate static checker: add size mismatch inside substructure

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Signed-off-by: Amit Shah <amit.shah@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

Re: [Qemu-devel] [PATCH v4 00/18] migration: add static analysis tool to check vmstate compat

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Hello,
>
> v4:
>  - Return value capped to 255 to prevent wrap-around (Eric Blake)
>
> v3:
>  - Python script returns an error code: 0 if no errors, positive for
>    the number of errors identified.
>
> v2:
>  - Tabs->spaces (Dave Gilbert)
>  - Several changes to the python script to make it more python-like
>    (Vitaly Kuznetsov)
>  - Don't store the empty fields created by VMSTATE_VALIDATE in the
>    json output
>
> This series adds a static vmstate checker to check for breakage of
> live migration by analyzing the vmstate information between different
> QEMU versions.
>
> In patch 1, QEMU is modified to add a -dump-vmstate commandline
> option, which takes a filename as the argument.  When invoked, QEMU
> dumps the vmstate info in JSON format for the current machine type to
> the file.  This patch is loosely based on a version from Andreas
> Färber.
>
> This JSON file is then fed into the Python script introduced in patch
> 2.  The script takes 'src' and 'dest' arguments, indicating the
> direction of migration.  The script then performs a series of checks
> and spews out information on inconsistencies it finds in the data.
>
> Two stripped-down versions of JSON dumps are included in this
> patchset (patch 3).
>
> Patches 4 - 18 contain modifications to those dumps, to show the
> checks that are performed by the script.  The result of running the
> script against the final version of those files is appended below.
>
> The checks are to be performed for a particular machine type, and
> comparing different machine types is bound to turn up false-positives:
> e.g.
>
> (in a qemu 2.0 tree):
> ./x86_64-softmmu/qemu-system-x86_64 -dump-vmstate qemu-2.0.json
>
> (in a qemu 2.2 tree:)
> ./x86_64-softmmu/qemu-system-x86_64 -dump-vmstate -M pc-i440fx-2.0 \
>    qemu-2.2-m2.0.json
>
> ./scripts/vmstate-static-checker.py -s qemu-2.0.json -d qemu-2.2-m2.0.json
>
> should not show any output.

This usage should go at the beggining of the script or somewhere at Documantion/

> The idea is to include this script in 'make check', ensuring new
> commits don't break migration.

Can we add this as:

make check-vmstate-sections or whatever, and then we can discuss later
if integrate it on make check or not?

I would assume something like:
- dump things on qemu-2.0 (yes, need to be done by hand applying the
  first patch)
- dump things on current qemu as qemu-2.1, qemu-2.0.95 or whatever
- check what changed from 2.0 and form last saved from 2.1, so we can
  commit the changes that we are happy with and reverse the ones that we
  are not happy with

Makes sense?

Thanks, Juan.

Re: [Qemu-devel] [PATCH v4 00/18] migration: add static analysis tool to check vmstate compat

[Amit Shah]

On (Wed) 18 Jun 2014 [12:49:21], Juan Quintela wrote:
> Amit Shah <amit.shah@redhat.com> wrote:
> > Hello,
> >
> > v4:
> >  - Return value capped to 255 to prevent wrap-around (Eric Blake)
> >
> > v3:
> >  - Python script returns an error code: 0 if no errors, positive for
> >    the number of errors identified.
> >
> > v2:
> >  - Tabs->spaces (Dave Gilbert)
> >  - Several changes to the python script to make it more python-like
> >    (Vitaly Kuznetsov)
> >  - Don't store the empty fields created by VMSTATE_VALIDATE in the
> >    json output
> >
> > This series adds a static vmstate checker to check for breakage of
> > live migration by analyzing the vmstate information between different
> > QEMU versions.
> >
> > In patch 1, QEMU is modified to add a -dump-vmstate commandline
> > option, which takes a filename as the argument.  When invoked, QEMU
> > dumps the vmstate info in JSON format for the current machine type to
> > the file.  This patch is loosely based on a version from Andreas
> > Färber.
> >
> > This JSON file is then fed into the Python script introduced in patch
> > 2.  The script takes 'src' and 'dest' arguments, indicating the
> > direction of migration.  The script then performs a series of checks
> > and spews out information on inconsistencies it finds in the data.
> >
> > Two stripped-down versions of JSON dumps are included in this
> > patchset (patch 3).
> >
> > Patches 4 - 18 contain modifications to those dumps, to show the
> > checks that are performed by the script.  The result of running the
> > script against the final version of those files is appended below.
> >
> > The checks are to be performed for a particular machine type, and
> > comparing different machine types is bound to turn up false-positives:
> > e.g.
> >
> > (in a qemu 2.0 tree):
> > ./x86_64-softmmu/qemu-system-x86_64 -dump-vmstate qemu-2.0.json
> >
> > (in a qemu 2.2 tree:)
> > ./x86_64-softmmu/qemu-system-x86_64 -dump-vmstate -M pc-i440fx-2.0 \
> >    qemu-2.2-m2.0.json
> >
> > ./scripts/vmstate-static-checker.py -s qemu-2.0.json -d qemu-2.2-m2.0.json
> >
> > should not show any output.
> 
> This usage should go at the beggining of the script or somewhere at Documantion/

I've mentioned it in words instead of direct example; but sure I will
add it to the commit msg.

> > The idea is to include this script in 'make check', ensuring new
> > commits don't break migration.
> 
> Can we add this as:
> 
> make check-vmstate-sections or whatever, and then we can discuss later
> if integrate it on make check or not?

Yes, makes sense.

> I would assume something like:
> - dump things on qemu-2.0 (yes, need to be done by hand applying the
>   first patch)
> - dump things on current qemu as qemu-2.1, qemu-2.0.95 or whatever
> - check what changed from 2.0 and form last saved from 2.1, so we can
>   commit the changes that we are happy with and reverse the ones that we
>   are not happy with

I'm thinking each 'git push' gets us a upstream.json file somewhere,
so a 'make check' or 'make check-vmstate-sections' as you suggest
above compares against the current upstream, so the script flags any
breakage in this patch series (assuming 'make check' is run on top of
a patch series against origin/master).

If all goes according to plan (TM), we will never have a situation
where we commit stuff and then have to revert it -- because things
will be flagged in 'make check' before any push operation.


		Amit