qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH v3 00/18] migration: add static analysis tool to check vmstate compat
@ 2014-06-17 13:19 Amit Shah
  2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 01/18] migration: dump vmstate info as a json file for static analysis Amit Shah
                   ` (18 more replies)
  0 siblings, 19 replies; 22+ messages in thread
From: Amit Shah @ 2014-06-17 13:19 UTC (permalink / raw)
  To: qemu list
  Cc: Juan Quintela, Markus Armbruster, Alexander Graf,
	Dr. David Alan Gilbert, Amit Shah, Paolo Bonzini,
	Andreas Färber

Hello,

v3:
 - Python script returns an error code: 0 if no errors, positive for
   the number of errors identified.

v2:
 - Tabs->spaces (Dave Gilbert)
 - Several changes to the python script to make it more python-like
   (Vitaly Kuznetsov)
 - Don't store the empty fields created by VMSTATE_VALIDATE in the
   json output

This series adds a static vmstate checker to check for breakage of
live migration by analyzing the vmstate information between different
QEMU versions.

In patch 1, QEMU is modified to add a -dump-vmstate commandline
option, which takes a filename as the argument.  When invoked, QEMU
dumps the vmstate info in JSON format for the current machine type to
the file.  This patch is loosely based on a version from Andreas
Färber.

This JSON file is then fed into the Python script introduced in patch
2.  The script takes 'src' and 'dest' arguments, indicating the
direction of migration.  The script then performs a series of checks
and spews out information on inconsistencies it finds in the data.

Two stripped-down versions of JSON dumps are included in this
patchset (patch 3).

Patches 4 - 18 contain modifications to those dumps, to show the
checks that are performed by the script.  The result of running the
script against the final version of those files is appended below.

The checks are to be performed for a particular machine type, and
comparing different machine types is bound to turn up false-positives:
e.g.

(in a qemu 2.0 tree):
./x86_64-softmmu/qemu-system-x86_64 -dump-vmstate qemu-2.0.json

(in a qemu 2.2 tree:)
./x86_64-softmmu/qemu-system-x86_64 -dump-vmstate -M pc-i440fx-2.0 \
   qemu-2.2-m2.0.json

./scripts/vmstate-static-checker.py -s qemu-2.0.json -d qemu-2.2-m2.0.json

should not show any output.

The idea is to include this script in 'make check', ensuring new
commits don't break migration.

Later, this script can also be baked into the release process: vmstate
dumps for released versions of qemu for various machine types can be
stored in a directory in the tree.  At the time of freezing the tree
for releases, (like -rc2), the dump for the current release can be
updated.  A policy that says "No more live migration-breaking changes
can be accepted post -rc2" can be put in place.  Also, for checks for
older machine types on the current tree with the saved dumps should
not indicate any regressions.

I expect there to be a few false positives at the start (though there
aren't any right now).  The script will keep evolving, though, to
include new scenarios, to make it more helpful.

A later project would be to also store other guest-visible information
and use that output for more checks (including lspci output from
inside guests).

As mentioned earlier, this is the result of running the script against
the sample json data included in this patchset:

$ ./scripts/vmstate-static-checker.py --src ./tests/vmstate-static-checker-data/dump1.json  --dest ./tests/vmstate-static-checker-data/dump2.json 
Section "usb-kbd" Description "usb-kbd" Field "kbd.keycodes" size mismatch: 4 , 2
Section "PIIX3-xen" Description "PIIX3": minimum version error: 1 < 2
Section "PIIX3-xen" Description "PIIX3": Entry "Subsections" missing
Section "tpci200": Description "tpci200" missing, got "tpci2002" instead; skipping
Section "megasas", Description "PCIDevice": expected field "irq_state", while dest has no further fields
Section "SUNW,fdtwo" Description "fdc": version error: 2 > 1
Section "SUNW,fdtwo", Description "fdrive": Subsection "fdrive/media_rate" not found
Section "fusbh200-ehci-usb" version error: 2 > 1
Section "fusbh200-ehci-usb", Description "ehci-core": expected field "usbsts", got "usbsts_pending"; skipping rest
Section "intel-hda-generic", Description "intel-hda", Field "pci": missing description
Section "cfi.pflash01": Entry "Description" missing
Section "pci-serial-4x" Description "pci-serial-multi": Entry "Fields" missing
Warning: checking incompatible machine types: "pc-i440fx-2.1", "pc-i440fx-2.2"
Section "fw_cfg" does not exist in dest

$ echo $?
13

Amit Shah (18):
  migration: dump vmstate info as a json file for static analysis
  vmstate-static-checker: script to validate vmstate changes
  tests: vmstate static checker: add dump1 and dump2 files
  tests: vmstate static checker: incompat machine types
  tests: vmstate static checker: add version error in main section
  tests: vmstate static checker: version mismatch inside a Description
  tests: vmstate static checker: minimum_version_id check
  tests: vmstate static checker: remove a section
  tests: vmstate static checker: remove a field
  tests: vmstate static checker: remove last field in a struct
  tests: vmstate static checker: change description name
  tests: vmstate static checker: remove Fields
  tests: vmstate static checker: remove Description
  tests: vmstate static checker: remove Description inside Fields
  tests: vmstate static checker: remove a subsection
  tests: vmstate static checker: remove Subsections
  tests: vmstate static checker: add substructure for usb-kbd for hid
    section
  tests: vmstate static checker: add size mismatch inside substructure

 include/migration/vmstate.h                  |    2 +
 qemu-options.hx                              |    9 +
 savevm.c                                     |  139 +++
 scripts/vmstate-static-checker.py            |  326 ++++++++
 tests/vmstate-static-checker-data/dump1.json | 1163 ++++++++++++++++++++++++++
 tests/vmstate-static-checker-data/dump2.json |  968 +++++++++++++++++++++
 vl.c                                         |   13 +
 7 files changed, 2620 insertions(+)
 create mode 100755 scripts/vmstate-static-checker.py
 create mode 100644 tests/vmstate-static-checker-data/dump1.json
 create mode 100644 tests/vmstate-static-checker-data/dump2.json

-- 
1.9.3

^ permalink raw reply	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2014-06-17 15:25 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-06-17 13:19 [Qemu-devel] [PATCH v3 00/18] migration: add static analysis tool to check vmstate compat Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 01/18] migration: dump vmstate info as a json file for static analysis Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 02/18] vmstate-static-checker: script to validate vmstate changes Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 03/18] tests: vmstate static checker: add dump1 and dump2 files Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 04/18] tests: vmstate static checker: incompat machine types Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 05/18] tests: vmstate static checker: add version error in main section Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 06/18] tests: vmstate static checker: version mismatch inside a Description Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 07/18] tests: vmstate static checker: minimum_version_id check Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 08/18] tests: vmstate static checker: remove a section Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 09/18] tests: vmstate static checker: remove a field Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 10/18] tests: vmstate static checker: remove last field in a struct Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 11/18] tests: vmstate static checker: change description name Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 12/18] tests: vmstate static checker: remove Fields Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 13/18] tests: vmstate static checker: remove Description Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 14/18] tests: vmstate static checker: remove Description inside Fields Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 15/18] tests: vmstate static checker: remove a subsection Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 16/18] tests: vmstate static checker: remove Subsections Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 17/18] tests: vmstate static checker: add substructure for usb-kbd for hid section Amit Shah
2014-06-17 13:19 ` [Qemu-devel] [PATCH v3 18/18] tests: vmstate static checker: add size mismatch inside substructure Amit Shah
2014-06-17 13:34 ` [Qemu-devel] [PATCH v3 00/18] migration: add static analysis tool to check vmstate compat Eric Blake
2014-06-17 13:59   ` Amit Shah
2014-06-17 15:25     ` Eric Blake

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).