[PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization

[Wei Liu <wei.liu2@citrix.com>]

This series contains all patches necessary to fully preserve domain
configurations across save / restore.

It's divided to several parts:
1. first few patches are bugfixes, doc updates
2. JSON infrastructure to manipulate domain configuration
3. new "xl-json" format
4. update stored domain configuration as we update runtime domain configs

I've run test-amd64-amd64-xl and test-amd64-amd64-xl-qemuu-debianhvm-amd64
in OSSTest standalone mode, this series passed both testcases.

OSSTest is only able to test first 3 parts of the series. So I also did some
manual testing for device hotplug. The main problem discovered is that when a
device is removed, but the backend times out, the device is considered not
removed. But in fact in that case (backend timed out) the device is force
removed. This series contains a patch to fix that behavior.

And for the 4th part of this series, the synchronization is done in xl level.
AIUI libxl userdata store contains the data of from the user of this library,
so "xl-json" format is xl centric. It's also possible to push the
synchronization to libxl level, but that looks wrong to me. Correct me if I've
mistaken the usage of libxl userdata store.

Wei.

Changes in V4:
* minor bug fixes to parser
* push option parsing down to libxl level
* introduce deep copy functions
* handle domain configuration changes when setting memory targets
* handle domain configuration changes when plug / unplug device

Changes in V3:
* remove libxl__integer_parse_json and add functions for specific
  integer types
* relax some checkings on invalid inputs
* libxl__json_parse_callback now takes gc instead of ctx
* rewrite "allow basic JSON type objects generation"
* ... and address some other misc things mentioned by IanC

Changes in V2:
* address IanC's comments on JSON parser
* add "xl-json" format patch


Wei Liu (24):
  xl / libxl: push parsing of SSID and CPU pool ID down to libxl
  libxl: fix memory leak in libxl_cpuid_dispose
  libxl.h: document the paradigm of using libxl types
  libxl.h: move / add some libxl defbool #define here
  libxl IDL: rename json_fn to json_gen_fn
  libxl_json: introduce libx__object_from_json
  libxl_internal: make JSON_* types a bit-field
  libxl_internal: introduce libxl__json_object_is_{null,number,double}
  libxl_json: introduce parser functions for builtin types
  libxl_json: allow basic JSON type objects generation
  libxl/gentypes.py: include discriminator in JSON output
  libxl IDL: generate code to parse libxl__json_object to libxl_FOO
    struct
  libxl/gentest.py: test JSON parser
  libxl: introduce libxl_key_value_list_length
  libxl: introduce libxl_cpuid_policy_list_length
  libxl: copy function for builtin types
  libxl IDL: generate deep copy functions
  libxl/gentest.py: test deep copy functions
  xl: introduce and use "xl-json" format
  xl: introduce load/save_domain_config
  xl: update domain configuration when running mem-set and mem-max
  xl: update domain configuration when we hotplug a device
  libxl: consider force removal of device successful
  xl: update configuration when we unplug a device

 docs/man/xl.cfg.pod.5                |    3 +-
 docs/man/xl.pod.1                    |   21 +
 tools/libxl/gentest.py               |   64 ++-
 tools/libxl/gentypes.py              |  198 +++++++++-
 tools/libxl/idl.py                   |   38 +-
 tools/libxl/idl.txt                  |   21 +-
 tools/libxl/libxl.c                  |  106 ++++-
 tools/libxl/libxl.h                  |   40 +-
 tools/libxl/libxl_cpuid.c            |  131 ++++++-
 tools/libxl/libxl_create.c           |   57 +++
 tools/libxl/libxl_device.c           |   12 +-
 tools/libxl/libxl_dm.c               |    4 +
 tools/libxl/libxl_internal.h         |   44 ++-
 tools/libxl/libxl_json.c             |  355 ++++++++++++++---
 tools/libxl/libxl_json.h             |   36 +-
 tools/libxl/libxl_nocpuid.c          |   13 +
 tools/libxl/libxl_types.idl          |   43 +-
 tools/libxl/libxl_types_internal.idl |    4 +-
 tools/libxl/libxl_utils.c            |   56 +++
 tools/libxl/libxl_utils.h            |    7 +
 tools/libxl/libxl_uuid.c             |   12 +
 tools/libxl/libxl_uuid.h             |    4 +
 tools/libxl/xl_cmdimpl.c             |  715 ++++++++++++++++++++++++----------
 tools/libxl/xl_cmdtable.c            |   11 +-
 tools/libxl/xl_sxp.c                 |    7 +-
 25 files changed, 1662 insertions(+), 340 deletions(-)

-- 
1.7.10.4