* [PATCH 00 of 10] libxl: IDL improvements and JSON generation
@ 2011-06-23 16:20 Ian Campbell
2011-06-23 16:20 ` [PATCH 01 of 10] build: define "move-if-changed" make macro Ian Campbell
` (10 more replies)
0 siblings, 11 replies; 13+ messages in thread
From: Ian Campbell @ 2011-06-23 16:20 UTC (permalink / raw)
To: xen-devel; +Cc: Ian Campbell
This patch series makes some improvements to the IDL language and
generator and then goes on to implement JSON generation for IDL
defined datatypes.
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH 01 of 10] build: define "move-if-changed" make macro 2011-06-23 16:20 [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Campbell @ 2011-06-23 16:20 ` Ian Campbell 2011-06-23 16:20 ` [PATCH 02 of 10] libxl: IDL: s/anonynous/anonymous/ Ian Campbell ` (9 subsequent siblings) 10 siblings, 0 replies; 13+ messages in thread From: Ian Campbell @ 2011-06-23 16:20 UTC (permalink / raw) To: xen-devel; +Cc: Ian Campbell # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1308843343 -3600 # Node ID edb8e868a964365a3ffe5c7d17467f9f08bf2803 # Parent 4fa92ff6c6156c2723d716a48c041dcb2c92938b build: define "move-if-changed" make macro. Use it to replace various places which (should) use the if ! cmp -s ...; then mv ....; fi pattern. Also add an else clause to cleanup the unchanged temporary file. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 4fa92ff6c615 -r edb8e868a964 Config.mk --- a/Config.mk Thu Jun 23 16:35:43 2011 +0100 +++ b/Config.mk Thu Jun 23 16:35:43 2011 +0100 @@ -118,6 +118,16 @@ define buildmakevars2shellvars export XEN_ROOT="$(XEN_ROOT)" endef +# +# Compare $(1) and $(2) and replace $(2) with $(1) if they differ +# +# Typically $(1) is a newly generated file and $(2) is the target file +# being regenerated. This prevents changing the timestamp of $(2) only +# due to being auto regenereated with the same contents. +define move-if-changed + if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi +endef + buildmakevars2file = $(eval $(call buildmakevars2file-closure,$(1))) define buildmakevars2file-closure .PHONY: genpath @@ -133,7 +143,7 @@ define buildmakevars2file-closure echo "XEN_CONFIG_DIR=\"$(XEN_CONFIG_DIR)\"" >> $(1).tmp; \ echo "XEN_SCRIPT_DIR=\"$(XEN_SCRIPT_DIR)\"" >> $(1).tmp; \ echo "XEN_LOCK_DIR=\"$(XEN_LOCK_DIR)\"" >> $(1).tmp; \ - if ! cmp $(1).tmp $(1); then mv -f $(1).tmp $(1); fi + $(call move-if-changed,$(1).tmp,$(1)) endef ifeq ($(debug),y) diff -r 4fa92ff6c615 -r edb8e868a964 tools/libxl/Makefile --- a/tools/libxl/Makefile Thu Jun 23 16:35:43 2011 +0100 +++ b/tools/libxl/Makefile Thu Jun 23 16:35:43 2011 +0100 @@ -74,7 +74,8 @@ genpath-target = $(call buildmakevars2fi _libxl_paths.h: genpath sed -e "s/\([^=]*\)=\(.*\)/#define \1 \2/g" $@.tmp >$@.2.tmp - if ! cmp -s $@.2.tmp $@; then mv -f $@.2.tmp $@; fi + rm -f $@.tmp + $(call move-if-changed,$@.2.tmp,$@) libxl_paths.c: _libxl_paths.h @@ -84,8 +85,8 @@ libxl.h: _libxl_types.h _libxl_%.h _libxl_%.c: libxl.idl gen%.py libxl%.py $(PYTHON) gen$*.py libxl.idl __libxl_$*.h __libxl_$*.c - mv __libxl_$*.h _libxl_$*.h - mv __libxl_$*.c _libxl_$*.c + $(call move-if-changed,__libxl_$*.h,_libxl_$*.h) + $(call move-if-changed,__libxl_$*.c,_libxl_$*.c) libxenlight.so: libxenlight.so.$(MAJOR) ln -sf $< $@ ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 02 of 10] libxl: IDL: s/anonynous/anonymous/ 2011-06-23 16:20 [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Campbell 2011-06-23 16:20 ` [PATCH 01 of 10] build: define "move-if-changed" make macro Ian Campbell @ 2011-06-23 16:20 ` Ian Campbell 2011-06-23 16:20 ` [PATCH 03 of 10] libxl: IDL: handle generation of pass-by-reference arguments Ian Campbell ` (8 subsequent siblings) 10 siblings, 0 replies; 13+ messages in thread From: Ian Campbell @ 2011-06-23 16:20 UTC (permalink / raw) To: xen-devel; +Cc: Ian Campbell # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1308845904 -3600 # Node ID ba383ce75f8bba3e08a84c660c8383c26d23aaa8 # Parent edb8e868a964365a3ffe5c7d17467f9f08bf2803 libxl: IDL: s/anonynous/anonymous/ Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r edb8e868a964 -r ba383ce75f8b tools/libxl/gentypes.py --- a/tools/libxl/gentypes.py Thu Jun 23 16:35:43 2011 +0100 +++ b/tools/libxl/gentypes.py Thu Jun 23 17:18:24 2011 +0100 @@ -101,7 +101,7 @@ def libxl_C_type_destroy(ty, v, indent = elif isinstance(ty, libxltypes.Struct) and (parent is None or ty.destructor_fn is None): for f in [f for f in ty.fields if not f.const]: - if f.name is None: # Anonynous struct + if f.name is None: # Anonymous struct s += libxl_C_type_destroy(f.type, deref, "", deref) else: s += libxl_C_type_destroy(f.type, deref + f.name, "", deref) ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 03 of 10] libxl: IDL: handle generation of pass-by-reference arguments 2011-06-23 16:20 [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Campbell 2011-06-23 16:20 ` [PATCH 01 of 10] build: define "move-if-changed" make macro Ian Campbell 2011-06-23 16:20 ` [PATCH 02 of 10] libxl: IDL: s/anonynous/anonymous/ Ian Campbell @ 2011-06-23 16:20 ` Ian Campbell 2011-06-23 16:20 ` [PATCH 04 of 10] libxl: IDL: refactor code to massage a type into a function argument Ian Campbell ` (7 subsequent siblings) 10 siblings, 0 replies; 13+ messages in thread From: Ian Campbell @ 2011-06-23 16:20 UTC (permalink / raw) To: xen-devel; +Cc: Ian Campbell # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1308845917 -3600 # Node ID 4186a6ca6799797e24fafa5fe39e55b474b86bf0 # Parent ba383ce75f8bba3e08a84c660c8383c26d23aaa8 libxl: IDL: handle generation of pass-by-reference arguments. Up until now everything with a destructor function happened to be pass-by-reference so the current code worked but this will not be the case for *_to_string and *_to_json. Put some infrastructure in place and use it. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r ba383ce75f8b -r 4186a6ca6799 tools/libxl/gentypes.py --- a/tools/libxl/gentypes.py Thu Jun 23 17:18:24 2011 +0100 +++ b/tools/libxl/gentypes.py Thu Jun 23 17:18:37 2011 +0100 @@ -178,10 +178,10 @@ if __name__ == '__main__': for ty in types: f.write(libxl_C_type_define(ty) + ";\n") if ty.destructor_fn is not None: - f.write("void %s(%s *p);\n" % (ty.destructor_fn, ty.typename)) + f.write("void %s(%s);\n" % (ty.destructor_fn, ty.make_arg("p"))) if isinstance(ty, libxltypes.Enumeration): - f.write("const char *%s_to_string(%s e);\n" % (ty.typename, ty.typename)) - f.write("int %s_from_string(const char *s, %s *e);\n" % (ty.typename, ty.typename)) + f.write("const char *%s_to_string(%s);\n" % (ty.typename, ty.make_arg("p"))) + f.write("int %s_from_string(const char *s, %s);\n" % (ty.typename, ty.make_arg("e", passby=libxltypes.PASS_BY_REFERENCE))) f.write("extern libxl_enum_string_table %s_string_table[];\n" % (ty.typename)) f.write("\n") @@ -213,7 +213,7 @@ if __name__ == '__main__': """ % " ".join(sys.argv)) for ty in [t for t in types if t.destructor_fn is not None and t.autogenerate_destructor]: - f.write("void %s(%s *p)\n" % (ty.destructor_fn, ty.typename)) + f.write("void %s(%s)\n" % (ty.destructor_fn, ty.make_arg("p"))) f.write("{\n") f.write(libxl_C_type_destroy(ty, "p")) f.write(" memset(p, LIBXL_DTOR_POISON, sizeof(*p));\n") diff -r ba383ce75f8b -r 4186a6ca6799 tools/libxl/libxl.idl --- a/tools/libxl/libxl.idl Thu Jun 23 17:18:24 2011 +0100 +++ b/tools/libxl/libxl.idl Thu Jun 23 17:18:37 2011 +0100 @@ -4,8 +4,8 @@ # libxl_domid = Builtin("domid") -libxl_uuid = Builtin("uuid") -libxl_mac = Builtin("mac") +libxl_uuid = Builtin("uuid", passby=PASS_BY_REFERENCE) +libxl_mac = Builtin("mac", passby=PASS_BY_REFERENCE) libxl_cpumap = Builtin("cpumap", destructor_fn="libxl_cpumap_destroy", passby=PASS_BY_REFERENCE) libxl_cpuarray = Builtin("cpuarray", destructor_fn="libxl_cpuarray_destroy", passby=PASS_BY_REFERENCE) libxl_cpuid_policy_list = Builtin("cpuid_policy_list", destructor_fn="libxl_cpuid_destroy", passby=PASS_BY_REFERENCE) diff -r ba383ce75f8b -r 4186a6ca6799 tools/libxl/libxltypes.py --- a/tools/libxl/libxltypes.py Thu Jun 23 17:18:24 2011 +0100 +++ b/tools/libxl/libxltypes.py Thu Jun 23 17:18:37 2011 +0100 @@ -42,6 +42,14 @@ class Type(object): def marshal_out(self): return self.dir in [DIR_OUT, DIR_BOTH] + def make_arg(self, n, passby=None): + if passby is None: passby = self.passby + + if passby == PASS_BY_REFERENCE: + return "%s *%s" % (self.typename, n) + else: + return "%s %s" % (self.typename, n) + class Builtin(Type): """Builtin type""" def __init__(self, typename, **kwargs): ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 04 of 10] libxl: IDL: refactor code to massage a type into a function argument 2011-06-23 16:20 [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Campbell ` (2 preceding siblings ...) 2011-06-23 16:20 ` [PATCH 03 of 10] libxl: IDL: handle generation of pass-by-reference arguments Ian Campbell @ 2011-06-23 16:20 ` Ian Campbell 2011-06-23 16:20 ` [PATCH 05 of 10] libxl: IDL: remove libxl_C_type_of Ian Campbell ` (6 subsequent siblings) 10 siblings, 0 replies; 13+ messages in thread From: Ian Campbell @ 2011-06-23 16:20 UTC (permalink / raw) To: xen-devel; +Cc: Ian Campbell # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1308845927 -3600 # Node ID 34279f2359cbaaef7c16287e023cae1b023f02d3 # Parent 4186a6ca6799797e24fafa5fe39e55b474b86bf0 libxl: IDL: refactor code to massage a type into a function argument Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 4186a6ca6799 -r 34279f2359cb tools/libxl/gentypes.py --- a/tools/libxl/gentypes.py Thu Jun 23 17:18:37 2011 +0100 +++ b/tools/libxl/gentypes.py Thu Jun 23 17:18:47 2011 +0100 @@ -84,11 +84,6 @@ def libxl_C_type_destroy(ty, v, indent = else: deref = v + "." - if ty.passby == libxltypes.PASS_BY_REFERENCE and parent is not None: - makeref = "&" - else: - makeref = "" - s = "" if isinstance(ty, libxltypes.KeyedUnion): if parent is None: @@ -107,7 +102,7 @@ def libxl_C_type_destroy(ty, v, indent = s += libxl_C_type_destroy(f.type, deref + f.name, "", deref) else: if ty.destructor_fn is not None: - s += "%s(%s);\n" % (ty.destructor_fn, makeref + v) + s += "%s(%s);\n" % (ty.destructor_fn, ty.pass_arg(v, parent is None)) if s != "": s = indent + s diff -r 4186a6ca6799 -r 34279f2359cb tools/libxl/libxltypes.py --- a/tools/libxl/libxltypes.py Thu Jun 23 17:18:37 2011 +0100 +++ b/tools/libxl/libxltypes.py Thu Jun 23 17:18:47 2011 +0100 @@ -50,6 +50,21 @@ class Type(object): else: return "%s %s" % (self.typename, n) + def pass_arg(self, n, isref=None, passby=None): + if passby is None: passby = self.passby + if isref is None: isref = self.passby == PASS_BY_REFERENCE + + if passby == PASS_BY_REFERENCE: + if isref: + return "%s" % (n) + else: + return "&%s" % (n) + else: + if isref: + return "*%s" % (n) + else: + return "%s" % (n) + class Builtin(Type): """Builtin type""" def __init__(self, typename, **kwargs): ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 05 of 10] libxl: IDL: remove libxl_C_type_of 2011-06-23 16:20 [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Campbell ` (3 preceding siblings ...) 2011-06-23 16:20 ` [PATCH 04 of 10] libxl: IDL: refactor code to massage a type into a function argument Ian Campbell @ 2011-06-23 16:20 ` Ian Campbell 2011-06-23 16:20 ` [PATCH 06 of 10] libxl: IDL: add helper to generate references to Aggregate type members Ian Campbell ` (5 subsequent siblings) 10 siblings, 0 replies; 13+ messages in thread From: Ian Campbell @ 2011-06-23 16:20 UTC (permalink / raw) To: xen-devel; +Cc: Ian Campbell # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1308845928 -3600 # Node ID 2b91546fb417458d3fa29da85aed44bddb91854f # Parent 34279f2359cbaaef7c16287e023cae1b023f02d3 libxl: IDL: remove libxl_C_type_of It's not really adding much... Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 34279f2359cb -r 2b91546fb417 tools/libxl/gentypes.py --- a/tools/libxl/gentypes.py Thu Jun 23 17:18:47 2011 +0100 +++ b/tools/libxl/gentypes.py Thu Jun 23 17:18:48 2011 +0100 @@ -17,10 +17,7 @@ def format_comment(level, comment): s += "%s */" % indent s += "\n" return s - -def libxl_C_type_of(ty): - return ty.typename - + def libxl_C_instance_of(ty, instancename): if isinstance(ty, libxltypes.Aggregate) and ty.typename is None: if instancename is None: @@ -28,7 +25,7 @@ def libxl_C_instance_of(ty, instancename else: return libxl_C_type_define(ty) + " " + instancename else: - return libxl_C_type_of(ty) + " " + instancename + return ty.typename + " " + instancename def libxl_C_type_define(ty, indent = ""): s = "" ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 06 of 10] libxl: IDL: add helper to generate references to Aggregate type members 2011-06-23 16:20 [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Campbell ` (4 preceding siblings ...) 2011-06-23 16:20 ` [PATCH 05 of 10] libxl: IDL: remove libxl_C_type_of Ian Campbell @ 2011-06-23 16:20 ` Ian Campbell 2011-06-23 16:20 ` [PATCH 07 of 10] libxl: IDL: bring command line handling in genwrap.py into one place Ian Campbell ` (4 subsequent siblings) 10 siblings, 0 replies; 13+ messages in thread From: Ian Campbell @ 2011-06-23 16:20 UTC (permalink / raw) To: xen-devel; +Cc: Ian Campbell # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1308845933 -3600 # Node ID 273be439b67afbeefdf00e68efb5b09412a25c82 # Parent 2b91546fb417458d3fa29da85aed44bddb91854f libxl: IDL: add helper to generate references to Aggregate type members. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 2b91546fb417 -r 273be439b67a tools/libxl/gentypes.py --- a/tools/libxl/gentypes.py Thu Jun 23 17:18:48 2011 +0100 +++ b/tools/libxl/gentypes.py Thu Jun 23 17:18:53 2011 +0100 @@ -76,27 +76,21 @@ def libxl_C_type_define(ty, indent = "") return s.replace("\n", "\n%s" % indent) def libxl_C_type_destroy(ty, v, indent = " ", parent = None): - if parent is None: - deref = v + "->" - else: - deref = v + "." s = "" if isinstance(ty, libxltypes.KeyedUnion): if parent is None: raise Exception("KeyedUnion type must have a parent") for f in ty.fields: + (nparent,fexpr) = ty.member(v, f, parent is None) keyvar_expr = f.keyvar_expr % (parent + ty.keyvar_name) s += "if (" + keyvar_expr + ") {\n" - s += libxl_C_type_destroy(f.type, deref + f.name, indent + " ", deref) + s += libxl_C_type_destroy(f.type, fexpr, indent + " ", nparent) s += "}\n" elif isinstance(ty, libxltypes.Struct) and (parent is None or ty.destructor_fn is None): for f in [f for f in ty.fields if not f.const]: - - if f.name is None: # Anonymous struct - s += libxl_C_type_destroy(f.type, deref, "", deref) - else: - s += libxl_C_type_destroy(f.type, deref + f.name, "", deref) + (nparent,fexpr) = ty.member(v, f, parent is None) + s += libxl_C_type_destroy(f.type, fexpr, "", nparent) else: if ty.destructor_fn is not None: s += "%s(%s);\n" % (ty.destructor_fn, ty.pass_arg(v, parent is None)) diff -r 2b91546fb417 -r 273be439b67a tools/libxl/libxltypes.py --- a/tools/libxl/libxltypes.py Thu Jun 23 17:18:48 2011 +0100 +++ b/tools/libxl/libxltypes.py Thu Jun 23 17:18:53 2011 +0100 @@ -147,6 +147,24 @@ class Aggregate(Type): n,t,const,comment = f self.fields.append(Field(t,n,const=const,comment=comment)) + # Returns a tuple (stem, field-expr) + # + # field-expr is a C expression for a field "f" within the struct + # "v". + # + # stem is the stem common to both "f" and any other sibbling field + # within the "v". + def member(self, v, f, isref): + if isref: + deref = v + "->" + else: + deref = v + "." + + if f.name is None: # Anonymous + return (deref, deref) + else: + return (deref, deref + f.name) + class Struct(Aggregate): def __init__(self, name, fields, **kwargs): kwargs.setdefault('passby', PASS_BY_REFERENCE) ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 07 of 10] libxl: IDL: bring command line handling in genwrap.py into one place 2011-06-23 16:20 [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Campbell ` (5 preceding siblings ...) 2011-06-23 16:20 ` [PATCH 06 of 10] libxl: IDL: add helper to generate references to Aggregate type members Ian Campbell @ 2011-06-23 16:20 ` Ian Campbell 2011-06-23 16:20 ` [PATCH 08 of 10] libxl: rename testenum->testidl Ian Campbell ` (3 subsequent siblings) 10 siblings, 0 replies; 13+ messages in thread From: Ian Campbell @ 2011-06-23 16:20 UTC (permalink / raw) To: xen-devel; +Cc: Ian Campbell # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1308845948 -3600 # Node ID 956eef6b25e72a6fb76c43813c4367c29a9860cb # Parent 273be439b67afbeefdf00e68efb5b09412a25c82 libxl: IDL: bring command line handling in genwrap.py into one place. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 273be439b67a -r 956eef6b25e7 tools/libxl/gentypes.py --- a/tools/libxl/gentypes.py Thu Jun 23 17:18:53 2011 +0100 +++ b/tools/libxl/gentypes.py Thu Jun 23 17:19:08 2011 +0100 @@ -137,14 +137,14 @@ def libxl_C_enum_from_string(ty, str, e, if __name__ == '__main__': - if len(sys.argv) < 4: + if len(sys.argv) != 4: print >>sys.stderr, "Usage: gentypes.py <idl> <header> <implementation>" sys.exit(1) - idl = sys.argv[1] + (_, idl, header, impl) = sys.argv + (_,types) = libxltypes.parse(idl) - header = sys.argv[2] print "outputting libxl type definitions to %s" % header f = open(header, "w") @@ -174,7 +174,6 @@ if __name__ == '__main__': f.write("""#endif /* __LIBXL_TYPES_H */\n""") f.close() - impl = sys.argv[3] print "outputting libxl type implementations to %s" % impl f = open(impl, "w") ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 08 of 10] libxl: rename testenum->testidl 2011-06-23 16:20 [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Campbell ` (6 preceding siblings ...) 2011-06-23 16:20 ` [PATCH 07 of 10] libxl: IDL: bring command line handling in genwrap.py into one place Ian Campbell @ 2011-06-23 16:20 ` Ian Campbell 2011-06-23 16:21 ` [PATCH 09 of 10] libxl: add LIBXL_MAC_{FMT, FMTLEN, BYTES} Ian Campbell ` (2 subsequent siblings) 10 siblings, 0 replies; 13+ messages in thread From: Ian Campbell @ 2011-06-23 16:20 UTC (permalink / raw) To: xen-devel; +Cc: Ian Campbell # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1308845949 -3600 # Node ID dea9208b98db2b1b301a17b13d21558fa0a0edc7 # Parent 956eef6b25e72a6fb76c43813c4367c29a9860cb libxl: rename testenum->testidl I plan to add some non-Enumeration tests. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 956eef6b25e7 -r dea9208b98db .hgignore --- a/.hgignore Thu Jun 23 17:19:08 2011 +0100 +++ b/.hgignore Thu Jun 23 17:19:09 2011 +0100 @@ -187,8 +187,8 @@ ^tools/libxl/_.*\.c$ ^tools/libxl/libxlu_cfg_y\.output$ ^tools/libxl/xl$ -^tools/libxl/testenum$ -^tools/libxl/testenum\.c$ +^tools/libxl/testidl$ +^tools/libxl/testidl\.c$ ^tools/libaio/src/.*\.ol$ ^tools/libaio/src/.*\.os$ ^tools/misc/cpuperf/cpuperf-perfcntr$ diff -r 956eef6b25e7 -r dea9208b98db tools/libxl/Makefile --- a/tools/libxl/Makefile Thu Jun 23 17:19:08 2011 +0100 +++ b/tools/libxl/Makefile Thu Jun 23 17:19:09 2011 +0100 @@ -44,16 +44,16 @@ AUTOSRCS= libxlu_cfg_y.c libxlu_cfg_l.c LIBXLU_OBJS = libxlu_cfg_y.o libxlu_cfg_l.o libxlu_cfg.o $(LIBXLU_OBJS): CFLAGS += $(CFLAGS_libxenctrl) # For xentoollog.h -CLIENTS = xl testenum +CLIENTS = xl testidl XL_OBJS = xl.o xl_cmdimpl.o xl_cmdtable.o $(XL_OBJS): CFLAGS += $(CFLAGS_libxenctrl) # For xentoollog.h $(XL_OBJS): CFLAGS += $(CFLAGS_libxenlight) -testenum.o: CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenlight) -testenum.c: libxl.idl gentest.py - $(PYTHON) gentest.py libxl.idl testenum.c.new - mv testenum.c.new testenum.c +testidl.o: CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenlight) +testidl.c: libxl.idl gentest.py + $(PYTHON) gentest.py libxl.idl testidl.c.new + mv testidl.c.new testidl.c .PHONY: all all: $(CLIENTS) libxenlight.so libxenlight.a libxlutil.so libxlutil.a \ @@ -115,8 +115,8 @@ libxlutil.a: $(LIBXLU_OBJS) xl: $(XL_OBJS) libxlutil.so libxenlight.so $(CC) $(LDFLAGS) -o $@ $(XL_OBJS) libxlutil.so $(LDLIBS_libxenlight) $(LDLIBS_libxenctrl) -testenum: testenum.o libxlutil.so libxenlight.so - $(CC) $(LDFLAGS) -o $@ testenum.o libxlutil.so $(LDLIBS_libxenlight) $(LDLIBS_libxenctrl) +testidl: testidl.o libxlutil.so libxenlight.so + $(CC) $(LDFLAGS) -o $@ testidl.o libxlutil.so $(LDLIBS_libxenlight) $(LDLIBS_libxenctrl) .PHONY: install install: all ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 09 of 10] libxl: add LIBXL_MAC_{FMT, FMTLEN, BYTES} 2011-06-23 16:20 [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Campbell ` (7 preceding siblings ...) 2011-06-23 16:20 ` [PATCH 08 of 10] libxl: rename testenum->testidl Ian Campbell @ 2011-06-23 16:21 ` Ian Campbell 2011-06-23 16:21 ` [PATCH 10 of 10] libxl: IDL: autogenerate functions to produce JSON from libxl data structures Ian Campbell 2011-06-24 14:54 ` [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Jackson 10 siblings, 0 replies; 13+ messages in thread From: Ian Campbell @ 2011-06-23 16:21 UTC (permalink / raw) To: xen-devel; +Cc: Ian Campbell # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1308845956 -3600 # Node ID 4e751a05df3c323aea8fb291961a21d13de4afda # Parent dea9208b98db2b1b301a17b13d21558fa0a0edc7 libxl: add LIBXL_MAC_{FMT,FMTLEN,BYTES} Modelled after LIBXL_UUID_... (where I also add FMTLEN). signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r dea9208b98db -r 4e751a05df3c tools/libxl/libxl.c --- a/tools/libxl/libxl.c Thu Jun 23 17:19:09 2011 +0100 +++ b/tools/libxl/libxl.c Thu Jun 23 17:19:16 2011 +0100 @@ -1284,9 +1284,8 @@ int libxl_device_nic_add(libxl_ctx *ctx, nic->script)); } flexarray_append(back, "mac"); - flexarray_append(back, libxl__sprintf(&gc, "%02x:%02x:%02x:%02x:%02x:%02x", - nic->mac[0], nic->mac[1], nic->mac[2], - nic->mac[3], nic->mac[4], nic->mac[5])); + flexarray_append(back,libxl__sprintf(&gc, + LIBXL_MAC_FMT, LIBXL_MAC_BYTES(nic->mac))); if (nic->ip) { flexarray_append(back, "ip"); flexarray_append(back, libxl__strdup(&gc, nic->ip)); @@ -1304,9 +1303,8 @@ int libxl_device_nic_add(libxl_ctx *ctx, flexarray_append(front, "handle"); flexarray_append(front, libxl__sprintf(&gc, "%d", nic->devid)); flexarray_append(front, "mac"); - flexarray_append(front, libxl__sprintf(&gc, "%02x:%02x:%02x:%02x:%02x:%02x", - nic->mac[0], nic->mac[1], nic->mac[2], - nic->mac[3], nic->mac[4], nic->mac[5])); + flexarray_append(front, libxl__sprintf(&gc, + LIBXL_MAC_FMT, LIBXL_MAC_BYTES(nic->mac))); libxl__device_generic_add(&gc, &device, libxl__xs_kvs_of_flexarray(&gc, back, back->count), libxl__xs_kvs_of_flexarray(&gc, front, front->count)); diff -r dea9208b98db -r 4e751a05df3c tools/libxl/libxl.h --- a/tools/libxl/libxl.h Thu Jun 23 17:19:09 2011 +0100 +++ b/tools/libxl/libxl.h Thu Jun 23 17:19:16 2011 +0100 @@ -139,6 +139,9 @@ #include <libxl_uuid.h> typedef uint8_t libxl_mac[6]; +#define LIBXL_MAC_FMT "%02hhx:%02hhx:%02hhx:%02hhx:%02hhx:%02hhx" +#define LIBXL_MAC_FMTLEN ((2*6)+5) /* 6 hex bytes plus 5 colons */ +#define LIBXL_MAC_BYTES(mac) mac[0], mac[1], mac[2], mac[3], mac[4], mac[5] typedef char **libxl_string_list; void libxl_string_list_destroy(libxl_string_list *sl); diff -r dea9208b98db -r 4e751a05df3c tools/libxl/libxl_dm.c --- a/tools/libxl/libxl_dm.c Thu Jun 23 17:19:09 2011 +0100 +++ b/tools/libxl/libxl_dm.c Thu Jun 23 17:19:16 2011 +0100 @@ -177,9 +177,8 @@ static char ** libxl__build_device_model } for (i = 0; i < num_vifs; i++) { if (vifs[i].nictype == LIBXL_NIC_TYPE_IOEMU) { - char *smac = libxl__sprintf(gc, "%02x:%02x:%02x:%02x:%02x:%02x", - vifs[i].mac[0], vifs[i].mac[1], vifs[i].mac[2], - vifs[i].mac[3], vifs[i].mac[4], vifs[i].mac[5]); + char *smac = libxl__sprintf(gc, + LIBXL_MAC_FMT, LIBXL_MAC_BYTES(vifs[i].mac)); char *ifname; if (!vifs[i].ifname) ifname = libxl__sprintf(gc, "tap%d.%d", info->domid, vifs[i].devid); @@ -368,9 +367,8 @@ static char ** libxl__build_device_model } for (i = 0; i < num_vifs; i++) { if (vifs[i].nictype == LIBXL_NIC_TYPE_IOEMU) { - char *smac = libxl__sprintf(gc, "%02x:%02x:%02x:%02x:%02x:%02x", - vifs[i].mac[0], vifs[i].mac[1], vifs[i].mac[2], - vifs[i].mac[3], vifs[i].mac[4], vifs[i].mac[5]); + char *smac = libxl__sprintf(gc, + LIBXL_MAC_FMT, LIBXL_MAC_BYTES(vifs[i].mac)); char *ifname; if (!vifs[i].ifname) { ifname = libxl__sprintf(gc, "tap%d.%d", info->domid, vifs[i].devid); diff -r dea9208b98db -r 4e751a05df3c tools/libxl/libxl_uuid.h --- a/tools/libxl/libxl_uuid.h Thu Jun 23 17:19:09 2011 +0100 +++ b/tools/libxl/libxl_uuid.h Thu Jun 23 17:19:16 2011 +0100 @@ -16,6 +16,7 @@ #define __LIBXL_UUID_H__ #define LIBXL_UUID_FMT "%02hhx%02hhx%02hhx%02hhx-%02hhx%02hhx-%02hhx%02hhx-%02hhx%02hhx-%02hhx%02hhx%02hhx%02hhx%02hhx%02hhx" +#define LIBXL_UUID_FMTLEN ((2*16)+4) /* 16 hex bytes plus 4 hypens */ #define LIBXL__UUID_BYTES(uuid) uuid[0], uuid[1], uuid[2], uuid[3], \ uuid[4], uuid[5], uuid[6], uuid[7], \ uuid[8], uuid[9], uuid[10], uuid[11], \ diff -r dea9208b98db -r 4e751a05df3c tools/libxl/xl_cmdimpl.c --- a/tools/libxl/xl_cmdimpl.c Thu Jun 23 17:19:09 2011 +0100 +++ b/tools/libxl/xl_cmdimpl.c Thu Jun 23 17:19:16 2011 +0100 @@ -4094,9 +4094,7 @@ int main_networklist(int argc, char **ar /* Idx BE */ printf("%-3d %-2d ", nics[i].devid, nics[i].backend_id); /* MAC */ - printf("%02x:%02x:%02x:%02x:%02x:%02x ", - nics[i].mac[0], nics[i].mac[1], nics[i].mac[2], - nics[i].mac[3], nics[i].mac[4], nics[i].mac[5]); + printf(LIBXL_MAC_FMT, LIBXL_MAC_BYTES(nics[i].mac)); /* Hdl Sta evch txr/rxr BE-path */ printf("%6d %5d %6d %5d/%-11d %-30s\n", nics[i].devid, nics[i].state, nics[i].evtch, ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 10 of 10] libxl: IDL: autogenerate functions to produce JSON from libxl data structures 2011-06-23 16:20 [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Campbell ` (8 preceding siblings ...) 2011-06-23 16:21 ` [PATCH 09 of 10] libxl: add LIBXL_MAC_{FMT, FMTLEN, BYTES} Ian Campbell @ 2011-06-23 16:21 ` Ian Campbell 2011-06-23 16:23 ` Ian Campbell 2011-06-24 14:54 ` [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Jackson 10 siblings, 1 reply; 13+ messages in thread From: Ian Campbell @ 2011-06-23 16:21 UTC (permalink / raw) To: xen-devel; +Cc: Ian Campbell # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1308845967 -3600 # Node ID f004439ac18259226136d68d533407661dd55232 # Parent 4e751a05df3c323aea8fb291961a21d13de4afda libxl: IDL: autogenerate functions to produce JSON from libxl data structures. Two functions are provided. TYPE_gen_json exposes an interface which is compatible with the YAGL generator infrastructure. TYPE_to_string uses this to produce a pretty printed string. The TYPE_gen_json functions are defined in a new header libxl_json.h which is not exposed via libxl.h due to the use of YAGL datatypes to avoid poluting the namespace us libxl users which don't use the library themselves. If a libxl user is interested in integrating at the YAGL level then it should #include this file itself. Also update testidl to generate a random version of each IDL datastructure and convert it to JSON. Unfortunately this requires a libxl_ctx and therefore the test must be run on a Xen system now. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 4e751a05df3c -r f004439ac182 tools/libxl/Makefile --- a/tools/libxl/Makefile Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/Makefile Thu Jun 23 17:19:27 2011 +0100 @@ -32,9 +32,12 @@ endif LIBXL_OBJS-$(CONFIG_X86) += libxl_cpuid.o LIBXL_OBJS-$(CONFIG_IA64) += libxl_nocpuid.o +LIBXL_LIBS += -lyajl + LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \ libxl_dom.o libxl_exec.o libxl_xshelp.o libxl_device.o \ - libxl_internal.o libxl_utils.o libxl_uuid.o $(LIBXL_OBJS-y) + libxl_internal.o libxl_utils.o libxl_uuid.o libxl_json.o \ + $(LIBXL_OBJS-y) LIBXL_OBJS += _libxl_types.o libxl_flask.o $(LIBXL_OBJS): CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenguest) $(CFLAGS_libxenstore) $(CFLAGS_libblktapctl) @@ -81,11 +84,14 @@ libxl_paths.c: _libxl_paths.h libxl.h: _libxl_types.h +libxl_json.h: _libxl_types_json.h + $(LIBXL_OBJS) $(LIBXLU_OBJS) $(XL_OBJS): libxl.h -_libxl_%.h _libxl_%.c: libxl.idl gen%.py libxl%.py - $(PYTHON) gen$*.py libxl.idl __libxl_$*.h __libxl_$*.c +_libxl_%.h _libxl_%_json.h _libxl_%.c: libxl.idl gen%.py libxltypes.py + $(PYTHON) gen$*.py libxl.idl __libxl_$*.h __libxl_$*_json.h __libxl_$*.c $(call move-if-changed,__libxl_$*.h,_libxl_$*.h) + $(call move-if-changed,__libxl_$*_json.h,_libxl_$*_json.h) $(call move-if-changed,__libxl_$*.c,_libxl_$*.c) libxenlight.so: libxenlight.so.$(MAJOR) @@ -133,7 +139,7 @@ install: all ln -sf libxlutil.so.$(XLUMAJOR).$(XLUMINOR) $(DESTDIR)$(LIBDIR)/libxlutil.so.$(XLUMAJOR) ln -sf libxlutil.so.$(XLUMAJOR) $(DESTDIR)$(LIBDIR)/libxlutil.so $(INSTALL_DATA) libxlutil.a $(DESTDIR)$(LIBDIR) - $(INSTALL_DATA) libxl.h _libxl_types.h libxl_utils.h libxl_uuid.h $(DESTDIR)$(INCLUDEDIR) + $(INSTALL_DATA) libxl.h libxl_json.h _libxl_types.h _libxl_types_json.h libxl_utils.h libxl_uuid.h $(DESTDIR)$(INCLUDEDIR) $(INSTALL_DATA) bash-completion $(DESTDIR)$(BASH_COMPLETION_DIR)/xl.sh .PHONY: clean diff -r 4e751a05df3c -r f004439ac182 tools/libxl/gentest.py --- a/tools/libxl/gentest.py Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/gentest.py Thu Jun 23 17:19:27 2011 +0100 @@ -5,6 +5,7 @@ import re import random import libxltypes + def randomize_char(c): if random.random() < 0.5: return str.lower(c) @@ -15,6 +16,49 @@ def randomize_case(s): r = [randomize_char(c) for c in s] return "".join(r) +def randomize_enum(e): + return random.choice([v.name for v in e.values]) + +handcoded = ["libxl_cpumap", "libxl_key_value_list", + "libxl_cpuid_policy_list", "libxl_file_reference", + "libxl_string_list", "libxl_cpuarray"] + +def gen_rand_init(ty, v, indent = " ", parent = None): + s = "" + if isinstance(ty, libxltypes.Enumeration): + s += "%s = %s;\n" % (ty.pass_arg(v, parent is None), randomize_enum(ty)) + elif isinstance(ty, libxltypes.KeyedUnion): + if parent is None: + raise Exception("KeyedUnion type must have a parent") + for f in ty.fields: + (nparent,fexpr) = ty.member(v, f, parent is None) + keyvar_expr = f.keyvar_expr % (parent + ty.keyvar_name) + s += "if (" + keyvar_expr + ") {\n" + s += gen_rand_init(f.type, fexpr, indent + " ", nparent) + s += "}\n" + elif isinstance(ty, libxltypes.Struct) and (parent is None or ty.json_fn is None): + for f in [f for f in ty.fields if not f.const]: + (nparent,fexpr) = ty.member(v, f, parent is None) + s += gen_rand_init(f.type, fexpr, "", nparent) + elif hasattr(ty, "rand_init") and ty.rand_init is not None: + s += "%s(%s);\n" % (ty.rand_init, ty.pass_arg(v, isref=parent is None, passby=libxltypes.PASS_BY_REFERENCE)) + elif ty.typename in ["libxl_uuid", "libxl_mac", "libxl_hwcap"]: + s += "rand_bytes((uint8_t *)%s, sizeof(*%s));\n" % (v,v) + elif ty.typename in ["libxl_domid"] or isinstance(ty, libxltypes.Number): + s += "%s = rand() %% (sizeof(%s)*8);\n" % (ty.pass_arg(v, parent is None), ty.pass_arg(v, parent is None)) + elif ty.typename in ["bool"]: + s += "%s = rand() %% 2;\n" % v + elif ty.typename in ["char *"]: + s += "%s = rand_str();\n" % v + elif ty.typename in handcoded: + raise Exception("Gen for handcoded %s" % ty.typename) + else: + raise Exception("Cannot randomly init %s" % ty.typename) + + if s != "": + s = indent + s + return s.replace("\n", "\n%s" % indent).rstrip(indent) + if __name__ == '__main__': if len(sys.argv) < 3: print >>sys.stderr, "Usage: gentest.py <idl> <implementation>" @@ -23,23 +67,191 @@ if __name__ == '__main__': random.seed() idl = sys.argv[1] - (_,types) = libxltypes.parse(idl) + (builtins,types) = libxltypes.parse(idl) impl = sys.argv[2] f = open(impl, "w") f.write(""" #include <stdio.h> +#include <stdlib.h> +#include <string.h> + #include \"libxl.h\" +#include \"libxl_utils.h\" +static char *rand_str(void) +{ + int i, sz = rand() % 32; + char *s = malloc(sz+1); + for (i=0; i<sz; i++) + s[i] = 'a' + (rand() % 26); + s[i] = '\\0'; + return s; +} + +static void rand_bytes(uint8_t *p, size_t sz) +{ + int i; + for (i=0; i<sz; i++) + p[i] = rand() % 256; + //p[i] = i; +} + +static void libxl_cpumap_rand_init(libxl_cpumap *cpumap) +{ + int i; + cpumap->size = rand() % 16; + cpumap->map = calloc(cpumap->size, sizeof(*cpumap->map)); + libxl_for_each_cpu(i, *cpumap) { + if (rand() % 2) + libxl_cpumap_set(cpumap, i); + else + libxl_cpumap_reset(cpumap, i); + } +} + +static void libxl_key_value_list_rand_init(libxl_key_value_list *pkvl) +{ + int i, nr_kvp = rand() % 16; + libxl_key_value_list kvl = calloc(nr_kvp+1, 2*sizeof(char *)); + + for (i = 0; i<2*nr_kvp; i += 2) { + kvl[i] = rand_str(); + if (rand() % 8) + kvl[i+1] = rand_str(); + else + kvl[i+1] = NULL; + } + kvl[i] = NULL; + kvl[i+1] = NULL; + *pkvl = kvl; +} + +static void libxl_cpuid_policy_list_rand_init(libxl_cpuid_policy_list *pp) +{ + int i, nr_policies = rand() % 16; + struct { + const char *n; + int w; + } options[] = { + /* A random selection from libxl_cpuid_parse_config */ + {"maxleaf", 32}, + {"family", 8}, + {"model", 8}, + {"stepping", 4}, + {"localapicid", 8}, + {"proccount", 8}, + {"clflush", 8}, + {"brandid", 8}, + {"f16c", 1}, + {"avx", 1}, + {"osxsave", 1}, + {"xsave", 1}, + {"aes", 1}, + {"popcnt", 1}, + {"movbe", 1}, + {"x2apic", 1}, + {"sse4.2", 1}, + {"sse4.1", 1}, + {"dca", 1}, + {"pdcm", 1}, + {"procpkg", 6}, + }; + const int nr_options = sizeof(options)/sizeof(options[0]); + char buf[64]; + libxl_cpuid_policy_list p = NULL; + + for (i = 0; i < nr_policies; i++) { + int opt = rand() % nr_options; + int val = rand() % (1<<options[opt].w); + snprintf(buf, 64, "%s=%#x", options[opt].n, val); + libxl_cpuid_parse_config(&p, buf); + } + *pp = p; +} + +static void libxl_file_reference_rand_init(libxl_file_reference *p) +{ + memset(p, 0, sizeof(*p)); + if (rand() % 8) + p->path = rand_str(); +} + +static void libxl_string_list_rand_init(libxl_string_list *p) +{ + int i, nr = rand() % 16; + libxl_string_list l = calloc(nr+1, sizeof(char *)); + + for (i = 0; i<nr; i++) { + l[i] = rand_str(); + } + l[i] = NULL; + *p = l; +} + +static void libxl_cpuarray_rand_init(libxl_cpuarray *p) +{ + int i; + /* Up to 16 VCPUs on 32 PCPUS */ + p->entries = rand() % 16; + p->array = calloc(p->entries, sizeof(*p->array)); + for (i = 0; i < p->entries; i++) { + int r = rand() % 32*1.5; /* 2:1 valid:invalid */ + if (r >= 32) + p->array[i] = LIBXL_CPUARRAY_INVALID_ENTRY; + else + p->array[i] = r; + } +} +""") + for ty in builtins + types: + if ty.typename not in handcoded: + f.write("static void %s_rand_init(%s);\n" % (ty.typename, ty.make_arg("p", passby=libxltypes.PASS_BY_REFERENCE))) + f.write("static void %s_rand_init(%s)\n" % (ty.typename, ty.make_arg("p", passby=libxltypes.PASS_BY_REFERENCE))) + f.write("{\n") + f.write(gen_rand_init(ty, "p")) + f.write("}\n") + f.write("\n") + ty.rand_init = "%s_rand_init" % ty.typename + + f.write(""" int main(int argc, char **argv) { """) - for ty in [t for t in types if isinstance(t,libxltypes.Enumeration)]: + for ty in types: f.write(" %s %s_val;\n" % (ty.typename, ty.typename)) - f.write(" int rc;\n") - f.write("\n") - + f.write(""" + int rc; + char *s; + xentoollog_logger_stdiostream *logger; + libxl_ctx *ctx; + + logger = xtl_createlogger_stdiostream(stderr, XTL_DETAIL, 0); + if (!logger) exit(1); + + if (libxl_ctx_alloc(&ctx, LIBXL_VERSION, (xentoollog_logger*)logger)) { + fprintf(stderr, "cannot init xl context\\n"); + exit(1); + } +""") + f.write(" printf(\"Testing TYPE_to_json()\\n\");\n") + f.write(" printf(\"----------------------\\n\");\n") + f.write(" printf(\"\\n\");\n") + for ty in [t for t in types if t.json_fn is not None]: + arg = ty.typename + "_val" + f.write(" %s_rand_init(%s);\n" % (ty.typename, ty.pass_arg(arg, isref=False, passby=libxltypes.PASS_BY_REFERENCE))) + f.write(" s = %s_to_json(ctx, %s);\n" % (ty.typename, ty.pass_arg(arg, isref=False))) + f.write(" printf(\"%%s: %%s\\n\", \"%s\", s);\n" % ty.typename) + f.write(" if (s == NULL) abort();\n") + f.write(" free(s);\n") + if ty.destructor_fn is not None: + f.write(" %s(&%s_val);\n" % (ty.destructor_fn, ty.typename)) + f.write("\n") + + f.write(" printf(\"Testing Enumerations\\n\");\n") + f.write(" printf(\"--------------------\\n\");\n") + f.write(" printf(\"\\n\");\n") for ty in [t for t in types if isinstance(t,libxltypes.Enumeration)]: f.write(" printf(\"%s -- to string:\\n\");\n" % (ty.typename)) for v in ty.values: @@ -47,6 +259,12 @@ int main(int argc, char **argv) (v.valuename, v.name, ty.typename, v.name)) f.write("\n") + f.write(" printf(\"%s -- to JSON:\\n\");\n" % (ty.typename)) + for v in ty.values: + f.write(" printf(\"\\t%s = %%d = %%s\", %s, %s_to_json(ctx, %s));\n" %\ + (v.valuename, v.name, ty.typename, v.name)) + f.write("\n") + f.write(" printf(\"%s -- from string:\\n\");\n" % (ty.typename)) for v in [v.valuename for v in ty.values] + ["AN INVALID VALUE"]: n = randomize_case(v) @@ -58,6 +276,11 @@ int main(int argc, char **argv) (v, n, ty.typename)) f.write("\n") - f.write("""return 0; + f.write(""" + + libxl_ctx_free(ctx); + xtl_logger_destroy((xentoollog_logger*)logger); + + return 0; } """) diff -r 4e751a05df3c -r f004439ac182 tools/libxl/gentypes.py --- a/tools/libxl/gentypes.py Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/gentypes.py Thu Jun 23 17:19:27 2011 +0100 @@ -29,7 +29,6 @@ def libxl_C_instance_of(ty, instancename def libxl_C_type_define(ty, indent = ""): s = "" - if isinstance(ty, libxltypes.Enumeration): if ty.comment is not None: s += format_comment(0, ty.comment) @@ -76,7 +75,6 @@ def libxl_C_type_define(ty, indent = "") return s.replace("\n", "\n%s" % indent) def libxl_C_type_destroy(ty, v, indent = " ", parent = None): - s = "" if isinstance(ty, libxltypes.KeyedUnion): if parent is None: @@ -99,6 +97,65 @@ def libxl_C_type_destroy(ty, v, indent = s = indent + s return s.replace("\n", "\n%s" % indent).rstrip(indent) +def libxl_C_type_gen_json(ty, v, indent = " ", parent = None): + s = "" + if parent is None: + s += "yajl_gen_status s;\n" + if isinstance(ty, libxltypes.Enumeration): + s += "{\n" + s += " const char *se = %s_to_string(%s);\n" % (ty.typename, ty.pass_arg(v, parent is None)) + s += " if (se)\n" + s += " s = yajl_gen_string(hand, (const unsigned char *)se, strlen(se));\n" + s += " else\n" + s += " s = yajl_gen_null(hand);\n" + s += " if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + s += "}\n" + elif isinstance(ty, libxltypes.KeyedUnion): + if parent is None: + raise Exception("KeyedUnion type must have a parent") + for f in ty.fields: + (nparent,fexpr) = ty.member(v, f, parent is None) + keyvar_expr = f.keyvar_expr % (parent + ty.keyvar_name) + s += "if (" + keyvar_expr + ") {\n" + s += libxl_C_type_gen_json(f.type, fexpr, indent + " ", nparent) + s += "}\n" + elif isinstance(ty, libxltypes.Struct) and (parent is None or ty.json_fn is None): + s += "s = yajl_gen_map_open(hand);\n" + s += "if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + for f in [f for f in ty.fields if not f.const]: + (nparent,fexpr) = ty.member(v, f, parent is None) + s += "s = yajl_gen_string(hand, (const unsigned char *)\"%s\", sizeof(\"%s\")-1);\n" % (f.name, f.name) + s += "if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + s += libxl_C_type_gen_json(f.type, fexpr, "", nparent) + s += "s = yajl_gen_map_close(hand);\n" + s += "if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + else: + if ty.json_fn is not None: + s += "s = %s(hand, %s);\n" % (ty.json_fn, ty.pass_arg(v, parent is None)) + s += "if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + + if parent is None: + s += "out:\n" + s += "return s;\n" + + if s != "": + s = indent + s + return s.replace("\n", "\n%s" % indent).rstrip(indent) + +def libxl_C_type_to_json(ty, v, indent = " "): + s = "" + gen = "(libxl__gen_json_callback)&%s_gen_json" % ty.typename + s += "return libxl__object_to_json(ctx, \"%s\", %s, (void *)%s);\n" % (ty.typename, gen, ty.pass_arg(v, passby=libxltypes.PASS_BY_REFERENCE)) + + if s != "": + s = indent + s + return s.replace("\n", "\n%s" % indent).rstrip(indent) + def libxl_C_enum_to_string(ty, e, indent = " "): s = "" s += "switch(%s) {\n" % e @@ -137,11 +194,11 @@ def libxl_C_enum_from_string(ty, str, e, if __name__ == '__main__': - if len(sys.argv) != 4: - print >>sys.stderr, "Usage: gentypes.py <idl> <header> <implementation>" + if len(sys.argv) != 5: + print >>sys.stderr, "Usage: gentypes.py <idl> <header> <header-json> <implementation>" sys.exit(1) - (_, idl, header, impl) = sys.argv + (_, idl, header, header_json, impl) = sys.argv (_,types) = libxltypes.parse(idl) @@ -165,6 +222,8 @@ if __name__ == '__main__': f.write(libxl_C_type_define(ty) + ";\n") if ty.destructor_fn is not None: f.write("void %s(%s);\n" % (ty.destructor_fn, ty.make_arg("p"))) + if ty.json_fn is not None: + f.write("char *%s_to_json(libxl_ctx *ctx, %s);\n" % (ty.typename, ty.make_arg("p"))) if isinstance(ty, libxltypes.Enumeration): f.write("const char *%s_to_string(%s);\n" % (ty.typename, ty.make_arg("p"))) f.write("int %s_from_string(const char *s, %s);\n" % (ty.typename, ty.make_arg("e", passby=libxltypes.PASS_BY_REFERENCE))) @@ -174,6 +233,29 @@ if __name__ == '__main__': f.write("""#endif /* __LIBXL_TYPES_H */\n""") f.close() + print "outputting libxl JSON definitions to %s" % header_json + + f = open(header_json, "w") + + f.write("""#ifndef __LIBXL_TYPES_JSON_H +#define __LIBXL_TYPES_JSON_H + +/* + * DO NOT EDIT. + * + * This file is autogenerated by + * "%s" + */ + +""" % " ".join(sys.argv)) + + for ty in [ty for ty in types if ty.json_fn is not None]: + f.write("yajl_gen_status %s_gen_json(yajl_gen hand, %s);\n" % (ty.typename, ty.make_arg("p", passby=libxltypes.PASS_BY_REFERENCE))) + + f.write("\n") + f.write("""#endif /* __LIBXL_TYPES_JSON_H */\n""") + f.close() + print "outputting libxl type implementations to %s" % impl f = open(impl, "w") @@ -220,5 +302,17 @@ if __name__ == '__main__': f.write("}\n") f.write("\n") + for ty in [t for t in types if t.json_fn is not None]: + f.write("yajl_gen_status %s_gen_json(yajl_gen hand, %s)\n" % (ty.typename, ty.make_arg("p", passby=libxltypes.PASS_BY_REFERENCE))) + f.write("{\n") + f.write(libxl_C_type_gen_json(ty, "p")) + f.write("}\n") + f.write("\n") + + f.write("char *%s_to_json(libxl_ctx *ctx, %s)\n" % (ty.typename, ty.make_arg("p"))) + f.write("{\n") + f.write(libxl_C_type_to_json(ty, "p")) + f.write("}\n") + f.write("\n") f.close() diff -r 4e751a05df3c -r f004439ac182 tools/libxl/idl.txt --- a/tools/libxl/idl.txt Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/idl.txt Thu Jun 23 17:19:27 2011 +0100 @@ -49,6 +49,15 @@ Type.autogenerate_destructor: (default: Indicates if the above named Type.destructor_fn should be autogenerated. +Type.json_fn: (default: typename + "_gen_json" or None if type == None) + + The name of the C function which will generate a YAJL data structure + representing this type. + +Type.autogenerate_json: (default: True) + + Indicates if the above named Type.json_fn should be autogenerated. + Other simple type-Classes ------------------------- diff -r 4e751a05df3c -r f004439ac182 tools/libxl/libxl.h --- a/tools/libxl/libxl.h Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/libxl.h Thu Jun 23 17:19:27 2011 +0100 @@ -199,10 +199,10 @@ typedef struct { int v; } libxl_enum_string_table; +typedef struct libxl__ctx libxl_ctx; + #include "_libxl_types.h" -typedef struct libxl__ctx libxl_ctx; - const libxl_version_info* libxl_get_version_info(libxl_ctx *ctx); typedef struct { diff -r 4e751a05df3c -r f004439ac182 tools/libxl/libxl.idl --- a/tools/libxl/libxl.idl Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/libxl.idl Thu Jun 23 17:19:27 2011 +0100 @@ -3,7 +3,8 @@ # Builtin libxl types # -libxl_domid = Builtin("domid") +libxl_domid = Builtin("domid", json_fn = "yajl_gen_integer", autogenerate_json = False) + libxl_uuid = Builtin("uuid", passby=PASS_BY_REFERENCE) libxl_mac = Builtin("mac", passby=PASS_BY_REFERENCE) libxl_cpumap = Builtin("cpumap", destructor_fn="libxl_cpumap_destroy", passby=PASS_BY_REFERENCE) diff -r 4e751a05df3c -r f004439ac182 tools/libxl/libxl_internal.h --- a/tools/libxl/libxl_internal.h Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/libxl_internal.h Thu Jun 23 17:19:27 2011 +0100 @@ -35,6 +35,7 @@ #include "flexarray.h" #include "libxl_utils.h" +#include "libxl_json.h" #define LIBXL_DESTROY_TIMEOUT 10 #define LIBXL_DEVICE_MODEL_START_TIMEOUT 10 @@ -328,6 +329,12 @@ _hidden char *libxl__cpupoolid_to_name(l _hidden int libxl__enum_from_string(const libxl_enum_string_table *t, const char *s, int *e); +_hidden yajl_gen_status libxl__string_gen_json(yajl_gen hand, const char *p); + +typedef yajl_gen_status (*libxl__gen_json_callback)(yajl_gen hand, void *); +_hidden char *libxl__object_to_json(libxl_ctx *ctx, const char *type, + libxl__gen_json_callback gen, void *p); + /* holds the CPUID response for a single CPUID leaf * input contains the value of the EAX and ECX register, * and each policy string contains a filter to apply to diff -r 4e751a05df3c -r f004439ac182 tools/libxl/libxl_json.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/libxl/libxl_json.c Thu Jun 23 17:19:27 2011 +0100 @@ -0,0 +1,287 @@ +/* + * Copyright (C) 2011 Citrix Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published + * by the Free Software Foundation; version 2.1 only. with the special + * exception on linking described in file LICENSE. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + */ + +#include <string.h> + +#include <libxl.h> +#include <libxl_internal.h> + +yajl_gen_status yajl_gen_asciiz(yajl_gen hand, const char *str) +{ + return yajl_gen_string(hand, (const unsigned char *)str, strlen(str)); +} + +yajl_gen_status libxl_uuid_gen_json(yajl_gen hand, + libxl_uuid *uuid) +{ + char buf[LIBXL_UUID_FMTLEN+1]; + snprintf(buf, sizeof(buf), LIBXL_UUID_FMT, LIBXL_UUID_BYTES((*uuid))); + return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_UUID_FMTLEN); +} + +yajl_gen_status libxl_cpumap_gen_json(yajl_gen hand, + libxl_cpumap *cpumap) +{ + yajl_gen_status s; + int i; + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + libxl_for_each_cpu(i, *cpumap) { + if (libxl_cpumap_test(cpumap, i)) { + s = yajl_gen_integer(hand, i); + if (s != yajl_gen_status_ok) goto out; + } + } + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand, + libxl_key_value_list *pkvl) +{ + libxl_key_value_list kvl = *pkvl; + yajl_gen_status s; + int i; + + s = yajl_gen_map_open(hand); + if (s != yajl_gen_status_ok) goto out; + + if (!kvl) goto empty; + + for (i = 0; kvl[i] != NULL; i += 2) { + s = yajl_gen_asciiz(hand, kvl[i]); + if (s != yajl_gen_status_ok) goto out; + if (kvl[i + 1]) + s = yajl_gen_asciiz(hand, kvl[i+1]); + else + s = yajl_gen_null(hand); + if (s != yajl_gen_status_ok) goto out; + } +empty: + s = yajl_gen_map_close(hand); +out: + return s; +} + +yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand, + libxl_cpuid_policy_list *pcpuid) +{ + libxl_cpuid_policy_list cpuid = *pcpuid; + yajl_gen_status s; + const char *input_names[2] = { "leaf", "subleaf" }; + const char *policy_names[4] = { "eax", "ebx", "ecx", "edx" }; + int i, j; + + /* + * Aiming for: + * [ + * { 'leaf': 'val-eax', + * 'subleaf': 'val-edx', + * 'ebx': 'filter', + * 'ecx': 'filter', + * 'edx': 'filter' }, ], + * { 'leaf': 'val-eax', ..., 'eax': 'filter', ... }, + * ... etc ... + * } + */ + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + if (cpuid == NULL) goto empty; + + for (i = 0; cpuid[i].input[0] != XEN_CPUID_INPUT_UNUSED; i++) { + s = yajl_gen_map_open(hand); + if (s != yajl_gen_status_ok) goto out; + + for (j = 0; j < 2; j++) { + if (cpuid[i].input[j] != XEN_CPUID_INPUT_UNUSED) { + s = yajl_gen_asciiz(hand, input_names[j]); + if (s != yajl_gen_status_ok) goto out; + s = yajl_gen_integer(hand, cpuid[i].input[j]); + if (s != yajl_gen_status_ok) goto out; + } + } + + for (j = 0; j < 4; j++) { + if (cpuid[i].policy[j] != NULL) { + s = yajl_gen_asciiz(hand, policy_names[j]); + if (s != yajl_gen_status_ok) goto out; + s = yajl_gen_string(hand, + (const unsigned char *)cpuid[i].policy[j], 32); + if (s != yajl_gen_status_ok) goto out; + } + } + s = yajl_gen_map_close(hand); + if (s != yajl_gen_status_ok) goto out; + } + +empty: + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, libxl_string_list *pl) +{ + libxl_string_list l = *pl; + yajl_gen_status s; + int i; + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + if (!l) goto empty; + + for (i = 0; l[i] != NULL; i++) { + s = yajl_gen_asciiz(hand, l[i]); + if (s != yajl_gen_status_ok) goto out; + } +empty: + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *mac) +{ + char buf[LIBXL_MAC_FMTLEN+1]; + snprintf(buf, sizeof(buf), LIBXL_MAC_FMT, LIBXL_MAC_BYTES((*mac))); + return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_MAC_FMTLEN); +} + +yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand, + libxl_hwcap p) +{ + yajl_gen_status s; + int i; + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + for(i=0; i<4; i++) { + s = yajl_gen_integer(hand, p[i]); + if (s != yajl_gen_status_ok) goto out; + } + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_cpuarray_gen_json(yajl_gen hand, + libxl_cpuarray *cpuarray) +{ + yajl_gen_status s; + int i; + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + for(i=0; i<cpuarray->entries; i++) { + if (cpuarray->array[i] == LIBXL_CPUARRAY_INVALID_ENTRY) + s = yajl_gen_null(hand); + else + s = yajl_gen_integer(hand, cpuarray->array[i]); + if (s != yajl_gen_status_ok) goto out; + } + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_file_reference_gen_json(yajl_gen hand, + libxl_file_reference *p) +{ + if (p->path) + return yajl_gen_asciiz(hand, p->path); + else + return yajl_gen_null(hand); +} + +yajl_gen_status libxl__string_gen_json(yajl_gen hand, + const char *p) +{ + if (p) + return yajl_gen_asciiz(hand, p); + else + return yajl_gen_null(hand); +} + +static const char *yajl_gen_status_to_string(yajl_gen_status s) +{ + switch (s) { + case yajl_gen_status_ok: abort(); + case yajl_gen_keys_must_be_strings: + return "keys must be strings"; + case yajl_max_depth_exceeded: + return "max depth exceeded"; + case yajl_gen_in_error_state: + return "in error state"; + case yajl_gen_generation_complete: + return "generation complete"; + case yajl_gen_invalid_number: + return "invalid number"; + case yajl_gen_no_buf: + return "no buffer"; +#if 0 /* This is in the docs but not implemented in the version I am running. */ + case yajl_gen_invalid_string: + return "invalid string"; +#endif + default: + return "unknown error"; + } +} + +char *libxl__object_to_json(libxl_ctx *ctx, const char *type, + libxl__gen_json_callback gen, void *p) +{ + yajl_gen_config conf = { 1, " " }; + const unsigned char *buf; + char *ret = NULL; + unsigned int len = 0; + yajl_gen_status s; + yajl_gen hand; + + hand = yajl_gen_alloc(&conf, NULL); + if (!hand) + return NULL; + + s = gen(hand, p); + if (s != yajl_gen_status_ok) + goto out; + + s = yajl_gen_get_buf(hand, &buf, &len); + if (s != yajl_gen_status_ok) + goto out; + ret = strdup((const char *)buf); + +out: + yajl_gen_free(hand); + + if (s != yajl_gen_status_ok) { + LIBXL__LOG(ctx, LIBXL__LOG_ERROR, + "unable to convert %s to JSON representation. " + "YAJL error code %d: %s", type, + s, yajl_gen_status_to_string(s)); + } else if (!ret) { + LIBXL__LOG(ctx, LIBXL__LOG_ERROR, + "unable to allocate space for to JSON representation of %s", + type); + } + + return ret; +} diff -r 4e751a05df3c -r f004439ac182 tools/libxl/libxl_json.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/libxl/libxl_json.h Thu Jun 23 17:19:27 2011 +0100 @@ -0,0 +1,41 @@ +/* + * Copyright (C) 2011 Citrix Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published + * by the Free Software Foundation; version 2.1 only. with the special + * exception on linking described in file LICENSE. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + */ + +#ifndef LIBXL_JSON_H +#define LIBXL_JSON_H + +#include <yajl/yajl_gen.h> + +#include <_libxl_types_json.h> + +yajl_gen_status libxl_uuid_gen_json(yajl_gen hand, + libxl_uuid *p); +yajl_gen_status libxl_cpumap_gen_json(yajl_gen hand, + libxl_cpumap *p); +yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand, + libxl_key_value_list *p); +yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand, + libxl_cpuid_policy_list *p); +yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, + libxl_string_list *p); +yajl_gen_status libxl_mac_gen_json(yajl_gen hand, + libxl_mac *p); +yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand, + libxl_hwcap p); +yajl_gen_status libxl_cpuarray_gen_json(yajl_gen hand, + libxl_cpuarray *p); +yajl_gen_status libxl_file_reference_gen_json(yajl_gen hand, + libxl_file_reference *p); + +#endif /* LIBXL_JSON_H */ diff -r 4e751a05df3c -r f004439ac182 tools/libxl/libxltypes.py --- a/tools/libxl/libxltypes.py Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/libxltypes.py Thu Jun 23 17:19:27 2011 +0100 @@ -37,6 +37,13 @@ class Type(object): self.autogenerate_destructor = kwargs.setdefault('autogenerate_destructor', True) + if self.typename is not None: + self.json_fn = kwargs.setdefault('json_fn', self.typename + "_gen_json") + else: + self.json_fn = kwargs.setdefault('json_fn', None) + + self.autogenerate_json = kwargs.setdefault('autogenerate_json', True) + def marshal_in(self): return self.dir in [DIR_IN, DIR_BOTH] def marshal_out(self): @@ -70,6 +77,7 @@ class Builtin(Type): def __init__(self, typename, **kwargs): kwargs.setdefault('destructor_fn', None) kwargs.setdefault('autogenerate_destructor', False) + kwargs.setdefault('autogenerate_json', False) Type.__init__(self, typename, **kwargs) class Number(Builtin): @@ -77,6 +85,7 @@ class Number(Builtin): kwargs.setdefault('namespace', None) kwargs.setdefault('destructor_fn', None) kwargs.setdefault('signed', False) + kwargs.setdefault('json_fn', "yajl_gen_integer") self.signed = kwargs['signed'] Builtin.__init__(self, ctype, **kwargs) @@ -145,6 +154,8 @@ class Aggregate(Type): comment = None else: n,t,const,comment = f + if n is None: + raise ValueError self.fields.append(Field(t,n,const=const,comment=comment)) # Returns a tuple (stem, field-expr) @@ -199,7 +210,10 @@ class KeyedUnion(Aggregate): # void = Builtin("void *", namespace = None) -bool = Builtin("bool", namespace = None) +bool = Builtin("bool", namespace = None, + json_fn = "yajl_gen_bool", + autogenerate_json = False) + size_t = Number("size_t", namespace = None) integer = Number("int", namespace = None, signed = True) @@ -209,7 +223,9 @@ uint16 = UInt(16) uint32 = UInt(32) uint64 = UInt(64) -string = Builtin("char *", namespace = None, destructor_fn = "free") +string = Builtin("char *", namespace = None, destructor_fn = "free", + json_fn = "libxl__string_gen_json", + autogenerate_json = False) class OrderedDict(dict): """A dictionary which remembers insertion order. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 10 of 10] libxl: IDL: autogenerate functions to produce JSON from libxl data structures 2011-06-23 16:21 ` [PATCH 10 of 10] libxl: IDL: autogenerate functions to produce JSON from libxl data structures Ian Campbell @ 2011-06-23 16:23 ` Ian Campbell 0 siblings, 0 replies; 13+ messages in thread From: Ian Campbell @ 2011-06-23 16:23 UTC (permalink / raw) To: xen-devel@lists.xensource.com Minor update (missing "static") because I forgot to qrefresh before sending. I also forgot to mention that this uses the same library (YAJL) as Anthony's QMP patch, I made the Makefile bit look the same to keep any conflict reslution minimal. # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1308846102 -3600 # Node ID fc8319c378641194e028a9ae3353b95b1c05539a # Parent 4e751a05df3c323aea8fb291961a21d13de4afda libxl: IDL: autogenerate functions to produce JSON from libxl data structures. Two functions are provided. TYPE_gen_json exposes an interface which is compatible with the YAGL generator infrastructure. TYPE_to_string uses this to produce a pretty printed string. The TYPE_gen_json functions are defined in a new header libxl_json.h which is not exposed via libxl.h due to the use of YAGL datatypes to avoid poluting the namespace us libxl users which don't use the library themselves. If a libxl user is interested in integrating at the YAGL level then it should #include this file itself. Also update testidl to generate a random version of each IDL datastructure and convert it to JSON. Unfortunately this requires a libxl_ctx and therefore the test must be run on a Xen system now. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 4e751a05df3c -r fc8319c37864 tools/libxl/Makefile --- a/tools/libxl/Makefile Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/Makefile Thu Jun 23 17:21:42 2011 +0100 @@ -32,9 +32,12 @@ endif LIBXL_OBJS-$(CONFIG_X86) += libxl_cpuid.o LIBXL_OBJS-$(CONFIG_IA64) += libxl_nocpuid.o +LIBXL_LIBS += -lyajl + LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \ libxl_dom.o libxl_exec.o libxl_xshelp.o libxl_device.o \ - libxl_internal.o libxl_utils.o libxl_uuid.o $(LIBXL_OBJS-y) + libxl_internal.o libxl_utils.o libxl_uuid.o libxl_json.o \ + $(LIBXL_OBJS-y) LIBXL_OBJS += _libxl_types.o libxl_flask.o $(LIBXL_OBJS): CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenguest) $(CFLAGS_libxenstore) $(CFLAGS_libblktapctl) @@ -81,11 +84,14 @@ libxl_paths.c: _libxl_paths.h libxl.h: _libxl_types.h +libxl_json.h: _libxl_types_json.h + $(LIBXL_OBJS) $(LIBXLU_OBJS) $(XL_OBJS): libxl.h -_libxl_%.h _libxl_%.c: libxl.idl gen%.py libxl%.py - $(PYTHON) gen$*.py libxl.idl __libxl_$*.h __libxl_$*.c +_libxl_%.h _libxl_%_json.h _libxl_%.c: libxl.idl gen%.py libxltypes.py + $(PYTHON) gen$*.py libxl.idl __libxl_$*.h __libxl_$*_json.h __libxl_$*.c $(call move-if-changed,__libxl_$*.h,_libxl_$*.h) + $(call move-if-changed,__libxl_$*_json.h,_libxl_$*_json.h) $(call move-if-changed,__libxl_$*.c,_libxl_$*.c) libxenlight.so: libxenlight.so.$(MAJOR) @@ -133,7 +139,7 @@ install: all ln -sf libxlutil.so.$(XLUMAJOR).$(XLUMINOR) $(DESTDIR)$(LIBDIR)/libxlutil.so.$(XLUMAJOR) ln -sf libxlutil.so.$(XLUMAJOR) $(DESTDIR)$(LIBDIR)/libxlutil.so $(INSTALL_DATA) libxlutil.a $(DESTDIR)$(LIBDIR) - $(INSTALL_DATA) libxl.h _libxl_types.h libxl_utils.h libxl_uuid.h $(DESTDIR)$(INCLUDEDIR) + $(INSTALL_DATA) libxl.h libxl_json.h _libxl_types.h _libxl_types_json.h libxl_utils.h libxl_uuid.h $(DESTDIR)$(INCLUDEDIR) $(INSTALL_DATA) bash-completion $(DESTDIR)$(BASH_COMPLETION_DIR)/xl.sh .PHONY: clean diff -r 4e751a05df3c -r fc8319c37864 tools/libxl/gentest.py --- a/tools/libxl/gentest.py Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/gentest.py Thu Jun 23 17:21:42 2011 +0100 @@ -5,6 +5,7 @@ import re import random import libxltypes + def randomize_char(c): if random.random() < 0.5: return str.lower(c) @@ -15,6 +16,49 @@ def randomize_case(s): r = [randomize_char(c) for c in s] return "".join(r) +def randomize_enum(e): + return random.choice([v.name for v in e.values]) + +handcoded = ["libxl_cpumap", "libxl_key_value_list", + "libxl_cpuid_policy_list", "libxl_file_reference", + "libxl_string_list", "libxl_cpuarray"] + +def gen_rand_init(ty, v, indent = " ", parent = None): + s = "" + if isinstance(ty, libxltypes.Enumeration): + s += "%s = %s;\n" % (ty.pass_arg(v, parent is None), randomize_enum(ty)) + elif isinstance(ty, libxltypes.KeyedUnion): + if parent is None: + raise Exception("KeyedUnion type must have a parent") + for f in ty.fields: + (nparent,fexpr) = ty.member(v, f, parent is None) + keyvar_expr = f.keyvar_expr % (parent + ty.keyvar_name) + s += "if (" + keyvar_expr + ") {\n" + s += gen_rand_init(f.type, fexpr, indent + " ", nparent) + s += "}\n" + elif isinstance(ty, libxltypes.Struct) and (parent is None or ty.json_fn is None): + for f in [f for f in ty.fields if not f.const]: + (nparent,fexpr) = ty.member(v, f, parent is None) + s += gen_rand_init(f.type, fexpr, "", nparent) + elif hasattr(ty, "rand_init") and ty.rand_init is not None: + s += "%s(%s);\n" % (ty.rand_init, ty.pass_arg(v, isref=parent is None, passby=libxltypes.PASS_BY_REFERENCE)) + elif ty.typename in ["libxl_uuid", "libxl_mac", "libxl_hwcap"]: + s += "rand_bytes((uint8_t *)%s, sizeof(*%s));\n" % (v,v) + elif ty.typename in ["libxl_domid"] or isinstance(ty, libxltypes.Number): + s += "%s = rand() %% (sizeof(%s)*8);\n" % (ty.pass_arg(v, parent is None), ty.pass_arg(v, parent is None)) + elif ty.typename in ["bool"]: + s += "%s = rand() %% 2;\n" % v + elif ty.typename in ["char *"]: + s += "%s = rand_str();\n" % v + elif ty.typename in handcoded: + raise Exception("Gen for handcoded %s" % ty.typename) + else: + raise Exception("Cannot randomly init %s" % ty.typename) + + if s != "": + s = indent + s + return s.replace("\n", "\n%s" % indent).rstrip(indent) + if __name__ == '__main__': if len(sys.argv) < 3: print >>sys.stderr, "Usage: gentest.py <idl> <implementation>" @@ -23,23 +67,191 @@ if __name__ == '__main__': random.seed() idl = sys.argv[1] - (_,types) = libxltypes.parse(idl) + (builtins,types) = libxltypes.parse(idl) impl = sys.argv[2] f = open(impl, "w") f.write(""" #include <stdio.h> +#include <stdlib.h> +#include <string.h> + #include \"libxl.h\" +#include \"libxl_utils.h\" +static char *rand_str(void) +{ + int i, sz = rand() % 32; + char *s = malloc(sz+1); + for (i=0; i<sz; i++) + s[i] = 'a' + (rand() % 26); + s[i] = '\\0'; + return s; +} + +static void rand_bytes(uint8_t *p, size_t sz) +{ + int i; + for (i=0; i<sz; i++) + p[i] = rand() % 256; + //p[i] = i; +} + +static void libxl_cpumap_rand_init(libxl_cpumap *cpumap) +{ + int i; + cpumap->size = rand() % 16; + cpumap->map = calloc(cpumap->size, sizeof(*cpumap->map)); + libxl_for_each_cpu(i, *cpumap) { + if (rand() % 2) + libxl_cpumap_set(cpumap, i); + else + libxl_cpumap_reset(cpumap, i); + } +} + +static void libxl_key_value_list_rand_init(libxl_key_value_list *pkvl) +{ + int i, nr_kvp = rand() % 16; + libxl_key_value_list kvl = calloc(nr_kvp+1, 2*sizeof(char *)); + + for (i = 0; i<2*nr_kvp; i += 2) { + kvl[i] = rand_str(); + if (rand() % 8) + kvl[i+1] = rand_str(); + else + kvl[i+1] = NULL; + } + kvl[i] = NULL; + kvl[i+1] = NULL; + *pkvl = kvl; +} + +static void libxl_cpuid_policy_list_rand_init(libxl_cpuid_policy_list *pp) +{ + int i, nr_policies = rand() % 16; + struct { + const char *n; + int w; + } options[] = { + /* A random selection from libxl_cpuid_parse_config */ + {"maxleaf", 32}, + {"family", 8}, + {"model", 8}, + {"stepping", 4}, + {"localapicid", 8}, + {"proccount", 8}, + {"clflush", 8}, + {"brandid", 8}, + {"f16c", 1}, + {"avx", 1}, + {"osxsave", 1}, + {"xsave", 1}, + {"aes", 1}, + {"popcnt", 1}, + {"movbe", 1}, + {"x2apic", 1}, + {"sse4.2", 1}, + {"sse4.1", 1}, + {"dca", 1}, + {"pdcm", 1}, + {"procpkg", 6}, + }; + const int nr_options = sizeof(options)/sizeof(options[0]); + char buf[64]; + libxl_cpuid_policy_list p = NULL; + + for (i = 0; i < nr_policies; i++) { + int opt = rand() % nr_options; + int val = rand() % (1<<options[opt].w); + snprintf(buf, 64, "%s=%#x", options[opt].n, val); + libxl_cpuid_parse_config(&p, buf); + } + *pp = p; +} + +static void libxl_file_reference_rand_init(libxl_file_reference *p) +{ + memset(p, 0, sizeof(*p)); + if (rand() % 8) + p->path = rand_str(); +} + +static void libxl_string_list_rand_init(libxl_string_list *p) +{ + int i, nr = rand() % 16; + libxl_string_list l = calloc(nr+1, sizeof(char *)); + + for (i = 0; i<nr; i++) { + l[i] = rand_str(); + } + l[i] = NULL; + *p = l; +} + +static void libxl_cpuarray_rand_init(libxl_cpuarray *p) +{ + int i; + /* Up to 16 VCPUs on 32 PCPUS */ + p->entries = rand() % 16; + p->array = calloc(p->entries, sizeof(*p->array)); + for (i = 0; i < p->entries; i++) { + int r = rand() % 32*1.5; /* 2:1 valid:invalid */ + if (r >= 32) + p->array[i] = LIBXL_CPUARRAY_INVALID_ENTRY; + else + p->array[i] = r; + } +} +""") + for ty in builtins + types: + if ty.typename not in handcoded: + f.write("static void %s_rand_init(%s);\n" % (ty.typename, ty.make_arg("p", passby=libxltypes.PASS_BY_REFERENCE))) + f.write("static void %s_rand_init(%s)\n" % (ty.typename, ty.make_arg("p", passby=libxltypes.PASS_BY_REFERENCE))) + f.write("{\n") + f.write(gen_rand_init(ty, "p")) + f.write("}\n") + f.write("\n") + ty.rand_init = "%s_rand_init" % ty.typename + + f.write(""" int main(int argc, char **argv) { """) - for ty in [t for t in types if isinstance(t,libxltypes.Enumeration)]: + for ty in types: f.write(" %s %s_val;\n" % (ty.typename, ty.typename)) - f.write(" int rc;\n") - f.write("\n") - + f.write(""" + int rc; + char *s; + xentoollog_logger_stdiostream *logger; + libxl_ctx *ctx; + + logger = xtl_createlogger_stdiostream(stderr, XTL_DETAIL, 0); + if (!logger) exit(1); + + if (libxl_ctx_alloc(&ctx, LIBXL_VERSION, (xentoollog_logger*)logger)) { + fprintf(stderr, "cannot init xl context\\n"); + exit(1); + } +""") + f.write(" printf(\"Testing TYPE_to_json()\\n\");\n") + f.write(" printf(\"----------------------\\n\");\n") + f.write(" printf(\"\\n\");\n") + for ty in [t for t in types if t.json_fn is not None]: + arg = ty.typename + "_val" + f.write(" %s_rand_init(%s);\n" % (ty.typename, ty.pass_arg(arg, isref=False, passby=libxltypes.PASS_BY_REFERENCE))) + f.write(" s = %s_to_json(ctx, %s);\n" % (ty.typename, ty.pass_arg(arg, isref=False))) + f.write(" printf(\"%%s: %%s\\n\", \"%s\", s);\n" % ty.typename) + f.write(" if (s == NULL) abort();\n") + f.write(" free(s);\n") + if ty.destructor_fn is not None: + f.write(" %s(&%s_val);\n" % (ty.destructor_fn, ty.typename)) + f.write("\n") + + f.write(" printf(\"Testing Enumerations\\n\");\n") + f.write(" printf(\"--------------------\\n\");\n") + f.write(" printf(\"\\n\");\n") for ty in [t for t in types if isinstance(t,libxltypes.Enumeration)]: f.write(" printf(\"%s -- to string:\\n\");\n" % (ty.typename)) for v in ty.values: @@ -47,6 +259,12 @@ int main(int argc, char **argv) (v.valuename, v.name, ty.typename, v.name)) f.write("\n") + f.write(" printf(\"%s -- to JSON:\\n\");\n" % (ty.typename)) + for v in ty.values: + f.write(" printf(\"\\t%s = %%d = %%s\", %s, %s_to_json(ctx, %s));\n" %\ + (v.valuename, v.name, ty.typename, v.name)) + f.write("\n") + f.write(" printf(\"%s -- from string:\\n\");\n" % (ty.typename)) for v in [v.valuename for v in ty.values] + ["AN INVALID VALUE"]: n = randomize_case(v) @@ -58,6 +276,11 @@ int main(int argc, char **argv) (v, n, ty.typename)) f.write("\n") - f.write("""return 0; + f.write(""" + + libxl_ctx_free(ctx); + xtl_logger_destroy((xentoollog_logger*)logger); + + return 0; } """) diff -r 4e751a05df3c -r fc8319c37864 tools/libxl/gentypes.py --- a/tools/libxl/gentypes.py Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/gentypes.py Thu Jun 23 17:21:42 2011 +0100 @@ -29,7 +29,6 @@ def libxl_C_instance_of(ty, instancename def libxl_C_type_define(ty, indent = ""): s = "" - if isinstance(ty, libxltypes.Enumeration): if ty.comment is not None: s += format_comment(0, ty.comment) @@ -76,7 +75,6 @@ def libxl_C_type_define(ty, indent = "") return s.replace("\n", "\n%s" % indent) def libxl_C_type_destroy(ty, v, indent = " ", parent = None): - s = "" if isinstance(ty, libxltypes.KeyedUnion): if parent is None: @@ -99,6 +97,65 @@ def libxl_C_type_destroy(ty, v, indent = s = indent + s return s.replace("\n", "\n%s" % indent).rstrip(indent) +def libxl_C_type_gen_json(ty, v, indent = " ", parent = None): + s = "" + if parent is None: + s += "yajl_gen_status s;\n" + if isinstance(ty, libxltypes.Enumeration): + s += "{\n" + s += " const char *se = %s_to_string(%s);\n" % (ty.typename, ty.pass_arg(v, parent is None)) + s += " if (se)\n" + s += " s = yajl_gen_string(hand, (const unsigned char *)se, strlen(se));\n" + s += " else\n" + s += " s = yajl_gen_null(hand);\n" + s += " if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + s += "}\n" + elif isinstance(ty, libxltypes.KeyedUnion): + if parent is None: + raise Exception("KeyedUnion type must have a parent") + for f in ty.fields: + (nparent,fexpr) = ty.member(v, f, parent is None) + keyvar_expr = f.keyvar_expr % (parent + ty.keyvar_name) + s += "if (" + keyvar_expr + ") {\n" + s += libxl_C_type_gen_json(f.type, fexpr, indent + " ", nparent) + s += "}\n" + elif isinstance(ty, libxltypes.Struct) and (parent is None or ty.json_fn is None): + s += "s = yajl_gen_map_open(hand);\n" + s += "if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + for f in [f for f in ty.fields if not f.const]: + (nparent,fexpr) = ty.member(v, f, parent is None) + s += "s = yajl_gen_string(hand, (const unsigned char *)\"%s\", sizeof(\"%s\")-1);\n" % (f.name, f.name) + s += "if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + s += libxl_C_type_gen_json(f.type, fexpr, "", nparent) + s += "s = yajl_gen_map_close(hand);\n" + s += "if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + else: + if ty.json_fn is not None: + s += "s = %s(hand, %s);\n" % (ty.json_fn, ty.pass_arg(v, parent is None)) + s += "if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + + if parent is None: + s += "out:\n" + s += "return s;\n" + + if s != "": + s = indent + s + return s.replace("\n", "\n%s" % indent).rstrip(indent) + +def libxl_C_type_to_json(ty, v, indent = " "): + s = "" + gen = "(libxl__gen_json_callback)&%s_gen_json" % ty.typename + s += "return libxl__object_to_json(ctx, \"%s\", %s, (void *)%s);\n" % (ty.typename, gen, ty.pass_arg(v, passby=libxltypes.PASS_BY_REFERENCE)) + + if s != "": + s = indent + s + return s.replace("\n", "\n%s" % indent).rstrip(indent) + def libxl_C_enum_to_string(ty, e, indent = " "): s = "" s += "switch(%s) {\n" % e @@ -137,11 +194,11 @@ def libxl_C_enum_from_string(ty, str, e, if __name__ == '__main__': - if len(sys.argv) != 4: - print >>sys.stderr, "Usage: gentypes.py <idl> <header> <implementation>" + if len(sys.argv) != 5: + print >>sys.stderr, "Usage: gentypes.py <idl> <header> <header-json> <implementation>" sys.exit(1) - (_, idl, header, impl) = sys.argv + (_, idl, header, header_json, impl) = sys.argv (_,types) = libxltypes.parse(idl) @@ -165,6 +222,8 @@ if __name__ == '__main__': f.write(libxl_C_type_define(ty) + ";\n") if ty.destructor_fn is not None: f.write("void %s(%s);\n" % (ty.destructor_fn, ty.make_arg("p"))) + if ty.json_fn is not None: + f.write("char *%s_to_json(libxl_ctx *ctx, %s);\n" % (ty.typename, ty.make_arg("p"))) if isinstance(ty, libxltypes.Enumeration): f.write("const char *%s_to_string(%s);\n" % (ty.typename, ty.make_arg("p"))) f.write("int %s_from_string(const char *s, %s);\n" % (ty.typename, ty.make_arg("e", passby=libxltypes.PASS_BY_REFERENCE))) @@ -174,6 +233,29 @@ if __name__ == '__main__': f.write("""#endif /* __LIBXL_TYPES_H */\n""") f.close() + print "outputting libxl JSON definitions to %s" % header_json + + f = open(header_json, "w") + + f.write("""#ifndef __LIBXL_TYPES_JSON_H +#define __LIBXL_TYPES_JSON_H + +/* + * DO NOT EDIT. + * + * This file is autogenerated by + * "%s" + */ + +""" % " ".join(sys.argv)) + + for ty in [ty for ty in types if ty.json_fn is not None]: + f.write("yajl_gen_status %s_gen_json(yajl_gen hand, %s);\n" % (ty.typename, ty.make_arg("p", passby=libxltypes.PASS_BY_REFERENCE))) + + f.write("\n") + f.write("""#endif /* __LIBXL_TYPES_JSON_H */\n""") + f.close() + print "outputting libxl type implementations to %s" % impl f = open(impl, "w") @@ -220,5 +302,17 @@ if __name__ == '__main__': f.write("}\n") f.write("\n") + for ty in [t for t in types if t.json_fn is not None]: + f.write("yajl_gen_status %s_gen_json(yajl_gen hand, %s)\n" % (ty.typename, ty.make_arg("p", passby=libxltypes.PASS_BY_REFERENCE))) + f.write("{\n") + f.write(libxl_C_type_gen_json(ty, "p")) + f.write("}\n") + f.write("\n") + + f.write("char *%s_to_json(libxl_ctx *ctx, %s)\n" % (ty.typename, ty.make_arg("p"))) + f.write("{\n") + f.write(libxl_C_type_to_json(ty, "p")) + f.write("}\n") + f.write("\n") f.close() diff -r 4e751a05df3c -r fc8319c37864 tools/libxl/idl.txt --- a/tools/libxl/idl.txt Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/idl.txt Thu Jun 23 17:21:42 2011 +0100 @@ -49,6 +49,15 @@ Type.autogenerate_destructor: (default: Indicates if the above named Type.destructor_fn should be autogenerated. +Type.json_fn: (default: typename + "_gen_json" or None if type == None) + + The name of the C function which will generate a YAJL data structure + representing this type. + +Type.autogenerate_json: (default: True) + + Indicates if the above named Type.json_fn should be autogenerated. + Other simple type-Classes ------------------------- diff -r 4e751a05df3c -r fc8319c37864 tools/libxl/libxl.h --- a/tools/libxl/libxl.h Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/libxl.h Thu Jun 23 17:21:42 2011 +0100 @@ -199,10 +199,10 @@ typedef struct { int v; } libxl_enum_string_table; +typedef struct libxl__ctx libxl_ctx; + #include "_libxl_types.h" -typedef struct libxl__ctx libxl_ctx; - const libxl_version_info* libxl_get_version_info(libxl_ctx *ctx); typedef struct { diff -r 4e751a05df3c -r fc8319c37864 tools/libxl/libxl.idl --- a/tools/libxl/libxl.idl Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/libxl.idl Thu Jun 23 17:21:42 2011 +0100 @@ -3,7 +3,8 @@ # Builtin libxl types # -libxl_domid = Builtin("domid") +libxl_domid = Builtin("domid", json_fn = "yajl_gen_integer", autogenerate_json = False) + libxl_uuid = Builtin("uuid", passby=PASS_BY_REFERENCE) libxl_mac = Builtin("mac", passby=PASS_BY_REFERENCE) libxl_cpumap = Builtin("cpumap", destructor_fn="libxl_cpumap_destroy", passby=PASS_BY_REFERENCE) diff -r 4e751a05df3c -r fc8319c37864 tools/libxl/libxl_internal.h --- a/tools/libxl/libxl_internal.h Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/libxl_internal.h Thu Jun 23 17:21:42 2011 +0100 @@ -35,6 +35,7 @@ #include "flexarray.h" #include "libxl_utils.h" +#include "libxl_json.h" #define LIBXL_DESTROY_TIMEOUT 10 #define LIBXL_DEVICE_MODEL_START_TIMEOUT 10 @@ -328,6 +329,12 @@ _hidden char *libxl__cpupoolid_to_name(l _hidden int libxl__enum_from_string(const libxl_enum_string_table *t, const char *s, int *e); +_hidden yajl_gen_status libxl__string_gen_json(yajl_gen hand, const char *p); + +typedef yajl_gen_status (*libxl__gen_json_callback)(yajl_gen hand, void *); +_hidden char *libxl__object_to_json(libxl_ctx *ctx, const char *type, + libxl__gen_json_callback gen, void *p); + /* holds the CPUID response for a single CPUID leaf * input contains the value of the EAX and ECX register, * and each policy string contains a filter to apply to diff -r 4e751a05df3c -r fc8319c37864 tools/libxl/libxl_json.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/libxl/libxl_json.c Thu Jun 23 17:21:42 2011 +0100 @@ -0,0 +1,287 @@ +/* + * Copyright (C) 2011 Citrix Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published + * by the Free Software Foundation; version 2.1 only. with the special + * exception on linking described in file LICENSE. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + */ + +#include <string.h> + +#include <libxl.h> +#include <libxl_internal.h> + +static yajl_gen_status yajl_gen_asciiz(yajl_gen hand, const char *str) +{ + return yajl_gen_string(hand, (const unsigned char *)str, strlen(str)); +} + +yajl_gen_status libxl_uuid_gen_json(yajl_gen hand, + libxl_uuid *uuid) +{ + char buf[LIBXL_UUID_FMTLEN+1]; + snprintf(buf, sizeof(buf), LIBXL_UUID_FMT, LIBXL_UUID_BYTES((*uuid))); + return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_UUID_FMTLEN); +} + +yajl_gen_status libxl_cpumap_gen_json(yajl_gen hand, + libxl_cpumap *cpumap) +{ + yajl_gen_status s; + int i; + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + libxl_for_each_cpu(i, *cpumap) { + if (libxl_cpumap_test(cpumap, i)) { + s = yajl_gen_integer(hand, i); + if (s != yajl_gen_status_ok) goto out; + } + } + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand, + libxl_key_value_list *pkvl) +{ + libxl_key_value_list kvl = *pkvl; + yajl_gen_status s; + int i; + + s = yajl_gen_map_open(hand); + if (s != yajl_gen_status_ok) goto out; + + if (!kvl) goto empty; + + for (i = 0; kvl[i] != NULL; i += 2) { + s = yajl_gen_asciiz(hand, kvl[i]); + if (s != yajl_gen_status_ok) goto out; + if (kvl[i + 1]) + s = yajl_gen_asciiz(hand, kvl[i+1]); + else + s = yajl_gen_null(hand); + if (s != yajl_gen_status_ok) goto out; + } +empty: + s = yajl_gen_map_close(hand); +out: + return s; +} + +yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand, + libxl_cpuid_policy_list *pcpuid) +{ + libxl_cpuid_policy_list cpuid = *pcpuid; + yajl_gen_status s; + const char *input_names[2] = { "leaf", "subleaf" }; + const char *policy_names[4] = { "eax", "ebx", "ecx", "edx" }; + int i, j; + + /* + * Aiming for: + * [ + * { 'leaf': 'val-eax', + * 'subleaf': 'val-edx', + * 'ebx': 'filter', + * 'ecx': 'filter', + * 'edx': 'filter' }, ], + * { 'leaf': 'val-eax', ..., 'eax': 'filter', ... }, + * ... etc ... + * } + */ + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + if (cpuid == NULL) goto empty; + + for (i = 0; cpuid[i].input[0] != XEN_CPUID_INPUT_UNUSED; i++) { + s = yajl_gen_map_open(hand); + if (s != yajl_gen_status_ok) goto out; + + for (j = 0; j < 2; j++) { + if (cpuid[i].input[j] != XEN_CPUID_INPUT_UNUSED) { + s = yajl_gen_asciiz(hand, input_names[j]); + if (s != yajl_gen_status_ok) goto out; + s = yajl_gen_integer(hand, cpuid[i].input[j]); + if (s != yajl_gen_status_ok) goto out; + } + } + + for (j = 0; j < 4; j++) { + if (cpuid[i].policy[j] != NULL) { + s = yajl_gen_asciiz(hand, policy_names[j]); + if (s != yajl_gen_status_ok) goto out; + s = yajl_gen_string(hand, + (const unsigned char *)cpuid[i].policy[j], 32); + if (s != yajl_gen_status_ok) goto out; + } + } + s = yajl_gen_map_close(hand); + if (s != yajl_gen_status_ok) goto out; + } + +empty: + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, libxl_string_list *pl) +{ + libxl_string_list l = *pl; + yajl_gen_status s; + int i; + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + if (!l) goto empty; + + for (i = 0; l[i] != NULL; i++) { + s = yajl_gen_asciiz(hand, l[i]); + if (s != yajl_gen_status_ok) goto out; + } +empty: + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *mac) +{ + char buf[LIBXL_MAC_FMTLEN+1]; + snprintf(buf, sizeof(buf), LIBXL_MAC_FMT, LIBXL_MAC_BYTES((*mac))); + return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_MAC_FMTLEN); +} + +yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand, + libxl_hwcap p) +{ + yajl_gen_status s; + int i; + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + for(i=0; i<4; i++) { + s = yajl_gen_integer(hand, p[i]); + if (s != yajl_gen_status_ok) goto out; + } + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_cpuarray_gen_json(yajl_gen hand, + libxl_cpuarray *cpuarray) +{ + yajl_gen_status s; + int i; + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + for(i=0; i<cpuarray->entries; i++) { + if (cpuarray->array[i] == LIBXL_CPUARRAY_INVALID_ENTRY) + s = yajl_gen_null(hand); + else + s = yajl_gen_integer(hand, cpuarray->array[i]); + if (s != yajl_gen_status_ok) goto out; + } + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_file_reference_gen_json(yajl_gen hand, + libxl_file_reference *p) +{ + if (p->path) + return yajl_gen_asciiz(hand, p->path); + else + return yajl_gen_null(hand); +} + +yajl_gen_status libxl__string_gen_json(yajl_gen hand, + const char *p) +{ + if (p) + return yajl_gen_asciiz(hand, p); + else + return yajl_gen_null(hand); +} + +static const char *yajl_gen_status_to_string(yajl_gen_status s) +{ + switch (s) { + case yajl_gen_status_ok: abort(); + case yajl_gen_keys_must_be_strings: + return "keys must be strings"; + case yajl_max_depth_exceeded: + return "max depth exceeded"; + case yajl_gen_in_error_state: + return "in error state"; + case yajl_gen_generation_complete: + return "generation complete"; + case yajl_gen_invalid_number: + return "invalid number"; + case yajl_gen_no_buf: + return "no buffer"; +#if 0 /* This is in the docs but not implemented in the version I am running. */ + case yajl_gen_invalid_string: + return "invalid string"; +#endif + default: + return "unknown error"; + } +} + +char *libxl__object_to_json(libxl_ctx *ctx, const char *type, + libxl__gen_json_callback gen, void *p) +{ + yajl_gen_config conf = { 1, " " }; + const unsigned char *buf; + char *ret = NULL; + unsigned int len = 0; + yajl_gen_status s; + yajl_gen hand; + + hand = yajl_gen_alloc(&conf, NULL); + if (!hand) + return NULL; + + s = gen(hand, p); + if (s != yajl_gen_status_ok) + goto out; + + s = yajl_gen_get_buf(hand, &buf, &len); + if (s != yajl_gen_status_ok) + goto out; + ret = strdup((const char *)buf); + +out: + yajl_gen_free(hand); + + if (s != yajl_gen_status_ok) { + LIBXL__LOG(ctx, LIBXL__LOG_ERROR, + "unable to convert %s to JSON representation. " + "YAJL error code %d: %s", type, + s, yajl_gen_status_to_string(s)); + } else if (!ret) { + LIBXL__LOG(ctx, LIBXL__LOG_ERROR, + "unable to allocate space for to JSON representation of %s", + type); + } + + return ret; +} diff -r 4e751a05df3c -r fc8319c37864 tools/libxl/libxl_json.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/libxl/libxl_json.h Thu Jun 23 17:21:42 2011 +0100 @@ -0,0 +1,41 @@ +/* + * Copyright (C) 2011 Citrix Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published + * by the Free Software Foundation; version 2.1 only. with the special + * exception on linking described in file LICENSE. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + */ + +#ifndef LIBXL_JSON_H +#define LIBXL_JSON_H + +#include <yajl/yajl_gen.h> + +#include <_libxl_types_json.h> + +yajl_gen_status libxl_uuid_gen_json(yajl_gen hand, + libxl_uuid *p); +yajl_gen_status libxl_cpumap_gen_json(yajl_gen hand, + libxl_cpumap *p); +yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand, + libxl_key_value_list *p); +yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand, + libxl_cpuid_policy_list *p); +yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, + libxl_string_list *p); +yajl_gen_status libxl_mac_gen_json(yajl_gen hand, + libxl_mac *p); +yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand, + libxl_hwcap p); +yajl_gen_status libxl_cpuarray_gen_json(yajl_gen hand, + libxl_cpuarray *p); +yajl_gen_status libxl_file_reference_gen_json(yajl_gen hand, + libxl_file_reference *p); + +#endif /* LIBXL_JSON_H */ diff -r 4e751a05df3c -r fc8319c37864 tools/libxl/libxltypes.py --- a/tools/libxl/libxltypes.py Thu Jun 23 17:19:16 2011 +0100 +++ b/tools/libxl/libxltypes.py Thu Jun 23 17:21:42 2011 +0100 @@ -37,6 +37,13 @@ class Type(object): self.autogenerate_destructor = kwargs.setdefault('autogenerate_destructor', True) + if self.typename is not None: + self.json_fn = kwargs.setdefault('json_fn', self.typename + "_gen_json") + else: + self.json_fn = kwargs.setdefault('json_fn', None) + + self.autogenerate_json = kwargs.setdefault('autogenerate_json', True) + def marshal_in(self): return self.dir in [DIR_IN, DIR_BOTH] def marshal_out(self): @@ -70,6 +77,7 @@ class Builtin(Type): def __init__(self, typename, **kwargs): kwargs.setdefault('destructor_fn', None) kwargs.setdefault('autogenerate_destructor', False) + kwargs.setdefault('autogenerate_json', False) Type.__init__(self, typename, **kwargs) class Number(Builtin): @@ -77,6 +85,7 @@ class Number(Builtin): kwargs.setdefault('namespace', None) kwargs.setdefault('destructor_fn', None) kwargs.setdefault('signed', False) + kwargs.setdefault('json_fn', "yajl_gen_integer") self.signed = kwargs['signed'] Builtin.__init__(self, ctype, **kwargs) @@ -145,6 +154,8 @@ class Aggregate(Type): comment = None else: n,t,const,comment = f + if n is None: + raise ValueError self.fields.append(Field(t,n,const=const,comment=comment)) # Returns a tuple (stem, field-expr) @@ -199,7 +210,10 @@ class KeyedUnion(Aggregate): # void = Builtin("void *", namespace = None) -bool = Builtin("bool", namespace = None) +bool = Builtin("bool", namespace = None, + json_fn = "yajl_gen_bool", + autogenerate_json = False) + size_t = Number("size_t", namespace = None) integer = Number("int", namespace = None, signed = True) @@ -209,7 +223,9 @@ uint16 = UInt(16) uint32 = UInt(32) uint64 = UInt(64) -string = Builtin("char *", namespace = None, destructor_fn = "free") +string = Builtin("char *", namespace = None, destructor_fn = "free", + json_fn = "libxl__string_gen_json", + autogenerate_json = False) class OrderedDict(dict): """A dictionary which remembers insertion order. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 00 of 10] libxl: IDL improvements and JSON generation 2011-06-23 16:20 [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Campbell ` (9 preceding siblings ...) 2011-06-23 16:21 ` [PATCH 10 of 10] libxl: IDL: autogenerate functions to produce JSON from libxl data structures Ian Campbell @ 2011-06-24 14:54 ` Ian Jackson 10 siblings, 0 replies; 13+ messages in thread From: Ian Jackson @ 2011-06-24 14:54 UTC (permalink / raw) To: Ian Campbell; +Cc: xen-devel Ian Campbell writes ("[Xen-devel] [PATCH 00 of 10] libxl: IDL improvements and JSON generation"): > This patch series makes some improvements to the IDL language and > generator and then goes on to implement JSON generation for IDL > defined datatypes. Nice work. Acked-by: Ian Jackson <ian.jackson@eu.citrix.com> If there are no adverse comments or last-minute changes :-) I will apply this early next week. Ian. ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2011-06-24 14:54 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-06-23 16:20 [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Campbell
2011-06-23 16:20 ` [PATCH 01 of 10] build: define "move-if-changed" make macro Ian Campbell
2011-06-23 16:20 ` [PATCH 02 of 10] libxl: IDL: s/anonynous/anonymous/ Ian Campbell
2011-06-23 16:20 ` [PATCH 03 of 10] libxl: IDL: handle generation of pass-by-reference arguments Ian Campbell
2011-06-23 16:20 ` [PATCH 04 of 10] libxl: IDL: refactor code to massage a type into a function argument Ian Campbell
2011-06-23 16:20 ` [PATCH 05 of 10] libxl: IDL: remove libxl_C_type_of Ian Campbell
2011-06-23 16:20 ` [PATCH 06 of 10] libxl: IDL: add helper to generate references to Aggregate type members Ian Campbell
2011-06-23 16:20 ` [PATCH 07 of 10] libxl: IDL: bring command line handling in genwrap.py into one place Ian Campbell
2011-06-23 16:20 ` [PATCH 08 of 10] libxl: rename testenum->testidl Ian Campbell
2011-06-23 16:21 ` [PATCH 09 of 10] libxl: add LIBXL_MAC_{FMT, FMTLEN, BYTES} Ian Campbell
2011-06-23 16:21 ` [PATCH 10 of 10] libxl: IDL: autogenerate functions to produce JSON from libxl data structures Ian Campbell
2011-06-23 16:23 ` Ian Campbell
2011-06-24 14:54 ` [PATCH 00 of 10] libxl: IDL improvements and JSON generation Ian Jackson
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.