* [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files
@ 2014-02-11 13:00 Lluís Vilanova
2014-02-11 13:00 ` [Qemu-devel] [PATCH v3 1/3] qapi: Use an explicit input file Lluís Vilanova
` (3 more replies)
0 siblings, 4 replies; 9+ messages in thread
From: Lluís Vilanova @ 2014-02-11 13:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Markus Armbruster, Luiz Capitulino
Adds the "include(...)" primitive to the syntax of QAPI schema files, allowing
these to be modularized into multiple per-topic files in the future.
Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
Changes in v3:
* Fix documentation examples regarding how the input file is passed to the
scripts.
* Add documentation for the 'include' directive.
* Detect inclusion loops.
* Fix "tests/qapi-schema/test-qapi.py" and "tests/Makefile" to use an explicit
input file when running tests.
* Fix QAPI tests to cope with an explicit input file.
* Add tests for the "include" directive.
Changes in v2:
* Change the scripts to use an explicit input file instead of standard input.
* Fix "tests/Makefile" to use the new argument.
* Get the input directory for the "include" directive from the input file
dirname.
Lluís Vilanova (3):
qapi: Use an explicit input file
qapi: Add a primitive to include other files from a QAPI schema file
qapi: Add tests for the "include" directive
Makefile | 24 ++++++++++---
docs/qapi-code-gen.txt | 12 +++++-
scripts/qapi-commands.py | 10 ++++-
scripts/qapi-types.py | 9 +++--
scripts/qapi-visit.py | 9 +++--
scripts/qapi.py | 47 +++++++++++++++++++++++--
tests/Makefile | 17 ++++++---
tests/qapi-schema/funny-char.err | 2 +
tests/qapi-schema/include-cycle.err | 1 +
tests/qapi-schema/include-cycle.exit | 1 +
tests/qapi-schema/include-cycle.json | 1 +
tests/qapi-schema/include-cycle.out | 0
tests/qapi-schema/include-nested-err.err | 1 +
tests/qapi-schema/include-nested-err.exit | 1 +
tests/qapi-schema/include-nested-err.json | 1 +
tests/qapi-schema/include-nested-err.out | 0
tests/qapi-schema/include-no-file.err | 1 +
tests/qapi-schema/include-no-file.exit | 1 +
tests/qapi-schema/include-no-file.json | 1 +
tests/qapi-schema/include-no-file.out | 0
tests/qapi-schema/include-simple-sub.json | 2 +
tests/qapi-schema/include-simple.err | 0
tests/qapi-schema/include-simple.exit | 1 +
tests/qapi-schema/include-simple.json | 1 +
tests/qapi-schema/include-simple.out | 3 ++
tests/qapi-schema/missing-colon.err | 2 +
tests/qapi-schema/missing-comma-list.err | 2 +
tests/qapi-schema/missing-comma-object.err | 2 +
tests/qapi-schema/non-objects.err | 2 +
tests/qapi-schema/quoted-structural-chars.err | 2 +
tests/qapi-schema/test-qapi.py | 3 +-
tests/qapi-schema/trailing-comma-list.err | 2 +
tests/qapi-schema/trailing-comma-object.err | 2 +
tests/qapi-schema/unclosed-list.err | 2 +
tests/qapi-schema/unclosed-object.err | 2 +
tests/qapi-schema/unclosed-string.err | 2 +
36 files changed, 131 insertions(+), 38 deletions(-)
create mode 100644 tests/qapi-schema/include-cycle.err
create mode 100644 tests/qapi-schema/include-cycle.exit
create mode 100644 tests/qapi-schema/include-cycle.json
create mode 100644 tests/qapi-schema/include-cycle.out
create mode 100644 tests/qapi-schema/include-nested-err.err
create mode 100644 tests/qapi-schema/include-nested-err.exit
create mode 100644 tests/qapi-schema/include-nested-err.json
create mode 100644 tests/qapi-schema/include-nested-err.out
create mode 100644 tests/qapi-schema/include-no-file.err
create mode 100644 tests/qapi-schema/include-no-file.exit
create mode 100644 tests/qapi-schema/include-no-file.json
create mode 100644 tests/qapi-schema/include-no-file.out
create mode 100644 tests/qapi-schema/include-simple-sub.json
create mode 100644 tests/qapi-schema/include-simple.err
create mode 100644 tests/qapi-schema/include-simple.exit
create mode 100644 tests/qapi-schema/include-simple.json
create mode 100644 tests/qapi-schema/include-simple.out
To: qemu-devel@nongnu.org
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Qemu-devel] [PATCH v3 1/3] qapi: Use an explicit input file
2014-02-11 13:00 [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files Lluís Vilanova
@ 2014-02-11 13:00 ` Lluís Vilanova
2014-02-11 13:00 ` [Qemu-devel] [PATCH v3 2/3] qapi: Add a primitive to include other files from a QAPI schema file Lluís Vilanova
` (2 subsequent siblings)
3 siblings, 0 replies; 9+ messages in thread
From: Lluís Vilanova @ 2014-02-11 13:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Markus Armbruster, Luiz Capitulino
Use an explicit input file on the command-line instead of reading from standard input
Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
Makefile | 24 ++++++++++++++++++------
docs/qapi-code-gen.txt | 4 ++--
scripts/qapi-commands.py | 10 +++++++---
scripts/qapi-types.py | 9 ++++++---
scripts/qapi-visit.py | 9 ++++++---
scripts/qapi.py | 15 +++++++++++----
tests/Makefile | 14 ++++++++++----
tests/qapi-schema/funny-char.err | 2 +-
tests/qapi-schema/missing-colon.err | 2 +-
tests/qapi-schema/missing-comma-list.err | 2 +-
tests/qapi-schema/missing-comma-object.err | 2 +-
tests/qapi-schema/non-objects.err | 2 +-
tests/qapi-schema/quoted-structural-chars.err | 2 +-
tests/qapi-schema/test-qapi.py | 3 ++-
tests/qapi-schema/trailing-comma-list.err | 2 +-
tests/qapi-schema/trailing-comma-object.err | 2 +-
tests/qapi-schema/unclosed-list.err | 2 +-
tests/qapi-schema/unclosed-object.err | 2 +-
tests/qapi-schema/unclosed-string.err | 2 +-
19 files changed, 73 insertions(+), 37 deletions(-)
diff --git a/Makefile b/Makefile
index a9b6e39..50a8e38 100644
--- a/Makefile
+++ b/Makefile
@@ -212,23 +212,35 @@ qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py
qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h :\
$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \
+ $(gen-out-type) -i "$<" -o qga/qapi-generated -p "qga-", \
+ " GEN $@")
qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h :\
$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \
+ $(gen-out-type) -i "$<" -o qga/qapi-generated -p "qga-", \
+ " GEN $@")
qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c :\
$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
+ $(gen-out-type) -i "$<" -o qga/qapi-generated -p "qga-", \
+ " GEN $@")
qapi-types.c qapi-types.h :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py $(gen-out-type) -o "." -b < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \
+ $(gen-out-type) -i "$<" -o "." -b, \
+ " GEN $@")
qapi-visit.c qapi-visit.h :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py $(gen-out-type) -o "." -b < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \
+ $(gen-out-type) -i "$<" -o "." -b, \
+ " GEN $@")
qmp-commands.h qmp-marshal.c :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -m -o "." < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
+ $(gen-out-type) -i "$<" -o "." -m, \
+ " GEN $@")
QGALIB_GEN=$(addprefix qga/qapi-generated/, qga-qapi-types.h qga-qapi-visit.h qga-qmp-commands.h)
$(qga-obj-y) qemu-ga.o: $(QGALIB_GEN)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 0728f36..2e9f036 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -220,7 +220,7 @@ created code.
Example:
mdroth@illuin:~/w/qemu2.git$ python scripts/qapi-types.py \
- --output-dir="qapi-generated" --prefix="example-" < example-schema.json
+ --input-file=example-schema.json --output-dir="qapi-generated" --prefix="example-"
mdroth@illuin:~/w/qemu2.git$ cat qapi-generated/example-qapi-types.c
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
@@ -290,7 +290,7 @@ $(prefix)qapi-visit.h: declarations for previously mentioned visitor
Example:
mdroth@illuin:~/w/qemu2.git$ python scripts/qapi-visit.py \
- --output-dir="qapi-generated" --prefix="example-" < example-schema.json
+ --input-file=example-schema.json --output-dir="qapi-generated" --prefix="example-"
mdroth@illuin:~/w/qemu2.git$ cat qapi-generated/example-qapi-visit.c
/* THIS FILE IS AUTOMATICALLY GENERATED, DO NOT MODIFY */
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index b12b696..1657f21 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -389,13 +389,15 @@ def gen_command_def_prologue(prefix="", proxy=False):
try:
- opts, args = getopt.gnu_getopt(sys.argv[1:], "chp:o:m",
+ opts, args = getopt.gnu_getopt(sys.argv[1:], "chp:i:o:m",
["source", "header", "prefix=",
- "output-dir=", "type=", "middle"])
+ "input-file=", "output-dir=",
+ "type=", "middle"])
except getopt.GetoptError, err:
print str(err)
sys.exit(1)
+input_file = ""
output_dir = ""
prefix = ""
dispatch_type = "sync"
@@ -409,6 +411,8 @@ do_h = False
for o, a in opts:
if o in ("-p", "--prefix"):
prefix = a
+ elif o in ("-i", "--input-file"):
+ input_file = a
elif o in ("-o", "--output-dir"):
output_dir = a + "/"
elif o in ("-t", "--type"):
@@ -440,7 +444,7 @@ except os.error, e:
if e.errno != errno.EEXIST:
raise
-exprs = parse_schema(sys.stdin)
+exprs = parse_schema(input_file)
commands = filter(lambda expr: expr.has_key('command'), exprs)
commands = filter(lambda expr: not expr.has_key('gen'), commands)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 4a1652b..7304543 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -282,14 +282,15 @@ void qapi_free_%(type)s(%(c_type)s obj)
try:
- opts, args = getopt.gnu_getopt(sys.argv[1:], "chbp:o:",
+ opts, args = getopt.gnu_getopt(sys.argv[1:], "chbp:i:o:",
["source", "header", "builtins",
- "prefix=", "output-dir="])
+ "prefix=", "input-file=", "output-dir="])
except getopt.GetoptError, err:
print str(err)
sys.exit(1)
output_dir = ""
+input_file = ""
prefix = ""
c_file = 'qapi-types.c'
h_file = 'qapi-types.h'
@@ -301,6 +302,8 @@ do_builtins = False
for o, a in opts:
if o in ("-p", "--prefix"):
prefix = a
+ elif o in ("-i", "--input-file"):
+ input_file = a
elif o in ("-o", "--output-dir"):
output_dir = a + "/"
elif o in ("-c", "--source"):
@@ -381,7 +384,7 @@ fdecl.write(mcgen('''
''',
guard=guardname(h_file)))
-exprs = parse_schema(sys.stdin)
+exprs = parse_schema(input_file)
exprs = filter(lambda expr: not expr.has_key('gen'), exprs)
fdecl.write(guardstart("QAPI_TYPES_BUILTIN_STRUCT_DECL"))
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 65f1a54..856f969 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -383,13 +383,14 @@ void visit_type_%(name)s(Visitor *m, %(name)s * obj, const char *name, Error **e
name=name)
try:
- opts, args = getopt.gnu_getopt(sys.argv[1:], "chbp:o:",
+ opts, args = getopt.gnu_getopt(sys.argv[1:], "chbp:i:o:",
["source", "header", "builtins", "prefix=",
- "output-dir="])
+ "input-file=", "output-dir="])
except getopt.GetoptError, err:
print str(err)
sys.exit(1)
+input_file = ""
output_dir = ""
prefix = ""
c_file = 'qapi-visit.c'
@@ -402,6 +403,8 @@ do_builtins = False
for o, a in opts:
if o in ("-p", "--prefix"):
prefix = a
+ elif o in ("-i", "--input-file"):
+ input_file = a
elif o in ("-o", "--output-dir"):
output_dir = a + "/"
elif o in ("-c", "--source"):
@@ -480,7 +483,7 @@ fdecl.write(mcgen('''
''',
prefix=prefix, guard=guardname(h_file)))
-exprs = parse_schema(sys.stdin)
+exprs = parse_schema(input_file)
# to avoid header dependency hell, we always generate declarations
# for built-in types in our header files and simply guard them
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 9b3de4c..59c2b9b 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -12,6 +12,7 @@
# See the COPYING.LIB file in the top-level directory.
from ordereddict import OrderedDict
+import os
import sys
builtin_types = [
@@ -37,6 +38,7 @@ builtin_type_qtypes = {
class QAPISchemaError(Exception):
def __init__(self, schema, msg):
+ self.base = schema.error_base
self.fp = schema.fp
self.msg = msg
self.line = self.col = 1
@@ -50,12 +52,17 @@ class QAPISchemaError(Exception):
self.col += 1
def __str__(self):
- return "%s:%s:%s: %s" % (self.fp.name, self.line, self.col, self.msg)
+ name = os.path.relpath(self.fp.name, self.base)
+ return "%s:%s:%s: %s" % (name, self.line, self.col, self.msg)
class QAPISchema:
- def __init__(self, fp):
+ def __init__(self, fp, error_base=None):
self.fp = fp
+ if error_base is None:
+ self.error_base = os.getcwd()
+ else:
+ self.error_base = error_base
self.src = fp.read()
if self.src == '' or self.src[-1] != '\n':
self.src += '\n'
@@ -158,9 +165,9 @@ class QAPISchema:
raise QAPISchemaError(self, 'Expected "{", "[" or string')
return expr
-def parse_schema(fp):
+def parse_schema(input_path, error_base=None):
try:
- schema = QAPISchema(fp)
+ schema = QAPISchema(open(input_path, "r"), error_base)
except QAPISchemaError, e:
print >>sys.stderr, e
exit(1)
diff --git a/tests/Makefile b/tests/Makefile
index fd36eee..947e255 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -174,13 +174,19 @@ tests/test-vmstate$(EXESUF): tests/test-vmstate.o \
tests/test-qapi-types.c tests/test-qapi-types.h :\
$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \
+ $(gen-out-type) -i "$<" -o tests -p "test-", \
+ " GEN $@")
tests/test-qapi-visit.c tests/test-qapi-visit.h :\
$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-visit.py
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \
+ $(gen-out-type) -i "$<" -o tests -p "test-", \
+ " GEN $@")
tests/test-qmp-commands.h tests/test-qmp-marshal.c :\
$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-commands.py
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
+ $(gen-out-type) -i "$<" -o tests -p "test-", \
+ " GEN $@")
tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) libqemuutil.a libqemustub.a
tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y) libqemuutil.a libqemustub.a
@@ -304,7 +310,7 @@ check-tests/test-qapi.py: tests/test-qapi.py
.PHONY: $(patsubst %, check-%, $(check-qapi-schema-y))
$(patsubst %, check-%, $(check-qapi-schema-y)): check-%.json: $(SRC_PATH)/%.json
- $(call quiet-command, PYTHONPATH=$(SRC_PATH)/scripts $(PYTHON) $(SRC_PATH)/tests/qapi-schema/test-qapi.py <$^ >$*.test.out 2>$*.test.err; echo $$? >$*.test.exit, " TEST $*.out")
+ $(call quiet-command, PYTHONPATH=$(SRC_PATH)/scripts $(PYTHON) $(SRC_PATH)/tests/qapi-schema/test-qapi.py "$^" >$*.test.out 2>$*.test.err; echo $$? >$*.test.exit, " TEST $*.out")
@diff -q $(SRC_PATH)/$*.out $*.test.out
@diff -q $(SRC_PATH)/$*.err $*.test.err
@diff -q $(SRC_PATH)/$*.exit $*.test.exit
diff --git a/tests/qapi-schema/funny-char.err b/tests/qapi-schema/funny-char.err
index d3dd293..ee65869 100644
--- a/tests/qapi-schema/funny-char.err
+++ b/tests/qapi-schema/funny-char.err
@@ -1 +1 @@
-<stdin>:2:36: Stray ";"
+funny-char.json:2:36: Stray ";"
diff --git a/tests/qapi-schema/missing-colon.err b/tests/qapi-schema/missing-colon.err
index 9f2a355..676cce5 100644
--- a/tests/qapi-schema/missing-colon.err
+++ b/tests/qapi-schema/missing-colon.err
@@ -1 +1 @@
-<stdin>:1:10: Expected ":"
+missing-colon.json:1:10: Expected ":"
diff --git a/tests/qapi-schema/missing-comma-list.err b/tests/qapi-schema/missing-comma-list.err
index 4fe0700..d0ed8c3 100644
--- a/tests/qapi-schema/missing-comma-list.err
+++ b/tests/qapi-schema/missing-comma-list.err
@@ -1 +1 @@
-<stdin>:2:20: Expected "," or "]"
+missing-comma-list.json:2:20: Expected "," or "]"
diff --git a/tests/qapi-schema/missing-comma-object.err b/tests/qapi-schema/missing-comma-object.err
index b0121b5..ad9b457 100644
--- a/tests/qapi-schema/missing-comma-object.err
+++ b/tests/qapi-schema/missing-comma-object.err
@@ -1 +1 @@
-<stdin>:2:3: Expected "," or "}"
+missing-comma-object.json:2:3: Expected "," or "}"
diff --git a/tests/qapi-schema/non-objects.err b/tests/qapi-schema/non-objects.err
index a6c2dc2..e958cf0 100644
--- a/tests/qapi-schema/non-objects.err
+++ b/tests/qapi-schema/non-objects.err
@@ -1 +1 @@
-<stdin>:1:1: Expected "{"
+non-objects.json:1:1: Expected "{"
diff --git a/tests/qapi-schema/quoted-structural-chars.err b/tests/qapi-schema/quoted-structural-chars.err
index a6c2dc2..77732d0 100644
--- a/tests/qapi-schema/quoted-structural-chars.err
+++ b/tests/qapi-schema/quoted-structural-chars.err
@@ -1 +1 @@
-<stdin>:1:1: Expected "{"
+quoted-structural-chars.json:1:1: Expected "{"
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index b3d1e1d..f97e886 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -12,10 +12,11 @@
from qapi import *
from pprint import pprint
+import os
import sys
try:
- exprs = parse_schema(sys.stdin)
+ exprs = parse_schema(sys.argv[1], os.path.dirname(sys.argv[1]))
except SystemExit:
raise
except:
diff --git a/tests/qapi-schema/trailing-comma-list.err b/tests/qapi-schema/trailing-comma-list.err
index ff839a3..13b79f9 100644
--- a/tests/qapi-schema/trailing-comma-list.err
+++ b/tests/qapi-schema/trailing-comma-list.err
@@ -1 +1 @@
-<stdin>:2:36: Expected "{", "[" or string
+trailing-comma-list.json:2:36: Expected "{", "[" or string
diff --git a/tests/qapi-schema/trailing-comma-object.err b/tests/qapi-schema/trailing-comma-object.err
index f540962..d1d57f0 100644
--- a/tests/qapi-schema/trailing-comma-object.err
+++ b/tests/qapi-schema/trailing-comma-object.err
@@ -1 +1 @@
-<stdin>:2:38: Expected string
+trailing-comma-object.json:2:38: Expected string
diff --git a/tests/qapi-schema/unclosed-list.err b/tests/qapi-schema/unclosed-list.err
index 0e837a7..1a699a2 100644
--- a/tests/qapi-schema/unclosed-list.err
+++ b/tests/qapi-schema/unclosed-list.err
@@ -1 +1 @@
-<stdin>:1:20: Expected "," or "]"
+unclosed-list.json:1:20: Expected "," or "]"
diff --git a/tests/qapi-schema/unclosed-object.err b/tests/qapi-schema/unclosed-object.err
index e6dc950..3ddb126 100644
--- a/tests/qapi-schema/unclosed-object.err
+++ b/tests/qapi-schema/unclosed-object.err
@@ -1 +1 @@
-<stdin>:1:21: Expected "," or "}"
+unclosed-object.json:1:21: Expected "," or "}"
diff --git a/tests/qapi-schema/unclosed-string.err b/tests/qapi-schema/unclosed-string.err
index 948d883..cdd3dca 100644
--- a/tests/qapi-schema/unclosed-string.err
+++ b/tests/qapi-schema/unclosed-string.err
@@ -1 +1 @@
-<stdin>:1:11: Missing terminating "'"
+unclosed-string.json:1:11: Missing terminating "'"
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [Qemu-devel] [PATCH v3 2/3] qapi: Add a primitive to include other files from a QAPI schema file
2014-02-11 13:00 [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files Lluís Vilanova
2014-02-11 13:00 ` [Qemu-devel] [PATCH v3 1/3] qapi: Use an explicit input file Lluís Vilanova
@ 2014-02-11 13:00 ` Lluís Vilanova
2014-02-27 21:14 ` Eric Blake
2014-02-11 13:00 ` [Qemu-devel] [PATCH v3 3/3] qapi: Add tests for the "include" directive Lluís Vilanova
2014-02-27 15:31 ` [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files Luiz Capitulino
3 siblings, 1 reply; 9+ messages in thread
From: Lluís Vilanova @ 2014-02-11 13:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Markus Armbruster, Luiz Capitulino
Adds the "include(...)" primitive to the syntax of QAPI schema files.
Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
docs/qapi-code-gen.txt | 8 ++++++++
scripts/qapi.py | 36 ++++++++++++++++++++++++++++++++++--
2 files changed, 42 insertions(+), 2 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 2e9f036..e007807 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -40,6 +40,14 @@ enumeration types and union types.
Generally speaking, types definitions should always use CamelCase for the type
names. Command names should be all lower case with words separated by a hyphen.
+The QAPI schema definitions can be modularized using the 'include' directive:
+
+ include("sub-system/qapi.json")
+
+All paths are interpreted as relative to the initial input file passed to the
+QAPI parsing scripts.
+
+
=== Complex types ===
A complex type is a dictionary containing a single key whose value is a
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 59c2b9b..a3e2a30 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -11,10 +11,14 @@
# This work is licensed under the terms of the GNU GPLv2.
# See the COPYING.LIB file in the top-level directory.
+import os
+import re
from ordereddict import OrderedDict
import os
import sys
+include_cre = re.compile("include\(\"([^\"]*)\"\)")
+
builtin_types = [
'str', 'int', 'number', 'bool',
'int8', 'int16', 'int32', 'int64',
@@ -57,12 +61,16 @@ class QAPISchemaError(Exception):
class QAPISchema:
- def __init__(self, fp, error_base=None):
+ def __init__(self, fp, input_dir, error_base=None, included=[]):
self.fp = fp
if error_base is None:
self.error_base = os.getcwd()
else:
self.error_base = error_base
+ input_path = os.path.abspath(fp.name)
+ input_path = os.path.relpath(input_path, self.error_base)
+ self.included = included + [input_path]
+ self.input_dir = input_dir
self.src = fp.read()
if self.src == '' or self.src[-1] != '\n':
self.src += '\n'
@@ -107,6 +115,29 @@ class QAPISchema:
if self.cursor == len(self.src):
self.tok = None
return
+ elif self.tok == 'i':
+ include_src = self.src[self.cursor-1:]
+ include_match = include_cre.match(include_src)
+ if include_match is not None:
+ include_path = os.path.join(self.input_dir,
+ include_match.group(1))
+ include_path = os.path.abspath(include_path)
+ if not os.path.isfile(include_path):
+ raise QAPISchemaError(
+ self,
+ 'Non-existing included file "%s"' %
+ include_match.group(1))
+ include_path_rel = os.path.relpath(include_path,
+ self.error_base)
+ included = self.included + [include_path_rel]
+ if include_path_rel in self.included:
+ raise QAPISchemaError(self, "Inifite inclusion loop: %s"
+ % " -> ".join(included))
+ include_schema = QAPISchema(open(include_path, "r"),
+ self.input_dir, self.error_base,
+ included)
+ self.exprs += include_schema.exprs
+ self.cursor += include_match.span()[1] - 1
elif not self.tok.isspace():
raise QAPISchemaError(self, 'Stray "%s"' % self.tok)
@@ -166,8 +197,9 @@ class QAPISchema:
return expr
def parse_schema(input_path, error_base=None):
+ input_dir = os.path.dirname(input_path)
try:
- schema = QAPISchema(open(input_path, "r"), error_base)
+ schema = QAPISchema(open(input_path, "r"), input_dir, error_base)
except QAPISchemaError, e:
print >>sys.stderr, e
exit(1)
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [Qemu-devel] [PATCH v3 3/3] qapi: Add tests for the "include" directive
2014-02-11 13:00 [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files Lluís Vilanova
2014-02-11 13:00 ` [Qemu-devel] [PATCH v3 1/3] qapi: Use an explicit input file Lluís Vilanova
2014-02-11 13:00 ` [Qemu-devel] [PATCH v3 2/3] qapi: Add a primitive to include other files from a QAPI schema file Lluís Vilanova
@ 2014-02-11 13:00 ` Lluís Vilanova
2014-02-27 21:16 ` Eric Blake
2014-02-27 15:31 ` [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files Luiz Capitulino
3 siblings, 1 reply; 9+ messages in thread
From: Lluís Vilanova @ 2014-02-11 13:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Markus Armbruster, Luiz Capitulino
Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
tests/Makefile | 3 ++-
tests/qapi-schema/include-cycle.err | 1 +
tests/qapi-schema/include-cycle.exit | 1 +
tests/qapi-schema/include-cycle.json | 1 +
tests/qapi-schema/include-cycle.out | 0
tests/qapi-schema/include-nested-err.err | 1 +
tests/qapi-schema/include-nested-err.exit | 1 +
tests/qapi-schema/include-nested-err.json | 1 +
tests/qapi-schema/include-nested-err.out | 0
tests/qapi-schema/include-no-file.err | 1 +
tests/qapi-schema/include-no-file.exit | 1 +
tests/qapi-schema/include-no-file.json | 1 +
tests/qapi-schema/include-no-file.out | 0
tests/qapi-schema/include-simple-sub.json | 2 ++
tests/qapi-schema/include-simple.err | 0
tests/qapi-schema/include-simple.exit | 1 +
tests/qapi-schema/include-simple.json | 1 +
tests/qapi-schema/include-simple.out | 3 +++
18 files changed, 18 insertions(+), 1 deletion(-)
create mode 100644 tests/qapi-schema/include-cycle.err
create mode 100644 tests/qapi-schema/include-cycle.exit
create mode 100644 tests/qapi-schema/include-cycle.json
create mode 100644 tests/qapi-schema/include-cycle.out
create mode 100644 tests/qapi-schema/include-nested-err.err
create mode 100644 tests/qapi-schema/include-nested-err.exit
create mode 100644 tests/qapi-schema/include-nested-err.json
create mode 100644 tests/qapi-schema/include-nested-err.out
create mode 100644 tests/qapi-schema/include-no-file.err
create mode 100644 tests/qapi-schema/include-no-file.exit
create mode 100644 tests/qapi-schema/include-no-file.json
create mode 100644 tests/qapi-schema/include-no-file.out
create mode 100644 tests/qapi-schema/include-simple-sub.json
create mode 100644 tests/qapi-schema/include-simple.err
create mode 100644 tests/qapi-schema/include-simple.exit
create mode 100644 tests/qapi-schema/include-simple.json
create mode 100644 tests/qapi-schema/include-simple.out
diff --git a/tests/Makefile b/tests/Makefile
index 947e255..9270ade 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -124,7 +124,8 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
missing-comma-object.json non-objects.json \
qapi-schema-test.json quoted-structural-chars.json \
trailing-comma-list.json trailing-comma-object.json \
- unclosed-list.json unclosed-object.json unclosed-string.json)
+ unclosed-list.json unclosed-object.json unclosed-string.json\
+ include-simple.json include-nested-err.json include-no-file.json include-cycle.json)
GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h tests/test-qmp-commands.h
diff --git a/tests/qapi-schema/include-cycle.err b/tests/qapi-schema/include-cycle.err
new file mode 100644
index 0000000..3f4290c
--- /dev/null
+++ b/tests/qapi-schema/include-cycle.err
@@ -0,0 +1 @@
+include-cycle.json:1:1: Inifite inclusion loop: include-cycle.json -> include-cycle.json
diff --git a/tests/qapi-schema/include-cycle.exit b/tests/qapi-schema/include-cycle.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-cycle.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-cycle.json b/tests/qapi-schema/include-cycle.json
new file mode 100644
index 0000000..90ffa6a
--- /dev/null
+++ b/tests/qapi-schema/include-cycle.json
@@ -0,0 +1 @@
+include("include-cycle.json")
diff --git a/tests/qapi-schema/include-cycle.out b/tests/qapi-schema/include-cycle.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-nested-err.err b/tests/qapi-schema/include-nested-err.err
new file mode 100644
index 0000000..676cce5
--- /dev/null
+++ b/tests/qapi-schema/include-nested-err.err
@@ -0,0 +1 @@
+missing-colon.json:1:10: Expected ":"
diff --git a/tests/qapi-schema/include-nested-err.exit b/tests/qapi-schema/include-nested-err.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-nested-err.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-nested-err.json b/tests/qapi-schema/include-nested-err.json
new file mode 100644
index 0000000..ff53883
--- /dev/null
+++ b/tests/qapi-schema/include-nested-err.json
@@ -0,0 +1 @@
+include("missing-colon.json")
diff --git a/tests/qapi-schema/include-nested-err.out b/tests/qapi-schema/include-nested-err.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-no-file.err b/tests/qapi-schema/include-no-file.err
new file mode 100644
index 0000000..24832c1
--- /dev/null
+++ b/tests/qapi-schema/include-no-file.err
@@ -0,0 +1 @@
+include-no-file.json:1:1: Non-existing included file "include-no-file-sub.json"
diff --git a/tests/qapi-schema/include-no-file.exit b/tests/qapi-schema/include-no-file.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-no-file.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-no-file.json b/tests/qapi-schema/include-no-file.json
new file mode 100644
index 0000000..df762c2
--- /dev/null
+++ b/tests/qapi-schema/include-no-file.json
@@ -0,0 +1 @@
+include("include-no-file-sub.json")
diff --git a/tests/qapi-schema/include-no-file.out b/tests/qapi-schema/include-no-file.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-simple-sub.json b/tests/qapi-schema/include-simple-sub.json
new file mode 100644
index 0000000..4bd4af4
--- /dev/null
+++ b/tests/qapi-schema/include-simple-sub.json
@@ -0,0 +1,2 @@
+{ 'enum': 'Status',
+ 'data': [ 'good', 'bad', 'ugly' ] }
diff --git a/tests/qapi-schema/include-simple.err b/tests/qapi-schema/include-simple.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-simple.exit b/tests/qapi-schema/include-simple.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/include-simple.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/include-simple.json b/tests/qapi-schema/include-simple.json
new file mode 100644
index 0000000..31c81d1
--- /dev/null
+++ b/tests/qapi-schema/include-simple.json
@@ -0,0 +1 @@
+include("include-simple-sub.json")
diff --git a/tests/qapi-schema/include-simple.out b/tests/qapi-schema/include-simple.out
new file mode 100644
index 0000000..e3bd904
--- /dev/null
+++ b/tests/qapi-schema/include-simple.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
+['Status']
+[]
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files
2014-02-11 13:00 [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files Lluís Vilanova
` (2 preceding siblings ...)
2014-02-11 13:00 ` [Qemu-devel] [PATCH v3 3/3] qapi: Add tests for the "include" directive Lluís Vilanova
@ 2014-02-27 15:31 ` Luiz Capitulino
2014-02-27 21:17 ` Eric Blake
2014-03-01 9:10 ` Markus Armbruster
3 siblings, 2 replies; 9+ messages in thread
From: Luiz Capitulino @ 2014-02-27 15:31 UTC (permalink / raw)
To: Lluís Vilanova; +Cc: qemu-devel, Markus Armbruster
On Tue, 11 Feb 2014 14:00:14 +0100
Lluís Vilanova <vilanova@ac.upc.edu> wrote:
> Adds the "include(...)" primitive to the syntax of QAPI schema files, allowing
> these to be modularized into multiple per-topic files in the future.
Markus, Eric, you commented/reviewed v2 of this series. Any comments
against v3?
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qemu-devel] [PATCH v3 2/3] qapi: Add a primitive to include other files from a QAPI schema file
2014-02-11 13:00 ` [Qemu-devel] [PATCH v3 2/3] qapi: Add a primitive to include other files from a QAPI schema file Lluís Vilanova
@ 2014-02-27 21:14 ` Eric Blake
0 siblings, 0 replies; 9+ messages in thread
From: Eric Blake @ 2014-02-27 21:14 UTC (permalink / raw)
To: Lluís Vilanova, qemu-devel; +Cc: Markus Armbruster, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 536 bytes --]
On 02/11/2014 06:00 AM, Lluís Vilanova wrote:
> Adds the "include(...)" primitive to the syntax of QAPI schema files.
>
> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
> + included = self.included + [include_path_rel]
> + if include_path_rel in self.included:
> + raise QAPISchemaError(self, "Inifite inclusion loop: %s"
s/Inifite/Infinite/
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qemu-devel] [PATCH v3 3/3] qapi: Add tests for the "include" directive
2014-02-11 13:00 ` [Qemu-devel] [PATCH v3 3/3] qapi: Add tests for the "include" directive Lluís Vilanova
@ 2014-02-27 21:16 ` Eric Blake
0 siblings, 0 replies; 9+ messages in thread
From: Eric Blake @ 2014-02-27 21:16 UTC (permalink / raw)
To: Lluís Vilanova, qemu-devel; +Cc: Markus Armbruster, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 1295 bytes --]
On 02/11/2014 06:00 AM, Lluís Vilanova wrote:
> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
> ---
> +++ b/tests/Makefile
> @@ -124,7 +124,8 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
> missing-comma-object.json non-objects.json \
> qapi-schema-test.json quoted-structural-chars.json \
> trailing-comma-list.json trailing-comma-object.json \
> - unclosed-list.json unclosed-object.json unclosed-string.json)
> + unclosed-list.json unclosed-object.json unclosed-string.json\
The other lines have a space before \, but that's cosmetic. This will
conflict with Wenchao's series to add enum discriminators of unions, but
that should be an obvious conflict resolution for whichever series goes
in second.
> +++ b/tests/qapi-schema/include-cycle.err
> @@ -0,0 +1 @@
> +include-cycle.json:1:1: Inifite inclusion loop: include-cycle.json -> include-cycle.json
Once the typo in 2/3 is fixed, this needs fixing too.
You tested a single-cycle file, but it might also be worth adding a
circular chain (A includes B, B includes C, C includes A) to prove it
also triggers the cycle-detection logic.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files
2014-02-27 15:31 ` [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files Luiz Capitulino
@ 2014-02-27 21:17 ` Eric Blake
2014-03-01 9:10 ` Markus Armbruster
1 sibling, 0 replies; 9+ messages in thread
From: Eric Blake @ 2014-02-27 21:17 UTC (permalink / raw)
To: Luiz Capitulino, Lluís Vilanova; +Cc: qemu-devel, Markus Armbruster
[-- Attachment #1: Type: text/plain, Size: 812 bytes --]
On 02/27/2014 08:31 AM, Luiz Capitulino wrote:
> On Tue, 11 Feb 2014 14:00:14 +0100
> Lluís Vilanova <vilanova@ac.upc.edu> wrote:
>
>> Adds the "include(...)" primitive to the syntax of QAPI schema files, allowing
>> these to be modularized into multiple per-topic files in the future.
>
> Markus, Eric, you commented/reviewed v2 of this series. Any comments
> against v3?
A couple nits that still need fixing (see individual messages), but if
you feel comfortable fixing them now instead of waiting for a v4, then
go ahead and add:
Series: Reviewed-by: Eric Blake <eblake@redhat.com>
Then again, adding a cycle test for chained inclusion cycles in test 3
may be best with a v4.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files
2014-02-27 15:31 ` [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files Luiz Capitulino
2014-02-27 21:17 ` Eric Blake
@ 2014-03-01 9:10 ` Markus Armbruster
1 sibling, 0 replies; 9+ messages in thread
From: Markus Armbruster @ 2014-03-01 9:10 UTC (permalink / raw)
To: Luiz Capitulino; +Cc: Lluís Vilanova, qemu-devel
Luiz Capitulino <lcapitulino@redhat.com> writes:
> On Tue, 11 Feb 2014 14:00:14 +0100
> Lluís Vilanova <vilanova@ac.upc.edu> wrote:
>
>> Adds the "include(...)" primitive to the syntax of QAPI schema files, allowing
>> these to be modularized into multiple per-topic files in the future.
>
> Markus, Eric, you commented/reviewed v2 of this series. Any comments
> against v3?
I partially reviewed v4 instead.
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2014-03-01 9:11 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-02-11 13:00 [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files Lluís Vilanova
2014-02-11 13:00 ` [Qemu-devel] [PATCH v3 1/3] qapi: Use an explicit input file Lluís Vilanova
2014-02-11 13:00 ` [Qemu-devel] [PATCH v3 2/3] qapi: Add a primitive to include other files from a QAPI schema file Lluís Vilanova
2014-02-27 21:14 ` Eric Blake
2014-02-11 13:00 ` [Qemu-devel] [PATCH v3 3/3] qapi: Add tests for the "include" directive Lluís Vilanova
2014-02-27 21:16 ` Eric Blake
2014-02-27 15:31 ` [Qemu-devel] [PATCH v3 0/3] qapi: Allow modularization of QAPI schema files Luiz Capitulino
2014-02-27 21:17 ` Eric Blake
2014-03-01 9:10 ` Markus Armbruster
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).