* [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
@ 2014-05-02 13:52 Lluís Vilanova
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 1/4] qapi: [trivial] Break long command lines Lluís Vilanova
` (5 more replies)
0 siblings, 6 replies; 26+ messages in thread
From: Lluís Vilanova @ 2014-05-02 13:52 UTC (permalink / raw)
To: qemu-devel; +Cc: Benoît Canet, Markus Armbruster, Luiz Capitulino
Adds an include primitive to the syntax of QAPI schema files, allowing these to
be modularized into multiple per-topic files in the future.
Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
---
Changes in v12:
* Fix "Reviewed-by" lines.
Changes in v11:
* Rename "tests/include-after-err*" to "tests/include-before-err*" (Markus
Armbruster).
* Do not use too many line breaks in "test/Makefile" (Eric Blake).
Changes in v10:
* Various argument/variable name changes (Markus Armbruster).
* Fix path replacement command in error files when executing tests (Markus
Armbruster).
* Catch IOError and translate it to QAPIExprError (Markus Armbruster).
Changes in v9:
* Do not catch unknown exceptions in "tests/qapi-schema/test-qapi.py".
* Show primitive syntax in commit message.
* Do not pre-initialize 'input_file' in "scripts/qapi-commands.py".
* Use a one-liner for test error message checking.
Changes in v8:
* Do not show absolute paths in error messages.
Changes in v7:
* Add tests for relative path inclusion.
* Print inclusion path on all errors.
* Add test to ensure errors after an include have the correct context.
* Squash "qapi.py" changes and tests into a single patch.
* Add test for 'include' directive format.
* s/file path/file name/.
* s/included/include/.
* Move "-i" to the end of the command.
* Fix GNUism when using sed.
Changes in v6:
* Split changes for long-line breaking in makefiles.
* Put documentation on a separate section; reference recursiveness.
* Check (and test) for non-string include arguments (tanks to Benoît Canet).
Changes in v5:
* Rebase on b3706fa.
* Remove 'error_base' argument in 'parse_schema'; fix test checks instead.
* Implement include directive using JSON syntax.
Changes in v4:
* Rebase on 3e890c7.
* Minor cosmetic changes.
* Fix recording of included files in case of a cycle error.
* Add a more complex include cycle test.
Changes in v3:
* Fix documentation examples regarding how the input file is passed to the
scripts.
* Add documentation for the 'include' directive.
* Detect inclusion loops.
* Fix "tests/qapi-schema/test-qapi.py" and "tests/Makefile" to use an explicit
input file when running tests.
* Fix QAPI tests to cope with an explicit input file.
* Add tests for the "include" directive.
Changes in v2:
* Change the scripts to use an explicit input file instead of standard input.
* Fix "tests/Makefile" to use the new argument.
* Get the input directory for the "include" directive from the input file
dirname.
Lluís Vilanova (4):
qapi: [trivial] Break long command lines
qapi: [trivial] Do not catch unknown exceptions in "test-qapi.py"
qapi: Use an explicit input file
qapi: Add a primitive to include other files from a QAPI schema file
Makefile | 24 +++++--
docs/qapi-code-gen.txt | 15 ++++
scripts/qapi-commands.py | 9 ++-
scripts/qapi-types.py | 9 ++-
scripts/qapi-visit.py | 9 ++-
scripts/qapi.py | 70 ++++++++++++++++----
tests/Makefile | 26 ++++++-
tests/qapi-schema/duplicate-key.err | 2 -
.../qapi-schema/flat-union-invalid-branch-key.err | 2 -
.../flat-union-invalid-discriminator.err | 2 -
tests/qapi-schema/flat-union-no-base.err | 2 -
.../flat-union-string-discriminator.err | 2 -
tests/qapi-schema/funny-char.err | 2 -
tests/qapi-schema/include-before-err.err | 1
tests/qapi-schema/include-before-err.exit | 1
tests/qapi-schema/include-before-err.json | 2 +
tests/qapi-schema/include-before-err.out | 0
tests/qapi-schema/include-cycle-b.json | 1
tests/qapi-schema/include-cycle-c.json | 1
tests/qapi-schema/include-cycle.err | 3 +
tests/qapi-schema/include-cycle.exit | 1
tests/qapi-schema/include-cycle.json | 1
tests/qapi-schema/include-cycle.out | 0
tests/qapi-schema/include-format-err.err | 1
tests/qapi-schema/include-format-err.exit | 1
tests/qapi-schema/include-format-err.json | 2 +
tests/qapi-schema/include-format-err.out | 0
tests/qapi-schema/include-nested-err.err | 2 +
tests/qapi-schema/include-nested-err.exit | 1
tests/qapi-schema/include-nested-err.json | 1
tests/qapi-schema/include-nested-err.out | 0
tests/qapi-schema/include-no-file.err | 1
tests/qapi-schema/include-no-file.exit | 1
tests/qapi-schema/include-no-file.json | 1
tests/qapi-schema/include-no-file.out | 0
tests/qapi-schema/include-non-file.err | 1
tests/qapi-schema/include-non-file.exit | 1
tests/qapi-schema/include-non-file.json | 1
tests/qapi-schema/include-non-file.out | 0
tests/qapi-schema/include-relpath-sub.json | 2 +
tests/qapi-schema/include-relpath.err | 0
tests/qapi-schema/include-relpath.exit | 1
tests/qapi-schema/include-relpath.json | 1
tests/qapi-schema/include-relpath.out | 3 +
tests/qapi-schema/include-self-cycle.err | 1
tests/qapi-schema/include-self-cycle.exit | 1
tests/qapi-schema/include-self-cycle.json | 1
tests/qapi-schema/include-self-cycle.out | 0
tests/qapi-schema/include-simple-sub.json | 2 +
tests/qapi-schema/include-simple.err | 0
tests/qapi-schema/include-simple.exit | 1
tests/qapi-schema/include-simple.json | 1
tests/qapi-schema/include-simple.out | 3 +
tests/qapi-schema/include/relpath.json | 1
tests/qapi-schema/missing-colon.err | 2 -
tests/qapi-schema/missing-comma-list.err | 2 -
tests/qapi-schema/missing-comma-object.err | 2 -
tests/qapi-schema/non-objects.err | 2 -
tests/qapi-schema/quoted-structural-chars.err | 2 -
tests/qapi-schema/test-qapi.py | 6 +-
tests/qapi-schema/trailing-comma-list.err | 2 -
tests/qapi-schema/trailing-comma-object.err | 2 -
tests/qapi-schema/unclosed-list.err | 2 -
tests/qapi-schema/unclosed-object.err | 2 -
tests/qapi-schema/unclosed-string.err | 2 -
tests/qapi-schema/union-invalid-base.err | 2 -
66 files changed, 187 insertions(+), 58 deletions(-)
create mode 100644 tests/qapi-schema/include-before-err.err
create mode 100644 tests/qapi-schema/include-before-err.exit
create mode 100644 tests/qapi-schema/include-before-err.json
create mode 100644 tests/qapi-schema/include-before-err.out
create mode 100644 tests/qapi-schema/include-cycle-b.json
create mode 100644 tests/qapi-schema/include-cycle-c.json
create mode 100644 tests/qapi-schema/include-cycle.err
create mode 100644 tests/qapi-schema/include-cycle.exit
create mode 100644 tests/qapi-schema/include-cycle.json
create mode 100644 tests/qapi-schema/include-cycle.out
create mode 100644 tests/qapi-schema/include-format-err.err
create mode 100644 tests/qapi-schema/include-format-err.exit
create mode 100644 tests/qapi-schema/include-format-err.json
create mode 100644 tests/qapi-schema/include-format-err.out
create mode 100644 tests/qapi-schema/include-nested-err.err
create mode 100644 tests/qapi-schema/include-nested-err.exit
create mode 100644 tests/qapi-schema/include-nested-err.json
create mode 100644 tests/qapi-schema/include-nested-err.out
create mode 100644 tests/qapi-schema/include-no-file.err
create mode 100644 tests/qapi-schema/include-no-file.exit
create mode 100644 tests/qapi-schema/include-no-file.json
create mode 100644 tests/qapi-schema/include-no-file.out
create mode 100644 tests/qapi-schema/include-non-file.err
create mode 100644 tests/qapi-schema/include-non-file.exit
create mode 100644 tests/qapi-schema/include-non-file.json
create mode 100644 tests/qapi-schema/include-non-file.out
create mode 100644 tests/qapi-schema/include-relpath-sub.json
create mode 100644 tests/qapi-schema/include-relpath.err
create mode 100644 tests/qapi-schema/include-relpath.exit
create mode 100644 tests/qapi-schema/include-relpath.json
create mode 100644 tests/qapi-schema/include-relpath.out
create mode 100644 tests/qapi-schema/include-self-cycle.err
create mode 100644 tests/qapi-schema/include-self-cycle.exit
create mode 100644 tests/qapi-schema/include-self-cycle.json
create mode 100644 tests/qapi-schema/include-self-cycle.out
create mode 100644 tests/qapi-schema/include-simple-sub.json
create mode 100644 tests/qapi-schema/include-simple.err
create mode 100644 tests/qapi-schema/include-simple.exit
create mode 100644 tests/qapi-schema/include-simple.json
create mode 100644 tests/qapi-schema/include-simple.out
create mode 100644 tests/qapi-schema/include/relpath.json
To: qemu-devel@nongnu.org
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Eric Blake <eblake@redhat.com>
Cc: Benoît Canet <benoit.canet@irqsave.net>
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH v12 1/4] qapi: [trivial] Break long command lines
2014-05-02 13:52 [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files Lluís Vilanova
@ 2014-05-02 13:52 ` Lluís Vilanova
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 2/4] qapi: [trivial] Do not catch unknown exceptions in "test-qapi.py" Lluís Vilanova
` (4 subsequent siblings)
5 siblings, 0 replies; 26+ messages in thread
From: Lluís Vilanova @ 2014-05-02 13:52 UTC (permalink / raw)
To: qemu-devel; +Cc: Benoît Canet, Markus Armbruster, Luiz Capitulino
Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
---
Makefile | 24 ++++++++++++++++++------
tests/Makefile | 18 ++++++++++++++----
2 files changed, 32 insertions(+), 10 deletions(-)
diff --git a/Makefile b/Makefile
index ec74039..84345ee 100644
--- a/Makefile
+++ b/Makefile
@@ -237,23 +237,35 @@ qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py
qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h :\
$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \
+ $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, \
+ " GEN $@")
qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h :\
$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \
+ $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, \
+ " GEN $@")
qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c :\
$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
+ $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, \
+ " GEN $@")
qapi-types.c qapi-types.h :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py $(gen-out-type) -o "." -b < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \
+ $(gen-out-type) -o "." -b < $<, \
+ " GEN $@")
qapi-visit.c qapi-visit.h :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py $(gen-out-type) -o "." -b < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \
+ $(gen-out-type) -o "." -b < $<, \
+ " GEN $@")
qmp-commands.h qmp-marshal.c :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -m -o "." < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
+ $(gen-out-type) -o "." -m < $<, \
+ " GEN $@")
QGALIB_GEN=$(addprefix qga/qapi-generated/, qga-qapi-types.h qga-qapi-visit.h qga-qmp-commands.h)
$(qga-obj-y) qemu-ga.o: $(QGALIB_GEN)
diff --git a/tests/Makefile b/tests/Makefile
index 2d021fb..02ba6dd 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -216,13 +216,19 @@ tests/test-vmstate$(EXESUF): tests/test-vmstate.o \
tests/test-qapi-types.c tests/test-qapi-types.h :\
$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \
+ $(gen-out-type) -o tests -p "test-" < $<, \
+ " GEN $@")
tests/test-qapi-visit.c tests/test-qapi-visit.h :\
$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-visit.py
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \
+ $(gen-out-type) -o tests -p "test-" < $<, \
+ " GEN $@")
tests/test-qmp-commands.h tests/test-qmp-marshal.c :\
$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-commands.py
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
+ $(gen-out-type) -o tests -p "test-" < $<, \
+ " GEN $@")
tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) libqemuutil.a libqemustub.a
tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y) libqemuutil.a libqemustub.a
@@ -362,7 +368,11 @@ check-tests/test-qapi.py: tests/test-qapi.py
.PHONY: $(patsubst %, check-%, $(check-qapi-schema-y))
$(patsubst %, check-%, $(check-qapi-schema-y)): check-%.json: $(SRC_PATH)/%.json
- $(call quiet-command, PYTHONPATH=$(SRC_PATH)/scripts $(PYTHON) $(SRC_PATH)/tests/qapi-schema/test-qapi.py <$^ >$*.test.out 2>$*.test.err; echo $$? >$*.test.exit, " TEST $*.out")
+ $(call quiet-command, PYTHONPATH=$(SRC_PATH)/scripts \
+ $(PYTHON) $(SRC_PATH)/tests/qapi-schema/test-qapi.py \
+ <$^ >$*.test.out 2>$*.test.err; \
+ echo $$? >$*.test.exit, \
+ " TEST $*.out")
@diff -q $(SRC_PATH)/$*.out $*.test.out
@diff -q $(SRC_PATH)/$*.err $*.test.err
@diff -q $(SRC_PATH)/$*.exit $*.test.exit
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH v12 2/4] qapi: [trivial] Do not catch unknown exceptions in "test-qapi.py"
2014-05-02 13:52 [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files Lluís Vilanova
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 1/4] qapi: [trivial] Break long command lines Lluís Vilanova
@ 2014-05-02 13:52 ` Lluís Vilanova
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 3/4] qapi: Use an explicit input file Lluís Vilanova
` (3 subsequent siblings)
5 siblings, 0 replies; 26+ messages in thread
From: Lluís Vilanova @ 2014-05-02 13:52 UTC (permalink / raw)
To: qemu-devel; +Cc: Benoît Canet, Markus Armbruster, Luiz Capitulino
Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
---
tests/qapi-schema/test-qapi.py | 3 ---
1 file changed, 3 deletions(-)
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index b3d1e1d..5a26ef3 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -18,9 +18,6 @@ try:
exprs = parse_schema(sys.stdin)
except SystemExit:
raise
-except:
- print >>sys.stderr, "Crashed:", sys.exc_info()[0]
- exit(1)
pprint(exprs)
pprint(enum_types)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH v12 3/4] qapi: Use an explicit input file
2014-05-02 13:52 [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files Lluís Vilanova
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 1/4] qapi: [trivial] Break long command lines Lluís Vilanova
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 2/4] qapi: [trivial] Do not catch unknown exceptions in "test-qapi.py" Lluís Vilanova
@ 2014-05-02 13:52 ` Lluís Vilanova
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 4/4] qapi: Add a primitive to include other files from a QAPI schema file Lluís Vilanova
` (2 subsequent siblings)
5 siblings, 0 replies; 26+ messages in thread
From: Lluís Vilanova @ 2014-05-02 13:52 UTC (permalink / raw)
To: qemu-devel; +Cc: Benoît Canet, Markus Armbruster, Luiz Capitulino
Use an explicit input file on the command-line instead of reading from standard
input.
It also outputs the proper file name when there's an error.
Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
---
Makefile | 12 ++++++------
docs/qapi-code-gen.txt | 4 ++--
scripts/qapi-commands.py | 9 ++++++---
scripts/qapi-types.py | 9 ++++++---
scripts/qapi-visit.py | 9 ++++++---
scripts/qapi.py | 5 +++--
tests/Makefile | 11 ++++++-----
tests/qapi-schema/duplicate-key.err | 2 +-
.../qapi-schema/flat-union-invalid-branch-key.err | 2 +-
.../flat-union-invalid-discriminator.err | 2 +-
tests/qapi-schema/flat-union-no-base.err | 2 +-
.../flat-union-string-discriminator.err | 2 +-
tests/qapi-schema/funny-char.err | 2 +-
tests/qapi-schema/missing-colon.err | 2 +-
tests/qapi-schema/missing-comma-list.err | 2 +-
tests/qapi-schema/missing-comma-object.err | 2 +-
tests/qapi-schema/non-objects.err | 2 +-
tests/qapi-schema/quoted-structural-chars.err | 2 +-
tests/qapi-schema/test-qapi.py | 3 ++-
tests/qapi-schema/trailing-comma-list.err | 2 +-
tests/qapi-schema/trailing-comma-object.err | 2 +-
tests/qapi-schema/unclosed-list.err | 2 +-
tests/qapi-schema/unclosed-object.err | 2 +-
tests/qapi-schema/unclosed-string.err | 2 +-
tests/qapi-schema/union-invalid-base.err | 2 +-
25 files changed, 54 insertions(+), 42 deletions(-)
diff --git a/Makefile b/Makefile
index 84345ee..ac6a047 100644
--- a/Makefile
+++ b/Makefile
@@ -238,33 +238,33 @@ qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py
qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h :\
$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \
- $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, \
+ $(gen-out-type) -o qga/qapi-generated -p "qga-" -i $<, \
" GEN $@")
qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h :\
$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \
- $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, \
+ $(gen-out-type) -o qga/qapi-generated -p "qga-" -i $<, \
" GEN $@")
qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c :\
$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
- $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, \
+ $(gen-out-type) -o qga/qapi-generated -p "qga-" -i $<, \
" GEN $@")
qapi-types.c qapi-types.h :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \
- $(gen-out-type) -o "." -b < $<, \
+ $(gen-out-type) -o "." -b -i $<, \
" GEN $@")
qapi-visit.c qapi-visit.h :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \
- $(gen-out-type) -o "." -b < $<, \
+ $(gen-out-type) -o "." -b -i $<, \
" GEN $@")
qmp-commands.h qmp-marshal.c :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
- $(gen-out-type) -o "." -m < $<, \
+ $(gen-out-type) -o "." -m -i $<, \
" GEN $@")
QGALIB_GEN=$(addprefix qga/qapi-generated/, qga-qapi-types.h qga-qapi-visit.h qga-qmp-commands.h)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index d78921f..63b03cf 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -221,7 +221,7 @@ created code.
Example:
mdroth@illuin:~/w/qemu2.git$ python scripts/qapi-types.py \
- --output-dir="qapi-generated" --prefix="example-" < example-schema.json
+ --output-dir="qapi-generated" --prefix="example-" --input-file=example-schema.json
mdroth@illuin:~/w/qemu2.git$ cat qapi-generated/example-qapi-types.c
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
@@ -291,7 +291,7 @@ $(prefix)qapi-visit.h: declarations for previously mentioned visitor
Example:
mdroth@illuin:~/w/qemu2.git$ python scripts/qapi-visit.py \
- --output-dir="qapi-generated" --prefix="example-" < example-schema.json
+ --output-dir="qapi-generated" --prefix="example-" --input-file=example-schema.json
mdroth@illuin:~/w/qemu2.git$ cat qapi-generated/example-qapi-visit.c
/* THIS FILE IS AUTOMATICALLY GENERATED, DO NOT MODIFY */
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 9734ab0..8d9096f 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -369,9 +369,10 @@ def gen_command_def_prologue(prefix="", proxy=False):
try:
- opts, args = getopt.gnu_getopt(sys.argv[1:], "chp:o:m",
+ opts, args = getopt.gnu_getopt(sys.argv[1:], "chp:i:o:m",
["source", "header", "prefix=",
- "output-dir=", "type=", "middle"])
+ "input-file=", "output-dir=",
+ "type=", "middle"])
except getopt.GetoptError, err:
print str(err)
sys.exit(1)
@@ -389,6 +390,8 @@ do_h = False
for o, a in opts:
if o in ("-p", "--prefix"):
prefix = a
+ elif o in ("-i", "--input-file"):
+ input_file = a
elif o in ("-o", "--output-dir"):
output_dir = a + "/"
elif o in ("-t", "--type"):
@@ -420,7 +423,7 @@ except os.error, e:
if e.errno != errno.EEXIST:
raise
-exprs = parse_schema(sys.stdin)
+exprs = parse_schema(input_file)
commands = filter(lambda expr: expr.has_key('command'), exprs)
commands = filter(lambda expr: not expr.has_key('gen'), commands)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 10864ef..b463232 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -279,14 +279,15 @@ void qapi_free_%(type)s(%(c_type)s obj)
try:
- opts, args = getopt.gnu_getopt(sys.argv[1:], "chbp:o:",
+ opts, args = getopt.gnu_getopt(sys.argv[1:], "chbp:i:o:",
["source", "header", "builtins",
- "prefix=", "output-dir="])
+ "prefix=", "input-file=", "output-dir="])
except getopt.GetoptError, err:
print str(err)
sys.exit(1)
output_dir = ""
+input_file = ""
prefix = ""
c_file = 'qapi-types.c'
h_file = 'qapi-types.h'
@@ -298,6 +299,8 @@ do_builtins = False
for o, a in opts:
if o in ("-p", "--prefix"):
prefix = a
+ elif o in ("-i", "--input-file"):
+ input_file = a
elif o in ("-o", "--output-dir"):
output_dir = a + "/"
elif o in ("-c", "--source"):
@@ -378,7 +381,7 @@ fdecl.write(mcgen('''
''',
guard=guardname(h_file)))
-exprs = parse_schema(sys.stdin)
+exprs = parse_schema(input_file)
exprs = filter(lambda expr: not expr.has_key('gen'), exprs)
fdecl.write(guardstart("QAPI_TYPES_BUILTIN_STRUCT_DECL"))
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 45ce3a9..c6579be 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -397,13 +397,14 @@ void visit_type_%(name)s(Visitor *m, %(name)s * obj, const char *name, Error **e
name=name)
try:
- opts, args = getopt.gnu_getopt(sys.argv[1:], "chbp:o:",
+ opts, args = getopt.gnu_getopt(sys.argv[1:], "chbp:i:o:",
["source", "header", "builtins", "prefix=",
- "output-dir="])
+ "input-file=", "output-dir="])
except getopt.GetoptError, err:
print str(err)
sys.exit(1)
+input_file = ""
output_dir = ""
prefix = ""
c_file = 'qapi-visit.c'
@@ -416,6 +417,8 @@ do_builtins = False
for o, a in opts:
if o in ("-p", "--prefix"):
prefix = a
+ elif o in ("-i", "--input-file"):
+ input_file = a
elif o in ("-o", "--output-dir"):
output_dir = a + "/"
elif o in ("-c", "--source"):
@@ -494,7 +497,7 @@ fdecl.write(mcgen('''
''',
prefix=prefix, guard=guardname(h_file)))
-exprs = parse_schema(sys.stdin)
+exprs = parse_schema(input_file)
# to avoid header dependency hell, we always generate declarations
# for built-in types in our header files and simply guard them
diff --git a/scripts/qapi.py b/scripts/qapi.py
index b474c39..07a7a82 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -12,6 +12,7 @@
# See the COPYING file in the top-level directory.
from ordereddict import OrderedDict
+import os
import sys
builtin_types = [
@@ -263,9 +264,9 @@ def check_exprs(schema):
if expr.has_key('union'):
check_union(expr, expr_elem['info'])
-def parse_schema(fp):
+def parse_schema(input_file):
try:
- schema = QAPISchema(fp)
+ schema = QAPISchema(open(input_file, "r"))
except QAPISchemaError, e:
print >>sys.stderr, e
exit(1)
diff --git a/tests/Makefile b/tests/Makefile
index 02ba6dd..3aa9b0b 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -217,17 +217,17 @@ tests/test-vmstate$(EXESUF): tests/test-vmstate.o \
tests/test-qapi-types.c tests/test-qapi-types.h :\
$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \
- $(gen-out-type) -o tests -p "test-" < $<, \
+ $(gen-out-type) -o tests -p "test-" -i $<, \
" GEN $@")
tests/test-qapi-visit.c tests/test-qapi-visit.h :\
$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-visit.py
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \
- $(gen-out-type) -o tests -p "test-" < $<, \
+ $(gen-out-type) -o tests -p "test-" -i $<, \
" GEN $@")
tests/test-qmp-commands.h tests/test-qmp-marshal.c :\
$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-commands.py
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
- $(gen-out-type) -o tests -p "test-" < $<, \
+ $(gen-out-type) -o tests -p "test-" -i $<, \
" GEN $@")
tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) libqemuutil.a libqemustub.a
@@ -370,11 +370,12 @@ check-tests/test-qapi.py: tests/test-qapi.py
$(patsubst %, check-%, $(check-qapi-schema-y)): check-%.json: $(SRC_PATH)/%.json
$(call quiet-command, PYTHONPATH=$(SRC_PATH)/scripts \
$(PYTHON) $(SRC_PATH)/tests/qapi-schema/test-qapi.py \
- <$^ >$*.test.out 2>$*.test.err; \
+ $^ >$*.test.out 2>$*.test.err; \
echo $$? >$*.test.exit, \
" TEST $*.out")
@diff -q $(SRC_PATH)/$*.out $*.test.out
- @diff -q $(SRC_PATH)/$*.err $*.test.err
+ @# Sanitize error messages (make them independent of build directory)
+ @perl -p -e 's|\Q$(SRC_PATH)\E/||g' $*.test.err | diff -q $(SRC_PATH)/$*.err -
@diff -q $(SRC_PATH)/$*.exit $*.test.exit
# Consolidated targets
diff --git a/tests/qapi-schema/duplicate-key.err b/tests/qapi-schema/duplicate-key.err
index 0801c6a..768b276 100644
--- a/tests/qapi-schema/duplicate-key.err
+++ b/tests/qapi-schema/duplicate-key.err
@@ -1 +1 @@
-<stdin>:2:10: Duplicate key "key"
+tests/qapi-schema/duplicate-key.json:2:10: Duplicate key "key"
diff --git a/tests/qapi-schema/flat-union-invalid-branch-key.err b/tests/qapi-schema/flat-union-invalid-branch-key.err
index 1125caf..ccf72d2 100644
--- a/tests/qapi-schema/flat-union-invalid-branch-key.err
+++ b/tests/qapi-schema/flat-union-invalid-branch-key.err
@@ -1 +1 @@
-<stdin>:13: Discriminator value 'value_wrong' is not found in enum 'TestEnum'
+tests/qapi-schema/flat-union-invalid-branch-key.json:13: Discriminator value 'value_wrong' is not found in enum 'TestEnum'
diff --git a/tests/qapi-schema/flat-union-invalid-discriminator.err b/tests/qapi-schema/flat-union-invalid-discriminator.err
index cad9dbf..790b675 100644
--- a/tests/qapi-schema/flat-union-invalid-discriminator.err
+++ b/tests/qapi-schema/flat-union-invalid-discriminator.err
@@ -1 +1 @@
-<stdin>:13: Discriminator 'enum_wrong' is not a member of base type 'TestBase'
+tests/qapi-schema/flat-union-invalid-discriminator.json:13: Discriminator 'enum_wrong' is not a member of base type 'TestBase'
diff --git a/tests/qapi-schema/flat-union-no-base.err b/tests/qapi-schema/flat-union-no-base.err
index e2d7443..a59749e 100644
--- a/tests/qapi-schema/flat-union-no-base.err
+++ b/tests/qapi-schema/flat-union-no-base.err
@@ -1 +1 @@
-<stdin>:7: Flat union 'TestUnion' must have a base field
+tests/qapi-schema/flat-union-no-base.json:7: Flat union 'TestUnion' must have a base field
diff --git a/tests/qapi-schema/flat-union-string-discriminator.err b/tests/qapi-schema/flat-union-string-discriminator.err
index 8748270..200016b 100644
--- a/tests/qapi-schema/flat-union-string-discriminator.err
+++ b/tests/qapi-schema/flat-union-string-discriminator.err
@@ -1 +1 @@
-<stdin>:13: Discriminator 'kind' must be of enumeration type
+tests/qapi-schema/flat-union-string-discriminator.json:13: Discriminator 'kind' must be of enumeration type
diff --git a/tests/qapi-schema/funny-char.err b/tests/qapi-schema/funny-char.err
index d3dd293..bfc890c 100644
--- a/tests/qapi-schema/funny-char.err
+++ b/tests/qapi-schema/funny-char.err
@@ -1 +1 @@
-<stdin>:2:36: Stray ";"
+tests/qapi-schema/funny-char.json:2:36: Stray ";"
diff --git a/tests/qapi-schema/missing-colon.err b/tests/qapi-schema/missing-colon.err
index 9f2a355..d9d66b3 100644
--- a/tests/qapi-schema/missing-colon.err
+++ b/tests/qapi-schema/missing-colon.err
@@ -1 +1 @@
-<stdin>:1:10: Expected ":"
+tests/qapi-schema/missing-colon.json:1:10: Expected ":"
diff --git a/tests/qapi-schema/missing-comma-list.err b/tests/qapi-schema/missing-comma-list.err
index 4fe0700..e73d277 100644
--- a/tests/qapi-schema/missing-comma-list.err
+++ b/tests/qapi-schema/missing-comma-list.err
@@ -1 +1 @@
-<stdin>:2:20: Expected "," or "]"
+tests/qapi-schema/missing-comma-list.json:2:20: Expected "," or "]"
diff --git a/tests/qapi-schema/missing-comma-object.err b/tests/qapi-schema/missing-comma-object.err
index b0121b5..52b3a8a 100644
--- a/tests/qapi-schema/missing-comma-object.err
+++ b/tests/qapi-schema/missing-comma-object.err
@@ -1 +1 @@
-<stdin>:2:3: Expected "," or "}"
+tests/qapi-schema/missing-comma-object.json:2:3: Expected "," or "}"
diff --git a/tests/qapi-schema/non-objects.err b/tests/qapi-schema/non-objects.err
index a6c2dc2..334f0c9 100644
--- a/tests/qapi-schema/non-objects.err
+++ b/tests/qapi-schema/non-objects.err
@@ -1 +1 @@
-<stdin>:1:1: Expected "{"
+tests/qapi-schema/non-objects.json:1:1: Expected "{"
diff --git a/tests/qapi-schema/quoted-structural-chars.err b/tests/qapi-schema/quoted-structural-chars.err
index a6c2dc2..9b18384 100644
--- a/tests/qapi-schema/quoted-structural-chars.err
+++ b/tests/qapi-schema/quoted-structural-chars.err
@@ -1 +1 @@
-<stdin>:1:1: Expected "{"
+tests/qapi-schema/quoted-structural-chars.json:1:1: Expected "{"
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index 5a26ef3..634ef2d 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -12,10 +12,11 @@
from qapi import *
from pprint import pprint
+import os
import sys
try:
- exprs = parse_schema(sys.stdin)
+ exprs = parse_schema(sys.argv[1])
except SystemExit:
raise
diff --git a/tests/qapi-schema/trailing-comma-list.err b/tests/qapi-schema/trailing-comma-list.err
index ff839a3..24c24b0 100644
--- a/tests/qapi-schema/trailing-comma-list.err
+++ b/tests/qapi-schema/trailing-comma-list.err
@@ -1 +1 @@
-<stdin>:2:36: Expected "{", "[" or string
+tests/qapi-schema/trailing-comma-list.json:2:36: Expected "{", "[" or string
diff --git a/tests/qapi-schema/trailing-comma-object.err b/tests/qapi-schema/trailing-comma-object.err
index f540962..30bce5e 100644
--- a/tests/qapi-schema/trailing-comma-object.err
+++ b/tests/qapi-schema/trailing-comma-object.err
@@ -1 +1 @@
-<stdin>:2:38: Expected string
+tests/qapi-schema/trailing-comma-object.json:2:38: Expected string
diff --git a/tests/qapi-schema/unclosed-list.err b/tests/qapi-schema/unclosed-list.err
index 0e837a7..fb41a86 100644
--- a/tests/qapi-schema/unclosed-list.err
+++ b/tests/qapi-schema/unclosed-list.err
@@ -1 +1 @@
-<stdin>:1:20: Expected "," or "]"
+tests/qapi-schema/unclosed-list.json:1:20: Expected "," or "]"
diff --git a/tests/qapi-schema/unclosed-object.err b/tests/qapi-schema/unclosed-object.err
index e6dc950..db3deed 100644
--- a/tests/qapi-schema/unclosed-object.err
+++ b/tests/qapi-schema/unclosed-object.err
@@ -1 +1 @@
-<stdin>:1:21: Expected "," or "}"
+tests/qapi-schema/unclosed-object.json:1:21: Expected "," or "}"
diff --git a/tests/qapi-schema/unclosed-string.err b/tests/qapi-schema/unclosed-string.err
index 948d883..12b1870 100644
--- a/tests/qapi-schema/unclosed-string.err
+++ b/tests/qapi-schema/unclosed-string.err
@@ -1 +1 @@
-<stdin>:1:11: Missing terminating "'"
+tests/qapi-schema/unclosed-string.json:1:11: Missing terminating "'"
diff --git a/tests/qapi-schema/union-invalid-base.err b/tests/qapi-schema/union-invalid-base.err
index dd8e3d1..938f969 100644
--- a/tests/qapi-schema/union-invalid-base.err
+++ b/tests/qapi-schema/union-invalid-base.err
@@ -1 +1 @@
-<stdin>:7: Base 'TestBaseWrong' is not a valid type
+tests/qapi-schema/union-invalid-base.json:7: Base 'TestBaseWrong' is not a valid type
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH v12 4/4] qapi: Add a primitive to include other files from a QAPI schema file
2014-05-02 13:52 [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files Lluís Vilanova
` (2 preceding siblings ...)
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 3/4] qapi: Use an explicit input file Lluís Vilanova
@ 2014-05-02 13:52 ` Lluís Vilanova
2014-05-05 20:31 ` Luiz Capitulino
2014-05-07 18:46 ` Lluís Vilanova
2014-05-05 18:21 ` [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files Luiz Capitulino
2014-05-06 13:07 ` Benoît Canet
5 siblings, 2 replies; 26+ messages in thread
From: Lluís Vilanova @ 2014-05-02 13:52 UTC (permalink / raw)
To: qemu-devel; +Cc: Benoît Canet, Markus Armbruster, Luiz Capitulino
The primitive uses JSON syntax, and include paths are relative to the file using the directive:
{ 'include': 'path/to/file.json' }
Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
---
docs/qapi-code-gen.txt | 11 +++++
scripts/qapi.py | 65 +++++++++++++++++++++++-----
tests/Makefile | 5 ++
tests/qapi-schema/include-before-err.err | 1
tests/qapi-schema/include-before-err.exit | 1
tests/qapi-schema/include-before-err.json | 2 +
tests/qapi-schema/include-before-err.out | 0
tests/qapi-schema/include-cycle-b.json | 1
tests/qapi-schema/include-cycle-c.json | 1
tests/qapi-schema/include-cycle.err | 3 +
tests/qapi-schema/include-cycle.exit | 1
tests/qapi-schema/include-cycle.json | 1
tests/qapi-schema/include-cycle.out | 0
tests/qapi-schema/include-format-err.err | 1
tests/qapi-schema/include-format-err.exit | 1
tests/qapi-schema/include-format-err.json | 2 +
tests/qapi-schema/include-format-err.out | 0
tests/qapi-schema/include-nested-err.err | 2 +
tests/qapi-schema/include-nested-err.exit | 1
tests/qapi-schema/include-nested-err.json | 1
tests/qapi-schema/include-nested-err.out | 0
tests/qapi-schema/include-no-file.err | 1
tests/qapi-schema/include-no-file.exit | 1
tests/qapi-schema/include-no-file.json | 1
tests/qapi-schema/include-no-file.out | 0
tests/qapi-schema/include-non-file.err | 1
tests/qapi-schema/include-non-file.exit | 1
tests/qapi-schema/include-non-file.json | 1
tests/qapi-schema/include-non-file.out | 0
tests/qapi-schema/include-relpath-sub.json | 2 +
tests/qapi-schema/include-relpath.err | 0
tests/qapi-schema/include-relpath.exit | 1
tests/qapi-schema/include-relpath.json | 1
tests/qapi-schema/include-relpath.out | 3 +
tests/qapi-schema/include-self-cycle.err | 1
tests/qapi-schema/include-self-cycle.exit | 1
tests/qapi-schema/include-self-cycle.json | 1
tests/qapi-schema/include-self-cycle.out | 0
tests/qapi-schema/include-simple-sub.json | 2 +
tests/qapi-schema/include-simple.err | 0
tests/qapi-schema/include-simple.exit | 1
tests/qapi-schema/include-simple.json | 1
tests/qapi-schema/include-simple.out | 3 +
tests/qapi-schema/include/relpath.json | 1
44 files changed, 111 insertions(+), 13 deletions(-)
create mode 100644 tests/qapi-schema/include-before-err.err
create mode 100644 tests/qapi-schema/include-before-err.exit
create mode 100644 tests/qapi-schema/include-before-err.json
create mode 100644 tests/qapi-schema/include-before-err.out
create mode 100644 tests/qapi-schema/include-cycle-b.json
create mode 100644 tests/qapi-schema/include-cycle-c.json
create mode 100644 tests/qapi-schema/include-cycle.err
create mode 100644 tests/qapi-schema/include-cycle.exit
create mode 100644 tests/qapi-schema/include-cycle.json
create mode 100644 tests/qapi-schema/include-cycle.out
create mode 100644 tests/qapi-schema/include-format-err.err
create mode 100644 tests/qapi-schema/include-format-err.exit
create mode 100644 tests/qapi-schema/include-format-err.json
create mode 100644 tests/qapi-schema/include-format-err.out
create mode 100644 tests/qapi-schema/include-nested-err.err
create mode 100644 tests/qapi-schema/include-nested-err.exit
create mode 100644 tests/qapi-schema/include-nested-err.json
create mode 100644 tests/qapi-schema/include-nested-err.out
create mode 100644 tests/qapi-schema/include-no-file.err
create mode 100644 tests/qapi-schema/include-no-file.exit
create mode 100644 tests/qapi-schema/include-no-file.json
create mode 100644 tests/qapi-schema/include-no-file.out
create mode 100644 tests/qapi-schema/include-non-file.err
create mode 100644 tests/qapi-schema/include-non-file.exit
create mode 100644 tests/qapi-schema/include-non-file.json
create mode 100644 tests/qapi-schema/include-non-file.out
create mode 100644 tests/qapi-schema/include-relpath-sub.json
create mode 100644 tests/qapi-schema/include-relpath.err
create mode 100644 tests/qapi-schema/include-relpath.exit
create mode 100644 tests/qapi-schema/include-relpath.json
create mode 100644 tests/qapi-schema/include-relpath.out
create mode 100644 tests/qapi-schema/include-self-cycle.err
create mode 100644 tests/qapi-schema/include-self-cycle.exit
create mode 100644 tests/qapi-schema/include-self-cycle.json
create mode 100644 tests/qapi-schema/include-self-cycle.out
create mode 100644 tests/qapi-schema/include-simple-sub.json
create mode 100644 tests/qapi-schema/include-simple.err
create mode 100644 tests/qapi-schema/include-simple.exit
create mode 100644 tests/qapi-schema/include-simple.json
create mode 100644 tests/qapi-schema/include-simple.out
create mode 100644 tests/qapi-schema/include/relpath.json
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 63b03cf..051d109 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -40,6 +40,17 @@ enumeration types and union types.
Generally speaking, types definitions should always use CamelCase for the type
names. Command names should be all lower case with words separated by a hyphen.
+
+=== Includes ===
+
+The QAPI schema definitions can be modularized using the 'include' directive:
+
+ { 'include': 'path/to/file.json'}
+
+The directive is evaluated recursively, and include paths are relative to the
+file using the directive.
+
+
=== Complex types ===
A complex type is a dictionary containing a single key whose value is a
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 07a7a82..78050ef 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -11,6 +11,8 @@
# This work is licensed under the terms of the GNU GPL, version 2.
# See the COPYING file in the top-level directory.
+import os
+import re
from ordereddict import OrderedDict
import os
import sys
@@ -36,9 +38,17 @@ builtin_type_qtypes = {
'uint64': 'QTYPE_QINT',
}
+def error_path(parent):
+ res = ""
+ while parent:
+ res = ("In file included from %s:%d:\n" % (parent['file'],
+ parent['line'])) + res
+ parent = parent['parent']
+ return res
+
class QAPISchemaError(Exception):
def __init__(self, schema, msg):
- self.fp = schema.fp
+ self.input_file = schema.input_file
self.msg = msg
self.col = 1
self.line = schema.line
@@ -47,23 +57,31 @@ class QAPISchemaError(Exception):
self.col = (self.col + 7) % 8 + 1
else:
self.col += 1
+ self.info = schema.parent_info
def __str__(self):
- return "%s:%s:%s: %s" % (self.fp.name, self.line, self.col, self.msg)
+ return error_path(self.info) + \
+ "%s:%d:%d: %s" % (self.input_file, self.line, self.col, self.msg)
class QAPIExprError(Exception):
def __init__(self, expr_info, msg):
- self.fp = expr_info['fp']
- self.line = expr_info['line']
+ self.info = expr_info
self.msg = msg
def __str__(self):
- return "%s:%s: %s" % (self.fp.name, self.line, self.msg)
+ return error_path(self.info['parent']) + \
+ "%s:%d: %s" % (self.info['file'], self.info['line'], self.msg)
class QAPISchema:
- def __init__(self, fp):
- self.fp = fp
+ def __init__(self, fp, input_relname=None, include_hist=[], parent_info=None):
+ input_fname = os.path.abspath(fp.name)
+ if input_relname is None:
+ input_relname = fp.name
+ self.input_dir = os.path.dirname(input_fname)
+ self.input_file = input_relname
+ self.include_hist = include_hist + [(input_relname, input_fname)]
+ self.parent_info = parent_info
self.src = fp.read()
if self.src == '' or self.src[-1] != '\n':
self.src += '\n'
@@ -74,10 +92,33 @@ class QAPISchema:
self.accept()
while self.tok != None:
- expr_info = {'fp': fp, 'line': self.line}
- expr_elem = {'expr': self.get_expr(False),
- 'info': expr_info}
- self.exprs.append(expr_elem)
+ expr_info = {'file': input_relname, 'line': self.line, 'parent': self.parent_info}
+ expr = self.get_expr(False)
+ if isinstance(expr, dict) and "include" in expr:
+ if len(expr) != 1:
+ raise QAPIExprError(expr_info, "Invalid 'include' directive")
+ include = expr["include"]
+ if not isinstance(include, str):
+ raise QAPIExprError(expr_info,
+ 'Expected a file name (string), got: %s'
+ % include)
+ include_path = os.path.join(self.input_dir, include)
+ if any(include_path == elem[1]
+ for elem in self.include_hist):
+ raise QAPIExprError(expr_info, "Inclusion loop for %s"
+ % include)
+ try:
+ fobj = open(include_path, 'r')
+ except IOError as e:
+ raise QAPIExprError(expr_info,
+ '%s: %s' % (e.strerror, include))
+ exprs_include = QAPISchema(fobj, include,
+ self.include_hist, expr_info)
+ self.exprs.extend(exprs_include.exprs)
+ else:
+ expr_elem = {'expr': expr,
+ 'info': expr_info}
+ self.exprs.append(expr_elem)
def accept(self):
while True:
@@ -267,7 +308,7 @@ def check_exprs(schema):
def parse_schema(input_file):
try:
schema = QAPISchema(open(input_file, "r"))
- except QAPISchemaError, e:
+ except (QAPISchemaError, QAPIExprError), e:
print >>sys.stderr, e
exit(1)
diff --git a/tests/Makefile b/tests/Makefile
index 3aa9b0b..409bd24 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -164,7 +164,10 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
duplicate-key.json union-invalid-base.json flat-union-no-base.json \
flat-union-invalid-discriminator.json \
flat-union-invalid-branch-key.json flat-union-reverse-define.json \
- flat-union-string-discriminator.json)
+ flat-union-string-discriminator.json \
+ include-simple.json include-relpath.json include-format-err.json \
+ include-non-file.json include-no-file.json include-before-err.json \
+ include-nested-err.json include-self-cycle.json include-cycle.json)
GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h tests/test-qmp-commands.h
diff --git a/tests/qapi-schema/include-before-err.err b/tests/qapi-schema/include-before-err.err
new file mode 100644
index 0000000..eaf632a
--- /dev/null
+++ b/tests/qapi-schema/include-before-err.err
@@ -0,0 +1 @@
+tests/qapi-schema/include-after-err.json:2:13: Expected ":"
diff --git a/tests/qapi-schema/include-before-err.exit b/tests/qapi-schema/include-before-err.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-before-err.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-before-err.json b/tests/qapi-schema/include-before-err.json
new file mode 100644
index 0000000..afb6cb6
--- /dev/null
+++ b/tests/qapi-schema/include-before-err.json
@@ -0,0 +1,2 @@
+{ 'include': 'include-simple-sub.json' }
+{ 'command' 'missing-colon' }
diff --git a/tests/qapi-schema/include-before-err.out b/tests/qapi-schema/include-before-err.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-cycle-b.json b/tests/qapi-schema/include-cycle-b.json
new file mode 100644
index 0000000..4fa985d
--- /dev/null
+++ b/tests/qapi-schema/include-cycle-b.json
@@ -0,0 +1 @@
+{ 'include': 'include-cycle-c.json' }
diff --git a/tests/qapi-schema/include-cycle-c.json b/tests/qapi-schema/include-cycle-c.json
new file mode 100644
index 0000000..d12b592
--- /dev/null
+++ b/tests/qapi-schema/include-cycle-c.json
@@ -0,0 +1 @@
+{ 'include': 'include-cycle.json' }
diff --git a/tests/qapi-schema/include-cycle.err b/tests/qapi-schema/include-cycle.err
new file mode 100644
index 0000000..602cf62
--- /dev/null
+++ b/tests/qapi-schema/include-cycle.err
@@ -0,0 +1,3 @@
+In file included from tests/qapi-schema/include-cycle.json:1:
+In file included from include-cycle-b.json:1:
+include-cycle-c.json:1: Inclusion loop for include-cycle.json
diff --git a/tests/qapi-schema/include-cycle.exit b/tests/qapi-schema/include-cycle.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-cycle.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-cycle.json b/tests/qapi-schema/include-cycle.json
new file mode 100644
index 0000000..6fcf1eb
--- /dev/null
+++ b/tests/qapi-schema/include-cycle.json
@@ -0,0 +1 @@
+{ 'include': 'include-cycle-b.json' }
diff --git a/tests/qapi-schema/include-cycle.out b/tests/qapi-schema/include-cycle.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-format-err.err b/tests/qapi-schema/include-format-err.err
new file mode 100644
index 0000000..721ff4e
--- /dev/null
+++ b/tests/qapi-schema/include-format-err.err
@@ -0,0 +1 @@
+tests/qapi-schema/include-format-err.json:1: Invalid 'include' directive
diff --git a/tests/qapi-schema/include-format-err.exit b/tests/qapi-schema/include-format-err.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-format-err.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-format-err.json b/tests/qapi-schema/include-format-err.json
new file mode 100644
index 0000000..44980f0
--- /dev/null
+++ b/tests/qapi-schema/include-format-err.json
@@ -0,0 +1,2 @@
+{ 'include': 'include-simple-sub.json',
+ 'foo': 'bar' }
diff --git a/tests/qapi-schema/include-format-err.out b/tests/qapi-schema/include-format-err.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-nested-err.err b/tests/qapi-schema/include-nested-err.err
new file mode 100644
index 0000000..1dacbda
--- /dev/null
+++ b/tests/qapi-schema/include-nested-err.err
@@ -0,0 +1,2 @@
+In file included from tests/qapi-schema/include-nested-err.json:1:
+missing-colon.json:1:10: Expected ":"
diff --git a/tests/qapi-schema/include-nested-err.exit b/tests/qapi-schema/include-nested-err.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-nested-err.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-nested-err.json b/tests/qapi-schema/include-nested-err.json
new file mode 100644
index 0000000..5631e56
--- /dev/null
+++ b/tests/qapi-schema/include-nested-err.json
@@ -0,0 +1 @@
+{ 'include': 'missing-colon.json' }
diff --git a/tests/qapi-schema/include-nested-err.out b/tests/qapi-schema/include-nested-err.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-no-file.err b/tests/qapi-schema/include-no-file.err
new file mode 100644
index 0000000..d5b9b22
--- /dev/null
+++ b/tests/qapi-schema/include-no-file.err
@@ -0,0 +1 @@
+tests/qapi-schema/include-no-file.json:1: No such file or directory: include-no-file-sub.json
diff --git a/tests/qapi-schema/include-no-file.exit b/tests/qapi-schema/include-no-file.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-no-file.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-no-file.json b/tests/qapi-schema/include-no-file.json
new file mode 100644
index 0000000..9249ebd
--- /dev/null
+++ b/tests/qapi-schema/include-no-file.json
@@ -0,0 +1 @@
+{ 'include': 'include-no-file-sub.json' }
diff --git a/tests/qapi-schema/include-no-file.out b/tests/qapi-schema/include-no-file.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-non-file.err b/tests/qapi-schema/include-non-file.err
new file mode 100644
index 0000000..9658c78
--- /dev/null
+++ b/tests/qapi-schema/include-non-file.err
@@ -0,0 +1 @@
+tests/qapi-schema/include-non-file.json:1: Expected a file name (string), got: ['foo', 'bar']
diff --git a/tests/qapi-schema/include-non-file.exit b/tests/qapi-schema/include-non-file.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-non-file.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-non-file.json b/tests/qapi-schema/include-non-file.json
new file mode 100644
index 0000000..cd43c3f
--- /dev/null
+++ b/tests/qapi-schema/include-non-file.json
@@ -0,0 +1 @@
+{ 'include': [ 'foo', 'bar' ] }
diff --git a/tests/qapi-schema/include-non-file.out b/tests/qapi-schema/include-non-file.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-relpath-sub.json b/tests/qapi-schema/include-relpath-sub.json
new file mode 100644
index 0000000..4bd4af4
--- /dev/null
+++ b/tests/qapi-schema/include-relpath-sub.json
@@ -0,0 +1,2 @@
+{ 'enum': 'Status',
+ 'data': [ 'good', 'bad', 'ugly' ] }
diff --git a/tests/qapi-schema/include-relpath.err b/tests/qapi-schema/include-relpath.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-relpath.exit b/tests/qapi-schema/include-relpath.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/include-relpath.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/include-relpath.json b/tests/qapi-schema/include-relpath.json
new file mode 100644
index 0000000..05018f3
--- /dev/null
+++ b/tests/qapi-schema/include-relpath.json
@@ -0,0 +1 @@
+{ 'include': 'include/relpath.json' }
diff --git a/tests/qapi-schema/include-relpath.out b/tests/qapi-schema/include-relpath.out
new file mode 100644
index 0000000..4ce3dcf
--- /dev/null
+++ b/tests/qapi-schema/include-relpath.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
+[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
+[]
diff --git a/tests/qapi-schema/include-self-cycle.err b/tests/qapi-schema/include-self-cycle.err
new file mode 100644
index 0000000..981742a
--- /dev/null
+++ b/tests/qapi-schema/include-self-cycle.err
@@ -0,0 +1 @@
+tests/qapi-schema/include-self-cycle.json:1: Inclusion loop for include-self-cycle.json
diff --git a/tests/qapi-schema/include-self-cycle.exit b/tests/qapi-schema/include-self-cycle.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-self-cycle.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-self-cycle.json b/tests/qapi-schema/include-self-cycle.json
new file mode 100644
index 0000000..55fb1b5
--- /dev/null
+++ b/tests/qapi-schema/include-self-cycle.json
@@ -0,0 +1 @@
+{ 'include': 'include-self-cycle.json' }
diff --git a/tests/qapi-schema/include-self-cycle.out b/tests/qapi-schema/include-self-cycle.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-simple-sub.json b/tests/qapi-schema/include-simple-sub.json
new file mode 100644
index 0000000..4bd4af4
--- /dev/null
+++ b/tests/qapi-schema/include-simple-sub.json
@@ -0,0 +1,2 @@
+{ 'enum': 'Status',
+ 'data': [ 'good', 'bad', 'ugly' ] }
diff --git a/tests/qapi-schema/include-simple.err b/tests/qapi-schema/include-simple.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-simple.exit b/tests/qapi-schema/include-simple.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/include-simple.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/include-simple.json b/tests/qapi-schema/include-simple.json
new file mode 100644
index 0000000..1dd391a
--- /dev/null
+++ b/tests/qapi-schema/include-simple.json
@@ -0,0 +1 @@
+{ 'include': 'include-simple-sub.json' }
diff --git a/tests/qapi-schema/include-simple.out b/tests/qapi-schema/include-simple.out
new file mode 100644
index 0000000..4ce3dcf
--- /dev/null
+++ b/tests/qapi-schema/include-simple.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
+[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
+[]
diff --git a/tests/qapi-schema/include/relpath.json b/tests/qapi-schema/include/relpath.json
new file mode 100644
index 0000000..45dee24
--- /dev/null
+++ b/tests/qapi-schema/include/relpath.json
@@ -0,0 +1 @@
+{ 'include': '../include-relpath-sub.json' }
^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-02 13:52 [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files Lluís Vilanova
` (3 preceding siblings ...)
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 4/4] qapi: Add a primitive to include other files from a QAPI schema file Lluís Vilanova
@ 2014-05-05 18:21 ` Luiz Capitulino
2014-05-07 16:36 ` Luiz Capitulino
2014-05-06 13:07 ` Benoît Canet
5 siblings, 1 reply; 26+ messages in thread
From: Luiz Capitulino @ 2014-05-05 18:21 UTC (permalink / raw)
To: Lluís Vilanova; +Cc: Benoît Canet, qemu-devel, Markus Armbruster
On Fri, 2 May 2014 15:52:19 +0200
Lluís Vilanova <vilanova@ac.upc.edu> wrote:
> Adds an include primitive to the syntax of QAPI schema files, allowing these to
> be modularized into multiple per-topic files in the future.
>
> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
Applied to the qmp branch, thanks.
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 4/4] qapi: Add a primitive to include other files from a QAPI schema file
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 4/4] qapi: Add a primitive to include other files from a QAPI schema file Lluís Vilanova
@ 2014-05-05 20:31 ` Luiz Capitulino
2014-05-05 21:15 ` Lluís Vilanova
2014-05-07 18:46 ` Lluís Vilanova
1 sibling, 1 reply; 26+ messages in thread
From: Luiz Capitulino @ 2014-05-05 20:31 UTC (permalink / raw)
To: Lluís Vilanova; +Cc: Benoît Canet, qemu-devel, Markus Armbruster
On Fri, 2 May 2014 15:52:41 +0200
Lluís Vilanova <vilanova@ac.upc.edu> wrote:
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -11,6 +11,8 @@
> # This work is licensed under the terms of the GNU GPL, version 2.
> # See the COPYING file in the top-level directory.
>
> +import os
> +import re
> from ordereddict import OrderedDict
> import os
> import sys
> @@ -36,9 +38,17 @@ builtin_type_qtypes = {
> 'uint64': 'QTYPE_QINT',
> }
You're importing os twice in this series. I saw this while applying an
unrelated patch on top. I'll drop the import added by this patch, please
reply ACKing or NACKing.
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 4/4] qapi: Add a primitive to include other files from a QAPI schema file
2014-05-05 20:31 ` Luiz Capitulino
@ 2014-05-05 21:15 ` Lluís Vilanova
0 siblings, 0 replies; 26+ messages in thread
From: Lluís Vilanova @ 2014-05-05 21:15 UTC (permalink / raw)
To: Luiz Capitulino; +Cc: Benoît Canet, qemu-devel, Markus Armbruster
Luiz Capitulino writes:
> On Fri, 2 May 2014 15:52:41 +0200
> Lluís Vilanova <vilanova@ac.upc.edu> wrote:
>> --- a/scripts/qapi.py
>> +++ b/scripts/qapi.py
>> @@ -11,6 +11,8 @@
>> # This work is licensed under the terms of the GNU GPL, version 2.
>> # See the COPYING file in the top-level directory.
>>
>> +import os
>> +import re
>> from ordereddict import OrderedDict
>> import os
>> import sys
>> @@ -36,9 +38,17 @@ builtin_type_qtypes = {
>> 'uint64': 'QTYPE_QINT',
>> }
> You're importing os twice in this series. I saw this while applying an
> unrelated patch on top. I'll drop the import added by this patch, please
> reply ACKing or NACKing.
Ok.
Sorry for the silly mistake.
Lluis
--
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-02 13:52 [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files Lluís Vilanova
` (4 preceding siblings ...)
2014-05-05 18:21 ` [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files Luiz Capitulino
@ 2014-05-06 13:07 ` Benoît Canet
2014-05-06 13:27 ` Luiz Capitulino
2014-05-06 15:02 ` Eric Blake
5 siblings, 2 replies; 26+ messages in thread
From: Benoît Canet @ 2014-05-06 13:07 UTC (permalink / raw)
To: Lluís Vilanova
Cc: Benoît Canet, Markus Armbruster, qemu-devel, Luiz Capitulino
The Friday 02 May 2014 à 15:52:19 (+0200), Lluís Vilanova wrote :
Hello,
I am trying to use this series to modularise the block API.
Here are my finding.
I tried to make a qmp/block.json including VM state related API.
block.json include a qmp/block-core.json containing only true block stuff.
When generating and compiling block-core.json to link it with qemu-nbd
I saw that some of the block stuff needed ErrorClass so I went the route
of creating a qmp/common.json containing ErrorClass.
common.json being included in block-core.json and in qapi-schema.json it
quickly lead some code being generated in double and the compilation to choke.
What do you think would be the best solution to fix this ?
(Fix the generator ? Make include ignore second inclusion of the same file ?)
Best regards
Benoît
> Adds an include primitive to the syntax of QAPI schema files, allowing these to
> be modularized into multiple per-topic files in the future.
>
> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
> ---
>
> Changes in v12:
>
> * Fix "Reviewed-by" lines.
>
> Changes in v11:
>
> * Rename "tests/include-after-err*" to "tests/include-before-err*" (Markus
> Armbruster).
> * Do not use too many line breaks in "test/Makefile" (Eric Blake).
>
> Changes in v10:
>
> * Various argument/variable name changes (Markus Armbruster).
> * Fix path replacement command in error files when executing tests (Markus
> Armbruster).
> * Catch IOError and translate it to QAPIExprError (Markus Armbruster).
>
> Changes in v9:
>
> * Do not catch unknown exceptions in "tests/qapi-schema/test-qapi.py".
> * Show primitive syntax in commit message.
> * Do not pre-initialize 'input_file' in "scripts/qapi-commands.py".
> * Use a one-liner for test error message checking.
>
> Changes in v8:
>
> * Do not show absolute paths in error messages.
>
> Changes in v7:
>
> * Add tests for relative path inclusion.
> * Print inclusion path on all errors.
> * Add test to ensure errors after an include have the correct context.
> * Squash "qapi.py" changes and tests into a single patch.
> * Add test for 'include' directive format.
> * s/file path/file name/.
> * s/included/include/.
> * Move "-i" to the end of the command.
> * Fix GNUism when using sed.
>
> Changes in v6:
>
> * Split changes for long-line breaking in makefiles.
> * Put documentation on a separate section; reference recursiveness.
> * Check (and test) for non-string include arguments (tanks to Benoît Canet).
>
> Changes in v5:
>
> * Rebase on b3706fa.
> * Remove 'error_base' argument in 'parse_schema'; fix test checks instead.
> * Implement include directive using JSON syntax.
>
> Changes in v4:
>
> * Rebase on 3e890c7.
> * Minor cosmetic changes.
> * Fix recording of included files in case of a cycle error.
> * Add a more complex include cycle test.
>
>
> Changes in v3:
>
> * Fix documentation examples regarding how the input file is passed to the
> scripts.
> * Add documentation for the 'include' directive.
> * Detect inclusion loops.
> * Fix "tests/qapi-schema/test-qapi.py" and "tests/Makefile" to use an explicit
> input file when running tests.
> * Fix QAPI tests to cope with an explicit input file.
> * Add tests for the "include" directive.
>
>
> Changes in v2:
>
> * Change the scripts to use an explicit input file instead of standard input.
> * Fix "tests/Makefile" to use the new argument.
> * Get the input directory for the "include" directive from the input file
> dirname.
>
>
> Lluís Vilanova (4):
> qapi: [trivial] Break long command lines
> qapi: [trivial] Do not catch unknown exceptions in "test-qapi.py"
> qapi: Use an explicit input file
> qapi: Add a primitive to include other files from a QAPI schema file
>
>
> Makefile | 24 +++++--
> docs/qapi-code-gen.txt | 15 ++++
> scripts/qapi-commands.py | 9 ++-
> scripts/qapi-types.py | 9 ++-
> scripts/qapi-visit.py | 9 ++-
> scripts/qapi.py | 70 ++++++++++++++++----
> tests/Makefile | 26 ++++++-
> tests/qapi-schema/duplicate-key.err | 2 -
> .../qapi-schema/flat-union-invalid-branch-key.err | 2 -
> .../flat-union-invalid-discriminator.err | 2 -
> tests/qapi-schema/flat-union-no-base.err | 2 -
> .../flat-union-string-discriminator.err | 2 -
> tests/qapi-schema/funny-char.err | 2 -
> tests/qapi-schema/include-before-err.err | 1
> tests/qapi-schema/include-before-err.exit | 1
> tests/qapi-schema/include-before-err.json | 2 +
> tests/qapi-schema/include-before-err.out | 0
> tests/qapi-schema/include-cycle-b.json | 1
> tests/qapi-schema/include-cycle-c.json | 1
> tests/qapi-schema/include-cycle.err | 3 +
> tests/qapi-schema/include-cycle.exit | 1
> tests/qapi-schema/include-cycle.json | 1
> tests/qapi-schema/include-cycle.out | 0
> tests/qapi-schema/include-format-err.err | 1
> tests/qapi-schema/include-format-err.exit | 1
> tests/qapi-schema/include-format-err.json | 2 +
> tests/qapi-schema/include-format-err.out | 0
> tests/qapi-schema/include-nested-err.err | 2 +
> tests/qapi-schema/include-nested-err.exit | 1
> tests/qapi-schema/include-nested-err.json | 1
> tests/qapi-schema/include-nested-err.out | 0
> tests/qapi-schema/include-no-file.err | 1
> tests/qapi-schema/include-no-file.exit | 1
> tests/qapi-schema/include-no-file.json | 1
> tests/qapi-schema/include-no-file.out | 0
> tests/qapi-schema/include-non-file.err | 1
> tests/qapi-schema/include-non-file.exit | 1
> tests/qapi-schema/include-non-file.json | 1
> tests/qapi-schema/include-non-file.out | 0
> tests/qapi-schema/include-relpath-sub.json | 2 +
> tests/qapi-schema/include-relpath.err | 0
> tests/qapi-schema/include-relpath.exit | 1
> tests/qapi-schema/include-relpath.json | 1
> tests/qapi-schema/include-relpath.out | 3 +
> tests/qapi-schema/include-self-cycle.err | 1
> tests/qapi-schema/include-self-cycle.exit | 1
> tests/qapi-schema/include-self-cycle.json | 1
> tests/qapi-schema/include-self-cycle.out | 0
> tests/qapi-schema/include-simple-sub.json | 2 +
> tests/qapi-schema/include-simple.err | 0
> tests/qapi-schema/include-simple.exit | 1
> tests/qapi-schema/include-simple.json | 1
> tests/qapi-schema/include-simple.out | 3 +
> tests/qapi-schema/include/relpath.json | 1
> tests/qapi-schema/missing-colon.err | 2 -
> tests/qapi-schema/missing-comma-list.err | 2 -
> tests/qapi-schema/missing-comma-object.err | 2 -
> tests/qapi-schema/non-objects.err | 2 -
> tests/qapi-schema/quoted-structural-chars.err | 2 -
> tests/qapi-schema/test-qapi.py | 6 +-
> tests/qapi-schema/trailing-comma-list.err | 2 -
> tests/qapi-schema/trailing-comma-object.err | 2 -
> tests/qapi-schema/unclosed-list.err | 2 -
> tests/qapi-schema/unclosed-object.err | 2 -
> tests/qapi-schema/unclosed-string.err | 2 -
> tests/qapi-schema/union-invalid-base.err | 2 -
> 66 files changed, 187 insertions(+), 58 deletions(-)
> create mode 100644 tests/qapi-schema/include-before-err.err
> create mode 100644 tests/qapi-schema/include-before-err.exit
> create mode 100644 tests/qapi-schema/include-before-err.json
> create mode 100644 tests/qapi-schema/include-before-err.out
> create mode 100644 tests/qapi-schema/include-cycle-b.json
> create mode 100644 tests/qapi-schema/include-cycle-c.json
> create mode 100644 tests/qapi-schema/include-cycle.err
> create mode 100644 tests/qapi-schema/include-cycle.exit
> create mode 100644 tests/qapi-schema/include-cycle.json
> create mode 100644 tests/qapi-schema/include-cycle.out
> create mode 100644 tests/qapi-schema/include-format-err.err
> create mode 100644 tests/qapi-schema/include-format-err.exit
> create mode 100644 tests/qapi-schema/include-format-err.json
> create mode 100644 tests/qapi-schema/include-format-err.out
> create mode 100644 tests/qapi-schema/include-nested-err.err
> create mode 100644 tests/qapi-schema/include-nested-err.exit
> create mode 100644 tests/qapi-schema/include-nested-err.json
> create mode 100644 tests/qapi-schema/include-nested-err.out
> create mode 100644 tests/qapi-schema/include-no-file.err
> create mode 100644 tests/qapi-schema/include-no-file.exit
> create mode 100644 tests/qapi-schema/include-no-file.json
> create mode 100644 tests/qapi-schema/include-no-file.out
> create mode 100644 tests/qapi-schema/include-non-file.err
> create mode 100644 tests/qapi-schema/include-non-file.exit
> create mode 100644 tests/qapi-schema/include-non-file.json
> create mode 100644 tests/qapi-schema/include-non-file.out
> create mode 100644 tests/qapi-schema/include-relpath-sub.json
> create mode 100644 tests/qapi-schema/include-relpath.err
> create mode 100644 tests/qapi-schema/include-relpath.exit
> create mode 100644 tests/qapi-schema/include-relpath.json
> create mode 100644 tests/qapi-schema/include-relpath.out
> create mode 100644 tests/qapi-schema/include-self-cycle.err
> create mode 100644 tests/qapi-schema/include-self-cycle.exit
> create mode 100644 tests/qapi-schema/include-self-cycle.json
> create mode 100644 tests/qapi-schema/include-self-cycle.out
> create mode 100644 tests/qapi-schema/include-simple-sub.json
> create mode 100644 tests/qapi-schema/include-simple.err
> create mode 100644 tests/qapi-schema/include-simple.exit
> create mode 100644 tests/qapi-schema/include-simple.json
> create mode 100644 tests/qapi-schema/include-simple.out
> create mode 100644 tests/qapi-schema/include/relpath.json
>
>
> To: qemu-devel@nongnu.org
> Cc: Luiz Capitulino <lcapitulino@redhat.com>
> Cc: Markus Armbruster <armbru@redhat.com>
> Cc: Eric Blake <eblake@redhat.com>
> Cc: Benoît Canet <benoit.canet@irqsave.net>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-06 13:07 ` Benoît Canet
@ 2014-05-06 13:27 ` Luiz Capitulino
2014-05-06 14:05 ` Benoît Canet
2014-05-06 14:55 ` Eric Blake
2014-05-06 15:02 ` Eric Blake
1 sibling, 2 replies; 26+ messages in thread
From: Luiz Capitulino @ 2014-05-06 13:27 UTC (permalink / raw)
To: Benoît Canet; +Cc: Markus Armbruster, Lluís Vilanova, qemu-devel
On Tue, 6 May 2014 15:07:40 +0200
Benoît Canet <benoit.canet@irqsave.net> wrote:
> I am trying to use this series to modularise the block API.
>
> Here are my finding.
>
> I tried to make a qmp/block.json including VM state related API.
> block.json include a qmp/block-core.json containing only true block stuff.
>
> When generating and compiling block-core.json to link it with qemu-nbd
> I saw that some of the block stuff needed ErrorClass so I went the route
> of creating a qmp/common.json containing ErrorClass.
>
> common.json being included in block-core.json and in qapi-schema.json it
> quickly lead some code being generated in double and the compilation to choke.
>
> What do you think would be the best solution to fix this ?
> (Fix the generator ? Make include ignore second inclusion of the same file ?)
Make qapi-schema.json a sort of master file and include everything?
Eventually, we might want to have if/defs and whatnot. But having a master
file seems a reasonable first step to me. I actually thought this was the
intention. Unless I got it wrong, of course.
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-06 13:27 ` Luiz Capitulino
@ 2014-05-06 14:05 ` Benoît Canet
2014-05-06 14:55 ` Eric Blake
1 sibling, 0 replies; 26+ messages in thread
From: Benoît Canet @ 2014-05-06 14:05 UTC (permalink / raw)
To: Luiz Capitulino
Cc: Benoît Canet, qemu-devel, Markus Armbruster,
Lluís Vilanova
The Tuesday 06 May 2014 à 09:27:25 (-0400), Luiz Capitulino wrote :
> On Tue, 6 May 2014 15:07:40 +0200
> Benoît Canet <benoit.canet@irqsave.net> wrote:
>
> > I am trying to use this series to modularise the block API.
> >
> > Here are my finding.
> >
> > I tried to make a qmp/block.json including VM state related API.
> > block.json include a qmp/block-core.json containing only true block stuff.
> >
> > When generating and compiling block-core.json to link it with qemu-nbd
> > I saw that some of the block stuff needed ErrorClass so I went the route
> > of creating a qmp/common.json containing ErrorClass.
> >
> > common.json being included in block-core.json and in qapi-schema.json it
> > quickly lead some code being generated in double and the compilation to choke.
> >
> > What do you think would be the best solution to fix this ?
> > (Fix the generator ? Make include ignore second inclusion of the same file ?)
>
> Make qapi-schema.json a sort of master file and include everything?
My problem is that I need something smaller than qapi-schema.json to link against
qemu-nbd.
>
> Eventually, we might want to have if/defs and whatnot. But having a master
> file seems a reasonable first step to me. I actually thought this was the
> intention. Unless I got it wrong, of course.
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-06 13:27 ` Luiz Capitulino
2014-05-06 14:05 ` Benoît Canet
@ 2014-05-06 14:55 ` Eric Blake
2014-05-06 16:55 ` Luiz Capitulino
2014-05-07 19:44 ` Markus Armbruster
1 sibling, 2 replies; 26+ messages in thread
From: Eric Blake @ 2014-05-06 14:55 UTC (permalink / raw)
To: Luiz Capitulino, Benoît Canet
Cc: Markus Armbruster, Lluís Vilanova, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 2341 bytes --]
On 05/06/2014 07:27 AM, Luiz Capitulino wrote:
> On Tue, 6 May 2014 15:07:40 +0200
> Benoît Canet <benoit.canet@irqsave.net> wrote:
>
>> I am trying to use this series to modularise the block API.
>>
>> Here are my finding.
>>
>> I tried to make a qmp/block.json including VM state related API.
>> block.json include a qmp/block-core.json containing only true block stuff.
>>
>> When generating and compiling block-core.json to link it with qemu-nbd
>> I saw that some of the block stuff needed ErrorClass so I went the route
>> of creating a qmp/common.json containing ErrorClass.
>>
>> common.json being included in block-core.json and in qapi-schema.json it
>> quickly lead some code being generated in double and the compilation to choke.
>>
>> What do you think would be the best solution to fix this ?
>> (Fix the generator ? Make include ignore second inclusion of the same file ?)
>
> Make qapi-schema.json a sort of master file and include everything?
Won't cut it, if we want to support a subset of files in other contexts.
You either have to do:
qemu-schema.json:
include common
include block
include other
qemu-block:
include common
include block
where block does not work in isolation, and has to be wrapped; but now
we have two different wrappers depending on the two different clients
that want a different subset. Or you do:
qemu-schema.json:
include common
include block
include other
block.json:
include common
now block.json is standalone, and qemu-schema.json ends up including
common through two different paths.
>
> Eventually, we might want to have if/defs and whatnot. But having a master
> file seems a reasonable first step to me. I actually thought this was the
> intention. Unless I got it wrong, of course.
Ifdefs may be a bit much. If we add them, then we can worry about
explicit include guards, the same as the C preprocessor. But for now,
I'd be perfectly fine with a followup patch that includes a file's
contents exactly once, no matter how many times it is included (that is,
act as if include guards were implicitly present, since we lack
conditionals, so include files are currently idempotent).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-06 13:07 ` Benoît Canet
2014-05-06 13:27 ` Luiz Capitulino
@ 2014-05-06 15:02 ` Eric Blake
1 sibling, 0 replies; 26+ messages in thread
From: Eric Blake @ 2014-05-06 15:02 UTC (permalink / raw)
To: Benoît Canet, Lluís Vilanova
Cc: Markus Armbruster, qemu-devel, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 691 bytes --]
On 05/06/2014 07:07 AM, Benoît Canet wrote:
>
> common.json being included in block-core.json and in qapi-schema.json it
> quickly lead some code being generated in double and the compilation to choke.
>
> What do you think would be the best solution to fix this ?
> (Fix the generator ? Make include ignore second inclusion of the same file ?)
I'd argue for the latter - update the parser and the documentation to
state that if a file is included twice, the second inclusion does
nothing. Obviously, the testsuite needs enhancement to test the
solution, as well.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-06 14:55 ` Eric Blake
@ 2014-05-06 16:55 ` Luiz Capitulino
2014-05-06 17:11 ` Eric Blake
2014-05-07 19:44 ` Markus Armbruster
1 sibling, 1 reply; 26+ messages in thread
From: Luiz Capitulino @ 2014-05-06 16:55 UTC (permalink / raw)
To: Eric Blake
Cc: Benoît Canet, Markus Armbruster, Lluís Vilanova,
qemu-devel
On Tue, 06 May 2014 08:55:52 -0600
Eric Blake <eblake@redhat.com> wrote:
> > Eventually, we might want to have if/defs and whatnot. But having a master
> > file seems a reasonable first step to me. I actually thought this was the
> > intention. Unless I got it wrong, of course.
>
> Ifdefs may be a bit much. If we add them, then we can worry about
> explicit include guards, the same as the C preprocessor. But for now,
> I'd be perfectly fine with a followup patch that includes a file's
> contents exactly once, no matter how many times it is included (that is,
> act as if include guards were implicitly present, since we lack
> conditionals, so include files are currently idempotent).
OK. Does it make sense to merge the current series without that
modification?
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-06 16:55 ` Luiz Capitulino
@ 2014-05-06 17:11 ` Eric Blake
0 siblings, 0 replies; 26+ messages in thread
From: Eric Blake @ 2014-05-06 17:11 UTC (permalink / raw)
To: Luiz Capitulino
Cc: Benoît Canet, Markus Armbruster, Lluís Vilanova,
qemu-devel
[-- Attachment #1: Type: text/plain, Size: 1171 bytes --]
On 05/06/2014 10:55 AM, Luiz Capitulino wrote:
> On Tue, 06 May 2014 08:55:52 -0600
> Eric Blake <eblake@redhat.com> wrote:
>
>>> Eventually, we might want to have if/defs and whatnot. But having a master
>>> file seems a reasonable first step to me. I actually thought this was the
>>> intention. Unless I got it wrong, of course.
>>
>> Ifdefs may be a bit much. If we add them, then we can worry about
>> explicit include guards, the same as the C preprocessor. But for now,
>> I'd be perfectly fine with a followup patch that includes a file's
>> contents exactly once, no matter how many times it is included (that is,
>> act as if include guards were implicitly present, since we lack
>> conditionals, so include files are currently idempotent).
>
> OK. Does it make sense to merge the current series without that
> modification?
Yes. Idempotent inclusion as a followup patch is just fine; and the
current series is still useful for some clients without waiting for
idempotent inclusion to make it useful for even more clients.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-05 18:21 ` [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files Luiz Capitulino
@ 2014-05-07 16:36 ` Luiz Capitulino
2014-05-07 17:07 ` Lluís Vilanova
0 siblings, 1 reply; 26+ messages in thread
From: Luiz Capitulino @ 2014-05-07 16:36 UTC (permalink / raw)
To: Luiz Capitulino
Cc: Benoît Canet, Markus Armbruster, Lluís Vilanova,
qemu-devel
On Mon, 5 May 2014 14:21:26 -0400
Luiz Capitulino <lcapitulino@redhat.com> wrote:
> On Fri, 2 May 2014 15:52:19 +0200
> Lluís Vilanova <vilanova@ac.upc.edu> wrote:
>
> > Adds an include primitive to the syntax of QAPI schema files, allowing these to
> > be modularized into multiple per-topic files in the future.
> >
> > Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
> > Reviewed-by: Eric Blake <eblake@redhat.com>
> > Reviewed-by: Markus Armbruster <armbru@redhat.com>
>
> Applied to the qmp branch, thanks.
make check fails for me:
TEST tests/qapi-schema/include-before-err.out
Files /home/lcapitulino/work/src/upstream/qmp-unstable/tests/qapi-schema/include-before-err.err and - differ
make: *** [check-tests/qapi-schema/include-before-err.json] Error 1
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-07 16:36 ` Luiz Capitulino
@ 2014-05-07 17:07 ` Lluís Vilanova
2014-05-07 17:14 ` Luiz Capitulino
0 siblings, 1 reply; 26+ messages in thread
From: Lluís Vilanova @ 2014-05-07 17:07 UTC (permalink / raw)
To: Luiz Capitulino; +Cc: Benoît Canet, qemu-devel, Markus Armbruster
Luiz Capitulino writes:
> On Mon, 5 May 2014 14:21:26 -0400
> Luiz Capitulino <lcapitulino@redhat.com> wrote:
>> On Fri, 2 May 2014 15:52:19 +0200
>> Lluís Vilanova <vilanova@ac.upc.edu> wrote:
>>
>> > Adds an include primitive to the syntax of QAPI schema files, allowing these to
>> > be modularized into multiple per-topic files in the future.
>> >
>> > Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
>> > Reviewed-by: Eric Blake <eblake@redhat.com>
>> > Reviewed-by: Markus Armbruster <armbru@redhat.com>
>>
>> Applied to the qmp branch, thanks.
> make check fails for me:
> TEST tests/qapi-schema/include-before-err.out
> Files /home/lcapitulino/work/src/upstream/qmp-unstable/tests/qapi-schema/include-before-err.err and - differ
> make: *** [check-tests/qapi-schema/include-before-err.json] Error 1
Sorry, it's my fault. I though I did rerun the tests. The change is:
sed -i -e 's/after/before' tests/qapi-schema/include-before-err.err
Should I resend the series?
Thanks,
Lluis
--
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-07 17:07 ` Lluís Vilanova
@ 2014-05-07 17:14 ` Luiz Capitulino
2014-05-07 17:20 ` Lluís Vilanova
0 siblings, 1 reply; 26+ messages in thread
From: Luiz Capitulino @ 2014-05-07 17:14 UTC (permalink / raw)
To: Lluís Vilanova; +Cc: Benoît Canet, qemu-devel, Markus Armbruster
On Wed, 07 May 2014 19:07:56 +0200
Lluís Vilanova <vilanova@ac.upc.edu> wrote:
> Luiz Capitulino writes:
>
> > On Mon, 5 May 2014 14:21:26 -0400
> > Luiz Capitulino <lcapitulino@redhat.com> wrote:
>
> >> On Fri, 2 May 2014 15:52:19 +0200
> >> Lluís Vilanova <vilanova@ac.upc.edu> wrote:
> >>
> >> > Adds an include primitive to the syntax of QAPI schema files, allowing these to
> >> > be modularized into multiple per-topic files in the future.
> >> >
> >> > Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
> >> > Reviewed-by: Eric Blake <eblake@redhat.com>
> >> > Reviewed-by: Markus Armbruster <armbru@redhat.com>
> >>
> >> Applied to the qmp branch, thanks.
>
> > make check fails for me:
>
> > TEST tests/qapi-schema/include-before-err.out
> > Files /home/lcapitulino/work/src/upstream/qmp-unstable/tests/qapi-schema/include-before-err.err and - differ
> > make: *** [check-tests/qapi-schema/include-before-err.json] Error 1
>
> Sorry, it's my fault. I though I did rerun the tests. The change is:
>
> sed -i -e 's/after/before' tests/qapi-schema/include-before-err.err
>
> Should I resend the series?
It would be nice if you can resend only the affected patch.
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-07 17:14 ` Luiz Capitulino
@ 2014-05-07 17:20 ` Lluís Vilanova
2014-05-07 17:23 ` Luiz Capitulino
0 siblings, 1 reply; 26+ messages in thread
From: Lluís Vilanova @ 2014-05-07 17:20 UTC (permalink / raw)
To: Luiz Capitulino; +Cc: Benoît Canet, qemu-devel, Markus Armbruster
Luiz Capitulino writes:
> On Wed, 07 May 2014 19:07:56 +0200
> Lluís Vilanova <vilanova@ac.upc.edu> wrote:
>> Luiz Capitulino writes:
>>
>> > On Mon, 5 May 2014 14:21:26 -0400
>> > Luiz Capitulino <lcapitulino@redhat.com> wrote:
>>
>> >> On Fri, 2 May 2014 15:52:19 +0200
>> >> Lluís Vilanova <vilanova@ac.upc.edu> wrote:
>> >>
>> >> > Adds an include primitive to the syntax of QAPI schema files, allowing these to
>> >> > be modularized into multiple per-topic files in the future.
>> >> >
>> >> > Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
>> >> > Reviewed-by: Eric Blake <eblake@redhat.com>
>> >> > Reviewed-by: Markus Armbruster <armbru@redhat.com>
>> >>
>> >> Applied to the qmp branch, thanks.
>>
>> > make check fails for me:
>>
>> > TEST tests/qapi-schema/include-before-err.out
>> > Files /home/lcapitulino/work/src/upstream/qmp-unstable/tests/qapi-schema/include-before-err.err and - differ
>> > make: *** [check-tests/qapi-schema/include-before-err.json] Error 1
>>
>> Sorry, it's my fault. I though I did rerun the tests. The change is:
>>
>> sed -i -e 's/after/before' tests/qapi-schema/include-before-err.err
>>
>> Should I resend the series?
> It would be nice if you can resend only the affected patch.
You mean resend a fixed version of the original patch, or a patch on top of the
series?
Thanks,
Lluis
--
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-07 17:20 ` Lluís Vilanova
@ 2014-05-07 17:23 ` Luiz Capitulino
0 siblings, 0 replies; 26+ messages in thread
From: Luiz Capitulino @ 2014-05-07 17:23 UTC (permalink / raw)
To: Lluís Vilanova; +Cc: Benoît Canet, qemu-devel, Markus Armbruster
On Wed, 07 May 2014 19:20:14 +0200
Lluís Vilanova <vilanova@ac.upc.edu> wrote:
> Luiz Capitulino writes:
>
> > On Wed, 07 May 2014 19:07:56 +0200
> > Lluís Vilanova <vilanova@ac.upc.edu> wrote:
>
> >> Luiz Capitulino writes:
> >>
> >> > On Mon, 5 May 2014 14:21:26 -0400
> >> > Luiz Capitulino <lcapitulino@redhat.com> wrote:
> >>
> >> >> On Fri, 2 May 2014 15:52:19 +0200
> >> >> Lluís Vilanova <vilanova@ac.upc.edu> wrote:
> >> >>
> >> >> > Adds an include primitive to the syntax of QAPI schema files, allowing these to
> >> >> > be modularized into multiple per-topic files in the future.
> >> >> >
> >> >> > Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
> >> >> > Reviewed-by: Eric Blake <eblake@redhat.com>
> >> >> > Reviewed-by: Markus Armbruster <armbru@redhat.com>
> >> >>
> >> >> Applied to the qmp branch, thanks.
> >>
> >> > make check fails for me:
> >>
> >> > TEST tests/qapi-schema/include-before-err.out
> >> > Files /home/lcapitulino/work/src/upstream/qmp-unstable/tests/qapi-schema/include-before-err.err and - differ
> >> > make: *** [check-tests/qapi-schema/include-before-err.json] Error 1
> >>
> >> Sorry, it's my fault. I though I did rerun the tests. The change is:
> >>
> >> sed -i -e 's/after/before' tests/qapi-schema/include-before-err.err
> >>
> >> Should I resend the series?
>
> > It would be nice if you can resend only the affected patch.
>
> You mean resend a fixed version of the original patch, or a patch on top of the
> series?
A fixed version of the original patch. You can do it by replying to the
original patch.
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH v12 4/4] qapi: Add a primitive to include other files from a QAPI schema file
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 4/4] qapi: Add a primitive to include other files from a QAPI schema file Lluís Vilanova
2014-05-05 20:31 ` Luiz Capitulino
@ 2014-05-07 18:46 ` Lluís Vilanova
2014-05-07 18:48 ` Lluís Vilanova
` (2 more replies)
1 sibling, 3 replies; 26+ messages in thread
From: Lluís Vilanova @ 2014-05-07 18:46 UTC (permalink / raw)
To: qemu-devel; +Cc: Benoît Canet, Markus Armbruster, Luiz Capitulino
The primitive uses JSON syntax, and include paths are relative to the file using the directive:
{ 'include': 'path/to/file.json' }
Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
---
docs/qapi-code-gen.txt | 11 +++++
scripts/qapi.py | 64 +++++++++++++++++++++++-----
tests/Makefile | 5 ++
tests/qapi-schema/include-before-err.err | 1
tests/qapi-schema/include-before-err.exit | 1
tests/qapi-schema/include-before-err.json | 2 +
tests/qapi-schema/include-before-err.out | 0
tests/qapi-schema/include-cycle-b.json | 1
tests/qapi-schema/include-cycle-c.json | 1
tests/qapi-schema/include-cycle.err | 3 +
tests/qapi-schema/include-cycle.exit | 1
tests/qapi-schema/include-cycle.json | 1
tests/qapi-schema/include-cycle.out | 0
tests/qapi-schema/include-format-err.err | 1
tests/qapi-schema/include-format-err.exit | 1
tests/qapi-schema/include-format-err.json | 2 +
tests/qapi-schema/include-format-err.out | 0
tests/qapi-schema/include-nested-err.err | 2 +
tests/qapi-schema/include-nested-err.exit | 1
tests/qapi-schema/include-nested-err.json | 1
tests/qapi-schema/include-nested-err.out | 0
tests/qapi-schema/include-no-file.err | 1
tests/qapi-schema/include-no-file.exit | 1
tests/qapi-schema/include-no-file.json | 1
tests/qapi-schema/include-no-file.out | 0
tests/qapi-schema/include-non-file.err | 1
tests/qapi-schema/include-non-file.exit | 1
tests/qapi-schema/include-non-file.json | 1
tests/qapi-schema/include-non-file.out | 0
tests/qapi-schema/include-relpath-sub.json | 2 +
tests/qapi-schema/include-relpath.err | 0
tests/qapi-schema/include-relpath.exit | 1
tests/qapi-schema/include-relpath.json | 1
tests/qapi-schema/include-relpath.out | 3 +
tests/qapi-schema/include-self-cycle.err | 1
tests/qapi-schema/include-self-cycle.exit | 1
tests/qapi-schema/include-self-cycle.json | 1
tests/qapi-schema/include-self-cycle.out | 0
tests/qapi-schema/include-simple-sub.json | 2 +
tests/qapi-schema/include-simple.err | 0
tests/qapi-schema/include-simple.exit | 1
tests/qapi-schema/include-simple.json | 1
tests/qapi-schema/include-simple.out | 3 +
tests/qapi-schema/include/relpath.json | 1
44 files changed, 110 insertions(+), 13 deletions(-)
create mode 100644 tests/qapi-schema/include-before-err.err
create mode 100644 tests/qapi-schema/include-before-err.exit
create mode 100644 tests/qapi-schema/include-before-err.json
create mode 100644 tests/qapi-schema/include-before-err.out
create mode 100644 tests/qapi-schema/include-cycle-b.json
create mode 100644 tests/qapi-schema/include-cycle-c.json
create mode 100644 tests/qapi-schema/include-cycle.err
create mode 100644 tests/qapi-schema/include-cycle.exit
create mode 100644 tests/qapi-schema/include-cycle.json
create mode 100644 tests/qapi-schema/include-cycle.out
create mode 100644 tests/qapi-schema/include-format-err.err
create mode 100644 tests/qapi-schema/include-format-err.exit
create mode 100644 tests/qapi-schema/include-format-err.json
create mode 100644 tests/qapi-schema/include-format-err.out
create mode 100644 tests/qapi-schema/include-nested-err.err
create mode 100644 tests/qapi-schema/include-nested-err.exit
create mode 100644 tests/qapi-schema/include-nested-err.json
create mode 100644 tests/qapi-schema/include-nested-err.out
create mode 100644 tests/qapi-schema/include-no-file.err
create mode 100644 tests/qapi-schema/include-no-file.exit
create mode 100644 tests/qapi-schema/include-no-file.json
create mode 100644 tests/qapi-schema/include-no-file.out
create mode 100644 tests/qapi-schema/include-non-file.err
create mode 100644 tests/qapi-schema/include-non-file.exit
create mode 100644 tests/qapi-schema/include-non-file.json
create mode 100644 tests/qapi-schema/include-non-file.out
create mode 100644 tests/qapi-schema/include-relpath-sub.json
create mode 100644 tests/qapi-schema/include-relpath.err
create mode 100644 tests/qapi-schema/include-relpath.exit
create mode 100644 tests/qapi-schema/include-relpath.json
create mode 100644 tests/qapi-schema/include-relpath.out
create mode 100644 tests/qapi-schema/include-self-cycle.err
create mode 100644 tests/qapi-schema/include-self-cycle.exit
create mode 100644 tests/qapi-schema/include-self-cycle.json
create mode 100644 tests/qapi-schema/include-self-cycle.out
create mode 100644 tests/qapi-schema/include-simple-sub.json
create mode 100644 tests/qapi-schema/include-simple.err
create mode 100644 tests/qapi-schema/include-simple.exit
create mode 100644 tests/qapi-schema/include-simple.json
create mode 100644 tests/qapi-schema/include-simple.out
create mode 100644 tests/qapi-schema/include/relpath.json
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 63b03cf..051d109 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -40,6 +40,17 @@ enumeration types and union types.
Generally speaking, types definitions should always use CamelCase for the type
names. Command names should be all lower case with words separated by a hyphen.
+
+=== Includes ===
+
+The QAPI schema definitions can be modularized using the 'include' directive:
+
+ { 'include': 'path/to/file.json'}
+
+The directive is evaluated recursively, and include paths are relative to the
+file using the directive.
+
+
=== Complex types ===
A complex type is a dictionary containing a single key whose value is a
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 07a7a82..ec806aa 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -11,6 +11,7 @@
# This work is licensed under the terms of the GNU GPL, version 2.
# See the COPYING file in the top-level directory.
+import re
from ordereddict import OrderedDict
import os
import sys
@@ -36,9 +37,17 @@ builtin_type_qtypes = {
'uint64': 'QTYPE_QINT',
}
+def error_path(parent):
+ res = ""
+ while parent:
+ res = ("In file included from %s:%d:\n" % (parent['file'],
+ parent['line'])) + res
+ parent = parent['parent']
+ return res
+
class QAPISchemaError(Exception):
def __init__(self, schema, msg):
- self.fp = schema.fp
+ self.input_file = schema.input_file
self.msg = msg
self.col = 1
self.line = schema.line
@@ -47,23 +56,31 @@ class QAPISchemaError(Exception):
self.col = (self.col + 7) % 8 + 1
else:
self.col += 1
+ self.info = schema.parent_info
def __str__(self):
- return "%s:%s:%s: %s" % (self.fp.name, self.line, self.col, self.msg)
+ return error_path(self.info) + \
+ "%s:%d:%d: %s" % (self.input_file, self.line, self.col, self.msg)
class QAPIExprError(Exception):
def __init__(self, expr_info, msg):
- self.fp = expr_info['fp']
- self.line = expr_info['line']
+ self.info = expr_info
self.msg = msg
def __str__(self):
- return "%s:%s: %s" % (self.fp.name, self.line, self.msg)
+ return error_path(self.info['parent']) + \
+ "%s:%d: %s" % (self.info['file'], self.info['line'], self.msg)
class QAPISchema:
- def __init__(self, fp):
- self.fp = fp
+ def __init__(self, fp, input_relname=None, include_hist=[], parent_info=None):
+ input_fname = os.path.abspath(fp.name)
+ if input_relname is None:
+ input_relname = fp.name
+ self.input_dir = os.path.dirname(input_fname)
+ self.input_file = input_relname
+ self.include_hist = include_hist + [(input_relname, input_fname)]
+ self.parent_info = parent_info
self.src = fp.read()
if self.src == '' or self.src[-1] != '\n':
self.src += '\n'
@@ -74,10 +91,33 @@ class QAPISchema:
self.accept()
while self.tok != None:
- expr_info = {'fp': fp, 'line': self.line}
- expr_elem = {'expr': self.get_expr(False),
- 'info': expr_info}
- self.exprs.append(expr_elem)
+ expr_info = {'file': input_relname, 'line': self.line, 'parent': self.parent_info}
+ expr = self.get_expr(False)
+ if isinstance(expr, dict) and "include" in expr:
+ if len(expr) != 1:
+ raise QAPIExprError(expr_info, "Invalid 'include' directive")
+ include = expr["include"]
+ if not isinstance(include, str):
+ raise QAPIExprError(expr_info,
+ 'Expected a file name (string), got: %s'
+ % include)
+ include_path = os.path.join(self.input_dir, include)
+ if any(include_path == elem[1]
+ for elem in self.include_hist):
+ raise QAPIExprError(expr_info, "Inclusion loop for %s"
+ % include)
+ try:
+ fobj = open(include_path, 'r')
+ except IOError as e:
+ raise QAPIExprError(expr_info,
+ '%s: %s' % (e.strerror, include))
+ exprs_include = QAPISchema(fobj, include,
+ self.include_hist, expr_info)
+ self.exprs.extend(exprs_include.exprs)
+ else:
+ expr_elem = {'expr': expr,
+ 'info': expr_info}
+ self.exprs.append(expr_elem)
def accept(self):
while True:
@@ -267,7 +307,7 @@ def check_exprs(schema):
def parse_schema(input_file):
try:
schema = QAPISchema(open(input_file, "r"))
- except QAPISchemaError, e:
+ except (QAPISchemaError, QAPIExprError), e:
print >>sys.stderr, e
exit(1)
diff --git a/tests/Makefile b/tests/Makefile
index 3aa9b0b..409bd24 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -164,7 +164,10 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
duplicate-key.json union-invalid-base.json flat-union-no-base.json \
flat-union-invalid-discriminator.json \
flat-union-invalid-branch-key.json flat-union-reverse-define.json \
- flat-union-string-discriminator.json)
+ flat-union-string-discriminator.json \
+ include-simple.json include-relpath.json include-format-err.json \
+ include-non-file.json include-no-file.json include-before-err.json \
+ include-nested-err.json include-self-cycle.json include-cycle.json)
GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h tests/test-qmp-commands.h
diff --git a/tests/qapi-schema/include-before-err.err b/tests/qapi-schema/include-before-err.err
new file mode 100644
index 0000000..5565275
--- /dev/null
+++ b/tests/qapi-schema/include-before-err.err
@@ -0,0 +1 @@
+tests/qapi-schema/include-before-err.json:2:13: Expected ":"
diff --git a/tests/qapi-schema/include-before-err.exit b/tests/qapi-schema/include-before-err.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-before-err.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-before-err.json b/tests/qapi-schema/include-before-err.json
new file mode 100644
index 0000000..afb6cb6
--- /dev/null
+++ b/tests/qapi-schema/include-before-err.json
@@ -0,0 +1,2 @@
+{ 'include': 'include-simple-sub.json' }
+{ 'command' 'missing-colon' }
diff --git a/tests/qapi-schema/include-before-err.out b/tests/qapi-schema/include-before-err.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-cycle-b.json b/tests/qapi-schema/include-cycle-b.json
new file mode 100644
index 0000000..4fa985d
--- /dev/null
+++ b/tests/qapi-schema/include-cycle-b.json
@@ -0,0 +1 @@
+{ 'include': 'include-cycle-c.json' }
diff --git a/tests/qapi-schema/include-cycle-c.json b/tests/qapi-schema/include-cycle-c.json
new file mode 100644
index 0000000..d12b592
--- /dev/null
+++ b/tests/qapi-schema/include-cycle-c.json
@@ -0,0 +1 @@
+{ 'include': 'include-cycle.json' }
diff --git a/tests/qapi-schema/include-cycle.err b/tests/qapi-schema/include-cycle.err
new file mode 100644
index 0000000..602cf62
--- /dev/null
+++ b/tests/qapi-schema/include-cycle.err
@@ -0,0 +1,3 @@
+In file included from tests/qapi-schema/include-cycle.json:1:
+In file included from include-cycle-b.json:1:
+include-cycle-c.json:1: Inclusion loop for include-cycle.json
diff --git a/tests/qapi-schema/include-cycle.exit b/tests/qapi-schema/include-cycle.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-cycle.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-cycle.json b/tests/qapi-schema/include-cycle.json
new file mode 100644
index 0000000..6fcf1eb
--- /dev/null
+++ b/tests/qapi-schema/include-cycle.json
@@ -0,0 +1 @@
+{ 'include': 'include-cycle-b.json' }
diff --git a/tests/qapi-schema/include-cycle.out b/tests/qapi-schema/include-cycle.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-format-err.err b/tests/qapi-schema/include-format-err.err
new file mode 100644
index 0000000..721ff4e
--- /dev/null
+++ b/tests/qapi-schema/include-format-err.err
@@ -0,0 +1 @@
+tests/qapi-schema/include-format-err.json:1: Invalid 'include' directive
diff --git a/tests/qapi-schema/include-format-err.exit b/tests/qapi-schema/include-format-err.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-format-err.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-format-err.json b/tests/qapi-schema/include-format-err.json
new file mode 100644
index 0000000..44980f0
--- /dev/null
+++ b/tests/qapi-schema/include-format-err.json
@@ -0,0 +1,2 @@
+{ 'include': 'include-simple-sub.json',
+ 'foo': 'bar' }
diff --git a/tests/qapi-schema/include-format-err.out b/tests/qapi-schema/include-format-err.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-nested-err.err b/tests/qapi-schema/include-nested-err.err
new file mode 100644
index 0000000..1dacbda
--- /dev/null
+++ b/tests/qapi-schema/include-nested-err.err
@@ -0,0 +1,2 @@
+In file included from tests/qapi-schema/include-nested-err.json:1:
+missing-colon.json:1:10: Expected ":"
diff --git a/tests/qapi-schema/include-nested-err.exit b/tests/qapi-schema/include-nested-err.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-nested-err.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-nested-err.json b/tests/qapi-schema/include-nested-err.json
new file mode 100644
index 0000000..5631e56
--- /dev/null
+++ b/tests/qapi-schema/include-nested-err.json
@@ -0,0 +1 @@
+{ 'include': 'missing-colon.json' }
diff --git a/tests/qapi-schema/include-nested-err.out b/tests/qapi-schema/include-nested-err.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-no-file.err b/tests/qapi-schema/include-no-file.err
new file mode 100644
index 0000000..d5b9b22
--- /dev/null
+++ b/tests/qapi-schema/include-no-file.err
@@ -0,0 +1 @@
+tests/qapi-schema/include-no-file.json:1: No such file or directory: include-no-file-sub.json
diff --git a/tests/qapi-schema/include-no-file.exit b/tests/qapi-schema/include-no-file.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-no-file.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-no-file.json b/tests/qapi-schema/include-no-file.json
new file mode 100644
index 0000000..9249ebd
--- /dev/null
+++ b/tests/qapi-schema/include-no-file.json
@@ -0,0 +1 @@
+{ 'include': 'include-no-file-sub.json' }
diff --git a/tests/qapi-schema/include-no-file.out b/tests/qapi-schema/include-no-file.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-non-file.err b/tests/qapi-schema/include-non-file.err
new file mode 100644
index 0000000..9658c78
--- /dev/null
+++ b/tests/qapi-schema/include-non-file.err
@@ -0,0 +1 @@
+tests/qapi-schema/include-non-file.json:1: Expected a file name (string), got: ['foo', 'bar']
diff --git a/tests/qapi-schema/include-non-file.exit b/tests/qapi-schema/include-non-file.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-non-file.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-non-file.json b/tests/qapi-schema/include-non-file.json
new file mode 100644
index 0000000..cd43c3f
--- /dev/null
+++ b/tests/qapi-schema/include-non-file.json
@@ -0,0 +1 @@
+{ 'include': [ 'foo', 'bar' ] }
diff --git a/tests/qapi-schema/include-non-file.out b/tests/qapi-schema/include-non-file.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-relpath-sub.json b/tests/qapi-schema/include-relpath-sub.json
new file mode 100644
index 0000000..4bd4af4
--- /dev/null
+++ b/tests/qapi-schema/include-relpath-sub.json
@@ -0,0 +1,2 @@
+{ 'enum': 'Status',
+ 'data': [ 'good', 'bad', 'ugly' ] }
diff --git a/tests/qapi-schema/include-relpath.err b/tests/qapi-schema/include-relpath.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-relpath.exit b/tests/qapi-schema/include-relpath.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/include-relpath.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/include-relpath.json b/tests/qapi-schema/include-relpath.json
new file mode 100644
index 0000000..05018f3
--- /dev/null
+++ b/tests/qapi-schema/include-relpath.json
@@ -0,0 +1 @@
+{ 'include': 'include/relpath.json' }
diff --git a/tests/qapi-schema/include-relpath.out b/tests/qapi-schema/include-relpath.out
new file mode 100644
index 0000000..4ce3dcf
--- /dev/null
+++ b/tests/qapi-schema/include-relpath.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
+[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
+[]
diff --git a/tests/qapi-schema/include-self-cycle.err b/tests/qapi-schema/include-self-cycle.err
new file mode 100644
index 0000000..981742a
--- /dev/null
+++ b/tests/qapi-schema/include-self-cycle.err
@@ -0,0 +1 @@
+tests/qapi-schema/include-self-cycle.json:1: Inclusion loop for include-self-cycle.json
diff --git a/tests/qapi-schema/include-self-cycle.exit b/tests/qapi-schema/include-self-cycle.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/include-self-cycle.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/include-self-cycle.json b/tests/qapi-schema/include-self-cycle.json
new file mode 100644
index 0000000..55fb1b5
--- /dev/null
+++ b/tests/qapi-schema/include-self-cycle.json
@@ -0,0 +1 @@
+{ 'include': 'include-self-cycle.json' }
diff --git a/tests/qapi-schema/include-self-cycle.out b/tests/qapi-schema/include-self-cycle.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-simple-sub.json b/tests/qapi-schema/include-simple-sub.json
new file mode 100644
index 0000000..4bd4af4
--- /dev/null
+++ b/tests/qapi-schema/include-simple-sub.json
@@ -0,0 +1,2 @@
+{ 'enum': 'Status',
+ 'data': [ 'good', 'bad', 'ugly' ] }
diff --git a/tests/qapi-schema/include-simple.err b/tests/qapi-schema/include-simple.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/include-simple.exit b/tests/qapi-schema/include-simple.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/include-simple.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/include-simple.json b/tests/qapi-schema/include-simple.json
new file mode 100644
index 0000000..1dd391a
--- /dev/null
+++ b/tests/qapi-schema/include-simple.json
@@ -0,0 +1 @@
+{ 'include': 'include-simple-sub.json' }
diff --git a/tests/qapi-schema/include-simple.out b/tests/qapi-schema/include-simple.out
new file mode 100644
index 0000000..4ce3dcf
--- /dev/null
+++ b/tests/qapi-schema/include-simple.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
+[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
+[]
diff --git a/tests/qapi-schema/include/relpath.json b/tests/qapi-schema/include/relpath.json
new file mode 100644
index 0000000..45dee24
--- /dev/null
+++ b/tests/qapi-schema/include/relpath.json
@@ -0,0 +1 @@
+{ 'include': '../include-relpath-sub.json' }
^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 4/4] qapi: Add a primitive to include other files from a QAPI schema file
2014-05-07 18:46 ` Lluís Vilanova
@ 2014-05-07 18:48 ` Lluís Vilanova
2014-05-07 20:44 ` Luiz Capitulino
2014-05-18 11:44 ` Peter Maydell
2 siblings, 0 replies; 26+ messages in thread
From: Lluís Vilanova @ 2014-05-07 18:48 UTC (permalink / raw)
To: qemu-devel; +Cc: Benoît Canet, Markus Armbruster, Luiz Capitulino
This resend fixes a couple things:
* Remove double import of os package
* Incorrect error checking in the "include-before-err" test.
Thanks,
Lluis
Lluís Vilanova writes:
> The primitive uses JSON syntax, and include paths are relative to the file using the directive:
> { 'include': 'path/to/file.json' }
> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
> ---
> docs/qapi-code-gen.txt | 11 +++++
> scripts/qapi.py | 64 +++++++++++++++++++++++-----
> tests/Makefile | 5 ++
> tests/qapi-schema/include-before-err.err | 1
> tests/qapi-schema/include-before-err.exit | 1
> tests/qapi-schema/include-before-err.json | 2 +
> tests/qapi-schema/include-before-err.out | 0
> tests/qapi-schema/include-cycle-b.json | 1
> tests/qapi-schema/include-cycle-c.json | 1
> tests/qapi-schema/include-cycle.err | 3 +
> tests/qapi-schema/include-cycle.exit | 1
> tests/qapi-schema/include-cycle.json | 1
> tests/qapi-schema/include-cycle.out | 0
> tests/qapi-schema/include-format-err.err | 1
> tests/qapi-schema/include-format-err.exit | 1
> tests/qapi-schema/include-format-err.json | 2 +
> tests/qapi-schema/include-format-err.out | 0
> tests/qapi-schema/include-nested-err.err | 2 +
> tests/qapi-schema/include-nested-err.exit | 1
> tests/qapi-schema/include-nested-err.json | 1
> tests/qapi-schema/include-nested-err.out | 0
> tests/qapi-schema/include-no-file.err | 1
> tests/qapi-schema/include-no-file.exit | 1
> tests/qapi-schema/include-no-file.json | 1
> tests/qapi-schema/include-no-file.out | 0
> tests/qapi-schema/include-non-file.err | 1
> tests/qapi-schema/include-non-file.exit | 1
> tests/qapi-schema/include-non-file.json | 1
> tests/qapi-schema/include-non-file.out | 0
> tests/qapi-schema/include-relpath-sub.json | 2 +
> tests/qapi-schema/include-relpath.err | 0
> tests/qapi-schema/include-relpath.exit | 1
> tests/qapi-schema/include-relpath.json | 1
> tests/qapi-schema/include-relpath.out | 3 +
> tests/qapi-schema/include-self-cycle.err | 1
> tests/qapi-schema/include-self-cycle.exit | 1
> tests/qapi-schema/include-self-cycle.json | 1
> tests/qapi-schema/include-self-cycle.out | 0
> tests/qapi-schema/include-simple-sub.json | 2 +
> tests/qapi-schema/include-simple.err | 0
> tests/qapi-schema/include-simple.exit | 1
> tests/qapi-schema/include-simple.json | 1
> tests/qapi-schema/include-simple.out | 3 +
> tests/qapi-schema/include/relpath.json | 1
> 44 files changed, 110 insertions(+), 13 deletions(-)
> create mode 100644 tests/qapi-schema/include-before-err.err
> create mode 100644 tests/qapi-schema/include-before-err.exit
> create mode 100644 tests/qapi-schema/include-before-err.json
> create mode 100644 tests/qapi-schema/include-before-err.out
> create mode 100644 tests/qapi-schema/include-cycle-b.json
> create mode 100644 tests/qapi-schema/include-cycle-c.json
> create mode 100644 tests/qapi-schema/include-cycle.err
> create mode 100644 tests/qapi-schema/include-cycle.exit
> create mode 100644 tests/qapi-schema/include-cycle.json
> create mode 100644 tests/qapi-schema/include-cycle.out
> create mode 100644 tests/qapi-schema/include-format-err.err
> create mode 100644 tests/qapi-schema/include-format-err.exit
> create mode 100644 tests/qapi-schema/include-format-err.json
> create mode 100644 tests/qapi-schema/include-format-err.out
> create mode 100644 tests/qapi-schema/include-nested-err.err
> create mode 100644 tests/qapi-schema/include-nested-err.exit
> create mode 100644 tests/qapi-schema/include-nested-err.json
> create mode 100644 tests/qapi-schema/include-nested-err.out
> create mode 100644 tests/qapi-schema/include-no-file.err
> create mode 100644 tests/qapi-schema/include-no-file.exit
> create mode 100644 tests/qapi-schema/include-no-file.json
> create mode 100644 tests/qapi-schema/include-no-file.out
> create mode 100644 tests/qapi-schema/include-non-file.err
> create mode 100644 tests/qapi-schema/include-non-file.exit
> create mode 100644 tests/qapi-schema/include-non-file.json
> create mode 100644 tests/qapi-schema/include-non-file.out
> create mode 100644 tests/qapi-schema/include-relpath-sub.json
> create mode 100644 tests/qapi-schema/include-relpath.err
> create mode 100644 tests/qapi-schema/include-relpath.exit
> create mode 100644 tests/qapi-schema/include-relpath.json
> create mode 100644 tests/qapi-schema/include-relpath.out
> create mode 100644 tests/qapi-schema/include-self-cycle.err
> create mode 100644 tests/qapi-schema/include-self-cycle.exit
> create mode 100644 tests/qapi-schema/include-self-cycle.json
> create mode 100644 tests/qapi-schema/include-self-cycle.out
> create mode 100644 tests/qapi-schema/include-simple-sub.json
> create mode 100644 tests/qapi-schema/include-simple.err
> create mode 100644 tests/qapi-schema/include-simple.exit
> create mode 100644 tests/qapi-schema/include-simple.json
> create mode 100644 tests/qapi-schema/include-simple.out
> create mode 100644 tests/qapi-schema/include/relpath.json
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index 63b03cf..051d109 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -40,6 +40,17 @@ enumeration types and union types.
> Generally speaking, types definitions should always use CamelCase for the type
> names. Command names should be all lower case with words separated by a hyphen.
> +
> +=== Includes ===
> +
> +The QAPI schema definitions can be modularized using the 'include' directive:
> +
> + { 'include': 'path/to/file.json'}
> +
> +The directive is evaluated recursively, and include paths are relative to the
> +file using the directive.
> +
> +
> === Complex types ===
> A complex type is a dictionary containing a single key whose value is a
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 07a7a82..ec806aa 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -11,6 +11,7 @@
> # This work is licensed under the terms of the GNU GPL, version 2.
> # See the COPYING file in the top-level directory.
> +import re
> from ordereddict import OrderedDict
> import os
> import sys
> @@ -36,9 +37,17 @@ builtin_type_qtypes = {
> 'uint64': 'QTYPE_QINT',
> }
> +def error_path(parent):
> + res = ""
> + while parent:
> + res = ("In file included from %s:%d:\n" % (parent['file'],
> + parent['line'])) + res
> + parent = parent['parent']
> + return res
> +
> class QAPISchemaError(Exception):
> def __init__(self, schema, msg):
> - self.fp = schema.fp
> + self.input_file = schema.input_file
> self.msg = msg
> self.col = 1
> self.line = schema.line
> @@ -47,23 +56,31 @@ class QAPISchemaError(Exception):
> self.col = (self.col + 7) % 8 + 1
> else:
> self.col += 1
> + self.info = schema.parent_info
> def __str__(self):
> - return "%s:%s:%s: %s" % (self.fp.name, self.line, self.col, self.msg)
> + return error_path(self.info) + \
> + "%s:%d:%d: %s" % (self.input_file, self.line, self.col, self.msg)
> class QAPIExprError(Exception):
> def __init__(self, expr_info, msg):
> - self.fp = expr_info['fp']
> - self.line = expr_info['line']
> + self.info = expr_info
> self.msg = msg
> def __str__(self):
> - return "%s:%s: %s" % (self.fp.name, self.line, self.msg)
> + return error_path(self.info['parent']) + \
> + "%s:%d: %s" % (self.info['file'], self.info['line'], self.msg)
> class QAPISchema:
> - def __init__(self, fp):
> - self.fp = fp
> + def __init__(self, fp, input_relname=None, include_hist=[], parent_info=None):
> + input_fname = os.path.abspath(fp.name)
> + if input_relname is None:
> + input_relname = fp.name
> + self.input_dir = os.path.dirname(input_fname)
> + self.input_file = input_relname
> + self.include_hist = include_hist + [(input_relname, input_fname)]
> + self.parent_info = parent_info
> self.src = fp.read()
> if self.src == '' or self.src[-1] != '\n':
> self.src += '\n'
> @@ -74,10 +91,33 @@ class QAPISchema:
> self.accept()
> while self.tok != None:
> - expr_info = {'fp': fp, 'line': self.line}
> - expr_elem = {'expr': self.get_expr(False),
> - 'info': expr_info}
> - self.exprs.append(expr_elem)
> + expr_info = {'file': input_relname, 'line': self.line, 'parent': self.parent_info}
> + expr = self.get_expr(False)
> + if isinstance(expr, dict) and "include" in expr:
> + if len(expr) != 1:
> + raise QAPIExprError(expr_info, "Invalid 'include' directive")
> + include = expr["include"]
> + if not isinstance(include, str):
> + raise QAPIExprError(expr_info,
> + 'Expected a file name (string), got: %s'
> + % include)
> + include_path = os.path.join(self.input_dir, include)
> + if any(include_path == elem[1]
> + for elem in self.include_hist):
> + raise QAPIExprError(expr_info, "Inclusion loop for %s"
> + % include)
> + try:
> + fobj = open(include_path, 'r')
> + except IOError as e:
> + raise QAPIExprError(expr_info,
> + '%s: %s' % (e.strerror, include))
> + exprs_include = QAPISchema(fobj, include,
> + self.include_hist, expr_info)
> + self.exprs.extend(exprs_include.exprs)
> + else:
> + expr_elem = {'expr': expr,
> + 'info': expr_info}
> + self.exprs.append(expr_elem)
> def accept(self):
> while True:
> @@ -267,7 +307,7 @@ def check_exprs(schema):
> def parse_schema(input_file):
> try:
> schema = QAPISchema(open(input_file, "r"))
> - except QAPISchemaError, e:
> + except (QAPISchemaError, QAPIExprError), e:
> print >>sys.stderr, e
> exit(1)
> diff --git a/tests/Makefile b/tests/Makefile
> index 3aa9b0b..409bd24 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -164,7 +164,10 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
> duplicate-key.json union-invalid-base.json flat-union-no-base.json \
> flat-union-invalid-discriminator.json \
> flat-union-invalid-branch-key.json flat-union-reverse-define.json \
> - flat-union-string-discriminator.json)
> + flat-union-string-discriminator.json \
> + include-simple.json include-relpath.json include-format-err.json \
> + include-non-file.json include-no-file.json include-before-err.json \
> + include-nested-err.json include-self-cycle.json include-cycle.json)
> GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h tests/test-qmp-commands.h
> diff --git a/tests/qapi-schema/include-before-err.err b/tests/qapi-schema/include-before-err.err
> new file mode 100644
> index 0000000..5565275
> --- /dev/null
> +++ b/tests/qapi-schema/include-before-err.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-before-err.json:2:13: Expected ":"
> diff --git a/tests/qapi-schema/include-before-err.exit b/tests/qapi-schema/include-before-err.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-before-err.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-before-err.json b/tests/qapi-schema/include-before-err.json
> new file mode 100644
> index 0000000..afb6cb6
> --- /dev/null
> +++ b/tests/qapi-schema/include-before-err.json
> @@ -0,0 +1,2 @@
> +{ 'include': 'include-simple-sub.json' }
> +{ 'command' 'missing-colon' }
> diff --git a/tests/qapi-schema/include-before-err.out b/tests/qapi-schema/include-before-err.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-cycle-b.json b/tests/qapi-schema/include-cycle-b.json
> new file mode 100644
> index 0000000..4fa985d
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle-b.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-cycle-c.json' }
> diff --git a/tests/qapi-schema/include-cycle-c.json b/tests/qapi-schema/include-cycle-c.json
> new file mode 100644
> index 0000000..d12b592
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle-c.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-cycle.json' }
> diff --git a/tests/qapi-schema/include-cycle.err b/tests/qapi-schema/include-cycle.err
> new file mode 100644
> index 0000000..602cf62
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle.err
> @@ -0,0 +1,3 @@
> +In file included from tests/qapi-schema/include-cycle.json:1:
> +In file included from include-cycle-b.json:1:
> +include-cycle-c.json:1: Inclusion loop for include-cycle.json
> diff --git a/tests/qapi-schema/include-cycle.exit b/tests/qapi-schema/include-cycle.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-cycle.json b/tests/qapi-schema/include-cycle.json
> new file mode 100644
> index 0000000..6fcf1eb
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-cycle-b.json' }
> diff --git a/tests/qapi-schema/include-cycle.out b/tests/qapi-schema/include-cycle.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-format-err.err b/tests/qapi-schema/include-format-err.err
> new file mode 100644
> index 0000000..721ff4e
> --- /dev/null
> +++ b/tests/qapi-schema/include-format-err.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-format-err.json:1: Invalid 'include' directive
> diff --git a/tests/qapi-schema/include-format-err.exit b/tests/qapi-schema/include-format-err.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-format-err.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-format-err.json b/tests/qapi-schema/include-format-err.json
> new file mode 100644
> index 0000000..44980f0
> --- /dev/null
> +++ b/tests/qapi-schema/include-format-err.json
> @@ -0,0 +1,2 @@
> +{ 'include': 'include-simple-sub.json',
> + 'foo': 'bar' }
> diff --git a/tests/qapi-schema/include-format-err.out b/tests/qapi-schema/include-format-err.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-nested-err.err b/tests/qapi-schema/include-nested-err.err
> new file mode 100644
> index 0000000..1dacbda
> --- /dev/null
> +++ b/tests/qapi-schema/include-nested-err.err
> @@ -0,0 +1,2 @@
> +In file included from tests/qapi-schema/include-nested-err.json:1:
> +missing-colon.json:1:10: Expected ":"
> diff --git a/tests/qapi-schema/include-nested-err.exit b/tests/qapi-schema/include-nested-err.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-nested-err.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-nested-err.json b/tests/qapi-schema/include-nested-err.json
> new file mode 100644
> index 0000000..5631e56
> --- /dev/null
> +++ b/tests/qapi-schema/include-nested-err.json
> @@ -0,0 +1 @@
> +{ 'include': 'missing-colon.json' }
> diff --git a/tests/qapi-schema/include-nested-err.out b/tests/qapi-schema/include-nested-err.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-no-file.err b/tests/qapi-schema/include-no-file.err
> new file mode 100644
> index 0000000..d5b9b22
> --- /dev/null
> +++ b/tests/qapi-schema/include-no-file.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-no-file.json:1: No such file or directory: include-no-file-sub.json
> diff --git a/tests/qapi-schema/include-no-file.exit b/tests/qapi-schema/include-no-file.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-no-file.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-no-file.json b/tests/qapi-schema/include-no-file.json
> new file mode 100644
> index 0000000..9249ebd
> --- /dev/null
> +++ b/tests/qapi-schema/include-no-file.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-no-file-sub.json' }
> diff --git a/tests/qapi-schema/include-no-file.out b/tests/qapi-schema/include-no-file.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-non-file.err b/tests/qapi-schema/include-non-file.err
> new file mode 100644
> index 0000000..9658c78
> --- /dev/null
> +++ b/tests/qapi-schema/include-non-file.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-non-file.json:1: Expected a file name (string), got: ['foo', 'bar']
> diff --git a/tests/qapi-schema/include-non-file.exit b/tests/qapi-schema/include-non-file.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-non-file.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-non-file.json b/tests/qapi-schema/include-non-file.json
> new file mode 100644
> index 0000000..cd43c3f
> --- /dev/null
> +++ b/tests/qapi-schema/include-non-file.json
> @@ -0,0 +1 @@
> +{ 'include': [ 'foo', 'bar' ] }
> diff --git a/tests/qapi-schema/include-non-file.out b/tests/qapi-schema/include-non-file.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-relpath-sub.json b/tests/qapi-schema/include-relpath-sub.json
> new file mode 100644
> index 0000000..4bd4af4
> --- /dev/null
> +++ b/tests/qapi-schema/include-relpath-sub.json
> @@ -0,0 +1,2 @@
> +{ 'enum': 'Status',
> + 'data': [ 'good', 'bad', 'ugly' ] }
> diff --git a/tests/qapi-schema/include-relpath.err b/tests/qapi-schema/include-relpath.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-relpath.exit b/tests/qapi-schema/include-relpath.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/include-relpath.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/include-relpath.json b/tests/qapi-schema/include-relpath.json
> new file mode 100644
> index 0000000..05018f3
> --- /dev/null
> +++ b/tests/qapi-schema/include-relpath.json
> @@ -0,0 +1 @@
> +{ 'include': 'include/relpath.json' }
> diff --git a/tests/qapi-schema/include-relpath.out b/tests/qapi-schema/include-relpath.out
> new file mode 100644
> index 0000000..4ce3dcf
> --- /dev/null
> +++ b/tests/qapi-schema/include-relpath.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
> +[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
> +[]
> diff --git a/tests/qapi-schema/include-self-cycle.err b/tests/qapi-schema/include-self-cycle.err
> new file mode 100644
> index 0000000..981742a
> --- /dev/null
> +++ b/tests/qapi-schema/include-self-cycle.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-self-cycle.json:1: Inclusion loop for include-self-cycle.json
> diff --git a/tests/qapi-schema/include-self-cycle.exit b/tests/qapi-schema/include-self-cycle.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-self-cycle.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-self-cycle.json b/tests/qapi-schema/include-self-cycle.json
> new file mode 100644
> index 0000000..55fb1b5
> --- /dev/null
> +++ b/tests/qapi-schema/include-self-cycle.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-self-cycle.json' }
> diff --git a/tests/qapi-schema/include-self-cycle.out b/tests/qapi-schema/include-self-cycle.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-simple-sub.json b/tests/qapi-schema/include-simple-sub.json
> new file mode 100644
> index 0000000..4bd4af4
> --- /dev/null
> +++ b/tests/qapi-schema/include-simple-sub.json
> @@ -0,0 +1,2 @@
> +{ 'enum': 'Status',
> + 'data': [ 'good', 'bad', 'ugly' ] }
> diff --git a/tests/qapi-schema/include-simple.err b/tests/qapi-schema/include-simple.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-simple.exit b/tests/qapi-schema/include-simple.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/include-simple.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/include-simple.json b/tests/qapi-schema/include-simple.json
> new file mode 100644
> index 0000000..1dd391a
> --- /dev/null
> +++ b/tests/qapi-schema/include-simple.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-simple-sub.json' }
> diff --git a/tests/qapi-schema/include-simple.out b/tests/qapi-schema/include-simple.out
> new file mode 100644
> index 0000000..4ce3dcf
> --- /dev/null
> +++ b/tests/qapi-schema/include-simple.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
> +[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
> +[]
> diff --git a/tests/qapi-schema/include/relpath.json b/tests/qapi-schema/include/relpath.json
> new file mode 100644
> index 0000000..45dee24
> --- /dev/null
> +++ b/tests/qapi-schema/include/relpath.json
> @@ -0,0 +1 @@
> +{ 'include': '../include-relpath-sub.json' }
--
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-06 14:55 ` Eric Blake
2014-05-06 16:55 ` Luiz Capitulino
@ 2014-05-07 19:44 ` Markus Armbruster
2014-05-07 20:01 ` Benoît Canet
1 sibling, 1 reply; 26+ messages in thread
From: Markus Armbruster @ 2014-05-07 19:44 UTC (permalink / raw)
To: Eric Blake
Cc: Benoît Canet, qemu-devel, Lluís Vilanova,
Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> On 05/06/2014 07:27 AM, Luiz Capitulino wrote:
>> On Tue, 6 May 2014 15:07:40 +0200
>> Benoît Canet <benoit.canet@irqsave.net> wrote:
>>
>>> I am trying to use this series to modularise the block API.
>>>
>>> Here are my finding.
>>>
>>> I tried to make a qmp/block.json including VM state related API.
>>> block.json include a qmp/block-core.json containing only true block stuff.
>>>
>>> When generating and compiling block-core.json to link it with qemu-nbd
>>> I saw that some of the block stuff needed ErrorClass so I went the route
>>> of creating a qmp/common.json containing ErrorClass.
>>>
>>> common.json being included in block-core.json and in qapi-schema.json it
>>> quickly lead some code being generated in double and the
>>> compilation to choke.
>>>
>>> What do you think would be the best solution to fix this ?
>>> (Fix the generator ? Make include ignore second inclusion of the same file ?)
>>
>> Make qapi-schema.json a sort of master file and include everything?
>
> Won't cut it, if we want to support a subset of files in other contexts.
> You either have to do:
>
> qemu-schema.json:
> include common
> include block
> include other
>
> qemu-block:
> include common
> include block
>
> where block does not work in isolation, and has to be wrapped; but now
> we have two different wrappers depending on the two different clients
> that want a different subset.
Won't win beauty contests, but it isn't exactly terrible, either.
> Or you do:
>
> qemu-schema.json:
> include common
> include block
> include other
> block.json:
> include common
>
> now block.json is standalone, and qemu-schema.json ends up including
> common through two different paths.
Yes.
>> Eventually, we might want to have if/defs and whatnot. But having a master
>> file seems a reasonable first step to me. I actually thought this was the
>> intention. Unless I got it wrong, of course.
>
> Ifdefs may be a bit much. If we add them, then we can worry about
> explicit include guards, the same as the C preprocessor. But for now,
> I'd be perfectly fine with a followup patch that includes a file's
> contents exactly once, no matter how many times it is included (that is,
> act as if include guards were implicitly present, since we lack
> conditionals, so include files are currently idempotent).
As long as the meaning of an include doesn't depend on its environment
(and I don't see that change for QAPI schemata), making the include
idempotent is the right thing.
Benoît, Lluís, would either of you be willing to tackle this?
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files
2014-05-07 19:44 ` Markus Armbruster
@ 2014-05-07 20:01 ` Benoît Canet
0 siblings, 0 replies; 26+ messages in thread
From: Benoît Canet @ 2014-05-07 20:01 UTC (permalink / raw)
To: Markus Armbruster
Cc: Benoît Canet, Luiz Capitulino, qemu-devel,
Lluís Vilanova
The Wednesday 07 May 2014 à 21:44:06 (+0200), Markus Armbruster wrote :
> Eric Blake <eblake@redhat.com> writes:
>
> > On 05/06/2014 07:27 AM, Luiz Capitulino wrote:
> >> On Tue, 6 May 2014 15:07:40 +0200
> >> Benoît Canet <benoit.canet@irqsave.net> wrote:
> >>
> >>> I am trying to use this series to modularise the block API.
> >>>
> >>> Here are my finding.
> >>>
> >>> I tried to make a qmp/block.json including VM state related API.
> >>> block.json include a qmp/block-core.json containing only true block stuff.
> >>>
> >>> When generating and compiling block-core.json to link it with qemu-nbd
> >>> I saw that some of the block stuff needed ErrorClass so I went the route
> >>> of creating a qmp/common.json containing ErrorClass.
> >>>
> >>> common.json being included in block-core.json and in qapi-schema.json it
> >>> quickly lead some code being generated in double and the
> >>> compilation to choke.
> >>>
> >>> What do you think would be the best solution to fix this ?
> >>> (Fix the generator ? Make include ignore second inclusion of the same file ?)
> >>
> >> Make qapi-schema.json a sort of master file and include everything?
> >
> > Won't cut it, if we want to support a subset of files in other contexts.
> > You either have to do:
> >
> > qemu-schema.json:
> > include common
> > include block
> > include other
> >
> > qemu-block:
> > include common
> > include block
> >
> > where block does not work in isolation, and has to be wrapped; but now
> > we have two different wrappers depending on the two different clients
> > that want a different subset.
>
> Won't win beauty contests, but it isn't exactly terrible, either.
>
> > Or you do:
> >
> > qemu-schema.json:
> > include common
> > include block
> > include other
> > block.json:
> > include common
> >
> > now block.json is standalone, and qemu-schema.json ends up including
> > common through two different paths.
>
> Yes.
>
> >> Eventually, we might want to have if/defs and whatnot. But having a master
> >> file seems a reasonable first step to me. I actually thought this was the
> >> intention. Unless I got it wrong, of course.
> >
> > Ifdefs may be a bit much. If we add them, then we can worry about
> > explicit include guards, the same as the C preprocessor. But for now,
> > I'd be perfectly fine with a followup patch that includes a file's
> > contents exactly once, no matter how many times it is included (that is,
> > act as if include guards were implicitly present, since we lack
> > conditionals, so include files are currently idempotent).
>
> As long as the meaning of an include doesn't depend on its environment
> (and I don't see that change for QAPI schemata), making the include
> idempotent is the right thing.
>
> Benoît, Lluís, would either of you be willing to tackle this?
I need it badly I will do it.
Best regards
Benoît
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 4/4] qapi: Add a primitive to include other files from a QAPI schema file
2014-05-07 18:46 ` Lluís Vilanova
2014-05-07 18:48 ` Lluís Vilanova
@ 2014-05-07 20:44 ` Luiz Capitulino
2014-05-18 11:44 ` Peter Maydell
2 siblings, 0 replies; 26+ messages in thread
From: Luiz Capitulino @ 2014-05-07 20:44 UTC (permalink / raw)
To: Lluís Vilanova; +Cc: Benoît Canet, qemu-devel, Markus Armbruster
On Wed, 07 May 2014 20:46:15 +0200
Lluís Vilanova <vilanova@ac.upc.edu> wrote:
> The primitive uses JSON syntax, and include paths are relative to the file using the directive:
>
> { 'include': 'path/to/file.json' }
>
> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
I've replaced the patch and now the tests pass. Thanks Lluís!
> ---
> docs/qapi-code-gen.txt | 11 +++++
> scripts/qapi.py | 64 +++++++++++++++++++++++-----
> tests/Makefile | 5 ++
> tests/qapi-schema/include-before-err.err | 1
> tests/qapi-schema/include-before-err.exit | 1
> tests/qapi-schema/include-before-err.json | 2 +
> tests/qapi-schema/include-before-err.out | 0
> tests/qapi-schema/include-cycle-b.json | 1
> tests/qapi-schema/include-cycle-c.json | 1
> tests/qapi-schema/include-cycle.err | 3 +
> tests/qapi-schema/include-cycle.exit | 1
> tests/qapi-schema/include-cycle.json | 1
> tests/qapi-schema/include-cycle.out | 0
> tests/qapi-schema/include-format-err.err | 1
> tests/qapi-schema/include-format-err.exit | 1
> tests/qapi-schema/include-format-err.json | 2 +
> tests/qapi-schema/include-format-err.out | 0
> tests/qapi-schema/include-nested-err.err | 2 +
> tests/qapi-schema/include-nested-err.exit | 1
> tests/qapi-schema/include-nested-err.json | 1
> tests/qapi-schema/include-nested-err.out | 0
> tests/qapi-schema/include-no-file.err | 1
> tests/qapi-schema/include-no-file.exit | 1
> tests/qapi-schema/include-no-file.json | 1
> tests/qapi-schema/include-no-file.out | 0
> tests/qapi-schema/include-non-file.err | 1
> tests/qapi-schema/include-non-file.exit | 1
> tests/qapi-schema/include-non-file.json | 1
> tests/qapi-schema/include-non-file.out | 0
> tests/qapi-schema/include-relpath-sub.json | 2 +
> tests/qapi-schema/include-relpath.err | 0
> tests/qapi-schema/include-relpath.exit | 1
> tests/qapi-schema/include-relpath.json | 1
> tests/qapi-schema/include-relpath.out | 3 +
> tests/qapi-schema/include-self-cycle.err | 1
> tests/qapi-schema/include-self-cycle.exit | 1
> tests/qapi-schema/include-self-cycle.json | 1
> tests/qapi-schema/include-self-cycle.out | 0
> tests/qapi-schema/include-simple-sub.json | 2 +
> tests/qapi-schema/include-simple.err | 0
> tests/qapi-schema/include-simple.exit | 1
> tests/qapi-schema/include-simple.json | 1
> tests/qapi-schema/include-simple.out | 3 +
> tests/qapi-schema/include/relpath.json | 1
> 44 files changed, 110 insertions(+), 13 deletions(-)
> create mode 100644 tests/qapi-schema/include-before-err.err
> create mode 100644 tests/qapi-schema/include-before-err.exit
> create mode 100644 tests/qapi-schema/include-before-err.json
> create mode 100644 tests/qapi-schema/include-before-err.out
> create mode 100644 tests/qapi-schema/include-cycle-b.json
> create mode 100644 tests/qapi-schema/include-cycle-c.json
> create mode 100644 tests/qapi-schema/include-cycle.err
> create mode 100644 tests/qapi-schema/include-cycle.exit
> create mode 100644 tests/qapi-schema/include-cycle.json
> create mode 100644 tests/qapi-schema/include-cycle.out
> create mode 100644 tests/qapi-schema/include-format-err.err
> create mode 100644 tests/qapi-schema/include-format-err.exit
> create mode 100644 tests/qapi-schema/include-format-err.json
> create mode 100644 tests/qapi-schema/include-format-err.out
> create mode 100644 tests/qapi-schema/include-nested-err.err
> create mode 100644 tests/qapi-schema/include-nested-err.exit
> create mode 100644 tests/qapi-schema/include-nested-err.json
> create mode 100644 tests/qapi-schema/include-nested-err.out
> create mode 100644 tests/qapi-schema/include-no-file.err
> create mode 100644 tests/qapi-schema/include-no-file.exit
> create mode 100644 tests/qapi-schema/include-no-file.json
> create mode 100644 tests/qapi-schema/include-no-file.out
> create mode 100644 tests/qapi-schema/include-non-file.err
> create mode 100644 tests/qapi-schema/include-non-file.exit
> create mode 100644 tests/qapi-schema/include-non-file.json
> create mode 100644 tests/qapi-schema/include-non-file.out
> create mode 100644 tests/qapi-schema/include-relpath-sub.json
> create mode 100644 tests/qapi-schema/include-relpath.err
> create mode 100644 tests/qapi-schema/include-relpath.exit
> create mode 100644 tests/qapi-schema/include-relpath.json
> create mode 100644 tests/qapi-schema/include-relpath.out
> create mode 100644 tests/qapi-schema/include-self-cycle.err
> create mode 100644 tests/qapi-schema/include-self-cycle.exit
> create mode 100644 tests/qapi-schema/include-self-cycle.json
> create mode 100644 tests/qapi-schema/include-self-cycle.out
> create mode 100644 tests/qapi-schema/include-simple-sub.json
> create mode 100644 tests/qapi-schema/include-simple.err
> create mode 100644 tests/qapi-schema/include-simple.exit
> create mode 100644 tests/qapi-schema/include-simple.json
> create mode 100644 tests/qapi-schema/include-simple.out
> create mode 100644 tests/qapi-schema/include/relpath.json
>
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index 63b03cf..051d109 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -40,6 +40,17 @@ enumeration types and union types.
> Generally speaking, types definitions should always use CamelCase for the type
> names. Command names should be all lower case with words separated by a hyphen.
>
> +
> +=== Includes ===
> +
> +The QAPI schema definitions can be modularized using the 'include' directive:
> +
> + { 'include': 'path/to/file.json'}
> +
> +The directive is evaluated recursively, and include paths are relative to the
> +file using the directive.
> +
> +
> === Complex types ===
>
> A complex type is a dictionary containing a single key whose value is a
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 07a7a82..ec806aa 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -11,6 +11,7 @@
> # This work is licensed under the terms of the GNU GPL, version 2.
> # See the COPYING file in the top-level directory.
>
> +import re
> from ordereddict import OrderedDict
> import os
> import sys
> @@ -36,9 +37,17 @@ builtin_type_qtypes = {
> 'uint64': 'QTYPE_QINT',
> }
>
> +def error_path(parent):
> + res = ""
> + while parent:
> + res = ("In file included from %s:%d:\n" % (parent['file'],
> + parent['line'])) + res
> + parent = parent['parent']
> + return res
> +
> class QAPISchemaError(Exception):
> def __init__(self, schema, msg):
> - self.fp = schema.fp
> + self.input_file = schema.input_file
> self.msg = msg
> self.col = 1
> self.line = schema.line
> @@ -47,23 +56,31 @@ class QAPISchemaError(Exception):
> self.col = (self.col + 7) % 8 + 1
> else:
> self.col += 1
> + self.info = schema.parent_info
>
> def __str__(self):
> - return "%s:%s:%s: %s" % (self.fp.name, self.line, self.col, self.msg)
> + return error_path(self.info) + \
> + "%s:%d:%d: %s" % (self.input_file, self.line, self.col, self.msg)
>
> class QAPIExprError(Exception):
> def __init__(self, expr_info, msg):
> - self.fp = expr_info['fp']
> - self.line = expr_info['line']
> + self.info = expr_info
> self.msg = msg
>
> def __str__(self):
> - return "%s:%s: %s" % (self.fp.name, self.line, self.msg)
> + return error_path(self.info['parent']) + \
> + "%s:%d: %s" % (self.info['file'], self.info['line'], self.msg)
>
> class QAPISchema:
>
> - def __init__(self, fp):
> - self.fp = fp
> + def __init__(self, fp, input_relname=None, include_hist=[], parent_info=None):
> + input_fname = os.path.abspath(fp.name)
> + if input_relname is None:
> + input_relname = fp.name
> + self.input_dir = os.path.dirname(input_fname)
> + self.input_file = input_relname
> + self.include_hist = include_hist + [(input_relname, input_fname)]
> + self.parent_info = parent_info
> self.src = fp.read()
> if self.src == '' or self.src[-1] != '\n':
> self.src += '\n'
> @@ -74,10 +91,33 @@ class QAPISchema:
> self.accept()
>
> while self.tok != None:
> - expr_info = {'fp': fp, 'line': self.line}
> - expr_elem = {'expr': self.get_expr(False),
> - 'info': expr_info}
> - self.exprs.append(expr_elem)
> + expr_info = {'file': input_relname, 'line': self.line, 'parent': self.parent_info}
> + expr = self.get_expr(False)
> + if isinstance(expr, dict) and "include" in expr:
> + if len(expr) != 1:
> + raise QAPIExprError(expr_info, "Invalid 'include' directive")
> + include = expr["include"]
> + if not isinstance(include, str):
> + raise QAPIExprError(expr_info,
> + 'Expected a file name (string), got: %s'
> + % include)
> + include_path = os.path.join(self.input_dir, include)
> + if any(include_path == elem[1]
> + for elem in self.include_hist):
> + raise QAPIExprError(expr_info, "Inclusion loop for %s"
> + % include)
> + try:
> + fobj = open(include_path, 'r')
> + except IOError as e:
> + raise QAPIExprError(expr_info,
> + '%s: %s' % (e.strerror, include))
> + exprs_include = QAPISchema(fobj, include,
> + self.include_hist, expr_info)
> + self.exprs.extend(exprs_include.exprs)
> + else:
> + expr_elem = {'expr': expr,
> + 'info': expr_info}
> + self.exprs.append(expr_elem)
>
> def accept(self):
> while True:
> @@ -267,7 +307,7 @@ def check_exprs(schema):
> def parse_schema(input_file):
> try:
> schema = QAPISchema(open(input_file, "r"))
> - except QAPISchemaError, e:
> + except (QAPISchemaError, QAPIExprError), e:
> print >>sys.stderr, e
> exit(1)
>
> diff --git a/tests/Makefile b/tests/Makefile
> index 3aa9b0b..409bd24 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -164,7 +164,10 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
> duplicate-key.json union-invalid-base.json flat-union-no-base.json \
> flat-union-invalid-discriminator.json \
> flat-union-invalid-branch-key.json flat-union-reverse-define.json \
> - flat-union-string-discriminator.json)
> + flat-union-string-discriminator.json \
> + include-simple.json include-relpath.json include-format-err.json \
> + include-non-file.json include-no-file.json include-before-err.json \
> + include-nested-err.json include-self-cycle.json include-cycle.json)
>
> GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h tests/test-qmp-commands.h
>
> diff --git a/tests/qapi-schema/include-before-err.err b/tests/qapi-schema/include-before-err.err
> new file mode 100644
> index 0000000..5565275
> --- /dev/null
> +++ b/tests/qapi-schema/include-before-err.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-before-err.json:2:13: Expected ":"
> diff --git a/tests/qapi-schema/include-before-err.exit b/tests/qapi-schema/include-before-err.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-before-err.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-before-err.json b/tests/qapi-schema/include-before-err.json
> new file mode 100644
> index 0000000..afb6cb6
> --- /dev/null
> +++ b/tests/qapi-schema/include-before-err.json
> @@ -0,0 +1,2 @@
> +{ 'include': 'include-simple-sub.json' }
> +{ 'command' 'missing-colon' }
> diff --git a/tests/qapi-schema/include-before-err.out b/tests/qapi-schema/include-before-err.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-cycle-b.json b/tests/qapi-schema/include-cycle-b.json
> new file mode 100644
> index 0000000..4fa985d
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle-b.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-cycle-c.json' }
> diff --git a/tests/qapi-schema/include-cycle-c.json b/tests/qapi-schema/include-cycle-c.json
> new file mode 100644
> index 0000000..d12b592
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle-c.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-cycle.json' }
> diff --git a/tests/qapi-schema/include-cycle.err b/tests/qapi-schema/include-cycle.err
> new file mode 100644
> index 0000000..602cf62
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle.err
> @@ -0,0 +1,3 @@
> +In file included from tests/qapi-schema/include-cycle.json:1:
> +In file included from include-cycle-b.json:1:
> +include-cycle-c.json:1: Inclusion loop for include-cycle.json
> diff --git a/tests/qapi-schema/include-cycle.exit b/tests/qapi-schema/include-cycle.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-cycle.json b/tests/qapi-schema/include-cycle.json
> new file mode 100644
> index 0000000..6fcf1eb
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-cycle-b.json' }
> diff --git a/tests/qapi-schema/include-cycle.out b/tests/qapi-schema/include-cycle.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-format-err.err b/tests/qapi-schema/include-format-err.err
> new file mode 100644
> index 0000000..721ff4e
> --- /dev/null
> +++ b/tests/qapi-schema/include-format-err.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-format-err.json:1: Invalid 'include' directive
> diff --git a/tests/qapi-schema/include-format-err.exit b/tests/qapi-schema/include-format-err.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-format-err.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-format-err.json b/tests/qapi-schema/include-format-err.json
> new file mode 100644
> index 0000000..44980f0
> --- /dev/null
> +++ b/tests/qapi-schema/include-format-err.json
> @@ -0,0 +1,2 @@
> +{ 'include': 'include-simple-sub.json',
> + 'foo': 'bar' }
> diff --git a/tests/qapi-schema/include-format-err.out b/tests/qapi-schema/include-format-err.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-nested-err.err b/tests/qapi-schema/include-nested-err.err
> new file mode 100644
> index 0000000..1dacbda
> --- /dev/null
> +++ b/tests/qapi-schema/include-nested-err.err
> @@ -0,0 +1,2 @@
> +In file included from tests/qapi-schema/include-nested-err.json:1:
> +missing-colon.json:1:10: Expected ":"
> diff --git a/tests/qapi-schema/include-nested-err.exit b/tests/qapi-schema/include-nested-err.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-nested-err.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-nested-err.json b/tests/qapi-schema/include-nested-err.json
> new file mode 100644
> index 0000000..5631e56
> --- /dev/null
> +++ b/tests/qapi-schema/include-nested-err.json
> @@ -0,0 +1 @@
> +{ 'include': 'missing-colon.json' }
> diff --git a/tests/qapi-schema/include-nested-err.out b/tests/qapi-schema/include-nested-err.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-no-file.err b/tests/qapi-schema/include-no-file.err
> new file mode 100644
> index 0000000..d5b9b22
> --- /dev/null
> +++ b/tests/qapi-schema/include-no-file.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-no-file.json:1: No such file or directory: include-no-file-sub.json
> diff --git a/tests/qapi-schema/include-no-file.exit b/tests/qapi-schema/include-no-file.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-no-file.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-no-file.json b/tests/qapi-schema/include-no-file.json
> new file mode 100644
> index 0000000..9249ebd
> --- /dev/null
> +++ b/tests/qapi-schema/include-no-file.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-no-file-sub.json' }
> diff --git a/tests/qapi-schema/include-no-file.out b/tests/qapi-schema/include-no-file.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-non-file.err b/tests/qapi-schema/include-non-file.err
> new file mode 100644
> index 0000000..9658c78
> --- /dev/null
> +++ b/tests/qapi-schema/include-non-file.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-non-file.json:1: Expected a file name (string), got: ['foo', 'bar']
> diff --git a/tests/qapi-schema/include-non-file.exit b/tests/qapi-schema/include-non-file.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-non-file.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-non-file.json b/tests/qapi-schema/include-non-file.json
> new file mode 100644
> index 0000000..cd43c3f
> --- /dev/null
> +++ b/tests/qapi-schema/include-non-file.json
> @@ -0,0 +1 @@
> +{ 'include': [ 'foo', 'bar' ] }
> diff --git a/tests/qapi-schema/include-non-file.out b/tests/qapi-schema/include-non-file.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-relpath-sub.json b/tests/qapi-schema/include-relpath-sub.json
> new file mode 100644
> index 0000000..4bd4af4
> --- /dev/null
> +++ b/tests/qapi-schema/include-relpath-sub.json
> @@ -0,0 +1,2 @@
> +{ 'enum': 'Status',
> + 'data': [ 'good', 'bad', 'ugly' ] }
> diff --git a/tests/qapi-schema/include-relpath.err b/tests/qapi-schema/include-relpath.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-relpath.exit b/tests/qapi-schema/include-relpath.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/include-relpath.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/include-relpath.json b/tests/qapi-schema/include-relpath.json
> new file mode 100644
> index 0000000..05018f3
> --- /dev/null
> +++ b/tests/qapi-schema/include-relpath.json
> @@ -0,0 +1 @@
> +{ 'include': 'include/relpath.json' }
> diff --git a/tests/qapi-schema/include-relpath.out b/tests/qapi-schema/include-relpath.out
> new file mode 100644
> index 0000000..4ce3dcf
> --- /dev/null
> +++ b/tests/qapi-schema/include-relpath.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
> +[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
> +[]
> diff --git a/tests/qapi-schema/include-self-cycle.err b/tests/qapi-schema/include-self-cycle.err
> new file mode 100644
> index 0000000..981742a
> --- /dev/null
> +++ b/tests/qapi-schema/include-self-cycle.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-self-cycle.json:1: Inclusion loop for include-self-cycle.json
> diff --git a/tests/qapi-schema/include-self-cycle.exit b/tests/qapi-schema/include-self-cycle.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-self-cycle.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-self-cycle.json b/tests/qapi-schema/include-self-cycle.json
> new file mode 100644
> index 0000000..55fb1b5
> --- /dev/null
> +++ b/tests/qapi-schema/include-self-cycle.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-self-cycle.json' }
> diff --git a/tests/qapi-schema/include-self-cycle.out b/tests/qapi-schema/include-self-cycle.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-simple-sub.json b/tests/qapi-schema/include-simple-sub.json
> new file mode 100644
> index 0000000..4bd4af4
> --- /dev/null
> +++ b/tests/qapi-schema/include-simple-sub.json
> @@ -0,0 +1,2 @@
> +{ 'enum': 'Status',
> + 'data': [ 'good', 'bad', 'ugly' ] }
> diff --git a/tests/qapi-schema/include-simple.err b/tests/qapi-schema/include-simple.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-simple.exit b/tests/qapi-schema/include-simple.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/include-simple.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/include-simple.json b/tests/qapi-schema/include-simple.json
> new file mode 100644
> index 0000000..1dd391a
> --- /dev/null
> +++ b/tests/qapi-schema/include-simple.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-simple-sub.json' }
> diff --git a/tests/qapi-schema/include-simple.out b/tests/qapi-schema/include-simple.out
> new file mode 100644
> index 0000000..4ce3dcf
> --- /dev/null
> +++ b/tests/qapi-schema/include-simple.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
> +[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
> +[]
> diff --git a/tests/qapi-schema/include/relpath.json b/tests/qapi-schema/include/relpath.json
> new file mode 100644
> index 0000000..45dee24
> --- /dev/null
> +++ b/tests/qapi-schema/include/relpath.json
> @@ -0,0 +1 @@
> +{ 'include': '../include-relpath-sub.json' }
>
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH v12 4/4] qapi: Add a primitive to include other files from a QAPI schema file
2014-05-07 18:46 ` Lluís Vilanova
2014-05-07 18:48 ` Lluís Vilanova
2014-05-07 20:44 ` Luiz Capitulino
@ 2014-05-18 11:44 ` Peter Maydell
2 siblings, 0 replies; 26+ messages in thread
From: Peter Maydell @ 2014-05-18 11:44 UTC (permalink / raw)
To: QEMU Developers, Benoît Canet, Markus Armbruster,
Luiz Capitulino, Andreas Färber
On 7 May 2014 19:46, Lluís Vilanova <vilanova@ac.upc.edu> wrote:
> The primitive uses JSON syntax, and include paths are relative to the file using the directive:
>
> { 'include': 'path/to/file.json' }
>
> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
> @@ -74,10 +91,33 @@ class QAPISchema:
> self.accept()
>
> while self.tok != None:
> - expr_info = {'fp': fp, 'line': self.line}
> - expr_elem = {'expr': self.get_expr(False),
> - 'info': expr_info}
> - self.exprs.append(expr_elem)
> + expr_info = {'file': input_relname, 'line': self.line, 'parent': self.parent_info}
> + expr = self.get_expr(False)
> + if isinstance(expr, dict) and "include" in expr:
> + if len(expr) != 1:
> + raise QAPIExprError(expr_info, "Invalid 'include' directive")
> + include = expr["include"]
> + if not isinstance(include, str):
> + raise QAPIExprError(expr_info,
> + 'Expected a file name (string), got: %s'
> + % include)
> + include_path = os.path.join(self.input_dir, include)
> + if any(include_path == elem[1]
> + for elem in self.include_hist):
> + raise QAPIExprError(expr_info, "Inclusion loop for %s"
> + % include)
> + try:
> + fobj = open(include_path, 'r')
> + except IOError as e:
Hi. I'm afraid "except ... as" is only supported in Python 2.6,
and we have to support Python 2.4 still (as Andreas mentioned
in another thread this breaks compilation on MacOSX 10.5).
Please can you fix this to use the older syntax (cf also commit
21e0043bad which fixed an earlier instance of this problem)?
thanks
-- PMM
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2014-05-18 11:44 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-05-02 13:52 [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files Lluís Vilanova
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 1/4] qapi: [trivial] Break long command lines Lluís Vilanova
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 2/4] qapi: [trivial] Do not catch unknown exceptions in "test-qapi.py" Lluís Vilanova
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 3/4] qapi: Use an explicit input file Lluís Vilanova
2014-05-02 13:52 ` [Qemu-devel] [PATCH v12 4/4] qapi: Add a primitive to include other files from a QAPI schema file Lluís Vilanova
2014-05-05 20:31 ` Luiz Capitulino
2014-05-05 21:15 ` Lluís Vilanova
2014-05-07 18:46 ` Lluís Vilanova
2014-05-07 18:48 ` Lluís Vilanova
2014-05-07 20:44 ` Luiz Capitulino
2014-05-18 11:44 ` Peter Maydell
2014-05-05 18:21 ` [Qemu-devel] [PATCH v12 0/4] qapi: Allow modularization of QAPI schema files Luiz Capitulino
2014-05-07 16:36 ` Luiz Capitulino
2014-05-07 17:07 ` Lluís Vilanova
2014-05-07 17:14 ` Luiz Capitulino
2014-05-07 17:20 ` Lluís Vilanova
2014-05-07 17:23 ` Luiz Capitulino
2014-05-06 13:07 ` Benoît Canet
2014-05-06 13:27 ` Luiz Capitulino
2014-05-06 14:05 ` Benoît Canet
2014-05-06 14:55 ` Eric Blake
2014-05-06 16:55 ` Luiz Capitulino
2014-05-06 17:11 ` Eric Blake
2014-05-07 19:44 ` Markus Armbruster
2014-05-07 20:01 ` Benoît Canet
2014-05-06 15:02 ` Eric Blake
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).