From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Stefan Hajnoczi" <stefanha@redhat.com>,
"Michael Roth" <michael.roth@amd.com>,
"Markus Armbruster" <armbru@redhat.com>,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Philippe Mathieu-Daudé" <philmd@linaro.org>,
"Mads Ynddal" <mads@ynddal.dk>,
"Alex Bennée" <alex.bennee@linaro.org>,
"Thomas Huth" <thuth@redhat.com>,
"Daniel P. Berrangé" <berrange@redhat.com>
Subject: [PATCH v3 8/8] qapi: switch to use QEMU_TEST_REGENERATE env var
Date: Wed, 6 Aug 2025 17:48:32 +0100 [thread overview]
Message-ID: <20250806164832.1382919-9-berrange@redhat.com> (raw)
In-Reply-To: <20250806164832.1382919-1-berrange@redhat.com>
The QAPI_TEST_UPDATE env var can be set when running the QAPI
schema tests to regenerate the reference output. For consistent
naming with the tracetool test, change the env var name to
QEMU_TEST_REGENERATE.
The test is modified to provide a hint about use of the new
env var and it is also added to the developer documentation.document its usage.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
docs/devel/testing/main.rst | 12 ++++++++++++
tests/qapi-schema/test-qapi.py | 7 +++++--
2 files changed, 17 insertions(+), 2 deletions(-)
diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
index 11f05c0006..fe233fcf7a 100644
--- a/docs/devel/testing/main.rst
+++ b/docs/devel/testing/main.rst
@@ -178,6 +178,18 @@ parser (either fixing a bug or extending/modifying the syntax). To do this:
``qapi-schema += foo.json``
+The reference output can be automatically updated to match the latest QAPI
+code generator by running the tests with the QEMU_TEST_REGENERATE environment
+variable set.
+
+.. code::
+
+ QEMU_TEST_REGENERATE=1 make check-qapi-schema
+
+The resulting changes must be reviewed by the author to ensure they match
+the intended results, before adding the updated reference output to the
+same commit that alters the generator code.
+
.. _tracetool-tests:
Tracetool tests
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index 4be930228c..cf7fb8a6df 100755
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -165,7 +165,7 @@ def test_and_diff(test_name, dir_name, update):
if actual_out == expected_out and actual_err == expected_err:
return 0
- print("%s %s" % (test_name, 'UPDATE' if update else 'FAIL'),
+ print("%s: %s" % (test_name, 'UPDATE' if update else 'FAIL'),
file=sys.stderr)
out_diff = difflib.unified_diff(expected_out, actual_out, outfp.name)
err_diff = difflib.unified_diff(expected_err, actual_err, errfp.name)
@@ -173,6 +173,9 @@ def test_and_diff(test_name, dir_name, update):
sys.stdout.writelines(err_diff)
if not update:
+ print(("\n%s: set QEMU_TEST_REGENERATE=1 to recreate reference output" +
+ "if the QAPI schema generator was intentionally changed") % test_name,
+ file=sys.stderr)
return 1
try:
@@ -197,7 +200,7 @@ def main(argv):
parser.add_argument('-d', '--dir', action='store', default='',
help="directory containing tests")
parser.add_argument('-u', '--update', action='store_true',
- default='QAPI_TEST_UPDATE' in os.environ,
+ default='QEMU_TEST_REGENERATE' in os.environ,
help="update expected test results")
parser.add_argument('tests', nargs='*', metavar='TEST', action='store')
args = parser.parse_args()
--
2.50.1
next prev parent reply other threads:[~2025-08-06 17:13 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-08-06 16:48 [PATCH v3 0/8] tracetool: add test suite to improve reviewability Daniel P. Berrangé
2025-08-06 16:48 ` [PATCH v3 1/8] checkpatch: cull trailing '*/' in SPDX check Daniel P. Berrangé
2025-08-06 16:48 ` [PATCH v3 2/8] tracetool: eliminate trailing whitespace in C format Daniel P. Berrangé
2025-08-06 16:48 ` [PATCH v3 3/8] tracetool: avoid space after "*" in arg types Daniel P. Berrangé
2025-08-06 16:48 ` [PATCH v3 4/8] tracetool: include SPDX-License-Identifier in generated files Daniel P. Berrangé
2025-08-06 16:48 ` [PATCH v3 5/8] tracetool: support "-" as a shorthand for stdout Daniel P. Berrangé
2025-08-07 19:46 ` Stefan Hajnoczi
2025-08-18 15:07 ` Daniel P. Berrangé
2025-08-18 17:55 ` Stefan Hajnoczi
2025-08-19 15:58 ` Daniel P. Berrangé
2025-08-06 16:48 ` [PATCH v3 6/8] tracetool: add test suite for tracetool with reference output Daniel P. Berrangé
2025-08-06 16:48 ` [PATCH v3 7/8] tracetool: drop the probe "__nocheck__" wrapping Daniel P. Berrangé
2025-08-06 16:48 ` Daniel P. Berrangé [this message]
2025-08-08 6:46 ` [PATCH v3 8/8] qapi: switch to use QEMU_TEST_REGENERATE env var Markus Armbruster
2025-08-18 15:09 ` Daniel P. Berrangé
2025-08-25 12:01 ` Markus Armbruster
2025-08-27 14:11 ` Daniel P. Berrangé
2025-08-07 20:06 ` [PATCH v3 0/8] tracetool: add test suite to improve reviewability Stefan Hajnoczi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250806164832.1382919-9-berrange@redhat.com \
--to=berrange@redhat.com \
--cc=alex.bennee@linaro.org \
--cc=armbru@redhat.com \
--cc=mads@ynddal.dk \
--cc=michael.roth@amd.com \
--cc=pbonzini@redhat.com \
--cc=philmd@linaro.org \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@redhat.com \
--cc=thuth@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).