From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Kevin Wolf" <kwolf@redhat.com>,
"Vladimir Sementsov-Ogievskiy" <vsementsov@virtuozzo.com>,
"Daniel P. Berrangé" <berrange@redhat.com>,
qemu-block@nongnu.org, "Markus Armbruster" <armbru@redhat.com>,
"Hanna Reitz" <hreitz@redhat.com>,
"Eric Blake" <eblake@redhat.com>
Subject: [PATCH v2 06/12] tests/qemu-iotests: add QEMU_IOTESTS_REGEN=1 to update reference file
Date: Fri, 4 Mar 2022 19:36:04 +0000 [thread overview]
Message-ID: <20220304193610.3293146-7-berrange@redhat.com> (raw)
In-Reply-To: <20220304193610.3293146-1-berrange@redhat.com>
When developing an I/O test it is typical to add some logic to the
test script, run it to view the output diff, and then apply the
output diff to the reference file. This can be drastically simplified
by letting the test runner update the reference file in place.
By setting 'QEMU_IOTESTS_REGEN=1', the test runner will report the
failure and show the diff, but at the same time update the reference
file. So next time the I/O test is run it will succeed.
Continuing to display the diff when updating the reference gives the
developer a chance to review what was changed.
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/qemu-iotests/testrunner.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
index 9a94273975..8a82696a6b 100644
--- a/tests/qemu-iotests/testrunner.py
+++ b/tests/qemu-iotests/testrunner.py
@@ -25,6 +25,7 @@
import contextlib
import json
import termios
+import shutil
import sys
from multiprocessing import Pool
from contextlib import contextmanager
@@ -320,6 +321,11 @@ def do_run_test(self, test: str, mp: bool) -> TestResult:
diff = file_diff(str(f_reference), str(f_bad))
if diff:
+ if os.environ.get("QEMU_IOTESTS_REGEN", None) is not None:
+ shutil.copyfile(str(f_bad), str(f_reference))
+ print("########################################")
+ print("##### REFERENCE FILE UPDATED #####")
+ print("########################################")
return TestResult(status='fail', elapsed=elapsed,
description=f'output mismatch (see {f_bad})',
diff=diff, casenotrun=casenotrun)
--
2.34.1
next prev parent reply other threads:[~2022-03-04 19:53 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-04 19:35 [PATCH v2 00/12] nbd: enable use of TLS on non-TCP transports and other TLS improvements Daniel P. Berrangé
2022-03-04 19:35 ` [PATCH v2 01/12] crypto: mandate a hostname when checking x509 creds on a client Daniel P. Berrangé
2022-03-04 19:36 ` [PATCH v2 02/12] block: pass desired TLS hostname through from block driver client Daniel P. Berrangé
2022-03-04 19:36 ` [PATCH v2 03/12] block/nbd: support override of hostname for TLS certificate validation Daniel P. Berrangé
2022-03-04 19:36 ` [PATCH v2 04/12] qemu-nbd: add --tls-hostname option " Daniel P. Berrangé
2022-03-04 19:36 ` [PATCH v2 05/12] block/nbd: don't restrict TLS usage to IP sockets Daniel P. Berrangé
2022-03-04 19:36 ` Daniel P. Berrangé [this message]
2022-03-04 19:36 ` [PATCH v2 07/12] tests/qemu-iotests: expand _filter_nbd rules Daniel P. Berrangé
2022-03-04 19:36 ` [PATCH v2 08/12] tests/qemu-iotests: introduce filter for qemu-nbd export list Daniel P. Berrangé
2022-03-04 20:38 ` Eric Blake
2022-03-04 19:36 ` [PATCH v2 09/12] tests/qemu-iotests: convert NBD TLS test to use standard filters Daniel P. Berrangé
2022-03-04 19:36 ` [PATCH v2 10/12] tests/qemu-iotests: validate NBD TLS with hostname mismatch Daniel P. Berrangé
2022-03-04 19:36 ` [PATCH v2 11/12] tests/qemu-iotests: validate NBD TLS with UNIX sockets Daniel P. Berrangé
2022-03-04 21:08 ` Eric Blake
2022-03-04 19:36 ` [PATCH v2 12/12] tests/qemu-iotests: validate NBD TLS with UNIX sockets and PSK Daniel P. Berrangé
2022-03-04 21:27 ` Eric Blake
2022-03-07 16:59 ` Daniel P. Berrangé
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220304193610.3293146-7-berrange@redhat.com \
--to=berrange@redhat.com \
--cc=armbru@redhat.com \
--cc=eblake@redhat.com \
--cc=hreitz@redhat.com \
--cc=kwolf@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=vsementsov@virtuozzo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).