qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Paolo Bonzini <pbonzini@redhat.com>
To: qemu-devel@nongnu.org
Cc: eesposit@redhat.com, kwolf@redhat.com, vsementsov@virtuozzo.com,
	qemu-block@nongnu.org, mreitz@redhat.com
Subject: [PATCH v2 1/5] qemu-iotests: do not buffer the test output
Date: Tue, 23 Mar 2021 19:19:23 +0100	[thread overview]
Message-ID: <20210323181928.311862-2-pbonzini@redhat.com> (raw)
In-Reply-To: <20210323181928.311862-1-pbonzini@redhat.com>

Instead of buffering the test output into a StringIO, patch it on
the fly by wrapping sys.stdout's write method.  This can be
done unconditionally, even if using -d, which makes execute_unittest
a bit simpler.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/qemu-iotests/iotests.py | 68 ++++++++++++++++++++---------------
 1 file changed, 40 insertions(+), 28 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 90d0b62523..0521235030 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -32,7 +32,7 @@
 import sys
 import time
 from typing import (Any, Callable, Dict, Iterable,
-                    List, Optional, Sequence, Tuple, TypeVar)
+                    List, Optional, Sequence, Tuple, Type, TypeVar)
 import unittest
 
 from contextlib import contextmanager
@@ -1271,37 +1271,49 @@ def func_wrapper(*args, **kwargs):
             return func(*args, **kwargs)
     return func_wrapper
 
+# We need to filter out the time taken from the output so that
+# qemu-iotest can reliably diff the results against master output,
+# and hide skipped tests from the reference output.
+
+class ReproducibleTestResult(unittest.TextTestResult):
+    def addSkip(self, test, reason):
+        # Same as TextTestResult, but print dot instead of "s"
+        unittest.TestResult.addSkip(self, test, reason)
+        if self.showAll:
+            self.stream.writeln("skipped {0!r}".format(reason))
+        elif self.dots:
+            self.stream.write(".")
+            self.stream.flush()
+
+class ReproducibleStreamWrapper(object):
+    def __init__(self, stream):
+        self.stream = stream
+
+    def __getattr__(self, attr):
+        if attr in ('stream', '__getstate__'):
+            raise AttributeError(attr)
+        return getattr(self.stream,attr)
+
+    def write(self, arg=None):
+        arg = re.sub(r'Ran (\d+) tests? in [\d.]+s', r'Ran \1 tests', arg)
+        arg = re.sub(r' \(skipped=\d+\)', r'', arg)
+        self.stream.write(arg)
+
+class ReproducibleTestRunner(unittest.TextTestRunner):
+    def __init__(self, stream: Optional[io.TextIOBase] = None,
+                 resultclass: Type = ReproducibleTestResult, *args, **kwargs):
+        rstream = ReproducibleStreamWrapper(stream or sys.stdout)
+        super().__init__(stream=rstream,           # type: ignore
+                         descriptions=True,
+                         resultclass=resultclass,
+                         *args, **kwargs)
+
 def execute_unittest(debug=False):
     """Executes unittests within the calling module."""
 
     verbosity = 2 if debug else 1
-
-    if debug:
-        output = sys.stdout
-    else:
-        # We need to filter out the time taken from the output so that
-        # qemu-iotest can reliably diff the results against master output.
-        output = io.StringIO()
-
-    runner = unittest.TextTestRunner(stream=output, descriptions=True,
-                                     verbosity=verbosity)
-    try:
-        # unittest.main() will use sys.exit(); so expect a SystemExit
-        # exception
-        unittest.main(testRunner=runner)
-    finally:
-        # We need to filter out the time taken from the output so that
-        # qemu-iotest can reliably diff the results against master output.
-        if not debug:
-            out = output.getvalue()
-            out = re.sub(r'Ran (\d+) tests? in [\d.]+s', r'Ran \1 tests', out)
-
-            # Hide skipped tests from the reference output
-            out = re.sub(r'OK \(skipped=\d+\)', 'OK', out)
-            out_first_line, out_rest = out.split('\n', 1)
-            out = out_first_line.replace('s', '.') + '\n' + out_rest
-
-            sys.stderr.write(out)
+    runner = unittest.ReproducibleTestRunner(verbosity=verbosity)
+    unittest.main(testRunner=runner)
 
 def execute_setup_common(supported_fmts: Sequence[str] = (),
                          supported_platforms: Sequence[str] = (),
-- 
2.30.1




  reply	other threads:[~2021-03-23 19:09 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-23 18:19 [PATCH v2 0/5] qemu-iotests: quality of life improvements Paolo Bonzini
2021-03-23 18:19 ` Paolo Bonzini [this message]
2021-03-23 18:54   ` [PATCH v2 1/5] qemu-iotests: do not buffer the test output Vladimir Sementsov-Ogievskiy
2021-03-23 18:57     ` Vladimir Sementsov-Ogievskiy
2021-03-23 18:19 ` [PATCH v2 2/5] qemu-iotests: allow passing unittest.main arguments to the test scripts Paolo Bonzini
2021-03-23 19:02   ` Vladimir Sementsov-Ogievskiy
2021-03-23 19:17   ` Vladimir Sementsov-Ogievskiy
2021-03-23 21:22     ` Paolo Bonzini
2021-03-24  7:34       ` Vladimir Sementsov-Ogievskiy
2021-03-23 18:19 ` [PATCH v2 3/5] qemu-iotests: move command line and environment handling from TestRunner to TestEnv Paolo Bonzini
2021-03-23 18:19 ` [PATCH v2 4/5] qemu-iotests: let "check" spawn an arbitrary test command Paolo Bonzini
2021-03-23 19:12   ` Vladimir Sementsov-Ogievskiy
2021-03-23 21:20     ` Paolo Bonzini
2021-03-24  7:36       ` Vladimir Sementsov-Ogievskiy
2021-03-23 18:19 ` [PATCH v2 5/5] qemu-iotests: fix case of SOCK_DIR already in the environment Paolo Bonzini
2021-03-24 15:17 ` [PATCH v2 0/5] qemu-iotests: quality of life improvements Emanuele Giuseppe Esposito
2021-03-25 18:15 ` Max Reitz
2021-03-26  6:50   ` Paolo Bonzini
2021-03-26 13:40 ` Max Reitz

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210323181928.311862-2-pbonzini@redhat.com \
    --to=pbonzini@redhat.com \
    --cc=eesposit@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=mreitz@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=vsementsov@virtuozzo.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).