All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Paolo Bonzini" <pbonzini@redhat.com>,
	"Pierrick Bouvier" <pierrick.bouvier@oss.qualcomm.com>,
	"Zhao Liu" <zhao1.liu@intel.com>,
	"Peter Maydell" <peter.maydell@linaro.org>,
	"Matthew Rosato" <mjrosato@linux.ibm.com>,
	"Halil Pasic" <pasic@linux.ibm.com>,
	"Philippe Mathieu-Daudé" <philmd@oss.qualcomm.com>,
	"Strahinja Jankovic" <strahinja.p.jankovic@gmail.com>,
	"Niek Linnenbank" <nieklinnenbank@gmail.com>,
	"Daniel P. Berrangé" <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	qemu-s390x@nongnu.org, qemu-arm@nongnu.org,
	"Christian Borntraeger" <borntraeger@linux.ibm.com>,
	qemu-ppc@nongnu.org, "Thomas Huth" <th.huth+qemu@posteo.eu>,
	"Hervé Poussineau" <hpoussin@reactos.org>,
	"Eric Farman" <farman@linux.ibm.com>
Subject: [PATCH 12/12] tests/functional: optionally display download progress
Date: Fri, 20 Nov 2026 12:14:58 +0000	[thread overview]
Message-ID: <20261120121458.3723625-13-berrange@redhat.com> (raw)
In-Reply-To: <20261120121458.3723625-1-berrange@redhat.com>

If "QEMU_TEST_VERBOSE=1" is set then display the file download
progress. This is not enabled by default since meson buffers
output, but if running the test file directly it can be helpful
to see. This requires the 'tqdm' python package to be present
on the system and will have no effect if missing.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 docs/devel/testing/functional.rst   |  6 ++++++
 tests/functional/qemu_test/asset.py | 29 +++++++++++++++++++++++++++--
 2 files changed, 33 insertions(+), 2 deletions(-)

diff --git a/docs/devel/testing/functional.rst b/docs/devel/testing/functional.rst
index 60a427d175d..8aac3ef9508 100644
--- a/docs/devel/testing/functional.rst
+++ b/docs/devel/testing/functional.rst
@@ -329,6 +329,12 @@ default, but the location can be changed by setting the
 To force the test suite to re-download the cache, even if still valid,
 set the ``QEMU_TEST_REFRESH_CACHE`` environment variable.
 
+To see progress information during the downloads, set ``QEMU_TEST_VERBOSE``
+to ``1``. Since meson buffers output and only display it on completion,
+this is only suitable if running the test precaching directly::
+
+    QEMU_TEST_VERBOSE=1 ./run tests/functional/mips/test_malta.py
+
 Skipping tests
 --------------
 
diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
index 7bebdd4d481..aa071ea1960 100644
--- a/tests/functional/qemu_test/asset.py
+++ b/tests/functional/qemu_test/asset.py
@@ -18,9 +18,13 @@
 import urllib.request
 from time import sleep
 from pathlib import Path
-from shutil import copyfileobj
 from urllib.error import HTTPError, URLError
 
+try:
+    from tqdm import tqdm
+except:
+    tqdm = None
+
 
 class AssetError(Exception):
     '''This exception will be raised if an asset is not usable'''
@@ -139,6 +143,14 @@ def _save_time_stamp(self):
         self.cache_file.with_suffix(".stamp").write_text(f"{int(time.time())}")
 
     def _try_to_fetch(self, tmp_cache_file):
+        def copyfileprog(prg, src, dst):
+            rd = src.read
+            wr = dst.write
+            while buf := rd(256 * 1024):
+                wr(buf)
+                if prg is not None:
+                    prg.update(len(buf))
+
         for _retries in range(3):
             try:
                 with tmp_cache_file.open("xb") as dst:
@@ -152,8 +164,21 @@ def _try_to_fetch(self, tmp_cache_file):
                         }
                     )
                     with urllib.request.urlopen(req) as resp:
-                        copyfileobj(resp, dst)
                         length_hdr = resp.getheader("Content-Length")
+                        verbose = os.environ.get("QEMU_TEST_VERBOSE", False)
+                        if tqdm is not None and verbose:
+                            args = {
+                                "desc": os.path.basename(self.url)
+                            }
+                            if length_hdr is not None:
+                                args["total"] = int(length_hdr)
+                            prg = tqdm(**args)
+                            try:
+                                copyfileprog(prg, resp, dst)
+                            finally:
+                                prg.close()
+                        else:
+                            copyfileprog(None, resp, dst)
 
                 # Verify downloaded file size against length metadata, if
                 # available.
-- 
2.56.1



      parent reply	other threads:[~2026-11-20 12:17 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-20 12:14 [PATCH 00/12] tests/functional: misc fixes to functional test suite Daniel P. Berrangé
2026-11-20 12:14 ` [PATCH 01/12] tests/functional: update to newer sunxi kernel package Daniel P. Berrangé
2026-11-20 15:17   ` Thomas Huth
2026-11-20 12:14 ` [PATCH 02/12] tests/functional: update sha digests for tuxrun kernels Daniel P. Berrangé
2026-11-20 12:23   ` Marc-André Lureau
2026-11-20 12:14 ` [PATCH 03/12] tests/functional: update URL to rs6000 firmware Daniel P. Berrangé
2026-11-20 12:14 ` [PATCH 04/12] tests/functional: add Accept header when downloading assets Daniel P. Berrangé
2026-11-20 12:14 ` [PATCH 05/12] tests/functional: check if we are served an Anubis challenge Daniel P. Berrangé
2026-11-20 13:01   ` BALATON Zoltan
2026-11-20 13:06     ` BALATON Zoltan
2026-11-20 12:14 ` [PATCH 06/12] tests/functional: don't write timestamp unless successful Daniel P. Berrangé
2026-11-20 12:14 ` [PATCH 07/12] tests/functional: delete tmp file on interrupt Daniel P. Berrangé
2026-11-20 12:14 ` [PATCH 08/12] tests/functional: honour make jobs in precache Daniel P. Berrangé
2026-11-20 12:14 ` [PATCH 09/12] tests/functional: don't display stack trace on download fail Daniel P. Berrangé
2026-11-20 12:14 ` [PATCH 10/12] tests/functional: always raise AssetError when download fails Daniel P. Berrangé
2026-11-20 12:14 ` [PATCH 11/12] tests/functional: only run pre-cache once per class Daniel P. Berrangé
2026-11-20 12:14 ` Daniel P. Berrangé [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261120121458.3723625-13-berrange@redhat.com \
    --to=berrange@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=borntraeger@linux.ibm.com \
    --cc=farman@linux.ibm.com \
    --cc=hpoussin@reactos.org \
    --cc=mjrosato@linux.ibm.com \
    --cc=mst@redhat.com \
    --cc=nieklinnenbank@gmail.com \
    --cc=pasic@linux.ibm.com \
    --cc=pbonzini@redhat.com \
    --cc=peter.maydell@linaro.org \
    --cc=philmd@oss.qualcomm.com \
    --cc=pierrick.bouvier@oss.qualcomm.com \
    --cc=qemu-arm@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=qemu-ppc@nongnu.org \
    --cc=qemu-s390x@nongnu.org \
    --cc=strahinja.p.jankovic@gmail.com \
    --cc=th.huth+qemu@posteo.eu \
    --cc=zhao1.liu@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.