From: Nicholas Piggin <npiggin@gmail.com>
To: Thomas Huth <thuth@redhat.com>
Cc: "Nicholas Piggin" <npiggin@gmail.com>,
"Philippe Mathieu-Daudé" <philmd@linaro.org>,
"Daniel P. Berrangé" <berrange@redhat.com>,
qemu-devel@nongnu.org
Subject: [PATCH 2/3] tests/functional/asset: Verify downloaded size
Date: Wed, 12 Mar 2025 15:17:37 +1000 [thread overview]
Message-ID: <20250312051739.938441-3-npiggin@gmail.com> (raw)
In-Reply-To: <20250312051739.938441-1-npiggin@gmail.com>
If the server provides a Content-Length header, use that to verify the
size of the downloaded file. This catches cases where the connection
terminates early, and gives the opportunity to retry. Without this, the
checksum will likely mismatch and fail without retry.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
tests/functional/qemu_test/asset.py | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
index 6a1c92ffbef..d34e8f5e2ad 100644
--- a/tests/functional/qemu_test/asset.py
+++ b/tests/functional/qemu_test/asset.py
@@ -124,6 +124,22 @@ def fetch(self):
with tmp_cache_file.open("xb") as dst:
with urllib.request.urlopen(self.url) as resp:
copyfileobj(resp, dst)
+ length_hdr = resp.getheader("Content-Length")
+
+ # Verify downloaded file size against length metadata, if
+ # available. dst must be out of scope before testing st_size
+ # because # copyfileobj returns before all buffers are
+ # flushed to filesystem.
+ if length_hdr:
+ length = int(length_hdr)
+ if tmp_cache_file.stat().st_size != length:
+ print("st_size %ld", tmp_cache_file.stat().st_size)
+ self.log.error("Unable to download %s: "
+ "connection closed before "
+ "transfer complete",
+ self.url)
+ tmp_cache_file.unlink()
+ continue
break
except FileExistsError:
self.log.debug("%s already exists, "
--
2.47.1
next prev parent reply other threads:[~2025-03-12 5:18 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-03-12 5:17 [PATCH 0/3] tests/functional/asset: improve partial-download handling Nicholas Piggin
2025-03-12 5:17 ` [PATCH 1/3] tests/functional/asset: Fail assert fetch when retries are exceeded Nicholas Piggin
2025-03-12 6:40 ` Thomas Huth
2025-03-12 8:13 ` Daniel P. Berrangé
2025-03-12 5:17 ` Nicholas Piggin [this message]
2025-03-12 6:49 ` [PATCH 2/3] tests/functional/asset: Verify downloaded size Thomas Huth
2025-03-12 5:17 ` [PATCH 3/3] tests/functional/asset: Add AssetError exception class Nicholas Piggin
2025-03-12 6:56 ` Thomas Huth
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250312051739.938441-3-npiggin@gmail.com \
--to=npiggin@gmail.com \
--cc=berrange@redhat.com \
--cc=philmd@linaro.org \
--cc=qemu-devel@nongnu.org \
--cc=thuth@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).