qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Nicholas Piggin <npiggin@gmail.com>
To: Thomas Huth <thuth@redhat.com>
Cc: "Nicholas Piggin" <npiggin@gmail.com>,
	"Philippe Mathieu-Daudé" <philmd@linaro.org>,
	"Daniel P. Berrangé" <berrange@redhat.com>,
	qemu-devel@nongnu.org
Subject: [PATCH v2 2/3] tests/functional/asset: Verify downloaded size
Date: Wed, 12 Mar 2025 22:25:57 +1000	[thread overview]
Message-ID: <20250312122559.944533-3-npiggin@gmail.com> (raw)
In-Reply-To: <20250312122559.944533-1-npiggin@gmail.com>

If the server provides a Content-Length header, use that to verify the
size of the downloaded file. This catches cases where the connection
terminates early, and gives the opportunity to retry. Without this, the
checksum will likely mismatch and fail without retry.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 tests/functional/qemu_test/asset.py | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
index 27dd839e705..6bbfb9e1cad 100644
--- a/tests/functional/qemu_test/asset.py
+++ b/tests/functional/qemu_test/asset.py
@@ -121,6 +121,20 @@ def fetch(self):
                 with tmp_cache_file.open("xb") as dst:
                     with urllib.request.urlopen(self.url) as resp:
                         copyfileobj(resp, dst)
+                        length_hdr = resp.getheader("Content-Length")
+
+                # Verify downloaded file size against length metadata, if
+                # available.
+                if length_hdr is not None:
+                    length = int(length_hdr)
+                    fsize = tmp_cache_file.stat().st_size
+                    if fsize != length:
+                        self.log.error("Unable to download %s: "
+                                       "connection closed before "
+                                       "transfer complete (%d/%d)",
+                                       self.url, fsize, length)
+                        tmp_cache_file.unlink()
+                        continue
                 break
             except FileExistsError:
                 self.log.debug("%s already exists, "
-- 
2.47.1



  parent reply	other threads:[~2025-03-12 12:26 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-03-12 12:25 [PATCH v2 0/3] tests/functional/asset: improve partial-download handling Nicholas Piggin
2025-03-12 12:25 ` [PATCH v2 1/3] tests/functional/asset: Fail assert fetch when retries are exceeded Nicholas Piggin
2025-03-12 12:27   ` Daniel P. Berrangé
2025-03-12 12:25 ` Nicholas Piggin [this message]
2025-03-12 12:27   ` [PATCH v2 2/3] tests/functional/asset: Verify downloaded size Daniel P. Berrangé
2025-03-12 12:25 ` [PATCH v2 3/3] tests/functional/asset: Add AssetError exception class Nicholas Piggin
2025-03-12 12:29   ` Daniel P. Berrangé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250312122559.944533-3-npiggin@gmail.com \
    --to=npiggin@gmail.com \
    --cc=berrange@redhat.com \
    --cc=philmd@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=thuth@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).