[PATCH] fetch/wget: rename the partial download on checksum mismatch

[PATCH] fetch/wget: rename the partial download on checksum mismatch

[Ricardo Salveti]

The wget fetcher downloads into a .tmp file which is resumed with
--continue and is shared with the mirrors of the same file. When it
fails checksum verification it is left in place, so MIRRORS reuse the
bad file instead of downloading their own copy and, without a mirror,
every retry fails the same way until it is removed by hand. Rename it
with rename_bad_checksum(), as done for any other file with a bad
checksum, so it can be inspected and the next attempt starts from
scratch. Mirror downloads skip this check and are still handled by the
existing code once moved into place.

Seen with parallel CI runners sharing DL_DIR on Amazon FSx for OpenZFS
mounted over NFSv4.2. What corrupts the file there is not addressed,
only the recovery on retry. Tested with the new tests, which fail
without the fix.

AI-Generated: Uses Claude Code (Claude Fable 5.1)
Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com>
---
 lib/bb/fetch/__init__.py | 11 ++++++---
 lib/bb/fetch/wget.py     |  8 +++++-
 lib/bb/tests/fetch.py    | 53 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 67 insertions(+), 5 deletions(-)

diff --git a/lib/bb/fetch/__init__.py b/lib/bb/fetch/__init__.py
index fb6278439..eb3483a48 100644
--- a/lib/bb/fetch/__init__.py
+++ b/lib/bb/fetch/__init__.py
@@ -1079,7 +1079,7 @@ def build_mirroruris(origud, mirrors, ld):
 
     return uris, uds
 
-def rename_bad_checksum(ud, suffix):
+def rename_bad_checksum(ud, suffix, localpath=None):
     """
     Renames files to have suffix from parameter
     """
@@ -1087,10 +1087,13 @@ def rename_bad_checksum(ud, suffix):
     if ud.localpath is None:
         return
 
+    if localpath is None:
+        localpath = ud.localpath
+
     new_localpath = "%s_bad-checksum_%s" % (ud.localpath, suffix)
-    bb.warn("Renaming %s to %s" % (ud.localpath, new_localpath))
-    if not bb.utils.movefile(ud.localpath, new_localpath):
-        bb.warn("Renaming %s to %s failed, grep movefile in log.do_fetch to see why" % (ud.localpath, new_localpath))
+    bb.warn("Renaming %s to %s" % (localpath, new_localpath))
+    if not bb.utils.movefile(localpath, new_localpath):
+        bb.warn("Renaming %s to %s failed, grep movefile in log.do_fetch to see why" % (localpath, new_localpath))
 
 
 def try_mirror_url(fetch, origud, ud, ld, check = False):
diff --git a/lib/bb/fetch/wget.py b/lib/bb/fetch/wget.py
index 972c84048..6259c1245 100644
--- a/lib/bb/fetch/wget.py
+++ b/lib/bb/fetch/wget.py
@@ -167,7 +167,13 @@ class Wget(FetchMethod):
         # Try and verify any checksum now, meaning if it isn't correct, we don't remove the
         # original file, which might be a race (imagine two recipes referencing the same
         # source, one with an incorrect checksum)
-        bb.fetch.verify_checksum(ud, d, localpath=localpath, fatal_nochecksum=False)
+        try:
+            bb.fetch.verify_checksum(ud, d, localpath=localpath, fatal_nochecksum=False)
+        except bb.fetch.ChecksumError as e:
+            # The download is resumed with --continue, so a bad file left in place
+            # would be reused by the mirrors and by every later attempt
+            bb.fetch.rename_bad_checksum(ud, e.checksum, localpath=localpath)
+            raise
 
         # Remove the ".tmp" and move the file into position atomically
         # Our lock prevents multiple writers but mirroring code may grab incomplete files
diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py
index 71b0a63fe..9627b3b1b 100644
--- a/lib/bb/tests/fetch.py
+++ b/lib/bb/tests/fetch.py
@@ -1706,6 +1706,59 @@ class FetchLatestVersionTest(FetcherTest):
                 r = bb.utils.vercmp_string(verstring, v_larger)
                 self.assertTrue(r == -1, msg="Package %s, version: %s <= %s" % (k[0], v_larger, verstring))
 
+class WgetChecksumTest(FetcherTest):
+    content = b"bitbake wget test data\n" * 64
+    # As large as the real file, so wget --continue has nothing left to fetch
+    # and keeps it as is
+    corrupt = b"x" * len(content)
+
+    def setUp(self):
+        super().setUp()
+        self.serverdir = os.path.join(self.tempdir, "server")
+        for subdir in ["upstream", "mirror"]:
+            os.makedirs(os.path.join(self.serverdir, subdir))
+        self.server = HTTPService(self.serverdir, host="127.0.0.1")
+        self.server.start()
+        self.baseurl = "http://127.0.0.1:%s" % self.server.port
+        self.url = "%s/upstream/test.bin;sha256sum=%s" % (
+            self.baseurl, hashlib.sha256(self.content).hexdigest())
+
+    def tearDown(self):
+        self.server.stop()
+        super().tearDown()
+
+    def write(self, path, data):
+        with open(path, "wb") as f:
+            f.write(data)
+
+    def assertDownloaded(self):
+        with open(os.path.join(self.dldir, "test.bin"), "rb") as f:
+            self.assertEqual(f.read(), self.content)
+        self.assertFalse(os.path.exists(os.path.join(self.dldir, "test.bin.tmp")))
+        self.assertTrue(os.path.exists(os.path.join(self.dldir,
+            "test.bin_bad-checksum_%s" % hashlib.sha256(self.corrupt).hexdigest())))
+
+    def test_wget_corrupt_partial_download(self):
+        self.write(os.path.join(self.serverdir, "upstream", "test.bin"), self.content)
+        self.write(os.path.join(self.dldir, "test.bin.tmp"), self.corrupt)
+
+        fetcher = bb.fetch.Fetch([self.url], self.d)
+        with self.assertRaises(bb.fetch.FetchError):
+            fetcher.download()
+        self.assertFalse(os.path.exists(os.path.join(self.dldir, "test.bin.tmp")))
+
+        fetcher.download()
+        self.assertDownloaded()
+
+    def test_wget_mirror_after_checksum_failure(self):
+        self.write(os.path.join(self.serverdir, "upstream", "test.bin"), self.corrupt)
+        self.write(os.path.join(self.serverdir, "mirror", "test.bin"), self.content)
+        self.d.setVar("MIRRORS", "%s/upstream/ %s/mirror/" % (self.baseurl, self.baseurl))
+
+        fetcher = bb.fetch.Fetch([self.url], self.d)
+        fetcher.download()
+        self.assertDownloaded()
+
 class FetchCheckStatusTest(FetcherTest):
     test_wget_uris = ["https://downloads.yoctoproject.org/releases/sato/sato-engine-0.1.tar.gz",
                       "https://downloads.yoctoproject.org/releases/sato/sato-engine-0.2.tar.gz",
-- 
2.43.0

Re: [bitbake-devel] [PATCH] fetch/wget: rename the partial download on checksum mismatch

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 752 bytes --]

On Tue, 29 Sept 2026 at 03:28, Ricardo Salveti via lists.openembedded.org
<ricardo.salveti=oss.qualcomm.com@lists.openembedded.org> wrote:

> Seen with parallel CI runners sharing DL_DIR on Amazon FSx for OpenZFS
> mounted over NFSv4.2. What corrupts the file there is not addressed,
> only the recovery on retry. Tested with the new tests, which fail
> without the fix.
>

What is the plan for addressing the actual issue? Has it been reported? Is
it investigated?

The problem with such workarounds is that they remove all incentives to fix
the actual issue. Worse yet, they can obscure that the issue exists to
begin with, or that it has appeared on a system update where previously
there wasn't any (e.g. a new bug).

I'm firmly unconvinced.

Alex

[-- Attachment #2: Type: text/html, Size: 1540 bytes --]

Re: [bitbake-devel] [PATCH] fetch/wget: rename the partial download on checksum mismatch

[Ricardo Salveti]

On Tue, Sep 29, 2026 at 1:48 PM Alexander Kanavin
<alex.kanavin@gmail.com> wrote:
>
> On Tue, 29 Sept 2026 at 03:28, Ricardo Salveti via lists.openembedded.org <ricardo.salveti=oss.qualcomm.com@lists.openembedded.org> wrote:
>>
>> Seen with parallel CI runners sharing DL_DIR on Amazon FSx for OpenZFS
>> mounted over NFSv4.2. What corrupts the file there is not addressed,
>> only the recovery on retry. Tested with the new tests, which fail
>> without the fix.
>
> What is the plan for addressing the actual issue? Has it been reported? Is it investigated?

Yes, it was reported to AWS and we are following up with them in
parallel (about how the file gets corrupted on that storage).

> The problem with such workarounds is that they remove all incentives to fix the actual issue. Worse yet, they can obscure that the issue exists to begin with, or that it has appeared on a system update where previously there wasn't any (e.g. a new bug).

I agree with the concern in general, but I don't think it applies here.

The fetcher issue is valid on its own, the shared storage is just how
we found it. Once a .tmp file has the wrong content, for whatever
reason, it is never discarded: the next download of the same file
resumes it with --continue, mirrors included, so it aborts every
future build until someone removes it by hand. This reproduces on
current master with a local http server and nothing else:

 - upstream serves a file which doesn't match the recipe checksum
(e.g. a tarball regenerated upstream) and a mirror has the right one:
the  fetch fails, as the mirror download reuses the .tmp file left by
the upstream one instead of fetching its own copy. With the patch the
mirror is used, which is what the fetcher intends ("will attempt other
sources if available").
 - a bad .tmp file and no mirror: every attempt fails. With the patch
the first attempt still fails and the next one works.

Both are covered by the tests added in the patch.

It also doesn't hide anything. The checksum failure is still reported,
and still fails the task when there is no mirror to fall back to, and
the bad file is kept as <file>_bad-checksum_<sum> with a warning, same
as the fetcher already does for any other file with a bad checksum.

Today the only way out is to delete the .tmp file, which is what
destroys the evidence.

Thanks,

Ricardo

Re: [bitbake-devel] [PATCH] fetch/wget: rename the partial download on checksum mismatch

[Richard Purdie]

Hi Ricardo,

On Tue, 2026-09-29 at 01:28 +0000, Ricardo Salveti via lists.openembedded.org wrote:
> diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py
> index 71b0a63fe..9627b3b1b 100644
> --- a/lib/bb/tests/fetch.py
> +++ b/lib/bb/tests/fetch.py
> @@ -1706,6 +1706,59 @@ class FetchLatestVersionTest(FetcherTest):
>                  r = bb.utils.vercmp_string(verstring, v_larger)
>                  self.assertTrue(r == -1, msg="Package %s, version: %s <= %s" % (k[0], v_larger, verstring))
>  
> +class WgetChecksumTest(FetcherTest):
> +    content = b"bitbake wget test data\n" * 64
> +    # As large as the real file, so wget --continue has nothing left to fetch
> +    # and keeps it as is
> +    corrupt = b"x" * len(content)
> +
> +    def setUp(self):
> +        super().setUp()
> +        self.serverdir = os.path.join(self.tempdir, "server")
> +        for subdir in ["upstream", "mirror"]:
> +            os.makedirs(os.path.join(self.serverdir, subdir))
> +        self.server = HTTPService(self.serverdir, host="127.0.0.1")
> +        self.server.start()
> +        self.baseurl = "http://127.0.0.1:%s" % self.server.port
> +        self.url = "%s/upstream/test.bin;sha256sum=%s" % (
> +            self.baseurl, hashlib.sha256(self.content).hexdigest())
> +
> +    def tearDown(self):
> +        self.server.stop()
> +        super().tearDown()
> +
> +    def write(self, path, data):
> +        with open(path, "wb") as f:
> +            f.write(data)
> +
> +    def assertDownloaded(self):
> +        with open(os.path.join(self.dldir, "test.bin"), "rb") as f:
> +            self.assertEqual(f.read(), self.content)
> +        self.assertFalse(os.path.exists(os.path.join(self.dldir, "test.bin.tmp")))
> +        self.assertTrue(os.path.exists(os.path.join(self.dldir,
> +            "test.bin_bad-checksum_%s" % hashlib.sha256(self.corrupt).hexdigest())))
> +
> +    def test_wget_corrupt_partial_download(self):
> +        self.write(os.path.join(self.serverdir, "upstream", "test.bin"), self.content)
> +        self.write(os.path.join(self.dldir, "test.bin.tmp"), self.corrupt)
> +
> +        fetcher = bb.fetch.Fetch([self.url], self.d)
> +        with self.assertRaises(bb.fetch.FetchError):
> +            fetcher.download()
> +        self.assertFalse(os.path.exists(os.path.join(self.dldir, "test.bin.tmp")))
> +
> +        fetcher.download()
> +        self.assertDownloaded()
> +
> +    def test_wget_mirror_after_checksum_failure(self):
> +        self.write(os.path.join(self.serverdir, "upstream", "test.bin"), self.corrupt)
> +        self.write(os.path.join(self.serverdir, "mirror", "test.bin"), self.content)
> +        self.d.setVar("MIRRORS", "%s/upstream/ %s/mirror/" % (self.baseurl, self.baseurl))
> +
> +        fetcher = bb.fetch.Fetch([self.url], self.d)
> +        fetcher.download()
> +        self.assertDownloaded()
> +
>  class FetchCheckStatusTest(FetcherTest):
>      test_wget_uris = ["https://downloads.yoctoproject.org/releases/sato/sato-engine-0.1.tar.gz",
>                        "https://downloads.yoctoproject.org/releases/sato/sato-engine-0.2.tar.gz",
> 

I merged this but the testcase is intermittently failing, e.g.:

https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/4786/steps/14/logs/stdio

which I'm struggling to understand why. You can see the rename in the
logs so it implies the checksum didn't match the expected value, or the
file was somehow deleted...

Cheers,

Richard

Re: [bitbake-devel] [PATCH] fetch/wget: rename the partial download on checksum mismatch

[Richard Purdie]

On Fri, 2026-10-02 at 10:06 +0100, Richard Purdie via lists.openembedded.org wrote:
> Hi Ricardo,
> 
> On Tue, 2026-09-29 at 01:28 +0000, Ricardo Salveti via lists.openembedded.org wrote:
> > diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py
> > index 71b0a63fe..9627b3b1b 100644
> > --- a/lib/bb/tests/fetch.py
> > +++ b/lib/bb/tests/fetch.py
> > @@ -1706,6 +1706,59 @@ class FetchLatestVersionTest(FetcherTest):
> >                  r = bb.utils.vercmp_string(verstring, v_larger)
> >                  self.assertTrue(r == -1, msg="Package %s, version: %s <= %s" % (k[0], v_larger, verstring))
> >  
> > +class WgetChecksumTest(FetcherTest):
> > +    content = b"bitbake wget test data\n" * 64
> > +    # As large as the real file, so wget --continue has nothing left to fetch
> > +    # and keeps it as is
> > +    corrupt = b"x" * len(content)
> > +
> > +    def setUp(self):
> > +        super().setUp()
> > +        self.serverdir = os.path.join(self.tempdir, "server")
> > +        for subdir in ["upstream", "mirror"]:
> > +            os.makedirs(os.path.join(self.serverdir, subdir))
> > +        self.server = HTTPService(self.serverdir, host="127.0.0.1")
> > +        self.server.start()
> > +        self.baseurl = "http://127.0.0.1:%s" % self.server.port
> > +        self.url = "%s/upstream/test.bin;sha256sum=%s" % (
> > +            self.baseurl, hashlib.sha256(self.content).hexdigest())
> > +
> > +    def tearDown(self):
> > +        self.server.stop()
> > +        super().tearDown()
> > +
> > +    def write(self, path, data):
> > +        with open(path, "wb") as f:
> > +            f.write(data)
> > +
> > +    def assertDownloaded(self):
> > +        with open(os.path.join(self.dldir, "test.bin"), "rb") as f:
> > +            self.assertEqual(f.read(), self.content)
> > +        self.assertFalse(os.path.exists(os.path.join(self.dldir, "test.bin.tmp")))
> > +        self.assertTrue(os.path.exists(os.path.join(self.dldir,
> > +            "test.bin_bad-checksum_%s" % hashlib.sha256(self.corrupt).hexdigest())))
> > +
> > +    def test_wget_corrupt_partial_download(self):
> > +        self.write(os.path.join(self.serverdir, "upstream", "test.bin"), self.content)
> > +        self.write(os.path.join(self.dldir, "test.bin.tmp"), self.corrupt)
> > +
> > +        fetcher = bb.fetch.Fetch([self.url], self.d)
> > +        with self.assertRaises(bb.fetch.FetchError):
> > +            fetcher.download()
> > +        self.assertFalse(os.path.exists(os.path.join(self.dldir, "test.bin.tmp")))
> > +
> > +        fetcher.download()
> > +        self.assertDownloaded()
> > +
> > +    def test_wget_mirror_after_checksum_failure(self):
> > +        self.write(os.path.join(self.serverdir, "upstream", "test.bin"), self.corrupt)
> > +        self.write(os.path.join(self.serverdir, "mirror", "test.bin"), self.content)
> > +        self.d.setVar("MIRRORS", "%s/upstream/ %s/mirror/" % (self.baseurl, self.baseurl))
> > +
> > +        fetcher = bb.fetch.Fetch([self.url], self.d)
> > +        fetcher.download()
> > +        self.assertDownloaded()
> > +
> >  class FetchCheckStatusTest(FetcherTest):
> >      test_wget_uris = ["https://downloads.yoctoproject.org/releases/sato/sato-engine-0.1.tar.gz",
> >                        "https://downloads.yoctoproject.org/releases/sato/sato-engine-0.2.tar.gz",
> > 
> 
> I merged this but the testcase is intermittently failing, e.g.:
> 
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/4786/steps/14/logs/stdio
> 
> which I'm struggling to understand why. You can see the rename in the
> logs so it implies the checksum didn't match the expected value, or the
> file was somehow deleted...

I realised this is only happening on fedora45, which we don't have in
main testing yet as we're still validating it works ok, there are some
issues there. I found the checksum of the bad-checksum file was clearly
from. From the debug output I then added:

1472
total 12
drwxr-xr-x. 2 pokybuild pokybuild  100 Oct  2 09:16 .
drwx------. 5 pokybuild pokybuild  100 Oct  2 09:16 ..
-rw-r--r--. 1 pokybuild pokybuild 1472 Oct  2 09:16 test.bin
-rw-r--r--. 1 pokybuild pokybuild 2944 Oct  2 09:16 test.bin_bad-checksum_4d10580369033e7582482ddb05c76f38df07ad7daac927cd515909e564e26269
-rw-r--r--. 1 pokybuild pokybuild  545 Oct  2 09:16 test.bin.done
total 4
drwxr-xr-x. 2 pokybuild pokybuild   60 Oct  2 09:16 .
drwxr-xr-x. 4 pokybuild pokybuild   80 Oct  2 09:16 ..
-rw-r--r--. 1 pokybuild pokybuild 1472 Oct  2 09:16 test.bin

which shows the bad-checksum file appears to actually have two copies
of the downloaded data based on the file size. That seems odd.

$ wget --version
GNU Wget2 2.2.1 - multithreaded metalink/file/website downloader

+digest +https +ssl/gnutls +ipv6 +iri +large-file +nls -ntlm -opie +psl -hsts
+iconv +idn2 +zlib -lzma +brotlidec +zstd -bzip2 -lzip +http2 +gpgme

A bug in wget or a bug in httpserver in 3.15?

Cheers,

Richard

Re: [bitbake-devel] [PATCH] fetch/wget: rename the partial download on checksum mismatch

[Ricardo Salveti]

On Fri, Oct 2, 2026 at 11:20 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Fri, 2026-10-02 at 10:06 +0100, Richard Purdie via lists.openembedded.org wrote:
> > Hi Ricardo,
> >
> > On Tue, 2026-09-29 at 01:28 +0000, Ricardo Salveti via lists.openembedded.org wrote:
> > > diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py
> > > index 71b0a63fe..9627b3b1b 100644
> > > --- a/lib/bb/tests/fetch.py
> > > +++ b/lib/bb/tests/fetch.py
> > > @@ -1706,6 +1706,59 @@ class FetchLatestVersionTest(FetcherTest):
> > >                  r = bb.utils.vercmp_string(verstring, v_larger)
> > >                  self.assertTrue(r == -1, msg="Package %s, version: %s <= %s" % (k[0], v_larger, verstring))
> > >
> > > +class WgetChecksumTest(FetcherTest):
> > > +    content = b"bitbake wget test data\n" * 64
> > > +    # As large as the real file, so wget --continue has nothing left to fetch
> > > +    # and keeps it as is
> > > +    corrupt = b"x" * len(content)
> > > +
> > > +    def setUp(self):
> > > +        super().setUp()
> > > +        self.serverdir = os.path.join(self.tempdir, "server")
> > > +        for subdir in ["upstream", "mirror"]:
> > > +            os.makedirs(os.path.join(self.serverdir, subdir))
> > > +        self.server = HTTPService(self.serverdir, host="127.0.0.1")
> > > +        self.server.start()
> > > +        self.baseurl = "http://127.0.0.1:%s" % self.server.port
> > > +        self.url = "%s/upstream/test.bin;sha256sum=%s" % (
> > > +            self.baseurl, hashlib.sha256(self.content).hexdigest())
> > > +
> > > +    def tearDown(self):
> > > +        self.server.stop()
> > > +        super().tearDown()
> > > +
> > > +    def write(self, path, data):
> > > +        with open(path, "wb") as f:
> > > +            f.write(data)
> > > +
> > > +    def assertDownloaded(self):
> > > +        with open(os.path.join(self.dldir, "test.bin"), "rb") as f:
> > > +            self.assertEqual(f.read(), self.content)
> > > +        self.assertFalse(os.path.exists(os.path.join(self.dldir, "test.bin.tmp")))
> > > +        self.assertTrue(os.path.exists(os.path.join(self.dldir,
> > > +            "test.bin_bad-checksum_%s" % hashlib.sha256(self.corrupt).hexdigest())))
> > > +
> > > +    def test_wget_corrupt_partial_download(self):
> > > +        self.write(os.path.join(self.serverdir, "upstream", "test.bin"), self.content)
> > > +        self.write(os.path.join(self.dldir, "test.bin.tmp"), self.corrupt)
> > > +
> > > +        fetcher = bb.fetch.Fetch([self.url], self.d)
> > > +        with self.assertRaises(bb.fetch.FetchError):
> > > +            fetcher.download()
> > > +        self.assertFalse(os.path.exists(os.path.join(self.dldir, "test.bin.tmp")))
> > > +
> > > +        fetcher.download()
> > > +        self.assertDownloaded()
> > > +
> > > +    def test_wget_mirror_after_checksum_failure(self):
> > > +        self.write(os.path.join(self.serverdir, "upstream", "test.bin"), self.corrupt)
> > > +        self.write(os.path.join(self.serverdir, "mirror", "test.bin"), self.content)
> > > +        self.d.setVar("MIRRORS", "%s/upstream/ %s/mirror/" % (self.baseurl, self.baseurl))
> > > +
> > > +        fetcher = bb.fetch.Fetch([self.url], self.d)
> > > +        fetcher.download()
> > > +        self.assertDownloaded()
> > > +
> > >  class FetchCheckStatusTest(FetcherTest):
> > >      test_wget_uris = ["https://downloads.yoctoproject.org/releases/sato/sato-engine-0.1.tar.gz",
> > >                        "https://downloads.yoctoproject.org/releases/sato/sato-engine-0.2.tar.gz",
> > >
> >
> > I merged this but the testcase is intermittently failing, e.g.:
> >
> > https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/4786/steps/14/logs/stdio
> >
> > which I'm struggling to understand why. You can see the rename in the
> > logs so it implies the checksum didn't match the expected value, or the
> > file was somehow deleted...
>
> I realised this is only happening on fedora45, which we don't have in
> main testing yet as we're still validating it works ok, there are some
> issues there. I found the checksum of the bad-checksum file was clearly
> from. From the debug output I then added:
>
> 1472
> total 12
> drwxr-xr-x. 2 pokybuild pokybuild  100 Oct  2 09:16 .
> drwx------. 5 pokybuild pokybuild  100 Oct  2 09:16 ..
> -rw-r--r--. 1 pokybuild pokybuild 1472 Oct  2 09:16 test.bin
> -rw-r--r--. 1 pokybuild pokybuild 2944 Oct  2 09:16 test.bin_bad-checksum_4d10580369033e7582482ddb05c76f38df07ad7daac927cd515909e564e26269
> -rw-r--r--. 1 pokybuild pokybuild  545 Oct  2 09:16 test.bin.done
> total 4
> drwxr-xr-x. 2 pokybuild pokybuild   60 Oct  2 09:16 .
> drwxr-xr-x. 4 pokybuild pokybuild   80 Oct  2 09:16 ..
> -rw-r--r--. 1 pokybuild pokybuild 1472 Oct  2 09:16 test.bin
>
> which shows the bad-checksum file appears to actually have two copies
> of the downloaded data based on the file size. That seems odd.
>
> $ wget --version
> GNU Wget2 2.2.1 - multithreaded metalink/file/website downloader
>
> +digest +https +ssl/gnutls +ipv6 +iri +large-file +nls -ntlm -opie +psl -hsts
> +iconv +idn2 +zlib -lzma +brotlidec +zstd -bzip2 -lzip +http2 +gpgme
>
> A bug in wget or a bug in httpserver in 3.15?

Yeah, this seems to be a different behavior in wget2 2.2.1, it ends up
rewriting the file with a successful fetch, let me send another fix.

Thanks,

Ricardo

[PATCH] tests/fetch: don't rely on wget --continue in the corrupt partial test

[Ricardo Salveti]

test_wget_corrupt_partial_download expected wget --continue to leave an
existing .tmp file as large as the remote one alone when the server
ignores Range, which is what GNU wget 1.21.4 does. wget2 2.1.0 rewrites
the file instead, so the fetch succeeds, and wget2 2.2.1 (Fedora 45)
appends the whole download to it, so the bad file ends up under another
checksum than the one the test looks for. Don't run wget for that first
fetch, so the fetcher sees the partial download as it was left,
whichever wget is in use.

Tested with wget 1.21.4, wget2 2.1.0 and wget2 2.2.1 (Fedora 45). The
test still fails without the fetcher fix.

Fixes: 37b9c56ae3b2 ("fetch/wget: rename the partial download on checksum mismatch")
Reported-by: Richard Purdie <richard.purdie@linuxfoundation.org>
AI-Generated: Uses Claude Code (Claude Opus 5.5)
Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com>
---
 lib/bb/tests/fetch.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py
index 9627b3b1b..0f29cdf11 100644
--- a/lib/bb/tests/fetch.py
+++ b/lib/bb/tests/fetch.py
@@ -1708,8 +1708,6 @@ class FetchLatestVersionTest(FetcherTest):
 
 class WgetChecksumTest(FetcherTest):
     content = b"bitbake wget test data\n" * 64
-    # As large as the real file, so wget --continue has nothing left to fetch
-    # and keeps it as is
     corrupt = b"x" * len(content)
 
     def setUp(self):
@@ -1743,8 +1741,10 @@ class WgetChecksumTest(FetcherTest):
         self.write(os.path.join(self.dldir, "test.bin.tmp"), self.corrupt)
 
         fetcher = bb.fetch.Fetch([self.url], self.d)
-        with self.assertRaises(bb.fetch.FetchError):
-            fetcher.download()
+        # Skip wget, so the fetcher verifies the file seeded above as is
+        with unittest.mock.patch.object(bb.fetch.wget.Wget, "_runwget"):
+            with self.assertRaises(bb.fetch.FetchError):
+                fetcher.download()
         self.assertFalse(os.path.exists(os.path.join(self.dldir, "test.bin.tmp")))
 
         fetcher.download()
-- 
2.43.0

Re: [bitbake-devel] [PATCH] tests/fetch: don't rely on wget --continue in the corrupt partial test

[Yoann Congal]

On Tue Oct 6, 2026 at 11:33 PM CEST, Ricardo Salveti via lists.openembedded.org wrote:
> test_wget_corrupt_partial_download expected wget --continue to leave an
> existing .tmp file as large as the remote one alone when the server
> ignores Range, which is what GNU wget 1.21.4 does. wget2 2.1.0 rewrites
> the file instead, so the fetch succeeds, and wget2 2.2.1 (Fedora 45)
> appends the whole download to it, so the bad file ends up under another
> checksum than the one the test looks for. Don't run wget for that first
> fetch, so the fetcher sees the partial download as it was left,
> whichever wget is in use.
>
> Tested with wget 1.21.4, wget2 2.1.0 and wget2 2.2.1 (Fedora 45). The
> test still fails without the fetcher fix.
>
> Fixes: 37b9c56ae3b2 ("fetch/wget: rename the partial download on checksum mismatch")
> Reported-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> AI-Generated: Uses Claude Code (Claude Opus 5.5)
> Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com>
> ---
>  lib/bb/tests/fetch.py | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)

Hello,

Do you think this could be a fix for
https://bugzilla.yoctoproject.org/show_bug.cgi?id=16153 ?
(previous patch definitely sound related)

Thanks!
-- 
Yoann Congal
Smile ECS

Re: [bitbake-devel] [PATCH] fetch/wget: rename the partial download on checksum mismatch

[Mathieu Dubois-Briand]

On Tue Sep 29, 2026 at 3:28 AM CEST, Ricardo Salveti via lists.openembedded.org wrote:
> The wget fetcher downloads into a .tmp file which is resumed with
> --continue and is shared with the mirrors of the same file. When it
> fails checksum verification it is left in place, so MIRRORS reuse the
> bad file instead of downloading their own copy and, without a mirror,
> every retry fails the same way until it is removed by hand. Rename it
> with rename_bad_checksum(), as done for any other file with a bad
> checksum, so it can be inspected and the next attempt starts from
> scratch. Mirror downloads skip this check and are still handled by the
> existing code once moved into place.
>
> Seen with parallel CI runners sharing DL_DIR on Amazon FSx for OpenZFS
> mounted over NFSv4.2. What corrupts the file there is not addressed,
> only the recovery on retry. Tested with the new tests, which fail
> without the fix.
>
> AI-Generated: Uses Claude Code (Claude Fable 5.1)
> Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com>
> ---

Hi Ricardo,

I suspect this patch is responsible of intermittent failures we can see
during test_wget_corrupt_partial_download selftest, only on Fedora
workers.

https://bugzilla.yoctoproject.org/show_bug.cgi?id=16445

Do you have any clue about what might be wrong here?

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [bitbake-devel] [PATCH] tests/fetch: don't rely on wget --continue in the corrupt partial test

[Ricardo Salveti]

Hi,

On Wed, Oct 7, 2026 at 12:50 AM Yoann Congal <yoann.congal@smile.fr> wrote:
>
> On Tue Oct 6, 2026 at 11:33 PM CEST, Ricardo Salveti via lists.openembedded.org wrote:
> > test_wget_corrupt_partial_download expected wget --continue to leave an
> > existing .tmp file as large as the remote one alone when the server
> > ignores Range, which is what GNU wget 1.21.4 does. wget2 2.1.0 rewrites
> > the file instead, so the fetch succeeds, and wget2 2.2.1 (Fedora 45)
> > appends the whole download to it, so the bad file ends up under another
> > checksum than the one the test looks for. Don't run wget for that first
> > fetch, so the fetcher sees the partial download as it was left,
> > whichever wget is in use.
> >
> > Tested with wget 1.21.4, wget2 2.1.0 and wget2 2.2.1 (Fedora 45). The
> > test still fails without the fetcher fix.
> >
> > Fixes: 37b9c56ae3b2 ("fetch/wget: rename the partial download on checksum mismatch")
> > Reported-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > AI-Generated: Uses Claude Code (Claude Opus 5.5)
> > Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com>
> > ---
> >  lib/bb/tests/fetch.py | 8 ++++----
> >  1 file changed, 4 insertions(+), 4 deletions(-)
>
> Hello,
>
> Do you think this could be a fix for
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=16153 ?
> (previous patch definitely sound related)

Yes, the merged one (37b9c56ae) fixes the sequence described in the
bug: the upstream download fails its checksum, and the mirror is then
fetched with --continue on top of the .tmp left by it, so a bigger
mirror file ends up garbled (corrupt head, mirror tail). With
37b9c56ae the .tmp is renamed to <file>_bad-checksum_<sum> as soon as
the checksum fails, so the mirror download starts from scratch and
there is nothing for --continue to resume.

Having issues with my account in bugzilla, would you mind updating the bug?

Thanks!

Ricardo

Re: [bitbake-devel] [PATCH] fetch/wget: rename the partial download on checksum mismatch

[Ricardo Salveti]

Hi,

On Thu, Oct 8, 2026 at 8:59 AM Mathieu Dubois-Briand
<mathieu.dubois-briand@bootlin.com> wrote:
>
> On Tue Sep 29, 2026 at 3:28 AM CEST, Ricardo Salveti via lists.openembedded.org wrote:
> > The wget fetcher downloads into a .tmp file which is resumed with
> > --continue and is shared with the mirrors of the same file. When it
> > fails checksum verification it is left in place, so MIRRORS reuse the
> > bad file instead of downloading their own copy and, without a mirror,
> > every retry fails the same way until it is removed by hand. Rename it
> > with rename_bad_checksum(), as done for any other file with a bad
> > checksum, so it can be inspected and the next attempt starts from
> > scratch. Mirror downloads skip this check and are still handled by the
> > existing code once moved into place.
> >
> > Seen with parallel CI runners sharing DL_DIR on Amazon FSx for OpenZFS
> > mounted over NFSv4.2. What corrupts the file there is not addressed,
> > only the recovery on retry. Tested with the new tests, which fail
> > without the fix.
> >
> > AI-Generated: Uses Claude Code (Claude Fable 5.1)
> > Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com>
> > ---
>
> Hi Ricardo,
>
> I suspect this patch is responsible of intermittent failures we can see
> during test_wget_corrupt_partial_download selftest, only on Fedora
> workers.
>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=16445
>
> Do you have any clue about what might be wrong here?

Should be fixed with
https://patchwork.yoctoproject.org/project/bitbake/patch/20261006213334.3360919-1-ricardo.salveti@oss.qualcomm.com/,
mind giving it a try?

Thanks,

Ricardo

Re: [bitbake-devel] [PATCH] tests/fetch: don't rely on wget --continue in the corrupt partial test

[Yoann Congal]

On Thu Oct 8, 2026 at 4:07 PM CEST, Ricardo Salveti wrote:
> Hi,
>
> On Wed, Oct 7, 2026 at 12:50 AM Yoann Congal <yoann.congal@smile.fr> wrote:
>>
>> On Tue Oct 6, 2026 at 11:33 PM CEST, Ricardo Salveti via lists.openembedded.org wrote:
>> > test_wget_corrupt_partial_download expected wget --continue to leave an
>> > existing .tmp file as large as the remote one alone when the server
>> > ignores Range, which is what GNU wget 1.21.4 does. wget2 2.1.0 rewrites
>> > the file instead, so the fetch succeeds, and wget2 2.2.1 (Fedora 45)
>> > appends the whole download to it, so the bad file ends up under another
>> > checksum than the one the test looks for. Don't run wget for that first
>> > fetch, so the fetcher sees the partial download as it was left,
>> > whichever wget is in use.
>> >
>> > Tested with wget 1.21.4, wget2 2.1.0 and wget2 2.2.1 (Fedora 45). The
>> > test still fails without the fetcher fix.
>> >
>> > Fixes: 37b9c56ae3b2 ("fetch/wget: rename the partial download on checksum mismatch")
>> > Reported-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>> > AI-Generated: Uses Claude Code (Claude Opus 5.5)
>> > Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com>
>> > ---
>> >  lib/bb/tests/fetch.py | 8 ++++----
>> >  1 file changed, 4 insertions(+), 4 deletions(-)
>>
>> Hello,
>>
>> Do you think this could be a fix for
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=16153 ?
>> (previous patch definitely sound related)
>
> Yes, the merged one (37b9c56ae) fixes the sequence described in the
> bug: the upstream download fails its checksum, and the mirror is then
> fetched with --continue on top of the .tmp left by it, so a bigger
> mirror file ends up garbled (corrupt head, mirror tail). With
> 37b9c56ae the .tmp is renamed to <file>_bad-checksum_<sum> as soon as
> the checksum fails, so the mirror download starts from scratch and
> there is nothing for --continue to resume.
>
> Having issues with my account in bugzilla, would you mind updating the bug?

I've put links in the bug so we won't lose the relation to this patch.
But, I'd rather wait for the followup "tests/fetch: don't rely on wget
--continue in the corrupt partial test" to merge before closing the bug.

Regards,
-- 
Yoann Congal
Smile ECS

Re: [bitbake-devel] [PATCH] fetch/wget: rename the partial download on checksum mismatch

[Mathieu Dubois-Briand]

On Thu Oct 8, 2026 at 4:15 PM CEST, Ricardo Salveti wrote:
> Hi,
>
> On Thu, Oct 8, 2026 at 8:59 AM Mathieu Dubois-Briand
> <mathieu.dubois-briand@bootlin.com> wrote:
>>
>> On Tue Sep 29, 2026 at 3:28 AM CEST, Ricardo Salveti via lists.openembedded.org wrote:
>> > The wget fetcher downloads into a .tmp file which is resumed with
>> > --continue and is shared with the mirrors of the same file. When it
>> > fails checksum verification it is left in place, so MIRRORS reuse the
>> > bad file instead of downloading their own copy and, without a mirror,
>> > every retry fails the same way until it is removed by hand. Rename it
>> > with rename_bad_checksum(), as done for any other file with a bad
>> > checksum, so it can be inspected and the next attempt starts from
>> > scratch. Mirror downloads skip this check and are still handled by the
>> > existing code once moved into place.
>> >
>> > Seen with parallel CI runners sharing DL_DIR on Amazon FSx for OpenZFS
>> > mounted over NFSv4.2. What corrupts the file there is not addressed,
>> > only the recovery on retry. Tested with the new tests, which fail
>> > without the fix.
>> >
>> > AI-Generated: Uses Claude Code (Claude Fable 5.1)
>> > Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com>
>> > ---
>>
>> Hi Ricardo,
>>
>> I suspect this patch is responsible of intermittent failures we can see
>> during test_wget_corrupt_partial_download selftest, only on Fedora
>> workers.
>>
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=16445
>>
>> Do you have any clue about what might be wrong here?
>
> Should be fixed with
> https://patchwork.yoctoproject.org/project/bitbake/patch/20261006213334.3360919-1-ricardo.salveti@oss.qualcomm.com/,
> mind giving it a try?
>
> Thanks,
>
> Ricardo

Sorry, I thought it was still failing, but this is something else. I'll
keep you updated.

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com