Broken NetBSD Orange Pi image URL in QEMU tests

Broken NetBSD Orange Pi image URL in QEMU tests

[Stefan Hajnoczi]

Hi,
CI jobs that run test_arm_orangepi.py are failing:
https://gitlab.com/qemu-project/qemu/-/jobs/9390048284#L1138

Please consider how to resolve this so the CI job passes again. If you
are in contact with the archive.netbsd.org administrators, maybe
contacting them will lead to a fix. Otherwise please update the QEMU
test to use a URL that works or remove the test.

The NetBSD Orange Pi image fails to download cleanly:

$ curl -O 'https://archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/gzimg/armv7.img.gz'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0  303M    0 2048k    0     0  1098k      0  0:04:42  0:00:01  0:04:41 1098k
curl: (18) end of response with 315646186 bytes missing

Thanks,
Stefan

Re: Broken NetBSD Orange Pi image URL in QEMU tests

[Thomas Huth]

On 13/03/2025 03.22, Stefan Hajnoczi wrote:
> Hi,
> CI jobs that run test_arm_orangepi.py are failing:
> https://gitlab.com/qemu-project/qemu/-/jobs/9390048284#L1138
> 
> Please consider how to resolve this so the CI job passes again. If you
> are in contact with the archive.netbsd.org administrators, maybe
> contacting them will lead to a fix. Otherwise please update the QEMU
> test to use a URL that works or remove the test.
> 
> The NetBSD Orange Pi image fails to download cleanly:
> 
> $ curl -O 'https://archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/gzimg/armv7.img.gz'
>    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
>                                   Dload  Upload   Total   Spent    Left  Speed
>    0  303M    0 2048k    0     0  1098k      0  0:04:42  0:00:01  0:04:41 1098k
> curl: (18) end of response with 315646186 bytes missing

FYI, Nicholas already provided a nice patch series to skip the test in case 
of such incomplete downloads:

  https://lore.kernel.org/qemu-devel/20250312130002.945508-1-npiggin@gmail.com/

I'll try to assemble a pull request with these patches today.

  Thomas

Re: Broken NetBSD Orange Pi image URL in QEMU tests

[Stefan Hajnoczi]

On Thu, Mar 13, 2025 at 1:48 PM Thomas Huth <thuth@redhat.com> wrote:
>
> On 13/03/2025 03.22, Stefan Hajnoczi wrote:
> > Hi,
> > CI jobs that run test_arm_orangepi.py are failing:
> > https://gitlab.com/qemu-project/qemu/-/jobs/9390048284#L1138
> >
> > Please consider how to resolve this so the CI job passes again. If you
> > are in contact with the archive.netbsd.org administrators, maybe
> > contacting them will lead to a fix. Otherwise please update the QEMU
> > test to use a URL that works or remove the test.
> >
> > The NetBSD Orange Pi image fails to download cleanly:
> >
> > $ curl -O 'https://archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/gzimg/armv7.img.gz'
> >    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
> >                                   Dload  Upload   Total   Spent    Left  Speed
> >    0  303M    0 2048k    0     0  1098k      0  0:04:42  0:00:01  0:04:41 1098k
> > curl: (18) end of response with 315646186 bytes missing
>
> FYI, Nicholas already provided a nice patch series to skip the test in case
> of such incomplete downloads:
>
>   https://lore.kernel.org/qemu-devel/20250312130002.945508-1-npiggin@gmail.com/
>
> I'll try to assemble a pull request with these patches today.

Thank you!

Stefan

Re: Broken NetBSD Orange Pi image URL in QEMU tests

[Niek Linnenbank]

[-- Attachment #1: Type: text/plain, Size: 1693 bytes --]

Hello Stefan, Thomas,

Thanks for reporting this. I'll try to spend some time to figure out the
issue and perhaps propose a new image for this test, if needed.

Regards,
Niek

On Thu, Mar 13, 2025 at 8:39 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:

> On Thu, Mar 13, 2025 at 1:48 PM Thomas Huth <thuth@redhat.com> wrote:
> >
> > On 13/03/2025 03.22, Stefan Hajnoczi wrote:
> > > Hi,
> > > CI jobs that run test_arm_orangepi.py are failing:
> > > https://gitlab.com/qemu-project/qemu/-/jobs/9390048284#L1138
> > >
> > > Please consider how to resolve this so the CI job passes again. If you
> > > are in contact with the archive.netbsd.org administrators, maybe
> > > contacting them will lead to a fix. Otherwise please update the QEMU
> > > test to use a URL that works or remove the test.
> > >
> > > The NetBSD Orange Pi image fails to download cleanly:
> > >
> > > $ curl -O '
> https://archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/gzimg/armv7.img.gz
> '
> > >    % Total    % Received % Xferd  Average Speed   Time    Time
>  Time  Current
> > >                                   Dload  Upload   Total   Spent
> Left  Speed
> > >    0  303M    0 2048k    0     0  1098k      0  0:04:42  0:00:01
> 0:04:41 1098k
> > > curl: (18) end of response with 315646186 bytes missing
> >
> > FYI, Nicholas already provided a nice patch series to skip the test in
> case
> > of such incomplete downloads:
> >
> >
> https://lore.kernel.org/qemu-devel/20250312130002.945508-1-npiggin@gmail.com/
> >
> > I'll try to assemble a pull request with these patches today.
>
> Thank you!
>
> Stefan
>


-- 
Niek Linnenbank

[-- Attachment #2: Type: text/html, Size: 2962 bytes --]

Re: Broken NetBSD Orange Pi image URL in QEMU tests

[Niek Linnenbank]

[-- Attachment #1: Type: text/plain, Size: 2699 bytes --]

Hello Stefan,

As of today, it seems the URL is working properly again. I've done a few
downloads without any error.
What I did notice is that NetBSD provides a 'cdn.netbsd.org' also, but I
can't see any noticable difference yet.

Tests are also running OK now, at least on my local system:

$ QEMU_TEST_ALLOW_LARGE_STORAGE=1 pyvenv/bin/python3
../tests/functional/test_arm_orangepi.py
TAP version 13
ok 1 test_arm_orangepi.BananaPiMachine.test_arm_orangepi
ok 2 test_arm_orangepi.BananaPiMachine.test_arm_orangepi_armbian
ok 3 test_arm_orangepi.BananaPiMachine.test_arm_orangepi_initrd
ok 4 test_arm_orangepi.BananaPiMachine.test_arm_orangepi_sd
ok 5 test_arm_orangepi.BananaPiMachine.test_arm_orangepi_uboot_netbsd9
1..5

Can you confirm the URL is working again properly on CI also?

Regards
Niek

On Thu, Mar 13, 2025 at 10:22 PM Niek Linnenbank <nieklinnenbank@gmail.com>
wrote:

> Hello Stefan, Thomas,
>
> Thanks for reporting this. I'll try to spend some time to figure out the
> issue and perhaps propose a new image for this test, if needed.
>
> Regards,
> Niek
>
> On Thu, Mar 13, 2025 at 8:39 AM Stefan Hajnoczi <stefanha@gmail.com>
> wrote:
>
>> On Thu, Mar 13, 2025 at 1:48 PM Thomas Huth <thuth@redhat.com> wrote:
>> >
>> > On 13/03/2025 03.22, Stefan Hajnoczi wrote:
>> > > Hi,
>> > > CI jobs that run test_arm_orangepi.py are failing:
>> > > https://gitlab.com/qemu-project/qemu/-/jobs/9390048284#L1138
>> > >
>> > > Please consider how to resolve this so the CI job passes again. If you
>> > > are in contact with the archive.netbsd.org administrators, maybe
>> > > contacting them will lead to a fix. Otherwise please update the QEMU
>> > > test to use a URL that works or remove the test.
>> > >
>> > > The NetBSD Orange Pi image fails to download cleanly:
>> > >
>> > > $ curl -O '
>> https://archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/gzimg/armv7.img.gz
>> '
>> > >    % Total    % Received % Xferd  Average Speed   Time    Time
>>  Time  Current
>> > >                                   Dload  Upload   Total   Spent
>> Left  Speed
>> > >    0  303M    0 2048k    0     0  1098k      0  0:04:42  0:00:01
>> 0:04:41 1098k
>> > > curl: (18) end of response with 315646186 bytes missing
>> >
>> > FYI, Nicholas already provided a nice patch series to skip the test in
>> case
>> > of such incomplete downloads:
>> >
>> >
>> https://lore.kernel.org/qemu-devel/20250312130002.945508-1-npiggin@gmail.com/
>> >
>> > I'll try to assemble a pull request with these patches today.
>>
>> Thank you!
>>
>> Stefan
>>
>
>
> --
> Niek Linnenbank
>
>

-- 
Niek Linnenbank

[-- Attachment #2: Type: text/html, Size: 4556 bytes --]

Re: Broken NetBSD Orange Pi image URL in QEMU tests

[Stefan Hajnoczi]

On Sat, Mar 15, 2025 at 5:01 PM Niek Linnenbank
<nieklinnenbank@gmail.com> wrote:
>
> Hello Stefan,
>
> As of today, it seems the URL is working properly again. I've done a few downloads without any error.
> What I did notice is that NetBSD provides a 'cdn.netbsd.org' also, but I can't see any noticable difference yet.
>
> Tests are also running OK now, at least on my local system:
>
> $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 pyvenv/bin/python3 ../tests/functional/test_arm_orangepi.py
> TAP version 13
> ok 1 test_arm_orangepi.BananaPiMachine.test_arm_orangepi
> ok 2 test_arm_orangepi.BananaPiMachine.test_arm_orangepi_armbian
> ok 3 test_arm_orangepi.BananaPiMachine.test_arm_orangepi_initrd
> ok 4 test_arm_orangepi.BananaPiMachine.test_arm_orangepi_sd
> ok 5 test_arm_orangepi.BananaPiMachine.test_arm_orangepi_uboot_netbsd9
> 1..5
>
> Can you confirm the URL is working again properly on CI also?

Thanks, Niek. The URL is working again for me.

Stefan

>
> Regards
> Niek
>
> On Thu, Mar 13, 2025 at 10:22 PM Niek Linnenbank <nieklinnenbank@gmail.com> wrote:
>>
>> Hello Stefan, Thomas,
>>
>> Thanks for reporting this. I'll try to spend some time to figure out the issue and perhaps propose a new image for this test, if needed.
>>
>> Regards,
>> Niek
>>
>> On Thu, Mar 13, 2025 at 8:39 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
>>>
>>> On Thu, Mar 13, 2025 at 1:48 PM Thomas Huth <thuth@redhat.com> wrote:
>>> >
>>> > On 13/03/2025 03.22, Stefan Hajnoczi wrote:
>>> > > Hi,
>>> > > CI jobs that run test_arm_orangepi.py are failing:
>>> > > https://gitlab.com/qemu-project/qemu/-/jobs/9390048284#L1138
>>> > >
>>> > > Please consider how to resolve this so the CI job passes again. If you
>>> > > are in contact with the archive.netbsd.org administrators, maybe
>>> > > contacting them will lead to a fix. Otherwise please update the QEMU
>>> > > test to use a URL that works or remove the test.
>>> > >
>>> > > The NetBSD Orange Pi image fails to download cleanly:
>>> > >
>>> > > $ curl -O 'https://archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/gzimg/armv7.img.gz'
>>> > >    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
>>> > >                                   Dload  Upload   Total   Spent    Left  Speed
>>> > >    0  303M    0 2048k    0     0  1098k      0  0:04:42  0:00:01  0:04:41 1098k
>>> > > curl: (18) end of response with 315646186 bytes missing
>>> >
>>> > FYI, Nicholas already provided a nice patch series to skip the test in case
>>> > of such incomplete downloads:
>>> >
>>> >   https://lore.kernel.org/qemu-devel/20250312130002.945508-1-npiggin@gmail.com/
>>> >
>>> > I'll try to assemble a pull request with these patches today.
>>>
>>> Thank you!
>>>
>>> Stefan
>>
>>
>>
>> --
>> Niek Linnenbank
>>
>
>
> --
> Niek Linnenbank
>

Re: Broken NetBSD Orange Pi image URL in QEMU tests

[Thomas Huth]

On 15/03/2025 22.01, Niek Linnenbank wrote:
> Hello Stefan,
> 
> As of today, it seems the URL is working properly again. I've done a few 
> downloads without any error.
> What I did notice is that NetBSD provides a 'cdn.netbsd.org <http:// 
> cdn.netbsd.org>' also, but I can't see any noticable difference yet.

Oh, weird, in commit 380f7268b7ba4a6 I had to switch the URL from the cdn to 
the archive one since the file was not available on the cdn server 
anymore... but now it seems to be back! Certainly something to keep in mind 
in case we face problems with this asset again.

  Thomas