Re: Functional tests precache behaviour

[Pierrick Bouvier <pierrick.bouvier@linaro.org>]

On 4/30/25 8:00 AM, Thomas Huth wrote:
> On 30/04/2025 16.34, Pierrick Bouvier wrote:
>> Hi folks,
>>
>> $ ninja -C build precache-functional
>> 2025-04-30 07:23:20,382 - qemu-test - ERROR - Unable to download https://
>> archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/
>> gzimg/armv7.img.gz: HTTP error 503
>> 2025-04-30 07:23:23,131 - qemu-test - ERROR - Unable to download https://
>> archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/
>> gzimg/armv7.img.gz: HTTP error 503
>> 2025-04-30 07:23:25,870 - qemu-test - ERROR - Unable to download https://
>> archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/
>> gzimg/armv7.img.gz: HTTP error 503
>> 2025-04-30 07:23:25,871 - qemu-test - ERROR - https://archive.netbsd.org/
>> pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/gzimg/armv7.img.gz:
>> Download retries exceeded: skipping asset precache
>> $ echo $?
>> 0
>>
>> Since we silently skip the asset precaching, how can we identify that an
>> asset is not available anymore (temporarily or not)?
>> Should we rely on test itself failing when trying to download again this asset?
> 
> The current logic fails hard for 404 errors, so if the asset is completely
> gone, we should notice it. For other error codes, we assume that it is only
> a temporary server problem that will hopefully be fixed on the server side
> sooner or later.
>

Sounds good.
Should we replicate this semantic when running the test itself?
It would be more useful to skip it because an asset is missing instead 
of reporting an error, except if it's a 404 error.

>    Thomas
>