Re: [PATCH v2 31/31] tests/functional: ignore errors when caching assets, except for 404

[Thomas Huth <thuth@redhat.com>]

On 11/12/2024 18.26, Daniel P. Berrangé wrote:
> We see periodic errors caching assets due to a combination of transient
> networking and server problems. With the previous patch to skip running
> a test when it has missing assets, we can now treat most cache download
> errors as non-fatal.
> 
> Only HTTP 404 is retained as fatal, since it is a strong indicator of
> a fully broken test rather than a transient error.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   tests/functional/qemu_test/asset.py | 14 +++++++++++++-
>   1 file changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
> index 39832b2587..f0730695f0 100644
> --- a/tests/functional/qemu_test/asset.py
> +++ b/tests/functional/qemu_test/asset.py
> @@ -15,6 +15,7 @@
>   from time import sleep
>   from pathlib import Path
>   from shutil import copyfileobj
> +from urllib.error import HTTPError
>   
>   
>   # Instances of this class must be declared as class level variables
> @@ -170,7 +171,18 @@ def precache_test(test):
>           for name, asset in vars(test.__class__).items():
>               if name.startswith("ASSET_") and type(asset) == Asset:
>                   log.info("Attempting to cache '%s'" % asset)
> -                asset.fetch()
> +                try:
> +                    asset.fetch()
> +                except HTTPError as e:
> +                    # Treat 404 as fatal, since it is highly likely to
> +                    # indicate a broken test rather than a transient
> +                    # server or networking problem
> +                    if e.code == 404:
> +                        raise
> +
> +                    log.debug(f"HTTP error {e.code} from {asset.url} " +
> +                              "skipping asset precache")
> +
>           log.removeHandler(handler)
>   
>       def precache_suite(suite):

Reviewed-by: Thomas Huth <thuth@redhat.com>