Re: [PATCH] tests/functional: Bump timeouts of functional tests

[Pierrick Bouvier <pierrick.bouvier@linaro.org>]

On 11/6/24 09:09, Thomas Huth wrote:
> When building QEMU with "--enable-debug" and running the tests
> in parallel with "make -j$(nproc) check-functional", many tests are
> still timing out due to our conservative timeout settings. Bump
> the timeouts of the problematic tests and also increase the default
> timeout to 90 seconds (from 60 seconds) to be on the safe side.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   tests/functional/meson.build | 24 +++++++++++++-----------
>   1 file changed, 13 insertions(+), 11 deletions(-)
> 
> diff --git a/tests/functional/meson.build b/tests/functional/meson.build
> index d5296bff8b..3561f987a6 100644
> --- a/tests/functional/meson.build
> +++ b/tests/functional/meson.build
> @@ -11,24 +11,26 @@ endif
>   
>   # Timeouts for individual tests that can be slow e.g. with debugging enabled
>   test_timeouts = {
> -  'aarch64_raspi4' : 120,
> +  'aarch64_raspi4' : 480,
>     'aarch64_sbsaref' : 600,
> -  'aarch64_virt' : 360,
> -  'acpi_bits' : 240,
> +  'aarch64_virt' : 720,
> +  'acpi_bits' : 420,
>     'arm_aspeed' : 600,
> -  'arm_bpim2u' : 360,
> +  'arm_bpim2u' : 500,
> +  'arm_collie' : 180,
>     'arm_orangepi' : 540,
>     'arm_raspi2' : 120,
> -  'arm_tuxrun' : 120,
> +  'arm_tuxrun' : 240,
>     'arm_sx1' : 360,
>     'mips_malta' : 120,
>     'netdev_ethtool' : 180,
>     'ppc_40p' : 240,
>     'ppc64_hv' : 1000,
> -  'ppc64_powernv' : 240,
> -  'ppc64_pseries' : 240,
> -  'ppc64_tuxrun' : 240,
> -  's390x_ccw_virtio' : 240,
> +  'ppc64_powernv' : 480,
> +  'ppc64_pseries' : 480,
> +  'ppc64_tuxrun' : 420,
> +  'riscv64_tuxrun' : 120,
> +  's390x_ccw_virtio' : 420,
>   }
>   
>   tests_generic_system = [
> @@ -273,8 +275,8 @@ foreach speed : ['quick', 'thorough']
>              env: test_env,
>              args: [testpath],
>              protocol: 'tap',
> -           timeout: test_timeouts.get(test, 60),
> -           priority: test_timeouts.get(test, 60),
> +           timeout: test_timeouts.get(test, 90),
> +           priority: test_timeouts.get(test, 90),
>              suite: suites)
>       endforeach
>     endforeach

I noticed by --enable-debug in configure is a combination of enabling 
checks (enable-debug-tcg + graph + mutex), and deactivating optimizations.

Would it be worth keeping the optimizations and runtime checks instead? 
This way, there would be no more "timeout" issue.

I'm not sure which added value we get from O0, except for debugging 
locally QEMU.