Re: [PATCH v3 24/24] tests/avocado: disable BootLinuxPPC64 test in CI

[Thomas Huth <thuth@redhat.com>]

On 28/02/2023 20.06, Alex Bennée wrote:
> This test is exceptionally heavyweight (nearly 330s) compared to the
> two (both endians) TuxRun baseline tests which complete in under 160s.
> The coverage is slightly reduced but a more directed test could make
> up the difference.
> 
> tests/avocado/tuxrun_baselines.py:TuxRunBaselineTest.test_ppc64:
> 
> Overall coverage rate:
>    lines......: 9.6% (44110 of 458817 lines)
>    functions..: 16.5% (6767 of 41054 functions)
>    branches...: 6.0% (13395 of 222634 branches)
> 
> tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg:
> 
> Overall coverage rate:
>    lines......: 11.6% (53408 of 458817 lines)
>    functions..: 18.7% (7691 of 41054 functions)
>    branches...: 7.9% (17692 of 224218 branches)
> 
> So lets skip for GITLAB_CI and also unless AVOCADO_TIMEOUT_EXPECTED is
> specified by the user.

The explanation sounds somewhat implausible to me. AVOCADO_TIMEOUT_EXPECTED 
should be for jobs where we are not sure whether the job really finishes in 
time, e.g. when compiling QEMU with debug flags enabled, and not for jobs 
that simply run a little bit longer (in the latter case, it would be enough 
to simply bump the timeout setting a little bit if necessary). So did you 
check whether you really run into timeout issues here when compiling QEMU 
with debug flags?

Anyway, if you add AVOCADO_TIMEOUT_EXPECTED, then I think you don't need 
GITLAB_CI anymore, since we certainly don't set AVOCADO_TIMEOUT_EXPECTED in 
the gitlab CI.

  Thomas