[PATCH] tests/fp: Make mul and div tests have a longer timeout

[PATCH] tests/fp: Make mul and div tests have a longer timeout

[Peter Maydell]

At the moment we run all fp-test tests except for the muladd ones
with the default meson test timeout of 30s. This is plenty for
most of the test cases, but for multiplication and division we
can sometimes hit the timeout if the CI runner is going slow.

Add support to meson.build for a way to override the timeout on
a per test basis, and use it to set the timeout to 60s for
fp-test-rem, fp-test-div and fp-test-mul. We can use this new
generic mechanism also to set the timeout for mulAdd rather
than hardcoding it.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
On my local system these tests finish in less than 5 seconds
each, but on the cross-i686-system CI job they can take 5 or 6
times as long. Example:
https://gitlab.com/qemu-project/qemu/-/jobs/7844908223
---
 tests/fp/meson.build | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/tests/fp/meson.build b/tests/fp/meson.build
index 114b4b483ea..9059a247521 100644
--- a/tests/fp/meson.build
+++ b/tests/fp/meson.build
@@ -7,6 +7,16 @@ if host_os == 'windows'
   subdir_done()
 endif
 
+# By default tests run with the usual 30s timeout; particularly
+# slow tests can have that overridden here. The keys here are
+# the testnames without their fp-test- prefix.
+slow_fp_tests = {
+  'rem': 60,
+  'div': 60,
+  'mul': 60,
+  'mulAdd': 180,
+}
+
 sfcflags = [
   # softfloat defines
   '-DSOFTFLOAT_ROUND_ODD',
@@ -109,6 +119,7 @@ fptest_rounding_args = ['-r', 'all']
 foreach k, v : softfloat_conv_tests
   test('fp-test-' + k, fptest,
        args: fptest_args + fptest_rounding_args + v.split(),
+       timeout: slow_fp_tests.get(k, 30),
        suite: ['softfloat', 'softfloat-conv'])
 endforeach
 
@@ -116,6 +127,7 @@ foreach k, v : softfloat_tests
   test('fp-test-' + k, fptest,
        args: fptest_args + fptest_rounding_args +
              ['f16_' + k, 'f32_' + k, 'f64_' + k, 'f128_' + k, 'extF80_' + k],
+       timeout: slow_fp_tests.get(k, 30),
        suite: ['softfloat', 'softfloat-' + v])
 endforeach
 
@@ -124,7 +136,8 @@ test('fp-test-mulAdd', fptest,
      # no fptest_rounding_args
      args: fptest_args +
            ['f16_mulAdd', 'f32_mulAdd', 'f64_mulAdd', 'f128_mulAdd'],
-     suite: ['softfloat-slow', 'softfloat-ops-slow', 'slow'], timeout: 180)
+     timeout: slow_fp_tests.get('mulAdd', 30),
+     suite: ['softfloat-slow', 'softfloat-ops-slow', 'slow'])
 
 executable(
   'fp-bench',
@@ -140,4 +153,5 @@ fptestlog2 = executable(
   c_args: fpcflags,
 )
 test('fp-test-log2', fptestlog2,
+     timeout: slow_fp_tests.get('log2', 30),
      suite: ['softfloat', 'softfloat-ops'])
-- 
2.34.1

Re: [PATCH] tests/fp: Make mul and div tests have a longer timeout

[Richard Henderson]

On 9/17/24 16:16, Peter Maydell wrote:
> At the moment we run all fp-test tests except for the muladd ones
> with the default meson test timeout of 30s. This is plenty for
> most of the test cases, but for multiplication and division we
> can sometimes hit the timeout if the CI runner is going slow.
> 
> Add support to meson.build for a way to override the timeout on
> a per test basis, and use it to set the timeout to 60s for
> fp-test-rem, fp-test-div and fp-test-mul. We can use this new
> generic mechanism also to set the timeout for mulAdd rather
> than hardcoding it.
> 
> Signed-off-by: Peter Maydell<peter.maydell@linaro.org>
> ---
> On my local system these tests finish in less than 5 seconds
> each, but on the cross-i686-system CI job they can take 5 or 6
> times as long. Example:
> https://gitlab.com/qemu-project/qemu/-/jobs/7844908223
> ---
>   tests/fp/meson.build | 16 +++++++++++++++-
>   1 file changed, 15 insertions(+), 1 deletion(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH] tests/fp: Make mul and div tests have a longer timeout

[Peter Maydell]

On Wed, 18 Sept 2024 at 15:19, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 9/17/24 16:16, Peter Maydell wrote:
> > At the moment we run all fp-test tests except for the muladd ones
> > with the default meson test timeout of 30s. This is plenty for
> > most of the test cases, but for multiplication and division we
> > can sometimes hit the timeout if the CI runner is going slow.
> >
> > Add support to meson.build for a way to override the timeout on
> > a per test basis, and use it to set the timeout to 60s for
> > fp-test-rem, fp-test-div and fp-test-mul. We can use this new
> > generic mechanism also to set the timeout for mulAdd rather
> > than hardcoding it.
> >
> > Signed-off-by: Peter Maydell<peter.maydell@linaro.org>
> > ---
> > On my local system these tests finish in less than 5 seconds
> > each, but on the cross-i686-system CI job they can take 5 or 6
> > times as long. Example:
> > https://gitlab.com/qemu-project/qemu/-/jobs/7844908223
> > ---
> >   tests/fp/meson.build | 16 +++++++++++++++-
> >   1 file changed, 15 insertions(+), 1 deletion(-)
>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

Thanks; I've applied this directly to see if it helps with
some of the CI flakiness.

-- PMM