Re: [RFC PATCH] tests/qtest: Don't parallelize migration-test

["Alex Bennée" <alex.bennee@linaro.org>]

Peter Maydell <peter.maydell@linaro.org> writes:

> The migration-test is a long-running test whose subtests all launch
> at least two QEMU processes.  This means that if for example the host
> has 4 CPUs then 'make check' defaults to a parallelism of 5, and if
> we launch 5 migration-tests in parallel then we will be running 10
> QEMU instances on a 4 CPU system.  If the system is not very fast
> then the test can spuriously time out because the different tests are
> all stealing CPU from each other.  This seems to particularly be a
> problem on our S390 CI job and the cross-i686-tci CI job.
>
> Force meson to run migration-test non-parallel, so there is never any
> other test running at the same time as it.  This will slow down
> overall test execution time somewhat, but hopefully make our CI less
> flaky.
>
> The downside is that because each migration-test instance runs for
> between 2 and 5 minutes and we run it for five architectures this
> significantly increases the runtime.  For an all-architectures build
> on my local machine 'make check -j8' goes from
>
>  real    8m19.127s
>  user    31m47.534s
>  sys     19m42.650s
>
> to
>
>  real    20m31.218s
>  user    32m48.712s
>  sys     19m52.133s
>
> more than doubling the wallclock runtime.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> Also, looking at these figures we spend a *lot* of our overall
> 'make check' time on migration-test. Do we really need to do
> that much for every architecture?

I guess one question is are we getting value from all the extra
migration tests? There certainly seem to be some sub-tests that are
slower than the others and I assume testing a small delta on the tests
before it.

On s390x it seems the native test runs pretty much to the same time as
the other TCG guests. Do we exercise any extra migration code by running
tests for every architecture as opposed to one KVM/native hyp and one
TCG one?

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro