From: "Alex Bennée" <alex.bennee@linaro.org>
To: Peter Maydell <peter.maydell@linaro.org>
Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>,
"QEMU Developers" <qemu-devel@nongnu.org>
Subject: Re: [PATCH] tests/acceptance/boot_linux: Skip slow Aarch64 'virt' machine TCG test
Date: Thu, 07 May 2020 21:32:41 +0100 [thread overview]
Message-ID: <871rnv5vva.fsf@linaro.org> (raw)
In-Reply-To: <CAFEAcA800Lhgqa6gOWmA5M2D3+Xj0hYcJw9Tkget4yvW5fFs+A@mail.gmail.com>
Peter Maydell <peter.maydell@linaro.org> writes:
> On Thu, 7 May 2020 at 19:38, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>>
>> Philippe Mathieu-Daudé <philmd@redhat.com> writes:
>>
>> > The BootLinuxAarch64.test_virt_tcg is reported to take >7min to run.
>> > Add a possibility to users to skip this particular test, by setting
>> > the AVOCADO_SKIP_SLOW_TESTS environment variable:
>> >
>> > $ AVOCADO_SKIP_SLOW_TESTS=please make check-acceptance
>> > ...
>> > (05/88) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_tcg: SKIP: Test takes >7min
>> > ...
>>
>> 7m = 420s - I'm seeing:
>>
>> hyperfine "./tests/venv/bin/avocado run ./tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_tcg"
>> Benchmark #1: ./tests/venv/bin/avocado run ./tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_tcg
>> Time (mean ± σ): 162.179 s ± 3.138 s [User: 204.726 s, System: 9.663 s]
>> Range (min … max): 158.651 s … 170.036 s 10 runs
>>
>> Is this on very slow hardware?
>
> Intel(R) Xeon(R) W-2145 CPU @ 3.70GHz 16-core
> clang sanitizer build, which probably slows it down a bit.
>
> But even 200 seconds is an order of magnitude slower than any
> of the other tests that check-acceptance runs. I think we
> should be aiming for tests here to be ~30 seconds at most,
> or the whole thing will take forever by the time we have
> decent coverage of most machines.
I think we could say the same for a whole bunch of the tests that are
based on full distro downloads. What is this one running under the hood
anyway?
>
> thanks
> -- PMM
--
Alex Bennée
next prev parent reply other threads:[~2020-05-07 20:33 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-07 16:22 [PATCH] tests/acceptance/boot_linux: Skip slow Aarch64 'virt' machine TCG test Philippe Mathieu-Daudé
2020-05-07 18:38 ` Alex Bennée
2020-05-07 18:45 ` Peter Maydell
2020-05-07 20:32 ` Alex Bennée [this message]
2020-05-08 13:07 ` Philippe Mathieu-Daudé
2020-05-09 13:18 ` Lukas Straub
2020-05-11 9:11 ` Peter Maydell
2020-05-11 16:51 ` Lukas Straub
2020-05-26 9:14 ` Philippe Mathieu-Daudé
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=871rnv5vva.fsf@linaro.org \
--to=alex.bennee@linaro.org \
--cc=peter.maydell@linaro.org \
--cc=philmd@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.