Re: [Qemu-devel] [PULL 10/19] tests/boot_linux_console: increase timeout

[Cleber Rosa <crosa@redhat.com>]

On Wed, May 08, 2019 at 12:28:24PM +0200, Gerd Hoffmann wrote:
> On Thu, May 02, 2019 at 09:41:21PM -0300, Eduardo Habkost wrote:
> > From: Cleber Rosa <crosa@redhat.com>
> > 
> > When running on very low powered environments, some tests may time out
> > causing false negatives.  As a conservative change, and for
> > considering that human time (investigating false negatives) is worth
> > more than some extra machine cycles (and time), let's increase the
> > overall timeout.
> 
> Tests can also timeout due to slow downloads of test kernels.
> Any chance to run the downloads without timeout?

I acknowledge this is an issue, and have thought about two possible
ways to solve it:

 1) Downloading/caching/checking all the test assets in a job "pre-tests"
    plugin.  This would involve/require:

   - https://trello.com/c/WPd4FrIy/1479-add-support-to-specify-assets-in-test-docstring
   - https://trello.com/c/CKP7YS6G/1481-on-cache-check-for-asset-fetcher

 2) Report the test phase (say, setUp()) to the test runner, which
    would allow the runner to:

   - Not take that time into account for the test (similar to
     https://github.com/avocado-framework/avocado/pull/3076)
   - Retry the setUp a number of times (if the test opts in or doesn't
     opt out)

I'm very much interested in your opinion so we can evolve the idea into
implementation.

Thanks!
- Cleber

> 
> cheers,
>   Gerd
>