Re: [Qemu-devel] [PATCH v2 2/2] Boot Linux Console Test: add a test for the Raspberry Pi 2

[Cleber Rosa <crosa@redhat.com>]

----- Original Message -----
> From: "Philippe Mathieu-Daudé" <philmd@redhat.com>
> To: "Cleber Rosa" <crosa@redhat.com>, "Eduardo Habkost" <ehabkost@redhat.com>
> Cc: qemu-devel@nongnu.org, "Peter Maydell" <peter.maydell@linaro.org>, qemu-arm@nongnu.org, "Philippe Mathieu-Daudé"
> <f4bug@amsat.org>, "Andrew Baumann" <Andrew.Baumann@microsoft.com>, "Gerd Hoffmann" <kraxel@redhat.com>
> Sent: Thursday, May 23, 2019 5:29:22 AM
> Subject: Re: [Qemu-devel] [PATCH v2 2/2] Boot Linux Console Test: add a test for the Raspberry Pi 2
> 
> On 5/22/19 11:05 PM, Cleber Rosa wrote:
> > ----- Original Message -----
> >> From: "Eduardo Habkost" <ehabkost@redhat.com>
> >> On Wed, Mar 13, 2019 at 12:45:41AM +0100, Philippe Mathieu-Daudé wrote:
> >>> From: Philippe Mathieu-Daudé <f4bug@amsat.org>
> >>>
> >>> Similar to the x86_64/pc test, it boots a Linux kernel on a raspi2
> >>> board and verify the serial is working.
> >>>
> >>> If ARM is a target being built, "make check-acceptance" will
> >>> automatically include this test by the use of the "arch:arm" tags.
> >>>
> >>> Alternatively, this test can be run using:
> >>>
> >>>     $ avocado run -t arch:arm tests/acceptance
> >>>     $ avocado run -t machine:raspi2 tests/acceptance
> >>>
> >>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> >>
> >> We're getting timeouts on travis-ci:
> >> https://travis-ci.org/ehabkost/qemu/jobs/535468057#L3289
> >>
> >> I don't know yet if the guest is hanging, or if we just need to
> >> increase the timeout.  I could reproduce the timeout locally,
> >> too.
> 
> That's odd, I can't reproduce (this test is quicker than the following
> test_arm_emcraft_sf2 which start u-boot then timeouts and start the kernel).
> 
> My guess is network issues, since this test use a different mirror:
> archive.raspberrypi.org
> 

It could be a network issue, it could be something else.  I think the very
first step, and I'd urge us to get that on master ASAP, is to show the
entire logs in CI, I mean:

---

diff --git a/.travis.yml b/.travis.yml
index 6fd89b3d91..fd8f6ca2d2 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -225,7 +225,7 @@ matrix:
     # Acceptance (Functional) tests
     - env:
         - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
-        - TEST_CMD="make check-acceptance"
+        - TEST_CMD="make AVOCADO_SHOW=test check-acceptance"
       addons:
         apt:
           packages:

---

That way we can know for sure what's going on. 

> Gerd already raised this problem (timeout reached while fetching
> artifacts) to Cleber.
> Cleber, can we add test_setup() methods that use different timeouts?
> 

Not in a released Avocado version.  Interestingly enough, I wrote a PoC
that adds different timeouts to tearDown()[1], that can be used the same
way for setUp(), and it looks like Intel DAOS is already using those
patches[2].

So, I guess I can work on a non-PoC version for that.

- Cleber.

[1] - https://github.com/avocado-framework/avocado/pull/3076
[2] - https://github.com/daos-stack/daos/commit/084ec23461e7bd9b997d4b6f5e8095a4eaffc685

> Regards,
> 
> Phil.
> 
> > It may be related to:
> > 
> >  https://bugs.launchpad.net/qemu/+bug/1829779
> > 
> > And this is a proof that we urgently need to have a better
> > way of presenting/storing test artifacts.  The brief output
> > is nice when everything goes well, but makes the test results
> > close to useless once a failure happens.
> > 
> > Philippe showed us how GitLab allows CI jobs to preserve
> > artifacts, so maybe the solution is to move the loads there.
> > 
> > - Cleber.
> > 
>