[U-Boot] [RFC PATCH] test: py: Disable sleep test for qemu targets

[Tom Rini <trini@konsulko.com>]

On Mon, Dec 04, 2017 at 03:21:04PM +0100, Michal Simek wrote:
> On 4.12.2017 15:03, Tom Rini wrote:
> > On Mon, Dec 04, 2017 at 02:55:45PM +0100, Michal Simek wrote:
> >> On 1.12.2017 23:44, Tom Rini wrote:
> >>> On Fri, Dec 01, 2017 at 10:07:54AM -0700, Stephen Warren wrote:
> >>>> On 12/01/2017 08:19 AM, Michal Simek wrote:
> >>>>> Hi,
> >>>>>
> >>>>> On 1.12.2017 16:06, Heinrich Schuchardt wrote:
> >>>>>>
> >>>>>>
> >>>>>> On 12/01/2017 03:46 PM, Michal Simek wrote:
> >>>>>>> Qemu for arm32/arm64 has a problem with time setup.
> >>>>>>
> >>>>>> Wouldn't it be preferable to fix the root cause?
> >>>>>
> >>>>> Definitely that would be the best and IIRC I have tried to convince our
> >>>>> qemu guy to do that but they have never done that.
> >>>>
> >>>> What is the exact failure condition? Is it simply that the test is still
> >>>> slightly too strict about which delays it accepts, or is sleep outright
> >>>> broken?
> >>>>
> >>>> You can use command-line option -k to avoid some tests. For example "-k not
> >>>> sleep". That way, we don't have to hard-code the dependency into the test
> >>>> source. Depending on the root cause (issue in U-Boot, or issue in just your
> >>>> local version of qemu, or something that will never work) this might be
> >>>> better?
> >>>
> >>> Even with the most recent relaxing of the sleep test requirements, I can
> >>> still (depending on overall system load) have 'sleep' take too long, on
> >>> QEMU.  I think it might have been half a second slow, but I don't have
> >>> the log handy anymore.  Both locally and in travis we -k not sleep all
> >>> of the qemu instances.
> >>
> >> ok. By locally do you mean just using -k not sleep?
> > 
> > Yes, I have that in my CI scripts and similar.
> 
> Wouldn't be easier to keep this in uboot-test-hooks repo with other
> target setting?

Or do as you did did and mark the tests as not allowed for qemu, yes.

> What we are trying to do is that our testing group will run these tests
> for me that's why it is just easier for me to change local
> uboot-test-hooks repo instead of communicate with them what -k not XXX
> parameters to add to certain scripts.
> 
> It means in loop they will just run all tests on qemu, local targets and
> in boardfarm. It is probably not big deal to tell them to add -k not
> sleep for all qemu runs but I know that for some i2c testing qemu
> doesn't emulate these devices that's why these tests fails. And the
> amount of tests which we shouldn't run on qemu will probably grow.

Well, I'm still open to possibly tweaking the allowed variance in the
sleep test.  OTOH, if we just say "no QEMU" here, we can then go back to
"sleep should be pretty darn accurate on HW" for the test too, and
perhaps that's best.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20171204/fd43c499/attachment.sig>