Re: [xen-4.5-testing test] 56898: regressions - FAIL

[Ian Campbell <ian.campbell@citrix.com>]

On Fri, 2015-05-22 at 08:48 +0100, Jan Beulich wrote:
> >>> On 22.05.15 at 09:19, <ian.campbell@citrix.com> wrote:
> > On Fri, 2015-05-22 at 08:11 +0100, Jan Beulich wrote:
> >> >>> On 21.05.15 at 21:30, <osstest@xenbits.xen.org> wrote:
> >> > flight 56898 xen-4.5-testing real [real]
> >> > http://logs.test-lab.xenproject.org/osstest/logs/56898/ 
> >> > 
> >> > Regressions :-(
> >> > 
> >> > Tests which did not succeed and are blocking,
> >> > including tests which could not be run:
> >> >  test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop         fail REGR. vs. 56728
> >> 
> >> This is recurring (i.e. presumably real), but none of the few changes
> >> under test appear to be related in any way. And going through the
> >> logs I can't spot anything suspicious either. Does anyone else have
> >> a clue?
> > 
> > For a long time this test was marked "never passed". However I recently
> > added osstest support for using the ACPI shutdown method against certain
> > guests when configured to do so and configured win7.
> > 
> > However it is starting to look like the ACPI shutdown method is
> > unreliable, since this now seems to be failing intermittently. I haven't
> > had a chance to analyse it yet, but it seems like it might also be
> > specific to Windows 7.
> > 
> > In this case, as with the other one I looked at earlier in the week, the
> > guest vnc screenshot shows no sign that it is considering shutting down.
> > 
> >> >  test-amd64-amd64-rumpuserxen-amd64 15 
> >> > rumpuserxen-demo-xenstorels/xenstorels.repeat fail REGR. vs. 56728
> >> 
> >> I suppose this (also recurring, and also seemingly unrelated to
> >> any of the commits under test)
> > 
> > Correct, this is a long standing heisenbug. I recently increased the
> > number of iterations used in this test to (hopefully) reduce the
> > incidences of false passes.
> > 
> > I would force push any flight which failed only this case.
> 
> Together with your explanation on the other failure this then perhaps
> is enough reason to actually do a force push here (which iiuc will at
> once make both of them allowable failures going forward).

Until the next spurious pass, which looks to be about 1 in 10.

But yes, I think force pushing the win7 shutdown issue while we discuss
in the other thread would be reasonable, pending a decision there
whether to whitelist this particular failure or not when Ian gets back.

Ian.