Re: [linux-4.1 test] 60785: tolerable FAIL - PUSHED

[Ian Campbell <ian.campbell@citrix.com>]

On Fri, 2015-08-21 at 17:24 +0000, osstest service owner wrote:
> flight 60785 linux-4.1 real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/60785/
> 
> Failures :-/ but no regressions.
> 
> Tests which are failing intermittently (not blocking):
>  test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 19 guest-start/debianhvm.repeat fail in 60746 pass in 60785
>  test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 16 guest-start/debianhvm.repeat fail pass in 60746
>  test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 13 guest-localmigrate fail pass in 60746
> 
> Regressions which are regarded as allowable (not blocking):
>  test-amd64-amd64-rumpuserxen-amd64 15 rumpuserxen-demo-xenstorels/xenstorels.repeat fail REGR. vs. 60654
>  test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 15 guest-localmigrate.2 fail blocked in 60654
>  test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 13 guest-localmigrate fail in 60746 like 60654

>  test-amd64-i386-xl-xsm       14 guest-saverestore            fail   like 60654
>  test-amd64-i386-xl           14 guest-saverestore            fail   like 60654
>  test-amd64-i386-pair        21 guest-migrate/src_host/dst_host fail like 60654

As noted in 
http://lists.xen.org/archives/html/xen-devel/2015-09/msg00306.html these
three are regressions vs. 3.14 which the bisector has fingered with being
3a9ace0147d4 "tools/libxc+libxl+xl: Restore v2
streams".

linux-linux is also failing in what looks like the same manner.

We perhaps ought to consider whether we think "migration broken with newer
kernels" should be a blocker for 4.6.

The bisector was working on the linux-4.1, so ended up fingering the Xen
change which exposed the bug, since it didn't consider older kernels. I'm
going to see if I can engineer some adhoc runs which will cause the
bisector to instead try and find a commit between 3.18 and 4.1 which was
tickled by the Xen change, since I suspect that will be the actual
underlying buggy thing. If any one has any intuition on where to start
looking that might help speed things along...

Also, the fact these are "fail like NNNN" is a bit odd since the history[0]
shows this used to pass, hence a bad thing got pushed at some point.

59811 and 59837 correctly has them as "fail REGR. vs. 59393", 

59909 considered one of the three "fail like 59936-bisect" and the others
as "fail REGR. vs. 59393".

Then 59960 has two as "fail like NNNNNN-bisect" and one as "fail REGR. vs.
59393".

60030 is the "bad" push which uses "fail like NNNNNN-bisect" to justify
ignoring all three.

IIRC there was a bug in the osstest machinery (now fixed by Ian) which lead
to this, but I mention it in case I'm recalling incorrectly.

Ian.

[0] http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64
-i386-pair/linux-4.1.html