From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anthony PERARD Subject: Re: [xen-unstable test] 24250: tolerable FAIL Date: Wed, 8 Jan 2014 12:55:32 +0000 Message-ID: <20140108125531.GC1696@perard.uk.xensource.com> References: <52CC01350200007800111164@nat28.tlf.novell.com> <1389178188.4883.19.camel@kazak.uk.xensource.com> <52CD3DBF02000078001117A2@nat28.tlf.novell.com> <1389179573.4883.30.camel@kazak.uk.xensource.com> <1389180290.4883.34.camel@kazak.uk.xensource.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1W0sfc-0000N3-Gv for xen-devel@lists.xenproject.org; Wed, 08 Jan 2014 12:55:40 +0000 Content-Disposition: inline In-Reply-To: <1389180290.4883.34.camel@kazak.uk.xensource.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Campbell Cc: George Dunlap , xen-devel , Stefano Stabellini , ian.jackson@eu.citrix.com, Jan Beulich List-Id: xen-devel@lists.xenproject.org On Wed, Jan 08, 2014 at 11:24:50AM +0000, Ian Campbell wrote: > On Wed, 2014-01-08 at 11:12 +0000, Ian Campbell wrote: > > On Wed, 2014-01-08 at 10:59 +0000, Jan Beulich wrote: > > > >>> On 08.01.14 at 11:49, Ian Campbell wrote: > > > > On Tue, 2014-01-07 at 12:29 +0000, Jan Beulich wrote: > > > >> >>> On 06.01.14 at 10:36, xen.org wrote: > > > >> > flight 24250 xen-unstable real [real] > > > >> > http://www.chiark.greenend.org.uk/~xensrcts/logs/24250/ > > > >> > > > > >> > Failures :-/ but no regressions. > > > >> > > > > >> > Regressions which are regarded as allowable (not blocking): > > > >> > test-armhf-armhf-xl 7 debian-install fail like > > > > 24146 > > > >> > test-amd64-i386-xl-winxpsp3-vcpus1 7 windows-install fail like > > > > 23938 > > > >> > test-amd64-i386-xl-win7-amd64 7 windows-install fail like > > > > 24146 > > > >> > > > >> These windows-install failures have been pretty persistent for > > > >> the last month or two. I've been looking at the logs from the > > > >> hypervisor side a number of times without spotting anything. It'd > > > >> be nice to know whether anyone also did so from the tools and > > > >> qemu sides... In any event we will need to do something about > > > >> this before 4.4 goes out. > > > > > > > > http://www.chiark.greenend.org.uk/~xensrcts/logs/24250/test-amd64-i386-xl-wi > > > > n7-amd64/win.guest.osstest--vnc.jpeg > > > > > > > > says that Windows experienced an unexpected error. > > > > > > > > http://www.chiark.greenend.org.uk/~xensrcts/logs/24250/test-amd64-i386-xl-wi > > > > nxpsp3-vcpus1/win.guest.osstest--vnc.jpeg > > > > > > > > is a blue screen "BAD_POOL_CALLER". > > > > FWIW It's code 0x7 which > > http://msdn.microsoft.com/en-us/library/windows/hardware/ff560185%28v=vs.85%29.aspx > > says is "The current thread attempted to free the pool, which was > > already freed.". > > > > The Internet(tm) seems to think this is often a driver issue. Not that > > this helps us much! > > > > > > I think this is unlikely to be a toolstack thing, but as to whether it > > > > is a Xen or a Windows issue I wouldn't like to say. I had a look through > > > > the toolstack logs anyway and didn't see anything untoward. > > > > > > Right, neither did I. I was particularly thinking of qemu though, > > > since I think these pretty persistent failures started around the > > > time the qemu tree upgrade was done. Of course this could just > > > be coincidence with a hypervisor side change having bad effects. > > > > If there is a correlation then it would be interesting to investigate -- > > Anthony/Stefano could you guys have a look please. > > The earliest instance I could see with logs was 22371 (10/12/2013). > > 21288 (30/10/2013) has a windows-install failure but the logs have > expired so I cannot tell if it was the same failure. > > I didn't look at the vnc for every failure I spotted, there were a lot > like these, and a lot where Windows was just sat at its login screen > (quite long standing issue I think? Thought not to be us?). > > There were a smattering of other failures too e.g.: > http://www.chiark.greenend.org.uk/~xensrcts/logs/22466/test-amd64-i386-xl-win7-amd64/win.guest.osstest--vnc.jpeg > http://www.chiark.greenend.org.uk/~xensrcts/logs/22455/test-amd64-i386-xl-win7-amd64/win.guest.osstest--vnc.jpeg There is also this one: http://www.chiark.greenend.org.uk/~xensrcts/logs/23724/test-amd64-amd64-xl-qemuu-winxpsp3/win.guest.osstest--vnc.jpeg an issue with ntfs.sys. Or this one: http://www.chiark.greenend.org.uk/~xensrcts/logs/23035/test-amd64-i386-xl-winxpsp3-vcpus1/win.guest.osstest--vnc.jpeg which say "a device driver has pool" and google respond: http://www.faultwire.com/solutions-fatal_error/A-device-driver-has-pool-0x000000C5-*1198.html "A device driver has a bug that attempted to access memory either nonexistent memory or memory it is not allowed to access." So, maybe a emulated disk issue or memory issue ? I'll try to reproduce the issue. > (bearing in mind that I didn't check all of them, if there was an easy > way to data mine the screen shots for all the failures of this test into > a directory it might be easier to scan) -- Anthony PERARD