Unresolved - qemu-dm crash under high dom0 disk load

Unresolved - qemu-dm crash under high dom0 disk load

[Randy McAnally]

This is regarding a long unresolved issue with HVM qemu-dm crashing under 
heavy Dom0 IO load.

I decided to make a new thread on this seeing that I found many old ones 
scattered about with no resolution: 

Here are old threads regarding the same/similar issue 

http://lists.xensource.com/archives/html/xen-users/2007-08/msg00135.html 
http://lists.xensource.com/archives/html/xen-users/2007-07/msg00624.html 

Old bug reports of the issue: 

http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=542 
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1038 

In 1038, spam@abma.de _incorrectly_ reports it is fixed in 3.1.1 -- Not only 
can I find any evidence of this, I can EASILY reproduce the crash over and 
over with 3.1.3-rc2. 

I have a dev. Xen box available for anyone who wants access for stack 
traces, ect.

Unfortunately I myself do not have enough knowledge to find the problem 
myself, however I am available to instruct on how to produce the bug, run 
stack traces, help organize this effort, and provide unrestricted access to 
this dev machine if needed.

-- 
Randy McAnally
Fast Serv Networks, LLC
619-819-8252 x201
619-819-9443 fax

Re: Unresolved - qemu-dm crash under high dom0 disk load

[Keir Fraser]

[-- Attachment #1: Type: text/plain, Size: 1226 bytes --]

Please try the attached patch.

 -- Keir

On 30/1/08 16:09, "Randy McAnally" <rsm@fast-serv.com> wrote:

> This is regarding a long unresolved issue with HVM qemu-dm crashing under
> heavy Dom0 IO load.
> 
> I decided to make a new thread on this seeing that I found many old ones
> scattered about with no resolution:
> 
> Here are old threads regarding the same/similar issue
> 
> http://lists.xensource.com/archives/html/xen-users/2007-08/msg00135.html
> http://lists.xensource.com/archives/html/xen-users/2007-07/msg00624.html
> 
> Old bug reports of the issue:
> 
> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=542
> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1038
> 
> In 1038, spam@abma.de _incorrectly_ reports it is fixed in 3.1.1 -- Not only
> can I find any evidence of this, I can EASILY reproduce the crash over and
> over with 3.1.3-rc2.
> 
> I have a dev. Xen box available for anyone who wants access for stack
> traces, ect.
> 
> Unfortunately I myself do not have enough knowledge to find the problem
> myself, however I am available to instruct on how to produce the bug, run
> stack traces, help organize this effort, and provide unrestricted access to
> this dev machine if needed.


[-- Attachment #2: ioemu-ide-thread-race --]
[-- Type: application/octet-stream, Size: 680 bytes --]

Fix race where a bm can get kicked off twice, resulting in an ioemu crash
because the bm is no longer valid.

diff -r a2b4ffd594a8 tools/ioemu/hw/ide.c
--- a/tools/ioemu/hw/ide.c	Thu Jun 14 14:26:39 2007 +0100
+++ b/tools/ioemu/hw/ide.c	Thu Jun 14 14:30:51 2007 +0100
@@ -427,7 +427,8 @@ static void *dma_thread_func(void* opaqu
         if (rv != 0) {
             if (read(file_pipes[0], &req, sizeof(req)) == 0)
                 return NULL;
-            dma_thread_loop(req);
+	    if (req->status & BM_STATUS_DMAING)
+		dma_thread_loop(req);
         } else {
             if (suspend_requested)  {
                 /* Need to tidy up the DMA thread so that we don't end up 

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Unresolved - qemu-dm crash under high dom0 disk load

[Randy McAnally]

So far, so good.  Haven't been able to crash a DomU.

-- 
Randy McAnally
Fast Serv Networks, LLC
619-819-8252 x201
619-819-9443 fax


---------- Original Message -----------
From: Keir Fraser <Keir.Fraser@cl.cam.ac.uk>
To: Randy McAnally <rsm@fast-serv.com>, <xen-devel@lists.xensource.com>
Sent: Wed, 30 Jan 2008 16:32:36 +0000
Subject: Re: [Xen-devel] Unresolved - qemu-dm crash under high dom0 disk load

> Please try the attached patch.
> 
>  -- Keir
> 
> On 30/1/08 16:09, "Randy McAnally" <rsm@fast-serv.com> wrote:
> 
> > This is regarding a long unresolved issue with HVM qemu-dm crashing under
> > heavy Dom0 IO load.
> > 
> > I decided to make a new thread on this seeing that I found many old ones
> > scattered about with no resolution:
> > 
> > Here are old threads regarding the same/similar issue
> > 
> > http://lists.xensource.com/archives/html/xen-users/2007-08/msg00135.html
> > http://lists.xensource.com/archives/html/xen-users/2007-07/msg00624.html
> > 
> > Old bug reports of the issue:
> > 
> > http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=542
> > http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1038
> > 
> > In 1038, spam@abma.de _incorrectly_ reports it is fixed in 3.1.1 -- Not 
only
> > can I find any evidence of this, I can EASILY reproduce the crash over 
and
> > over with 3.1.3-rc2.
> > 
> > I have a dev. Xen box available for anyone who wants access for stack
> > traces, ect.
> > 
> > Unfortunately I myself do not have enough knowledge to find the problem
> > myself, however I am available to instruct on how to produce the bug, run
> > stack traces, help organize this effort, and provide unrestricted access 
to
> > this dev machine if needed.
------- End of Original Message -------

RE: Unresolved - qemu-dm crash under high dom0 disk load

[Dan Magenheimer]

If this solves the problem, it would be nice to slide it into 3.1.3 final.

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com]On Behalf Of Keir Fraser
> Sent: Wednesday, January 30, 2008 9:33 AM
> To: Randy McAnally; xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] Unresolved - qemu-dm crash under 
> high dom0 disk
> load
> 
> 
> Please try the attached patch.
> 
>  -- Keir
> 
> On 30/1/08 16:09, "Randy McAnally" <rsm@fast-serv.com> wrote:
> 
> > This is regarding a long unresolved issue with HVM qemu-dm 
> crashing under
> > heavy Dom0 IO load.
> >
> > I decided to make a new thread on this seeing that I found 
> many old ones
> > scattered about with no resolution:
> >
> > Here are old threads regarding the same/similar issue
> >
> > 
> http://lists.xensource.com/archives/html/xen-users/2007-08/msg
00135.html
> http://lists.xensource.com/archives/html/xen-users/2007-07/msg00624.html
>
> Old bug reports of the issue:
>
> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=542
> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1038
>
> In 1038, spam@abma.de _incorrectly_ reports it is fixed in 3.1.1 -- Not only
> can I find any evidence of this, I can EASILY reproduce the crash over and
> over with 3.1.3-rc2.
>
> I have a dev. Xen box available for anyone who wants access for stack
> traces, ect.
>
> Unfortunately I myself do not have enough knowledge to find the problem
> myself, however I am available to instruct on how to produce the bug, run
> stack traces, help organize this effort, and provide unrestricted access to
> this dev machine if needed.

Re: Unresolved - qemu-dm crash under high dom0 disk load

[Keir Fraser]

Sure. I'm confident to slip this one in without doing another rc.

 -- Keir

On 30/1/08 17:37, "Dan Magenheimer" <dan.magenheimer@oracle.com> wrote:

> If this solves the problem, it would be nice to slide it into 3.1.3 final.
> 
>> -----Original Message-----
>> From: xen-devel-bounces@lists.xensource.com
>> [mailto:xen-devel-bounces@lists.xensource.com]On Behalf Of Keir Fraser
>> Sent: Wednesday, January 30, 2008 9:33 AM
>> To: Randy McAnally; xen-devel@lists.xensource.com
>> Subject: Re: [Xen-devel] Unresolved - qemu-dm crash under
>> high dom0 disk
>> load
>> 
>> 
>> Please try the attached patch.
>> 
>>  -- Keir
>> 
>> On 30/1/08 16:09, "Randy McAnally" <rsm@fast-serv.com> wrote:
>> 
>>> This is regarding a long unresolved issue with HVM qemu-dm
>> crashing under
>>> heavy Dom0 IO load.
>>> 
>>> I decided to make a new thread on this seeing that I found
>> many old ones
>>> scattered about with no resolution:
>>> 
>>> Here are old threads regarding the same/similar issue
>>> 
>>> 
>> http://lists.xensource.com/archives/html/xen-users/2007-08/msg
> 00135.html
>> http://lists.xensource.com/archives/html/xen-users/2007-07/msg00624.html
>> 
>> Old bug reports of the issue:
>> 
>> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=542
>> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1038
>> 
>> In 1038, spam@abma.de _incorrectly_ reports it is fixed in 3.1.1 -- Not only
>> can I find any evidence of this, I can EASILY reproduce the crash over and
>> over with 3.1.3-rc2.
>> 
>> I have a dev. Xen box available for anyone who wants access for stack
>> traces, ect.
>> 
>> Unfortunately I myself do not have enough knowledge to find the problem
>> myself, however I am available to instruct on how to produce the bug, run
>> stack traces, help organize this effort, and provide unrestricted access to
>> this dev machine if needed.
> 
> 
> 

Re: Unresolved - qemu-dm crash under high dom0 disk load

[Randy McAnally]

This certainly solves the issues I've been having.  Thanks so much for all 
your hard work, Keir.

-- 
Randy McAnally
Fast Serv Networks, LLC
619-819-8252 x201
619-819-9443 fax


---------- Original Message -----------
From: Keir Fraser <Keir.Fraser@cl.cam.ac.uk>
To: "dan.magenheimer@oracle.com" <dan.magenheimer@oracle.com>, Randy 
McAnally <rsm@fast-serv.com>, "xen-devel@lists.xensource.com" <xen-
devel@lists.xensource.com>
Sent: Wed, 30 Jan 2008 17:40:22 +0000
Subject: Re: [Xen-devel] Unresolved - qemu-dm crash under high dom0 disk load

> Sure. I'm confident to slip this one in without doing another rc.
> 
>  -- Keir
> 
> On 30/1/08 17:37, "Dan Magenheimer" <dan.magenheimer@oracle.com> wrote:
> 
> > If this solves the problem, it would be nice to slide it into 3.1.3 
final.
> > 
> >> -----Original Message-----
> >> From: xen-devel-bounces@lists.xensource.com
> >> [mailto:xen-devel-bounces@lists.xensource.com]On Behalf Of Keir Fraser
> >> Sent: Wednesday, January 30, 2008 9:33 AM
> >> To: Randy McAnally; xen-devel@lists.xensource.com
> >> Subject: Re: [Xen-devel] Unresolved - qemu-dm crash under
> >> high dom0 disk
> >> load
> >> 
> >> 
> >> Please try the attached patch.
> >> 
> >>  -- Keir
> >> 
> >> On 30/1/08 16:09, "Randy McAnally" <rsm@fast-serv.com> wrote:
> >> 
> >>> This is regarding a long unresolved issue with HVM qemu-dm
> >> crashing under
> >>> heavy Dom0 IO load.
> >>> 
> >>> I decided to make a new thread on this seeing that I found
> >> many old ones
> >>> scattered about with no resolution:
> >>> 
> >>> Here are old threads regarding the same/similar issue
> >>> 
> >>> 
> >> http://lists.xensource.com/archives/html/xen-users/2007-08/msg
> > 00135.html
> >> http://lists.xensource.com/archives/html/xen-users/2007-07/msg00624.html
> >> 
> >> Old bug reports of the issue:
> >> 
> >> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=542
> >> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1038
> >> 
> >> In 1038, spam@abma.de _incorrectly_ reports it is fixed in 3.1.1 -- Not 
only
> >> can I find any evidence of this, I can EASILY reproduce the crash over 
and
> >> over with 3.1.3-rc2.
> >> 
> >> I have a dev. Xen box available for anyone who wants access for stack
> >> traces, ect.
> >> 
> >> Unfortunately I myself do not have enough knowledge to find the problem
> >> myself, however I am available to instruct on how to produce the bug, 
run
> >> stack traces, help organize this effort, and provide unrestricted 
access to
> >> this dev machine if needed.
> > 
> > 
> >
------- End of Original Message -------