DomU not rebooting in 3.0.1

DomU not rebooting in 3.0.1

[Brian Hays]

[-- Attachment #1.1: Type: text/plain, Size: 517 bytes --]

In 3.0.1 it appears that reboots from within a domU result in the domU
shutting down but not coming back up.

/var/log/xen-hotplug.log reports something similar to "xenstore-list: could
not read path backend/vbd/40"

Attempts to manually start the domU up again via "xm create" fail until
first running "xenstore-rm backend/vbd/40"

This has happened on several occasions within different domU's on different
machines. Is there a known workaround... Is this already fixed in unstable?

Thank you,
Brian

[-- Attachment #1.2: Type: text/html, Size: 600 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: DomU not rebooting in 3.0.1

[Ewan Mellor]

On Sat, Feb 25, 2006 at 11:31:57AM -0500, Brian Hays wrote:

> In 3.0.1 it appears that reboots from within a domU result in the domU
> shutting down but not coming back up.
> 
> /var/log/xen-hotplug.log reports something similar to "xenstore-list:
> could not read path backend/vbd/40"
> 
> Attempts to manually start the domU up again via "xm create" fail until
> first running "xenstore-rm backend/vbd/40"
> 
> This has happened on several occasions within different domU's on
> different machines. Is there a known workaround... Is this already fixed
> in unstable?

This is being tracked as bug #514.  I think I have a fix, which should be in
today, all being well.

Ewan.

Re: DomU not rebooting in 3.0.1

[Sean Dague]

[-- Attachment #1.1: Type: text/plain, Size: 1345 bytes --]

On Mon, Feb 27, 2006 at 03:33:17PM +0000, Ewan Mellor wrote:
> On Sat, Feb 25, 2006 at 11:31:57AM -0500, Brian Hays wrote:
> 
> > In 3.0.1 it appears that reboots from within a domU result in the domU
> > shutting down but not coming back up.
> > 
> > /var/log/xen-hotplug.log reports something similar to "xenstore-list:
> > could not read path backend/vbd/40"
> > 
> > Attempts to manually start the domU up again via "xm create" fail until
> > first running "xenstore-rm backend/vbd/40"
> > 
> > This has happened on several occasions within different domU's on
> > different machines. Is there a known workaround... Is this already fixed
> > in unstable?
> 
> This is being tracked as bug #514.  I think I have a fix, which should be in
> today, all being well.

Ewan, I didn't notice 514, but I submitted a related bug
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=549.  It appears that
the big deciding factor on whether or not I see the race is is the DomU that
is rebooting running on a fully independant CPU from Dom0.  If it is, it
breaks every time, if it isn't (i.e. it is on the *same* phys CPU as Dom0)
it works every time.

	-Sean

-- 
Sean Dague
IBM Linux Technology Center                     email: japh@us.ibm.com
Open Hypervisor Team                           alt: sldague@us.ibm.com

[-- Attachment #1.2: Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: DomU not rebooting in 3.0.1

[Brian Hays]

[-- Attachment #1.1: Type: text/plain, Size: 1582 bytes --]

My domU is on the same physical CPU as dom0 but still doesn't come back from
reboots.

Brian

On 2/27/06, Sean Dague <japh@us.ibm.com> wrote:
>
> On Mon, Feb 27, 2006 at 03:33:17PM +0000, Ewan Mellor wrote:
> > On Sat, Feb 25, 2006 at 11:31:57AM -0500, Brian Hays wrote:
> >
> > > In 3.0.1 it appears that reboots from within a domU result in the domU
> > > shutting down but not coming back up.
> > >
> > > /var/log/xen-hotplug.log reports something similar to "xenstore-list:
> > > could not read path backend/vbd/40"
> > >
> > > Attempts to manually start the domU up again via "xm create" fail
> until
> > > first running "xenstore-rm backend/vbd/40"
> > >
> > > This has happened on several occasions within different domU's on
> > > different machines. Is there a known workaround... Is this already
> fixed
> > > in unstable?
> >
> > This is being tracked as bug #514.  I think I have a fix, which should
> be in
> > today, all being well.
>
> Ewan, I didn't notice 514, but I submitted a related bug
> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=549.  It appears
> that
> the big deciding factor on whether or not I see the race is is the DomU
> that
> is rebooting running on a fully independant CPU from Dom0.  If it is, it
> breaks every time, if it isn't (i.e. it is on the *same* phys CPU as Dom0)
> it works every time.
>
>         -Sean
>
> --
> Sean Dague
> IBM Linux Technology Center                     email: japh@us.ibm.com
> Open Hypervisor Team                           alt: sldague@us.ibm.com
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 2421 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: DomU not rebooting in 3.0.1

[Ewan Mellor]

On Mon, Feb 27, 2006 at 02:06:14PM -0500, Sean Dague wrote:

> On Mon, Feb 27, 2006 at 03:33:17PM +0000, Ewan Mellor wrote:
> > On Sat, Feb 25, 2006 at 11:31:57AM -0500, Brian Hays wrote:
> > 
> > > In 3.0.1 it appears that reboots from within a domU result in the domU
> > > shutting down but not coming back up.
> > > 
> > > /var/log/xen-hotplug.log reports something similar to "xenstore-list:
> > > could not read path backend/vbd/40"
> > > 
> > > Attempts to manually start the domU up again via "xm create" fail until
> > > first running "xenstore-rm backend/vbd/40"
> > > 
> > > This has happened on several occasions within different domU's on
> > > different machines. Is there a known workaround... Is this already fixed
> > > in unstable?
> > 
> > This is being tracked as bug #514.  I think I have a fix, which should be in
> > today, all being well.
> 
> Ewan, I didn't notice 514, but I submitted a related bug
> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=549.  It appears that
> the big deciding factor on whether or not I see the race is is the DomU that
> is rebooting running on a fully independant CPU from Dom0.  If it is, it
> breaks every time, if it isn't (i.e. it is on the *same* phys CPU as Dom0)
> it works every time.

Please could you use xen-bugtool to attach your logs to that bug #549 -- I'm
not sure it's the same root cause.

Ewan.

Re: DomU not rebooting in 3.0.1

[Sean Dague]

[-- Attachment #1.1: Type: text/plain, Size: 1752 bytes --]

On Mon, Feb 27, 2006 at 07:13:53PM +0000, Ewan Mellor wrote:
> On Mon, Feb 27, 2006 at 02:06:14PM -0500, Sean Dague wrote:
> 
> > On Mon, Feb 27, 2006 at 03:33:17PM +0000, Ewan Mellor wrote:
> > > On Sat, Feb 25, 2006 at 11:31:57AM -0500, Brian Hays wrote:
> > > 
> > > > In 3.0.1 it appears that reboots from within a domU result in the domU
> > > > shutting down but not coming back up.
> > > > 
> > > > /var/log/xen-hotplug.log reports something similar to "xenstore-list:
> > > > could not read path backend/vbd/40"
> > > > 
> > > > Attempts to manually start the domU up again via "xm create" fail until
> > > > first running "xenstore-rm backend/vbd/40"
> > > > 
> > > > This has happened on several occasions within different domU's on
> > > > different machines. Is there a known workaround... Is this already fixed
> > > > in unstable?
> > > 
> > > This is being tracked as bug #514.  I think I have a fix, which should be in
> > > today, all being well.
> > 
> > Ewan, I didn't notice 514, but I submitted a related bug
> > http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=549.  It appears that
> > the big deciding factor on whether or not I see the race is is the DomU that
> > is rebooting running on a fully independant CPU from Dom0.  If it is, it
> > breaks every time, if it isn't (i.e. it is on the *same* phys CPU as Dom0)
> > it works every time.
> 
> Please could you use xen-bugtool to attach your logs to that bug #549 -- I'm
> not sure it's the same root cause.

No problem, xen-bugtool now run with files attached.

	-Sean

-- 
Sean Dague
IBM Linux Technology Center                     email: japh@us.ibm.com
Open Hypervisor Team                           alt: sldague@us.ibm.com

[-- Attachment #1.2: Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: DomU not rebooting in 3.0.1

[Tiago Teresa Teodosio]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sean Dague wrote:
> On Mon, Feb 27, 2006 at 07:13:53PM +0000, Ewan Mellor wrote:
> 
>>On Mon, Feb 27, 2006 at 02:06:14PM -0500, Sean Dague wrote:
>>
>>
>>>On Mon, Feb 27, 2006 at 03:33:17PM +0000, Ewan Mellor wrote:
>>>
>>>>On Sat, Feb 25, 2006 at 11:31:57AM -0500, Brian Hays wrote:
>>>>
>>>>
>>>>>In 3.0.1 it appears that reboots from within a domU result in the domU
>>>>>shutting down but not coming back up.
>>>>>
>>>>>/var/log/xen-hotplug.log reports something similar to "xenstore-list:
>>>>>could not read path backend/vbd/40"
>>>>>
>>>>>Attempts to manually start the domU up again via "xm create" fail until
>>>>>first running "xenstore-rm backend/vbd/40"
>>>>>
>>>>>This has happened on several occasions within different domU's on
>>>>>different machines. Is there a known workaround... Is this already fixed
>>>>>in unstable?
>>>>
>>>>This is being tracked as bug #514.  I think I have a fix, which should be in
>>>>today, all being well.
>>>
>>>Ewan, I didn't notice 514, but I submitted a related bug
>>>http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=549.  It appears that
>>>the big deciding factor on whether or not I see the race is is the DomU that
>>>is rebooting running on a fully independant CPU from Dom0.  If it is, it
>>>breaks every time, if it isn't (i.e. it is on the *same* phys CPU as Dom0)
>>>it works every time.

Hi

I had the same problem in 3.0.0 and the workaround I found was to assign
more than one vcpu to the DomU. (The Dom0 has only runs only on one cpu)

I left xen to assign the cpus to each VM so I can not tell if the fact
of running on the same cpu of the Dom0 has any effect.

If it is of any help, the physical machine is a Dual Xeon 3.0 GHz, so
the hypervisor sees 4 cpus.

Bye

>>
>>Please could you use xen-bugtool to attach your logs to that bug #549 -- I'm
>>not sure it's the same root cause.
> 
> 
> No problem, xen-bugtool now run with files attached.
> 
> 	-Sean
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


- --
Tiago Teresa Teodódio
tiago@rnl.ist.utl.pt

Administração da Rede das Novas Licenciaturas
Instituto Superior Técnico

web: http://www.rnl.ist.utl.pt
email: rnl@rnl.ist.utl.pt
tel: +351 218 41 77 71

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFEBirTQO/Tafxh35cRAhi3AKDNe9RHp7VLON7EP5j0TFR5FDz64ACg0C35
3fF0FG864dUmcm2hnNAUcgU=
=C+kk
-----END PGP SIGNATURE-----

Re: DomU not rebooting in 3.0.1

[Ewan Mellor]

On Mon, Feb 27, 2006 at 02:06:14PM -0500, Sean Dague wrote:

> On Mon, Feb 27, 2006 at 03:33:17PM +0000, Ewan Mellor wrote:
> > On Sat, Feb 25, 2006 at 11:31:57AM -0500, Brian Hays wrote:
> > 
> > > In 3.0.1 it appears that reboots from within a domU result in the domU
> > > shutting down but not coming back up.
> > > 
> > > /var/log/xen-hotplug.log reports something similar to "xenstore-list:
> > > could not read path backend/vbd/40"
> > > 
> > > Attempts to manually start the domU up again via "xm create" fail until
> > > first running "xenstore-rm backend/vbd/40"
> > > 
> > > This has happened on several occasions within different domU's on
> > > different machines. Is there a known workaround... Is this already fixed
> > > in unstable?
> > 
> > This is being tracked as bug #514.  I think I have a fix, which should be in
> > today, all being well.
> 
> Ewan, I didn't notice 514, but I submitted a related bug
> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=549.  It appears that
> the big deciding factor on whether or not I see the race is is the DomU that
> is rebooting running on a fully independant CPU from Dom0.  If it is, it
> breaks every time, if it isn't (i.e. it is on the *same* phys CPU as Dom0)
> it works every time.

Sean, your #549 is not the same as #514, because your /var/log/xen-hotplug.log
show neither the message "xenstore-list: could not read path
backend/vbd/40" nor "xenstore-list: could not read path /local/domain/1/vm".
One of these messages is expected if you are hitting bug #514.

Your xen-hotplug.log does show

/etc/xen/scripts/vif-bridge: line 41: brctl: command not found

Obviously, I would expect your vif to be broken with this message showing.  I
wouldn't expect your vbd to be broken, unless there's some cascading failure
that I don't know about.

Nothing jumps out at me otherwise from your logs, so you're going to have to
dig a bit more.  As a first suggestion, I'd run xenstore-ls with the guest up
but broken, and see if there are any error messages in the store.  Xend loses
the error messages when it reboots a domain, because there's no "wait for the
devices" phase in the reboot logic.

Ewan.