changeset:6989 vm sticking around in shutdown state

changeset:6989 vm sticking around in shutdown state

[Ted Kaczmarek]

changeset:6989 
FC4, Tyan 2462 SMP Athlon

Had a domU get stuck in shutdown state after issuing an init 0 from its
console.

Issuing an xm destroy had no affect.

On a good note, the memory assigned to it was given back and I cloned
the config with a new name and the domU was recreated. Also the 
xend-debug.log is very quite, meaning not a single line written to it.

It took a few restarts of that particular domU to make it happen.

Questions?

Is their any difference between issuing an init 0 at a domU's console
versus doing an xm shutdown from the domO?


Looking over the xend.log nothing is jumping out at me, what should I be
looking for if indeed their is something that should be their with
relation to the vm sticking around?


Best Regards,
Ted

Re: changeset:6989 vm sticking around in shutdown state

[Dan Smith]

TK> I cloned the config with a new name and the domU was recreated.

This brings me back to the topic of renaming stuck domains.  It seems
to me that having the tools prevent you from re-creating a domain just
because a shell of the domain is stuck will get annoying.  Admins may
end up cloning a config file and tweaking the name several times to
get the tools to allow recreation of one of their domUs without having
to reboot the host.

I know that the domains *shouldn't* get stuck once everything firms
up, but can't we have the tools be more user-friendly if the problem
does arise in a production environment?

How about adding a "force" flag to "xm create" that would rename any
stuck domains so that the creation can proceed?

-- 
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms@us.ibm.com

Re: changeset:6989 vm sticking around in shutdown state

[Ted Kaczmarek]

On Thu, 2005-09-22 at 06:57 -0700, Dan Smith wrote:
> TK> I cloned the config with a new name and the domU was recreated.
> 
> This brings me back to the topic of renaming stuck domains.  It seems
> to me that having the tools prevent you from re-creating a domain just
> because a shell of the domain is stuck will get annoying.  Admins may
> end up cloning a config file and tweaking the name several times to
> get the tools to allow recreation of one of their domUs without having
> to reboot the host.
> 
> I know that the domains *shouldn't* get stuck once everything firms
> up, but can't we have the tools be more user-friendly if the problem
> does arise in a production environment?
> 
> How about adding a "force" flag to "xm create" that would rename any
> stuck domains so that the creation can proceed?

Sound like a good starting point, you thinking about appending an
instance to the name? 

I am of the mindset that something like allow creation with the same
name, but invalidate the old one or vice versa. This would also keep
track of how many times such an issue occured on a particular domU.

xm list
Name              Id  Mem(MB)  CPU VCPU(s)  State  Time(s)
Domain-0           0      182    -      2   r----    970.7
debian		   3      128    -      2   -b---    502.4
debian.1	   3.1      0    -      2   ---s-     91.1

xm list
Name              Id  Mem(MB)  CPU VCPU(s)  State  Time(s)
Domain-0           0      182    -      2   r----    970.7
debian		   3        0    -      2   ---s-    502.4
debian.1	   3.1    128    -      2   -b---     91.1

I am still rather green with this, but from a dumb ___ admin/user
perspective this would be very helpful :-)

Regards,
Ted

Re: changeset:6989 vm sticking around in shutdown state

[Dan Smith]

TK> Sound like a good starting point, you thinking about appending an
TK> instance to the name?

We previously discussed adding something like -zombie-X to the name,
where X is the old numeric domid.  That would make it evident that the
domain was hanging around after death.

TK> I am of the mindset that something like allow creation with the
TK> same name, but invalidate the old one or vice versa. 

Yea, so, since Xend looks things up mostly by name, we need to create
the new domain with the real name, and mangle the name of the zombie
so that it mostly gets ignored.

TK> This would also keep track of how many times such an issue occured
TK> on a particular domU.

Well, it would keep track of stuck instances, true.  I don't think we
want to keep track of anything that we don't have to.  That's what the
logs are for :)

TK> Name           Id  Mem(MB)  CPU VCPU(s)  State  Time(s)
TK> Domain-0       0      182    -      2   r----    970.7
TK> debian	   3      128    -      2   -b---    502.4
TK> debian.1	   3.1      0    -      2   ---s-     91.1

I think it would/should look like this:
 
Name              Id  Mem(MB)  CPU VCPU(s)  State  Time(s)
Domain-0           0      182    -      2   r----    970.7
debian-zombie-3	   3        0    -      2   ---s-    502.4
debian  	   4      128    -      2   -b---     91.1

-- 
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms@us.ibm.com

RE: changeset:6989 vm sticking around in shutdown state

[Ian Pratt]

 > TK> Sound like a good starting point, you thinking about appending an

> TK> instance to the name?
> 
> We previously discussed adding something like -zombie-X to 
> the name, where X is the old numeric domid.  That would make 
> it evident that the domain was hanging around after death.

-zombie-domid seems like a good approach. I don't think anyone has
posted a patch yet...

Ian 

Re: changeset:6989 vm sticking around in shutdown state

[Dan Smith]

IP> -zombie-domid seems like a good approach. I don't think anyone has
IP> posted a patch yet...

I did:

http://lists.xensource.com/archives/html/xen-devel/2005-09/msg00565.html

-- 
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms@us.ibm.com