[Cluster-devel] virDomainDump() API (equivalent to xm dump) in libvirt?

[Cluster-devel] virDomainDump() API (equivalent to xm dump) in libvirt?

[Lon Hohberger]

There's already one for saving live domains:

int virDomainSave(virDomainPtr domain, const char *to);

Why not use a similar API for a function which does the equivalent of
'xm dump' ?

int virDomainDump(virDomainPtr domain, const char *to, int flags);

Create a kernel dump of the domain in the path contained in *to.  This
API must fail if the domain is not in a panicked/crashed state.

If you set the lowest-bit in flags, however, you can override this
behavior: if the lowest bit is set, the domain will first be crashed
(similar to SysRq-C) - then a dump will be taken from that.


Why?

In the event that a cluster node needs to fence a Xen domain (or other
management tools, to be sure), we could use the API to generate a core
of the domain (if it panicked) to help debug the problem prior to
restarting the domain.

The 'crash-before-dump' bit is just something someone might want to use
at some point; I don't think it is absolutely necessary - but it doesn't
hurt anything either.

-- Lon

[Cluster-devel] Re: [Libvir] virDomainDump() API (equivalent to xm dump) in libvirt?

[Daniel Veillard]

On Fri, Nov 03, 2006 at 10:38:18AM -0500, Lon Hohberger wrote:
> There's already one for saving live domains:
> 
> int virDomainSave(virDomainPtr domain, const char *to);
> 
> Why not use a similar API for a function which does the equivalent of
> 'xm dump' ?

  I assume you mean 'xm dump-core' actually 

> int virDomainDump(virDomainPtr domain, const char *to, int flags);
> 
> Create a kernel dump of the domain in the path contained in *to.  This
> API must fail if the domain is not in a panicked/crashed state.
> If you set the lowest-bit in flags, however, you can override this
> behavior: if the lowest bit is set, the domain will first be crashed
> (similar to SysRq-C) - then a dump will be taken from that.
> 
> 
> Why?
> 
> In the event that a cluster node needs to fence a Xen domain (or other
> management tools, to be sure), we could use the API to generate a core
> of the domain (if it panicked) to help debug the problem prior to
> restarting the domain.
> 
> The 'crash-before-dump' bit is just something someone might want to use
> at some point; I don't think it is absolutely necessary - but it doesn't
> hurt anything either.

  Okay I can see how this would be useful, the questions I would have would be:
    - how generic is this, i.e. suppose a different hypervisor back-end
      would this still make sense. I guess yes, for example with an UML
      back-end we could check the process status and force a dump with a
      signal and move the core to the given file not trivial but same semantic
      would be doable.
    - can we implement it with current xen, again yes, we should be able
      if we have a full connection (root) to do the same as 'xm dump-core'
    - is the API clean enough, I guess the semantic is relatively clear
      instead of stating 'If you set the lowest-bit in flags' I would rather
      define a DumpFlags enum and state that flags is an or'ing of any of them
      I would probably name the function virDomainDumpCore though to not
      confuse with virDomainSave 

 So yes, why not, you want to work on it ? Or should I (or any candidate).

Daniel

-- 
Red Hat Virtualization group http://redhat.com/virtualization/
Daniel Veillard      | virtualization library  http://libvirt.org/
veillard at redhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine  http://rpmfind.net/

[Cluster-devel] Re: [Libvir] virDomainDump() API (equivalent to xm dump) in libvirt?

[Lon Hohberger]

On Fri, 2006-11-03 at 10:53 -0500, Daniel Veillard wrote:

>   Okay I can see how this would be useful, the questions I would have would be:
>     - how generic is this, i.e. suppose a different hypervisor back-end
>       would this still make sense. I guess yes, for example with an UML
>       back-end we could check the process status and force a dump with a
>       signal and move the core to the given file not trivial but same semantic
>       would be doable.

This I don't know a lot about.  I hadn't considered the implications of
using this on non-Xen guests. :o  I think I agree with you, though.  The
only weird part I can think of with UML is that because it behaves like
a normal process, we'd have to tweak the process's core limit before
causing it to dump...?

>     - can we implement it with current xen, again yes, we should be able
>       if we have a full connection (root) to do the same as 'xm dump-core'

Yes, given that xm dump-core can do it ;)

>     - is the API clean enough, I guess the semantic is relatively clear
>       instead of stating 'If you set the lowest-bit in flags' I would rather
>       define a DumpFlags enum and state that flags is an or'ing of any of them
>       I would probably name the function virDomainDumpCore though to not
>       confuse with virDomainSave 

Totally correct here, it should be an enum of flags (with only one
defined initially; I don't know what other flags there might be.

>  So yes, why not, you want to work on it ? Or should I (or any candidate).

Give me a few; I'll check on this.  I don't have problems trying it, but
I will have a learning curve of course.

-- Lon

[Cluster-devel] Re: [Libvir] virDomainDump() API (equivalent to xm dump) in libvirt?

[Lon Hohberger]

On Fri, 2006-11-03 at 11:36 -0500, Lon Hohberger wrote:
> >  So yes, why not, you want to work on it ? Or should I (or any candidate).
> 
> Give me a few; I'll check on this.  I don't have problems trying it, but
> I will have a learning curve of course.
> 

Right now, I don't have the time to ramp up my knowledge of libvirt /
xend bits :(

-- Lon

[Cluster-devel] Re: [Libvir] virDomainDump() API (equivalent to xm dump) in libvirt?

[Daniel Veillard]

On Fri, Nov 03, 2006 at 11:36:32AM -0500, Lon Hohberger wrote:
> On Fri, 2006-11-03 at 10:53 -0500, Daniel Veillard wrote:
> >     - can we implement it with current xen, again yes, we should be able
> >       if we have a full connection (root) to do the same as 'xm dump-core'
> 
> Yes, given that xm dump-core can do it ;)
> 
> >     - is the API clean enough, I guess the semantic is relatively clear
> >       instead of stating 'If you set the lowest-bit in flags' I would rather
> >       define a DumpFlags enum and state that flags is an or'ing of any of them
> >       I would probably name the function virDomainDumpCore though to not
> >       confuse with virDomainSave 
> 
> Totally correct here, it should be an enum of flags (with only one
> defined initially; I don't know what other flags there might be.
> 
> >  So yes, why not, you want to work on it ? Or should I (or any candidate).
> 
> Give me a few; I'll check on this.  I don't have problems trying it, but
> I will have a learning curve of course.

  Let's see if I was to do it I would:
    0/ check out a CVS tree http://libvirt.org/downloads.html
    1/ look how xm communicate with xend on that command
    2/ isolate another similar entry point from xend_internal.c and 
       see how to generate the same kind of command but for the core dump
       for example xenDaemonDomainSave ()
       test by linking statically from a small test program
 then the next steps are rather integration problems
    3/ extend the back-end driver to add a DumpCore entry point and the
       enum in the header (beware it's include/libvirt/libvirt.h.in)
    4/ modify all the drivers to have NULL entry points except for the
       virDriver xenDaemonDriver one adding the new function at the end.
    5/ create the real entry point in libvirt.c, add the initial check for
       the flag and the domain status

 basically if you just have 1 and 2, sending the diff here I can take care
quickly of the purely integration details,

  hope this helps,

Daniel

-- 
Red Hat Virtualization group http://redhat.com/virtualization/
Daniel Veillard      | virtualization library  http://libvirt.org/
veillard at redhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine  http://rpmfind.net/

[Cluster-devel] Re: [Libvir] virDomainDump() API (equivalent to xm dump) in libvirt?

[Daniel Veillard]

On Fri, Nov 03, 2006 at 11:56:55AM -0500, Daniel Veillard wrote:
> On Fri, Nov 03, 2006 at 11:36:32AM -0500, Lon Hohberger wrote:
> > On Fri, 2006-11-03 at 10:53 -0500, Daniel Veillard wrote:
> > >     - can we implement it with current xen, again yes, we should be able
> > >       if we have a full connection (root) to do the same as 'xm dump-core'
> > 
> > Yes, given that xm dump-core can do it ;)
> > 
> > >     - is the API clean enough, I guess the semantic is relatively clear
> > >       instead of stating 'If you set the lowest-bit in flags' I would rather
> > >       define a DumpFlags enum and state that flags is an or'ing of any of them
> > >       I would probably name the function virDomainDumpCore though to not
> > >       confuse with virDomainSave 
> > 
> > Totally correct here, it should be an enum of flags (with only one
> > defined initially; I don't know what other flags there might be.
> > 
> > >  So yes, why not, you want to work on it ? Or should I (or any candidate).
> > 
> > Give me a few; I'll check on this.  I don't have problems trying it, but
> > I will have a learning curve of course.
> 
>   Let's see if I was to do it I would:
>     0/ check out a CVS tree http://libvirt.org/downloads.html
>     1/ look how xm communicate with xend on that command

  Bad news, I looked at this and it seems xm can only use the XML-RPC interface
to xend to request the dump command, while libvirt uses the HTTP POST interface.
Adding the support would either require to hack xend or add support for the
intermediary xml-rpc access which was added in 3.0.2 and likely to disapear soon
when the new Xen API is used by xm.

  Unfortunately more complex than one could think initially :-\

Daniel

-- 
Red Hat Virtualization group http://redhat.com/virtualization/
Daniel Veillard      | virtualization library  http://libvirt.org/
veillard at redhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine  http://rpmfind.net/

[Cluster-devel] Re: [Libvir] virDomainDump() API (equivalent to xm dump) in libvirt?

[Daniel Veillard]

On Thu, Nov 16, 2006 at 10:36:29AM -0500, Daniel Veillard wrote:
> On Fri, Nov 03, 2006 at 11:56:55AM -0500, Daniel Veillard wrote:
> > On Fri, Nov 03, 2006 at 11:36:32AM -0500, Lon Hohberger wrote:
> > > On Fri, 2006-11-03 at 10:53 -0500, Daniel Veillard wrote:
> > > >     - can we implement it with current xen, again yes, we should be able
> > > >       if we have a full connection (root) to do the same as 'xm dump-core'
> > > 
> > > Yes, given that xm dump-core can do it ;)
> > > 
> > > >     - is the API clean enough, I guess the semantic is relatively clear
> > > >       instead of stating 'If you set the lowest-bit in flags' I would rather
> > > >       define a DumpFlags enum and state that flags is an or'ing of any of them
> > > >       I would probably name the function virDomainDumpCore though to not
> > > >       confuse with virDomainSave 
> > > 
> > > Totally correct here, it should be an enum of flags (with only one
> > > defined initially; I don't know what other flags there might be.
> > > 
> > > >  So yes, why not, you want to work on it ? Or should I (or any candidate).
> > > 
> > > Give me a few; I'll check on this.  I don't have problems trying it, but
> > > I will have a learning curve of course.
> > 
> >   Let's see if I was to do it I would:
> >     0/ check out a CVS tree http://libvirt.org/downloads.html
> >     1/ look how xm communicate with xend on that command
> 
>   Bad news, I looked at this and it seems xm can only use the XML-RPC interface
> to xend to request the dump command, while libvirt uses the HTTP POST interface.
> Adding the support would either require to hack xend or add support for the
> intermediary xml-rpc access which was added in 3.0.2 and likely to disapear soon
> when the new Xen API is used by xm.

  Good news: the fix at the xend level is fairly small and has been accepted
upstream:
  http://lists.xensource.com/archives/html/xen-devel/2006-11/msg01120.html
  http://lists.xensource.com/archives/html/xen-devel/2006-11/msg01121.html

  So adding dump support was a matter of introducing the new API and calling
the usual xend back-end code. I added it and commited it, it's a new entry
point in the driver interface, I still kept the extra flags argument, even if
upon inspection xend doesn't use any of the 'live' or 'crash' argument it
exposes at this API, I just didn't defined any constant for it at this point.
So the new entry is just:

  int  virDomainCoreDump (virDomainPtr domain, const char *to, int flags);

  I commited it to CVS, I also added the 'dump' command to virsh taking the
filename as the only argument (after the domain name or id).

Daniel

-- 
Red Hat Virtualization group http://redhat.com/virtualization/
Daniel Veillard      | virtualization library  http://libvirt.org/
veillard at redhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine  http://rpmfind.net/