Libvir: a simple C virtualization control library

Libvir: a simple C virtualization control library

[Daniel Veillard]

  This mail is to present a new project being started:

       Libvir: a simple C virtualization control library

 The libvir library is born from the need for a simpler userland C library
to watch and control Xen domains. Among the design goal are:

   - being able to provide API and ABI guarantee
   - LGPL to be able to use it in a variety of contexts
   - pure C, minimizing dependancies
   - aiming at full documentation coverage

 A typical example of use case should be the applet displaying local Xen
domains status in Fedora Core desktop.

 The current state is a small library to get informations and interract
with existing domains, the design is not frozen, though the existing code
should work as is, it is considered mostly as a way to bootstrap and
seed the project, we are seeking interest from others. The library could
be extended in various ways potentally supporting other virtualization
mechanisms like QEmu, allowing to start new domains, etc. as long as the
API is kept simple enough.

 The project is hosted at :
     http://libvir.org/
API  http://libvir.org/html/libvir-libvir.html

See the download page at http://libvir.org/downloads.html to get sources
or CVS checkout, and https://www.redhat.com/mailman/listinfo/libvir-list
for the mailing-list informations, which would be the best place to discuss
this for those interested in this project,

   yours,

Daniel

-- 
Daniel Veillard      | Red Hat http://redhat.com/
veillard@redhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

Re: Libvir: a simple C virtualization control library

[Hollis Blanchard]

On Thursday 15 December 2005 09:03, Daniel Veillard wrote:
>  The libvir library is born from the need for a simpler userland C library
> to watch and control Xen domains.

I'm curious why libxc isn't good enough. Is the emphasis here on "simpler"? 
From what I've seen of it so far, I'm not sure I'd call libxc overly 
complicated...

The reason I'm interested is that right now the PPC port is carrying some 
libxc hacks for domain creation, which already have caused merge conflicts. 
There's no pressing need for us to throw out our hacks at the moment, but 
longer term if it's difficult for us to fit into libxc then maybe libvir 
would be a better fit.

-- 
Hollis Blanchard
IBM Linux Technology Center

Re: Libvir: a simple C virtualization control library

[Daniel Veillard]

On Thu, Dec 15, 2005 at 11:26:12AM -0600, Hollis Blanchard wrote:
> On Thursday 15 December 2005 09:03, Daniel Veillard wrote:
> >  The libvir library is born from the need for a simpler userland C library
> > to watch and control Xen domains.
> 
> I'm curious why libxc isn't good enough. Is the emphasis here on "simpler"? 
> >From what I've seen of it so far, I'm not sure I'd call libxc overly 
> complicated...

  I would say simpler to use, I'm not really targetting the same kind of
developpers I guess application and tools developpers not system programmers.
To me libxc is very low level, the high level abstractions are available on
top of the python classes but not at the C level. Basically if you want to
reuse Xen at the application level, you are pushed to Python + GPL which
is not necessarily an easy spot to stay in.

> The reason I'm interested is that right now the PPC port is carrying some 
> libxc hacks for domain creation, which already have caused merge conflicts. 
> There's no pressing need for us to throw out our hacks at the moment, but 
> longer term if it's difficult for us to fit into libxc then maybe libvir 
> would be a better fit.

  I don't think of libvir as a replacement for libxc, so a-priori I'm not sure
it really fits, especially as libvir has no domain creation API yet, but the
library will go where the user base will drive it.

Daniel

-- 
Daniel Veillard      | Red Hat http://redhat.com/
veillard@redhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

Re: Libvir: a simple C virtualization control library

[Anthony Liguori]

Hi Daniel,

I think the goal of having a library that allows applications to 
interact with Xen is a great idea.

I suggest though that instead of taking the current approach of 
reimplementing a hypercall library, you consider just using Xend's HTTP 
interface.

xm/xend communicate via an web services API that's based on 
S-Expressions.  Xend can listen on a domain socket and/or a TCP socket 
for incoming connections so both transports should be supported.

You can access this interface by doing an HTTP request with the 
Content-Type set to application/sxp.  The protocol uses the URL to 
identify the command and options and returns s-expressions.  You can get 
a feeling for the supported commands by enabling the TCP server and 
going to http://localhost:8000/xend/domain/

This would allow you to have an LGPL control library yet not rewrite all 
of Xend.  It also requires minimal dependencies since I believe libxml2 
already has an HTTP client (though you'd have to add a domain socket 
transport).

An interesting thing to do too would be to attempt to add support to 
Xend for another mime type (like text/xml) and perhaps even support 
XML-RPC.  This would be really excellent long term as it could eliminate 
a ton of code in Xend and xm by just reusing the python XML-RPC support.

Regards,

Anthony Liguori

Daniel Veillard wrote:

>  This mail is to present a new project being started:
>
>       Libvir: a simple C virtualization control library
>
> The libvir library is born from the need for a simpler userland C library
>to watch and control Xen domains. Among the design goal are:
>
>   - being able to provide API and ABI guarantee
>   - LGPL to be able to use it in a variety of contexts
>   - pure C, minimizing dependancies
>   - aiming at full documentation coverage
>
> A typical example of use case should be the applet displaying local Xen
>domains status in Fedora Core desktop.
>
> The current state is a small library to get informations and interract
>with existing domains, the design is not frozen, though the existing code
>should work as is, it is considered mostly as a way to bootstrap and
>seed the project, we are seeking interest from others. The library could
>be extended in various ways potentally supporting other virtualization
>mechanisms like QEmu, allowing to start new domains, etc. as long as the
>API is kept simple enough.
>
> The project is hosted at :
>     http://libvir.org/
>API  http://libvir.org/html/libvir-libvir.html
>
>See the download page at http://libvir.org/downloads.html to get sources
>or CVS checkout, and https://www.redhat.com/mailman/listinfo/libvir-list
>for the mailing-list informations, which would be the best place to discuss
>this for those interested in this project,
>
>   yours,
>
>Daniel
>
>  
>

Re: Libvir: a simple C virtualization control library

[Daniel Veillard]

On Sun, Dec 18, 2005 at 01:41:03PM -0600, Anthony Liguori wrote:
> Hi Daniel,

  Hi Anthony,

> I think the goal of having a library that allows applications to 
> interact with Xen is a great idea.

  thanks !

> I suggest though that instead of taking the current approach of 
> reimplementing a hypercall library, you consider just using Xend's HTTP 
> interface.
> 
> xm/xend communicate via an web services API that's based on 
> S-Expressions.  Xend can listen on a domain socket and/or a TCP socket 
> for incoming connections so both transports should be supported.
> 
> You can access this interface by doing an HTTP request with the 
> Content-Type set to application/sxp.  The protocol uses the URL to 
> identify the command and options and returns s-expressions.  You can get 
> a feeling for the supported commands by enabling the TCP server and 
> going to http://localhost:8000/xend/domain/

  Okay, I will check this specific API too, thanks :-) !

  One can take a technical look, there is pros and cons for both approaches,
for example querying another process though localhost TCP + HTTP + Python
marshalling + python interpreter and back to get state informations for
a domain when this could be a simple hypervisor call in pure C, well
the HTTP RPC approach does not sound fantastic. When creating a new domain,
I agree that the performance aspect is neglectible. From a technical view
again using directly the hypercalls introduce an ABI dependancy, which the
Xen authors may or may not want to garantee at this point.

  Then there is non-technical viewpoint, I am personally (I don't represent
my employer here) uneasy with a framework purely GPLed in user land
especially as a lot of the work is being done in user space. To me GPL is fine
for completely self-contained work with standard open APIs like a kernel can
be, the competition of ideas and implementations (which is the core factor
why OSS/free software end up being the best) happens either inside the project
or between projects reimplementing the standard APIs (e.g. Linux/BSD/Solaris).
But having a project GPLed without standard APIs hence creating an unicity of
the implementation by the nature of the licence I don't feel great about.
I understand the motivation to drive people to collabrate on a single
implementation when the project is bootstrapping, but using the licence as
a mechanism for this sounds wrong to me, Xen 3.0 and upcoming integration
in Linux main kernel show the project is getting ready to become mainstream,
and now is time to define the standard APIs and make sure the licence don't
constrain the adoption.

  Not having contributed yet to Xen, my viewpoint can be seen as presumptuous,
I am sorry for this, I recognize the huge amount of work which has been done
and is still being done, but ideally libvir should not exist because the 
problems of a standard, easy to reuse API for Xen should not exist. I don't
think a GPL library is an answer, Xen and virtualization in general will
be used by a variety of tools, and I want to see an healthy competition
of project bringing tools to the users, and not just corporate, cluster users.
I hope virtualization will reach the masses, and will integrate seamlessly
in the user's desktop, that's why I think Xen is fantastic, and why I think
we should not bet on a single collection of tools, getting there will require
open APIs and competing open-source projects building those users tools.

> This would allow you to have an LGPL control library yet not rewrite all 
> of Xend.  It also requires minimal dependencies since I believe libxml2 
> already has an HTTP client (though you'd have to add a domain socket 
> transport).

   libvir doesn't require libxml2 :-) the dependancy may be added later
if interfaces using XML as input get exposed though. There is also all
the security and reliability aspects of opening up an HTTP server with
side effect (even if limited to localhost) especially when it's is exposing
a full programming environment.

> An interesting thing to do too would be to attempt to add support to 
> Xend for another mime type (like text/xml) and perhaps even support 
> XML-RPC.  This would be really excellent long term as it could eliminate 
> a ton of code in Xend and xm by just reusing the python XML-RPC support.

  Using XML would force to tighten up some grey area, like for example
what is the encoding of a domain name string in a /etc/xen config file
(it's python so all bets are off at the moment). But this would add one more
layer of processing and dependancies. For something like creating a domain
or querying/changing long lived settings that would be just fine, but for
anything like monitoring this could be just too much.

   Thanks for the suggestion again, yes I will look at this too, this may
be the best technical solution in the current situation :-)

Daniel

-- 
Daniel Veillard      | Red Hat http://redhat.com/
veillard@redhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/