libxenlight or libxenheavy?

libxenlight or libxenheavy?

[Andres Lagar-Cavilla]

Aside of the tongue in cheek title, I'd like to get a feel for where is 
libxenlight going. I love having a library that gives me three 
straight-forward C calls to create a domain, and I think it's an 
excellent vehicle to writing control stacks.

But some of the latest patches have grown/bloated the library in 
directions I don't think are useful. This an obviously subjective take 
on the matter, but here are two examples:

- Managing tapdisk2 devices in libxenlight: why at all? An upper-level 
control stack can (will have to) vet the configuration stanza of the 
tapdisk2 process, and it can then launch it and manage its life-cycle 
(i.e. echo appropriate commands to the sysfs interface). One of the 
great advantages of tapdisk2 is that it looks like a regular block 
device: /dev/xen/blktap-2/tapdev0. Libxenlight doesn't need to know this 
is any different from a regular block device ...

- Asynchronous notifications via xenstore watches: I've seen at least 
two locations (device deletion during destroy and waiting for domain 
death) where a watch on a xenstore path is placed by the library, and 
later xs_read_watch is called. According to my limited understanding, 
this could read *any* firing watch placed by the same process, and the 
code will discard it unless it's the one we are looking for. Thus 
destroying information useful to someone else. I cannot have two 
concurrent (or even interleaved) calls to libxenlight on these code 
paths, because they could read each other's watches. Why not leave these 
to an upper-level stack, which in all likelihood will have to deal with 
lots of asynchronous events? As it stands, I have to write my code 
*around* libxenlight, which kind of defeats the purpose.

Anyway, these are two observations. Libxenlight is great. And to buy 
back some of the love I spent ranting, I'll follow this with 14 patches. 
These are all based off20522: abc6183f486e 
<http://xenbits.xen.org/xen-unstable.hg?rev/abc6183f486e>. Some are more 
RFC-ish in nature, but there a fair bit of fixes. They apply -p1 
(sorry). Just let me know if you need a rebasing or whatever.

Andres

Re: libxenlight or libxenheavy?

[Vincent Hanquez]

On Mon, Nov 30, 2009 at 07:08:47PM +0000, Andres Lagar-Cavilla wrote:
> Aside of the tongue in cheek title, I'd like to get a feel for where is 
> libxenlight going. I love having a library that gives me three 
> straight-forward C calls to create a domain, and I think it's an 
> excellent vehicle to writing control stacks.
> 
> But some of the latest patches have grown/bloated the library in 
> directions I don't think are useful. This an obviously subjective take 
> on the matter, but here are two examples:

I agree with your observation. the goal for libxenlight is not to provide
a full hand on solutions, but a bunch of helper functions to do one thing
and only one thing "plug a disk", "suspend a domain".
 
I think however that whilst it's not going only in the right direction,
the early stage of the library means that we need for testing a lot
of things that shouldn't be in the API.

once we reach a stage where everything works, i'm sure that integrating
into proper management stack will means that the API will simplify as
things become obsolete or useless.

> - Managing tapdisk2 devices in libxenlight: why at all? An upper-level 
> control stack can (will have to) vet the configuration stanza of the 
> tapdisk2 process, and it can then launch it and manage its life-cycle 
> (i.e. echo appropriate commands to the sysfs interface). One of the 
> great advantages of tapdisk2 is that it looks like a regular block 
> device: /dev/xen/blktap-2/tapdev0. Libxenlight doesn't need to know this 
> is any different from a regular block device ...

indeed.

> 
> - Asynchronous notifications via xenstore watches: I've seen at least 
> two locations (device deletion during destroy and waiting for domain 
> death) where a watch on a xenstore path is placed by the library, and 
> later xs_read_watch is called. According to my limited understanding, 
> this could read *any* firing watch placed by the same process, and the 
> code will discard it unless it's the one we are looking for. Thus 
> destroying information useful to someone else. I cannot have two 
> concurrent (or even interleaved) calls to libxenlight on these code 
> paths, because they could read each other's watches. Why not leave these 
> to an upper-level stack, which in all likelihood will have to deal with 
> lots of asynchronous events? As it stands, I have to write my code 
> *around* libxenlight, which kind of defeats the purpose.

This whole part is the mess, i wouldn't rely on the lib for now for thoses.
the opensource xs library need some cleanup in the watch processing department
in the first place.

-- 
Vincent

Re: libxenlight or libxenheavy?

[Vincent Hanquez]

Apart for the patches i commented one:

Acked-by: Vincent Hanquez <vincent.hanquez@eu.citrix.com>

Thanks !
-- 
Vincent

Re: libxenlight or libxenheavy?

[Stefano Stabellini]

On Mon, 30 Nov 2009, Andres Lagar-Cavilla wrote:
> Aside of the tongue in cheek title, I'd like to get a feel for where is 
> libxenlight going. I love having a library that gives me three 
> straight-forward C calls to create a domain, and I think it's an 
> excellent vehicle to writing control stacks.
> 
> But some of the latest patches have grown/bloated the library in 
> directions I don't think are useful. This an obviously subjective take 
> on the matter, but here are two examples:

First of all I am really glad that you brought this up, because it is
very important that we get this clear: libxenlight must remain
lightweight and small at all costs.
Obviously everyone has a different feeling about what's small and what's
not but that is the general idea.


> - Managing tapdisk2 devices in libxenlight: why at all? An upper-level 
> control stack can (will have to) vet the configuration stanza of the 
> tapdisk2 process, and it can then launch it and manage its life-cycle 
> (i.e. echo appropriate commands to the sysfs interface). One of the 
> great advantages of tapdisk2 is that it looks like a regular block 
> device: /dev/xen/blktap-2/tapdev0. Libxenlight doesn't need to know this 
> is any different from a regular block device ...

On this point I disagree with you: I am glad that blktap2 is simpler
but it shouldn't be threated differently from blktap1 on this basis.
Besides I wanted to be able to use blktap2 in place of blktap1
transparently whenever possible.
Finally, if your high level toolstack really wants, can still manage
tapdisk2 devices directly.


> - Asynchronous notifications via xenstore watches: I've seen at least 
> two locations (device deletion during destroy and waiting for domain 
> death) where a watch on a xenstore path is placed by the library, and 
> later xs_read_watch is called. According to my limited understanding, 
> this could read *any* firing watch placed by the same process, and the 
> code will discard it unless it's the one we are looking for. Thus 
> destroying information useful to someone else. I cannot have two 
> concurrent (or even interleaved) calls to libxenlight on these code 
> paths, because they could read each other's watches. Why not leave these 
> to an upper-level stack, which in all likelihood will have to deal with 
> lots of asynchronous events? As it stands, I have to write my code 
> *around* libxenlight, which kind of defeats the purpose.


You have a very good point here and I think we should fix this soon.

That said, let me just explain why we need these watches: we want to
abstract xenstore from the high level toolstack, so that it doesn't have
to access xenstore unless it really wants to.

Re: libxenlight or libxenheavy?

[Jean Guyader]

2009/11/30 Andres Lagar-Cavilla <andres@lagarcavilla.com>:
> Aside of the tongue in cheek title, I'd like to get a feel for where is
> libxenlight going. I love having a library that gives me three
> straight-forward C calls to create a domain, and I think it's an excellent
> vehicle to writing control stacks.
>
> But some of the latest patches have grown/bloated the library in directions
> I don't think are useful. This an obviously subjective take on the matter,
> but here are two examples:
>
> - Managing tapdisk2 devices in libxenlight: why at all? An upper-level
> control stack can (will have to) vet the configuration stanza of the
> tapdisk2 process, and it can then launch it and manage its life-cycle (i.e.
> echo appropriate commands to the sysfs interface). One of the great
> advantages of tapdisk2 is that it looks like a regular block device:
> /dev/xen/blktap-2/tapdev0. Libxenlight doesn't need to know this is any
> different from a regular block device ...
>

I don't think that really a bad thing. The top level toolstack can
still just give block
device to libxl and libxl will have no idea whether or not it's a
blktap2 device and
obviously at this point will be responsible of the creation and
destruction of those one.

I believe it's important to have a basic VHD support into the low
level library so you don't
need to have anything on the top to be able to use VHD.

Jean

> - Asynchronous notifications via xenstore watches: I've seen at least two
> locations (device deletion during destroy and waiting for domain death)
> where a watch on a xenstore path is placed by the library, and later
> xs_read_watch is called. According to my limited understanding, this could
> read *any* firing watch placed by the same process, and the code will
> discard it unless it's the one we are looking for. Thus destroying
> information useful to someone else. I cannot have two concurrent (or even
> interleaved) calls to libxenlight on these code paths, because they could
> read each other's watches. Why not leave these to an upper-level stack,
> which in all likelihood will have to deal with lots of asynchronous events?
> As it stands, I have to write my code *around* libxenlight, which kind of
> defeats the purpose.
>
> Anyway, these are two observations. Libxenlight is great. And to buy back
> some of the love I spent ranting, I'll follow this with 14 patches. These
> are all based off20522: abc6183f486e
> <http://xenbits.xen.org/xen-unstable.hg?rev/abc6183f486e>. Some are more
> RFC-ish in nature, but there a fair bit of fixes. They apply -p1 (sorry).
> Just let me know if you need a rebasing or whatever.
>

Re: libxenlight or libxenheavy?

[Stefano Stabellini]

On Tue, 1 Dec 2009, Vincent Hanquez wrote:
> Apart for the patches i commented one:
> 
> Acked-by: Vincent Hanquez <vincent.hanquez@eu.citrix.com>
> 

Same for me: all acked apart from the one I commented (also I agree with
Vincent's comments as well).

Re: libxenlight or libxenheavy?

[Keir Fraser]

Okay, I think I'm checking in Andres' patches 2,4,5,6,8,11,12,13 with Acks
from Stefano and Vincent. We'll see how well they apply out of sequence...

 -- Keir

On 01/12/2009 11:41, "Stefano Stabellini" <stefano.stabellini@eu.citrix.com>
wrote:

> On Tue, 1 Dec 2009, Vincent Hanquez wrote:
>> Apart for the patches i commented one:
>> 
>> Acked-by: Vincent Hanquez <vincent.hanquez@eu.citrix.com>
>> 
> 
> Same for me: all acked apart from the one I commented (also I agree with
> Vincent's comments as well).
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: libxenlight or libxenheavy?

[Keir Fraser]

Applied all but patch 2/14, of the below list. It depended on patch 1/14
which was nacked. All others I list below are applied as of c/s 20538.

 -- Keir

On 01/12/2009 12:14, "Keir Fraser" <keir.fraser@eu.citrix.com> wrote:

> Okay, I think I'm checking in Andres' patches 2,4,5,6,8,11,12,13 with Acks
> from Stefano and Vincent. We'll see how well they apply out of sequence...
> 
>  -- Keir
> 
> On 01/12/2009 11:41, "Stefano Stabellini" <stefano.stabellini@eu.citrix.com>
> wrote:
> 
>> On Tue, 1 Dec 2009, Vincent Hanquez wrote:
>>> Apart for the patches i commented one:
>>> 
>>> Acked-by: Vincent Hanquez <vincent.hanquez@eu.citrix.com>
>>> 
>> 
>> Same for me: all acked apart from the one I commented (also I agree with
>> Vincent's comments as well).
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: libxenlight or libxenheavy?

[Stefano Stabellini]

I would not apply also 3/14 and 7/14 because of Vincent's comments.

On Tue, 1 Dec 2009, Keir Fraser wrote:
> Applied all but patch 2/14, of the below list. It depended on patch 1/14
> which was nacked. All others I list below are applied as of c/s 20538.
> 
>  -- Keir
> 

Re: libxenlight or libxenheavy?

[Keir Fraser]

Too late. Issues will have to be fixed up in followup patches.

 -- Keir

On 01/12/2009 14:00, "Stefano Stabellini" <Stefano.Stabellini@eu.citrix.com>
wrote:

> I would not apply also 3/14 and 7/14 because of Vincent's comments.
> 
> On Tue, 1 Dec 2009, Keir Fraser wrote:
>> Applied all but patch 2/14, of the below list. It depended on patch 1/14
>> which was nacked. All others I list below are applied as of c/s 20538.
>> 
>>  -- Keir
>> 
> 

Re: libxenlight or libxenheavy?

[Andres Lagar-Cavilla]

I think 3/14 (8mb slack) is a non-issue because no real memory is 
allocated. Perhaps a confussion due to the patch description.
I'll see what I can do about 7/14 (no dm for pv). Later today. As well 
as Stefano's comments on 1/14 which should allow 2/14 as well
Andres
Keir Fraser wrote:
> Too late. Issues will have to be fixed up in followup patches.
>
>  -- Keir
>
> On 01/12/2009 14:00, "Stefano Stabellini" <Stefano.Stabellini@eu.citrix.com>
> wrote:
>
>   
>> I would not apply also 3/14 and 7/14 because of Vincent's comments.
>>
>> On Tue, 1 Dec 2009, Keir Fraser wrote:
>>     
>>> Applied all but patch 2/14, of the below list. It depended on patch 1/14
>>> which was nacked. All others I list below are applied as of c/s 20538.
>>>
>>>  -- Keir
>>>
>>>       
>
>
>