KVM call agenda for June, Tuesday 15th

KVM call agenda for June, Tuesday 15th

[Juan Quintela]

Hi

Please send in any agenda items you are interested in covering.

Thanks, Juan.

[Qemu-devel] KVM call agenda for June, Tuesday 15th

[Juan Quintela]

Hi

Please send in any agenda items you are interested in covering.

Thanks, Juan.

Re: [Qemu-devel] KVM call agenda for June, Tuesday 15th

[Anthony Liguori]

On 05/14/2012 07:41 AM, Juan Quintela wrote:
>
> Hi
>
> Please send in any agenda items you are interested in covering.

-open-fd-hook proposal

The discussion seems to have tapered out without a consensus.

Regards,

Anthony Liguori

>
> Thanks, Juan.
>

Re: [Qemu-devel] KVM call agenda for June, Tuesday 15th

[Anthony Liguori]

On 05/14/2012 07:41 AM, Juan Quintela wrote:
>
> Hi
>
> Please send in any agenda items you are interested in covering.

-open-fd-hook proposal

The discussion seems to have tapered out without a consensus.

Regards,

Anthony Liguori

>
> Thanks, Juan.
>

Re: [Qemu-devel] KVM call agenda for June, Tuesday 15th

[Paolo Bonzini]

Il 14/05/2012 17:42, Anthony Liguori ha scritto:
> On 05/14/2012 07:41 AM, Juan Quintela wrote:
>>
>> Hi
>>
>> Please send in any agenda items you are interested in covering.
> 
> -open-fd-hook proposal
> 
> The discussion seems to have tapered out without a consensus.

1.1 changelog, anything missing at http://wiki.qemu.org/ChangeLog/Next?

Paolo

Re: [Qemu-devel] KVM call agenda for June, Tuesday 15th

[Paolo Bonzini]

Il 14/05/2012 17:42, Anthony Liguori ha scritto:
> On 05/14/2012 07:41 AM, Juan Quintela wrote:
>>
>> Hi
>>
>> Please send in any agenda items you are interested in covering.
> 
> -open-fd-hook proposal
> 
> The discussion seems to have tapered out without a consensus.

1.1 changelog, anything missing at http://wiki.qemu.org/ChangeLog/Next?

Paolo

Re: [Qemu-devel] KVM call agenda for June, Tuesday 15th

[Kevin Wolf]

Am 14.05.2012 17:42, schrieb Anthony Liguori:
> On 05/14/2012 07:41 AM, Juan Quintela wrote:
>>
>> Hi
>>
>> Please send in any agenda items you are interested in covering.
> 
> -open-fd-hook proposal
> 
> The discussion seems to have tapered out without a consensus.

I asked for really good reasons to justify it, and the only response you
gave was along the lines of "because it's easy to implement".

Actually, I think it's not very easy at all when you start considering
cases like libvirt crashing (or restarting during an upgrade, as Daniel
mentioned). Currently we have a very simple unidirectional structure:
qemu is a standalone program that keeps running on its own. libvirt is
the user of qemu. Often enough it's already hard to get things working
correctly in error cases with this simple structure - do you really want
to have qemu depend on an RPC to libvirt?

You're right that the proper fix would be in the kernel, but in qemu a
much better solution that RPCs to libvirt is allowing all QMP commands
that open new files to pass a block device description that can contain
a fd. This would much better than first getting an open command via QMP
and then using an RPC to ask back what we're really meant to open.

To the full extent we're going to get this with blockdev-add (which is
what we should really start working on now rather than on hacks like
-open-fd-hook), but if you like hacks, much (if not all) of it is
already possible today with the 'existing' mode of live snapshots.

So if you want -open-fd-hook, please give us use cases that can't be
solved by passing the file descriptor directly in the QMP command (or on
the command line). I believe you might find some obscure ones that don't
work today, but I don't really expect that it enables anything that
-blockdev wouldn't enable.

Kevin

Re: [Qemu-devel] KVM call agenda for June, Tuesday 15th

[Kevin Wolf]

Am 14.05.2012 17:42, schrieb Anthony Liguori:
> On 05/14/2012 07:41 AM, Juan Quintela wrote:
>>
>> Hi
>>
>> Please send in any agenda items you are interested in covering.
> 
> -open-fd-hook proposal
> 
> The discussion seems to have tapered out without a consensus.

I asked for really good reasons to justify it, and the only response you
gave was along the lines of "because it's easy to implement".

Actually, I think it's not very easy at all when you start considering
cases like libvirt crashing (or restarting during an upgrade, as Daniel
mentioned). Currently we have a very simple unidirectional structure:
qemu is a standalone program that keeps running on its own. libvirt is
the user of qemu. Often enough it's already hard to get things working
correctly in error cases with this simple structure - do you really want
to have qemu depend on an RPC to libvirt?

You're right that the proper fix would be in the kernel, but in qemu a
much better solution that RPCs to libvirt is allowing all QMP commands
that open new files to pass a block device description that can contain
a fd. This would much better than first getting an open command via QMP
and then using an RPC to ask back what we're really meant to open.

To the full extent we're going to get this with blockdev-add (which is
what we should really start working on now rather than on hacks like
-open-fd-hook), but if you like hacks, much (if not all) of it is
already possible today with the 'existing' mode of live snapshots.

So if you want -open-fd-hook, please give us use cases that can't be
solved by passing the file descriptor directly in the QMP command (or on
the command line). I believe you might find some obscure ones that don't
work today, but I don't really expect that it enables anything that
-blockdev wouldn't enable.

Kevin

Re: [Qemu-devel] KVM call agenda for June, Tuesday 15th

[Anthony Liguori]

On 05/15/2012 03:51 AM, Kevin Wolf wrote:
> Am 14.05.2012 17:42, schrieb Anthony Liguori:
>> On 05/14/2012 07:41 AM, Juan Quintela wrote:
>>>
>>> Hi
>>>
>>> Please send in any agenda items you are interested in covering.
>>
>> -open-fd-hook proposal
>>
>> The discussion seems to have tapered out without a consensus.
>
> I asked for really good reasons to justify it, and the only response you
> gave was along the lines of "because it's easy to implement".

Is this not a really good reason?  It's a simple and complete way to solve a 
real problem.

> Actually, I think it's not very easy at all when you start considering
> cases like libvirt crashing (or restarting during an upgrade, as Daniel
> mentioned).

I think this is just an implementation detail.  We could switch to a named unix 
domain socket and also support reconnect.

> Currently we have a very simple unidirectional structure:
> qemu is a standalone program that keeps running on its own. libvirt is
> the user of qemu. Often enough it's already hard to get things working
> correctly in error cases with this simple structure - do you really want
> to have qemu depend on an RPC to libvirt?

Yes.  We're relying on libvirt for a *syscall* that the kernel isn't processing 
correctly.  I'm not advocating a general mechanism where we defer larges parts 
of QEMU to libvirt.  This is specifically the open() syscall.

> You're right that the proper fix would be in the kernel, but in qemu a
> much better solution that RPCs to libvirt is allowing all QMP commands
> that open new files to pass a block device description that can contain
> a fd.

I don't agree that this is an obviously better solution.  For example, it 
mandates that libvirt parse image formats to determine the backing file chains.

OTOH, the open() RPC allows libvirt to avoid parsing image formats.  It could do 
something as simple as have the user specify a white list of image files the 
guest is allowed to access in the domain XML and validate against that.

It removes considerable complexity from libvirt as it doesn't have to construct 
a potentially complex set of blockdev arguments.

> This would much better than first getting an open command via QMP
> and then using an RPC to ask back what we're really meant to open.
>
> To the full extent we're going to get this with blockdev-add (which is
> what we should really start working on now rather than on hacks like
> -open-fd-hook), but if you like hacks, much (if not all) of it is
> already possible today with the 'existing' mode of live snapshots.

I really don't think that blockdev is an elegant solution to this problem.  It 
pushes an awful lot of complexity to libvirt (or any management tool).

I actually think Avi's original idea of a filename dictionary is a better 
approach than blockdev for solving this problem.

That is not to say that I think blockdev doesn't have it's own merits, but I 
don't think we should be mandating its usage for management tools to do 
something simple.

Regards,

Anthony Liguori

>
> So if you want -open-fd-hook, please give us use cases that can't be
> solved by passing the file descriptor directly in the QMP command (or on
> the command line). I believe you might find some obscure ones that don't
> work today, but I don't really expect that it enables anything that
> -blockdev wouldn't enable.
>
> Kevin
>

Re: [Qemu-devel] KVM call agenda for June, Tuesday 15th

[Anthony Liguori]

On 05/15/2012 03:51 AM, Kevin Wolf wrote:
> Am 14.05.2012 17:42, schrieb Anthony Liguori:
>> On 05/14/2012 07:41 AM, Juan Quintela wrote:
>>>
>>> Hi
>>>
>>> Please send in any agenda items you are interested in covering.
>>
>> -open-fd-hook proposal
>>
>> The discussion seems to have tapered out without a consensus.
>
> I asked for really good reasons to justify it, and the only response you
> gave was along the lines of "because it's easy to implement".

Is this not a really good reason?  It's a simple and complete way to solve a 
real problem.

> Actually, I think it's not very easy at all when you start considering
> cases like libvirt crashing (or restarting during an upgrade, as Daniel
> mentioned).

I think this is just an implementation detail.  We could switch to a named unix 
domain socket and also support reconnect.

> Currently we have a very simple unidirectional structure:
> qemu is a standalone program that keeps running on its own. libvirt is
> the user of qemu. Often enough it's already hard to get things working
> correctly in error cases with this simple structure - do you really want
> to have qemu depend on an RPC to libvirt?

Yes.  We're relying on libvirt for a *syscall* that the kernel isn't processing 
correctly.  I'm not advocating a general mechanism where we defer larges parts 
of QEMU to libvirt.  This is specifically the open() syscall.

> You're right that the proper fix would be in the kernel, but in qemu a
> much better solution that RPCs to libvirt is allowing all QMP commands
> that open new files to pass a block device description that can contain
> a fd.

I don't agree that this is an obviously better solution.  For example, it 
mandates that libvirt parse image formats to determine the backing file chains.

OTOH, the open() RPC allows libvirt to avoid parsing image formats.  It could do 
something as simple as have the user specify a white list of image files the 
guest is allowed to access in the domain XML and validate against that.

It removes considerable complexity from libvirt as it doesn't have to construct 
a potentially complex set of blockdev arguments.

> This would much better than first getting an open command via QMP
> and then using an RPC to ask back what we're really meant to open.
>
> To the full extent we're going to get this with blockdev-add (which is
> what we should really start working on now rather than on hacks like
> -open-fd-hook), but if you like hacks, much (if not all) of it is
> already possible today with the 'existing' mode of live snapshots.

I really don't think that blockdev is an elegant solution to this problem.  It 
pushes an awful lot of complexity to libvirt (or any management tool).

I actually think Avi's original idea of a filename dictionary is a better 
approach than blockdev for solving this problem.

That is not to say that I think blockdev doesn't have it's own merits, but I 
don't think we should be mandating its usage for management tools to do 
something simple.

Regards,

Anthony Liguori

>
> So if you want -open-fd-hook, please give us use cases that can't be
> solved by passing the file descriptor directly in the QMP command (or on
> the command line). I believe you might find some obscure ones that don't
> work today, but I don't really expect that it enables anything that
> -blockdev wouldn't enable.
>
> Kevin
>

Re: [Qemu-devel] KVM call agenda for June, Tuesday 15th

[Daniel P. Berrange]

On Tue, May 15, 2012 at 08:44:14AM -0500, Anthony Liguori wrote:
> On 05/15/2012 03:51 AM, Kevin Wolf wrote:
> >Currently we have a very simple unidirectional structure:
> >qemu is a standalone program that keeps running on its own. libvirt is
> >the user of qemu. Often enough it's already hard to get things working
> >correctly in error cases with this simple structure - do you really want
> >to have qemu depend on an RPC to libvirt?
> 
> Yes.  We're relying on libvirt for a *syscall* that the kernel isn't
> processing correctly.  I'm not advocating a general mechanism where
> we defer larges parts of QEMU to libvirt.  This is specifically the
> open() syscall.
> 
> >You're right that the proper fix would be in the kernel, but in qemu a
> >much better solution that RPCs to libvirt is allowing all QMP commands
> >that open new files to pass a block device description that can contain
> >a fd.
> 
> I don't agree that this is an obviously better solution.  For
> example, it mandates that libvirt parse image formats to determine
> the backing file chains.

I think that the question of parsing image formats is tangential
to this QEMU impl choice.

> OTOH, the open() RPC allows libvirt to avoid parsing image formats.
> It could do something as simple as have the user specify a white
> list of image files the guest is allowed to access in the domain XML
> and validate against that.
>
> It removes considerable complexity from libvirt as it doesn't have
> to construct a potentially complex set of blockdev arguments.

I don't really think this QEMU approach to a callback for arbitrary
files simplifies libvirt's life in any way. In fact I think it will
actually complicate our life, because instead of being able to
provide all the information/resources required at one time, we need
have to wait to get async callbacks some time later. We then have to
try and figure out whether the file being request is actually allowed
by the config.


> >This would much better than first getting an open command via QMP
> >and then using an RPC to ask back what we're really meant to open.
> >
> >To the full extent we're going to get this with blockdev-add (which is
> >what we should really start working on now rather than on hacks like
> >-open-fd-hook), but if you like hacks, much (if not all) of it is
> >already possible today with the 'existing' mode of live snapshots.
> 
> I really don't think that blockdev is an elegant solution to this
> problem.  It pushes an awful lot of complexity to libvirt (or any
> management tool).
> 
> I actually think Avi's original idea of a filename dictionary is a
> better approach than blockdev for solving this problem.

While I raise blockdev as an alternative approach, I am open to
other alternative ways to provide this config via the CLI or
monitor. Basically anything that isn't this generic file open
callback.

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|