[Qemu-devel] KVM call agenda for Oct 19

[Qemu-devel] KVM call agenda for Oct 19

[Juan Quintela]

Please send in any agenda items you are interested in covering.

thanks,

Juan.

[Qemu-devel] Re: KVM call agenda for Oct 19

[Chris Wright]

* Juan Quintela (quintela@redhat.com) wrote:
> 
> Please send in any agenda items you are interested in covering.

- 0.13.X -stable handoff
- 0.14 planning
- threadlet work
- virtfs proposals

Re: [Qemu-devel] Re: KVM call agenda for Oct 19

[Dor Laor]

On 10/19/2010 04:11 AM, Chris Wright wrote:
> * Juan Quintela (quintela@redhat.com) wrote:
>>
>> Please send in any agenda items you are interested in covering.
>
> - 0.13.X -stable handoff
> - 0.14 planning
> - threadlet work
> - virtfs proposals
>

- Live snapshots
   - We were asked to add this feature for external qcow2
     images. Will simple approach of fsync + tracking each requested
     backing file (it can be per vDisk) and re-open the new image would
     be accepted?
   - Integration with FS freeze for consistent guest app snapshot
     Many apps do not sync their ram state to disk correctly or frequent
     enough. Physical world backup software calls fs freeze on xfs and
     VSS for windows to make the backup consistent.
     In order to integrated this with live snapshots we need a guest
     agent to trigger the guest fs freeze.
     We can either have qemu communicate with the agent directly through
     virtio-serial or have a mgmt daemon use virtio-serial to
     communicate with the guest in addition to QMP messages about the
     live snapshot state.
     Preferences? The first solution complicates qemu while the second
     complicates mgmt.

Re: [Qemu-devel] Re: KVM call agenda for Oct 19

[Avi Kivity]

  On 10/19/2010 02:48 PM, Dor Laor wrote:
> On 10/19/2010 04:11 AM, Chris Wright wrote:
>> * Juan Quintela (quintela@redhat.com) wrote:
>>>
>>> Please send in any agenda items you are interested in covering.
>>
>> - 0.13.X -stable handoff
>> - 0.14 planning
>> - threadlet work
>> - virtfs proposals
>>
>
> - Live snapshots
>   - We were asked to add this feature for external qcow2
>     images. Will simple approach of fsync + tracking each requested
>     backing file (it can be per vDisk) and re-open the new image would
>     be accepted?
>   - Integration with FS freeze for consistent guest app snapshot
>     Many apps do not sync their ram state to disk correctly or frequent
>     enough. Physical world backup software calls fs freeze on xfs and
>     VSS for windows to make the backup consistent.
>     In order to integrated this with live snapshots we need a guest
>     agent to trigger the guest fs freeze.
>     We can either have qemu communicate with the agent directly through
>     virtio-serial or have a mgmt daemon use virtio-serial to
>     communicate with the guest in addition to QMP messages about the
>     live snapshot state.
>     Preferences? The first solution complicates qemu while the second
>     complicates mgmt.

Third option, make the freeze path management -> qemu -> virtio-blk -> 
guest kernel -> file systems.  The advantage is that it's easy to 
associate file systems with a block device this way.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Re: KVM call agenda for Oct 19

[Dor Laor]

On 10/19/2010 02:55 PM, Avi Kivity wrote:
> On 10/19/2010 02:48 PM, Dor Laor wrote:
>> On 10/19/2010 04:11 AM, Chris Wright wrote:
>>> * Juan Quintela (quintela@redhat.com) wrote:
>>>>
>>>> Please send in any agenda items you are interested in covering.
>>>
>>> - 0.13.X -stable handoff
>>> - 0.14 planning
>>> - threadlet work
>>> - virtfs proposals
>>>
>>
>> - Live snapshots
>> - We were asked to add this feature for external qcow2
>> images. Will simple approach of fsync + tracking each requested
>> backing file (it can be per vDisk) and re-open the new image would
>> be accepted?
>> - Integration with FS freeze for consistent guest app snapshot
>> Many apps do not sync their ram state to disk correctly or frequent
>> enough. Physical world backup software calls fs freeze on xfs and
>> VSS for windows to make the backup consistent.
>> In order to integrated this with live snapshots we need a guest
>> agent to trigger the guest fs freeze.
>> We can either have qemu communicate with the agent directly through
>> virtio-serial or have a mgmt daemon use virtio-serial to
>> communicate with the guest in addition to QMP messages about the
>> live snapshot state.
>> Preferences? The first solution complicates qemu while the second
>> complicates mgmt.
>
> Third option, make the freeze path management -> qemu -> virtio-blk ->
> guest kernel -> file systems. The advantage is that it's easy to
> associate file systems with a block device this way.

OTH the userspace freeze path already exist and now you create another 
path. What about FS that span over LVM with multiple drives? IDE/SCSI?

Re: [Qemu-devel] Re: KVM call agenda for Oct 19

[Avi Kivity]

  On 10/19/2010 02:58 PM, Dor Laor wrote:
> On 10/19/2010 02:55 PM, Avi Kivity wrote:
>> On 10/19/2010 02:48 PM, Dor Laor wrote:
>>> On 10/19/2010 04:11 AM, Chris Wright wrote:
>>>> * Juan Quintela (quintela@redhat.com) wrote:
>>>>>
>>>>> Please send in any agenda items you are interested in covering.
>>>>
>>>> - 0.13.X -stable handoff
>>>> - 0.14 planning
>>>> - threadlet work
>>>> - virtfs proposals
>>>>
>>>
>>> - Live snapshots
>>> - We were asked to add this feature for external qcow2
>>> images. Will simple approach of fsync + tracking each requested
>>> backing file (it can be per vDisk) and re-open the new image would
>>> be accepted?
>>> - Integration with FS freeze for consistent guest app snapshot
>>> Many apps do not sync their ram state to disk correctly or frequent
>>> enough. Physical world backup software calls fs freeze on xfs and
>>> VSS for windows to make the backup consistent.
>>> In order to integrated this with live snapshots we need a guest
>>> agent to trigger the guest fs freeze.
>>> We can either have qemu communicate with the agent directly through
>>> virtio-serial or have a mgmt daemon use virtio-serial to
>>> communicate with the guest in addition to QMP messages about the
>>> live snapshot state.
>>> Preferences? The first solution complicates qemu while the second
>>> complicates mgmt.
>>
>> Third option, make the freeze path management -> qemu -> virtio-blk ->
>> guest kernel -> file systems. The advantage is that it's easy to
>> associate file systems with a block device this way.
>
> OTH the userspace freeze path already exist and now you create another 
> path. 

I guess we would still have a userspace daemon; instead of talking to 
virtio-serial it talks to virtio-blk.  So:

   management -> qemu -> virtio-blk -> guest driver -> kernel fs 
resolver -> daemon -> apps

Yuck.

> What about FS that span over LVM with multiple drives? IDE/SCSI?

Good points.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Re: KVM call agenda for Oct 19

[Anthony Liguori]

On 10/19/2010 08:03 AM, Avi Kivity wrote:
>  On 10/19/2010 02:58 PM, Dor Laor wrote:
>> On 10/19/2010 02:55 PM, Avi Kivity wrote:
>>> On 10/19/2010 02:48 PM, Dor Laor wrote:
>>>> On 10/19/2010 04:11 AM, Chris Wright wrote:
>>>>> * Juan Quintela (quintela@redhat.com) wrote:
>>>>>>
>>>>>> Please send in any agenda items you are interested in covering.
>>>>>
>>>>> - 0.13.X -stable handoff
>>>>> - 0.14 planning
>>>>> - threadlet work
>>>>> - virtfs proposals
>>>>>
>>>>
>>>> - Live snapshots
>>>> - We were asked to add this feature for external qcow2
>>>> images. Will simple approach of fsync + tracking each requested
>>>> backing file (it can be per vDisk) and re-open the new image would
>>>> be accepted?
>>>> - Integration with FS freeze for consistent guest app snapshot
>>>> Many apps do not sync their ram state to disk correctly or frequent
>>>> enough. Physical world backup software calls fs freeze on xfs and
>>>> VSS for windows to make the backup consistent.
>>>> In order to integrated this with live snapshots we need a guest
>>>> agent to trigger the guest fs freeze.
>>>> We can either have qemu communicate with the agent directly through
>>>> virtio-serial or have a mgmt daemon use virtio-serial to
>>>> communicate with the guest in addition to QMP messages about the
>>>> live snapshot state.
>>>> Preferences? The first solution complicates qemu while the second
>>>> complicates mgmt.
>>>
>>> Third option, make the freeze path management -> qemu -> virtio-blk ->
>>> guest kernel -> file systems. The advantage is that it's easy to
>>> associate file systems with a block device this way.
>>
>> OTH the userspace freeze path already exist and now you create 
>> another path. 
>
> I guess we would still have a userspace daemon; instead of talking to 
> virtio-serial it talks to virtio-blk.  So:
>
>   management -> qemu -> virtio-blk -> guest driver -> kernel fs 
> resolver -> daemon -> apps
>
> Yuck.

Yeah, in Windows, I'm pretty sure the freeze API is a userspace 
concept.  Various apps can hook into it to serialize their state.

At the risk of stealing Mike's thunder, we've actually been working on a 
simple guest agent exactly for this type of task.  Mike's planning an 
RFC for later this week but for those that are interested the repo is at 
http://repo.or.cz/w/qemu/mdroth.git

Regards,

Anthony Liguori
>
>> What about FS that span over LVM with multiple drives? IDE/SCSI?
>
> Good points.
>

Re: [Qemu-devel] Re: KVM call agenda for Oct 19

[Anthony Liguori]

On 10/19/2010 07:48 AM, Dor Laor wrote:
> On 10/19/2010 04:11 AM, Chris Wright wrote:
>> * Juan Quintela (quintela@redhat.com) wrote:
>>>
>>> Please send in any agenda items you are interested in covering.
>>
>> - 0.13.X -stable handoff
>> - 0.14 planning
>> - threadlet work
>> - virtfs proposals
>>
>
> - Live snapshots
>   - We were asked to add this feature for external qcow2
>     images. Will simple approach of fsync + tracking each requested
>     backing file (it can be per vDisk) and re-open the new image would
>     be accepted?

I had assumed that this would involve:

qemu -hda windows.img

(qemu) snapshot ide0-disk0 snap0.img

1) create snap0.img internally by doing the equivalent of `qemu-img 
create -f qcow2 -b windows.img snap0.img'
2) bdrv_flush('ide0-disk0')
3) bdrv_open(snap0.img)
4) bdrv_close(windows.img)
5) rename('windows.img', 'windows.img.tmp')
6) rename('snap0.img', 'windows.img')
7) rename('windows.img.tmp', 'snap0.img')

Regards,

Anthony Liguori

>   - Integration with FS freeze for consistent guest app snapshot
>     Many apps do not sync their ram state to disk correctly or frequent
>     enough. Physical world backup software calls fs freeze on xfs and
>     VSS for windows to make the backup consistent.
>     In order to integrated this with live snapshots we need a guest
>     agent to trigger the guest fs freeze.
>     We can either have qemu communicate with the agent directly through
>     virtio-serial or have a mgmt daemon use virtio-serial to
>     communicate with the guest in addition to QMP messages about the
>     live snapshot state.
>     Preferences? The first solution complicates qemu while the second
>     complicates mgmt.
> -- 
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Qemu-devel] Re: KVM call agenda for Oct 19

[Avi Kivity]

  On 10/19/2010 03:22 PM, Anthony Liguori wrote:
>
> I had assumed that this would involve:
>
> qemu -hda windows.img
>
> (qemu) snapshot ide0-disk0 snap0.img
>
> 1) create snap0.img internally by doing the equivalent of `qemu-img 
> create -f qcow2 -b windows.img snap0.img'
> 2) bdrv_flush('ide0-disk0')
> 3) bdrv_open(snap0.img)
> 4) bdrv_close(windows.img)
> 5) rename('windows.img', 'windows.img.tmp')
> 6) rename('snap0.img', 'windows.img')
> 7) rename('windows.img.tmp', 'snap0.img')
>

Looks reasonable.

Would be interesting to look at this as a use case for the threading 
work.  We should eventually be able to create a snapshot without 
stalling vcpus (stalling I/O of course allowed).

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Re: KVM call agenda for Oct 19

[Anthony Liguori]

On 10/19/2010 07:48 AM, Dor Laor wrote:
> On 10/19/2010 04:11 AM, Chris Wright wrote:
>> * Juan Quintela (quintela@redhat.com) wrote:
>>>
>>> Please send in any agenda items you are interested in covering.
>>
>> - 0.13.X -stable handoff
>> - 0.14 planning
>> - threadlet work
>> - virtfs proposals
>>
>
> - Live snapshots
>   - We were asked to add this feature for external qcow2
>     images. Will simple approach of fsync + tracking each requested
>     backing file (it can be per vDisk) and re-open the new image would
>     be accepted?
>   - Integration with FS freeze for consistent guest app snapshot
>     Many apps do not sync their ram state to disk correctly or frequent
>     enough. Physical world backup software calls fs freeze on xfs and
>     VSS for windows to make the backup consistent.
>     In order to integrated this with live snapshots we need a guest
>     agent to trigger the guest fs freeze.
>     We can either have qemu communicate with the agent directly through
>     virtio-serial or have a mgmt daemon use virtio-serial to
>     communicate with the guest in addition to QMP messages about the
>     live snapshot state.
>     Preferences? The first solution complicates qemu while the second
>     complicates mgmt.

- usb-ccid (aka external device modules)

We probably won't get to it for today's call, but we should try to queue 
this topic up for discussion.  We have a similar situation with vtpm 
(existing device model that wants to integrate with QEMU).  My position 
so far has been that we should avoid external device models because of 
difficulty integrating QEMU features with external device models.

However, I'd like to hear opinions from a wider audience.

Regards,

Anthony Liguori

> -- 
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Qemu-devel] Re: KVM call agenda for Oct 19

[Anthony Liguori]

On 10/19/2010 08:27 AM, Avi Kivity wrote:
>  On 10/19/2010 03:22 PM, Anthony Liguori wrote:
>>
>> I had assumed that this would involve:
>>
>> qemu -hda windows.img
>>
>> (qemu) snapshot ide0-disk0 snap0.img
>>
>> 1) create snap0.img internally by doing the equivalent of `qemu-img 
>> create -f qcow2 -b windows.img snap0.img'
>> 2) bdrv_flush('ide0-disk0')
>> 3) bdrv_open(snap0.img)
>> 4) bdrv_close(windows.img)
>> 5) rename('windows.img', 'windows.img.tmp')
>> 6) rename('snap0.img', 'windows.img')
>> 7) rename('windows.img.tmp', 'snap0.img')
>>
>
> Looks reasonable.
>
> Would be interesting to look at this as a use case for the threading 
> work.  We should eventually be able to create a snapshot without 
> stalling vcpus (stalling I/O of course allowed).

If we had another block-level command, like bdrv_aio_freeze(), that 
queued all pending requests until the given callback completed, it would 
be very easy to do this entirely asynchronously.  For instance:

bdrv_aio_freeze(create_snapshot)

create_snapshot():
   bdrv_aio_flush(done_flush)

done_flush():
   bdrv_open(...)
   bdrv_close(...)
   ...

Of course, closing a device while it's being frozen is probably a recipe 
for disaster but you get the idea :-)

Regards,

Anthony Liguori

Re: [Qemu-devel] Re: KVM call agenda for Oct 19

[Stefan Hajnoczi]

On Tue, Oct 19, 2010 at 2:33 PM, Anthony Liguori <anthony@codemonkey.ws> wrote:
> On 10/19/2010 08:27 AM, Avi Kivity wrote:
>>
>>  On 10/19/2010 03:22 PM, Anthony Liguori wrote:
>>>
>>> I had assumed that this would involve:
>>>
>>> qemu -hda windows.img
>>>
>>> (qemu) snapshot ide0-disk0 snap0.img
>>>
>>> 1) create snap0.img internally by doing the equivalent of `qemu-img
>>> create -f qcow2 -b windows.img snap0.img'
>>> 2) bdrv_flush('ide0-disk0')
>>> 3) bdrv_open(snap0.img)
>>> 4) bdrv_close(windows.img)
>>> 5) rename('windows.img', 'windows.img.tmp')
>>> 6) rename('snap0.img', 'windows.img')
>>> 7) rename('windows.img.tmp', 'snap0.img')
>>>
>>
>> Looks reasonable.
>>
>> Would be interesting to look at this as a use case for the threading work.
>>  We should eventually be able to create a snapshot without stalling vcpus
>> (stalling I/O of course allowed).
>
> If we had another block-level command, like bdrv_aio_freeze(), that queued
> all pending requests until the given callback completed, it would be very
> easy to do this entirely asynchronously.  For instance:
>
> bdrv_aio_freeze(create_snapshot)
>
> create_snapshot():
>  bdrv_aio_flush(done_flush)
>
> done_flush():
>  bdrv_open(...)
>  bdrv_close(...)
>  ...
>
> Of course, closing a device while it's being frozen is probably a recipe for
> disaster but you get the idea :-)

bdrv_aio_freeze() or any mechanism to deal with pending requests in
the generic block code would be a good step for future "live" support
of other operations like truncate.

Stefan

Re: [Qemu-devel] Re: KVM call agenda for Oct 19

[Avi Kivity]

  On 10/19/2010 03:38 PM, Stefan Hajnoczi wrote:
> bdrv_aio_freeze() or any mechanism to deal with pending requests in
> the generic block code would be a good step for future "live" support
> of other operations like truncate.

+ logical disk grow, etc.

-- 
error compiling committee.c: too many arguments to function