[Qemu-devel] Image streaming gives live block copy for free (and vice versa)

[Qemu-devel] Image streaming gives live block copy for free (and vice versa)

[Stefan Hajnoczi]

Perhaps someone has been saying this all along but I want to spell it
out that image streaming and live block copy are equivalent in theory.
I just realized this last night.  In practice we might choose one
implementation or two different ones for performance reasons.

If any of these are unclear please let me know and I'll try to post diagrams.

Live block copy using image streaming
-------------------------------------

1. Create the destination image file and use the source image as the
backing file.
2. Quiesce I/O and pause VM.
3. Switch to destination image.
4. Resume VM.
5. Start streaming destination image in order to copy source image
data into destination file.
6. Streaming completes and disables the backing file, leaving the live
copied destination image that no longer depends on the source image.

There is no need for dirty block tracking because image streaming will
only copy over unallocated clusters.  There are no phases to the copy
process because the guest is writing to the destination file already
and does not dirty the source file.

Implementing live block copy without switch is also possible using the
block-mirror driver to update both the source image and the
destination image.  This would require making the backing file
writable though.

Image streaming using live block copy
-------------------------------------

1. Create destination image.
2. Start a live block copy to copy the source image data into destination file.
3. When live block copy reaches switch state, quiesce I/O and pause VM.
4. Switch to destination image, which now contains the flattened source image.
5. Resume VM.
7. Delete source image.

This approach copies the contents of the source image (and its backing
file) into the destination file.  You need to have two times the disk
space, since towards the end of live block copy you have two copies of
the image.  The QED image streaming patches I posted do not need twice
the disk space because they work in-place.

Call now!  We'll give you live snapshot merge using image streaming for FREE
----------------------------------------------------------------------------

1. Quiesce I/O and pause VM.
2. Free clusters in the snapshot that are allocated in the
copy-on-write delta file.
3. Make the copy-on-write delta file the backing file of the snapshot.
(Now the snapshot image has most clusters allocated except those which
were COWed due to a write after the snapshot was taken)
4. Resume VM.
5. Start streaming the snapshot image in order to copy the COW data
back into the snapshot file.
6. Streaming completes and disables the backing file, leaving the
merged snapshot.
7. Delete the COW file.

This approach is much more handwavy.  We needed to invert the backing
file relationship between snapshot and COW file, as well as freeing
clusters in order to make image streaming copy the data backing into
the snapshot file.

Stefan

Re: [Qemu-devel] Image streaming gives live block copy for free (and vice versa)

[Dor Laor]

On 06/17/2011 08:53 AM, Stefan Hajnoczi wrote:
> Perhaps someone has been saying this all along but I want to spell it
> out that image streaming and live block copy are equivalent in theory.
> I just realized this last night.  In practice we might choose one
> implementation or two different ones for performance reasons.
>
> If any of these are unclear please let me know and I'll try to post diagrams.
>
> Live block copy using image streaming
> -------------------------------------
>
> 1. Create the destination image file and use the source image as the
> backing file.
> 2. Quiesce I/O and pause VM.
> 3. Switch to destination image.
> 4. Resume VM.
> 5. Start streaming destination image in order to copy source image
> data into destination file.
> 6. Streaming completes and disables the backing file, leaving the live
> copied destination image that no longer depends on the source image.

Well streaming is copying just using post-copy approach.
Both pre and post are required:
  - post-copy (aka streaming) for fast live block migration of the VM
    for cpu load balance.
  - pre-copy (aka live block copy)
    If you manage to get a network outage between the source and the
    destination storage systems, you will manage to keep running the VM
    on the source side.

Like Kevin noted, the implementation and the functionality of these two 
features are almost identical, that's why we can have a single 
implementation with some options to enable pre or post copy.

>
> There is no need for dirty block tracking because image streaming will
> only copy over unallocated clusters.  There are no phases to the copy
> process because the guest is writing to the destination file already
> and does not dirty the source file.
>
> Implementing live block copy without switch is also possible using the
> block-mirror driver to update both the source image and the
> destination image.  This would require making the backing file
> writable though.
>
> Image streaming using live block copy
> -------------------------------------
>
> 1. Create destination image.
> 2. Start a live block copy to copy the source image data into destination file.
> 3. When live block copy reaches switch state, quiesce I/O and pause VM.
> 4. Switch to destination image, which now contains the flattened source image.
> 5. Resume VM.
> 7. Delete source image.
>
> This approach copies the contents of the source image (and its backing
> file) into the destination file.  You need to have two times the disk
> space, since towards the end of live block copy you have two copies of
> the image.  The QED image streaming patches I posted do not need twice
> the disk space because they work in-place.
>
> Call now!  We'll give you live snapshot merge using image streaming for FREE
> ----------------------------------------------------------------------------
>
> 1. Quiesce I/O and pause VM.
> 2. Free clusters in the snapshot that are allocated in the
> copy-on-write delta file.
> 3. Make the copy-on-write delta file the backing file of the snapshot.
> (Now the snapshot image has most clusters allocated except those which
> were COWed due to a write after the snapshot was taken)
> 4. Resume VM.
> 5. Start streaming the snapshot image in order to copy the COW data
> back into the snapshot file.
> 6. Streaming completes and disables the backing file, leaving the
> merged snapshot.
> 7. Delete the COW file.
>
> This approach is much more handwavy.  We needed to invert the backing
> file relationship between snapshot and COW file, as well as freeing
> clusters in order to make image streaming copy the data backing into
> the snapshot file.
>
> Stefan
>

Re: [Qemu-devel] Image streaming gives live block copy for free (and vice versa)

[Stefan Hajnoczi]

On Sun, Jun 19, 2011 at 5:12 PM, Dor Laor <dlaor@redhat.com> wrote:
> On 06/17/2011 08:53 AM, Stefan Hajnoczi wrote:
>>
>> Perhaps someone has been saying this all along but I want to spell it
>> out that image streaming and live block copy are equivalent in theory.
>> I just realized this last night.  In practice we might choose one
>> implementation or two different ones for performance reasons.
>>
>> If any of these are unclear please let me know and I'll try to post
>> diagrams.
>>
>> Live block copy using image streaming
>> -------------------------------------
>>
>> 1. Create the destination image file and use the source image as the
>> backing file.
>> 2. Quiesce I/O and pause VM.
>> 3. Switch to destination image.
>> 4. Resume VM.
>> 5. Start streaming destination image in order to copy source image
>> data into destination file.
>> 6. Streaming completes and disables the backing file, leaving the live
>> copied destination image that no longer depends on the source image.
>
> Well streaming is copying just using post-copy approach.
> Both pre and post are required:
>  - post-copy (aka streaming) for fast live block migration of the VM
>   for cpu load balance.
>  - pre-copy (aka live block copy)
>   If you manage to get a network outage between the source and the
>   destination storage systems, you will manage to keep running the VM
>   on the source side.

Out of interest, are you brainstorming using live block copy and image
streaming for pre- and post-copy live migration or do you have
concrete plans to update libvirt to use these mechanisms?

So far I've been focussing on the fast provisioning use case where
image streaming helps, but eventually I would like to improve the
state of storage migration too.

Stefan

Re: [Qemu-devel] Image streaming gives live block copy for free (and vice versa)

[Dor Laor]

On 06/24/2011 12:35 PM, Stefan Hajnoczi wrote:
> On Sun, Jun 19, 2011 at 5:12 PM, Dor Laor<dlaor@redhat.com>  wrote:
>> On 06/17/2011 08:53 AM, Stefan Hajnoczi wrote:
>>>
>>> Perhaps someone has been saying this all along but I want to spell it
>>> out that image streaming and live block copy are equivalent in theory.
>>> I just realized this last night.  In practice we might choose one
>>> implementation or two different ones for performance reasons.
>>>
>>> If any of these are unclear please let me know and I'll try to post
>>> diagrams.
>>>
>>> Live block copy using image streaming
>>> -------------------------------------
>>>
>>> 1. Create the destination image file and use the source image as the
>>> backing file.
>>> 2. Quiesce I/O and pause VM.
>>> 3. Switch to destination image.
>>> 4. Resume VM.
>>> 5. Start streaming destination image in order to copy source image
>>> data into destination file.
>>> 6. Streaming completes and disables the backing file, leaving the live
>>> copied destination image that no longer depends on the source image.
>>
>> Well streaming is copying just using post-copy approach.
>> Both pre and post are required:
>>   - post-copy (aka streaming) for fast live block migration of the VM
>>    for cpu load balance.
>>   - pre-copy (aka live block copy)
>>    If you manage to get a network outage between the source and the
>>    destination storage systems, you will manage to keep running the VM
>>    on the source side.
>
> Out of interest, are you brainstorming using live block copy and image
> streaming for pre- and post-copy live migration or do you have
> concrete plans to update libvirt to use these mechanisms?

Adding Eric, Daniel and Dave from the libvirt team.

>
> So far I've been focussing on the fast provisioning use case where
> image streaming helps, but eventually I would like to improve the
> state of storage migration too.

In general it seems to me that we need to reuse the design so fast 
provision, live block migration, live snapshot (merge) and non shared 
storage option will all use the same code and the same interfaces.

>
> Stefan
>
>
>