[Qemu-devel] QCOW rebases on running VMs

[Qemu-devel] QCOW rebases on running VMs

[Alex Bligh]

This question concerns the safety of rebases on backing disks whilst a VM is running from a writing disk.

Assume that we have the following structure:

    QEMU ------->   Writing disk
                    ....
                    backing disk N
                    ....
                    backing disk 1
                    ....
                    backing disk 0

If QEMU is not running, I can happily use 'qemu-img rebase' to merge backing disks 1 ... N (i.e. so backing disk N is now based on backing disk 0).

Is it safe to do this whilst the server is running using the writing disk which is based on backing disk N?

It is attractive to think it might be, as even if QEMU has a stale view of the contents of backing disk N, backing disks N-1 to 1 (inclusive) will still be present, QEMU will still keep an fd open for them, and it will (thus) still be able to read the correct sectors from them.

If not, do I need to use QMP magic instead?

-- 
Alex Bligh

Re: [Qemu-devel] QCOW rebases on running VMs

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 2545 bytes --]

On 27.01.2016 19:04, Alex Bligh wrote:
> This question concerns the safety of rebases on backing disks whilst a VM is running from a writing disk.
> 
> Assume that we have the following structure:
> 
>     QEMU ------->   Writing disk
>                     ....
>                     backing disk N
>                     ....
>                     backing disk 1
>                     ....
>                     backing disk 0
> 
> If QEMU is not running, I can happily use 'qemu-img rebase' to merge backing disks 1 ... N (i.e. so backing disk N is now based on backing disk 0).
> 
> Is it safe to do this whilst the server is running using the writing disk which is based on backing disk N?

Maybe. You would rebase 1 through N into a new file and change the
backing file of the writing disk to the new file; because qemu is not
writing to the backing disks, that should probably work, but it's not
nice and may break horribly without prior notice.

> It is attractive to think it might be, as even if QEMU has a stale view of the contents of backing disk N, backing disks N-1 to 1 (inclusive) will still be present, QEMU will still keep an fd open for them, and it will (thus) still be able to read the correct sectors from them.
> 
> If not, do I need to use QMP magic instead?

This is always the best option, and I would strongly recommend doing so.
block-stream or block-commit should be your friend. block-stream streams
data from backing files into backed files, block-commit pushes data from
backed files into backing files; so block-stream transmits data from
bottom to top, and block-commit transmits it from top to bottom.

With block-commit, you can collapse the backing chain to e.g. (from top
to bottom):

Writing disk -> backing disk 1..N -> backing disk 0

like this (I think):

{ 'execute': 'block-commit',
  'arguments': { 'device': '$drive',
                 'base': 'backing disk 1',
                 'top': 'backing disk N' } }

The "backing disk 1..N" is the file which contained "backing disk 1" and
now contains all the data from disks 1 through N.


With block-stream, you can collapse it into

Writing disk -> backing disk 0

like this:

{ 'execute': 'block-stream',
  'arguments': { 'device': '$drive',
                 'base': 'backing disk 0' } }

Here, the writing disk would contain all the data from backing disks 1
through N. You could omit the "base" option, in which case the whole
backing chain would be collapsed into the writing disk.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Qemu-devel] QCOW rebases on running VMs

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1525 bytes --]

On 01/27/2016 11:04 AM, Alex Bligh wrote:
> This question concerns the safety of rebases on backing disks whilst a VM is running from a writing disk.
> 
> Assume that we have the following structure:
> 
>     QEMU ------->   Writing disk
>                     ....
>                     backing disk N
>                     ....
>                     backing disk 1
>                     ....
>                     backing disk 0
> 
> If QEMU is not running, I can happily use 'qemu-img rebase' to merge backing disks 1 ... N (i.e. so backing disk N is now based on backing disk 0).
> 
> Is it safe to do this whilst the server is running using the writing disk which is based on backing disk N?

Only through the running qemu.

> 
> It is attractive to think it might be, as even if QEMU has a stale view of the contents of backing disk N, backing disks N-1 to 1 (inclusive) will still be present, QEMU will still keep an fd open for them, and it will (thus) still be able to read the correct sectors from them.

No. Any change to the backing chain in use by an active qemu process is
liable to corrupt that file and/or qemu's view of the data to present to
the guest.  The only safe way to collapse a backing chain while qemu is
running is via monitor (QMP or HMP) commands to that qemu process.

> 
> If not, do I need to use QMP magic instead?

Yes. block-commit is your friend.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] QCOW rebases on running VMs

[Alex Bligh]

[-- Attachment #1: Type: text/plain, Size: 446 bytes --]


On 27 Jan 2016, at 20:47, Eric Blake <eblake@redhat.com> wrote:

>> 
>> 
>> If not, do I need to use QMP magic instead?
> 
> Yes. block-commit is your friend.

Sadly what I need is block-stream to an intermediate layer (as I
want to merge in the other direction); I don't think that proposal
was ever implemented.

http://wiki.qemu.org/Features/Snapshots#Streaming_to_an_Intermediate_Layer_.5Bproposal.5D

--
Alex Bligh





[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]

Re: [Qemu-devel] QCOW rebases on running VMs

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]

On 01/28/2016 05:36 AM, Alex Bligh wrote:
> 
> On 27 Jan 2016, at 20:47, Eric Blake <eblake@redhat.com> wrote:
> 
>>>
>>>
>>> If not, do I need to use QMP magic instead?
>>
>> Yes. block-commit is your friend.
> 
> Sadly what I need is block-stream to an intermediate layer (as I
> want to merge in the other direction); I don't think that proposal
> was ever implemented.
> 
> http://wiki.qemu.org/Features/Snapshots#Streaming_to_an_Intermediate_Layer_.5Bproposal.5D

Alberto was working on it last; this was the most recent mail I found on
the topic (just to the qemu-block list):
https://lists.gnu.org/archive/html/qemu-block/2015-12/msg00214.html

so although I recall some RFC patches having been posted, you are right
that it still needs some work to be merged in.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] QCOW rebases on running VMs

[Alberto Garcia]

On Thu, Jan 28, 2016 at 08:25:18AM -0700, Eric Blake wrote:

> > Sadly what I need is block-stream to an intermediate layer (as I
> > want to merge in the other direction); I don't think that proposal
> > was ever implemented.
> > 
> > http://wiki.qemu.org/Features/Snapshots#Streaming_to_an_Intermediate_Layer_.5Bproposal.5D
> 
> Alberto was working on it last; this was the most recent mail I found on
> the topic (just to the qemu-block list):
> https://lists.gnu.org/archive/html/qemu-block/2015-12/msg00214.html
> 
> so although I recall some RFC patches having been posted, you are
> right that it still needs some work to be merged in.

The intermediate block streaming code was finished and ready to be
merged about half a year ago. Unfortunately there were conflicts with
other changes that were happening at the same time so it had to be
rolled back.

Here's the latest version of the series:

   https://lists.gnu.org/archive/html/qemu-devel/2015-06/msg05754.html

And here's a recent summary of the problems that prevent it from being
merged:

   https://lists.gnu.org/archive/html/qemu-block/2015-12/msg00180.html

My plan is to continue working on it.

Berto