From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=36081 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PxNmP-0003vU-0i for qemu-devel@nongnu.org; Wed, 09 Mar 2011 13:06:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PxNmM-0002HR-SK for qemu-devel@nongnu.org; Wed, 09 Mar 2011 13:06:36 -0500 Received: from web161607.mail.bf1.yahoo.com ([98.139.210.176]:43394) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1PxNmM-0002H9-P6 for qemu-devel@nongnu.org; Wed, 09 Mar 2011 13:06:34 -0500 Message-ID: <732519.22405.qm@web161607.mail.bf1.yahoo.com> Date: Wed, 9 Mar 2011 10:06:33 -0800 (PST) From: SAURAV LAHIRI MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-694468262-1299693993=:22405" Subject: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org --0-694468262-1299693993=:22405 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Hello, I attempted to create snapshots outside the original qcow2 disk, a feature = that is available with 0.14.0, but facing an unexpected behaviour as descri= bed below. Any help is greatly appreciated. I am using qemu version 0.14 # qemu --version QEMU emulator version 0.14.0, Copyright (c) 2003-2008 Fabrice Bellard # qemu-img --version qemu-img version 0.14.0, Copyright (c) 2004-2008 Fabrice Bellard usage: qemu-img command [command options] QEMU disk image utility Following steps were executed to copy a snapshot outside the original qcow2= but looks like there is an issue. 1. VM(vm1) brought up with a qcow2 image(/home/user1/lucid-vm2). This is a = lucid 10.04 VM. 2. Next snapshot created for the qcow2 =A0=A0 #qemu-img snapshot -c snap1 /home/user1/lucid-vm2 3. Lets see if the snapshot got created. =A0=A0 =A0# qemu-img snapshot -l /home/user1/lucid-vm2=20 =A0=A0 =A0Snapshot list: =A0=A0 =A0ID=A0=A0=A0=A0=A0=A0=A0 TAG=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0 VM SIZE=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 DATE=A0= =A0=A0=A0=A0=A0 VM CLOCK =A0=A0 =A01=A0=A0=A0=A0=A0=A0=A0=A0 snap1=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0 0 2011-03-09 23:17:53=A0=A0 00:00:00.000 4. Next inside the running VM(vm2) a new file is created. Since the snapsho= t was =A0=A0=A0 created before creating this file. The snapshot should not have t= his file. =A0=A0 #touch samplefile 5. Now the snapshot snap1 is copied outside the original qcow2 image. =A0=A0 qemu-img convert -f qcow2 -O qcow2 -s snap1 /home/user1/lucid-vm2 /h= ome/user1/lucid-vm2-snap1 6. Next shutdown VM(vm1) and bring it up the snapshot image /home/user1/luc= id-vm2-snap1.=20 =A0=A0 Since the snapshot was created before creating the file named "sampl= efile" it is to =A0=A0 be expected that the file should not be there in the snapshot image.= But the file does appear=A0 in the snapshot image =A0=A0 #ls samplefile =A0=A0 samplefile =A0 =A0 Regards sl =0A=0A=0A --0-694468262-1299693993=:22405 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Hello,
I attempted to create snapshots out= side the original qcow2 disk, a feature that is available with 0.14.0, but = facing an unexpected behaviour as described below. Any help is greatly appr= eciated.

I am using qemu version 0.14

# qemu --version
QEM= U emulator version 0.14.0, Copyright (c) 2003-2008 Fabrice Bellard

<= br># qemu-img --version
qemu-img version 0.14.0, Copyright (c) 2004-2008= Fabrice Bellard
usage: qemu-img command [command options]
QEMU disk = image utility


Following steps were executed to copy a snapshot o= utside the original qcow2 but looks like there is an issue.

1. VM(vm= 1) brought up with a qcow2 image(/home/user1/lucid-vm2). This is a lucid 10= .04 VM.

2. Next snapshot created for the qcow2
   #qemu= -img snapshot -c snap1 /home/user1/lucid-vm2

3. Lets see if the snapshot got created.
    # qemu-img snapshot -l /home/u= ser1/lucid-vm2
    Snapshot list:
    = ID        TAG    &nb= sp;            VM SI= ZE            &= nbsp;   DATE       VM CLOCK
&nbs= p;   1         snap1 = ;            &n= bsp;       0 2011-03-09 23:17:53   = 00:00:00.000


4. Next inside the running VM(vm2) a new file is cr= eated. Since the snapshot was
    created before creating= this file. The snapshot should not have this file.
   #touch = samplefile

5. Now the snapshot snap1 is copied outside the original qcow2 image.
   qemu-img convert -f qcow2 -O qcow2 -= s snap1 /home/user1/lucid-vm2 /home/user1/lucid-vm2-snap1

6. Next sh= utdown VM(vm1) and bring it up the snapshot image /home/user1/lucid-vm2-sna= p1.
   Since the snapshot was created before creating the fil= e named "samplefile" it is to
   be expected that the file sho= uld not be there in the snapshot image. But the file does appear  in t= he snapshot image
   #ls samplefile
   samplefile=
   

Regards
sl


=0A=0A= =0A=0A --0-694468262-1299693993=:22405-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=53833 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PxPRj-0008Hi-7v for qemu-devel@nongnu.org; Wed, 09 Mar 2011 14:53:24 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PxPRi-0000ef-BB for qemu-devel@nongnu.org; Wed, 09 Mar 2011 14:53:23 -0500 Received: from mail-yw0-f45.google.com ([209.85.213.45]:58916) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1PxPRi-0000eb-8q for qemu-devel@nongnu.org; Wed, 09 Mar 2011 14:53:22 -0500 Received: by ywl41 with SMTP id 41so446321ywl.4 for ; Wed, 09 Mar 2011 11:53:21 -0800 (PST) MIME-Version: 1.0 In-Reply-To: <732519.22405.qm@web161607.mail.bf1.yahoo.com> References: <732519.22405.qm@web161607.mail.bf1.yahoo.com> Date: Wed, 9 Mar 2011 19:53:21 +0000 Message-ID: Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 From: Stefan Hajnoczi Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: SAURAV LAHIRI Cc: qemu-devel@nongnu.org On Wed, Mar 9, 2011 at 6:06 PM, SAURAV LAHIRI wro= te: > Following steps were executed to copy a snapshot outside the original qco= w2 but looks like there is an issue. > > 1. VM(vm1) brought up with a qcow2 image(/home/user1/lucid-vm2). This is = a lucid 10.04 VM. > > 2. Next snapshot created for the qcow2 > =A0=A0 #qemu-img snapshot -c snap1 /home/user1/lucid-vm2 It is not safe to run qemu-img while the VM is using the image file. Stefan From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=39742 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PxV2o-0000Gy-LN for qemu-devel@nongnu.org; Wed, 09 Mar 2011 20:52:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PxV2n-0006Rm-65 for qemu-devel@nongnu.org; Wed, 09 Mar 2011 20:52:02 -0500 Received: from web161620.mail.bf1.yahoo.com ([98.139.211.142]:37334) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1PxV2m-0006Rg-Ug for qemu-devel@nongnu.org; Wed, 09 Mar 2011 20:52:01 -0500 Message-ID: <9678.79494.qm@web161620.mail.bf1.yahoo.com> Date: Wed, 9 Mar 2011 17:51:59 -0800 (PST) From: SAURAV LAHIRI Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1958729861-1299721919=:79494" List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: qemu-devel@nongnu.org --0-1958729861-1299721919=:79494 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Thank you very much Stefan, Based on your suggestion I tried out the follow= ing two scenarios. Scenario 1: 1) I executed following with the vm in shutdown state. "#qemu-img snapshot -c snap1 /home/user1/lucid-vm2"=20 2) Brought the VM Up. 3) Inside the vm create a file. #touch samplefile 4) Shutdown the vm and copy the snapshot outside the original qcow2 #qemu-img convert -f qcow2 -O qcow2 -s snap1 /home/user1/lucid-vm2 /home/lu= cid-vm2-snap1 Result : When I bring up the vm do not see the samplefile which is the expe= cted behaviour. Scenario 2: =0A1) I executed following with the vm in shutdown state. =0A"#qemu-img snapshot -c snap1 /home/user1/lucid-vm2"=20 =0A 2) Bring up the VM. 3) Inside the vm create a file. =0A#touch samplefile =0A 4) VM is NOT Shutdown and copy the snapshot outside the original qcow2 =0A =0A#qemu-img convert -f qcow2 -O qcow2 -s snap1 /home/user1/lucid-vm2 /home= /lucid-vm2-snap1 =0A =0AResult : When I bring up the vm do not see the samplefile which is the e= xpected behaviour. =0A Is Scenario 2 safe where in the copying the snapshot outside the original q= cow2 is being executed with the VM running. This is because if this is safe= then this could be an approach as it would not require a long downtime for= the VM.=20 Regards sl --- On Wed, 9/3/11, Stefan Hajnoczi wrote: From: Stefan Hajnoczi Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.1= 4.0 To: "SAURAV LAHIRI" Cc: qemu-devel@nongnu.org Date: Wednesday, 9 March, 2011, 15:53 On Wed, Mar 9, 2011 at 6:06 PM, SAURAV LAHIRI wro= te: > Following steps were executed to copy a snapshot outside the original qco= w2 but looks like there is an issue. > > 1. VM(vm1) brought up with a qcow2 image(/home/user1/lucid-vm2). This is = a lucid 10.04 VM. > > 2. Next snapshot created for the qcow2 > =A0=A0 #qemu-img snapshot -c snap1 /home/user1/lucid-vm2 It is not safe to run qemu-img while the VM is using the image file. Stefan =0A=0A=0A --0-1958729861-1299721919=:79494 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Thank you very much Stefan, Based on your sug= gestion I tried out the following two scenarios.

Scenario 1:
1) I= executed following with the vm in shutdown state.
"#qemu-img snapshot -= c snap1 /home/user1/lucid-vm2"

2) Brought the VM Up.

3) Insi= de the vm create a file.
#touch samplefile

4) Shutdown the vm and= copy the snapshot outside the original qcow2

#qemu-img convert -f q= cow2 -O qcow2 -s snap1 /home/user1/lucid-vm2 /home/lucid-vm2-snap1

R= esult : When I bring up the vm do not see the samplefile which is the expec= ted behaviour.

Scenario 2:
=0A1) I executed following with the vm= in shutdown state.
=0A"#qemu-img snapshot -c snap1 /home/user1/lucid-vm= 2"
=0A
2) Bring up the VM.

3) Inside the vm create a file.=0A#touch samplefile
=0A
4) VM is NOT Shutdown and copy the snapshot= outside the original qcow2
=0A
=0A#qemu-img convert -f qcow2 -O qcow= 2 -s snap1 /home/user1/lucid-vm2 /home/lucid-vm2-snap1
=0A
=0AResult = : When I bring up the vm do not see the samplefile which is the expected be= haviour.
=0A

Is Scenario 2 safe where in the copying the snapshot= outside the original qcow2 is being executed with the VM running. This is = because if this is safe then this could be an approach as it would not requ= ire a long downtime for the VM.

Regards
sl

--- On Wed,= 9/3/11, Stefan Hajnoczi <stefanha@gmail.com> wrote:

From: Stefan Hajnoczi <stefanha@gmail.com>=
Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu= 0.14.0
To: "SAURAV LAHIRI" <saurav_lahiri@yahoo.com>
Cc: qemu-= devel@nongnu.org
Date: Wednesday, 9 March, 2011, 15:53

On Wed, Mar 9, 2011 at 6:06 PM, SAURAV LAHIRI <saurav_lahiri@yahoo.com> wrote:
> Following steps w= ere executed to copy a snapshot outside the original qcow2 but looks like there is an issu= e.
>
> 1. VM(vm1) brought up with a qcow2 image(/home/user1/luc= id-vm2). This is a lucid 10.04 VM.
>
> 2. Next snapshot created= for the qcow2
>    #qemu-img snapshot -c snap1 /home/user1= /lucid-vm2

It is not safe to run qemu-img while the VM is using the = image file.

Stefan

=0A= =0A=0A=0A --0-1958729861-1299721919=:79494-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=60812 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Pxc98-0001I8-I8 for qemu-devel@nongnu.org; Thu, 10 Mar 2011 04:27:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Pxc96-00079E-QJ for qemu-devel@nongnu.org; Thu, 10 Mar 2011 04:27:02 -0500 Received: from mail-yx0-f173.google.com ([209.85.213.173]:39067) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Pxc96-000790-Nu for qemu-devel@nongnu.org; Thu, 10 Mar 2011 04:27:00 -0500 Received: by yxk8 with SMTP id 8so704574yxk.4 for ; Thu, 10 Mar 2011 01:27:00 -0800 (PST) MIME-Version: 1.0 In-Reply-To: <9678.79494.qm@web161620.mail.bf1.yahoo.com> References: <9678.79494.qm@web161620.mail.bf1.yahoo.com> Date: Thu, 10 Mar 2011 09:27:00 +0000 Message-ID: Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 From: Stefan Hajnoczi Content-Type: text/plain; charset=ISO-8859-1 List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: SAURAV LAHIRI Cc: Jes Sorensen , qemu-devel@nongnu.org On Thu, Mar 10, 2011 at 1:51 AM, SAURAV LAHIRI wrote: > Scenario 1: > 1) I executed following with the vm in shutdown state. > "#qemu-img snapshot -c snap1 /home/user1/lucid-vm2" Here you are snapshotting the current disk image and storing the snapshot away as "snap1". > 2) Brought the VM Up. > > 3) Inside the vm create a file. > #touch samplefile Now you modified the current disk image but "snap1" remains unchanged. > 4) Shutdown the vm and copy the snapshot outside the original qcow2 > > #qemu-img convert -f qcow2 -O qcow2 -s snap1 /home/user1/lucid-vm2 /home/lucid-vm2-snap1 > > Result : When I bring up the vm do not see the samplefile which is the expected behaviour. Are you bringing up the VM with lucid-vm2 (which should have samplefile) or lucid-vm2-snap1 (which should not have samplefile)? > Scenario 2: > 1) I executed following with the vm in shutdown state. > "#qemu-img snapshot -c snap1 /home/user1/lucid-vm2" > > 2) Bring up the VM. > > 3) Inside the vm create a file. > #touch samplefile > > 4) VM is NOT Shutdown and copy the snapshot outside the original qcow2 > > #qemu-img convert -f qcow2 -O qcow2 -s snap1 /home/user1/lucid-vm2 /home/lucid-vm2-snap1 > > Result : When I bring up the vm do not see the samplefile which is the expected behaviour. > > > Is Scenario 2 safe where in the copying the snapshot outside the original qcow2 is being executed with the VM running. This is because if this is safe then this could be an approach as it would not require a long downtime for the VM. There is no guarantee that qemu-img will work on an image file that is open by a running VM. I have CCed Jes who has been working on a live snapshot mechanism. He recently added the snapshot_blkdev monitor command that takes a snapshot of a block device while the VM is running. A new image file is created based off the original image file (which will no longer be modified), all new disk writes go to the new image file. It is safe to perform read-only access to the original image file. There currently is no support to merge the snapshot changes back into the original image while the VM is running, but I think that is the next planned step. If you can describe your snapshot use case at a higher level that might be useful. Stefan From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=56568 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PxcEo-0004Ev-M5 for qemu-devel@nongnu.org; Thu, 10 Mar 2011 04:32:55 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PxcEn-0008BX-Cz for qemu-devel@nongnu.org; Thu, 10 Mar 2011 04:32:54 -0500 Received: from mx1.redhat.com ([209.132.183.28]:18273) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1PxcEn-0008BM-4g for qemu-devel@nongnu.org; Thu, 10 Mar 2011 04:32:53 -0500 Message-ID: <4D789AB4.4010202@redhat.com> Date: Thu, 10 Mar 2011 10:32:36 +0100 From: Jes Sorensen MIME-Version: 1.0 Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 References: <9678.79494.qm@web161620.mail.bf1.yahoo.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: SAURAV LAHIRI , qemu-devel@nongnu.org On 03/10/11 10:27, Stefan Hajnoczi wrote: >> Is Scenario 2 safe where in the copying the snapshot outside the original qcow2 is being executed with the VM running. This is because if this is safe then this could be an approach as it would not require a long downtime for the VM. > > There is no guarantee that qemu-img will work on an image file that is > open by a running VM. I think the guarantee here is that you're guaranteed it will go horribly wrong if you try to do so..... > I have CCed Jes who has been working on a live snapshot mechanism. He > recently added the snapshot_blkdev monitor command that takes a > snapshot of a block device while the VM is running. A new image file > is created based off the original image file (which will no longer be > modified), all new disk writes go to the new image file. It is safe > to perform read-only access to the original image file. There > currently is no support to merge the snapshot changes back into the > original image while the VM is running, but I think that is the next > planned step. Yes, keep in mind that the live snapshot is only for external snapshot files, it doesn't deal with internal snapshots. Cheers, Jes From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=49446 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PxcdC-0006Ta-F9 for qemu-devel@nongnu.org; Thu, 10 Mar 2011 04:58:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Pxcd9-0005Fx-H3 for qemu-devel@nongnu.org; Thu, 10 Mar 2011 04:58:06 -0500 Received: from mail-yx0-f173.google.com ([209.85.213.173]:39610) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Pxcd9-0005Ff-Em for qemu-devel@nongnu.org; Thu, 10 Mar 2011 04:58:03 -0500 Received: by yxk8 with SMTP id 8so714896yxk.4 for ; Thu, 10 Mar 2011 01:58:02 -0800 (PST) MIME-Version: 1.0 In-Reply-To: <4D789AB4.4010202@redhat.com> References: <9678.79494.qm@web161620.mail.bf1.yahoo.com> <4D789AB4.4010202@redhat.com> Date: Thu, 10 Mar 2011 09:58:02 +0000 Message-ID: Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 From: Stefan Hajnoczi Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Jes Sorensen Cc: SAURAV LAHIRI , qemu-devel@nongnu.org On Thu, Mar 10, 2011 at 9:32 AM, Jes Sorensen wro= te: > On 03/10/11 10:27, Stefan Hajnoczi wrote: >> I have CCed Jes who has been working on a live snapshot mechanism. =A0He >> recently added the snapshot_blkdev monitor command that takes a >> snapshot of a block device while the VM is running. =A0A new image file >> is created based off the original image file (which will no longer be >> modified), all new disk writes go to the new image file. =A0It is safe >> to perform read-only access to the original image file. =A0There >> currently is no support to merge the snapshot changes back into the >> original image while the VM is running, but I think that is the next >> planned step. > > Yes, keep in mind that the live snapshot is only for external snapshot > files, it doesn't deal with internal snapshots. Yep, that's why I'm interested in Saurav's use case. Many use cases work with either internal or external snapshot but it depends on the details. Stefan From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=50361 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PxceQ-000788-Nn for qemu-devel@nongnu.org; Thu, 10 Mar 2011 04:59:24 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PxceP-0005fv-PN for qemu-devel@nongnu.org; Thu, 10 Mar 2011 04:59:22 -0500 Received: from mx1.redhat.com ([209.132.183.28]:45990) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1PxceP-0005fb-Fn for qemu-devel@nongnu.org; Thu, 10 Mar 2011 04:59:21 -0500 Message-ID: <4D78A0E8.7000606@redhat.com> Date: Thu, 10 Mar 2011 10:59:04 +0100 From: Jes Sorensen MIME-Version: 1.0 Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 References: <9678.79494.qm@web161620.mail.bf1.yahoo.com> <4D789AB4.4010202@redhat.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: SAURAV LAHIRI , qemu-devel@nongnu.org On 03/10/11 10:58, Stefan Hajnoczi wrote: > On Thu, Mar 10, 2011 at 9:32 AM, Jes Sorensen wrote: >> On 03/10/11 10:27, Stefan Hajnoczi wrote: >>> I have CCed Jes who has been working on a live snapshot mechanism. He >>> recently added the snapshot_blkdev monitor command that takes a >>> snapshot of a block device while the VM is running. A new image file >>> is created based off the original image file (which will no longer be >>> modified), all new disk writes go to the new image file. It is safe >>> to perform read-only access to the original image file. There >>> currently is no support to merge the snapshot changes back into the >>> original image while the VM is running, but I think that is the next >>> planned step. >> >> Yes, keep in mind that the live snapshot is only for external snapshot >> files, it doesn't deal with internal snapshots. > > Yep, that's why I'm interested in Saurav's use case. Many use cases > work with either internal or external snapshot but it depends on the > details. Actually I think there's very little reason to keep internal snapshot support. It doesn't buy us much, but it adds unnecessary complexity. Cheers, Jes From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=49749 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Pxlyu-0004Vi-Hn for qemu-devel@nongnu.org; Thu, 10 Mar 2011 14:57:10 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Pxlyt-0005iY-0U for qemu-devel@nongnu.org; Thu, 10 Mar 2011 14:57:08 -0500 Received: from web161602.mail.bf1.yahoo.com ([98.139.210.171]:34860) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1Pxlys-0005iR-Ps for qemu-devel@nongnu.org; Thu, 10 Mar 2011 14:57:06 -0500 Message-ID: <102956.23165.qm@web161602.mail.bf1.yahoo.com> Date: Thu, 10 Mar 2011 11:57:05 -0800 (PST) From: SAURAV LAHIRI Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 In-Reply-To: <4D78A0E8.7000606@redhat.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-708760848-1299787025=:23165" List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi , Jes Sorensen Cc: qemu-devel@nongnu.org --0-708760848-1299787025=:23165 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Thank you Stefan and Jes for providing further inputs. Details on use case: The high level use case is that of being able to backup user specified disk= s of a VM without having to bring down the VM.=20 That was the reason that I had started of with running the "qemu-img snapsh= ot -c snap1..." on a running vm. So when I have a snapshot i can back this = up. Without a frozen and consistent snapshot, backing up the qcow2 disk ima= ge would cause serious issues on restore. But after your inputs shelved the= plans of creating snapshot on running vm. So next approach planned did take into account that=A0 I have to shutdown t= he VM for creating the snapshot. It would appear that, since the internal s= napshot creation is an instantaneous process I could bring down the vm, cre= ate an internal snapshot, bring up the VM. and this entire thing would take= a minimal time.=20 Once the VM is up I can continue with converting the internal snapshot to a= n external snapshot and then back up the external snapshot to my backup loc= ation. The duration of conversion from internal to external obviously would= depend on the size of the original qcow2 disk. But since the VM is up the = duration would not concern me much as long as it completes within some prac= tical time limits. snapshot_blkdev: Regarding this=A0 I do have a couple of questions. 1. If the snapshot cannot be merged then it could mean that there are sever= al snapshot files. One readonly=A0 for each of the previous snapshots and t= he last one being the active one, which handles all the current writes. Pos= t backup If do have to restore to a particular snapshot then i would probab= ly have to copy all the files in the chain and maintain the entire chain. B= ut would it not affect read performance if several snapshot files are maint= ained, particularly if the VM is hosting a database like mysql ? Could you = please clarify. 2. I have seen that at times the qemu monitor command is not able to connne= ct to=A0 the monitor socket as libvirt it seems controls the monitor socket= . If I shutdown libvirt then commands like socat is able to connect. But si= nce my current environment does use libvirt, shutting down libvirt is not a= n option. Is there any way around this ? Appreciate your help. Regards sl --- On Thu, 10/3/11, Jes Sorensen wrote: From: Jes Sorensen Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.1= 4.0 To: "Stefan Hajnoczi" Cc: "SAURAV LAHIRI" , qemu-devel@nongnu.org Date: Thursday, 10 March, 2011, 5:59 On 03/10/11 10:58, Stefan Hajnoczi wrote: > On Thu, Mar 10, 2011 at 9:32 AM, Jes Sorensen w= rote: >> On 03/10/11 10:27, Stefan Hajnoczi wrote: >>> I have CCed Jes who has been working on a live snapshot mechanism.=A0 H= e >>> recently added the snapshot_blkdev monitor command that takes a >>> snapshot of a block device while the VM is running.=A0 A new image file >>> is created based off the original image file (which will no longer be >>> modified), all new disk writes go to the new image file.=A0 It is safe >>> to perform read-only access to the original image file.=A0 There >>> currently is no support to merge the snapshot changes back into the >>> original image while the VM is running, but I think that is the next >>> planned step. >> >> Yes, keep in mind that the live snapshot is only for external snapshot >> files, it doesn't deal with internal snapshots. >=20 > Yep, that's why I'm interested in Saurav's use case.=A0 Many use cases > work with either internal or external snapshot but it depends on the > details. Actually I think there's very little reason to keep internal snapshot support. It doesn't buy us much, but it adds unnecessary complexity. Cheers, Jes =0A=0A=0A --0-708760848-1299787025=:23165 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Thank you Stefan and Jes for providing furthe= r inputs.

Details on use case:

The high level use case is tha= t of being able to backup user specified disks of a VM without having to br= ing down the VM.

That was the reason that I had started of with run= ning the "qemu-img snapshot -c snap1..." on a running vm. So when I have a = snapshot i can back this up. Without a frozen and consistent snapshot, back= ing up the qcow2 disk image would cause serious issues on restore. But afte= r your inputs shelved the plans of creating snapshot on running vm.

= So next approach planned did take into account that  I have to shutdow= n the VM for creating the snapshot. It would appear that, since the interna= l snapshot creation is an instantaneous process I could bring down the vm, = create an internal snapshot, bring up the VM. and this entire thing would t= ake a minimal time.

Once the VM is up I can continue with converting t= he internal snapshot to an external snapshot and then back up the external = snapshot to my backup location. The duration of conversion from internal to= external obviously would depend on the size of the original qcow2 disk. Bu= t since the VM is up the duration would not concern me much as long as it c= ompletes within some practical time limits.


snapshot_blkdev: Reg= arding this  I do have a couple of questions.

1. If the snapsho= t cannot be merged then it could mean that there are several snapshot files= . One readonly  for each of the previous snapshots and the last one be= ing the active one, which handles all the current writes. Post backup If do= have to restore to a particular snapshot then i would probably have to cop= y all the files in the chain and maintain the entire chain. But would it no= t affect read performance if several snapshot files are maintained, particularly if the VM is hosting a database like mysql ? Could you please= clarify.

2. I have seen that at times the qemu monitor command is n= ot able to connnect to  the monitor socket as libvirt it seems control= s the monitor socket. If I shutdown libvirt then commands like socat is abl= e to connect. But since my current environment does use libvirt, shutting d= own libvirt is not an option. Is there any way around this ?

Appreci= ate your help.

Regards
sl









=
--- On Thu, 10/3/11, Jes Sorensen <Jes.Sorensen@redhat.com>= wrote:

From: Jes Sorensen <Jes.= Sorensen@redhat.com>
Subject: Re: [Qemu-devel] Issue with snapshot ou= tside qcow2 disk - qemu 0.14.0
To: "Stefan Hajnoczi" <stefanha@gmail.= com>
Cc: "SAURAV LAHIRI" <saurav_lahiri@yahoo.com>, qemu-devel@nongnu.org
Date: Thursday, 10 March, 2011, 5:59

On 03/10/11 10:58, Stefan Hajnoczi wrote:
> On Th= u, Mar 10, 2011 at 9:32 AM, Jes Sorensen <Jes.Soren= sen@redhat.com> wrote:
>> On 03/10/11 10:27, Stefan Hajnocz= i wrote:
>>> I have CCed Jes who has been working on a live sna= pshot mechanism.  He
>>> recently added the snapshot_blkde= v monitor command that takes a
>>> snapshot of a block device w= hile the VM is running.  A new image file
>>> is created b= ased off the original image file (which will no longer be
>>> m= odified), all new disk writes go to the new image file.  It is safe>>> to perform read-only access to the original image file. = There
>>> currently is no support to merge the snapshot change= s back into the
>>> original image while the VM is running, but = I think that is the next
>>> planned step.
>>
>&= gt; Yes, keep in mind that the live snapshot is only for external snapshot<= br>>> files, it doesn't deal with internal snapshots.
>
>= ; Yep, that's why I'm interested in Saurav's use case.  Many use cases=
> work with either internal or external snapshot but it depends on t= he
> details.

Actually I think there's very little reason to k= eep internal snapshot
support. It doesn't buy us much, but it adds unnec= essary complexity.

Cheers,
Jes


=0A=0A=0A=0A --0-708760848-1299787025=:23165-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=37548 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Pxn2D-0006fr-7T for qemu-devel@nongnu.org; Thu, 10 Mar 2011 16:04:39 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Pxn2A-0001zb-KY for qemu-devel@nongnu.org; Thu, 10 Mar 2011 16:04:36 -0500 Received: from mail-gx0-f173.google.com ([209.85.161.173]:61172) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Pxn2A-0001zR-IH for qemu-devel@nongnu.org; Thu, 10 Mar 2011 16:04:34 -0500 Received: by gxk28 with SMTP id 28so1009070gxk.4 for ; Thu, 10 Mar 2011 13:04:33 -0800 (PST) MIME-Version: 1.0 In-Reply-To: <102956.23165.qm@web161602.mail.bf1.yahoo.com> References: <4D78A0E8.7000606@redhat.com> <102956.23165.qm@web161602.mail.bf1.yahoo.com> Date: Thu, 10 Mar 2011 21:04:32 +0000 Message-ID: Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 From: Stefan Hajnoczi Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: SAURAV LAHIRI Cc: Jes Sorensen , qemu-devel@nongnu.org On Thu, Mar 10, 2011 at 7:57 PM, SAURAV LAHIRI wr= ote: > The high level use case is that of being able to backup user specified di= sks of a VM without having to bring down the VM. Excellent, that sounds exactly like Jes is addressing so future QEMU/KVM releases will hopefully have the live snapshot/merge capability. > snapshot_blkdev: Regarding this=A0 I do have a couple of questions. > > 1. If the snapshot cannot be merged then it could mean that there are sev= eral snapshot files. One readonly=A0 for each of the previous snapshots and= the last one being the active one, which handles all the current writes. P= ost backup If do have to restore to a particular snapshot then i would prob= ably have to copy all the files in the chain and maintain the entire chain.= But would it not affect read performance if several snapshot files are mai= ntained, particularly if the VM is hosting a database like mysql ? Could yo= u please clarify. If the VM is not running you can use the qemu-img commit command to merge the snapshot back down into the base image. After that you only have one image file again and can restart the VM. Hopefully the deltas are small enough that this process is quick. In the future a live merge command will take care of this and avoid the downtime. > 2. I have seen that at times the qemu monitor command is not able to conn= nect to=A0 the monitor socket as libvirt it seems controls the monitor sock= et. If I shutdown libvirt then commands like socat is able to connect. But = since my current environment does use libvirt, shutting down libvirt is not= an option. Is there any way around this ? New versions of libvirt have a virsh "qemu-monitor-command" command you can use to send a QEMU monitor command. Stefan From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=50267 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PxvtW-0007Ww-40 for qemu-devel@nongnu.org; Fri, 11 Mar 2011 01:32:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PxvtV-0002SR-2h for qemu-devel@nongnu.org; Fri, 11 Mar 2011 01:32:13 -0500 Received: from mx1.redhat.com ([209.132.183.28]:16129) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1PxvtU-0002RQ-P5 for qemu-devel@nongnu.org; Fri, 11 Mar 2011 01:32:13 -0500 Message-ID: <4D79C1D7.1030105@redhat.com> Date: Fri, 11 Mar 2011 07:31:51 +0100 From: Jes Sorensen MIME-Version: 1.0 Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 References: <4D78A0E8.7000606@redhat.com> <102956.23165.qm@web161602.mail.bf1.yahoo.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: SAURAV LAHIRI , qemu-devel@nongnu.org On 03/10/11 22:04, Stefan Hajnoczi wrote: > On Thu, Mar 10, 2011 at 7:57 PM, SAURAV LAHIRI wrote: >> The high level use case is that of being able to backup user specified disks of a VM without having to bring down the VM. > > Excellent, that sounds exactly like Jes is addressing so future > QEMU/KVM releases will hopefully have the live snapshot/merge > capability. > >> snapshot_blkdev: Regarding this I do have a couple of questions. >> >> 1. If the snapshot cannot be merged then it could mean that there are several snapshot files. One readonly for each of the previous snapshots and the last one being the active one, which handles all the current writes. Post backup If do have to restore to a particular snapshot then i would probably have to copy all the files in the chain and maintain the entire chain. But would it not affect read performance if several snapshot files are maintained, particularly if the VM is hosting a database like mysql ? Could you please clarify. > > If the VM is not running you can use the qemu-img commit command to > merge the snapshot back down into the base image. After that you only > have one image file again and can restart the VM. Hopefully the > deltas are small enough that this process is quick. > > In the future a live merge command will take care of this and avoid > the downtime. Yep, qemu-img convert should be able to copy it into a single image so you can delete the chain. Cheers, Jes From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=44401 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PxyoY-00057O-O1 for qemu-devel@nongnu.org; Fri, 11 Mar 2011 04:39:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PxyoW-0004wE-VL for qemu-devel@nongnu.org; Fri, 11 Mar 2011 04:39:18 -0500 Received: from web161605.mail.bf1.yahoo.com ([98.139.210.174]:38429) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1PxyoW-0004vr-P8 for qemu-devel@nongnu.org; Fri, 11 Mar 2011 04:39:16 -0500 Message-ID: <920767.76291.qm@web161605.mail.bf1.yahoo.com> Date: Fri, 11 Mar 2011 01:39:15 -0800 (PST) From: SAURAV LAHIRI Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 In-Reply-To: <4D79C1D7.1030105@redhat.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1235860719-1299836355=:76291" List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi , Jes Sorensen Cc: qemu-devel@nongnu.org --0-1235860719-1299836355=:76291 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Thank you Stefan, Jes. So it appears that snapshot_blkdev is the way to go = for vm disk backup with running vms. In regard to merging changes, assuming that we go snapshot_blkdev rightaway= . Stefan's suggestion: "qemu-img commit" Jes's suggestion: "qemu-img convert" Does qemu-img convert apply to running VM's. In that case it would appear t= o be the more practical approach(since vm shutdown would not be required). Also incase If i have interpreted "qemu-img convert" incorrectly and does r= equire a VM shutdown. Then when is expected time when the "live merge" will= be available. Thanks and Regards Saurav Lahiri --- On Fri, 11/3/11, Jes Sorensen wrote: From: Jes Sorensen Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.1= 4.0 To: "Stefan Hajnoczi" Cc: "SAURAV LAHIRI" , qemu-devel@nongnu.org Date: Friday, 11 March, 2011, 2:31 On 03/10/11 22:04, Stefan Hajnoczi wrote: > On Thu, Mar 10, 2011 at 7:57 PM, SAURAV LAHIRI = wrote: >> The high level use case is that of being able to backup user specified d= isks of a VM without having to bring down the VM. >=20 > Excellent, that sounds exactly like Jes is addressing so future > QEMU/KVM releases will hopefully have the live snapshot/merge > capability. >=20 >> snapshot_blkdev: Regarding this=A0 I do have a couple of questions. >> >> 1. If the snapshot cannot be merged then it could mean that there are se= veral snapshot files. One readonly=A0 for each of the previous snapshots an= d the last one being the active one, which handles all the current writes. = Post backup If do have to restore to a particular snapshot then i would pro= bably have to copy all the files in the chain and maintain the entire chain= . But would it not affect read performance if several snapshot files are ma= intained, particularly if the VM is hosting a database like mysql ? Could y= ou please clarify. >=20 > If the VM is not running you can use the qemu-img commit command to > merge the snapshot back down into the base image.=A0 After that you only > have one image file again and can restart the VM.=A0 Hopefully the > deltas are small enough that this process is quick. >=20 > In the future a live merge command will take care of this and avoid > the downtime. Yep, qemu-img convert should be able to copy it into a single image so you can delete the chain. Cheers, Jes =0A=0A=0A --0-1235860719-1299836355=:76291 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Thank you Stefan, Jes. So it appears that sna= pshot_blkdev is the way to go for vm disk backup with running vms.

I= n regard to merging changes, assuming that we go snapshot_blkdev rightaway.=
Stefan's suggestion: "qemu-img commit"
Jes's suggestion: "qemu-img c= onvert"

Does qemu-img convert apply to running VM's. In that case it= would appear to be the more practical approach(since vm shutdown would not= be required).


Also incase If i have interpreted "qemu-img conve= rt" incorrectly and does require a VM shutdown. Then when is expected time = when the "live merge" will be available.


Thanks and Regards
S= aurav Lahiri

--- On Fri, 11/3/11, Jes Sorensen <Jes.Sorense= n@redhat.com> wrote:

From: Jes Sorensen <Jes.Sorensen@redhat.com>
Subject: Re: [Qemu-devel] Issu= e with snapshot outside qcow2 disk - qemu 0.14.0
To: "Stefan Hajnoczi" &= lt;stefanha@gmail.com>
Cc: "SAURAV LAHIRI" <saurav_lahiri@yahoo.co= m>, qemu-devel@nongnu.org
Date: Friday, 11 March, 2011, 2:31

<= div class=3D"plainMail">On 03/10/11 22:04, Stefan Hajnoczi wrote:
> O= n Thu, Mar 10, 2011 at 7:57 PM, SAURAV LAHIRI <saur= av_lahiri@yahoo.com> wrote:
>> The high level use case is t= hat of being able to backup user specified disks of a VM without having to = bring down the VM.
>
> Excellent, that sounds exactly like Jes= is addressing so future
> QEMU/KVM releases will hopefully have the = live snapshot/merge
> capability.
>
>> snapshot_blkde= v: Regarding this  I do have a couple of questions.
>>
>> 1. If the snapshot cannot be merged the= n it could mean that there are several snapshot files. One readonly  f= or each of the previous snapshots and the last one being the active one, wh= ich handles all the current writes. Post backup If do have to restore to a = particular snapshot then i would probably have to copy all the files in the= chain and maintain the entire chain. But would it not affect read performa= nce if several snapshot files are maintained, particularly if the VM is hos= ting a database like mysql ? Could you please clarify.
>
> If = the VM is not running you can use the qemu-img commit command to
> me= rge the snapshot back down into the base image.  After that you only> have one image file again and can restart the VM.  Hopefully th= e
> deltas are small enough that this process is quick.
>
&= gt; In the future a live merge command will take care of this and avoid
> the downtime.

Yep, qemu-img convert should be able to= copy it into a single image so
you can delete the chain.

Cheers,=
Jes


=0A=0A=0A=0A --0-1235860719-1299836355=:76291-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=49290 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Pxyzh-0003xk-FN for qemu-devel@nongnu.org; Fri, 11 Mar 2011 04:50:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Pxyzf-0000aq-Oq for qemu-devel@nongnu.org; Fri, 11 Mar 2011 04:50:48 -0500 Received: from mx1.redhat.com ([209.132.183.28]:43506) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Pxyzf-0000aI-Cx for qemu-devel@nongnu.org; Fri, 11 Mar 2011 04:50:47 -0500 Message-ID: <4D79F063.7040007@redhat.com> Date: Fri, 11 Mar 2011 10:50:27 +0100 From: Jes Sorensen MIME-Version: 1.0 Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 References: <920767.76291.qm@web161605.mail.bf1.yahoo.com> In-Reply-To: <920767.76291.qm@web161605.mail.bf1.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: SAURAV LAHIRI Cc: Stefan Hajnoczi , qemu-devel@nongnu.org On 03/11/11 10:39, SAURAV LAHIRI wrote: > Thank you Stefan, Jes. So it appears that snapshot_blkdev is the way > to go for vm disk backup with running vms. > > In regard to merging changes, assuming that we go snapshot_blkdev > rightaway. Stefan's suggestion: "qemu-img commit" Jes's suggestion: > "qemu-img convert" > > Does qemu-img convert apply to running VM's. In that case it would > appear to be the more practical approach(since vm shutdown would not > be required). > > Also incase If i have interpreted "qemu-img convert" incorrectly and > does require a VM shutdown. Then when is expected time when the "live > merge" will be available. I believe commit only applies to images with internal files. If you use convert then it doesn't modify the actual images, so lets say you have a chain like this: original->snapshotA->snapshotB original and snapshotA are read-only when snapshotB is running. Therefore you should be able to use convert to simply copy snapshotA into a new image file snapshotX and save that for your backup. If you later restore, you have a single image file you can boot from. What you cannot do is to create the new snapshotX file and switch to it as the backing file for snapshotB while you are up and running. Cheers, Jes From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=49160 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Py0wP-00041R-Kv for qemu-devel@nongnu.org; Fri, 11 Mar 2011 06:55:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Py0wK-0000WR-Gx for qemu-devel@nongnu.org; Fri, 11 Mar 2011 06:55:33 -0500 Received: from mail-yw0-f45.google.com ([209.85.213.45]:35068) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Py0wK-0000WK-CD for qemu-devel@nongnu.org; Fri, 11 Mar 2011 06:55:28 -0500 Received: by ywl41 with SMTP id 41so1357050ywl.4 for ; Fri, 11 Mar 2011 03:55:27 -0800 (PST) MIME-Version: 1.0 In-Reply-To: <4D79F063.7040007@redhat.com> References: <920767.76291.qm@web161605.mail.bf1.yahoo.com> <4D79F063.7040007@redhat.com> Date: Fri, 11 Mar 2011 11:55:27 +0000 Message-ID: Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 From: Stefan Hajnoczi Content-Type: text/plain; charset=ISO-8859-1 List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: SAURAV LAHIRI Cc: Jes Sorensen , qemu-devel@nongnu.org On Fri, Mar 11, 2011 at 9:50 AM, Jes Sorensen wrote: > On 03/11/11 10:39, SAURAV LAHIRI wrote: >> Thank you Stefan, Jes. So it appears that snapshot_blkdev is the way >> to go for vm disk backup with running vms. >> >> In regard to merging changes, assuming that we go snapshot_blkdev >> rightaway. Stefan's suggestion: "qemu-img commit" Jes's suggestion: >> "qemu-img convert" >> >> Does qemu-img convert apply to running VM's. In that case it would >> appear to be the more practical approach(since vm shutdown would not >> be required). >> >> Also incase If i have interpreted "qemu-img convert" incorrectly and >> does require a VM shutdown. Then when is expected time when the "live >> merge" will be available. > > I believe commit only applies to images with internal files. If you use Nope, qemu-img commit only applies to images with backing files (aka external snapshots). > convert then it doesn't modify the actual images, so lets say you have a > chain like this: > > original->snapshotA->snapshotB > > original and snapshotA are read-only when snapshotB is running. > Therefore you should be able to use convert to simply copy snapshotA > into a new image file snapshotX and save that for your backup. If you > later restore, you have a single image file you can boot from. > > What you cannot do is to create the new snapshotX file and switch to it > as the backing file for snapshotB while you are up and running. qemu-img commit merges the deltas from a qcow2 (or qed or other copy-on-write image format) into the backing file. If you have the following: qemu-img create -f qcow2 -o backing_file=master.img,backing_fmt=raw instance.qcow2 Then the following command will merge the delta from instance.qcow2 back into master.img: qemu-img commit instance.qcow2 My suggestion was to shut down the VM and run qemu-img commit so that you can move back to the original disk image file which has now been updated with the deltas. You no longer need the file with the deltas. So qemu-img commit lets you merge down the chain of snapshots, whereas qemu-img convert lets you copy a full disk image into a new image file. To be clear, qemu-img should not be used on an image file that is currently open by a running VM. Stefan From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=47534 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Py5t6-0004yi-Lq for qemu-devel@nongnu.org; Fri, 11 Mar 2011 12:12:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Py5t4-0007ij-JE for qemu-devel@nongnu.org; Fri, 11 Mar 2011 12:12:28 -0500 Received: from web161605.mail.bf1.yahoo.com ([98.139.210.174]:21809) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1Py5t4-0007iV-9d for qemu-devel@nongnu.org; Fri, 11 Mar 2011 12:12:26 -0500 Message-ID: <646085.78086.qm@web161605.mail.bf1.yahoo.com> Date: Fri, 11 Mar 2011 09:12:25 -0800 (PST) From: SAURAV LAHIRI Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 In-Reply-To: <4D79F063.7040007@redhat.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1243797253-1299863545=:78086" List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Jes Sorensen Cc: Stefan Hajnoczi , qemu-devel@nongnu.org --0-1243797253-1299863545=:78086 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Thank you very much Stefan and Jes for brining about clarity to the issue o= f snapshots in qemu.=A0 One last question which we would be interested in i= s when is the back merge of snapshots for running vms expected to be releas= ed. Regards sl --- On Fri, 11/3/11, Jes Sorensen wrote: From: Jes Sorensen Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.1= 4.0 To: "SAURAV LAHIRI" Cc: "Stefan Hajnoczi" , qemu-devel@nongnu.org Date: Friday, 11 March, 2011, 5:50 On 03/11/11 10:39, SAURAV LAHIRI wrote: > Thank you Stefan, Jes. So it appears that snapshot_blkdev is the way > to go for vm disk backup with running vms. >=20 > In regard to merging changes, assuming that we go snapshot_blkdev > rightaway. Stefan's suggestion: "qemu-img commit" Jes's suggestion: > "qemu-img convert" >=20 > Does qemu-img convert apply to running VM's. In that case it would > appear to be the more practical approach(since vm shutdown would not > be required). >=20 > Also incase If i have interpreted "qemu-img convert" incorrectly and > does require a VM shutdown. Then when is expected time when the "live > merge" will be available. I believe commit only applies to images with internal files. If you use convert then it doesn't modify the actual images, so lets say you have a chain like this: original->snapshotA->snapshotB original and snapshotA are read-only when snapshotB is running. Therefore you should be able to use convert to simply copy snapshotA into a new image file snapshotX and save that for your backup. If you later restore, you have a single image file you can boot from. What you cannot do is to create the new snapshotX file and switch to it as the backing file for snapshotB while you are up and running. Cheers, Jes =0A=0A=0A --0-1243797253-1299863545=:78086 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Thank you very much Stefan and Jes for brinin= g about clarity to the issue of snapshots in qemu.  One last question = which we would be interested in is when is the back merge of snapshots for = running vms expected to be released.

Regards
sl

--- On = Fri, 11/3/11, Jes Sorensen <Jes.Sorensen@redhat.com> wrote= :

From: Jes Sorensen <Jes.Sorensen@redha= t.com>
Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 di= sk - qemu 0.14.0
To: "SAURAV LAHIRI" <saurav_lahiri@yahoo.com>
= Cc: "Stefan Hajnoczi" <stefanha@gmail.com>, qemu-devel@nongnu.org
= Date: Friday, 11 March, 2011, 5:50

On 03/11= /11 10:39, SAURAV LAHIRI wrote:
> Thank you Stefan, Jes. So it appear= s that snapshot_blkdev is the way
> to go for vm disk backup with running v= ms.
>
> In regard to merging changes, assuming that we go snap= shot_blkdev
> rightaway. Stefan's suggestion: "qemu-img commit" Jes's= suggestion:
> "qemu-img convert"
>
> Does qemu-img conv= ert apply to running VM's. In that case it would
> appear to be the m= ore practical approach(since vm shutdown would not
> be required).>
> Also incase If i have interpreted "qemu-img convert" incorre= ctly and
> does require a VM shutdown. Then when is expected time whe= n the "live
> merge" will be available.

I believe commit only = applies to images with internal files. If you use
convert then it doesn'= t modify the actual images, so lets say you have a
chain like this:
<= br>original->snapshotA->snapshotB

original and snapshotA are r= ead-only when snapshotB is running.
Therefore you should be able to use convert to simply copy snapshotA
into a new image file snapshotX= and save that for your backup. If you
later restore, you have a single = image file you can boot from.

What you cannot do is to create the ne= w snapshotX file and switch to it
as the backing file for snapshotB whil= e you are up and running.

Cheers,
Jes

<= /td>

=0A=0A=0A=0A --0-1243797253-1299863545=:78086-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=33900 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Py7lA-0004dp-0y for qemu-devel@nongnu.org; Fri, 11 Mar 2011 14:12:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Py7l8-0001O6-MD for qemu-devel@nongnu.org; Fri, 11 Mar 2011 14:12:23 -0500 Received: from web161603.mail.bf1.yahoo.com ([98.139.210.172]:24423) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1Py7l8-0001O0-HC for qemu-devel@nongnu.org; Fri, 11 Mar 2011 14:12:22 -0500 Message-ID: <720386.19420.qm@web161603.mail.bf1.yahoo.com> Date: Fri, 11 Mar 2011 11:12:21 -0800 (PST) From: SAURAV LAHIRI Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.14.0 In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-193469796-1299870741=:19420" List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: Jes Sorensen , qemu-devel@nongnu.org --0-193469796-1299870741=:19420 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Thank you Stefan that does clarify on how to correctly use "qemu-img commit= " Regards sl --- On Fri, 11/3/11, Stefan Hajnoczi wrote: From: Stefan Hajnoczi Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qemu 0.1= 4.0 To: "SAURAV LAHIRI" Cc: "Jes Sorensen" , qemu-devel@nongnu.org Date: Friday, 11 March, 2011, 7:55 On Fri, Mar 11, 2011 at 9:50 AM, Jes Sorensen wro= te: > On 03/11/11 10:39, SAURAV LAHIRI wrote: >> Thank you Stefan, Jes. So it appears that snapshot_blkdev is the way >> to go for vm disk backup with running vms. >> >> In regard to merging changes, assuming that we go snapshot_blkdev >> rightaway. Stefan's suggestion: "qemu-img commit" Jes's suggestion: >> "qemu-img convert" >> >> Does qemu-img convert apply to running VM's. In that case it would >> appear to be the more practical approach(since vm shutdown would not >> be required). >> >> Also incase If i have interpreted "qemu-img convert" incorrectly and >> does require a VM shutdown. Then when is expected time when the "live >> merge" will be available. > > I believe commit only applies to images with internal files. If you use Nope, qemu-img commit only applies to images with backing files (aka external snapshots). > convert then it doesn't modify the actual images, so lets say you have a > chain like this: > > original->snapshotA->snapshotB > > original and snapshotA are read-only when snapshotB is running. > Therefore you should be able to use convert to simply copy snapshotA > into a new image file snapshotX and save that for your backup. If you > later restore, you have a single image file you can boot from. > > What you cannot do is to create the new snapshotX file and switch to it > as the backing file for snapshotB while you are up and running. qemu-img commit merges the deltas from a qcow2 (or qed or other copy-on-write image format) into the backing file.=A0 If you have the following: qemu-img create -f qcow2 -o backing_file=3Dmaster.img,backing_fmt=3Draw instance.qcow2 Then the following command will merge the delta from instance.qcow2 back into master.img: qemu-img commit instance.qcow2 My suggestion was to shut down the VM and run qemu-img commit so that you can move back to the original disk image file which has now been updated with the deltas.=A0 You no longer need the file with the deltas. So qemu-img commit lets you merge down the chain of snapshots, whereas qemu-img convert lets you copy a full disk image into a new image file. To be clear, qemu-img should not be used on an image file that is currently open by a running VM. Stefan =0A=0A=0A --0-193469796-1299870741=:19420 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Thank you Stefan that does clarify on how to = correctly use "qemu-img commit "

Regards
sl

--- On Fri,= 11/3/11, Stefan Hajnoczi <stefanha@gmail.com> wrote:
<= blockquote style=3D"border-left: 2px solid rgb(16, 16, 255); margin-left: 5= px; padding-left: 5px;">
From: Stefan Hajnoczi <stefanha@gmail.com>= ;
Subject: Re: [Qemu-devel] Issue with snapshot outside qcow2 disk - qem= u 0.14.0
To: "SAURAV LAHIRI" <saurav_lahiri@yahoo.com>
Cc: "Jes= Sorensen" <Jes.Sorensen@redhat.com>, qemu-devel@nongnu.org
Date: = Friday, 11 March, 2011, 7:55

On Fri, Mar 11= , 2011 at 9:50 AM, Jes Sorensen <Jes.Sorensen@redha= t.com> wrote:
> On 03/11/11 10:39, SAURAV LAHIRI wrote:
>= ;> Thank you Stefan, Jes. So it appears that snapshot_blkdev is the way
>>= to go for vm disk backup with running vms.
>>
>> In rega= rd to merging changes, assuming that we go snapshot_blkdev
>> righ= taway. Stefan's suggestion: "qemu-img commit" Jes's suggestion:
>>= "qemu-img convert"
>>
>> Does qemu-img convert apply to = running VM's. In that case it would
>> appear to be the more pract= ical approach(since vm shutdown would not
>> be required).
>= >
>> Also incase If i have interpreted "qemu-img convert" incor= rectly and
>> does require a VM shutdown. Then when is expected ti= me when the "live
>> merge" will be available.
>
> I b= elieve commit only applies to images with internal files. If you use
Nope, qemu-img commit only applies to images with backing files (aka
ex= ternal snapshots).

> convert then it doesn't modify the actual images, so lets say you have a
> chain like this:
>
= > original->snapshotA->snapshotB
>
> original and snap= shotA are read-only when snapshotB is running.
> Therefore you should= be able to use convert to simply copy snapshotA
> into a new image f= ile snapshotX and save that for your backup. If you
> later restore, = you have a single image file you can boot from.
>
> What you ca= nnot do is to create the new snapshotX file and switch to it
> as the= backing file for snapshotB while you are up and running.

qemu-img c= ommit merges the deltas from a qcow2 (or qed or other
copy-on-write imag= e format) into the backing file.  If you have the
following:
qemu-img create -f qcow2 -o backing_file=3Dmaster.img,backing_fmt=3Drawinstance.qcow2

Then the following command will merge the delta from= instance.qcow2
back into master.img:

qemu-img commit instance.qcow2

My suggestion was to shut down the VM and run qemu-i= mg commit so that
you can move back to the original disk image file whic= h has now been
updated with the deltas.  You no longer need the fil= e with the deltas.

So qemu-img commit lets you merge down the chain = of snapshots, whereas
qemu-img convert lets you copy a full disk image i= nto a new image
file.

To be clear, qemu-img should not be used on= an image file that is
currently open by a running VM.

Stefan
=

=0A=0A=0A --0-193469796-1299870741=:19420--