KVM backup and snapshots

KVM backup and snapshots

[Dietmar Maurer]

What is the suggested way to backup a running kvm instance which uses
several disk images? Currently I simply use a LVM2 snapshot if all disk
images resides on one lvm volume. But what if it uses several lvm
volumes? 

Or is it possible to make a consistent backup without lvm?

- Dietmar 

Re: KVM backup and snapshots

[Javier Guerra]

On Thu, Aug 7, 2008 at 9:23 AM, Dietmar Maurer <dietmar@proxmox.com> wrote:
>
> What is the suggested way to backup a running kvm instance which uses
> several disk images? Currently I simply use a LVM2 snapshot if all disk
> images resides on one lvm volume. But what if it uses several lvm
> volumes?

i'd try to suspend KVM, do all LVM snapshots, unsuspend KVM.
hopefully it would only mean a few seconds of dead time.  can several
LVM snapshots be created in parallel? lots of testing ahead...


-- 
Javier

AW: KVM backup and snapshots

[Dietmar Maurer]

> > What is the suggested way to backup a running kvm instance 
> which uses 
> > several disk images? Currently I simply use a LVM2 snapshot if all 
> > disk images resides on one lvm volume. But what if it uses 
> several lvm 
> > volumes?
> 
> i'd try to suspend KVM, do all LVM snapshots, unsuspend KVM.
> hopefully it would only mean a few seconds of dead time.  can 

I guess that would work, but I want to avoid any downtime. Also
Making more that one LVM snapshot is clumsy.

> several LVM snapshots be created in parallel? lots of testing ahead...

I thought about using 1 lvm volume, but splitting that into slices
somehow, which can then be used as kvm disks - maybe by implementing a
very simple filesystem (block mapper). The problem with this approach is
that adding/deleting a new disk would mean to grow/shrink an lvm
partition, which is slow. 

What do you think? What would be the best way to implement this?

- Dietmar

Re: KVM backup and snapshots

[Javier Guerra]

On Thu, Aug 7, 2008 at 9:51 AM, Dietmar Maurer <dietmar@proxmox.com> wrote:
> I thought about using 1 lvm volume, but splitting that into slices
> somehow, which can then be used as kvm disks - maybe by implementing a
> very simple filesystem (block mapper). The problem with this approach is
> that adding/deleting a new disk would mean to grow/shrink an lvm
> partition, which is slow.

you could run LVM in the VM.  be careful about block scanning tools on
Dom0, could mistake the LVM structure inside a LV for the 'outer' one.
 (reiserfsck has this problem with image files)


-- 
Javier

AW: KVM backup and snapshots

[Dietmar Maurer]

> you could run LVM in the VM.  be careful about block scanning 
> tools on Dom0, could mistake the LVM structure inside a LV 
> for the 'outer' one.
>  (reiserfsck has this problem with image files)

No, thats no option, because I cant depend on the guest (i.e lvm2 is not
available for windows)

- Dietmar

Re: KVM backup and snapshots

[Soren Hansen]

[-- Attachment #1: Type: text/plain, Size: 1025 bytes --]

On Thu, Aug 07, 2008 at 09:57:17AM -0500, Javier Guerra wrote:
>> I thought about using 1 lvm volume, but splitting that into slices
>> somehow, which can then be used as kvm disks - maybe by implementing
>> a very simple filesystem (block mapper). The problem with this
>> approach is that adding/deleting a new disk would mean to grow/shrink
>> an lvm partition, which is slow.
> you could run LVM in the VM.  be careful about block scanning tools on
> Dom0, could mistake the LVM structure inside a LV for the 'outer' one.
> (reiserfsck has this problem with image files)

Yes, I noticed that this is not common knowledge. Besides, it's
something that can't hurt being repeated every so often:

 * Don't ever, ever store images of reiser file systems on reiserfs!! *

Theodore T'so explained it rather well:

   http://linuxmafia.com/faq/Filesystems/reiserfs.html

-- 
Soren Hansen               | 
Virtualisation specialist  | Ubuntu Server Team
Canonical Ltd.             | http://www.ubuntu.com/

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 315 bytes --]

AW: KVM backup and snapshots

[Dietmar Maurer]

Or maybe it possible to modify kvm to store all changes made to a vm
(opposite off -snapshot option)? 

> you could run LVM in the VM.  be careful about block scanning 
> tools on Dom0, could mistake the LVM structure inside a LV 
> for the 'outer' one.
>  (reiserfsck has this problem with image files)

Re: KVM backup and snapshots

[Daniel P. Berrange]

On Thu, Aug 07, 2008 at 09:57:17AM -0500, Javier Guerra wrote:
> On Thu, Aug 7, 2008 at 9:51 AM, Dietmar Maurer <dietmar@proxmox.com> wrote:
> > I thought about using 1 lvm volume, but splitting that into slices
> > somehow, which can then be used as kvm disks - maybe by implementing a
> > very simple filesystem (block mapper). The problem with this approach is
> > that adding/deleting a new disk would mean to grow/shrink an lvm
> > partition, which is slow.
> 
> you could run LVM in the VM.  be careful about block scanning tools on
> Dom0, could mistake the LVM structure inside a LV for the 'outer' one.
>  (reiserfsck has this problem with image files)

This is an example of why you should use nested partition tables and not
export a whole block device to the guest - always partition the block
device on your host into at least one partition, and expose this as the
virtual disk to the guest.  The guest then partitions this and puts the
partition in as the physical volume. No dom0 tool will accidentally
see the guest LVM volumes  now.

In the few occassions when you do need to, you can access this nested
partition table in the dom0 using kpartx.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

AW: KVM backup and snapshots

[Dietmar Maurer]

> like a 'savevm' without qcow2 requirements?
> 
> would be great.

Yes, something like:

# savediff [filname] [maxsize]

After a backup you can revert all changes, so you basically have a
snapshot.

- Dietmar

AW: KVM backup and snapshots

[Dietmar Maurer]

Maybe this would also simplify/fasten online migration without shared
storage, because you know what blocks changed.

> On Thu, Aug 7, 2008 at 10:29 AM, Dietmar Maurer 
> <dietmar@proxmox.com> wrote:
> > Or maybe it possible to modify kvm to store all changes 
> made to a vm 
> > (opposite off -snapshot option)?
> 
> like a 'savevm' without qcow2 requirements?

Re: AW: KVM backup and snapshots

[Anthony Liguori]

Dietmar Maurer wrote:
>> like a 'savevm' without qcow2 requirements?
>>
>> would be great.
>>     
>
> Yes, something like:
>
> # savediff [filname] [maxsize]
>
> After a backup you can revert all changes, so you basically have a
> snapshot.
>   

Actually, I think a simple change to the 'change' monitor command would 
suffice.  If it were changed to allow non-ejectable block devices to be 
changed atomically, perhaps with a force flag, you could do the following:

qemu -hda foo.img

# create temporary qcow2 file that backs to foo.img
qemu-img create -f qcow2 -b foo.img tmp-hda.img

# in the monitor
(qemu) change -f ide0-hd tmp-hda.img
# do whatever you need with foo.img
(qemu) stop
(qemu) commit ide0-hd  # sync tmp-hda.img with foo.img
(qemu) change -f ide0-hd foo.img
(qemu) continue

obviously, if you script it, you'll minimize downtime.  We could get 
smarter and support a "live" commit that allowed a guest to keep running 
too which would make the downtime near-zero.

N.B. your base image can be anything, including an LVM partition.

Regards,

Anthony Liguori

> - Dietmar
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>