RBD Backup

RBD Backup

[Stefan Priebe - Profihost AG]

Hello list,

is there a recommanded way to backup rbd images / disks?

Or is it just
rbd snap create BACKUP
rbd export BACKUP
rbd snap rm BACKUP

Is the snap needed at all? Or is an export save? Is there a way to make 
sure the image is consistent?

Is it possible to use the BACKUP file as a loop device or something else 
so that i'm able to mount the partitions from the backup file?

Thanks!

Greets Stefan

Re: RBD Backup

[Wido den Hollander]

Hi,

On 11/21/2012 09:37 PM, Stefan Priebe - Profihost AG wrote:
> Hello list,
>
> is there a recommanded way to backup rbd images / disks?
>
> Or is it just
> rbd snap create BACKUP
> rbd export BACKUP

You should use:

rbd export --snap BACKUP <img> <dest>

> rbd snap rm BACKUP
>
> Is the snap needed at all? Or is an export save? Is there a way to make
> sure the image is consistent?
>

While reading rbd.cc it doesn't seem like running export on a running VM 
is safe, so you should snapshot before.

The snapshot isn't consistent since it has no way of telling the VM to 
flush it's buffers.

To make it consistent you have to run "sync" (In the VM) just prior to 
creating the snapshot.

> Is it possible to use the BACKUP file as a loop device or something else
> so that i'm able to mount the partitions from the backup file?
>

You can do something like:

rbd export --snap BACKUP image1 /mnt/backup/image1.img
losetup /mnt/backup/image1.img
kpartx -a /dev/loop0

Now you will have the partitions from the RBD image available in 
/dev/mapper/loop0pX

Wido

> Thanks!
>
> Greets Stefan
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RBD Backup

[Stefan Priebe - Profihost AG]

Hi Wido,

thanks for all your explanations.

This doesn't seem to work:
> rbd export --snap BACKUP <img> <dest>

rbd -p kvmpool1 export --snap BACKUP vm-101-disk-1 /vm-101-disk-1.img 

rbd: error setting snapshot context: (2) No such file or directory

Or should i still create and delete a snapshot named BACKUP before doing 
this?

Greets,
Stefan

Re: RBD Backup

[Wido den Hollander]

Hi,

On 11/21/2012 09:56 PM, Stefan Priebe - Profihost AG wrote:
> Hi Wido,
>
> thanks for all your explanations.
>
> This doesn't seem to work:
>> rbd export --snap BACKUP <img> <dest>
>
> rbd -p kvmpool1 export --snap BACKUP vm-101-disk-1 /vm-101-disk-1.img
> rbd: error setting snapshot context: (2) No such file or directory
>
> Or should i still create and delete a snapshot named BACKUP before doing
> this?
>

Yes, you should create the snapshot first before exporting it. Export 
does not create the snapshot for you.

Wido

> Greets,
> Stefan
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RBD Backup

[Stefan Priebe - Profihost AG]

Hi,

Am 21.11.2012 14:47, schrieb Wido den Hollander:
> The snapshot isn't consistent since it has no way of telling the VM to
> flush it's buffers.
>
> To make it consistent you have to run "sync" (In the VM) just prior to
> creating the snapshot.

Mhm but between executing sync and executing snap is again time to store 
data.

> rbd export --snap BACKUP image1 /mnt/backup/image1.img
> losetup /mnt/backup/image1.img
>
> kpartx -a /dev/loop0
>
> Now you will have the partitions from the RBD image available in
> /dev/mapper/loop0pX
Works fine!

Greets,
Stefan

Re: RBD Backup

[Wido den Hollander]

On 11/22/2012 06:57 PM, Stefan Priebe - Profihost AG wrote:
> Hi,
>
> Am 21.11.2012 14:47, schrieb Wido den Hollander:
>> The snapshot isn't consistent since it has no way of telling the VM to
>> flush it's buffers.
>>
>> To make it consistent you have to run "sync" (In the VM) just prior to
>> creating the snapshot.
>
> Mhm but between executing sync and executing snap is again time to store
> data.
>

True. That is always a problem with snapshots. I always regard data 
written to disk in the last 30 seconds as being in the "danger zone".

When you use libvirt and QCOW2 as a backing store for your virtual 
machine you can also snapshot with libvirt. It will not only snapshot 
the disk, but it will also store the memory contents from the virtual 
machine so you have a consistent state of the virtual machine.

This has a drawback however, since when you give the VM 16GB of memory, 
you have to store 16GB of data.

Right now this doesn't work yet with RBD, but there is a feature request 
in the tracker. I can't seem to find it right now.

What you could do is:

$ ssh root@virtual-machine "sync"
$ rbd snap create vm-disk@snap1
$ rbd export --snap snap1 vm-disk /mnt/backup/vm-disk_snap1.img

This way you have a pretty consistent snapshot.

Wido

>> rbd export --snap BACKUP image1 /mnt/backup/image1.img
>> losetup /mnt/backup/image1.img
>>
>> kpartx -a /dev/loop0
>>
>> Now you will have the partitions from the RBD image available in
>> /dev/mapper/loop0pX
> Works fine!
>
> Greets,
> Stefan

Re: RBD Backup

[Josh Durgin]

On 11/22/2012 05:13 AM, Wido den Hollander wrote:
>
>
> On 11/22/2012 06:57 PM, Stefan Priebe - Profihost AG wrote:
>> Hi,
>>
>> Am 21.11.2012 14:47, schrieb Wido den Hollander:
>>> The snapshot isn't consistent since it has no way of telling the VM to
>>> flush it's buffers.
>>>
>>> To make it consistent you have to run "sync" (In the VM) just prior to
>>> creating the snapshot.
>>
>> Mhm but between executing sync and executing snap is again time to store
>> data.
>>
>
> True. That is always a problem with snapshots. I always regard data
> written to disk in the last 30 seconds as being in the "danger zone".
>
> When you use libvirt and QCOW2 as a backing store for your virtual
> machine you can also snapshot with libvirt. It will not only snapshot
> the disk, but it will also store the memory contents from the virtual
> machine so you have a consistent state of the virtual machine.
>
> This has a drawback however, since when you give the VM 16GB of memory,
> you have to store 16GB of data.
>
> Right now this doesn't work yet with RBD, but there is a feature request
> in the tracker. I can't seem to find it right now.
>
> What you could do is:
>
> $ ssh root@virtual-machine "sync"
> $ rbd snap create vm-disk@snap1
> $ rbd export --snap snap1 vm-disk /mnt/backup/vm-disk_snap1.img
>
> This way you have a pretty consistent snapshot.

You can get an entirely consistent snapshot using xfs_freeze to
stop I/O to the fs until you thaw it. It's done at the vfs level
these days, so it works on all filesystems.

Josh

>>> rbd export --snap BACKUP image1 /mnt/backup/image1.img
>>> losetup /mnt/backup/image1.img
>>>
>>> kpartx -a /dev/loop0
>>>
>>> Now you will have the partitions from the RBD image available in
>>> /dev/mapper/loop0pX
>> Works fine!
>>
>> Greets,
>> Stefan

Re: RBD Backup

[Stefan Priebe - Profihost AG]

Hi Josh,

Am 22.11.2012 22:08, schrieb Josh Durgin:

>> This way you have a pretty consistent snapshot.
>
> You can get an entirely consistent snapshot using xfs_freeze to
> stop I/O to the fs until you thaw it. It's done at the vfs level
> these days, so it works on all filesystems.

Great thing we even use XFS ;-) but when i do
xfs_freeze -f /

it just hangs and i can't do anything until i reset the whole VM.

Stefan