[Qemu-devel] Online image backup

[Qemu-devel] Online image backup

[Matteo Bertini]

Hello everyone,

is there any support for an online image backup? Or some direction to
have it?

Imagine I have an emulated server. How can I make an online backup of
the server state?

If I'm right in the actual design this is very hard.

Or there are some image internals update rules that permit a consistent
copy possible?

Regards,
Matteo Bertini

Re: [Qemu-devel] Online image backup

[Fabrice Bellard]

Matteo Bertini wrote:
> Hello everyone,
> 
> is there any support for an online image backup? Or some direction to
> have it?
> 
> Imagine I have an emulated server. How can I make an online backup of
> the server state?
> 
> If I'm right in the actual design this is very hard.
> 
> Or there are some image internals update rules that permit a consistent
> copy possible?

It should not be difficult provided the VM is using the QEMU qcow2 image
format (doing a backup can be seen as a kind of snapshot).

Fabrice.

[Qemu-devel] Re: Online image backup

[Matteo Bertini]

Il 21-11-2007 0:02, Fabrice Bellard ha scritto:
> Matteo Bertini wrote:
>> Hello everyone,
>>
>> is there any support for an online image backup? Or some direction to
>> have it?
>>
>> Imagine I have an emulated server. How can I make an online backup of
>> the server state?
>>
>> If I'm right in the actual design this is very hard.
>>
>> Or there are some image internals update rules that permit a consistent
>> copy possible?
> 
> It should not be difficult provided the VM is using the QEMU qcow2 image
> format (doing a backup can be seen as a kind of snapshot).
> 

I have tested that is quite hard to corrupt an image, even copying
during a savevm, at least the last completely saved one is loadvm-able.
And that is good, but I didn't really undestand what appends at
filesystem level...

Am I somewhat granted that after a savevm and a copy (online) the image
is loadvm-able and the filesystem is back to the last saved state?

Thank you,
Matteo Bertini

Re: [Qemu-devel] Online image backup

[Lorenzo Mancini]

> Matteo Bertini wrote:
>> Hello everyone,
>> 
>> is there any support for an online image backup? Or some direction to
>> have it?
>> 
>> Imagine I have an emulated server. How can I make an online backup of
>> the server state?
>> 
>> If I'm right in the actual design this is very hard.
>> 
>> Or there are some image internals update rules that permit a consistent
>> copy possible?
> 
> It should not be difficult provided the VM is using the QEMU qcow2 image
> format (doing a backup can be seen as a kind of snapshot).

Fabrice, that's just part of the problem.  A full automated remote 
backup service for qemu VMs should work like this:

  1. send a "savevm state" command to qemu;
  2. wait for savevm completion;
  3. perform an *online* copy (cp, rsync, whatever) of the .qcow2 image, 
while it's still running in qemu;
  4. ...repeat for all running VMs.

Now suppose you lose the host machine and must revert to backup.  A 
recovery scenario would be:

  1. copy the .qcow2 image on the new host machine;
  2. start qemu with the "-loadvm state" commandline switch.


The weak spot of this configuration is the third step of backup: if you 
perform an online copy (and that's your only real choice, since you 
don't want to powerdown the VMs every night just to backup them), you 
are copying a file while it's always modifying, and chances are high 
that the copy will be corrupted (i.e. different from any state the 
original went through), since you can't perform the copy with a single read.

However, if the .qcow2 file format is designed in such a way that even 
while copying it online, the resulting copy will certainly be corrupted 
but at least the states inside of it will be loadvm-able, that saves the 
day.


If that's not the case, one could fallback to perform the online copy 
between a couple of "stop" and "cont" commands, so you're sure that the 
underlying .qcow2 image is frozen while you're reading it (...is it?), 
but again that's suboptimal since you force the VMs to be down for some 
time.


Thanks in advance!

-- 
Lorenzo Mancini

Re: [Qemu-devel] Online image backup

[Ricardo Almeida]

Hi,

Discalmer: I'm just a user of QEmu :) <comments bellow>

> Fabrice, that's just part of the problem.  A full automated remote
> backup service for qemu VMs should work like this:
>
>   1. send a "savevm state" command to qemu;
>   2. wait for savevm completion;

2.1 Switch to snapshot mode

>   3. perform an *online* copy (cp, rsync, whatever) of the .qcow2 image,
> while it's still running in qemu;

3.1 Pause VM
3.2 Apply snapshot changes to local and remote copy (remote must be
running something that can apply the update)
3.3 Back to normal mode

>   4. ...repeat for all running VMs.
>

[...]

> The weak spot of this configuration is the third step of backup: if you
> perform an online copy (and that's your only real choice, since you
> don't want to powerdown the VMs every night just to backup them), you
> are copying a file while it's always modifying, and chances are high
> that the copy will be corrupted (i.e. different from any state the
> original went through), since you can't perform the copy with a single read.

See the above added steps :)
Is this a bad idea?

Regards and keep up the good work :),
Ricardo Almeida

Re: [Qemu-devel] Online image backup

[Lorenzo Mancini]

Ricardo Almeida wrote:

>> Fabrice, that's just part of the problem.  A full automated remote
>> backup service for qemu VMs should work like this:
>>
>>   1. send a "savevm state" command to qemu;
>>   2. wait for savevm completion;
> 
> 2.1 Switch to snapshot mode

How do you switch to snapshot mode *while* the VM is running, i.e. 
without stopping and restarting qemu?  The only way I know to enter that 
mode is using the -snapshot switch at invocation time and I don't think 
it can be done online.


>>   3. perform an *online* copy (cp, rsync, whatever) of the .qcow2 image,
>> while it's still running in qemu;
> 
> 3.1 Pause VM
> 3.2 Apply snapshot changes to local and remote copy (remote must be
> running something that can apply the update)
> 3.3 Back to normal mode

If executing step 2.1 is actually possible, you can skip step 3.1 and 
3.2 because the remote copy is already consistent with the state the VM 
went throught at the time of execution of step 2.1 .  If it was 
possible, it would be just fine for automated daily backups.

-- 
Lorenzo Mancini

Re: [Qemu-devel] Online image backup

[Ricardo Almeida]

> > 2.1 Switch to snapshot mode
>
> How do you switch to snapshot mode *while* the VM is running, i.e.
> without stopping and restarting qemu?  The only way I know to enter that
> mode is using the -snapshot switch at invocation time and I don't think
> it can be done online.

As I said, I'm just a user ;) I don't see a reason why this can't be
implemented... This is the devel list, so if a good idea comes up
someone can implement it ;)

> If executing step 2.1 is actually possible, you can skip step 3.1 and
> 3.2 because the remote copy is already consistent with the state the VM
> went throught at the time of execution of step 2.1 .  If it was
> possible, it would be just fine for automated daily backups.

Yes, you're right. I wanted to update a little more but it doesn't
really make much sense...