Re: Livebackup feature for qemu/qemu-kvm

[Jagane Sundar <jagane@sundar.org>]

On 5/2/2011 7:36 AM, Jes Sorensen wrote:
> On 05/02/11 00:23, Jagane Sundar wrote:
>> Having a live backup feature for kvm will make it a better solution for
>> IaaS clouds compared to xen. I would like to solicit feedback from all
>> of you folks involved in the block subsystem of qemu. Stefan mentioned
>> that Jes is the person most intimately involved in the block subsystem,
>> so Jes - your feedback is particularly important.
> Jagane,
>
> Reading your page, the first thing I stumble upon under 'Use Cases' is
> the reference to EBS storage. What is that?
EBS stands for Elastic Block Storage - Amazon EC2's shared storage 
solution. This is the storage
that comes with guarantees, since it is replicated across machines.
> Under details, I think it is not a good idea to rely on QEMU looking for
> any files with specific file name suffixes. It really should be
> specified on the command line by the user or admin tool.
That's a good idea. Perhaps another attribute in the drive description list,
just like type=virtio, maybe backup=livebackup.
> Other questions, how do you plan to handle crashes or loss of network
> connection in the middle of a livebackup? How about handling corrupted
> livebackup files?
Crashes of various software:
1. qemu crashes during normal operation of the VM:
     When this happens, the livebackup_client is forced to do a full 
backup the next
     time around. Here's how: livebackup writes out the in-memory dirty 
bitmap
     to a dirty bitmap file only at the time of orderly shutdown of 
qemu. Hence,
     the mtime of the virtual disk file is later than the mtime  of the 
livebackup
     dirty bitmap file. This causes livebackup to consider the dirty 
bitmap invalid,
     and forces the livebackup_client to do a full backup next time around.
2. qemu crashes while livebackup is in progress:
     In this case also, the livebackup_client is forced to do a full 
backup the next
     time around. The dirty bitmap file, the COW file used to store 
blocks written
     while a livebackup is in progress, are all deleted, and the 
livebackup client
    is forced to do a full backup next time around.
3. livebackup_client crashes while livebackup is in progress:
     In this case, a new livebackup_client may be started, and it can 
redo the last
     type of backup it was doing - an incremental backup or a full 
backup. Note
     all the blocks of the last backup type need to be transferred over 
again, the
     qemu livebackup code does not keep track of what block the client 
was at.
     It does not need to be a forced full backup.
> In general it looks interesting, you could consider submitting a
> presentation about Livebackup to the KVM Forum 2011.

Glad to know that you think it is interesting. Also, thanks for the 
pointer to
KVM Forum 2011, Jes. It looks like I have a few more weeks to get an 
abstract
in for KVM forum 2011. I will do so.

Jagane