Re: [Qemu-devel] Microcheckpointing: Memory-VCPU / Disk State consistency

[Walid Nouri <walid.nouri@gmail.com>]

Hello Michael, Hello Paolo
i have „studied“ the available documentation/Information and tried to 
get an idea of the QEMU live block operation possibilities.

I think the MC protocol doesn’t need synchronous block device 
replication because primary and secondary VM are not synchronous. The 
state of the primary is allays ahead of the state of the secondary. When 
the primary is in epoch(n) the secondary is in epoch(n-1).

What MC needs is a block device agnostic, controlled and asynchronous 
approach for replicating the contents of block devices and its state 
changes to the secondary VM while the primary VM is running. 
Asynchronous block transfer is important to allow maximum performance 
for the primary VM, while keeping the secondary VM updated with state 
changes.

The block device replication should be possible in two stages or modes.

The first stage is the live copy of all block devices of the primary to 
the secondary. This is necessary if the secondary doesn’t have an 
existing image which is in sync with the primary at the time MC has 
started. This is not very convenient but as far as I know actually there 
is no mechanism for persistent dirty bitmap in QEMU.

The second stage (mode) is the replication of block device state changes 
(modified blocks)  to keep the image on the secondary in sync with the 
primary. The mirrored blocks must be buffered in ram (block buffer) 
until the complete Checkpoint (RAM, vCPU, device state) can be committed.

For keeping the complete system state consistent on the secondary system 
there must be a possibility for MC to commit/discard block device state 
changes. In normal operation the mirrored block device state changes 
(block buffer) are committed to disk when the complete checkpoint is 
committed. In case of a crash of the primary system while transferring a 
checkpoint the data in the block buffer corresponding to the failed 
Checkpoint must be discarded.

The storage architecture should be “shared nothing” so that no shared 
storage is required and primary/secondary can have separate block device 
images.

I think this can be achieved by drive-mirror and a filter block driver. 
Another approach could be to exploit the block migration functionality 
of live migration with a filter block driver.

The drive-mirror (and live migration) does not rely on shared storage 
and allow live block device copy and incremental syncing.

A block buffer can be implemented with a QEMU filter block driver. It 
should sit at the same position as the Quorum driver in the block driver 
hierarchy. When using block filter approach MC will be transparent and 
block device agnostic.

The block buffer filter must have an Interface which allows MC control 
the commits or discards of block device state changes. I have no idea 
where to put such an interface to stay conform with QEMU coding style.


I’m sure there are alternative and better approaches and I’m open for 
any ideas


Walid

Am 17.08.2014 11:52, schrieb Paolo Bonzini:
> Il 11/08/2014 22:15, Michael R. Hines ha scritto:
>> Excellent question: QEMU does have a feature called "drive-mirror"
>> in block/mirror.c that was introduced a couple of years ago. I'm not
>> sure what the
>> adoption rate of the feature is, but I would start with that one.
>
> block/mirror.c is asynchronous, and there's no support for communicating
> checkpoints back to the master.  However, the quorum disk driver could
> be what you need.
>
> There's also a series on the mailing list that lets quorum read only
> from the primary, so that quorum can still do replication and fault
> tolerance, but skip fault detection.
>
> Paolo
>
>> There is also a second fault tolerance implementation that works a
>> little differently called
>> "COLO" - you may have seen those emails on the list too, but their
>> method does not require a disk replication solution, if I recall correctly.
>