Re: [Qemu-devel] feature proposal: checkpoint-assisted migration

["Dr. David Alan Gilbert" <dgilbert@redhat.com>]

* Thomas Knauth (thomas.knauth@googlemail.com) wrote:
> Dear list,
> 
> my research revolves around cloud computing, virtual machines and
> migration. In this context I came across the following: a recent study
> by IBM indicates that a typical VM only migrates between a small set
> of physical servers; often just two.
> 
> The potential for optimization is clear. By storing a snapshot of the
> VM's memory on the migration source, we can reuse (some) of this
> information on a subsequent incoming migration.
> 
> In the course of our research we implemented a prototype of this
> feature within kvm/qemu. We would like to contribute it to mainline,
> but it needs cleanup and proper testing. As is the nature with
> research prototypes, the code is ugly and not well integrated with the
> existing kvm/qemu codebase. To avoid confusion and irritation, I want
> to mention that I have little experience in contributing to large
> open-source projects. So if I violate some unwritten protocol or best
> practises, please be patient.
> 
> Initially, I'm hoping to get some feedback on the current state of the
> implementation. It would be immensely helpful if someone more
> intimately familiar with the migration code/framework could comment on
> the prototyp's current state. The code very likely needs restructuring
> to make it fit better into the overall codebase. Getting information
> on what needs to change and how to change it would be my goal.
> 
> The prototype also touches the migration protocol. Changes in this
> part probably need discussion. The basic idea is that if a block of
> memory (e.g., a 4 KiB page) already exists at the migration
> destination, than the source only sends a checksum of the block
> (currently MD5). The destination uses the checksum to find the
> corresponding block, e.g., by reading it from local storage (instead
> of transferring it over the network). This definitely reduces the
> migration traffic and usually also the overall migration time.
> 
> We currently use MD5 checksums to identify (un)modified blocks. For
> strict ping-pong migration, where a VM only migrates between two
> servers, there is also the possibility to use dirty page tracking to
> identify modified pages. This has not been implemented so far. We are
> also unclear about the potential performance tradeoffs this might
> entail and how it would interact with the dirty page tracking code
> during a live migration.

I like your basic idea, and I kind of agreed with your argument that
if it's good enough for rsync then it's good enough; however, then I
found:
  https://github.com/therealmik/rsync-collision

which complicates the argument!  Those are 700byte blocks, so I guess
the chance of a collision on a 4kB page must be less likely; but I'd
want a crypto guy to say what was actually safeish.

> Our research also includes a look at real world data to motivate that
> this optimization actually does make sense in practise. If you are
> interested, you can find a draft of the relevant paper at:
> 
> https://www.dropbox.com/s/v7qzim8exmji6j5/paper.pdf?dl=0
> 
> Keep in mind that the paper is not published yet and, hence, work in progress.
> 
> As you can see, there are many open/unanswered questions, but I'm
> hopeful that this feature will eventually become part of kvm/qemu such
> that everyone can benefit from it.
> 
> Please find the current code at
> https://bitbucket.org/tknauth/vecycle-qemu/branch/checkpoint-assisted-migration

That asks me to login just to read it; which is a very odd thing
to have.    I suggest you post your code to the list, with basically
this message at the start of the series, saying it's very new and you
expect it needs lots of changes, that way people can more easily
look at it.

Dave

> 
> Looking forward to your feedback,
> Thomas.
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK