Re: [RFC PATCH 00/20] Kemari for KVM v0.1

[Yoshiaki Tamura <tamura.yoshiaki@lab.ntt.co.jp>]

Anthony Liguori wrote:
> On 04/21/2010 12:57 AM, Yoshiaki Tamura wrote:
>> Hi all,
>>
>> We have been implementing the prototype of Kemari for KVM, and we're
>> sending
>> this message to share what we have now and TODO lists. Hopefully, we
>> would like
>> to get early feedback to keep us in the right direction. Although
>> advanced
>> approaches in the TODO lists are fascinating, we would like to run
>> this project
>> step by step while absorbing comments from the community. The current
>> code is
>> based on qemu-kvm.git 2b644fd0e737407133c88054ba498e772ce01f27.
>>
>> For those who are new to Kemari for KVM, please take a look at the
>> following RFC which we posted last year.
>>
>> http://www.mail-archive.com/kvm@vger.kernel.org/msg25022.html
>>
>> The transmission/transaction protocol, and most of the control logic is
>> implemented in QEMU. However, we needed a hack in KVM to prevent rip from
>> proceeding before synchronizing VMs. It may also need some plumbing in
>> the
>> kernel side to guarantee replayability of certain events and
>> instructions,
>> integrate the RAS capabilities of newer x86 hardware with the HA
>> stack, as well
>> as for optimization purposes, for example.
>>
>> Before going into details, we would like to show how Kemari looks. We
>> prepared
>> a demonstration video at the following location. For those who are not
>> interested in the code, please take a look.
>> The demonstration scenario is,
>>
>> 1. Play with a guest VM that has virtio-blk and virtio-net.
>> # The guest image should be a NFS/SAN.
>> 2. Start Kemari to synchronize the VM by running the following command
>> in QEMU.
>> Just add "-k" option to usual migrate command.
>> migrate -d -k tcp:192.168.0.20:4444
>> 3. Check the status by calling info migrate.
>> 4. Go back to the VM to play chess animation.
>> 5. Kill the the VM. (VNC client also disappears)
>> 6. Press "c" to continue the VM on the other host.
>> 7. Bring up the VNC client (Sorry, it pops outside of video capture.)
>> 8. Confirm that the chess animation ends, browser works fine, then
>> shutdown.
>>
>> http://www.osrg.net/kemari/download/kemari-kvm-fc11.mov
>>
>> The repository contains all patches we're sending with this message.
>> For those
>> who want to try, pull the following repository. At running configure,
>> please
>> put --enable-ft-mode. Also you need to apply a patch attached at the
>> end of
>> this message to your KVM.
>>
>> git://kemari.git.sourceforge.net/gitroot/kemari/kemari
>>
>> In addition to usual migrate environment and command, add "-k" to run.
>>
>> The patch set consists of following components.
>>
>> - bit-based dirty bitmap. (I have posted v4 for upstream QEMU on April
>> 2o)
>> - writev() support to QEMUFile and FdMigrationState.
>> - FT transaction sender/receiver
>> - event tap that triggers FT transaction.
>> - virtio-blk, virtio-net support for event tap.
>
> This series looks quite nice!

Thanks for your kind words!

> I think it would make sense to separate out the things that are actually
> optimizations (like the dirty bitmap changes and the writev/readv
> changes) and to attempt to justify them with actual performance data.

I agree with the separation plan.

For dirty bitmap change, Avi and I discussed on patchset for upsream QEMU while 
you were offline (Sorry, if I was wrong).  Could you also take a look?

http://lists.gnu.org/archive/html/qemu-devel/2010-04/msg01396.html

Regarding writev, I agree that it should be backed with actual data, otherwise 
it should be removed.  We attemped to do everything that may reduce the overhead 
of the transaction.

> I'd prefer not to modify the live migration protocol ABI and it doesn't
> seem to be necessary if we're willing to add options to the -incoming
> flag. We also want to be a bit more generic with respect to IO.

I totally agree with your approach not to change the protocol ABI.  Can we add 
an option to -incoming?  Like, -incoming ft_mode, for example
Regarding the IO, let me reply to the next message.

> Otherwise, the series looks very close to being mergable.

Thank you for your comment on each patch.

To be honest, I wasn't that confident because I'm a newbie to KVM/QEMU and 
struggled for how to implement in an acceptable way.

Thanks,

Yoshi

>
> Regards,
>
> Anthony Liguori
>
>
>