Re: [Qemu-devel] [PATCH v9 00/14] rdma: migration support

["Michael R. Hines" <mrhines@linux.vnet.ibm.com>]

On 06/15/2013 06:51 PM, Chegu Vinod wrote:
> On 6/14/2013 1:35 PM, mrhines@linux.vnet.ibm.com wrote:
>> From: "Michael R. Hines" <mrhines@us.ibm.com>
>>
>> Changes since v8:
>>      For very large virtual machines, pinning can take a long time.
>>      While this does not affect the migration's *actual* time itself,
>>      it is still important for the user to know what's going on and
>>      to know what component of the total time is actual taken up by
>>      pinning.
>>
>>      For example, using a 14GB virtual machine, pinning can take as
>>      long as 5 seconds, for which the user would not otherwise know
>>      what was happening.
>>
>> Reviewed-by: Eric Blake <eblake@redhat.com>
>> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
>
> Reviewed-by: Chegu Vinod <chegu_vinod@hp.com>
> Tested-by: Chegu Vinod <chegu_vinod@hp.com>
>

Thanks...

>
> Thx
> Vinod
>>
>> Wiki: http://wiki.qemu.org/Features/RDMALiveMigration
>> Github: git@github.com:hinesmr/qemu.git
>>
>> Here is a brief summary of total migration time and downtime using RDMA:
>>
>> Using a 40gbps infiniband link performing a worst-case stress test,
>> using an 8GB RAM virtual machine:
>> Using the following command:
>>
>> $ apt-get install stress
>> $ stress --vm-bytes 7500M --vm 1 --vm-keep
>>
>> RESULTS:
>>
>> 1. Migration throughput: 26 gigabits/second.
>> 2. Downtime (stop time) varies between 15 and 100 milliseconds.
>>
>> EFFECTS of memory registration on bulk phase round:
>>
>> For example, in the same 8GB RAM example with all 8GB of memory in
>> active use and the VM itself is completely idle using the same 40 gbps
>> infiniband link:
>>
>> 1. x-rdma-pin-all disabled total time: approximately 7.5 seconds @ 
>> 9.5 Gbps
>> 2. x-rdma-pin-all enabled total time: approximately 4 seconds @ 26 Gbps
>>
>> These numbers would of course scale up to whatever size virtual machine
>> you have to migrate using RDMA.
>>
>> Enabling this feature does *not* have any measurable affect on
>> migration *downtime*. This is because, without this feature, all of the
>> memory will have already been registered already in advance during
>> the bulk round and does not need to be re-registered during the 
>> successive
>> iteration rounds.
>>
>> The following changes since commit 
>> f3aa844bbb2922a5b8393d17620eca7d7e921ab3:
>>
>>    build: include config-{, all-}devices.mak after defining 
>> CONFIG_SOFTMMU and CONFIG_USER_ONLY (2013-04-24 12:18:41 -0500)
>>
>> are available in the git repository at:
>>
>>    git@github.com:hinesmr/qemu.git rdma_patch_v9
>>
>> for you to fetch changes up to 75e6fac1f642885b93cefe6e1874d648e9850f8f:
>>
>>    rdma: send pc.ram (2013-04-24 14:55:01 -0400)
>>
>> ----------------------------------------------------------------
>> Michael R. Hines (14):
>>    rdma: add documentation
>>    rdma: introduce qemu_update_position()
>>    rdma: export yield_until_fd_readable()
>>    rdma: export throughput w/ MigrationStats QMP
>>    rdma: introduce qemu_file_mode_is_not_valid()
>>    rdma: export qemu_fflush()
>>    rdma: introduce ram_handle_compressed()
>>    rdma: introduce qemu_ram_foreach_block()
>>    rdma: new QEMUFileOps hooks
>>    rdma: introduce capability x-rdma-pin-all
>>    rdma: core logic
>>    rdma: send pc.ram
>>    rdma: fix mlock() freezes and accounting
>>    rdma: add pin-all accounting timestamp to QMP statistics
>>
>>   Makefile.objs                 |    1 +
>>   arch_init.c                   |   69 +-
>>   configure                     |   29 +
>>   docs/rdma.txt                 |  415 ++++++
>>   exec.c                        |    9 +
>>   hmp.c                         |    6 +
>>   include/block/coroutine.h     |    6 +
>>   include/exec/cpu-common.h     |    5 +
>>   include/migration/migration.h |   32 +
>>   include/migration/qemu-file.h |   32 +
>>   migration-rdma.c              | 2831 
>> +++++++++++++++++++++++++++++++++++++++++
>>   migration.c                   |   36 +-
>>   qapi-schema.json              |   15 +-
>>   qemu-coroutine-io.c           |   23 +
>>   savevm.c                      |  114 +-
>>   15 files changed, 3574 insertions(+), 49 deletions(-)
>>   create mode 100644 docs/rdma.txt
>>   create mode 100644 migration-rdma.c
>>
>
>