Re: [Qemu-devel] [PATCH v9 00/14] rdma: migration support

[Chegu Vinod <chegu_vinod@hp.com>]

On 6/14/2013 1:35 PM, mrhines@linux.vnet.ibm.com wrote:
> From: "Michael R. Hines" <mrhines@us.ibm.com>
>
> Changes since v8:
>      For very large virtual machines, pinning can take a long time.
>      While this does not affect the migration's *actual* time itself,
>      it is still important for the user to know what's going on and
>      to know what component of the total time is actual taken up by
>      pinning.
>
>      For example, using a 14GB virtual machine, pinning can take as
>      long as 5 seconds, for which the user would not otherwise know
>      what was happening.
>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Chegu Vinod <chegu_vinod@hp.com>
Tested-by: Chegu Vinod <chegu_vinod@hp.com>


Thx
Vinod
>
> Wiki: http://wiki.qemu.org/Features/RDMALiveMigration
> Github: git@github.com:hinesmr/qemu.git
>
> Here is a brief summary of total migration time and downtime using RDMA:
>
> Using a 40gbps infiniband link performing a worst-case stress test,
> using an 8GB RAM virtual machine:
> Using the following command:
>
> $ apt-get install stress
> $ stress --vm-bytes 7500M --vm 1 --vm-keep
>
> RESULTS:
>
> 1. Migration throughput: 26 gigabits/second.
> 2. Downtime (stop time) varies between 15 and 100 milliseconds.
>
> EFFECTS of memory registration on bulk phase round:
>
> For example, in the same 8GB RAM example with all 8GB of memory in
> active use and the VM itself is completely idle using the same 40 gbps
> infiniband link:
>
> 1. x-rdma-pin-all disabled total time: approximately 7.5 seconds @ 9.5 Gbps
> 2. x-rdma-pin-all enabled total time: approximately 4 seconds @ 26 Gbps
>
> These numbers would of course scale up to whatever size virtual machine
> you have to migrate using RDMA.
>
> Enabling this feature does *not* have any measurable affect on
> migration *downtime*. This is because, without this feature, all of the
> memory will have already been registered already in advance during
> the bulk round and does not need to be re-registered during the successive
> iteration rounds.
>
> The following changes since commit f3aa844bbb2922a5b8393d17620eca7d7e921ab3:
>
>    build: include config-{, all-}devices.mak after defining CONFIG_SOFTMMU and CONFIG_USER_ONLY (2013-04-24 12:18:41 -0500)
>
> are available in the git repository at:
>
>    git@github.com:hinesmr/qemu.git rdma_patch_v9
>
> for you to fetch changes up to 75e6fac1f642885b93cefe6e1874d648e9850f8f:
>
>    rdma: send pc.ram (2013-04-24 14:55:01 -0400)
>
> ----------------------------------------------------------------
> Michael R. Hines (14):
>    rdma: add documentation
>    rdma: introduce qemu_update_position()
>    rdma: export yield_until_fd_readable()
>    rdma: export throughput w/ MigrationStats QMP
>    rdma: introduce qemu_file_mode_is_not_valid()
>    rdma: export qemu_fflush()
>    rdma: introduce ram_handle_compressed()
>    rdma: introduce qemu_ram_foreach_block()
>    rdma: new QEMUFileOps hooks
>    rdma: introduce capability x-rdma-pin-all
>    rdma: core logic
>    rdma: send pc.ram
>    rdma: fix mlock() freezes and accounting
>    rdma: add pin-all accounting timestamp to QMP statistics
>
>   Makefile.objs                 |    1 +
>   arch_init.c                   |   69 +-
>   configure                     |   29 +
>   docs/rdma.txt                 |  415 ++++++
>   exec.c                        |    9 +
>   hmp.c                         |    6 +
>   include/block/coroutine.h     |    6 +
>   include/exec/cpu-common.h     |    5 +
>   include/migration/migration.h |   32 +
>   include/migration/qemu-file.h |   32 +
>   migration-rdma.c              | 2831 +++++++++++++++++++++++++++++++++++++++++
>   migration.c                   |   36 +-
>   qapi-schema.json              |   15 +-
>   qemu-coroutine-io.c           |   23 +
>   savevm.c                      |  114 +-
>   15 files changed, 3574 insertions(+), 49 deletions(-)
>   create mode 100644 docs/rdma.txt
>   create mode 100644 migration-rdma.c
>