From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:55422) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UUhMY-00031r-Ol for qemu-devel@nongnu.org; Tue, 23 Apr 2013 13:50:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UUhMT-0007e6-LK for qemu-devel@nongnu.org; Tue, 23 Apr 2013 13:50:42 -0400 Received: from e23smtp04.au.ibm.com ([202.81.31.146]:37899) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UUhMT-0007bn-3f for qemu-devel@nongnu.org; Tue, 23 Apr 2013 13:50:37 -0400 Received: from /spool/local by e23smtp04.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 24 Apr 2013 03:38:24 +1000 Received: from d23relay04.au.ibm.com (d23relay04.au.ibm.com [9.190.234.120]) by d23dlp03.au.ibm.com (Postfix) with ESMTP id 4E348357804A for ; Wed, 24 Apr 2013 03:50:11 +1000 (EST) Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay04.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r3NHaa9J48496844 for ; Wed, 24 Apr 2013 03:36:37 +1000 Received: from d23av03.au.ibm.com (loopback [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r3NHoAtC015714 for ; Wed, 24 Apr 2013 03:50:10 +1000 From: Anthony Liguori In-Reply-To: <1366682139-22122-1-git-send-email-mrhines@linux.vnet.ibm.com> References: <1366682139-22122-1-git-send-email-mrhines@linux.vnet.ibm.com> Date: Tue, 23 Apr 2013 12:50:03 -0500 Message-ID: <87li89dtmc.fsf@codemonkey.ws> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: [Qemu-devel] [PATCH v5 00/12] rdma: migration support List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: mrhines@linux.vnet.ibm.com, quintela@redhat.com Cc: qemu-devel@nongnu.org, owasserm@redhat.com, Bulent Abali , Michael R Hines , Gokul B Kandiraju , pbonzini@redhat.com mrhines@linux.vnet.ibm.com writes: > From: "Michael R. Hines" > > Juan, Please pull. I assume this is actually v6, not v5? I don't see collected Reviewed-bys... That said, we're pretty close to hard freeze. I think this should wait until 1.6 opens up although I'm open to suggestion if people think this is low risk. I don't like the idea of adding a new protocol this close to the end of a cycle. Regards, Anthony Liguori > > Changes since v4: > > - Re-ran checkpatch.pl > - Added new QEMUFileOps function: qemu_get_max_size() > - Renamed capability to x-pin-all, disabled by default > - Added numbers for x-pin-all to performance section in docs/rdma.txt > - Included performance numbers in this cover letter > - Converted throughput patch to a MigrationStats statistic in QMP > - Better QMP error message delivery > - Updated documentation > - Moved docs/rdma.txt up to top of patch series > - Fixed all v4 changes requested > - Finished additional cleanup requests > - Updated copyright for migration-rdma.c > > Wiki: http://wiki.qemu.org/Features/RDMALiveMigration > Github: git@github.com:hinesmr/qemu.git > > Here is a brief summary of total migration time and downtime using RDMA: > > Using a 40gbps infiniband link performing a worst-case stress test, > using an 8GB RAM virtual machine: > Using the following command: > > $ apt-get install stress > $ stress --vm-bytes 7500M --vm 1 --vm-keep > > RESULTS: > > 1. Migration throughput: 26 gigabits/second. > 2. Downtime (stop time) varies between 15 and 100 milliseconds. > > EFFECTS of memory registration on bulk phase round: > > For example, in the same 8GB RAM example with all 8GB of memory in > active use and the VM itself is completely idle using the same 40 gbps > infiniband link: > > 1. x-rdma-pin-all disabled total time: approximately 7.5 seconds @ 9.5 Gbps > 2. x-rdma-pin-all enabled total time: approximately 4 seconds @ 26 Gbps > > These numbers would of course scale up to whatever size virtual machine > you have to migrate using RDMA. > > Enabling this feature does *not* have any measurable affect on > migration *downtime*. This is because, without this feature, all of the > memory will have already been registered already in advance during > the bulk round and does not need to be re-registered during the successive > iteration rounds. > > Michael R. Hines (12): > rdma: add documentation > rdma: export yield_until_fd_readable() > rdma: export throughput w/ MigrationStats QMP > rdma: introduce qemu_get_max_size() > rdma: introduce qemu_file_mode_is_not_valid() > rdma: export qemu_fflush() > rdma: introduce ram_handle_compressed() > rdma: introduce qemu_ram_foreach_block() > rdma: new QEMUFileOps hooks > rdma: introduce capability x-rdma-pin-all > rdma: core logic > rdma: send pc.ram > > Makefile.objs | 1 + > arch_init.c | 59 +- > configure | 29 + > docs/rdma.txt | 404 ++++++ > exec.c | 9 + > hmp.c | 2 + > include/block/coroutine.h | 6 + > include/exec/cpu-common.h | 5 + > include/migration/migration.h | 24 + > include/migration/qemu-file.h | 44 + > migration-rdma.c | 2727 +++++++++++++++++++++++++++++++++++++++++ > migration.c | 22 +- > qapi-schema.json | 12 +- > qemu-coroutine-io.c | 23 + > savevm.c | 133 +- > 15 files changed, 3451 insertions(+), 49 deletions(-) > create mode 100644 docs/rdma.txt > create mode 100644 migration-rdma.c > > -- > 1.7.10.4