From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:44704) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UHQbn-0005ys-7L for qemu-devel@nongnu.org; Sun, 17 Mar 2013 23:19:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UHQbh-0004Zn-7z for qemu-devel@nongnu.org; Sun, 17 Mar 2013 23:19:35 -0400 Received: from e9.ny.us.ibm.com ([32.97.182.139]:57467) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UHQbh-0004ZG-3T for qemu-devel@nongnu.org; Sun, 17 Mar 2013 23:19:29 -0400 Received: from /spool/local by e9.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Sun, 17 Mar 2013 23:19:28 -0400 Received: from d01relay06.pok.ibm.com (d01relay06.pok.ibm.com [9.56.227.116]) by d01dlp01.pok.ibm.com (Postfix) with ESMTP id D25CD38C804D for ; Sun, 17 Mar 2013 23:19:25 -0400 (EDT) Received: from d01av04.pok.ibm.com (d01av04.pok.ibm.com [9.56.224.64]) by d01relay06.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r2I3JPd129819004 for ; Sun, 17 Mar 2013 23:19:25 -0400 Received: from d01av04.pok.ibm.com (loopback [127.0.0.1]) by d01av04.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r2I3JPNY009348 for ; Sun, 17 Mar 2013 23:19:25 -0400 From: mrhines@linux.vnet.ibm.com Date: Sun, 17 Mar 2013 23:19:03 -0400 Message-Id: <1363576743-6146-11-git-send-email-mrhines@linux.vnet.ibm.com> In-Reply-To: <1363576743-6146-1-git-send-email-mrhines@linux.vnet.ibm.com> References: <1363576743-6146-1-git-send-email-mrhines@linux.vnet.ibm.com> Subject: [Qemu-devel] [RFC PATCH RDMA support v4: 10/10] send pc.ram over RDMA List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: aliguori@us.ibm.com, mst@redhat.com, owasserm@redhat.com, abali@us.ibm.com, mrhines@us.ibm.com, gokul@us.ibm.com, pbonzini@redhat.com From: "Michael R. Hines" Signed-off-by: Michael R. Hines --- arch_init.c | 28 +++++++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/arch_init.c b/arch_init.c index 98e2bc6..b013cc8 100644 --- a/arch_init.c +++ b/arch_init.c @@ -45,6 +45,7 @@ #include "exec/address-spaces.h" #include "hw/pcspk.h" #include "migration/page_cache.h" +#include "migration/rdma.h" #include "qemu/config-file.h" #include "qmp-commands.h" #include "trace.h" @@ -225,6 +226,18 @@ static void acct_clear(void) memset(&acct_info, 0, sizeof(acct_info)); } +/* + * RDMA pc.ram doesn't go through QEMUFile directly, + * but still needs to be accounted for... + */ +uint64_t delta_norm_mig_bytes_transferred(void) +{ + static uint64_t last_norm_pages = 0; + uint64_t delta_bytes = (acct_info.norm_pages - last_norm_pages) * TARGET_PAGE_SIZE; + last_norm_pages = acct_info.norm_pages; + return delta_bytes; +} + uint64_t dup_mig_bytes_transferred(void) { return acct_info.dup_pages * TARGET_PAGE_SIZE; @@ -463,7 +476,11 @@ static int ram_save_block(QEMUFile *f, bool last_stage) /* In doubt sent page as normal */ bytes_sent = -1; - if (is_dup_page(p)) { + if (migrate_use_rdma(f)) { + /* for now, mapping the page is slower than RDMA */ + acct_info.norm_pages++; + bytes_sent = save_rdma_page(f, block->offset, offset, cont, TARGET_PAGE_SIZE); + } else if (is_dup_page(p)) { acct_info.dup_pages++; bytes_sent = save_block_hdr(f, block, offset, cont, RAM_SAVE_FLAG_COMPRESS); @@ -648,6 +665,15 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) qemu_mutex_unlock_ramlist(); + /* + * Don't go to the next iteration without + * ensuring RDMA transfers have completed. + */ + if ((ret = qemu_drain(f)) < 0) { + fprintf(stderr, "failed to drain RDMA first!\n"); + return ret; + } + if (ret < 0) { bytes_transferred += total_sent; return ret; -- 1.7.10.4