From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:42903) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UJ3FI-00062F-9f for qemu-devel@nongnu.org; Fri, 22 Mar 2013 10:47:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UJ3FG-0007e6-Fm for qemu-devel@nongnu.org; Fri, 22 Mar 2013 10:47:04 -0400 Received: from mx1.redhat.com ([209.132.183.28]:19418) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UJ3FG-0007dW-7r for qemu-devel@nongnu.org; Fri, 22 Mar 2013 10:47:02 -0400 From: Orit Wasserman Date: Fri, 22 Mar 2013 16:47:56 +0200 Message-Id: <1363963683-26157-1-git-send-email-owasserm@redhat.com> Subject: [Qemu-devel] [PATCH v5 0/7] Migration: Remove copying of guest ram pages List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: mst@redhat.com, quintela@redhat.com, Orit Wasserman , pbonzini@redhat.com, chegu_vinod@hp.com In migration all data is copied to a static buffer in QEMUFile, this hurts our network bandwidth and CPU usage especially with large guests. We switched to iovec for storing different buffers to send (even a byte field is considered as a buffer) and use sendmsg to send the iovec. Adjacent iovecs are coalesced to create a bigger buffer instead of many small buffers. Guest memory pages are not copied by calling a new function qemu_put_buffer_async. The page header data and device state data are still copied into the static buffer. This data consists of a lot of bytes and integer fields and the static buffer is used to store it during batching. git repository: git://github.com/oritwas/qemu.git sendv_v2 Changes from v4: return ssize_t for writev_buffer ops. Fix other Eric's comments. Squash patch 8 (coalesce adjacent iovecs) into patch 4. Changes from v3: Use a variable for iov_size Change f->bytes_xfer +=1 to f->bytes_xfer++ Remove unneeded "More optimized qemu_put_be64/32/16" patch Move code to the right patch Rename qemu_put_buffer_no_copy to qemu_put_buffer_async Use function for updating the iovec that detect adjacent iovecs and coalesce them. Change from v2: Always send data for the iovec even if writev_buffer is not implemented. Coalesce adjacent iovecs to create one big buffer from small adjacent buffer. Changes from v1: Use iov_send for socket. Make writev_buffer optional and if it is not implemented use put_buffer Future work: Make number of iovec changeable Orit Wasserman (7): Add QemuFileWritevBuffer QemuFileOps Add socket_writev_buffer function Update bytes_xfer in qemu_put_byte Store the data to send also in iovec Use writev ops if available Add qemu_put_buffer_async Use qemu_put_buffer_async for guest memory pages arch_init.c | 2 +- include/migration/qemu-file.h | 12 +++++ savevm.c | 103 +++++++++++++++++++++++++++++++++++------- 3 files changed, 100 insertions(+), 17 deletions(-) -- 1.7.11.7