From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:45700) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UIcMM-0007WD-OJ for qemu-devel@nongnu.org; Thu, 21 Mar 2013 06:04:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UIcML-0008Fh-8x for qemu-devel@nongnu.org; Thu, 21 Mar 2013 06:04:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:27920) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UIcMK-0008Fa-VR for qemu-devel@nongnu.org; Thu, 21 Mar 2013 06:04:33 -0400 Message-ID: <514ADB78.6010805@redhat.com> Date: Thu, 21 Mar 2013 12:05:44 +0200 From: Orit Wasserman MIME-Version: 1.0 References: <1363856971-4601-1-git-send-email-owasserm@redhat.com> <514AD2EC.40700@redhat.com> In-Reply-To: <514AD2EC.40700@redhat.com> Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC 00/12] Migration: Remove copying of guest ram pages List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: mst@redhat.com, chegu_vinod@hp.com, qemu-devel@nongnu.org, quintela@redhat.com On 03/21/2013 11:29 AM, Paolo Bonzini wrote: > Il 21/03/2013 10:09, Orit Wasserman ha scritto: >> In migration all data is copied to a static buffer in QEMUFile, >> this hurts our network bandwidth and CPU usage especially with large guests. >> We switched to iovec for storing different buffers to send (even a byte field is >> considered as a buffer) and use writev to send the iovec. >> writev was chosen (as apposed to sendmsg) because it supprts non socket fds. >> >> Guest memory pages are not copied by calling a new function >> qemu_put_buffer_no_copy. >> The page header data and device state data are still copied into the static >> buffer. This data consists of a lot of bytes and integer fields and the static >> buffer is used to store it during batching. >> Another improvement is changing qemu_putbe64/32/16 to create a single >> buffer instead of several byte sized buffer. > > Very nice! I just disagree on making writev_buffer mandatory; instead, > a QemuFileOps could choose between implementing either put_buffer or > writev_buffer. This removes the duplicate code you have between > iov_writev and block_writev_buffer. Sure, I can change the code that if put_buffer exists to revert to the previous implementation. Orit > > Thanks, > > Paolo > >> Orit Wasserman (12): >> Add iov_writev to use writev to send iovec (also for files) >> Add QemuFileWritevBuffer QemuFileOps >> Add socket_writev_buffer function >> Add stdio_writev_buffer function >> Add block_writev_buffer function >> Update bytes_xfer in qemu_put_byte >> Store the data to send also in iovec >> Use writev ops instead of put_buffer ops >> More optimized qemu_put_be64/32/16 >> Add qemu_put_buffer_no_copy >> Use qemu_put_buffer_no_copy for guest memory pages >> Bye Bye put_buffer >> >> arch_init.c | 2 +- >> include/migration/qemu-file.h | 20 ++++--- >> include/qemu/iov.h | 12 ++++ >> savevm.c | 130 +++++++++++++++++++++++++----------------- >> util/iov.c | 36 ++++++++++++ >> 5 files changed, 139 insertions(+), 61 deletions(-) >> >