From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:34895) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QqR5p-0005IX-82 for qemu-devel@nongnu.org; Mon, 08 Aug 2011 10:46:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QqR5n-0004SO-RG for qemu-devel@nongnu.org; Mon, 08 Aug 2011 10:46:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:26028) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QqR5n-0004SG-FJ for qemu-devel@nongnu.org; Mon, 08 Aug 2011 10:46:11 -0400 Message-ID: <4E3FF6AE.8030004@redhat.com> Date: Mon, 08 Aug 2011 17:46:06 +0300 From: Avi Kivity MIME-Version: 1.0 References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v4] XBZRLE delta for live migration of large memory apps List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Shribman, Aidan" Cc: Blue Swirl , Stefan Hajnoczi , qemu-devel Developers On 08/08/2011 11:42 AM, Shribman, Aidan wrote: > Subject: [PATCH v4] XBZRLE delta for live migration of large memory apps > From: Aidan Shribman > > By using XBZRLE (Xor Binary Zero Run-Length-Encoding) we can reduce VM downtime > and total live-migration time of VMs running memory write intensive workloads > typical of large enterprise applications such as SAP ERP Systems, and generally > speaking for any application with a sparse memory update pattern. > > On the sender side XBZRLE is used as a compact delta encoding of page updates, > retrieving the old page content from an LRU cache (default size of 64 MB). The > receiving side uses the existing page content and XBZRLE to decode the new page > content. > > Work was originally based on research results published VEE 2011: Evaluation of > Delta Compression Techniques for Efficient Live Migration of Large Virtual > Machines by Benoit, Svard, Tordsson and Elmroth. Additionally the delta encoder > XBRLE was improved further using XBZRLE instead. > > XBZRLE has a sustained bandwidth of 2-2.5 GB/s for typical workloads making it > ideal for in-line, real-time encoding such as is needed for live-migration. > > A typical usage scenario: > {qemu} migrate_set_cachesize 256m > {qemu} migrate -x -d tcp:destination.host:4444 > {qemu} info migrate > ... > transferred ram-duplicate: A kbytes > transferred ram-duplicate: B pages > transferred ram-normal: C kbytes > transferred ram-normal: D pages > transferred ram-xbrle: E kbytes > transferred ram-xbrle: F pages > overflow ram-xbrle: G pages > cache-hit ram-xbrle: H pages > cache-lookup ram-xbrle: J pages > > Testing: live migration with XBZRLE completed in 110 seconds, without live > migration was not able to complete. > > A simple synthetic memory r/w load generator: > .. include > .. include > .. int main() > .. { > .. char *buf = (char *) calloc(4096, 4096); > .. while (1) { > .. int i; > .. for (i = 0; i< 4096 * 4; i++) { > .. buf[i * 4096 / 4]++; > .. } > .. printf("."); > .. } > .. } > > Please provide documentation in docs/ of the compression format. IMO it should be disabled by default (with an option to disable it, via, sat, migrate-set-options, so we can migrate to older hosts). The protocol should allow XBZRLE to turn itself off if it detects that it isn't effective. -- error compiling committee.c: too many arguments to function