From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:34534) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Ri6Ol-0005fE-BQ for qemu-devel@nongnu.org; Tue, 03 Jan 2012 10:35:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Ri6Oi-0006wz-MY for qemu-devel@nongnu.org; Tue, 03 Jan 2012 10:35:35 -0500 Received: from mx1.redhat.com ([209.132.183.28]:50602) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Ri6Oi-0006ws-6U for qemu-devel@nongnu.org; Tue, 03 Jan 2012 10:35:32 -0500 From: Orit Wasserman Date: Tue, 3 Jan 2012 17:34:30 +0200 Message-Id: <1325604879-15862-1-git-send-email-owasserm@redhat.com> Subject: [Qemu-devel] [PATCH v5 0/9] XBZRLE delta for live migration of large memory apps List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Petter Svard , stefanha@gmail.com, quintela@redhat.com, blauwirbel@gmail.com, Orit Wasserman , Benoit Hudzia , Aidan Shribman Changes from v4: 1) Rebase 2) divide patch into 9 patches 3) move memory allocation into cache_insert By using XBZRLE (Xor Binary Zero Run-Length-Encoding) we can reduce VM downtime and total live-migration time of VMs running memory write intensive workloads typical of large enterprise applications such as SAP ERP Systems, and generally speaking for any application with a sparse memory update pattern. On the sender side XBZRLE is used as a compact delta encoding of page updates, retrieving the old page content from an LRU cache (default size of 64 MB). The receiving side uses the existing page content and XBZRLE to decode the new page content. Work was originally based on research results published VEE 2011: Evaluation of Delta Compression Techniques for Efficient Live Migration of Large Virtual Machines by Benoit, Svard, Tordsson and Elmroth. Additionally the delta encoder XBRLE was improved further using XBZRLE instead. XBZRLE has a sustained bandwidth of 2-2.5 GB/s for typical workloads making it ideal for in-line, real-time encoding such as is needed for live-migration. A typical usage scenario: {qemu} migrate_set_cachesize 256m {qemu} migrate -x -d tcp:destination.host:4444 {qemu} info migrate ... transferred ram-duplicate: A kbytes transferred ram-duplicate: B pages transferred ram-normal: C kbytes transferred ram-normal: D pages transferred ram-xbrle: E kbytes transferred ram-xbrle: F pages overflow ram-xbrle: G pages cache-hit ram-xbrle: H pages cache-lookup ram-xbrle: J pages Testing: live migration with XBZRLE completed in 110 seconds, without live migration was not able to complete. A simple synthetic memory r/w load generator: .. include .. include .. int main() .. { .. char *buf = (char *) calloc(4096, 4096); .. while (1) { .. int i; .. for (i = 0; i < 4096 * 4; i++) { .. buf[i * 4096 / 4]++; .. } .. printf("."); .. } .. } Signed-off-by: Benoit Hudzia Signed-off-by: Petter Svard Signed-off-by: Aidan Shribman arch_init.c | 546 +++++++++++++++++++++++++++++++++++++++++++++++++---- block-migration.c | 4 +- hmp-commands.hx | 34 +++- hw/hw.h | 4 +- migration.c | 51 +++++- migration.h | 19 ++ qmp-commands.hx | 44 ++++- savevm.c | 11 +- sysemu.h | 4 +- 9 files changed, 653 insertions(+), 64 deletions(-) -- 1.7.6.5