From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34203) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WKS37-0001JU-6k for qemu-devel@nongnu.org; Mon, 03 Mar 2014 07:32:57 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WKS2y-0004SW-Or for qemu-devel@nongnu.org; Mon, 03 Mar 2014 07:32:49 -0500 Received: from mail-ee0-x22e.google.com ([2a00:1450:4013:c00::22e]:42503) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WKS2y-0004S6-I6 for qemu-devel@nongnu.org; Mon, 03 Mar 2014 07:32:40 -0500 Received: by mail-ee0-f46.google.com with SMTP id t10so1220836eei.33 for ; Mon, 03 Mar 2014 04:32:39 -0800 (PST) Date: Mon, 3 Mar 2014 13:32:34 +0100 From: Stefan Hajnoczi Message-ID: <20140303123234.GC21055@stefanha-thinkpad.redhat.com> References: <615092B2FD0E7648B6E4B43E029BCFB84D578044@SZXEMA503-MBS.china.huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <615092B2FD0E7648B6E4B43E029BCFB84D578044@SZXEMA503-MBS.china.huawei.com> Subject: Re: [Qemu-devel] [RFC]VM live snapshot proposal List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Huangpeng (Peter)" Cc: "kwolf@redhat.com" , Pavel Hrdina , Zhanghailiang , KVM devel mailing list , "qemu-devel@nongnu.org" , Paolo Bonzini , Wenchao Xia On Mon, Mar 03, 2014 at 01:13:41AM +0000, Huangpeng (Peter) wrote: Just to summarize the idea of live savevm for people joining the discussion: It should be possible to save a snapshot of the guest (including memory, devices, and disk) without noticable downtime. The 'savevm' command pauses the guest until the snapshot has been completed and therefore doesn't meet the requirements. > Here I have another proposal, based on the live-migration scheme, add consistent > memory state tracking and saving. > The idea is simple: > 1.First round use live-migration to save all memory to a snapshot file. > 2.intercept the action of memory-modify, save old pages to a temporary file and mark dirty-bits, > 3.Merge temporary file to the original snapshot file > > Detailed process: > (1)Pause VM > (2) Save the device status to a temporary file (live-migration already supported ) > (3) Make disk snapshot > (4) Enable page dirty log and old dirty pages save function(which we need to add) > (5) Resume VM > (6) Begin the first round of iteration, we save the entire contents of the VM memory pages > to the snapshot file > (7) In the second round of iteration , we save the old page to the snapshot file > (8) Merge data of device status which is pre-saved in temporary files to the snapshot file > (8) End ram snapshot and some cleanup work > > Due to memory-modifications may happen in kvm, qemu, or vhost, the key-part is how we > can provide common page-modify-tracking-and-saving api, we completed a prototype by > simply add modified-page tracking/saving function in qemu, and it seems worked fine. Yes, this is the tricky part. To be honest, I think this is the reason no one has submitted patches - it's a hard task and the win isn't that great (you can already migrate to file). But back to the options: If the host has enough free memory to fork QEMU, a small helper process can be used to save the copy-on-write memory snapshot (thanks to fork(2) semantics). The hard part about the fork(2) approach is that QEMU isn't really designed to fork, so work is necessary to reach a quiescent state for the child process. If there is not enough memory to fork, then a synchronous approach to catching guest memory writes is needed. I'm not sure if a good mechanism for that exists but the simplest would be mprotect(2) and a signal handler (which will make the guest run very slowly). Stefan