From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38309) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1buKPX-000854-Mq for qemu-devel@nongnu.org; Wed, 12 Oct 2016 10:21:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1buKPR-00041e-S1 for qemu-devel@nongnu.org; Wed, 12 Oct 2016 10:21:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51682) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1buKPR-00041E-LI for qemu-devel@nongnu.org; Wed, 12 Oct 2016 10:21:29 -0400 Date: Wed, 12 Oct 2016 15:21:23 +0100 From: "Dr. David Alan Gilbert" Message-ID: <20161012142121.GA13343@work-vm> References: <20161012140442.GB15590@stefanha-x1.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161012140442.GB15590@stefanha-x1.localdomain> Subject: Re: [Qemu-devel] Async savevm using userfaultfd(2) List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: qemu-devel@nongnu.org, jsnow@redhat.com, "Denis V. Lunev" , Rik van Riel , Andrea Arcangeli * Stefan Hajnoczi (stefanha@gmail.com) wrote: > John and I recently discussed asynchronous savevm and I wanted to post > the ideas so they aren't forgotten. (We're not actively working on this > feature.) > > Asynchronous savevm has the same effect as the 'savevm' monitor command: > it saves RAM, device state, and a snapshot of all disks at the point in > time the command was issued. > > The current 'savevm' monitor command is synchronous so the guest and > QEMU monitor are blocked while the operation runs (it can take a > while!). Asynchronous savevm has the advantage of allowing the guest > and QEMU monitor to continue while the operation is running. > > This sounds similar to live migration to file but remember that live > migration's consistency point is when the guest is paused at the end of > the iteration phase. The user has no control over *when* live migration > captures the guest state. Therefore it's not a useful command for > taking snapshots of guest state at a specific point in time - we need > asynchronous savevm for that. > > Async savevm must copy-on-write guest RAM so the guest can continue > writing to memory while the snapshot is being saved. Rik van Riel > suggested using userfaultfd(2) to do this on Linux. > > Unlike post-copy live migration, we want to track memory writes (instead > of missing page faults). The userfaultfd(2) flag > UFFDIO_REGISTER_MODE_WP provides these semantics. Unfortunately I think > UFFDIO_REGISTER_MODE_WP is not yet implemented? A prototype of this has already been written by Hailiang Zhang; see https://lists.gnu.org/archive/html/qemu-devel/2016-08/msg03441.html > Once UFFDIO_REGISTER_MODE_WP is available QEMU can catch writes to guest > RAM and copy the original pages to a buffer. If memory is dirtied too > quickly then it's necessary to throttle the guest or fail the savevm > operation. The only limit there is the size of the buffer, waiting for space will do the throttling. Dave > > Perhaps this approach can be prototyped with mprotect and a SIGSEGV > handler if anyone wants to get async savevm going. I don't know if > there are any disadvantages to mprotecting guest RAM that the kvm kernel > module is using. Hopefully in-kernel devices and vhost will continue to > work. > > Stefan -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK