From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53837) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1buUZJ-0004ai-7r for qemu-devel@nongnu.org; Wed, 12 Oct 2016 21:12:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1buUZF-0003Wy-0v for qemu-devel@nongnu.org; Wed, 12 Oct 2016 21:12:20 -0400 Received: from szxga01-in.huawei.com ([58.251.152.64]:18034) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1buUZ6-00036x-Ck for qemu-devel@nongnu.org; Wed, 12 Oct 2016 21:12:16 -0400 References: <20161012140442.GB15590@stefanha-x1.localdomain> <20161012142121.GA13343@work-vm> From: Hailiang Zhang Message-ID: <57FEDF2F.20007@huawei.com> Date: Thu, 13 Oct 2016 09:11:11 +0800 MIME-Version: 1.0 In-Reply-To: <20161012142121.GA13343@work-vm> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] Async savevm using userfaultfd(2) List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert" , Stefan Hajnoczi Cc: Andrea Arcangeli , "Denis V. Lunev" , jsnow@redhat.com, qemu-devel@nongnu.org On 2016/10/12 22:21, Dr. David Alan Gilbert wrote: > * Stefan Hajnoczi (stefanha@gmail.com) wrote: >> John and I recently discussed asynchronous savevm and I wanted to post >> the ideas so they aren't forgotten. (We're not actively working on this >> feature.) >> >> Asynchronous savevm has the same effect as the 'savevm' monitor command: >> it saves RAM, device state, and a snapshot of all disks at the point in >> time the command was issued. >> >> The current 'savevm' monitor command is synchronous so the guest and >> QEMU monitor are blocked while the operation runs (it can take a >> while!). Asynchronous savevm has the advantage of allowing the guest >> and QEMU monitor to continue while the operation is running. >> >> This sounds similar to live migration to file but remember that live >> migration's consistency point is when the guest is paused at the end of >> the iteration phase. The user has no control over *when* live migration >> captures the guest state. Therefore it's not a useful command for >> taking snapshots of guest state at a specific point in time - we need >> asynchronous savevm for that. >> >> Async savevm must copy-on-write guest RAM so the guest can continue >> writing to memory while the snapshot is being saved. Rik van Riel >> suggested using userfaultfd(2) to do this on Linux. >> >> Unlike post-copy live migration, we want to track memory writes (instead >> of missing page faults). The userfaultfd(2) flag >> UFFDIO_REGISTER_MODE_WP provides these semantics. Unfortunately I think >> UFFDIO_REGISTER_MODE_WP is not yet implemented? > > A prototype of this has already been written by Hailiang Zhang; > see https://lists.gnu.org/archive/html/qemu-devel/2016-08/msg03441.html > Yes, I have updated it to 2th version in private, but unfortunately, there are still some problems with UFFDIO_REGISTER_MODE_WP API in kernel, It still can't support KVM, (only supports tcg mode). I have given feedback to Andrea, but got no response ... :( http://www.mail-archive.com/qemu-devel@nongnu.org/msg394897.html >> Once UFFDIO_REGISTER_MODE_WP is available QEMU can catch writes to guest >> RAM and copy the original pages to a buffer. If memory is dirtied too >> quickly then it's necessary to throttle the guest or fail the savevm >> operation. > > The only limit there is the size of the buffer, waiting for space will > do the throttling. > Yes, We can optimize it by extend the size of buffer and use multiple fds to handle the user fault. Hailiang > Dave > >> >> Perhaps this approach can be prototyped with mprotect and a SIGSEGV >> handler if anyone wants to get async savevm going. I don't know if >> there are any disadvantages to mprotecting guest RAM that the kvm kernel >> module is using. Hopefully in-kernel devices and vhost will continue to >> work. >> >> Stefan > > > -- > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > > > . >