From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marcelo Tosatti Subject: Re: [RFC] Moving dirty bitmaps to userspace - Double buffering approach Date: Mon, 15 Mar 2010 05:33:06 -0300 Message-ID: <20100315083306.GA30179@amt.cnet> References: <4B94B3D3.2010801@oss.ntt.co.jp> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: kvm@vger.kernel.org, Avi Kivity To: Takuya Yoshikawa Return-path: Received: from mx1.redhat.com ([209.132.183.28]:49445 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1759857Ab0COIeH (ORCPT ); Mon, 15 Mar 2010 04:34:07 -0400 Content-Disposition: inline In-Reply-To: <4B94B3D3.2010801@oss.ntt.co.jp> Sender: kvm-owner@vger.kernel.org List-ID: On Mon, Mar 08, 2010 at 05:22:43PM +0900, Takuya Yoshikawa wrote: > Hi, I would like to hear your comments about the following plan: > > Moving dirty bitmaps to userspace > - Double buffering approach > > especially I would be glad if I can hear some advice about how > to keep the compatibility. > > Thanks in advance, > Takuya > > > --- > Overview: > > Last time, I submitted a patch > "make get dirty log ioctl return the first dirty page's position" > http://www.spinics.net/lists/kvm/msg29724.html > and got some new better ideas from Avi. > > As a result, I agreed to try to eliminate the bitmap allocation > done in the x86 KVM every time when we execute get dirty log by > using "double buffering" approach. > > > Here is my plan: > > - move the dirty bitmap allocation to userspace > > We allocate bitmaps in the userspace and register them by ioctl. > Once a bitmap is registered, we do not touch it from userspace > and let the kernel modify it directly until we switch to the next > bitmap. We use "double buffering" at this switch point: userspace > give the kernel a new bitmap by ioctl and the kernel switch the > bitmap atomically to new one. > > After succeeded in this switch, we can read the old bitmap freely > in the userspace and free it if we want: needless to say we can > also reuse it at the next switch. > > > - implementation details > > Although it may be possible to touch the bitmap from the kernel > side without doing kmap, I think kmapping the bitmap is better. > So we may use the following functions paying enough attention to > the preemption control. > - get_user_pages() > - kmap_atomic() > > > - compatibility issues > > What I am facing now are the compatibility issues. We have to > support both the userspace and kernel side bitmap allocations > to let the current qemu and KVM work properly. > > 1. From the kernel side, we have to care bitmap allocations done in both > the kvm_vm_ioctl_set_memory_region() and kvm_vm_ioctl_get_dirty_log(). > > 2. From the userspace side, we have to check the new api's availability > and determine which way we use, e.g. by using check extension ioctl. > > The most problematic is 1, kernel side. We have to be able to know > by which way current bitmap allocation is being done using flags or > something. In the case of set memory region, we have to judge whether > we allocate a bitmap, and if not we have to register a bitmap later > by another api: set memory region is not restricted to the dirty log > issues and need more care than get dirty log. > > Are there any good ways to solve this kind of problems? You can introduce a new get_dirty_log ioctl that passes the address of the next bitmap in userspace, and use it (after pinning with get_user_pages), instead of vmalloc'ing.