Re: [RFC] Moving dirty bitmaps to userspace - Double buffering approach

[Marcelo Tosatti <mtosatti@redhat.com>]

On Mon, Mar 08, 2010 at 05:22:43PM +0900, Takuya Yoshikawa wrote:
> Hi, I would like to hear your comments about the following plan:
> 
>   Moving dirty bitmaps to userspace
>     - Double buffering approach
> 
> especially I would be glad if I can hear some advice about how
> to keep the compatibility.
> 
> Thanks in advance,
>   Takuya
> 
> 
> ---
> Overview:
> 
> Last time, I submitted a patch
>   "make get dirty log ioctl return the first dirty page's position"
>    http://www.spinics.net/lists/kvm/msg29724.html
> and got some new better ideas from Avi.
> 
> As a result, I agreed to try to eliminate the bitmap allocation
> done in the x86 KVM every time when we execute get dirty log by
> using "double buffering" approach.
> 
> 
> Here is my plan:
> 
> - move the dirty bitmap allocation to userspace
> 
> We allocate bitmaps in the userspace and register them by ioctl.
> Once a bitmap is registered, we do not touch it from userspace
> and let the kernel modify it directly until we switch to the next
> bitmap. We use "double buffering" at this switch point: userspace
> give the kernel a new bitmap by ioctl and the kernel switch the
> bitmap atomically to new one.
>
> After succeeded in this switch, we can read the old bitmap freely
> in the userspace and free it if we want: needless to say we can
> also reuse it at the next switch.
> 
> 
> - implementation details
> 
> Although it may be possible to touch the bitmap from the kernel
> side without doing kmap, I think kmapping the bitmap is better.
> So we may use the following functions paying enough attention to
> the preemption control.
>   - get_user_pages()
>   - kmap_atomic()
> 
> 
> - compatibility issues
> 
> What I am facing now are the compatibility issues. We have to
> support both the userspace and kernel side bitmap allocations
> to let the current qemu and KVM work properly.
> 
> 1. From the kernel side, we have to care bitmap allocations done in both
> the kvm_vm_ioctl_set_memory_region() and kvm_vm_ioctl_get_dirty_log().
> 
> 2. From the userspace side, we have to check the new api's availability
> and determine which way we use, e.g. by using check extension ioctl.
> 
> The most problematic is 1, kernel side. We have to be able to know
> by which way current bitmap allocation is being done using flags or
> something. In the case of set memory region, we have to judge whether
> we allocate a bitmap, and if not we have to register a bitmap later
> by another api: set memory region is not restricted to the dirty log
> issues and need more care than get dirty log.
> 
> Are there any good ways to solve this kind of problems?

You can introduce a new get_dirty_log ioctl that passes the address
of the next bitmap in userspace, and use it (after pinning with
get_user_pages), instead of vmalloc'ing.