[RFC] Moving dirty bitmaps to userspace - Double buffering approach

[Takuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>]

Hi, I would like to hear your comments about the following plan:

   Moving dirty bitmaps to userspace
     - Double buffering approach

especially I would be glad if I can hear some advice about how
to keep the compatibility.

Thanks in advance,
   Takuya


---
Overview:

Last time, I submitted a patch
   "make get dirty log ioctl return the first dirty page's position"
    http://www.spinics.net/lists/kvm/msg29724.html
and got some new better ideas from Avi.

As a result, I agreed to try to eliminate the bitmap allocation
done in the x86 KVM every time when we execute get dirty log by
using "double buffering" approach.


Here is my plan:

- move the dirty bitmap allocation to userspace

We allocate bitmaps in the userspace and register them by ioctl.
Once a bitmap is registered, we do not touch it from userspace
and let the kernel modify it directly until we switch to the next
bitmap. We use "double buffering" at this switch point: userspace
give the kernel a new bitmap by ioctl and the kernel switch the
bitmap atomically to new one.

After succeeded in this switch, we can read the old bitmap freely
in the userspace and free it if we want: needless to say we can
also reuse it at the next switch.


- implementation details

Although it may be possible to touch the bitmap from the kernel
side without doing kmap, I think kmapping the bitmap is better.
So we may use the following functions paying enough attention to
the preemption control.
   - get_user_pages()
   - kmap_atomic()


- compatibility issues

What I am facing now are the compatibility issues. We have to
support both the userspace and kernel side bitmap allocations
to let the current qemu and KVM work properly.

1. From the kernel side, we have to care bitmap allocations done in both
the kvm_vm_ioctl_set_memory_region() and kvm_vm_ioctl_get_dirty_log().

2. From the userspace side, we have to check the new api's availability
and determine which way we use, e.g. by using check extension ioctl.

The most problematic is 1, kernel side. We have to be able to know
by which way current bitmap allocation is being done using flags or
something. In the case of set memory region, we have to judge whether
we allocate a bitmap, and if not we have to register a bitmap later
by another api: set memory region is not restricted to the dirty log
issues and need more care than get dirty log.

Are there any good ways to solve this kind of problems?