Re: [PATCH 00/11] KVM: x86: track guest page access

[Andrea Arcangeli <aarcange@redhat.com>]

On Tue, Dec 01, 2015 at 11:17:30AM +0100, Paolo Bonzini wrote:
> 
> 
> On 30/11/2015 19:26, Xiao Guangrong wrote:
> > This patchset introduces the feature which allows us to track page
> > access in guest. Currently, only write access tracking is implemented
> > in this version.
> > 
> > Four APIs are introduces:
> > - kvm_page_track_add_page(kvm, gfn, mode), single guest page @gfn is
> >   added into the track pool of the guest instance represented by @kvm,
> >   @mode specifies which kind of access on the @gfn is tracked
> >   
> > - kvm_page_track_remove_page(kvm, gfn, mode), is the opposed operation
> >   of kvm_page_track_add_page() which removes @gfn from the tracking pool.
> >   gfn is no tracked after its last user is gone
> > 
> > - kvm_page_track_register_notifier(kvm, n), register a notifier so that
> >   the event triggered by page tracking will be received, at that time,
> >   the callback of n->track_write() will be called
> > 
> > - kvm_page_track_unregister_notifier(kvm, n), does the opposed operation
> >   of kvm_page_track_register_notifier(), which unlinks the notifier and
> >   stops receiving the tracked event
> > 
> > The first user of page track is non-leaf shadow page tables as they are
> > always write protected. It also gains performance improvement because
> > page track speeds up page fault handler for the tracked pages. The
> > performance result of kernel building is as followings:
> > 
> >    before           after
> > real 461.63       real 455.48
> > user 4529.55      user 4557.88
> > sys 1995.39       sys 1922.57
> 
> For KVM-GT, as far as I know Andrea Arcangeli is working on extending
> userfaultfd to tracking write faults only.  Perhaps KVM-GT can do

I was a bit busy lately with the KSMscale design change and to fix a
THP purely theoretical issue, but the userfaultfd write tracking is
already become available here:

http://www.spinics.net/lists/linux-mm/msg97422.html

I'll be merging it soon in my tree after a thoughtful review.

> something similar, where KVM gets the write tracking functionality for
> free through the MMU notifiers.  Any thoughts on this?
> 
> Applying your technique to non-leaf shadow pages actually makes this
> series quite interesting. :)  Shadow paging is still in use for nested
> EPT, so it's always a good idea to speed it up.

I don't have the full picture of how userfaultfd write tracking could
also fit in the leaf/non-leaf shadow pagetable write tracking yet but
it's good to think about it.

In the userfaultfd case the write notification would always arrive
first through the uffd and it would be received by the qemu userfault
thread, it's then the uffd memory protect ioctl invoked by the qemu
userfault thread (to handle the write fault in userland and wake up
the thread stuck in handle_userfault()) that would also flush the
secondary MMU TLB through MMU notifier and get rid of the readonly
spte (or update it to read-write with change_pte in the best case).

Thanks,
Andrea