Re: [PATCH/RFC 0/8] numa - Migrate-on-Fault

[Lee Schermerhorn <Lee.Schermerhorn@hp.com>]

On Wed, 2010-11-17 at 19:10 +0200, Avi Kivity wrote:
> On 11/11/2010 09:44 PM, Lee Schermerhorn wrote:
> > This series of patches implements page migration in the fault path.
> >
> > !!! N.B., Need to consider iteraction with KSM and Transparent Huge
> > !!! Pages.
> >
> > The basic idea is that when a fault handler such as do_swap_page()
> > finds a cached page with zero mappings that is otherwise "stable"--
> > e.g., no I/O in progress--this is a good opportunity to check whether the
> > page resides on the node indicated by the mempolicy in the current context.
> >
> > We only attempt to migrate when there are zero mappings because 1) we can
> > easily migrate the page--don't have to go through the effort of removing
> > all mappings and 2) default policy--a common case--can give different
> > answers from different tasks running on different nodes.  Checking the
> > policy when there are zero mappings effectively implements a "first touch"
> > placement policy.
> 
> A couple of kvm-related notes:
> - kvm page faults are significantly more expensive than ordinary page 
> faults; this will affect the cost/benefit tradeoff
> - kvm faults go through get_user_pages_fast(), not the ordinary fault 
> path.  Will the code handle this?

Don't know, Avi.  These patches pre-date all of the kvm, ksm,
transparent huge pages work.  I wasn't running any of these on the tests
I ran last week before posting.  When I rebased to the 9nov mmotm, I
attempted to test on a RHEL6.0 install, so probably transparent
hugepages and maybe ksm daemons were running.  Certainly libvirtd was
running, but I don't think I had any guests running.  But, it occurs to
me that the null pointer deref I saw on this config could have been the
result of interactions with newer virt features in RHEL6.

That said, the 'migrate-on-fault' mechanism which is at the heart of
lazy migration is currently hooked [only] into do_swap_page().  So, when
get_user_pages(_fast)() calls the page fault handler, the pages should
be migrated if they're found in the correct state and are "misplaced".
The interactions I'm more worried about are in the filtering of pages
selected for automigration--probably want to skip THPs and KSM'd pages.

Lee