Re: Hugepage regression

[Andrew Morton <akpm@osdl.org>]

On Tue, 10 Oct 2006 10:35:50 -0700
"Chen, Kenneth W" <kenneth.w.chen@intel.com> wrote:

> David Gibson wrote on Tuesday, October 10, 2006 2:16 AM
> > > > It seems commit fe1668ae5bf0145014c71797febd9ad5670d5d05 causes a
> > > > hugepage regression.  A git bisect points the finger at that commit
> > > > for causing an oops in the 'alloc-instantiate-race' test from the
> > > > libhugetlbfs testsuite.
> > > > 
> > > > Still looking to determine the reason it breaks things.
> > > > 
> > > 
> > > It's assuming that unmap_hugepage_range() is always freeing these pages. 
> > > If the page is shared by another mapping, bad things will happen: the
> > > threads fight over page->lru.
> > > 
> > > Doing
> > > 
> > > +	if (page_count(page) == 1)
> > > 		list_add(&page->lru, &page_list);
> > > 
> > > might help.  But then we miss the tlb flush in rare racy conditions.
> > 
> > Well, there'd need to be an else doing a put_page(), too.
> > 
> > Looks like the fundamental problem is that a list is not a suitable
> > data structure for gathering here, since it's not truly local.  We
> > should probably change it to a small array, like in the normal tlb
> > gather structure.  If we run out of space we can force the tlb flush
> > and keep going.
> 
> 
> With the pending shared page table for hugetlb currently sitting in -mm,
> we serialize the all hugetlb unmap with a per file i_mmap_lock.  This
> race could well be solved by that pending patch?
> 
> http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.19-rc1/2.6.19-rc1-mm1/broken-out/shared-page-table-for-hugetlb-page-v
> 4.patch
> 

We need something for 2.6.19 though.  As David indicates, not using
page->lru should fix it (pagevec_add, pagevec_release would suit).

Or just a separate TBL invalidation per page.  Is that likely to be
particularly expensive?  It's the first one which hurts?