Re: [PATCH] mm: Do not stall in synchronous compaction for THP allocations

[Andrew Morton <akpm@linux-foundation.org>]

On Fri, 11 Nov 2011 10:01:56 +0000
Mel Gorman <mgorman@suse.de> wrote:

> On Thu, Nov 10, 2011 at 03:12:11PM -0800, Andrew Morton wrote:
> > On Thu, 10 Nov 2011 16:13:31 +0000
> > Mel Gorman <mgorman@suse.de> wrote:
> > 
> > > This patch once again prevents sync migration for transparent
> > > hugepage allocations as it is preferable to fail a THP allocation
> > > than stall.
> > 
> > Who said?  ;)
> 
> Everyone who ever complained about stalls due to writing to USB :)

oh, here it is.  Who broke mesage threading?

> In the last few releases, we have squashed a number of stalls due
> to writing pages from the end of the LRU but sync compaction is a
> relatively recent new root cause of stalls.
> 
> > Presumably some people would prefer to get lots of
> > huge pages for their 1000-hour compute job, and waiting a bit to get
> > those pages is acceptable.
> > 
> 
> A 1000-hour compute job will have its pages collapsed into hugepages by
> khugepaged so they might not have the huge pages at the very beginning
> but they get them. With khugepaged in place, there should be no need for
> an additional tuneable.

OK...

> > Do we have the accounting in place for us to be able to determine how
> > many huge page allocation attempts failed due to this change?
> > 
> 
> thp_fault_fallback is the big one. It is incremented if we fail to
> 	allocate a hugepage during fault in either
> 	do_huge_pmd_anonymous_page or do_huge_pmd_wp_page_fallback
> 
> thp_collapse_alloc_failed is also very interesting. It is incremented
> 	if khugepaged tried to collapse pages into a hugepage and
> 	failed the allocation
> 
> The user has the  option of monitoring their compute jobs hugepage
> usage by reading /proc/PID/smaps and looking at the AnonHugePages
> count for the large mappings of interest.

Fair enough.  One slight problem though:

akpm:/usr/src/25> grep -r thp_collapse_alloc_failed Documentation 
akpm:/usr/src/25>