+ mm-vmscan-scale-number-of-pages-reclaimed-by-reclaim-compaction-only-in-direct-reclaim.patch added to -mm tree

+ mm-vmscan-scale-number-of-pages-reclaimed-by-reclaim-compaction-only-in-direct-reclaim.patch added to -mm tree

[akpm]

The patch titled
     Subject: mm: vmscan: scale number of pages reclaimed by reclaim/compaction only in direct reclaim
has been added to the -mm tree.  Its filename is
     mm-vmscan-scale-number-of-pages-reclaimed-by-reclaim-compaction-only-in-direct-reclaim.patch

Before you just go and hit "reply", please:
   a) Consider who else should be cc'ed
   b) Prefer to cc a suitable mailing list as well
   c) Ideally: find the original patch on the mailing list and do a
      reply-to-all to that, adding suitable additional cc's

*** Remember to use Documentation/SubmitChecklist when testing your code ***

The -mm tree is included into linux-next and is updated
there every 3-4 working days

------------------------------------------------------
From: Mel Gorman <mgorman@suse.de>
Subject: mm: vmscan: scale number of pages reclaimed by reclaim/compaction only in direct reclaim

Jiri Slaby reported the following:

	(It's an effective revert of "mm: vmscan: scale number of pages
	reclaimed by reclaim/compaction based on failures".)
	Given kswapd had hours of runtime in ps/top output yesterday in the
	morning and after the revert it's now 2 minutes in sum for the last 24h,
	I would say, it's gone.

The intention of the patch in question was to compensate for the loss of
lumpy reclaim.  Part of the reason lumpy reclaim worked is because it
aggressively reclaimed pages and this patch was meant to be a sane
compromise.

When compaction fails, it gets deferred and both compaction and
reclaim/compaction is deferred avoid excessive reclaim.  However, since
commit c6543459 ("mm: remove __GFP_NO_KSWAPD"), kswapd is woken up each
time and continues reclaiming which was not taken into account when the
patch was developed.

As it is not taking deferred compaction into account in this path it scans
aggressively before falling out and making the compaction_deferred check
in compaction_ready.  This patch avoids kswapd scaling pages for reclaim
and leaves the aggressive reclaim to the process attempting the THP
allocation.

Signed-off-by: Mel Gorman <mgorman@suse.de>
Reported-by: Jiri Slaby <jslaby@suse.cz>
Cc: Rik van Riel <riel@redhat.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Valdis Kletnieks <Valdis.Kletnieks@vt.edu>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/vmscan.c |   10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff -puN mm/vmscan.c~mm-vmscan-scale-number-of-pages-reclaimed-by-reclaim-compaction-only-in-direct-reclaim mm/vmscan.c
--- a/mm/vmscan.c~mm-vmscan-scale-number-of-pages-reclaimed-by-reclaim-compaction-only-in-direct-reclaim
+++ a/mm/vmscan.c
@@ -1763,14 +1763,20 @@ static bool in_reclaim_compaction(struct
 #ifdef CONFIG_COMPACTION
 /*
  * If compaction is deferred for sc->order then scale the number of pages
- * reclaimed based on the number of consecutive allocation failures
+ * reclaimed based on the number of consecutive allocation failures. This
+ * scaling only happens for direct reclaim as it is about to attempt
+ * compaction. If compaction fails, future allocations will be deferred
+ * and reclaim avoided. On the other hand, kswapd does not take compaction
+ * deferral into account so if it scaled, it could scan excessively even
+ * though allocations are temporarily not being attempted.
  */
 static unsigned long scale_for_compaction(unsigned long pages_for_compaction,
 			struct lruvec *lruvec, struct scan_control *sc)
 {
 	struct zone *zone = lruvec_zone(lruvec);
 
-	if (zone->compact_order_failed <= sc->order)
+	if (zone->compact_order_failed <= sc->order &&
+	    !current_is_kswapd())
 		pages_for_compaction <<= zone->compact_defer_shift;
 	return pages_for_compaction;
 }
_

Patches currently in -mm which might be from mgorman@suse.de are

mm-vmscan-scale-number-of-pages-reclaimed-by-reclaim-compaction-only-in-direct-reclaim.patch

Re: + mm-vmscan-scale-number-of-pages-reclaimed-by-reclaim-compaction-only-in-direct-reclaim.patch added to -mm tree

[Andrew Morton]

On Sun, 28 Oct 2012 21:22:15 +0000
Mel Gorman <mgorman@suse.de> wrote:

> > Hi, this one breaks UML build:
> > In function ???is_thp_alloc???,
> >     inlined from ???__alloc_pages_slowpath??? at mm/page_alloc.c:2427:20,
> >     inlined from ???__alloc_pages_nodemask??? at mm/page_alloc.c:2645:8:
> > mm/page_alloc.c:2383:15: error: call to ???__build_bug_failed??? declared
> > with attribute error: BUILD_BUG failed
> > 
> > That arch does not support THP.
> > 
> 
> ---8<---
> mm: page_alloc: Do not wake kswapd if the request is for THP but deferred
>
> Since commit c6543459 (mm: remove __GFP_NO_KSWAPD), kswapd gets woken
> for every THP request in the slow path. If compaction has been deferred
> the waker will not compact or enter direct reclaim on its own behalf
> but kswapd is still woken to reclaim free pages that no one may consume.
> If compaction was deferred because pages and slab was not reclaimable
> then kswapd is just consuming cycles for no gain.
> 
> This patch avoids waking kswapd if the compaction has been deferred.
> It'll still wake when compaction is running to reduce the latency of
> THP allocations.

umwhat?  I'd struggling to see how this patch will fix the UML build.

> index bb90971..e72674c 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -2378,6 +2378,15 @@ bool gfp_pfmemalloc_allowed(gfp_t gfp_mask)
>  	return !!(gfp_to_alloc_flags(gfp_mask) & ALLOC_NO_WATERMARKS);
>  }
>  
> +/* Returns true if the allocation is likely for THP */

Comment is a bit awkward/ambiguous.  Perhaps "if this allocation is
probably asking for a THP hugepage".

And why the "likely"/"probably"?  Is it so hard to know for certain?


Anyway, I think I'll duck this patch under the assumption you pasted the
wrong thing into that email.


I currently have
mm-vmscan-scale-number-of-pages-reclaimed-by-reclaim-compaction-only-in-direct-reclaim.patch
queued for 3.7, but on hold due to Valdis's bug report.