* [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC.
@ 2009-04-21 5:22 KOSAKI Motohiro
2009-04-21 7:12 ` Minchan Kim
2009-04-22 14:32 ` Mel Gorman
0 siblings, 2 replies; 6+ messages in thread
From: KOSAKI Motohiro @ 2009-04-21 5:22 UTC (permalink / raw)
To: Andy Whitcroft, Peter Zijlstra, Mel Gorman, Rik van Riel, LKML,
linux-mm, Andrew Morton, Lee Schermerhorn
Cc: kosaki.motohiro
Subject: [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC.
commit 33c120ed2843090e2bd316de1588b8bf8b96cbde (more aggressively use lumpy reclaim)
change lumpy reclaim using condition. but it isn't enough change.
lumpy reclaim don't only mean isolate neighber page, but also do pageout as synchronous.
this patch does it.
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Cc: Andy Whitcroft <apw@shadowen.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: Rik van Riel <riel@redhat.com>
Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
---
mm/vmscan.c | 29 +++++++++++++++--------------
1 file changed, 15 insertions(+), 14 deletions(-)
Index: b/mm/vmscan.c
===================================================================
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1049,6 +1049,19 @@ static unsigned long shrink_inactive_lis
unsigned long nr_scanned = 0;
unsigned long nr_reclaimed = 0;
struct zone_reclaim_stat *reclaim_stat = get_reclaim_stat(zone, sc);
+ int lumpy_reclaim = 0;
+
+ /*
+ * If we need a large contiguous chunk of memory, or have
+ * trouble getting a small set of contiguous pages, we
+ * will reclaim both active and inactive pages.
+ *
+ * We use the same threshold as pageout congestion_wait below.
+ */
+ if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
+ lumpy_reclaim = 1;
+ else if (sc->order && priority < DEF_PRIORITY - 2)
+ lumpy_reclaim = 1;
pagevec_init(&pvec, 1);
@@ -1061,19 +1074,7 @@ static unsigned long shrink_inactive_lis
unsigned long nr_freed;
unsigned long nr_active;
unsigned int count[NR_LRU_LISTS] = { 0, };
- int mode = ISOLATE_INACTIVE;
-
- /*
- * If we need a large contiguous chunk of memory, or have
- * trouble getting a small set of contiguous pages, we
- * will reclaim both active and inactive pages.
- *
- * We use the same threshold as pageout congestion_wait below.
- */
- if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
- mode = ISOLATE_BOTH;
- else if (sc->order && priority < DEF_PRIORITY - 2)
- mode = ISOLATE_BOTH;
+ int mode = lumpy_reclaim ? ISOLATE_BOTH : ISOLATE_INACTIVE;
nr_taken = sc->isolate_pages(sc->swap_cluster_max,
&page_list, &nr_scan, sc->order, mode,
@@ -1110,7 +1111,7 @@ static unsigned long shrink_inactive_lis
* but that should be acceptable to the caller
*/
if (nr_freed < nr_taken && !current_is_kswapd() &&
- sc->order > PAGE_ALLOC_COSTLY_ORDER) {
+ lumpy_reclaim) {
congestion_wait(WRITE, HZ/10);
/*
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC.
2009-04-21 5:22 [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC KOSAKI Motohiro
@ 2009-04-21 7:12 ` Minchan Kim
2009-04-21 7:21 ` KOSAKI Motohiro
2009-04-22 14:32 ` Mel Gorman
1 sibling, 1 reply; 6+ messages in thread
From: Minchan Kim @ 2009-04-21 7:12 UTC (permalink / raw)
To: KOSAKI Motohiro
Cc: Andy Whitcroft, Peter Zijlstra, Mel Gorman, Rik van Riel, LKML,
linux-mm, Andrew Morton, Lee Schermerhorn, Minchan Kim
Hi, Kosaki-san.
On Tue, 21 Apr 2009 14:22:27 +0900 (JST)
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> Subject: [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC.
>
> commit 33c120ed2843090e2bd316de1588b8bf8b96cbde (more aggressively use lumpy reclaim)
> change lumpy reclaim using condition. but it isn't enough change.
>
> lumpy reclaim don't only mean isolate neighber page, but also do pageout as synchronous.
> this patch does it.
I agree.
Andi added synchronous lumpy reclaim with c661b078fd62abe06fd11fab4ac5e4eeafe26b6d.
At that time, lumpy reclaim is not agressive.
His intension is just for high-order users.(above PAGE_ALLOC_COSTLY_ORDER).
After some time, Rik added aggressive lumpy reclaim with 33c120ed2843090e2bd316de1588b8bf8b96cbde.
His intension is that do lumpy reclaim when high-order users and trouble getting a small set of contiguous pages.
So we also have to add synchronous pageout for small set of contiguous pages.
Nice catch!.
Reviewed-by: Minchan Kim <Minchan.kim@gmail.com>
BTW, Do you have any number ?
--
Kinds Regards
Minchan Kim
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC.
2009-04-21 7:12 ` Minchan Kim
@ 2009-04-21 7:21 ` KOSAKI Motohiro
2009-04-21 7:39 ` Minchan Kim
0 siblings, 1 reply; 6+ messages in thread
From: KOSAKI Motohiro @ 2009-04-21 7:21 UTC (permalink / raw)
To: Minchan Kim
Cc: kosaki.motohiro, Andy Whitcroft, Peter Zijlstra, Mel Gorman,
Rik van Riel, LKML, linux-mm, Andrew Morton, Lee Schermerhorn
> Hi, Kosaki-san.
>
> On Tue, 21 Apr 2009 14:22:27 +0900 (JST)
> KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>
> > Subject: [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC.
> >
> > commit 33c120ed2843090e2bd316de1588b8bf8b96cbde (more aggressively use lumpy reclaim)
> > change lumpy reclaim using condition. but it isn't enough change.
> >
> > lumpy reclaim don't only mean isolate neighber page, but also do pageout as synchronous.
> > this patch does it.
>
> I agree.
>
> Andi added synchronous lumpy reclaim with c661b078fd62abe06fd11fab4ac5e4eeafe26b6d.
> At that time, lumpy reclaim is not agressive.
> His intension is just for high-order users.(above PAGE_ALLOC_COSTLY_ORDER).
>
> After some time, Rik added aggressive lumpy reclaim with 33c120ed2843090e2bd316de1588b8bf8b96cbde.
> His intension is that do lumpy reclaim when high-order users and trouble getting a small set of contiguous pages.
>
> So we also have to add synchronous pageout for small set of contiguous pages.
> Nice catch!.
>
> Reviewed-by: Minchan Kim <Minchan.kim@gmail.com>
>
> BTW, Do you have any number ?
No.
Actually, this logic only run when system is strongly memory stavation
or fragment. not normal case.
At that time, another slowdown thing hide synchronous reclaim latency, I think.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC.
2009-04-21 7:21 ` KOSAKI Motohiro
@ 2009-04-21 7:39 ` Minchan Kim
0 siblings, 0 replies; 6+ messages in thread
From: Minchan Kim @ 2009-04-21 7:39 UTC (permalink / raw)
To: KOSAKI Motohiro
Cc: Minchan Kim, Andy Whitcroft, Peter Zijlstra, Mel Gorman,
Rik van Riel, LKML, linux-mm, Andrew Morton, Lee Schermerhorn
On Tue, 21 Apr 2009 16:21:18 +0900 (JST)
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> > Hi, Kosaki-san.
> >
> > On Tue, 21 Apr 2009 14:22:27 +0900 (JST)
> > KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> >
> > > Subject: [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC.
> > >
> > > commit 33c120ed2843090e2bd316de1588b8bf8b96cbde (more aggressively use lumpy reclaim)
> > > change lumpy reclaim using condition. but it isn't enough change.
> > >
> > > lumpy reclaim don't only mean isolate neighber page, but also do pageout as synchronous.
> > > this patch does it.
> >
> > I agree.
> >
> > Andi added synchronous lumpy reclaim with c661b078fd62abe06fd11fab4ac5e4eeafe26b6d.
> > At that time, lumpy reclaim is not agressive.
> > His intension is just for high-order users.(above PAGE_ALLOC_COSTLY_ORDER).
> >
> > After some time, Rik added aggressive lumpy reclaim with 33c120ed2843090e2bd316de1588b8bf8b96cbde.
> > His intension is that do lumpy reclaim when high-order users and trouble getting a small set of contiguous pages.
> >
> > So we also have to add synchronous pageout for small set of contiguous pages.
> > Nice catch!.
> >
> > Reviewed-by: Minchan Kim <Minchan.kim@gmail.com>
> >
> > BTW, Do you have any number ?
>
> No.
>
> Actually, this logic only run when system is strongly memory stavation
> or fragment. not normal case.
>
> At that time, another slowdown thing hide synchronous reclaim latency, I think.
>
Yes. I think it's hard measure, too.
I was just out of curiosity if server guy have a any benchmark method. ;-)
--
Kinds Regards
Minchan Kim
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC.
2009-04-21 5:22 [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC KOSAKI Motohiro
2009-04-21 7:12 ` Minchan Kim
@ 2009-04-22 14:32 ` Mel Gorman
2009-04-23 1:20 ` KOSAKI Motohiro
1 sibling, 1 reply; 6+ messages in thread
From: Mel Gorman @ 2009-04-22 14:32 UTC (permalink / raw)
To: KOSAKI Motohiro
Cc: Andy Whitcroft, Peter Zijlstra, Rik van Riel, LKML, linux-mm,
Andrew Morton, Lee Schermerhorn
On Tue, Apr 21, 2009 at 02:22:27PM +0900, KOSAKI Motohiro wrote:
> Subject: [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC.
>
> commit 33c120ed2843090e2bd316de1588b8bf8b96cbde (more aggressively use lumpy reclaim)
> change lumpy reclaim using condition. but it isn't enough change.
>
> lumpy reclaim don't only mean isolate neighber page, but also do pageout as synchronous.
> this patch does it.
>
> Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
> Cc: Andy Whitcroft <apw@shadowen.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Mel Gorman <mel@csn.ul.ie>
> Cc: Rik van Riel <riel@redhat.com>
> Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Seems fair although the changelog could be better. Maybe something like?
====
Commit 33c120ed2843090e2bd316de1588b8bf8b96cbde increased how aggressive
lumpy reclaim was by isolating both active and inactive pages for asynchronous
lumpy reclaim on costly-high-order pages and for cheap-high-order when memory
pressure is high. However, if the system is under heavy pressure and there
are dirty pages, asynchronous IO may not be sufficient to reclaim a suitable
page in time.
This patch causes the caller to enter synchronous lumpy reclaim for
costly-high-order pages and for cheap-high-order pages when under memory
pressure.
====
Whether the changelog is updated or not though;
Reviewed-by: Mel Gorman <mel@csn.ul.ie>
> ---
> mm/vmscan.c | 29 +++++++++++++++--------------
> 1 file changed, 15 insertions(+), 14 deletions(-)
>
> Index: b/mm/vmscan.c
> ===================================================================
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -1049,6 +1049,19 @@ static unsigned long shrink_inactive_lis
> unsigned long nr_scanned = 0;
> unsigned long nr_reclaimed = 0;
> struct zone_reclaim_stat *reclaim_stat = get_reclaim_stat(zone, sc);
> + int lumpy_reclaim = 0;
> +
> + /*
> + * If we need a large contiguous chunk of memory, or have
> + * trouble getting a small set of contiguous pages, we
> + * will reclaim both active and inactive pages.
> + *
> + * We use the same threshold as pageout congestion_wait below.
> + */
> + if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
> + lumpy_reclaim = 1;
> + else if (sc->order && priority < DEF_PRIORITY - 2)
> + lumpy_reclaim = 1;
>
> pagevec_init(&pvec, 1);
>
> @@ -1061,19 +1074,7 @@ static unsigned long shrink_inactive_lis
> unsigned long nr_freed;
> unsigned long nr_active;
> unsigned int count[NR_LRU_LISTS] = { 0, };
> - int mode = ISOLATE_INACTIVE;
> -
> - /*
> - * If we need a large contiguous chunk of memory, or have
> - * trouble getting a small set of contiguous pages, we
> - * will reclaim both active and inactive pages.
> - *
> - * We use the same threshold as pageout congestion_wait below.
> - */
> - if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
> - mode = ISOLATE_BOTH;
> - else if (sc->order && priority < DEF_PRIORITY - 2)
> - mode = ISOLATE_BOTH;
> + int mode = lumpy_reclaim ? ISOLATE_BOTH : ISOLATE_INACTIVE;
>
> nr_taken = sc->isolate_pages(sc->swap_cluster_max,
> &page_list, &nr_scan, sc->order, mode,
> @@ -1110,7 +1111,7 @@ static unsigned long shrink_inactive_lis
> * but that should be acceptable to the caller
> */
> if (nr_freed < nr_taken && !current_is_kswapd() &&
> - sc->order > PAGE_ALLOC_COSTLY_ORDER) {
> + lumpy_reclaim) {
> congestion_wait(WRITE, HZ/10);
>
> /*
>
>
--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC.
2009-04-22 14:32 ` Mel Gorman
@ 2009-04-23 1:20 ` KOSAKI Motohiro
0 siblings, 0 replies; 6+ messages in thread
From: KOSAKI Motohiro @ 2009-04-23 1:20 UTC (permalink / raw)
To: Mel Gorman, Andrew Morton
Cc: kosaki.motohiro, Andy Whitcroft, Peter Zijlstra, Rik van Riel,
LKML, linux-mm, Lee Schermerhorn
> On Tue, Apr 21, 2009 at 02:22:27PM +0900, KOSAKI Motohiro wrote:
> > Subject: [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC.
> >
> > commit 33c120ed2843090e2bd316de1588b8bf8b96cbde (more aggressively use lumpy reclaim)
> > change lumpy reclaim using condition. but it isn't enough change.
> >
> > lumpy reclaim don't only mean isolate neighber page, but also do pageout as synchronous.
> > this patch does it.
> >
> > Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
> > Cc: Andy Whitcroft <apw@shadowen.org>
> > Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > Cc: Mel Gorman <mel@csn.ul.ie>
> > Cc: Rik van Riel <riel@redhat.com>
> > Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>
> Seems fair although the changelog could be better. Maybe something like?
>
> ====
>
> Commit 33c120ed2843090e2bd316de1588b8bf8b96cbde increased how aggressive
> lumpy reclaim was by isolating both active and inactive pages for asynchronous
> lumpy reclaim on costly-high-order pages and for cheap-high-order when memory
> pressure is high. However, if the system is under heavy pressure and there
> are dirty pages, asynchronous IO may not be sufficient to reclaim a suitable
> page in time.
>
> This patch causes the caller to enter synchronous lumpy reclaim for
> costly-high-order pages and for cheap-high-order pages when under memory
> pressure.
> ====
>
> Whether the changelog is updated or not though;
>
> Reviewed-by: Mel Gorman <mel@csn.ul.ie>
Cool!
Andrew, Could you please replace vmscan-low-order-lumpy-reclaim-also-should-use-pageout_io_sync.patch
with following patch?
===================================
Subject: vmscan: low order lumpy reclaim also should use PAGEOUT_IO_SYNC
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Commit 33c120ed2843090e2bd316de1588b8bf8b96cbde increased how aggressive
lumpy reclaim was by isolating both active and inactive pages for asynchronous
lumpy reclaim on costly-high-order pages and for cheap-high-order when memory
pressure is high. However, if the system is under heavy pressure and there
are dirty pages, asynchronous IO may not be sufficient to reclaim a suitable
page in time.
This patch causes the caller to enter synchronous lumpy reclaim for
costly-high-order pages and for cheap-high-order pages when under memory
pressure.
Minchan.kim@gmail.com said:
Andy added synchronous lumpy reclaim with
c661b078fd62abe06fd11fab4ac5e4eeafe26b6d. At that time, lumpy reclaim is
not agressive. His intension is just for high-order users.(above
PAGE_ALLOC_COSTLY_ORDER).
After some time, Rik added aggressive lumpy reclaim with
33c120ed2843090e2bd316de1588b8bf8b96cbde. His intention was to do lumpy
reclaim when high-order users and trouble getting a small set of
contiguous pages.
So we also have to add synchronous pageout for small set of contiguous
pages.
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Cc: Andy Whitcroft <apw@shadowen.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Rik van Riel <riel@redhat.com>
Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Reviewed-by: Minchan Kim <Minchan.kim@gmail.com>
Reviewed-by: Mel Gorman <mel@csn.ul.ie>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
mm/vmscan.c | 29 +++++++++++++++--------------
1 file changed, 15 insertions(+), 14 deletions(-)
diff -puN mm/vmscan.c~vmscan-low-order-lumpy-reclaim-also-should-use-pageout_io_sync mm/vmscan.c
--- a/mm/vmscan.c~vmscan-low-order-lumpy-reclaim-also-should-use-pageout_io_sync
+++ a/mm/vmscan.c
@@ -1059,6 +1059,19 @@ static unsigned long shrink_inactive_lis
unsigned long nr_scanned = 0;
unsigned long nr_reclaimed = 0;
struct zone_reclaim_stat *reclaim_stat = get_reclaim_stat(zone, sc);
+ int lumpy_reclaim = 0;
+
+ /*
+ * If we need a large contiguous chunk of memory, or have
+ * trouble getting a small set of contiguous pages, we
+ * will reclaim both active and inactive pages.
+ *
+ * We use the same threshold as pageout congestion_wait below.
+ */
+ if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
+ lumpy_reclaim = 1;
+ else if (sc->order && priority < DEF_PRIORITY - 2)
+ lumpy_reclaim = 1;
pagevec_init(&pvec, 1);
@@ -1071,19 +1084,7 @@ static unsigned long shrink_inactive_lis
unsigned long nr_freed;
unsigned long nr_active;
unsigned int count[NR_LRU_LISTS] = { 0, };
- int mode = ISOLATE_INACTIVE;
-
- /*
- * If we need a large contiguous chunk of memory, or have
- * trouble getting a small set of contiguous pages, we
- * will reclaim both active and inactive pages.
- *
- * We use the same threshold as pageout congestion_wait below.
- */
- if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
- mode = ISOLATE_BOTH;
- else if (sc->order && priority < DEF_PRIORITY - 2)
- mode = ISOLATE_BOTH;
+ int mode = lumpy_reclaim ? ISOLATE_BOTH : ISOLATE_INACTIVE;
nr_taken = sc->isolate_pages(sc->swap_cluster_max,
&page_list, &nr_scan, sc->order, mode,
@@ -1120,7 +1121,7 @@ static unsigned long shrink_inactive_lis
* but that should be acceptable to the caller
*/
if (nr_freed < nr_taken && !current_is_kswapd() &&
- sc->order > PAGE_ALLOC_COSTLY_ORDER) {
+ lumpy_reclaim) {
congestion_wait(WRITE, HZ/10);
/*
_
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2009-04-23 1:20 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2009-04-21 5:22 [PATCH] low order lumpy reclaim also should use PAGEOUT_IO_SYNC KOSAKI Motohiro
2009-04-21 7:12 ` Minchan Kim
2009-04-21 7:21 ` KOSAKI Motohiro
2009-04-21 7:39 ` Minchan Kim
2009-04-22 14:32 ` Mel Gorman
2009-04-23 1:20 ` KOSAKI Motohiro
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).