[PATCH] switch free memory back to MIGRATE_MOVABLE

[PATCH] switch free memory back to MIGRATE_MOVABLE

[Shaohua Li]

When page is back to buddy and its order is bigger than pageblock_order, we can
switch its type to MIGRATE_MOVABLE. This can reduce fragmentation. The patch
has obvious effect when read a block device and then drop caches.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
---
 mm/page_alloc.c |    9 +++++++++
 1 file changed, 9 insertions(+)

Index: linux/mm/page_alloc.c
===================================================================
--- linux.orig/mm/page_alloc.c	2009-07-10 11:36:07.000000000 +0800
+++ linux/mm/page_alloc.c	2009-07-13 09:25:21.000000000 +0800
@@ -475,6 +475,15 @@ static inline void __free_one_page(struc
 		order++;
 	}
 	set_page_order(page, order);
+
+	if (order >= pageblock_order && migratetype != MIGRATE_MOVABLE) {
+		int i;
+
+		migratetype = MIGRATE_MOVABLE;
+		for (i = 0; i < (1 << (order - pageblock_order)); i++)
+			set_pageblock_migratetype(page +
+				i * pageblock_nr_pages, MIGRATE_MOVABLE);
+	}
 	list_add(&page->lru,
 		&zone->free_area[order].free_list[migratetype]);
 	zone->free_area[order].nr_free++;

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] switch free memory back to MIGRATE_MOVABLE

[KOSAKI Motohiro]

> When page is back to buddy and its order is bigger than pageblock_order, we can
> switch its type to MIGRATE_MOVABLE. This can reduce fragmentation. The patch
> has obvious effect when read a block device and then drop caches.
> 
> Signed-off-by: Shaohua Li <shaohua.li@intel.com>

This patch change hot path, but there is no performance mesurement description.
Also, I don't like modification buddy core for only drop caches.



> ---
>  mm/page_alloc.c |    9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> Index: linux/mm/page_alloc.c
> ===================================================================
> --- linux.orig/mm/page_alloc.c	2009-07-10 11:36:07.000000000 +0800
> +++ linux/mm/page_alloc.c	2009-07-13 09:25:21.000000000 +0800
> @@ -475,6 +475,15 @@ static inline void __free_one_page(struc
>  		order++;
>  	}
>  	set_page_order(page, order);
> +
> +	if (order >= pageblock_order && migratetype != MIGRATE_MOVABLE) {
> +		int i;
> +
> +		migratetype = MIGRATE_MOVABLE;
> +		for (i = 0; i < (1 << (order - pageblock_order)); i++)
> +			set_pageblock_migratetype(page +
> +				i * pageblock_nr_pages, MIGRATE_MOVABLE);
> +	}
>  	list_add(&page->lru,
>  		&zone->free_area[order].free_list[migratetype]);
>  	zone->free_area[order].nr_free++;
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] switch free memory back to MIGRATE_MOVABLE

[KAMEZAWA Hiroyuki]

On Mon, 13 Jul 2009 11:47:46 +0900 (JST)
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:

> > When page is back to buddy and its order is bigger than pageblock_order, we can
> > switch its type to MIGRATE_MOVABLE. This can reduce fragmentation. The patch
> > has obvious effect when read a block device and then drop caches.
> > 
> > Signed-off-by: Shaohua Li <shaohua.li@intel.com>
> 
> This patch change hot path, but there is no performance mesurement description.
> Also, I don't like modification buddy core for only drop caches.
> 
Li, does this patch imply fallback of migration type doesn't work well ?
What is the bad case ?

-Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] switch free memory back to MIGRATE_MOVABLE

[Shaohua Li]

On Mon, Jul 13, 2009 at 10:58:03AM +0800, KAMEZAWA Hiroyuki wrote:
> On Mon, 13 Jul 2009 11:47:46 +0900 (JST)
> KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> 
> > > When page is back to buddy and its order is bigger than pageblock_order, we can
> > > switch its type to MIGRATE_MOVABLE. This can reduce fragmentation. The patch
> > > has obvious effect when read a block device and then drop caches.
> > > 
> > > Signed-off-by: Shaohua Li <shaohua.li@intel.com>
> > 
> > This patch change hot path, but there is no performance mesurement description.
> > Also, I don't like modification buddy core for only drop caches.
> > 
> Li, does this patch imply fallback of migration type doesn't work well ?
> What is the bad case ?
The page is initialized as migrate_movable, and then switch to reclaimable or
something else when fallback occurs, but its type remains even the page gets
freed. When the page gets freed, its type actually can be switch back to movable,
this is what the patch does.

Thanks,
Shaohua

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] switch free memory back to MIGRATE_MOVABLE

[KAMEZAWA Hiroyuki]

On Mon, 13 Jul 2009 11:04:44 +0800
Shaohua Li <shaohua.li@intel.com> wrote:

> On Mon, Jul 13, 2009 at 10:58:03AM +0800, KAMEZAWA Hiroyuki wrote:
> > On Mon, 13 Jul 2009 11:47:46 +0900 (JST)
> > KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> > 
> > > > When page is back to buddy and its order is bigger than pageblock_order, we can
> > > > switch its type to MIGRATE_MOVABLE. This can reduce fragmentation. The patch
> > > > has obvious effect when read a block device and then drop caches.
> > > > 
> > > > Signed-off-by: Shaohua Li <shaohua.li@intel.com>
> > > 
> > > This patch change hot path, but there is no performance mesurement description.
> > > Also, I don't like modification buddy core for only drop caches.
> > > 
> > Li, does this patch imply fallback of migration type doesn't work well ?
> > What is the bad case ?
> The page is initialized as migrate_movable, and then switch to reclaimable or
> something else when fallback occurs, but its type remains even the page gets
> freed. When the page gets freed, its type actually can be switch back to movable,
> this is what the patch does.
> 
Then, what is the benefits ? Changing this Movable here is better than fallback and
find this chunk again in lazy way ?

Thanks,
-Kame


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] switch free memory back to MIGRATE_MOVABLE

[KOSAKI Motohiro]

> On Mon, Jul 13, 2009 at 10:58:03AM +0800, KAMEZAWA Hiroyuki wrote:
> > On Mon, 13 Jul 2009 11:47:46 +0900 (JST)
> > KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> > 
> > > > When page is back to buddy and its order is bigger than pageblock_order, we can
> > > > switch its type to MIGRATE_MOVABLE. This can reduce fragmentation. The patch
> > > > has obvious effect when read a block device and then drop caches.
> > > > 
> > > > Signed-off-by: Shaohua Li <shaohua.li@intel.com>
> > > 
> > > This patch change hot path, but there is no performance mesurement description.
> > > Also, I don't like modification buddy core for only drop caches.
> > > 
> > Li, does this patch imply fallback of migration type doesn't work well ?
> > What is the bad case ?
> The page is initialized as migrate_movable, and then switch to reclaimable or
> something else when fallback occurs, but its type remains even the page gets
> freed. When the page gets freed, its type actually can be switch back to movable,
> this is what the patch does.

This answer is not actual answer.
Why do you think __rmqueue_fallback() doesn't works well? Do you have
any test-case or found a bug by review?





--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] switch free memory back to MIGRATE_MOVABLE

[Shaohua Li]

On Mon, Jul 13, 2009 at 11:08:14AM +0800, KOSAKI Motohiro wrote:
> > On Mon, Jul 13, 2009 at 10:58:03AM +0800, KAMEZAWA Hiroyuki wrote:
> > > On Mon, 13 Jul 2009 11:47:46 +0900 (JST)
> > > KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> > > 
> > > > > When page is back to buddy and its order is bigger than pageblock_order, we can
> > > > > switch its type to MIGRATE_MOVABLE. This can reduce fragmentation. The patch
> > > > > has obvious effect when read a block device and then drop caches.
> > > > > 
> > > > > Signed-off-by: Shaohua Li <shaohua.li@intel.com>
> > > > 
> > > > This patch change hot path, but there is no performance mesurement description.
> > > > Also, I don't like modification buddy core for only drop caches.
> > > > 
> > > Li, does this patch imply fallback of migration type doesn't work well ?
> > > What is the bad case ?
> > The page is initialized as migrate_movable, and then switch to reclaimable or
> > something else when fallback occurs, but its type remains even the page gets
> > freed. When the page gets freed, its type actually can be switch back to movable,
> > this is what the patch does.
> 
> This answer is not actual answer.
> Why do you think __rmqueue_fallback() doesn't works well? Do you have
> any test-case or found a bug by review?
I never said __rmqueue_fallback() doesn't work well. The page is already freed, switching
back the pageblock to movable might make next page allocation (non-movable) skip this
pageblock. So this could potentially reduce fragmentation and improve memory offline.
But your guys are right, I have no number if this will impact performance.

Thanks,
Shaohua

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] switch free memory back to MIGRATE_MOVABLE

[KAMEZAWA Hiroyuki]

On Mon, 13 Jul 2009 11:18:01 +0800
Shaohua Li <shaohua.li@intel.com> wrote:

> On Mon, Jul 13, 2009 at 11:08:14AM +0800, KOSAKI Motohiro wrote:
> > > On Mon, Jul 13, 2009 at 10:58:03AM +0800, KAMEZAWA Hiroyuki wrote:
> > > > On Mon, 13 Jul 2009 11:47:46 +0900 (JST)
> > > > KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> > > > 
> > > > > > When page is back to buddy and its order is bigger than pageblock_order, we can
> > > > > > switch its type to MIGRATE_MOVABLE. This can reduce fragmentation. The patch
> > > > > > has obvious effect when read a block device and then drop caches.
> > > > > > 
> > > > > > Signed-off-by: Shaohua Li <shaohua.li@intel.com>
> > > > > 
> > > > > This patch change hot path, but there is no performance mesurement description.
> > > > > Also, I don't like modification buddy core for only drop caches.
> > > > > 
> > > > Li, does this patch imply fallback of migration type doesn't work well ?
> > > > What is the bad case ?
> > > The page is initialized as migrate_movable, and then switch to reclaimable or
> > > something else when fallback occurs, but its type remains even the page gets
> > > freed. When the page gets freed, its type actually can be switch back to movable,
> > > this is what the patch does.
> > 
> > This answer is not actual answer.
> > Why do you think __rmqueue_fallback() doesn't works well? Do you have
> > any test-case or found a bug by review?
> I never said __rmqueue_fallback() doesn't work well. The page is already freed, switching
> back the pageblock to movable might make next page allocation (non-movable) skip this
> pageblock. So this could potentially reduce fragmentation and improve memory offline.
> But your guys are right, I have no number if this will impact performance.
> 
If this is for memory offlining, plz mention that at first ;)
IIUC, if this can be a problem, fixing memory offline itself is better. No ?
At implementing memory unplug, I had no problems because I assumes ZONE_MOVABLE.
But ok, I welcome enhances to memory unplug.

If this part is bad for you.
4714         /*
4715          * In future, more migrate types will be able to be isolation target.
4716          */
4717         if (get_pageblock_migratetype(page) != MIGRATE_MOVABLE)
4718                 goto out;

plz fix this to do more precise work for zid != ZONE_MOVABLE zones.
As I wrote in comments. My codes assumes ZONE_MOVABLE in many parts because I want
100%-success memory offline. 

Thanks,
-Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] switch free memory back to MIGRATE_MOVABLE

[Mel Gorman]

On Mon, Jul 13, 2009 at 10:30:30AM +0800, Shaohua Li wrote:
> When page is back to buddy and its order is bigger than pageblock_order, we can
> switch its type to MIGRATE_MOVABLE. This can reduce fragmentation. The patch
> has obvious effect when read a block device and then drop caches.
> 
> Signed-off-by: Shaohua Li <shaohua.li@intel.com>

NAK.

There is no point making this check in the free path, it can be left at
whatever type it is. rmqueue fallback will already find blocks like this and
switch the type again if necessary. The only time you might care is memory
off-lining and at that point, you can check if a free page spans the
pageblock and if so, ignore the existing migrate type.

> ---
>  mm/page_alloc.c |    9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> Index: linux/mm/page_alloc.c
> ===================================================================
> --- linux.orig/mm/page_alloc.c	2009-07-10 11:36:07.000000000 +0800
> +++ linux/mm/page_alloc.c	2009-07-13 09:25:21.000000000 +0800
> @@ -475,6 +475,15 @@ static inline void __free_one_page(struc
>  		order++;
>  	}
>  	set_page_order(page, order);
> +
> +	if (order >= pageblock_order && migratetype != MIGRATE_MOVABLE) {
> +		int i;
> +
> +		migratetype = MIGRATE_MOVABLE;
> +		for (i = 0; i < (1 << (order - pageblock_order)); i++)
> +			set_pageblock_migratetype(page +
> +				i * pageblock_nr_pages, MIGRATE_MOVABLE);
> +	}
>  	list_add(&page->lru,
>  		&zone->free_area[order].free_list[migratetype]);
>  	zone->free_area[order].nr_free++;
> 

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>