[RFC PATCH 1/3] mm, page_alloc: free HIGHATOMIC page directly to the allocator

[RFC PATCH 1/3] mm, page_alloc: free HIGHATOMIC page directly to the allocator

[Wenwei Tao]

From: Wenwei Tao <ww.tao0320@gmail.com>

Some pages might have already been allocated before reserve
the pageblock as HIGHATOMIC. When free these pages, put them
directly to the allocator instead of the pcp lists since they
might have the chance to be merged to high order pages.

Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com>
---
 mm/page_alloc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 6903b69..19f9e76 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2412,7 +2412,8 @@ void free_hot_cold_page(struct page *page, bool cold)
 	 * excessively into the page allocator
 	 */
 	if (migratetype >= MIGRATE_PCPTYPES) {
-		if (unlikely(is_migrate_isolate(migratetype))) {
+		if (unlikely(is_migrate_isolate(migratetype) ||
+				migratetype == MIGRATE_HIGHATOMIC)) {
 			free_one_page(zone, page, pfn, 0, migratetype);
 			goto out;
 		}
-- 
1.8.3.1


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 1/3] mm, page_alloc: free HIGHATOMIC page directly to the allocator

[Vlastimil Babka]

On 06/18/2016 11:34 AM, Wenwei Tao wrote:
> From: Wenwei Tao <ww.tao0320@gmail.com>
> 
> Some pages might have already been allocated before reserve
> the pageblock as HIGHATOMIC. When free these pages, put them
> directly to the allocator instead of the pcp lists since they
> might have the chance to be merged to high order pages.

Are there some data showing the improvement, or just theoretical?

> Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com>
> ---
>  mm/page_alloc.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 6903b69..19f9e76 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -2412,7 +2412,8 @@ void free_hot_cold_page(struct page *page, bool cold)

The full comment that's here for context:

/*
 * We only track unmovable, reclaimable and movable on pcp lists.
 * Free ISOLATE pages back to the allocator because they are being
 * offlined but treat RESERVE as movable pages so we can get those
 * areas back if necessary. Otherwise, we may have to free
 * excessively into the page allocator
 */

That comment looks outdated as it refers to RESERVE, which was replaced
by HIGHATOMIC. But there's some reasoning why these pages go to
pcplists. I'd expect the "free excessively" part isn't as bad as
highatomic reserves are quite limited. They also shouldn't be used for
order-0 allocations, which is what this function is about, so I would
expect both the impact on "free excessively" and the improvement of
merging to be minimal?

>  	 * excessively into the page allocator
>  	 */
>  	if (migratetype >= MIGRATE_PCPTYPES) {
> -		if (unlikely(is_migrate_isolate(migratetype))) {
> +		if (unlikely(is_migrate_isolate(migratetype) ||
> +				migratetype == MIGRATE_HIGHATOMIC)) {
>  			free_one_page(zone, page, pfn, 0, migratetype);
>  			goto out;
>  		}

In any case your patch highlighted that this code could be imho
optimized like below.

if (unlikely(migratetype >= MIGRATE_PCPTYPES))
   if (is_migrate_cma(migratetype)) {
       migratetype = MIGRATE_MOVABLE;
   } else {
       free_one_page(zone, page, pfn, 0, migratetype);
       goto out;
   }
}

That's less branches than your patch, and even less than originally if
CMA is not enabled (or with ZONE_CMA).

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC PATCH 2/3] mm, page_alloc: get page migreate type again when related to highatomic

[Wenwei Tao]

From: Wenwei Tao <ww.tao0320@gmail.com>

The migratetype might get staled, pages might have become highatomic when
we try to free them to the allocator, we might not want to put highatomic
pages into other buddy lists, since they are reserved only for atomic high
order use. And also highatomic pages could have been unreserved,
put them into the hightatomic buddy list might exceed the limit of
highatomic pages. So get the pages migreate type again to put them into
the right lists.

Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com>
---
 mm/page_alloc.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 19f9e76..b72b771 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1079,9 +1079,11 @@ static void free_pcppages_bulk(struct zone *zone, int count,
 	int batch_free = 0;
 	unsigned long nr_scanned;
 	bool isolated_pageblocks;
+	bool reserved_highatomic;
 
 	spin_lock(&zone->lock);
 	isolated_pageblocks = has_isolate_pageblock(zone);
+	reserved_highatomic = !!zone->nr_reserved_highatomic;
 	nr_scanned = zone_page_state(zone, NR_PAGES_SCANNED);
 	if (nr_scanned)
 		__mod_zone_page_state(zone, NR_PAGES_SCANNED, -nr_scanned);
@@ -1118,8 +1120,10 @@ static void free_pcppages_bulk(struct zone *zone, int count,
 			mt = get_pcppage_migratetype(page);
 			/* MIGRATE_ISOLATE page should not go to pcplists */
 			VM_BUG_ON_PAGE(is_migrate_isolate(mt), page);
+			VM_BUG_ON_PAGE(mt == MIGRATE_HIGHATOMIC, page);
 			/* Pageblock could have been isolated meanwhile */
-			if (unlikely(isolated_pageblocks))
+			if (unlikely(isolated_pageblocks ||
+					reserved_highatomic))
 				mt = get_pageblock_migratetype(page);
 
 			if (bulkfree_pcp_prepare(page))
@@ -1144,7 +1148,9 @@ static void free_one_page(struct zone *zone,
 		__mod_zone_page_state(zone, NR_PAGES_SCANNED, -nr_scanned);
 
 	if (unlikely(has_isolate_pageblock(zone) ||
-		is_migrate_isolate(migratetype))) {
+		zone->nr_reserved_highatomic ||
+		is_migrate_isolate(migratetype) ||
+		migratetype == MIGRATE_HIGHATOMIC)) {
 		migratetype = get_pfnblock_migratetype(page, pfn);
 	}
 	__free_one_page(page, pfn, zone, order, migratetype);
-- 
1.8.3.1


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC PATCH 3/3] mm, page_alloc: prevent merge freepages between highatomic and others

[Wenwei Tao]

From: Wenwei Tao <ww.tao0320@gmail.com>

We might not want other migrate types pin highatomic pageblock
away since it's reserved for high order use. And also we might
not want reserve high atomic pages out of limit, we can add
check in __free_one_page but this might be costly, so just stop
the merging.

Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com>
---
 mm/page_alloc.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index b72b771..ffce9b5 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -832,7 +832,8 @@ continue_merging:
 		 * We don't want to hit this code for the more frequent
 		 * low-order merging.
 		 */
-		if (unlikely(has_isolate_pageblock(zone))) {
+		if (unlikely(has_isolate_pageblock(zone) ||
+				zone->nr_reserved_highatomic)) {
 			int buddy_mt;
 
 			buddy_idx = __find_buddy_index(page_idx, order);
@@ -841,7 +842,9 @@ continue_merging:
 
 			if (migratetype != buddy_mt
 					&& (is_migrate_isolate(migratetype) ||
-						is_migrate_isolate(buddy_mt)))
+					is_migrate_isolate(buddy_mt) ||
+					migratetype == MIGRATE_HIGHATOMIC ||
+					buddy_mt == MIGRATE_HIGHATOMIC))
 				goto done_merging;
 		}
 		max_order++;
-- 
1.8.3.1


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Fwd: [RFC PATCH 1/3] mm, page_alloc: free HIGHATOMIC page directly to the allocator

[Wenwei Tao]

Hi,
The original message is somehow determined to be junk mail and
rejected by the system.
Forward this message.

---------- Forwarded message ----------
From: Wenwei Tao <ww.tao0320@gmail.com>
Date: 2016-06-19 10:40 GMT+08:00
Subject: Re: [RFC PATCH 1/3] mm, page_alloc: free HIGHATOMIC page
directly to the allocator
To: Vlastimil Babka <vbabka@suse.cz>
抄送： Wenwei Tao <wwtao0320@163.com>, akpm@linux-foundation.org,
mgorman@techsingularity.net, mhocko@suse.com, rientjes@google.com,
kirill.shutemov@linux.intel.com, iamjoonsoo.kim@lge.com,
izumi.taku@jp.fujitsu.com, Johannes Weiner <hannes@cmpxchg.org>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org


2016-06-18 18:14 GMT+08:00 Vlastimil Babka <vbabka@suse.cz>:
> On 06/18/2016 11:34 AM, Wenwei Tao wrote:
>> From: Wenwei Tao <ww.tao0320@gmail.com>
>>
>> Some pages might have already been allocated before reserve
>> the pageblock as HIGHATOMIC. When free these pages, put them
>> directly to the allocator instead of the pcp lists since they
>> might have the chance to be merged to high order pages.
>
> Are there some data showing the improvement, or just theoretical?
>

It's just theoretical. I read the mm code and try to understand it,
think this might be an optimization.

>> Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com>
>> ---
>>  mm/page_alloc.c | 3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index 6903b69..19f9e76 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -2412,7 +2412,8 @@ void free_hot_cold_page(struct page *page, bool cold)
>
> The full comment that's here for context:
>
> /*
>  * We only track unmovable, reclaimable and movable on pcp lists.
>  * Free ISOLATE pages back to the allocator because they are being
>  * offlined but treat RESERVE as movable pages so we can get those
>  * areas back if necessary. Otherwise, we may have to free
>  * excessively into the page allocator
>  */
>
> That comment looks outdated as it refers to RESERVE, which was replaced
> by HIGHATOMIC. But there's some reasoning why these pages go to
> pcplists. I'd expect the "free excessively" part isn't as bad as
> highatomic reserves are quite limited. They also shouldn't be used for
> order-0 allocations, which is what this function is about, so I would
> expect both the impact on "free excessively" and the improvement of
> merging to be minimal?
>
>>        * excessively into the page allocator
>>        */
>>       if (migratetype >= MIGRATE_PCPTYPES) {
>> -             if (unlikely(is_migrate_isolate(migratetype))) {
>> +             if (unlikely(is_migrate_isolate(migratetype) ||
>> +                             migratetype == MIGRATE_HIGHATOMIC)) {
>>                       free_one_page(zone, page, pfn, 0, migratetype);
>>                       goto out;
>>               }
>
> In any case your patch highlighted that this code could be imho
> optimized like below.
>
> if (unlikely(migratetype >= MIGRATE_PCPTYPES))
>    if (is_migrate_cma(migratetype)) {
>        migratetype = MIGRATE_MOVABLE;
>    } else {
>        free_one_page(zone, page, pfn, 0, migratetype);
>        goto out;
>    }
> }
>
> That's less branches than your patch, and even less than originally if
> CMA is not enabled (or with ZONE_CMA).

Yeah, this looks better.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>