[PATCH v3 0/5] cma: fix watermark checking

[PATCH v3 0/5] cma: fix watermark checking

[Bartlomiej Zolnierkiewicz]

Free pages belonging to Contiguous Memory Allocator (CMA) areas cannot be
used by unmovable allocations and this fact should be accounted for while
doing zone watermark checking.  Additionaly while CMA pages are isolated
they shouldn't be included in the total number of free pages (as they
cannot be allocated while they are isolated).  The following patch series
should fix both issues.  It is based on top of recent Minchan's CMA series
(https://lkml.org/lkml/2012/8/14/81 "[RFC 0/2] Reduce alloc_contig_range
latency").

v2:
- no need to call get_pageblock_migratetype() in free_one_page() in patch #1
  (thanks to review from Michal Nazarewicz)
- fix issues pointed in http://www.spinics.net/lists/linux-mm/msg41017.html
  in patch #2 (ditto)
- remove no longer needed is_cma_pageblock() from patch #2

v3:
- fix tracing in free_pcppages_bulk()
- fix counting of free CMA pages (broken by v2)


Bartlomiej Zolnierkiewicz (4):
  mm: fix tracing in free_pcppages_bulk()
  cma: fix counting of isolated pages
  cma: count free CMA pages
  cma: fix watermark checking

Marek Szyprowski (1):
  mm: add accounting for CMA pages and use them for watermark
    calculation

 include/linux/mmzone.h |  3 +-
 mm/compaction.c        | 11 ++++----
 mm/page_alloc.c        | 77 +++++++++++++++++++++++++++++++++++++++-----------
 mm/page_isolation.c    | 20 +++++++++++--
 mm/vmscan.c            |  4 +--
 mm/vmstat.c            |  1 +
 6 files changed, 89 insertions(+), 27 deletions(-)

-- 
1.7.11.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v3 1/5] mm: fix tracing in free_pcppages_bulk()

[Bartlomiej Zolnierkiewicz]

page->private gets re-used in __free_one_page() to store page order
so migratetype value must be cached locally.

Fixes regression introduced in a701623 ("mm: fix migratetype bug
which slowed swapping").

Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Michal Nazarewicz <mina86@mina86.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Hugh Dickins <hughd@google.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
---
 mm/page_alloc.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 93a3433..e9da55c 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -668,12 +668,15 @@ static void free_pcppages_bulk(struct zone *zone, int count,
 			batch_free = to_free;
 
 		do {
+			int mt;
+
 			page = list_entry(list->prev, struct page, lru);
 			/* must delete as __free_one_page list manipulates */
 			list_del(&page->lru);
+			mt = page_private(page);
 			/* MIGRATE_MOVABLE list may include MIGRATE_RESERVEs */
-			__free_one_page(page, zone, 0, page_private(page));
-			trace_mm_page_pcpu_drain(page, 0, page_private(page));
+			__free_one_page(page, zone, 0, mt);
+			trace_mm_page_pcpu_drain(page, 0, mt);
 		} while (--to_free && --batch_free && !list_empty(list));
 	}
 	__mod_zone_page_state(zone, NR_FREE_PAGES, count);
-- 
1.7.11.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v3 2/5] cma: fix counting of isolated pages

[Bartlomiej Zolnierkiewicz]

Isolated free pages shouldn't be accounted to NR_FREE_PAGES counter.
Fix it by properly decreasing/increasing NR_FREE_PAGES counter in
set_migratetype_isolate()/unset_migratetype_isolate() and removing
counter adjustment for isolated pages from free_one_page() and
split_free_page().

Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Michal Nazarewicz <mina86@mina86.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Hugh Dickins <hughd@google.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
---
 mm/page_alloc.c     |  7 +++++--
 mm/page_isolation.c | 13 ++++++++++---
 2 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index e9da55c..3acdf0f 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -691,7 +691,8 @@ static void free_one_page(struct zone *zone, struct page *page, int order,
 	zone->pages_scanned = 0;
 
 	__free_one_page(page, zone, order, migratetype);
-	__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
+	if (migratetype != MIGRATE_ISOLATE)
+		__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
 	spin_unlock(&zone->lock);
 }
 
@@ -1414,7 +1415,9 @@ int split_free_page(struct page *page, bool check_wmark)
 	list_del(&page->lru);
 	zone->free_area[order].nr_free--;
 	rmv_page_order(page);
-	__mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
+
+	if (get_pageblock_migratetype(page) != MIGRATE_ISOLATE)
+		__mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
 
 	/* Split into individual pages */
 	set_page_refcounted(page);
diff --git a/mm/page_isolation.c b/mm/page_isolation.c
index 247d1f1..d210cc8 100644
--- a/mm/page_isolation.c
+++ b/mm/page_isolation.c
@@ -76,8 +76,12 @@ int set_migratetype_isolate(struct page *page)
 
 out:
 	if (!ret) {
+		unsigned long nr_pages;
+
 		set_pageblock_isolate(page);
-		move_freepages_block(zone, page, MIGRATE_ISOLATE);
+		nr_pages = move_freepages_block(zone, page, MIGRATE_ISOLATE);
+
+		__mod_zone_page_state(zone, NR_FREE_PAGES, -nr_pages);
 	}
 
 	spin_unlock_irqrestore(&zone->lock, flags);
@@ -89,12 +93,15 @@ out:
 void unset_migratetype_isolate(struct page *page, unsigned migratetype)
 {
 	struct zone *zone;
-	unsigned long flags;
+	unsigned long flags, nr_pages;
+
 	zone = page_zone(page);
+
 	spin_lock_irqsave(&zone->lock, flags);
 	if (get_pageblock_migratetype(page) != MIGRATE_ISOLATE)
 		goto out;
-	move_freepages_block(zone, page, migratetype);
+	nr_pages = move_freepages_block(zone, page, migratetype);
+	__mod_zone_page_state(zone, NR_FREE_PAGES, nr_pages);
 	restore_pageblock_isolate(page, migratetype);
 out:
 	spin_unlock_irqrestore(&zone->lock, flags);
-- 
1.7.11.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v3 3/5] cma: count free CMA pages

[Bartlomiej Zolnierkiewicz]

Add NR_FREE_CMA_PAGES counter to be later used for checking watermark
in __zone_watermark_ok().  For simplicity and to avoid #ifdef hell make
this counter always available (not only when CONFIG_CMA=y).

Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Michal Nazarewicz <mina86@mina86.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Hugh Dickins <hughd@google.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
---
 include/linux/mmzone.h |  1 +
 mm/page_alloc.c        | 36 ++++++++++++++++++++++++++++++++----
 mm/page_isolation.c    |  7 +++++++
 mm/vmstat.c            |  1 +
 4 files changed, 41 insertions(+), 4 deletions(-)

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index ca034a1..904889d 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -140,6 +140,7 @@ enum zone_stat_item {
 	NUMA_OTHER,		/* allocation from other node */
 #endif
 	NR_ANON_TRANSPARENT_HUGEPAGES,
+	NR_FREE_CMA_PAGES,
 	NR_VM_ZONE_STAT_ITEMS };
 
 /*
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 3acdf0f..5bb0cda 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -559,6 +559,9 @@ static inline void __free_one_page(struct page *page,
 			clear_page_guard_flag(buddy);
 			set_page_private(page, 0);
 			__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
+			if (is_migrate_cma(migratetype))
+				__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
+						      1 << order);
 		} else {
 			list_del(&buddy->lru);
 			zone->free_area[order].nr_free--;
@@ -677,6 +680,8 @@ static void free_pcppages_bulk(struct zone *zone, int count,
 			/* MIGRATE_MOVABLE list may include MIGRATE_RESERVEs */
 			__free_one_page(page, zone, 0, mt);
 			trace_mm_page_pcpu_drain(page, 0, mt);
+			if (is_migrate_cma(mt))
+				__mod_zone_page_state(zone, NR_FREE_CMA_PAGES, 1);
 		} while (--to_free && --batch_free && !list_empty(list));
 	}
 	__mod_zone_page_state(zone, NR_FREE_PAGES, count);
@@ -691,8 +696,12 @@ static void free_one_page(struct zone *zone, struct page *page, int order,
 	zone->pages_scanned = 0;
 
 	__free_one_page(page, zone, order, migratetype);
-	if (migratetype != MIGRATE_ISOLATE)
+	if (migratetype != MIGRATE_ISOLATE) {
 		__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
+		if (is_migrate_cma(migratetype))
+			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
+					      1 << order);
+	}
 	spin_unlock(&zone->lock);
 }
 
@@ -816,6 +825,9 @@ static inline void expand(struct zone *zone, struct page *page,
 			set_page_private(&page[size], high);
 			/* Guard pages are not available for any usage */
 			__mod_zone_page_state(zone, NR_FREE_PAGES, -(1 << high));
+			if (is_migrate_cma(migratetype))
+				__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
+						      -(1 << high));
 			continue;
 		}
 #endif
@@ -1141,6 +1153,9 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order,
 		}
 		set_page_private(page, mt);
 		list = &page->lru;
+		if (is_migrate_cma(mt))
+			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
+					      -(1 << order));
 	}
 	__mod_zone_page_state(zone, NR_FREE_PAGES, -(i << order));
 	spin_unlock(&zone->lock);
@@ -1398,6 +1413,7 @@ int split_free_page(struct page *page, bool check_wmark)
 	unsigned int order;
 	unsigned long watermark;
 	struct zone *zone;
+	int mt;
 
 	BUG_ON(!PageBuddy(page));
 
@@ -1416,8 +1432,13 @@ int split_free_page(struct page *page, bool check_wmark)
 	zone->free_area[order].nr_free--;
 	rmv_page_order(page);
 
-	if (get_pageblock_migratetype(page) != MIGRATE_ISOLATE)
+	mt = get_pageblock_migratetype(page);
+	if (mt != MIGRATE_ISOLATE) {
 		__mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
+		if (is_migrate_cma(mt))
+			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
+					      -(1UL << order));
+	}
 
 	/* Split into individual pages */
 	set_page_refcounted(page);
@@ -1492,6 +1513,9 @@ again:
 		spin_unlock(&zone->lock);
 		if (!page)
 			goto failed;
+		if (is_migrate_cma(get_pageblock_migratetype(page)))
+			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
+					      -(1 << order));
 		__mod_zone_page_state(zone, NR_FREE_PAGES, -(1 << order));
 	}
 
@@ -2860,7 +2884,8 @@ void show_free_areas(unsigned int filter)
 		" unevictable:%lu"
 		" dirty:%lu writeback:%lu unstable:%lu\n"
 		" free:%lu slab_reclaimable:%lu slab_unreclaimable:%lu\n"
-		" mapped:%lu shmem:%lu pagetables:%lu bounce:%lu\n",
+		" mapped:%lu shmem:%lu pagetables:%lu bounce:%lu\n"
+		" free_cma:%lu\n",
 		global_page_state(NR_ACTIVE_ANON),
 		global_page_state(NR_INACTIVE_ANON),
 		global_page_state(NR_ISOLATED_ANON),
@@ -2877,7 +2902,8 @@ void show_free_areas(unsigned int filter)
 		global_page_state(NR_FILE_MAPPED),
 		global_page_state(NR_SHMEM),
 		global_page_state(NR_PAGETABLE),
-		global_page_state(NR_BOUNCE));
+		global_page_state(NR_BOUNCE),
+		global_page_state(NR_FREE_CMA_PAGES));
 
 	for_each_populated_zone(zone) {
 		int i;
@@ -2909,6 +2935,7 @@ void show_free_areas(unsigned int filter)
 			" pagetables:%lukB"
 			" unstable:%lukB"
 			" bounce:%lukB"
+			" free_cma:%lukB"
 			" writeback_tmp:%lukB"
 			" pages_scanned:%lu"
 			" all_unreclaimable? %s"
@@ -2938,6 +2965,7 @@ void show_free_areas(unsigned int filter)
 			K(zone_page_state(zone, NR_PAGETABLE)),
 			K(zone_page_state(zone, NR_UNSTABLE_NFS)),
 			K(zone_page_state(zone, NR_BOUNCE)),
+			K(zone_page_state(zone, NR_FREE_CMA_PAGES)),
 			K(zone_page_state(zone, NR_WRITEBACK_TEMP)),
 			zone->pages_scanned,
 			(zone->all_unreclaimable ? "yes" : "no")
diff --git a/mm/page_isolation.c b/mm/page_isolation.c
index d210cc8..6ead34d 100644
--- a/mm/page_isolation.c
+++ b/mm/page_isolation.c
@@ -77,11 +77,15 @@ int set_migratetype_isolate(struct page *page)
 out:
 	if (!ret) {
 		unsigned long nr_pages;
+		int mt = get_pageblock_migratetype(page);
 
 		set_pageblock_isolate(page);
 		nr_pages = move_freepages_block(zone, page, MIGRATE_ISOLATE);
 
 		__mod_zone_page_state(zone, NR_FREE_PAGES, -nr_pages);
+		if (is_migrate_cma(mt))
+			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
+					      -nr_pages);
 	}
 
 	spin_unlock_irqrestore(&zone->lock, flags);
@@ -102,6 +106,9 @@ void unset_migratetype_isolate(struct page *page, unsigned migratetype)
 		goto out;
 	nr_pages = move_freepages_block(zone, page, migratetype);
 	__mod_zone_page_state(zone, NR_FREE_PAGES, nr_pages);
+	if (is_migrate_cma(migratetype))
+		__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
+				      nr_pages);
 	restore_pageblock_isolate(page, migratetype);
 out:
 	spin_unlock_irqrestore(&zone->lock, flags);
diff --git a/mm/vmstat.c b/mm/vmstat.c
index df7a674..7c102e6 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -722,6 +722,7 @@ const char * const vmstat_text[] = {
 	"numa_other",
 #endif
 	"nr_anon_transparent_hugepages",
+	"nr_free_cma",
 	"nr_dirty_threshold",
 	"nr_dirty_background_threshold",
 
-- 
1.7.11.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v3 4/5] mm: add accounting for CMA pages and use them for watermark calculation

[Bartlomiej Zolnierkiewicz]

From: Marek Szyprowski <m.szyprowski@samsung.com>

During watermark check we need to decrease available free pages number
by free CMA pages number because unmovable allocations cannot use pages
from CMA areas.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Michal Nazarewicz <mina86@mina86.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Hugh Dickins <hughd@google.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
---
 mm/page_alloc.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 5bb0cda..2166774 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1628,7 +1628,7 @@ static inline bool should_fail_alloc_page(gfp_t gfp_mask, unsigned int order)
  * of the allocation.
  */
 static bool __zone_watermark_ok(struct zone *z, int order, unsigned long mark,
-		      int classzone_idx, int alloc_flags, long free_pages)
+		      int classzone_idx, int alloc_flags, long free_pages, long free_cma_pages)
 {
 	/* free_pages my go negative - that's OK */
 	long min = mark;
@@ -1641,7 +1641,7 @@ static bool __zone_watermark_ok(struct zone *z, int order, unsigned long mark,
 	if (alloc_flags & ALLOC_HARDER)
 		min -= min / 4;
 
-	if (free_pages <= min + lowmem_reserve)
+	if (free_pages - free_cma_pages <= min + lowmem_reserve)
 		return false;
 	for (o = 0; o < order; o++) {
 		/* At the next order, this order's pages become unavailable */
@@ -1674,13 +1674,15 @@ bool zone_watermark_ok(struct zone *z, int order, unsigned long mark,
 		      int classzone_idx, int alloc_flags)
 {
 	return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags,
-					zone_page_state(z, NR_FREE_PAGES));
+					zone_page_state(z, NR_FREE_PAGES),
+					zone_page_state(z, NR_FREE_CMA_PAGES));
 }
 
 bool zone_watermark_ok_safe(struct zone *z, int order, unsigned long mark,
 		      int classzone_idx, int alloc_flags)
 {
 	long free_pages = zone_page_state(z, NR_FREE_PAGES);
+	long free_cma_pages = zone_page_state(z, NR_FREE_CMA_PAGES);
 
 	if (z->percpu_drift_mark && free_pages < z->percpu_drift_mark)
 		free_pages = zone_page_state_snapshot(z, NR_FREE_PAGES);
@@ -1694,7 +1696,7 @@ bool zone_watermark_ok_safe(struct zone *z, int order, unsigned long mark,
 	 */
 	free_pages -= nr_zone_isolate_freepages(z);
 	return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags,
-								free_pages);
+					free_pages, free_cma_pages);
 }
 
 #ifdef CONFIG_NUMA
-- 
1.7.11.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v3 5/5] cma: fix watermark checking

[Bartlomiej Zolnierkiewicz]

Pass GFP flags to [__]zone_watermark_ok() and use them to account
free CMA pages only when necessary (there is no need to check
watermark against only non-CMA free pages for movable allocations).

Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Michal Nazarewicz <mina86@mina86.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Hugh Dickins <hughd@google.com>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
---
 include/linux/mmzone.h |  2 +-
 mm/compaction.c        | 11 ++++++-----
 mm/page_alloc.c        | 29 +++++++++++++++++++----------
 mm/vmscan.c            |  4 ++--
 4 files changed, 28 insertions(+), 18 deletions(-)

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 904889d..308bb91 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -725,7 +725,7 @@ extern struct mutex zonelists_mutex;
 void build_all_zonelists(pg_data_t *pgdat, struct zone *zone);
 void wakeup_kswapd(struct zone *zone, int order, enum zone_type classzone_idx);
 bool zone_watermark_ok(struct zone *z, int order, unsigned long mark,
-		int classzone_idx, int alloc_flags);
+		int classzone_idx, int alloc_flags, gfp_t gfp_flags);
 bool zone_watermark_ok_safe(struct zone *z, int order, unsigned long mark,
 		int classzone_idx, int alloc_flags);
 enum memmap_context {
diff --git a/mm/compaction.c b/mm/compaction.c
index 4b902aa..080175a 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -684,7 +684,7 @@ static int compact_finished(struct zone *zone,
 	watermark = low_wmark_pages(zone);
 	watermark += (1 << cc->order);
 
-	if (!zone_watermark_ok(zone, cc->order, watermark, 0, 0))
+	if (!zone_watermark_ok(zone, cc->order, watermark, 0, 0, 0))
 		return COMPACT_CONTINUE;
 
 	/* Direct compactor: Is a suitable page free? */
@@ -726,7 +726,7 @@ unsigned long compaction_suitable(struct zone *zone, int order)
 	 * allocated and for a short time, the footprint is higher
 	 */
 	watermark = low_wmark_pages(zone) + (2UL << order);
-	if (!zone_watermark_ok(zone, 0, watermark, 0, 0))
+	if (!zone_watermark_ok(zone, 0, watermark, 0, 0, 0))
 		return COMPACT_SKIPPED;
 
 	/*
@@ -745,7 +745,7 @@ unsigned long compaction_suitable(struct zone *zone, int order)
 		return COMPACT_SKIPPED;
 
 	if (fragindex == -1000 && zone_watermark_ok(zone, order, watermark,
-	    0, 0))
+	    0, 0, 0))
 		return COMPACT_PARTIAL;
 
 	return COMPACT_CONTINUE;
@@ -889,7 +889,8 @@ unsigned long try_to_compact_pages(struct zonelist *zonelist,
 		rc = max(status, rc);
 
 		/* If a normal allocation would succeed, stop compacting */
-		if (zone_watermark_ok(zone, order, low_wmark_pages(zone), 0, 0))
+		if (zone_watermark_ok(zone, order, low_wmark_pages(zone), 0, 0,
+				      gfp_mask))
 			break;
 	}
 
@@ -920,7 +921,7 @@ static int __compact_pgdat(pg_data_t *pgdat, struct compact_control *cc)
 
 		if (cc->order > 0) {
 			int ok = zone_watermark_ok(zone, cc->order,
-						low_wmark_pages(zone), 0, 0);
+						low_wmark_pages(zone), 0, 0, 0);
 			if (ok && cc->order >= zone->compact_order_failed)
 				zone->compact_order_failed = cc->order + 1;
 			/* Currently async compaction is never deferred. */
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 2166774..5912a8c 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1423,7 +1423,7 @@ int split_free_page(struct page *page, bool check_wmark)
 	if (check_wmark) {
 		/* Obey watermarks as if the page was being allocated */
 		watermark = low_wmark_pages(zone) + (1 << order);
-		if (!zone_watermark_ok(zone, 0, watermark, 0, 0))
+		if (!zone_watermark_ok(zone, 0, watermark, 0, 0, 0))
 			return 0;
 	}
 
@@ -1628,12 +1628,13 @@ static inline bool should_fail_alloc_page(gfp_t gfp_mask, unsigned int order)
  * of the allocation.
  */
 static bool __zone_watermark_ok(struct zone *z, int order, unsigned long mark,
-		      int classzone_idx, int alloc_flags, long free_pages, long free_cma_pages)
+		      int classzone_idx, int alloc_flags, long free_pages,
+		      long free_cma_pages, gfp_t gfp_flags)
 {
 	/* free_pages my go negative - that's OK */
 	long min = mark;
 	long lowmem_reserve = z->lowmem_reserve[classzone_idx];
-	int o;
+	int mt, o;
 
 	free_pages -= (1 << order) - 1;
 	if (alloc_flags & ALLOC_HIGH)
@@ -1641,8 +1642,14 @@ static bool __zone_watermark_ok(struct zone *z, int order, unsigned long mark,
 	if (alloc_flags & ALLOC_HARDER)
 		min -= min / 4;
 
-	if (free_pages - free_cma_pages <= min + lowmem_reserve)
-		return false;
+	mt = allocflags_to_migratetype(gfp_flags);
+	if (mt == MIGRATE_MOVABLE) {
+		if (free_pages <= min + lowmem_reserve)
+			return false;
+	} else {
+		if (free_pages - free_cma_pages <= min + lowmem_reserve)
+			return false;
+	}
 	for (o = 0; o < order; o++) {
 		/* At the next order, this order's pages become unavailable */
 		free_pages -= z->free_area[o].nr_free << o;
@@ -1671,11 +1678,12 @@ static inline unsigned long nr_zone_isolate_freepages(struct zone *zone)
 #endif
 
 bool zone_watermark_ok(struct zone *z, int order, unsigned long mark,
-		      int classzone_idx, int alloc_flags)
+		      int classzone_idx, int alloc_flags, gfp_t gfp_flags)
 {
 	return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags,
 					zone_page_state(z, NR_FREE_PAGES),
-					zone_page_state(z, NR_FREE_CMA_PAGES));
+					zone_page_state(z, NR_FREE_CMA_PAGES),
+					gfp_flags);
 }
 
 bool zone_watermark_ok_safe(struct zone *z, int order, unsigned long mark,
@@ -1696,7 +1704,7 @@ bool zone_watermark_ok_safe(struct zone *z, int order, unsigned long mark,
 	 */
 	free_pages -= nr_zone_isolate_freepages(z);
 	return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags,
-					free_pages, free_cma_pages);
+					free_pages, free_cma_pages, 0);
 }
 
 #ifdef CONFIG_NUMA
@@ -1906,7 +1914,7 @@ zonelist_scan:
 
 			mark = zone->watermark[alloc_flags & ALLOC_WMARK_MASK];
 			if (zone_watermark_ok(zone, order, mark,
-				    classzone_idx, alloc_flags))
+				    classzone_idx, alloc_flags, gfp_mask))
 				goto try_this_zone;
 
 			if (NUMA_BUILD && !did_zlc_setup && nr_online_nodes > 1) {
@@ -1942,7 +1950,8 @@ zonelist_scan:
 			default:
 				/* did we reclaim enough */
 				if (!zone_watermark_ok(zone, order, mark,
-						classzone_idx, alloc_flags))
+						classzone_idx, alloc_flags,
+						gfp_mask))
 					goto this_zone_full;
 			}
 		}
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 8d01243..4a10038b 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2777,14 +2777,14 @@ out:
 
 			/* Confirm the zone is balanced for order-0 */
 			if (!zone_watermark_ok(zone, 0,
-					high_wmark_pages(zone), 0, 0)) {
+					high_wmark_pages(zone), 0, 0, 0)) {
 				order = sc.order = 0;
 				goto loop_again;
 			}
 
 			/* Check if the memory needs to be defragmented. */
 			if (zone_watermark_ok(zone, order,
-				    low_wmark_pages(zone), *classzone_idx, 0))
+				    low_wmark_pages(zone), *classzone_idx, 0, 0))
 				zones_need_compaction = 0;
 
 			/* If balanced, clear the congested flag */
-- 
1.7.11.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v3 1/5] mm: fix tracing in free_pcppages_bulk()

[Mel Gorman]

On Tue, Sep 04, 2012 at 03:26:21PM +0200, Bartlomiej Zolnierkiewicz wrote:
> page->private gets re-used in __free_one_page() to store page order
> so migratetype value must be cached locally.
> 
> Fixes regression introduced in a701623 ("mm: fix migratetype bug
> which slowed swapping").
> 

This is unrelated to the rest of the series and should be sent on its
own but otherwise.

Acked-by: Mel Gorman <mgorman@suse.de>

-- 
Mel Gorman
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v3 2/5] cma: fix counting of isolated pages

[Mel Gorman]

On Tue, Sep 04, 2012 at 03:26:22PM +0200, Bartlomiej Zolnierkiewicz wrote:
> Isolated free pages shouldn't be accounted to NR_FREE_PAGES counter.
> Fix it by properly decreasing/increasing NR_FREE_PAGES counter in
> set_migratetype_isolate()/unset_migratetype_isolate() and removing
> counter adjustment for isolated pages from free_one_page() and
> split_free_page().
> 
> Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> Cc: Michal Nazarewicz <mina86@mina86.com>
> Cc: Minchan Kim <minchan@kernel.org>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Hugh Dickins <hughd@google.com>
> Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
> Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
> ---
>  mm/page_alloc.c     |  7 +++++--
>  mm/page_isolation.c | 13 ++++++++++---
>  2 files changed, 15 insertions(+), 5 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index e9da55c..3acdf0f 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -691,7 +691,8 @@ static void free_one_page(struct zone *zone, struct page *page, int order,
>  	zone->pages_scanned = 0;
>  
>  	__free_one_page(page, zone, order, migratetype);
> -	__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
> +	if (migratetype != MIGRATE_ISOLATE)
> +		__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
>  	spin_unlock(&zone->lock);
>  }
>  
> @@ -1414,7 +1415,9 @@ int split_free_page(struct page *page, bool check_wmark)
>  	list_del(&page->lru);
>  	zone->free_area[order].nr_free--;
>  	rmv_page_order(page);
> -	__mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
> +
> +	if (get_pageblock_migratetype(page) != MIGRATE_ISOLATE)
> +		__mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
>  

Are you *sure* about this part? The page is already free so the
NR_FREE_PAGES counters should already be correct. It feels to me that it
should be the caller that fixes up NR_FREE_PAGES if necessary.

Have you tested this with THP? I have a suspicion that the free page
accounting gets broken when page migration is used there.

>  	/* Split into individual pages */
>  	set_page_refcounted(page);
> diff --git a/mm/page_isolation.c b/mm/page_isolation.c
> index 247d1f1..d210cc8 100644
> --- a/mm/page_isolation.c
> +++ b/mm/page_isolation.c
> @@ -76,8 +76,12 @@ int set_migratetype_isolate(struct page *page)
>  
>  out:
>  	if (!ret) {
> +		unsigned long nr_pages;
> +
>  		set_pageblock_isolate(page);
> -		move_freepages_block(zone, page, MIGRATE_ISOLATE);
> +		nr_pages = move_freepages_block(zone, page, MIGRATE_ISOLATE);
> +
> +		__mod_zone_page_state(zone, NR_FREE_PAGES, -nr_pages);
>  	}
>  
>  	spin_unlock_irqrestore(&zone->lock, flags);
> @@ -89,12 +93,15 @@ out:
>  void unset_migratetype_isolate(struct page *page, unsigned migratetype)
>  {
>  	struct zone *zone;
> -	unsigned long flags;
> +	unsigned long flags, nr_pages;
> +
>  	zone = page_zone(page);
> +

unnecessary whitespace change.

>  	spin_lock_irqsave(&zone->lock, flags);
>  	if (get_pageblock_migratetype(page) != MIGRATE_ISOLATE)
>  		goto out;
> -	move_freepages_block(zone, page, migratetype);
> +	nr_pages = move_freepages_block(zone, page, migratetype);
> +	__mod_zone_page_state(zone, NR_FREE_PAGES, nr_pages);
>  	restore_pageblock_isolate(page, migratetype);
>  out:
>  	spin_unlock_irqrestore(&zone->lock, flags);
> -- 
> 1.7.11.3
> 

-- 
Mel Gorman
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v3 2/5] cma: fix counting of isolated pages

[Bartlomiej Zolnierkiewicz]

On Wednesday 05 September 2012 13:08:47 Mel Gorman wrote:
> On Tue, Sep 04, 2012 at 03:26:22PM +0200, Bartlomiej Zolnierkiewicz wrote:
> > Isolated free pages shouldn't be accounted to NR_FREE_PAGES counter.
> > Fix it by properly decreasing/increasing NR_FREE_PAGES counter in
> > set_migratetype_isolate()/unset_migratetype_isolate() and removing
> > counter adjustment for isolated pages from free_one_page() and
> > split_free_page().
> > 
> > Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> > Cc: Michal Nazarewicz <mina86@mina86.com>
> > Cc: Minchan Kim <minchan@kernel.org>
> > Cc: Mel Gorman <mgorman@suse.de>
> > Cc: Hugh Dickins <hughd@google.com>
> > Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
> > Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
> > ---
> >  mm/page_alloc.c     |  7 +++++--
> >  mm/page_isolation.c | 13 ++++++++++---
> >  2 files changed, 15 insertions(+), 5 deletions(-)
> > 
> > diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> > index e9da55c..3acdf0f 100644
> > --- a/mm/page_alloc.c
> > +++ b/mm/page_alloc.c
> > @@ -691,7 +691,8 @@ static void free_one_page(struct zone *zone, struct page *page, int order,
> >  	zone->pages_scanned = 0;
> >  
> >  	__free_one_page(page, zone, order, migratetype);
> > -	__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
> > +	if (migratetype != MIGRATE_ISOLATE)
> > +		__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
> >  	spin_unlock(&zone->lock);
> >  }
> >  
> > @@ -1414,7 +1415,9 @@ int split_free_page(struct page *page, bool check_wmark)
> >  	list_del(&page->lru);
> >  	zone->free_area[order].nr_free--;
> >  	rmv_page_order(page);
> > -	__mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
> > +
> > +	if (get_pageblock_migratetype(page) != MIGRATE_ISOLATE)
> > +		__mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
> >  
> 
> Are you *sure* about this part? The page is already free so the
> NR_FREE_PAGES counters should already be correct. It feels to me that it

The isolated page is not counted as free so the counter shouldn't be
adjusted here (IOW we shouldn't decrease the counter as it was already
decreased in set_migratetype_isolate() earlier).

> should be the caller that fixes up NR_FREE_PAGES if necessary.

split_free_page() is only called from isolate_freepages_block() so
the fixup for MIGRATE_ISOLATE case can be added there if needed..

> Have you tested this with THP? I have a suspicion that the free page
> accounting gets broken when page migration is used there.

No I haven't tested it with THP but I don't see how it could break
because of this patch (please explain the potential failure scenario
a bit more).

Best regards,
--
Bartlomiej Zolnierkiewicz
Samsung Poland R&D Center

> >  	/* Split into individual pages */
> >  	set_page_refcounted(page);
> > diff --git a/mm/page_isolation.c b/mm/page_isolation.c
> > index 247d1f1..d210cc8 100644
> > --- a/mm/page_isolation.c
> > +++ b/mm/page_isolation.c
> > @@ -76,8 +76,12 @@ int set_migratetype_isolate(struct page *page)
> >  
> >  out:
> >  	if (!ret) {
> > +		unsigned long nr_pages;
> > +
> >  		set_pageblock_isolate(page);
> > -		move_freepages_block(zone, page, MIGRATE_ISOLATE);
> > +		nr_pages = move_freepages_block(zone, page, MIGRATE_ISOLATE);
> > +
> > +		__mod_zone_page_state(zone, NR_FREE_PAGES, -nr_pages);
> >  	}
> >  
> >  	spin_unlock_irqrestore(&zone->lock, flags);
> > @@ -89,12 +93,15 @@ out:
> >  void unset_migratetype_isolate(struct page *page, unsigned migratetype)
> >  {
> >  	struct zone *zone;
> > -	unsigned long flags;
> > +	unsigned long flags, nr_pages;
> > +
> >  	zone = page_zone(page);
> > +
> 
> unnecessary whitespace change.
> 
> >  	spin_lock_irqsave(&zone->lock, flags);
> >  	if (get_pageblock_migratetype(page) != MIGRATE_ISOLATE)
> >  		goto out;
> > -	move_freepages_block(zone, page, migratetype);
> > +	nr_pages = move_freepages_block(zone, page, migratetype);
> > +	__mod_zone_page_state(zone, NR_FREE_PAGES, nr_pages);
> >  	restore_pageblock_isolate(page, migratetype);
> >  out:
> >  	spin_unlock_irqrestore(&zone->lock, flags);

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v3 1/5] mm: fix tracing in free_pcppages_bulk()

[Minchan Kim]

On Tue, Sep 04, 2012 at 03:26:21PM +0200, Bartlomiej Zolnierkiewicz wrote:
> page->private gets re-used in __free_one_page() to store page order

Please write down result of end-user by this bug.
"So trace_mm_page_pcpu_drain may print order instead of migratetype"

> so migratetype value must be cached locally.
> 
> Fixes regression introduced in a701623 ("mm: fix migratetype bug
> which slowed swapping").
> 
> Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> Cc: Michal Nazarewicz <mina86@mina86.com>
> Cc: Minchan Kim <minchan@kernel.org>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Hugh Dickins <hughd@google.com>
> Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
> Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Minchan Kim <minchan@kernel.org>

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v3 2/5] cma: fix counting of isolated pages

[Minchan Kim]

On Tue, Sep 04, 2012 at 03:26:22PM +0200, Bartlomiej Zolnierkiewicz wrote:
> Isolated free pages shouldn't be accounted to NR_FREE_PAGES counter.
> Fix it by properly decreasing/increasing NR_FREE_PAGES counter in
> set_migratetype_isolate()/unset_migratetype_isolate() and removing
> counter adjustment for isolated pages from free_one_page() and
> split_free_page().
> 
> Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> Cc: Michal Nazarewicz <mina86@mina86.com>
> Cc: Minchan Kim <minchan@kernel.org>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Hugh Dickins <hughd@google.com>
> Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
> Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
> ---
>  mm/page_alloc.c     |  7 +++++--
>  mm/page_isolation.c | 13 ++++++++++---
>  2 files changed, 15 insertions(+), 5 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index e9da55c..3acdf0f 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -691,7 +691,8 @@ static void free_one_page(struct zone *zone, struct page *page, int order,
>  	zone->pages_scanned = 0;
>  
>  	__free_one_page(page, zone, order, migratetype);
> -	__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
> +	if (migratetype != MIGRATE_ISOLATE)

We can add unlikely.

> +		__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
>  	spin_unlock(&zone->lock);
>  }
>  
> @@ -1414,7 +1415,9 @@ int split_free_page(struct page *page, bool check_wmark)
>  	list_del(&page->lru);
>  	zone->free_area[order].nr_free--;
>  	rmv_page_order(page);
> -	__mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
> +
> +	if (get_pageblock_migratetype(page) != MIGRATE_ISOLATE)

The get_pageblock_migratetype isn't cheap.
You can use get_freepage_migratetype.

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v3 3/5] cma: count free CMA pages

[Minchan Kim]

On Tue, Sep 04, 2012 at 03:26:23PM +0200, Bartlomiej Zolnierkiewicz wrote:
> Add NR_FREE_CMA_PAGES counter to be later used for checking watermark
> in __zone_watermark_ok().  For simplicity and to avoid #ifdef hell make
> this counter always available (not only when CONFIG_CMA=y).

I would like to hide it in case of !CONFIG_CMA.
Otherwise, does it really make ifdef hell?

Many part of your code uses below pattern.

                       __mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
                       if (is_migrate_cma(migratetype))
                               __mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
                                                     1 << order);

So how about this?

        NR_ANON_TRANSPARENT_HUGEPAGES,
+#ifdef CONFIG_CMA
+       NR_FREE_CMA_PAGES,
+#endif
        NR_VM_ZONE_STAT_ITEMS };

#ifdef CONFIG_CMA
#  define is_migrate_cma(migratetype) unlikely((migratetype) == MIGRATE_CMA)
#  define cma_wmark_pages(zone) zone->min_cma_pages
#else
#  define is_migrate_cma(migratetype) false
#  define cma_wmark_pages(zone) 0
#  define NR_FREE_CMA_PAGES     0
#endif

void __mod_zone_freepage_state(struct zone *zone, int nr_pages,
                               int migratetype)
{
       __mod_zone_page_state(zone, NR_FREE_PAGES, nr_pages);
       if (is_migrate_cma(migratetype))
               __mod_zone_page_state(zone, NR_FREE_CMA_PAGES, nr_pages);
}


> 
> Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> Cc: Michal Nazarewicz <mina86@mina86.com>
> Cc: Minchan Kim <minchan@kernel.org>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Hugh Dickins <hughd@google.com>
> Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
> Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
> ---
>  include/linux/mmzone.h |  1 +
>  mm/page_alloc.c        | 36 ++++++++++++++++++++++++++++++++----
>  mm/page_isolation.c    |  7 +++++++
>  mm/vmstat.c            |  1 +
>  4 files changed, 41 insertions(+), 4 deletions(-)
> 
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index ca034a1..904889d 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -140,6 +140,7 @@ enum zone_stat_item {
>  	NUMA_OTHER,		/* allocation from other node */
>  #endif
>  	NR_ANON_TRANSPARENT_HUGEPAGES,
> +	NR_FREE_CMA_PAGES,
>  	NR_VM_ZONE_STAT_ITEMS };
>  
>  /*
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 3acdf0f..5bb0cda 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -559,6 +559,9 @@ static inline void __free_one_page(struct page *page,
>  			clear_page_guard_flag(buddy);
>  			set_page_private(page, 0);
>  			__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
> +			if (is_migrate_cma(migratetype))
> +				__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> +						      1 << order);
>  		} else {
>  			list_del(&buddy->lru);
>  			zone->free_area[order].nr_free--;
> @@ -677,6 +680,8 @@ static void free_pcppages_bulk(struct zone *zone, int count,
>  			/* MIGRATE_MOVABLE list may include MIGRATE_RESERVEs */
>  			__free_one_page(page, zone, 0, mt);
>  			trace_mm_page_pcpu_drain(page, 0, mt);
> +			if (is_migrate_cma(mt))
> +				__mod_zone_page_state(zone, NR_FREE_CMA_PAGES, 1);
>  		} while (--to_free && --batch_free && !list_empty(list));
>  	}
>  	__mod_zone_page_state(zone, NR_FREE_PAGES, count);
> @@ -691,8 +696,12 @@ static void free_one_page(struct zone *zone, struct page *page, int order,
>  	zone->pages_scanned = 0;
>  
>  	__free_one_page(page, zone, order, migratetype);
> -	if (migratetype != MIGRATE_ISOLATE)
> +	if (migratetype != MIGRATE_ISOLATE) {
>  		__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
> +		if (is_migrate_cma(migratetype))
> +			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> +					      1 << order);
> +	}
>  	spin_unlock(&zone->lock);
>  }
>  
> @@ -816,6 +825,9 @@ static inline void expand(struct zone *zone, struct page *page,
>  			set_page_private(&page[size], high);
>  			/* Guard pages are not available for any usage */
>  			__mod_zone_page_state(zone, NR_FREE_PAGES, -(1 << high));
> +			if (is_migrate_cma(migratetype))
> +				__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> +						      -(1 << high));
>  			continue;
>  		}
>  #endif
> @@ -1141,6 +1153,9 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order,
>  		}
>  		set_page_private(page, mt);
>  		list = &page->lru;
> +		if (is_migrate_cma(mt))
> +			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> +					      -(1 << order));
>  	}
>  	__mod_zone_page_state(zone, NR_FREE_PAGES, -(i << order));
>  	spin_unlock(&zone->lock);
> @@ -1398,6 +1413,7 @@ int split_free_page(struct page *page, bool check_wmark)
>  	unsigned int order;
>  	unsigned long watermark;
>  	struct zone *zone;
> +	int mt;
>  
>  	BUG_ON(!PageBuddy(page));
>  
> @@ -1416,8 +1432,13 @@ int split_free_page(struct page *page, bool check_wmark)
>  	zone->free_area[order].nr_free--;
>  	rmv_page_order(page);
>  
> -	if (get_pageblock_migratetype(page) != MIGRATE_ISOLATE)
> +	mt = get_pageblock_migratetype(page);
> +	if (mt != MIGRATE_ISOLATE) {
>  		__mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
> +		if (is_migrate_cma(mt))
> +			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> +					      -(1UL << order));
> +	}
>  
>  	/* Split into individual pages */
>  	set_page_refcounted(page);
> @@ -1492,6 +1513,9 @@ again:
>  		spin_unlock(&zone->lock);
>  		if (!page)
>  			goto failed;
> +		if (is_migrate_cma(get_pageblock_migratetype(page)))
> +			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> +					      -(1 << order));
>  		__mod_zone_page_state(zone, NR_FREE_PAGES, -(1 << order));
>  	}
>  
> @@ -2860,7 +2884,8 @@ void show_free_areas(unsigned int filter)
>  		" unevictable:%lu"
>  		" dirty:%lu writeback:%lu unstable:%lu\n"
>  		" free:%lu slab_reclaimable:%lu slab_unreclaimable:%lu\n"
> -		" mapped:%lu shmem:%lu pagetables:%lu bounce:%lu\n",
> +		" mapped:%lu shmem:%lu pagetables:%lu bounce:%lu\n"
> +		" free_cma:%lu\n",
>  		global_page_state(NR_ACTIVE_ANON),
>  		global_page_state(NR_INACTIVE_ANON),
>  		global_page_state(NR_ISOLATED_ANON),
> @@ -2877,7 +2902,8 @@ void show_free_areas(unsigned int filter)
>  		global_page_state(NR_FILE_MAPPED),
>  		global_page_state(NR_SHMEM),
>  		global_page_state(NR_PAGETABLE),
> -		global_page_state(NR_BOUNCE));
> +		global_page_state(NR_BOUNCE),
> +		global_page_state(NR_FREE_CMA_PAGES));
>  
>  	for_each_populated_zone(zone) {
>  		int i;
> @@ -2909,6 +2935,7 @@ void show_free_areas(unsigned int filter)
>  			" pagetables:%lukB"
>  			" unstable:%lukB"
>  			" bounce:%lukB"
> +			" free_cma:%lukB"
>  			" writeback_tmp:%lukB"
>  			" pages_scanned:%lu"
>  			" all_unreclaimable? %s"
> @@ -2938,6 +2965,7 @@ void show_free_areas(unsigned int filter)
>  			K(zone_page_state(zone, NR_PAGETABLE)),
>  			K(zone_page_state(zone, NR_UNSTABLE_NFS)),
>  			K(zone_page_state(zone, NR_BOUNCE)),
> +			K(zone_page_state(zone, NR_FREE_CMA_PAGES)),
>  			K(zone_page_state(zone, NR_WRITEBACK_TEMP)),
>  			zone->pages_scanned,
>  			(zone->all_unreclaimable ? "yes" : "no")
> diff --git a/mm/page_isolation.c b/mm/page_isolation.c
> index d210cc8..6ead34d 100644
> --- a/mm/page_isolation.c
> +++ b/mm/page_isolation.c
> @@ -77,11 +77,15 @@ int set_migratetype_isolate(struct page *page)
>  out:
>  	if (!ret) {
>  		unsigned long nr_pages;
> +		int mt = get_pageblock_migratetype(page);
>  
>  		set_pageblock_isolate(page);
>  		nr_pages = move_freepages_block(zone, page, MIGRATE_ISOLATE);
>  
>  		__mod_zone_page_state(zone, NR_FREE_PAGES, -nr_pages);
> +		if (is_migrate_cma(mt))
> +			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> +					      -nr_pages);
>  	}
>  
>  	spin_unlock_irqrestore(&zone->lock, flags);
> @@ -102,6 +106,9 @@ void unset_migratetype_isolate(struct page *page, unsigned migratetype)
>  		goto out;
>  	nr_pages = move_freepages_block(zone, page, migratetype);
>  	__mod_zone_page_state(zone, NR_FREE_PAGES, nr_pages);
> +	if (is_migrate_cma(migratetype))
> +		__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> +				      nr_pages);
>  	restore_pageblock_isolate(page, migratetype);
>  out:
>  	spin_unlock_irqrestore(&zone->lock, flags);
> diff --git a/mm/vmstat.c b/mm/vmstat.c
> index df7a674..7c102e6 100644
> --- a/mm/vmstat.c
> +++ b/mm/vmstat.c
> @@ -722,6 +722,7 @@ const char * const vmstat_text[] = {
>  	"numa_other",
>  #endif
>  	"nr_anon_transparent_hugepages",
> +	"nr_free_cma",
>  	"nr_dirty_threshold",
>  	"nr_dirty_background_threshold",
>  
> -- 
> 1.7.11.3
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v3 4/5] mm: add accounting for CMA pages and use them for watermark calculation

[Minchan Kim]

On Tue, Sep 04, 2012 at 03:26:24PM +0200, Bartlomiej Zolnierkiewicz wrote:
> From: Marek Szyprowski <m.szyprowski@samsung.com>
> 
> During watermark check we need to decrease available free pages number
> by free CMA pages number because unmovable allocations cannot use pages
> from CMA areas.

This patch could be fold into 5/5.

> 
> Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
> Cc: Michal Nazarewicz <mina86@mina86.com>
> Cc: Minchan Kim <minchan@kernel.org>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Hugh Dickins <hughd@google.com>
> Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
> Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
> ---
>  mm/page_alloc.c | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 5bb0cda..2166774 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -1628,7 +1628,7 @@ static inline bool should_fail_alloc_page(gfp_t gfp_mask, unsigned int order)
>   * of the allocation.
>   */
>  static bool __zone_watermark_ok(struct zone *z, int order, unsigned long mark,
> -		      int classzone_idx, int alloc_flags, long free_pages)
> +		      int classzone_idx, int alloc_flags, long free_pages, long free_cma_pages)
>  {
>  	/* free_pages my go negative - that's OK */
>  	long min = mark;
> @@ -1641,7 +1641,7 @@ static bool __zone_watermark_ok(struct zone *z, int order, unsigned long mark,
>  	if (alloc_flags & ALLOC_HARDER)
>  		min -= min / 4;
>  
> -	if (free_pages <= min + lowmem_reserve)
> +	if (free_pages - free_cma_pages <= min + lowmem_reserve)
>  		return false;
>  	for (o = 0; o < order; o++) {
>  		/* At the next order, this order's pages become unavailable */
> @@ -1674,13 +1674,15 @@ bool zone_watermark_ok(struct zone *z, int order, unsigned long mark,
>  		      int classzone_idx, int alloc_flags)
>  {
>  	return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags,
> -					zone_page_state(z, NR_FREE_PAGES));
> +					zone_page_state(z, NR_FREE_PAGES),
> +					zone_page_state(z, NR_FREE_CMA_PAGES));
>  }
>  
>  bool zone_watermark_ok_safe(struct zone *z, int order, unsigned long mark,
>  		      int classzone_idx, int alloc_flags)
>  {
>  	long free_pages = zone_page_state(z, NR_FREE_PAGES);
> +	long free_cma_pages = zone_page_state(z, NR_FREE_CMA_PAGES);
>  
>  	if (z->percpu_drift_mark && free_pages < z->percpu_drift_mark)
>  		free_pages = zone_page_state_snapshot(z, NR_FREE_PAGES);
> @@ -1694,7 +1696,7 @@ bool zone_watermark_ok_safe(struct zone *z, int order, unsigned long mark,
>  	 */
>  	free_pages -= nr_zone_isolate_freepages(z);
>  	return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags,
> -								free_pages);
> +					free_pages, free_cma_pages);
>  }
>  
>  #ifdef CONFIG_NUMA
> -- 
> 1.7.11.3
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v3 5/5] cma: fix watermark checking

[Minchan Kim]

On Tue, Sep 04, 2012 at 03:26:25PM +0200, Bartlomiej Zolnierkiewicz wrote:
> Pass GFP flags to [__]zone_watermark_ok() and use them to account
> free CMA pages only when necessary (there is no need to check
> watermark against only non-CMA free pages for movable allocations).

I want to make it zero-overhead in case of !CONFIG_CMA.
We can reduce the number of zone_watermark_ok's argument and in case of !CONFIG_CMA,
overhead would be zero.

How about this?
(Below is what I want to show the *concept*, NOT completed, NOT compile tested)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 009ac28..61c592a 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1514,6 +1514,8 @@ failed:
 #define ALLOC_HIGH		0x20 /* __GFP_HIGH set */
 #define ALLOC_CPUSET		0x40 /* check for correct cpuset */
 
+#define ALLOC_CMA		0x80
+
 #ifdef CONFIG_FAIL_PAGE_ALLOC
 
 static struct {
@@ -1608,7 +1610,10 @@ static bool __zone_watermark_ok(struct zone *z, int order, unsigned long mark,
 		min -= min / 2;
 	if (alloc_flags & ALLOC_HARDER)
 		min -= min / 4;
-
+#ifdef CONFIG_CMA
+	if (alloc_flags & ALLOC_CMA)
+		free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
+#endif
 	if (free_pages <= min + lowmem_reserve)
 		return false;
 	for (o = 0; o < order; o++) {
@@ -2303,7 +2308,10 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
 				 unlikely(test_thread_flag(TIF_MEMDIE))))
 			alloc_flags |= ALLOC_NO_WATERMARKS;
 	}
-
+#ifdef CONFIG_CMA
+	if (allocflags_to_migratetype(gfp_mask) == MIGRATE_MOVABLE)
+			alloc_flags |= ALLOC_CMA;
+#endif
 	return alloc_flags;
 }
 
@@ -2533,6 +2541,7 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order,
 	struct page *page = NULL;
 	int migratetype = allocflags_to_migratetype(gfp_mask);
 	unsigned int cpuset_mems_cookie;
+	int alloc_flags = ALLOC_WMARK_LOW|ALLOC_CPUSET;
 
 	gfp_mask &= gfp_allowed_mask;
 
@@ -2561,9 +2570,13 @@ retry_cpuset:
 	if (!preferred_zone)
 		goto out;
 
+#ifdef CONFIG_CMA
+	if (allocflags_to_migratetype(gfp_mask) == MIGRATE_MOVABLE)
+		alloc_flags |= ALLOC_CMA;
+#endif
 	/* First allocation attempt */
 	page = get_page_from_freelist(gfp_mask|__GFP_HARDWALL, nodemask, order,
-			zonelist, high_zoneidx, ALLOC_WMARK_LOW|ALLOC_CPUSET,
+			zonelist, high_zoneidx, alloc_flags,
 			preferred_zone, migratetype);
 	if (unlikely(!page))
 		page = __alloc_pages_slowpath(gfp_mask, order,
> 
> Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> Cc: Michal Nazarewicz <mina86@mina86.com>
> Cc: Minchan Kim <minchan@kernel.org>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Hugh Dickins <hughd@google.com>
> Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
> Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
> ---
>  include/linux/mmzone.h |  2 +-
>  mm/compaction.c        | 11 ++++++-----
>  mm/page_alloc.c        | 29 +++++++++++++++++++----------
>  mm/vmscan.c            |  4 ++--
>  4 files changed, 28 insertions(+), 18 deletions(-)
> 
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index 904889d..308bb91 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -725,7 +725,7 @@ extern struct mutex zonelists_mutex;
>  void build_all_zonelists(pg_data_t *pgdat, struct zone *zone);
>  void wakeup_kswapd(struct zone *zone, int order, enum zone_type classzone_idx);
>  bool zone_watermark_ok(struct zone *z, int order, unsigned long mark,
> -		int classzone_idx, int alloc_flags);
> +		int classzone_idx, int alloc_flags, gfp_t gfp_flags);
>  bool zone_watermark_ok_safe(struct zone *z, int order, unsigned long mark,
>  		int classzone_idx, int alloc_flags);
>  enum memmap_context {
> diff --git a/mm/compaction.c b/mm/compaction.c
> index 4b902aa..080175a 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -684,7 +684,7 @@ static int compact_finished(struct zone *zone,
>  	watermark = low_wmark_pages(zone);
>  	watermark += (1 << cc->order);
>  
> -	if (!zone_watermark_ok(zone, cc->order, watermark, 0, 0))
> +	if (!zone_watermark_ok(zone, cc->order, watermark, 0, 0, 0))
>  		return COMPACT_CONTINUE;
>  
>  	/* Direct compactor: Is a suitable page free? */
> @@ -726,7 +726,7 @@ unsigned long compaction_suitable(struct zone *zone, int order)
>  	 * allocated and for a short time, the footprint is higher
>  	 */
>  	watermark = low_wmark_pages(zone) + (2UL << order);
> -	if (!zone_watermark_ok(zone, 0, watermark, 0, 0))
> +	if (!zone_watermark_ok(zone, 0, watermark, 0, 0, 0))
>  		return COMPACT_SKIPPED;
>  
>  	/*
> @@ -745,7 +745,7 @@ unsigned long compaction_suitable(struct zone *zone, int order)
>  		return COMPACT_SKIPPED;
>  
>  	if (fragindex == -1000 && zone_watermark_ok(zone, order, watermark,
> -	    0, 0))
> +	    0, 0, 0))
>  		return COMPACT_PARTIAL;
>  
>  	return COMPACT_CONTINUE;
> @@ -889,7 +889,8 @@ unsigned long try_to_compact_pages(struct zonelist *zonelist,
>  		rc = max(status, rc);
>  
>  		/* If a normal allocation would succeed, stop compacting */
> -		if (zone_watermark_ok(zone, order, low_wmark_pages(zone), 0, 0))
> +		if (zone_watermark_ok(zone, order, low_wmark_pages(zone), 0, 0,
> +				      gfp_mask))
>  			break;
>  	}
>  
> @@ -920,7 +921,7 @@ static int __compact_pgdat(pg_data_t *pgdat, struct compact_control *cc)
>  
>  		if (cc->order > 0) {
>  			int ok = zone_watermark_ok(zone, cc->order,
> -						low_wmark_pages(zone), 0, 0);
> +						low_wmark_pages(zone), 0, 0, 0);
>  			if (ok && cc->order >= zone->compact_order_failed)
>  				zone->compact_order_failed = cc->order + 1;
>  			/* Currently async compaction is never deferred. */
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 2166774..5912a8c 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -1423,7 +1423,7 @@ int split_free_page(struct page *page, bool check_wmark)
>  	if (check_wmark) {
>  		/* Obey watermarks as if the page was being allocated */
>  		watermark = low_wmark_pages(zone) + (1 << order);
> -		if (!zone_watermark_ok(zone, 0, watermark, 0, 0))
> +		if (!zone_watermark_ok(zone, 0, watermark, 0, 0, 0))
>  			return 0;
>  	}
>  
> @@ -1628,12 +1628,13 @@ static inline bool should_fail_alloc_page(gfp_t gfp_mask, unsigned int order)
>   * of the allocation.
>   */
>  static bool __zone_watermark_ok(struct zone *z, int order, unsigned long mark,
> -		      int classzone_idx, int alloc_flags, long free_pages, long free_cma_pages)
> +		      int classzone_idx, int alloc_flags, long free_pages,
> +		      long free_cma_pages, gfp_t gfp_flags)
>  {
>  	/* free_pages my go negative - that's OK */
>  	long min = mark;
>  	long lowmem_reserve = z->lowmem_reserve[classzone_idx];
> -	int o;
> +	int mt, o;
>  
>  	free_pages -= (1 << order) - 1;
>  	if (alloc_flags & ALLOC_HIGH)
> @@ -1641,8 +1642,14 @@ static bool __zone_watermark_ok(struct zone *z, int order, unsigned long mark,
>  	if (alloc_flags & ALLOC_HARDER)
>  		min -= min / 4;
>  
> -	if (free_pages - free_cma_pages <= min + lowmem_reserve)
> -		return false;
> +	mt = allocflags_to_migratetype(gfp_flags);
> +	if (mt == MIGRATE_MOVABLE) {
> +		if (free_pages <= min + lowmem_reserve)
> +			return false;
> +	} else {
> +		if (free_pages - free_cma_pages <= min + lowmem_reserve)
> +			return false;
> +	}
>  	for (o = 0; o < order; o++) {
>  		/* At the next order, this order's pages become unavailable */
>  		free_pages -= z->free_area[o].nr_free << o;
> @@ -1671,11 +1678,12 @@ static inline unsigned long nr_zone_isolate_freepages(struct zone *zone)
>  #endif
>  
>  bool zone_watermark_ok(struct zone *z, int order, unsigned long mark,
> -		      int classzone_idx, int alloc_flags)
> +		      int classzone_idx, int alloc_flags, gfp_t gfp_flags)
>  {
>  	return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags,
>  					zone_page_state(z, NR_FREE_PAGES),
> -					zone_page_state(z, NR_FREE_CMA_PAGES));
> +					zone_page_state(z, NR_FREE_CMA_PAGES),
> +					gfp_flags);
>  }
>  
>  bool zone_watermark_ok_safe(struct zone *z, int order, unsigned long mark,
> @@ -1696,7 +1704,7 @@ bool zone_watermark_ok_safe(struct zone *z, int order, unsigned long mark,
>  	 */
>  	free_pages -= nr_zone_isolate_freepages(z);
>  	return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags,
> -					free_pages, free_cma_pages);
> +					free_pages, free_cma_pages, 0);
>  }
>  
>  #ifdef CONFIG_NUMA
> @@ -1906,7 +1914,7 @@ zonelist_scan:
>  
>  			mark = zone->watermark[alloc_flags & ALLOC_WMARK_MASK];
>  			if (zone_watermark_ok(zone, order, mark,
> -				    classzone_idx, alloc_flags))
> +				    classzone_idx, alloc_flags, gfp_mask))
>  				goto try_this_zone;
>  
>  			if (NUMA_BUILD && !did_zlc_setup && nr_online_nodes > 1) {
> @@ -1942,7 +1950,8 @@ zonelist_scan:
>  			default:
>  				/* did we reclaim enough */
>  				if (!zone_watermark_ok(zone, order, mark,
> -						classzone_idx, alloc_flags))
> +						classzone_idx, alloc_flags,
> +						gfp_mask))
>  					goto this_zone_full;
>  			}
>  		}
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 8d01243..4a10038b 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -2777,14 +2777,14 @@ out:
>  
>  			/* Confirm the zone is balanced for order-0 */
>  			if (!zone_watermark_ok(zone, 0,
> -					high_wmark_pages(zone), 0, 0)) {
> +					high_wmark_pages(zone), 0, 0, 0)) {
>  				order = sc.order = 0;
>  				goto loop_again;
>  			}
>  
>  			/* Check if the memory needs to be defragmented. */
>  			if (zone_watermark_ok(zone, order,
> -				    low_wmark_pages(zone), *classzone_idx, 0))
> +				    low_wmark_pages(zone), *classzone_idx, 0, 0))
>  				zones_need_compaction = 0;
>  
>  			/* If balanced, clear the congested flag */
> -- 
> 1.7.11.3
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v3 2/5] cma: fix counting of isolated pages

[Bartlomiej Zolnierkiewicz]

On Friday 14 September 2012 04:26:20 Minchan Kim wrote:
> On Tue, Sep 04, 2012 at 03:26:22PM +0200, Bartlomiej Zolnierkiewicz wrote:
> > Isolated free pages shouldn't be accounted to NR_FREE_PAGES counter.
> > Fix it by properly decreasing/increasing NR_FREE_PAGES counter in
> > set_migratetype_isolate()/unset_migratetype_isolate() and removing
> > counter adjustment for isolated pages from free_one_page() and
> > split_free_page().
> > 
> > Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> > Cc: Michal Nazarewicz <mina86@mina86.com>
> > Cc: Minchan Kim <minchan@kernel.org>
> > Cc: Mel Gorman <mgorman@suse.de>
> > Cc: Hugh Dickins <hughd@google.com>
> > Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
> > Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
> > ---
> >  mm/page_alloc.c     |  7 +++++--
> >  mm/page_isolation.c | 13 ++++++++++---
> >  2 files changed, 15 insertions(+), 5 deletions(-)
> > 
> > diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> > index e9da55c..3acdf0f 100644
> > --- a/mm/page_alloc.c
> > +++ b/mm/page_alloc.c
> > @@ -691,7 +691,8 @@ static void free_one_page(struct zone *zone, struct page *page, int order,
> >  	zone->pages_scanned = 0;
> >  
> >  	__free_one_page(page, zone, order, migratetype);
> > -	__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
> > +	if (migratetype != MIGRATE_ISOLATE)
> 
> We can add unlikely.

Ok.

> > +		__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
> >  	spin_unlock(&zone->lock);
> >  }
> >  
> > @@ -1414,7 +1415,9 @@ int split_free_page(struct page *page, bool check_wmark)
> >  	list_del(&page->lru);
> >  	zone->free_area[order].nr_free--;
> >  	rmv_page_order(page);
> > -	__mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
> > +
> > +	if (get_pageblock_migratetype(page) != MIGRATE_ISOLATE)
> 
> The get_pageblock_migratetype isn't cheap.
> You can use get_freepage_migratetype.

get_freepage_migratetype() is not yet upstream and I don't want to add
more dependencies to these patches so I'll update this later.

Best regards,
--
Bartlomiej Zolnierkiewicz
Samsung Poland R&D Center

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v3 3/5] cma: count free CMA pages

[Bartlomiej Zolnierkiewicz]

On Friday 14 September 2012 05:02:24 Minchan Kim wrote:
> On Tue, Sep 04, 2012 at 03:26:23PM +0200, Bartlomiej Zolnierkiewicz wrote:
> > Add NR_FREE_CMA_PAGES counter to be later used for checking watermark
> > in __zone_watermark_ok().  For simplicity and to avoid #ifdef hell make
> > this counter always available (not only when CONFIG_CMA=y).
> 
> I would like to hide it in case of !CONFIG_CMA.
> Otherwise, does it really make ifdef hell?
> 
> Many part of your code uses below pattern.
> 
>                        __mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
>                        if (is_migrate_cma(migratetype))
>                                __mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
>                                                      1 << order);
> 
> So how about this?
> 
>         NR_ANON_TRANSPARENT_HUGEPAGES,
> +#ifdef CONFIG_CMA
> +       NR_FREE_CMA_PAGES,
> +#endif
>         NR_VM_ZONE_STAT_ITEMS };
> 
> #ifdef CONFIG_CMA
> #  define is_migrate_cma(migratetype) unlikely((migratetype) == MIGRATE_CMA)
> #  define cma_wmark_pages(zone) zone->min_cma_pages
> #else
> #  define is_migrate_cma(migratetype) false
> #  define cma_wmark_pages(zone) 0
> #  define NR_FREE_CMA_PAGES     0
> #endif

I worry that this wouldn't work for show_free_areas() (0 would mean that
NR_FREE_PAGES counter would be used instead) without adding ifdef hell..

> void __mod_zone_freepage_state(struct zone *zone, int nr_pages,
>                                int migratetype)
> {
>        __mod_zone_page_state(zone, NR_FREE_PAGES, nr_pages);
>        if (is_migrate_cma(migratetype))
>                __mod_zone_page_state(zone, NR_FREE_CMA_PAGES, nr_pages);
> }

This is a good idea and I'll fix the patch accordingly.

Best regards,
--
Bartlomiej Zolnierkiewicz
Samsung Poland R&D Center

> 
> > 
> > Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> > Cc: Michal Nazarewicz <mina86@mina86.com>
> > Cc: Minchan Kim <minchan@kernel.org>
> > Cc: Mel Gorman <mgorman@suse.de>
> > Cc: Hugh Dickins <hughd@google.com>
> > Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
> > Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
> > ---
> >  include/linux/mmzone.h |  1 +
> >  mm/page_alloc.c        | 36 ++++++++++++++++++++++++++++++++----
> >  mm/page_isolation.c    |  7 +++++++
> >  mm/vmstat.c            |  1 +
> >  4 files changed, 41 insertions(+), 4 deletions(-)
> > 
> > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> > index ca034a1..904889d 100644
> > --- a/include/linux/mmzone.h
> > +++ b/include/linux/mmzone.h
> > @@ -140,6 +140,7 @@ enum zone_stat_item {
> >  	NUMA_OTHER,		/* allocation from other node */
> >  #endif
> >  	NR_ANON_TRANSPARENT_HUGEPAGES,
> > +	NR_FREE_CMA_PAGES,
> >  	NR_VM_ZONE_STAT_ITEMS };
> >  
> >  /*
> > diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> > index 3acdf0f..5bb0cda 100644
> > --- a/mm/page_alloc.c
> > +++ b/mm/page_alloc.c
> > @@ -559,6 +559,9 @@ static inline void __free_one_page(struct page *page,
> >  			clear_page_guard_flag(buddy);
> >  			set_page_private(page, 0);
> >  			__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
> > +			if (is_migrate_cma(migratetype))
> > +				__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> > +						      1 << order);
> >  		} else {
> >  			list_del(&buddy->lru);
> >  			zone->free_area[order].nr_free--;
> > @@ -677,6 +680,8 @@ static void free_pcppages_bulk(struct zone *zone, int count,
> >  			/* MIGRATE_MOVABLE list may include MIGRATE_RESERVEs */
> >  			__free_one_page(page, zone, 0, mt);
> >  			trace_mm_page_pcpu_drain(page, 0, mt);
> > +			if (is_migrate_cma(mt))
> > +				__mod_zone_page_state(zone, NR_FREE_CMA_PAGES, 1);
> >  		} while (--to_free && --batch_free && !list_empty(list));
> >  	}
> >  	__mod_zone_page_state(zone, NR_FREE_PAGES, count);
> > @@ -691,8 +696,12 @@ static void free_one_page(struct zone *zone, struct page *page, int order,
> >  	zone->pages_scanned = 0;
> >  
> >  	__free_one_page(page, zone, order, migratetype);
> > -	if (migratetype != MIGRATE_ISOLATE)
> > +	if (migratetype != MIGRATE_ISOLATE) {
> >  		__mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
> > +		if (is_migrate_cma(migratetype))
> > +			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> > +					      1 << order);
> > +	}
> >  	spin_unlock(&zone->lock);
> >  }
> >  
> > @@ -816,6 +825,9 @@ static inline void expand(struct zone *zone, struct page *page,
> >  			set_page_private(&page[size], high);
> >  			/* Guard pages are not available for any usage */
> >  			__mod_zone_page_state(zone, NR_FREE_PAGES, -(1 << high));
> > +			if (is_migrate_cma(migratetype))
> > +				__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> > +						      -(1 << high));
> >  			continue;
> >  		}
> >  #endif
> > @@ -1141,6 +1153,9 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order,
> >  		}
> >  		set_page_private(page, mt);
> >  		list = &page->lru;
> > +		if (is_migrate_cma(mt))
> > +			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> > +					      -(1 << order));
> >  	}
> >  	__mod_zone_page_state(zone, NR_FREE_PAGES, -(i << order));
> >  	spin_unlock(&zone->lock);
> > @@ -1398,6 +1413,7 @@ int split_free_page(struct page *page, bool check_wmark)
> >  	unsigned int order;
> >  	unsigned long watermark;
> >  	struct zone *zone;
> > +	int mt;
> >  
> >  	BUG_ON(!PageBuddy(page));
> >  
> > @@ -1416,8 +1432,13 @@ int split_free_page(struct page *page, bool check_wmark)
> >  	zone->free_area[order].nr_free--;
> >  	rmv_page_order(page);
> >  
> > -	if (get_pageblock_migratetype(page) != MIGRATE_ISOLATE)
> > +	mt = get_pageblock_migratetype(page);
> > +	if (mt != MIGRATE_ISOLATE) {
> >  		__mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
> > +		if (is_migrate_cma(mt))
> > +			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> > +					      -(1UL << order));
> > +	}
> >  
> >  	/* Split into individual pages */
> >  	set_page_refcounted(page);
> > @@ -1492,6 +1513,9 @@ again:
> >  		spin_unlock(&zone->lock);
> >  		if (!page)
> >  			goto failed;
> > +		if (is_migrate_cma(get_pageblock_migratetype(page)))
> > +			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> > +					      -(1 << order));
> >  		__mod_zone_page_state(zone, NR_FREE_PAGES, -(1 << order));
> >  	}
> >  
> > @@ -2860,7 +2884,8 @@ void show_free_areas(unsigned int filter)
> >  		" unevictable:%lu"
> >  		" dirty:%lu writeback:%lu unstable:%lu\n"
> >  		" free:%lu slab_reclaimable:%lu slab_unreclaimable:%lu\n"
> > -		" mapped:%lu shmem:%lu pagetables:%lu bounce:%lu\n",
> > +		" mapped:%lu shmem:%lu pagetables:%lu bounce:%lu\n"
> > +		" free_cma:%lu\n",
> >  		global_page_state(NR_ACTIVE_ANON),
> >  		global_page_state(NR_INACTIVE_ANON),
> >  		global_page_state(NR_ISOLATED_ANON),
> > @@ -2877,7 +2902,8 @@ void show_free_areas(unsigned int filter)
> >  		global_page_state(NR_FILE_MAPPED),
> >  		global_page_state(NR_SHMEM),
> >  		global_page_state(NR_PAGETABLE),
> > -		global_page_state(NR_BOUNCE));
> > +		global_page_state(NR_BOUNCE),
> > +		global_page_state(NR_FREE_CMA_PAGES));
> >  
> >  	for_each_populated_zone(zone) {
> >  		int i;
> > @@ -2909,6 +2935,7 @@ void show_free_areas(unsigned int filter)
> >  			" pagetables:%lukB"
> >  			" unstable:%lukB"
> >  			" bounce:%lukB"
> > +			" free_cma:%lukB"
> >  			" writeback_tmp:%lukB"
> >  			" pages_scanned:%lu"
> >  			" all_unreclaimable? %s"
> > @@ -2938,6 +2965,7 @@ void show_free_areas(unsigned int filter)
> >  			K(zone_page_state(zone, NR_PAGETABLE)),
> >  			K(zone_page_state(zone, NR_UNSTABLE_NFS)),
> >  			K(zone_page_state(zone, NR_BOUNCE)),
> > +			K(zone_page_state(zone, NR_FREE_CMA_PAGES)),
> >  			K(zone_page_state(zone, NR_WRITEBACK_TEMP)),
> >  			zone->pages_scanned,
> >  			(zone->all_unreclaimable ? "yes" : "no")
> > diff --git a/mm/page_isolation.c b/mm/page_isolation.c
> > index d210cc8..6ead34d 100644
> > --- a/mm/page_isolation.c
> > +++ b/mm/page_isolation.c
> > @@ -77,11 +77,15 @@ int set_migratetype_isolate(struct page *page)
> >  out:
> >  	if (!ret) {
> >  		unsigned long nr_pages;
> > +		int mt = get_pageblock_migratetype(page);
> >  
> >  		set_pageblock_isolate(page);
> >  		nr_pages = move_freepages_block(zone, page, MIGRATE_ISOLATE);
> >  
> >  		__mod_zone_page_state(zone, NR_FREE_PAGES, -nr_pages);
> > +		if (is_migrate_cma(mt))
> > +			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> > +					      -nr_pages);
> >  	}
> >  
> >  	spin_unlock_irqrestore(&zone->lock, flags);
> > @@ -102,6 +106,9 @@ void unset_migratetype_isolate(struct page *page, unsigned migratetype)
> >  		goto out;
> >  	nr_pages = move_freepages_block(zone, page, migratetype);
> >  	__mod_zone_page_state(zone, NR_FREE_PAGES, nr_pages);
> > +	if (is_migrate_cma(migratetype))
> > +		__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> > +				      nr_pages);
> >  	restore_pageblock_isolate(page, migratetype);
> >  out:
> >  	spin_unlock_irqrestore(&zone->lock, flags);
> > diff --git a/mm/vmstat.c b/mm/vmstat.c
> > index df7a674..7c102e6 100644
> > --- a/mm/vmstat.c
> > +++ b/mm/vmstat.c
> > @@ -722,6 +722,7 @@ const char * const vmstat_text[] = {
> >  	"numa_other",
> >  #endif
> >  	"nr_anon_transparent_hugepages",
> > +	"nr_free_cma",
> >  	"nr_dirty_threshold",
> >  	"nr_dirty_background_threshold",
> >  

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v3 5/5] cma: fix watermark checking

[Bartlomiej Zolnierkiewicz]

On Friday 14 September 2012 06:13:34 Minchan Kim wrote:
> On Tue, Sep 04, 2012 at 03:26:25PM +0200, Bartlomiej Zolnierkiewicz wrote:
> > Pass GFP flags to [__]zone_watermark_ok() and use them to account
> > free CMA pages only when necessary (there is no need to check
> > watermark against only non-CMA free pages for movable allocations).
> 
> I want to make it zero-overhead in case of !CONFIG_CMA.
> We can reduce the number of zone_watermark_ok's argument and in case of !CONFIG_CMA,
> overhead would be zero.
> 
> How about this?
> (Below is what I want to show the *concept*, NOT completed, NOT compile tested)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 009ac28..61c592a 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -1514,6 +1514,8 @@ failed:
>  #define ALLOC_HIGH		0x20 /* __GFP_HIGH set */
>  #define ALLOC_CPUSET		0x40 /* check for correct cpuset */
>  
> +#define ALLOC_CMA		0x80
> +
>  #ifdef CONFIG_FAIL_PAGE_ALLOC
>  
>  static struct {
> @@ -1608,7 +1610,10 @@ static bool __zone_watermark_ok(struct zone *z, int order, unsigned long mark,
>  		min -= min / 2;
>  	if (alloc_flags & ALLOC_HARDER)
>  		min -= min / 4;
> -
> +#ifdef CONFIG_CMA
> +	if (alloc_flags & ALLOC_CMA)

This should be (!(alloc_flags & ALLOC_CMA)) because we want to decrease
free pages when the flag is not set (== unmovable allocation).

> +		free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
> +#endif
>  	if (free_pages <= min + lowmem_reserve)
>  		return false;
>  	for (o = 0; o < order; o++) {
> @@ -2303,7 +2308,10 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
>  				 unlikely(test_thread_flag(TIF_MEMDIE))))
>  			alloc_flags |= ALLOC_NO_WATERMARKS;
>  	}
> -
> +#ifdef CONFIG_CMA
> +	if (allocflags_to_migratetype(gfp_mask) == MIGRATE_MOVABLE)
> +			alloc_flags |= ALLOC_CMA;
> +#endif
>  	return alloc_flags;
>  }
>  
> @@ -2533,6 +2541,7 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order,
>  	struct page *page = NULL;
>  	int migratetype = allocflags_to_migratetype(gfp_mask);
>  	unsigned int cpuset_mems_cookie;
> +	int alloc_flags = ALLOC_WMARK_LOW|ALLOC_CPUSET;
>  
>  	gfp_mask &= gfp_allowed_mask;
>  
> @@ -2561,9 +2570,13 @@ retry_cpuset:
>  	if (!preferred_zone)
>  		goto out;
>  
> +#ifdef CONFIG_CMA
> +	if (allocflags_to_migratetype(gfp_mask) == MIGRATE_MOVABLE)
> +		alloc_flags |= ALLOC_CMA;
> +#endif
>  	/* First allocation attempt */
>  	page = get_page_from_freelist(gfp_mask|__GFP_HARDWALL, nodemask, order,
> -			zonelist, high_zoneidx, ALLOC_WMARK_LOW|ALLOC_CPUSET,
> +			zonelist, high_zoneidx, alloc_flags,
>  			preferred_zone, migratetype);
>  	if (unlikely(!page))
>  		page = __alloc_pages_slowpath(gfp_mask, order,

Otherwise the change to ALLOC_CMA looks good and I'll do it in the next
revision of the patchset.

Thanks for review & useful ideas!

Best regards,
--
Bartlomiej Zolnierkiewicz
Samsung Poland R&D Center

> > Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> > Cc: Michal Nazarewicz <mina86@mina86.com>
> > Cc: Minchan Kim <minchan@kernel.org>
> > Cc: Mel Gorman <mgorman@suse.de>
> > Cc: Hugh Dickins <hughd@google.com>
> > Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
> > Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
> > ---
> >  include/linux/mmzone.h |  2 +-
> >  mm/compaction.c        | 11 ++++++-----
> >  mm/page_alloc.c        | 29 +++++++++++++++++++----------
> >  mm/vmscan.c            |  4 ++--
> >  4 files changed, 28 insertions(+), 18 deletions(-)
> > 
> > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> > index 904889d..308bb91 100644
> > --- a/include/linux/mmzone.h
> > +++ b/include/linux/mmzone.h
> > @@ -725,7 +725,7 @@ extern struct mutex zonelists_mutex;
> >  void build_all_zonelists(pg_data_t *pgdat, struct zone *zone);
> >  void wakeup_kswapd(struct zone *zone, int order, enum zone_type classzone_idx);
> >  bool zone_watermark_ok(struct zone *z, int order, unsigned long mark,
> > -		int classzone_idx, int alloc_flags);
> > +		int classzone_idx, int alloc_flags, gfp_t gfp_flags);
> >  bool zone_watermark_ok_safe(struct zone *z, int order, unsigned long mark,
> >  		int classzone_idx, int alloc_flags);
> >  enum memmap_context {
> > diff --git a/mm/compaction.c b/mm/compaction.c
> > index 4b902aa..080175a 100644
> > --- a/mm/compaction.c
> > +++ b/mm/compaction.c
> > @@ -684,7 +684,7 @@ static int compact_finished(struct zone *zone,
> >  	watermark = low_wmark_pages(zone);
> >  	watermark += (1 << cc->order);
> >  
> > -	if (!zone_watermark_ok(zone, cc->order, watermark, 0, 0))
> > +	if (!zone_watermark_ok(zone, cc->order, watermark, 0, 0, 0))
> >  		return COMPACT_CONTINUE;
> >  
> >  	/* Direct compactor: Is a suitable page free? */
> > @@ -726,7 +726,7 @@ unsigned long compaction_suitable(struct zone *zone, int order)
> >  	 * allocated and for a short time, the footprint is higher
> >  	 */
> >  	watermark = low_wmark_pages(zone) + (2UL << order);
> > -	if (!zone_watermark_ok(zone, 0, watermark, 0, 0))
> > +	if (!zone_watermark_ok(zone, 0, watermark, 0, 0, 0))
> >  		return COMPACT_SKIPPED;
> >  
> >  	/*
> > @@ -745,7 +745,7 @@ unsigned long compaction_suitable(struct zone *zone, int order)
> >  		return COMPACT_SKIPPED;
> >  
> >  	if (fragindex == -1000 && zone_watermark_ok(zone, order, watermark,
> > -	    0, 0))
> > +	    0, 0, 0))
> >  		return COMPACT_PARTIAL;
> >  
> >  	return COMPACT_CONTINUE;
> > @@ -889,7 +889,8 @@ unsigned long try_to_compact_pages(struct zonelist *zonelist,
> >  		rc = max(status, rc);
> >  
> >  		/* If a normal allocation would succeed, stop compacting */
> > -		if (zone_watermark_ok(zone, order, low_wmark_pages(zone), 0, 0))
> > +		if (zone_watermark_ok(zone, order, low_wmark_pages(zone), 0, 0,
> > +				      gfp_mask))
> >  			break;
> >  	}
> >  
> > @@ -920,7 +921,7 @@ static int __compact_pgdat(pg_data_t *pgdat, struct compact_control *cc)
> >  
> >  		if (cc->order > 0) {
> >  			int ok = zone_watermark_ok(zone, cc->order,
> > -						low_wmark_pages(zone), 0, 0);
> > +						low_wmark_pages(zone), 0, 0, 0);
> >  			if (ok && cc->order >= zone->compact_order_failed)
> >  				zone->compact_order_failed = cc->order + 1;
> >  			/* Currently async compaction is never deferred. */
> > diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> > index 2166774..5912a8c 100644
> > --- a/mm/page_alloc.c
> > +++ b/mm/page_alloc.c
> > @@ -1423,7 +1423,7 @@ int split_free_page(struct page *page, bool check_wmark)
> >  	if (check_wmark) {
> >  		/* Obey watermarks as if the page was being allocated */
> >  		watermark = low_wmark_pages(zone) + (1 << order);
> > -		if (!zone_watermark_ok(zone, 0, watermark, 0, 0))
> > +		if (!zone_watermark_ok(zone, 0, watermark, 0, 0, 0))
> >  			return 0;
> >  	}
> >  
> > @@ -1628,12 +1628,13 @@ static inline bool should_fail_alloc_page(gfp_t gfp_mask, unsigned int order)
> >   * of the allocation.
> >   */
> >  static bool __zone_watermark_ok(struct zone *z, int order, unsigned long mark,
> > -		      int classzone_idx, int alloc_flags, long free_pages, long free_cma_pages)
> > +		      int classzone_idx, int alloc_flags, long free_pages,
> > +		      long free_cma_pages, gfp_t gfp_flags)
> >  {
> >  	/* free_pages my go negative - that's OK */
> >  	long min = mark;
> >  	long lowmem_reserve = z->lowmem_reserve[classzone_idx];
> > -	int o;
> > +	int mt, o;
> >  
> >  	free_pages -= (1 << order) - 1;
> >  	if (alloc_flags & ALLOC_HIGH)
> > @@ -1641,8 +1642,14 @@ static bool __zone_watermark_ok(struct zone *z, int order, unsigned long mark,
> >  	if (alloc_flags & ALLOC_HARDER)
> >  		min -= min / 4;
> >  
> > -	if (free_pages - free_cma_pages <= min + lowmem_reserve)
> > -		return false;
> > +	mt = allocflags_to_migratetype(gfp_flags);
> > +	if (mt == MIGRATE_MOVABLE) {
> > +		if (free_pages <= min + lowmem_reserve)
> > +			return false;
> > +	} else {
> > +		if (free_pages - free_cma_pages <= min + lowmem_reserve)
> > +			return false;
> > +	}
> >  	for (o = 0; o < order; o++) {
> >  		/* At the next order, this order's pages become unavailable */
> >  		free_pages -= z->free_area[o].nr_free << o;
> > @@ -1671,11 +1678,12 @@ static inline unsigned long nr_zone_isolate_freepages(struct zone *zone)
> >  #endif
> >  
> >  bool zone_watermark_ok(struct zone *z, int order, unsigned long mark,
> > -		      int classzone_idx, int alloc_flags)
> > +		      int classzone_idx, int alloc_flags, gfp_t gfp_flags)
> >  {
> >  	return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags,
> >  					zone_page_state(z, NR_FREE_PAGES),
> > -					zone_page_state(z, NR_FREE_CMA_PAGES));
> > +					zone_page_state(z, NR_FREE_CMA_PAGES),
> > +					gfp_flags);
> >  }
> >  
> >  bool zone_watermark_ok_safe(struct zone *z, int order, unsigned long mark,
> > @@ -1696,7 +1704,7 @@ bool zone_watermark_ok_safe(struct zone *z, int order, unsigned long mark,
> >  	 */
> >  	free_pages -= nr_zone_isolate_freepages(z);
> >  	return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags,
> > -					free_pages, free_cma_pages);
> > +					free_pages, free_cma_pages, 0);
> >  }
> >  
> >  #ifdef CONFIG_NUMA
> > @@ -1906,7 +1914,7 @@ zonelist_scan:
> >  
> >  			mark = zone->watermark[alloc_flags & ALLOC_WMARK_MASK];
> >  			if (zone_watermark_ok(zone, order, mark,
> > -				    classzone_idx, alloc_flags))
> > +				    classzone_idx, alloc_flags, gfp_mask))
> >  				goto try_this_zone;
> >  
> >  			if (NUMA_BUILD && !did_zlc_setup && nr_online_nodes > 1) {
> > @@ -1942,7 +1950,8 @@ zonelist_scan:
> >  			default:
> >  				/* did we reclaim enough */
> >  				if (!zone_watermark_ok(zone, order, mark,
> > -						classzone_idx, alloc_flags))
> > +						classzone_idx, alloc_flags,
> > +						gfp_mask))
> >  					goto this_zone_full;
> >  			}
> >  		}
> > diff --git a/mm/vmscan.c b/mm/vmscan.c
> > index 8d01243..4a10038b 100644
> > --- a/mm/vmscan.c
> > +++ b/mm/vmscan.c
> > @@ -2777,14 +2777,14 @@ out:
> >  
> >  			/* Confirm the zone is balanced for order-0 */
> >  			if (!zone_watermark_ok(zone, 0,
> > -					high_wmark_pages(zone), 0, 0)) {
> > +					high_wmark_pages(zone), 0, 0, 0)) {
> >  				order = sc.order = 0;
> >  				goto loop_again;
> >  			}
> >  
> >  			/* Check if the memory needs to be defragmented. */
> >  			if (zone_watermark_ok(zone, order,
> > -				    low_wmark_pages(zone), *classzone_idx, 0))
> > +				    low_wmark_pages(zone), *classzone_idx, 0, 0))
> >  				zones_need_compaction = 0;
> >  
> >  			/* If balanced, clear the congested flag */

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>