[PATCH v3 00/18] Folio conversion of memcg

[PATCH v3 00/18] Folio conversion of memcg

[Matthew Wilcox (Oracle)]

After Michel's comments on "Folio-enabling the page cache", I thought it
best to split out the memcg patches from the rest of the page cache folio
patches and redo them to focus on correctness (ie always passing a folio).

This is fundamentally for review rather than application.  I've rebased
on Linus' current tree, which includes the recent patchbomb from akpm.
That upstream version won't boot on any system I have available, and I'm
not comfortable asking for patches to be applied unless I can actually
try them.  That said, these patches were fine on top of 5.13.

There are still a few functions which take pages, but they rely on other
conversions happening first, which in turn rely on this set of patches,
so I think this is a good place to stop, with the understanding that
there will be more patches later.

Some of the commit logs may be garbled ... I haven't fully taken into
account all of Muchun's recent rework.  All of this work is visible in
context in a git tree here:
https://git.infradead.org/users/willy/pagecache.git/shortlog/refs/heads/folio

Matthew Wilcox (Oracle) (18):
  mm: Add folio_nid()
  mm/memcg: Remove 'page' parameter to mem_cgroup_charge_statistics()
  mm/memcg: Use the node id in mem_cgroup_update_tree()
  mm/memcg: Remove soft_limit_tree_node()
  mm/memcg: Convert memcg_check_events to take a node ID
  mm/memcg: Add folio_memcg() and related functions
  mm/memcg: Convert commit_charge() to take a folio
  mm/memcg: Convert mem_cgroup_charge() to take a folio
  mm/memcg: Convert uncharge_page() to uncharge_folio()
  mm/memcg: Convert mem_cgroup_uncharge() to take a folio
  mm/memcg: Convert mem_cgroup_migrate() to take folios
  mm/memcg: Convert mem_cgroup_track_foreign_dirty_slowpath() to folio
  mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()
  mm/memcg: Convert mem_cgroup_move_account() to use a folio
  mm/memcg: Add mem_cgroup_folio_lruvec()
  mm/memcg: Add folio_lruvec_lock() and similar functions
  mm/memcg: Add folio_lruvec_relock_irq() and
    folio_lruvec_relock_irqsave()
  mm/workingset: Convert workingset_activation to take a folio

 include/linux/memcontrol.h       | 249 +++++++++++++++----------
 include/linux/mm.h               |   5 +
 include/linux/swap.h             |   2 +-
 include/trace/events/writeback.h |   8 +-
 kernel/events/uprobes.c          |   3 +-
 mm/compaction.c                  |   2 +-
 mm/filemap.c                     |   8 +-
 mm/huge_memory.c                 |   2 +-
 mm/khugepaged.c                  |   8 +-
 mm/ksm.c                         |   3 +-
 mm/memcontrol.c                  | 311 +++++++++++++++----------------
 mm/memory-failure.c              |   2 +-
 mm/memory.c                      |   9 +-
 mm/memremap.c                    |   2 +-
 mm/migrate.c                     |   6 +-
 mm/page_alloc.c                  |   2 +-
 mm/shmem.c                       |   7 +-
 mm/swap.c                        |   4 +-
 mm/userfaultfd.c                 |   2 +-
 mm/vmscan.c                      |   2 +-
 mm/workingset.c                  |  10 +-
 21 files changed, 355 insertions(+), 292 deletions(-)

-- 
2.30.2

[PATCH v3 01/18] mm: Add folio_nid()

[Matthew Wilcox (Oracle)]

This is the folio equivalent of page_to_nid().

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 include/linux/mm.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/include/linux/mm.h b/include/linux/mm.h
index 7ad3c8a115ae..48d2ccd9f8c3 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1429,6 +1429,11 @@ static inline int page_to_nid(const struct page *page)
 }
 #endif
 
+static inline int folio_nid(const struct folio *folio)
+{
+	return page_to_nid(&folio->page);
+}
+
 #ifdef CONFIG_NUMA_BALANCING
 static inline int cpu_pid_to_cpupid(int cpu, int pid)
 {
-- 
2.30.2

Re: [PATCH v3 01/18] mm: Add folio_nid()

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>

[PATCH v3 02/18] mm/memcg: Remove 'page' parameter to mem_cgroup_charge_statistics()

[Matthew Wilcox (Oracle)]

The last use of 'page' was removed by commit 468c398233da ("mm:
memcontrol: switch to native NR_ANON_THPS counter"), so we can now remove
the parameter from the function.

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
Acked-by: Michal Hocko <mhocko-IBi9RG/b67k@public.gmane.org>
---
 mm/memcontrol.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 4ee243ce6135..25cad0fb7d4e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -826,7 +826,6 @@ static unsigned long memcg_events_local(struct mem_cgroup *memcg, int event)
 }
 
 static void mem_cgroup_charge_statistics(struct mem_cgroup *memcg,
-					 struct page *page,
 					 int nr_pages)
 {
 	/* pagein of a big page is an event. So, ignore page size */
@@ -5687,9 +5686,9 @@ static int mem_cgroup_move_account(struct page *page,
 	ret = 0;
 
 	local_irq_disable();
-	mem_cgroup_charge_statistics(to, page, nr_pages);
+	mem_cgroup_charge_statistics(to, nr_pages);
 	memcg_check_events(to, page);
-	mem_cgroup_charge_statistics(from, page, -nr_pages);
+	mem_cgroup_charge_statistics(from, -nr_pages);
 	memcg_check_events(from, page);
 	local_irq_enable();
 out_unlock:
@@ -6710,7 +6709,7 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
 	commit_charge(page, memcg);
 
 	local_irq_disable();
-	mem_cgroup_charge_statistics(memcg, page, nr_pages);
+	mem_cgroup_charge_statistics(memcg, nr_pages);
 	memcg_check_events(memcg, page);
 	local_irq_enable();
 out:
@@ -7001,7 +7000,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 	commit_charge(newpage, memcg);
 
 	local_irq_save(flags);
-	mem_cgroup_charge_statistics(memcg, newpage, nr_pages);
+	mem_cgroup_charge_statistics(memcg, nr_pages);
 	memcg_check_events(memcg, newpage);
 	local_irq_restore(flags);
 }
@@ -7231,7 +7230,7 @@ void mem_cgroup_swapout(struct page *page, swp_entry_t entry)
 	 * only synchronisation we have for updating the per-CPU variables.
 	 */
 	VM_BUG_ON(!irqs_disabled());
-	mem_cgroup_charge_statistics(memcg, page, -nr_entries);
+	mem_cgroup_charge_statistics(memcg, -nr_entries);
 	memcg_check_events(memcg, page);
 
 	css_put(&memcg->css);
-- 
2.30.2

Re: [PATCH v3 02/18] mm/memcg: Remove 'page' parameter to mem_cgroup_charge_statistics()

[Johannes Weiner]

On Wed, Jun 30, 2021 at 05:00:18AM +0100, Matthew Wilcox (Oracle) wrote:
> The last use of 'page' was removed by commit 468c398233da ("mm:
> memcontrol: switch to native NR_ANON_THPS counter"), so we can now remove
> the parameter from the function.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
> Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
> Acked-by: Michal Hocko <mhocko-IBi9RG/b67k@public.gmane.org>

Acked-by: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>

[PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree()

[Matthew Wilcox (Oracle)]

By using the node id in mem_cgroup_update_tree(), we can delete
soft_limit_tree_from_page() and mem_cgroup_page_nodeinfo().  Saves 42
bytes of kernel text on my config.

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 mm/memcontrol.c | 24 ++++--------------------
 1 file changed, 4 insertions(+), 20 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 25cad0fb7d4e..29b28a050707 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -446,28 +446,12 @@ ino_t page_cgroup_ino(struct page *page)
 	return ino;
 }
 
-static struct mem_cgroup_per_node *
-mem_cgroup_page_nodeinfo(struct mem_cgroup *memcg, struct page *page)
-{
-	int nid = page_to_nid(page);
-
-	return memcg->nodeinfo[nid];
-}
-
 static struct mem_cgroup_tree_per_node *
 soft_limit_tree_node(int nid)
 {
 	return soft_limit_tree.rb_tree_per_node[nid];
 }
 
-static struct mem_cgroup_tree_per_node *
-soft_limit_tree_from_page(struct page *page)
-{
-	int nid = page_to_nid(page);
-
-	return soft_limit_tree.rb_tree_per_node[nid];
-}
-
 static void __mem_cgroup_insert_exceeded(struct mem_cgroup_per_node *mz,
 					 struct mem_cgroup_tree_per_node *mctz,
 					 unsigned long new_usage_in_excess)
@@ -538,13 +522,13 @@ static unsigned long soft_limit_excess(struct mem_cgroup *memcg)
 	return excess;
 }
 
-static void mem_cgroup_update_tree(struct mem_cgroup *memcg, struct page *page)
+static void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid)
 {
 	unsigned long excess;
 	struct mem_cgroup_per_node *mz;
 	struct mem_cgroup_tree_per_node *mctz;
 
-	mctz = soft_limit_tree_from_page(page);
+	mctz = soft_limit_tree_node(nid);
 	if (!mctz)
 		return;
 	/*
@@ -552,7 +536,7 @@ static void mem_cgroup_update_tree(struct mem_cgroup *memcg, struct page *page)
 	 * because their event counter is not touched.
 	 */
 	for (; memcg; memcg = parent_mem_cgroup(memcg)) {
-		mz = mem_cgroup_page_nodeinfo(memcg, page);
+		mz = memcg->nodeinfo[nid];
 		excess = soft_limit_excess(memcg);
 		/*
 		 * We have to update the tree if mz is on RB-tree or
@@ -879,7 +863,7 @@ static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
 						MEM_CGROUP_TARGET_SOFTLIMIT);
 		mem_cgroup_threshold(memcg);
 		if (unlikely(do_softlimit))
-			mem_cgroup_update_tree(memcg, page);
+			mem_cgroup_update_tree(memcg, page_to_nid(page));
 	}
 }
 
-- 
2.30.2

Re: [PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree()

[Michal Hocko]

On Wed 30-06-21 05:00:19, Matthew Wilcox wrote:
> By using the node id in mem_cgroup_update_tree(), we can delete
> soft_limit_tree_from_page() and mem_cgroup_page_nodeinfo().  Saves 42
> bytes of kernel text on my config.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>

Acked-by: Michal Hocko <mhocko-IBi9RG/b67k@public.gmane.org>

Thanks!

> ---
>  mm/memcontrol.c | 24 ++++--------------------
>  1 file changed, 4 insertions(+), 20 deletions(-)
> 
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 25cad0fb7d4e..29b28a050707 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -446,28 +446,12 @@ ino_t page_cgroup_ino(struct page *page)
>  	return ino;
>  }
>  
> -static struct mem_cgroup_per_node *
> -mem_cgroup_page_nodeinfo(struct mem_cgroup *memcg, struct page *page)
> -{
> -	int nid = page_to_nid(page);
> -
> -	return memcg->nodeinfo[nid];
> -}
> -
>  static struct mem_cgroup_tree_per_node *
>  soft_limit_tree_node(int nid)
>  {
>  	return soft_limit_tree.rb_tree_per_node[nid];
>  }
>  
> -static struct mem_cgroup_tree_per_node *
> -soft_limit_tree_from_page(struct page *page)
> -{
> -	int nid = page_to_nid(page);
> -
> -	return soft_limit_tree.rb_tree_per_node[nid];
> -}
> -
>  static void __mem_cgroup_insert_exceeded(struct mem_cgroup_per_node *mz,
>  					 struct mem_cgroup_tree_per_node *mctz,
>  					 unsigned long new_usage_in_excess)
> @@ -538,13 +522,13 @@ static unsigned long soft_limit_excess(struct mem_cgroup *memcg)
>  	return excess;
>  }
>  
> -static void mem_cgroup_update_tree(struct mem_cgroup *memcg, struct page *page)
> +static void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid)
>  {
>  	unsigned long excess;
>  	struct mem_cgroup_per_node *mz;
>  	struct mem_cgroup_tree_per_node *mctz;
>  
> -	mctz = soft_limit_tree_from_page(page);
> +	mctz = soft_limit_tree_node(nid);
>  	if (!mctz)
>  		return;
>  	/*
> @@ -552,7 +536,7 @@ static void mem_cgroup_update_tree(struct mem_cgroup *memcg, struct page *page)
>  	 * because their event counter is not touched.
>  	 */
>  	for (; memcg; memcg = parent_mem_cgroup(memcg)) {
> -		mz = mem_cgroup_page_nodeinfo(memcg, page);
> +		mz = memcg->nodeinfo[nid];
>  		excess = soft_limit_excess(memcg);
>  		/*
>  		 * We have to update the tree if mz is on RB-tree or
> @@ -879,7 +863,7 @@ static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
>  						MEM_CGROUP_TARGET_SOFTLIMIT);
>  		mem_cgroup_threshold(memcg);
>  		if (unlikely(do_softlimit))
> -			mem_cgroup_update_tree(memcg, page);
> +			mem_cgroup_update_tree(memcg, page_to_nid(page));
>  	}
>  }
>  
> -- 
> 2.30.2

-- 
Michal Hocko
SUSE Labs

Re: [PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree()

[Johannes Weiner]

On Wed, Jun 30, 2021 at 05:00:19AM +0100, Matthew Wilcox (Oracle) wrote:
> By using the node id in mem_cgroup_update_tree(), we can delete
> soft_limit_tree_from_page() and mem_cgroup_page_nodeinfo().  Saves 42
> bytes of kernel text on my config.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>

Acked-by: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>

Re: [PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree()

[Christoph Hellwig]

On Wed, Jun 30, 2021 at 05:00:19AM +0100, Matthew Wilcox (Oracle) wrote:
> By using the node id in mem_cgroup_update_tree(), we can delete
> soft_limit_tree_from_page() and mem_cgroup_page_nodeinfo().  Saves 42
> bytes of kernel text on my config.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>

[PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node()

[Matthew Wilcox (Oracle)]

Opencode this one-line function in its three callers.

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 mm/memcontrol.c | 12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 29b28a050707..29fdb70dca42 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -446,12 +446,6 @@ ino_t page_cgroup_ino(struct page *page)
 	return ino;
 }
 
-static struct mem_cgroup_tree_per_node *
-soft_limit_tree_node(int nid)
-{
-	return soft_limit_tree.rb_tree_per_node[nid];
-}
-
 static void __mem_cgroup_insert_exceeded(struct mem_cgroup_per_node *mz,
 					 struct mem_cgroup_tree_per_node *mctz,
 					 unsigned long new_usage_in_excess)
@@ -528,7 +522,7 @@ static void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid)
 	struct mem_cgroup_per_node *mz;
 	struct mem_cgroup_tree_per_node *mctz;
 
-	mctz = soft_limit_tree_node(nid);
+	mctz = soft_limit_tree.rb_tree_per_node[nid];
 	if (!mctz)
 		return;
 	/*
@@ -567,7 +561,7 @@ static void mem_cgroup_remove_from_trees(struct mem_cgroup *memcg)
 
 	for_each_node(nid) {
 		mz = memcg->nodeinfo[nid];
-		mctz = soft_limit_tree_node(nid);
+		mctz = soft_limit_tree.rb_tree_per_node[nid];
 		if (mctz)
 			mem_cgroup_remove_exceeded(mz, mctz);
 	}
@@ -3415,7 +3409,7 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
 	if (order > 0)
 		return 0;
 
-	mctz = soft_limit_tree_node(pgdat->node_id);
+	mctz = soft_limit_tree.rb_tree_per_node[pgdat->node_id];
 
 	/*
 	 * Do not even bother to check the largest node if the root
-- 
2.30.2

Re: [PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node()

[Michal Hocko]

On Wed 30-06-21 05:00:20, Matthew Wilcox wrote:
> Opencode this one-line function in its three callers.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>

Acked-by: Michal Hocko <mhocko-IBi9RG/b67k@public.gmane.org>

> ---
>  mm/memcontrol.c | 12 +++---------
>  1 file changed, 3 insertions(+), 9 deletions(-)
> 
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 29b28a050707..29fdb70dca42 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -446,12 +446,6 @@ ino_t page_cgroup_ino(struct page *page)
>  	return ino;
>  }
>  
> -static struct mem_cgroup_tree_per_node *
> -soft_limit_tree_node(int nid)
> -{
> -	return soft_limit_tree.rb_tree_per_node[nid];
> -}
> -
>  static void __mem_cgroup_insert_exceeded(struct mem_cgroup_per_node *mz,
>  					 struct mem_cgroup_tree_per_node *mctz,
>  					 unsigned long new_usage_in_excess)
> @@ -528,7 +522,7 @@ static void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid)
>  	struct mem_cgroup_per_node *mz;
>  	struct mem_cgroup_tree_per_node *mctz;
>  
> -	mctz = soft_limit_tree_node(nid);
> +	mctz = soft_limit_tree.rb_tree_per_node[nid];
>  	if (!mctz)
>  		return;
>  	/*
> @@ -567,7 +561,7 @@ static void mem_cgroup_remove_from_trees(struct mem_cgroup *memcg)
>  
>  	for_each_node(nid) {
>  		mz = memcg->nodeinfo[nid];
> -		mctz = soft_limit_tree_node(nid);
> +		mctz = soft_limit_tree.rb_tree_per_node[nid];
>  		if (mctz)
>  			mem_cgroup_remove_exceeded(mz, mctz);
>  	}
> @@ -3415,7 +3409,7 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
>  	if (order > 0)
>  		return 0;
>  
> -	mctz = soft_limit_tree_node(pgdat->node_id);
> +	mctz = soft_limit_tree.rb_tree_per_node[pgdat->node_id];
>  
>  	/*
>  	 * Do not even bother to check the largest node if the root
> -- 
> 2.30.2

-- 
Michal Hocko
SUSE Labs

Re: [PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node()

[Johannes Weiner]

On Wed, Jun 30, 2021 at 05:00:20AM +0100, Matthew Wilcox (Oracle) wrote:
> Opencode this one-line function in its three callers.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>

Acked-by: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>

Re: [PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node()

[Christoph Hellwig]

On Wed, Jun 30, 2021 at 05:00:20AM +0100, Matthew Wilcox (Oracle) wrote:
> Opencode this one-line function in its three callers.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>

[PATCH v3 05/18] mm/memcg: Convert memcg_check_events to take a node ID

[Matthew Wilcox (Oracle)]

memcg_check_events only uses the page's nid, so call page_to_nid in the
callers to make the folio conversion easier.

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 mm/memcontrol.c | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 29fdb70dca42..5d143d46a8a4 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -846,7 +846,7 @@ static bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg,
  * Check events in order.
  *
  */
-static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
+static void memcg_check_events(struct mem_cgroup *memcg, int nid)
 {
 	/* threshold event is triggered in finer grain than soft limit */
 	if (unlikely(mem_cgroup_event_ratelimit(memcg,
@@ -857,7 +857,7 @@ static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
 						MEM_CGROUP_TARGET_SOFTLIMIT);
 		mem_cgroup_threshold(memcg);
 		if (unlikely(do_softlimit))
-			mem_cgroup_update_tree(memcg, page_to_nid(page));
+			mem_cgroup_update_tree(memcg, nid);
 	}
 }
 
@@ -5573,7 +5573,7 @@ static int mem_cgroup_move_account(struct page *page,
 	struct lruvec *from_vec, *to_vec;
 	struct pglist_data *pgdat;
 	unsigned int nr_pages = compound ? thp_nr_pages(page) : 1;
-	int ret;
+	int nid, ret;
 
 	VM_BUG_ON(from == to);
 	VM_BUG_ON_PAGE(PageLRU(page), page);
@@ -5662,12 +5662,13 @@ static int mem_cgroup_move_account(struct page *page,
 	__unlock_page_memcg(from);
 
 	ret = 0;
+	nid = page_to_nid(page);
 
 	local_irq_disable();
 	mem_cgroup_charge_statistics(to, nr_pages);
-	memcg_check_events(to, page);
+	memcg_check_events(to, nid);
 	mem_cgroup_charge_statistics(from, -nr_pages);
-	memcg_check_events(from, page);
+	memcg_check_events(from, nid);
 	local_irq_enable();
 out_unlock:
 	unlock_page(page);
@@ -6688,7 +6689,7 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
 
 	local_irq_disable();
 	mem_cgroup_charge_statistics(memcg, nr_pages);
-	memcg_check_events(memcg, page);
+	memcg_check_events(memcg, page_to_nid(page));
 	local_irq_enable();
 out:
 	return ret;
@@ -6796,7 +6797,7 @@ struct uncharge_gather {
 	unsigned long nr_memory;
 	unsigned long pgpgout;
 	unsigned long nr_kmem;
-	struct page *dummy_page;
+	int nid;
 };
 
 static inline void uncharge_gather_clear(struct uncharge_gather *ug)
@@ -6820,7 +6821,7 @@ static void uncharge_batch(const struct uncharge_gather *ug)
 	local_irq_save(flags);
 	__count_memcg_events(ug->memcg, PGPGOUT, ug->pgpgout);
 	__this_cpu_add(ug->memcg->vmstats_percpu->nr_page_events, ug->nr_memory);
-	memcg_check_events(ug->memcg, ug->dummy_page);
+	memcg_check_events(ug->memcg, ug->nid);
 	local_irq_restore(flags);
 
 	/* drop reference from uncharge_page */
@@ -6861,7 +6862,7 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
 			uncharge_gather_clear(ug);
 		}
 		ug->memcg = memcg;
-		ug->dummy_page = page;
+		ug->nid = page_to_nid(page);
 
 		/* pairs with css_put in uncharge_batch */
 		css_get(&memcg->css);
@@ -6979,7 +6980,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 
 	local_irq_save(flags);
 	mem_cgroup_charge_statistics(memcg, nr_pages);
-	memcg_check_events(memcg, newpage);
+	memcg_check_events(memcg, page_to_nid(newpage));
 	local_irq_restore(flags);
 }
 
@@ -7209,7 +7210,7 @@ void mem_cgroup_swapout(struct page *page, swp_entry_t entry)
 	 */
 	VM_BUG_ON(!irqs_disabled());
 	mem_cgroup_charge_statistics(memcg, -nr_entries);
-	memcg_check_events(memcg, page);
+	memcg_check_events(memcg, page_to_nid(page));
 
 	css_put(&memcg->css);
 }
-- 
2.30.2

Re: [PATCH v3 05/18] mm/memcg: Convert memcg_check_events to take a node ID

[Michal Hocko]

On Wed 30-06-21 05:00:21, Matthew Wilcox wrote:
> memcg_check_events only uses the page's nid, so call page_to_nid in the
> callers to make the folio conversion easier.

It will also make the interface slightly easier to follow as there
shouldn't be any real reason to take the page for these events.
So this is a good cleanup in general.

> Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>

Acked-by: Michal Hocko <mhocko-IBi9RG/b67k@public.gmane.org>

Thanks.
> ---
>  mm/memcontrol.c | 23 ++++++++++++-----------
>  1 file changed, 12 insertions(+), 11 deletions(-)
> 
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 29fdb70dca42..5d143d46a8a4 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -846,7 +846,7 @@ static bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg,
>   * Check events in order.
>   *
>   */
> -static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
> +static void memcg_check_events(struct mem_cgroup *memcg, int nid)
>  {
>  	/* threshold event is triggered in finer grain than soft limit */
>  	if (unlikely(mem_cgroup_event_ratelimit(memcg,
> @@ -857,7 +857,7 @@ static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
>  						MEM_CGROUP_TARGET_SOFTLIMIT);
>  		mem_cgroup_threshold(memcg);
>  		if (unlikely(do_softlimit))
> -			mem_cgroup_update_tree(memcg, page_to_nid(page));
> +			mem_cgroup_update_tree(memcg, nid);
>  	}
>  }
>  
> @@ -5573,7 +5573,7 @@ static int mem_cgroup_move_account(struct page *page,
>  	struct lruvec *from_vec, *to_vec;
>  	struct pglist_data *pgdat;
>  	unsigned int nr_pages = compound ? thp_nr_pages(page) : 1;
> -	int ret;
> +	int nid, ret;
>  
>  	VM_BUG_ON(from == to);
>  	VM_BUG_ON_PAGE(PageLRU(page), page);
> @@ -5662,12 +5662,13 @@ static int mem_cgroup_move_account(struct page *page,
>  	__unlock_page_memcg(from);
>  
>  	ret = 0;
> +	nid = page_to_nid(page);
>  
>  	local_irq_disable();
>  	mem_cgroup_charge_statistics(to, nr_pages);
> -	memcg_check_events(to, page);
> +	memcg_check_events(to, nid);
>  	mem_cgroup_charge_statistics(from, -nr_pages);
> -	memcg_check_events(from, page);
> +	memcg_check_events(from, nid);
>  	local_irq_enable();
>  out_unlock:
>  	unlock_page(page);
> @@ -6688,7 +6689,7 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
>  
>  	local_irq_disable();
>  	mem_cgroup_charge_statistics(memcg, nr_pages);
> -	memcg_check_events(memcg, page);
> +	memcg_check_events(memcg, page_to_nid(page));
>  	local_irq_enable();
>  out:
>  	return ret;
> @@ -6796,7 +6797,7 @@ struct uncharge_gather {
>  	unsigned long nr_memory;
>  	unsigned long pgpgout;
>  	unsigned long nr_kmem;
> -	struct page *dummy_page;
> +	int nid;
>  };
>  
>  static inline void uncharge_gather_clear(struct uncharge_gather *ug)
> @@ -6820,7 +6821,7 @@ static void uncharge_batch(const struct uncharge_gather *ug)
>  	local_irq_save(flags);
>  	__count_memcg_events(ug->memcg, PGPGOUT, ug->pgpgout);
>  	__this_cpu_add(ug->memcg->vmstats_percpu->nr_page_events, ug->nr_memory);
> -	memcg_check_events(ug->memcg, ug->dummy_page);
> +	memcg_check_events(ug->memcg, ug->nid);
>  	local_irq_restore(flags);
>  
>  	/* drop reference from uncharge_page */
> @@ -6861,7 +6862,7 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
>  			uncharge_gather_clear(ug);
>  		}
>  		ug->memcg = memcg;
> -		ug->dummy_page = page;
> +		ug->nid = page_to_nid(page);
>  
>  		/* pairs with css_put in uncharge_batch */
>  		css_get(&memcg->css);
> @@ -6979,7 +6980,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
>  
>  	local_irq_save(flags);
>  	mem_cgroup_charge_statistics(memcg, nr_pages);
> -	memcg_check_events(memcg, newpage);
> +	memcg_check_events(memcg, page_to_nid(newpage));
>  	local_irq_restore(flags);
>  }
>  
> @@ -7209,7 +7210,7 @@ void mem_cgroup_swapout(struct page *page, swp_entry_t entry)
>  	 */
>  	VM_BUG_ON(!irqs_disabled());
>  	mem_cgroup_charge_statistics(memcg, -nr_entries);
> -	memcg_check_events(memcg, page);
> +	memcg_check_events(memcg, page_to_nid(page));
>  
>  	css_put(&memcg->css);
>  }
> -- 
> 2.30.2

-- 
Michal Hocko
SUSE Labs

Re: [PATCH v3 05/18] mm/memcg: Convert memcg_check_events to take a node ID

[Michal Hocko]

On Wed 30-06-21 08:58:20, Michal Hocko wrote:
> On Wed 30-06-21 05:00:21, Matthew Wilcox wrote:
> > memcg_check_events only uses the page's nid, so call page_to_nid in the
> > callers to make the folio conversion easier.
> 
> It will also make the interface slightly easier to follow as there
> shouldn't be any real reason to take the page for these events.
> So this is a good cleanup in general.
> 
> > Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
> 
> Acked-by: Michal Hocko <mhocko-IBi9RG/b67k@public.gmane.org>

Btw. patches 2-5 seem good enough to go to Andrew even without the rest
so that you do not have to carry them along with the rest which is quite
large I can imagine.
-- 
Michal Hocko
SUSE Labs

Re: [PATCH v3 05/18] mm/memcg: Convert memcg_check_events to take a node ID

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>

[PATCH v3 06/18] mm/memcg: Add folio_memcg() and related functions

[Matthew Wilcox (Oracle)]

memcg information is only stored in the head page, so the memcg
subsystem needs to assure that all accesses are to the head page.
The first step is converting page_memcg() to folio_memcg().

Retain page_memcg() as a wrapper around folio_memcg() and PageMemcgKmem()
as a wrapper around folio_memcg_kmem() but convert __page_memcg() to
__folio_memcg() and __page_objcg() to __folio_objcg().

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 include/linux/memcontrol.h | 105 +++++++++++++++++++++----------------
 mm/memcontrol.c            |  21 ++++----
 2 files changed, 73 insertions(+), 53 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 6d66037be646..92689fb2dab4 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -372,6 +372,7 @@ enum page_memcg_data_flags {
 #define MEMCG_DATA_FLAGS_MASK (__NR_MEMCG_DATA_FLAGS - 1)
 
 static inline bool PageMemcgKmem(struct page *page);
+static inline bool folio_memcg_kmem(struct folio *folio);
 
 /*
  * After the initialization objcg->memcg is always pointing at
@@ -386,73 +387,78 @@ static inline struct mem_cgroup *obj_cgroup_memcg(struct obj_cgroup *objcg)
 }
 
 /*
- * __page_memcg - get the memory cgroup associated with a non-kmem page
- * @page: a pointer to the page struct
+ * __folio_memcg - Get the memory cgroup associated with a non-kmem folio
+ * @folio: Pointer to the folio.
  *
- * Returns a pointer to the memory cgroup associated with the page,
- * or NULL. This function assumes that the page is known to have a
+ * Returns a pointer to the memory cgroup associated with the folio,
+ * or NULL. This function assumes that the folio is known to have a
  * proper memory cgroup pointer. It's not safe to call this function
- * against some type of pages, e.g. slab pages or ex-slab pages or
- * kmem pages.
+ * against some type of folios, e.g. slab folios or ex-slab folios or
+ * kmem folios.
  */
-static inline struct mem_cgroup *__page_memcg(struct page *page)
+static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
 {
-	unsigned long memcg_data = page->memcg_data;
+	unsigned long memcg_data = folio->memcg_data;
 
-	VM_BUG_ON_PAGE(PageSlab(page), page);
-	VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_OBJCGS, page);
-	VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_KMEM, page);
+	VM_BUG_ON_FOLIO(folio_slab(folio), folio);
+	VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
+	VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_KMEM, folio);
 
 	return (struct mem_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
 }
 
 /*
- * __page_objcg - get the object cgroup associated with a kmem page
- * @page: a pointer to the page struct
+ * __folio_objcg - get the object cgroup associated with a kmem folio.
+ * @folio: Pointer to the folio.
  *
- * Returns a pointer to the object cgroup associated with the page,
- * or NULL. This function assumes that the page is known to have a
+ * Returns a pointer to the object cgroup associated with the folio,
+ * or NULL. This function assumes that the folio is known to have a
  * proper object cgroup pointer. It's not safe to call this function
- * against some type of pages, e.g. slab pages or ex-slab pages or
- * LRU pages.
+ * against some type of folios, e.g. slab folios or ex-slab folios or
+ * LRU folios.
  */
-static inline struct obj_cgroup *__page_objcg(struct page *page)
+static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
 {
-	unsigned long memcg_data = page->memcg_data;
+	unsigned long memcg_data = folio->memcg_data;
 
-	VM_BUG_ON_PAGE(PageSlab(page), page);
-	VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_OBJCGS, page);
-	VM_BUG_ON_PAGE(!(memcg_data & MEMCG_DATA_KMEM), page);
+	VM_BUG_ON_FOLIO(folio_slab(folio), folio);
+	VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
+	VM_BUG_ON_FOLIO(!(memcg_data & MEMCG_DATA_KMEM), folio);
 
 	return (struct obj_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
 }
 
 /*
- * page_memcg - get the memory cgroup associated with a page
- * @page: a pointer to the page struct
+ * folio_memcg - Get the memory cgroup associated with a folio.
+ * @folio: Pointer to the folio.
  *
- * Returns a pointer to the memory cgroup associated with the page,
- * or NULL. This function assumes that the page is known to have a
+ * Returns a pointer to the memory cgroup associated with the folio,
+ * or NULL. This function assumes that the folio is known to have a
  * proper memory cgroup pointer. It's not safe to call this function
- * against some type of pages, e.g. slab pages or ex-slab pages.
+ * against some type of folios, e.g. slab folios or ex-slab folios.
  *
- * For a non-kmem page any of the following ensures page and memcg binding
+ * For a non-kmem folio any of the following ensures folio and memcg binding
  * stability:
  *
- * - the page lock
+ * - the folio lock
  * - LRU isolation
  * - lock_page_memcg()
  * - exclusive reference
  *
- * For a kmem page a caller should hold an rcu read lock to protect memcg
- * associated with a kmem page from being released.
+ * For a kmem folio a caller should hold an rcu read lock to protect memcg
+ * associated with a kmem folio from being released.
  */
-static inline struct mem_cgroup *page_memcg(struct page *page)
+static inline struct mem_cgroup *folio_memcg(struct folio *folio)
 {
-	if (PageMemcgKmem(page))
-		return obj_cgroup_memcg(__page_objcg(page));
+	if (folio_memcg_kmem(folio))
+		return obj_cgroup_memcg(__folio_objcg(folio));
 	else
-		return __page_memcg(page);
+		return __folio_memcg(folio);
+}
+
+static inline struct mem_cgroup *page_memcg(struct page *page)
+{
+	return folio_memcg(page_folio(page));
 }
 
 /*
@@ -525,17 +531,18 @@ static inline struct mem_cgroup *page_memcg_check(struct page *page)
 
 #ifdef CONFIG_MEMCG_KMEM
 /*
- * PageMemcgKmem - check if the page has MemcgKmem flag set
- * @page: a pointer to the page struct
+ * folio_memcg_kmem - Check if the folio has the memcg_kmem flag set.
+ * @folio: Pointer to the folio.
  *
- * Checks if the page has MemcgKmem flag set. The caller must ensure that
- * the page has an associated memory cgroup. It's not safe to call this function
- * against some types of pages, e.g. slab pages.
+ * Checks if the folio has MemcgKmem flag set. The caller must ensure
+ * that the folio has an associated memory cgroup. It's not safe to call
+ * this function against some types of folios, e.g. slab folios.
  */
-static inline bool PageMemcgKmem(struct page *page)
+static inline bool folio_memcg_kmem(struct folio *folio)
 {
-	VM_BUG_ON_PAGE(page->memcg_data & MEMCG_DATA_OBJCGS, page);
-	return page->memcg_data & MEMCG_DATA_KMEM;
+	VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
+	VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
+	return folio->memcg_data & MEMCG_DATA_KMEM;
 }
 
 /*
@@ -579,7 +586,7 @@ static inline struct obj_cgroup **page_objcgs_check(struct page *page)
 }
 
 #else
-static inline bool PageMemcgKmem(struct page *page)
+static inline bool folio_memcg_kmem(struct folio *folio)
 {
 	return false;
 }
@@ -595,6 +602,11 @@ static inline struct obj_cgroup **page_objcgs_check(struct page *page)
 }
 #endif
 
+static inline bool PageMemcgKmem(struct page *page)
+{
+	return folio_memcg_kmem(page_folio(page));
+}
+
 static __always_inline bool memcg_stat_item_in_bytes(int idx)
 {
 	if (idx == MEMCG_PERCPU_B)
@@ -1122,6 +1134,11 @@ static inline struct mem_cgroup *page_memcg_check(struct page *page)
 	return NULL;
 }
 
+static inline bool folio_memcg_kmem(struct folio *folio)
+{
+	return false;
+}
+
 static inline bool PageMemcgKmem(struct page *page)
 {
 	return false;
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 5d143d46a8a4..f369bbaf584b 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -3045,15 +3045,16 @@ int __memcg_kmem_charge_page(struct page *page, gfp_t gfp, int order)
  */
 void __memcg_kmem_uncharge_page(struct page *page, int order)
 {
+	struct folio *folio = page_folio(page);
 	struct obj_cgroup *objcg;
 	unsigned int nr_pages = 1 << order;
 
-	if (!PageMemcgKmem(page))
+	if (!folio_memcg_kmem(folio))
 		return;
 
-	objcg = __page_objcg(page);
+	objcg = __folio_objcg(folio);
 	obj_cgroup_uncharge_pages(objcg, nr_pages);
-	page->memcg_data = 0;
+	folio->memcg_data = 0;
 	obj_cgroup_put(objcg);
 }
 
@@ -3285,17 +3286,18 @@ void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size)
  */
 void split_page_memcg(struct page *head, unsigned int nr)
 {
-	struct mem_cgroup *memcg = page_memcg(head);
+	struct folio *folio = page_folio(head);
+	struct mem_cgroup *memcg = folio_memcg(folio);
 	int i;
 
 	if (mem_cgroup_disabled() || !memcg)
 		return;
 
 	for (i = 1; i < nr; i++)
-		head[i].memcg_data = head->memcg_data;
+		folio_page(folio, i)->memcg_data = folio->memcg_data;
 
-	if (PageMemcgKmem(head))
-		obj_cgroup_get_many(__page_objcg(head), nr - 1);
+	if (folio_memcg_kmem(folio))
+		obj_cgroup_get_many(__folio_objcg(folio), nr - 1);
 	else
 		css_get_many(&memcg->css, nr - 1);
 }
@@ -6830,6 +6832,7 @@ static void uncharge_batch(const struct uncharge_gather *ug)
 
 static void uncharge_page(struct page *page, struct uncharge_gather *ug)
 {
+	struct folio *folio = page_folio(page);
 	unsigned long nr_pages;
 	struct mem_cgroup *memcg;
 	struct obj_cgroup *objcg;
@@ -6843,14 +6846,14 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
 	 * exclusive access to the page.
 	 */
 	if (use_objcg) {
-		objcg = __page_objcg(page);
+		objcg = __folio_objcg(folio);
 		/*
 		 * This get matches the put at the end of the function and
 		 * kmem pages do not hold memcg references anymore.
 		 */
 		memcg = get_mem_cgroup_from_objcg(objcg);
 	} else {
-		memcg = __page_memcg(page);
+		memcg = __folio_memcg(folio);
 	}
 
 	if (!memcg)
-- 
2.30.2

Re: [PATCH v3 06/18] mm/memcg: Add folio_memcg() and related functions

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 31770 bytes --]

Hi "Matthew,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[also build test ERROR on next-20210629]
[cannot apply to hnaz-linux-mm/master tip/perf/core cgroup/for-next v5.13]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 007b350a58754a93ca9fe50c498cc27780171153
config: x86_64-randconfig-a003-20210628 (attached as .config)
compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project 8d21d5472501460933e78aead04cf59579025ba4)
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # install x86_64 cross compiling tool for clang build
        # apt-get install binutils-x86-64-linux-gnu
        # https://github.com/0day-ci/linux/commit/615adf6169006fa1bc5e19d2105ff9a333bef423
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
        git checkout 615adf6169006fa1bc5e19d2105ff9a333bef423
        # save the attached .config to linux build tree
        mkdir build_dir
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross O=build_dir ARCH=x86_64 olddefconfig

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

All errors (new ones prefixed by >>):

   In file included from arch/x86/kernel/asm-offsets.c:13:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   In file included from include/linux/memcontrol.h:13:
   In file included from include/linux/cgroup.h:28:
   In file included from include/linux/cgroup-defs.h:22:
   In file included from include/linux/bpf-cgroup.h:5:
   In file included from include/linux/bpf.h:20:
   In file included from include/linux/kallsyms.h:12:
   include/linux/mm.h:1382:42: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline int folio_nid(const struct folio *folio)
                                            ^
   include/linux/mm.h:1384:27: error: incomplete definition of type 'struct folio'
           return page_to_nid(&folio->page);
                               ~~~~~^
   include/linux/mm.h:1382:42: note: forward declaration of 'struct folio'
   static inline int folio_nid(const struct folio *folio)
                                            ^
   In file included from arch/x86/kernel/asm-offsets.c:13:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:375:44: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline bool folio_memcg_kmem(struct folio *folio);
                                              ^
   include/linux/memcontrol.h:399:55: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
                                                         ^
>> include/linux/memcontrol.h:401:34: error: incomplete definition of type 'struct folio'
           unsigned long memcg_data = folio->memcg_data;
                                      ~~~~~^
   include/linux/memcontrol.h:399:55: note: forward declaration of 'struct folio'
   static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
                                                         ^
>> include/linux/memcontrol.h:403:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio_slab(folio), folio);
           ^
>> include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio_slab(folio), folio);
                           ^
   include/linux/memcontrol.h:420:55: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
                                                         ^
   include/linux/memcontrol.h:422:34: error: incomplete definition of type 'struct folio'
           unsigned long memcg_data = folio->memcg_data;
                                      ~~~~~^
   include/linux/memcontrol.h:420:55: note: forward declaration of 'struct folio'
   static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
                                                         ^
   include/linux/memcontrol.h:424:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio_slab(folio), folio);
           ^
   include/linux/memcontrol.h:424:18: error: implicit declaration of function 'folio_slab' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio_slab(folio), folio);
                           ^
   include/linux/memcontrol.h:451:53: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline struct mem_cgroup *folio_memcg(struct folio *folio)
                                                       ^
   include/linux/memcontrol.h:453:23: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
           if (folio_memcg_kmem(folio))
                                ^~~~~
   include/linux/memcontrol.h:375:51: note: passing argument to parameter 'folio' here
   static inline bool folio_memcg_kmem(struct folio *folio);
                                                     ^
   include/linux/memcontrol.h:454:41: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
                   return obj_cgroup_memcg(__folio_objcg(folio));
                                                         ^~~~~
   include/linux/memcontrol.h:420:62: note: passing argument to parameter 'folio' here
   static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
                                                                ^
   include/linux/memcontrol.h:456:24: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
                   return __folio_memcg(folio);
                                        ^~~~~
   include/linux/memcontrol.h:399:62: note: passing argument to parameter 'folio' here
   static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
                                                                ^
   include/linux/memcontrol.h:461:21: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration]
           return folio_memcg(page_folio(page));
                              ^
   include/linux/memcontrol.h:461:21: warning: incompatible integer to pointer conversion passing 'int' to parameter of type 'struct folio *' [-Wint-conversion]
           return folio_memcg(page_folio(page));
                              ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:451:60: note: passing argument to parameter 'folio' here
   static inline struct mem_cgroup *folio_memcg(struct folio *folio)
                                                              ^
   include/linux/memcontrol.h:541:44: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline bool folio_memcg_kmem(struct folio *folio)
                                              ^
   include/linux/memcontrol.h:541:20: error: conflicting types for 'folio_memcg_kmem'
   static inline bool folio_memcg_kmem(struct folio *folio)
                      ^
   include/linux/memcontrol.h:375:20: note: previous declaration is here
   static inline bool folio_memcg_kmem(struct folio *folio);
                      ^
   include/linux/memcontrol.h:543:35: error: incomplete definition of type 'struct folio'
           VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
                                       ~~~~~^
   include/linux/mmdebug.h:76:60: note: expanded from macro 'VM_BUG_ON_PGFLAGS'
   #define VM_BUG_ON_PGFLAGS(cond, page) BUILD_BUG_ON_INVALID(cond)
                                                              ^~~~
   include/linux/build_bug.h:30:63: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                                                 ^
   include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
   static inline bool folio_memcg_kmem(struct folio *folio)
                                              ^
   include/linux/memcontrol.h:544:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
           ^
   include/linux/memcontrol.h:544:23: error: incomplete definition of type 'struct folio'
           VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
                           ~~~~~^
   include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
   static inline bool folio_memcg_kmem(struct folio *folio)
                                              ^
   include/linux/memcontrol.h:545:14: error: incomplete definition of type 'struct folio'
           return folio->memcg_data & MEMCG_DATA_KMEM;
                  ~~~~~^
   include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
   static inline bool folio_memcg_kmem(struct folio *folio)
                                              ^
   include/linux/memcontrol.h:607:26: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration]
           return folio_memcg_kmem(page_folio(page));
                                   ^
   include/linux/memcontrol.h:607:26: warning: incompatible integer to pointer conversion passing 'int' to parameter of type 'struct folio *' [-Wint-conversion]
           return folio_memcg_kmem(page_folio(page));
                                   ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:375:51: note: passing argument to parameter 'folio' here
   static inline bool folio_memcg_kmem(struct folio *folio);
                                                     ^
   8 warnings and 17 errors generated.
--
   In file included from arch/x86/kernel/asm-offsets.c:13:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   In file included from include/linux/memcontrol.h:13:
   In file included from include/linux/cgroup.h:28:
   In file included from include/linux/cgroup-defs.h:22:
   In file included from include/linux/bpf-cgroup.h:5:
   In file included from include/linux/bpf.h:20:
   In file included from include/linux/kallsyms.h:12:
   include/linux/mm.h:1382:42: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline int folio_nid(const struct folio *folio)
                                            ^
   include/linux/mm.h:1384:27: error: incomplete definition of type 'struct folio'
           return page_to_nid(&folio->page);
                               ~~~~~^
   include/linux/mm.h:1382:42: note: forward declaration of 'struct folio'
   static inline int folio_nid(const struct folio *folio)
                                            ^
   In file included from arch/x86/kernel/asm-offsets.c:13:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:375:44: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline bool folio_memcg_kmem(struct folio *folio);
                                              ^
   include/linux/memcontrol.h:399:55: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
                                                         ^
>> include/linux/memcontrol.h:401:34: error: incomplete definition of type 'struct folio'
           unsigned long memcg_data = folio->memcg_data;
                                      ~~~~~^
   include/linux/memcontrol.h:399:55: note: forward declaration of 'struct folio'
   static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
                                                         ^
>> include/linux/memcontrol.h:403:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio_slab(folio), folio);
           ^
>> include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio_slab(folio), folio);
                           ^
   include/linux/memcontrol.h:420:55: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
                                                         ^
   include/linux/memcontrol.h:422:34: error: incomplete definition of type 'struct folio'
           unsigned long memcg_data = folio->memcg_data;
                                      ~~~~~^
   include/linux/memcontrol.h:420:55: note: forward declaration of 'struct folio'
   static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
                                                         ^
   include/linux/memcontrol.h:424:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio_slab(folio), folio);
           ^
   include/linux/memcontrol.h:424:18: error: implicit declaration of function 'folio_slab' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio_slab(folio), folio);
                           ^
   include/linux/memcontrol.h:451:53: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline struct mem_cgroup *folio_memcg(struct folio *folio)
                                                       ^
   include/linux/memcontrol.h:453:23: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
           if (folio_memcg_kmem(folio))
                                ^~~~~
   include/linux/memcontrol.h:375:51: note: passing argument to parameter 'folio' here
   static inline bool folio_memcg_kmem(struct folio *folio);
                                                     ^
   include/linux/memcontrol.h:454:41: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
                   return obj_cgroup_memcg(__folio_objcg(folio));
                                                         ^~~~~
   include/linux/memcontrol.h:420:62: note: passing argument to parameter 'folio' here
   static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
                                                                ^
   include/linux/memcontrol.h:456:24: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
                   return __folio_memcg(folio);
                                        ^~~~~
   include/linux/memcontrol.h:399:62: note: passing argument to parameter 'folio' here
   static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
                                                                ^
   include/linux/memcontrol.h:461:21: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration]
           return folio_memcg(page_folio(page));
                              ^
   include/linux/memcontrol.h:461:21: warning: incompatible integer to pointer conversion passing 'int' to parameter of type 'struct folio *' [-Wint-conversion]
           return folio_memcg(page_folio(page));
                              ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:451:60: note: passing argument to parameter 'folio' here
   static inline struct mem_cgroup *folio_memcg(struct folio *folio)
                                                              ^
   include/linux/memcontrol.h:541:44: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline bool folio_memcg_kmem(struct folio *folio)
                                              ^
   include/linux/memcontrol.h:541:20: error: conflicting types for 'folio_memcg_kmem'
   static inline bool folio_memcg_kmem(struct folio *folio)
                      ^
   include/linux/memcontrol.h:375:20: note: previous declaration is here
   static inline bool folio_memcg_kmem(struct folio *folio);
                      ^
   include/linux/memcontrol.h:543:35: error: incomplete definition of type 'struct folio'
           VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
                                       ~~~~~^
   include/linux/mmdebug.h:76:60: note: expanded from macro 'VM_BUG_ON_PGFLAGS'
   #define VM_BUG_ON_PGFLAGS(cond, page) BUILD_BUG_ON_INVALID(cond)
                                                              ^~~~
   include/linux/build_bug.h:30:63: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                                                 ^
   include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
   static inline bool folio_memcg_kmem(struct folio *folio)
                                              ^
   include/linux/memcontrol.h:544:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
           ^
   include/linux/memcontrol.h:544:23: error: incomplete definition of type 'struct folio'
           VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
                           ~~~~~^
   include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
   static inline bool folio_memcg_kmem(struct folio *folio)
                                              ^
   include/linux/memcontrol.h:545:14: error: incomplete definition of type 'struct folio'
           return folio->memcg_data & MEMCG_DATA_KMEM;
                  ~~~~~^
   include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
   static inline bool folio_memcg_kmem(struct folio *folio)
                                              ^
   include/linux/memcontrol.h:607:26: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration]
           return folio_memcg_kmem(page_folio(page));
                                   ^
   include/linux/memcontrol.h:607:26: warning: incompatible integer to pointer conversion passing 'int' to parameter of type 'struct folio *' [-Wint-conversion]
           return folio_memcg_kmem(page_folio(page));
                                   ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:375:51: note: passing argument to parameter 'folio' here
   static inline bool folio_memcg_kmem(struct folio *folio);
                                                     ^
   8 warnings and 17 errors generated.
   make[2]: *** [scripts/Makefile.build:117: arch/x86/kernel/asm-offsets.s] Error 1
   make[2]: Target '__build' not remade because of errors.
   make[1]: *** [Makefile:1235: prepare0] Error 2
   make[1]: Target 'modules_prepare' not remade because of errors.
   make: *** [Makefile:215: __sub-make] Error 2
   make: Target 'modules_prepare' not remade because of errors.
--
   In file included from arch/x86/kernel/asm-offsets.c:13:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   In file included from include/linux/memcontrol.h:13:
   In file included from include/linux/cgroup.h:28:
   In file included from include/linux/cgroup-defs.h:22:
   In file included from include/linux/bpf-cgroup.h:5:
   In file included from include/linux/bpf.h:20:
   In file included from include/linux/kallsyms.h:12:
   include/linux/mm.h:1382:42: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline int folio_nid(const struct folio *folio)
                                            ^
   include/linux/mm.h:1384:27: error: incomplete definition of type 'struct folio'
           return page_to_nid(&folio->page);
                               ~~~~~^
   include/linux/mm.h:1382:42: note: forward declaration of 'struct folio'
   static inline int folio_nid(const struct folio *folio)
                                            ^
   In file included from arch/x86/kernel/asm-offsets.c:13:
   In file included from include/linux/suspend.h:5:
   In file included from include/linux/swap.h:9:
   include/linux/memcontrol.h:375:44: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline bool folio_memcg_kmem(struct folio *folio);
                                              ^
   include/linux/memcontrol.h:399:55: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
                                                         ^
>> include/linux/memcontrol.h:401:34: error: incomplete definition of type 'struct folio'
           unsigned long memcg_data = folio->memcg_data;
                                      ~~~~~^
   include/linux/memcontrol.h:399:55: note: forward declaration of 'struct folio'
   static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
                                                         ^
>> include/linux/memcontrol.h:403:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio_slab(folio), folio);
           ^
>> include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio_slab(folio), folio);
                           ^
   include/linux/memcontrol.h:420:55: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
                                                         ^
   include/linux/memcontrol.h:422:34: error: incomplete definition of type 'struct folio'
           unsigned long memcg_data = folio->memcg_data;
                                      ~~~~~^
   include/linux/memcontrol.h:420:55: note: forward declaration of 'struct folio'
   static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
                                                         ^
   include/linux/memcontrol.h:424:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio_slab(folio), folio);
           ^
   include/linux/memcontrol.h:424:18: error: implicit declaration of function 'folio_slab' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio_slab(folio), folio);
                           ^
   include/linux/memcontrol.h:451:53: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline struct mem_cgroup *folio_memcg(struct folio *folio)
                                                       ^
   include/linux/memcontrol.h:453:23: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
           if (folio_memcg_kmem(folio))
                                ^~~~~
   include/linux/memcontrol.h:375:51: note: passing argument to parameter 'folio' here
   static inline bool folio_memcg_kmem(struct folio *folio);
                                                     ^
   include/linux/memcontrol.h:454:41: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
                   return obj_cgroup_memcg(__folio_objcg(folio));
                                                         ^~~~~
   include/linux/memcontrol.h:420:62: note: passing argument to parameter 'folio' here
   static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
                                                                ^
   include/linux/memcontrol.h:456:24: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
                   return __folio_memcg(folio);
                                        ^~~~~
   include/linux/memcontrol.h:399:62: note: passing argument to parameter 'folio' here
   static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
                                                                ^
   include/linux/memcontrol.h:461:21: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration]
           return folio_memcg(page_folio(page));
                              ^
   include/linux/memcontrol.h:461:21: warning: incompatible integer to pointer conversion passing 'int' to parameter of type 'struct folio *' [-Wint-conversion]
           return folio_memcg(page_folio(page));
                              ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:451:60: note: passing argument to parameter 'folio' here
   static inline struct mem_cgroup *folio_memcg(struct folio *folio)
                                                              ^
   include/linux/memcontrol.h:541:44: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
   static inline bool folio_memcg_kmem(struct folio *folio)
                                              ^
   include/linux/memcontrol.h:541:20: error: conflicting types for 'folio_memcg_kmem'
   static inline bool folio_memcg_kmem(struct folio *folio)
                      ^
   include/linux/memcontrol.h:375:20: note: previous declaration is here
   static inline bool folio_memcg_kmem(struct folio *folio);
                      ^
   include/linux/memcontrol.h:543:35: error: incomplete definition of type 'struct folio'
           VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
                                       ~~~~~^
   include/linux/mmdebug.h:76:60: note: expanded from macro 'VM_BUG_ON_PGFLAGS'
   #define VM_BUG_ON_PGFLAGS(cond, page) BUILD_BUG_ON_INVALID(cond)
                                                              ^~~~
   include/linux/build_bug.h:30:63: note: expanded from macro 'BUILD_BUG_ON_INVALID'
   #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
                                                                 ^
   include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
   static inline bool folio_memcg_kmem(struct folio *folio)
                                              ^
   include/linux/memcontrol.h:544:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
           VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
           ^
   include/linux/memcontrol.h:544:23: error: incomplete definition of type 'struct folio'
           VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
                           ~~~~~^
   include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
   static inline bool folio_memcg_kmem(struct folio *folio)
                                              ^
   include/linux/memcontrol.h:545:14: error: incomplete definition of type 'struct folio'
           return folio->memcg_data & MEMCG_DATA_KMEM;
                  ~~~~~^
   include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
   static inline bool folio_memcg_kmem(struct folio *folio)
                                              ^
   include/linux/memcontrol.h:607:26: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration]
           return folio_memcg_kmem(page_folio(page));
                                   ^
   include/linux/memcontrol.h:607:26: warning: incompatible integer to pointer conversion passing 'int' to parameter of type 'struct folio *' [-Wint-conversion]
           return folio_memcg_kmem(page_folio(page));
                                   ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:375:51: note: passing argument to parameter 'folio' here
   static inline bool folio_memcg_kmem(struct folio *folio);
                                                     ^
   8 warnings and 17 errors generated.
   make[2]: *** [scripts/Makefile.build:117: arch/x86/kernel/asm-offsets.s] Error 1
   make[2]: Target '__build' not remade because of errors.
   make[1]: *** [Makefile:1235: prepare0] Error 2
   make[1]: Target 'prepare' not remade because of errors.
   make: *** [Makefile:215: __sub-make] Error 2
   make: Target 'prepare' not remade because of errors.


vim +401 include/linux/memcontrol.h

   373	
   374	static inline bool PageMemcgKmem(struct page *page);
 > 375	static inline bool folio_memcg_kmem(struct folio *folio);
   376	
   377	/*
   378	 * After the initialization objcg->memcg is always pointing at
   379	 * a valid memcg, but can be atomically swapped to the parent memcg.
   380	 *
   381	 * The caller must ensure that the returned memcg won't be released:
   382	 * e.g. acquire the rcu_read_lock or css_set_lock.
   383	 */
   384	static inline struct mem_cgroup *obj_cgroup_memcg(struct obj_cgroup *objcg)
   385	{
   386		return READ_ONCE(objcg->memcg);
   387	}
   388	
   389	/*
   390	 * __folio_memcg - Get the memory cgroup associated with a non-kmem folio
   391	 * @folio: Pointer to the folio.
   392	 *
   393	 * Returns a pointer to the memory cgroup associated with the folio,
   394	 * or NULL. This function assumes that the folio is known to have a
   395	 * proper memory cgroup pointer. It's not safe to call this function
   396	 * against some type of folios, e.g. slab folios or ex-slab folios or
   397	 * kmem folios.
   398	 */
 > 399	static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
   400	{
 > 401		unsigned long memcg_data = folio->memcg_data;
   402	
 > 403		VM_BUG_ON_FOLIO(folio_slab(folio), folio);
   404		VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
   405		VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_KMEM, folio);
   406	
   407		return (struct mem_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
   408	}
   409	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 49428 bytes --]

Re: [PATCH v3 06/18] mm/memcg: Add folio_memcg() and related functions

[Christoph Hellwig]

On Wed, Jun 30, 2021 at 05:00:22AM +0100, Matthew Wilcox (Oracle) wrote:
> memcg information is only stored in the head page, so the memcg
> subsystem needs to assure that all accesses are to the head page.
> The first step is converting page_memcg() to folio_memcg().
> 
> Retain page_memcg() as a wrapper around folio_memcg() and PageMemcgKmem()
> as a wrapper around folio_memcg_kmem() but convert __page_memcg() to
> __folio_memcg() and __page_objcg() to __folio_objcg().
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
> ---
>  include/linux/memcontrol.h | 105 +++++++++++++++++++++----------------
>  mm/memcontrol.c            |  21 ++++----
>  2 files changed, 73 insertions(+), 53 deletions(-)
> 
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index 6d66037be646..92689fb2dab4 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -372,6 +372,7 @@ enum page_memcg_data_flags {
>  #define MEMCG_DATA_FLAGS_MASK (__NR_MEMCG_DATA_FLAGS - 1)
>  
>  static inline bool PageMemcgKmem(struct page *page);
> +static inline bool folio_memcg_kmem(struct folio *folio);
>  
>  /*
>   * After the initialization objcg->memcg is always pointing at
> @@ -386,73 +387,78 @@ static inline struct mem_cgroup *obj_cgroup_memcg(struct obj_cgroup *objcg)
>  }
>  
>  /*
> - * __page_memcg - get the memory cgroup associated with a non-kmem page
> - * @page: a pointer to the page struct
> + * __folio_memcg - Get the memory cgroup associated with a non-kmem folio
> + * @folio: Pointer to the folio.
>   *
> - * Returns a pointer to the memory cgroup associated with the page,
> - * or NULL. This function assumes that the page is known to have a
> + * Returns a pointer to the memory cgroup associated with the folio,
> + * or NULL. This function assumes that the folio is known to have a
>   * proper memory cgroup pointer. It's not safe to call this function
> - * against some type of pages, e.g. slab pages or ex-slab pages or
> - * kmem pages.
> + * against some type of folios, e.g. slab folios or ex-slab folios or
> + * kmem folios.
>   */
> -static inline struct mem_cgroup *__page_memcg(struct page *page)
> +static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
>  {
> -	unsigned long memcg_data = page->memcg_data;
> +	unsigned long memcg_data = folio->memcg_data;
>  
> -	VM_BUG_ON_PAGE(PageSlab(page), page);
> -	VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_OBJCGS, page);
> -	VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_KMEM, page);
> +	VM_BUG_ON_FOLIO(folio_slab(folio), folio);
> +	VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
> +	VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_KMEM, folio);
>  
>  	return (struct mem_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
>  }
>  
>  /*
> - * __page_objcg - get the object cgroup associated with a kmem page
> - * @page: a pointer to the page struct
> + * __folio_objcg - get the object cgroup associated with a kmem folio.
> + * @folio: Pointer to the folio.
>   *
> - * Returns a pointer to the object cgroup associated with the page,
> - * or NULL. This function assumes that the page is known to have a
> + * Returns a pointer to the object cgroup associated with the folio,
> + * or NULL. This function assumes that the folio is known to have a
>   * proper object cgroup pointer. It's not safe to call this function
> - * against some type of pages, e.g. slab pages or ex-slab pages or
> - * LRU pages.
> + * against some type of folios, e.g. slab folios or ex-slab folios or
> + * LRU folios.
>   */
> -static inline struct obj_cgroup *__page_objcg(struct page *page)
> +static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
>  {
> -	unsigned long memcg_data = page->memcg_data;
> +	unsigned long memcg_data = folio->memcg_data;
>  
> -	VM_BUG_ON_PAGE(PageSlab(page), page);
> -	VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_OBJCGS, page);
> -	VM_BUG_ON_PAGE(!(memcg_data & MEMCG_DATA_KMEM), page);
> +	VM_BUG_ON_FOLIO(folio_slab(folio), folio);
> +	VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
> +	VM_BUG_ON_FOLIO(!(memcg_data & MEMCG_DATA_KMEM), folio);
>  
>  	return (struct obj_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
>  }
>  
>  /*
> - * page_memcg - get the memory cgroup associated with a page
> - * @page: a pointer to the page struct
> + * folio_memcg - Get the memory cgroup associated with a folio.
> + * @folio: Pointer to the folio.
>   *
> - * Returns a pointer to the memory cgroup associated with the page,
> - * or NULL. This function assumes that the page is known to have a
> + * Returns a pointer to the memory cgroup associated with the folio,
> + * or NULL. This function assumes that the folio is known to have a
>   * proper memory cgroup pointer. It's not safe to call this function
> - * against some type of pages, e.g. slab pages or ex-slab pages.
> + * against some type of folios, e.g. slab folios or ex-slab folios.
>   *
> - * For a non-kmem page any of the following ensures page and memcg binding
> + * For a non-kmem folio any of the following ensures folio and memcg binding
>   * stability:
>   *
> - * - the page lock
> + * - the folio lock
>   * - LRU isolation
>   * - lock_page_memcg()
>   * - exclusive reference
>   *
> - * For a kmem page a caller should hold an rcu read lock to protect memcg
> - * associated with a kmem page from being released.
> + * For a kmem folio a caller should hold an rcu read lock to protect memcg
> + * associated with a kmem folio from being released.
>   */
> -static inline struct mem_cgroup *page_memcg(struct page *page)
> +static inline struct mem_cgroup *folio_memcg(struct folio *folio)
>  {
> -	if (PageMemcgKmem(page))
> -		return obj_cgroup_memcg(__page_objcg(page));
> +	if (folio_memcg_kmem(folio))
> +		return obj_cgroup_memcg(__folio_objcg(folio));
>  	else
> +		return __folio_memcg(folio);

Nit: it might be worth to drop the pointless else while you're at it.
therwise looks good:


Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>

[PATCH v3 07/18] mm/memcg: Convert commit_charge() to take a folio

[Matthew Wilcox (Oracle)]

The memcg_data is only set on the head page, so enforce that by
typing it as a folio.

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
Acked-by: Michal Hocko <mhocko-IBi9RG/b67k@public.gmane.org>
---
 mm/memcontrol.c | 27 +++++++++++++--------------
 1 file changed, 13 insertions(+), 14 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index f369bbaf584b..727bd578ca7d 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -2764,9 +2764,9 @@ static void cancel_charge(struct mem_cgroup *memcg, unsigned int nr_pages)
 }
 #endif
 
-static void commit_charge(struct page *page, struct mem_cgroup *memcg)
+static void commit_charge(struct folio *folio, struct mem_cgroup *memcg)
 {
-	VM_BUG_ON_PAGE(page_memcg(page), page);
+	VM_BUG_ON_FOLIO(folio_memcg(folio), folio);
 	/*
 	 * Any of the following ensures page's memcg stability:
 	 *
@@ -2775,7 +2775,7 @@ static void commit_charge(struct page *page, struct mem_cgroup *memcg)
 	 * - lock_page_memcg()
 	 * - exclusive reference
 	 */
-	page->memcg_data = (unsigned long)memcg;
+	folio->memcg_data = (unsigned long)memcg;
 }
 
 static struct mem_cgroup *get_mem_cgroup_from_objcg(struct obj_cgroup *objcg)
@@ -6679,7 +6679,8 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root,
 static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
 			       gfp_t gfp)
 {
-	unsigned int nr_pages = thp_nr_pages(page);
+	struct folio *folio = page_folio(page);
+	unsigned int nr_pages = folio_nr_pages(folio);
 	int ret;
 
 	ret = try_charge(memcg, gfp, nr_pages);
@@ -6687,7 +6688,7 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
 		goto out;
 
 	css_get(&memcg->css);
-	commit_charge(page, memcg);
+	commit_charge(folio, memcg);
 
 	local_irq_disable();
 	mem_cgroup_charge_statistics(memcg, nr_pages);
@@ -6947,21 +6948,21 @@ void mem_cgroup_uncharge_list(struct list_head *page_list)
  */
 void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 {
+	struct folio *newfolio = page_folio(newpage);
 	struct mem_cgroup *memcg;
-	unsigned int nr_pages;
+	unsigned int nr_pages = folio_nr_pages(newfolio);
 	unsigned long flags;
 
 	VM_BUG_ON_PAGE(!PageLocked(oldpage), oldpage);
-	VM_BUG_ON_PAGE(!PageLocked(newpage), newpage);
-	VM_BUG_ON_PAGE(PageAnon(oldpage) != PageAnon(newpage), newpage);
-	VM_BUG_ON_PAGE(PageTransHuge(oldpage) != PageTransHuge(newpage),
-		       newpage);
+	VM_BUG_ON_FOLIO(!folio_locked(newfolio), newfolio);
+	VM_BUG_ON_FOLIO(PageAnon(oldpage) != folio_anon(newfolio), newfolio);
+	VM_BUG_ON_FOLIO(compound_nr(oldpage) != nr_pages, newfolio);
 
 	if (mem_cgroup_disabled())
 		return;
 
 	/* Page cache replacement: new page already charged? */
-	if (page_memcg(newpage))
+	if (folio_memcg(newfolio))
 		return;
 
 	memcg = page_memcg(oldpage);
@@ -6970,8 +6971,6 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 		return;
 
 	/* Force-charge the new page. The old one will be freed soon */
-	nr_pages = thp_nr_pages(newpage);
-
 	if (!mem_cgroup_is_root(memcg)) {
 		page_counter_charge(&memcg->memory, nr_pages);
 		if (do_memsw_account())
@@ -6979,7 +6978,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 	}
 
 	css_get(&memcg->css);
-	commit_charge(newpage, memcg);
+	commit_charge(newfolio, memcg);
 
 	local_irq_save(flags);
 	mem_cgroup_charge_statistics(memcg, nr_pages);
-- 
2.30.2

[PATCH v3 08/18] mm/memcg: Convert mem_cgroup_charge() to take a folio

[Matthew Wilcox (Oracle)]

Convert all callers of mem_cgroup_charge() to call page_folio() on the
page they're currently passing in.  Many of them will be converted to
use folios themselves soon.

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 include/linux/memcontrol.h |  6 +++---
 kernel/events/uprobes.c    |  3 ++-
 mm/filemap.c               |  2 +-
 mm/huge_memory.c           |  2 +-
 mm/khugepaged.c            |  4 ++--
 mm/ksm.c                   |  3 ++-
 mm/memcontrol.c            | 26 +++++++++++++-------------
 mm/memory.c                |  9 +++++----
 mm/migrate.c               |  2 +-
 mm/shmem.c                 |  2 +-
 mm/userfaultfd.c           |  2 +-
 11 files changed, 32 insertions(+), 29 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 92689fb2dab4..90d48b0e3191 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -705,7 +705,7 @@ static inline bool mem_cgroup_below_min(struct mem_cgroup *memcg)
 		page_counter_read(&memcg->memory);
 }
 
-int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask);
+int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
 int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
 				  gfp_t gfp, swp_entry_t entry);
 void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry);
@@ -1186,8 +1186,8 @@ static inline bool mem_cgroup_below_min(struct mem_cgroup *memcg)
 	return false;
 }
 
-static inline int mem_cgroup_charge(struct page *page, struct mm_struct *mm,
-				    gfp_t gfp_mask)
+static inline int mem_cgroup_charge(struct folio *folio,
+		struct mm_struct *mm, gfp_t gfp)
 {
 	return 0;
 }
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index af24dc3febbe..6357c3580d07 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -167,7 +167,8 @@ static int __replace_page(struct vm_area_struct *vma, unsigned long addr,
 				addr + PAGE_SIZE);
 
 	if (new_page) {
-		err = mem_cgroup_charge(new_page, vma->vm_mm, GFP_KERNEL);
+		err = mem_cgroup_charge(page_folio(new_page), vma->vm_mm,
+					GFP_KERNEL);
 		if (err)
 			return err;
 	}
diff --git a/mm/filemap.c b/mm/filemap.c
index 3fb48130f980..9600bca84162 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -872,7 +872,7 @@ noinline int __add_to_page_cache_locked(struct page *page,
 	page->index = offset;
 
 	if (!huge) {
-		error = mem_cgroup_charge(page, NULL, gfp);
+		error = mem_cgroup_charge(page_folio(page), NULL, gfp);
 		if (error)
 			goto error;
 		charged = true;
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 6d2a0119fc58..662f92bb1953 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -594,7 +594,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf,
 
 	VM_BUG_ON_PAGE(!PageCompound(page), page);
 
-	if (mem_cgroup_charge(page, vma->vm_mm, gfp)) {
+	if (mem_cgroup_charge(page_folio(page), vma->vm_mm, gfp)) {
 		put_page(page);
 		count_vm_event(THP_FAULT_FALLBACK);
 		count_vm_event(THP_FAULT_FALLBACK_CHARGE);
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 6c0185fdd815..0daa21fbdd71 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1088,7 +1088,7 @@ static void collapse_huge_page(struct mm_struct *mm,
 		goto out_nolock;
 	}
 
-	if (unlikely(mem_cgroup_charge(new_page, mm, gfp))) {
+	if (unlikely(mem_cgroup_charge(page_folio(new_page), mm, gfp))) {
 		result = SCAN_CGROUP_CHARGE_FAIL;
 		goto out_nolock;
 	}
@@ -1659,7 +1659,7 @@ static void collapse_file(struct mm_struct *mm,
 		goto out;
 	}
 
-	if (unlikely(mem_cgroup_charge(new_page, mm, gfp))) {
+	if (unlikely(mem_cgroup_charge(page_folio(new_page), mm, gfp))) {
 		result = SCAN_CGROUP_CHARGE_FAIL;
 		goto out;
 	}
diff --git a/mm/ksm.c b/mm/ksm.c
index 3fa9bc8a67cf..23d36b59f997 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -2580,7 +2580,8 @@ struct page *ksm_might_need_to_copy(struct page *page,
 		return page;		/* let do_swap_page report the error */
 
 	new_page = alloc_page_vma(GFP_HIGHUSER_MOVABLE, vma, address);
-	if (new_page && mem_cgroup_charge(new_page, vma->vm_mm, GFP_KERNEL)) {
+	if (new_page &&
+	    mem_cgroup_charge(page_folio(new_page), vma->vm_mm, GFP_KERNEL)) {
 		put_page(new_page);
 		new_page = NULL;
 	}
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 727bd578ca7d..1dc02ecd3257 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6676,10 +6676,9 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root,
 			atomic_long_read(&parent->memory.children_low_usage)));
 }
 
-static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
+static int __mem_cgroup_charge(struct folio *folio, struct mem_cgroup *memcg,
 			       gfp_t gfp)
 {
-	struct folio *folio = page_folio(page);
 	unsigned int nr_pages = folio_nr_pages(folio);
 	int ret;
 
@@ -6692,27 +6691,27 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
 
 	local_irq_disable();
 	mem_cgroup_charge_statistics(memcg, nr_pages);
-	memcg_check_events(memcg, page_to_nid(page));
+	memcg_check_events(memcg, folio_nid(folio));
 	local_irq_enable();
 out:
 	return ret;
 }
 
 /**
- * mem_cgroup_charge - charge a newly allocated page to a cgroup
- * @page: page to charge
- * @mm: mm context of the victim
- * @gfp_mask: reclaim mode
+ * mem_cgroup_charge - Charge a newly allocated folio to a cgroup.
+ * @folio: Folio to charge.
+ * @mm: mm context of the allocating task.
+ * @gfp: reclaim mode
  *
- * Try to charge @page to the memcg that @mm belongs to, reclaiming
- * pages according to @gfp_mask if necessary. if @mm is NULL, try to
+ * Try to charge @folio to the memcg that @mm belongs to, reclaiming
+ * pages according to @gfp if necessary.  If @mm is NULL, try to
  * charge to the active memcg.
  *
- * Do not use this for pages allocated for swapin.
+ * Do not use this for folios allocated for swapin.
  *
  * Returns 0 on success. Otherwise, an error code is returned.
  */
-int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask)
+int mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, gfp_t gfp)
 {
 	struct mem_cgroup *memcg;
 	int ret;
@@ -6721,7 +6720,7 @@ int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask)
 		return 0;
 
 	memcg = get_mem_cgroup_from_mm(mm);
-	ret = __mem_cgroup_charge(page, memcg, gfp_mask);
+	ret = __mem_cgroup_charge(folio, memcg, gfp);
 	css_put(&memcg->css);
 
 	return ret;
@@ -6742,6 +6741,7 @@ int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask)
 int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
 				  gfp_t gfp, swp_entry_t entry)
 {
+	struct folio *folio = page_folio(page);
 	struct mem_cgroup *memcg;
 	unsigned short id;
 	int ret;
@@ -6756,7 +6756,7 @@ int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
 		memcg = get_mem_cgroup_from_mm(mm);
 	rcu_read_unlock();
 
-	ret = __mem_cgroup_charge(page, memcg, gfp);
+	ret = __mem_cgroup_charge(folio, memcg, gfp);
 
 	css_put(&memcg->css);
 	return ret;
diff --git a/mm/memory.c b/mm/memory.c
index afff080caf8b..2e7a568e2c15 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -915,7 +915,7 @@ page_copy_prealloc(struct mm_struct *src_mm, struct vm_area_struct *vma,
 	if (!new_page)
 		return NULL;
 
-	if (mem_cgroup_charge(new_page, src_mm, GFP_KERNEL)) {
+	if (mem_cgroup_charge(page_folio(new_page), src_mm, GFP_KERNEL)) {
 		put_page(new_page);
 		return NULL;
 	}
@@ -2922,7 +2922,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
 		}
 	}
 
-	if (mem_cgroup_charge(new_page, mm, GFP_KERNEL))
+	if (mem_cgroup_charge(page_folio(new_page), mm, GFP_KERNEL))
 		goto oom_free_new;
 	cgroup_throttle_swaprate(new_page, GFP_KERNEL);
 
@@ -3640,7 +3640,7 @@ static vm_fault_t do_anonymous_page(struct vm_fault *vmf)
 	if (!page)
 		goto oom;
 
-	if (mem_cgroup_charge(page, vma->vm_mm, GFP_KERNEL))
+	if (mem_cgroup_charge(page_folio(page), vma->vm_mm, GFP_KERNEL))
 		goto oom_free_page;
 	cgroup_throttle_swaprate(page, GFP_KERNEL);
 
@@ -4053,7 +4053,8 @@ static vm_fault_t do_cow_fault(struct vm_fault *vmf)
 	if (!vmf->cow_page)
 		return VM_FAULT_OOM;
 
-	if (mem_cgroup_charge(vmf->cow_page, vma->vm_mm, GFP_KERNEL)) {
+	if (mem_cgroup_charge(page_folio(vmf->cow_page), vma->vm_mm,
+				GFP_KERNEL)) {
 		put_page(vmf->cow_page);
 		return VM_FAULT_OOM;
 	}
diff --git a/mm/migrate.c b/mm/migrate.c
index 380ca57b9031..94efe09bb2a0 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2914,7 +2914,7 @@ static void migrate_vma_insert_page(struct migrate_vma *migrate,
 
 	if (unlikely(anon_vma_prepare(vma)))
 		goto abort;
-	if (mem_cgroup_charge(page, vma->vm_mm, GFP_KERNEL))
+	if (mem_cgroup_charge(page_folio(page), vma->vm_mm, GFP_KERNEL))
 		goto abort;
 
 	/*
diff --git a/mm/shmem.c b/mm/shmem.c
index 6268b9b4e41a..3cc5ddd5cc6b 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -685,7 +685,7 @@ static int shmem_add_to_page_cache(struct page *page,
 	page->index = index;
 
 	if (!PageSwapCache(page)) {
-		error = mem_cgroup_charge(page, charge_mm, gfp);
+		error = mem_cgroup_charge(page_folio(page), charge_mm, gfp);
 		if (error) {
 			if (PageTransHuge(page)) {
 				count_vm_event(THP_FILE_FALLBACK);
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 63a73e164d55..bc8a2c3e5aea 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -96,7 +96,7 @@ static int mcopy_atomic_pte(struct mm_struct *dst_mm,
 	__SetPageUptodate(page);
 
 	ret = -ENOMEM;
-	if (mem_cgroup_charge(page, dst_mm, GFP_KERNEL))
+	if (mem_cgroup_charge(page_folio(page), dst_mm, GFP_KERNEL))
 		goto out_release;
 
 	_dst_pte = pte_mkdirty(mk_pte(page, dst_vma->vm_page_prot));
-- 
2.30.2

Re: [PATCH v3 08/18] mm/memcg: Convert mem_cgroup_charge() to take a folio

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 21121 bytes --]

Hi "Matthew,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on linus/master]
[also build test WARNING on next-20210629]
[cannot apply to hnaz-linux-mm/master tip/perf/core cgroup/for-next v5.13]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 007b350a58754a93ca9fe50c498cc27780171153
config: sparc64-randconfig-r002-20210628 (attached as .config)
compiler: sparc64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/0day-ci/linux/commit/014d3a00c94acec84ed571a521e10b9fe77a95cc
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
        git checkout 014d3a00c94acec84ed571a521e10b9fe77a95cc
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=sparc64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

All warnings (new ones prefixed by >>):

   include/linux/mm.h:1382:42: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1382 | static inline int folio_nid(const struct folio *folio)
         |                                          ^~~~~
   include/linux/mm.h: In function 'folio_nid':
   include/linux/mm.h:1384:27: error: dereferencing pointer to incomplete type 'const struct folio'
    1384 |  return page_to_nid(&folio->page);
         |                           ^~
   In file included from include/linux/rmap.h:12,
                    from mm/huge_memory.c:16:
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:375:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                            ^~~~~
   include/linux/memcontrol.h:399:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_memcg':
   include/linux/memcontrol.h:401:34: error: dereferencing pointer to incomplete type 'struct folio'
     401 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
   include/linux/memcontrol.h:403:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO'; did you mean 'VM_BUG_ON_MM'? [-Werror=implicit-function-declaration]
     403 |  VM_BUG_ON_FOLIO(folio_slab(folio), folio);
         |  ^~~~~~~~~~~~~~~
         |  VM_BUG_ON_MM
   include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror=implicit-function-declaration]
     403 |  VM_BUG_ON_FOLIO(folio_slab(folio), folio);
         |                  ^~~~~~~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:420:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_objcg':
   include/linux/memcontrol.h:422:34: error: dereferencing pointer to incomplete type 'struct folio'
     422 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:451:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                                     ^~~~~
   include/linux/memcontrol.h: In function 'folio_memcg':
   include/linux/memcontrol.h:453:23: error: passing argument 1 of 'folio_memcg_kmem' from incompatible pointer type [-Werror=incompatible-pointer-types]
     453 |  if (folio_memcg_kmem(folio))
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:375:51: note: expected 'struct folio *' but argument is of type 'struct folio *'
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                     ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h:454:41: error: passing argument 1 of '__folio_objcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     454 |   return obj_cgroup_memcg(__folio_objcg(folio));
         |                                         ^~~~~
         |                                         |
         |                                         struct folio *
   include/linux/memcontrol.h:420:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h:456:24: error: passing argument 1 of '__folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     456 |   return __folio_memcg(folio);
         |                        ^~~~~
         |                        |
         |                        struct folio *
   include/linux/memcontrol.h:399:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: In function 'page_memcg':
   include/linux/memcontrol.h:461:21: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
     461 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~
   include/linux/memcontrol.h:461:21: warning: passing argument 1 of 'folio_memcg' makes pointer from integer without a cast [-Wint-conversion]
     461 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~~~~~~~
         |                     |
         |                     int
   include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'int'
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                              ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:589:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     589 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
   include/linux/memcontrol.h:589:20: error: conflicting types for 'folio_memcg_kmem'
     589 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                    ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:375:20: note: previous declaration of 'folio_memcg_kmem' was here
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h: In function 'PageMemcgKmem':
   include/linux/memcontrol.h:607:26: warning: passing argument 1 of 'folio_memcg_kmem' makes pointer from integer without a cast [-Wint-conversion]
     607 |  return folio_memcg_kmem(page_folio(page));
         |                          ^~~~~~~~~~~~~~~~
         |                          |
         |                          int
   include/linux/memcontrol.h:589:51: note: expected 'struct folio *' but argument is of type 'int'
     589 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                     ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:708:30: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     708 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
         |                              ^~~~~
   mm/huge_memory.c: In function '__do_huge_pmd_anonymous_page':
>> mm/huge_memory.c:597:24: warning: passing argument 1 of 'mem_cgroup_charge' makes pointer from integer without a cast [-Wint-conversion]
     597 |  if (mem_cgroup_charge(page_folio(page), vma->vm_mm, gfp)) {
         |                        ^~~~~~~~~~~~~~~~
         |                        |
         |                        int
   In file included from include/linux/rmap.h:12,
                    from mm/huge_memory.c:16:
   include/linux/memcontrol.h:708:23: note: expected 'struct folio *' but argument is of type 'int'
     708 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
         |                       ^~~~~~~~~~~~~~
   mm/huge_memory.c: At top level:
   include/linux/memcontrol.h:375:20: warning: 'folio_memcg_kmem' used but never defined
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors
--
    1384 |  return page_to_nid(&folio->page);
         |                           ^~
   In file included from include/linux/rmap.h:12,
                    from mm/khugepaged.c:9:
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:375:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                            ^~~~~
   include/linux/memcontrol.h:399:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_memcg':
   include/linux/memcontrol.h:401:34: error: dereferencing pointer to incomplete type 'struct folio'
     401 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
   include/linux/memcontrol.h:403:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO'; did you mean 'VM_BUG_ON_MM'? [-Werror=implicit-function-declaration]
     403 |  VM_BUG_ON_FOLIO(folio_slab(folio), folio);
         |  ^~~~~~~~~~~~~~~
         |  VM_BUG_ON_MM
   include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror=implicit-function-declaration]
     403 |  VM_BUG_ON_FOLIO(folio_slab(folio), folio);
         |                  ^~~~~~~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:420:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_objcg':
   include/linux/memcontrol.h:422:34: error: dereferencing pointer to incomplete type 'struct folio'
     422 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:451:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                                     ^~~~~
   include/linux/memcontrol.h: In function 'folio_memcg':
   include/linux/memcontrol.h:453:23: error: passing argument 1 of 'folio_memcg_kmem' from incompatible pointer type [-Werror=incompatible-pointer-types]
     453 |  if (folio_memcg_kmem(folio))
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:375:51: note: expected 'struct folio *' but argument is of type 'struct folio *'
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                     ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h:454:41: error: passing argument 1 of '__folio_objcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     454 |   return obj_cgroup_memcg(__folio_objcg(folio));
         |                                         ^~~~~
         |                                         |
         |                                         struct folio *
   include/linux/memcontrol.h:420:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h:456:24: error: passing argument 1 of '__folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     456 |   return __folio_memcg(folio);
         |                        ^~~~~
         |                        |
         |                        struct folio *
   include/linux/memcontrol.h:399:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: In function 'page_memcg':
   include/linux/memcontrol.h:461:21: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
     461 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~
   include/linux/memcontrol.h:461:21: warning: passing argument 1 of 'folio_memcg' makes pointer from integer without a cast [-Wint-conversion]
     461 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~~~~~~~
         |                     |
         |                     int
   include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'int'
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                              ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:589:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     589 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
   include/linux/memcontrol.h:589:20: error: conflicting types for 'folio_memcg_kmem'
     589 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                    ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:375:20: note: previous declaration of 'folio_memcg_kmem' was here
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h: In function 'PageMemcgKmem':
   include/linux/memcontrol.h:607:26: warning: passing argument 1 of 'folio_memcg_kmem' makes pointer from integer without a cast [-Wint-conversion]
     607 |  return folio_memcg_kmem(page_folio(page));
         |                          ^~~~~~~~~~~~~~~~
         |                          |
         |                          int
   include/linux/memcontrol.h:589:51: note: expected 'struct folio *' but argument is of type 'int'
     589 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                     ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:708:30: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     708 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
         |                              ^~~~~
   In file included from arch/sparc/include/asm/bug.h:6,
                    from include/linux/bug.h:5,
                    from include/linux/mmdebug.h:5,
                    from include/linux/mm.h:9,
                    from mm/khugepaged.c:4:
   mm/khugepaged.c: In function 'collapse_huge_page':
>> mm/khugepaged.c:1091:33: warning: passing argument 1 of 'mem_cgroup_charge' makes pointer from integer without a cast [-Wint-conversion]
    1091 |  if (unlikely(mem_cgroup_charge(page_folio(new_page), mm, gfp))) {
         |                                 ^~~~~~~~~~~~~~~~~~~~
         |                                 |
         |                                 int
   include/linux/compiler.h:78:42: note: in definition of macro 'unlikely'
      78 | # define unlikely(x) __builtin_expect(!!(x), 0)
         |                                          ^
   In file included from include/linux/rmap.h:12,
                    from mm/khugepaged.c:9:
   include/linux/memcontrol.h:708:23: note: expected 'struct folio *' but argument is of type 'int'
     708 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
         |                       ^~~~~~~~~~~~~~
   mm/khugepaged.c: At top level:
   include/linux/memcontrol.h:375:20: warning: 'folio_memcg_kmem' used but never defined
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors


vim +/mem_cgroup_charge +597 mm/huge_memory.c

   586	
   587	static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf,
   588				struct page *page, gfp_t gfp)
   589	{
   590		struct vm_area_struct *vma = vmf->vma;
   591		pgtable_t pgtable;
   592		unsigned long haddr = vmf->address & HPAGE_PMD_MASK;
   593		vm_fault_t ret = 0;
   594	
   595		VM_BUG_ON_PAGE(!PageCompound(page), page);
   596	
 > 597		if (mem_cgroup_charge(page_folio(page), vma->vm_mm, gfp)) {
   598			put_page(page);
   599			count_vm_event(THP_FAULT_FALLBACK);
   600			count_vm_event(THP_FAULT_FALLBACK_CHARGE);
   601			return VM_FAULT_FALLBACK;
   602		}
   603		cgroup_throttle_swaprate(page, gfp);
   604	
   605		pgtable = pte_alloc_one(vma->vm_mm);
   606		if (unlikely(!pgtable)) {
   607			ret = VM_FAULT_OOM;
   608			goto release;
   609		}
   610	
   611		clear_huge_page(page, vmf->address, HPAGE_PMD_NR);
   612		/*
   613		 * The memory barrier inside __SetPageUptodate makes sure that
   614		 * clear_huge_page writes become visible before the set_pmd_at()
   615		 * write.
   616		 */
   617		__SetPageUptodate(page);
   618	
   619		vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd);
   620		if (unlikely(!pmd_none(*vmf->pmd))) {
   621			goto unlock_release;
   622		} else {
   623			pmd_t entry;
   624	
   625			ret = check_stable_address_space(vma->vm_mm);
   626			if (ret)
   627				goto unlock_release;
   628	
   629			/* Deliver the page fault to userland */
   630			if (userfaultfd_missing(vma)) {
   631				spin_unlock(vmf->ptl);
   632				put_page(page);
   633				pte_free(vma->vm_mm, pgtable);
   634				ret = handle_userfault(vmf, VM_UFFD_MISSING);
   635				VM_BUG_ON(ret & VM_FAULT_FALLBACK);
   636				return ret;
   637			}
   638	
   639			entry = mk_huge_pmd(page, vma->vm_page_prot);
   640			entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma);
   641			page_add_new_anon_rmap(page, vma, haddr, true);
   642			lru_cache_add_inactive_or_unevictable(page, vma);
   643			pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, pgtable);
   644			set_pmd_at(vma->vm_mm, haddr, vmf->pmd, entry);
   645			update_mmu_cache_pmd(vma, vmf->address, vmf->pmd);
   646			add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR);
   647			mm_inc_nr_ptes(vma->vm_mm);
   648			spin_unlock(vmf->ptl);
   649			count_vm_event(THP_FAULT_ALLOC);
   650			count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
   651		}
   652	
   653		return 0;
   654	unlock_release:
   655		spin_unlock(vmf->ptl);
   656	release:
   657		if (pgtable)
   658			pte_free(vma->vm_mm, pgtable);
   659		put_page(page);
   660		return ret;
   661	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 26865 bytes --]

Re: [PATCH v3 08/18] mm/memcg: Convert mem_cgroup_charge() to take a folio

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>

[PATCH v3 09/18] mm/memcg: Convert uncharge_page() to uncharge_folio()

[Matthew Wilcox (Oracle)]

Use a folio rather than a page to ensure that we're only operating on
base or head pages, and not tail pages.

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 mm/memcontrol.c | 29 ++++++++++++++---------------
 1 file changed, 14 insertions(+), 15 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 1dc02ecd3257..21b791935957 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6827,24 +6827,23 @@ static void uncharge_batch(const struct uncharge_gather *ug)
 	memcg_check_events(ug->memcg, ug->nid);
 	local_irq_restore(flags);
 
-	/* drop reference from uncharge_page */
+	/* drop reference from uncharge_folio */
 	css_put(&ug->memcg->css);
 }
 
-static void uncharge_page(struct page *page, struct uncharge_gather *ug)
+static void uncharge_folio(struct folio *folio, struct uncharge_gather *ug)
 {
-	struct folio *folio = page_folio(page);
 	unsigned long nr_pages;
 	struct mem_cgroup *memcg;
 	struct obj_cgroup *objcg;
-	bool use_objcg = PageMemcgKmem(page);
+	bool use_objcg = folio_memcg_kmem(folio);
 
-	VM_BUG_ON_PAGE(PageLRU(page), page);
+	VM_BUG_ON_FOLIO(folio_lru(folio), folio);
 
 	/*
 	 * Nobody should be changing or seriously looking at
-	 * page memcg or objcg at this point, we have fully
-	 * exclusive access to the page.
+	 * folio memcg or objcg at this point, we have fully
+	 * exclusive access to the folio.
 	 */
 	if (use_objcg) {
 		objcg = __folio_objcg(folio);
@@ -6866,19 +6865,19 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
 			uncharge_gather_clear(ug);
 		}
 		ug->memcg = memcg;
-		ug->nid = page_to_nid(page);
+		ug->nid = folio_nid(folio);
 
 		/* pairs with css_put in uncharge_batch */
 		css_get(&memcg->css);
 	}
 
-	nr_pages = compound_nr(page);
+	nr_pages = folio_nr_pages(folio);
 
 	if (use_objcg) {
 		ug->nr_memory += nr_pages;
 		ug->nr_kmem += nr_pages;
 
-		page->memcg_data = 0;
+		folio->memcg_data = 0;
 		obj_cgroup_put(objcg);
 	} else {
 		/* LRU pages aren't accounted at the root level */
@@ -6886,7 +6885,7 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
 			ug->nr_memory += nr_pages;
 		ug->pgpgout++;
 
-		page->memcg_data = 0;
+		folio->memcg_data = 0;
 	}
 
 	css_put(&memcg->css);
@@ -6910,7 +6909,7 @@ void mem_cgroup_uncharge(struct page *page)
 		return;
 
 	uncharge_gather_clear(&ug);
-	uncharge_page(page, &ug);
+	uncharge_folio(page_folio(page), &ug);
 	uncharge_batch(&ug);
 }
 
@@ -6924,14 +6923,14 @@ void mem_cgroup_uncharge(struct page *page)
 void mem_cgroup_uncharge_list(struct list_head *page_list)
 {
 	struct uncharge_gather ug;
-	struct page *page;
+	struct folio *folio;
 
 	if (mem_cgroup_disabled())
 		return;
 
 	uncharge_gather_clear(&ug);
-	list_for_each_entry(page, page_list, lru)
-		uncharge_page(page, &ug);
+	list_for_each_entry(folio, page_list, lru)
+		uncharge_folio(folio, &ug);
 	if (ug.memcg)
 		uncharge_batch(&ug);
 }
-- 
2.30.2

Re: [PATCH v3 09/18] mm/memcg: Convert uncharge_page() to uncharge_folio()

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>

[PATCH v3 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio

[Matthew Wilcox (Oracle)]

Convert all the callers to call page_folio().

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 include/linux/memcontrol.h |  4 ++--
 mm/filemap.c               |  2 +-
 mm/khugepaged.c            |  4 ++--
 mm/memcontrol.c            | 14 +++++++-------
 mm/memory-failure.c        |  2 +-
 mm/memremap.c              |  2 +-
 mm/page_alloc.c            |  2 +-
 mm/swap.c                  |  2 +-
 8 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 90d48b0e3191..d6386a2b9d7a 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -710,7 +710,7 @@ int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
 				  gfp_t gfp, swp_entry_t entry);
 void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry);
 
-void mem_cgroup_uncharge(struct page *page);
+void mem_cgroup_uncharge(struct folio *);
 void mem_cgroup_uncharge_list(struct list_head *page_list);
 
 void mem_cgroup_migrate(struct page *oldpage, struct page *newpage);
@@ -1202,7 +1202,7 @@ static inline void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry)
 {
 }
 
-static inline void mem_cgroup_uncharge(struct page *page)
+static inline void mem_cgroup_uncharge(struct folio *folio)
 {
 }
 
diff --git a/mm/filemap.c b/mm/filemap.c
index 9600bca84162..0008ada132c4 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -923,7 +923,7 @@ noinline int __add_to_page_cache_locked(struct page *page,
 	if (xas_error(&xas)) {
 		error = xas_error(&xas);
 		if (charged)
-			mem_cgroup_uncharge(page);
+			mem_cgroup_uncharge(page_folio(page));
 		goto error;
 	}
 
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 0daa21fbdd71..988a230c7a41 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1212,7 +1212,7 @@ static void collapse_huge_page(struct mm_struct *mm,
 	mmap_write_unlock(mm);
 out_nolock:
 	if (!IS_ERR_OR_NULL(*hpage))
-		mem_cgroup_uncharge(*hpage);
+		mem_cgroup_uncharge(page_folio(*hpage));
 	trace_mm_collapse_huge_page(mm, isolated, result);
 	return;
 }
@@ -1963,7 +1963,7 @@ static void collapse_file(struct mm_struct *mm,
 out:
 	VM_BUG_ON(!list_empty(&pagelist));
 	if (!IS_ERR_OR_NULL(*hpage))
-		mem_cgroup_uncharge(*hpage);
+		mem_cgroup_uncharge(page_folio(*hpage));
 	/* TODO: tracepoints */
 }
 
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 21b791935957..90a53f554371 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6892,24 +6892,24 @@ static void uncharge_folio(struct folio *folio, struct uncharge_gather *ug)
 }
 
 /**
- * mem_cgroup_uncharge - uncharge a page
- * @page: page to uncharge
+ * mem_cgroup_uncharge - Uncharge a folio.
+ * @folio: Folio to uncharge.
  *
- * Uncharge a page previously charged with mem_cgroup_charge().
+ * Uncharge a folio previously charged with folio_charge_cgroup().
  */
-void mem_cgroup_uncharge(struct page *page)
+void mem_cgroup_uncharge(struct folio *folio)
 {
 	struct uncharge_gather ug;
 
 	if (mem_cgroup_disabled())
 		return;
 
-	/* Don't touch page->lru of any random page, pre-check: */
-	if (!page_memcg(page))
+	/* Don't touch folio->lru of any random page, pre-check: */
+	if (!folio_memcg(folio))
 		return;
 
 	uncharge_gather_clear(&ug);
-	uncharge_folio(page_folio(page), &ug);
+	uncharge_folio(folio, &ug);
 	uncharge_batch(&ug);
 }
 
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index e5a1531f7f4e..7ada5959b5ad 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -750,7 +750,7 @@ static int delete_from_lru_cache(struct page *p)
 		 * Poisoned page might never drop its ref count to 0 so we have
 		 * to uncharge it manually from its memcg.
 		 */
-		mem_cgroup_uncharge(p);
+		mem_cgroup_uncharge(page_folio(p));
 
 		/*
 		 * drop the page count elevated by isolate_lru_page()
diff --git a/mm/memremap.c b/mm/memremap.c
index 15a074ffb8d7..6eac40f9f62a 100644
--- a/mm/memremap.c
+++ b/mm/memremap.c
@@ -508,7 +508,7 @@ void free_devmap_managed_page(struct page *page)
 
 	__ClearPageWaiters(page);
 
-	mem_cgroup_uncharge(page);
+	mem_cgroup_uncharge(page_folio(page));
 
 	/*
 	 * When a device_private page is freed, the page->mapping field
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 0817d88383d5..5a5fcd4f21a8 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -737,7 +737,7 @@ static inline void free_the_page(struct page *page, unsigned int order)
 
 void free_compound_page(struct page *page)
 {
-	mem_cgroup_uncharge(page);
+	mem_cgroup_uncharge(page_folio(page));
 	free_the_page(page, compound_order(page));
 }
 
diff --git a/mm/swap.c b/mm/swap.c
index 6954cfebab4f..8ba62a930370 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -94,7 +94,7 @@ static void __page_cache_release(struct page *page)
 static void __put_single_page(struct page *page)
 {
 	__page_cache_release(page);
-	mem_cgroup_uncharge(page);
+	mem_cgroup_uncharge(page_folio(page));
 	free_unref_page(page, 0);
 }
 
-- 
2.30.2

Re: [PATCH v3 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 15521 bytes --]

Hi "Matthew,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on linus/master]
[also build test WARNING on next-20210630]
[cannot apply to hnaz-linux-mm/master tip/perf/core cgroup/for-next v5.13]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 007b350a58754a93ca9fe50c498cc27780171153
config: sparc64-randconfig-r002-20210628 (attached as .config)
compiler: sparc64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/0day-ci/linux/commit/b527e805e7d7066a8fea14ff4a49f53454c355a1
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
        git checkout b527e805e7d7066a8fea14ff4a49f53454c355a1
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=sparc64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

All warnings (new ones prefixed by >>):

     403 |  VM_BUG_ON_FOLIO(folio_slab(folio), folio);
         |  ^~~~~~~~~~~~~~~
         |  VM_BUG_ON_MM
   include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror=implicit-function-declaration]
     403 |  VM_BUG_ON_FOLIO(folio_slab(folio), folio);
         |                  ^~~~~~~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:420:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_objcg':
   include/linux/memcontrol.h:422:34: error: dereferencing pointer to incomplete type 'struct folio'
     422 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:451:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                                     ^~~~~
   include/linux/memcontrol.h: In function 'folio_memcg':
   include/linux/memcontrol.h:453:23: error: passing argument 1 of 'folio_memcg_kmem' from incompatible pointer type [-Werror=incompatible-pointer-types]
     453 |  if (folio_memcg_kmem(folio))
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:375:51: note: expected 'struct folio *' but argument is of type 'struct folio *'
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                     ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h:454:41: error: passing argument 1 of '__folio_objcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     454 |   return obj_cgroup_memcg(__folio_objcg(folio));
         |                                         ^~~~~
         |                                         |
         |                                         struct folio *
   include/linux/memcontrol.h:420:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h:456:24: error: passing argument 1 of '__folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     456 |   return __folio_memcg(folio);
         |                        ^~~~~
         |                        |
         |                        struct folio *
   include/linux/memcontrol.h:399:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: In function 'page_memcg':
   include/linux/memcontrol.h:461:21: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
     461 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~
   include/linux/memcontrol.h:461:21: warning: passing argument 1 of 'folio_memcg' makes pointer from integer without a cast [-Wint-conversion]
     461 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~~~~~~~
         |                     |
         |                     int
   include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'int'
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                              ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:589:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     589 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
   include/linux/memcontrol.h:589:20: error: conflicting types for 'folio_memcg_kmem'
     589 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                    ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:375:20: note: previous declaration of 'folio_memcg_kmem' was here
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h: In function 'PageMemcgKmem':
   include/linux/memcontrol.h:607:26: warning: passing argument 1 of 'folio_memcg_kmem' makes pointer from integer without a cast [-Wint-conversion]
     607 |  return folio_memcg_kmem(page_folio(page));
         |                          ^~~~~~~~~~~~~~~~
         |                          |
         |                          int
   include/linux/memcontrol.h:589:51: note: expected 'struct folio *' but argument is of type 'int'
     589 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                     ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:708:30: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     708 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
         |                              ^~~~~
   include/linux/memcontrol.h:713:33: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     713 | void mem_cgroup_uncharge(struct folio *);
         |                                 ^~~~~
   In file included from arch/sparc/include/asm/bug.h:6,
                    from include/linux/bug.h:5,
                    from include/linux/mmdebug.h:5,
                    from include/linux/mm.h:9,
                    from mm/khugepaged.c:4:
   mm/khugepaged.c: In function 'collapse_huge_page':
   mm/khugepaged.c:1091:33: warning: passing argument 1 of 'mem_cgroup_charge' makes pointer from integer without a cast [-Wint-conversion]
    1091 |  if (unlikely(mem_cgroup_charge(page_folio(new_page), mm, gfp))) {
         |                                 ^~~~~~~~~~~~~~~~~~~~
         |                                 |
         |                                 int
   include/linux/compiler.h:78:42: note: in definition of macro 'unlikely'
      78 | # define unlikely(x) __builtin_expect(!!(x), 0)
         |                                          ^
   In file included from include/linux/rmap.h:12,
                    from mm/khugepaged.c:9:
   include/linux/memcontrol.h:708:23: note: expected 'struct folio *' but argument is of type 'int'
     708 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
         |                       ^~~~~~~~~~~~~~
>> mm/khugepaged.c:1215:23: warning: passing argument 1 of 'mem_cgroup_uncharge' makes pointer from integer without a cast [-Wint-conversion]
    1215 |   mem_cgroup_uncharge(page_folio(*hpage));
         |                       ^~~~~~~~~~~~~~~~~~
         |                       |
         |                       int
   In file included from include/linux/rmap.h:12,
                    from mm/khugepaged.c:9:
   include/linux/memcontrol.h:713:26: note: expected 'struct folio *' but argument is of type 'int'
     713 | void mem_cgroup_uncharge(struct folio *);
         |                          ^~~~~~~~~~~~~~
   mm/khugepaged.c: At top level:
   include/linux/memcontrol.h:375:20: warning: 'folio_memcg_kmem' used but never defined
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors


vim +/mem_cgroup_uncharge +1215 mm/khugepaged.c

  1056	
  1057	static void collapse_huge_page(struct mm_struct *mm,
  1058					   unsigned long address,
  1059					   struct page **hpage,
  1060					   int node, int referenced, int unmapped)
  1061	{
  1062		LIST_HEAD(compound_pagelist);
  1063		pmd_t *pmd, _pmd;
  1064		pte_t *pte;
  1065		pgtable_t pgtable;
  1066		struct page *new_page;
  1067		spinlock_t *pmd_ptl, *pte_ptl;
  1068		int isolated = 0, result = 0;
  1069		struct vm_area_struct *vma;
  1070		struct mmu_notifier_range range;
  1071		gfp_t gfp;
  1072	
  1073		VM_BUG_ON(address & ~HPAGE_PMD_MASK);
  1074	
  1075		/* Only allocate from the target node */
  1076		gfp = alloc_hugepage_khugepaged_gfpmask() | __GFP_THISNODE;
  1077	
  1078		/*
  1079		 * Before allocating the hugepage, release the mmap_lock read lock.
  1080		 * The allocation can take potentially a long time if it involves
  1081		 * sync compaction, and we do not need to hold the mmap_lock during
  1082		 * that. We will recheck the vma after taking it again in write mode.
  1083		 */
  1084		mmap_read_unlock(mm);
  1085		new_page = khugepaged_alloc_page(hpage, gfp, node);
  1086		if (!new_page) {
  1087			result = SCAN_ALLOC_HUGE_PAGE_FAIL;
  1088			goto out_nolock;
  1089		}
  1090	
  1091		if (unlikely(mem_cgroup_charge(page_folio(new_page), mm, gfp))) {
  1092			result = SCAN_CGROUP_CHARGE_FAIL;
  1093			goto out_nolock;
  1094		}
  1095		count_memcg_page_event(new_page, THP_COLLAPSE_ALLOC);
  1096	
  1097		mmap_read_lock(mm);
  1098		result = hugepage_vma_revalidate(mm, address, &vma);
  1099		if (result) {
  1100			mmap_read_unlock(mm);
  1101			goto out_nolock;
  1102		}
  1103	
  1104		pmd = mm_find_pmd(mm, address);
  1105		if (!pmd) {
  1106			result = SCAN_PMD_NULL;
  1107			mmap_read_unlock(mm);
  1108			goto out_nolock;
  1109		}
  1110	
  1111		/*
  1112		 * __collapse_huge_page_swapin always returns with mmap_lock locked.
  1113		 * If it fails, we release mmap_lock and jump out_nolock.
  1114		 * Continuing to collapse causes inconsistency.
  1115		 */
  1116		if (unmapped && !__collapse_huge_page_swapin(mm, vma, address,
  1117							     pmd, referenced)) {
  1118			mmap_read_unlock(mm);
  1119			goto out_nolock;
  1120		}
  1121	
  1122		mmap_read_unlock(mm);
  1123		/*
  1124		 * Prevent all access to pagetables with the exception of
  1125		 * gup_fast later handled by the ptep_clear_flush and the VM
  1126		 * handled by the anon_vma lock + PG_lock.
  1127		 */
  1128		mmap_write_lock(mm);
  1129		result = hugepage_vma_revalidate(mm, address, &vma);
  1130		if (result)
  1131			goto out_up_write;
  1132		/* check if the pmd is still valid */
  1133		if (mm_find_pmd(mm, address) != pmd)
  1134			goto out_up_write;
  1135	
  1136		anon_vma_lock_write(vma->anon_vma);
  1137	
  1138		mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, NULL, mm,
  1139					address, address + HPAGE_PMD_SIZE);
  1140		mmu_notifier_invalidate_range_start(&range);
  1141	
  1142		pte = pte_offset_map(pmd, address);
  1143		pte_ptl = pte_lockptr(mm, pmd);
  1144	
  1145		pmd_ptl = pmd_lock(mm, pmd); /* probably unnecessary */
  1146		/*
  1147		 * After this gup_fast can't run anymore. This also removes
  1148		 * any huge TLB entry from the CPU so we won't allow
  1149		 * huge and small TLB entries for the same virtual address
  1150		 * to avoid the risk of CPU bugs in that area.
  1151		 */
  1152		_pmd = pmdp_collapse_flush(vma, address, pmd);
  1153		spin_unlock(pmd_ptl);
  1154		mmu_notifier_invalidate_range_end(&range);
  1155	
  1156		spin_lock(pte_ptl);
  1157		isolated = __collapse_huge_page_isolate(vma, address, pte,
  1158				&compound_pagelist);
  1159		spin_unlock(pte_ptl);
  1160	
  1161		if (unlikely(!isolated)) {
  1162			pte_unmap(pte);
  1163			spin_lock(pmd_ptl);
  1164			BUG_ON(!pmd_none(*pmd));
  1165			/*
  1166			 * We can only use set_pmd_at when establishing
  1167			 * hugepmds and never for establishing regular pmds that
  1168			 * points to regular pagetables. Use pmd_populate for that
  1169			 */
  1170			pmd_populate(mm, pmd, pmd_pgtable(_pmd));
  1171			spin_unlock(pmd_ptl);
  1172			anon_vma_unlock_write(vma->anon_vma);
  1173			result = SCAN_FAIL;
  1174			goto out_up_write;
  1175		}
  1176	
  1177		/*
  1178		 * All pages are isolated and locked so anon_vma rmap
  1179		 * can't run anymore.
  1180		 */
  1181		anon_vma_unlock_write(vma->anon_vma);
  1182	
  1183		__collapse_huge_page_copy(pte, new_page, vma, address, pte_ptl,
  1184				&compound_pagelist);
  1185		pte_unmap(pte);
  1186		/*
  1187		 * spin_lock() below is not the equivalent of smp_wmb(), but
  1188		 * the smp_wmb() inside __SetPageUptodate() can be reused to
  1189		 * avoid the copy_huge_page writes to become visible after
  1190		 * the set_pmd_at() write.
  1191		 */
  1192		__SetPageUptodate(new_page);
  1193		pgtable = pmd_pgtable(_pmd);
  1194	
  1195		_pmd = mk_huge_pmd(new_page, vma->vm_page_prot);
  1196		_pmd = maybe_pmd_mkwrite(pmd_mkdirty(_pmd), vma);
  1197	
  1198		spin_lock(pmd_ptl);
  1199		BUG_ON(!pmd_none(*pmd));
  1200		page_add_new_anon_rmap(new_page, vma, address, true);
  1201		lru_cache_add_inactive_or_unevictable(new_page, vma);
  1202		pgtable_trans_huge_deposit(mm, pmd, pgtable);
  1203		set_pmd_at(mm, address, pmd, _pmd);
  1204		update_mmu_cache_pmd(vma, address, pmd);
  1205		spin_unlock(pmd_ptl);
  1206	
  1207		*hpage = NULL;
  1208	
  1209		khugepaged_pages_collapsed++;
  1210		result = SCAN_SUCCEED;
  1211	out_up_write:
  1212		mmap_write_unlock(mm);
  1213	out_nolock:
  1214		if (!IS_ERR_OR_NULL(*hpage))
> 1215			mem_cgroup_uncharge(page_folio(*hpage));
  1216		trace_mm_collapse_huge_page(mm, isolated, result);
  1217		return;
  1218	}
  1219	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 26865 bytes --]

Re: [PATCH v3 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio

[Christoph Hellwig]

On Wed, Jun 30, 2021 at 05:00:26AM +0100, Matthew Wilcox (Oracle) wrote:
> Convert all the callers to call page_folio().
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
> ---
>  include/linux/memcontrol.h |  4 ++--
>  mm/filemap.c               |  2 +-
>  mm/khugepaged.c            |  4 ++--
>  mm/memcontrol.c            | 14 +++++++-------
>  mm/memory-failure.c        |  2 +-
>  mm/memremap.c              |  2 +-
>  mm/page_alloc.c            |  2 +-
>  mm/swap.c                  |  2 +-
>  8 files changed, 16 insertions(+), 16 deletions(-)
> 
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index 90d48b0e3191..d6386a2b9d7a 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -710,7 +710,7 @@ int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
>  				  gfp_t gfp, swp_entry_t entry);
>  void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry);
>  
> -void mem_cgroup_uncharge(struct page *page);
> +void mem_cgroup_uncharge(struct folio *);

why do you drop the parameter name?

Otherwise looks good:

Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>

Re: [PATCH v3 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio

[Matthew Wilcox]

On Thu, Jul 01, 2021 at 08:17:48AM +0100, Christoph Hellwig wrote:
> On Wed, Jun 30, 2021 at 05:00:26AM +0100, Matthew Wilcox (Oracle) wrote:
> > -void mem_cgroup_uncharge(struct page *page);
> > +void mem_cgroup_uncharge(struct folio *);
> 
> why do you drop the parameter name?

I usually do where it's 'struct foo *foo' or 'foo_t foo'.  If the best
you can do is say "it's a page", well, yes, I knew that from the type.
But since you've complained, I've added the pointless 'folio' to it.

> Otherwise looks good:
> 
> Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>

[PATCH v3 11/18] mm/memcg: Convert mem_cgroup_migrate() to take folios

[Matthew Wilcox (Oracle)]

Convert all callers of mem_cgroup_migrate() to call page_folio() first.

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 include/linux/memcontrol.h |  4 ++--
 mm/filemap.c               |  4 +++-
 mm/memcontrol.c            | 35 +++++++++++++++++------------------
 mm/migrate.c               |  4 +++-
 mm/shmem.c                 |  5 ++++-
 5 files changed, 29 insertions(+), 23 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index d6386a2b9d7a..2c57a405acd2 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -713,7 +713,7 @@ void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry);
 void mem_cgroup_uncharge(struct folio *);
 void mem_cgroup_uncharge_list(struct list_head *page_list);
 
-void mem_cgroup_migrate(struct page *oldpage, struct page *newpage);
+void mem_cgroup_migrate(struct folio *old, struct folio *new);
 
 /**
  * mem_cgroup_lruvec - get the lru list vector for a memcg & node
@@ -1210,7 +1210,7 @@ static inline void mem_cgroup_uncharge_list(struct list_head *page_list)
 {
 }
 
-static inline void mem_cgroup_migrate(struct page *old, struct page *new)
+static inline void mem_cgroup_migrate(struct folio *old, struct folio *new)
 {
 }
 
diff --git a/mm/filemap.c b/mm/filemap.c
index 0008ada132c4..964f1643dd97 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -817,6 +817,8 @@ EXPORT_SYMBOL(file_write_and_wait_range);
  */
 void replace_page_cache_page(struct page *old, struct page *new)
 {
+	struct folio *fold = page_folio(old);
+	struct folio *fnew = page_folio(new);
 	struct address_space *mapping = old->mapping;
 	void (*freepage)(struct page *) = mapping->a_ops->freepage;
 	pgoff_t offset = old->index;
@@ -831,7 +833,7 @@ void replace_page_cache_page(struct page *old, struct page *new)
 	new->mapping = mapping;
 	new->index = offset;
 
-	mem_cgroup_migrate(old, new);
+	mem_cgroup_migrate(fold, fnew);
 
 	xas_lock_irqsave(&xas, flags);
 	xas_store(&xas, new);
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 90a53f554371..4ce2f2eb81d8 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6936,36 +6936,35 @@ void mem_cgroup_uncharge_list(struct list_head *page_list)
 }
 
 /**
- * mem_cgroup_migrate - charge a page's replacement
- * @oldpage: currently circulating page
- * @newpage: replacement page
+ * mem_cgroup_migrate - Charge a folio's replacement.
+ * @old: Currently circulating folio.
+ * @new: Replacement folio.
  *
- * Charge @newpage as a replacement page for @oldpage. @oldpage will
+ * Charge @new as a replacement folio for @old. @old will
  * be uncharged upon free.
  *
- * Both pages must be locked, @newpage->mapping must be set up.
+ * Both folios must be locked, @new->mapping must be set up.
  */
-void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
+void mem_cgroup_migrate(struct folio *old, struct folio *new)
 {
-	struct folio *newfolio = page_folio(newpage);
 	struct mem_cgroup *memcg;
-	unsigned int nr_pages = folio_nr_pages(newfolio);
+	unsigned int nr_pages = folio_nr_pages(new);
 	unsigned long flags;
 
-	VM_BUG_ON_PAGE(!PageLocked(oldpage), oldpage);
-	VM_BUG_ON_FOLIO(!folio_locked(newfolio), newfolio);
-	VM_BUG_ON_FOLIO(PageAnon(oldpage) != folio_anon(newfolio), newfolio);
-	VM_BUG_ON_FOLIO(compound_nr(oldpage) != nr_pages, newfolio);
+	VM_BUG_ON_FOLIO(!folio_locked(old), old);
+	VM_BUG_ON_FOLIO(!folio_locked(new), new);
+	VM_BUG_ON_FOLIO(folio_anon(old) != folio_anon(new), new);
+	VM_BUG_ON_FOLIO(folio_nr_pages(old) != nr_pages, new);
 
 	if (mem_cgroup_disabled())
 		return;
 
-	/* Page cache replacement: new page already charged? */
-	if (folio_memcg(newfolio))
+	/* Page cache replacement: new folio already charged? */
+	if (folio_memcg(new))
 		return;
 
-	memcg = page_memcg(oldpage);
-	VM_WARN_ON_ONCE_PAGE(!memcg, oldpage);
+	memcg = folio_memcg(old);
+	VM_WARN_ON_ONCE_FOLIO(!memcg, old);
 	if (!memcg)
 		return;
 
@@ -6977,11 +6976,11 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 	}
 
 	css_get(&memcg->css);
-	commit_charge(newfolio, memcg);
+	commit_charge(new, memcg);
 
 	local_irq_save(flags);
 	mem_cgroup_charge_statistics(memcg, nr_pages);
-	memcg_check_events(memcg, page_to_nid(newpage));
+	memcg_check_events(memcg, folio_nid(new));
 	local_irq_restore(flags);
 }
 
diff --git a/mm/migrate.c b/mm/migrate.c
index 94efe09bb2a0..f71e72f9c812 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -582,6 +582,8 @@ static void copy_huge_page(struct page *dst, struct page *src)
  */
 void migrate_page_states(struct page *newpage, struct page *page)
 {
+	struct folio *folio = page_folio(page);
+	struct folio *newfolio = page_folio(newpage);
 	int cpupid;
 
 	if (PageError(page))
@@ -646,7 +648,7 @@ void migrate_page_states(struct page *newpage, struct page *page)
 	copy_page_owner(page, newpage);
 
 	if (!PageHuge(page))
-		mem_cgroup_migrate(page, newpage);
+		mem_cgroup_migrate(folio, newfolio);
 }
 EXPORT_SYMBOL(migrate_page_states);
 
diff --git a/mm/shmem.c b/mm/shmem.c
index 3cc5ddd5cc6b..1e172fb40fb3 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -1619,6 +1619,7 @@ static int shmem_replace_page(struct page **pagep, gfp_t gfp,
 				struct shmem_inode_info *info, pgoff_t index)
 {
 	struct page *oldpage, *newpage;
+	struct folio *old, *new;
 	struct address_space *swap_mapping;
 	swp_entry_t entry;
 	pgoff_t swap_index;
@@ -1655,7 +1656,9 @@ static int shmem_replace_page(struct page **pagep, gfp_t gfp,
 	xa_lock_irq(&swap_mapping->i_pages);
 	error = shmem_replace_entry(swap_mapping, swap_index, oldpage, newpage);
 	if (!error) {
-		mem_cgroup_migrate(oldpage, newpage);
+		old = page_folio(oldpage);
+		new = page_folio(newpage);
+		mem_cgroup_migrate(old, new);
 		__inc_lruvec_page_state(newpage, NR_FILE_PAGES);
 		__dec_lruvec_page_state(oldpage, NR_FILE_PAGES);
 	}
-- 
2.30.2

Re: [PATCH v3 11/18] mm/memcg: Convert mem_cgroup_migrate() to take folios

[Christoph Hellwig]

On Wed, Jun 30, 2021 at 05:00:27AM +0100, Matthew Wilcox (Oracle) wrote:
> Convert all callers of mem_cgroup_migrate() to call page_folio() first.

Looks good,

Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>

[PATCH v3 12/18] mm/memcg: Convert mem_cgroup_track_foreign_dirty_slowpath() to folio

[Matthew Wilcox (Oracle)]

The page was only being used for the memcg and to gather trace
information, so this is a simple conversion.  The only caller of
mem_cgroup_track_foreign_dirty() will be converted to folios in a later
patch, so doing this now makes that patch simpler.

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
---
 include/linux/memcontrol.h       | 7 ++++---
 include/trace/events/writeback.h | 8 ++++----
 mm/memcontrol.c                  | 6 +++---
 3 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 2c57a405acd2..ef79f9c0b296 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -1556,17 +1556,18 @@ void mem_cgroup_wb_stats(struct bdi_writeback *wb, unsigned long *pfilepages,
 			 unsigned long *pheadroom, unsigned long *pdirty,
 			 unsigned long *pwriteback);
 
-void mem_cgroup_track_foreign_dirty_slowpath(struct page *page,
+void mem_cgroup_track_foreign_dirty_slowpath(struct folio *folio,
 					     struct bdi_writeback *wb);
 
 static inline void mem_cgroup_track_foreign_dirty(struct page *page,
 						  struct bdi_writeback *wb)
 {
+	struct folio *folio = page_folio(page);
 	if (mem_cgroup_disabled())
 		return;
 
-	if (unlikely(&page_memcg(page)->css != wb->memcg_css))
-		mem_cgroup_track_foreign_dirty_slowpath(page, wb);
+	if (unlikely(&folio_memcg(folio)->css != wb->memcg_css))
+		mem_cgroup_track_foreign_dirty_slowpath(folio, wb);
 }
 
 void mem_cgroup_flush_foreign(struct bdi_writeback *wb);
diff --git a/include/trace/events/writeback.h b/include/trace/events/writeback.h
index 1efa463c4979..80b24801bbf7 100644
--- a/include/trace/events/writeback.h
+++ b/include/trace/events/writeback.h
@@ -235,9 +235,9 @@ TRACE_EVENT(inode_switch_wbs,
 
 TRACE_EVENT(track_foreign_dirty,
 
-	TP_PROTO(struct page *page, struct bdi_writeback *wb),
+	TP_PROTO(struct folio *folio, struct bdi_writeback *wb),
 
-	TP_ARGS(page, wb),
+	TP_ARGS(folio, wb),
 
 	TP_STRUCT__entry(
 		__array(char,		name, 32)
@@ -249,7 +249,7 @@ TRACE_EVENT(track_foreign_dirty,
 	),
 
 	TP_fast_assign(
-		struct address_space *mapping = page_mapping(page);
+		struct address_space *mapping = folio_mapping(folio);
 		struct inode *inode = mapping ? mapping->host : NULL;
 
 		strscpy_pad(__entry->name, bdi_dev_name(wb->bdi), 32);
@@ -257,7 +257,7 @@ TRACE_EVENT(track_foreign_dirty,
 		__entry->ino		= inode ? inode->i_ino : 0;
 		__entry->memcg_id	= wb->memcg_css->id;
 		__entry->cgroup_ino	= __trace_wb_assign_cgroup(wb);
-		__entry->page_cgroup_ino = cgroup_ino(page_memcg(page)->css.cgroup);
+		__entry->page_cgroup_ino = cgroup_ino(folio_memcg(folio)->css.cgroup);
 	),
 
 	TP_printk("bdi %s[%llu]: ino=%lu memcg_id=%u cgroup_ino=%lu page_cgroup_ino=%lu",
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 4ce2f2eb81d8..b925bdce0c6e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -4566,17 +4566,17 @@ void mem_cgroup_wb_stats(struct bdi_writeback *wb, unsigned long *pfilepages,
  * As being wrong occasionally doesn't matter, updates and accesses to the
  * records are lockless and racy.
  */
-void mem_cgroup_track_foreign_dirty_slowpath(struct page *page,
+void mem_cgroup_track_foreign_dirty_slowpath(struct folio *folio,
 					     struct bdi_writeback *wb)
 {
-	struct mem_cgroup *memcg = page_memcg(page);
+	struct mem_cgroup *memcg = folio_memcg(folio);
 	struct memcg_cgwb_frn *frn;
 	u64 now = get_jiffies_64();
 	u64 oldest_at = now;
 	int oldest = -1;
 	int i;
 
-	trace_track_foreign_dirty(page, wb);
+	trace_track_foreign_dirty(folio, wb);
 
 	/*
 	 * Pick the slot to use.  If there is already a slot for @wb, keep
-- 
2.30.2

[PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()

[Matthew Wilcox (Oracle)]

These are the folio equivalents of lock_page_memcg() and
unlock_page_memcg().  Reimplement them as wrappers.

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 include/linux/memcontrol.h | 10 +++++++++
 mm/memcontrol.c            | 45 ++++++++++++++++++++++++--------------
 2 files changed, 39 insertions(+), 16 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index ef79f9c0b296..279ea2640365 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -951,6 +951,8 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg);
 extern bool cgroup_memory_noswap;
 #endif
 
+void folio_memcg_lock(struct folio *folio);
+void folio_memcg_unlock(struct folio *folio);
 void lock_page_memcg(struct page *page);
 void unlock_page_memcg(struct page *page);
 
@@ -1363,6 +1365,14 @@ static inline void unlock_page_memcg(struct page *page)
 {
 }
 
+static inline void folio_memcg_lock(struct folio *folio)
+{
+}
+
+static inline void folio_memcg_unlock(struct folio *folio)
+{
+}
+
 static inline void mem_cgroup_handle_over_high(void)
 {
 }
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index b925bdce0c6e..b94a6122f27d 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1960,18 +1960,17 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg)
 }
 
 /**
- * lock_page_memcg - lock a page and memcg binding
- * @page: the page
+ * folio_memcg_lock - Bind a folio to its memcg.
+ * @folio: The folio.
  *
- * This function protects unlocked LRU pages from being moved to
+ * This function prevents unlocked LRU folios from being moved to
  * another cgroup.
  *
- * It ensures lifetime of the locked memcg. Caller is responsible
- * for the lifetime of the page.
+ * It ensures lifetime of the bound memcg.  The caller is responsible
+ * for the lifetime of the folio.
  */
-void lock_page_memcg(struct page *page)
+void folio_memcg_lock(struct folio *folio)
 {
-	struct page *head = compound_head(page); /* rmap on tail pages */
 	struct mem_cgroup *memcg;
 	unsigned long flags;
 
@@ -1985,7 +1984,7 @@ void lock_page_memcg(struct page *page)
 	if (mem_cgroup_disabled())
 		return;
 again:
-	memcg = page_memcg(head);
+	memcg = folio_memcg(folio);
 	if (unlikely(!memcg))
 		return;
 
@@ -1999,7 +1998,7 @@ void lock_page_memcg(struct page *page)
 		return;
 
 	spin_lock_irqsave(&memcg->move_lock, flags);
-	if (memcg != page_memcg(head)) {
+	if (memcg != folio_memcg(folio)) {
 		spin_unlock_irqrestore(&memcg->move_lock, flags);
 		goto again;
 	}
@@ -2013,9 +2012,15 @@ void lock_page_memcg(struct page *page)
 	memcg->move_lock_task = current;
 	memcg->move_lock_flags = flags;
 }
+EXPORT_SYMBOL(folio_memcg_lock);
+
+void lock_page_memcg(struct page *page)
+{
+	folio_memcg_lock(page_folio(page));
+}
 EXPORT_SYMBOL(lock_page_memcg);
 
-static void __unlock_page_memcg(struct mem_cgroup *memcg)
+static void __memcg_unlock(struct mem_cgroup *memcg)
 {
 	if (memcg && memcg->move_lock_task == current) {
 		unsigned long flags = memcg->move_lock_flags;
@@ -2030,14 +2035,22 @@ static void __unlock_page_memcg(struct mem_cgroup *memcg)
 }
 
 /**
- * unlock_page_memcg - unlock a page and memcg binding
- * @page: the page
+ * folio_memcg_unlock - Release the binding between a folio and its memcg.
+ * @folio: The folio.
+ *
+ * This releases the binding created by folio_memcg_lock().  This does
+ * not change the accounting of this folio to its memcg, but it does
+ * permit others to change it.
  */
-void unlock_page_memcg(struct page *page)
+void folio_memcg_unlock(struct folio *folio)
 {
-	struct page *head = compound_head(page);
+	__memcg_unlock(folio_memcg(folio));
+}
+EXPORT_SYMBOL(folio_memcg_unlock);
 
-	__unlock_page_memcg(page_memcg(head));
+void unlock_page_memcg(struct page *page)
+{
+	folio_memcg_unlock(page_folio(page));
 }
 EXPORT_SYMBOL(unlock_page_memcg);
 
@@ -5661,7 +5674,7 @@ static int mem_cgroup_move_account(struct page *page,
 
 	page->memcg_data = (unsigned long)to;
 
-	__unlock_page_memcg(from);
+	__memcg_unlock(from);
 
 	ret = 0;
 	nid = page_to_nid(page);
-- 
2.30.2

Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()

[Michal Hocko]

On Wed 30-06-21 05:00:29, Matthew Wilcox wrote:
> These are the folio equivalents of lock_page_memcg() and
> unlock_page_memcg().  Reimplement them as wrappers.

Is there any reason why you haven't followed the same approach as for
the previous patches. I mean callers can call page_folio and then
lock_page_memcg wrapper shouldn't be really needed.

I do not really want to be annoying here but I have to say that I like
the conversion by previous patches much better than this wrapper
approach as mentioned during the previous review already. If you have
some reasons to stick with this approach for this particular case then
make it explicit in the changelog.

> Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
> ---
>  include/linux/memcontrol.h | 10 +++++++++
>  mm/memcontrol.c            | 45 ++++++++++++++++++++++++--------------
>  2 files changed, 39 insertions(+), 16 deletions(-)
> 
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index ef79f9c0b296..279ea2640365 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -951,6 +951,8 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg);
>  extern bool cgroup_memory_noswap;
>  #endif
>  
> +void folio_memcg_lock(struct folio *folio);
> +void folio_memcg_unlock(struct folio *folio);
>  void lock_page_memcg(struct page *page);
>  void unlock_page_memcg(struct page *page);
>  
> @@ -1363,6 +1365,14 @@ static inline void unlock_page_memcg(struct page *page)
>  {
>  }
>  
> +static inline void folio_memcg_lock(struct folio *folio)
> +{
> +}
> +
> +static inline void folio_memcg_unlock(struct folio *folio)
> +{
> +}
> +
>  static inline void mem_cgroup_handle_over_high(void)
>  {
>  }
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index b925bdce0c6e..b94a6122f27d 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -1960,18 +1960,17 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg)
>  }
>  
>  /**
> - * lock_page_memcg - lock a page and memcg binding
> - * @page: the page
> + * folio_memcg_lock - Bind a folio to its memcg.
> + * @folio: The folio.
>   *
> - * This function protects unlocked LRU pages from being moved to
> + * This function prevents unlocked LRU folios from being moved to
>   * another cgroup.
>   *
> - * It ensures lifetime of the locked memcg. Caller is responsible
> - * for the lifetime of the page.
> + * It ensures lifetime of the bound memcg.  The caller is responsible
> + * for the lifetime of the folio.
>   */
> -void lock_page_memcg(struct page *page)
> +void folio_memcg_lock(struct folio *folio)
>  {
> -	struct page *head = compound_head(page); /* rmap on tail pages */
>  	struct mem_cgroup *memcg;
>  	unsigned long flags;
>  
> @@ -1985,7 +1984,7 @@ void lock_page_memcg(struct page *page)
>  	if (mem_cgroup_disabled())
>  		return;
>  again:
> -	memcg = page_memcg(head);
> +	memcg = folio_memcg(folio);
>  	if (unlikely(!memcg))
>  		return;
>  
> @@ -1999,7 +1998,7 @@ void lock_page_memcg(struct page *page)
>  		return;
>  
>  	spin_lock_irqsave(&memcg->move_lock, flags);
> -	if (memcg != page_memcg(head)) {
> +	if (memcg != folio_memcg(folio)) {
>  		spin_unlock_irqrestore(&memcg->move_lock, flags);
>  		goto again;
>  	}
> @@ -2013,9 +2012,15 @@ void lock_page_memcg(struct page *page)
>  	memcg->move_lock_task = current;
>  	memcg->move_lock_flags = flags;
>  }
> +EXPORT_SYMBOL(folio_memcg_lock);
> +
> +void lock_page_memcg(struct page *page)
> +{
> +	folio_memcg_lock(page_folio(page));
> +}
>  EXPORT_SYMBOL(lock_page_memcg);
>  
> -static void __unlock_page_memcg(struct mem_cgroup *memcg)
> +static void __memcg_unlock(struct mem_cgroup *memcg)
>  {
>  	if (memcg && memcg->move_lock_task == current) {
>  		unsigned long flags = memcg->move_lock_flags;
> @@ -2030,14 +2035,22 @@ static void __unlock_page_memcg(struct mem_cgroup *memcg)
>  }
>  
>  /**
> - * unlock_page_memcg - unlock a page and memcg binding
> - * @page: the page
> + * folio_memcg_unlock - Release the binding between a folio and its memcg.
> + * @folio: The folio.
> + *
> + * This releases the binding created by folio_memcg_lock().  This does
> + * not change the accounting of this folio to its memcg, but it does
> + * permit others to change it.
>   */
> -void unlock_page_memcg(struct page *page)
> +void folio_memcg_unlock(struct folio *folio)
>  {
> -	struct page *head = compound_head(page);
> +	__memcg_unlock(folio_memcg(folio));
> +}
> +EXPORT_SYMBOL(folio_memcg_unlock);
>  
> -	__unlock_page_memcg(page_memcg(head));
> +void unlock_page_memcg(struct page *page)
> +{
> +	folio_memcg_unlock(page_folio(page));
>  }
>  EXPORT_SYMBOL(unlock_page_memcg);
>  
> @@ -5661,7 +5674,7 @@ static int mem_cgroup_move_account(struct page *page,
>  
>  	page->memcg_data = (unsigned long)to;
>  
> -	__unlock_page_memcg(from);
> +	__memcg_unlock(from);
>  
>  	ret = 0;
>  	nid = page_to_nid(page);
> -- 
> 2.30.2

-- 
Michal Hocko
SUSE Labs

Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()

[Matthew Wilcox]

On Wed, Jun 30, 2021 at 10:32:02AM +0200, Michal Hocko wrote:
> On Wed 30-06-21 05:00:29, Matthew Wilcox wrote:
> > These are the folio equivalents of lock_page_memcg() and
> > unlock_page_memcg().  Reimplement them as wrappers.
> 
> Is there any reason why you haven't followed the same approach as for
> the previous patches. I mean callers can call page_folio and then
> lock_page_memcg wrapper shouldn't be really needed.

At this point in the patch series there are ~20 places which call
lock_page_memcg().  I think it makes more sense to leave the wrapper
in place, and then we can remove the wrapper once all/most of these
places are converted to use folios.  There are another 5 conversions
already in the patch series, eg here:

https://git.infradead.org/users/willy/pagecache.git/commitdiff/a41c942c8e4b41df30be128ef6998ff1849fa36a

> I do not really want to be annoying here but I have to say that I like
> the conversion by previous patches much better than this wrapper
> approach as mentioned during the previous review already. If you have
> some reasons to stick with this approach for this particular case then
> make it explicit in the changelog.

OK, I can point to the number of callers as a reason to keep the
wrappers in place.  I intended to just do the conversion here, but
seeing the number of callers made me reconsider.

Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()

[Michal Hocko]

On Wed 07-07-21 16:10:49, Matthew Wilcox wrote:
[...]
> > I do not really want to be annoying here but I have to say that I like
> > the conversion by previous patches much better than this wrapper
> > approach as mentioned during the previous review already. If you have
> > some reasons to stick with this approach for this particular case then
> > make it explicit in the changelog.
> 
> OK, I can point to the number of callers as a reason to keep the
> wrappers in place.  I intended to just do the conversion here, but
> seeing the number of callers made me reconsider.

OK, fair enough. My worry is that we will have this lingering for way
too long. People simply tend to copy code... Anyway, please add a
comment warning that the wrapper shouldn't be used in any new code at
least.

Thanks!
-- 
Michal Hocko
SUSE Labs

Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()

[Johannes Weiner]

On Wed, Jun 30, 2021 at 05:00:29AM +0100, Matthew Wilcox (Oracle) wrote:
> -static void __unlock_page_memcg(struct mem_cgroup *memcg)
> +static void __memcg_unlock(struct mem_cgroup *memcg)

This is too generic a name. There are several locks in the memcg, and
this one only locks the page->memcg bindings in the group.

Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()

[Matthew Wilcox]

On Wed, Jul 07, 2021 at 01:08:51PM -0400, Johannes Weiner wrote:
> On Wed, Jun 30, 2021 at 05:00:29AM +0100, Matthew Wilcox (Oracle) wrote:
> > -static void __unlock_page_memcg(struct mem_cgroup *memcg)
> > +static void __memcg_unlock(struct mem_cgroup *memcg)
> 
> This is too generic a name. There are several locks in the memcg, and
> this one only locks the page->memcg bindings in the group.

Fair.  __memcg_move_unlock looks like the right name to me?

Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()

[Johannes Weiner]

On Wed, Jul 07, 2021 at 08:28:39PM +0100, Matthew Wilcox wrote:
> On Wed, Jul 07, 2021 at 01:08:51PM -0400, Johannes Weiner wrote:
> > On Wed, Jun 30, 2021 at 05:00:29AM +0100, Matthew Wilcox (Oracle) wrote:
> > > -static void __unlock_page_memcg(struct mem_cgroup *memcg)
> > > +static void __memcg_unlock(struct mem_cgroup *memcg)
> > 
> > This is too generic a name. There are several locks in the memcg, and
> > this one only locks the page->memcg bindings in the group.
> 
> Fair.  __memcg_move_unlock looks like the right name to me?

Could you please elaborate what the problem with the current name is?

mem_cgroup_move_account() does this:

	lock_page_memcg(page);
	page->memcg = to;
	__unlock_page_memcg(from);

It locks and unlocks the page->memcg binding which can be done coming
from the page or the memcg. The current names are symmetrical to
reflect that it's the same lock.

We could switch them both to move_lock, but as per the other email,
lock_page_memcg() was chosen to resemble lock_page(). Because from a
memcg POV they're interchangeable - the former is just a more narrowly
scoped version for contexts that don't hold the page lock. It used to
be called something else and we had several contexts taking redundant
locks on accident because this hierarchy wasn't clear.

I don't mind fixing poorly chosen or misleading naming schemes, but I
think we need better explanations to overcome the reasoning behind the
existing names, not just the assumption that there weren't any.

Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()

[Matthew Wilcox]

On Wed, Jul 07, 2021 at 04:41:05PM -0400, Johannes Weiner wrote:
> On Wed, Jul 07, 2021 at 08:28:39PM +0100, Matthew Wilcox wrote:
> > On Wed, Jul 07, 2021 at 01:08:51PM -0400, Johannes Weiner wrote:
> > > On Wed, Jun 30, 2021 at 05:00:29AM +0100, Matthew Wilcox (Oracle) wrote:
> > > > -static void __unlock_page_memcg(struct mem_cgroup *memcg)
> > > > +static void __memcg_unlock(struct mem_cgroup *memcg)
> > > 
> > > This is too generic a name. There are several locks in the memcg, and
> > > this one only locks the page->memcg bindings in the group.
> > 
> > Fair.  __memcg_move_unlock looks like the right name to me?
> 
> Could you please elaborate what the problem with the current name is?
> 
> mem_cgroup_move_account() does this:
> 
> 	lock_page_memcg(page);
> 	page->memcg = to;
> 	__unlock_page_memcg(from);
> 
> It locks and unlocks the page->memcg binding which can be done coming
> from the page or the memcg. The current names are symmetrical to
> reflect that it's the same lock.

OK, so in the prerequisite series to this patch, lock_page() becomes
folio_lock().  This series turns lock_page_memcg() into
folio_memcg_lock().  As a minimum, then, this needs to turn into
__folio_memcg_unlock().

> We could switch them both to move_lock, but as per the other email,
> lock_page_memcg() was chosen to resemble lock_page(). Because from a
> memcg POV they're interchangeable - the former is just a more narrowly
> scoped version for contexts that don't hold the page lock. It used to
> be called something else and we had several contexts taking redundant
> locks on accident because this hierarchy wasn't clear.

Unfortunately, it's still not clear.  I've answered questions from
people who think that they have the page locked because they called
lock_page_memcg() ;-(

[PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio

[Matthew Wilcox (Oracle)]

This saves dozens of bytes of text by eliminating a lot of calls to
compound_head().

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 mm/memcontrol.c | 37 +++++++++++++++++++------------------
 1 file changed, 19 insertions(+), 18 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index b94a6122f27d..95795b65ae3e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -5585,38 +5585,39 @@ static int mem_cgroup_move_account(struct page *page,
 				   struct mem_cgroup *from,
 				   struct mem_cgroup *to)
 {
+	struct folio *folio = page_folio(page);
 	struct lruvec *from_vec, *to_vec;
 	struct pglist_data *pgdat;
-	unsigned int nr_pages = compound ? thp_nr_pages(page) : 1;
+	unsigned int nr_pages = compound ? folio_nr_pages(folio) : 1;
 	int nid, ret;
 
 	VM_BUG_ON(from == to);
-	VM_BUG_ON_PAGE(PageLRU(page), page);
-	VM_BUG_ON(compound && !PageTransHuge(page));
+	VM_BUG_ON_FOLIO(folio_lru(folio), folio);
+	VM_BUG_ON(compound && !folio_multi(folio));
 
 	/*
 	 * Prevent mem_cgroup_migrate() from looking at
 	 * page's memory cgroup of its source page while we change it.
 	 */
 	ret = -EBUSY;
-	if (!trylock_page(page))
+	if (!folio_trylock(folio))
 		goto out;
 
 	ret = -EINVAL;
-	if (page_memcg(page) != from)
+	if (folio_memcg(folio) != from)
 		goto out_unlock;
 
-	pgdat = page_pgdat(page);
+	pgdat = folio_pgdat(folio);
 	from_vec = mem_cgroup_lruvec(from, pgdat);
 	to_vec = mem_cgroup_lruvec(to, pgdat);
 
-	lock_page_memcg(page);
+	folio_memcg_lock(folio);
 
-	if (PageAnon(page)) {
-		if (page_mapped(page)) {
+	if (folio_anon(folio)) {
+		if (folio_mapped(folio)) {
 			__mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages);
 			__mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages);
-			if (PageTransHuge(page)) {
+			if (folio_multi(folio)) {
 				__mod_lruvec_state(from_vec, NR_ANON_THPS,
 						   -nr_pages);
 				__mod_lruvec_state(to_vec, NR_ANON_THPS,
@@ -5627,18 +5628,18 @@ static int mem_cgroup_move_account(struct page *page,
 		__mod_lruvec_state(from_vec, NR_FILE_PAGES, -nr_pages);
 		__mod_lruvec_state(to_vec, NR_FILE_PAGES, nr_pages);
 
-		if (PageSwapBacked(page)) {
+		if (folio_swapbacked(folio)) {
 			__mod_lruvec_state(from_vec, NR_SHMEM, -nr_pages);
 			__mod_lruvec_state(to_vec, NR_SHMEM, nr_pages);
 		}
 
-		if (page_mapped(page)) {
+		if (folio_mapped(folio)) {
 			__mod_lruvec_state(from_vec, NR_FILE_MAPPED, -nr_pages);
 			__mod_lruvec_state(to_vec, NR_FILE_MAPPED, nr_pages);
 		}
 
-		if (PageDirty(page)) {
-			struct address_space *mapping = page_mapping(page);
+		if (folio_dirty(folio)) {
+			struct address_space *mapping = folio_mapping(folio);
 
 			if (mapping_can_writeback(mapping)) {
 				__mod_lruvec_state(from_vec, NR_FILE_DIRTY,
@@ -5649,7 +5650,7 @@ static int mem_cgroup_move_account(struct page *page,
 		}
 	}
 
-	if (PageWriteback(page)) {
+	if (folio_writeback(folio)) {
 		__mod_lruvec_state(from_vec, NR_WRITEBACK, -nr_pages);
 		__mod_lruvec_state(to_vec, NR_WRITEBACK, nr_pages);
 	}
@@ -5672,12 +5673,12 @@ static int mem_cgroup_move_account(struct page *page,
 	css_get(&to->css);
 	css_put(&from->css);
 
-	page->memcg_data = (unsigned long)to;
+	folio->memcg_data = (unsigned long)to;
 
 	__memcg_unlock(from);
 
 	ret = 0;
-	nid = page_to_nid(page);
+	nid = folio_nid(folio);
 
 	local_irq_disable();
 	mem_cgroup_charge_statistics(to, nr_pages);
@@ -5686,7 +5687,7 @@ static int mem_cgroup_move_account(struct page *page,
 	memcg_check_events(from, nid);
 	local_irq_enable();
 out_unlock:
-	unlock_page(page);
+	folio_unlock(folio);
 out:
 	return ret;
 }
-- 
2.30.2

Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio

[Michal Hocko]

On Wed 30-06-21 05:00:30, Matthew Wilcox wrote:
> This saves dozens of bytes of text by eliminating a lot of calls to
> compound_head().
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
> ---
>  mm/memcontrol.c | 37 +++++++++++++++++++------------------
>  1 file changed, 19 insertions(+), 18 deletions(-)
> 
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index b94a6122f27d..95795b65ae3e 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -5585,38 +5585,39 @@ static int mem_cgroup_move_account(struct page *page,
>  				   struct mem_cgroup *from,
>  				   struct mem_cgroup *to)
>  {
> +	struct folio *folio = page_folio(page);
>  	struct lruvec *from_vec, *to_vec;
>  	struct pglist_data *pgdat;
> -	unsigned int nr_pages = compound ? thp_nr_pages(page) : 1;
> +	unsigned int nr_pages = compound ? folio_nr_pages(folio) : 1;
>  	int nid, ret;
>  
>  	VM_BUG_ON(from == to);
> -	VM_BUG_ON_PAGE(PageLRU(page), page);
> -	VM_BUG_ON(compound && !PageTransHuge(page));
> +	VM_BUG_ON_FOLIO(folio_lru(folio), folio);
> +	VM_BUG_ON(compound && !folio_multi(folio));
>  
>  	/*
>  	 * Prevent mem_cgroup_migrate() from looking at
>  	 * page's memory cgroup of its source page while we change it.
>  	 */
>  	ret = -EBUSY;
> -	if (!trylock_page(page))
> +	if (!folio_trylock(folio))
>  		goto out;
>  
>  	ret = -EINVAL;
> -	if (page_memcg(page) != from)
> +	if (folio_memcg(folio) != from)
>  		goto out_unlock;
>  
> -	pgdat = page_pgdat(page);
> +	pgdat = folio_pgdat(folio);
>  	from_vec = mem_cgroup_lruvec(from, pgdat);
>  	to_vec = mem_cgroup_lruvec(to, pgdat);
>  
> -	lock_page_memcg(page);
> +	folio_memcg_lock(folio);
>  
> -	if (PageAnon(page)) {
> -		if (page_mapped(page)) {
> +	if (folio_anon(folio)) {
> +		if (folio_mapped(folio)) {
>  			__mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages);
>  			__mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages);
> -			if (PageTransHuge(page)) {
> +			if (folio_multi(folio)) {

Shouldn't be folio_transhuge? The resulting code is the same but
folio_transhuge is more explicit and matches the THP aspect.

-- 
Michal Hocko
SUSE Labs

Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio

[Matthew Wilcox]

On Wed, Jun 30, 2021 at 10:30:38AM +0200, Michal Hocko wrote:
> > -	if (PageAnon(page)) {
> > -		if (page_mapped(page)) {
> > +	if (folio_anon(folio)) {
> > +		if (folio_mapped(folio)) {
> >  			__mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages);
> >  			__mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages);
> > -			if (PageTransHuge(page)) {
> > +			if (folio_multi(folio)) {
> 
> Shouldn't be folio_transhuge? The resulting code is the same but
> folio_transhuge is more explicit and matches the THP aspect.

I genuinely don't know.  For the benefit of those reading along, the
important part of the context is:

                if (folio_mapped(folio)) {
                        __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages);
                        __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages);
                        if (folio_multi(folio)) {
                                __mod_lruvec_state(from_vec, NR_ANON_THPS,
                                                   -nr_pages);
                                __mod_lruvec_state(to_vec, NR_ANON_THPS,
                                                   nr_pages);
                        }
                }

We need to decide what 'NR_ANON_THPS' means in a folio-based world where
we have folios of all orders.  Does it count only the number of pages
in folios >= HPAGE_PMD_SIZE?  Or does it count the number of pages in
folios > PAGE_SIZE?

Similar question (and I suspect the same answer) for NR_SHMEM_THPS and
NR_FILE_THPS.  Right now, I've been accounting any multi-page folio as
a THP, but I don't have a good sense of what the answer should be.

Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio

[Michal Hocko]

On Wed 30-06-21 12:22:48, Matthew Wilcox wrote:
> On Wed, Jun 30, 2021 at 10:30:38AM +0200, Michal Hocko wrote:
> > > -	if (PageAnon(page)) {
> > > -		if (page_mapped(page)) {
> > > +	if (folio_anon(folio)) {
> > > +		if (folio_mapped(folio)) {
> > >  			__mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages);
> > >  			__mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages);
> > > -			if (PageTransHuge(page)) {
> > > +			if (folio_multi(folio)) {
> > 
> > Shouldn't be folio_transhuge? The resulting code is the same but
> > folio_transhuge is more explicit and matches the THP aspect.
> 
> I genuinely don't know.  For the benefit of those reading along, the
> important part of the context is:
> 
>                 if (folio_mapped(folio)) {
>                         __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages);
>                         __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages);
>                         if (folio_multi(folio)) {
>                                 __mod_lruvec_state(from_vec, NR_ANON_THPS,
>                                                    -nr_pages);
>                                 __mod_lruvec_state(to_vec, NR_ANON_THPS,
>                                                    nr_pages);
>                         }
>                 }
> 
> We need to decide what 'NR_ANON_THPS' means in a folio-based world where
> we have folios of all orders.  Does it count only the number of pages
> in folios >= HPAGE_PMD_SIZE?  Or does it count the number of pages in
> folios > PAGE_SIZE?

At this stage we only have PMD based, right? I believe it would be
simpler to stick with that at the moment and change that to a more
generic way along with other places which need updating.

Wrt. counters they do count pages so in this case this shouldn't be a
problem. But we do have counters for pmd mappings and that might need
some care.
-- 
Michal Hocko
SUSE Labs

Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio

[Matthew Wilcox]

On Wed, Jun 30, 2021 at 02:20:38PM +0200, Michal Hocko wrote:
> On Wed 30-06-21 12:22:48, Matthew Wilcox wrote:
> > We need to decide what 'NR_ANON_THPS' means in a folio-based world where
> > we have folios of all orders.  Does it count only the number of pages
> > in folios >= HPAGE_PMD_SIZE?  Or does it count the number of pages in
> > folios > PAGE_SIZE?
> 
> At this stage we only have PMD based, right? I believe it would be
> simpler to stick with that at the moment and change that to a more
> generic way along with other places which need updating.
> 
> Wrt. counters they do count pages so in this case this shouldn't be a
> problem. But we do have counters for pmd mappings and that might need
> some care.

Looking at how these are reported:

        show_val_kb(m, "AnonHugePages:  ",
                    global_node_page_state(NR_ANON_THPS));
        show_val_kb(m, "ShmemHugePages: ",
                    global_node_page_state(NR_SHMEM_THPS));
        show_val_kb(m, "ShmemPmdMapped: ",
                    global_node_page_state(NR_SHMEM_PMDMAPPED));
        show_val_kb(m, "FileHugePages:  ",
                    global_node_page_state(NR_FILE_THPS));
        show_val_kb(m, "FilePmdMapped:  ",
                    global_node_page_state(NR_FILE_PMDMAPPED));

it specifically refers to 'HugePages', so I think we need to only
count folios with order >= PMD_ORDER.  I'll make that change to
folio_transhuge() and use folio_transhuge() here.

Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio

[Michal Hocko]

On Wed 30-06-21 13:31:17, Matthew Wilcox wrote:
> On Wed, Jun 30, 2021 at 02:20:38PM +0200, Michal Hocko wrote:
> > On Wed 30-06-21 12:22:48, Matthew Wilcox wrote:
> > > We need to decide what 'NR_ANON_THPS' means in a folio-based world where
> > > we have folios of all orders.  Does it count only the number of pages
> > > in folios >= HPAGE_PMD_SIZE?  Or does it count the number of pages in
> > > folios > PAGE_SIZE?
> > 
> > At this stage we only have PMD based, right? I believe it would be
> > simpler to stick with that at the moment and change that to a more
> > generic way along with other places which need updating.
> > 
> > Wrt. counters they do count pages so in this case this shouldn't be a
> > problem. But we do have counters for pmd mappings and that might need
> > some care.
> 
> Looking at how these are reported:
> 
>         show_val_kb(m, "AnonHugePages:  ",
>                     global_node_page_state(NR_ANON_THPS));
>         show_val_kb(m, "ShmemHugePages: ",
>                     global_node_page_state(NR_SHMEM_THPS));
>         show_val_kb(m, "ShmemPmdMapped: ",
>                     global_node_page_state(NR_SHMEM_PMDMAPPED));
>         show_val_kb(m, "FileHugePages:  ",
>                     global_node_page_state(NR_FILE_THPS));
>         show_val_kb(m, "FilePmdMapped:  ",
>                     global_node_page_state(NR_FILE_PMDMAPPED));
> 
> it specifically refers to 'HugePages', so I think we need to only
> count folios with order >= PMD_ORDER.

Why? The presented value is in kB. It gives us a cumulative number of
transparent large pages.  Sure breakdown to respective orders would be
impossible in general but the same would be the case if order > PMD_ORDER.

I am not really sure how useful that information is in practice but that
is a different story.

> I'll make that change to
> folio_transhuge() and use folio_transhuge() here.

Thanks!

-- 
Michal Hocko
SUSE Labs

Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio

[Matthew Wilcox]

On Wed, Jun 30, 2021 at 02:45:33PM +0200, Michal Hocko wrote:
> On Wed 30-06-21 13:31:17, Matthew Wilcox wrote:
> > On Wed, Jun 30, 2021 at 02:20:38PM +0200, Michal Hocko wrote:
> > > On Wed 30-06-21 12:22:48, Matthew Wilcox wrote:
> > > > We need to decide what 'NR_ANON_THPS' means in a folio-based world where
> > > > we have folios of all orders.  Does it count only the number of pages
> > > > in folios >= HPAGE_PMD_SIZE?  Or does it count the number of pages in
> > > > folios > PAGE_SIZE?
> > > 
> > > At this stage we only have PMD based, right? I believe it would be
> > > simpler to stick with that at the moment and change that to a more
> > > generic way along with other places which need updating.
> > > 
> > > Wrt. counters they do count pages so in this case this shouldn't be a
> > > problem. But we do have counters for pmd mappings and that might need
> > > some care.
> > 
> > Looking at how these are reported:
> > 
> >         show_val_kb(m, "AnonHugePages:  ",
> >                     global_node_page_state(NR_ANON_THPS));
> >         show_val_kb(m, "ShmemHugePages: ",
> >                     global_node_page_state(NR_SHMEM_THPS));
> >         show_val_kb(m, "ShmemPmdMapped: ",
> >                     global_node_page_state(NR_SHMEM_PMDMAPPED));
> >         show_val_kb(m, "FileHugePages:  ",
> >                     global_node_page_state(NR_FILE_THPS));
> >         show_val_kb(m, "FilePmdMapped:  ",
> >                     global_node_page_state(NR_FILE_PMDMAPPED));
> > 
> > it specifically refers to 'HugePages', so I think we need to only
> > count folios with order >= PMD_ORDER.
> 
> Why? The presented value is in kB. It gives us a cumulative number of
> transparent large pages.  Sure breakdown to respective orders would be
> impossible in general but the same would be the case if order > PMD_ORDER.
> 
> I am not really sure how useful that information is in practice but that
> is a different story.

The scenario I'm thinking about is a situation where we have gigabytes
of memory in the page cache in 16k-64k chunks and we'll see
FileHugePages: 5219348 kB
FilePmdMapped:       0 kB

which might cause the slightly-too-clever user to think there's a
problem.

Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio

[Michal Hocko]

On Wed 07-07-21 16:25:07, Matthew Wilcox wrote:
> On Wed, Jun 30, 2021 at 02:45:33PM +0200, Michal Hocko wrote:
> > On Wed 30-06-21 13:31:17, Matthew Wilcox wrote:
> > > On Wed, Jun 30, 2021 at 02:20:38PM +0200, Michal Hocko wrote:
> > > > On Wed 30-06-21 12:22:48, Matthew Wilcox wrote:
> > > > > We need to decide what 'NR_ANON_THPS' means in a folio-based world where
> > > > > we have folios of all orders.  Does it count only the number of pages
> > > > > in folios >= HPAGE_PMD_SIZE?  Or does it count the number of pages in
> > > > > folios > PAGE_SIZE?
> > > > 
> > > > At this stage we only have PMD based, right? I believe it would be
> > > > simpler to stick with that at the moment and change that to a more
> > > > generic way along with other places which need updating.
> > > > 
> > > > Wrt. counters they do count pages so in this case this shouldn't be a
> > > > problem. But we do have counters for pmd mappings and that might need
> > > > some care.
> > > 
> > > Looking at how these are reported:
> > > 
> > >         show_val_kb(m, "AnonHugePages:  ",
> > >                     global_node_page_state(NR_ANON_THPS));
> > >         show_val_kb(m, "ShmemHugePages: ",
> > >                     global_node_page_state(NR_SHMEM_THPS));
> > >         show_val_kb(m, "ShmemPmdMapped: ",
> > >                     global_node_page_state(NR_SHMEM_PMDMAPPED));
> > >         show_val_kb(m, "FileHugePages:  ",
> > >                     global_node_page_state(NR_FILE_THPS));
> > >         show_val_kb(m, "FilePmdMapped:  ",
> > >                     global_node_page_state(NR_FILE_PMDMAPPED));
> > > 
> > > it specifically refers to 'HugePages', so I think we need to only
> > > count folios with order >= PMD_ORDER.
> > 
> > Why? The presented value is in kB. It gives us a cumulative number of
> > transparent large pages.  Sure breakdown to respective orders would be
> > impossible in general but the same would be the case if order > PMD_ORDER.
> > 
> > I am not really sure how useful that information is in practice but that
> > is a different story.
> 
> The scenario I'm thinking about is a situation where we have gigabytes
> of memory in the page cache in 16k-64k chunks and we'll see
> FileHugePages: 5219348 kB
> FilePmdMapped:       0 kB
> 
> which might cause the slightly-too-clever user to think there's a
> problem.

Well, cases like this one shouldn't be really hard to explain though so
I wouldn't be worried about this.

-- 
Michal Hocko
SUSE Labs

[PATCH v3 15/18] mm/memcg: Add mem_cgroup_folio_lruvec()

[Matthew Wilcox (Oracle)]

This is the folio equivalent of mem_cgroup_page_lruvec().

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 include/linux/memcontrol.h | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 279ea2640365..a7e1ccbc7ed6 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -752,18 +752,17 @@ static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg,
 }
 
 /**
- * mem_cgroup_page_lruvec - return lruvec for isolating/putting an LRU page
- * @page: the page
+ * mem_cgroup_folio_lruvec - return lruvec for isolating/putting an LRU folio
+ * @folio: Pointer to the folio.
  *
- * This function relies on page->mem_cgroup being stable.
+ * This function relies on folio->mem_cgroup being stable.
  */
-static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page)
+static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
 {
-	pg_data_t *pgdat = page_pgdat(page);
-	struct mem_cgroup *memcg = page_memcg(page);
+	struct mem_cgroup *memcg = folio_memcg(folio);
 
-	VM_WARN_ON_ONCE_PAGE(!memcg && !mem_cgroup_disabled(), page);
-	return mem_cgroup_lruvec(memcg, pgdat);
+	VM_WARN_ON_ONCE_FOLIO(!memcg && !mem_cgroup_disabled(), folio);
+	return mem_cgroup_lruvec(memcg, folio_pgdat(folio));
 }
 
 struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
@@ -1222,10 +1221,9 @@ static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg,
 	return &pgdat->__lruvec;
 }
 
-static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page)
+static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
 {
-	pg_data_t *pgdat = page_pgdat(page);
-
+	struct pglist_data *pgdat = folio_pgdat(folio);
 	return &pgdat->__lruvec;
 }
 
@@ -1485,6 +1483,11 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
 }
 #endif /* CONFIG_MEMCG */
 
+static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page)
+{
+	return mem_cgroup_folio_lruvec(page_folio(page));
+}
+
 static inline void __inc_lruvec_kmem_state(void *p, enum node_stat_item idx)
 {
 	__mod_lruvec_kmem_state(p, idx, 1);
-- 
2.30.2

Re: [PATCH v3 15/18] mm/memcg: Add mem_cgroup_folio_lruvec()

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 11172 bytes --]

Hi "Matthew,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on linus/master]
[also build test WARNING on next-20210629]
[cannot apply to hnaz-linux-mm/master tip/perf/core cgroup/for-next v5.13]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 007b350a58754a93ca9fe50c498cc27780171153
config: arm64-randconfig-c024-20210630 (attached as .config)
compiler: aarch64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/0day-ci/linux/commit/c7ce93681b922cb2b709109c21e8206ef623e0be
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
        git checkout c7ce93681b922cb2b709109c21e8206ef623e0be
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

All warnings (new ones prefixed by >>):

   In file included from arch/arm64/kernel/asm-offsets.c:12:
   include/linux/mm.h:1382:42: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1382 | static inline int folio_nid(const struct folio *folio)
         |                                          ^~~~~
   include/linux/mm.h: In function 'folio_nid':
   include/linux/mm.h:1384:27: error: dereferencing pointer to incomplete type 'const struct folio'
    1384 |  return page_to_nid(&folio->page);
         |                           ^~
   In file included from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/arm64/kernel/asm-offsets.c:16:
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:1138:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1138 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
   include/linux/memcontrol.h:1190:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1190 | static inline int mem_cgroup_charge(struct folio *folio,
         |                                            ^~~~~
   include/linux/memcontrol.h:1206:47: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1206 | static inline void mem_cgroup_uncharge(struct folio *folio)
         |                                               ^~~~~
   include/linux/memcontrol.h:1214:46: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1214 | static inline void mem_cgroup_migrate(struct folio *old, struct folio *new)
         |                                              ^~~~~
   include/linux/memcontrol.h:1224:61: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1224 | static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
         |                                                             ^~~~~
   include/linux/memcontrol.h: In function 'mem_cgroup_folio_lruvec':
   include/linux/memcontrol.h:1226:30: error: implicit declaration of function 'folio_pgdat'; did you mean 'folio_nid'? [-Werror=implicit-function-declaration]
    1226 |  struct pglist_data *pgdat = folio_pgdat(folio);
         |                              ^~~~~~~~~~~
         |                              folio_nid
>> include/linux/memcontrol.h:1226:30: warning: initialization of 'struct pglist_data *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:1366:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1366 | static inline void folio_memcg_lock(struct folio *folio)
         |                                            ^~~~~
   include/linux/memcontrol.h:1370:46: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1370 | static inline void folio_memcg_unlock(struct folio *folio)
         |                                              ^~~~~
   include/linux/memcontrol.h: In function 'mem_cgroup_page_lruvec':
   include/linux/memcontrol.h:1488:33: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
    1488 |  return mem_cgroup_folio_lruvec(page_folio(page));
         |                                 ^~~~~~~~~~
   include/linux/memcontrol.h:1488:33: warning: passing argument 1 of 'mem_cgroup_folio_lruvec' makes pointer from integer without a cast [-Wint-conversion]
    1488 |  return mem_cgroup_folio_lruvec(page_folio(page));
         |                                 ^~~~~~~~~~~~~~~~
         |                                 |
         |                                 int
   include/linux/memcontrol.h:1224:68: note: expected 'struct folio *' but argument is of type 'int'
    1224 | static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
         |                                                      ~~~~~~~~~~~~~~^~~~~
   cc1: some warnings being treated as errors
--
   In file included from arch/arm64/kernel/asm-offsets.c:12:
   include/linux/mm.h:1382:42: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1382 | static inline int folio_nid(const struct folio *folio)
         |                                          ^~~~~
   include/linux/mm.h: In function 'folio_nid':
   include/linux/mm.h:1384:27: error: dereferencing pointer to incomplete type 'const struct folio'
    1384 |  return page_to_nid(&folio->page);
         |                           ^~
   In file included from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/arm64/kernel/asm-offsets.c:16:
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:1138:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1138 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
   include/linux/memcontrol.h:1190:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1190 | static inline int mem_cgroup_charge(struct folio *folio,
         |                                            ^~~~~
   include/linux/memcontrol.h:1206:47: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1206 | static inline void mem_cgroup_uncharge(struct folio *folio)
         |                                               ^~~~~
   include/linux/memcontrol.h:1214:46: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1214 | static inline void mem_cgroup_migrate(struct folio *old, struct folio *new)
         |                                              ^~~~~
   include/linux/memcontrol.h:1224:61: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1224 | static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
         |                                                             ^~~~~
   include/linux/memcontrol.h: In function 'mem_cgroup_folio_lruvec':
   include/linux/memcontrol.h:1226:30: error: implicit declaration of function 'folio_pgdat'; did you mean 'folio_nid'? [-Werror=implicit-function-declaration]
    1226 |  struct pglist_data *pgdat = folio_pgdat(folio);
         |                              ^~~~~~~~~~~
         |                              folio_nid
>> include/linux/memcontrol.h:1226:30: warning: initialization of 'struct pglist_data *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:1366:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1366 | static inline void folio_memcg_lock(struct folio *folio)
         |                                            ^~~~~
   include/linux/memcontrol.h:1370:46: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1370 | static inline void folio_memcg_unlock(struct folio *folio)
         |                                              ^~~~~
   include/linux/memcontrol.h: In function 'mem_cgroup_page_lruvec':
   include/linux/memcontrol.h:1488:33: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
    1488 |  return mem_cgroup_folio_lruvec(page_folio(page));
         |                                 ^~~~~~~~~~
   include/linux/memcontrol.h:1488:33: warning: passing argument 1 of 'mem_cgroup_folio_lruvec' makes pointer from integer without a cast [-Wint-conversion]
    1488 |  return mem_cgroup_folio_lruvec(page_folio(page));
         |                                 ^~~~~~~~~~~~~~~~
         |                                 |
         |                                 int
   include/linux/memcontrol.h:1224:68: note: expected 'struct folio *' but argument is of type 'int'
    1224 | static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
         |                                                      ~~~~~~~~~~~~~~^~~~~
   cc1: some warnings being treated as errors
   make[2]: *** [scripts/Makefile.build:117: arch/arm64/kernel/asm-offsets.s] Error 1
   make[2]: Target '__build' not remade because of errors.
   make[1]: *** [Makefile:1235: prepare0] Error 2
   make[1]: Target 'prepare' not remade because of errors.
   make: *** [Makefile:215: __sub-make] Error 2
   make: Target 'prepare' not remade because of errors.


vim +1226 include/linux/memcontrol.h

  1223	
  1224	static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
  1225	{
> 1226		struct pglist_data *pgdat = folio_pgdat(folio);
  1227		return &pgdat->__lruvec;
  1228	}
  1229	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 41649 bytes --]

Re: [PATCH v3 15/18] mm/memcg: Add mem_cgroup_folio_lruvec()

[Matthew Wilcox]

On Wed, Jun 30, 2021 at 05:00:31AM +0100, Matthew Wilcox (Oracle) wrote:
> This is the folio equivalent of mem_cgroup_page_lruvec().

I'm just going through and removing the wrappers.

Why is this function called this?  There's an odd mix of

lock_page_memcg()
page_memcg()
count_memcg_page_event()
split_page_memcg()

vs

mem_cgroup_charge()
mem_cgroup_swapin_charge_page()
mem_cgroup_lruvec()
mem_cgroup_from_task()

I'd really like to call this function folio_lruvec().  It happens to
behave differently if the folio is part of a memcg, but conceptually,
lruvecs aren't particularly tied to memcgs.


... are there any other functions you'd like to rename while I'm
changing pages to folios?

Re: [PATCH v3 15/18] mm/memcg: Add mem_cgroup_folio_lruvec()

[Johannes Weiner]

On Wed, Jun 30, 2021 at 08:18:33PM +0100, Matthew Wilcox wrote:
> On Wed, Jun 30, 2021 at 05:00:31AM +0100, Matthew Wilcox (Oracle) wrote:
> > This is the folio equivalent of mem_cgroup_page_lruvec().
> 
> I'm just going through and removing the wrappers.
> 
> Why is this function called this?  There's an odd mix of
> 
> lock_page_memcg()

This was chosen to match lock_page().

> page_memcg()

And this to match page_mapping(), page_zone() etc.

> count_memcg_page_event()

count_vm_event()

> split_page_memcg()

split_page(), split_page_owner()

> mem_cgroup_charge()
> mem_cgroup_swapin_charge_page()

These are larger, heavier subsystem API calls that modify all kinds of
state, not just the page. Hence the namespacing.

With the smaller getter/setter type functions on pages we have
traditionally used <verb>_<object> rather than page_<verb>, simply
because the page is such a low-level object and many functions do
sequences of page manipulations. Namespacing would turn them into:

	page_do_this(page);
	page_set_that(page);
	page_lock(page);
	if (page_is_blah(page))
		page_mark_foo(page);
	page_unlock(page);
	page_put(page);

which is hard on the reader because it obscures the salient part of
each line behind repetetive boiler plate.

> mem_cgroup_lruvec()

This is arguably not a namespace prefix, but rather an accessor
function to look up the memcg's lruvec.

> mem_cgroup_from_task()

This is a pattern to look up memcgs from various objects:

- mem_cgroup_from_css()
- mem_cgroup_from_counter()
- mem_cgroup_from_id()
- mem_cgroup_from_seq()
- mem_cgroup_from_obj()

and we used to have mem_cgroup_from_page() at some point...

> I'd really like to call this function folio_lruvec().

That would be a better name indeed.

However, pairing renames with conversion is worrisome because it means
having two unintuitively diverging names for the same operation in the
API during a time where everybody has to relearn the code base already.

Please either precede it with a rename to page_lruvec(), or keep the
existing naming pattern in the conversion.

> It happens to behave differently if the folio is part of a memcg,
> but conceptually, lruvecs aren't particularly tied to memcgs.

Conceptually, lruvecs are always tied to a memcg. On !CONFIG_MEMCG
kernels that just happens to be the root cgroup.

But because it's an accessor to a page attribute, dropping the
namespacing prefix is in line with things like page_memcg().

[PATCH v3 16/18] mm/memcg: Add folio_lruvec_lock() and similar functions

[Matthew Wilcox (Oracle)]

These are the folio equivalents of lock_page_lruvec() and
similar functions.  Retain lock_page_lruvec() as wrappers so we
don't have to convert all these functions twice.  Also convert
lruvec_memcg_debug() to take a folio.

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 include/linux/memcontrol.h | 39 +++++++++++++++++++++++++++-----------
 mm/compaction.c            |  2 +-
 mm/memcontrol.c            | 33 ++++++++++++++------------------
 3 files changed, 43 insertions(+), 31 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index a7e1ccbc7ed6..b21a77669277 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -769,15 +769,16 @@ struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
 
 struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm);
 
-struct lruvec *lock_page_lruvec(struct page *page);
-struct lruvec *lock_page_lruvec_irq(struct page *page);
-struct lruvec *lock_page_lruvec_irqsave(struct page *page,
+struct lruvec *folio_lruvec_lock(struct folio *folio);
+struct lruvec *folio_lruvec_lock_irq(struct folio *folio);
+struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
 						unsigned long *flags);
 
 #ifdef CONFIG_DEBUG_VM
-void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page);
+void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio);
 #else
-static inline void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
+static inline
+void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio)
 {
 }
 #endif
@@ -1257,26 +1258,26 @@ static inline void mem_cgroup_put(struct mem_cgroup *memcg)
 {
 }
 
-static inline struct lruvec *lock_page_lruvec(struct page *page)
+static inline struct lruvec *folio_lruvec_lock(struct folio *folio)
 {
-	struct pglist_data *pgdat = page_pgdat(page);
+	struct pglist_data *pgdat = folio_pgdat(folio);
 
 	spin_lock(&pgdat->__lruvec.lru_lock);
 	return &pgdat->__lruvec;
 }
 
-static inline struct lruvec *lock_page_lruvec_irq(struct page *page)
+static inline struct lruvec *folio_lruvec_lock_irq(struct folio *folio)
 {
-	struct pglist_data *pgdat = page_pgdat(page);
+	struct pglist_data *pgdat = folio_pgdat(folio);
 
 	spin_lock_irq(&pgdat->__lruvec.lru_lock);
 	return &pgdat->__lruvec;
 }
 
-static inline struct lruvec *lock_page_lruvec_irqsave(struct page *page,
+static inline struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
 		unsigned long *flagsp)
 {
-	struct pglist_data *pgdat = page_pgdat(page);
+	struct pglist_data *pgdat = folio_pgdat(folio);
 
 	spin_lock_irqsave(&pgdat->__lruvec.lru_lock, *flagsp);
 	return &pgdat->__lruvec;
@@ -1488,6 +1489,22 @@ static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page)
 	return mem_cgroup_folio_lruvec(page_folio(page));
 }
 
+static inline struct lruvec *lock_page_lruvec(struct page *page)
+{
+	return folio_lruvec_lock(page_folio(page));
+}
+
+static inline struct lruvec *lock_page_lruvec_irq(struct page *page)
+{
+	return folio_lruvec_lock_irq(page_folio(page));
+}
+
+static inline struct lruvec *lock_page_lruvec_irqsave(struct page *page,
+						unsigned long *flags)
+{
+	return folio_lruvec_lock_irqsave(page_folio(page), flags);
+}
+
 static inline void __inc_lruvec_kmem_state(void *p, enum node_stat_item idx)
 {
 	__mod_lruvec_kmem_state(p, idx, 1);
diff --git a/mm/compaction.c b/mm/compaction.c
index 3a509fbf2bea..8b0da04b70f2 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -1038,7 +1038,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 			compact_lock_irqsave(&lruvec->lru_lock, &flags, cc);
 			locked = lruvec;
 
-			lruvec_memcg_debug(lruvec, page);
+			lruvec_memcg_debug(lruvec, page_folio(page));
 
 			/* Try get exclusive access under lock */
 			if (!skip_updated) {
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 95795b65ae3e..23b166917def 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1153,19 +1153,19 @@ int mem_cgroup_scan_tasks(struct mem_cgroup *memcg,
 }
 
 #ifdef CONFIG_DEBUG_VM
-void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
+void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio)
 {
 	struct mem_cgroup *memcg;
 
 	if (mem_cgroup_disabled())
 		return;
 
-	memcg = page_memcg(page);
+	memcg = folio_memcg(folio);
 
 	if (!memcg)
-		VM_BUG_ON_PAGE(lruvec_memcg(lruvec) != root_mem_cgroup, page);
+		VM_BUG_ON_FOLIO(lruvec_memcg(lruvec) != root_mem_cgroup, folio);
 	else
-		VM_BUG_ON_PAGE(lruvec_memcg(lruvec) != memcg, page);
+		VM_BUG_ON_FOLIO(lruvec_memcg(lruvec) != memcg, folio);
 }
 #endif
 
@@ -1179,38 +1179,33 @@ void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
  * - lock_page_memcg()
  * - page->_refcount is zero
  */
-struct lruvec *lock_page_lruvec(struct page *page)
+struct lruvec *folio_lruvec_lock(struct folio *folio)
 {
-	struct lruvec *lruvec;
+	struct lruvec *lruvec = mem_cgroup_folio_lruvec(folio);
 
-	lruvec = mem_cgroup_page_lruvec(page);
 	spin_lock(&lruvec->lru_lock);
-
-	lruvec_memcg_debug(lruvec, page);
+	lruvec_memcg_debug(lruvec, folio);
 
 	return lruvec;
 }
 
-struct lruvec *lock_page_lruvec_irq(struct page *page)
+struct lruvec *folio_lruvec_lock_irq(struct folio *folio)
 {
-	struct lruvec *lruvec;
+	struct lruvec *lruvec = mem_cgroup_folio_lruvec(folio);
 
-	lruvec = mem_cgroup_page_lruvec(page);
 	spin_lock_irq(&lruvec->lru_lock);
-
-	lruvec_memcg_debug(lruvec, page);
+	lruvec_memcg_debug(lruvec, folio);
 
 	return lruvec;
 }
 
-struct lruvec *lock_page_lruvec_irqsave(struct page *page, unsigned long *flags)
+struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
+		unsigned long *flags)
 {
-	struct lruvec *lruvec;
+	struct lruvec *lruvec = mem_cgroup_folio_lruvec(folio);
 
-	lruvec = mem_cgroup_page_lruvec(page);
 	spin_lock_irqsave(&lruvec->lru_lock, *flags);
-
-	lruvec_memcg_debug(lruvec, page);
+	lruvec_memcg_debug(lruvec, folio);
 
 	return lruvec;
 }
-- 
2.30.2

Re: [PATCH v3 16/18] mm/memcg: Add folio_lruvec_lock() and similar functions

[Michal Hocko]

On Wed 30-06-21 05:00:32, Matthew Wilcox wrote:
> These are the folio equivalents of lock_page_lruvec() and
> similar functions.  Retain lock_page_lruvec() as wrappers so we
> don't have to convert all these functions twice.  Also convert
> lruvec_memcg_debug() to take a folio.

Here again I would rather not go with wrappers. I can see how changing
all the callers (20+) is annoying but they all should be pretty trivial
so if we can avoid that I would go with a direct folio_lruvec_{un}lock

> Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
> ---
>  include/linux/memcontrol.h | 39 +++++++++++++++++++++++++++-----------
>  mm/compaction.c            |  2 +-
>  mm/memcontrol.c            | 33 ++++++++++++++------------------
>  3 files changed, 43 insertions(+), 31 deletions(-)
> 
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index a7e1ccbc7ed6..b21a77669277 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -769,15 +769,16 @@ struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
>  
>  struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm);
>  
> -struct lruvec *lock_page_lruvec(struct page *page);
> -struct lruvec *lock_page_lruvec_irq(struct page *page);
> -struct lruvec *lock_page_lruvec_irqsave(struct page *page,
> +struct lruvec *folio_lruvec_lock(struct folio *folio);
> +struct lruvec *folio_lruvec_lock_irq(struct folio *folio);
> +struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
>  						unsigned long *flags);
>  
>  #ifdef CONFIG_DEBUG_VM
> -void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page);
> +void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio);
>  #else
> -static inline void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
> +static inline
> +void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio)
>  {
>  }
>  #endif
> @@ -1257,26 +1258,26 @@ static inline void mem_cgroup_put(struct mem_cgroup *memcg)
>  {
>  }
>  
> -static inline struct lruvec *lock_page_lruvec(struct page *page)
> +static inline struct lruvec *folio_lruvec_lock(struct folio *folio)
>  {
> -	struct pglist_data *pgdat = page_pgdat(page);
> +	struct pglist_data *pgdat = folio_pgdat(folio);
>  
>  	spin_lock(&pgdat->__lruvec.lru_lock);
>  	return &pgdat->__lruvec;
>  }
>  
> -static inline struct lruvec *lock_page_lruvec_irq(struct page *page)
> +static inline struct lruvec *folio_lruvec_lock_irq(struct folio *folio)
>  {
> -	struct pglist_data *pgdat = page_pgdat(page);
> +	struct pglist_data *pgdat = folio_pgdat(folio);
>  
>  	spin_lock_irq(&pgdat->__lruvec.lru_lock);
>  	return &pgdat->__lruvec;
>  }
>  
> -static inline struct lruvec *lock_page_lruvec_irqsave(struct page *page,
> +static inline struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
>  		unsigned long *flagsp)
>  {
> -	struct pglist_data *pgdat = page_pgdat(page);
> +	struct pglist_data *pgdat = folio_pgdat(folio);
>  
>  	spin_lock_irqsave(&pgdat->__lruvec.lru_lock, *flagsp);
>  	return &pgdat->__lruvec;
> @@ -1488,6 +1489,22 @@ static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page)
>  	return mem_cgroup_folio_lruvec(page_folio(page));
>  }
>  
> +static inline struct lruvec *lock_page_lruvec(struct page *page)
> +{
> +	return folio_lruvec_lock(page_folio(page));
> +}
> +
> +static inline struct lruvec *lock_page_lruvec_irq(struct page *page)
> +{
> +	return folio_lruvec_lock_irq(page_folio(page));
> +}
> +
> +static inline struct lruvec *lock_page_lruvec_irqsave(struct page *page,
> +						unsigned long *flags)
> +{
> +	return folio_lruvec_lock_irqsave(page_folio(page), flags);
> +}
> +
>  static inline void __inc_lruvec_kmem_state(void *p, enum node_stat_item idx)
>  {
>  	__mod_lruvec_kmem_state(p, idx, 1);
> diff --git a/mm/compaction.c b/mm/compaction.c
> index 3a509fbf2bea..8b0da04b70f2 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -1038,7 +1038,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
>  			compact_lock_irqsave(&lruvec->lru_lock, &flags, cc);
>  			locked = lruvec;
>  
> -			lruvec_memcg_debug(lruvec, page);
> +			lruvec_memcg_debug(lruvec, page_folio(page));
>  
>  			/* Try get exclusive access under lock */
>  			if (!skip_updated) {
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 95795b65ae3e..23b166917def 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -1153,19 +1153,19 @@ int mem_cgroup_scan_tasks(struct mem_cgroup *memcg,
>  }
>  
>  #ifdef CONFIG_DEBUG_VM
> -void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
> +void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio)
>  {
>  	struct mem_cgroup *memcg;
>  
>  	if (mem_cgroup_disabled())
>  		return;
>  
> -	memcg = page_memcg(page);
> +	memcg = folio_memcg(folio);
>  
>  	if (!memcg)
> -		VM_BUG_ON_PAGE(lruvec_memcg(lruvec) != root_mem_cgroup, page);
> +		VM_BUG_ON_FOLIO(lruvec_memcg(lruvec) != root_mem_cgroup, folio);
>  	else
> -		VM_BUG_ON_PAGE(lruvec_memcg(lruvec) != memcg, page);
> +		VM_BUG_ON_FOLIO(lruvec_memcg(lruvec) != memcg, folio);
>  }
>  #endif
>  
> @@ -1179,38 +1179,33 @@ void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
>   * - lock_page_memcg()
>   * - page->_refcount is zero
>   */
> -struct lruvec *lock_page_lruvec(struct page *page)
> +struct lruvec *folio_lruvec_lock(struct folio *folio)
>  {
> -	struct lruvec *lruvec;
> +	struct lruvec *lruvec = mem_cgroup_folio_lruvec(folio);
>  
> -	lruvec = mem_cgroup_page_lruvec(page);
>  	spin_lock(&lruvec->lru_lock);
> -
> -	lruvec_memcg_debug(lruvec, page);
> +	lruvec_memcg_debug(lruvec, folio);
>  
>  	return lruvec;
>  }
>  
> -struct lruvec *lock_page_lruvec_irq(struct page *page)
> +struct lruvec *folio_lruvec_lock_irq(struct folio *folio)
>  {
> -	struct lruvec *lruvec;
> +	struct lruvec *lruvec = mem_cgroup_folio_lruvec(folio);
>  
> -	lruvec = mem_cgroup_page_lruvec(page);
>  	spin_lock_irq(&lruvec->lru_lock);
> -
> -	lruvec_memcg_debug(lruvec, page);
> +	lruvec_memcg_debug(lruvec, folio);
>  
>  	return lruvec;
>  }
>  
> -struct lruvec *lock_page_lruvec_irqsave(struct page *page, unsigned long *flags)
> +struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
> +		unsigned long *flags)
>  {
> -	struct lruvec *lruvec;
> +	struct lruvec *lruvec = mem_cgroup_folio_lruvec(folio);
>  
> -	lruvec = mem_cgroup_page_lruvec(page);
>  	spin_lock_irqsave(&lruvec->lru_lock, *flags);
> -
> -	lruvec_memcg_debug(lruvec, page);
> +	lruvec_memcg_debug(lruvec, folio);
>  
>  	return lruvec;
>  }
> -- 
> 2.30.2

-- 
Michal Hocko
SUSE Labs

[PATCH v3 17/18] mm/memcg: Add folio_lruvec_relock_irq() and folio_lruvec_relock_irqsave()

[Matthew Wilcox (Oracle)]

These are the folio equivalents of relock_page_lruvec_irq() and
folio_lruvec_relock_irqsave(), which are retained as compatibility wrappers.
Also convert lruvec_holds_page_lru_lock() to folio_lruvec_holds_lru_lock().

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
---
 include/linux/memcontrol.h | 31 ++++++++++++++++++++++---------
 mm/vmscan.c                |  2 +-
 2 files changed, 23 insertions(+), 10 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index b21a77669277..e6b5e8fbf770 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -1545,40 +1545,53 @@ static inline void unlock_page_lruvec_irqrestore(struct lruvec *lruvec,
 }
 
 /* Test requires a stable page->memcg binding, see page_memcg() */
-static inline bool page_matches_lruvec(struct page *page, struct lruvec *lruvec)
+static inline bool folio_matches_lruvec(struct folio *folio,
+		struct lruvec *lruvec)
 {
-	return lruvec_pgdat(lruvec) == page_pgdat(page) &&
-	       lruvec_memcg(lruvec) == page_memcg(page);
+	return lruvec_pgdat(lruvec) == folio_pgdat(folio) &&
+	       lruvec_memcg(lruvec) == folio_memcg(folio);
 }
 
 /* Don't lock again iff page's lruvec locked */
-static inline struct lruvec *relock_page_lruvec_irq(struct page *page,
+static inline struct lruvec *folio_lruvec_relock_irq(struct folio *folio,
 		struct lruvec *locked_lruvec)
 {
 	if (locked_lruvec) {
-		if (page_matches_lruvec(page, locked_lruvec))
+		if (folio_matches_lruvec(folio, locked_lruvec))
 			return locked_lruvec;
 
 		unlock_page_lruvec_irq(locked_lruvec);
 	}
 
-	return lock_page_lruvec_irq(page);
+	return folio_lruvec_lock_irq(folio);
 }
 
 /* Don't lock again iff page's lruvec locked */
-static inline struct lruvec *relock_page_lruvec_irqsave(struct page *page,
+static inline struct lruvec *folio_lruvec_relock_irqsave(struct folio *folio,
 		struct lruvec *locked_lruvec, unsigned long *flags)
 {
 	if (locked_lruvec) {
-		if (page_matches_lruvec(page, locked_lruvec))
+		if (folio_matches_lruvec(folio, locked_lruvec))
 			return locked_lruvec;
 
 		unlock_page_lruvec_irqrestore(locked_lruvec, *flags);
 	}
 
-	return lock_page_lruvec_irqsave(page, flags);
+	return folio_lruvec_lock_irqsave(folio, flags);
 }
 
+static inline struct lruvec *relock_page_lruvec_irq(struct page *page,
+		struct lruvec *locked_lruvec)
+{
+	return folio_lruvec_relock_irq(page_folio(page), locked_lruvec);
+}
+
+static inline struct lruvec *relock_page_lruvec_irqsave(struct page *page,
+		struct lruvec *locked_lruvec, unsigned long *flags)
+{
+	return folio_lruvec_relock_irqsave(page_folio(page), locked_lruvec,
+			flags);
+}
 #ifdef CONFIG_CGROUP_WRITEBACK
 
 struct wb_domain *mem_cgroup_wb_domain(struct bdi_writeback *wb);
diff --git a/mm/vmscan.c b/mm/vmscan.c
index d7c3cb8688dd..a8d8f4673451 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2063,7 +2063,7 @@ static unsigned int move_pages_to_lru(struct lruvec *lruvec,
 		 * All pages were isolated from the same lruvec (and isolation
 		 * inhibits memcg migration).
 		 */
-		VM_BUG_ON_PAGE(!page_matches_lruvec(page, lruvec), page);
+		VM_BUG_ON_PAGE(!folio_matches_lruvec(page_folio(page), lruvec), page);
 		add_page_to_lru_list(page, lruvec);
 		nr_pages = thp_nr_pages(page);
 		nr_moved += nr_pages;
-- 
2.30.2

Re: [PATCH v3 17/18] mm/memcg: Add folio_lruvec_relock_irq() and folio_lruvec_relock_irqsave()

[Michal Hocko]

On Wed 30-06-21 05:00:33, Matthew Wilcox wrote:
> These are the folio equivalents of relock_page_lruvec_irq() and
> folio_lruvec_relock_irqsave(), which are retained as compatibility wrappers.
> Also convert lruvec_holds_page_lru_lock() to folio_lruvec_holds_lru_lock().

$ git grep relock_page_lruvec_irqsave
include/linux/memcontrol.h:static inline struct lruvec *relock_page_lruvec_irqsave(struct page *page,

on your tree.
-- 
Michal Hocko
SUSE Labs

[PATCH v3 18/18] mm/workingset: Convert workingset_activation to take a folio

[Matthew Wilcox (Oracle)]

This function already assumed it was being passed a head page.  No real
change here, except that thp_nr_pages() compiles away on kernels with
THP compiled out while folio_nr_pages() is always present.  Also convert
page_memcg_rcu() to folio_memcg_rcu().

Signed-off-by: Matthew Wilcox (Oracle) <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
Reviewed-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
---
 include/linux/memcontrol.h | 18 +++++++++---------
 include/linux/swap.h       |  2 +-
 mm/swap.c                  |  2 +-
 mm/workingset.c            | 10 +++++-----
 4 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index e6b5e8fbf770..be131c28b3bc 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -462,19 +462,19 @@ static inline struct mem_cgroup *page_memcg(struct page *page)
 }
 
 /*
- * page_memcg_rcu - locklessly get the memory cgroup associated with a page
- * @page: a pointer to the page struct
+ * folio_memcg_rcu - Locklessly get the memory cgroup associated with a folio.
+ * @folio: Pointer to the folio.
  *
- * Returns a pointer to the memory cgroup associated with the page,
- * or NULL. This function assumes that the page is known to have a
+ * Returns a pointer to the memory cgroup associated with the folio,
+ * or NULL. This function assumes that the folio is known to have a
  * proper memory cgroup pointer. It's not safe to call this function
- * against some type of pages, e.g. slab pages or ex-slab pages.
+ * against some type of folios, e.g. slab folios or ex-slab folios.
  */
-static inline struct mem_cgroup *page_memcg_rcu(struct page *page)
+static inline struct mem_cgroup *folio_memcg_rcu(struct folio *folio)
 {
-	unsigned long memcg_data = READ_ONCE(page->memcg_data);
+	unsigned long memcg_data = READ_ONCE(folio->memcg_data);
 
-	VM_BUG_ON_PAGE(PageSlab(page), page);
+	VM_BUG_ON_FOLIO(folio_slab(folio), folio);
 	WARN_ON_ONCE(!rcu_read_lock_held());
 
 	if (memcg_data & MEMCG_DATA_KMEM) {
@@ -1125,7 +1125,7 @@ static inline struct mem_cgroup *page_memcg(struct page *page)
 	return NULL;
 }
 
-static inline struct mem_cgroup *page_memcg_rcu(struct page *page)
+static inline struct mem_cgroup *folio_memcg_rcu(struct folio *folio)
 {
 	WARN_ON_ONCE(!rcu_read_lock_held());
 	return NULL;
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 950dd96007ad..614bbef65777 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -325,7 +325,7 @@ static inline swp_entry_t folio_swap_entry(struct folio *folio)
 void workingset_age_nonresident(struct lruvec *lruvec, unsigned long nr_pages);
 void *workingset_eviction(struct page *page, struct mem_cgroup *target_memcg);
 void workingset_refault(struct page *page, void *shadow);
-void workingset_activation(struct page *page);
+void workingset_activation(struct folio *folio);
 
 /* Only track the nodes of mappings with shadow entries */
 void workingset_update_node(struct xa_node *node);
diff --git a/mm/swap.c b/mm/swap.c
index 8ba62a930370..3c817717af0c 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -447,7 +447,7 @@ void mark_page_accessed(struct page *page)
 		else
 			__lru_cache_activate_page(page);
 		ClearPageReferenced(page);
-		workingset_activation(page);
+		workingset_activation(page_folio(page));
 	}
 	if (page_is_idle(page))
 		clear_page_idle(page);
diff --git a/mm/workingset.c b/mm/workingset.c
index 4f7a306ce75a..86e239ec0306 100644
--- a/mm/workingset.c
+++ b/mm/workingset.c
@@ -390,9 +390,9 @@ void workingset_refault(struct page *page, void *shadow)
 
 /**
  * workingset_activation - note a page activation
- * @page: page that is being activated
+ * @folio: Folio that is being activated.
  */
-void workingset_activation(struct page *page)
+void workingset_activation(struct folio *folio)
 {
 	struct mem_cgroup *memcg;
 	struct lruvec *lruvec;
@@ -405,11 +405,11 @@ void workingset_activation(struct page *page)
 	 * XXX: See workingset_refault() - this should return
 	 * root_mem_cgroup even for !CONFIG_MEMCG.
 	 */
-	memcg = page_memcg_rcu(page);
+	memcg = folio_memcg_rcu(folio);
 	if (!mem_cgroup_disabled() && !memcg)
 		goto out;
-	lruvec = mem_cgroup_page_lruvec(page);
-	workingset_age_nonresident(lruvec, thp_nr_pages(page));
+	lruvec = mem_cgroup_folio_lruvec(folio);
+	workingset_age_nonresident(lruvec, folio_nr_pages(folio));
 out:
 	rcu_read_unlock();
 }
-- 
2.30.2

Re: [PATCH v3 00/18] Folio conversion of memcg

[Michal Hocko]

On Wed 30-06-21 05:00:16, Matthew Wilcox wrote:
> After Michel's comments on "Folio-enabling the page cache", I thought it
> best to split out the memcg patches from the rest of the page cache folio
> patches and redo them to focus on correctness (ie always passing a folio).
> 
> This is fundamentally for review rather than application.  I've rebased
> on Linus' current tree, which includes the recent patchbomb from akpm.
> That upstream version won't boot on any system I have available, and I'm
> not comfortable asking for patches to be applied unless I can actually
> try them.  That said, these patches were fine on top of 5.13.
> 
> There are still a few functions which take pages, but they rely on other
> conversions happening first, which in turn rely on this set of patches,
> so I think this is a good place to stop, with the understanding that
> there will be more patches later.

Thanks for considering the last review feedback. After looking at this
pile the conversion is really straightforward and reasonably easy to
review. There are still some cases where you opted to go with
compatibility wrappers and I would still like to avoid those as it
somehow obfuscates the conversion IMHO. As I've said previously, though,
I do understand that this a tedious work so I do not want to make it
more annoying unnecessarily for you. The replacement with page_folio for
the replaced API looks better to me. If you have strong reasons (e.g.
due to dependencies etc.) please make it explicit in the respective
changelogs.

Thanks!
-- 
Michal Hocko
SUSE Labs