[PATCH 0/2] mm: fix PMD level mTHP accounting bugs

[PATCH 0/2] mm: fix PMD level mTHP accounting bugs

[Nico Pache]

While running selftests I noticed the PMD level per-mTHP stats (nr_anon)
remained elevated after each run. After further investigation I noticed
this accounting error occurs for both the migration.private_anon_htlb_test
and the HMM tests.

In the HMM case this is due to folio_add_new_anon_rmap() incrementing the
mTHP stats, but never containing a corresponding decrement in
free_zone_device_folio(). We solve this by making sure to decrement the
counter when freeing device memory.

In the migration case, we are incrementing this counter without first
checking whether this folio is a hugetlb folio, which relies on a separate
accounting system. We solve this by adding the proper hugetlb check before
incrementing this counter.

With these changes in place, the two tests no longer cause elevated PMD
level accounting issues.

Co-developed-by: David Hildenbrand <david@kernel.org>
Signed-off-by: David Hildenbrand <david@kernel.org>
Signed-off-by: Nico Pache <npache@redhat.com>

Nico Pache (2):
  mm: decrement MTHP_STAT_NR_ANON in free_zone_device_folio()
  mm/migrate: exclude hugetlb folios from MTHP_STAT_NR_ANON accounting

 mm/memremap.c | 1 +
 mm/migrate.c  | 6 ++++--
 2 files changed, 5 insertions(+), 2 deletions(-)

-- 
2.54.0

[PATCH 1/2] mm: decrement MTHP_STAT_NR_ANON in free_zone_device_folio()

[Nico Pache]

When a zone device folio is mapped as anonymous, folio_add_new_anon_rmap()
increments MTHP_STAT_NR_ANON. The corresponding decrement lives in
__free_pages_prepare() in page_alloc.c, but zone device folios are freed
via free_zone_device_folio() which never calls __free_pages_prepare().
This causes nr_anon to remain permanently elevated after zone device
folios are freed.

Add the missing mod_mthp_stat() decrement to free_zone_device_folio()
so that the counter is properly balanced.

Fixes: 5d65c8d758f2 ("mm: count the number of anonymous THPs per size")
Co-developed-by: David Hildenbrand <david@kernel.org>
Signed-off-by: David Hildenbrand <david@kernel.org>
Signed-off-by: Nico Pache <npache@redhat.com>
---
 mm/memremap.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mm/memremap.c b/mm/memremap.c
index 81766d822400..accba23aef28 100644
--- a/mm/memremap.c
+++ b/mm/memremap.c
@@ -425,6 +425,7 @@ void free_zone_device_folio(struct folio *folio)
 	mem_cgroup_uncharge(folio);
 
 	if (folio_test_anon(folio)) {
+		mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, -1);
 		for (i = 0; i < nr; i++)
 			__ClearPageAnonExclusive(folio_page(folio, i));
 	}
-- 
2.54.0

Re: [PATCH 1/2] mm: decrement MTHP_STAT_NR_ANON in free_zone_device_folio()

[Zi Yan]

On Thu Jul 2, 2026 at 1:25 PM EDT, Nico Pache wrote:
> When a zone device folio is mapped as anonymous, folio_add_new_anon_rmap()
> increments MTHP_STAT_NR_ANON. The corresponding decrement lives in
> __free_pages_prepare() in page_alloc.c, but zone device folios are freed
> via free_zone_device_folio() which never calls __free_pages_prepare().
> This causes nr_anon to remain permanently elevated after zone device
> folios are freed.
>
> Add the missing mod_mthp_stat() decrement to free_zone_device_folio()
> so that the counter is properly balanced.
>
> Fixes: 5d65c8d758f2 ("mm: count the number of anonymous THPs per size")
> Co-developed-by: David Hildenbrand <david@kernel.org>
> Signed-off-by: David Hildenbrand <david@kernel.org>
> Signed-off-by: Nico Pache <npache@redhat.com>
> ---
>  mm/memremap.c | 1 +
>  1 file changed, 1 insertion(+)
>
LGTM.

Reviewed-by: Zi Yan <ziy@nvidia.com>


-- 
Best Regards,
Yan, Zi

[PATCH 2/2] mm/migrate: exclude hugetlb folios from MTHP_STAT_NR_ANON accounting

[Nico Pache]

__folio_migrate_mapping() increments MTHP_STAT_NR_ANON for the
destination folio when `folio_test_anon(folio) && folio_test_large(folio)`
is true. However, hugetlb folios satisfy both conditions despite having a
completely separate accounting system; they use hugetlb_add_anon_rmap()
which does not touch mTHP stats, and their free path also bypasses the
mTHP decrement in __free_pages_prepare().

This causes MTHP_STAT_NR_ANON to be incremented on each hugetlb
migration without a corresponding decrement, permanently inflating the
nr_anon counter.

Add a !folio_test_hugetlb() check to both places in
__folio_migrate_mapping() so that only actual mTHP folios are counted.

Fixes: 5d65c8d758f2 ("mm: count the number of anonymous THPs per size")
Co-developed-by: David Hildenbrand <david@kernel.org>
Signed-off-by: David Hildenbrand <david@kernel.org>
Signed-off-by: Nico Pache <npache@redhat.com>
---
 mm/migrate.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/mm/migrate.c b/mm/migrate.c
index d9b23909d716..9fd50ea25d2d 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -590,7 +590,8 @@ static int __folio_migrate_mapping(struct address_space *mapping,
 		/* No turning back from here */
 		newfolio->index = folio->index;
 		newfolio->mapping = folio->mapping;
-		if (folio_test_anon(folio) && folio_test_large(folio))
+		if (folio_test_anon(folio) && folio_test_large(folio) &&
+		    !folio_test_hugetlb(folio))
 			mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, 1);
 		if (folio_test_swapbacked(folio))
 			__folio_set_swapbacked(newfolio);
@@ -623,7 +624,8 @@ static int __folio_migrate_mapping(struct address_space *mapping,
 	 */
 	newfolio->index = folio->index;
 	newfolio->mapping = folio->mapping;
-	if (folio_test_anon(folio) && folio_test_large(folio))
+	if (folio_test_anon(folio) && folio_test_large(folio) &&
+	    !folio_test_hugetlb(folio))
 		mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, 1);
 	folio_ref_add(newfolio, nr); /* add cache reference */
 	if (folio_test_swapbacked(folio))
-- 
2.54.0

Re: [PATCH 2/2] mm/migrate: exclude hugetlb folios from MTHP_STAT_NR_ANON accounting

[Zi Yan]

On Thu Jul 2, 2026 at 1:25 PM EDT, Nico Pache wrote:
> __folio_migrate_mapping() increments MTHP_STAT_NR_ANON for the
> destination folio when `folio_test_anon(folio) && folio_test_large(folio)`
> is true. However, hugetlb folios satisfy both conditions despite having a
> completely separate accounting system; they use hugetlb_add_anon_rmap()
> which does not touch mTHP stats, and their free path also bypasses the
> mTHP decrement in __free_pages_prepare().
>
> This causes MTHP_STAT_NR_ANON to be incremented on each hugetlb
> migration without a corresponding decrement, permanently inflating the
> nr_anon counter.
>
> Add a !folio_test_hugetlb() check to both places in
> __folio_migrate_mapping() so that only actual mTHP folios are counted.
>
> Fixes: 5d65c8d758f2 ("mm: count the number of anonymous THPs per size")
> Co-developed-by: David Hildenbrand <david@kernel.org>
> Signed-off-by: David Hildenbrand <david@kernel.org>
> Signed-off-by: Nico Pache <npache@redhat.com>
> ---
>  mm/migrate.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
LGTM. Thanks.

Reviewed-by: Zi Yan <ziy@nvidia.com>

-- 
Best Regards,
Yan, Zi

Re: [PATCH 2/2] mm/migrate: exclude hugetlb folios from MTHP_STAT_NR_ANON accounting

[Baolin Wang]

On 7/3/26 1:25 AM, Nico Pache wrote:
> __folio_migrate_mapping() increments MTHP_STAT_NR_ANON for the
> destination folio when `folio_test_anon(folio) && folio_test_large(folio)`
> is true. However, hugetlb folios satisfy both conditions despite having a
> completely separate accounting system; they use hugetlb_add_anon_rmap()
> which does not touch mTHP stats, and their free path also bypasses the
> mTHP decrement in __free_pages_prepare().
> 
> This causes MTHP_STAT_NR_ANON to be incremented on each hugetlb
> migration without a corresponding decrement, permanently inflating the
> nr_anon counter.
> 
> Add a !folio_test_hugetlb() check to both places in
> __folio_migrate_mapping() so that only actual mTHP folios are counted.
> 
> Fixes: 5d65c8d758f2 ("mm: count the number of anonymous THPs per size")
> Co-developed-by: David Hildenbrand <david@kernel.org>
> Signed-off-by: David Hildenbrand <david@kernel.org>
> Signed-off-by: Nico Pache <npache@redhat.com>
> ---
>   mm/migrate.c | 6 ++++--
>   1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/migrate.c b/mm/migrate.c
> index d9b23909d716..9fd50ea25d2d 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -590,7 +590,8 @@ static int __folio_migrate_mapping(struct address_space *mapping,
>   		/* No turning back from here */
>   		newfolio->index = folio->index;
>   		newfolio->mapping = folio->mapping;
> -		if (folio_test_anon(folio) && folio_test_large(folio))
> +		if (folio_test_anon(folio) && folio_test_large(folio) &&
> +		    !folio_test_hugetlb(folio))
>   			mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, 1);

Yes, anonymous hugetlb mappings can reach here, so this check looks 
reasonable to me.

>   		if (folio_test_swapbacked(folio))
>   			__folio_set_swapbacked(newfolio);
> @@ -623,7 +624,8 @@ static int __folio_migrate_mapping(struct address_space *mapping,
>   	 */
>   	newfolio->index = folio->index;
>   	newfolio->mapping = folio->mapping;
> -	if (folio_test_anon(folio) && folio_test_large(folio))
> +	if (folio_test_anon(folio) && folio_test_large(folio) &&
> +	    !folio_test_hugetlb(folio))

If the hugetlb folio has a non-NULL mapping (i.e., it's a shared 
mapping), we should migrate it via 
hugetlbfs_migrate_folio()->migrate_huge_page_move_mapping(). In other 
words, shared hugetlb mappings should not reach this path, so this 
hugetlb check can be dropped. Or am I missing something?

>   		mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, 1);
>   	folio_ref_add(newfolio, nr); /* add cache reference */
>   	if (folio_test_swapbacked(folio))

Re: [PATCH 2/2] mm/migrate: exclude hugetlb folios from MTHP_STAT_NR_ANON accounting

[David Hildenbrand (Arm)]

>>       newfolio->index = folio->index;
>>       newfolio->mapping = folio->mapping;
>> -    if (folio_test_anon(folio) && folio_test_large(folio))
>> +    if (folio_test_anon(folio) && folio_test_large(folio) &&
>> +        !folio_test_hugetlb(folio))
> 
> If the hugetlb folio has a non-NULL mapping (i.e., it's a shared mapping), we
> should migrate it via hugetlbfs_migrate_folio()-
>>migrate_huge_page_move_mapping(). In other words, shared hugetlb mappings
> should not reach this path, so this hugetlb check can be dropped. Or am I
> missing something?

Good point: we should only reach here with anonymous folios that are in the
swapcache. That doesn't apply to hugetlb.

-- 
Cheers,

David

Re: [PATCH 2/2] mm/migrate: exclude hugetlb folios from MTHP_STAT_NR_ANON accounting

[Richard Cheng]

On Fri, Jul 03, 2026 at 09:13:59AM +0800, David Hildenbrand (Arm) wrote:
> 
> >>       newfolio->index = folio->index;
> >>       newfolio->mapping = folio->mapping;
> >> -    if (folio_test_anon(folio) && folio_test_large(folio))
> >> +    if (folio_test_anon(folio) && folio_test_large(folio) &&
> >> +        !folio_test_hugetlb(folio))
> > 
> > If the hugetlb folio has a non-NULL mapping (i.e., it's a shared mapping), we
> > should migrate it via hugetlbfs_migrate_folio()-
> >>migrate_huge_page_move_mapping(). In other words, shared hugetlb mappings
> > should not reach this path, so this hugetlb check can be dropped. Or am I
> > missing something?
> 
> Good point: we should only reach here with anonymous folios that are in the
> swapcache. That doesn't apply to hugetlb.
> 
> -- 
> Cheers,
> 
> David
>

I think this is right. THe second site requires folio_test_anon() with a
non-NULL mapping, i. e. an anon folio in the swap cache, and hugetlb folios
never enter the swap cache. Shared hugetlb goes through migrate_huge_page_move_mapping()
as you say, and the only other callers of folio_migrate_mapping() never pass
hugetlb  folios either. So the check in the second hunk can't fire.

--Richard 

Re: [PATCH 2/2] mm/migrate: exclude hugetlb folios from MTHP_STAT_NR_ANON accounting

[Nico Pache]

On Thu, Jul 2, 2026 at 9:10 PM Baolin Wang
<baolin.wang@linux.alibaba.com> wrote:
>
>
>
> On 7/3/26 1:25 AM, Nico Pache wrote:
> > __folio_migrate_mapping() increments MTHP_STAT_NR_ANON for the
> > destination folio when `folio_test_anon(folio) && folio_test_large(folio)`
> > is true. However, hugetlb folios satisfy both conditions despite having a
> > completely separate accounting system; they use hugetlb_add_anon_rmap()
> > which does not touch mTHP stats, and their free path also bypasses the
> > mTHP decrement in __free_pages_prepare().
> >
> > This causes MTHP_STAT_NR_ANON to be incremented on each hugetlb
> > migration without a corresponding decrement, permanently inflating the
> > nr_anon counter.
> >
> > Add a !folio_test_hugetlb() check to both places in
> > __folio_migrate_mapping() so that only actual mTHP folios are counted.
> >
> > Fixes: 5d65c8d758f2 ("mm: count the number of anonymous THPs per size")
> > Co-developed-by: David Hildenbrand <david@kernel.org>
> > Signed-off-by: David Hildenbrand <david@kernel.org>
> > Signed-off-by: Nico Pache <npache@redhat.com>
> > ---
> >   mm/migrate.c | 6 ++++--
> >   1 file changed, 4 insertions(+), 2 deletions(-)
> >
> > diff --git a/mm/migrate.c b/mm/migrate.c
> > index d9b23909d716..9fd50ea25d2d 100644
> > --- a/mm/migrate.c
> > +++ b/mm/migrate.c
> > @@ -590,7 +590,8 @@ static int __folio_migrate_mapping(struct address_space *mapping,
> >               /* No turning back from here */
> >               newfolio->index = folio->index;
> >               newfolio->mapping = folio->mapping;
> > -             if (folio_test_anon(folio) && folio_test_large(folio))
> > +             if (folio_test_anon(folio) && folio_test_large(folio) &&
> > +                 !folio_test_hugetlb(folio))
> >                       mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, 1);
>
> Yes, anonymous hugetlb mappings can reach here, so this check looks
> reasonable to me.
>
> >               if (folio_test_swapbacked(folio))
> >                       __folio_set_swapbacked(newfolio);
> > @@ -623,7 +624,8 @@ static int __folio_migrate_mapping(struct address_space *mapping,
> >        */
> >       newfolio->index = folio->index;
> >       newfolio->mapping = folio->mapping;
> > -     if (folio_test_anon(folio) && folio_test_large(folio))
> > +     if (folio_test_anon(folio) && folio_test_large(folio) &&
> > +         !folio_test_hugetlb(folio))
>
> If the hugetlb folio has a non-NULL mapping (i.e., it's a shared
> mapping), we should migrate it via
> hugetlbfs_migrate_folio()->migrate_huge_page_move_mapping(). In other
> words, shared hugetlb mappings should not reach this path, so this
> hugetlb check can be dropped. Or am I missing something?

I double checked (and others seem to have confirmed), you are correct;
these hugetlb pages shouldn't be reachable here.

Ill drop this hunk and retest :)

Thank you,
-- Nico

>
> >               mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, 1);
> >       folio_ref_add(newfolio, nr); /* add cache reference */
> >       if (folio_test_swapbacked(folio))
>