[PATCH] mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()

[PATCH] mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()

[Konstantin Khorenko]

copy_huge_pud() adds the child's rmap of a device DAX PUD with
folio_add_file_rmap_pud(). That helper maps a folio afresh: it updates
NR_FILE_MAPPED when the folio becomes mapped and mlocks it in a
VM_LOCKED VMA. Neither applies at fork: the parent keeps the folio mapped
under the page table lock, and dup_mmap() clears VM_LOCKED in the
child's VMA.

copy_present_ptes() duplicates the rmap of a PTE with
folio_dup_file_rmap_ptes(), which only raises the mapcounts. Add
folio_dup_file_rmap_pud() for the PUD level and use it in copy_huge_pud().

No functional change.

Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
---
David's fix for the device DAX PUD fork bug, commit d90bdc02e36e
("mm/huge_memory: properly handle refcounted PUD mappings on fork"),
went in before v2 of my series for the same bug [1].

This is the follow-up David asked for [2].

The Reviewed-by tags are from v2 patch 1, whose rmap.h hunk and
copy_huge_pud() line this patch carries unchanged.

The selftest from v2 [3] is not included, as it needs a real devdax
device. It is there for anyone who wants to run it.

Based on thp-7.5.misc of the mm/core tree.

[1] https://lore.kernel.org/all/20261105-vstor-150143-copy_huge_pud-folio-refcount-v2-0-2828fe5a51ed@virtuozzo.com/
[2] https://lore.kernel.org/all/6f035f97-8b55-47ad-bd52-833b86b8ae8f@kernel.org/
[3] https://lore.kernel.org/all/20261105-vstor-150143-copy_huge_pud-folio-refcount-v2-2-2828fe5a51ed@virtuozzo.com/

Assisted-by: LLM
---
 include/linux/rmap.h | 22 ++++++++++++++++++++++
 mm/huge_memory.c     |  2 +-
 2 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/include/linux/rmap.h b/include/linux/rmap.h
index 74cca0e3c7264..52fff8dc74213 100644
--- a/include/linux/rmap.h
+++ b/include/linux/rmap.h
@@ -564,6 +564,28 @@ static inline void folio_dup_file_rmap_pmd(struct folio *folio,
 #endif
 }
 
+/**
+ * folio_dup_file_rmap_pud - duplicate a PUD mapping of a page range of a folio
+ * @folio:	The folio to duplicate the mapping of
+ * @page:	The first page to duplicate the mapping of
+ * @dst_vma:	The destination vm area
+ *
+ * The page range of the folio is defined by [page, page + HPAGE_PUD_NR)
+ *
+ * The caller needs to hold the page table lock.
+ */
+static inline void folio_dup_file_rmap_pud(struct folio *folio,
+		struct page *page, struct vm_area_struct *dst_vma)
+{
+#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && \
+	defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)
+	__folio_dup_file_rmap(folio, page, HPAGE_PUD_NR, dst_vma,
+			      PGTABLE_LEVEL_PUD);
+#else
+	WARN_ON_ONCE(true);
+#endif
+}
+
 static __always_inline int __folio_try_dup_anon_rmap(struct folio *folio,
 		struct page *page, int nr_pages, struct vm_area_struct *dst_vma,
 		struct vm_area_struct *src_vma, enum pgtable_level level)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index d0e19d9e6b651..13caf81cb51af 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2100,7 +2100,7 @@ int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma
 		 * TODO: once we support anonymous pages, use
 		 * folio_try_dup_anon_rmap_*() and split if duplicating fails.
 		 */
-		folio_add_file_rmap_pud(folio, page, dst_vma);
+		folio_dup_file_rmap_pud(folio, page, dst_vma);
 		add_mm_counter(dst_mm, mm_counter_file(folio), HPAGE_PUD_NR);
 	}
 

---
base-commit: 31ea5515bf595046effc74a5bafdcc26320d2d86
change-id: 20261106-khorenko-vstor-151183-copy_huge_pud-dup-rmap-820094c56b90

Best regards,
--  
Konstantin Khorenko,
Virtuozzo Linux Kernel Team

Re: [PATCH] mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()

[David Hildenbrand (Arm)]

On 11/6/26 18:05, Konstantin Khorenko wrote:
> copy_huge_pud() adds the child's rmap of a device DAX PUD with
> folio_add_file_rmap_pud(). That helper maps a folio afresh: it updates
> NR_FILE_MAPPED when the folio becomes mapped and mlocks it in a
> VM_LOCKED VMA. Neither applies at fork: the parent keeps the folio mapped
> under the page table lock, and dup_mmap() clears VM_LOCKED in the
> child's VMA.
> 
> copy_present_ptes() duplicates the rmap of a PTE with
> folio_dup_file_rmap_ptes(), which only raises the mapcounts. Add
> folio_dup_file_rmap_pud() for the PUD level and use it in copy_huge_pud().
> 
> No functional change.
> 
> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
> Reviewed-by: Zi Yan <ziy@nvidia.com>
> Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
> ---
> David's fix for the device DAX PUD fork bug, commit d90bdc02e36e
> ("mm/huge_memory: properly handle refcounted PUD mappings on fork"),
> went in before v2 of my series for the same bug [1].
> 
> This is the follow-up David asked for [2].
> 
> The Reviewed-by tags are from v2 patch 1, whose rmap.h hunk and
> copy_huge_pud() line this patch carries unchanged.
> 
> The selftest from v2 [3] is not included, as it needs a real devdax
> device. It is there for anyone who wants to run it.
> 
> Based on thp-7.5.misc of the mm/core tree.
> 
> [1] https://lore.kernel.org/all/20261105-vstor-150143-copy_huge_pud-folio-refcount-v2-0-2828fe5a51ed@virtuozzo.com/
> [2] https://lore.kernel.org/all/6f035f97-8b55-47ad-bd52-833b86b8ae8f@kernel.org/
> [3] https://lore.kernel.org/all/20261105-vstor-150143-copy_huge_pud-folio-refcount-v2-2-2828fe5a51ed@virtuozzo.com/
> 
> Assisted-by: LLM
> ---

Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>

Thanks, I'll wait a bit and then pick this up.


-- 
Cheers,

David

Re: [PATCH] mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found no new issues
in this patch, but noticed 1 related pre-existing issue it does not
introduce:
- [Medium] mm: hardware A/D bits discarded for PUD folios in
  zap_huge_pud() [1]
--

commit a6bb79712598f2da7fdcc94c9b1b5f8b9aad8851
Author: Konstantin Khorenko <khorenko@virtuozzo.com>

mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()

This commit adds folio_dup_file_rmap_pud() to duplicate file rmap mapcounts
at the PUD level and uses it in copy_huge_pud() instead of
folio_add_file_rmap_pud().

> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index d0e19d9e6b651..13caf81cb51af 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2100,7 +2100,7 @@ int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma
>  		 * TODO: once we support anonymous pages, use
>  		 * folio_try_dup_anon_rmap_*() and split if duplicating fails.
>  		 */
> -		folio_add_file_rmap_pud(folio, page, dst_vma);
> +		folio_dup_file_rmap_pud(folio, page, dst_vma);
>  		add_mm_counter(dst_mm, mm_counter_file(folio), HPAGE_PUD_NR);
>  	}

[Severity: Medium]
This problem wasn't introduced by this patch, but does zap_huge_pud()
discard hardware accessed and dirty bits on the PUD folio [1]?

mm/huge_memory.c:zap_huge_pud() {
	...
	orig_pud = pudp_huge_get_and_clear_full(vma, addr, pud, tlb->fullmm);
	...
	} else {
		...
		page = pud_page(orig_pud);
		folio = page_folio(page);
		folio_remove_rmap_pud(folio, page, vma);
		add_mm_counter(tlb->mm, mm_counter_file(folio), -HPAGE_PUD_NR);
	...
}

Unlike zap_huge_pmd(), which checks the cleared entry and calls
folio_mark_accessed() and folio_mark_dirty(), zap_huge_pud() removes the
PUD rmap without propagating the young and dirty bits from orig_pud to the
folio.  When unmapping a refcounted Device DAX PUD mapping, those hardware
A/D bits are lost, leaving the folio with stale access and dirty state.

[1] https://sashiko.dev/#/bug/linux-d2fca843-ce3d-48bf-9ef3-bf87872ea6ec

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-khorenko-vstor-151183-copy_huge_pud-dup-rmap-v1-1-cf8ee4fafabb@virtuozzo.com?part=1

Re: [PATCH] mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()

[David Hildenbrand (Arm)]

On 11/7/26 18:05, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found no new issues
> in this patch, but noticed 1 related pre-existing issue it does not
> introduce:
> - [Medium] mm: hardware A/D bits discarded for PUD folios in
>   zap_huge_pud() [1]
> --
> 
> commit a6bb79712598f2da7fdcc94c9b1b5f8b9aad8851
> Author: Konstantin Khorenko <khorenko@virtuozzo.com>
> 
> mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()
> 
> This commit adds folio_dup_file_rmap_pud() to duplicate file rmap mapcounts
> at the PUD level and uses it in copy_huge_pud() instead of
> folio_add_file_rmap_pud().
> 
>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>> index d0e19d9e6b651..13caf81cb51af 100644
>> --- a/mm/huge_memory.c
>> +++ b/mm/huge_memory.c
>> @@ -2100,7 +2100,7 @@ int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma
>>  		 * TODO: once we support anonymous pages, use
>>  		 * folio_try_dup_anon_rmap_*() and split if duplicating fails.
>>  		 */
>> -		folio_add_file_rmap_pud(folio, page, dst_vma);
>> +		folio_dup_file_rmap_pud(folio, page, dst_vma);
>>  		add_mm_counter(dst_mm, mm_counter_file(folio), HPAGE_PUD_NR);
>>  	}
> 
> [Severity: Medium]
> This problem wasn't introduced by this patch, but does zap_huge_pud()
> discard hardware accessed and dirty bits on the PUD folio [1]?
> 
> mm/huge_memory.c:zap_huge_pud() {
> 	...
> 	orig_pud = pudp_huge_get_and_clear_full(vma, addr, pud, tlb->fullmm);
> 	...
> 	} else {
> 		...
> 		page = pud_page(orig_pud);
> 		folio = page_folio(page);
> 		folio_remove_rmap_pud(folio, page, vma);
> 		add_mm_counter(tlb->mm, mm_counter_file(folio), -HPAGE_PUD_NR);
> 	...
> }
> 
> Unlike zap_huge_pmd(), which checks the cleared entry and calls
> folio_mark_accessed() and folio_mark_dirty(), zap_huge_pud() removes the
> PUD rmap without propagating the young and dirty bits from orig_pud to the
> folio.  When unmapping a refcounted Device DAX PUD mapping, those hardware
> A/D bits are lost, leaving the folio with stale access and dirty state.

Probably not that relevant for fsdax, which is the only one supporting PUD THPs
IIRC.

-- 
Cheers,

David

Re: [PATCH] mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()

[David Hildenbrand]

On Fri, 06 Nov 2026 18:05:20 +0100, Konstantin Khorenko wrote:
> copy_huge_pud() adds the child's rmap of a device DAX PUD with
> folio_add_file_rmap_pud(). That helper maps a folio afresh: it updates
> NR_FILE_MAPPED when the folio becomes mapped and mlocks it in a
> VM_LOCKED VMA. Neither applies at fork: the parent keeps the folio mapped
> under the page table lock, and dup_mmap() clears VM_LOCKED in the
> child's VMA.
> 
> [...]

Applied, thanks!

tree: mm/core.git
branch: thp-7.5.misc

Applied patches will appear soon in mm-next, followed by linux-next.

Please report any outstanding bugs that were missed during review, so we
can either drop the patch series or decide how to fix it up.

It is encouraged to provide Acked-bys, Reviewed-bys, and Tested-bys even
though the patches have already been applied. If possible, patch trailers
will be updated.

Note that commit hashes shown below are subject to change due to rebase,
trailer updates or similar. Further, this is not a guarantee that the
patches will go upstream. If in doubt, please check the listed branch.

[1/1] mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()
      https://git.kernel.org/mm/core/c/f403b8aa

-- 
Cheers,

David