[PATCH] mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()

[PATCH] mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()

[Konstantin Khorenko]

copy_huge_pud() adds the child's rmap of a device DAX PUD with
folio_add_file_rmap_pud(). That helper maps a folio afresh: it updates
NR_FILE_MAPPED when the folio becomes mapped and mlocks it in a
VM_LOCKED VMA. Neither applies at fork: the parent keeps the folio mapped
under the page table lock, and dup_mmap() clears VM_LOCKED in the
child's VMA.

copy_present_ptes() duplicates the rmap of a PTE with
folio_dup_file_rmap_ptes(), which only raises the mapcounts. Add
folio_dup_file_rmap_pud() for the PUD level and use it in copy_huge_pud().

No functional change.

Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
---
David's fix for the device DAX PUD fork bug, commit d90bdc02e36e
("mm/huge_memory: properly handle refcounted PUD mappings on fork"),
went in before v2 of my series for the same bug [1].

This is the follow-up David asked for [2].

The Reviewed-by tags are from v2 patch 1, whose rmap.h hunk and
copy_huge_pud() line this patch carries unchanged.

The selftest from v2 [3] is not included, as it needs a real devdax
device. It is there for anyone who wants to run it.

Based on thp-7.5.misc of the mm/core tree.

[1] https://lore.kernel.org/all/20261105-vstor-150143-copy_huge_pud-folio-refcount-v2-0-2828fe5a51ed@virtuozzo.com/
[2] https://lore.kernel.org/all/6f035f97-8b55-47ad-bd52-833b86b8ae8f@kernel.org/
[3] https://lore.kernel.org/all/20261105-vstor-150143-copy_huge_pud-folio-refcount-v2-2-2828fe5a51ed@virtuozzo.com/

Assisted-by: LLM
---
 include/linux/rmap.h | 22 ++++++++++++++++++++++
 mm/huge_memory.c     |  2 +-
 2 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/include/linux/rmap.h b/include/linux/rmap.h
index 74cca0e3c7264..52fff8dc74213 100644
--- a/include/linux/rmap.h
+++ b/include/linux/rmap.h
@@ -564,6 +564,28 @@ static inline void folio_dup_file_rmap_pmd(struct folio *folio,
 #endif
 }
 
+/**
+ * folio_dup_file_rmap_pud - duplicate a PUD mapping of a page range of a folio
+ * @folio:	The folio to duplicate the mapping of
+ * @page:	The first page to duplicate the mapping of
+ * @dst_vma:	The destination vm area
+ *
+ * The page range of the folio is defined by [page, page + HPAGE_PUD_NR)
+ *
+ * The caller needs to hold the page table lock.
+ */
+static inline void folio_dup_file_rmap_pud(struct folio *folio,
+		struct page *page, struct vm_area_struct *dst_vma)
+{
+#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && \
+	defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)
+	__folio_dup_file_rmap(folio, page, HPAGE_PUD_NR, dst_vma,
+			      PGTABLE_LEVEL_PUD);
+#else
+	WARN_ON_ONCE(true);
+#endif
+}
+
 static __always_inline int __folio_try_dup_anon_rmap(struct folio *folio,
 		struct page *page, int nr_pages, struct vm_area_struct *dst_vma,
 		struct vm_area_struct *src_vma, enum pgtable_level level)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index d0e19d9e6b651..13caf81cb51af 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2100,7 +2100,7 @@ int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma
 		 * TODO: once we support anonymous pages, use
 		 * folio_try_dup_anon_rmap_*() and split if duplicating fails.
 		 */
-		folio_add_file_rmap_pud(folio, page, dst_vma);
+		folio_dup_file_rmap_pud(folio, page, dst_vma);
 		add_mm_counter(dst_mm, mm_counter_file(folio), HPAGE_PUD_NR);
 	}
 

---
base-commit: 31ea5515bf595046effc74a5bafdcc26320d2d86
change-id: 20261106-khorenko-vstor-151183-copy_huge_pud-dup-rmap-820094c56b90

Best regards,
--  
Konstantin Khorenko,
Virtuozzo Linux Kernel Team

Re: [PATCH] mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()

[David Hildenbrand (Arm)]

On 11/6/26 18:05, Konstantin Khorenko wrote:
> copy_huge_pud() adds the child's rmap of a device DAX PUD with
> folio_add_file_rmap_pud(). That helper maps a folio afresh: it updates
> NR_FILE_MAPPED when the folio becomes mapped and mlocks it in a
> VM_LOCKED VMA. Neither applies at fork: the parent keeps the folio mapped
> under the page table lock, and dup_mmap() clears VM_LOCKED in the
> child's VMA.
> 
> copy_present_ptes() duplicates the rmap of a PTE with
> folio_dup_file_rmap_ptes(), which only raises the mapcounts. Add
> folio_dup_file_rmap_pud() for the PUD level and use it in copy_huge_pud().
> 
> No functional change.
> 
> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
> Reviewed-by: Zi Yan <ziy@nvidia.com>
> Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
> ---
> David's fix for the device DAX PUD fork bug, commit d90bdc02e36e
> ("mm/huge_memory: properly handle refcounted PUD mappings on fork"),
> went in before v2 of my series for the same bug [1].
> 
> This is the follow-up David asked for [2].
> 
> The Reviewed-by tags are from v2 patch 1, whose rmap.h hunk and
> copy_huge_pud() line this patch carries unchanged.
> 
> The selftest from v2 [3] is not included, as it needs a real devdax
> device. It is there for anyone who wants to run it.
> 
> Based on thp-7.5.misc of the mm/core tree.
> 
> [1] https://lore.kernel.org/all/20261105-vstor-150143-copy_huge_pud-folio-refcount-v2-0-2828fe5a51ed@virtuozzo.com/
> [2] https://lore.kernel.org/all/6f035f97-8b55-47ad-bd52-833b86b8ae8f@kernel.org/
> [3] https://lore.kernel.org/all/20261105-vstor-150143-copy_huge_pud-folio-refcount-v2-2-2828fe5a51ed@virtuozzo.com/
> 
> Assisted-by: LLM
> ---

Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>

Thanks, I'll wait a bit and then pick this up.


-- 
Cheers,

David

Re: [PATCH] mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found no new issues
in this patch, but noticed 1 related pre-existing issue it does not
introduce:
- [Medium] mm: hardware A/D bits discarded for PUD folios in
  zap_huge_pud() [1]
--

commit a6bb79712598f2da7fdcc94c9b1b5f8b9aad8851
Author: Konstantin Khorenko <khorenko@virtuozzo.com>

mm/huge_memory: use folio_dup_file_rmap_pud() in copy_huge_pud()

This commit adds folio_dup_file_rmap_pud() to duplicate file rmap mapcounts
at the PUD level and uses it in copy_huge_pud() instead of
folio_add_file_rmap_pud().

> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index d0e19d9e6b651..13caf81cb51af 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2100,7 +2100,7 @@ int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma
>  		 * TODO: once we support anonymous pages, use
>  		 * folio_try_dup_anon_rmap_*() and split if duplicating fails.
>  		 */
> -		folio_add_file_rmap_pud(folio, page, dst_vma);
> +		folio_dup_file_rmap_pud(folio, page, dst_vma);
>  		add_mm_counter(dst_mm, mm_counter_file(folio), HPAGE_PUD_NR);
>  	}

[Severity: Medium]
This problem wasn't introduced by this patch, but does zap_huge_pud()
discard hardware accessed and dirty bits on the PUD folio [1]?

mm/huge_memory.c:zap_huge_pud() {
	...
	orig_pud = pudp_huge_get_and_clear_full(vma, addr, pud, tlb->fullmm);
	...
	} else {
		...
		page = pud_page(orig_pud);
		folio = page_folio(page);
		folio_remove_rmap_pud(folio, page, vma);
		add_mm_counter(tlb->mm, mm_counter_file(folio), -HPAGE_PUD_NR);
	...
}

Unlike zap_huge_pmd(), which checks the cleared entry and calls
folio_mark_accessed() and folio_mark_dirty(), zap_huge_pud() removes the
PUD rmap without propagating the young and dirty bits from orig_pud to the
folio.  When unmapping a refcounted Device DAX PUD mapping, those hardware
A/D bits are lost, leaving the folio with stale access and dirty state.

[1] https://sashiko.dev/#/bug/linux-d2fca843-ce3d-48bf-9ef3-bf87872ea6ec

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106-khorenko-vstor-151183-copy_huge_pud-dup-rmap-v1-1-cf8ee4fafabb@virtuozzo.com?part=1