[PATCH] mm/hugetlb_vmemmap: fix PMD split TLB flush race

[PATCH] mm/hugetlb_vmemmap: fix PMD split TLB flush race

[Muchun Song]

Bulk HVO optimization converts vmemmap PMDs for a list of folios into
PTE tables before issuing a single global TLB flush. After publishing
a PTE table, a concurrent single-folio optimization can observe it and
remap its PTEs before the deferred flush:

CPU0                             CPU1
hugetlb_vmemmap_optimize_folios()
  hugetlb_vmemmap_pteify_folio()
    vmemmap_split_pmd()
      pmd_populate_kernel()
      /* TLB flush deferred */
                                 hugetlb_vmemmap_optimize_folio()
                                   vmemmap_pmd_entry()
                                   /* sees a PTE table */
                                   vmemmap_optimize_pte()
                                     set_pte_at()
  flush_tlb_all()

This can create overlapping TLB entries at different granularities
that map the same virtual address to different physical pages.

Serialize deferred PMD splitting against PTE remapping with init_mm's
mmap lock. Hold it for write across the entire pteify batch and the
final TLB flush, while optimize and restore remaps hold it for read.
A remapper can then observe a split PMD only after stale PMD
translations have been invalidated.

Fixes: f4b7e3efaddb ("hugetlb: batch PMD split for bulk vmemmap dedup")
Cc: stable@vger.kernel.org
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/bug/linux-fbdc3410-a4cf-457e-88e6-684380fbadfe
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
---
 mm/hugetlb_vmemmap.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c
index ba8e11e399b2..ba6559bef742 100644
--- a/mm/hugetlb_vmemmap.c
+++ b/mm/hugetlb_vmemmap.c
@@ -158,10 +158,8 @@ static int vmemmap_remap_range(unsigned long start, unsigned long end,
 
 	VM_BUG_ON(!PAGE_ALIGNED(start | end));
 
-	mmap_read_lock(&init_mm);
 	ret = walk_kernel_page_table_range(start, end, &vmemmap_remap_ops,
 				    NULL, walk);
-	mmap_read_unlock(&init_mm);
 	if (ret)
 		return ret;
 
@@ -308,6 +306,9 @@ static inline int vmemmap_restore_range(unsigned long start, unsigned long end,
  * is skipped, the caller must flush TLBs after all pteify operations are complete
  * before remapping the folio's vmemmap pages.
  *
+ * Note: The caller must hold init_mm's mmap write lock for all pteify operations
+ * and the final TLB flush.
+ *
  * Return: %0 on success, negative error code otherwise.
  */
 static int vmemmap_pteify_folio(const struct folio *folio, unsigned long flags)
@@ -318,6 +319,8 @@ static int vmemmap_pteify_folio(const struct folio *folio, unsigned long flags)
 		.flags		= flags,
 	};
 
+	mmap_assert_write_locked(&init_mm);
+
 	return vmemmap_remap_range(start, end, &walk);
 }
 
@@ -358,6 +361,7 @@ static int vmemmap_optimize_folio(const struct folio *folio,
 	if (ret)
 		return ret;
 
+	guard(mmap_read_lock)(&init_mm);
 	ret = vmemmap_remap_range(start, end, &walk);
 	if (!ret || !walk.nr_walked)
 		return ret;
@@ -392,6 +396,8 @@ static int vmemmap_restore_folio(const struct folio *folio, unsigned long flags)
 	if (ret)
 		return ret;
 
+	guard(mmap_read_lock)(&init_mm);
+
 	return vmemmap_restore_range(start, end, &vmemmap_pages, flags);
 }
 
@@ -559,6 +565,7 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l
 	if (list_empty(folio_list))
 		return;
 
+	mmap_write_lock(&init_mm);
 	list_for_each_entry(folio, folio_list, lru) {
 		int ret;
 
@@ -583,11 +590,13 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l
 			break;
 	}
 
+	if (nr_to_optimize)
+		flush_tlb_all();
+	mmap_write_unlock(&init_mm);
+
 	if (!nr_to_optimize)
 		return;
 
-	flush_tlb_all();
-
 	list_for_each_entry(folio, folio_list, lru) {
 		int ret;
 

base-commit: b95f8bb86713c860ce8536d22a988b949b884d5f
-- 
2.54.0

Re: [PATCH] mm/hugetlb_vmemmap: fix PMD split TLB flush race

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [High] mm/hugetlb_vmemmap: deadlock from GFP_KERNEL alloc under
  init_mm.mmap_lock
--

commit c4de2ad9c999ebd1524c99a20d1c19a13ab7ec8f
Author: Muchun Song <muchun.song@linux.dev>

mm/hugetlb_vmemmap: fix PMD split TLB flush race

This commit serializes batched vmemmap PMD splitting and its final TLB flush
against concurrent PTE remapping using init_mm.mmap_lock.

> diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c
> index ba8e11e399b24..ba6559bef742a 100644
> --- a/mm/hugetlb_vmemmap.c
> +++ b/mm/hugetlb_vmemmap.c
[ ... ]
> @@ -559,6 +565,7 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l
>  	if (list_empty(folio_list))
>  		return;
>  
> +	mmap_write_lock(&init_mm);
>  	list_for_each_entry(folio, folio_list, lru) {
>  		int ret;
>  

[Severity: High]
Can holding init_mm.mmap_lock for write across hugetlb_vmemmap_pteify_folio()
deadlock in memory reclaim?

hugetlb_vmemmap_pteify_folio() walks the kernel page tables and calls
vmemmap_split_pmd() -> pte_alloc_one_kernel(&init_mm), which allocates a page
table with GFP_PGTABLE_KERNEL (GFP_KERNEL).  Reclaim shrinkers such as
ttm_pool_shrinker_scan() take mmap_read_lock(&init_mm) when restoring direct
map attributes via set_pages_wb() -> change_page_attr_set_clr():

hugetlb_vmemmap_optimize_folios()
  mmap_write_lock(&init_mm)
  hugetlb_vmemmap_pteify_folio()
    ...
      vmemmap_split_pmd()
        pte_alloc_one_kernel(&init_mm)  /* GFP_KERNEL */
          <direct reclaim>
            ttm_pool_shrinker_scan()
              ...
                change_page_attr_set_clr()
                  mmap_read_lock(&init_mm)  <-- self-deadlock

Because fs_reclaim nests outside init_mm.mmap_lock, this also triggers a
lockdep splat on the first vmemmap PMD split (including under
mmap_read_lock(&init_mm) in vmemmap_optimize_folio()) and can ABBA deadlock
against kswapd.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261110112456.62827-1-songmuchun@bytedance.com?part=1

Re: [PATCH] mm/hugetlb_vmemmap: fix PMD split TLB flush race

[Muchun Song]

> On Nov 10, 2026, at 20:01, sashiko-bot@kernel.org wrote:
> 
> Thank you for your contribution! Sashiko AI review found 1 potential
> issue to consider:
> - [High] mm/hugetlb_vmemmap: deadlock from GFP_KERNEL alloc under
>  init_mm.mmap_lock
> --
> 
> commit c4de2ad9c999ebd1524c99a20d1c19a13ab7ec8f
> Author: Muchun Song <muchun.song@linux.dev>
> 
> mm/hugetlb_vmemmap: fix PMD split TLB flush race
> 
> This commit serializes batched vmemmap PMD splitting and its final TLB flush
> against concurrent PTE remapping using init_mm.mmap_lock.
> 
>> diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c
>> index ba8e11e399b24..ba6559bef742a 100644
>> --- a/mm/hugetlb_vmemmap.c
>> +++ b/mm/hugetlb_vmemmap.c
> [ ... ]
>> @@ -559,6 +565,7 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l
>> 	if (list_empty(folio_list))
>> 		return;
>> 
>> +	mmap_write_lock(&init_mm);
>> 	list_for_each_entry(folio, folio_list, lru) {
>> 		int ret;
>> 
> 
> [Severity: High]
> Can holding init_mm.mmap_lock for write across hugetlb_vmemmap_pteify_folio()
> deadlock in memory reclaim?

A valid point, I'll fix it ASAP.

THanks.

> 
> hugetlb_vmemmap_pteify_folio() walks the kernel page tables and calls
> vmemmap_split_pmd() -> pte_alloc_one_kernel(&init_mm), which allocates a page
> table with GFP_PGTABLE_KERNEL (GFP_KERNEL).  Reclaim shrinkers such as
> ttm_pool_shrinker_scan() take mmap_read_lock(&init_mm) when restoring direct
> map attributes via set_pages_wb() -> change_page_attr_set_clr():
> 
> hugetlb_vmemmap_optimize_folios()
>  mmap_write_lock(&init_mm)
>  hugetlb_vmemmap_pteify_folio()
>    ...
>      vmemmap_split_pmd()
>        pte_alloc_one_kernel(&init_mm)  /* GFP_KERNEL */
>          <direct reclaim>
>            ttm_pool_shrinker_scan()
>              ...
>                change_page_attr_set_clr()
>                  mmap_read_lock(&init_mm)  <-- self-deadlock
> 
> Because fs_reclaim nests outside init_mm.mmap_lock, this also triggers a
> lockdep splat on the first vmemmap PMD split (including under
> mmap_read_lock(&init_mm) in vmemmap_optimize_folio()) and can ABBA deadlock
> against kswapd.
> 
> -- 
> Sashiko AI review · https://sashiko.dev/#/patchset/20261110112456.62827-1-songmuchun@bytedance.com?part=1