[PATCH v4 0/9] mm/vmalloc: fix up vmap error handling

[PATCH v4 0/9] mm/vmalloc: fix up vmap error handling

[Hao Ge]

__vmap_pages_range_noflush() and friends can install some PTEs before
failing and leave them mapped, and the kernel callers do not agree on
who cleans them up. pcpu_map_pages() and kmsan_ioremap_page_range()
roll back what they mapped before the failure, while
vm_module_tags_populate() and the __GFP_NOFAIL retry loop in
__vmalloc_area_node() rely on the mapping functions cleaning up and
do not call anything like vunmap_range() themselves. When the same
range is mapped again, the attempt hits the leftovers and fails with
a warning in vmap_pages_pte_range(). A leftover huge pmd is worse.
During the retry, vmap_try_huge_pmd() calls pmd_free_pte_page() on
it, and the x86 implementation cannot tell a leaf entry from a page
table and frees the mapped page.

After discussing with Suren and Ulad [1], we move the rollback into
the mapping functions. A failed vmap leaves nothing mapped, and the
callers no longer clean up after it.

Patch 1 is the KMSAN part. The shadow and the origin of a range are
mapped by two separate calls, so the rollback there has to cover
both. Patch 2 is its disabled-KMSAN counterpart. Patches 3 to 5 bug
out on an unaligned range at the entry of the three range walkers.
Patch 6 only flushes the cache once the mapping succeeded. Patch 7
adds the rollback to __vmap_pages_range_noflush() and
vmap_page_range(). Patches 8 and 9 drop the powerpc and percpu
cleanups that are not needed anymore.

Changes since v3:
- drop v3 patch 3. The clamp papered over the caller bug, a bad
  range now BUG_ONs at the unmap entry
- drop v3 patch 4. The reworked walk (24bbe1c56f63a) propagates
  the errors already
- drop v3 patch 6. The walk below checks the range already
- the map side checks moved from the pte level to the entries of
  vmap_range_noflush() and the walk, and became BUG_ON, in line
  with the unmap side
- rebase onto the walk rework, commit messages updated accordingly

[1] https://lore.kernel.org/all/20260915070001.113559-3-hao.ge@linux.dev/

v3 link: https://lore.kernel.org/all/20261026080329.140244-1-hao.ge@linux.dev/
v2 link: https://lore.kernel.org/all/20261019054531.87652-1-hao.ge@linux.dev/
v1 link: https://lore.kernel.org/all/20261009063619.112313-1-hao.ge@linux.dev/

Hao Ge (9):
  mm/kmsan: undo the shadow mapping when the origin mapping fails
  mm/kmsan: skip vunmap_range_noflush when KMSAN is disabled
  mm/vmalloc: BUG_ON() on an unaligned range in vmap_range_noflush()
  mm/vmalloc: BUG_ON() on an unaligned range in
    vmap_pages_range_noflush_walk()
  mm/vmalloc: BUG_ON() on an unaligned range in __vunmap_range_noflush()
  mm/vmalloc: only flush the cache after the vmap succeeded
  mm/vmalloc: undo partial mappings inside the mapping functions
  powerpc: drop redundant unmaps of failed vmap mappings
  mm/percpu: stop unmapping the CPU that failed to map

 arch/powerpc/kernel/isa-bridge.c |  5 +--
 arch/powerpc/kernel/pci_64.c     |  4 +-
 include/linux/kmsan.h            |  8 +++-
 mm/kmsan/hooks.c                 | 21 ++++++---
 mm/kmsan/shadow.c                |  9 +++-
 mm/percpu-vm.c                   |  5 ++-
 mm/vmalloc.c                     | 75 ++++++++++++++++++++++++--------
 mm/vmalloc.h                     |  6 ++-
 8 files changed, 94 insertions(+), 39 deletions(-)

-- 
2.25.1

[PATCH v4 1/9] mm/kmsan: undo the shadow mapping when the origin mapping fails

[Hao Ge]

kmsan_vmap_pages_range_noflush() first maps the shadow pages and then
the origin pages. If the second mapping fails, the first one is left
mapped, and the callers do not roll it back. The next vmap of the
same metadata range hits the stale PTEs again and fails with a
WARN_ON() and -EBUSY in vmap_pages_pte_range().

Undo the shadow mapping on that error path, the same way
kmsan_ioremap_page_range() cleans up after a partial failure.
__vunmap_range_noflush() only clears the PTEs; nothing has touched
the shadow mapping, so no TLB flush is needed.

Fixes: 47ebd0310e89 ("mm: kmsan: handle alloc failures in kmsan_vmap_pages_range_noflush()")
Cc: stable@vger.kernel.org
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Dev Jain <dev.jain@arm.com>
Signed-off-by: Hao Ge <hao.ge@linux.dev>
---
 mm/kmsan/shadow.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/mm/kmsan/shadow.c b/mm/kmsan/shadow.c
index 0c88d89bf0d6..2166086d3dc3 100644
--- a/mm/kmsan/shadow.c
+++ b/mm/kmsan/shadow.c
@@ -258,6 +258,10 @@ int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
 					    o_pages, page_shift);
 	kmsan_leave_runtime();
 	if (mapped) {
+		/* Undo the shadow mapping set up above. */
+		kmsan_enter_runtime();
+		__vunmap_range_noflush(shadow_start, shadow_end);
+		kmsan_leave_runtime();
 		err = mapped;
 		goto ret;
 	}
-- 
2.25.1

[PATCH v4 2/9] mm/kmsan: skip vunmap_range_noflush when KMSAN is disabled

[Hao Ge]

kmsan_vmap_pages_range_noflush() returns 0 when kmsan_enabled is
false.  The unmap side does not, so it still walks the shadow and
origin page tables even though KMSAN never installed any metadata
there.

Add the matching early return.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Reviewed-by: Dev Jain <dev.jain@arm.com>
Signed-off-by: Hao Ge <hao.ge@linux.dev>
---
 mm/kmsan/hooks.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c
index a706666db097..107f66841274 100644
--- a/mm/kmsan/hooks.c
+++ b/mm/kmsan/hooks.c
@@ -138,6 +138,9 @@ static unsigned long vmalloc_origin(unsigned long addr)
 
 void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end)
 {
+	if (!kmsan_enabled)
+		return;
+
 	__vunmap_range_noflush(vmalloc_shadow(start), vmalloc_shadow(end));
 	__vunmap_range_noflush(vmalloc_origin(start), vmalloc_origin(end));
 	flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end));
-- 
2.25.1

[PATCH v4 3/9] mm/vmalloc: BUG_ON() on an unaligned range in vmap_range_noflush()

[Hao Ge]

commit a739e6b557af ("mm: vmalloc: WARN_ON if mapping size is not
PAGE_SIZE aligned") put the check in vmap_pte_range(), which is too
late, because the upper levels may already have installed page
tables for a range that is rejected anyway.

The pte loop maps whole pages and stops only when addr lands
exactly on end.  An unaligned end lets addr jump over it and the
loop keeps mapping past the range.  An unaligned start maps the
page below the range and usually misses end too.  Such a range is
never a legitimate request, and the corruption it leaves behind is
hard to attribute to the caller afterwards, so bug out on addr and
end at the entry of vmap_range_noflush() and drop the check from
vmap_pte_range().

Signed-off-by: Hao Ge <hao.ge@linux.dev>
---
 mm/vmalloc.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 093b8d5f7b0e..27f2674a5556 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -125,9 +125,6 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
 	unsigned long size;
 	unsigned int nr_pages;
 
-	if (WARN_ON_ONCE(!PAGE_ALIGNED(end - addr)))
-		return -EINVAL;
-
 	pfn = phys_addr >> PAGE_SHIFT;
 	pte = pte_alloc_kernel_track(pmd, addr, mask);
 	if (!pte)
@@ -347,6 +344,7 @@ static int vmap_range_noflush(unsigned long addr, unsigned long end,
 	 */
 	might_sleep();
 	BUG_ON(addr >= end);
+	BUG_ON(!PAGE_ALIGNED(addr) || !PAGE_ALIGNED(end));
 
 	start = addr;
 	pgd = pgd_offset_k(addr);
-- 
2.25.1

[PATCH v4 4/9] mm/vmalloc: BUG_ON() on an unaligned range in vmap_pages_range_noflush_walk()

[Hao Ge]

The pte loop of vmap_pages_pte_range() maps whole pages and stops
only when addr lands exactly on end.  An unaligned end lets addr
jump over it and the loop keeps mapping into whatever follows the
range.  An unaligned start maps the page below the range with its
first pte.  That corruption is impossible to debug afterwards, so
bug out on addr and end at the entry of
vmap_pages_range_noflush_walk().

Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Hao Ge <hao.ge@linux.dev>
---
 mm/vmalloc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 27f2674a5556..86fbb1d2b27a 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -698,6 +698,8 @@ static int vmap_pages_range_noflush_walk(unsigned long addr, unsigned long end,
 	pgtbl_mod_mask mask = 0;
 
 	BUG_ON(addr >= end);
+	BUG_ON(!PAGE_ALIGNED(addr) || !PAGE_ALIGNED(end));
+
 	pgd = pgd_offset_k(addr);
 	do {
 		next = pgd_addr_end(addr, end);
-- 
2.25.1

[PATCH v4 5/9] mm/vmalloc: BUG_ON() on an unaligned range in __vunmap_range_noflush()

[Hao Ge]

The PTE loop of the unmap path stops only at addr == end.  With an
unaligned end addr never lands on end, so the loop walks past the
range and keeps clearing.  With an unaligned start the loop begins
in the page containing start, which extends below the range.  Either
way the unmap clears the mappings adjacent to the range, and that is
nearly impossible to debug afterwards.

vmap only installs page-aligned, page-granular mappings, so an
unaligned unmap request is never a legitimate teardown, and there is
no way to tell which pages the caller meant.  Check addr and end at
the entry of __vunmap_range_noflush() and bug out before any PTE is
cleared.

Signed-off-by: Hao Ge <hao.ge@linux.dev>
---
 mm/vmalloc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 86fbb1d2b27a..0565cbfe325c 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -518,6 +518,8 @@ void __vunmap_range_noflush(unsigned long start, unsigned long end)
 	pgtbl_mod_mask mask = 0;
 
 	BUG_ON(addr >= end);
+	BUG_ON(!PAGE_ALIGNED(addr) || !PAGE_ALIGNED(end));
+
 	pgd = pgd_offset_k(addr);
 	do {
 		next = pgd_addr_end(addr, end);
-- 
2.25.1

[PATCH v4 6/9] mm/vmalloc: only flush the cache after the vmap succeeded

[Hao Ge]

vmap_page_range() and __vmap_pages_range() call flush_cache_vmap()
before they even look at the error code. A failed mapping is never
used, so the flush is pointless at best.

According to the Sashiko review it is worse than pointless on UML:
flush_cache_vmap() there is flush_tlb_kernel_range(), which syncs the
PTEs into the host OS. Flushing a half-installed mapping therefore
leaves host mappings behind for a range that was just freed again.

Only flush when the mapping, and for vmap_page_range() also the KMSAN
metadata, is fully set up.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Hao Ge <hao.ge@linux.dev>
---
 mm/vmalloc.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 0565cbfe325c..3f98056f8c7e 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -369,10 +369,12 @@ int vmap_page_range(unsigned long addr, unsigned long end,
 
 	err = vmap_range_noflush(addr, end, phys_addr, pgprot_nx(prot),
 				 ioremap_max_page_shift);
-	flush_cache_vmap(addr, end);
 	if (!err)
 		err = kmsan_ioremap_page_range(addr, end, phys_addr, prot,
 					       ioremap_max_page_shift);
+	if (!err)
+		flush_cache_vmap(addr, end);
+
 	return err;
 }
 
@@ -758,7 +760,9 @@ static int __vmap_pages_range(unsigned long addr, unsigned long end,
 	int err;
 
 	err = vmap_pages_range_noflush(addr, end, prot, pages, page_shift, gfp_mask);
-	flush_cache_vmap(addr, end);
+	if (!err)
+		flush_cache_vmap(addr, end);
+
 	return err;
 }
 
-- 
2.25.1

[PATCH v4 7/9] mm/vmalloc: undo partial mappings inside the mapping functions

[Hao Ge]

The vmap entry points may install some PTEs and then fail, leaving a
partial mapping behind.

The callers do not agree on who cleans it up. pcpu_map_pages() and
kmsan_ioremap_page_range() roll back what they mapped, while
vm_module_tags_populate() and the __GFP_NOFAIL retry loop in
__vmalloc_area_node() expect a clean range from the mapping function.
For them the leftovers stay mapped, and the next attempt to map the
same range trips the WARN_ON(!pte_none()) in vmap_pages_pte_range().
A leftover huge pmd is even worse. During the retry, vmap_try_huge_pmd()
calls pmd_free_pte_page() on it, and the x86 implementation cannot
tell a leaf entry from a page table and frees the mapped page.

After discussing with Suren and Ulad, we decided to put the rollback
into the entry points of the vmap API rather than into every
low-level helper. __vmap_pages_range_noflush() undoes the whole range
it was asked to map when it fails, and vmap_page_range() undoes its
range for the ioremap-style mappings.

The rollback is __vunmap_range_noflush(), which only clears the PTEs.
No TLB flush is needed, nothing has touched the mapping yet.
vmap_pages_range_noflush() also drops the KMSAN metadata of the
failed mapping.

The undo walk skips its cond_resched() when the caller's gfp mask
cannot block. __vmalloc() with GFP_ATOMIC or GFP_NOWAIT can run in
atomic context, where cond_resched() cannot schedule.
__vunmap_range_noflush() takes a can_resched argument for that, which
the sleepable callers pass as true.

Suggested-by: Uladyslau Rezki <urezki@gmail.com>
Signed-off-by: Hao Ge <hao.ge@linux.dev>
---
 include/linux/kmsan.h |  8 ++++--
 mm/kmsan/hooks.c      | 18 ++++++++-----
 mm/kmsan/shadow.c     |  7 ++---
 mm/vmalloc.c          | 61 +++++++++++++++++++++++++++++++++----------
 mm/vmalloc.h          |  6 +++--
 5 files changed, 72 insertions(+), 28 deletions(-)

diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h
index 7da9fd506b39..0016c032d1e7 100644
--- a/include/linux/kmsan.h
+++ b/include/linux/kmsan.h
@@ -150,11 +150,14 @@ int __must_check kmsan_vmap_pages_range_noflush(unsigned long start,
  * kmsan_vunmap_kernel_range_noflush() - Notify KMSAN about a vunmap.
  * @start: start of vunmapped range.
  * @end:   end of vunmapped range.
+ * @can_resched: whether the unmap walk may call cond_resched(); pass
+ * false from atomic context.
  *
  * KMSAN unmaps the contiguous metadata ranges created by
  * kmsan_map_kernel_range_noflush().
  */
-void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end);
+void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end,
+				bool can_resched);
 
 /**
  * kmsan_ioremap_page_range() - Notify KMSAN about a ioremap_page_range() call.
@@ -355,7 +358,8 @@ static inline int __must_check kmsan_vmap_pages_range_noflush(
 }
 
 static inline void kmsan_vunmap_range_noflush(unsigned long start,
-					      unsigned long end)
+					      unsigned long end,
+					      bool can_resched)
 {
 }
 
diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c
index 107f66841274..0cd1edaa1120 100644
--- a/mm/kmsan/hooks.c
+++ b/mm/kmsan/hooks.c
@@ -136,13 +136,16 @@ static unsigned long vmalloc_origin(unsigned long addr)
 						 KMSAN_META_ORIGIN);
 }
 
-void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end)
+void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end,
+				bool can_resched)
 {
 	if (!kmsan_enabled)
 		return;
 
-	__vunmap_range_noflush(vmalloc_shadow(start), vmalloc_shadow(end));
-	__vunmap_range_noflush(vmalloc_origin(start), vmalloc_origin(end));
+	__vunmap_range_noflush(vmalloc_shadow(start), vmalloc_shadow(end),
+			       can_resched);
+	__vunmap_range_noflush(vmalloc_origin(start), vmalloc_origin(end),
+			       can_resched);
 	flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end));
 	flush_cache_vmap(vmalloc_origin(start), vmalloc_origin(end));
 }
@@ -171,8 +174,9 @@ static void kmsan_iounmap_pages(unsigned long start, unsigned long end)
 		if (origin)
 			__free_pages(origin, KMSAN_IOREMAP_META_ORDER);
 	}
-	__vunmap_range_noflush(shadow_start, shadow_end);
-	__vunmap_range_noflush(origin_start, origin_end);
+	/* ioremap metadata mappings are always GFP_KERNEL, so resched is safe. */
+	__vunmap_range_noflush(shadow_start, shadow_end, true);
+	__vunmap_range_noflush(origin_start, origin_end, true);
 	flush_tlb_kernel_range(shadow_start, shadow_end);
 	flush_tlb_kernel_range(origin_start, origin_end);
 }
@@ -207,7 +211,7 @@ int kmsan_ioremap_page_range(unsigned long start, unsigned long end,
 		mapped = __vmap_pages_range_noflush(
 			vmalloc_shadow(start + off),
 			vmalloc_shadow(start + off + PAGE_SIZE), prot, &shadow,
-			PAGE_SHIFT);
+			PAGE_SHIFT, gfp_mask);
 		if (mapped) {
 			err = mapped;
 			goto ret;
@@ -217,7 +221,7 @@ int kmsan_ioremap_page_range(unsigned long start, unsigned long end,
 		mapped = __vmap_pages_range_noflush(
 			vmalloc_origin(start + off),
 			vmalloc_origin(start + off + PAGE_SIZE), prot, &origin,
-			PAGE_SHIFT);
+			PAGE_SHIFT, gfp_mask);
 		if (mapped) {
 			err = mapped;
 			goto ret;
diff --git a/mm/kmsan/shadow.c b/mm/kmsan/shadow.c
index 2166086d3dc3..452760a333e4 100644
--- a/mm/kmsan/shadow.c
+++ b/mm/kmsan/shadow.c
@@ -247,7 +247,7 @@ int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
 	origin_end = vmalloc_meta((void *)end, KMSAN_META_ORIGIN);
 	kmsan_enter_runtime();
 	mapped = __vmap_pages_range_noflush(shadow_start, shadow_end, prot,
-					    s_pages, page_shift);
+					    s_pages, page_shift, gfp_mask);
 	kmsan_leave_runtime();
 	if (mapped) {
 		err = mapped;
@@ -255,12 +255,13 @@ int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
 	}
 	kmsan_enter_runtime();
 	mapped = __vmap_pages_range_noflush(origin_start, origin_end, prot,
-					    o_pages, page_shift);
+					    o_pages, page_shift, gfp_mask);
 	kmsan_leave_runtime();
 	if (mapped) {
 		/* Undo the shadow mapping set up above. */
 		kmsan_enter_runtime();
-		__vunmap_range_noflush(shadow_start, shadow_end);
+		__vunmap_range_noflush(shadow_start, shadow_end,
+				       gfpflags_allow_blocking(gfp_mask));
 		kmsan_leave_runtime();
 		err = mapped;
 		goto ret;
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 3f98056f8c7e..d1986f68eab4 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -372,8 +372,12 @@ int vmap_page_range(unsigned long addr, unsigned long end,
 	if (!err)
 		err = kmsan_ioremap_page_range(addr, end, phys_addr, prot,
 					       ioremap_max_page_shift);
-	if (!err)
+	if (err) {
+		/* Callers are ioremap and arch setup code, always sleepable. */
+		__vunmap_range_noflush(addr, end, true);
+	} else {
 		flush_cache_vmap(addr, end);
+	}
 
 	return err;
 }
@@ -429,7 +433,7 @@ static void vunmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
 }
 
 static void vunmap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end,
-			     pgtbl_mod_mask *mask)
+			     pgtbl_mod_mask *mask, bool can_resched)
 {
 	pmd_t *pmd;
 	unsigned long next;
@@ -451,12 +455,14 @@ static void vunmap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end,
 			continue;
 		vunmap_pte_range(pmd, addr, next, mask);
 
-		cond_resched();
+		/* The undo path may run in atomic context. */
+		if (can_resched)
+			cond_resched();
 	} while (pmd++, addr = next, addr != end);
 }
 
 static void vunmap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end,
-			     pgtbl_mod_mask *mask)
+			     pgtbl_mod_mask *mask, bool can_resched)
 {
 	pud_t *pud;
 	unsigned long next;
@@ -476,12 +482,12 @@ static void vunmap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end,
 		}
 		if (pud_none_or_clear_bad(pud))
 			continue;
-		vunmap_pmd_range(pud, addr, next, mask);
+		vunmap_pmd_range(pud, addr, next, mask, can_resched);
 	} while (pud++, addr = next, addr != end);
 }
 
 static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end,
-			     pgtbl_mod_mask *mask)
+			     pgtbl_mod_mask *mask, bool can_resched)
 {
 	p4d_t *p4d;
 	unsigned long next;
@@ -496,7 +502,7 @@ static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end,
 
 		if (p4d_none_or_clear_bad(p4d))
 			continue;
-		vunmap_pud_range(p4d, addr, next, mask);
+		vunmap_pud_range(p4d, addr, next, mask, can_resched);
 	} while (p4d++, addr = next, addr != end);
 }
 
@@ -512,7 +518,8 @@ static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end,
  *
  * This is an internal function only. Do not use outside mm/.
  */
-void __vunmap_range_noflush(unsigned long start, unsigned long end)
+void __vunmap_range_noflush(unsigned long start, unsigned long end,
+			    bool can_resched)
 {
 	unsigned long next;
 	pgd_t *pgd;
@@ -529,7 +536,7 @@ void __vunmap_range_noflush(unsigned long start, unsigned long end)
 			mask |= PGTBL_PGD_MODIFIED;
 		if (pgd_none_or_clear_bad(pgd))
 			continue;
-		vunmap_p4d_range(pgd, addr, next, &mask);
+		vunmap_p4d_range(pgd, addr, next, &mask, can_resched);
 	} while (pgd++, addr = next, addr != end);
 
 	if (mask & ARCH_PAGE_TABLE_SYNC_MASK)
@@ -538,8 +545,9 @@ void __vunmap_range_noflush(unsigned long start, unsigned long end)
 
 void vunmap_range_noflush(unsigned long start, unsigned long end)
 {
-	kmsan_vunmap_range_noflush(start, end);
-	__vunmap_range_noflush(start, end);
+	/* The unmap paths are sleepable, keep the resched point. */
+	kmsan_vunmap_range_noflush(start, end, true);
+	__vunmap_range_noflush(start, end, true);
 }
 
 /**
@@ -727,18 +735,32 @@ static int vmap_pages_range_noflush_walk(unsigned long addr, unsigned long end,
  * The caller is responsible for calling flush_cache_vmap() after this
  * function returns successfully and before the addresses are accessed.
  *
+ * On failure the mapping is torn down before returning, so a retry
+ * sees a clean range. When @gfp_mask cannot block, the teardown
+ * skips cond_resched(), it may run in atomic context.
+ *
  * This is an internal function only. Do not use outside mm/.
  */
 int __vmap_pages_range_noflush(unsigned long addr, unsigned long end,
-		pgprot_t prot, struct page **pages, unsigned int page_shift)
+		pgprot_t prot, struct page **pages, unsigned int page_shift,
+		gfp_t gfp_mask)
 {
+	int err;
+
 	if (WARN_ON_ONCE(page_shift < PAGE_SHIFT))
 		return -EINVAL;
 
 	if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC))
 		page_shift = PAGE_SHIFT;
 
-	return vmap_pages_range_noflush_walk(addr, end, prot, pages, page_shift);
+	err = vmap_pages_range_noflush_walk(addr, end, prot, pages, page_shift);
+
+	/* Undo the PTEs installed before the failure. */
+	if (err)
+		__vunmap_range_noflush(addr, end,
+				       gfpflags_allow_blocking(gfp_mask));
+
+	return err;
 }
 
 int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
@@ -750,7 +772,18 @@ int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
 
 	if (ret)
 		return ret;
-	return __vmap_pages_range_noflush(addr, end, prot, pages, page_shift);
+
+	ret = __vmap_pages_range_noflush(addr, end, prot, pages, page_shift,
+					 gfp_mask);
+	/*
+	 * The page tables undo themselves on failure. Tear down the
+	 * metadata that was fully set up before the mapping failed.
+	 */
+	if (ret)
+		kmsan_vunmap_range_noflush(addr, end,
+					   gfpflags_allow_blocking(gfp_mask));
+
+	return ret;
 }
 
 static int __vmap_pages_range(unsigned long addr, unsigned long end,
diff --git a/mm/vmalloc.h b/mm/vmalloc.h
index d8372a4f65c4..9ea28845f713 100644
--- a/mm/vmalloc.h
+++ b/mm/vmalloc.h
@@ -40,10 +40,12 @@ void clear_vm_uninitialized_flag(struct vm_struct *vm);
 
 int __must_check __vmap_pages_range_noflush(unsigned long addr,
 		unsigned long end, pgprot_t prot,
-		struct page **pages, unsigned int page_shift);
+		struct page **pages, unsigned int page_shift,
+		gfp_t gfp_mask);
 
 void vunmap_range_noflush(unsigned long start, unsigned long end);
 
-void __vunmap_range_noflush(unsigned long start, unsigned long end);
+void __vunmap_range_noflush(unsigned long start, unsigned long end,
+			    bool can_resched);
 
 #endif /* __MM_VMALLOC_H */
-- 
2.25.1

[PATCH v4 8/9] powerpc: drop redundant unmaps of failed vmap mappings

[Hao Ge]

vmap_page_range() and ioremap_page_range() clear the PTEs they
installed when they fail, so the vunmap_range() in remap_isa_base()
and ioremap_phb() has nothing left to remove.

Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
Signed-off-by: Hao Ge <hao.ge@linux.dev>
---
 arch/powerpc/kernel/isa-bridge.c | 5 ++---
 arch/powerpc/kernel/pci_64.c     | 4 +---
 2 files changed, 3 insertions(+), 6 deletions(-)

diff --git a/arch/powerpc/kernel/isa-bridge.c b/arch/powerpc/kernel/isa-bridge.c
index 5c064485197a..93029d85e6ea 100644
--- a/arch/powerpc/kernel/isa-bridge.c
+++ b/arch/powerpc/kernel/isa-bridge.c
@@ -46,9 +46,8 @@ static void remap_isa_base(phys_addr_t pa, unsigned long size)
 	WARN_ON_ONCE(size & ~PAGE_MASK);
 
 	if (slab_is_available()) {
-		if (vmap_page_range(ISA_IO_BASE, ISA_IO_BASE + size, pa,
-				    pgprot_noncached(PAGE_KERNEL)))
-			vunmap_range(ISA_IO_BASE, ISA_IO_BASE + size);
+		vmap_page_range(ISA_IO_BASE, ISA_IO_BASE + size, pa,
+				pgprot_noncached(PAGE_KERNEL));
 	} else {
 		early_ioremap_range(ISA_IO_BASE, pa, size,
 				pgprot_noncached(PAGE_KERNEL));
diff --git a/arch/powerpc/kernel/pci_64.c b/arch/powerpc/kernel/pci_64.c
index e27342ef128b..f74269636d5e 100644
--- a/arch/powerpc/kernel/pci_64.c
+++ b/arch/powerpc/kernel/pci_64.c
@@ -139,10 +139,8 @@ void __iomem *ioremap_phb(phys_addr_t paddr, unsigned long size)
 
 	addr = (unsigned long)area->addr;
 	if (ioremap_page_range(addr, addr + size, paddr,
-			pgprot_noncached(PAGE_KERNEL))) {
-		vunmap_range(addr, addr + size);
+			pgprot_noncached(PAGE_KERNEL)))
 		return NULL;
-	}
 
 	return (void __iomem *)addr;
 }
-- 
2.25.1

[PATCH v4 9/9] mm/percpu: stop unmapping the CPU that failed to map

[Hao Ge]

__pcpu_map_pages() clears the PTEs it installed when it fails, so
pcpu_map_pages() only has to tear down the CPUs it mapped before
the failing one.

Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
Signed-off-by: Hao Ge <hao.ge@linux.dev>
---
 mm/percpu-vm.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/mm/percpu-vm.c b/mm/percpu-vm.c
index 509d8901835c..59d3a336a6bd 100644
--- a/mm/percpu-vm.c
+++ b/mm/percpu-vm.c
@@ -252,10 +252,11 @@ static int pcpu_map_pages(struct pcpu_chunk *chunk, struct page **pages,
 	return 0;
 err:
 	for_each_possible_cpu(tcpu) {
-		__pcpu_unmap_pages(pcpu_chunk_addr(chunk, tcpu, page_start),
-				   page_end - page_start);
+		/* The failing CPU's partial mapping undoes itself */
 		if (tcpu == cpu)
 			break;
+		__pcpu_unmap_pages(pcpu_chunk_addr(chunk, tcpu, page_start),
+				   page_end - page_start);
 	}
 	pcpu_post_unmap_tlb_flush(chunk, page_start, page_end);
 	return err;
-- 
2.25.1

Re: [PATCH v4 3/9] mm/vmalloc: BUG_ON() on an unaligned range in vmap_range_noflush()

[Hao Ge]

On 2026/11/10 19:25, Hao Ge wrote:
> commit a739e6b557af ("mm: vmalloc: WARN_ON if mapping size is not
> PAGE_SIZE aligned") put the check in vmap_pte_range(), which is too
> late, because the upper levels may already have installed page
> tables for a range that is rejected anyway.
> 
> The pte loop maps whole pages and stops only when addr lands
> exactly on end.  An unaligned end lets addr jump over it and the
> loop keeps mapping past the range.  An unaligned start maps the
> page below the range and usually misses end too.  Such a range is
> never a legitimate request, and the corruption it leaves behind is
> hard to attribute to the caller afterwards, so bug out on addr and
> end at the entry of vmap_range_noflush() and drop the check from
> vmap_pte_range().
> 
> Signed-off-by: Hao Ge <hao.ge@linux.dev>
> ---
>  mm/vmalloc.c | 4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
> 
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 093b8d5f7b0e..27f2674a5556 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -125,9 +125,6 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
>  	unsigned long size;
>  	unsigned int nr_pages;
>  
> -	if (WARN_ON_ONCE(!PAGE_ALIGNED(end - addr)))
> -		return -EINVAL;
> -
>  	pfn = phys_addr >> PAGE_SHIFT;
>  	pte = pte_alloc_kernel_track(pmd, addr, mask);
>  	if (!pte)
> @@ -347,6 +344,7 @@ static int vmap_range_noflush(unsigned long addr, unsigned long end,
>  	 */
>  	might_sleep();
>  	BUG_ON(addr >= end);
> +	BUG_ON(!PAGE_ALIGNED(addr) || !PAGE_ALIGNED(end));

I know BUG_ON() is ugly and it was not my first choice either, but I
have not figured out a better way to handle __vunmap_range_noflush().

If an unaligned range is passed to it, the pte loop may clear PTEs
adjacent to the range. Say a caller passes [3, 12) while [0, 3)
belongs to somebody else. The loop operates on whole pages, so it
starts at the page containing 3 and clears [0, 3) with it. That
silently corrupts a live mapping, and the corruption is not detected
until the owner touches the page. Bugs like that are nearly
impossible to debug afterwards.

It is not feasible for __vunmap_range_noflush() to just return an
invalid-argument error upfront. Its job is to clear PTEs, and an
early return leaves the residual PTEs behind.

I also thought about warning and returning -EINVAL on the map side
and keeping BUG_ON() only in __vunmap_range_noflush(), but the
rollback kills that idea.

int __vmap_pages_range_noflush(unsigned long addr, unsigned long end,
                pgprot_t prot, struct page **pages, unsigned int page_shift,
                gfp_t gfp_mask)
{
        int err;

        if (WARN_ON_ONCE(page_shift < PAGE_SHIFT))
                return -EINVAL;

        if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC))
                page_shift = PAGE_SHIFT;

        err = vmap_pages_range_noflush_walk(addr, end, prot, pages, page_shift);

        /* Undo the PTEs installed before the failure. */
        if (err)
                __vunmap_range_noflush(addr, end,
                                       gfpflags_allow_blocking(gfp_mask));

        return err;
}

If the walk WARNs and returns -EINVAL for a bad range, the undo
above passes that same range to __vunmap_range_noflush(), and we
BUG right after the WARN. vmap_page_range() runs into the same
thing on its error path.

So the map side cannot warn while the unmap side BUGs, the rollback
drags the rejected range into the unmap anyway. And the unmap side
cannot warn and skip either, that leaves the stale PTEs.

Also, these checks are the last line of defense. All in-tree callers
pass page aligned, non-empty ranges, and I am not aware of anyone ever
hitting this line. So for correct code the BUG_ON never fires at all,
no user will ever see it. The only one who can trip it is the developer
who just passed the bad range, and for him a stack trace right at the
spot is much better than chasing a silent corruption afterwards.

I settled on BUG_ON here. It may not be an ideal solution, but if anyone
has a better idea or alternative approach, please let me know.
I would greatly appreciate it.

Thanks
Best Regards
Hao

>  
>  	start = addr;
>  	pgd = pgd_offset_k(addr);

Re: [PATCH v4 0/9] mm/vmalloc: fix up vmap error handling

[Suren Baghdasaryan]

On Tue, Nov 10, 2026 at 11:24 AM Hao Ge <hao.ge@linux.dev> wrote:
>
> __vmap_pages_range_noflush() and friends can install some PTEs before
> failing and leave them mapped, and the kernel callers do not agree on
> who cleans them up. pcpu_map_pages() and kmsan_ioremap_page_range()
> roll back what they mapped before the failure, while
> vm_module_tags_populate() and the __GFP_NOFAIL retry loop in
> __vmalloc_area_node() rely on the mapping functions cleaning up and
> do not call anything like vunmap_range() themselves. When the same
> range is mapped again, the attempt hits the leftovers and fails with
> a warning in vmap_pages_pte_range(). A leftover huge pmd is worse.
> During the retry, vmap_try_huge_pmd() calls pmd_free_pte_page() on
> it, and the x86 implementation cannot tell a leaf entry from a page
> table and frees the mapped page.

Please note that there is a new function called
vmap_pages_range_batched() which should also unmap its incomplete
mappings on failure. This function was introduced by a patch called
"mm/vmalloc: map contiguous pages in batches for vmap() if possible"
in mm-new. I think it should be handled by this series as well.

checkpatch.pl also complains about "Reported-by: should be immediately
followed by Closes" on some of the patches, which can be easily fixed.

>
> After discussing with Suren and Ulad [1], we move the rollback into
> the mapping functions. A failed vmap leaves nothing mapped, and the
> callers no longer clean up after it.
>
> Patch 1 is the KMSAN part. The shadow and the origin of a range are
> mapped by two separate calls, so the rollback there has to cover
> both. Patch 2 is its disabled-KMSAN counterpart. Patches 3 to 5 bug
> out on an unaligned range at the entry of the three range walkers.
> Patch 6 only flushes the cache once the mapping succeeded. Patch 7
> adds the rollback to __vmap_pages_range_noflush() and
> vmap_page_range(). Patches 8 and 9 drop the powerpc and percpu
> cleanups that are not needed anymore.
>
> Changes since v3:
> - drop v3 patch 3. The clamp papered over the caller bug, a bad
>   range now BUG_ONs at the unmap entry
> - drop v3 patch 4. The reworked walk (24bbe1c56f63a) propagates
>   the errors already
> - drop v3 patch 6. The walk below checks the range already
> - the map side checks moved from the pte level to the entries of
>   vmap_range_noflush() and the walk, and became BUG_ON, in line
>   with the unmap side
> - rebase onto the walk rework, commit messages updated accordingly
>
> [1] https://lore.kernel.org/all/20260915070001.113559-3-hao.ge@linux.dev/
>
> v3 link: https://lore.kernel.org/all/20261026080329.140244-1-hao.ge@linux.dev/
> v2 link: https://lore.kernel.org/all/20261019054531.87652-1-hao.ge@linux.dev/
> v1 link: https://lore.kernel.org/all/20261009063619.112313-1-hao.ge@linux.dev/
>
> Hao Ge (9):
>   mm/kmsan: undo the shadow mapping when the origin mapping fails
>   mm/kmsan: skip vunmap_range_noflush when KMSAN is disabled
>   mm/vmalloc: BUG_ON() on an unaligned range in vmap_range_noflush()
>   mm/vmalloc: BUG_ON() on an unaligned range in
>     vmap_pages_range_noflush_walk()
>   mm/vmalloc: BUG_ON() on an unaligned range in __vunmap_range_noflush()
>   mm/vmalloc: only flush the cache after the vmap succeeded
>   mm/vmalloc: undo partial mappings inside the mapping functions
>   powerpc: drop redundant unmaps of failed vmap mappings
>   mm/percpu: stop unmapping the CPU that failed to map
>
>  arch/powerpc/kernel/isa-bridge.c |  5 +--
>  arch/powerpc/kernel/pci_64.c     |  4 +-
>  include/linux/kmsan.h            |  8 +++-
>  mm/kmsan/hooks.c                 | 21 ++++++---
>  mm/kmsan/shadow.c                |  9 +++-
>  mm/percpu-vm.c                   |  5 ++-
>  mm/vmalloc.c                     | 75 ++++++++++++++++++++++++--------
>  mm/vmalloc.h                     |  6 ++-
>  8 files changed, 94 insertions(+), 39 deletions(-)
>
> --
> 2.25.1

Re: [PATCH v4 1/9] mm/kmsan: undo the shadow mapping when the origin mapping fails

[Suren Baghdasaryan]

On Tue, Nov 10, 2026 at 11:24 AM Hao Ge <hao.ge@linux.dev> wrote:
>
> kmsan_vmap_pages_range_noflush() first maps the shadow pages and then
> the origin pages. If the second mapping fails, the first one is left
> mapped, and the callers do not roll it back. The next vmap of the
> same metadata range hits the stale PTEs again and fails with a
> WARN_ON() and -EBUSY in vmap_pages_pte_range().
>
> Undo the shadow mapping on that error path, the same way
> kmsan_ioremap_page_range() cleans up after a partial failure.
> __vunmap_range_noflush() only clears the PTEs; nothing has touched
> the shadow mapping, so no TLB flush is needed.
>
> Fixes: 47ebd0310e89 ("mm: kmsan: handle alloc failures in kmsan_vmap_pages_range_noflush()")
> Cc: stable@vger.kernel.org
> Reviewed-by: Suren Baghdasaryan <surenb@google.com>
> Reviewed-by: Dev Jain <dev.jain@arm.com>
> Signed-off-by: Hao Ge <hao.ge@linux.dev>
> ---
>  mm/kmsan/shadow.c | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/mm/kmsan/shadow.c b/mm/kmsan/shadow.c
> index 0c88d89bf0d6..2166086d3dc3 100644
> --- a/mm/kmsan/shadow.c
> +++ b/mm/kmsan/shadow.c
> @@ -258,6 +258,10 @@ int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
>                                             o_pages, page_shift);
>         kmsan_leave_runtime();
>         if (mapped) {
> +               /* Undo the shadow mapping set up above. */
> +               kmsan_enter_runtime();
> +               __vunmap_range_noflush(shadow_start, shadow_end);

Sorry, there is a problem here. You introduce "can_resched" check to
avoid calling cond_resched() later in the series (in patch #7), but at
this point __vunmap_range_noflush() can be called with GFP flags which
do not allow blocking. Before recent Ulad's change called "kmsan:
remove hard-coded GFP_KERNEL flags" this function was called only with
GPF_KERNEL but now that's not the case. I think you can move
introduction of can_resched flag into this patch.

> +               kmsan_leave_runtime();
>                 err = mapped;
>                 goto ret;
>         }
> --
> 2.25.1
>

Re: [PATCH v4 2/9] mm/kmsan: skip vunmap_range_noflush when KMSAN is disabled

[Suren Baghdasaryan]

On Tue, Nov 10, 2026 at 11:24 AM Hao Ge <hao.ge@linux.dev> wrote:
>
> kmsan_vmap_pages_range_noflush() returns 0 when kmsan_enabled is
> false.  The unmap side does not, so it still walks the shadow and
> origin page tables even though KMSAN never installed any metadata
> there.

Note that KMSAN_WARN_ON() can reset kmsan_enabled flag, so the fact
that it's false does not mean that metadata was never installed. I
think a proper bailout should depend on CONFIG_KMSAN, not on
kmsan_enabled.

>
> Add the matching early return.
>
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Reviewed-by: Dev Jain <dev.jain@arm.com>
> Signed-off-by: Hao Ge <hao.ge@linux.dev>
> ---
>  mm/kmsan/hooks.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c
> index a706666db097..107f66841274 100644
> --- a/mm/kmsan/hooks.c
> +++ b/mm/kmsan/hooks.c
> @@ -138,6 +138,9 @@ static unsigned long vmalloc_origin(unsigned long addr)
>
>  void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end)
>  {
> +       if (!kmsan_enabled)
> +               return;
> +
>         __vunmap_range_noflush(vmalloc_shadow(start), vmalloc_shadow(end));
>         __vunmap_range_noflush(vmalloc_origin(start), vmalloc_origin(end));
>         flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end));
> --
> 2.25.1
>

Re: [PATCH v4 3/9] mm/vmalloc: BUG_ON() on an unaligned range in vmap_range_noflush()

[Suren Baghdasaryan]

On Tue, Nov 10, 2026 at 12:26 PM Hao Ge <hao.ge@linux.dev> wrote:
>
>
>
> On 2026/11/10 19:25, Hao Ge wrote:
> > commit a739e6b557af ("mm: vmalloc: WARN_ON if mapping size is not
> > PAGE_SIZE aligned") put the check in vmap_pte_range(), which is too
> > late, because the upper levels may already have installed page
> > tables for a range that is rejected anyway.
> >
> > The pte loop maps whole pages and stops only when addr lands
> > exactly on end.  An unaligned end lets addr jump over it and the
> > loop keeps mapping past the range.  An unaligned start maps the
> > page below the range and usually misses end too.  Such a range is
> > never a legitimate request, and the corruption it leaves behind is
> > hard to attribute to the caller afterwards, so bug out on addr and
> > end at the entry of vmap_range_noflush() and drop the check from
> > vmap_pte_range().
> >
> > Signed-off-by: Hao Ge <hao.ge@linux.dev>
> > ---
> >  mm/vmalloc.c | 4 +---
> >  1 file changed, 1 insertion(+), 3 deletions(-)
> >
> > diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> > index 093b8d5f7b0e..27f2674a5556 100644
> > --- a/mm/vmalloc.c
> > +++ b/mm/vmalloc.c
> > @@ -125,9 +125,6 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
> >       unsigned long size;
> >       unsigned int nr_pages;
> >
> > -     if (WARN_ON_ONCE(!PAGE_ALIGNED(end - addr)))
> > -             return -EINVAL;
> > -
> >       pfn = phys_addr >> PAGE_SHIFT;
> >       pte = pte_alloc_kernel_track(pmd, addr, mask);
> >       if (!pte)
> > @@ -347,6 +344,7 @@ static int vmap_range_noflush(unsigned long addr, unsigned long end,
> >        */
> >       might_sleep();
> >       BUG_ON(addr >= end);
> > +     BUG_ON(!PAGE_ALIGNED(addr) || !PAGE_ALIGNED(end));
>
> I know BUG_ON() is ugly and it was not my first choice either, but I
> have not figured out a better way to handle __vunmap_range_noflush().
>
> If an unaligned range is passed to it, the pte loop may clear PTEs
> adjacent to the range. Say a caller passes [3, 12) while [0, 3)
> belongs to somebody else. The loop operates on whole pages, so it
> starts at the page containing 3 and clears [0, 3) with it. That
> silently corrupts a live mapping, and the corruption is not detected
> until the owner touches the page. Bugs like that are nearly
> impossible to debug afterwards.
>
> It is not feasible for __vunmap_range_noflush() to just return an
> invalid-argument error upfront. Its job is to clear PTEs, and an
> early return leaves the residual PTEs behind.
>
> I also thought about warning and returning -EINVAL on the map side
> and keeping BUG_ON() only in __vunmap_range_noflush(), but the
> rollback kills that idea.
>
> int __vmap_pages_range_noflush(unsigned long addr, unsigned long end,
>                 pgprot_t prot, struct page **pages, unsigned int page_shift,
>                 gfp_t gfp_mask)
> {
>         int err;
>
>         if (WARN_ON_ONCE(page_shift < PAGE_SHIFT))
>                 return -EINVAL;
>
>         if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC))
>                 page_shift = PAGE_SHIFT;
>
>         err = vmap_pages_range_noflush_walk(addr, end, prot, pages, page_shift);
>
>         /* Undo the PTEs installed before the failure. */
>         if (err)
>                 __vunmap_range_noflush(addr, end,
>                                        gfpflags_allow_blocking(gfp_mask));
>
>         return err;
> }
>
> If the walk WARNs and returns -EINVAL for a bad range, the undo
> above passes that same range to __vunmap_range_noflush(), and we
> BUG right after the WARN. vmap_page_range() runs into the same
> thing on its error path.
>
> So the map side cannot warn while the unmap side BUGs, the rollback
> drags the rejected range into the unmap anyway. And the unmap side
> cannot warn and skip either, that leaves the stale PTEs.
>
> Also, these checks are the last line of defense. All in-tree callers
> pass page aligned, non-empty ranges, and I am not aware of anyone ever
> hitting this line. So for correct code the BUG_ON never fires at all,
> no user will ever see it. The only one who can trip it is the developer
> who just passed the bad range, and for him a stack trace right at the
> spot is much better than chasing a silent corruption afterwards.
>
> I settled on BUG_ON here. It may not be an ideal solution, but if anyone
> has a better idea or alternative approach, please let me know.
> I would greatly appreciate it.

I don't see a better way to handle this situation but I would add a
WARN_ON_ONCE() alignment check at the beginning of vmap_page_range()
too because it is used in some of the exported functions like
ioremap_page() and pci_remap_iospace(). If someone passes an unaliged
address to these functions your change would turn a warning+function
failure into a panic.

>
> Thanks
> Best Regards
> Hao
>
> >
> >       start = addr;
> >       pgd = pgd_offset_k(addr);

Re: [PATCH v4 4/9] mm/vmalloc: BUG_ON() on an unaligned range in vmap_pages_range_noflush_walk()

[Suren Baghdasaryan]

On Tue, Nov 10, 2026 at 11:24 AM Hao Ge <hao.ge@linux.dev> wrote:
>
> The pte loop of vmap_pages_pte_range() maps whole pages and stops
> only when addr lands exactly on end.  An unaligned end lets addr
> jump over it and the loop keeps mapping into whatever follows the
> range.  An unaligned start maps the page below the range with its
> first pte.  That corruption is impossible to debug afterwards, so
> bug out on addr and end at the entry of
> vmap_pages_range_noflush_walk().
>
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Signed-off-by: Hao Ge <hao.ge@linux.dev>
> ---
>  mm/vmalloc.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 27f2674a5556..86fbb1d2b27a 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -698,6 +698,8 @@ static int vmap_pages_range_noflush_walk(unsigned long addr, unsigned long end,
>         pgtbl_mod_mask mask = 0;
>
>         BUG_ON(addr >= end);
> +       BUG_ON(!PAGE_ALIGNED(addr) || !PAGE_ALIGNED(end));

Instead of BUG_ON() here can we error out early in this function's
callers (with or without a warning)? This is a static function with
only 2 callers. vmap_pages_range_batched() uses it in a loop but I
believe if we check the original address alignment then consequent
calls are guaranteed to have aligned addresses.

> +
>         pgd = pgd_offset_k(addr);
>         do {
>                 next = pgd_addr_end(addr, end);
> --
> 2.25.1
>

Re: [PATCH v4 5/9] mm/vmalloc: BUG_ON() on an unaligned range in __vunmap_range_noflush()

[Suren Baghdasaryan]

On Tue, Nov 10, 2026 at 11:25 AM Hao Ge <hao.ge@linux.dev> wrote:
>
> The PTE loop of the unmap path stops only at addr == end.  With an
> unaligned end addr never lands on end, so the loop walks past the
> range and keeps clearing.  With an unaligned start the loop begins
> in the page containing start, which extends below the range.  Either
> way the unmap clears the mappings adjacent to the range, and that is
> nearly impossible to debug afterwards.
>
> vmap only installs page-aligned, page-granular mappings, so an
> unaligned unmap request is never a legitimate teardown, and there is
> no way to tell which pages the caller meant.  Check addr and end at
> the entry of __vunmap_range_noflush() and bug out before any PTE is
> cleared.
>
> Signed-off-by: Hao Ge <hao.ge@linux.dev>
> ---
>  mm/vmalloc.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 86fbb1d2b27a..0565cbfe325c 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -518,6 +518,8 @@ void __vunmap_range_noflush(unsigned long start, unsigned long end)
>         pgtbl_mod_mask mask = 0;
>
>         BUG_ON(addr >= end);
> +       BUG_ON(!PAGE_ALIGNED(addr) || !PAGE_ALIGNED(end));

Again, not pretty but I don't see a better way that does not involve
adding early exit checks to all its users.

> +
>         pgd = pgd_offset_k(addr);
>         do {
>                 next = pgd_addr_end(addr, end);
> --
> 2.25.1
>

Re: [PATCH v4 6/9] mm/vmalloc: only flush the cache after the vmap succeeded

[Suren Baghdasaryan]

On Tue, Nov 10, 2026 at 11:25 AM Hao Ge <hao.ge@linux.dev> wrote:
>
> vmap_page_range() and __vmap_pages_range() call flush_cache_vmap()

New vmap_pages_range_batched() function has the same issue, so needs fixing too.

> before they even look at the error code. A failed mapping is never
> used, so the flush is pointless at best.
>
> According to the Sashiko review it is worse than pointless on UML:
> flush_cache_vmap() there is flush_tlb_kernel_range(), which syncs the
> PTEs into the host OS. Flushing a half-installed mapping therefore
> leaves host mappings behind for a range that was just freed again.
>
> Only flush when the mapping, and for vmap_page_range() also the KMSAN
> metadata, is fully set up.
>
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Signed-off-by: Hao Ge <hao.ge@linux.dev>
> ---
>  mm/vmalloc.c | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 0565cbfe325c..3f98056f8c7e 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -369,10 +369,12 @@ int vmap_page_range(unsigned long addr, unsigned long end,
>
>         err = vmap_range_noflush(addr, end, phys_addr, pgprot_nx(prot),
>                                  ioremap_max_page_shift);
> -       flush_cache_vmap(addr, end);
>         if (!err)
>                 err = kmsan_ioremap_page_range(addr, end, phys_addr, prot,
>                                                ioremap_max_page_shift);
> +       if (!err)
> +               flush_cache_vmap(addr, end);
> +
>         return err;
>  }
>
> @@ -758,7 +760,9 @@ static int __vmap_pages_range(unsigned long addr, unsigned long end,
>         int err;
>
>         err = vmap_pages_range_noflush(addr, end, prot, pages, page_shift, gfp_mask);
> -       flush_cache_vmap(addr, end);
> +       if (!err)
> +               flush_cache_vmap(addr, end);
> +
>         return err;
>  }
>
> --
> 2.25.1
>

Re: [PATCH v4 7/9] mm/vmalloc: undo partial mappings inside the mapping functions

[Suren Baghdasaryan]

On Tue, Nov 10, 2026 at 11:25 AM Hao Ge <hao.ge@linux.dev> wrote:
>
> The vmap entry points may install some PTEs and then fail, leaving a
> partial mapping behind.
>
> The callers do not agree on who cleans it up. pcpu_map_pages() and
> kmsan_ioremap_page_range() roll back what they mapped, while
> vm_module_tags_populate() and the __GFP_NOFAIL retry loop in
> __vmalloc_area_node() expect a clean range from the mapping function.
> For them the leftovers stay mapped, and the next attempt to map the
> same range trips the WARN_ON(!pte_none()) in vmap_pages_pte_range().
> A leftover huge pmd is even worse. During the retry, vmap_try_huge_pmd()
> calls pmd_free_pte_page() on it, and the x86 implementation cannot
> tell a leaf entry from a page table and frees the mapped page.
>
> After discussing with Suren and Ulad, we decided to put the rollback
> into the entry points of the vmap API rather than into every
> low-level helper. __vmap_pages_range_noflush() undoes the whole range
> it was asked to map when it fails, and vmap_page_range() undoes its
> range for the ioremap-style mappings.
>
> The rollback is __vunmap_range_noflush(), which only clears the PTEs.
> No TLB flush is needed, nothing has touched the mapping yet.
> vmap_pages_range_noflush() also drops the KMSAN metadata of the
> failed mapping.
>
> The undo walk skips its cond_resched() when the caller's gfp mask
> cannot block. __vmalloc() with GFP_ATOMIC or GFP_NOWAIT can run in
> atomic context, where cond_resched() cannot schedule.
> __vunmap_range_noflush() takes a can_resched argument for that, which
> the sleepable callers pass as true.
>
> Suggested-by: Uladyslau Rezki <urezki@gmail.com>
> Signed-off-by: Hao Ge <hao.ge@linux.dev>
> ---
>  include/linux/kmsan.h |  8 ++++--
>  mm/kmsan/hooks.c      | 18 ++++++++-----
>  mm/kmsan/shadow.c     |  7 ++---
>  mm/vmalloc.c          | 61 +++++++++++++++++++++++++++++++++----------
>  mm/vmalloc.h          |  6 +++--
>  5 files changed, 72 insertions(+), 28 deletions(-)
>
> diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h
> index 7da9fd506b39..0016c032d1e7 100644
> --- a/include/linux/kmsan.h
> +++ b/include/linux/kmsan.h
> @@ -150,11 +150,14 @@ int __must_check kmsan_vmap_pages_range_noflush(unsigned long start,
>   * kmsan_vunmap_kernel_range_noflush() - Notify KMSAN about a vunmap.
>   * @start: start of vunmapped range.
>   * @end:   end of vunmapped range.
> + * @can_resched: whether the unmap walk may call cond_resched(); pass
> + * false from atomic context.
>   *
>   * KMSAN unmaps the contiguous metadata ranges created by
>   * kmsan_map_kernel_range_noflush().
>   */
> -void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end);
> +void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end,
> +                               bool can_resched);
>
>  /**
>   * kmsan_ioremap_page_range() - Notify KMSAN about a ioremap_page_range() call.
> @@ -355,7 +358,8 @@ static inline int __must_check kmsan_vmap_pages_range_noflush(
>  }
>
>  static inline void kmsan_vunmap_range_noflush(unsigned long start,
> -                                             unsigned long end)
> +                                             unsigned long end,
> +                                             bool can_resched)
>  {
>  }
>
> diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c
> index 107f66841274..0cd1edaa1120 100644
> --- a/mm/kmsan/hooks.c
> +++ b/mm/kmsan/hooks.c
> @@ -136,13 +136,16 @@ static unsigned long vmalloc_origin(unsigned long addr)
>                                                  KMSAN_META_ORIGIN);
>  }
>
> -void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end)
> +void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end,
> +                               bool can_resched)
>  {
>         if (!kmsan_enabled)
>                 return;
>
> -       __vunmap_range_noflush(vmalloc_shadow(start), vmalloc_shadow(end));
> -       __vunmap_range_noflush(vmalloc_origin(start), vmalloc_origin(end));
> +       __vunmap_range_noflush(vmalloc_shadow(start), vmalloc_shadow(end),
> +                              can_resched);
> +       __vunmap_range_noflush(vmalloc_origin(start), vmalloc_origin(end),
> +                              can_resched);
>         flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end));
>         flush_cache_vmap(vmalloc_origin(start), vmalloc_origin(end));
>  }
> @@ -171,8 +174,9 @@ static void kmsan_iounmap_pages(unsigned long start, unsigned long end)
>                 if (origin)
>                         __free_pages(origin, KMSAN_IOREMAP_META_ORDER);
>         }
> -       __vunmap_range_noflush(shadow_start, shadow_end);
> -       __vunmap_range_noflush(origin_start, origin_end);
> +       /* ioremap metadata mappings are always GFP_KERNEL, so resched is safe. */
> +       __vunmap_range_noflush(shadow_start, shadow_end, true);
> +       __vunmap_range_noflush(origin_start, origin_end, true);
>         flush_tlb_kernel_range(shadow_start, shadow_end);
>         flush_tlb_kernel_range(origin_start, origin_end);
>  }
> @@ -207,7 +211,7 @@ int kmsan_ioremap_page_range(unsigned long start, unsigned long end,
>                 mapped = __vmap_pages_range_noflush(
>                         vmalloc_shadow(start + off),
>                         vmalloc_shadow(start + off + PAGE_SIZE), prot, &shadow,
> -                       PAGE_SHIFT);
> +                       PAGE_SHIFT, gfp_mask);
>                 if (mapped) {
>                         err = mapped;
>                         goto ret;
> @@ -217,7 +221,7 @@ int kmsan_ioremap_page_range(unsigned long start, unsigned long end,
>                 mapped = __vmap_pages_range_noflush(
>                         vmalloc_origin(start + off),
>                         vmalloc_origin(start + off + PAGE_SIZE), prot, &origin,
> -                       PAGE_SHIFT);
> +                       PAGE_SHIFT, gfp_mask);
>                 if (mapped) {
>                         err = mapped;
>                         goto ret;
> diff --git a/mm/kmsan/shadow.c b/mm/kmsan/shadow.c
> index 2166086d3dc3..452760a333e4 100644
> --- a/mm/kmsan/shadow.c
> +++ b/mm/kmsan/shadow.c
> @@ -247,7 +247,7 @@ int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
>         origin_end = vmalloc_meta((void *)end, KMSAN_META_ORIGIN);
>         kmsan_enter_runtime();
>         mapped = __vmap_pages_range_noflush(shadow_start, shadow_end, prot,
> -                                           s_pages, page_shift);
> +                                           s_pages, page_shift, gfp_mask);
>         kmsan_leave_runtime();
>         if (mapped) {
>                 err = mapped;
> @@ -255,12 +255,13 @@ int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
>         }
>         kmsan_enter_runtime();
>         mapped = __vmap_pages_range_noflush(origin_start, origin_end, prot,
> -                                           o_pages, page_shift);
> +                                           o_pages, page_shift, gfp_mask);
>         kmsan_leave_runtime();
>         if (mapped) {
>                 /* Undo the shadow mapping set up above. */
>                 kmsan_enter_runtime();
> -               __vunmap_range_noflush(shadow_start, shadow_end);
> +               __vunmap_range_noflush(shadow_start, shadow_end,
> +                                      gfpflags_allow_blocking(gfp_mask));
>                 kmsan_leave_runtime();
>                 err = mapped;
>                 goto ret;
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 3f98056f8c7e..d1986f68eab4 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -372,8 +372,12 @@ int vmap_page_range(unsigned long addr, unsigned long end,
>         if (!err)
>                 err = kmsan_ioremap_page_range(addr, end, phys_addr, prot,
>                                                ioremap_max_page_shift);
> -       if (!err)
> +       if (err) {
> +               /* Callers are ioremap and arch setup code, always sleepable. */
> +               __vunmap_range_noflush(addr, end, true);
> +       } else {
>                 flush_cache_vmap(addr, end);
> +       }
>
>         return err;
>  }
> @@ -429,7 +433,7 @@ static void vunmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
>  }
>
>  static void vunmap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end,
> -                            pgtbl_mod_mask *mask)
> +                            pgtbl_mod_mask *mask, bool can_resched)
>  {
>         pmd_t *pmd;
>         unsigned long next;
> @@ -451,12 +455,14 @@ static void vunmap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end,
>                         continue;
>                 vunmap_pte_range(pmd, addr, next, mask);
>
> -               cond_resched();
> +               /* The undo path may run in atomic context. */
> +               if (can_resched)
> +                       cond_resched();
>         } while (pmd++, addr = next, addr != end);
>  }
>
>  static void vunmap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end,
> -                            pgtbl_mod_mask *mask)
> +                            pgtbl_mod_mask *mask, bool can_resched)
>  {
>         pud_t *pud;
>         unsigned long next;
> @@ -476,12 +482,12 @@ static void vunmap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end,
>                 }
>                 if (pud_none_or_clear_bad(pud))
>                         continue;
> -               vunmap_pmd_range(pud, addr, next, mask);
> +               vunmap_pmd_range(pud, addr, next, mask, can_resched);
>         } while (pud++, addr = next, addr != end);
>  }
>
>  static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end,
> -                            pgtbl_mod_mask *mask)
> +                            pgtbl_mod_mask *mask, bool can_resched)
>  {
>         p4d_t *p4d;
>         unsigned long next;
> @@ -496,7 +502,7 @@ static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end,
>
>                 if (p4d_none_or_clear_bad(p4d))
>                         continue;
> -               vunmap_pud_range(p4d, addr, next, mask);
> +               vunmap_pud_range(p4d, addr, next, mask, can_resched);
>         } while (p4d++, addr = next, addr != end);
>  }
>
> @@ -512,7 +518,8 @@ static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end,
>   *
>   * This is an internal function only. Do not use outside mm/.
>   */
> -void __vunmap_range_noflush(unsigned long start, unsigned long end)
> +void __vunmap_range_noflush(unsigned long start, unsigned long end,
> +                           bool can_resched)
>  {
>         unsigned long next;
>         pgd_t *pgd;
> @@ -529,7 +536,7 @@ void __vunmap_range_noflush(unsigned long start, unsigned long end)
>                         mask |= PGTBL_PGD_MODIFIED;
>                 if (pgd_none_or_clear_bad(pgd))
>                         continue;
> -               vunmap_p4d_range(pgd, addr, next, &mask);
> +               vunmap_p4d_range(pgd, addr, next, &mask, can_resched);
>         } while (pgd++, addr = next, addr != end);
>
>         if (mask & ARCH_PAGE_TABLE_SYNC_MASK)
> @@ -538,8 +545,9 @@ void __vunmap_range_noflush(unsigned long start, unsigned long end)
>
>  void vunmap_range_noflush(unsigned long start, unsigned long end)
>  {
> -       kmsan_vunmap_range_noflush(start, end);
> -       __vunmap_range_noflush(start, end);
> +       /* The unmap paths are sleepable, keep the resched point. */
> +       kmsan_vunmap_range_noflush(start, end, true);
> +       __vunmap_range_noflush(start, end, true);
>  }
>
>  /**
> @@ -727,18 +735,32 @@ static int vmap_pages_range_noflush_walk(unsigned long addr, unsigned long end,
>   * The caller is responsible for calling flush_cache_vmap() after this
>   * function returns successfully and before the addresses are accessed.
>   *
> + * On failure the mapping is torn down before returning, so a retry
> + * sees a clean range. When @gfp_mask cannot block, the teardown
> + * skips cond_resched(), it may run in atomic context.
> + *
>   * This is an internal function only. Do not use outside mm/.
>   */
>  int __vmap_pages_range_noflush(unsigned long addr, unsigned long end,
> -               pgprot_t prot, struct page **pages, unsigned int page_shift)
> +               pgprot_t prot, struct page **pages, unsigned int page_shift,
> +               gfp_t gfp_mask)
>  {
> +       int err;
> +
>         if (WARN_ON_ONCE(page_shift < PAGE_SHIFT))
>                 return -EINVAL;
>
>         if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC))
>                 page_shift = PAGE_SHIFT;
>
> -       return vmap_pages_range_noflush_walk(addr, end, prot, pages, page_shift);
> +       err = vmap_pages_range_noflush_walk(addr, end, prot, pages, page_shift);
> +
> +       /* Undo the PTEs installed before the failure. */
> +       if (err)
> +               __vunmap_range_noflush(addr, end,
> +                                      gfpflags_allow_blocking(gfp_mask));

I remember we discussed this before that __vunmap_range_noflush() will
be unmapping the entire range between addr and end instead of the
range vmap_pages_range_noflush_walk() actially mapped and your reply
was that this keeps things simple and avoids adding extra code in the
common path when we do not fail. This was bothering me, so I looked a
bit deeper and I think there is an issue with this approach.
vmap_pages_pte_range() fails with -EBUSY if it encounters an already
installed PTE. So, if this happens, we immediately bail out of mapping
the range and will be unmapping the entire range, including these
existing PTEs. I don't think that's ok.

A fix might be quite simple. vmap_pages_range_noflush_walk() already
counts how many pages we processed in nr variable. If we propagate
that one level up then we could unmap only the range we actually
mapped.

> +
> +       return err;
>  }
>
>  int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
> @@ -750,7 +772,18 @@ int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
>
>         if (ret)
>                 return ret;
> -       return __vmap_pages_range_noflush(addr, end, prot, pages, page_shift);
> +
> +       ret = __vmap_pages_range_noflush(addr, end, prot, pages, page_shift,
> +                                        gfp_mask);
> +       /*
> +        * The page tables undo themselves on failure. Tear down the
> +        * metadata that was fully set up before the mapping failed.
> +        */
> +       if (ret)
> +               kmsan_vunmap_range_noflush(addr, end,
> +                                          gfpflags_allow_blocking(gfp_mask));
> +
> +       return ret;
>  }
>
>  static int __vmap_pages_range(unsigned long addr, unsigned long end,
> diff --git a/mm/vmalloc.h b/mm/vmalloc.h
> index d8372a4f65c4..9ea28845f713 100644
> --- a/mm/vmalloc.h
> +++ b/mm/vmalloc.h
> @@ -40,10 +40,12 @@ void clear_vm_uninitialized_flag(struct vm_struct *vm);
>
>  int __must_check __vmap_pages_range_noflush(unsigned long addr,
>                 unsigned long end, pgprot_t prot,
> -               struct page **pages, unsigned int page_shift);
> +               struct page **pages, unsigned int page_shift,
> +               gfp_t gfp_mask);
>
>  void vunmap_range_noflush(unsigned long start, unsigned long end);
>
> -void __vunmap_range_noflush(unsigned long start, unsigned long end);
> +void __vunmap_range_noflush(unsigned long start, unsigned long end,
> +                           bool can_resched);
>
>  #endif /* __MM_VMALLOC_H */
> --
> 2.25.1
>

Re: [PATCH v4 7/9] mm/vmalloc: undo partial mappings inside the mapping functions

[Hao Ge]

Hi Suren

Thanks for your review.

On 2026/11/12 09:15, Suren Baghdasaryan wrote:
> On Tue, Nov 10, 2026 at 11:25 AM Hao Ge <hao.ge@linux.dev> wrote:
>>
>> The vmap entry points may install some PTEs and then fail, leaving a
>> partial mapping behind.
>>
>> The callers do not agree on who cleans it up. pcpu_map_pages() and
>> kmsan_ioremap_page_range() roll back what they mapped, while
>> vm_module_tags_populate() and the __GFP_NOFAIL retry loop in
>> __vmalloc_area_node() expect a clean range from the mapping function.
>> For them the leftovers stay mapped, and the next attempt to map the
>> same range trips the WARN_ON(!pte_none()) in vmap_pages_pte_range().
>> A leftover huge pmd is even worse. During the retry, vmap_try_huge_pmd()
>> calls pmd_free_pte_page() on it, and the x86 implementation cannot
>> tell a leaf entry from a page table and frees the mapped page.
>>
>> After discussing with Suren and Ulad, we decided to put the rollback
>> into the entry points of the vmap API rather than into every
>> low-level helper. __vmap_pages_range_noflush() undoes the whole range
>> it was asked to map when it fails, and vmap_page_range() undoes its
>> range for the ioremap-style mappings.
>>
>> The rollback is __vunmap_range_noflush(), which only clears the PTEs.
>> No TLB flush is needed, nothing has touched the mapping yet.
>> vmap_pages_range_noflush() also drops the KMSAN metadata of the
>> failed mapping.
>>
>> The undo walk skips its cond_resched() when the caller's gfp mask
>> cannot block. __vmalloc() with GFP_ATOMIC or GFP_NOWAIT can run in
>> atomic context, where cond_resched() cannot schedule.
>> __vunmap_range_noflush() takes a can_resched argument for that, which
>> the sleepable callers pass as true.
>>
>> Suggested-by: Uladyslau Rezki <urezki@gmail.com>
>> Signed-off-by: Hao Ge <hao.ge@linux.dev>
>> ---
>>  include/linux/kmsan.h |  8 ++++--
>>  mm/kmsan/hooks.c      | 18 ++++++++-----
>>  mm/kmsan/shadow.c     |  7 ++---
>>  mm/vmalloc.c          | 61 +++++++++++++++++++++++++++++++++----------
>>  mm/vmalloc.h          |  6 +++--
>>  5 files changed, 72 insertions(+), 28 deletions(-)
>>
>> diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h
>> index 7da9fd506b39..0016c032d1e7 100644
>> --- a/include/linux/kmsan.h
>> +++ b/include/linux/kmsan.h
>> @@ -150,11 +150,14 @@ int __must_check kmsan_vmap_pages_range_noflush(unsigned long start,
>>   * kmsan_vunmap_kernel_range_noflush() - Notify KMSAN about a vunmap.
>>   * @start: start of vunmapped range.
>>   * @end:   end of vunmapped range.
>> + * @can_resched: whether the unmap walk may call cond_resched(); pass
>> + * false from atomic context.
>>   *
>>   * KMSAN unmaps the contiguous metadata ranges created by
>>   * kmsan_map_kernel_range_noflush().
>>   */
>> -void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end);
>> +void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end,
>> +                               bool can_resched);
>>
>>  /**
>>   * kmsan_ioremap_page_range() - Notify KMSAN about a ioremap_page_range() call.
>> @@ -355,7 +358,8 @@ static inline int __must_check kmsan_vmap_pages_range_noflush(
>>  }
>>
>>  static inline void kmsan_vunmap_range_noflush(unsigned long start,
>> -                                             unsigned long end)
>> +                                             unsigned long end,
>> +                                             bool can_resched)
>>  {
>>  }
>>
>> diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c
>> index 107f66841274..0cd1edaa1120 100644
>> --- a/mm/kmsan/hooks.c
>> +++ b/mm/kmsan/hooks.c
>> @@ -136,13 +136,16 @@ static unsigned long vmalloc_origin(unsigned long addr)
>>                                                  KMSAN_META_ORIGIN);
>>  }
>>
>> -void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end)
>> +void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end,
>> +                               bool can_resched)
>>  {
>>         if (!kmsan_enabled)
>>                 return;
>>
>> -       __vunmap_range_noflush(vmalloc_shadow(start), vmalloc_shadow(end));
>> -       __vunmap_range_noflush(vmalloc_origin(start), vmalloc_origin(end));
>> +       __vunmap_range_noflush(vmalloc_shadow(start), vmalloc_shadow(end),
>> +                              can_resched);
>> +       __vunmap_range_noflush(vmalloc_origin(start), vmalloc_origin(end),
>> +                              can_resched);
>>         flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end));
>>         flush_cache_vmap(vmalloc_origin(start), vmalloc_origin(end));
>>  }
>> @@ -171,8 +174,9 @@ static void kmsan_iounmap_pages(unsigned long start, unsigned long end)
>>                 if (origin)
>>                         __free_pages(origin, KMSAN_IOREMAP_META_ORDER);
>>         }
>> -       __vunmap_range_noflush(shadow_start, shadow_end);
>> -       __vunmap_range_noflush(origin_start, origin_end);
>> +       /* ioremap metadata mappings are always GFP_KERNEL, so resched is safe. */
>> +       __vunmap_range_noflush(shadow_start, shadow_end, true);
>> +       __vunmap_range_noflush(origin_start, origin_end, true);
>>         flush_tlb_kernel_range(shadow_start, shadow_end);
>>         flush_tlb_kernel_range(origin_start, origin_end);
>>  }
>> @@ -207,7 +211,7 @@ int kmsan_ioremap_page_range(unsigned long start, unsigned long end,
>>                 mapped = __vmap_pages_range_noflush(
>>                         vmalloc_shadow(start + off),
>>                         vmalloc_shadow(start + off + PAGE_SIZE), prot, &shadow,
>> -                       PAGE_SHIFT);
>> +                       PAGE_SHIFT, gfp_mask);
>>                 if (mapped) {
>>                         err = mapped;
>>                         goto ret;
>> @@ -217,7 +221,7 @@ int kmsan_ioremap_page_range(unsigned long start, unsigned long end,
>>                 mapped = __vmap_pages_range_noflush(
>>                         vmalloc_origin(start + off),
>>                         vmalloc_origin(start + off + PAGE_SIZE), prot, &origin,
>> -                       PAGE_SHIFT);
>> +                       PAGE_SHIFT, gfp_mask);
>>                 if (mapped) {
>>                         err = mapped;
>>                         goto ret;
>> diff --git a/mm/kmsan/shadow.c b/mm/kmsan/shadow.c
>> index 2166086d3dc3..452760a333e4 100644
>> --- a/mm/kmsan/shadow.c
>> +++ b/mm/kmsan/shadow.c
>> @@ -247,7 +247,7 @@ int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
>>         origin_end = vmalloc_meta((void *)end, KMSAN_META_ORIGIN);
>>         kmsan_enter_runtime();
>>         mapped = __vmap_pages_range_noflush(shadow_start, shadow_end, prot,
>> -                                           s_pages, page_shift);
>> +                                           s_pages, page_shift, gfp_mask);
>>         kmsan_leave_runtime();
>>         if (mapped) {
>>                 err = mapped;
>> @@ -255,12 +255,13 @@ int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
>>         }
>>         kmsan_enter_runtime();
>>         mapped = __vmap_pages_range_noflush(origin_start, origin_end, prot,
>> -                                           o_pages, page_shift);
>> +                                           o_pages, page_shift, gfp_mask);
>>         kmsan_leave_runtime();
>>         if (mapped) {
>>                 /* Undo the shadow mapping set up above. */
>>                 kmsan_enter_runtime();
>> -               __vunmap_range_noflush(shadow_start, shadow_end);
>> +               __vunmap_range_noflush(shadow_start, shadow_end,
>> +                                      gfpflags_allow_blocking(gfp_mask));
>>                 kmsan_leave_runtime();
>>                 err = mapped;
>>                 goto ret;
>> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
>> index 3f98056f8c7e..d1986f68eab4 100644
>> --- a/mm/vmalloc.c
>> +++ b/mm/vmalloc.c
>> @@ -372,8 +372,12 @@ int vmap_page_range(unsigned long addr, unsigned long end,
>>         if (!err)
>>                 err = kmsan_ioremap_page_range(addr, end, phys_addr, prot,
>>                                                ioremap_max_page_shift);
>> -       if (!err)
>> +       if (err) {
>> +               /* Callers are ioremap and arch setup code, always sleepable. */
>> +               __vunmap_range_noflush(addr, end, true);
>> +       } else {
>>                 flush_cache_vmap(addr, end);
>> +       }
>>
>>         return err;
>>  }
>> @@ -429,7 +433,7 @@ static void vunmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
>>  }
>>
>>  static void vunmap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end,
>> -                            pgtbl_mod_mask *mask)
>> +                            pgtbl_mod_mask *mask, bool can_resched)
>>  {
>>         pmd_t *pmd;
>>         unsigned long next;
>> @@ -451,12 +455,14 @@ static void vunmap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end,
>>                         continue;
>>                 vunmap_pte_range(pmd, addr, next, mask);
>>
>> -               cond_resched();
>> +               /* The undo path may run in atomic context. */
>> +               if (can_resched)
>> +                       cond_resched();
>>         } while (pmd++, addr = next, addr != end);
>>  }
>>
>>  static void vunmap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end,
>> -                            pgtbl_mod_mask *mask)
>> +                            pgtbl_mod_mask *mask, bool can_resched)
>>  {
>>         pud_t *pud;
>>         unsigned long next;
>> @@ -476,12 +482,12 @@ static void vunmap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end,
>>                 }
>>                 if (pud_none_or_clear_bad(pud))
>>                         continue;
>> -               vunmap_pmd_range(pud, addr, next, mask);
>> +               vunmap_pmd_range(pud, addr, next, mask, can_resched);
>>         } while (pud++, addr = next, addr != end);
>>  }
>>
>>  static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end,
>> -                            pgtbl_mod_mask *mask)
>> +                            pgtbl_mod_mask *mask, bool can_resched)
>>  {
>>         p4d_t *p4d;
>>         unsigned long next;
>> @@ -496,7 +502,7 @@ static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end,
>>
>>                 if (p4d_none_or_clear_bad(p4d))
>>                         continue;
>> -               vunmap_pud_range(p4d, addr, next, mask);
>> +               vunmap_pud_range(p4d, addr, next, mask, can_resched);
>>         } while (p4d++, addr = next, addr != end);
>>  }
>>
>> @@ -512,7 +518,8 @@ static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end,
>>   *
>>   * This is an internal function only. Do not use outside mm/.
>>   */
>> -void __vunmap_range_noflush(unsigned long start, unsigned long end)
>> +void __vunmap_range_noflush(unsigned long start, unsigned long end,
>> +                           bool can_resched)
>>  {
>>         unsigned long next;
>>         pgd_t *pgd;
>> @@ -529,7 +536,7 @@ void __vunmap_range_noflush(unsigned long start, unsigned long end)
>>                         mask |= PGTBL_PGD_MODIFIED;
>>                 if (pgd_none_or_clear_bad(pgd))
>>                         continue;
>> -               vunmap_p4d_range(pgd, addr, next, &mask);
>> +               vunmap_p4d_range(pgd, addr, next, &mask, can_resched);
>>         } while (pgd++, addr = next, addr != end);
>>
>>         if (mask & ARCH_PAGE_TABLE_SYNC_MASK)
>> @@ -538,8 +545,9 @@ void __vunmap_range_noflush(unsigned long start, unsigned long end)
>>
>>  void vunmap_range_noflush(unsigned long start, unsigned long end)
>>  {
>> -       kmsan_vunmap_range_noflush(start, end);
>> -       __vunmap_range_noflush(start, end);
>> +       /* The unmap paths are sleepable, keep the resched point. */
>> +       kmsan_vunmap_range_noflush(start, end, true);
>> +       __vunmap_range_noflush(start, end, true);
>>  }
>>
>>  /**
>> @@ -727,18 +735,32 @@ static int vmap_pages_range_noflush_walk(unsigned long addr, unsigned long end,
>>   * The caller is responsible for calling flush_cache_vmap() after this
>>   * function returns successfully and before the addresses are accessed.
>>   *
>> + * On failure the mapping is torn down before returning, so a retry
>> + * sees a clean range. When @gfp_mask cannot block, the teardown
>> + * skips cond_resched(), it may run in atomic context.
>> + *
>>   * This is an internal function only. Do not use outside mm/.
>>   */
>>  int __vmap_pages_range_noflush(unsigned long addr, unsigned long end,
>> -               pgprot_t prot, struct page **pages, unsigned int page_shift)
>> +               pgprot_t prot, struct page **pages, unsigned int page_shift,
>> +               gfp_t gfp_mask)
>>  {
>> +       int err;
>> +
>>         if (WARN_ON_ONCE(page_shift < PAGE_SHIFT))
>>                 return -EINVAL;
>>
>>         if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC))
>>                 page_shift = PAGE_SHIFT;
>>
>> -       return vmap_pages_range_noflush_walk(addr, end, prot, pages, page_shift);
>> +       err = vmap_pages_range_noflush_walk(addr, end, prot, pages, page_shift);
>> +
>> +       /* Undo the PTEs installed before the failure. */
>> +       if (err)
>> +               __vunmap_range_noflush(addr, end,
>> +                                      gfpflags_allow_blocking(gfp_mask));
> 
> I remember we discussed this before that __vunmap_range_noflush() will
> be unmapping the entire range between addr and end instead of the
> range vmap_pages_range_noflush_walk() actially mapped and your reply
> was that this keeps things simple and avoids adding extra code in the
> common path when we do not fail. This was bothering me, so I looked a
> bit deeper and I think there is an issue with this approach.
> vmap_pages_pte_range() fails with -EBUSY if it encounters an already
> installed PTE. So, if this happens, we immediately bail out of mapping
> the range and will be unmapping the entire range, including these
> existing PTEs. I don't think that's ok.
>

You were right, I did not really think the -EBUSY case through.
when the walk trips over a PTE that is already there, everything
from that point on is somebody else's mapping, and the undo as
it stood would clear it. sorry for this.


> A fix might be quite simple. vmap_pages_range_noflush_walk() already
> counts how many pages we processed in nr variable. If we propagate
> that one level up then we could unmap only the range we actually
> mapped.

Right, thank you very much for helping sort this out. This makes
it a lot clearer.

Thanks
Best Regards
Hao

> 
>> +
>> +       return err;
>>  }
>>
>>  int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
>> @@ -750,7 +772,18 @@ int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
>>
>>         if (ret)
>>                 return ret;
>> -       return __vmap_pages_range_noflush(addr, end, prot, pages, page_shift);
>> +
>> +       ret = __vmap_pages_range_noflush(addr, end, prot, pages, page_shift,
>> +                                        gfp_mask);
>> +       /*
>> +        * The page tables undo themselves on failure. Tear down the
>> +        * metadata that was fully set up before the mapping failed.
>> +        */
>> +       if (ret)
>> +               kmsan_vunmap_range_noflush(addr, end,
>> +                                          gfpflags_allow_blocking(gfp_mask));
>> +
>> +       return ret;
>>  }
>>
>>  static int __vmap_pages_range(unsigned long addr, unsigned long end,
>> diff --git a/mm/vmalloc.h b/mm/vmalloc.h
>> index d8372a4f65c4..9ea28845f713 100644
>> --- a/mm/vmalloc.h
>> +++ b/mm/vmalloc.h
>> @@ -40,10 +40,12 @@ void clear_vm_uninitialized_flag(struct vm_struct *vm);
>>
>>  int __must_check __vmap_pages_range_noflush(unsigned long addr,
>>                 unsigned long end, pgprot_t prot,
>> -               struct page **pages, unsigned int page_shift);
>> +               struct page **pages, unsigned int page_shift,
>> +               gfp_t gfp_mask);
>>
>>  void vunmap_range_noflush(unsigned long start, unsigned long end);
>>
>> -void __vunmap_range_noflush(unsigned long start, unsigned long end);
>> +void __vunmap_range_noflush(unsigned long start, unsigned long end,
>> +                           bool can_resched);
>>
>>  #endif /* __MM_VMALLOC_H */
>> --
>> 2.25.1
>>

Re: [PATCH v4 6/9] mm/vmalloc: only flush the cache after the vmap succeeded

[Hao Ge]

On 2026/11/12 08:30, Suren Baghdasaryan wrote:
> On Tue, Nov 10, 2026 at 11:25 AM Hao Ge <hao.ge@linux.dev> wrote:
>>
>> vmap_page_range() and __vmap_pages_range() call flush_cache_vmap()
> 
> New vmap_pages_range_batched() function has the same issue, so needs fixing too.

Ack. will do next version.

> 
>> before they even look at the error code. A failed mapping is never
>> used, so the flush is pointless at best.
>>
>> According to the Sashiko review it is worse than pointless on UML:
>> flush_cache_vmap() there is flush_tlb_kernel_range(), which syncs the
>> PTEs into the host OS. Flushing a half-installed mapping therefore
>> leaves host mappings behind for a range that was just freed again.
>>
>> Only flush when the mapping, and for vmap_page_range() also the KMSAN
>> metadata, is fully set up.
>>
>> Reported-by: Sashiko <sashiko-bot@kernel.org>
>> Signed-off-by: Hao Ge <hao.ge@linux.dev>
>> ---
>>  mm/vmalloc.c | 8 ++++++--
>>  1 file changed, 6 insertions(+), 2 deletions(-)
>>
>> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
>> index 0565cbfe325c..3f98056f8c7e 100644
>> --- a/mm/vmalloc.c
>> +++ b/mm/vmalloc.c
>> @@ -369,10 +369,12 @@ int vmap_page_range(unsigned long addr, unsigned long end,
>>
>>         err = vmap_range_noflush(addr, end, phys_addr, pgprot_nx(prot),
>>                                  ioremap_max_page_shift);
>> -       flush_cache_vmap(addr, end);
>>         if (!err)
>>                 err = kmsan_ioremap_page_range(addr, end, phys_addr, prot,
>>                                                ioremap_max_page_shift);
>> +       if (!err)
>> +               flush_cache_vmap(addr, end);
>> +
>>         return err;
>>  }
>>
>> @@ -758,7 +760,9 @@ static int __vmap_pages_range(unsigned long addr, unsigned long end,
>>         int err;
>>
>>         err = vmap_pages_range_noflush(addr, end, prot, pages, page_shift, gfp_mask);
>> -       flush_cache_vmap(addr, end);
>> +       if (!err)
>> +               flush_cache_vmap(addr, end);
>> +
>>         return err;
>>  }
>>
>> --
>> 2.25.1
>>

Re: [PATCH v4 4/9] mm/vmalloc: BUG_ON() on an unaligned range in vmap_pages_range_noflush_walk()

[Hao Ge]

Hi Suren

On 2026/11/12 08:16, Suren Baghdasaryan wrote:
> On Tue, Nov 10, 2026 at 11:24 AM Hao Ge <hao.ge@linux.dev> wrote:
>>
>> The pte loop of vmap_pages_pte_range() maps whole pages and stops
>> only when addr lands exactly on end.  An unaligned end lets addr
>> jump over it and the loop keeps mapping into whatever follows the
>> range.  An unaligned start maps the page below the range with its
>> first pte.  That corruption is impossible to debug afterwards, so
>> bug out on addr and end at the entry of
>> vmap_pages_range_noflush_walk().
>>
>> Reported-by: Sashiko <sashiko-bot@kernel.org>
>> Signed-off-by: Hao Ge <hao.ge@linux.dev>
>> ---
>>  mm/vmalloc.c | 2 ++
>>  1 file changed, 2 insertions(+)
>>
>> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
>> index 27f2674a5556..86fbb1d2b27a 100644
>> --- a/mm/vmalloc.c
>> +++ b/mm/vmalloc.c
>> @@ -698,6 +698,8 @@ static int vmap_pages_range_noflush_walk(unsigned long addr, unsigned long end,
>>         pgtbl_mod_mask mask = 0;
>>
>>         BUG_ON(addr >= end);
>> +       BUG_ON(!PAGE_ALIGNED(addr) || !PAGE_ALIGNED(end));
> 
> Instead of BUG_ON() here can we error out early in this function's
> callers (with or without a warning)? This is a static function with
> only 2 callers. vmap_pages_range_batched() uses it in a loop but I
> believe if we check the original address alignment then consequent
> calls are guaranteed to have aligned addresses.
>

One thing that worries me about checking in the callers, a new caller
of the walk can simply forget. 

If we take the approach raised after my patch 3, handling the
unaligned range with WARN instead of BUG, I am happy to do the WARN
here. Even the BUG_ON(addr >= end) above it can become WARN, because
it only fails this mapping. If the module is not that important, we
can ignore it and let the system keep going.

Thanks
Best Regards
Hao

>> +
>>         pgd = pgd_offset_k(addr);
>>         do {
>>                 next = pgd_addr_end(addr, end);
>> --
>> 2.25.1
>>

Re: [PATCH v4 2/9] mm/kmsan: skip vunmap_range_noflush when KMSAN is disabled

[Hao Ge]

Hi Suren

On 2026/11/12 06:10, Suren Baghdasaryan wrote:
> On Tue, Nov 10, 2026 at 11:24 AM Hao Ge <hao.ge@linux.dev> wrote:
>>
>> kmsan_vmap_pages_range_noflush() returns 0 when kmsan_enabled is
>> false.  The unmap side does not, so it still walks the shadow and
>> origin page tables even though KMSAN never installed any metadata
>> there.
> 
> Note that KMSAN_WARN_ON() can reset kmsan_enabled flag, so the fact
> that it's false does not mean that metadata was never installed. I
> think a proper bailout should depend on CONFIG_KMSAN, not on
> kmsan_enabled.

Sorry, I did not realize this logic.

I will go through the code again and will probably drop
this patch in the next version.

Thanks
Best Regards
Hao

> 
>>
>> Add the matching early return.
>>
>> Reported-by: Sashiko <sashiko-bot@kernel.org>
>> Reviewed-by: Dev Jain <dev.jain@arm.com>
>> Signed-off-by: Hao Ge <hao.ge@linux.dev>
>> ---
>>  mm/kmsan/hooks.c | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c
>> index a706666db097..107f66841274 100644
>> --- a/mm/kmsan/hooks.c
>> +++ b/mm/kmsan/hooks.c
>> @@ -138,6 +138,9 @@ static unsigned long vmalloc_origin(unsigned long addr)
>>
>>  void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end)
>>  {
>> +       if (!kmsan_enabled)
>> +               return;
>> +
>>         __vunmap_range_noflush(vmalloc_shadow(start), vmalloc_shadow(end));
>>         __vunmap_range_noflush(vmalloc_origin(start), vmalloc_origin(end));
>>         flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end));
>> --
>> 2.25.1
>>

Re: [PATCH v4 1/9] mm/kmsan: undo the shadow mapping when the origin mapping fails

[Hao Ge]

Hi Suren

On 2026/11/12 05:03, Suren Baghdasaryan wrote:
> On Tue, Nov 10, 2026 at 11:24 AM Hao Ge <hao.ge@linux.dev> wrote:
>>
>> kmsan_vmap_pages_range_noflush() first maps the shadow pages and then
>> the origin pages. If the second mapping fails, the first one is left
>> mapped, and the callers do not roll it back. The next vmap of the
>> same metadata range hits the stale PTEs again and fails with a
>> WARN_ON() and -EBUSY in vmap_pages_pte_range().
>>
>> Undo the shadow mapping on that error path, the same way
>> kmsan_ioremap_page_range() cleans up after a partial failure.
>> __vunmap_range_noflush() only clears the PTEs; nothing has touched
>> the shadow mapping, so no TLB flush is needed.
>>
>> Fixes: 47ebd0310e89 ("mm: kmsan: handle alloc failures in kmsan_vmap_pages_range_noflush()")
>> Cc: stable@vger.kernel.org
>> Reviewed-by: Suren Baghdasaryan <surenb@google.com>
>> Reviewed-by: Dev Jain <dev.jain@arm.com>
>> Signed-off-by: Hao Ge <hao.ge@linux.dev>
>> ---
>>  mm/kmsan/shadow.c | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/mm/kmsan/shadow.c b/mm/kmsan/shadow.c
>> index 0c88d89bf0d6..2166086d3dc3 100644
>> --- a/mm/kmsan/shadow.c
>> +++ b/mm/kmsan/shadow.c
>> @@ -258,6 +258,10 @@ int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
>>                                             o_pages, page_shift);
>>         kmsan_leave_runtime();
>>         if (mapped) {
>> +               /* Undo the shadow mapping set up above. */
>> +               kmsan_enter_runtime();
>> +               __vunmap_range_noflush(shadow_start, shadow_end);
> 
> Sorry, there is a problem here. You introduce "can_resched" check to
> avoid calling cond_resched() later in the series (in patch #7), but at
> this point __vunmap_range_noflush() can be called with GFP flags which
> do not allow blocking. Before recent Ulad's change called "kmsan:
> remove hard-coded GFP_KERNEL flags" this function was called only with
> GPF_KERNEL but now that's not the case. I think you can move
> introduction of can_resched flag into this patch.

Agree. I just went back and looked at Ulad's b186a94227b7 ("kmsan:
remove hard-coded GFP_KERNEL flags").

So will do next version. But perhaps it is better to make it a separate
prep patch in the same series? Because besides this call site, the other
places that call __vunmap_range_noflush() need to be changed as well.

Thanks
Best Regards
Hao

> 
>> +               kmsan_leave_runtime();
>>                 err = mapped;
>>                 goto ret;
>>         }
>> --
>> 2.25.1
>>

Re: [PATCH v4 4/9] mm/vmalloc: BUG_ON() on an unaligned range in vmap_pages_range_noflush_walk()

[Uladzislau Rezki]

On Tue, Nov 10, 2026 at 07:25:41PM +0800, Hao Ge wrote:
> The pte loop of vmap_pages_pte_range() maps whole pages and stops
> only when addr lands exactly on end.  An unaligned end lets addr
> jump over it and the loop keeps mapping into whatever follows the
> range.  An unaligned start maps the page below the range with its
> first pte.  That corruption is impossible to debug afterwards, so
> bug out on addr and end at the entry of
> vmap_pages_range_noflush_walk().
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Signed-off-by: Hao Ge <hao.ge@linux.dev>
> ---
>  mm/vmalloc.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 27f2674a5556..86fbb1d2b27a 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -698,6 +698,8 @@ static int vmap_pages_range_noflush_walk(unsigned long addr, unsigned long end,
>  	pgtbl_mod_mask mask = 0;
>  
>  	BUG_ON(addr >= end);
> +	BUG_ON(!PAGE_ALIGNED(addr) || !PAGE_ALIGNED(end));
> +
>
But why do we add this? We used to live without it and as far i see
we have not gotten any report about any unaligned addresses un-maping.

The vmap allocator grantees that va->va_start/end have to be page
aligned.

--
Uladzislau Rezki

Re: [PATCH v4 5/9] mm/vmalloc: BUG_ON() on an unaligned range in __vunmap_range_noflush()

[Uladzislau Rezki]

On Tue, Nov 10, 2026 at 07:25:42PM +0800, Hao Ge wrote:
> The PTE loop of the unmap path stops only at addr == end.  With an
> unaligned end addr never lands on end, so the loop walks past the
> range and keeps clearing.  With an unaligned start the loop begins
> in the page containing start, which extends below the range.  Either
> way the unmap clears the mappings adjacent to the range, and that is
> nearly impossible to debug afterwards.
> 
> vmap only installs page-aligned, page-granular mappings, so an
> unaligned unmap request is never a legitimate teardown, and there is
> no way to tell which pages the caller meant.  Check addr and end at
> the entry of __vunmap_range_noflush() and bug out before any PTE is
> cleared.
> 
> Signed-off-by: Hao Ge <hao.ge@linux.dev>
> ---
>  mm/vmalloc.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 86fbb1d2b27a..0565cbfe325c 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -518,6 +518,8 @@ void __vunmap_range_noflush(unsigned long start, unsigned long end)
>  	pgtbl_mod_mask mask = 0;
>  
>  	BUG_ON(addr >= end);
> +	BUG_ON(!PAGE_ALIGNED(addr) || !PAGE_ALIGNED(end));
> +
>  	pgd = pgd_offset_k(addr);
>  	do {
>  		next = pgd_addr_end(addr, end);
> -- 
> 2.25.1
> 
I am not sure this is good to stop machine at all. WARN_ON_ONCE()
and return?

--
Uladzislau Rezki

Re: [PATCH v4 6/9] mm/vmalloc: only flush the cache after the vmap succeeded

[Uladzislau Rezki]

On Tue, Nov 10, 2026 at 07:25:43PM +0800, Hao Ge wrote:
> vmap_page_range() and __vmap_pages_range() call flush_cache_vmap()
> before they even look at the error code. A failed mapping is never
> used, so the flush is pointless at best.
> 
> According to the Sashiko review it is worse than pointless on UML:
> flush_cache_vmap() there is flush_tlb_kernel_range(), which syncs the
> PTEs into the host OS. Flushing a half-installed mapping therefore
> leaves host mappings behind for a range that was just freed again.
> 
> Only flush when the mapping, and for vmap_page_range() also the KMSAN
> metadata, is fully set up.
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Signed-off-by: Hao Ge <hao.ge@linux.dev>
> ---
>  mm/vmalloc.c | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 0565cbfe325c..3f98056f8c7e 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -369,10 +369,12 @@ int vmap_page_range(unsigned long addr, unsigned long end,
>  
>  	err = vmap_range_noflush(addr, end, phys_addr, pgprot_nx(prot),
>  				 ioremap_max_page_shift);
> -	flush_cache_vmap(addr, end);
>  	if (!err)
>  		err = kmsan_ioremap_page_range(addr, end, phys_addr, prot,
>  					       ioremap_max_page_shift);
> +	if (!err)
> +		flush_cache_vmap(addr, end);
> +
>  	return err;
>  }
>  
> @@ -758,7 +760,9 @@ static int __vmap_pages_range(unsigned long addr, unsigned long end,
>  	int err;
>  
>  	err = vmap_pages_range_noflush(addr, end, prot, pages, page_shift, gfp_mask);
> -	flush_cache_vmap(addr, end);
> +	if (!err)
> +		flush_cache_vmap(addr, end);
> +
>  	return err;
>  }
>  
> -- 
> 2.25.1
> 
Like Suren noted, vmap_pages_range_batched() should be fixed also.

--
Uladzislau Rezki

Re: [PATCH v4 7/9] mm/vmalloc: undo partial mappings inside the mapping functions

[Uladzislau Rezki]

On Tue, Nov 10, 2026 at 07:25:44PM +0800, Hao Ge wrote:
> The vmap entry points may install some PTEs and then fail, leaving a
> partial mapping behind.
> 
> The callers do not agree on who cleans it up. pcpu_map_pages() and
> kmsan_ioremap_page_range() roll back what they mapped, while
> vm_module_tags_populate() and the __GFP_NOFAIL retry loop in
> __vmalloc_area_node() expect a clean range from the mapping function.
> For them the leftovers stay mapped, and the next attempt to map the
> same range trips the WARN_ON(!pte_none()) in vmap_pages_pte_range().
> A leftover huge pmd is even worse. During the retry, vmap_try_huge_pmd()
> calls pmd_free_pte_page() on it, and the x86 implementation cannot
> tell a leaf entry from a page table and frees the mapped page.
> 
> After discussing with Suren and Ulad, we decided to put the rollback
> into the entry points of the vmap API rather than into every
> low-level helper. __vmap_pages_range_noflush() undoes the whole range
> it was asked to map when it fails, and vmap_page_range() undoes its
> range for the ioremap-style mappings.
> 
> The rollback is __vunmap_range_noflush(), which only clears the PTEs.
> No TLB flush is needed, nothing has touched the mapping yet.
> vmap_pages_range_noflush() also drops the KMSAN metadata of the
> failed mapping.
> 
> The undo walk skips its cond_resched() when the caller's gfp mask
> cannot block. __vmalloc() with GFP_ATOMIC or GFP_NOWAIT can run in
> atomic context, where cond_resched() cannot schedule.
> __vunmap_range_noflush() takes a can_resched argument for that, which
> the sleepable callers pass as true.
> 
> Suggested-by: Uladyslau Rezki <urezki@gmail.com>
> Signed-off-by: Hao Ge <hao.ge@linux.dev>
> ---
>  include/linux/kmsan.h |  8 ++++--
>  mm/kmsan/hooks.c      | 18 ++++++++-----
>  mm/kmsan/shadow.c     |  7 ++---
>  mm/vmalloc.c          | 61 +++++++++++++++++++++++++++++++++----------
>  mm/vmalloc.h          |  6 +++--
>  5 files changed, 72 insertions(+), 28 deletions(-)
> 
> diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h
> index 7da9fd506b39..0016c032d1e7 100644
> --- a/include/linux/kmsan.h
> +++ b/include/linux/kmsan.h
> @@ -150,11 +150,14 @@ int __must_check kmsan_vmap_pages_range_noflush(unsigned long start,
>   * kmsan_vunmap_kernel_range_noflush() - Notify KMSAN about a vunmap.
>   * @start: start of vunmapped range.
>   * @end:   end of vunmapped range.
> + * @can_resched: whether the unmap walk may call cond_resched(); pass
> + * false from atomic context.
>   *
>   * KMSAN unmaps the contiguous metadata ranges created by
>   * kmsan_map_kernel_range_noflush().
>   */
> -void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end);
> +void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end,
> +				bool can_resched);
>  
>  /**
>   * kmsan_ioremap_page_range() - Notify KMSAN about a ioremap_page_range() call.
> @@ -355,7 +358,8 @@ static inline int __must_check kmsan_vmap_pages_range_noflush(
>  }
>  
>  static inline void kmsan_vunmap_range_noflush(unsigned long start,
> -					      unsigned long end)
> +					      unsigned long end,
> +					      bool can_resched)
>  {
>  }
>  
> diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c
> index 107f66841274..0cd1edaa1120 100644
> --- a/mm/kmsan/hooks.c
> +++ b/mm/kmsan/hooks.c
> @@ -136,13 +136,16 @@ static unsigned long vmalloc_origin(unsigned long addr)
>  						 KMSAN_META_ORIGIN);
>  }
>  
> -void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end)
> +void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end,
> +				bool can_resched)
>  {
>  	if (!kmsan_enabled)
>  		return;
>  
> -	__vunmap_range_noflush(vmalloc_shadow(start), vmalloc_shadow(end));
> -	__vunmap_range_noflush(vmalloc_origin(start), vmalloc_origin(end));
> +	__vunmap_range_noflush(vmalloc_shadow(start), vmalloc_shadow(end),
> +			       can_resched);
> +	__vunmap_range_noflush(vmalloc_origin(start), vmalloc_origin(end),
> +			       can_resched);
>  	flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end));
>  	flush_cache_vmap(vmalloc_origin(start), vmalloc_origin(end));
>  }
> @@ -171,8 +174,9 @@ static void kmsan_iounmap_pages(unsigned long start, unsigned long end)
>  		if (origin)
>  			__free_pages(origin, KMSAN_IOREMAP_META_ORDER);
>  	}
> -	__vunmap_range_noflush(shadow_start, shadow_end);
> -	__vunmap_range_noflush(origin_start, origin_end);
> +	/* ioremap metadata mappings are always GFP_KERNEL, so resched is safe. */
> +	__vunmap_range_noflush(shadow_start, shadow_end, true);
> +	__vunmap_range_noflush(origin_start, origin_end, true);
>  	flush_tlb_kernel_range(shadow_start, shadow_end);
>  	flush_tlb_kernel_range(origin_start, origin_end);
>  }
> @@ -207,7 +211,7 @@ int kmsan_ioremap_page_range(unsigned long start, unsigned long end,
>  		mapped = __vmap_pages_range_noflush(
>  			vmalloc_shadow(start + off),
>  			vmalloc_shadow(start + off + PAGE_SIZE), prot, &shadow,
> -			PAGE_SHIFT);
> +			PAGE_SHIFT, gfp_mask);
>  		if (mapped) {
>  			err = mapped;
>  			goto ret;
> @@ -217,7 +221,7 @@ int kmsan_ioremap_page_range(unsigned long start, unsigned long end,
>  		mapped = __vmap_pages_range_noflush(
>  			vmalloc_origin(start + off),
>  			vmalloc_origin(start + off + PAGE_SIZE), prot, &origin,
> -			PAGE_SHIFT);
> +			PAGE_SHIFT, gfp_mask);
>  		if (mapped) {
>  			err = mapped;
>  			goto ret;
> diff --git a/mm/kmsan/shadow.c b/mm/kmsan/shadow.c
> index 2166086d3dc3..452760a333e4 100644
> --- a/mm/kmsan/shadow.c
> +++ b/mm/kmsan/shadow.c
> @@ -247,7 +247,7 @@ int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
>  	origin_end = vmalloc_meta((void *)end, KMSAN_META_ORIGIN);
>  	kmsan_enter_runtime();
>  	mapped = __vmap_pages_range_noflush(shadow_start, shadow_end, prot,
> -					    s_pages, page_shift);
> +					    s_pages, page_shift, gfp_mask);
>  	kmsan_leave_runtime();
>  	if (mapped) {
>  		err = mapped;
> @@ -255,12 +255,13 @@ int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
>  	}
>  	kmsan_enter_runtime();
>  	mapped = __vmap_pages_range_noflush(origin_start, origin_end, prot,
> -					    o_pages, page_shift);
> +					    o_pages, page_shift, gfp_mask);
>  	kmsan_leave_runtime();
>  	if (mapped) {
>  		/* Undo the shadow mapping set up above. */
>  		kmsan_enter_runtime();
> -		__vunmap_range_noflush(shadow_start, shadow_end);
> +		__vunmap_range_noflush(shadow_start, shadow_end,
> +				       gfpflags_allow_blocking(gfp_mask));
>  		kmsan_leave_runtime();
>  		err = mapped;
>  		goto ret;
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 3f98056f8c7e..d1986f68eab4 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -372,8 +372,12 @@ int vmap_page_range(unsigned long addr, unsigned long end,
>  	if (!err)
>  		err = kmsan_ioremap_page_range(addr, end, phys_addr, prot,
>  					       ioremap_max_page_shift);
> -	if (!err)
> +	if (err) {
> +		/* Callers are ioremap and arch setup code, always sleepable. */
> +		__vunmap_range_noflush(addr, end, true);
> +	} else {
>  		flush_cache_vmap(addr, end);
> +	}
>  
>  	return err;
>  }
> @@ -429,7 +433,7 @@ static void vunmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
>  }
>  
>  static void vunmap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end,
> -			     pgtbl_mod_mask *mask)
> +			     pgtbl_mod_mask *mask, bool can_resched)
>  {
>  	pmd_t *pmd;
>  	unsigned long next;
> @@ -451,12 +455,14 @@ static void vunmap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end,
>  			continue;
>  		vunmap_pte_range(pmd, addr, next, mask);
>  
> -		cond_resched();
> +		/* The undo path may run in atomic context. */
> +		if (can_resched)
> +			cond_resched();
>  	} while (pmd++, addr = next, addr != end);
>  }
>  
>  static void vunmap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end,
> -			     pgtbl_mod_mask *mask)
> +			     pgtbl_mod_mask *mask, bool can_resched)
>  {
>  	pud_t *pud;
>  	unsigned long next;
> @@ -476,12 +482,12 @@ static void vunmap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end,
>  		}
>  		if (pud_none_or_clear_bad(pud))
>  			continue;
> -		vunmap_pmd_range(pud, addr, next, mask);
> +		vunmap_pmd_range(pud, addr, next, mask, can_resched);
>  	} while (pud++, addr = next, addr != end);
>  }
>  
>  static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end,
> -			     pgtbl_mod_mask *mask)
> +			     pgtbl_mod_mask *mask, bool can_resched)
>  {
>  	p4d_t *p4d;
>  	unsigned long next;
> @@ -496,7 +502,7 @@ static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end,
>  
>  		if (p4d_none_or_clear_bad(p4d))
>  			continue;
> -		vunmap_pud_range(p4d, addr, next, mask);
> +		vunmap_pud_range(p4d, addr, next, mask, can_resched);
>  	} while (p4d++, addr = next, addr != end);
>  }
>  
> @@ -512,7 +518,8 @@ static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end,
>   *
>   * This is an internal function only. Do not use outside mm/.
>   */
> -void __vunmap_range_noflush(unsigned long start, unsigned long end)
> +void __vunmap_range_noflush(unsigned long start, unsigned long end,
> +			    bool can_resched)
>  {
>  	unsigned long next;
>  	pgd_t *pgd;
> @@ -529,7 +536,7 @@ void __vunmap_range_noflush(unsigned long start, unsigned long end)
>  			mask |= PGTBL_PGD_MODIFIED;
>  		if (pgd_none_or_clear_bad(pgd))
>  			continue;
> -		vunmap_p4d_range(pgd, addr, next, &mask);
> +		vunmap_p4d_range(pgd, addr, next, &mask, can_resched);
>  	} while (pgd++, addr = next, addr != end);
>  
>  	if (mask & ARCH_PAGE_TABLE_SYNC_MASK)
> @@ -538,8 +545,9 @@ void __vunmap_range_noflush(unsigned long start, unsigned long end)
>  
>  void vunmap_range_noflush(unsigned long start, unsigned long end)
>  {
> -	kmsan_vunmap_range_noflush(start, end);
> -	__vunmap_range_noflush(start, end);
> +	/* The unmap paths are sleepable, keep the resched point. */
> +	kmsan_vunmap_range_noflush(start, end, true);
> +	__vunmap_range_noflush(start, end, true);
>  }
>  
>  /**
> @@ -727,18 +735,32 @@ static int vmap_pages_range_noflush_walk(unsigned long addr, unsigned long end,
>   * The caller is responsible for calling flush_cache_vmap() after this
>   * function returns successfully and before the addresses are accessed.
>   *
> + * On failure the mapping is torn down before returning, so a retry
> + * sees a clean range. When @gfp_mask cannot block, the teardown
> + * skips cond_resched(), it may run in atomic context.
> + *
>   * This is an internal function only. Do not use outside mm/.
>   */
>  int __vmap_pages_range_noflush(unsigned long addr, unsigned long end,
> -		pgprot_t prot, struct page **pages, unsigned int page_shift)
> +		pgprot_t prot, struct page **pages, unsigned int page_shift,
> +		gfp_t gfp_mask)
>  {
> +	int err;
> +
>  	if (WARN_ON_ONCE(page_shift < PAGE_SHIFT))
>  		return -EINVAL;
>  
>  	if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC))
>  		page_shift = PAGE_SHIFT;
>  
> -	return vmap_pages_range_noflush_walk(addr, end, prot, pages, page_shift);
> +	err = vmap_pages_range_noflush_walk(addr, end, prot, pages, page_shift);
> +
> +	/* Undo the PTEs installed before the failure. */
> +	if (err)
> +		__vunmap_range_noflush(addr, end,
> +				       gfpflags_allow_blocking(gfp_mask));
> +
> +	return err;
>  }
>  
>  int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
> @@ -750,7 +772,18 @@ int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
>  
>  	if (ret)
>  		return ret;
> -	return __vmap_pages_range_noflush(addr, end, prot, pages, page_shift);
> +
> +	ret = __vmap_pages_range_noflush(addr, end, prot, pages, page_shift,
> +					 gfp_mask);
> +	/*
> +	 * The page tables undo themselves on failure. Tear down the
> +	 * metadata that was fully set up before the mapping failed.
> +	 */
> +	if (ret)
> +		kmsan_vunmap_range_noflush(addr, end,
> +					   gfpflags_allow_blocking(gfp_mask));
> +
> +	return ret;
>  }
>  
>  static int __vmap_pages_range(unsigned long addr, unsigned long end,
> diff --git a/mm/vmalloc.h b/mm/vmalloc.h
> index d8372a4f65c4..9ea28845f713 100644
> --- a/mm/vmalloc.h
> +++ b/mm/vmalloc.h
> @@ -40,10 +40,12 @@ void clear_vm_uninitialized_flag(struct vm_struct *vm);
>  
>  int __must_check __vmap_pages_range_noflush(unsigned long addr,
>  		unsigned long end, pgprot_t prot,
> -		struct page **pages, unsigned int page_shift);
> +		struct page **pages, unsigned int page_shift,
> +		gfp_t gfp_mask);
>  
>  void vunmap_range_noflush(unsigned long start, unsigned long end);
>  
> -void __vunmap_range_noflush(unsigned long start, unsigned long end);
> +void __vunmap_range_noflush(unsigned long start, unsigned long end,
> +			    bool can_resched);
>  
>  #endif /* __MM_VMALLOC_H */
> -- 
> 2.25.1
> 
OK. Waiting for v5.

--
Uladzislau Rezki

Re: [PATCH v4 3/9] mm/vmalloc: BUG_ON() on an unaligned range in vmap_range_noflush()

[Uladzislau Rezki]

On Tue, Nov 10, 2026 at 07:25:40PM +0800, Hao Ge wrote:
> commit a739e6b557af ("mm: vmalloc: WARN_ON if mapping size is not
> PAGE_SIZE aligned") put the check in vmap_pte_range(), which is too
> late, because the upper levels may already have installed page
> tables for a range that is rejected anyway.
> 
> The pte loop maps whole pages and stops only when addr lands
> exactly on end.  An unaligned end lets addr jump over it and the
> loop keeps mapping past the range.  An unaligned start maps the
> page below the range and usually misses end too.  Such a range is
> never a legitimate request, and the corruption it leaves behind is
> hard to attribute to the caller afterwards, so bug out on addr and
> end at the entry of vmap_range_noflush() and drop the check from
> vmap_pte_range().
> 
> Signed-off-by: Hao Ge <hao.ge@linux.dev>
> ---
>  mm/vmalloc.c | 4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
> 
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 093b8d5f7b0e..27f2674a5556 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -125,9 +125,6 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
>  	unsigned long size;
>  	unsigned int nr_pages;
>  
> -	if (WARN_ON_ONCE(!PAGE_ALIGNED(end - addr)))
> -		return -EINVAL;
> -
>  	pfn = phys_addr >> PAGE_SHIFT;
>  	pte = pte_alloc_kernel_track(pmd, addr, mask);
>  	if (!pte)
> @@ -347,6 +344,7 @@ static int vmap_range_noflush(unsigned long addr, unsigned long end,
>  	 */
>  	might_sleep();
>  	BUG_ON(addr >= end);
> +	BUG_ON(!PAGE_ALIGNED(addr) || !PAGE_ALIGNED(end));
>  
>  	start = addr;
>  	pgd = pgd_offset_k(addr);
> -- 
> 2.25.1
> 
Like Suren proposed, there is only one user of vmap_range_noflush() it
is vmap_page_range() function. You can add warn-on-once() there and
bail out.

Do not add BUG_ON().

--
Uladzislau Rezki

Re: [PATCH v4 1/9] mm/kmsan: undo the shadow mapping when the origin mapping fails

[Suren Baghdasaryan]

On Wed, Nov 11, 2026 at 11:49 PM Hao Ge <hao.ge@linux.dev> wrote:
>
> Hi Suren
>
> On 2026/11/12 05:03, Suren Baghdasaryan wrote:
> > On Tue, Nov 10, 2026 at 11:24 AM Hao Ge <hao.ge@linux.dev> wrote:
> >>
> >> kmsan_vmap_pages_range_noflush() first maps the shadow pages and then
> >> the origin pages. If the second mapping fails, the first one is left
> >> mapped, and the callers do not roll it back. The next vmap of the
> >> same metadata range hits the stale PTEs again and fails with a
> >> WARN_ON() and -EBUSY in vmap_pages_pte_range().
> >>
> >> Undo the shadow mapping on that error path, the same way
> >> kmsan_ioremap_page_range() cleans up after a partial failure.
> >> __vunmap_range_noflush() only clears the PTEs; nothing has touched
> >> the shadow mapping, so no TLB flush is needed.
> >>
> >> Fixes: 47ebd0310e89 ("mm: kmsan: handle alloc failures in kmsan_vmap_pages_range_noflush()")
> >> Cc: stable@vger.kernel.org
> >> Reviewed-by: Suren Baghdasaryan <surenb@google.com>
> >> Reviewed-by: Dev Jain <dev.jain@arm.com>
> >> Signed-off-by: Hao Ge <hao.ge@linux.dev>
> >> ---
> >>  mm/kmsan/shadow.c | 4 ++++
> >>  1 file changed, 4 insertions(+)
> >>
> >> diff --git a/mm/kmsan/shadow.c b/mm/kmsan/shadow.c
> >> index 0c88d89bf0d6..2166086d3dc3 100644
> >> --- a/mm/kmsan/shadow.c
> >> +++ b/mm/kmsan/shadow.c
> >> @@ -258,6 +258,10 @@ int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
> >>                                             o_pages, page_shift);
> >>         kmsan_leave_runtime();
> >>         if (mapped) {
> >> +               /* Undo the shadow mapping set up above. */
> >> +               kmsan_enter_runtime();
> >> +               __vunmap_range_noflush(shadow_start, shadow_end);
> >
> > Sorry, there is a problem here. You introduce "can_resched" check to
> > avoid calling cond_resched() later in the series (in patch #7), but at
> > this point __vunmap_range_noflush() can be called with GFP flags which
> > do not allow blocking. Before recent Ulad's change called "kmsan:
> > remove hard-coded GFP_KERNEL flags" this function was called only with
> > GPF_KERNEL but now that's not the case. I think you can move
> > introduction of can_resched flag into this patch.
>
> Agree. I just went back and looked at Ulad's b186a94227b7 ("kmsan:
> remove hard-coded GFP_KERNEL flags").
>
> So will do next version. But perhaps it is better to make it a separate
> prep patch in the same series? Because besides this call site, the other
> places that call __vunmap_range_noflush() need to be changed as well.

Agree. Moving that into a separate patch at the beginning of the
series with its own changelog and explanation would be cleaner.

>
> Thanks
> Best Regards
> Hao
>
> >
> >> +               kmsan_leave_runtime();
> >>                 err = mapped;
> >>                 goto ret;
> >>         }
> >> --
> >> 2.25.1
> >>