From: Hao Ge <hao.ge@linux.dev>
To: Suren Baghdasaryan <surenb@google.com>,
Uladzislau Rezki <urezki@gmail.com>, Dev Jain <dev.jain@arm.com>,
Andrew Morton <akpm@linux-foundation.org>,
Madhavan Srinivasan <maddy@linux.ibm.com>,
Michael Ellerman <mpe@ellerman.id.au>,
Nicholas Piggin <npiggin@gmail.com>,
"Christophe Leroy (CS GROUP)" <chleroy@kernel.org>,
"Ritesh Harjani (IBM)" <ritesh.list@gmail.com>,
Shrikanth Hegde <sshegde@linux.ibm.com>,
Alexander Potapenko <glider@google.com>,
Marco Elver <elver@google.com>,
Dmitry Vyukov <dvyukov@google.com>,
Dennis Zhou <dennis@kernel.org>, Tejun Heo <tj@kernel.org>,
Christoph Lameter <cl@gentwo.org>
Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
kasan-dev@googlegroups.com, linux-mm@kvack.org,
Hao Ge <hao.ge@linux.dev>
Subject: [PATCH v5 09/11] mm/vmalloc: undo what vmap_pages_range_batched() mapped on failure
Date: Mon, 16 Nov 2026 15:13:23 +0800 [thread overview]
Message-ID: <20261116071325.263682-10-hao.ge@linux.dev> (raw)
In-Reply-To: <20261116071325.263682-1-hao.ge@linux.dev>
vmap_pages_range_batched() maps the range batch by batch. When a
batch fails, the batches before it are installed, and the failing
one may have installed some of its pages. mapped_end follows the
end of the last installed page, and the error path undoes
[addr, mapped_end), never past the point where the walk stopped.
The KMSAN metadata is set up for the whole range up front, so it is
torn down as a whole when the mapping fails, like in
vmap_pages_range_noflush(). The only caller, vmap(), also calls
vunmap() on its error path; the double teardown only walks over PTEs
that are already cleared.
Signed-off-by: Hao Ge <hao.ge@linux.dev>
---
mm/vmalloc.c | 22 ++++++++++++++++------
1 file changed, 16 insertions(+), 6 deletions(-)
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 28e65073624a..aa80bc273ec0 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -3748,13 +3748,14 @@ static int vmap_pages_range_batched(unsigned long addr, unsigned long end,
const unsigned int nr_pages = (end - addr) >> PAGE_SHIFT;
unsigned int prev_shift = 0, batch_idx = 0, nr_mapped;
unsigned long batch_start = addr, batch_end = addr;
+ unsigned long mapped_end = addr;
int err;
err = kmsan_vmap_pages_range_noflush(addr, end, prot, pages,
PAGE_SHIFT, GFP_KERNEL);
if (err)
- goto out;
+ return err;
for (unsigned int i = 0; i < nr_pages; ) {
unsigned int shift = PAGE_SHIFT +
@@ -3764,9 +3765,10 @@ static int vmap_pages_range_batched(unsigned long addr, unsigned long end,
prev_shift = shift;
if (shift != prev_shift) {
- err = vmap_pages_range_noflush_walk(batch_start, batch_end,
- prot, pages + batch_idx, prev_shift,
- &nr_mapped);
+ err = vmap_pages_range_noflush_walk(batch_start,
+ batch_end, prot, pages + batch_idx,
+ prev_shift, &nr_mapped);
+ mapped_end = batch_start + (nr_mapped * PAGE_SIZE);
if (err)
goto out;
prev_shift = shift;
@@ -3786,13 +3788,21 @@ static int vmap_pages_range_batched(unsigned long addr, unsigned long end,
}
/* Remaining */
- if (batch_start < end)
+ if (batch_start < end) {
err = vmap_pages_range_noflush_walk(batch_start, end, prot,
pages + batch_idx, prev_shift, &nr_mapped);
+ mapped_end = batch_start + (nr_mapped * PAGE_SIZE);
+ }
out:
- if (!err)
+ if (err) {
+ /* Undo the part the batches installed before they failed. */
+ if (mapped_end > addr)
+ __vunmap_range_noflush(addr, mapped_end, true);
+ kmsan_vunmap_range_noflush(addr, end, true);
+ } else {
flush_cache_vmap(addr, end);
+ }
return err;
}
--
2.25.1
next prev parent reply other threads:[~2026-11-16 7:14 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-16 7:13 [PATCH v5 00/11] mm/vmalloc: fix up vmap error handling Hao Ge
2026-11-16 7:13 ` [PATCH v5 01/11] mm/vmalloc: skip cond_resched() in the vmap unmap paths when atomic Hao Ge
2026-11-16 7:13 ` [PATCH v5 02/11] mm/kmsan: undo the shadow mapping when the origin mapping fails Hao Ge
2026-11-16 7:13 ` [PATCH v5 03/11] mm/vmalloc: reject an unaligned range in vmap_page_range() Hao Ge
2026-11-17 5:53 ` Dev Jain
2026-11-16 7:13 ` [PATCH v5 04/11] mm/vmalloc: reject an unaligned range in __vunmap_range_noflush() Hao Ge
2026-11-17 6:05 ` Dev Jain
2026-11-16 7:13 ` [PATCH v5 05/11] mm/vmalloc: only flush the cache after the vmap succeeded Hao Ge
2026-11-16 7:13 ` [PATCH v5 06/11] mm/vmalloc: undo what vmap_page_range() mapped on failure Hao Ge
2026-11-16 7:13 ` [PATCH v5 07/11] mm/vmalloc: undo what __vmap_pages_range_noflush() " Hao Ge
2026-11-17 7:15 ` sashiko-bot
2026-11-17 7:27 ` Hao Ge
2026-11-16 7:13 ` [PATCH v5 08/11] mm/vmalloc: drop the KMSAN metadata in vmap_pages_range_noflush() " Hao Ge
2026-11-16 7:13 ` Hao Ge [this message]
2026-11-16 7:13 ` [PATCH v5 10/11] powerpc: drop redundant unmaps of failed vmap mappings Hao Ge
2026-11-16 7:13 ` [PATCH v5 11/11] mm/percpu: stop unmapping the CPU that failed to map Hao Ge
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261116071325.263682-10-hao.ge@linux.dev \
--to=hao.ge@linux.dev \
--cc=akpm@linux-foundation.org \
--cc=chleroy@kernel.org \
--cc=cl@gentwo.org \
--cc=dennis@kernel.org \
--cc=dev.jain@arm.com \
--cc=dvyukov@google.com \
--cc=elver@google.com \
--cc=glider@google.com \
--cc=kasan-dev@googlegroups.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=maddy@linux.ibm.com \
--cc=mpe@ellerman.id.au \
--cc=npiggin@gmail.com \
--cc=ritesh.list@gmail.com \
--cc=sshegde@linux.ibm.com \
--cc=surenb@google.com \
--cc=tj@kernel.org \
--cc=urezki@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.