* FAILED: patch "[PATCH] mm: do not unintentionally overwrite atomic VMA flags" failed to apply to 7.3-stable tree
@ 2026-11-10 1:01 Sasha Levin
2026-11-11 20:02 ` [PATCH 7.3.y] mm: do not unintentionally overwrite atomic VMA flags Sasha Levin
0 siblings, 1 reply; 2+ messages in thread
From: Sasha Levin @ 2026-11-10 1:01 UTC (permalink / raw)
To: ljs, akpm, david, pfalcato, gourry, liam, vbabka, rppt, surenb,
mhocko, jannh, dave, sj, kees, ccross
Cc: stable
The patch below does not apply to the 7.3-stable tree.
If someone wants it applied there, or to any other stable or longterm
tree, then please email the backport, including the original git commit
id to <stable@vger.kernel.org>.
To reproduce the conflict and resubmit, you may use the following commands:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-7.3.y
git checkout FETCH_HEAD
git cherry-pick -x 13fc99f46399c9d38d1b54ecf0a3f387ba04d05d
# <resolve conflicts, build, test, etc.>
git commit -s
git send-email --to '<stable@vger.kernel.org>' --in-reply-to '20261109200132-failed-7.3-13fc99f46399@kernel.org' --subject-prefix 'PATCH 7.3.y' 'HEAD^..'
Thanks,
Sasha
------------------ original commit in Linus's tree ------------------
From 13fc99f46399c9d38d1b54ecf0a3f387ba04d05d Mon Sep 17 00:00:00 2001
From: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>
Date: Fri, 23 Oct 2026 14:47:27 +0100
Subject: [PATCH] mm: do not unintentionally overwrite atomic VMA flags
Patch series "mm: avoid losing atomic VMA flags", v1.
Since commit 2ab7f1bbafc9 ("mm/madvise: allow guard page install/remove
under VMA lock"), VMA_MAYBE_GUARD_BIT can be set atomically under the VMA
read lock.
However, a number of places wrongly assume the flags are stable under the
read lock, which can cause stale atomic VMA flags to overwrite existing
valid ones.
Additionally, one case in which VMA_SOFTDIRTY_BIT was overwritten is also
addressed.
This series addresses each of these, while keeping the logic inline with
existing conventions.
Each patch in this series is a hotfix.
This patch (of 4):
madvise, mlock and mprotect take a copy of a VMA's flags holding only the
VMA read lock, later writing it back under the write lock (in
madvise_update_vma(), mlock_fixup(), mlock_vma_pages_range() and
mprotect_fixup()).
This was done on the assumption that VMA flags cannot change with a read
lock held.
However this changed in commit 2ab7f1bbafc9 ("mm/madvise: allow guard page
install/remove under VMA lock"), which allowed VM_ATOMIC_SET_ALLOWED
flags (currently only VMA_MAYBE_GUARD_BIT) to be written concurrently under
a VMA read lock.
The actual write itself is safe as this is done under the VMA write lock
at which point no further concurrent writes can occur, however at this
point the stale flags mean that these callers can unintentionally overwrite
atomic flags incorrectly.
The consequences of this are significant - a missing VMA_MAYBE_GUARD_BIT
could result in page tables not being copied on fork (the vma_needs_copy()
check can return false) or being overwritten on file-backed THP
collapse (the file_backed_vma_is_retractable() can return true).
Resolve the issue by always taking care to retain existing atomic flags
when overwriting VMA flags, replacing the inconsistently named
vma_flags_reset_once() with vma_overwrite_flags() to do so, and always
invoking it when flags are overwritten like this.
Reads are a problem too - madvise, mlock and mprotect all compare VMA flags
without a VMA write lock held via vma_flags_same_pair().
This has more serious impact, as concurrent changes to atomic VMA flags can
cause two sets of VMA flags to be wrongly considered different.
For mprotect and madvise this causes a useless split, but mlock is impacted
more seriously as it can result in mm->locked_vm underflow.
Fix this by updating vma_flags_same_pair() to disregard
VM_ATOMIC_SET_ALLOWED VMAs - VMA equality is the same as VMA mergeability,
and atomic VMA flags are disregarded when comparing VMA flags for merge -
so this is justified semantically.
Additionally, remove the redundant vma_flags_same_mask() and the unused
vma_flags_same() as part of the change, and replace VM_ATOMIC_SET_ALLOWED
with VMA_ATOMIC_SET_ALLOWED, using the new VMA flag API.
Finally, update userland VMA tests to reflect the changes.
Link: https://lore.kernel.org/20261023-fix-vma-atomic-v1-0-8ff4fae1fc85@kernel.org
Link: https://lore.kernel.org/20261023-fix-vma-atomic-v1-1-8ff4fae1fc85@kernel.org
Fixes: 2ab7f1bbafc9 ("mm/madvise: allow guard page install/remove under VMA lock")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Suggested-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Tested-by: Gregory Price (Meta) <gourry@gourry.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Jann Horn <jannh@google.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: SJ Park <sj@kernel.org>
Cc: Kees Cook <kees@kernel.org>
Cc: Colin Cross <ccross@google.com>
Cc: <stable@vger.kernel.org>
---
include/linux/mm.h | 166 ++++++++++++++++----------------
mm/madvise.c | 4 +-
mm/mlock.c | 6 +-
mm/mprotect.c | 2 +-
tools/testing/vma/include/dup.h | 76 +++++++++------
tools/testing/vma/shared.h | 16 +--
6 files changed, 146 insertions(+), 124 deletions(-)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 9f167950926a1..6af3b7008f1ad 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -602,7 +602,7 @@ enum {
#define VMA_LOCKED_MASK mk_vma_flags(VMA_LOCKED_BIT, VMA_LOCKONFAULT_BIT)
/* These flags can be updated atomically via VMA/mmap read lock. */
-#define VM_ATOMIC_SET_ALLOWED VM_MAYBE_GUARD
+#define VMA_ATOMIC_SET_ALLOWED mk_vma_flags(VMA_MAYBE_GUARD_BIT)
/* Arch-specific flags to clear when updating VM flags on protection change */
#ifndef VM_ARCH_CLEAR
@@ -987,22 +987,6 @@ static inline void vm_flags_reset(struct vm_area_struct *vma,
vm_flags_init(vma, flags);
}
-static inline void vma_flags_reset_once(struct vm_area_struct *vma,
- vma_flags_t *flags)
-{
- const unsigned long word = flags->__vma_flags[0];
-
- /* It is assumed only the first system word must be written once. */
- vma_flags_overwrite_word_once(&vma->flags, word);
- /* The remainder can be copied normally. */
- if (NUM_VMA_FLAG_BITS > BITS_PER_LONG) {
- unsigned long *dst = &vma->flags.__vma_flags[1];
- const unsigned long *src = &flags->__vma_flags[1];
-
- bitmap_copy(dst, src, NUM_VMA_FLAG_BITS - BITS_PER_LONG);
- }
-}
-
static inline void vm_flags_set(struct vm_area_struct *vma,
vm_flags_t flags)
{
@@ -1039,48 +1023,6 @@ static inline void vm_flags_mod(struct vm_area_struct *vma,
__vm_flags_mod(vma, set, clear);
}
-static __always_inline bool __vma_atomic_valid_flag(struct vm_area_struct *vma,
- vma_flag_t bit)
-{
- const vm_flags_t mask = BIT((__force int)bit);
-
- /* Only specific flags are permitted */
- if (WARN_ON_ONCE(!(mask & VM_ATOMIC_SET_ALLOWED)))
- return false;
-
- return true;
-}
-
-/*
- * Set VMA flag atomically. Requires only VMA/mmap read lock. Only specific
- * valid flags are allowed to do this.
- */
-static __always_inline void vma_set_atomic_flag(struct vm_area_struct *vma,
- vma_flag_t bit)
-{
- unsigned long *bitmap = vma->flags.__vma_flags;
-
- vma_assert_stabilised(vma);
- if (__vma_atomic_valid_flag(vma, bit))
- set_bit((__force int)bit, bitmap);
-}
-
-/*
- * Test for VMA flag atomically. Requires no locks. Only specific valid flags
- * are allowed to do this.
- *
- * This is necessarily racey, so callers must ensure that serialisation is
- * achieved through some other means, or that races are permissible.
- */
-static __always_inline bool vma_test_atomic_flag(struct vm_area_struct *vma,
- vma_flag_t bit)
-{
- if (__vma_atomic_valid_flag(vma, bit))
- return test_bit((__force int)bit, &vma->vm_flags);
-
- return false;
-}
-
/* Set an individual VMA flag in flags, non-atomically. */
static __always_inline void vma_flags_set_flag(vma_flags_t *flags,
vma_flag_t bit)
@@ -1302,34 +1244,24 @@ static __always_inline vma_flags_t vma_flags_diff_pair(const vma_flags_t *flags,
return dst;
}
-/* Determine if flags and flags_other have precisely the same flags set. */
+/*
+ * Determine if flags and flags_other have precisely the same flags set,
+ * excluding atomic VMA flags.
+ */
static __always_inline bool vma_flags_same_pair(const vma_flags_t *flags,
- const vma_flags_t *flags_other)
+ const vma_flags_t *flags_other)
{
- const unsigned long *bitmap = flags->__vma_flags;
- const unsigned long *bitmap_other = flags_other->__vma_flags;
-
- return bitmap_equal(bitmap, bitmap_other, NUM_VMA_FLAG_BITS);
-}
+ vma_flags_t diff = vma_flags_diff_pair(flags, flags_other);
-/* Determine if flags and flags_other have precisely the same flags set. */
-static __always_inline bool vma_flags_same_mask(const vma_flags_t *flags,
- vma_flags_t flags_other)
-{
- const unsigned long *bitmap = flags->__vma_flags;
- const unsigned long *bitmap_other = flags_other.__vma_flags;
+ /*
+ * Atomic VMA flags may be set concurrently under the VMA read lock, so
+ * are disregarded for equality testing purposes.
+ */
+ vma_flags_clear_mask(&diff, VMA_ATOMIC_SET_ALLOWED);
- return bitmap_equal(bitmap, bitmap_other, NUM_VMA_FLAG_BITS);
+ return vma_flags_empty(&diff);
}
-/*
- * Helper macro to determine if only the specific flags are set, e.g.:
- *
- * if (vma_flags_same(&flags, VMA_WRITE_BIT) { ... }
- */
-#define vma_flags_same(flags, ...) \
- vma_flags_same_mask(flags, mk_vma_flags(__VA_ARGS__))
-
/*
* Test whether a specific flag in the VMA is set, e.g.:
*
@@ -1436,6 +1368,78 @@ static __always_inline void vma_clear_flags_mask(struct vm_area_struct *vma,
#define vma_clear_flags(vma, ...) \
vma_clear_flags_mask(vma, mk_vma_flags(__VA_ARGS__))
+static __always_inline bool __vma_atomic_valid_flag(struct vm_area_struct *vma,
+ vma_flag_t bit)
+{
+ const vma_flags_t mask = VMA_ATOMIC_SET_ALLOWED;
+
+ /* Is this flag allowed to be set atomically? */
+ if (WARN_ON_ONCE(!vma_flags_test(&mask, bit)))
+ return false;
+
+ return true;
+}
+
+/*
+ * Set VMA flag atomically. Requires only VMA/mmap read lock. Only specific
+ * valid flags are allowed to do this.
+ */
+static __always_inline void vma_set_atomic_flag(struct vm_area_struct *vma,
+ vma_flag_t bit)
+{
+ unsigned long *bitmap = vma->flags.__vma_flags;
+
+ vma_assert_stabilised(vma);
+ if (__vma_atomic_valid_flag(vma, bit))
+ set_bit((__force int)bit, bitmap);
+}
+
+/*
+ * Test for VMA flag atomically. Requires no locks. Only specific valid flags
+ * are allowed to do this.
+ *
+ * This is necessarily racey, so callers must ensure that serialisation is
+ * achieved through some other means, or that races are permissible.
+ */
+static __always_inline bool vma_test_atomic_flag(struct vm_area_struct *vma,
+ vma_flag_t bit)
+{
+ if (__vma_atomic_valid_flag(vma, bit))
+ return test_bit((__force int)bit, &vma->vm_flags);
+
+ return false;
+}
+
+/*
+ * Overwrite the VMA's flags with new ones, preserving atomic VMA flags in the
+ * process. The write is done once and once only.
+ */
+static inline void vma_overwrite_flags(struct vm_area_struct *vma,
+ vma_flags_t new_flags)
+{
+ vma_flags_t atomic;
+
+ vma_assert_write_locked(vma);
+
+ /*
+ * Preserve the VMA's atomic flags, which are kept stable by the VMA
+ * write lock.
+ */
+ atomic = vma_flags_and_mask(&vma->flags, VMA_ATOMIC_SET_ALLOWED);
+ vma_flags_clear_mask(&new_flags, VMA_ATOMIC_SET_ALLOWED);
+ vma_flags_set_mask(&new_flags, atomic);
+
+ /* It is assumed only the first system word must be written once. */
+ vma_flags_overwrite_word_once(&vma->flags, new_flags.__vma_flags[0]);
+ /* The remainder can be copied normally. */
+ if (NUM_VMA_FLAG_BITS > BITS_PER_LONG) {
+ unsigned long *dst = &vma->flags.__vma_flags[1];
+ const unsigned long *src = &new_flags.__vma_flags[1];
+
+ bitmap_copy(dst, src, NUM_VMA_FLAG_BITS - BITS_PER_LONG);
+ }
+}
+
/*
* Test whether a specific VMA flag is set in a VMA descriptor, e.g.:
*
diff --git a/mm/madvise.c b/mm/madvise.c
index 75b409e97dbf2..b2e00efcaa542 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -158,7 +158,7 @@ static int madvise_update_vma(vm_flags_t new_flags,
bool set_new_anon_name = madv_behavior->behavior == __MADV_SET_ANON_VMA_NAME;
VMA_ITERATOR(vmi, madv_behavior->mm, range->start);
- if (vma_flags_same_mask(&vma->flags, new_vma_flags) &&
+ if (vma_flags_same_pair(&vma->flags, &new_vma_flags) &&
(!set_new_anon_name ||
anon_vma_name_eq(anon_vma_name(vma), anon_name)))
return 0;
@@ -177,7 +177,7 @@ static int madvise_update_vma(vm_flags_t new_flags,
/* vm_flags is protected by the mmap_lock held in write mode. */
vma_start_write(vma);
- vma->flags = new_vma_flags;
+ vma_overwrite_flags(vma, new_vma_flags);
/*
* If the vma become good for khugepaged to scan,
* register it here without waiting a page fault that
diff --git a/mm/mlock.c b/mm/mlock.c
index 67ae9b18b9235..1c70a78293944 100644
--- a/mm/mlock.c
+++ b/mm/mlock.c
@@ -432,14 +432,14 @@ static void mlock_vma_pages_range(struct vm_area_struct *vma,
}
vma_start_write(vma);
- vma_flags_reset_once(vma, &walk_flags);
+ vma_overwrite_flags(vma, walk_flags);
lru_add_drain();
walk_page_range_vma(vma, start, end, &mlock_walk_ops, (void *)lock);
lru_add_drain();
if (lock)
- vma_flags_reset_once(vma, new_vma_flags);
+ vma_overwrite_flags(vma, *new_vma_flags);
}
/*
@@ -494,7 +494,7 @@ static int mlock_fixup(struct vma_iterator *vmi, struct vm_area_struct *vma,
vma_flags_test(&old_vma_flags, VMA_LOCKED_BIT)) {
/* No work to do, and mlocking twice would be wrong */
vma_start_write(vma);
- vma->flags = *new_vma_flags;
+ vma_overwrite_flags(vma, *new_vma_flags);
} else {
mlock_vma_pages_range(vma, start, end, new_vma_flags);
}
diff --git a/mm/mprotect.c b/mm/mprotect.c
index 754333698c1ea..e18d4e12fc341 100644
--- a/mm/mprotect.c
+++ b/mm/mprotect.c
@@ -833,7 +833,7 @@ mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb,
* held in write mode.
*/
vma_start_write(vma);
- vma_flags_reset_once(vma, &new_vma_flags);
+ vma_overwrite_flags(vma, new_vma_flags);
if (vma_wants_manual_pte_write_upgrade(vma))
mm_cp_flags |= MM_CP_TRY_CHANGE_WRITABLE;
vma_set_page_prot(vma);
diff --git a/tools/testing/vma/include/dup.h b/tools/testing/vma/include/dup.h
index e0eab4eca998e..a139774054828 100644
--- a/tools/testing/vma/include/dup.h
+++ b/tools/testing/vma/include/dup.h
@@ -366,6 +366,9 @@ enum {
#define VMA_LOCKED_MASK mk_vma_flags(VMA_LOCKED_BIT, VMA_LOCKONFAULT_BIT)
+/* These flags can be updated atomically via VMA/mmap read lock. */
+#define VMA_ATOMIC_SET_ALLOWED mk_vma_flags(VMA_MAYBE_GUARD_BIT)
+
#define RLIMIT_STACK 3 /* max stack size */
#define RLIMIT_MEMLOCK 8 /* max locked-in-memory address space */
@@ -895,22 +898,6 @@ static inline void vm_flags_reset(struct vm_area_struct *vma,
vm_flags_init(vma, flags);
}
-static inline void vma_flags_reset_once(struct vm_area_struct *vma,
- vma_flags_t *flags)
-{
- const unsigned long word = flags->__vma_flags[0];
-
- /* It is assumed only the first system word must be written once. */
- vma_flags_overwrite_word_once(&vma->flags, word);
- /* The remainder can be copied normally. */
- if (NUM_VMA_FLAG_BITS > BITS_PER_LONG) {
- unsigned long *dst = &vma->flags.__vma_flags[1];
- const unsigned long *src = &flags->__vma_flags[1];
-
- bitmap_copy(dst, src, NUM_VMA_FLAG_BITS - BITS_PER_LONG);
- }
-}
-
static inline void vm_flags_set(struct vm_area_struct *vma,
vm_flags_t flags)
{
@@ -1037,27 +1024,24 @@ static __always_inline vma_flags_t vma_flags_diff_pair(const vma_flags_t *flags,
return dst;
}
+/*
+ * Determine if flags and flags_other have precisely the same flags set,
+ * excluding atomic VMA flags.
+ */
static __always_inline bool vma_flags_same_pair(const vma_flags_t *flags,
- const vma_flags_t *flags_other)
+ const vma_flags_t *flags_other)
{
- const unsigned long *bitmap = flags->__vma_flags;
- const unsigned long *bitmap_other = flags_other->__vma_flags;
-
- return bitmap_equal(bitmap, bitmap_other, NUM_VMA_FLAG_BITS);
-}
+ vma_flags_t diff = vma_flags_diff_pair(flags, flags_other);
-static __always_inline bool vma_flags_same_mask(const vma_flags_t *flags,
- vma_flags_t flags_other)
-{
- const unsigned long *bitmap = flags->__vma_flags;
- const unsigned long *bitmap_other = flags_other.__vma_flags;
+ /*
+ * Atomic VMA flags may be set concurrently under the VMA read lock, so
+ * are disregarded for equality testing purposes.
+ */
+ vma_flags_clear_mask(&diff, VMA_ATOMIC_SET_ALLOWED);
- return bitmap_equal(bitmap, bitmap_other, NUM_VMA_FLAG_BITS);
+ return vma_flags_empty(&diff);
}
-#define vma_flags_same(flags, ...) \
- vma_flags_same_mask(flags, mk_vma_flags(__VA_ARGS__))
-
static __always_inline bool vma_test(const struct vm_area_struct *vma,
vma_flag_t bit)
{
@@ -1106,6 +1090,36 @@ static __always_inline void vma_clear_flags_mask(struct vm_area_struct *vma,
#define vma_clear_flags(vma, ...) \
vma_clear_flags_mask(vma, mk_vma_flags(__VA_ARGS__))
+/*
+ * Overwrite the VMA's flags with new ones, preserving atomic VMA flags in the
+ * process. The write is done once and once only.
+ */
+static inline void vma_overwrite_flags(struct vm_area_struct *vma,
+ vma_flags_t new_flags)
+{
+ vma_flags_t atomic;
+
+ vma_assert_write_locked(vma);
+
+ /*
+ * Preserve the VMA's atomic flags, which are kept stable by the VMA
+ * write lock.
+ */
+ atomic = vma_flags_and_mask(&vma->flags, VMA_ATOMIC_SET_ALLOWED);
+ vma_flags_clear_mask(&new_flags, VMA_ATOMIC_SET_ALLOWED);
+ vma_flags_set_mask(&new_flags, atomic);
+
+ /* It is assumed only the first system word must be written once. */
+ vma_flags_overwrite_word_once(&vma->flags, new_flags.__vma_flags[0]);
+ /* The remainder can be copied normally. */
+ if (NUM_VMA_FLAG_BITS > BITS_PER_LONG) {
+ unsigned long *dst = &vma->flags.__vma_flags[1];
+ const unsigned long *src = &new_flags.__vma_flags[1];
+
+ bitmap_copy(dst, src, NUM_VMA_FLAG_BITS - BITS_PER_LONG);
+ }
+}
+
static __always_inline bool vma_desc_test(const struct vm_area_desc *desc,
vma_flag_t bit)
{
diff --git a/tools/testing/vma/shared.h b/tools/testing/vma/shared.h
index 97cd7a679dc19..646edf367d7c4 100644
--- a/tools/testing/vma/shared.h
+++ b/tools/testing/vma/shared.h
@@ -51,17 +51,21 @@
__TO_SCALAR(__val1), __TO_SCALAR(__val2)); \
} while (0)
-#define ASSERT_FLAGS_SAME_MASK(_flags, _flags_other) \
- ASSERT_TRUE(vma_flags_same_mask((_flags), (_flags_other)))
+#define ASSERT_FLAGS_SAME_MASK(_flags, _flags_other) do { \
+ const vma_flags_t __flags_other = (_flags_other); \
+ ASSERT_TRUE(vma_flags_same_pair((_flags), &__flags_other)); \
+ } while (0)
-#define ASSERT_FLAGS_NOT_SAME_MASK(_flags, _flags_other) \
- ASSERT_FALSE(vma_flags_same_mask((_flags), (_flags_other)))
+#define ASSERT_FLAGS_NOT_SAME_MASK(_flags, _flags_other) do { \
+ const vma_flags_t __flags_other = (_flags_other); \
+ ASSERT_FALSE(vma_flags_same_pair((_flags), &__flags_other)); \
+ } while (0)
#define ASSERT_FLAGS_SAME(_flags, ...) \
- ASSERT_TRUE(vma_flags_same(_flags, __VA_ARGS__))
+ ASSERT_FLAGS_SAME_MASK(_flags, mk_vma_flags(__VA_ARGS__))
#define ASSERT_FLAGS_NOT_SAME(_flags, ...) \
- ASSERT_FALSE(vma_flags_same(_flags, __VA_ARGS__))
+ ASSERT_FLAGS_NOT_SAME_MASK(_flags, mk_vma_flags(__VA_ARGS__))
#define ASSERT_FLAGS_EMPTY(_flags) \
ASSERT_TRUE(vma_flags_empty(_flags))
--
2.53.0
^ permalink raw reply related [flat|nested] 2+ messages in thread* [PATCH 7.3.y] mm: do not unintentionally overwrite atomic VMA flags
2026-11-10 1:01 FAILED: patch "[PATCH] mm: do not unintentionally overwrite atomic VMA flags" failed to apply to 7.3-stable tree Sasha Levin
@ 2026-11-11 20:02 ` Sasha Levin
0 siblings, 0 replies; 2+ messages in thread
From: Sasha Levin @ 2026-11-11 20:02 UTC (permalink / raw)
To: stable
Cc: Lorenzo Stoakes (ARM), Andrew Morton, David Hildenbrand (Arm),
Pedro Falcato, Gregory Price (Meta), Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jann Horn, Davidlohr Bueso, SJ Park, Kees Cook, Colin Cross,
Sasha Levin
From: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>
[ Upstream commit 13fc99f46399c9d38d1b54ecf0a3f387ba04d05d ]
Patch series "mm: avoid losing atomic VMA flags", v1.
Since commit 2ab7f1bbafc9 ("mm/madvise: allow guard page install/remove
under VMA lock"), VMA_MAYBE_GUARD_BIT can be set atomically under the VMA
read lock.
However, a number of places wrongly assume the flags are stable under the
read lock, which can cause stale atomic VMA flags to overwrite existing
valid ones.
Additionally, one case in which VMA_SOFTDIRTY_BIT was overwritten is also
addressed.
This series addresses each of these, while keeping the logic inline with
existing conventions.
Each patch in this series is a hotfix.
This patch (of 4):
madvise, mlock and mprotect take a copy of a VMA's flags holding only the
VMA read lock, later writing it back under the write lock (in
madvise_update_vma(), mlock_fixup(), mlock_vma_pages_range() and
mprotect_fixup()).
This was done on the assumption that VMA flags cannot change with a read
lock held.
However this changed in commit 2ab7f1bbafc9 ("mm/madvise: allow guard page
install/remove under VMA lock"), which allowed VM_ATOMIC_SET_ALLOWED
flags (currently only VMA_MAYBE_GUARD_BIT) to be written concurrently under
a VMA read lock.
The actual write itself is safe as this is done under the VMA write lock
at which point no further concurrent writes can occur, however at this
point the stale flags mean that these callers can unintentionally overwrite
atomic flags incorrectly.
The consequences of this are significant - a missing VMA_MAYBE_GUARD_BIT
could result in page tables not being copied on fork (the vma_needs_copy()
check can return false) or being overwritten on file-backed THP
collapse (the file_backed_vma_is_retractable() can return true).
Resolve the issue by always taking care to retain existing atomic flags
when overwriting VMA flags, replacing the inconsistently named
vma_flags_reset_once() with vma_overwrite_flags() to do so, and always
invoking it when flags are overwritten like this.
Reads are a problem too - madvise, mlock and mprotect all compare VMA flags
without a VMA write lock held via vma_flags_same_pair().
This has more serious impact, as concurrent changes to atomic VMA flags can
cause two sets of VMA flags to be wrongly considered different.
For mprotect and madvise this causes a useless split, but mlock is impacted
more seriously as it can result in mm->locked_vm underflow.
Fix this by updating vma_flags_same_pair() to disregard
VM_ATOMIC_SET_ALLOWED VMAs - VMA equality is the same as VMA mergeability,
and atomic VMA flags are disregarded when comparing VMA flags for merge -
so this is justified semantically.
Additionally, remove the redundant vma_flags_same_mask() and the unused
vma_flags_same() as part of the change, and replace VM_ATOMIC_SET_ALLOWED
with VMA_ATOMIC_SET_ALLOWED, using the new VMA flag API.
Finally, update userland VMA tests to reflect the changes.
Link: https://lore.kernel.org/20261023-fix-vma-atomic-v1-0-8ff4fae1fc85@kernel.org
Link: https://lore.kernel.org/20261023-fix-vma-atomic-v1-1-8ff4fae1fc85@kernel.org
Fixes: 2ab7f1bbafc9 ("mm/madvise: allow guard page install/remove under VMA lock")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Suggested-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Tested-by: Gregory Price (Meta) <gourry@gourry.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Jann Horn <jannh@google.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: SJ Park <sj@kernel.org>
Cc: Kees Cook <kees@kernel.org>
Cc: Colin Cross <ccross@google.com>
Cc: <stable@vger.kernel.org>
[ sashal: adapted mlock page-walk flag writes to the existing temporary VMA_IO_BIT protocol instead of walk_flags. ]
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
include/linux/mm.h | 166 ++++++++++++++++----------------
mm/madvise.c | 4 +-
mm/mlock.c | 6 +-
mm/mprotect.c | 2 +-
tools/testing/vma/include/dup.h | 76 +++++++++------
tools/testing/vma/shared.h | 16 +--
6 files changed, 146 insertions(+), 124 deletions(-)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index dd09c438fa23e..cfdf0e3039563 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -622,7 +622,7 @@ enum {
#define VMA_LOCKED_MASK mk_vma_flags(VMA_LOCKED_BIT, VMA_LOCKONFAULT_BIT)
/* These flags can be updated atomically via VMA/mmap read lock. */
-#define VM_ATOMIC_SET_ALLOWED VM_MAYBE_GUARD
+#define VMA_ATOMIC_SET_ALLOWED mk_vma_flags(VMA_MAYBE_GUARD_BIT)
/* Arch-specific flags to clear when updating VM flags on protection change */
#ifndef VM_ARCH_CLEAR
@@ -1019,22 +1019,6 @@ static inline void vm_flags_reset(struct vm_area_struct *vma,
vm_flags_init(vma, flags);
}
-static inline void vma_flags_reset_once(struct vm_area_struct *vma,
- vma_flags_t *flags)
-{
- const unsigned long word = flags->__vma_flags[0];
-
- /* It is assumed only the first system word must be written once. */
- vma_flags_overwrite_word_once(&vma->flags, word);
- /* The remainder can be copied normally. */
- if (NUM_VMA_FLAG_BITS > BITS_PER_LONG) {
- unsigned long *dst = &vma->flags.__vma_flags[1];
- const unsigned long *src = &flags->__vma_flags[1];
-
- bitmap_copy(dst, src, NUM_VMA_FLAG_BITS - BITS_PER_LONG);
- }
-}
-
static inline void vm_flags_set(struct vm_area_struct *vma,
vm_flags_t flags)
{
@@ -1071,48 +1055,6 @@ static inline void vm_flags_mod(struct vm_area_struct *vma,
__vm_flags_mod(vma, set, clear);
}
-static __always_inline bool __vma_atomic_valid_flag(struct vm_area_struct *vma,
- vma_flag_t bit)
-{
- const vm_flags_t mask = BIT((__force int)bit);
-
- /* Only specific flags are permitted */
- if (WARN_ON_ONCE(!(mask & VM_ATOMIC_SET_ALLOWED)))
- return false;
-
- return true;
-}
-
-/*
- * Set VMA flag atomically. Requires only VMA/mmap read lock. Only specific
- * valid flags are allowed to do this.
- */
-static __always_inline void vma_set_atomic_flag(struct vm_area_struct *vma,
- vma_flag_t bit)
-{
- unsigned long *bitmap = vma->flags.__vma_flags;
-
- vma_assert_stabilised(vma);
- if (__vma_atomic_valid_flag(vma, bit))
- set_bit((__force int)bit, bitmap);
-}
-
-/*
- * Test for VMA flag atomically. Requires no locks. Only specific valid flags
- * are allowed to do this.
- *
- * This is necessarily racey, so callers must ensure that serialisation is
- * achieved through some other means, or that races are permissible.
- */
-static __always_inline bool vma_test_atomic_flag(struct vm_area_struct *vma,
- vma_flag_t bit)
-{
- if (__vma_atomic_valid_flag(vma, bit))
- return test_bit((__force int)bit, &vma->vm_flags);
-
- return false;
-}
-
/* Set an individual VMA flag in flags, non-atomically. */
static __always_inline void vma_flags_set_flag(vma_flags_t *flags,
vma_flag_t bit)
@@ -1334,34 +1276,24 @@ static __always_inline vma_flags_t vma_flags_diff_pair(const vma_flags_t *flags,
return dst;
}
-/* Determine if flags and flags_other have precisely the same flags set. */
+/*
+ * Determine if flags and flags_other have precisely the same flags set,
+ * excluding atomic VMA flags.
+ */
static __always_inline bool vma_flags_same_pair(const vma_flags_t *flags,
- const vma_flags_t *flags_other)
+ const vma_flags_t *flags_other)
{
- const unsigned long *bitmap = flags->__vma_flags;
- const unsigned long *bitmap_other = flags_other->__vma_flags;
-
- return bitmap_equal(bitmap, bitmap_other, NUM_VMA_FLAG_BITS);
-}
+ vma_flags_t diff = vma_flags_diff_pair(flags, flags_other);
-/* Determine if flags and flags_other have precisely the same flags set. */
-static __always_inline bool vma_flags_same_mask(const vma_flags_t *flags,
- vma_flags_t flags_other)
-{
- const unsigned long *bitmap = flags->__vma_flags;
- const unsigned long *bitmap_other = flags_other.__vma_flags;
+ /*
+ * Atomic VMA flags may be set concurrently under the VMA read lock, so
+ * are disregarded for equality testing purposes.
+ */
+ vma_flags_clear_mask(&diff, VMA_ATOMIC_SET_ALLOWED);
- return bitmap_equal(bitmap, bitmap_other, NUM_VMA_FLAG_BITS);
+ return vma_flags_empty(&diff);
}
-/*
- * Helper macro to determine if only the specific flags are set, e.g.:
- *
- * if (vma_flags_same(&flags, VMA_WRITE_BIT) { ... }
- */
-#define vma_flags_same(flags, ...) \
- vma_flags_same_mask(flags, mk_vma_flags(__VA_ARGS__))
-
/*
* Test whether a specific flag in the VMA is set, e.g.:
*
@@ -1468,6 +1400,78 @@ static __always_inline void vma_clear_flags_mask(struct vm_area_struct *vma,
#define vma_clear_flags(vma, ...) \
vma_clear_flags_mask(vma, mk_vma_flags(__VA_ARGS__))
+static __always_inline bool __vma_atomic_valid_flag(struct vm_area_struct *vma,
+ vma_flag_t bit)
+{
+ const vma_flags_t mask = VMA_ATOMIC_SET_ALLOWED;
+
+ /* Is this flag allowed to be set atomically? */
+ if (WARN_ON_ONCE(!vma_flags_test(&mask, bit)))
+ return false;
+
+ return true;
+}
+
+/*
+ * Set VMA flag atomically. Requires only VMA/mmap read lock. Only specific
+ * valid flags are allowed to do this.
+ */
+static __always_inline void vma_set_atomic_flag(struct vm_area_struct *vma,
+ vma_flag_t bit)
+{
+ unsigned long *bitmap = vma->flags.__vma_flags;
+
+ vma_assert_stabilised(vma);
+ if (__vma_atomic_valid_flag(vma, bit))
+ set_bit((__force int)bit, bitmap);
+}
+
+/*
+ * Test for VMA flag atomically. Requires no locks. Only specific valid flags
+ * are allowed to do this.
+ *
+ * This is necessarily racey, so callers must ensure that serialisation is
+ * achieved through some other means, or that races are permissible.
+ */
+static __always_inline bool vma_test_atomic_flag(struct vm_area_struct *vma,
+ vma_flag_t bit)
+{
+ if (__vma_atomic_valid_flag(vma, bit))
+ return test_bit((__force int)bit, &vma->vm_flags);
+
+ return false;
+}
+
+/*
+ * Overwrite the VMA's flags with new ones, preserving atomic VMA flags in the
+ * process. The write is done once and once only.
+ */
+static inline void vma_overwrite_flags(struct vm_area_struct *vma,
+ vma_flags_t new_flags)
+{
+ vma_flags_t atomic;
+
+ vma_assert_write_locked(vma);
+
+ /*
+ * Preserve the VMA's atomic flags, which are kept stable by the VMA
+ * write lock.
+ */
+ atomic = vma_flags_and_mask(&vma->flags, VMA_ATOMIC_SET_ALLOWED);
+ vma_flags_clear_mask(&new_flags, VMA_ATOMIC_SET_ALLOWED);
+ vma_flags_set_mask(&new_flags, atomic);
+
+ /* It is assumed only the first system word must be written once. */
+ vma_flags_overwrite_word_once(&vma->flags, new_flags.__vma_flags[0]);
+ /* The remainder can be copied normally. */
+ if (NUM_VMA_FLAG_BITS > BITS_PER_LONG) {
+ unsigned long *dst = &vma->flags.__vma_flags[1];
+ const unsigned long *src = &new_flags.__vma_flags[1];
+
+ bitmap_copy(dst, src, NUM_VMA_FLAG_BITS - BITS_PER_LONG);
+ }
+}
+
/*
* Test whether a specific VMA flag is set in a VMA descriptor, e.g.:
*
diff --git a/mm/madvise.c b/mm/madvise.c
index 222c10eb61eb4..9d64e15dfdb48 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -158,7 +158,7 @@ static int madvise_update_vma(vm_flags_t new_flags,
bool set_new_anon_name = madv_behavior->behavior == __MADV_SET_ANON_VMA_NAME;
VMA_ITERATOR(vmi, madv_behavior->mm, range->start);
- if (vma_flags_same_mask(&vma->flags, new_vma_flags) &&
+ if (vma_flags_same_pair(&vma->flags, &new_vma_flags) &&
(!set_new_anon_name ||
anon_vma_name_eq(anon_vma_name(vma), anon_name)))
return 0;
@@ -177,7 +177,7 @@ static int madvise_update_vma(vm_flags_t new_flags,
/* vm_flags is protected by the mmap_lock held in write mode. */
vma_start_write(vma);
- vma->flags = new_vma_flags;
+ vma_overwrite_flags(vma, new_vma_flags);
/*
* If the vma become good for khugepaged to scan,
* register it here without waiting a page fault that
diff --git a/mm/mlock.c b/mm/mlock.c
index 4fb43cddafa84..d88683ca2a6fe 100644
--- a/mm/mlock.c
+++ b/mm/mlock.c
@@ -443,7 +443,7 @@ static void mlock_vma_pages_range(struct vm_area_struct *vma,
if (vma_flags_test(new_vma_flags, VMA_LOCKED_BIT))
vma_flags_set(new_vma_flags, VMA_IO_BIT);
vma_start_write(vma);
- vma_flags_reset_once(vma, new_vma_flags);
+ vma_overwrite_flags(vma, *new_vma_flags);
lru_add_drain();
walk_page_range_vma(vma, start, end, &mlock_walk_ops, NULL);
@@ -451,7 +451,7 @@ static void mlock_vma_pages_range(struct vm_area_struct *vma,
if (vma_flags_test(new_vma_flags, VMA_IO_BIT)) {
vma_flags_clear(new_vma_flags, VMA_IO_BIT);
- vma_flags_reset_once(vma, new_vma_flags);
+ vma_overwrite_flags(vma, *new_vma_flags);
}
}
@@ -507,7 +507,7 @@ static int mlock_fixup(struct vma_iterator *vmi, struct vm_area_struct *vma,
vma_flags_test(&old_vma_flags, VMA_LOCKED_BIT)) {
/* No work to do, and mlocking twice would be wrong */
vma_start_write(vma);
- vma->flags = *new_vma_flags;
+ vma_overwrite_flags(vma, *new_vma_flags);
} else {
mlock_vma_pages_range(vma, start, end, new_vma_flags);
}
diff --git a/mm/mprotect.c b/mm/mprotect.c
index 2888ee638d872..e923a3bf4f680 100644
--- a/mm/mprotect.c
+++ b/mm/mprotect.c
@@ -834,7 +834,7 @@ mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb,
* held in write mode.
*/
vma_start_write(vma);
- vma_flags_reset_once(vma, &new_vma_flags);
+ vma_overwrite_flags(vma, new_vma_flags);
if (vma_wants_manual_pte_write_upgrade(vma))
mm_cp_flags |= MM_CP_TRY_CHANGE_WRITABLE;
vma_set_page_prot(vma);
diff --git a/tools/testing/vma/include/dup.h b/tools/testing/vma/include/dup.h
index 4c58487b764e9..e7724b2ca2ff8 100644
--- a/tools/testing/vma/include/dup.h
+++ b/tools/testing/vma/include/dup.h
@@ -374,6 +374,9 @@ enum {
#define VMA_LOCKED_MASK mk_vma_flags(VMA_LOCKED_BIT, VMA_LOCKONFAULT_BIT)
+/* These flags can be updated atomically via VMA/mmap read lock. */
+#define VMA_ATOMIC_SET_ALLOWED mk_vma_flags(VMA_MAYBE_GUARD_BIT)
+
#define RLIMIT_STACK 3 /* max stack size */
#define RLIMIT_MEMLOCK 8 /* max locked-in-memory address space */
@@ -899,22 +902,6 @@ static inline void vm_flags_reset(struct vm_area_struct *vma,
vm_flags_init(vma, flags);
}
-static inline void vma_flags_reset_once(struct vm_area_struct *vma,
- vma_flags_t *flags)
-{
- const unsigned long word = flags->__vma_flags[0];
-
- /* It is assumed only the first system word must be written once. */
- vma_flags_overwrite_word_once(&vma->flags, word);
- /* The remainder can be copied normally. */
- if (NUM_VMA_FLAG_BITS > BITS_PER_LONG) {
- unsigned long *dst = &vma->flags.__vma_flags[1];
- const unsigned long *src = &flags->__vma_flags[1];
-
- bitmap_copy(dst, src, NUM_VMA_FLAG_BITS - BITS_PER_LONG);
- }
-}
-
static inline void vm_flags_set(struct vm_area_struct *vma,
vm_flags_t flags)
{
@@ -1041,27 +1028,24 @@ static __always_inline vma_flags_t vma_flags_diff_pair(const vma_flags_t *flags,
return dst;
}
+/*
+ * Determine if flags and flags_other have precisely the same flags set,
+ * excluding atomic VMA flags.
+ */
static __always_inline bool vma_flags_same_pair(const vma_flags_t *flags,
- const vma_flags_t *flags_other)
+ const vma_flags_t *flags_other)
{
- const unsigned long *bitmap = flags->__vma_flags;
- const unsigned long *bitmap_other = flags_other->__vma_flags;
-
- return bitmap_equal(bitmap, bitmap_other, NUM_VMA_FLAG_BITS);
-}
+ vma_flags_t diff = vma_flags_diff_pair(flags, flags_other);
-static __always_inline bool vma_flags_same_mask(const vma_flags_t *flags,
- vma_flags_t flags_other)
-{
- const unsigned long *bitmap = flags->__vma_flags;
- const unsigned long *bitmap_other = flags_other.__vma_flags;
+ /*
+ * Atomic VMA flags may be set concurrently under the VMA read lock, so
+ * are disregarded for equality testing purposes.
+ */
+ vma_flags_clear_mask(&diff, VMA_ATOMIC_SET_ALLOWED);
- return bitmap_equal(bitmap, bitmap_other, NUM_VMA_FLAG_BITS);
+ return vma_flags_empty(&diff);
}
-#define vma_flags_same(flags, ...) \
- vma_flags_same_mask(flags, mk_vma_flags(__VA_ARGS__))
-
static __always_inline bool vma_test(const struct vm_area_struct *vma,
vma_flag_t bit)
{
@@ -1110,6 +1094,36 @@ static __always_inline void vma_clear_flags_mask(struct vm_area_struct *vma,
#define vma_clear_flags(vma, ...) \
vma_clear_flags_mask(vma, mk_vma_flags(__VA_ARGS__))
+/*
+ * Overwrite the VMA's flags with new ones, preserving atomic VMA flags in the
+ * process. The write is done once and once only.
+ */
+static inline void vma_overwrite_flags(struct vm_area_struct *vma,
+ vma_flags_t new_flags)
+{
+ vma_flags_t atomic;
+
+ vma_assert_write_locked(vma);
+
+ /*
+ * Preserve the VMA's atomic flags, which are kept stable by the VMA
+ * write lock.
+ */
+ atomic = vma_flags_and_mask(&vma->flags, VMA_ATOMIC_SET_ALLOWED);
+ vma_flags_clear_mask(&new_flags, VMA_ATOMIC_SET_ALLOWED);
+ vma_flags_set_mask(&new_flags, atomic);
+
+ /* It is assumed only the first system word must be written once. */
+ vma_flags_overwrite_word_once(&vma->flags, new_flags.__vma_flags[0]);
+ /* The remainder can be copied normally. */
+ if (NUM_VMA_FLAG_BITS > BITS_PER_LONG) {
+ unsigned long *dst = &vma->flags.__vma_flags[1];
+ const unsigned long *src = &new_flags.__vma_flags[1];
+
+ bitmap_copy(dst, src, NUM_VMA_FLAG_BITS - BITS_PER_LONG);
+ }
+}
+
static __always_inline bool vma_desc_test(const struct vm_area_desc *desc,
vma_flag_t bit)
{
diff --git a/tools/testing/vma/shared.h b/tools/testing/vma/shared.h
index 97cd7a679dc19..646edf367d7c4 100644
--- a/tools/testing/vma/shared.h
+++ b/tools/testing/vma/shared.h
@@ -51,17 +51,21 @@
__TO_SCALAR(__val1), __TO_SCALAR(__val2)); \
} while (0)
-#define ASSERT_FLAGS_SAME_MASK(_flags, _flags_other) \
- ASSERT_TRUE(vma_flags_same_mask((_flags), (_flags_other)))
+#define ASSERT_FLAGS_SAME_MASK(_flags, _flags_other) do { \
+ const vma_flags_t __flags_other = (_flags_other); \
+ ASSERT_TRUE(vma_flags_same_pair((_flags), &__flags_other)); \
+ } while (0)
-#define ASSERT_FLAGS_NOT_SAME_MASK(_flags, _flags_other) \
- ASSERT_FALSE(vma_flags_same_mask((_flags), (_flags_other)))
+#define ASSERT_FLAGS_NOT_SAME_MASK(_flags, _flags_other) do { \
+ const vma_flags_t __flags_other = (_flags_other); \
+ ASSERT_FALSE(vma_flags_same_pair((_flags), &__flags_other)); \
+ } while (0)
#define ASSERT_FLAGS_SAME(_flags, ...) \
- ASSERT_TRUE(vma_flags_same(_flags, __VA_ARGS__))
+ ASSERT_FLAGS_SAME_MASK(_flags, mk_vma_flags(__VA_ARGS__))
#define ASSERT_FLAGS_NOT_SAME(_flags, ...) \
- ASSERT_FALSE(vma_flags_same(_flags, __VA_ARGS__))
+ ASSERT_FLAGS_NOT_SAME_MASK(_flags, mk_vma_flags(__VA_ARGS__))
#define ASSERT_FLAGS_EMPTY(_flags) \
ASSERT_TRUE(vma_flags_empty(_flags))
--
2.53.0
^ permalink raw reply related [flat|nested] 2+ messages in thread
end of thread, other threads:[~2026-11-11 20:02 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-10 1:01 FAILED: patch "[PATCH] mm: do not unintentionally overwrite atomic VMA flags" failed to apply to 7.3-stable tree Sasha Levin
2026-11-11 20:02 ` [PATCH 7.3.y] mm: do not unintentionally overwrite atomic VMA flags Sasha Levin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox