[PATCH v3 0/5] mm: Unconditional per-VMA locks and cleanups

[PATCH v3 0/5] mm: Unconditional per-VMA locks and cleanups

[Suren Baghdasaryan]

v2 version of this patchset [1] was written by Dave Hansen and per his
request, I'm taking over this series.

tl;dr: Make per-VMA locks available in all configs. Simplify some
of the per-VMA lock users now that they can rely on them being
always available.

Binder and networking folks: Your code is the target of the cleanups.
I'm cc'ing you now on v2 because there's emerging consensus on the mm
side that the approach here is sane. I'm not quite sure how this pile
would get merged, but ack/review tags would be appreciated if this
looks good to you.

Longer version:

When working on some x86 shadow stack code, it was a real pain to
avoid causing recursive locking problems with mmap_lock. One way
to avoid those was to avoid mmap_lock and use per-VMA locks instead.
They are great, but they are not available in all configs which
makes them unusable in generic code, or if you want to completely
avoid mmap_lock.

Make per-VMA locks available in all configs. Right now, they are
only available on select architectures when SMP and MMU are enabled.
But all of the primitives that per-VMA locks are built on (RCU, maple
trees, refcounts) work just fine without SMP or MMU.

The only real downside is that making VMAs a wee bit bigger on !MMU
and !SMP builds.

The upside is much cleaner code, lower complexity and less #ifdeffery.

Clean up a binder VMA locking site now that it can rely on per-VMA
locks.

Building on top of universally-available per-VMA locks, introduce a
new helper. Since the new API does not require callers to have a
fallback to mmap_lock, it's much easier to use. Callers can
potentially replace this very common kernel idiom:

	mmap_read_lock(mm);
	vma = vma_lookup()
	// fiddle with vma
	mmap_read_unlock(mm);

with:

	vma = vma_start_read_unlocked(mm, address);
	// fiddle with vma
	vma_end_read(vma);

Which avoids mmap_lock entirely in the fast path.

Use that new API for another binder site and one in the TCP code.

Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: linux-mm@kvack.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Arve Hjønnevåg <arve@android.com>
Cc: Todd Kjos <tkjos@android.com>
Cc: Christian Brauner <christian@brauner.io>
Cc: Carlos Llamas <cmllamas@google.com>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: David Ahern <dsahern@kernel.org>
Cc: netdev@vger.kernel.org

Changes from v2 [1]:
Patch 1
- Removed new CONFIG_PER_VMA_LOCK usage in kernel/bpf/stackmap.c
- Modified lock_vma_under_rcu Rust implementation, per Alice Ryhl
Patch 2
- Added binder_alloc_is_mapped() check to allow page freeing if the vma is
unmapped, per Alice Ryhl and Carlos Llamas
- Removed earlier Reviewed-by's and Acked-by's
Patch 3
- Modified comments for vma_start_read_locked_nested(),
vma_start_read_locked() and uffd_lock_vma(), per Vlastimil Babka
- Updated vma_start_read_unlocked() to call vma_lookup(), per Dave Hansen
- Added a check for VMA to be valid before calling
vma_start_read_locked(), per Vlastimil Babka
- Replaced guard with mmap_read_lock/mmap_read_unlock
- Modified the comment to indicate mmap_locking is temporary,
per Vlastimil Babka
Patch 4
- Removed earlier Reviewed-by's and Acked-by's
- Added Rust version of vma_start_read_unlocked() and used it in Rust
version of the binder driver, per Alice Ryhl
Patch 5
- Replaced lock_vma_under_rcu_wait() with vma_start_read_unlocked() in the
changelog, per Vlastimil Babka

Applies cleanly over mm-unstable

[1] https://lore.kernel.org/all/20260610230409.A44D29FA@davehans-spike.ostc.intel.com/

Dave Hansen (5):
  mm: Make per-VMA locks available universally
  binder: Make shrinker rely solely on per-VMA lock
  mm: Add RCU-based VMA lookup helper that waits for writers
  binder: Remove mmap_lock fallback
  tcp: Remove mmap_lock fallback path

 arch/arm/Kconfig                       |  1 -
 arch/arm64/Kconfig                     |  1 -
 arch/loongarch/Kconfig                 |  1 -
 arch/powerpc/platforms/powernv/Kconfig |  1 -
 arch/powerpc/platforms/pseries/Kconfig |  1 -
 arch/riscv/Kconfig                     |  1 -
 arch/s390/Kconfig                      |  1 -
 arch/x86/Kconfig                       |  2 -
 drivers/android/binder/page_range.rs   | 19 +-----
 drivers/android/binder_alloc.c         | 46 ++++++-------
 fs/proc/internal.h                     |  2 -
 fs/proc/task_mmu.c                     | 93 --------------------------
 include/linux/mm.h                     | 12 ----
 include/linux/mm_types.h               |  8 +--
 include/linux/mmap_lock.h              | 65 +++---------------
 kernel/bpf/stackmap.c                  | 16 +----
 kernel/bpf/task_iter.c                 |  5 --
 kernel/fork.c                          |  2 -
 mm/Kconfig                             | 13 ----
 mm/Kconfig.debug                       |  1 -
 mm/debug.c                             |  4 --
 mm/init-mm.c                           |  2 -
 mm/memory.c                            |  2 -
 mm/mmap_lock.c                         | 53 ++++++++-------
 mm/pagewalk.c                          |  2 -
 mm/rmap.c                              |  2 -
 mm/userfaultfd.c                       | 61 ++---------------
 net/ipv4/tcp.c                         | 31 +++------
 rust/kernel/mm.rs                      | 38 +++++++----
 tools/testing/vma/include/dup.h        |  5 +-
 tools/testing/vma/vma_internal.h       |  1 -
 31 files changed, 106 insertions(+), 386 deletions(-)


base-commit: 94f9b3980dd446b56acf1dfed649e9b32a9f3813
-- 
2.55.0.508.g3f0d502094-goog

[PATCH v3 1/5] mm: Make per-VMA locks available universally

[Suren Baghdasaryan]

From: Dave Hansen <dave.hansen@linux.intel.com>

The per-VMA locks have been around for several years. They've had some
bugs worked out of them and have seen quite wide use. However, they
are still only available when architectures explicitly enable them.
Remove the conditional compilation around the per-VMA locks, making
them available on all architectures and configs.

The approach up to now seemed to be to add ARCH_SUPPORTS_PER_VMA_LOCK
when the architecture started using per-VMA locks in the fault
handler. But, contrary to the naming, the Kconfig option does not
really indicate whether the architecture supports per-VMA locks or
not. It is more of a marker for whether the architecture is likely to
benefit from per-VMA locks.

To me, the most important thing side-effect of universal availability
is letting per-VMA locks be used in SMP=n configs. This lets us use
per-VMA locking in all x86 code without fallbacks.

Overall, this just generally makes the kernel simpler. Just look at
the diffstat. It also opens the door to users that want to use the
per-VMA locks in common code. Doing *that* brings additional
simplifications.

The downside of this is adding some fields to vm_area_struct and
mm_struct. There are likely ways to optimize this, especially for
things like SMP=n configs. For now, do the simplest thing: use the
same implementation everywhere.

Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: linux-mm@kvack.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Arve Hjønnevåg <arve@android.com>
Cc: Todd Kjos <tkjos@android.com>
Cc: Christian Brauner <christian@brauner.io>
Cc: Carlos Llamas <cmllamas@google.com>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: David Ahern <dsahern@kernel.org>
Cc: netdev@vger.kernel.org
---
 arch/arm/Kconfig                       |  1 -
 arch/arm64/Kconfig                     |  1 -
 arch/loongarch/Kconfig                 |  1 -
 arch/powerpc/platforms/powernv/Kconfig |  1 -
 arch/powerpc/platforms/pseries/Kconfig |  1 -
 arch/riscv/Kconfig                     |  1 -
 arch/s390/Kconfig                      |  1 -
 arch/x86/Kconfig                       |  2 -
 fs/proc/internal.h                     |  2 -
 fs/proc/task_mmu.c                     | 93 --------------------------
 include/linux/mm.h                     | 12 ----
 include/linux/mm_types.h               |  8 +--
 include/linux/mmap_lock.h              | 50 --------------
 kernel/bpf/stackmap.c                  | 16 +----
 kernel/bpf/task_iter.c                 |  5 --
 kernel/fork.c                          |  2 -
 mm/Kconfig                             | 13 ----
 mm/Kconfig.debug                       |  1 -
 mm/debug.c                             |  4 --
 mm/init-mm.c                           |  2 -
 mm/memory.c                            |  2 -
 mm/mmap_lock.c                         | 24 -------
 mm/pagewalk.c                          |  2 -
 mm/rmap.c                              |  2 -
 mm/userfaultfd.c                       | 55 ---------------
 rust/kernel/mm.rs                      | 22 +++---
 tools/testing/vma/include/dup.h        |  5 +-
 tools/testing/vma/vma_internal.h       |  1 -
 28 files changed, 13 insertions(+), 317 deletions(-)

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 9187240a02db..f815209167cd 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -41,7 +41,6 @@ config ARM
 	select ARCH_SUPPORTS_ATOMIC_RMW
 	select ARCH_SUPPORTS_CFI
 	select ARCH_SUPPORTS_HUGETLBFS if ARM_LPAE
-	select ARCH_SUPPORTS_PER_VMA_LOCK
 	select ARCH_SUPPORTS_RT
 	select ARCH_USE_BUILTIN_BSWAP
 	select ARCH_USE_CMPXCHG_LOCKREF
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 11a9c534b7b4..21eb64b24a2c 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -81,7 +81,6 @@ config ARM64
 	select ARCH_HAS_PTE_PROTNONE
 	select ARCH_SUPPORTS_NUMA_BALANCING
 	select ARCH_SUPPORTS_PAGE_TABLE_CHECK
-	select ARCH_SUPPORTS_PER_VMA_LOCK
 	select ARCH_SUPPORTS_HUGE_PFNMAP if TRANSPARENT_HUGEPAGE
 	select ARCH_SUPPORTS_RT
 	select ARCH_SUPPORTS_SCHED_SMT
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index e20acbe5fe7b..7741e39eca2b 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -69,7 +69,6 @@ config LOONGARCH
 	select ARCH_SUPPORTS_MSEAL_SYSTEM_MAPPINGS
 	select ARCH_HAS_PTE_PROTNONE if 64BIT
 	select ARCH_SUPPORTS_NUMA_BALANCING if NUMA
-	select ARCH_SUPPORTS_PER_VMA_LOCK
 	select ARCH_SUPPORTS_RT
 	select ARCH_SUPPORTS_SCHED_SMT if SMP
 	select ARCH_SUPPORTS_SCHED_MC  if SMP
diff --git a/arch/powerpc/platforms/powernv/Kconfig b/arch/powerpc/platforms/powernv/Kconfig
index b5ad7c173ef0..dd8f6060fb7a 100644
--- a/arch/powerpc/platforms/powernv/Kconfig
+++ b/arch/powerpc/platforms/powernv/Kconfig
@@ -17,7 +17,6 @@ config PPC_POWERNV
 	select PPC_DOORBELL
 	select MMU_NOTIFIER
 	select FORCE_SMP
-	select ARCH_SUPPORTS_PER_VMA_LOCK
 	select PPC_RADIX_BROADCAST_TLBIE if PPC_RADIX_MMU
 	default y
 
diff --git a/arch/powerpc/platforms/pseries/Kconfig b/arch/powerpc/platforms/pseries/Kconfig
index 74910ce3a541..7d125e288f6e 100644
--- a/arch/powerpc/platforms/pseries/Kconfig
+++ b/arch/powerpc/platforms/pseries/Kconfig
@@ -23,7 +23,6 @@ config PPC_PSERIES
 	select HOTPLUG_CPU
 	select FORCE_SMP
 	select SWIOTLB
-	select ARCH_SUPPORTS_PER_VMA_LOCK
 	select PPC_RADIX_BROADCAST_TLBIE if PPC_RADIX_MMU
 	default y
 
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 7b9c373d82fa..faa85a031fe5 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -70,7 +70,6 @@ config RISCV
 	select ARCH_SUPPORTS_LTO_CLANG_THIN
 	select ARCH_SUPPORTS_MSEAL_SYSTEM_MAPPINGS if 64BIT && MMU
 	select ARCH_SUPPORTS_PAGE_TABLE_CHECK if MMU
-	select ARCH_SUPPORTS_PER_VMA_LOCK if MMU
 	select ARCH_HAS_PTE_PROTNONE if MMU
 	select ARCH_SUPPORTS_RT
 	select ARCH_SUPPORTS_SHADOW_CALL_STACK if HAVE_SHADOW_CALL_STACK
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index ab8fccc2cc4e..d1274bca8c39 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -151,7 +151,6 @@ config S390
 	select ARCH_HAS_PTE_PROTNONE
 	select ARCH_SUPPORTS_NUMA_BALANCING
 	select ARCH_SUPPORTS_PAGE_TABLE_CHECK
-	select ARCH_SUPPORTS_PER_VMA_LOCK
 	select ARCH_USE_BUILTIN_BSWAP
 	select ARCH_USE_CMPXCHG_LOCKREF
 	select ARCH_USE_SYM_ANNOTATIONS
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index fb298e219179..79479d29576f 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -27,7 +27,6 @@ config X86_64
 	select ARCH_HAS_GIGANTIC_PAGE
 	select ARCH_SUPPORTS_MSEAL_SYSTEM_MAPPINGS
 	select ARCH_SUPPORTS_INT128 if CC_HAS_INT128
-	select ARCH_SUPPORTS_PER_VMA_LOCK
 	select ARCH_SUPPORTS_HUGE_PFNMAP if TRANSPARENT_HUGEPAGE
 	select HAVE_ARCH_SOFT_DIRTY
 	select MODULES_USE_ELF_RELA
@@ -1846,7 +1845,6 @@ config X86_USER_SHADOW_STACK
 	bool "X86 userspace shadow stack"
 	depends on AS_WRUSS
 	depends on X86_64
-	depends on PER_VMA_LOCK
 	select ARCH_USES_HIGH_VMA_FLAGS
 	select ARCH_HAS_USER_SHADOW_STACK
 	select X86_CET
diff --git a/fs/proc/internal.h b/fs/proc/internal.h
index b232e1098117..6713757da099 100644
--- a/fs/proc/internal.h
+++ b/fs/proc/internal.h
@@ -385,10 +385,8 @@ struct mem_size_stats;
 
 struct proc_maps_locking_ctx {
 	struct mm_struct *mm;
-#ifdef CONFIG_PER_VMA_LOCK
 	bool mmap_locked;
 	struct vm_area_struct *locked_vma;
-#endif
 };
 
 struct proc_maps_private {
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 817e3e0f9194..096bf0b0b9e0 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -130,8 +130,6 @@ static void release_task_mempolicy(struct proc_maps_private *priv)
 }
 #endif
 
-#ifdef CONFIG_PER_VMA_LOCK
-
 static inline int lock_ctx_mm(struct proc_maps_locking_ctx *lock_ctx)
 {
 	int ret = mmap_read_lock_killable(lock_ctx->mm);
@@ -233,46 +231,6 @@ static inline void reacquire_rcu(struct proc_maps_private *priv)
 	vma_iter_set(&priv->iter, priv->lock_ctx.locked_vma->vm_end);
 }
 
-#else /* CONFIG_PER_VMA_LOCK */
-
-static inline int lock_ctx_mm(struct proc_maps_locking_ctx *lock_ctx)
-{
-	return mmap_read_lock_killable(lock_ctx->mm);
-}
-
-static inline void unlock_ctx_mm(struct proc_maps_locking_ctx *lock_ctx)
-{
-	mmap_read_unlock(lock_ctx->mm);
-}
-
-static inline bool lock_vma_range(struct seq_file *m,
-				  struct proc_maps_locking_ctx *lock_ctx)
-{
-	return lock_ctx_mm(lock_ctx) == 0;
-}
-
-static inline void unlock_vma_range(struct proc_maps_locking_ctx *lock_ctx)
-{
-	unlock_ctx_mm(lock_ctx);
-}
-
-static struct vm_area_struct *get_next_vma(struct proc_maps_private *priv,
-					   loff_t last_pos)
-{
-	return vma_next(&priv->iter);
-}
-
-static inline bool fallback_to_mmap_lock(struct proc_maps_private *priv,
-					 loff_t pos)
-{
-	return false;
-}
-
-static inline void drop_rcu(struct proc_maps_private *priv) {}
-static inline void reacquire_rcu(struct proc_maps_private *priv) {}
-
-#endif /* CONFIG_PER_VMA_LOCK */
-
 static struct vm_area_struct *proc_get_vma(struct seq_file *m, loff_t *ppos)
 {
 	struct proc_maps_private *priv = m->private;
@@ -560,8 +518,6 @@ static int pid_maps_open(struct inode *inode, struct file *file)
 		PROCMAP_QUERY_VMA_FLAGS				\
 )
 
-#ifdef CONFIG_PER_VMA_LOCK
-
 static int query_vma_setup(struct proc_maps_locking_ctx *lock_ctx)
 {
 	reset_lock_ctx(lock_ctx);
@@ -612,26 +568,6 @@ static struct vm_area_struct *query_vma_find_by_addr(struct proc_maps_locking_ct
 	return vma;
 }
 
-#else /* CONFIG_PER_VMA_LOCK */
-
-static int query_vma_setup(struct proc_maps_locking_ctx *lock_ctx)
-{
-	return mmap_read_lock_killable(lock_ctx->mm);
-}
-
-static void query_vma_teardown(struct proc_maps_locking_ctx *lock_ctx)
-{
-	mmap_read_unlock(lock_ctx->mm);
-}
-
-static struct vm_area_struct *query_vma_find_by_addr(struct proc_maps_locking_ctx *lock_ctx,
-						     unsigned long addr)
-{
-	return find_vma(lock_ctx->mm, addr);
-}
-
-#endif  /* CONFIG_PER_VMA_LOCK */
-
 static struct vm_area_struct *query_matching_vma(struct proc_maps_locking_ctx *lock_ctx,
 						 unsigned long addr, u32 flags)
 {
@@ -1314,8 +1250,6 @@ static const struct mm_walk_ops smaps_shmem_walk_ops = {
 	.walk_lock		= PGWALK_RDLOCK,
 };
 
-#ifdef CONFIG_PER_VMA_LOCK
-
 static const struct mm_walk_ops smaps_walk_vma_lock_ops = {
 	.pmd_entry		= smaps_pte_range,
 	.hugetlb_entry		= smaps_hugetlb_range,
@@ -1345,22 +1279,6 @@ get_smaps_shmem_walk_ops(struct proc_maps_private *priv)
 	return &smaps_shmem_walk_vma_lock_ops;
 }
 
-#else /* CONFIG_PER_VMA_LOCK */
-
-static inline const struct mm_walk_ops *
-get_smaps_walk_ops(struct proc_maps_private *priv)
-{
-	return &smaps_walk_ops;
-}
-
-static inline const struct mm_walk_ops *
-get_smaps_shmem_walk_ops(struct proc_maps_private *priv)
-{
-	return &smaps_shmem_walk_ops;
-}
-
-#endif /* CONFIG_PER_VMA_LOCK */
-
 /*
  * Gather mem stats from @vma with the indicated beginning
  * address @start, and keep them in @mss.
@@ -3497,7 +3415,6 @@ static const struct mm_walk_ops show_numa_ops = {
 	.walk_lock = PGWALK_RDLOCK,
 };
 
-#ifdef CONFIG_PER_VMA_LOCK
 static const struct mm_walk_ops show_numa_vma_lock_ops = {
 	.hugetlb_entry = gather_hugetlb_stats,
 	.pmd_entry = gather_pte_stats,
@@ -3512,16 +3429,6 @@ get_show_numa_ops(struct proc_maps_private *priv)
 	return &show_numa_vma_lock_ops;
 }
 
-#else /* CONFIG_PER_VMA_LOCK */
-
-static inline const struct mm_walk_ops *
-get_show_numa_ops(struct proc_maps_private *priv)
-{
-	return &show_numa_ops;
-}
-
-#endif /* CONFIG_PER_VMA_LOCK */
-
 /*
  * Display pages allocated per node and memory policy via /proc.
  */
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 7fabe6c66b4b..d9850f846242 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -931,7 +931,6 @@ static inline void vma_numab_state_free(struct vm_area_struct *vma) {}
  * These must be here rather than mmap_lock.h as dependent on vm_fault type,
  * declared in this header.
  */
-#ifdef CONFIG_PER_VMA_LOCK
 static inline void release_fault_lock(struct vm_fault *vmf)
 {
 	if (vmf->flags & FAULT_FLAG_VMA_LOCK)
@@ -947,17 +946,6 @@ static inline void assert_fault_locked(const struct vm_fault *vmf)
 	else
 		mmap_assert_locked(vmf->vma->vm_mm);
 }
-#else
-static inline void release_fault_lock(struct vm_fault *vmf)
-{
-	mmap_read_unlock(vmf->vma->vm_mm);
-}
-
-static inline void assert_fault_locked(const struct vm_fault *vmf)
-{
-	mmap_assert_locked(vmf->vma->vm_mm);
-}
-#endif /* CONFIG_PER_VMA_LOCK */
 
 static inline bool mm_flags_test(int flag, const struct mm_struct *mm)
 {
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index b5d4cd3b067b..d8e246fd09d3 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -950,7 +950,6 @@ struct vm_area_struct {
 		vma_flags_t flags;
 	};
 
-#ifdef CONFIG_PER_VMA_LOCK
 	/*
 	 * Can only be written (using WRITE_ONCE()) while holding both:
 	 *  - mmap_lock (in write mode)
@@ -966,7 +965,7 @@ struct vm_area_struct {
 	 * slowpath.
 	 */
 	unsigned int vm_lock_seq;
-#endif
+
 	/*
 	 * Low 32-bits of virtual page offset.
 	 * See vma_start_virt_pgoff() comment for details.
@@ -1003,7 +1002,6 @@ struct vm_area_struct {
 #ifdef CONFIG_NUMA_BALANCING
 	struct vma_numab_state *numab_state;	/* NUMA Balancing state */
 #endif
-#ifdef CONFIG_PER_VMA_LOCK
 	/*
 	 * Used to keep track of firstly, whether the VMA is attached, secondly,
 	 * if attached, how many read locks are taken, and thirdly, if the
@@ -1046,7 +1044,6 @@ struct vm_area_struct {
 #ifdef CONFIG_DEBUG_LOCK_ALLOC
 	struct lockdep_map vmlock_dep_map;
 #endif
-#endif
 #ifdef CONFIG_64BIT
 	/*
 	 * High 32-bits of virtual page offset.
@@ -1254,7 +1251,6 @@ struct mm_struct {
 					  * init_mm.mmlist, and are protected
 					  * by mmlist_lock
 					  */
-#ifdef CONFIG_PER_VMA_LOCK
 		struct rcuwait vma_writer_wait;
 		/*
 		 * This field has lock-like semantics, meaning it is sometimes
@@ -1274,7 +1270,7 @@ struct mm_struct {
 		 * mmap_lock.
 		 */
 		seqcount_t mm_lock_seq;
-#endif
+
 		struct futex_mm_data	futex;
 
 		unsigned long hiwater_rss; /* High-watermark of RSS usage */
diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
index 87f77e3da77f..eb32b482434e 100644
--- a/include/linux/mmap_lock.h
+++ b/include/linux/mmap_lock.h
@@ -76,8 +76,6 @@ static inline void mmap_assert_write_locked(const struct mm_struct *mm)
 	rwsem_assert_held_write(&mm->mmap_lock);
 }
 
-#ifdef CONFIG_PER_VMA_LOCK
-
 #ifdef CONFIG_LOCKDEP
 #define __vma_lockdep_map(vma) (&vma->vmlock_dep_map)
 #else
@@ -484,54 +482,6 @@ struct vm_area_struct *lock_next_vma(struct mm_struct *mm,
 				     struct vma_iterator *iter,
 				     unsigned long address);
 
-#else /* CONFIG_PER_VMA_LOCK */
-
-static inline void mm_lock_seqcount_init(struct mm_struct *mm) {}
-static inline void mm_lock_seqcount_begin(struct mm_struct *mm) {}
-static inline void mm_lock_seqcount_end(struct mm_struct *mm) {}
-
-static inline bool mmap_lock_speculate_try_begin(struct mm_struct *mm, unsigned int *seq)
-{
-	return false;
-}
-
-static inline bool mmap_lock_speculate_retry(struct mm_struct *mm, unsigned int seq)
-{
-	return true;
-}
-static inline void vma_lock_init(struct vm_area_struct *vma, bool reset_refcnt) {}
-static inline void vma_end_read(struct vm_area_struct *vma) {}
-static inline void vma_start_write(struct vm_area_struct *vma) {}
-static inline __must_check
-int vma_start_write_killable(struct vm_area_struct *vma) { return 0; }
-static inline void vma_assert_write_locked(struct vm_area_struct *vma)
-		{ mmap_assert_write_locked(vma->vm_mm); }
-static inline bool vma_is_attached(struct vm_area_struct *vma)
-		{ return true; }
-static inline void vma_assert_attached(struct vm_area_struct *vma) {}
-static inline void vma_assert_detached(struct vm_area_struct *vma) {}
-static inline void vma_mark_attached(struct vm_area_struct *vma) {}
-static inline void vma_mark_detached(struct vm_area_struct *vma) {}
-
-static inline struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
-		unsigned long address)
-{
-	return NULL;
-}
-
-static inline void vma_assert_locked(struct vm_area_struct *vma)
-{
-	mmap_assert_locked(vma->vm_mm);
-}
-
-static inline void vma_assert_stabilised(struct vm_area_struct *vma)
-{
-	/* If no VMA locks, then either mmap lock suffices to stabilise. */
-	mmap_assert_locked(vma->vm_mm);
-}
-
-#endif /* CONFIG_PER_VMA_LOCK */
-
 static inline void vma_assert_can_modify(struct vm_area_struct *vma)
 {
 	if (vma_is_attached(vma))
diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
index 41fe87d7302f..8848e26ef581 100644
--- a/kernel/bpf/stackmap.c
+++ b/kernel/bpf/stackmap.c
@@ -272,13 +272,8 @@ struct stack_map_vma_lock {
 /*
  * Acquire a stable read-side reference on the VMA covering @ip.
  *
- * With CONFIG_PER_VMA_LOCK=y this returns a VMA with its per-VMA read
- * lock held and mmap_lock dropped, so the caller may sleep.
- *
- * With CONFIG_PER_VMA_LOCK=n it returns a VMA with mmap_lock still
- * held; the caller must snapshot any fields it needs and pin vm_file
- * with get_file() before stack_map_unlock_vma() drops mmap_lock, as
- * the VMA may be split, merged, or freed after that.
+ * This returns a VMA with its per-VMA read lock held and mmap_lock
+ * dropped, so the caller may sleep.
  *
  * Returns NULL on failure, in which case no lock is held.
  */
@@ -288,7 +283,6 @@ stack_map_lock_vma(struct stack_map_vma_lock *lock, unsigned long ip)
 	struct mm_struct *mm = lock->mm;
 	struct vm_area_struct *vma;
 
-	/* noop under !CONFIG_PER_VMA_LOCK */
 	vma = lock_vma_under_rcu(mm, ip);
 	if (vma) {
 		lock->vma = vma;
@@ -308,13 +302,11 @@ stack_map_lock_vma(struct stack_map_vma_lock *lock, unsigned long ip)
 		return NULL;
 	}
 
-#ifdef CONFIG_PER_VMA_LOCK
 	if (!vma_start_read_locked(vma)) {
 		mmap_read_unlock(mm);
 		return NULL;
 	}
 	mmap_read_unlock(mm);
-#endif
 
 	lock->vma = vma;
 	return vma;
@@ -322,11 +314,7 @@ stack_map_lock_vma(struct stack_map_vma_lock *lock, unsigned long ip)
 
 static void stack_map_unlock_vma(struct stack_map_vma_lock *lock)
 {
-#ifdef CONFIG_PER_VMA_LOCK
 	vma_end_read(lock->vma);
-#else
-	mmap_read_unlock(lock->mm);
-#endif
 	lock->vma = NULL;
 }
 
diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c
index e791ae065c39..6cf815bc84be 100644
--- a/kernel/bpf/task_iter.c
+++ b/kernel/bpf/task_iter.c
@@ -835,11 +835,6 @@ __bpf_kfunc int bpf_iter_task_vma_new(struct bpf_iter_task_vma *it,
 	BUILD_BUG_ON(sizeof(struct bpf_iter_task_vma_kern) != sizeof(struct bpf_iter_task_vma));
 	BUILD_BUG_ON(__alignof__(struct bpf_iter_task_vma_kern) != __alignof__(struct bpf_iter_task_vma));
 
-	if (!IS_ENABLED(CONFIG_PER_VMA_LOCK)) {
-		kit->data = NULL;
-		return -EOPNOTSUPP;
-	}
-
 	/*
 	 * Reject irqs-disabled contexts including NMI. Operations used
 	 * by _next() and _destroy() (vma_end_read, fput, bpf_iter_mmput_async)
diff --git a/kernel/fork.c b/kernel/fork.c
index f0e2e131a9a5..ff91f5f66c80 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1077,9 +1077,7 @@ static void mmap_init_lock(struct mm_struct *mm)
 {
 	init_rwsem(&mm->mmap_lock);
 	mm_lock_seqcount_init(mm);
-#ifdef CONFIG_PER_VMA_LOCK
 	rcuwait_init(&mm->vma_writer_wait);
-#endif
 }
 
 static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p)
diff --git a/mm/Kconfig b/mm/Kconfig
index 060190e12bce..77103b46b679 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -1425,19 +1425,6 @@ config LRU_GEN_STATS
 config LRU_GEN_WALKS_MMU
 	def_bool y
 	depends on LRU_GEN && ARCH_HAS_HW_PTE_YOUNG
-# }
-
-config ARCH_SUPPORTS_PER_VMA_LOCK
-       def_bool n
-
-config PER_VMA_LOCK
-	def_bool y
-	depends on ARCH_SUPPORTS_PER_VMA_LOCK && MMU && SMP
-	help
-	  Allow per-vma locking during page fault handling.
-
-	  This feature allows locking each virtual memory area separately when
-	  handling page faults instead of taking mmap_lock.
 
 config LOCK_MM_AND_FIND_VMA
 	bool
diff --git a/mm/Kconfig.debug b/mm/Kconfig.debug
index 5737a504efbb..1dd150edfe71 100644
--- a/mm/Kconfig.debug
+++ b/mm/Kconfig.debug
@@ -310,7 +310,6 @@ config DEBUG_KMEMLEAK_VERBOSE
 
 config PER_VMA_LOCK_STATS
 	bool "Statistics for per-vma locks"
-	depends on PER_VMA_LOCK
 	help
 	  Say Y here to enable success, retry and failure counters of page
 	  faults handled under protection of per-vma locks. When enabled, the
diff --git a/mm/debug.c b/mm/debug.c
index 9a0297b3988d..655e6bcc0e8d 100644
--- a/mm/debug.c
+++ b/mm/debug.c
@@ -157,17 +157,13 @@ void dump_vma(const struct vm_area_struct *vma)
 	pr_emerg("vma %px start %px end %px mm %px\n"
 		"prot %lx anon_vma %px vm_ops %px\n"
 		"pgoff %lx file %px private_data %px\n"
-#ifdef CONFIG_PER_VMA_LOCK
 		"refcnt %x\n"
-#endif
 		"flags: %#lx(%pGv)\n",
 		vma, (void *)vma->vm_start, (void *)vma->vm_end, vma->vm_mm,
 		(unsigned long)pgprot_val(vma->vm_page_prot),
 		vma->anon_vma, vma->vm_ops, vma_start_pgoff(vma),
 		vma->vm_file, vma->vm_private_data,
-#ifdef CONFIG_PER_VMA_LOCK
 		refcount_read(&vma->vm_refcnt),
-#endif
 		vma->vm_flags, &vma->vm_flags);
 }
 EXPORT_SYMBOL(dump_vma);
diff --git a/mm/init-mm.c b/mm/init-mm.c
index 3e792aad7626..a1bb2c2d0284 100644
--- a/mm/init-mm.c
+++ b/mm/init-mm.c
@@ -39,10 +39,8 @@ struct mm_struct init_mm = {
 	.page_table_lock =  __SPIN_LOCK_UNLOCKED(init_mm.page_table_lock),
 	.arg_lock	=  __SPIN_LOCK_UNLOCKED(init_mm.arg_lock),
 	.mmlist		= LIST_HEAD_INIT(init_mm.mmlist),
-#ifdef CONFIG_PER_VMA_LOCK
 	.vma_writer_wait = __RCUWAIT_INITIALIZER(init_mm.vma_writer_wait),
 	.mm_lock_seq	= SEQCNT_ZERO(init_mm.mm_lock_seq),
-#endif
 #ifdef CONFIG_SCHED_MM_CID
 	.mm_cid.lock = __RAW_SPIN_LOCK_UNLOCKED(init_mm.mm_cid.lock),
 #endif
diff --git a/mm/memory.c b/mm/memory.c
index a620d425ec95..f109f6c87b28 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -6813,7 +6813,6 @@ static vm_fault_t sanitize_fault_flags(struct vm_area_struct *vma,
 				 !is_cow_mapping(vma->vm_flags)))
 			return VM_FAULT_SIGSEGV;
 	}
-#ifdef CONFIG_PER_VMA_LOCK
 	/*
 	 * Per-VMA locks can't be used with FAULT_FLAG_RETRY_NOWAIT because of
 	 * the assumption that lock is dropped on VM_FAULT_RETRY.
@@ -6822,7 +6821,6 @@ static vm_fault_t sanitize_fault_flags(struct vm_area_struct *vma,
 			(FAULT_FLAG_VMA_LOCK | FAULT_FLAG_RETRY_NOWAIT)) ==
 			(FAULT_FLAG_VMA_LOCK | FAULT_FLAG_RETRY_NOWAIT)))
 		return VM_FAULT_SIGSEGV;
-#endif
 
 	return 0;
 }
diff --git a/mm/mmap_lock.c b/mm/mmap_lock.c
index 898c2ef1e958..e20d01e8d38f 100644
--- a/mm/mmap_lock.c
+++ b/mm/mmap_lock.c
@@ -43,9 +43,6 @@ void __mmap_lock_do_trace_released(struct mm_struct *mm, bool write)
 EXPORT_SYMBOL(__mmap_lock_do_trace_released);
 #endif /* CONFIG_TRACING */
 
-#ifdef CONFIG_MMU
-#ifdef CONFIG_PER_VMA_LOCK
-
 /* State shared across __vma_[start, end]_exclude_readers. */
 struct vma_exclude_readers_state {
 	/* Input parameters. */
@@ -431,7 +428,6 @@ struct vm_area_struct *lock_next_vma(struct mm_struct *mm,
 
 	return vma;
 }
-#endif /* CONFIG_PER_VMA_LOCK */
 
 #ifdef CONFIG_LOCK_MM_AND_FIND_VMA
 #include <linux/extable.h>
@@ -548,23 +544,3 @@ struct vm_area_struct *lock_mm_and_find_vma(struct mm_struct *mm,
 	return NULL;
 }
 #endif /* CONFIG_LOCK_MM_AND_FIND_VMA */
-
-#else /* CONFIG_MMU */
-
-/*
- * At least xtensa ends up having protection faults even with no
- * MMU.. No stack expansion, at least.
- */
-struct vm_area_struct *lock_mm_and_find_vma(struct mm_struct *mm,
-			unsigned long addr, struct pt_regs *regs)
-{
-	struct vm_area_struct *vma;
-
-	mmap_read_lock(mm);
-	vma = vma_lookup(mm, addr);
-	if (!vma)
-		mmap_read_unlock(mm);
-	return vma;
-}
-
-#endif /* CONFIG_MMU */
diff --git a/mm/pagewalk.c b/mm/pagewalk.c
index ed4860c01936..fbcf64c59a97 100644
--- a/mm/pagewalk.c
+++ b/mm/pagewalk.c
@@ -446,7 +446,6 @@ static inline void process_mm_walk_lock(struct mm_struct *mm,
 static inline void process_vma_walk_lock(struct vm_area_struct *vma,
 					 enum page_walk_lock walk_lock)
 {
-#ifdef CONFIG_PER_VMA_LOCK
 	switch (walk_lock) {
 	case PGWALK_WRLOCK:
 		vma_start_write(vma);
@@ -461,7 +460,6 @@ static inline void process_vma_walk_lock(struct vm_area_struct *vma,
 		/* PGWALK_RDLOCK is handled by process_mm_walk_lock */
 		break;
 	}
-#endif
 }
 
 /*
diff --git a/mm/rmap.c b/mm/rmap.c
index b917431759ee..4e4a4b747977 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -260,11 +260,9 @@ static void check_anon_vma_clone(struct vm_area_struct *dst,
 	/* For the anon_vma to be compatible, it can only be singular. */
 	VM_WARN_ON_ONCE(operation == VMA_OP_MERGE_UNFAULTED &&
 			!list_is_singular(&src->anon_vma_chain));
-#ifdef CONFIG_PER_VMA_LOCK
 	/* Only merging an unfaulted VMA leaves the destination attached. */
 	VM_WARN_ON_ONCE(operation != VMA_OP_MERGE_UNFAULTED &&
 			vma_is_attached(dst));
-#endif
 }
 
 static void maybe_reuse_anon_vma(struct vm_area_struct *dst,
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 258b03182a78..edd90892f8cc 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -122,7 +122,6 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
 	return vma;
 }
 
-#ifdef CONFIG_PER_VMA_LOCK
 /*
  * uffd_lock_vma() - Lookup and lock vma corresponding to @address.
  * @mm: mm to search vma in.
@@ -182,34 +181,6 @@ static void uffd_mfill_unlock(struct vm_area_struct *vma)
 	vma_end_read(vma);
 }
 
-#else
-
-static struct vm_area_struct *uffd_mfill_lock(struct mm_struct *dst_mm,
-					      unsigned long dst_start,
-					      unsigned long len)
-{
-	struct vm_area_struct *dst_vma;
-
-	mmap_read_lock(dst_mm);
-	dst_vma = find_vma_and_prepare_anon(dst_mm, dst_start);
-	if (IS_ERR(dst_vma))
-		goto out_unlock;
-
-	if (validate_dst_vma(dst_vma, dst_start + len))
-		return dst_vma;
-
-	dst_vma = ERR_PTR(-ENOENT);
-out_unlock:
-	mmap_read_unlock(dst_mm);
-	return dst_vma;
-}
-
-static void uffd_mfill_unlock(struct vm_area_struct *vma)
-{
-	mmap_read_unlock(vma->vm_mm);
-}
-#endif
-
 static void mfill_put_vma(struct mfill_state *state)
 {
 	if (!state->vma)
@@ -1852,7 +1823,6 @@ int find_vmas_mm_locked(struct mm_struct *mm,
 	return 0;
 }
 
-#ifdef CONFIG_PER_VMA_LOCK
 static int uffd_move_lock(struct mm_struct *mm,
 			  unsigned long dst_start,
 			  unsigned long src_start,
@@ -1927,31 +1897,6 @@ static void uffd_move_unlock(struct vm_area_struct *dst_vma,
 		vma_end_read(dst_vma);
 }
 
-#else
-
-static int uffd_move_lock(struct mm_struct *mm,
-			  unsigned long dst_start,
-			  unsigned long src_start,
-			  struct vm_area_struct **dst_vmap,
-			  struct vm_area_struct **src_vmap)
-{
-	int err;
-
-	mmap_read_lock(mm);
-	err = find_vmas_mm_locked(mm, dst_start, src_start, dst_vmap, src_vmap);
-	if (err)
-		mmap_read_unlock(mm);
-	return err;
-}
-
-static void uffd_move_unlock(struct vm_area_struct *dst_vma,
-			     struct vm_area_struct *src_vma)
-{
-	mmap_assert_locked(src_vma->vm_mm);
-	mmap_read_unlock(dst_vma->vm_mm);
-}
-#endif
-
 /**
  * move_pages - move arbitrary anonymous pages of an existing vma
  * @ctx: pointer to the userfaultfd context
diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs
index 4764d7b68f2a..2633e704c83d 100644
--- a/rust/kernel/mm.rs
+++ b/rust/kernel/mm.rs
@@ -174,26 +174,20 @@ pub unsafe fn from_raw<'a>(ptr: *const bindings::mm_struct) -> &'a MmWithUser {
     /// When per-vma locks are disabled, this always returns `None`.
     #[inline]
     pub fn lock_vma_under_rcu(&self, vma_addr: usize) -> Option<VmaReadGuard<'_>> {
-        #[cfg(CONFIG_PER_VMA_LOCK)]
         {
             // SAFETY: Calling `bindings::lock_vma_under_rcu` is always okay given an mm where
             // `mm_users` is non-zero.
             let vma = unsafe { bindings::lock_vma_under_rcu(self.as_raw(), vma_addr) };
-            if !vma.is_null() {
-                return Some(VmaReadGuard {
-                    // SAFETY: If `lock_vma_under_rcu` returns a non-null ptr, then it points at a
-                    // valid vma. The vma is stable for as long as the vma read lock is held.
-                    vma: unsafe { VmaRef::from_raw(vma) },
-                    _nts: NotThreadSafe,
-                });
+            if vma.is_null() {
+                return None;
             }
+            Some(VmaReadGuard {
+                // SAFETY: If `lock_vma_under_rcu` returns a non-null ptr, then it points at a
+                // valid vma. The vma is stable for as long as the vma read lock is held.
+                vma: unsafe { VmaRef::from_raw(vma) },
+                _nts: NotThreadSafe,
+            })
         }
-
-        // Silence warnings about unused variables.
-        #[cfg(not(CONFIG_PER_VMA_LOCK))]
-        let _ = vma_addr;
-
-        None
     }
 
     /// Lock the mmap read lock.
diff --git a/tools/testing/vma/include/dup.h b/tools/testing/vma/include/dup.h
index 800e5fa02d78..362feda28526 100644
--- a/tools/testing/vma/include/dup.h
+++ b/tools/testing/vma/include/dup.h
@@ -582,7 +582,6 @@ struct vm_area_struct {
 		vma_flags_t flags;
 	};
 
-#ifdef CONFIG_PER_VMA_LOCK
 	/*
 	 * Can only be written (using WRITE_ONCE()) while holding both:
 	 *  - mmap_lock (in write mode)
@@ -598,7 +597,7 @@ struct vm_area_struct {
 	 * slowpath.
 	 */
 	unsigned int vm_lock_seq;
-#endif
+
 	unsigned int __vm_virt_pgoff_lo;
 
 	/*
@@ -632,10 +631,8 @@ struct vm_area_struct {
 #ifdef CONFIG_NUMA_BALANCING
 	struct vma_numab_state *numab_state;	/* NUMA Balancing state */
 #endif
-#ifdef CONFIG_PER_VMA_LOCK
 	/* Unstable RCU readers are allowed to read this. */
 	refcount_t vm_refcnt;
-#endif
 #ifdef CONFIG_64BIT
 	unsigned int __vm_virt_pgoff_hi;
 #endif
diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h
index 8a48b231aa7a..54d5c3360aa2 100644
--- a/tools/testing/vma/vma_internal.h
+++ b/tools/testing/vma/vma_internal.h
@@ -15,7 +15,6 @@
 #include <stdlib.h>
 
 #define CONFIG_MMU		1
-#define CONFIG_PER_VMA_LOCK	1
 
 #ifdef __CONCAT
 #undef __CONCAT
-- 
2.55.0.508.g3f0d502094-goog

Re: [PATCH v3 1/5] mm: Make per-VMA locks available universally

[Lorenzo Stoakes (ARM)]

On Sun, Aug 02, 2026 at 02:54:55PM -0700, Suren Baghdasaryan wrote:
> From: Dave Hansen <dave.hansen@linux.intel.com>
>
> The per-VMA locks have been around for several years. They've had some
> bugs worked out of them and have seen quite wide use. However, they
> are still only available when architectures explicitly enable them.
> Remove the conditional compilation around the per-VMA locks, making
> them available on all architectures and configs.
>
> The approach up to now seemed to be to add ARCH_SUPPORTS_PER_VMA_LOCK
> when the architecture started using per-VMA locks in the fault
> handler. But, contrary to the naming, the Kconfig option does not
> really indicate whether the architecture supports per-VMA locks or
> not. It is more of a marker for whether the architecture is likely to
> benefit from per-VMA locks.
>
> To me, the most important thing side-effect of universal availability
> is letting per-VMA locks be used in SMP=n configs. This lets us use
> per-VMA locking in all x86 code without fallbacks.
>
> Overall, this just generally makes the kernel simpler. Just look at
> the diffstat. It also opens the door to users that want to use the
> per-VMA locks in common code. Doing *that* brings additional
> simplifications.
>
> The downside of this is adding some fields to vm_area_struct and
> mm_struct. There are likely ways to optimize this, especially for
> things like SMP=n configs. For now, do the simplest thing: use the
> same implementation everywhere.
>
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Signed-off-by: Suren Baghdasaryan <surenb@google.com>

Thanks a lot for this! Great change.

I see you also got the VMA userland test changes as well :)

All LGTM so:

Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>

> Cc: Suren Baghdasaryan <surenb@google.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> Cc: Lorenzo Stoakes <ljs@kernel.org>
> Cc: Vlastimil Babka <vbabka@kernel.org>
> Cc: Shakeel Butt <shakeel.butt@linux.dev>
> Cc: linux-mm@kvack.org
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Arve Hjønnevåg <arve@android.com>
> Cc: Todd Kjos <tkjos@android.com>
> Cc: Christian Brauner <christian@brauner.io>
> Cc: Carlos Llamas <cmllamas@google.com>
> Cc: Alice Ryhl <aliceryhl@google.com>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: David Ahern <dsahern@kernel.org>
> Cc: netdev@vger.kernel.org
> ---
>  arch/arm/Kconfig                       |  1 -
>  arch/arm64/Kconfig                     |  1 -
>  arch/loongarch/Kconfig                 |  1 -
>  arch/powerpc/platforms/powernv/Kconfig |  1 -
>  arch/powerpc/platforms/pseries/Kconfig |  1 -
>  arch/riscv/Kconfig                     |  1 -
>  arch/s390/Kconfig                      |  1 -
>  arch/x86/Kconfig                       |  2 -
>  fs/proc/internal.h                     |  2 -
>  fs/proc/task_mmu.c                     | 93 --------------------------
>  include/linux/mm.h                     | 12 ----
>  include/linux/mm_types.h               |  8 +--
>  include/linux/mmap_lock.h              | 50 --------------
>  kernel/bpf/stackmap.c                  | 16 +----
>  kernel/bpf/task_iter.c                 |  5 --
>  kernel/fork.c                          |  2 -
>  mm/Kconfig                             | 13 ----
>  mm/Kconfig.debug                       |  1 -
>  mm/debug.c                             |  4 --
>  mm/init-mm.c                           |  2 -
>  mm/memory.c                            |  2 -
>  mm/mmap_lock.c                         | 24 -------
>  mm/pagewalk.c                          |  2 -
>  mm/rmap.c                              |  2 -
>  mm/userfaultfd.c                       | 55 ---------------
>  rust/kernel/mm.rs                      | 22 +++---
>  tools/testing/vma/include/dup.h        |  5 +-
>  tools/testing/vma/vma_internal.h       |  1 -
>  28 files changed, 13 insertions(+), 317 deletions(-)

Look at all that red :) lovely!

>
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index 9187240a02db..f815209167cd 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -41,7 +41,6 @@ config ARM
>  	select ARCH_SUPPORTS_ATOMIC_RMW
>  	select ARCH_SUPPORTS_CFI
>  	select ARCH_SUPPORTS_HUGETLBFS if ARM_LPAE
> -	select ARCH_SUPPORTS_PER_VMA_LOCK
>  	select ARCH_SUPPORTS_RT
>  	select ARCH_USE_BUILTIN_BSWAP
>  	select ARCH_USE_CMPXCHG_LOCKREF
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 11a9c534b7b4..21eb64b24a2c 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -81,7 +81,6 @@ config ARM64
>  	select ARCH_HAS_PTE_PROTNONE
>  	select ARCH_SUPPORTS_NUMA_BALANCING
>  	select ARCH_SUPPORTS_PAGE_TABLE_CHECK
> -	select ARCH_SUPPORTS_PER_VMA_LOCK
>  	select ARCH_SUPPORTS_HUGE_PFNMAP if TRANSPARENT_HUGEPAGE
>  	select ARCH_SUPPORTS_RT
>  	select ARCH_SUPPORTS_SCHED_SMT
> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
> index e20acbe5fe7b..7741e39eca2b 100644
> --- a/arch/loongarch/Kconfig
> +++ b/arch/loongarch/Kconfig
> @@ -69,7 +69,6 @@ config LOONGARCH
>  	select ARCH_SUPPORTS_MSEAL_SYSTEM_MAPPINGS
>  	select ARCH_HAS_PTE_PROTNONE if 64BIT
>  	select ARCH_SUPPORTS_NUMA_BALANCING if NUMA
> -	select ARCH_SUPPORTS_PER_VMA_LOCK
>  	select ARCH_SUPPORTS_RT
>  	select ARCH_SUPPORTS_SCHED_SMT if SMP
>  	select ARCH_SUPPORTS_SCHED_MC  if SMP
> diff --git a/arch/powerpc/platforms/powernv/Kconfig b/arch/powerpc/platforms/powernv/Kconfig
> index b5ad7c173ef0..dd8f6060fb7a 100644
> --- a/arch/powerpc/platforms/powernv/Kconfig
> +++ b/arch/powerpc/platforms/powernv/Kconfig
> @@ -17,7 +17,6 @@ config PPC_POWERNV
>  	select PPC_DOORBELL
>  	select MMU_NOTIFIER
>  	select FORCE_SMP
> -	select ARCH_SUPPORTS_PER_VMA_LOCK
>  	select PPC_RADIX_BROADCAST_TLBIE if PPC_RADIX_MMU
>  	default y
>
> diff --git a/arch/powerpc/platforms/pseries/Kconfig b/arch/powerpc/platforms/pseries/Kconfig
> index 74910ce3a541..7d125e288f6e 100644
> --- a/arch/powerpc/platforms/pseries/Kconfig
> +++ b/arch/powerpc/platforms/pseries/Kconfig
> @@ -23,7 +23,6 @@ config PPC_PSERIES
>  	select HOTPLUG_CPU
>  	select FORCE_SMP
>  	select SWIOTLB
> -	select ARCH_SUPPORTS_PER_VMA_LOCK
>  	select PPC_RADIX_BROADCAST_TLBIE if PPC_RADIX_MMU
>  	default y
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 7b9c373d82fa..faa85a031fe5 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -70,7 +70,6 @@ config RISCV
>  	select ARCH_SUPPORTS_LTO_CLANG_THIN
>  	select ARCH_SUPPORTS_MSEAL_SYSTEM_MAPPINGS if 64BIT && MMU
>  	select ARCH_SUPPORTS_PAGE_TABLE_CHECK if MMU
> -	select ARCH_SUPPORTS_PER_VMA_LOCK if MMU
>  	select ARCH_HAS_PTE_PROTNONE if MMU
>  	select ARCH_SUPPORTS_RT
>  	select ARCH_SUPPORTS_SHADOW_CALL_STACK if HAVE_SHADOW_CALL_STACK
> diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
> index ab8fccc2cc4e..d1274bca8c39 100644
> --- a/arch/s390/Kconfig
> +++ b/arch/s390/Kconfig
> @@ -151,7 +151,6 @@ config S390
>  	select ARCH_HAS_PTE_PROTNONE
>  	select ARCH_SUPPORTS_NUMA_BALANCING
>  	select ARCH_SUPPORTS_PAGE_TABLE_CHECK
> -	select ARCH_SUPPORTS_PER_VMA_LOCK
>  	select ARCH_USE_BUILTIN_BSWAP
>  	select ARCH_USE_CMPXCHG_LOCKREF
>  	select ARCH_USE_SYM_ANNOTATIONS
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index fb298e219179..79479d29576f 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -27,7 +27,6 @@ config X86_64
>  	select ARCH_HAS_GIGANTIC_PAGE
>  	select ARCH_SUPPORTS_MSEAL_SYSTEM_MAPPINGS
>  	select ARCH_SUPPORTS_INT128 if CC_HAS_INT128
> -	select ARCH_SUPPORTS_PER_VMA_LOCK
>  	select ARCH_SUPPORTS_HUGE_PFNMAP if TRANSPARENT_HUGEPAGE
>  	select HAVE_ARCH_SOFT_DIRTY
>  	select MODULES_USE_ELF_RELA
> @@ -1846,7 +1845,6 @@ config X86_USER_SHADOW_STACK
>  	bool "X86 userspace shadow stack"
>  	depends on AS_WRUSS
>  	depends on X86_64
> -	depends on PER_VMA_LOCK
>  	select ARCH_USES_HIGH_VMA_FLAGS
>  	select ARCH_HAS_USER_SHADOW_STACK
>  	select X86_CET
> diff --git a/fs/proc/internal.h b/fs/proc/internal.h
> index b232e1098117..6713757da099 100644
> --- a/fs/proc/internal.h
> +++ b/fs/proc/internal.h
> @@ -385,10 +385,8 @@ struct mem_size_stats;
>
>  struct proc_maps_locking_ctx {
>  	struct mm_struct *mm;
> -#ifdef CONFIG_PER_VMA_LOCK
>  	bool mmap_locked;
>  	struct vm_area_struct *locked_vma;
> -#endif
>  };
>
>  struct proc_maps_private {
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index 817e3e0f9194..096bf0b0b9e0 100644
> --- a/fs/proc/task_mmu.c
> +++ b/fs/proc/task_mmu.c
> @@ -130,8 +130,6 @@ static void release_task_mempolicy(struct proc_maps_private *priv)
>  }
>  #endif
>
> -#ifdef CONFIG_PER_VMA_LOCK
> -
>  static inline int lock_ctx_mm(struct proc_maps_locking_ctx *lock_ctx)
>  {
>  	int ret = mmap_read_lock_killable(lock_ctx->mm);
> @@ -233,46 +231,6 @@ static inline void reacquire_rcu(struct proc_maps_private *priv)
>  	vma_iter_set(&priv->iter, priv->lock_ctx.locked_vma->vm_end);
>  }
>
> -#else /* CONFIG_PER_VMA_LOCK */
> -
> -static inline int lock_ctx_mm(struct proc_maps_locking_ctx *lock_ctx)
> -{
> -	return mmap_read_lock_killable(lock_ctx->mm);
> -}
> -
> -static inline void unlock_ctx_mm(struct proc_maps_locking_ctx *lock_ctx)
> -{
> -	mmap_read_unlock(lock_ctx->mm);
> -}
> -
> -static inline bool lock_vma_range(struct seq_file *m,
> -				  struct proc_maps_locking_ctx *lock_ctx)
> -{
> -	return lock_ctx_mm(lock_ctx) == 0;
> -}
> -
> -static inline void unlock_vma_range(struct proc_maps_locking_ctx *lock_ctx)
> -{
> -	unlock_ctx_mm(lock_ctx);
> -}
> -
> -static struct vm_area_struct *get_next_vma(struct proc_maps_private *priv,
> -					   loff_t last_pos)
> -{
> -	return vma_next(&priv->iter);
> -}
> -
> -static inline bool fallback_to_mmap_lock(struct proc_maps_private *priv,
> -					 loff_t pos)
> -{
> -	return false;
> -}
> -
> -static inline void drop_rcu(struct proc_maps_private *priv) {}
> -static inline void reacquire_rcu(struct proc_maps_private *priv) {}
> -
> -#endif /* CONFIG_PER_VMA_LOCK */
> -
>  static struct vm_area_struct *proc_get_vma(struct seq_file *m, loff_t *ppos)
>  {
>  	struct proc_maps_private *priv = m->private;
> @@ -560,8 +518,6 @@ static int pid_maps_open(struct inode *inode, struct file *file)
>  		PROCMAP_QUERY_VMA_FLAGS				\
>  )
>
> -#ifdef CONFIG_PER_VMA_LOCK
> -
>  static int query_vma_setup(struct proc_maps_locking_ctx *lock_ctx)
>  {
>  	reset_lock_ctx(lock_ctx);
> @@ -612,26 +568,6 @@ static struct vm_area_struct *query_vma_find_by_addr(struct proc_maps_locking_ct
>  	return vma;
>  }
>
> -#else /* CONFIG_PER_VMA_LOCK */
> -
> -static int query_vma_setup(struct proc_maps_locking_ctx *lock_ctx)
> -{
> -	return mmap_read_lock_killable(lock_ctx->mm);
> -}
> -
> -static void query_vma_teardown(struct proc_maps_locking_ctx *lock_ctx)
> -{
> -	mmap_read_unlock(lock_ctx->mm);
> -}
> -
> -static struct vm_area_struct *query_vma_find_by_addr(struct proc_maps_locking_ctx *lock_ctx,
> -						     unsigned long addr)
> -{
> -	return find_vma(lock_ctx->mm, addr);
> -}
> -
> -#endif  /* CONFIG_PER_VMA_LOCK */
> -
>  static struct vm_area_struct *query_matching_vma(struct proc_maps_locking_ctx *lock_ctx,
>  						 unsigned long addr, u32 flags)
>  {
> @@ -1314,8 +1250,6 @@ static const struct mm_walk_ops smaps_shmem_walk_ops = {
>  	.walk_lock		= PGWALK_RDLOCK,
>  };
>
> -#ifdef CONFIG_PER_VMA_LOCK
> -
>  static const struct mm_walk_ops smaps_walk_vma_lock_ops = {
>  	.pmd_entry		= smaps_pte_range,
>  	.hugetlb_entry		= smaps_hugetlb_range,
> @@ -1345,22 +1279,6 @@ get_smaps_shmem_walk_ops(struct proc_maps_private *priv)
>  	return &smaps_shmem_walk_vma_lock_ops;
>  }
>
> -#else /* CONFIG_PER_VMA_LOCK */
> -
> -static inline const struct mm_walk_ops *
> -get_smaps_walk_ops(struct proc_maps_private *priv)
> -{
> -	return &smaps_walk_ops;
> -}
> -
> -static inline const struct mm_walk_ops *
> -get_smaps_shmem_walk_ops(struct proc_maps_private *priv)
> -{
> -	return &smaps_shmem_walk_ops;
> -}
> -
> -#endif /* CONFIG_PER_VMA_LOCK */
> -
>  /*
>   * Gather mem stats from @vma with the indicated beginning
>   * address @start, and keep them in @mss.
> @@ -3497,7 +3415,6 @@ static const struct mm_walk_ops show_numa_ops = {
>  	.walk_lock = PGWALK_RDLOCK,
>  };
>
> -#ifdef CONFIG_PER_VMA_LOCK
>  static const struct mm_walk_ops show_numa_vma_lock_ops = {
>  	.hugetlb_entry = gather_hugetlb_stats,
>  	.pmd_entry = gather_pte_stats,
> @@ -3512,16 +3429,6 @@ get_show_numa_ops(struct proc_maps_private *priv)
>  	return &show_numa_vma_lock_ops;
>  }
>
> -#else /* CONFIG_PER_VMA_LOCK */
> -
> -static inline const struct mm_walk_ops *
> -get_show_numa_ops(struct proc_maps_private *priv)
> -{
> -	return &show_numa_ops;
> -}
> -
> -#endif /* CONFIG_PER_VMA_LOCK */
> -
>  /*
>   * Display pages allocated per node and memory policy via /proc.
>   */
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index 7fabe6c66b4b..d9850f846242 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -931,7 +931,6 @@ static inline void vma_numab_state_free(struct vm_area_struct *vma) {}
>   * These must be here rather than mmap_lock.h as dependent on vm_fault type,
>   * declared in this header.
>   */
> -#ifdef CONFIG_PER_VMA_LOCK
>  static inline void release_fault_lock(struct vm_fault *vmf)
>  {
>  	if (vmf->flags & FAULT_FLAG_VMA_LOCK)
> @@ -947,17 +946,6 @@ static inline void assert_fault_locked(const struct vm_fault *vmf)
>  	else
>  		mmap_assert_locked(vmf->vma->vm_mm);
>  }
> -#else
> -static inline void release_fault_lock(struct vm_fault *vmf)
> -{
> -	mmap_read_unlock(vmf->vma->vm_mm);
> -}
> -
> -static inline void assert_fault_locked(const struct vm_fault *vmf)
> -{
> -	mmap_assert_locked(vmf->vma->vm_mm);
> -}
> -#endif /* CONFIG_PER_VMA_LOCK */
>
>  static inline bool mm_flags_test(int flag, const struct mm_struct *mm)
>  {
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index b5d4cd3b067b..d8e246fd09d3 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -950,7 +950,6 @@ struct vm_area_struct {
>  		vma_flags_t flags;
>  	};
>
> -#ifdef CONFIG_PER_VMA_LOCK
>  	/*
>  	 * Can only be written (using WRITE_ONCE()) while holding both:
>  	 *  - mmap_lock (in write mode)
> @@ -966,7 +965,7 @@ struct vm_area_struct {
>  	 * slowpath.
>  	 */
>  	unsigned int vm_lock_seq;
> -#endif
> +
>  	/*
>  	 * Low 32-bits of virtual page offset.
>  	 * See vma_start_virt_pgoff() comment for details.
> @@ -1003,7 +1002,6 @@ struct vm_area_struct {
>  #ifdef CONFIG_NUMA_BALANCING
>  	struct vma_numab_state *numab_state;	/* NUMA Balancing state */
>  #endif
> -#ifdef CONFIG_PER_VMA_LOCK
>  	/*
>  	 * Used to keep track of firstly, whether the VMA is attached, secondly,
>  	 * if attached, how many read locks are taken, and thirdly, if the
> @@ -1046,7 +1044,6 @@ struct vm_area_struct {
>  #ifdef CONFIG_DEBUG_LOCK_ALLOC
>  	struct lockdep_map vmlock_dep_map;
>  #endif
> -#endif
>  #ifdef CONFIG_64BIT
>  	/*
>  	 * High 32-bits of virtual page offset.
> @@ -1254,7 +1251,6 @@ struct mm_struct {
>  					  * init_mm.mmlist, and are protected
>  					  * by mmlist_lock
>  					  */
> -#ifdef CONFIG_PER_VMA_LOCK
>  		struct rcuwait vma_writer_wait;
>  		/*
>  		 * This field has lock-like semantics, meaning it is sometimes
> @@ -1274,7 +1270,7 @@ struct mm_struct {
>  		 * mmap_lock.
>  		 */
>  		seqcount_t mm_lock_seq;
> -#endif
> +
>  		struct futex_mm_data	futex;
>
>  		unsigned long hiwater_rss; /* High-watermark of RSS usage */
> diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
> index 87f77e3da77f..eb32b482434e 100644
> --- a/include/linux/mmap_lock.h
> +++ b/include/linux/mmap_lock.h
> @@ -76,8 +76,6 @@ static inline void mmap_assert_write_locked(const struct mm_struct *mm)
>  	rwsem_assert_held_write(&mm->mmap_lock);
>  }
>
> -#ifdef CONFIG_PER_VMA_LOCK
> -
>  #ifdef CONFIG_LOCKDEP
>  #define __vma_lockdep_map(vma) (&vma->vmlock_dep_map)
>  #else
> @@ -484,54 +482,6 @@ struct vm_area_struct *lock_next_vma(struct mm_struct *mm,
>  				     struct vma_iterator *iter,
>  				     unsigned long address);
>
> -#else /* CONFIG_PER_VMA_LOCK */
> -
> -static inline void mm_lock_seqcount_init(struct mm_struct *mm) {}
> -static inline void mm_lock_seqcount_begin(struct mm_struct *mm) {}
> -static inline void mm_lock_seqcount_end(struct mm_struct *mm) {}
> -
> -static inline bool mmap_lock_speculate_try_begin(struct mm_struct *mm, unsigned int *seq)
> -{
> -	return false;
> -}
> -
> -static inline bool mmap_lock_speculate_retry(struct mm_struct *mm, unsigned int seq)
> -{
> -	return true;
> -}
> -static inline void vma_lock_init(struct vm_area_struct *vma, bool reset_refcnt) {}
> -static inline void vma_end_read(struct vm_area_struct *vma) {}
> -static inline void vma_start_write(struct vm_area_struct *vma) {}
> -static inline __must_check
> -int vma_start_write_killable(struct vm_area_struct *vma) { return 0; }
> -static inline void vma_assert_write_locked(struct vm_area_struct *vma)
> -		{ mmap_assert_write_locked(vma->vm_mm); }
> -static inline bool vma_is_attached(struct vm_area_struct *vma)
> -		{ return true; }
> -static inline void vma_assert_attached(struct vm_area_struct *vma) {}
> -static inline void vma_assert_detached(struct vm_area_struct *vma) {}
> -static inline void vma_mark_attached(struct vm_area_struct *vma) {}
> -static inline void vma_mark_detached(struct vm_area_struct *vma) {}
> -
> -static inline struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
> -		unsigned long address)
> -{
> -	return NULL;
> -}
> -
> -static inline void vma_assert_locked(struct vm_area_struct *vma)
> -{
> -	mmap_assert_locked(vma->vm_mm);
> -}
> -
> -static inline void vma_assert_stabilised(struct vm_area_struct *vma)
> -{
> -	/* If no VMA locks, then either mmap lock suffices to stabilise. */
> -	mmap_assert_locked(vma->vm_mm);
> -}
> -
> -#endif /* CONFIG_PER_VMA_LOCK */
> -
>  static inline void vma_assert_can_modify(struct vm_area_struct *vma)
>  {
>  	if (vma_is_attached(vma))
> diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
> index 41fe87d7302f..8848e26ef581 100644
> --- a/kernel/bpf/stackmap.c
> +++ b/kernel/bpf/stackmap.c
> @@ -272,13 +272,8 @@ struct stack_map_vma_lock {
>  /*
>   * Acquire a stable read-side reference on the VMA covering @ip.
>   *
> - * With CONFIG_PER_VMA_LOCK=y this returns a VMA with its per-VMA read
> - * lock held and mmap_lock dropped, so the caller may sleep.
> - *
> - * With CONFIG_PER_VMA_LOCK=n it returns a VMA with mmap_lock still
> - * held; the caller must snapshot any fields it needs and pin vm_file
> - * with get_file() before stack_map_unlock_vma() drops mmap_lock, as
> - * the VMA may be split, merged, or freed after that.
> + * This returns a VMA with its per-VMA read lock held and mmap_lock
> + * dropped, so the caller may sleep.
>   *
>   * Returns NULL on failure, in which case no lock is held.
>   */
> @@ -288,7 +283,6 @@ stack_map_lock_vma(struct stack_map_vma_lock *lock, unsigned long ip)
>  	struct mm_struct *mm = lock->mm;
>  	struct vm_area_struct *vma;
>
> -	/* noop under !CONFIG_PER_VMA_LOCK */
>  	vma = lock_vma_under_rcu(mm, ip);
>  	if (vma) {
>  		lock->vma = vma;
> @@ -308,13 +302,11 @@ stack_map_lock_vma(struct stack_map_vma_lock *lock, unsigned long ip)
>  		return NULL;
>  	}
>
> -#ifdef CONFIG_PER_VMA_LOCK
>  	if (!vma_start_read_locked(vma)) {
>  		mmap_read_unlock(mm);
>  		return NULL;
>  	}
>  	mmap_read_unlock(mm);
> -#endif
>
>  	lock->vma = vma;
>  	return vma;
> @@ -322,11 +314,7 @@ stack_map_lock_vma(struct stack_map_vma_lock *lock, unsigned long ip)
>
>  static void stack_map_unlock_vma(struct stack_map_vma_lock *lock)
>  {
> -#ifdef CONFIG_PER_VMA_LOCK
>  	vma_end_read(lock->vma);
> -#else
> -	mmap_read_unlock(lock->mm);
> -#endif
>  	lock->vma = NULL;
>  }
>
> diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c
> index e791ae065c39..6cf815bc84be 100644
> --- a/kernel/bpf/task_iter.c
> +++ b/kernel/bpf/task_iter.c
> @@ -835,11 +835,6 @@ __bpf_kfunc int bpf_iter_task_vma_new(struct bpf_iter_task_vma *it,
>  	BUILD_BUG_ON(sizeof(struct bpf_iter_task_vma_kern) != sizeof(struct bpf_iter_task_vma));
>  	BUILD_BUG_ON(__alignof__(struct bpf_iter_task_vma_kern) != __alignof__(struct bpf_iter_task_vma));
>
> -	if (!IS_ENABLED(CONFIG_PER_VMA_LOCK)) {
> -		kit->data = NULL;
> -		return -EOPNOTSUPP;
> -	}
> -
>  	/*
>  	 * Reject irqs-disabled contexts including NMI. Operations used
>  	 * by _next() and _destroy() (vma_end_read, fput, bpf_iter_mmput_async)
> diff --git a/kernel/fork.c b/kernel/fork.c
> index f0e2e131a9a5..ff91f5f66c80 100644
> --- a/kernel/fork.c
> +++ b/kernel/fork.c
> @@ -1077,9 +1077,7 @@ static void mmap_init_lock(struct mm_struct *mm)
>  {
>  	init_rwsem(&mm->mmap_lock);
>  	mm_lock_seqcount_init(mm);
> -#ifdef CONFIG_PER_VMA_LOCK
>  	rcuwait_init(&mm->vma_writer_wait);
> -#endif
>  }
>
>  static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p)
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 060190e12bce..77103b46b679 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -1425,19 +1425,6 @@ config LRU_GEN_STATS
>  config LRU_GEN_WALKS_MMU
>  	def_bool y
>  	depends on LRU_GEN && ARCH_HAS_HW_PTE_YOUNG
> -# }
> -
> -config ARCH_SUPPORTS_PER_VMA_LOCK
> -       def_bool n
> -
> -config PER_VMA_LOCK
> -	def_bool y
> -	depends on ARCH_SUPPORTS_PER_VMA_LOCK && MMU && SMP
> -	help
> -	  Allow per-vma locking during page fault handling.
> -
> -	  This feature allows locking each virtual memory area separately when
> -	  handling page faults instead of taking mmap_lock.
>
>  config LOCK_MM_AND_FIND_VMA
>  	bool
> diff --git a/mm/Kconfig.debug b/mm/Kconfig.debug
> index 5737a504efbb..1dd150edfe71 100644
> --- a/mm/Kconfig.debug
> +++ b/mm/Kconfig.debug
> @@ -310,7 +310,6 @@ config DEBUG_KMEMLEAK_VERBOSE
>
>  config PER_VMA_LOCK_STATS
>  	bool "Statistics for per-vma locks"
> -	depends on PER_VMA_LOCK
>  	help
>  	  Say Y here to enable success, retry and failure counters of page
>  	  faults handled under protection of per-vma locks. When enabled, the
> diff --git a/mm/debug.c b/mm/debug.c
> index 9a0297b3988d..655e6bcc0e8d 100644
> --- a/mm/debug.c
> +++ b/mm/debug.c
> @@ -157,17 +157,13 @@ void dump_vma(const struct vm_area_struct *vma)
>  	pr_emerg("vma %px start %px end %px mm %px\n"
>  		"prot %lx anon_vma %px vm_ops %px\n"
>  		"pgoff %lx file %px private_data %px\n"
> -#ifdef CONFIG_PER_VMA_LOCK
>  		"refcnt %x\n"
> -#endif
>  		"flags: %#lx(%pGv)\n",
>  		vma, (void *)vma->vm_start, (void *)vma->vm_end, vma->vm_mm,
>  		(unsigned long)pgprot_val(vma->vm_page_prot),
>  		vma->anon_vma, vma->vm_ops, vma_start_pgoff(vma),
>  		vma->vm_file, vma->vm_private_data,
> -#ifdef CONFIG_PER_VMA_LOCK
>  		refcount_read(&vma->vm_refcnt),
> -#endif
>  		vma->vm_flags, &vma->vm_flags);
>  }
>  EXPORT_SYMBOL(dump_vma);
> diff --git a/mm/init-mm.c b/mm/init-mm.c
> index 3e792aad7626..a1bb2c2d0284 100644
> --- a/mm/init-mm.c
> +++ b/mm/init-mm.c
> @@ -39,10 +39,8 @@ struct mm_struct init_mm = {
>  	.page_table_lock =  __SPIN_LOCK_UNLOCKED(init_mm.page_table_lock),
>  	.arg_lock	=  __SPIN_LOCK_UNLOCKED(init_mm.arg_lock),
>  	.mmlist		= LIST_HEAD_INIT(init_mm.mmlist),
> -#ifdef CONFIG_PER_VMA_LOCK
>  	.vma_writer_wait = __RCUWAIT_INITIALIZER(init_mm.vma_writer_wait),
>  	.mm_lock_seq	= SEQCNT_ZERO(init_mm.mm_lock_seq),
> -#endif
>  #ifdef CONFIG_SCHED_MM_CID
>  	.mm_cid.lock = __RAW_SPIN_LOCK_UNLOCKED(init_mm.mm_cid.lock),
>  #endif
> diff --git a/mm/memory.c b/mm/memory.c
> index a620d425ec95..f109f6c87b28 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -6813,7 +6813,6 @@ static vm_fault_t sanitize_fault_flags(struct vm_area_struct *vma,
>  				 !is_cow_mapping(vma->vm_flags)))
>  			return VM_FAULT_SIGSEGV;
>  	}
> -#ifdef CONFIG_PER_VMA_LOCK
>  	/*
>  	 * Per-VMA locks can't be used with FAULT_FLAG_RETRY_NOWAIT because of
>  	 * the assumption that lock is dropped on VM_FAULT_RETRY.
> @@ -6822,7 +6821,6 @@ static vm_fault_t sanitize_fault_flags(struct vm_area_struct *vma,
>  			(FAULT_FLAG_VMA_LOCK | FAULT_FLAG_RETRY_NOWAIT)) ==
>  			(FAULT_FLAG_VMA_LOCK | FAULT_FLAG_RETRY_NOWAIT)))
>  		return VM_FAULT_SIGSEGV;
> -#endif
>
>  	return 0;
>  }
> diff --git a/mm/mmap_lock.c b/mm/mmap_lock.c
> index 898c2ef1e958..e20d01e8d38f 100644
> --- a/mm/mmap_lock.c
> +++ b/mm/mmap_lock.c
> @@ -43,9 +43,6 @@ void __mmap_lock_do_trace_released(struct mm_struct *mm, bool write)
>  EXPORT_SYMBOL(__mmap_lock_do_trace_released);
>  #endif /* CONFIG_TRACING */
>
> -#ifdef CONFIG_MMU
> -#ifdef CONFIG_PER_VMA_LOCK
> -
>  /* State shared across __vma_[start, end]_exclude_readers. */
>  struct vma_exclude_readers_state {
>  	/* Input parameters. */
> @@ -431,7 +428,6 @@ struct vm_area_struct *lock_next_vma(struct mm_struct *mm,
>
>  	return vma;
>  }
> -#endif /* CONFIG_PER_VMA_LOCK */
>
>  #ifdef CONFIG_LOCK_MM_AND_FIND_VMA
>  #include <linux/extable.h>
> @@ -548,23 +544,3 @@ struct vm_area_struct *lock_mm_and_find_vma(struct mm_struct *mm,
>  	return NULL;
>  }
>  #endif /* CONFIG_LOCK_MM_AND_FIND_VMA */
> -
> -#else /* CONFIG_MMU */
> -
> -/*
> - * At least xtensa ends up having protection faults even with no
> - * MMU.. No stack expansion, at least.
> - */
> -struct vm_area_struct *lock_mm_and_find_vma(struct mm_struct *mm,
> -			unsigned long addr, struct pt_regs *regs)
> -{
> -	struct vm_area_struct *vma;
> -
> -	mmap_read_lock(mm);
> -	vma = vma_lookup(mm, addr);
> -	if (!vma)
> -		mmap_read_unlock(mm);
> -	return vma;
> -}
> -
> -#endif /* CONFIG_MMU */
> diff --git a/mm/pagewalk.c b/mm/pagewalk.c
> index ed4860c01936..fbcf64c59a97 100644
> --- a/mm/pagewalk.c
> +++ b/mm/pagewalk.c
> @@ -446,7 +446,6 @@ static inline void process_mm_walk_lock(struct mm_struct *mm,
>  static inline void process_vma_walk_lock(struct vm_area_struct *vma,
>  					 enum page_walk_lock walk_lock)
>  {
> -#ifdef CONFIG_PER_VMA_LOCK
>  	switch (walk_lock) {
>  	case PGWALK_WRLOCK:
>  		vma_start_write(vma);
> @@ -461,7 +460,6 @@ static inline void process_vma_walk_lock(struct vm_area_struct *vma,
>  		/* PGWALK_RDLOCK is handled by process_mm_walk_lock */
>  		break;
>  	}
> -#endif
>  }
>
>  /*
> diff --git a/mm/rmap.c b/mm/rmap.c
> index b917431759ee..4e4a4b747977 100644
> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> @@ -260,11 +260,9 @@ static void check_anon_vma_clone(struct vm_area_struct *dst,
>  	/* For the anon_vma to be compatible, it can only be singular. */
>  	VM_WARN_ON_ONCE(operation == VMA_OP_MERGE_UNFAULTED &&
>  			!list_is_singular(&src->anon_vma_chain));
> -#ifdef CONFIG_PER_VMA_LOCK
>  	/* Only merging an unfaulted VMA leaves the destination attached. */
>  	VM_WARN_ON_ONCE(operation != VMA_OP_MERGE_UNFAULTED &&
>  			vma_is_attached(dst));
> -#endif
>  }
>
>  static void maybe_reuse_anon_vma(struct vm_area_struct *dst,
> diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
> index 258b03182a78..edd90892f8cc 100644
> --- a/mm/userfaultfd.c
> +++ b/mm/userfaultfd.c
> @@ -122,7 +122,6 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
>  	return vma;
>  }
>
> -#ifdef CONFIG_PER_VMA_LOCK
>  /*
>   * uffd_lock_vma() - Lookup and lock vma corresponding to @address.
>   * @mm: mm to search vma in.
> @@ -182,34 +181,6 @@ static void uffd_mfill_unlock(struct vm_area_struct *vma)
>  	vma_end_read(vma);
>  }
>
> -#else
> -
> -static struct vm_area_struct *uffd_mfill_lock(struct mm_struct *dst_mm,
> -					      unsigned long dst_start,
> -					      unsigned long len)
> -{
> -	struct vm_area_struct *dst_vma;
> -
> -	mmap_read_lock(dst_mm);
> -	dst_vma = find_vma_and_prepare_anon(dst_mm, dst_start);
> -	if (IS_ERR(dst_vma))
> -		goto out_unlock;
> -
> -	if (validate_dst_vma(dst_vma, dst_start + len))
> -		return dst_vma;
> -
> -	dst_vma = ERR_PTR(-ENOENT);
> -out_unlock:
> -	mmap_read_unlock(dst_mm);
> -	return dst_vma;
> -}
> -
> -static void uffd_mfill_unlock(struct vm_area_struct *vma)
> -{
> -	mmap_read_unlock(vma->vm_mm);
> -}
> -#endif
> -
>  static void mfill_put_vma(struct mfill_state *state)
>  {
>  	if (!state->vma)
> @@ -1852,7 +1823,6 @@ int find_vmas_mm_locked(struct mm_struct *mm,
>  	return 0;
>  }
>
> -#ifdef CONFIG_PER_VMA_LOCK
>  static int uffd_move_lock(struct mm_struct *mm,
>  			  unsigned long dst_start,
>  			  unsigned long src_start,
> @@ -1927,31 +1897,6 @@ static void uffd_move_unlock(struct vm_area_struct *dst_vma,
>  		vma_end_read(dst_vma);
>  }
>
> -#else
> -
> -static int uffd_move_lock(struct mm_struct *mm,
> -			  unsigned long dst_start,
> -			  unsigned long src_start,
> -			  struct vm_area_struct **dst_vmap,
> -			  struct vm_area_struct **src_vmap)
> -{
> -	int err;
> -
> -	mmap_read_lock(mm);
> -	err = find_vmas_mm_locked(mm, dst_start, src_start, dst_vmap, src_vmap);
> -	if (err)
> -		mmap_read_unlock(mm);
> -	return err;
> -}
> -
> -static void uffd_move_unlock(struct vm_area_struct *dst_vma,
> -			     struct vm_area_struct *src_vma)
> -{
> -	mmap_assert_locked(src_vma->vm_mm);
> -	mmap_read_unlock(dst_vma->vm_mm);
> -}
> -#endif
> -
>  /**
>   * move_pages - move arbitrary anonymous pages of an existing vma
>   * @ctx: pointer to the userfaultfd context
> diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs
> index 4764d7b68f2a..2633e704c83d 100644
> --- a/rust/kernel/mm.rs
> +++ b/rust/kernel/mm.rs
> @@ -174,26 +174,20 @@ pub unsafe fn from_raw<'a>(ptr: *const bindings::mm_struct) -> &'a MmWithUser {
>      /// When per-vma locks are disabled, this always returns `None`.
>      #[inline]
>      pub fn lock_vma_under_rcu(&self, vma_addr: usize) -> Option<VmaReadGuard<'_>> {
> -        #[cfg(CONFIG_PER_VMA_LOCK)]
>          {
>              // SAFETY: Calling `bindings::lock_vma_under_rcu` is always okay given an mm where
>              // `mm_users` is non-zero.
>              let vma = unsafe { bindings::lock_vma_under_rcu(self.as_raw(), vma_addr) };
> -            if !vma.is_null() {
> -                return Some(VmaReadGuard {
> -                    // SAFETY: If `lock_vma_under_rcu` returns a non-null ptr, then it points at a
> -                    // valid vma. The vma is stable for as long as the vma read lock is held.
> -                    vma: unsafe { VmaRef::from_raw(vma) },
> -                    _nts: NotThreadSafe,
> -                });
> +            if vma.is_null() {
> +                return None;
>              }
> +            Some(VmaReadGuard {
> +                // SAFETY: If `lock_vma_under_rcu` returns a non-null ptr, then it points at a
> +                // valid vma. The vma is stable for as long as the vma read lock is held.
> +                vma: unsafe { VmaRef::from_raw(vma) },
> +                _nts: NotThreadSafe,
> +            })
>          }
> -
> -        // Silence warnings about unused variables.
> -        #[cfg(not(CONFIG_PER_VMA_LOCK))]
> -        let _ = vma_addr;
> -
> -        None
>      }
>
>      /// Lock the mmap read lock.
> diff --git a/tools/testing/vma/include/dup.h b/tools/testing/vma/include/dup.h
> index 800e5fa02d78..362feda28526 100644
> --- a/tools/testing/vma/include/dup.h
> +++ b/tools/testing/vma/include/dup.h
> @@ -582,7 +582,6 @@ struct vm_area_struct {
>  		vma_flags_t flags;
>  	};
>
> -#ifdef CONFIG_PER_VMA_LOCK
>  	/*
>  	 * Can only be written (using WRITE_ONCE()) while holding both:
>  	 *  - mmap_lock (in write mode)
> @@ -598,7 +597,7 @@ struct vm_area_struct {
>  	 * slowpath.
>  	 */
>  	unsigned int vm_lock_seq;
> -#endif
> +
>  	unsigned int __vm_virt_pgoff_lo;
>
>  	/*
> @@ -632,10 +631,8 @@ struct vm_area_struct {
>  #ifdef CONFIG_NUMA_BALANCING
>  	struct vma_numab_state *numab_state;	/* NUMA Balancing state */
>  #endif
> -#ifdef CONFIG_PER_VMA_LOCK
>  	/* Unstable RCU readers are allowed to read this. */
>  	refcount_t vm_refcnt;
> -#endif
>  #ifdef CONFIG_64BIT
>  	unsigned int __vm_virt_pgoff_hi;
>  #endif
> diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h
> index 8a48b231aa7a..54d5c3360aa2 100644
> --- a/tools/testing/vma/vma_internal.h
> +++ b/tools/testing/vma/vma_internal.h
> @@ -15,7 +15,6 @@
>  #include <stdlib.h>
>
>  #define CONFIG_MMU		1
> -#define CONFIG_PER_VMA_LOCK	1
>
>  #ifdef __CONCAT
>  #undef __CONCAT
> --
> 2.55.0.508.g3f0d502094-goog
>

--
Cheers, Lorenzo

Re: [PATCH v3 1/5] mm: Make per-VMA locks available universally

[Vlastimil Babka (SUSE)]

On 8/2/26 23:54, Suren Baghdasaryan wrote:
> From: Dave Hansen <dave.hansen@linux.intel.com>
> 
> The per-VMA locks have been around for several years. They've had some
> bugs worked out of them and have seen quite wide use. However, they
> are still only available when architectures explicitly enable them.
> Remove the conditional compilation around the per-VMA locks, making
> them available on all architectures and configs.
> 
> The approach up to now seemed to be to add ARCH_SUPPORTS_PER_VMA_LOCK
> when the architecture started using per-VMA locks in the fault
> handler. But, contrary to the naming, the Kconfig option does not
> really indicate whether the architecture supports per-VMA locks or
> not. It is more of a marker for whether the architecture is likely to
> benefit from per-VMA locks.
> 
> To me, the most important thing side-effect of universal availability
> is letting per-VMA locks be used in SMP=n configs. This lets us use
> per-VMA locking in all x86 code without fallbacks.
> 
> Overall, this just generally makes the kernel simpler. Just look at
> the diffstat. It also opens the door to users that want to use the
> per-VMA locks in common code. Doing *that* brings additional
> simplifications.
> 
> The downside of this is adding some fields to vm_area_struct and
> mm_struct. There are likely ways to optimize this, especially for
> things like SMP=n configs. For now, do the simplest thing: use the
> same implementation everywhere.
> 
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Signed-off-by: Suren Baghdasaryan <surenb@google.com>

Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>

Nits:

> diff --git a/mm/Kconfig b/mm/Kconfig
> index 060190e12bce..77103b46b679 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -1425,19 +1425,6 @@ config LRU_GEN_STATS
>  config LRU_GEN_WALKS_MMU
>  	def_bool y
>  	depends on LRU_GEN && ARCH_HAS_HW_PTE_YOUNG
> -# }

I think either leave this in place, or remove the line above with "#
multi-gen LRU {" as well?

> -
> -config ARCH_SUPPORTS_PER_VMA_LOCK
> -       def_bool n
> -
> -config PER_VMA_LOCK
> -	def_bool y
> -	depends on ARCH_SUPPORTS_PER_VMA_LOCK && MMU && SMP
> -	help
> -	  Allow per-vma locking during page fault handling.
> -
> -	  This feature allows locking each virtual memory area separately when
> -	  handling page faults instead of taking mmap_lock.
>  
>  config LOCK_MM_AND_FIND_VMA
>  	bool

...

> diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs
> index 4764d7b68f2a..2633e704c83d 100644
> --- a/rust/kernel/mm.rs
> +++ b/rust/kernel/mm.rs
> @@ -174,26 +174,20 @@ pub unsafe fn from_raw<'a>(ptr: *const bindings::mm_struct) -> &'a MmWithUser {
>      /// When per-vma locks are disabled, this always returns `None`.
>      #[inline]
>      pub fn lock_vma_under_rcu(&self, vma_addr: usize) -> Option<VmaReadGuard<'_>> {
> -        #[cfg(CONFIG_PER_VMA_LOCK)]
>          {
>              // SAFETY: Calling `bindings::lock_vma_under_rcu` is always okay given an mm where
>              // `mm_users` is non-zero.
>              let vma = unsafe { bindings::lock_vma_under_rcu(self.as_raw(), vma_addr) };
> -            if !vma.is_null() {
> -                return Some(VmaReadGuard {
> -                    // SAFETY: If `lock_vma_under_rcu` returns a non-null ptr, then it points at a
> -                    // valid vma. The vma is stable for as long as the vma read lock is held.
> -                    vma: unsafe { VmaRef::from_raw(vma) },
> -                    _nts: NotThreadSafe,
> -                });
> +            if vma.is_null() {
> +                return None;
>              }
> +            Some(VmaReadGuard {
> +                // SAFETY: If `lock_vma_under_rcu` returns a non-null ptr, then it points at a
> +                // valid vma. The vma is stable for as long as the vma read lock is held.
> +                vma: unsafe { VmaRef::from_raw(vma) },
> +                _nts: NotThreadSafe,
> +            })
>          }

With the "#[cfg(CONFIG_PER_VMA_LOCK)]" does this still need to be in a { }
block?

> -
> -        // Silence warnings about unused variables.
> -        #[cfg(not(CONFIG_PER_VMA_LOCK))]
> -        let _ = vma_addr;
> -
> -        None
>      }
>  
>      /// Lock the mmap read lock.

Re: [PATCH v3 1/5] mm: Make per-VMA locks available universally

[Suren Baghdasaryan]

On Mon, Aug 3, 2026 at 7:01 AM Vlastimil Babka (SUSE) <vbabka@kernel.org> wrote:
>
> On 8/2/26 23:54, Suren Baghdasaryan wrote:
> > From: Dave Hansen <dave.hansen@linux.intel.com>
> >
> > The per-VMA locks have been around for several years. They've had some
> > bugs worked out of them and have seen quite wide use. However, they
> > are still only available when architectures explicitly enable them.
> > Remove the conditional compilation around the per-VMA locks, making
> > them available on all architectures and configs.
> >
> > The approach up to now seemed to be to add ARCH_SUPPORTS_PER_VMA_LOCK
> > when the architecture started using per-VMA locks in the fault
> > handler. But, contrary to the naming, the Kconfig option does not
> > really indicate whether the architecture supports per-VMA locks or
> > not. It is more of a marker for whether the architecture is likely to
> > benefit from per-VMA locks.
> >
> > To me, the most important thing side-effect of universal availability
> > is letting per-VMA locks be used in SMP=n configs. This lets us use
> > per-VMA locking in all x86 code without fallbacks.
> >
> > Overall, this just generally makes the kernel simpler. Just look at
> > the diffstat. It also opens the door to users that want to use the
> > per-VMA locks in common code. Doing *that* brings additional
> > simplifications.
> >
> > The downside of this is adding some fields to vm_area_struct and
> > mm_struct. There are likely ways to optimize this, especially for
> > things like SMP=n configs. For now, do the simplest thing: use the
> > same implementation everywhere.
> >
> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
>
> Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
>
> Nits:
>
> > diff --git a/mm/Kconfig b/mm/Kconfig
> > index 060190e12bce..77103b46b679 100644
> > --- a/mm/Kconfig
> > +++ b/mm/Kconfig
> > @@ -1425,19 +1425,6 @@ config LRU_GEN_STATS
> >  config LRU_GEN_WALKS_MMU
> >       def_bool y
> >       depends on LRU_GEN && ARCH_HAS_HW_PTE_YOUNG
> > -# }
>
> I think either leave this in place, or remove the line above with "#
> multi-gen LRU {" as well?

Ah, missed it. I'll keep it in place.

>
> > -
> > -config ARCH_SUPPORTS_PER_VMA_LOCK
> > -       def_bool n
> > -
> > -config PER_VMA_LOCK
> > -     def_bool y
> > -     depends on ARCH_SUPPORTS_PER_VMA_LOCK && MMU && SMP
> > -     help
> > -       Allow per-vma locking during page fault handling.
> > -
> > -       This feature allows locking each virtual memory area separately when
> > -       handling page faults instead of taking mmap_lock.
> >
> >  config LOCK_MM_AND_FIND_VMA
> >       bool
>
> ...
>
> > diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs
> > index 4764d7b68f2a..2633e704c83d 100644
> > --- a/rust/kernel/mm.rs
> > +++ b/rust/kernel/mm.rs
> > @@ -174,26 +174,20 @@ pub unsafe fn from_raw<'a>(ptr: *const bindings::mm_struct) -> &'a MmWithUser {
> >      /// When per-vma locks are disabled, this always returns `None`.
> >      #[inline]
> >      pub fn lock_vma_under_rcu(&self, vma_addr: usize) -> Option<VmaReadGuard<'_>> {
> > -        #[cfg(CONFIG_PER_VMA_LOCK)]
> >          {
> >              // SAFETY: Calling `bindings::lock_vma_under_rcu` is always okay given an mm where
> >              // `mm_users` is non-zero.
> >              let vma = unsafe { bindings::lock_vma_under_rcu(self.as_raw(), vma_addr) };
> > -            if !vma.is_null() {
> > -                return Some(VmaReadGuard {
> > -                    // SAFETY: If `lock_vma_under_rcu` returns a non-null ptr, then it points at a
> > -                    // valid vma. The vma is stable for as long as the vma read lock is held.
> > -                    vma: unsafe { VmaRef::from_raw(vma) },
> > -                    _nts: NotThreadSafe,
> > -                });
> > +            if vma.is_null() {
> > +                return None;
> >              }
> > +            Some(VmaReadGuard {
> > +                // SAFETY: If `lock_vma_under_rcu` returns a non-null ptr, then it points at a
> > +                // valid vma. The vma is stable for as long as the vma read lock is held.
> > +                vma: unsafe { VmaRef::from_raw(vma) },
> > +                _nts: NotThreadSafe,
> > +            })
> >          }
>
> With the "#[cfg(CONFIG_PER_VMA_LOCK)]" does this still need to be in a { }
> block?

I don't think it needs to be. I'll remove the extra braces.
Thanks!

>
> > -
> > -        // Silence warnings about unused variables.
> > -        #[cfg(not(CONFIG_PER_VMA_LOCK))]
> > -        let _ = vma_addr;
> > -
> > -        None
> >      }
> >
> >      /// Lock the mmap read lock.

Re: [PATCH v3 1/5] mm: Make per-VMA locks available universally

[Suren Baghdasaryan]

On Sun, Aug 2, 2026 at 2:55 PM Suren Baghdasaryan <surenb@google.com> wrote:
>
> From: Dave Hansen <dave.hansen@linux.intel.com>
>
> The per-VMA locks have been around for several years. They've had some
> bugs worked out of them and have seen quite wide use. However, they
> are still only available when architectures explicitly enable them.
> Remove the conditional compilation around the per-VMA locks, making
> them available on all architectures and configs.
>
> The approach up to now seemed to be to add ARCH_SUPPORTS_PER_VMA_LOCK
> when the architecture started using per-VMA locks in the fault
> handler. But, contrary to the naming, the Kconfig option does not
> really indicate whether the architecture supports per-VMA locks or
> not. It is more of a marker for whether the architecture is likely to
> benefit from per-VMA locks.
>
> To me, the most important thing side-effect of universal availability
> is letting per-VMA locks be used in SMP=n configs. This lets us use
> per-VMA locking in all x86 code without fallbacks.
>
> Overall, this just generally makes the kernel simpler. Just look at
> the diffstat. It also opens the door to users that want to use the
> per-VMA locks in common code. Doing *that* brings additional
> simplifications.
>
> The downside of this is adding some fields to vm_area_struct and
> mm_struct. There are likely ways to optimize this, especially for
> things like SMP=n configs. For now, do the simplest thing: use the
> same implementation everywhere.
>
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> Cc: Suren Baghdasaryan <surenb@google.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> Cc: Lorenzo Stoakes <ljs@kernel.org>
> Cc: Vlastimil Babka <vbabka@kernel.org>
> Cc: Shakeel Butt <shakeel.butt@linux.dev>
> Cc: linux-mm@kvack.org
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Arve Hjønnevåg <arve@android.com>
> Cc: Todd Kjos <tkjos@android.com>
> Cc: Christian Brauner <christian@brauner.io>
> Cc: Carlos Llamas <cmllamas@google.com>
> Cc: Alice Ryhl <aliceryhl@google.com>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: David Ahern <dsahern@kernel.org>
> Cc: netdev@vger.kernel.org
> ---
>  arch/arm/Kconfig                       |  1 -
>  arch/arm64/Kconfig                     |  1 -
>  arch/loongarch/Kconfig                 |  1 -
>  arch/powerpc/platforms/powernv/Kconfig |  1 -
>  arch/powerpc/platforms/pseries/Kconfig |  1 -
>  arch/riscv/Kconfig                     |  1 -
>  arch/s390/Kconfig                      |  1 -
>  arch/x86/Kconfig                       |  2 -
>  fs/proc/internal.h                     |  2 -
>  fs/proc/task_mmu.c                     | 93 --------------------------
>  include/linux/mm.h                     | 12 ----
>  include/linux/mm_types.h               |  8 +--
>  include/linux/mmap_lock.h              | 50 --------------
>  kernel/bpf/stackmap.c                  | 16 +----
>  kernel/bpf/task_iter.c                 |  5 --
>  kernel/fork.c                          |  2 -
>  mm/Kconfig                             | 13 ----
>  mm/Kconfig.debug                       |  1 -
>  mm/debug.c                             |  4 --
>  mm/init-mm.c                           |  2 -
>  mm/memory.c                            |  2 -
>  mm/mmap_lock.c                         | 24 -------
>  mm/pagewalk.c                          |  2 -
>  mm/rmap.c                              |  2 -
>  mm/userfaultfd.c                       | 55 ---------------
>  rust/kernel/mm.rs                      | 22 +++---
>  tools/testing/vma/include/dup.h        |  5 +-
>  tools/testing/vma/vma_internal.h       |  1 -
>  28 files changed, 13 insertions(+), 317 deletions(-)
>
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index 9187240a02db..f815209167cd 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -41,7 +41,6 @@ config ARM
>         select ARCH_SUPPORTS_ATOMIC_RMW
>         select ARCH_SUPPORTS_CFI
>         select ARCH_SUPPORTS_HUGETLBFS if ARM_LPAE
> -       select ARCH_SUPPORTS_PER_VMA_LOCK
>         select ARCH_SUPPORTS_RT
>         select ARCH_USE_BUILTIN_BSWAP
>         select ARCH_USE_CMPXCHG_LOCKREF
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 11a9c534b7b4..21eb64b24a2c 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -81,7 +81,6 @@ config ARM64
>         select ARCH_HAS_PTE_PROTNONE
>         select ARCH_SUPPORTS_NUMA_BALANCING
>         select ARCH_SUPPORTS_PAGE_TABLE_CHECK
> -       select ARCH_SUPPORTS_PER_VMA_LOCK
>         select ARCH_SUPPORTS_HUGE_PFNMAP if TRANSPARENT_HUGEPAGE
>         select ARCH_SUPPORTS_RT
>         select ARCH_SUPPORTS_SCHED_SMT
> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
> index e20acbe5fe7b..7741e39eca2b 100644
> --- a/arch/loongarch/Kconfig
> +++ b/arch/loongarch/Kconfig
> @@ -69,7 +69,6 @@ config LOONGARCH
>         select ARCH_SUPPORTS_MSEAL_SYSTEM_MAPPINGS
>         select ARCH_HAS_PTE_PROTNONE if 64BIT
>         select ARCH_SUPPORTS_NUMA_BALANCING if NUMA
> -       select ARCH_SUPPORTS_PER_VMA_LOCK
>         select ARCH_SUPPORTS_RT
>         select ARCH_SUPPORTS_SCHED_SMT if SMP
>         select ARCH_SUPPORTS_SCHED_MC  if SMP
> diff --git a/arch/powerpc/platforms/powernv/Kconfig b/arch/powerpc/platforms/powernv/Kconfig
> index b5ad7c173ef0..dd8f6060fb7a 100644
> --- a/arch/powerpc/platforms/powernv/Kconfig
> +++ b/arch/powerpc/platforms/powernv/Kconfig
> @@ -17,7 +17,6 @@ config PPC_POWERNV
>         select PPC_DOORBELL
>         select MMU_NOTIFIER
>         select FORCE_SMP
> -       select ARCH_SUPPORTS_PER_VMA_LOCK
>         select PPC_RADIX_BROADCAST_TLBIE if PPC_RADIX_MMU
>         default y
>
> diff --git a/arch/powerpc/platforms/pseries/Kconfig b/arch/powerpc/platforms/pseries/Kconfig
> index 74910ce3a541..7d125e288f6e 100644
> --- a/arch/powerpc/platforms/pseries/Kconfig
> +++ b/arch/powerpc/platforms/pseries/Kconfig
> @@ -23,7 +23,6 @@ config PPC_PSERIES
>         select HOTPLUG_CPU
>         select FORCE_SMP
>         select SWIOTLB
> -       select ARCH_SUPPORTS_PER_VMA_LOCK
>         select PPC_RADIX_BROADCAST_TLBIE if PPC_RADIX_MMU
>         default y
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 7b9c373d82fa..faa85a031fe5 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -70,7 +70,6 @@ config RISCV
>         select ARCH_SUPPORTS_LTO_CLANG_THIN
>         select ARCH_SUPPORTS_MSEAL_SYSTEM_MAPPINGS if 64BIT && MMU
>         select ARCH_SUPPORTS_PAGE_TABLE_CHECK if MMU
> -       select ARCH_SUPPORTS_PER_VMA_LOCK if MMU
>         select ARCH_HAS_PTE_PROTNONE if MMU
>         select ARCH_SUPPORTS_RT
>         select ARCH_SUPPORTS_SHADOW_CALL_STACK if HAVE_SHADOW_CALL_STACK
> diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
> index ab8fccc2cc4e..d1274bca8c39 100644
> --- a/arch/s390/Kconfig
> +++ b/arch/s390/Kconfig
> @@ -151,7 +151,6 @@ config S390
>         select ARCH_HAS_PTE_PROTNONE
>         select ARCH_SUPPORTS_NUMA_BALANCING
>         select ARCH_SUPPORTS_PAGE_TABLE_CHECK
> -       select ARCH_SUPPORTS_PER_VMA_LOCK
>         select ARCH_USE_BUILTIN_BSWAP
>         select ARCH_USE_CMPXCHG_LOCKREF
>         select ARCH_USE_SYM_ANNOTATIONS
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index fb298e219179..79479d29576f 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -27,7 +27,6 @@ config X86_64
>         select ARCH_HAS_GIGANTIC_PAGE
>         select ARCH_SUPPORTS_MSEAL_SYSTEM_MAPPINGS
>         select ARCH_SUPPORTS_INT128 if CC_HAS_INT128
> -       select ARCH_SUPPORTS_PER_VMA_LOCK
>         select ARCH_SUPPORTS_HUGE_PFNMAP if TRANSPARENT_HUGEPAGE
>         select HAVE_ARCH_SOFT_DIRTY
>         select MODULES_USE_ELF_RELA
> @@ -1846,7 +1845,6 @@ config X86_USER_SHADOW_STACK
>         bool "X86 userspace shadow stack"
>         depends on AS_WRUSS
>         depends on X86_64
> -       depends on PER_VMA_LOCK
>         select ARCH_USES_HIGH_VMA_FLAGS
>         select ARCH_HAS_USER_SHADOW_STACK
>         select X86_CET
> diff --git a/fs/proc/internal.h b/fs/proc/internal.h
> index b232e1098117..6713757da099 100644
> --- a/fs/proc/internal.h
> +++ b/fs/proc/internal.h
> @@ -385,10 +385,8 @@ struct mem_size_stats;
>
>  struct proc_maps_locking_ctx {
>         struct mm_struct *mm;
> -#ifdef CONFIG_PER_VMA_LOCK
>         bool mmap_locked;
>         struct vm_area_struct *locked_vma;
> -#endif
>  };
>
>  struct proc_maps_private {
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index 817e3e0f9194..096bf0b0b9e0 100644
> --- a/fs/proc/task_mmu.c
> +++ b/fs/proc/task_mmu.c
> @@ -130,8 +130,6 @@ static void release_task_mempolicy(struct proc_maps_private *priv)
>  }
>  #endif
>
> -#ifdef CONFIG_PER_VMA_LOCK
> -
>  static inline int lock_ctx_mm(struct proc_maps_locking_ctx *lock_ctx)
>  {
>         int ret = mmap_read_lock_killable(lock_ctx->mm);
> @@ -233,46 +231,6 @@ static inline void reacquire_rcu(struct proc_maps_private *priv)
>         vma_iter_set(&priv->iter, priv->lock_ctx.locked_vma->vm_end);
>  }
>
> -#else /* CONFIG_PER_VMA_LOCK */
> -
> -static inline int lock_ctx_mm(struct proc_maps_locking_ctx *lock_ctx)
> -{
> -       return mmap_read_lock_killable(lock_ctx->mm);
> -}
> -
> -static inline void unlock_ctx_mm(struct proc_maps_locking_ctx *lock_ctx)
> -{
> -       mmap_read_unlock(lock_ctx->mm);
> -}
> -
> -static inline bool lock_vma_range(struct seq_file *m,
> -                                 struct proc_maps_locking_ctx *lock_ctx)
> -{
> -       return lock_ctx_mm(lock_ctx) == 0;
> -}
> -
> -static inline void unlock_vma_range(struct proc_maps_locking_ctx *lock_ctx)
> -{
> -       unlock_ctx_mm(lock_ctx);
> -}
> -
> -static struct vm_area_struct *get_next_vma(struct proc_maps_private *priv,
> -                                          loff_t last_pos)
> -{
> -       return vma_next(&priv->iter);
> -}
> -
> -static inline bool fallback_to_mmap_lock(struct proc_maps_private *priv,
> -                                        loff_t pos)
> -{
> -       return false;
> -}
> -
> -static inline void drop_rcu(struct proc_maps_private *priv) {}
> -static inline void reacquire_rcu(struct proc_maps_private *priv) {}
> -
> -#endif /* CONFIG_PER_VMA_LOCK */
> -
>  static struct vm_area_struct *proc_get_vma(struct seq_file *m, loff_t *ppos)
>  {
>         struct proc_maps_private *priv = m->private;
> @@ -560,8 +518,6 @@ static int pid_maps_open(struct inode *inode, struct file *file)
>                 PROCMAP_QUERY_VMA_FLAGS                         \
>  )
>
> -#ifdef CONFIG_PER_VMA_LOCK
> -
>  static int query_vma_setup(struct proc_maps_locking_ctx *lock_ctx)
>  {
>         reset_lock_ctx(lock_ctx);
> @@ -612,26 +568,6 @@ static struct vm_area_struct *query_vma_find_by_addr(struct proc_maps_locking_ct
>         return vma;
>  }
>
> -#else /* CONFIG_PER_VMA_LOCK */
> -
> -static int query_vma_setup(struct proc_maps_locking_ctx *lock_ctx)
> -{
> -       return mmap_read_lock_killable(lock_ctx->mm);
> -}
> -
> -static void query_vma_teardown(struct proc_maps_locking_ctx *lock_ctx)
> -{
> -       mmap_read_unlock(lock_ctx->mm);
> -}
> -
> -static struct vm_area_struct *query_vma_find_by_addr(struct proc_maps_locking_ctx *lock_ctx,
> -                                                    unsigned long addr)
> -{
> -       return find_vma(lock_ctx->mm, addr);
> -}
> -
> -#endif  /* CONFIG_PER_VMA_LOCK */
> -
>  static struct vm_area_struct *query_matching_vma(struct proc_maps_locking_ctx *lock_ctx,
>                                                  unsigned long addr, u32 flags)
>  {
> @@ -1314,8 +1250,6 @@ static const struct mm_walk_ops smaps_shmem_walk_ops = {
>         .walk_lock              = PGWALK_RDLOCK,
>  };
>
> -#ifdef CONFIG_PER_VMA_LOCK
> -
>  static const struct mm_walk_ops smaps_walk_vma_lock_ops = {
>         .pmd_entry              = smaps_pte_range,
>         .hugetlb_entry          = smaps_hugetlb_range,
> @@ -1345,22 +1279,6 @@ get_smaps_shmem_walk_ops(struct proc_maps_private *priv)
>         return &smaps_shmem_walk_vma_lock_ops;
>  }
>
> -#else /* CONFIG_PER_VMA_LOCK */
> -
> -static inline const struct mm_walk_ops *
> -get_smaps_walk_ops(struct proc_maps_private *priv)
> -{
> -       return &smaps_walk_ops;
> -}
> -
> -static inline const struct mm_walk_ops *
> -get_smaps_shmem_walk_ops(struct proc_maps_private *priv)
> -{
> -       return &smaps_shmem_walk_ops;
> -}
> -
> -#endif /* CONFIG_PER_VMA_LOCK */
> -
>  /*
>   * Gather mem stats from @vma with the indicated beginning
>   * address @start, and keep them in @mss.
> @@ -3497,7 +3415,6 @@ static const struct mm_walk_ops show_numa_ops = {
>         .walk_lock = PGWALK_RDLOCK,
>  };
>
> -#ifdef CONFIG_PER_VMA_LOCK
>  static const struct mm_walk_ops show_numa_vma_lock_ops = {
>         .hugetlb_entry = gather_hugetlb_stats,
>         .pmd_entry = gather_pte_stats,
> @@ -3512,16 +3429,6 @@ get_show_numa_ops(struct proc_maps_private *priv)
>         return &show_numa_vma_lock_ops;
>  }
>
> -#else /* CONFIG_PER_VMA_LOCK */
> -
> -static inline const struct mm_walk_ops *
> -get_show_numa_ops(struct proc_maps_private *priv)
> -{
> -       return &show_numa_ops;
> -}
> -
> -#endif /* CONFIG_PER_VMA_LOCK */
> -
>  /*
>   * Display pages allocated per node and memory policy via /proc.
>   */
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index 7fabe6c66b4b..d9850f846242 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -931,7 +931,6 @@ static inline void vma_numab_state_free(struct vm_area_struct *vma) {}
>   * These must be here rather than mmap_lock.h as dependent on vm_fault type,
>   * declared in this header.
>   */
> -#ifdef CONFIG_PER_VMA_LOCK
>  static inline void release_fault_lock(struct vm_fault *vmf)
>  {
>         if (vmf->flags & FAULT_FLAG_VMA_LOCK)
> @@ -947,17 +946,6 @@ static inline void assert_fault_locked(const struct vm_fault *vmf)
>         else
>                 mmap_assert_locked(vmf->vma->vm_mm);
>  }
> -#else
> -static inline void release_fault_lock(struct vm_fault *vmf)
> -{
> -       mmap_read_unlock(vmf->vma->vm_mm);
> -}
> -
> -static inline void assert_fault_locked(const struct vm_fault *vmf)
> -{
> -       mmap_assert_locked(vmf->vma->vm_mm);
> -}
> -#endif /* CONFIG_PER_VMA_LOCK */
>
>  static inline bool mm_flags_test(int flag, const struct mm_struct *mm)
>  {
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index b5d4cd3b067b..d8e246fd09d3 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -950,7 +950,6 @@ struct vm_area_struct {
>                 vma_flags_t flags;
>         };
>
> -#ifdef CONFIG_PER_VMA_LOCK
>         /*
>          * Can only be written (using WRITE_ONCE()) while holding both:
>          *  - mmap_lock (in write mode)
> @@ -966,7 +965,7 @@ struct vm_area_struct {
>          * slowpath.
>          */
>         unsigned int vm_lock_seq;
> -#endif
> +
>         /*
>          * Low 32-bits of virtual page offset.
>          * See vma_start_virt_pgoff() comment for details.
> @@ -1003,7 +1002,6 @@ struct vm_area_struct {
>  #ifdef CONFIG_NUMA_BALANCING
>         struct vma_numab_state *numab_state;    /* NUMA Balancing state */
>  #endif
> -#ifdef CONFIG_PER_VMA_LOCK
>         /*
>          * Used to keep track of firstly, whether the VMA is attached, secondly,
>          * if attached, how many read locks are taken, and thirdly, if the
> @@ -1046,7 +1044,6 @@ struct vm_area_struct {
>  #ifdef CONFIG_DEBUG_LOCK_ALLOC
>         struct lockdep_map vmlock_dep_map;
>  #endif
> -#endif
>  #ifdef CONFIG_64BIT
>         /*
>          * High 32-bits of virtual page offset.
> @@ -1254,7 +1251,6 @@ struct mm_struct {
>                                           * init_mm.mmlist, and are protected
>                                           * by mmlist_lock
>                                           */
> -#ifdef CONFIG_PER_VMA_LOCK
>                 struct rcuwait vma_writer_wait;
>                 /*
>                  * This field has lock-like semantics, meaning it is sometimes
> @@ -1274,7 +1270,7 @@ struct mm_struct {
>                  * mmap_lock.
>                  */
>                 seqcount_t mm_lock_seq;
> -#endif
> +
>                 struct futex_mm_data    futex;
>
>                 unsigned long hiwater_rss; /* High-watermark of RSS usage */
> diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
> index 87f77e3da77f..eb32b482434e 100644
> --- a/include/linux/mmap_lock.h
> +++ b/include/linux/mmap_lock.h
> @@ -76,8 +76,6 @@ static inline void mmap_assert_write_locked(const struct mm_struct *mm)
>         rwsem_assert_held_write(&mm->mmap_lock);
>  }
>
> -#ifdef CONFIG_PER_VMA_LOCK
> -
>  #ifdef CONFIG_LOCKDEP
>  #define __vma_lockdep_map(vma) (&vma->vmlock_dep_map)
>  #else
> @@ -484,54 +482,6 @@ struct vm_area_struct *lock_next_vma(struct mm_struct *mm,
>                                      struct vma_iterator *iter,
>                                      unsigned long address);
>
> -#else /* CONFIG_PER_VMA_LOCK */
> -
> -static inline void mm_lock_seqcount_init(struct mm_struct *mm) {}
> -static inline void mm_lock_seqcount_begin(struct mm_struct *mm) {}
> -static inline void mm_lock_seqcount_end(struct mm_struct *mm) {}
> -
> -static inline bool mmap_lock_speculate_try_begin(struct mm_struct *mm, unsigned int *seq)
> -{
> -       return false;
> -}
> -
> -static inline bool mmap_lock_speculate_retry(struct mm_struct *mm, unsigned int seq)
> -{
> -       return true;
> -}
> -static inline void vma_lock_init(struct vm_area_struct *vma, bool reset_refcnt) {}
> -static inline void vma_end_read(struct vm_area_struct *vma) {}
> -static inline void vma_start_write(struct vm_area_struct *vma) {}
> -static inline __must_check
> -int vma_start_write_killable(struct vm_area_struct *vma) { return 0; }
> -static inline void vma_assert_write_locked(struct vm_area_struct *vma)
> -               { mmap_assert_write_locked(vma->vm_mm); }
> -static inline bool vma_is_attached(struct vm_area_struct *vma)
> -               { return true; }
> -static inline void vma_assert_attached(struct vm_area_struct *vma) {}
> -static inline void vma_assert_detached(struct vm_area_struct *vma) {}
> -static inline void vma_mark_attached(struct vm_area_struct *vma) {}
> -static inline void vma_mark_detached(struct vm_area_struct *vma) {}
> -
> -static inline struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
> -               unsigned long address)
> -{
> -       return NULL;
> -}
> -
> -static inline void vma_assert_locked(struct vm_area_struct *vma)
> -{
> -       mmap_assert_locked(vma->vm_mm);
> -}
> -
> -static inline void vma_assert_stabilised(struct vm_area_struct *vma)
> -{
> -       /* If no VMA locks, then either mmap lock suffices to stabilise. */
> -       mmap_assert_locked(vma->vm_mm);
> -}
> -
> -#endif /* CONFIG_PER_VMA_LOCK */

Now that I'm looking closer into this, I think we would break NOMMU
case because nommu.c does not take VMA write locks at all. So,
lock_vma_under_rcu() for example would always succeed.

Extra per_VMA lock-related fields in the vm_area_struct and mm_struct
would also inflate NOMMU structure sizes without them being used. I'm
not sure if this is an issue we should consider.

> -
>  static inline void vma_assert_can_modify(struct vm_area_struct *vma)
>  {
>         if (vma_is_attached(vma))
> diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
> index 41fe87d7302f..8848e26ef581 100644
> --- a/kernel/bpf/stackmap.c
> +++ b/kernel/bpf/stackmap.c
> @@ -272,13 +272,8 @@ struct stack_map_vma_lock {
>  /*
>   * Acquire a stable read-side reference on the VMA covering @ip.
>   *
> - * With CONFIG_PER_VMA_LOCK=y this returns a VMA with its per-VMA read
> - * lock held and mmap_lock dropped, so the caller may sleep.
> - *
> - * With CONFIG_PER_VMA_LOCK=n it returns a VMA with mmap_lock still
> - * held; the caller must snapshot any fields it needs and pin vm_file
> - * with get_file() before stack_map_unlock_vma() drops mmap_lock, as
> - * the VMA may be split, merged, or freed after that.
> + * This returns a VMA with its per-VMA read lock held and mmap_lock
> + * dropped, so the caller may sleep.
>   *
>   * Returns NULL on failure, in which case no lock is held.
>   */
> @@ -288,7 +283,6 @@ stack_map_lock_vma(struct stack_map_vma_lock *lock, unsigned long ip)
>         struct mm_struct *mm = lock->mm;
>         struct vm_area_struct *vma;
>
> -       /* noop under !CONFIG_PER_VMA_LOCK */
>         vma = lock_vma_under_rcu(mm, ip);
>         if (vma) {
>                 lock->vma = vma;
> @@ -308,13 +302,11 @@ stack_map_lock_vma(struct stack_map_vma_lock *lock, unsigned long ip)
>                 return NULL;
>         }
>
> -#ifdef CONFIG_PER_VMA_LOCK
>         if (!vma_start_read_locked(vma)) {
>                 mmap_read_unlock(mm);
>                 return NULL;
>         }
>         mmap_read_unlock(mm);
> -#endif
>
>         lock->vma = vma;
>         return vma;
> @@ -322,11 +314,7 @@ stack_map_lock_vma(struct stack_map_vma_lock *lock, unsigned long ip)
>
>  static void stack_map_unlock_vma(struct stack_map_vma_lock *lock)
>  {
> -#ifdef CONFIG_PER_VMA_LOCK
>         vma_end_read(lock->vma);
> -#else
> -       mmap_read_unlock(lock->mm);
> -#endif
>         lock->vma = NULL;
>  }
>
> diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c
> index e791ae065c39..6cf815bc84be 100644
> --- a/kernel/bpf/task_iter.c
> +++ b/kernel/bpf/task_iter.c
> @@ -835,11 +835,6 @@ __bpf_kfunc int bpf_iter_task_vma_new(struct bpf_iter_task_vma *it,
>         BUILD_BUG_ON(sizeof(struct bpf_iter_task_vma_kern) != sizeof(struct bpf_iter_task_vma));
>         BUILD_BUG_ON(__alignof__(struct bpf_iter_task_vma_kern) != __alignof__(struct bpf_iter_task_vma));
>
> -       if (!IS_ENABLED(CONFIG_PER_VMA_LOCK)) {
> -               kit->data = NULL;
> -               return -EOPNOTSUPP;
> -       }
> -
>         /*
>          * Reject irqs-disabled contexts including NMI. Operations used
>          * by _next() and _destroy() (vma_end_read, fput, bpf_iter_mmput_async)
> diff --git a/kernel/fork.c b/kernel/fork.c
> index f0e2e131a9a5..ff91f5f66c80 100644
> --- a/kernel/fork.c
> +++ b/kernel/fork.c
> @@ -1077,9 +1077,7 @@ static void mmap_init_lock(struct mm_struct *mm)
>  {
>         init_rwsem(&mm->mmap_lock);
>         mm_lock_seqcount_init(mm);
> -#ifdef CONFIG_PER_VMA_LOCK
>         rcuwait_init(&mm->vma_writer_wait);
> -#endif
>  }
>
>  static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p)
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 060190e12bce..77103b46b679 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -1425,19 +1425,6 @@ config LRU_GEN_STATS
>  config LRU_GEN_WALKS_MMU
>         def_bool y
>         depends on LRU_GEN && ARCH_HAS_HW_PTE_YOUNG
> -# }
> -
> -config ARCH_SUPPORTS_PER_VMA_LOCK
> -       def_bool n
> -
> -config PER_VMA_LOCK
> -       def_bool y
> -       depends on ARCH_SUPPORTS_PER_VMA_LOCK && MMU && SMP
> -       help
> -         Allow per-vma locking during page fault handling.
> -
> -         This feature allows locking each virtual memory area separately when
> -         handling page faults instead of taking mmap_lock.
>
>  config LOCK_MM_AND_FIND_VMA
>         bool
> diff --git a/mm/Kconfig.debug b/mm/Kconfig.debug
> index 5737a504efbb..1dd150edfe71 100644
> --- a/mm/Kconfig.debug
> +++ b/mm/Kconfig.debug
> @@ -310,7 +310,6 @@ config DEBUG_KMEMLEAK_VERBOSE
>
>  config PER_VMA_LOCK_STATS
>         bool "Statistics for per-vma locks"
> -       depends on PER_VMA_LOCK
>         help
>           Say Y here to enable success, retry and failure counters of page
>           faults handled under protection of per-vma locks. When enabled, the
> diff --git a/mm/debug.c b/mm/debug.c
> index 9a0297b3988d..655e6bcc0e8d 100644
> --- a/mm/debug.c
> +++ b/mm/debug.c
> @@ -157,17 +157,13 @@ void dump_vma(const struct vm_area_struct *vma)
>         pr_emerg("vma %px start %px end %px mm %px\n"
>                 "prot %lx anon_vma %px vm_ops %px\n"
>                 "pgoff %lx file %px private_data %px\n"
> -#ifdef CONFIG_PER_VMA_LOCK
>                 "refcnt %x\n"
> -#endif
>                 "flags: %#lx(%pGv)\n",
>                 vma, (void *)vma->vm_start, (void *)vma->vm_end, vma->vm_mm,
>                 (unsigned long)pgprot_val(vma->vm_page_prot),
>                 vma->anon_vma, vma->vm_ops, vma_start_pgoff(vma),
>                 vma->vm_file, vma->vm_private_data,
> -#ifdef CONFIG_PER_VMA_LOCK
>                 refcount_read(&vma->vm_refcnt),
> -#endif
>                 vma->vm_flags, &vma->vm_flags);
>  }
>  EXPORT_SYMBOL(dump_vma);
> diff --git a/mm/init-mm.c b/mm/init-mm.c
> index 3e792aad7626..a1bb2c2d0284 100644
> --- a/mm/init-mm.c
> +++ b/mm/init-mm.c
> @@ -39,10 +39,8 @@ struct mm_struct init_mm = {
>         .page_table_lock =  __SPIN_LOCK_UNLOCKED(init_mm.page_table_lock),
>         .arg_lock       =  __SPIN_LOCK_UNLOCKED(init_mm.arg_lock),
>         .mmlist         = LIST_HEAD_INIT(init_mm.mmlist),
> -#ifdef CONFIG_PER_VMA_LOCK
>         .vma_writer_wait = __RCUWAIT_INITIALIZER(init_mm.vma_writer_wait),
>         .mm_lock_seq    = SEQCNT_ZERO(init_mm.mm_lock_seq),
> -#endif
>  #ifdef CONFIG_SCHED_MM_CID
>         .mm_cid.lock = __RAW_SPIN_LOCK_UNLOCKED(init_mm.mm_cid.lock),
>  #endif
> diff --git a/mm/memory.c b/mm/memory.c
> index a620d425ec95..f109f6c87b28 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -6813,7 +6813,6 @@ static vm_fault_t sanitize_fault_flags(struct vm_area_struct *vma,
>                                  !is_cow_mapping(vma->vm_flags)))
>                         return VM_FAULT_SIGSEGV;
>         }
> -#ifdef CONFIG_PER_VMA_LOCK
>         /*
>          * Per-VMA locks can't be used with FAULT_FLAG_RETRY_NOWAIT because of
>          * the assumption that lock is dropped on VM_FAULT_RETRY.
> @@ -6822,7 +6821,6 @@ static vm_fault_t sanitize_fault_flags(struct vm_area_struct *vma,
>                         (FAULT_FLAG_VMA_LOCK | FAULT_FLAG_RETRY_NOWAIT)) ==
>                         (FAULT_FLAG_VMA_LOCK | FAULT_FLAG_RETRY_NOWAIT)))
>                 return VM_FAULT_SIGSEGV;
> -#endif
>
>         return 0;
>  }
> diff --git a/mm/mmap_lock.c b/mm/mmap_lock.c
> index 898c2ef1e958..e20d01e8d38f 100644
> --- a/mm/mmap_lock.c
> +++ b/mm/mmap_lock.c
> @@ -43,9 +43,6 @@ void __mmap_lock_do_trace_released(struct mm_struct *mm, bool write)
>  EXPORT_SYMBOL(__mmap_lock_do_trace_released);
>  #endif /* CONFIG_TRACING */
>
> -#ifdef CONFIG_MMU
> -#ifdef CONFIG_PER_VMA_LOCK
> -
>  /* State shared across __vma_[start, end]_exclude_readers. */
>  struct vma_exclude_readers_state {
>         /* Input parameters. */
> @@ -431,7 +428,6 @@ struct vm_area_struct *lock_next_vma(struct mm_struct *mm,
>
>         return vma;
>  }
> -#endif /* CONFIG_PER_VMA_LOCK */
>
>  #ifdef CONFIG_LOCK_MM_AND_FIND_VMA
>  #include <linux/extable.h>
> @@ -548,23 +544,3 @@ struct vm_area_struct *lock_mm_and_find_vma(struct mm_struct *mm,
>         return NULL;
>  }
>  #endif /* CONFIG_LOCK_MM_AND_FIND_VMA */
> -
> -#else /* CONFIG_MMU */
> -
> -/*
> - * At least xtensa ends up having protection faults even with no
> - * MMU.. No stack expansion, at least.
> - */
> -struct vm_area_struct *lock_mm_and_find_vma(struct mm_struct *mm,
> -                       unsigned long addr, struct pt_regs *regs)
> -{
> -       struct vm_area_struct *vma;
> -
> -       mmap_read_lock(mm);
> -       vma = vma_lookup(mm, addr);
> -       if (!vma)
> -               mmap_read_unlock(mm);
> -       return vma;
> -}
> -
> -#endif /* CONFIG_MMU */
> diff --git a/mm/pagewalk.c b/mm/pagewalk.c
> index ed4860c01936..fbcf64c59a97 100644
> --- a/mm/pagewalk.c
> +++ b/mm/pagewalk.c
> @@ -446,7 +446,6 @@ static inline void process_mm_walk_lock(struct mm_struct *mm,
>  static inline void process_vma_walk_lock(struct vm_area_struct *vma,
>                                          enum page_walk_lock walk_lock)
>  {
> -#ifdef CONFIG_PER_VMA_LOCK
>         switch (walk_lock) {
>         case PGWALK_WRLOCK:
>                 vma_start_write(vma);
> @@ -461,7 +460,6 @@ static inline void process_vma_walk_lock(struct vm_area_struct *vma,
>                 /* PGWALK_RDLOCK is handled by process_mm_walk_lock */
>                 break;
>         }
> -#endif
>  }
>
>  /*
> diff --git a/mm/rmap.c b/mm/rmap.c
> index b917431759ee..4e4a4b747977 100644
> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> @@ -260,11 +260,9 @@ static void check_anon_vma_clone(struct vm_area_struct *dst,
>         /* For the anon_vma to be compatible, it can only be singular. */
>         VM_WARN_ON_ONCE(operation == VMA_OP_MERGE_UNFAULTED &&
>                         !list_is_singular(&src->anon_vma_chain));
> -#ifdef CONFIG_PER_VMA_LOCK
>         /* Only merging an unfaulted VMA leaves the destination attached. */
>         VM_WARN_ON_ONCE(operation != VMA_OP_MERGE_UNFAULTED &&
>                         vma_is_attached(dst));
> -#endif
>  }
>
>  static void maybe_reuse_anon_vma(struct vm_area_struct *dst,
> diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
> index 258b03182a78..edd90892f8cc 100644
> --- a/mm/userfaultfd.c
> +++ b/mm/userfaultfd.c
> @@ -122,7 +122,6 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
>         return vma;
>  }
>
> -#ifdef CONFIG_PER_VMA_LOCK
>  /*
>   * uffd_lock_vma() - Lookup and lock vma corresponding to @address.
>   * @mm: mm to search vma in.
> @@ -182,34 +181,6 @@ static void uffd_mfill_unlock(struct vm_area_struct *vma)
>         vma_end_read(vma);
>  }
>
> -#else
> -
> -static struct vm_area_struct *uffd_mfill_lock(struct mm_struct *dst_mm,
> -                                             unsigned long dst_start,
> -                                             unsigned long len)
> -{
> -       struct vm_area_struct *dst_vma;
> -
> -       mmap_read_lock(dst_mm);
> -       dst_vma = find_vma_and_prepare_anon(dst_mm, dst_start);
> -       if (IS_ERR(dst_vma))
> -               goto out_unlock;
> -
> -       if (validate_dst_vma(dst_vma, dst_start + len))
> -               return dst_vma;
> -
> -       dst_vma = ERR_PTR(-ENOENT);
> -out_unlock:
> -       mmap_read_unlock(dst_mm);
> -       return dst_vma;
> -}
> -
> -static void uffd_mfill_unlock(struct vm_area_struct *vma)
> -{
> -       mmap_read_unlock(vma->vm_mm);
> -}
> -#endif
> -
>  static void mfill_put_vma(struct mfill_state *state)
>  {
>         if (!state->vma)
> @@ -1852,7 +1823,6 @@ int find_vmas_mm_locked(struct mm_struct *mm,
>         return 0;
>  }
>
> -#ifdef CONFIG_PER_VMA_LOCK
>  static int uffd_move_lock(struct mm_struct *mm,
>                           unsigned long dst_start,
>                           unsigned long src_start,
> @@ -1927,31 +1897,6 @@ static void uffd_move_unlock(struct vm_area_struct *dst_vma,
>                 vma_end_read(dst_vma);
>  }
>
> -#else
> -
> -static int uffd_move_lock(struct mm_struct *mm,
> -                         unsigned long dst_start,
> -                         unsigned long src_start,
> -                         struct vm_area_struct **dst_vmap,
> -                         struct vm_area_struct **src_vmap)
> -{
> -       int err;
> -
> -       mmap_read_lock(mm);
> -       err = find_vmas_mm_locked(mm, dst_start, src_start, dst_vmap, src_vmap);
> -       if (err)
> -               mmap_read_unlock(mm);
> -       return err;
> -}
> -
> -static void uffd_move_unlock(struct vm_area_struct *dst_vma,
> -                            struct vm_area_struct *src_vma)
> -{
> -       mmap_assert_locked(src_vma->vm_mm);
> -       mmap_read_unlock(dst_vma->vm_mm);
> -}
> -#endif
> -
>  /**
>   * move_pages - move arbitrary anonymous pages of an existing vma
>   * @ctx: pointer to the userfaultfd context
> diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs
> index 4764d7b68f2a..2633e704c83d 100644
> --- a/rust/kernel/mm.rs
> +++ b/rust/kernel/mm.rs
> @@ -174,26 +174,20 @@ pub unsafe fn from_raw<'a>(ptr: *const bindings::mm_struct) -> &'a MmWithUser {
>      /// When per-vma locks are disabled, this always returns `None`.
>      #[inline]
>      pub fn lock_vma_under_rcu(&self, vma_addr: usize) -> Option<VmaReadGuard<'_>> {
> -        #[cfg(CONFIG_PER_VMA_LOCK)]
>          {
>              // SAFETY: Calling `bindings::lock_vma_under_rcu` is always okay given an mm where
>              // `mm_users` is non-zero.
>              let vma = unsafe { bindings::lock_vma_under_rcu(self.as_raw(), vma_addr) };
> -            if !vma.is_null() {
> -                return Some(VmaReadGuard {
> -                    // SAFETY: If `lock_vma_under_rcu` returns a non-null ptr, then it points at a
> -                    // valid vma. The vma is stable for as long as the vma read lock is held.
> -                    vma: unsafe { VmaRef::from_raw(vma) },
> -                    _nts: NotThreadSafe,
> -                });
> +            if vma.is_null() {
> +                return None;
>              }
> +            Some(VmaReadGuard {
> +                // SAFETY: If `lock_vma_under_rcu` returns a non-null ptr, then it points at a
> +                // valid vma. The vma is stable for as long as the vma read lock is held.
> +                vma: unsafe { VmaRef::from_raw(vma) },
> +                _nts: NotThreadSafe,
> +            })
>          }
> -
> -        // Silence warnings about unused variables.
> -        #[cfg(not(CONFIG_PER_VMA_LOCK))]
> -        let _ = vma_addr;
> -
> -        None
>      }
>
>      /// Lock the mmap read lock.
> diff --git a/tools/testing/vma/include/dup.h b/tools/testing/vma/include/dup.h
> index 800e5fa02d78..362feda28526 100644
> --- a/tools/testing/vma/include/dup.h
> +++ b/tools/testing/vma/include/dup.h
> @@ -582,7 +582,6 @@ struct vm_area_struct {
>                 vma_flags_t flags;
>         };
>
> -#ifdef CONFIG_PER_VMA_LOCK
>         /*
>          * Can only be written (using WRITE_ONCE()) while holding both:
>          *  - mmap_lock (in write mode)
> @@ -598,7 +597,7 @@ struct vm_area_struct {
>          * slowpath.
>          */
>         unsigned int vm_lock_seq;
> -#endif
> +
>         unsigned int __vm_virt_pgoff_lo;
>
>         /*
> @@ -632,10 +631,8 @@ struct vm_area_struct {
>  #ifdef CONFIG_NUMA_BALANCING
>         struct vma_numab_state *numab_state;    /* NUMA Balancing state */
>  #endif
> -#ifdef CONFIG_PER_VMA_LOCK
>         /* Unstable RCU readers are allowed to read this. */
>         refcount_t vm_refcnt;
> -#endif
>  #ifdef CONFIG_64BIT
>         unsigned int __vm_virt_pgoff_hi;
>  #endif
> diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h
> index 8a48b231aa7a..54d5c3360aa2 100644
> --- a/tools/testing/vma/vma_internal.h
> +++ b/tools/testing/vma/vma_internal.h
> @@ -15,7 +15,6 @@
>  #include <stdlib.h>
>
>  #define CONFIG_MMU             1
> -#define CONFIG_PER_VMA_LOCK    1
>
>  #ifdef __CONCAT
>  #undef __CONCAT
> --
> 2.55.0.508.g3f0d502094-goog
>

Re: [PATCH v3 1/5] mm: Make per-VMA locks available universally

[Lorenzo Stoakes (ARM)]

On Mon, Aug 03, 2026 at 08:24:44AM -0700, Suren Baghdasaryan wrote:
> On Sun, Aug 2, 2026 at 2:55 PM Suren Baghdasaryan <surenb@google.com> wrote:
> >
> > From: Dave Hansen <dave.hansen@linux.intel.com>
> >
> > The per-VMA locks have been around for several years. They've had some
> > bugs worked out of them and have seen quite wide use. However, they
> > are still only available when architectures explicitly enable them.
> > Remove the conditional compilation around the per-VMA locks, making
> > them available on all architectures and configs.
> >
> > The approach up to now seemed to be to add ARCH_SUPPORTS_PER_VMA_LOCK
> > when the architecture started using per-VMA locks in the fault
> > handler. But, contrary to the naming, the Kconfig option does not
> > really indicate whether the architecture supports per-VMA locks or
> > not. It is more of a marker for whether the architecture is likely to
> > benefit from per-VMA locks.
> >
> > To me, the most important thing side-effect of universal availability
> > is letting per-VMA locks be used in SMP=n configs. This lets us use
> > per-VMA locking in all x86 code without fallbacks.
> >
> > Overall, this just generally makes the kernel simpler. Just look at
> > the diffstat. It also opens the door to users that want to use the
> > per-VMA locks in common code. Doing *that* brings additional
> > simplifications.
> >
> > The downside of this is adding some fields to vm_area_struct and
> > mm_struct. There are likely ways to optimize this, especially for
> > things like SMP=n configs. For now, do the simplest thing: use the
> > same implementation everywhere.
> >
> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > Cc: Suren Baghdasaryan <surenb@google.com>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> > Cc: Lorenzo Stoakes <ljs@kernel.org>
> > Cc: Vlastimil Babka <vbabka@kernel.org>
> > Cc: Shakeel Butt <shakeel.butt@linux.dev>
> > Cc: linux-mm@kvack.org
> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Cc: Arve Hjønnevåg <arve@android.com>
> > Cc: Todd Kjos <tkjos@android.com>
> > Cc: Christian Brauner <christian@brauner.io>
> > Cc: Carlos Llamas <cmllamas@google.com>
> > Cc: Alice Ryhl <aliceryhl@google.com>
> > Cc: "David S. Miller" <davem@davemloft.net>
> > Cc: David Ahern <dsahern@kernel.org>
> > Cc: netdev@vger.kernel.org
> > ---

<snip>

> > -#endif /* CONFIG_PER_VMA_LOCK */
>
> Now that I'm looking closer into this, I think we would break NOMMU
> case because nommu.c does not take VMA write locks at all. So,
> lock_vma_under_rcu() for example would always succeed.

I don't think anything's broken actually.

Per-VMA locks was gated on CONFIG_MMU so nothing there assumes per-VMA flags,
but now you have stuff that happens that didn't before but:

* vm_area_free() -> vma_assert_detached() - fine - it's always detached in nommu.
* vm_area_dup() -> vma_lock_init() - no asserts, just sets refcount to 0 (correct).

AFAICT nothing else.

So seems fine to me?

>
> Extra per_VMA lock-related fields in the vm_area_struct and mm_struct
> would also inflate NOMMU structure sizes without them being used. I'm
> not sure if this is an issue we should consider.

As nommu co-maintainer, no it's not :) I won't have that stuff blocking
important changes for real arches.

Go ahead! :)

--
Cheers, Lorenzo

Re: [PATCH v3 1/5] mm: Make per-VMA locks available universally

[Suren Baghdasaryan]

On Mon, Aug 3, 2026 at 9:08 AM Lorenzo Stoakes (ARM) <ljs@kernel.org> wrote:
>
> On Mon, Aug 03, 2026 at 08:24:44AM -0700, Suren Baghdasaryan wrote:
> > On Sun, Aug 2, 2026 at 2:55 PM Suren Baghdasaryan <surenb@google.com> wrote:
> > >
> > > From: Dave Hansen <dave.hansen@linux.intel.com>
> > >
> > > The per-VMA locks have been around for several years. They've had some
> > > bugs worked out of them and have seen quite wide use. However, they
> > > are still only available when architectures explicitly enable them.
> > > Remove the conditional compilation around the per-VMA locks, making
> > > them available on all architectures and configs.
> > >
> > > The approach up to now seemed to be to add ARCH_SUPPORTS_PER_VMA_LOCK
> > > when the architecture started using per-VMA locks in the fault
> > > handler. But, contrary to the naming, the Kconfig option does not
> > > really indicate whether the architecture supports per-VMA locks or
> > > not. It is more of a marker for whether the architecture is likely to
> > > benefit from per-VMA locks.
> > >
> > > To me, the most important thing side-effect of universal availability
> > > is letting per-VMA locks be used in SMP=n configs. This lets us use
> > > per-VMA locking in all x86 code without fallbacks.
> > >
> > > Overall, this just generally makes the kernel simpler. Just look at
> > > the diffstat. It also opens the door to users that want to use the
> > > per-VMA locks in common code. Doing *that* brings additional
> > > simplifications.
> > >
> > > The downside of this is adding some fields to vm_area_struct and
> > > mm_struct. There are likely ways to optimize this, especially for
> > > things like SMP=n configs. For now, do the simplest thing: use the
> > > same implementation everywhere.
> > >
> > > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > > Cc: Suren Baghdasaryan <surenb@google.com>
> > > Cc: Andrew Morton <akpm@linux-foundation.org>
> > > Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> > > Cc: Lorenzo Stoakes <ljs@kernel.org>
> > > Cc: Vlastimil Babka <vbabka@kernel.org>
> > > Cc: Shakeel Butt <shakeel.butt@linux.dev>
> > > Cc: linux-mm@kvack.org
> > > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > > Cc: Arve Hjønnevåg <arve@android.com>
> > > Cc: Todd Kjos <tkjos@android.com>
> > > Cc: Christian Brauner <christian@brauner.io>
> > > Cc: Carlos Llamas <cmllamas@google.com>
> > > Cc: Alice Ryhl <aliceryhl@google.com>
> > > Cc: "David S. Miller" <davem@davemloft.net>
> > > Cc: David Ahern <dsahern@kernel.org>
> > > Cc: netdev@vger.kernel.org
> > > ---
>
> <snip>
>
> > > -#endif /* CONFIG_PER_VMA_LOCK */
> >
> > Now that I'm looking closer into this, I think we would break NOMMU
> > case because nommu.c does not take VMA write locks at all. So,
> > lock_vma_under_rcu() for example would always succeed.
>
> I don't think anything's broken actually.
>
> Per-VMA locks was gated on CONFIG_MMU so nothing there assumes per-VMA flags,
> but now you have stuff that happens that didn't before but:
>
> * vm_area_free() -> vma_assert_detached() - fine - it's always detached in nommu.
> * vm_area_dup() -> vma_lock_init() - no asserts, just sets refcount to 0 (correct).
>
> AFAICT nothing else.
>
> So seems fine to me?

One place that needs fixing is in BPF iterators. They use
lock_vma_under_rcu() which would always succeeds even if the VMA is
being modified. Instead of removing !IS_ENABLED(CONFIG_PER_VMA_LOCK)
check in bpf_iter_task_vma_new() I'll need to change it to
!IS_ENABLED(CONFIG_MMU).
I'll look into other parts and try to verify they do not affect NOMMU
but ultimately I would like to run some tests with this config, which
is not trivial...

>
> >
> > Extra per_VMA lock-related fields in the vm_area_struct and mm_struct
> > would also inflate NOMMU structure sizes without them being used. I'm
> > not sure if this is an issue we should consider.
>
> As nommu co-maintainer, no it's not :) I won't have that stuff blocking
> important changes for real arches.
>
> Go ahead! :)
>
> --
> Cheers, Lorenzo

Re: [PATCH v3 1/5] mm: Make per-VMA locks available universally

[Suren Baghdasaryan]

On Mon, Aug 3, 2026 at 10:41 AM Suren Baghdasaryan <surenb@google.com> wrote:
>
> On Mon, Aug 3, 2026 at 9:08 AM Lorenzo Stoakes (ARM) <ljs@kernel.org> wrote:
> >
> > On Mon, Aug 03, 2026 at 08:24:44AM -0700, Suren Baghdasaryan wrote:
> > > On Sun, Aug 2, 2026 at 2:55 PM Suren Baghdasaryan <surenb@google.com> wrote:
> > > >
> > > > From: Dave Hansen <dave.hansen@linux.intel.com>
> > > >
> > > > The per-VMA locks have been around for several years. They've had some
> > > > bugs worked out of them and have seen quite wide use. However, they
> > > > are still only available when architectures explicitly enable them.
> > > > Remove the conditional compilation around the per-VMA locks, making
> > > > them available on all architectures and configs.
> > > >
> > > > The approach up to now seemed to be to add ARCH_SUPPORTS_PER_VMA_LOCK
> > > > when the architecture started using per-VMA locks in the fault
> > > > handler. But, contrary to the naming, the Kconfig option does not
> > > > really indicate whether the architecture supports per-VMA locks or
> > > > not. It is more of a marker for whether the architecture is likely to
> > > > benefit from per-VMA locks.
> > > >
> > > > To me, the most important thing side-effect of universal availability
> > > > is letting per-VMA locks be used in SMP=n configs. This lets us use
> > > > per-VMA locking in all x86 code without fallbacks.
> > > >
> > > > Overall, this just generally makes the kernel simpler. Just look at
> > > > the diffstat. It also opens the door to users that want to use the
> > > > per-VMA locks in common code. Doing *that* brings additional
> > > > simplifications.
> > > >
> > > > The downside of this is adding some fields to vm_area_struct and
> > > > mm_struct. There are likely ways to optimize this, especially for
> > > > things like SMP=n configs. For now, do the simplest thing: use the
> > > > same implementation everywhere.
> > > >
> > > > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > > > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > > > Cc: Suren Baghdasaryan <surenb@google.com>
> > > > Cc: Andrew Morton <akpm@linux-foundation.org>
> > > > Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> > > > Cc: Lorenzo Stoakes <ljs@kernel.org>
> > > > Cc: Vlastimil Babka <vbabka@kernel.org>
> > > > Cc: Shakeel Butt <shakeel.butt@linux.dev>
> > > > Cc: linux-mm@kvack.org
> > > > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > > > Cc: Arve Hjønnevåg <arve@android.com>
> > > > Cc: Todd Kjos <tkjos@android.com>
> > > > Cc: Christian Brauner <christian@brauner.io>
> > > > Cc: Carlos Llamas <cmllamas@google.com>
> > > > Cc: Alice Ryhl <aliceryhl@google.com>
> > > > Cc: "David S. Miller" <davem@davemloft.net>
> > > > Cc: David Ahern <dsahern@kernel.org>
> > > > Cc: netdev@vger.kernel.org
> > > > ---
> >
> > <snip>
> >
> > > > -#endif /* CONFIG_PER_VMA_LOCK */
> > >
> > > Now that I'm looking closer into this, I think we would break NOMMU
> > > case because nommu.c does not take VMA write locks at all. So,
> > > lock_vma_under_rcu() for example would always succeed.
> >
> > I don't think anything's broken actually.
> >
> > Per-VMA locks was gated on CONFIG_MMU so nothing there assumes per-VMA flags,
> > but now you have stuff that happens that didn't before but:
> >
> > * vm_area_free() -> vma_assert_detached() - fine - it's always detached in nommu.
> > * vm_area_dup() -> vma_lock_init() - no asserts, just sets refcount to 0 (correct).
> >
> > AFAICT nothing else.
> >
> > So seems fine to me?
>
> One place that needs fixing is in BPF iterators. They use
> lock_vma_under_rcu() which would always succeeds even if the VMA is
> being modified. Instead of removing !IS_ENABLED(CONFIG_PER_VMA_LOCK)
> check in bpf_iter_task_vma_new() I'll need to change it to
> !IS_ENABLED(CONFIG_MMU).
> I'll look into other parts and try to verify they do not affect NOMMU
> but ultimately I would like to run some tests with this config, which
> is not trivial...
>
> >
> > >
> > > Extra per_VMA lock-related fields in the vm_area_struct and mm_struct
> > > would also inflate NOMMU structure sizes without them being used. I'm
> > > not sure if this is an issue we should consider.
> >
> > As nommu co-maintainer, no it's not :) I won't have that stuff blocking
> > important changes for real arches.
> >
> > Go ahead! :)

Thanks for confirmation!

> >
> > --
> > Cheers, Lorenzo

Re: [PATCH v3 1/5] mm: Make per-VMA locks available universally

[Jann Horn]

On Mon, Aug 3, 2026 at 6:08 PM Lorenzo Stoakes (ARM) <ljs@kernel.org> wrote:
> On Mon, Aug 03, 2026 at 08:24:44AM -0700, Suren Baghdasaryan wrote:
> > On Sun, Aug 2, 2026 at 2:55 PM Suren Baghdasaryan <surenb@google.com> wrote:
> > > -#endif /* CONFIG_PER_VMA_LOCK */
> >
> > Now that I'm looking closer into this, I think we would break NOMMU
> > case because nommu.c does not take VMA write locks at all. So,
> > lock_vma_under_rcu() for example would always succeed.
>
> I don't think anything's broken actually.
>
> Per-VMA locks was gated on CONFIG_MMU so nothing there assumes per-VMA flags,
> but now you have stuff that happens that didn't before but:
>
> * vm_area_free() -> vma_assert_detached() - fine - it's always detached in nommu.

I don't think that's true, AFAICS vma_mark_attached() is called from
vma_iter_store_new() which is called from the nommu version of
do_mmap(). But I think you're right that one blunt workaround for this
would be to add a hack that prevents ever marking VMAs in nommu as
attached...

> * vm_area_dup() -> vma_lock_init() - no asserts, just sets refcount to 0 (correct).

Re: [PATCH v3 1/5] mm: Make per-VMA locks available universally

[Jann Horn]

On Sun, Aug 2, 2026 at 11:55 PM Suren Baghdasaryan <surenb@google.com> wrote:
> The per-VMA locks have been around for several years. They've had some
> bugs worked out of them and have seen quite wide use. However, they
> are still only available when architectures explicitly enable them.
> Remove the conditional compilation around the per-VMA locks, making
> them available on all architectures and configs.

Reviewed-by: Jann Horn <jannh@google.com>

I've grepped around in arch-specific code in the architectures that
weren't already opted in to look for stuff that takes the mmap lock in
write mode and manually mutates VMAs, and couldn't find anything, so I
think this is correct.

Re: [PATCH v3 1/5] mm: Make per-VMA locks available universally

[Suren Baghdasaryan]

On Mon, Aug 3, 2026 at 12:34 PM Jann Horn <jannh@google.com> wrote:
>
> On Sun, Aug 2, 2026 at 11:55 PM Suren Baghdasaryan <surenb@google.com> wrote:
> > The per-VMA locks have been around for several years. They've had some
> > bugs worked out of them and have seen quite wide use. However, they
> > are still only available when architectures explicitly enable them.
> > Remove the conditional compilation around the per-VMA locks, making
> > them available on all architectures and configs.
>
> Reviewed-by: Jann Horn <jannh@google.com>
>
> I've grepped around in arch-specific code in the architectures that
> weren't already opted in to look for stuff that takes the mmap lock in
> write mode and manually mutates VMAs, and couldn't find anything, so I
> think this is correct.

Thanks Jann!

[PATCH v3 2/5] binder: Make shrinker rely solely on per-VMA lock

[Suren Baghdasaryan]

From: Dave Hansen <dave.hansen@linux.intel.com>

tl;dr: lock_vma_under_rcu() is already a trylock. No need to do both
it and mmap_read_trylock().

Long Version:

== Background ==

Historically, binder used an mmap_read_trylock() in its shrinker code.
This ensures that reclaim is not blocked on an mmap_lock. Commit
95bc2d4a9020 ("binder: use per-vma lock in page reclaiming") added
support for the per-VMA lock, but left mmap_read_trylock() as a
fallback.

This was presumably because the per-VMA locking can fail for several
reasons and most (all?) lock_vma_under_rcu() callers have a fallback
to mmap_read_trylock().

== Problem ==

The fallback is not worth the complexity here. lock_vma_under_rcu() is
essentially already a non-blocking trylock. The main reason it fails
is also the reason mmap_read_trylock() fails: something is holding
mmap_write_lock().

The only remedy for a collision with mmap_write_lock() is to wait,
which this code can not do. So the "fallback" after
lock_vma_under_rcu() failure is not really a fallback: it is really
likely to just be retrying in vain. That retry in an of itself isn't
horrible. But it adds complexity.

== Solution ==

Now that per-VMA locks are universally available, lock_vma_under_rcu()
will not persistently fail. Rely on it alone and simplify the code.

Full disclosure: I originally tried to do this with
lock_vma_under_rcu_wait(), but it did not fit well with the mmap_lock
trylock semantics. Claude caught this in a review and suggested the
approach in this path. It seemed sane to me. So, Suggesed-by: Claude,
I guess.

Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: linux-mm@kvack.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Arve Hjønnevåg <arve@android.com>
Cc: Todd Kjos <tkjos@android.com>
Cc: Christian Brauner <christian@brauner.io>
Cc: Carlos Llamas <cmllamas@google.com>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: David Ahern <dsahern@kernel.org>
Cc: netdev@vger.kernel.org
---
 drivers/android/binder_alloc.c | 29 +++++++++++++++--------------
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/drivers/android/binder_alloc.c b/drivers/android/binder_alloc.c
index e4488ad86a65..84104ba04e30 100644
--- a/drivers/android/binder_alloc.c
+++ b/drivers/android/binder_alloc.c
@@ -1142,7 +1142,6 @@ enum lru_status binder_alloc_free_page(struct list_head *item,
 	struct vm_area_struct *vma;
 	struct page *page_to_free;
 	unsigned long page_addr;
-	int mm_locked = 0;
 	size_t index;
 
 	if (!mmget_not_zero(mm))
@@ -1151,14 +1150,20 @@ enum lru_status binder_alloc_free_page(struct list_head *item,
 	index = mdata->page_index;
 	page_addr = alloc->vm_start + index * PAGE_SIZE;
 
-	/* attempt per-vma lock first */
+	/*
+	 * Attempt per-vma lock. This is essentially a
+	 * "trylock". It can fail even if the VMA exists
+	 * for 'page_addr'.
+	 */
 	vma = lock_vma_under_rcu(mm, page_addr);
 	if (!vma) {
-		/* fall back to mmap_lock */
-		if (!mmap_read_trylock(mm))
-			goto err_mmap_read_lock_failed;
-		mm_locked = 1;
-		vma = vma_lookup(mm, page_addr);
+		/*
+		 * If the vma exists, we can't continue because we cannot
+		 * remove the page from the vma. However, if the vma was
+		 * unmapped, it's okay to continue.
+		 */
+		if (binder_alloc_is_mapped(alloc))
+			goto err_vma_lock_failed;
 	}
 
 	if (!mutex_trylock(&alloc->mutex))
@@ -1191,9 +1196,7 @@ enum lru_status binder_alloc_free_page(struct list_head *item,
 	}
 
 	mutex_unlock(&alloc->mutex);
-	if (mm_locked)
-		mmap_read_unlock(mm);
-	else
+	if (vma)
 		vma_end_read(vma);
 	mmput_async(mm);
 	binder_free_page(page_to_free);
@@ -1203,11 +1206,9 @@ enum lru_status binder_alloc_free_page(struct list_head *item,
 err_invalid_vma:
 	mutex_unlock(&alloc->mutex);
 err_get_alloc_mutex_failed:
-	if (mm_locked)
-		mmap_read_unlock(mm);
-	else
+	if (vma)
 		vma_end_read(vma);
-err_mmap_read_lock_failed:
+err_vma_lock_failed:
 	mmput_async(mm);
 err_mmget:
 	return LRU_SKIP;
-- 
2.55.0.508.g3f0d502094-goog

Re: [PATCH v3 2/5] binder: Make shrinker rely solely on per-VMA lock

[Alice Ryhl]

On Sun, Aug 02, 2026 at 02:54:56PM -0700, Suren Baghdasaryan wrote:
> From: Dave Hansen <dave.hansen@linux.intel.com>
> 
> tl;dr: lock_vma_under_rcu() is already a trylock. No need to do both
> it and mmap_read_trylock().
> 
> Long Version:
> 
> == Background ==
> 
> Historically, binder used an mmap_read_trylock() in its shrinker code.
> This ensures that reclaim is not blocked on an mmap_lock. Commit
> 95bc2d4a9020 ("binder: use per-vma lock in page reclaiming") added
> support for the per-VMA lock, but left mmap_read_trylock() as a
> fallback.
> 
> This was presumably because the per-VMA locking can fail for several
> reasons and most (all?) lock_vma_under_rcu() callers have a fallback
> to mmap_read_trylock().
> 
> == Problem ==
> 
> The fallback is not worth the complexity here. lock_vma_under_rcu() is
> essentially already a non-blocking trylock. The main reason it fails
> is also the reason mmap_read_trylock() fails: something is holding
> mmap_write_lock().
> 
> The only remedy for a collision with mmap_write_lock() is to wait,
> which this code can not do. So the "fallback" after
> lock_vma_under_rcu() failure is not really a fallback: it is really
> likely to just be retrying in vain. That retry in an of itself isn't
> horrible. But it adds complexity.
> 
> == Solution ==
> 
> Now that per-VMA locks are universally available, lock_vma_under_rcu()
> will not persistently fail. Rely on it alone and simplify the code.
> 
> Full disclosure: I originally tried to do this with
> lock_vma_under_rcu_wait(), but it did not fit well with the mmap_lock
> trylock semantics. Claude caught this in a review and suggested the
> approach in this path. It seemed sane to me. So, Suggesed-by: Claude,
> I guess.
> 
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Signed-off-by: Suren Baghdasaryan <surenb@google.com>

Reviewed-by: Alice Ryhl <aliceryhl@google.com>

Re: [PATCH v3 2/5] binder: Make shrinker rely solely on per-VMA lock

[Lorenzo Stoakes (ARM)]

On Mon, Aug 03, 2026 at 09:48:19AM +0000, Alice Ryhl wrote:
> On Sun, Aug 02, 2026 at 02:54:56PM -0700, Suren Baghdasaryan wrote:
> > From: Dave Hansen <dave.hansen@linux.intel.com>
> >
> > tl;dr: lock_vma_under_rcu() is already a trylock. No need to do both
> > it and mmap_read_trylock().
> >
> > Long Version:
> >
> > == Background ==
> >
> > Historically, binder used an mmap_read_trylock() in its shrinker code.
> > This ensures that reclaim is not blocked on an mmap_lock. Commit
> > 95bc2d4a9020 ("binder: use per-vma lock in page reclaiming") added
> > support for the per-VMA lock, but left mmap_read_trylock() as a
> > fallback.
> >
> > This was presumably because the per-VMA locking can fail for several
> > reasons and most (all?) lock_vma_under_rcu() callers have a fallback
> > to mmap_read_trylock().
> >
> > == Problem ==
> >
> > The fallback is not worth the complexity here. lock_vma_under_rcu() is
> > essentially already a non-blocking trylock. The main reason it fails
> > is also the reason mmap_read_trylock() fails: something is holding
> > mmap_write_lock().
> >
> > The only remedy for a collision with mmap_write_lock() is to wait,
> > which this code can not do. So the "fallback" after
> > lock_vma_under_rcu() failure is not really a fallback: it is really
> > likely to just be retrying in vain. That retry in an of itself isn't
> > horrible. But it adds complexity.
> >
> > == Solution ==
> >
> > Now that per-VMA locks are universally available, lock_vma_under_rcu()
> > will not persistently fail. Rely on it alone and simplify the code.
> >
> > Full disclosure: I originally tried to do this with
> > lock_vma_under_rcu_wait(), but it did not fit well with the mmap_lock
> > trylock semantics. Claude caught this in a review and suggested the
> > approach in this path. It seemed sane to me. So, Suggesed-by: Claude,
> > I guess.
> >
> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
>
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>

Is there anything on the rust side that needs to be changed also?

--
Cheers, Lorenzo

Re: [PATCH v3 2/5] binder: Make shrinker rely solely on per-VMA lock

[Lorenzo Stoakes (ARM)]

On Mon, Aug 03, 2026 at 11:50:41AM +0100, Lorenzo Stoakes (ARM) wrote:
> On Mon, Aug 03, 2026 at 09:48:19AM +0000, Alice Ryhl wrote:
> > On Sun, Aug 02, 2026 at 02:54:56PM -0700, Suren Baghdasaryan wrote:
> > > From: Dave Hansen <dave.hansen@linux.intel.com>
> > >
> > > tl;dr: lock_vma_under_rcu() is already a trylock. No need to do both
> > > it and mmap_read_trylock().
> > >
> > > Long Version:
> > >
> > > == Background ==
> > >
> > > Historically, binder used an mmap_read_trylock() in its shrinker code.
> > > This ensures that reclaim is not blocked on an mmap_lock. Commit
> > > 95bc2d4a9020 ("binder: use per-vma lock in page reclaiming") added
> > > support for the per-VMA lock, but left mmap_read_trylock() as a
> > > fallback.
> > >
> > > This was presumably because the per-VMA locking can fail for several
> > > reasons and most (all?) lock_vma_under_rcu() callers have a fallback
> > > to mmap_read_trylock().
> > >
> > > == Problem ==
> > >
> > > The fallback is not worth the complexity here. lock_vma_under_rcu() is
> > > essentially already a non-blocking trylock. The main reason it fails
> > > is also the reason mmap_read_trylock() fails: something is holding
> > > mmap_write_lock().
> > >
> > > The only remedy for a collision with mmap_write_lock() is to wait,
> > > which this code can not do. So the "fallback" after
> > > lock_vma_under_rcu() failure is not really a fallback: it is really
> > > likely to just be retrying in vain. That retry in an of itself isn't
> > > horrible. But it adds complexity.
> > >
> > > == Solution ==
> > >
> > > Now that per-VMA locks are universally available, lock_vma_under_rcu()
> > > will not persistently fail. Rely on it alone and simplify the code.
> > >
> > > Full disclosure: I originally tried to do this with
> > > lock_vma_under_rcu_wait(), but it did not fit well with the mmap_lock
> > > trylock semantics. Claude caught this in a review and suggested the
> > > approach in this path. It seemed sane to me. So, Suggesed-by: Claude,
> > > I guess.
> > >
> > > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> >
> > Reviewed-by: Alice Ryhl <aliceryhl@google.com>
>
> Is there anything on the rust side that needs to be changed also?

Ah ok addressed in 4/5, disregard :)

>
> --
> Cheers, Lorenzo

--
Cheers, Lorenzo

Re: [PATCH v3 2/5] binder: Make shrinker rely solely on per-VMA lock

[Lorenzo Stoakes (ARM)]

On Mon, Aug 03, 2026 at 12:11:42PM +0100, Lorenzo Stoakes (ARM) wrote:
> On Mon, Aug 03, 2026 at 11:50:41AM +0100, Lorenzo Stoakes (ARM) wrote:
> > On Mon, Aug 03, 2026 at 09:48:19AM +0000, Alice Ryhl wrote:
> > > On Sun, Aug 02, 2026 at 02:54:56PM -0700, Suren Baghdasaryan wrote:
> > > > From: Dave Hansen <dave.hansen@linux.intel.com>
> > > >
> > > > tl;dr: lock_vma_under_rcu() is already a trylock. No need to do both
> > > > it and mmap_read_trylock().
> > > >
> > > > Long Version:
> > > >
> > > > == Background ==
> > > >
> > > > Historically, binder used an mmap_read_trylock() in its shrinker code.
> > > > This ensures that reclaim is not blocked on an mmap_lock. Commit
> > > > 95bc2d4a9020 ("binder: use per-vma lock in page reclaiming") added
> > > > support for the per-VMA lock, but left mmap_read_trylock() as a
> > > > fallback.
> > > >
> > > > This was presumably because the per-VMA locking can fail for several
> > > > reasons and most (all?) lock_vma_under_rcu() callers have a fallback
> > > > to mmap_read_trylock().
> > > >
> > > > == Problem ==
> > > >
> > > > The fallback is not worth the complexity here. lock_vma_under_rcu() is
> > > > essentially already a non-blocking trylock. The main reason it fails
> > > > is also the reason mmap_read_trylock() fails: something is holding
> > > > mmap_write_lock().
> > > >
> > > > The only remedy for a collision with mmap_write_lock() is to wait,
> > > > which this code can not do. So the "fallback" after
> > > > lock_vma_under_rcu() failure is not really a fallback: it is really
> > > > likely to just be retrying in vain. That retry in an of itself isn't
> > > > horrible. But it adds complexity.
> > > >
> > > > == Solution ==
> > > >
> > > > Now that per-VMA locks are universally available, lock_vma_under_rcu()
> > > > will not persistently fail. Rely on it alone and simplify the code.
> > > >
> > > > Full disclosure: I originally tried to do this with
> > > > lock_vma_under_rcu_wait(), but it did not fit well with the mmap_lock
> > > > trylock semantics. Claude caught this in a review and suggested the
> > > > approach in this path. It seemed sane to me. So, Suggesed-by: Claude,
> > > > I guess.
> > > >
> > > > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > > > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > >
> > > Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> >
> > Is there anything on the rust side that needs to be changed also?
>
> Ah ok addressed in 4/5, disregard :)

OK disregard the disregard :)) that addresses something different. So question
remains :P

--
Cheers, Lorenzo

Re: [PATCH v3 2/5] binder: Make shrinker rely solely on per-VMA lock

[Suren Baghdasaryan]

On Mon, Aug 3, 2026 at 3:50 AM Lorenzo Stoakes (ARM) <ljs@kernel.org> wrote:
>
> On Mon, Aug 03, 2026 at 09:48:19AM +0000, Alice Ryhl wrote:
> > On Sun, Aug 02, 2026 at 02:54:56PM -0700, Suren Baghdasaryan wrote:
> > > From: Dave Hansen <dave.hansen@linux.intel.com>
> > >
> > > tl;dr: lock_vma_under_rcu() is already a trylock. No need to do both
> > > it and mmap_read_trylock().
> > >
> > > Long Version:
> > >
> > > == Background ==
> > >
> > > Historically, binder used an mmap_read_trylock() in its shrinker code.
> > > This ensures that reclaim is not blocked on an mmap_lock. Commit
> > > 95bc2d4a9020 ("binder: use per-vma lock in page reclaiming") added
> > > support for the per-VMA lock, but left mmap_read_trylock() as a
> > > fallback.
> > >
> > > This was presumably because the per-VMA locking can fail for several
> > > reasons and most (all?) lock_vma_under_rcu() callers have a fallback
> > > to mmap_read_trylock().
> > >
> > > == Problem ==
> > >
> > > The fallback is not worth the complexity here. lock_vma_under_rcu() is
> > > essentially already a non-blocking trylock. The main reason it fails
> > > is also the reason mmap_read_trylock() fails: something is holding
> > > mmap_write_lock().
> > >
> > > The only remedy for a collision with mmap_write_lock() is to wait,
> > > which this code can not do. So the "fallback" after
> > > lock_vma_under_rcu() failure is not really a fallback: it is really
> > > likely to just be retrying in vain. That retry in an of itself isn't
> > > horrible. But it adds complexity.
> > >
> > > == Solution ==
> > >
> > > Now that per-VMA locks are universally available, lock_vma_under_rcu()
> > > will not persistently fail. Rely on it alone and simplify the code.
> > >
> > > Full disclosure: I originally tried to do this with
> > > lock_vma_under_rcu_wait(), but it did not fit well with the mmap_lock
> > > trylock semantics. Claude caught this in a review and suggested the
> > > approach in this path. It seemed sane to me. So, Suggesed-by: Claude,
> > > I guess.
> > >
> > > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> >
> > Reviewed-by: Alice Ryhl <aliceryhl@google.com>
>
> Is there anything on the rust side that needs to be changed also?

No. I think the corresponding function is rust_shrink_free_page() and
it does not have mmap_fallback logic. IIUC, it just returns LRU_SKIP
on failure.

>
> --
> Cheers, Lorenzo

Re: [PATCH v3 2/5] binder: Make shrinker rely solely on per-VMA lock

[Lorenzo Stoakes (ARM)]

On Sun, Aug 02, 2026 at 02:54:56PM -0700, Suren Baghdasaryan wrote:
> From: Dave Hansen <dave.hansen@linux.intel.com>
>
> tl;dr: lock_vma_under_rcu() is already a trylock. No need to do both
> it and mmap_read_trylock().
>
> Long Version:
>
> == Background ==
>
> Historically, binder used an mmap_read_trylock() in its shrinker code.
> This ensures that reclaim is not blocked on an mmap_lock. Commit
> 95bc2d4a9020 ("binder: use per-vma lock in page reclaiming") added
> support for the per-VMA lock, but left mmap_read_trylock() as a
> fallback.
>
> This was presumably because the per-VMA locking can fail for several
> reasons and most (all?) lock_vma_under_rcu() callers have a fallback
> to mmap_read_trylock().
>
> == Problem ==
>
> The fallback is not worth the complexity here. lock_vma_under_rcu() is
> essentially already a non-blocking trylock. The main reason it fails
> is also the reason mmap_read_trylock() fails: something is holding
> mmap_write_lock().
>
> The only remedy for a collision with mmap_write_lock() is to wait,
> which this code can not do. So the "fallback" after
> lock_vma_under_rcu() failure is not really a fallback: it is really
> likely to just be retrying in vain. That retry in an of itself isn't
> horrible. But it adds complexity.
>
> == Solution ==
>
> Now that per-VMA locks are universally available, lock_vma_under_rcu()
> will not persistently fail. Rely on it alone and simplify the code.
>
> Full disclosure: I originally tried to do this with
> lock_vma_under_rcu_wait(), but it did not fit well with the mmap_lock
> trylock semantics. Claude caught this in a review and suggested the
> approach in this path. It seemed sane to me. So, Suggesed-by: Claude,
> I guess.
>
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> Cc: Vlastimil Babka <vbabka@kernel.org>
> Cc: Shakeel Butt <shakeel.butt@linux.dev>
> Cc: linux-mm@kvack.org
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Arve Hjønnevåg <arve@android.com>
> Cc: Todd Kjos <tkjos@android.com>
> Cc: Christian Brauner <christian@brauner.io>
> Cc: Carlos Llamas <cmllamas@google.com>
> Cc: Alice Ryhl <aliceryhl@google.com>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: David Ahern <dsahern@kernel.org>
> Cc: netdev@vger.kernel.org
> ---
>  drivers/android/binder_alloc.c | 29 +++++++++++++++--------------
>  1 file changed, 15 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/android/binder_alloc.c b/drivers/android/binder_alloc.c
> index e4488ad86a65..84104ba04e30 100644
> --- a/drivers/android/binder_alloc.c
> +++ b/drivers/android/binder_alloc.c
> @@ -1142,7 +1142,6 @@ enum lru_status binder_alloc_free_page(struct list_head *item,
>  	struct vm_area_struct *vma;
>  	struct page *page_to_free;
>  	unsigned long page_addr;
> -	int mm_locked = 0;
>  	size_t index;
>
>  	if (!mmget_not_zero(mm))
> @@ -1151,14 +1150,20 @@ enum lru_status binder_alloc_free_page(struct list_head *item,
>  	index = mdata->page_index;
>  	page_addr = alloc->vm_start + index * PAGE_SIZE;
>
> -	/* attempt per-vma lock first */
> +	/*
> +	 * Attempt per-vma lock. This is essentially a
> +	 * "trylock". It can fail even if the VMA exists
> +	 * for 'page_addr'.
> +	 */

This makes me wonder whether lock_vma_under_rcu() should really become
vma_trylock() at some point in time? :)

Or at least have 'trylock' in the name.

>  	vma = lock_vma_under_rcu(mm, page_addr);
>  	if (!vma) {
> -		/* fall back to mmap_lock */
> -		if (!mmap_read_trylock(mm))
> -			goto err_mmap_read_lock_failed;
> -		mm_locked = 1;
> -		vma = vma_lookup(mm, page_addr);
> +		/*
> +		 * If the vma exists, we can't continue because we cannot
> +		 * remove the page from the vma. However, if the vma was
> +		 * unmapped, it's okay to continue.
> +		 */
> +		if (binder_alloc_is_mapped(alloc))
> +			goto err_vma_lock_failed;

Hmm, it seems a bit odd to me that you also have:

	if (vma && !binder_alloc_is_mapped(alloc))
		goto err_invalid_vma;

Below?

So you have:

Before:

                        	|binder_alloc_is_mapped()?
				|yes   	no
			--------|-----------------
	vma is mapped?	yes	|OK	abort
			no	|OK	OK

Now:

                        	|binder_alloc_is_mapped()?
				|yes   	no
			--------|-----------------
	vma is mapped? maybe	|abort	OK
			yes	|OK	abort
			no	|OK	OK

The 'maybe' is because the VMA trylock failed.

So the issue is you might have a case where the VMA _is_ mapped but
!binder_alloc_is_mapped(), which previously aborted because of the vma &&
!binder_alloc_is_mapped() check.

It seems like:

	/*
	 * Since a binder_alloc can only be mapped once, we ensure
	 * the vma corresponds to this mapping by checking whether
	 * the binder_alloc is still mapped.
	 */
	if (vma && !binder_alloc_is_mapped(alloc))
		goto err_invalid_vma;

Is testing for a specific scenario 'we found a VMA but it turns out it's
invalid' and aborting if so.

So either this check should be removed or you should uncondtionally abort if
!vma I think?


>  	}
>
>  	if (!mutex_trylock(&alloc->mutex))
> @@ -1191,9 +1196,7 @@ enum lru_status binder_alloc_free_page(struct list_head *item,
>  	}
>
>  	mutex_unlock(&alloc->mutex);
> -	if (mm_locked)
> -		mmap_read_unlock(mm);
> -	else
> +	if (vma)
>  		vma_end_read(vma);
>  	mmput_async(mm);
>  	binder_free_page(page_to_free);
> @@ -1203,11 +1206,9 @@ enum lru_status binder_alloc_free_page(struct list_head *item,
>  err_invalid_vma:
>  	mutex_unlock(&alloc->mutex);
>  err_get_alloc_mutex_failed:
> -	if (mm_locked)
> -		mmap_read_unlock(mm);
> -	else
> +	if (vma)
>  		vma_end_read(vma);
> -err_mmap_read_lock_failed:
> +err_vma_lock_failed:
>  	mmput_async(mm);
>  err_mmget:
>  	return LRU_SKIP;
> --
> 2.55.0.508.g3f0d502094-goog
>

--
Cheers, Lorenzo

Re: [PATCH v3 2/5] binder: Make shrinker rely solely on per-VMA lock

[Suren Baghdasaryan]

On Mon, Aug 3, 2026 at 4:10 AM Lorenzo Stoakes (ARM) <ljs@kernel.org> wrote:
>
> On Sun, Aug 02, 2026 at 02:54:56PM -0700, Suren Baghdasaryan wrote:
> > From: Dave Hansen <dave.hansen@linux.intel.com>
> >
> > tl;dr: lock_vma_under_rcu() is already a trylock. No need to do both
> > it and mmap_read_trylock().
> >
> > Long Version:
> >
> > == Background ==
> >
> > Historically, binder used an mmap_read_trylock() in its shrinker code.
> > This ensures that reclaim is not blocked on an mmap_lock. Commit
> > 95bc2d4a9020 ("binder: use per-vma lock in page reclaiming") added
> > support for the per-VMA lock, but left mmap_read_trylock() as a
> > fallback.
> >
> > This was presumably because the per-VMA locking can fail for several
> > reasons and most (all?) lock_vma_under_rcu() callers have a fallback
> > to mmap_read_trylock().
> >
> > == Problem ==
> >
> > The fallback is not worth the complexity here. lock_vma_under_rcu() is
> > essentially already a non-blocking trylock. The main reason it fails
> > is also the reason mmap_read_trylock() fails: something is holding
> > mmap_write_lock().
> >
> > The only remedy for a collision with mmap_write_lock() is to wait,
> > which this code can not do. So the "fallback" after
> > lock_vma_under_rcu() failure is not really a fallback: it is really
> > likely to just be retrying in vain. That retry in an of itself isn't
> > horrible. But it adds complexity.
> >
> > == Solution ==
> >
> > Now that per-VMA locks are universally available, lock_vma_under_rcu()
> > will not persistently fail. Rely on it alone and simplify the code.
> >
> > Full disclosure: I originally tried to do this with
> > lock_vma_under_rcu_wait(), but it did not fit well with the mmap_lock
> > trylock semantics. Claude caught this in a review and suggested the
> > approach in this path. It seemed sane to me. So, Suggesed-by: Claude,
> > I guess.
> >
> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> > Cc: Vlastimil Babka <vbabka@kernel.org>
> > Cc: Shakeel Butt <shakeel.butt@linux.dev>
> > Cc: linux-mm@kvack.org
> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Cc: Arve Hjønnevåg <arve@android.com>
> > Cc: Todd Kjos <tkjos@android.com>
> > Cc: Christian Brauner <christian@brauner.io>
> > Cc: Carlos Llamas <cmllamas@google.com>
> > Cc: Alice Ryhl <aliceryhl@google.com>
> > Cc: "David S. Miller" <davem@davemloft.net>
> > Cc: David Ahern <dsahern@kernel.org>
> > Cc: netdev@vger.kernel.org
> > ---
> >  drivers/android/binder_alloc.c | 29 +++++++++++++++--------------
> >  1 file changed, 15 insertions(+), 14 deletions(-)
> >
> > diff --git a/drivers/android/binder_alloc.c b/drivers/android/binder_alloc.c
> > index e4488ad86a65..84104ba04e30 100644
> > --- a/drivers/android/binder_alloc.c
> > +++ b/drivers/android/binder_alloc.c
> > @@ -1142,7 +1142,6 @@ enum lru_status binder_alloc_free_page(struct list_head *item,
> >       struct vm_area_struct *vma;
> >       struct page *page_to_free;
> >       unsigned long page_addr;
> > -     int mm_locked = 0;
> >       size_t index;
> >
> >       if (!mmget_not_zero(mm))
> > @@ -1151,14 +1150,20 @@ enum lru_status binder_alloc_free_page(struct list_head *item,
> >       index = mdata->page_index;
> >       page_addr = alloc->vm_start + index * PAGE_SIZE;
> >
> > -     /* attempt per-vma lock first */
> > +     /*
> > +      * Attempt per-vma lock. This is essentially a
> > +      * "trylock". It can fail even if the VMA exists
> > +      * for 'page_addr'.
> > +      */
>
> This makes me wonder whether lock_vma_under_rcu() should really become
> vma_trylock() at some point in time? :)
>
> Or at least have 'trylock' in the name.

Makes sense. I think I'll postpone renames until after the series are
merged. Don't want to mix too many changes together.

>
> >       vma = lock_vma_under_rcu(mm, page_addr);
> >       if (!vma) {
> > -             /* fall back to mmap_lock */
> > -             if (!mmap_read_trylock(mm))
> > -                     goto err_mmap_read_lock_failed;
> > -             mm_locked = 1;
> > -             vma = vma_lookup(mm, page_addr);
> > +             /*
> > +              * If the vma exists, we can't continue because we cannot
> > +              * remove the page from the vma. However, if the vma was
> > +              * unmapped, it's okay to continue.
> > +              */
> > +             if (binder_alloc_is_mapped(alloc))
> > +                     goto err_vma_lock_failed;
>
> Hmm, it seems a bit odd to me that you also have:
>
>         if (vma && !binder_alloc_is_mapped(alloc))
>                 goto err_invalid_vma;
>
> Below?
>
> So you have:
>
> Before:
>
>                                 |binder_alloc_is_mapped()?
>                                 |yes    no
>                         --------|-----------------
>         vma is mapped?  yes     |OK     abort
>                         no      |OK     OK
>
> Now:
>
>                                 |binder_alloc_is_mapped()?
>                                 |yes    no
>                         --------|-----------------
>         vma is mapped? maybe    |abort  OK
>                         yes     |OK     abort
>                         no      |OK     OK
>
> The 'maybe' is because the VMA trylock failed.

The case you are considering is lock_vma_under_rcu() failed for some
reason other than lock contention (say seqno overflow). In that case
we get vma==NULL and binder_alloc_is_mapped() is called without any
lock (VMA or mmap lock) being held. I'm not sure if this is a real
problem since I see other places calling binder_alloc_is_mapped()
without locking. Alice, Carlos, is this a problem?

>
> So the issue is you might have a case where the VMA _is_ mapped but
> !binder_alloc_is_mapped(), which previously aborted because of the vma &&
> !binder_alloc_is_mapped() check.
>
> It seems like:
>
>         /*
>          * Since a binder_alloc can only be mapped once, we ensure
>          * the vma corresponds to this mapping by checking whether
>          * the binder_alloc is still mapped.
>          */
>         if (vma && !binder_alloc_is_mapped(alloc))
>                 goto err_invalid_vma;
>
> Is testing for a specific scenario 'we found a VMA but it turns out it's
> invalid' and aborting if so.
>
> So either this check should be removed or you should uncondtionally abort if
> !vma I think?

I think this check is fine because it basically checks
binder_alloc_is_mapped() after stabilizing the address range.
Unconditionally aborting if !vma would prevent us from freeing the
page if the VMA was already unmapped. The ultimate question is whether
we can rely on binder_alloc_is_mapped() alone when freeing that page.
IOW, VMA might still be in the VMA tree but
binder_alloc_is_mapped()==false, can we free the page?

>
>
> >       }
> >
> >       if (!mutex_trylock(&alloc->mutex))
> > @@ -1191,9 +1196,7 @@ enum lru_status binder_alloc_free_page(struct list_head *item,
> >       }
> >
> >       mutex_unlock(&alloc->mutex);
> > -     if (mm_locked)
> > -             mmap_read_unlock(mm);
> > -     else
> > +     if (vma)
> >               vma_end_read(vma);
> >       mmput_async(mm);
> >       binder_free_page(page_to_free);
> > @@ -1203,11 +1206,9 @@ enum lru_status binder_alloc_free_page(struct list_head *item,
> >  err_invalid_vma:
> >       mutex_unlock(&alloc->mutex);
> >  err_get_alloc_mutex_failed:
> > -     if (mm_locked)
> > -             mmap_read_unlock(mm);
> > -     else
> > +     if (vma)
> >               vma_end_read(vma);
> > -err_mmap_read_lock_failed:
> > +err_vma_lock_failed:
> >       mmput_async(mm);
> >  err_mmget:
> >       return LRU_SKIP;
> > --
> > 2.55.0.508.g3f0d502094-goog
> >
>
> --
> Cheers, Lorenzo

[PATCH v3 3/5] mm: Add RCU-based VMA lookup helper that waits for writers

[Suren Baghdasaryan]

From: Dave Hansen <dave.hansen@linux.intel.com>

== Background ==

There are basically two parallel ways to look up a VMA: the
traditional way, which is protected by mmap_read_lock, and the RCU-based
per-VMA lock way which is based on RCU and refcounts.

== Problem ==

The mmap_lock one is more straightforward to use but it has a big
disadvantage in that it can not be mixed with page faults since those
can take mmap_lock for read, which can deadlock when mixed with nested
page faults and parallel writers.
For example:

	mmap_read_lock(mm);
	// Another thread does mmap_write_lock().
	// New mmap_lock readers are blocked.
	vma = vma_lookup(mm, address);
	// This deadlocks on mmap_read_lock() if it faults:
	copy_from_user(address);
	mmap_read_unlock(mm);

The per-VMA lock can be mixed with faults, but they can fail and need to
be able to fall back to the traditional way.

== Solution ==

Add a variant of the RCU-based lookup that waits for writers. This is
basically the same as the existing RCU-based lookup, but on a failure to
lock it temporarily takes mmap_lock for read and waits for writers
to finish before locking the VMA, dropping the mmap_lock and returning
the locked VMA. This has some advantages:

 1. Callers do not need to have a fallback path for when they
    collide with writers.
 2. It can be used in contexts where page faults can happen because
    it can take the mmap_lock for read but never *holds* it.
 3. Its fast path does not require taking mmap_lock for read.

Basically, when applied correctly, this approach results in faster
*and* simpler code.

Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: linux-mm@kvack.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Arve Hjønnevåg <arve@android.com>
Cc: Todd Kjos <tkjos@android.com>
Cc: Christian Brauner <christian@brauner.io>
Cc: Carlos Llamas <cmllamas@google.com>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: David Ahern <dsahern@kernel.org>
Cc: netdev@vger.kernel.org
---
 include/linux/mmap_lock.h | 15 +++++++++++----
 mm/mmap_lock.c            | 29 +++++++++++++++++++++++++++++
 mm/userfaultfd.c          |  6 ++++--
 3 files changed, 44 insertions(+), 6 deletions(-)

diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
index eb32b482434e..fdd8f5cf5722 100644
--- a/include/linux/mmap_lock.h
+++ b/include/linux/mmap_lock.h
@@ -228,10 +228,12 @@ static inline void vma_refcount_put(struct vm_area_struct *vma)
 }
 
 /*
- * Use only while holding mmap read lock which guarantees that locking will not
- * fail (nobody can concurrently write-lock the vma). vma_start_read() should
+ * Use only while holding mmap read lock which guarantees that vma lock is not
+ * contended (nobody can concurrently write-lock the vma). vma_start_read() should
  * not be used in such cases because it might fail due to mm_lock_seq overflow.
  * This functionality is used to obtain vma read lock and drop the mmap read lock.
+ * VMA can't be detached while we are holding mmap lock, therefore in practice this
+ * function can fail only when there are so many readers that vm_refcnt overflows.
  */
 static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int subclass)
 {
@@ -247,16 +249,21 @@ static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int
 }
 
 /*
- * Use only while holding mmap read lock which guarantees that locking will not
- * fail (nobody can concurrently write-lock the vma). vma_start_read() should
+ * Use only while holding mmap read lock which guarantees that vma lock is not
+ * contended (nobody can concurrently write-lock the vma). vma_start_read() should
  * not be used in such cases because it might fail due to mm_lock_seq overflow.
  * This functionality is used to obtain vma read lock and drop the mmap read lock.
+ * VMA can't be detached while we are holding mmap lock, therefore in practice this
+ * function can fail only when there are so many readers that vm_refcnt overflows.
  */
 static inline bool vma_start_read_locked(struct vm_area_struct *vma)
 {
 	return vma_start_read_locked_nested(vma, 0);
 }
 
+struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
+					       unsigned long address);
+
 static inline void vma_end_read(struct vm_area_struct *vma)
 {
 	vma_refcount_put(vma);
diff --git a/mm/mmap_lock.c b/mm/mmap_lock.c
index e20d01e8d38f..6ff05e68e61b 100644
--- a/mm/mmap_lock.c
+++ b/mm/mmap_lock.c
@@ -338,6 +338,35 @@ struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
 	return NULL;
 }
 
+/*
+ * Find the VMA covering 'address' and lock it for reading. Waits for writers to
+ * finish if the VMA is being modified. Returns NULL if there is no VMA covering
+ * 'address'.
+ *
+ * Use only in code paths where no mmap_lock and no VMA lock is held.
+ *
+ * The fast path does not take mmap_lock.
+ */
+struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
+					       unsigned long address)
+{
+	struct vm_area_struct *vma;
+
+	/* Fast path: return stable VMA covering 'address': */
+	vma = lock_vma_under_rcu(mm, address);
+	if (vma)
+		return vma;
+
+	/* Slow path: preclude VMA writers by temporarily getting mmap read lock. */
+	mmap_read_lock(mm);
+	vma = vma_lookup(mm, address);
+	if (vma && !vma_start_read_locked(vma))
+		vma = NULL;
+	mmap_read_unlock(mm);
+
+	return vma;
+}
+
 static struct vm_area_struct *lock_next_vma_under_mmap_lock(struct mm_struct *mm,
 							    struct vma_iterator *vmi,
 							    unsigned long from_addr)
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index edd90892f8cc..c3a0c38a3dc3 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -129,8 +129,10 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
  *
  * Should be called without holding mmap_lock.
  *
- * Return: A locked vma containing @address, -ENOENT if no vma is found, or
- * -ENOMEM if anon_vma couldn't be allocated.
+ * Return: A locked vma containing @address, -ENOENT if no vma is found,
+ * -ENOMEM if anon_vma couldn't be allocated, or -EAGAIN if vma refcount
+ * overflow happened due to high number of readers and the caller should
+ * retry later.
  */
 static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,
 				       unsigned long address)
-- 
2.55.0.508.g3f0d502094-goog

Re: [PATCH v3 3/5] mm: Add RCU-based VMA lookup helper that waits for writers

[Lorenzo Stoakes (ARM)]

On Sun, Aug 02, 2026 at 02:54:57PM -0700, Suren Baghdasaryan wrote:
> From: Dave Hansen <dave.hansen@linux.intel.com>
>
> == Background ==
>
> There are basically two parallel ways to look up a VMA: the
> traditional way, which is protected by mmap_read_lock, and the RCU-based
> per-VMA lock way which is based on RCU and refcounts.
>
> == Problem ==
>
> The mmap_lock one is more straightforward to use but it has a big
> disadvantage in that it can not be mixed with page faults since those
> can take mmap_lock for read, which can deadlock when mixed with nested
> page faults and parallel writers.
> For example:
>
> 	mmap_read_lock(mm);
> 	// Another thread does mmap_write_lock().
> 	// New mmap_lock readers are blocked.
> 	vma = vma_lookup(mm, address);
> 	// This deadlocks on mmap_read_lock() if it faults:
> 	copy_from_user(address);
> 	mmap_read_unlock(mm);
>
> The per-VMA lock can be mixed with faults, but they can fail and need to
> be able to fall back to the traditional way.
>
> == Solution ==
>
> Add a variant of the RCU-based lookup that waits for writers. This is
> basically the same as the existing RCU-based lookup, but on a failure to
> lock it temporarily takes mmap_lock for read and waits for writers
> to finish before locking the VMA, dropping the mmap_lock and returning
> the locked VMA. This has some advantages:
>
>  1. Callers do not need to have a fallback path for when they
>     collide with writers.
>  2. It can be used in contexts where page faults can happen because
>     it can take the mmap_lock for read but never *holds* it.
>  3. Its fast path does not require taking mmap_lock for read.
>
> Basically, when applied correctly, this approach results in faster
> *and* simpler code.
>

Would be nice to have a:

Suggested-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>

Here given https://lore.kernel.org/linux-mm/af4Zx0gJIWbdDeY2@lucifer/ :)

> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> Cc: Suren Baghdasaryan <surenb@google.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> Cc: Lorenzo Stoakes <ljs@kernel.org>
> Cc: Vlastimil Babka <vbabka@kernel.org>
> Cc: Shakeel Butt <shakeel.butt@linux.dev>
> Cc: linux-mm@kvack.org
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Arve Hjønnevåg <arve@android.com>
> Cc: Todd Kjos <tkjos@android.com>
> Cc: Christian Brauner <christian@brauner.io>
> Cc: Carlos Llamas <cmllamas@google.com>
> Cc: Alice Ryhl <aliceryhl@google.com>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: David Ahern <dsahern@kernel.org>
> Cc: netdev@vger.kernel.org
> ---
>  include/linux/mmap_lock.h | 15 +++++++++++----
>  mm/mmap_lock.c            | 29 +++++++++++++++++++++++++++++
>  mm/userfaultfd.c          |  6 ++++--
>  3 files changed, 44 insertions(+), 6 deletions(-)
>
> diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
> index eb32b482434e..fdd8f5cf5722 100644
> --- a/include/linux/mmap_lock.h
> +++ b/include/linux/mmap_lock.h
> @@ -228,10 +228,12 @@ static inline void vma_refcount_put(struct vm_area_struct *vma)
>  }
>
>  /*
> - * Use only while holding mmap read lock which guarantees that locking will not
> - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
> + * Use only while holding mmap read lock which guarantees that vma lock is not
> + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
>   * not be used in such cases because it might fail due to mm_lock_seq overflow.
>   * This functionality is used to obtain vma read lock and drop the mmap read lock.
> + * VMA can't be detached while we are holding mmap lock, therefore in practice this
> + * function can fail only when there are so many readers that vm_refcnt overflows.
>   */
>  static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int subclass)
>  {
> @@ -247,16 +249,21 @@ static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int
>  }
>
>  /*
> - * Use only while holding mmap read lock which guarantees that locking will not
> - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
> + * Use only while holding mmap read lock which guarantees that vma lock is not
> + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
>   * not be used in such cases because it might fail due to mm_lock_seq overflow.
>   * This functionality is used to obtain vma read lock and drop the mmap read lock.
> + * VMA can't be detached while we are holding mmap lock, therefore in practice this
> + * function can fail only when there are so many readers that vm_refcnt overflows.
>   */
>  static inline bool vma_start_read_locked(struct vm_area_struct *vma)
>  {
>  	return vma_start_read_locked_nested(vma, 0);
>  }
>
> +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
> +					       unsigned long address);
> +
>  static inline void vma_end_read(struct vm_area_struct *vma)
>  {
>  	vma_refcount_put(vma);
> diff --git a/mm/mmap_lock.c b/mm/mmap_lock.c
> index e20d01e8d38f..6ff05e68e61b 100644
> --- a/mm/mmap_lock.c
> +++ b/mm/mmap_lock.c
> @@ -338,6 +338,35 @@ struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
>  	return NULL;
>  }
>
> +/*

Why not a kdoc comment?

> + * Find the VMA covering 'address' and lock it for reading. Waits for writers to
> + * finish if the VMA is being modified. Returns NULL if there is no VMA covering
> + * 'address'.
> + *
> + * Use only in code paths where no mmap_lock and no VMA lock is held.

Well, a VMA read lock can be held which would make this a noop essentially.

If a VMA write lock is held you're also ok as the mmap read lock will preclude
an mmap write lock, meaning vma_end_write_all() will have been called and the
write lock released.

So I think you can just drop this line?

> + *
> + * The fast path does not take mmap_lock.
> + */
> +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
> +					       unsigned long address)
> +{
> +	struct vm_area_struct *vma;
> +
> +	/* Fast path: return stable VMA covering 'address': */
> +	vma = lock_vma_under_rcu(mm, address);
> +	if (vma)
> +		return vma;
> +
> +	/* Slow path: preclude VMA writers by temporarily getting mmap read lock. */
> +	mmap_read_lock(mm);
> +	vma = vma_lookup(mm, address);
> +	if (vma && !vma_start_read_locked(vma))

This maybe warrants an unlikely() given it can only happen if refcount
overflows? Also worth having a comment to that effect here?

> +		vma = NULL;
> +	mmap_read_unlock(mm);
> +
> +	return vma;
> +}
> +
>  static struct vm_area_struct *lock_next_vma_under_mmap_lock(struct mm_struct *mm,
>  							    struct vma_iterator *vmi,
>  							    unsigned long from_addr)
> diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
> index edd90892f8cc..c3a0c38a3dc3 100644
> --- a/mm/userfaultfd.c
> +++ b/mm/userfaultfd.c
> @@ -129,8 +129,10 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
>   *
>   * Should be called without holding mmap_lock.
>   *
> - * Return: A locked vma containing @address, -ENOENT if no vma is found, or
> - * -ENOMEM if anon_vma couldn't be allocated.
> + * Return: A locked vma containing @address, -ENOENT if no vma is found,
> + * -ENOMEM if anon_vma couldn't be allocated, or -EAGAIN if vma refcount
> + * overflow happened due to high number of readers and the caller should
> + * retry later.

I'm guessing you're fixing this up as part of the patch? But it feels a bit
random, I mean fine but you should mention this change in the commit message +
explain why.

>   */
>  static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,
>  				       unsigned long address)
> --
> 2.55.0.508.g3f0d502094-goog
>

--
Cheers, Lorenzo

Re: [PATCH v3 3/5] mm: Add RCU-based VMA lookup helper that waits for writers

[Suren Baghdasaryan]

On Mon, Aug 3, 2026 at 4:28 AM Lorenzo Stoakes (ARM) <ljs@kernel.org> wrote:
>
> On Sun, Aug 02, 2026 at 02:54:57PM -0700, Suren Baghdasaryan wrote:
> > From: Dave Hansen <dave.hansen@linux.intel.com>
> >
> > == Background ==
> >
> > There are basically two parallel ways to look up a VMA: the
> > traditional way, which is protected by mmap_read_lock, and the RCU-based
> > per-VMA lock way which is based on RCU and refcounts.
> >
> > == Problem ==
> >
> > The mmap_lock one is more straightforward to use but it has a big
> > disadvantage in that it can not be mixed with page faults since those
> > can take mmap_lock for read, which can deadlock when mixed with nested
> > page faults and parallel writers.
> > For example:
> >
> >       mmap_read_lock(mm);
> >       // Another thread does mmap_write_lock().
> >       // New mmap_lock readers are blocked.
> >       vma = vma_lookup(mm, address);
> >       // This deadlocks on mmap_read_lock() if it faults:
> >       copy_from_user(address);
> >       mmap_read_unlock(mm);
> >
> > The per-VMA lock can be mixed with faults, but they can fail and need to
> > be able to fall back to the traditional way.
> >
> > == Solution ==
> >
> > Add a variant of the RCU-based lookup that waits for writers. This is
> > basically the same as the existing RCU-based lookup, but on a failure to
> > lock it temporarily takes mmap_lock for read and waits for writers
> > to finish before locking the VMA, dropping the mmap_lock and returning
> > the locked VMA. This has some advantages:
> >
> >  1. Callers do not need to have a fallback path for when they
> >     collide with writers.
> >  2. It can be used in contexts where page faults can happen because
> >     it can take the mmap_lock for read but never *holds* it.
> >  3. Its fast path does not require taking mmap_lock for read.
> >
> > Basically, when applied correctly, this approach results in faster
> > *and* simpler code.
> >
>
> Would be nice to have a:
>
> Suggested-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>

Ack.

>
> Here given https://lore.kernel.org/linux-mm/af4Zx0gJIWbdDeY2@lucifer/ :)
>
> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > Cc: Suren Baghdasaryan <surenb@google.com>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> > Cc: Lorenzo Stoakes <ljs@kernel.org>
> > Cc: Vlastimil Babka <vbabka@kernel.org>
> > Cc: Shakeel Butt <shakeel.butt@linux.dev>
> > Cc: linux-mm@kvack.org
> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Cc: Arve Hjønnevåg <arve@android.com>
> > Cc: Todd Kjos <tkjos@android.com>
> > Cc: Christian Brauner <christian@brauner.io>
> > Cc: Carlos Llamas <cmllamas@google.com>
> > Cc: Alice Ryhl <aliceryhl@google.com>
> > Cc: "David S. Miller" <davem@davemloft.net>
> > Cc: David Ahern <dsahern@kernel.org>
> > Cc: netdev@vger.kernel.org
> > ---
> >  include/linux/mmap_lock.h | 15 +++++++++++----
> >  mm/mmap_lock.c            | 29 +++++++++++++++++++++++++++++
> >  mm/userfaultfd.c          |  6 ++++--
> >  3 files changed, 44 insertions(+), 6 deletions(-)
> >
> > diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
> > index eb32b482434e..fdd8f5cf5722 100644
> > --- a/include/linux/mmap_lock.h
> > +++ b/include/linux/mmap_lock.h
> > @@ -228,10 +228,12 @@ static inline void vma_refcount_put(struct vm_area_struct *vma)
> >  }
> >
> >  /*
> > - * Use only while holding mmap read lock which guarantees that locking will not
> > - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
> > + * Use only while holding mmap read lock which guarantees that vma lock is not
> > + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
> >   * not be used in such cases because it might fail due to mm_lock_seq overflow.
> >   * This functionality is used to obtain vma read lock and drop the mmap read lock.
> > + * VMA can't be detached while we are holding mmap lock, therefore in practice this
> > + * function can fail only when there are so many readers that vm_refcnt overflows.
> >   */
> >  static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int subclass)
> >  {
> > @@ -247,16 +249,21 @@ static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int
> >  }
> >
> >  /*
> > - * Use only while holding mmap read lock which guarantees that locking will not
> > - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
> > + * Use only while holding mmap read lock which guarantees that vma lock is not
> > + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
> >   * not be used in such cases because it might fail due to mm_lock_seq overflow.
> >   * This functionality is used to obtain vma read lock and drop the mmap read lock.
> > + * VMA can't be detached while we are holding mmap lock, therefore in practice this
> > + * function can fail only when there are so many readers that vm_refcnt overflows.
> >   */
> >  static inline bool vma_start_read_locked(struct vm_area_struct *vma)
> >  {
> >       return vma_start_read_locked_nested(vma, 0);
> >  }
> >
> > +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
> > +                                            unsigned long address);
> > +
> >  static inline void vma_end_read(struct vm_area_struct *vma)
> >  {
> >       vma_refcount_put(vma);
> > diff --git a/mm/mmap_lock.c b/mm/mmap_lock.c
> > index e20d01e8d38f..6ff05e68e61b 100644
> > --- a/mm/mmap_lock.c
> > +++ b/mm/mmap_lock.c
> > @@ -338,6 +338,35 @@ struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
> >       return NULL;
> >  }
> >
> > +/*
>
> Why not a kdoc comment?

Indeed. Will change.

>
> > + * Find the VMA covering 'address' and lock it for reading. Waits for writers to
> > + * finish if the VMA is being modified. Returns NULL if there is no VMA covering
> > + * 'address'.
> > + *
> > + * Use only in code paths where no mmap_lock and no VMA lock is held.
>
> Well, a VMA read lock can be held which would make this a noop essentially.
>
> If a VMA write lock is held you're also ok as the mmap read lock will preclude
> an mmap write lock, meaning vma_end_write_all() will have been called and the
> write lock released.
>
> So I think you can just drop this line?

Maybe instead of this I should say: Use when mmap_lock is not held,
otherwise use vma_start_read_locked() ? I think that was the original
reason for this comment.

>
> > + *
> > + * The fast path does not take mmap_lock.
> > + */
> > +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
> > +                                            unsigned long address)
> > +{
> > +     struct vm_area_struct *vma;
> > +
> > +     /* Fast path: return stable VMA covering 'address': */
> > +     vma = lock_vma_under_rcu(mm, address);
> > +     if (vma)
> > +             return vma;
> > +
> > +     /* Slow path: preclude VMA writers by temporarily getting mmap read lock. */
> > +     mmap_read_lock(mm);
> > +     vma = vma_lookup(mm, address);
> > +     if (vma && !vma_start_read_locked(vma))
>
> This maybe warrants an unlikely() given it can only happen if refcount
> overflows? Also worth having a comment to that effect here?

vma_start_read_locked()
  vma_start_read_locked()
    vma_start_read_locked_nested()
      unlikely(!__refcount_inc_not_zero_limited_acquire())

already contains "unlikely" clause. Do we need to add it in all its users?
I can modify the comment for vma_start_read_locked() stating that
refcount overflow is very unlikely. Would that work or do you want
callers to have that too?

>
> > +             vma = NULL;
> > +     mmap_read_unlock(mm);
> > +
> > +     return vma;
> > +}
> > +
> >  static struct vm_area_struct *lock_next_vma_under_mmap_lock(struct mm_struct *mm,
> >                                                           struct vma_iterator *vmi,
> >                                                           unsigned long from_addr)
> > diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
> > index edd90892f8cc..c3a0c38a3dc3 100644
> > --- a/mm/userfaultfd.c
> > +++ b/mm/userfaultfd.c
> > @@ -129,8 +129,10 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
> >   *
> >   * Should be called without holding mmap_lock.
> >   *
> > - * Return: A locked vma containing @address, -ENOENT if no vma is found, or
> > - * -ENOMEM if anon_vma couldn't be allocated.
> > + * Return: A locked vma containing @address, -ENOENT if no vma is found,
> > + * -ENOMEM if anon_vma couldn't be allocated, or -EAGAIN if vma refcount
> > + * overflow happened due to high number of readers and the caller should
> > + * retry later.
>
> I'm guessing you're fixing this up as part of the patch? But it feels a bit
> random, I mean fine but you should mention this change in the commit message +
> explain why.

I'm fixing that because Vlastimil asked about these inconsistencies in
his previous review :)
I can package these fixes as a separate patch or add a comment in the
changelog, smth like "While at it, fix the comments for related
functions". Would that work?

>
> >   */
> >  static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,
> >                                      unsigned long address)
> > --
> > 2.55.0.508.g3f0d502094-goog
> >
>
> --
> Cheers, Lorenzo

Re: [PATCH v3 3/5] mm: Add RCU-based VMA lookup helper that waits for writers

[Vlastimil Babka (SUSE)]

On 8/2/26 23:54, Suren Baghdasaryan wrote:
> From: Dave Hansen <dave.hansen@linux.intel.com>
> 
> == Background ==
> 
> There are basically two parallel ways to look up a VMA: the
> traditional way, which is protected by mmap_read_lock, and the RCU-based
> per-VMA lock way which is based on RCU and refcounts.
> 
> == Problem ==
> 
> The mmap_lock one is more straightforward to use but it has a big
> disadvantage in that it can not be mixed with page faults since those
> can take mmap_lock for read, which can deadlock when mixed with nested
> page faults and parallel writers.
> For example:
> 
> 	mmap_read_lock(mm);
> 	// Another thread does mmap_write_lock().
> 	// New mmap_lock readers are blocked.
> 	vma = vma_lookup(mm, address);
> 	// This deadlocks on mmap_read_lock() if it faults:
> 	copy_from_user(address);
> 	mmap_read_unlock(mm);
> 
> The per-VMA lock can be mixed with faults, but they can fail and need to
> be able to fall back to the traditional way.
> 
> == Solution ==
> 
> Add a variant of the RCU-based lookup that waits for writers. This is
> basically the same as the existing RCU-based lookup, but on a failure to
> lock it temporarily takes mmap_lock for read and waits for writers
> to finish before locking the VMA, dropping the mmap_lock and returning
> the locked VMA. This has some advantages:

Maybe mention that the helper is called vma_start_read_unlocked()?

> 
>  1. Callers do not need to have a fallback path for when they
>     collide with writers.
>  2. It can be used in contexts where page faults can happen because
>     it can take the mmap_lock for read but never *holds* it.
>  3. Its fast path does not require taking mmap_lock for read.
> 
> Basically, when applied correctly, this approach results in faster
> *and* simpler code.
> 
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> Cc: Suren Baghdasaryan <surenb@google.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> Cc: Lorenzo Stoakes <ljs@kernel.org>
> Cc: Vlastimil Babka <vbabka@kernel.org>
> Cc: Shakeel Butt <shakeel.butt@linux.dev>
> Cc: linux-mm@kvack.org
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Arve Hjønnevåg <arve@android.com>
> Cc: Todd Kjos <tkjos@android.com>
> Cc: Christian Brauner <christian@brauner.io>
> Cc: Carlos Llamas <cmllamas@google.com>
> Cc: Alice Ryhl <aliceryhl@google.com>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: David Ahern <dsahern@kernel.org>
> Cc: netdev@vger.kernel.org
> ---
>  include/linux/mmap_lock.h | 15 +++++++++++----
>  mm/mmap_lock.c            | 29 +++++++++++++++++++++++++++++
>  mm/userfaultfd.c          |  6 ++++--
>  3 files changed, 44 insertions(+), 6 deletions(-)
> 
> diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
> index eb32b482434e..fdd8f5cf5722 100644
> --- a/include/linux/mmap_lock.h
> +++ b/include/linux/mmap_lock.h
> @@ -228,10 +228,12 @@ static inline void vma_refcount_put(struct vm_area_struct *vma)
>  }
>  
>  /*
> - * Use only while holding mmap read lock which guarantees that locking will not
> - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
> + * Use only while holding mmap read lock which guarantees that vma lock is not
> + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
>   * not be used in such cases because it might fail due to mm_lock_seq overflow.
>   * This functionality is used to obtain vma read lock and drop the mmap read lock.
> + * VMA can't be detached while we are holding mmap lock, therefore in practice this
> + * function can fail only when there are so many readers that vm_refcnt overflows.
>   */
>  static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int subclass)
>  {
> @@ -247,16 +249,21 @@ static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int
>  }
>  
>  /*
> - * Use only while holding mmap read lock which guarantees that locking will not
> - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
> + * Use only while holding mmap read lock which guarantees that vma lock is not
> + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
>   * not be used in such cases because it might fail due to mm_lock_seq overflow.
>   * This functionality is used to obtain vma read lock and drop the mmap read lock.
> + * VMA can't be detached while we are holding mmap lock, therefore in practice this
> + * function can fail only when there are so many readers that vm_refcnt overflows.
>   */
>  static inline bool vma_start_read_locked(struct vm_area_struct *vma)
>  {
>  	return vma_start_read_locked_nested(vma, 0);
>  }
>  
> +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
> +					       unsigned long address);
> +
>  static inline void vma_end_read(struct vm_area_struct *vma)
>  {
>  	vma_refcount_put(vma);
> diff --git a/mm/mmap_lock.c b/mm/mmap_lock.c
> index e20d01e8d38f..6ff05e68e61b 100644
> --- a/mm/mmap_lock.c
> +++ b/mm/mmap_lock.c
> @@ -338,6 +338,35 @@ struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
>  	return NULL;
>  }
>  
> +/*
> + * Find the VMA covering 'address' and lock it for reading. Waits for writers to
> + * finish if the VMA is being modified. Returns NULL if there is no VMA covering
> + * 'address'.

Hm but it can also return NULL when vm_refcnt overflows, in theory.
Should we also return -EAGAIN (like uffd_lock_vma() below), or just retry in
here and hope for the best? The latter would be simpler for the users.
(AFAICS due to VM_REFCNT_LIMIT we never end up triggering the refcount
saturation)

> + *
> + * Use only in code paths where no mmap_lock and no VMA lock is held.
> + *
> + * The fast path does not take mmap_lock.
> + */
> +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
> +					       unsigned long address)
> +{
> +	struct vm_area_struct *vma;
> +
> +	/* Fast path: return stable VMA covering 'address': */
> +	vma = lock_vma_under_rcu(mm, address);
> +	if (vma)
> +		return vma;
> +
> +	/* Slow path: preclude VMA writers by temporarily getting mmap read lock. */
> +	mmap_read_lock(mm);
> +	vma = vma_lookup(mm, address);
> +	if (vma && !vma_start_read_locked(vma))
> +		vma = NULL;
> +	mmap_read_unlock(mm);
> +
> +	return vma;
> +}
> +
>  static struct vm_area_struct *lock_next_vma_under_mmap_lock(struct mm_struct *mm,
>  							    struct vma_iterator *vmi,
>  							    unsigned long from_addr)
> diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
> index edd90892f8cc..c3a0c38a3dc3 100644
> --- a/mm/userfaultfd.c
> +++ b/mm/userfaultfd.c
> @@ -129,8 +129,10 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
>   *
>   * Should be called without holding mmap_lock.
>   *
> - * Return: A locked vma containing @address, -ENOENT if no vma is found, or
> - * -ENOMEM if anon_vma couldn't be allocated.
> + * Return: A locked vma containing @address, -ENOENT if no vma is found,
> + * -ENOMEM if anon_vma couldn't be allocated, or -EAGAIN if vma refcount
> + * overflow happened due to high number of readers and the caller should
> + * retry later.
>   */
>  static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,
>  				       unsigned long address)

Re: [PATCH v3 3/5] mm: Add RCU-based VMA lookup helper that waits for writers

[Lorenzo Stoakes (ARM)]

On Mon, Aug 03, 2026 at 04:55:19PM +0200, Vlastimil Babka (SUSE) wrote:
> On 8/2/26 23:54, Suren Baghdasaryan wrote:
> > From: Dave Hansen <dave.hansen@linux.intel.com>
> >
> > == Background ==
> >
> > There are basically two parallel ways to look up a VMA: the
> > traditional way, which is protected by mmap_read_lock, and the RCU-based
> > per-VMA lock way which is based on RCU and refcounts.
> >
> > == Problem ==
> >
> > The mmap_lock one is more straightforward to use but it has a big
> > disadvantage in that it can not be mixed with page faults since those
> > can take mmap_lock for read, which can deadlock when mixed with nested
> > page faults and parallel writers.
> > For example:
> >
> > 	mmap_read_lock(mm);
> > 	// Another thread does mmap_write_lock().
> > 	// New mmap_lock readers are blocked.
> > 	vma = vma_lookup(mm, address);
> > 	// This deadlocks on mmap_read_lock() if it faults:
> > 	copy_from_user(address);
> > 	mmap_read_unlock(mm);
> >
> > The per-VMA lock can be mixed with faults, but they can fail and need to
> > be able to fall back to the traditional way.
> >
> > == Solution ==
> >
> > Add a variant of the RCU-based lookup that waits for writers. This is
> > basically the same as the existing RCU-based lookup, but on a failure to
> > lock it temporarily takes mmap_lock for read and waits for writers
> > to finish before locking the VMA, dropping the mmap_lock and returning
> > the locked VMA. This has some advantages:
>
> Maybe mention that the helper is called vma_start_read_unlocked()?
>
> >
> >  1. Callers do not need to have a fallback path for when they
> >     collide with writers.
> >  2. It can be used in contexts where page faults can happen because
> >     it can take the mmap_lock for read but never *holds* it.
> >  3. Its fast path does not require taking mmap_lock for read.
> >
> > Basically, when applied correctly, this approach results in faster
> > *and* simpler code.
> >
> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > Cc: Suren Baghdasaryan <surenb@google.com>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> > Cc: Lorenzo Stoakes <ljs@kernel.org>
> > Cc: Vlastimil Babka <vbabka@kernel.org>
> > Cc: Shakeel Butt <shakeel.butt@linux.dev>
> > Cc: linux-mm@kvack.org
> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Cc: Arve Hjønnevåg <arve@android.com>
> > Cc: Todd Kjos <tkjos@android.com>
> > Cc: Christian Brauner <christian@brauner.io>
> > Cc: Carlos Llamas <cmllamas@google.com>
> > Cc: Alice Ryhl <aliceryhl@google.com>
> > Cc: "David S. Miller" <davem@davemloft.net>
> > Cc: David Ahern <dsahern@kernel.org>
> > Cc: netdev@vger.kernel.org
> > ---
> >  include/linux/mmap_lock.h | 15 +++++++++++----
> >  mm/mmap_lock.c            | 29 +++++++++++++++++++++++++++++
> >  mm/userfaultfd.c          |  6 ++++--
> >  3 files changed, 44 insertions(+), 6 deletions(-)
> >
> > diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
> > index eb32b482434e..fdd8f5cf5722 100644
> > --- a/include/linux/mmap_lock.h
> > +++ b/include/linux/mmap_lock.h
> > @@ -228,10 +228,12 @@ static inline void vma_refcount_put(struct vm_area_struct *vma)
> >  }
> >
> >  /*
> > - * Use only while holding mmap read lock which guarantees that locking will not
> > - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
> > + * Use only while holding mmap read lock which guarantees that vma lock is not
> > + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
> >   * not be used in such cases because it might fail due to mm_lock_seq overflow.
> >   * This functionality is used to obtain vma read lock and drop the mmap read lock.
> > + * VMA can't be detached while we are holding mmap lock, therefore in practice this
> > + * function can fail only when there are so many readers that vm_refcnt overflows.
> >   */
> >  static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int subclass)
> >  {
> > @@ -247,16 +249,21 @@ static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int
> >  }
> >
> >  /*
> > - * Use only while holding mmap read lock which guarantees that locking will not
> > - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
> > + * Use only while holding mmap read lock which guarantees that vma lock is not
> > + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
> >   * not be used in such cases because it might fail due to mm_lock_seq overflow.
> >   * This functionality is used to obtain vma read lock and drop the mmap read lock.
> > + * VMA can't be detached while we are holding mmap lock, therefore in practice this
> > + * function can fail only when there are so many readers that vm_refcnt overflows.
> >   */
> >  static inline bool vma_start_read_locked(struct vm_area_struct *vma)
> >  {
> >  	return vma_start_read_locked_nested(vma, 0);
> >  }
> >
> > +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
> > +					       unsigned long address);
> > +
> >  static inline void vma_end_read(struct vm_area_struct *vma)
> >  {
> >  	vma_refcount_put(vma);
> > diff --git a/mm/mmap_lock.c b/mm/mmap_lock.c
> > index e20d01e8d38f..6ff05e68e61b 100644
> > --- a/mm/mmap_lock.c
> > +++ b/mm/mmap_lock.c
> > @@ -338,6 +338,35 @@ struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
> >  	return NULL;
> >  }
> >
> > +/*
> > + * Find the VMA covering 'address' and lock it for reading. Waits for writers to
> > + * finish if the VMA is being modified. Returns NULL if there is no VMA covering
> > + * 'address'.
>
> Hm but it can also return NULL when vm_refcnt overflows, in theory.
> Should we also return -EAGAIN (like uffd_lock_vma() below), or just retry in
> here and hope for the best? The latter would be simpler for the users.
> (AFAICS due to VM_REFCNT_LIMIT we never end up triggering the refcount
> saturation)

The problem is everything's unlocked so 'didn't find a VMA' doesn't really mean
much more than 'something went wrong' because hey maybe if you check again now
you'll find something :)

So I think this might be a feature more than a bug, especially given overflow is
not exactly likely.

>
> > + *
> > + * Use only in code paths where no mmap_lock and no VMA lock is held.
> > + *
> > + * The fast path does not take mmap_lock.
> > + */
> > +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
> > +					       unsigned long address)
> > +{
> > +	struct vm_area_struct *vma;
> > +
> > +	/* Fast path: return stable VMA covering 'address': */
> > +	vma = lock_vma_under_rcu(mm, address);
> > +	if (vma)
> > +		return vma;
> > +
> > +	/* Slow path: preclude VMA writers by temporarily getting mmap read lock. */
> > +	mmap_read_lock(mm);
> > +	vma = vma_lookup(mm, address);
> > +	if (vma && !vma_start_read_locked(vma))
> > +		vma = NULL;
> > +	mmap_read_unlock(mm);
> > +
> > +	return vma;
> > +}
> > +
> >  static struct vm_area_struct *lock_next_vma_under_mmap_lock(struct mm_struct *mm,
> >  							    struct vma_iterator *vmi,
> >  							    unsigned long from_addr)
> > diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
> > index edd90892f8cc..c3a0c38a3dc3 100644
> > --- a/mm/userfaultfd.c
> > +++ b/mm/userfaultfd.c
> > @@ -129,8 +129,10 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
> >   *
> >   * Should be called without holding mmap_lock.
> >   *
> > - * Return: A locked vma containing @address, -ENOENT if no vma is found, or
> > - * -ENOMEM if anon_vma couldn't be allocated.
> > + * Return: A locked vma containing @address, -ENOENT if no vma is found,
> > + * -ENOMEM if anon_vma couldn't be allocated, or -EAGAIN if vma refcount
> > + * overflow happened due to high number of readers and the caller should
> > + * retry later.
> >   */
> >  static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,
> >  				       unsigned long address)
>

--
Cheers, Lorenzo

Re: [PATCH v3 3/5] mm: Add RCU-based VMA lookup helper that waits for writers

[Vlastimil Babka (SUSE)]

On 8/3/26 17:00, Lorenzo Stoakes (ARM) wrote:
> On Mon, Aug 03, 2026 at 04:55:19PM +0200, Vlastimil Babka (SUSE) wrote:
>> On 8/2/26 23:54, Suren Baghdasaryan wrote:
>> > From: Dave Hansen <dave.hansen@linux.intel.com>
>> >
>> > == Background ==
>> >
>> > There are basically two parallel ways to look up a VMA: the
>> > traditional way, which is protected by mmap_read_lock, and the RCU-based
>> > per-VMA lock way which is based on RCU and refcounts.
>> >
>> > == Problem ==
>> >
>> > The mmap_lock one is more straightforward to use but it has a big
>> > disadvantage in that it can not be mixed with page faults since those
>> > can take mmap_lock for read, which can deadlock when mixed with nested
>> > page faults and parallel writers.
>> > For example:
>> >
>> > 	mmap_read_lock(mm);
>> > 	// Another thread does mmap_write_lock().
>> > 	// New mmap_lock readers are blocked.
>> > 	vma = vma_lookup(mm, address);
>> > 	// This deadlocks on mmap_read_lock() if it faults:
>> > 	copy_from_user(address);
>> > 	mmap_read_unlock(mm);
>> >
>> > The per-VMA lock can be mixed with faults, but they can fail and need to
>> > be able to fall back to the traditional way.
>> >
>> > == Solution ==
>> >
>> > Add a variant of the RCU-based lookup that waits for writers. This is
>> > basically the same as the existing RCU-based lookup, but on a failure to
>> > lock it temporarily takes mmap_lock for read and waits for writers
>> > to finish before locking the VMA, dropping the mmap_lock and returning
>> > the locked VMA. This has some advantages:
>>
>> Maybe mention that the helper is called vma_start_read_unlocked()?
>>
>> >
>> >  1. Callers do not need to have a fallback path for when they
>> >     collide with writers.
>> >  2. It can be used in contexts where page faults can happen because
>> >     it can take the mmap_lock for read but never *holds* it.
>> >  3. Its fast path does not require taking mmap_lock for read.
>> >
>> > Basically, when applied correctly, this approach results in faster
>> > *and* simpler code.
>> >
>> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
>> > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
>> > Cc: Suren Baghdasaryan <surenb@google.com>
>> > Cc: Andrew Morton <akpm@linux-foundation.org>
>> > Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
>> > Cc: Lorenzo Stoakes <ljs@kernel.org>
>> > Cc: Vlastimil Babka <vbabka@kernel.org>
>> > Cc: Shakeel Butt <shakeel.butt@linux.dev>
>> > Cc: linux-mm@kvack.org
>> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>> > Cc: Arve Hjønnevåg <arve@android.com>
>> > Cc: Todd Kjos <tkjos@android.com>
>> > Cc: Christian Brauner <christian@brauner.io>
>> > Cc: Carlos Llamas <cmllamas@google.com>
>> > Cc: Alice Ryhl <aliceryhl@google.com>
>> > Cc: "David S. Miller" <davem@davemloft.net>
>> > Cc: David Ahern <dsahern@kernel.org>
>> > Cc: netdev@vger.kernel.org
>> > ---
>> >  include/linux/mmap_lock.h | 15 +++++++++++----
>> >  mm/mmap_lock.c            | 29 +++++++++++++++++++++++++++++
>> >  mm/userfaultfd.c          |  6 ++++--
>> >  3 files changed, 44 insertions(+), 6 deletions(-)
>> >
>> > diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
>> > index eb32b482434e..fdd8f5cf5722 100644
>> > --- a/include/linux/mmap_lock.h
>> > +++ b/include/linux/mmap_lock.h
>> > @@ -228,10 +228,12 @@ static inline void vma_refcount_put(struct vm_area_struct *vma)
>> >  }
>> >
>> >  /*
>> > - * Use only while holding mmap read lock which guarantees that locking will not
>> > - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
>> > + * Use only while holding mmap read lock which guarantees that vma lock is not
>> > + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
>> >   * not be used in such cases because it might fail due to mm_lock_seq overflow.
>> >   * This functionality is used to obtain vma read lock and drop the mmap read lock.
>> > + * VMA can't be detached while we are holding mmap lock, therefore in practice this
>> > + * function can fail only when there are so many readers that vm_refcnt overflows.
>> >   */
>> >  static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int subclass)
>> >  {
>> > @@ -247,16 +249,21 @@ static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int
>> >  }
>> >
>> >  /*
>> > - * Use only while holding mmap read lock which guarantees that locking will not
>> > - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
>> > + * Use only while holding mmap read lock which guarantees that vma lock is not
>> > + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
>> >   * not be used in such cases because it might fail due to mm_lock_seq overflow.
>> >   * This functionality is used to obtain vma read lock and drop the mmap read lock.
>> > + * VMA can't be detached while we are holding mmap lock, therefore in practice this
>> > + * function can fail only when there are so many readers that vm_refcnt overflows.
>> >   */
>> >  static inline bool vma_start_read_locked(struct vm_area_struct *vma)
>> >  {
>> >  	return vma_start_read_locked_nested(vma, 0);
>> >  }
>> >
>> > +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
>> > +					       unsigned long address);
>> > +
>> >  static inline void vma_end_read(struct vm_area_struct *vma)
>> >  {
>> >  	vma_refcount_put(vma);
>> > diff --git a/mm/mmap_lock.c b/mm/mmap_lock.c
>> > index e20d01e8d38f..6ff05e68e61b 100644
>> > --- a/mm/mmap_lock.c
>> > +++ b/mm/mmap_lock.c
>> > @@ -338,6 +338,35 @@ struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
>> >  	return NULL;
>> >  }
>> >
>> > +/*
>> > + * Find the VMA covering 'address' and lock it for reading. Waits for writers to
>> > + * finish if the VMA is being modified. Returns NULL if there is no VMA covering
>> > + * 'address'.
>>
>> Hm but it can also return NULL when vm_refcnt overflows, in theory.
>> Should we also return -EAGAIN (like uffd_lock_vma() below), or just retry in
>> here and hope for the best? The latter would be simpler for the users.
>> (AFAICS due to VM_REFCNT_LIMIT we never end up triggering the refcount
>> saturation)
> 
> The problem is everything's unlocked so 'didn't find a VMA' doesn't really mean
> much more than 'something went wrong' because hey maybe if you check again now
> you'll find something :)

Well there might be use cases where you know that either there's a vma with
your address and then you need to do something with it, or there's not and
then you don't. And it can't suddenly appear after you check.

So in that case treating that spurious NULL as "there's no vma so I don't
need to do anything" would be wrong.

The usages in 4/5 and 5/5 seem like they are not this case though. So it's
fine. But perhaps worth just mentioning it in the comment then.

> So I think this might be a feature more than a bug, especially given overflow is
> not exactly likely.
> 
>>
>> > + *
>> > + * Use only in code paths where no mmap_lock and no VMA lock is held.
>> > + *
>> > + * The fast path does not take mmap_lock.
>> > + */
>> > +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
>> > +					       unsigned long address)
>> > +{
>> > +	struct vm_area_struct *vma;
>> > +
>> > +	/* Fast path: return stable VMA covering 'address': */
>> > +	vma = lock_vma_under_rcu(mm, address);
>> > +	if (vma)
>> > +		return vma;
>> > +
>> > +	/* Slow path: preclude VMA writers by temporarily getting mmap read lock. */
>> > +	mmap_read_lock(mm);
>> > +	vma = vma_lookup(mm, address);
>> > +	if (vma && !vma_start_read_locked(vma))
>> > +		vma = NULL;
>> > +	mmap_read_unlock(mm);
>> > +
>> > +	return vma;
>> > +}
>> > +
>> >  static struct vm_area_struct *lock_next_vma_under_mmap_lock(struct mm_struct *mm,
>> >  							    struct vma_iterator *vmi,
>> >  							    unsigned long from_addr)
>> > diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
>> > index edd90892f8cc..c3a0c38a3dc3 100644
>> > --- a/mm/userfaultfd.c
>> > +++ b/mm/userfaultfd.c
>> > @@ -129,8 +129,10 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
>> >   *
>> >   * Should be called without holding mmap_lock.
>> >   *
>> > - * Return: A locked vma containing @address, -ENOENT if no vma is found, or
>> > - * -ENOMEM if anon_vma couldn't be allocated.
>> > + * Return: A locked vma containing @address, -ENOENT if no vma is found,
>> > + * -ENOMEM if anon_vma couldn't be allocated, or -EAGAIN if vma refcount
>> > + * overflow happened due to high number of readers and the caller should
>> > + * retry later.
>> >   */
>> >  static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,
>> >  				       unsigned long address)
>>
> 
> --
> Cheers, Lorenzo

Re: [PATCH v3 3/5] mm: Add RCU-based VMA lookup helper that waits for writers

[Lorenzo Stoakes (ARM)]

On Mon, Aug 03, 2026 at 06:24:34PM +0200, Vlastimil Babka (SUSE) wrote:
> On 8/3/26 17:00, Lorenzo Stoakes (ARM) wrote:
> > On Mon, Aug 03, 2026 at 04:55:19PM +0200, Vlastimil Babka (SUSE) wrote:
> >> On 8/2/26 23:54, Suren Baghdasaryan wrote:
> >> > From: Dave Hansen <dave.hansen@linux.intel.com>
> >> >
> >> > == Background ==
> >> >
> >> > There are basically two parallel ways to look up a VMA: the
> >> > traditional way, which is protected by mmap_read_lock, and the RCU-based
> >> > per-VMA lock way which is based on RCU and refcounts.
> >> >
> >> > == Problem ==
> >> >
> >> > The mmap_lock one is more straightforward to use but it has a big
> >> > disadvantage in that it can not be mixed with page faults since those
> >> > can take mmap_lock for read, which can deadlock when mixed with nested
> >> > page faults and parallel writers.
> >> > For example:
> >> >
> >> > 	mmap_read_lock(mm);
> >> > 	// Another thread does mmap_write_lock().
> >> > 	// New mmap_lock readers are blocked.
> >> > 	vma = vma_lookup(mm, address);
> >> > 	// This deadlocks on mmap_read_lock() if it faults:
> >> > 	copy_from_user(address);
> >> > 	mmap_read_unlock(mm);
> >> >
> >> > The per-VMA lock can be mixed with faults, but they can fail and need to
> >> > be able to fall back to the traditional way.
> >> >
> >> > == Solution ==
> >> >
> >> > Add a variant of the RCU-based lookup that waits for writers. This is
> >> > basically the same as the existing RCU-based lookup, but on a failure to
> >> > lock it temporarily takes mmap_lock for read and waits for writers
> >> > to finish before locking the VMA, dropping the mmap_lock and returning
> >> > the locked VMA. This has some advantages:
> >>
> >> Maybe mention that the helper is called vma_start_read_unlocked()?
> >>
> >> >
> >> >  1. Callers do not need to have a fallback path for when they
> >> >     collide with writers.
> >> >  2. It can be used in contexts where page faults can happen because
> >> >     it can take the mmap_lock for read but never *holds* it.
> >> >  3. Its fast path does not require taking mmap_lock for read.
> >> >
> >> > Basically, when applied correctly, this approach results in faster
> >> > *and* simpler code.
> >> >
> >> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> >> > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> >> > Cc: Suren Baghdasaryan <surenb@google.com>
> >> > Cc: Andrew Morton <akpm@linux-foundation.org>
> >> > Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> >> > Cc: Lorenzo Stoakes <ljs@kernel.org>
> >> > Cc: Vlastimil Babka <vbabka@kernel.org>
> >> > Cc: Shakeel Butt <shakeel.butt@linux.dev>
> >> > Cc: linux-mm@kvack.org
> >> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> >> > Cc: Arve Hjønnevåg <arve@android.com>
> >> > Cc: Todd Kjos <tkjos@android.com>
> >> > Cc: Christian Brauner <christian@brauner.io>
> >> > Cc: Carlos Llamas <cmllamas@google.com>
> >> > Cc: Alice Ryhl <aliceryhl@google.com>
> >> > Cc: "David S. Miller" <davem@davemloft.net>
> >> > Cc: David Ahern <dsahern@kernel.org>
> >> > Cc: netdev@vger.kernel.org
> >> > ---
> >> >  include/linux/mmap_lock.h | 15 +++++++++++----
> >> >  mm/mmap_lock.c            | 29 +++++++++++++++++++++++++++++
> >> >  mm/userfaultfd.c          |  6 ++++--
> >> >  3 files changed, 44 insertions(+), 6 deletions(-)
> >> >
> >> > diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
> >> > index eb32b482434e..fdd8f5cf5722 100644
> >> > --- a/include/linux/mmap_lock.h
> >> > +++ b/include/linux/mmap_lock.h
> >> > @@ -228,10 +228,12 @@ static inline void vma_refcount_put(struct vm_area_struct *vma)
> >> >  }
> >> >
> >> >  /*
> >> > - * Use only while holding mmap read lock which guarantees that locking will not
> >> > - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
> >> > + * Use only while holding mmap read lock which guarantees that vma lock is not
> >> > + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
> >> >   * not be used in such cases because it might fail due to mm_lock_seq overflow.
> >> >   * This functionality is used to obtain vma read lock and drop the mmap read lock.
> >> > + * VMA can't be detached while we are holding mmap lock, therefore in practice this
> >> > + * function can fail only when there are so many readers that vm_refcnt overflows.
> >> >   */
> >> >  static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int subclass)
> >> >  {
> >> > @@ -247,16 +249,21 @@ static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int
> >> >  }
> >> >
> >> >  /*
> >> > - * Use only while holding mmap read lock which guarantees that locking will not
> >> > - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
> >> > + * Use only while holding mmap read lock which guarantees that vma lock is not
> >> > + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
> >> >   * not be used in such cases because it might fail due to mm_lock_seq overflow.
> >> >   * This functionality is used to obtain vma read lock and drop the mmap read lock.
> >> > + * VMA can't be detached while we are holding mmap lock, therefore in practice this
> >> > + * function can fail only when there are so many readers that vm_refcnt overflows.
> >> >   */
> >> >  static inline bool vma_start_read_locked(struct vm_area_struct *vma)
> >> >  {
> >> >  	return vma_start_read_locked_nested(vma, 0);
> >> >  }
> >> >
> >> > +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
> >> > +					       unsigned long address);
> >> > +
> >> >  static inline void vma_end_read(struct vm_area_struct *vma)
> >> >  {
> >> >  	vma_refcount_put(vma);
> >> > diff --git a/mm/mmap_lock.c b/mm/mmap_lock.c
> >> > index e20d01e8d38f..6ff05e68e61b 100644
> >> > --- a/mm/mmap_lock.c
> >> > +++ b/mm/mmap_lock.c
> >> > @@ -338,6 +338,35 @@ struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
> >> >  	return NULL;
> >> >  }
> >> >
> >> > +/*
> >> > + * Find the VMA covering 'address' and lock it for reading. Waits for writers to
> >> > + * finish if the VMA is being modified. Returns NULL if there is no VMA covering
> >> > + * 'address'.
> >>
> >> Hm but it can also return NULL when vm_refcnt overflows, in theory.
> >> Should we also return -EAGAIN (like uffd_lock_vma() below), or just retry in
> >> here and hope for the best? The latter would be simpler for the users.
> >> (AFAICS due to VM_REFCNT_LIMIT we never end up triggering the refcount
> >> saturation)
> >
> > The problem is everything's unlocked so 'didn't find a VMA' doesn't really mean
> > much more than 'something went wrong' because hey maybe if you check again now
> > you'll find something :)
>
> Well there might be use cases where you know that either there's a vma with
> your address and then you need to do something with it, or there's not and
> then you don't. And it can't suddenly appear after you check.

You don't hold a lock that prevents new VMAs appearing/disappearing
spontaneously at the point you call lock_vma_under_rcu(), or after you drop the
mmap read lock, only that at the point of checking a VMA spans address, so
there's nothing preventing a VMA suddenly appearing after you check right? Or it
not being the one you wanted?

And checking to see if it's 'really the one you meant' is itself fraught (see
the whole uffd saga on that).

Point I'm making is that in any case where you'd actually care you'd need to
take a stronger lock anyway, so it's actually potentially dangerous to
differentiate between the two.

Given the overflow is very very unlikely I think it's also not a big deal to not
differentiate anyway.

>
> So in that case treating that spurious NULL as "there's no vma so I don't
> need to do anything" would be wrong.
>
> The usages in 4/5 and 5/5 seem like they are not this case though. So it's
> fine. But perhaps worth just mentioning it in the comment then.

Agree this is worth spelling out in the comment (I raised similarly).

Maybe something like:

	If a VMA exists which spans @address, return that VMA, read-locked.

	If no VMA is mapped there or, very unlikely, a reference count overflow
	occurred, return NULL.

	Nothing prevents VMAs being unmapped/mapped before or after the VMA is
	looked up, if a stronger guarantee is required, take an mmap lock.

>
> > So I think this might be a feature more than a bug, especially given overflow is
> > not exactly likely.
> >
> >>
> >> > + *
> >> > + * Use only in code paths where no mmap_lock and no VMA lock is held.
> >> > + *
> >> > + * The fast path does not take mmap_lock.
> >> > + */
> >> > +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
> >> > +					       unsigned long address)
> >> > +{
> >> > +	struct vm_area_struct *vma;
> >> > +
> >> > +	/* Fast path: return stable VMA covering 'address': */
> >> > +	vma = lock_vma_under_rcu(mm, address);
> >> > +	if (vma)
> >> > +		return vma;
> >> > +
> >> > +	/* Slow path: preclude VMA writers by temporarily getting mmap read lock. */
> >> > +	mmap_read_lock(mm);
> >> > +	vma = vma_lookup(mm, address);
> >> > +	if (vma && !vma_start_read_locked(vma))
> >> > +		vma = NULL;
> >> > +	mmap_read_unlock(mm);
> >> > +
> >> > +	return vma;
> >> > +}
> >> > +
> >> >  static struct vm_area_struct *lock_next_vma_under_mmap_lock(struct mm_struct *mm,
> >> >  							    struct vma_iterator *vmi,
> >> >  							    unsigned long from_addr)
> >> > diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
> >> > index edd90892f8cc..c3a0c38a3dc3 100644
> >> > --- a/mm/userfaultfd.c
> >> > +++ b/mm/userfaultfd.c
> >> > @@ -129,8 +129,10 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
> >> >   *
> >> >   * Should be called without holding mmap_lock.
> >> >   *
> >> > - * Return: A locked vma containing @address, -ENOENT if no vma is found, or
> >> > - * -ENOMEM if anon_vma couldn't be allocated.
> >> > + * Return: A locked vma containing @address, -ENOENT if no vma is found,
> >> > + * -ENOMEM if anon_vma couldn't be allocated, or -EAGAIN if vma refcount
> >> > + * overflow happened due to high number of readers and the caller should
> >> > + * retry later.
> >> >   */
> >> >  static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,
> >> >  				       unsigned long address)
> >>
> >
> > --
> > Cheers, Lorenzo
>

--
Cheers, Lorenzo

Re: [PATCH v3 3/5] mm: Add RCU-based VMA lookup helper that waits for writers

[Suren Baghdasaryan]

On Mon, Aug 3, 2026 at 9:44 AM Lorenzo Stoakes (ARM) <ljs@kernel.org> wrote:
>
> On Mon, Aug 03, 2026 at 06:24:34PM +0200, Vlastimil Babka (SUSE) wrote:
> > On 8/3/26 17:00, Lorenzo Stoakes (ARM) wrote:
> > > On Mon, Aug 03, 2026 at 04:55:19PM +0200, Vlastimil Babka (SUSE) wrote:
> > >> On 8/2/26 23:54, Suren Baghdasaryan wrote:
> > >> > From: Dave Hansen <dave.hansen@linux.intel.com>
> > >> >
> > >> > == Background ==
> > >> >
> > >> > There are basically two parallel ways to look up a VMA: the
> > >> > traditional way, which is protected by mmap_read_lock, and the RCU-based
> > >> > per-VMA lock way which is based on RCU and refcounts.
> > >> >
> > >> > == Problem ==
> > >> >
> > >> > The mmap_lock one is more straightforward to use but it has a big
> > >> > disadvantage in that it can not be mixed with page faults since those
> > >> > can take mmap_lock for read, which can deadlock when mixed with nested
> > >> > page faults and parallel writers.
> > >> > For example:
> > >> >
> > >> >  mmap_read_lock(mm);
> > >> >  // Another thread does mmap_write_lock().
> > >> >  // New mmap_lock readers are blocked.
> > >> >  vma = vma_lookup(mm, address);
> > >> >  // This deadlocks on mmap_read_lock() if it faults:
> > >> >  copy_from_user(address);
> > >> >  mmap_read_unlock(mm);
> > >> >
> > >> > The per-VMA lock can be mixed with faults, but they can fail and need to
> > >> > be able to fall back to the traditional way.
> > >> >
> > >> > == Solution ==
> > >> >
> > >> > Add a variant of the RCU-based lookup that waits for writers. This is
> > >> > basically the same as the existing RCU-based lookup, but on a failure to
> > >> > lock it temporarily takes mmap_lock for read and waits for writers
> > >> > to finish before locking the VMA, dropping the mmap_lock and returning
> > >> > the locked VMA. This has some advantages:
> > >>
> > >> Maybe mention that the helper is called vma_start_read_unlocked()?

Ack.

> > >>
> > >> >
> > >> >  1. Callers do not need to have a fallback path for when they
> > >> >     collide with writers.
> > >> >  2. It can be used in contexts where page faults can happen because
> > >> >     it can take the mmap_lock for read but never *holds* it.
> > >> >  3. Its fast path does not require taking mmap_lock for read.
> > >> >
> > >> > Basically, when applied correctly, this approach results in faster
> > >> > *and* simpler code.
> > >> >
> > >> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > >> > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > >> > Cc: Suren Baghdasaryan <surenb@google.com>
> > >> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > >> > Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
> > >> > Cc: Lorenzo Stoakes <ljs@kernel.org>
> > >> > Cc: Vlastimil Babka <vbabka@kernel.org>
> > >> > Cc: Shakeel Butt <shakeel.butt@linux.dev>
> > >> > Cc: linux-mm@kvack.org
> > >> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > >> > Cc: Arve Hjønnevåg <arve@android.com>
> > >> > Cc: Todd Kjos <tkjos@android.com>
> > >> > Cc: Christian Brauner <christian@brauner.io>
> > >> > Cc: Carlos Llamas <cmllamas@google.com>
> > >> > Cc: Alice Ryhl <aliceryhl@google.com>
> > >> > Cc: "David S. Miller" <davem@davemloft.net>
> > >> > Cc: David Ahern <dsahern@kernel.org>
> > >> > Cc: netdev@vger.kernel.org
> > >> > ---
> > >> >  include/linux/mmap_lock.h | 15 +++++++++++----
> > >> >  mm/mmap_lock.c            | 29 +++++++++++++++++++++++++++++
> > >> >  mm/userfaultfd.c          |  6 ++++--
> > >> >  3 files changed, 44 insertions(+), 6 deletions(-)
> > >> >
> > >> > diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
> > >> > index eb32b482434e..fdd8f5cf5722 100644
> > >> > --- a/include/linux/mmap_lock.h
> > >> > +++ b/include/linux/mmap_lock.h
> > >> > @@ -228,10 +228,12 @@ static inline void vma_refcount_put(struct vm_area_struct *vma)
> > >> >  }
> > >> >
> > >> >  /*
> > >> > - * Use only while holding mmap read lock which guarantees that locking will not
> > >> > - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
> > >> > + * Use only while holding mmap read lock which guarantees that vma lock is not
> > >> > + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
> > >> >   * not be used in such cases because it might fail due to mm_lock_seq overflow.
> > >> >   * This functionality is used to obtain vma read lock and drop the mmap read lock.
> > >> > + * VMA can't be detached while we are holding mmap lock, therefore in practice this
> > >> > + * function can fail only when there are so many readers that vm_refcnt overflows.
> > >> >   */
> > >> >  static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int subclass)
> > >> >  {
> > >> > @@ -247,16 +249,21 @@ static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int
> > >> >  }
> > >> >
> > >> >  /*
> > >> > - * Use only while holding mmap read lock which guarantees that locking will not
> > >> > - * fail (nobody can concurrently write-lock the vma). vma_start_read() should
> > >> > + * Use only while holding mmap read lock which guarantees that vma lock is not
> > >> > + * contended (nobody can concurrently write-lock the vma). vma_start_read() should
> > >> >   * not be used in such cases because it might fail due to mm_lock_seq overflow.
> > >> >   * This functionality is used to obtain vma read lock and drop the mmap read lock.
> > >> > + * VMA can't be detached while we are holding mmap lock, therefore in practice this
> > >> > + * function can fail only when there are so many readers that vm_refcnt overflows.
> > >> >   */
> > >> >  static inline bool vma_start_read_locked(struct vm_area_struct *vma)
> > >> >  {
> > >> >          return vma_start_read_locked_nested(vma, 0);
> > >> >  }
> > >> >
> > >> > +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
> > >> > +                                               unsigned long address);
> > >> > +
> > >> >  static inline void vma_end_read(struct vm_area_struct *vma)
> > >> >  {
> > >> >          vma_refcount_put(vma);
> > >> > diff --git a/mm/mmap_lock.c b/mm/mmap_lock.c
> > >> > index e20d01e8d38f..6ff05e68e61b 100644
> > >> > --- a/mm/mmap_lock.c
> > >> > +++ b/mm/mmap_lock.c
> > >> > @@ -338,6 +338,35 @@ struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
> > >> >          return NULL;
> > >> >  }
> > >> >
> > >> > +/*
> > >> > + * Find the VMA covering 'address' and lock it for reading. Waits for writers to
> > >> > + * finish if the VMA is being modified. Returns NULL if there is no VMA covering
> > >> > + * 'address'.
> > >>
> > >> Hm but it can also return NULL when vm_refcnt overflows, in theory.
> > >> Should we also return -EAGAIN (like uffd_lock_vma() below), or just retry in
> > >> here and hope for the best? The latter would be simpler for the users.
> > >> (AFAICS due to VM_REFCNT_LIMIT we never end up triggering the refcount
> > >> saturation)
> > >
> > > The problem is everything's unlocked so 'didn't find a VMA' doesn't really mean
> > > much more than 'something went wrong' because hey maybe if you check again now
> > > you'll find something :)
> >
> > Well there might be use cases where you know that either there's a vma with
> > your address and then you need to do something with it, or there's not and
> > then you don't. And it can't suddenly appear after you check.
>
> You don't hold a lock that prevents new VMAs appearing/disappearing
> spontaneously at the point you call lock_vma_under_rcu(), or after you drop the
> mmap read lock, only that at the point of checking a VMA spans address, so
> there's nothing preventing a VMA suddenly appearing after you check right? Or it
> not being the one you wanted?
>
> And checking to see if it's 'really the one you meant' is itself fraught (see
> the whole uffd saga on that).
>
> Point I'm making is that in any case where you'd actually care you'd need to
> take a stronger lock anyway, so it's actually potentially dangerous to
> differentiate between the two.

Yeah, I tend to agree with Lorenzo that  when lock_vma_under_rcu()
fails, we should not make any assumptions about the reason because the
range is not locked and therefore is not stable. Any assumption risks
being wrong if a race occurs.

>
> Given the overflow is very very unlikely I think it's also not a big deal to not
> differentiate anyway.
>
> >
> > So in that case treating that spurious NULL as "there's no vma so I don't
> > need to do anything" would be wrong.
> >
> > The usages in 4/5 and 5/5 seem like they are not this case though. So it's
> > fine. But perhaps worth just mentioning it in the comment then.
>
> Agree this is worth spelling out in the comment (I raised similarly).
>
> Maybe something like:
>
>         If a VMA exists which spans @address, return that VMA, read-locked.
>
>         If no VMA is mapped there or, very unlikely, a reference count overflow
>         occurred, return NULL.
>
>         Nothing prevents VMAs being unmapped/mapped before or after the VMA is
>         looked up, if a stronger guarantee is required, take an mmap lock.

This last statement is true only if the function returns NULL, so I
think it should be in the same paragraph as the "If no VMA is
mapped..." sentence and prepended with "In this case...". So:

       If no VMA is mapped there or, very unlikely, a reference count overflow
       occurred, return NULL. In this case, nothing prevents VMAs
being unmapped/
       mapped before or after the VMA is looked up, if a stronger guarantee is
       required, take an mmap lock.

Does that sounds good?

>
> >
> > > So I think this might be a feature more than a bug, especially given overflow is
> > > not exactly likely.
> > >
> > >>
> > >> > + *
> > >> > + * Use only in code paths where no mmap_lock and no VMA lock is held.
> > >> > + *
> > >> > + * The fast path does not take mmap_lock.
> > >> > + */
> > >> > +struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
> > >> > +                                               unsigned long address)
> > >> > +{
> > >> > +        struct vm_area_struct *vma;
> > >> > +
> > >> > +        /* Fast path: return stable VMA covering 'address': */
> > >> > +        vma = lock_vma_under_rcu(mm, address);
> > >> > +        if (vma)
> > >> > +                return vma;
> > >> > +
> > >> > +        /* Slow path: preclude VMA writers by temporarily getting mmap read lock. */
> > >> > +        mmap_read_lock(mm);
> > >> > +        vma = vma_lookup(mm, address);
> > >> > +        if (vma && !vma_start_read_locked(vma))
> > >> > +                vma = NULL;
> > >> > +        mmap_read_unlock(mm);
> > >> > +
> > >> > +        return vma;
> > >> > +}
> > >> > +
> > >> >  static struct vm_area_struct *lock_next_vma_under_mmap_lock(struct mm_struct *mm,
> > >> >                                                              struct vma_iterator *vmi,
> > >> >                                                              unsigned long from_addr)
> > >> > diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
> > >> > index edd90892f8cc..c3a0c38a3dc3 100644
> > >> > --- a/mm/userfaultfd.c
> > >> > +++ b/mm/userfaultfd.c
> > >> > @@ -129,8 +129,10 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
> > >> >   *
> > >> >   * Should be called without holding mmap_lock.
> > >> >   *
> > >> > - * Return: A locked vma containing @address, -ENOENT if no vma is found, or
> > >> > - * -ENOMEM if anon_vma couldn't be allocated.
> > >> > + * Return: A locked vma containing @address, -ENOENT if no vma is found,
> > >> > + * -ENOMEM if anon_vma couldn't be allocated, or -EAGAIN if vma refcount
> > >> > + * overflow happened due to high number of readers and the caller should
> > >> > + * retry later.
> > >> >   */
> > >> >  static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,
> > >> >                                         unsigned long address)
> > >>
> > >
> > > --
> > > Cheers, Lorenzo
> >
>
> --
> Cheers, Lorenzo

[PATCH v3 4/5] binder: Remove mmap_lock fallback

[Suren Baghdasaryan]

From: Dave Hansen <dave.hansen@linux.intel.com>

Previously, the per-VMA locking could fail in the face of writers
which necessitate a fallback to mmap_lock. The new
vma_start_read_unlocked() will wait for writers instead of failing.

Use the new helper. Wait for writers. Remove the fallback to mmap_lock.

Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Liam R. Howlett <Liam.Howlett@oracle.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: linux-mm@kvack.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Arve Hjønnevåg <arve@android.com>
Cc: Todd Kjos <tkjos@android.com>
Cc: Christian Brauner <christian@brauner.io>
Cc: Carlos Llamas <cmllamas@google.com>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: David Ahern <dsahern@kernel.org>
Cc: netdev@vger.kernel.org
---
 drivers/android/binder/page_range.rs | 19 +++----------------
 drivers/android/binder_alloc.c       | 17 +++++------------
 rust/kernel/mm.rs                    | 18 ++++++++++++++++++
 3 files changed, 26 insertions(+), 28 deletions(-)

diff --git a/drivers/android/binder/page_range.rs b/drivers/android/binder/page_range.rs
index e82a5523804f..f7ad88a0d806 100644
--- a/drivers/android/binder/page_range.rs
+++ b/drivers/android/binder/page_range.rs
@@ -439,22 +439,9 @@ unsafe fn use_page_slow(&self, i: usize) -> Result<()> {
         // workqueue.
         let mm = MmWithUser::into_mmput_async(self.mm.mmget_not_zero().ok_or(ESRCH)?);
         {
-            let vma_read;
-            let mmap_read;
-            let vma = if let Some(ret) = mm.lock_vma_under_rcu(vma_addr) {
-                vma_read = ret;
-                check_vma(&vma_read, self)
-            } else {
-                mmap_read = mm.mmap_read_lock();
-                mmap_read
-                    .vma_lookup(vma_addr)
-                    .and_then(|vma| check_vma(vma, self))
-            };
-
-            match vma {
-                Some(vma) => vma.vm_insert_page(user_page_addr, &new_page)?,
-                None => return Err(ESRCH),
-            }
+            let vma_read_guard = mm.vma_start_read_unlocked(vma_addr).ok_or(ESRCH)?;
+            let vma = check_vma(&vma_read_guard, self).ok_or(ESRCH)?;
+            vma.vm_insert_page(user_page_addr, &new_page)?;
         }
 
         let inner = self.lock.lock();
diff --git a/drivers/android/binder_alloc.c b/drivers/android/binder_alloc.c
index 84104ba04e30..519dcded19b2 100644
--- a/drivers/android/binder_alloc.c
+++ b/drivers/android/binder_alloc.c
@@ -259,21 +259,14 @@ static int binder_page_insert(struct binder_alloc *alloc,
 	struct vm_area_struct *vma;
 	int ret = -ESRCH;
 
-	/* attempt per-vma lock first */
-	vma = lock_vma_under_rcu(mm, addr);
-	if (vma) {
-		if (binder_alloc_is_mapped(alloc))
-			ret = vm_insert_page(vma, addr, page);
-		vma_end_read(vma);
+	vma = vma_start_read_unlocked(mm, addr);
+	if (!vma)
 		return ret;
-	}
 
-	/* fall back to mmap_lock */
-	mmap_read_lock(mm);
-	vma = vma_lookup(mm, addr);
-	if (vma && binder_alloc_is_mapped(alloc))
+	if (binder_alloc_is_mapped(alloc))
 		ret = vm_insert_page(vma, addr, page);
-	mmap_read_unlock(mm);
+
+	vma_end_read(vma);
 
 	return ret;
 }
diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs
index 2633e704c83d..877fad68be9c 100644
--- a/rust/kernel/mm.rs
+++ b/rust/kernel/mm.rs
@@ -190,6 +190,24 @@ pub fn lock_vma_under_rcu(&self, vma_addr: usize) -> Option<VmaReadGuard<'_>> {
         }
     }
 
+    /// Find the VMA covering 'address' and lock it for reading. Waits for writers to finish if the
+    /// VMA is being modified.
+    #[inline]
+    pub fn vma_start_read_unlocked(&self, vma_addr: usize) -> Option<VmaReadGuard<'_>> {
+        // SAFETY: We may invoke `vma_start_read_unlocked` because we know this `mm` has non-zero
+        // `mm_users`.
+        let vma = unsafe { bindings::vma_start_read_unlocked(self.as_raw(), vma_addr) };
+        if vma.is_null() {
+            return None;
+        }
+        Some(VmaReadGuard {
+            // SAFETY: If `vma_start_read_unlocked` returns a non-null ptr, then it points at a
+            // valid vma. The vma is stable for as long as the vma read lock is held.
+            vma: unsafe { VmaRef::from_raw(vma) },
+            _nts: NotThreadSafe,
+        })
+    }
+
     /// Lock the mmap read lock.
     #[inline]
     pub fn mmap_read_lock(&self) -> MmapReadGuard<'_> {
-- 
2.55.0.508.g3f0d502094-goog

Re: [PATCH v3 4/5] binder: Remove mmap_lock fallback

[Alice Ryhl]

On Sun, Aug 02, 2026 at 02:54:58PM -0700, Suren Baghdasaryan wrote:
> From: Dave Hansen <dave.hansen@linux.intel.com>
> 
> Previously, the per-VMA locking could fail in the face of writers
> which necessitate a fallback to mmap_lock. The new
> vma_start_read_unlocked() will wait for writers instead of failing.
> 
> Use the new helper. Wait for writers. Remove the fallback to mmap_lock.
> 
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Signed-off-by: Suren Baghdasaryan <surenb@google.com>

Reviewed-by: Alice Ryhl <aliceryhl@google.com>

Re: [PATCH v3 4/5] binder: Remove mmap_lock fallback

[Suren Baghdasaryan]

On Mon, Aug 3, 2026 at 3:34 AM Alice Ryhl <aliceryhl@google.com> wrote:
>
> On Sun, Aug 02, 2026 at 02:54:58PM -0700, Suren Baghdasaryan wrote:
> > From: Dave Hansen <dave.hansen@linux.intel.com>
> >
> > Previously, the per-VMA locking could fail in the face of writers
> > which necessitate a fallback to mmap_lock. The new
> > vma_start_read_unlocked() will wait for writers instead of failing.
> >
> > Use the new helper. Wait for writers. Remove the fallback to mmap_lock.
> >
> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
>
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>

Thanks!

Re: [PATCH v3 4/5] binder: Remove mmap_lock fallback

[Lorenzo Stoakes (ARM)]

On Sun, Aug 02, 2026 at 02:54:58PM -0700, Suren Baghdasaryan wrote:
> From: Dave Hansen <dave.hansen@linux.intel.com>
>
> Previously, the per-VMA locking could fail in the face of writers
> which necessitate a fallback to mmap_lock. The new
> vma_start_read_unlocked() will wait for writers instead of failing.
>
> Use the new helper. Wait for writers. Remove the fallback to mmap_lock.
>
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Signed-off-by: Suren Baghdasaryan <surenb@google.com>

LGTM, just a nit below.

Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>

> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Liam R. Howlett <Liam.Howlett@oracle.com>
> Cc: Vlastimil Babka <vbabka@kernel.org>
> Cc: Shakeel Butt <shakeel.butt@linux.dev>
> Cc: linux-mm@kvack.org
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Arve Hjønnevåg <arve@android.com>
> Cc: Todd Kjos <tkjos@android.com>
> Cc: Christian Brauner <christian@brauner.io>
> Cc: Carlos Llamas <cmllamas@google.com>
> Cc: Alice Ryhl <aliceryhl@google.com>
> Cc: David S. Miller <davem@davemloft.net>
> Cc: David Ahern <dsahern@kernel.org>
> Cc: netdev@vger.kernel.org
> ---
>  drivers/android/binder/page_range.rs | 19 +++----------------
>  drivers/android/binder_alloc.c       | 17 +++++------------
>  rust/kernel/mm.rs                    | 18 ++++++++++++++++++
>  3 files changed, 26 insertions(+), 28 deletions(-)
>
> diff --git a/drivers/android/binder/page_range.rs b/drivers/android/binder/page_range.rs
> index e82a5523804f..f7ad88a0d806 100644
> --- a/drivers/android/binder/page_range.rs
> +++ b/drivers/android/binder/page_range.rs
> @@ -439,22 +439,9 @@ unsafe fn use_page_slow(&self, i: usize) -> Result<()> {
>          // workqueue.
>          let mm = MmWithUser::into_mmput_async(self.mm.mmget_not_zero().ok_or(ESRCH)?);
>          {
> -            let vma_read;
> -            let mmap_read;
> -            let vma = if let Some(ret) = mm.lock_vma_under_rcu(vma_addr) {
> -                vma_read = ret;
> -                check_vma(&vma_read, self)
> -            } else {
> -                mmap_read = mm.mmap_read_lock();
> -                mmap_read
> -                    .vma_lookup(vma_addr)
> -                    .and_then(|vma| check_vma(vma, self))
> -            };
> -
> -            match vma {
> -                Some(vma) => vma.vm_insert_page(user_page_addr, &new_page)?,
> -                None => return Err(ESRCH),
> -            }
> +            let vma_read_guard = mm.vma_start_read_unlocked(vma_addr).ok_or(ESRCH)?;
> +            let vma = check_vma(&vma_read_guard, self).ok_or(ESRCH)?;
> +            vma.vm_insert_page(user_page_addr, &new_page)?;
>          }
>
>          let inner = self.lock.lock();
> diff --git a/drivers/android/binder_alloc.c b/drivers/android/binder_alloc.c
> index 84104ba04e30..519dcded19b2 100644
> --- a/drivers/android/binder_alloc.c
> +++ b/drivers/android/binder_alloc.c
> @@ -259,21 +259,14 @@ static int binder_page_insert(struct binder_alloc *alloc,
>  	struct vm_area_struct *vma;
>  	int ret = -ESRCH;
>
> -	/* attempt per-vma lock first */
> -	vma = lock_vma_under_rcu(mm, addr);
> -	if (vma) {
> -		if (binder_alloc_is_mapped(alloc))
> -			ret = vm_insert_page(vma, addr, page);
> -		vma_end_read(vma);
> +	vma = vma_start_read_unlocked(mm, addr);
> +	if (!vma)
>  		return ret;
> -	}
>
> -	/* fall back to mmap_lock */
> -	mmap_read_lock(mm);
> -	vma = vma_lookup(mm, addr);
> -	if (vma && binder_alloc_is_mapped(alloc))
> +	if (binder_alloc_is_mapped(alloc))
>  		ret = vm_insert_page(vma, addr, page);
> -	mmap_read_unlock(mm);
> +
> +	vma_end_read(vma);

Nice cleanup :)

>
>  	return ret;
>  }
> diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs
> index 2633e704c83d..877fad68be9c 100644
> --- a/rust/kernel/mm.rs
> +++ b/rust/kernel/mm.rs
> @@ -190,6 +190,24 @@ pub fn lock_vma_under_rcu(&self, vma_addr: usize) -> Option<VmaReadGuard<'_>> {
>          }
>      }
>
> +    /// Find the VMA covering 'address' and lock it for reading. Waits for writers to finish if the
> +    /// VMA is being modified.

This seems a little inconsistent with the C version's comment, should they not be the same?

> +    #[inline]
> +    pub fn vma_start_read_unlocked(&self, vma_addr: usize) -> Option<VmaReadGuard<'_>> {
> +        // SAFETY: We may invoke `vma_start_read_unlocked` because we know this `mm` has non-zero
> +        // `mm_users`.
> +        let vma = unsafe { bindings::vma_start_read_unlocked(self.as_raw(), vma_addr) };
> +        if vma.is_null() {
> +            return None;
> +        }
> +        Some(VmaReadGuard {
> +            // SAFETY: If `vma_start_read_unlocked` returns a non-null ptr, then it points at a
> +            // valid vma. The vma is stable for as long as the vma read lock is held.
> +            vma: unsafe { VmaRef::from_raw(vma) },
> +            _nts: NotThreadSafe,
> +        })
> +    }
> +
>      /// Lock the mmap read lock.
>      #[inline]
>      pub fn mmap_read_lock(&self) -> MmapReadGuard<'_> {
> --
> 2.55.0.508.g3f0d502094-goog
>

--
Cheers, Lorenzo

Re: [PATCH v3 4/5] binder: Remove mmap_lock fallback

[Suren Baghdasaryan]

On Mon, Aug 3, 2026 at 4:33 AM Lorenzo Stoakes (ARM) <ljs@kernel.org> wrote:
>
> On Sun, Aug 02, 2026 at 02:54:58PM -0700, Suren Baghdasaryan wrote:
> > From: Dave Hansen <dave.hansen@linux.intel.com>
> >
> > Previously, the per-VMA locking could fail in the face of writers
> > which necessitate a fallback to mmap_lock. The new
> > vma_start_read_unlocked() will wait for writers instead of failing.
> >
> > Use the new helper. Wait for writers. Remove the fallback to mmap_lock.
> >
> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
>
> LGTM, just a nit below.
>
> Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>

Thanks!

>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: Liam R. Howlett <Liam.Howlett@oracle.com>
> > Cc: Vlastimil Babka <vbabka@kernel.org>
> > Cc: Shakeel Butt <shakeel.butt@linux.dev>
> > Cc: linux-mm@kvack.org
> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Cc: Arve Hjønnevåg <arve@android.com>
> > Cc: Todd Kjos <tkjos@android.com>
> > Cc: Christian Brauner <christian@brauner.io>
> > Cc: Carlos Llamas <cmllamas@google.com>
> > Cc: Alice Ryhl <aliceryhl@google.com>
> > Cc: David S. Miller <davem@davemloft.net>
> > Cc: David Ahern <dsahern@kernel.org>
> > Cc: netdev@vger.kernel.org
> > ---
> >  drivers/android/binder/page_range.rs | 19 +++----------------
> >  drivers/android/binder_alloc.c       | 17 +++++------------
> >  rust/kernel/mm.rs                    | 18 ++++++++++++++++++
> >  3 files changed, 26 insertions(+), 28 deletions(-)
> >
> > diff --git a/drivers/android/binder/page_range.rs b/drivers/android/binder/page_range.rs
> > index e82a5523804f..f7ad88a0d806 100644
> > --- a/drivers/android/binder/page_range.rs
> > +++ b/drivers/android/binder/page_range.rs
> > @@ -439,22 +439,9 @@ unsafe fn use_page_slow(&self, i: usize) -> Result<()> {
> >          // workqueue.
> >          let mm = MmWithUser::into_mmput_async(self.mm.mmget_not_zero().ok_or(ESRCH)?);
> >          {
> > -            let vma_read;
> > -            let mmap_read;
> > -            let vma = if let Some(ret) = mm.lock_vma_under_rcu(vma_addr) {
> > -                vma_read = ret;
> > -                check_vma(&vma_read, self)
> > -            } else {
> > -                mmap_read = mm.mmap_read_lock();
> > -                mmap_read
> > -                    .vma_lookup(vma_addr)
> > -                    .and_then(|vma| check_vma(vma, self))
> > -            };
> > -
> > -            match vma {
> > -                Some(vma) => vma.vm_insert_page(user_page_addr, &new_page)?,
> > -                None => return Err(ESRCH),
> > -            }
> > +            let vma_read_guard = mm.vma_start_read_unlocked(vma_addr).ok_or(ESRCH)?;
> > +            let vma = check_vma(&vma_read_guard, self).ok_or(ESRCH)?;
> > +            vma.vm_insert_page(user_page_addr, &new_page)?;
> >          }
> >
> >          let inner = self.lock.lock();
> > diff --git a/drivers/android/binder_alloc.c b/drivers/android/binder_alloc.c
> > index 84104ba04e30..519dcded19b2 100644
> > --- a/drivers/android/binder_alloc.c
> > +++ b/drivers/android/binder_alloc.c
> > @@ -259,21 +259,14 @@ static int binder_page_insert(struct binder_alloc *alloc,
> >       struct vm_area_struct *vma;
> >       int ret = -ESRCH;
> >
> > -     /* attempt per-vma lock first */
> > -     vma = lock_vma_under_rcu(mm, addr);
> > -     if (vma) {
> > -             if (binder_alloc_is_mapped(alloc))
> > -                     ret = vm_insert_page(vma, addr, page);
> > -             vma_end_read(vma);
> > +     vma = vma_start_read_unlocked(mm, addr);
> > +     if (!vma)
> >               return ret;
> > -     }
> >
> > -     /* fall back to mmap_lock */
> > -     mmap_read_lock(mm);
> > -     vma = vma_lookup(mm, addr);
> > -     if (vma && binder_alloc_is_mapped(alloc))
> > +     if (binder_alloc_is_mapped(alloc))
> >               ret = vm_insert_page(vma, addr, page);
> > -     mmap_read_unlock(mm);
> > +
> > +     vma_end_read(vma);
>
> Nice cleanup :)
>
> >
> >       return ret;
> >  }
> > diff --git a/rust/kernel/mm.rs b/rust/kernel/mm.rs
> > index 2633e704c83d..877fad68be9c 100644
> > --- a/rust/kernel/mm.rs
> > +++ b/rust/kernel/mm.rs
> > @@ -190,6 +190,24 @@ pub fn lock_vma_under_rcu(&self, vma_addr: usize) -> Option<VmaReadGuard<'_>> {
> >          }
> >      }
> >
> > +    /// Find the VMA covering 'address' and lock it for reading. Waits for writers to finish if the
> > +    /// VMA is being modified.
>
> This seems a little inconsistent with the C version's comment, should they not be the same?

Ack. Will change.

>
> > +    #[inline]
> > +    pub fn vma_start_read_unlocked(&self, vma_addr: usize) -> Option<VmaReadGuard<'_>> {
> > +        // SAFETY: We may invoke `vma_start_read_unlocked` because we know this `mm` has non-zero
> > +        // `mm_users`.
> > +        let vma = unsafe { bindings::vma_start_read_unlocked(self.as_raw(), vma_addr) };
> > +        if vma.is_null() {
> > +            return None;
> > +        }
> > +        Some(VmaReadGuard {
> > +            // SAFETY: If `vma_start_read_unlocked` returns a non-null ptr, then it points at a
> > +            // valid vma. The vma is stable for as long as the vma read lock is held.
> > +            vma: unsafe { VmaRef::from_raw(vma) },
> > +            _nts: NotThreadSafe,
> > +        })
> > +    }
> > +
> >      /// Lock the mmap read lock.
> >      #[inline]
> >      pub fn mmap_read_lock(&self) -> MmapReadGuard<'_> {
> > --
> > 2.55.0.508.g3f0d502094-goog
> >
>
> --
> Cheers, Lorenzo

[PATCH v3 5/5] tcp: Remove mmap_lock fallback path

[Suren Baghdasaryan]

From: Dave Hansen <dave.hansen@linux.intel.com>

Previously, the per-VMA locking could fail in the face of writers
which necessitates a fallback to mmap_lock. The new
lock_vma_under_rcu_wait() will wait for writers instead of failing.

Use the new helper. Wait for writers. Remove the fallback to mmap_lock.

This really is a nice cleanup. It removes the need to pass the lock
state back and forth to find_tcp_vma().

Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Acked-by: Lorenzo Stoakes <ljs@kernel.org>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Tested-by: syzbot@syzkaller.appspotmail.com
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Liam R. Howlett <Liam.Howlett@oracle.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: linux-mm@kvack.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Arve Hjønnevåg <arve@android.com>
Cc: Todd Kjos <tkjos@android.com>
Cc: Christian Brauner <christian@brauner.io>
Cc: Carlos Llamas <cmllamas@google.com>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: David Ahern <dsahern@kernel.org>
Cc: netdev@vger.kernel.org
---
 net/ipv4/tcp.c | 31 +++++++++----------------------
 1 file changed, 9 insertions(+), 22 deletions(-)

diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 455441f1b694..62defe70f3ce 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2167,27 +2167,18 @@ static void tcp_zc_finalize_rx_tstamp(struct sock *sk,
 }
 
 static struct vm_area_struct *find_tcp_vma(struct mm_struct *mm,
-					   unsigned long address,
-					   bool *mmap_locked)
+					   unsigned long address)
 {
-	struct vm_area_struct *vma = lock_vma_under_rcu(mm, address);
+	struct vm_area_struct *vma = vma_start_read_unlocked(mm, address);
 
-	if (vma) {
-		if (vma->vm_ops != &tcp_vm_ops) {
-			vma_end_read(vma);
-			return NULL;
-		}
-		*mmap_locked = false;
-		return vma;
-	}
+	if (!vma)
+		return NULL;
 
-	mmap_read_lock(mm);
-	vma = vma_lookup(mm, address);
-	if (!vma || vma->vm_ops != &tcp_vm_ops) {
-		mmap_read_unlock(mm);
+	if (vma->vm_ops != &tcp_vm_ops) {
+		vma_end_read(vma);
 		return NULL;
 	}
-	*mmap_locked = true;
+
 	return vma;
 }
 
@@ -2208,7 +2199,6 @@ static int tcp_zerocopy_receive(struct sock *sk,
 	u32 seq = tp->copied_seq;
 	u32 total_bytes_to_map;
 	int inq = tcp_inq(sk);
-	bool mmap_locked;
 	int ret;
 
 	zc->copybuf_len = 0;
@@ -2233,7 +2223,7 @@ static int tcp_zerocopy_receive(struct sock *sk,
 		return 0;
 	}
 
-	vma = find_tcp_vma(current->mm, address, &mmap_locked);
+	vma = find_tcp_vma(current->mm, address);
 	if (!vma)
 		return -EINVAL;
 
@@ -2315,10 +2305,7 @@ static int tcp_zerocopy_receive(struct sock *sk,
 						   zc, total_bytes_to_map);
 	}
 out:
-	if (mmap_locked)
-		mmap_read_unlock(current->mm);
-	else
-		vma_end_read(vma);
+	vma_end_read(vma);
 	/* Try to copy straggler data. */
 	if (!ret)
 		copylen = tcp_zc_handle_leftover(zc, sk, skb, &seq, copybuf_len, tss);
-- 
2.55.0.508.g3f0d502094-goog

Re: [PATCH v3 0/5] mm: Unconditional per-VMA locks and cleanups

[Barry Song]

On Mon, Aug 3, 2026 at 5:58 AM Suren Baghdasaryan <surenb@google.com> wrote:
>
> v2 version of this patchset [1] was written by Dave Hansen and per his
> request, I'm taking over this series.
>
> tl;dr: Make per-VMA locks available in all configs. Simplify some
> of the per-VMA lock users now that they can rely on them being
> always available.
>
> Binder and networking folks: Your code is the target of the cleanups.
> I'm cc'ing you now on v2 because there's emerging consensus on the mm
> side that the approach here is sane. I'm not quite sure how this pile
> would get merged, but ack/review tags would be appreciated if this
> looks good to you.
>
> Longer version:
>
> When working on some x86 shadow stack code, it was a real pain to
> avoid causing recursive locking problems with mmap_lock. One way
> to avoid those was to avoid mmap_lock and use per-VMA locks instead.
> They are great, but they are not available in all configs which
> makes them unusable in generic code, or if you want to completely
> avoid mmap_lock.
>
> Make per-VMA locks available in all configs. Right now, they are
> only available on select architectures when SMP and MMU are enabled.
> But all of the primitives that per-VMA locks are built on (RCU, maple
> trees, refcounts) work just fine without SMP or MMU.
>
> The only real downside is that making VMAs a wee bit bigger on !MMU
> and !SMP builds.
>
> The upside is much cleaner code, lower complexity and less #ifdeffery.
>
> Clean up a binder VMA locking site now that it can rely on per-VMA
> locks.
>
> Building on top of universally-available per-VMA locks, introduce a
> new helper. Since the new API does not require callers to have a
> fallback to mmap_lock, it's much easier to use. Callers can
> potentially replace this very common kernel idiom:
>
>         mmap_read_lock(mm);
>         vma = vma_lookup()
>         // fiddle with vma
>         mmap_read_unlock(mm);
>
> with:
>
>         vma = vma_start_read_unlocked(mm, address);
>         // fiddle with vma
>         vma_end_read(vma);
>
> Which avoids mmap_lock entirely in the fast path.
>
> Use that new API for another binder site and one in the TCP code.

Nice, Suren and Dave.

I wonder if we could use the same approach in the page fault
path. Instead of falling back to mmap_lock when
lock_vma_under_rcu() fails the first time, could we wait for the
writer to finish and then retry acquiring the VMA lock?
For example:

diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 85e23388f9bb..684f38cc4e74 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -677,7 +677,7 @@ static int __kprobes do_page_fault(unsigned long
far, unsigned long esr,
        if (!(mm_flags & FAULT_FLAG_USER))
                goto lock_mmap;

-       vma = lock_vma_under_rcu(mm, addr);
+       vma = vma_start_read_unlocked(mm, addr);
        if (!vma)
                goto lock_mmap;

diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 45b99c3b1442..a3a4c4741e30 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -1331,7 +1331,7 @@ void do_user_addr_fault(struct pt_regs *regs,
        if (!(flags & FAULT_FLAG_USER))
                goto lock_mmap;

-       vma = lock_vma_under_rcu(mm, address);
+       vma = vma_start_read_unlocked(mm, address);
        if (!vma)
                goto lock_mmap;

Best Regards
Barry

Re: [PATCH v3 0/5] mm: Unconditional per-VMA locks and cleanups

[Suren Baghdasaryan]

On Sun, Aug 2, 2026 at 7:11 PM Barry Song <baohua@kernel.org> wrote:
>
> On Mon, Aug 3, 2026 at 5:58 AM Suren Baghdasaryan <surenb@google.com> wrote:
> >
> > v2 version of this patchset [1] was written by Dave Hansen and per his
> > request, I'm taking over this series.
> >
> > tl;dr: Make per-VMA locks available in all configs. Simplify some
> > of the per-VMA lock users now that they can rely on them being
> > always available.
> >
> > Binder and networking folks: Your code is the target of the cleanups.
> > I'm cc'ing you now on v2 because there's emerging consensus on the mm
> > side that the approach here is sane. I'm not quite sure how this pile
> > would get merged, but ack/review tags would be appreciated if this
> > looks good to you.
> >
> > Longer version:
> >
> > When working on some x86 shadow stack code, it was a real pain to
> > avoid causing recursive locking problems with mmap_lock. One way
> > to avoid those was to avoid mmap_lock and use per-VMA locks instead.
> > They are great, but they are not available in all configs which
> > makes them unusable in generic code, or if you want to completely
> > avoid mmap_lock.
> >
> > Make per-VMA locks available in all configs. Right now, they are
> > only available on select architectures when SMP and MMU are enabled.
> > But all of the primitives that per-VMA locks are built on (RCU, maple
> > trees, refcounts) work just fine without SMP or MMU.
> >
> > The only real downside is that making VMAs a wee bit bigger on !MMU
> > and !SMP builds.
> >
> > The upside is much cleaner code, lower complexity and less #ifdeffery.
> >
> > Clean up a binder VMA locking site now that it can rely on per-VMA
> > locks.
> >
> > Building on top of universally-available per-VMA locks, introduce a
> > new helper. Since the new API does not require callers to have a
> > fallback to mmap_lock, it's much easier to use. Callers can
> > potentially replace this very common kernel idiom:
> >
> >         mmap_read_lock(mm);
> >         vma = vma_lookup()
> >         // fiddle with vma
> >         mmap_read_unlock(mm);
> >
> > with:
> >
> >         vma = vma_start_read_unlocked(mm, address);
> >         // fiddle with vma
> >         vma_end_read(vma);
> >
> > Which avoids mmap_lock entirely in the fast path.
> >
> > Use that new API for another binder site and one in the TCP code.
>
> Nice, Suren and Dave.
>
> I wonder if we could use the same approach in the page fault
> path. Instead of falling back to mmap_lock when
> lock_vma_under_rcu() fails the first time, could we wait for the
> writer to finish and then retry acquiring the VMA lock?

Yeah, we might be able to do that. Matthew is working on moving common
page-fault handling code into a single arch-independent place. Your
suggested change would be simpler if done after Matthew's refactoring.

> For example:
>
> diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
> index 85e23388f9bb..684f38cc4e74 100644
> --- a/arch/arm64/mm/fault.c
> +++ b/arch/arm64/mm/fault.c
> @@ -677,7 +677,7 @@ static int __kprobes do_page_fault(unsigned long
> far, unsigned long esr,
>         if (!(mm_flags & FAULT_FLAG_USER))
>                 goto lock_mmap;
>
> -       vma = lock_vma_under_rcu(mm, addr);
> +       vma = vma_start_read_unlocked(mm, addr);
>         if (!vma)
>                 goto lock_mmap;
>
> diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> index 45b99c3b1442..a3a4c4741e30 100644
> --- a/arch/x86/mm/fault.c
> +++ b/arch/x86/mm/fault.c
> @@ -1331,7 +1331,7 @@ void do_user_addr_fault(struct pt_regs *regs,
>         if (!(flags & FAULT_FLAG_USER))
>                 goto lock_mmap;
>
> -       vma = lock_vma_under_rcu(mm, address);
> +       vma = vma_start_read_unlocked(mm, address);
>         if (!vma)
>                 goto lock_mmap;
>
> Best Regards
> Barry