[RFC PATCH v1 0/2] mm: use VMA lock for kernel faults on user addresses

[RFC PATCH v1 0/2] mm: use VMA lock for kernel faults on user addresses

[Barry Song (Xiaomi)]

Right now, kernel faults on user addresses, such as those from
copy_from_user() and copy_to_user(), unconditionally fall back
to the mmap_lock path.

This patchset switches them to the per-VMA lock path for three
reasons:

1. These faults are common. On a typical Ubuntu system, hundreds
   of kernel faults on user addresses occur every second in
   python3, apt-esm-hook, package-data-do, teamviewerd, bash,
   scudo, cscope, gnome-shell, systemd, and many other
   applications. Handling these faults under mmap_lock
   unnecessarily increases lock contention.

2. It removes one obstacle to simplifying filemap_fault().
   Matthew has proposed removing the page fault retry path and
   performing I/O while holding locks [1]. Based on that
   approach, Hongru has already reported regressions caused by
   performing I/O under mmap_lock for kernel faults on user
   addresses [2]. This patchset removes that obstacle.

   Another source of I/O under mmap_lock is GUP, which also
   relies on mmap_lock today and has been reported by Hongru to
   exhibit similar regressions [2]. It appears Rik van Riel may
   be addressing this separately [3].

3. The current implementation is inconsistent. Although kernel
   faults on user addresses always fall back to mmap_lock,
   arch/*/mm/fault.c still performs !user_mode checks in the
   per-VMA lock path, making that code effectively dead.

As an RFC, this patchset demonstrates the approach on x86 and
arm64 only. Other architectures will need to be updated as well.

[1] https://lore.kernel.org/linux-mm/20260625195040.2508362-1-willy@infradead.org/
[2] https://lore.kernel.org/linux-mm/20260712132759.2030823-1-zhanghongru@xiaomi.com/
[3] https://lore.kernel.org/linux-mm/20260724222934.1463812-1-riel@surriel.com/

Barry Song (Xiaomi) (2):
  x86/mm: use VMA lock for kernel faults on user addresses
  arm64/mm: use VMA lock for kernel faults on user addresses

 arch/arm64/mm/fault.c | 4 +++-
 arch/x86/mm/fault.c   | 3 ---
 2 files changed, 3 insertions(+), 4 deletions(-)

-- 
2.39.3 (Apple Git-146)

[RFC PATCH v1 1/2] x86/mm: use VMA lock for kernel faults on user addresses

[Barry Song (Xiaomi)]

Use the VMA lock for kernel faults on user addresses. This also
makes the existing code below meaningful:

  /* Quick path to respond to signals */
  if (fault_signal_pending(fault, regs)) {
          if (!user_mode(regs))
                  kernelmode_fixup_or_oops(regs, error_code, address,
                                           SIGBUS, BUS_ADRERR,
                                           ARCH_DEFAULT_PKEY);
          return;
  }

Right now, the code above is dead because !user_mode always
takes the mmap_lock path.

Co-developed-by: Bo Zhang <zhangbo56@xiaomi.com>
Signed-off-by: Bo Zhang <zhangbo56@xiaomi.com>
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
 arch/x86/mm/fault.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 45b99c3b1442..c22b74e0eeaf 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -1328,9 +1328,6 @@ void do_user_addr_fault(struct pt_regs *regs,
 	}
 #endif
 
-	if (!(flags & FAULT_FLAG_USER))
-		goto lock_mmap;
-
 	vma = lock_vma_under_rcu(mm, address);
 	if (!vma)
 		goto lock_mmap;
-- 
2.39.3 (Apple Git-146)

[RFC PATCH v1 2/2] arm64/mm: use VMA lock for kernel faults on user addresses

[Barry Song (Xiaomi)]

Use the VMA lock for kernel faults on user addresses. This also
makes the existing code below meaningful:

/* Quick path to respond to signals */
if (fault_signal_pending(fault, regs)) {
	if (!user_mode(regs))
		goto no_context;
	return 0;
}

Right now, the code above is dead because !user_mode always
takes the mmap_lock path.

Co-developed-by: Bo Zhang <zhangbo56@xiaomi.com>
Signed-off-by: Bo Zhang <zhangbo56@xiaomi.com>
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
 arch/arm64/mm/fault.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 85e23388f9bb..241f1ab07ab3 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -607,6 +607,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr,
 	unsigned int mm_flags = FAULT_FLAG_DEFAULT;
 	unsigned long addr = untagged_addr(far);
 	struct vm_area_struct *vma;
+	bool uaccess = false;
 	int si_code;
 	int pkey = -1;
 
@@ -663,6 +664,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr,
 		if (!insn_may_access_user(regs->pc, esr))
 			die_kernel_fault("access to user memory outside uaccess routines",
 					 addr, esr, regs);
+		uaccess = true;
 	}
 
 	if (is_pkvm_stage2_abort(esr)) {
@@ -674,7 +676,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr,
 
 	perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, addr);
 
-	if (!(mm_flags & FAULT_FLAG_USER))
+	if (!(mm_flags & FAULT_FLAG_USER) && !uaccess)
 		goto lock_mmap;
 
 	vma = lock_vma_under_rcu(mm, addr);
-- 
2.39.3 (Apple Git-146)

Re: [RFC PATCH v1 2/2] arm64/mm: use VMA lock for kernel faults on user addresses

[Barry Song]

On Sun, Aug 2, 2026 at 3:40 PM Barry Song (Xiaomi) <baohua@kernel.org> wrote:
>
> Use the VMA lock for kernel faults on user addresses. This also
> makes the existing code below meaningful:
>
> /* Quick path to respond to signals */
> if (fault_signal_pending(fault, regs)) {
>         if (!user_mode(regs))
>                 goto no_context;
>         return 0;
> }
>
> Right now, the code above is dead because !user_mode always
> takes the mmap_lock path.
>
> Co-developed-by: Bo Zhang <zhangbo56@xiaomi.com>
> Signed-off-by: Bo Zhang <zhangbo56@xiaomi.com>
> Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
> ---
>  arch/arm64/mm/fault.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
> index 85e23388f9bb..241f1ab07ab3 100644
> --- a/arch/arm64/mm/fault.c
> +++ b/arch/arm64/mm/fault.c
> @@ -607,6 +607,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr,
>         unsigned int mm_flags = FAULT_FLAG_DEFAULT;
>         unsigned long addr = untagged_addr(far);
>         struct vm_area_struct *vma;
> +       bool uaccess = false;
>         int si_code;
>         int pkey = -1;
>
> @@ -663,6 +664,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr,
>                 if (!insn_may_access_user(regs->pc, esr))
>                         die_kernel_fault("access to user memory outside uaccess routines",
>                                          addr, esr, regs);
> +               uaccess = true;

https://sashiko.dev/#/patchset/20260802074018.73887-1-baohua%40kernel.org

"Does this conditional bypass translation faults? Because this block is
guarded by:
if (is_ttbr0_addr(addr) && is_el1_permission_fault(addr, esr, regs)) {
it appears uaccess is only set for permission faults like CoW or PAN
violations.
If a kernel uaccess routine accesses an unmapped user address, a translation
fault occurs, making is_el1_permission_fault() evaluate to false.
Would this cause demand paging in uaccess routines to fall back to the slow
lock_mmap path, missing the intended optimization?"

Good catch! I should have only modified a single line. Then
unmapped PTEs would also benefit from the VMA lock:

diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 85e23388f9bb..e1b406d667aa 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -674,7 +674,7 @@ static int __kprobes do_page_fault(unsigned long
far, unsigned long esr,

        perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, addr);

-       if (!(mm_flags & FAULT_FLAG_USER))
+       if (!(mm_flags & FAULT_FLAG_USER) && !is_ttbr0_addr(addr))
                goto lock_mmap;

        vma = lock_vma_under_rcu(mm, addr);

Re: [RFC PATCH v1 1/2] x86/mm: use VMA lock for kernel faults on user addresses

[Lorenzo Stoakes (ARM)]

On Sun, Aug 02, 2026 at 03:40:17PM +0800, Barry Song (Xiaomi) wrote:
> Use the VMA lock for kernel faults on user addresses. This also
> makes the existing code below meaningful:
>
>   /* Quick path to respond to signals */
>   if (fault_signal_pending(fault, regs)) {
>           if (!user_mode(regs))
>                   kernelmode_fixup_or_oops(regs, error_code, address,
>                                            SIGBUS, BUS_ADRERR,
>                                            ARCH_DEFAULT_PKEY);
>           return;
>   }

Hmm yeah :)

Bit weird it uses user_mode(regs) and the early exit uses the just-set
'flags & FAULT_FLAG_USER' too.

Some horrible duplication here too... the user_mod_regs() etc. code is just
duplicated in the mmap lock path.

I think you mentioned it on Suren/Dave's series but vma_start_read_unlocked()
would avoid all this and could lead to a nicely red patch.

>
> Right now, the code above is dead because !user_mode always
> takes the mmap_lock path.
>
> Co-developed-by: Bo Zhang <zhangbo56@xiaomi.com>
> Signed-off-by: Bo Zhang <zhangbo56@xiaomi.com>
> Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
> ---
>  arch/x86/mm/fault.c | 3 ---
>  1 file changed, 3 deletions(-)
>
> diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> index 45b99c3b1442..c22b74e0eeaf 100644
> --- a/arch/x86/mm/fault.c
> +++ b/arch/x86/mm/fault.c
> @@ -1328,9 +1328,6 @@ void do_user_addr_fault(struct pt_regs *regs,
>  	}
>  #endif
>
> -	if (!(flags & FAULT_FLAG_USER))
> -		goto lock_mmap;
> -

I'm not sure why kernel faults of userland memory were excluded initially
(Suren?)

Given that it's OK to take the mmap lock here and we're necessarily in process
context anyway surely it's OK to take the VMA lock?

It looks fine to me but want Suren's input.

Also given vma_start_read_unlocked() is coming maybe that's better for a
cleanup.

And finally - Willy is working on a grand clean up of this stuff _I think_ so
you probably should coordinate with him also?

>  	vma = lock_vma_under_rcu(mm, address);
>  	if (!vma)
>  		goto lock_mmap;
> --
> 2.39.3 (Apple Git-146)
>

--
Cheers, Lorenzo

Re: [RFC PATCH v1 2/2] arm64/mm: use VMA lock for kernel faults on user addresses

[Lorenzo Stoakes (ARM)]

On Sun, Aug 02, 2026 at 04:49:04PM +0800, Barry Song wrote:
> On Sun, Aug 2, 2026 at 3:40 PM Barry Song (Xiaomi) <baohua@kernel.org> wrote:
> >
> > Use the VMA lock for kernel faults on user addresses. This also
> > makes the existing code below meaningful:
> >
> > /* Quick path to respond to signals */
> > if (fault_signal_pending(fault, regs)) {
> >         if (!user_mode(regs))
> >                 goto no_context;
> >         return 0;
> > }
> >
> > Right now, the code above is dead because !user_mode always
> > takes the mmap_lock path.
> >
> > Co-developed-by: Bo Zhang <zhangbo56@xiaomi.com>
> > Signed-off-by: Bo Zhang <zhangbo56@xiaomi.com>
> > Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
> > ---
> >  arch/arm64/mm/fault.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
> > index 85e23388f9bb..241f1ab07ab3 100644
> > --- a/arch/arm64/mm/fault.c
> > +++ b/arch/arm64/mm/fault.c
> > @@ -607,6 +607,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr,
> >         unsigned int mm_flags = FAULT_FLAG_DEFAULT;
> >         unsigned long addr = untagged_addr(far);
> >         struct vm_area_struct *vma;
> > +       bool uaccess = false;
> >         int si_code;
> >         int pkey = -1;
> >
> > @@ -663,6 +664,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr,
> >                 if (!insn_may_access_user(regs->pc, esr))
> >                         die_kernel_fault("access to user memory outside uaccess routines",
> >                                          addr, esr, regs);
> > +               uaccess = true;
>
> https://sashiko.dev/#/patchset/20260802074018.73887-1-baohua%40kernel.org
>
> "Does this conditional bypass translation faults? Because this block is
> guarded by:
> if (is_ttbr0_addr(addr) && is_el1_permission_fault(addr, esr, regs)) {
> it appears uaccess is only set for permission faults like CoW or PAN
> violations.
> If a kernel uaccess routine accesses an unmapped user address, a translation
> fault occurs, making is_el1_permission_fault() evaluate to false.
> Would this cause demand paging in uaccess routines to fall back to the slow
> lock_mmap path, missing the intended optimization?"
>
> Good catch! I should have only modified a single line. Then
> unmapped PTEs would also benefit from the VMA lock:
>
> diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
> index 85e23388f9bb..e1b406d667aa 100644
> --- a/arch/arm64/mm/fault.c
> +++ b/arch/arm64/mm/fault.c
> @@ -674,7 +674,7 @@ static int __kprobes do_page_fault(unsigned long
> far, unsigned long esr,
>
>         perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, addr);
>
> -       if (!(mm_flags & FAULT_FLAG_USER))
> +       if (!(mm_flags & FAULT_FLAG_USER) && !is_ttbr0_addr(addr))
>                 goto lock_mmap;

I mean honestly it feels like, with the permanent VMA flags change, that maybe
this really needs to wait for Matthew's series?

There's duplication here too of course :)

But it begs the question of every other arch that does VMA page faulting.

So feels better suited as part of that change I think?

Maybe Matthew has thoughts.

>
>         vma = lock_vma_under_rcu(mm, addr);

--
Cheers, Lorenzo

Re: [RFC PATCH v1 1/2] x86/mm: use VMA lock for kernel faults on user addresses

[Suren Baghdasaryan]

On Mon, Aug 3, 2026 at 8:18 AM Lorenzo Stoakes (ARM) <ljs@kernel.org> wrote:
>
> On Sun, Aug 02, 2026 at 03:40:17PM +0800, Barry Song (Xiaomi) wrote:
> > Use the VMA lock for kernel faults on user addresses. This also
> > makes the existing code below meaningful:
> >
> >   /* Quick path to respond to signals */
> >   if (fault_signal_pending(fault, regs)) {
> >           if (!user_mode(regs))
> >                   kernelmode_fixup_or_oops(regs, error_code, address,
> >                                            SIGBUS, BUS_ADRERR,
> >                                            ARCH_DEFAULT_PKEY);
> >           return;
> >   }
>
> Hmm yeah :)
>
> Bit weird it uses user_mode(regs) and the early exit uses the just-set
> 'flags & FAULT_FLAG_USER' too.
>
> Some horrible duplication here too... the user_mod_regs() etc. code is just
> duplicated in the mmap lock path.
>
> I think you mentioned it on Suren/Dave's series but vma_start_read_unlocked()
> would avoid all this and could lead to a nicely red patch.
>
> >
> > Right now, the code above is dead because !user_mode always
> > takes the mmap_lock path.
> >
> > Co-developed-by: Bo Zhang <zhangbo56@xiaomi.com>
> > Signed-off-by: Bo Zhang <zhangbo56@xiaomi.com>
> > Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
> > ---
> >  arch/x86/mm/fault.c | 3 ---
> >  1 file changed, 3 deletions(-)
> >
> > diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> > index 45b99c3b1442..c22b74e0eeaf 100644
> > --- a/arch/x86/mm/fault.c
> > +++ b/arch/x86/mm/fault.c
> > @@ -1328,9 +1328,6 @@ void do_user_addr_fault(struct pt_regs *regs,
> >       }
> >  #endif
> >
> > -     if (!(flags & FAULT_FLAG_USER))
> > -             goto lock_mmap;
> > -
>
> I'm not sure why kernel faults of userland memory were excluded initially
> (Suren?)
>
> Given that it's OK to take the mmap lock here and we're necessarily in process
> context anyway surely it's OK to take the VMA lock?
>
> It looks fine to me but want Suren's input.

Yeah, I don't recall the specific reason it was excluded and there was
no discussion about it. Probably wanted to limit the initial series to
the most common cases and avoid any side-effects. But indeed, if it's
safe to take mmap_lock then it should be safe to take VMA lock.

>
> Also given vma_start_read_unlocked() is coming maybe that's better for a
> cleanup.
>
> And finally - Willy is working on a grand clean up of this stuff _I think_ so
> you probably should coordinate with him also?
>
> >       vma = lock_vma_under_rcu(mm, address);
> >       if (!vma)
> >               goto lock_mmap;
> > --
> > 2.39.3 (Apple Git-146)
> >
>
> --
> Cheers, Lorenzo

Re: [RFC PATCH v1 1/2] x86/mm: use VMA lock for kernel faults on user addresses

[Barry Song]

On Mon, Aug 3, 2026 at 11:18 PM Lorenzo Stoakes (ARM) <ljs@kernel.org> wrote:
>
> On Sun, Aug 02, 2026 at 03:40:17PM +0800, Barry Song (Xiaomi) wrote:
> > Use the VMA lock for kernel faults on user addresses. This also
> > makes the existing code below meaningful:
> >
> >   /* Quick path to respond to signals */
> >   if (fault_signal_pending(fault, regs)) {
> >           if (!user_mode(regs))
> >                   kernelmode_fixup_or_oops(regs, error_code, address,
> >                                            SIGBUS, BUS_ADRERR,
> >                                            ARCH_DEFAULT_PKEY);
> >           return;
> >   }
>
> Hmm yeah :)
>
> Bit weird it uses user_mode(regs) and the early exit uses the just-set
> 'flags & FAULT_FLAG_USER' too.
>
> Some horrible duplication here too... the user_mod_regs() etc. code is just
> duplicated in the mmap lock path.
>
> I think you mentioned it on Suren/Dave's series but vma_start_read_unlocked()
> would avoid all this and could lead to a nicely red patch.

vma_start_read_unlocked() could completely avoid falling back to
mmap_lock for binder. For page faults, however, there are still
paths that must retry under mmap_lock, such as
__vmf_anon_prepare() and swapcache_is_device_private().
So we may not end up with something as clean as binder, but I
think we can still achieve a cleaner design than what we have
today.

>
> >
> > Right now, the code above is dead because !user_mode always
> > takes the mmap_lock path.
> >
> > Co-developed-by: Bo Zhang <zhangbo56@xiaomi.com>
> > Signed-off-by: Bo Zhang <zhangbo56@xiaomi.com>
> > Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
> > ---
> >  arch/x86/mm/fault.c | 3 ---
> >  1 file changed, 3 deletions(-)
> >
> > diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> > index 45b99c3b1442..c22b74e0eeaf 100644
> > --- a/arch/x86/mm/fault.c
> > +++ b/arch/x86/mm/fault.c
> > @@ -1328,9 +1328,6 @@ void do_user_addr_fault(struct pt_regs *regs,
> >       }
> >  #endif
> >
> > -     if (!(flags & FAULT_FLAG_USER))
> > -             goto lock_mmap;
> > -
>
> I'm not sure why kernel faults of userland memory were excluded initially
> (Suren?)
>
> Given that it's OK to take the mmap lock here and we're necessarily in process
> context anyway surely it's OK to take the VMA lock?
>
> It looks fine to me but want Suren's input.
>
> Also given vma_start_read_unlocked() is coming maybe that's better for a
> cleanup.
>
> And finally - Willy is working on a grand clean up of this stuff _I think_ so
> you probably should coordinate with him also?

I have no problem waiting for Matthew's work. It is the right
thing to refactor the duplicated code across architectures for
the VMA lock page fault path.

Thanks
Barry