[PATCH 1/1] arch/mm/fault: fix major fault accounting when retrying under per-VMA lock

[PATCH 1/1] arch/mm/fault: fix major fault accounting when retrying under per-VMA lock

[Suren Baghdasaryan]

A test [1] in Android test suite started failing after [2] was merged.
It turns out that after handling a major fault under per-VMA lock, the
process major fault counter does not register that fault as major.
Before [2] read faults would be done under mmap_lock, in which case
FAULT_FLAG_TRIED flag is set before retrying. That in turn causes
mm_account_fault() to account the fault as major once retry completes.
With per-VMA locks we often retry because a fault can't be handled
without locking the whole mm using mmap_lock. Therefore such retries
do not set FAULT_FLAG_TRIED flag. This logic does not work after [2]
because we can now handle read major faults under per-VMA lock and
upon retry the fact there was a major fault gets lost. Fix this by
setting FAULT_FLAG_TRIED after retrying under per-VMA lock if
VM_FAULT_MAJOR was returned. Ideally we would use an additional
VM_FAULT bit to indicate the reason for the retry (could not handle
under per-VMA lock vs other reason) but this simpler solution seems
to work, so keeping it simple.

[1] https://cs.android.com/android/platform/superproject/+/master:test/vts-testcase/kernel/api/drop_caches_prop/drop_caches_test.cpp
[2] https://lore.kernel.org/all/20231006195318.4087158-6-willy@infradead.org/

Fixes: 12214eba1992 ("mm: handle read faults under the VMA lock")
Cc: Matthew Wilcox <willy@infradead.org>
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
---
 arch/arm64/mm/fault.c   | 2 ++
 arch/powerpc/mm/fault.c | 2 ++
 arch/riscv/mm/fault.c   | 2 ++
 arch/s390/mm/fault.c    | 3 +++
 arch/x86/mm/fault.c     | 2 ++
 5 files changed, 11 insertions(+)

diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 460d799e1296..55f6455a8284 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -607,6 +607,8 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr,
 		goto done;
 	}
 	count_vm_vma_lock_event(VMA_LOCK_RETRY);
+	if (fault & VM_FAULT_MAJOR)
+		mm_flags |= FAULT_FLAG_TRIED;
 
 	/* Quick path to respond to signals */
 	if (fault_signal_pending(fault, regs)) {
diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c
index 9e49ede2bc1c..53335ae21a40 100644
--- a/arch/powerpc/mm/fault.c
+++ b/arch/powerpc/mm/fault.c
@@ -497,6 +497,8 @@ static int ___do_page_fault(struct pt_regs *regs, unsigned long address,
 		goto done;
 	}
 	count_vm_vma_lock_event(VMA_LOCK_RETRY);
+	if (fault & VM_FAULT_MAJOR)
+		flags |= FAULT_FLAG_TRIED;
 
 	if (fault_signal_pending(fault, regs))
 		return user_mode(regs) ? 0 : SIGBUS;
diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c
index 90d4ba36d1d0..081339ddf47e 100644
--- a/arch/riscv/mm/fault.c
+++ b/arch/riscv/mm/fault.c
@@ -304,6 +304,8 @@ void handle_page_fault(struct pt_regs *regs)
 		goto done;
 	}
 	count_vm_vma_lock_event(VMA_LOCK_RETRY);
+	if (fault & VM_FAULT_MAJOR)
+		flags |= FAULT_FLAG_TRIED;
 
 	if (fault_signal_pending(fault, regs)) {
 		if (!user_mode(regs))
diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c
index 249aefcf7c4e..ab4098886e56 100644
--- a/arch/s390/mm/fault.c
+++ b/arch/s390/mm/fault.c
@@ -337,6 +337,9 @@ static void do_exception(struct pt_regs *regs, int access)
 		return;
 	}
 	count_vm_vma_lock_event(VMA_LOCK_RETRY);
+	if (fault & VM_FAULT_MAJOR)
+		flags |= FAULT_FLAG_TRIED;
+
 	/* Quick path to respond to signals */
 	if (fault_signal_pending(fault, regs)) {
 		if (!user_mode(regs))
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index ab778eac1952..679b09cfe241 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -1370,6 +1370,8 @@ void do_user_addr_fault(struct pt_regs *regs,
 		goto done;
 	}
 	count_vm_vma_lock_event(VMA_LOCK_RETRY);
+	if (fault & VM_FAULT_MAJOR)
+		flags |= FAULT_FLAG_TRIED;
 
 	/* Quick path to respond to signals */
 	if (fault_signal_pending(fault, regs)) {
-- 
2.43.0.472.g3155946c3a-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 1/1] arch/mm/fault: fix major fault accounting when retrying under per-VMA lock

[patchwork-bot+linux-riscv]

Hello:

This patch was applied to riscv/linux.git (fixes)
by Andrew Morton <akpm@linux-foundation.org>:

On Tue, 26 Dec 2023 13:46:10 -0800 you wrote:
> A test [1] in Android test suite started failing after [2] was merged.
> It turns out that after handling a major fault under per-VMA lock, the
> process major fault counter does not register that fault as major.
> Before [2] read faults would be done under mmap_lock, in which case
> FAULT_FLAG_TRIED flag is set before retrying. That in turn causes
> mm_account_fault() to account the fault as major once retry completes.
> With per-VMA locks we often retry because a fault can't be handled
> without locking the whole mm using mmap_lock. Therefore such retries
> do not set FAULT_FLAG_TRIED flag. This logic does not work after [2]
> because we can now handle read major faults under per-VMA lock and
> upon retry the fact there was a major fault gets lost. Fix this by
> setting FAULT_FLAG_TRIED after retrying under per-VMA lock if
> VM_FAULT_MAJOR was returned. Ideally we would use an additional
> VM_FAULT bit to indicate the reason for the retry (could not handle
> under per-VMA lock vs other reason) but this simpler solution seems
> to work, so keeping it simple.
> 
> [...]

Here is the summary with links:
  - [1/1] arch/mm/fault: fix major fault accounting when retrying under per-VMA lock
    https://git.kernel.org/riscv/c/46e714c729c8

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 1/1] arch/mm/fault: fix major fault accounting when retrying under per-VMA lock

[Russell King (Oracle)]

On Sat, Jan 20, 2024 at 09:09:47PM +0000, patchwork-bot+linux-riscv@kernel.org wrote:
> Hello:
> 
> This patch was applied to riscv/linux.git (fixes)
> by Andrew Morton <akpm@linux-foundation.org>:
> 
> On Tue, 26 Dec 2023 13:46:10 -0800 you wrote:
> > A test [1] in Android test suite started failing after [2] was merged.
> > It turns out that after handling a major fault under per-VMA lock, the
> > process major fault counter does not register that fault as major.
> > Before [2] read faults would be done under mmap_lock, in which case
> > FAULT_FLAG_TRIED flag is set before retrying. That in turn causes
> > mm_account_fault() to account the fault as major once retry completes.
> > With per-VMA locks we often retry because a fault can't be handled
> > without locking the whole mm using mmap_lock. Therefore such retries
> > do not set FAULT_FLAG_TRIED flag. This logic does not work after [2]
> > because we can now handle read major faults under per-VMA lock and
> > upon retry the fact there was a major fault gets lost. Fix this by
> > setting FAULT_FLAG_TRIED after retrying under per-VMA lock if
> > VM_FAULT_MAJOR was returned. Ideally we would use an additional
> > VM_FAULT bit to indicate the reason for the retry (could not handle
> > under per-VMA lock vs other reason) but this simpler solution seems
> > to work, so keeping it simple.
> > 
> > [...]
> 
> Here is the summary with links:
>   - [1/1] arch/mm/fault: fix major fault accounting when retrying under per-VMA lock
>     https://git.kernel.org/riscv/c/46e714c729c8
> 
> You are awesome, thank you!

Now that 32-bit ARM has support for the per-VMA lock, does that also
need to be patched?

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 1/1] arch/mm/fault: fix major fault accounting when retrying under per-VMA lock

[Suren Baghdasaryan]

On Sat, Jan 20, 2024 at 1:15 PM Russell King (Oracle)
<linux@armlinux.org.uk> wrote:
>
> On Sat, Jan 20, 2024 at 09:09:47PM +0000, patchwork-bot+linux-riscv@kernel.org wrote:
> > Hello:
> >
> > This patch was applied to riscv/linux.git (fixes)
> > by Andrew Morton <akpm@linux-foundation.org>:
> >
> > On Tue, 26 Dec 2023 13:46:10 -0800 you wrote:
> > > A test [1] in Android test suite started failing after [2] was merged.
> > > It turns out that after handling a major fault under per-VMA lock, the
> > > process major fault counter does not register that fault as major.
> > > Before [2] read faults would be done under mmap_lock, in which case
> > > FAULT_FLAG_TRIED flag is set before retrying. That in turn causes
> > > mm_account_fault() to account the fault as major once retry completes.
> > > With per-VMA locks we often retry because a fault can't be handled
> > > without locking the whole mm using mmap_lock. Therefore such retries
> > > do not set FAULT_FLAG_TRIED flag. This logic does not work after [2]
> > > because we can now handle read major faults under per-VMA lock and
> > > upon retry the fact there was a major fault gets lost. Fix this by
> > > setting FAULT_FLAG_TRIED after retrying under per-VMA lock if
> > > VM_FAULT_MAJOR was returned. Ideally we would use an additional
> > > VM_FAULT bit to indicate the reason for the retry (could not handle
> > > under per-VMA lock vs other reason) but this simpler solution seems
> > > to work, so keeping it simple.
> > >
> > > [...]
> >
> > Here is the summary with links:
> >   - [1/1] arch/mm/fault: fix major fault accounting when retrying under per-VMA lock
> >     https://git.kernel.org/riscv/c/46e714c729c8
> >
> > You are awesome, thank you!
>
> Now that 32-bit ARM has support for the per-VMA lock, does that also
> need to be patched?

Yes, I think so. I missed the ARM32 change that added support for
per-VMA locks. Will post a similar patch for it tomorrow.
Thanks,
Suren.

>
> --
> RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
> FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 1/1] arch/mm/fault: fix major fault accounting when retrying under per-VMA lock

[Suren Baghdasaryan]

On Sun, Jan 21, 2024 at 11:38 PM Suren Baghdasaryan <surenb@google.com> wrote:
>
> On Sat, Jan 20, 2024 at 1:15 PM Russell King (Oracle)
> <linux@armlinux.org.uk> wrote:
> >
> > On Sat, Jan 20, 2024 at 09:09:47PM +0000, patchwork-bot+linux-riscv@kernel.org wrote:
> > > Hello:
> > >
> > > This patch was applied to riscv/linux.git (fixes)
> > > by Andrew Morton <akpm@linux-foundation.org>:
> > >
> > > On Tue, 26 Dec 2023 13:46:10 -0800 you wrote:
> > > > A test [1] in Android test suite started failing after [2] was merged.
> > > > It turns out that after handling a major fault under per-VMA lock, the
> > > > process major fault counter does not register that fault as major.
> > > > Before [2] read faults would be done under mmap_lock, in which case
> > > > FAULT_FLAG_TRIED flag is set before retrying. That in turn causes
> > > > mm_account_fault() to account the fault as major once retry completes.
> > > > With per-VMA locks we often retry because a fault can't be handled
> > > > without locking the whole mm using mmap_lock. Therefore such retries
> > > > do not set FAULT_FLAG_TRIED flag. This logic does not work after [2]
> > > > because we can now handle read major faults under per-VMA lock and
> > > > upon retry the fact there was a major fault gets lost. Fix this by
> > > > setting FAULT_FLAG_TRIED after retrying under per-VMA lock if
> > > > VM_FAULT_MAJOR was returned. Ideally we would use an additional
> > > > VM_FAULT bit to indicate the reason for the retry (could not handle
> > > > under per-VMA lock vs other reason) but this simpler solution seems
> > > > to work, so keeping it simple.
> > > >
> > > > [...]
> > >
> > > Here is the summary with links:
> > >   - [1/1] arch/mm/fault: fix major fault accounting when retrying under per-VMA lock
> > >     https://git.kernel.org/riscv/c/46e714c729c8
> > >
> > > You are awesome, thank you!
> >
> > Now that 32-bit ARM has support for the per-VMA lock, does that also
> > need to be patched?
>
> Yes, I think so. I missed the ARM32 change that added support for
> per-VMA locks. Will post a similar patch for it tomorrow.

Fix for ARM posted at
https://lore.kernel.org/all/20240123064305.2829244-1-surenb@google.com/

> Thanks,
> Suren.
>
> >
> > --
> > RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
> > FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel