[PATCH 6/6] x86/mm: Avoid mmap lock for shadow stack pop fast path

[Dave Hansen <dave.hansen@linux.intel.com>]

From: Dave Hansen <dave.hansen@linux.intel.com>

The shadow stack code needs to look at the VMA from which it is
reading a userspace "token" to ensure that the memory is shadow stack
memory. If it did not do this, it might read the token from
non-shadow-stack memory, which could result in a control flow hijack.

But that lookup requires two things:
 * Looking at a VMA, which must be locked
 * Touching userspace

That's a bit of a pain because mmap_lock can not be held while
touching userspace. So the code has to drop the lock, touch userspace,
then re-acquire the lock and check if the VMA might have changed.

The current implementation does with a combination of holding
mmap_lock and looping if the VMA might have changed. It works great.
But the lock_vma_under_rcu_wait() API is a little simpler and also
does not use mmap_lock in its fast path.

Switch to lock_vma_under_rcu_wait().

BTW, this does swap in a mmap_read_lock() for
mmap_read_lock_killable().  That obviously isn't ideal, but it's
trivially fixable with another variant of the helper. I'd apprecaite
if we could handwave that away for the moment. :)

Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: linux-mm@kvack.org
---

 b/arch/x86/kernel/shstk.c |   47 ++++++++++++++++------------------------------
 1 file changed, 17 insertions(+), 30 deletions(-)

diff -puN arch/x86/kernel/shstk.c~shstk-pop-rcu arch/x86/kernel/shstk.c
--- a/arch/x86/kernel/shstk.c~shstk-pop-rcu	2026-04-29 11:18:52.425697858 -0700
+++ b/arch/x86/kernel/shstk.c	2026-04-29 11:18:52.428697973 -0700
@@ -326,8 +326,9 @@ static int shstk_push_sigframe(unsigned
 
 static int shstk_pop_sigframe(unsigned long *ssp)
 {
+	struct vm_area_struct *vma;
 	unsigned long token_addr;
-	unsigned int seq;
+	int err;
 
 	/*
 	 * It is possible for the SSP to be off the end of a shadow stack by 4
@@ -338,35 +339,21 @@ static int shstk_pop_sigframe(unsigned l
 	if (!IS_ALIGNED(*ssp, 8))
 		return -EINVAL;
 
-	do {
-		struct vm_area_struct *vma;
-		bool valid_vma;
-		int err;
-
-		if (mmap_read_lock_killable(current->mm))
-			return -EINTR;
-
-		vma = find_vma(current->mm, *ssp);
-		valid_vma = vma && (vma->vm_flags & VM_SHADOW_STACK);
-
-		/*
-		 * VMAs can change between get_shstk_data() and find_vma().
-		 * Watch for changes and ensure that 'token_addr' comes from
-		 * 'vma' by recording a seqcount.
-		 *
-		 * Ignore the return value of mmap_lock_speculate_try_begin()
-		 * because the mmap lock excludes the possibility of writers.
-		 */
-		mmap_lock_speculate_try_begin(current->mm, &seq);
-		mmap_read_unlock(current->mm);
-
-		if (!valid_vma)
-			return -EINVAL;
-
-		err = get_shstk_data(&token_addr, (unsigned long __user *)*ssp);
-		if (err)
-			return err;
-	} while (mmap_lock_speculate_retry(current->mm, seq));
+	vma = lock_vma_under_rcu_wait(current->mm, *ssp);
+	if (!vma)
+		return -EINVAL;
+
+	if (!(vma->vm_flags & VM_SHADOW_STACK)) {
+		vma_end_read(vma);
+		return -EINVAL;
+	}
+
+	err = get_shstk_data(&token_addr, (unsigned long __user *)*ssp);
+
+	vma_end_read(vma);
+
+	if (err)
+		return err;
 
 	/* Restore SSP aligned? */
 	if (unlikely(!IS_ALIGNED(token_addr, 8)))
_