[PATCH 6/6] x86/mm: Avoid mmap lock for shadow stack pop fast path

Dave Hansen
2026-04-29T18:20:08Z

From: Dave Hansen <dave.hansen@linux.intel.com>

The shadow stack code needs to look at the VMA from which it is
reading a userspace "token" to ensure that the memory is shadow stack
memory. If it did not do this, it might read the token from
non-shadow-stack memory, which could result in a control flow hijack.

But that lookup requires two things:
* Looking at a VMA, which must be locked
* Touching userspace

That's a bit of a pain because mmap_lock can not be held while
touching userspace. So the code has to drop the lock, touch userspace,
then re-acquire the lock and check if the VMA might have changed.

The current implementation does with a combination of holding
mmap_lock and looping if the VMA might have changed. It works great.
But the lock_vma_under_rcu_wait() API is a little simpler and also
does not use mmap_lock in its fast path.

Switch to lock_vma_under_rcu_wait().

BTW, this does swap in a mmap_read_lock() for
mmap_read_lock_killable(). That obviously isn't ideal, but it's
trivially fixable with another variant of the helper. I'd apprecaite
if we could handwave that away for the moment. :)

Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: linux-mm@kvack.org
---

b/arch/x86/kernel/shstk.c | 47 ++++++++++++++++------------------------------
1 file changed, 17 insertions(+), 30 deletions(-)

diff -puN arch/x86/kernel/shstk.c~shstk-pop-rcu arch/x86/kernel/shstk.c
--- a/arch/x86/kernel/shstk.c~shstk-pop-rcu 2026-04-29 11:18:52.425697858 -0700
+++ b/arch/x86/kernel/shstk.c 2026-04-29 11:18:52.428697973 -0700
@@ -326,8 +326,9 @@ static int shstk_push_sigframe(unsigned

static int shstk_pop_sigframe(unsigned long *ssp)
{
+ struct vm_area_struct *vma;
unsigned long token_addr;
- unsigned int seq;
+ int err;

/*
* It is possible for the SSP to be off the end of a shadow stack by 4
@@ -338,35 +339,21 @@ static int shstk_pop_sigframe(unsigned l
if (!IS_ALIGNED(*ssp, 8))
return -EINVAL;

- do {
- struct vm_area_struct *vma;
- bool valid_vma;
- int err;
-
- if (mmap_read_lock_killable(current->mm))
- return -EINTR;
-
- vma = find_vma(current->mm, *ssp);
- valid_vma = vma && (vma->vm_flags & VM_SHADOW_STACK);
-
- /*
- * VMAs can change between get_shstk_data() and find_vma().
- * Watch for changes and ensure that 'token_addr' comes from
- * 'vma' by recording a seqcount.
- *
- * Ignore the return value of mmap_lock_speculate_try_begin()
- * because the mmap lock excludes the possibility of writers.
- */
- mmap_lock_speculate_try_begin(current->mm, &seq);
- mmap_read_unlock(current->mm);
-
- if (!valid_vma)
- return -EINVAL;
-
- err = get_shstk_data(&token_addr, (unsigned long __user *)*ssp);
- if (err)
- return err;
- } while (mmap_lock_speculate_retry(current->mm, seq));
+ vma = lock_vma_under_rcu_wait(current->mm, *ssp);
+ if (!vma)
+ return -EINVAL;
+
+ if (!(vma->vm_flags & VM_SHADOW_STACK)) {
+ vma_end_read(vma);
+ return -EINVAL;
+ }
+
+ err = get_shstk_data(&token_addr, (unsigned long __user *)*ssp);
+
+ vma_end_read(vma);
+
+ if (err)
+ return err;

/* Restore SSP aligned? */
if (unlikely(!IS_ALIGNED(token_addr, 8)))
_