From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D60A413222 for ; Wed, 29 Apr 2026 18:20:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777486809; cv=none; b=KLkpipDKBLEaW2fyVMDggImXBTY7GWN80mA2BTCWdW3k0Xbuos9hUnMC06VO/BSHiAkvA1WuH3CQdhLCgtgvQd6EwYcEyf4WFCqBVSv1qfzvox7d2pcphPZZHK1lw8mvP4Pg6zUzXC9omlOTkz7PjyNWABaqhX5aSZt4eQXJYyI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777486809; c=relaxed/simple; bh=7dFDPzrIAbzJLwmxK2im4erAqAsRrcXN0YYEP3dJALk=; h=Subject:To:Cc:From:Date:References:In-Reply-To:Message-Id; b=P0i40NAtJLbOa638XN51lCLIZqafohpIEyb9F75wUwSW/tH4xntzIqZn82UkEZtx1AYNPsuLELkH//5DdWB2GDHyYdTJfrs337s2cBctJP4NsKt40dag0z5CvD0Y3+22pAmpMgW6V9EIUUoh0OJy6m+OTsrEGlkgUlHFozqnO/Q= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=YyNZs0HM; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="YyNZs0HM" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1777486808; x=1809022808; h=subject:to:cc:from:date:references:in-reply-to: message-id; bh=7dFDPzrIAbzJLwmxK2im4erAqAsRrcXN0YYEP3dJALk=; b=YyNZs0HM21EEYkYUPCRf8OjfRexZYzFg2ojbgR/dxQPRcbbANZ7zdMf1 yg4fUhK0W1Sm7rX8LW3YSIRi6DyvVyLZQ/IIO7xG02NGRHu7gI/pB0QSZ MuFDtfw4MPI9Peaz3+tGN4jxn+buO6Y8JhxBI8XisWGuilwB3QpAKmZxw ABAiwFfUFZW1Oat+99xWuEKC209BRXXRSG+bBeLKPzv9xqTke0reVj9T2 NgDNmroInbAnoFZrjRyMRT+tGkZODbcV2pjyXR/H/GcpKeiyupOHh/akV 1kULNJ1Fi87HKQwIj49kOiU2p7htjjuX0+gWIGhpucMQDDyD2mgT/EvDA g==; X-CSE-ConnectionGUID: RW7OSy2RT+aIp1aJjK6kWg== X-CSE-MsgGUID: 9/8lzUKCRCKmmrJsS7EHDw== X-IronPort-AV: E=McAfee;i="6800,10657,11771"; a="95990141" X-IronPort-AV: E=Sophos;i="6.23,206,1770624000"; d="scan'208";a="95990141" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Apr 2026 11:20:06 -0700 X-CSE-ConnectionGUID: PD/Hu5SWQpmU0LRPzM05AQ== X-CSE-MsgGUID: xbkb5ieTSnKs2CexBURa/Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,206,1770624000"; d="scan'208";a="239336473" Received: from davehans-spike.ostc.intel.com (HELO localhost.localdomain) ([10.165.164.11]) by fmviesa005.fm.intel.com with ESMTP; 29 Apr 2026 11:20:06 -0700 Subject: [PATCH 6/6] x86/mm: Avoid mmap lock for shadow stack pop fast path To: linux-kernel@vger.kernel.org Cc: Dave Hansen , Andrew Morton , "Liam R. Howlett" , linux-mm@kvack.org, Lorenzo Stoakes , Shakeel Butt , Suren Baghdasaryan , Vlastimil Babka From: Dave Hansen Date: Wed, 29 Apr 2026 11:20:05 -0700 References: <20260429181954.F50224AE@davehans-spike.ostc.intel.com> In-Reply-To: <20260429181954.F50224AE@davehans-spike.ostc.intel.com> Message-Id: <20260429182005.00BF70D8@davehans-spike.ostc.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Dave Hansen The shadow stack code needs to look at the VMA from which it is reading a userspace "token" to ensure that the memory is shadow stack memory. If it did not do this, it might read the token from non-shadow-stack memory, which could result in a control flow hijack. But that lookup requires two things: * Looking at a VMA, which must be locked * Touching userspace That's a bit of a pain because mmap_lock can not be held while touching userspace. So the code has to drop the lock, touch userspace, then re-acquire the lock and check if the VMA might have changed. The current implementation does with a combination of holding mmap_lock and looping if the VMA might have changed. It works great. But the lock_vma_under_rcu_wait() API is a little simpler and also does not use mmap_lock in its fast path. Switch to lock_vma_under_rcu_wait(). BTW, this does swap in a mmap_read_lock() for mmap_read_lock_killable(). That obviously isn't ideal, but it's trivially fixable with another variant of the helper. I'd apprecaite if we could handwave that away for the moment. :) Signed-off-by: Dave Hansen Cc: Suren Baghdasaryan Cc: Andrew Morton Cc: "Liam R. Howlett" Cc: Lorenzo Stoakes Cc: Vlastimil Babka Cc: Shakeel Butt Cc: linux-mm@kvack.org --- b/arch/x86/kernel/shstk.c | 47 ++++++++++++++++------------------------------ 1 file changed, 17 insertions(+), 30 deletions(-) diff -puN arch/x86/kernel/shstk.c~shstk-pop-rcu arch/x86/kernel/shstk.c --- a/arch/x86/kernel/shstk.c~shstk-pop-rcu 2026-04-29 11:18:52.425697858 -0700 +++ b/arch/x86/kernel/shstk.c 2026-04-29 11:18:52.428697973 -0700 @@ -326,8 +326,9 @@ static int shstk_push_sigframe(unsigned static int shstk_pop_sigframe(unsigned long *ssp) { + struct vm_area_struct *vma; unsigned long token_addr; - unsigned int seq; + int err; /* * It is possible for the SSP to be off the end of a shadow stack by 4 @@ -338,35 +339,21 @@ static int shstk_pop_sigframe(unsigned l if (!IS_ALIGNED(*ssp, 8)) return -EINVAL; - do { - struct vm_area_struct *vma; - bool valid_vma; - int err; - - if (mmap_read_lock_killable(current->mm)) - return -EINTR; - - vma = find_vma(current->mm, *ssp); - valid_vma = vma && (vma->vm_flags & VM_SHADOW_STACK); - - /* - * VMAs can change between get_shstk_data() and find_vma(). - * Watch for changes and ensure that 'token_addr' comes from - * 'vma' by recording a seqcount. - * - * Ignore the return value of mmap_lock_speculate_try_begin() - * because the mmap lock excludes the possibility of writers. - */ - mmap_lock_speculate_try_begin(current->mm, &seq); - mmap_read_unlock(current->mm); - - if (!valid_vma) - return -EINVAL; - - err = get_shstk_data(&token_addr, (unsigned long __user *)*ssp); - if (err) - return err; - } while (mmap_lock_speculate_retry(current->mm, seq)); + vma = lock_vma_under_rcu_wait(current->mm, *ssp); + if (!vma) + return -EINVAL; + + if (!(vma->vm_flags & VM_SHADOW_STACK)) { + vma_end_read(vma); + return -EINVAL; + } + + err = get_shstk_data(&token_addr, (unsigned long __user *)*ssp); + + vma_end_read(vma); + + if (err) + return err; /* Restore SSP aligned? */ if (unlikely(!IS_ALIGNED(token_addr, 8))) _