From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3589BC4338F for ; Fri, 20 Aug 2021 18:19:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D1C3C61209 for ; Fri, 20 Aug 2021 18:19:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D1C3C61209 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 37E3E8D0011; Fri, 20 Aug 2021 14:18:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 13C418D0016; Fri, 20 Aug 2021 14:18:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DC3B68D0011; Fri, 20 Aug 2021 14:18:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0229.hostedemail.com [216.40.44.229]) by kanga.kvack.org (Postfix) with ESMTP id 909118D0014 for ; Fri, 20 Aug 2021 14:18:55 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 3CFFC18041D14 for ; Fri, 20 Aug 2021 18:18:55 +0000 (UTC) X-FDA: 78496270230.11.441DCBF Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf27.hostedemail.com (Postfix) with ESMTP id 8D09A70000A9 for ; Fri, 20 Aug 2021 18:18:54 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6200,9189,10082"; a="197075226" X-IronPort-AV: E=Sophos;i="5.84,338,1620716400"; d="scan'208";a="197075226" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Aug 2021 11:18:51 -0700 X-IronPort-AV: E=Sophos;i="5.84,338,1620716400"; d="scan'208";a="533074772" Received: from yyu32-desk.sc.intel.com ([143.183.136.146]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Aug 2021 11:18:51 -0700 From: Yu-cheng Yu To: x86@kernel.org, "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H.J. Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Dave Martin , Weijiang Yang , Pengfei Xu , Haitao Huang , Rick P Edgecombe Cc: Yu-cheng Yu Subject: [PATCH v29 26/32] x86/cet/shstk: Introduce shadow stack token setup/verify routines Date: Fri, 20 Aug 2021 11:11:55 -0700 Message-Id: <20210820181201.31490-27-yu-cheng.yu@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20210820181201.31490-1-yu-cheng.yu@intel.com> References: <20210820181201.31490-1-yu-cheng.yu@intel.com> MIME-Version: 1.0 Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none); spf=none (imf27.hostedemail.com: domain of yu-cheng.yu@intel.com has no SPF policy when checking 192.55.52.151) smtp.mailfrom=yu-cheng.yu@intel.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 8D09A70000A9 X-Stat-Signature: 8rpsj8hr3nqaa3zsryz7thbg6qpw35gt X-HE-Tag: 1629483534-858256 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A shadow stack restore token marks a restore point of the shadow stack, a= nd the address in a token must point directly above the token, which is with= in the same shadow stack. This is distinctively different from other pointe= rs on the shadow stack, since those pointers point to executable code area. The restore token can be used as an extra protection for signal handling. To deliver a signal, create a shadow stack restore token and put the toke= n and the signal restorer address on the shadow stack. In sigreturn, verif= y the token and restore from it the shadow stack pointer. Introduce token setup and verify routines. Also introduce WRUSS, which i= s a kernel-mode instruction but writes directly to user shadow stack. It i= s used to construct user signal stack as described above. Signed-off-by: Yu-cheng Yu Cc: Kees Cook --- v29: - Update comments for the use of get_xsave_addr(). v28: - Add comments for get_xsave_addr(). v27: - For shstk_check_rstor_token(), instead of an input param, use current shadow stack pointer. - In response to comments, fix/simplify a few syntax/format issues. v25: - Update inline assembly syntax, use %[]. - Change token address from (unsigned long) to (u64/u32 __user *). - Change -EPERM to -EFAULT. --- arch/x86/include/asm/cet.h | 7 ++ arch/x86/include/asm/special_insns.h | 30 ++++++ arch/x86/kernel/shstk.c | 140 +++++++++++++++++++++++++++ 3 files changed, 177 insertions(+) diff --git a/arch/x86/include/asm/cet.h b/arch/x86/include/asm/cet.h index 4314a41ab3c9..aa533700ba31 100644 --- a/arch/x86/include/asm/cet.h +++ b/arch/x86/include/asm/cet.h @@ -21,6 +21,9 @@ int shstk_alloc_thread_stack(struct task_struct *p, uns= igned long clone_flags, unsigned long stack_size); void shstk_free(struct task_struct *p); void shstk_disable(void); +int shstk_setup_rstor_token(bool ia32, unsigned long restorer, + unsigned long *new_ssp); +int shstk_check_rstor_token(bool ia32, unsigned long *new_ssp); #else static inline int shstk_setup(void) { return 0; } static inline int shstk_alloc_thread_stack(struct task_struct *p, @@ -28,6 +31,10 @@ static inline int shstk_alloc_thread_stack(struct task= _struct *p, unsigned long stack_size) { return 0; } static inline void shstk_free(struct task_struct *p) {} static inline void shstk_disable(void) {} +static inline int shstk_setup_rstor_token(bool ia32, unsigned long resto= rer, + unsigned long *new_ssp) { return 0; } +static inline int shstk_check_rstor_token(bool ia32, + unsigned long *new_ssp) { return 0; } #endif =20 #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/= special_insns.h index f3fbb84ff8a7..c6df3773b44c 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -222,6 +222,36 @@ static inline void clwb(volatile void *__p) : [pax] "a" (p)); } =20 +#ifdef CONFIG_X86_SHADOW_STACK +static inline int write_user_shstk_32(u32 __user *addr, u32 val) +{ + if (WARN_ONCE(!IS_ENABLED(CONFIG_IA32_EMULATION) && + !IS_ENABLED(CONFIG_X86_X32), + "%s used but not supported.\n", __func__)) { + return -EFAULT; + } + + asm_volatile_goto("1: wrussd %[val], (%[addr])\n" + _ASM_EXTABLE(1b, %l[fail]) + :: [addr] "r" (addr), [val] "r" (val) + :: fail); + return 0; +fail: + return -EFAULT; +} + +static inline int write_user_shstk_64(u64 __user *addr, u64 val) +{ + asm_volatile_goto("1: wrussq %[val], (%[addr])\n" + _ASM_EXTABLE(1b, %l[fail]) + :: [addr] "r" (addr), [val] "r" (val) + :: fail); + return 0; +fail: + return -EFAULT; +} +#endif /* CONFIG_X86_SHADOW_STACK */ + #define nop() asm volatile ("nop") =20 static inline void serialize(void) diff --git a/arch/x86/kernel/shstk.c b/arch/x86/kernel/shstk.c index 7c1ca2476a5e..548d0552f9b3 100644 --- a/arch/x86/kernel/shstk.c +++ b/arch/x86/kernel/shstk.c @@ -20,6 +20,7 @@ #include #include #include +#include =20 static void start_update_msrs(void) { @@ -193,3 +194,142 @@ void shstk_disable(void) =20 shstk_free(current); } + +static unsigned long get_user_shstk_addr(void) +{ + struct fpu *fpu =3D ¤t->thread.fpu; + unsigned long ssp =3D 0; + + fpregs_lock(); + + if (fpregs_state_valid(fpu, smp_processor_id())) { + rdmsrl(MSR_IA32_PL3_SSP, ssp); + } else { + struct cet_user_state *p; + + /* + * When !fpregs_state_valid() and get_xsave_addr() returns + * null, XFEAUTRE_CET_USER is in init state. Shadow stack + * pointer is null in this case, so return zero. This can + * happen when shadow stack is enabled, but its xstates in + * memory is corrupted. + */ + p =3D get_xsave_addr(&fpu->state.xsave, XFEATURE_CET_USER); + if (p) + ssp =3D p->user_ssp; + } + + fpregs_unlock(); + + return ssp; +} + +/* + * Create a restore token on the shadow stack. A token is always 8-byte + * and aligned to 8. + */ +static int create_rstor_token(bool ia32, unsigned long ssp, + unsigned long *token_addr) +{ + unsigned long addr; + + /* Aligned to 8 is aligned to 4, so test 8 first */ + if ((!ia32 && !IS_ALIGNED(ssp, 8)) || !IS_ALIGNED(ssp, 4)) + return -EINVAL; + + addr =3D ALIGN_DOWN(ssp, 8) - 8; + + /* Is the token for 64-bit? */ + if (!ia32) + ssp |=3D BIT(0); + + if (write_user_shstk_64((u64 __user *)addr, (u64)ssp)) + return -EFAULT; + + *token_addr =3D addr; + + return 0; +} + +/* + * Create a restore token on shadow stack, and then push the user-mode + * function return address. + */ +int shstk_setup_rstor_token(bool ia32, unsigned long ret_addr, + unsigned long *new_ssp) +{ + struct thread_shstk *shstk =3D ¤t->thread.shstk; + unsigned long ssp, token_addr; + int err; + + if (!shstk->size) + return 0; + + if (!ret_addr) + return -EINVAL; + + ssp =3D get_user_shstk_addr(); + if (!ssp) + return -EINVAL; + + err =3D create_rstor_token(ia32, ssp, &token_addr); + if (err) + return err; + + if (ia32) { + ssp =3D token_addr - sizeof(u32); + err =3D write_user_shstk_32((u32 __user *)ssp, (u32)ret_addr); + } else { + ssp =3D token_addr - sizeof(u64); + err =3D write_user_shstk_64((u64 __user *)ssp, (u64)ret_addr); + } + + if (!err) + *new_ssp =3D ssp; + + return err; +} + +/* + * Verify token_addr points to a valid token, and then set *new_ssp + * according to the token. + */ +int shstk_check_rstor_token(bool proc32, unsigned long *new_ssp) +{ + unsigned long token_addr; + unsigned long token; + bool shstk32; + + token_addr =3D get_user_shstk_addr(); + + if (get_user(token, (unsigned long __user *)token_addr)) + return -EFAULT; + + /* Is mode flag correct? */ + shstk32 =3D !(token & BIT(0)); + if (proc32 ^ shstk32) + return -EINVAL; + + /* Is busy flag set? */ + if (token & BIT(1)) + return -EINVAL; + + /* Mask out flags */ + token &=3D ~3UL; + + /* + * Restore address aligned? + */ + if ((!proc32 && !IS_ALIGNED(token, 8)) || !IS_ALIGNED(token, 4)) + return -EINVAL; + + /* + * Token placed properly? + */ + if (((ALIGN_DOWN(token, 8) - 8) !=3D token_addr) || token >=3D TASK_SIZ= E_MAX) + return -EINVAL; + + *new_ssp =3D token; + + return 0; +} --=20 2.21.0