From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1A0BC47258 for ; Thu, 25 Jan 2024 17:30:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2322C6B0081; Thu, 25 Jan 2024 12:30:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E1336B0082; Thu, 25 Jan 2024 12:30:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05B0F6B0083; Thu, 25 Jan 2024 12:30:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E7EAE6B0081 for ; Thu, 25 Jan 2024 12:30:15 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1F44D40E3B for ; Thu, 25 Jan 2024 17:30:14 +0000 (UTC) X-FDA: 81718521948.29.6F8F84B Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by imf30.hostedemail.com (Postfix) with ESMTP id 0154980021 for ; Thu, 25 Jan 2024 17:30:10 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=gUzxXacU; spf=pass (imf30.hostedemail.com: domain of debug@rivosinc.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=debug@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706203811; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FiMcoXLtake3SsYp4VgIXSl2ZPSDDwizQGFnhQxFN8w=; b=KvAxnhzy0j4xYdKwAav3lpS2LlCv8QeEE+VYZnHHSj0n2eWzkJeLabEPVZv7SrgYI5z4FC zbyFGpRoIV6p4Abze6qJ7+Y8hicJpSVsDhDj+qNvl5jkE/O4t+ZnGCAP3olviPuzHki0lp KA1aeoNlk8muHPNlAzaGW2GBB/cW+ew= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=gUzxXacU; spf=pass (imf30.hostedemail.com: domain of debug@rivosinc.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=debug@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706203811; a=rsa-sha256; cv=none; b=njHJzltkSSNx/k64TOeWdXDFgl/2nGtzYa4JZI+oruy0GR6E1CGFlCHkuh88k2b9cmL1uo pszw1w/+Rrl1jnPbFc18NTLj/5dXJE7uq321QBVJ6OByD1W437DASwoyoivH3kAYzpV5Gh +AEfe8iWwfdQt4y76bRI53Ph59468Ng= Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-6d9f94b9186so6803488b3a.0 for ; Thu, 25 Jan 2024 09:30:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1706203810; x=1706808610; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=FiMcoXLtake3SsYp4VgIXSl2ZPSDDwizQGFnhQxFN8w=; b=gUzxXacUukaKHsVQWsFsms4ZawB+slwD10hCBPPk26KBU8xq9ojiPiuTNffidkja3T wqUZS8rI3R/IKaHqMRv130ykG+qPgYfHR7+35RpICCuGSeAVTzYmErKg4XUaUGt8Mlzm o9C7ihN73QeUpz681dE2xa/+z/0Pwh22CtK41ULWpKfNB9hiJQYhAb555bCok6XekPCC 7chs0YkopUO3Ph5ukl+ctOasTybu/Bsc4D5DCDTbwZj0rC0vHRI55TOIaCjOzeTObhqj np64Y1Vu44x5SiJ+FFjNwZR8NyLjLdn5Olw2hNKeFCB495ne4DbwTKb+9uoQ+6zHpa3Y J9hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706203810; x=1706808610; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=FiMcoXLtake3SsYp4VgIXSl2ZPSDDwizQGFnhQxFN8w=; b=DrgPkFE+iIJEGGh8EXVoOygnVGtu6U70zuX6fSByyTXqzIefgJS2/+6boNasRMpfkH AFjGcB/dG5af09cvEZKdDxjLVxKS2sqDhTKhgHM1bo1D9cdYmyQVd6VVCwU3SRqDkeGS lR7ahbLLHmMhZS06YfSQGG2HQML3IP9DXiIVEH72q2EKa5VVhr/Cjz+pyZ3WXRoHghhD F5ClaFIWH2mErjF3BrHzP4tKI3ln5RBJZHA31ApMD0OiaBE870VatzZjAtNIuWRUDozy HcCtp52145gwMlJyS9gDGxp1mDGlScQlNqCXqrlSdFWMbYQoMrz0mwsBAvqsNLBcDVyE PKCA== X-Gm-Message-State: AOJu0YwrTkUnLeBsIYaBoVxul86C07ffTkqdxsBRiXuKtp8+M0Y9EaSk 1yKfrvmki9LZb2B28oPAx0dxHdClkLpeROy/NDiT/qubZoeeBAQgcGWNZXK3D5U= X-Google-Smtp-Source: AGHT+IFiBmJ9bUWly4Y63SGZXbMGx4CdDV3C4ohhFPjAa5O+ssE0qkHaRA6CwOtYsFJogn6onLaUzQ== X-Received: by 2002:aa7:8755:0:b0:6d9:be3e:19a0 with SMTP id g21-20020aa78755000000b006d9be3e19a0mr32266pfo.48.1706203809641; Thu, 25 Jan 2024 09:30:09 -0800 (PST) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id v6-20020aa78086000000b006dbda1b19f7sm9099781pff.159.2024.01.25.09.30.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 09:30:09 -0800 (PST) Date: Thu, 25 Jan 2024 09:30:04 -0800 From: Deepak Gupta To: Stefan O'Rear Cc: rick.p.edgecombe@intel.com, broonie@kernel.org, Szabolcs.Nagy@arm.com, "kito.cheng@sifive.com" , Kees Cook , Andrew Jones , paul.walmsley@sifive.com, Palmer Dabbelt , Conor Dooley , cleger@rivosinc.com, Atish Patra , Alexandre Ghiti , =?iso-8859-1?Q?Bj=F6rn_T=F6pel?= , Alexandre Ghiti , Jonathan Corbet , Albert Ou , oleg@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, "Eric W. Biederman" , shuah@kernel.org, Christian Brauner , guoren , samitolvanen@google.com, Evan Green , xiao.w.wang@intel.com, Anup Patel , mchitale@ventanamicro.com, waylingii@gmail.com, greentime.hu@sifive.com, Heiko Stuebner , Jisheng Zhang , shikemeng@huaweicloud.com, david@redhat.com, Charlie Jenkins , panqinglin2020@iscas.ac.cn, willy@infradead.org, Vincent Chen , Andy Chiu , Greg Ungerer , jeeheng.sia@starfivetech.com, mason.huo@starfivetech.com, ancientmodern4@gmail.com, mathis.salmen@matsal.de, cuiyunhui@bytedance.com, bhe@redhat.com, chenjiahao16@huawei.com, ruscur@russell.cc, bgray@linux.ibm.com, alx@kernel.org, baruch@tkos.co.il, zhangqing@loongson.cn, Catalin Marinas , revest@chromium.org, josh@joshtriplett.org, joey.gouly@arm.com, shr@devkernel.io, omosnace@redhat.com, ojeda@kernel.org, jhubbard@nvidia.com, linux-doc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kselftest@vger.kernel.org Subject: Re: [RFC PATCH v1 07/28] riscv: kernel handling on trap entry/exit for user cfi Message-ID: References: <20240125062739.1339782-1-debug@rivosinc.com> <20240125062739.1339782-8-debug@rivosinc.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 0154980021 X-Rspam-User: X-Stat-Signature: nnfebrmm3a6shyxiwcispmbtd1nydnpt X-Rspamd-Server: rspam01 X-HE-Tag: 1706203810-286255 X-HE-Meta: U2FsdGVkX1+Cmu0C0mA2B+UbBxg3+LWTJQoQgmS7S7Qzz7kIyAUmGIR16OEIoXhpboeIKq2SzrlkoB5RKxXDHV7sm8QpILD/FrVdGC2g9XNc36/ZZWfOqmqWr6DTkZPMlGbpmc+9pw8iZgXV8WsMgzO5oz0V/wLwMUowkdQxQZPXJeeXLQiVmDMZqaLVXvThrim9J9MkFujWpL8oTEHF/dcHwL4G3EMNOQWGbkXj/fzMrXni/sor3Q2wjyVGjKbksn65x5wMCV5cWZNzNk3zwqCwLcDD/JwHf9z5PJ2j0g6dhfbFuLSD7sICDNnLtHss9kuoizJFpB1CRu8ck3lmMrn8SWyleHJbxzSMomJZvp8yVy7FxUcfP/o715zthFSHM4BocxAUB8ipScb1gJqZypFaY5BETvW192CfDKN+a4x+fGoITqvvX/wZx9D8iIY2E9HaxGLExot8lyNrjHv/pWadAMLQi6ovvfw4Qjj/0KGfI4lHdfosNLB7zd9ZcnIPvsoJ6ZdED7i/rTHN7NUS0eceJsM/8hZzpAdk/4tZnf+jouriuaI0y6YnwMH2ashl5Ok5pOe6lduh8DE6S5JUnIauCHNzuzgFZ2QCRn9jocxYjjsaDRCaHHOHUXF8syZcjJgsTs9mce1mvvfBjjzmgFBs4oC5QKOTiFv56fkkOncwPJj3R7AQmptYZf4UmlxgUXsvtJW54W0/jZdGkcITK0GiFkpK3f+MJx4/fMg1M+o6anais07iM+lSUjz/fbYNfINkk94/BPIDtGLXu+oEhHzjY4tvxFTJqT4GsbelaFg59rQRbz4iWqjOkg69rL1CpyR8P3MHgAQgnb/xfXBWwjuylTQbPg/YaDayo7r4exuZce1fDHaSWkPD5s1t2TblBIvWmpDJ9HtGGHGg7GjwrfFmnnfOGrCRAPD4I5CULoWFguU5Pi5XotUc35G+Po9lCGj/k1ugZUe4Awrhw2l sDPAXOP2 p26c4PJJKV1s8FQd4JfsTHGBCqJUZCB02urTsYW5BJoiCiUv1KoITl/Z8TZC0CGhbudyJRu/YcPyNTsn57qdLKGFEyo/QzWXjRY7OnIMOwRqhn19R30/3l5rDETggxGnvzNFtLckCkDR1cUl3xwtObuCpS8TcZ+yEOyW/2dqSgPpiIjyPguae9bfA+aZaRraAazeC2CYK4azIaD4KfYihYmzK9LTLpf+on9YyesuExnAz9Y1G+e3h50OKusAHILap2bbt/mETndE3gxlFSF/I1wYRdLE/NPvRXLTh95G7DL8s63j4dix3YH1Q33+NDfZ0GoxpBPpfxc31pCbnYxJvzd5xI3tvycDQOVudRriUHZa+v6mV0iN91MK0XBYygfpr1eImdNELVIrOuWTKoeF4GDPN/WmxYqrybAG4/fNbjFVTrlS0kPQwMYjk1jhp+LPGNrl/drmgE16GQZiq/Z9GDn5gnGoyommtXEU5DOp8o1vgl05Rc+BOfCdEkIS2vmQ7B7hEC5UduMbdGGqMXwANozBQo8qkZePscx8wdYf6f7Wk7IaD2l2FCQTPwiayMgmmrpkfu4lgrV6VhK8V2242FW1nai7tXUxIfTnm6GXnC8X2qtTFftkQbRhHNSgzBz/sgh1+8mGJC4bEsFI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jan 25, 2024 at 02:29:01AM -0500, Stefan O'Rear wrote: >On Thu, Jan 25, 2024, at 1:21 AM, debug@rivosinc.com wrote: >> From: Deepak Gupta >> >> Carves out space in arch specific thread struct for cfi status and shadow stack >> in usermode on riscv. >> >> This patch does following >> - defines a new structure cfi_status with status bit for cfi feature >> - defines shadow stack pointer, base and size in cfi_status structure >> - defines offsets to new member fields in thread in asm-offsets.c >> - Saves and restore shadow stack pointer on trap entry (U --> S) and exit >> (S --> U) >> >> Signed-off-by: Deepak Gupta >> --- >> arch/riscv/include/asm/processor.h | 1 + >> arch/riscv/include/asm/thread_info.h | 3 +++ >> arch/riscv/include/asm/usercfi.h | 24 ++++++++++++++++++++++++ >> arch/riscv/kernel/asm-offsets.c | 5 ++++- >> arch/riscv/kernel/entry.S | 25 +++++++++++++++++++++++++ >> 5 files changed, 57 insertions(+), 1 deletion(-) >> create mode 100644 arch/riscv/include/asm/usercfi.h >> >> diff --git a/arch/riscv/include/asm/processor.h >> b/arch/riscv/include/asm/processor.h >> index ee2f51787ff8..d4dc298880fc 100644 >> --- a/arch/riscv/include/asm/processor.h >> +++ b/arch/riscv/include/asm/processor.h >> @@ -14,6 +14,7 @@ >> >> #include >> #include >> +#include >> >> #ifdef CONFIG_64BIT >> #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1)) >> diff --git a/arch/riscv/include/asm/thread_info.h >> b/arch/riscv/include/asm/thread_info.h >> index 320bc899a63b..6a2acecec546 100644 >> --- a/arch/riscv/include/asm/thread_info.h >> +++ b/arch/riscv/include/asm/thread_info.h >> @@ -58,6 +58,9 @@ struct thread_info { >> int cpu; >> unsigned long syscall_work; /* SYSCALL_WORK_ flags */ >> unsigned long envcfg; >> +#ifdef CONFIG_RISCV_USER_CFI >> + struct cfi_status user_cfi_state; >> +#endif >> #ifdef CONFIG_SHADOW_CALL_STACK >> void *scs_base; >> void *scs_sp; >> diff --git a/arch/riscv/include/asm/usercfi.h >> b/arch/riscv/include/asm/usercfi.h >> new file mode 100644 >> index 000000000000..080d7077d12c >> --- /dev/null >> +++ b/arch/riscv/include/asm/usercfi.h >> @@ -0,0 +1,24 @@ >> +/* SPDX-License-Identifier: GPL-2.0 >> + * Copyright (C) 2023 Rivos, Inc. >> + * Deepak Gupta >> + */ >> +#ifndef _ASM_RISCV_USERCFI_H >> +#define _ASM_RISCV_USERCFI_H >> + >> +#ifndef __ASSEMBLY__ >> +#include >> + >> +#ifdef CONFIG_RISCV_USER_CFI >> +struct cfi_status { >> + unsigned long ubcfi_en : 1; /* Enable for backward cfi. */ >> + unsigned long rsvd : ((sizeof(unsigned long)*8) - 1); >> + unsigned long user_shdw_stk; /* Current user shadow stack pointer */ >> + unsigned long shdw_stk_base; /* Base address of shadow stack */ >> + unsigned long shdw_stk_size; /* size of shadow stack */ >> +}; >> + >> +#endif /* CONFIG_RISCV_USER_CFI */ >> + >> +#endif /* __ASSEMBLY__ */ >> + >> +#endif /* _ASM_RISCV_USERCFI_H */ >> diff --git a/arch/riscv/kernel/asm-offsets.c >> b/arch/riscv/kernel/asm-offsets.c >> index cdd8f095c30c..5e1f412e96ba 100644 >> --- a/arch/riscv/kernel/asm-offsets.c >> +++ b/arch/riscv/kernel/asm-offsets.c >> @@ -43,8 +43,11 @@ void asm_offsets(void) >> #ifdef CONFIG_SHADOW_CALL_STACK >> OFFSET(TASK_TI_SCS_SP, task_struct, thread_info.scs_sp); >> #endif >> - >> OFFSET(TASK_TI_CPU_NUM, task_struct, thread_info.cpu); >> +#ifdef CONFIG_RISCV_USER_CFI >> + OFFSET(TASK_TI_CFI_STATUS, task_struct, thread_info.user_cfi_state); >> + OFFSET(TASK_TI_USER_SSP, task_struct, >> thread_info.user_cfi_state.user_shdw_stk); >> +#endif >> OFFSET(TASK_THREAD_F0, task_struct, thread.fstate.f[0]); >> OFFSET(TASK_THREAD_F1, task_struct, thread.fstate.f[1]); >> OFFSET(TASK_THREAD_F2, task_struct, thread.fstate.f[2]); >> diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S >> index 63c3855ba80d..410659e2eadb 100644 >> --- a/arch/riscv/kernel/entry.S >> +++ b/arch/riscv/kernel/entry.S >> @@ -49,6 +49,21 @@ SYM_CODE_START(handle_exception) >> REG_S x5, PT_T0(sp) >> save_from_x6_to_x31 >> >> +#ifdef CONFIG_RISCV_USER_CFI >> + /* >> + * we need to save cfi status only when previous mode was U >> + */ >> + csrr s2, CSR_STATUS >> + andi s2, s2, SR_SPP >> + bnez s2, skip_bcfi_save >> + /* load cfi status word */ >> + lw s3, TASK_TI_CFI_STATUS(tp) >> + andi s3, s3, 1 >> + beqz s3, skip_bcfi_save >> + csrr s3, CSR_SSP >> + REG_S s3, TASK_TI_USER_SSP(tp) /* save user ssp in thread_info */ >> +skip_bcfi_save: >> +#endif >> /* >> * Disable user-mode memory access as it should only be set in the >> * actual user copy routines. >> @@ -141,6 +156,16 @@ SYM_CODE_START_NOALIGN(ret_from_exception) >> * structures again. >> */ >> csrw CSR_SCRATCH, tp >> + >> +#ifdef CONFIG_RISCV_USER_CFI >> + lw s3, TASK_TI_CFI_STATUS(tp) >> + andi s3, s3, 1 >> + beqz s3, skip_bcfi_resume >> + REG_L s3, TASK_TI_USER_SSP(tp) /* restore user ssp from thread struct */ >> + csrw CSR_SSP, s3 >> +skip_bcfi_resume: >> +#endif >> + > >We shouldn't need any of this in the entry/exit code, at least as long as >the kernel itself is not using Zicfiss. ssp can keep its value in the >kernel and swap it on task switches. Our entry/exit code is rather short >and I'd like to keep it that way. I kept it here because sooner or later we will need to establish kernel shadow stack. Kernel shadow stack on riscv (compared to other arches) kernel actually will be easier to support and adopt because there is already support for shadow call stack (SCS, [1]). Difference between existing shadow call stack (SCS) and `zicfiss` based kernel shadow stack would be - In prolog instead of using `sd`, we will be inserting `sspush` to save ret addr - In epilog instead of using `ld` and compare, we will be inserting `sspopchk` So a lot underlying work and functional testing for shadow kernel stack is already carried out with SCS patches. It would be easier and faster to re-use SCS patches to support `zicfiss` based shadow stack. I don't have favorites here, if overwhelving opinion of community here is to take this logic into task switching and re-work this logic back into entry.S whenever shadow stack for kernel patches are posted, I can do that as well. [1] - https://lore.kernel.org/all/20230828195833.756747-8-samitolvanen@google.com/ > >-s > >> 1: >> REG_L a0, PT_STATUS(sp) >> /* >> -- >> 2.43.0 >> >> >> _______________________________________________ >> linux-riscv mailing list >> linux-riscv@lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/linux-riscv