From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4C77ACD8CBD for ; Tue, 9 Jun 2026 06:02:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=O445o4ESkIUUbAh3NEe9EeNDpRYYGHMWnSZOZMJZQEs=; b=qJnWTWdm4nBRxT 1tQ6XfQfZiHAUeDwDdx4gEF9ql+9df3ms+uxOHSQ3wX3/jHftCj/nC7k/ljp7Lo9Bf4Nj1k6zqmWe n/qFPoVV//wEMmV/HCr4BjTM6uIg4x+1zGsLNhePACPpmmBF5ERCxbPU+nZsfbwlpRQdXZXqtQtlT RDOzLI1AroPZUkFwgkNXSeKVTelPQ8fdTtzjGvYm9UtuB4MMZkZaz9LfrwLhA41N5wHZMFv7vP+9B Gq+yLJXs4GLUFgLvFOI8sPa82E3Xjc/+rQydtibjuHSKByTBKtorfQUl+4LlK7LerzP/d9RggBW2H 6rbANMjyKPnd32IdZl/g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wWpXv-00000004nOA-2mjF; Tue, 09 Jun 2026 06:02:43 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.99.1 #2 (Red Hat Linux)) id 1wWpXs-00000004nIV-1HkY for opensbi@bombadil.infradead.org; Tue, 09 Jun 2026 06:02:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=gsxkDcWxP78dEk+nguc5R9RaH/KLwo0iNr9638RVVnc=; b=ic4RV6m2GGHAkIqLvzHvOqr0wX pe9PxokJRS7xCXZujNMbDY6dTlWHmcvRyi06jykQzOtx/cNv8Dgnj7EeFvf4tDT35FJa4X9eH/a0d 2dU0n0vkreeYOh5ZZNWKYZvpZhRIuLi9kz0Ar3v1lVqeX58GYAVJeJUW5OdsynjzQJqaDSU9+fvxL lGihG604n6vqICQCCcXWTKkz8FCOy8NiP9ZPMF2lAtj5QbOg9b4lfVCMPE31PPb36Nh6McMFavb3E 0NMOdkM9lvMEPSbSP1VWIx0HywEX6Qb3G5S4FQxXMB+HQl3xnYqMaqbLur4S1cwE4QstFLqYwzaqp SRv8J0sQ==; Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by desiato.infradead.org with esmtps (Exim 4.99.2 #2 (Red Hat Linux)) id 1wWpXo-0000000202g-2FKp for opensbi@lists.infradead.org; Tue, 09 Jun 2026 06:02:38 +0000 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-2bf114b0cf9so45021415ad.2 for ; Mon, 08 Jun 2026 23:02:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780984954; x=1781589754; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gsxkDcWxP78dEk+nguc5R9RaH/KLwo0iNr9638RVVnc=; b=bB+f/plQW3OsGPpaAn6Zw8lBPArS62sPf6+nQB1/jl53mEmVAjoaHePBWakJWfB82w vGMoqYa/A/DErSagrzO+S+koI028nxe/n8bXqxmWSjr5dbe+8JokzUbE/Rx7xXGC/bHw A/d7BcWCzA2/ZnNVkXaGe/kCQ1V+ywvrx9pCUWyLlOOtJzDCD0tLgAcc9PC/uz9FUDMI 2VlJQV/hyJaRvwNAoIvU4carkH4qmN/EIjQCpUEeWsXBSacNqzE9qRU0Jd77eRRNSebb 6AO9wiAX30kqo5DI60HrfLQ/y6W9SiTVpneB8ciU+sCr4bK1LPEVSnRZlHfKk+mHWLT7 C58A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780984954; x=1781589754; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=gsxkDcWxP78dEk+nguc5R9RaH/KLwo0iNr9638RVVnc=; b=lDXfEBY1mhFjVtj2QnLiO3MJFKeLmu5VH+wdfGgndl5QAwOnVCP/gcFqWIsAT/nKXy e1NUvcQ9FnwsNmQHHzr7dPkm5G2mCSW6W5BtYg/v0NT3kf+5c9UN6D6U8lSFjHDvqu9+ aSV2rquPiKSUSwujKKqiYjwtx32qPpfiNL+HQMefgej5icmHU3lRIIo1TeUbgTefiIhu h3Sq8EliwTaNtnZkDEoojQ4nFbabqaVoVVaiXfVoenH+ssCe9Ybig0OcdUed5+UyiQcJ ClJeNmMPvaqAQFROsyOM8onLUXaz44YYEde/aPKnCjbQVmbf0llo3K3VdukqJTKC1496 1U+A== X-Gm-Message-State: AOJu0Yzd2S5KZ+Qjyy9EyWA0GXq4qJuWft0mBVW4NDzhDvlUZoYRMjet zinF2R2XPWPx2kYUTeyhCg7E/IBc5txC1kkj/fceTXp+jBTim5mMYTpAjUlRbQ== X-Gm-Gg: Acq92OH5lMH1hcCiENwm+3soABjbc8OteItx+J+1F9aWD/NWTLWygjP3hIhPBxZSgjz WvlmCOpyrpuMqIYnYvK5HI9e5nprQd2Pv8rJVDcoKx36czchjMsBNQ3sjuXd7oLTl20EeqrAbPc o+hyQMA7NfeFOhlIginzY46HVbpD9oBOxlvqW1Ng7sKHhmlSRtg1/NKjxhCSiFatFnTDqn7+VOS V9MJoodDA0InCpuJWJxowx8W692j1b7zNb0MdArMsqVi2lcqAtwVL7i477Bxdc/fBpg0vzOIYyZ 90kLMzpDTYlnN6BulO8HHApGL0z4KVnY8rOL7vOqkL1bjy3/ijCx3D52hrGKEYwPSrgoKCxR8wD /3to2aplzLlEiqR+GxvFdXSuyiRHANVyP+xBdtx8yYVCTBAPMta7uWF0M9OAe+YumiQPkwZo7mb Y/3cpn4/GtWlfrxebP5HnSkU7SieRrBlZuThA= X-Received: by 2002:a17:903:4b28:b0:2b0:7d3d:756a with SMTP id d9443c01a7336-2c1e89579ccmr196233775ad.35.1780984954200; Mon, 08 Jun 2026 23:02:34 -0700 (PDT) Received: from m91p.airy.home ([172.92.174.155]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2c164f9ed6csm202721105ad.31.2026.06.08.23.02.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 23:02:33 -0700 (PDT) From: Bo Gan To: opensbi@lists.infradead.org, wangruikang@iscas.ac.cn, dramforever@live.com, andrew.jones@oss.qualcomm.com Cc: cleger@rivosinc.com, pjw@kernel.org, asrinivasan@oss.tenstorrent.com Subject: [PATCH v2 3/4] lib: sbi: Add variable-length unprivilege access functions Date: Mon, 8 Jun 2026 23:00:23 -0700 Message-Id: <20260609060024.706-4-ganboing@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260609060024.706-1-ganboing@gmail.com> References: <20260609060024.706-1-ganboing@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260609_070236_988163_216F4BC3 X-CRM114-Status: GOOD ( 16.98 ) X-BeenThere: opensbi@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "opensbi" Errors-To: opensbi-bounces+opensbi=archiver.kernel.org@lists.infradead.org sbi_load/store_loop read/write variable-length buffer unprivileged. Both function use the widest aligned 8/4/2/1 byte load/stores in each loop to reduce the total number of iterations. Also switch the scalar/vector misaligned handlers to make use of such functions to simplify code. Miscellaneous: remove the unnecessary [taddr] in inline assembly Signed-off-by: Bo Gan --- include/sbi/sbi_unpriv.h | 16 +++++++ lib/sbi/sbi_trap_ldst.c | 24 ++++------- lib/sbi/sbi_trap_v_ldst.c | 38 ++++++++--------- lib/sbi/sbi_unpriv.c | 88 ++++++++++++++++++++++++++++++++++++--- 4 files changed, 123 insertions(+), 43 deletions(-) diff --git a/include/sbi/sbi_unpriv.h b/include/sbi/sbi_unpriv.h index 8cbd3de0..226cf175 100644 --- a/include/sbi/sbi_unpriv.h +++ b/include/sbi/sbi_unpriv.h @@ -15,6 +15,16 @@ struct sbi_scratch; struct sbi_trap_info; +union sbi_unpriv_data { + u8 b; + u16 h; + u32 w; +#if __riscv_xlen == 64 + u64 d; +#endif + u8 bytes[__riscv_xlen / 8]; +}; + #define DECLARE_UNPRIVILEGED_LOAD_FUNCTION(type) \ type sbi_load_##type(const type *addr, \ struct sbi_trap_info *trap); @@ -36,6 +46,12 @@ DECLARE_UNPRIVILEGED_LOAD_FUNCTION(u64) DECLARE_UNPRIVILEGED_STORE_FUNCTION(u64) DECLARE_UNPRIVILEGED_LOAD_FUNCTION(ulong) +void sbi_load_loop(u8 *buffer, ulong addr, ulong len, + struct sbi_trap_info *trap); + +void sbi_store_loop(u8 *buffer, ulong addr, ulong len, + struct sbi_trap_info *trap); + ulong sbi_get_insn(ulong mepc, struct sbi_trap_info *trap); #endif diff --git a/lib/sbi/sbi_trap_ldst.c b/lib/sbi/sbi_trap_ldst.c index 5f7de662..e726520f 100644 --- a/lib/sbi/sbi_trap_ldst.c +++ b/lib/sbi/sbi_trap_ldst.c @@ -471,7 +471,6 @@ static int sbi_misaligned_ld_emulator(ulong insn, int rlen, ulong addr, const struct sbi_trap_info *orig_trap = &tcntx->trap; struct sbi_trap_regs *regs = &tcntx->regs; struct sbi_trap_info uptrap; - int i; if (!rlen) { if (IS_VECTOR_LOAD_STORE(insn)) @@ -484,13 +483,10 @@ static int sbi_misaligned_ld_emulator(ulong insn, int rlen, ulong addr, if (addr != orig_trap->tval) return SBI_EFAIL; - for (i = 0; i < rlen; i++) { - out_val->data_bytes[i] = - sbi_load_u8((void *)(addr + i), &uptrap); - if (uptrap.cause) { - sbi_misaligned_tinst_fixup(orig_trap, &uptrap); - return sbi_trap_redirect(regs, &uptrap); - } + sbi_load_loop(out_val->data_bytes, addr, rlen, &uptrap); + if (uptrap.cause) { + sbi_misaligned_tinst_fixup(orig_trap, &uptrap); + return sbi_trap_redirect(regs, &uptrap); } return rlen; } @@ -507,7 +503,6 @@ static int sbi_misaligned_st_emulator(ulong insn, int wlen, ulong addr, const struct sbi_trap_info *orig_trap = &tcntx->trap; struct sbi_trap_regs *regs = &tcntx->regs; struct sbi_trap_info uptrap; - int i; if (!wlen) { if (IS_VECTOR_LOAD_STORE(insn)) @@ -520,13 +515,10 @@ static int sbi_misaligned_st_emulator(ulong insn, int wlen, ulong addr, if (addr != orig_trap->tval) return SBI_EFAIL; - for (i = 0; i < wlen; i++) { - sbi_store_u8((void *)(addr + i), - in_val.data_bytes[i], &uptrap); - if (uptrap.cause) { - sbi_misaligned_tinst_fixup(orig_trap, &uptrap); - return sbi_trap_redirect(regs, &uptrap); - } + sbi_store_loop(in_val.data_bytes, addr, wlen, &uptrap); + if (uptrap.cause) { + sbi_misaligned_tinst_fixup(orig_trap, &uptrap); + return sbi_trap_redirect(regs, &uptrap); } return wlen; } diff --git a/lib/sbi/sbi_trap_v_ldst.c b/lib/sbi/sbi_trap_v_ldst.c index e16e3def..02f7d6cc 100644 --- a/lib/sbi/sbi_trap_v_ldst.c +++ b/lib/sbi/sbi_trap_v_ldst.c @@ -229,20 +229,17 @@ int sbi_misaligned_v_ld_emulator(ulong insn, struct sbi_trap_context *tcntx) /* obtain load data from memory */ for (ulong seg = 0; seg < nf; seg++) { - for (ulong i = 0; i < len; i++) { - bytes[seg * len + i] = - sbi_load_u8((void *)(addr + seg * len + i), - &uptrap); - - if (uptrap.cause) { - if (IS_FAULT_ONLY_FIRST_LOAD(insn) && vstart != 0) { - vl = vstart; - break; - } - vsetvl(vl, vtype); - sbi_misaligned_v_tinst_fixup(&uptrap); - return sbi_trap_redirect(regs, &uptrap); + sbi_load_loop(bytes + seg * len, + addr + seg * len, len, &uptrap); + + if (uptrap.cause) { + if (IS_FAULT_ONLY_FIRST_LOAD(insn) && vstart != 0) { + vl = vstart; + break; } + vsetvl(vl, vtype); + sbi_misaligned_v_tinst_fixup(&uptrap); + return sbi_trap_redirect(regs, &uptrap); } } @@ -332,14 +329,13 @@ int sbi_misaligned_v_st_emulator(ulong insn, struct sbi_trap_context *tcntx) /* write store data to memory */ for (ulong seg = 0; seg < nf; seg++) { - for (ulong i = 0; i < len; i++) { - sbi_store_u8((void *)(addr + seg * len + i), - bytes[seg * len + i], &uptrap); - if (uptrap.cause) { - vsetvl(vl, vtype); - sbi_misaligned_v_tinst_fixup(&uptrap); - return sbi_trap_redirect(regs, &uptrap); - } + sbi_store_loop(bytes + seg * len, + addr + seg * len, len, &uptrap); + + if (uptrap.cause) { + vsetvl(vl, vtype); + sbi_misaligned_v_tinst_fixup(&uptrap); + return sbi_trap_redirect(regs, &uptrap); } } } while (++vstart < vl); diff --git a/lib/sbi/sbi_unpriv.c b/lib/sbi/sbi_unpriv.c index f9bbec59..14b8f3cb 100644 --- a/lib/sbi/sbi_unpriv.c +++ b/lib/sbi/sbi_unpriv.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include @@ -22,13 +23,12 @@ type sbi_load_##type(const type *addr, \ struct sbi_trap_info *trap) \ { \ - register ulong tinfo asm("a3"); \ + register ulong tinfo asm("a3") = (ulong)trap; \ register ulong mstatus = 0; \ register ulong mtvec = (ulong)sbi_hart_expected_trap; \ type ret = 0; \ trap->cause = 0; \ asm volatile( \ - "add %[tinfo], %[taddr], zero\n" \ "csrrw %[mtvec], " STR(CSR_MTVEC) ", %[mtvec]\n" \ "csrrs %[mstatus], " STR(CSR_MSTATUS) ", %[mprv]\n" \ ".option push\n" \ @@ -39,8 +39,7 @@ "csrw " STR(CSR_MTVEC) ", %[mtvec]" \ : [mstatus] "+&r"(mstatus), [mtvec] "+&r"(mtvec), \ [tinfo] "+&r"(tinfo), [ret] "=&r"(ret) \ - : [addr] "m"(*addr), [mprv] "r"(MSTATUS_MPRV), \ - [taddr] "r"((ulong)trap) \ + : [addr] "m"(*addr), [mprv] "r"(MSTATUS_MPRV) \ : "a4", "memory"); \ return ret; \ } @@ -54,7 +53,6 @@ register ulong mtvec = (ulong)sbi_hart_expected_trap; \ trap->cause = 0; \ asm volatile( \ - "add %[tinfo], %[taddr], zero\n" \ "csrrw %[mtvec], " STR(CSR_MTVEC) ", %[mtvec]\n" \ "csrrs %[mstatus], " STR(CSR_MSTATUS) ", %[mprv]\n" \ ".option push\n" \ @@ -66,7 +64,7 @@ : [mstatus] "+&r"(mstatus), [mtvec] "+&r"(mtvec), \ [tinfo] "+&r"(tinfo) \ : [addr] "m"(*addr), [mprv] "r"(MSTATUS_MPRV), \ - [val] "r"(val), [taddr] "r"((ulong)trap) \ + [val] "r"(val) \ : "a4", "memory"); \ } @@ -116,6 +114,84 @@ void sbi_store_u64(u64 *addr, u64 val, # error "Unexpected __riscv_xlen" #endif +void sbi_load_loop(u8 *buffer, ulong addr, ulong len, + struct sbi_trap_info *trap) +{ + union sbi_unpriv_data data; + + trap->cause = 0; + while (len) { + unsigned int width = __riscv_xlen / 8; + void *ptr = (void*)addr; + + while (len < width || (addr & (width - 1))) + width /= 2; + + switch (width) { + case 1: + data.b = sbi_load_u8(ptr, trap); + break; + case 2: + data.h = sbi_load_u16(ptr, trap); + break; + case 4: + data.w = sbi_load_u32(ptr, trap); + break; +#if __riscv_xlen == 64 + case 8: + data.d = sbi_load_u64(ptr, trap); + break; +#endif + } + if (trap->cause) + return; + + sbi_memcpy(buffer, data.bytes, width); + len -= width; + addr += width; + buffer += width; + } +} + +void sbi_store_loop(u8 *buffer, ulong addr, ulong len, + struct sbi_trap_info *trap) +{ + union sbi_unpriv_data data; + + trap->cause = 0; + while (len) { + unsigned int width = __riscv_xlen / 8; + void *ptr = (void*)addr; + + while (len < width || (addr & (width - 1))) + width /= 2; + + sbi_memcpy(data.bytes, buffer, width); + switch (width) { + case 1: + sbi_store_u8(ptr, data.b, trap); + break; + case 2: + sbi_store_u16(ptr, data.h, trap); + break; + case 4: + sbi_store_u32(ptr, data.w, trap); + break; +#if __riscv_xlen == 64 + case 8: + sbi_store_u64(ptr, data.d, trap); + break; +#endif + } + if (trap->cause) + return; + + len -= width; + addr += width; + buffer += width; + } +} + ulong sbi_get_insn(ulong mepc, struct sbi_trap_info *trap) { register ulong tinfo asm("a3"); -- 2.34.1 -- opensbi mailing list opensbi@lists.infradead.org http://lists.infradead.org/mailman/listinfo/opensbi