* [PATCH v3] target/riscv: Enable Zicbo[m,z,p] instructions @ 2022-02-10 16:34 Christoph Muellner 2022-02-10 16:48 ` Philipp Tomsich 2022-02-11 2:41 ` Weiwei Li 0 siblings, 2 replies; 5+ messages in thread From: Christoph Muellner @ 2022-02-10 16:34 UTC (permalink / raw) To: Atish Patra, Anup Patel, Frédéric Pétrot, Palmer Dabbelt, Alistair Francis, Bin Meng, qemu-riscv, qemu-devel, Philipp Tomsich, Richard Henderson, Weiwei Li Cc: Christoph Muellner The RISC-V base cache management operation ISA extension has been ratified [1]. This patch adds support for the defined instructions. The cmo.prefetch instructions are nops for QEMU (no emulation of the memory hierarchy, no illegal instructions, no permission faults, no traps), therefore there's only a comment where they would be decoded. The other cbo* instructions are moved into an overlap group to resolve the overlapping pattern with the LQ instruction. The cbo.zero zeros a configurable amount of bytes. Similar to other extensions (e.g. atomic instructions), the trap behavior is limited such, that only the page permissions are checked (ignoring other optional protection mechanisms like PMA or PMP). [1] https://wiki.riscv.org/display/TECH/Recently+Ratified+Extensions v3: - Enable by default (like zb*) - Rename flags Zicbo* -> zicbo* (like zb*) - Rename ext_zicbo* -> ext_icbo* (like ext_icsr) - Rename trans_zicbo.c.inc -> trans_rvzicbo.c.inc (like all others) - Simplify prefetch instruction support to a single comment - Rebase on top of github-alistair23/riscv-to-apply.next plus the Priv v1.12 series from github-atishp04/priv_1_12_support_v3 v2: - Fix overlapping instruction encoding with LQ instructions - Drop CSR related changes and rebase on Priv 1.12 patchset Co-developed-by: Philipp Tomsich <philipp.tomsich@vrull.eu> Signed-off-by: Christoph Muellner <cmuellner@linux.com> --- target/riscv/cpu.c | 3 + target/riscv/cpu.h | 3 + target/riscv/helper.h | 5 ++ target/riscv/insn32.decode | 16 +++- target/riscv/insn_trans/trans_rvzicbo.c.inc | 57 +++++++++++++ target/riscv/op_helper.c | 94 +++++++++++++++++++++ target/riscv/translate.c | 1 + 7 files changed, 178 insertions(+), 1 deletion(-) create mode 100644 target/riscv/insn_trans/trans_rvzicbo.c.inc diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 39ffb883fc..cbd0a34318 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -764,6 +764,9 @@ static Property riscv_cpu_properties[] = { DEFINE_PROP_BOOL("Counters", RISCVCPU, cfg.ext_counters, true), DEFINE_PROP_BOOL("Zifencei", RISCVCPU, cfg.ext_ifencei, true), DEFINE_PROP_BOOL("Zicsr", RISCVCPU, cfg.ext_icsr, true), + DEFINE_PROP_BOOL("zicbom", RISCVCPU, cfg.ext_icbom, true), + DEFINE_PROP_BOOL("zicboz", RISCVCPU, cfg.ext_icboz, true), + DEFINE_PROP_UINT16("cbozlen", RISCVCPU, cfg.cbozlen, 64), DEFINE_PROP_BOOL("Zfh", RISCVCPU, cfg.ext_zfh, false), DEFINE_PROP_BOOL("Zfhmin", RISCVCPU, cfg.ext_zfhmin, false), DEFINE_PROP_BOOL("Zve32f", RISCVCPU, cfg.ext_zve32f, false), diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index fe80caeec0..7bd2fd26d6 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -368,6 +368,8 @@ struct RISCVCPUConfig { bool ext_counters; bool ext_ifencei; bool ext_icsr; + bool ext_icbom; + bool ext_icboz; bool ext_zfh; bool ext_zfhmin; bool ext_zve32f; @@ -382,6 +384,7 @@ struct RISCVCPUConfig { char *vext_spec; uint16_t vlen; uint16_t elen; + uint16_t cbozlen; bool mmu; bool pmp; bool epmp; diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 72cc2582f4..ef1944da8f 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -92,6 +92,11 @@ DEF_HELPER_FLAGS_2(fcvt_h_l, TCG_CALL_NO_RWG, i64, env, tl) DEF_HELPER_FLAGS_2(fcvt_h_lu, TCG_CALL_NO_RWG, i64, env, tl) DEF_HELPER_FLAGS_1(fclass_h, TCG_CALL_NO_RWG_SE, tl, i64) +/* Cache-block operations */ +DEF_HELPER_2(cbo_clean_flush, void, env, tl) +DEF_HELPER_2(cbo_inval, void, env, tl) +DEF_HELPER_2(cbo_zero, void, env, tl) + /* Special functions */ DEF_HELPER_2(csrr, tl, env, int) DEF_HELPER_3(csrw, void, env, int, tl) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 5bbedc254c..d5f8329970 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -128,6 +128,7 @@ addi ............ ..... 000 ..... 0010011 @i slti ............ ..... 010 ..... 0010011 @i sltiu ............ ..... 011 ..... 0010011 @i xori ............ ..... 100 ..... 0010011 @i +# cbo.prefetch_{i,r,m} instructions are ori with rd=x0 and not decoded. ori ............ ..... 110 ..... 0010011 @i andi ............ ..... 111 ..... 0010011 @i slli 00000. ...... ..... 001 ..... 0010011 @sh @@ -168,7 +169,20 @@ sraw 0100000 ..... ..... 101 ..... 0111011 @r # *** RV128I Base Instruction Set (in addition to RV64I) *** ldu ............ ..... 111 ..... 0000011 @i -lq ............ ..... 010 ..... 0001111 @i +{ + [ + # *** RV32 Zicbom Standard Extension *** + cbo_clean 0000000 00001 ..... 010 00000 0001111 @sfence_vm + cbo_flush 0000000 00010 ..... 010 00000 0001111 @sfence_vm + cbo_inval 0000000 00000 ..... 010 00000 0001111 @sfence_vm + + # *** RV32 Zicboz Standard Extension *** + cbo_zero 0000000 00100 ..... 010 00000 0001111 @sfence_vm + ] + + # *** RVI128 lq *** + lq ............ ..... 010 ..... 0001111 @i +} sq ............ ..... 100 ..... 0100011 @s addid ............ ..... 000 ..... 1011011 @i sllid 000000 ...... ..... 001 ..... 1011011 @sh6 diff --git a/target/riscv/insn_trans/trans_rvzicbo.c.inc b/target/riscv/insn_trans/trans_rvzicbo.c.inc new file mode 100644 index 0000000000..35c277261f --- /dev/null +++ b/target/riscv/insn_trans/trans_rvzicbo.c.inc @@ -0,0 +1,57 @@ +/* + * RISC-V translation routines for the RISC-V CBO Extension. + * + * Copyright (c) 2021 Philipp Tomsich, philipp.tomsich@vrull.eu + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see <http://www.gnu.org/licenses/>. + */ + +#define REQUIRE_ZICBOM(ctx) do { \ + if (!RISCV_CPU(ctx->cs)->cfg.ext_icbom) { \ + return false; \ + } \ +} while (0) + +#define REQUIRE_ZICBOZ(ctx) do { \ + if (!RISCV_CPU(ctx->cs)->cfg.ext_icboz) { \ + return false; \ + } \ +} while (0) + +static bool trans_cbo_clean(DisasContext *ctx, arg_cbo_clean *a) +{ + REQUIRE_ZICBOM(ctx); + gen_helper_cbo_clean_flush(cpu_env, cpu_gpr[a->rs1]); + return true; +} + +static bool trans_cbo_flush(DisasContext *ctx, arg_cbo_flush *a) +{ + REQUIRE_ZICBOM(ctx); + gen_helper_cbo_clean_flush(cpu_env, cpu_gpr[a->rs1]); + return true; +} + +static bool trans_cbo_inval(DisasContext *ctx, arg_cbo_inval *a) +{ + REQUIRE_ZICBOM(ctx); + gen_helper_cbo_inval(cpu_env, cpu_gpr[a->rs1]); + return true; +} + +static bool trans_cbo_zero(DisasContext *ctx, arg_cbo_zero *a) +{ + REQUIRE_ZICBOZ(ctx); + gen_helper_cbo_zero(cpu_env, cpu_gpr[a->rs1]); + return true; +} diff --git a/target/riscv/op_helper.c b/target/riscv/op_helper.c index 1a75ba11e6..21762bc2e0 100644 --- a/target/riscv/op_helper.c +++ b/target/riscv/op_helper.c @@ -3,6 +3,7 @@ * * Copyright (c) 2016-2017 Sagar Karandikar, sagark@eecs.berkeley.edu * Copyright (c) 2017-2018 SiFive, Inc. + * Copyright (c) 2022 VRULL GmbH * * This program is free software; you can redistribute it and/or modify it * under the terms and conditions of the GNU General Public License, @@ -114,6 +115,99 @@ target_ulong helper_csrrw_i128(CPURISCVState *env, int csr, return int128_getlo(rv); } + +/* helper_zicbo_envcfg + * + * Raise virtual exceptions and illegal instruction exceptions for + * Zicbo[mz] instructions based on the settings of [mhs]envcfg as + * specified in section 2.5.1 of the CMO specification. + */ +static void helper_zicbo_envcfg(CPURISCVState *env, target_ulong envbits, + uintptr_t ra) +{ +#ifndef CONFIG_USER_ONLY + /* Check for virtual instruction exceptions first, as we don't see + * VU and VS reflected in env->priv (these are just the translated + * U and S stated with virtualisation enabled. + */ + if (riscv_cpu_virt_enabled(env) && + (((env->priv < PRV_H) && !get_field(env->henvcfg, envbits)) || + ((env->priv < PRV_S) && !get_field(env->senvcfg, envbits)))) { + riscv_raise_exception(env, RISCV_EXCP_VIRT_INSTRUCTION_FAULT, ra); + } + + if (((env->priv < PRV_M) && !get_field(env->menvcfg, envbits)) || + ((env->priv < PRV_S) && !get_field(env->senvcfg, envbits))) { + riscv_raise_exception(env, RISCV_EXCP_ILLEGAL_INST, ra); + } +#endif +} + +/* helper_zicbom_access + * + * Check access permissions (LOAD, STORE or FETCH as specified in section + * 2.5.2 of the CMO specification) for Zicbom, raising either store + * page-fault (non-virtualised) or store guest-page fault (virtualised). + */ +static void helper_zicbom_access(CPURISCVState *env, target_ulong address, + uintptr_t ra) +{ + void* phost; + int ret = TLB_INVALID_MASK; + MMUAccessType access_type = MMU_DATA_LOAD; + + while (ret == TLB_INVALID_MASK && access_type <= MMU_INST_FETCH) { + ret = probe_access_flags(env, address, access_type++, + cpu_mmu_index(env, false), + true, &phost, ra); + } + + if (ret == TLB_INVALID_MASK) { + uint32_t exc = RISCV_EXCP_STORE_PAGE_FAULT; + +#ifndef CONFIG_USER_ONLY + /* User-mode emulation does not have virtualisation. */ + if (riscv_cpu_virt_enabled(env)) { + exc = RISCV_EXCP_STORE_GUEST_AMO_ACCESS_FAULT; + } +#endif + riscv_raise_exception(env, exc, ra); + } +} + +void helper_cbo_clean_flush(CPURISCVState *env, target_ulong address) +{ + uintptr_t ra = GETPC(); + helper_zicbo_envcfg(env, MENVCFG_CBCFE, ra); + helper_zicbom_access(env, address, ra); +} + +void helper_cbo_inval(CPURISCVState *env, target_ulong address) +{ + uintptr_t ra = GETPC(); + helper_zicbo_envcfg(env, MENVCFG_CBIE, ra); + helper_zicbom_access(env, address, ra); +} + +void helper_cbo_zero(CPURISCVState *env, target_ulong address) +{ + uintptr_t ra = GETPC(); + helper_zicbo_envcfg(env, MENVCFG_CBZE, ra); + + /* Get the size of the cache block for zero instructions. */ + RISCVCPU *cpu = env_archcpu(env); + uint16_t cbozlen = cpu->cfg.cbozlen; + + /* Mask off low-bits to align-down to the cache-block. */ + address &= ~(cbozlen - 1); + + void* mem = probe_access(env, address, cbozlen, MMU_DATA_STORE, + cpu_mmu_index(env, false), GETPC()); + + /* Zero the block */ + memset(mem, 0, cbozlen); +} + #ifndef CONFIG_USER_ONLY target_ulong helper_sret(CPURISCVState *env) diff --git a/target/riscv/translate.c b/target/riscv/translate.c index eaf5a72c81..0ee2ce85ec 100644 --- a/target/riscv/translate.c +++ b/target/riscv/translate.c @@ -861,6 +861,7 @@ static uint32_t opcode_at(DisasContextBase *dcbase, target_ulong pc) #include "insn_trans/trans_rvv.c.inc" #include "insn_trans/trans_rvb.c.inc" #include "insn_trans/trans_rvzfh.c.inc" +#include "insn_trans/trans_rvzicbo.c.inc" #include "insn_trans/trans_privileged.c.inc" #include "insn_trans/trans_xventanacondops.c.inc" -- 2.35.1 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH v3] target/riscv: Enable Zicbo[m,z,p] instructions 2022-02-10 16:34 [PATCH v3] target/riscv: Enable Zicbo[m,z,p] instructions Christoph Muellner @ 2022-02-10 16:48 ` Philipp Tomsich 2022-02-10 22:10 ` Richard Henderson 2022-02-11 2:41 ` Weiwei Li 1 sibling, 1 reply; 5+ messages in thread From: Philipp Tomsich @ 2022-02-10 16:48 UTC (permalink / raw) To: Christoph Muellner Cc: Weiwei Li, qemu-riscv, Anup Patel, Bin Meng, Atish Patra, qemu-devel, Richard Henderson, Alistair Francis, Palmer Dabbelt, Frédéric Pétrot [-- Attachment #1: Type: text/plain, Size: 13163 bytes --] On Thu, 10 Feb 2022 at 17:34, Christoph Muellner <cmuellner@linux.com> wrote: > The RISC-V base cache management operation ISA extension has been > ratified [1]. This patch adds support for the defined instructions. > > The cmo.prefetch instructions are nops for QEMU (no emulation of the memory > hierarchy, no illegal instructions, no permission faults, no traps), > therefore there's only a comment where they would be decoded. > > The other cbo* instructions are moved into an overlap group to > resolve the overlapping pattern with the LQ instruction. > The cbo.zero zeros a configurable amount of bytes. > Similar to other extensions (e.g. atomic instructions), > the trap behavior is limited such, that only the page permissions > are checked (ignoring other optional protection mechanisms like > PMA or PMP). > > [1] https://wiki.riscv.org/display/TECH/Recently+Ratified+Extensions > > v3: > - Enable by default (like zb*) > - Rename flags Zicbo* -> zicbo* (like zb*) > - Rename ext_zicbo* -> ext_icbo* (like ext_icsr) > - Rename trans_zicbo.c.inc -> trans_rvzicbo.c.inc (like all others) > - Simplify prefetch instruction support to a single comment > - Rebase on top of github-alistair23/riscv-to-apply.next plus the > Priv v1.12 series from github-atishp04/priv_1_12_support_v3 > > v2: > - Fix overlapping instruction encoding with LQ instructions > - Drop CSR related changes and rebase on Priv 1.12 patchset > > Co-developed-by: Philipp Tomsich <philipp.tomsich@vrull.eu> > Signed-off-by: Christoph Muellner <cmuellner@linux.com> > --- > target/riscv/cpu.c | 3 + > target/riscv/cpu.h | 3 + > target/riscv/helper.h | 5 ++ > target/riscv/insn32.decode | 16 +++- > target/riscv/insn_trans/trans_rvzicbo.c.inc | 57 +++++++++++++ > target/riscv/op_helper.c | 94 +++++++++++++++++++++ > target/riscv/translate.c | 1 + > 7 files changed, 178 insertions(+), 1 deletion(-) > create mode 100644 target/riscv/insn_trans/trans_rvzicbo.c.inc > > diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c > index 39ffb883fc..cbd0a34318 100644 > --- a/target/riscv/cpu.c > +++ b/target/riscv/cpu.c > @@ -764,6 +764,9 @@ static Property riscv_cpu_properties[] = { > DEFINE_PROP_BOOL("Counters", RISCVCPU, cfg.ext_counters, true), > DEFINE_PROP_BOOL("Zifencei", RISCVCPU, cfg.ext_ifencei, true), > DEFINE_PROP_BOOL("Zicsr", RISCVCPU, cfg.ext_icsr, true), > + DEFINE_PROP_BOOL("zicbom", RISCVCPU, cfg.ext_icbom, true), > + DEFINE_PROP_BOOL("zicboz", RISCVCPU, cfg.ext_icboz, true), > + DEFINE_PROP_UINT16("cbozlen", RISCVCPU, cfg.cbozlen, 64), > DEFINE_PROP_BOOL("Zfh", RISCVCPU, cfg.ext_zfh, false), > DEFINE_PROP_BOOL("Zfhmin", RISCVCPU, cfg.ext_zfhmin, false), > DEFINE_PROP_BOOL("Zve32f", RISCVCPU, cfg.ext_zve32f, false), > diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h > index fe80caeec0..7bd2fd26d6 100644 > --- a/target/riscv/cpu.h > +++ b/target/riscv/cpu.h > @@ -368,6 +368,8 @@ struct RISCVCPUConfig { > bool ext_counters; > bool ext_ifencei; > bool ext_icsr; > + bool ext_icbom; > + bool ext_icboz; > bool ext_zfh; > bool ext_zfhmin; > bool ext_zve32f; > @@ -382,6 +384,7 @@ struct RISCVCPUConfig { > char *vext_spec; > uint16_t vlen; > uint16_t elen; > + uint16_t cbozlen; > bool mmu; > bool pmp; > bool epmp; > diff --git a/target/riscv/helper.h b/target/riscv/helper.h > index 72cc2582f4..ef1944da8f 100644 > --- a/target/riscv/helper.h > +++ b/target/riscv/helper.h > @@ -92,6 +92,11 @@ DEF_HELPER_FLAGS_2(fcvt_h_l, TCG_CALL_NO_RWG, i64, env, > tl) > DEF_HELPER_FLAGS_2(fcvt_h_lu, TCG_CALL_NO_RWG, i64, env, tl) > DEF_HELPER_FLAGS_1(fclass_h, TCG_CALL_NO_RWG_SE, tl, i64) > > +/* Cache-block operations */ > +DEF_HELPER_2(cbo_clean_flush, void, env, tl) > +DEF_HELPER_2(cbo_inval, void, env, tl) > +DEF_HELPER_2(cbo_zero, void, env, tl) > + > /* Special functions */ > DEF_HELPER_2(csrr, tl, env, int) > DEF_HELPER_3(csrw, void, env, int, tl) > diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode > index 5bbedc254c..d5f8329970 100644 > --- a/target/riscv/insn32.decode > +++ b/target/riscv/insn32.decode > @@ -128,6 +128,7 @@ addi ............ ..... 000 ..... 0010011 @i > slti ............ ..... 010 ..... 0010011 @i > sltiu ............ ..... 011 ..... 0010011 @i > xori ............ ..... 100 ..... 0010011 @i > +# cbo.prefetch_{i,r,m} instructions are ori with rd=x0 and not decoded. > ori ............ ..... 110 ..... 0010011 @i > andi ............ ..... 111 ..... 0010011 @i > slli 00000. ...... ..... 001 ..... 0010011 @sh > @@ -168,7 +169,20 @@ sraw 0100000 ..... ..... 101 ..... 0111011 @r > > # *** RV128I Base Instruction Set (in addition to RV64I) *** > ldu ............ ..... 111 ..... 0000011 @i > -lq ............ ..... 010 ..... 0001111 @i > +{ > + [ > + # *** RV32 Zicbom Standard Extension *** > + cbo_clean 0000000 00001 ..... 010 00000 0001111 @sfence_vm > + cbo_flush 0000000 00010 ..... 010 00000 0001111 @sfence_vm > + cbo_inval 0000000 00000 ..... 010 00000 0001111 @sfence_vm > + > + # *** RV32 Zicboz Standard Extension *** > + cbo_zero 0000000 00100 ..... 010 00000 0001111 @sfence_vm > + ] > + > + # *** RVI128 lq *** > + lq ............ ..... 010 ..... 0001111 @i > +} > sq ............ ..... 100 ..... 0100011 @s > addid ............ ..... 000 ..... 1011011 @i > sllid 000000 ...... ..... 001 ..... 1011011 @sh6 > diff --git a/target/riscv/insn_trans/trans_rvzicbo.c.inc > b/target/riscv/insn_trans/trans_rvzicbo.c.inc > new file mode 100644 > index 0000000000..35c277261f > --- /dev/null > +++ b/target/riscv/insn_trans/trans_rvzicbo.c.inc > @@ -0,0 +1,57 @@ > +/* > + * RISC-V translation routines for the RISC-V CBO Extension. > + * > + * Copyright (c) 2021 Philipp Tomsich, philipp.tomsich@vrull.eu > + * > + * This program is free software; you can redistribute it and/or modify it > + * under the terms and conditions of the GNU General Public License, > + * version 2 or later, as published by the Free Software Foundation. > + * > + * This program is distributed in the hope it will be useful, but WITHOUT > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License > for > + * more details. > + * > + * You should have received a copy of the GNU General Public License > along with > + * this program. If not, see <http://www.gnu.org/licenses/>. > + */ > + > +#define REQUIRE_ZICBOM(ctx) do { \ > + if (!RISCV_CPU(ctx->cs)->cfg.ext_icbom) { \ > + return false; \ > + } \ > +} while (0) > The exception semantics seem to be broken here: if Zicbom is not implemented, but the requirements for lq (i.e. rv128) are satisfied, then this needs to be passed on to lq: "lq zero, 0(rs1)" is still expected to raise exceptions based on the permissions for the address at 0(rs1). There are multiple ways to do this, including: 1) perform a tail-call to trans_lq, in case Zicbom is not enabled (instead of just returning false); 2) use the table-based dispatch (added for XVentanaCondOps) and hook a Zicbom disptacher before the RVI dispatcher: if Zicbom then falls through, the RVI dispatcher would drop into trans_lq; > + > +#define REQUIRE_ZICBOZ(ctx) do { \ > + if (!RISCV_CPU(ctx->cs)->cfg.ext_icboz) { \ > + return false; \ > + } \ > +} while (0) > + > +static bool trans_cbo_clean(DisasContext *ctx, arg_cbo_clean *a) > +{ > + REQUIRE_ZICBOM(ctx); > + gen_helper_cbo_clean_flush(cpu_env, cpu_gpr[a->rs1]); > + return true; > +} > + > +static bool trans_cbo_flush(DisasContext *ctx, arg_cbo_flush *a) > +{ > + REQUIRE_ZICBOM(ctx); > + gen_helper_cbo_clean_flush(cpu_env, cpu_gpr[a->rs1]); > + return true; > +} > + > +static bool trans_cbo_inval(DisasContext *ctx, arg_cbo_inval *a) > +{ > + REQUIRE_ZICBOM(ctx); > + gen_helper_cbo_inval(cpu_env, cpu_gpr[a->rs1]); > + return true; > +} > + > +static bool trans_cbo_zero(DisasContext *ctx, arg_cbo_zero *a) > +{ > + REQUIRE_ZICBOZ(ctx); > + gen_helper_cbo_zero(cpu_env, cpu_gpr[a->rs1]); > + return true; > +} > diff --git a/target/riscv/op_helper.c b/target/riscv/op_helper.c > index 1a75ba11e6..21762bc2e0 100644 > --- a/target/riscv/op_helper.c > +++ b/target/riscv/op_helper.c > @@ -3,6 +3,7 @@ > * > * Copyright (c) 2016-2017 Sagar Karandikar, sagark@eecs.berkeley.edu > * Copyright (c) 2017-2018 SiFive, Inc. > + * Copyright (c) 2022 VRULL GmbH > * > * This program is free software; you can redistribute it and/or modify it > * under the terms and conditions of the GNU General Public License, > @@ -114,6 +115,99 @@ target_ulong helper_csrrw_i128(CPURISCVState *env, > int csr, > return int128_getlo(rv); > } > > + > +/* helper_zicbo_envcfg > + * > + * Raise virtual exceptions and illegal instruction exceptions for > + * Zicbo[mz] instructions based on the settings of [mhs]envcfg as > + * specified in section 2.5.1 of the CMO specification. > + */ > +static void helper_zicbo_envcfg(CPURISCVState *env, target_ulong envbits, > + uintptr_t ra) > +{ > +#ifndef CONFIG_USER_ONLY > + /* Check for virtual instruction exceptions first, as we don't see > + * VU and VS reflected in env->priv (these are just the translated > + * U and S stated with virtualisation enabled. > + */ > + if (riscv_cpu_virt_enabled(env) && > + (((env->priv < PRV_H) && !get_field(env->henvcfg, envbits)) || > + ((env->priv < PRV_S) && !get_field(env->senvcfg, envbits)))) { > + riscv_raise_exception(env, RISCV_EXCP_VIRT_INSTRUCTION_FAULT, ra); > + } > + > + if (((env->priv < PRV_M) && !get_field(env->menvcfg, envbits)) || > + ((env->priv < PRV_S) && !get_field(env->senvcfg, envbits))) { > + riscv_raise_exception(env, RISCV_EXCP_ILLEGAL_INST, ra); > + } > +#endif > +} > + > +/* helper_zicbom_access > + * > + * Check access permissions (LOAD, STORE or FETCH as specified in section > + * 2.5.2 of the CMO specification) for Zicbom, raising either store > + * page-fault (non-virtualised) or store guest-page fault (virtualised). > + */ > +static void helper_zicbom_access(CPURISCVState *env, target_ulong address, > + uintptr_t ra) > +{ > + void* phost; > + int ret = TLB_INVALID_MASK; > + MMUAccessType access_type = MMU_DATA_LOAD; > + > + while (ret == TLB_INVALID_MASK && access_type <= MMU_INST_FETCH) { > + ret = probe_access_flags(env, address, access_type++, > + cpu_mmu_index(env, false), > + true, &phost, ra); > + } > + > + if (ret == TLB_INVALID_MASK) { > + uint32_t exc = RISCV_EXCP_STORE_PAGE_FAULT; > + > +#ifndef CONFIG_USER_ONLY > + /* User-mode emulation does not have virtualisation. */ > + if (riscv_cpu_virt_enabled(env)) { > + exc = RISCV_EXCP_STORE_GUEST_AMO_ACCESS_FAULT; > + } > +#endif > + riscv_raise_exception(env, exc, ra); > + } > +} > + > +void helper_cbo_clean_flush(CPURISCVState *env, target_ulong address) > +{ > + uintptr_t ra = GETPC(); > + helper_zicbo_envcfg(env, MENVCFG_CBCFE, ra); > + helper_zicbom_access(env, address, ra); > +} > + > +void helper_cbo_inval(CPURISCVState *env, target_ulong address) > +{ > + uintptr_t ra = GETPC(); > + helper_zicbo_envcfg(env, MENVCFG_CBIE, ra); > + helper_zicbom_access(env, address, ra); > +} > + > +void helper_cbo_zero(CPURISCVState *env, target_ulong address) > +{ > + uintptr_t ra = GETPC(); > + helper_zicbo_envcfg(env, MENVCFG_CBZE, ra); > + > + /* Get the size of the cache block for zero instructions. */ > + RISCVCPU *cpu = env_archcpu(env); > + uint16_t cbozlen = cpu->cfg.cbozlen; > + > + /* Mask off low-bits to align-down to the cache-block. */ > + address &= ~(cbozlen - 1); > + > + void* mem = probe_access(env, address, cbozlen, MMU_DATA_STORE, > + cpu_mmu_index(env, false), GETPC()); > + > + /* Zero the block */ > + memset(mem, 0, cbozlen); > +} > + > #ifndef CONFIG_USER_ONLY > > target_ulong helper_sret(CPURISCVState *env) > diff --git a/target/riscv/translate.c b/target/riscv/translate.c > index eaf5a72c81..0ee2ce85ec 100644 > --- a/target/riscv/translate.c > +++ b/target/riscv/translate.c > @@ -861,6 +861,7 @@ static uint32_t opcode_at(DisasContextBase *dcbase, > target_ulong pc) > #include "insn_trans/trans_rvv.c.inc" > #include "insn_trans/trans_rvb.c.inc" > #include "insn_trans/trans_rvzfh.c.inc" > +#include "insn_trans/trans_rvzicbo.c.inc" > #include "insn_trans/trans_privileged.c.inc" > #include "insn_trans/trans_xventanacondops.c.inc" > > -- > 2.35.1 > > [-- Attachment #2: Type: text/html, Size: 15992 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v3] target/riscv: Enable Zicbo[m,z,p] instructions 2022-02-10 16:48 ` Philipp Tomsich @ 2022-02-10 22:10 ` Richard Henderson 0 siblings, 0 replies; 5+ messages in thread From: Richard Henderson @ 2022-02-10 22:10 UTC (permalink / raw) To: Philipp Tomsich, Christoph Muellner Cc: Weiwei Li, qemu-riscv, Anup Patel, Bin Meng, Atish Patra, qemu-devel, Alistair Francis, Palmer Dabbelt, Frédéric Pétrot On 2/11/22 03:48, Philipp Tomsich wrote: > -lq ............ ..... 010 ..... 0001111 @i > +{ > + [ > + # *** RV32 Zicbom Standard Extension *** > + cbo_clean 0000000 00001 ..... 010 00000 0001111 @sfence_vm > + cbo_flush 0000000 00010 ..... 010 00000 0001111 @sfence_vm > + cbo_inval 0000000 00000 ..... 010 00000 0001111 @sfence_vm > + > + # *** RV32 Zicboz Standard Extension *** > + cbo_zero 0000000 00100 ..... 010 00000 0001111 @sfence_vm > + ] > + > + # *** RVI128 lq *** > + lq ............ ..... 010 ..... 0001111 @i > +} ... > +#define REQUIRE_ZICBOM(ctx) do { \ > + if (!RISCV_CPU(ctx->cs)->cfg.ext_icbom) { \ > + return false; \ > + } \ > +} while (0) > > > The exception semantics seem to be broken here: if Zicbom is not implemented, but the > requirements for lq (i.e. rv128) are satisfied, then this needs to be passed on to lq: "lq > zero, 0(rs1)" is still expected to raise exceptions based on the permissions for the > address at 0(rs1). > > There are multiple ways to do this, including: > 1) perform a tail-call to trans_lq, in case Zicbom is not enabled (instead of just > returning false); > 2) use the table-based dispatch (added for XVentanaCondOps) and hook a Zicbom > disptacher before the RVI dispatcher: if Zicbom then falls through, the RVI dispatcher > would drop into trans_lq; No, returning false will cause the next pattern in the { } group to be matched. No need for other workarounds. r~ ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v3] target/riscv: Enable Zicbo[m,z,p] instructions 2022-02-10 16:34 [PATCH v3] target/riscv: Enable Zicbo[m,z,p] instructions Christoph Muellner 2022-02-10 16:48 ` Philipp Tomsich @ 2022-02-11 2:41 ` Weiwei Li 2022-02-16 13:51 ` Christoph Müllner 1 sibling, 1 reply; 5+ messages in thread From: Weiwei Li @ 2022-02-11 2:41 UTC (permalink / raw) To: Christoph Muellner, Atish Patra, Anup Patel, Frédéric Pétrot, Palmer Dabbelt, Alistair Francis, Bin Meng, qemu-riscv, qemu-devel, Philipp Tomsich, Richard Henderson, Weiwei Li 在 2022/2/11 上午12:34, Christoph Muellner 写道: > The RISC-V base cache management operation ISA extension has been > ratified [1]. This patch adds support for the defined instructions. > > The cmo.prefetch instructions are nops for QEMU (no emulation of the memory > hierarchy, no illegal instructions, no permission faults, no traps), > therefore there's only a comment where they would be decoded. > > The other cbo* instructions are moved into an overlap group to > resolve the overlapping pattern with the LQ instruction. > The cbo.zero zeros a configurable amount of bytes. > Similar to other extensions (e.g. atomic instructions), > the trap behavior is limited such, that only the page permissions > are checked (ignoring other optional protection mechanisms like > PMA or PMP). > > [1] https://wiki.riscv.org/display/TECH/Recently+Ratified+Extensions > > v3: > - Enable by default (like zb*) > - Rename flags Zicbo* -> zicbo* (like zb*) > - Rename ext_zicbo* -> ext_icbo* (like ext_icsr) > - Rename trans_zicbo.c.inc -> trans_rvzicbo.c.inc (like all others) > - Simplify prefetch instruction support to a single comment > - Rebase on top of github-alistair23/riscv-to-apply.next plus the > Priv v1.12 series from github-atishp04/priv_1_12_support_v3 > > v2: > - Fix overlapping instruction encoding with LQ instructions > - Drop CSR related changes and rebase on Priv 1.12 patchset > > Co-developed-by: Philipp Tomsich <philipp.tomsich@vrull.eu> > Signed-off-by: Christoph Muellner <cmuellner@linux.com> > --- > target/riscv/cpu.c | 3 + > target/riscv/cpu.h | 3 + > target/riscv/helper.h | 5 ++ > target/riscv/insn32.decode | 16 +++- > target/riscv/insn_trans/trans_rvzicbo.c.inc | 57 +++++++++++++ > target/riscv/op_helper.c | 94 +++++++++++++++++++++ > target/riscv/translate.c | 1 + > 7 files changed, 178 insertions(+), 1 deletion(-) > create mode 100644 target/riscv/insn_trans/trans_rvzicbo.c.inc > > diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c > index 39ffb883fc..cbd0a34318 100644 > --- a/target/riscv/cpu.c > +++ b/target/riscv/cpu.c > @@ -764,6 +764,9 @@ static Property riscv_cpu_properties[] = { > DEFINE_PROP_BOOL("Counters", RISCVCPU, cfg.ext_counters, true), > DEFINE_PROP_BOOL("Zifencei", RISCVCPU, cfg.ext_ifencei, true), > DEFINE_PROP_BOOL("Zicsr", RISCVCPU, cfg.ext_icsr, true), > + DEFINE_PROP_BOOL("zicbom", RISCVCPU, cfg.ext_icbom, true), > + DEFINE_PROP_BOOL("zicboz", RISCVCPU, cfg.ext_icboz, true), > + DEFINE_PROP_UINT16("cbozlen", RISCVCPU, cfg.cbozlen, 64), > DEFINE_PROP_BOOL("Zfh", RISCVCPU, cfg.ext_zfh, false), > DEFINE_PROP_BOOL("Zfhmin", RISCVCPU, cfg.ext_zfhmin, false), > DEFINE_PROP_BOOL("Zve32f", RISCVCPU, cfg.ext_zve32f, false), > diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h > index fe80caeec0..7bd2fd26d6 100644 > --- a/target/riscv/cpu.h > +++ b/target/riscv/cpu.h > @@ -368,6 +368,8 @@ struct RISCVCPUConfig { > bool ext_counters; > bool ext_ifencei; > bool ext_icsr; > + bool ext_icbom; > + bool ext_icboz; > bool ext_zfh; > bool ext_zfhmin; > bool ext_zve32f; > @@ -382,6 +384,7 @@ struct RISCVCPUConfig { > char *vext_spec; > uint16_t vlen; > uint16_t elen; > + uint16_t cbozlen; > bool mmu; > bool pmp; > bool epmp; > diff --git a/target/riscv/helper.h b/target/riscv/helper.h > index 72cc2582f4..ef1944da8f 100644 > --- a/target/riscv/helper.h > +++ b/target/riscv/helper.h > @@ -92,6 +92,11 @@ DEF_HELPER_FLAGS_2(fcvt_h_l, TCG_CALL_NO_RWG, i64, env, tl) > DEF_HELPER_FLAGS_2(fcvt_h_lu, TCG_CALL_NO_RWG, i64, env, tl) > DEF_HELPER_FLAGS_1(fclass_h, TCG_CALL_NO_RWG_SE, tl, i64) > > +/* Cache-block operations */ > +DEF_HELPER_2(cbo_clean_flush, void, env, tl) > +DEF_HELPER_2(cbo_inval, void, env, tl) > +DEF_HELPER_2(cbo_zero, void, env, tl) > + > /* Special functions */ > DEF_HELPER_2(csrr, tl, env, int) > DEF_HELPER_3(csrw, void, env, int, tl) > diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode > index 5bbedc254c..d5f8329970 100644 > --- a/target/riscv/insn32.decode > +++ b/target/riscv/insn32.decode > @@ -128,6 +128,7 @@ addi ............ ..... 000 ..... 0010011 @i > slti ............ ..... 010 ..... 0010011 @i > sltiu ............ ..... 011 ..... 0010011 @i > xori ............ ..... 100 ..... 0010011 @i > +# cbo.prefetch_{i,r,m} instructions are ori with rd=x0 and not decoded. > ori ............ ..... 110 ..... 0010011 @i > andi ............ ..... 111 ..... 0010011 @i > slli 00000. ...... ..... 001 ..... 0010011 @sh > @@ -168,7 +169,20 @@ sraw 0100000 ..... ..... 101 ..... 0111011 @r > > # *** RV128I Base Instruction Set (in addition to RV64I) *** > ldu ............ ..... 111 ..... 0000011 @i > -lq ............ ..... 010 ..... 0001111 @i > +{ > + [ > + # *** RV32 Zicbom Standard Extension *** > + cbo_clean 0000000 00001 ..... 010 00000 0001111 @sfence_vm > + cbo_flush 0000000 00010 ..... 010 00000 0001111 @sfence_vm > + cbo_inval 0000000 00000 ..... 010 00000 0001111 @sfence_vm > + > + # *** RV32 Zicboz Standard Extension *** > + cbo_zero 0000000 00100 ..... 010 00000 0001111 @sfence_vm > + ] > + > + # *** RVI128 lq *** > + lq ............ ..... 010 ..... 0001111 @i > +} > sq ............ ..... 100 ..... 0100011 @s > addid ............ ..... 000 ..... 1011011 @i > sllid 000000 ...... ..... 001 ..... 1011011 @sh6 > diff --git a/target/riscv/insn_trans/trans_rvzicbo.c.inc b/target/riscv/insn_trans/trans_rvzicbo.c.inc > new file mode 100644 > index 0000000000..35c277261f > --- /dev/null > +++ b/target/riscv/insn_trans/trans_rvzicbo.c.inc > @@ -0,0 +1,57 @@ > +/* > + * RISC-V translation routines for the RISC-V CBO Extension. > + * > + * Copyright (c) 2021 Philipp Tomsich, philipp.tomsich@vrull.eu > + * > + * This program is free software; you can redistribute it and/or modify it > + * under the terms and conditions of the GNU General Public License, > + * version 2 or later, as published by the Free Software Foundation. > + * > + * This program is distributed in the hope it will be useful, but WITHOUT > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for > + * more details. > + * > + * You should have received a copy of the GNU General Public License along with > + * this program. If not, see <http://www.gnu.org/licenses/>. > + */ > + > +#define REQUIRE_ZICBOM(ctx) do { \ > + if (!RISCV_CPU(ctx->cs)->cfg.ext_icbom) { \ > + return false; \ > + } \ > +} while (0) > + It's better to use ctx->cfg_ptr->ext_icbom instead. > +#define REQUIRE_ZICBOZ(ctx) do { \ > + if (!RISCV_CPU(ctx->cs)->cfg.ext_icboz) { \ > + return false; \ > + } \ > +} while (0) > + > +static bool trans_cbo_clean(DisasContext *ctx, arg_cbo_clean *a) > +{ > + REQUIRE_ZICBOM(ctx); > + gen_helper_cbo_clean_flush(cpu_env, cpu_gpr[a->rs1]); > + return true; > +} > + > +static bool trans_cbo_flush(DisasContext *ctx, arg_cbo_flush *a) > +{ > + REQUIRE_ZICBOM(ctx); > + gen_helper_cbo_clean_flush(cpu_env, cpu_gpr[a->rs1]); > + return true; > +} > + > +static bool trans_cbo_inval(DisasContext *ctx, arg_cbo_inval *a) > +{ > + REQUIRE_ZICBOM(ctx); > + gen_helper_cbo_inval(cpu_env, cpu_gpr[a->rs1]); > + return true; > +} > + > +static bool trans_cbo_zero(DisasContext *ctx, arg_cbo_zero *a) > +{ > + REQUIRE_ZICBOZ(ctx); > + gen_helper_cbo_zero(cpu_env, cpu_gpr[a->rs1]); > + return true; > +} > diff --git a/target/riscv/op_helper.c b/target/riscv/op_helper.c > index 1a75ba11e6..21762bc2e0 100644 > --- a/target/riscv/op_helper.c > +++ b/target/riscv/op_helper.c > @@ -3,6 +3,7 @@ > * > * Copyright (c) 2016-2017 Sagar Karandikar, sagark@eecs.berkeley.edu > * Copyright (c) 2017-2018 SiFive, Inc. > + * Copyright (c) 2022 VRULL GmbH > * > * This program is free software; you can redistribute it and/or modify it > * under the terms and conditions of the GNU General Public License, > @@ -114,6 +115,99 @@ target_ulong helper_csrrw_i128(CPURISCVState *env, int csr, > return int128_getlo(rv); > } > > + > +/* helper_zicbo_envcfg > + * > + * Raise virtual exceptions and illegal instruction exceptions for > + * Zicbo[mz] instructions based on the settings of [mhs]envcfg as > + * specified in section 2.5.1 of the CMO specification. > + */ > +static void helper_zicbo_envcfg(CPURISCVState *env, target_ulong envbits, > + uintptr_t ra) > +{ > +#ifndef CONFIG_USER_ONLY > + /* Check for virtual instruction exceptions first, as we don't see > + * VU and VS reflected in env->priv (these are just the translated > + * U and S stated with virtualisation enabled. > + */ > + if (riscv_cpu_virt_enabled(env) && > + (((env->priv < PRV_H) && !get_field(env->henvcfg, envbits)) || > + ((env->priv < PRV_S) && !get_field(env->senvcfg, envbits)))) { > + riscv_raise_exception(env, RISCV_EXCP_VIRT_INSTRUCTION_FAULT, ra); > + } > + > + if (((env->priv < PRV_M) && !get_field(env->menvcfg, envbits)) || > + ((env->priv < PRV_S) && !get_field(env->senvcfg, envbits))) { > + riscv_raise_exception(env, RISCV_EXCP_ILLEGAL_INST, ra); > + } > +#endif > +} > + > +/* helper_zicbom_access > + * > + * Check access permissions (LOAD, STORE or FETCH as specified in section > + * 2.5.2 of the CMO specification) for Zicbom, raising either store > + * page-fault (non-virtualised) or store guest-page fault (virtualised). > + */ > +static void helper_zicbom_access(CPURISCVState *env, target_ulong address, > + uintptr_t ra) > +{ > + void* phost; > + int ret = TLB_INVALID_MASK; > + MMUAccessType access_type = MMU_DATA_LOAD; > + > + while (ret == TLB_INVALID_MASK && access_type <= MMU_INST_FETCH) { > + ret = probe_access_flags(env, address, access_type++, > + cpu_mmu_index(env, false), > + true, &phost, ra); Using probe_access_flags here will make the size of pmp check for final physical address to be zero. Maybe it's better to use probe_access with cbozlen as size and address aligned to cbozlen. > + } > + > + if (ret == TLB_INVALID_MASK) { > + uint32_t exc = RISCV_EXCP_STORE_PAGE_FAULT; > + > +#ifndef CONFIG_USER_ONLY > + /* User-mode emulation does not have virtualisation. */ > + if (riscv_cpu_virt_enabled(env)) { > + exc = RISCV_EXCP_STORE_GUEST_AMO_ACCESS_FAULT; > + } > +#endif > + riscv_raise_exception(env, exc, ra); If pmp violation triggers, the exception type should be RISCV_EXCP_STORE_AMO_ACCESS_FAULT. And It seems that not all exception triggered when virtualisation is enabled is guest page fault(RISCV_EXCP_STORE_GUEST_*_ACCESS_FAULT) . The exception triggered in first stage(G-Stage) is normal page fault. Regards, Weiwei Li > + } > +} > + > +void helper_cbo_clean_flush(CPURISCVState *env, target_ulong address) > +{ > + uintptr_t ra = GETPC(); > + helper_zicbo_envcfg(env, MENVCFG_CBCFE, ra); > + helper_zicbom_access(env, address, ra); > +} > + > +void helper_cbo_inval(CPURISCVState *env, target_ulong address) > +{ > + uintptr_t ra = GETPC(); > + helper_zicbo_envcfg(env, MENVCFG_CBIE, ra); > + helper_zicbom_access(env, address, ra); > +} > + > +void helper_cbo_zero(CPURISCVState *env, target_ulong address) > +{ > + uintptr_t ra = GETPC(); > + helper_zicbo_envcfg(env, MENVCFG_CBZE, ra); > + > + /* Get the size of the cache block for zero instructions. */ > + RISCVCPU *cpu = env_archcpu(env); > + uint16_t cbozlen = cpu->cfg.cbozlen; > + > + /* Mask off low-bits to align-down to the cache-block. */ > + address &= ~(cbozlen - 1); > + > + void* mem = probe_access(env, address, cbozlen, MMU_DATA_STORE, > + cpu_mmu_index(env, false), GETPC()); > + > + /* Zero the block */ > + memset(mem, 0, cbozlen); > +} > + > #ifndef CONFIG_USER_ONLY > > target_ulong helper_sret(CPURISCVState *env) > diff --git a/target/riscv/translate.c b/target/riscv/translate.c > index eaf5a72c81..0ee2ce85ec 100644 > --- a/target/riscv/translate.c > +++ b/target/riscv/translate.c > @@ -861,6 +861,7 @@ static uint32_t opcode_at(DisasContextBase *dcbase, target_ulong pc) > #include "insn_trans/trans_rvv.c.inc" > #include "insn_trans/trans_rvb.c.inc" > #include "insn_trans/trans_rvzfh.c.inc" > +#include "insn_trans/trans_rvzicbo.c.inc" > #include "insn_trans/trans_privileged.c.inc" > #include "insn_trans/trans_xventanacondops.c.inc" > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v3] target/riscv: Enable Zicbo[m,z,p] instructions 2022-02-11 2:41 ` Weiwei Li @ 2022-02-16 13:51 ` Christoph Müllner 0 siblings, 0 replies; 5+ messages in thread From: Christoph Müllner @ 2022-02-16 13:51 UTC (permalink / raw) To: Weiwei Li Cc: open list:RISC-V, Anup Patel, Bin Meng, Atish Patra, qemu-devel@nongnu.org Developers, Philipp Tomsich, Richard Henderson, Alistair Francis, Palmer Dabbelt, Frédéric Pétrot [-- Attachment #1: Type: text/plain, Size: 15908 bytes --] On Fri, Feb 11, 2022 at 3:41 AM Weiwei Li <liweiwei@iscas.ac.cn> wrote: > > 在 2022/2/11 上午12:34, Christoph Muellner 写道: > > The RISC-V base cache management operation ISA extension has been > > ratified [1]. This patch adds support for the defined instructions. > > > > The cmo.prefetch instructions are nops for QEMU (no emulation of the > memory > > hierarchy, no illegal instructions, no permission faults, no traps), > > therefore there's only a comment where they would be decoded. > > > > The other cbo* instructions are moved into an overlap group to > > resolve the overlapping pattern with the LQ instruction. > > The cbo.zero zeros a configurable amount of bytes. > > Similar to other extensions (e.g. atomic instructions), > > the trap behavior is limited such, that only the page permissions > > are checked (ignoring other optional protection mechanisms like > > PMA or PMP). > > > > [1] https://wiki.riscv.org/display/TECH/Recently+Ratified+Extensions > > > > v3: > > - Enable by default (like zb*) > > - Rename flags Zicbo* -> zicbo* (like zb*) > > - Rename ext_zicbo* -> ext_icbo* (like ext_icsr) > > - Rename trans_zicbo.c.inc -> trans_rvzicbo.c.inc (like all others) > > - Simplify prefetch instruction support to a single comment > > - Rebase on top of github-alistair23/riscv-to-apply.next plus the > > Priv v1.12 series from github-atishp04/priv_1_12_support_v3 > > > > v2: > > - Fix overlapping instruction encoding with LQ instructions > > - Drop CSR related changes and rebase on Priv 1.12 patchset > > > > Co-developed-by: Philipp Tomsich <philipp.tomsich@vrull.eu> > > Signed-off-by: Christoph Muellner <cmuellner@linux.com> > > --- > > target/riscv/cpu.c | 3 + > > target/riscv/cpu.h | 3 + > > target/riscv/helper.h | 5 ++ > > target/riscv/insn32.decode | 16 +++- > > target/riscv/insn_trans/trans_rvzicbo.c.inc | 57 +++++++++++++ > > target/riscv/op_helper.c | 94 +++++++++++++++++++++ > > target/riscv/translate.c | 1 + > > 7 files changed, 178 insertions(+), 1 deletion(-) > > create mode 100644 target/riscv/insn_trans/trans_rvzicbo.c.inc > > > > diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c > > index 39ffb883fc..cbd0a34318 100644 > > --- a/target/riscv/cpu.c > > +++ b/target/riscv/cpu.c > > @@ -764,6 +764,9 @@ static Property riscv_cpu_properties[] = { > > DEFINE_PROP_BOOL("Counters", RISCVCPU, cfg.ext_counters, true), > > DEFINE_PROP_BOOL("Zifencei", RISCVCPU, cfg.ext_ifencei, true), > > DEFINE_PROP_BOOL("Zicsr", RISCVCPU, cfg.ext_icsr, true), > > + DEFINE_PROP_BOOL("zicbom", RISCVCPU, cfg.ext_icbom, true), > > + DEFINE_PROP_BOOL("zicboz", RISCVCPU, cfg.ext_icboz, true), > > + DEFINE_PROP_UINT16("cbozlen", RISCVCPU, cfg.cbozlen, 64), > > DEFINE_PROP_BOOL("Zfh", RISCVCPU, cfg.ext_zfh, false), > > DEFINE_PROP_BOOL("Zfhmin", RISCVCPU, cfg.ext_zfhmin, false), > > DEFINE_PROP_BOOL("Zve32f", RISCVCPU, cfg.ext_zve32f, false), > > diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h > > index fe80caeec0..7bd2fd26d6 100644 > > --- a/target/riscv/cpu.h > > +++ b/target/riscv/cpu.h > > @@ -368,6 +368,8 @@ struct RISCVCPUConfig { > > bool ext_counters; > > bool ext_ifencei; > > bool ext_icsr; > > + bool ext_icbom; > > + bool ext_icboz; > > bool ext_zfh; > > bool ext_zfhmin; > > bool ext_zve32f; > > @@ -382,6 +384,7 @@ struct RISCVCPUConfig { > > char *vext_spec; > > uint16_t vlen; > > uint16_t elen; > > + uint16_t cbozlen; > > bool mmu; > > bool pmp; > > bool epmp; > > diff --git a/target/riscv/helper.h b/target/riscv/helper.h > > index 72cc2582f4..ef1944da8f 100644 > > --- a/target/riscv/helper.h > > +++ b/target/riscv/helper.h > > @@ -92,6 +92,11 @@ DEF_HELPER_FLAGS_2(fcvt_h_l, TCG_CALL_NO_RWG, i64, > env, tl) > > DEF_HELPER_FLAGS_2(fcvt_h_lu, TCG_CALL_NO_RWG, i64, env, tl) > > DEF_HELPER_FLAGS_1(fclass_h, TCG_CALL_NO_RWG_SE, tl, i64) > > > > +/* Cache-block operations */ > > +DEF_HELPER_2(cbo_clean_flush, void, env, tl) > > +DEF_HELPER_2(cbo_inval, void, env, tl) > > +DEF_HELPER_2(cbo_zero, void, env, tl) > > + > > /* Special functions */ > > DEF_HELPER_2(csrr, tl, env, int) > > DEF_HELPER_3(csrw, void, env, int, tl) > > diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode > > index 5bbedc254c..d5f8329970 100644 > > --- a/target/riscv/insn32.decode > > +++ b/target/riscv/insn32.decode > > @@ -128,6 +128,7 @@ addi ............ ..... 000 ..... 0010011 @i > > slti ............ ..... 010 ..... 0010011 @i > > sltiu ............ ..... 011 ..... 0010011 @i > > xori ............ ..... 100 ..... 0010011 @i > > +# cbo.prefetch_{i,r,m} instructions are ori with rd=x0 and not decoded. > > ori ............ ..... 110 ..... 0010011 @i > > andi ............ ..... 111 ..... 0010011 @i > > slli 00000. ...... ..... 001 ..... 0010011 @sh > > @@ -168,7 +169,20 @@ sraw 0100000 ..... ..... 101 ..... 0111011 @r > > > > # *** RV128I Base Instruction Set (in addition to RV64I) *** > > ldu ............ ..... 111 ..... 0000011 @i > > -lq ............ ..... 010 ..... 0001111 @i > > +{ > > + [ > > + # *** RV32 Zicbom Standard Extension *** > > + cbo_clean 0000000 00001 ..... 010 00000 0001111 @sfence_vm > > + cbo_flush 0000000 00010 ..... 010 00000 0001111 @sfence_vm > > + cbo_inval 0000000 00000 ..... 010 00000 0001111 @sfence_vm > > + > > + # *** RV32 Zicboz Standard Extension *** > > + cbo_zero 0000000 00100 ..... 010 00000 0001111 @sfence_vm > > + ] > > + > > + # *** RVI128 lq *** > > + lq ............ ..... 010 ..... 0001111 @i > > +} > > sq ............ ..... 100 ..... 0100011 @s > > addid ............ ..... 000 ..... 1011011 @i > > sllid 000000 ...... ..... 001 ..... 1011011 @sh6 > > diff --git a/target/riscv/insn_trans/trans_rvzicbo.c.inc > b/target/riscv/insn_trans/trans_rvzicbo.c.inc > > new file mode 100644 > > index 0000000000..35c277261f > > --- /dev/null > > +++ b/target/riscv/insn_trans/trans_rvzicbo.c.inc > > @@ -0,0 +1,57 @@ > > +/* > > + * RISC-V translation routines for the RISC-V CBO Extension. > > + * > > + * Copyright (c) 2021 Philipp Tomsich, philipp.tomsich@vrull.eu > > + * > > + * This program is free software; you can redistribute it and/or modify > it > > + * under the terms and conditions of the GNU General Public License, > > + * version 2 or later, as published by the Free Software Foundation. > > + * > > + * This program is distributed in the hope it will be useful, but > WITHOUT > > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public > License for > > + * more details. > > + * > > + * You should have received a copy of the GNU General Public License > along with > > + * this program. If not, see <http://www.gnu.org/licenses/>. > > + */ > > + > > +#define REQUIRE_ZICBOM(ctx) do { \ > > + if (!RISCV_CPU(ctx->cs)->cfg.ext_icbom) { \ > > + return false; \ > > + } \ > > +} while (0) > > + > It's better to use ctx->cfg_ptr->ext_icbom instead. > Ok. > > +#define REQUIRE_ZICBOZ(ctx) do { \ > > + if (!RISCV_CPU(ctx->cs)->cfg.ext_icboz) { \ > > + return false; \ > > + } \ > > +} while (0) > > + > > +static bool trans_cbo_clean(DisasContext *ctx, arg_cbo_clean *a) > > +{ > > + REQUIRE_ZICBOM(ctx); > > + gen_helper_cbo_clean_flush(cpu_env, cpu_gpr[a->rs1]); > > + return true; > > +} > > + > > +static bool trans_cbo_flush(DisasContext *ctx, arg_cbo_flush *a) > > +{ > > + REQUIRE_ZICBOM(ctx); > > + gen_helper_cbo_clean_flush(cpu_env, cpu_gpr[a->rs1]); > > + return true; > > +} > > + > > +static bool trans_cbo_inval(DisasContext *ctx, arg_cbo_inval *a) > > +{ > > + REQUIRE_ZICBOM(ctx); > > + gen_helper_cbo_inval(cpu_env, cpu_gpr[a->rs1]); > > + return true; > > +} > > + > > +static bool trans_cbo_zero(DisasContext *ctx, arg_cbo_zero *a) > > +{ > > + REQUIRE_ZICBOZ(ctx); > > + gen_helper_cbo_zero(cpu_env, cpu_gpr[a->rs1]); > > + return true; > > +} > > diff --git a/target/riscv/op_helper.c b/target/riscv/op_helper.c > > index 1a75ba11e6..21762bc2e0 100644 > > --- a/target/riscv/op_helper.c > > +++ b/target/riscv/op_helper.c > > @@ -3,6 +3,7 @@ > > * > > * Copyright (c) 2016-2017 Sagar Karandikar, sagark@eecs.berkeley.edu > > * Copyright (c) 2017-2018 SiFive, Inc. > > + * Copyright (c) 2022 VRULL GmbH > > * > > * This program is free software; you can redistribute it and/or > modify it > > * under the terms and conditions of the GNU General Public License, > > @@ -114,6 +115,99 @@ target_ulong helper_csrrw_i128(CPURISCVState *env, > int csr, > > return int128_getlo(rv); > > } > > > > + > > +/* helper_zicbo_envcfg > > + * > > + * Raise virtual exceptions and illegal instruction exceptions for > > + * Zicbo[mz] instructions based on the settings of [mhs]envcfg as > > + * specified in section 2.5.1 of the CMO specification. > > + */ > > +static void helper_zicbo_envcfg(CPURISCVState *env, target_ulong > envbits, > > + uintptr_t ra) > > +{ > > +#ifndef CONFIG_USER_ONLY > > + /* Check for virtual instruction exceptions first, as we don't see > > + * VU and VS reflected in env->priv (these are just the translated > > + * U and S stated with virtualisation enabled. > > + */ > > + if (riscv_cpu_virt_enabled(env) && > > + (((env->priv < PRV_H) && !get_field(env->henvcfg, envbits)) || > > + ((env->priv < PRV_S) && !get_field(env->senvcfg, envbits)))) { > > + riscv_raise_exception(env, RISCV_EXCP_VIRT_INSTRUCTION_FAULT, > ra); > > + } > > + > > + if (((env->priv < PRV_M) && !get_field(env->menvcfg, envbits)) || > > + ((env->priv < PRV_S) && !get_field(env->senvcfg, envbits))) { > > + riscv_raise_exception(env, RISCV_EXCP_ILLEGAL_INST, ra); > > + } > > +#endif > > +} > > + > > +/* helper_zicbom_access > > + * > > + * Check access permissions (LOAD, STORE or FETCH as specified in > section > > + * 2.5.2 of the CMO specification) for Zicbom, raising either store > > + * page-fault (non-virtualised) or store guest-page fault (virtualised). > > + */ > > +static void helper_zicbom_access(CPURISCVState *env, target_ulong > address, > > + uintptr_t ra) > > +{ > > + void* phost; > > + int ret = TLB_INVALID_MASK; > > + MMUAccessType access_type = MMU_DATA_LOAD; > > + > > + while (ret == TLB_INVALID_MASK && access_type <= MMU_INST_FETCH) { > > + ret = probe_access_flags(env, address, access_type++, > > + cpu_mmu_index(env, false), > > + true, &phost, ra); > > Using probe_access_flags here will make the size of pmp check for > final physical address to be zero. > > Maybe it's better to use probe_access with cbozlen as size and address > aligned to cbozlen. > > The current API let's me to decide between: * probe_access() -> implies nonfault == false (-> no control over the faults) * probe_access_flags() -> implies size == 0 (-> access to the whole page) The size value is indeed forwarded to pmp_hart_has_privs(), but there is a comment, which states: If size is unknown (0), assume that all bytes from addr to the end of the page will be accessed. ...and the code below does: pmp_size = -(addr | TARGET_PAGE_MASK); So, indeed we might raise a PMP error outside of the cache line. Given the API restriction, I'll do the following: * Add a new call probe_access_flags_with_size() * Introduce a cbom block size env variable similar to cbozlen. > > > + } > > + > > + if (ret == TLB_INVALID_MASK) { > > + uint32_t exc = RISCV_EXCP_STORE_PAGE_FAULT; > > + > > +#ifndef CONFIG_USER_ONLY > > + /* User-mode emulation does not have virtualisation. */ > > + if (riscv_cpu_virt_enabled(env)) { > > + exc = RISCV_EXCP_STORE_GUEST_AMO_ACCESS_FAULT; > > + } > > +#endif > > + riscv_raise_exception(env, exc, ra); > > If pmp violation triggers, the exception type should be > RISCV_EXCP_STORE_AMO_ACCESS_FAULT. > The probe_access*() API does not allow distinguishing different reasons for a translation error (we just get TLB_INVALID_MASK). Looking into the details of the current PMP implementation in QEMU, I agree that translation errors could be analysed in more detail. ... > And It seems that not all exception triggered when virtualisation is > enabled is guest page fault(RISCV_EXCP_STORE_GUEST_*_ACCESS_FAULT) . > The exception triggered in first stage(G-Stage) is normal page fault. > ...here we have the same issue: The probe_access*() calls don't provide the necessary details of the translation error. And the RISC-V backend does not provide convenient APIs to get better information. However, looking at target/riscv/cpu_helper.c, I can see that the function raise_mmu_exception() creates exactly the faults that we need for access_type==MMU_DATA_STORE. Based on that, here is how I intend to solve the issue: * change the probe_access_flags() loop as mentioned above * in the error-paths (if ret == TLB_INVALID_MASK): call probe_access(STORE, nonfault=false) to create the correct exception Thanks, Christoph > > > + } > > +} > > + > > +void helper_cbo_clean_flush(CPURISCVState *env, target_ulong address) > > +{ > > + uintptr_t ra = GETPC(); > > + helper_zicbo_envcfg(env, MENVCFG_CBCFE, ra); > > + helper_zicbom_access(env, address, ra); > > +} > > + > > +void helper_cbo_inval(CPURISCVState *env, target_ulong address) > > +{ > > + uintptr_t ra = GETPC(); > > + helper_zicbo_envcfg(env, MENVCFG_CBIE, ra); > > + helper_zicbom_access(env, address, ra); > > +} > > + > > +void helper_cbo_zero(CPURISCVState *env, target_ulong address) > > +{ > > + uintptr_t ra = GETPC(); > > + helper_zicbo_envcfg(env, MENVCFG_CBZE, ra); > > + > > + /* Get the size of the cache block for zero instructions. */ > > + RISCVCPU *cpu = env_archcpu(env); > > + uint16_t cbozlen = cpu->cfg.cbozlen; > > + > > + /* Mask off low-bits to align-down to the cache-block. */ > > + address &= ~(cbozlen - 1); > > + > > + void* mem = probe_access(env, address, cbozlen, MMU_DATA_STORE, > > + cpu_mmu_index(env, false), GETPC()); > > + > > + /* Zero the block */ > > + memset(mem, 0, cbozlen); > > +} > > + > > #ifndef CONFIG_USER_ONLY > > > > target_ulong helper_sret(CPURISCVState *env) > > diff --git a/target/riscv/translate.c b/target/riscv/translate.c > > index eaf5a72c81..0ee2ce85ec 100644 > > --- a/target/riscv/translate.c > > +++ b/target/riscv/translate.c > > @@ -861,6 +861,7 @@ static uint32_t opcode_at(DisasContextBase *dcbase, > target_ulong pc) > > #include "insn_trans/trans_rvv.c.inc" > > #include "insn_trans/trans_rvb.c.inc" > > #include "insn_trans/trans_rvzfh.c.inc" > > +#include "insn_trans/trans_rvzicbo.c.inc" > > #include "insn_trans/trans_privileged.c.inc" > > #include "insn_trans/trans_xventanacondops.c.inc" > > > [-- Attachment #2: Type: text/html, Size: 20361 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2022-02-16 13:53 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-02-10 16:34 [PATCH v3] target/riscv: Enable Zicbo[m,z,p] instructions Christoph Muellner 2022-02-10 16:48 ` Philipp Tomsich 2022-02-10 22:10 ` Richard Henderson 2022-02-11 2:41 ` Weiwei Li 2022-02-16 13:51 ` Christoph Müllner
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).