[PATCH v1 0/7] target/riscv: Support SXL32 on RV64 CPU

[PATCH v1 0/7] target/riscv: Support SXL32 on RV64 CPU

[LIU Zhiwei]

From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>

We have implemented UXL32 on QEMU already. It enables us to run RV32
applications on RV64 Linux on QEMU. Similarly, RISCV specification
doesn't limit the SXLEN to be the same with MXLEN. In this patch set,
we will go forward to implement SXL32 on RV64 CPU.

SXL is one filed in mstatus. It defines the SXLEN. After CPU reset,
SXL will be same with MXL in misa.

According to the RISC-V privilege specification, Smode CSR with SXLEN
length will usually be read and written in an SXLEN view, no matter
current XLEN is 32bit or 64bit. Thus if MXLEN=64 and SXLEN=32, M mode
register will read or write them in the same way under RV32 layout.

In this patch set, we carefully process the Smode CSR write and read
for SXL32. This includes explicitly CSR write/read, and implicitly
CSR read such as satp read in PTW(page table walk).

Another thing we do in this patch set is that we introduce a property
of CPU, named sxl32. It provides us a way to override the default sxl64
for RV64 cpu after cpu reset. The reason is OpenSBI currently doesn't
dynamically probe or configure SXLEN. When we want to run RV64 OpenSBI
in M mode and RV32 Linux in S mode, OpenSBI will not prepare the SXL32
environment for RV32 Linux. sxl32 property will also forbidden writing SXL
field in mstatus, which means QEMU can ensure an SXL32 environment for Linux.

We've successfully booted 64-bit OpenSBI and 32-bit Linux system on
RV64 CPU, with a minor fix of Linux.

Tested by follow steps:

 - Prepare Linux rv32 Image && rv64 Image
  Please apply the patch of linux before test. You can get the patch in
https://github.com/romanheros/linux/commit/8444fb3c913d64c20b1e5334a29a66061eb3f5fe.

	$ cd linux
	$ make ARCH=riscv CROSS_COMPILE=riscv32-unknown-linux-gnu- O=../linux-rv32 defconfig
	$ make ARCH=riscv CROSS_COMPILE=riscv32-unknown-linux-gnu- O=../build-rv32 32-bit.config
	$ make ARCH=riscv CROSS_COMPILE=riscv32-unknown-linux-gnu- O=../linux-rv32/ -j $(nproc)
	$ make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- O=../linux-rv64 defconfig
	$ make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- O=../linux-rv64/ -j $(nproc)
	(Get Image in ../linux-rv32/arch/riscv/boot)

 - Prepare QEMU
 	$ git clone https://github.com/romanheros/qemu.git -b riscv-upstream-sxl32-v1
	$ cd qemu
	$ ./configure --target-list="riscv64-softmmu" --enable-slirp
	$ cd build; ninja

- Run 32-bit Linux boot on 64-bit RISC-V with Opensbi 64 :
	$ ./build/qemu-system-riscv64 -cpu rv64,sxl32=on -M virt -nographic \
		-bios opensbi/build/platform/generic/firmware/fw_dynamic.bin \
		-kernel linux-rv32/arch/riscv/boot/Image \
		-append "root=/dev/vda ro console=ttyS0" \
		-drive file=qemu_riscv32_virt_defconfig/images/rootfs.ext2,format=raw,id=hd0 \
		-netdev user,id=net0 -device virtio-net-device,netdev=net0
```
OpenSBI v1.4-112-g7b4bc9d
Build time: 2024-06-17 06:57:42 +0000
Build compiler: gcc version 13.2.0 (gc891d8dc23e-dirty)
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name             : riscv-virtio,qemu
Platform Features         : medeleg
Platform HART Count       : 1
Platform IPI Device       : aclint-mswi
Platform Timer Device     : aclint-mtimer @ 10000000Hz
Platform Console Device   : uart8250
Platform HSM Device       : ---
Platform PMU Device       : ---
Platform Reboot Device    : syscon-reboot
Platform Shutdown Device  : syscon-poweroff
Platform Suspend Device   : ---
Platform CPPC Device      : ---
Firmware Base             : 0x80000000
Firmware Size             : 583 KB
Firmware RW Offset        : 0x80000
Firmware RW Size          : 71 KB
Firmware Heap Offset      : 0x89000
Firmware Heap Size        : 35 KB (total), 2 KB (reserved), 11 KB (used), 21 KB (free)
Firmware Scratch Size     : 4096 B (total), 400 B (used), 3696 B (free)
Runtime SBI Version       : 2.0

Domain0 Name              : root
Domain0 Boot HART         : 0
Domain0 HARTs             : 0*
Domain0 Region00          : 0x0000000000100000-0x0000000000100fff M: (I,R,W) S/U: (R,W)
Domain0 Region01          : 0x0000000010000000-0x0000000010000fff M: (I,R,W) S/U: (R,W)
Domain0 Region02          : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: ()
Domain0 Region03          : 0x0000000080080000-0x000000008009ffff M: (R,W) S/U: ()
Domain0 Region04          : 0x0000000080000000-0x000000008007ffff M: (R,X) S/U: ()
Domain0 Region05          : 0x000000000c400000-0x000000000c5fffff M: (I,R,W) S/U: (R,W)
Domain0 Region06          : 0x000000000c000000-0x000000000c3fffff M: (I,R,W) S/U: (R,W)
Domain0 Region07          : 0x0000000000000000-0xffffffffffffffff M: () S/U: (R,W,X)
Domain0 Next Address      : 0x0000000080200000
Domain0 Next Arg1         : 0x0000000087e00000
Domain0 Next Mode         : S-mode
Domain0 SysReset          : yes
Domain0 SysSuspend        : yes

Boot HART ID              : 0
Boot HART Domain          : root
Boot HART Priv Version    : v1.12
Boot HART Base ISA        : rv64imafdch
Boot HART ISA Extensions  : sstc,zicntr,zihpm,zicboz,zicbom,sdtrig
Boot HART PMP Count       : 16
Boot HART PMP Granularity : 2 bits
Boot HART PMP Address Bits: 54
Boot HART MHPM Info       : 16 (0x0007fff8)
Boot HART Debug Triggers  : 2 triggers
Boot HART MIDELEG         : 0x0000000000001666
Boot HART MEDELEG         : 0x0000000000f0b509
[    0.000000] Linux version 6.10.0-rc3-00100-gec0abe37042f (developer@95e6e39b54aa) (riscv32-unknown-linux-gnu-gcc (gc891d8dc23e-dirty) 13.2.0, GNU ld (GNU Binutils) 2.42) #1 SMP Sat Jun 15 13:01:52 UTC 2024
[    0.000000] random: crng init done
[    0.000000] Machine model: riscv-virtio,qemu
[    0.000000] SBI specification v2.0 detected
[    0.000000] SBI implementation ID=0x1 Version=0x10004
[    0.000000] SBI TIME extension detected
[    0.000000] SBI IPI extension detected
[    0.000000] SBI RFENCE extension detected
[    0.000000] SBI SRST extension detected
[    0.000000] SBI DBCN extension detected
[    0.000000] efi: UEFI not found.
[    0.000000] OF: reserved mem: 0x0000000080000000..0x000000008007ffff (512 KiB) nomap non-reusable mmode_resv1@80000000
[    0.000000] OF: reserved mem: 0x0000000080080000..0x000000008009ffff (128 KiB) nomap non-reusable mmode_resv0@80080000
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080000000-0x0000000087ffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x000000008009ffff]
[    0.000000]   node   0: [mem 0x00000000800a0000-0x0000000087ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x0000000087ffffff]
[    0.000000] SBI HSM extension detected
[    0.000000] riscv: base ISA extensions acdfhim
[    0.000000] riscv: ELF capabilities acdfim
......
Starting network: udhcpc: started, v1.36.1
udhcpc: broadcasting discover
udhcpc: broadcasting select for 10.0.2.15, server 10.0.2.2
udhcpc: lease of 10.0.2.15 obtained from 10.0.2.2, lease time 86400
deleting routers
adding dns 10.0.2.3
OK

Welcome to Buildroot
buildroot login: root
# cat /proc/cpuinfo
processor       : 0
hart            : 0
isa             : rv64imafdch_zicbom_zicboz_zicntr_zicsr_zifencei_zihintntl_zihintpause_zihpm_zfa_zba_zbb_zbc_zbs_sstc
mmu             : sv57
mvendorid       : 0x0
marchid         : 0x0
mimpid          : 0x0
hart isa        : rv64imafdch_zicbom_zicboz_zicntr_zicsr_zifencei_zihintntl_zihintpause_zihpm_zfa_zba_zbb_zbc_zbs_sstc
```

This doesn't mean we have addressed all the challenges for this problem. The to do list is here:
1. OpenSBI need accept a parameter from firmware the next boot level is SXL32 instead of only S mode.
2. OpenSBI should dynamically configure the SXL32 environment for next boot.
3. OpenSBI should ensure SBI calls for RV32 are right implemented even M mode is RV64.
4. OpenSBI should override details of machine configuration for RV32 Linux, such as rv64i ISA description in DTB.

Having all of them been addressed, we can allow the SXL write in mstatus. But this todo list depends on
a runnable environment, which is provided by this patch set.

This patch set is based on the patch set[1] under review. You can also fetch the branch from my repo[2].

[1]: https://mail.gnu.org/archive/html/qemu-riscv/2024-09/msg00501.html
[2]: https://github.com/romanheros/qemu/tree/riscv-upstream-sxl32-v1

TANG Tiancheng (7):
  target/riscv: Fix sstatus read and write
  target/riscv: Fix satp read and write implicitly or explicitly.
  target/riscv: Read pte and satp based on SXL in PTW
  hw/riscv: Align kernel to 4MB when sxl32 is on.
  target/riscv: Enable 32-bit only registers for RV64 with sxl32
  target/riscv: Reset SXL and UXL according to sxl32
  target/riscv: Expose sxl32 configuration in RISC-V CPU

 hw/riscv/boot.c           |  3 ++-
 target/riscv/cpu.c        | 14 ++++++++++++--
 target/riscv/cpu_cfg.h    |  4 ++++
 target/riscv/cpu_helper.c |  4 ++--
 target/riscv/csr.c        | 38 +++++++++++++++++++++++++-------------
 5 files changed, 45 insertions(+), 18 deletions(-)

-- 
2.43.0

[PATCH v1 1/7] target/riscv: Fix sstatus read and write

[LIU Zhiwei]

From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>

Sstatus is SXLEN bits in length and always has the layout determined by
the SXL configuration, regardless of the current XLEN.

Signed-off-by: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
Fixes: b550f89457 (target/riscv: Compute mstatus.sd on demand)
Fixes: f310df58bd (target/riscv: Enable uxl field write)
---
 target/riscv/csr.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/target/riscv/csr.c b/target/riscv/csr.c
index ea3560342c..b33cc1ec23 100644
--- a/target/riscv/csr.c
+++ b/target/riscv/csr.c
@@ -2893,7 +2893,7 @@ static RISCVException read_sstatus_i128(CPURISCVState *env, int csrno,
 {
     uint64_t mask = sstatus_v1_10_mask;
     uint64_t sstatus = env->mstatus & mask;
-    if (env->xl != MXL_RV32 || env->debugger) {
+    if (riscv_cpu_sxl(env) != MXL_RV32 || env->debugger) {
         mask |= SSTATUS64_UXL;
     }
 
@@ -2905,11 +2905,10 @@ static RISCVException read_sstatus(CPURISCVState *env, int csrno,
                                    target_ulong *val)
 {
     target_ulong mask = (sstatus_v1_10_mask);
-    if (env->xl != MXL_RV32 || env->debugger) {
+    if (riscv_cpu_sxl(env) != MXL_RV32 || env->debugger) {
         mask |= SSTATUS64_UXL;
     }
-    /* TODO: Use SXL not MXL. */
-    *val = add_status_sd(riscv_cpu_mxl(env), env->mstatus & mask);
+    *val = add_status_sd(riscv_cpu_sxl(env), env->mstatus & mask);
     return RISCV_EXCP_NONE;
 }
 
@@ -2918,7 +2917,7 @@ static RISCVException write_sstatus(CPURISCVState *env, int csrno,
 {
     target_ulong mask = (sstatus_v1_10_mask);
 
-    if (env->xl != MXL_RV32 || env->debugger) {
+    if (riscv_cpu_sxl(env) != MXL_RV32 || env->debugger) {
         if ((val & SSTATUS64_UXL) != 0) {
             mask |= SSTATUS64_UXL;
         }
-- 
2.43.0

[PATCH v1 2/7] target/riscv: Fix satp read and write implicitly or explicitly.

[LIU Zhiwei]

From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>

CSR satp is SXLEN bits in length and always has the $layout determined by
the SXL configuration, regardless of the current XLEN.

Only process CSR satp, as we still don't have a riscv_cpu_vsxl API
currently.

Added sxl32 property to control sxlen as 32 in s-mode for QEMU RV64.

Signed-off-by: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
Fixes: c7b9517188 (RISC-V: Implement modular CSR helper interface)
Reviewed-by: Liu Zhiwei <zhiwei_liu@linux.alibaba.com>
---
 target/riscv/cpu_cfg.h |  4 ++++
 target/riscv/csr.c     | 25 +++++++++++++++++++------
 2 files changed, 23 insertions(+), 6 deletions(-)

diff --git a/target/riscv/cpu_cfg.h b/target/riscv/cpu_cfg.h
index 8b272fb826..cdbd2afe29 100644
--- a/target/riscv/cpu_cfg.h
+++ b/target/riscv/cpu_cfg.h
@@ -173,6 +173,10 @@ struct RISCVCPUConfig {
     bool short_isa_string;
 
 #ifndef CONFIG_USER_ONLY
+    /*
+     * true when RV64 QEMU running with mxlen==64 but sxlen==32.
+     */
+    bool sxl32;
     RISCVSATPMap satp_mode;
 #endif
 };
diff --git a/target/riscv/csr.c b/target/riscv/csr.c
index b33cc1ec23..93a5cf87ed 100644
--- a/target/riscv/csr.c
+++ b/target/riscv/csr.c
@@ -1504,16 +1504,29 @@ static RISCVException read_mstatus(CPURISCVState *env, int csrno,
 
 static bool validate_vm(CPURISCVState *env, target_ulong vm)
 {
-    uint64_t mode_supported = riscv_cpu_cfg(env)->satp_mode.map;
+    uint64_t mode_supported = 0;
+    if (riscv_cpu_cfg(env)->sxl32 && (riscv_cpu_mxl(env) != MXL_RV32)) {
+        mode_supported = (1 << VM_1_10_MBARE) | (1 << VM_1_10_SV32);
+    } else {
+        mode_supported = riscv_cpu_cfg(env)->satp_mode.map;
+    }
     return get_field(mode_supported, (1 << vm));
 }
 
 static target_ulong legalize_xatp(CPURISCVState *env, target_ulong old_xatp,
-                                  target_ulong val)
+                                  target_ulong val, int csrno)
 {
     target_ulong mask;
     bool vm;
-    if (riscv_cpu_mxl(env) == MXL_RV32) {
+    RISCVMXL xl;
+
+    if (csrno == CSR_SATP) {
+        xl = riscv_cpu_sxl(env);
+    } else {
+        xl = riscv_cpu_mxl(env);
+    }
+
+    if (xl == MXL_RV32) {
         vm = validate_vm(env, get_field(val, SATP32_MODE));
         mask = (val ^ old_xatp) & (SATP32_MODE | SATP32_ASID | SATP32_PPN);
     } else {
@@ -3316,7 +3329,7 @@ static RISCVException write_satp(CPURISCVState *env, int csrno,
         return RISCV_EXCP_NONE;
     }
 
-    env->satp = legalize_xatp(env, env->satp, val);
+    env->satp = legalize_xatp(env, env->satp, val, csrno);
     return RISCV_EXCP_NONE;
 }
 
@@ -3834,7 +3847,7 @@ static RISCVException read_hgatp(CPURISCVState *env, int csrno,
 static RISCVException write_hgatp(CPURISCVState *env, int csrno,
                                   target_ulong val)
 {
-    env->hgatp = legalize_xatp(env, env->hgatp, val);
+    env->hgatp = legalize_xatp(env, env->hgatp, val, csrno);
     return RISCV_EXCP_NONE;
 }
 
@@ -4116,7 +4129,7 @@ static RISCVException read_vsatp(CPURISCVState *env, int csrno,
 static RISCVException write_vsatp(CPURISCVState *env, int csrno,
                                   target_ulong val)
 {
-    env->vsatp = legalize_xatp(env, env->vsatp, val);
+    env->vsatp = legalize_xatp(env, env->vsatp, val, csrno);
     return RISCV_EXCP_NONE;
 }
 
-- 
2.43.0

[PATCH v1 3/7] target/riscv: Read pte and satp based on SXL in PTW

[LIU Zhiwei]

From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>

Satp and PTE are always SXLEN-bit. when SXLEN is 32,
read PTE as 4 bytes, and treat satp as SATP32.

Signed-off-by: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
---
 target/riscv/cpu_helper.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index 077f6d77c3..773789e02e 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -851,7 +851,7 @@ static int get_physical_address(CPURISCVState *env, hwaddr *physical,
                 vm = get_field(env->vsatp, SATP64_MODE);
             }
         } else {
-            if (riscv_cpu_mxl(env) == MXL_RV32) {
+            if (riscv_cpu_sxl(env) == MXL_RV32) {
                 base = (hwaddr)get_field(env->satp, SATP32_PPN) << PGSHIFT;
                 vm = get_field(env->satp, SATP32_MODE);
             } else {
@@ -972,7 +972,7 @@ restart:
             return TRANSLATE_PMP_FAIL;
         }
 
-        if (riscv_cpu_mxl(env) == MXL_RV32) {
+        if (riscv_cpu_sxl(env) == MXL_RV32) {
             pte = address_space_ldl(cs->as, pte_addr, attrs, &res);
         } else {
             pte = address_space_ldq(cs->as, pte_addr, attrs, &res);
-- 
2.43.0

[PATCH v1 4/7] hw/riscv: Align kernel to 4MB when sxl32 is on.

[LIU Zhiwei]

From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>

RISC-V always requires 4MB alignment for RV32.

Signed-off-by: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
---
 hw/riscv/boot.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
index 1a2c1ff9e0..7ce0d8f08f 100644
--- a/hw/riscv/boot.c
+++ b/hw/riscv/boot.c
@@ -69,7 +69,8 @@ char *riscv_plic_hart_config_string(int hart_count)
 
 target_ulong riscv_calc_kernel_start_addr(RISCVHartArrayState *harts,
                                           target_ulong firmware_end_addr) {
-    if (riscv_is_32bit(harts)) {
+    RISCVCPU *cpu = &harts->harts[0];
+    if (riscv_is_32bit(harts) || cpu->cfg.sxl32) {
         return QEMU_ALIGN_UP(firmware_end_addr, 4 * MiB);
     } else {
         return QEMU_ALIGN_UP(firmware_end_addr, 2 * MiB);
-- 
2.43.0

[PATCH v1 5/7] target/riscv: Enable 32-bit only registers for RV64 with sxl32

[LIU Zhiwei]

From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>

Allow reading 32-bit only registers like timeh and stimecmph when
booting a 32-bit Linux kernel on RV64 when sxl32 is true.

Signed-off-by: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
---
 target/riscv/csr.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/riscv/csr.c b/target/riscv/csr.c
index 93a5cf87ed..c412ac8e31 100644
--- a/target/riscv/csr.c
+++ b/target/riscv/csr.c
@@ -161,7 +161,7 @@ skip_ext_pmu_check:
 
 static RISCVException ctr32(CPURISCVState *env, int csrno)
 {
-    if (riscv_cpu_mxl(env) != MXL_RV32) {
+    if (env->xl != MXL_RV32) {
         return RISCV_EXCP_ILLEGAL_INST;
     }
 
@@ -481,7 +481,7 @@ static RISCVException sstc(CPURISCVState *env, int csrno)
 
 static RISCVException sstc_32(CPURISCVState *env, int csrno)
 {
-    if (riscv_cpu_mxl(env) != MXL_RV32) {
+    if (env->xl != MXL_RV32) {
         return RISCV_EXCP_ILLEGAL_INST;
     }
 
-- 
2.43.0

[PATCH v1 6/7] target/riscv: Reset SXL and UXL according to sxl32

[LIU Zhiwei]

From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>

When boot a 32-bit system, sxl and uxl should be set to 1 by OpenSBI. However,
OpenSBI does not support this feature.

We temporarily force QEMU reset SXL and UXL to MXL_RV32 when sxl32 is enabled.

Signed-off-by: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
---
 target/riscv/cpu.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index baf8fba467..9dbbb1ca77 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -933,8 +933,17 @@ static void riscv_cpu_reset_hold(Object *obj, ResetType type)
          * The reset status of SXL/UXL is undefined, but mstatus is WARL
          * and we must ensure that the value after init is valid for read.
          */
-        env->mstatus = set_field(env->mstatus, MSTATUS64_SXL, env->misa_mxl);
-        env->mstatus = set_field(env->mstatus, MSTATUS64_UXL, env->misa_mxl);
+        if (cpu->cfg.sxl32) {
+            env->mstatus = set_field(env->mstatus, MSTATUS64_SXL,
+                                      MXL_RV32);
+            env->mstatus = set_field(env->mstatus, MSTATUS64_UXL,
+                                      MXL_RV32);
+        } else {
+            env->mstatus = set_field(env->mstatus, MSTATUS64_SXL,
+                                      env->misa_mxl);
+            env->mstatus = set_field(env->mstatus, MSTATUS64_UXL,
+                                      env->misa_mxl);
+        }
         if (riscv_has_ext(env, RVH)) {
             env->vsstatus = set_field(env->vsstatus,
                                       MSTATUS64_SXL, env->misa_mxl);
-- 
2.43.0

[PATCH v1 7/7] target/riscv: Expose sxl32 configuration in RISC-V CPU

[LIU Zhiwei]

From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>

Enable with "-cpu rv64,sxl32=on".
When sxl32 is enabled, RV64 can boot 32-bit Linux with
64-bit Opensbi while requiring to make minor modifications
to the Linux kernel source code.

How to patch linux:
https://git

Signed-off-by: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
---
 target/riscv/cpu.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index 9dbbb1ca77..86984b7f8f 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -2665,6 +2665,7 @@ static Property riscv_cpu_properties[] = {
 
 #ifndef CONFIG_USER_ONLY
     DEFINE_PROP_UINT64("resetvec", RISCVCPU, env.resetvec, DEFAULT_RSTVEC),
+    DEFINE_PROP_BOOL("sxl32", RISCVCPU, cfg.sxl32, false),
 #endif
 
     DEFINE_PROP_BOOL("short-isa-string", RISCVCPU, cfg.short_isa_string, false),
-- 
2.43.0

Re: [PATCH v1 1/7] target/riscv: Fix sstatus read and write

[Alistair Francis]

On Mon, Oct 7, 2024 at 1:35 PM LIU Zhiwei <zhiwei_liu@linux.alibaba.com> wrote:
>
> From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
>
> Sstatus is SXLEN bits in length and always has the layout determined by
> the SXL configuration, regardless of the current XLEN.
>
> Signed-off-by: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
> Fixes: b550f89457 (target/riscv: Compute mstatus.sd on demand)
> Fixes: f310df58bd (target/riscv: Enable uxl field write)

Reviewed-by: Alistair Francis <alistair.francis@wdc.com>

Alistair

> ---
>  target/riscv/csr.c | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)
>
> diff --git a/target/riscv/csr.c b/target/riscv/csr.c
> index ea3560342c..b33cc1ec23 100644
> --- a/target/riscv/csr.c
> +++ b/target/riscv/csr.c
> @@ -2893,7 +2893,7 @@ static RISCVException read_sstatus_i128(CPURISCVState *env, int csrno,
>  {
>      uint64_t mask = sstatus_v1_10_mask;
>      uint64_t sstatus = env->mstatus & mask;
> -    if (env->xl != MXL_RV32 || env->debugger) {
> +    if (riscv_cpu_sxl(env) != MXL_RV32 || env->debugger) {
>          mask |= SSTATUS64_UXL;
>      }
>
> @@ -2905,11 +2905,10 @@ static RISCVException read_sstatus(CPURISCVState *env, int csrno,
>                                     target_ulong *val)
>  {
>      target_ulong mask = (sstatus_v1_10_mask);
> -    if (env->xl != MXL_RV32 || env->debugger) {
> +    if (riscv_cpu_sxl(env) != MXL_RV32 || env->debugger) {
>          mask |= SSTATUS64_UXL;
>      }
> -    /* TODO: Use SXL not MXL. */
> -    *val = add_status_sd(riscv_cpu_mxl(env), env->mstatus & mask);
> +    *val = add_status_sd(riscv_cpu_sxl(env), env->mstatus & mask);
>      return RISCV_EXCP_NONE;
>  }
>
> @@ -2918,7 +2917,7 @@ static RISCVException write_sstatus(CPURISCVState *env, int csrno,
>  {
>      target_ulong mask = (sstatus_v1_10_mask);
>
> -    if (env->xl != MXL_RV32 || env->debugger) {
> +    if (riscv_cpu_sxl(env) != MXL_RV32 || env->debugger) {
>          if ((val & SSTATUS64_UXL) != 0) {
>              mask |= SSTATUS64_UXL;
>          }
> --
> 2.43.0
>
>

Re: [PATCH v1 0/7] target/riscv: Support SXL32 on RV64 CPU

[Alistair Francis]

On Mon, Oct 7, 2024 at 1:35 PM LIU Zhiwei <zhiwei_liu@linux.alibaba.com> wrote:
>
> From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
>
> We have implemented UXL32 on QEMU already. It enables us to run RV32
> applications on RV64 Linux on QEMU. Similarly, RISCV specification
> doesn't limit the SXLEN to be the same with MXLEN. In this patch set,
> we will go forward to implement SXL32 on RV64 CPU.
>
> SXL is one filed in mstatus. It defines the SXLEN. After CPU reset,
> SXL will be same with MXL in misa.
>
> According to the RISC-V privilege specification, Smode CSR with SXLEN
> length will usually be read and written in an SXLEN view, no matter
> current XLEN is 32bit or 64bit. Thus if MXLEN=64 and SXLEN=32, M mode
> register will read or write them in the same way under RV32 layout.
>
> In this patch set, we carefully process the Smode CSR write and read
> for SXL32. This includes explicitly CSR write/read, and implicitly
> CSR read such as satp read in PTW(page table walk).
>
> Another thing we do in this patch set is that we introduce a property
> of CPU, named sxl32. It provides us a way to override the default sxl64
> for RV64 cpu after cpu reset. The reason is OpenSBI currently doesn't
> dynamically probe or configure SXLEN. When we want to run RV64 OpenSBI
> in M mode and RV32 Linux in S mode, OpenSBI will not prepare the SXL32
> environment for RV32 Linux. sxl32 property will also forbidden writing SXL
> field in mstatus, which means QEMU can ensure an SXL32 environment for Linux.

This should be fixed in OpenSBI then. We don't really want a property
in QEMU to do something that the guest should be setting.

I guess the spec doesn't mandate what this is on reset though... So in
theory a 64-bit hart could have a SXLEN reset value of 32-bits, which
is what this property allows modelling. That does seem pretty odd
though and I think it makes more sense to just defer this to OpenSBI
to handle.

Alistair

>
> We've successfully booted 64-bit OpenSBI and 32-bit Linux system on
> RV64 CPU, with a minor fix of Linux.
>
> Tested by follow steps:
>
>  - Prepare Linux rv32 Image && rv64 Image
>   Please apply the patch of linux before test. You can get the patch in
> https://github.com/romanheros/linux/commit/8444fb3c913d64c20b1e5334a29a66061eb3f5fe.
>
>         $ cd linux
>         $ make ARCH=riscv CROSS_COMPILE=riscv32-unknown-linux-gnu- O=../linux-rv32 defconfig
>         $ make ARCH=riscv CROSS_COMPILE=riscv32-unknown-linux-gnu- O=../build-rv32 32-bit.config
>         $ make ARCH=riscv CROSS_COMPILE=riscv32-unknown-linux-gnu- O=../linux-rv32/ -j $(nproc)
>         $ make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- O=../linux-rv64 defconfig
>         $ make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- O=../linux-rv64/ -j $(nproc)
>         (Get Image in ../linux-rv32/arch/riscv/boot)
>
>  - Prepare QEMU
>         $ git clone https://github.com/romanheros/qemu.git -b riscv-upstream-sxl32-v1
>         $ cd qemu
>         $ ./configure --target-list="riscv64-softmmu" --enable-slirp
>         $ cd build; ninja
>
> - Run 32-bit Linux boot on 64-bit RISC-V with Opensbi 64 :
>         $ ./build/qemu-system-riscv64 -cpu rv64,sxl32=on -M virt -nographic \
>                 -bios opensbi/build/platform/generic/firmware/fw_dynamic.bin \
>                 -kernel linux-rv32/arch/riscv/boot/Image \
>                 -append "root=/dev/vda ro console=ttyS0" \
>                 -drive file=qemu_riscv32_virt_defconfig/images/rootfs.ext2,format=raw,id=hd0 \
>                 -netdev user,id=net0 -device virtio-net-device,netdev=net0
> ```
> OpenSBI v1.4-112-g7b4bc9d
> Build time: 2024-06-17 06:57:42 +0000
> Build compiler: gcc version 13.2.0 (gc891d8dc23e-dirty)
>    ____                    _____ ____ _____
>   / __ \                  / ____|  _ \_   _|
>  | |  | |_ __   ___ _ __ | (___ | |_) || |
>  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
>  | |__| | |_) |  __/ | | |____) | |_) || |_
>   \____/| .__/ \___|_| |_|_____/|____/_____|
>         | |
>         |_|
>
> Platform Name             : riscv-virtio,qemu
> Platform Features         : medeleg
> Platform HART Count       : 1
> Platform IPI Device       : aclint-mswi
> Platform Timer Device     : aclint-mtimer @ 10000000Hz
> Platform Console Device   : uart8250
> Platform HSM Device       : ---
> Platform PMU Device       : ---
> Platform Reboot Device    : syscon-reboot
> Platform Shutdown Device  : syscon-poweroff
> Platform Suspend Device   : ---
> Platform CPPC Device      : ---
> Firmware Base             : 0x80000000
> Firmware Size             : 583 KB
> Firmware RW Offset        : 0x80000
> Firmware RW Size          : 71 KB
> Firmware Heap Offset      : 0x89000
> Firmware Heap Size        : 35 KB (total), 2 KB (reserved), 11 KB (used), 21 KB (free)
> Firmware Scratch Size     : 4096 B (total), 400 B (used), 3696 B (free)
> Runtime SBI Version       : 2.0
>
> Domain0 Name              : root
> Domain0 Boot HART         : 0
> Domain0 HARTs             : 0*
> Domain0 Region00          : 0x0000000000100000-0x0000000000100fff M: (I,R,W) S/U: (R,W)
> Domain0 Region01          : 0x0000000010000000-0x0000000010000fff M: (I,R,W) S/U: (R,W)
> Domain0 Region02          : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: ()
> Domain0 Region03          : 0x0000000080080000-0x000000008009ffff M: (R,W) S/U: ()
> Domain0 Region04          : 0x0000000080000000-0x000000008007ffff M: (R,X) S/U: ()
> Domain0 Region05          : 0x000000000c400000-0x000000000c5fffff M: (I,R,W) S/U: (R,W)
> Domain0 Region06          : 0x000000000c000000-0x000000000c3fffff M: (I,R,W) S/U: (R,W)
> Domain0 Region07          : 0x0000000000000000-0xffffffffffffffff M: () S/U: (R,W,X)
> Domain0 Next Address      : 0x0000000080200000
> Domain0 Next Arg1         : 0x0000000087e00000
> Domain0 Next Mode         : S-mode
> Domain0 SysReset          : yes
> Domain0 SysSuspend        : yes
>
> Boot HART ID              : 0
> Boot HART Domain          : root
> Boot HART Priv Version    : v1.12
> Boot HART Base ISA        : rv64imafdch
> Boot HART ISA Extensions  : sstc,zicntr,zihpm,zicboz,zicbom,sdtrig
> Boot HART PMP Count       : 16
> Boot HART PMP Granularity : 2 bits
> Boot HART PMP Address Bits: 54
> Boot HART MHPM Info       : 16 (0x0007fff8)
> Boot HART Debug Triggers  : 2 triggers
> Boot HART MIDELEG         : 0x0000000000001666
> Boot HART MEDELEG         : 0x0000000000f0b509
> [    0.000000] Linux version 6.10.0-rc3-00100-gec0abe37042f (developer@95e6e39b54aa) (riscv32-unknown-linux-gnu-gcc (gc891d8dc23e-dirty) 13.2.0, GNU ld (GNU Binutils) 2.42) #1 SMP Sat Jun 15 13:01:52 UTC 2024
> [    0.000000] random: crng init done
> [    0.000000] Machine model: riscv-virtio,qemu
> [    0.000000] SBI specification v2.0 detected
> [    0.000000] SBI implementation ID=0x1 Version=0x10004
> [    0.000000] SBI TIME extension detected
> [    0.000000] SBI IPI extension detected
> [    0.000000] SBI RFENCE extension detected
> [    0.000000] SBI SRST extension detected
> [    0.000000] SBI DBCN extension detected
> [    0.000000] efi: UEFI not found.
> [    0.000000] OF: reserved mem: 0x0000000080000000..0x000000008007ffff (512 KiB) nomap non-reusable mmode_resv1@80000000
> [    0.000000] OF: reserved mem: 0x0000000080080000..0x000000008009ffff (128 KiB) nomap non-reusable mmode_resv0@80080000
> [    0.000000] Zone ranges:
> [    0.000000]   DMA32    [mem 0x0000000080000000-0x0000000087ffffff]
> [    0.000000]   Normal   empty
> [    0.000000] Movable zone start for each node
> [    0.000000] Early memory node ranges
> [    0.000000]   node   0: [mem 0x0000000080000000-0x000000008009ffff]
> [    0.000000]   node   0: [mem 0x00000000800a0000-0x0000000087ffffff]
> [    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x0000000087ffffff]
> [    0.000000] SBI HSM extension detected
> [    0.000000] riscv: base ISA extensions acdfhim
> [    0.000000] riscv: ELF capabilities acdfim
> ......
> Starting network: udhcpc: started, v1.36.1
> udhcpc: broadcasting discover
> udhcpc: broadcasting select for 10.0.2.15, server 10.0.2.2
> udhcpc: lease of 10.0.2.15 obtained from 10.0.2.2, lease time 86400
> deleting routers
> adding dns 10.0.2.3
> OK
>
> Welcome to Buildroot
> buildroot login: root
> # cat /proc/cpuinfo
> processor       : 0
> hart            : 0
> isa             : rv64imafdch_zicbom_zicboz_zicntr_zicsr_zifencei_zihintntl_zihintpause_zihpm_zfa_zba_zbb_zbc_zbs_sstc
> mmu             : sv57
> mvendorid       : 0x0
> marchid         : 0x0
> mimpid          : 0x0
> hart isa        : rv64imafdch_zicbom_zicboz_zicntr_zicsr_zifencei_zihintntl_zihintpause_zihpm_zfa_zba_zbb_zbc_zbs_sstc
> ```
>
> This doesn't mean we have addressed all the challenges for this problem. The to do list is here:
> 1. OpenSBI need accept a parameter from firmware the next boot level is SXL32 instead of only S mode.
> 2. OpenSBI should dynamically configure the SXL32 environment for next boot.
> 3. OpenSBI should ensure SBI calls for RV32 are right implemented even M mode is RV64.
> 4. OpenSBI should override details of machine configuration for RV32 Linux, such as rv64i ISA description in DTB.
>
> Having all of them been addressed, we can allow the SXL write in mstatus. But this todo list depends on
> a runnable environment, which is provided by this patch set.
>
> This patch set is based on the patch set[1] under review. You can also fetch the branch from my repo[2].
>
> [1]: https://mail.gnu.org/archive/html/qemu-riscv/2024-09/msg00501.html
> [2]: https://github.com/romanheros/qemu/tree/riscv-upstream-sxl32-v1
>
> TANG Tiancheng (7):
>   target/riscv: Fix sstatus read and write
>   target/riscv: Fix satp read and write implicitly or explicitly.
>   target/riscv: Read pte and satp based on SXL in PTW
>   hw/riscv: Align kernel to 4MB when sxl32 is on.
>   target/riscv: Enable 32-bit only registers for RV64 with sxl32
>   target/riscv: Reset SXL and UXL according to sxl32
>   target/riscv: Expose sxl32 configuration in RISC-V CPU
>
>  hw/riscv/boot.c           |  3 ++-
>  target/riscv/cpu.c        | 14 ++++++++++++--
>  target/riscv/cpu_cfg.h    |  4 ++++
>  target/riscv/cpu_helper.c |  4 ++--
>  target/riscv/csr.c        | 38 +++++++++++++++++++++++++-------------
>  5 files changed, 45 insertions(+), 18 deletions(-)
>
> --
> 2.43.0
>
>

Re: [PATCH v1 7/7] target/riscv: Expose sxl32 configuration in RISC-V CPU

[Alistair Francis]

On Mon, Oct 7, 2024 at 2:19 PM LIU Zhiwei <zhiwei_liu@linux.alibaba.com> wrote:
>
> From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
>
> Enable with "-cpu rv64,sxl32=on".
> When sxl32 is enabled, RV64 can boot 32-bit Linux with
> 64-bit Opensbi while requiring to make minor modifications
> to the Linux kernel source code.
>
> How to patch linux:
> https://git
>
> Signed-off-by: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
> ---
>  target/riscv/cpu.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
> index 9dbbb1ca77..86984b7f8f 100644
> --- a/target/riscv/cpu.c
> +++ b/target/riscv/cpu.c
> @@ -2665,6 +2665,7 @@ static Property riscv_cpu_properties[] = {
>
>  #ifndef CONFIG_USER_ONLY
>      DEFINE_PROP_UINT64("resetvec", RISCVCPU, env.resetvec, DEFAULT_RSTVEC),
> +    DEFINE_PROP_BOOL("sxl32", RISCVCPU, cfg.sxl32, false),

I don't think we should add this (see the cover letter), but if we did
it would need documentation

Alistair

>  #endif
>
>      DEFINE_PROP_BOOL("short-isa-string", RISCVCPU, cfg.short_isa_string, false),
> --
> 2.43.0
>
>

Re: [PATCH v1 2/7] target/riscv: Fix satp read and write implicitly or explicitly.

[Alistair Francis]

On Mon, Oct 7, 2024 at 1:35 PM LIU Zhiwei <zhiwei_liu@linux.alibaba.com> wrote:
>
> From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
>
> CSR satp is SXLEN bits in length and always has the $layout determined by
> the SXL configuration, regardless of the current XLEN.
>
> Only process CSR satp, as we still don't have a riscv_cpu_vsxl API
> currently.
>
> Added sxl32 property to control sxlen as 32 in s-mode for QEMU RV64.
>
> Signed-off-by: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
> Fixes: c7b9517188 (RISC-V: Implement modular CSR helper interface)
> Reviewed-by: Liu Zhiwei <zhiwei_liu@linux.alibaba.com>
> ---
>  target/riscv/cpu_cfg.h |  4 ++++
>  target/riscv/csr.c     | 25 +++++++++++++++++++------
>  2 files changed, 23 insertions(+), 6 deletions(-)
>
> diff --git a/target/riscv/cpu_cfg.h b/target/riscv/cpu_cfg.h
> index 8b272fb826..cdbd2afe29 100644
> --- a/target/riscv/cpu_cfg.h
> +++ b/target/riscv/cpu_cfg.h
> @@ -173,6 +173,10 @@ struct RISCVCPUConfig {
>      bool short_isa_string;
>
>  #ifndef CONFIG_USER_ONLY
> +    /*
> +     * true when RV64 QEMU running with mxlen==64 but sxlen==32.
> +     */
> +    bool sxl32;
>      RISCVSATPMap satp_mode;
>  #endif
>  };
> diff --git a/target/riscv/csr.c b/target/riscv/csr.c
> index b33cc1ec23..93a5cf87ed 100644
> --- a/target/riscv/csr.c
> +++ b/target/riscv/csr.c
> @@ -1504,16 +1504,29 @@ static RISCVException read_mstatus(CPURISCVState *env, int csrno,
>
>  static bool validate_vm(CPURISCVState *env, target_ulong vm)
>  {
> -    uint64_t mode_supported = riscv_cpu_cfg(env)->satp_mode.map;
> +    uint64_t mode_supported = 0;
> +    if (riscv_cpu_cfg(env)->sxl32 && (riscv_cpu_mxl(env) != MXL_RV32)) {

This should be using the actual SXLEN values, not this special
property. What happens if the property is set to false but the guest
software sets SXLEN to 32-bits? This will break

Alistair

> +        mode_supported = (1 << VM_1_10_MBARE) | (1 << VM_1_10_SV32);
> +    } else {
> +        mode_supported = riscv_cpu_cfg(env)->satp_mode.map;
> +    }
>      return get_field(mode_supported, (1 << vm));
>  }
>
>  static target_ulong legalize_xatp(CPURISCVState *env, target_ulong old_xatp,
> -                                  target_ulong val)
> +                                  target_ulong val, int csrno)
>  {
>      target_ulong mask;
>      bool vm;
> -    if (riscv_cpu_mxl(env) == MXL_RV32) {
> +    RISCVMXL xl;
> +
> +    if (csrno == CSR_SATP) {
> +        xl = riscv_cpu_sxl(env);
> +    } else {
> +        xl = riscv_cpu_mxl(env);
> +    }
> +
> +    if (xl == MXL_RV32) {
>          vm = validate_vm(env, get_field(val, SATP32_MODE));
>          mask = (val ^ old_xatp) & (SATP32_MODE | SATP32_ASID | SATP32_PPN);
>      } else {
> @@ -3316,7 +3329,7 @@ static RISCVException write_satp(CPURISCVState *env, int csrno,
>          return RISCV_EXCP_NONE;
>      }
>
> -    env->satp = legalize_xatp(env, env->satp, val);
> +    env->satp = legalize_xatp(env, env->satp, val, csrno);
>      return RISCV_EXCP_NONE;
>  }
>
> @@ -3834,7 +3847,7 @@ static RISCVException read_hgatp(CPURISCVState *env, int csrno,
>  static RISCVException write_hgatp(CPURISCVState *env, int csrno,
>                                    target_ulong val)
>  {
> -    env->hgatp = legalize_xatp(env, env->hgatp, val);
> +    env->hgatp = legalize_xatp(env, env->hgatp, val, csrno);
>      return RISCV_EXCP_NONE;
>  }
>
> @@ -4116,7 +4129,7 @@ static RISCVException read_vsatp(CPURISCVState *env, int csrno,
>  static RISCVException write_vsatp(CPURISCVState *env, int csrno,
>                                    target_ulong val)
>  {
> -    env->vsatp = legalize_xatp(env, env->vsatp, val);
> +    env->vsatp = legalize_xatp(env, env->vsatp, val, csrno);
>      return RISCV_EXCP_NONE;
>  }
>
> --
> 2.43.0
>
>

Re: [PATCH v1 3/7] target/riscv: Read pte and satp based on SXL in PTW

[Alistair Francis]

On Mon, Oct 7, 2024 at 1:36 PM LIU Zhiwei <zhiwei_liu@linux.alibaba.com> wrote:
>
> From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
>
> Satp and PTE are always SXLEN-bit. when SXLEN is 32,
> read PTE as 4 bytes, and treat satp as SATP32.
>
> Signed-off-by: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>

Reviewed-by: Alistair Francis <alistair.francis@wdc.com>

Alistair

> ---
>  target/riscv/cpu_helper.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
> index 077f6d77c3..773789e02e 100644
> --- a/target/riscv/cpu_helper.c
> +++ b/target/riscv/cpu_helper.c
> @@ -851,7 +851,7 @@ static int get_physical_address(CPURISCVState *env, hwaddr *physical,
>                  vm = get_field(env->vsatp, SATP64_MODE);
>              }
>          } else {
> -            if (riscv_cpu_mxl(env) == MXL_RV32) {
> +            if (riscv_cpu_sxl(env) == MXL_RV32) {
>                  base = (hwaddr)get_field(env->satp, SATP32_PPN) << PGSHIFT;
>                  vm = get_field(env->satp, SATP32_MODE);
>              } else {
> @@ -972,7 +972,7 @@ restart:
>              return TRANSLATE_PMP_FAIL;
>          }
>
> -        if (riscv_cpu_mxl(env) == MXL_RV32) {
> +        if (riscv_cpu_sxl(env) == MXL_RV32) {
>              pte = address_space_ldl(cs->as, pte_addr, attrs, &res);
>          } else {
>              pte = address_space_ldq(cs->as, pte_addr, attrs, &res);
> --
> 2.43.0
>
>

Re: [PATCH v1 5/7] target/riscv: Enable 32-bit only registers for RV64 with sxl32

[Alistair Francis]

On Mon, Oct 7, 2024 at 1:52 PM LIU Zhiwei <zhiwei_liu@linux.alibaba.com> wrote:
>
> From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
>
> Allow reading 32-bit only registers like timeh and stimecmph when
> booting a 32-bit Linux kernel on RV64 when sxl32 is true.
>
> Signed-off-by: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>
> ---
>  target/riscv/csr.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/target/riscv/csr.c b/target/riscv/csr.c
> index 93a5cf87ed..c412ac8e31 100644
> --- a/target/riscv/csr.c
> +++ b/target/riscv/csr.c
> @@ -161,7 +161,7 @@ skip_ext_pmu_check:
>
>  static RISCVException ctr32(CPURISCVState *env, int csrno)
>  {
> -    if (riscv_cpu_mxl(env) != MXL_RV32) {
> +    if (env->xl != MXL_RV32) {

Why not riscv_cpu_sxl()?

Alistair

Re: [PATCH v1 0/7] target/riscv: Support SXL32 on RV64 CPU

[LIU Zhiwei]

[-- Attachment #1: Type: text/plain, Size: 10702 bytes --]


On 2024/10/11 11:55, Alistair Francis wrote:
> On Mon, Oct 7, 2024 at 1:35 PM LIU Zhiwei<zhiwei_liu@linux.alibaba.com> wrote:
>> From: TANG Tiancheng<tangtiancheng.ttc@alibaba-inc.com>
>>
>> We have implemented UXL32 on QEMU already. It enables us to run RV32
>> applications on RV64 Linux on QEMU. Similarly, RISCV specification
>> doesn't limit the SXLEN to be the same with MXLEN. In this patch set,
>> we will go forward to implement SXL32 on RV64 CPU.
>>
>> SXL is one filed in mstatus. It defines the SXLEN. After CPU reset,
>> SXL will be same with MXL in misa.
>>
>> According to the RISC-V privilege specification, Smode CSR with SXLEN
>> length will usually be read and written in an SXLEN view, no matter
>> current XLEN is 32bit or 64bit. Thus if MXLEN=64 and SXLEN=32, M mode
>> register will read or write them in the same way under RV32 layout.
>>
>> In this patch set, we carefully process the Smode CSR write and read
>> for SXL32. This includes explicitly CSR write/read, and implicitly
>> CSR read such as satp read in PTW(page table walk).
>>
>> Another thing we do in this patch set is that we introduce a property
>> of CPU, named sxl32. It provides us a way to override the default sxl64
>> for RV64 cpu after cpu reset. The reason is OpenSBI currently doesn't
>> dynamically probe or configure SXLEN. When we want to run RV64 OpenSBI
>> in M mode and RV32 Linux in S mode, OpenSBI will not prepare the SXL32
>> environment for RV32 Linux. sxl32 property will also forbidden writing SXL
>> field in mstatus, which means QEMU can ensure an SXL32 environment for Linux.
> This should be fixed in OpenSBI then. We don't really want a property
> in QEMU to do something that the guest should be setting.

Agree. I will remove sxl32 property next version.

Thanks for review.

Zhiwei

>
> I guess the spec doesn't mandate what this is on reset though... So in
> theory a 64-bit hart could have a SXLEN reset value of 32-bits, which
> is what this property allows modelling. That does seem pretty odd
> though and I think it makes more sense to just defer this to OpenSBI
> to handle.
> Alistair
>
>> We've successfully booted 64-bit OpenSBI and 32-bit Linux system on
>> RV64 CPU, with a minor fix of Linux.
>>
>> Tested by follow steps:
>>
>>   - Prepare Linux rv32 Image && rv64 Image
>>    Please apply the patch of linux before test. You can get the patch in
>> https://github.com/romanheros/linux/commit/8444fb3c913d64c20b1e5334a29a66061eb3f5fe.
>>
>>          $ cd linux
>>          $ make ARCH=riscv CROSS_COMPILE=riscv32-unknown-linux-gnu- O=../linux-rv32 defconfig
>>          $ make ARCH=riscv CROSS_COMPILE=riscv32-unknown-linux-gnu- O=../build-rv32 32-bit.config
>>          $ make ARCH=riscv CROSS_COMPILE=riscv32-unknown-linux-gnu- O=../linux-rv32/ -j $(nproc)
>>          $ make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- O=../linux-rv64 defconfig
>>          $ make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- O=../linux-rv64/ -j $(nproc)
>>          (Get Image in ../linux-rv32/arch/riscv/boot)
>>
>>   - Prepare QEMU
>>          $ git clonehttps://github.com/romanheros/qemu.git -b riscv-upstream-sxl32-v1
>>          $ cd qemu
>>          $ ./configure --target-list="riscv64-softmmu" --enable-slirp
>>          $ cd build; ninja
>>
>> - Run 32-bit Linux boot on 64-bit RISC-V with Opensbi 64 :
>>          $ ./build/qemu-system-riscv64 -cpu rv64,sxl32=on -M virt -nographic \
>>                  -bios opensbi/build/platform/generic/firmware/fw_dynamic.bin \
>>                  -kernel linux-rv32/arch/riscv/boot/Image \
>>                  -append "root=/dev/vda ro console=ttyS0" \
>>                  -drive file=qemu_riscv32_virt_defconfig/images/rootfs.ext2,format=raw,id=hd0 \
>>                  -netdev user,id=net0 -device virtio-net-device,netdev=net0
>> ```
>> OpenSBI v1.4-112-g7b4bc9d
>> Build time: 2024-06-17 06:57:42 +0000
>> Build compiler: gcc version 13.2.0 (gc891d8dc23e-dirty)
>>     ____                    _____ ____ _____
>>    / __ \                  / ____|  _ \_   _|
>>   | |  | |_ __   ___ _ __ | (___ | |_) || |
>>   | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
>>   | |__| | |_) |  __/ | | |____) | |_) || |_
>>    \____/| .__/ \___|_| |_|_____/|____/_____|
>>          | |
>>          |_|
>>
>> Platform Name             : riscv-virtio,qemu
>> Platform Features         : medeleg
>> Platform HART Count       : 1
>> Platform IPI Device       : aclint-mswi
>> Platform Timer Device     : aclint-mtimer @ 10000000Hz
>> Platform Console Device   : uart8250
>> Platform HSM Device       : ---
>> Platform PMU Device       : ---
>> Platform Reboot Device    : syscon-reboot
>> Platform Shutdown Device  : syscon-poweroff
>> Platform Suspend Device   : ---
>> Platform CPPC Device      : ---
>> Firmware Base             : 0x80000000
>> Firmware Size             : 583 KB
>> Firmware RW Offset        : 0x80000
>> Firmware RW Size          : 71 KB
>> Firmware Heap Offset      : 0x89000
>> Firmware Heap Size        : 35 KB (total), 2 KB (reserved), 11 KB (used), 21 KB (free)
>> Firmware Scratch Size     : 4096 B (total), 400 B (used), 3696 B (free)
>> Runtime SBI Version       : 2.0
>>
>> Domain0 Name              : root
>> Domain0 Boot HART         : 0
>> Domain0 HARTs             : 0*
>> Domain0 Region00          : 0x0000000000100000-0x0000000000100fff M: (I,R,W) S/U: (R,W)
>> Domain0 Region01          : 0x0000000010000000-0x0000000010000fff M: (I,R,W) S/U: (R,W)
>> Domain0 Region02          : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: ()
>> Domain0 Region03          : 0x0000000080080000-0x000000008009ffff M: (R,W) S/U: ()
>> Domain0 Region04          : 0x0000000080000000-0x000000008007ffff M: (R,X) S/U: ()
>> Domain0 Region05          : 0x000000000c400000-0x000000000c5fffff M: (I,R,W) S/U: (R,W)
>> Domain0 Region06          : 0x000000000c000000-0x000000000c3fffff M: (I,R,W) S/U: (R,W)
>> Domain0 Region07          : 0x0000000000000000-0xffffffffffffffff M: () S/U: (R,W,X)
>> Domain0 Next Address      : 0x0000000080200000
>> Domain0 Next Arg1         : 0x0000000087e00000
>> Domain0 Next Mode         : S-mode
>> Domain0 SysReset          : yes
>> Domain0 SysSuspend        : yes
>>
>> Boot HART ID              : 0
>> Boot HART Domain          : root
>> Boot HART Priv Version    : v1.12
>> Boot HART Base ISA        : rv64imafdch
>> Boot HART ISA Extensions  : sstc,zicntr,zihpm,zicboz,zicbom,sdtrig
>> Boot HART PMP Count       : 16
>> Boot HART PMP Granularity : 2 bits
>> Boot HART PMP Address Bits: 54
>> Boot HART MHPM Info       : 16 (0x0007fff8)
>> Boot HART Debug Triggers  : 2 triggers
>> Boot HART MIDELEG         : 0x0000000000001666
>> Boot HART MEDELEG         : 0x0000000000f0b509
>> [    0.000000] Linux version 6.10.0-rc3-00100-gec0abe37042f (developer@95e6e39b54aa) (riscv32-unknown-linux-gnu-gcc (gc891d8dc23e-dirty) 13.2.0, GNU ld (GNU Binutils) 2.42) #1 SMP Sat Jun 15 13:01:52 UTC 2024
>> [    0.000000] random: crng init done
>> [    0.000000] Machine model: riscv-virtio,qemu
>> [    0.000000] SBI specification v2.0 detected
>> [    0.000000] SBI implementation ID=0x1 Version=0x10004
>> [    0.000000] SBI TIME extension detected
>> [    0.000000] SBI IPI extension detected
>> [    0.000000] SBI RFENCE extension detected
>> [    0.000000] SBI SRST extension detected
>> [    0.000000] SBI DBCN extension detected
>> [    0.000000] efi: UEFI not found.
>> [    0.000000] OF: reserved mem: 0x0000000080000000..0x000000008007ffff (512 KiB) nomap non-reusable mmode_resv1@80000000
>> [    0.000000] OF: reserved mem: 0x0000000080080000..0x000000008009ffff (128 KiB) nomap non-reusable mmode_resv0@80080000
>> [    0.000000] Zone ranges:
>> [    0.000000]   DMA32    [mem 0x0000000080000000-0x0000000087ffffff]
>> [    0.000000]   Normal   empty
>> [    0.000000] Movable zone start for each node
>> [    0.000000] Early memory node ranges
>> [    0.000000]   node   0: [mem 0x0000000080000000-0x000000008009ffff]
>> [    0.000000]   node   0: [mem 0x00000000800a0000-0x0000000087ffffff]
>> [    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x0000000087ffffff]
>> [    0.000000] SBI HSM extension detected
>> [    0.000000] riscv: base ISA extensions acdfhim
>> [    0.000000] riscv: ELF capabilities acdfim
>> ......
>> Starting network: udhcpc: started, v1.36.1
>> udhcpc: broadcasting discover
>> udhcpc: broadcasting select for 10.0.2.15, server 10.0.2.2
>> udhcpc: lease of 10.0.2.15 obtained from 10.0.2.2, lease time 86400
>> deleting routers
>> adding dns 10.0.2.3
>> OK
>>
>> Welcome to Buildroot
>> buildroot login: root
>> # cat /proc/cpuinfo
>> processor       : 0
>> hart            : 0
>> isa             : rv64imafdch_zicbom_zicboz_zicntr_zicsr_zifencei_zihintntl_zihintpause_zihpm_zfa_zba_zbb_zbc_zbs_sstc
>> mmu             : sv57
>> mvendorid       : 0x0
>> marchid         : 0x0
>> mimpid          : 0x0
>> hart isa        : rv64imafdch_zicbom_zicboz_zicntr_zicsr_zifencei_zihintntl_zihintpause_zihpm_zfa_zba_zbb_zbc_zbs_sstc
>> ```
>>
>> This doesn't mean we have addressed all the challenges for this problem. The to do list is here:
>> 1. OpenSBI need accept a parameter from firmware the next boot level is SXL32 instead of only S mode.
>> 2. OpenSBI should dynamically configure the SXL32 environment for next boot.
>> 3. OpenSBI should ensure SBI calls for RV32 are right implemented even M mode is RV64.
>> 4. OpenSBI should override details of machine configuration for RV32 Linux, such as rv64i ISA description in DTB.
>>
>> Having all of them been addressed, we can allow the SXL write in mstatus. But this todo list depends on
>> a runnable environment, which is provided by this patch set.
>>
>> This patch set is based on the patch set[1] under review. You can also fetch the branch from my repo[2].
>>
>> [1]:https://mail.gnu.org/archive/html/qemu-riscv/2024-09/msg00501.html
>> [2]:https://github.com/romanheros/qemu/tree/riscv-upstream-sxl32-v1
>>
>> TANG Tiancheng (7):
>>    target/riscv: Fix sstatus read and write
>>    target/riscv: Fix satp read and write implicitly or explicitly.
>>    target/riscv: Read pte and satp based on SXL in PTW
>>    hw/riscv: Align kernel to 4MB when sxl32 is on.
>>    target/riscv: Enable 32-bit only registers for RV64 with sxl32
>>    target/riscv: Reset SXL and UXL according to sxl32
>>    target/riscv: Expose sxl32 configuration in RISC-V CPU
>>
>>   hw/riscv/boot.c           |  3 ++-
>>   target/riscv/cpu.c        | 14 ++++++++++++--
>>   target/riscv/cpu_cfg.h    |  4 ++++
>>   target/riscv/cpu_helper.c |  4 ++--
>>   target/riscv/csr.c        | 38 +++++++++++++++++++++++++-------------
>>   5 files changed, 45 insertions(+), 18 deletions(-)
>>
>> --
>> 2.43.0
>>
>>

[-- Attachment #2: Type: text/html, Size: 11808 bytes --]