[PATCH] qemuriscv: Enable Sv39 memory address scheme by default

[PATCH] qemuriscv: Enable Sv39 memory address scheme by default

[Yash.Shinde]

From: Yash Shinde <Yash.Shinde@windriver.com>

Currently, GCC Asan supports only Sv39 on RISC-V due to custom allocator configurations optimized for large allocations.
These configurations are incompatible with larger address spaces like Sv48/Sv57.
This fix enables RISC-V's configuration for Sv39 by default when running `runqemu nographic`,
eliminating the need for manual configuration via command line flags (`qemuparams="-cpu rv64, sv39=true"`).

Fixes [YOCTO #15691]
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15691

Reference: https://git.openembedded.org/openembedded-core/commit/?id=4b4450ff695ef73bf7a2437e142d2e0730d6a547

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
---
 meta/conf/machine/include/riscv/qemuriscv.inc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/conf/machine/include/riscv/qemuriscv.inc b/meta/conf/machine/include/riscv/qemuriscv.inc
index d01d988eee..6170ce65f2 100644
--- a/meta/conf/machine/include/riscv/qemuriscv.inc
+++ b/meta/conf/machine/include/riscv/qemuriscv.inc
@@ -27,6 +27,7 @@ UBOOT_ENTRYPOINT:riscv64 = "0x80200000"
 # qemuboot options
 QB_SMP ?= "-smp 4"
 QB_KERNEL_CMDLINE_APPEND = "earlycon=sbi"
+QB_CPU = "-cpu rv64, sv39=true"
 QB_MACHINE = "-machine virt"
 QB_DEFAULT_BIOS = "fw_jump.elf"
 QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"
-- 
2.43.0

Re: [PATCH] qemuriscv: Enable Sv39 memory address scheme by default

[Yash Shinde]

[-- Attachment #1: Type: text/plain, Size: 2735 bytes --]

Tested Asan test as follows with command "runqemu nographic"
(with IMAGE_INSTALL:append = " libasan  libasan-dev"):

root@qemuriscv64:~# gcc -g -fsanitize=address -o asan_test asan_test.c

root@qemuriscv64:~# ./asan_test

Running AddressSanitizer test...

=================================================================

==262==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x003f9380002a at pc 0x002add509b88 bp 0x003ff6510ec0 sp 0x003ff6510e90

WRITE of size 1 at 0x003f9380002a thread T0

#0 0x2add509b86 in cause_buffer_overflow /home/root/a.c:9

#1 0x2add509ca2 in main /home/root/a.c:16

#2 0x3f95a917b4 (/lib/libc.so.6+0x277b4) (BuildId: 89440b2a8f8e9f78366a816e4685a39d6a4de4c5)

#3 0x3f95a9185c in __libc_start_main (/lib/libc.so.6+0x2785c) (BuildId: 89440b2a8f8e9f78366a816e4685a39d6a4de4c5)

#4 0x2add509a1e in _start ../sysdeps/riscv/start.S:67

Address 0x003f9380002a is located in stack of thread T0 at offset 42 in frame

#0 0x2add509ac4 in cause_buffer_overflow /home/root/a.c:5

This frame has 1 object(s):

[32, 42) 'buffer' (line 6) <== Memory access at offset 42 overflows this variable

HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork

(longjmp and C++ exceptions *are* supported)

SUMMARY: AddressSanitizer: stack-buffer-overflow /home/root/a.c:9 in cause_buffer_overflow

Shadow bytes around the buggy address:

0x003f937ffd80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0x003f937ffe00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0x003f937ffe80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0x003f937fff00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0x003f937fff80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

=>0x003f93800000: f1 f1 f1 f1 00[02]f3 f3 00 00 00 00 00 00 00 00

0x003f93800080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0x003f93800100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0x003f93800180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0x003f93800200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0x003f93800280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Shadow byte legend (one shadow byte represents 8 application bytes):

Addressable: 00

Partially addressable: 01 02 03 04 05 06 07

Heap left redzone: fa

Freed heap region: fd

Stack left redzone: f1

Stack mid redzone: f2

Stack right redzone: f3

Stack after return: f5

Stack use after scope: f8

Global redzone: f9

Global init order: f6

Poisoned by user: f7

Container overflow: fc

Array cookie: ac

Intra object redzone: bb

ASan internal: fe

Left alloca redzone: ca

Right alloca redzone: cb

[-- Attachment #2: Type: text/html, Size: 9403 bytes --]

Re: [PATCH] qemuriscv: Enable Sv39 memory address scheme by default

[Randy MacLeod]

[-- Attachment #1: Type: text/plain, Size: 2971 bytes --]

Add Raj, Khem, Kexin and Zhe who know more about RISC-V than I do.
Kexin and Zhe are on holiday for a week or more.
CC a few people who have contributed to riscv* conf.

I am not all that familiar with either qemu internals or riscv 
specifications so any input is appreciated.

I have some comments and questions below.

On 2025-01-28 5:16 a.m., Yash.Shinde@windriver.com wrote:
> From: Yash Shinde<Yash.Shinde@windriver.com>
>
> Currently, GCC Asan supports only Sv39 on RISC-V due to custom allocator configurations optimized for large allocations.
> These configurations are incompatible with larger address spaces like Sv48/Sv57.
> This fix enables RISC-V's configuration for Sv39 by default when running `runqemu nographic`,
> eliminating the need for manual configuration via command line flags (`qemuparams="-cpu rv64, sv39=true"`).
>
> Fixes [YOCTO #15691]
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15691
>
> Reference:https://git.openembedded.org/openembedded-core/commit/?id=4b4450ff695ef73bf7a2437e142d2e0730d6a547
>
> Signed-off-by: Yash Shinde<Yash.Shinde@windriver.com>
> ---
>   meta/conf/machine/include/riscv/qemuriscv.inc | 1 +
>   1 file changed, 1 insertion(+)
>
> diff --git a/meta/conf/machine/include/riscv/qemuriscv.inc b/meta/conf/machine/include/riscv/qemuriscv.inc
> index d01d988eee..6170ce65f2 100644
> --- a/meta/conf/machine/include/riscv/qemuriscv.inc
> +++ b/meta/conf/machine/include/riscv/qemuriscv.inc
> @@ -27,6 +27,7 @@ UBOOT_ENTRYPOINT:riscv64 = "0x80200000"
>   # qemuboot options
>   QB_SMP ?= "-smp 4"
>   QB_KERNEL_CMDLINE_APPEND = "earlycon=sbi"
> +QB_CPU = "-cpu rv64, sv39=true"

Thanks Yash.

To save everyone from doing the math, sv39 gives the system 512 GB 
virtual address space.

Should we use the ?= operator here so that users can easily override 
this specification
just as is done for  QB_SMP two lines above?

Please summarize what testing you did.

Did you test with TCLIBC=musl ?

Since you are changing qemuriscv.inc you and the YP AB should test 
qemuriscv32 and qemuriscv64.
qemuriscv32 should, if I'm understanding how this works, be unaffected 
since it has to use sv32 mode.

Did you find out why sv39 isn't the default for qemu?
If not, please check the git history and it there isn't an answer there, 
ask one of the qemu developers
who work on risc-v. It would be nice to have a sentence about that in 
the commit log.

Did you consider a different RISC-V cpu sub-arch such as rva22s64 ?

https://github.com/qemu/qemu/commit/3ba8462c4ca55720239fdc6fa30b9870d22a8814

I'm only asking because it seems like it might be a good fit for us and 
I noticed it when
searching the qemu commit logs looking for sv39.


../Randy




>   QB_MACHINE = "-machine virt"
>   QB_DEFAULT_BIOS = "fw_jump.elf"
>   QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"


-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2: Type: text/html, Size: 4864 bytes --]

Re: [PATCH] qemuriscv: Enable Sv39 memory address scheme by default

[Khem Raj]

I am fine with this change as is.

On Tue, Jan 28, 2025 at 7:51 AM Randy MacLeod
<randy.macleod@windriver.com> wrote:
>
> Add Raj, Khem, Kexin and Zhe who know more about RISC-V than I do.
> Kexin and Zhe are on holiday for a week or more.
> CC a few people who have contributed to riscv* conf.
>
> I am not all that familiar with either qemu internals or riscv specifications so any input is appreciated.
>
> I have some comments and questions below.
>
> On 2025-01-28 5:16 a.m., Yash.Shinde@windriver.com wrote:
>
> From: Yash Shinde <Yash.Shinde@windriver.com>
>
> Currently, GCC Asan supports only Sv39 on RISC-V due to custom allocator configurations optimized for large allocations.
> These configurations are incompatible with larger address spaces like Sv48/Sv57.
> This fix enables RISC-V's configuration for Sv39 by default when running `runqemu nographic`,
> eliminating the need for manual configuration via command line flags (`qemuparams="-cpu rv64, sv39=true"`).
>
> Fixes [YOCTO #15691]
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15691
>
> Reference: https://git.openembedded.org/openembedded-core/commit/?id=4b4450ff695ef73bf7a2437e142d2e0730d6a547
>
> Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
> ---
>  meta/conf/machine/include/riscv/qemuriscv.inc | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/meta/conf/machine/include/riscv/qemuriscv.inc b/meta/conf/machine/include/riscv/qemuriscv.inc
> index d01d988eee..6170ce65f2 100644
> --- a/meta/conf/machine/include/riscv/qemuriscv.inc
> +++ b/meta/conf/machine/include/riscv/qemuriscv.inc
> @@ -27,6 +27,7 @@ UBOOT_ENTRYPOINT:riscv64 = "0x80200000"
>  # qemuboot options
>  QB_SMP ?= "-smp 4"
>  QB_KERNEL_CMDLINE_APPEND = "earlycon=sbi"
> +QB_CPU = "-cpu rv64, sv39=true"
>
> Thanks Yash.
>
> To save everyone from doing the math, sv39 gives the system 512 GB virtual address space.
>
> Should we use the ?= operator here so that users can easily override this specification
> just as is done for  QB_SMP two lines above?
>
> Please summarize what testing you did.
>
> Did you test with TCLIBC=musl ?
>
> Since you are changing qemuriscv.inc you and the YP AB should test qemuriscv32 and qemuriscv64.
> qemuriscv32 should, if I'm understanding how this works, be unaffected since it has to use sv32 mode.
>
> Did you find out why sv39 isn't the default for qemu?
> If not, please check the git history and it there isn't an answer there, ask one of the qemu developers
> who work on risc-v. It would be nice to have a sentence about that in the commit log.
>
> Did you consider a different RISC-V cpu sub-arch such as rva22s64 ?
>
>    https://github.com/qemu/qemu/commit/3ba8462c4ca55720239fdc6fa30b9870d22a8814
>
> I'm only asking because it seems like it might be a good fit for us and I noticed it when
> searching the qemu commit logs looking for sv39.
>
>
> ../Randy
>
>
>
>
>  QB_MACHINE = "-machine virt"
>  QB_DEFAULT_BIOS = "fw_jump.elf"
>  QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"
>
>
> --
> # Randy MacLeod
> # Wind River Linux

Re: [OE-core] [PATCH] qemuriscv: Enable Sv39 memory address scheme by default

[Alistair Francis]

On Wed, Jan 29, 2025 at 1:51 AM Randy MacLeod via
lists.openembedded.org
<randy.macleod=windriver.com@lists.openembedded.org> wrote:
>
> Add Raj, Khem, Kexin and Zhe who know more about RISC-V than I do.
> Kexin and Zhe are on holiday for a week or more.
> CC a few people who have contributed to riscv* conf.
>
> I am not all that familiar with either qemu internals or riscv specifications so any input is appreciated.
>
> I have some comments and questions below.
>
> On 2025-01-28 5:16 a.m., Yash.Shinde@windriver.com wrote:
>
> From: Yash Shinde <Yash.Shinde@windriver.com>
>
> Currently, GCC Asan supports only Sv39 on RISC-V due to custom allocator configurations optimized for large allocations.
> These configurations are incompatible with larger address spaces like Sv48/Sv57.
> This fix enables RISC-V's configuration for Sv39 by default when running `runqemu nographic`,
> eliminating the need for manual configuration via command line flags (`qemuparams="-cpu rv64, sv39=true"`).
>
> Fixes [YOCTO #15691]
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15691
>
> Reference: https://git.openembedded.org/openembedded-core/commit/?id=4b4450ff695ef73bf7a2437e142d2e0730d6a547
>
> Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
> ---
>  meta/conf/machine/include/riscv/qemuriscv.inc | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/meta/conf/machine/include/riscv/qemuriscv.inc b/meta/conf/machine/include/riscv/qemuriscv.inc
> index d01d988eee..6170ce65f2 100644
> --- a/meta/conf/machine/include/riscv/qemuriscv.inc
> +++ b/meta/conf/machine/include/riscv/qemuriscv.inc
> @@ -27,6 +27,7 @@ UBOOT_ENTRYPOINT:riscv64 = "0x80200000"
>  # qemuboot options
>  QB_SMP ?= "-smp 4"
>  QB_KERNEL_CMDLINE_APPEND = "earlycon=sbi"
> +QB_CPU = "-cpu rv64, sv39=true"
>
> Thanks Yash.
>
> To save everyone from doing the math, sv39 gives the system 512 GB virtual address space.
>
> Should we use the ?= operator here so that users can easily override this specification
> just as is done for  QB_SMP two lines above?
>
> Please summarize what testing you did.
>
> Did you test with TCLIBC=musl ?
>
> Since you are changing qemuriscv.inc you and the YP AB should test qemuriscv32 and qemuriscv64.
> qemuriscv32 should, if I'm understanding how this works, be unaffected since it has to use sv32 mode.
>
> Did you find out why sv39 isn't the default for qemu?

QEMU defaults to the max supported SATP mode. I'm not sure there is a
specific reason, that's just what we have always done since we allowed
a user to specify the SATP mode.

> If not, please check the git history and it there isn't an answer there, ask one of the qemu developers
> who work on risc-v. It would be nice to have a sentence about that in the commit log.
>
> Did you consider a different RISC-V cpu sub-arch such as rva22s64 ?

I think defaulting to a profile (RVA22S64) would be the way to go,
that will also enable a known set of extensions.

Alistair

>
>    https://github.com/qemu/qemu/commit/3ba8462c4ca55720239fdc6fa30b9870d22a8814
>
> I'm only asking because it seems like it might be a good fit for us and I noticed it when
> searching the qemu commit logs looking for sv39.
>
>
> ../Randy
>
>
>
>
>  QB_MACHINE = "-machine virt"
>  QB_DEFAULT_BIOS = "fw_jump.elf"
>  QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"
>
>
> --
> # Randy MacLeod
> # Wind River Linux
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#210329): https://lists.openembedded.org/g/openembedded-core/message/210329
> Mute This Topic: https://lists.openembedded.org/mt/110856569/3619028
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alistair23@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [PATCH] qemuriscv: Enable Sv39 memory address scheme by default

[Yash Shinde]

[-- Attachment #1: Type: text/plain, Size: 6802 bytes --]


On 29-01-2025 06:08, Alistair Francis wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On Wed, Jan 29, 2025 at 1:51 AM Randy MacLeod via
> lists.openembedded.org
> <randy.macleod=windriver.com@lists.openembedded.org> wrote:
>> Add Raj, Khem, Kexin and Zhe who know more about RISC-V than I do.
>> Kexin and Zhe are on holiday for a week or more.
>> CC a few people who have contributed to riscv* conf.
>>
>> I am not all that familiar with either qemu internals or riscv specifications so any input is appreciated.
>>
>> I have some comments and questions below.
>>
>> On 2025-01-28 5:16 a.m.,Yash.Shinde@windriver.com wrote:
>>
>> From: Yash Shinde<Yash.Shinde@windriver.com>
>>
>> Currently, GCC Asan supports only Sv39 on RISC-V due to custom allocator configurations optimized for large allocations.
>> These configurations are incompatible with larger address spaces like Sv48/Sv57.
>> This fix enables RISC-V's configuration for Sv39 by default when running `runqemu nographic`,
>> eliminating the need for manual configuration via command line flags (`qemuparams="-cpu rv64, sv39=true"`).
>>
>> Fixes [YOCTO #15691]
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15691
>>
>> Reference:https://git.openembedded.org/openembedded-core/commit/?id=4b4450ff695ef73bf7a2437e142d2e0730d6a547
>>
>> Signed-off-by: Yash Shinde<Yash.Shinde@windriver.com>
>> ---
>>   meta/conf/machine/include/riscv/qemuriscv.inc | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/meta/conf/machine/include/riscv/qemuriscv.inc b/meta/conf/machine/include/riscv/qemuriscv.inc
>> index d01d988eee..6170ce65f2 100644
>> --- a/meta/conf/machine/include/riscv/qemuriscv.inc
>> +++ b/meta/conf/machine/include/riscv/qemuriscv.inc
>> @@ -27,6 +27,7 @@ UBOOT_ENTRYPOINT:riscv64 = "0x80200000"
>>   # qemuboot options
>>   QB_SMP ?= "-smp 4"
>>   QB_KERNEL_CMDLINE_APPEND = "earlycon=sbi"
>> +QB_CPU = "-cpu rv64, sv39=true"
>>
>> Thanks Yash.
>>
>> To save everyone from doing the math, sv39 gives the system 512 GB virtual address space.
>>
>> Should we use the ?= operator here so that users can easily override this specification
>> just as is done for  QB_SMP two lines above?
>>
>> Please summarize what testing you did.
I tested a buffer overflow case for asan by booting qemuriscv64 and 
using the command "runqemu nographic",
without passing any additional qemu parameters, as mentioned in the 
follow-up message to the patch:
https://lists.openembedded.org/g/openembedded-core/message/210317
>>
>> Did you test with TCLIBC=musl ?
Musl builds for asan are still broken as mentioned earlier in 
"gcc-sanitizers.inc" :
/
# musl is currently broken entirely
COMPATIBLE_HOST_libc-musl = 'null'/

http://git.openembedded.org/openembedded-core/tree/meta/recipes-devtools/gcc/gcc-sanitizers.inc#n88

When built with TCLIBC="musl", following error is seen:/
"gcc-sanitizers RPROVIDES libasan but was skipped: incompatible with 
host riscv64-poky-linux-musl (not in COMPATIBLE_HOST)"/
>>
>> Since you are changing qemuriscv.inc you and the YP AB should test qemuriscv32 and qemuriscv64.
>> qemuriscv32 should, if I'm understanding how this works, be unaffected since it has to use sv32 mode.
As mentioned in the below gcc mailing list, risc-v 32 does not officialy 
support GCC's ASan and some of other 32-bit targets. So, this should not 
affect it.
https://gcc.gnu.org/pipermail/gcc-patches/2020-October/557406.html


Also, there is no compatible host for riscv-32 in gcc-sanitizers.inc:
https://git.openembedded.org/openembedded-core/commit/?id=7d90d2a70f0184ad715e9917d3e7aa096cf98f79

Note: While testing this I found that the "-cpu rv64, sv39=true" also 
passes when building for risc-v 32.
I will send a V2 where it only applies to risc-v 64.

>>
>> Did you find out why sv39 isn't the default for qemu?
> QEMU defaults to the max supported SATP mode. I'm not sure there is a
> specific reason, that's just what we have always done since we allowed
> a user to specify the SATP mode.
Supporting to the above comments, here is the commit that allows users 
to specify the SATP mode

https://github.com/qemu/qemu/commit/6f23aaeb9be13d4a26b7f863533c316105481640

It seems that Risc-V has multiple sizes for addressable memory and Linux 
probes for the machine's support at startup.
Thus, the reason would be to give the flexibility to users to set it 
rather than having a fixed default SATP mode.
The authors/reviewers of the commit can put more light on this topic.

>> If not, please check the git history and it there isn't an answer there, ask one of the qemu developers
>> who work on risc-v. It would be nice to have a sentence about that in the commit log.
>>
>> Did you consider a different RISC-V cpu sub-arch such as rva22s64 ?
> I think defaulting to a profile (RVA22S64) would be the way to go,
> that will also enable a known set of extensions.
>
> Alistair
>
>>     https://github.com/qemu/qemu/commit/3ba8462c4ca55720239fdc6fa30b9870d22a8814
>>
>> I'm only asking because it seems like it might be a good fit for us and I noticed it when
>> searching the qemu commit logs looking for sv39.
I added  "-cpu rv64,sv39=true" options for risc-v by default as 
mentioned in the bug requirements.

The support for RVA22S64 looks good to have  provided that it doesn't 
break other risc-v builds/tests being done in Yocto.
Following commit adds the RVA22S64 profile and corresponding extensions 
in qemu.
https://github.com/qemu/qemu/commit/af651969eb23ecf018bcaa8e0761b58425de550c

<https://github.com/qemu/qemu/commit/af651969eb23ecf018bcaa8e0761b58425de550c>I 
also tested with passing /"QB_CPU = "-cpu rv64i, rva22s64=true"" /in 
/qemuriscv.inc/ file and asan works with that as well.
I will send a V2 with setting "rva22s64" profile for risc-v 64 only as 
follows:

*/QB_CPU:riscv64 ?= "-cpu rv64, rva22s64=true"/
*

Regards,
Yash
>>
>>
>> ../Randy
>>
>>
>>
>>
>>   QB_MACHINE = "-machine virt"
>>   QB_DEFAULT_BIOS = "fw_jump.elf"
>>   QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"
>>
>>
>> --
>> # Randy MacLeod
>> # Wind River Linux
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#210329):https://lists.openembedded.org/g/openembedded-core/message/210329
>> Mute This Topic:https://lists.openembedded.org/mt/110856569/3619028
>> Group Owner:openembedded-core+owner@lists.openembedded.org
>> Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [alistair23@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>

[-- Attachment #2: Type: text/html, Size: 11060 bytes --]

Re: [OE-core] [PATCH] qemuriscv: Enable Sv39 memory address scheme by default

[Randy MacLeod]

[-- Attachment #1: Type: text/plain, Size: 7936 bytes --]

On 2025-01-29 7:12 a.m., Yash Shinde wrote:
>
>
> On 29-01-2025 06:08, Alistair Francis wrote:
>> CAUTION: This email comes from a non Wind River email account!
>> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>
>> On Wed, Jan 29, 2025 at 1:51 AM Randy MacLeod via
>> lists.openembedded.org
>> <randy.macleod=windriver.com@lists.openembedded.org> wrote:
>>> Add Raj, Khem, Kexin and Zhe who know more about RISC-V than I do.
>>> Kexin and Zhe are on holiday for a week or more.
>>> CC a few people who have contributed to riscv* conf.
>>>
>>> I am not all that familiar with either qemu internals or riscv specifications so any input is appreciated.
>>>
>>> I have some comments and questions below.
>>>
>>> On 2025-01-28 5:16 a.m.,Yash.Shinde@windriver.com wrote:
>>>
>>> From: Yash Shinde<Yash.Shinde@windriver.com>
>>>
>>> Currently, GCC Asan supports only Sv39 on RISC-V due to custom allocator configurations optimized for large allocations.
>>> These configurations are incompatible with larger address spaces like Sv48/Sv57.
>>> This fix enables RISC-V's configuration for Sv39 by default when running `runqemu nographic`,
>>> eliminating the need for manual configuration via command line flags (`qemuparams="-cpu rv64, sv39=true"`).
>>>
>>> Fixes [YOCTO #15691]
>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15691
>>>
>>> Reference:https://git.openembedded.org/openembedded-core/commit/?id=4b4450ff695ef73bf7a2437e142d2e0730d6a547
>>>
>>> Signed-off-by: Yash Shinde<Yash.Shinde@windriver.com>
>>> ---
>>>   meta/conf/machine/include/riscv/qemuriscv.inc | 1 +
>>>   1 file changed, 1 insertion(+)
>>>
>>> diff --git a/meta/conf/machine/include/riscv/qemuriscv.inc b/meta/conf/machine/include/riscv/qemuriscv.inc
>>> index d01d988eee..6170ce65f2 100644
>>> --- a/meta/conf/machine/include/riscv/qemuriscv.inc
>>> +++ b/meta/conf/machine/include/riscv/qemuriscv.inc
>>> @@ -27,6 +27,7 @@ UBOOT_ENTRYPOINT:riscv64 = "0x80200000"
>>>   # qemuboot options
>>>   QB_SMP ?= "-smp 4"
>>>   QB_KERNEL_CMDLINE_APPEND = "earlycon=sbi"
>>> +QB_CPU = "-cpu rv64, sv39=true"
>>>
>>> Thanks Yash.
>>>
>>> To save everyone from doing the math, sv39 gives the system 512 GB virtual address space.
>>>
>>> Should we use the ?= operator here so that users can easily override this specification
>>> just as is done for  QB_SMP two lines above?
>>>
>>> Please summarize what testing you did.
> I tested a buffer overflow case for asan by booting qemuriscv64 and 
> using the command "runqemu nographic",
> without passing any additional qemu parameters, as mentioned in the 
> follow-up message to the patch:
> https://lists.openembedded.org/g/openembedded-core/message/210317
>>> Did you test with TCLIBC=musl ?
> Musl builds for asan are still broken as mentioned earlier in 
> "gcc-sanitizers.inc" :
> /
> # musl is currently broken entirely
> COMPATIBLE_HOST_libc-musl = 'null'/
>
> http://git.openembedded.org/openembedded-core/tree/meta/recipes-devtools/gcc/gcc-sanitizers.inc#n88
>
> When built with TCLIBC="musl", following error is seen:/
> "gcc-sanitizers RPROVIDES libasan but was skipped: incompatible with 
> host riscv64-poky-linux-musl (not in COMPATIBLE_HOST)"/
>>> Since you are changing qemuriscv.inc you and the YP AB should test qemuriscv32 and qemuriscv64.
>>> qemuriscv32 should, if I'm understanding how this works, be unaffected since it has to use sv32 mode.
> As mentioned in the below gcc mailing list, risc-v 32 does not 
> officialy support GCC's ASan and some of other 32-bit targets. So, 
> this should not affect it.
> https://gcc.gnu.org/pipermail/gcc-patches/2020-October/557406.html
>
>
> Also, there is no compatible host for riscv-32 in gcc-sanitizers.inc:
> https://git.openembedded.org/openembedded-core/commit/?id=7d90d2a70f0184ad715e9917d3e7aa096cf98f79
>
> Note: While testing this I found that the "-cpu rv64, sv39=true" also 
> passes when building for risc-v 32.
> I will send a V2 where it only applies to risc-v 64.
>

Thanks for duplicating and explaining your testing. I noticed your test 
email after I had replied
to the main thread because you didn't CC me on the testing email. Oops 
... on both our parts.

Anyway, this started out being about just GCC ASan but if you're 
changing qemu's config,
you'll have to build that emulator and use it a bit before sending in 
v2.  Perhaps run the full
GCC test suite and say openssl's ptests. The rest of the ptest will 
likely be run as part of the YP AB
testing.



>>> Did you find out why sv39 isn't the default for qemu?
>> QEMU defaults to the max supported SATP mode. I'm not sure there is a
>> specific reason, that's just what we have always done since we allowed
>> a user to specify the SATP mode.
> Supporting to the above comments, here is the commit that allows users 
> to specify the SATP mode
>
> https://github.com/qemu/qemu/commit/6f23aaeb9be13d4a26b7f863533c316105481640
>
> It seems that Risc-V has multiple sizes for addressable memory and 
> Linux probes for the machine's support at startup.
> Thus, the reason would be to give the flexibility to users to set it 
> rather than having a fixed default SATP mode.
> The authors/reviewers of the commit can put more light on this topic.
>
>>> If not, please check the git history and it there isn't an answer there, ask one of the qemu developers
>>> who work on risc-v. It would be nice to have a sentence about that in the commit log.
>>>
>>> Did you consider a different RISC-V cpu sub-arch such as rva22s64 ?
>> I think defaulting to a profile (RVA22S64) would be the way to go,
>> that will also enable a known set of extensions.
>>
>> Alistair
>>
>>>     https://github.com/qemu/qemu/commit/3ba8462c4ca55720239fdc6fa30b9870d22a8814
>>>
>>> I'm only asking because it seems like it might be a good fit for us and I noticed it when
>>> searching the qemu commit logs looking for sv39.
> I added  "-cpu rv64,sv39=true" options for risc-v by default as 
> mentioned in the bug requirements.
>
> The support for RVA22S64 looks good to have  provided that it doesn't 
> break other risc-v builds/tests being done in Yocto.
> Following commit adds the RVA22S64 profile and corresponding 
> extensions in qemu.
> https://github.com/qemu/qemu/commit/af651969eb23ecf018bcaa8e0761b58425de550c
>
> <https://github.com/qemu/qemu/commit/af651969eb23ecf018bcaa8e0761b58425de550c>I 
> also tested with passing /"QB_CPU = "-cpu rv64i, rva22s64=true"" /in 
> /qemuriscv.inc/ file and asan works with that as well.


Good.



> I will send a V2 with setting "rva22s64" profile for risc-v 64 only as 
> follows:
>
> */QB_CPU:riscv64 ?= "-cpu rv64, rva22s64=true"/
> *


Using the weak operator "?=" to allow overrides seems like a good choice 
at least for now.

Do you really need both
    "-cpu rv64, rva22s64=true"
rather than just:

    "-cpu rva22s64"

?

Remember, I don't know much about the qemu internals so maybe this is 
obvious to you.

../Randy



> **
>
> Regards,
> Yash
>>> ../Randy
>>>
>>>
>>>
>>>
>>>   QB_MACHINE = "-machine virt"
>>>   QB_DEFAULT_BIOS = "fw_jump.elf"
>>>   QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"
>>>
>>>
>>> --
>>> # Randy MacLeod
>>> # Wind River Linux
>>>
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#210329):https://lists.openembedded.org/g/openembedded-core/message/210329
>>> Mute This Topic:https://lists.openembedded.org/mt/110856569/3619028
>>> Group Owner:openembedded-core+owner@lists.openembedded.org
>>> Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [alistair23@gmail.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>

-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2: Type: text/html, Size: 13251 bytes --]

Re: [OE-core] [PATCH] qemuriscv: Enable Sv39 memory address scheme by default

[Yash Shinde]

[-- Attachment #1: Type: text/plain, Size: 3485 bytes --]


On 30-01-2025 00:58, Randy MacLeod wrote:
>
>
> Thanks for duplicating and explaining your testing. I noticed your 
> test email after I had replied
> to the main thread because you didn't CC me on the testing email. Oops 
> ... on both our parts.
>
> Anyway, this started out being about just GCC ASan but if you're 
> changing qemu's config,
> you'll have to build that emulator and use it a bit before sending in 
> v2.  Perhaps run the full
> GCC test suite and say openssl's ptests. The rest of the ptest will 
> likely be run as part of the YP AB
> testing.
>
>
I tested openssl ptest and the results are positive.
The full GCC test suite is still running as it takes long time to be 
completed.
I would send a V2 after that.
>
>
>>>> Did you find out why sv39 isn't the default for qemu?
>>> QEMU defaults to the max supported SATP mode. I'm not sure there is a
>>> specific reason, that's just what we have always done since we allowed
>>> a user to specify the SATP mode.
>> Supporting to the above comments, here is the commit that allows 
>> users to specify the SATP mode
>>
>> https://github.com/qemu/qemu/commit/6f23aaeb9be13d4a26b7f863533c316105481640
>>
>> It seems that Risc-V has multiple sizes for addressable memory and 
>> Linux probes for the machine's support at startup.
>> Thus, the reason would be to give the flexibility to users to set it 
>> rather than having a fixed default SATP mode.
>> The authors/reviewers of the commit can put more light on this topic.
>>
>>>> If not, please check the git history and it there isn't an answer there, ask one of the qemu developers
>>>> who work on risc-v. It would be nice to have a sentence about that in the commit log.
>>>>
>>>> Did you consider a different RISC-V cpu sub-arch such as rva22s64 ?
>>> I think defaulting to a profile (RVA22S64) would be the way to go,
>>> that will also enable a known set of extensions.
>>>
>>> Alistair
>>>
>>>>     https://github.com/qemu/qemu/commit/3ba8462c4ca55720239fdc6fa30b9870d22a8814
>>>>
>>>> I'm only asking because it seems like it might be a good fit for us and I noticed it when
>>>> searching the qemu commit logs looking for sv39.
>> I added  "-cpu rv64,sv39=true" options for risc-v by default as 
>> mentioned in the bug requirements.
>>
>> The support for RVA22S64 looks good to have  provided that it doesn't 
>> break other risc-v builds/tests being done in Yocto.
>> Following commit adds the RVA22S64 profile and corresponding 
>> extensions in qemu.
>> https://github.com/qemu/qemu/commit/af651969eb23ecf018bcaa8e0761b58425de550c
>>
>> <https://github.com/qemu/qemu/commit/af651969eb23ecf018bcaa8e0761b58425de550c>I 
>> also tested with passing /"QB_CPU = "-cpu rv64i, rva22s64=true"" /in 
>> /qemuriscv.inc/ file and asan works with that as well.
>
>
> Good.
>
>
>
>> I will send a V2 with setting "rva22s64" profile for risc-v 64 only 
>> as follows:
>>
>> */QB_CPU:riscv64 ?= "-cpu rv64, rva22s64=true"/
>> *
>
>
> Using the weak operator "?=" to allow overrides seems like a good 
> choice at least for now.
>
> Do you really need both
>    "-cpu rv64, rva22s64=true"
> rather than just:
>
>    "-cpu rva22s64"
>
> ?
>
> Remember, I don't know much about the qemu internals so maybe this is 
> obvious to you.
>
Thanks for the suggestion. "-cpu rva22s64" does work.
Even I am new to the qemu internals.


Regards,
Yash
>
> ../Randy
>
>
>

[-- Attachment #2: Type: text/html, Size: 6054 bytes --]

Re: [OE-core] [PATCH] qemuriscv: Enable Sv39 memory address scheme by default

[Khem Raj]

On Wed, Jan 29, 2025 at 11:28 AM Randy MacLeod
<randy.macleod@windriver.com> wrote:
>
> On 2025-01-29 7:12 a.m., Yash Shinde wrote:
>
>
> On 29-01-2025 06:08, Alistair Francis wrote:
>
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On Wed, Jan 29, 2025 at 1:51 AM Randy MacLeod via
> lists.openembedded.org
> <randy.macleod=windriver.com@lists.openembedded.org> wrote:
>
> Add Raj, Khem, Kexin and Zhe who know more about RISC-V than I do.
> Kexin and Zhe are on holiday for a week or more.
> CC a few people who have contributed to riscv* conf.
>
> I am not all that familiar with either qemu internals or riscv specifications so any input is appreciated.
>
> I have some comments and questions below.
>
> On 2025-01-28 5:16 a.m., Yash.Shinde@windriver.com wrote:
>
> From: Yash Shinde <Yash.Shinde@windriver.com>
>
> Currently, GCC Asan supports only Sv39 on RISC-V due to custom allocator configurations optimized for large allocations.
> These configurations are incompatible with larger address spaces like Sv48/Sv57.
> This fix enables RISC-V's configuration for Sv39 by default when running `runqemu nographic`,
> eliminating the need for manual configuration via command line flags (`qemuparams="-cpu rv64, sv39=true"`).
>
> Fixes [YOCTO #15691]
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15691
>
> Reference: https://git.openembedded.org/openembedded-core/commit/?id=4b4450ff695ef73bf7a2437e142d2e0730d6a547
>
> Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
> ---
>  meta/conf/machine/include/riscv/qemuriscv.inc | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/meta/conf/machine/include/riscv/qemuriscv.inc b/meta/conf/machine/include/riscv/qemuriscv.inc
> index d01d988eee..6170ce65f2 100644
> --- a/meta/conf/machine/include/riscv/qemuriscv.inc
> +++ b/meta/conf/machine/include/riscv/qemuriscv.inc
> @@ -27,6 +27,7 @@ UBOOT_ENTRYPOINT:riscv64 = "0x80200000"
>  # qemuboot options
>  QB_SMP ?= "-smp 4"
>  QB_KERNEL_CMDLINE_APPEND = "earlycon=sbi"
> +QB_CPU = "-cpu rv64, sv39=true"
>
> Thanks Yash.
>
> To save everyone from doing the math, sv39 gives the system 512 GB virtual address space.
>
> Should we use the ?= operator here so that users can easily override this specification
> just as is done for  QB_SMP two lines above?
>
> Please summarize what testing you did.
>
> I tested a buffer overflow case for asan by booting qemuriscv64 and using the command "runqemu nographic",
> without passing any additional qemu parameters, as mentioned in the follow-up message to the patch:
> https://lists.openembedded.org/g/openembedded-core/message/210317
>
> Did you test with TCLIBC=musl ?
>
> Musl builds for asan are still broken as mentioned earlier in "gcc-sanitizers.inc" :
>
> # musl is currently broken entirely
> COMPATIBLE_HOST_libc-musl = 'null'
>
> http://git.openembedded.org/openembedded-core/tree/meta/recipes-devtools/gcc/gcc-sanitizers.inc#n88
>
> When built with TCLIBC="musl", following error is seen:
> "gcc-sanitizers RPROVIDES libasan but was skipped: incompatible with host riscv64-poky-linux-musl (not in COMPATIBLE_HOST)"
>
> Since you are changing qemuriscv.inc you and the YP AB should test qemuriscv32 and qemuriscv64.
> qemuriscv32 should, if I'm understanding how this works, be unaffected since it has to use sv32 mode.
>
> As mentioned in the below gcc mailing list, risc-v 32 does not officialy support GCC's ASan and some of other 32-bit targets. So, this should not affect it.
> https://gcc.gnu.org/pipermail/gcc-patches/2020-October/557406.html
>
>
> Also, there is no compatible host for riscv-32 in gcc-sanitizers.inc:
> https://git.openembedded.org/openembedded-core/commit/?id=7d90d2a70f0184ad715e9917d3e7aa096cf98f79
>
> Note: While testing this I found that the "-cpu rv64, sv39=true" also passes when building for risc-v 32.
> I will send a V2 where it only applies to risc-v 64.
>
>
> Thanks for duplicating and explaining your testing. I noticed your test email after I had replied
> to the main thread because you didn't CC me on the testing email. Oops ... on both our parts.
>
> Anyway, this started out being about just GCC ASan but if you're changing qemu's config,
> you'll have to build that emulator and use it a bit before sending in v2.  Perhaps run the full
> GCC test suite and say openssl's ptests. The rest of the ptest will likely be run as part of the YP AB
> testing.
>
>
>
> Did you find out why sv39 isn't the default for qemu?
>
> QEMU defaults to the max supported SATP mode. I'm not sure there is a
> specific reason, that's just what we have always done since we allowed
> a user to specify the SATP mode.
>
> Supporting to the above comments, here is the commit that allows users to specify the SATP mode
>
> https://github.com/qemu/qemu/commit/6f23aaeb9be13d4a26b7f863533c316105481640
>
> It seems that Risc-V has multiple sizes for addressable memory and Linux probes for the machine's support at startup.
> Thus, the reason would be to give the flexibility to users to set it rather than having a fixed default SATP mode.
> The authors/reviewers of the commit can put more light on this topic.
>
> If not, please check the git history and it there isn't an answer there, ask one of the qemu developers
> who work on risc-v. It would be nice to have a sentence about that in the commit log.
>
> Did you consider a different RISC-V cpu sub-arch such as rva22s64 ?
>
> I think defaulting to a profile (RVA22S64) would be the way to go,
> that will also enable a known set of extensions.
>
> Alistair
>
>    https://github.com/qemu/qemu/commit/3ba8462c4ca55720239fdc6fa30b9870d22a8814
>
> I'm only asking because it seems like it might be a good fit for us and I noticed it when
> searching the qemu commit logs looking for sv39.
>
> I added  "-cpu rv64,sv39=true" options for risc-v by default as mentioned in the bug requirements.
>
> The support for RVA22S64 looks good to have  provided that it doesn't break other risc-v builds/tests being done in Yocto.
> Following commit adds the RVA22S64 profile and corresponding extensions in qemu.
> https://github.com/qemu/qemu/commit/af651969eb23ecf018bcaa8e0761b58425de550c
>
> I also tested with passing "QB_CPU = "-cpu rv64i, rva22s64=true""  in qemuriscv.inc file and asan works with that as well.
>
>
> Good.
>
>
>
> I will send a V2 with setting "rva22s64" profile for risc-v 64 only as follows:
>
> QB_CPU:riscv64 ?= "-cpu rv64, rva22s64=true"
>
>
> Using the weak operator "?=" to allow overrides seems like a good choice at least for now.
>
> Do you really need both
>    "-cpu rv64, rva22s64=true"
> rather than just:
>
>    "-cpu rva22s64"
>
> ?


"-cpu rva22s64" directly selects the RVA22S64 processor model as the
complete CPU specification
"-cpu rv64,rva22s64=true" starts with a base RV64 processor and
explicitly enables the RVA22S64 extension as an additional feature

The end result should be functionally equivalent - both will give you
a 64-bit RISC-V processor with the RVA22S64 instruction set extension
enabled. The second format with the explicit feature flag is more
verbose but can be useful when you want to be explicit about enabled
features or combine multiple extensions.



>
> Remember, I don't know much about the qemu internals so maybe this is obvious to you.
>
> ../Randy
>
>
>
>
>
> Regards,
> Yash
>
> ../Randy
>
>
>
>
>  QB_MACHINE = "-machine virt"
>  QB_DEFAULT_BIOS = "fw_jump.elf"
>  QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"
>
>
> --
> # Randy MacLeod
> # Wind River Linux
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#210329): https://lists.openembedded.org/g/openembedded-core/message/210329
> Mute This Topic: https://lists.openembedded.org/mt/110856569/3619028
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alistair23@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
> --
> # Randy MacLeod
> # Wind River Linux