KVM: riscv: selftests: sv39 support needed?

KVM: riscv: selftests: sv39 support needed?

[wu.fei9]

I was running kvm selftests on qemu-system-riscv64 with "-cpu rva23s64", which
uses sv39 by default, then I got the following error. After some debugging I
found sv39 is not supported for selftests, should we add sv39 support? I can
prepare a patch if it's needed.

# ./arch_timer
Random seed: 0x6b8b4567
==== Test Assertion Failure ====
  arch_timer.c:73: false
  pid=144 tid=146 errno=0 - Success

Regards,
Fei.

-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

Re: KVM: riscv: selftests: sv39 support needed?

[Andrew Jones]

On Wed, Sep 17, 2025 at 10:11:05AM +0800, wu.fei9@sanechips.com.cn wrote:
> I was running kvm selftests on qemu-system-riscv64 with "-cpu rva23s64", which
> uses sv39 by default, then I got the following error. After some debugging I
> found sv39 is not supported for selftests, should we add sv39 support? I can
> prepare a patch if it's needed.

Yes, please.

> 
> # ./arch_timer
> Random seed: 0x6b8b4567
> ==== Test Assertion Failure ====
>   arch_timer.c:73: false
>   pid=144 tid=146 errno=0 - Success

We can avoid errors for the default case by adding a
kvm_selftest_arch_init() function for riscv which calls
guest_modes_append_default(). We'll also need to modify
guest_modes_append_default() to configure the default using information
from KVM that can be obtained with a get-one-reg of the satp_mode config
register. OIOW, use satp_mode but otherwise just mimic arm64's setup.

Thanks,
drew

-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

Re: KVM: riscv: selftests: sv39 support needed?

[Wu Fei]

On 9/18/25 22:13, Andrew Jones wrote:
> On Wed, Sep 17, 2025 at 10:11:05AM +0800, wu.fei9@sanechips.com.cn wrote:
>> I was running kvm selftests on qemu-system-riscv64 with "-cpu rva23s64", which
>> uses sv39 by default, then I got the following error. After some debugging I
>> found sv39 is not supported for selftests, should we add sv39 support? I can
>> prepare a patch if it's needed.
> Yes, please.


Should we add another mode as VM_MODE_P41V39_4K? The smallest VA is 
47bits on the existing modes.

Also, is it necessary to add mode for sv57? Or we can just use some 
existing ones such as P52V48 instead.

>
>> # ./arch_timer
>> Random seed: 0x6b8b4567
>> ==== Test Assertion Failure ====
>>    arch_timer.c:73: false
>>    pid=144 tid=146 errno=0 - Success
> We can avoid errors for the default case by adding a
> kvm_selftest_arch_init() function for riscv which calls
> guest_modes_append_default(). We'll also need to modify
> guest_modes_append_default() to configure the default using information
> from KVM that can be obtained with a get-one-reg of the satp_mode config
> register. OIOW, use satp_mode but otherwise just mimic arm64's setup.


Is the current kvm_check_cap(KVM_CAP_VM_GPA_BITS) equal to satp_mode,

e.g. gpa 41-bits always means 8 in satp_mode?


Thanks,

Fei.


>
> Thanks,
> drew
>


-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

Re: KVM: riscv: selftests: sv39 support needed?

[Andrew Jones]

On Thu, Oct 09, 2025 at 08:40:03PM +0800, Wu Fei wrote:
> 
> On 9/18/25 22:13, Andrew Jones wrote:
> > On Wed, Sep 17, 2025 at 10:11:05AM +0800, wu.fei9@sanechips.com.cn wrote:
> > > I was running kvm selftests on qemu-system-riscv64 with "-cpu rva23s64", which
> > > uses sv39 by default, then I got the following error. After some debugging I
> > > found sv39 is not supported for selftests, should we add sv39 support? I can
> > > prepare a patch if it's needed.
> > Yes, please.
> 
> 
> Should we add another mode as VM_MODE_P41V39_4K? The smallest VA is 47bits
> on the existing modes.

Why P41? sv39 supports a 56-bit physical address. To support sv39 I'd
expect to add a VM_MODE_P56V39_4K. And, afaict, the reason we don't have a
VM_MODE_P56V48_4K is simply because when riscv support was added it
didn't add any new modes. We could VM_MODE_P56V48_4K as part of this work.

> 
> Also, is it necessary to add mode for sv57? Or we can just use some existing
> ones such as P52V48 instead.

We need new modes that support 57-bit virtual addresses to test that
support, and I'd like to see that. Again we have a 56-bit physical
address, so the new mode would be VM_MODE_P56V57_4K.

> 
> > 
> > > # ./arch_timer
> > > Random seed: 0x6b8b4567
> > > ==== Test Assertion Failure ====
> > >    arch_timer.c:73: false
> > >    pid=144 tid=146 errno=0 - Success
> > We can avoid errors for the default case by adding a
> > kvm_selftest_arch_init() function for riscv which calls
> > guest_modes_append_default(). We'll also need to modify
> > guest_modes_append_default() to configure the default using information
> > from KVM that can be obtained with a get-one-reg of the satp_mode config
> > register. OIOW, use satp_mode but otherwise just mimic arm64's setup.
> 
> 
> Is the current kvm_check_cap(KVM_CAP_VM_GPA_BITS) equal to satp_mode,
> 
> e.g. gpa 41-bits always means 8 in satp_mode?

No, KVM_CAP_VM_GPA_BITS only tells you the number of physical bits of the
host, which dictates the maximum mode you can choose (see
lib/guest_modes.c). You can get the maximum satp_mode with get-one-reg
RISCV_CONFIG_REG(satp_mode) and then use that or anything smaller than
that for the mode in the test. See lib/guest_modes.c again to see how
there's support in the test framework for looping over modes in order to
run the same tests on multiple modes. To do that, the test will change
the guest configured mode with a set-one-reg of satp_mode.

Thanks,
drew

-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

Re: KVM: riscv: selftests: sv39 support needed?

[Wu Fei]

On 10/9/25 22:40, Andrew Jones wrote:
> On Thu, Oct 09, 2025 at 08:40:03PM +0800, Wu Fei wrote:
>>
>> On 9/18/25 22:13, Andrew Jones wrote:
>>> On Wed, Sep 17, 2025 at 10:11:05AM +0800, wu.fei9@sanechips.com.cn wrote:
>>>> I was running kvm selftests on qemu-system-riscv64 with "-cpu rva23s64", which
>>>> uses sv39 by default, then I got the following error. After some debugging I
>>>> found sv39 is not supported for selftests, should we add sv39 support? I can
>>>> prepare a patch if it's needed.
>>> Yes, please.
>>
>>
>> Should we add another mode as VM_MODE_P41V39_4K? The smallest VA is 47bits
>> on the existing modes.
> 
> Why P41? sv39 supports a 56-bit physical address. To support sv39 I'd
> expect to add a VM_MODE_P56V39_4K. And, afaict, the reason we don't have a
> VM_MODE_P56V48_4K is simply because when riscv support was added it
> didn't add any new modes. We could VM_MODE_P56V48_4K as part of this work.

It's about mode for guest? Which is also limited by hgapt.mode, suppose 
hgapt.mode is Sv39x4, the valid gpa bits is 41.

> 
>>
>> Also, is it necessary to add mode for sv57? Or we can just use some existing
>> ones such as P52V48 instead.
> 
> We need new modes that support 57-bit virtual addresses to test that
> support, and I'd like to see that. Again we have a 56-bit physical
> address, so the new mode would be VM_MODE_P56V57_4K.
> 
>>
>>>
>>>> # ./arch_timer
>>>> Random seed: 0x6b8b4567
>>>> ==== Test Assertion Failure ====
>>>>     arch_timer.c:73: false
>>>>     pid=144 tid=146 errno=0 - Success
>>> We can avoid errors for the default case by adding a
>>> kvm_selftest_arch_init() function for riscv which calls
>>> guest_modes_append_default(). We'll also need to modify
>>> guest_modes_append_default() to configure the default using information
>>> from KVM that can be obtained with a get-one-reg of the satp_mode config
>>> register. OIOW, use satp_mode but otherwise just mimic arm64's setup.
>>
>>
>> Is the current kvm_check_cap(KVM_CAP_VM_GPA_BITS) equal to satp_mode,
>>
>> e.g. gpa 41-bits always means 8 in satp_mode?
> 
> No, KVM_CAP_VM_GPA_BITS only tells you the number of physical bits of the
> host, which dictates the maximum mode you can choose (see
> lib/guest_modes.c). You can get the maximum satp_mode with get-one-reg
> RISCV_CONFIG_REG(satp_mode) and then use that or anything smaller than
> that for the mode in the test. See lib/guest_modes.c again to see how
> there's support in the test framework for looping over modes in order to
> run the same tests on multiple modes. To do that, the test will change
> the guest configured mode with a set-one-reg of satp_mode.

Got it, just check the difference of the two methods:
* KVM_CAP_VM_GPA_BITS - value from hgatp.mode, which is set during 
kvm_riscv_gstage_mode_detect(), represents the hardware capability
* RISCV_CONFIG_REG(satp_mode) - value from satp_mode, which is set 
during set_satp_mode(), decided by both hardware capability and kernel 
cmdline

It looks like current implementation tries to ensure vsatp.mode equals 
to host satp.mode (I don't think it's necessary from spec point of 
view), so that only one mode can be set for guest, in 
kvm_riscv_vcpu_set_reg_config:

         case KVM_REG_RISCV_CONFIG_REG(satp_mode):
                 if (reg_val != (satp_mode >> SATP_MODE_SHIFT))
                         return -EINVAL;
                 break;

Thanks,
Fei.

> 
> Thanks,
> drew


-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

Re: KVM: riscv: selftests: sv39 support needed?

[Andrew Jones]

On Fri, Oct 10, 2025 at 10:11:47PM +0800, Wu Fei wrote:
> On 10/9/25 22:40, Andrew Jones wrote:
> > On Thu, Oct 09, 2025 at 08:40:03PM +0800, Wu Fei wrote:
> > > 
> > > On 9/18/25 22:13, Andrew Jones wrote:
> > > > On Wed, Sep 17, 2025 at 10:11:05AM +0800, wu.fei9@sanechips.com.cn wrote:
> > > > > I was running kvm selftests on qemu-system-riscv64 with "-cpu rva23s64", which
> > > > > uses sv39 by default, then I got the following error. After some debugging I
> > > > > found sv39 is not supported for selftests, should we add sv39 support? I can
> > > > > prepare a patch if it's needed.
> > > > Yes, please.
> > > 
> > > 
> > > Should we add another mode as VM_MODE_P41V39_4K? The smallest VA is 47bits
> > > on the existing modes.
> > 
> > Why P41? sv39 supports a 56-bit physical address. To support sv39 I'd
> > expect to add a VM_MODE_P56V39_4K. And, afaict, the reason we don't have a
> > VM_MODE_P56V48_4K is simply because when riscv support was added it
> > didn't add any new modes. We could VM_MODE_P56V48_4K as part of this work.
> 
> It's about mode for guest? Which is also limited by hgapt.mode, suppose
> hgapt.mode is Sv39x4, the valid gpa bits is 41.

You're right. I forgot what the PXX meant in the mode, but reviewing the
implementation for Arm has now reminded me. Indeed we want
VM_MODE_P41V39_4K.

> 
> > 
> > > 
> > > Also, is it necessary to add mode for sv57? Or we can just use some existing
> > > ones such as P52V48 instead.
> > 
> > We need new modes that support 57-bit virtual addresses to test that
> > support, and I'd like to see that. Again we have a 56-bit physical
> > address, so the new mode would be VM_MODE_P56V57_4K.
> > 
> > > 
> > > > 
> > > > > # ./arch_timer
> > > > > Random seed: 0x6b8b4567
> > > > > ==== Test Assertion Failure ====
> > > > >     arch_timer.c:73: false
> > > > >     pid=144 tid=146 errno=0 - Success
> > > > We can avoid errors for the default case by adding a
> > > > kvm_selftest_arch_init() function for riscv which calls
> > > > guest_modes_append_default(). We'll also need to modify
> > > > guest_modes_append_default() to configure the default using information
> > > > from KVM that can be obtained with a get-one-reg of the satp_mode config
> > > > register. OIOW, use satp_mode but otherwise just mimic arm64's setup.
> > > 
> > > 
> > > Is the current kvm_check_cap(KVM_CAP_VM_GPA_BITS) equal to satp_mode,
> > > 
> > > e.g. gpa 41-bits always means 8 in satp_mode?
> > 
> > No, KVM_CAP_VM_GPA_BITS only tells you the number of physical bits of the
> > host, which dictates the maximum mode you can choose (see
> > lib/guest_modes.c). You can get the maximum satp_mode with get-one-reg
> > RISCV_CONFIG_REG(satp_mode) and then use that or anything smaller than
> > that for the mode in the test. See lib/guest_modes.c again to see how
> > there's support in the test framework for looping over modes in order to
> > run the same tests on multiple modes. To do that, the test will change
> > the guest configured mode with a set-one-reg of satp_mode.

I'm wrong again here. We don't need to use set-one-reg for satp_mode and
it won't work (as you've pointed out below) anyway. The guest should just
use the modes we tell it to use. More about setting satp_mode below.

> 
> Got it, just check the difference of the two methods:
> * KVM_CAP_VM_GPA_BITS - value from hgatp.mode, which is set during
> kvm_riscv_gstage_mode_detect(), represents the hardware capability
> * RISCV_CONFIG_REG(satp_mode) - value from satp_mode, which is set during
> set_satp_mode(), decided by both hardware capability and kernel cmdline
> 
> It looks like current implementation tries to ensure vsatp.mode equals to
> host satp.mode (I don't think it's necessary from spec point of view), so
> that only one mode can be set for guest, in kvm_riscv_vcpu_set_reg_config:
> 
>         case KVM_REG_RISCV_CONFIG_REG(satp_mode):
>                 if (reg_val != (satp_mode >> SATP_MODE_SHIFT))
>                         return -EINVAL;
>                 break;

Right, the spec doesn't restrict the guest to the same mode as the host,
just to the same or narrower. We could relax this check. Of course we
have to trust the guest to do what it's told, not what the hardware says
is possible, anyway though, since satp accesses don't trap. With that in
mind, a VMM doesn't even need this set-one-reg to configure guests since
it knows the maximum width it can describe to the guest from the
get-one-reg of satp and KVM can't do anything with a mode set by the VMM
anyway.

So, as said above, for the multiple guest mode tests the guest will just
need to set the mode we tell it to set.

Thanks,
drew

-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv