[PATCH] RISC-V: KVM: Using user-mode pte within kvm_riscv_gstage_ioremap

[PATCH] RISC-V: KVM: Using user-mode pte within kvm_riscv_gstage_ioremap

[fangyu.yu]

From: Fangyu Yu <fangyu.yu@linux.alibaba.com>

Currently we use kvm_riscv_gstage_ioremap to map IMSIC gpa to the spa of
guest interrupt file within IMSIC.

The PAGE_KERNEL_IO property does not include user mode settings, so when
accessing the IMSIC address in the virtual machine,  a  guest page fault
will occur, this is not expected.

According to the RISC-V Privileged Architecture Spec, for G-stage address
translation, all memory accesses are considered to be user-level accesses
as though executed in Umode.

Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
---
 arch/riscv/kvm/mmu.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index 1087ea74567b..800064e96ef6 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -351,6 +351,7 @@ int kvm_riscv_gstage_ioremap(struct kvm *kvm, gpa_t gpa,
 	int ret = 0;
 	unsigned long pfn;
 	phys_addr_t addr, end;
+	pgprot_t prot;
 	struct kvm_mmu_memory_cache pcache = {
 		.gfp_custom = (in_atomic) ? GFP_ATOMIC | __GFP_ACCOUNT : 0,
 		.gfp_zero = __GFP_ZERO,
@@ -359,8 +360,11 @@ int kvm_riscv_gstage_ioremap(struct kvm *kvm, gpa_t gpa,
 	end = (gpa + size + PAGE_SIZE - 1) & PAGE_MASK;
 	pfn = __phys_to_pfn(hpa);
 
+	prot = pgprot_noncached(PAGE_WRITE);
+
 	for (addr = gpa; addr < end; addr += PAGE_SIZE) {
-		pte = pfn_pte(pfn, PAGE_KERNEL_IO);
+		pte = pfn_pte(pfn, prot);
+		pte = pte_mkdirty(pte);
 
 		if (!writable)
 			pte = pte_wrprotect(pte);
-- 
2.39.3 (Apple Git-146)

Re: [PATCH] RISC-V: KVM: Using user-mode pte within kvm_riscv_gstage_ioremap

[Nutty.Liu]

On 8/7/2025 3:07 PM, fangyu.yu@linux.alibaba.com wrote:
> From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
>
> Currently we use kvm_riscv_gstage_ioremap to map IMSIC gpa to the spa of
> guest interrupt file within IMSIC.
>
> The PAGE_KERNEL_IO property does not include user mode settings, so when
> accessing the IMSIC address in the virtual machine,  a  guest page fault
> will occur, this is not expected.
>
> According to the RISC-V Privileged Architecture Spec, for G-stage address
> translation, all memory accesses are considered to be user-level accesses
> as though executed in Umode.
>
> Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
> ---
>   arch/riscv/kvm/mmu.c | 6 +++++-
>   1 file changed, 5 insertions(+), 1 deletion(-)
Reviewed-by: Nutty Liu <nutty.liu@hotmail.com>

Thanks,
Nutty
> diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
> index 1087ea74567b..800064e96ef6 100644
> --- a/arch/riscv/kvm/mmu.c
> +++ b/arch/riscv/kvm/mmu.c
> @@ -351,6 +351,7 @@ int kvm_riscv_gstage_ioremap(struct kvm *kvm, gpa_t gpa,
>   	int ret = 0;
>   	unsigned long pfn;
>   	phys_addr_t addr, end;
> +	pgprot_t prot;
>   	struct kvm_mmu_memory_cache pcache = {
>   		.gfp_custom = (in_atomic) ? GFP_ATOMIC | __GFP_ACCOUNT : 0,
>   		.gfp_zero = __GFP_ZERO,
> @@ -359,8 +360,11 @@ int kvm_riscv_gstage_ioremap(struct kvm *kvm, gpa_t gpa,
>   	end = (gpa + size + PAGE_SIZE - 1) & PAGE_MASK;
>   	pfn = __phys_to_pfn(hpa);
>   
> +	prot = pgprot_noncached(PAGE_WRITE);
> +
>   	for (addr = gpa; addr < end; addr += PAGE_SIZE) {
> -		pte = pfn_pte(pfn, PAGE_KERNEL_IO);
> +		pte = pfn_pte(pfn, prot);
> +		pte = pte_mkdirty(pte);
>   
>   		if (!writable)
>   			pte = pte_wrprotect(pte);

Re: [PATCH] RISC-V: KVM: Using user-mode pte within kvm_riscv_gstage_ioremap

[Radim Krčmář]

2025-08-07T15:07:29+08:00, <fangyu.yu@linux.alibaba.com>:
> From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
>
> Currently we use kvm_riscv_gstage_ioremap to map IMSIC gpa to the spa of
                                                                    ^^^
                                                                    hpa?

> guest interrupt file within IMSIC.
>
> The PAGE_KERNEL_IO property does not include user mode settings, so when
> accessing the IMSIC address in the virtual machine,  a  guest page fault
> will occur, this is not expected.

PAGE_KERNEL_IO also set the reserved G bit, so you're fixing two issues
with a single change. :)

> According to the RISC-V Privileged Architecture Spec, for G-stage address
> translation, all memory accesses are considered to be user-level accesses
> as though executed in Umode.

What implementation are you using?  I would have assume that the
original code was tested on QEMU, so we might have a bug there.

> Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
> ---
> diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
> @@ -359,8 +360,11 @@ int kvm_riscv_gstage_ioremap(struct kvm *kvm, gpa_t gpa,
>  	end = (gpa + size + PAGE_SIZE - 1) & PAGE_MASK;
>  	pfn = __phys_to_pfn(hpa);
>  
> +	prot = pgprot_noncached(PAGE_WRITE);
> +
>  	for (addr = gpa; addr < end; addr += PAGE_SIZE) {
> -		pte = pfn_pte(pfn, PAGE_KERNEL_IO);
> +		pte = pfn_pte(pfn, prot);
> +		pte = pte_mkdirty(pte);

Is it necessary to dirty the pte?

It was dirtied before, so it definitely doesn't hurt,

Reviewed-by: Radim Krčmář <rkrcmar@ventanamicro.com>

Thanks.

Re: [PATCH] RISC-V: KVM: Using user-mode pte within kvm_riscv_gstage_ioremap

[fangyu.yu]

>> From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
>>
>> Currently we use kvm_riscv_gstage_ioremap to map IMSIC gpa to the spa of
>                                                                    ^^^
>                                                                    hpa?
>

Yes, I think they mean the same thing, RISC-V IOMMU Spec defines spa
(Supervisor Physical Address).

>> guest interrupt file within IMSIC.
>>
>> The PAGE_KERNEL_IO property does not include user mode settings, so when
>> accessing the IMSIC address in the virtual machine,  a  guest page fault
>> will occur, this is not expected.
>
>PAGE_KERNEL_IO also set the reserved G bit, so you're fixing two issues
>with a single change. :)
>

Right, The G bit in all G-stage PTEs is reserved for future standard use.

>> According to the RISC-V Privileged Architecture Spec, for G-stage address
>> translation, all memory accesses are considered to be user-level accesses
>> as though executed in Umode.
>
>What implementation are you using?  I would have assume that the
>original code was tested on QEMU, so we might have a bug there.
>

This issue can be reproduced using QEMU.
Since kvm has registered the MMIO Bus for IMSIC gpa, when a guest
page fault occurs, it will call the imsic_mmio_write function,the
guest irq will be written to the guest interrupt file by kvm.

>> Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
>> ---
>> diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
>> @@ -359,8 +360,11 @@ int kvm_riscv_gstage_ioremap(struct kvm *kvm, gpa_t gpa,
>>  	end = (gpa + size + PAGE_SIZE - 1) & PAGE_MASK;
>>  	pfn = __phys_to_pfn(hpa);
>>  
>> +	prot = pgprot_noncached(PAGE_WRITE);
>> +
>>  	for (addr = gpa; addr < end; addr += PAGE_SIZE) {
>> -		pte = pfn_pte(pfn, PAGE_KERNEL_IO);
>> +		pte = pfn_pte(pfn, prot);
>> +		pte = pte_mkdirty(pte);
>
>Is it necessary to dirty the pte?
>
>It was dirtied before, so it definitely doesn't hurt,
>

Make pte dirty is necessary(for hardware without Svadu), and here is
the first time to make this pte dirty.

>Reviewed-by: Radim Krčmář <rkrcmar@ventanamicro.com>
>
>Thanks.

Thanks,
fangyu

Re: [PATCH] RISC-V: KVM: Using user-mode pte within kvm_riscv_gstage_ioremap

[Radim Krčmář]

2025-08-09T11:20:20+08:00, <fangyu.yu@linux.alibaba.com>:
>>> From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
>>> According to the RISC-V Privileged Architecture Spec, for G-stage address
>>> translation, all memory accesses are considered to be user-level accesses
>>> as though executed in Umode.
>>
>>What implementation are you using?  I would have assume that the
>>original code was tested on QEMU, so we might have a bug there.
>>
>
> This issue can be reproduced using QEMU.
> Since kvm has registered the MMIO Bus for IMSIC gpa, when a guest
> page fault occurs, it will call the imsic_mmio_write function,the
> guest irq will be written to the guest interrupt file by kvm.

Oh, so the interrupts were "just" slower.  Great job catching that!

>>> ---
>>> diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
>>> @@ -359,8 +360,11 @@ int kvm_riscv_gstage_ioremap(struct kvm *kvm, gpa_t gpa,
>>> +		pte = pte_mkdirty(pte);
>>
>>Is it necessary to dirty the pte?
>>
>>It was dirtied before, so it definitely doesn't hurt,
>
> Make pte dirty is necessary(for hardware without Svadu), and here is
> the first time to make this pte dirty.

Right, we would get a pointless trap otherwise,

Thanks.

Re: [PATCH] RISC-V: KVM: Using user-mode pte within kvm_riscv_gstage_ioremap

[Anup Patel]

On Thu, Aug 7, 2025 at 12:37 PM <fangyu.yu@linux.alibaba.com> wrote:
>
> From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
>
> Currently we use kvm_riscv_gstage_ioremap to map IMSIC gpa to the spa of
> guest interrupt file within IMSIC.
>
> The PAGE_KERNEL_IO property does not include user mode settings, so when
> accessing the IMSIC address in the virtual machine,  a  guest page fault
> will occur, this is not expected.
>
> According to the RISC-V Privileged Architecture Spec, for G-stage address
> translation, all memory accesses are considered to be user-level accesses
> as though executed in Umode.
>
> Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>

Overall, a good fix. Thanks!

The patch subject and description needs improvements. Also, there is no
Fixes tag which is required for backporting.

I have taken care of the above things at the time of merging this patch.

Queued this patch as fixes for Linux-6.17

Thanks,
Anup

> ---
>  arch/riscv/kvm/mmu.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
> index 1087ea74567b..800064e96ef6 100644
> --- a/arch/riscv/kvm/mmu.c
> +++ b/arch/riscv/kvm/mmu.c
> @@ -351,6 +351,7 @@ int kvm_riscv_gstage_ioremap(struct kvm *kvm, gpa_t gpa,
>         int ret = 0;
>         unsigned long pfn;
>         phys_addr_t addr, end;
> +       pgprot_t prot;
>         struct kvm_mmu_memory_cache pcache = {
>                 .gfp_custom = (in_atomic) ? GFP_ATOMIC | __GFP_ACCOUNT : 0,
>                 .gfp_zero = __GFP_ZERO,
> @@ -359,8 +360,11 @@ int kvm_riscv_gstage_ioremap(struct kvm *kvm, gpa_t gpa,
>         end = (gpa + size + PAGE_SIZE - 1) & PAGE_MASK;
>         pfn = __phys_to_pfn(hpa);
>
> +       prot = pgprot_noncached(PAGE_WRITE);
> +
>         for (addr = gpa; addr < end; addr += PAGE_SIZE) {
> -               pte = pfn_pte(pfn, PAGE_KERNEL_IO);
> +               pte = pfn_pte(pfn, prot);
> +               pte = pte_mkdirty(pte);
>
>                 if (!writable)
>                         pte = pte_wrprotect(pte);
> --
> 2.39.3 (Apple Git-146)
>

Re: [PATCH] RISC-V: KVM: Using user-mode pte within kvm_riscv_gstage_ioremap

[fangyu.yu]

>>
>> From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
>>
>> Currently we use kvm_riscv_gstage_ioremap to map IMSIC gpa to the spa of
>> guest interrupt file within IMSIC.
>>
>> The PAGE_KERNEL_IO property does not include user mode settings, so when
>> accessing the IMSIC address in the virtual machine,  a  guest page fault
>> will occur, this is not expected.
>>
>> According to the RISC-V Privileged Architecture Spec, for G-stage address
>> translation, all memory accesses are considered to be user-level accesses
>> as though executed in Umode.
>>
>> Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
>
>Overall, a good fix. Thanks!
>
>The patch subject and description needs improvements. Also, there is no
>Fixes tag which is required for backporting.
>
>I have taken care of the above things at the time of merging this patch.
>
>Queued this patch as fixes for Linux-6.17
>
>Thanks,
>Anup
>

Thanks for your review.
I will send a v2 patch to fix these comments.

Thanks,
fangyu

>> ---
>>  arch/riscv/kvm/mmu.c | 6 +++++-
>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
>> index 1087ea74567b..800064e96ef6 100644
>> --- a/arch/riscv/kvm/mmu.c
>> +++ b/arch/riscv/kvm/mmu.c
>> @@ -351,6 +351,7 @@ int kvm_riscv_gstage_ioremap(struct kvm *kvm, gpa_t gpa,
>>         int ret = 0;
>>         unsigned long pfn;
>>         phys_addr_t addr, end;
>> +       pgprot_t prot;
>>         struct kvm_mmu_memory_cache pcache = {
>>                 .gfp_custom = (in_atomic) ? GFP_ATOMIC | __GFP_ACCOUNT : 0,
>>                 .gfp_zero = __GFP_ZERO,
>> @@ -359,8 +360,11 @@ int kvm_riscv_gstage_ioremap(struct kvm *kvm, gpa_t gpa,
>>         end = (gpa + size + PAGE_SIZE - 1) & PAGE_MASK;
>>         pfn = __phys_to_pfn(hpa);
>>
>> +       prot = pgprot_noncached(PAGE_WRITE);
>> +
>>         for (addr = gpa; addr < end; addr += PAGE_SIZE) {
>> -               pte = pfn_pte(pfn, PAGE_KERNEL_IO);
>> +               pte = pfn_pte(pfn, prot);
>> +               pte = pte_mkdirty(pte);
>>
>>                 if (!writable)
>>                         pte = pte_wrprotect(pte);
>> --
>> 2.39.3 (Apple Git-146)
>>

Re: [PATCH] RISC-V: KVM: Using user-mode pte within kvm_riscv_gstage_ioremap

[Anup Patel]

On Tue, Aug 19, 2025 at 6:56 AM <fangyu.yu@linux.alibaba.com> wrote:
>
> >>
> >> From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
> >>
> >> Currently we use kvm_riscv_gstage_ioremap to map IMSIC gpa to the spa of
> >> guest interrupt file within IMSIC.
> >>
> >> The PAGE_KERNEL_IO property does not include user mode settings, so when
> >> accessing the IMSIC address in the virtual machine,  a  guest page fault
> >> will occur, this is not expected.
> >>
> >> According to the RISC-V Privileged Architecture Spec, for G-stage address
> >> translation, all memory accesses are considered to be user-level accesses
> >> as though executed in Umode.
> >>
> >> Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
> >
> >Overall, a good fix. Thanks!
> >
> >The patch subject and description needs improvements. Also, there is no
> >Fixes tag which is required for backporting.
> >
> >I have taken care of the above things at the time of merging this patch.
> >
> >Queued this patch as fixes for Linux-6.17
> >
> >Thanks,
> >Anup
> >
>
> Thanks for your review.
> I will send a v2 patch to fix these comments.

No need, it's already part of my riscv_kvm_fixes branch at:
https//github.com/kvm-riscv/linux.git

Regards,
Anup