* [PATCH v5 0/2] RISC-V: KVM: Huge page recovery during disable-dirty-log
@ 2026-09-14 5:59 ` Wang Yechao
0 siblings, 0 replies; 10+ messages in thread
From: Wang Yechao @ 2026-09-14 5:59 UTC (permalink / raw)
To: Anup Patel, kvm, kvm-riscv, linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra,
Alexandre Ghiti, Wang Yechao
When dirty logging is enabled, the gstage mappings are split into
4K pages to track dirty pages. If the migration fails or is canceled,
in order to keep the VM's performance consistent with that before
dirty logging was enabled, we need to recover the gstage huge page mappings
when dirty logging is disabled.
With this series, dirty_log_perf_test shows a decrease in the number of
vCPU faults:
$ perf stat -e kvm:kvm_page_fault \
/dirty_log_perf_test -s anonymous_hugetlb_1gb -v 1 -e -b 1G
Before: 1,574,460 kvm:kvm_page_fault
After : 788,048 kvm:kvm_page_fault
Changes in v5:
Fix the issues reported by sashiko
- Use _PAGE_PFN_SHIFT instead of pfn_pte to make huge pte.
- Validate host size via hugetlb_mapping_size() before recovery.
- Skip bad advancement after mmu_recover_huge_pages_range.
Changes in v4:
- Rebased on Linux v7.3-rc2.
- Squash Patch01 into Patch03.
- The 1G mapping recovery is performed only after all 2M leaf recover succeed.
https://lore.kernel.org/linux-riscv/20260913070352.3626956-1-wang.yechao255@zte.com.cn
Changes in v3:
- Rebased on https://github.com/kvm-riscv/linux/tree/riscv_kvm_queue
- Add support to recover 1GB mappings for gstage.
- Refactor kvm_arch_commit_memory_region to be identical to arm64.
(sashiko AI Reviews)
- Scan the child page table for the first valid leaf PTE to recover.
- Add the memslot boundaries check when recover hugepages.
- Move the cond_resched_lock to the start loop iteration.
- The kvm_page_fault tracepoint be placed before the conditional block.
- Remove the fault_supports_gstage_huge_mapping check.
https://lore.kernel.org/linux-riscv/20260605160019772GJy5SDmWQQF7sSqvJ0JtC@zte.com.cn/
Changes in v2:
- Rebased on Linux v7.1-rc1.
- Skip recovering huge pte if the first child pte is not leaf.
- Link to v2:
https://lore.kernel.org/linux-riscv/20260429173002565bdp1o7P0bvOs6yTz64yNx@zte.com.cn/
- Link to v1:
https://lore.kernel.org/linux-riscv/20260407171052241tmZDFGusMP_wlEsBVVtJo@zte.com.cn/
Wang Yechao (2):
RISC-V: KVM: add tracepoint for guest page faults
RISC-V: KVM: Recover gstage huge page mappings during
disable-dirty-log
arch/riscv/include/asm/kvm_gstage.h | 4 +
arch/riscv/kvm/gstage.c | 67 +++++++++++++++
arch/riscv/kvm/mmu.c | 123 +++++++++++++++++++++++++++-
arch/riscv/kvm/trace.h | 22 +++++
arch/riscv/kvm/vcpu_exit.c | 2 +
5 files changed, 217 insertions(+), 1 deletion(-)
--
2.27.0
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v5 0/2] RISC-V: KVM: Huge page recovery during disable-dirty-log
@ 2026-09-14 5:59 ` Wang Yechao
0 siblings, 0 replies; 10+ messages in thread
From: Wang Yechao @ 2026-09-14 5:59 UTC (permalink / raw)
To: Anup Patel, kvm, kvm-riscv, linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra,
Alexandre Ghiti, Wang Yechao
When dirty logging is enabled, the gstage mappings are split into
4K pages to track dirty pages. If the migration fails or is canceled,
in order to keep the VM's performance consistent with that before
dirty logging was enabled, we need to recover the gstage huge page mappings
when dirty logging is disabled.
With this series, dirty_log_perf_test shows a decrease in the number of
vCPU faults:
$ perf stat -e kvm:kvm_page_fault \
/dirty_log_perf_test -s anonymous_hugetlb_1gb -v 1 -e -b 1G
Before: 1,574,460 kvm:kvm_page_fault
After : 788,048 kvm:kvm_page_fault
Changes in v5:
Fix the issues reported by sashiko
- Use _PAGE_PFN_SHIFT instead of pfn_pte to make huge pte.
- Validate host size via hugetlb_mapping_size() before recovery.
- Skip bad advancement after mmu_recover_huge_pages_range.
Changes in v4:
- Rebased on Linux v7.3-rc2.
- Squash Patch01 into Patch03.
- The 1G mapping recovery is performed only after all 2M leaf recover succeed.
https://lore.kernel.org/linux-riscv/20260913070352.3626956-1-wang.yechao255@zte.com.cn
Changes in v3:
- Rebased on https://github.com/kvm-riscv/linux/tree/riscv_kvm_queue
- Add support to recover 1GB mappings for gstage.
- Refactor kvm_arch_commit_memory_region to be identical to arm64.
(sashiko AI Reviews)
- Scan the child page table for the first valid leaf PTE to recover.
- Add the memslot boundaries check when recover hugepages.
- Move the cond_resched_lock to the start loop iteration.
- The kvm_page_fault tracepoint be placed before the conditional block.
- Remove the fault_supports_gstage_huge_mapping check.
https://lore.kernel.org/linux-riscv/20260605160019772GJy5SDmWQQF7sSqvJ0JtC@zte.com.cn/
Changes in v2:
- Rebased on Linux v7.1-rc1.
- Skip recovering huge pte if the first child pte is not leaf.
- Link to v2:
https://lore.kernel.org/linux-riscv/20260429173002565bdp1o7P0bvOs6yTz64yNx@zte.com.cn/
- Link to v1:
https://lore.kernel.org/linux-riscv/20260407171052241tmZDFGusMP_wlEsBVVtJo@zte.com.cn/
Wang Yechao (2):
RISC-V: KVM: add tracepoint for guest page faults
RISC-V: KVM: Recover gstage huge page mappings during
disable-dirty-log
arch/riscv/include/asm/kvm_gstage.h | 4 +
arch/riscv/kvm/gstage.c | 67 +++++++++++++++
arch/riscv/kvm/mmu.c | 123 +++++++++++++++++++++++++++-
arch/riscv/kvm/trace.h | 22 +++++
arch/riscv/kvm/vcpu_exit.c | 2 +
5 files changed, 217 insertions(+), 1 deletion(-)
--
2.27.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v5 0/2] RISC-V: KVM: Huge page recovery during disable-dirty-log
@ 2026-09-14 5:59 ` Wang Yechao
0 siblings, 0 replies; 10+ messages in thread
From: Wang Yechao @ 2026-09-14 5:59 UTC (permalink / raw)
To: Anup Patel, kvm, kvm-riscv, linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra,
Alexandre Ghiti, Wang Yechao
When dirty logging is enabled, the gstage mappings are split into
4K pages to track dirty pages. If the migration fails or is canceled,
in order to keep the VM's performance consistent with that before
dirty logging was enabled, we need to recover the gstage huge page mappings
when dirty logging is disabled.
With this series, dirty_log_perf_test shows a decrease in the number of
vCPU faults:
$ perf stat -e kvm:kvm_page_fault \
/dirty_log_perf_test -s anonymous_hugetlb_1gb -v 1 -e -b 1G
Before: 1,574,460 kvm:kvm_page_fault
After : 788,048 kvm:kvm_page_fault
Changes in v5:
Fix the issues reported by sashiko
- Use _PAGE_PFN_SHIFT instead of pfn_pte to make huge pte.
- Validate host size via hugetlb_mapping_size() before recovery.
- Skip bad advancement after mmu_recover_huge_pages_range.
Changes in v4:
- Rebased on Linux v7.3-rc2.
- Squash Patch01 into Patch03.
- The 1G mapping recovery is performed only after all 2M leaf recover succeed.
https://lore.kernel.org/linux-riscv/20260913070352.3626956-1-wang.yechao255@zte.com.cn
Changes in v3:
- Rebased on https://github.com/kvm-riscv/linux/tree/riscv_kvm_queue
- Add support to recover 1GB mappings for gstage.
- Refactor kvm_arch_commit_memory_region to be identical to arm64.
(sashiko AI Reviews)
- Scan the child page table for the first valid leaf PTE to recover.
- Add the memslot boundaries check when recover hugepages.
- Move the cond_resched_lock to the start loop iteration.
- The kvm_page_fault tracepoint be placed before the conditional block.
- Remove the fault_supports_gstage_huge_mapping check.
https://lore.kernel.org/linux-riscv/20260605160019772GJy5SDmWQQF7sSqvJ0JtC@zte.com.cn/
Changes in v2:
- Rebased on Linux v7.1-rc1.
- Skip recovering huge pte if the first child pte is not leaf.
- Link to v2:
https://lore.kernel.org/linux-riscv/20260429173002565bdp1o7P0bvOs6yTz64yNx@zte.com.cn/
- Link to v1:
https://lore.kernel.org/linux-riscv/20260407171052241tmZDFGusMP_wlEsBVVtJo@zte.com.cn/
Wang Yechao (2):
RISC-V: KVM: add tracepoint for guest page faults
RISC-V: KVM: Recover gstage huge page mappings during
disable-dirty-log
arch/riscv/include/asm/kvm_gstage.h | 4 +
arch/riscv/kvm/gstage.c | 67 +++++++++++++++
arch/riscv/kvm/mmu.c | 123 +++++++++++++++++++++++++++-
arch/riscv/kvm/trace.h | 22 +++++
arch/riscv/kvm/vcpu_exit.c | 2 +
5 files changed, 217 insertions(+), 1 deletion(-)
--
2.27.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v5 1/2] RISC-V: KVM: add tracepoint for guest page faults
2026-09-14 5:59 ` Wang Yechao
(?)
@ 2026-09-14 5:59 ` Wang Yechao
-1 siblings, 0 replies; 10+ messages in thread
From: Wang Yechao @ 2026-09-14 5:59 UTC (permalink / raw)
To: Anup Patel, kvm, kvm-riscv, linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra,
Alexandre Ghiti, Wang Yechao
Add the kvm_page_fault event tracepoint to count the number of KVM
guest page faults.
Signed-off-by: Wang Yechao <wang.yechao255@zte.com.cn>
---
arch/riscv/kvm/trace.h | 22 ++++++++++++++++++++++
arch/riscv/kvm/vcpu_exit.c | 2 ++
2 files changed, 24 insertions(+)
diff --git a/arch/riscv/kvm/trace.h b/arch/riscv/kvm/trace.h
index 23a5fd1b686e..07fcf5b6059b 100644
--- a/arch/riscv/kvm/trace.h
+++ b/arch/riscv/kvm/trace.h
@@ -133,6 +133,28 @@ TRACE_EVENT(kvm_vcpu_irq,
__entry->vcpu_id, __entry->irq, __entry->level)
);
+TRACE_EVENT(kvm_page_fault,
+ TP_PROTO(struct kvm_vcpu *vcpu, u64 fault_address, u64 error_code),
+ TP_ARGS(vcpu, fault_address, error_code),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, vcpu_id)
+ __field(u64, fault_address)
+ __field(u64, error_code)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu->vcpu_id;
+ __entry->fault_address = fault_address;
+ __entry->error_code = error_code;
+ ),
+
+ TP_printk("vcpu %u address 0x%016llx error_code 0x%llx",
+ __entry->vcpu_id,
+ __entry->fault_address,
+ __entry->error_code)
+);
+
#endif /* _TRACE_KVM_H */
#undef TRACE_INCLUDE_PATH
diff --git a/arch/riscv/kvm/vcpu_exit.c b/arch/riscv/kvm/vcpu_exit.c
index 88e0c369b354..de0262d0fd95 100644
--- a/arch/riscv/kvm/vcpu_exit.c
+++ b/arch/riscv/kvm/vcpu_exit.c
@@ -29,6 +29,8 @@ static int gstage_page_fault(struct kvm_vcpu *vcpu, struct kvm_run *run,
memslot = gfn_to_memslot(vcpu->kvm, gfn);
hva = gfn_to_hva_memslot_prot(memslot, gfn, &writable);
+ trace_kvm_page_fault(vcpu, fault_addr, trap->scause);
+
if (kvm_is_error_hva(hva) ||
(trap->scause == EXC_STORE_GUEST_PAGE_FAULT && !writable)) {
switch (trap->scause) {
--
2.27.0
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v5 1/2] RISC-V: KVM: add tracepoint for guest page faults
@ 2026-09-14 5:59 ` Wang Yechao
0 siblings, 0 replies; 10+ messages in thread
From: Wang Yechao @ 2026-09-14 5:59 UTC (permalink / raw)
To: Anup Patel, kvm, kvm-riscv, linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra,
Alexandre Ghiti, Wang Yechao
Add the kvm_page_fault event tracepoint to count the number of KVM
guest page faults.
Signed-off-by: Wang Yechao <wang.yechao255@zte.com.cn>
---
arch/riscv/kvm/trace.h | 22 ++++++++++++++++++++++
arch/riscv/kvm/vcpu_exit.c | 2 ++
2 files changed, 24 insertions(+)
diff --git a/arch/riscv/kvm/trace.h b/arch/riscv/kvm/trace.h
index 23a5fd1b686e..07fcf5b6059b 100644
--- a/arch/riscv/kvm/trace.h
+++ b/arch/riscv/kvm/trace.h
@@ -133,6 +133,28 @@ TRACE_EVENT(kvm_vcpu_irq,
__entry->vcpu_id, __entry->irq, __entry->level)
);
+TRACE_EVENT(kvm_page_fault,
+ TP_PROTO(struct kvm_vcpu *vcpu, u64 fault_address, u64 error_code),
+ TP_ARGS(vcpu, fault_address, error_code),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, vcpu_id)
+ __field(u64, fault_address)
+ __field(u64, error_code)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu->vcpu_id;
+ __entry->fault_address = fault_address;
+ __entry->error_code = error_code;
+ ),
+
+ TP_printk("vcpu %u address 0x%016llx error_code 0x%llx",
+ __entry->vcpu_id,
+ __entry->fault_address,
+ __entry->error_code)
+);
+
#endif /* _TRACE_KVM_H */
#undef TRACE_INCLUDE_PATH
diff --git a/arch/riscv/kvm/vcpu_exit.c b/arch/riscv/kvm/vcpu_exit.c
index 88e0c369b354..de0262d0fd95 100644
--- a/arch/riscv/kvm/vcpu_exit.c
+++ b/arch/riscv/kvm/vcpu_exit.c
@@ -29,6 +29,8 @@ static int gstage_page_fault(struct kvm_vcpu *vcpu, struct kvm_run *run,
memslot = gfn_to_memslot(vcpu->kvm, gfn);
hva = gfn_to_hva_memslot_prot(memslot, gfn, &writable);
+ trace_kvm_page_fault(vcpu, fault_addr, trap->scause);
+
if (kvm_is_error_hva(hva) ||
(trap->scause == EXC_STORE_GUEST_PAGE_FAULT && !writable)) {
switch (trap->scause) {
--
2.27.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v5 1/2] RISC-V: KVM: add tracepoint for guest page faults
@ 2026-09-14 5:59 ` Wang Yechao
0 siblings, 0 replies; 10+ messages in thread
From: Wang Yechao @ 2026-09-14 5:59 UTC (permalink / raw)
To: Anup Patel, kvm, kvm-riscv, linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra,
Alexandre Ghiti, Wang Yechao
Add the kvm_page_fault event tracepoint to count the number of KVM
guest page faults.
Signed-off-by: Wang Yechao <wang.yechao255@zte.com.cn>
---
arch/riscv/kvm/trace.h | 22 ++++++++++++++++++++++
arch/riscv/kvm/vcpu_exit.c | 2 ++
2 files changed, 24 insertions(+)
diff --git a/arch/riscv/kvm/trace.h b/arch/riscv/kvm/trace.h
index 23a5fd1b686e..07fcf5b6059b 100644
--- a/arch/riscv/kvm/trace.h
+++ b/arch/riscv/kvm/trace.h
@@ -133,6 +133,28 @@ TRACE_EVENT(kvm_vcpu_irq,
__entry->vcpu_id, __entry->irq, __entry->level)
);
+TRACE_EVENT(kvm_page_fault,
+ TP_PROTO(struct kvm_vcpu *vcpu, u64 fault_address, u64 error_code),
+ TP_ARGS(vcpu, fault_address, error_code),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, vcpu_id)
+ __field(u64, fault_address)
+ __field(u64, error_code)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu->vcpu_id;
+ __entry->fault_address = fault_address;
+ __entry->error_code = error_code;
+ ),
+
+ TP_printk("vcpu %u address 0x%016llx error_code 0x%llx",
+ __entry->vcpu_id,
+ __entry->fault_address,
+ __entry->error_code)
+);
+
#endif /* _TRACE_KVM_H */
#undef TRACE_INCLUDE_PATH
diff --git a/arch/riscv/kvm/vcpu_exit.c b/arch/riscv/kvm/vcpu_exit.c
index 88e0c369b354..de0262d0fd95 100644
--- a/arch/riscv/kvm/vcpu_exit.c
+++ b/arch/riscv/kvm/vcpu_exit.c
@@ -29,6 +29,8 @@ static int gstage_page_fault(struct kvm_vcpu *vcpu, struct kvm_run *run,
memslot = gfn_to_memslot(vcpu->kvm, gfn);
hva = gfn_to_hva_memslot_prot(memslot, gfn, &writable);
+ trace_kvm_page_fault(vcpu, fault_addr, trap->scause);
+
if (kvm_is_error_hva(hva) ||
(trap->scause == EXC_STORE_GUEST_PAGE_FAULT && !writable)) {
switch (trap->scause) {
--
2.27.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v5 2/2] RISC-V: KVM: Recover gstage huge page mappings during disable-dirty-log
2026-09-14 5:59 ` Wang Yechao
(?)
@ 2026-09-14 5:59 ` Wang Yechao
-1 siblings, 0 replies; 10+ messages in thread
From: Wang Yechao @ 2026-09-14 5:59 UTC (permalink / raw)
To: Anup Patel, kvm, kvm-riscv, linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra,
Alexandre Ghiti, Wang Yechao
When dirty logging is enabled, the gstage mappings are split into
4K pages to track dirty pages. If the migration fails or is canceled,
in order to keep the VM's performance consistent with that before
dirty logging was enabled, the gstage huge page mappings are recoverd
when dirty logging is disabled.
With this patch, dirty_log_perf_test shows a decrease in the number of
vCPU faults:
$ perf stat -e kvm:kvm_page_fault \
./dirty_log_perf_test -s anonymous_hugetlb_1gb -v 1 -e -b 1G
Before: 1,574,460 kvm:kvm_page_fault
After : 788,048 kvm:kvm_page_fault
Signed-off-by: Wang Yechao <wang.yechao255@zte.com.cn>
---
arch/riscv/include/asm/kvm_gstage.h | 4 +
arch/riscv/kvm/gstage.c | 67 +++++++++++++++
arch/riscv/kvm/mmu.c | 123 +++++++++++++++++++++++++++-
3 files changed, 193 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/include/asm/kvm_gstage.h b/arch/riscv/include/asm/kvm_gstage.h
index aaf080ba1b77..6dd0a96b336f 100644
--- a/arch/riscv/include/asm/kvm_gstage.h
+++ b/arch/riscv/include/asm/kvm_gstage.h
@@ -68,6 +68,10 @@ bool kvm_riscv_gstage_split_huge(struct kvm_gstage *gstage,
struct kvm_mmu_memory_cache *pcache,
gpa_t addr, u32 target_level, bool flush);
+bool kvm_riscv_gstage_recover_huge(struct kvm_gstage *gstage, gpa_t addr,
+ unsigned long target_page_size,
+ unsigned long *page_size);
+
enum kvm_riscv_gstage_op {
GSTAGE_OP_NOP = 0, /* Nothing */
GSTAGE_OP_CLEAR, /* Clear/Unmap */
diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
index e5002cb9cbef..2bbe052b9780 100644
--- a/arch/riscv/kvm/gstage.c
+++ b/arch/riscv/kvm/gstage.c
@@ -369,6 +369,73 @@ bool kvm_riscv_gstage_split_huge(struct kvm_gstage *gstage,
return need_flush;
}
+static inline unsigned long make_huge_pte(unsigned long child_pte, int index,
+ unsigned long child_page_size)
+{
+ unsigned long huge_pte = child_pte;
+ unsigned long child_pfn_offset;
+
+ child_pfn_offset = index * (child_page_size / PAGE_SIZE);
+ huge_pte -= (child_pfn_offset << _PAGE_PFN_SHIFT);
+
+ return huge_pte;
+}
+
+bool kvm_riscv_gstage_recover_huge(struct kvm_gstage *gstage, gpa_t addr,
+ unsigned long target_page_size,
+ unsigned long *page_size)
+{
+ u32 current_level = gstage->pgd_levels - 1;
+ pte_t *next_ptep = (pte_t *)gstage->pgd;
+ u32 target_level, out_level;
+ pte_t *ptep, *child_ptep;
+ unsigned long huge_pte;
+ bool recovered = false;
+ int ret, i;
+
+ out_level = 0;
+ ret = gstage_page_size_to_level(gstage, target_page_size, &target_level);
+ if (ret)
+ goto out;
+
+ while (current_level >= target_level) {
+ ptep = (pte_t *)&next_ptep[gstage_pte_index(gstage, addr, current_level)];
+
+ out_level = current_level;
+ if (!pte_val(ptep_get(ptep)))
+ goto out;
+
+ /* The mapping is already a huge page mapping. */
+ if (gstage_pte_leaf(ptep)) {
+ recovered = true;
+ goto out;
+ }
+
+ next_ptep = (pte_t *)gstage_pte_page_vaddr(ptep_get(ptep));
+ current_level--;
+ }
+
+ for (i = 0; i < PTRS_PER_PTE; i++) {
+ child_ptep = (pte_t *)&next_ptep[i];
+ if (!gstage_pte_leaf(child_ptep))
+ continue;
+
+ huge_pte = make_huge_pte(pte_val(ptep_get(child_ptep)),
+ i, target_page_size / PTRS_PER_PTE);
+ set_pte(ptep, __pte(huge_pte));
+ gstage_tlb_flush(gstage, target_level, addr);
+ put_page(virt_to_page(next_ptep));
+ recovered = true;
+
+ break;
+ }
+
+out:
+ gstage_level_to_page_size(gstage, out_level, page_size);
+
+ return recovered;
+}
+
bool kvm_riscv_gstage_op_pte(struct kvm_gstage *gstage, gpa_t addr,
pte_t *ptep, u32 ptep_level, enum kvm_riscv_gstage_op op)
{
diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index 6035b5ec9503..cb1d9d275bc6 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -19,6 +19,8 @@
static bool __read_mostly eager_page_split = true;
module_param(eager_page_split, bool, 0644);
+static void mmu_recover_huge_pages(struct kvm *kvm, int slot);
+
static void mmu_wp_memory_region(struct kvm *kvm, int slot)
{
struct kvm_memslots *slots = kvm_memslots(kvm);
@@ -241,17 +243,44 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
const struct kvm_memory_slot *new,
enum kvm_mr_change change)
{
+ bool log_dirty_pages = new && new->flags & KVM_MEM_LOG_DIRTY_PAGES;
+
/*
* At this point memslot has been committed and dirty pages will be
* tracked while the memory slot is write protected.
*/
- if (change != KVM_MR_DELETE && new->flags & KVM_MEM_LOG_DIRTY_PAGES) {
+ if (log_dirty_pages) {
+ if (change == KVM_MR_DELETE)
+ return;
+
if (kvm_dirty_log_manual_protect_and_init_set(kvm))
return;
+
mmu_wp_memory_region(kvm, new->id);
if (READ_ONCE(eager_page_split))
mmu_split_memory_region(kvm, new->id);
+
+ } else {
+ /*
+ * Only when change == KVM_MR_FLAGS_ONLY, this branch handles the
+ * disable-dirty-log case. For other changes (KVM_MR_CREATE,
+ * KVM_MR_DELETE, KVM_MR_MOVE), there is no need to recover
+ * huge pages.
+ */
+ if (change != KVM_MR_FLAGS_ONLY)
+ return;
+
+ /*
+ * Recover huge page mappings in the slot now that dirty logging
+ * is disabled, i.e. now that KVM does not have to track guest
+ * writes at 4KiB granularity.
+ *
+ * Dirty logging might be disabled by userspace if an ongoing VM
+ * live migration is cancelled and the VM must continue running
+ * on the source.
+ */
+ mmu_recover_huge_pages(kvm, new->id);
}
}
@@ -810,3 +839,95 @@ void kvm_riscv_mmu_update_hgatp(struct kvm_vcpu *vcpu)
if (!kvm_riscv_gstage_vmid_bits())
kvm_riscv_local_hfence_gvma_all();
}
+
+static void mmu_recover_huge_pages_range(struct kvm_gstage *gstage,
+ unsigned long page_size,
+ gpa_t range_start,
+ gpa_t range_end)
+{
+ phys_addr_t start = range_start;
+ phys_addr_t end = range_end;
+ unsigned long out_sz = 0;
+ bool recovered = true;
+
+ /*
+ * Recover 2MB hugepages mapping within the range.
+ */
+ while (start < end) {
+ recovered &= kvm_riscv_gstage_recover_huge(gstage, start,
+ PMD_SIZE, &out_sz);
+ start += out_sz;
+ }
+
+ /*
+ * If 1GB hugepages are desired, try to recover the whole range
+ * as one 1GB hugepages mapping.
+ */
+ if (recovered && (page_size == PUD_SIZE)) {
+ start = range_start;
+ kvm_riscv_gstage_recover_huge(gstage, start, PUD_SIZE, &out_sz);
+ }
+}
+
+static void mmu_recover_huge_pages(struct kvm *kvm, int slot)
+{
+ struct kvm_memslots *slots = kvm_memslots(kvm);
+ struct kvm_memory_slot *memslot = id_to_memslot(slots, slot);
+ unsigned long hva = gfn_to_hva(kvm, memslot->base_gfn);
+ phys_addr_t start = memslot->base_gfn << PAGE_SHIFT;
+ phys_addr_t end = (memslot->base_gfn + memslot->npages) << PAGE_SHIFT;
+ phys_addr_t addr = start;
+ struct kvm_gstage gstage;
+ unsigned long page_size;
+ phys_addr_t range_start;
+ phys_addr_t range_end;
+
+ kvm_riscv_gstage_init(&gstage, kvm);
+
+ write_lock(&kvm->mmu_lock);
+
+ while (addr < end) {
+ /*
+ * If a very large memslot is mapped exclusively with
+ * 4KB host pages, or too many hugepages need to recover,
+ * release the kvm->mmu_lock to prevent starvation and
+ * lockup detector warnings.
+ */
+ cond_resched_rwlock_write(&kvm->mmu_lock);
+
+ if (!kvm->arch.pgd)
+ break;
+
+ hva = gfn_to_hva(kvm, addr >> PAGE_SHIFT);
+ page_size = get_hva_mapping_size(kvm, hva);
+ if (page_size == PAGE_SIZE) {
+ addr += page_size;
+ continue;
+ }
+
+ page_size = hugetlb_mapping_size(memslot, hva, page_size);
+ if (page_size == PAGE_SIZE) {
+ addr += page_size;
+ continue;
+ }
+
+ range_start = ALIGN_DOWN(addr, page_size);
+ range_end = range_start + page_size;
+
+ /*
+ * Make sure the recover range [range_start, range_end)
+ * is within the slot range.
+ */
+ if (range_start < start || range_end > end) {
+ addr = range_end;
+ continue;
+ }
+
+ mmu_recover_huge_pages_range(&gstage, page_size,
+ range_start, range_end);
+
+ addr = range_end;
+ }
+
+ write_unlock(&kvm->mmu_lock);
+}
--
2.27.0
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v5 2/2] RISC-V: KVM: Recover gstage huge page mappings during disable-dirty-log
@ 2026-09-14 5:59 ` Wang Yechao
0 siblings, 0 replies; 10+ messages in thread
From: Wang Yechao @ 2026-09-14 5:59 UTC (permalink / raw)
To: Anup Patel, kvm, kvm-riscv, linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra,
Alexandre Ghiti, Wang Yechao
When dirty logging is enabled, the gstage mappings are split into
4K pages to track dirty pages. If the migration fails or is canceled,
in order to keep the VM's performance consistent with that before
dirty logging was enabled, the gstage huge page mappings are recoverd
when dirty logging is disabled.
With this patch, dirty_log_perf_test shows a decrease in the number of
vCPU faults:
$ perf stat -e kvm:kvm_page_fault \
./dirty_log_perf_test -s anonymous_hugetlb_1gb -v 1 -e -b 1G
Before: 1,574,460 kvm:kvm_page_fault
After : 788,048 kvm:kvm_page_fault
Signed-off-by: Wang Yechao <wang.yechao255@zte.com.cn>
---
arch/riscv/include/asm/kvm_gstage.h | 4 +
arch/riscv/kvm/gstage.c | 67 +++++++++++++++
arch/riscv/kvm/mmu.c | 123 +++++++++++++++++++++++++++-
3 files changed, 193 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/include/asm/kvm_gstage.h b/arch/riscv/include/asm/kvm_gstage.h
index aaf080ba1b77..6dd0a96b336f 100644
--- a/arch/riscv/include/asm/kvm_gstage.h
+++ b/arch/riscv/include/asm/kvm_gstage.h
@@ -68,6 +68,10 @@ bool kvm_riscv_gstage_split_huge(struct kvm_gstage *gstage,
struct kvm_mmu_memory_cache *pcache,
gpa_t addr, u32 target_level, bool flush);
+bool kvm_riscv_gstage_recover_huge(struct kvm_gstage *gstage, gpa_t addr,
+ unsigned long target_page_size,
+ unsigned long *page_size);
+
enum kvm_riscv_gstage_op {
GSTAGE_OP_NOP = 0, /* Nothing */
GSTAGE_OP_CLEAR, /* Clear/Unmap */
diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
index e5002cb9cbef..2bbe052b9780 100644
--- a/arch/riscv/kvm/gstage.c
+++ b/arch/riscv/kvm/gstage.c
@@ -369,6 +369,73 @@ bool kvm_riscv_gstage_split_huge(struct kvm_gstage *gstage,
return need_flush;
}
+static inline unsigned long make_huge_pte(unsigned long child_pte, int index,
+ unsigned long child_page_size)
+{
+ unsigned long huge_pte = child_pte;
+ unsigned long child_pfn_offset;
+
+ child_pfn_offset = index * (child_page_size / PAGE_SIZE);
+ huge_pte -= (child_pfn_offset << _PAGE_PFN_SHIFT);
+
+ return huge_pte;
+}
+
+bool kvm_riscv_gstage_recover_huge(struct kvm_gstage *gstage, gpa_t addr,
+ unsigned long target_page_size,
+ unsigned long *page_size)
+{
+ u32 current_level = gstage->pgd_levels - 1;
+ pte_t *next_ptep = (pte_t *)gstage->pgd;
+ u32 target_level, out_level;
+ pte_t *ptep, *child_ptep;
+ unsigned long huge_pte;
+ bool recovered = false;
+ int ret, i;
+
+ out_level = 0;
+ ret = gstage_page_size_to_level(gstage, target_page_size, &target_level);
+ if (ret)
+ goto out;
+
+ while (current_level >= target_level) {
+ ptep = (pte_t *)&next_ptep[gstage_pte_index(gstage, addr, current_level)];
+
+ out_level = current_level;
+ if (!pte_val(ptep_get(ptep)))
+ goto out;
+
+ /* The mapping is already a huge page mapping. */
+ if (gstage_pte_leaf(ptep)) {
+ recovered = true;
+ goto out;
+ }
+
+ next_ptep = (pte_t *)gstage_pte_page_vaddr(ptep_get(ptep));
+ current_level--;
+ }
+
+ for (i = 0; i < PTRS_PER_PTE; i++) {
+ child_ptep = (pte_t *)&next_ptep[i];
+ if (!gstage_pte_leaf(child_ptep))
+ continue;
+
+ huge_pte = make_huge_pte(pte_val(ptep_get(child_ptep)),
+ i, target_page_size / PTRS_PER_PTE);
+ set_pte(ptep, __pte(huge_pte));
+ gstage_tlb_flush(gstage, target_level, addr);
+ put_page(virt_to_page(next_ptep));
+ recovered = true;
+
+ break;
+ }
+
+out:
+ gstage_level_to_page_size(gstage, out_level, page_size);
+
+ return recovered;
+}
+
bool kvm_riscv_gstage_op_pte(struct kvm_gstage *gstage, gpa_t addr,
pte_t *ptep, u32 ptep_level, enum kvm_riscv_gstage_op op)
{
diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index 6035b5ec9503..cb1d9d275bc6 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -19,6 +19,8 @@
static bool __read_mostly eager_page_split = true;
module_param(eager_page_split, bool, 0644);
+static void mmu_recover_huge_pages(struct kvm *kvm, int slot);
+
static void mmu_wp_memory_region(struct kvm *kvm, int slot)
{
struct kvm_memslots *slots = kvm_memslots(kvm);
@@ -241,17 +243,44 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
const struct kvm_memory_slot *new,
enum kvm_mr_change change)
{
+ bool log_dirty_pages = new && new->flags & KVM_MEM_LOG_DIRTY_PAGES;
+
/*
* At this point memslot has been committed and dirty pages will be
* tracked while the memory slot is write protected.
*/
- if (change != KVM_MR_DELETE && new->flags & KVM_MEM_LOG_DIRTY_PAGES) {
+ if (log_dirty_pages) {
+ if (change == KVM_MR_DELETE)
+ return;
+
if (kvm_dirty_log_manual_protect_and_init_set(kvm))
return;
+
mmu_wp_memory_region(kvm, new->id);
if (READ_ONCE(eager_page_split))
mmu_split_memory_region(kvm, new->id);
+
+ } else {
+ /*
+ * Only when change == KVM_MR_FLAGS_ONLY, this branch handles the
+ * disable-dirty-log case. For other changes (KVM_MR_CREATE,
+ * KVM_MR_DELETE, KVM_MR_MOVE), there is no need to recover
+ * huge pages.
+ */
+ if (change != KVM_MR_FLAGS_ONLY)
+ return;
+
+ /*
+ * Recover huge page mappings in the slot now that dirty logging
+ * is disabled, i.e. now that KVM does not have to track guest
+ * writes at 4KiB granularity.
+ *
+ * Dirty logging might be disabled by userspace if an ongoing VM
+ * live migration is cancelled and the VM must continue running
+ * on the source.
+ */
+ mmu_recover_huge_pages(kvm, new->id);
}
}
@@ -810,3 +839,95 @@ void kvm_riscv_mmu_update_hgatp(struct kvm_vcpu *vcpu)
if (!kvm_riscv_gstage_vmid_bits())
kvm_riscv_local_hfence_gvma_all();
}
+
+static void mmu_recover_huge_pages_range(struct kvm_gstage *gstage,
+ unsigned long page_size,
+ gpa_t range_start,
+ gpa_t range_end)
+{
+ phys_addr_t start = range_start;
+ phys_addr_t end = range_end;
+ unsigned long out_sz = 0;
+ bool recovered = true;
+
+ /*
+ * Recover 2MB hugepages mapping within the range.
+ */
+ while (start < end) {
+ recovered &= kvm_riscv_gstage_recover_huge(gstage, start,
+ PMD_SIZE, &out_sz);
+ start += out_sz;
+ }
+
+ /*
+ * If 1GB hugepages are desired, try to recover the whole range
+ * as one 1GB hugepages mapping.
+ */
+ if (recovered && (page_size == PUD_SIZE)) {
+ start = range_start;
+ kvm_riscv_gstage_recover_huge(gstage, start, PUD_SIZE, &out_sz);
+ }
+}
+
+static void mmu_recover_huge_pages(struct kvm *kvm, int slot)
+{
+ struct kvm_memslots *slots = kvm_memslots(kvm);
+ struct kvm_memory_slot *memslot = id_to_memslot(slots, slot);
+ unsigned long hva = gfn_to_hva(kvm, memslot->base_gfn);
+ phys_addr_t start = memslot->base_gfn << PAGE_SHIFT;
+ phys_addr_t end = (memslot->base_gfn + memslot->npages) << PAGE_SHIFT;
+ phys_addr_t addr = start;
+ struct kvm_gstage gstage;
+ unsigned long page_size;
+ phys_addr_t range_start;
+ phys_addr_t range_end;
+
+ kvm_riscv_gstage_init(&gstage, kvm);
+
+ write_lock(&kvm->mmu_lock);
+
+ while (addr < end) {
+ /*
+ * If a very large memslot is mapped exclusively with
+ * 4KB host pages, or too many hugepages need to recover,
+ * release the kvm->mmu_lock to prevent starvation and
+ * lockup detector warnings.
+ */
+ cond_resched_rwlock_write(&kvm->mmu_lock);
+
+ if (!kvm->arch.pgd)
+ break;
+
+ hva = gfn_to_hva(kvm, addr >> PAGE_SHIFT);
+ page_size = get_hva_mapping_size(kvm, hva);
+ if (page_size == PAGE_SIZE) {
+ addr += page_size;
+ continue;
+ }
+
+ page_size = hugetlb_mapping_size(memslot, hva, page_size);
+ if (page_size == PAGE_SIZE) {
+ addr += page_size;
+ continue;
+ }
+
+ range_start = ALIGN_DOWN(addr, page_size);
+ range_end = range_start + page_size;
+
+ /*
+ * Make sure the recover range [range_start, range_end)
+ * is within the slot range.
+ */
+ if (range_start < start || range_end > end) {
+ addr = range_end;
+ continue;
+ }
+
+ mmu_recover_huge_pages_range(&gstage, page_size,
+ range_start, range_end);
+
+ addr = range_end;
+ }
+
+ write_unlock(&kvm->mmu_lock);
+}
--
2.27.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v5 2/2] RISC-V: KVM: Recover gstage huge page mappings during disable-dirty-log
@ 2026-09-14 5:59 ` Wang Yechao
0 siblings, 0 replies; 10+ messages in thread
From: Wang Yechao @ 2026-09-14 5:59 UTC (permalink / raw)
To: Anup Patel, kvm, kvm-riscv, linux-riscv, linux-kernel
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra,
Alexandre Ghiti, Wang Yechao
When dirty logging is enabled, the gstage mappings are split into
4K pages to track dirty pages. If the migration fails or is canceled,
in order to keep the VM's performance consistent with that before
dirty logging was enabled, the gstage huge page mappings are recoverd
when dirty logging is disabled.
With this patch, dirty_log_perf_test shows a decrease in the number of
vCPU faults:
$ perf stat -e kvm:kvm_page_fault \
./dirty_log_perf_test -s anonymous_hugetlb_1gb -v 1 -e -b 1G
Before: 1,574,460 kvm:kvm_page_fault
After : 788,048 kvm:kvm_page_fault
Signed-off-by: Wang Yechao <wang.yechao255@zte.com.cn>
---
arch/riscv/include/asm/kvm_gstage.h | 4 +
arch/riscv/kvm/gstage.c | 67 +++++++++++++++
arch/riscv/kvm/mmu.c | 123 +++++++++++++++++++++++++++-
3 files changed, 193 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/include/asm/kvm_gstage.h b/arch/riscv/include/asm/kvm_gstage.h
index aaf080ba1b77..6dd0a96b336f 100644
--- a/arch/riscv/include/asm/kvm_gstage.h
+++ b/arch/riscv/include/asm/kvm_gstage.h
@@ -68,6 +68,10 @@ bool kvm_riscv_gstage_split_huge(struct kvm_gstage *gstage,
struct kvm_mmu_memory_cache *pcache,
gpa_t addr, u32 target_level, bool flush);
+bool kvm_riscv_gstage_recover_huge(struct kvm_gstage *gstage, gpa_t addr,
+ unsigned long target_page_size,
+ unsigned long *page_size);
+
enum kvm_riscv_gstage_op {
GSTAGE_OP_NOP = 0, /* Nothing */
GSTAGE_OP_CLEAR, /* Clear/Unmap */
diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
index e5002cb9cbef..2bbe052b9780 100644
--- a/arch/riscv/kvm/gstage.c
+++ b/arch/riscv/kvm/gstage.c
@@ -369,6 +369,73 @@ bool kvm_riscv_gstage_split_huge(struct kvm_gstage *gstage,
return need_flush;
}
+static inline unsigned long make_huge_pte(unsigned long child_pte, int index,
+ unsigned long child_page_size)
+{
+ unsigned long huge_pte = child_pte;
+ unsigned long child_pfn_offset;
+
+ child_pfn_offset = index * (child_page_size / PAGE_SIZE);
+ huge_pte -= (child_pfn_offset << _PAGE_PFN_SHIFT);
+
+ return huge_pte;
+}
+
+bool kvm_riscv_gstage_recover_huge(struct kvm_gstage *gstage, gpa_t addr,
+ unsigned long target_page_size,
+ unsigned long *page_size)
+{
+ u32 current_level = gstage->pgd_levels - 1;
+ pte_t *next_ptep = (pte_t *)gstage->pgd;
+ u32 target_level, out_level;
+ pte_t *ptep, *child_ptep;
+ unsigned long huge_pte;
+ bool recovered = false;
+ int ret, i;
+
+ out_level = 0;
+ ret = gstage_page_size_to_level(gstage, target_page_size, &target_level);
+ if (ret)
+ goto out;
+
+ while (current_level >= target_level) {
+ ptep = (pte_t *)&next_ptep[gstage_pte_index(gstage, addr, current_level)];
+
+ out_level = current_level;
+ if (!pte_val(ptep_get(ptep)))
+ goto out;
+
+ /* The mapping is already a huge page mapping. */
+ if (gstage_pte_leaf(ptep)) {
+ recovered = true;
+ goto out;
+ }
+
+ next_ptep = (pte_t *)gstage_pte_page_vaddr(ptep_get(ptep));
+ current_level--;
+ }
+
+ for (i = 0; i < PTRS_PER_PTE; i++) {
+ child_ptep = (pte_t *)&next_ptep[i];
+ if (!gstage_pte_leaf(child_ptep))
+ continue;
+
+ huge_pte = make_huge_pte(pte_val(ptep_get(child_ptep)),
+ i, target_page_size / PTRS_PER_PTE);
+ set_pte(ptep, __pte(huge_pte));
+ gstage_tlb_flush(gstage, target_level, addr);
+ put_page(virt_to_page(next_ptep));
+ recovered = true;
+
+ break;
+ }
+
+out:
+ gstage_level_to_page_size(gstage, out_level, page_size);
+
+ return recovered;
+}
+
bool kvm_riscv_gstage_op_pte(struct kvm_gstage *gstage, gpa_t addr,
pte_t *ptep, u32 ptep_level, enum kvm_riscv_gstage_op op)
{
diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index 6035b5ec9503..cb1d9d275bc6 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -19,6 +19,8 @@
static bool __read_mostly eager_page_split = true;
module_param(eager_page_split, bool, 0644);
+static void mmu_recover_huge_pages(struct kvm *kvm, int slot);
+
static void mmu_wp_memory_region(struct kvm *kvm, int slot)
{
struct kvm_memslots *slots = kvm_memslots(kvm);
@@ -241,17 +243,44 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
const struct kvm_memory_slot *new,
enum kvm_mr_change change)
{
+ bool log_dirty_pages = new && new->flags & KVM_MEM_LOG_DIRTY_PAGES;
+
/*
* At this point memslot has been committed and dirty pages will be
* tracked while the memory slot is write protected.
*/
- if (change != KVM_MR_DELETE && new->flags & KVM_MEM_LOG_DIRTY_PAGES) {
+ if (log_dirty_pages) {
+ if (change == KVM_MR_DELETE)
+ return;
+
if (kvm_dirty_log_manual_protect_and_init_set(kvm))
return;
+
mmu_wp_memory_region(kvm, new->id);
if (READ_ONCE(eager_page_split))
mmu_split_memory_region(kvm, new->id);
+
+ } else {
+ /*
+ * Only when change == KVM_MR_FLAGS_ONLY, this branch handles the
+ * disable-dirty-log case. For other changes (KVM_MR_CREATE,
+ * KVM_MR_DELETE, KVM_MR_MOVE), there is no need to recover
+ * huge pages.
+ */
+ if (change != KVM_MR_FLAGS_ONLY)
+ return;
+
+ /*
+ * Recover huge page mappings in the slot now that dirty logging
+ * is disabled, i.e. now that KVM does not have to track guest
+ * writes at 4KiB granularity.
+ *
+ * Dirty logging might be disabled by userspace if an ongoing VM
+ * live migration is cancelled and the VM must continue running
+ * on the source.
+ */
+ mmu_recover_huge_pages(kvm, new->id);
}
}
@@ -810,3 +839,95 @@ void kvm_riscv_mmu_update_hgatp(struct kvm_vcpu *vcpu)
if (!kvm_riscv_gstage_vmid_bits())
kvm_riscv_local_hfence_gvma_all();
}
+
+static void mmu_recover_huge_pages_range(struct kvm_gstage *gstage,
+ unsigned long page_size,
+ gpa_t range_start,
+ gpa_t range_end)
+{
+ phys_addr_t start = range_start;
+ phys_addr_t end = range_end;
+ unsigned long out_sz = 0;
+ bool recovered = true;
+
+ /*
+ * Recover 2MB hugepages mapping within the range.
+ */
+ while (start < end) {
+ recovered &= kvm_riscv_gstage_recover_huge(gstage, start,
+ PMD_SIZE, &out_sz);
+ start += out_sz;
+ }
+
+ /*
+ * If 1GB hugepages are desired, try to recover the whole range
+ * as one 1GB hugepages mapping.
+ */
+ if (recovered && (page_size == PUD_SIZE)) {
+ start = range_start;
+ kvm_riscv_gstage_recover_huge(gstage, start, PUD_SIZE, &out_sz);
+ }
+}
+
+static void mmu_recover_huge_pages(struct kvm *kvm, int slot)
+{
+ struct kvm_memslots *slots = kvm_memslots(kvm);
+ struct kvm_memory_slot *memslot = id_to_memslot(slots, slot);
+ unsigned long hva = gfn_to_hva(kvm, memslot->base_gfn);
+ phys_addr_t start = memslot->base_gfn << PAGE_SHIFT;
+ phys_addr_t end = (memslot->base_gfn + memslot->npages) << PAGE_SHIFT;
+ phys_addr_t addr = start;
+ struct kvm_gstage gstage;
+ unsigned long page_size;
+ phys_addr_t range_start;
+ phys_addr_t range_end;
+
+ kvm_riscv_gstage_init(&gstage, kvm);
+
+ write_lock(&kvm->mmu_lock);
+
+ while (addr < end) {
+ /*
+ * If a very large memslot is mapped exclusively with
+ * 4KB host pages, or too many hugepages need to recover,
+ * release the kvm->mmu_lock to prevent starvation and
+ * lockup detector warnings.
+ */
+ cond_resched_rwlock_write(&kvm->mmu_lock);
+
+ if (!kvm->arch.pgd)
+ break;
+
+ hva = gfn_to_hva(kvm, addr >> PAGE_SHIFT);
+ page_size = get_hva_mapping_size(kvm, hva);
+ if (page_size == PAGE_SIZE) {
+ addr += page_size;
+ continue;
+ }
+
+ page_size = hugetlb_mapping_size(memslot, hva, page_size);
+ if (page_size == PAGE_SIZE) {
+ addr += page_size;
+ continue;
+ }
+
+ range_start = ALIGN_DOWN(addr, page_size);
+ range_end = range_start + page_size;
+
+ /*
+ * Make sure the recover range [range_start, range_end)
+ * is within the slot range.
+ */
+ if (range_start < start || range_end > end) {
+ addr = range_end;
+ continue;
+ }
+
+ mmu_recover_huge_pages_range(&gstage, page_size,
+ range_start, range_end);
+
+ addr = range_end;
+ }
+
+ write_unlock(&kvm->mmu_lock);
+}
--
2.27.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v5 2/2] RISC-V: KVM: Recover gstage huge page mappings during disable-dirty-log
2026-09-14 5:59 ` Wang Yechao
(?)
(?)
@ 2026-09-14 6:22 ` sashiko-bot
-1 siblings, 0 replies; 10+ messages in thread
From: sashiko-bot @ 2026-09-14 6:22 UTC (permalink / raw)
To: Wang Yechao; +Cc: kvm
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Reconstructing huge pages leaves stale 4K TLB entries, which can cause infinite page fault loops.
--
commit adebe96417592e1fb6f444f66872994bafd575bb
Author: Wang Yechao <wang.yechao255@zte.com.cn>
RISC-V: KVM: Recover gstage huge page mappings during disable-dirty-log
This commit recovers gstage huge page mappings when dirty logging is disabled.
It is intended to restore vm performance to pre-dirty-logging levels if a
migration fails or is canceled.
> diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
> index e5002cb9cbef1..2bbe052b97806 100644
> --- a/arch/riscv/kvm/gstage.c
> +++ b/arch/riscv/kvm/gstage.c
[ ... ]
> +bool kvm_riscv_gstage_recover_huge(struct kvm_gstage *gstage, gpa_t addr,
> + unsigned long target_page_size,
> + unsigned long *page_size)
> +{
[ ... ]
> + for (i = 0; i < PTRS_PER_PTE; i++) {
> + child_ptep = (pte_t *)&next_ptep[i];
> + if (!gstage_pte_leaf(child_ptep))
> + continue;
> +
> + huge_pte = make_huge_pte(pte_val(ptep_get(child_ptep)),
> + i, target_page_size / PTRS_PER_PTE);
> + set_pte(ptep, __pte(huge_pte));
> + gstage_tlb_flush(gstage, target_level, addr);
[Severity: High]
Does this tlb flush leave stale 4KB entries in the hardware tlb?
When dirty logging is enabled, a 2MB huge page is split into 4KB read-only
pages. If dirty logging is then disabled and kvm reconstructs the 2MB huge
page by replacing the non-leaf pte with a 2MB leaf pte, this code calls
gstage_tlb_flush() for the target level.
This passes the huge page order to kvm_riscv_local_hfence_gvma_vmid_gpa(),
which executes a single hfence.gvma instruction for the 2MB-aligned base
address. Because hfence.gvma with a specific address only invalidates
translations matching exactly that address, could the 4KB tlb entries for
sub-pages (like addr + 4KB) remain cached in the hardware tlb?
If the guest attempts to write to one of these stale 4KB pages, it triggers
a guest store page fault. The kvm page fault handler checks the gstage page
tables, observes the new 2MB pte is already writable, issues no tlb flush,
and returns to the guest. Would the guest then retry the write instruction,
hit the same stale read-only 4KB tlb entry, and fault again, creating an
infinite loop?
> + put_page(virt_to_page(next_ptep));
> + recovered = true;
> +
> + break;
> + }
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260914055936.3672758-1-wang.yechao255@zte.com.cn?part=2
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2026-09-14 6:22 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-14 5:59 [PATCH v5 0/2] RISC-V: KVM: Huge page recovery during disable-dirty-log Wang Yechao
2026-09-14 5:59 ` Wang Yechao
2026-09-14 5:59 ` Wang Yechao
2026-09-14 5:59 ` [PATCH v5 1/2] RISC-V: KVM: add tracepoint for guest page faults Wang Yechao
2026-09-14 5:59 ` Wang Yechao
2026-09-14 5:59 ` Wang Yechao
2026-09-14 5:59 ` [PATCH v5 2/2] RISC-V: KVM: Recover gstage huge page mappings during disable-dirty-log Wang Yechao
2026-09-14 5:59 ` Wang Yechao
2026-09-14 5:59 ` Wang Yechao
2026-09-14 6:22 ` sashiko-bot
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.