* [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
@ 2026-06-16 6:17 ` Xie Bo
0 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-06-16 6:17 UTC (permalink / raw)
To: Anup Patel, kvm-riscv
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm, linux-riscv,
linux-kernel, stable, Xie Bo
RISC-V KVM tracks guest pending interrupts with irqs_pending and
irqs_pending_mask. The existing code uses atomic bitops and models the
state as a multiple-producer, single-consumer queue where the vCPU is
the consumer.
The atomic bitops make each individual bitmap operation atomic, but
they do not make the pair of irqs_pending and irqs_pending_mask an
atomic state transition. The vCPU interrupt sync path is also not a
pure consumer: it writes both bitmaps when it reflects guest-visible
HVIP changes back into KVM state.
For example, kvm_riscv_vcpu_set_interrupt() can set IRQ_VS_SOFT in
irqs_pending while the target vCPU is concurrently syncing a
guest-cleared HVIP.VSSIP bit. If sync observes irqs_pending_mask before
the producer sets it, sync can set the mask and clear irqs_pending. The
producer then sets the mask and kicks the vCPU, but the pending bit has
already been lost. A subsequent flush updates HVIP without VSSIP, and
the guest can remain blocked in WFI even though it has work queued.
Serialize all updates to irqs_pending and irqs_pending_mask with a
per-vCPU raw spinlock. This intentionally replaces the lockless
pending/mask protocol with one small critical section per vCPU, so the
pending bit and the dirty mask are updated as one state transition.
Use the same lock for sync, flush, reset, and userspace CSR writes that
clear the dirty mask, so a newly injected interrupt cannot be
overwritten by a concurrent HVIP sync.
Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
Cc: stable@vger.kernel.org
Signed-off-by: Xie Bo <xb@ultrarisc.com>
---
arch/riscv/include/asm/kvm_host.h | 10 +++++-----
arch/riscv/kvm/aia.c | 28 +++++++++++++++++++++-------
arch/riscv/kvm/vcpu.c | 27 ++++++++++++++++++++++-----
arch/riscv/kvm/vcpu_onereg.c | 13 +++++++++----
4 files changed, 57 insertions(+), 21 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index 75b0a951c..97e42645c 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -207,13 +207,13 @@ struct kvm_vcpu_arch {
/*
* VCPU interrupts
*
- * We have a lockless approach for tracking pending VCPU interrupts
- * implemented using atomic bitops. The irqs_pending bitmap represent
- * pending interrupts whereas irqs_pending_mask represent bits changed
- * in irqs_pending. Our approach is modeled around multiple producer
- * and single consumer problem where the consumer is the VCPU itself.
+ * The irqs_pending bitmap represents pending interrupts whereas
+ * irqs_pending_mask represents bits changed in irqs_pending. Updates
+ * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
+ * drop a newly injected interrupt while syncing guest-visible HVIP.
*/
#define KVM_RISCV_VCPU_NR_IRQS 64
+ raw_spinlock_t irqs_pending_lock;
DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index 5ec503288..821d2cb6d 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -50,17 +50,21 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long mask, val;
+ unsigned long flags;
if (!kvm_riscv_aia_available())
return;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ mask = vcpu->arch.irqs_pending_mask[1];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ val = vcpu->arch.irqs_pending[1] & mask;
csr->hviph &= ~mask;
csr->hviph |= val;
}
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
}
void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
@@ -205,6 +209,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
+#ifdef CONFIG_32BIT
+ unsigned long flags;
+#endif
if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
return -ENOENT;
@@ -214,11 +221,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
reg_num = array_index_nospec(reg_num, regs_max);
if (kvm_riscv_aia_available()) {
- ((unsigned long *)csr)[reg_num] = val;
-
#ifdef CONFIG_32BIT
- if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = val;
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
+ flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = val;
+ }
+#else
+ ((unsigned long *)csr)[reg_num] = val;
#endif
}
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index a73690eda..b04730ccd 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
{
+ unsigned long flags;
bool loaded;
/**
@@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
kvm_riscv_vcpu_aia_reset(vcpu);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_riscv_vcpu_pmu_reset(vcpu);
@@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
/* Setup VCPU hfence queue */
spin_lock_init(&vcpu->arch.hfence_lock);
+ raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
spin_lock_init(&vcpu->arch.reset_state.lock);
@@ -352,14 +356,18 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long mask, val;
+ unsigned long flags;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ mask = vcpu->arch.irqs_pending_mask[0];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ val = vcpu->arch.irqs_pending[0] & mask;
csr->hvip &= ~mask;
csr->hvip |= val;
}
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
/* Flush AIA high interrupts */
kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
@@ -368,6 +376,7 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
{
unsigned long hvip;
+ unsigned long flags;
struct kvm_vcpu_arch *v = &vcpu->arch;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
@@ -376,6 +385,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
/* Sync-up HVIP.VSSIP bit changes does by Guest */
hvip = ncsr_read(CSR_HVIP);
+ raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
if (hvip & (1UL << IRQ_VS_SOFT)) {
if (!test_and_set_bit(IRQ_VS_SOFT,
@@ -394,6 +404,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
!test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
clear_bit(IRQ_PMU_OVF, v->irqs_pending);
}
+ raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
/* Sync-up AIA high interrupts */
kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
@@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, and external
* interrupts when irq is one of the local interrupts
@@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
set_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_vcpu_kick(vcpu);
@@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, counter overflow and external
* interrupts when irq is one of the local interrupts
@@ -439,9 +455,10 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
clear_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
return 0;
}
diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
index bb920e892..cba368294 100644
--- a/arch/riscv/kvm/vcpu_onereg.c
+++ b/arch/riscv/kvm/vcpu_onereg.c
@@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
+ unsigned long flags;
if (reg_num >= regs_max)
return -ENOENT;
@@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
reg_val <<= VSIP_TO_HVIP_SHIFT;
}
- ((unsigned long *)csr)[reg_num] = reg_val;
-
- if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ }
return 0;
}
base-commit: 481329ec5b31d2c48ac6b8b703c8008133884c7e
--
2.54.0
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
@ 2026-06-16 6:17 ` Xie Bo
0 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-06-16 6:17 UTC (permalink / raw)
To: Anup Patel, kvm-riscv
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm, linux-riscv,
linux-kernel, stable, Xie Bo
RISC-V KVM tracks guest pending interrupts with irqs_pending and
irqs_pending_mask. The existing code uses atomic bitops and models the
state as a multiple-producer, single-consumer queue where the vCPU is
the consumer.
The atomic bitops make each individual bitmap operation atomic, but
they do not make the pair of irqs_pending and irqs_pending_mask an
atomic state transition. The vCPU interrupt sync path is also not a
pure consumer: it writes both bitmaps when it reflects guest-visible
HVIP changes back into KVM state.
For example, kvm_riscv_vcpu_set_interrupt() can set IRQ_VS_SOFT in
irqs_pending while the target vCPU is concurrently syncing a
guest-cleared HVIP.VSSIP bit. If sync observes irqs_pending_mask before
the producer sets it, sync can set the mask and clear irqs_pending. The
producer then sets the mask and kicks the vCPU, but the pending bit has
already been lost. A subsequent flush updates HVIP without VSSIP, and
the guest can remain blocked in WFI even though it has work queued.
Serialize all updates to irqs_pending and irqs_pending_mask with a
per-vCPU raw spinlock. This intentionally replaces the lockless
pending/mask protocol with one small critical section per vCPU, so the
pending bit and the dirty mask are updated as one state transition.
Use the same lock for sync, flush, reset, and userspace CSR writes that
clear the dirty mask, so a newly injected interrupt cannot be
overwritten by a concurrent HVIP sync.
Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
Cc: stable@vger.kernel.org
Signed-off-by: Xie Bo <xb@ultrarisc.com>
---
arch/riscv/include/asm/kvm_host.h | 10 +++++-----
arch/riscv/kvm/aia.c | 28 +++++++++++++++++++++-------
arch/riscv/kvm/vcpu.c | 27 ++++++++++++++++++++++-----
arch/riscv/kvm/vcpu_onereg.c | 13 +++++++++----
4 files changed, 57 insertions(+), 21 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index 75b0a951c..97e42645c 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -207,13 +207,13 @@ struct kvm_vcpu_arch {
/*
* VCPU interrupts
*
- * We have a lockless approach for tracking pending VCPU interrupts
- * implemented using atomic bitops. The irqs_pending bitmap represent
- * pending interrupts whereas irqs_pending_mask represent bits changed
- * in irqs_pending. Our approach is modeled around multiple producer
- * and single consumer problem where the consumer is the VCPU itself.
+ * The irqs_pending bitmap represents pending interrupts whereas
+ * irqs_pending_mask represents bits changed in irqs_pending. Updates
+ * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
+ * drop a newly injected interrupt while syncing guest-visible HVIP.
*/
#define KVM_RISCV_VCPU_NR_IRQS 64
+ raw_spinlock_t irqs_pending_lock;
DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index 5ec503288..821d2cb6d 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -50,17 +50,21 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long mask, val;
+ unsigned long flags;
if (!kvm_riscv_aia_available())
return;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ mask = vcpu->arch.irqs_pending_mask[1];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ val = vcpu->arch.irqs_pending[1] & mask;
csr->hviph &= ~mask;
csr->hviph |= val;
}
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
}
void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
@@ -205,6 +209,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
+#ifdef CONFIG_32BIT
+ unsigned long flags;
+#endif
if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
return -ENOENT;
@@ -214,11 +221,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
reg_num = array_index_nospec(reg_num, regs_max);
if (kvm_riscv_aia_available()) {
- ((unsigned long *)csr)[reg_num] = val;
-
#ifdef CONFIG_32BIT
- if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = val;
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
+ flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = val;
+ }
+#else
+ ((unsigned long *)csr)[reg_num] = val;
#endif
}
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index a73690eda..b04730ccd 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
{
+ unsigned long flags;
bool loaded;
/**
@@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
kvm_riscv_vcpu_aia_reset(vcpu);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_riscv_vcpu_pmu_reset(vcpu);
@@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
/* Setup VCPU hfence queue */
spin_lock_init(&vcpu->arch.hfence_lock);
+ raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
spin_lock_init(&vcpu->arch.reset_state.lock);
@@ -352,14 +356,18 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long mask, val;
+ unsigned long flags;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ mask = vcpu->arch.irqs_pending_mask[0];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ val = vcpu->arch.irqs_pending[0] & mask;
csr->hvip &= ~mask;
csr->hvip |= val;
}
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
/* Flush AIA high interrupts */
kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
@@ -368,6 +376,7 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
{
unsigned long hvip;
+ unsigned long flags;
struct kvm_vcpu_arch *v = &vcpu->arch;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
@@ -376,6 +385,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
/* Sync-up HVIP.VSSIP bit changes does by Guest */
hvip = ncsr_read(CSR_HVIP);
+ raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
if (hvip & (1UL << IRQ_VS_SOFT)) {
if (!test_and_set_bit(IRQ_VS_SOFT,
@@ -394,6 +404,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
!test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
clear_bit(IRQ_PMU_OVF, v->irqs_pending);
}
+ raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
/* Sync-up AIA high interrupts */
kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
@@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, and external
* interrupts when irq is one of the local interrupts
@@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
set_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_vcpu_kick(vcpu);
@@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, counter overflow and external
* interrupts when irq is one of the local interrupts
@@ -439,9 +455,10 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
clear_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
return 0;
}
diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
index bb920e892..cba368294 100644
--- a/arch/riscv/kvm/vcpu_onereg.c
+++ b/arch/riscv/kvm/vcpu_onereg.c
@@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
+ unsigned long flags;
if (reg_num >= regs_max)
return -ENOENT;
@@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
reg_val <<= VSIP_TO_HVIP_SHIFT;
}
- ((unsigned long *)csr)[reg_num] = reg_val;
-
- if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ }
return 0;
}
base-commit: 481329ec5b31d2c48ac6b8b703c8008133884c7e
--
2.54.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
@ 2026-06-16 6:17 ` Xie Bo
0 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-06-16 6:17 UTC (permalink / raw)
To: Anup Patel, kvm-riscv
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm, linux-riscv,
linux-kernel, stable, Xie Bo
RISC-V KVM tracks guest pending interrupts with irqs_pending and
irqs_pending_mask. The existing code uses atomic bitops and models the
state as a multiple-producer, single-consumer queue where the vCPU is
the consumer.
The atomic bitops make each individual bitmap operation atomic, but
they do not make the pair of irqs_pending and irqs_pending_mask an
atomic state transition. The vCPU interrupt sync path is also not a
pure consumer: it writes both bitmaps when it reflects guest-visible
HVIP changes back into KVM state.
For example, kvm_riscv_vcpu_set_interrupt() can set IRQ_VS_SOFT in
irqs_pending while the target vCPU is concurrently syncing a
guest-cleared HVIP.VSSIP bit. If sync observes irqs_pending_mask before
the producer sets it, sync can set the mask and clear irqs_pending. The
producer then sets the mask and kicks the vCPU, but the pending bit has
already been lost. A subsequent flush updates HVIP without VSSIP, and
the guest can remain blocked in WFI even though it has work queued.
Serialize all updates to irqs_pending and irqs_pending_mask with a
per-vCPU raw spinlock. This intentionally replaces the lockless
pending/mask protocol with one small critical section per vCPU, so the
pending bit and the dirty mask are updated as one state transition.
Use the same lock for sync, flush, reset, and userspace CSR writes that
clear the dirty mask, so a newly injected interrupt cannot be
overwritten by a concurrent HVIP sync.
Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
Cc: stable@vger.kernel.org
Signed-off-by: Xie Bo <xb@ultrarisc.com>
---
arch/riscv/include/asm/kvm_host.h | 10 +++++-----
arch/riscv/kvm/aia.c | 28 +++++++++++++++++++++-------
arch/riscv/kvm/vcpu.c | 27 ++++++++++++++++++++++-----
arch/riscv/kvm/vcpu_onereg.c | 13 +++++++++----
4 files changed, 57 insertions(+), 21 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index 75b0a951c..97e42645c 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -207,13 +207,13 @@ struct kvm_vcpu_arch {
/*
* VCPU interrupts
*
- * We have a lockless approach for tracking pending VCPU interrupts
- * implemented using atomic bitops. The irqs_pending bitmap represent
- * pending interrupts whereas irqs_pending_mask represent bits changed
- * in irqs_pending. Our approach is modeled around multiple producer
- * and single consumer problem where the consumer is the VCPU itself.
+ * The irqs_pending bitmap represents pending interrupts whereas
+ * irqs_pending_mask represents bits changed in irqs_pending. Updates
+ * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
+ * drop a newly injected interrupt while syncing guest-visible HVIP.
*/
#define KVM_RISCV_VCPU_NR_IRQS 64
+ raw_spinlock_t irqs_pending_lock;
DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index 5ec503288..821d2cb6d 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -50,17 +50,21 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long mask, val;
+ unsigned long flags;
if (!kvm_riscv_aia_available())
return;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ mask = vcpu->arch.irqs_pending_mask[1];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ val = vcpu->arch.irqs_pending[1] & mask;
csr->hviph &= ~mask;
csr->hviph |= val;
}
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
}
void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
@@ -205,6 +209,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
+#ifdef CONFIG_32BIT
+ unsigned long flags;
+#endif
if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
return -ENOENT;
@@ -214,11 +221,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
reg_num = array_index_nospec(reg_num, regs_max);
if (kvm_riscv_aia_available()) {
- ((unsigned long *)csr)[reg_num] = val;
-
#ifdef CONFIG_32BIT
- if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = val;
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
+ flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = val;
+ }
+#else
+ ((unsigned long *)csr)[reg_num] = val;
#endif
}
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index a73690eda..b04730ccd 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
{
+ unsigned long flags;
bool loaded;
/**
@@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
kvm_riscv_vcpu_aia_reset(vcpu);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_riscv_vcpu_pmu_reset(vcpu);
@@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
/* Setup VCPU hfence queue */
spin_lock_init(&vcpu->arch.hfence_lock);
+ raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
spin_lock_init(&vcpu->arch.reset_state.lock);
@@ -352,14 +356,18 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long mask, val;
+ unsigned long flags;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ mask = vcpu->arch.irqs_pending_mask[0];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ val = vcpu->arch.irqs_pending[0] & mask;
csr->hvip &= ~mask;
csr->hvip |= val;
}
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
/* Flush AIA high interrupts */
kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
@@ -368,6 +376,7 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
{
unsigned long hvip;
+ unsigned long flags;
struct kvm_vcpu_arch *v = &vcpu->arch;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
@@ -376,6 +385,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
/* Sync-up HVIP.VSSIP bit changes does by Guest */
hvip = ncsr_read(CSR_HVIP);
+ raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
if (hvip & (1UL << IRQ_VS_SOFT)) {
if (!test_and_set_bit(IRQ_VS_SOFT,
@@ -394,6 +404,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
!test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
clear_bit(IRQ_PMU_OVF, v->irqs_pending);
}
+ raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
/* Sync-up AIA high interrupts */
kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
@@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, and external
* interrupts when irq is one of the local interrupts
@@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
set_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_vcpu_kick(vcpu);
@@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, counter overflow and external
* interrupts when irq is one of the local interrupts
@@ -439,9 +455,10 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
clear_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
return 0;
}
diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
index bb920e892..cba368294 100644
--- a/arch/riscv/kvm/vcpu_onereg.c
+++ b/arch/riscv/kvm/vcpu_onereg.c
@@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
+ unsigned long flags;
if (reg_num >= regs_max)
return -ENOENT;
@@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
reg_val <<= VSIP_TO_HVIP_SHIFT;
}
- ((unsigned long *)csr)[reg_num] = reg_val;
-
- if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ }
return 0;
}
base-commit: 481329ec5b31d2c48ac6b8b703c8008133884c7e
--
2.54.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
2026-06-16 6:17 ` Xie Bo
(?)
@ 2026-07-07 8:27 ` 谢波
-1 siblings, 0 replies; 26+ messages in thread
From: 谢波 @ 2026-07-07 8:27 UTC (permalink / raw)
To: Anup Patel, kvm-riscv
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm, linux-riscv,
linux-kernel, stable
Hi,
Just a friendly ping on this patch.
Could someone please take a look when convenient?
Thanks,
Xie Bo
> -----原始邮件-----
> 发件人: "Xie Bo" <xb@ultrarisc.com>
> 发送时间:2026-06-16 14:17:58 (星期二)
> 收件人: "Anup Patel" <anup@brainfault.org>, kvm-riscv@lists.infradead.org
> 抄送: "Atish Patra" <atish.patra@linux.dev>, "Paul Walmsley" <pjw@kernel.org>, "Palmer Dabbelt" <palmer@dabbelt.com>, "Albert Ou" <aou@eecs.berkeley.edu>, "Alexandre Ghiti" <alex@ghiti.fr>, "Paolo Bonzini" <pbonzini@redhat.com>, "Alexander Graf" <graf@amazon.com>, kvm@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, "Xie Bo" <xb@ultrarisc.com>
> 主题: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
>
> RISC-V KVM tracks guest pending interrupts with irqs_pending and
> irqs_pending_mask. The existing code uses atomic bitops and models the
> state as a multiple-producer, single-consumer queue where the vCPU is
> the consumer.
>
> The atomic bitops make each individual bitmap operation atomic, but
> they do not make the pair of irqs_pending and irqs_pending_mask an
> atomic state transition. The vCPU interrupt sync path is also not a
> pure consumer: it writes both bitmaps when it reflects guest-visible
> HVIP changes back into KVM state.
>
> For example, kvm_riscv_vcpu_set_interrupt() can set IRQ_VS_SOFT in
> irqs_pending while the target vCPU is concurrently syncing a
> guest-cleared HVIP.VSSIP bit. If sync observes irqs_pending_mask before
> the producer sets it, sync can set the mask and clear irqs_pending. The
> producer then sets the mask and kicks the vCPU, but the pending bit has
> already been lost. A subsequent flush updates HVIP without VSSIP, and
> the guest can remain blocked in WFI even though it has work queued.
>
> Serialize all updates to irqs_pending and irqs_pending_mask with a
> per-vCPU raw spinlock. This intentionally replaces the lockless
> pending/mask protocol with one small critical section per vCPU, so the
> pending bit and the dirty mask are updated as one state transition.
> Use the same lock for sync, flush, reset, and userspace CSR writes that
> clear the dirty mask, so a newly injected interrupt cannot be
> overwritten by a concurrent HVIP sync.
>
> Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
> Cc: stable@vger.kernel.org
> Signed-off-by: Xie Bo <xb@ultrarisc.com>
> ---
> arch/riscv/include/asm/kvm_host.h | 10 +++++-----
> arch/riscv/kvm/aia.c | 28 +++++++++++++++++++++-------
> arch/riscv/kvm/vcpu.c | 27 ++++++++++++++++++++++-----
> arch/riscv/kvm/vcpu_onereg.c | 13 +++++++++----
> 4 files changed, 57 insertions(+), 21 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
> index 75b0a951c..97e42645c 100644
> --- a/arch/riscv/include/asm/kvm_host.h
> +++ b/arch/riscv/include/asm/kvm_host.h
> @@ -207,13 +207,13 @@ struct kvm_vcpu_arch {
> /*
> * VCPU interrupts
> *
> - * We have a lockless approach for tracking pending VCPU interrupts
> - * implemented using atomic bitops. The irqs_pending bitmap represent
> - * pending interrupts whereas irqs_pending_mask represent bits changed
> - * in irqs_pending. Our approach is modeled around multiple producer
> - * and single consumer problem where the consumer is the VCPU itself.
> + * The irqs_pending bitmap represents pending interrupts whereas
> + * irqs_pending_mask represents bits changed in irqs_pending. Updates
> + * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
> + * drop a newly injected interrupt while syncing guest-visible HVIP.
> */
> #define KVM_RISCV_VCPU_NR_IRQS 64
> + raw_spinlock_t irqs_pending_lock;
> DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
>
> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index 5ec503288..821d2cb6d 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
> @@ -50,17 +50,21 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> if (!kvm_riscv_aia_available())
> return;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[1];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + val = vcpu->arch.irqs_pending[1] & mask;
>
> csr->hviph &= ~mask;
> csr->hviph |= val;
> }
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> }
>
> void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
> @@ -205,6 +209,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
> +#ifdef CONFIG_32BIT
> + unsigned long flags;
> +#endif
>
> if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
> return -ENOENT;
> @@ -214,11 +221,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> reg_num = array_index_nospec(reg_num, regs_max);
>
> if (kvm_riscv_aia_available()) {
> - ((unsigned long *)csr)[reg_num] = val;
> -
> #ifdef CONFIG_32BIT
> - if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = val;
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
> + flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = val;
> + }
> +#else
> + ((unsigned long *)csr)[reg_num] = val;
> #endif
> }
>
> diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> index a73690eda..b04730ccd 100644
> --- a/arch/riscv/kvm/vcpu.c
> +++ b/arch/riscv/kvm/vcpu.c
> @@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
>
> static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
> {
> + unsigned long flags;
> bool loaded;
>
> /**
> @@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
>
> kvm_riscv_vcpu_aia_reset(vcpu);
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_riscv_vcpu_pmu_reset(vcpu);
>
> @@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
>
> /* Setup VCPU hfence queue */
> spin_lock_init(&vcpu->arch.hfence_lock);
> + raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
>
> spin_lock_init(&vcpu->arch.reset_state.lock);
>
> @@ -352,14 +356,18 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[0];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + val = vcpu->arch.irqs_pending[0] & mask;
>
> csr->hvip &= ~mask;
> csr->hvip |= val;
> }
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> /* Flush AIA high interrupts */
> kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
> @@ -368,6 +376,7 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> unsigned long hvip;
> + unsigned long flags;
> struct kvm_vcpu_arch *v = &vcpu->arch;
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
>
> @@ -376,6 +385,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> /* Sync-up HVIP.VSSIP bit changes does by Guest */
> hvip = ncsr_read(CSR_HVIP);
> + raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
> if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
> if (hvip & (1UL << IRQ_VS_SOFT)) {
> if (!test_and_set_bit(IRQ_VS_SOFT,
> @@ -394,6 +404,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
> clear_bit(IRQ_PMU_OVF, v->irqs_pending);
> }
> + raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
>
> /* Sync-up AIA high interrupts */
> kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
> @@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, and external
> * interrupts when irq is one of the local interrupts
> @@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> set_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_vcpu_kick(vcpu);
>
> @@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
>
> int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, counter overflow and external
> * interrupts when irq is one of the local interrupts
> @@ -439,9 +455,10 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> clear_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> return 0;
> }
> diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
> index bb920e892..cba368294 100644
> --- a/arch/riscv/kvm/vcpu_onereg.c
> +++ b/arch/riscv/kvm/vcpu_onereg.c
> @@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
> + unsigned long flags;
>
> if (reg_num >= regs_max)
> return -ENOENT;
> @@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> reg_val <<= VSIP_TO_HVIP_SHIFT;
> }
>
> - ((unsigned long *)csr)[reg_num] = reg_val;
> -
> - if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + }
>
> return 0;
> }
>
> base-commit: 481329ec5b31d2c48ac6b8b703c8008133884c7e
> --
> 2.54.0
______________________www.ultrarisc.com
重要提示：本邮件包括附件的内容是受法律保护的保密信息，如果您不是指定收件人，请立即将本邮件删除，法律禁止任何非法的披露、复制、传播或以任何方式使用本邮件。本邮件中包含的意见、建议是基于或受到我方表达和定义的条款及条件的限定，如无我方的正式书面澄清或授权，不可被单独作为任何情形下的证据或依据。感谢您的理解与配合。版权所有。IMPORTANT NOTICE: This email, including its attachment if any, is confidential. If you are not the intended recipient, please delete it from your computer immediately. Any disclosure, copying, or distribution of this message, or taking of any action based on it is strictly prohibited. Any opinions and suggestions contained in this email are subject to the terms and conditions expressed and defined by us and should not be relied upon unconditionally under any circumstances unless they are confirmed in official written clarification or authorization from us. Thank you for your understanding and cooperation.All rights reserved.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
@ 2026-07-07 8:27 ` 谢波
0 siblings, 0 replies; 26+ messages in thread
From: 谢波 @ 2026-07-07 8:27 UTC (permalink / raw)
To: Anup Patel, kvm-riscv
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm, linux-riscv,
linux-kernel, stable
Hi,
Just a friendly ping on this patch.
Could someone please take a look when convenient?
Thanks,
Xie Bo
> -----原始邮件-----
> 发件人: "Xie Bo" <xb@ultrarisc.com>
> 发送时间:2026-06-16 14:17:58 (星期二)
> 收件人: "Anup Patel" <anup@brainfault.org>, kvm-riscv@lists.infradead.org
> 抄送: "Atish Patra" <atish.patra@linux.dev>, "Paul Walmsley" <pjw@kernel.org>, "Palmer Dabbelt" <palmer@dabbelt.com>, "Albert Ou" <aou@eecs.berkeley.edu>, "Alexandre Ghiti" <alex@ghiti.fr>, "Paolo Bonzini" <pbonzini@redhat.com>, "Alexander Graf" <graf@amazon.com>, kvm@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, "Xie Bo" <xb@ultrarisc.com>
> 主题: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
>
> RISC-V KVM tracks guest pending interrupts with irqs_pending and
> irqs_pending_mask. The existing code uses atomic bitops and models the
> state as a multiple-producer, single-consumer queue where the vCPU is
> the consumer.
>
> The atomic bitops make each individual bitmap operation atomic, but
> they do not make the pair of irqs_pending and irqs_pending_mask an
> atomic state transition. The vCPU interrupt sync path is also not a
> pure consumer: it writes both bitmaps when it reflects guest-visible
> HVIP changes back into KVM state.
>
> For example, kvm_riscv_vcpu_set_interrupt() can set IRQ_VS_SOFT in
> irqs_pending while the target vCPU is concurrently syncing a
> guest-cleared HVIP.VSSIP bit. If sync observes irqs_pending_mask before
> the producer sets it, sync can set the mask and clear irqs_pending. The
> producer then sets the mask and kicks the vCPU, but the pending bit has
> already been lost. A subsequent flush updates HVIP without VSSIP, and
> the guest can remain blocked in WFI even though it has work queued.
>
> Serialize all updates to irqs_pending and irqs_pending_mask with a
> per-vCPU raw spinlock. This intentionally replaces the lockless
> pending/mask protocol with one small critical section per vCPU, so the
> pending bit and the dirty mask are updated as one state transition.
> Use the same lock for sync, flush, reset, and userspace CSR writes that
> clear the dirty mask, so a newly injected interrupt cannot be
> overwritten by a concurrent HVIP sync.
>
> Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
> Cc: stable@vger.kernel.org
> Signed-off-by: Xie Bo <xb@ultrarisc.com>
> ---
> arch/riscv/include/asm/kvm_host.h | 10 +++++-----
> arch/riscv/kvm/aia.c | 28 +++++++++++++++++++++-------
> arch/riscv/kvm/vcpu.c | 27 ++++++++++++++++++++++-----
> arch/riscv/kvm/vcpu_onereg.c | 13 +++++++++----
> 4 files changed, 57 insertions(+), 21 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
> index 75b0a951c..97e42645c 100644
> --- a/arch/riscv/include/asm/kvm_host.h
> +++ b/arch/riscv/include/asm/kvm_host.h
> @@ -207,13 +207,13 @@ struct kvm_vcpu_arch {
> /*
> * VCPU interrupts
> *
> - * We have a lockless approach for tracking pending VCPU interrupts
> - * implemented using atomic bitops. The irqs_pending bitmap represent
> - * pending interrupts whereas irqs_pending_mask represent bits changed
> - * in irqs_pending. Our approach is modeled around multiple producer
> - * and single consumer problem where the consumer is the VCPU itself.
> + * The irqs_pending bitmap represents pending interrupts whereas
> + * irqs_pending_mask represents bits changed in irqs_pending. Updates
> + * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
> + * drop a newly injected interrupt while syncing guest-visible HVIP.
> */
> #define KVM_RISCV_VCPU_NR_IRQS 64
> + raw_spinlock_t irqs_pending_lock;
> DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
>
> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index 5ec503288..821d2cb6d 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
> @@ -50,17 +50,21 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> if (!kvm_riscv_aia_available())
> return;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[1];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + val = vcpu->arch.irqs_pending[1] & mask;
>
> csr->hviph &= ~mask;
> csr->hviph |= val;
> }
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> }
>
> void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
> @@ -205,6 +209,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
> +#ifdef CONFIG_32BIT
> + unsigned long flags;
> +#endif
>
> if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
> return -ENOENT;
> @@ -214,11 +221,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> reg_num = array_index_nospec(reg_num, regs_max);
>
> if (kvm_riscv_aia_available()) {
> - ((unsigned long *)csr)[reg_num] = val;
> -
> #ifdef CONFIG_32BIT
> - if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = val;
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
> + flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = val;
> + }
> +#else
> + ((unsigned long *)csr)[reg_num] = val;
> #endif
> }
>
> diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> index a73690eda..b04730ccd 100644
> --- a/arch/riscv/kvm/vcpu.c
> +++ b/arch/riscv/kvm/vcpu.c
> @@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
>
> static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
> {
> + unsigned long flags;
> bool loaded;
>
> /**
> @@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
>
> kvm_riscv_vcpu_aia_reset(vcpu);
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_riscv_vcpu_pmu_reset(vcpu);
>
> @@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
>
> /* Setup VCPU hfence queue */
> spin_lock_init(&vcpu->arch.hfence_lock);
> + raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
>
> spin_lock_init(&vcpu->arch.reset_state.lock);
>
> @@ -352,14 +356,18 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[0];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + val = vcpu->arch.irqs_pending[0] & mask;
>
> csr->hvip &= ~mask;
> csr->hvip |= val;
> }
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> /* Flush AIA high interrupts */
> kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
> @@ -368,6 +376,7 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> unsigned long hvip;
> + unsigned long flags;
> struct kvm_vcpu_arch *v = &vcpu->arch;
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
>
> @@ -376,6 +385,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> /* Sync-up HVIP.VSSIP bit changes does by Guest */
> hvip = ncsr_read(CSR_HVIP);
> + raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
> if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
> if (hvip & (1UL << IRQ_VS_SOFT)) {
> if (!test_and_set_bit(IRQ_VS_SOFT,
> @@ -394,6 +404,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
> clear_bit(IRQ_PMU_OVF, v->irqs_pending);
> }
> + raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
>
> /* Sync-up AIA high interrupts */
> kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
> @@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, and external
> * interrupts when irq is one of the local interrupts
> @@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> set_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_vcpu_kick(vcpu);
>
> @@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
>
> int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, counter overflow and external
> * interrupts when irq is one of the local interrupts
> @@ -439,9 +455,10 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> clear_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> return 0;
> }
> diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
> index bb920e892..cba368294 100644
> --- a/arch/riscv/kvm/vcpu_onereg.c
> +++ b/arch/riscv/kvm/vcpu_onereg.c
> @@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
> + unsigned long flags;
>
> if (reg_num >= regs_max)
> return -ENOENT;
> @@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> reg_val <<= VSIP_TO_HVIP_SHIFT;
> }
>
> - ((unsigned long *)csr)[reg_num] = reg_val;
> -
> - if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + }
>
> return 0;
> }
>
> base-commit: 481329ec5b31d2c48ac6b8b703c8008133884c7e
> --
> 2.54.0
______________________www.ultrarisc.com
重要提示：本邮件包括附件的内容是受法律保护的保密信息，如果您不是指定收件人，请立即将本邮件删除，法律禁止任何非法的披露、复制、传播或以任何方式使用本邮件。本邮件中包含的意见、建议是基于或受到我方表达和定义的条款及条件的限定，如无我方的正式书面澄清或授权，不可被单独作为任何情形下的证据或依据。感谢您的理解与配合。版权所有。IMPORTANT NOTICE: This email, including its attachment if any, is confidential. If you are not the intended recipient, please delete it from your computer immediately. Any disclosure, copying, or distribution of this message, or taking of any action based on it is strictly prohibited. Any opinions and suggestions contained in this email are subject to the terms and conditions expressed and defined by us and should not be relied upon unconditionally under any circumstances unless they are confirmed in official written clarification or authorization from us. Thank you for your understanding and cooperation.All rights reserved.
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
@ 2026-07-07 8:27 ` 谢波
0 siblings, 0 replies; 26+ messages in thread
From: 谢波 @ 2026-07-07 8:27 UTC (permalink / raw)
To: Anup Patel, kvm-riscv
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm, linux-riscv,
linux-kernel, stable
Hi,
Just a friendly ping on this patch.
Could someone please take a look when convenient?
Thanks,
Xie Bo
> -----原始邮件-----
> 发件人: "Xie Bo" <xb@ultrarisc.com>
> 发送时间:2026-06-16 14:17:58 (星期二)
> 收件人: "Anup Patel" <anup@brainfault.org>, kvm-riscv@lists.infradead.org
> 抄送: "Atish Patra" <atish.patra@linux.dev>, "Paul Walmsley" <pjw@kernel.org>, "Palmer Dabbelt" <palmer@dabbelt.com>, "Albert Ou" <aou@eecs.berkeley.edu>, "Alexandre Ghiti" <alex@ghiti.fr>, "Paolo Bonzini" <pbonzini@redhat.com>, "Alexander Graf" <graf@amazon.com>, kvm@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, "Xie Bo" <xb@ultrarisc.com>
> 主题: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
>
> RISC-V KVM tracks guest pending interrupts with irqs_pending and
> irqs_pending_mask. The existing code uses atomic bitops and models the
> state as a multiple-producer, single-consumer queue where the vCPU is
> the consumer.
>
> The atomic bitops make each individual bitmap operation atomic, but
> they do not make the pair of irqs_pending and irqs_pending_mask an
> atomic state transition. The vCPU interrupt sync path is also not a
> pure consumer: it writes both bitmaps when it reflects guest-visible
> HVIP changes back into KVM state.
>
> For example, kvm_riscv_vcpu_set_interrupt() can set IRQ_VS_SOFT in
> irqs_pending while the target vCPU is concurrently syncing a
> guest-cleared HVIP.VSSIP bit. If sync observes irqs_pending_mask before
> the producer sets it, sync can set the mask and clear irqs_pending. The
> producer then sets the mask and kicks the vCPU, but the pending bit has
> already been lost. A subsequent flush updates HVIP without VSSIP, and
> the guest can remain blocked in WFI even though it has work queued.
>
> Serialize all updates to irqs_pending and irqs_pending_mask with a
> per-vCPU raw spinlock. This intentionally replaces the lockless
> pending/mask protocol with one small critical section per vCPU, so the
> pending bit and the dirty mask are updated as one state transition.
> Use the same lock for sync, flush, reset, and userspace CSR writes that
> clear the dirty mask, so a newly injected interrupt cannot be
> overwritten by a concurrent HVIP sync.
>
> Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
> Cc: stable@vger.kernel.org
> Signed-off-by: Xie Bo <xb@ultrarisc.com>
> ---
> arch/riscv/include/asm/kvm_host.h | 10 +++++-----
> arch/riscv/kvm/aia.c | 28 +++++++++++++++++++++-------
> arch/riscv/kvm/vcpu.c | 27 ++++++++++++++++++++++-----
> arch/riscv/kvm/vcpu_onereg.c | 13 +++++++++----
> 4 files changed, 57 insertions(+), 21 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
> index 75b0a951c..97e42645c 100644
> --- a/arch/riscv/include/asm/kvm_host.h
> +++ b/arch/riscv/include/asm/kvm_host.h
> @@ -207,13 +207,13 @@ struct kvm_vcpu_arch {
> /*
> * VCPU interrupts
> *
> - * We have a lockless approach for tracking pending VCPU interrupts
> - * implemented using atomic bitops. The irqs_pending bitmap represent
> - * pending interrupts whereas irqs_pending_mask represent bits changed
> - * in irqs_pending. Our approach is modeled around multiple producer
> - * and single consumer problem where the consumer is the VCPU itself.
> + * The irqs_pending bitmap represents pending interrupts whereas
> + * irqs_pending_mask represents bits changed in irqs_pending. Updates
> + * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
> + * drop a newly injected interrupt while syncing guest-visible HVIP.
> */
> #define KVM_RISCV_VCPU_NR_IRQS 64
> + raw_spinlock_t irqs_pending_lock;
> DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
>
> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index 5ec503288..821d2cb6d 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
> @@ -50,17 +50,21 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> if (!kvm_riscv_aia_available())
> return;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[1];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + val = vcpu->arch.irqs_pending[1] & mask;
>
> csr->hviph &= ~mask;
> csr->hviph |= val;
> }
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> }
>
> void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
> @@ -205,6 +209,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
> +#ifdef CONFIG_32BIT
> + unsigned long flags;
> +#endif
>
> if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
> return -ENOENT;
> @@ -214,11 +221,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> reg_num = array_index_nospec(reg_num, regs_max);
>
> if (kvm_riscv_aia_available()) {
> - ((unsigned long *)csr)[reg_num] = val;
> -
> #ifdef CONFIG_32BIT
> - if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = val;
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
> + flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = val;
> + }
> +#else
> + ((unsigned long *)csr)[reg_num] = val;
> #endif
> }
>
> diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> index a73690eda..b04730ccd 100644
> --- a/arch/riscv/kvm/vcpu.c
> +++ b/arch/riscv/kvm/vcpu.c
> @@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
>
> static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
> {
> + unsigned long flags;
> bool loaded;
>
> /**
> @@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
>
> kvm_riscv_vcpu_aia_reset(vcpu);
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_riscv_vcpu_pmu_reset(vcpu);
>
> @@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
>
> /* Setup VCPU hfence queue */
> spin_lock_init(&vcpu->arch.hfence_lock);
> + raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
>
> spin_lock_init(&vcpu->arch.reset_state.lock);
>
> @@ -352,14 +356,18 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[0];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + val = vcpu->arch.irqs_pending[0] & mask;
>
> csr->hvip &= ~mask;
> csr->hvip |= val;
> }
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> /* Flush AIA high interrupts */
> kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
> @@ -368,6 +376,7 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> unsigned long hvip;
> + unsigned long flags;
> struct kvm_vcpu_arch *v = &vcpu->arch;
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
>
> @@ -376,6 +385,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> /* Sync-up HVIP.VSSIP bit changes does by Guest */
> hvip = ncsr_read(CSR_HVIP);
> + raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
> if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
> if (hvip & (1UL << IRQ_VS_SOFT)) {
> if (!test_and_set_bit(IRQ_VS_SOFT,
> @@ -394,6 +404,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
> clear_bit(IRQ_PMU_OVF, v->irqs_pending);
> }
> + raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
>
> /* Sync-up AIA high interrupts */
> kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
> @@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, and external
> * interrupts when irq is one of the local interrupts
> @@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> set_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_vcpu_kick(vcpu);
>
> @@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
>
> int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, counter overflow and external
> * interrupts when irq is one of the local interrupts
> @@ -439,9 +455,10 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> clear_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> return 0;
> }
> diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
> index bb920e892..cba368294 100644
> --- a/arch/riscv/kvm/vcpu_onereg.c
> +++ b/arch/riscv/kvm/vcpu_onereg.c
> @@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
> + unsigned long flags;
>
> if (reg_num >= regs_max)
> return -ENOENT;
> @@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> reg_val <<= VSIP_TO_HVIP_SHIFT;
> }
>
> - ((unsigned long *)csr)[reg_num] = reg_val;
> -
> - if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + }
>
> return 0;
> }
>
> base-commit: 481329ec5b31d2c48ac6b8b703c8008133884c7e
> --
> 2.54.0
______________________www.ultrarisc.com
重要提示：本邮件包括附件的内容是受法律保护的保密信息，如果您不是指定收件人，请立即将本邮件删除，法律禁止任何非法的披露、复制、传播或以任何方式使用本邮件。本邮件中包含的意见、建议是基于或受到我方表达和定义的条款及条件的限定，如无我方的正式书面澄清或授权，不可被单独作为任何情形下的证据或依据。感谢您的理解与配合。版权所有。IMPORTANT NOTICE: This email, including its attachment if any, is confidential. If you are not the intended recipient, please delete it from your computer immediately. Any disclosure, copying, or distribution of this message, or taking of any action based on it is strictly prohibited. Any opinions and suggestions contained in this email are subject to the terms and conditions expressed and defined by us and should not be relied upon unconditionally under any circumstances unless they are confirmed in official written clarification or authorization from us. Thank you for your understanding and cooperation.All rights reserved.
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
2026-06-16 6:17 ` Xie Bo
(?)
@ 2026-07-13 4:43 ` Anup Patel
-1 siblings, 0 replies; 26+ messages in thread
From: Anup Patel @ 2026-07-13 4:43 UTC (permalink / raw)
To: Xie Bo
Cc: kvm-riscv, Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm, linux-riscv,
linux-kernel, stable
On Tue, Jun 16, 2026 at 11:48 AM Xie Bo <xb@ultrarisc.com> wrote:
>
> RISC-V KVM tracks guest pending interrupts with irqs_pending and
> irqs_pending_mask. The existing code uses atomic bitops and models the
> state as a multiple-producer, single-consumer queue where the vCPU is
> the consumer.
>
> The atomic bitops make each individual bitmap operation atomic, but
> they do not make the pair of irqs_pending and irqs_pending_mask an
> atomic state transition. The vCPU interrupt sync path is also not a
> pure consumer: it writes both bitmaps when it reflects guest-visible
> HVIP changes back into KVM state.
>
> For example, kvm_riscv_vcpu_set_interrupt() can set IRQ_VS_SOFT in
> irqs_pending while the target vCPU is concurrently syncing a
> guest-cleared HVIP.VSSIP bit. If sync observes irqs_pending_mask before
> the producer sets it, sync can set the mask and clear irqs_pending. The
> producer then sets the mask and kicks the vCPU, but the pending bit has
> already been lost. A subsequent flush updates HVIP without VSSIP, and
> the guest can remain blocked in WFI even though it has work queued.
>
> Serialize all updates to irqs_pending and irqs_pending_mask with a
> per-vCPU raw spinlock. This intentionally replaces the lockless
> pending/mask protocol with one small critical section per vCPU, so the
> pending bit and the dirty mask are updated as one state transition.
> Use the same lock for sync, flush, reset, and userspace CSR writes that
> clear the dirty mask, so a newly injected interrupt cannot be
> overwritten by a concurrent HVIP sync.
Overall, I agree with the race condition and proposed solution. The
race-condition is primarily because for VSSIP the VCPU acts as both
producer and consumer which breaks the lockless multi-producer
single-consumer approach.
I have the following suggestions:
1) Now that we have a spinlock protecting irqs_pending* bitmaps,
it is better to do non-atomic bitmap operations on these bitmaps
2) To minimize the lock/unlock dance, various kvm_riscv_vcpu_aia_xyz()
functions must be called with irqs_pending_lock held
3) The kvm_riscv_vcpu_has_interrupts() function must also be updated
to use the irqs_pending_lock
>
> Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
> Cc: stable@vger.kernel.org
> Signed-off-by: Xie Bo <xb@ultrarisc.com>
> ---
> arch/riscv/include/asm/kvm_host.h | 10 +++++-----
> arch/riscv/kvm/aia.c | 28 +++++++++++++++++++++-------
> arch/riscv/kvm/vcpu.c | 27 ++++++++++++++++++++++-----
> arch/riscv/kvm/vcpu_onereg.c | 13 +++++++++----
> 4 files changed, 57 insertions(+), 21 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
> index 75b0a951c..97e42645c 100644
> --- a/arch/riscv/include/asm/kvm_host.h
> +++ b/arch/riscv/include/asm/kvm_host.h
> @@ -207,13 +207,13 @@ struct kvm_vcpu_arch {
> /*
> * VCPU interrupts
> *
> - * We have a lockless approach for tracking pending VCPU interrupts
> - * implemented using atomic bitops. The irqs_pending bitmap represent
> - * pending interrupts whereas irqs_pending_mask represent bits changed
> - * in irqs_pending. Our approach is modeled around multiple producer
> - * and single consumer problem where the consumer is the VCPU itself.
> + * The irqs_pending bitmap represents pending interrupts whereas
> + * irqs_pending_mask represents bits changed in irqs_pending. Updates
> + * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
> + * drop a newly injected interrupt while syncing guest-visible HVIP.
> */
> #define KVM_RISCV_VCPU_NR_IRQS 64
> + raw_spinlock_t irqs_pending_lock;
> DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
>
> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index 5ec503288..821d2cb6d 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
> @@ -50,17 +50,21 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> if (!kvm_riscv_aia_available())
> return;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[1];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + val = vcpu->arch.irqs_pending[1] & mask;
>
> csr->hviph &= ~mask;
> csr->hviph |= val;
> }
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> }
>
> void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
> @@ -205,6 +209,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
> +#ifdef CONFIG_32BIT
> + unsigned long flags;
> +#endif
>
> if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
> return -ENOENT;
> @@ -214,11 +221,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> reg_num = array_index_nospec(reg_num, regs_max);
>
> if (kvm_riscv_aia_available()) {
> - ((unsigned long *)csr)[reg_num] = val;
> -
> #ifdef CONFIG_32BIT
> - if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = val;
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
> + flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = val;
> + }
> +#else
> + ((unsigned long *)csr)[reg_num] = val;
> #endif
> }
>
> diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> index a73690eda..b04730ccd 100644
> --- a/arch/riscv/kvm/vcpu.c
> +++ b/arch/riscv/kvm/vcpu.c
> @@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
>
> static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
> {
> + unsigned long flags;
> bool loaded;
>
> /**
> @@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
>
> kvm_riscv_vcpu_aia_reset(vcpu);
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_riscv_vcpu_pmu_reset(vcpu);
>
> @@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
>
> /* Setup VCPU hfence queue */
> spin_lock_init(&vcpu->arch.hfence_lock);
> + raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
>
> spin_lock_init(&vcpu->arch.reset_state.lock);
>
> @@ -352,14 +356,18 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[0];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + val = vcpu->arch.irqs_pending[0] & mask;
>
> csr->hvip &= ~mask;
> csr->hvip |= val;
> }
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> /* Flush AIA high interrupts */
> kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
> @@ -368,6 +376,7 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> unsigned long hvip;
> + unsigned long flags;
> struct kvm_vcpu_arch *v = &vcpu->arch;
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
>
> @@ -376,6 +385,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> /* Sync-up HVIP.VSSIP bit changes does by Guest */
> hvip = ncsr_read(CSR_HVIP);
> + raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
> if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
> if (hvip & (1UL << IRQ_VS_SOFT)) {
> if (!test_and_set_bit(IRQ_VS_SOFT,
> @@ -394,6 +404,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
> clear_bit(IRQ_PMU_OVF, v->irqs_pending);
> }
> + raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
>
> /* Sync-up AIA high interrupts */
> kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
> @@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, and external
> * interrupts when irq is one of the local interrupts
> @@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> set_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_vcpu_kick(vcpu);
>
> @@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
>
> int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, counter overflow and external
> * interrupts when irq is one of the local interrupts
> @@ -439,9 +455,10 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> clear_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> return 0;
> }
> diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
> index bb920e892..cba368294 100644
> --- a/arch/riscv/kvm/vcpu_onereg.c
> +++ b/arch/riscv/kvm/vcpu_onereg.c
> @@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
> + unsigned long flags;
>
> if (reg_num >= regs_max)
> return -ENOENT;
> @@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> reg_val <<= VSIP_TO_HVIP_SHIFT;
> }
>
> - ((unsigned long *)csr)[reg_num] = reg_val;
> -
> - if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + }
>
> return 0;
> }
>
> base-commit: 481329ec5b31d2c48ac6b8b703c8008133884c7e
> --
> 2.54.0
>
Regards,
Anup
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
@ 2026-07-13 4:43 ` Anup Patel
0 siblings, 0 replies; 26+ messages in thread
From: Anup Patel @ 2026-07-13 4:43 UTC (permalink / raw)
To: Xie Bo
Cc: kvm-riscv, Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm, linux-riscv,
linux-kernel, stable
On Tue, Jun 16, 2026 at 11:48 AM Xie Bo <xb@ultrarisc.com> wrote:
>
> RISC-V KVM tracks guest pending interrupts with irqs_pending and
> irqs_pending_mask. The existing code uses atomic bitops and models the
> state as a multiple-producer, single-consumer queue where the vCPU is
> the consumer.
>
> The atomic bitops make each individual bitmap operation atomic, but
> they do not make the pair of irqs_pending and irqs_pending_mask an
> atomic state transition. The vCPU interrupt sync path is also not a
> pure consumer: it writes both bitmaps when it reflects guest-visible
> HVIP changes back into KVM state.
>
> For example, kvm_riscv_vcpu_set_interrupt() can set IRQ_VS_SOFT in
> irqs_pending while the target vCPU is concurrently syncing a
> guest-cleared HVIP.VSSIP bit. If sync observes irqs_pending_mask before
> the producer sets it, sync can set the mask and clear irqs_pending. The
> producer then sets the mask and kicks the vCPU, but the pending bit has
> already been lost. A subsequent flush updates HVIP without VSSIP, and
> the guest can remain blocked in WFI even though it has work queued.
>
> Serialize all updates to irqs_pending and irqs_pending_mask with a
> per-vCPU raw spinlock. This intentionally replaces the lockless
> pending/mask protocol with one small critical section per vCPU, so the
> pending bit and the dirty mask are updated as one state transition.
> Use the same lock for sync, flush, reset, and userspace CSR writes that
> clear the dirty mask, so a newly injected interrupt cannot be
> overwritten by a concurrent HVIP sync.
Overall, I agree with the race condition and proposed solution. The
race-condition is primarily because for VSSIP the VCPU acts as both
producer and consumer which breaks the lockless multi-producer
single-consumer approach.
I have the following suggestions:
1) Now that we have a spinlock protecting irqs_pending* bitmaps,
it is better to do non-atomic bitmap operations on these bitmaps
2) To minimize the lock/unlock dance, various kvm_riscv_vcpu_aia_xyz()
functions must be called with irqs_pending_lock held
3) The kvm_riscv_vcpu_has_interrupts() function must also be updated
to use the irqs_pending_lock
>
> Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
> Cc: stable@vger.kernel.org
> Signed-off-by: Xie Bo <xb@ultrarisc.com>
> ---
> arch/riscv/include/asm/kvm_host.h | 10 +++++-----
> arch/riscv/kvm/aia.c | 28 +++++++++++++++++++++-------
> arch/riscv/kvm/vcpu.c | 27 ++++++++++++++++++++++-----
> arch/riscv/kvm/vcpu_onereg.c | 13 +++++++++----
> 4 files changed, 57 insertions(+), 21 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
> index 75b0a951c..97e42645c 100644
> --- a/arch/riscv/include/asm/kvm_host.h
> +++ b/arch/riscv/include/asm/kvm_host.h
> @@ -207,13 +207,13 @@ struct kvm_vcpu_arch {
> /*
> * VCPU interrupts
> *
> - * We have a lockless approach for tracking pending VCPU interrupts
> - * implemented using atomic bitops. The irqs_pending bitmap represent
> - * pending interrupts whereas irqs_pending_mask represent bits changed
> - * in irqs_pending. Our approach is modeled around multiple producer
> - * and single consumer problem where the consumer is the VCPU itself.
> + * The irqs_pending bitmap represents pending interrupts whereas
> + * irqs_pending_mask represents bits changed in irqs_pending. Updates
> + * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
> + * drop a newly injected interrupt while syncing guest-visible HVIP.
> */
> #define KVM_RISCV_VCPU_NR_IRQS 64
> + raw_spinlock_t irqs_pending_lock;
> DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
>
> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index 5ec503288..821d2cb6d 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
> @@ -50,17 +50,21 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> if (!kvm_riscv_aia_available())
> return;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[1];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + val = vcpu->arch.irqs_pending[1] & mask;
>
> csr->hviph &= ~mask;
> csr->hviph |= val;
> }
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> }
>
> void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
> @@ -205,6 +209,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
> +#ifdef CONFIG_32BIT
> + unsigned long flags;
> +#endif
>
> if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
> return -ENOENT;
> @@ -214,11 +221,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> reg_num = array_index_nospec(reg_num, regs_max);
>
> if (kvm_riscv_aia_available()) {
> - ((unsigned long *)csr)[reg_num] = val;
> -
> #ifdef CONFIG_32BIT
> - if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = val;
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
> + flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = val;
> + }
> +#else
> + ((unsigned long *)csr)[reg_num] = val;
> #endif
> }
>
> diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> index a73690eda..b04730ccd 100644
> --- a/arch/riscv/kvm/vcpu.c
> +++ b/arch/riscv/kvm/vcpu.c
> @@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
>
> static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
> {
> + unsigned long flags;
> bool loaded;
>
> /**
> @@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
>
> kvm_riscv_vcpu_aia_reset(vcpu);
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_riscv_vcpu_pmu_reset(vcpu);
>
> @@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
>
> /* Setup VCPU hfence queue */
> spin_lock_init(&vcpu->arch.hfence_lock);
> + raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
>
> spin_lock_init(&vcpu->arch.reset_state.lock);
>
> @@ -352,14 +356,18 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[0];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + val = vcpu->arch.irqs_pending[0] & mask;
>
> csr->hvip &= ~mask;
> csr->hvip |= val;
> }
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> /* Flush AIA high interrupts */
> kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
> @@ -368,6 +376,7 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> unsigned long hvip;
> + unsigned long flags;
> struct kvm_vcpu_arch *v = &vcpu->arch;
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
>
> @@ -376,6 +385,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> /* Sync-up HVIP.VSSIP bit changes does by Guest */
> hvip = ncsr_read(CSR_HVIP);
> + raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
> if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
> if (hvip & (1UL << IRQ_VS_SOFT)) {
> if (!test_and_set_bit(IRQ_VS_SOFT,
> @@ -394,6 +404,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
> clear_bit(IRQ_PMU_OVF, v->irqs_pending);
> }
> + raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
>
> /* Sync-up AIA high interrupts */
> kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
> @@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, and external
> * interrupts when irq is one of the local interrupts
> @@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> set_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_vcpu_kick(vcpu);
>
> @@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
>
> int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, counter overflow and external
> * interrupts when irq is one of the local interrupts
> @@ -439,9 +455,10 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> clear_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> return 0;
> }
> diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
> index bb920e892..cba368294 100644
> --- a/arch/riscv/kvm/vcpu_onereg.c
> +++ b/arch/riscv/kvm/vcpu_onereg.c
> @@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
> + unsigned long flags;
>
> if (reg_num >= regs_max)
> return -ENOENT;
> @@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> reg_val <<= VSIP_TO_HVIP_SHIFT;
> }
>
> - ((unsigned long *)csr)[reg_num] = reg_val;
> -
> - if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + }
>
> return 0;
> }
>
> base-commit: 481329ec5b31d2c48ac6b8b703c8008133884c7e
> --
> 2.54.0
>
Regards,
Anup
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
@ 2026-07-13 4:43 ` Anup Patel
0 siblings, 0 replies; 26+ messages in thread
From: Anup Patel @ 2026-07-13 4:43 UTC (permalink / raw)
To: Xie Bo
Cc: kvm-riscv, Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm, linux-riscv,
linux-kernel, stable
On Tue, Jun 16, 2026 at 11:48 AM Xie Bo <xb@ultrarisc.com> wrote:
>
> RISC-V KVM tracks guest pending interrupts with irqs_pending and
> irqs_pending_mask. The existing code uses atomic bitops and models the
> state as a multiple-producer, single-consumer queue where the vCPU is
> the consumer.
>
> The atomic bitops make each individual bitmap operation atomic, but
> they do not make the pair of irqs_pending and irqs_pending_mask an
> atomic state transition. The vCPU interrupt sync path is also not a
> pure consumer: it writes both bitmaps when it reflects guest-visible
> HVIP changes back into KVM state.
>
> For example, kvm_riscv_vcpu_set_interrupt() can set IRQ_VS_SOFT in
> irqs_pending while the target vCPU is concurrently syncing a
> guest-cleared HVIP.VSSIP bit. If sync observes irqs_pending_mask before
> the producer sets it, sync can set the mask and clear irqs_pending. The
> producer then sets the mask and kicks the vCPU, but the pending bit has
> already been lost. A subsequent flush updates HVIP without VSSIP, and
> the guest can remain blocked in WFI even though it has work queued.
>
> Serialize all updates to irqs_pending and irqs_pending_mask with a
> per-vCPU raw spinlock. This intentionally replaces the lockless
> pending/mask protocol with one small critical section per vCPU, so the
> pending bit and the dirty mask are updated as one state transition.
> Use the same lock for sync, flush, reset, and userspace CSR writes that
> clear the dirty mask, so a newly injected interrupt cannot be
> overwritten by a concurrent HVIP sync.
Overall, I agree with the race condition and proposed solution. The
race-condition is primarily because for VSSIP the VCPU acts as both
producer and consumer which breaks the lockless multi-producer
single-consumer approach.
I have the following suggestions:
1) Now that we have a spinlock protecting irqs_pending* bitmaps,
it is better to do non-atomic bitmap operations on these bitmaps
2) To minimize the lock/unlock dance, various kvm_riscv_vcpu_aia_xyz()
functions must be called with irqs_pending_lock held
3) The kvm_riscv_vcpu_has_interrupts() function must also be updated
to use the irqs_pending_lock
>
> Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
> Cc: stable@vger.kernel.org
> Signed-off-by: Xie Bo <xb@ultrarisc.com>
> ---
> arch/riscv/include/asm/kvm_host.h | 10 +++++-----
> arch/riscv/kvm/aia.c | 28 +++++++++++++++++++++-------
> arch/riscv/kvm/vcpu.c | 27 ++++++++++++++++++++++-----
> arch/riscv/kvm/vcpu_onereg.c | 13 +++++++++----
> 4 files changed, 57 insertions(+), 21 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
> index 75b0a951c..97e42645c 100644
> --- a/arch/riscv/include/asm/kvm_host.h
> +++ b/arch/riscv/include/asm/kvm_host.h
> @@ -207,13 +207,13 @@ struct kvm_vcpu_arch {
> /*
> * VCPU interrupts
> *
> - * We have a lockless approach for tracking pending VCPU interrupts
> - * implemented using atomic bitops. The irqs_pending bitmap represent
> - * pending interrupts whereas irqs_pending_mask represent bits changed
> - * in irqs_pending. Our approach is modeled around multiple producer
> - * and single consumer problem where the consumer is the VCPU itself.
> + * The irqs_pending bitmap represents pending interrupts whereas
> + * irqs_pending_mask represents bits changed in irqs_pending. Updates
> + * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
> + * drop a newly injected interrupt while syncing guest-visible HVIP.
> */
> #define KVM_RISCV_VCPU_NR_IRQS 64
> + raw_spinlock_t irqs_pending_lock;
> DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
>
> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index 5ec503288..821d2cb6d 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
> @@ -50,17 +50,21 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> if (!kvm_riscv_aia_available())
> return;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[1];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + val = vcpu->arch.irqs_pending[1] & mask;
>
> csr->hviph &= ~mask;
> csr->hviph |= val;
> }
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> }
>
> void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
> @@ -205,6 +209,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
> +#ifdef CONFIG_32BIT
> + unsigned long flags;
> +#endif
>
> if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
> return -ENOENT;
> @@ -214,11 +221,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> reg_num = array_index_nospec(reg_num, regs_max);
>
> if (kvm_riscv_aia_available()) {
> - ((unsigned long *)csr)[reg_num] = val;
> -
> #ifdef CONFIG_32BIT
> - if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = val;
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
> + flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = val;
> + }
> +#else
> + ((unsigned long *)csr)[reg_num] = val;
> #endif
> }
>
> diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> index a73690eda..b04730ccd 100644
> --- a/arch/riscv/kvm/vcpu.c
> +++ b/arch/riscv/kvm/vcpu.c
> @@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
>
> static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
> {
> + unsigned long flags;
> bool loaded;
>
> /**
> @@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
>
> kvm_riscv_vcpu_aia_reset(vcpu);
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_riscv_vcpu_pmu_reset(vcpu);
>
> @@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
>
> /* Setup VCPU hfence queue */
> spin_lock_init(&vcpu->arch.hfence_lock);
> + raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
>
> spin_lock_init(&vcpu->arch.reset_state.lock);
>
> @@ -352,14 +356,18 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[0];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + val = vcpu->arch.irqs_pending[0] & mask;
>
> csr->hvip &= ~mask;
> csr->hvip |= val;
> }
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> /* Flush AIA high interrupts */
> kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
> @@ -368,6 +376,7 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> unsigned long hvip;
> + unsigned long flags;
> struct kvm_vcpu_arch *v = &vcpu->arch;
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
>
> @@ -376,6 +385,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> /* Sync-up HVIP.VSSIP bit changes does by Guest */
> hvip = ncsr_read(CSR_HVIP);
> + raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
> if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
> if (hvip & (1UL << IRQ_VS_SOFT)) {
> if (!test_and_set_bit(IRQ_VS_SOFT,
> @@ -394,6 +404,7 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
> clear_bit(IRQ_PMU_OVF, v->irqs_pending);
> }
> + raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
>
> /* Sync-up AIA high interrupts */
> kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
> @@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, and external
> * interrupts when irq is one of the local interrupts
> @@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> set_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_vcpu_kick(vcpu);
>
> @@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
>
> int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, counter overflow and external
> * interrupts when irq is one of the local interrupts
> @@ -439,9 +455,10 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> clear_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> return 0;
> }
> diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
> index bb920e892..cba368294 100644
> --- a/arch/riscv/kvm/vcpu_onereg.c
> +++ b/arch/riscv/kvm/vcpu_onereg.c
> @@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
> + unsigned long flags;
>
> if (reg_num >= regs_max)
> return -ENOENT;
> @@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> reg_val <<= VSIP_TO_HVIP_SHIFT;
> }
>
> - ((unsigned long *)csr)[reg_num] = reg_val;
> -
> - if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + }
>
> return 0;
> }
>
> base-commit: 481329ec5b31d2c48ac6b8b703c8008133884c7e
> --
> 2.54.0
>
Regards,
Anup
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
2026-07-13 4:43 ` Anup Patel
(?)
@ 2026-07-13 6:48 ` Xie Bo
-1 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-07-13 6:48 UTC (permalink / raw)
To: Anup Patel
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable, Xie Bo
Hi Anup,
I have addressed all three suggestions in v3:
- use non-atomic bitmap operations while holding irqs_pending_lock;
- hold irqs_pending_lock across the AIA sync, flush, and pending checks,
with lockdep assertions in the AIA helpers; and
- protect kvm_riscv_vcpu_has_interrupts() with the same lock.
I also rebased the patch onto Linux 7.2-rc3. Both RV64 and RV32 KVM
builds pass, and checkpatch reports no issues.
The v3 patch follows this reply.
Regards,
Xie Bo
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
@ 2026-07-13 6:48 ` Xie Bo
0 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-07-13 6:48 UTC (permalink / raw)
To: Anup Patel
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable, Xie Bo
Hi Anup,
I have addressed all three suggestions in v3:
- use non-atomic bitmap operations while holding irqs_pending_lock;
- hold irqs_pending_lock across the AIA sync, flush, and pending checks,
with lockdep assertions in the AIA helpers; and
- protect kvm_riscv_vcpu_has_interrupts() with the same lock.
I also rebased the patch onto Linux 7.2-rc3. Both RV64 and RV32 KVM
builds pass, and checkpatch reports no issues.
The v3 patch follows this reply.
Regards,
Xie Bo
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
@ 2026-07-13 6:48 ` Xie Bo
0 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-07-13 6:48 UTC (permalink / raw)
To: Anup Patel
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable, Xie Bo
Hi Anup,
I have addressed all three suggestions in v3:
- use non-atomic bitmap operations while holding irqs_pending_lock;
- hold irqs_pending_lock across the AIA sync, flush, and pending checks,
with lockdep assertions in the AIA helpers; and
- protect kvm_riscv_vcpu_has_interrupts() with the same lock.
I also rebased the patch onto Linux 7.2-rc3. Both RV64 and RV32 KVM
builds pass, and checkpatch reports no issues.
The v3 patch follows this reply.
Regards,
Xie Bo
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PATCH v3] RISC-V: KVM: Serialize virtual interrupt pending state updates
2026-07-13 6:48 ` Xie Bo
(?)
@ 2026-07-13 6:48 ` Xie Bo
-1 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-07-13 6:48 UTC (permalink / raw)
To: Anup Patel
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable, Xie Bo
RISC-V KVM tracks guest interrupt state with two bitmaps:
- irqs_pending: interrupts that should be visible to the guest
- irqs_pending_mask: interrupts whose pending state changed
The current code updates those bitmaps with independent atomic bitops
and assumes a multiple-producer, single-consumer protocol. That model
does not actually hold.
kvm_riscv_vcpu_sync_interrupts() is not a pure consumer. When the guest
changes guest-visible HVIP state, sync_interrupts() writes both
irqs_pending and irqs_pending_mask to reflect the new guest state back
into KVM state. As a result, irqs_pending and irqs_pending_mask form a
single logical state transition, but they are not updated atomically as
a pair.
This allows a race where a newly injected interrupt is lost. For
example:
CPU0 CPU1
---- ----
kvm_riscv_vcpu_set_interrupt(VS_SOFT)
set_bit(VS_SOFT, irqs_pending)
kvm_riscv_vcpu_sync_interrupts()
sees guest-cleared HVIP.VSSIP
sets irqs_pending_mask
clears irqs_pending
set_bit(VS_SOFT, irqs_pending_mask)
kvm_vcpu_kick()
After that interleaving, a later flush can update HVIP without VSSIP
even though a new virtual interrupt was injected. In practice, the
guest can remain blocked in WFI with work pending.
The same pending/mask protocol is shared by VS soft interrupts, PMU
overflow delivery, and AIA high interrupt synchronization, so the race
is not limited to one interrupt source.
Fix this by serializing all updates to irqs_pending and
irqs_pending_mask with a per-vCPU raw spinlock. This keeps the pending
bit and the dirty mask as one state transition across:
- set/unset interrupt
- guest HVIP sync
- interrupt flush to guest CSR state
- vCPU reset
- AIA CSR writes that clear dirty state
Use non-atomic bitmap operations while holding the lock. Hold the lock
across the AIA sync, flush, and pending checks as well, so both bitmap
words share the same serialization domain.
This intentionally replaces the existing lockless protocol instead of
trying to repair it with additional barriers. The problem is not memory
ordering on a single field; it is that two separate bitmaps encode one
shared state machine while both producers and sync paths can modify
them. A per-vCPU raw spinlock keeps the fix small, local, and suitable
for backporting.
Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
Cc: stable@vger.kernel.org
Signed-off-by: Xie Bo <xb@ultrarisc.com>
---
Changes in v3:
- Rebase onto Linux 7.2-rc3.
- Use non-atomic bitmap operations under irqs_pending_lock.
- Hold irqs_pending_lock across the AIA sync/flush helpers and add
lockdep assertions for their locking contract.
- Protect kvm_riscv_vcpu_has_interrupts() with irqs_pending_lock.
Changes in v2:
- Expand the race description and user-visible failure mode.
arch/riscv/include/asm/kvm_host.h | 10 ++---
arch/riscv/kvm/aia.c | 33 ++++++++++++----
arch/riscv/kvm/vcpu.c | 63 +++++++++++++++++++++----------
arch/riscv/kvm/vcpu_onereg.c | 13 +++++--
4 files changed, 82 insertions(+), 37 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index 60017ceec9d..e2d5808169e 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -209,13 +209,13 @@ struct kvm_vcpu_arch {
/*
* VCPU interrupts
*
- * We have a lockless approach for tracking pending VCPU interrupts
- * implemented using atomic bitops. The irqs_pending bitmap represent
- * pending interrupts whereas irqs_pending_mask represent bits changed
- * in irqs_pending. Our approach is modeled around multiple producer
- * and single consumer problem where the consumer is the VCPU itself.
+ * The irqs_pending bitmap represents pending interrupts whereas
+ * irqs_pending_mask represents bits changed in irqs_pending. Updates
+ * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
+ * drop a newly injected interrupt while syncing guest-visible HVIP.
*/
#define KVM_RISCV_VCPU_NR_IRQS 64
+ raw_spinlock_t irqs_pending_lock;
DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index bafb009c5ce..225e4aed81a 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -53,12 +53,15 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long mask, val;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (!kvm_riscv_aia_available())
return;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
+ mask = vcpu->arch.irqs_pending_mask[1];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ val = vcpu->arch.irqs_pending[1] & mask;
csr->hviph &= ~mask;
csr->hviph |= val;
@@ -69,6 +72,8 @@ void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (kvm_riscv_aia_available())
csr->vsieh = ncsr_read(CSR_VSIEH);
}
@@ -78,11 +83,13 @@ bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
{
unsigned long seip;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (!kvm_riscv_aia_available())
return false;
#ifdef CONFIG_32BIT
- if (READ_ONCE(vcpu->arch.irqs_pending[1]) &
+ if (vcpu->arch.irqs_pending[1] &
(vcpu->arch.aia_context.guest_csr.vsieh & upper_32_bits(mask)))
return true;
#endif
@@ -207,6 +214,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
+#ifdef CONFIG_32BIT
+ unsigned long flags;
+#endif
if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
return -ENOENT;
@@ -216,11 +226,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
reg_num = array_index_nospec(reg_num, regs_max);
if (kvm_riscv_aia_available()) {
- ((unsigned long *)csr)[reg_num] = val;
-
#ifdef CONFIG_32BIT
- if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = val;
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
+ flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = val;
+ }
+#else
+ ((unsigned long *)csr)[reg_num] = val;
#endif
}
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index cf6e231e76e..7d8d20839d4 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
{
+ unsigned long flags;
bool loaded;
/**
@@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
kvm_riscv_vcpu_aia_reset(vcpu);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_riscv_vcpu_pmu_reset(vcpu);
@@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
/* Setup VCPU hfence queue */
spin_lock_init(&vcpu->arch.hfence_lock);
+ raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
spin_lock_init(&vcpu->arch.reset_state.lock);
@@ -352,10 +356,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long mask, val;
+ unsigned long flags;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ mask = vcpu->arch.irqs_pending_mask[0];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ val = vcpu->arch.irqs_pending[0] & mask;
csr->hvip &= ~mask;
csr->hvip |= val;
@@ -363,11 +370,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
/* Flush AIA high interrupts */
kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
}
void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
{
unsigned long hvip;
+ unsigned long flags;
struct kvm_vcpu_arch *v = &vcpu->arch;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
@@ -376,27 +385,29 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
/* Sync-up HVIP.VSSIP bit changes does by Guest */
hvip = ncsr_read(CSR_HVIP);
+ raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
if (hvip & (1UL << IRQ_VS_SOFT)) {
- if (!test_and_set_bit(IRQ_VS_SOFT,
- v->irqs_pending_mask))
- set_bit(IRQ_VS_SOFT, v->irqs_pending);
+ if (!__test_and_set_bit(IRQ_VS_SOFT,
+ v->irqs_pending_mask))
+ __set_bit(IRQ_VS_SOFT, v->irqs_pending);
} else {
- if (!test_and_set_bit(IRQ_VS_SOFT,
- v->irqs_pending_mask))
- clear_bit(IRQ_VS_SOFT, v->irqs_pending);
+ if (!__test_and_set_bit(IRQ_VS_SOFT,
+ v->irqs_pending_mask))
+ __clear_bit(IRQ_VS_SOFT, v->irqs_pending);
}
}
/* Sync up the HVIP.LCOFIP bit changes (only clear) by the guest */
if ((csr->hvip ^ hvip) & (1UL << IRQ_PMU_OVF)) {
if (!(hvip & (1UL << IRQ_PMU_OVF)) &&
- !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
- clear_bit(IRQ_PMU_OVF, v->irqs_pending);
+ !__test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
+ __clear_bit(IRQ_PMU_OVF, v->irqs_pending);
}
/* Sync-up AIA high interrupts */
kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
+ raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
/* Sync-up timer CSRs */
kvm_riscv_vcpu_timer_sync(vcpu);
@@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, and external
* interrupts when irq is one of the local interrupts
@@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
- set_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
- set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ __set_bit(irq, vcpu->arch.irqs_pending);
+ __set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_vcpu_kick(vcpu);
@@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, counter overflow and external
* interrupts when irq is one of the local interrupts
@@ -439,26 +455,33 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
- clear_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
- set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ __clear_bit(irq, vcpu->arch.irqs_pending);
+ __set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
return 0;
}
bool kvm_riscv_vcpu_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
{
+ unsigned long flags;
unsigned long ie;
+ bool ret;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
ie = ((vcpu->arch.guest_csr.vsie & VSIP_VALID_MASK)
<< VSIP_TO_HVIP_SHIFT) & (unsigned long)mask;
ie |= vcpu->arch.guest_csr.vsie & ~IRQ_LOCAL_MASK &
(unsigned long)mask;
- if (READ_ONCE(vcpu->arch.irqs_pending[0]) & ie)
- return true;
+ ret = vcpu->arch.irqs_pending[0] & ie;
/* Check AIA high interrupts */
- return kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
+ if (!ret)
+ ret = kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
+
+ return ret;
}
void __kvm_riscv_vcpu_power_off(struct kvm_vcpu *vcpu)
diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
index bb920e8923c..cba3682944b 100644
--- a/arch/riscv/kvm/vcpu_onereg.c
+++ b/arch/riscv/kvm/vcpu_onereg.c
@@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
+ unsigned long flags;
if (reg_num >= regs_max)
return -ENOENT;
@@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
reg_val <<= VSIP_TO_HVIP_SHIFT;
}
- ((unsigned long *)csr)[reg_num] = reg_val;
-
- if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ }
return 0;
}
--
2.54.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH v3] RISC-V: KVM: Serialize virtual interrupt pending state updates
@ 2026-07-13 6:48 ` Xie Bo
0 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-07-13 6:48 UTC (permalink / raw)
To: Anup Patel
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable, Xie Bo
RISC-V KVM tracks guest interrupt state with two bitmaps:
- irqs_pending: interrupts that should be visible to the guest
- irqs_pending_mask: interrupts whose pending state changed
The current code updates those bitmaps with independent atomic bitops
and assumes a multiple-producer, single-consumer protocol. That model
does not actually hold.
kvm_riscv_vcpu_sync_interrupts() is not a pure consumer. When the guest
changes guest-visible HVIP state, sync_interrupts() writes both
irqs_pending and irqs_pending_mask to reflect the new guest state back
into KVM state. As a result, irqs_pending and irqs_pending_mask form a
single logical state transition, but they are not updated atomically as
a pair.
This allows a race where a newly injected interrupt is lost. For
example:
CPU0 CPU1
---- ----
kvm_riscv_vcpu_set_interrupt(VS_SOFT)
set_bit(VS_SOFT, irqs_pending)
kvm_riscv_vcpu_sync_interrupts()
sees guest-cleared HVIP.VSSIP
sets irqs_pending_mask
clears irqs_pending
set_bit(VS_SOFT, irqs_pending_mask)
kvm_vcpu_kick()
After that interleaving, a later flush can update HVIP without VSSIP
even though a new virtual interrupt was injected. In practice, the
guest can remain blocked in WFI with work pending.
The same pending/mask protocol is shared by VS soft interrupts, PMU
overflow delivery, and AIA high interrupt synchronization, so the race
is not limited to one interrupt source.
Fix this by serializing all updates to irqs_pending and
irqs_pending_mask with a per-vCPU raw spinlock. This keeps the pending
bit and the dirty mask as one state transition across:
- set/unset interrupt
- guest HVIP sync
- interrupt flush to guest CSR state
- vCPU reset
- AIA CSR writes that clear dirty state
Use non-atomic bitmap operations while holding the lock. Hold the lock
across the AIA sync, flush, and pending checks as well, so both bitmap
words share the same serialization domain.
This intentionally replaces the existing lockless protocol instead of
trying to repair it with additional barriers. The problem is not memory
ordering on a single field; it is that two separate bitmaps encode one
shared state machine while both producers and sync paths can modify
them. A per-vCPU raw spinlock keeps the fix small, local, and suitable
for backporting.
Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
Cc: stable@vger.kernel.org
Signed-off-by: Xie Bo <xb@ultrarisc.com>
---
Changes in v3:
- Rebase onto Linux 7.2-rc3.
- Use non-atomic bitmap operations under irqs_pending_lock.
- Hold irqs_pending_lock across the AIA sync/flush helpers and add
lockdep assertions for their locking contract.
- Protect kvm_riscv_vcpu_has_interrupts() with irqs_pending_lock.
Changes in v2:
- Expand the race description and user-visible failure mode.
arch/riscv/include/asm/kvm_host.h | 10 ++---
arch/riscv/kvm/aia.c | 33 ++++++++++++----
arch/riscv/kvm/vcpu.c | 63 +++++++++++++++++++++----------
arch/riscv/kvm/vcpu_onereg.c | 13 +++++--
4 files changed, 82 insertions(+), 37 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index 60017ceec9d..e2d5808169e 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -209,13 +209,13 @@ struct kvm_vcpu_arch {
/*
* VCPU interrupts
*
- * We have a lockless approach for tracking pending VCPU interrupts
- * implemented using atomic bitops. The irqs_pending bitmap represent
- * pending interrupts whereas irqs_pending_mask represent bits changed
- * in irqs_pending. Our approach is modeled around multiple producer
- * and single consumer problem where the consumer is the VCPU itself.
+ * The irqs_pending bitmap represents pending interrupts whereas
+ * irqs_pending_mask represents bits changed in irqs_pending. Updates
+ * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
+ * drop a newly injected interrupt while syncing guest-visible HVIP.
*/
#define KVM_RISCV_VCPU_NR_IRQS 64
+ raw_spinlock_t irqs_pending_lock;
DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index bafb009c5ce..225e4aed81a 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -53,12 +53,15 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long mask, val;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (!kvm_riscv_aia_available())
return;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
+ mask = vcpu->arch.irqs_pending_mask[1];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ val = vcpu->arch.irqs_pending[1] & mask;
csr->hviph &= ~mask;
csr->hviph |= val;
@@ -69,6 +72,8 @@ void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (kvm_riscv_aia_available())
csr->vsieh = ncsr_read(CSR_VSIEH);
}
@@ -78,11 +83,13 @@ bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
{
unsigned long seip;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (!kvm_riscv_aia_available())
return false;
#ifdef CONFIG_32BIT
- if (READ_ONCE(vcpu->arch.irqs_pending[1]) &
+ if (vcpu->arch.irqs_pending[1] &
(vcpu->arch.aia_context.guest_csr.vsieh & upper_32_bits(mask)))
return true;
#endif
@@ -207,6 +214,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
+#ifdef CONFIG_32BIT
+ unsigned long flags;
+#endif
if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
return -ENOENT;
@@ -216,11 +226,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
reg_num = array_index_nospec(reg_num, regs_max);
if (kvm_riscv_aia_available()) {
- ((unsigned long *)csr)[reg_num] = val;
-
#ifdef CONFIG_32BIT
- if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = val;
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
+ flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = val;
+ }
+#else
+ ((unsigned long *)csr)[reg_num] = val;
#endif
}
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index cf6e231e76e..7d8d20839d4 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
{
+ unsigned long flags;
bool loaded;
/**
@@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
kvm_riscv_vcpu_aia_reset(vcpu);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_riscv_vcpu_pmu_reset(vcpu);
@@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
/* Setup VCPU hfence queue */
spin_lock_init(&vcpu->arch.hfence_lock);
+ raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
spin_lock_init(&vcpu->arch.reset_state.lock);
@@ -352,10 +356,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long mask, val;
+ unsigned long flags;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ mask = vcpu->arch.irqs_pending_mask[0];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ val = vcpu->arch.irqs_pending[0] & mask;
csr->hvip &= ~mask;
csr->hvip |= val;
@@ -363,11 +370,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
/* Flush AIA high interrupts */
kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
}
void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
{
unsigned long hvip;
+ unsigned long flags;
struct kvm_vcpu_arch *v = &vcpu->arch;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
@@ -376,27 +385,29 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
/* Sync-up HVIP.VSSIP bit changes does by Guest */
hvip = ncsr_read(CSR_HVIP);
+ raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
if (hvip & (1UL << IRQ_VS_SOFT)) {
- if (!test_and_set_bit(IRQ_VS_SOFT,
- v->irqs_pending_mask))
- set_bit(IRQ_VS_SOFT, v->irqs_pending);
+ if (!__test_and_set_bit(IRQ_VS_SOFT,
+ v->irqs_pending_mask))
+ __set_bit(IRQ_VS_SOFT, v->irqs_pending);
} else {
- if (!test_and_set_bit(IRQ_VS_SOFT,
- v->irqs_pending_mask))
- clear_bit(IRQ_VS_SOFT, v->irqs_pending);
+ if (!__test_and_set_bit(IRQ_VS_SOFT,
+ v->irqs_pending_mask))
+ __clear_bit(IRQ_VS_SOFT, v->irqs_pending);
}
}
/* Sync up the HVIP.LCOFIP bit changes (only clear) by the guest */
if ((csr->hvip ^ hvip) & (1UL << IRQ_PMU_OVF)) {
if (!(hvip & (1UL << IRQ_PMU_OVF)) &&
- !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
- clear_bit(IRQ_PMU_OVF, v->irqs_pending);
+ !__test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
+ __clear_bit(IRQ_PMU_OVF, v->irqs_pending);
}
/* Sync-up AIA high interrupts */
kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
+ raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
/* Sync-up timer CSRs */
kvm_riscv_vcpu_timer_sync(vcpu);
@@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, and external
* interrupts when irq is one of the local interrupts
@@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
- set_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
- set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ __set_bit(irq, vcpu->arch.irqs_pending);
+ __set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_vcpu_kick(vcpu);
@@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, counter overflow and external
* interrupts when irq is one of the local interrupts
@@ -439,26 +455,33 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
- clear_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
- set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ __clear_bit(irq, vcpu->arch.irqs_pending);
+ __set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
return 0;
}
bool kvm_riscv_vcpu_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
{
+ unsigned long flags;
unsigned long ie;
+ bool ret;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
ie = ((vcpu->arch.guest_csr.vsie & VSIP_VALID_MASK)
<< VSIP_TO_HVIP_SHIFT) & (unsigned long)mask;
ie |= vcpu->arch.guest_csr.vsie & ~IRQ_LOCAL_MASK &
(unsigned long)mask;
- if (READ_ONCE(vcpu->arch.irqs_pending[0]) & ie)
- return true;
+ ret = vcpu->arch.irqs_pending[0] & ie;
/* Check AIA high interrupts */
- return kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
+ if (!ret)
+ ret = kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
+
+ return ret;
}
void __kvm_riscv_vcpu_power_off(struct kvm_vcpu *vcpu)
diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
index bb920e8923c..cba3682944b 100644
--- a/arch/riscv/kvm/vcpu_onereg.c
+++ b/arch/riscv/kvm/vcpu_onereg.c
@@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
+ unsigned long flags;
if (reg_num >= regs_max)
return -ENOENT;
@@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
reg_val <<= VSIP_TO_HVIP_SHIFT;
}
- ((unsigned long *)csr)[reg_num] = reg_val;
-
- if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ }
return 0;
}
--
2.54.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH v3] RISC-V: KVM: Serialize virtual interrupt pending state updates
@ 2026-07-13 6:48 ` Xie Bo
0 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-07-13 6:48 UTC (permalink / raw)
To: Anup Patel
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable, Xie Bo
RISC-V KVM tracks guest interrupt state with two bitmaps:
- irqs_pending: interrupts that should be visible to the guest
- irqs_pending_mask: interrupts whose pending state changed
The current code updates those bitmaps with independent atomic bitops
and assumes a multiple-producer, single-consumer protocol. That model
does not actually hold.
kvm_riscv_vcpu_sync_interrupts() is not a pure consumer. When the guest
changes guest-visible HVIP state, sync_interrupts() writes both
irqs_pending and irqs_pending_mask to reflect the new guest state back
into KVM state. As a result, irqs_pending and irqs_pending_mask form a
single logical state transition, but they are not updated atomically as
a pair.
This allows a race where a newly injected interrupt is lost. For
example:
CPU0 CPU1
---- ----
kvm_riscv_vcpu_set_interrupt(VS_SOFT)
set_bit(VS_SOFT, irqs_pending)
kvm_riscv_vcpu_sync_interrupts()
sees guest-cleared HVIP.VSSIP
sets irqs_pending_mask
clears irqs_pending
set_bit(VS_SOFT, irqs_pending_mask)
kvm_vcpu_kick()
After that interleaving, a later flush can update HVIP without VSSIP
even though a new virtual interrupt was injected. In practice, the
guest can remain blocked in WFI with work pending.
The same pending/mask protocol is shared by VS soft interrupts, PMU
overflow delivery, and AIA high interrupt synchronization, so the race
is not limited to one interrupt source.
Fix this by serializing all updates to irqs_pending and
irqs_pending_mask with a per-vCPU raw spinlock. This keeps the pending
bit and the dirty mask as one state transition across:
- set/unset interrupt
- guest HVIP sync
- interrupt flush to guest CSR state
- vCPU reset
- AIA CSR writes that clear dirty state
Use non-atomic bitmap operations while holding the lock. Hold the lock
across the AIA sync, flush, and pending checks as well, so both bitmap
words share the same serialization domain.
This intentionally replaces the existing lockless protocol instead of
trying to repair it with additional barriers. The problem is not memory
ordering on a single field; it is that two separate bitmaps encode one
shared state machine while both producers and sync paths can modify
them. A per-vCPU raw spinlock keeps the fix small, local, and suitable
for backporting.
Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
Cc: stable@vger.kernel.org
Signed-off-by: Xie Bo <xb@ultrarisc.com>
---
Changes in v3:
- Rebase onto Linux 7.2-rc3.
- Use non-atomic bitmap operations under irqs_pending_lock.
- Hold irqs_pending_lock across the AIA sync/flush helpers and add
lockdep assertions for their locking contract.
- Protect kvm_riscv_vcpu_has_interrupts() with irqs_pending_lock.
Changes in v2:
- Expand the race description and user-visible failure mode.
arch/riscv/include/asm/kvm_host.h | 10 ++---
arch/riscv/kvm/aia.c | 33 ++++++++++++----
arch/riscv/kvm/vcpu.c | 63 +++++++++++++++++++++----------
arch/riscv/kvm/vcpu_onereg.c | 13 +++++--
4 files changed, 82 insertions(+), 37 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index 60017ceec9d..e2d5808169e 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -209,13 +209,13 @@ struct kvm_vcpu_arch {
/*
* VCPU interrupts
*
- * We have a lockless approach for tracking pending VCPU interrupts
- * implemented using atomic bitops. The irqs_pending bitmap represent
- * pending interrupts whereas irqs_pending_mask represent bits changed
- * in irqs_pending. Our approach is modeled around multiple producer
- * and single consumer problem where the consumer is the VCPU itself.
+ * The irqs_pending bitmap represents pending interrupts whereas
+ * irqs_pending_mask represents bits changed in irqs_pending. Updates
+ * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
+ * drop a newly injected interrupt while syncing guest-visible HVIP.
*/
#define KVM_RISCV_VCPU_NR_IRQS 64
+ raw_spinlock_t irqs_pending_lock;
DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index bafb009c5ce..225e4aed81a 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -53,12 +53,15 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long mask, val;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (!kvm_riscv_aia_available())
return;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
+ mask = vcpu->arch.irqs_pending_mask[1];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ val = vcpu->arch.irqs_pending[1] & mask;
csr->hviph &= ~mask;
csr->hviph |= val;
@@ -69,6 +72,8 @@ void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (kvm_riscv_aia_available())
csr->vsieh = ncsr_read(CSR_VSIEH);
}
@@ -78,11 +83,13 @@ bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
{
unsigned long seip;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (!kvm_riscv_aia_available())
return false;
#ifdef CONFIG_32BIT
- if (READ_ONCE(vcpu->arch.irqs_pending[1]) &
+ if (vcpu->arch.irqs_pending[1] &
(vcpu->arch.aia_context.guest_csr.vsieh & upper_32_bits(mask)))
return true;
#endif
@@ -207,6 +214,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
+#ifdef CONFIG_32BIT
+ unsigned long flags;
+#endif
if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
return -ENOENT;
@@ -216,11 +226,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
reg_num = array_index_nospec(reg_num, regs_max);
if (kvm_riscv_aia_available()) {
- ((unsigned long *)csr)[reg_num] = val;
-
#ifdef CONFIG_32BIT
- if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = val;
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
+ flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = val;
+ }
+#else
+ ((unsigned long *)csr)[reg_num] = val;
#endif
}
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index cf6e231e76e..7d8d20839d4 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
{
+ unsigned long flags;
bool loaded;
/**
@@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
kvm_riscv_vcpu_aia_reset(vcpu);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_riscv_vcpu_pmu_reset(vcpu);
@@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
/* Setup VCPU hfence queue */
spin_lock_init(&vcpu->arch.hfence_lock);
+ raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
spin_lock_init(&vcpu->arch.reset_state.lock);
@@ -352,10 +356,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long mask, val;
+ unsigned long flags;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ mask = vcpu->arch.irqs_pending_mask[0];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ val = vcpu->arch.irqs_pending[0] & mask;
csr->hvip &= ~mask;
csr->hvip |= val;
@@ -363,11 +370,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
/* Flush AIA high interrupts */
kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
}
void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
{
unsigned long hvip;
+ unsigned long flags;
struct kvm_vcpu_arch *v = &vcpu->arch;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
@@ -376,27 +385,29 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
/* Sync-up HVIP.VSSIP bit changes does by Guest */
hvip = ncsr_read(CSR_HVIP);
+ raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
if (hvip & (1UL << IRQ_VS_SOFT)) {
- if (!test_and_set_bit(IRQ_VS_SOFT,
- v->irqs_pending_mask))
- set_bit(IRQ_VS_SOFT, v->irqs_pending);
+ if (!__test_and_set_bit(IRQ_VS_SOFT,
+ v->irqs_pending_mask))
+ __set_bit(IRQ_VS_SOFT, v->irqs_pending);
} else {
- if (!test_and_set_bit(IRQ_VS_SOFT,
- v->irqs_pending_mask))
- clear_bit(IRQ_VS_SOFT, v->irqs_pending);
+ if (!__test_and_set_bit(IRQ_VS_SOFT,
+ v->irqs_pending_mask))
+ __clear_bit(IRQ_VS_SOFT, v->irqs_pending);
}
}
/* Sync up the HVIP.LCOFIP bit changes (only clear) by the guest */
if ((csr->hvip ^ hvip) & (1UL << IRQ_PMU_OVF)) {
if (!(hvip & (1UL << IRQ_PMU_OVF)) &&
- !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
- clear_bit(IRQ_PMU_OVF, v->irqs_pending);
+ !__test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
+ __clear_bit(IRQ_PMU_OVF, v->irqs_pending);
}
/* Sync-up AIA high interrupts */
kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
+ raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
/* Sync-up timer CSRs */
kvm_riscv_vcpu_timer_sync(vcpu);
@@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, and external
* interrupts when irq is one of the local interrupts
@@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
- set_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
- set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ __set_bit(irq, vcpu->arch.irqs_pending);
+ __set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_vcpu_kick(vcpu);
@@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, counter overflow and external
* interrupts when irq is one of the local interrupts
@@ -439,26 +455,33 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
- clear_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
- set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ __clear_bit(irq, vcpu->arch.irqs_pending);
+ __set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
return 0;
}
bool kvm_riscv_vcpu_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
{
+ unsigned long flags;
unsigned long ie;
+ bool ret;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
ie = ((vcpu->arch.guest_csr.vsie & VSIP_VALID_MASK)
<< VSIP_TO_HVIP_SHIFT) & (unsigned long)mask;
ie |= vcpu->arch.guest_csr.vsie & ~IRQ_LOCAL_MASK &
(unsigned long)mask;
- if (READ_ONCE(vcpu->arch.irqs_pending[0]) & ie)
- return true;
+ ret = vcpu->arch.irqs_pending[0] & ie;
/* Check AIA high interrupts */
- return kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
+ if (!ret)
+ ret = kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
+
+ return ret;
}
void __kvm_riscv_vcpu_power_off(struct kvm_vcpu *vcpu)
diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
index bb920e8923c..cba3682944b 100644
--- a/arch/riscv/kvm/vcpu_onereg.c
+++ b/arch/riscv/kvm/vcpu_onereg.c
@@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
+ unsigned long flags;
if (reg_num >= regs_max)
return -ENOENT;
@@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
reg_val <<= VSIP_TO_HVIP_SHIFT;
}
- ((unsigned long *)csr)[reg_num] = reg_val;
-
- if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ }
return 0;
}
--
2.54.0
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [PATCH v3] RISC-V: KVM: Serialize virtual interrupt pending state updates
2026-07-13 6:48 ` Xie Bo
(?)
(?)
@ 2026-07-13 7:06 ` sashiko-bot
-1 siblings, 0 replies; 26+ messages in thread
From: sashiko-bot @ 2026-07-13 7:06 UTC (permalink / raw)
To: Xie Bo; +Cc: kvm
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [High] Acquiring a sleepable rwlock (`imsic->vsfile_lock`) while holding a raw spinlock (`vcpu->arch.irqs_pending_lock`) causes a PREEMPT_RT sleep-in-atomic BUG, PROVE_RAW_LOCK_NESTING splats, and introduces an ABBA deadlock.
Pre-existing issues:
- [High] Desynchronization between `irqs_pending` and `hvip` when user-space writes the `sip` CSR, potentially causing infinite WFI wakeup loops.
--
--- Patch [1]: [PATCH v3] RISC-V: KVM: Serialize virtual interrupt pending state updates ---
commit 84ae8e6240f7f5635c2560729711fc1ceb24234d
Author: Xie Bo <xb@ultrarisc.com>
RISC-V: KVM: Serialize virtual interrupt pending state updates
This commit replaces the lockless atomic operations for the irqs_pending
and irqs_pending_mask bitmaps with a per-vCPU raw spinlock. This groups
the updates into a single state transition to avoid losing interrupts.
> diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> index cf6e231e76e25..7d8d20839d46e 100644
> --- a/arch/riscv/kvm/vcpu.c
> +++ b/arch/riscv/kvm/vcpu.c
[ ... ]
> @@ -455,26 +469,33 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
[ ... ]
> bool kvm_riscv_vcpu_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
> {
> + unsigned long flags;
> unsigned long ie;
> + bool ret;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> ie = ((vcpu->arch.guest_csr.vsie & VSIP_VALID_MASK)
> << VSIP_TO_HVIP_SHIFT) & (unsigned long)mask;
> ie |= vcpu->arch.guest_csr.vsie & ~IRQ_LOCAL_MASK &
> (unsigned long)mask;
> - if (READ_ONCE(vcpu->arch.irqs_pending[0]) & ie)
> - return true;
> + ret = vcpu->arch.irqs_pending[0] & ie;
>
> /* Check AIA high interrupts */
> - return kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
> + if (!ret)
> + ret = kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> +
> + return ret;
> }
[Severity: High]
Does holding the raw irqs_pending_lock across kvm_riscv_vcpu_aia_has_interrupts()
cause a lock nesting issue and potential ABBA deadlock?
If kvm_riscv_vcpu_aia_has_interrupts() falls through to
kvm_riscv_vcpu_aia_imsic_has_interrupt(), that function acquires
imsic->vsfile_lock (an rwlock_t, which is sleepable on PREEMPT_RT).
Acquiring a sleepable lock while holding a raw spinlock triggers a
sleep-in-atomic bug and PROVE_RAW_LOCK_NESTING warnings.
Additionally, can this sequence deadlock against concurrent userspace
ioctl calls?
For example, a KVM_SET_DEVICE_ATTR call on the IMSIC triggers
kvm_riscv_aia_imsic_rw_attr(), which acquires imsic->vsfile_lock first:
kvm_riscv_aia_imsic_rw_attr()
read_lock_irqsave(&imsic->vsfile_lock, flags);
...
imsic_swfile_extirq_update(vcpu);
kvm_riscv_vcpu_set_interrupt();
raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
If a vCPU thread running kvm_riscv_vcpu_has_interrupts() takes
irqs_pending_lock first, wouldn't these paths deadlock?
> diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
> index bb920e8923c93..cba3682944b69 100644
> --- a/arch/riscv/kvm/vcpu_onereg.c
> +++ b/arch/riscv/kvm/vcpu_onereg.c
[ ... ]
> @@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
[ ... ]
> - if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + }
[Severity: High]
This is a pre-existing issue, but does this clear of irqs_pending_mask leave
stale bits in irqs_pending?
If userspace writes 0 to the sip CSR, this code clears irqs_pending_mask[0]
and updates csr->hvip, but it does not clear any previously injected
interrupts in irqs_pending[0].
Because kvm_riscv_vcpu_has_interrupts() checks irqs_pending directly, can
this cause the vCPU to endlessly wake from WFI, finding no hardware interrupt
in HVIP, and immediately sleeping again?
The same situation appears to exist for the siph CSR update in
kvm_riscv_vcpu_aia_set_csr().
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260713064815.1276212-2-xb@ultrarisc.com?part=1
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
2026-07-13 4:43 ` Anup Patel
(?)
@ 2026-07-13 7:33 ` Xie Bo
-1 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-07-13 7:33 UTC (permalink / raw)
To: Anup Patel
Cc: Xie Bo, Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable
Hi Anup,
After sending v3, I found that kvm_riscv_vcpu_has_interrupts() held
irqs_pending_lock while the AIA helper could acquire the IMSIC
vsfile_lock. Since vsfile_lock is sleepable on PREEMPT_RT, this creates
a sleep-in-atomic issue. IMSIC update paths can also acquire the locks
in the reverse order, creating an ABBA lock ordering cycle.
I fixed this in v4 by splitting the AIA high pending bitmap check from
the IMSIC VS-file check. The bitmap check remains protected by
irqs_pending_lock, while the IMSIC check runs after releasing it.
The v4 patch is still based on Linux 7.2-rc3. Both RV64 and RV32 KVM
builds pass, and checkpatch reports no issues.
The v4 patch follows this reply.
Regards,
Xie Bo
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
@ 2026-07-13 7:33 ` Xie Bo
0 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-07-13 7:33 UTC (permalink / raw)
To: Anup Patel
Cc: Xie Bo, Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable
Hi Anup,
After sending v3, I found that kvm_riscv_vcpu_has_interrupts() held
irqs_pending_lock while the AIA helper could acquire the IMSIC
vsfile_lock. Since vsfile_lock is sleepable on PREEMPT_RT, this creates
a sleep-in-atomic issue. IMSIC update paths can also acquire the locks
in the reverse order, creating an ABBA lock ordering cycle.
I fixed this in v4 by splitting the AIA high pending bitmap check from
the IMSIC VS-file check. The bitmap check remains protected by
irqs_pending_lock, while the IMSIC check runs after releasing it.
The v4 patch is still based on Linux 7.2-rc3. Both RV64 and RV32 KVM
builds pass, and checkpatch reports no issues.
The v4 patch follows this reply.
Regards,
Xie Bo
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync
@ 2026-07-13 7:33 ` Xie Bo
0 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-07-13 7:33 UTC (permalink / raw)
To: Anup Patel
Cc: Xie Bo, Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable
Hi Anup,
After sending v3, I found that kvm_riscv_vcpu_has_interrupts() held
irqs_pending_lock while the AIA helper could acquire the IMSIC
vsfile_lock. Since vsfile_lock is sleepable on PREEMPT_RT, this creates
a sleep-in-atomic issue. IMSIC update paths can also acquire the locks
in the reverse order, creating an ABBA lock ordering cycle.
I fixed this in v4 by splitting the AIA high pending bitmap check from
the IMSIC VS-file check. The bitmap check remains protected by
irqs_pending_lock, while the IMSIC check runs after releasing it.
The v4 patch is still based on Linux 7.2-rc3. Both RV64 and RV32 KVM
builds pass, and checkpatch reports no issues.
The v4 patch follows this reply.
Regards,
Xie Bo
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PATCH v4] RISC-V: KVM: Serialize virtual interrupt pending state updates
2026-07-13 7:33 ` Xie Bo
(?)
@ 2026-07-13 7:33 ` Xie Bo
-1 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-07-13 7:33 UTC (permalink / raw)
To: Anup Patel
Cc: Xie Bo, Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable
RISC-V KVM tracks guest interrupt state with two bitmaps:
- irqs_pending: interrupts that should be visible to the guest
- irqs_pending_mask: interrupts whose pending state changed
The current code updates those bitmaps with independent atomic bitops
and assumes a multiple-producer, single-consumer protocol. That model
does not actually hold.
kvm_riscv_vcpu_sync_interrupts() is not a pure consumer. When the guest
changes guest-visible HVIP state, sync_interrupts() writes both
irqs_pending and irqs_pending_mask to reflect the new guest state back
into KVM state. As a result, irqs_pending and irqs_pending_mask form a
single logical state transition, but they are not updated atomically as
a pair.
This allows a race where a newly injected interrupt is lost. For
example:
CPU0 CPU1
---- ----
kvm_riscv_vcpu_set_interrupt(VS_SOFT)
set_bit(VS_SOFT, irqs_pending)
kvm_riscv_vcpu_sync_interrupts()
sees guest-cleared HVIP.VSSIP
sets irqs_pending_mask
clears irqs_pending
set_bit(VS_SOFT, irqs_pending_mask)
kvm_vcpu_kick()
After that interleaving, a later flush can update HVIP without VSSIP
even though a new virtual interrupt was injected. In practice, the
guest can remain blocked in WFI with work pending.
The same pending/mask protocol is shared by VS soft interrupts, PMU
overflow delivery, and AIA high interrupt synchronization, so the race
is not limited to one interrupt source.
Fix this by serializing all updates to irqs_pending and
irqs_pending_mask with a per-vCPU raw spinlock. This keeps the pending
bit and the dirty mask as one state transition across:
- set/unset interrupt
- guest HVIP sync
- interrupt flush to guest CSR state
- vCPU reset
- AIA CSR writes that clear dirty state
Use non-atomic bitmap operations while holding the lock. Hold the lock
across the AIA sync, flush, and pending checks as well, so both bitmap
words share the same serialization domain.
This intentionally replaces the existing lockless protocol instead of
trying to repair it with additional barriers. The problem is not memory
ordering on a single field; it is that two separate bitmaps encode one
shared state machine while both producers and sync paths can modify
them. A per-vCPU raw spinlock keeps the fix small, local, and suitable
for backporting.
Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
Cc: stable@vger.kernel.org
Signed-off-by: Xie Bo <xb@ultrarisc.com>
---
Changes in v4:
- Split the AIA pending bitmap check from the IMSIC VS-file check.
- Release irqs_pending_lock before the IMSIC check takes vsfile_lock,
avoiding sleep-in-atomic on PREEMPT_RT and an ABBA lock ordering cycle.
Changes in v3:
- Rebase onto Linux 7.2-rc3.
- Use non-atomic bitmap operations under irqs_pending_lock.
- Hold irqs_pending_lock across the AIA sync/flush helpers and add
lockdep assertions for their locking contract.
- Protect kvm_riscv_vcpu_has_interrupts() with irqs_pending_lock.
Changes in v2:
- Expand the race description and user-visible failure mode.
arch/riscv/include/asm/kvm_aia.h | 2 +
arch/riscv/include/asm/kvm_host.h | 10 ++---
arch/riscv/kvm/aia.c | 45 +++++++++++++++-----
arch/riscv/kvm/vcpu.c | 69 +++++++++++++++++++++----------
arch/riscv/kvm/vcpu_onereg.c | 13 ++++--
5 files changed, 99 insertions(+), 40 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_aia.h
index c67ec5ac0a1..1fe146675e2 100644
--- a/arch/riscv/include/asm/kvm_aia.h
+++ b/arch/riscv/include/asm/kvm_aia.h
@@ -124,6 +124,8 @@ static inline void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
{
}
#endif
+bool kvm_riscv_vcpu_aia_has_pending_interrupts(struct kvm_vcpu *vcpu,
+ u64 mask);
bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask);
void kvm_riscv_vcpu_aia_update_hvip(struct kvm_vcpu *vcpu);
diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index 60017ceec9d..e2d5808169e 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -209,13 +209,13 @@ struct kvm_vcpu_arch {
/*
* VCPU interrupts
*
- * We have a lockless approach for tracking pending VCPU interrupts
- * implemented using atomic bitops. The irqs_pending bitmap represent
- * pending interrupts whereas irqs_pending_mask represent bits changed
- * in irqs_pending. Our approach is modeled around multiple producer
- * and single consumer problem where the consumer is the VCPU itself.
+ * The irqs_pending bitmap represents pending interrupts whereas
+ * irqs_pending_mask represents bits changed in irqs_pending. Updates
+ * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
+ * drop a newly injected interrupt while syncing guest-visible HVIP.
*/
#define KVM_RISCV_VCPU_NR_IRQS 64
+ raw_spinlock_t irqs_pending_lock;
DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index bafb009c5ce..001e83032f6 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -53,12 +53,15 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long mask, val;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (!kvm_riscv_aia_available())
return;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
+ mask = vcpu->arch.irqs_pending_mask[1];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ val = vcpu->arch.irqs_pending[1] & mask;
csr->hviph &= ~mask;
csr->hviph |= val;
@@ -69,23 +72,35 @@ void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (kvm_riscv_aia_available())
csr->vsieh = ncsr_read(CSR_VSIEH);
}
#endif
-bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
+bool kvm_riscv_vcpu_aia_has_pending_interrupts(struct kvm_vcpu *vcpu,
+ u64 mask)
{
- unsigned long seip;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
if (!kvm_riscv_aia_available())
return false;
#ifdef CONFIG_32BIT
- if (READ_ONCE(vcpu->arch.irqs_pending[1]) &
+ if (vcpu->arch.irqs_pending[1] &
(vcpu->arch.aia_context.guest_csr.vsieh & upper_32_bits(mask)))
return true;
#endif
+ return false;
+}
+
+bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
+{
+ unsigned long seip;
+
+ if (!kvm_riscv_aia_available())
+ return false;
seip = vcpu->arch.guest_csr.vsie;
seip &= (unsigned long)mask;
@@ -207,6 +222,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
+#ifdef CONFIG_32BIT
+ unsigned long flags;
+#endif
if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
return -ENOENT;
@@ -216,11 +234,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
reg_num = array_index_nospec(reg_num, regs_max);
if (kvm_riscv_aia_available()) {
- ((unsigned long *)csr)[reg_num] = val;
-
#ifdef CONFIG_32BIT
- if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = val;
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
+ flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = val;
+ }
+#else
+ ((unsigned long *)csr)[reg_num] = val;
#endif
}
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index cf6e231e76e..99c929ec0ac 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
{
+ unsigned long flags;
bool loaded;
/**
@@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
kvm_riscv_vcpu_aia_reset(vcpu);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_riscv_vcpu_pmu_reset(vcpu);
@@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
/* Setup VCPU hfence queue */
spin_lock_init(&vcpu->arch.hfence_lock);
+ raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
spin_lock_init(&vcpu->arch.reset_state.lock);
@@ -352,10 +356,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long mask, val;
+ unsigned long flags;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ mask = vcpu->arch.irqs_pending_mask[0];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ val = vcpu->arch.irqs_pending[0] & mask;
csr->hvip &= ~mask;
csr->hvip |= val;
@@ -363,11 +370,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
/* Flush AIA high interrupts */
kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
}
void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
{
unsigned long hvip;
+ unsigned long flags;
struct kvm_vcpu_arch *v = &vcpu->arch;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
@@ -376,27 +385,29 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
/* Sync-up HVIP.VSSIP bit changes does by Guest */
hvip = ncsr_read(CSR_HVIP);
+ raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
if (hvip & (1UL << IRQ_VS_SOFT)) {
- if (!test_and_set_bit(IRQ_VS_SOFT,
- v->irqs_pending_mask))
- set_bit(IRQ_VS_SOFT, v->irqs_pending);
+ if (!__test_and_set_bit(IRQ_VS_SOFT,
+ v->irqs_pending_mask))
+ __set_bit(IRQ_VS_SOFT, v->irqs_pending);
} else {
- if (!test_and_set_bit(IRQ_VS_SOFT,
- v->irqs_pending_mask))
- clear_bit(IRQ_VS_SOFT, v->irqs_pending);
+ if (!__test_and_set_bit(IRQ_VS_SOFT,
+ v->irqs_pending_mask))
+ __clear_bit(IRQ_VS_SOFT, v->irqs_pending);
}
}
/* Sync up the HVIP.LCOFIP bit changes (only clear) by the guest */
if ((csr->hvip ^ hvip) & (1UL << IRQ_PMU_OVF)) {
if (!(hvip & (1UL << IRQ_PMU_OVF)) &&
- !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
- clear_bit(IRQ_PMU_OVF, v->irqs_pending);
+ !__test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
+ __clear_bit(IRQ_PMU_OVF, v->irqs_pending);
}
/* Sync-up AIA high interrupts */
kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
+ raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
/* Sync-up timer CSRs */
kvm_riscv_vcpu_timer_sync(vcpu);
@@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, and external
* interrupts when irq is one of the local interrupts
@@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
- set_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
- set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ __set_bit(irq, vcpu->arch.irqs_pending);
+ __set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_vcpu_kick(vcpu);
@@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, counter overflow and external
* interrupts when irq is one of the local interrupts
@@ -439,26 +455,37 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
- clear_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
- set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ __clear_bit(irq, vcpu->arch.irqs_pending);
+ __set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
return 0;
}
bool kvm_riscv_vcpu_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
{
+ unsigned long flags;
unsigned long ie;
+ bool ret;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
ie = ((vcpu->arch.guest_csr.vsie & VSIP_VALID_MASK)
<< VSIP_TO_HVIP_SHIFT) & (unsigned long)mask;
ie |= vcpu->arch.guest_csr.vsie & ~IRQ_LOCAL_MASK &
(unsigned long)mask;
- if (READ_ONCE(vcpu->arch.irqs_pending[0]) & ie)
- return true;
+ ret = vcpu->arch.irqs_pending[0] & ie;
+
+ /* Check AIA high pending bitmap while holding irqs_pending_lock */
+ if (!ret)
+ ret = kvm_riscv_vcpu_aia_has_pending_interrupts(vcpu, mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
- /* Check AIA high interrupts */
- return kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
+ /* IMSIC interrupt check takes vsfile_lock, which can sleep on RT. */
+ if (!ret)
+ ret = kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
+
+ return ret;
}
void __kvm_riscv_vcpu_power_off(struct kvm_vcpu *vcpu)
diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
index bb920e8923c..cba3682944b 100644
--- a/arch/riscv/kvm/vcpu_onereg.c
+++ b/arch/riscv/kvm/vcpu_onereg.c
@@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
+ unsigned long flags;
if (reg_num >= regs_max)
return -ENOENT;
@@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
reg_val <<= VSIP_TO_HVIP_SHIFT;
}
- ((unsigned long *)csr)[reg_num] = reg_val;
-
- if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ }
return 0;
}
--
2.54.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH v4] RISC-V: KVM: Serialize virtual interrupt pending state updates
@ 2026-07-13 7:33 ` Xie Bo
0 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-07-13 7:33 UTC (permalink / raw)
To: Anup Patel
Cc: Xie Bo, Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable
RISC-V KVM tracks guest interrupt state with two bitmaps:
- irqs_pending: interrupts that should be visible to the guest
- irqs_pending_mask: interrupts whose pending state changed
The current code updates those bitmaps with independent atomic bitops
and assumes a multiple-producer, single-consumer protocol. That model
does not actually hold.
kvm_riscv_vcpu_sync_interrupts() is not a pure consumer. When the guest
changes guest-visible HVIP state, sync_interrupts() writes both
irqs_pending and irqs_pending_mask to reflect the new guest state back
into KVM state. As a result, irqs_pending and irqs_pending_mask form a
single logical state transition, but they are not updated atomically as
a pair.
This allows a race where a newly injected interrupt is lost. For
example:
CPU0 CPU1
---- ----
kvm_riscv_vcpu_set_interrupt(VS_SOFT)
set_bit(VS_SOFT, irqs_pending)
kvm_riscv_vcpu_sync_interrupts()
sees guest-cleared HVIP.VSSIP
sets irqs_pending_mask
clears irqs_pending
set_bit(VS_SOFT, irqs_pending_mask)
kvm_vcpu_kick()
After that interleaving, a later flush can update HVIP without VSSIP
even though a new virtual interrupt was injected. In practice, the
guest can remain blocked in WFI with work pending.
The same pending/mask protocol is shared by VS soft interrupts, PMU
overflow delivery, and AIA high interrupt synchronization, so the race
is not limited to one interrupt source.
Fix this by serializing all updates to irqs_pending and
irqs_pending_mask with a per-vCPU raw spinlock. This keeps the pending
bit and the dirty mask as one state transition across:
- set/unset interrupt
- guest HVIP sync
- interrupt flush to guest CSR state
- vCPU reset
- AIA CSR writes that clear dirty state
Use non-atomic bitmap operations while holding the lock. Hold the lock
across the AIA sync, flush, and pending checks as well, so both bitmap
words share the same serialization domain.
This intentionally replaces the existing lockless protocol instead of
trying to repair it with additional barriers. The problem is not memory
ordering on a single field; it is that two separate bitmaps encode one
shared state machine while both producers and sync paths can modify
them. A per-vCPU raw spinlock keeps the fix small, local, and suitable
for backporting.
Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
Cc: stable@vger.kernel.org
Signed-off-by: Xie Bo <xb@ultrarisc.com>
---
Changes in v4:
- Split the AIA pending bitmap check from the IMSIC VS-file check.
- Release irqs_pending_lock before the IMSIC check takes vsfile_lock,
avoiding sleep-in-atomic on PREEMPT_RT and an ABBA lock ordering cycle.
Changes in v3:
- Rebase onto Linux 7.2-rc3.
- Use non-atomic bitmap operations under irqs_pending_lock.
- Hold irqs_pending_lock across the AIA sync/flush helpers and add
lockdep assertions for their locking contract.
- Protect kvm_riscv_vcpu_has_interrupts() with irqs_pending_lock.
Changes in v2:
- Expand the race description and user-visible failure mode.
arch/riscv/include/asm/kvm_aia.h | 2 +
arch/riscv/include/asm/kvm_host.h | 10 ++---
arch/riscv/kvm/aia.c | 45 +++++++++++++++-----
arch/riscv/kvm/vcpu.c | 69 +++++++++++++++++++++----------
arch/riscv/kvm/vcpu_onereg.c | 13 ++++--
5 files changed, 99 insertions(+), 40 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_aia.h
index c67ec5ac0a1..1fe146675e2 100644
--- a/arch/riscv/include/asm/kvm_aia.h
+++ b/arch/riscv/include/asm/kvm_aia.h
@@ -124,6 +124,8 @@ static inline void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
{
}
#endif
+bool kvm_riscv_vcpu_aia_has_pending_interrupts(struct kvm_vcpu *vcpu,
+ u64 mask);
bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask);
void kvm_riscv_vcpu_aia_update_hvip(struct kvm_vcpu *vcpu);
diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index 60017ceec9d..e2d5808169e 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -209,13 +209,13 @@ struct kvm_vcpu_arch {
/*
* VCPU interrupts
*
- * We have a lockless approach for tracking pending VCPU interrupts
- * implemented using atomic bitops. The irqs_pending bitmap represent
- * pending interrupts whereas irqs_pending_mask represent bits changed
- * in irqs_pending. Our approach is modeled around multiple producer
- * and single consumer problem where the consumer is the VCPU itself.
+ * The irqs_pending bitmap represents pending interrupts whereas
+ * irqs_pending_mask represents bits changed in irqs_pending. Updates
+ * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
+ * drop a newly injected interrupt while syncing guest-visible HVIP.
*/
#define KVM_RISCV_VCPU_NR_IRQS 64
+ raw_spinlock_t irqs_pending_lock;
DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index bafb009c5ce..001e83032f6 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -53,12 +53,15 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long mask, val;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (!kvm_riscv_aia_available())
return;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
+ mask = vcpu->arch.irqs_pending_mask[1];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ val = vcpu->arch.irqs_pending[1] & mask;
csr->hviph &= ~mask;
csr->hviph |= val;
@@ -69,23 +72,35 @@ void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (kvm_riscv_aia_available())
csr->vsieh = ncsr_read(CSR_VSIEH);
}
#endif
-bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
+bool kvm_riscv_vcpu_aia_has_pending_interrupts(struct kvm_vcpu *vcpu,
+ u64 mask)
{
- unsigned long seip;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
if (!kvm_riscv_aia_available())
return false;
#ifdef CONFIG_32BIT
- if (READ_ONCE(vcpu->arch.irqs_pending[1]) &
+ if (vcpu->arch.irqs_pending[1] &
(vcpu->arch.aia_context.guest_csr.vsieh & upper_32_bits(mask)))
return true;
#endif
+ return false;
+}
+
+bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
+{
+ unsigned long seip;
+
+ if (!kvm_riscv_aia_available())
+ return false;
seip = vcpu->arch.guest_csr.vsie;
seip &= (unsigned long)mask;
@@ -207,6 +222,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
+#ifdef CONFIG_32BIT
+ unsigned long flags;
+#endif
if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
return -ENOENT;
@@ -216,11 +234,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
reg_num = array_index_nospec(reg_num, regs_max);
if (kvm_riscv_aia_available()) {
- ((unsigned long *)csr)[reg_num] = val;
-
#ifdef CONFIG_32BIT
- if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = val;
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
+ flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = val;
+ }
+#else
+ ((unsigned long *)csr)[reg_num] = val;
#endif
}
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index cf6e231e76e..99c929ec0ac 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
{
+ unsigned long flags;
bool loaded;
/**
@@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
kvm_riscv_vcpu_aia_reset(vcpu);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_riscv_vcpu_pmu_reset(vcpu);
@@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
/* Setup VCPU hfence queue */
spin_lock_init(&vcpu->arch.hfence_lock);
+ raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
spin_lock_init(&vcpu->arch.reset_state.lock);
@@ -352,10 +356,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long mask, val;
+ unsigned long flags;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ mask = vcpu->arch.irqs_pending_mask[0];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ val = vcpu->arch.irqs_pending[0] & mask;
csr->hvip &= ~mask;
csr->hvip |= val;
@@ -363,11 +370,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
/* Flush AIA high interrupts */
kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
}
void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
{
unsigned long hvip;
+ unsigned long flags;
struct kvm_vcpu_arch *v = &vcpu->arch;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
@@ -376,27 +385,29 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
/* Sync-up HVIP.VSSIP bit changes does by Guest */
hvip = ncsr_read(CSR_HVIP);
+ raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
if (hvip & (1UL << IRQ_VS_SOFT)) {
- if (!test_and_set_bit(IRQ_VS_SOFT,
- v->irqs_pending_mask))
- set_bit(IRQ_VS_SOFT, v->irqs_pending);
+ if (!__test_and_set_bit(IRQ_VS_SOFT,
+ v->irqs_pending_mask))
+ __set_bit(IRQ_VS_SOFT, v->irqs_pending);
} else {
- if (!test_and_set_bit(IRQ_VS_SOFT,
- v->irqs_pending_mask))
- clear_bit(IRQ_VS_SOFT, v->irqs_pending);
+ if (!__test_and_set_bit(IRQ_VS_SOFT,
+ v->irqs_pending_mask))
+ __clear_bit(IRQ_VS_SOFT, v->irqs_pending);
}
}
/* Sync up the HVIP.LCOFIP bit changes (only clear) by the guest */
if ((csr->hvip ^ hvip) & (1UL << IRQ_PMU_OVF)) {
if (!(hvip & (1UL << IRQ_PMU_OVF)) &&
- !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
- clear_bit(IRQ_PMU_OVF, v->irqs_pending);
+ !__test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
+ __clear_bit(IRQ_PMU_OVF, v->irqs_pending);
}
/* Sync-up AIA high interrupts */
kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
+ raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
/* Sync-up timer CSRs */
kvm_riscv_vcpu_timer_sync(vcpu);
@@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, and external
* interrupts when irq is one of the local interrupts
@@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
- set_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
- set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ __set_bit(irq, vcpu->arch.irqs_pending);
+ __set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_vcpu_kick(vcpu);
@@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, counter overflow and external
* interrupts when irq is one of the local interrupts
@@ -439,26 +455,37 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
- clear_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
- set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ __clear_bit(irq, vcpu->arch.irqs_pending);
+ __set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
return 0;
}
bool kvm_riscv_vcpu_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
{
+ unsigned long flags;
unsigned long ie;
+ bool ret;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
ie = ((vcpu->arch.guest_csr.vsie & VSIP_VALID_MASK)
<< VSIP_TO_HVIP_SHIFT) & (unsigned long)mask;
ie |= vcpu->arch.guest_csr.vsie & ~IRQ_LOCAL_MASK &
(unsigned long)mask;
- if (READ_ONCE(vcpu->arch.irqs_pending[0]) & ie)
- return true;
+ ret = vcpu->arch.irqs_pending[0] & ie;
+
+ /* Check AIA high pending bitmap while holding irqs_pending_lock */
+ if (!ret)
+ ret = kvm_riscv_vcpu_aia_has_pending_interrupts(vcpu, mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
- /* Check AIA high interrupts */
- return kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
+ /* IMSIC interrupt check takes vsfile_lock, which can sleep on RT. */
+ if (!ret)
+ ret = kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
+
+ return ret;
}
void __kvm_riscv_vcpu_power_off(struct kvm_vcpu *vcpu)
diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
index bb920e8923c..cba3682944b 100644
--- a/arch/riscv/kvm/vcpu_onereg.c
+++ b/arch/riscv/kvm/vcpu_onereg.c
@@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
+ unsigned long flags;
if (reg_num >= regs_max)
return -ENOENT;
@@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
reg_val <<= VSIP_TO_HVIP_SHIFT;
}
- ((unsigned long *)csr)[reg_num] = reg_val;
-
- if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ }
return 0;
}
--
2.54.0
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH v4] RISC-V: KVM: Serialize virtual interrupt pending state updates
@ 2026-07-13 7:33 ` Xie Bo
0 siblings, 0 replies; 26+ messages in thread
From: Xie Bo @ 2026-07-13 7:33 UTC (permalink / raw)
To: Anup Patel
Cc: Xie Bo, Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable
RISC-V KVM tracks guest interrupt state with two bitmaps:
- irqs_pending: interrupts that should be visible to the guest
- irqs_pending_mask: interrupts whose pending state changed
The current code updates those bitmaps with independent atomic bitops
and assumes a multiple-producer, single-consumer protocol. That model
does not actually hold.
kvm_riscv_vcpu_sync_interrupts() is not a pure consumer. When the guest
changes guest-visible HVIP state, sync_interrupts() writes both
irqs_pending and irqs_pending_mask to reflect the new guest state back
into KVM state. As a result, irqs_pending and irqs_pending_mask form a
single logical state transition, but they are not updated atomically as
a pair.
This allows a race where a newly injected interrupt is lost. For
example:
CPU0 CPU1
---- ----
kvm_riscv_vcpu_set_interrupt(VS_SOFT)
set_bit(VS_SOFT, irqs_pending)
kvm_riscv_vcpu_sync_interrupts()
sees guest-cleared HVIP.VSSIP
sets irqs_pending_mask
clears irqs_pending
set_bit(VS_SOFT, irqs_pending_mask)
kvm_vcpu_kick()
After that interleaving, a later flush can update HVIP without VSSIP
even though a new virtual interrupt was injected. In practice, the
guest can remain blocked in WFI with work pending.
The same pending/mask protocol is shared by VS soft interrupts, PMU
overflow delivery, and AIA high interrupt synchronization, so the race
is not limited to one interrupt source.
Fix this by serializing all updates to irqs_pending and
irqs_pending_mask with a per-vCPU raw spinlock. This keeps the pending
bit and the dirty mask as one state transition across:
- set/unset interrupt
- guest HVIP sync
- interrupt flush to guest CSR state
- vCPU reset
- AIA CSR writes that clear dirty state
Use non-atomic bitmap operations while holding the lock. Hold the lock
across the AIA sync, flush, and pending checks as well, so both bitmap
words share the same serialization domain.
This intentionally replaces the existing lockless protocol instead of
trying to repair it with additional barriers. The problem is not memory
ordering on a single field; it is that two separate bitmaps encode one
shared state machine while both producers and sync paths can modify
them. A per-vCPU raw spinlock keeps the fix small, local, and suitable
for backporting.
Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
Cc: stable@vger.kernel.org
Signed-off-by: Xie Bo <xb@ultrarisc.com>
---
Changes in v4:
- Split the AIA pending bitmap check from the IMSIC VS-file check.
- Release irqs_pending_lock before the IMSIC check takes vsfile_lock,
avoiding sleep-in-atomic on PREEMPT_RT and an ABBA lock ordering cycle.
Changes in v3:
- Rebase onto Linux 7.2-rc3.
- Use non-atomic bitmap operations under irqs_pending_lock.
- Hold irqs_pending_lock across the AIA sync/flush helpers and add
lockdep assertions for their locking contract.
- Protect kvm_riscv_vcpu_has_interrupts() with irqs_pending_lock.
Changes in v2:
- Expand the race description and user-visible failure mode.
arch/riscv/include/asm/kvm_aia.h | 2 +
arch/riscv/include/asm/kvm_host.h | 10 ++---
arch/riscv/kvm/aia.c | 45 +++++++++++++++-----
arch/riscv/kvm/vcpu.c | 69 +++++++++++++++++++++----------
arch/riscv/kvm/vcpu_onereg.c | 13 ++++--
5 files changed, 99 insertions(+), 40 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_aia.h
index c67ec5ac0a1..1fe146675e2 100644
--- a/arch/riscv/include/asm/kvm_aia.h
+++ b/arch/riscv/include/asm/kvm_aia.h
@@ -124,6 +124,8 @@ static inline void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
{
}
#endif
+bool kvm_riscv_vcpu_aia_has_pending_interrupts(struct kvm_vcpu *vcpu,
+ u64 mask);
bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask);
void kvm_riscv_vcpu_aia_update_hvip(struct kvm_vcpu *vcpu);
diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index 60017ceec9d..e2d5808169e 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -209,13 +209,13 @@ struct kvm_vcpu_arch {
/*
* VCPU interrupts
*
- * We have a lockless approach for tracking pending VCPU interrupts
- * implemented using atomic bitops. The irqs_pending bitmap represent
- * pending interrupts whereas irqs_pending_mask represent bits changed
- * in irqs_pending. Our approach is modeled around multiple producer
- * and single consumer problem where the consumer is the VCPU itself.
+ * The irqs_pending bitmap represents pending interrupts whereas
+ * irqs_pending_mask represents bits changed in irqs_pending. Updates
+ * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
+ * drop a newly injected interrupt while syncing guest-visible HVIP.
*/
#define KVM_RISCV_VCPU_NR_IRQS 64
+ raw_spinlock_t irqs_pending_lock;
DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
index bafb009c5ce..001e83032f6 100644
--- a/arch/riscv/kvm/aia.c
+++ b/arch/riscv/kvm/aia.c
@@ -53,12 +53,15 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long mask, val;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (!kvm_riscv_aia_available())
return;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
+ mask = vcpu->arch.irqs_pending_mask[1];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ val = vcpu->arch.irqs_pending[1] & mask;
csr->hviph &= ~mask;
csr->hviph |= val;
@@ -69,23 +72,35 @@ void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
+
if (kvm_riscv_aia_available())
csr->vsieh = ncsr_read(CSR_VSIEH);
}
#endif
-bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
+bool kvm_riscv_vcpu_aia_has_pending_interrupts(struct kvm_vcpu *vcpu,
+ u64 mask)
{
- unsigned long seip;
+ lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
if (!kvm_riscv_aia_available())
return false;
#ifdef CONFIG_32BIT
- if (READ_ONCE(vcpu->arch.irqs_pending[1]) &
+ if (vcpu->arch.irqs_pending[1] &
(vcpu->arch.aia_context.guest_csr.vsieh & upper_32_bits(mask)))
return true;
#endif
+ return false;
+}
+
+bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
+{
+ unsigned long seip;
+
+ if (!kvm_riscv_aia_available())
+ return false;
seip = vcpu->arch.guest_csr.vsie;
seip &= (unsigned long)mask;
@@ -207,6 +222,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
+#ifdef CONFIG_32BIT
+ unsigned long flags;
+#endif
if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
return -ENOENT;
@@ -216,11 +234,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
reg_num = array_index_nospec(reg_num, regs_max);
if (kvm_riscv_aia_available()) {
- ((unsigned long *)csr)[reg_num] = val;
-
#ifdef CONFIG_32BIT
- if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = val;
+ vcpu->arch.irqs_pending_mask[1] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
+ flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = val;
+ }
+#else
+ ((unsigned long *)csr)[reg_num] = val;
#endif
}
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index cf6e231e76e..99c929ec0ac 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
{
+ unsigned long flags;
bool loaded;
/**
@@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
kvm_riscv_vcpu_aia_reset(vcpu);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_riscv_vcpu_pmu_reset(vcpu);
@@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
/* Setup VCPU hfence queue */
spin_lock_init(&vcpu->arch.hfence_lock);
+ raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
spin_lock_init(&vcpu->arch.reset_state.lock);
@@ -352,10 +356,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long mask, val;
+ unsigned long flags;
- if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
- mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
- val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ mask = vcpu->arch.irqs_pending_mask[0];
+ if (mask) {
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ val = vcpu->arch.irqs_pending[0] & mask;
csr->hvip &= ~mask;
csr->hvip |= val;
@@ -363,11 +370,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
/* Flush AIA high interrupts */
kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
}
void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
{
unsigned long hvip;
+ unsigned long flags;
struct kvm_vcpu_arch *v = &vcpu->arch;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
@@ -376,27 +385,29 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
/* Sync-up HVIP.VSSIP bit changes does by Guest */
hvip = ncsr_read(CSR_HVIP);
+ raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
if (hvip & (1UL << IRQ_VS_SOFT)) {
- if (!test_and_set_bit(IRQ_VS_SOFT,
- v->irqs_pending_mask))
- set_bit(IRQ_VS_SOFT, v->irqs_pending);
+ if (!__test_and_set_bit(IRQ_VS_SOFT,
+ v->irqs_pending_mask))
+ __set_bit(IRQ_VS_SOFT, v->irqs_pending);
} else {
- if (!test_and_set_bit(IRQ_VS_SOFT,
- v->irqs_pending_mask))
- clear_bit(IRQ_VS_SOFT, v->irqs_pending);
+ if (!__test_and_set_bit(IRQ_VS_SOFT,
+ v->irqs_pending_mask))
+ __clear_bit(IRQ_VS_SOFT, v->irqs_pending);
}
}
/* Sync up the HVIP.LCOFIP bit changes (only clear) by the guest */
if ((csr->hvip ^ hvip) & (1UL << IRQ_PMU_OVF)) {
if (!(hvip & (1UL << IRQ_PMU_OVF)) &&
- !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
- clear_bit(IRQ_PMU_OVF, v->irqs_pending);
+ !__test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
+ __clear_bit(IRQ_PMU_OVF, v->irqs_pending);
}
/* Sync-up AIA high interrupts */
kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
+ raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
/* Sync-up timer CSRs */
kvm_riscv_vcpu_timer_sync(vcpu);
@@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, and external
* interrupts when irq is one of the local interrupts
@@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
- set_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
- set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ __set_bit(irq, vcpu->arch.irqs_pending);
+ __set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
kvm_vcpu_kick(vcpu);
@@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
{
+ unsigned long flags;
+
/*
* We only allow VS-mode software, timer, counter overflow and external
* interrupts when irq is one of the local interrupts
@@ -439,26 +455,37 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
irq != IRQ_PMU_OVF)
return -EINVAL;
- clear_bit(irq, vcpu->arch.irqs_pending);
- smp_mb__before_atomic();
- set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ __clear_bit(irq, vcpu->arch.irqs_pending);
+ __set_bit(irq, vcpu->arch.irqs_pending_mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
return 0;
}
bool kvm_riscv_vcpu_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
{
+ unsigned long flags;
unsigned long ie;
+ bool ret;
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
ie = ((vcpu->arch.guest_csr.vsie & VSIP_VALID_MASK)
<< VSIP_TO_HVIP_SHIFT) & (unsigned long)mask;
ie |= vcpu->arch.guest_csr.vsie & ~IRQ_LOCAL_MASK &
(unsigned long)mask;
- if (READ_ONCE(vcpu->arch.irqs_pending[0]) & ie)
- return true;
+ ret = vcpu->arch.irqs_pending[0] & ie;
+
+ /* Check AIA high pending bitmap while holding irqs_pending_lock */
+ if (!ret)
+ ret = kvm_riscv_vcpu_aia_has_pending_interrupts(vcpu, mask);
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
- /* Check AIA high interrupts */
- return kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
+ /* IMSIC interrupt check takes vsfile_lock, which can sleep on RT. */
+ if (!ret)
+ ret = kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
+
+ return ret;
}
void __kvm_riscv_vcpu_power_off(struct kvm_vcpu *vcpu)
diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
index bb920e8923c..cba3682944b 100644
--- a/arch/riscv/kvm/vcpu_onereg.c
+++ b/arch/riscv/kvm/vcpu_onereg.c
@@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
{
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
+ unsigned long flags;
if (reg_num >= regs_max)
return -ENOENT;
@@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
reg_val <<= VSIP_TO_HVIP_SHIFT;
}
- ((unsigned long *)csr)[reg_num] = reg_val;
-
- if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
- WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
+ if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
+ raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ vcpu->arch.irqs_pending_mask[0] = 0;
+ raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
+ } else {
+ ((unsigned long *)csr)[reg_num] = reg_val;
+ }
return 0;
}
--
2.54.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [PATCH v4] RISC-V: KVM: Serialize virtual interrupt pending state updates
2026-07-13 7:33 ` Xie Bo
(?)
(?)
@ 2026-07-13 7:47 ` sashiko-bot
-1 siblings, 0 replies; 26+ messages in thread
From: sashiko-bot @ 2026-07-13 7:47 UTC (permalink / raw)
To: Xie Bo; +Cc: kvm
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Restoring guest `sip` (or `siph`) CSR via KVM_SET_ONE_REG drops pending interrupts from KVM's internal `irqs_pending` tracker, causing a vCPU stall if the guest executes WFI.
--
--- Patch [1]: [PATCH v4] RISC-V: KVM: Serialize virtual interrupt pending state updates ---
commit 445393d6bd22386cd7da7af1c9dd4c9d3be703a4
Author: Xie Bo <xb@ultrarisc.com>
RISC-V: KVM: Serialize virtual interrupt pending state updates
RISC-V KVM tracks guest interrupt state with two bitmaps: irqs_pending and
irqs_pending_mask. This commit serializes all updates to these bitmaps
with a per-vCPU raw spinlock to fix a race condition where injected
interrupts could be lost.
> diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
> index bb920e8923c93..cba3682944b69 100644
> --- a/arch/riscv/kvm/vcpu_onereg.c
> +++ b/arch/riscv/kvm/vcpu_onereg.c
[ ... ]
> @@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> reg_val <<= VSIP_TO_HVIP_SHIFT;
> }
>
> - ((unsigned long *)csr)[reg_num] = reg_val;
> -
> - if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + vcpu->arch.irqs_pending_mask[0] = 0;
[Severity: High]
This is a pre-existing issue, but does restoring the guest sip (or siph)
CSR via KVM_SET_ONE_REG drop pending interrupts from KVM's internal
irqs_pending tracker?
If userspace (e.g., QEMU) migrating a VM writes to KVM_REG_RISCV_CSR_REG(sip),
csr->hvip is updated and irqs_pending_mask[0] is cleared. However,
irqs_pending[0] is not updated with the restored state.
Later, if the guest executes WFI with interrupts locally disabled but
globally expected to wake it, kvm_riscv_vcpu_has_interrupts() will return
false because it solely checks irqs_pending:
arch/riscv/kvm/vcpu.c:kvm_riscv_vcpu_has_interrupts() {
...
ret = vcpu->arch.irqs_pending[0] & ie;
...
}
Could this cause a VM hang or stalled vCPU after live migration if it enters
WFI?
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + }
>
> return 0;
> }
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260713073346.1293408-2-xb@ultrarisc.com?part=1
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v4] RISC-V: KVM: Serialize virtual interrupt pending state updates
2026-07-13 7:33 ` Xie Bo
(?)
@ 2026-07-14 12:47 ` Anup Patel
-1 siblings, 0 replies; 26+ messages in thread
From: Anup Patel @ 2026-07-14 12:47 UTC (permalink / raw)
To: Xie Bo
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable
On Mon, Jul 13, 2026 at 1:04 PM Xie Bo <xb@ultrarisc.com> wrote:
>
> RISC-V KVM tracks guest interrupt state with two bitmaps:
>
> - irqs_pending: interrupts that should be visible to the guest
> - irqs_pending_mask: interrupts whose pending state changed
>
> The current code updates those bitmaps with independent atomic bitops
> and assumes a multiple-producer, single-consumer protocol. That model
> does not actually hold.
>
> kvm_riscv_vcpu_sync_interrupts() is not a pure consumer. When the guest
> changes guest-visible HVIP state, sync_interrupts() writes both
> irqs_pending and irqs_pending_mask to reflect the new guest state back
> into KVM state. As a result, irqs_pending and irqs_pending_mask form a
> single logical state transition, but they are not updated atomically as
> a pair.
>
> This allows a race where a newly injected interrupt is lost. For
> example:
>
> CPU0 CPU1
> ---- ----
> kvm_riscv_vcpu_set_interrupt(VS_SOFT)
> set_bit(VS_SOFT, irqs_pending)
> kvm_riscv_vcpu_sync_interrupts()
> sees guest-cleared HVIP.VSSIP
> sets irqs_pending_mask
> clears irqs_pending
s/clears irqs_pending/clear_bit(IRQ_VS_SOFT, irqs_pending)
> set_bit(VS_SOFT, irqs_pending_mask)
> kvm_vcpu_kick()
>
> After that interleaving, a later flush can update HVIP without VSSIP
> even though a new virtual interrupt was injected. In practice, the
> guest can remain blocked in WFI with work pending.
>
> The same pending/mask protocol is shared by VS soft interrupts, PMU
> overflow delivery, and AIA high interrupt synchronization, so the race
> is not limited to one interrupt source.
>
> Fix this by serializing all updates to irqs_pending and
> irqs_pending_mask with a per-vCPU raw spinlock. This keeps the pending
> bit and the dirty mask as one state transition across:
>
> - set/unset interrupt
> - guest HVIP sync
> - interrupt flush to guest CSR state
> - vCPU reset
> - AIA CSR writes that clear dirty state
>
> Use non-atomic bitmap operations while holding the lock. Hold the lock
> across the AIA sync, flush, and pending checks as well, so both bitmap
> words share the same serialization domain.
>
> This intentionally replaces the existing lockless protocol instead of
> trying to repair it with additional barriers. The problem is not memory
> ordering on a single field; it is that two separate bitmaps encode one
> shared state machine while both producers and sync paths can modify
> them. A per-vCPU raw spinlock keeps the fix small, local, and suitable
> for backporting.
>
> Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
> Cc: stable@vger.kernel.org
> Signed-off-by: Xie Bo <xb@ultrarisc.com>
> ---
> Changes in v4:
> - Split the AIA pending bitmap check from the IMSIC VS-file check.
> - Release irqs_pending_lock before the IMSIC check takes vsfile_lock,
> avoiding sleep-in-atomic on PREEMPT_RT and an ABBA lock ordering cycle.
>
> Changes in v3:
> - Rebase onto Linux 7.2-rc3.
> - Use non-atomic bitmap operations under irqs_pending_lock.
> - Hold irqs_pending_lock across the AIA sync/flush helpers and add
> lockdep assertions for their locking contract.
> - Protect kvm_riscv_vcpu_has_interrupts() with irqs_pending_lock.
>
> Changes in v2:
> - Expand the race description and user-visible failure mode.
>
> arch/riscv/include/asm/kvm_aia.h | 2 +
> arch/riscv/include/asm/kvm_host.h | 10 ++---
> arch/riscv/kvm/aia.c | 45 +++++++++++++++-----
> arch/riscv/kvm/vcpu.c | 69 +++++++++++++++++++++----------
> arch/riscv/kvm/vcpu_onereg.c | 13 ++++--
> 5 files changed, 99 insertions(+), 40 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_aia.h
> index c67ec5ac0a1..1fe146675e2 100644
> --- a/arch/riscv/include/asm/kvm_aia.h
> +++ b/arch/riscv/include/asm/kvm_aia.h
> @@ -124,6 +124,8 @@ static inline void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> }
> #endif
> +bool kvm_riscv_vcpu_aia_has_pending_interrupts(struct kvm_vcpu *vcpu,
> + u64 mask);
> bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask);
This split is looking very ugly and unmaintainable.
For now, let's not split kvm_riscv_vcpu_aia_has_interrupts() instead let
kvm_riscv_vcpu_aia_has_interrupts() explicity take irqs_pending_lock.
>
> void kvm_riscv_vcpu_aia_update_hvip(struct kvm_vcpu *vcpu);
> diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
> index 60017ceec9d..e2d5808169e 100644
> --- a/arch/riscv/include/asm/kvm_host.h
> +++ b/arch/riscv/include/asm/kvm_host.h
> @@ -209,13 +209,13 @@ struct kvm_vcpu_arch {
> /*
> * VCPU interrupts
> *
> - * We have a lockless approach for tracking pending VCPU interrupts
> - * implemented using atomic bitops. The irqs_pending bitmap represent
> - * pending interrupts whereas irqs_pending_mask represent bits changed
> - * in irqs_pending. Our approach is modeled around multiple producer
> - * and single consumer problem where the consumer is the VCPU itself.
> + * The irqs_pending bitmap represents pending interrupts whereas
> + * irqs_pending_mask represents bits changed in irqs_pending. Updates
> + * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
> + * drop a newly injected interrupt while syncing guest-visible HVIP.
> */
> #define KVM_RISCV_VCPU_NR_IRQS 64
> + raw_spinlock_t irqs_pending_lock;
> DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
>
> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index bafb009c5ce..001e83032f6 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
> @@ -53,12 +53,15 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long mask, val;
>
> + lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
> +
> if (!kvm_riscv_aia_available())
> return;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
> + mask = vcpu->arch.irqs_pending_mask[1];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + val = vcpu->arch.irqs_pending[1] & mask;
>
> csr->hviph &= ~mask;
> csr->hviph |= val;
> @@ -69,23 +72,35 @@ void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
>
> + lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
> +
> if (kvm_riscv_aia_available())
> csr->vsieh = ncsr_read(CSR_VSIEH);
> }
> #endif
>
> -bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
> +bool kvm_riscv_vcpu_aia_has_pending_interrupts(struct kvm_vcpu *vcpu,
> + u64 mask)
> {
> - unsigned long seip;
> + lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
>
> if (!kvm_riscv_aia_available())
> return false;
>
> #ifdef CONFIG_32BIT
> - if (READ_ONCE(vcpu->arch.irqs_pending[1]) &
> + if (vcpu->arch.irqs_pending[1] &
> (vcpu->arch.aia_context.guest_csr.vsieh & upper_32_bits(mask)))
> return true;
> #endif
> + return false;
> +}
> +
> +bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
> +{
> + unsigned long seip;
> +
> + if (!kvm_riscv_aia_available())
> + return false;
>
> seip = vcpu->arch.guest_csr.vsie;
> seip &= (unsigned long)mask;
> @@ -207,6 +222,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
> +#ifdef CONFIG_32BIT
> + unsigned long flags;
> +#endif
>
> if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
> return -ENOENT;
> @@ -216,11 +234,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> reg_num = array_index_nospec(reg_num, regs_max);
>
> if (kvm_riscv_aia_available()) {
> - ((unsigned long *)csr)[reg_num] = val;
> -
> #ifdef CONFIG_32BIT
> - if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = val;
The hvip in guest context is not under critical section so don't bring hviph
under critical section.
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
> + flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = val;
> + }
> +#else
> + ((unsigned long *)csr)[reg_num] = val;
> #endif
> }
>
> diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> index cf6e231e76e..99c929ec0ac 100644
> --- a/arch/riscv/kvm/vcpu.c
> +++ b/arch/riscv/kvm/vcpu.c
> @@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
>
> static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
> {
> + unsigned long flags;
> bool loaded;
>
> /**
> @@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
>
> kvm_riscv_vcpu_aia_reset(vcpu);
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_riscv_vcpu_pmu_reset(vcpu);
>
> @@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
>
> /* Setup VCPU hfence queue */
> spin_lock_init(&vcpu->arch.hfence_lock);
> + raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
>
> spin_lock_init(&vcpu->arch.reset_state.lock);
>
> @@ -352,10 +356,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[0];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + val = vcpu->arch.irqs_pending[0] & mask;
>
> csr->hvip &= ~mask;
> csr->hvip |= val;
> @@ -363,11 +370,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
>
> /* Flush AIA high interrupts */
> kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> }
>
> void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> unsigned long hvip;
> + unsigned long flags;
> struct kvm_vcpu_arch *v = &vcpu->arch;
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
>
> @@ -376,27 +385,29 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> /* Sync-up HVIP.VSSIP bit changes does by Guest */
> hvip = ncsr_read(CSR_HVIP);
> + raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
Add newline before and after raw_spin_lock_irqsave() to make it readable.
> if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
> if (hvip & (1UL << IRQ_VS_SOFT)) {
> - if (!test_and_set_bit(IRQ_VS_SOFT,
> - v->irqs_pending_mask))
> - set_bit(IRQ_VS_SOFT, v->irqs_pending);
> + if (!__test_and_set_bit(IRQ_VS_SOFT,
> + v->irqs_pending_mask))
> + __set_bit(IRQ_VS_SOFT, v->irqs_pending);
> } else {
> - if (!test_and_set_bit(IRQ_VS_SOFT,
> - v->irqs_pending_mask))
> - clear_bit(IRQ_VS_SOFT, v->irqs_pending);
> + if (!__test_and_set_bit(IRQ_VS_SOFT,
> + v->irqs_pending_mask))
> + __clear_bit(IRQ_VS_SOFT, v->irqs_pending);
> }
> }
>
> /* Sync up the HVIP.LCOFIP bit changes (only clear) by the guest */
> if ((csr->hvip ^ hvip) & (1UL << IRQ_PMU_OVF)) {
> if (!(hvip & (1UL << IRQ_PMU_OVF)) &&
> - !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
> - clear_bit(IRQ_PMU_OVF, v->irqs_pending);
> + !__test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
> + __clear_bit(IRQ_PMU_OVF, v->irqs_pending);
> }
>
> /* Sync-up AIA high interrupts */
> kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
Add newline here before raw_spin_unlock_irqrestore().
> + raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
>
> /* Sync-up timer CSRs */
> kvm_riscv_vcpu_timer_sync(vcpu);
> @@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, and external
> * interrupts when irq is one of the local interrupts
> @@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> - set_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> - set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + __set_bit(irq, vcpu->arch.irqs_pending);
> + __set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_vcpu_kick(vcpu);
>
> @@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
>
> int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, counter overflow and external
> * interrupts when irq is one of the local interrupts
> @@ -439,26 +455,37 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> - clear_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> - set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + __clear_bit(irq, vcpu->arch.irqs_pending);
> + __set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> return 0;
> }
>
> bool kvm_riscv_vcpu_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
> {
> + unsigned long flags;
> unsigned long ie;
> + bool ret;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> ie = ((vcpu->arch.guest_csr.vsie & VSIP_VALID_MASK)
> << VSIP_TO_HVIP_SHIFT) & (unsigned long)mask;
> ie |= vcpu->arch.guest_csr.vsie & ~IRQ_LOCAL_MASK &
> (unsigned long)mask;
> - if (READ_ONCE(vcpu->arch.irqs_pending[0]) & ie)
> - return true;
> + ret = vcpu->arch.irqs_pending[0] & ie;
> +
> + /* Check AIA high pending bitmap while holding irqs_pending_lock */
> + if (!ret)
> + ret = kvm_riscv_vcpu_aia_has_pending_interrupts(vcpu, mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> - /* Check AIA high interrupts */
> - return kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
> + /* IMSIC interrupt check takes vsfile_lock, which can sleep on RT. */
> + if (!ret)
> + ret = kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
> +
> + return ret;
> }
>
> void __kvm_riscv_vcpu_power_off(struct kvm_vcpu *vcpu)
> diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
> index bb920e8923c..cba3682944b 100644
> --- a/arch/riscv/kvm/vcpu_onereg.c
> +++ b/arch/riscv/kvm/vcpu_onereg.c
> @@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
> + unsigned long flags;
>
> if (reg_num >= regs_max)
> return -ENOENT;
> @@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> reg_val <<= VSIP_TO_HVIP_SHIFT;
> }
>
> - ((unsigned long *)csr)[reg_num] = reg_val;
> -
> - if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = reg_val;
Same comment as above.
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + }
>
> return 0;
> }
> --
> 2.54.0
>
Regards,
Anup
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v4] RISC-V: KVM: Serialize virtual interrupt pending state updates
@ 2026-07-14 12:47 ` Anup Patel
0 siblings, 0 replies; 26+ messages in thread
From: Anup Patel @ 2026-07-14 12:47 UTC (permalink / raw)
To: Xie Bo
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable
On Mon, Jul 13, 2026 at 1:04 PM Xie Bo <xb@ultrarisc.com> wrote:
>
> RISC-V KVM tracks guest interrupt state with two bitmaps:
>
> - irqs_pending: interrupts that should be visible to the guest
> - irqs_pending_mask: interrupts whose pending state changed
>
> The current code updates those bitmaps with independent atomic bitops
> and assumes a multiple-producer, single-consumer protocol. That model
> does not actually hold.
>
> kvm_riscv_vcpu_sync_interrupts() is not a pure consumer. When the guest
> changes guest-visible HVIP state, sync_interrupts() writes both
> irqs_pending and irqs_pending_mask to reflect the new guest state back
> into KVM state. As a result, irqs_pending and irqs_pending_mask form a
> single logical state transition, but they are not updated atomically as
> a pair.
>
> This allows a race where a newly injected interrupt is lost. For
> example:
>
> CPU0 CPU1
> ---- ----
> kvm_riscv_vcpu_set_interrupt(VS_SOFT)
> set_bit(VS_SOFT, irqs_pending)
> kvm_riscv_vcpu_sync_interrupts()
> sees guest-cleared HVIP.VSSIP
> sets irqs_pending_mask
> clears irqs_pending
s/clears irqs_pending/clear_bit(IRQ_VS_SOFT, irqs_pending)
> set_bit(VS_SOFT, irqs_pending_mask)
> kvm_vcpu_kick()
>
> After that interleaving, a later flush can update HVIP without VSSIP
> even though a new virtual interrupt was injected. In practice, the
> guest can remain blocked in WFI with work pending.
>
> The same pending/mask protocol is shared by VS soft interrupts, PMU
> overflow delivery, and AIA high interrupt synchronization, so the race
> is not limited to one interrupt source.
>
> Fix this by serializing all updates to irqs_pending and
> irqs_pending_mask with a per-vCPU raw spinlock. This keeps the pending
> bit and the dirty mask as one state transition across:
>
> - set/unset interrupt
> - guest HVIP sync
> - interrupt flush to guest CSR state
> - vCPU reset
> - AIA CSR writes that clear dirty state
>
> Use non-atomic bitmap operations while holding the lock. Hold the lock
> across the AIA sync, flush, and pending checks as well, so both bitmap
> words share the same serialization domain.
>
> This intentionally replaces the existing lockless protocol instead of
> trying to repair it with additional barriers. The problem is not memory
> ordering on a single field; it is that two separate bitmaps encode one
> shared state machine while both producers and sync paths can modify
> them. A per-vCPU raw spinlock keeps the fix small, local, and suitable
> for backporting.
>
> Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
> Cc: stable@vger.kernel.org
> Signed-off-by: Xie Bo <xb@ultrarisc.com>
> ---
> Changes in v4:
> - Split the AIA pending bitmap check from the IMSIC VS-file check.
> - Release irqs_pending_lock before the IMSIC check takes vsfile_lock,
> avoiding sleep-in-atomic on PREEMPT_RT and an ABBA lock ordering cycle.
>
> Changes in v3:
> - Rebase onto Linux 7.2-rc3.
> - Use non-atomic bitmap operations under irqs_pending_lock.
> - Hold irqs_pending_lock across the AIA sync/flush helpers and add
> lockdep assertions for their locking contract.
> - Protect kvm_riscv_vcpu_has_interrupts() with irqs_pending_lock.
>
> Changes in v2:
> - Expand the race description and user-visible failure mode.
>
> arch/riscv/include/asm/kvm_aia.h | 2 +
> arch/riscv/include/asm/kvm_host.h | 10 ++---
> arch/riscv/kvm/aia.c | 45 +++++++++++++++-----
> arch/riscv/kvm/vcpu.c | 69 +++++++++++++++++++++----------
> arch/riscv/kvm/vcpu_onereg.c | 13 ++++--
> 5 files changed, 99 insertions(+), 40 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_aia.h
> index c67ec5ac0a1..1fe146675e2 100644
> --- a/arch/riscv/include/asm/kvm_aia.h
> +++ b/arch/riscv/include/asm/kvm_aia.h
> @@ -124,6 +124,8 @@ static inline void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> }
> #endif
> +bool kvm_riscv_vcpu_aia_has_pending_interrupts(struct kvm_vcpu *vcpu,
> + u64 mask);
> bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask);
This split is looking very ugly and unmaintainable.
For now, let's not split kvm_riscv_vcpu_aia_has_interrupts() instead let
kvm_riscv_vcpu_aia_has_interrupts() explicity take irqs_pending_lock.
>
> void kvm_riscv_vcpu_aia_update_hvip(struct kvm_vcpu *vcpu);
> diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
> index 60017ceec9d..e2d5808169e 100644
> --- a/arch/riscv/include/asm/kvm_host.h
> +++ b/arch/riscv/include/asm/kvm_host.h
> @@ -209,13 +209,13 @@ struct kvm_vcpu_arch {
> /*
> * VCPU interrupts
> *
> - * We have a lockless approach for tracking pending VCPU interrupts
> - * implemented using atomic bitops. The irqs_pending bitmap represent
> - * pending interrupts whereas irqs_pending_mask represent bits changed
> - * in irqs_pending. Our approach is modeled around multiple producer
> - * and single consumer problem where the consumer is the VCPU itself.
> + * The irqs_pending bitmap represents pending interrupts whereas
> + * irqs_pending_mask represents bits changed in irqs_pending. Updates
> + * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
> + * drop a newly injected interrupt while syncing guest-visible HVIP.
> */
> #define KVM_RISCV_VCPU_NR_IRQS 64
> + raw_spinlock_t irqs_pending_lock;
> DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
>
> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index bafb009c5ce..001e83032f6 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
> @@ -53,12 +53,15 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long mask, val;
>
> + lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
> +
> if (!kvm_riscv_aia_available())
> return;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
> + mask = vcpu->arch.irqs_pending_mask[1];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + val = vcpu->arch.irqs_pending[1] & mask;
>
> csr->hviph &= ~mask;
> csr->hviph |= val;
> @@ -69,23 +72,35 @@ void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
>
> + lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
> +
> if (kvm_riscv_aia_available())
> csr->vsieh = ncsr_read(CSR_VSIEH);
> }
> #endif
>
> -bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
> +bool kvm_riscv_vcpu_aia_has_pending_interrupts(struct kvm_vcpu *vcpu,
> + u64 mask)
> {
> - unsigned long seip;
> + lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
>
> if (!kvm_riscv_aia_available())
> return false;
>
> #ifdef CONFIG_32BIT
> - if (READ_ONCE(vcpu->arch.irqs_pending[1]) &
> + if (vcpu->arch.irqs_pending[1] &
> (vcpu->arch.aia_context.guest_csr.vsieh & upper_32_bits(mask)))
> return true;
> #endif
> + return false;
> +}
> +
> +bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
> +{
> + unsigned long seip;
> +
> + if (!kvm_riscv_aia_available())
> + return false;
>
> seip = vcpu->arch.guest_csr.vsie;
> seip &= (unsigned long)mask;
> @@ -207,6 +222,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
> +#ifdef CONFIG_32BIT
> + unsigned long flags;
> +#endif
>
> if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
> return -ENOENT;
> @@ -216,11 +234,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> reg_num = array_index_nospec(reg_num, regs_max);
>
> if (kvm_riscv_aia_available()) {
> - ((unsigned long *)csr)[reg_num] = val;
> -
> #ifdef CONFIG_32BIT
> - if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = val;
The hvip in guest context is not under critical section so don't bring hviph
under critical section.
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
> + flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = val;
> + }
> +#else
> + ((unsigned long *)csr)[reg_num] = val;
> #endif
> }
>
> diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> index cf6e231e76e..99c929ec0ac 100644
> --- a/arch/riscv/kvm/vcpu.c
> +++ b/arch/riscv/kvm/vcpu.c
> @@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
>
> static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
> {
> + unsigned long flags;
> bool loaded;
>
> /**
> @@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
>
> kvm_riscv_vcpu_aia_reset(vcpu);
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_riscv_vcpu_pmu_reset(vcpu);
>
> @@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
>
> /* Setup VCPU hfence queue */
> spin_lock_init(&vcpu->arch.hfence_lock);
> + raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
>
> spin_lock_init(&vcpu->arch.reset_state.lock);
>
> @@ -352,10 +356,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[0];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + val = vcpu->arch.irqs_pending[0] & mask;
>
> csr->hvip &= ~mask;
> csr->hvip |= val;
> @@ -363,11 +370,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
>
> /* Flush AIA high interrupts */
> kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> }
>
> void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> unsigned long hvip;
> + unsigned long flags;
> struct kvm_vcpu_arch *v = &vcpu->arch;
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
>
> @@ -376,27 +385,29 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> /* Sync-up HVIP.VSSIP bit changes does by Guest */
> hvip = ncsr_read(CSR_HVIP);
> + raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
Add newline before and after raw_spin_lock_irqsave() to make it readable.
> if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
> if (hvip & (1UL << IRQ_VS_SOFT)) {
> - if (!test_and_set_bit(IRQ_VS_SOFT,
> - v->irqs_pending_mask))
> - set_bit(IRQ_VS_SOFT, v->irqs_pending);
> + if (!__test_and_set_bit(IRQ_VS_SOFT,
> + v->irqs_pending_mask))
> + __set_bit(IRQ_VS_SOFT, v->irqs_pending);
> } else {
> - if (!test_and_set_bit(IRQ_VS_SOFT,
> - v->irqs_pending_mask))
> - clear_bit(IRQ_VS_SOFT, v->irqs_pending);
> + if (!__test_and_set_bit(IRQ_VS_SOFT,
> + v->irqs_pending_mask))
> + __clear_bit(IRQ_VS_SOFT, v->irqs_pending);
> }
> }
>
> /* Sync up the HVIP.LCOFIP bit changes (only clear) by the guest */
> if ((csr->hvip ^ hvip) & (1UL << IRQ_PMU_OVF)) {
> if (!(hvip & (1UL << IRQ_PMU_OVF)) &&
> - !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
> - clear_bit(IRQ_PMU_OVF, v->irqs_pending);
> + !__test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
> + __clear_bit(IRQ_PMU_OVF, v->irqs_pending);
> }
>
> /* Sync-up AIA high interrupts */
> kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
Add newline here before raw_spin_unlock_irqrestore().
> + raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
>
> /* Sync-up timer CSRs */
> kvm_riscv_vcpu_timer_sync(vcpu);
> @@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, and external
> * interrupts when irq is one of the local interrupts
> @@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> - set_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> - set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + __set_bit(irq, vcpu->arch.irqs_pending);
> + __set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_vcpu_kick(vcpu);
>
> @@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
>
> int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, counter overflow and external
> * interrupts when irq is one of the local interrupts
> @@ -439,26 +455,37 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> - clear_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> - set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + __clear_bit(irq, vcpu->arch.irqs_pending);
> + __set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> return 0;
> }
>
> bool kvm_riscv_vcpu_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
> {
> + unsigned long flags;
> unsigned long ie;
> + bool ret;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> ie = ((vcpu->arch.guest_csr.vsie & VSIP_VALID_MASK)
> << VSIP_TO_HVIP_SHIFT) & (unsigned long)mask;
> ie |= vcpu->arch.guest_csr.vsie & ~IRQ_LOCAL_MASK &
> (unsigned long)mask;
> - if (READ_ONCE(vcpu->arch.irqs_pending[0]) & ie)
> - return true;
> + ret = vcpu->arch.irqs_pending[0] & ie;
> +
> + /* Check AIA high pending bitmap while holding irqs_pending_lock */
> + if (!ret)
> + ret = kvm_riscv_vcpu_aia_has_pending_interrupts(vcpu, mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> - /* Check AIA high interrupts */
> - return kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
> + /* IMSIC interrupt check takes vsfile_lock, which can sleep on RT. */
> + if (!ret)
> + ret = kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
> +
> + return ret;
> }
>
> void __kvm_riscv_vcpu_power_off(struct kvm_vcpu *vcpu)
> diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
> index bb920e8923c..cba3682944b 100644
> --- a/arch/riscv/kvm/vcpu_onereg.c
> +++ b/arch/riscv/kvm/vcpu_onereg.c
> @@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
> + unsigned long flags;
>
> if (reg_num >= regs_max)
> return -ENOENT;
> @@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> reg_val <<= VSIP_TO_HVIP_SHIFT;
> }
>
> - ((unsigned long *)csr)[reg_num] = reg_val;
> -
> - if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = reg_val;
Same comment as above.
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + }
>
> return 0;
> }
> --
> 2.54.0
>
Regards,
Anup
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v4] RISC-V: KVM: Serialize virtual interrupt pending state updates
@ 2026-07-14 12:47 ` Anup Patel
0 siblings, 0 replies; 26+ messages in thread
From: Anup Patel @ 2026-07-14 12:47 UTC (permalink / raw)
To: Xie Bo
Cc: Atish Patra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Alexandre Ghiti, Paolo Bonzini, Alexander Graf, kvm-riscv, kvm,
linux-riscv, linux-kernel, stable
On Mon, Jul 13, 2026 at 1:04 PM Xie Bo <xb@ultrarisc.com> wrote:
>
> RISC-V KVM tracks guest interrupt state with two bitmaps:
>
> - irqs_pending: interrupts that should be visible to the guest
> - irqs_pending_mask: interrupts whose pending state changed
>
> The current code updates those bitmaps with independent atomic bitops
> and assumes a multiple-producer, single-consumer protocol. That model
> does not actually hold.
>
> kvm_riscv_vcpu_sync_interrupts() is not a pure consumer. When the guest
> changes guest-visible HVIP state, sync_interrupts() writes both
> irqs_pending and irqs_pending_mask to reflect the new guest state back
> into KVM state. As a result, irqs_pending and irqs_pending_mask form a
> single logical state transition, but they are not updated atomically as
> a pair.
>
> This allows a race where a newly injected interrupt is lost. For
> example:
>
> CPU0 CPU1
> ---- ----
> kvm_riscv_vcpu_set_interrupt(VS_SOFT)
> set_bit(VS_SOFT, irqs_pending)
> kvm_riscv_vcpu_sync_interrupts()
> sees guest-cleared HVIP.VSSIP
> sets irqs_pending_mask
> clears irqs_pending
s/clears irqs_pending/clear_bit(IRQ_VS_SOFT, irqs_pending)
> set_bit(VS_SOFT, irqs_pending_mask)
> kvm_vcpu_kick()
>
> After that interleaving, a later flush can update HVIP without VSSIP
> even though a new virtual interrupt was injected. In practice, the
> guest can remain blocked in WFI with work pending.
>
> The same pending/mask protocol is shared by VS soft interrupts, PMU
> overflow delivery, and AIA high interrupt synchronization, so the race
> is not limited to one interrupt source.
>
> Fix this by serializing all updates to irqs_pending and
> irqs_pending_mask with a per-vCPU raw spinlock. This keeps the pending
> bit and the dirty mask as one state transition across:
>
> - set/unset interrupt
> - guest HVIP sync
> - interrupt flush to guest CSR state
> - vCPU reset
> - AIA CSR writes that clear dirty state
>
> Use non-atomic bitmap operations while holding the lock. Hold the lock
> across the AIA sync, flush, and pending checks as well, so both bitmap
> words share the same serialization domain.
>
> This intentionally replaces the existing lockless protocol instead of
> trying to repair it with additional barriers. The problem is not memory
> ordering on a single field; it is that two separate bitmaps encode one
> shared state machine while both producers and sync paths can modify
> them. A per-vCPU raw spinlock keeps the fix small, local, and suitable
> for backporting.
>
> Fixes: cce69aff689e ("RISC-V: KVM: Implement VCPU interrupts and requests handling")
> Cc: stable@vger.kernel.org
> Signed-off-by: Xie Bo <xb@ultrarisc.com>
> ---
> Changes in v4:
> - Split the AIA pending bitmap check from the IMSIC VS-file check.
> - Release irqs_pending_lock before the IMSIC check takes vsfile_lock,
> avoiding sleep-in-atomic on PREEMPT_RT and an ABBA lock ordering cycle.
>
> Changes in v3:
> - Rebase onto Linux 7.2-rc3.
> - Use non-atomic bitmap operations under irqs_pending_lock.
> - Hold irqs_pending_lock across the AIA sync/flush helpers and add
> lockdep assertions for their locking contract.
> - Protect kvm_riscv_vcpu_has_interrupts() with irqs_pending_lock.
>
> Changes in v2:
> - Expand the race description and user-visible failure mode.
>
> arch/riscv/include/asm/kvm_aia.h | 2 +
> arch/riscv/include/asm/kvm_host.h | 10 ++---
> arch/riscv/kvm/aia.c | 45 +++++++++++++++-----
> arch/riscv/kvm/vcpu.c | 69 +++++++++++++++++++++----------
> arch/riscv/kvm/vcpu_onereg.c | 13 ++++--
> 5 files changed, 99 insertions(+), 40 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_aia.h b/arch/riscv/include/asm/kvm_aia.h
> index c67ec5ac0a1..1fe146675e2 100644
> --- a/arch/riscv/include/asm/kvm_aia.h
> +++ b/arch/riscv/include/asm/kvm_aia.h
> @@ -124,6 +124,8 @@ static inline void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> }
> #endif
> +bool kvm_riscv_vcpu_aia_has_pending_interrupts(struct kvm_vcpu *vcpu,
> + u64 mask);
> bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask);
This split is looking very ugly and unmaintainable.
For now, let's not split kvm_riscv_vcpu_aia_has_interrupts() instead let
kvm_riscv_vcpu_aia_has_interrupts() explicity take irqs_pending_lock.
>
> void kvm_riscv_vcpu_aia_update_hvip(struct kvm_vcpu *vcpu);
> diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
> index 60017ceec9d..e2d5808169e 100644
> --- a/arch/riscv/include/asm/kvm_host.h
> +++ b/arch/riscv/include/asm/kvm_host.h
> @@ -209,13 +209,13 @@ struct kvm_vcpu_arch {
> /*
> * VCPU interrupts
> *
> - * We have a lockless approach for tracking pending VCPU interrupts
> - * implemented using atomic bitops. The irqs_pending bitmap represent
> - * pending interrupts whereas irqs_pending_mask represent bits changed
> - * in irqs_pending. Our approach is modeled around multiple producer
> - * and single consumer problem where the consumer is the VCPU itself.
> + * The irqs_pending bitmap represents pending interrupts whereas
> + * irqs_pending_mask represents bits changed in irqs_pending. Updates
> + * to these bitmaps are serialized so vcpu interrupt sync/flush cannot
> + * drop a newly injected interrupt while syncing guest-visible HVIP.
> */
> #define KVM_RISCV_VCPU_NR_IRQS 64
> + raw_spinlock_t irqs_pending_lock;
> DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
>
> diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c
> index bafb009c5ce..001e83032f6 100644
> --- a/arch/riscv/kvm/aia.c
> +++ b/arch/riscv/kvm/aia.c
> @@ -53,12 +53,15 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu)
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long mask, val;
>
> + lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
> +
> if (!kvm_riscv_aia_available())
> return;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask;
> + mask = vcpu->arch.irqs_pending_mask[1];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + val = vcpu->arch.irqs_pending[1] & mask;
>
> csr->hviph &= ~mask;
> csr->hviph |= val;
> @@ -69,23 +72,35 @@ void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
>
> + lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
> +
> if (kvm_riscv_aia_available())
> csr->vsieh = ncsr_read(CSR_VSIEH);
> }
> #endif
>
> -bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
> +bool kvm_riscv_vcpu_aia_has_pending_interrupts(struct kvm_vcpu *vcpu,
> + u64 mask)
> {
> - unsigned long seip;
> + lockdep_assert_held(&vcpu->arch.irqs_pending_lock);
>
> if (!kvm_riscv_aia_available())
> return false;
>
> #ifdef CONFIG_32BIT
> - if (READ_ONCE(vcpu->arch.irqs_pending[1]) &
> + if (vcpu->arch.irqs_pending[1] &
> (vcpu->arch.aia_context.guest_csr.vsieh & upper_32_bits(mask)))
> return true;
> #endif
> + return false;
> +}
> +
> +bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
> +{
> + unsigned long seip;
> +
> + if (!kvm_riscv_aia_available())
> + return false;
>
> seip = vcpu->arch.guest_csr.vsie;
> seip &= (unsigned long)mask;
> @@ -207,6 +222,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long);
> +#ifdef CONFIG_32BIT
> + unsigned long flags;
> +#endif
>
> if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA))
> return -ENOENT;
> @@ -216,11 +234,18 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu,
> reg_num = array_index_nospec(reg_num, regs_max);
>
> if (kvm_riscv_aia_available()) {
> - ((unsigned long *)csr)[reg_num] = val;
> -
> #ifdef CONFIG_32BIT
> - if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = val;
The hvip in guest context is not under critical section so don't bring hviph
under critical section.
> + vcpu->arch.irqs_pending_mask[1] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock,
> + flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = val;
> + }
> +#else
> + ((unsigned long *)csr)[reg_num] = val;
> #endif
> }
>
> diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> index cf6e231e76e..99c929ec0ac 100644
> --- a/arch/riscv/kvm/vcpu.c
> +++ b/arch/riscv/kvm/vcpu.c
> @@ -80,6 +80,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu,
>
> static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
> {
> + unsigned long flags;
> bool loaded;
>
> /**
> @@ -104,8 +105,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset)
>
> kvm_riscv_vcpu_aia_reset(vcpu);
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS);
> bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_riscv_vcpu_pmu_reset(vcpu);
>
> @@ -151,6 +154,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
>
> /* Setup VCPU hfence queue */
> spin_lock_init(&vcpu->arch.hfence_lock);
> + raw_spin_lock_init(&vcpu->arch.irqs_pending_lock);
>
> spin_lock_init(&vcpu->arch.reset_state.lock);
>
> @@ -352,10 +356,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long mask, val;
> + unsigned long flags;
>
> - if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) {
> - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0);
> - val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask;
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + mask = vcpu->arch.irqs_pending_mask[0];
> + if (mask) {
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + val = vcpu->arch.irqs_pending[0] & mask;
>
> csr->hvip &= ~mask;
> csr->hvip |= val;
> @@ -363,11 +370,13 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu)
>
> /* Flush AIA high interrupts */
> kvm_riscv_vcpu_aia_flush_interrupts(vcpu);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> }
>
> void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
> {
> unsigned long hvip;
> + unsigned long flags;
> struct kvm_vcpu_arch *v = &vcpu->arch;
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
>
> @@ -376,27 +385,29 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> /* Sync-up HVIP.VSSIP bit changes does by Guest */
> hvip = ncsr_read(CSR_HVIP);
> + raw_spin_lock_irqsave(&v->irqs_pending_lock, flags);
Add newline before and after raw_spin_lock_irqsave() to make it readable.
> if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) {
> if (hvip & (1UL << IRQ_VS_SOFT)) {
> - if (!test_and_set_bit(IRQ_VS_SOFT,
> - v->irqs_pending_mask))
> - set_bit(IRQ_VS_SOFT, v->irqs_pending);
> + if (!__test_and_set_bit(IRQ_VS_SOFT,
> + v->irqs_pending_mask))
> + __set_bit(IRQ_VS_SOFT, v->irqs_pending);
> } else {
> - if (!test_and_set_bit(IRQ_VS_SOFT,
> - v->irqs_pending_mask))
> - clear_bit(IRQ_VS_SOFT, v->irqs_pending);
> + if (!__test_and_set_bit(IRQ_VS_SOFT,
> + v->irqs_pending_mask))
> + __clear_bit(IRQ_VS_SOFT, v->irqs_pending);
> }
> }
>
> /* Sync up the HVIP.LCOFIP bit changes (only clear) by the guest */
> if ((csr->hvip ^ hvip) & (1UL << IRQ_PMU_OVF)) {
> if (!(hvip & (1UL << IRQ_PMU_OVF)) &&
> - !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
> - clear_bit(IRQ_PMU_OVF, v->irqs_pending);
> + !__test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask))
> + __clear_bit(IRQ_PMU_OVF, v->irqs_pending);
> }
>
> /* Sync-up AIA high interrupts */
> kvm_riscv_vcpu_aia_sync_interrupts(vcpu);
Add newline here before raw_spin_unlock_irqrestore().
> + raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags);
>
> /* Sync-up timer CSRs */
> kvm_riscv_vcpu_timer_sync(vcpu);
> @@ -404,6 +415,8 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu)
>
> int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, and external
> * interrupts when irq is one of the local interrupts
> @@ -416,9 +429,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> - set_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> - set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + __set_bit(irq, vcpu->arch.irqs_pending);
> + __set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> kvm_vcpu_kick(vcpu);
>
> @@ -427,6 +441,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
>
> int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> {
> + unsigned long flags;
> +
> /*
> * We only allow VS-mode software, timer, counter overflow and external
> * interrupts when irq is one of the local interrupts
> @@ -439,26 +455,37 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq)
> irq != IRQ_PMU_OVF)
> return -EINVAL;
>
> - clear_bit(irq, vcpu->arch.irqs_pending);
> - smp_mb__before_atomic();
> - set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + __clear_bit(irq, vcpu->arch.irqs_pending);
> + __set_bit(irq, vcpu->arch.irqs_pending_mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> return 0;
> }
>
> bool kvm_riscv_vcpu_has_interrupts(struct kvm_vcpu *vcpu, u64 mask)
> {
> + unsigned long flags;
> unsigned long ie;
> + bool ret;
>
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> ie = ((vcpu->arch.guest_csr.vsie & VSIP_VALID_MASK)
> << VSIP_TO_HVIP_SHIFT) & (unsigned long)mask;
> ie |= vcpu->arch.guest_csr.vsie & ~IRQ_LOCAL_MASK &
> (unsigned long)mask;
> - if (READ_ONCE(vcpu->arch.irqs_pending[0]) & ie)
> - return true;
> + ret = vcpu->arch.irqs_pending[0] & ie;
> +
> + /* Check AIA high pending bitmap while holding irqs_pending_lock */
> + if (!ret)
> + ret = kvm_riscv_vcpu_aia_has_pending_interrupts(vcpu, mask);
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
>
> - /* Check AIA high interrupts */
> - return kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
> + /* IMSIC interrupt check takes vsfile_lock, which can sleep on RT. */
> + if (!ret)
> + ret = kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask);
> +
> + return ret;
> }
>
> void __kvm_riscv_vcpu_power_off(struct kvm_vcpu *vcpu)
> diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
> index bb920e8923c..cba3682944b 100644
> --- a/arch/riscv/kvm/vcpu_onereg.c
> +++ b/arch/riscv/kvm/vcpu_onereg.c
> @@ -298,6 +298,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> {
> struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
> unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long);
> + unsigned long flags;
>
> if (reg_num >= regs_max)
> return -ENOENT;
> @@ -309,10 +310,14 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
> reg_val <<= VSIP_TO_HVIP_SHIFT;
> }
>
> - ((unsigned long *)csr)[reg_num] = reg_val;
> -
> - if (reg_num == KVM_REG_RISCV_CSR_REG(sip))
> - WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0);
> + if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) {
> + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags);
> + ((unsigned long *)csr)[reg_num] = reg_val;
Same comment as above.
> + vcpu->arch.irqs_pending_mask[0] = 0;
> + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags);
> + } else {
> + ((unsigned long *)csr)[reg_num] = reg_val;
> + }
>
> return 0;
> }
> --
> 2.54.0
>
Regards,
Anup
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2026-07-14 12:48 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-06-16 6:17 [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync Xie Bo
2026-06-16 6:17 ` Xie Bo
2026-06-16 6:17 ` Xie Bo
2026-07-07 8:27 ` 谢波
2026-07-07 8:27 ` 谢波
2026-07-07 8:27 ` 谢波
2026-07-13 4:43 ` Anup Patel
2026-07-13 4:43 ` Anup Patel
2026-07-13 4:43 ` Anup Patel
2026-07-13 6:48 ` Xie Bo
2026-07-13 6:48 ` Xie Bo
2026-07-13 6:48 ` Xie Bo
2026-07-13 6:48 ` [PATCH v3] RISC-V: KVM: Serialize virtual interrupt pending state updates Xie Bo
2026-07-13 6:48 ` Xie Bo
2026-07-13 6:48 ` Xie Bo
2026-07-13 7:06 ` sashiko-bot
2026-07-13 7:33 ` [PATCH] RISC-V: KVM: Fix lost virtual interrupts during IRQ sync Xie Bo
2026-07-13 7:33 ` Xie Bo
2026-07-13 7:33 ` Xie Bo
2026-07-13 7:33 ` [PATCH v4] RISC-V: KVM: Serialize virtual interrupt pending state updates Xie Bo
2026-07-13 7:33 ` Xie Bo
2026-07-13 7:33 ` Xie Bo
2026-07-13 7:47 ` sashiko-bot
2026-07-14 12:47 ` Anup Patel
2026-07-14 12:47 ` Anup Patel
2026-07-14 12:47 ` Anup Patel
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.