[PATCH 0/7] KVM: arm64: Fix cross-CPU vcpu pausing

[PATCH 0/7] KVM: arm64: Fix cross-CPU vcpu pausing

[Marc Zyngier]

This series is a follow-up on what I previously posted at [1], and has
now grown a few more fixes, and inherited an improvement from Oliver.
Some of the fixes have already been merged, so what we have here
serves a slightly different goal.

As we are simplifying (and fixing) a bunch of vgic behaviours, it
becomes necessary to pause one (or all) vcpu(s) and *reliably*
introspect the state of interrupts.

It appears that the current KVM behaviour is remarkably broken:

- OUTSIDE_GUEST_MODE is published too early, before we have resynced
  the state, making it unreliable

- IN_GUEST_MODE is published too late, once we have already placed
  volatile state on the CPU (interrupts, for example), and making the
  in-memory state stale (edge pending bits are a particular sore spot)

- self-generated requests are sometimes posted too late, after we have
  committed to entering the guest, and are therefore delayed

- the pausing mechanism itself is unreliable, as it has no requirement
  to be called whilst holding a global lock, and therefore is able to
  nest, while it is backed by a simple boolean

This series addresses these defects, and builds on top of it to:

- add pausing to a single vcpu in addition to the full VM pausing

- convert LPI disabling and the MOVALL command to vcpu pausing

Patches on top of -rc6, most of them being Cc stable without any
qualification (this has been broken for as long as I can remember).

[1] https://lore.kernel.org/r/20260929093548.3598547-1-maz@kernel.org

Marc Zyngier (6):
  KVM: arm64: nv: Move L2->L1 switch on interrupt early
  KVM: arm64: Move OUTSIDE_GUEST_MODE publication past context being
    saved
  KVM: arm64: Move IN_GUEST_MODE ahead of state being loaded
  KVM: arm64: Turn vcpu->arch.pause into a counter
  KVM: arm64: vgic: Stop the target vcpu when disabling LPIs
  KVM: arm64: vgic-its: Stop the VM when handling MOVALL

Oliver Upton (1):
  KVM: arm64: Add helpers to halt a vCPU

 arch/arm64/include/asm/kvm_host.h  |  9 ++--
 arch/arm64/kvm/arm.c               | 69 ++++++++++++++++++++----------
 arch/arm64/kvm/nested.c            |  8 ++++
 arch/arm64/kvm/vgic/vgic-its.c     | 11 +++++
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 10 +++++
 arch/arm64/kvm/vgic/vgic.c         |  8 +---
 6 files changed, 81 insertions(+), 34 deletions(-)

-- 
2.47.3

[PATCH 1/7] KVM: arm64: nv: Move L2->L1 switch on interrupt early

[Marc Zyngier]

NV request generation to switch from L2 to L1 on interrupt is buried
into the vgic flush code, while we should be able to deliver requests
as early as possible.

Similarly, MI generation happens *after* the need for a request has been
evaluated, which means we potentially delay the maintenance interrupt by
a full exit/entry cycle.

Move these two things to happen much earlier in the entry code, and make
sure the MI generation and PMU interrupt serve as input to the request
switching back from L2 to L1.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
---
 arch/arm64/kvm/arm.c       | 4 ++--
 arch/arm64/kvm/nested.c    | 8 ++++++++
 arch/arm64/kvm/vgic/vgic.c | 8 +-------
 3 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 0576c2022ef57..73706464d2653 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1314,11 +1314,11 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
 		 */
 		preempt_disable();
 
-		kvm_nested_flush_hwstate(vcpu);
-
 		if (kvm_vcpu_has_pmu(vcpu))
 			kvm_pmu_flush_hwstate(vcpu);
 
+		kvm_nested_flush_hwstate(vcpu);
+
 		local_irq_disable();
 
 		kvm_vgic_flush_hwstate(vcpu);
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index b191365d97cc2..2fac0f2678122 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -2046,6 +2046,9 @@ void check_nested_vcpu_requests(struct kvm_vcpu *vcpu)
  *
  * What follows is some slop to (1) react to exception routing / masking and (2)
  * preserve the pending SError state across translation regimes.
+ *
+ * Also generate the maintenance interrupt state, and finally check that if
+ * we need to take an interrupt targetting the guest hypervisor.
  */
 void kvm_nested_flush_hwstate(struct kvm_vcpu *vcpu)
 {
@@ -2054,6 +2057,11 @@ void kvm_nested_flush_hwstate(struct kvm_vcpu *vcpu)
 
 	if (unlikely(vcpu_test_and_clear_flag(vcpu, NESTED_SERROR_PENDING)))
 		kvm_inject_serror_esr(vcpu, vcpu_get_vsesr(vcpu));
+
+	vgic_v3_nested_update_mi(vcpu);
+
+	if (vgic_state_is_nested(vcpu) && kvm_vgic_vcpu_pending_irq(vcpu))
+		kvm_make_request(KVM_REQ_GUEST_HYP_IRQ_PENDING, vcpu);
 }
 
 void kvm_nested_sync_hwstate(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index 5cf5a1ef86cdd..7061b4ab206c9 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -1152,7 +1152,7 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
 	 *
 	 *   We do that by placing a request to ourselves which will
 	 *   abort the entry procedure and inject the exception at the
-	 *   beginning of the run loop.
+	 *   beginning of the run loop (see kvm_nested_flush_hwstate()).
 	 *
 	 * - Otherwise, do exactly *NOTHING* apart from enabling the virtual
 	 *   CPU interface. The guest state is already loaded, and we can
@@ -1162,16 +1162,10 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
 	 * maintenance interrupt state, as it may fire.
 	 */
 	if (vgic_state_is_nested(vcpu)) {
-		if (kvm_vgic_vcpu_pending_irq(vcpu))
-			kvm_make_request(KVM_REQ_GUEST_HYP_IRQ_PENDING, vcpu);
-
 		vgic_v3_flush_nested(vcpu);
 		return;
 	}
 
-	if (vcpu_has_nv(vcpu))
-		vgic_v3_nested_update_mi(vcpu);
-
 	DEBUG_SPINLOCK_BUG_ON(!irqs_disabled());
 
 	vgic_flush_state(vcpu);
-- 
2.47.3

Re: [PATCH 1/7] KVM: arm64: nv: Move L2->L1 switch on interrupt early

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] kvm: Outdated comment block in kvm_vgic_flush_hwstate()
--

commit 8603da24ffa95dd560817192077ee16c16570bf2
Author: Marc Zyngier <maz@kernel.org>

KVM: arm64: nv: Move L2->L1 switch on interrupt early

This commit moves the nested request generation for L2->L1 switch on
interrupt and the maintenance interrupt (MI) generation to earlier in
the vCPU entry sequence.

> diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
> index b25303d9919fd..1748e7758681b 100644
> --- a/arch/arm64/kvm/vgic/vgic.c
> +++ b/arch/arm64/kvm/vgic/vgic.c
> @@ -1141,7 +1141,7 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
[ ... ]
>  	 * - Otherwise, do exactly *NOTHING* apart from enabling the virtual
>  	 *   CPU interface. The guest state is already loaded, and we can
>  	 *   carry on with running it.
>  	 *
>  	 * If we have NV, but are not in a nested state, compute the
>  	 * maintenance interrupt state, as it may fire.
>  	 */
>  	if (vgic_state_is_nested(vcpu)) {

[Severity: Low]
This isn't a bug, but should this comment block in kvm_vgic_flush_hwstate()
be updated?

The patch moves the execution of vgic_v3_nested_update_mi() to
kvm_nested_flush_hwstate(), so the note about computing the maintenance
interrupt state here appears to be outdated now.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261015100539.1011760-1-maz@kernel.org?part=1

[PATCH 2/7] KVM: arm64: Move OUTSIDE_GUEST_MODE publication past context being saved

[Marc Zyngier]

OUTSIDE_GUEST_MODE indicates to the rest of KVM that the vcpu has
exited. So far, this is set as soon as we return from the notional
hypervisor (which with VHE is just a function call away).

However, code that uses OUTSIDE_GUEST_MODE as a synchronisation point
relies on the vcpu state to have been written back to the in-memory
structures. Clearly, this is not what is happening

Move the publication of OUTSIDE_GUEST_MODE to the point where the state
is actually written, and give this write release semantics to ensure the
correct ordering.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
---
 arch/arm64/kvm/arm.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 73706464d2653..8b46fafdd3494 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1334,13 +1334,13 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
 		smp_store_mb(vcpu->mode, IN_GUEST_MODE);
 
 		if (ret <= 0 || kvm_vcpu_exit_request(vcpu, &ret)) {
-			vcpu->mode = OUTSIDE_GUEST_MODE;
 			isb(); /* Ensure work in x_flush_hwstate is committed */
 			if (kvm_vcpu_has_pmu(vcpu))
 				kvm_pmu_sync_hwstate(vcpu);
 			if (unlikely(!irqchip_in_kernel(vcpu->kvm)))
 				kvm_timer_sync_user(vcpu);
 			kvm_vgic_sync_hwstate(vcpu);
+			smp_store_release(&vcpu->mode, OUTSIDE_GUEST_MODE);
 			local_irq_enable();
 			preempt_enable();
 			continue;
@@ -1356,7 +1356,6 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
 
 		ret = kvm_arm_vcpu_enter_exit(vcpu);
 
-		vcpu->mode = OUTSIDE_GUEST_MODE;
 		vcpu->stat.exits++;
 		/*
 		 * Back from guest
@@ -1390,6 +1389,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
 
 		kvm_arch_vcpu_ctxsync_fp(vcpu);
 
+		/*
+		 * All the state has been synchronised, let advertise
+		 * we're outside of the guest.
+		 */
+		smp_store_release(&vcpu->mode, OUTSIDE_GUEST_MODE);
+
 		/*
 		 * We must ensure that any pending interrupts are taken before
 		 * we exit guest timing so that timer ticks are accounted as
-- 
2.47.3

Re: [PATCH 2/7] KVM: arm64: Move OUTSIDE_GUEST_MODE publication past context being saved

[Fuad Tabba]

Hi Marc,

On Thu, 15 Oct 2026 11:05:34 +0100, Marc Zyngier <maz@kernel.org> wrote:
[...]
> However, code that uses OUTSIDE_GUEST_MODE as a synchronisation point
> relies on the vcpu state to have been written back to the in-memory
> structures. Clearly, this is not what is happening

nit: missing full stop.

[...]
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
[...]
> @@ -1390,6 +1389,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
>
>  		kvm_arch_vcpu_ctxsync_fp(vcpu);
>
> +		/*
> +		 * All the state has been synchronised, let advertise
> +		 * we're outside of the guest.
> +		 */
> +		smp_store_release(&vcpu->mode, OUTSIDE_GUEST_MODE);

nit: could this say the vgic state rather than all the state? In pKVM,
a non-protected guest's context comes back later, in
handle_exit_pkvm_state() or on demand, and with VHE the EL1 sysregs
stay on the CPU until vcpu_put(). Also "let's advertise".

Cheers,
/fuad

Re: [PATCH 2/7] KVM: arm64: Move OUTSIDE_GUEST_MODE publication past context being saved

[Marc Zyngier]

On Thu, 15 Oct 2026 13:57:13 +0100,
Fuad Tabba <fuad.tabba@linux.dev> wrote:
> 
> Hi Marc,
> 
> On Thu, 15 Oct 2026 11:05:34 +0100, Marc Zyngier <maz@kernel.org> wrote:
> [...]
> > However, code that uses OUTSIDE_GUEST_MODE as a synchronisation point
> > relies on the vcpu state to have been written back to the in-memory
> > structures. Clearly, this is not what is happening
> 
> nit: missing full stop.
> 
> [...]
> > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> [...]
> > @@ -1390,6 +1389,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
> >
> >  		kvm_arch_vcpu_ctxsync_fp(vcpu);
> >
> > +		/*
> > +		 * All the state has been synchronised, let advertise
> > +		 * we're outside of the guest.
> > +		 */
> > +		smp_store_release(&vcpu->mode, OUTSIDE_GUEST_MODE);
> 
> nit: could this say the vgic state rather than all the state? In pKVM,
> a non-protected guest's context comes back later, in
> handle_exit_pkvm_state() or on demand, and with VHE the EL1 sysregs
> stay on the CPU until vcpu_put(). Also "let's advertise".

I don't really want to make this vgic specific, as there could be
more. But I can more precisely describe what state were talking about:

diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 8b46fafdd3494..7ce9f3bfa7fff 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1390,8 +1390,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
 		kvm_arch_vcpu_ctxsync_fp(vcpu);
 
 		/*
-		 * All the state has been synchronised, let advertise
-		 * we're outside of the guest.
+		 * The state that was pushed into the CPU on entering
+		 * the guest has been synchronised back, let's
+		 * advertise we're outside of the guest.
 		 */
 		smp_store_release(&vcpu->mode, OUTSIDE_GUEST_MODE);
 

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH 2/7] KVM: arm64: Move OUTSIDE_GUEST_MODE publication past context being saved

[Fuad Tabba]

On Thu, 15 Oct 2026 at 14:52, Marc Zyngier <maz@kernel.org> wrote:
[...]
> I don't really want to make this vgic specific, as there could be
> more. But I can more precisely describe what state were talking about:
>
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 8b46fafdd3494..7ce9f3bfa7fff 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -1390,8 +1390,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
>                 kvm_arch_vcpu_ctxsync_fp(vcpu);
>
>                 /*
> -                * All the state has been synchronised, let advertise
> -                * we're outside of the guest.
> +                * The state that was pushed into the CPU on entering
> +                * the guest has been synchronised back, let's
> +                * advertise we're outside of the guest.
>                  */

lgtm,
/fuad

[PATCH 3/7] KVM: arm64: Move IN_GUEST_MODE ahead of state being loaded

[Marc Zyngier]

IN_GUEST_MODE is advertised too late, when state has already been moved
onto CPU registers and cannot be observed by another CPU anymore.

Move the IN_GUEST_MODE publication ahead of the state being loaded, and
immediately check for pending request. This ensures that there is no
window of opportunity for another CPU to observe OUTSIDE_GUEST_MODE and
yet for state to be loaded.

This also simplifies the "state rollback", which solely amounts to
synchronising the timer when no irqchip is present.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
---
 arch/arm64/kvm/arm.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 8b46fafdd3494..9956049e592ab 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1321,31 +1321,33 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
 
 		local_irq_disable();
 
-		kvm_vgic_flush_hwstate(vcpu);
-
-		kvm_pmu_update_vcpu_events(vcpu);
-
 		/*
 		 * Ensure we set mode to IN_GUEST_MODE after we disable
 		 * interrupts and before the final VCPU requests check.
 		 * See the comment in kvm_vcpu_exiting_guest_mode() and
 		 * Documentation/virt/kvm/vcpu-requests.rst
+		 *
+		 * Requests to self must be posted before this point, or
+		 * won't be taken into account until the next exit. No state
+		 * loading other than what happens at vcpu_load() time can
+		 * be done before we have checked for pending requests.
 		 */
 		smp_store_mb(vcpu->mode, IN_GUEST_MODE);
 
 		if (ret <= 0 || kvm_vcpu_exit_request(vcpu, &ret)) {
-			isb(); /* Ensure work in x_flush_hwstate is committed */
-			if (kvm_vcpu_has_pmu(vcpu))
-				kvm_pmu_sync_hwstate(vcpu);
 			if (unlikely(!irqchip_in_kernel(vcpu->kvm)))
 				kvm_timer_sync_user(vcpu);
-			kvm_vgic_sync_hwstate(vcpu);
 			smp_store_release(&vcpu->mode, OUTSIDE_GUEST_MODE);
 			local_irq_enable();
 			preempt_enable();
 			continue;
 		}
 
+		/* From here onwards, we are commited to entering the guest */
+		kvm_vgic_flush_hwstate(vcpu);
+
+		kvm_pmu_update_vcpu_events(vcpu);
+
 		kvm_arch_vcpu_ctxflush_fp(vcpu);
 
 		/**************************************************************
-- 
2.47.3

[PATCH 4/7] KVM: arm64: Turn vcpu->arch.pause into a counter

[Marc Zyngier]

We have situations where we can call kvm_arm_halt_guest() without
holding config_lock, which means that halt and resume can overlap in
funny ways when called from separate contexts, and result in situations
where a vcpu is resumed while other parts of KVM assume it is halted.
Consequences are left to the imagination of the reader.

Fix this sorry situation by turning kvm_vcpu_arch::pause into an
atomic counter, which makes the races described above harmless.

Fixes: 3b92830ad41b2 ("KVM: arm/arm64: implement kvm_arm_[halt,resume]_guest")
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba < fuad.tabba@linux.dev>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
---
 arch/arm64/include/asm/kvm_host.h |  7 +++----
 arch/arm64/kvm/arm.c              | 21 ++++++++++++---------
 2 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index cd9b9d2462f96..9c068f27f9b8a 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -889,11 +889,10 @@ struct kvm_vcpu_arch {
 	/*
 	 * Don't run the guest (internal implementation need).
 	 *
-	 * Contrary to the flags above, this is set/cleared outside of
-	 * a vcpu context, and thus cannot be mixed with the flags
-	 * themselves (or the flag accesses need to be made atomic).
+	 * Contrary to the flags above, this is updated outside of
+	 * a vcpu context, and thus cannot be mixed with the flags.
 	 */
-	bool pause;
+	atomic_t pause;
 
 	/*
 	 * We maintain more than a single set of debug registers to support
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 9956049e592ab..07bf8b2e683d5 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -565,6 +565,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
 	kvm_arm_pvtime_vcpu_init(&vcpu->arch);
 
 	vcpu->arch.hw_mmu = &vcpu->kvm->arch.mmu;
+	atomic_set(&vcpu->arch.pause, 0);
 
 	/*
 	 * This vCPU may have been created after mpidr_data was initialized.
@@ -839,6 +840,11 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
 	return ret;
 }
 
+static bool vcpu_can_run(struct kvm_vcpu *vcpu)
+{
+	return !kvm_arm_vcpu_stopped(vcpu) && !atomic_read(&vcpu->arch.pause);
+}
+
 /**
  * kvm_arch_vcpu_runnable - determine if the vcpu can be scheduled
  * @v:		The VCPU pointer
@@ -854,8 +860,7 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
 		      (kvm_timer_should_notify_user(v) ||
 		       kvm_pmu_should_notify_user(v)));
 
-	return ((irq_lines || kvm_vgic_vcpu_pending_irq(v))
-		&& !kvm_arm_vcpu_stopped(v) && !v->arch.pause);
+	return ((irq_lines || kvm_vgic_vcpu_pending_irq(v)) && vcpu_can_run(v));
 }
 
 bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu)
@@ -1018,7 +1023,7 @@ void kvm_arm_halt_guest(struct kvm *kvm)
 	struct kvm_vcpu *vcpu;
 
 	kvm_for_each_vcpu(i, vcpu, kvm)
-		vcpu->arch.pause = true;
+		atomic_inc(&vcpu->arch.pause);
 	kvm_make_all_cpus_request(kvm, KVM_REQ_SLEEP);
 }
 
@@ -1028,8 +1033,8 @@ void kvm_arm_resume_guest(struct kvm *kvm)
 	struct kvm_vcpu *vcpu;
 
 	kvm_for_each_vcpu(i, vcpu, kvm) {
-		vcpu->arch.pause = false;
-		__kvm_vcpu_wake_up(vcpu);
+		if (atomic_dec_and_test(&vcpu->arch.pause))
+			__kvm_vcpu_wake_up(vcpu);
 	}
 }
 
@@ -1037,11 +1042,9 @@ static void kvm_vcpu_sleep(struct kvm_vcpu *vcpu)
 {
 	struct rcuwait *wait = kvm_arch_vcpu_get_wait(vcpu);
 
-	rcuwait_wait_event(wait,
-			   (!kvm_arm_vcpu_stopped(vcpu)) && (!vcpu->arch.pause),
-			   TASK_INTERRUPTIBLE);
+	rcuwait_wait_event(wait, vcpu_can_run(vcpu), TASK_INTERRUPTIBLE);
 
-	if (kvm_arm_vcpu_stopped(vcpu) || vcpu->arch.pause) {
+	if (!vcpu_can_run(vcpu)) {
 		/* Awaken to handle a signal, request we sleep again later. */
 		kvm_make_request(KVM_REQ_SLEEP, vcpu);
 	}
-- 
2.47.3

[PATCH 5/7] KVM: arm64: Add helpers to halt a vCPU

[Marc Zyngier]

From: Oliver Upton <oupton@kernel.org>

Currently we pause the entire VM when modifying VGIC state that requires
state to be synchronized back from hardware. This is perfectly fine for
heavyweight operations but not ideal for 'simple' things like migrating
an active/pending IRQ.

Add helpers for halting a single vCPU such that they can be used to
manipulate the AP list remotely.

Signed-off-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/20260929212925.31775-2-oupton@kernel.org
[maz: fixed to use kvm_make_vcpus_request_mask(), guaranteeing that the
      caller waits for the target vcpu to handle the IPI]
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/include/asm/kvm_host.h |  2 ++
 arch/arm64/kvm/arm.c              | 21 +++++++++++++++++----
 2 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 9c068f27f9b8a..7b4f7b6d94b8d 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -1254,6 +1254,8 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,
 
 void kvm_arm_halt_guest(struct kvm *kvm);
 void kvm_arm_resume_guest(struct kvm *kvm);
+void kvm_pause_vcpu(struct kvm_vcpu *vcpu);
+void kvm_resume_vcpu(struct kvm_vcpu *vcpu);
 
 #define vcpu_has_run_once(vcpu)	(!!READ_ONCE((vcpu)->pid))
 
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 07bf8b2e683d5..a2f1606822614 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1017,6 +1017,21 @@ bool kvm_arch_intc_initialized(struct kvm *kvm)
 	return vgic_initialized(kvm);
 }
 
+void kvm_pause_vcpu(struct kvm_vcpu *vcpu)
+{
+	DECLARE_BITMAP(vcpu_bitmap, KVM_MAX_VCPUS) = { };
+
+	atomic_inc(&vcpu->arch.pause);
+	__set_bit(vcpu->vcpu_idx, vcpu_bitmap);
+	kvm_make_vcpus_request_mask(vcpu->kvm, KVM_REQ_SLEEP, vcpu_bitmap);
+}
+
+void kvm_resume_vcpu(struct kvm_vcpu *vcpu)
+{
+	if (atomic_dec_and_test(&vcpu->arch.pause))
+		__kvm_vcpu_wake_up(vcpu);
+}
+
 void kvm_arm_halt_guest(struct kvm *kvm)
 {
 	unsigned long i;
@@ -1032,10 +1047,8 @@ void kvm_arm_resume_guest(struct kvm *kvm)
 	unsigned long i;
 	struct kvm_vcpu *vcpu;
 
-	kvm_for_each_vcpu(i, vcpu, kvm) {
-		if (atomic_dec_and_test(&vcpu->arch.pause))
-			__kvm_vcpu_wake_up(vcpu);
-	}
+	kvm_for_each_vcpu(i, vcpu, kvm)
+		kvm_resume_vcpu(vcpu);
 }
 
 static void kvm_vcpu_sleep(struct kvm_vcpu *vcpu)
-- 
2.47.3

Re: [PATCH 5/7] KVM: arm64: Add helpers to halt a vCPU

[Fuad Tabba]

Hi Marc,

On Thu, 15 Oct 2026 11:05:37 +0100, Oliver Upton <oupton@kernel.org> wrote:
[...]
> Signed-off-by: Oliver Upton <oupton@kernel.org>
> Link: https://patch.msgid.link/20260929212925.31775-2-oupton@kernel.org
> Signed-off-by: Marc Zyngier <maz@kernel.org>

Should this one have a Cc: stable too? Patch 6 is tagged for stable and
calls kvm_pause_vcpu() and kvm_resume_vcpu(), so it won't build in
stable without this patch.

Cheers,
/fuad

Re: [PATCH 5/7] KVM: arm64: Add helpers to halt a vCPU

[Marc Zyngier]

On Thu, 15 Oct 2026 13:57:16 +0100,
Fuad Tabba <fuad.tabba@linux.dev> wrote:
> 
> Hi Marc,
> 
> On Thu, 15 Oct 2026 11:05:37 +0100, Oliver Upton <oupton@kernel.org> wrote:
> [...]
> > Signed-off-by: Oliver Upton <oupton@kernel.org>
> > Link: https://patch.msgid.link/20260929212925.31775-2-oupton@kernel.org
> > Signed-off-by: Marc Zyngier <maz@kernel.org>
> 
> Should this one have a Cc: stable too? Patch 6 is tagged for stable and
> calls kvm_pause_vcpu() and kvm_resume_vcpu(), so it won't build in
> stable without this patch.

My take is that there is no need to backport this patch at all. A
valid backport of patch #6 should simply halt the full VM, not just a
single vcpu.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH 5/7] KVM: arm64: Add helpers to halt a vCPU

[Fuad Tabba]

On Thu, 15 Oct 2026 at 14:13, Marc Zyngier <maz@kernel.org> wrote:
>
> On Thu, 15 Oct 2026 13:57:16 +0100,
> Fuad Tabba <fuad.tabba@linux.dev> wrote:
> >
> > Hi Marc,
> >
> > On Thu, 15 Oct 2026 11:05:37 +0100, Oliver Upton <oupton@kernel.org> wrote:
> > [...]
> > > Signed-off-by: Oliver Upton <oupton@kernel.org>
> > > Link: https://patch.msgid.link/20260929212925.31775-2-oupton@kernel.org
> > > Signed-off-by: Marc Zyngier <maz@kernel.org>
> >
> > Should this one have a Cc: stable too? Patch 6 is tagged for stable and
> > calls kvm_pause_vcpu() and kvm_resume_vcpu(), so it won't build in
> > stable without this patch.
>
> My take is that there is no need to backport this patch at all. A
> valid backport of patch #6 should simply halt the full VM, not just a
> single vcpu.

Makes sense, so this will be resolved on the backport itself.

/fuad

> Thanks,
>
>         M.
>
> --
> Without deviation from the norm, progress is not possible.

[PATCH 6/7] KVM: arm64: vgic: Stop the target vcpu when disabling LPIs

[Marc Zyngier]

Disabling LPIs is pretty nasty, as it directly messes with the AP list
of the affected CPU, which could be running... This has the potential to
lead to really bad behaviours, and we should not be doing that.

Take example on the way the Active state is handled and simply pause the
the vcpu so that we are sure it is in a quiescent state, and the state be
safely manipulated.

Nobody has any expectation of performance for this anyway.

Fixes: 96085b949672d ("KVM: arm/arm64: vgic-v3: Retire pending interrupts on disabling LPIs")
Co-developed-by: Oliver Upton <oupton@kernel.org>
Signed-off-by: Oliver Upton <oupton@kernel.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
---
 arch/arm64/kvm/vgic/vgic-mmio-v3.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/arm64/kvm/vgic/vgic-mmio-v3.c b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
index 5913a20d83019..f5bd29fbe3a90 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
@@ -303,9 +303,19 @@ static void vgic_mmio_write_v3r_ctlr(struct kvm_vcpu *vcpu,
 		if (ctlr != GICR_CTLR_ENABLE_LPIS)
 			return;
 
+		/*
+		 * Yes, disabling LPIs is painful, since it can be done from
+		 * a *remote* vcpu! So let's not take any chance, and make
+		 * sure that the target vCPU has written their LRs back to the
+		 * irq structures, and release any reference they would have.
+		 *
+		 * If it hurts, don't do it.
+		 */
+		kvm_pause_vcpu(vcpu);
 		vgic_flush_pending_lpis(vcpu);
 		vgic_its_invalidate_all_caches(vcpu->kvm);
 		atomic_set_release(&vgic_cpu->ctlr, 0);
+		kvm_resume_vcpu(vcpu);
 	} else {
 		ctlr = atomic_cmpxchg_acquire(&vgic_cpu->ctlr, 0,
 					      GICR_CTLR_ENABLE_LPIS);
-- 
2.47.3

[PATCH 7/7] KVM: arm64: vgic-its: Stop the VM when handling MOVALL

[Marc Zyngier]

MOVALL is a royal pain, as it requires us to iterate over all the LPIs
and move them around if they are on the correct source vcpu, and
requiring locking for each of them.

This locking can result in contention for vcpus that enter and exit
the guest, and hinders forward progress. In order to make sure such
process is not contended, just stop the VM altogether.

Nobody in their right mind uses MOVALL anyway, as this is a rather
braindead aspect of the GICv3 architecture.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/vgic/vgic-its.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c
index c10a11373e106..a9949ab4e4133 100644
--- a/arch/arm64/kvm/vgic/vgic-its.c
+++ b/arch/arm64/kvm/vgic/vgic-its.c
@@ -1382,6 +1382,15 @@ static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
 	if (vcpu1 == vcpu2)
 		return 0;
 
+	/*
+	 * Bulk operations such as MOVALL are a pain, as they can clash
+	 * badly locking-wise with other vcpus entering and exiting the
+	 * guest, should they be affected by it. Stopping the guest is a
+	 * safer bet to ensure uncontended access and ultimately forward
+	 * progress. Yeah...
+	 */
+	kvm_arm_halt_guest(kvm);
+
 	xa_for_each(&dist->lpi_xa, intid, irq) {
 		irq = vgic_get_irq(kvm, intid);
 		if (!irq)
@@ -1394,6 +1403,8 @@ static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
 
 	vgic_its_invalidate_cache(its);
 
+	kvm_arm_resume_guest(kvm);
+
 	return 0;
 }
 
-- 
2.47.3

Re: [PATCH 0/7] KVM: arm64: Fix cross-CPU vcpu pausing

[Fuad Tabba]

Hi Marc,

On Thu, 15 Oct 2026 11:05:32 +0100, Marc Zyngier <maz@kernel.org> wrote:
[...]
> This series addresses these defects, and builds on top of it to:
>
> - add pausing to a single vcpu in addition to the full VM pausing
>
> - convert LPI disabling and the MOVALL command to vcpu pausing
>
> Patches on top of -rc6, most of them being Cc stable without any
> qualification (this has been broken for as long as I can remember).

I'll send a couple of nits on patches 2 and 5 as replies to those
patches.

For the series:
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Tested on QEMU on top of -rc6: npVM and pVM boots in pKVM, npVM boots
with nVHE and VHE hosts, and vgic_irq and vgic_lpi_stress in all three
and with kvm-arm.mode=nested. My reproducer has vcpu 0 clear vcpu 1's
EnableLPIs while the LPI is in vcpu 1's LR: on -rc6 the LPI is still
pending on vcpu 1 afterwards, and with the series it isn't. With my
vgic-its v2 [1] on top, my MOVALL selftest and my shared LPI and DISCARD
reproducers pass, including with PROVE_LOCKING and with debug hooks
that force the DISCARD race.

Tested-by: Fuad Tabba <fuad.tabba@linux.dev>

v2 of "Fix LPI double mapping and DISCARD" [1] applies on top of this
series as it is. Would you take it as it is, or would you like me to
respin it?

Cheers,
/fuad

[1] https://lore.kernel.org/all/20261013081646.933312-1-fuad.tabba@linux.dev/

Re: [PATCH 0/7] KVM: arm64: Fix cross-CPU vcpu pausing

[Marc Zyngier]

On Thu, 15 Oct 2026 13:57:12 +0100,
Fuad Tabba <fuad.tabba@linux.dev> wrote:
> 
> Hi Marc,
> 
> On Thu, 15 Oct 2026 11:05:32 +0100, Marc Zyngier <maz@kernel.org> wrote:
> [...]
> > This series addresses these defects, and builds on top of it to:
> >
> > - add pausing to a single vcpu in addition to the full VM pausing
> >
> > - convert LPI disabling and the MOVALL command to vcpu pausing
> >
> > Patches on top of -rc6, most of them being Cc stable without any
> > qualification (this has been broken for as long as I can remember).
> 
> I'll send a couple of nits on patches 2 and 5 as replies to those
> patches.

Thanks, I'll tidy that up as I apply them.

> 
> For the series:
> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
> 
> Tested on QEMU on top of -rc6: npVM and pVM boots in pKVM, npVM boots
> with nVHE and VHE hosts, and vgic_irq and vgic_lpi_stress in all three
> and with kvm-arm.mode=nested. My reproducer has vcpu 0 clear vcpu 1's
> EnableLPIs while the LPI is in vcpu 1's LR: on -rc6 the LPI is still
> pending on vcpu 1 afterwards, and with the series it isn't. With my
> vgic-its v2 [1] on top, my MOVALL selftest and my shared LPI and DISCARD
> reproducers pass, including with PROVE_LOCKING and with debug hooks
> that force the DISCARD race.
> 
> Tested-by: Fuad Tabba <fuad.tabba@linux.dev>

Great, thanks again.

> 
> v2 of "Fix LPI double mapping and DISCARD" [1] applies on top of this
> series as it is. Would you take it as it is, or would you like me to
> respin it?

I can stash it on top as is. Looks like the first set of 7.4 fixes is
going to be pretty vgic-heavy anyway.

Cheers,

	M.

-- 
Without deviation from the norm, progress is not possible.