[PATCH] KVM: s390: improve interrupt cpu for wakeup

[PATCH] KVM: s390: improve interrupt cpu for wakeup

[Christian Borntraeger]

Turns out that picking an idle CPU for floating interrupts has some
negative side effects. The guest will keep the IO workload on its CPU
and rather use an IPI from the interrupt CPU instead of moving workload.
For example a guest with 2 vCPUss and 1 fio process might run that fio on
vcpu1. If after diag500 both vCPUs are idle then vcpu0 is woken up. The
guest will then do an IPI from vcpu0 to vcpu1.

So lets change the heuristics and prefer the last CPU that went to
sleep. This one is likely still in halt polling and can be woken up
quickly.

This patch shows significant improvements in terms of bandwidth or
cpu consumption for fio and uperf workloads and seems to be a net
win.

Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
---
 arch/s390/include/asm/kvm_host.h |  2 +-
 arch/s390/kvm/interrupt.c        | 20 +++++++++-----------
 2 files changed, 10 insertions(+), 12 deletions(-)

diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
index f870d09515cc1..95d15416c39d2 100644
--- a/arch/s390/include/asm/kvm_host.h
+++ b/arch/s390/include/asm/kvm_host.h
@@ -356,7 +356,7 @@ struct kvm_s390_float_interrupt {
 	int counters[FIRQ_MAX_COUNT];
 	struct kvm_s390_mchk_info mchk;
 	struct kvm_s390_ext_info srv_signal;
-	int next_rr_cpu;
+	int last_sleep_cpu;
 	struct mutex ais_lock;
 	u8 simm;
 	u8 nimm;
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index 60c360c18690f..b8e6f82e92c3f 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -1322,6 +1322,7 @@ int kvm_s390_handle_wait(struct kvm_vcpu *vcpu)
 	VCPU_EVENT(vcpu, 4, "enabled wait: %llu ns", sltime);
 no_timer:
 	kvm_vcpu_srcu_read_unlock(vcpu);
+	vcpu->kvm->arch.float_int.last_sleep_cpu = vcpu->vcpu_idx;
 	kvm_vcpu_halt(vcpu);
 	vcpu->valid_wakeup = false;
 	__unset_cpu_idle(vcpu);
@@ -1948,18 +1949,15 @@ static void __floating_irq_kick(struct kvm *kvm, u64 type)
 	if (!online_vcpus)
 		return;
 
-	/* find idle VCPUs first, then round robin */
-	sigcpu = find_first_bit(kvm->arch.idle_mask, online_vcpus);
-	if (sigcpu == online_vcpus) {
-		do {
-			sigcpu = kvm->arch.float_int.next_rr_cpu++;
-			kvm->arch.float_int.next_rr_cpu %= online_vcpus;
-			/* avoid endless loops if all vcpus are stopped */
-			if (nr_tries++ >= online_vcpus)
-				return;
-		} while (is_vcpu_stopped(kvm_get_vcpu(kvm, sigcpu)));
+	for (sigcpu = kvm->arch.float_int.last_sleep_cpu; ; sigcpu++) {
+		sigcpu %= online_vcpus;
+		dst_vcpu = kvm_get_vcpu(kvm, sigcpu);
+		if (!is_vcpu_stopped(dst_vcpu))
+			break;
+		/* avoid endless loops if all vcpus are stopped */
+		if (nr_tries++ >= online_vcpus)
+			return;
 	}
-	dst_vcpu = kvm_get_vcpu(kvm, sigcpu);
 
 	/* make the VCPU drop out of the SIE, or wake it up if sleeping */
 	switch (type) {
-- 
2.43.5

Re: [PATCH] KVM: s390: improve interrupt cpu for wakeup

[Christian Borntraeger]

Am 04.09.25 um 13:39 schrieb Christian Borntraeger:
> Turns out that picking an idle CPU for floating interrupts has some
> negative side effects. The guest will keep the IO workload on its CPU
> and rather use an IPI from the interrupt CPU instead of moving workload.
> For example a guest with 2 vCPUss and 1 fio process might run that fio on
> vcpu1. If after diag500 both vCPUs are idle then vcpu0 is woken up. The
> guest will then do an IPI from vcpu0 to vcpu1.
> 
> So lets change the heuristics and prefer the last CPU that went to
> sleep. This one is likely still in halt polling and can be woken up
> quickly.
> 
> This patch shows significant improvements in terms of bandwidth or
> cpu consumption for fio and uperf workloads and seems to be a net
> win.
> 
> Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>


applied internally for CI regression runs. Will push to next if successful.

Re: [PATCH] KVM: s390: improve interrupt cpu for wakeup

[Christoph Schlameuss]

With nit fixed.

Reviewed-by: Christoph Schlameuß <schlameuss@de.ibm.com>

On Thu Sep 4, 2025 at 1:39 PM CEST, Christian Borntraeger wrote:
> Turns out that picking an idle CPU for floating interrupts has some
> negative side effects. The guest will keep the IO workload on its CPU
> and rather use an IPI from the interrupt CPU instead of moving workload.
> For example a guest with 2 vCPUss and 1 fio process might run that fio on
----------------------------------^
nit: vCPUs

> vcpu1. If after diag500 both vCPUs are idle then vcpu0 is woken up. The
> guest will then do an IPI from vcpu0 to vcpu1.
>
> So lets change the heuristics and prefer the last CPU that went to
> sleep. This one is likely still in halt polling and can be woken up
> quickly.
>
> This patch shows significant improvements in terms of bandwidth or
> cpu consumption for fio and uperf workloads and seems to be a net
> win.
>
> Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
> ---
>  arch/s390/include/asm/kvm_host.h |  2 +-
>  arch/s390/kvm/interrupt.c        | 20 +++++++++-----------
>  2 files changed, 10 insertions(+), 12 deletions(-)