From: Sean Christopherson <seanjc@google.com>
To: Satish Kharat <satishkh@cisco.com>
Cc: pbonzini@redhat.com, kvm@vger.kernel.org, x86@kernel.org,
linux-kernel@vger.kernel.org, tglx@kernel.org, mingo@redhat.com,
bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com,
stable@vger.kernel.org
Subject: Re: [PATCH] KVM: x86: Start IRQ bypass with local interrupts enabled
Date: Mon, 16 Nov 2026 13:26:15 -0800 [thread overview]
Message-ID: <avt098dxfwl0FIGE@seanjc-mac.roam.internal> (raw)
In-Reply-To: <20261116-b4-kvm-irq-bypass-irqs-20261116-v1-1-6cc721a062fd@cisco.com>
On Mon, Nov 16, 2026, Satish Kharat wrote:
> Move the first IRQ-bypass notification after releasing irqfds.lock so
> that the vendor callback runs with local interrupts enabled.
>
> On Intel, pi_start_bypass() issues KVM_REQ_UNBLOCK to the vCPUs through
> kvm_make_all_cpus_request(). If another CPU needs to be kicked, this can
> reach smp_call_function_many(), which requires local interrupts enabled.
> Calling the hook inside spin_lock_irq() violates that requirement and
> triggered the following warning during VFIO MSI-X setup:
>
> WARNING: kernel/smp.c:943 at smp_call_function_many_cond
> kvm_make_all_cpus_request [kvm]
> kvm_arch_irq_bypass_add_producer [kvm]
> __connect [irqbypass]
> irq_bypass_register_producer [irqbypass]
> vfio_msi_set_vector_signal [vfio_pci_core]
>
> The SMP API documents a possible deadlock in this context, including for
> non-waiting calls. No deadlock was observed in this case.
>
> Keep producer publication, bypass accounting and IRTE error rollback under
> the existing lock. The IRQ bypass mutex still serializes producer
> callbacks after irqfds.lock is released. Notify only for a successful
> first attachment, after any IRTE failure has been ruled out.
>
> An LLM assisted with implementation, code review and test automation.
LOL, I couldn't tell. :-D
> diff --git a/arch/x86/kvm/irq.c b/arch/x86/kvm/irq.c
> index 7f8b092b4479..611b3e57b09a 100644
> --- a/arch/x86/kvm/irq.c
> +++ b/arch/x86/kvm/irq.c
> @@ -485,13 +485,13 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
> struct kvm_kernel_irqfd *irqfd =
> container_of(cons, struct kvm_kernel_irqfd, consumer);
> struct kvm *kvm = irqfd->kvm;
> + bool start_bypass;
> int ret = 0;
>
> spin_lock_irq(&kvm->irqfds.lock);
> irqfd->producer = prod;
>
> - if (!kvm->arch.nr_possible_bypass_irqs++)
> - kvm_x86_call(pi_start_bypass)(kvm);
> + start_bypass = !kvm->arch.nr_possible_bypass_irqs++;
>
> if (irqfd->irq_entry.type == KVM_IRQ_ROUTING_MSI) {
> ret = kvm_pi_update_irte(irqfd, &irqfd->irq_entry);
> @@ -502,6 +502,13 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
> }
> spin_unlock_irq(&kvm->irqfds.lock);
>
> + /*
> + * Wake vCPUs with IRQs enabled. The IRQ bypass mutex keeps producer
> + * callbacks serialized after dropping irqfds.lock.
> + */
> + if (!ret && start_bypass)
> + kvm_x86_call(pi_start_bypass)(kvm);
> +
> return ret;
> }
I would much rather fix this by having common code send resched IPIs for requests
that don't need to wait, same as KVM does for single-vCPU requests. Because the
current task needs to be sleepable if a request needs to wait, i.e. it's not just
IRQs that could be problematic, and callers shouldn't have to deal with internal
implementation details (and those details really shouldn't be different for single
vs. multiple vCPU requests).
Compile tested only. I'll write a changelog and post if it pans out.
--
From: Sean Christopherson <seanjc@google.com>
Date: Mon, 16 Nov 2026 13:21:51 -0800
Subject: [PATCH] KVM: Send reschedule IPIs instead of SMP function calls for
non-waiting requests
<write a changelog if this actually works>
Fixes: 77e1b8332d1d ("KVM: x86: Decouple device assignment from IRQ bypass")
Cc: stable@vger.kernel.org
Reported-by: Satish Kharat <satishkh@cisco.com>
Closes: https://lore.kernel.org/all/20261116-b4-kvm-irq-bypass-irqs-20261116-v1-1-6cc721a062fd@cisco.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
virt/kvm/kvm_main.c | 17 ++++++++++++++---
1 file changed, 14 insertions(+), 3 deletions(-)
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index cf4cb586f140..0b733746884c 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -218,7 +218,10 @@ static inline bool kvm_kick_many_cpus(struct cpumask *cpus, bool wait)
if (cpumask_empty(cpus))
return false;
- smp_call_function_many(cpus, ack_kick, NULL, wait);
+ /* See kvm_vcpu_kick() and kvm_make_vcpu_request() for details. */
+ if (wait)
+ smp_call_function_many(cpus, ack_kick, NULL, wait);
+
return true;
}
@@ -241,12 +244,20 @@ static void kvm_make_vcpu_request(struct kvm_vcpu *vcpu, unsigned int req,
* satisfied if the vCPU migrates. Entering READING_SHADOW_PAGE_TABLES
* after this point is also OK, as the requirement is only that KVM wait
* for vCPUs that were reading SPTEs _before_ any changes were
- * finalized. See kvm_vcpu_kick() for more details on handling requests.
+ * finalized.
+ *
+ * See kvm_vcpu_kick() for more details on handling requests, including
+ * the use of a reschedule IPI for non-waiting requests. If waiting is
+ * required, batch the SMP function call until after all to-be-kicked
+ * CPUs are gathered, to avoid waiting multiple times.
*/
if (kvm_request_needs_ipi(vcpu, req)) {
cpu = READ_ONCE(vcpu->cpu);
- if (cpu != -1 && cpu != current_cpu)
+ if (cpu != -1 && cpu != current_cpu) {
+ if (!(req & KVM_REQUEST_WAIT))
+ smp_send_reschedule(cpu);
__cpumask_set_cpu(cpu, tmp);
+ }
}
}
base-commit: d21eef2084a9af87ac2a783bc7c2b175ba114e4e
--
prev parent reply other threads:[~2026-11-16 21:26 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-16 20:38 [PATCH] KVM: x86: Start IRQ bypass with local interrupts enabled Satish Kharat
2026-11-16 21:26 ` Sean Christopherson [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=avt098dxfwl0FIGE@seanjc-mac.roam.internal \
--to=seanjc@google.com \
--cc=bp@alien8.de \
--cc=dave.hansen@linux.intel.com \
--cc=hpa@zytor.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=pbonzini@redhat.com \
--cc=satishkh@cisco.com \
--cc=stable@vger.kernel.org \
--cc=tglx@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox