[PATCH v4 0/9] KVM: Improve IRQ assignment for device passthrough

Jan Kiszka
2010-11-08T11:22:13Z

Nine patches (yeah, it's getting more and more) to improve "classic"
device assigment /wrt IRQs. Highlight is the last one that resolves the
host IRQ sharing issue for all PCI 2.3 devices. Quite essential when
passing non-MSI-ready devices like many USB host controllers.

As there were concerns regarding the overhead of IRQ masking via the PCI
config space, I did some micro-benchmarks. Well, the concerns are valid:

disable_irq_nosync: ~600 cycles
pci_2_3_irq_check_and_mask: ~6000 cycles (EHCI)
~22000 cycles (AR9287, with peaks >100000)

Specifically the varying impact of the device like in the Atheros case
is worrying (this device is actually known to cause horrible latencies
to the host, but who knows what other devices do). So I decided to go
with PCI-2.3 masking as default off in the to-be-sent qemu-kvm patch.
Maybe something to consider vor VFIO as well.

Changes in v4:
- Allow user space to push its INTx mask state to the kernel and
respect this on IRQ masking and delivery
- Specify IRQF_ONESHOT for threaded IRQ of assigned device
- Switch IRQ subsystem locking from RCU to SRCU
- Fix for SRCU struct leakage
- Small cleanup for kvm_vm_ioctl_assigned_device
- Documentation for device assignment API (please check carefully if I
got it right)

Changes in v3:
- Save/restore PCI device state across guest usage
- Make PCI-2.3-based IRQ sharing configurable by user space
- Fix potential nesting of pci_block_user_cfg_access
- Do not track PCI-level IRQ masking in software, user space may change
it concurrently
- Optimize kvm_assigned_dev_ack_irq for the case another IRQ is already
pending
- Cleanups according to review comments

Changes in v2:
- Reworked IRQ forwarding path to use threaded IRQs (direct signalling
from IRQ context does not work out of the box and may be too lengthy)
- Refactored host IRQ naming of assigned devices (cosmetic change)
- Avoid unmask on ack when the next IRQ is pending, rather reassert the
guest line (PCI-2.3 patch)
- Refactored PCI-2.3 patch (but still no control knob for shared mode -
is that a must?)

Jan Kiszka (9):
KVM: Fix srcu struct leakage
KVM: Switch IRQ subsystem to SRCU
KVM: Clear assigned guest IRQ on release
KVM: Switch assigned device IRQ forwarding to threaded handler
KVM: Refactor IRQ names of assigned devices
KVM: Save/restore state of assigned PCI device
KVM: Clean up kvm_vm_ioctl_assigned_device
KVM: Document device assigment API
KVM: Allow host IRQ sharing for passed-through PCI 2.3 devices

Documentation/kvm/api.txt | 203 ++++++++++++++++++++++++
arch/x86/kvm/x86.c | 1 +
include/linux/kvm.h | 6 +
include/linux/kvm_host.h | 16 +--
virt/kvm/assigned-dev.c | 383 ++++++++++++++++++++++++++++++++++----------
virt/kvm/irq_comm.c | 31 ++--
virt/kvm/kvm_main.c | 20 ++-
7 files changed, 541 insertions(+), 119 deletions(-)