[RFC PATCH 0/5] Add support for AMD IOMMU GAPPI

[Sairaj Kodilkar <sarunkod@amd.com>]

Introduction
============
On newer generation of AMD processor, the IOMMU AVIC/x2AVIC feature can be
enabled with the new Guest APIC Physical Processor Interrupt (GAPPI) mode,
which is an alternative mode for handling AVIC guest interrupts to
non-running vcpus (i.e. IRTE[IsRun]=0).

With GAPPI enabled, the IOMMU delivers the posted interrupt to the physical
CPU described by the IRTE destination fields, with the wake up vector in
ga_tag.

Please see more detail about GAPPI in section 2.2.5.4 Guest APIC Physical
Processor Interrupt of the AMD I/O Virtualization Technology (IOMMU)
Specification [1]. 


Implementation Details
======================
GAPPI reuses the posted-interrupt wakeup path introduced for Intel VMX: the
IOMMU sets ga_tag to POSTED_INTR_WAKEUP_VECTOR and SVM registers the handler
with kvm_set_posted_intr_wakeup_handler().

SVM maintains a per-CPU list of vCPUs that are scheduled out.
When a CPU receives a GAPPI interrupt from the IOMMU, the handler
walks that list, finds vCPUs with a pending IRR bit, and wakes them.

The IRTE destination is chosen as the last host CPU where the vCPU ran, to
reduce unnecessary VMEXITs from GAPPI deliveries.

The first patch refactors the SVM/IOMMU interface: apicid (formerly cpu)
denotes the running vCPU's host APIC ID or, when scheduled out, the APIC ID
of the pCPU hosting the vCPU on its GAPPI wakeup list. Explicit running and
posted-interrupt flags replace the old ga_log_intr boolean, since apicid no
longer implies vCPU state on its own.


Advantages
==========
With GALOG, IOMMU can only generate a single interrupt using the MMIO offset
0x180h (XT IOMMU GA Log Interrupt Control Register) and appends the vCPU
information to the GALOG buffer. hypervisor has to scan this list in order to
wakeup the vCPUs which can introduce significant latency and even cause buffer
overflow under high interrupt rate.

GAPPI resolves this problem by distributing the posted interrupt across
multiple CPUs. 

[1] https://docs.amd.com/v/u/en-US/48882_3.11_IOMMU_PUB

Sairaj Kodilkar (5):
  iommu/amd: kvm/svm: Improve API between SVM and AMD IOMMU
  iommu/amd: Configure IRTE to use the GAPPI for posted interrupts
  kvm/svm: Introduce per-CPU lock and wakeup queue
  kvm/svm: Update the per-CPU wakeup-list during vCPU load and unload
  iommu/amd: Provide kernel command line option to enable GAPPI

 arch/x86/include/asm/irq_remapping.h |   4 +-
 arch/x86/kvm/svm/avic.c              | 136 +++++++++++++++++++++++----
 arch/x86/kvm/svm/svm.c               |   2 +
 arch/x86/kvm/svm/svm.h               |   5 +
 drivers/iommu/amd/amd_iommu.h        |   1 +
 drivers/iommu/amd/amd_iommu_types.h  |   6 +-
 drivers/iommu/amd/init.c             |  23 ++++-
 drivers/iommu/amd/iommu.c            |  41 +++++---
 include/linux/amd-iommu.h            |  15 ++-
 9 files changed, 192 insertions(+), 41 deletions(-)


base-commit: 8cd9520d35a6c38db6567e97dd93b1f11f185dc6
-- 
2.34.1