Re: [PATCH v4 0/7] Add support for AMD IOMMU GAPPI

[Sairaj Kodilkar <sarunkod@amd.com>]

On 8/21/2026 11:26 AM, Sairaj Kodilkar wrote:
> Introduction
> ============
> On newer generations of AMD processors, IOMMU AVIC/x2AVIC guest-mode interrupt
> remapping can use Guest APIC Physical Processor Interrupt (GAPPI) as an
> alternative host-notification path when device interrupts target a vCPU that
> is not running (IRTE[IsRun] = 0).
> 
> With AVIC enabled, the IOMMU posts device interrupts into the guest virtual
> APIC backing page. When the vCPU is not running, KVM must additionally be
> notified so it can schedule the vCPU.
> 
> The legacy notification path is the GA log (GALOG): the IOMMU appends vCPU
> tags to a shared GA log buffer and raises a single GA log interrupt.  KVM
> registers a notifier and scans the buffer to decide which vCPUs to wake.
> Under heavy interrupt load this adds latency and can overflow the buffer
> because all wakeups funnel through one interrupt and one shared log.
> 
> Guest APIC Physical Processor Interrupt (GAPPI), defined in section 2.2.5.4
> of the AMD I/O Virtualization Technology (IOMMU) Specification [1], is an
> alternative.  With GAPPI enabled, the IOMMU still updates the guest vAPIC
> backing page IRR, but may deliver a physical APIC interrupt directly to
> IRTE[Destination], using IRTE[GATag][7:0] as the vector.  This distributes
> host wakeup notifications across CPUs instead of centralizing them in a
> log buffer.
> 
> This series programs guest-mode IRTEs accordingly: IRTE[Destination] carries
> the target host physical APIC ID, IRTE[GATag] is set to
> POSTED_INTR_WAKEUP_VECTOR, and IRTE[GAPPIDis] / IRTE[GALogIntr] are set
> based on whether KVM requests host wakeup.  GAPPI is selected at boot via
> the amd_iommu=gappi kernel parameter on capable hardware, otherwise the
> existing GA log path is unchanged.
> 
> 
> SVM/AMD IOMMU interface changes
> ===============================
> The first four patches refactor the SVM/AMD IOMMU interface ahead of GAPPI.
> 
> The cpu field is renamed to apicid because it carries the host physical
> APIC ID for IRTE[Destination], not a Linux CPU number.
> 
> The ga_log_intr boolean is renamed to wakeup_intr (and the synthetic
> AVIC_PHYSICAL_ID_ENTRY_GA_LOG_INTR shadow bit to
> AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR). wakeup_intr describes KVM's intent
> (request host wakeup while the vCPU is not running), not a specific hardware
> mechanism.
> 
> A separate is_running boolean is added to IOMMU interface because GAPPI
> requires a valid apicid in IRTE[Destination] even when the vCPU is not running. 
> The prior encoding (apicid >= 0 means running, apicid == -1 means not running)
> no longer works once apicid carries the GAPPI destination while IRTE[IsRun] is
> clear.  The IOMMU driver keys IRTE[IsRun] and destination programming off this
> explicit boolean instead of inferring running state from apicid.
> 
> 
> KVM GAPPI wakeup scheme
> =======================
> SVM follows the Intel posted-interrupt wakeup model already used by VMX.
> Each pCPU maintains a list of blocked vCPUs that may be woken by a GAPPI
> delivery to that CPU.  When a vCPU blocks while waiting for a device
> interrupt, SVM enqueues it on the wakeup list of the pCPU on which it was
> previously running (gappi_cpu) and passes that pCPU's physical APIC ID to
> the IOMMU to program IRTE[Destination].  The rationale is that the vCPU is
> likely to run again on the same pCPU, which is common when vCPUs are pinned;
> targeting GAPPI notifications there reduces unnecessary VMEXITs from GAPPI
> deliveries on other CPUs.  When the vCPU is scheduled in again, it is
> removed from the list and IRTE[Destination] is updated to the current pCPU.
> 
> List maintenance is moved from VMX layer to common KVM layer so that both VMX
> and SVM layer can share the same list handling code. 
> 
> Note
> ====
> kvm_pi_enable_wakeup_handler() and kvm_pi_disable_wakeup_handler() take the
> target pCPU explicitly instead of using vcpu->cpu, because on SVM the two can
> differ. avic_vcpu_load() returns early while the vCPU is blocking and leaves
> the target wakeup CPU untouched, but the generic layer still refreshes
> vcpu->cpu on every load. Using vcpu->cpu would then unlink the vCPU under the
> wrong per-pCPU lock, so SVM tracks the pCPU it last programmed in
> svm->gappi_cpu and passes that instead.
> 
> 
> Changes since v3
> ================
> https://lore.kernel.org/all/20260713105033.15405-1-sarunkod@amd.com/
> 
> Patch 3:
>   - Make amd_iommu_activate_guest_mode() as wrapper around
>     __amd_iommu_activate_guest_mode() to in order to hardcode is_running
>     flag to "true" for external calls
> 
> Patch 5:
>   - Newly introduced, moves wakeup list handling from VMX to the KVM layer
> 
> Patch 6:
>   - Use common list handling functions introduced in patch 5.
>   - Remove list addition during pi_update_irte by moving list handling
>     functions before list_empty(&svm->ir_list) check.
> 
> 
> Changes since v2
> ================
> https://lore.kernel.org/linux-iommu/20260708091408.12106-1-sarunkod@amd.com/
> 
> Patch[1-6]
>   - Expand commit messages to explain GAPPI, the interface changes, and the
>     per-CPU wakeup list scheme [Sean].
> 
> Patch[1-3]
>   - Split the monolithic SVM/IOMMU API refactor into four preparatory
>     patches [Sean]
>   - Rename posted_intr to wakeup_intr to reflect host wakeup intent, not
>     guest interrupt posting [Sean]
>   - Pass vCPU running status with a extra parameter (is_running) instead of
>     flags.
> 
> Patch[4,5]
>   - Move ga_tag=POSTED_INTR_WAKEUP_VECTOR setting from IOMMU to SVM layer.
> 
> 
> Changes since V1:
> ================
> https://lore.kernel.org/all/20260626105906.14577-1-sarunkod@amd.com/
> 
> Patch4
>     - Disable interrupts while holding wakeup list lock inside [sashiko]
>       avic_add_vcpu_to_gappi_wakeup_list and avic_remove_vcpu_from_gappi_wakeup_list
>     - Unregister posted_intr_wakeup_handler during module unload [sashiko]
> 
> Patch5
>     - Disable GAPPI feature during kexec and suspend path [sashiko]
> 
> 
> ------
> [1] https://docs.amd.com/v/u/en-US/48882_3.11_IOMMU_PUB
> 
> ------
> Base: 89516a1c6837156f43525ef49fdbd781145942f5

Apologies for wrong base commit, Please use following base commit

Base: 8cd9520d35a6c38db6567e97dd93b1f11f185dc6

> 
> Sairaj Kodilkar (7):
>   iommu/amd: KVM: SVM: Rename cpu to apicid in IOMMU interface
>   iommu/amd: KVM: SVM: Rename ga_log_intr to wakeup_intr in IOMMU
>     interface
>   iommu/amd: KVM: SVM: Add explicit vCPU running state to IOMMU
>     interface
>   iommu/amd: Program guest-mode IRTEs for GAPPI wakeup when IRTE[IsRun]
>     = 0
>   KVM: VMX: Factor out wakeup list handling code to KVM
>   KVM: SVM: Add support for AMD IOMMU Guest APIC Physical Processor
>     Interrupt (GAPPI)
>   iommu/amd: Provide kernel command line option to enable GAPPI
> 
>  .../admin-guide/kernel-parameters.txt         |   3 +-
>  arch/x86/include/asm/irq_remapping.h          |   5 +-
>  arch/x86/include/asm/kvm-x86-ops.h            |   1 +
>  arch/x86/include/asm/kvm_host.h               |   1 +
>  arch/x86/include/asm/svm.h                    |   9 +-
>  arch/x86/kvm/svm/avic.c                       | 109 +++++++++++++----
>  arch/x86/kvm/svm/svm.c                        |   3 +
>  arch/x86/kvm/svm/svm.h                        |   5 +
>  arch/x86/kvm/vmx/common.h                     |   3 -
>  arch/x86/kvm/vmx/main.c                       |   6 +
>  arch/x86/kvm/vmx/posted_intr.c                |  82 +------------
>  arch/x86/kvm/vmx/tdx.c                        |   1 -
>  arch/x86/kvm/vmx/vmx.c                        |  14 +--
>  arch/x86/kvm/vmx/x86_ops.h                    |   1 +
>  arch/x86/kvm/x86.c                            | 111 ++++++++++++++++++
>  drivers/iommu/amd/amd_iommu_types.h           |   6 +-
>  drivers/iommu/amd/init.c                      |  31 ++++-
>  drivers/iommu/amd/iommu.c                     |  65 ++++++----
>  include/linux/amd-iommu.h                     |  16 ++-
>  include/linux/kvm_host.h                      |   6 +
>  20 files changed, 323 insertions(+), 155 deletions(-)
>