[GIT PULL] KVM: x86: SVM+SEV changes

[Sean Christopherson <seanjc@google.com>]

This is the full set of SVM+SEV changes.  The end goal of the SEV changes, after
fixing a few fatal bugs, is to add a lockdep assertion to ensure that kvm->lock
is held when checking if the VM is an SEV+ guest.  This is at least the second
fatal bug we've had due to SEV+ state being unwound on failure, and lack of
formal-ish rules makes it hard to reason about the safety of any related code,
e.g. when reviewing new code.

This has a superficial (I can't even figure out why git treats it as a conflict,
I think it's both deleting white space or something?) syntactic conflict with the
"vmxon" PULL request; just take this one.

There's a syntactic conflict with the "nested" PULL request (this is what I
see when merging the "nested" one first):

@@@ -870,8 -881,8 +886,8 @@@ void svm_enable_lbrv(struct kvm_vcpu *v
  
  static void __svm_disable_lbrv(struct kvm_vcpu *vcpu)
  {
-       KVM_BUG_ON(sev_es_guest(vcpu->kvm), vcpu->kvm);
+       KVM_BUG_ON(is_sev_es_guest(vcpu), vcpu->kvm);
 -      to_svm(vcpu)->vmcb->control.virt_ext &= ~LBR_CTL_ENABLE_MASK;
 +      to_svm(vcpu)->vmcb->control.misc_ctl2 &= ~SVM_MISC2_ENABLE_V_LBR;
  }

and a semantic conflict with kvm/master due to the CR8 interception fix:

diff --cc arch/x86/kvm/svm/avic.c
index 2885c5993ebc,7056c4891f93..adf211860949
--- a/arch/x86/kvm/svm/avic.c
+++ b/arch/x86/kvm/svm/avic.c
@@@ -226,9 -237,6 +237,9 @@@ static void avic_deactivate_vmcb(struc
        vmcb->control.int_ctl &= ~(AVIC_ENABLE_MASK | X2APIC_MODE_MASK);
        vmcb->control.avic_physical_id &= ~AVIC_PHYSICAL_MAX_INDEX_MASK;
  
-       if (!sev_es_guest(svm->vcpu.kvm))
++      if (!is_sev_es_guest(&svm->vcpu))
 +              svm_set_intercept(svm, INTERCEPT_CR8_WRITE);
 +
        /*
         * If running nested and the guest uses its own MSR bitmap, there
         * is no need to update L0's msr bitmap


The following changes since commit 11439c4635edd669ae435eec308f4ab8a0804808:

  Linux 7.0-rc2 (2026-03-01 15:39:31 -0800)

are available in the Git repository at:

  https://github.com/kvm-x86/linux.git tags/kvm-x86-svm-7.1

for you to fetch changes up to bc0932cf9b9917e826871db947398aa2b62789b2:

  KVM: SEV: Goto an existing error label if charging misc_cg for an ASID fails (2026-04-09 12:00:24 -0700)

----------------------------------------------------------------
KVM SVM changes for 7.1

 - Fix and optimize IRQ window inhibit handling for AVIC (the tracking needs to
   be per-vCPU, e.g. so that KVM doesn't prematurely re-enable AVIC if multiple
   vCPUs have to-be-injected IRQs).

 - Fix an undefined behavior warning where a crafty userspace can read the
   "avic" module param before it's fully initialized.

 - Fix a (likely benign) bug in the "OS-visible workarounds" handling, where
   KVM could clobber state when enabling virtualization on multiple CPUs in
   parallel, and clean up and optimize the code.

 - Drop a WARN in KVM_MEMORY_ENCRYPT_REG_REGION where KVM complains about a
   "too large" size based purely on user input, and clean up and harden the
   related pinning code.

 - Disallow synchronizing a VMSA of an already-launched/encrypted vCPU, as
   doing so for an SNP guest will trigger an RMP violation #PF and crash the
   host.

 - Protect all of sev_mem_enc_register_region() with kvm->lock to ensure
   sev_guest() is stable for the entire of the function.

 - Lock all vCPUs when synchronizing VMSAs for SNP guests to ensure the VMSA
   page isn't actively being used.

 - Overhaul KVM's APIs for detecting SEV+ guests so that VM-scoped queries are
   required to hold kvm->lock (KVM has had multiple bugs due "is SEV?" checks
   becoming stale), enforced by lockdep.  Add and use vCPU-scoped APIs when
   possible/appropriate, as all checks that originate from a vCPU are
   guaranteed to be stable.

 - Convert a pile of kvm->lock SEV code to guard().

----------------------------------------------------------------
Carlos López (5):
      KVM: SEV: use mutex guard in snp_launch_update()
      KVM: SEV: use mutex guard in sev_mem_enc_ioctl()
      KVM: SEV: use mutex guard in sev_mem_enc_unregister_region()
      KVM: SEV: use mutex guard in snp_handle_guest_req()
      KVM: SVM: Move lock-protected allocation of SEV ASID into a separate helper

Gal Pressman (1):
      KVM: SVM: Fix UBSAN warning when reading avic parameter

Li RongQing (1):
      KVM: SVM: Mark module parameters as __ro_after_init for security and performance

Sean Christopherson (30):
      KVM: SVM: Fix clearing IRQ window inhibit with nested guests
      KVM: SVM: Fix IRQ window inhibit handling across multiple vCPUs
      KVM: SVM: Optimize IRQ window inhibit handling
      KVM: Isolate apicv_update_lock and apicv_nr_irq_window_req in a cacheline
      KVM: SVM: Serialize updates to global OS-Visible Workarounds variables
      KVM: SVM: Skip OSVW MSR reads if KVM is treating all errata as present
      KVM: SVM: Extract OS-visible workarounds setup to helper function
      KVM: SVM: Skip OSVW variable updates if current CPU's errata are a subset
      KVM: SVM: Skip OSVW MSR reads if current CPU doesn't support the feature
      KVM: SEV: Drop WARN on large size for KVM_MEMORY_ENCRYPT_REG_REGION
      KVM: SEV: Drop useless sanity checks in sev_mem_enc_register_region()
      KVM: SEV: Disallow pinning more pages than exist in the system
      KVM: SEV: Use PFN_DOWN() to simplify "number of pages" math when pinning memory
      KVM: SEV: Use kvzalloc_objs() when pinning userpages
      KVM: selftests: Remove duplicate LAUNCH_UPDATE_VMSA call in SEV-ES migrate test
      KVM: SEV: Reject attempts to sync VMSA of an already-launched/encrypted vCPU
      KVM: SEV: Protect *all* of sev_mem_enc_register_region() with kvm->lock
      KVM: SEV: Disallow LAUNCH_FINISH if vCPUs are actively being created
      KVM: SEV: Lock all vCPUs when synchronzing VMSAs for SNP launch finish
      KVM: SEV: Lock all vCPUs for the duration of SEV-ES VMSA synchronization
      KVM: SEV: Provide vCPU-scoped accessors for detecting SEV+ guests
      KVM: SEV: Add quad-underscore version of VM-scoped APIs to detect SEV+ guests
      KVM: SEV: Document the SEV-ES check when querying SMM support as "safe"
      KVM: SEV: Move standard VM-scoped helpers to detect SEV+ guests to sev.c
      KVM: SEV: Move SEV-specific VM initialization to sev.c
      KVM: SEV: WARN on unhandled VM type when initializing VM
      KVM: SEV: Hide "struct kvm_sev_info" behind CONFIG_KVM_AMD_SEV=y
      KVM: SEV: Document that checking for SEV+ guests when reclaiming memory is "safe"
      KVM: SEV: Assert that kvm->lock is held when querying SEV+ support
      KVM: SEV: Goto an existing error label if charging misc_cg for an ASID fails

 arch/x86/include/asm/kvm_host.h                    |  29 +-
 arch/x86/kvm/svm/avic.c                            |  17 +-
 arch/x86/kvm/svm/sev.c                             | 374 ++++++++++++---------
 arch/x86/kvm/svm/svm.c                             | 270 ++++++++-------
 arch/x86/kvm/svm/svm.h                             |  37 +-
 arch/x86/kvm/x86.c                                 |  45 ++-
 include/linux/kvm_host.h                           |   7 +
 .../testing/selftests/kvm/x86/sev_migrate_tests.c  |   2 -
 8 files changed, 487 insertions(+), 294 deletions(-)