Re: [PATCH] KVM: x86/mmu: Cope with the SS bit being set in #NPF exit codes

[Sean Christopherson <seanjc@google.com>]

On Fri, Oct 09, 2026, Paolo Bonzini wrote:
> When running Hyper-V from Windows 11, KVM is seeing the SS bit set
> in nested page fault error codes produced for shadowed NPT.  The AMD
> manual is unclear about when SS is set, including whether it can be set
> when SupervisorShadowStackEn (bit 6) is not set in the MISC_CTL field
> of the VMCB; in fact, bisection shows that while SHSTK was available
> to guests since Linux 6.18, this started happening with commit
> 687ee95c1b6d ("KVM: nSVM: enable GMET for guests"), which seems unrelated.

It's not unrelated.  As you note below, this is purely because Hyper-V only uses
CET when GMET is available.

> Nevertheless, the unexpected SS bit causes out of bounds accesses in
> permission_fault(), seen as either WARNs or UBSAN reports:
> 
>   kernel: UBSAN: array-index-out-of-bounds in arch/x86/kvm/mmu.h:225:27
>   kernel: index 34 is out of range for type 'u16 [16]'
> 
> (in this case, the page fault error code is 34*2=0x44, i.e. SS|U; the
> report shows SS|U|W happening as well).  Fix the handling of unknown
> error codes by dropping SS and (with a WARN) any bits not among those
> that permission_fault() expects to receive.
> 
> At the same time, given the uncertainty about when hardware sets SS,
> follow the processor's behavior when constructing the fault's error
> code, and preserve the SS bit as passed to FNAME(walk_addr_generic).
> While the behavior is surprising, there's a possibility that Hyper-V
> expects it (Hyper-V does not even enable CET unless it finds MBEC/GMET!),
> so retain the information when reflecting the fault to L1 instead of
> unconditionally discarding it.
> 
> Note that, even though KVM always tries to run L1 with GMET enabled, it
> copies the GMET-enable bit of the VMCB12 to the VMCB02 when L1 requests
> usage of NPT.  Therefore, if the theory suggested by the bisection result
> is correct and GMET enables setting the SS bit as well, this would not
> affect hypervisors that enable NPT but not GMET.
> 
> Reported-by: SimonP <simonp.git@mailbox.org>

Closes: https://lore.kernel.org/all/asUqYEisPp9MJ8mQ@rika.bernkas.tel

> Fixes: 687ee95c1b6d ("KVM: nSVM: enable GMET for guests")
> Cc: stable@vger.kernel.org # 7.2+

This is wrong, I reproduced this with the KUT CET test with gmet=1.  It needs to
be (I would just leave off the kernel version number and let the Fixes take the
backports where it needs to go).

  Fixes: 8db428fd5229 ("KVM: SVM: Enable shadow stack virtualization for SVM")
  Cc: stable@vger.kernel.org

With the above tweaks and the removal of user_fault:

  Reviewed-by: Sean Christopherson <seanjc@google.com>

The only reason the KUT test doesn't hit the WARN is because L0 KVM never takes
a #NPF on the push to the shadow stack.  Thanks to KVM's prefetching and KUT's
handling of free pages (the allocator writes to the page being allocated), it's
all but impossible for the shadow stack push to hita #NPF in L0's domain.

With hacks in KVM and KUT to zap all SPTEs just before running the shadow stack
test:

diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 81a5b32a4bd0b..2ab55fd910d64 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -7683,6 +7683,7 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm)
 {
 	kvm_mmu_zap_all(kvm);
 }
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_arch_flush_shadow_all);
 
 static void kvm_mmu_zap_memslot_pages_and_flush(struct kvm *kvm,
 						struct kvm_memory_slot *slot,
diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index 757c8ea1ee818..a050ebb633b36 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -1828,6 +1828,12 @@ int nested_svm_exit_special(struct vcpu_svm *svm)
 		 * of L1.
 		 */
 		return NESTED_EXIT_HOST;
+	case SVM_EXIT_MSR:
+		if (!svm->vmcb->control.exit_info_1)
+			break;
+
+		kvm_arch_flush_shadow_all(vcpu->kvm);
+		break;
 	default:
 		break;
 	}

diff --git a/x86/cet.c b/x86/cet.c
index bb031b80..050cc1c7 100644
--- a/x86/cet.c
+++ b/x86/cet.c
@@ -225,10 +225,14 @@ static void test_shstk(void)
 	wrmsr(MSR_IA32_PL3_SSP, (u64)(shstk_virt + 0x1000));
 
 	printf("Running user mode Shadow Stack tests\n");
+	wrmsr(MSR_IA32_UCODE_REV, 0);
 	run_in_user(cet_shstk_func, CP_VECTOR, 0, 0, 0, 0, &rvc);
 	report(rvc && exception_error_code() == CP_ERR_NEAR_RET,
 	       "NEAR RET shadow-stack protection test");
 
+	if (1)
+		return;
+
 	run_in_user(cet_shstk_far_ret, CP_VECTOR, 0, 0, 0, 0, &rvc);
 	report(rvc && exception_error_code() == CP_ERR_FAR_RET,
 	       "FAR RET shadow-stack protection test");


I can trigger the WARN:

 ------------[ cut here ]------------
 WARNING: ./arch/x86/kvm/mmu.h:305 at paging64_walk_addr_generic+0x493/0x630 [kvm], CPU#475: CPU 27/KVM/453497
 Modules linked in: kvm_amd kvm irqbypass vfat fat dummy bridge stp llc ccp k10temp i2c_piix4 cdc_ncm cdc_eem cdc_ether usbnet mii xhci_pci xhci_hcd ehci_pci ehci_hcd [last unloaded: kvm]
 CPU: 475 UID: 0 PID: 453497 Comm: CPU 27/KVM Tainted: G S   U              7.3.0-smp--dfc8813a3132f-seanjc-sink #71 PREEMPT 
 Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER
 Hardware name: Google Astoria-Turin/astoria, BIOS 0.20260526.0-0 05/26/2026
 RIP: 0010:paging64_walk_addr_generic+0x493/0x630 [kvm]
 Call Trace:
  <TASK>
  paging64_page_fault+0x55/0x2a0 [kvm]
  kvm_mmu_do_page_fault+0x138/0x210 [kvm]
  kvm_mmu_page_fault+0xce/0x400 [kvm]
  npf_interception+0x14b/0x320 [kvm_amd]
  vcpu_run+0x1571/0x1f00 [kvm]
  kvm_arch_vcpu_ioctl_run+0x2c6/0x600 [kvm]
  kvm_vcpu_ioctl+0x580/0x6b0 [kvm]
  __se_sys_ioctl+0x6d/0xb0
  do_syscall_64+0xfe/0x460
  entry_SYSCALL_64_after_hwframe+0x4b/0x53
 RIP: 0033:0x7f9e1677a17b
  </TASK>

Note, the early return in test_shstk() is necessary to reproduce the splat in my
setup.  I assume it's due to a change in the generated test code that causes KVM
to fault-in (prefetch?) the shadow stack page.  I didn't bother digging that deep.