[PATCH 0/3] KVM: x86/SVM: Fixes for AMD ERAPS virtualization

[PATCH 0/3] KVM: x86/SVM: Fixes for AMD ERAPS virtualization

[Jim Mattson]

This series fixes three issues in KVM's virtualization of Enhanced Return
Address Predictor Security (ERAPS):

Patch 1 moves ALLOW_LARGER_RAP configuration from init_vmcb() to
svm_vcpu_after_set_cpuid(). On initial vCPU creation, init_vmcb() runs
before guest CPUID is configured via KVM_SET_CPUID2, leaving the BSP
without ALLOW_LARGER_RAP. Configuring ALLOW_LARGER_RAP in
svm_vcpu_after_set_cpuid() handles all vCPUs in one place.

Patch 2 moves ERAPS dirty tracking into svm_flush_tlb_asid(). AMD ERAPS
architecturally requires clearing the Return Address Predictor (RAP/RSB) on
all implicit TLB invalidations. Centralizing dirty tracking in
svm_flush_tlb_asid() ensures that local/current ASID flushes
(e.g. same-value CR3 reloads under shadow paging, CR4.PAE/SMEP toggles,
single-context INVPCID) correctly mark ERAPS dirty and clear the RAP on the
next VMRUN.

Patch 3 adds a KVM_REQ_TLB_FLUSH_GUEST request to kvm_mtrr_set_msr() when
MTRR MSRs are written via WRMSR. Per AMD APM and Intel SDM, MTRR writes are
implicit TLB invalidations. Requesting a guest TLB flush restores x86
architectural compliance and ensures ERAPS RAP clearing on MTRR writes.

Jim Mattson (3):
  KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid()
  KVM: SVM: Dirty ERAPS register on all ASID TLB flushes
  KVM: x86: Flush guest TLB on MTRR MSR writes

 arch/x86/kvm/mtrr.c    |  1 +
 arch/x86/kvm/svm/svm.c | 19 ++++++++-----------
 2 files changed, 9 insertions(+), 11 deletions(-)

-- 
2.55.0.229.g6434b31f56-goog

[PATCH 1/3] KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid()

[Jim Mattson]

On initial vCPU creation, init_vmcb() is called before userspace configures
guest CPUID via KVM_SET_CPUID2. Consequently, ERAP_CONTROL_ALLOW_LARGER_RAP
is never set during initial vCPU creation. If an AP later receives an INIT
signal after CPUID has been set, init_vmcb() would set the bit for that AP,
but the BSP would be left with ALLOW_LARGER_RAP clear.

Move the ALLOW_LARGER_RAP configuration from init_vmcb() into
svm_vcpu_after_set_cpuid() to dynamically set or clear the bit in vmcb01's
erap_ctl whenever guest CPUID changes.

Remove the now redundaant code to set ALLOW_LARGER_RAP inside init_vmcb().

Fixes: db5e82496492 ("KVM: SVM: Virtualize and advertise support for ERAPS")
Assisted-by: Gemini:Gemini-Next
Signed-off-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/kvm/svm/svm.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 4d2bacd00ec4..b76c6a76ce37 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -1243,9 +1243,6 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)
 		svm_clr_intercept(svm, INTERCEPT_PAUSE);
 	}
 
-	if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS))
-		svm->vmcb->control.erap_ctl |= ERAP_CONTROL_ALLOW_LARGER_RAP;
-
 	if (enable_apicv && irqchip_in_kernel(vcpu->kvm))
 		avic_init_vmcb(svm, vmcb);
 
@@ -4736,6 +4733,11 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 	if (guest_cpuid_is_intel_compatible(vcpu))
 		guest_cpu_cap_clear(vcpu, X86_FEATURE_V_VMSAVE_VMLOAD);
 
+	if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS))
+		svm->vmcb01.ptr->control.erap_ctl |= ERAP_CONTROL_ALLOW_LARGER_RAP;
+	else
+		svm->vmcb01.ptr->control.erap_ctl &= ~ERAP_CONTROL_ALLOW_LARGER_RAP;
+
 	if (is_sev_guest(vcpu))
 		sev_vcpu_after_set_cpuid(svm);
 }
-- 
2.55.0.229.g6434b31f56-goog

[PATCH 2/3] KVM: SVM: Dirty ERAPS register on all ASID TLB flushes

[Jim Mattson]

Per the AMD APM, ERAPS clears the Return Address Predictor (RAP/RSB) on all
implicit TLB invalidations (e.g. modifying certain CR[04] bits).

Move kvm_register_mark_dirty(vcpu, VCPU_REG_ERAPS) into
svm_flush_tlb_asid() so that any ASID-level TLB flush marks ERAPS dirty.
Centralizing the ERAPS dirty call in svm_flush_tlb_asid() ensures that all
ASID flushes (flush_tlb_current, flush_tlb_all, flush_tlb_guest) properly
set ERAP_CONTROL_CLEAR_RAP on the next VMRUN.

Remove the now redundant svm_flush_tlb_guest() wrapper.

Fixes: db5e82496492 ("KVM: SVM: Virtualize and advertise support for ERAPS")
Assisted-by: Gemini:Gemini-Next
Signed-off-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/kvm/svm/svm.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index b76c6a76ce37..2514c6516fa1 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -4168,6 +4168,8 @@ static void svm_flush_tlb_asid(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
+	kvm_register_mark_dirty(vcpu, VCPU_REG_ERAPS);
+
 	/*
 	 * Unlike VMX, SVM doesn't provide a way to flush only NPT TLB entries.
 	 * A TLB flush for the current ASID flushes both "host" and "guest" TLB
@@ -4226,13 +4228,6 @@ static void svm_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t gva)
 	invlpga(gva, svm->vmcb->control.asid);
 }
 
-static void svm_flush_tlb_guest(struct kvm_vcpu *vcpu)
-{
-	kvm_register_mark_dirty(vcpu, VCPU_REG_ERAPS);
-
-	svm_flush_tlb_asid(vcpu);
-}
-
 static inline void sync_cr8_to_lapic(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
@@ -5380,7 +5375,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = {
 	.flush_tlb_all = svm_flush_tlb_all,
 	.flush_tlb_current = svm_flush_tlb_current,
 	.flush_tlb_gva = svm_flush_tlb_gva,
-	.flush_tlb_guest = svm_flush_tlb_guest,
+	.flush_tlb_guest = svm_flush_tlb_asid,
 
 	.vcpu_pre_run = svm_vcpu_pre_run,
 	.vcpu_run = svm_vcpu_run,
-- 
2.55.0.229.g6434b31f56-goog

[PATCH 3/3] KVM: x86: Flush guest TLB on MTRR MSR writes

[Jim Mattson]

Per both the AMD APM and the Intel SDM, writing to an MTRR with WRMSR is an
implicit TLB invalidation that invalidates all TLB entries (including
global entries). [The APM uses the word, "update," which could be construed
to mean "modify," but it is unclear.]

Previously, kvm_mtrr_set_msr() stored the updated MTRR state without
requesting a TLB flush. This broke x86 architectural compliance by leaving
existing TLB entries in hardware. Additionally, on AMD CPUs supporting
ERAPS, omitting the TLB flush request meant VCPU_REG_ERAPS was never
dirtied on MTRR writes, leaving the Return Address Predictor (RAP/RSB)
uncleared.

Issue a KVM_REQ_TLB_FLUSH_GUEST request on every MTRR MSR write to match
x86 architectural semantics.

Fixes: 9ba075a664df ("KVM: MTRR support")
Assisted-by: Gemini:Gemini-Next
Signed-off-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/kvm/mtrr.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/kvm/mtrr.c b/arch/x86/kvm/mtrr.c
index 6f74e2b27c1e..ca5e5c7f7022 100644
--- a/arch/x86/kvm/mtrr.c
+++ b/arch/x86/kvm/mtrr.c
@@ -105,6 +105,7 @@ int kvm_mtrr_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data)
 		return 1;
 
 	*mtrr = data;
+	kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu);
 	return 0;
 }
 
-- 
2.55.0.229.g6434b31f56-goog

Re: [PATCH 1/3] KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid()

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The ERAP_CONTROL_ALLOW_LARGER_RAP bit in the VMCB is permanently lost if the vCPU is reset due to a triple-fault (SHUTDOWN intercept).
--

--- Patch [1]: [PATCH 1/3] KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid() ---
commit 7fe0a6077a4e214f36b5f568e5752ff88dbf1752
Author: Jim Mattson <jmattson@google.com>

KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid()

This commit moves the configuration of the ALLOW_LARGER_RAP bit from
init_vmcb() to svm_vcpu_after_set_cpuid() so that it is properly set
when userspace configures the guest CPUID. It removes the previous
initialization logic from init_vmcb().

> diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
> index ef69a51ab27f9..11e023abc7ab9 100644
> --- a/arch/x86/kvm/svm/svm.c
> +++ b/arch/x86/kvm/svm/svm.c
> @@ -1243,9 +1243,6 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)
>  		svm_clr_intercept(svm, INTERCEPT_PAUSE);
>  	}
>  
> -	if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS))
> -		svm->vmcb->control.erap_ctl |= ERAP_CONTROL_ALLOW_LARGER_RAP;
> -

[Severity: High]
Does removing this initialization cause the ALLOW_LARGER_RAP bit to be
permanently lost after a triple-fault?

When a guest triggers a triple fault, KVM intercepts the shutdown and wipes
the VMCB:

arch/x86/kvm/svm/svm.c:shutdown_interception() {
    ...
    clear_page(svm->vmcb);
    kvm_vcpu_reset(vcpu, true);
    ...
}

kvm_vcpu_reset() then calls init_vmcb() to reinitialize the VMCB. Since the
ALLOW_LARGER_RAP setup is removed from init_vmcb(), and userspace typically
does not call KVM_SET_CPUID2 during a soft VM reset, will the ERAPS capability
remain disabled indefinitely after the reset?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260716232524.2092085-1-jmattson@google.com?part=1