[PATCH v5 00/10] KVM: x86: nSVM: Improve PAT virtualization

[PATCH v5 00/10] KVM: x86: nSVM: Improve PAT virtualization

[Jim Mattson]

Currently, KVM's implementation of nested SVM treats the PAT MSR the same
way whether or not nested NPT is enabled: L1 and L2 share a single
PAT. However, the APM specifies that when nested NPT is enabled, the host
(L1) and the guest (L2) should have independent PATs: hPAT for L1 and gPAT
for L2. This patch series implements the architectural specification in
KVM.

Use the existing PAT MSR (vcpu->arch.pat) for hPAT. Add a new field,
svm->nested.gpat, for gPAT. With nested NPT enabled, redirect guest
accesses to the IA32_PAT MSR to gPAT. All other accesses, including
userspace accesses via KVM_{GET,SET}_MSRS, continue to reference hPAT.  The
special handling of userspace accesses ensures save/restore forward
compatibility (i.e. resuming a new checkpoint on an older kernel). When an
old kernel restores a checkpoint from a new kernel, the gPAT will be lost,
and L2 will simply use L1's PAT, which is the existing behavior of the old
kernel anyway.

v1: https://lore.kernel.org/kvm/20260113003016.3511895-1-jmattson@google.com/
v2: https://lore.kernel.org/kvm/20260115232154.3021475-1-jmattson@google.com/
v3: https://lore.kernel.org/kvm/20260205214326.1029278-1-jmattson@google.com/
v4: https://lore.kernel.org/kvm/20260212155905.3448571-1-jmattson@google.com/

  v4 -> v5:
  * Separate commit to remove vmcb_is_dirty() from first v4 commit [Yosry, Sean]
  * Introduce svm_get_pat and svm_set_pat to keep all hPAT vs gPAT logic
    together in one place.
  * Remove the no longer common logic for get/set IA32_PAT. [Sean]
  * Make vmcb02's g_pat authoritative for gPAT.
  * Clear legacy_gpat_semantics when forcing the vCPU out of guest mode and
    when processing a second KVM_SET_NESTED_STATE that doesn't have legacy
    semantics. [Sean]

Note that this series should be applied after Yosry's v5 "Nested SVM fixes,
cleanups, and hardening."

Jim Mattson (10):
  KVM: SVM: Remove vmcb_is_dirty()
  KVM: x86: nSVM: Clear VMCB_NPT clean bit when updating hPAT from guest
    mode
  KVM: x86: nSVM: Cache and validate vmcb12 g_pat
  KVM: x86: nSVM: Set vmcb02.g_pat correctly for nested NPT
  KVM: x86: nSVM: Redirect IA32_PAT accesses to either hPAT or gPAT
  KVM: x86: Remove common handling of MSR_IA32_CR_PAT
  KVM: x86: nSVM: Save gPAT to vmcb12.g_pat on VMEXIT
  KVM: x86: nSVM: Save/restore gPAT with KVM_{GET,SET}_NESTED_STATE
  KVM: x86: nSVM: Handle restore of legacy nested state
  KVM: selftests: nSVM: Add svm_nested_pat test

 arch/x86/include/uapi/asm/kvm.h               |   5 +
 arch/x86/kvm/svm/nested.c                     |  63 +++-
 arch/x86/kvm/svm/svm.c                        |  57 +++-
 arch/x86/kvm/svm/svm.h                        |  20 +-
 arch/x86/kvm/vmx/vmx.c                        |   9 +-
 arch/x86/kvm/x86.c                            |   9 -
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/x86/svm_nested_pat_test.c   | 298 ++++++++++++++++++
 8 files changed, 422 insertions(+), 40 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/svm_nested_pat_test.c

-- 
2.53.0.371.g1d285c8824-goog

[PATCH v5 01/10] KVM: x86: SVM: Remove vmcb_is_dirty()

[Jim Mattson]

After commit dd26d1b5d6ed ("KVM: nSVM: Cache all used fields from VMCB12"),
vmcb_is_dirty() has no callers. Remove the function.

Signed-off-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/kvm/svm/svm.h | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index 0bb93879abfe..8f9e6a39659c 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -434,11 +434,6 @@ static inline void vmcb_mark_dirty(struct vmcb *vmcb, int bit)
 	vmcb->control.clean &= ~(1 << bit);
 }
 
-static inline bool vmcb_is_dirty(struct vmcb *vmcb, int bit)
-{
-        return !test_bit(bit, (unsigned long *)&vmcb->control.clean);
-}
-
 static inline bool vmcb12_is_dirty(struct vmcb_ctrl_area_cached *control, int bit)
 {
 	return !test_bit(bit, (unsigned long *)&control->clean);
-- 
2.53.0.371.g1d285c8824-goog

[PATCH v5 02/10] KVM: x86: nSVM: Clear VMCB_NPT clean bit when updating hPAT from guest mode

[Jim Mattson]

When running an L2 guest and writing to MSR_IA32_CR_PAT, the host PAT value
is stored in both vmcb01's g_pat field and vmcb02's g_pat field, but the
clean bit was only being cleared for vmcb02.

Introduce the helper vmcb_set_gpat() which sets vmcb->save.g_pat and marks
the VMCB dirty for VMCB_NPT. Use this helper in both svm_set_msr() for
updating vmcb01 and in nested_vmcb02_compute_g_pat() for updating vmcb02,
ensuring both VMCBs' NPT fields are properly marked dirty.

Fixes: 4995a3685f1b ("KVM: SVM: Use a separate vmcb for the nested L2 guest")
Signed-off-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/kvm/svm/nested.c | 2 +-
 arch/x86/kvm/svm/svm.c    | 3 +--
 arch/x86/kvm/svm/svm.h    | 6 ++++++
 3 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index d80b1bde6630..b72a1f3c4144 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -707,7 +707,7 @@ void nested_vmcb02_compute_g_pat(struct vcpu_svm *svm)
 		return;
 
 	/* FIXME: merge g_pat from vmcb01 and vmcb12.  */
-	svm->nested.vmcb02.ptr->save.g_pat = svm->vmcb01.ptr->save.g_pat;
+	vmcb_set_gpat(svm->nested.vmcb02.ptr, svm->vmcb01.ptr->save.g_pat);
 }
 
 static void nested_vmcb02_prepare_save(struct vcpu_svm *svm)
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index be9d562fabde..6c41f2317777 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -2924,10 +2924,9 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
 		if (ret)
 			break;
 
-		svm->vmcb01.ptr->save.g_pat = data;
+		vmcb_set_gpat(svm->vmcb01.ptr, data);
 		if (is_guest_mode(vcpu))
 			nested_vmcb02_compute_g_pat(svm);
-		vmcb_mark_dirty(svm->vmcb, VMCB_NPT);
 		break;
 	case MSR_IA32_SPEC_CTRL:
 		if (!msr->host_initiated &&
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index 8f9e6a39659c..9850ed01e16e 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -439,6 +439,12 @@ static inline bool vmcb12_is_dirty(struct vmcb_ctrl_area_cached *control, int bi
 	return !test_bit(bit, (unsigned long *)&control->clean);
 }
 
+static inline void vmcb_set_gpat(struct vmcb *vmcb, u64 data)
+{
+	vmcb->save.g_pat = data;
+	vmcb_mark_dirty(vmcb, VMCB_NPT);
+}
+
 static __always_inline struct vcpu_svm *to_svm(struct kvm_vcpu *vcpu)
 {
 	return container_of(vcpu, struct vcpu_svm, vcpu);
-- 
2.53.0.371.g1d285c8824-goog

[PATCH v5 03/10] KVM: x86: nSVM: Cache and validate vmcb12 g_pat

[Jim Mattson]

Cache g_pat from vmcb12 in vmcb_save_area_cached to avoid TOCTTOU issues,
and add a validity check so that when nested paging is enabled for vmcb12,
an invalid g_pat at emulated VMRUN causes an immediate VMEXIT with exit
code VMEXIT_INVALID, as specified in the APM, volume 2: "Nested Paging and
VMRUN/VMEXIT."

Fixes: 3d6368ef580a ("KVM: SVM: Add VMRUN handler")
Signed-off-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/kvm/svm/nested.c | 17 +++++++++++++----
 arch/x86/kvm/svm/svm.h    |  1 +
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index b72a1f3c4144..91b35adb83f8 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -426,7 +426,8 @@ static bool nested_vmcb_check_controls(struct kvm_vcpu *vcpu,
 
 /* Common checks that apply to both L1 and L2 state.  */
 static bool nested_vmcb_check_save(struct kvm_vcpu *vcpu,
-				   struct vmcb_save_area_cached *save)
+				   struct vmcb_save_area_cached *save,
+				   bool check_gpat)
 {
 	if (CC(!(save->efer & EFER_SVME)))
 		return false;
@@ -462,6 +463,9 @@ static bool nested_vmcb_check_save(struct kvm_vcpu *vcpu,
 	if (CC(!kvm_valid_efer(vcpu, save->efer)))
 		return false;
 
+	if (check_gpat && CC(!kvm_pat_valid(save->g_pat)))
+		return false;
+
 	return true;
 }
 
@@ -573,6 +577,7 @@ static void __nested_copy_vmcb_save_to_cache(struct vmcb_save_area_cached *to,
 
 	to->rax = from->rax;
 	to->cr2 = from->cr2;
+	to->g_pat = from->g_pat;
 
 	svm_copy_lbrs(to, from);
 }
@@ -1036,7 +1041,8 @@ int enter_svm_guest_mode(struct kvm_vcpu *vcpu, u64 vmcb12_gpa, bool from_vmrun)
 
 	enter_guest_mode(vcpu);
 
-	if (!nested_vmcb_check_save(vcpu, &svm->nested.save) ||
+	if (!nested_vmcb_check_save(vcpu, &svm->nested.save,
+				    nested_npt_enabled(svm)) ||
 	    !nested_vmcb_check_controls(vcpu, &svm->nested.ctl,
 					svm->vmcb01.ptr->save.cr0))
 		return -EINVAL;
@@ -2006,13 +2012,16 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
 
 	/*
 	 * Validate host state saved from before VMRUN (see
-	 * nested_svm_check_permissions).
+	 * nested_svm_check_permissions). Note that the g_pat field is not
+	 * validated, because (a) it may have been clobbered by SMM before
+	 * KVM_GET_NESTED_STATE, and (b) it is not loaded at emulated
+	 * #VMEXIT.
 	 */
 	__nested_copy_vmcb_save_to_cache(&save_cached, save);
 	if (!(save->cr0 & X86_CR0_PG) ||
 	    !(save->cr0 & X86_CR0_PE) ||
 	    (save->rflags & X86_EFLAGS_VM) ||
-	    !nested_vmcb_check_save(vcpu, &save_cached))
+	    !nested_vmcb_check_save(vcpu, &save_cached, false))
 		goto out_free;
 
 
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index 9850ed01e16e..a49c48459e0b 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -161,6 +161,7 @@ struct vmcb_save_area_cached {
 	u64 isst_addr;
 	u64 rax;
 	u64 cr2;
+	u64 g_pat;
 	u64 dbgctl;
 	u64 br_from;
 	u64 br_to;
-- 
2.53.0.371.g1d285c8824-goog

[PATCH v5 04/10] KVM: x86: nSVM: Set vmcb02.g_pat correctly for nested NPT

[Jim Mattson]

When nested NPT is enabled in vmcb12, copy the (cached and validated)
vmcb12 g_pat field to the guest PAT register. Under KVM, the guest PAT
register lives in svm->nested.save.g_pat.

When NPT is enabled, but nested NPT is disabled, copy L1's IA32_PAT MSR to
the vmcb02 g_pat field, since L2 shares the IA32_PAT MSR with L1.

When NPT is disabled, the g_pat field is ignored by hardware.

Fixes: 15038e147247 ("KVM: SVM: obey guest PAT")
Signed-off-by: Jim Mattson <jmattson@google.com>
Reviewed-by: Yosry Ahmed <yosry.ahmed@linux.dev>
---
 arch/x86/kvm/svm/nested.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index 91b35adb83f8..dc8275837120 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -724,9 +724,6 @@ static void nested_vmcb02_prepare_save(struct vcpu_svm *svm)
 	struct vmcb *vmcb02 = svm->nested.vmcb02.ptr;
 	struct kvm_vcpu *vcpu = &svm->vcpu;
 
-	nested_vmcb02_compute_g_pat(svm);
-	vmcb_mark_dirty(vmcb02, VMCB_NPT);
-
 	/* Load the nested guest state */
 	if (svm->nested.vmcb12_gpa != svm->nested.last_vmcb12_gpa) {
 		new_vmcb12 = true;
@@ -757,6 +754,13 @@ static void nested_vmcb02_prepare_save(struct vcpu_svm *svm)
 		vmcb_mark_dirty(vmcb02, VMCB_CET);
 	}
 
+	if (nested_npt_enabled(svm)) {
+		if (unlikely(new_vmcb12 || vmcb12_is_dirty(control, VMCB_NPT)))
+			vmcb_set_gpat(vmcb02, svm->nested.save.g_pat);
+	} else if (npt_enabled) {
+		vmcb_set_gpat(vmcb02, vcpu->arch.pat);
+	}
+
 	kvm_set_rflags(vcpu, save->rflags | X86_EFLAGS_FIXED);
 
 	svm_set_efer(vcpu, svm->nested.save.efer);
-- 
2.53.0.371.g1d285c8824-goog

[PATCH v5 05/10] KVM: x86: nSVM: Redirect IA32_PAT accesses to either hPAT or gPAT

[Jim Mattson]

When the vCPU is in guest mode with nested NPT enabled, guest accesses to
IA32_PAT are redirected to the gPAT register, which is stored in VMCB02's
g_pat field.

Non-guest accesses (e.g. from userspace) to IA32_PAT are always redirected
to hPAT, which is stored in vcpu->arch.pat.

This is architected behavior. It also makes it possible to restore a new
checkpoint on an old kernel with reasonable semantics. After the restore,
gPAT will be lost, and L2 will run on L1's PAT. Note that the old kernel
would have always run L2 on L1's PAT.

Add WARN_ON_ONCE to flag any host-initiated accesses originating from KVM
itself rather than userspace.

Fixes: 15038e147247 ("KVM: SVM: obey guest PAT")
Signed-off-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/kvm/svm/nested.c |  9 -------
 arch/x86/kvm/svm/svm.c    | 52 ++++++++++++++++++++++++++++++++++-----
 arch/x86/kvm/svm/svm.h    |  1 -
 3 files changed, 46 insertions(+), 16 deletions(-)

diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index dc8275837120..69b577a4915c 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -706,15 +706,6 @@ static int nested_svm_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3,
 	return 0;
 }
 
-void nested_vmcb02_compute_g_pat(struct vcpu_svm *svm)
-{
-	if (!svm->nested.vmcb02.ptr)
-		return;
-
-	/* FIXME: merge g_pat from vmcb01 and vmcb12.  */
-	vmcb_set_gpat(svm->nested.vmcb02.ptr, svm->vmcb01.ptr->save.g_pat);
-}
-
 static void nested_vmcb02_prepare_save(struct vcpu_svm *svm)
 {
 	struct vmcb_ctrl_area_cached *control = &svm->nested.ctl;
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 6c41f2317777..00dba10991a5 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -2715,6 +2715,46 @@ static bool sev_es_prevent_msr_access(struct kvm_vcpu *vcpu,
 	       !msr_write_intercepted(vcpu, msr_info->index);
 }
 
+static bool svm_pat_accesses_gpat(struct kvm_vcpu *vcpu, bool from_host)
+{
+	struct vcpu_svm *svm = to_svm(vcpu);
+
+	/*
+	 * When nested NPT is enabled, L2 has a separate PAT from
+	 * L1.  Guest accesses to IA32_PAT while running L2 target
+	 * L2's gPAT; host-initiated accesses always target L1's
+	 * hPAT for backward and forward KVM_SET_MSRS compatibility
+	 * with older kernels.
+	 */
+	WARN_ON_ONCE(from_host && vcpu->wants_to_run);
+	return !from_host && is_guest_mode(vcpu) && nested_npt_enabled(svm);
+}
+
+static u64 svm_get_pat(struct kvm_vcpu *vcpu, bool from_host)
+{
+	if (svm_pat_accesses_gpat(vcpu, from_host))
+		return to_svm(vcpu)->vmcb->save.g_pat;
+	else
+		return vcpu->arch.pat;
+}
+
+static void svm_set_pat(struct kvm_vcpu *vcpu, bool from_host, u64 data)
+{
+	struct vcpu_svm *svm = to_svm(vcpu);
+
+	if (svm_pat_accesses_gpat(vcpu, from_host)) {
+		vmcb_set_gpat(svm->vmcb, data);
+	} else {
+		svm->vcpu.arch.pat = data;
+		if (npt_enabled) {
+			vmcb_set_gpat(svm->vmcb01.ptr, data);
+			if (is_guest_mode(&svm->vcpu) &&
+			    !nested_npt_enabled(svm))
+				vmcb_set_gpat(svm->vmcb, data);
+		}
+	}
+}
+
 static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
@@ -2837,6 +2877,9 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	case MSR_AMD64_DE_CFG:
 		msr_info->data = svm->msr_decfg;
 		break;
+	case MSR_IA32_CR_PAT:
+		msr_info->data = svm_get_pat(vcpu, msr_info->host_initiated);
+		break;
 	default:
 		return kvm_get_msr_common(vcpu, msr_info);
 	}
@@ -2920,13 +2963,10 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
 
 		break;
 	case MSR_IA32_CR_PAT:
-		ret = kvm_set_msr_common(vcpu, msr);
-		if (ret)
-			break;
+		if (!kvm_pat_valid(data))
+			return 1;
 
-		vmcb_set_gpat(svm->vmcb01.ptr, data);
-		if (is_guest_mode(vcpu))
-			nested_vmcb02_compute_g_pat(svm);
+		svm_set_pat(vcpu, msr->host_initiated, data);
 		break;
 	case MSR_IA32_SPEC_CTRL:
 		if (!msr->host_initiated &&
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index a49c48459e0b..58b0b935d049 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -840,7 +840,6 @@ void nested_copy_vmcb_control_to_cache(struct vcpu_svm *svm,
 void nested_copy_vmcb_save_to_cache(struct vcpu_svm *svm,
 				    struct vmcb_save_area *save);
 void nested_sync_control_from_vmcb02(struct vcpu_svm *svm);
-void nested_vmcb02_compute_g_pat(struct vcpu_svm *svm);
 void svm_switch_vmcb(struct vcpu_svm *svm, struct kvm_vmcb_info *target_vmcb);
 
 extern struct kvm_x86_nested_ops svm_nested_ops;
-- 
2.53.0.371.g1d285c8824-goog

[PATCH v5 06/10] KVM: x86: Remove common handling of MSR_IA32_CR_PAT

[Jim Mattson]

SVM now has completely independent handling of MSR_IA32_CR_PAT in
svm_get_msr() and svm_set_msr().

To avoid any confusion, move the logic for MSR_IA32_CR_PAT from
kvm_get_msr_common() and kvm_set_msr_common() into vmx_get_msr() and
vmx_set_msr().

Signed-off-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/kvm/vmx/vmx.c | 9 ++++++---
 arch/x86/kvm/x86.c     | 9 ---------
 2 files changed, 6 insertions(+), 12 deletions(-)

diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 967b58a8ab9d..f5127dbd9104 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -2146,6 +2146,9 @@ int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		    !(vcpu->arch.arch_capabilities & ARCH_CAP_TSX_CTRL_MSR))
 			return 1;
 		goto find_uret_msr;
+	case MSR_IA32_CR_PAT:
+		msr_info->data = vcpu->arch.pat;
+		break;
 	case MSR_IA32_UMWAIT_CONTROL:
 		if (!msr_info->host_initiated && !vmx_has_waitpkg(vmx))
 			return 1;
@@ -2468,10 +2471,10 @@ int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			return 1;
 		goto find_uret_msr;
 	case MSR_IA32_CR_PAT:
-		ret = kvm_set_msr_common(vcpu, msr_info);
-		if (ret)
-			break;
+		if (!kvm_pat_valid(data))
+			return 1;
 
+		vcpu->arch.pat = data;
 		if (is_guest_mode(vcpu) &&
 		    get_vmcs12(vcpu)->vm_exit_controls & VM_EXIT_SAVE_IA32_PAT)
 			get_vmcs12(vcpu)->guest_ia32_pat = data;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 416899b5dbe4..41936f83a17f 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -4025,12 +4025,6 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			return 1;
 		}
 		break;
-	case MSR_IA32_CR_PAT:
-		if (!kvm_pat_valid(data))
-			return 1;
-
-		vcpu->arch.pat = data;
-		break;
 	case MTRRphysBase_MSR(0) ... MSR_MTRRfix4K_F8000:
 	case MSR_MTRRdefType:
 		return kvm_mtrr_set_msr(vcpu, msr, data);
@@ -4436,9 +4430,6 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		msr_info->data = kvm_scale_tsc(rdtsc(), ratio) + offset;
 		break;
 	}
-	case MSR_IA32_CR_PAT:
-		msr_info->data = vcpu->arch.pat;
-		break;
 	case MSR_MTRRcap:
 	case MTRRphysBase_MSR(0) ... MSR_MTRRfix4K_F8000:
 	case MSR_MTRRdefType:
-- 
2.53.0.371.g1d285c8824-goog

[PATCH v5 07/10] KVM: x86: nSVM: Save gPAT to vmcb12.g_pat on VMEXIT

[Jim Mattson]

According to the APM volume 3 pseudo-code for "VMRUN," when nested paging
is enabled in the vmcb, the guest PAT register (gPAT) is saved to the vmcb
on emulated VMEXIT.

When nested NPT is enabled, save the vmcb02 g_pat field to the vmcb12 g_pat
field on emulated VMEXIT.

Fixes: 15038e147247 ("KVM: SVM: obey guest PAT")
Signed-off-by: Jim Mattson <jmattson@google.com>
Reviewed-by: Yosry Ahmed <yosry.ahmed@linux.dev>
---
 arch/x86/kvm/svm/nested.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index 69b577a4915c..26f758e294ab 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -1312,6 +1312,9 @@ void nested_svm_vmexit(struct vcpu_svm *svm)
 	vmcb12->save.dr6    = svm->vcpu.arch.dr6;
 	vmcb12->save.cpl    = vmcb02->save.cpl;
 
+	if (nested_npt_enabled(svm))
+		vmcb12->save.g_pat = vmcb02->save.g_pat;
+
 	if (guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) {
 		vmcb12->save.s_cet	= vmcb02->save.s_cet;
 		vmcb12->save.isst_addr	= vmcb02->save.isst_addr;
-- 
2.53.0.371.g1d285c8824-goog

[PATCH v5 08/10] KVM: x86: nSVM: Save/restore gPAT with KVM_{GET,SET}_NESTED_STATE

[Jim Mattson]

Add a 'flags' field to the SVM nested state header, and use bit 0 of the
flags to indicate that gPAT is stored in the nested state.

If in guest mode with NPT enabled, store the current vmcb->save.g_pat value
into the header of the nested state, and set the flag.

Note that struct kvm_svm_nested_state_hdr is included in a union padded to
120 bytes, so there is room to add the flags field and the gpat field
without changing any offsets.

Fixes: cc440cdad5b7 ("KVM: nSVM: implement KVM_GET_NESTED_STATE and KVM_SET_NESTED_STATE")
Signed-off-by: Jim Mattson <jmattson@google.com>
Reviewed-by: Yosry Ahmed <yosry.ahmed@linux.dev>
---
 arch/x86/include/uapi/asm/kvm.h |  5 +++++
 arch/x86/kvm/svm/nested.c       | 17 +++++++++++++++++
 2 files changed, 22 insertions(+)

diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
index 846a63215ce1..664d04d1db3f 100644
--- a/arch/x86/include/uapi/asm/kvm.h
+++ b/arch/x86/include/uapi/asm/kvm.h
@@ -495,6 +495,8 @@ struct kvm_sync_regs {
 
 #define KVM_STATE_VMX_PREEMPTION_TIMER_DEADLINE	0x00000001
 
+#define KVM_STATE_SVM_VALID_GPAT	0x00000001
+
 /* vendor-independent attributes for system fd (group 0) */
 #define KVM_X86_GRP_SYSTEM		0
 #  define KVM_X86_XCOMP_GUEST_SUPP	0
@@ -531,6 +533,9 @@ struct kvm_svm_nested_state_data {
 
 struct kvm_svm_nested_state_hdr {
 	__u64 vmcb_pa;
+	__u32 flags;
+	__u32 reserved;
+	__u64 gpat;
 };
 
 /* for KVM_CAP_NESTED_STATE */
diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index 26f758e294ab..5a35277f2364 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -1893,6 +1893,10 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,
 	/* First fill in the header and copy it out.  */
 	if (is_guest_mode(vcpu)) {
 		kvm_state.hdr.svm.vmcb_pa = svm->nested.vmcb12_gpa;
+		if (nested_npt_enabled(svm)) {
+			kvm_state.hdr.svm.flags |= KVM_STATE_SVM_VALID_GPAT;
+			kvm_state.hdr.svm.gpat = svm->vmcb->save.g_pat;
+		}
 		kvm_state.size += KVM_STATE_NESTED_SVM_VMCB_SIZE;
 		kvm_state.flags |= KVM_STATE_NESTED_GUEST_MODE;
 
@@ -2022,6 +2026,14 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
 	    !nested_vmcb_check_save(vcpu, &save_cached, false))
 		goto out_free;
 
+	/*
+	 * Validate gPAT, if provided. This is done separately from the
+	 * vmcb_save_area_cached validation above, because gPAT is L2
+	 * state, but the vmcb_save_area_cached is populated with L1 state.
+	 */
+	if ((kvm_state->hdr.svm.flags & KVM_STATE_SVM_VALID_GPAT) &&
+	    !kvm_pat_valid(kvm_state->hdr.svm.gpat))
+		goto out_free;
 
 	/*
 	 * All checks done, we can enter guest mode. Userspace provides
@@ -2062,6 +2074,11 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
 		goto out_free;
 
 	svm_switch_vmcb(svm, &svm->nested.vmcb02);
+
+	if (nested_npt_enabled(svm) &&
+	    (kvm_state->hdr.svm.flags & KVM_STATE_SVM_VALID_GPAT))
+		vmcb_set_gpat(svm->vmcb, kvm_state->hdr.svm.gpat);
+
 	nested_vmcb02_prepare_control(svm, svm->vmcb->save.rip, svm->vmcb->save.cs.base);
 
 	svm->nested.force_msr_bitmap_recalc = true;
-- 
2.53.0.371.g1d285c8824-goog

[PATCH v5 09/10] KVM: x86: nSVM: Handle restore of legacy nested state

[Jim Mattson]

When nested NPT is enabled and KVM_SET_NESTED_STATE is used to restore an
old checkpoint (without a valid gPAT), the current IA32_PAT value must be
used as L2's gPAT.

Unfortunately, checkpoint restore is non-atomic, and the order in which
state components are restored is not specified. Hence, the current IA32_PAT
value may be restored by KVM_SET_MSRS after KVM_SET_NESTED_STATE.  To
further complicate matters, there may be a KVM_GET_NESTED_STATE before the
next KVM_RUN.

Introduce a new boolean, svm->nested.legacy_gpat_semantics. When set, hPAT
updates are also applied to gPAT, preserving the old behavior (i.e. L2
shares L1's PAT). Set this boolean when restoring legacy state (i.e. nested
NPT is enabled, but no GPAT is provided) in KVM_SET_NESTED_STATE. Clear
this boolean in svm_vcpu_pre_run(), to ensure that hPAT and gPAT are
decoupled before the vCPU resumes execution. Also clear this boolean when
the vCPU is forced out of guest mode by svm_leave_nested().

Signed-off-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/kvm/svm/nested.c | 13 ++++++++++---
 arch/x86/kvm/svm/svm.c    |  8 ++++++--
 arch/x86/kvm/svm/svm.h    |  9 +++++++++
 3 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index 5a35277f2364..b68eddcbc217 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -1537,6 +1537,7 @@ void svm_leave_nested(struct kvm_vcpu *vcpu)
 	if (is_guest_mode(vcpu)) {
 		svm->nested.nested_run_pending = 0;
 		svm->nested.vmcb12_gpa = INVALID_GPA;
+		svm->nested.legacy_gpat_semantics = false;
 
 		leave_guest_mode(vcpu);
 
@@ -2075,9 +2076,15 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
 
 	svm_switch_vmcb(svm, &svm->nested.vmcb02);
 
-	if (nested_npt_enabled(svm) &&
-	    (kvm_state->hdr.svm.flags & KVM_STATE_SVM_VALID_GPAT))
-		vmcb_set_gpat(svm->vmcb, kvm_state->hdr.svm.gpat);
+	svm->nested.legacy_gpat_semantics =
+		nested_npt_enabled(svm) &&
+		!(kvm_state->hdr.svm.flags & KVM_STATE_SVM_VALID_GPAT);
+	if (nested_npt_enabled(svm)) {
+		u64 g_pat = svm->nested.legacy_gpat_semantics ?
+			    vcpu->arch.pat : kvm_state->hdr.svm.gpat;
+
+		vmcb_set_gpat(svm->nested.vmcb02.ptr, g_pat);
+	}
 
 	nested_vmcb02_prepare_control(svm, svm->vmcb->save.rip, svm->vmcb->save.cs.base);
 
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 00dba10991a5..ac45702f566e 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -2727,7 +2727,8 @@ static bool svm_pat_accesses_gpat(struct kvm_vcpu *vcpu, bool from_host)
 	 * with older kernels.
 	 */
 	WARN_ON_ONCE(from_host && vcpu->wants_to_run);
-	return !from_host && is_guest_mode(vcpu) && nested_npt_enabled(svm);
+	return !svm->nested.legacy_gpat_semantics && !from_host &&
+		is_guest_mode(vcpu) && nested_npt_enabled(svm);
 }
 
 static u64 svm_get_pat(struct kvm_vcpu *vcpu, bool from_host)
@@ -2749,7 +2750,8 @@ static void svm_set_pat(struct kvm_vcpu *vcpu, bool from_host, u64 data)
 		if (npt_enabled) {
 			vmcb_set_gpat(svm->vmcb01.ptr, data);
 			if (is_guest_mode(&svm->vcpu) &&
-			    !nested_npt_enabled(svm))
+			    (svm->nested.legacy_gpat_semantics ||
+			     !nested_npt_enabled(svm)))
 				vmcb_set_gpat(svm->vmcb, data);
 		}
 	}
@@ -4262,6 +4264,8 @@ static int svm_vcpu_pre_run(struct kvm_vcpu *vcpu)
 	if (to_kvm_sev_info(vcpu->kvm)->need_init)
 		return -EINVAL;
 
+	to_svm(vcpu)->nested.legacy_gpat_semantics = false;
+
 	return 1;
 }
 
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index 58b0b935d049..626efef878a5 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -238,6 +238,15 @@ struct svm_nested_state {
 	 * on its side.
 	 */
 	bool force_msr_bitmap_recalc;
+
+	/*
+	 * Indicates that a legacy nested state (without a valid gPAT) was
+	 * recently restored. Until the next KVM_RUN, updates to hPAT are
+	 * also applied to gPAT, preserving legacy behavior (i.e. L2 shares
+	 * L1's PAT). Because checkpoint restore is non-atomic, this
+	 * complication is necessary for backward compatibility.
+	 */
+	bool legacy_gpat_semantics;
 };
 
 struct vcpu_sev_es_state {
-- 
2.53.0.371.g1d285c8824-goog

[PATCH v5 10/10] KVM: selftests: nSVM: Add svm_nested_pat test

[Jim Mattson]

Verify that KVM correctly virtualizes the host PAT MSR and the guest PAT
register for nested SVM guests.

With nested NPT disabled:
 * L1 and L2 share the same PAT
 * The vmcb12.g_pat is ignored

With nested NPT enabled:
 * An invalid g_pat in vmcb12 causes VMEXIT_INVALID
 * RDMSR(IA32_PAT) from L2 returns the value of the guest PAT register
 * WRMSR(IA32_PAT) from L2 is reflected in vmcb12's g_pat on VMEXIT
 * RDMSR(IA32_PAT) from L1 returns the value of the host PAT MSR
 * Save/restore with the vCPU in guest mode preserves both hPAT and gPAT

Signed-off-by: Jim Mattson <jmattson@google.com>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/x86/svm_nested_pat_test.c   | 298 ++++++++++++++++++
 2 files changed, 299 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/svm_nested_pat_test.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 36b48e766e49..08a7bec34e4b 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -110,6 +110,7 @@ TEST_GEN_PROGS_x86 += x86/state_test
 TEST_GEN_PROGS_x86 += x86/vmx_preemption_timer_test
 TEST_GEN_PROGS_x86 += x86/svm_vmcall_test
 TEST_GEN_PROGS_x86 += x86/svm_int_ctl_test
+TEST_GEN_PROGS_x86 += x86/svm_nested_pat_test
 TEST_GEN_PROGS_x86 += x86/svm_nested_shutdown_test
 TEST_GEN_PROGS_x86 += x86/svm_nested_soft_inject_test
 TEST_GEN_PROGS_x86 += x86/svm_lbr_nested_state
diff --git a/tools/testing/selftests/kvm/x86/svm_nested_pat_test.c b/tools/testing/selftests/kvm/x86/svm_nested_pat_test.c
new file mode 100644
index 000000000000..08c1428969b0
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/svm_nested_pat_test.c
@@ -0,0 +1,298 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * KVM nested SVM PAT test
+ *
+ * Copyright (C) 2026, Google LLC.
+ *
+ * Test that KVM correctly virtualizes the PAT MSR and VMCB g_pat field
+ * for nested SVM guests:
+ *
+ * o With nested NPT disabled:
+ *     - L1 and L2 share the same PAT
+ *     - The vmcb12.g_pat is ignored
+ * o With nested NPT enabled:
+ *     - Invalid g_pat in vmcb12 should cause VMEXIT_INVALID
+ *     - L2 should see vmcb12.g_pat via RDMSR, not L1's PAT
+ *     - L2's writes to PAT should be saved to vmcb12 on exit
+ *     - L1's PAT should be restored after #VMEXIT from L2
+ *     - State save/restore should preserve both L1's and L2's PAT values
+ */
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+#include "svm_util.h"
+
+#define L2_GUEST_STACK_SIZE 256
+
+#define PAT_DEFAULT		0x0007040600070406ULL
+#define L1_PAT_VALUE		0x0007040600070404ULL  /* Change PA0 to WT */
+#define L2_VMCB12_PAT		0x0606060606060606ULL  /* All WB */
+#define L2_PAT_MODIFIED		0x0606060606060604ULL  /* Change PA0 to WT */
+#define INVALID_PAT_VALUE	0x0808080808080808ULL  /* 8 is reserved */
+
+/*
+ * Shared state between L1 and L2 for verification.
+ */
+struct pat_test_data {
+	uint64_t l2_pat_read;
+	uint64_t l2_pat_after_write;
+	uint64_t l1_pat_after_vmexit;
+	uint64_t vmcb12_gpat_after_exit;
+	bool l2_done;
+};
+
+static struct pat_test_data *pat_data;
+
+static void l2_guest_code(void)
+{
+	pat_data->l2_pat_read = rdmsr(MSR_IA32_CR_PAT);
+	wrmsr(MSR_IA32_CR_PAT, L2_PAT_MODIFIED);
+	pat_data->l2_pat_after_write = rdmsr(MSR_IA32_CR_PAT);
+	pat_data->l2_done = true;
+	vmmcall();
+}
+
+static void l2_guest_code_saverestoretest(void)
+{
+	pat_data->l2_pat_read = rdmsr(MSR_IA32_CR_PAT);
+
+	GUEST_SYNC(1);
+	GUEST_ASSERT_EQ(rdmsr(MSR_IA32_CR_PAT), pat_data->l2_pat_read);
+
+	wrmsr(MSR_IA32_CR_PAT, L2_PAT_MODIFIED);
+	pat_data->l2_pat_after_write = rdmsr(MSR_IA32_CR_PAT);
+
+	GUEST_SYNC(2);
+	GUEST_ASSERT_EQ(rdmsr(MSR_IA32_CR_PAT), L2_PAT_MODIFIED);
+
+	pat_data->l2_done = true;
+	vmmcall();
+}
+
+static void l2_guest_code_multi_vmentry(void)
+{
+	pat_data->l2_pat_read = rdmsr(MSR_IA32_CR_PAT);
+	wrmsr(MSR_IA32_CR_PAT, L2_PAT_MODIFIED);
+	pat_data->l2_pat_after_write = rdmsr(MSR_IA32_CR_PAT);
+	vmmcall();
+
+	pat_data->l2_pat_read = rdmsr(MSR_IA32_CR_PAT);
+	pat_data->l2_done = true;
+	vmmcall();
+}
+
+static struct vmcb *l1_common_setup(struct svm_test_data *svm,
+				    struct pat_test_data *data,
+				    void *l2_guest_code,
+				    void *l2_guest_stack)
+{
+	struct vmcb *vmcb = svm->vmcb;
+
+	pat_data = data;
+
+	wrmsr(MSR_IA32_CR_PAT, L1_PAT_VALUE);
+	GUEST_ASSERT_EQ(rdmsr(MSR_IA32_CR_PAT), L1_PAT_VALUE);
+
+	generic_svm_setup(svm, l2_guest_code, l2_guest_stack);
+
+	vmcb->save.g_pat = L2_VMCB12_PAT;
+	vmcb->control.intercept &= ~(1ULL << INTERCEPT_MSR_PROT);
+
+	return vmcb;
+}
+
+static void l1_assert_l2_state(struct pat_test_data *data, uint64_t expected_pat_read)
+{
+	GUEST_ASSERT(data->l2_done);
+	GUEST_ASSERT_EQ(data->l2_pat_read, expected_pat_read);
+	GUEST_ASSERT_EQ(data->l2_pat_after_write, L2_PAT_MODIFIED);
+}
+
+static void l1_svm_code_npt_disabled(struct svm_test_data *svm,
+				     struct pat_test_data *data)
+{
+	unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
+	struct vmcb *vmcb;
+
+	vmcb = l1_common_setup(svm, data, l2_guest_code,
+			       &l2_guest_stack[L2_GUEST_STACK_SIZE]);
+
+	run_guest(vmcb, svm->vmcb_gpa);
+
+	GUEST_ASSERT_EQ(vmcb->control.exit_code, SVM_EXIT_VMMCALL);
+	l1_assert_l2_state(data, L1_PAT_VALUE);
+
+	data->l1_pat_after_vmexit = rdmsr(MSR_IA32_CR_PAT);
+	GUEST_ASSERT_EQ(data->l1_pat_after_vmexit, L2_PAT_MODIFIED);
+
+	GUEST_DONE();
+}
+
+static void l1_svm_code_invalid_gpat(struct svm_test_data *svm,
+				     struct pat_test_data *data)
+{
+	unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
+	struct vmcb *vmcb;
+
+	vmcb = l1_common_setup(svm, data, l2_guest_code,
+			       &l2_guest_stack[L2_GUEST_STACK_SIZE]);
+
+	vmcb->save.g_pat = INVALID_PAT_VALUE;
+
+	run_guest(vmcb, svm->vmcb_gpa);
+
+	GUEST_ASSERT_EQ(vmcb->control.exit_code, SVM_EXIT_ERR);
+	GUEST_ASSERT(!data->l2_done);
+
+	GUEST_DONE();
+}
+
+static void l1_svm_code_npt_enabled(struct svm_test_data *svm,
+				    struct pat_test_data *data)
+{
+	unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
+	struct vmcb *vmcb;
+
+	vmcb = l1_common_setup(svm, data, l2_guest_code,
+			       &l2_guest_stack[L2_GUEST_STACK_SIZE]);
+
+	run_guest(vmcb, svm->vmcb_gpa);
+
+	GUEST_ASSERT_EQ(vmcb->control.exit_code, SVM_EXIT_VMMCALL);
+	l1_assert_l2_state(data, L2_VMCB12_PAT);
+
+	data->vmcb12_gpat_after_exit = vmcb->save.g_pat;
+	GUEST_ASSERT_EQ(data->vmcb12_gpat_after_exit, L2_PAT_MODIFIED);
+
+	data->l1_pat_after_vmexit = rdmsr(MSR_IA32_CR_PAT);
+	GUEST_ASSERT_EQ(data->l1_pat_after_vmexit, L1_PAT_VALUE);
+
+	GUEST_DONE();
+}
+
+static void l1_svm_code_saverestore(struct svm_test_data *svm,
+				    struct pat_test_data *data)
+{
+	unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
+	struct vmcb *vmcb;
+
+	vmcb = l1_common_setup(svm, data, l2_guest_code_saverestoretest,
+			       &l2_guest_stack[L2_GUEST_STACK_SIZE]);
+
+	run_guest(vmcb, svm->vmcb_gpa);
+
+	GUEST_ASSERT_EQ(vmcb->control.exit_code, SVM_EXIT_VMMCALL);
+	GUEST_ASSERT(data->l2_done);
+
+	GUEST_ASSERT_EQ(rdmsr(MSR_IA32_CR_PAT), L1_PAT_VALUE);
+	GUEST_ASSERT_EQ(vmcb->save.g_pat, L2_PAT_MODIFIED);
+
+	GUEST_DONE();
+}
+
+static void l1_svm_code_multi_vmentry(struct svm_test_data *svm,
+				      struct pat_test_data *data)
+{
+	unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
+	struct vmcb *vmcb;
+
+	vmcb = l1_common_setup(svm, data, l2_guest_code_multi_vmentry,
+			       &l2_guest_stack[L2_GUEST_STACK_SIZE]);
+
+	run_guest(vmcb, svm->vmcb_gpa);
+	GUEST_ASSERT_EQ(vmcb->control.exit_code, SVM_EXIT_VMMCALL);
+
+	GUEST_ASSERT_EQ(data->l2_pat_after_write, L2_PAT_MODIFIED);
+	GUEST_ASSERT_EQ(vmcb->save.g_pat, L2_PAT_MODIFIED);
+	GUEST_ASSERT_EQ(rdmsr(MSR_IA32_CR_PAT), L1_PAT_VALUE);
+
+	vmcb->save.rip += 3;  /* vmmcall */
+	run_guest(vmcb, svm->vmcb_gpa);
+
+	GUEST_ASSERT_EQ(vmcb->control.exit_code, SVM_EXIT_VMMCALL);
+	GUEST_ASSERT(data->l2_done);
+	GUEST_ASSERT_EQ(data->l2_pat_read, L2_PAT_MODIFIED);
+	GUEST_ASSERT_EQ(rdmsr(MSR_IA32_CR_PAT), L1_PAT_VALUE);
+
+	GUEST_DONE();
+}
+
+static void run_test(void *l1_code, const char *test_name, bool npt_enabled,
+		     bool do_save_restore)
+{
+	struct pat_test_data *data_hva;
+	vm_vaddr_t svm_gva, data_gva;
+	struct kvm_x86_state *state;
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	struct ucall uc;
+
+	pr_info("Testing: %s\n", test_name);
+
+	vm = vm_create_with_one_vcpu(&vcpu, l1_code);
+	if (npt_enabled)
+		vm_enable_npt(vm);
+
+	vcpu_alloc_svm(vm, &svm_gva);
+
+	data_gva = vm_vaddr_alloc_page(vm);
+	data_hva = addr_gva2hva(vm, data_gva);
+	memset(data_hva, 0, sizeof(*data_hva));
+
+	if (npt_enabled)
+		tdp_identity_map_default_memslots(vm);
+
+	vcpu_args_set(vcpu, 2, svm_gva, data_gva);
+
+	for (;;) {
+		vcpu_run(vcpu);
+		TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_IO);
+
+		switch (get_ucall(vcpu, &uc)) {
+		case UCALL_ABORT:
+			REPORT_GUEST_ASSERT(uc);
+			/* NOT REACHED */
+		case UCALL_SYNC:
+			if (do_save_restore) {
+				pr_info("  Save/restore at sync point %ld\n",
+					uc.args[1]);
+				state = vcpu_save_state(vcpu);
+				kvm_vm_release(vm);
+				vcpu = vm_recreate_with_one_vcpu(vm);
+				vcpu_load_state(vcpu, state);
+				kvm_x86_state_cleanup(state);
+			}
+			break;
+		case UCALL_DONE:
+			pr_info("  PASSED\n");
+			kvm_vm_free(vm);
+			return;
+		default:
+			TEST_FAIL("Unknown ucall %lu", uc.cmd);
+		}
+	}
+}
+
+int main(int argc, char *argv[])
+{
+	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_SVM));
+	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_NPT));
+	TEST_REQUIRE(kvm_has_cap(KVM_CAP_NESTED_STATE));
+
+	run_test(l1_svm_code_npt_disabled, "nested NPT disabled", false, false);
+
+	run_test(l1_svm_code_invalid_gpat, "invalid g_pat", true, false);
+
+	run_test(l1_svm_code_npt_enabled, "nested NPT enabled", true, false);
+
+	run_test(l1_svm_code_saverestore, "save/restore", true, true);
+
+	run_test(l1_svm_code_multi_vmentry, "multiple entries", true, false);
+
+	return 0;
+}
-- 
2.53.0.371.g1d285c8824-goog

Re: [PATCH v5 05/10] KVM: x86: nSVM: Redirect IA32_PAT accesses to either hPAT or gPAT

[Sean Christopherson]

On Mon, Feb 23, 2026, Jim Mattson wrote:
> +static void svm_set_pat(struct kvm_vcpu *vcpu, bool from_host, u64 data)
> +{
> +	struct vcpu_svm *svm = to_svm(vcpu);
> +
> +	if (svm_pat_accesses_gpat(vcpu, from_host)) {
> +		vmcb_set_gpat(svm->vmcb, data);
> +	} else {
> +		svm->vcpu.arch.pat = data;
> +		if (npt_enabled) {
> +			vmcb_set_gpat(svm->vmcb01.ptr, data);
> +			if (is_guest_mode(&svm->vcpu) &&
> +			    !nested_npt_enabled(svm))
> +				vmcb_set_gpat(svm->vmcb, data);
> +		}
> +	}

Overall, this LGTM.  For this particular code, any objection to using early
returns to reduce indentation?  The else branch above is a bit gnarly, especially
when legacy_gpat_semantics comes along.

I.e. end up with this

  static void svm_set_pat(struct kvm_vcpu *vcpu, bool from_host, u64 data)
  {
	struct vcpu_svm *svm = to_svm(vcpu);

	if (svm_pat_accesses_gpat(vcpu, from_host)) {
		vmcb_set_gpat(svm->vmcb, data);
		return;
	}

	svm->vcpu.arch.pat = data;

	if (!npt_enabled)
		return;

	vmcb_set_gpat(svm->vmcb01.ptr, data);
	if (is_guest_mode(&svm->vcpu) &&
	    (svm->nested.legacy_gpat_semantics || !nested_npt_enabled(svm)))
		vmcb_set_gpat(svm->vmcb, data);
  }

I can fixup when applying (unless you and/or Yosry object).

Re: [PATCH v5 05/10] KVM: x86: nSVM: Redirect IA32_PAT accesses to either hPAT or gPAT

[Jim Mattson]

On Tue, Feb 24, 2026 at 9:43 AM Sean Christopherson <seanjc@google.com> wrote:
> Overall, this LGTM.  For this particular code, any objection to using early
> returns to reduce indentation?  The else branch above is a bit gnarly, especially
> when legacy_gpat_semantics comes along.
> ...
> I can fixup when applying (unless you and/or Yosry object).

No objection.

Re: [PATCH v5 05/10] KVM: x86: nSVM: Redirect IA32_PAT accesses to either hPAT or gPAT

[Yosry Ahmed]

On Tue, Feb 24, 2026 at 9:43 AM Sean Christopherson <seanjc@google.com> wrote:
>
> On Mon, Feb 23, 2026, Jim Mattson wrote:
> > +static void svm_set_pat(struct kvm_vcpu *vcpu, bool from_host, u64 data)
> > +{
> > +     struct vcpu_svm *svm = to_svm(vcpu);
> > +
> > +     if (svm_pat_accesses_gpat(vcpu, from_host)) {
> > +             vmcb_set_gpat(svm->vmcb, data);
> > +     } else {
> > +             svm->vcpu.arch.pat = data;
> > +             if (npt_enabled) {
> > +                     vmcb_set_gpat(svm->vmcb01.ptr, data);
> > +                     if (is_guest_mode(&svm->vcpu) &&
> > +                         !nested_npt_enabled(svm))
> > +                             vmcb_set_gpat(svm->vmcb, data);
> > +             }
> > +     }
>
> Overall, this LGTM.  For this particular code, any objection to using early
> returns to reduce indentation?  The else branch above is a bit gnarly, especially
> when legacy_gpat_semantics comes along.
>
> I.e. end up with this
>
>   static void svm_set_pat(struct kvm_vcpu *vcpu, bool from_host, u64 data)
>   {
>         struct vcpu_svm *svm = to_svm(vcpu);
>
>         if (svm_pat_accesses_gpat(vcpu, from_host)) {
>                 vmcb_set_gpat(svm->vmcb, data);
>                 return;
>         }
>
>         svm->vcpu.arch.pat = data;
>
>         if (!npt_enabled)
>                 return;
>
>         vmcb_set_gpat(svm->vmcb01.ptr, data);
>         if (is_guest_mode(&svm->vcpu) &&
>             (svm->nested.legacy_gpat_semantics || !nested_npt_enabled(svm)))
>                 vmcb_set_gpat(svm->vmcb, data);
>   }
>
> I can fixup when applying (unless you and/or Yosry object).

LGTM.

Re: [PATCH v5 09/10] KVM: x86: nSVM: Handle restore of legacy nested state

[Sean Christopherson]

On Mon, Feb 23, 2026, Jim Mattson wrote:
> @@ -2075,9 +2076,15 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
>  
>  	svm_switch_vmcb(svm, &svm->nested.vmcb02);
>  
> -	if (nested_npt_enabled(svm) &&
> -	    (kvm_state->hdr.svm.flags & KVM_STATE_SVM_VALID_GPAT))
> -		vmcb_set_gpat(svm->vmcb, kvm_state->hdr.svm.gpat);
> +	svm->nested.legacy_gpat_semantics =
> +		nested_npt_enabled(svm) &&
> +		!(kvm_state->hdr.svm.flags & KVM_STATE_SVM_VALID_GPAT);
> +	if (nested_npt_enabled(svm)) {
> +		u64 g_pat = svm->nested.legacy_gpat_semantics ?
> +			    vcpu->arch.pat : kvm_state->hdr.svm.gpat;

This is all a bit gnarly, e.g. the indentation and wrapping, as well as the logic
(not that it's wrong, just a bit hard to follow the chain of events).

Rather than set legacy_gpat_semantics directly, what if we clear it by default,
and then set it %true in the exact path where KVM uses legacy semantics.

	svm->nested.legacy_gpat_semantics = false;
	if (nested_npt_enabled(svm)) {
		if (kvm_state->hdr.svm.flags & KVM_STATE_SVM_VALID_GPAT) {
			vmcb_set_gpat(svm->vmcb, kvm_state->hdr.svm.gpat);
		} else {
			svm->nested.legacy_gpat_semantics = true;
			vmcb_set_gpat(svm->vmcb, vcpu->arch.pat);
		}
	}

As a bonus, if the previous patch is deliberately "bad" and does:

	if (nested_npt_enabled(svm)) {
		if (kvm_state->hdr.svm.flags & KVM_STATE_SVM_VALID_GPAT)
			vmcb_set_gpat(svm->vmcb, kvm_state->hdr.svm.gpat);
	}

then the diff for this snippet shrinks to:

@@ -2025,9 +2026,14 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
 
        svm_switch_vmcb(svm, &svm->nested.vmcb02);
 
+       svm->nested.legacy_gpat_semantics = false;
        if (nested_npt_enabled(svm)) {
-               if (kvm_state->hdr.svm.flags & KVM_STATE_SVM_VALID_GPAT)
+               if (kvm_state->hdr.svm.flags & KVM_STATE_SVM_VALID_GPAT) {
                        vmcb_set_gpat(svm->vmcb, kvm_state->hdr.svm.gpat);
+               } else {
+                       svm->nested.legacy_gpat_semantics = true;
+                       vmcb_set_gpat(svm->vmcb, vcpu->arch.pat);
+               }
        }
 
        nested_vmcb02_prepare_control(svm);

> +
> +		vmcb_set_gpat(svm->nested.vmcb02.ptr, g_pat);

I don't like the switch from svm->vmcb to svm->nested.vmcb02.ptr.  For better or
worse, the existing code uses svm->vmcb, so I think it makes sense to use that.
If we want to explicitly use vmcb02, then we should capture svm->nested.vmcb02.ptr
locally as vmcb02 (in a future cleanup patch).


> +	}
>  
>  	nested_vmcb02_prepare_control(svm, svm->vmcb->save.rip, svm->vmcb->save.cs.base);
>  
> diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
> index 00dba10991a5..ac45702f566e 100644
> --- a/arch/x86/kvm/svm/svm.c
> +++ b/arch/x86/kvm/svm/svm.c
> @@ -2727,7 +2727,8 @@ static bool svm_pat_accesses_gpat(struct kvm_vcpu *vcpu, bool from_host)
>  	 * with older kernels.
>  	 */
>  	WARN_ON_ONCE(from_host && vcpu->wants_to_run);
> -	return !from_host && is_guest_mode(vcpu) && nested_npt_enabled(svm);
> +	return !svm->nested.legacy_gpat_semantics && !from_host &&
> +		is_guest_mode(vcpu) && nested_npt_enabled(svm);

Align indentation (it's a shame return wasn't spelled retturn, it would save many
spaces).

Re: [PATCH v5 08/10] KVM: x86: nSVM: Save/restore gPAT with KVM_{GET,SET}_NESTED_STATE

[Sean Christopherson]

On Mon, Feb 23, 2026, Jim Mattson wrote:
> Add a 'flags' field to the SVM nested state header, and use bit 0 of the
> flags to indicate that gPAT is stored in the nested state.
> 
> If in guest mode with NPT enabled, store the current vmcb->save.g_pat value
> into the header of the nested state, and set the flag.
> 
> Note that struct kvm_svm_nested_state_hdr is included in a union padded to
> 120 bytes, so there is room to add the flags field and the gpat field
> without changing any offsets.
> 
> Fixes: cc440cdad5b7 ("KVM: nSVM: implement KVM_GET_NESTED_STATE and KVM_SET_NESTED_STATE")
> Signed-off-by: Jim Mattson <jmattson@google.com>
> Reviewed-by: Yosry Ahmed <yosry.ahmed@linux.dev>
> ---
>  arch/x86/include/uapi/asm/kvm.h |  5 +++++
>  arch/x86/kvm/svm/nested.c       | 17 +++++++++++++++++
>  2 files changed, 22 insertions(+)
> 
> diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
> index 846a63215ce1..664d04d1db3f 100644
> --- a/arch/x86/include/uapi/asm/kvm.h
> +++ b/arch/x86/include/uapi/asm/kvm.h
> @@ -495,6 +495,8 @@ struct kvm_sync_regs {
>  
>  #define KVM_STATE_VMX_PREEMPTION_TIMER_DEADLINE	0x00000001
>  
> +#define KVM_STATE_SVM_VALID_GPAT	0x00000001
> +
>  /* vendor-independent attributes for system fd (group 0) */
>  #define KVM_X86_GRP_SYSTEM		0
>  #  define KVM_X86_XCOMP_GUEST_SUPP	0
> @@ -531,6 +533,9 @@ struct kvm_svm_nested_state_data {
>  
>  struct kvm_svm_nested_state_hdr {
>  	__u64 vmcb_pa;
> +	__u32 flags;
> +	__u32 reserved;
> +	__u64 gpat;
>  };
>  
>  /* for KVM_CAP_NESTED_STATE */
> diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
> index 26f758e294ab..5a35277f2364 100644
> --- a/arch/x86/kvm/svm/nested.c
> +++ b/arch/x86/kvm/svm/nested.c
> @@ -1893,6 +1893,10 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,
>  	/* First fill in the header and copy it out.  */
>  	if (is_guest_mode(vcpu)) {
>  		kvm_state.hdr.svm.vmcb_pa = svm->nested.vmcb12_gpa;
> +		if (nested_npt_enabled(svm)) {
> +			kvm_state.hdr.svm.flags |= KVM_STATE_SVM_VALID_GPAT;

Bugger.  This isn't going to work.  KVM doesn't reserve any bytes in the header
for SVM, so there's no guarantee/requirement that old userspace won't provide
garbage.  The flag needs to go in kvm_nested_state.flags.  We only reserved space
for 16 flags between VMX and SVM, but that's a future problem as we can always
add e.g. KVM_STATE_NESTED_EXT_FLAGS when we've exausted the current space.

Ugh.  And vmx_set_nested_state() doesn't check for unsupported flags, at all.
But that too is largely a future problem though, i.e. is a non-issue until nVMX
wants to add a new flag.  *sigh*

Anyways, for gPAT, unless I'm missing something, I'm going to squash this:

diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
index 664d04d1db3f..0c1f97c9a2d8 100644
--- a/arch/x86/include/uapi/asm/kvm.h
+++ b/arch/x86/include/uapi/asm/kvm.h
@@ -485,6 +485,7 @@ struct kvm_sync_regs {
 #define KVM_STATE_NESTED_EVMCS         0x00000004
 #define KVM_STATE_NESTED_MTF_PENDING   0x00000008
 #define KVM_STATE_NESTED_GIF_SET       0x00000100
+#define KVM_STATE_NESTED_GPAT_VALID    0x00000200
 
 #define KVM_STATE_NESTED_SMM_GUEST_MODE        0x00000001
 #define KVM_STATE_NESTED_SMM_VMXON     0x00000002
@@ -495,8 +496,6 @@ struct kvm_sync_regs {
 
 #define KVM_STATE_VMX_PREEMPTION_TIMER_DEADLINE        0x00000001
 
-#define KVM_STATE_SVM_VALID_GPAT       0x00000001
-
 /* vendor-independent attributes for system fd (group 0) */
 #define KVM_X86_GRP_SYSTEM             0
 #  define KVM_X86_XCOMP_GUEST_SUPP     0
@@ -533,8 +532,6 @@ struct kvm_svm_nested_state_data {
 
 struct kvm_svm_nested_state_hdr {
        __u64 vmcb_pa;
-       __u32 flags;
-       __u32 reserved;
        __u64 gpat;
 };
 
diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index 991ee4c03363..099bf8ac10ee 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -1848,7 +1848,7 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,
        if (is_guest_mode(vcpu)) {
                kvm_state.hdr.svm.vmcb_pa = svm->nested.vmcb12_gpa;
                if (nested_npt_enabled(svm)) {
-                       kvm_state.hdr.svm.flags |= KVM_STATE_SVM_VALID_GPAT;
+                       kvm_state->flags |= KVM_STATE_NESTED_GPAT_VALID;
                        kvm_state.hdr.svm.gpat = svm->vmcb->save.g_pat;
                }
                kvm_state.size += KVM_STATE_NESTED_SVM_VMCB_SIZE;
@@ -1914,7 +1914,8 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
 
        if (kvm_state->flags & ~(KVM_STATE_NESTED_GUEST_MODE |
                                 KVM_STATE_NESTED_RUN_PENDING |
-                                KVM_STATE_NESTED_GIF_SET))
+                                KVM_STATE_NESTED_GIF_SET |
+                                KVM_STATE_NESTED_GPAT_VALID))
                return -EINVAL;
 
        /*
@@ -1984,7 +1985,7 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
         * vmcb_save_area_cached validation above, because gPAT is L2
         * state, but the vmcb_save_area_cached is populated with L1 state.
         */
-       if ((kvm_state->hdr.svm.flags & KVM_STATE_SVM_VALID_GPAT) &&
+       if ((kvm_state->flags & KVM_STATE_NESTED_GPAT_VALID) &&
            !kvm_pat_valid(kvm_state->hdr.svm.gpat))
                goto out_free;
 
@@ -2013,7 +2014,7 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
        svm_switch_vmcb(svm, &svm->nested.vmcb02);
 
        if (nested_npt_enabled(svm)) {
-               if (kvm_state->hdr.svm.flags & KVM_STATE_SVM_VALID_GPAT)
+               if (kvm_state->flags & KVM_STATE_NESTED_GPAT_VALID)
                        vmcb_set_gpat(svm->vmcb, kvm_state->hdr.svm.gpat);
        }
--

One could argue we should even be more paranoid and do this as well:

	if (!(kvm_state->flags & KVM_STATE_NESTED_GUEST_MODE)) {
		if (kvm_state->flags & KVM_STATE_NESTED_GPAT_VALID)
			return -EINVAL:

		svm_leave_nested(vcpu);
		svm_set_gif(svm, !!(kvm_state->flags & KVM_STATE_NESTED_GIF_SET));
		return 0;
	}

but KVM doesn't enforce GUEST_MODE for KVM_STATE_NESTED_RUN_PENDING, and it's
easy to just ignore gPAT, so I'm inclined to not bother.

Re: [PATCH v5 08/10] KVM: x86: nSVM: Save/restore gPAT with KVM_{GET,SET}_NESTED_STATE

[Jim Mattson]

On Wed, Mar 4, 2026 at 9:11 AM Sean Christopherson <seanjc@google.com> wrote:
> diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
> index 991ee4c03363..099bf8ac10ee 100644
> --- a/arch/x86/kvm/svm/nested.c
> +++ b/arch/x86/kvm/svm/nested.c
> @@ -1848,7 +1848,7 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,
>         if (is_guest_mode(vcpu)) {
>                 kvm_state.hdr.svm.vmcb_pa = svm->nested.vmcb12_gpa;
>                 if (nested_npt_enabled(svm)) {
> -                       kvm_state.hdr.svm.flags |= KVM_STATE_SVM_VALID_GPAT;
> +                       kvm_state->flags |= KVM_STATE_NESTED_GPAT_VALID;
>                         kvm_state.hdr.svm.gpat = svm->vmcb->save.g_pat;
>                 }
>                 kvm_state.size += KVM_STATE_NESTED_SVM_VMCB_SIZE;
> @@ -1914,7 +1914,8 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
>
>         if (kvm_state->flags & ~(KVM_STATE_NESTED_GUEST_MODE |
>                                  KVM_STATE_NESTED_RUN_PENDING |
> -                                KVM_STATE_NESTED_GIF_SET))
> +                                KVM_STATE_NESTED_GIF_SET |
> +                                KVM_STATE_NESTED_GPAT_VALID))
>                 return -EINVAL;

Unless I'm missing something, this breaks forward compatibility
completely. An older kernel will refuse to accept a nested state blob
with GPAT_VALID set.

Re: [PATCH v5 08/10] KVM: x86: nSVM: Save/restore gPAT with KVM_{GET,SET}_NESTED_STATE

[Sean Christopherson]

On Wed, Mar 04, 2026, Jim Mattson wrote:
> On Wed, Mar 4, 2026 at 9:11 AM Sean Christopherson <seanjc@google.com> wrote:
> > diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
> > index 991ee4c03363..099bf8ac10ee 100644
> > --- a/arch/x86/kvm/svm/nested.c
> > +++ b/arch/x86/kvm/svm/nested.c
> > @@ -1848,7 +1848,7 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,
> >         if (is_guest_mode(vcpu)) {
> >                 kvm_state.hdr.svm.vmcb_pa = svm->nested.vmcb12_gpa;
> >                 if (nested_npt_enabled(svm)) {
> > -                       kvm_state.hdr.svm.flags |= KVM_STATE_SVM_VALID_GPAT;
> > +                       kvm_state->flags |= KVM_STATE_NESTED_GPAT_VALID;
> >                         kvm_state.hdr.svm.gpat = svm->vmcb->save.g_pat;
> >                 }
> >                 kvm_state.size += KVM_STATE_NESTED_SVM_VMCB_SIZE;
> > @@ -1914,7 +1914,8 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
> >
> >         if (kvm_state->flags & ~(KVM_STATE_NESTED_GUEST_MODE |
> >                                  KVM_STATE_NESTED_RUN_PENDING |
> > -                                KVM_STATE_NESTED_GIF_SET))
> > +                                KVM_STATE_NESTED_GIF_SET |
> > +                                KVM_STATE_NESTED_GPAT_VALID))
> >                 return -EINVAL;
> 
> Unless I'm missing something, this breaks forward compatibility
> completely. An older kernel will refuse to accept a nested state blob
> with GPAT_VALID set.

Argh, so we've painted ourselves into an impossible situation by restricting the
set of valid flags.  I.e. VMX's omission of checks on unknown flags is a feature,
not a bug.

Chatted with Jim offlist, and he pointed out that KVM's standard way to deal with
this is to make setting the flag opt-in, e.g. KVM_CAP_X86_TRIPLE_FAULT_EVENT and
KVM_CAP_EXCEPTION_PAYLOAD.

As much as I want to retroactively change KVM's documentation to state doing
KVM_SET_NESTED_STATE with data that didn't come from KVM_GET_NESTED_STATE is
unsupported, that feels too restrictive and could really bite us in the future.
And it doesn't help if there's already userspace that's putting garbage into the
header.

So yeah, I don't see a better option than adding yet another capability.

Can you send a new version based on `kvm-x86 next`?  (give me ~hour to drop these
and push).  This has snowballed beyond what I'm comfortable doing as fixup. :-(

Re: [PATCH v5 08/10] KVM: x86: nSVM: Save/restore gPAT with KVM_{GET,SET}_NESTED_STATE

[Yosry Ahmed]

On Wed, Mar 4, 2026 at 11:11 AM Sean Christopherson <seanjc@google.com> wrote:
>
> On Wed, Mar 04, 2026, Jim Mattson wrote:
> > On Wed, Mar 4, 2026 at 9:11 AM Sean Christopherson <seanjc@google.com> wrote:
> > > diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
> > > index 991ee4c03363..099bf8ac10ee 100644
> > > --- a/arch/x86/kvm/svm/nested.c
> > > +++ b/arch/x86/kvm/svm/nested.c
> > > @@ -1848,7 +1848,7 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,
> > >         if (is_guest_mode(vcpu)) {
> > >                 kvm_state.hdr.svm.vmcb_pa = svm->nested.vmcb12_gpa;
> > >                 if (nested_npt_enabled(svm)) {
> > > -                       kvm_state.hdr.svm.flags |= KVM_STATE_SVM_VALID_GPAT;
> > > +                       kvm_state->flags |= KVM_STATE_NESTED_GPAT_VALID;
> > >                         kvm_state.hdr.svm.gpat = svm->vmcb->save.g_pat;
> > >                 }
> > >                 kvm_state.size += KVM_STATE_NESTED_SVM_VMCB_SIZE;
> > > @@ -1914,7 +1914,8 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
> > >
> > >         if (kvm_state->flags & ~(KVM_STATE_NESTED_GUEST_MODE |
> > >                                  KVM_STATE_NESTED_RUN_PENDING |
> > > -                                KVM_STATE_NESTED_GIF_SET))
> > > +                                KVM_STATE_NESTED_GIF_SET |
> > > +                                KVM_STATE_NESTED_GPAT_VALID))
> > >                 return -EINVAL;
> >
> > Unless I'm missing something, this breaks forward compatibility
> > completely. An older kernel will refuse to accept a nested state blob
> > with GPAT_VALID set.
>
> Argh, so we've painted ourselves into an impossible situation by restricting the
> set of valid flags.  I.e. VMX's omission of checks on unknown flags is a feature,
> not a bug.
>
> Chatted with Jim offlist, and he pointed out that KVM's standard way to deal with
> this is to make setting the flag opt-in, e.g. KVM_CAP_X86_TRIPLE_FAULT_EVENT and
> KVM_CAP_EXCEPTION_PAYLOAD.
>
> As much as I want to retroactively change KVM's documentation to state doing
> KVM_SET_NESTED_STATE with data that didn't come from KVM_GET_NESTED_STATE is
> unsupported, that feels too restrictive and could really bite us in the future.
> And it doesn't help if there's already userspace that's putting garbage into the
> header.
>
> So yeah, I don't see a better option than adding yet another capability.
>
> Can you send a new version based on `kvm-x86 next`?  (give me ~hour to drop these
> and push).  This has snowballed beyond what I'm comfortable doing as fixup. :-(

Will the current patches still be reachable through
kvm-x86-next-2026.03.03? I imagine Jim will want to pull those and
change them directly as they have all your fixups (rather than
reconstructing them).

Re: [PATCH v5 08/10] KVM: x86: nSVM: Save/restore gPAT with KVM_{GET,SET}_NESTED_STATE

[Sean Christopherson]

On Wed, Mar 04, 2026, Yosry Ahmed wrote:
> On Wed, Mar 4, 2026 at 11:11 AM Sean Christopherson <seanjc@google.com> wrote:
> >
> > On Wed, Mar 04, 2026, Jim Mattson wrote:
> > > On Wed, Mar 4, 2026 at 9:11 AM Sean Christopherson <seanjc@google.com> wrote:
> > > > diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
> > > > index 991ee4c03363..099bf8ac10ee 100644
> > > > --- a/arch/x86/kvm/svm/nested.c
> > > > +++ b/arch/x86/kvm/svm/nested.c
> > > > @@ -1848,7 +1848,7 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,
> > > >         if (is_guest_mode(vcpu)) {
> > > >                 kvm_state.hdr.svm.vmcb_pa = svm->nested.vmcb12_gpa;
> > > >                 if (nested_npt_enabled(svm)) {
> > > > -                       kvm_state.hdr.svm.flags |= KVM_STATE_SVM_VALID_GPAT;
> > > > +                       kvm_state->flags |= KVM_STATE_NESTED_GPAT_VALID;
> > > >                         kvm_state.hdr.svm.gpat = svm->vmcb->save.g_pat;
> > > >                 }
> > > >                 kvm_state.size += KVM_STATE_NESTED_SVM_VMCB_SIZE;
> > > > @@ -1914,7 +1914,8 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
> > > >
> > > >         if (kvm_state->flags & ~(KVM_STATE_NESTED_GUEST_MODE |
> > > >                                  KVM_STATE_NESTED_RUN_PENDING |
> > > > -                                KVM_STATE_NESTED_GIF_SET))
> > > > +                                KVM_STATE_NESTED_GIF_SET |
> > > > +                                KVM_STATE_NESTED_GPAT_VALID))
> > > >                 return -EINVAL;
> > >
> > > Unless I'm missing something, this breaks forward compatibility
> > > completely. An older kernel will refuse to accept a nested state blob
> > > with GPAT_VALID set.
> >
> > Argh, so we've painted ourselves into an impossible situation by restricting the
> > set of valid flags.  I.e. VMX's omission of checks on unknown flags is a feature,
> > not a bug.
> >
> > Chatted with Jim offlist, and he pointed out that KVM's standard way to deal with
> > this is to make setting the flag opt-in, e.g. KVM_CAP_X86_TRIPLE_FAULT_EVENT and
> > KVM_CAP_EXCEPTION_PAYLOAD.
> >
> > As much as I want to retroactively change KVM's documentation to state doing
> > KVM_SET_NESTED_STATE with data that didn't come from KVM_GET_NESTED_STATE is
> > unsupported, that feels too restrictive and could really bite us in the future.
> > And it doesn't help if there's already userspace that's putting garbage into the
> > header.
> >
> > So yeah, I don't see a better option than adding yet another capability.
> >
> > Can you send a new version based on `kvm-x86 next`?  (give me ~hour to drop these
> > and push).  This has snowballed beyond what I'm comfortable doing as fixup. :-(
> 
> Will the current patches still be reachable through
> kvm-x86-next-2026.03.03? I imagine Jim will want to pull those and
> change them directly as they have all your fixups (rather than
> reconstructing them).

Oh, yeah!  I was actually going to self-respond with that suggestion, then squirrel!

Re: [PATCH v5 00/10] KVM: x86: nSVM: Improve PAT virtualization

[Sean Christopherson]

On Mon, 23 Feb 2026 16:54:38 -0800, Jim Mattson wrote:
> Currently, KVM's implementation of nested SVM treats the PAT MSR the same
> way whether or not nested NPT is enabled: L1 and L2 share a single
> PAT. However, the APM specifies that when nested NPT is enabled, the host
> (L1) and the guest (L2) should have independent PATs: hPAT for L1 and gPAT
> for L2. This patch series implements the architectural specification in
> KVM.
> 
> [...]

Applied patch 1 to kvm-x86 nested.  I'll wait for a new version for the gPAT
stuff.

[01/10] KVM: x86: SVM: Remove vmcb_is_dirty()
        https://github.com/kvm-x86/linux/commit/66b207f175f1

--
https://github.com/kvm-x86/linux/tree/next

Re: [PATCH v5 08/10] KVM: x86: nSVM: Save/restore gPAT with KVM_{GET,SET}_NESTED_STATE

[Jim Mattson]

On Wed, Mar 4, 2026 at 11:55 AM Sean Christopherson <seanjc@google.com> wrote:
>
> On Wed, Mar 04, 2026, Yosry Ahmed wrote:
> > On Wed, Mar 4, 2026 at 11:11 AM Sean Christopherson <seanjc@google.com> wrote:
> > >
> > > On Wed, Mar 04, 2026, Jim Mattson wrote:
> > > > On Wed, Mar 4, 2026 at 9:11 AM Sean Christopherson <seanjc@google.com> wrote:
> > > > > diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
> > > > > index 991ee4c03363..099bf8ac10ee 100644
> > > > > --- a/arch/x86/kvm/svm/nested.c
> > > > > +++ b/arch/x86/kvm/svm/nested.c
> > > > > @@ -1848,7 +1848,7 @@ static int svm_get_nested_state(struct kvm_vcpu *vcpu,
> > > > >         if (is_guest_mode(vcpu)) {
> > > > >                 kvm_state.hdr.svm.vmcb_pa = svm->nested.vmcb12_gpa;
> > > > >                 if (nested_npt_enabled(svm)) {
> > > > > -                       kvm_state.hdr.svm.flags |= KVM_STATE_SVM_VALID_GPAT;
> > > > > +                       kvm_state->flags |= KVM_STATE_NESTED_GPAT_VALID;
> > > > >                         kvm_state.hdr.svm.gpat = svm->vmcb->save.g_pat;
> > > > >                 }
> > > > >                 kvm_state.size += KVM_STATE_NESTED_SVM_VMCB_SIZE;
> > > > > @@ -1914,7 +1914,8 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
> > > > >
> > > > >         if (kvm_state->flags & ~(KVM_STATE_NESTED_GUEST_MODE |
> > > > >                                  KVM_STATE_NESTED_RUN_PENDING |
> > > > > -                                KVM_STATE_NESTED_GIF_SET))
> > > > > +                                KVM_STATE_NESTED_GIF_SET |
> > > > > +                                KVM_STATE_NESTED_GPAT_VALID))
> > > > >                 return -EINVAL;
> > > >
> > > > Unless I'm missing something, this breaks forward compatibility
> > > > completely. An older kernel will refuse to accept a nested state blob
> > > > with GPAT_VALID set.
> > >
> > > Argh, so we've painted ourselves into an impossible situation by restricting the
> > > set of valid flags.  I.e. VMX's omission of checks on unknown flags is a feature,
> > > not a bug.
> > >
> > > Chatted with Jim offlist, and he pointed out that KVM's standard way to deal with
> > > this is to make setting the flag opt-in, e.g. KVM_CAP_X86_TRIPLE_FAULT_EVENT and
> > > KVM_CAP_EXCEPTION_PAYLOAD.
> > >
> > > As much as I want to retroactively change KVM's documentation to state doing
> > > KVM_SET_NESTED_STATE with data that didn't come from KVM_GET_NESTED_STATE is
> > > unsupported, that feels too restrictive and could really bite us in the future.
> > > And it doesn't help if there's already userspace that's putting garbage into the
> > > header.
> > >
> > > So yeah, I don't see a better option than adding yet another capability.

Capability or quirk?

/me ducks.