[PATCH v3 0/6] KVM: VMX: Handle the immediate form of MSR instructions

[PATCH v3 0/6] KVM: VMX: Handle the immediate form of MSR instructions

[Sean Christopherson]

On behalf of Xin, to avoid having to resolve conflicts when applying.
This applies on the fastpath cleanup series:
https://lore.kernel.org/all/20250805190526.1453366-1-seanjc@google.com

This patch set handles two newly introduced VM exit reasons associated
with the immediate form of MSR instructions to ensure proper
virtualization of these instructions.

The immediate form of MSR access instructions are primarily motivated
by performance, not code size: by having the MSR number in an immediate,
it is available *much* earlier in the pipeline, which allows the
hardware much more leeway about how a particular MSR is handled.

For proper virtualization of the immediate form of MSR instructions,
Intel VMX architecture adds the following changes:

  1) The immediate form of RDMSR uses VM exit reason 84.

  2) The immediate form of WRMSRNS uses VM exit reason 85.

  3) For both VM exit reasons 84 and 85, the exit qualification is set
     to the MSR address causing the VM exit.

  4) Bits 3 ~ 6 of the VM exit instruction information field represent
     the operand register used in the immediate form of MSR instruction.

  5) The VM-exit instruction length field records the size of the
     immediate form of the MSR instruction.

Note: The VMX specification for the immediate form of MSR instructions
was inadvertently omitted from the last published ISE, but it will be
included in the upcoming edition.

Linux bare metal support of the immediate form of MSR instructions is
still under development; however, the KVM support effort is proceeding
independently of the bare metal implementation.

v3:
 - Rebase on the fastpath cleanups.
 - Split patches to better isolate the functional changes.
 - Massage and expand on a changelogs.
 - Make a handful of (mostly) stylistic changes (shouldn't affect
   functionality, key word "should").

v2: https://lore.kernel.org/all/20250802001520.3142577-1-xin@zytor.com
v1: https://lore.kernel.org/lkml/20250730174605.1614792-1-xin@zytor.com

Sean Christopherson (1):
  KVM: x86: Rename local "ecx" variables to "msr" and "pmc" as
    appropriate

Xin Li (5):
  x86/cpufeatures: Add a CPU feature bit for MSR immediate form
    instructions
  KVM: x86: Rename handle_fastpath_set_msr_irqoff() to
    handle_fastpath_wrmsr()
  KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel
  KVM: VMX: Support the immediate form of WRMSRNS in the VM-Exit
    fastpath
  KVM: x86: Advertise support for the immediate form of MSR instructions

 arch/x86/include/asm/cpufeatures.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  4 ++
 arch/x86/include/uapi/asm/vmx.h    |  6 +-
 arch/x86/kernel/cpu/scattered.c    |  1 +
 arch/x86/kvm/cpuid.c               |  6 +-
 arch/x86/kvm/reverse_cpuid.h       |  5 ++
 arch/x86/kvm/svm/svm.c             |  8 ++-
 arch/x86/kvm/vmx/nested.c          | 13 ++++-
 arch/x86/kvm/vmx/vmx.c             | 26 ++++++++-
 arch/x86/kvm/vmx/vmx.h             |  5 ++
 arch/x86/kvm/x86.c                 | 94 ++++++++++++++++++++++--------
 arch/x86/kvm/x86.h                 |  3 +-
 12 files changed, 139 insertions(+), 33 deletions(-)


base-commit: 53d61a43a7973f812caa08fa922b607574befef4
-- 
2.50.1.565.gc32cd1483b-goog

[PATCH v3 1/6] x86/cpufeatures: Add a CPU feature bit for MSR immediate form instructions

[Sean Christopherson]

From: Xin Li <xin@zytor.com>

The immediate form of MSR access instructions are primarily motivated
by performance, not code size: by having the MSR number in an immediate,
it is available *much* earlier in the pipeline, which allows the
hardware much more leeway about how a particular MSR is handled.

Use a scattered CPU feature bit for MSR immediate form instructions.

Suggested-by: Borislav Petkov (AMD) <bp@alien8.de>
Signed-off-by: Xin Li (Intel) <xin@zytor.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/include/asm/cpufeatures.h | 1 +
 arch/x86/kernel/cpu/scattered.c    | 1 +
 2 files changed, 2 insertions(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 286d509f9363..75b43bbe2a6d 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -491,6 +491,7 @@
 #define X86_FEATURE_TSA_SQ_NO		(21*32+11) /* AMD CPU not vulnerable to TSA-SQ */
 #define X86_FEATURE_TSA_L1_NO		(21*32+12) /* AMD CPU not vulnerable to TSA-L1 */
 #define X86_FEATURE_CLEAR_CPU_BUF_VM	(21*32+13) /* Clear CPU buffers using VERW before VMRUN */
+#define X86_FEATURE_MSR_IMM		(21*32+14) /* MSR immediate form instructions */
 
 /*
  * BUG word(s)
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index b4a1f6732a3a..5fe19bbe538e 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -27,6 +27,7 @@ static const struct cpuid_bit cpuid_bits[] = {
 	{ X86_FEATURE_APERFMPERF,		CPUID_ECX,  0, 0x00000006, 0 },
 	{ X86_FEATURE_EPB,			CPUID_ECX,  3, 0x00000006, 0 },
 	{ X86_FEATURE_INTEL_PPIN,		CPUID_EBX,  0, 0x00000007, 1 },
+	{ X86_FEATURE_MSR_IMM,			CPUID_ECX,  5, 0x00000007, 1 },
 	{ X86_FEATURE_APX,			CPUID_EDX, 21, 0x00000007, 1 },
 	{ X86_FEATURE_RRSBA_CTRL,		CPUID_EDX,  2, 0x00000007, 2 },
 	{ X86_FEATURE_BHI_CTRL,			CPUID_EDX,  4, 0x00000007, 2 },
-- 
2.50.1.565.gc32cd1483b-goog

[PATCH v3 2/6] KVM: x86: Rename local "ecx" variables to "msr" and "pmc" as appropriate

[Sean Christopherson]

Rename "ecx" variables in {RD,WR}MSR and RDPMC helpers to "msr" and "pmc"
respectively, in anticipation of adding support for the immediate variants
of RDMSR and WRMSRNS, and to better document what the variables hold
(versus where the data originated).

No functional change intended.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/kvm/x86.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 5af2c5aed0f2..d3afb875133e 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -1572,10 +1572,10 @@ EXPORT_SYMBOL_GPL(kvm_get_dr);
 
 int kvm_emulate_rdpmc(struct kvm_vcpu *vcpu)
 {
-	u32 ecx = kvm_rcx_read(vcpu);
+	u32 pmc = kvm_rcx_read(vcpu);
 	u64 data;
 
-	if (kvm_pmu_rdpmc(vcpu, ecx, &data)) {
+	if (kvm_pmu_rdpmc(vcpu, pmc, &data)) {
 		kvm_inject_gp(vcpu, 0);
 		return 1;
 	}
@@ -2026,23 +2026,23 @@ static int kvm_msr_user_space(struct kvm_vcpu *vcpu, u32 index,
 
 int kvm_emulate_rdmsr(struct kvm_vcpu *vcpu)
 {
-	u32 ecx = kvm_rcx_read(vcpu);
+	u32 msr = kvm_rcx_read(vcpu);
 	u64 data;
 	int r;
 
-	r = kvm_get_msr_with_filter(vcpu, ecx, &data);
+	r = kvm_get_msr_with_filter(vcpu, msr, &data);
 
 	if (!r) {
-		trace_kvm_msr_read(ecx, data);
+		trace_kvm_msr_read(msr, data);
 
 		kvm_rax_write(vcpu, data & -1u);
 		kvm_rdx_write(vcpu, (data >> 32) & -1u);
 	} else {
 		/* MSR read failed? See if we should ask user space */
-		if (kvm_msr_user_space(vcpu, ecx, KVM_EXIT_X86_RDMSR, 0,
+		if (kvm_msr_user_space(vcpu, msr, KVM_EXIT_X86_RDMSR, 0,
 				       complete_fast_rdmsr, r))
 			return 0;
-		trace_kvm_msr_read_ex(ecx);
+		trace_kvm_msr_read_ex(msr);
 	}
 
 	return kvm_x86_call(complete_emulated_msr)(vcpu, r);
@@ -2051,23 +2051,23 @@ EXPORT_SYMBOL_GPL(kvm_emulate_rdmsr);
 
 int kvm_emulate_wrmsr(struct kvm_vcpu *vcpu)
 {
-	u32 ecx = kvm_rcx_read(vcpu);
+	u32 msr = kvm_rcx_read(vcpu);
 	u64 data = kvm_read_edx_eax(vcpu);
 	int r;
 
-	r = kvm_set_msr_with_filter(vcpu, ecx, data);
+	r = kvm_set_msr_with_filter(vcpu, msr, data);
 
 	if (!r) {
-		trace_kvm_msr_write(ecx, data);
+		trace_kvm_msr_write(msr, data);
 	} else {
 		/* MSR write failed? See if we should ask user space */
-		if (kvm_msr_user_space(vcpu, ecx, KVM_EXIT_X86_WRMSR, data,
+		if (kvm_msr_user_space(vcpu, msr, KVM_EXIT_X86_WRMSR, data,
 				       complete_fast_msr_access, r))
 			return 0;
 		/* Signal all other negative errors to userspace */
 		if (r < 0)
 			return r;
-		trace_kvm_msr_write_ex(ecx, data);
+		trace_kvm_msr_write_ex(msr, data);
 	}
 
 	return kvm_x86_call(complete_emulated_msr)(vcpu, r);
-- 
2.50.1.565.gc32cd1483b-goog

Re: [PATCH v3 2/6] KVM: x86: Rename local "ecx" variables to "msr" and "pmc" as appropriate

[Xin Li]

On 8/5/2025 1:22 PM, Sean Christopherson wrote:
> Rename "ecx" variables in {RD,WR}MSR and RDPMC helpers to "msr" and "pmc"
> respectively, in anticipation of adding support for the immediate variants
> of RDMSR and WRMSRNS, and to better document what the variables hold
> (versus where the data originated).
> 
> No functional change intended.
> 
> Signed-off-by: Sean Christopherson<seanjc@google.com>

Yeah, this makes the immediate MSR handling changes in the following 
patches prominent.

[PATCH v3 3/6] KVM: x86: Rename handle_fastpath_set_msr_irqoff() to handle_fastpath_wrmsr()

[Sean Christopherson]

From: Xin Li <xin@zytor.com>

Rename the WRMSR fastpath API to drop "irqoff", as that information is
redundant (the fastpath always runs with IRQs disabled), and to prepare
for adding a fastpath for the immediate variant of WRMSRNS.

No functional change intended.

Signed-off-by: Xin Li (Intel) <xin@zytor.com>
[sean: split to separate patch, write changelog]
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/kvm/svm/svm.c | 2 +-
 arch/x86/kvm/vmx/vmx.c | 2 +-
 arch/x86/kvm/x86.c     | 4 ++--
 arch/x86/kvm/x86.h     | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index f7e1e665a826..ca550c4fa174 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -4197,7 +4197,7 @@ static fastpath_t svm_exit_handlers_fastpath(struct kvm_vcpu *vcpu)
 	case SVM_EXIT_MSR:
 		if (!control->exit_info_1)
 			break;
-		return handle_fastpath_set_msr_irqoff(vcpu);
+		return handle_fastpath_wrmsr(vcpu);
 	case SVM_EXIT_HLT:
 		return handle_fastpath_hlt(vcpu);
 	case SVM_EXIT_INVD:
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 95765db52992..ae2c8c10e5d2 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -7170,7 +7170,7 @@ static fastpath_t vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu,
 
 	switch (vmx_get_exit_reason(vcpu).basic) {
 	case EXIT_REASON_MSR_WRITE:
-		return handle_fastpath_set_msr_irqoff(vcpu);
+		return handle_fastpath_wrmsr(vcpu);
 	case EXIT_REASON_PREEMPTION_TIMER:
 		return handle_fastpath_preemption_timer(vcpu, force_immediate_exit);
 	case EXIT_REASON_HLT:
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index d3afb875133e..6470f0ab2060 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2142,7 +2142,7 @@ static inline bool kvm_vcpu_exit_request(struct kvm_vcpu *vcpu)
 	       kvm_request_pending(vcpu) || xfer_to_guest_mode_work_pending();
 }
 
-fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu)
+fastpath_t handle_fastpath_wrmsr(struct kvm_vcpu *vcpu)
 {
 	u64 data = kvm_read_edx_eax(vcpu);
 	u32 msr = kvm_rcx_read(vcpu);
@@ -2167,7 +2167,7 @@ fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu)
 
 	return EXIT_FASTPATH_REENTER_GUEST;
 }
-EXPORT_SYMBOL_GPL(handle_fastpath_set_msr_irqoff);
+EXPORT_SYMBOL_GPL(handle_fastpath_wrmsr);
 
 /*
  * Adapt set_msr() to msr_io()'s calling convention
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index 46220b04cdf2..2dab9c9d6199 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -437,7 +437,7 @@ int x86_decode_emulated_instruction(struct kvm_vcpu *vcpu, int emulation_type,
 				    void *insn, int insn_len);
 int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
 			    int emulation_type, void *insn, int insn_len);
-fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu);
+fastpath_t handle_fastpath_wrmsr(struct kvm_vcpu *vcpu);
 fastpath_t handle_fastpath_hlt(struct kvm_vcpu *vcpu);
 fastpath_t handle_fastpath_invd(struct kvm_vcpu *vcpu);
 
-- 
2.50.1.565.gc32cd1483b-goog

Re: [PATCH v3 3/6] KVM: x86: Rename handle_fastpath_set_msr_irqoff() to handle_fastpath_wrmsr()

[Xin Li]

On 8/5/2025 1:22 PM, Sean Christopherson wrote:
> From: Xin Li <xin@zytor.com>
> 
> Rename the WRMSR fastpath API to drop "irqoff", as that information is
> redundant (the fastpath always runs with IRQs disabled), and to prepare
> for adding a fastpath for the immediate variant of WRMSRNS.
> 
> No functional change intended.
> 

I think it makes sense to add:

Suggested-by: Sean Christopherson <seanjc@google.com>

I didn't add it in v2 because the changes are part of a patch that
handles the immediate form MSR instructions.

> Signed-off-by: Xin Li (Intel) <xin@zytor.com>
> [sean: split to separate patch, write changelog]
> Signed-off-by: Sean Christopherson <seanjc@google.com>

[PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel

[Sean Christopherson]

From: Xin Li <xin@zytor.com>

Add support for the immediate forms of RDMSR and WRMSRNS (currently
Intel-only).  The immediate variants are only valid in 64-bit mode, and
use a single general purpose register for the data (the register is also
encoded in the instruction, i.e. not implicit like regular RDMSR/WRMSR).

The immediate variants are primarily motivated by performance, not code
size: by having the MSR index in an immediate, it is available *much*
earlier in the CPU pipeline, which allows hardware much more leeway about
how a particular MSR is handled.

Intel VMX support for the immediate forms of MSR accesses communicates
exit information to the host as follows:

  1) The immediate form of RDMSR uses VM-Exit Reason 84.

  2) The immediate form of WRMSRNS uses VM-Exit Reason 85.

  3) For both VM-Exit reasons 84 and 85, the Exit Qualification field is
     set to the MSR index that triggered the VM-Exit.

  4) Bits 3 ~ 6 of the VM-Exit Instruction Information field are set to
     the register encoding used by the immediate form of the instruction,
     i.e. the destination register for RDMSR, and the source for WRMSRNS.

  5) The VM-Exit Instruction Length field records the size of the
     immediate form of the MSR instruction.

To deal with userspace RDMSR exits, stash the destination register in a
new kvm_vcpu_arch field, similar to cui_linear_rip, pio, etc.
Alternatively, the register could be saved in kvm_run.msr or re-retrieved
from the VMCS, but the former would require sanitizing the value to ensure
userspace doesn't clobber the value to an out-of-bounds index, and the
latter would require a new one-off kvm_x86_ops hook.

Don't bother adding support for the instructions in KVM's emulator, as the
only way for RDMSR/WRMSR to be encountered is if KVM is emulating large
swaths of code due to invalid guest state, and a vCPU cannot have invalid
guest state while in 64-bit mode.

Signed-off-by: Xin Li (Intel) <xin@zytor.com>
[sean: minor tweaks, massage and expand changelog]
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/include/asm/kvm_host.h |  3 ++
 arch/x86/include/uapi/asm/vmx.h |  6 +++-
 arch/x86/kvm/vmx/nested.c       | 13 ++++++--
 arch/x86/kvm/vmx/vmx.c          | 21 +++++++++++++
 arch/x86/kvm/vmx/vmx.h          |  5 +++
 arch/x86/kvm/x86.c              | 55 +++++++++++++++++++++++++++------
 6 files changed, 90 insertions(+), 13 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index d7680612ba1e..dbdec6025fde 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -929,6 +929,7 @@ struct kvm_vcpu_arch {
 	bool emulate_regs_need_sync_from_vcpu;
 	int (*complete_userspace_io)(struct kvm_vcpu *vcpu);
 	unsigned long cui_linear_rip;
+	int cui_rdmsr_imm_reg;
 
 	gpa_t time;
 	s8  pvclock_tsc_shift;
@@ -2158,7 +2159,9 @@ int __kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data, bool host_initiat
 int kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data);
 int kvm_set_msr(struct kvm_vcpu *vcpu, u32 index, u64 data);
 int kvm_emulate_rdmsr(struct kvm_vcpu *vcpu);
+int kvm_emulate_rdmsr_imm(struct kvm_vcpu *vcpu, u32 msr, int reg);
 int kvm_emulate_wrmsr(struct kvm_vcpu *vcpu);
+int kvm_emulate_wrmsr_imm(struct kvm_vcpu *vcpu, u32 msr, int reg);
 int kvm_emulate_as_nop(struct kvm_vcpu *vcpu);
 int kvm_emulate_invd(struct kvm_vcpu *vcpu);
 int kvm_emulate_mwait(struct kvm_vcpu *vcpu);
diff --git a/arch/x86/include/uapi/asm/vmx.h b/arch/x86/include/uapi/asm/vmx.h
index f0f4a4cf84a7..9792e329343e 100644
--- a/arch/x86/include/uapi/asm/vmx.h
+++ b/arch/x86/include/uapi/asm/vmx.h
@@ -94,6 +94,8 @@
 #define EXIT_REASON_BUS_LOCK            74
 #define EXIT_REASON_NOTIFY              75
 #define EXIT_REASON_TDCALL              77
+#define EXIT_REASON_MSR_READ_IMM        84
+#define EXIT_REASON_MSR_WRITE_IMM       85
 
 #define VMX_EXIT_REASONS \
 	{ EXIT_REASON_EXCEPTION_NMI,         "EXCEPTION_NMI" }, \
@@ -158,7 +160,9 @@
 	{ EXIT_REASON_TPAUSE,                "TPAUSE" }, \
 	{ EXIT_REASON_BUS_LOCK,              "BUS_LOCK" }, \
 	{ EXIT_REASON_NOTIFY,                "NOTIFY" }, \
-	{ EXIT_REASON_TDCALL,                "TDCALL" }
+	{ EXIT_REASON_TDCALL,                "TDCALL" }, \
+	{ EXIT_REASON_MSR_READ_IMM,          "MSR_READ_IMM" }, \
+	{ EXIT_REASON_MSR_WRITE_IMM,         "MSR_WRITE_IMM" }
 
 #define VMX_EXIT_REASON_FLAGS \
 	{ VMX_EXIT_REASONS_FAILED_VMENTRY,	"FAILED_VMENTRY" }
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index db2fd4eedc90..798776dddd43 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -6216,19 +6216,26 @@ static bool nested_vmx_exit_handled_msr(struct kvm_vcpu *vcpu,
 					struct vmcs12 *vmcs12,
 					union vmx_exit_reason exit_reason)
 {
-	u32 msr_index = kvm_rcx_read(vcpu);
+	u32 msr_index;
 	gpa_t bitmap;
 
 	if (!nested_cpu_has(vmcs12, CPU_BASED_USE_MSR_BITMAPS))
 		return true;
 
+	if (exit_reason.basic == EXIT_REASON_MSR_READ_IMM ||
+	    exit_reason.basic == EXIT_REASON_MSR_WRITE_IMM)
+		msr_index = vmx_get_exit_qual(vcpu);
+	else
+		msr_index = kvm_rcx_read(vcpu);
+
 	/*
 	 * The MSR_BITMAP page is divided into four 1024-byte bitmaps,
 	 * for the four combinations of read/write and low/high MSR numbers.
 	 * First we need to figure out which of the four to use:
 	 */
 	bitmap = vmcs12->msr_bitmap;
-	if (exit_reason.basic == EXIT_REASON_MSR_WRITE)
+	if (exit_reason.basic == EXIT_REASON_MSR_WRITE ||
+	    exit_reason.basic == EXIT_REASON_MSR_WRITE_IMM)
 		bitmap += 2048;
 	if (msr_index >= 0xc0000000) {
 		msr_index -= 0xc0000000;
@@ -6527,6 +6534,8 @@ static bool nested_vmx_l1_wants_exit(struct kvm_vcpu *vcpu,
 		return nested_cpu_has2(vmcs12, SECONDARY_EXEC_DESC);
 	case EXIT_REASON_MSR_READ:
 	case EXIT_REASON_MSR_WRITE:
+	case EXIT_REASON_MSR_READ_IMM:
+	case EXIT_REASON_MSR_WRITE_IMM:
 		return nested_vmx_exit_handled_msr(vcpu, vmcs12, exit_reason);
 	case EXIT_REASON_INVALID_STATE:
 		return true;
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index ae2c8c10e5d2..44423d5f0e27 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -6003,6 +6003,23 @@ static int handle_notify(struct kvm_vcpu *vcpu)
 	return 1;
 }
 
+static int vmx_get_msr_imm_reg(struct kvm_vcpu *vcpu)
+{
+	return vmx_get_instr_info_reg(vmcs_read32(VMX_INSTRUCTION_INFO));
+}
+
+static int handle_rdmsr_imm(struct kvm_vcpu *vcpu)
+{
+	return kvm_emulate_rdmsr_imm(vcpu, vmx_get_exit_qual(vcpu),
+				     vmx_get_msr_imm_reg(vcpu));
+}
+
+static int handle_wrmsr_imm(struct kvm_vcpu *vcpu)
+{
+	return kvm_emulate_wrmsr_imm(vcpu, vmx_get_exit_qual(vcpu),
+				     vmx_get_msr_imm_reg(vcpu));
+}
+
 /*
  * The exit handlers return 1 if the exit was handled fully and guest execution
  * may resume.  Otherwise they set the kvm_run parameter to indicate what needs
@@ -6061,6 +6078,8 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = {
 	[EXIT_REASON_ENCLS]		      = handle_encls,
 	[EXIT_REASON_BUS_LOCK]                = handle_bus_lock_vmexit,
 	[EXIT_REASON_NOTIFY]		      = handle_notify,
+	[EXIT_REASON_MSR_READ_IMM]            = handle_rdmsr_imm,
+	[EXIT_REASON_MSR_WRITE_IMM]           = handle_wrmsr_imm,
 };
 
 static const int kvm_vmx_max_exit_handlers =
@@ -6495,6 +6514,8 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)
 #ifdef CONFIG_MITIGATION_RETPOLINE
 	if (exit_reason.basic == EXIT_REASON_MSR_WRITE)
 		return kvm_emulate_wrmsr(vcpu);
+	else if (exit_reason.basic == EXIT_REASON_MSR_WRITE_IMM)
+		return handle_wrmsr_imm(vcpu);
 	else if (exit_reason.basic == EXIT_REASON_PREEMPTION_TIMER)
 		return handle_preemption_timer(vcpu);
 	else if (exit_reason.basic == EXIT_REASON_INTERRUPT_WINDOW)
diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h
index d3389baf3ab3..24d65dac5e89 100644
--- a/arch/x86/kvm/vmx/vmx.h
+++ b/arch/x86/kvm/vmx/vmx.h
@@ -706,6 +706,11 @@ static inline bool vmx_guest_state_valid(struct kvm_vcpu *vcpu)
 
 void dump_vmcs(struct kvm_vcpu *vcpu);
 
+static inline int vmx_get_instr_info_reg(u32 vmx_instr_info)
+{
+	return (vmx_instr_info >> 3) & 0xf;
+}
+
 static inline int vmx_get_instr_info_reg2(u32 vmx_instr_info)
 {
 	return (vmx_instr_info >> 28) & 0xf;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 6470f0ab2060..79c3074dbd60 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -1990,6 +1990,15 @@ static int complete_fast_rdmsr(struct kvm_vcpu *vcpu)
 	return complete_fast_msr_access(vcpu);
 }
 
+static int complete_fast_rdmsr_imm(struct kvm_vcpu *vcpu)
+{
+	if (!vcpu->run->msr.error)
+		kvm_register_write(vcpu, vcpu->arch.cui_rdmsr_imm_reg,
+				   vcpu->run->msr.data);
+
+	return complete_fast_msr_access(vcpu);
+}
+
 static u64 kvm_msr_reason(int r)
 {
 	switch (r) {
@@ -2024,39 +2033,53 @@ static int kvm_msr_user_space(struct kvm_vcpu *vcpu, u32 index,
 	return 1;
 }
 
-int kvm_emulate_rdmsr(struct kvm_vcpu *vcpu)
+static int __kvm_emulate_rdmsr(struct kvm_vcpu *vcpu, u32 msr, int reg,
+			       int (*complete_rdmsr)(struct kvm_vcpu *))
 {
-	u32 msr = kvm_rcx_read(vcpu);
 	u64 data;
 	int r;
 
 	r = kvm_get_msr_with_filter(vcpu, msr, &data);
-
 	if (!r) {
 		trace_kvm_msr_read(msr, data);
 
-		kvm_rax_write(vcpu, data & -1u);
-		kvm_rdx_write(vcpu, (data >> 32) & -1u);
+		if (reg < 0) {
+			kvm_rax_write(vcpu, data & -1u);
+			kvm_rdx_write(vcpu, (data >> 32) & -1u);
+		} else {
+			kvm_register_write(vcpu, reg, data);
+		}
 	} else {
 		/* MSR read failed? See if we should ask user space */
 		if (kvm_msr_user_space(vcpu, msr, KVM_EXIT_X86_RDMSR, 0,
-				       complete_fast_rdmsr, r))
+				       complete_rdmsr, r))
 			return 0;
 		trace_kvm_msr_read_ex(msr);
 	}
 
 	return kvm_x86_call(complete_emulated_msr)(vcpu, r);
 }
+
+int kvm_emulate_rdmsr(struct kvm_vcpu *vcpu)
+{
+	return __kvm_emulate_rdmsr(vcpu, kvm_rcx_read(vcpu), -1,
+				   complete_fast_rdmsr);
+}
 EXPORT_SYMBOL_GPL(kvm_emulate_rdmsr);
 
-int kvm_emulate_wrmsr(struct kvm_vcpu *vcpu)
+int kvm_emulate_rdmsr_imm(struct kvm_vcpu *vcpu, u32 msr, int reg)
+{
+	vcpu->arch.cui_rdmsr_imm_reg = reg;
+
+	return __kvm_emulate_rdmsr(vcpu, msr, reg, complete_fast_rdmsr_imm);
+}
+EXPORT_SYMBOL_GPL(kvm_emulate_rdmsr_imm);
+
+static int __kvm_emulate_wrmsr(struct kvm_vcpu *vcpu, u32 msr, u64 data)
 {
-	u32 msr = kvm_rcx_read(vcpu);
-	u64 data = kvm_read_edx_eax(vcpu);
 	int r;
 
 	r = kvm_set_msr_with_filter(vcpu, msr, data);
-
 	if (!r) {
 		trace_kvm_msr_write(msr, data);
 	} else {
@@ -2072,8 +2095,20 @@ int kvm_emulate_wrmsr(struct kvm_vcpu *vcpu)
 
 	return kvm_x86_call(complete_emulated_msr)(vcpu, r);
 }
+
+int kvm_emulate_wrmsr(struct kvm_vcpu *vcpu)
+{
+	return __kvm_emulate_wrmsr(vcpu, kvm_rcx_read(vcpu),
+				   kvm_read_edx_eax(vcpu));
+}
 EXPORT_SYMBOL_GPL(kvm_emulate_wrmsr);
 
+int kvm_emulate_wrmsr_imm(struct kvm_vcpu *vcpu, u32 msr, int reg)
+{
+	return __kvm_emulate_wrmsr(vcpu, msr, kvm_register_read(vcpu, reg));
+}
+EXPORT_SYMBOL_GPL(kvm_emulate_wrmsr_imm);
+
 int kvm_emulate_as_nop(struct kvm_vcpu *vcpu)
 {
 	return kvm_skip_emulated_instruction(vcpu);
-- 
2.50.1.565.gc32cd1483b-goog

Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel

[Xiaoyao Li]

On 8/6/2025 4:22 AM, Sean Christopherson wrote:
> +static int handle_rdmsr_imm(struct kvm_vcpu *vcpu)
> +{
> +	return kvm_emulate_rdmsr_imm(vcpu, vmx_get_exit_qual(vcpu),
> +				     vmx_get_msr_imm_reg(vcpu));
> +}
> +
> +static int handle_wrmsr_imm(struct kvm_vcpu *vcpu)
> +{
> +	return kvm_emulate_wrmsr_imm(vcpu, vmx_get_exit_qual(vcpu),
> +				     vmx_get_msr_imm_reg(vcpu));
> +}

We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)

Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel

[Binbin Wu]

On 9/1/2025 12:13 PM, Xiaoyao Li wrote:
> On 8/6/2025 4:22 AM, Sean Christopherson wrote:
>> +static int handle_rdmsr_imm(struct kvm_vcpu *vcpu)
>> +{
>> +    return kvm_emulate_rdmsr_imm(vcpu, vmx_get_exit_qual(vcpu),
>> +                     vmx_get_msr_imm_reg(vcpu));
>> +}
>> +
>> +static int handle_wrmsr_imm(struct kvm_vcpu *vcpu)
>> +{
>> +    return kvm_emulate_wrmsr_imm(vcpu, vmx_get_exit_qual(vcpu),
>> +                     vmx_get_msr_imm_reg(vcpu));
>> +}
>
> We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
>

Indeed.

There is a virtualization hole of this feature for the accesses to the MSRs not
intercepted. IIUIC, there is no other control in VMX for this feature. If the
feature is supported in hardware, the guest will succeed when it accesses to the
MSRs not intercepted even when the feature is not exposed to the guest, but the
guest will get #UD when access to the MSRs intercepted if KVM injects #UD.

But I guess this is the guest's fault by not following the CPUID, KVM should
still follow the spec?

Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel

[Xin Li]

On 8/31/2025 11:34 PM, Binbin Wu wrote:
>> We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
>>
> 
> Indeed.

Good catch!

> 
> There is a virtualization hole of this feature for the accesses to the MSRs 
> not
> intercepted. IIUIC, there is no other control in VMX for this feature. If the
> feature is supported in hardware, the guest will succeed when it accesses 
> to the
> MSRs not intercepted even when the feature is not exposed to the guest, but 
> the
> guest will get #UD when access to the MSRs intercepted if KVM injects #UD.

hpa mentioned this when I just started the work.  But I managed to forget
it later... Sigh!

> 
> But I guess this is the guest's fault by not following the CPUID, KVM should
> still follow the spec?

I think we should still inject #UD when a MSR is intercepted by KVM.

Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel

[Xiaoyao Li]

On 9/1/2025 3:04 PM, Xin Li wrote:
> On 8/31/2025 11:34 PM, Binbin Wu wrote:
>>> We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
>>>
>>
>> Indeed.
> 
> Good catch!
> 
>>
>> There is a virtualization hole of this feature for the accesses to the 
>> MSRs not
>> intercepted. IIUIC, there is no other control in VMX for this feature. 
>> If the
>> feature is supported in hardware, the guest will succeed when it 
>> accesses to the
>> MSRs not intercepted even when the feature is not exposed to the 
>> guest, but the
>> guest will get #UD when access to the MSRs intercepted if KVM injects 
>> #UD.
> 
> hpa mentioned this when I just started the work.  But I managed to forget
> it later... Sigh!
> 
>>
>> But I guess this is the guest's fault by not following the CPUID, KVM 
>> should
>> still follow the spec?
> 
> I think we should still inject #UD when a MSR is intercepted by KVM.
> 

For handle_wrmsr_imm(), it seems we need to check 
guest_cpu_cap_has(X86_FEATURE_WRMSRNS) as well, since immediate form of 
MSR write is only supported on WRMSRNS instruction.

It leads to another topic, do we need to bother checking the opcode of 
the instruction on EXIT_REASON_MSR_WRITE and inject #UD when it is 
WRMSRNS instuction and !guest_cpu_cap_has(X86_FEATURE_WRMSRNS)?

WRMSRNS has virtualization hole as well, but KVM at least can emulate 
the architectural behavior when the write on MSRs are not pass through.

Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel

[Sean Christopherson]

On Mon, Sep 01, 2025, Xiaoyao Li wrote:
> On 9/1/2025 3:04 PM, Xin Li wrote:
> > On 8/31/2025 11:34 PM, Binbin Wu wrote:
> > > > We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
> > > > 
> > > 
> > > Indeed.
> > 
> > Good catch!
> > 
> > > 
> > > There is a virtualization hole of this feature for the accesses to the
> > > MSRs not intercepted. IIUIC, there is no other control in VMX for this
> > > feature. If the feature is supported in hardware, the guest will succeed
> > > when it accesses to the MSRs not intercepted even when the feature is not
> > > exposed to the guest, but the guest will get #UD when access to the MSRs
> > > intercepted if KVM injects #UD.
> > 
> > hpa mentioned this when I just started the work.  But I managed to forget
> > it later... Sigh!
> > 
> > > 
> > > But I guess this is the guest's fault by not following the CPUID,
> > > KVM should
> > > still follow the spec?
> > 
> > I think we should still inject #UD when a MSR is intercepted by KVM.

Hmm, no, inconsistent behavior (from the guest's perspective) is likely worse
than eating with the virtualization hole.  Practically speaking, the only guest
that's going to be surprised by the hole is a guest that's fuzzing opcodes, and
a guest that's fuzzing opcodes at CPL0 isn't is going to create an inherently
unstable environment no matter what.

Though that raises the question of whether or not KVM should emulate WRMSRNS and
whatever the official name for the "RDMSR with immediate" instruction is (I can't
find it in the SDM).  I'm leaning "no", because outside of forced emulation, KVM
should only "need" to emulate the instructions if Unrestricted Guest is disabled,
the instructions should only be supported on CPUs with unrestricted guest, there's
no sane reason (other than testing) to run a guest without Unrestricted Guest,
and using the instructions in Big RM would be quite bizarre.  On the other hand,
adding emulation support should be quite easy...

Side topic, does RDMSRLIST have any VMX controls?

> For handle_wrmsr_imm(), it seems we need to check
> guest_cpu_cap_has(X86_FEATURE_WRMSRNS) as well, since immediate form of MSR
> write is only supported on WRMSRNS instruction.
> 
> It leads to another topic, do we need to bother checking the opcode of the
> instruction on EXIT_REASON_MSR_WRITE and inject #UD when it is WRMSRNS
> instuction and !guest_cpu_cap_has(X86_FEATURE_WRMSRNS)?
> 
> WRMSRNS has virtualization hole as well, but KVM at least can emulate the
> architectural behavior when the write on MSRs are not pass through.

Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel

[Xin Li]

On 9/2/2025 7:50 AM, Sean Christopherson wrote:
> On Mon, Sep 01, 2025, Xiaoyao Li wrote:
>> On 9/1/2025 3:04 PM, Xin Li wrote:
>>> On 8/31/2025 11:34 PM, Binbin Wu wrote:
>>>>> We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
>>>>>
>>>>
>>>> Indeed.
>>>
>>> Good catch!
>>>
>>>>
>>>> There is a virtualization hole of this feature for the accesses to the
>>>> MSRs not intercepted. IIUIC, there is no other control in VMX for this
>>>> feature. If the feature is supported in hardware, the guest will succeed
>>>> when it accesses to the MSRs not intercepted even when the feature is not
>>>> exposed to the guest, but the guest will get #UD when access to the MSRs
>>>> intercepted if KVM injects #UD.
>>>
>>> hpa mentioned this when I just started the work.  But I managed to forget
>>> it later... Sigh!
>>>
>>>>
>>>> But I guess this is the guest's fault by not following the CPUID,
>>>> KVM should
>>>> still follow the spec?
>>>
>>> I think we should still inject #UD when a MSR is intercepted by KVM.
> 
> Hmm, no, inconsistent behavior (from the guest's perspective) is likely worse
> than eating with the virtualization hole.  Practically speaking, the only guest
> that's going to be surprised by the hole is a guest that's fuzzing opcodes, and
> a guest that's fuzzing opcodes at CPL0 isn't is going to create an inherently
> unstable environment no matter what.

Hmm, a malicious guest could *smartly* avoid causing such vmexits.  So more
or less that is wasteful and no objection here.

> 
> Though that raises the question of whether or not KVM should emulate WRMSRNS and
> whatever the official name for the "RDMSR with immediate" instruction is (I can't
> find it in the SDM).  I'm leaning "no", because outside of forced emulation, KVM
> should only "need" to emulate the instructions if Unrestricted Guest is disabled,
> the instructions should only be supported on CPUs with unrestricted guest, there's
> no sane reason (other than testing) to run a guest without Unrestricted Guest,
> and using the instructions in Big RM would be quite bizarre.  On the other hand,
> adding emulation support should be quite easy...
> 
> Side topic, does RDMSRLIST have any VMX controls?

Yes, bit 6 of Tertiary Processor-Based VM-Execution Controls, below is C&P
from Intel SDM:

Enable MSR-list instructions control: if this control is 0, any execution 
of RDMSRLIST or WRMSRLIST causes a #UD.

If the control is 1, the instruction commences normally, writing one MSR at
a time. Writes to certain MSRs are treated specially as described above for
WRMSR and WRMSRNS. In addition, attempts to access specific MSRs may cause 
VM exits.

For RDMSRLIST and WRMSRLIST, the exit qualification depends on the setting
of the “use MSR bitmaps” VM-execution control. If the control is 0, the
exit qualification is zero. If the control is 1, the exit qualification
is the index of the MSR whose access caused the VM exit.

Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel

[Xiaoyao Li]

On 9/2/2025 10:50 PM, Sean Christopherson wrote:
> On Mon, Sep 01, 2025, Xiaoyao Li wrote:
>> On 9/1/2025 3:04 PM, Xin Li wrote:
>>> On 8/31/2025 11:34 PM, Binbin Wu wrote:
>>>>> We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
>>>>>
>>>>
>>>> Indeed.
>>>
>>> Good catch!
>>>
>>>>
>>>> There is a virtualization hole of this feature for the accesses to the
>>>> MSRs not intercepted. IIUIC, there is no other control in VMX for this
>>>> feature. If the feature is supported in hardware, the guest will succeed
>>>> when it accesses to the MSRs not intercepted even when the feature is not
>>>> exposed to the guest, but the guest will get #UD when access to the MSRs
>>>> intercepted if KVM injects #UD.
>>>
>>> hpa mentioned this when I just started the work.  But I managed to forget
>>> it later... Sigh!
>>>
>>>>
>>>> But I guess this is the guest's fault by not following the CPUID,
>>>> KVM should
>>>> still follow the spec?
>>>
>>> I think we should still inject #UD when a MSR is intercepted by KVM.
> 
> Hmm, no, inconsistent behavior (from the guest's perspective) is likely worse
> than eating with the virtualization hole.  

Then could we document this design decision somewhere?

I believe people won't stop wondering why not inject #UD when no guest 
CPUID, when reading the code.

> Practically speaking, the only guest
> that's going to be surprised by the hole is a guest that's fuzzing opcodes, and
> a guest that's fuzzing opcodes at CPL0 isn't is going to create an inherently
> unstable environment no matter what.
> 
> Though that raises the question of whether or not KVM should emulate WRMSRNS and
> whatever the official name for the "RDMSR with immediate" instruction is (I can't
> find it in the SDM).  

do you mean because guest might be able to use immediate form of MSR 
access even if the CPUID doesn't advertise it, should KVM emulate it on 
platform doesn't support it, to make sure immediate form of MSR access 
is always supported?

> I'm leaning "no", because outside of forced emulation, KVM
> should only "need" to emulate the instructions if Unrestricted Guest is disabled,
> the instructions should only be supported on CPUs with unrestricted guest, there's
> no sane reason (other than testing) to run a guest without Unrestricted Guest,
> and using the instructions in Big RM would be quite bizarre.  On the other hand,
> adding emulation support should be quite easy...
> 
> Side topic, does RDMSRLIST have any VMX controls?
> 
>> For handle_wrmsr_imm(), it seems we need to check
>> guest_cpu_cap_has(X86_FEATURE_WRMSRNS) as well, since immediate form of MSR
>> write is only supported on WRMSRNS instruction.
>>
>> It leads to another topic, do we need to bother checking the opcode of the
>> instruction on EXIT_REASON_MSR_WRITE and inject #UD when it is WRMSRNS
>> instuction and !guest_cpu_cap_has(X86_FEATURE_WRMSRNS)?
>>
>> WRMSRNS has virtualization hole as well, but KVM at least can emulate the
>> architectural behavior when the write on MSRs are not pass through.

Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel

[Sean Christopherson]

On Wed, Sep 03, 2025, Xiaoyao Li wrote:
> On 9/2/2025 10:50 PM, Sean Christopherson wrote:
> > On Mon, Sep 01, 2025, Xiaoyao Li wrote:
> > > On 9/1/2025 3:04 PM, Xin Li wrote:
> > > > On 8/31/2025 11:34 PM, Binbin Wu wrote:
> > > > > > We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
> > > > > > 
> > > > > 
> > > > > Indeed.
> > > > 
> > > > Good catch!
> > > > 
> > > > > 
> > > > > There is a virtualization hole of this feature for the accesses to the
> > > > > MSRs not intercepted. IIUIC, there is no other control in VMX for this
> > > > > feature. If the feature is supported in hardware, the guest will succeed
> > > > > when it accesses to the MSRs not intercepted even when the feature is not
> > > > > exposed to the guest, but the guest will get #UD when access to the MSRs
> > > > > intercepted if KVM injects #UD.
> > > > 
> > > > hpa mentioned this when I just started the work.  But I managed to forget
> > > > it later... Sigh!
> > > > 
> > > > > 
> > > > > But I guess this is the guest's fault by not following the CPUID,
> > > > > KVM should
> > > > > still follow the spec?
> > > > 
> > > > I think we should still inject #UD when a MSR is intercepted by KVM.
> > 
> > Hmm, no, inconsistent behavior (from the guest's perspective) is likely worse
> > than eating with the virtualization hole.
> 
> Then could we document this design decision somewhere?

Yeah, Documentation/virt/kvm/x86/errata.rst would be the place for that.

> I believe people won't stop wondering why not inject #UD when no guest
> CPUID, when reading the code.
> 
> > Practically speaking, the only guest that's going to be surprised by the
> > hole is a guest that's fuzzing opcodes, and a guest that's fuzzing opcodes
> > at CPL0 isn't is going to create an inherently unstable environment no
> > matter what.
> > 
> > Though that raises the question of whether or not KVM should emulate WRMSRNS and
> > whatever the official name for the "RDMSR with immediate" instruction is (I can't
> > find it in the SDM).
> 
> do you mean because guest might be able to use immediate form of MSR access
> even if the CPUID doesn't advertise it, should KVM emulate it on platform
> doesn't support it, to make sure immediate form of MSR access is always
> supported?

No, I'm calling out that as implemented, KVM doesn't support emulating WRMSRNS
at all.  Most of me just doesn't care.  The only way to encounter WRMSRNS would
be to disable unrestricted guest and execute WRMSRNS in an odd context, or to
force emulation.  Adding emulation support just for forced emulation is absurd,
and adding WRMSRNS emulation for !URG is almost as ridiculous.

[PATCH v3 5/6] KVM: VMX: Support the immediate form of WRMSRNS in the VM-Exit fastpath

[Sean Christopherson]

From: Xin Li <xin@zytor.com>

Add support for handling "WRMSRNS with an immediate" VM-Exits in KVM's
fastpath.  On Intel, all writes to the x2APIC ICR and to the TSC Deadline
MSR are non-serializing, i.e. it's highly likely guest kernels will switch
to using WRMSRNS when possible.  And in general, any MSR written via
WRMSRNS is probably worth handling in the fastpath, as the entire point of
WRMSRNS is to shave cycles in hot paths.

Signed-off-by: Xin Li (Intel) <xin@zytor.com>
[sean: rewrite changelog, split rename to separate patch]
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/kvm/vmx/vmx.c |  3 +++
 arch/x86/kvm/x86.c     | 17 +++++++++++++----
 arch/x86/kvm/x86.h     |  1 +
 3 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 44423d5f0e27..a3f0d458be9d 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -7192,6 +7192,9 @@ static fastpath_t vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu,
 	switch (vmx_get_exit_reason(vcpu).basic) {
 	case EXIT_REASON_MSR_WRITE:
 		return handle_fastpath_wrmsr(vcpu);
+	case EXIT_REASON_MSR_WRITE_IMM:
+		return handle_fastpath_wrmsr_imm(vcpu, vmx_get_exit_qual(vcpu),
+						 vmx_get_msr_imm_reg(vcpu));
 	case EXIT_REASON_PREEMPTION_TIMER:
 		return handle_fastpath_preemption_timer(vcpu, force_immediate_exit);
 	case EXIT_REASON_HLT:
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 79c3074dbd60..68b95ab4b23f 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2177,11 +2177,8 @@ static inline bool kvm_vcpu_exit_request(struct kvm_vcpu *vcpu)
 	       kvm_request_pending(vcpu) || xfer_to_guest_mode_work_pending();
 }
 
-fastpath_t handle_fastpath_wrmsr(struct kvm_vcpu *vcpu)
+static fastpath_t __handle_fastpath_wrmsr(struct kvm_vcpu *vcpu, u32 msr, u64 data)
 {
-	u64 data = kvm_read_edx_eax(vcpu);
-	u32 msr = kvm_rcx_read(vcpu);
-
 	switch (msr) {
 	case APIC_BASE_MSR + (APIC_ICR >> 4):
 		if (!lapic_in_kernel(vcpu) || !apic_x2apic_mode(vcpu->arch.apic) ||
@@ -2202,8 +2199,20 @@ fastpath_t handle_fastpath_wrmsr(struct kvm_vcpu *vcpu)
 
 	return EXIT_FASTPATH_REENTER_GUEST;
 }
+
+fastpath_t handle_fastpath_wrmsr(struct kvm_vcpu *vcpu)
+{
+	return __handle_fastpath_wrmsr(vcpu, kvm_rcx_read(vcpu),
+				       kvm_read_edx_eax(vcpu));
+}
 EXPORT_SYMBOL_GPL(handle_fastpath_wrmsr);
 
+fastpath_t handle_fastpath_wrmsr_imm(struct kvm_vcpu *vcpu, u32 msr, int reg)
+{
+	return __handle_fastpath_wrmsr(vcpu, msr, kvm_register_read(vcpu, reg));
+}
+EXPORT_SYMBOL_GPL(handle_fastpath_wrmsr_imm);
+
 /*
  * Adapt set_msr() to msr_io()'s calling convention
  */
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index 2dab9c9d6199..eb3088684e8a 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -438,6 +438,7 @@ int x86_decode_emulated_instruction(struct kvm_vcpu *vcpu, int emulation_type,
 int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
 			    int emulation_type, void *insn, int insn_len);
 fastpath_t handle_fastpath_wrmsr(struct kvm_vcpu *vcpu);
+fastpath_t handle_fastpath_wrmsr_imm(struct kvm_vcpu *vcpu, u32 msr, int reg);
 fastpath_t handle_fastpath_hlt(struct kvm_vcpu *vcpu);
 fastpath_t handle_fastpath_invd(struct kvm_vcpu *vcpu);
 
-- 
2.50.1.565.gc32cd1483b-goog

[PATCH v3 6/6] KVM: x86: Advertise support for the immediate form of MSR instructions

[Sean Christopherson]

From: Xin Li <xin@zytor.com>

Advertise support for the immediate form of MSR instructions to userspace
if the instructions are supported by the underlying CPU, and KVM is using
VMX, i.e. is running on an Intel-compatible CPU.

For SVM, explicitly clear X86_FEATURE_MSR_IMM to ensure KVM doesn't over-
report support if AMD-compatible CPUs ever implement the immediate forms,
as SVM will likely require explicit enablement in KVM.

Signed-off-by: Xin Li (Intel) <xin@zytor.com>
[sean: massage changelog]
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/include/asm/kvm_host.h | 1 +
 arch/x86/kvm/cpuid.c            | 6 +++++-
 arch/x86/kvm/reverse_cpuid.h    | 5 +++++
 arch/x86/kvm/svm/svm.c          | 6 +++++-
 4 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index dbdec6025fde..735b5d1e62dd 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -774,6 +774,7 @@ enum kvm_only_cpuid_leafs {
 	CPUID_7_2_EDX,
 	CPUID_24_0_EBX,
 	CPUID_8000_0021_ECX,
+	CPUID_7_1_ECX,
 	NR_KVM_CPU_CAPS,
 
 	NKVMCAPINTS = NR_KVM_CPU_CAPS - NCAPINTS,
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index e2836a255b16..eaaa9203d4d9 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -985,6 +985,10 @@ void kvm_set_cpu_caps(void)
 		F(LAM),
 	);
 
+	kvm_cpu_cap_init(CPUID_7_1_ECX,
+		SCATTERED_F(MSR_IMM),
+	);
+
 	kvm_cpu_cap_init(CPUID_7_1_EDX,
 		F(AVX_VNNI_INT8),
 		F(AVX_NE_CONVERT),
@@ -1411,9 +1415,9 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
 				goto out;
 
 			cpuid_entry_override(entry, CPUID_7_1_EAX);
+			cpuid_entry_override(entry, CPUID_7_1_ECX);
 			cpuid_entry_override(entry, CPUID_7_1_EDX);
 			entry->ebx = 0;
-			entry->ecx = 0;
 		}
 		if (max_idx >= 2) {
 			entry = do_host_cpuid(array, function, 2);
diff --git a/arch/x86/kvm/reverse_cpuid.h b/arch/x86/kvm/reverse_cpuid.h
index c53b92379e6e..743ab25ba787 100644
--- a/arch/x86/kvm/reverse_cpuid.h
+++ b/arch/x86/kvm/reverse_cpuid.h
@@ -25,6 +25,9 @@
 #define KVM_X86_FEATURE_SGX2		KVM_X86_FEATURE(CPUID_12_EAX, 1)
 #define KVM_X86_FEATURE_SGX_EDECCSSA	KVM_X86_FEATURE(CPUID_12_EAX, 11)
 
+/* Intel-defined sub-features, CPUID level 0x00000007:1 (ECX) */
+#define KVM_X86_FEATURE_MSR_IMM		KVM_X86_FEATURE(CPUID_7_1_ECX, 5)
+
 /* Intel-defined sub-features, CPUID level 0x00000007:1 (EDX) */
 #define X86_FEATURE_AVX_VNNI_INT8       KVM_X86_FEATURE(CPUID_7_1_EDX, 4)
 #define X86_FEATURE_AVX_NE_CONVERT      KVM_X86_FEATURE(CPUID_7_1_EDX, 5)
@@ -87,6 +90,7 @@ static const struct cpuid_reg reverse_cpuid[] = {
 	[CPUID_7_2_EDX]       = {         7, 2, CPUID_EDX},
 	[CPUID_24_0_EBX]      = {      0x24, 0, CPUID_EBX},
 	[CPUID_8000_0021_ECX] = {0x80000021, 0, CPUID_ECX},
+	[CPUID_7_1_ECX]       = {         7, 1, CPUID_ECX},
 };
 
 /*
@@ -128,6 +132,7 @@ static __always_inline u32 __feature_translate(int x86_feature)
 	KVM_X86_TRANSLATE_FEATURE(BHI_CTRL);
 	KVM_X86_TRANSLATE_FEATURE(TSA_SQ_NO);
 	KVM_X86_TRANSLATE_FEATURE(TSA_L1_NO);
+	KVM_X86_TRANSLATE_FEATURE(MSR_IMM);
 	default:
 		return x86_feature;
 	}
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index ca550c4fa174..7e7821ee8ee1 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -5311,8 +5311,12 @@ static __init void svm_set_cpu_caps(void)
 	/* CPUID 0x8000001F (SME/SEV features) */
 	sev_set_cpu_caps();
 
-	/* Don't advertise Bus Lock Detect to guest if SVM support is absent */
+	/*
+	 * Clear capabilities that are automatically configured by common code,
+	 * but that require explicit SVM support (that isn't yet implemented).
+	 */
 	kvm_cpu_cap_clear(X86_FEATURE_BUS_LOCK_DETECT);
+	kvm_cpu_cap_clear(X86_FEATURE_MSR_IMM);
 }
 
 static __init int svm_hardware_setup(void)
-- 
2.50.1.565.gc32cd1483b-goog

Re: [PATCH v3 0/6] KVM: VMX: Handle the immediate form of MSR instructions

[Xin Li]

On 8/5/2025 1:22 PM, Sean Christopherson wrote:
> On behalf of Xin, to avoid having to resolve conflicts when applying.
> This applies on the fastpath cleanup series:
> https://lore.kernel.org/all/20250805190526.1453366-1-seanjc@google.com
>

Thanks for being considerate, and this is a much better use of time :)
> v3:
>   - Rebase on the fastpath cleanups.
>   - Split patches to better isolate the functional changes.
>   - Massage and expand on a changelogs.
>   - Make a handful of (mostly) stylistic changes (shouldn't affect
>     functionality, key word "should").

Tested after applying both patch sets, it works well.

Re: [PATCH v3 0/6] KVM: VMX: Handle the immediate form of MSR instructions

[Sean Christopherson]

On Tue, 05 Aug 2025 13:22:18 -0700, Sean Christopherson wrote:
> On behalf of Xin, to avoid having to resolve conflicts when applying.
> This applies on the fastpath cleanup series:
> https://lore.kernel.org/all/20250805190526.1453366-1-seanjc@google.com
> 
> This patch set handles two newly introduced VM exit reasons associated
> with the immediate form of MSR instructions to ensure proper
> virtualization of these instructions.
> 
> [...]

Applied to kvm-x86 misc, thanks!

[1/6] x86/cpufeatures: Add a CPU feature bit for MSR immediate form instructions
      https://github.com/kvm-x86/linux/commit/3c7cb8414533
[2/6] KVM: x86: Rename local "ecx" variables to "msr" and "pmc" as appropriate
      https://github.com/kvm-x86/linux/commit/ec400f6c2f27
[3/6] KVM: x86: Rename handle_fastpath_set_msr_irqoff() to handle_fastpath_wrmsr()
      https://github.com/kvm-x86/linux/commit/87a877de367d
[4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel
      https://github.com/kvm-x86/linux/commit/885df2d2109a
[5/6] KVM: VMX: Support the immediate form of WRMSRNS in the VM-Exit fastpath
      https://github.com/kvm-x86/linux/commit/ec93675a3251
[6/6] KVM: x86: Advertise support for the immediate form of MSR instructions
      https://github.com/kvm-x86/linux/commit/d90ebf5a06ec

--
https://github.com/kvm-x86/linux/tree/next