[PATCH v2 00/36] KVM: VMX APIC timer virtualization support

[PATCH v2 00/36] KVM: VMX APIC timer virtualization support

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

This patch series is v2 of VMX APIC timer virtualization support.

qemu patches doesn't need update.
https://lore.kernel.org/kvm/e41ffc4af96b8b4ee3a65f8da1f52dac7f3f4913.1770116952.git.isaku.yamahata@intel.com/

v2 patch for kvm-unit-tests update will be posted.
(v1 https://lore.kernel.org/kvm/7acdd9974effabe5dc461aa755eacf9fb0697467.1770116601.git.isaku.yamahata@intel.com/)

Changes v1 -> v2:
https://lore.kernel.org/kvm/cover.1770116050.git.isaku.yamahata@intel.com/
- Rebased to kvm-x86 and v7.0-rcX.
- Fixed an issue found by syzbot.  Require in-kernel APIC for nested
  VMX APIC timer virtualization.
- Add test case for nested VMX APIC timer virtualization with in-kernel
  APIC.
- Avoid compile time error by clearing the feature bit explicitly instead
  of clearing the bit of KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL.
- Fix compile errors. EXPORT_SYMBOL_FOR_KVM_INTERNAL.
- selftests: Use GUEST_ASSERT_EQ() where meaningful.
- selftests: Use safe_halt(), sti(), cli() instead of raw inline assembly.
- selftests: Require serialize instruction for tests that uses it.

Background
==========
X86 provides the TSC deadline timer as the primary local timer
interrupt source.  Currently, KVM intercepts the guest programming of
the timer and emulates it using either the host OS timer or the VMX
preemption timer.


Problem
=======
VMM emulation causes high latency.  Some workloads require lower
latency, such as gaming applications, while there have been efforts to
reduce latency in the past, a hardware extension can reduce it further
by eliminating VM Exits.


Solution
========
Hardware Extension
------------------
The APIC timer virtualization [1] allows the guest to directly access
the TSC DEADLINE MSR and receive timer interrupts without VM Exits.

It introduces
- A feature bit in the tertiary processor-based VM-execution controls
- Guest deadline: 64-bit physical deadline (host TSC value)
- Guest deadline shadow: 64-bit virtual deadline (virtualized TSC
  value with TSC offset and multiplier)
- Virtual timer vector: interrupt vector to inject on timeout.

Implementation
--------------
Add hooks to the LAPIC timer emulation and implement them in the VMX
backend.  Enable the feature when available, falling back to
software/preemption timer in the following cases
One-shot or periodic APIC timer:
  The hardware supports only the TSC deadline timer
Masked the timer interrupt in LVTT:
  The hardware doesn't respect the emulated LVTT and always generates an
  interrupt on timeout.
vCPU blocking/unblocking:
  The hardware generates an interrupt while the vCPU is running.  The KVM
  must wake up from vCPU blocking by getting the latest TSC
  deadline and setting a software timer before blocking the vCPU.
VM Entry to L2 vCPU:
  If the L1 timer interrupt fires while the L2 vCPU is running, the
  expected behavior is a VM Exit from L2 to L1, followed by an interrupt
  injection into the L1 vCPU.

nVMX Support
------------
Support nVMX to address the benchmark result below.  Emulate related
MSRs and VMCS individually.
MSRs: capability reporting registers of primary/tertiary processor-based
      VM-execution controls.
VMCS fields: primary/tertiary VM-execution controls, guest deadline,
             guest deadline shadow, and virtual timer vector.

Patch Organization
------------------
The patch is organized into 5 parts as follows.

Patches  1- 8: VMX support (feature probe, hooks to KVM LAPIC, VMX hooks)
Patches  9-18: nVMX support (implement emulation of MSR and VMCS fields)
Patches 19-23: Expose the feature to the user
Patches 24-31: KVM selftests
Patches 32   : Documentation update

https://lore.kernel.org/kvm/e41ffc4af96b8b4ee3a65f8da1f52dac7f3f4913.1770116952.git.isaku.yamahata@intel.com/
(KVM unit tests turned out test case issue. It needs fixes.)
https://lore.kernel.org/kvm/7acdd9974effabe5dc461aa755eacf9fb0697467.1770116601.git.isaku.yamahata@intel.com/


Test
====
The following tests were conducted:  The newly added test case as a
part of KVM selftests, KVM unit tests, and cyclic test included in
rt-tests [2].  Selftests and KVM unit tests were run on platforms with
and without APIC timer virtualization.


Benchmark Results
=================
cyclictest
----------
10-minute run of
cyclictest --quiet --nsecs --smp --mlockall --priority=95 --policy=fifo
# of vCPU: host 256, L1 and L2: 16

Legends:
L1 or L2: cyclic test run as L1/L2 process
Y: feature enabled
N: feature disabled

Run in
|       APIC timer virtualization
|       |       nested APIC timer virtualization
|       |       |       min reduction %
|       |       |       |       avg reduction %
|       |       |       |       |
L1	N	-
L1	Y	-	21%	21% (compared to L1 N)

L2	N	N
L2	Y	N	4%	-2% (compared to L2 N N)
L2	Y	Y	75%	51% (compared to L2 N N)

Micro benchmark: Timer latency
------------------------------
10-minute run of custom micro benchmark, timer_latency.
# of vCPU: host 256, L1 and L2: 16

Legends:
L1: the benchmark run in L0 Linux.
L2: the benchmark run in L1 Linux.
Y: feature enabled
N: feature disabled

Run as
|       APIC timer virtualization
|       |       nested APIC timer virtualization
|       |       |       HLT or busy
|       |       |       |       min reduction %
|       |       |       |       |       avg reduction %
|       |       |       |       |       |
L1	N	-	HLT
L1	Y	-	HLT	49%	24% (compared to L1 N HLT)

L1	N	-	busy
L1	Y	-	busy	63%	61% (compared to L1 N busy)

L2	N	N	HLT
L2	Y	N	HLT	-19%	-3% (compared to L2 N N HLT)
L2	Y	Y	HLT	99%	27% (compared to L2 N N HLT)

L2	N	N	busy
L2	Y	N	busy	-5%	-4% (compared to L2 N N busy)
L2	Y	Y	busy	99%	97% (compared to L2 N N busy)


[1] Intel Architecture Instruction Set Extensions and Future Features
September 2025 319433-059
Chapter 8 APIC-TIMER VIRTUALIZATION
https://cdrdv2.intel.com/v1/dl/getContent/671368

[2] rt-tests
https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git/


Isaku Yamahata (29):
  KVM: x86/lapic: Wire DEADLINE MSR update to guest virtual TSC deadline
  KVM: VMX: Update APIC timer virtualization on apicv changed
  KVM: nVMX: Disallow/allow guest APIC timer virtualization switch
    to/from L2
  KVM: nVMX: Pass struct msr_data to VMX MSRs emulation
  KVM: nVMX: Supports VMX tertiary controls and GUEST_APIC_TIMER bit
  KVM: nVMX: Add alignment check for struct vmcs12
  KVM: nVMX: Add tertiary VM-execution control VMCS support
  KVM: nVMX: Update intercept on TSC deadline MSR
  KVM: nVMX: Handle virtual timer vector VMCS field
  KVM: VMX: Make vmx_calc_deadline_l1_to_host() non-static
  KVM: nVMX: Enable guest deadline and its shadow VMCS field
  KVM: nVMX: Add VM entry checks related to APIC timer virtualization
  KVM: nVMX: Add check vmread/vmwrite on tertiary control
  KVM: nVMX: Add check VMCS index for guest timer virtualization
  KVM: VMX: Advertise tertiary controls to the user space
  KVM: VMX: Enable APIC timer virtualization
  KVM: nVMX: Introduce module parameter for nested APIC timer
    virtualization
  KVM: selftests: Add a test to measure local timer latency
  KVM: selftests: Add nVMX support to timer_latency test case
  KVM: selftests: Add test for nVMX MSR_IA32_VMX_PROCBASED_CTLS3
  KVM: selftests: Add test vmx_set_nested_state_test with EVMCS disabled
  KVM: selftests: Add tests nested state of APIC timer virtualization
  KVM: selftests: Add VMCS access test to APIC timer virtualization
  KVM: selftests: Add serialize() helper and X86_FEATURE_SERIALIZE
  KVM: selftests: Test cases for L1 APIC timer virtualization
  KVM: selftests: Add tests for nVMX to vmx_apic_timer_virt
  KVM: selftests: Add a global option to disable in-kernel irqchip
  KVM: selftests: Test VMX apic timer virt with inkernel apic disable
  Documentation: KVM: x86: Update documentation of struct vmcs12

Yang Zhong (7):
  KVM: VMX: Detect APIC timer virtualization bit
  KVM: x86: Implement APIC virt timer helpers with callbacks
  KVM: x86/lapic: Start/stop sw/hv timer on vCPU un/block
  KVM: x86/lapic: Add a trace point for guest virtual timer
  KVM: VMX: Implement the hooks for VMX guest virtual deadline timer
  KVM: VMX: dump_vmcs() support the guest virt timer
  KVM: VMX: Introduce module parameter for APIC virt timer support

 Documentation/virt/kvm/x86/nested-vmx.rst     |  14 +-
 arch/x86/include/asm/kvm-x86-ops.h            |   5 +
 arch/x86/include/asm/kvm_host.h               |   6 +
 arch/x86/include/asm/vmx.h                    |   6 +
 arch/x86/include/asm/vmxfeatures.h            |   1 +
 arch/x86/kvm/lapic.c                          | 157 +++-
 arch/x86/kvm/lapic.h                          |  18 +
 arch/x86/kvm/trace.h                          |  16 +
 arch/x86/kvm/vmx/capabilities.h               |   8 +
 arch/x86/kvm/vmx/hyperv.c                     |  17 +
 arch/x86/kvm/vmx/main.c                       |   5 +
 arch/x86/kvm/vmx/nested.c                     | 212 +++++-
 arch/x86/kvm/vmx/nested.h                     |  39 +-
 arch/x86/kvm/vmx/vmcs12.c                     |   8 +
 arch/x86/kvm/vmx/vmcs12.h                     |  13 +-
 arch/x86/kvm/vmx/vmcs_shadow_fields.h         |   1 +
 arch/x86/kvm/vmx/vmx.c                        | 148 +++-
 arch/x86/kvm/vmx/vmx.h                        |   7 +-
 arch/x86/kvm/vmx/x86_ops.h                    |   5 +
 arch/x86/kvm/x86.c                            |   8 +-
 arch/x86/kvm/x86.h                            |   2 +-
 tools/testing/selftests/kvm/Makefile.kvm      |   4 +
 .../testing/selftests/kvm/include/x86/apic.h  |   1 +
 .../selftests/kvm/include/x86/kvm_util_arch.h |   1 +
 .../selftests/kvm/include/x86/processor.h     |   7 +
 tools/testing/selftests/kvm/include/x86/vmx.h |  14 +
 .../testing/selftests/kvm/lib/x86/processor.c |   5 +-
 .../selftests/kvm/x86/nested_set_state_test.c | 250 +++++++
 .../testing/selftests/kvm/x86/timer_latency.c | 704 ++++++++++++++++++
 .../kvm/x86/vmx_apic_timer_virt_test.c        | 510 +++++++++++++
 .../kvm/x86/vmx_apic_timer_virt_vmcs_test.c   | 464 ++++++++++++
 .../testing/selftests/kvm/x86/vmx_msrs_test.c |  53 ++
 .../kvm/x86/vmx_tertiary_ctls_test.c          | 168 +++++
 33 files changed, 2853 insertions(+), 24 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/timer_latency.c
 create mode 100644 tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_vmcs_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/vmx_tertiary_ctls_test.c


base-commit: 5128b972fb2801ad9aca54d990a75611ab5283a9
-- 
2.45.2

[PATCH v2 01/36] KVM: VMX: Detect APIC timer virtualization bit

[isaku.yamahata]

From: Yang Zhong <yang.zhong@linux.intel.com>

Detect the APIC timer virtualization feature by setting the bit (bit 8) in
the tertiary processor-based VM-execution controls.  Additionally, define
the new related VMCS fields necessary for managing this feature.

Do not enable the feature bit in the tertiary VM exec control yet until the
supporting logic is implemented.

Signed-off-by: Yang Zhong <yang.zhong@linux.intel.com>
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
Changes:
v1 -> v2:
- Change KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL to include apic timer
  virt bit to avoid compile time errors.  Clear the bit when
  adjust_vmx_controls64() is called instead.
---
 arch/x86/include/asm/vmx.h         |  6 ++++++
 arch/x86/include/asm/vmxfeatures.h |  1 +
 arch/x86/kvm/vmx/vmx.c             | 20 +++++++++++++++++++-
 arch/x86/kvm/vmx/vmx.h             |  3 ++-
 4 files changed, 28 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h
index 37080382df54..a9fe10cf1b4d 100644
--- a/arch/x86/include/asm/vmx.h
+++ b/arch/x86/include/asm/vmx.h
@@ -96,6 +96,7 @@ struct vmcs {
  * Definitions of Tertiary Processor-Based VM-Execution Controls.
  */
 #define TERTIARY_EXEC_IPI_VIRT			VMCS_CONTROL_BIT(IPI_VIRT)
+#define TERTIARY_EXEC_GUEST_APIC_TIMER		VMCS_CONTROL_BIT(GUEST_APIC_TIMER)
 
 #define PIN_BASED_EXT_INTR_MASK                 VMCS_CONTROL_BIT(INTR_EXITING)
 #define PIN_BASED_NMI_EXITING                   VMCS_CONTROL_BIT(NMI_EXITING)
@@ -204,6 +205,7 @@ enum vmcs_field {
 	VIRTUAL_PROCESSOR_ID            = 0x00000000,
 	POSTED_INTR_NV                  = 0x00000002,
 	LAST_PID_POINTER_INDEX		= 0x00000008,
+	GUEST_APIC_TIMER_VECTOR         = 0x0000000a,
 	GUEST_ES_SELECTOR               = 0x00000800,
 	GUEST_CS_SELECTOR               = 0x00000802,
 	GUEST_SS_SELECTOR               = 0x00000804,
@@ -274,6 +276,8 @@ enum vmcs_field {
 	SHARED_EPT_POINTER		= 0x0000203C,
 	PID_POINTER_TABLE		= 0x00002042,
 	PID_POINTER_TABLE_HIGH		= 0x00002043,
+	GUEST_DEADLINE_VIR              = 0x0000204e,
+	GUEST_DEADLINE_VIR_HIGH         = 0x0000204f,
 	GUEST_PHYSICAL_ADDRESS          = 0x00002400,
 	GUEST_PHYSICAL_ADDRESS_HIGH     = 0x00002401,
 	VMCS_LINK_POINTER               = 0x00002800,
@@ -298,6 +302,8 @@ enum vmcs_field {
 	GUEST_BNDCFGS_HIGH              = 0x00002813,
 	GUEST_IA32_RTIT_CTL		= 0x00002814,
 	GUEST_IA32_RTIT_CTL_HIGH	= 0x00002815,
+	GUEST_DEADLINE_PHY              = 0x00002830,
+	GUEST_DEADLINE_PHY_HIGH         = 0x00002831,
 	HOST_IA32_PAT			= 0x00002c00,
 	HOST_IA32_PAT_HIGH		= 0x00002c01,
 	HOST_IA32_EFER			= 0x00002c02,
diff --git a/arch/x86/include/asm/vmxfeatures.h b/arch/x86/include/asm/vmxfeatures.h
index 09b1d7e607c1..f2eb4243bae4 100644
--- a/arch/x86/include/asm/vmxfeatures.h
+++ b/arch/x86/include/asm/vmxfeatures.h
@@ -90,4 +90,5 @@
 
 /* Tertiary Processor-Based VM-Execution Controls, word 3 */
 #define VMX_FEATURE_IPI_VIRT		( 3*32+  4) /* "ipi_virt" Enable IPI virtualization */
+#define VMX_FEATURE_GUEST_APIC_TIMER	( 3*32+  8) /* Enable virtual APIC tsc deadline */
 #endif /* _ASM_X86_VMXFEATURES_H */
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 9302c16571cd..4ccb2e42322d 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -2791,9 +2791,20 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 
 	if (_cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS)
 		_cpu_based_3rd_exec_control =
-			adjust_vmx_controls64(KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL,
+			adjust_vmx_controls64(KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL
+					      /*
+					       * Disable apic timer
+					       * virtualization until the logic
+					       * is imlemented.
+					       * Once it's supported, add
+					       * TERTIARY_EXEC_GUEST_APIC_TIMER.
+					       */
+					      & ~TERTIARY_EXEC_GUEST_APIC_TIMER,
 					      MSR_IA32_VMX_PROCBASED_CTLS3);
 
+	if (!(_cpu_based_2nd_exec_control & SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY))
+		_cpu_based_3rd_exec_control &= ~TERTIARY_EXEC_GUEST_APIC_TIMER;
+
 	if (adjust_vmx_controls(KVM_REQUIRED_VMX_VM_EXIT_CONTROLS,
 				KVM_OPTIONAL_VMX_VM_EXIT_CONTROLS,
 				MSR_IA32_VMX_EXIT_CTLS,
@@ -4636,6 +4647,13 @@ static u64 vmx_tertiary_exec_control(struct vcpu_vmx *vmx)
 	if (!enable_ipiv || !kvm_vcpu_apicv_active(&vmx->vcpu))
 		exec_control &= ~TERTIARY_EXEC_IPI_VIRT;
 
+	/*
+	 * APIC timer virtualization is supported only for TSC deadline mode.
+	 * Disable for one-shot/periodic mode.  Dynamically set/clear the bit
+	 * on the guest timer mode change.  Disable on reset state.
+	 */
+	exec_control &= ~TERTIARY_EXEC_GUEST_APIC_TIMER;
+
 	return exec_control;
 }
 
diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h
index 70bfe81dea54..9a61f6bd8cc0 100644
--- a/arch/x86/kvm/vmx/vmx.h
+++ b/arch/x86/kvm/vmx/vmx.h
@@ -575,7 +575,8 @@ static inline u8 vmx_get_rvi(void)
 
 #define KVM_REQUIRED_VMX_TERTIARY_VM_EXEC_CONTROL 0
 #define KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL			\
-	(TERTIARY_EXEC_IPI_VIRT)
+	(TERTIARY_EXEC_IPI_VIRT |					\
+	 TERTIARY_EXEC_GUEST_APIC_TIMER)
 
 #define BUILD_CONTROLS_SHADOW(lname, uname, bits)						\
 static inline void lname##_controls_set(struct vcpu_vmx *vmx, u##bits val)			\
-- 
2.45.2

[PATCH v2 02/36] KVM: x86: Implement APIC virt timer helpers with callbacks

[isaku.yamahata]

From: Yang Zhong <yang.zhong@linux.intel.com>

Add an additional APIC emulation mode for APIC timer virtualization.  When
the guest programs the local APIC timer mode, switch to this new emulation
mode using the newly added hooks when APIC timer virtualization is
available.

Add five x86 KVM callbacks for APIC timer virtualization.  These callbacks
are analogous to those used for the preemption timer and will be invoked by
kvm/lapic.  These helpers start/stop the timer once the APIC virt timer
feature is enabled and the guest sets the MSR_IA32_TSC_DEADLINE.  Upon
updating the TSC deadline mode in the APIC_LVTT register, KVM's LAPIC will
initiate the APIC virt timer instead of the preemption timer.

Signed-off-by: Yang Zhong <yang.zhong@linux.intel.com>
Co-developed-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/include/asm/kvm-x86-ops.h |  5 ++
 arch/x86/include/asm/kvm_host.h    |  6 +++
 arch/x86/kvm/lapic.c               | 82 +++++++++++++++++++++++++++++-
 arch/x86/kvm/lapic.h               |  1 +
 4 files changed, 92 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h
index de709fb5bd76..09f664aa72c1 100644
--- a/arch/x86/include/asm/kvm-x86-ops.h
+++ b/arch/x86/include/asm/kvm-x86-ops.h
@@ -118,6 +118,11 @@ KVM_X86_OP_OPTIONAL_RET0(dy_apicv_has_pending_interrupt)
 KVM_X86_OP_OPTIONAL(protected_apic_has_interrupt)
 KVM_X86_OP_OPTIONAL(set_hv_timer)
 KVM_X86_OP_OPTIONAL(cancel_hv_timer)
+KVM_X86_OP_OPTIONAL(can_use_apic_virt_timer)
+KVM_X86_OP_OPTIONAL(set_apic_virt_timer)
+KVM_X86_OP_OPTIONAL(cancel_apic_virt_timer)
+KVM_X86_OP_OPTIONAL(set_guest_tsc_deadline_virt)
+KVM_X86_OP_OPTIONAL(get_guest_tsc_deadline_virt)
 KVM_X86_OP(setup_mce)
 #ifdef CONFIG_KVM_SMM
 KVM_X86_OP(smi_allowed)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index c94556fefb75..345ea21b45f8 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1934,6 +1934,12 @@ struct kvm_x86_ops {
 	int (*set_hv_timer)(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc,
 			    bool *expired);
 	void (*cancel_hv_timer)(struct kvm_vcpu *vcpu);
+	bool (*can_use_apic_virt_timer)(struct kvm_vcpu *vcpu);
+	void (*set_apic_virt_timer)(struct kvm_vcpu *vcpu, u16 vector);
+	void (*cancel_apic_virt_timer)(struct kvm_vcpu *vcpu);
+	void (*set_guest_tsc_deadline_virt)(struct kvm_vcpu *vcpu,
+					    u64 tscdeadline);
+	u64 (*get_guest_tsc_deadline_virt)(struct kvm_vcpu *vcpu);
 
 	void (*setup_mce)(struct kvm_vcpu *vcpu);
 
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 9381c58d4c85..0d91834d972c 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -188,7 +188,8 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)
 static bool kvm_can_post_timer_interrupt(struct kvm_vcpu *vcpu)
 {
 	return pi_inject_timer && kvm_vcpu_apicv_active(vcpu) &&
-		(kvm_mwait_in_guest(vcpu->kvm) || kvm_hlt_in_guest(vcpu->kvm));
+		(kvm_mwait_in_guest(vcpu->kvm) || kvm_hlt_in_guest(vcpu->kvm)) &&
+		!vcpu->arch.apic->lapic_timer.apic_virt_timer_in_use;
 }
 
 static bool kvm_can_use_hv_timer(struct kvm_vcpu *vcpu)
@@ -1864,17 +1865,82 @@ static void limit_periodic_timer_frequency(struct kvm_lapic *apic)
 }
 
 static void cancel_hv_timer(struct kvm_lapic *apic);
+static void cancel_apic_virt_timer(struct kvm_lapic *apic);
 
-static void cancel_apic_timer(struct kvm_lapic *apic)
+static void __cancel_apic_timer(struct kvm_lapic *apic)
 {
 	hrtimer_cancel(&apic->lapic_timer.timer);
 	preempt_disable();
 	if (apic->lapic_timer.hv_timer_in_use)
 		cancel_hv_timer(apic);
+	else if (apic->lapic_timer.apic_virt_timer_in_use)
+		cancel_apic_virt_timer(apic);
 	preempt_enable();
+}
+
+static void cancel_apic_timer(struct kvm_lapic *apic)
+{
+	__cancel_apic_timer(apic);
 	atomic_set(&apic->lapic_timer.pending, 0);
 }
 
+static void start_apic_timer(struct kvm_lapic *apic);
+
+static void cancel_apic_virt_timer(struct kvm_lapic *apic)
+{
+	struct kvm_vcpu *vcpu = apic->vcpu;
+
+	apic->lapic_timer.tscdeadline = kvm_x86_call(get_guest_tsc_deadline_virt)(vcpu);
+	kvm_x86_call(set_guest_tsc_deadline_virt)(vcpu, 0);
+
+	kvm_x86_call(cancel_apic_virt_timer)(vcpu);
+	apic->lapic_timer.apic_virt_timer_in_use = false;
+}
+
+static void apic_cancel_apic_virt_timer(struct kvm_lapic *apic)
+{
+	if (!apic->lapic_timer.apic_virt_timer_in_use)
+		return;
+
+	cancel_apic_virt_timer(apic);
+	start_apic_timer(apic);
+}
+
+static void apic_set_apic_virt_timer(struct kvm_lapic *apic)
+{
+	struct kvm_timer *ktimer = &apic->lapic_timer;
+	struct kvm_vcpu *vcpu = apic->vcpu;
+	u8 vector;
+	u32 reg;
+
+	if (apic->lapic_timer.apic_virt_timer_in_use)
+		return;
+
+	reg = kvm_lapic_get_reg(apic, APIC_LVTT);
+	vector = reg & APIC_VECTOR_MASK;
+
+	__cancel_apic_timer(apic);
+	kvm_x86_call(set_apic_virt_timer)(vcpu, vector);
+	kvm_x86_call(set_guest_tsc_deadline_virt)(vcpu, ktimer->tscdeadline);
+	ktimer->apic_virt_timer_in_use = true;
+}
+
+static bool kvm_can_use_apic_virt_timer(struct kvm_vcpu *vcpu)
+{
+	return kvm_x86_ops.can_use_apic_virt_timer &&
+		lapic_in_kernel(vcpu) &&
+		apic_lvt_enabled(vcpu->arch.apic, APIC_LVTT) &&
+		kvm_x86_call(can_use_apic_virt_timer)(vcpu);
+}
+
+static void apic_update_apic_virt_timer(struct kvm_lapic *apic)
+{
+	if (kvm_can_use_apic_virt_timer(apic->vcpu))
+		apic_set_apic_virt_timer(apic);
+	else
+		apic_cancel_apic_virt_timer(apic);
+}
+
 static void apic_update_lvtt(struct kvm_lapic *apic)
 {
 	u32 timer_mode = kvm_lapic_get_reg(apic, APIC_LVTT) &
@@ -1887,10 +1953,19 @@ static void apic_update_lvtt(struct kvm_lapic *apic)
 			kvm_lapic_set_reg(apic, APIC_TMICT, 0);
 			apic->lapic_timer.period = 0;
 			apic->lapic_timer.tscdeadline = 0;
+
+			if (apic->lapic_timer.apic_virt_timer_in_use)
+				kvm_x86_call(set_guest_tsc_deadline_virt)(apic->vcpu, 0);
 		}
 		apic->lapic_timer.timer_mode = timer_mode;
 		limit_periodic_timer_frequency(apic);
 	}
+
+	/*
+	 * Update on not only timer mode change, but also mask change
+	 * for the case of timer_mode = TSCDEADLINE, mask = 1.
+	 */
+	apic_update_apic_virt_timer(apic);
 }
 
 /*
@@ -2312,6 +2387,9 @@ static void restart_apic_timer(struct kvm_lapic *apic)
 	if (!apic_lvtt_period(apic) && atomic_read(&apic->lapic_timer.pending))
 		goto out;
 
+	if (apic->lapic_timer.apic_virt_timer_in_use)
+		goto out;
+
 	if (!start_hv_timer(apic))
 		start_sw_timer(apic);
 out:
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index 274885af4ebc..ba1090f9a3c8 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -57,6 +57,7 @@ struct kvm_timer {
 	u32 timer_advance_ns;
 	atomic_t pending;			/* accumulated triggered timers */
 	bool hv_timer_in_use;
+	bool apic_virt_timer_in_use;
 };
 
 struct kvm_lapic {
-- 
2.45.2

[PATCH v2 03/36] KVM: x86/lapic: Start/stop sw/hv timer on vCPU un/block

[isaku.yamahata]

From: Yang Zhong <yang.zhong@linux.intel.com>

Switch to use software timer when a vCPU is blocked by instructions such as
HLT similar to the hv timer case.

The guest deadline shadow field is read to obtain the guest_tsc value,
which is then used to program an hrtimer for the duration of the vCPU
block. Upon completion of the block, if the LAPIC timer has pending
interrupts, the LAPIC timer is transitioned to APIC virt timer mode to
continue providing timer services to the guest.
Set the guest TSC deadline to 0 when injecting a timer interrupt, as the
TSC deadline is cleared to zero when a timer interrupt is injected.  Do so
when injecting a timer interrupt to the vCPU.

Signed-off-by: Yang Zhong <yang.zhong@linux.intel.com>
Co-developed-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
Changes:
v1 -> v2:
- Add inkernel apic check as apic timer virtualizaion requires it.
---
 arch/x86/kvm/lapic.c | 20 +++++++++++++++++---
 arch/x86/kvm/lapic.h |  9 +++++++++
 arch/x86/kvm/x86.c   |  7 +++++--
 3 files changed, 31 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 0d91834d972c..ab59722e291e 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -2089,6 +2089,8 @@ static void kvm_apic_inject_pending_timer_irqs(struct kvm_lapic *apic)
 	kvm_apic_local_deliver(apic, APIC_LVTT);
 	if (apic_lvtt_tscdeadline(apic)) {
 		ktimer->tscdeadline = 0;
+		if (apic->lapic_timer.apic_virt_timer_in_use)
+			kvm_x86_call(set_guest_tsc_deadline_virt)(apic->vcpu, 0);
 	} else if (apic_lvtt_oneshot(apic)) {
 		ktimer->tscdeadline = 0;
 		ktimer->target_expiration = 0;
@@ -2368,8 +2370,10 @@ static void start_sw_timer(struct kvm_lapic *apic)
 	struct kvm_timer *ktimer = &apic->lapic_timer;
 
 	WARN_ON(preemptible());
-	if (apic->lapic_timer.hv_timer_in_use)
+	if (apic->lapic_timer.hv_timer_in_use) {
 		cancel_hv_timer(apic);
+		trace_kvm_hv_timer_state(apic->vcpu->vcpu_id, false);
+	}
 	if (!apic_lvtt_period(apic) && atomic_read(&ktimer->pending))
 		return;
 
@@ -2377,7 +2381,6 @@ static void start_sw_timer(struct kvm_lapic *apic)
 		start_sw_period(apic);
 	else if (apic_lvtt_tscdeadline(apic))
 		start_sw_tscdeadline(apic);
-	trace_kvm_hv_timer_state(apic->vcpu->vcpu_id, false);
 }
 
 static void restart_apic_timer(struct kvm_lapic *apic)
@@ -2422,13 +2425,24 @@ void kvm_lapic_switch_to_hv_timer(struct kvm_vcpu *vcpu)
 	restart_apic_timer(vcpu->arch.apic);
 }
 
+void kvm_lapic_switch_to_apic_virt_timer(struct kvm_vcpu *vcpu)
+{
+	hrtimer_cancel(&vcpu->arch.apic->lapic_timer.timer);
+}
+
 void kvm_lapic_switch_to_sw_timer(struct kvm_vcpu *vcpu)
 {
 	struct kvm_lapic *apic = vcpu->arch.apic;
 
 	preempt_disable();
+
+	if (apic->lapic_timer.apic_virt_timer_in_use)
+		apic->lapic_timer.tscdeadline =
+			kvm_x86_call(get_guest_tsc_deadline_virt)(vcpu);
+
 	/* Possibly the TSC deadline timer is not enabled yet */
-	if (apic->lapic_timer.hv_timer_in_use)
+	if (apic->lapic_timer.hv_timer_in_use ||
+	    apic->lapic_timer.apic_virt_timer_in_use)
 		start_sw_timer(apic);
 	preempt_enable();
 }
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index ba1090f9a3c8..5e96299c31f7 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -250,10 +250,19 @@ bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct kvm_lapic_irq *irq,
 			     struct kvm_vcpu **dest_vcpu);
 void kvm_lapic_switch_to_sw_timer(struct kvm_vcpu *vcpu);
 void kvm_lapic_switch_to_hv_timer(struct kvm_vcpu *vcpu);
+void kvm_lapic_switch_to_apic_virt_timer(struct kvm_vcpu *vcpu);
 void kvm_lapic_expired_hv_timer(struct kvm_vcpu *vcpu);
 bool kvm_lapic_hv_timer_in_use(struct kvm_vcpu *vcpu);
 void kvm_lapic_restart_hv_timer(struct kvm_vcpu *vcpu);
 
+static inline bool kvm_lapic_apic_virt_timer_in_use(struct kvm_vcpu *vcpu)
+{
+	if (!lapic_in_kernel(vcpu))
+		return false;
+
+	return vcpu->arch.apic->lapic_timer.apic_virt_timer_in_use;
+}
+
 static inline enum lapic_mode kvm_apic_mode(u64 apic_base)
 {
 	return apic_base & (MSR_IA32_APICBASE_ENABLE | X2APIC_ENABLE);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 879cdeb6adde..1922e8699101 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -11661,7 +11661,7 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu)
 /* Called within kvm->srcu read side.  */
 static inline int vcpu_block(struct kvm_vcpu *vcpu)
 {
-	bool hv_timer;
+	bool hv_timer, virt_timer;
 
 	if (!kvm_arch_vcpu_runnable(vcpu)) {
 		/*
@@ -11672,7 +11672,8 @@ static inline int vcpu_block(struct kvm_vcpu *vcpu)
 		 * timer before blocking.
 		 */
 		hv_timer = kvm_lapic_hv_timer_in_use(vcpu);
-		if (hv_timer)
+		virt_timer = kvm_lapic_apic_virt_timer_in_use(vcpu);
+		if (hv_timer || virt_timer)
 			kvm_lapic_switch_to_sw_timer(vcpu);
 
 		kvm_vcpu_srcu_read_unlock(vcpu);
@@ -11684,6 +11685,8 @@ static inline int vcpu_block(struct kvm_vcpu *vcpu)
 
 		if (hv_timer)
 			kvm_lapic_switch_to_hv_timer(vcpu);
+		else if (virt_timer)
+			kvm_lapic_switch_to_apic_virt_timer(vcpu);
 
 		/*
 		 * If the vCPU is not runnable, a signal or another host event
-- 
2.45.2

[PATCH v2 04/36] KVM: x86/lapic: Wire DEADLINE MSR update to guest virtual TSC deadline

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Wire userpace read/write (KVM_GET_MSRS, KVM_SET_MSRS) of TSCDEADLINE MSR to
the vendor backend to update the VMCS field of GUEST TSCDEADLINE and
GUEST TSCDEADLINE shadow.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/lapic.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index ab59722e291e..10aa7040686f 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -2802,6 +2802,10 @@ u64 kvm_get_lapic_tscdeadline_msr(struct kvm_vcpu *vcpu)
 	if (!kvm_apic_present(vcpu) || !apic_lvtt_tscdeadline(apic))
 		return 0;
 
+	if (apic->lapic_timer.apic_virt_timer_in_use)
+		apic->lapic_timer.tscdeadline =
+			kvm_x86_call(get_guest_tsc_deadline_virt)(vcpu);
+
 	return apic->lapic_timer.tscdeadline;
 }
 
@@ -2814,6 +2818,8 @@ void kvm_set_lapic_tscdeadline_msr(struct kvm_vcpu *vcpu, u64 data)
 
 	hrtimer_cancel(&apic->lapic_timer.timer);
 	apic->lapic_timer.tscdeadline = data;
+	if (apic->lapic_timer.apic_virt_timer_in_use)
+		kvm_x86_call(set_guest_tsc_deadline_virt)(vcpu, data);
 	start_apic_timer(apic);
 }
 
-- 
2.45.2

[PATCH v2 05/36] KVM: x86/lapic: Add a trace point for guest virtual timer

[isaku.yamahata]

From: Yang Zhong <yang.zhong@linux.intel.com>

Add a trace point for changing the guest virtual timer, similar to the hv
timer case.

Signed-off-by: Yang Zhong <yang.zhong@linux.intel.com>
Co-developed-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/lapic.c |  5 +++++
 arch/x86/kvm/trace.h | 16 ++++++++++++++++
 2 files changed, 21 insertions(+)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 10aa7040686f..abbd51c4da7f 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -1895,6 +1895,9 @@ static void cancel_apic_virt_timer(struct kvm_lapic *apic)
 
 	kvm_x86_call(cancel_apic_virt_timer)(vcpu);
 	apic->lapic_timer.apic_virt_timer_in_use = false;
+
+	trace_kvm_apic_virt_timer_state(vcpu->vcpu_id,
+					apic->lapic_timer.apic_virt_timer_in_use);
 }
 
 static void apic_cancel_apic_virt_timer(struct kvm_lapic *apic)
@@ -1923,6 +1926,8 @@ static void apic_set_apic_virt_timer(struct kvm_lapic *apic)
 	kvm_x86_call(set_apic_virt_timer)(vcpu, vector);
 	kvm_x86_call(set_guest_tsc_deadline_virt)(vcpu, ktimer->tscdeadline);
 	ktimer->apic_virt_timer_in_use = true;
+
+	trace_kvm_apic_virt_timer_state(vcpu->vcpu_id, ktimer->apic_virt_timer_in_use);
 }
 
 static bool kvm_can_use_apic_virt_timer(struct kvm_vcpu *vcpu)
diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h
index e7fdbe9efc90..ca9456955254 100644
--- a/arch/x86/kvm/trace.h
+++ b/arch/x86/kvm/trace.h
@@ -1649,6 +1649,22 @@ TRACE_EVENT(kvm_hv_timer_state,
 			__entry->hv_timer_in_use)
 );
 
+TRACE_EVENT(kvm_apic_virt_timer_state,
+		TP_PROTO(unsigned int vcpu_id, unsigned int apic_virt_timer_in_use),
+		TP_ARGS(vcpu_id, apic_virt_timer_in_use),
+		TP_STRUCT__entry(
+			__field(unsigned int, vcpu_id)
+			__field(unsigned int, apic_virt_timer_in_use)
+			),
+		TP_fast_assign(
+			__entry->vcpu_id = vcpu_id;
+			__entry->apic_virt_timer_in_use = apic_virt_timer_in_use;
+			),
+		TP_printk("vcpu_id %x apic_virt_timer %x",
+			__entry->vcpu_id,
+			__entry->apic_virt_timer_in_use)
+);
+
 /*
  * Tracepoint for kvm_hv_flush_tlb.
  */
-- 
2.45.2

[PATCH v2 06/36] KVM: VMX: Implement the hooks for VMX guest virtual deadline timer

[isaku.yamahata]

From: Yang Zhong <yang.zhong@linux.intel.com>

Implement the hooks for the VMX backend for APIC timer virtualization to
access the related VMCS fields.

Signed-off-by: Yang Zhong <yang.zhong@linux.intel.com>
Co-developed-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
Changes:
v1 -> v2:
Move kvm_cpu_apicv_active() before kvm_lapic_lvtt_timer_mode() as
it checks in-kernel apic check.
---
 arch/x86/kvm/lapic.h            |  5 ++
 arch/x86/kvm/vmx/capabilities.h |  6 +++
 arch/x86/kvm/vmx/main.c         |  5 ++
 arch/x86/kvm/vmx/vmx.c          | 83 ++++++++++++++++++++++++++++++++-
 arch/x86/kvm/vmx/x86_ops.h      |  5 ++
 5 files changed, 103 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index 5e96299c31f7..2f510503f5b3 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -237,6 +237,11 @@ static inline int kvm_lapic_latched_init(struct kvm_vcpu *vcpu)
 	return lapic_in_kernel(vcpu) && test_bit(KVM_APIC_INIT, &vcpu->arch.apic->pending_events);
 }
 
+static inline int kvm_lapic_lvtt_timer_mode(struct kvm_vcpu *vcpu)
+{
+	return vcpu->arch.apic->lapic_timer.timer_mode;
+}
+
 bool kvm_apic_pending_eoi(struct kvm_vcpu *vcpu, int vector);
 
 bool kvm_lapic_suppress_eoi_broadcast(struct kvm_lapic *apic);
diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilities.h
index 4e371c93ae16..c5cb098f579b 100644
--- a/arch/x86/kvm/vmx/capabilities.h
+++ b/arch/x86/kvm/vmx/capabilities.h
@@ -90,6 +90,12 @@ static inline bool cpu_has_vmx_preemption_timer(void)
 		PIN_BASED_VMX_PREEMPTION_TIMER;
 }
 
+static inline bool cpu_has_vmx_apic_timer_virt(void)
+{
+	return vmcs_config.cpu_based_3rd_exec_ctrl &
+		TERTIARY_EXEC_GUEST_APIC_TIMER;
+}
+
 static inline bool cpu_has_vmx_posted_intr(void)
 {
 	return vmcs_config.pin_based_exec_ctrl & PIN_BASED_POSTED_INTR;
diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c
index dbebddf648be..ed20c859def3 100644
--- a/arch/x86/kvm/vmx/main.c
+++ b/arch/x86/kvm/vmx/main.c
@@ -994,6 +994,11 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 #ifdef CONFIG_X86_64
 	.set_hv_timer = vt_op(set_hv_timer),
 	.cancel_hv_timer = vt_op(cancel_hv_timer),
+	.can_use_apic_virt_timer = vmx_can_use_apic_virt_timer,
+	.set_apic_virt_timer = vmx_set_apic_virt_timer,
+	.cancel_apic_virt_timer = vmx_cancel_apic_virt_timer,
+	.set_guest_tsc_deadline_virt = vmx_set_guest_tsc_deadline_virt,
+	.get_guest_tsc_deadline_virt = vmx_get_guest_tsc_deadline_virt,
 #endif
 
 	.setup_mce = vt_op(setup_mce),
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 4ccb2e42322d..b70641bfecab 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -2802,7 +2802,8 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 					      & ~TERTIARY_EXEC_GUEST_APIC_TIMER,
 					      MSR_IA32_VMX_PROCBASED_CTLS3);
 
-	if (!(_cpu_based_2nd_exec_control & SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY))
+	if (!IS_ENABLED(CONFIG_X86_64) ||
+	    !(_cpu_based_2nd_exec_control & SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY))
 		_cpu_based_3rd_exec_control &= ~TERTIARY_EXEC_GUEST_APIC_TIMER;
 
 	if (adjust_vmx_controls(KVM_REQUIRED_VMX_VM_EXIT_CONTROLS,
@@ -8364,6 +8365,86 @@ void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu)
 {
 	to_vmx(vcpu)->hv_deadline_tsc = -1;
 }
+
+bool vmx_can_use_apic_virt_timer(struct kvm_vcpu *vcpu)
+{
+	if (vcpu->kvm->arch.vm_type != KVM_X86_DEFAULT_VM)
+		return false;
+
+	return cpu_has_vmx_apic_timer_virt() &&
+		/* SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY and in-kernel apic */
+		kvm_vcpu_apicv_active(vcpu) &&
+		/* VMX guest virtual timer supports only TSC deadline mode. */
+		kvm_lapic_lvtt_timer_mode(vcpu) == APIC_LVT_TIMER_TSCDEADLINE &&
+		/* KVM doesn't use RDTSC existing. Safeguard. */
+		!(exec_controls_get(to_vmx(vcpu)) & CPU_BASED_RDTSC_EXITING);
+}
+
+void vmx_set_apic_virt_timer(struct kvm_vcpu *vcpu, u16 vector)
+{
+	vmcs_write16(GUEST_APIC_TIMER_VECTOR, vector);
+	vmx_disable_intercept_for_msr(vcpu, MSR_IA32_TSC_DEADLINE, MSR_TYPE_RW);
+	tertiary_exec_controls_setbit(to_vmx(vcpu), TERTIARY_EXEC_GUEST_APIC_TIMER);
+}
+
+void vmx_cancel_apic_virt_timer(struct kvm_vcpu *vcpu)
+{
+	vmx_enable_intercept_for_msr(vcpu, MSR_IA32_TSC_DEADLINE, MSR_TYPE_RW);
+	tertiary_exec_controls_clearbit(to_vmx(vcpu), TERTIARY_EXEC_GUEST_APIC_TIMER);
+}
+
+static u64 vmx_calc_deadline_l1_to_host(struct kvm_vcpu *vcpu, u64 l1_tsc)
+{
+	u64 host_tsc_now = rdtsc();
+	u64 l1_tsc_now = kvm_read_l1_tsc(vcpu, host_tsc_now);
+	u64 host_tsc;
+
+	/* 0 means that timer is disarmed. */
+	if (!l1_tsc)
+		return 0;
+
+	host_tsc = l1_tsc - vcpu->arch.l1_tsc_offset;
+	if (vcpu->arch.l1_tsc_scaling_ratio != kvm_caps.default_tsc_scaling_ratio)
+		if (u64_shl_div_u64(l1_tsc,
+				    kvm_caps.tsc_scaling_ratio_frac_bits,
+				    vcpu->arch.l1_tsc_scaling_ratio,
+				    &host_tsc))
+			host_tsc = ~0ull;
+
+	/*
+	 * Clamp the result on overflow.
+	 * TSC deadline isn't supposed to overflow in practice.
+	 * ~0ull is considered that the timer is armed, but won't fire in
+	 * practical time frame.
+	 */
+	if (l1_tsc > l1_tsc_now && host_tsc <= host_tsc_now)
+		host_tsc = ~0ull;
+	/*
+	 * Clamp the result on underflow.
+	 * The past value means fire the timer immediately.
+	 * Pick the obvious past value.
+	 */
+	if (l1_tsc <= l1_tsc_now && host_tsc > host_tsc_now)
+		host_tsc = 1ull;
+
+	if (!host_tsc)
+		host_tsc = 1ull;
+
+	return host_tsc;
+}
+
+void vmx_set_guest_tsc_deadline_virt(struct kvm_vcpu *vcpu,
+				     u64 guest_deadline_virt)
+{
+	vmcs_write64(GUEST_DEADLINE_VIR, guest_deadline_virt);
+	vmcs_write64(GUEST_DEADLINE_PHY,
+		     vmx_calc_deadline_l1_to_host(vcpu, guest_deadline_virt));
+}
+
+u64 vmx_get_guest_tsc_deadline_virt(struct kvm_vcpu *vcpu)
+{
+	return vmcs_read64(GUEST_DEADLINE_VIR);
+}
 #endif
 
 void vmx_update_cpu_dirty_logging(struct kvm_vcpu *vcpu)
diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h
index d09abeac2b56..364050e0427c 100644
--- a/arch/x86/kvm/vmx/x86_ops.h
+++ b/arch/x86/kvm/vmx/x86_ops.h
@@ -117,6 +117,11 @@ void vmx_update_cpu_dirty_logging(struct kvm_vcpu *vcpu);
 int vmx_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc,
 		     bool *expired);
 void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu);
+bool vmx_can_use_apic_virt_timer(struct kvm_vcpu *vcpu);
+void vmx_set_apic_virt_timer(struct kvm_vcpu *vcpu, u16 vector);
+void vmx_cancel_apic_virt_timer(struct kvm_vcpu *vcpu);
+void vmx_set_guest_tsc_deadline_virt(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc);
+u64 vmx_get_guest_tsc_deadline_virt(struct kvm_vcpu *vcpu);
 #endif
 void vmx_setup_mce(struct kvm_vcpu *vcpu);
 
-- 
2.45.2

[PATCH v2 07/36] KVM: VMX: Update APIC timer virtualization on apicv changed

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

VMX APIC timer virtualization requires the virtual-interrupt delivery.
It's part of KVM apicv, and when apicv state is changed, the
refresh_apicv_exec_ctrl() callback is invoked.  Export a lapic function to
update which timer to use for APIC timer emulation and make the VMX backend
call it.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
--
Changes v1 -> v2:
- Use EXPORT_SYMBOL_FOR_KVM_INTERNAL() instead of EXPORT_SYMBOL_GPL().
- Add in-kernel apic check.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/lapic.c   | 9 +++++++++
 arch/x86/kvm/lapic.h   | 1 +
 arch/x86/kvm/vmx/vmx.c | 2 ++
 3 files changed, 12 insertions(+)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index abbd51c4da7f..a3c1a81e63e2 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -1946,6 +1946,15 @@ static void apic_update_apic_virt_timer(struct kvm_lapic *apic)
 		apic_cancel_apic_virt_timer(apic);
 }
 
+void kvm_update_apic_virt_timer(struct kvm_vcpu *vcpu)
+{
+	if (!lapic_in_kernel(vcpu))
+		return;
+
+	apic_update_apic_virt_timer(vcpu->arch.apic);
+}
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_update_apic_virt_timer);
+
 static void apic_update_lvtt(struct kvm_lapic *apic)
 {
 	u32 timer_mode = kvm_lapic_get_reg(apic, APIC_LVTT) &
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index 2f510503f5b3..0571b7438328 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -259,6 +259,7 @@ void kvm_lapic_switch_to_apic_virt_timer(struct kvm_vcpu *vcpu);
 void kvm_lapic_expired_hv_timer(struct kvm_vcpu *vcpu);
 bool kvm_lapic_hv_timer_in_use(struct kvm_vcpu *vcpu);
 void kvm_lapic_restart_hv_timer(struct kvm_vcpu *vcpu);
+void kvm_update_apic_virt_timer(struct kvm_vcpu *vcpu);
 
 static inline bool kvm_lapic_apic_virt_timer_in_use(struct kvm_vcpu *vcpu)
 {
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index b70641bfecab..5d36f2b632e9 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -4591,6 +4591,8 @@ void vmx_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu)
 						 kvm_vcpu_apicv_active(vcpu));
 
 	vmx_update_msr_bitmap_x2apic(vcpu);
+
+	kvm_update_apic_virt_timer(vcpu);
 }
 
 static u32 vmx_exec_control(struct vcpu_vmx *vmx)
-- 
2.45.2

[PATCH v2 08/36] KVM: nVMX: Disallow/allow guest APIC timer virtualization switch to/from L2

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Disable guest APIC timer virtualization on nested VMEnter, enable it on
nested vmexit.

With VMX APIC timer virtualization, the CPU directly injects a guest timer
interrupt without VMExit.  When the L1 APIC timer fires while running the
nested (L2) vCPU, KVM should emulate VMExit from L2 to L1.  Switch to the
hv timer (preemption timer) or the sw timer when VMEntering from L1 to L2,
switch to guest APIC timer virtualization when VM exiting from L2 to L1.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
--
Changes v1->v2:
- Use EXPORT_SYMBOL_FOR_KVM_INTERNAL() instead of EXPORT_SYMBOL_GPL().
- Add in-kernel apic check.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/lapic.c      | 35 +++++++++++++++++++++++++++++++++++
 arch/x86/kvm/lapic.h      |  2 ++
 arch/x86/kvm/vmx/nested.c | 13 +++++++++++++
 arch/x86/kvm/vmx/vmx.c    |  5 +++++
 4 files changed, 55 insertions(+)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index a3c1a81e63e2..03667ca6357e 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -1909,6 +1909,41 @@ static void apic_cancel_apic_virt_timer(struct kvm_lapic *apic)
 	start_apic_timer(apic);
 }
 
+void kvm_sync_apic_virt_timer(struct kvm_vcpu *vcpu)
+{
+	struct kvm_lapic *apic = vcpu->arch.apic;
+
+	WARN_ON_ONCE(is_guest_mode(vcpu));
+
+	if (!lapic_in_kernel(vcpu))
+		return;
+
+	if (!apic->lapic_timer.apic_virt_timer_in_use)
+		return;
+
+	apic->lapic_timer.tscdeadline = kvm_x86_call(get_guest_tsc_deadline_virt)(vcpu);
+}
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_sync_apic_virt_timer);
+
+void kvm_cancel_apic_virt_timer(struct kvm_vcpu *vcpu)
+{
+	struct kvm_lapic *apic = vcpu->arch.apic;
+
+	WARN_ON_ONCE(!is_guest_mode(vcpu));
+
+	if (!lapic_in_kernel(vcpu))
+		return;
+
+	if (!apic->lapic_timer.apic_virt_timer_in_use)
+		return;
+
+	apic->lapic_timer.apic_virt_timer_in_use = false;
+	trace_kvm_apic_virt_timer_state(vcpu->vcpu_id, false);
+
+	start_apic_timer(apic);
+}
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_cancel_apic_virt_timer);
+
 static void apic_set_apic_virt_timer(struct kvm_lapic *apic)
 {
 	struct kvm_timer *ktimer = &apic->lapic_timer;
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index 0571b7438328..8e7ee5f3a01d 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -260,6 +260,8 @@ void kvm_lapic_expired_hv_timer(struct kvm_vcpu *vcpu);
 bool kvm_lapic_hv_timer_in_use(struct kvm_vcpu *vcpu);
 void kvm_lapic_restart_hv_timer(struct kvm_vcpu *vcpu);
 void kvm_update_apic_virt_timer(struct kvm_vcpu *vcpu);
+void kvm_sync_apic_virt_timer(struct kvm_vcpu *vcpu);
+void kvm_cancel_apic_virt_timer(struct kvm_vcpu *vcpu);
 
 static inline bool kvm_lapic_apic_virt_timer_in_use(struct kvm_vcpu *vcpu)
 {
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 101588914cbb..c475b8c94807 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -3656,6 +3656,8 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,
 	if (!enable_ept)
 		vmcs_writel(GUEST_CR3, vcpu->arch.cr3);
 
+	kvm_sync_apic_virt_timer(vcpu);
+
 	vmx_switch_vmcs(vcpu, &vmx->nested.vmcs02);
 
 	prepare_vmcs02_early(vmx, &vmx->vmcs01, vmcs12);
@@ -3731,6 +3733,14 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,
 		vmx_start_preemption_timer(vcpu, timer_value);
 	}
 
+	/*
+	 * Disable apic virtual timer for L1 to use sw timer (hr timer) or
+	 * hypervisor timer (VMX preemption timer).
+	 * When L1 timer interrupt occurs during running L2, KVM emulates
+	 * VMExit from L2 to L1.  Not directly injecting the interrupt into L2.
+	 */
+	kvm_cancel_apic_virt_timer(vcpu);
+
 	/*
 	 * Note no nested_vmx_succeed or nested_vmx_fail here. At this point
 	 * we are no longer running L1, and VMLAUNCH/VMRESUME has not yet
@@ -5158,6 +5168,9 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
 	/* in case we halted in L2 */
 	kvm_set_mp_state(vcpu, KVM_MP_STATE_RUNNABLE);
 
+	/* If apic virtual timer is supported, switch back to it. */
+	kvm_update_apic_virt_timer(vcpu);
+
 	if (likely(!vmx->fail)) {
 		if (vm_exit_reason != -1)
 			trace_kvm_nested_vmexit_inject(vmcs12->vm_exit_reason,
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 5d36f2b632e9..9d5a493b6fe5 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -8373,6 +8373,9 @@ bool vmx_can_use_apic_virt_timer(struct kvm_vcpu *vcpu)
 	if (vcpu->kvm->arch.vm_type != KVM_X86_DEFAULT_VM)
 		return false;
 
+	if (is_guest_mode(vcpu))
+		return false;
+
 	return cpu_has_vmx_apic_timer_virt() &&
 		/* SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY and in-kernel apic */
 		kvm_vcpu_apicv_active(vcpu) &&
@@ -8384,6 +8387,8 @@ bool vmx_can_use_apic_virt_timer(struct kvm_vcpu *vcpu)
 
 void vmx_set_apic_virt_timer(struct kvm_vcpu *vcpu, u16 vector)
 {
+	WARN_ON_ONCE(is_guest_mode(vcpu));
+
 	vmcs_write16(GUEST_APIC_TIMER_VECTOR, vector);
 	vmx_disable_intercept_for_msr(vcpu, MSR_IA32_TSC_DEADLINE, MSR_TYPE_RW);
 	tertiary_exec_controls_setbit(to_vmx(vcpu), TERTIARY_EXEC_GUEST_APIC_TIMER);
-- 
2.45.2

[PATCH v2 09/36] KVM: nVMX: Pass struct msr_data to VMX MSRs emulation

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Pass struct msr_data, which has host_initiated member in addition to
msr_index and data to vmx_set/get_vmx_msr().

Processor-based tertiary control access depends on which initiated the
operation, the host or the guest. For host-initiated access (KVM_GET_MSRS,
KVM_SET_MSRS), if the host supports processor-based tertiary controls,
allow access.  If guest-initiated access (emulation for guest rdmsr/wrmsr),
allow/disallow based on guest tertiary controls is advertised to the guest
(guest processor-based control high &
CPU_BASED_ACTIVATE_TERTIARY_CONTROLS).  Prepare to add the check.

No functional change intended.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/vmx/nested.c |  9 +++++++--
 arch/x86/kvm/vmx/nested.h |  4 ++--
 arch/x86/kvm/vmx/vmx.c    | 18 ++++++++++++++----
 3 files changed, 23 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index c475b8c94807..5f0ac8acd768 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -1475,9 +1475,11 @@ static int vmx_restore_fixed0_msr(struct vcpu_vmx *vmx, u32 msr_index, u64 data)
  *
  * Returns 0 on success, non-0 otherwise.
  */
-int vmx_set_vmx_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data)
+int vmx_set_vmx_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	u32 msr_index = msr_info->index;
+	u64 data = msr_info->data;
 
 	/*
 	 * Don't allow changes to the VMX capability MSRs while the vCPU
@@ -1540,8 +1542,11 @@ int vmx_set_vmx_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data)
 }
 
 /* Returns 0 on success, non-0 otherwise. */
-int vmx_get_vmx_msr(struct nested_vmx_msrs *msrs, u32 msr_index, u64 *pdata)
+int vmx_get_vmx_msr(struct nested_vmx_msrs *msrs, struct msr_data *msr_info)
 {
+	u32 msr_index = msr_info->index;
+	u64 *pdata = &msr_info->data;
+
 	switch (msr_index) {
 	case MSR_IA32_VMX_BASIC:
 		*pdata = msrs->basic;
diff --git a/arch/x86/kvm/vmx/nested.h b/arch/x86/kvm/vmx/nested.h
index b844c5d59025..d0257447b7cb 100644
--- a/arch/x86/kvm/vmx/nested.h
+++ b/arch/x86/kvm/vmx/nested.h
@@ -47,8 +47,8 @@ static inline void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
 }
 
 void nested_sync_vmcs12_to_shadow(struct kvm_vcpu *vcpu);
-int vmx_set_vmx_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data);
-int vmx_get_vmx_msr(struct nested_vmx_msrs *msrs, u32 msr_index, u64 *pdata);
+int vmx_set_vmx_msr(struct kvm_vcpu *vcpu, struct msr_data *msr);
+int vmx_get_vmx_msr(struct nested_vmx_msrs *msrs, struct msr_data *msr);
 int get_vmx_mem_address(struct kvm_vcpu *vcpu, unsigned long exit_qualification,
 			u32 vmx_instruction_info, bool wr, int len, gva_t *ret);
 bool nested_vmx_check_io_bitmaps(struct kvm_vcpu *vcpu, unsigned int port,
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 9d5a493b6fe5..9177b693df1b 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -2070,11 +2070,22 @@ static inline bool is_vmx_feature_control_msr_valid(struct vcpu_vmx *vmx,
 
 int vmx_get_feature_msr(u32 msr, u64 *data)
 {
+	struct msr_data msr_info;
+	int r;
+
 	switch (msr) {
 	case KVM_FIRST_EMULATED_VMX_MSR ... KVM_LAST_EMULATED_VMX_MSR:
 		if (!nested)
 			return 1;
-		return vmx_get_vmx_msr(&vmcs_config.nested, msr, data);
+
+		msr_info = (struct msr_data) {
+			.index = msr,
+			.host_initiated = true,
+		};
+		r = vmx_get_vmx_msr(&vmcs_config.nested, &msr_info);
+		if (!r)
+			*data = msr_info.data;
+		return r;
 	default:
 		return KVM_MSR_RET_UNSUPPORTED;
 	}
@@ -2159,8 +2170,7 @@ int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	case KVM_FIRST_EMULATED_VMX_MSR ... KVM_LAST_EMULATED_VMX_MSR:
 		if (!guest_cpu_cap_has(vcpu, X86_FEATURE_VMX))
 			return 1;
-		if (vmx_get_vmx_msr(&vmx->nested.msrs, msr_info->index,
-				    &msr_info->data))
+		if (vmx_get_vmx_msr(&vmx->nested.msrs, msr_info))
 			return 1;
 #ifdef CONFIG_KVM_HYPERV
 		/*
@@ -2487,7 +2497,7 @@ int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			return 1; /* they are read-only */
 		if (!guest_cpu_cap_has(vcpu, X86_FEATURE_VMX))
 			return 1;
-		return vmx_set_vmx_msr(vcpu, msr_index, data);
+		return vmx_set_vmx_msr(vcpu, msr_info);
 	case MSR_IA32_RTIT_CTL:
 		if (!vmx_pt_mode_is_host_guest() ||
 			vmx_rtit_ctl_check(vcpu, data) ||
-- 
2.45.2

[PATCH v2 10/36] KVM: nVMX: Supports VMX tertiary controls and GUEST_APIC_TIMER bit

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Emulate MSR_IA32_VMX_PROCBASED_CTLS3 to advertise APIC timer virtualization
feature to the L2 guest.

Reported-by: syzbot+ci66a37fb2e2f8de71@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/kvm/6982f952.050a0220.3b3015.0012.GAE@google.com/
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
Changes:
v1 -> v2:
- When in-kernel lapic is disabled, disable nested apic timer
  virtualization.
- disable apic timer virtualization bit in nested tertiary vm exec control
  when in-kernel lapic is disabled.
---
 arch/x86/kvm/vmx/capabilities.h |  1 +
 arch/x86/kvm/vmx/hyperv.c       |  7 +++++
 arch/x86/kvm/vmx/nested.c       | 48 +++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/nested.h       | 11 ++++++++
 arch/x86/kvm/vmx/vmx.c          |  6 ++++-
 arch/x86/kvm/x86.h              |  2 +-
 6 files changed, 73 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilities.h
index c5cb098f579b..8d67be77f02c 100644
--- a/arch/x86/kvm/vmx/capabilities.h
+++ b/arch/x86/kvm/vmx/capabilities.h
@@ -47,6 +47,7 @@ struct nested_vmx_msrs {
 	u64 cr4_fixed1;
 	u64 vmcs_enum;
 	u64 vmfunc_controls;
+	u64 tertiary_ctls;
 };
 
 struct vmcs_config {
diff --git a/arch/x86/kvm/vmx/hyperv.c b/arch/x86/kvm/vmx/hyperv.c
index fa41d036acd4..2731c2e4b0e5 100644
--- a/arch/x86/kvm/vmx/hyperv.c
+++ b/arch/x86/kvm/vmx/hyperv.c
@@ -141,6 +141,13 @@ void nested_evmcs_filter_control_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 *
 	case MSR_IA32_VMX_PROCBASED_CTLS2:
 		ctl_high &= evmcs_get_supported_ctls(EVMCS_2NDEXEC);
 		break;
+	case MSR_IA32_VMX_PROCBASED_CTLS3:
+		/*
+		 * tertiary procbased controls are 64-bit. 0 means unsupported,
+		 * 1 supported.
+		 */
+		*pdata &= evmcs_get_supported_ctls(EVMCS_3RDEXEC);
+		return;
 	case MSR_IA32_VMX_TRUE_PINBASED_CTLS:
 	case MSR_IA32_VMX_PINBASED_CTLS:
 		ctl_high &= evmcs_get_supported_ctls(EVMCS_PINCTRL);
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 5f0ac8acd768..be6b92b3c66a 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -218,6 +218,11 @@ static inline bool vmx_control_verify(u32 control, u32 low, u32 high)
 	return fixed_bits_valid(control, low, high);
 }
 
+static inline bool vmx_control64_verify(u64 control, u64 msr)
+{
+	return !(control & ~msr);
+}
+
 static inline u64 vmx_control_msr(u32 low, u32 high)
 {
 	return low | ((u64)high << 32);
@@ -1511,6 +1516,25 @@ int vmx_set_vmx_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	case MSR_IA32_VMX_TRUE_ENTRY_CTLS:
 	case MSR_IA32_VMX_PROCBASED_CTLS2:
 		return vmx_restore_control_msr(vmx, msr_index, data);
+	case MSR_IA32_VMX_PROCBASED_CTLS3: {
+		u64 ctls3;
+
+		if (!__nested_cpu_supports_tertiary_ctls(&vmcs_config.nested))
+			return -EINVAL;
+
+		/* read-only for guest. */
+		if (!msr_info->host_initiated)
+			return -EINVAL;
+
+		ctls3 = vmcs_config.nested.tertiary_ctls;
+		if (!nested_cpu_can_support_apic_virt_timer(vcpu))
+			ctls3 &= ~TERTIARY_EXEC_GUEST_APIC_TIMER;
+
+		if (!vmx_control64_verify(data, ctls3))
+			return -EINVAL;
+		vmx->nested.msrs.tertiary_ctls = data;
+		return 0;
+	}
 	case MSR_IA32_VMX_MISC:
 		return vmx_restore_vmx_misc(vmx, data);
 	case MSR_IA32_VMX_CR0_FIXED0:
@@ -1608,6 +1632,16 @@ int vmx_get_vmx_msr(struct nested_vmx_msrs *msrs, struct msr_data *msr_info)
 			msrs->secondary_ctls_low,
 			msrs->secondary_ctls_high);
 		break;
+	case MSR_IA32_VMX_PROCBASED_CTLS3:
+		if (!__nested_cpu_supports_tertiary_ctls(&vmcs_config.nested))
+			return KVM_MSR_RET_UNSUPPORTED;
+
+		if (!msr_info->host_initiated &&
+		    !__nested_cpu_supports_tertiary_ctls(msrs))
+			return -EINVAL;
+
+		*pdata = msrs->tertiary_ctls;
+		break;
 	case MSR_IA32_VMX_EPT_VPID_CAP:
 		*pdata = msrs->ept_caps |
 			((u64)msrs->vpid_caps << 32);
@@ -7285,6 +7319,18 @@ static void nested_vmx_setup_secondary_ctls(u32 ept_caps,
 		msrs->secondary_ctls_high |= SECONDARY_EXEC_ENCLS_EXITING;
 }
 
+static void nested_vmx_setup_tertiary_ctls(struct vmcs_config *vmcs_conf,
+					   struct nested_vmx_msrs *msrs)
+{
+	msrs->tertiary_ctls = vmcs_conf->cpu_based_3rd_exec_ctrl;
+
+	msrs->tertiary_ctls &= TERTIARY_EXEC_GUEST_APIC_TIMER;
+
+	if (msrs->tertiary_ctls)
+		msrs->procbased_ctls_high |=
+			CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+}
+
 static void nested_vmx_setup_misc_data(struct vmcs_config *vmcs_conf,
 				       struct nested_vmx_msrs *msrs)
 {
@@ -7373,6 +7419,8 @@ void nested_vmx_setup_ctls_msrs(struct vmcs_config *vmcs_conf, u32 ept_caps)
 
 	nested_vmx_setup_secondary_ctls(ept_caps, vmcs_conf, msrs);
 
+	nested_vmx_setup_tertiary_ctls(vmcs_conf, msrs);
+
 	nested_vmx_setup_misc_data(vmcs_conf, msrs);
 
 	nested_vmx_setup_basic(msrs);
diff --git a/arch/x86/kvm/vmx/nested.h b/arch/x86/kvm/vmx/nested.h
index d0257447b7cb..8c25054a710e 100644
--- a/arch/x86/kvm/vmx/nested.h
+++ b/arch/x86/kvm/vmx/nested.h
@@ -152,6 +152,17 @@ static inline bool nested_cpu_has_vmx_shadow_vmcs(struct kvm_vcpu *vcpu)
 		SECONDARY_EXEC_SHADOW_VMCS;
 }
 
+static inline bool __nested_cpu_supports_tertiary_ctls(struct nested_vmx_msrs *msrs)
+{
+	return msrs->procbased_ctls_high & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+}
+
+/* APIC TIMER VIRTUALIZATION requires in-kernel lapic. */
+static inline bool nested_cpu_can_support_apic_virt_timer(struct kvm_vcpu *vcpu)
+{
+	return cpu_has_vmx_apic_timer_virt() && lapic_in_kernel(vcpu);
+}
+
 static inline bool nested_cpu_has(struct vmcs12 *vmcs12, u32 bit)
 {
 	return vmcs12->cpu_based_vm_exec_control & bit;
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 9177b693df1b..25f31103cb21 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -4985,9 +4985,13 @@ static void __vmx_vcpu_reset(struct kvm_vcpu *vcpu)
 	init_vmcs(vmx);
 
 	if (nested &&
-	    kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_STUFF_FEATURE_MSRS))
+	    kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_STUFF_FEATURE_MSRS)) {
 		memcpy(&vmx->nested.msrs, &vmcs_config.nested, sizeof(vmx->nested.msrs));
 
+		if (!nested_cpu_can_support_apic_virt_timer(vcpu))
+			vmx->nested.msrs.tertiary_ctls &= ~TERTIARY_EXEC_GUEST_APIC_TIMER;
+	}
+
 	vcpu_setup_sgx_lepubkeyhash(vcpu);
 
 	vmx->nested.posted_intr_nv = -1;
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index 44a28d343d40..1de31aae9c6c 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -92,7 +92,7 @@ do {											\
  * associated feature that KVM supports for nested virtualization.
  */
 #define KVM_FIRST_EMULATED_VMX_MSR	MSR_IA32_VMX_BASIC
-#define KVM_LAST_EMULATED_VMX_MSR	MSR_IA32_VMX_VMFUNC
+#define KVM_LAST_EMULATED_VMX_MSR	MSR_IA32_VMX_PROCBASED_CTLS3
 
 #define KVM_DEFAULT_PLE_GAP		128
 #define KVM_VMX_DEFAULT_PLE_WINDOW	4096
-- 
2.45.2

[PATCH v2 11/36] KVM: nVMX: Add alignment check for struct vmcs12

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

As struct vmcs12 has __packed attributes and it has manual alignment,
it's easy to add unaligned member without any warnings.  Add alignment
check to avoid accidental misalignment.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/vmx/vmcs12.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/vmx/vmcs12.h b/arch/x86/kvm/vmx/vmcs12.h
index 21cd1b75e4fd..8c9d4c22b960 100644
--- a/arch/x86/kvm/vmx/vmcs12.h
+++ b/arch/x86/kvm/vmx/vmcs12.h
@@ -217,7 +217,8 @@ struct __packed vmcs12 {
  * although appending fields and/or filling gaps is obviously allowed.
  */
 #define CHECK_OFFSET(field, loc) \
-	ASSERT_STRUCT_OFFSET(struct vmcs12, field, loc)
+	ASSERT_STRUCT_OFFSET(struct vmcs12, field, loc);		\
+	static_assert(loc % __alignof(((struct vmcs12 *)0)->field) == 0)
 
 static inline void vmx_check_vmcs12_offsets(void)
 {
-- 
2.45.2

[PATCH v2 12/36] KVM: nVMX: Add tertiary VM-execution control VMCS support

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Support tertiary processor-based VM-execution control VMCS field.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/vmx/hyperv.c | 10 ++++++++++
 arch/x86/kvm/vmx/nested.c | 17 +++++++++++++++++
 arch/x86/kvm/vmx/nested.h |  7 +++++++
 arch/x86/kvm/vmx/vmcs12.c |  1 +
 arch/x86/kvm/vmx/vmcs12.h |  3 ++-
 5 files changed, 37 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/vmx/hyperv.c b/arch/x86/kvm/vmx/hyperv.c
index 2731c2e4b0e5..70e210472681 100644
--- a/arch/x86/kvm/vmx/hyperv.c
+++ b/arch/x86/kvm/vmx/hyperv.c
@@ -166,6 +166,12 @@ static bool nested_evmcs_is_valid_controls(enum evmcs_ctrl_type ctrl_type,
 	return !(val & ~evmcs_get_supported_ctls(ctrl_type));
 }
 
+static bool nested_evmcs_is_valid_controls64(enum evmcs_ctrl_type ctrl_type,
+					     u64 val)
+{
+	return !(val & ~evmcs_get_supported_ctls(ctrl_type));
+}
+
 int nested_evmcs_check_controls(struct vmcs12 *vmcs12)
 {
 	if (CC(!nested_evmcs_is_valid_controls(EVMCS_PINCTRL,
@@ -188,6 +194,10 @@ int nested_evmcs_check_controls(struct vmcs12 *vmcs12)
 					       vmcs12->vm_entry_controls)))
 		return -EINVAL;
 
+	if (CC(!nested_evmcs_is_valid_controls64(EVMCS_3RDEXEC,
+						 vmcs12->tertiary_vm_exec_control)))
+		return -EINVAL;
+
 	/*
 	 * VM-Func controls are 64-bit, but KVM currently doesn't support any
 	 * controls in bits 63:32, i.e. dropping those bits on the consistency
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index be6b92b3c66a..1bd5e164e285 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -1815,6 +1815,7 @@ static void copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx, u32 hv_clean_fields
 		vmcs12->vm_exit_controls = evmcs->vm_exit_controls;
 		vmcs12->secondary_vm_exec_control =
 			evmcs->secondary_vm_exec_control;
+		vmcs12->tertiary_vm_exec_control = 0;
 	}
 
 	if (unlikely(!(hv_clean_fields &
@@ -2511,6 +2512,17 @@ static void prepare_vmcs02_early(struct vcpu_vmx *vmx, struct loaded_vmcs *vmcs0
 		secondary_exec_controls_set(vmx, exec_control);
 	}
 
+	/*
+	 * TERTIARY EXEC CONTROLS
+	 */
+	if (cpu_has_tertiary_exec_ctrls()) {
+		u64 ctls = 0;
+
+		/* guest apic timer virtualization will come */
+
+		tertiary_exec_controls_set(vmx, ctls);
+	}
+
 	/*
 	 * ENTRY CONTROLS
 	 *
@@ -2969,6 +2981,11 @@ static int nested_check_vm_execution_controls(struct kvm_vcpu *vcpu,
 				   vmx->nested.msrs.secondary_ctls_high)))
 		return -EINVAL;
 
+	if (nested_cpu_has(vmcs12, CPU_BASED_ACTIVATE_TERTIARY_CONTROLS) &&
+	    CC(!vmx_control64_verify(vmcs12->tertiary_vm_exec_control,
+				     vmx->nested.msrs.tertiary_ctls)))
+		return -EINVAL;
+
 	if (CC(vmcs12->cr3_target_count > nested_cpu_vmx_misc_cr3_count(vcpu)) ||
 	    nested_vmx_check_io_bitmap_controls(vcpu, vmcs12) ||
 	    nested_vmx_check_msr_bitmap_controls(vcpu, vmcs12) ||
diff --git a/arch/x86/kvm/vmx/nested.h b/arch/x86/kvm/vmx/nested.h
index 8c25054a710e..52bf035bcc03 100644
--- a/arch/x86/kvm/vmx/nested.h
+++ b/arch/x86/kvm/vmx/nested.h
@@ -175,6 +175,13 @@ static inline bool nested_cpu_has2(struct vmcs12 *vmcs12, u32 bit)
 		(vmcs12->secondary_vm_exec_control & bit);
 }
 
+static inline bool nested_cpu_has3(struct vmcs12 *vmcs12, u64 bit)
+{
+	return (vmcs12->cpu_based_vm_exec_control &
+			CPU_BASED_ACTIVATE_TERTIARY_CONTROLS) &&
+		(vmcs12->tertiary_vm_exec_control & bit);
+}
+
 static inline bool nested_cpu_has_preemption_timer(struct vmcs12 *vmcs12)
 {
 	return vmcs12->pin_based_vm_exec_control &
diff --git a/arch/x86/kvm/vmx/vmcs12.c b/arch/x86/kvm/vmx/vmcs12.c
index 1ebe67c384ad..e2e2a99c8aa9 100644
--- a/arch/x86/kvm/vmx/vmcs12.c
+++ b/arch/x86/kvm/vmx/vmcs12.c
@@ -38,6 +38,7 @@ static const u16 kvm_supported_vmcs12_field_offsets[] __initconst = {
 	FIELD64(PML_ADDRESS, pml_address),
 	FIELD64(TSC_OFFSET, tsc_offset),
 	FIELD64(TSC_MULTIPLIER, tsc_multiplier),
+	FIELD64(TERTIARY_VM_EXEC_CONTROL, tertiary_vm_exec_control),
 	FIELD64(VIRTUAL_APIC_PAGE_ADDR, virtual_apic_page_addr),
 	FIELD64(APIC_ACCESS_ADDR, apic_access_addr),
 	FIELD64(POSTED_INTR_DESC_ADDR, posted_intr_desc_addr),
diff --git a/arch/x86/kvm/vmx/vmcs12.h b/arch/x86/kvm/vmx/vmcs12.h
index 8c9d4c22b960..b7d30a2cf23f 100644
--- a/arch/x86/kvm/vmx/vmcs12.h
+++ b/arch/x86/kvm/vmx/vmcs12.h
@@ -71,7 +71,7 @@ struct __packed vmcs12 {
 	u64 pml_address;
 	u64 encls_exiting_bitmap;
 	u64 tsc_multiplier;
-	u64 padding64[1]; /* room for future expansion */
+	u64 tertiary_vm_exec_control;
 	/*
 	 * To allow migration of L1 (complete with its L2 guests) between
 	 * machines of different natural widths (32 or 64 bit), we cannot have
@@ -262,6 +262,7 @@ static inline void vmx_check_vmcs12_offsets(void)
 	CHECK_OFFSET(pml_address, 312);
 	CHECK_OFFSET(encls_exiting_bitmap, 320);
 	CHECK_OFFSET(tsc_multiplier, 328);
+	CHECK_OFFSET(tertiary_vm_exec_control, 336);
 	CHECK_OFFSET(cr0_guest_host_mask, 344);
 	CHECK_OFFSET(cr4_guest_host_mask, 352);
 	CHECK_OFFSET(cr0_read_shadow, 360);
-- 
2.45.2

[PATCH v2 13/36] KVM: nVMX: Update intercept on TSC deadline MSR

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

When APIC timer virtualization is enabled, the hardware handles the access
to the guest TSC deadline MSR, not by the VMM.  Disable/enable MSR
intercept on TSC DEADLINE MSR based on the APIC timer virtualization bit of
tertiary processor-based execution control.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/vmx/nested.c | 22 ++++++++++++++++++++++
 arch/x86/kvm/vmx/nested.h |  5 +++++
 2 files changed, 27 insertions(+)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 1bd5e164e285..8bb8734cc690 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -598,6 +598,26 @@ static inline void enable_x2apic_msr_intercepts(unsigned long *msr_bitmap)
 	}
 }
 
+static inline void prepare_tsc_deadline_msr_intercepts(struct vmcs12 *vmcs12,
+						       unsigned long *msr_bitmap_l1,
+						       unsigned long *msr_bitmap_l0)
+{
+	if (nested_cpu_has_guest_apic_timer(vmcs12)) {
+		if (vmx_test_msr_bitmap_read(msr_bitmap_l1, MSR_IA32_TSC_DEADLINE))
+			vmx_set_msr_bitmap_read(msr_bitmap_l0, MSR_IA32_TSC_DEADLINE);
+		else
+			vmx_clear_msr_bitmap_read(msr_bitmap_l0, MSR_IA32_TSC_DEADLINE);
+
+		if (vmx_test_msr_bitmap_write(msr_bitmap_l1, MSR_IA32_TSC_DEADLINE))
+			vmx_set_msr_bitmap_write(msr_bitmap_l0, MSR_IA32_TSC_DEADLINE);
+		else
+			vmx_clear_msr_bitmap_write(msr_bitmap_l0, MSR_IA32_TSC_DEADLINE);
+	} else {
+		vmx_set_msr_bitmap_read(msr_bitmap_l0, MSR_IA32_TSC_DEADLINE);
+		vmx_set_msr_bitmap_write(msr_bitmap_l0, MSR_IA32_TSC_DEADLINE);
+	}
+}
+
 #define BUILD_NVMX_MSR_INTERCEPT_HELPER(rw)					\
 static inline									\
 void nested_vmx_set_msr_##rw##_intercept(struct vcpu_vmx *vmx,			\
@@ -745,6 +765,8 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu,
 		}
 	}
 
+	prepare_tsc_deadline_msr_intercepts(vmcs12, msr_bitmap_l1, msr_bitmap_l0);
+
 	/*
 	 * Always check vmcs01's bitmap to honor userspace MSR filters and any
 	 * other runtime changes to vmcs01's bitmap, e.g. dynamic pass-through.
diff --git a/arch/x86/kvm/vmx/nested.h b/arch/x86/kvm/vmx/nested.h
index 52bf035bcc03..6df2cfb20d87 100644
--- a/arch/x86/kvm/vmx/nested.h
+++ b/arch/x86/kvm/vmx/nested.h
@@ -286,6 +286,11 @@ static inline bool nested_cpu_has_encls_exit(struct vmcs12 *vmcs12)
 	return nested_cpu_has2(vmcs12, SECONDARY_EXEC_ENCLS_EXITING);
 }
 
+static inline bool nested_cpu_has_guest_apic_timer(struct vmcs12 *vmcs12)
+{
+	return nested_cpu_has3(vmcs12, TERTIARY_EXEC_GUEST_APIC_TIMER);
+}
+
 /*
  * if fixed0[i] == 1: val[i] must be 1
  * if fixed1[i] == 0: val[i] must be 0
-- 
2.45.2

[PATCH v2 14/36] KVM: nVMX: Handle virtual timer vector VMCS field

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Support virtual timer vector VMCS field.
Opportunistically add a size check of struct vmcs12.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
Changes:
v1 -> v2:
- update for 5fdf86e7353c ("KVM: nVMX: Disallow access to vmcs12 fields
  that aren't supported by "hardware"") to use cpu_has_vmcs12_field().
---
 arch/x86/kvm/vmx/nested.c             | 6 +++++-
 arch/x86/kvm/vmx/vmcs12.c             | 5 +++++
 arch/x86/kvm/vmx/vmcs12.h             | 2 ++
 arch/x86/kvm/vmx/vmcs_shadow_fields.h | 1 +
 4 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 8bb8734cc690..b7561f8f4565 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -2540,7 +2540,8 @@ static void prepare_vmcs02_early(struct vcpu_vmx *vmx, struct loaded_vmcs *vmcs0
 	if (cpu_has_tertiary_exec_ctrls()) {
 		u64 ctls = 0;
 
-		/* guest apic timer virtualization will come */
+		if (nested_cpu_has_guest_apic_timer(vmcs12))
+			ctls |= TERTIARY_EXEC_GUEST_APIC_TIMER;
 
 		tertiary_exec_controls_set(vmx, ctls);
 	}
@@ -2734,6 +2735,9 @@ static void prepare_vmcs02_rare(struct vcpu_vmx *vmx, struct vmcs12 *vmcs12)
 		vmcs_write64(EOI_EXIT_BITMAP3, vmcs12->eoi_exit_bitmap3);
 	}
 
+	if (nested_cpu_has_guest_apic_timer(vmcs12))
+		vmcs_write16(GUEST_APIC_TIMER_VECTOR, vmcs12->virtual_timer_vector);
+
 	/*
 	 * If vmcs12 is configured to save TSC on exit via the auto-store list,
 	 * append the MSR to vmcs02's auto-store list so that KVM effectively
diff --git a/arch/x86/kvm/vmx/vmcs12.c b/arch/x86/kvm/vmx/vmcs12.c
index e2e2a99c8aa9..dac796fc20f2 100644
--- a/arch/x86/kvm/vmx/vmcs12.c
+++ b/arch/x86/kvm/vmx/vmcs12.c
@@ -3,6 +3,8 @@
 
 #include "vmcs12.h"
 
+static_assert(sizeof(struct vmcs12) <= VMCS12_SIZE);
+
 #define VMCS12_OFFSET(x) offsetof(struct vmcs12, x)
 #define FIELD(number, name)	[ENC_TO_VMCS12_IDX(number)] = VMCS12_OFFSET(name)
 #define FIELD64(number, name)						\
@@ -22,6 +24,7 @@ static const u16 kvm_supported_vmcs12_field_offsets[] __initconst = {
 	FIELD(GUEST_TR_SELECTOR, guest_tr_selector),
 	FIELD(GUEST_INTR_STATUS, guest_intr_status),
 	FIELD(GUEST_PML_INDEX, guest_pml_index),
+	FIELD(GUEST_APIC_TIMER_VECTOR, virtual_timer_vector),
 	FIELD(HOST_ES_SELECTOR, host_es_selector),
 	FIELD(HOST_CS_SELECTOR, host_cs_selector),
 	FIELD(HOST_SS_SELECTOR, host_ss_selector),
@@ -204,6 +207,8 @@ static __init bool cpu_has_vmcs12_field(unsigned int idx)
 	case HOST_SSP:
 	case HOST_INTR_SSP_TABLE:
 		return cpu_has_load_cet_ctrl();
+	case GUEST_APIC_TIMER_VECTOR:
+		return cpu_has_vmx_apic_timer_virt();
 
 	/* KVM always emulates PML and the VMX preemption timer in software. */
 	case GUEST_PML_INDEX:
diff --git a/arch/x86/kvm/vmx/vmcs12.h b/arch/x86/kvm/vmx/vmcs12.h
index b7d30a2cf23f..7a20d6661da0 100644
--- a/arch/x86/kvm/vmx/vmcs12.h
+++ b/arch/x86/kvm/vmx/vmcs12.h
@@ -191,6 +191,7 @@ struct __packed vmcs12 {
 	u16 host_gs_selector;
 	u16 host_tr_selector;
 	u16 guest_pml_index;
+	u16 virtual_timer_vector;
 };
 
 /*
@@ -374,6 +375,7 @@ static inline void vmx_check_vmcs12_offsets(void)
 	CHECK_OFFSET(host_gs_selector, 992);
 	CHECK_OFFSET(host_tr_selector, 994);
 	CHECK_OFFSET(guest_pml_index, 996);
+	CHECK_OFFSET(virtual_timer_vector, 998);
 }
 
 extern u16 vmcs12_field_offsets[] __ro_after_init;
diff --git a/arch/x86/kvm/vmx/vmcs_shadow_fields.h b/arch/x86/kvm/vmx/vmcs_shadow_fields.h
index cad128d1657b..db1558d11c4c 100644
--- a/arch/x86/kvm/vmx/vmcs_shadow_fields.h
+++ b/arch/x86/kvm/vmx/vmcs_shadow_fields.h
@@ -34,6 +34,7 @@ BUILD_BUG_ON(1)
 /* 16-bits */
 SHADOW_FIELD_RW(GUEST_INTR_STATUS, guest_intr_status)
 SHADOW_FIELD_RW(GUEST_PML_INDEX, guest_pml_index)
+SHADOW_FIELD_RO(GUEST_APIC_TIMER_VECTOR, virtual_timer_vector)
 SHADOW_FIELD_RW(HOST_FS_SELECTOR, host_fs_selector)
 SHADOW_FIELD_RW(HOST_GS_SELECTOR, host_gs_selector)
 
-- 
2.45.2

[PATCH v2 15/36] KVM: VMX: Make vmx_calc_deadline_l1_to_host() non-static

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Remove static from vmx_calc_deadline_l1_to_host() and declare it in vmx.h.

As nVMX APIC timer virtualization will use vmx_calc_deadline_l1_to_host(),
make it available to nested.c.  Make u64_shl_div_u64() usable for X86_32
as vmx_calc_deadline_l1_to_host() is used for both X86_32 and X86_64.
Without this change, ARCH=i386 fails to compile.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/vmx/vmx.c | 41 ++++++++++++++++++++++++-----------------
 arch/x86/kvm/vmx/vmx.h |  2 ++
 2 files changed, 26 insertions(+), 17 deletions(-)

diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 25f31103cb21..bac3f33f7d73 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -8318,25 +8318,32 @@ int vmx_check_intercept(struct kvm_vcpu *vcpu,
 	return X86EMUL_INTERCEPTED;
 }
 
-#ifdef CONFIG_X86_64
 /* (a << shift) / divisor, return 1 if overflow otherwise 0 */
 static inline int u64_shl_div_u64(u64 a, unsigned int shift,
 				  u64 divisor, u64 *result)
 {
-	u64 low = a << shift, high = a >> (64 - shift);
+	u64 high = a >> (64 - shift);
+#ifdef CONFIG_X86_64
+	u64 low = a << shift;
+#endif
 
 	/* To avoid the overflow on divq */
 	if (high >= divisor)
 		return 1;
 
+#ifdef CONFIG_X86_64
 	/* Low hold the result, high hold rem which is discarded */
 	asm("divq %2\n\t" : "=a" (low), "=d" (high) :
 	    "rm" (divisor), "0" (low), "1" (high));
 	*result = low;
+#else
+	*result = mul_u64_u64_div_u64(a, 1ULL << shift, divisor);
+#endif
 
 	return 0;
 }
 
+#ifdef CONFIG_X86_64
 int vmx_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc,
 		     bool *expired)
 {
@@ -8414,7 +8421,21 @@ void vmx_cancel_apic_virt_timer(struct kvm_vcpu *vcpu)
 	tertiary_exec_controls_clearbit(to_vmx(vcpu), TERTIARY_EXEC_GUEST_APIC_TIMER);
 }
 
-static u64 vmx_calc_deadline_l1_to_host(struct kvm_vcpu *vcpu, u64 l1_tsc)
+void vmx_set_guest_tsc_deadline_virt(struct kvm_vcpu *vcpu,
+				     u64 guest_deadline_virt)
+{
+	vmcs_write64(GUEST_DEADLINE_VIR, guest_deadline_virt);
+	vmcs_write64(GUEST_DEADLINE_PHY,
+		     vmx_calc_deadline_l1_to_host(vcpu, guest_deadline_virt));
+}
+
+u64 vmx_get_guest_tsc_deadline_virt(struct kvm_vcpu *vcpu)
+{
+	return vmcs_read64(GUEST_DEADLINE_VIR);
+}
+#endif
+
+u64 vmx_calc_deadline_l1_to_host(struct kvm_vcpu *vcpu, u64 l1_tsc)
 {
 	u64 host_tsc_now = rdtsc();
 	u64 l1_tsc_now = kvm_read_l1_tsc(vcpu, host_tsc_now);
@@ -8454,20 +8475,6 @@ static u64 vmx_calc_deadline_l1_to_host(struct kvm_vcpu *vcpu, u64 l1_tsc)
 	return host_tsc;
 }
 
-void vmx_set_guest_tsc_deadline_virt(struct kvm_vcpu *vcpu,
-				     u64 guest_deadline_virt)
-{
-	vmcs_write64(GUEST_DEADLINE_VIR, guest_deadline_virt);
-	vmcs_write64(GUEST_DEADLINE_PHY,
-		     vmx_calc_deadline_l1_to_host(vcpu, guest_deadline_virt));
-}
-
-u64 vmx_get_guest_tsc_deadline_virt(struct kvm_vcpu *vcpu)
-{
-	return vmcs_read64(GUEST_DEADLINE_VIR);
-}
-#endif
-
 void vmx_update_cpu_dirty_logging(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h
index 9a61f6bd8cc0..6f1caf3ef5b7 100644
--- a/arch/x86/kvm/vmx/vmx.h
+++ b/arch/x86/kvm/vmx/vmx.h
@@ -394,6 +394,8 @@ static inline void vmx_enable_intercept_for_msr(struct kvm_vcpu *vcpu,
 u64 vmx_get_l2_tsc_offset(struct kvm_vcpu *vcpu);
 u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu);
 
+u64 vmx_calc_deadline_l1_to_host(struct kvm_vcpu *vcpu, u64 l1_tsc);
+
 gva_t vmx_get_untagged_addr(struct kvm_vcpu *vcpu, gva_t gva, unsigned int flags);
 
 void vmx_update_cpu_dirty_logging(struct kvm_vcpu *vcpu);
-- 
2.45.2

[PATCH v2 16/36] KVM: nVMX: Enable guest deadline and its shadow VMCS field

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Support the guest deadline and the guest deadline shadow VMCS field.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
Changes:
v1 -> v2:
- the offsets of vmcs fields changed due to rebase.
---
 arch/x86/kvm/vmx/nested.c | 48 +++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmcs12.c |  2 ++
 arch/x86/kvm/vmx/vmcs12.h |  5 ++++
 arch/x86/kvm/vmx/vmx.h    |  2 ++
 4 files changed, 57 insertions(+)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index b7561f8f4565..3f1318736205 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -2768,6 +2768,22 @@ static void prepare_vmcs02_rare(struct vcpu_vmx *vmx, struct vmcs12 *vmcs12)
 	set_cr4_guest_host_mask(vmx);
 }
 
+static void nested_guest_apic_timer(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12)
+{
+	u64 guest_deadline_shadow = vmcs12->guest_deadline_shadow;
+	u64 guest_deadline = vmcs12->guest_deadline;
+	struct vcpu_vmx *vmx = to_vmx(vcpu);
+
+	if (!vmx->nested.guest_deadline_dirty)
+		return;
+
+	guest_deadline = vmx_calc_deadline_l1_to_host(vcpu, guest_deadline);
+
+	vmcs_write64(GUEST_DEADLINE_PHY, guest_deadline);
+	vmcs_write64(GUEST_DEADLINE_VIR, guest_deadline_shadow);
+	vmx->nested.guest_deadline_dirty = false;
+}
+
 /*
  * prepare_vmcs02 is called when the L1 guest hypervisor runs its nested
  * L2 guest. L1 has a vmcs for L2 (vmcs12), and this function "merges" it
@@ -2845,6 +2861,9 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,
 	if (kvm_caps.has_tsc_control)
 		vmcs_write64(TSC_MULTIPLIER, vcpu->arch.tsc_scaling_ratio);
 
+	if (nested_cpu_has_guest_apic_timer(vmcs12))
+		nested_guest_apic_timer(vcpu, vmcs12);
+
 	nested_vmx_transition_tlb_flush(vcpu, vmcs12, true);
 
 	if (nested_cpu_has_ept(vmcs12))
@@ -4635,6 +4654,8 @@ static bool is_vmcs12_ext_field(unsigned long field)
 	case GUEST_IDTR_BASE:
 	case GUEST_PENDING_DBG_EXCEPTIONS:
 	case GUEST_BNDCFGS:
+	case GUEST_DEADLINE_PHY:
+	case GUEST_DEADLINE_VIR:
 		return true;
 	default:
 		break;
@@ -4685,6 +4706,24 @@ static void sync_vmcs02_to_vmcs12_rare(struct kvm_vcpu *vcpu,
 	vmcs12->guest_pending_dbg_exceptions =
 		vmcs_readl(GUEST_PENDING_DBG_EXCEPTIONS);
 
+	if (nested_cpu_has_guest_apic_timer(vmcs12)) {
+		u64 guest_deadline_shadow = vmcs_read64(GUEST_DEADLINE_VIR);
+		u64 guest_deadline = vmcs_read64(GUEST_DEADLINE_PHY);
+
+		if (guest_deadline) {
+			guest_deadline = kvm_read_l1_tsc(vcpu, guest_deadline);
+			if (!guest_deadline)
+				guest_deadline = 1;
+		}
+
+		vmcs12->guest_deadline = guest_deadline;
+		vmcs12->guest_deadline_shadow = guest_deadline_shadow;
+	} else if (vmx->nested.msrs.tertiary_ctls & TERTIARY_EXEC_GUEST_APIC_TIMER) {
+		vmcs12->guest_deadline = 0;
+		vmcs12->guest_deadline_shadow = 0;
+	}
+	vmx->nested.guest_deadline_dirty = false;
+
 	vmx->nested.need_sync_vmcs02_to_vmcs12_rare = false;
 }
 
@@ -5933,6 +5972,13 @@ static int handle_vmwrite(struct kvm_vcpu *vcpu)
 		vmx->nested.dirty_vmcs12 = true;
 	}
 
+	if (!is_guest_mode(vcpu) &&
+	    (field == GUEST_DEADLINE_PHY ||
+	     field == GUEST_DEADLINE_PHY_HIGH ||
+	     field == GUEST_DEADLINE_VIR ||
+	     field == GUEST_DEADLINE_VIR_HIGH))
+		vmx->nested.guest_deadline_dirty = true;
+
 	return nested_vmx_succeed(vcpu);
 }
 
@@ -5947,6 +5993,7 @@ static void set_current_vmptr(struct vcpu_vmx *vmx, gpa_t vmptr)
 	}
 	vmx->nested.dirty_vmcs12 = true;
 	vmx->nested.force_msr_bitmap_recalc = true;
+	vmx->nested.guest_deadline_dirty = true;
 }
 
 /* Emulate the VMPTRLD instruction */
@@ -7124,6 +7171,7 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu,
 
 	vmx->nested.dirty_vmcs12 = true;
 	vmx->nested.force_msr_bitmap_recalc = true;
+	vmx->nested.guest_deadline_dirty = true;
 	ret = nested_vmx_enter_non_root_mode(vcpu, false);
 	if (ret)
 		goto error_guest_mode;
diff --git a/arch/x86/kvm/vmx/vmcs12.c b/arch/x86/kvm/vmx/vmcs12.c
index dac796fc20f2..d2098767dd7a 100644
--- a/arch/x86/kvm/vmx/vmcs12.c
+++ b/arch/x86/kvm/vmx/vmcs12.c
@@ -70,6 +70,8 @@ static const u16 kvm_supported_vmcs12_field_offsets[] __initconst = {
 	FIELD64(HOST_IA32_PAT, host_ia32_pat),
 	FIELD64(HOST_IA32_EFER, host_ia32_efer),
 	FIELD64(HOST_IA32_PERF_GLOBAL_CTRL, host_ia32_perf_global_ctrl),
+	FIELD64(GUEST_DEADLINE_PHY, guest_deadline),
+	FIELD64(GUEST_DEADLINE_VIR, guest_deadline_shadow),
 	FIELD(PIN_BASED_VM_EXEC_CONTROL, pin_based_vm_exec_control),
 	FIELD(CPU_BASED_VM_EXEC_CONTROL, cpu_based_vm_exec_control),
 	FIELD(EXCEPTION_BITMAP, exception_bitmap),
diff --git a/arch/x86/kvm/vmx/vmcs12.h b/arch/x86/kvm/vmx/vmcs12.h
index 7a20d6661da0..0846033b6759 100644
--- a/arch/x86/kvm/vmx/vmcs12.h
+++ b/arch/x86/kvm/vmx/vmcs12.h
@@ -192,6 +192,9 @@ struct __packed vmcs12 {
 	u16 host_tr_selector;
 	u16 guest_pml_index;
 	u16 virtual_timer_vector;
+
+	u64 guest_deadline;
+	u64 guest_deadline_shadow;
 };
 
 /*
@@ -376,6 +379,8 @@ static inline void vmx_check_vmcs12_offsets(void)
 	CHECK_OFFSET(host_tr_selector, 994);
 	CHECK_OFFSET(guest_pml_index, 996);
 	CHECK_OFFSET(virtual_timer_vector, 998);
+	CHECK_OFFSET(guest_deadline, 1000);
+	CHECK_OFFSET(guest_deadline_shadow, 1008);
 }
 
 extern u16 vmcs12_field_offsets[] __ro_after_init;
diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h
index 6f1caf3ef5b7..fbbbae46e0b2 100644
--- a/arch/x86/kvm/vmx/vmx.h
+++ b/arch/x86/kvm/vmx/vmx.h
@@ -163,6 +163,8 @@ struct nested_vmx {
 	bool has_preemption_timer_deadline;
 	bool preemption_timer_expired;
 
+	bool guest_deadline_dirty;
+
 	/*
 	 * Used to snapshot MSRs that are conditionally loaded on VM-Enter in
 	 * order to propagate the guest's pre-VM-Enter value into vmcs02.  For
-- 
2.45.2

[PATCH v2 17/36] KVM: nVMX: Add VM entry checks related to APIC timer virtualization

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Add checks of VMX controls for APIC timer virtualization on VM entries.

The spec adds some checks on VMX controls related to the APIC timer
virtualization on VM entry.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/vmx/nested.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 3f1318736205..65f7260d02df 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -3069,6 +3069,12 @@ static int nested_check_vm_execution_controls(struct kvm_vcpu *vcpu,
 	    CC(!vmcs12->tsc_multiplier))
 		return -EINVAL;
 
+	if (nested_cpu_has_guest_apic_timer(vmcs12) &&
+	    (CC(!nested_cpu_has_vid(vmcs12)) ||
+	     CC(nested_cpu_has(vmcs12, CPU_BASED_RDTSC_EXITING)) ||
+	     CC(vmcs12->virtual_timer_vector > 255)))
+		return -EINVAL;
+
 	return 0;
 }
 
-- 
2.45.2

[PATCH v2 18/36] KVM: nVMX: Add check vmread/vmwrite on tertiary control

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Make the access to the tertiary processor-based VM control an error if the
guest VMX true processor-based controls don't report it.

Without this patch, the KVM unit test_vmread_vmwrite() fails because
vmread()/vmwrite() can succeeds with the index beyond
MSR_IA32_VMX_VMCS_ENUM when the tertiary processor-based VM-executing
controls aren't advertised to the guest.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/vmx/nested.c | 20 ++++++++++++++++++++
 arch/x86/kvm/vmx/nested.h |  5 +++++
 2 files changed, 25 insertions(+)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 65f7260d02df..562b5ffc6433 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -5768,6 +5768,16 @@ static int handle_vmresume(struct kvm_vcpu *vcpu)
 	return nested_vmx_run(vcpu, false);
 }
 
+static bool is_vmcs_field_valid(struct kvm_vcpu *vcpu, unsigned long field)
+{
+	if (!nested_cpu_supports_tertiary_ctls(vcpu) &&
+	    (field == TERTIARY_VM_EXEC_CONTROL ||
+	     field == TERTIARY_VM_EXEC_CONTROL_HIGH))
+		return false;
+
+	return true;
+}
+
 static int handle_vmread(struct kvm_vcpu *vcpu)
 {
 	struct vmcs12 *vmcs12 = is_guest_mode(vcpu) ? get_shadow_vmcs12(vcpu)
@@ -5798,6 +5808,9 @@ static int handle_vmread(struct kvm_vcpu *vcpu)
 		     get_vmcs12(vcpu)->vmcs_link_pointer == INVALID_GPA))
 			return nested_vmx_failInvalid(vcpu);
 
+		if (!is_vmcs_field_valid(vcpu, field))
+			return nested_vmx_fail(vcpu, VMXERR_UNSUPPORTED_VMCS_COMPONENT);
+
 		offset = get_vmcs12_field_offset(field);
 		if (offset < 0)
 			return nested_vmx_fail(vcpu, VMXERR_UNSUPPORTED_VMCS_COMPONENT);
@@ -5922,6 +5935,9 @@ static int handle_vmwrite(struct kvm_vcpu *vcpu)
 
 	field = kvm_register_read(vcpu, (((instr_info) >> 28) & 0xf));
 
+	if (!is_vmcs_field_valid(vcpu, field))
+		return nested_vmx_fail(vcpu, VMXERR_UNSUPPORTED_VMCS_COMPONENT);
+
 	offset = get_vmcs12_field_offset(field);
 	if (offset < 0)
 		return nested_vmx_fail(vcpu, VMXERR_UNSUPPORTED_VMCS_COMPONENT);
@@ -7170,6 +7186,10 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu,
 			kvm_state->hdr.vmx.preemption_timer_deadline;
 	}
 
+	if (!nested_cpu_supports_tertiary_ctls(vcpu) &&
+	    vmcs12->tertiary_vm_exec_control)
+		goto error_guest_mode;
+
 	if (nested_vmx_check_controls(vcpu, vmcs12) ||
 	    nested_vmx_check_host_state(vcpu, vmcs12) ||
 	    nested_vmx_check_guest_state(vcpu, vmcs12, &ignored))
diff --git a/arch/x86/kvm/vmx/nested.h b/arch/x86/kvm/vmx/nested.h
index 6df2cfb20d87..1100a8114dd9 100644
--- a/arch/x86/kvm/vmx/nested.h
+++ b/arch/x86/kvm/vmx/nested.h
@@ -157,6 +157,11 @@ static inline bool __nested_cpu_supports_tertiary_ctls(struct nested_vmx_msrs *m
 	return msrs->procbased_ctls_high & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
 }
 
+static inline bool nested_cpu_supports_tertiary_ctls(struct kvm_vcpu *vcpu)
+{
+	return __nested_cpu_supports_tertiary_ctls(&to_vmx(vcpu)->nested.msrs);
+}
+
 /* APIC TIMER VIRTUALIZATION requires in-kernel lapic. */
 static inline bool nested_cpu_can_support_apic_virt_timer(struct kvm_vcpu *vcpu)
 {
-- 
2.45.2

[PATCH v2 19/36] KVM: nVMX: Add check VMCS index for guest timer virtualization

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Make vmread/vmwrite to the VMCS fields an error if the guest
MSR_IA32_VMX_PROCBASED_CTLS3 doesn't advertise APIC timer virtualization.
Without this check, test_vmwrite_vmread of the KVM unit test fails.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/vmx/nested.c | 14 ++++++++++++++
 arch/x86/kvm/vmx/nested.h |  7 +++++++
 2 files changed, 21 insertions(+)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 562b5ffc6433..3cd29b005afe 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -5775,6 +5775,14 @@ static bool is_vmcs_field_valid(struct kvm_vcpu *vcpu, unsigned long field)
 	     field == TERTIARY_VM_EXEC_CONTROL_HIGH))
 		return false;
 
+	if (!nested_cpu_supports_guest_apic_timer(vcpu) &&
+	    (field == GUEST_APIC_TIMER_VECTOR ||
+	     field == GUEST_DEADLINE_VIR ||
+	     field == GUEST_DEADLINE_VIR_HIGH ||
+	     field == GUEST_DEADLINE_PHY ||
+	     field == GUEST_DEADLINE_PHY_HIGH))
+		return false;
+
 	return true;
 }
 
@@ -7190,6 +7198,12 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu,
 	    vmcs12->tertiary_vm_exec_control)
 		goto error_guest_mode;
 
+	if (!nested_cpu_supports_guest_apic_timer(vcpu) &&
+	    (vmcs12->virtual_timer_vector ||
+	     vmcs12->guest_deadline ||
+	     vmcs12->guest_deadline_shadow))
+		goto error_guest_mode;
+
 	if (nested_vmx_check_controls(vcpu, vmcs12) ||
 	    nested_vmx_check_host_state(vcpu, vmcs12) ||
 	    nested_vmx_check_guest_state(vcpu, vmcs12, &ignored))
diff --git a/arch/x86/kvm/vmx/nested.h b/arch/x86/kvm/vmx/nested.h
index 1100a8114dd9..d5d624150aca 100644
--- a/arch/x86/kvm/vmx/nested.h
+++ b/arch/x86/kvm/vmx/nested.h
@@ -291,6 +291,13 @@ static inline bool nested_cpu_has_encls_exit(struct vmcs12 *vmcs12)
 	return nested_cpu_has2(vmcs12, SECONDARY_EXEC_ENCLS_EXITING);
 }
 
+static inline bool nested_cpu_supports_guest_apic_timer(struct kvm_vcpu *vcpu)
+{
+	return nested_cpu_supports_tertiary_ctls(vcpu) &&
+		to_vmx(vcpu)->nested.msrs.tertiary_ctls &
+		TERTIARY_EXEC_GUEST_APIC_TIMER;
+}
+
 static inline bool nested_cpu_has_guest_apic_timer(struct vmcs12 *vmcs12)
 {
 	return nested_cpu_has3(vmcs12, TERTIARY_EXEC_GUEST_APIC_TIMER);
-- 
2.45.2

[PATCH v2 20/36] KVM: VMX: Advertise tertiary controls to the user space

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Make KVM_GET_MSR_INDEX_LIST, KVM_GET_MSR_FEATURE_INDEX_LIST, KVM_GET_MSRS
to advertise MSR_IA32_VMX_PROCBASED_CTLS3 to the user space VMM like QEMU.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/vmx/vmx.c | 3 +++
 arch/x86/kvm/x86.c     | 1 +
 2 files changed, 4 insertions(+)

diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index bac3f33f7d73..8c2ab0164714 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -7214,6 +7214,9 @@ bool vmx_has_emulated_msr(struct kvm *kvm, u32 index)
 		 */
 		return enable_unrestricted_guest || emulate_invalid_guest_state;
 	case KVM_FIRST_EMULATED_VMX_MSR ... KVM_LAST_EMULATED_VMX_MSR:
+		if (index == MSR_IA32_VMX_PROCBASED_CTLS3 &&
+		    !__nested_cpu_supports_tertiary_ctls(&vmcs_config.nested))
+			return false;
 		return nested;
 	case MSR_AMD64_VIRT_SPEC_CTRL:
 	case MSR_AMD64_TSC_RATIO:
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 1922e8699101..8a1ec0a39624 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -457,6 +457,7 @@ static const u32 emulated_msrs_all[] = {
 	MSR_IA32_VMX_PROCBASED_CTLS2,
 	MSR_IA32_VMX_EPT_VPID_CAP,
 	MSR_IA32_VMX_VMFUNC,
+	MSR_IA32_VMX_PROCBASED_CTLS3,
 
 	MSR_K7_HWCR,
 	MSR_KVM_POLL_CONTROL,
-- 
2.45.2

[PATCH v2 21/36] KVM: VMX: dump_vmcs() support the guest virt timer

[isaku.yamahata]

From: Yang Zhong <yang.zhong@linux.intel.com>

Add three VMCS fields from the guest virt timer into dump_vmcs().

Signed-off-by: Yang Zhong <yang.zhong@linux.intel.com>
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/vmx/vmx.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 8c2ab0164714..e665fc7f3377 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -6594,6 +6594,12 @@ void dump_vmcs(struct kvm_vcpu *vcpu)
 		vmx_dump_msrs("guest autoload", &vmx->msr_autoload.guest);
 	if (vmcs_read32(VM_EXIT_MSR_STORE_COUNT) > 0)
 		vmx_dump_msrs("autostore", &vmx->msr_autostore);
+	if (tertiary_exec_control & TERTIARY_EXEC_GUEST_APIC_TIMER) {
+		pr_err("DeadlinePhy = 0x%016llx\n", vmcs_read64(GUEST_DEADLINE_PHY));
+		pr_err("DeadlineVir = 0x%016llx\n", vmcs_read64(GUEST_DEADLINE_VIR));
+		pr_err("GuestApicTimerVector = 0x%04x\n",
+		       vmcs_read16(GUEST_APIC_TIMER_VECTOR));
+	}
 
 	if (vmentry_ctl & VM_ENTRY_LOAD_CET_STATE)
 		pr_err("S_CET = 0x%016lx, SSP = 0x%016lx, SSP TABLE = 0x%016lx\n",
-- 
2.45.2

[PATCH v2 22/36] KVM: VMX: Enable APIC timer virtualization

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Don't clear TERTIARY_EXEC_GUEST_APIC_TIMER bit unconditionally from
KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL when calculating supported bit.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
Changes:
v1 -> v2:
- Move the bit disable to the caller of adjust_vmx_controls64() to avoid
  compile time error because KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL is
  used for compile time check.
---
 arch/x86/kvm/vmx/vmx.c | 10 +---------
 1 file changed, 1 insertion(+), 9 deletions(-)

diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index e665fc7f3377..b3974125a902 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -2801,15 +2801,7 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 
 	if (_cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS)
 		_cpu_based_3rd_exec_control =
-			adjust_vmx_controls64(KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL
-					      /*
-					       * Disable apic timer
-					       * virtualization until the logic
-					       * is imlemented.
-					       * Once it's supported, add
-					       * TERTIARY_EXEC_GUEST_APIC_TIMER.
-					       */
-					      & ~TERTIARY_EXEC_GUEST_APIC_TIMER,
+			adjust_vmx_controls64(KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL,
 					      MSR_IA32_VMX_PROCBASED_CTLS3);
 
 	if (!IS_ENABLED(CONFIG_X86_64) ||
-- 
2.45.2

[PATCH v2 23/36] KVM: VMX: Introduce module parameter for APIC virt timer support

[isaku.yamahata]

From: Yang Zhong <yang.zhong@linux.intel.com>

Introduce a new module parameter, apic_timer_virt, to control the
virtualization of the APIC timer in KVM. The primary improvement offered by
APIC timer virtualization over the preemption timer is the passthrough of
MSR_IA32_TSC_DEADLINE to the VM. This passthrough capability reduces the
number of VM exits triggered by MSR write operations, thereby enhancing the
performance of the virtual machine.

The apic_timer_virt parameter is set to true by default on processor
platforms that support APIC timer virtualization.  On platforms that do not
support this feature, the parameter will indicate that APIC timer
virtualization is not available.

Signed-off-by: Yang Zhong <yang.zhong@linux.intel.com>
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/vmx/vmx.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index b3974125a902..0271514162df 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -119,6 +119,9 @@ module_param(enable_ipiv, bool, 0444);
 
 module_param(enable_device_posted_irqs, bool, 0444);
 
+static bool __read_mostly enable_apic_timer_virt = true;
+module_param_named(apic_timer_virt, enable_apic_timer_virt, bool, 0444);
+
 /*
  * If nested=1, nested virtualization is supported, i.e., guests may use
  * VMX and be a hypervisor for its own guests. If nested=0, guests may not
@@ -2804,7 +2807,7 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 			adjust_vmx_controls64(KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL,
 					      MSR_IA32_VMX_PROCBASED_CTLS3);
 
-	if (!IS_ENABLED(CONFIG_X86_64) ||
+	if (!IS_ENABLED(CONFIG_X86_64) || !enable_apic_timer_virt ||
 	    !(_cpu_based_2nd_exec_control & SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY))
 		_cpu_based_3rd_exec_control &= ~TERTIARY_EXEC_GUEST_APIC_TIMER;
 
@@ -8835,6 +8838,9 @@ __init int vmx_hardware_setup(void)
 	if (!cpu_has_vmx_preemption_timer())
 		enable_preemption_timer = false;
 
+	if (!cpu_has_vmx_apic_timer_virt())
+		enable_apic_timer_virt = false;
+
 	if (enable_preemption_timer) {
 		u64 use_timer_freq = 5000ULL * 1000 * 1000;
 
-- 
2.45.2

[PATCH v2 24/36] KVM: nVMX: Introduce module parameter for nested APIC timer virtualization

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Introduce a new module parameter, nested_apic_timer_virt, to control the
nested virtualization of the APIC timer in KVM.

The nested_apic_timer_virt parameter is set to true by default on processor
platforms that support APIC timer virtualization.  On platforms that do not
support this feature, the parameter will indicate that APIC timer
virtualization is not available.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
Changes:
v1 -> v2:
- use 0444 instead of S_IRUGO.
---
 arch/x86/kvm/vmx/capabilities.h |  1 +
 arch/x86/kvm/vmx/nested.c       | 13 ++++++++++++-
 arch/x86/kvm/vmx/vmx.c          |  2 +-
 3 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilities.h
index 8d67be77f02c..861334e15c01 100644
--- a/arch/x86/kvm/vmx/capabilities.h
+++ b/arch/x86/kvm/vmx/capabilities.h
@@ -15,6 +15,7 @@ extern bool __read_mostly enable_ept;
 extern bool __read_mostly enable_unrestricted_guest;
 extern bool __read_mostly enable_ept_ad_bits;
 extern bool __read_mostly enable_pml;
+extern bool __read_mostly enable_apic_timer_virt;
 extern int __read_mostly pt_mode;
 
 #define PT_MODE_SYSTEM		0
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 3cd29b005afe..60c7256298ce 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -27,6 +27,10 @@ module_param_named(enable_shadow_vmcs, enable_shadow_vmcs, bool, S_IRUGO);
 static bool __ro_after_init warn_on_missed_cc;
 module_param(warn_on_missed_cc, bool, 0444);
 
+static bool __read_mostly enable_nested_apic_timer_virt = true;
+module_param_named(nested_apic_timer_virt, enable_nested_apic_timer_virt, bool,
+		   0444);
+
 #define CC KVM_NESTED_VMENTER_CONSISTENCY_CHECK
 
 /*
@@ -7453,13 +7457,20 @@ static void nested_vmx_setup_secondary_ctls(u32 ept_caps,
 static void nested_vmx_setup_tertiary_ctls(struct vmcs_config *vmcs_conf,
 					   struct nested_vmx_msrs *msrs)
 {
-	msrs->tertiary_ctls = vmcs_conf->cpu_based_3rd_exec_ctrl;
+	enable_nested_apic_timer_virt &= enable_apic_timer_virt;
 
+	msrs->tertiary_ctls = vmcs_conf->cpu_based_3rd_exec_ctrl;
 	msrs->tertiary_ctls &= TERTIARY_EXEC_GUEST_APIC_TIMER;
 
+	if (!enable_nested_apic_timer_virt)
+		msrs->tertiary_ctls &= ~TERTIARY_EXEC_GUEST_APIC_TIMER;
+
 	if (msrs->tertiary_ctls)
 		msrs->procbased_ctls_high |=
 			CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+
+	if (!(msrs->tertiary_ctls & TERTIARY_EXEC_GUEST_APIC_TIMER))
+		enable_nested_apic_timer_virt = false;
 }
 
 static void nested_vmx_setup_misc_data(struct vmcs_config *vmcs_conf,
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 0271514162df..4d5414af750b 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -119,7 +119,7 @@ module_param(enable_ipiv, bool, 0444);
 
 module_param(enable_device_posted_irqs, bool, 0444);
 
-static bool __read_mostly enable_apic_timer_virt = true;
+bool __read_mostly enable_apic_timer_virt = true;
 module_param_named(apic_timer_virt, enable_apic_timer_virt, bool, 0444);
 
 /*
-- 
2.45.2

[PATCH v2 25/36] KVM: selftests: Add a test to measure local timer latency

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

A test case to measure local timer latency and to confirm that VMExit
doesn't happen on TSC DEADLINE MSR on the platform that supports VMX APIC
timer virtualization.  Or VMExit happens on the platform without the
feature.

This is inspired by kvm-unit-test x86/tscdeadline_latency.c.  The original
test records all latency, but this records only the max/min/avg of the
latency for simplicity.  It sets the local APIC timer (APIC oneshot or TSC
deadline) and the timer interrupt handler records the delay from the timer
value.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
Changes:
v1 -> v2:
- format fix in print_exit_stats() and help().
- static char * => static const char * const for checkpatch.
- use safe_halt(), sti(), cli() instead of inline assembly.
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../testing/selftests/kvm/x86/timer_latency.c | 578 ++++++++++++++++++
 2 files changed, 579 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/timer_latency.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index ba87cd31872b..3c2a5470c2ca 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -145,6 +145,7 @@ TEST_GEN_PROGS_x86 += x86/max_vcpuid_cap_test
 TEST_GEN_PROGS_x86 += x86/triple_fault_event_test
 TEST_GEN_PROGS_x86 += x86/recalc_apic_map_test
 TEST_GEN_PROGS_x86 += x86/aperfmperf_test
+TEST_GEN_PROGS_x86 += x86/timer_latency
 TEST_GEN_PROGS_x86 += access_tracking_perf_test
 TEST_GEN_PROGS_x86 += coalesced_io_test
 TEST_GEN_PROGS_x86 += dirty_log_perf_test
diff --git a/tools/testing/selftests/kvm/x86/timer_latency.c b/tools/testing/selftests/kvm/x86/timer_latency.c
new file mode 100644
index 000000000000..7c2e8225f299
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/timer_latency.c
@@ -0,0 +1,578 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2026 Intel Corporation
+ *
+ * Measure timer interrupt latency between time set to the local timer and
+ * interrupt arrival time.  Optionally print out max/min/avg of the latency.
+ */
+
+#include <stdio.h>
+#include <string.h>
+#include <stdatomic.h>
+#include <signal.h>
+#include <pthread.h>
+
+#include "kvm_util.h"
+#include "processor.h"
+#include "apic.h"
+
+#define LOCAL_TIMER_VECTOR	0xec
+
+#define TEST_DURATION_DEFAULT_IN_SEC   10
+
+/* Random number in ns, appropriate for timer interrupt */
+#define DEFAULT_TIMER_INC_NS	10000
+
+/* Twice 100Hz scheduler tick for nested virtualization. */
+#define DEFAULT_ALLOWED_TIMER_LATENCY_NS	(20 * 1000 * 1000)
+
+struct options {
+	bool use_oneshot_timer;
+	bool use_x2apic;
+	bool use_poll;
+
+	uint64_t timer_inc_ns;
+	uint64_t allowed_timer_latency_ns;
+
+	bool print_result;
+};
+
+static struct options options = {
+	.use_x2apic = true,
+	.timer_inc_ns = DEFAULT_TIMER_INC_NS,
+	.allowed_timer_latency_ns = DEFAULT_ALLOWED_TIMER_LATENCY_NS,
+};
+
+enum event_type {
+	EVENT_TIMER_HANDLER,
+	EVENT_HLT_WAKEUP,
+	EVENT_MAX,
+};
+
+struct test_sample {
+	uint64_t time_stamp;
+	enum event_type etype;
+	uint32_t latency;
+};
+
+struct test_latency_stat {
+	uint64_t sum;
+	uint64_t count;
+	uint32_t min;
+	uint32_t max;
+};
+
+struct test_shared_data {
+	atomic_bool stop_test;
+	atomic_bool terminated;
+	uint64_t tsc_khz;
+	uint64_t apic_bus_cycle_ns;
+	uint64_t allowed_timer_latency_tsc;
+
+	uint64_t timer_inc;
+
+	uint64_t hlt_count;
+	uint64_t timer_interrupt_set;
+	uint64_t timer_interrupt_received;
+
+	struct test_latency_stat latency_stat[EVENT_MAX];
+};
+
+#define GUEST_ASSERT_LATENCY(latency_tsc)				\
+	__GUEST_ASSERT((latency_tsc) <= data->allowed_timer_latency_tsc, \
+		       "too large timer latency %ld ns "		\
+		       "(requires %ld ns) %ld khz tsc",			\
+		       tsc_to_ns(data, latency_tsc),			\
+		       options.allowed_timer_latency_ns,		\
+		       data->tsc_khz)
+
+static struct test_shared_data shared_data;
+
+static u64 tsc_to_ns(struct test_shared_data *data, u64 tsc_delta)
+{
+	return tsc_delta * NSEC_PER_SEC / (data->tsc_khz * 1000);
+}
+
+static u64 ns_to_tsc(struct test_shared_data *data, u64 ns)
+{
+	return ns * (data->tsc_khz * 1000) / NSEC_PER_SEC;
+}
+
+static void latency_init(struct test_latency_stat *stat)
+{
+	stat->sum = 0;
+	stat->count = 0;
+	stat->min = -1;
+	stat->max = 0;
+}
+
+static void shared_data_init(struct test_shared_data *data)
+{
+	int i;
+
+	memset(data, 0, sizeof(*data));
+
+	for (i = 0; i < ARRAY_SIZE(data->latency_stat); i++)
+		latency_init(data->latency_stat + i);
+}
+
+static void stop_test(struct kvm_vm *vm, struct test_shared_data *data)
+{
+	atomic_store(&data->stop_test, true);
+	sync_global_to_guest(vm, data->stop_test);
+}
+
+static void guest_apic_enable(void)
+{
+	if (options.use_x2apic)
+		x2apic_enable();
+	else
+		xapic_enable();
+}
+
+static void guest_apic_write_reg(unsigned int reg, uint64_t val)
+{
+	if (options.use_x2apic)
+		x2apic_write_reg(reg, val);
+	else
+		xapic_write_reg(reg, val);
+}
+
+static void record_sample(struct test_shared_data *data, enum event_type etype,
+			 uint64_t ts, uint64_t latency)
+{
+	struct test_latency_stat *stat;
+
+	stat = &data->latency_stat[etype];
+
+	stat->count++;
+	stat->sum += latency;
+
+	if (stat->min > latency)
+		stat->min = latency;
+	if (stat->max < latency)
+		stat->max = latency;
+
+	if (etype == EVENT_TIMER_HANDLER &&
+	    latency > data->allowed_timer_latency_tsc) {
+		if (options.use_poll) {
+			GUEST_PRINTF("latency is too high %ld ns (> %ld ns)\n",
+				     tsc_to_ns(data, latency),
+				     options.allowed_timer_latency_ns);
+		} else
+			GUEST_ASSERT_LATENCY(latency);
+	}
+}
+
+static atomic_bool timer_interrupted;
+static atomic_uint_fast64_t timer_tsc;
+
+static inline bool tsc_before(u64 a, u64 b)
+{
+	return (s64)(a - b) < 0;
+}
+
+static void guest_timer_interrupt_handler(struct ex_regs *regs)
+{
+	uint64_t now = rdtsc();
+	uint64_t timer_tsc__ = atomic_load(&timer_tsc);
+
+	__GUEST_ASSERT(!atomic_load(&timer_interrupted),
+		       "timer handler is called multiple times per timer");
+	__GUEST_ASSERT(tsc_before(timer_tsc__, now),
+		       "timer is fired before armed time timer_tsc 0x%lx now 0x%lx",
+		       timer_tsc__, now);
+
+	record_sample(&shared_data, EVENT_TIMER_HANDLER, now, now - timer_tsc__);
+
+	shared_data.timer_interrupt_received++;
+	atomic_store(&timer_interrupted, true);
+	guest_apic_write_reg(APIC_EOI, 0);
+}
+
+static void __set_timer(struct test_shared_data *data,
+			uint64_t next_tsc, uint64_t apic_inc)
+{
+	if (options.use_oneshot_timer)
+		guest_apic_write_reg(APIC_TMICT, apic_inc);
+	else
+		wrmsr(MSR_IA32_TSC_DEADLINE, next_tsc);
+}
+
+static void set_timer(struct test_shared_data *data,
+		      uint64_t next_tsc, uint64_t apic_inc)
+{
+	atomic_store(&timer_tsc, next_tsc);
+	data->timer_interrupt_set++;
+	__set_timer(data, next_tsc, apic_inc);
+}
+
+static u64 to_apic_bus_cycle(struct test_shared_data *data, u64 tsc_delta)
+{
+	u64 ret;
+
+	if (!tsc_delta)
+		return 0;
+
+	ret = tsc_to_ns(data, tsc_delta) / data->apic_bus_cycle_ns;
+	if (!ret)
+		ret++;
+
+	return ret;
+}
+
+static void hlt_loop(struct test_shared_data *data)
+{
+	uint64_t inc, now, prev_tsc, next_tsc;
+
+	cli();
+	guest_apic_enable();
+
+	inc = data->timer_inc;
+
+	/* DIVISOR = 1 for oneshot timer case */
+	guest_apic_write_reg(APIC_TDCR, 0xb);
+	guest_apic_write_reg(APIC_LVTT,
+			     (options.use_oneshot_timer ?
+			      APIC_LVT_TIMER_ONESHOT :
+			      APIC_LVT_TIMER_TSCDEADLINE) |
+			     LOCAL_TIMER_VECTOR);
+
+	next_tsc = rdtsc() + inc;
+	if (!next_tsc)
+		next_tsc++;
+	atomic_store(&timer_interrupted, false);
+	set_timer(data, next_tsc, to_apic_bus_cycle(data, inc));
+
+	while (!atomic_load(&data->stop_test)) {
+		prev_tsc = rdtsc();
+
+		if (options.use_poll) {
+			sti();
+			while (!atomic_load(&timer_interrupted) &&
+			       rdtsc() < next_tsc + data->allowed_timer_latency_tsc)
+				cpu_relax();
+			cli();
+		} else {
+			/* "sti; hlt; cli" */
+			safe_halt();
+			cli();
+		}
+
+		now = rdtsc();
+
+		record_sample(data, EVENT_HLT_WAKEUP, now, now - prev_tsc);
+		data->hlt_count++;
+
+		if (atomic_load(&timer_interrupted)) {
+			while (next_tsc <= now)
+				next_tsc += inc;
+			if (!next_tsc)
+				next_tsc++;
+
+			atomic_store(&timer_interrupted, false);
+			set_timer(data, next_tsc,
+				  to_apic_bus_cycle(data, next_tsc - now));
+		} else {
+			uint64_t latency = now - next_tsc;
+
+			GUEST_ASSERT_LATENCY(latency);
+		}
+	}
+
+	/* Wait for the interrupt to arrive. */
+	now = rdtsc();
+	next_tsc = now + inc * 2;
+	sti();
+	while (now < next_tsc || !atomic_load(&timer_interrupted)) {
+		cpu_relax();
+		now = rdtsc();
+	}
+	cli();
+
+	/* Stop timer explicitly just in case. */
+	__set_timer(data, 0, 0);
+}
+
+static void guest_code(void)
+{
+	struct test_shared_data *data = &shared_data;
+
+	hlt_loop(data);
+
+	__GUEST_ASSERT(data->timer_interrupt_set == data->timer_interrupt_received,
+		       "timer interrupt lost set %ld received %ld",
+		       data->timer_interrupt_set, data->timer_interrupt_received);
+
+	GUEST_DONE();
+}
+
+static void __run_vcpu(struct kvm_vcpu *vcpu)
+{
+	struct ucall uc;
+
+	for (;;) {
+		vcpu_run(vcpu);
+
+		TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_IO);
+
+		switch (get_ucall(vcpu, &uc)) {
+		case UCALL_DONE:
+			pr_info("vcpu id %d passed\n", vcpu->id);
+			return;
+		case UCALL_ABORT:
+			REPORT_GUEST_ASSERT(uc);
+			return;
+		case UCALL_PRINTF:
+			pr_info("%s", uc.buffer);
+			continue;
+		default:
+			TEST_FAIL("Unexpected ucall cmd: %ld", uc.cmd);
+			return;
+		}
+
+		return;
+	}
+}
+
+static void *run_vcpu(void *args)
+{
+	struct kvm_vcpu *vcpu = args;
+
+	__run_vcpu(vcpu);
+
+	return NULL;
+}
+
+static void print_result_type(struct test_shared_data *data,
+			      enum event_type etype, const char *event_name)
+{
+	struct test_latency_stat *stat = &data->latency_stat[etype];
+	uint64_t avg = 0;
+
+	if (stat->count)
+		avg = stat->sum / stat->count;
+
+	pr_info("%s latency (%ld samples)\tmin %ld avg %ld max %ld ns\n",
+		event_name, stat->count,
+		tsc_to_ns(data, stat->min), tsc_to_ns(data, avg),
+		tsc_to_ns(data, stat->max));
+}
+
+static void print_result(struct test_shared_data *data)
+{
+	pr_info("guest timer: %s timer period %ld ns\n",
+		options.use_oneshot_timer ?
+		"APIC oneshot timer" : "tsc deadline",
+		options.timer_inc_ns);
+
+	pr_info("tsc_khz %ld apic_bus_cycle_ns %ld\n",
+		data->tsc_khz, data->apic_bus_cycle_ns);
+
+	pr_info("hlt %ld timer set %ld received %ld\n",
+		data->hlt_count,
+		data->timer_interrupt_set, data->timer_interrupt_received);
+
+	print_result_type(data, EVENT_TIMER_HANDLER, "timer interrupt");
+	print_result_type(data, EVENT_HLT_WAKEUP, "halt wakeup");
+}
+
+static void print_exit_stats(struct kvm_vcpu *vcpu)
+{
+	static const char * const stat_name[] = {
+		"exits",
+		"halt_exits",
+		"irq_exits",
+		"inject_tscdeadline"
+	};
+	uint64_t data;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(stat_name); i++) {
+		kvm_get_stat(&vcpu->stats, stat_name[i], &data, 1);
+		pr_info("%s: %ld ", stat_name[i], data);
+	}
+	pr_info("\n");
+}
+
+static void setup_timer_freq(struct kvm_vm *vm,
+			     struct test_shared_data *data)
+{
+	data->tsc_khz = __vm_ioctl(vm, KVM_GET_TSC_KHZ, NULL);
+	TEST_ASSERT(data->tsc_khz > 0, "KVM_GET_TSC_KHZ failed..");
+
+	data->apic_bus_cycle_ns = kvm_check_cap(KVM_CAP_X86_APIC_BUS_CYCLES_NS);
+	if (options.use_oneshot_timer)
+		data->timer_inc = options.timer_inc_ns * data->apic_bus_cycle_ns;
+	else
+		data->timer_inc = ns_to_tsc(data, options.timer_inc_ns);
+
+	data->allowed_timer_latency_tsc =
+		ns_to_tsc(data, options.allowed_timer_latency_ns);
+}
+
+static void setup(struct kvm_vm **vm__, struct kvm_vcpu **vcpu__)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+
+	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+	vm_install_exception_handler(vm, LOCAL_TIMER_VECTOR,
+				     guest_timer_interrupt_handler);
+	setup_timer_freq(vm, &shared_data);
+
+	if (!options.use_oneshot_timer)
+		vcpu_set_cpuid_feature(vcpu, X86_FEATURE_TSC_DEADLINE_TIMER);
+
+	sync_global_to_guest(vm, options);
+	sync_global_to_guest(vm, shared_data);
+
+	*vm__ = vm;
+	*vcpu__ = vcpu;
+}
+
+static void print_stats(struct kvm_vm *vm, struct kvm_vcpu *vcpu)
+{
+	if (options.print_result) {
+		sync_global_from_guest(vm, *&shared_data);
+		print_result(&shared_data);
+		print_exit_stats(vcpu);
+	}
+}
+
+static void sigterm_handler(int signum, siginfo_t *info, void *arg_)
+{
+	atomic_store(&shared_data.terminated, true);
+}
+
+static int run_test(unsigned int duration)
+{
+	struct kvm_vcpu *vcpu;
+	struct sigaction sa;
+	struct kvm_vm *vm;
+	pthread_t thread;
+	int r;
+
+	shared_data_init(&shared_data);
+
+	setup(&vm, &vcpu);
+
+	sa = (struct sigaction) {
+		.sa_sigaction = sigterm_handler,
+	};
+	sigemptyset(&sa.sa_mask);
+	r = sigaction(SIGTERM, &sa, NULL);
+	TEST_ASSERT(!r, "sigaction");
+
+	r = pthread_create(&thread, NULL, run_vcpu, vcpu);
+	TEST_ASSERT(!r, "pthread_create");
+
+	while (duration > 0 && !atomic_load(&shared_data.terminated)) {
+		duration = sleep(duration);
+		TEST_ASSERT(duration >= 0, "sleep");
+	}
+
+	if (atomic_load(&shared_data.terminated)) {
+		pr_info("terminated\n");
+		print_stats(vm, vcpu);
+		return -EINTR;
+	}
+
+	stop_test(vm, &shared_data);
+
+	r = pthread_join(thread, NULL);
+	TEST_ASSERT(!r, "pthread_join");
+
+	print_stats(vm, vcpu);
+
+	kvm_vm_free(vm);
+	return 0;
+}
+
+static void help(const char *name)
+{
+	puts("");
+	printf("usage: %s ", name);
+	printf("[-h] [-l] [-d duration_in_sec] [-a allowed_timer_latency] ");
+	printf("[-p period_in_ns] [-o] [-O] [-x] [-X]\n");
+	puts("");
+	printf("-h: Display this message.");
+	printf("-l: use idle loop instead of hlt\n");
+	printf("-d: specify test to run in second (default %d sec)\n",
+	       TEST_DURATION_DEFAULT_IN_SEC);
+	printf("-p: timer period in ns (default %d nsec)\n",
+	       DEFAULT_TIMER_INC_NS);
+	printf("-a: allowed timer latency in ns (default %d nsec)\n",
+	       DEFAULT_ALLOWED_TIMER_LATENCY_NS);
+	printf("-o: use APIC oneshot timer instead of TSC deadline timer\n");
+	printf("-t: use TSC deadline timer instead of APIC oneshot timer (default)\n");
+	printf("-P: print result stat\n");
+	printf("-x: use xAPIC mode\n");
+	printf("-X: use x2APIC mode (default)\n");
+	puts("");
+
+	exit(EXIT_SUCCESS);
+}
+
+int main(int argc, char **argv)
+{
+	int opt;
+	unsigned int duration = TEST_DURATION_DEFAULT_IN_SEC;
+
+	while ((opt = getopt(argc, argv, "hld:p:a:otxXP")) != -1) {
+		switch (opt) {
+		case 'l':
+			options.use_poll = true;
+			break;
+
+		case 'd':
+			duration = atoi_non_negative("test duration in sec", optarg);
+			break;
+		case 'p':
+			options.timer_inc_ns =
+				atoi_non_negative("timer period in nsec", optarg);
+			break;
+		case 'a':
+			options.allowed_timer_latency_ns =
+				atoi_non_negative("allowed timer latency in nsec",
+						  optarg);
+			break;
+
+
+		case 'x':
+			options.use_x2apic = false;
+			break;
+		case 'X':
+			options.use_x2apic = true;
+			break;
+
+		case 'o':
+			options.use_oneshot_timer = true;
+			break;
+		case 't':
+			options.use_oneshot_timer = false;
+			break;
+
+		case 'P':
+			options.print_result = true;
+			break;
+
+		case 'h':
+		default:
+			help(argv[0]);
+			break;
+		}
+	}
+
+	TEST_REQUIRE(kvm_has_cap(KVM_CAP_GET_TSC_KHZ));
+	TEST_REQUIRE(kvm_has_cap(KVM_CAP_VM_TSC_CONTROL));
+	if (!options.use_oneshot_timer)
+		TEST_REQUIRE(kvm_has_cap(KVM_CAP_TSC_DEADLINE_TIMER));
+	TEST_REQUIRE(kvm_has_cap(KVM_CAP_X86_APIC_BUS_CYCLES_NS));
+	if (options.use_x2apic)
+		TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_X2APIC));
+
+	run_test(duration);
+
+	return 0;
+}
-- 
2.45.2

[PATCH v2 26/36] KVM: selftests: Add nVMX support to timer_latency test case

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Support nVMX for the timer_latency test case to exercise the nVMX APIC
timer virtualization.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 tools/testing/selftests/kvm/include/x86/vmx.h |  10 ++
 .../testing/selftests/kvm/x86/timer_latency.c | 132 +++++++++++++++++-
 2 files changed, 139 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/kvm/include/x86/vmx.h b/tools/testing/selftests/kvm/include/x86/vmx.h
index 92b918700d24..4be103c7b367 100644
--- a/tools/testing/selftests/kvm/include/x86/vmx.h
+++ b/tools/testing/selftests/kvm/include/x86/vmx.h
@@ -24,6 +24,7 @@
 #define CPU_BASED_RDTSC_EXITING			0x00001000
 #define CPU_BASED_CR3_LOAD_EXITING		0x00008000
 #define CPU_BASED_CR3_STORE_EXITING		0x00010000
+#define CPU_BASED_ACTIVATE_TERTIARY_CONTROLS	0x00020000
 #define CPU_BASED_CR8_LOAD_EXITING		0x00080000
 #define CPU_BASED_CR8_STORE_EXITING		0x00100000
 #define CPU_BASED_TPR_SHADOW			0x00200000
@@ -63,6 +64,12 @@
 #define SECONDARY_ENABLE_XSAV_RESTORE		0x00100000
 #define SECONDARY_EXEC_TSC_SCALING		0x02000000
 
+/*
+ * Definitions of Tertiary Processor-Based VM-Execution Controls.
+ * It's 64 bit unlike primary/secondary processor based vm-execution controls.
+ */
+#define TERTIARY_EXEC_GUEST_APIC_TIMER		0x0000000000000100ULL
+
 #define PIN_BASED_EXT_INTR_MASK			0x00000001
 #define PIN_BASED_NMI_EXITING			0x00000008
 #define PIN_BASED_VIRTUAL_NMIS			0x00000020
@@ -104,6 +111,7 @@
 enum vmcs_field {
 	VIRTUAL_PROCESSOR_ID		= 0x00000000,
 	POSTED_INTR_NV			= 0x00000002,
+	GUEST_APIC_TIMER_VECTOR         = 0x0000000a,
 	GUEST_ES_SELECTOR		= 0x00000800,
 	GUEST_CS_SELECTOR		= 0x00000802,
 	GUEST_SS_SELECTOR		= 0x00000804,
@@ -163,6 +171,8 @@ enum vmcs_field {
 	ENCLS_EXITING_BITMAP_HIGH	= 0x0000202F,
 	TSC_MULTIPLIER			= 0x00002032,
 	TSC_MULTIPLIER_HIGH		= 0x00002033,
+	TERTIARY_VM_EXEC_CONTROL        = 0x00002034,
+	TERTIARY_VM_EXEC_CONTROL_HIGH   = 0x00002035,
 	GUEST_PHYSICAL_ADDRESS		= 0x00002400,
 	GUEST_PHYSICAL_ADDRESS_HIGH	= 0x00002401,
 	VMCS_LINK_POINTER		= 0x00002800,
diff --git a/tools/testing/selftests/kvm/x86/timer_latency.c b/tools/testing/selftests/kvm/x86/timer_latency.c
index 7c2e8225f299..17f41a0d15da 100644
--- a/tools/testing/selftests/kvm/x86/timer_latency.c
+++ b/tools/testing/selftests/kvm/x86/timer_latency.c
@@ -15,6 +15,10 @@
 #include "kvm_util.h"
 #include "processor.h"
 #include "apic.h"
+#include "vmx.h"
+
+#define L2_GUEST_STACK_SIZE 256
+static unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
 
 #define LOCAL_TIMER_VECTOR	0xec
 
@@ -30,6 +34,7 @@ struct options {
 	bool use_oneshot_timer;
 	bool use_x2apic;
 	bool use_poll;
+	bool nested;
 
 	uint64_t timer_inc_ns;
 	uint64_t allowed_timer_latency_ns;
@@ -307,6 +312,65 @@ static void guest_code(void)
 	GUEST_DONE();
 }
 
+static void l1_guest_code(struct vmx_pages *vmx_pages)
+{
+	union vmx_ctrl_msr ctls_msr, ctls2_msr;
+	uint64_t pin, ctls, ctls2, ctls3;
+
+	GUEST_ASSERT(prepare_for_vmx_operation(vmx_pages));
+	GUEST_ASSERT(load_vmcs(vmx_pages));
+	prepare_vmcs(vmx_pages, guest_code,
+		     &l2_guest_stack[L2_GUEST_STACK_SIZE]);
+
+	/* Check prerequisites */
+	GUEST_ASSERT(!rdmsr_safe(MSR_IA32_VMX_PROCBASED_CTLS, &ctls_msr.val));
+	GUEST_ASSERT(ctls_msr.clr & CPU_BASED_ACTIVATE_SECONDARY_CONTROLS);
+	GUEST_ASSERT(ctls_msr.clr & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS);
+
+	GUEST_ASSERT(!rdmsr_safe(MSR_IA32_VMX_PROCBASED_CTLS2, &ctls2_msr.val));
+	GUEST_ASSERT(ctls2_msr.clr & SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY);
+
+	GUEST_ASSERT(!rdmsr_safe(MSR_IA32_VMX_PROCBASED_CTLS3, &ctls3));
+	GUEST_ASSERT(ctls3 & TERTIARY_EXEC_GUEST_APIC_TIMER);
+
+	/*
+	 * SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY requires
+	 * PIN_BASED_EXT_INTR_MASK
+	 */
+	pin = vmreadz(PIN_BASED_VM_EXEC_CONTROL);
+	pin |= PIN_BASED_EXT_INTR_MASK;
+	GUEST_ASSERT(!vmwrite(PIN_BASED_VM_EXEC_CONTROL, pin));
+
+	ctls = vmreadz(CPU_BASED_VM_EXEC_CONTROL);
+	ctls |= CPU_BASED_USE_MSR_BITMAPS | CPU_BASED_TPR_SHADOW |
+		CPU_BASED_ACTIVATE_SECONDARY_CONTROLS |
+		CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+	GUEST_ASSERT(!vmwrite(CPU_BASED_VM_EXEC_CONTROL, ctls));
+
+	/* guest apic timer requires virtual interrutp delivery */
+	ctls2 = vmreadz(SECONDARY_VM_EXEC_CONTROL);
+	ctls2 |= SECONDARY_EXEC_VIRTUALIZE_X2APIC_MODE |
+		SECONDARY_EXEC_APIC_REGISTER_VIRT |
+		SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY;
+	vmwrite(SECONDARY_VM_EXEC_CONTROL, ctls2);
+
+	ctls3 = vmreadz(TERTIARY_VM_EXEC_CONTROL);
+	ctls3 |= TERTIARY_EXEC_GUEST_APIC_TIMER;
+	GUEST_ASSERT(!vmwrite(TERTIARY_VM_EXEC_CONTROL, ctls3));
+
+	/*
+	 * We don't emulate apic registers(including APIC_LVTT) for simplicity.
+	 * Directly set vector for timer interrupt instead.
+	 */
+	GUEST_ASSERT(!vmwrite(GUEST_APIC_TIMER_VECTOR, LOCAL_TIMER_VECTOR));
+
+	/* launch L2 */
+	GUEST_ASSERT(!vmlaunch());
+	GUEST_ASSERT_EQ(vmreadz(VM_EXIT_REASON), EXIT_REASON_VMCALL);
+
+	GUEST_DONE();
+}
+
 static void __run_vcpu(struct kvm_vcpu *vcpu)
 {
 	struct ucall uc;
@@ -411,12 +475,40 @@ static void setup_timer_freq(struct kvm_vm *vm,
 		ns_to_tsc(data, options.allowed_timer_latency_ns);
 }
 
+static void clear_msr_bitmap(struct vmx_pages *vmx, int msr)
+{
+	clear_bit(msr, vmx->msr_hva);
+	clear_bit(msr, vmx->msr_hva + 2048);
+}
+
 static void setup(struct kvm_vm **vm__, struct kvm_vcpu **vcpu__)
 {
 	struct kvm_vcpu *vcpu;
 	struct kvm_vm *vm;
 
-	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+	if (options.nested) {
+		vm_vaddr_t vmx_pages_gva = 0;
+		struct vmx_pages *vmx;
+
+		vm = vm_create_with_one_vcpu(&vcpu, l1_guest_code);
+
+		vmx = vcpu_alloc_vmx(vm, &vmx_pages_gva);
+		memset(vmx->msr_hva, 0xff, 4096);
+
+		/* Allow nested apic timer virtualization. */
+		clear_msr_bitmap(vmx, MSR_IA32_TSC_DEADLINE);
+
+		/*  Rely on x2apic virtualization. */
+		clear_msr_bitmap(vmx, MSR_IA32_APICBASE);
+		clear_msr_bitmap(vmx, APIC_BASE_MSR + (APIC_TDCR >> 4));
+		clear_msr_bitmap(vmx, APIC_BASE_MSR + (APIC_LVTT >> 4));
+		clear_msr_bitmap(vmx, APIC_BASE_MSR + (APIC_SPIV >> 4));
+		clear_msr_bitmap(vmx, APIC_BASE_MSR + (APIC_EOI >> 4));
+
+		vcpu_args_set(vcpu, 1, vmx_pages_gva);
+	} else
+		vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+
 	vm_install_exception_handler(vm, LOCAL_TIMER_VECTOR,
 				     guest_timer_interrupt_handler);
 	setup_timer_freq(vm, &shared_data);
@@ -509,6 +601,8 @@ static void help(const char *name)
 	printf("-P: print result stat\n");
 	printf("-x: use xAPIC mode\n");
 	printf("-X: use x2APIC mode (default)\n");
+	printf("-n: Only measure nested VM (L2)\n");
+	printf("-N: Don't measure nested VM (L2)\n");
 	puts("");
 
 	exit(EXIT_SUCCESS);
@@ -518,8 +612,10 @@ int main(int argc, char **argv)
 {
 	int opt;
 	unsigned int duration = TEST_DURATION_DEFAULT_IN_SEC;
+	bool nested_only = false;
+	bool no_nest = false;
 
-	while ((opt = getopt(argc, argv, "hld:p:a:otxXP")) != -1) {
+	while ((opt = getopt(argc, argv, "hld:p:a:otxXPnN")) != -1) {
 		switch (opt) {
 		case 'l':
 			options.use_poll = true;
@@ -557,6 +653,15 @@ int main(int argc, char **argv)
 			options.print_result = true;
 			break;
 
+		case 'n':
+			nested_only = true;
+			no_nest = false;
+			break;
+		case 'N':
+			nested_only = false;
+			no_nest = true;
+			break;
+
 		case 'h':
 		default:
 			help(argv[0]);
@@ -572,7 +677,28 @@ int main(int argc, char **argv)
 	if (options.use_x2apic)
 		TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_X2APIC));
 
-	run_test(duration);
+	if (!nested_only) {
+		options.nested = false;
+		run_test(duration);
+	}
+
+	if (!no_nest) {
+		union vmx_ctrl_msr ctls;
+		uint64_t ctls3;
+
+		ctls.val = kvm_get_feature_msr(MSR_IA32_VMX_TRUE_PROCBASED_CTLS);
+		TEST_REQUIRE(ctls.clr & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS);
+
+		ctls3 = kvm_get_feature_msr(MSR_IA32_VMX_PROCBASED_CTLS3);
+		TEST_REQUIRE(ctls3 & TERTIARY_EXEC_GUEST_APIC_TIMER);
+
+		/* L1 doesn't emulate HLT and memory-mapped APIC. */
+		options.use_poll = true;
+		options.use_oneshot_timer = false;
+
+		options.nested = true;
+		run_test(duration);
+	}
 
 	return 0;
 }
-- 
2.45.2

[PATCH v2 27/36] KVM: selftests: Add test for nVMX MSR_IA32_VMX_PROCBASED_CTLS3

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Add test case for nVMX MSR_IA32_VMX_PROCBASED_CTLS3 emulation.  Test if the
access to MSR_IA32_VMX_PROCBASED_CTLS3 to succeed or fail, depending on
whether the vCPU supports it or not.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 .../testing/selftests/kvm/x86/vmx_msrs_test.c | 53 +++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/vmx_msrs_test.c b/tools/testing/selftests/kvm/x86/vmx_msrs_test.c
index 90720b6205f4..3ec5b73b4f2f 100644
--- a/tools/testing/selftests/kvm/x86/vmx_msrs_test.c
+++ b/tools/testing/selftests/kvm/x86/vmx_msrs_test.c
@@ -48,6 +48,11 @@ static void vmx_fixed0and1_msr_test(struct kvm_vcpu *vcpu, uint32_t msr_index)
 
 static void vmx_save_restore_msrs_test(struct kvm_vcpu *vcpu)
 {
+	union vmx_ctrl_msr ctls;
+	const struct kvm_msr_list *feature_list;
+	bool ctl3_found = false;
+	int i;
+
 	vcpu_set_msr(vcpu, MSR_IA32_VMX_VMCS_ENUM, 0);
 	vcpu_set_msr(vcpu, MSR_IA32_VMX_VMCS_ENUM, -1ull);
 
@@ -65,6 +70,54 @@ static void vmx_save_restore_msrs_test(struct kvm_vcpu *vcpu)
 	vmx_fixed0and1_msr_test(vcpu, MSR_IA32_VMX_TRUE_EXIT_CTLS);
 	vmx_fixed0and1_msr_test(vcpu, MSR_IA32_VMX_TRUE_ENTRY_CTLS);
 	vmx_fixed1_msr_test(vcpu, MSR_IA32_VMX_VMFUNC, -1ull);
+
+	ctls.val = kvm_get_feature_msr(MSR_IA32_VMX_PROCBASED_CTLS);
+	TEST_ASSERT(!(ctls.set & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS),
+		    "CPU_BASED_ACTIVATE_TERTIARY_CONTROLS should be cleared.");
+
+	feature_list = kvm_get_feature_msr_index_list();
+	for (i = 0; i < feature_list->nmsrs; i++) {
+		if (feature_list->indices[i] == MSR_IA32_VMX_PROCBASED_CTLS3) {
+			ctl3_found = true;
+			break;
+		}
+	}
+
+	if (ctls.clr & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS) {
+		uint64_t kvm_ctls3, ctls3;
+
+		TEST_ASSERT(ctl3_found,
+			    "MSR_IA32_VMX_PROCBASED_CTLS3 was not in feature msr index list.");
+
+		kvm_ctls3 = kvm_get_feature_msr(MSR_IA32_VMX_PROCBASED_CTLS3);
+		ctls3 = vcpu_get_msr(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3);
+		TEST_ASSERT(kvm_ctls3 == ctls3,
+			    "msr values for kvm and vcpu must match.");
+
+		vcpu_set_msr(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3, 0);
+		vcpu_set_msr(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3, ctls3);
+		vmx_fixed1_msr_test(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3, ctls3);
+
+		/*
+		 * The kvm host should be able to get/set
+		 * MSR_IA32_VMX_PROCBASED_CTLS3 irrespective to the bit
+		 * CPU_BASED_ACTIVATE_TERTIARY_CONTROLS of
+		 * MSR_IA32_VMX_TRUE_PROCBASED_CTLS.
+		 */
+		ctls.val = vcpu_get_msr(vcpu, MSR_IA32_VMX_TRUE_PROCBASED_CTLS);
+		vcpu_set_msr(vcpu, MSR_IA32_VMX_TRUE_PROCBASED_CTLS,
+			     ctls.set & ~CPU_BASED_ACTIVATE_TERTIARY_CONTROLS);
+		vcpu_set_msr(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3, 0);
+		vcpu_set_msr(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3, ctls3);
+		vmx_fixed1_msr_test(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3, ctls3);
+		vcpu_set_msr(vcpu, MSR_IA32_VMX_TRUE_PROCBASED_CTLS, ctls.val);
+	} else {
+		TEST_ASSERT(!ctl3_found,
+			    "MSR_IA32_VMX_PROCBASED_CTLS3 was in feature msr index list.");
+
+		TEST_ASSERT(!_vcpu_set_msr(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3, 0),
+			    "setting MSR_IA32_VMX_PROCBASED_CTLS3 didn't fail.");
+	}
 }
 
 static void __ia32_feature_control_msr_test(struct kvm_vcpu *vcpu,
-- 
2.45.2

[PATCH v2 28/36] KVM: selftests: Add test vmx_set_nested_state_test with EVMCS disabled

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Hyper-V EVMCS capability, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, is optional.
To increase test coverage, when Hyper-V EVMCS is supported, additionally
run tests with EVMCS disabled.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 .../selftests/kvm/x86/nested_set_state_test.c       | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/tools/testing/selftests/kvm/x86/nested_set_state_test.c b/tools/testing/selftests/kvm/x86/nested_set_state_test.c
index 0f2102b43629..9651282df4d3 100644
--- a/tools/testing/selftests/kvm/x86/nested_set_state_test.c
+++ b/tools/testing/selftests/kvm/x86/nested_set_state_test.c
@@ -401,6 +401,19 @@ int main(int argc, char *argv[])
 	else
 		test_svm_nested_state(vcpu);
 
+	if (kvm_cpu_has(X86_FEATURE_VMX) && have_evmcs) {
+		/*
+		 * KVM_CAP_HYPERV_ENLIGHTENED_VMCS can be only enabled.
+		 * Because There is no way to disable it, re-create vm and vcpu.
+		 */
+		have_evmcs = false;
+		kvm_vm_free(vm);
+		vm = vm_create_with_one_vcpu(&vcpu, NULL);
+		vcpu_clear_cpuid_feature(vcpu, X86_FEATURE_VMX);
+
+		test_vmx_nested_state(vcpu);
+	}
+
 	kvm_vm_free(vm);
 	return 0;
 }
-- 
2.45.2

[PATCH v2 29/36] KVM: selftests: Add tests nested state of APIC timer virtualization

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Test vmread/vmwrite for the following VMCS fields related to apic timer
virtualization in vmx_set_nested_state_test.
- TERTIARY_EXEC_GUEST_APIC_TIMER bit in TERTIARY_VM_EXEC_CONTROL
- VIRTUAL_TIMER_VECTOR_OFFSET
- GUEST_DEADLINE_OFFSET
- GUEST_DEADLINE_SHADOW_OFFSET

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
Changes:
v1 -> v2:
- format fixes.
---
 tools/testing/selftests/kvm/include/x86/vmx.h |   4 +
 .../selftests/kvm/x86/nested_set_state_test.c | 237 ++++++++++++++++++
 2 files changed, 241 insertions(+)

diff --git a/tools/testing/selftests/kvm/include/x86/vmx.h b/tools/testing/selftests/kvm/include/x86/vmx.h
index 4be103c7b367..7a8fcf6f58ff 100644
--- a/tools/testing/selftests/kvm/include/x86/vmx.h
+++ b/tools/testing/selftests/kvm/include/x86/vmx.h
@@ -173,6 +173,8 @@ enum vmcs_field {
 	TSC_MULTIPLIER_HIGH		= 0x00002033,
 	TERTIARY_VM_EXEC_CONTROL        = 0x00002034,
 	TERTIARY_VM_EXEC_CONTROL_HIGH   = 0x00002035,
+	GUEST_DEADLINE_VIR              = 0x0000204e,
+	GUEST_DEADLINE_VIR_HIGH         = 0x0000204f,
 	GUEST_PHYSICAL_ADDRESS		= 0x00002400,
 	GUEST_PHYSICAL_ADDRESS_HIGH	= 0x00002401,
 	VMCS_LINK_POINTER		= 0x00002800,
@@ -195,6 +197,8 @@ enum vmcs_field {
 	GUEST_PDPTR3_HIGH		= 0x00002811,
 	GUEST_BNDCFGS			= 0x00002812,
 	GUEST_BNDCFGS_HIGH		= 0x00002813,
+	GUEST_DEADLINE_PHY              = 0x00002830,
+	GUEST_DEADLINE_PHY_HIGH         = 0x00002831,
 	HOST_IA32_PAT			= 0x00002c00,
 	HOST_IA32_PAT_HIGH		= 0x00002c01,
 	HOST_IA32_EFER			= 0x00002c02,
diff --git a/tools/testing/selftests/kvm/x86/nested_set_state_test.c b/tools/testing/selftests/kvm/x86/nested_set_state_test.c
index 9651282df4d3..ed668a006c06 100644
--- a/tools/testing/selftests/kvm/x86/nested_set_state_test.c
+++ b/tools/testing/selftests/kvm/x86/nested_set_state_test.c
@@ -24,6 +24,8 @@
 #define VMCS12_REVISION 0x11e57ed0
 
 bool have_evmcs;
+bool have_procbased_tertiary_ctls;
+bool have_apic_timer_virtualization;
 
 void test_nested_state(struct kvm_vcpu *vcpu, struct kvm_nested_state *state)
 {
@@ -84,6 +86,233 @@ void set_default_vmx_state(struct kvm_nested_state *state, int size)
 	set_revision_id_for_vmcs12(state, VMCS12_REVISION);
 }
 
+/* Those values are taken from arch/x86/kvm/vmx/vmcs12.h */
+#define VMCS_LINK_POINTER_OFFSET		176
+#define TERTIARY_VM_EXEC_CONTROL_OFFSET		336
+#define GUEST_CR0_OFFSET			424
+#define GUEST_CR4_OFFSET			440
+#define HOST_CR0_OFFSET				584
+#define HOST_CR4_OFFSET				600
+#define PIN_BASED_VM_EXEC_CONTROL_OFFSET	744
+#define CPU_BASED_VM_EXEC_CONTROL_OFFSET	748
+#define VM_EXIT_CONTROLS_OFFSET			768
+#define VM_ENTRY_CONTROLS_OFFSET		780
+#define SECONDARY_VM_EXEC_CONTROL_OFFSET	804
+#define HOST_CS_SELECTOR_OFFSET			984
+#define HOST_SS_SELECTOR_OFFSET			986
+#define HOST_TR_SELECTOR_OFFSET			994
+#define VIRTUAL_TIMER_VECTOR_OFFSET		998
+#define GUEST_DEADLINE_OFFSET			1000
+#define GUEST_DEADLINE_SHADOW_OFFSET		1008
+
+#define KERNEL_CS       0x8
+#define KERNEL_DS       0x10
+#define KERNEL_TSS      0x18
+
+/* vcpu with vmxon=false is needed to be able to set VMX MSRs. */
+static void nested_vmxoff(struct kvm_vcpu *vcpu, struct kvm_nested_state *state,
+			int state_sz)
+{
+	set_default_vmx_state(state, state_sz);
+	state->flags = 0;
+	state->hdr.vmx.vmxon_pa = -1ull;
+	state->hdr.vmx.vmcs12_pa = -1ull;
+	test_nested_state(vcpu, state);
+}
+
+static void get_control(struct kvm_vcpu *vcpu, uint32_t msr_index,
+			uint32_t *fixed0, uint32_t *fixed1)
+{
+	uint64_t ctls;
+
+	ctls = vcpu_get_msr(vcpu, msr_index);
+
+	*fixed0 = ctls & 0xffffffff;
+	*fixed1 = ctls >> 32;
+}
+
+static uint32_t *init_control(struct kvm_vcpu *vcpu,
+			      struct kvm_nested_state *state,
+			      uint32_t msr_index, size_t offset)
+{
+	uint32_t fixed0, fixed1, *vmcs32;
+
+	get_control(vcpu, msr_index, &fixed0, &fixed1);
+	vmcs32 = (uint32_t *)&state->data.vmx->vmcs12[offset];
+
+	*vmcs32 = fixed0;
+	*vmcs32 &= fixed1;
+
+	return vmcs32;
+}
+
+void set_guest_vmx_state(struct kvm_vcpu *vcpu, struct kvm_nested_state *state,
+			 int size)
+{
+	unsigned long cr0, cr4;
+	uint32_t *vmcs32;
+	uint64_t *vmcs64;
+
+	set_default_vmx_state(state, size);
+	state->flags |= KVM_STATE_NESTED_GUEST_MODE;
+
+	/* control */
+	init_control(vcpu, state, MSR_IA32_VMX_TRUE_PINBASED_CTLS,
+		     PIN_BASED_VM_EXEC_CONTROL_OFFSET);
+
+	vmcs32 = init_control(vcpu, state, MSR_IA32_VMX_TRUE_PROCBASED_CTLS,
+			      CPU_BASED_VM_EXEC_CONTROL_OFFSET);
+	*vmcs32 |= CPU_BASED_ACTIVATE_SECONDARY_CONTROLS;
+	*vmcs32 &= ~CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+
+	init_control(vcpu, state, MSR_IA32_VMX_PROCBASED_CTLS2,
+		     SECONDARY_VM_EXEC_CONTROL_OFFSET);
+
+	vmcs32 = init_control(vcpu, state, MSR_IA32_VMX_TRUE_EXIT_CTLS,
+			      VM_EXIT_CONTROLS_OFFSET);
+	*vmcs32 |= VM_EXIT_HOST_ADDR_SPACE_SIZE;
+
+	init_control(vcpu, state, MSR_IA32_VMX_TRUE_ENTRY_CTLS,
+		     VM_ENTRY_CONTROLS_OFFSET);
+
+	vmcs64 = (uint64_t *)&state->data.vmx->vmcs12[VMCS_LINK_POINTER_OFFSET];
+	*vmcs64 = -1ull;
+
+	/* host state */
+	cr0 = vcpu_get_msr(vcpu, MSR_IA32_VMX_CR0_FIXED0);
+	cr0 &= vcpu_get_msr(vcpu, MSR_IA32_VMX_CR0_FIXED1);
+	cr0 |= X86_CR0_PG;
+	*(unsigned long *)&state->data.vmx->vmcs12[HOST_CR0_OFFSET] = cr0;
+
+	cr4 = vcpu_get_msr(vcpu, MSR_IA32_VMX_CR4_FIXED0);
+	cr4 &= vcpu_get_msr(vcpu, MSR_IA32_VMX_CR4_FIXED1);
+	cr4 |= X86_CR4_PAE | X86_CR4_VMXE;
+	*(unsigned long *)&state->data.vmx->vmcs12[HOST_CR4_OFFSET] = cr4;
+
+	*(unsigned long *)&state->data.vmx->vmcs12[HOST_CS_SELECTOR_OFFSET] = KERNEL_CS;
+	*(unsigned long *)&state->data.vmx->vmcs12[HOST_TR_SELECTOR_OFFSET] = KERNEL_TSS;
+	*(unsigned long *)&state->data.vmx->vmcs12[HOST_SS_SELECTOR_OFFSET] = KERNEL_DS;
+
+	/* guest state */
+	cr0 = vcpu_get_msr(vcpu, MSR_IA32_VMX_CR0_FIXED0);
+	cr0 &= vcpu_get_msr(vcpu, MSR_IA32_VMX_CR0_FIXED1);
+	*(unsigned long *)&state->data.vmx->vmcs12[GUEST_CR0_OFFSET] = cr0;
+
+	cr4 = vcpu_get_msr(vcpu, MSR_IA32_VMX_CR4_FIXED0);
+	cr4 &= vcpu_get_msr(vcpu, MSR_IA32_VMX_CR4_FIXED1);
+	*(unsigned long *)&state->data.vmx->vmcs12[GUEST_CR4_OFFSET] = cr4;
+
+}
+
+static void test_tertiary_ctls(struct kvm_vcpu *vcpu,
+			       struct kvm_nested_state *state,
+			       int state_sz)
+{
+	union vmx_ctrl_msr msr;
+	uint16_t *vmcs16;
+	uint32_t *vmcs32;
+	uint64_t *vmcs64;
+	uint64_t ctls;
+
+	nested_vmxoff(vcpu, state, state_sz);
+
+	msr.val = vcpu_get_msr(vcpu, MSR_IA32_VMX_TRUE_PROCBASED_CTLS);
+	msr.clr |= CPU_BASED_ACTIVATE_SECONDARY_CONTROLS;
+	msr.clr |= CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+	vcpu_set_msr(vcpu, MSR_IA32_VMX_TRUE_PROCBASED_CTLS, msr.val);
+
+	ctls = vcpu_get_msr(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3);
+	ctls |= TERTIARY_EXEC_GUEST_APIC_TIMER;
+	vcpu_set_msr(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3, ctls);
+
+	set_default_vmx_state(state, state_sz);
+	test_nested_state(vcpu, state);
+
+	vmcs32 = (uint32_t *)&state->data.vmx->vmcs12[PIN_BASED_VM_EXEC_CONTROL_OFFSET];
+	*vmcs32 |= PIN_BASED_EXT_INTR_MASK;
+
+	vmcs32 = (uint32_t *)&state->data.vmx->vmcs12[CPU_BASED_VM_EXEC_CONTROL_OFFSET];
+	*vmcs32 |= CPU_BASED_TPR_SHADOW | CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+
+	vmcs32 = (uint32_t *)&state->data.vmx->vmcs12[SECONDARY_VM_EXEC_CONTROL_OFFSET];
+	*vmcs32 |= SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY;
+
+	vmcs64 = (uint64_t *)&state->data.vmx->vmcs12[TERTIARY_VM_EXEC_CONTROL_OFFSET];
+	*vmcs64 |= TERTIARY_EXEC_GUEST_APIC_TIMER;
+
+	vmcs16 = (uint16_t *)&state->data.vmx->vmcs12[VIRTUAL_TIMER_VECTOR_OFFSET];
+	*vmcs16 = 128;
+
+	vmcs64 = (uint64_t *)&state->data.vmx->vmcs12[GUEST_DEADLINE_OFFSET];
+	/* random non-zero value */
+	*vmcs64 = 0xffff;
+
+	vmcs64 = (uint64_t *)&state->data.vmx->vmcs12[GUEST_DEADLINE_SHADOW_OFFSET];
+	*vmcs64 = 0xffff;
+
+	test_nested_state(vcpu, state);
+}
+
+static void test_tertiary_ctls_disabled(struct kvm_vcpu *vcpu,
+					struct kvm_nested_state *state,
+					int state_sz)
+{
+	union vmx_ctrl_msr msr;
+	uint16_t *vmcs16;
+	uint32_t *vmcs32;
+	uint64_t *vmcs64;
+
+	nested_vmxoff(vcpu, state, state_sz);
+
+	msr.val = kvm_get_feature_msr(MSR_IA32_VMX_TRUE_PROCBASED_CTLS);
+	if (msr.clr & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS) {
+		msr.val = vcpu_get_msr(vcpu, MSR_IA32_VMX_TRUE_PROCBASED_CTLS);
+		msr.clr &= ~CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+		vcpu_set_msr(vcpu, MSR_IA32_VMX_TRUE_PROCBASED_CTLS, msr.val);
+		vcpu_set_msr(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3, 0);
+	}
+
+	set_guest_vmx_state(vcpu, state, state_sz);
+	test_nested_state(vcpu, state);
+
+	set_guest_vmx_state(vcpu, state, state_sz);
+	vmcs32 = (uint32_t *)&state->data.vmx->vmcs12[CPU_BASED_VM_EXEC_CONTROL_OFFSET];
+	*vmcs32 |= CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+	test_nested_state_expect_einval(vcpu, state);
+
+	set_guest_vmx_state(vcpu, state, state_sz);
+	vmcs64 = (uint64_t *)&state->data.vmx->vmcs12[TERTIARY_VM_EXEC_CONTROL_OFFSET];
+	*vmcs64 |= TERTIARY_EXEC_GUEST_APIC_TIMER;
+	test_nested_state_expect_einval(vcpu, state);
+
+	set_guest_vmx_state(vcpu, state, state_sz);
+	vmcs16 = (uint16_t *)&state->data.vmx->vmcs12[VIRTUAL_TIMER_VECTOR_OFFSET];
+	*vmcs16 = 128;
+	test_nested_state_expect_einval(vcpu, state);
+
+	set_guest_vmx_state(vcpu, state, state_sz);
+	vmcs64 = (uint64_t *)&state->data.vmx->vmcs12[GUEST_DEADLINE_OFFSET];
+	*vmcs64 = 0xffff;
+	test_nested_state_expect_einval(vcpu, state);
+
+	set_guest_vmx_state(vcpu, state, state_sz);
+	vmcs64 = (uint64_t *)&state->data.vmx->vmcs12[GUEST_DEADLINE_SHADOW_OFFSET];
+	*vmcs64 = 0xffff;
+	test_nested_state_expect_einval(vcpu, state);
+}
+
+static void test_vmx_tertiary_ctls(struct kvm_vcpu *vcpu,
+				   struct kvm_nested_state *state,
+				   int state_sz)
+{
+	vcpu_set_cpuid_feature(vcpu, X86_FEATURE_VMX);
+
+	if (have_procbased_tertiary_ctls)
+		test_tertiary_ctls(vcpu, state, state_sz);
+
+	test_tertiary_ctls_disabled(vcpu, state, state_sz);
+}
+
 void test_vmx_nested_state(struct kvm_vcpu *vcpu)
 {
 	/* Add a page for VMCS12. */
@@ -343,6 +572,8 @@ void test_svm_nested_state(struct kvm_vcpu *vcpu)
 	TEST_ASSERT_EQ(state->hdr.svm.vmcb_pa, 0);
 	TEST_ASSERT_EQ(state->flags, KVM_STATE_NESTED_GIF_SET);
 
+	test_vmx_tertiary_ctls(vcpu, state, state_sz);
+
 	free(state);
 }
 
@@ -353,6 +584,12 @@ int main(int argc, char *argv[])
 	struct kvm_vcpu *vcpu;
 
 	have_evmcs = kvm_check_cap(KVM_CAP_HYPERV_ENLIGHTENED_VMCS);
+	have_procbased_tertiary_ctls =
+		(kvm_get_feature_msr(MSR_IA32_VMX_TRUE_PROCBASED_CTLS) >> 32) &
+		CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+	have_apic_timer_virtualization = have_procbased_tertiary_ctls &&
+		(kvm_get_feature_msr(MSR_IA32_VMX_PROCBASED_CTLS3) &
+		 TERTIARY_EXEC_GUEST_APIC_TIMER);
 
 	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_VMX) ||
 		     kvm_cpu_has(X86_FEATURE_SVM));
-- 
2.45.2

[PATCH v2 30/36] KVM: selftests: Add VMCS access test to APIC timer virtualization

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Add test case for VMCS access of nVMX APIC timer virtualization.  It tests
nVMX emulation of a tertiary processor-based VM executing controls MSR,
tertiary processor-base VM execution controls, and related VMCS fields by
L1 vCPU and L2 vCPU.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../testing/selftests/kvm/include/x86/apic.h  |   1 +
 .../kvm/x86/vmx_apic_timer_virt_vmcs_test.c   | 464 ++++++++++++++++++
 3 files changed, 466 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_vmcs_test.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 3c2a5470c2ca..f0023bab1cd7 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -146,6 +146,7 @@ TEST_GEN_PROGS_x86 += x86/triple_fault_event_test
 TEST_GEN_PROGS_x86 += x86/recalc_apic_map_test
 TEST_GEN_PROGS_x86 += x86/aperfmperf_test
 TEST_GEN_PROGS_x86 += x86/timer_latency
+TEST_GEN_PROGS_x86 += x86/vmx_apic_timer_virt_vmcs_test
 TEST_GEN_PROGS_x86 += access_tracking_perf_test
 TEST_GEN_PROGS_x86 += coalesced_io_test
 TEST_GEN_PROGS_x86 += dirty_log_perf_test
diff --git a/tools/testing/selftests/kvm/include/x86/apic.h b/tools/testing/selftests/kvm/include/x86/apic.h
index 5ca6bacbd70e..b47e905aa4e8 100644
--- a/tools/testing/selftests/kvm/include/x86/apic.h
+++ b/tools/testing/selftests/kvm/include/x86/apic.h
@@ -35,6 +35,7 @@
 #define		APIC_SPIV_FOCUS_DISABLED	(1 << 9)
 #define		APIC_SPIV_APIC_ENABLED		(1 << 8)
 #define	APIC_ISR	0x100
+#define APIC_ISR_NR	0x8
 #define APIC_IRR	0x200
 #define	APIC_ICR	0x300
 #define	APIC_LVTCMCI	0x2f0
diff --git a/tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_vmcs_test.c b/tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_vmcs_test.c
new file mode 100644
index 000000000000..cd440b636585
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_vmcs_test.c
@@ -0,0 +1,464 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2025 Intel Corporation
+ *
+ * Tested vmread()/vmwrite() related to APIC timer virtualization in L1
+ * emulated by KVM.
+ */
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+#include "vmx.h"
+
+#include <string.h>
+#include <sys/ioctl.h>
+#include <stdatomic.h>
+
+bool have_procbased_tertiary_ctls;
+bool have_apic_timer_virtualization;
+
+#define L2_GUEST_STACK_SIZE 256
+static unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
+
+/* Any value [32, 255] for timer vector is okay. */
+#define TIMER_VECTOR   0xec
+
+static bool update_l2_tsc_deadline;
+static uint64_t l2_tsc_deadline;
+
+static void guest_timer_interrupt_handler(struct ex_regs *regs)
+{
+	x2apic_write_reg(APIC_EOI, 0);
+}
+
+static void l2_guest_code(void)
+{
+	cli();
+	x2apic_enable();
+	wrmsr(MSR_IA32_TSC_DEADLINE, 0);
+	x2apic_write_reg(APIC_LVTT, APIC_LVT_TIMER_TSCDEADLINE | TIMER_VECTOR);
+
+	vmcall();
+
+	while (true) {
+		/* reap pending timer interrupt. */
+		sti_nop_cli();
+
+		if (update_l2_tsc_deadline)
+			GUEST_ASSERT(!wrmsr_safe(MSR_IA32_TSC_DEADLINE, l2_tsc_deadline));
+
+		vmcall();
+	}
+
+	GUEST_FAIL("should not reached.");
+}
+
+static void setup_l2(struct vmx_pages *vmx_pages)
+{
+	uint64_t ctls, msr_val;
+	int r;
+
+	GUEST_ASSERT(!rdmsr_safe(MSR_IA32_VMX_TRUE_PROCBASED_CTLS,
+				 &msr_val));
+	GUEST_ASSERT_EQ(have_procbased_tertiary_ctls,
+			!!((msr_val >> 32) & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS));
+
+	r = rdmsr_safe(MSR_IA32_VMX_PROCBASED_CTLS3, &msr_val);
+	GUEST_ASSERT(have_procbased_tertiary_ctls == !r);
+	if (r)
+		msr_val = 0;
+	GUEST_ASSERT_EQ(have_apic_timer_virtualization,
+			!!(msr_val & TERTIARY_EXEC_GUEST_APIC_TIMER));
+
+	GUEST_ASSERT(prepare_for_vmx_operation(vmx_pages));
+	GUEST_ASSERT(load_vmcs(vmx_pages));
+	prepare_vmcs(vmx_pages, l2_guest_code,
+		     &l2_guest_stack[L2_GUEST_STACK_SIZE]);
+
+	GUEST_ASSERT(!vmread(PIN_BASED_VM_EXEC_CONTROL, &ctls));
+	ctls |= PIN_BASED_EXT_INTR_MASK;
+	GUEST_ASSERT(!vmwrite(PIN_BASED_VM_EXEC_CONTROL, ctls));
+
+	GUEST_ASSERT(!vmread(CPU_BASED_VM_EXEC_CONTROL, &ctls));
+	ctls |= CPU_BASED_USE_MSR_BITMAPS | CPU_BASED_TPR_SHADOW |
+		CPU_BASED_ACTIVATE_SECONDARY_CONTROLS;
+	if (have_procbased_tertiary_ctls)
+		ctls |= CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+	GUEST_ASSERT(!vmwrite(CPU_BASED_VM_EXEC_CONTROL, ctls));
+
+	GUEST_ASSERT(!vmread(SECONDARY_VM_EXEC_CONTROL, &ctls));
+	ctls |= SECONDARY_EXEC_VIRTUALIZE_X2APIC_MODE |
+		SECONDARY_EXEC_APIC_REGISTER_VIRT |
+		SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY;
+	GUEST_ASSERT(!vmwrite(SECONDARY_VM_EXEC_CONTROL, ctls));
+
+	if (have_procbased_tertiary_ctls) {
+		GUEST_ASSERT(!vmread(TERTIARY_VM_EXEC_CONTROL, &ctls));
+
+		ctls &= ~TERTIARY_EXEC_GUEST_APIC_TIMER;
+		GUEST_ASSERT(!vmwrite(TERTIARY_VM_EXEC_CONTROL, ctls));
+	} else {
+		GUEST_ASSERT(vmread(TERTIARY_VM_EXEC_CONTROL, &ctls));
+		ctls = 0;
+	}
+
+	ctls |= TERTIARY_EXEC_GUEST_APIC_TIMER;
+	GUEST_ASSERT_EQ(have_procbased_tertiary_ctls,
+			!vmwrite(TERTIARY_VM_EXEC_CONTROL, ctls));
+	if (have_procbased_tertiary_ctls && !have_apic_timer_virtualization) {
+		ctls &= ~TERTIARY_EXEC_GUEST_APIC_TIMER;
+		GUEST_ASSERT(!vmwrite(TERTIARY_VM_EXEC_CONTROL, ctls));
+	}
+
+	GUEST_ASSERT_EQ(have_apic_timer_virtualization,
+			!vmwrite(GUEST_APIC_TIMER_VECTOR, TIMER_VECTOR));
+}
+
+static void skip_guest_instruction(void)
+{
+	uint64_t guest_rip, length;
+
+	GUEST_ASSERT(!vmread(GUEST_RIP, &guest_rip));
+	GUEST_ASSERT(!vmread(VM_EXIT_INSTRUCTION_LEN, &length));
+
+	GUEST_ASSERT(!vmwrite(GUEST_RIP, guest_rip + length));
+	GUEST_ASSERT(!vmwrite(VM_EXIT_INSTRUCTION_LEN, 0));
+}
+
+static void l2_load_vmlaunch(struct vmx_pages *vmx_pages)
+{
+	GUEST_ASSERT(load_vmcs(vmx_pages));
+	skip_guest_instruction();
+	GUEST_ASSERT(!vmlaunch());
+	GUEST_ASSERT_EQ(vmreadz(VM_EXIT_REASON), EXIT_REASON_VMCALL);
+}
+
+struct vmcs_guest_deadline {
+	uint64_t vir;
+	uint64_t phy;
+};
+
+struct deadline_test {
+	struct vmcs_guest_deadline set;
+	struct vmcs_guest_deadline result;
+};
+
+static void test_vmclear_vmptrld(struct vmx_pages *vmx_pages)
+{
+	struct deadline_test tests[] = {
+		{
+			.set = {
+				.vir = ~0ull,
+				.phy = ~0ull,
+			},
+			.result = {
+				.vir = ~0ull,
+				.phy = ~0ull,
+			}
+		},
+		{
+			.set = {
+				.vir = ~0ull,
+				.phy = 0,
+			},
+			.result = {
+				.vir = ~0ull,
+				.phy = 0,
+			}
+		},
+		{
+			.set = {
+				.vir = ~0ull,
+				.phy = 1,
+			},
+			.result = {
+				.vir = 0,
+				.phy = 0,
+			}
+		},
+		{
+			.set = {
+				.vir = 0,
+				.phy = ~0ull,
+			},
+			.result = {
+				.vir = 0,
+				.phy = ~0ull,
+			}
+		},
+		{
+			.set = {
+				.vir = 1,
+				.phy = ~0ull,
+			},
+			.result = {
+				.vir = 1,
+				.phy = ~0ull,
+			}
+		},
+		{
+			.set = {
+				.vir = 1,
+				.phy = 1,
+			},
+			.result = {
+				.vir = 0,
+				.phy = 0,
+			}
+		},
+		{
+			.set = {
+				.vir = 1,
+				.phy = 0,
+			},
+			.result = {
+				.vir = 1,
+				.phy = 0,
+			}
+		},
+		{
+			.set = {
+				.vir = 0,
+				.phy = 1,
+			},
+			.result = {
+				.vir = 0,
+				.phy = 0,
+			}
+		},
+	};
+	int i;
+
+	if (!have_apic_timer_virtualization)
+		return;
+
+	update_l2_tsc_deadline = false;
+
+	/*
+	 * Test if KVM properly store/load TIMER_VECTOR, guest deadline of
+	 * vmcs area to/from memory.
+	 * Enforce KVM to store nested vmcs to memory and load it again.
+	 * load_vmcs() issues vmclear(), and then vmptrld()
+	 */
+	l2_load_vmlaunch(vmx_pages);
+	GUEST_ASSERT_EQ(vmreadz(GUEST_APIC_TIMER_VECTOR), TIMER_VECTOR);
+
+	for (i = 0; i < ARRAY_SIZE(tests); i++) {
+		struct deadline_test *test = &tests[i];
+		uint64_t vir, phy, val;
+
+		GUEST_ASSERT(!vmwrite(GUEST_DEADLINE_VIR, test->set.vir));
+		GUEST_ASSERT(!vmread(GUEST_DEADLINE_VIR, &val));
+		GUEST_ASSERT_EQ(test->set.vir, val);
+
+		GUEST_ASSERT(!vmwrite(GUEST_DEADLINE_PHY, test->set.phy));
+		GUEST_ASSERT(!vmread(GUEST_DEADLINE_PHY, &val));
+		GUEST_ASSERT_EQ(test->set.phy, val);
+
+		l2_load_vmlaunch(vmx_pages);
+
+		GUEST_ASSERT(!vmread(GUEST_DEADLINE_VIR, &vir));
+		GUEST_ASSERT(!vmread(GUEST_DEADLINE_PHY, &phy));
+
+		GUEST_ASSERT_EQ(vir, test->result.vir);
+		GUEST_ASSERT_EQ(!!phy, !!test->result.phy);
+	}
+}
+
+static void test_ctls(void)
+{
+	uint64_t ctls;
+
+	update_l2_tsc_deadline = false;
+
+	GUEST_ASSERT_EQ(!vmwrite(GUEST_APIC_TIMER_VECTOR, TIMER_VECTOR),
+			have_apic_timer_virtualization);
+	GUEST_ASSERT_EQ(!vmwrite(GUEST_DEADLINE_VIR, 0),
+			have_apic_timer_virtualization);
+	GUEST_ASSERT_EQ(!vmwrite(GUEST_DEADLINE_PHY, 0),
+			have_apic_timer_virtualization);
+
+	if (!have_procbased_tertiary_ctls) {
+		GUEST_ASSERT(!vmread(CPU_BASED_VM_EXEC_CONTROL, &ctls));
+		ctls |= CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+		GUEST_ASSERT(!vmwrite(CPU_BASED_VM_EXEC_CONTROL, ctls));
+
+		skip_guest_instruction();
+		GUEST_ASSERT(vmresume());
+
+		ctls &= ~CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+		GUEST_ASSERT(!vmwrite(CPU_BASED_VM_EXEC_CONTROL, ctls));
+	}
+
+	if (have_procbased_tertiary_ctls && !have_apic_timer_virtualization) {
+		GUEST_ASSERT(!vmread(TERTIARY_VM_EXEC_CONTROL, &ctls));
+		ctls |= TERTIARY_EXEC_GUEST_APIC_TIMER;
+		GUEST_ASSERT(!vmwrite(TERTIARY_VM_EXEC_CONTROL, ctls));
+
+		skip_guest_instruction();
+		GUEST_ASSERT(vmresume());
+
+		ctls &= ~TERTIARY_EXEC_GUEST_APIC_TIMER;
+		GUEST_ASSERT(!vmwrite(TERTIARY_VM_EXEC_CONTROL, ctls));
+	}
+}
+
+static void test_l2_set_deadline(void)
+{
+	uint64_t deadlines[] = {
+		0,
+		1,
+		2,
+
+		rdtsc() / 2,
+		rdtsc(),
+		rdtsc() * 2,
+
+		~0ull / 2 - 1,
+		~0ull / 2,
+		~0ull / 2 + 1,
+
+		~0ull - 1,
+		~0ull - 2,
+		~0ull,
+	};
+	int i;
+
+	update_l2_tsc_deadline = true;
+
+	for (i = 0; i < ARRAY_SIZE(deadlines); i++) {
+		uint64_t phy, vir;
+
+		l2_tsc_deadline = deadlines[i];
+
+		skip_guest_instruction();
+		GUEST_ASSERT(!vmresume());
+		GUEST_ASSERT_EQ(vmreadz(VM_EXIT_REASON), EXIT_REASON_VMCALL);
+
+		GUEST_ASSERT(!vmread(GUEST_DEADLINE_VIR, &vir));
+		GUEST_ASSERT(!vmread(GUEST_DEADLINE_PHY, &phy));
+
+		GUEST_ASSERT(!vir || vir == l2_tsc_deadline);
+	}
+}
+
+static void l1_guest_code(struct vmx_pages *vmx_pages)
+{
+	setup_l2(vmx_pages);
+
+	GUEST_ASSERT_EQ(have_apic_timer_virtualization,
+			!vmwrite(GUEST_DEADLINE_VIR, ~0ull));
+	GUEST_ASSERT_EQ(have_apic_timer_virtualization,
+			!vmwrite(GUEST_DEADLINE_PHY, ~0ull));
+
+	test_vmclear_vmptrld(vmx_pages);
+
+	update_l2_tsc_deadline = false;
+	l2_load_vmlaunch(vmx_pages);
+	test_ctls();
+
+	if (have_apic_timer_virtualization) {
+		update_l2_tsc_deadline = false;
+		l2_load_vmlaunch(vmx_pages);
+
+		test_l2_set_deadline();
+	}
+
+	GUEST_DONE();
+}
+
+static void run_vcpu(struct kvm_vcpu *vcpu)
+{
+	bool done = false;
+
+	while (!done) {
+		struct ucall uc;
+
+		vcpu_run(vcpu);
+		TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_IO);
+
+		switch (get_ucall(vcpu, &uc)) {
+		case UCALL_ABORT:
+			REPORT_GUEST_ASSERT(uc);
+			/* NOT REACHED */
+		case UCALL_PRINTF:
+			pr_info("%s", uc.buffer);
+			break;
+		case UCALL_DONE:
+			done = true;
+			break;
+		default:
+			TEST_ASSERT(false, "Unknown ucall %lu", uc.cmd);
+		}
+	}
+}
+
+static void test_apic_virtualization_vmcs(void)
+{
+	vm_vaddr_t vmx_pages_gva;
+
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	uint64_t ctls, ctls3;
+
+	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_VMX));
+
+	vm = vm_create_with_one_vcpu(&vcpu, l1_guest_code);
+	vcpu_alloc_vmx(vm, &vmx_pages_gva);
+	vcpu_args_set(vcpu, 1, vmx_pages_gva);
+
+	vcpu_set_cpuid_feature(vcpu, X86_FEATURE_TSC_DEADLINE_TIMER);
+
+	ctls = vcpu_get_msr(vcpu, MSR_IA32_VMX_TRUE_PROCBASED_CTLS);
+	if (have_procbased_tertiary_ctls) {
+		ctls |= (uint64_t)CPU_BASED_ACTIVATE_TERTIARY_CONTROLS << 32;
+		vcpu_set_msr(vcpu, MSR_IA32_VMX_TRUE_PROCBASED_CTLS, ctls);
+
+		ctls3 = vcpu_get_msr(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3);
+		if (have_apic_timer_virtualization)
+			ctls3 |= TERTIARY_EXEC_GUEST_APIC_TIMER;
+		else
+			ctls3 &= ~TERTIARY_EXEC_GUEST_APIC_TIMER;
+		vcpu_set_msr(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3, ctls3);
+	} else {
+		ctls &= ~((uint64_t)CPU_BASED_ACTIVATE_TERTIARY_CONTROLS << 32);
+		vcpu_set_msr(vcpu, MSR_IA32_VMX_TRUE_PROCBASED_CTLS, ctls);
+	}
+
+	/* For L2. */
+	vm_install_exception_handler(vm, TIMER_VECTOR,
+				     guest_timer_interrupt_handler);
+
+	sync_global_to_guest(vm, have_procbased_tertiary_ctls);
+	sync_global_to_guest(vm, have_apic_timer_virtualization);
+	run_vcpu(vcpu);
+
+	kvm_vm_free(vm);
+}
+
+int main(int argc, char *argv[])
+{
+	have_procbased_tertiary_ctls =
+		(kvm_get_feature_msr(MSR_IA32_VMX_TRUE_PROCBASED_CTLS) >> 32) &
+		CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+	have_apic_timer_virtualization = have_procbased_tertiary_ctls &&
+		(kvm_get_feature_msr(MSR_IA32_VMX_PROCBASED_CTLS3) &
+		 TERTIARY_EXEC_GUEST_APIC_TIMER);
+
+	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_VMX));
+
+	test_apic_virtualization_vmcs();
+
+	if (have_apic_timer_virtualization) {
+		have_apic_timer_virtualization = false;
+		test_apic_virtualization_vmcs();
+	}
+
+	if (have_procbased_tertiary_ctls) {
+		have_procbased_tertiary_ctls = false;
+		test_apic_virtualization_vmcs();
+	}
+
+	return 0;
+}
-- 
2.45.2

[PATCH v2 31/36] KVM: selftests: Add serialize() helper and X86_FEATURE_SERIALIZE

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

As the following test cases use serialize instruction, add feature
definition and a helper function for it.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
--
Changes v2:
- newly added
---
 tools/testing/selftests/kvm/include/x86/processor.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tools/testing/selftests/kvm/include/x86/processor.h b/tools/testing/selftests/kvm/include/x86/processor.h
index da94ebf16821..88e5acc1c03e 100644
--- a/tools/testing/selftests/kvm/include/x86/processor.h
+++ b/tools/testing/selftests/kvm/include/x86/processor.h
@@ -172,6 +172,7 @@ struct kvm_x86_cpu_feature {
 #define	X86_FEATURE_RDPID		KVM_X86_CPU_FEATURE(0x7, 0, ECX, 22)
 #define	X86_FEATURE_SGX_LC		KVM_X86_CPU_FEATURE(0x7, 0, ECX, 30)
 #define	X86_FEATURE_SHSTK		KVM_X86_CPU_FEATURE(0x7, 0, ECX, 7)
+#define	X86_FEATURE_SERIALIZE		KVM_X86_CPU_FEATURE(0x7, 0, EDX, 14)
 #define	X86_FEATURE_IBT			KVM_X86_CPU_FEATURE(0x7, 0, EDX, 20)
 #define	X86_FEATURE_AMX_TILE		KVM_X86_CPU_FEATURE(0x7, 0, EDX, 24)
 #define	X86_FEATURE_SPEC_CTRL		KVM_X86_CPU_FEATURE(0x7, 0, EDX, 26)
@@ -1433,6 +1434,12 @@ static inline void cli(void)
 	asm volatile ("cli");
 }
 
+static inline void serialize(void)
+{
+	/* serialize instruction. binutils >= 2.35 */
+	kvm_asm_safe(".byte 0x0f, 0x01, 0xe8");
+}
+
 void __vm_xsave_require_permission(uint64_t xfeature, const char *name);
 
 #define vm_xsave_require_permission(xfeature)	\
-- 
2.45.2

[PATCH v2 32/36] KVM: selftests: Test cases for L1 APIC timer virtualization

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Test nVMX APIC timer virtualization for L1 to see how KVM in L0 works.
It exercises KVM TSC deadline conversion between L0 and L1.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
Changes:
v1 -> v2:
- Add check serialize instruction support.
- Eliminate unused variable warnings.
- Add serialize() after disabling deadline timer.
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../kvm/x86/vmx_apic_timer_virt_test.c        | 317 ++++++++++++++++++
 2 files changed, 318 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_test.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index f0023bab1cd7..424ae9a56481 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -146,6 +146,7 @@ TEST_GEN_PROGS_x86 += x86/triple_fault_event_test
 TEST_GEN_PROGS_x86 += x86/recalc_apic_map_test
 TEST_GEN_PROGS_x86 += x86/aperfmperf_test
 TEST_GEN_PROGS_x86 += x86/timer_latency
+TEST_GEN_PROGS_x86 += x86/vmx_apic_timer_virt_test
 TEST_GEN_PROGS_x86 += x86/vmx_apic_timer_virt_vmcs_test
 TEST_GEN_PROGS_x86 += access_tracking_perf_test
 TEST_GEN_PROGS_x86 += coalesced_io_test
diff --git a/tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_test.c b/tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_test.c
new file mode 100644
index 000000000000..3eb544363570
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_test.c
@@ -0,0 +1,317 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2025 Intel Corporation
+ *
+ * Test timer expiration conversion and exercise various LVTT mode.
+ */
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+#include "vmx.h"
+
+#include <string.h>
+#include <sys/ioctl.h>
+#include <stdatomic.h>
+
+#include <linux/math64.h>
+
+static uint64_t host_tsc_khz;
+static uint64_t max_guest_tsc_khz;
+
+/* Any value [32, 255] for timer vector is okay. */
+#define TIMER_VECTOR   0xec
+
+static atomic_int timer_interrupted;
+
+static void guest_timer_interrupt_handler(struct ex_regs *regs)
+{
+	atomic_fetch_add(&timer_interrupted, 1);
+	x2apic_write_reg(APIC_EOI, 0);
+}
+
+static void reap_interrupt(void)
+{
+	GUEST_ASSERT(!wrmsr_safe(MSR_IA32_TSC_DEADLINE, 0));
+	serialize();
+	sti_nop_cli();
+}
+
+static void deadline_write_test(bool do_interrupt, bool mask,
+				uint64_t deadlines[], size_t nr_deadlines)
+{
+	int i;
+
+	for (i = 0; i < nr_deadlines; i++) {
+		uint64_t deadline = deadlines[i];
+		uint64_t val;
+
+		reap_interrupt();
+
+		atomic_store(&timer_interrupted, 0);
+		sti();
+		GUEST_ASSERT(!wrmsr_safe(MSR_IA32_TSC_DEADLINE, deadline));
+		/* serialize to wait for timer interrupt to fire. */
+		serialize();
+		cli();
+
+		GUEST_ASSERT(!rdmsr_safe(MSR_IA32_TSC_DEADLINE, &val));
+
+		if (do_interrupt) {
+			GUEST_ASSERT(val == 0);
+			if (mask || deadline == 0)
+				GUEST_ASSERT_EQ(atomic_load(&timer_interrupted), 0);
+			else
+				GUEST_ASSERT_EQ(atomic_load(&timer_interrupted), 1);
+		} else {
+			GUEST_ASSERT_EQ(val, deadline);
+			GUEST_ASSERT_EQ(atomic_load(&timer_interrupted), 0);
+		}
+	}
+}
+
+static void deadline_write_hlt_test(uint64_t deadlines[], size_t nr_deadlines)
+{
+	int i;
+
+	for (i = 0; i < nr_deadlines; i++) {
+		uint64_t deadline = deadlines[i];
+		uint64_t val;
+
+		reap_interrupt();
+
+		GUEST_ASSERT(deadline);
+
+		atomic_store(&timer_interrupted, 0);
+		GUEST_ASSERT(!wrmsr_safe(MSR_IA32_TSC_DEADLINE, deadline));
+
+		GUEST_ASSERT(!rdmsr_safe(MSR_IA32_TSC_DEADLINE, &val));
+		GUEST_ASSERT(val == deadline || val == 0);
+		GUEST_ASSERT_EQ(atomic_load(&timer_interrupted), 0);
+
+		asm volatile ("sti; hlt; nop; cli"
+			      /* L1 exit handler doesn't preserve GP registers. */
+			      : : : "cc", "memory",
+				"rax", "rbx", "rcx", "rdx", "rsi", "rdi", "rbp",
+				"r8", "r9", "r10", "r11", "r12", "r13", "r14",
+				"r15");
+
+		GUEST_ASSERT(!rdmsr_safe(MSR_IA32_TSC_DEADLINE, &val));
+		GUEST_ASSERT_EQ(val, 0);
+		GUEST_ASSERT_EQ(atomic_load(&timer_interrupted), 1);
+	}
+}
+
+static void deadline_no_int_test(void)
+{
+	uint64_t tsc = rdtsc();
+	uint64_t deadlines[] = {
+		0ull,
+		/* big values > tsc. */
+		max(~0ull - tsc, ~0ull / 2 + tsc / 2),
+		~0ull - 1,
+		~0ull - 2,
+		~0ull,
+	};
+
+	deadline_write_test(false, false, deadlines, ARRAY_SIZE(deadlines));
+}
+
+static void __deadline_int_test(bool do_interrupt, bool mask)
+{
+	uint64_t tsc = rdtsc();
+	uint64_t deadlines[] = {
+		0ull,
+		1ull,
+		2ull,
+		/* 1 msec past. tsc /2 is to avoid underflow. */
+		min(tsc - guest_tsc_khz, tsc / 2 + 1),
+		tsc,
+	};
+
+	deadline_write_test(do_interrupt, mask, deadlines, ARRAY_SIZE(deadlines));
+}
+
+static void deadline_int_test(void)
+{
+	__deadline_int_test(true, false);
+}
+
+static void deadline_int_mask_test(void)
+{
+	__deadline_int_test(true, true);
+}
+
+static void deadline_hlt_test(void)
+{
+	uint64_t tsc = rdtsc();
+	/* 1 msec future. */
+	uint64_t future = tsc + guest_tsc_khz;
+	uint64_t deadlines[] = {
+		1ull,
+		2ull,
+		/* pick a positive value between [0, tsc]. */
+		tsc > guest_tsc_khz ? tsc - guest_tsc_khz : tsc / 2 + 1,
+		tsc,
+		/* If overflow, pick near future value > tsc. */
+		future > tsc ? future : ~0ull / 2 + tsc / 2,
+	};
+
+	deadline_write_hlt_test(deadlines, ARRAY_SIZE(deadlines));
+}
+
+static void guest_code(void)
+{
+	x2apic_enable();
+
+	x2apic_write_reg(APIC_LVTT, APIC_LVT_TIMER_TSCDEADLINE | TIMER_VECTOR);
+	deadline_no_int_test();
+	deadline_int_test();
+	deadline_hlt_test();
+
+	x2apic_write_reg(APIC_LVTT, APIC_LVT_TIMER_TSCDEADLINE |
+			 APIC_LVT_MASKED | TIMER_VECTOR);
+	deadline_no_int_test();
+	deadline_int_mask_test();
+
+	GUEST_DONE();
+}
+
+static void run_vcpu(struct kvm_vcpu *vcpu)
+{
+	bool done = false;
+
+	while (!done) {
+		struct ucall uc;
+
+		vcpu_run(vcpu);
+		TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_IO);
+
+		switch (get_ucall(vcpu, &uc)) {
+		case UCALL_ABORT:
+			REPORT_GUEST_ASSERT(uc);
+			/* NOT REACHED */
+		case UCALL_SYNC:
+			break;
+		case UCALL_PRINTF:
+			pr_info("%s", uc.buffer);
+			break;
+		case UCALL_DONE:
+			done = true;
+			break;
+		default:
+			TEST_FAIL("Unknown ucall %lu", uc.cmd);
+		}
+	}
+}
+
+static int test_tsc_deadline(bool tsc_offset, uint64_t guest_tsc_khz__)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+
+	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+
+	if (guest_tsc_khz__) {
+		int ret;
+
+		ret = __vcpu_ioctl(vcpu, KVM_SET_TSC_KHZ, (void *)guest_tsc_khz__);
+		if (ret) {
+			kvm_vm_free(vm);
+			return ret;
+		}
+
+		guest_tsc_khz = guest_tsc_khz__;
+	}
+
+	if (tsc_offset) {
+		uint64_t offset;
+
+		__TEST_REQUIRE(!__vcpu_has_device_attr(vcpu, KVM_VCPU_TSC_CTRL,
+						       KVM_VCPU_TSC_OFFSET),
+			       "KVM_VCPU_TSC_OFFSET not supported; skipping test");
+
+		/*
+		 * Make the conversion guest deadline virt(L1) => phy (l0)
+		 * can overflow/underflow.
+		 */
+		offset = -rdtsc();
+		vcpu_device_attr_set(vcpu, KVM_VCPU_TSC_CTRL,
+				     KVM_VCPU_TSC_OFFSET, &offset);
+	}
+
+	vcpu_set_cpuid_feature(vcpu, X86_FEATURE_TSC_DEADLINE_TIMER);
+	vm_install_exception_handler(vm, TIMER_VECTOR,
+				     guest_timer_interrupt_handler);
+
+	sync_global_to_guest(vm, host_tsc_khz);
+	sync_global_to_guest(vm, guest_tsc_khz);
+	run_vcpu(vcpu);
+
+	kvm_vm_free(vm);
+
+	return 0;
+}
+
+static void test(void)
+{
+	uint64_t guest_tsc_khz__;
+	int r;
+
+	test_tsc_deadline(false, 0);
+	test_tsc_deadline(true, 0);
+
+	for (guest_tsc_khz__ = host_tsc_khz; guest_tsc_khz__ > 0;
+	     guest_tsc_khz__ >>= 1) {
+		r = test_tsc_deadline(false, guest_tsc_khz__);
+		if (r)
+			break;
+
+		test_tsc_deadline(true, guest_tsc_khz__);
+	}
+
+	for (guest_tsc_khz__ = host_tsc_khz; guest_tsc_khz__ < max_guest_tsc_khz;
+	     guest_tsc_khz__ <<= 1) {
+		r = test_tsc_deadline(false, guest_tsc_khz__);
+		if (r)
+			break;
+
+		test_tsc_deadline(true, guest_tsc_khz__);
+	}
+
+	test_tsc_deadline(false, max_guest_tsc_khz);
+	test_tsc_deadline(true, max_guest_tsc_khz);
+}
+
+int main(int argc, char *argv[])
+{
+	uint32_t eax_denominator, ebx_numerator, ecx_hz, edx;
+
+	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_X2APIC));
+	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_SERIALIZE));
+	TEST_REQUIRE(kvm_has_cap(KVM_CAP_TSC_CONTROL));
+
+	cpuid(0x15, &eax_denominator, &ebx_numerator, &ecx_hz, &edx);
+	TEST_REQUIRE(ebx_numerator > 0);
+	TEST_REQUIRE(eax_denominator > 0);
+
+	if (ecx_hz > 0)
+		host_tsc_khz = ecx_hz * ebx_numerator / eax_denominator / 1000;
+	else {
+		uint32_t eax_base_mhz, ebx, ecx;
+
+		cpuid(0x16, &eax_base_mhz, &ebx, &ecx, &edx);
+		host_tsc_khz = eax_base_mhz * 1000 * ebx_numerator /
+			eax_denominator;
+	}
+	TEST_REQUIRE(host_tsc_khz > 0);
+
+	/* See arch/x86/kvm/{x86.c, vmx/vmx.c}. There is no way for userspace to retrieve it. */
+#define KVM_VMX_TSC_MULTIPLIER_MAX	0xffffffffffffffffULL
+	max_guest_tsc_khz = min((uint64_t)0x7fffffffULL,
+				mul_u64_u32_shr(KVM_VMX_TSC_MULTIPLIER_MAX, host_tsc_khz, 48));
+
+	test();
+
+	return 0;
+}
-- 
2.45.2

[PATCH v2 33/36] KVM: selftests: Add tests for nVMX to vmx_apic_timer_virt

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Support nVMX for vmx_apic_timer_virt.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
Changes:
v1 -> v2:
- Fix compile error.
---
 .../kvm/x86/vmx_apic_timer_virt_test.c        | 209 +++++++++++++++++-
 1 file changed, 201 insertions(+), 8 deletions(-)

diff --git a/tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_test.c b/tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_test.c
index 3eb544363570..2779d0cac7ee 100644
--- a/tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_test.c
+++ b/tools/testing/selftests/kvm/x86/vmx_apic_timer_virt_test.c
@@ -16,6 +16,15 @@
 
 #include <linux/math64.h>
 
+static bool nested;
+
+#define L2_GUEST_STACK_SIZE 256
+static unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
+
+static uint64_t l2_tsc_offset;
+static uint64_t l2_tsc_multiplier;
+static uint64_t l2_tsc_khz;
+
 static uint64_t host_tsc_khz;
 static uint64_t max_guest_tsc_khz;
 
@@ -144,14 +153,15 @@ static void deadline_int_mask_test(void)
 
 static void deadline_hlt_test(void)
 {
+	uint64_t tsc_khz = nested ? l2_tsc_khz : guest_tsc_khz;
 	uint64_t tsc = rdtsc();
 	/* 1 msec future. */
-	uint64_t future = tsc + guest_tsc_khz;
+	uint64_t future = tsc + tsc_khz + 1;
 	uint64_t deadlines[] = {
 		1ull,
 		2ull,
 		/* pick a positive value between [0, tsc]. */
-		tsc > guest_tsc_khz ? tsc - guest_tsc_khz : tsc / 2 + 1,
+		tsc > tsc_khz ? tsc - tsc_khz : tsc / 2 + 1,
 		tsc,
 		/* If overflow, pick near future value > tsc. */
 		future > tsc ? future : ~0ull / 2 + tsc / 2,
@@ -169,10 +179,134 @@ static void guest_code(void)
 	deadline_int_test();
 	deadline_hlt_test();
 
-	x2apic_write_reg(APIC_LVTT, APIC_LVT_TIMER_TSCDEADLINE |
-			 APIC_LVT_MASKED | TIMER_VECTOR);
-	deadline_no_int_test();
-	deadline_int_mask_test();
+	/* L1 doesn't emulate LVTT entry so that mask is not supported. */
+	if (!nested) {
+		x2apic_write_reg(APIC_LVTT, APIC_LVT_TIMER_TSCDEADLINE |
+				 APIC_LVT_MASKED | TIMER_VECTOR);
+		deadline_no_int_test();
+		deadline_int_mask_test();
+	}
+
+	if (nested)
+		vmcall();
+	else
+		GUEST_DONE();
+}
+
+static void skip_guest_instruction(void)
+{
+	uint64_t guest_rip, length;
+
+	GUEST_ASSERT(!vmread(GUEST_RIP, &guest_rip));
+	GUEST_ASSERT(!vmread(VM_EXIT_INSTRUCTION_LEN, &length));
+
+	GUEST_ASSERT(!vmwrite(GUEST_RIP, guest_rip + length));
+	GUEST_ASSERT(!vmwrite(VM_EXIT_INSTRUCTION_LEN, 0));
+}
+
+static void l1_guest_code(struct vmx_pages *vmx_pages)
+{
+	union vmx_ctrl_msr ctls_msr, ctls2_msr;
+	uint64_t pin, ctls, ctls2, ctls3;
+	bool launch, done;
+
+	GUEST_ASSERT(prepare_for_vmx_operation(vmx_pages));
+	GUEST_ASSERT(load_vmcs(vmx_pages));
+	prepare_vmcs(vmx_pages, guest_code,
+		     &l2_guest_stack[L2_GUEST_STACK_SIZE]);
+
+	/* Check prerequisites */
+	GUEST_ASSERT(!rdmsr_safe(MSR_IA32_VMX_PROCBASED_CTLS, &ctls_msr.val));
+	GUEST_ASSERT(ctls_msr.clr & CPU_BASED_HLT_EXITING);
+	GUEST_ASSERT(ctls_msr.clr & CPU_BASED_ACTIVATE_SECONDARY_CONTROLS);
+	GUEST_ASSERT(ctls_msr.clr & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS);
+
+	GUEST_ASSERT(!rdmsr_safe(MSR_IA32_VMX_PROCBASED_CTLS2, &ctls2_msr.val));
+	GUEST_ASSERT(ctls2_msr.clr & SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY);
+
+	GUEST_ASSERT(!rdmsr_safe(MSR_IA32_VMX_PROCBASED_CTLS3, &ctls3));
+	GUEST_ASSERT(ctls3 & TERTIARY_EXEC_GUEST_APIC_TIMER);
+
+	pin = vmreadz(PIN_BASED_VM_EXEC_CONTROL);
+	pin |= PIN_BASED_EXT_INTR_MASK;
+	GUEST_ASSERT(!vmwrite(PIN_BASED_VM_EXEC_CONTROL, pin));
+
+	ctls = vmreadz(CPU_BASED_VM_EXEC_CONTROL);
+	ctls |= CPU_BASED_HLT_EXITING | CPU_BASED_USE_TSC_OFFSETTING |
+		CPU_BASED_USE_MSR_BITMAPS | CPU_BASED_TPR_SHADOW |
+		CPU_BASED_ACTIVATE_SECONDARY_CONTROLS |
+		CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+	GUEST_ASSERT(!vmwrite(CPU_BASED_VM_EXEC_CONTROL, ctls));
+
+	/* guest apic timer requires virtual interrutp delivery */
+	ctls2 = vmreadz(SECONDARY_VM_EXEC_CONTROL);
+	ctls2 |= SECONDARY_EXEC_VIRTUALIZE_X2APIC_MODE |
+		SECONDARY_EXEC_APIC_REGISTER_VIRT |
+		SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY;
+	vmwrite(SECONDARY_VM_EXEC_CONTROL, ctls2);
+
+	ctls3 = vmreadz(TERTIARY_VM_EXEC_CONTROL);
+	ctls3 |= TERTIARY_EXEC_GUEST_APIC_TIMER;
+	GUEST_ASSERT(!vmwrite(TERTIARY_VM_EXEC_CONTROL, ctls3));
+
+	/*
+	 * We don't emulate apic registers(including APIC_LVTT) for simplicity.
+	 * Directly set vector for timer interrupt instead.
+	 */
+	GUEST_ASSERT(!vmwrite(GUEST_APIC_TIMER_VECTOR, TIMER_VECTOR));
+
+	GUEST_ASSERT(!vmwrite(TSC_OFFSET, l2_tsc_offset));
+	if (l2_tsc_multiplier) {
+		GUEST_ASSERT(!vmwrite(TSC_MULTIPLIER, l2_tsc_multiplier));
+
+		GUEST_ASSERT(!vmread(SECONDARY_VM_EXEC_CONTROL, &ctls2));
+		ctls2 |= SECONDARY_EXEC_TSC_SCALING;
+		GUEST_ASSERT(!vmwrite(SECONDARY_VM_EXEC_CONTROL, ctls2));
+	} else {
+		GUEST_ASSERT(!vmread(SECONDARY_VM_EXEC_CONTROL, &ctls2));
+		ctls2 &= ~SECONDARY_EXEC_TSC_SCALING;
+		GUEST_ASSERT(!vmwrite(SECONDARY_VM_EXEC_CONTROL, ctls2));
+	}
+
+	/* launch L2 */
+	launch = true;
+	done = false;
+
+	while (!done) {
+		uint64_t reason;
+
+		if (launch) {
+			GUEST_ASSERT(!vmlaunch());
+			launch = false;
+		} else
+			GUEST_ASSERT(!vmresume());
+
+		GUEST_ASSERT(!vmread(VM_EXIT_REASON, &reason));
+
+		switch (reason) {
+		case EXIT_REASON_HLT: {
+			uint64_t phy, tsc;
+
+			skip_guest_instruction();
+			GUEST_ASSERT(!vmread(GUEST_DEADLINE_PHY, &phy));
+
+			/* Don't wait for more than 1 sec. */
+			tsc = rdtsc();
+			if (tsc < phy && tsc < ~0ULL - guest_tsc_khz)
+				GUEST_ASSERT(tsc + guest_tsc_khz * 1000 >= tsc);
+
+			while (tsc <= phy)
+				tsc = rdtsc();
+			break;
+		}
+		case EXIT_REASON_VMCALL:
+			done = true;
+			break;
+		default:
+			GUEST_FAIL("unexpected exit reason 0x%lx", reason);
+			break;
+		}
+	}
 
 	GUEST_DONE();
 }
@@ -210,9 +344,17 @@ static int test_tsc_deadline(bool tsc_offset, uint64_t guest_tsc_khz__)
 	struct kvm_vcpu *vcpu;
 	struct kvm_vm *vm;
 
-	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+	if (nested) {
+		vm_vaddr_t vmx_pages_gva = 0;
+
+		vm = vm_create_with_one_vcpu(&vcpu, l1_guest_code);
+		vcpu_alloc_vmx(vm, &vmx_pages_gva);
+		vcpu_args_set(vcpu, 1, vmx_pages_gva);
+	} else
+		vm = vm_create_with_one_vcpu(&vcpu, guest_code);
 
 	if (guest_tsc_khz__) {
+		uint64_t l1_tsc_multiplier;
 		int ret;
 
 		ret = __vcpu_ioctl(vcpu, KVM_SET_TSC_KHZ, (void *)guest_tsc_khz__);
@@ -222,7 +364,34 @@ static int test_tsc_deadline(bool tsc_offset, uint64_t guest_tsc_khz__)
 		}
 
 		guest_tsc_khz = guest_tsc_khz__;
-	}
+
+		/*
+		 * Pick same to L1 tsc multplier.  Any value to exercise
+		 * corner cases is okay.
+		 */
+		l1_tsc_multiplier = ((__uint128_t)guest_tsc_khz__ *
+				     (1ULL << 48)) / host_tsc_khz;
+		l2_tsc_multiplier = l1_tsc_multiplier;
+		/*
+		 * l1_multiplier * l2_multiplier needs to be represented in
+		 * the host.
+		 */
+		if ((__uint128_t)l1_tsc_multiplier * l2_tsc_multiplier >
+		    ((__uint128_t)1ULL << (63 + 48))) {
+
+			l2_tsc_multiplier = ((__uint128_t)1ULL << (63 + 48)) /
+				l1_tsc_multiplier;
+			if (!l2_tsc_multiplier)
+				l1_tsc_multiplier = 1;
+		}
+
+		l2_tsc_khz = ((__uint128_t)l2_tsc_multiplier * guest_tsc_khz__) >> 48;
+		if (!l2_tsc_khz) {
+			l2_tsc_multiplier = 1ULL << 48;
+			l2_tsc_khz = guest_tsc_khz__;
+		}
+	} else
+		l2_tsc_khz = host_tsc_khz;
 
 	if (tsc_offset) {
 		uint64_t offset;
@@ -238,6 +407,9 @@ static int test_tsc_deadline(bool tsc_offset, uint64_t guest_tsc_khz__)
 		offset = -rdtsc();
 		vcpu_device_attr_set(vcpu, KVM_VCPU_TSC_CTRL,
 				     KVM_VCPU_TSC_OFFSET, &offset);
+
+		/* Pick a non-zero value */
+		l2_tsc_offset = offset;
 	}
 
 	vcpu_set_cpuid_feature(vcpu, X86_FEATURE_TSC_DEADLINE_TIMER);
@@ -246,10 +418,18 @@ static int test_tsc_deadline(bool tsc_offset, uint64_t guest_tsc_khz__)
 
 	sync_global_to_guest(vm, host_tsc_khz);
 	sync_global_to_guest(vm, guest_tsc_khz);
+	sync_global_to_guest(vm, nested);
+	sync_global_to_guest(vm, l2_tsc_offset);
+	sync_global_to_guest(vm, l2_tsc_multiplier);
+	sync_global_to_guest(vm, l2_tsc_khz);
 	run_vcpu(vcpu);
 
 	kvm_vm_free(vm);
 
+	l2_tsc_offset = 0;
+	l2_tsc_multiplier = 0;
+	l2_tsc_khz = 0;
+
 	return 0;
 }
 
@@ -286,6 +466,8 @@ static void test(void)
 int main(int argc, char *argv[])
 {
 	uint32_t eax_denominator, ebx_numerator, ecx_hz, edx;
+	union vmx_ctrl_msr ctls;
+	uint64_t ctls3;
 
 	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_X2APIC));
 	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_SERIALIZE));
@@ -313,5 +495,16 @@ int main(int argc, char *argv[])
 
 	test();
 
+	ctls.val = kvm_get_feature_msr(MSR_IA32_VMX_TRUE_PROCBASED_CTLS);
+	if (ctls.clr & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS)
+		ctls3 = kvm_get_feature_msr(MSR_IA32_VMX_PROCBASED_CTLS3);
+	else
+		ctls3 = 0;
+	if (kvm_cpu_has(X86_FEATURE_VMX) &&
+	    ctls3 & TERTIARY_EXEC_GUEST_APIC_TIMER) {
+		nested = true;
+		test();
+	}
+
 	return 0;
 }
-- 
2.45.2

[PATCH v2 34/36] KVM: selftests: Add a global option to disable in-kernel irqchip

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

For test cases with in-kernel irqchip/apic disabled.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
--
Changes v1 -> v2:
- Newly added.
---
 tools/testing/selftests/kvm/include/x86/kvm_util_arch.h | 1 +
 tools/testing/selftests/kvm/lib/x86/processor.c         | 5 ++++-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/kvm/include/x86/kvm_util_arch.h b/tools/testing/selftests/kvm/include/x86/kvm_util_arch.h
index be35d26bb320..fe502956b2ad 100644
--- a/tools/testing/selftests/kvm/include/x86/kvm_util_arch.h
+++ b/tools/testing/selftests/kvm/include/x86/kvm_util_arch.h
@@ -9,6 +9,7 @@
 #include "test_util.h"
 
 extern bool is_forced_emulation_enabled;
+extern bool disable_inkernel_irqchip;
 
 struct pte_masks {
 	uint64_t present;
diff --git a/tools/testing/selftests/kvm/lib/x86/processor.c b/tools/testing/selftests/kvm/lib/x86/processor.c
index f4e8649071b6..2342ea227638 100644
--- a/tools/testing/selftests/kvm/lib/x86/processor.c
+++ b/tools/testing/selftests/kvm/lib/x86/processor.c
@@ -27,6 +27,7 @@ bool host_cpu_is_hygon;
 bool host_cpu_is_amd_compatible;
 bool is_forced_emulation_enabled;
 uint64_t guest_tsc_khz;
+bool disable_inkernel_irqchip;
 
 const char *ex_str(int vector)
 {
@@ -789,7 +790,8 @@ void kvm_arch_vm_post_create(struct kvm_vm *vm, unsigned int nr_vcpus)
 	TEST_ASSERT(kvm_has_cap(KVM_CAP_GET_TSC_KHZ),
 		    "Require KVM_GET_TSC_KHZ to provide udelay() to guest.");
 
-	vm_create_irqchip(vm);
+	if (!disable_inkernel_irqchip)
+		vm_create_irqchip(vm);
 	vm_init_descriptor_tables(vm);
 
 	sync_global_to_guest(vm, host_cpu_is_intel);
@@ -798,6 +800,7 @@ void kvm_arch_vm_post_create(struct kvm_vm *vm, unsigned int nr_vcpus)
 	sync_global_to_guest(vm, host_cpu_is_amd_compatible);
 	sync_global_to_guest(vm, is_forced_emulation_enabled);
 	sync_global_to_guest(vm, pmu_errata_mask);
+	sync_global_to_guest(vm, disable_inkernel_irqchip);
 
 	if (is_sev_vm(vm)) {
 		struct kvm_sev_init init = { 0 };
-- 
2.45.2

[PATCH v2 35/36] KVM: selftests: Test VMX apic timer virt with inkernel apic disable

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Add a test case to excercize APIC timer virtuliazation support with
inkernel APIC disabled as Syzbot reported GP fault if inkernel APIC is
disabled.

Reported-by: syzbot+ci66a37fb2e2f8de71@syzkaller.appspotmail.com
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
--
Changes v1 -> v2:
- Newly added.
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../kvm/x86/vmx_tertiary_ctls_test.c          | 168 ++++++++++++++++++
 2 files changed, 169 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/vmx_tertiary_ctls_test.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 424ae9a56481..928b665f8086 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -148,6 +148,7 @@ TEST_GEN_PROGS_x86 += x86/aperfmperf_test
 TEST_GEN_PROGS_x86 += x86/timer_latency
 TEST_GEN_PROGS_x86 += x86/vmx_apic_timer_virt_test
 TEST_GEN_PROGS_x86 += x86/vmx_apic_timer_virt_vmcs_test
+TEST_GEN_PROGS_x86 += x86/vmx_tertiary_ctls_test
 TEST_GEN_PROGS_x86 += access_tracking_perf_test
 TEST_GEN_PROGS_x86 += coalesced_io_test
 TEST_GEN_PROGS_x86 += dirty_log_perf_test
diff --git a/tools/testing/selftests/kvm/x86/vmx_tertiary_ctls_test.c b/tools/testing/selftests/kvm/x86/vmx_tertiary_ctls_test.c
new file mode 100644
index 000000000000..9ef120e00d6d
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/vmx_tertiary_ctls_test.c
@@ -0,0 +1,168 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * VMX tertiary controls test
+ *
+ * Copyright (C) 2026 Intel Corporation
+ */
+#include "kvm_util.h"
+#include "vmx.h"
+
+static bool has_guest_apic_timer_virt = true;
+#define TIMER_VECTOR	0xec
+
+#define L2_GUEST_STACK_SIZE 64
+static unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
+
+static void l2_guest_code(void)
+{
+	vmcall();
+
+	/* L1 stops L2 vcpu at above vmcall(). */
+	GUEST_FAIL("L2 should not reach here.");
+}
+
+static void setup_l2(struct vmx_pages *vmx_pages)
+{
+	uint64_t ctls;
+
+	GUEST_ASSERT(prepare_for_vmx_operation(vmx_pages));
+	GUEST_ASSERT(load_vmcs(vmx_pages));
+	prepare_vmcs(vmx_pages, l2_guest_code,
+		     &l2_guest_stack[L2_GUEST_STACK_SIZE]);
+
+	GUEST_ASSERT(!vmread(PIN_BASED_VM_EXEC_CONTROL, &ctls));
+	ctls |= PIN_BASED_EXT_INTR_MASK;
+	GUEST_ASSERT(!vmwrite(PIN_BASED_VM_EXEC_CONTROL, ctls));
+
+	GUEST_ASSERT(!vmread(CPU_BASED_VM_EXEC_CONTROL, &ctls));
+	ctls |= CPU_BASED_USE_MSR_BITMAPS | CPU_BASED_TPR_SHADOW |
+		CPU_BASED_ACTIVATE_SECONDARY_CONTROLS;
+	ctls |= CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
+	GUEST_ASSERT(!vmwrite(CPU_BASED_VM_EXEC_CONTROL, ctls));
+
+	GUEST_ASSERT(!vmread(SECONDARY_VM_EXEC_CONTROL, &ctls));
+	ctls |= SECONDARY_EXEC_VIRTUALIZE_X2APIC_MODE |
+		SECONDARY_EXEC_APIC_REGISTER_VIRT |
+		SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY;
+	GUEST_ASSERT(!vmwrite(SECONDARY_VM_EXEC_CONTROL, ctls));
+
+	GUEST_ASSERT(!vmread(TERTIARY_VM_EXEC_CONTROL, &ctls));
+	ctls |= TERTIARY_EXEC_GUEST_APIC_TIMER;
+	/*
+	 * vmwrite() succeeds even if unsuported bit is set.
+	 * follwoing vmenter will fail instead.
+	 */
+	GUEST_ASSERT(!vmwrite(TERTIARY_VM_EXEC_CONTROL, ctls));
+
+	if (has_guest_apic_timer_virt) {
+		GUEST_ASSERT(!vmwrite(GUEST_DEADLINE_VIR, 0));
+		GUEST_ASSERT(!vmwrite(GUEST_DEADLINE_PHY, 0));
+		GUEST_ASSERT(!vmwrite(GUEST_APIC_TIMER_VECTOR, TIMER_VECTOR));
+	} else {
+		GUEST_ASSERT(vmwrite(GUEST_DEADLINE_VIR, 0));
+		GUEST_ASSERT(vmwrite(GUEST_DEADLINE_PHY, 0));
+		GUEST_ASSERT(vmwrite(GUEST_APIC_TIMER_VECTOR, TIMER_VECTOR));
+	}
+}
+
+static void l1_guest_code(struct vmx_pages *vmx_pages)
+{
+	/* Prepare the VMCS for L2 execution. */
+	setup_l2(vmx_pages);
+
+	/* Run L2 */
+	if (!has_guest_apic_timer_virt) {
+		uint64_t ctls3;
+
+		GUEST_ASSERT(vmlaunch());
+
+		GUEST_ASSERT(!vmread(TERTIARY_VM_EXEC_CONTROL, &ctls3));
+		ctls3 &= ~TERTIARY_EXEC_GUEST_APIC_TIMER;
+		GUEST_ASSERT(!vmwrite(TERTIARY_VM_EXEC_CONTROL, ctls3));
+	}
+	GUEST_ASSERT(!vmlaunch());
+	GUEST_ASSERT(vmreadz(VM_EXIT_REASON) == EXIT_REASON_VMCALL);
+
+	GUEST_DONE();
+}
+
+static void vmx_tertiary_controls_test(void)
+{
+	uint64_t ctls, ctls3, r;
+	struct kvm_vcpu *vcpu;
+	vm_vaddr_t guest_gva;
+	struct kvm_vm *vm;
+	bool done;
+
+	vm = vm_create_with_one_vcpu(&vcpu, l1_guest_code);
+	vcpu_alloc_vmx(vm, &guest_gva);
+	vcpu_args_set(vcpu, 1, guest_gva);
+
+	ctls = vcpu_get_msr(vcpu, MSR_IA32_VMX_TRUE_PROCBASED_CTLS);
+	ctls |= (uint64_t)CPU_BASED_ACTIVATE_TERTIARY_CONTROLS << 32;
+	vcpu_set_msr(vcpu, MSR_IA32_VMX_TRUE_PROCBASED_CTLS, ctls);
+
+	ctls3 = vcpu_get_msr(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3);
+	ctls3 |= TERTIARY_EXEC_GUEST_APIC_TIMER;
+
+	r = _vcpu_set_msr(vcpu, MSR_IA32_VMX_PROCBASED_CTLS3, ctls3);
+	if (has_guest_apic_timer_virt)
+		TEST_ASSERT_MSR(r == 1,
+			       "KVM_SET_MSR on %s failed value = 0x%lx",
+				MSR_IA32_VMX_PROCBASED_CTLS3,
+				"MSR_IA32_VMX_PROCBASED_CTLS3",
+				ctls3);
+	else {
+		TEST_ASSERT_MSR(r != 1,
+				"KVM_SET_MSR on %s should fail value = 0x%lx",
+				MSR_IA32_VMX_PROCBASED_CTLS3,
+				"MSR_IA32_VMX_PROCBASED_CTLS3",
+				ctls3);
+	}
+	sync_global_to_guest(vm, has_guest_apic_timer_virt);
+
+	done = false;
+	while (!done) {
+		struct ucall uc;
+
+		vcpu_run(vcpu);
+		TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_IO);
+
+		switch (get_ucall(vcpu, &uc)) {
+		case UCALL_ABORT:
+			REPORT_GUEST_ASSERT(uc);
+			/* NOT REACHED */
+		case UCALL_DONE:
+			done = true;
+			break;
+		default:
+			TEST_ASSERT(false, "Unknown ucall %lu", uc.cmd);
+		}
+	}
+
+	kvm_vm_free(vm);
+}
+
+int main(void)
+{
+	union vmx_ctrl_msr ctls;
+	uint64_t ctls3;
+
+	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_VMX));
+
+	ctls.val = kvm_get_feature_msr(MSR_IA32_VMX_TRUE_PROCBASED_CTLS);
+	TEST_REQUIRE(ctls.clr & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS);
+
+	ctls3 = kvm_get_feature_msr(MSR_IA32_VMX_PROCBASED_CTLS3);
+	has_guest_apic_timer_virt = ctls3 & TERTIARY_EXEC_GUEST_APIC_TIMER;
+
+	disable_inkernel_irqchip = false;
+	vmx_tertiary_controls_test();
+
+	/* nested apic timer virtualization requires in-kernel apic */
+	disable_inkernel_irqchip = true;
+	has_guest_apic_timer_virt = false;
+	vmx_tertiary_controls_test();
+
+	return 0;
+}
-- 
2.45.2

[PATCH v2 36/36] Documentation: KVM: x86: Update documentation of struct vmcs12

[isaku.yamahata]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Update struct vmcs12 in the documentation to match the current
implementation.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 Documentation/virt/kvm/x86/nested-vmx.rst | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/Documentation/virt/kvm/x86/nested-vmx.rst b/Documentation/virt/kvm/x86/nested-vmx.rst
index ac2095d41f02..0269a89a1beb 100644
--- a/Documentation/virt/kvm/x86/nested-vmx.rst
+++ b/Documentation/virt/kvm/x86/nested-vmx.rst
@@ -113,7 +113,15 @@ struct shadow_vmcs is ever changed.
 		u64 guest_pdptr3;
 		u64 host_ia32_pat;
 		u64 host_ia32_efer;
-		u64 padding64[8]; /* room for future expansion */
+		u64 host_ia32_perf_global_ctrl;
+		u64 vmread_bitmap;
+		u64 vmwrite_bitmap;
+		u64 vm_function_control;
+		u64 eptp_list_address;
+		u64 pml_address;
+		u64 encls_exiting_bitmap;
+		u64 tsc_multiplier;
+		u64 tertiary_vm_exec_control;
 		natural_width cr0_guest_host_mask;
 		natural_width cr4_guest_host_mask;
 		natural_width cr0_read_shadow;
@@ -217,6 +225,10 @@ struct shadow_vmcs is ever changed.
 		u16 host_fs_selector;
 		u16 host_gs_selector;
 		u16 host_tr_selector;
+		u16 guest_pml_index;
+		u16 virtual_timer_vector;
+		u64 guest_deadline;
+		u64 guest_deadline_shadow;
 	};
 
 
-- 
2.45.2

Re: [PATCH v2 16/36] KVM: nVMX: Enable guest deadline and its shadow VMCS field

[kernel test robot]

Hi,

kernel test robot noticed the following build warnings:

[auto build test WARNING on 5128b972fb2801ad9aca54d990a75611ab5283a9]

url:    https://github.com/intel-lab-lkp/linux/commits/isaku-yamahata-intel-com/KVM-VMX-Detect-APIC-timer-virtualization-bit/20260306-020026
base:   5128b972fb2801ad9aca54d990a75611ab5283a9
patch link:    https://lore.kernel.org/r/e9d6845e7727f297bd702f5a05c8006f88fc7e8f.1772732517.git.isaku.yamahata%40intel.com
patch subject: [PATCH v2 16/36] KVM: nVMX: Enable guest deadline and its shadow VMCS field
config: i386-randconfig-061-20260306 (https://download.01.org/0day-ci/archive/20260306/202603061942.wdSsERnL-lkp@intel.com/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0
sparse: v0.6.5-rc1
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260306/202603061942.wdSsERnL-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202603061942.wdSsERnL-lkp@intel.com/

sparse warnings: (new ones prefixed by >>)
   arch/x86/kvm/vmx/nested.c: note: in included file (through arch/x86/kvm/vmx/vmx_onhyperv.h, arch/x86/kvm/vmx/vmx_ops.h, arch/x86/kvm/vmx/vmx.h, ...):
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100790 becomes 790)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a000a becomes a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190019 becomes 19)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190019 becomes 19)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80a88 becomes a88)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80608 becomes 608)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80108 becomes 108)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80388 becomes 388)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20482 becomes 482)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80b88 becomes b88)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100910 becomes 910)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80188 becomes 188)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80208 becomes 208)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80288 becomes 288)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a000a becomes a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100010 becomes 10)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100710 becomes 710)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100090 becomes 90)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20402 becomes 402)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80b88 becomes b88)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100790 becomes 790)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80d08 becomes d08)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100490 becomes 490)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100310 becomes 310)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100590 becomes 590)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100610 becomes 610)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100690 becomes 690)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120912 becomes 912)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100590 becomes 590)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a0a1a becomes a1a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a0a9a becomes a9a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a0b1a becomes b1a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a0a1a becomes a1a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a0a9a becomes a9a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a0b1a becomes b1a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20002 becomes 2)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20082 becomes 82)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20102 becomes 102)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20182 becomes 182)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20202 becomes 202)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20282 becomes 282)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20302 becomes 302)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20382 becomes 382)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120012 becomes 12)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120092 becomes 92)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120112 becomes 112)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120192 becomes 192)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120212 becomes 212)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120292 becomes 292)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120312 becomes 312)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120392 becomes 392)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120412 becomes 412)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120492 becomes 492)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120592 becomes 592)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120612 becomes 612)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120512 becomes 512)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120692 becomes 692)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120712 becomes 712)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120792 becomes 792)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120812 becomes 812)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120892 becomes 892)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a019a becomes 19a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a021a becomes 21a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a029a becomes 29a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a031a becomes 31a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a039a becomes 39a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a041a becomes 41a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a049a becomes 49a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a051a becomes 51a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a059a becomes 59a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a061a becomes 61a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120a92 becomes a92)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a089a becomes 89a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a091a becomes 91a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a099a becomes 99a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a028a becomes 28a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a030a becomes 30a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a038a becomes 38a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a040a becomes 40a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a048a becomes 48a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80b08 becomes b08)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100190 becomes 190)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100210 becomes 210)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100190 becomes 190)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100210 becomes 210)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80708 becomes 708)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80788 becomes 788)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80808 becomes 808)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80888 becomes 888)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100390 becomes 390)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100390 becomes 390)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100410 becomes 410)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100510 becomes 510)
>> arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a0c0a becomes c0a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (81388 becomes 1388)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a008a becomes 8a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a008a becomes 8a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a048a becomes 48a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (180018 becomes 18)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a010a becomes 10a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a010a becomes 10a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80408 becomes 408)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80c88 becomes c88)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (180118 becomes 118)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (180198 becomes 198)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a009a becomes 9a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a028a becomes 28a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a030a becomes 30a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a038a becomes 38a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a040a becomes 40a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80508 becomes 508)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80488 becomes 488)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100090 becomes 90)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80488 becomes 488)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80588 becomes 588)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100010 becomes 10)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100090 becomes 90)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100090 becomes 90)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a008a becomes 8a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a048a becomes 48a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a009a becomes 9a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20402 becomes 402)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20402 becomes 402)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a089a becomes 89a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20402 becomes 402)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20002 becomes 2)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20082 becomes 82)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20102 becomes 102)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20182 becomes 182)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20202 becomes 202)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20282 becomes 282)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20302 becomes 302)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20382 becomes 382)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120012 becomes 12)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120092 becomes 92)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120112 becomes 112)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120192 becomes 192)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120212 becomes 212)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120292 becomes 292)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120312 becomes 312)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120392 becomes 392)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120412 becomes 412)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120492 becomes 492)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120512 becomes 512)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120692 becomes 692)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120712 becomes 712)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120792 becomes 792)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120812 becomes 812)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120892 becomes 892)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a019a becomes 19a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a021a becomes 21a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a029a becomes 29a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a031a becomes 31a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a039a becomes 39a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a041a becomes 41a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a049a becomes 49a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a051a becomes 51a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a059a becomes 59a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a061a becomes 61a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a089a becomes 89a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (81388 becomes 1388)
>> arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a0c0a becomes c0a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a001a becomes 1a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (180118 becomes 118)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a011a becomes 11a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (180198 becomes 198)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a081a becomes 81a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120592 becomes 592)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120612 becomes 612)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120912 becomes 912)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a009a becomes 9a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a028a becomes 28a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a030a becomes 30a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a038a becomes 38a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a040a becomes 40a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190299 becomes 299)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (20402 becomes 402)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110391 becomes 391)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (180098 becomes 98)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120a92 becomes a92)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a091a becomes 91a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a099a becomes 99a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a061a becomes 61a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a059a becomes 59a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120492 becomes 492)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (120412 becomes 412)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a048a becomes 48a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a010a becomes 10a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a008a becomes 8a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a010a becomes 10a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a069a becomes 69a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a008a becomes 8a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a008a becomes 8a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a018a becomes 18a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (180118 becomes 118)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (180098 becomes 98)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (180198 becomes 198)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (1a009a becomes 9a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110011 becomes 11)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100390 becomes 390)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100410 becomes 410)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100510 becomes 510)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80408 becomes 408)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80c88 becomes c88)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190019 becomes 19)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110391 becomes 391)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190019 becomes 19)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110391 becomes 391)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190019 becomes 19)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110391 becomes 391)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (100790 becomes 790)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (a000a becomes a)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190019 becomes 19)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110391 becomes 391)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110391 becomes 391)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190019 becomes 19)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110391 becomes 391)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190019 becomes 19)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110111 becomes 111)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190019 becomes 19)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190019 becomes 19)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190019 becomes 19)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190019 becomes 19)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110391 becomes 391)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110111 becomes 111)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110111 becomes 111)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110011 becomes 11)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110111 becomes 111)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110191 becomes 191)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (190019 becomes 19)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (110311 becomes 311)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80988 becomes 988)
   arch/x86/kvm/vmx/hyperv_evmcs.h:133:30: sparse: sparse: cast truncates bits from constant value (80a08 becomes a08)

vim +133 arch/x86/kvm/vmx/hyperv_evmcs.h

e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  128  
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  129  static __always_inline int evmcs_field_offset(unsigned long field,
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  130  					      u16 *clean_field)
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  131  {
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  132  	const struct evmcs_field *evmcs_field;
c68feb605cc4743 Sean Christopherson 2026-01-15 @133  	unsigned int index = ENC_TO_VMCS12_IDX(field);
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  134  
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  135  	if (unlikely(index >= nr_evmcs_1_fields))
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  136  		return -ENOENT;
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  137  
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  138  	evmcs_field = &vmcs_field_to_evmcs_1[index];
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  139  
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  140  	/*
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  141  	 * Use offset=0 to detect holes in eVMCS. This offset belongs to
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  142  	 * 'revision_id' but this field has no encoding and is supposed to
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  143  	 * be accessed directly.
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  144  	 */
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  145  	if (unlikely(!evmcs_field->offset))
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  146  		return -ENOENT;
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  147  
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  148  	if (clean_field)
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  149  		*clean_field = evmcs_field->clean_field;
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  150  
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  151  	return evmcs_field->offset;
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  152  }
e7ad84db4d718e1 Vitaly Kuznetsov    2023-12-05  153  

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH v2 00/36] KVM: VMX APIC timer virtualization support

[Chao Gao]

On Thu, Mar 05, 2026 at 09:43:40AM -0800, isaku.yamahata@intel.com wrote:
>From: Isaku Yamahata <isaku.yamahata@intel.com>
>
>This patch series is v2 of VMX APIC timer virtualization support.

Hi,

I'm picking up the APIC timer virtualization work that Isaku posted [1].
While reviewing the series, I noticed that APIC timer virtualization and
the VMX preemption timer share fundamental properties:

  - Both run only in guest mode, requiring KVM to switch to a software
    hrtimer when the vCPU blocks.
  - Both are armed/disarmed by writing a VMCS field (a countdown for the
    preemption timer, an absolute TSC deadline for APIC timer
    virtualization).

These are exactly the properties the existing hv_timer infrastructure
was designed for. This v2 series treats APIC timer virtualization as
an entirely new mechanism, introducing a separate apic_virt_timer_in_use
flag with five dedicated hooks and helpers to arm/disarm and manage state
transitions in the common x86 flows (see [2]). I think we can avoid that.

The idea is to reuse set_hv_timer()/cancel_hv_timer() for APIC timer
virtualization, with two additions:

First, the pieces specific to APIC timer virtualization:

  - Configure the timer interrupt vector in the VMCS.
  - Disable MSR interception for TSC_DEADLINE when the guest is in
    deadline mode (so writes go directly to hardware).
  - Re-enable interception when the guest leaves deadline mode or masks
    the timer.

These are naturally scoped to LVTT register writes, so a single
update_lvtt() vendor hook is sufficient.

Second, tracking the timer state. With the preemption timer, KVM
intercepts every MSR_IA32_TSC_DEADLINE write and knows precisely when a
timer is armed. With APIC timer virtualization, the MSR is not
intercepted and the guest arms and re-arms the timer without KVM's
knowledge. When the vCPU halts, KVM needs to know whether a timer is
armed and its deadline to set up the software wakeup timer.

The solution is straightforward: on VM-exit, read back the deadline from
the VMCS and update KVM's existing tracking (hv_timer_in_use and
ktimer->tscdeadline). This is one vmcs_read64() per VM-exit, negligible
compared to the exit cost itself. The existing vcpu_block() path then
works unchanged.

I prototyped this approach [3] for enabling APIC timer virtualization for
L1 guests (nested support isn't considered). The result is roughly a third
of the code (110 LoC vs. 300 LoC), with details of hypervisor timers
(preemption timer or APIC timer virtualization) fully contained in VMX. The
only change to common x86 code is a single kvm_x86_call in apic_update_lvtt().

I plan to post a series based on this approach and would appreciate
feedback on the direction before doing so.

Thanks,
Chao

[1] https://lore.kernel.org/kvm/cover.1772732517.git.isaku.yamahata@intel.com/
[2] https://lore.kernel.org/kvm/727502c8f79850fae6a8a476a624e5902861db39.1772732517.git.isaku.yamahata@intel.com/
[3] https://github.com/gaochaointel/linux-dev/commits/43056263022fe57cc9a10ebf2828f0f29611d6c