* [PATCH v4 17/24] x86/virt/seamldr: Do TDX per-CPU initialization after updates
From: Chao Gao @ 2026-02-12 14:35 UTC (permalink / raw)
To: linux-coco, linux-kernel, kvm, x86
Cc: reinette.chatre, ira.weiny, kai.huang, dan.j.williams, yilun.xu,
sagis, vannapurve, paulmck, nik.borisov, zhenzhong.duan, seanjc,
rick.p.edgecombe, kas, dave.hansen, vishal.l.verma, binbin.wu,
tony.lindgren, Chao Gao, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin
In-Reply-To: <20260212143606.534586-1-chao.gao@intel.com>
After installing the new TDX module, each CPU should be initialized
again to make the CPU ready to run any other SEAMCALLs. So, call
tdx_cpu_enable() on all CPUs.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
---
arch/x86/virt/vmx/tdx/seamldr.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
index 4537311780b1..e29e6094c80b 100644
--- a/arch/x86/virt/vmx/tdx/seamldr.c
+++ b/arch/x86/virt/vmx/tdx/seamldr.c
@@ -200,6 +200,7 @@ enum tdp_state {
TDP_START,
TDP_SHUTDOWN,
TDP_CPU_INSTALL,
+ TDP_CPU_INIT,
TDP_DONE,
};
@@ -260,6 +261,9 @@ static int do_seamldr_install_module(void *seamldr_params)
args.rcx = __pa(seamldr_params);
ret = seamldr_call(P_SEAMLDR_INSTALL, &args);
break;
+ case TDP_CPU_INIT:
+ ret = tdx_cpu_enable();
+ break;
default:
break;
}
--
2.47.3
^ permalink raw reply related
* [PATCH v4 18/24] x86/virt/tdx: Restore TDX Module state
From: Chao Gao @ 2026-02-12 14:35 UTC (permalink / raw)
To: linux-coco, linux-kernel, kvm, x86
Cc: reinette.chatre, ira.weiny, kai.huang, dan.j.williams, yilun.xu,
sagis, vannapurve, paulmck, nik.borisov, zhenzhong.duan, seanjc,
rick.p.edgecombe, kas, dave.hansen, vishal.l.verma, binbin.wu,
tony.lindgren, Chao Gao, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin
In-Reply-To: <20260212143606.534586-1-chao.gao@intel.com>
TDX Module state was packed as handoff data during module shutdown. After
per-CPU initialization, the new module can restore TDX Module state from
handoff data to preserve running TDs.
Once the restoration is done, the TDX Module update is complete, which
means the new module is ready to handle requests from the host and guests.
Implement the new TDH.SYS.UPDATE SEAMCALL to restore TDX Module state
and invoke it for one CPU.
Note that Intel® Trust Domain Extensions (Intel® TDX) Module Base
Architecture Specification, Revision 348549-007, Chapter 4.5.5 states:
If TDH.SYS.UPDATE returns an error, then the host VMM can continue
with the non-update sequence (TDH.SYS.CONFIG, 15 TDH.SYS.KEY.CONFIG
etc.). In this case all existing TDs are lost. Alternatively, the host
VMM can request the P-SEAMLDR to update to another TDX Module. If that
update is successful, existing TDs are preserved
The two alternative error handling approaches are not implemented due to
their complexity and unclear benefits.
Also note that the location and the format of handoff data is defined by
the TDX Module. The new module knows where to get handoff data and how
to parse it. The kernel doesn't need to provide its location, format etc.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
---
v3:
- use seamcall_prerr() rather than raw seamcall() [Binbin]
- use pr_err() to print error message [Binbin]
---
arch/x86/virt/vmx/tdx/seamldr.c | 5 +++++
arch/x86/virt/vmx/tdx/tdx.c | 16 ++++++++++++++++
arch/x86/virt/vmx/tdx/tdx.h | 2 ++
3 files changed, 23 insertions(+)
diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
index e29e6094c80b..0ca802234695 100644
--- a/arch/x86/virt/vmx/tdx/seamldr.c
+++ b/arch/x86/virt/vmx/tdx/seamldr.c
@@ -201,6 +201,7 @@ enum tdp_state {
TDP_SHUTDOWN,
TDP_CPU_INSTALL,
TDP_CPU_INIT,
+ TDP_RUN_UPDATE,
TDP_DONE,
};
@@ -264,6 +265,10 @@ static int do_seamldr_install_module(void *seamldr_params)
case TDP_CPU_INIT:
ret = tdx_cpu_enable();
break;
+ case TDP_RUN_UPDATE:
+ if (primary)
+ ret = tdx_module_run_update();
+ break;
default:
break;
}
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index a1193efc1156..a8adb2c97e2f 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1202,6 +1202,22 @@ int tdx_module_shutdown(void)
return 0;
}
+int tdx_module_run_update(void)
+{
+ struct tdx_module_args args = {};
+ int ret;
+
+ ret = seamcall_prerr(TDH_SYS_UPDATE, &args);
+ if (ret) {
+ pr_err("TDX-Module update failed (%d)\n", ret);
+ tdx_module_status = TDX_MODULE_ERROR;
+ return ret;
+ }
+
+ tdx_module_status = TDX_MODULE_INITIALIZED;
+ return 0;
+}
+
static bool is_pamt_page(unsigned long phys)
{
struct tdmr_info_list *tdmr_list = &tdx_tdmr_list;
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index 1c4da9540ae0..0887debfd139 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -47,6 +47,7 @@
#define TDH_VP_WR 43
#define TDH_SYS_CONFIG 45
#define TDH_SYS_SHUTDOWN 52
+#define TDH_SYS_UPDATE 53
/*
* SEAMCALL leaf:
@@ -120,5 +121,6 @@ struct tdmr_info_list {
};
int tdx_module_shutdown(void);
+int tdx_module_run_update(void);
#endif
--
2.47.3
^ permalink raw reply related
* [PATCH v4 19/24] x86/virt/tdx: Update tdx_sysinfo and check features post-update
From: Chao Gao @ 2026-02-12 14:35 UTC (permalink / raw)
To: linux-coco, linux-kernel, kvm, x86
Cc: reinette.chatre, ira.weiny, kai.huang, dan.j.williams, yilun.xu,
sagis, vannapurve, paulmck, nik.borisov, zhenzhong.duan, seanjc,
rick.p.edgecombe, kas, dave.hansen, vishal.l.verma, binbin.wu,
tony.lindgren, Chao Gao, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin
In-Reply-To: <20260212143606.534586-1-chao.gao@intel.com>
tdx_sysinfo contains all metadata of the active TDX module, including
versions, supported features, and TDMR/TDCS/TDVPS information. These
values may change over updates. Blindly refreshing the entire tdx_sysinfo
could disrupt running software, as it may subtly rely on the previous state
unless proven otherwise.
Adopt a conservative approach, like microcode updates, by only refreshing
version information that does not affect functionality, while ignoring
all other changes. This is acceptable as new modules are required to
maintain backward compatibility.
Any updates to metadata beyond versions should be justified and reviewed on
a case-by-case basis.
Note that preallocating a tdx_sys_info buffer before updates is to avoid
having to handle -ENOMEM when updating tdx_sysinfo after a successful
update.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
---
v3:
- use 'old' instead of 'cur' as the local variable to represent the
sysinfo of the previous module [Binbin]
- combine if(ret) and WARN_ONCE(1, ...) to WARN_ONCE(ret, ...) [Binbin]
- Improve the print log messages after detecting new features from updates.
[Binbin]
v2:
- don't add a separate function for version and feature checks. Do them
directly in tdx_module_post_update()
- add a comment about preallocating a tdx_sys_info buffer in
seamldr_install_module().
---
arch/x86/virt/vmx/tdx/seamldr.c | 11 ++++++++-
arch/x86/virt/vmx/tdx/tdx.c | 43 +++++++++++++++++++++++++++++++++
arch/x86/virt/vmx/tdx/tdx.h | 3 +++
3 files changed, 56 insertions(+), 1 deletion(-)
diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
index 0ca802234695..3f37cc6c68ff 100644
--- a/arch/x86/virt/vmx/tdx/seamldr.c
+++ b/arch/x86/virt/vmx/tdx/seamldr.c
@@ -315,6 +315,15 @@ int seamldr_install_module(const u8 *data, u32 size)
if (WARN_ON_ONCE(!is_vmalloc_addr(data)))
return -EINVAL;
+ /*
+ * Preallocating a tdx_sys_info buffer before updates is to avoid having to
+ * handle -ENOMEM when updating tdx_sysinfo after a successful update.
+ */
+ struct tdx_sys_info *sysinfo __free(kfree) = kzalloc(sizeof(*sysinfo),
+ GFP_KERNEL);
+ if (!sysinfo)
+ return -ENOMEM;
+
struct seamldr_params *params __free(free_seamldr_params) =
init_seamldr_params(data, size);
if (IS_ERR(params))
@@ -332,6 +341,6 @@ int seamldr_install_module(const u8 *data, u32 size)
if (ret)
return ret;
- return 0;
+ return tdx_module_post_update(sysinfo);
}
EXPORT_SYMBOL_FOR_MODULES(seamldr_install_module, "tdx-host");
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index a8adb2c97e2f..3f5edbc33a4f 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1218,6 +1218,49 @@ int tdx_module_run_update(void)
return 0;
}
+/*
+ * Update tdx_sysinfo and check if any TDX module features changed after
+ * updates
+ */
+int tdx_module_post_update(struct tdx_sys_info *info)
+{
+ struct tdx_sys_info_version *old, *new;
+ int ret;
+
+ /* Shouldn't fail as the update has succeeded */
+ ret = get_tdx_sys_info(info);
+ if (WARN_ONCE(ret, "version retrieval failed after update, replace TDX Module\n"))
+ return ret;
+
+ old = &tdx_sysinfo.version;
+ new = &info->version;
+ pr_info("version %u.%u.%02u -> %u.%u.%02u\n", old->major_version,
+ old->minor_version,
+ old->update_version,
+ new->major_version,
+ new->minor_version,
+ new->update_version);
+
+ /*
+ * Blindly refreshing the entire tdx_sysinfo could disrupt running
+ * software, as it may subtly rely on the previous state unless
+ * proven otherwise.
+ *
+ * Only refresh version information (including handoff version)
+ * that does not affect functionality, and ignore all other
+ * changes.
+ */
+ tdx_sysinfo.version = info->version;
+ tdx_sysinfo.handoff = info->handoff;
+
+ if (!memcmp(&tdx_sysinfo, info, sizeof(*info)))
+ return 0;
+
+ pr_info("TDX module features have changed after updates, but might not take effect.\n");
+ pr_info("Please consider updating your BIOS to install the TDX Module.\n");
+ return 0;
+}
+
static bool is_pamt_page(unsigned long phys)
{
struct tdmr_info_list *tdmr_list = &tdx_tdmr_list;
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index 0887debfd139..d1807a476d3b 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -4,6 +4,8 @@
#include <linux/bits.h>
+#include <asm/tdx_global_metadata.h>
+
/*
* This file contains both macros and data structures defined by the TDX
* architecture and Linux defined software data structures and functions.
@@ -122,5 +124,6 @@ struct tdmr_info_list {
int tdx_module_shutdown(void);
int tdx_module_run_update(void);
+int tdx_module_post_update(struct tdx_sys_info *info);
#endif
--
2.47.3
^ permalink raw reply related
* [PATCH v4 20/24] x86/virt/tdx: Enable TDX Module runtime updates
From: Chao Gao @ 2026-02-12 14:35 UTC (permalink / raw)
To: linux-coco, linux-kernel, kvm, x86
Cc: reinette.chatre, ira.weiny, kai.huang, dan.j.williams, yilun.xu,
sagis, vannapurve, paulmck, nik.borisov, zhenzhong.duan, seanjc,
rick.p.edgecombe, kas, dave.hansen, vishal.l.verma, binbin.wu,
tony.lindgren, Chao Gao, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin
In-Reply-To: <20260212143606.534586-1-chao.gao@intel.com>
All pieces of TDX Module runtime updates are in place. Enable it if it
is supported.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
---
v4:
- s/BIT/BIT_ULL [Tony]
---
arch/x86/include/asm/tdx.h | 5 ++++-
arch/x86/virt/vmx/tdx/tdx.h | 3 ---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index ffadbf64d0c1..ad62a7be0443 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -32,6 +32,9 @@
#define TDX_SUCCESS 0ULL
#define TDX_RND_NO_ENTROPY 0x8000020300000000ULL
+/* Bit definitions of TDX_FEATURES0 metadata field */
+#define TDX_FEATURES0_TD_PRESERVING BIT_ULL(1)
+#define TDX_FEATURES0_NO_RBP_MOD BIT_ULL(18)
#ifndef __ASSEMBLER__
#include <uapi/asm/mce.h>
@@ -105,7 +108,7 @@ const struct tdx_sys_info *tdx_get_sysinfo(void);
static inline bool tdx_supports_runtime_update(const struct tdx_sys_info *sysinfo)
{
- return false; /* To be enabled when kernel is ready */
+ return sysinfo->features.tdx_features0 & TDX_FEATURES0_TD_PRESERVING;
}
int tdx_guest_keyid_alloc(void);
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index d1807a476d3b..749f4d74cb2c 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -88,9 +88,6 @@ struct tdmr_info {
DECLARE_FLEX_ARRAY(struct tdmr_reserved_area, reserved_areas);
} __packed __aligned(TDMR_INFO_ALIGNMENT);
-/* Bit definitions of TDX_FEATURES0 metadata field */
-#define TDX_FEATURES0_NO_RBP_MOD BIT(18)
-
/*
* Do not put any hardware-defined TDX structure representations below
* this comment!
--
2.47.3
^ permalink raw reply related
* [PATCH v4 21/24] x86/virt/tdx: Avoid updates during update-sensitive operations
From: Chao Gao @ 2026-02-12 14:35 UTC (permalink / raw)
To: linux-coco, linux-kernel, kvm, x86
Cc: reinette.chatre, ira.weiny, kai.huang, dan.j.williams, yilun.xu,
sagis, vannapurve, paulmck, nik.borisov, zhenzhong.duan, seanjc,
rick.p.edgecombe, kas, dave.hansen, vishal.l.verma, binbin.wu,
tony.lindgren, Chao Gao, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin, Paolo Bonzini
In-Reply-To: <20260212143606.534586-1-chao.gao@intel.com>
TDX Module updates may cause TD management operations to fail if they
occur during phases of the TD lifecycle that are sensitive to update
compatibility.
Currently, there are two update-sensitive scenarios:
- TD build, where TD Measurement Register (TDMR) accumulates over multiple
TDH.MEM.PAGE.ADD, TDH.MR.EXTEND and TDH.MR.FINALIZE calls.
- TD migration, where an intermediate crypto state is saved if a state
migration function (TDH.EXPORT.STATE.* or TDH.IMPORT.STATE.*) is
interrupted and restored when the function is resumed.
For example, if an update races with TD build operations, the TD
Measurement Register will become incorrect, causing the TD to fail
attestation.
The TDX Module offers two solutions:
1. Avoid updates during update-sensitive times
The host VMM can instruct TDH.SYS.SHUTDOWN to fail if any of the TDs
are currently in any update-sensitive cases.
2. Detect incompatibility after updates
On TDH.SYS.UPDATE, the host VMM can configure the TDX Module to detect
actual incompatibility cases. The TDX Module will then return a special
error to signal the incompatibility, allowing the host VMM to restart
the update-sensitive operations.
Implement option #1 to fail updates if the feature is available. Also,
distinguish this update failure from other failures by returning -EBUSY,
which will be converted to a firmware update error code indicating that the
firmware is busy.
Options like "do nothing" or option #2 are not viable [1] because the
former allows damage to propagate to multiple, potentially unknown
components (adding significant complexity to the whole ecosystem), while
the latter may make existing KVM ioctls unstable.
Based on a reference patch by Vishal [2].
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Link: https://lore.kernel.org/linux-coco/aQIbM5m09G0FYTzE@google.com/ # [1]
Link: https://lore.kernel.org/linux-coco/CAGtprH_oR44Vx9Z0cfxvq5-QbyLmy_+Gn3tWm3wzHPmC1nC0eg@mail.gmail.com/ # [2]
---
arch/x86/include/asm/tdx.h | 13 +++++++++++--
arch/x86/kvm/vmx/tdx_errno.h | 2 --
arch/x86/virt/vmx/tdx/tdx.c | 23 +++++++++++++++++++----
3 files changed, 30 insertions(+), 8 deletions(-)
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index ad62a7be0443..50a58160deef 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -26,15 +26,19 @@
#define TDX_SEAMCALL_GP (TDX_SW_ERROR | X86_TRAP_GP)
#define TDX_SEAMCALL_UD (TDX_SW_ERROR | X86_TRAP_UD)
+#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL
+
/*
* TDX module SEAMCALL leaf function error codes
*/
-#define TDX_SUCCESS 0ULL
-#define TDX_RND_NO_ENTROPY 0x8000020300000000ULL
+#define TDX_SUCCESS 0ULL
+#define TDX_RND_NO_ENTROPY 0x8000020300000000ULL
+#define TDX_UPDATE_COMPAT_SENSITIVE 0x8000051200000000ULL
/* Bit definitions of TDX_FEATURES0 metadata field */
#define TDX_FEATURES0_TD_PRESERVING BIT_ULL(1)
#define TDX_FEATURES0_NO_RBP_MOD BIT_ULL(18)
+#define TDX_FEATURES0_UPDATE_COMPAT BIT_ULL(47)
#ifndef __ASSEMBLER__
#include <uapi/asm/mce.h>
@@ -111,6 +115,11 @@ static inline bool tdx_supports_runtime_update(const struct tdx_sys_info *sysinf
return sysinfo->features.tdx_features0 & TDX_FEATURES0_TD_PRESERVING;
}
+static inline bool tdx_supports_update_compatibility(const struct tdx_sys_info *sysinfo)
+{
+ return sysinfo->features.tdx_features0 & TDX_FEATURES0_UPDATE_COMPAT;
+}
+
int tdx_guest_keyid_alloc(void);
u32 tdx_get_nr_guest_keyids(void);
void tdx_guest_keyid_free(unsigned int keyid);
diff --git a/arch/x86/kvm/vmx/tdx_errno.h b/arch/x86/kvm/vmx/tdx_errno.h
index 6ff4672c4181..215c00d76a94 100644
--- a/arch/x86/kvm/vmx/tdx_errno.h
+++ b/arch/x86/kvm/vmx/tdx_errno.h
@@ -4,8 +4,6 @@
#ifndef __KVM_X86_TDX_ERRNO_H
#define __KVM_X86_TDX_ERRNO_H
-#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL
-
/*
* TDX SEAMCALL Status Codes (returned in RAX)
*/
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 3f5edbc33a4f..2cf3a01d0b9c 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1176,10 +1176,13 @@ int tdx_enable(void)
}
EXPORT_SYMBOL_FOR_KVM(tdx_enable);
+#define TDX_SYS_SHUTDOWN_AVOID_COMPAT_SENSITIVE BIT(16)
+
int tdx_module_shutdown(void)
{
struct tdx_module_args args = {};
- int ret, cpu;
+ u64 ret;
+ int cpu;
/*
* Shut down the TDX Module and prepare handoff data for the next
@@ -1189,9 +1192,21 @@ int tdx_module_shutdown(void)
* modules as new modules likely have higher handoff version.
*/
args.rcx = tdx_sysinfo.handoff.module_hv;
- ret = seamcall_prerr(TDH_SYS_SHUTDOWN, &args);
- if (ret)
- return ret;
+
+ if (tdx_supports_update_compatibility(&tdx_sysinfo))
+ args.rcx |= TDX_SYS_SHUTDOWN_AVOID_COMPAT_SENSITIVE;
+
+ ret = seamcall(TDH_SYS_SHUTDOWN, &args);
+
+ /*
+ * Return -EBUSY to signal that there is one or more ongoing flows
+ * which may not be compatible with an updated TDX module, so that
+ * userspace can retry on this error.
+ */
+ if ((ret & TDX_SEAMCALL_STATUS_MASK) == TDX_UPDATE_COMPAT_SENSITIVE)
+ return -EBUSY;
+ else if (ret)
+ return -EIO;
tdx_module_status = TDX_MODULE_UNINITIALIZED;
sysinit_done = false;
--
2.47.3
^ permalink raw reply related
* [PATCH v4 23/24] x86/virt/tdx: Document TDX Module updates
From: Chao Gao @ 2026-02-12 14:35 UTC (permalink / raw)
To: linux-coco, linux-kernel, kvm, x86, linux-doc
Cc: reinette.chatre, ira.weiny, kai.huang, dan.j.williams, yilun.xu,
sagis, vannapurve, paulmck, nik.borisov, zhenzhong.duan, seanjc,
rick.p.edgecombe, kas, dave.hansen, vishal.l.verma, binbin.wu,
tony.lindgren, Chao Gao, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin, Jonathan Corbet
In-Reply-To: <20260212143606.534586-1-chao.gao@intel.com>
Document TDX Module updates as a subsection of "TDX Host Kernel Support" to
provide background information and cover key points that developers and
users may need to know, for example:
- update is done in stop_machine() context
- update instructions and results
- update policy and tooling
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
Documentation/arch/x86/tdx.rst | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/Documentation/arch/x86/tdx.rst b/Documentation/arch/x86/tdx.rst
index 61670e7df2f7..01ae560c7f66 100644
--- a/Documentation/arch/x86/tdx.rst
+++ b/Documentation/arch/x86/tdx.rst
@@ -99,6 +99,40 @@ initialize::
[..] virt/tdx: module initialization failed ...
+TDX Module Runtime Updates
+--------------------------
+
+The TDX architecture includes a persistent SEAM loader (P-SEAMLDR) that
+runs in SEAM mode separately from the TDX Module. The kernel can
+communicate with P-SEAMLDR to perform runtime updates of the TDX Module.
+
+During updates, the TDX Module becomes unresponsive to other TDX
+operations. To prevent components using TDX (such as KVM) from experiencing
+unexpected errors during updates, updates are performed in stop_machine()
+context.
+
+TDX Module updates have complex compatibility requirements; the new module
+must be compatible with the current CPU, P-SEAMLDR, and running TDX Module.
+Rather than implementing complex module selection and policy enforcement
+logic in the kernel, userspace is responsible for auditing and selecting
+appropriate updates.
+
+Updates use the standard firmware upload interface. See
+Documentation/driver-api/firmware/fw_upload.rst for detailed instructions
+
+Successful updates are logged in dmesg:
+ [..] virt/tdx: version 1.5.20 -> 1.5.24
+
+If updates failed, running TDs may be killed and further TDX operations may
+be not possible until reboot. For detailed error information, see
+Documentation/ABI/testing/sysfs-devices-faux-tdx-host.
+
+Given the risk of losing existing TDs, userspace should verify that the update
+is compatible with the current system and properly validated before applying it.
+A reference userspace tool that implements necessary checks is available at:
+
+ https://github.com/intel/confidential-computing.tdx.tdx-module.binaries
+
TDX Interaction to Other Kernel Components
------------------------------------------
--
2.47.3
^ permalink raw reply related
* [PATCH v4 22/24] coco/tdx-host: Document TDX Module update expectations
From: Chao Gao @ 2026-02-12 14:35 UTC (permalink / raw)
To: linux-coco, linux-kernel, kvm, x86
Cc: reinette.chatre, ira.weiny, kai.huang, dan.j.williams, yilun.xu,
sagis, vannapurve, paulmck, nik.borisov, zhenzhong.duan, seanjc,
rick.p.edgecombe, kas, dave.hansen, vishal.l.verma, binbin.wu,
tony.lindgren, Chao Gao
In-Reply-To: <20260212143606.534586-1-chao.gao@intel.com>
The TDX Module update protocol facilitates compatible runtime updates.
Document the compatibility criteria and indicators of various update
failures, including violations of the compatibility criteria.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
---
v4
- Drop "compat_capable" kernel ABI [Dan]
- Document Linux compatibility expectations and results of violating
them [Dan]
---
.../ABI/testing/sysfs-devices-faux-tdx-host | 53 +++++++++++++++++++
1 file changed, 53 insertions(+)
diff --git a/Documentation/ABI/testing/sysfs-devices-faux-tdx-host b/Documentation/ABI/testing/sysfs-devices-faux-tdx-host
index 88a9c0b2bdfe..fefe762998db 100644
--- a/Documentation/ABI/testing/sysfs-devices-faux-tdx-host
+++ b/Documentation/ABI/testing/sysfs-devices-faux-tdx-host
@@ -27,3 +27,56 @@ Description: (RO) Report the number of remaining updates. TDX maintains a
Interface Specification, Revision 343755-003, Chapter 3.3
"SEAMLDR_INFO" and Chapter 4.2 "SEAMLDR.INSTALL" for more
information.
+
+What: /sys/devices/faux/tdx_host/firmware/tdx_module
+Contact: linux-coco@lists.linux.dev
+Description: (Directory) The tdx_module directory implements the fw_upload
+ sysfs ABI, see Documentation/ABI/testing/sysfs-class-firmware
+ for the general description of the attributes @data, @cancel,
+ @error, @loading, @remaining_size, and @status. This ABI
+ facilitates "Compatible TDX Module Updates". A compatible update
+ is one that meets the following criteria:
+
+ Does not interrupt or interfere with any current TDX
+ operation or TD VM.
+
+ Does not invalidate any previously consumed Module metadata
+ values outside of the TEE_TCB_SVN_2 field (updated Security
+ Version Number) in TD Quotes.
+
+ Does not require validation of new Module metadata fields. By
+ implication, new Module features and capabilities are only
+ available by installing the Module at reboot (BIOS or EFI
+ helper loaded).
+
+ See tdx_host/firmware/tdx_module/error for information on
+ compatibility check failures and how to prevent them.
+
+What: /sys/devices/faux/tdx_host/firmware/tdx_module/error
+Contact: linux-coco@lists.linux.dev
+Description: (RO) See Documentation/ABI/testing/sysfs-class-firmware for
+ baseline expectations for this file. The <ERROR> part in the
+ <STATUS>:<ERROR> format can be:
+
+ "device-busy": Compatibility checks failed or not all CPUs
+ are online
+
+ "flash-wearout": The number of updates reached the limit.
+
+ "read-write-error": Memory allocation failed.
+
+ "hw-error": Cannot communicate with P-SEAMLDR or TDX Module.
+
+ "firmware-invalid": The provided TDX Module update is invalid
+ or other unexpected errors occurred.
+
+ "hw-error" or "firmware-invalid" may be fatal, causing all TDs
+ and the TDX Module to be lost and preventing further TDX
+ operations. This occurs when reading
+ /sys/devices/faux/tdx_host/version returns -ENXIO. For other
+ errors, TDs and the (previous) TDX Module stay running.
+
+ See tdxctl [1] documentation for how to detect compatible
+ updates and whether the current platform components catch errors
+ or let them leak and cause potential TD attestation failures.
+ [1]: <TBD - tdxctl link>
--
2.47.3
^ permalink raw reply related
* [PATCH v4 24/24] [NOT-FOR-REVIEW] x86/virt/seamldr: Save and restore current VMCS
From: Chao Gao @ 2026-02-12 14:35 UTC (permalink / raw)
To: linux-coco, linux-kernel, kvm, x86
Cc: reinette.chatre, ira.weiny, kai.huang, dan.j.williams, yilun.xu,
sagis, vannapurve, paulmck, nik.borisov, zhenzhong.duan, seanjc,
rick.p.edgecombe, kas, dave.hansen, vishal.l.verma, binbin.wu,
tony.lindgren, Chao Gao, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin
In-Reply-To: <20260212143606.534586-1-chao.gao@intel.com>
P-SEAMLDR calls clobber the current VMCS as documented in Intel® Trust
Domain CPU Architectural Extensions (May 2021 edition) Chapter 2.3 [1]:
SEAMRET from the P-SEAMLDR clears the current VMCS structure pointed
to by the current-VMCS pointer. A VMM that invokes the P-SEAMLDR using
SEAMCALL must reload the current-VMCS, if required, using the VMPTRLD
instruction.
Save and restore the current VMCS using VMPTRST and VMPTRLD instructions
to avoid breaking KVM.
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
This patch is needed for testing until microcode is updated to preserve
the current VMCS across P-SEAMLDR calls. Otherwise, if some normal VMs
are running before TDX Module updates, vmread/vmwrite errors may occur
immediately after updates.
---
arch/x86/include/asm/special_insns.h | 22 ++++++++++++++++++++++
arch/x86/virt/vmx/tdx/seamldr.c | 16 +++++++++++++++-
2 files changed, 37 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/special_insns.h
index 46aa2c9c1bda..a3e9a139b669 100644
--- a/arch/x86/include/asm/special_insns.h
+++ b/arch/x86/include/asm/special_insns.h
@@ -303,6 +303,28 @@ static __always_inline void tile_release(void)
asm volatile(".byte 0xc4, 0xe2, 0x78, 0x49, 0xc0");
}
+static inline int vmptrst(u64 *vmcs_pa)
+{
+ asm goto("1: vmptrst %0\n\t"
+ _ASM_EXTABLE(1b, %l[error])
+ : "=m" (*vmcs_pa) : : "cc" : error);
+
+ return 0;
+error:
+ return -EIO;
+}
+
+static inline int vmptrld(u64 vmcs_pa)
+{
+ asm goto("1: vmptrld %0\n\t"
+ "jna %l[error]\n\t"
+ _ASM_EXTABLE(1b, %l[error])
+ : : "m" (vmcs_pa) : "cc" : error);
+ return 0;
+error:
+ return -EIO;
+}
+
#endif /* __KERNEL__ */
#endif /* _ASM_X86_SPECIAL_INSNS_H */
diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
index 3f37cc6c68ff..02695307b8a0 100644
--- a/arch/x86/virt/vmx/tdx/seamldr.c
+++ b/arch/x86/virt/vmx/tdx/seamldr.c
@@ -16,6 +16,7 @@
#include <linux/stop_machine.h>
#include <asm/seamldr.h>
+#include <asm/special_insns.h>
#include "seamcall_internal.h"
#include "tdx.h"
@@ -59,12 +60,25 @@ static DEFINE_RAW_SPINLOCK(seamldr_lock);
static int seamldr_call(u64 fn, struct tdx_module_args *args)
{
+ u64 current_vmcs = -1ULL;
+ int ret;
+
/*
* Serialize P-SEAMLDR calls and disable interrupts as the calls
* can be made from IRQ context.
*/
guard(raw_spinlock_irqsave)(&seamldr_lock);
- return seamcall_prerr(fn, args);
+
+ /*
+ * P-SEAMLDR calls clobber the current VMCS. Save and restore it.
+ * -1 indicates invalid VMCS and no restoration is needed.
+ */
+ WARN_ON_ONCE(vmptrst(¤t_vmcs));
+ ret = seamcall_prerr(fn, args);
+ if (current_vmcs != -1ULL)
+ WARN_ON_ONCE(vmptrld(current_vmcs));
+
+ return ret;
}
int seamldr_get_info(struct seamldr_info *seamldr_info)
--
2.47.3
^ permalink raw reply related
* Re: [PATCH v4 00/24] Runtime TDX Module update support
From: Chao Gao @ 2026-02-12 14:46 UTC (permalink / raw)
To: linux-coco, linux-kernel, kvm, x86, linux-doc
Cc: reinette.chatre, ira.weiny, kai.huang, dan.j.williams, yilun.xu,
sagis, vannapurve, paulmck, nik.borisov, zhenzhong.duan, seanjc,
rick.p.edgecombe, kas, dave.hansen, vishal.l.verma, binbin.wu,
tony.lindgren, Borislav Petkov, H. Peter Anvin, Ingo Molnar,
Jonathan Corbet, Paolo Bonzini, Thomas Gleixner
In-Reply-To: <20260212143606.534586-1-chao.gao@intel.com>
On Thu, Feb 12, 2026 at 06:35:03AM -0800, Chao Gao wrote:
>
>Note that like v3, this v4 is not based on Sean's VMXON series to make this
>series more reviewable.
There are some conflicts between Sean's VMXON v2 and this TDX module update series:
1. tdx_cpu_enable() is unexported in the VMXON series but used in this series.
2. tdx_module_status is removed in the VMXON series but accessed in this series.
3. Several functions are tagged as __init but called in this series at runtime
Below is a sample diff showing how to resolve the conflicts. This series is not
ready for merge yet. The diff is posted just to give you a sense of how these
two series intersect:
diff --cc arch/x86/include/asm/tdx.h
index a149740b24e8,50a58160deef..000000000000
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@@ -97,57 -104,22 +104,21 @@@ static inline long tdx_kvm_hypercall(un
#endif /* CONFIG_INTEL_TDX_GUEST && CONFIG_KVM_GUEST */
#ifdef CONFIG_INTEL_TDX_HOST
- u64 __seamcall(u64 fn, struct tdx_module_args *args);
- u64 __seamcall_ret(u64 fn, struct tdx_module_args *args);
- u64 __seamcall_saved_ret(u64 fn, struct tdx_module_args *args);
void tdx_init(void);
+ int tdx_cpu_enable(void);
-int tdx_enable(void);
+ const char *tdx_dump_mce_info(struct mce *m);
+ const struct tdx_sys_info *tdx_get_sysinfo(void);
- #include <linux/preempt.h>
- #include <asm/archrandom.h>
- #include <asm/processor.h>
-
- typedef u64 (*sc_func_t)(u64 fn, struct tdx_module_args *args);
-
- static __always_inline u64 __seamcall_dirty_cache(sc_func_t func, u64 fn,
- struct tdx_module_args *args)
+ static inline bool tdx_supports_runtime_update(const struct tdx_sys_info *sysinfo)
{
- lockdep_assert_preemption_disabled();
-
- /*
- * SEAMCALLs are made to the TDX module and can generate dirty
- * cachelines of TDX private memory. Mark cache state incoherent
- * so that the cache can be flushed during kexec.
- *
- * This needs to be done before actually making the SEAMCALL,
- * because kexec-ing CPU could send NMI to stop remote CPUs,
- * in which case even disabling IRQ won't help here.
- */
- this_cpu_write(cache_state_incoherent, true);
-
- return func(fn, args);
+ return sysinfo->features.tdx_features0 & TDX_FEATURES0_TD_PRESERVING;
}
- static __always_inline u64 sc_retry(sc_func_t func, u64 fn,
- struct tdx_module_args *args)
+ static inline bool tdx_supports_update_compatibility(const struct tdx_sys_info *sysinfo)
{
- int retry = RDRAND_RETRY_LOOPS;
- u64 ret;
-
- do {
- preempt_disable();
- ret = __seamcall_dirty_cache(func, fn, args);
- preempt_enable();
- } while (ret == TDX_RND_NO_ENTROPY && --retry);
-
- return ret;
+ return sysinfo->features.tdx_features0 & TDX_FEATURES0_UPDATE_COMPAT;
}
- #define seamcall(_fn, _args) sc_retry(__seamcall, (_fn), (_args))
- #define seamcall_ret(_fn, _args) sc_retry(__seamcall_ret, (_fn), (_args))
- #define seamcall_saved_ret(_fn, _args) sc_retry(__seamcall_saved_ret, (_fn), (_args))
- const char *tdx_dump_mce_info(struct mce *m);
- const struct tdx_sys_info *tdx_get_sysinfo(void);
-
int tdx_guest_keyid_alloc(void);
u32 tdx_get_nr_guest_keyids(void);
void tdx_guest_keyid_free(unsigned int keyid);
diff --cc arch/x86/virt/vmx/tdx/tdx.c
index 55d3463e0e93,2cf3a01d0b9c..000000000000
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@@ -40,7 -39,8 +40,9 @@@
#include <asm/cpu_device_id.h>
#include <asm/processor.h>
#include <asm/mce.h>
+#include <asm/virt.h>
+
+ #include "seamcall_internal.h"
#include "tdx.h"
static u32 tdx_global_keyid __ro_after_init;
@@@ -53,57 -53,16 +55,15 @@@ static DEFINE_PER_CPU(bool, tdx_lp_init
static struct tdmr_info_list tdx_tdmr_list;
-static enum tdx_module_status_t tdx_module_status;
-static DEFINE_MUTEX(tdx_module_lock);
+ static bool sysinit_done;
+ static int sysinit_ret;
+
/* All TDX-usable memory regions. Protected by mem_hotplug_lock. */
static LIST_HEAD(tdx_memlist);
-static struct tdx_sys_info tdx_sysinfo;
+static struct tdx_sys_info tdx_sysinfo __ro_after_init;
+static bool tdx_module_initialized __ro_after_init;
- typedef void (*sc_err_func_t)(u64 fn, u64 err, struct tdx_module_args *args);
-
- static inline void seamcall_err(u64 fn, u64 err, struct tdx_module_args *args)
- {
- pr_err("SEAMCALL (0x%016llx) failed: 0x%016llx\n", fn, err);
- }
-
- static inline void seamcall_err_ret(u64 fn, u64 err,
- struct tdx_module_args *args)
- {
- seamcall_err(fn, err, args);
- pr_err("RCX 0x%016llx RDX 0x%016llx R08 0x%016llx\n",
- args->rcx, args->rdx, args->r8);
- pr_err("R09 0x%016llx R10 0x%016llx R11 0x%016llx\n",
- args->r9, args->r10, args->r11);
- }
-
- static __always_inline int sc_retry_prerr(sc_func_t func,
- sc_err_func_t err_func,
- u64 fn, struct tdx_module_args *args)
- {
- u64 sret = sc_retry(func, fn, args);
-
- if (sret == TDX_SUCCESS)
- return 0;
-
- if (sret == TDX_SEAMCALL_VMFAILINVALID)
- return -ENODEV;
-
- if (sret == TDX_SEAMCALL_GP)
- return -EOPNOTSUPP;
-
- if (sret == TDX_SEAMCALL_UD)
- return -EACCES;
-
- err_func(fn, sret, args);
- return -EIO;
- }
-
- #define seamcall_prerr(__fn, __args) \
- sc_retry_prerr(__seamcall, seamcall_err, (__fn), (__args))
-
- #define seamcall_prerr_ret(__fn, __args) \
- sc_retry_prerr(__seamcall_ret, seamcall_err_ret, (__fn), (__args))
-
/*
* Do the module global initialization once and return its result.
* It can be done on any cpu. It's always called with interrupts
@@@ -142,11 -99,17 +100,11 @@@ out
}
/**
- * tdx_cpu_enable - Enable TDX on local cpu
- *
- * Do one-time TDX module per-cpu initialization SEAMCALL (and TDX module
- * global initialization SEAMCALL if not done) on local cpu to make this
- * cpu be ready to run any other SEAMCALLs.
- *
- * Always call this function via IPI function calls.
- *
- * Return 0 on success, otherwise errors.
+ * Enable VMXON and then do one-time TDX module per-cpu initialization SEAMCALL
+ * (and TDX module global initialization SEAMCALL if not done) on local cpu to
+ * make this cpu be ready to run any other SEAMCALLs.
*/
- static int tdx_cpu_enable(void)
+ int tdx_cpu_enable(void)
{
struct tdx_module_args args = {};
int ret;
@@@ -1236,51 -1114,168 +1194,150 @@@ err_free_tdxmem
goto out_put_tdxmem;
}
-static int __tdx_enable(void)
+static __init int tdx_enable(void)
{
+ enum cpuhp_state state;
int ret;
- ret = init_tdx_module();
- if (ret) {
- pr_err("module initialization failed (%d)\n", ret);
- tdx_module_status = TDX_MODULE_ERROR;
- return ret;
+ if (!cpu_feature_enabled(X86_FEATURE_TDX_HOST_PLATFORM)) {
+ pr_err("TDX not supported by the host platform\n");
+ return -ENODEV;
}
- pr_info("module initialized\n");
- tdx_module_status = TDX_MODULE_INITIALIZED;
-
- return 0;
-}
+ if (!cpu_feature_enabled(X86_FEATURE_XSAVE)) {
+ pr_err("XSAVE is required for TDX\n");
+ return -EINVAL;
+ }
-/**
- * tdx_enable - Enable TDX module to make it ready to run TDX guests
- *
- * This function assumes the caller has: 1) held read lock of CPU hotplug
- * lock to prevent any new cpu from becoming online; 2) done both VMXON
- * and tdx_cpu_enable() on all online cpus.
- *
- * This function requires there's at least one online cpu for each CPU
- * package to succeed.
- *
- * This function can be called in parallel by multiple callers.
- *
- * Return 0 if TDX is enabled successfully, otherwise error.
- */
-int tdx_enable(void)
-{
- int ret;
+ if (!cpu_feature_enabled(X86_FEATURE_MOVDIR64B)) {
+ pr_err("MOVDIR64B is required for TDX\n");
+ return -EINVAL;
+ }
- if (!boot_cpu_has(X86_FEATURE_TDX_HOST_PLATFORM))
+ if (!cpu_feature_enabled(X86_FEATURE_SELFSNOOP)) {
+ pr_err("Self-snoop is required for TDX\n");
return -ENODEV;
+ }
- lockdep_assert_cpus_held();
-
- mutex_lock(&tdx_module_lock);
+ state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "virt/tdx:online",
+ tdx_online_cpu, tdx_offline_cpu);
+ if (state < 0)
+ return state;
- switch (tdx_module_status) {
- case TDX_MODULE_UNINITIALIZED:
- ret = __tdx_enable();
- break;
- case TDX_MODULE_INITIALIZED:
- /* Already initialized, great, tell the caller. */
- ret = 0;
- break;
- default:
- /* Failed to initialize in the previous attempts */
- ret = -EINVAL;
- break;
+ ret = init_tdx_module();
+ if (ret) {
+ pr_err("TDX-Module initialization failed (%d)\n", ret);
+ cpuhp_remove_state(state);
+ return ret;
}
- mutex_unlock(&tdx_module_lock);
+ register_syscore(&tdx_syscore);
- return ret;
+ tdx_module_initialized = true;
+ pr_info("TDX-Module initialized\n");
+ return 0;
}
-EXPORT_SYMBOL_FOR_KVM(tdx_enable);
+subsys_initcall(tdx_enable);
+ #define TDX_SYS_SHUTDOWN_AVOID_COMPAT_SENSITIVE BIT(16)
+
+ int tdx_module_shutdown(void)
+ {
+ struct tdx_module_args args = {};
+ u64 ret;
+ int cpu;
+
+ /*
+ * Shut down the TDX Module and prepare handoff data for the next
+ * TDX Module. This SEAMCALL requires a handoff version. Use the
+ * module's handoff version, as it is the highest version the
+ * module can produce and is more likely to be supported by new
+ * modules as new modules likely have higher handoff version.
+ */
+ args.rcx = tdx_sysinfo.handoff.module_hv;
+
+ if (tdx_supports_update_compatibility(&tdx_sysinfo))
+ args.rcx |= TDX_SYS_SHUTDOWN_AVOID_COMPAT_SENSITIVE;
+
+ ret = seamcall(TDH_SYS_SHUTDOWN, &args);
+
+ /*
+ * Return -EBUSY to signal that there is one or more ongoing flows
+ * which may not be compatible with an updated TDX module, so that
+ * userspace can retry on this error.
+ */
+ if ((ret & TDX_SEAMCALL_STATUS_MASK) == TDX_UPDATE_COMPAT_SENSITIVE)
+ return -EBUSY;
+ else if (ret)
+ return -EIO;
+
- tdx_module_status = TDX_MODULE_UNINITIALIZED;
++ tdx_module_initialized = false;
+ sysinit_done = false;
+ sysinit_ret = 0;
+
+ for_each_online_cpu(cpu)
+ per_cpu(tdx_lp_initialized, cpu) = false;
+ return 0;
+ }
+
+ int tdx_module_run_update(void)
+ {
+ struct tdx_module_args args = {};
+ int ret;
+
+ ret = seamcall_prerr(TDH_SYS_UPDATE, &args);
+ if (ret) {
+ pr_err("TDX-Module update failed (%d)\n", ret);
- tdx_module_status = TDX_MODULE_ERROR;
+ return ret;
+ }
+
- tdx_module_status = TDX_MODULE_INITIALIZED;
++ tdx_module_initialized = true;
+ return 0;
+ }
+
+ /*
+ * Update tdx_sysinfo and check if any TDX module features changed after
+ * updates
+ */
+ int tdx_module_post_update(struct tdx_sys_info *info)
+ {
+ struct tdx_sys_info_version *old, *new;
+ int ret;
+
+ /* Shouldn't fail as the update has succeeded */
+ ret = get_tdx_sys_info(info);
+ if (WARN_ONCE(ret, "version retrieval failed after update, replace TDX Module\n"))
+ return ret;
+
+ old = &tdx_sysinfo.version;
+ new = &info->version;
+ pr_info("version %u.%u.%02u -> %u.%u.%02u\n", old->major_version,
+ old->minor_version,
+ old->update_version,
+ new->major_version,
+ new->minor_version,
+ new->update_version);
+
+ /*
+ * Blindly refreshing the entire tdx_sysinfo could disrupt running
+ * software, as it may subtly rely on the previous state unless
+ * proven otherwise.
+ *
+ * Only refresh version information (including handoff version)
+ * that does not affect functionality, and ignore all other
+ * changes.
+ */
+ tdx_sysinfo.version = info->version;
+ tdx_sysinfo.handoff = info->handoff;
+
+ if (!memcmp(&tdx_sysinfo, info, sizeof(*info)))
+ return 0;
+
+ pr_info("TDX module features have changed after updates, but might not take effect.\n");
+ pr_info("Please consider updating your BIOS to install the TDX Module.\n");
+ return 0;
+ }
+
static bool is_pamt_page(unsigned long phys)
{
struct tdmr_info_list *tdmr_list = &tdx_tdmr_list;
@@@ -1530,12 -1525,17 +1587,12 @@@ void __init tdx_init(void
const struct tdx_sys_info *tdx_get_sysinfo(void)
{
- const struct tdx_sys_info *p = NULL;
-
- /* Make sure all fields in @tdx_sysinfo have been populated */
- mutex_lock(&tdx_module_lock);
- if (tdx_module_status == TDX_MODULE_INITIALIZED)
- p = (const struct tdx_sys_info *)&tdx_sysinfo;
- mutex_unlock(&tdx_module_lock);
+ if (!tdx_module_initialized)
+ return NULL;
- return p;
+ return (const struct tdx_sys_info *)&tdx_sysinfo;
}
- EXPORT_SYMBOL_FOR_KVM(tdx_get_sysinfo);
+ EXPORT_SYMBOL_FOR_MODULES(tdx_get_sysinfo, "kvm-intel,tdx-host");
u32 tdx_get_nr_guest_keyids(void)
{
diff --cc arch/x86/virt/vmx/tdx/tdx_global_metadata.c
index c7db393a9cfb,6aee10c36489..000000000000
--- a/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
+++ b/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
^ permalink raw reply
* Re: [PATCH v12 00/46] arm64: Support for Arm CCA in KVM
From: Mathieu Poirier @ 2026-02-12 17:48 UTC (permalink / raw)
To: Steven Price
Cc: kvm, kvmarm, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20251217101125.91098-1-steven.price@arm.com>
Hi Steven,
On Wed, Dec 17, 2025 at 10:10:37AM +0000, Steven Price wrote:
> This series adds support for running protected VMs using KVM under the
> Arm Confidential Compute Architecture (CCA). I've changed the uAPI
> following feedback from Marc.
>
> The main change is that rather than providing a multiplex CAP and
> expecting the VMM to drive the different stages of realm construction,
> there's now just a minimal interface and KVM performs the necessary
> operations when needed.
>
> This series is lightly tested and is meant as a demonstration of the new
> uAPI. There are a number of (known) rough corners in the implementation
> that I haven't dealt with properly.
>
> In particular please note that this series is still targetting RMM v1.0.
> There is an alpha quality version of RMM v2.0 available[1]. Feedback was
> that there are a number of blockers for merging with RMM v1.0 and so I
> expect to rework this series to support RMM v2.0 before it is merged.
> That will necessarily involve reworking the implementation.
>
> Specifically I'm expecting improvements in:
>
> * GIC handling - passing state in registers, and allowing the host to
> fully emulate the GIC by allowing trap bits to be set.
>
> * PMU handling - again providing flexibility to the host's emulation.
>
> * Page size/granule size mismatch. RMM v1.0 defines the granule as 4k,
> RMM v2.0 provide the option for the host to change the granule size.
> The intention is that Linux would simply set the granule size equal
> to its page size which will significantly simplify the management of
> granules.
>
> * Some performance improvement from the use of range-based map/unmap
> RMI calls.
>
> This series is based on v6.19-rc1. It is also available as a git
> repository:
>
> https://gitlab.arm.com/linux-arm/linux-cca cca-host/v12
>
> Work in progress changes for kvmtool are available from the git
> repository below:
>
> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/v10
The first thing to note is that branch cca/v10 does not compile due to function
realm_configure_parameters() not being called anywhere. Marking the function as
[[maybe_unused]] solved the problem on my side.
Using the FVP emulator, booting a Realm that includes EDK2 in its boot stack
worked. If EDK2 is not part of the boot stack and a kernel is booted directly
from lkvm, mounting the initrd fails. Looking into this issue further, I see
that from a Realm kernel's perspective, the content of the initrd is either
encrypted or has been trampled on.
I'd be happy to provide more details on the above, just let me know.
Thanks,
Mathieu
>
> [1] https://developer.arm.com/documentation/den0137/latest/
>
> Jean-Philippe Brucker (7):
> arm64: RMI: Propagate number of breakpoints and watchpoints to
> userspace
> arm64: RMI: Set breakpoint parameters through SET_ONE_REG
> arm64: RMI: Initialize PMCR.N with number counter supported by RMM
> arm64: RMI: Propagate max SVE vector length from RMM
> arm64: RMI: Configure max SVE vector length for a Realm
> arm64: RMI: Provide register list for unfinalized RMI RECs
> arm64: RMI: Provide accurate register list
>
> Joey Gouly (2):
> arm64: RMI: allow userspace to inject aborts
> arm64: RMI: support RSI_HOST_CALL
>
> Steven Price (34):
> arm64: RME: Handle Granule Protection Faults (GPFs)
> arm64: RMI: Add SMC definitions for calling the RMM
> arm64: RMI: Add wrappers for RMI calls
> arm64: RMI: Check for RMI support at KVM init
> arm64: RMI: Define the user ABI
> arm64: RMI: Basic infrastructure for creating a realm.
> KVM: arm64: Allow passing machine type in KVM creation
> arm64: RMI: RTT tear down
> arm64: RMI: Activate realm on first VCPU run
> arm64: RMI: Allocate/free RECs to match vCPUs
> KVM: arm64: vgic: Provide helper for number of list registers
> arm64: RMI: Support for the VGIC in realms
> KVM: arm64: Support timers in realm RECs
> arm64: RMI: Handle realm enter/exit
> arm64: RMI: Handle RMI_EXIT_RIPAS_CHANGE
> KVM: arm64: Handle realm MMIO emulation
> KVM: arm64: Expose support for private memory
> arm64: RMI: Allow populating initial contents
> arm64: RMI: Set RIPAS of initial memslots
> arm64: RMI: Create the realm descriptor
> arm64: RMI: Add a VMID allocator for realms
> arm64: RMI: Runtime faulting of memory
> KVM: arm64: Handle realm VCPU load
> KVM: arm64: Validate register access for a Realm VM
> KVM: arm64: Handle Realm PSCI requests
> KVM: arm64: WARN on injected undef exceptions
> arm64: Don't expose stolen time for realm guests
> arm64: RMI: Always use 4k pages for realms
> arm64: RMI: Prevent Device mappings for Realms
> HACK: Restore per-CPU cpu_armpmu pointer
> arm_pmu: Provide a mechanism for disabling the physical IRQ
> arm64: RMI: Enable PMU support with a realm guest
> KVM: arm64: Expose KVM_ARM_VCPU_REC to user space
> arm64: RMI: Enable realms to be created
>
> Suzuki K Poulose (3):
> kvm: arm64: Include kvm_emulate.h in kvm/arm_psci.h
> kvm: arm64: Don't expose unsupported capabilities for realm guests
> arm64: RMI: Allow checking SVE on VM instance
>
> Documentation/virt/kvm/api.rst | 78 +-
> arch/arm64/include/asm/kvm_emulate.h | 31 +
> arch/arm64/include/asm/kvm_host.h | 13 +-
> arch/arm64/include/asm/kvm_rmi.h | 137 +++
> arch/arm64/include/asm/rmi_cmds.h | 508 ++++++++
> arch/arm64/include/asm/rmi_smc.h | 269 +++++
> arch/arm64/include/asm/virt.h | 1 +
> arch/arm64/kernel/cpufeature.c | 1 +
> arch/arm64/kvm/Kconfig | 2 +
> arch/arm64/kvm/Makefile | 2 +-
> arch/arm64/kvm/arch_timer.c | 37 +-
> arch/arm64/kvm/arm.c | 179 ++-
> arch/arm64/kvm/guest.c | 95 +-
> arch/arm64/kvm/hypercalls.c | 4 +-
> arch/arm64/kvm/inject_fault.c | 5 +-
> arch/arm64/kvm/mmio.c | 16 +-
> arch/arm64/kvm/mmu.c | 214 +++-
> arch/arm64/kvm/pmu-emul.c | 6 +
> arch/arm64/kvm/psci.c | 30 +
> arch/arm64/kvm/reset.c | 13 +-
> arch/arm64/kvm/rmi-exit.c | 207 ++++
> arch/arm64/kvm/rmi.c | 1663 ++++++++++++++++++++++++++
> arch/arm64/kvm/sys_regs.c | 53 +-
> arch/arm64/kvm/vgic/vgic-init.c | 2 +-
> arch/arm64/kvm/vgic/vgic-v2.c | 6 +-
> arch/arm64/kvm/vgic/vgic-v3.c | 14 +-
> arch/arm64/kvm/vgic/vgic.c | 55 +-
> arch/arm64/kvm/vgic/vgic.h | 20 +-
> arch/arm64/mm/fault.c | 28 +-
> drivers/perf/arm_pmu.c | 20 +
> include/kvm/arm_arch_timer.h | 2 +
> include/kvm/arm_pmu.h | 4 +
> include/kvm/arm_psci.h | 2 +
> include/linux/perf/arm_pmu.h | 7 +
> include/uapi/linux/kvm.h | 42 +-
> 35 files changed, 3650 insertions(+), 116 deletions(-)
> create mode 100644 arch/arm64/include/asm/kvm_rmi.h
> create mode 100644 arch/arm64/include/asm/rmi_cmds.h
> create mode 100644 arch/arm64/include/asm/rmi_smc.h
> create mode 100644 arch/arm64/kvm/rmi-exit.c
> create mode 100644 arch/arm64/kvm/rmi.c
>
> --
> 2.43.0
>
>
^ permalink raw reply
* Re: [PATCH v4 22/24] coco/tdx-host: Document TDX Module update expectations
From: dan.j.williams @ 2026-02-12 21:59 UTC (permalink / raw)
To: Chao Gao, linux-coco, linux-kernel, kvm, x86
Cc: reinette.chatre, ira.weiny, kai.huang, dan.j.williams, yilun.xu,
sagis, vannapurve, paulmck, nik.borisov, zhenzhong.duan, seanjc,
rick.p.edgecombe, kas, dave.hansen, vishal.l.verma, binbin.wu,
tony.lindgren, Chao Gao
In-Reply-To: <20260212143606.534586-23-chao.gao@intel.com>
Chao Gao wrote:
> The TDX Module update protocol facilitates compatible runtime updates.
>
> Document the compatibility criteria and indicators of various update
> failures, including violations of the compatibility criteria.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Reviewed-by: Dan Williams <dan.j.williams@intel.com>
> ---
> v4
> - Drop "compat_capable" kernel ABI [Dan]
> - Document Linux compatibility expectations and results of violating
> them [Dan]
> ---
[..]
> +
> + See tdxctl [1] documentation for how to detect compatible
> + updates and whether the current platform components catch errors
> + or let them leak and cause potential TD attestation failures.
> + [1]: <TBD - tdxctl link>
Delete this paragraph. Do not carry dead documentation in the tree. You
might clarify in the changelog that until tooling arrives it is an
"update at your own risk" scenario in terms of encountering incompatible
updates. Otherwise, no point in documenting tdxctl vaporware in the
tree.
^ permalink raw reply
* Re: [RFC PATCH v5 43/45] *** DO NOT MERGE *** KVM: guest_memfd: Add pre-zap arch hook for shared<=>private conversion
From: Huang, Kai @ 2026-02-13 7:23 UTC (permalink / raw)
To: seanjc@google.com, x86@kernel.org, dave.hansen@linux.intel.com,
kas@kernel.org, bp@alien8.de, mingo@redhat.com,
pbonzini@redhat.com, tglx@kernel.org
Cc: Edgecombe, Rick P, ackerleytng@google.com, sagis@google.com,
Annapurve, Vishal, linux-kernel@vger.kernel.org, Zhao, Yan Y,
Li, Xiaoyao, kvm@vger.kernel.org, linux-coco@lists.linux.dev,
Yamahata, Isaku, binbin.wu@linux.intel.com
In-Reply-To: <20260129011517.3545883-44-seanjc@google.com>
On Wed, 2026-01-28 at 17:15 -0800, Sean Christopherson wrote:
> --- a/virt/kvm/Kconfig
> +++ b/virt/kvm/Kconfig
> @@ -125,3 +125,7 @@ config HAVE_KVM_ARCH_GMEM_INVALIDATE
> config HAVE_KVM_ARCH_GMEM_POPULATE
> bool
> depends on KVM_GUEST_MEMFD
> +
> +config HAVE_KVM_ARCH_GMEM_CONVERT
> + bool
> + depends on KVM_GUEST_MEMFD
> \ No newline at end of file
Just FYI:
It appears something went wrong when editing this file. I got below warning
when playing with this series:
virt/kvm/Kconfig:131:warning: no new line at end of file
^ permalink raw reply
* Re: [RFC PATCH v5 44/45] KVM: x86/mmu: Add support for splitting S-EPT hugepages on conversion
From: Sean Christopherson @ 2026-02-13 15:09 UTC (permalink / raw)
To: Yan Zhao
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Paolo Bonzini, linux-kernel, linux-coco, kvm,
Kai Huang, Rick Edgecombe, Vishal Annapurve, Ackerley Tng,
Sagi Shahar, Binbin Wu, Xiaoyao Li, Isaku Yamahata
In-Reply-To: <aYxBCINUG80GYfus@yzhao56-desk.sh.intel.com>
On Wed, Feb 11, 2026, Yan Zhao wrote:
> However, I have a question about kvm_tdp_mmu_try_split_huge_pages(), which is
> called by dirty page tracking related functions. I'm not sure if we might want
> to invoke them from a non-vCPU thread for mirror roots in the future. If that's
> the case, would they need some way to acquire this lock?
More than likely, yes. But that's a far future problem, at least as far as
upstream is concerned. I.e. I don't want to plan _that_ far ahead in terms of
writing code to avoid churn.
^ permalink raw reply
* Re: [RFC PATCH v5 08/45] KVM: x86/mmu: Propagate mirror SPTE removal to S-EPT in handle_changed_spte()
From: Sean Christopherson @ 2026-02-14 0:36 UTC (permalink / raw)
To: Yan Zhao
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Paolo Bonzini, linux-kernel, linux-coco, kvm,
Kai Huang, Rick Edgecombe, Vishal Annapurve, Ackerley Tng,
Sagi Shahar, Binbin Wu, Xiaoyao Li, Isaku Yamahata
In-Reply-To: <aYvmlBb6oR3lfWn2@yzhao56-desk.sh.intel.com>
On Wed, Feb 11, 2026, Yan Zhao wrote:
> On Tue, Feb 10, 2026 at 11:52:09AM -0800, Sean Christopherson wrote:
> > > > +static void handle_changed_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep,
> > > > + gfn_t gfn, u64 old_spte, u64 new_spte,
> > > > + int level, bool shared)
> > > > +{
> > > Do we need "WARN_ON_ONCE(is_mirror_sptep(sptep) && shared)" here ?
> >
> > No, because I want to call this code for all paths, including the fault path.
> Hmm. IIUC, handle_changed_spte() can't be invoked for mirror root under read
> mmu_lock.
> For read mmu_lock + mirror scenarios, they need to invoke
> tdp_mmu_set_spte_atomic() --> __handle_changed_spte().
Oh, sorry, I misread that. Now I see what you're saying. I think I'd still prefer
to omit the WARN? Because there's nothing inherently wrong with using
handle_changed_spte(). E.g. if the caller can somehow guarantee success, then
using handle_changed_spte() is a-ok.
> Besides, __handle_changed_spte() contains code like
> "kvm_update_page_stats(kvm, level, is_leaf ? 1 : -1);", which may have
> incorrectly updated the stats even if kvm_x86_call(set_external_spte)() fails
> later and the new_spte is never written to iter->sptep.
Oof, now _that_ is an actual problem. This is the least-ugly fix I can come up
with. Note, this will mean the trace order is "wrong" when removing a non-mirror
page table, as KVM will zap the page table before its children. I doubt that'll
be a problem in practice, so I'm inclined to take the simpler code.
diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
index d395da35d5e4..4ba789f2824d 100644
--- a/arch/x86/kvm/mmu/tdp_mmu.c
+++ b/arch/x86/kvm/mmu/tdp_mmu.c
@@ -493,6 +493,7 @@ static int __handle_changed_spte(struct kvm *kvm, struct kvm_mmu_page *sp,
bool is_leaf = is_present && is_last_spte(new_spte, level);
bool pfn_changed = spte_to_pfn(old_spte) != spte_to_pfn(new_spte);
int as_id = kvm_mmu_page_as_id(sp);
+ int r;
WARN_ON_ONCE(level > PT64_ROOT_MAX_LEVEL);
WARN_ON_ONCE(level < PG_LEVEL_4K);
@@ -524,8 +525,6 @@ static int __handle_changed_spte(struct kvm *kvm, struct kvm_mmu_page *sp,
if (old_spte == new_spte)
return 0;
- trace_kvm_tdp_mmu_spte_changed(as_id, gfn, level, old_spte, new_spte);
-
if (is_leaf)
check_spte_writable_invariants(new_spte);
@@ -554,9 +553,6 @@ static int __handle_changed_spte(struct kvm *kvm, struct kvm_mmu_page *sp,
return 0;
}
- if (is_leaf != was_leaf)
- kvm_update_page_stats(kvm, level, is_leaf ? 1 : -1);
-
/*
* Recursively handle child PTs if the change removed a subtree from
* the paging structure. Note the WARN on the PFN changing without the
@@ -567,11 +563,19 @@ static int __handle_changed_spte(struct kvm *kvm, struct kvm_mmu_page *sp,
* changes to the external SPTE.
*/
if (was_present && !was_leaf &&
- (is_leaf || !is_present || WARN_ON_ONCE(pfn_changed)))
+ (is_leaf || !is_present || WARN_ON_ONCE(pfn_changed))) {
handle_removed_pt(kvm, spte_to_child_pt(old_spte, level), shared);
- else if (is_mirror_sp(sp))
- return kvm_x86_call(set_external_spte)(kvm, gfn, old_spte,
- new_spte, level);
+ } else if (is_mirror_sp(sp)) {
+ r = kvm_x86_call(set_external_spte)(kvm, gfn, old_spte,
+ new_spte, level);
+ if (r)
+ return r;
+ }
+
+ trace_kvm_tdp_mmu_spte_changed(as_id, gfn, level, old_spte, new_spte);
+
+ if (is_leaf != was_leaf)
+ kvm_update_page_stats(kvm, level, is_leaf ? 1 : -1);
return 0;
}
> > > 3. set *iter->sptep to new_spte
> > >
> > > what if __handle_changed_spte() reads *iter->sptep in step 2?
> >
> > For the most part, "don't do that". There are an infinite number of "what ifs".
> > I agree that re-reading iter->sptep is slightly more likely than other "what ifs",
> > but then if we convert to a boolean it creates the "what if we swap the order of
> > @as_id and @is_mirror_sp"? Given that @old_spte is provided, IMO re-reading the
> > SPTE from memory will stand out.
> As my above concern, re-reading SPTE in __handle_changed_spte() will just get
> value FROZEN_SPTE instead of the value of new_spte.
>
> > That said, I think we can have the best of both worlds. Rather than pass @as_id
> > and @sptep, pass the @sp, i.e. the owning kvm_mmu_page. That would address your
> > concern about re-reading the sptep, without needing another boolean.
> Hmm, my intention of passing boolean is to avoid re-reading sptep, because
> in step 2, we pass new_spte instead of the real value in sptep (which is
> FROZEN_SPTE for mirror sp) to __handle_changed_spte().
> So, passing @sp may not help?
It won't prevent someone that's bound and determined to introduce a bug from
re-reading the sptep, but it most definitely helps. To get at the sptep, someone
would have to compute its index based off @gfn and then look it up in @sp->spt.
At that point, they've earned the bug :-)
^ permalink raw reply related
* [PATCH v3 00/16] KVM: x86/tdx: Have TDX handle VMXON during bringup
From: Sean Christopherson @ 2026-02-14 1:26 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun, Dan Williams
Assuming I didn't break anything between v2 and v3, I think this is ready to
rip. Given the scope of the KVM changes, and that they extend outside of x86,
my preference is to take this through the KVM tree. But a stable topic branch
in tip would work too, though I think we'd want it sooner than later so that
it can be used as a base.
Chao, I deliberately omitted your Tested-by, as I shuffled things around enough
while splitting up the main patch that I'm not 100% positive I didn't regress
anything relative to v2.
The idea here is to extract _only_ VMXON+VMXOFF and EFER.SVME toggling. AFAIK
there's no second user of SVM, i.e. no equivalent to TDX, but I wanted to keep
things as symmetrical as possible.
TDX isn't a hypervisor, and isn't trying to be a hypervisor. Specifically, TDX
should _never_ have it's own VMCSes (that are visible to the host; the
TDX-Module has it's own VMCSes to do SEAMCALL/SEAMRET), and so there is simply
no reason to move that functionality out of KVM.
With that out of the way, dealing with VMXON/VMXOFF and EFER.SVME is a fairly
simple refcounting game.
v3:
- https://lore.kernel.org/all/20251206011054.494190-1-seanjc@google.com
- Split up the move from KVM => virt into smaller patches. [Dan]
- Collect reviews. [Dan, Chao, Dave]
- Update sample dmesg output and hotplug angle in docs. [Chao]
- Add comments in kvm_arch_shutdown() to try and explain the madness. [Dave]
- Add a largely superfluous smp_wmb() in kvm_arch_shutdown() to provide a
convienent location for documenting the flow. [Dave]
- Disable preemption in x86_virt_{get,put}_ref() so that changes in how
KVM and/or TDX use the APIs doesn't result in bugs. [Xu]
- Add a patch to drop the bogus "IRQs must be disabled" rule in
tdx_cpu_enable().
- Tag more TDX helpers as __init. [Chao]
- Don't treat loading kvm-intel.ko with tdx=1 as fatal if the system doesn't
have a TDX-Module available. [Chao]
v2:
- Initialize the TDX-Module via subsys initcall instead of during
tdx_init(). [Rick]
- Isolate the __init and __ro_after_init changes. [Rick]
- Use ida_is_empty() instead of manually tracking HKID usage. [Dan]
- Don't do weird things with the refcounts when virt_rebooting is
true. [Chao]
- Drop unnecessary setting of virt_rebooting in KVM code. [Chao]
- Rework things to have less X86_FEATURE_FOO code. [Rick]
- Consolidate the CPU hotplug callbacks. [Chao]
v1 (RFC):
- https://lore.kernel.org/all/20251010220403.987927-1-seanjc@google.com
Chao Gao (1):
x86/virt/tdx: KVM: Consolidate TDX CPU hotplug handling
Sean Christopherson (15):
KVM: x86: Move kvm_rebooting to x86
KVM: VMX: Move architectural "vmcs" and "vmcs_hdr" structures to
public vmx.h
KVM: x86: Move "kvm_rebooting" to kernel as "virt_rebooting"
KVM: VMX: Unconditionally allocate root VMCSes during boot CPU bringup
x86/virt: Force-clear X86_FEATURE_VMX if configuring root VMCS fails
KVM: VMX: Move core VMXON enablement to kernel
KVM: SVM: Move core EFER.SVME enablement to kernel
KVM: x86: Move bulk of emergency virtualizaton logic to virt subsystem
x86/virt: Add refcounting of VMX/SVM usage to support multiple
in-kernel users
x86/virt/tdx: Drop the outdated requirement that TDX be enabled in IRQ
context
KVM: x86/tdx: Do VMXON and TDX-Module initialization during subsys
init
x86/virt/tdx: Tag a pile of functions as __init, and globals as
__ro_after_init
x86/virt/tdx: Use ida_is_empty() to detect if any TDs may be running
KVM: Bury kvm_{en,dis}able_virtualization() in kvm_main.c once more
KVM: TDX: Fold tdx_bringup() into tdx_hardware_setup()
Documentation/arch/x86/tdx.rst | 36 +-
arch/x86/events/intel/pt.c | 1 -
arch/x86/include/asm/kvm_host.h | 3 +-
arch/x86/include/asm/reboot.h | 11 -
arch/x86/include/asm/tdx.h | 4 -
arch/x86/include/asm/virt.h | 26 ++
arch/x86/include/asm/vmx.h | 11 +
arch/x86/kernel/cpu/common.c | 2 +
arch/x86/kernel/crash.c | 3 +-
arch/x86/kernel/reboot.c | 63 +---
arch/x86/kernel/smp.c | 5 +-
arch/x86/kvm/svm/svm.c | 34 +-
arch/x86/kvm/svm/vmenter.S | 10 +-
arch/x86/kvm/vmx/main.c | 19 +-
arch/x86/kvm/vmx/tdx.c | 210 ++----------
arch/x86/kvm/vmx/tdx.h | 8 +-
arch/x86/kvm/vmx/vmcs.h | 11 -
arch/x86/kvm/vmx/vmenter.S | 2 +-
arch/x86/kvm/vmx/vmx.c | 138 +-------
arch/x86/kvm/x86.c | 29 +-
arch/x86/virt/Makefile | 2 +
arch/x86/virt/hw.c | 359 ++++++++++++++++++++
arch/x86/virt/vmx/tdx/tdx.c | 321 +++++++++--------
arch/x86/virt/vmx/tdx/tdx.h | 8 -
arch/x86/virt/vmx/tdx/tdx_global_metadata.c | 10 +-
include/linux/kvm_host.h | 16 +-
virt/kvm/kvm_main.c | 31 +-
27 files changed, 717 insertions(+), 656 deletions(-)
create mode 100644 arch/x86/include/asm/virt.h
create mode 100644 arch/x86/virt/hw.c
base-commit: 183bb0ce8c77b0fd1fb25874112bc8751a461e49
--
2.53.0.310.g728cabbaf7-goog
^ permalink raw reply
* [PATCH v3 01/16] KVM: x86: Move kvm_rebooting to x86
From: Sean Christopherson @ 2026-02-14 1:26 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun, Dan Williams
In-Reply-To: <20260214012702.2368778-1-seanjc@google.com>
Move kvm_rebooting, which is only read by x86, to KVM x86 so that it can
be moved again to core x86 code. Add a "shutdown" arch hook to facilate
setting the flag in KVM x86, along with a pile of comments to provide more
context around what KVM x86 is doing and why.
Reviewed-by: Chao Gao <chao.gao@intel.com>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/x86.c | 22 ++++++++++++++++++++++
arch/x86/kvm/x86.h | 1 +
include/linux/kvm_host.h | 8 +++++++-
virt/kvm/kvm_main.c | 14 +++++++-------
4 files changed, 37 insertions(+), 8 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index db3f393192d9..77edc24f8309 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -700,6 +700,9 @@ static void drop_user_return_notifiers(void)
kvm_on_user_return(&msrs->urn);
}
+__visible bool kvm_rebooting;
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_rebooting);
+
/*
* Handle a fault on a hardware virtualization (VMX or SVM) instruction.
*
@@ -13178,6 +13181,25 @@ int kvm_arch_enable_virtualization_cpu(void)
return 0;
}
+void kvm_arch_shutdown(void)
+{
+ /*
+ * Set kvm_rebooting to indicate that KVM has asynchronously disabled
+ * hardware virtualization, i.e. that errors and/or exceptions on SVM
+ * and VMX instructions are expected and should be ignored.
+ */
+ kvm_rebooting = true;
+
+ /*
+ * Ensure kvm_rebooting is visible before IPIs are sent to other CPUs
+ * to disable virtualization. Effectively pairs with the reception of
+ * the IPI (kvm_rebooting is read in task/exception context, but only
+ * _needs_ to be read as %true after the IPI function callback disables
+ * virtualization).
+ */
+ smp_wmb();
+}
+
void kvm_arch_disable_virtualization_cpu(void)
{
kvm_x86_call(disable_virtualization_cpu)();
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index 94d4f07aaaa0..b314649e5c02 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -54,6 +54,7 @@ struct kvm_host_values {
u64 arch_capabilities;
};
+extern bool kvm_rebooting;
void kvm_spurious_fault(void);
#define SIZE_OF_MEMSLOTS_HASHTABLE \
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 2c7d76262898..981b55c0a3a7 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -1630,6 +1630,13 @@ static inline void kvm_create_vcpu_debugfs(struct kvm_vcpu *vcpu) {}
#endif
#ifdef CONFIG_KVM_GENERIC_HARDWARE_ENABLING
+/*
+ * kvm_arch_shutdown() is invoked immediately prior to forcefully disabling
+ * hardware virtualization on all CPUs via IPI function calls (in preparation
+ * for shutdown or reboot), e.g. to allow arch code to prepare for disabling
+ * virtualization while KVM may be actively running vCPUs.
+ */
+void kvm_arch_shutdown(void);
/*
* kvm_arch_{enable,disable}_virtualization() are called on one CPU, under
* kvm_usage_lock, immediately after/before 0=>1 and 1=>0 transitions of
@@ -2305,7 +2312,6 @@ static inline bool kvm_check_request(int req, struct kvm_vcpu *vcpu)
#ifdef CONFIG_KVM_GENERIC_HARDWARE_ENABLING
extern bool enable_virt_at_load;
-extern bool kvm_rebooting;
#endif
extern unsigned int halt_poll_ns;
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 571cf0d6ec01..e081e7244299 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -5593,13 +5593,15 @@ bool enable_virt_at_load = true;
module_param(enable_virt_at_load, bool, 0444);
EXPORT_SYMBOL_FOR_KVM_INTERNAL(enable_virt_at_load);
-__visible bool kvm_rebooting;
-EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_rebooting);
-
static DEFINE_PER_CPU(bool, virtualization_enabled);
static DEFINE_MUTEX(kvm_usage_lock);
static int kvm_usage_count;
+__weak void kvm_arch_shutdown(void)
+{
+
+}
+
__weak void kvm_arch_enable_virtualization(void)
{
@@ -5653,10 +5655,9 @@ static int kvm_offline_cpu(unsigned int cpu)
static void kvm_shutdown(void *data)
{
+ kvm_arch_shutdown();
+
/*
- * Disable hardware virtualization and set kvm_rebooting to indicate
- * that KVM has asynchronously disabled hardware virtualization, i.e.
- * that relevant errors and exceptions aren't entirely unexpected.
* Some flavors of hardware virtualization need to be disabled before
* transferring control to firmware (to perform shutdown/reboot), e.g.
* on x86, virtualization can block INIT interrupts, which are used by
@@ -5665,7 +5666,6 @@ static void kvm_shutdown(void *data)
* 100% comprehensive.
*/
pr_info("kvm: exiting hardware virtualization\n");
- kvm_rebooting = true;
on_each_cpu(kvm_disable_virtualization_cpu, NULL, 1);
}
--
2.53.0.310.g728cabbaf7-goog
^ permalink raw reply related
* [PATCH v3 02/16] KVM: VMX: Move architectural "vmcs" and "vmcs_hdr" structures to public vmx.h
From: Sean Christopherson @ 2026-02-14 1:26 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun, Dan Williams
In-Reply-To: <20260214012702.2368778-1-seanjc@google.com>
Move "struct vmcs" and "struct vmcs_hdr" to asm/vmx.h in anticipation of
moving VMXON/VMXOFF to the core kernel (VMXON requires a "root" VMCS with
the appropriate revision ID in its header).
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/vmx.h | 11 +++++++++++
arch/x86/kvm/vmx/vmcs.h | 11 -----------
2 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h
index b92ff87e3560..37080382df54 100644
--- a/arch/x86/include/asm/vmx.h
+++ b/arch/x86/include/asm/vmx.h
@@ -20,6 +20,17 @@
#include <asm/trapnr.h>
#include <asm/vmxfeatures.h>
+struct vmcs_hdr {
+ u32 revision_id:31;
+ u32 shadow_vmcs:1;
+};
+
+struct vmcs {
+ struct vmcs_hdr hdr;
+ u32 abort;
+ char data[];
+};
+
#define VMCS_CONTROL_BIT(x) BIT(VMX_FEATURE_##x & 0x1f)
/*
diff --git a/arch/x86/kvm/vmx/vmcs.h b/arch/x86/kvm/vmx/vmcs.h
index 66d747e265b1..1f16ddeae9cb 100644
--- a/arch/x86/kvm/vmx/vmcs.h
+++ b/arch/x86/kvm/vmx/vmcs.h
@@ -22,17 +22,6 @@
#define VMCS12_IDX_TO_ENC(idx) ROL16(idx, 10)
#define ENC_TO_VMCS12_IDX(enc) ROL16(enc, 6)
-struct vmcs_hdr {
- u32 revision_id:31;
- u32 shadow_vmcs:1;
-};
-
-struct vmcs {
- struct vmcs_hdr hdr;
- u32 abort;
- char data[];
-};
-
DECLARE_PER_CPU(struct vmcs *, current_vmcs);
/*
--
2.53.0.310.g728cabbaf7-goog
^ permalink raw reply related
* [PATCH v3 03/16] KVM: x86: Move "kvm_rebooting" to kernel as "virt_rebooting"
From: Sean Christopherson @ 2026-02-14 1:26 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun, Dan Williams
In-Reply-To: <20260214012702.2368778-1-seanjc@google.com>
Move "kvm_rebooting" to the kernel, exported for KVM, as one of many steps
towards extracting the innermost VMXON and EFER.SVME management logic out
of KVM and into to core x86.
For lack of a better name, call the new file "hw.c", to yield "virt
hardware" when combined with its parent directory.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/virt.h | 11 +++++++++++
arch/x86/kvm/svm/svm.c | 3 ++-
arch/x86/kvm/svm/vmenter.S | 10 +++++-----
arch/x86/kvm/vmx/tdx.c | 3 ++-
arch/x86/kvm/vmx/vmenter.S | 2 +-
arch/x86/kvm/vmx/vmx.c | 5 +++--
arch/x86/kvm/x86.c | 17 ++++++++---------
arch/x86/kvm/x86.h | 1 -
arch/x86/virt/Makefile | 2 ++
arch/x86/virt/hw.c | 7 +++++++
10 files changed, 41 insertions(+), 20 deletions(-)
create mode 100644 arch/x86/include/asm/virt.h
create mode 100644 arch/x86/virt/hw.c
diff --git a/arch/x86/include/asm/virt.h b/arch/x86/include/asm/virt.h
new file mode 100644
index 000000000000..131b9bf9ef3c
--- /dev/null
+++ b/arch/x86/include/asm/virt.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASM_X86_VIRT_H
+#define _ASM_X86_VIRT_H
+
+#include <linux/types.h>
+
+#if IS_ENABLED(CONFIG_KVM_X86)
+extern bool virt_rebooting;
+#endif
+
+#endif /* _ASM_X86_VIRT_H */
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 8f8bc863e214..0ae66c770ebc 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -44,6 +44,7 @@
#include <asm/traps.h>
#include <asm/reboot.h>
#include <asm/fpu/api.h>
+#include <asm/virt.h>
#include <trace/events/ipi.h>
@@ -495,7 +496,7 @@ static inline void kvm_cpu_svm_disable(void)
static void svm_emergency_disable_virtualization_cpu(void)
{
- kvm_rebooting = true;
+ virt_rebooting = true;
kvm_cpu_svm_disable();
}
diff --git a/arch/x86/kvm/svm/vmenter.S b/arch/x86/kvm/svm/vmenter.S
index 3392bcadfb89..d47c5c93c991 100644
--- a/arch/x86/kvm/svm/vmenter.S
+++ b/arch/x86/kvm/svm/vmenter.S
@@ -298,16 +298,16 @@ SYM_FUNC_START(__svm_vcpu_run)
RESTORE_GUEST_SPEC_CTRL_BODY
RESTORE_HOST_SPEC_CTRL_BODY (%_ASM_SP)
-10: cmpb $0, _ASM_RIP(kvm_rebooting)
+10: cmpb $0, _ASM_RIP(virt_rebooting)
jne 2b
ud2
-30: cmpb $0, _ASM_RIP(kvm_rebooting)
+30: cmpb $0, _ASM_RIP(virt_rebooting)
jne 4b
ud2
-50: cmpb $0, _ASM_RIP(kvm_rebooting)
+50: cmpb $0, _ASM_RIP(virt_rebooting)
jne 6b
ud2
-70: cmpb $0, _ASM_RIP(kvm_rebooting)
+70: cmpb $0, _ASM_RIP(virt_rebooting)
jne 8b
ud2
@@ -394,7 +394,7 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run)
RESTORE_GUEST_SPEC_CTRL_BODY
RESTORE_HOST_SPEC_CTRL_BODY %sil
-3: cmpb $0, kvm_rebooting(%rip)
+3: cmpb $0, virt_rebooting(%rip)
jne 2b
ud2
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 5df9d32d2058..0c790eb0bfa6 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -6,6 +6,7 @@
#include <linux/misc_cgroup.h>
#include <linux/mmu_context.h>
#include <asm/tdx.h>
+#include <asm/virt.h>
#include "capabilities.h"
#include "mmu.h"
#include "x86_ops.h"
@@ -1994,7 +1995,7 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath)
* TDX_SEAMCALL_VMFAILINVALID.
*/
if (unlikely((vp_enter_ret & TDX_SW_ERROR) == TDX_SW_ERROR)) {
- KVM_BUG_ON(!kvm_rebooting, vcpu->kvm);
+ KVM_BUG_ON(!virt_rebooting, vcpu->kvm);
goto unhandled_exit;
}
diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S
index 4426d34811fc..8a481dae9cae 100644
--- a/arch/x86/kvm/vmx/vmenter.S
+++ b/arch/x86/kvm/vmx/vmenter.S
@@ -310,7 +310,7 @@ SYM_INNER_LABEL_ALIGN(vmx_vmexit, SYM_L_GLOBAL)
RET
.Lfixup:
- cmpb $0, _ASM_RIP(kvm_rebooting)
+ cmpb $0, _ASM_RIP(virt_rebooting)
jne .Lvmfail
ud2
.Lvmfail:
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 967b58a8ab9d..fc6e3b620866 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -48,6 +48,7 @@
#include <asm/msr.h>
#include <asm/mwait.h>
#include <asm/spec-ctrl.h>
+#include <asm/virt.h>
#include <asm/vmx.h>
#include <trace/events/ipi.h>
@@ -814,13 +815,13 @@ void vmx_emergency_disable_virtualization_cpu(void)
int cpu = raw_smp_processor_id();
struct loaded_vmcs *v;
- kvm_rebooting = true;
+ virt_rebooting = true;
/*
* Note, CR4.VMXE can be _cleared_ in NMI context, but it can only be
* set in task context. If this races with VMX is disabled by an NMI,
* VMCLEAR and VMXOFF may #UD, but KVM will eat those faults due to
- * kvm_rebooting set.
+ * virt_rebooting set.
*/
if (!(__read_cr4() & X86_CR4_VMXE))
return;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 77edc24f8309..69937d14f5e1 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -83,6 +83,8 @@
#include <asm/intel_pt.h>
#include <asm/emulate_prefix.h>
#include <asm/sgx.h>
+#include <asm/virt.h>
+
#include <clocksource/hyperv_timer.h>
#define CREATE_TRACE_POINTS
@@ -700,9 +702,6 @@ static void drop_user_return_notifiers(void)
kvm_on_user_return(&msrs->urn);
}
-__visible bool kvm_rebooting;
-EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_rebooting);
-
/*
* Handle a fault on a hardware virtualization (VMX or SVM) instruction.
*
@@ -713,7 +712,7 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_rebooting);
noinstr void kvm_spurious_fault(void)
{
/* Fault while not rebooting. We want the trace. */
- BUG_ON(!kvm_rebooting);
+ BUG_ON(!virt_rebooting);
}
EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_spurious_fault);
@@ -13184,16 +13183,16 @@ int kvm_arch_enable_virtualization_cpu(void)
void kvm_arch_shutdown(void)
{
/*
- * Set kvm_rebooting to indicate that KVM has asynchronously disabled
+ * Set virt_rebooting to indicate that KVM has asynchronously disabled
* hardware virtualization, i.e. that errors and/or exceptions on SVM
* and VMX instructions are expected and should be ignored.
*/
- kvm_rebooting = true;
+ virt_rebooting = true;
/*
- * Ensure kvm_rebooting is visible before IPIs are sent to other CPUs
+ * Ensure virt_rebooting is visible before IPIs are sent to other CPUs
* to disable virtualization. Effectively pairs with the reception of
- * the IPI (kvm_rebooting is read in task/exception context, but only
+ * the IPI (virt_rebooting is read in task/exception context, but only
* _needs_ to be read as %true after the IPI function callback disables
* virtualization).
*/
@@ -13214,7 +13213,7 @@ void kvm_arch_disable_virtualization_cpu(void)
* disable virtualization arrives. Handle the extreme edge case here
* instead of trying to account for it in the normal flows.
*/
- if (in_task() || WARN_ON_ONCE(!kvm_rebooting))
+ if (in_task() || WARN_ON_ONCE(!virt_rebooting))
drop_user_return_notifiers();
else
__module_get(THIS_MODULE);
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index b314649e5c02..94d4f07aaaa0 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -54,7 +54,6 @@ struct kvm_host_values {
u64 arch_capabilities;
};
-extern bool kvm_rebooting;
void kvm_spurious_fault(void);
#define SIZE_OF_MEMSLOTS_HASHTABLE \
diff --git a/arch/x86/virt/Makefile b/arch/x86/virt/Makefile
index ea343fc392dc..6e485751650c 100644
--- a/arch/x86/virt/Makefile
+++ b/arch/x86/virt/Makefile
@@ -1,2 +1,4 @@
# SPDX-License-Identifier: GPL-2.0-only
obj-y += svm/ vmx/
+
+obj-$(subst m,y,$(CONFIG_KVM_X86)) += hw.o
\ No newline at end of file
diff --git a/arch/x86/virt/hw.c b/arch/x86/virt/hw.c
new file mode 100644
index 000000000000..df3dc18d19b4
--- /dev/null
+++ b/arch/x86/virt/hw.c
@@ -0,0 +1,7 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <linux/kvm_types.h>
+
+#include <asm/virt.h>
+
+__visible bool virt_rebooting;
+EXPORT_SYMBOL_FOR_KVM(virt_rebooting);
--
2.53.0.310.g728cabbaf7-goog
^ permalink raw reply related
* [PATCH v3 04/16] KVM: VMX: Unconditionally allocate root VMCSes during boot CPU bringup
From: Sean Christopherson @ 2026-02-14 1:26 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun, Dan Williams
In-Reply-To: <20260214012702.2368778-1-seanjc@google.com>
Allocate the root VMCS (misleading called "vmxarea" and "kvm_area" in KVM)
for each possible CPU during early boot CPU bringup, before early TDX
initialization, so that TDX can eventually do VMXON on-demand (to make
SEAMCALLs) without needing to load kvm-intel.ko. Allocate the pages early
on, e.g. instead of trying to do so on-demand, to avoid having to juggle
allocation failures at runtime.
Opportunistically rename the per-CPU pointers to better reflect the role
of the VMCS. Use Intel's "root VMCS" terminology, e.g. from various VMCS
patents[1][2] and older SDMs, not the more opaque "VMXON region" used in
recent versions of the SDM. While it's possible the VMCS passed to VMXON
no longer serves as _the_ root VMCS on modern CPUs, it is still in effect
a "root mode VMCS", as described in the patents.
Link: https://patentimages.storage.googleapis.com/c7/e4/32/d7a7def5580667/WO2013101191A1.pdf [1]
Link: https://patentimages.storage.googleapis.com/13/f6/8d/1361fab8c33373/US20080163205A1.pdf [2]
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/virt.h | 13 ++++++-
arch/x86/kernel/cpu/common.c | 2 +
arch/x86/kvm/vmx/vmx.c | 58 ++---------------------------
arch/x86/virt/hw.c | 71 ++++++++++++++++++++++++++++++++++++
4 files changed, 89 insertions(+), 55 deletions(-)
diff --git a/arch/x86/include/asm/virt.h b/arch/x86/include/asm/virt.h
index 131b9bf9ef3c..0da6db4f5b0c 100644
--- a/arch/x86/include/asm/virt.h
+++ b/arch/x86/include/asm/virt.h
@@ -2,10 +2,21 @@
#ifndef _ASM_X86_VIRT_H
#define _ASM_X86_VIRT_H
-#include <linux/types.h>
+#include <linux/percpu-defs.h>
+
+#include <asm/reboot.h>
#if IS_ENABLED(CONFIG_KVM_X86)
extern bool virt_rebooting;
+
+void __init x86_virt_init(void);
+
+#if IS_ENABLED(CONFIG_KVM_INTEL)
+DECLARE_PER_CPU(struct vmcs *, root_vmcs);
+#endif
+
+#else
+static __always_inline void x86_virt_init(void) {}
#endif
#endif /* _ASM_X86_VIRT_H */
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index e7ab22fce3b5..dda9e41292db 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -71,6 +71,7 @@
#include <asm/traps.h>
#include <asm/sev.h>
#include <asm/tdx.h>
+#include <asm/virt.h>
#include <asm/posted_intr.h>
#include <asm/runtime-const.h>
@@ -2143,6 +2144,7 @@ static __init void identify_boot_cpu(void)
cpu_detect_tlb(&boot_cpu_data);
setup_cr_pinning();
+ x86_virt_init();
tsx_init();
tdx_init();
lkgs_init();
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index fc6e3b620866..abd4830f71d8 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -580,7 +580,6 @@ noinline void invept_error(unsigned long ext, u64 eptp)
vmx_insn_failed("invept failed: ext=0x%lx eptp=%llx\n", ext, eptp);
}
-static DEFINE_PER_CPU(struct vmcs *, vmxarea);
DEFINE_PER_CPU(struct vmcs *, current_vmcs);
/*
* We maintain a per-CPU linked-list of VMCS loaded on that CPU. This is needed
@@ -2934,6 +2933,9 @@ static bool __kvm_is_vmx_supported(void)
return false;
}
+ if (!per_cpu(root_vmcs, cpu))
+ return false;
+
return true;
}
@@ -3008,7 +3010,7 @@ static int kvm_cpu_vmxon(u64 vmxon_pointer)
int vmx_enable_virtualization_cpu(void)
{
int cpu = raw_smp_processor_id();
- u64 phys_addr = __pa(per_cpu(vmxarea, cpu));
+ u64 phys_addr = __pa(per_cpu(root_vmcs, cpu));
int r;
if (cr4_read_shadow() & X86_CR4_VMXE)
@@ -3129,47 +3131,6 @@ int alloc_loaded_vmcs(struct loaded_vmcs *loaded_vmcs)
return -ENOMEM;
}
-static void free_kvm_area(void)
-{
- int cpu;
-
- for_each_possible_cpu(cpu) {
- free_vmcs(per_cpu(vmxarea, cpu));
- per_cpu(vmxarea, cpu) = NULL;
- }
-}
-
-static __init int alloc_kvm_area(void)
-{
- int cpu;
-
- for_each_possible_cpu(cpu) {
- struct vmcs *vmcs;
-
- vmcs = alloc_vmcs_cpu(false, cpu, GFP_KERNEL);
- if (!vmcs) {
- free_kvm_area();
- return -ENOMEM;
- }
-
- /*
- * When eVMCS is enabled, alloc_vmcs_cpu() sets
- * vmcs->revision_id to KVM_EVMCS_VERSION instead of
- * revision_id reported by MSR_IA32_VMX_BASIC.
- *
- * However, even though not explicitly documented by
- * TLFS, VMXArea passed as VMXON argument should
- * still be marked with revision_id reported by
- * physical CPU.
- */
- if (kvm_is_using_evmcs())
- vmcs->hdr.revision_id = vmx_basic_vmcs_revision_id(vmcs_config.basic);
-
- per_cpu(vmxarea, cpu) = vmcs;
- }
- return 0;
-}
-
static void fix_pmode_seg(struct kvm_vcpu *vcpu, int seg,
struct kvm_segment *save)
{
@@ -8566,8 +8527,6 @@ void vmx_hardware_unsetup(void)
if (nested)
nested_vmx_hardware_unsetup();
-
- free_kvm_area();
}
void vmx_vm_destroy(struct kvm *kvm)
@@ -8870,10 +8829,6 @@ __init int vmx_hardware_setup(void)
return r;
}
- r = alloc_kvm_area();
- if (r)
- goto err_kvm_area;
-
kvm_set_posted_intr_wakeup_handler(pi_wakeup_handler);
/*
@@ -8900,11 +8855,6 @@ __init int vmx_hardware_setup(void)
kvm_caps.inapplicable_quirks &= ~KVM_X86_QUIRK_IGNORE_GUEST_PAT;
return 0;
-
-err_kvm_area:
- if (nested)
- nested_vmx_hardware_unsetup();
- return r;
}
void vmx_exit(void)
diff --git a/arch/x86/virt/hw.c b/arch/x86/virt/hw.c
index df3dc18d19b4..56972f594d90 100644
--- a/arch/x86/virt/hw.c
+++ b/arch/x86/virt/hw.c
@@ -1,7 +1,78 @@
// SPDX-License-Identifier: GPL-2.0-only
+#include <linux/cpu.h>
+#include <linux/cpumask.h>
+#include <linux/errno.h>
#include <linux/kvm_types.h>
+#include <linux/list.h>
+#include <linux/percpu.h>
+#include <asm/perf_event.h>
+#include <asm/processor.h>
#include <asm/virt.h>
+#include <asm/vmx.h>
__visible bool virt_rebooting;
EXPORT_SYMBOL_FOR_KVM(virt_rebooting);
+
+#if IS_ENABLED(CONFIG_KVM_INTEL)
+DEFINE_PER_CPU(struct vmcs *, root_vmcs);
+EXPORT_PER_CPU_SYMBOL(root_vmcs);
+
+static __init void x86_vmx_exit(void)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu) {
+ free_page((unsigned long)per_cpu(root_vmcs, cpu));
+ per_cpu(root_vmcs, cpu) = NULL;
+ }
+}
+
+static __init int x86_vmx_init(void)
+{
+ u64 basic_msr;
+ u32 rev_id;
+ int cpu;
+
+ if (!cpu_feature_enabled(X86_FEATURE_VMX))
+ return -EOPNOTSUPP;
+
+ rdmsrq(MSR_IA32_VMX_BASIC, basic_msr);
+
+ /* IA-32 SDM Vol 3B: VMCS size is never greater than 4kB. */
+ if (WARN_ON_ONCE(vmx_basic_vmcs_size(basic_msr) > PAGE_SIZE))
+ return -EIO;
+
+ /*
+ * Even if eVMCS is enabled (or will be enabled?), and even though not
+ * explicitly documented by TLFS, the root VMCS passed to VMXON should
+ * still be marked with the revision_id reported by the physical CPU.
+ */
+ rev_id = vmx_basic_vmcs_revision_id(basic_msr);
+
+ for_each_possible_cpu(cpu) {
+ int node = cpu_to_node(cpu);
+ struct page *page;
+ struct vmcs *vmcs;
+
+ page = __alloc_pages_node(node, GFP_KERNEL | __GFP_ZERO, 0);
+ if (!page) {
+ x86_vmx_exit();
+ return -ENOMEM;
+ }
+
+ vmcs = page_address(page);
+ vmcs->hdr.revision_id = rev_id;
+ per_cpu(root_vmcs, cpu) = vmcs;
+ }
+
+ return 0;
+}
+#else
+static __init int x86_vmx_init(void) { return -EOPNOTSUPP; }
+#endif
+
+void __init x86_virt_init(void)
+{
+ x86_vmx_init();
+}
--
2.53.0.310.g728cabbaf7-goog
^ permalink raw reply related
* [PATCH v3 05/16] x86/virt: Force-clear X86_FEATURE_VMX if configuring root VMCS fails
From: Sean Christopherson @ 2026-02-14 1:26 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun, Dan Williams
In-Reply-To: <20260214012702.2368778-1-seanjc@google.com>
If allocating and configuring a root VMCS fails, clear X86_FEATURE_VMX in
all CPUs so that KVM doesn't need to manually check root_vmcs. As added
bonuses, clearing VMX will reflect that VMX is unusable in /proc/cpuinfo,
and will avoid a futile auto-probe of kvm-intel.ko.
WARN if allocating a root VMCS page fails, e.g. to help users figure out
why VMX is broken in the unlikely scenario something goes sideways during
boot (and because the allocation should succeed unless there's a kernel
bug). Tweak KVM's error message to suggest checking kernel logs if VMX is
unsupported (in addition to checking BIOS).
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/vmx/vmx.c | 7 ++++---
arch/x86/virt/hw.c | 14 ++++++++++++--
2 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index abd4830f71d8..e767835a4f3a 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -2927,14 +2927,15 @@ static bool __kvm_is_vmx_supported(void)
return false;
}
- if (!this_cpu_has(X86_FEATURE_MSR_IA32_FEAT_CTL) ||
- !this_cpu_has(X86_FEATURE_VMX)) {
+ if (!this_cpu_has(X86_FEATURE_MSR_IA32_FEAT_CTL)) {
pr_err("VMX not enabled (by BIOS) in MSR_IA32_FEAT_CTL on CPU %d\n", cpu);
return false;
}
- if (!per_cpu(root_vmcs, cpu))
+ if (!this_cpu_has(X86_FEATURE_VMX)) {
+ pr_err("VMX not fully enabled on CPU %d. Check kernel logs and/or BIOS\n", cpu);
return false;
+ }
return true;
}
diff --git a/arch/x86/virt/hw.c b/arch/x86/virt/hw.c
index 56972f594d90..40495872fdfb 100644
--- a/arch/x86/virt/hw.c
+++ b/arch/x86/virt/hw.c
@@ -28,7 +28,7 @@ static __init void x86_vmx_exit(void)
}
}
-static __init int x86_vmx_init(void)
+static __init int __x86_vmx_init(void)
{
u64 basic_msr;
u32 rev_id;
@@ -56,7 +56,7 @@ static __init int x86_vmx_init(void)
struct vmcs *vmcs;
page = __alloc_pages_node(node, GFP_KERNEL | __GFP_ZERO, 0);
- if (!page) {
+ if (WARN_ON_ONCE(!page)) {
x86_vmx_exit();
return -ENOMEM;
}
@@ -68,6 +68,16 @@ static __init int x86_vmx_init(void)
return 0;
}
+
+static __init int x86_vmx_init(void)
+{
+ int r;
+
+ r = __x86_vmx_init();
+ if (r)
+ setup_clear_cpu_cap(X86_FEATURE_VMX);
+ return r;
+}
#else
static __init int x86_vmx_init(void) { return -EOPNOTSUPP; }
#endif
--
2.53.0.310.g728cabbaf7-goog
^ permalink raw reply related
* [PATCH v3 06/16] KVM: VMX: Move core VMXON enablement to kernel
From: Sean Christopherson @ 2026-02-14 1:26 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun, Dan Williams
In-Reply-To: <20260214012702.2368778-1-seanjc@google.com>
Move the innermost VMXON+VMXOFF logic out of KVM and into to core x86 so
that TDX can (eventually) force VMXON without having to rely on KVM being
loaded, e.g. to do SEAMCALLs during initialization.
Opportunistically update the comment regarding emergency disabling via NMI
to clarify that virt_rebooting will be set by _another_ emergency callback,
i.e. that virt_rebooting doesn't need to be set before VMCLEAR, only
before _this_ invocation does VMXOFF.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/events/intel/pt.c | 1 -
arch/x86/include/asm/virt.h | 6 +--
arch/x86/kvm/vmx/vmx.c | 73 +++----------------------------
arch/x86/virt/hw.c | 85 ++++++++++++++++++++++++++++++++++++-
4 files changed, 92 insertions(+), 73 deletions(-)
diff --git a/arch/x86/events/intel/pt.c b/arch/x86/events/intel/pt.c
index 44524a387c58..b5726b50e77d 100644
--- a/arch/x86/events/intel/pt.c
+++ b/arch/x86/events/intel/pt.c
@@ -1591,7 +1591,6 @@ void intel_pt_handle_vmx(int on)
local_irq_restore(flags);
}
-EXPORT_SYMBOL_FOR_KVM(intel_pt_handle_vmx);
/*
* PMU callbacks
diff --git a/arch/x86/include/asm/virt.h b/arch/x86/include/asm/virt.h
index 0da6db4f5b0c..cca0210a5c16 100644
--- a/arch/x86/include/asm/virt.h
+++ b/arch/x86/include/asm/virt.h
@@ -2,8 +2,6 @@
#ifndef _ASM_X86_VIRT_H
#define _ASM_X86_VIRT_H
-#include <linux/percpu-defs.h>
-
#include <asm/reboot.h>
#if IS_ENABLED(CONFIG_KVM_X86)
@@ -12,7 +10,9 @@ extern bool virt_rebooting;
void __init x86_virt_init(void);
#if IS_ENABLED(CONFIG_KVM_INTEL)
-DECLARE_PER_CPU(struct vmcs *, root_vmcs);
+int x86_vmx_enable_virtualization_cpu(void);
+int x86_vmx_disable_virtualization_cpu(void);
+void x86_vmx_emergency_disable_virtualization_cpu(void);
#endif
#else
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index e767835a4f3a..36238cc694fd 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -786,41 +786,16 @@ static int vmx_set_guest_uret_msr(struct vcpu_vmx *vmx,
return ret;
}
-/*
- * Disable VMX and clear CR4.VMXE (even if VMXOFF faults)
- *
- * Note, VMXOFF causes a #UD if the CPU is !post-VMXON, but it's impossible to
- * atomically track post-VMXON state, e.g. this may be called in NMI context.
- * Eat all faults as all other faults on VMXOFF faults are mode related, i.e.
- * faults are guaranteed to be due to the !post-VMXON check unless the CPU is
- * magically in RM, VM86, compat mode, or at CPL>0.
- */
-static int kvm_cpu_vmxoff(void)
-{
- asm goto("1: vmxoff\n\t"
- _ASM_EXTABLE(1b, %l[fault])
- ::: "cc", "memory" : fault);
-
- cr4_clear_bits(X86_CR4_VMXE);
- return 0;
-
-fault:
- cr4_clear_bits(X86_CR4_VMXE);
- return -EIO;
-}
-
void vmx_emergency_disable_virtualization_cpu(void)
{
int cpu = raw_smp_processor_id();
struct loaded_vmcs *v;
- virt_rebooting = true;
-
/*
* Note, CR4.VMXE can be _cleared_ in NMI context, but it can only be
- * set in task context. If this races with VMX is disabled by an NMI,
- * VMCLEAR and VMXOFF may #UD, but KVM will eat those faults due to
- * virt_rebooting set.
+ * set in task context. If this races with _another_ emergency call
+ * from NMI context, VMCLEAR may #UD, but KVM will eat those faults due
+ * to virt_rebooting being set by the interrupting NMI callback.
*/
if (!(__read_cr4() & X86_CR4_VMXE))
return;
@@ -832,7 +807,7 @@ void vmx_emergency_disable_virtualization_cpu(void)
vmcs_clear(v->shadow_vmcs);
}
- kvm_cpu_vmxoff();
+ x86_vmx_emergency_disable_virtualization_cpu();
}
static void __loaded_vmcs_clear(void *arg)
@@ -2988,34 +2963,9 @@ int vmx_check_processor_compat(void)
return 0;
}
-static int kvm_cpu_vmxon(u64 vmxon_pointer)
-{
- u64 msr;
-
- cr4_set_bits(X86_CR4_VMXE);
-
- asm goto("1: vmxon %[vmxon_pointer]\n\t"
- _ASM_EXTABLE(1b, %l[fault])
- : : [vmxon_pointer] "m"(vmxon_pointer)
- : : fault);
- return 0;
-
-fault:
- WARN_ONCE(1, "VMXON faulted, MSR_IA32_FEAT_CTL (0x3a) = 0x%llx\n",
- rdmsrq_safe(MSR_IA32_FEAT_CTL, &msr) ? 0xdeadbeef : msr);
- cr4_clear_bits(X86_CR4_VMXE);
-
- return -EFAULT;
-}
-
int vmx_enable_virtualization_cpu(void)
{
int cpu = raw_smp_processor_id();
- u64 phys_addr = __pa(per_cpu(root_vmcs, cpu));
- int r;
-
- if (cr4_read_shadow() & X86_CR4_VMXE)
- return -EBUSY;
/*
* This can happen if we hot-added a CPU but failed to allocate
@@ -3024,15 +2974,7 @@ int vmx_enable_virtualization_cpu(void)
if (kvm_is_using_evmcs() && !hv_get_vp_assist_page(cpu))
return -EFAULT;
- intel_pt_handle_vmx(1);
-
- r = kvm_cpu_vmxon(phys_addr);
- if (r) {
- intel_pt_handle_vmx(0);
- return r;
- }
-
- return 0;
+ return x86_vmx_enable_virtualization_cpu();
}
static void vmclear_local_loaded_vmcss(void)
@@ -3049,12 +2991,9 @@ void vmx_disable_virtualization_cpu(void)
{
vmclear_local_loaded_vmcss();
- if (kvm_cpu_vmxoff())
- kvm_spurious_fault();
+ x86_vmx_disable_virtualization_cpu();
hv_reset_evmcs();
-
- intel_pt_handle_vmx(0);
}
struct vmcs *alloc_vmcs_cpu(bool shadow, int cpu, gfp_t flags)
diff --git a/arch/x86/virt/hw.c b/arch/x86/virt/hw.c
index 40495872fdfb..dc426c2bc24a 100644
--- a/arch/x86/virt/hw.c
+++ b/arch/x86/virt/hw.c
@@ -15,8 +15,89 @@ __visible bool virt_rebooting;
EXPORT_SYMBOL_FOR_KVM(virt_rebooting);
#if IS_ENABLED(CONFIG_KVM_INTEL)
-DEFINE_PER_CPU(struct vmcs *, root_vmcs);
-EXPORT_PER_CPU_SYMBOL(root_vmcs);
+static DEFINE_PER_CPU(struct vmcs *, root_vmcs);
+
+static int x86_virt_cpu_vmxon(void)
+{
+ u64 vmxon_pointer = __pa(per_cpu(root_vmcs, raw_smp_processor_id()));
+ u64 msr;
+
+ cr4_set_bits(X86_CR4_VMXE);
+
+ asm goto("1: vmxon %[vmxon_pointer]\n\t"
+ _ASM_EXTABLE(1b, %l[fault])
+ : : [vmxon_pointer] "m"(vmxon_pointer)
+ : : fault);
+ return 0;
+
+fault:
+ WARN_ONCE(1, "VMXON faulted, MSR_IA32_FEAT_CTL (0x3a) = 0x%llx\n",
+ rdmsrq_safe(MSR_IA32_FEAT_CTL, &msr) ? 0xdeadbeef : msr);
+ cr4_clear_bits(X86_CR4_VMXE);
+
+ return -EFAULT;
+}
+
+int x86_vmx_enable_virtualization_cpu(void)
+{
+ int r;
+
+ if (cr4_read_shadow() & X86_CR4_VMXE)
+ return -EBUSY;
+
+ intel_pt_handle_vmx(1);
+
+ r = x86_virt_cpu_vmxon();
+ if (r) {
+ intel_pt_handle_vmx(0);
+ return r;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_FOR_KVM(x86_vmx_enable_virtualization_cpu);
+
+/*
+ * Disable VMX and clear CR4.VMXE (even if VMXOFF faults)
+ *
+ * Note, VMXOFF causes a #UD if the CPU is !post-VMXON, but it's impossible to
+ * atomically track post-VMXON state, e.g. this may be called in NMI context.
+ * Eat all faults as all other faults on VMXOFF faults are mode related, i.e.
+ * faults are guaranteed to be due to the !post-VMXON check unless the CPU is
+ * magically in RM, VM86, compat mode, or at CPL>0.
+ */
+int x86_vmx_disable_virtualization_cpu(void)
+{
+ int r = -EIO;
+
+ asm goto("1: vmxoff\n\t"
+ _ASM_EXTABLE(1b, %l[fault])
+ ::: "cc", "memory" : fault);
+ r = 0;
+
+fault:
+ cr4_clear_bits(X86_CR4_VMXE);
+ intel_pt_handle_vmx(0);
+ return r;
+}
+EXPORT_SYMBOL_FOR_KVM(x86_vmx_disable_virtualization_cpu);
+
+void x86_vmx_emergency_disable_virtualization_cpu(void)
+{
+ virt_rebooting = true;
+
+ /*
+ * Note, CR4.VMXE can be _cleared_ in NMI context, but it can only be
+ * set in task context. If this races with _another_ emergency call
+ * from NMI context, VMXOFF may #UD, but kernel will eat those faults
+ * due to virt_rebooting being set by the interrupting NMI callback.
+ */
+ if (!(__read_cr4() & X86_CR4_VMXE))
+ return;
+
+ x86_vmx_disable_virtualization_cpu();
+}
+EXPORT_SYMBOL_FOR_KVM(x86_vmx_emergency_disable_virtualization_cpu);
static __init void x86_vmx_exit(void)
{
--
2.53.0.310.g728cabbaf7-goog
^ permalink raw reply related
* [PATCH v3 07/16] KVM: SVM: Move core EFER.SVME enablement to kernel
From: Sean Christopherson @ 2026-02-14 1:26 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun, Dan Williams
In-Reply-To: <20260214012702.2368778-1-seanjc@google.com>
Move the innermost EFER.SVME logic out of KVM and into to core x86 to land
the SVM support alongside VMX support. This will allow providing a more
unified API from the kernel to KVM, and will allow moving the bulk of the
emergency disabling insanity out of KVM without having a weird split
between kernel and KVM for SVM vs. VMX.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/virt.h | 6 +++++
arch/x86/kvm/svm/svm.c | 33 +++++------------------
arch/x86/virt/hw.c | 53 +++++++++++++++++++++++++++++++++++++
3 files changed, 65 insertions(+), 27 deletions(-)
diff --git a/arch/x86/include/asm/virt.h b/arch/x86/include/asm/virt.h
index cca0210a5c16..9a0753eaa20c 100644
--- a/arch/x86/include/asm/virt.h
+++ b/arch/x86/include/asm/virt.h
@@ -15,6 +15,12 @@ int x86_vmx_disable_virtualization_cpu(void);
void x86_vmx_emergency_disable_virtualization_cpu(void);
#endif
+#if IS_ENABLED(CONFIG_KVM_AMD)
+int x86_svm_enable_virtualization_cpu(void);
+int x86_svm_disable_virtualization_cpu(void);
+void x86_svm_emergency_disable_virtualization_cpu(void);
+#endif
+
#else
static __always_inline void x86_virt_init(void) {}
#endif
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 0ae66c770ebc..5f033bf3ba83 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -478,27 +478,9 @@ static __always_inline struct sev_es_save_area *sev_es_host_save_area(struct svm
return &sd->save_area->host_sev_es_save;
}
-static inline void kvm_cpu_svm_disable(void)
-{
- uint64_t efer;
-
- wrmsrq(MSR_VM_HSAVE_PA, 0);
- rdmsrq(MSR_EFER, efer);
- if (efer & EFER_SVME) {
- /*
- * Force GIF=1 prior to disabling SVM, e.g. to ensure INIT and
- * NMI aren't blocked.
- */
- stgi();
- wrmsrq(MSR_EFER, efer & ~EFER_SVME);
- }
-}
-
static void svm_emergency_disable_virtualization_cpu(void)
{
- virt_rebooting = true;
-
- kvm_cpu_svm_disable();
+ wrmsrq(MSR_VM_HSAVE_PA, 0);
}
static void svm_disable_virtualization_cpu(void)
@@ -507,7 +489,7 @@ static void svm_disable_virtualization_cpu(void)
if (tsc_scaling)
__svm_write_tsc_multiplier(SVM_TSC_RATIO_DEFAULT);
- kvm_cpu_svm_disable();
+ x86_svm_disable_virtualization_cpu();
amd_pmu_disable_virt();
}
@@ -516,12 +498,12 @@ static int svm_enable_virtualization_cpu(void)
{
struct svm_cpu_data *sd;
- uint64_t efer;
int me = raw_smp_processor_id();
+ int r;
- rdmsrq(MSR_EFER, efer);
- if (efer & EFER_SVME)
- return -EBUSY;
+ r = x86_svm_enable_virtualization_cpu();
+ if (r)
+ return r;
sd = per_cpu_ptr(&svm_data, me);
sd->asid_generation = 1;
@@ -529,8 +511,6 @@ static int svm_enable_virtualization_cpu(void)
sd->next_asid = sd->max_asid + 1;
sd->min_asid = max_sev_asid + 1;
- wrmsrq(MSR_EFER, efer | EFER_SVME);
-
wrmsrq(MSR_VM_HSAVE_PA, sd->save_area_pa);
if (static_cpu_has(X86_FEATURE_TSCRATEMSR)) {
@@ -541,7 +521,6 @@ static int svm_enable_virtualization_cpu(void)
__svm_write_tsc_multiplier(SVM_TSC_RATIO_DEFAULT);
}
-
/*
* Get OSVW bits.
*
diff --git a/arch/x86/virt/hw.c b/arch/x86/virt/hw.c
index dc426c2bc24a..014e9dfab805 100644
--- a/arch/x86/virt/hw.c
+++ b/arch/x86/virt/hw.c
@@ -163,6 +163,59 @@ static __init int x86_vmx_init(void)
static __init int x86_vmx_init(void) { return -EOPNOTSUPP; }
#endif
+#if IS_ENABLED(CONFIG_KVM_AMD)
+int x86_svm_enable_virtualization_cpu(void)
+{
+ u64 efer;
+
+ if (!cpu_feature_enabled(X86_FEATURE_SVM))
+ return -EOPNOTSUPP;
+
+ rdmsrq(MSR_EFER, efer);
+ if (efer & EFER_SVME)
+ return -EBUSY;
+
+ wrmsrq(MSR_EFER, efer | EFER_SVME);
+ return 0;
+}
+EXPORT_SYMBOL_FOR_KVM(x86_svm_enable_virtualization_cpu);
+
+int x86_svm_disable_virtualization_cpu(void)
+{
+ int r = -EIO;
+ u64 efer;
+
+ /*
+ * Force GIF=1 prior to disabling SVM, e.g. to ensure INIT and
+ * NMI aren't blocked.
+ */
+ asm goto("1: stgi\n\t"
+ _ASM_EXTABLE(1b, %l[fault])
+ ::: "memory" : fault);
+ r = 0;
+
+fault:
+ rdmsrq(MSR_EFER, efer);
+ wrmsrq(MSR_EFER, efer & ~EFER_SVME);
+ return r;
+}
+EXPORT_SYMBOL_FOR_KVM(x86_svm_disable_virtualization_cpu);
+
+void x86_svm_emergency_disable_virtualization_cpu(void)
+{
+ u64 efer;
+
+ virt_rebooting = true;
+
+ rdmsrq(MSR_EFER, efer);
+ if (!(efer & EFER_SVME))
+ return;
+
+ x86_svm_disable_virtualization_cpu();
+}
+EXPORT_SYMBOL_FOR_KVM(x86_svm_emergency_disable_virtualization_cpu);
+#endif
+
void __init x86_virt_init(void)
{
x86_vmx_init();
--
2.53.0.310.g728cabbaf7-goog
^ permalink raw reply related
* [PATCH v3 08/16] KVM: x86: Move bulk of emergency virtualizaton logic to virt subsystem
From: Sean Christopherson @ 2026-02-14 1:26 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun, Dan Williams
In-Reply-To: <20260214012702.2368778-1-seanjc@google.com>
Move the majority of the code related to disabling hardware virtualization
in emergency from KVM into the virt subsystem so that virt can take full
ownership of the state of SVM/VMX. This will allow refcounting usage of
SVM/VMX so that KVM and the TDX subsystem can enable VMX without stomping
on each other.
To route the emergency callback to the "right" vendor code, add to avoid
mixing vendor and generic code, implement a x86_virt_ops structure to
track the emergency callback, along with the SVM vs. VMX (vs. "none")
feature that is active.
To avoid having to choose between SVM and VMX, simply refuse to enable
either if both are somehow supported. No known CPU supports both SVM and
VMX, and it's comically unlikely such a CPU will ever exist.
Leave KVM's clearing of loaded VMCSes and MSR_VM_HSAVE_PA in KVM, via a
callback explicitly scoped to KVM. Loading VMCSes and saving/restoring
host state are firmly tied to running VMs, and thus are (a) KVM's
responsibility and (b) operations that are still exclusively reserved for
KVM (as far as in-tree code is concerned). I.e. the contract being
established is that non-KVM subsystems can utilize virtualization, but for
all intents and purposes cannot act as full-blown hypervisors.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/kvm_host.h | 3 +-
arch/x86/include/asm/reboot.h | 11 ---
arch/x86/include/asm/virt.h | 9 ++-
arch/x86/kernel/crash.c | 3 +-
arch/x86/kernel/reboot.c | 63 ++--------------
arch/x86/kernel/smp.c | 5 +-
arch/x86/kvm/vmx/vmx.c | 11 ---
arch/x86/kvm/x86.c | 4 +-
arch/x86/virt/hw.c | 123 +++++++++++++++++++++++++++++---
9 files changed, 138 insertions(+), 94 deletions(-)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index ff07c45e3c73..0bda52fbcae5 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -40,7 +40,8 @@
#include <asm/irq_remapping.h>
#include <asm/kvm_page_track.h>
#include <asm/kvm_vcpu_regs.h>
-#include <asm/reboot.h>
+#include <asm/virt.h>
+
#include <hyperv/hvhdk.h>
#define __KVM_HAVE_ARCH_VCPU_DEBUGFS
diff --git a/arch/x86/include/asm/reboot.h b/arch/x86/include/asm/reboot.h
index ecd58ea9a837..a671a1145906 100644
--- a/arch/x86/include/asm/reboot.h
+++ b/arch/x86/include/asm/reboot.h
@@ -25,17 +25,6 @@ void __noreturn machine_real_restart(unsigned int type);
#define MRR_BIOS 0
#define MRR_APM 1
-typedef void (cpu_emergency_virt_cb)(void);
-#if IS_ENABLED(CONFIG_KVM_X86)
-void cpu_emergency_register_virt_callback(cpu_emergency_virt_cb *callback);
-void cpu_emergency_unregister_virt_callback(cpu_emergency_virt_cb *callback);
-void cpu_emergency_disable_virtualization(void);
-#else
-static inline void cpu_emergency_register_virt_callback(cpu_emergency_virt_cb *callback) {}
-static inline void cpu_emergency_unregister_virt_callback(cpu_emergency_virt_cb *callback) {}
-static inline void cpu_emergency_disable_virtualization(void) {}
-#endif /* CONFIG_KVM_X86 */
-
typedef void (*nmi_shootdown_cb)(int, struct pt_regs*);
void nmi_shootdown_cpus(nmi_shootdown_cb callback);
void run_crash_ipi_callback(struct pt_regs *regs);
diff --git a/arch/x86/include/asm/virt.h b/arch/x86/include/asm/virt.h
index 9a0753eaa20c..2c35534437e0 100644
--- a/arch/x86/include/asm/virt.h
+++ b/arch/x86/include/asm/virt.h
@@ -4,6 +4,8 @@
#include <asm/reboot.h>
+typedef void (cpu_emergency_virt_cb)(void);
+
#if IS_ENABLED(CONFIG_KVM_X86)
extern bool virt_rebooting;
@@ -12,17 +14,20 @@ void __init x86_virt_init(void);
#if IS_ENABLED(CONFIG_KVM_INTEL)
int x86_vmx_enable_virtualization_cpu(void);
int x86_vmx_disable_virtualization_cpu(void);
-void x86_vmx_emergency_disable_virtualization_cpu(void);
#endif
#if IS_ENABLED(CONFIG_KVM_AMD)
int x86_svm_enable_virtualization_cpu(void);
int x86_svm_disable_virtualization_cpu(void);
-void x86_svm_emergency_disable_virtualization_cpu(void);
#endif
+int x86_virt_emergency_disable_virtualization_cpu(void);
+
+void x86_virt_register_emergency_callback(cpu_emergency_virt_cb *callback);
+void x86_virt_unregister_emergency_callback(cpu_emergency_virt_cb *callback);
#else
static __always_inline void x86_virt_init(void) {}
+static inline int x86_virt_emergency_disable_virtualization_cpu(void) { return -ENOENT; }
#endif
#endif /* _ASM_X86_VIRT_H */
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index 335fd2ee9766..cd796818d94d 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -42,6 +42,7 @@
#include <asm/crash.h>
#include <asm/cmdline.h>
#include <asm/sev.h>
+#include <asm/virt.h>
/* Used while preparing memory map entries for second kernel */
struct crash_memmap_data {
@@ -111,7 +112,7 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
crash_smp_send_stop();
- cpu_emergency_disable_virtualization();
+ x86_virt_emergency_disable_virtualization_cpu();
/*
* Disable Intel PT to stop its logging
diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c
index 6032fa9ec753..0bab8863375a 100644
--- a/arch/x86/kernel/reboot.c
+++ b/arch/x86/kernel/reboot.c
@@ -27,6 +27,7 @@
#include <asm/cpu.h>
#include <asm/nmi.h>
#include <asm/smp.h>
+#include <asm/virt.h>
#include <linux/ctype.h>
#include <linux/mc146818rtc.h>
@@ -532,51 +533,6 @@ static inline void kb_wait(void)
static inline void nmi_shootdown_cpus_on_restart(void);
#if IS_ENABLED(CONFIG_KVM_X86)
-/* RCU-protected callback to disable virtualization prior to reboot. */
-static cpu_emergency_virt_cb __rcu *cpu_emergency_virt_callback;
-
-void cpu_emergency_register_virt_callback(cpu_emergency_virt_cb *callback)
-{
- if (WARN_ON_ONCE(rcu_access_pointer(cpu_emergency_virt_callback)))
- return;
-
- rcu_assign_pointer(cpu_emergency_virt_callback, callback);
-}
-EXPORT_SYMBOL_FOR_KVM(cpu_emergency_register_virt_callback);
-
-void cpu_emergency_unregister_virt_callback(cpu_emergency_virt_cb *callback)
-{
- if (WARN_ON_ONCE(rcu_access_pointer(cpu_emergency_virt_callback) != callback))
- return;
-
- rcu_assign_pointer(cpu_emergency_virt_callback, NULL);
- synchronize_rcu();
-}
-EXPORT_SYMBOL_FOR_KVM(cpu_emergency_unregister_virt_callback);
-
-/*
- * Disable virtualization, i.e. VMX or SVM, to ensure INIT is recognized during
- * reboot. VMX blocks INIT if the CPU is post-VMXON, and SVM blocks INIT if
- * GIF=0, i.e. if the crash occurred between CLGI and STGI.
- */
-void cpu_emergency_disable_virtualization(void)
-{
- cpu_emergency_virt_cb *callback;
-
- /*
- * IRQs must be disabled as KVM enables virtualization in hardware via
- * function call IPIs, i.e. IRQs need to be disabled to guarantee
- * virtualization stays disabled.
- */
- lockdep_assert_irqs_disabled();
-
- rcu_read_lock();
- callback = rcu_dereference(cpu_emergency_virt_callback);
- if (callback)
- callback();
- rcu_read_unlock();
-}
-
static void emergency_reboot_disable_virtualization(void)
{
local_irq_disable();
@@ -588,16 +544,11 @@ static void emergency_reboot_disable_virtualization(void)
* We can't take any locks and we may be on an inconsistent state, so
* use NMIs as IPIs to tell the other CPUs to disable VMX/SVM and halt.
*
- * Do the NMI shootdown even if virtualization is off on _this_ CPU, as
- * other CPUs may have virtualization enabled.
+ * Safely force _this_ CPU out of VMX/SVM operation, and if necessary,
+ * blast NMIs to force other CPUs out of VMX/SVM as well.k
*/
- if (rcu_access_pointer(cpu_emergency_virt_callback)) {
- /* Safely force _this_ CPU out of VMX/SVM operation. */
- cpu_emergency_disable_virtualization();
-
- /* Disable VMX/SVM and halt on other CPUs. */
+ if (!x86_virt_emergency_disable_virtualization_cpu())
nmi_shootdown_cpus_on_restart();
- }
}
#else
static void emergency_reboot_disable_virtualization(void) { }
@@ -875,10 +826,10 @@ static int crash_nmi_callback(unsigned int val, struct pt_regs *regs)
shootdown_callback(cpu, regs);
/*
- * Prepare the CPU for reboot _after_ invoking the callback so that the
- * callback can safely use virtualization instructions, e.g. VMCLEAR.
+ * Disable virtualization, as both VMX and SVM can block INIT and thus
+ * prevent AP bringup, e.g. in a kdump kernel or in firmware.
*/
- cpu_emergency_disable_virtualization();
+ x86_virt_emergency_disable_virtualization_cpu();
atomic_dec(&waiting_for_crash_ipi);
diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c
index b014e6d229f9..cbf95fe2b207 100644
--- a/arch/x86/kernel/smp.c
+++ b/arch/x86/kernel/smp.c
@@ -35,6 +35,7 @@
#include <asm/trace/irq_vectors.h>
#include <asm/kexec.h>
#include <asm/reboot.h>
+#include <asm/virt.h>
/*
* Some notes on x86 processor bugs affecting SMP operation:
@@ -124,7 +125,7 @@ static int smp_stop_nmi_callback(unsigned int val, struct pt_regs *regs)
if (raw_smp_processor_id() == atomic_read(&stopping_cpu))
return NMI_HANDLED;
- cpu_emergency_disable_virtualization();
+ x86_virt_emergency_disable_virtualization_cpu();
stop_this_cpu(NULL);
return NMI_HANDLED;
@@ -136,7 +137,7 @@ static int smp_stop_nmi_callback(unsigned int val, struct pt_regs *regs)
DEFINE_IDTENTRY_SYSVEC(sysvec_reboot)
{
apic_eoi();
- cpu_emergency_disable_virtualization();
+ x86_virt_emergency_disable_virtualization_cpu();
stop_this_cpu(NULL);
}
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 36238cc694fd..c02fd7e91809 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -791,23 +791,12 @@ void vmx_emergency_disable_virtualization_cpu(void)
int cpu = raw_smp_processor_id();
struct loaded_vmcs *v;
- /*
- * Note, CR4.VMXE can be _cleared_ in NMI context, but it can only be
- * set in task context. If this races with _another_ emergency call
- * from NMI context, VMCLEAR may #UD, but KVM will eat those faults due
- * to virt_rebooting being set by the interrupting NMI callback.
- */
- if (!(__read_cr4() & X86_CR4_VMXE))
- return;
-
list_for_each_entry(v, &per_cpu(loaded_vmcss_on_cpu, cpu),
loaded_vmcss_on_cpu_link) {
vmcs_clear(v->vmcs);
if (v->shadow_vmcs)
vmcs_clear(v->shadow_vmcs);
}
-
- x86_vmx_emergency_disable_virtualization_cpu();
}
static void __loaded_vmcs_clear(void *arg)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 69937d14f5e1..4f30acd639f3 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -13076,12 +13076,12 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_vcpu_deliver_sipi_vector);
void kvm_arch_enable_virtualization(void)
{
- cpu_emergency_register_virt_callback(kvm_x86_ops.emergency_disable_virtualization_cpu);
+ x86_virt_register_emergency_callback(kvm_x86_ops.emergency_disable_virtualization_cpu);
}
void kvm_arch_disable_virtualization(void)
{
- cpu_emergency_unregister_virt_callback(kvm_x86_ops.emergency_disable_virtualization_cpu);
+ x86_virt_unregister_emergency_callback(kvm_x86_ops.emergency_disable_virtualization_cpu);
}
int kvm_arch_enable_virtualization_cpu(void)
diff --git a/arch/x86/virt/hw.c b/arch/x86/virt/hw.c
index 014e9dfab805..73c8309ba3fb 100644
--- a/arch/x86/virt/hw.c
+++ b/arch/x86/virt/hw.c
@@ -11,9 +11,45 @@
#include <asm/virt.h>
#include <asm/vmx.h>
+struct x86_virt_ops {
+ int feature;
+ void (*emergency_disable_virtualization_cpu)(void);
+};
+static struct x86_virt_ops virt_ops __ro_after_init;
+
__visible bool virt_rebooting;
EXPORT_SYMBOL_FOR_KVM(virt_rebooting);
+static cpu_emergency_virt_cb __rcu *kvm_emergency_callback;
+
+void x86_virt_register_emergency_callback(cpu_emergency_virt_cb *callback)
+{
+ if (WARN_ON_ONCE(rcu_access_pointer(kvm_emergency_callback)))
+ return;
+
+ rcu_assign_pointer(kvm_emergency_callback, callback);
+}
+EXPORT_SYMBOL_FOR_KVM(x86_virt_register_emergency_callback);
+
+void x86_virt_unregister_emergency_callback(cpu_emergency_virt_cb *callback)
+{
+ if (WARN_ON_ONCE(rcu_access_pointer(kvm_emergency_callback) != callback))
+ return;
+
+ rcu_assign_pointer(kvm_emergency_callback, NULL);
+ synchronize_rcu();
+}
+EXPORT_SYMBOL_FOR_KVM(x86_virt_unregister_emergency_callback);
+
+static void x86_virt_invoke_kvm_emergency_callback(void)
+{
+ cpu_emergency_virt_cb *kvm_callback;
+
+ kvm_callback = rcu_dereference(kvm_emergency_callback);
+ if (kvm_callback)
+ kvm_callback();
+}
+
#if IS_ENABLED(CONFIG_KVM_INTEL)
static DEFINE_PER_CPU(struct vmcs *, root_vmcs);
@@ -42,6 +78,9 @@ int x86_vmx_enable_virtualization_cpu(void)
{
int r;
+ if (virt_ops.feature != X86_FEATURE_VMX)
+ return -EOPNOTSUPP;
+
if (cr4_read_shadow() & X86_CR4_VMXE)
return -EBUSY;
@@ -82,22 +121,24 @@ int x86_vmx_disable_virtualization_cpu(void)
}
EXPORT_SYMBOL_FOR_KVM(x86_vmx_disable_virtualization_cpu);
-void x86_vmx_emergency_disable_virtualization_cpu(void)
+static void x86_vmx_emergency_disable_virtualization_cpu(void)
{
virt_rebooting = true;
/*
* Note, CR4.VMXE can be _cleared_ in NMI context, but it can only be
* set in task context. If this races with _another_ emergency call
- * from NMI context, VMXOFF may #UD, but kernel will eat those faults
- * due to virt_rebooting being set by the interrupting NMI callback.
+ * from NMI context, VMCLEAR (in KVM) and VMXOFF may #UD, but KVM and
+ * the kernel will eat those faults due to virt_rebooting being set by
+ * the interrupting NMI callback.
*/
if (!(__read_cr4() & X86_CR4_VMXE))
return;
+ x86_virt_invoke_kvm_emergency_callback();
+
x86_vmx_disable_virtualization_cpu();
}
-EXPORT_SYMBOL_FOR_KVM(x86_vmx_emergency_disable_virtualization_cpu);
static __init void x86_vmx_exit(void)
{
@@ -111,6 +152,11 @@ static __init void x86_vmx_exit(void)
static __init int __x86_vmx_init(void)
{
+ const struct x86_virt_ops vmx_ops = {
+ .feature = X86_FEATURE_VMX,
+ .emergency_disable_virtualization_cpu = x86_vmx_emergency_disable_virtualization_cpu,
+ };
+
u64 basic_msr;
u32 rev_id;
int cpu;
@@ -147,6 +193,7 @@ static __init int __x86_vmx_init(void)
per_cpu(root_vmcs, cpu) = vmcs;
}
+ memcpy(&virt_ops, &vmx_ops, sizeof(virt_ops));
return 0;
}
@@ -161,6 +208,7 @@ static __init int x86_vmx_init(void)
}
#else
static __init int x86_vmx_init(void) { return -EOPNOTSUPP; }
+static __init void x86_vmx_exit(void) { }
#endif
#if IS_ENABLED(CONFIG_KVM_AMD)
@@ -168,7 +216,7 @@ int x86_svm_enable_virtualization_cpu(void)
{
u64 efer;
- if (!cpu_feature_enabled(X86_FEATURE_SVM))
+ if (virt_ops.feature != X86_FEATURE_SVM)
return -EOPNOTSUPP;
rdmsrq(MSR_EFER, efer);
@@ -201,7 +249,7 @@ int x86_svm_disable_virtualization_cpu(void)
}
EXPORT_SYMBOL_FOR_KVM(x86_svm_disable_virtualization_cpu);
-void x86_svm_emergency_disable_virtualization_cpu(void)
+static void x86_svm_emergency_disable_virtualization_cpu(void)
{
u64 efer;
@@ -211,12 +259,71 @@ void x86_svm_emergency_disable_virtualization_cpu(void)
if (!(efer & EFER_SVME))
return;
+ x86_virt_invoke_kvm_emergency_callback();
+
x86_svm_disable_virtualization_cpu();
}
-EXPORT_SYMBOL_FOR_KVM(x86_svm_emergency_disable_virtualization_cpu);
+
+static __init int x86_svm_init(void)
+{
+ const struct x86_virt_ops svm_ops = {
+ .feature = X86_FEATURE_SVM,
+ .emergency_disable_virtualization_cpu = x86_svm_emergency_disable_virtualization_cpu,
+ };
+
+ if (!cpu_feature_enabled(X86_FEATURE_SVM))
+ return -EOPNOTSUPP;
+
+ memcpy(&virt_ops, &svm_ops, sizeof(virt_ops));
+ return 0;
+}
+#else
+static __init int x86_svm_init(void) { return -EOPNOTSUPP; }
#endif
+/*
+ * Disable virtualization, i.e. VMX or SVM, to ensure INIT is recognized during
+ * reboot. VMX blocks INIT if the CPU is post-VMXON, and SVM blocks INIT if
+ * GIF=0, i.e. if the crash occurred between CLGI and STGI.
+ */
+int x86_virt_emergency_disable_virtualization_cpu(void)
+{
+ /* Ensure the !feature check can't get false positives. */
+ BUILD_BUG_ON(!X86_FEATURE_SVM || !X86_FEATURE_VMX);
+
+ if (!virt_ops.feature)
+ return -EOPNOTSUPP;
+
+ /*
+ * IRQs must be disabled as virtualization is enabled in hardware via
+ * function call IPIs, i.e. IRQs need to be disabled to guarantee
+ * virtualization stays disabled.
+ */
+ lockdep_assert_irqs_disabled();
+
+ /*
+ * Do the NMI shootdown even if virtualization is off on _this_ CPU, as
+ * other CPUs may have virtualization enabled.
+ *
+ * TODO: Track whether or not virtualization might be enabled on other
+ * CPUs? May not be worth avoiding the NMI shootdown...
+ */
+ virt_ops.emergency_disable_virtualization_cpu();
+ return 0;
+}
+
void __init x86_virt_init(void)
{
- x86_vmx_init();
+ /*
+ * Attempt to initialize both SVM and VMX, and simply use whichever one
+ * is present. Rsefuse to enable/use SVM or VMX if both are somehow
+ * supported. No known CPU supports both SVM and VMX.
+ */
+ bool has_vmx = !x86_vmx_init();
+ bool has_svm = !x86_svm_init();
+
+ if (WARN_ON_ONCE(has_vmx && has_svm)) {
+ x86_vmx_exit();
+ memset(&virt_ops, 0, sizeof(virt_ops));
+ }
}
--
2.53.0.310.g728cabbaf7-goog
^ permalink raw reply related
* [PATCH v3 09/16] x86/virt: Add refcounting of VMX/SVM usage to support multiple in-kernel users
From: Sean Christopherson @ 2026-02-14 1:26 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun, Dan Williams
In-Reply-To: <20260214012702.2368778-1-seanjc@google.com>
Implement a per-CPU refcounting scheme so that "users" of hardware
virtualization, e.g. KVM and the future TDX code, can co-exist without
pulling the rug out from under each other. E.g. if KVM were to disable
VMX on module unload or when the last KVM VM was destroyed, SEAMCALLs from
the TDX subsystem would #UD and panic the kernel.
Disable preemption in the get/put APIs to ensure virtualization is fully
enabled/disabled before returning to the caller. E.g. if the task were
preempted after a 0=>1 transition, the new task would see a 1=>2 and thus
return without enabling virtualization. Explicitly disable preemption
instead of requiring the caller to do so, because the need to disable
preemption is an artifact of the implementation. E.g. from KVM's
perspective there is no _need_ to disable preemption as KVM guarantees the
pCPU on which it is running is stable (but preemption is enabled).
Opportunistically abstract away SVM vs. VMX in the public APIs by using
X86_FEATURE_{SVM,VMX} to communicate what technology the caller wants to
enable and use.
Cc: Xu Yilun <yilun.xu@linux.intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/virt.h | 11 ++-----
arch/x86/kvm/svm/svm.c | 4 +--
arch/x86/kvm/vmx/vmx.c | 4 +--
arch/x86/virt/hw.c | 64 +++++++++++++++++++++++++++----------
4 files changed, 53 insertions(+), 30 deletions(-)
diff --git a/arch/x86/include/asm/virt.h b/arch/x86/include/asm/virt.h
index 2c35534437e0..1558a0673d06 100644
--- a/arch/x86/include/asm/virt.h
+++ b/arch/x86/include/asm/virt.h
@@ -11,15 +11,8 @@ extern bool virt_rebooting;
void __init x86_virt_init(void);
-#if IS_ENABLED(CONFIG_KVM_INTEL)
-int x86_vmx_enable_virtualization_cpu(void);
-int x86_vmx_disable_virtualization_cpu(void);
-#endif
-
-#if IS_ENABLED(CONFIG_KVM_AMD)
-int x86_svm_enable_virtualization_cpu(void);
-int x86_svm_disable_virtualization_cpu(void);
-#endif
+int x86_virt_get_ref(int feat);
+void x86_virt_put_ref(int feat);
int x86_virt_emergency_disable_virtualization_cpu(void);
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 5f033bf3ba83..539fb4306dce 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -489,7 +489,7 @@ static void svm_disable_virtualization_cpu(void)
if (tsc_scaling)
__svm_write_tsc_multiplier(SVM_TSC_RATIO_DEFAULT);
- x86_svm_disable_virtualization_cpu();
+ x86_virt_put_ref(X86_FEATURE_SVM);
amd_pmu_disable_virt();
}
@@ -501,7 +501,7 @@ static int svm_enable_virtualization_cpu(void)
int me = raw_smp_processor_id();
int r;
- r = x86_svm_enable_virtualization_cpu();
+ r = x86_virt_get_ref(X86_FEATURE_SVM);
if (r)
return r;
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index c02fd7e91809..6200cf4dbd26 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -2963,7 +2963,7 @@ int vmx_enable_virtualization_cpu(void)
if (kvm_is_using_evmcs() && !hv_get_vp_assist_page(cpu))
return -EFAULT;
- return x86_vmx_enable_virtualization_cpu();
+ return x86_virt_get_ref(X86_FEATURE_VMX);
}
static void vmclear_local_loaded_vmcss(void)
@@ -2980,7 +2980,7 @@ void vmx_disable_virtualization_cpu(void)
{
vmclear_local_loaded_vmcss();
- x86_vmx_disable_virtualization_cpu();
+ x86_virt_put_ref(X86_FEATURE_VMX);
hv_reset_evmcs();
}
diff --git a/arch/x86/virt/hw.c b/arch/x86/virt/hw.c
index 73c8309ba3fb..c898f16fe612 100644
--- a/arch/x86/virt/hw.c
+++ b/arch/x86/virt/hw.c
@@ -13,6 +13,8 @@
struct x86_virt_ops {
int feature;
+ int (*enable_virtualization_cpu)(void);
+ int (*disable_virtualization_cpu)(void);
void (*emergency_disable_virtualization_cpu)(void);
};
static struct x86_virt_ops virt_ops __ro_after_init;
@@ -20,6 +22,8 @@ static struct x86_virt_ops virt_ops __ro_after_init;
__visible bool virt_rebooting;
EXPORT_SYMBOL_FOR_KVM(virt_rebooting);
+static DEFINE_PER_CPU(int, virtualization_nr_users);
+
static cpu_emergency_virt_cb __rcu *kvm_emergency_callback;
void x86_virt_register_emergency_callback(cpu_emergency_virt_cb *callback)
@@ -74,13 +78,10 @@ static int x86_virt_cpu_vmxon(void)
return -EFAULT;
}
-int x86_vmx_enable_virtualization_cpu(void)
+static int x86_vmx_enable_virtualization_cpu(void)
{
int r;
- if (virt_ops.feature != X86_FEATURE_VMX)
- return -EOPNOTSUPP;
-
if (cr4_read_shadow() & X86_CR4_VMXE)
return -EBUSY;
@@ -94,7 +95,6 @@ int x86_vmx_enable_virtualization_cpu(void)
return 0;
}
-EXPORT_SYMBOL_FOR_KVM(x86_vmx_enable_virtualization_cpu);
/*
* Disable VMX and clear CR4.VMXE (even if VMXOFF faults)
@@ -105,7 +105,7 @@ EXPORT_SYMBOL_FOR_KVM(x86_vmx_enable_virtualization_cpu);
* faults are guaranteed to be due to the !post-VMXON check unless the CPU is
* magically in RM, VM86, compat mode, or at CPL>0.
*/
-int x86_vmx_disable_virtualization_cpu(void)
+static int x86_vmx_disable_virtualization_cpu(void)
{
int r = -EIO;
@@ -119,7 +119,6 @@ int x86_vmx_disable_virtualization_cpu(void)
intel_pt_handle_vmx(0);
return r;
}
-EXPORT_SYMBOL_FOR_KVM(x86_vmx_disable_virtualization_cpu);
static void x86_vmx_emergency_disable_virtualization_cpu(void)
{
@@ -154,6 +153,8 @@ static __init int __x86_vmx_init(void)
{
const struct x86_virt_ops vmx_ops = {
.feature = X86_FEATURE_VMX,
+ .enable_virtualization_cpu = x86_vmx_enable_virtualization_cpu,
+ .disable_virtualization_cpu = x86_vmx_disable_virtualization_cpu,
.emergency_disable_virtualization_cpu = x86_vmx_emergency_disable_virtualization_cpu,
};
@@ -212,13 +213,10 @@ static __init void x86_vmx_exit(void) { }
#endif
#if IS_ENABLED(CONFIG_KVM_AMD)
-int x86_svm_enable_virtualization_cpu(void)
+static int x86_svm_enable_virtualization_cpu(void)
{
u64 efer;
- if (virt_ops.feature != X86_FEATURE_SVM)
- return -EOPNOTSUPP;
-
rdmsrq(MSR_EFER, efer);
if (efer & EFER_SVME)
return -EBUSY;
@@ -226,9 +224,8 @@ int x86_svm_enable_virtualization_cpu(void)
wrmsrq(MSR_EFER, efer | EFER_SVME);
return 0;
}
-EXPORT_SYMBOL_FOR_KVM(x86_svm_enable_virtualization_cpu);
-int x86_svm_disable_virtualization_cpu(void)
+static int x86_svm_disable_virtualization_cpu(void)
{
int r = -EIO;
u64 efer;
@@ -247,7 +244,6 @@ int x86_svm_disable_virtualization_cpu(void)
wrmsrq(MSR_EFER, efer & ~EFER_SVME);
return r;
}
-EXPORT_SYMBOL_FOR_KVM(x86_svm_disable_virtualization_cpu);
static void x86_svm_emergency_disable_virtualization_cpu(void)
{
@@ -268,6 +264,8 @@ static __init int x86_svm_init(void)
{
const struct x86_virt_ops svm_ops = {
.feature = X86_FEATURE_SVM,
+ .enable_virtualization_cpu = x86_svm_enable_virtualization_cpu,
+ .disable_virtualization_cpu = x86_svm_disable_virtualization_cpu,
.emergency_disable_virtualization_cpu = x86_svm_emergency_disable_virtualization_cpu,
};
@@ -281,6 +279,41 @@ static __init int x86_svm_init(void)
static __init int x86_svm_init(void) { return -EOPNOTSUPP; }
#endif
+int x86_virt_get_ref(int feat)
+{
+ int r;
+
+ /* Ensure the !feature check can't get false positives. */
+ BUILD_BUG_ON(!X86_FEATURE_SVM || !X86_FEATURE_VMX);
+
+ if (!virt_ops.feature || virt_ops.feature != feat)
+ return -EOPNOTSUPP;
+
+ guard(preempt)();
+
+ if (this_cpu_inc_return(virtualization_nr_users) > 1)
+ return 0;
+
+ r = virt_ops.enable_virtualization_cpu();
+ if (r)
+ WARN_ON_ONCE(this_cpu_dec_return(virtualization_nr_users));
+
+ return r;
+}
+EXPORT_SYMBOL_FOR_KVM(x86_virt_get_ref);
+
+void x86_virt_put_ref(int feat)
+{
+ guard(preempt)();
+
+ if (WARN_ON_ONCE(!this_cpu_read(virtualization_nr_users)) ||
+ this_cpu_dec_return(virtualization_nr_users))
+ return;
+
+ BUG_ON(virt_ops.disable_virtualization_cpu() && !virt_rebooting);
+}
+EXPORT_SYMBOL_FOR_KVM(x86_virt_put_ref);
+
/*
* Disable virtualization, i.e. VMX or SVM, to ensure INIT is recognized during
* reboot. VMX blocks INIT if the CPU is post-VMXON, and SVM blocks INIT if
@@ -288,9 +321,6 @@ static __init int x86_svm_init(void) { return -EOPNOTSUPP; }
*/
int x86_virt_emergency_disable_virtualization_cpu(void)
{
- /* Ensure the !feature check can't get false positives. */
- BUILD_BUG_ON(!X86_FEATURE_SVM || !X86_FEATURE_VMX);
-
if (!virt_ops.feature)
return -EOPNOTSUPP;
--
2.53.0.310.g728cabbaf7-goog
^ permalink raw reply related
* [PATCH v3 10/16] x86/virt/tdx: Drop the outdated requirement that TDX be enabled in IRQ context
From: Sean Christopherson @ 2026-02-14 1:26 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Peter Zijlstra, Arnaldo Carvalho de Melo,
Namhyung Kim, Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, linux-perf-users, Chao Gao,
Xu Yilun, Dan Williams
In-Reply-To: <20260214012702.2368778-1-seanjc@google.com>
Remove TDX's outdated requirement that per-CPU enabling be done via IPI
function call, which was a stale artifact leftover from early versions of
the TDX enablement series. The requirement that IRQs be disabled should
have been dropped as part of the revamped series that relied on a the KVM
rework to enable VMX at module load.
In other words, the kernel's "requirement" was never a requirement at all,
but instead a reflection of how KVM enabled VMX (via IPI callback) when
the TDX subsystem code was merged.
Note, accessing per-CPU information is safe even without disabling IRQs,
as tdx_online_cpu() is invoked via a cpuhp callback, i.e. from a per-CPU
thread.
Link: https://lore.kernel.org/all/ZyJOiPQnBz31qLZ7@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/vmx/tdx.c | 9 +--------
arch/x86/virt/vmx/tdx/tdx.c | 4 ----
2 files changed, 1 insertion(+), 12 deletions(-)
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 0c790eb0bfa6..582469118b79 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -3294,17 +3294,10 @@ int tdx_gmem_max_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, bool is_private)
static int tdx_online_cpu(unsigned int cpu)
{
- unsigned long flags;
- int r;
-
/* Sanity check CPU is already in post-VMXON */
WARN_ON_ONCE(!(cr4_read_shadow() & X86_CR4_VMXE));
- local_irq_save(flags);
- r = tdx_cpu_enable();
- local_irq_restore(flags);
-
- return r;
+ return tdx_cpu_enable();
}
static int tdx_offline_cpu(unsigned int cpu)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 5ce4ebe99774..dfd82fac0498 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -148,8 +148,6 @@ static int try_init_module_global(void)
* global initialization SEAMCALL if not done) on local cpu to make this
* cpu be ready to run any other SEAMCALLs.
*
- * Always call this function via IPI function calls.
- *
* Return 0 on success, otherwise errors.
*/
int tdx_cpu_enable(void)
@@ -160,8 +158,6 @@ int tdx_cpu_enable(void)
if (!boot_cpu_has(X86_FEATURE_TDX_HOST_PLATFORM))
return -ENODEV;
- lockdep_assert_irqs_disabled();
-
if (__this_cpu_read(tdx_lp_initialized))
return 0;
--
2.53.0.310.g728cabbaf7-goog
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox