* Re: [PATCH 01/15] x86/virt/tdx: Read global metadata for TDX Module Extensions
From: Xiaoyao Li @ 2026-05-25 6:24 UTC (permalink / raw)
To: Xu Yilun, kas, djbw, rick.p.edgecombe, x86, peter.fang
Cc: linux-coco, linux-kernel, kvm, sohil.mehta, yilun.xu, baolu.lu,
zhenzhong.duan
In-Reply-To: <20260522034128.3144354-2-yilun.xu@linux.intel.com>
On 5/22/2026 11:41 AM, Xu Yilun wrote:
> Add reading of the global metadata for TDX Module Extensions.
>
> TDX Module Extensions is an add-on feature enumerated by TDX_FEATURES0.
> But for the Module's integrity, Linux requires that all features that a
> Module advertises must have a complete, valid set of metadata,
I doubt on this.
1. Is it a must that any new feature introduces new metadata field?
2. Linux only cares the integrity for the features it uses, not for all
the features.
> and the
> validation must succeed at core TDX initialization time.
>
> Check TDX_FEATURES0 before reading these metadata. If a feature is
> advertised, a failure in reading associated metadata causes the entire
> TDX initialization to fail, otherwise skip.
I'm not sure why we need to explain the behavior when the reading fails.
It's not different to other existing fields.
Instead, I think you can explain why we need to check TDX_FEATURES0_EXT
at first.
Anyway, I don't read it as a good changelog. It event doesn't tell what
the added fields are and why we need them.
^ permalink raw reply
* Re: [PATCH 01/15] x86/virt/tdx: Read global metadata for TDX Module Extensions
From: Xiaoyao Li @ 2026-05-25 6:54 UTC (permalink / raw)
To: Xu Yilun, kas, djbw, rick.p.edgecombe, x86, peter.fang
Cc: linux-coco, linux-kernel, kvm, sohil.mehta, yilun.xu, baolu.lu,
zhenzhong.duan
In-Reply-To: <20260522034128.3144354-2-yilun.xu@linux.intel.com>
On 5/22/2026 11:41 AM, Xu Yilun wrote:
...
> +static __init int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)
> +{
> + int ret = 0;
> + u64 val;
> +
> + if (!ret && !(ret = read_sys_metadata_field(0x3100000100000000, &val)))
> + sysinfo_ext->memory_pool_required_pages = val;
> + if (!ret && !(ret = read_sys_metadata_field(0x3100000000000001, &val)))
> + sysinfo_ext->ext_required = val;
> +
> + return ret;
> +}
> +
> static __init int get_tdx_sys_info(struct tdx_sys_info *sysinfo)
> {
> int ret = 0;
> @@ -116,5 +129,8 @@ static __init int get_tdx_sys_info(struct tdx_sys_info *sysinfo)
> ret = ret ?: get_tdx_sys_info_td_ctrl(&sysinfo->td_ctrl);
> ret = ret ?: get_tdx_sys_info_td_conf(&sysinfo->td_conf);
>
> + if (sysinfo->features.tdx_features0 & TDX_FEATURES0_EXT)
> + ret = ret ?: get_tdx_sys_info_ext(&sysinfo->ext);
Is it correct to read "memory_pool_required_pages" and "ext_required" so
early in get_tdx_sys_info()? get_tdx_sys_info() is called before
config_tdx_module() which calls TDH.SYS.CONFIG.
If I read the TDX module base spec correctly, the amount of memory for
extensions and EXT_REQUIRED field depends on the enabled features, which
is determined by TDH.SYS.CONFIG/TDH.SYS.UPDATE ?
> return ret;
> }
^ permalink raw reply
* Re: [PATCH v14 06/44] arm64: RMI: Check for RMI support at init
From: Gavin Shan @ 2026-05-25 6:58 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Shanker Donthineni,
Alper Gun, Aneesh Kumar K . V, Emi Kisanuki, Vishal Annapurve,
WeiLin.Chang, Lorenzo.Pieralisi2
In-Reply-To: <78425c0d-86c5-457f-b171-a4c8dd3acb7d@arm.com>
Hi Steve,
On 5/22/26 1:49 AM, Steven Price wrote:
> On 21/05/2026 01:39, Gavin Shan wrote:
>> On 5/13/26 11:17 PM, Steven Price wrote:
>>> Query the RMI version number and check if it is a compatible version.
>>> The first two feature registers are read and exposed for future code to
>>> use.
>>>
>>> Signed-off-by: Steven Price <steven.price@arm.com>
>>> ---
>>> v14:
>>> * This moves the basic RMI setup into the 'kernel' directory. This is
>>> because RMI will be used for some features outside of KVM so should
>>> be available even if KVM isn't compiled in.
>>> ---
>>> arch/arm64/include/asm/rmi_cmds.h | 3 ++
>>> arch/arm64/kernel/Makefile | 2 +-
>>> arch/arm64/kernel/cpufeature.c | 1 +
>>> arch/arm64/kernel/rmi.c | 65 +++++++++++++++++++++++++++++++
>>> 4 files changed, 70 insertions(+), 1 deletion(-)
>>> create mode 100644 arch/arm64/kernel/rmi.c
>>>
>>
>> [...]
>>
>>> diff --git a/arch/arm64/kernel/rmi.c b/arch/arm64/kernel/rmi.c
>>> new file mode 100644
>>> index 000000000000..99c1ccc35c11
>>> --- /dev/null
>>> +++ b/arch/arm64/kernel/rmi.c
>>> @@ -0,0 +1,65 @@
>>> +// SPDX-License-Identifier: GPL-2.0
>>> +/*
>>> + * Copyright (C) 2023-2025 ARM Ltd.
>>> + */
>>> +
>>> +#include <linux/memblock.h>
>>> +
>>> +#include <asm/rmi_cmds.h>
>>> +
>>> +unsigned long rmm_feat_reg0;
>>> +unsigned long rmm_feat_reg1;
>>> +
>>> +static int rmi_check_version(void)
>>> +{
>>> + struct arm_smccc_res res;
>>> + unsigned short version_major, version_minor;
>>> + unsigned long host_version = RMI_ABI_VERSION(RMI_ABI_MAJOR_VERSION,
>>> + RMI_ABI_MINOR_VERSION);
>>> + unsigned long aa64pfr0 =
>>> read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
>>> +
>>> + /* If RME isn't supported, then RMI can't be */
>>> + if (cpuid_feature_extract_unsigned_field(aa64pfr0,
>>> ID_AA64PFR0_EL1_RME_SHIFT) == 0)
>>> + return -ENXIO;
>>> +
>>> + arm_smccc_1_1_invoke(SMC_RMI_VERSION, host_version, &res);
>>> +
>>> + if (res.a0 == SMCCC_RET_NOT_SUPPORTED)
>>> + return -ENXIO;
>>> +
>>> + version_major = RMI_ABI_VERSION_GET_MAJOR(res.a1);
>>> + version_minor = RMI_ABI_VERSION_GET_MINOR(res.a1);
>>> +
>>> + if (res.a0 != RMI_SUCCESS) {
>>> + unsigned short high_version_major, high_version_minor;
>>> +
>>> + high_version_major = RMI_ABI_VERSION_GET_MAJOR(res.a2);
>>> + high_version_minor = RMI_ABI_VERSION_GET_MINOR(res.a2);
>>> +
>>> + pr_err("Unsupported RMI ABI (v%d.%d - v%d.%d) we want v%d.%d\n",
>>> + version_major, version_minor,
>>> + high_version_major, high_version_minor,
>>> + RMI_ABI_MAJOR_VERSION,
>>> + RMI_ABI_MINOR_VERSION);
>>> + return -ENXIO;
>>> + }
>>> +
>>> + pr_info("RMI ABI version %d.%d\n", version_major, version_minor);
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static int __init arm64_init_rmi(void)
>>> +{
>>> + /* Continue without realm support if we can't agree on a version */
>>> + if (rmi_check_version())
>>> + return 0;
>>
>> Is this still a valid point that we have to return zero on errors returned
>> from rmi_check_version() or other other function calls like rmi_features()?
>> arm64_init_rmi() is triggered by subsys_initcall() where the return value
>> needs to indicate success or failure. It's fine to return error code from
>> arm64_init_rmi() in the path.
>
> Hmm, I guess now this is moved to arm64 code this indeed doesn't need
> to. Within a module I believe an error return can fail the module loading.
>
> I'm not sure it really makes much difference though - if this
> initialisation fails then it's not really an error - it just means the
> feature is unavailable.
>
I think the return value would be consistent to the value of 'arm64_rmi_is_available'.
'arm64_rmi_is_available' is true when zero is returned, otherwise, 'arm64_rmi_is_available'
is false.
With the consistency between the return value and 'arm64_rmi_is_available', users are
able to know the value of 'arm64_rmi_is_available' through kernel parameter 'initcall_debug'.
With the kernel parameter, the initcalls including arm64_init_rmi() are traced and its
return value is outputted in the traced messages, seeing do_trace_initcall_start().
> Thanks,
> Steve
>
>>> +
>>> + if (WARN_ON(rmi_features(0, &rmm_feat_reg0)))
>>> + return 0;
>>> + if (WARN_ON(rmi_features(1, &rmm_feat_reg1)))
>>> + return 0;
>>> +
>>> + return 0;
>>> +}
>>> +subsys_initcall(arm64_init_rmi);
>>
Thanks,
Gavin
^ permalink raw reply
* Re: [PATCH 04/15] x86/virt/tdx: Enable the Extensions right after basic TDX Module init
From: Xiaoyao Li @ 2026-05-25 8:05 UTC (permalink / raw)
To: Xu Yilun, kas, djbw, rick.p.edgecombe, x86, peter.fang
Cc: linux-coco, linux-kernel, kvm, sohil.mehta, yilun.xu, baolu.lu,
zhenzhong.duan
In-Reply-To: <20260522034128.3144354-5-yilun.xu@linux.intel.com>
On 5/22/2026 11:41 AM, Xu Yilun wrote:
> The detailed initialization flow for TDX Module Extensions has been
> fully implemented. Enable the flow after basic TDX Module
> initialization.
>
> Theoretically, the Extensions doesn't need to be enabled right after
> basic TDX initialization. It could be enabled right before the first
> Extension SEAMCALL is issued. That would save or postpone memory usage.
> But it isn't worth the complexity, the needs for the Extensions are vast
> but the savings are little for a typical TDX capable system (about
> 0.001% of memory). So the Linux decision is to just enable it along with
> the basic TDX.
> Note that the Extensions initialization flow will still not start if no
> add-on features require Extensions. The enabling of add-on features will
> be in later patches. Until then, the system hasn't consumed extra memory.
based on the above, how about putting this patch before patch 02 and 03?
so that we can eliminate the churn of add "__init" and the
"__maybe_unused " in patch 02.
To be more safer, we can even make the code as
static bool tdx_supports_extension(void)
{
/* To be enabled when kernel is ready. */
return false;
}
static __init int init_tdx_ext(void)
{
if (!tdx_supports_extension())
return 0;
/* No feature requires TDX Module Extensions. */
if (!tdx_sysinfo.ext.ext_required)
return 0;
}
and after all the pieces implemented, we can change
tdx_supports_extension() to
static bool tdx_supports_extension(void)
{
/* To be enabled when kernel is ready. */
return !!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT);
}
> Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
> ---
> arch/x86/virt/vmx/tdx/tdx.c | 16 ++++++++++------
> 1 file changed, 10 insertions(+), 6 deletions(-)
>
> diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> index ff2b96c20d2b..dad5ec642723 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.c
> +++ b/arch/x86/virt/vmx/tdx/tdx.c
> @@ -1180,7 +1180,7 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)
> return 0;
> }
>
> -static void tdx_clflush_hpa_list(struct page *root, unsigned int nr_pages)
> +static __init void tdx_clflush_hpa_list(struct page *root, unsigned int nr_pages)
> {
> u64 *entries = page_to_virt(root);
> int i;
> @@ -1193,7 +1193,7 @@ static void tdx_clflush_hpa_list(struct page *root, unsigned int nr_pages)
> #define HPA_LIST_INFO_PFN GENMASK_U64(51, 12)
> #define HPA_LIST_INFO_LAST_ENTRY GENMASK_U64(63, 55)
>
> -static u64 to_hpa_list_info(struct page *root, unsigned int nr_pages)
> +static __init u64 to_hpa_list_info(struct page *root, unsigned int nr_pages)
> {
> return FIELD_PREP(HPA_LIST_INFO_FIRST_ENTRY, 0) |
> FIELD_PREP(HPA_LIST_INFO_PFN, page_to_pfn(root)) |
> @@ -1201,7 +1201,7 @@ static u64 to_hpa_list_info(struct page *root, unsigned int nr_pages)
> }
>
> /* Initialize the TDX Module Extensions then Extension-SEAMCALLs can be used */
> -static int tdx_ext_init(void)
> +static __init int tdx_ext_init(void)
> {
> struct tdx_module_args args = {};
> u64 r;
> @@ -1216,7 +1216,7 @@ static int tdx_ext_init(void)
> return 0;
> }
>
> -static int tdx_ext_mem_add(struct page *root, unsigned int nr_pages)
> +static __init int tdx_ext_mem_add(struct page *root, unsigned int nr_pages)
> {
> struct tdx_module_args args = {
> .rcx = to_hpa_list_info(root, nr_pages),
> @@ -1240,7 +1240,7 @@ static int tdx_ext_mem_add(struct page *root, unsigned int nr_pages)
> return 0;
> }
>
> -static int tdx_ext_mem_setup(void)
> +static __init int tdx_ext_mem_setup(void)
> {
> unsigned int nr_pages;
> struct page *page;
> @@ -1301,7 +1301,7 @@ static int tdx_ext_mem_setup(void)
> return ret;
> }
>
> -static int __maybe_unused init_tdx_ext(void)
> +static __init int init_tdx_ext(void)
> {
> int ret;
>
> @@ -1373,6 +1373,10 @@ static __init int init_tdx_module(void)
> if (ret)
> goto err_reset_pamts;
>
> + ret = init_tdx_ext();
> + if (ret)
> + goto err_reset_pamts;
> +
> pr_info("%lu KB allocated for PAMT\n", tdmrs_count_pamt_kb(&tdx_tdmr_list));
>
> out_put_tdxmem:
^ permalink raw reply
* Re: [PATCH 02/15] x86/virt/tdx: Add extra memory to TDX Module for Extensions
From: Xiaoyao Li @ 2026-05-25 8:56 UTC (permalink / raw)
To: Xu Yilun, kas, djbw, rick.p.edgecombe, x86, peter.fang
Cc: linux-coco, linux-kernel, kvm, sohil.mehta, yilun.xu, baolu.lu,
zhenzhong.duan
In-Reply-To: <20260522034128.3144354-3-yilun.xu@linux.intel.com>
On 5/22/2026 11:41 AM, Xu Yilun wrote:
> TDX Module introduces a new concept called "TDX Module Extensions" to
> support long running / hard-irq preemptible flows inside. This makes TDX
> Module capable of handling complex tasks through "Extension SEAMCALLs".
> Adding more memory to TDX Module is the first step to enable Extensions.
>
> Currently, TDX Module memory use is relatively static. But, the
> Extensions need to use memory more dynamically. While 'static' here
> means the kernel provides necessary amount of memory to TDX Module for
> its basic functionalities, 'dynamic' means extra memory is needed only
> if new add-on features are to be enabled. So add a new memory feeding
> process backed by a new SEAMCALL TDH.EXT.MEM.ADD.
>
> The process is mostly the same as adding PAMT. The kernel queries TDX
> Module how much memory needed, allocates it, hands it over, and never
> gets it back.
>
> TDH.EXT.MEM.ADD uses a new parameter type HPA_LIST_INFO to provide
> control (private) pages to TDX Module. This type represents a list of
> pages for TDX Module to access. It needs a 'root page' which contains
> the list of HPAs of the pages. It collapses the HPA of the root page
> and the number of valid HPAs into a 64 bit raw value for SEAMCALL
> parameters. The root page is always a medium, TDX Module never keeps
> the root page.
>
> Introduce a tdx_clflush_hpa_list() helper to flush shared cache before
> SEAMCALL, to avoid shared cache writeback damaging these private pages.
>
> For now, TDX Module Extensions consumes relatively large amount of
> memory (~50MB). Use contiguous page allocation to avoid permanently
> fragment too much memory. Print the allocation amount on TDX Module
> Extensions initialization for visibility.
>
> Co-developed-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
> Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
> ---
> arch/x86/virt/vmx/tdx/tdx.h | 1 +
> arch/x86/virt/vmx/tdx/tdx.c | 118 ++++++++++++++++++++++++++++++++++++
> 2 files changed, 119 insertions(+)
>
> diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
> index a5eec8e3cc71..2335f88bbb10 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.h
> +++ b/arch/x86/virt/vmx/tdx/tdx.h
> @@ -46,6 +46,7 @@
> #define TDH_PHYMEM_PAGE_WBINVD 41
> #define TDH_VP_WR 43
> #define TDH_SYS_CONFIG 45
> +#define TDH_EXT_MEM_ADD 61
> #define TDH_SYS_DISABLE 69
>
> /*
> diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> index c0c6281b08a5..622399d8da68 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.c
> +++ b/arch/x86/virt/vmx/tdx/tdx.c
> @@ -31,6 +31,7 @@
> #include <linux/syscore_ops.h>
> #include <linux/idr.h>
> #include <linux/kvm_types.h>
> +#include <linux/bitfield.h>
> #include <asm/page.h>
> #include <asm/special_insns.h>
> #include <asm/msr-index.h>
> @@ -1179,6 +1180,123 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)
> return 0;
> }
>
> +static void tdx_clflush_hpa_list(struct page *root, unsigned int nr_pages)
> +{
> + u64 *entries = page_to_virt(root);
> + int i;
> +
> + for (i = 0; i < nr_pages; i++)
> + clflush_cache_range(__va(entries[i]), PAGE_SIZE);
Is the page flush only needed when CLFLUSH_BEFORE_ALLOC is true?
If so, it inherits the same decision to always flush as what
tdx_clflush_page() did. Then, any chance we can use tdx_clflush_page()
here so that we have a single central place of the comment to explain
the kernel design decision.
> +}
> +
> +#define HPA_LIST_INFO_FIRST_ENTRY GENMASK_U64(11, 3)
> +#define HPA_LIST_INFO_PFN GENMASK_U64(51, 12)
> +#define HPA_LIST_INFO_LAST_ENTRY GENMASK_U64(63, 55)
> +
> +static u64 to_hpa_list_info(struct page *root, unsigned int nr_pages)
> +{
> + return FIELD_PREP(HPA_LIST_INFO_FIRST_ENTRY, 0) |
> + FIELD_PREP(HPA_LIST_INFO_PFN, page_to_pfn(root)) |
> + FIELD_PREP(HPA_LIST_INFO_LAST_ENTRY, nr_pages - 1);
> +}
> +
> +static int tdx_ext_mem_add(struct page *root, unsigned int nr_pages)
> +{
> + struct tdx_module_args args = {
> + .rcx = to_hpa_list_info(root, nr_pages),
> + };
> + u64 r;
> +
> + tdx_clflush_hpa_list(root, nr_pages);
> +
> + do {
> + /*
> + * TDH_EXT_MEM_ADD is designed to use output parameter RCX to
> + * override/update input parameter RCX, so the caller doesn't
> + * have to do manual parameter update on retry call.
> + */
> + r = seamcall_ret(TDH_EXT_MEM_ADD, &args);
> + } while (r == TDX_INTERRUPTED_RESUMABLE);
> +
> + if (r != TDX_SUCCESS)
> + return -EFAULT;
> +
> + return 0;
> +}
> +
> +static int tdx_ext_mem_setup(void)
> +{
> + unsigned int nr_pages;
> + struct page *page;
> + u64 *root;
> + unsigned int i;
> + int ret;
> +
> + nr_pages = tdx_sysinfo.ext.memory_pool_required_pages;
> + /*
> + * memory_pool_required_pages == 0 means no need to add pages,
> + * skip the memory setup.
> + */
> + if (!nr_pages)
> + return 0;
> +
> + root = kzalloc(PAGE_SIZE, GFP_KERNEL);
> + if (!root)
> + return -ENOMEM;
> +
> + page = alloc_contig_pages(nr_pages, GFP_KERNEL, numa_mem_id(),
> + &node_online_map);
> + if (!page) {
> + ret = -ENOMEM;
> + goto out_free_root;
> + }
> +
> + for (i = 0; i < nr_pages;) {
> + unsigned int nents = min(nr_pages - i,
> + PAGE_SIZE / sizeof(*root));
> + int j;
> +
> + for (j = 0; j < nents; j++)
> + root[j] = page_to_phys(page + i + j);
> +
> + ret = tdx_ext_mem_add(virt_to_page(root), nents);
> + /*
> + * No SEAMCALLs to reclaim the added pages. For simple error
> + * handling, leak all pages.
> + */
> + WARN_ON_ONCE(ret);
> + if (ret)
> + break;
> +
> + i += nents;
> + }
> +
> + /*
> + * Extensions memory can't be reclaimed once added, print out the
> + * amount, stop tracking it and free the root page, no matter success
> + * or failure.
> + */
> + pr_info("%lu KB allocated for TDX Module Extensions\n",
> + nr_pages * PAGE_SIZE / 1024);
> +
> +out_free_root:
> + kfree(root);
> +
> + return ret;
> +}
> +
> +static int __maybe_unused init_tdx_ext(void)
> +{
> + if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
> + return 0;
> +
> + /* No feature requires TDX Module Extensions. */
> + if (!tdx_sysinfo.ext.ext_required)
> + return 0;
> +
> + return tdx_ext_mem_setup();
> +}
> +
> static __init int init_tdx_module(void)
> {
> int ret;
^ permalink raw reply
* Re: [PATCH 03/15] x86/virt/tdx: Make TDX Module initialize Extensions
From: Xiaoyao Li @ 2026-05-25 8:58 UTC (permalink / raw)
To: Xu Yilun, kas, djbw, rick.p.edgecombe, x86, peter.fang
Cc: linux-coco, linux-kernel, kvm, sohil.mehta, yilun.xu, baolu.lu,
zhenzhong.duan
In-Reply-To: <20260522034128.3144354-4-yilun.xu@linux.intel.com>
On 5/22/2026 11:41 AM, Xu Yilun wrote:
> After providing all required memory to TDX Module, initialize TDX
> Module Extensions via TDH.EXT.INIT, so Extension-SEAMCALLs can be used.
>
> Co-developed-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
> Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> arch/x86/virt/vmx/tdx/tdx.h | 1 +
> arch/x86/virt/vmx/tdx/tdx.c | 24 +++++++++++++++++++++++-
> 2 files changed, 24 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
> index 2335f88bbb10..c5bffd118145 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.h
> +++ b/arch/x86/virt/vmx/tdx/tdx.h
> @@ -46,6 +46,7 @@
> #define TDH_PHYMEM_PAGE_WBINVD 41
> #define TDH_VP_WR 43
> #define TDH_SYS_CONFIG 45
> +#define TDH_EXT_INIT 60
> #define TDH_EXT_MEM_ADD 61
> #define TDH_SYS_DISABLE 69
>
> diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> index 622399d8da68..ff2b96c20d2b 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.c
> +++ b/arch/x86/virt/vmx/tdx/tdx.c
> @@ -1200,6 +1200,22 @@ static u64 to_hpa_list_info(struct page *root, unsigned int nr_pages)
> FIELD_PREP(HPA_LIST_INFO_LAST_ENTRY, nr_pages - 1);
> }
>
> +/* Initialize the TDX Module Extensions then Extension-SEAMCALLs can be used */
> +static int tdx_ext_init(void)
> +{
> + struct tdx_module_args args = {};
> + u64 r;
> +
> + do {
> + r = seamcall(TDH_EXT_INIT, &args);
> + } while (r == TDX_INTERRUPTED_RESUMABLE);
> +
> + if (r != TDX_SUCCESS)
> + return -EFAULT;
> +
> + return 0;
> +}
> +
> static int tdx_ext_mem_add(struct page *root, unsigned int nr_pages)
> {
> struct tdx_module_args args = {
> @@ -1287,6 +1303,8 @@ static int tdx_ext_mem_setup(void)
>
> static int __maybe_unused init_tdx_ext(void)
> {
> + int ret;
> +
> if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
> return 0;
>
> @@ -1294,7 +1312,11 @@ static int __maybe_unused init_tdx_ext(void)
> if (!tdx_sysinfo.ext.ext_required)
> return 0;
>
> - return tdx_ext_mem_setup();
> + ret = tdx_ext_mem_setup();
> + if (ret)
> + return ret;
> +
> + return tdx_ext_init();
> }
>
> static __init int init_tdx_module(void)
^ permalink raw reply
* Re: [RFC PATCH 14/15] x86/virt/tdx: Embed version info in SEAMCALL leaf function definitions
From: Xiaoyao Li @ 2026-05-25 9:00 UTC (permalink / raw)
To: Xu Yilun, kas, djbw, rick.p.edgecombe, x86, peter.fang
Cc: linux-coco, linux-kernel, kvm, sohil.mehta, yilun.xu, baolu.lu,
zhenzhong.duan
In-Reply-To: <20260522034128.3144354-15-yilun.xu@linux.intel.com>
On 5/22/2026 11:41 AM, Xu Yilun wrote:
> Embed version information in SEAMCALL leaf function definitions rather
> than let the caller open code them. For now, only TDH.VP.INIT is
> involved.
>
> Don't bother the caller to choose the SEAMCALL version if unnecessary.
> New version SEAMCALLs are guaranteed to be backward compatible, so
> ideally kernel doesn't need to keep version history and only uses the
> latest version SEAMCALLs.
>
> The concern is some old TDX Modules don't recognize new version
> SEAMCALLs. Multiple SEAMCALL versions co-exist when kernel should
> support these old Modules. As time goes by, the old Modules deprecate
> and old version SEAMCALL definitions should disappear.
>
> The old TDX Modules that only support TDH.VP.INIT v0 are all deprecated,
> so only provide the latest (v1) definition.
>
> Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
> ---
> arch/x86/virt/vmx/tdx/tdx.h | 23 ++++++++++++++---------
> arch/x86/virt/vmx/tdx/tdx.c | 4 ++--
> 2 files changed, 16 insertions(+), 11 deletions(-)
>
> diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
> index 01a7d7d8ada9..10aff23cd01f 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.h
> +++ b/arch/x86/virt/vmx/tdx/tdx.h
> @@ -2,6 +2,7 @@
> #ifndef _X86_VIRT_TDX_H
> #define _X86_VIRT_TDX_H
>
> +#include <linux/bitfield.h>
> #include <linux/bits.h>
>
> /*
> @@ -11,6 +12,18 @@
> * architectural definitions come first.
> */
>
> +/*
> + * SEAMCALL leaf:
> + *
> + * Bit 15:0 Leaf number
> + * Bit 23:16 Version number
> + */
> +#define SEAMCALL_LEAF GENMASK(15, 0)
> +#define SEAMCALL_VER GENMASK(23, 16)
> +
> +#define SEAMCALL_LEAF_VER(l, v) (FIELD_PREP(SEAMCALL_LEAF, l) | \
> + FIELD_PREP(SEAMCALL_VER, v))
> +
> /*
> * TDX module SEAMCALL leaf functions
> */
> @@ -31,7 +44,7 @@
> #define TDH_VP_CREATE 10
> #define TDH_MNG_KEY_FREEID 20
> #define TDH_MNG_INIT 21
> -#define TDH_VP_INIT 22
> +#define TDH_VP_INIT SEAMCALL_LEAF_VER(22, 1)
how about
#define TDH_VP_INIT 22
#define TDH_VP_INIT_V1 SEAMCALL_LEAF_VER(TDH_VP_INIT, 1)
and use TDH_VP_INIT_V1 below?
> #define TDH_PHYMEM_PAGE_RDMD 24
> #define TDH_VP_RD 26
> #define TDH_PHYMEM_PAGE_RECLAIM 28
> @@ -52,14 +65,6 @@
> #define TDH_QUOTE_GET 98
> #define TDH_QUOTE_INIT 100
>
> -/*
> - * SEAMCALL leaf:
> - *
> - * Bit 15:0 Leaf number
> - * Bit 23:16 Version number
> - */
> -#define TDX_VERSION_SHIFT 16
> -
> /* TDX page types */
> #define PT_NDA 0x0
> #define PT_RSVD 0x1
> diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> index 821f677e9a86..f7600f930c6e 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.c
> +++ b/arch/x86/virt/vmx/tdx/tdx.c
> @@ -2217,8 +2217,8 @@ u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid)
> .r8 = x2apicid,
> };
>
> - /* apicid requires version == 1. */
> - return seamcall(TDH_VP_INIT | (1ULL << TDX_VERSION_SHIFT), &args);
> + /* apicid requires version == 1. See TDH_VP_INIT definition.*/
> + return seamcall(TDH_VP_INIT, &args);
> }
> EXPORT_SYMBOL_FOR_KVM(tdh_vp_init);
>
^ permalink raw reply
* Re: [RFC PATCH 15/15] x86/virt/tdx: Enable TDX Quoting extension
From: Xiaoyao Li @ 2026-05-25 10:51 UTC (permalink / raw)
To: Tony Lindgren, Xu Yilun
Cc: kas, djbw, rick.p.edgecombe, x86, peter.fang, linux-coco,
linux-kernel, kvm, sohil.mehta, yilun.xu, baolu.lu,
zhenzhong.duan
In-Reply-To: <ahPbb1Ws9hBruJ2d@tlindgre-MOBL1>
On 5/25/2026 1:17 PM, Tony Lindgren wrote:
> On Fri, May 22, 2026 at 11:41:28AM +0800, Xu Yilun wrote:
>> From: Peter Fang <peter.fang@intel.com>
>>
>> TDX Module updates global metadata when add-on features are enabled.
>> Host should update the cached tdx_sysinfo to reflect these changes.
>
> This should be made clearer IMO. How about mention that get_tdx_sys_info()
> needs to get called again to reload the TDX module global metadata?
Ah ha! This patch answers my comment to patch 1:
https://lore.kernel.org/all/956fa1e6-2920-4b2e-8037-d4b9d812ae53@intel.com/
sysinfo_ext->memory_pool_required_pages and sysinfo_ext->ext_required
will be updated after extensions are enabled by TDH.SYS.CONFIG.
Patch 06 in this series already reads the tdx_sys_info_quote out of
get_tdx_sys_info(), which mean get_tdx_sys_info() doesn't ensure all the
global metadata will be update again.
So how about move the read of memory_pool_required_pages and
ext_required out of get_tdx_sys_info() and put them after
TDH.SYS.CONFIG, so that we don't need call get_tdx_sys_info() again?
>> --- a/arch/x86/virt/vmx/tdx/tdx.c
>> +++ b/arch/x86/virt/vmx/tdx/tdx.c
>> @@ -1049,6 +1049,7 @@ static __init int construct_tdmrs(struct list_head *tmb_list,
>> static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
>> u64 global_keyid)
>> {
>> + u64 seamcall_fn = TDH_SYS_CONFIG_V0;
>> struct tdx_module_args args = {};
>> u64 *tdmr_pa_array;
>> size_t array_sz;
>> @@ -1074,8 +1075,22 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
>> args.rcx = __pa(tdmr_pa_array);
>> args.rdx = tdmr_list->nr_consumed_tdmrs;
>> args.r8 = global_keyid;
>> - ret = seamcall_prerr(TDH_SYS_CONFIG, &args);
>>
>> + if (tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_QUOTE) {
>> + args.r9 |= TDX_FEATURES0_QUOTE;
>> + /* These parameters require version >= 1 */
>> + seamcall_fn = TDH_SYS_CONFIG;
>> + }
>> +
>> + ret = seamcall_prerr(seamcall_fn, &args);
>> + if (ret)
>> + goto free_tdmr;
>> +
>> + /* enabling TDX Quoting may change tdx_sysinfo, update it */
>> + if (tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_QUOTE)
>> + ret = get_tdx_sys_info(&tdx_sysinfo);
>
> The comment above helps, but the change in the handling will be easy to
> miss.
>
>> +free_tdmr:
>> /* Free the array as it is not required anymore. */
>> kfree(tdmr_pa_array);
>>
>
> So I think it would be good to also add a comment to get_tdx_sys_info()
> to make it easier for folks to follow that it may get called multiple
> times.
>
> Regards,
>
> Tony
^ permalink raw reply
* [PATCH v5 0/5] Add iommufd ioctls to support TSM operations
From: Aneesh Kumar K.V (Arm) @ 2026-05-25 15:48 UTC (permalink / raw)
To: linux-coco, iommu, linux-kernel, kvm
Cc: Aneesh Kumar K.V (Arm), Alexey Kardashevskiy, Bjorn Helgaas,
Dan Williams, Jason Gunthorpe, Joerg Roedel, Jonathan Cameron,
Kevin Tian, Nicolin Chen, Samuel Ortiz, Steven Price,
Suzuki K Poulose, Will Deacon, Xu Yilun, Shameer Kolothum,
Paolo Bonzini, Tony Krowiak, Halil Pasic, Jason Herne,
Harald Freudenberger, Holger Dengler, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Alex Williamson, Matthew Rosato, Farhan Ali,
Eric Farman, linux-s390
This patch series adds iommufd ioctl support for TSM-related operations.
These ioctls allow VMMs to perform TSM management tasks such as bind and
unbind operations, and to handle guest requests.
Changes from v4:
https://lore.kernel.org/all/20260427061005.901854-1-aneesh.kumar@kernel.org
* Switch VFIO/iommufd to use struct file *kvm_file instead of relying on
kvm->users_count references.
* Define TSM request scope values globally in iommufd.
* Rename the ioctl to IOMMU_VDEVICE_TSM_REQ.
* Address other review feedback.
Changes from v2:
https://lore.kernel.org/all/20260309111704.2330479-1-aneesh.kumar@kernel.org
* Bump the series revision to v4 to keep it in sync with the dependent CCA DA
patchsets. There was no v3 posting.
* Drop [PATCH v2 1/3] iommufd/viommu: Allow associating a KVM VM fd with a
vIOMMU
* Add two new patches to associate a struct kvm * with iommufd objects:
iommufd/device: Associate a kvm pointer to iommufd_device
iommufd/viommu: Associate a kvm pointer to iommufd_viommu
* Address review feedback
Changes from v1:
https://lore.kernel.org/all/20250728135216.48084-8-aneesh.kumar@kernel.org
* Rebase onto the latest kernel
* Address review feedback
* Drop the TSM map ioctl; the KVM prefault patch will be used instead to
ensure that private memory is preallocated
Cc: Alexey Kardashevskiy <aik@amd.com>
Cc: Bjorn Helgaas <helgaas@kernel.org>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Jonathan Cameron <jic23@kernel.org>
Cc: Kevin Tian <kevin.tian@intel.com>
Cc: Nicolin Chen <nicolinc@nvidia.com>
Cc: Samuel Ortiz <sameo@rivosinc.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Xu Yilun <yilun.xu@linux.intel.com>
Cc: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Tony Krowiak <akrowiak@linux.ibm.com>
Cc: Halil Pasic <pasic@linux.ibm.com>
Cc: Jason Herne <jjherne@linux.ibm.com>
Cc: Harald Freudenberger <freude@linux.ibm.com>
Cc: Holger Dengler <dengler@linux.ibm.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Alex Williamson <alex@shazbot.org>
Cc: Matthew Rosato <mjrosato@linux.ibm.com>
Cc: Farhan Ali <alifm@linux.ibm.com>
Cc: Eric Farman <farman@linux.ibm.com>
Cc: linux-s390@vger.kernel.org
Aneesh Kumar K.V (Arm) (3):
vfio: cache KVM VM file references instead of raw struct kvm pointers
iommufd/tsm: add vdevice TSM bind/unbind ioctl
iommufd/vdevice: add TSM request ioctl
Nicolin Chen (1):
iommufd/viommu: Keep a reference to the KVM file
Shameer Kolothum (1):
iommufd/device: Associate KVM file pointer with iommufd_device
drivers/iommu/iommufd/Makefile | 2 +
drivers/iommu/iommufd/device.c | 7 +-
drivers/iommu/iommufd/iommufd_private.h | 16 +++
drivers/iommu/iommufd/main.c | 6 ++
drivers/iommu/iommufd/selftest.c | 2 +-
drivers/iommu/iommufd/tsm.c | 130 ++++++++++++++++++++++++
drivers/iommu/iommufd/viommu.c | 9 ++
drivers/s390/crypto/vfio_ap_ops.c | 5 +-
drivers/vfio/device_cdev.c | 10 +-
drivers/vfio/group.c | 14 ++-
drivers/vfio/iommufd.c | 3 +-
drivers/vfio/pci/vfio_pci_zdev.c | 7 +-
drivers/vfio/vfio.h | 16 ++-
drivers/vfio/vfio_main.c | 81 ++++++++-------
drivers/virt/coco/tsm-core.c | 58 +++++++++++
include/linux/iommufd.h | 5 +-
include/linux/kvm_host.h | 3 +
include/linux/pci-tsm.h | 9 +-
include/linux/tsm.h | 42 ++++++++
include/linux/vfio.h | 17 +++-
include/uapi/linux/iommufd.h | 106 +++++++++++++++++++
virt/kvm/kvm_main.c | 2 +
22 files changed, 478 insertions(+), 72 deletions(-)
create mode 100644 drivers/iommu/iommufd/tsm.c
base-commit: 50897c955902c93ae71c38698abb910525ebdc89
--
2.43.0
^ permalink raw reply
* [PATCH v5 1/5] vfio: cache KVM VM file references instead of raw struct kvm pointers
From: Aneesh Kumar K.V (Arm) @ 2026-05-25 15:48 UTC (permalink / raw)
To: linux-coco, iommu, linux-kernel, kvm
Cc: Aneesh Kumar K.V (Arm), Alexey Kardashevskiy, Bjorn Helgaas,
Dan Williams, Jason Gunthorpe, Joerg Roedel, Jonathan Cameron,
Kevin Tian, Nicolin Chen, Samuel Ortiz, Steven Price,
Suzuki K Poulose, Will Deacon, Xu Yilun, Shameer Kolothum,
Paolo Bonzini, Tony Krowiak, Halil Pasic, Jason Herne,
Harald Freudenberger, Holger Dengler, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Alex Williamson, Matthew Rosato, Farhan Ali,
Eric Farman, linux-s390
In-Reply-To: <20260525154816.1029642-1-aneesh.kumar@kernel.org>
VFIO currently records struct kvm pointers on vfio_group, vfio_device_file
and the opened vfio_device. Switch VFIO to track the VM's struct file
instead, so VFIO and iommufd can use normal file references for VM lifetime
instead of depending on KVM's internal struct kvm refcounting.
KVM_CREATE_DEVICE binds the KVM VM lifetime to the KVM device fd lifetime.
For KVM_DEV_TYPE_VFIO, the KVM VFIO device fd also takes references to each
VFIO file added through KVM_DEV_VFIO_FILE_ADD. The KVM VFIO device fd
therefore owns both the internal KVM reference and the VFIO file references
in kvf->file.
KVM_DEV_VFIO_FILE_ADD further installs the VM file association into the
VFIO file. VFIO converts the struct kvm pointer to a VM file reference with
get_file_active(&kvm->_file), because the KVM device fd can keep struct kvm
alive after the original VM fd is already in final release.
The association intentionally pins the VM file until KVM_DEV_VFIO_FILE_DEL
or until the KVM VFIO device fd is released. This gives VFIO/iommufd a
stable VM file reference source without taking a dependency on KVM's struct
kvm lifetime. The KVM VFIO device release path clears the VFIO-side
association before dropping its VFIO file references.
When a VFIO device is opened or bound, VFIO takes an additional reference
from the associated VM file and stores it in vfio_device::kvm_file for
driver and iommufd use. That open-time reference is released from
vfio_device_put_kvm() when the VFIO device is closed or unbound.
This gives the ownership model:
- KVM device fd pins struct kvm through kvm->users_count
- KVM VFIO device fd pins VFIO files through kvf->file
- VFIO group/device-file state pins the VM file while associated with KVM
- vfio_device::kvm_file pins the VM file during active VFIO device use
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
drivers/s390/crypto/vfio_ap_ops.c | 5 +-
drivers/vfio/device_cdev.c | 10 ++--
drivers/vfio/group.c | 14 +++---
drivers/vfio/pci/vfio_pci_zdev.c | 7 +--
drivers/vfio/vfio.h | 16 ++++--
drivers/vfio/vfio_main.c | 81 ++++++++++++++++---------------
include/linux/kvm_host.h | 3 ++
include/linux/vfio.h | 17 ++++++-
virt/kvm/kvm_main.c | 2 +
9 files changed, 91 insertions(+), 64 deletions(-)
diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
index 44b3a1dcc1b3..05996a8fd860 100644
--- a/drivers/s390/crypto/vfio_ap_ops.c
+++ b/drivers/s390/crypto/vfio_ap_ops.c
@@ -2054,11 +2054,12 @@ static int vfio_ap_mdev_open_device(struct vfio_device *vdev)
{
struct ap_matrix_mdev *matrix_mdev =
container_of(vdev, struct ap_matrix_mdev, vdev);
+ struct kvm *kvm = vfio_device_get_kvm(vdev);
- if (!vdev->kvm)
+ if (!kvm)
return -EINVAL;
- return vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm);
+ return vfio_ap_mdev_set_kvm(matrix_mdev, kvm);
}
static void vfio_ap_mdev_close_device(struct vfio_device *vdev)
diff --git a/drivers/vfio/device_cdev.c b/drivers/vfio/device_cdev.c
index 54abf312cf04..ca75ab8eb7bd 100644
--- a/drivers/vfio/device_cdev.c
+++ b/drivers/vfio/device_cdev.c
@@ -56,7 +56,7 @@ int vfio_device_fops_cdev_open(struct inode *inode, struct file *filep)
static void vfio_df_get_kvm_safe(struct vfio_device_file *df)
{
spin_lock(&df->kvm_ref_lock);
- vfio_device_get_kvm_safe(df->device, df->kvm);
+ vfio_device_get_kvm_safe(df->device, df->kvm_file);
spin_unlock(&df->kvm_ref_lock);
}
@@ -133,10 +133,10 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df,
}
/*
- * Before the device open, get the KVM pointer currently
- * associated with the device file (if there is) and obtain
- * a reference. This reference is held until device closed.
- * Save the pointer in the device for use by drivers.
+ * Before the device open, get the VM struct file currently
+ * associated with the device file (if there is one) and obtain a
+ * reference. This reference is held until the device is closed.
+ * Save the file in the device for use by drivers.
*/
vfio_df_get_kvm_safe(df);
diff --git a/drivers/vfio/group.c b/drivers/vfio/group.c
index b2299e5bc6df..8950cfb9405d 100644
--- a/drivers/vfio/group.c
+++ b/drivers/vfio/group.c
@@ -163,7 +163,7 @@ static int vfio_group_ioctl_set_container(struct vfio_group *group,
static void vfio_device_group_get_kvm_safe(struct vfio_device *device)
{
spin_lock(&device->group->kvm_ref_lock);
- vfio_device_get_kvm_safe(device, device->group->kvm);
+ vfio_device_get_kvm_safe(device, device->group->kvm_file);
spin_unlock(&device->group->kvm_ref_lock);
}
@@ -181,10 +181,10 @@ static int vfio_df_group_open(struct vfio_device_file *df)
mutex_lock(&device->dev_set->lock);
/*
- * Before the first device open, get the KVM pointer currently
- * associated with the group (if there is one) and obtain a reference
- * now that will be held until the open_count reaches 0 again. Save
- * the pointer in the device for use by drivers.
+ * Before the first device open, get the VM struct file currently
+ * associated with the group (if there is one) and obtain a
+ * reference now that will be held until the open_count reaches 0
+ * again. Save the file in the device for use by drivers.
*/
if (device->open_count == 0)
vfio_device_group_get_kvm_safe(device);
@@ -862,9 +862,7 @@ bool vfio_group_enforced_coherent(struct vfio_group *group)
void vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm)
{
- spin_lock(&group->kvm_ref_lock);
- group->kvm = kvm;
- spin_unlock(&group->kvm_ref_lock);
+ vfio_kvm_file_replace(&group->kvm_file, &group->kvm_ref_lock, kvm);
}
/**
diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_zdev.c
index 0990fdb146b7..a9d8e6aa3839 100644
--- a/drivers/vfio/pci/vfio_pci_zdev.c
+++ b/drivers/vfio/pci/vfio_pci_zdev.c
@@ -144,15 +144,16 @@ int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device *vdev,
int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev)
{
struct zpci_dev *zdev = to_zpci(vdev->pdev);
+ struct kvm *kvm = vfio_device_get_kvm(&vdev->vdev);
if (!zdev)
return -ENODEV;
- if (!vdev->vdev.kvm)
+ if (!kvm)
return 0;
if (zpci_kvm_hook.kvm_register)
- return zpci_kvm_hook.kvm_register(zdev, vdev->vdev.kvm);
+ return zpci_kvm_hook.kvm_register(zdev, kvm);
return -ENOENT;
}
@@ -161,7 +162,7 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev)
{
struct zpci_dev *zdev = to_zpci(vdev->pdev);
- if (!zdev || !vdev->vdev.kvm)
+ if (!zdev || !vfio_device_get_kvm(&vdev->vdev))
return;
if (zpci_kvm_hook.kvm_unregister)
diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
index e4b72e79b7e3..41032104eb36 100644
--- a/drivers/vfio/vfio.h
+++ b/drivers/vfio/vfio.h
@@ -22,8 +22,8 @@ struct vfio_device_file {
u8 access_granted;
u32 devid; /* only valid when iommufd is valid */
- spinlock_t kvm_ref_lock; /* protect kvm field */
- struct kvm *kvm;
+ spinlock_t kvm_ref_lock; /* protect kvm_file */
+ struct file *kvm_file;
struct iommufd_ctx *iommufd; /* protected by struct vfio_device_set::lock */
};
@@ -88,7 +88,7 @@ struct vfio_group {
#endif
enum vfio_group_type type;
struct mutex group_lock;
- struct kvm *kvm;
+ struct file *kvm_file;
struct file *opened_file;
struct iommufd_ctx *iommufd;
spinlock_t kvm_ref_lock;
@@ -434,11 +434,17 @@ static inline void vfio_virqfd_exit(void)
#endif
#if IS_ENABLED(CONFIG_KVM)
-void vfio_device_get_kvm_safe(struct vfio_device *device, struct kvm *kvm);
+void vfio_kvm_file_replace(struct file **dst, spinlock_t *lock, struct kvm *kvm);
+void vfio_device_get_kvm_safe(struct vfio_device *device, struct file *kvm_file);
void vfio_device_put_kvm(struct vfio_device *device);
#else
+static inline void vfio_kvm_file_replace(struct file **dst,
+ spinlock_t *lock, struct kvm *kvm)
+{
+}
+
static inline void vfio_device_get_kvm_safe(struct vfio_device *device,
- struct kvm *kvm)
+ struct file *kvm_file)
{
}
diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index 6222376ab6ab..88c85a7b98c0 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -442,55 +442,61 @@ void vfio_unregister_group_dev(struct vfio_device *device)
EXPORT_SYMBOL_GPL(vfio_unregister_group_dev);
#if IS_ENABLED(CONFIG_KVM)
-void vfio_device_get_kvm_safe(struct vfio_device *device, struct kvm *kvm)
+void vfio_kvm_file_replace(struct file **dst, spinlock_t *lock, struct kvm *kvm)
{
- void (*pfn)(struct kvm *kvm);
- bool (*fn)(struct kvm *kvm);
- bool ret;
+ struct file *old_kvm_file, *new_kvm_file = NULL;
- lockdep_assert_held(&device->dev_set->lock);
+ /*
+ * @kvm can outlive the VM fd and its final __fput(). Only take a
+ * new reference if the VM file is still active.
+ */
+ if (kvm)
+ new_kvm_file = get_file_active(&kvm->_file);
- if (!kvm)
- return;
+ spin_lock(lock);
+ old_kvm_file = *dst;
+ *dst = new_kvm_file;
+ spin_unlock(lock);
- pfn = symbol_get(kvm_put_kvm);
- if (WARN_ON(!pfn))
- return;
+ if (old_kvm_file)
+ fput(old_kvm_file);
+}
- fn = symbol_get(kvm_get_kvm_safe);
- if (WARN_ON(!fn)) {
- symbol_put(kvm_put_kvm);
- return;
- }
+void vfio_device_get_kvm_safe(struct vfio_device *device, struct file *kvm_file)
+{
+ lockdep_assert_held(&device->dev_set->lock);
- ret = fn(kvm);
- symbol_put(kvm_get_kvm_safe);
- if (!ret) {
- symbol_put(kvm_put_kvm);
- return;
- }
+ /*
+ * Take a VM file reference if the KVM fd is still active.
+ */
+ if (kvm_file)
+ kvm_file = get_file(kvm_file);
- device->put_kvm = pfn;
- device->kvm = kvm;
+ device->kvm_file = kvm_file;
}
void vfio_device_put_kvm(struct vfio_device *device)
{
+ struct file *kvm_file;
+
lockdep_assert_held(&device->dev_set->lock);
- if (!device->kvm)
+ kvm_file = device->kvm_file;
+ if (!kvm_file)
return;
- if (WARN_ON(!device->put_kvm))
- goto clear;
+ device->kvm_file = NULL;
+ fput(kvm_file);
+}
- device->put_kvm(device->kvm);
- device->put_kvm = NULL;
- symbol_put(kvm_put_kvm);
+struct kvm *vfio_device_get_kvm(struct vfio_device *device)
+{
+ if (!device->kvm_file)
+ return NULL;
-clear:
- device->kvm = NULL;
+ return device->kvm_file->private_data;
}
+EXPORT_SYMBOL_GPL(vfio_device_get_kvm);
#endif
/* true if the vfio_device has open_device() called but not close_device() */
@@ -1518,13 +1524,10 @@ static void vfio_device_file_set_kvm(struct file *file, struct kvm *kvm)
struct vfio_device_file *df = file->private_data;
/*
- * The kvm is first recorded in the vfio_device_file, and will
- * be propagated to vfio_device::kvm when the file is bound to
- * iommufd successfully in the vfio device cdev path.
+ * Cache the VM file reference associated with this VFIO file so it
+ * can be pinned into vfio_device while the device is open.
*/
- spin_lock(&df->kvm_ref_lock);
- df->kvm = kvm;
- spin_unlock(&df->kvm_ref_lock);
+ vfio_kvm_file_replace(&df->kvm_file, &df->kvm_ref_lock, kvm);
}
/**
@@ -1532,8 +1535,8 @@ static void vfio_device_file_set_kvm(struct file *file, struct kvm *kvm)
* @file: VFIO group file or VFIO device file
* @kvm: KVM to link
*
- * When a VFIO device is first opened the KVM will be available in
- * device->kvm if one was associated with the file.
+ * When a VFIO device is first opened, VFIO caches a VM file reference if
+ * one was associated with the file.
*/
void vfio_file_set_kvm(struct file *file, struct kvm *kvm)
{
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 4c14aee1fb06..31afac5fb0ea 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -45,6 +45,8 @@
#include <asm/kvm_host.h>
#include <linux/kvm_dirty_ring.h>
+struct file;
+
#ifndef KVM_MAX_VCPU_IDS
#define KVM_MAX_VCPU_IDS KVM_MAX_VCPUS
#endif
@@ -861,6 +863,7 @@ struct kvm {
struct srcu_struct srcu;
struct srcu_struct irq_srcu;
pid_t userspace_pid;
+ struct file __rcu *_file;
bool override_halt_poll_ns;
unsigned int max_halt_poll_ns;
u32 dirty_ring_size;
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index 31b826efba00..bca1d00f7845 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -22,8 +22,22 @@ struct kvm;
struct iommufd_ctx;
struct iommufd_device;
struct iommufd_access;
+struct vfio_device;
struct vfio_info_cap;
+#if IS_ENABLED(CONFIG_KVM)
+/*
+ * Return the KVM associated with @vdev's kvm_file. The returned pointer
+ * is valid only while VFIO device open holds the kvm_file reference.
+ */
+struct kvm *vfio_device_get_kvm(struct vfio_device *vdev);
+#else
+static inline struct kvm *vfio_device_get_kvm(struct vfio_device *vdev)
+{
+ return NULL;
+}
+#endif
+
/*
* VFIO devices can be placed in a set, this allows all devices to share this
* structure and the VFIO core will provide a lock that is held around
@@ -54,7 +68,7 @@ struct vfio_device {
struct list_head dev_set_list;
unsigned int migration_flags;
u8 precopy_info_v2;
- struct kvm *kvm;
+ struct file *kvm_file;
/* Members below here are private, not for driver use */
unsigned int index;
@@ -66,7 +80,6 @@ struct vfio_device {
unsigned int open_count;
struct completion comp;
struct iommufd_access *iommufd_access;
- void (*put_kvm)(struct kvm *kvm);
struct inode *inode;
#if IS_ENABLED(CONFIG_IOMMUFD)
struct iommufd_device *iommufd_device;
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 89489996fbc1..011819c5c47c 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -1351,6 +1351,7 @@ static int kvm_vm_release(struct inode *inode, struct file *filp)
kvm_irqfd_release(kvm);
+ RCU_INIT_POINTER(kvm->_file, NULL);
kvm_put_kvm(kvm);
return 0;
}
@@ -5500,6 +5501,7 @@ static int kvm_dev_ioctl_create_vm(unsigned long type)
r = PTR_ERR(file);
goto put_kvm;
}
+ rcu_assign_pointer(kvm->_file, file);
/*
* Don't call kvm_put_kvm anymore at this point; file->f_op is
--
2.43.0
^ permalink raw reply related
* [PATCH v5 2/5] iommufd/device: Associate KVM file pointer with iommufd_device
From: Aneesh Kumar K.V (Arm) @ 2026-05-25 15:48 UTC (permalink / raw)
To: linux-coco, iommu, linux-kernel, kvm
Cc: Aneesh Kumar K.V (Arm), Alexey Kardashevskiy, Bjorn Helgaas,
Dan Williams, Jason Gunthorpe, Joerg Roedel, Jonathan Cameron,
Kevin Tian, Nicolin Chen, Samuel Ortiz, Steven Price,
Suzuki K Poulose, Will Deacon, Xu Yilun, Shameer Kolothum,
Paolo Bonzini, Tony Krowiak, Halil Pasic, Jason Herne,
Harald Freudenberger, Holger Dengler, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Alex Williamson, Matthew Rosato, Farhan Ali,
Eric Farman, linux-s390, Jason Gunthorpe
In-Reply-To: <20260525154816.1029642-1-aneesh.kumar@kernel.org>
From: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
TSM vDevice support needs access to the KVM associated with a VFIO device
after the device has been bound to iommufd.
Extend iommufd_device_bind() to accept the device's KVM file and store it
in the iommufd_device. The KVM file reference is owned by VFIO and is
already held for the duration of the device open path.
Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
[nicolinc: fix build error in iommufd_test_mock_domain()]
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
[aneesh.kumar: Switch to use kvm_file]
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
drivers/iommu/iommufd/device.c | 7 ++++++-
drivers/iommu/iommufd/iommufd_private.h | 2 ++
drivers/iommu/iommufd/selftest.c | 2 +-
drivers/vfio/iommufd.c | 3 ++-
include/linux/iommufd.h | 4 +++-
5 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/drivers/iommu/iommufd/device.c b/drivers/iommu/iommufd/device.c
index 170a7005f0bc..718abdc0e627 100644
--- a/drivers/iommu/iommufd/device.c
+++ b/drivers/iommu/iommufd/device.c
@@ -203,6 +203,7 @@ void iommufd_device_destroy(struct iommufd_object *obj)
* iommufd_device_bind - Bind a physical device to an iommu fd
* @ictx: iommufd file descriptor
* @dev: Pointer to a physical device struct
+ * @kvm_file: VM file if device belongs to a KVM VM
* @id: Output ID number to return to userspace for this device
*
* A successful bind establishes an ownership over the device and returns
@@ -216,7 +217,9 @@ void iommufd_device_destroy(struct iommufd_object *obj)
* The caller must undo this with iommufd_device_unbind()
*/
struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx,
- struct device *dev, u32 *id)
+ struct device *dev,
+ struct file *kvm_file,
+ u32 *id)
{
struct iommufd_device *idev;
struct iommufd_group *igroup;
@@ -266,6 +269,8 @@ struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx,
if (!iommufd_selftest_is_mock_dev(dev))
iommufd_ctx_get(ictx);
idev->dev = dev;
+ /* reference is already taken in vfio_df_ioctl_bind_iommufd() */
+ idev->kvm_file = kvm_file;
idev->enforce_cache_coherency =
device_iommu_capable(dev, IOMMU_CAP_ENFORCE_CACHE_COHERENCY);
/* The calling driver is a user until iommufd_device_unbind() */
diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h
index 6ac1965199e9..44eb026c206d 100644
--- a/drivers/iommu/iommufd/iommufd_private.h
+++ b/drivers/iommu/iommufd/iommufd_private.h
@@ -488,6 +488,8 @@ struct iommufd_device {
struct list_head group_item;
/* always the physical device */
struct device *dev;
+ /* ..and the VM file if available */
+ struct file *kvm_file;
bool enforce_cache_coherency;
struct iommufd_vdevice *vdev;
bool destroying;
diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c
index af07c642a526..a193390f9d07 100644
--- a/drivers/iommu/iommufd/selftest.c
+++ b/drivers/iommu/iommufd/selftest.c
@@ -1069,7 +1069,7 @@ static int iommufd_test_mock_domain(struct iommufd_ucmd *ucmd,
goto out_sobj;
}
- idev = iommufd_device_bind(ucmd->ictx, &sobj->idev.mock_dev->dev,
+ idev = iommufd_device_bind(ucmd->ictx, &sobj->idev.mock_dev->dev, NULL,
&idev_id);
if (IS_ERR(idev)) {
rc = PTR_ERR(idev);
diff --git a/drivers/vfio/iommufd.c b/drivers/vfio/iommufd.c
index a38d262c6028..d2d0bd9382a1 100644
--- a/drivers/vfio/iommufd.c
+++ b/drivers/vfio/iommufd.c
@@ -119,7 +119,8 @@ int vfio_iommufd_physical_bind(struct vfio_device *vdev,
{
struct iommufd_device *idev;
- idev = iommufd_device_bind(ictx, vdev->dev, out_device_id);
+ idev = iommufd_device_bind(ictx, vdev->dev, vdev->kvm_file,
+ out_device_id);
if (IS_ERR(idev))
return PTR_ERR(idev);
vdev->iommufd_device = idev;
diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h
index 6e7efe83bc5d..0a0bb4abfbd2 100644
--- a/include/linux/iommufd.h
+++ b/include/linux/iommufd.h
@@ -59,7 +59,9 @@ struct iommufd_object {
};
struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx,
- struct device *dev, u32 *id);
+ struct device *dev,
+ struct file *kvm_file,
+ u32 *id);
void iommufd_device_unbind(struct iommufd_device *idev);
int iommufd_device_attach(struct iommufd_device *idev, ioasid_t pasid,
--
2.43.0
^ permalink raw reply related
* [PATCH v5 3/5] iommufd/viommu: Keep a reference to the KVM file
From: Aneesh Kumar K.V (Arm) @ 2026-05-25 15:48 UTC (permalink / raw)
To: linux-coco, iommu, linux-kernel, kvm
Cc: Aneesh Kumar K.V (Arm), Alexey Kardashevskiy, Bjorn Helgaas,
Dan Williams, Jason Gunthorpe, Joerg Roedel, Jonathan Cameron,
Kevin Tian, Nicolin Chen, Samuel Ortiz, Steven Price,
Suzuki K Poulose, Will Deacon, Xu Yilun, Shameer Kolothum,
Paolo Bonzini, Tony Krowiak, Halil Pasic, Jason Herne,
Harald Freudenberger, Holger Dengler, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Alex Williamson, Matthew Rosato, Farhan Ali,
Eric Farman, linux-s390
In-Reply-To: <20260525154816.1029642-1-aneesh.kumar@kernel.org>
From: Nicolin Chen <nicolinc@nvidia.com>
The TSM vDevice operations need access to the KVM associated with the
device's vIOMMU. Save the device's KVM file in the iommufd_viommu when the
vIOMMU is allocated, and take a file reference so it remains valid for the
lifetime of the vIOMMU.
Release the reference when the vIOMMU is destroyed.
Based on an original patch by Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
[nicolinc: hold kvm's users_count]
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
[aneesh.kumar: Switch to use kvm_file]
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
drivers/iommu/iommufd/viommu.c | 5 +++++
include/linux/iommufd.h | 1 +
2 files changed, 6 insertions(+)
diff --git a/drivers/iommu/iommufd/viommu.c b/drivers/iommu/iommufd/viommu.c
index 4081deda9b33..bf5d58d55939 100644
--- a/drivers/iommu/iommufd/viommu.c
+++ b/drivers/iommu/iommufd/viommu.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES
*/
+#include <linux/file.h>
#include "iommufd_private.h"
void iommufd_viommu_destroy(struct iommufd_object *obj)
@@ -11,6 +12,8 @@ void iommufd_viommu_destroy(struct iommufd_object *obj)
if (viommu->ops && viommu->ops->destroy)
viommu->ops->destroy(viommu);
refcount_dec(&viommu->hwpt->common.obj.users);
+ if (viommu->kvm_file)
+ fput(viommu->kvm_file);
xa_destroy(&viommu->vdevs);
}
@@ -76,6 +79,8 @@ int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd *ucmd)
}
xa_init(&viommu->vdevs);
+ if (idev->kvm_file)
+ viommu->kvm_file = get_file(idev->kvm_file);
viommu->type = cmd->type;
viommu->ictx = ucmd->ictx;
viommu->hwpt = hwpt_paging;
diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h
index 0a0bb4abfbd2..3267717f676d 100644
--- a/include/linux/iommufd.h
+++ b/include/linux/iommufd.h
@@ -103,6 +103,7 @@ struct iommufd_viommu {
struct iommufd_ctx *ictx;
struct iommu_device *iommu_dev;
struct iommufd_hwpt_paging *hwpt;
+ struct file *kvm_file;
const struct iommufd_viommu_ops *ops;
--
2.43.0
^ permalink raw reply related
* [PATCH v5 4/5] iommufd/tsm: add vdevice TSM bind/unbind ioctl
From: Aneesh Kumar K.V (Arm) @ 2026-05-25 15:48 UTC (permalink / raw)
To: linux-coco, iommu, linux-kernel, kvm
Cc: Aneesh Kumar K.V (Arm), Alexey Kardashevskiy, Bjorn Helgaas,
Dan Williams, Jason Gunthorpe, Joerg Roedel, Jonathan Cameron,
Kevin Tian, Nicolin Chen, Samuel Ortiz, Steven Price,
Suzuki K Poulose, Will Deacon, Xu Yilun, Shameer Kolothum,
Paolo Bonzini, Tony Krowiak, Halil Pasic, Jason Herne,
Harald Freudenberger, Holger Dengler, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Alex Williamson, Matthew Rosato, Farhan Ali,
Eric Farman, linux-s390
In-Reply-To: <20260525154816.1029642-1-aneesh.kumar@kernel.org>
Introduce IOMMU_VDEVICE_TSM_OP to allow userspace to issue TSM bind/unbind
operations for an iommufd vdevice.
The new ioctl:
- looks up the vdevice object from vdevice_id
- resolves the associated KVM VM from the vIOMMU KVM file reference
- dispatches bind/unbind via tsm_bind()/tsm_unbind()
Also add common TSM helpers in tsm-core and wire vdevice teardown to unbind
the device from TSM state.
This provides iommufd plumbing to bind a TDI to a confidential guest through
the TSM layer.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
drivers/iommu/iommufd/Makefile | 2 +
drivers/iommu/iommufd/iommufd_private.h | 8 ++++
drivers/iommu/iommufd/main.c | 3 ++
drivers/iommu/iommufd/tsm.c | 62 +++++++++++++++++++++++++
drivers/iommu/iommufd/viommu.c | 4 ++
drivers/virt/coco/tsm-core.c | 19 ++++++++
include/linux/tsm.h | 17 +++++++
include/uapi/linux/iommufd.h | 26 +++++++++++
8 files changed, 141 insertions(+)
create mode 100644 drivers/iommu/iommufd/tsm.c
diff --git a/drivers/iommu/iommufd/Makefile b/drivers/iommu/iommufd/Makefile
index 71d692c9a8f4..431089089ee9 100644
--- a/drivers/iommu/iommufd/Makefile
+++ b/drivers/iommu/iommufd/Makefile
@@ -10,6 +10,8 @@ iommufd-y := \
vfio_compat.o \
viommu.o
+iommufd-$(CONFIG_TSM) += tsm.o
+
iommufd-$(CONFIG_IOMMUFD_TEST) += selftest.o
obj-$(CONFIG_IOMMUFD) += iommufd.o
diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h
index 44eb026c206d..8eea0c2c332b 100644
--- a/drivers/iommu/iommufd/iommufd_private.h
+++ b/drivers/iommu/iommufd/iommufd_private.h
@@ -699,6 +699,14 @@ void iommufd_vdevice_destroy(struct iommufd_object *obj);
void iommufd_vdevice_abort(struct iommufd_object *obj);
int iommufd_hw_queue_alloc_ioctl(struct iommufd_ucmd *ucmd);
void iommufd_hw_queue_destroy(struct iommufd_object *obj);
+#ifdef CONFIG_TSM
+int iommufd_vdevice_tsm_op_ioctl(struct iommufd_ucmd *ucmd);
+#else
+static inline int iommufd_vdevice_tsm_op_ioctl(struct iommufd_ucmd *ucmd)
+{
+ return -EOPNOTSUPP;
+}
+#endif
static inline struct iommufd_vdevice *
iommufd_get_vdevice(struct iommufd_ctx *ictx, u32 id)
diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c
index 8c6d43601afb..d73e6b391c6f 100644
--- a/drivers/iommu/iommufd/main.c
+++ b/drivers/iommu/iommufd/main.c
@@ -432,6 +432,7 @@ union ucmd_buffer {
struct iommu_veventq_alloc veventq;
struct iommu_vfio_ioas vfio_ioas;
struct iommu_viommu_alloc viommu;
+ struct iommu_vdevice_tsm_op tsm_op;
#ifdef CONFIG_IOMMUFD_TEST
struct iommu_test_cmd test;
#endif
@@ -493,6 +494,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[] = {
__reserved),
IOCTL_OP(IOMMU_VIOMMU_ALLOC, iommufd_viommu_alloc_ioctl,
struct iommu_viommu_alloc, out_viommu_id),
+ IOCTL_OP(IOMMU_VDEVICE_TSM_OP, iommufd_vdevice_tsm_op_ioctl,
+ struct iommu_vdevice_tsm_op, vdevice_id),
#ifdef CONFIG_IOMMUFD_TEST
IOCTL_OP(IOMMU_TEST_CMD, iommufd_test, struct iommu_test_cmd, last),
#endif
diff --git a/drivers/iommu/iommufd/tsm.c b/drivers/iommu/iommufd/tsm.c
new file mode 100644
index 000000000000..09ee668dbed9
--- /dev/null
+++ b/drivers/iommu/iommufd/tsm.c
@@ -0,0 +1,62 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2026 ARM Ltd.
+ */
+
+#include <linux/tsm.h>
+#include "iommufd_private.h"
+
+/**
+ * iommufd_vdevice_tsm_op_ioctl - Handle vdevice TSM operations
+ * @ucmd: user command data for IOMMU_VDEVICE_TSM_OP
+ *
+ * Currently only supports TSM bind/unbind operations
+ * Resolve @iommu_vdevice_tsm_op::vdevice_id to a vdevice and dispatch the
+ * requested bind/unbind operation through the TSM core.
+ *
+ * Return: 0 on success, or a negative error code on failure.
+ */
+int iommufd_vdevice_tsm_op_ioctl(struct iommufd_ucmd *ucmd)
+{
+ int rc;
+ struct kvm *kvm = NULL;
+ struct iommufd_vdevice *vdev;
+ struct iommu_vdevice_tsm_op *cmd = ucmd->cmd;
+
+ if (cmd->flags)
+ return -EOPNOTSUPP;
+
+ vdev = iommufd_get_vdevice(ucmd->ictx, cmd->vdevice_id);
+ if (IS_ERR(vdev))
+ return PTR_ERR(vdev);
+
+ if (vdev->viommu->kvm_file)
+ kvm = vdev->viommu->kvm_file->private_data;
+
+ if (!kvm) {
+ rc = -ENODEV;
+ goto out_put_vdev;
+ }
+
+ /* tsm layer will take care of parallel calls to tsm_bind/unbind */
+ switch (cmd->type) {
+ case IOMMU_VDEVICE_TSM_BIND:
+ rc = tsm_bind(vdev->idev->dev, kvm, vdev->virt_id);
+ break;
+ case IOMMU_VDEVICE_TSM_UNBIND:
+ rc = tsm_unbind(vdev->idev->dev);
+ break;
+ default:
+ rc = -EINVAL;
+ goto out_put_vdev;
+ }
+
+ if (rc)
+ goto out_put_vdev;
+
+ rc = iommufd_ucmd_respond(ucmd, sizeof(*cmd));
+
+out_put_vdev:
+ iommufd_put_object(ucmd->ictx, &vdev->obj);
+ return rc;
+}
diff --git a/drivers/iommu/iommufd/viommu.c b/drivers/iommu/iommufd/viommu.c
index bf5d58d55939..1b9379fcba84 100644
--- a/drivers/iommu/iommufd/viommu.c
+++ b/drivers/iommu/iommufd/viommu.c
@@ -3,6 +3,8 @@
*/
#include <linux/file.h>
#include "iommufd_private.h"
+#include <linux/cleanup.h>
+#include <linux/tsm.h>
void iommufd_viommu_destroy(struct iommufd_object *obj)
{
@@ -124,6 +126,8 @@ void iommufd_vdevice_abort(struct iommufd_object *obj)
lockdep_assert_held(&idev->igroup->lock);
+ tsm_unbind(idev->dev);
+
if (vdev->destroy)
vdev->destroy(vdev);
/* xa_cmpxchg is okay to fail if alloc failed xa_cmpxchg previously */
diff --git a/drivers/virt/coco/tsm-core.c b/drivers/virt/coco/tsm-core.c
index e784993353d8..3870d08ffe0d 100644
--- a/drivers/virt/coco/tsm-core.c
+++ b/drivers/virt/coco/tsm-core.c
@@ -108,6 +108,25 @@ void tsm_unregister(struct tsm_dev *tsm_dev)
}
EXPORT_SYMBOL_GPL(tsm_unregister);
+int tsm_bind(struct device *dev, struct kvm *kvm, u64 tdi_id)
+{
+ if (!dev_is_pci(dev))
+ return -EINVAL;
+
+ return pci_tsm_bind(to_pci_dev(dev), kvm, tdi_id);
+}
+EXPORT_SYMBOL_GPL(tsm_bind);
+
+int tsm_unbind(struct device *dev)
+{
+ if (!dev_is_pci(dev))
+ return -EINVAL;
+
+ pci_tsm_unbind(to_pci_dev(dev));
+ return 0;
+}
+EXPORT_SYMBOL_GPL(tsm_unbind);
+
static void tsm_release(struct device *dev)
{
struct tsm_dev *tsm_dev = container_of(dev, typeof(*tsm_dev), dev);
diff --git a/include/linux/tsm.h b/include/linux/tsm.h
index 381c53244c83..7b6df827321b 100644
--- a/include/linux/tsm.h
+++ b/include/linux/tsm.h
@@ -123,4 +123,21 @@ int tsm_report_unregister(const struct tsm_report_ops *ops);
struct tsm_dev *tsm_register(struct device *parent, struct pci_tsm_ops *ops);
void tsm_unregister(struct tsm_dev *tsm_dev);
struct tsm_dev *find_tsm_dev(int id);
+
+struct kvm;
+#ifdef CONFIG_TSM
+int tsm_bind(struct device *dev, struct kvm *kvm, u64 tdi_id);
+int tsm_unbind(struct device *dev);
+#else
+static inline int tsm_bind(struct device *dev, struct kvm *kvm, u64 tdi_id)
+{
+ return -EINVAL;
+}
+
+static inline int tsm_unbind(struct device *dev)
+{
+ return 0;
+}
+#endif
+
#endif /* __TSM_H */
diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h
index e998dfbd6960..66398efa31d1 100644
--- a/include/uapi/linux/iommufd.h
+++ b/include/uapi/linux/iommufd.h
@@ -57,6 +57,7 @@ enum {
IOMMUFD_CMD_IOAS_CHANGE_PROCESS = 0x92,
IOMMUFD_CMD_VEVENTQ_ALLOC = 0x93,
IOMMUFD_CMD_HW_QUEUE_ALLOC = 0x94,
+ IOMMUFD_CMD_VDEVICE_TSM_OP = 0x95,
};
/**
@@ -1143,6 +1144,31 @@ struct iommu_vdevice_alloc {
};
#define IOMMU_VDEVICE_ALLOC _IO(IOMMUFD_TYPE, IOMMUFD_CMD_VDEVICE_ALLOC)
+/**
+ * enum iommu_vdevice_tsm_op_type - operation type for struct iommu_vdevice_tsm_op
+ * @IOMMU_VDEVICE_TSM_BIND: Bind a vDevice to TSM
+ * @IOMMU_VDEVICE_TSM_UNBIND: Unbind a vDevice from TSM
+ */
+enum iommu_vdevice_tsm_op_type {
+ IOMMU_VDEVICE_TSM_BIND = 0x1,
+ IOMMU_VDEVICE_TSM_UNBIND,
+};
+
+/**
+ * struct iommu_vdevice_tsm_op - ioctl(IOMMU_VDEVICE_TSM_OP)
+ * @size: sizeof(struct iommu_vdevice_tsm_op)
+ * @type: Type of TSM operation. Must be defined in enum iommu_vdevice_tsm_op_type
+ * @flags: Must be 0
+ * @vdevice_id: Object handle for the vDevice. Returned from IOMMU_VDEVICE_ALLOC
+ */
+struct iommu_vdevice_tsm_op {
+ __u32 size;
+ __u32 type;
+ __u32 flags;
+ __u32 vdevice_id;
+};
+#define IOMMU_VDEVICE_TSM_OP _IO(IOMMUFD_TYPE, IOMMUFD_CMD_VDEVICE_TSM_OP)
+
/**
* struct iommu_ioas_change_process - ioctl(VFIO_IOAS_CHANGE_PROCESS)
* @size: sizeof(struct iommu_ioas_change_process)
--
2.43.0
^ permalink raw reply related
* [PATCH v5 5/5] iommufd/vdevice: add TSM request ioctl
From: Aneesh Kumar K.V (Arm) @ 2026-05-25 15:48 UTC (permalink / raw)
To: linux-coco, iommu, linux-kernel, kvm
Cc: Aneesh Kumar K.V (Arm), Alexey Kardashevskiy, Bjorn Helgaas,
Dan Williams, Jason Gunthorpe, Joerg Roedel, Jonathan Cameron,
Kevin Tian, Nicolin Chen, Samuel Ortiz, Steven Price,
Suzuki K Poulose, Will Deacon, Xu Yilun, Shameer Kolothum,
Paolo Bonzini, Tony Krowiak, Halil Pasic, Jason Herne,
Harald Freudenberger, Holger Dengler, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Alex Williamson, Matthew Rosato, Farhan Ali,
Eric Farman, linux-s390
In-Reply-To: <20260525154816.1029642-1-aneesh.kumar@kernel.org>
Add IOMMU_VDEVICE_TSM_REQUEST for issuing TSM guest request/response
transactions against an iommufd vdevice.
The ioctl takes a vdevice_id plus request/response user buffers and length
fields, and forwards the request through tsm_guest_req() to the PCI TSM
backend. This provides the host-side passthrough path used by CoCo guests
for TSM device attestation and acceptance flows after the device has been
bound to TSM.
Also add the supporting tsm_guest_req() helper and associated TSM core
interface definitions.
Based on changes from: Alexey Kardashevskiy <aik@amd.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
drivers/iommu/iommufd/iommufd_private.h | 6 ++
drivers/iommu/iommufd/main.c | 3 +
drivers/iommu/iommufd/tsm.c | 68 +++++++++++++++++++++
drivers/virt/coco/tsm-core.c | 39 ++++++++++++
include/linux/pci-tsm.h | 9 +--
include/linux/tsm.h | 25 ++++++++
include/uapi/linux/iommufd.h | 80 +++++++++++++++++++++++++
7 files changed, 226 insertions(+), 4 deletions(-)
diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h
index 8eea0c2c332b..0080895e9e92 100644
--- a/drivers/iommu/iommufd/iommufd_private.h
+++ b/drivers/iommu/iommufd/iommufd_private.h
@@ -701,11 +701,17 @@ int iommufd_hw_queue_alloc_ioctl(struct iommufd_ucmd *ucmd);
void iommufd_hw_queue_destroy(struct iommufd_object *obj);
#ifdef CONFIG_TSM
int iommufd_vdevice_tsm_op_ioctl(struct iommufd_ucmd *ucmd);
+int iommufd_vdevice_tsm_req_ioctl(struct iommufd_ucmd *ucmd);
#else
static inline int iommufd_vdevice_tsm_op_ioctl(struct iommufd_ucmd *ucmd)
{
return -EOPNOTSUPP;
}
+
+static inline int iommufd_vdevice_tsm_req_ioctl(struct iommufd_ucmd *ucmd)
+{
+ return -EOPNOTSUPP;
+}
#endif
static inline struct iommufd_vdevice *
diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c
index d73e6b391c6f..5f49b546ec92 100644
--- a/drivers/iommu/iommufd/main.c
+++ b/drivers/iommu/iommufd/main.c
@@ -433,6 +433,7 @@ union ucmd_buffer {
struct iommu_vfio_ioas vfio_ioas;
struct iommu_viommu_alloc viommu;
struct iommu_vdevice_tsm_op tsm_op;
+ struct iommu_vdevice_tsm_req tsm_req;
#ifdef CONFIG_IOMMUFD_TEST
struct iommu_test_cmd test;
#endif
@@ -496,6 +497,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[] = {
struct iommu_viommu_alloc, out_viommu_id),
IOCTL_OP(IOMMU_VDEVICE_TSM_OP, iommufd_vdevice_tsm_op_ioctl,
struct iommu_vdevice_tsm_op, vdevice_id),
+ IOCTL_OP(IOMMU_VDEVICE_TSM_REQ, iommufd_vdevice_tsm_req_ioctl,
+ struct iommu_vdevice_tsm_req, resp_uptr),
#ifdef CONFIG_IOMMUFD_TEST
IOCTL_OP(IOMMU_TEST_CMD, iommufd_test, struct iommu_test_cmd, last),
#endif
diff --git a/drivers/iommu/iommufd/tsm.c b/drivers/iommu/iommufd/tsm.c
index 09ee668dbed9..342fbdb6a6b9 100644
--- a/drivers/iommu/iommufd/tsm.c
+++ b/drivers/iommu/iommufd/tsm.c
@@ -60,3 +60,71 @@ int iommufd_vdevice_tsm_op_ioctl(struct iommufd_ucmd *ucmd)
iommufd_put_object(ucmd->ictx, &vdev->obj);
return rc;
}
+
+static bool iommufd_vdevice_tsm_req_scope_valid(u32 scope)
+{
+ if (scope > IOMMU_VDEVICE_TSM_REQ_SCOPE_PCI_LAST)
+ return false;
+
+ switch (scope) {
+ case IOMMU_VDEVICE_TSM_REQ_PCI_INFO:
+ case IOMMU_VDEVICE_TSM_REQ_PCI_STATE_CHANGE:
+ case IOMMU_VDEVICE_TSM_REQ_PCI_DEBUG_READ:
+ case IOMMU_VDEVICE_TSM_REQ_PCI_DEBUG_WRITE:
+ return true;
+ default:
+ return false;
+ }
+}
+
+/**
+ * iommufd_vdevice_tsm_req_ioctl - Forward TSM requests
+ * @ucmd: user command data for IOMMU_VDEVICE_TSM_REQ
+ *
+ * Resolve @iommu_vdevice_tsm_req::vdevice_id to a vdevice and pass the
+ * request/response buffers to the TSM core.
+ *
+ * Return:
+ * -errno on error.
+ * positive residue if response/request bytes were left unconsumed.
+ * if response buffer is provided, residue indicates the number of bytes
+ * not used in response buffer
+ * if there is no response buffer, residue indicates the number of bytes
+ * not consumed in req buffer
+ * 0 otherwise.
+ */
+int iommufd_vdevice_tsm_req_ioctl(struct iommufd_ucmd *ucmd)
+{
+ int rc;
+ struct iommufd_vdevice *vdev;
+ struct iommu_vdevice_tsm_req *cmd = ucmd->cmd;
+ struct tsm_guest_req_info info = {
+ .scope = cmd->scope,
+ .req = {
+ .user = u64_to_user_ptr(cmd->req_uptr),
+ .is_kernel = false,
+ },
+ .req_len = cmd->req_len,
+ .resp = {
+ .user = u64_to_user_ptr(cmd->resp_uptr),
+ .is_kernel = false,
+ },
+ .resp_len = cmd->resp_len,
+ };
+
+ if (cmd->__reserved)
+ return -EOPNOTSUPP;
+
+ if (!iommufd_vdevice_tsm_req_scope_valid(cmd->scope))
+ return -EINVAL;
+
+ vdev = iommufd_get_vdevice(ucmd->ictx, cmd->vdevice_id);
+ if (IS_ERR(vdev))
+ return PTR_ERR(vdev);
+
+ rc = tsm_guest_req(vdev->idev->dev, &info);
+
+ /* No inline response, hence we don't need to copy the response */
+ iommufd_put_object(ucmd->ictx, &vdev->obj);
+ return rc;
+}
diff --git a/drivers/virt/coco/tsm-core.c b/drivers/virt/coco/tsm-core.c
index 3870d08ffe0d..c24886851f9e 100644
--- a/drivers/virt/coco/tsm-core.c
+++ b/drivers/virt/coco/tsm-core.c
@@ -8,6 +8,7 @@
#include <linux/module.h>
#include <linux/cleanup.h>
#include <linux/pci-tsm.h>
+#include <uapi/linux/iommufd.h>
static void tsm_release(struct device *);
static const struct class tsm_class = {
@@ -127,6 +128,44 @@ int tsm_unbind(struct device *dev)
}
EXPORT_SYMBOL_GPL(tsm_unbind);
+static int tsm_pci_req_scope(u32 scope, enum pci_tsm_req_scope *pci_scope)
+{
+ switch (scope) {
+ case IOMMU_VDEVICE_TSM_REQ_PCI_INFO:
+ *pci_scope = PCI_TSM_REQ_INFO;
+ return 0;
+ case IOMMU_VDEVICE_TSM_REQ_PCI_STATE_CHANGE:
+ *pci_scope = PCI_TSM_REQ_STATE_CHANGE;
+ return 0;
+ case IOMMU_VDEVICE_TSM_REQ_PCI_DEBUG_READ:
+ *pci_scope = PCI_TSM_REQ_DEBUG_READ;
+ return 0;
+ case IOMMU_VDEVICE_TSM_REQ_PCI_DEBUG_WRITE:
+ *pci_scope = PCI_TSM_REQ_DEBUG_WRITE;
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
+ssize_t tsm_guest_req(struct device *dev, struct tsm_guest_req_info *info)
+{
+ int ret;
+ enum pci_tsm_req_scope pci_scope;
+
+ if (!dev_is_pci(dev))
+ return -EINVAL;
+
+ ret = tsm_pci_req_scope(info->scope, &pci_scope);
+ if (ret)
+ return ret;
+
+ return pci_tsm_guest_req(to_pci_dev(dev), pci_scope, info->req,
+ info->req_len, info->resp, info->resp_len,
+ NULL);
+}
+EXPORT_SYMBOL_GPL(tsm_guest_req);
+
static void tsm_release(struct device *dev)
{
struct tsm_dev *tsm_dev = container_of(dev, typeof(*tsm_dev), dev);
diff --git a/include/linux/pci-tsm.h b/include/linux/pci-tsm.h
index a6435aba03f9..ec2236a7a279 100644
--- a/include/linux/pci-tsm.h
+++ b/include/linux/pci-tsm.h
@@ -4,6 +4,7 @@
#include <linux/mutex.h>
#include <linux/pci.h>
#include <linux/sockptr.h>
+#include <uapi/linux/iommufd.h>
struct pci_tsm;
struct tsm_dev;
@@ -173,7 +174,7 @@ enum pci_tsm_req_scope {
* typical TDISP collateral information like Device Interface Reports.
* No device secrets are permitted, and no device state is changed.
*/
- PCI_TSM_REQ_INFO = 0,
+ PCI_TSM_REQ_INFO = IOMMU_VDEVICE_TSM_REQ_PCI_INFO,
/**
* @PCI_TSM_REQ_STATE_CHANGE: Request to change the TDISP state from
* UNLOCKED->LOCKED, LOCKED->RUN, or other architecture specific state
@@ -181,14 +182,14 @@ enum pci_tsm_req_scope {
* to TDISP) device / host state, configuration, or data change is
* permitted.
*/
- PCI_TSM_REQ_STATE_CHANGE = 1,
+ PCI_TSM_REQ_STATE_CHANGE = IOMMU_VDEVICE_TSM_REQ_PCI_STATE_CHANGE,
/**
* @PCI_TSM_REQ_DEBUG_READ: Read-only request for debug information
*
* A method to facilitate TVM information retrieval outside of typical
* TDISP operational requirements. No device secrets are permitted.
*/
- PCI_TSM_REQ_DEBUG_READ = 2,
+ PCI_TSM_REQ_DEBUG_READ = IOMMU_VDEVICE_TSM_REQ_PCI_DEBUG_READ,
/**
* @PCI_TSM_REQ_DEBUG_WRITE: Device state changes for debug purposes
*
@@ -196,7 +197,7 @@ enum pci_tsm_req_scope {
* the TDISP operational model. If allowed, requires CAP_SYS_RAW_IO, and
* will taint the kernel.
*/
- PCI_TSM_REQ_DEBUG_WRITE = 3,
+ PCI_TSM_REQ_DEBUG_WRITE = IOMMU_VDEVICE_TSM_REQ_PCI_DEBUG_WRITE,
};
#ifdef CONFIG_PCI_TSM
diff --git a/include/linux/tsm.h b/include/linux/tsm.h
index 7b6df827321b..6101a2a1db61 100644
--- a/include/linux/tsm.h
+++ b/include/linux/tsm.h
@@ -6,6 +6,7 @@
#include <linux/types.h>
#include <linux/uuid.h>
#include <linux/device.h>
+#include <linux/sockptr.h>
#define TSM_REPORT_INBLOB_MAX 64
#define TSM_REPORT_OUTBLOB_MAX SZ_16M
@@ -128,6 +129,23 @@ struct kvm;
#ifdef CONFIG_TSM
int tsm_bind(struct device *dev, struct kvm *kvm, u64 tdi_id);
int tsm_unbind(struct device *dev);
+
+/**
+ * struct tsm_guest_req_info - parameter for tsm_guest_req()
+ * @scope: iommufd allocated scope for tsm guest request
+ * @req: request data buffer filled by guest
+ * @req_len: the size of @req filled by guest
+ * @resp: response data buffer filled by host
+ * @resp_len: the size of @resp buffer filled by guest
+ */
+struct tsm_guest_req_info {
+ u32 scope;
+ sockptr_t req;
+ size_t req_len;
+ sockptr_t resp;
+ size_t resp_len;
+};
+ssize_t tsm_guest_req(struct device *dev, struct tsm_guest_req_info *info);
#else
static inline int tsm_bind(struct device *dev, struct kvm *kvm, u64 tdi_id)
{
@@ -138,6 +156,13 @@ static inline int tsm_unbind(struct device *dev)
{
return 0;
}
+
+struct tsm_guest_req_info;
+static inline ssize_t tsm_guest_req(struct device *dev,
+ struct tsm_guest_req_info *info)
+{
+ return -EINVAL;
+}
#endif
#endif /* __TSM_H */
diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h
index 66398efa31d1..7953e99a9671 100644
--- a/include/uapi/linux/iommufd.h
+++ b/include/uapi/linux/iommufd.h
@@ -58,6 +58,7 @@ enum {
IOMMUFD_CMD_VEVENTQ_ALLOC = 0x93,
IOMMUFD_CMD_HW_QUEUE_ALLOC = 0x94,
IOMMUFD_CMD_VDEVICE_TSM_OP = 0x95,
+ IOMMUFD_CMD_VDEVICE_TSM_REQ = 0x96,
};
/**
@@ -1373,4 +1374,83 @@ struct iommu_hw_queue_alloc {
__aligned_u64 length;
};
#define IOMMU_HW_QUEUE_ALLOC _IO(IOMMUFD_TYPE, IOMMUFD_CMD_HW_QUEUE_ALLOC)
+
+/*
+ * TSM request scope values are allocated by iommufd. Each device-bus transport
+ * gets a range from this number space.
+ */
+#define IOMMU_VDEVICE_TSM_REQ_SCOPE_PCI_BASE 0
+
+enum iommu_vdevice_tsm_req_scope {
+ /*
+ * Read-only, without side effects, request for typical TDISP
+ * collateral information like Device Interface Reports. No device
+ * secrets are permitted, and no device state is changed.
+ */
+ IOMMU_VDEVICE_TSM_REQ_PCI_INFO =
+ IOMMU_VDEVICE_TSM_REQ_SCOPE_PCI_BASE,
+ /*
+ * Request to change the TDISP state from UNLOCKED->LOCKED,
+ * LOCKED->RUN, or other architecture specific state changes to
+ * support those transitions for a TDI. No other device or host state,
+ * configuration, or data change is permitted.
+ */
+ IOMMU_VDEVICE_TSM_REQ_PCI_STATE_CHANGE =
+ IOMMU_VDEVICE_TSM_REQ_SCOPE_PCI_BASE + 1,
+ /*
+ * Read-only request for debug information outside of typical TDISP
+ * operational requirements. No device secrets are permitted.
+ */
+ IOMMU_VDEVICE_TSM_REQ_PCI_DEBUG_READ =
+ IOMMU_VDEVICE_TSM_REQ_SCOPE_PCI_BASE + 2,
+ /*
+ * Device state changes for debug purposes. The request may affect the
+ * operational state of the device outside of the TDISP operational
+ * model. If allowed, this requires CAP_SYS_RAW_IO and taints the
+ * kernel.
+ */
+ IOMMU_VDEVICE_TSM_REQ_PCI_DEBUG_WRITE =
+ IOMMU_VDEVICE_TSM_REQ_SCOPE_PCI_BASE + 3,
+ IOMMU_VDEVICE_TSM_REQ_SCOPE_PCI_LAST =
+ IOMMU_VDEVICE_TSM_REQ_PCI_DEBUG_WRITE,
+};
+
+/**
+ * struct iommu_vdevice_tsm_req - ioctl(IOMMU_VDEVICE_TSM_REQ)
+ * @size: sizeof(struct iommu_vdevice_tsm_req)
+ * @vdevice_id: vDevice ID the guest request is for
+ * @scope: One of enum iommu_vdevice_tsm_req_scope
+ * @req_len: Size in bytes of the input payload at @req_uptr
+ * @resp_len: Size in bytes of the output buffer at @resp_uptr
+ * @__reserved: Must be 0
+ * @req_uptr: Userspace pointer to the guest-provided request payload
+ * @resp_uptr: Userspace pointer to the guest response buffer
+ *
+ * Forward a TSM request to the TSM bound vDevice. This is intended for
+ * guest TSM/TDISP message transport where the host kernel only marshals
+ * bytes between userspace and the TSM implementation.
+ *
+ * Requests outside the iommufd allocated scope values are rejected. Lower
+ * layers may reject scope values that are valid in the global iommufd
+ * namespace, but not permitted for a specific bus.
+ *
+ * The request payload is read from @req_uptr/@req_len. If a response is
+ * expected, userspace provides @resp_uptr/@resp_len as writable storage for
+ * response bytes returned by the TSM path.
+ *
+ * The ioctl is only suitable for commands and results that the host kernel
+ * has no use, the host is only facilitating guest to TSM communication.
+ */
+struct iommu_vdevice_tsm_req {
+ __u32 size;
+ __u32 vdevice_id;
+ __u32 scope;
+ __u32 req_len;
+ __u32 resp_len;
+ __u32 __reserved;
+ __aligned_u64 req_uptr;
+ __aligned_u64 resp_uptr;
+};
+
+#define IOMMU_VDEVICE_TSM_REQ _IO(IOMMUFD_TYPE, IOMMUFD_CMD_VDEVICE_TSM_REQ)
#endif
--
2.43.0
^ permalink raw reply related
* [PATCH v6 02/11] x86/virt/tdx: Allocate page bitmap for Dynamic PAMT
From: Rick Edgecombe @ 2026-05-26 2:35 UTC (permalink / raw)
To: bp, dave.hansen, hpa, kas, kvm, linux-coco, linux-doc,
linux-kernel, mingo, nik.borisov, pbonzini, seanjc, tglx,
vannapurve, x86, chao.gao, yan.y.zhao, kai.huang
Cc: rick.p.edgecombe, Kirill A. Shutemov, Binbin Wu
In-Reply-To: <20260526023515.288829-1-rick.p.edgecombe@intel.com>
From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
The TDX Physical Address Metadata Table (PAMT) holds data about the
physical memory used by TDX, and must be allocated by the kernel during
TDX module initialization.
The exact size of the required PAMT memory is determined by the TDX module
and may vary between TDX module versions. Currently it is approximately
0.4% of the system memory. This is a significant commitment, especially if
it is not known upfront whether the machine will run any TDX guests.
Each memory region that the TDX module might use needs three separate PAMT
allocations. One for each supported page size (1GB, 2MB, 4KB). The
TDX module supports a new feature designed to reduce PAMT overhead called
Dynamic PAMT. At a high level, Dynamic PAMT still has the 1GB and 2MB
levels allocated on TDX module initialization, but the 4KB level is
allocated dynamically during runtime.
However, in the details, Dynamic PAMT still needs some smaller per 4KB
page scoped data (currently it is 1 bit per page). The TDX module exposes
the number of bits as a separate piece of metadata than the 4KB static
allocation for regular PAMT. Although the size is enumerated differently,
it is handed to the TDX module in the same way the 4KB page size PAMT
allocation is for regular, non-dynamic PAMT.
Begin to implement Dynamic PAMT in the kernel by reading the bits-per-page
needed for Dynamic PAMT. Calculate the size needed for the bitmap,
and use it instead of the 4KB size determined for normal PAMT, in the case
of Dynamic PAMT.
Unlike the existing metadata reading code, this code is not generated by a
script. So adjust the comment to be more generic. Also, start to adopt a
more normal kernel code style without the tenary statements and if
conditionals assignments that the auto generated code has.
Assisted-by: Sashiko:claude-opus-4-6
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Co-developed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
---
v6:
- Improve comment (Binbin)
- Log tweaks
- Mark tdmr_get_pamt_bitmap_sz() __init in response to upstream
changes
- Switch to more normal kernel code style, even though it differs from
the existing auto generated code.
---
arch/x86/include/asm/tdx.h | 5 +++++
arch/x86/include/asm/tdx_global_metadata.h | 3 +++
arch/x86/virt/vmx/tdx/tdx.c | 19 ++++++++++++++++++-
arch/x86/virt/vmx/tdx/tdx_global_metadata.c | 21 ++++++++++++++++++++-
4 files changed, 46 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index 503f9a3f46d61..82dc27aecf297 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -149,6 +149,11 @@ static __always_inline u64 sc_retry(sc_func_t func, u64 fn,
const char *tdx_dump_mce_info(struct mce *m);
const struct tdx_sys_info *tdx_get_sysinfo(void);
+static inline bool tdx_supports_dynamic_pamt(const struct tdx_sys_info *sysinfo)
+{
+ return false; /* To be enabled when kernel is ready */
+}
+
int tdx_guest_keyid_alloc(void);
u32 tdx_get_nr_guest_keyids(void);
void tdx_guest_keyid_free(unsigned int keyid);
diff --git a/arch/x86/include/asm/tdx_global_metadata.h b/arch/x86/include/asm/tdx_global_metadata.h
index 40689c8dc67eb..88040ddb51af4 100644
--- a/arch/x86/include/asm/tdx_global_metadata.h
+++ b/arch/x86/include/asm/tdx_global_metadata.h
@@ -21,6 +21,9 @@ struct tdx_sys_info_tdmr {
u16 pamt_4k_entry_size;
u16 pamt_2m_entry_size;
u16 pamt_1g_entry_size;
+
+ /* Optional metadata, if Dynamic PAMT is supported */
+ u8 pamt_page_bitmap_entry_bits;
};
struct tdx_sys_info_td_ctrl {
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 487f389f52f4b..9ebd192cb5c17 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -512,6 +512,18 @@ static __init int fill_out_tdmrs(struct list_head *tmb_list,
return 0;
}
+static __init unsigned long tdmr_get_pamt_bitmap_sz(struct tdmr_info *tdmr)
+{
+ unsigned long pamt_sz, nr_pamt_entries;
+ int bits_per_entry;
+
+ bits_per_entry = tdx_sysinfo.tdmr.pamt_page_bitmap_entry_bits;
+ nr_pamt_entries = tdmr->size >> PAGE_SHIFT;
+ pamt_sz = DIV_ROUND_UP(nr_pamt_entries * bits_per_entry, BITS_PER_BYTE);
+
+ return PAGE_ALIGN(pamt_sz);
+}
+
/*
* Calculate PAMT size given a TDMR and a page size. The returned
* PAMT size is always aligned up to 4K page boundary.
@@ -579,7 +591,12 @@ static __init int tdmr_set_up_pamt(struct tdmr_info *tdmr,
* Calculate the PAMT size for each TDX supported page size
* and the total PAMT size.
*/
- tdmr->pamt_4k_size = tdmr_get_pamt_sz(tdmr, TDX_PS_4K);
+ if (tdx_supports_dynamic_pamt(&tdx_sysinfo)) {
+ /* With Dynamic PAMT, PAMT_4K is replaced with a bitmap */
+ tdmr->pamt_4k_size = tdmr_get_pamt_bitmap_sz(tdmr);
+ } else {
+ tdmr->pamt_4k_size = tdmr_get_pamt_sz(tdmr, TDX_PS_4K);
+ }
tdmr->pamt_2m_size = tdmr_get_pamt_sz(tdmr, TDX_PS_2M);
tdmr->pamt_1g_size = tdmr_get_pamt_sz(tdmr, TDX_PS_1G);
tdmr_pamt_size = tdmr->pamt_4k_size + tdmr->pamt_2m_size + tdmr->pamt_1g_size;
diff --git a/arch/x86/virt/vmx/tdx/tdx_global_metadata.c b/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
index c7db393a9cfb1..7e8e913463be1 100644
--- a/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
+++ b/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0
/*
- * Automatically generated functions to read TDX global metadata.
+ * Functions to read TDX global metadata.
*
* This file doesn't compile on its own as it lacks of inclusion
* of SEAMCALL wrapper primitive which reads global metadata.
@@ -33,6 +33,18 @@ static __init int get_tdx_sys_info_features(struct tdx_sys_info_features *sysinf
return ret;
}
+static __init int get_tdx_sys_info_tdmr_dpamt(struct tdx_sys_info_tdmr *sysinfo_tdmr)
+{
+ int ret;
+ u64 val;
+
+ ret = read_sys_metadata_field(0x9100000100000013, &val);
+ if (!ret)
+ sysinfo_tdmr->pamt_page_bitmap_entry_bits = val;
+
+ return ret;
+}
+
static __init int get_tdx_sys_info_tdmr(struct tdx_sys_info_tdmr *sysinfo_tdmr)
{
int ret = 0;
@@ -116,5 +128,12 @@ static __init int get_tdx_sys_info(struct tdx_sys_info *sysinfo)
ret = ret ?: get_tdx_sys_info_td_ctrl(&sysinfo->td_ctrl);
ret = ret ?: get_tdx_sys_info_td_conf(&sysinfo->td_conf);
+ /*
+ * Don't treat a module that doesn't support Dynamic PAMT
+ * as a failure. Only read the metadata optionally.
+ */
+ if (!ret && tdx_supports_dynamic_pamt(sysinfo))
+ ret = get_tdx_sys_info_tdmr_dpamt(&sysinfo->tdmr);
+
return ret;
}
--
2.54.0
^ permalink raw reply related
* [PATCH v6 01/11] x86/virt/tdx: Simplify tdmr_get_pamt_sz()
From: Rick Edgecombe @ 2026-05-26 2:35 UTC (permalink / raw)
To: bp, dave.hansen, hpa, kas, kvm, linux-coco, linux-doc,
linux-kernel, mingo, nik.borisov, pbonzini, seanjc, tglx,
vannapurve, x86, chao.gao, yan.y.zhao, kai.huang
Cc: rick.p.edgecombe, Binbin Wu
In-Reply-To: <20260526023515.288829-1-rick.p.edgecombe@intel.com>
For each memory region that the TDX module might use (called TDMR), three
separate traditional PAMT allocations are needed. One for each supported
page size (1GB, 2MB, 4KB). These store information on each page in the
TDMR. In Linux, they are allocated out of one physically contiguous block,
in order to more efficiently use some internal TDX module book keeping
resources. So some simple math is needed to break the single large
allocation into three smaller allocations for each page size.
There are some commonalities in the math needed to calculate the base and
size for each smaller allocation, and so an effort was made to share logic
across the three. Unfortunately doing this turned out unnaturally tortured,
with a loop iterating over the three page sizes, only to call into a
function with cases statement for each page size. In the future Dynamic
PAMT will add more logic that is special to the 4KB page size, making the
benefit of the math sharing even more questionable.
Three is not a very high number, so get rid of the loop and just duplicate
the small calculation three times. In doing so, setup for future Dynamic
PAMT changes.
Since the loop that iterates over it is gone, further simplify the code by
dropping the array of intermediate size and base storage. Just store the
values to their final locations. Accept the small complication of having
to clear tdmr->pamt_4k_base in the error path, so that tdmr_do_pamt_func()
will not try to operate on the TDMR struct when attempting to free it.
Assisted-by: GitHub Copilot:claude-opus-4-6 Claude:claude-opus-4-7
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
---
v6:
- Drop {} by moving a comment (Binbin)
- Log tweaks
v4:
- Just refer to global var instead of passing pamt_entry_size around
(Xiaoyao)
- Remove setting pamt_4k_base to zero, because it already is zero.
Adjust the comment appropriately (Kai)
v3:
- New patch
---
arch/x86/virt/vmx/tdx/tdx.c | 93 ++++++++++++-------------------------
1 file changed, 29 insertions(+), 64 deletions(-)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 967482ae3c801..487f389f52f4b 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -516,31 +516,21 @@ static __init int fill_out_tdmrs(struct list_head *tmb_list,
* Calculate PAMT size given a TDMR and a page size. The returned
* PAMT size is always aligned up to 4K page boundary.
*/
-static __init unsigned long tdmr_get_pamt_sz(struct tdmr_info *tdmr, int pgsz,
- u16 pamt_entry_size)
+static __init unsigned long tdmr_get_pamt_sz(struct tdmr_info *tdmr, int pgsz)
{
unsigned long pamt_sz, nr_pamt_entries;
+ const int tdx_pg_size_shift[] = { PAGE_SHIFT, PMD_SHIFT, PUD_SHIFT };
+ const u16 pamt_entry_size[TDX_PS_NR] = {
+ tdx_sysinfo.tdmr.pamt_4k_entry_size,
+ tdx_sysinfo.tdmr.pamt_2m_entry_size,
+ tdx_sysinfo.tdmr.pamt_1g_entry_size,
+ };
- switch (pgsz) {
- case TDX_PS_4K:
- nr_pamt_entries = tdmr->size >> PAGE_SHIFT;
- break;
- case TDX_PS_2M:
- nr_pamt_entries = tdmr->size >> PMD_SHIFT;
- break;
- case TDX_PS_1G:
- nr_pamt_entries = tdmr->size >> PUD_SHIFT;
- break;
- default:
- WARN_ON_ONCE(1);
- return 0;
- }
+ nr_pamt_entries = tdmr->size >> tdx_pg_size_shift[pgsz];
+ pamt_sz = nr_pamt_entries * pamt_entry_size[pgsz];
- pamt_sz = nr_pamt_entries * pamt_entry_size;
/* TDX requires PAMT size must be 4K aligned */
- pamt_sz = ALIGN(pamt_sz, PAGE_SIZE);
-
- return pamt_sz;
+ return PAGE_ALIGN(pamt_sz);
}
/*
@@ -578,28 +568,21 @@ static __init int tdmr_get_nid(struct tdmr_info *tdmr, struct list_head *tmb_lis
* within @tdmr, and set up PAMTs for @tdmr.
*/
static __init int tdmr_set_up_pamt(struct tdmr_info *tdmr,
- struct list_head *tmb_list,
- u16 pamt_entry_size[])
+ struct list_head *tmb_list)
{
- unsigned long pamt_base[TDX_PS_NR];
- unsigned long pamt_size[TDX_PS_NR];
- unsigned long tdmr_pamt_base;
unsigned long tdmr_pamt_size;
struct page *pamt;
- int pgsz, nid;
-
+ int nid;
nid = tdmr_get_nid(tdmr, tmb_list);
/*
* Calculate the PAMT size for each TDX supported page size
* and the total PAMT size.
*/
- tdmr_pamt_size = 0;
- for (pgsz = TDX_PS_4K; pgsz < TDX_PS_NR; pgsz++) {
- pamt_size[pgsz] = tdmr_get_pamt_sz(tdmr, pgsz,
- pamt_entry_size[pgsz]);
- tdmr_pamt_size += pamt_size[pgsz];
- }
+ tdmr->pamt_4k_size = tdmr_get_pamt_sz(tdmr, TDX_PS_4K);
+ tdmr->pamt_2m_size = tdmr_get_pamt_sz(tdmr, TDX_PS_2M);
+ tdmr->pamt_1g_size = tdmr_get_pamt_sz(tdmr, TDX_PS_1G);
+ tdmr_pamt_size = tdmr->pamt_4k_size + tdmr->pamt_2m_size + tdmr->pamt_1g_size;
/*
* Allocate one chunk of physically contiguous memory for all
@@ -607,26 +590,18 @@ static __init int tdmr_set_up_pamt(struct tdmr_info *tdmr,
* in overlapped TDMRs.
*/
pamt = alloc_contig_pages(tdmr_pamt_size >> PAGE_SHIFT, GFP_KERNEL,
- nid, &node_online_map);
+ nid, &node_online_map);
+
+ /*
+ * tdmr->pamt_4k_base is still zero so the error
+ * path of the caller will skip freeing the pamt.
+ */
if (!pamt)
return -ENOMEM;
- /*
- * Break the contiguous allocation back up into the
- * individual PAMTs for each page size.
- */
- tdmr_pamt_base = page_to_pfn(pamt) << PAGE_SHIFT;
- for (pgsz = TDX_PS_4K; pgsz < TDX_PS_NR; pgsz++) {
- pamt_base[pgsz] = tdmr_pamt_base;
- tdmr_pamt_base += pamt_size[pgsz];
- }
-
- tdmr->pamt_4k_base = pamt_base[TDX_PS_4K];
- tdmr->pamt_4k_size = pamt_size[TDX_PS_4K];
- tdmr->pamt_2m_base = pamt_base[TDX_PS_2M];
- tdmr->pamt_2m_size = pamt_size[TDX_PS_2M];
- tdmr->pamt_1g_base = pamt_base[TDX_PS_1G];
- tdmr->pamt_1g_size = pamt_size[TDX_PS_1G];
+ tdmr->pamt_4k_base = page_to_phys(pamt);
+ tdmr->pamt_2m_base = tdmr->pamt_4k_base + tdmr->pamt_4k_size;
+ tdmr->pamt_1g_base = tdmr->pamt_2m_base + tdmr->pamt_2m_size;
return 0;
}
@@ -657,10 +632,7 @@ static __init void tdmr_do_pamt_func(struct tdmr_info *tdmr,
tdmr_get_pamt(tdmr, &pamt_base, &pamt_size);
/* Do nothing if PAMT hasn't been allocated for this TDMR */
- if (!pamt_size)
- return;
-
- if (WARN_ON_ONCE(!pamt_base))
+ if (!pamt_base)
return;
pamt_func(pamt_base, pamt_size);
@@ -686,14 +658,12 @@ static __init void tdmrs_free_pamt_all(struct tdmr_info_list *tdmr_list)
/* Allocate and set up PAMTs for all TDMRs */
static __init int tdmrs_set_up_pamt_all(struct tdmr_info_list *tdmr_list,
- struct list_head *tmb_list,
- u16 pamt_entry_size[])
+ struct list_head *tmb_list)
{
int i, ret = 0;
for (i = 0; i < tdmr_list->nr_consumed_tdmrs; i++) {
- ret = tdmr_set_up_pamt(tdmr_entry(tdmr_list, i), tmb_list,
- pamt_entry_size);
+ ret = tdmr_set_up_pamt(tdmr_entry(tdmr_list, i), tmb_list);
if (ret)
goto err;
}
@@ -970,18 +940,13 @@ static __init int construct_tdmrs(struct list_head *tmb_list,
struct tdmr_info_list *tdmr_list,
struct tdx_sys_info_tdmr *sysinfo_tdmr)
{
- u16 pamt_entry_size[TDX_PS_NR] = {
- sysinfo_tdmr->pamt_4k_entry_size,
- sysinfo_tdmr->pamt_2m_entry_size,
- sysinfo_tdmr->pamt_1g_entry_size,
- };
int ret;
ret = fill_out_tdmrs(tmb_list, tdmr_list);
if (ret)
return ret;
- ret = tdmrs_set_up_pamt_all(tdmr_list, tmb_list, pamt_entry_size);
+ ret = tdmrs_set_up_pamt_all(tdmr_list, tmb_list);
if (ret)
return ret;
--
2.54.0
^ permalink raw reply related
* [PATCH v6 03/11] x86/virt/tdx: Add tdx_alloc/free_control_page() helpers
From: Rick Edgecombe @ 2026-05-26 2:35 UTC (permalink / raw)
To: bp, dave.hansen, hpa, kas, kvm, linux-coco, linux-doc,
linux-kernel, mingo, nik.borisov, pbonzini, seanjc, tglx,
vannapurve, x86, chao.gao, yan.y.zhao, kai.huang
Cc: rick.p.edgecombe, Kirill A. Shutemov
In-Reply-To: <20260526023515.288829-1-rick.p.edgecombe@intel.com>
From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Add helpers to use when allocating or preparing pages that are handed to
the TDX-Module for use as control/S-EPT pages, and thus need Dynamic PAMT
adjustments.
The TDX module tracks some state for each page of physical memory that it
might use. It calls this state the PAMT. It includes separate state for
each page size a physical page could be utilized at within the TDX module
(1GB, 2MB, 4KB). In Dynamic PAMT, only the 4KB page size state is
allocated dynamically. So for pages that TDX will use as 2MB physically
contiguous pages, Dynamic PAMT backing is not needed.
KVM will need to hand pages to the TDX module that it will use at 4KB
granularity. So these pages will need Dynamic PAMT backing added before
they are used by the TDX module, and removed afterwards.
Add tdx_alloc_control_page() and tdx_free_control_page() to handle both
page allocation and Dynamic PAMT installation. Make them behave like
normal alloc/free functions where allocation can fail in the case of no
memory, but free (with any necessary Dynamic PAMT release) always
succeeds. Do this so they can support the existing TDX flows that require
teardowns to succeed.
Also create tdx_pamt_get/put() to handle installing Dynamic PAMT 4KB
backing for pages that are already allocated (such as KVM's use of S-EPT
page tables or guest private memory). Have them take a pfn instead of a
struct page, as future changes will want to use these helpers for guest
pages which are tracked by PFN.
Don't CLFLUSH the Dynamic PAMT pages handed to the TDX module, as is done
for some other SEAMCALLs, as the TDX docs specify that this is only
needed on "TD private memory or TD control structure page".
Since these allocations will be easily user triggerable, account the
memory.
Leave logic to handle concurrency issues for future changes.
Assisted-by: GitHub Copilot:claude-opus-4-6 Claude:claude-opus-4-7 Sashiko:claude-opus-4-6
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Co-developed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
---
v6:
The major change was to split out the concurrency stuff into a future
patch. It makes it easier to explain in the log. This one is the basic
functionality. Then the simple version of the concurrency and why in the
next patch. Also, to get rid of the dynamically sized DPAMT backing
support which was not based on a formal spec.
Details:
- Split out concurrency stuff into next patch because the log was too long
- Switch to fixed size pamt page arrays (Nikolay)
- Rename tdx_alloc_page()/tdx_free_page() to tdx_alloc_control_page()/
tdx_free_control_page() to reflect control/S-EPT purpose (Sean)
- Take gfp from the caller in tdx_alloc_control_page() (Sean)
- Narrow external API: make tdx_pamt_get()/tdx_pamt_put() static and
export only tdx_alloc_control_page()/tdx_free_control_page() (note:
dropped inline helpers since the discussion on Sean's series resulted
in them not being needed)
- Switch EXPORT_SYMBOL_GPL to EXPORT_SYMBOL_FOR_KVM (Sean)
- Use WARN_ON_ONCE() instead of pr_err() for TDX module failures (Sean)
- Fold alloc_pamt_array()/free_pamt_array() helpers back in and fix the
error-unwind index bug (dpamt_pages[i] -> [j])
- Adjustments after struct page->pfn
- Adjustments from dropping error helper patches
- Make the free error paths more normal
- Drop gfp_t arg in tdx_alloc_control_page(). In the Sean mega v5, it
was really needed because the kvm_mmu_memory_cache had a gfp_t it
needed something to do with. But this was still weird because that
version didn't handle allocating the DPAMT pages as the gfp_t. And in
the end all the callers pass GFP_KERNEL_ACCOUNT. So just drop the arg.
- Log tweaks
---
arch/x86/include/asm/tdx.h | 7 ++
arch/x86/virt/vmx/tdx/tdx.c | 159 ++++++++++++++++++++++++++++++++++++
arch/x86/virt/vmx/tdx/tdx.h | 2 +
3 files changed, 168 insertions(+)
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index 82dc27aecf297..74e75db5728c7 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -37,6 +37,7 @@
#include <uapi/asm/mce.h>
#include <asm/tdx_global_metadata.h>
+#include <linux/mm.h>
#include <linux/pgtable.h>
/*
@@ -160,6 +161,12 @@ void tdx_guest_keyid_free(unsigned int keyid);
void tdx_quirk_reset_paddr(unsigned long base, unsigned long size);
+/* Number PAMT pages to be provided to TDX module per 2MB region of PA */
+#define TDX_DPAMT_ENTRY_PAGE_CNT 2
+
+struct page *tdx_alloc_control_page(void);
+void tdx_free_control_page(struct page *page);
+
struct tdx_td {
/* TD root structure: */
struct page *tdr_page;
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 9ebd192cb5c17..9e0812d87ab06 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1919,6 +1919,165 @@ u64 tdh_phymem_page_wbinvd_hkid(u64 hkid, kvm_pfn_t pfn)
}
EXPORT_SYMBOL_FOR_KVM(tdh_phymem_page_wbinvd_hkid);
+static int alloc_pamt_array(struct page **pamt_pages)
+{
+ int i, j;
+
+ for (i = 0; i < TDX_DPAMT_ENTRY_PAGE_CNT; i++) {
+ pamt_pages[i] = alloc_page(GFP_KERNEL_ACCOUNT);
+ if (!pamt_pages[i])
+ goto err;
+ }
+
+ return 0;
+err:
+ for (j = 0; j < i; j++)
+ __free_page(pamt_pages[j]);
+ return -ENOMEM;
+}
+
+static void free_pamt_array(struct page **pamt_pages)
+{
+ for (int i = 0; i < TDX_DPAMT_ENTRY_PAGE_CNT; i++) {
+ /*
+ * Reset pages unconditionally to cover cases
+ * where they were passed to the TDX module.
+ */
+ tdx_quirk_reset_paddr(page_to_phys(pamt_pages[i]), PAGE_SIZE);
+
+ __free_page(pamt_pages[i]);
+ }
+}
+
+/*
+ * Calculate the arg needed for operating on the DPAMT backing for
+ * a given 4KB page.
+ */
+static u64 pamt_2mb_arg(kvm_pfn_t pfn)
+{
+ unsigned long hpa_2mb = ALIGN_DOWN(pfn << PAGE_SHIFT, PMD_SIZE);
+
+ return hpa_2mb | TDX_PS_2M;
+}
+
+/* Add PAMT backing for the given page. */
+static u64 tdh_phymem_pamt_add(kvm_pfn_t pfn, struct page **pamt_pages)
+{
+ struct tdx_module_args args = {
+ .rcx = pamt_2mb_arg(pfn),
+ .rdx = page_to_phys(pamt_pages[0]),
+ .r8 = page_to_phys(pamt_pages[1]),
+ };
+
+ return seamcall(TDH_PHYMEM_PAMT_ADD, &args);
+}
+
+/* Remove PAMT backing for the given page. */
+static u64 tdh_phymem_pamt_remove(kvm_pfn_t pfn, struct page **pamt_pages)
+{
+ struct tdx_module_args args = {
+ .rcx = pamt_2mb_arg(pfn),
+ };
+ u64 ret;
+
+ ret = seamcall_ret(TDH_PHYMEM_PAMT_REMOVE, &args);
+ if (ret)
+ return ret;
+
+ /* Copy PAMT pages out of the struct per the TDX ABI */
+ pamt_pages[0] = phys_to_page(args.rdx);
+ pamt_pages[1] = phys_to_page(args.r8);
+
+ return 0;
+}
+
+/* Allocate PAMT memory for the given page */
+static int tdx_pamt_get(kvm_pfn_t pfn)
+{
+ struct page *pamt_pages[TDX_DPAMT_ENTRY_PAGE_CNT];
+ u64 tdx_status;
+ int ret;
+
+ if (!tdx_supports_dynamic_pamt(&tdx_sysinfo))
+ return 0;
+
+ ret = alloc_pamt_array(pamt_pages);
+ if (ret)
+ return ret;
+
+ tdx_status = tdh_phymem_pamt_add(pfn, pamt_pages);
+ if (tdx_status != TDX_SUCCESS) {
+ ret = -EIO;
+ goto out_free;
+ }
+
+ return 0;
+out_free:
+ free_pamt_array(pamt_pages);
+ return ret;
+}
+
+/* Free PAMT memory for the given page */
+static void tdx_pamt_put(kvm_pfn_t pfn)
+{
+ struct page *pamt_pages[TDX_DPAMT_ENTRY_PAGE_CNT] = {};
+ u64 tdx_status;
+
+ if (!tdx_supports_dynamic_pamt(&tdx_sysinfo))
+ return;
+
+ tdx_status = tdh_phymem_pamt_remove(pfn, pamt_pages);
+
+ /*
+ * Don't free pamt_pages as it could hold garbage when
+ * tdh_phymem_pamt_remove() fails. Don't panic/BUG_ON(), as
+ * there is no risk of data corruption, but do yell loudly as
+ * failure indicates a kernel bug, memory is being leaked, and
+ * the dangling PAMT entry may cause future operations to fail.
+ */
+ if (WARN_ON_ONCE(tdx_status != TDX_SUCCESS))
+ return;
+
+ free_pamt_array(pamt_pages);
+}
+
+/*
+ * Return a page that can be gifted to the TDX-Module for use as a "control"
+ * page, i.e. pages that are used for control and S-EPT structures for a given
+ * TDX guest, and bound to said guest's HKID and thus obtain TDX protections,
+ * including PAMT tracking.
+ */
+struct page *tdx_alloc_control_page(void)
+{
+ struct page *page;
+
+ page = alloc_page(GFP_KERNEL_ACCOUNT);
+ if (!page)
+ return NULL;
+
+ if (tdx_pamt_get(page_to_pfn(page))) {
+ __free_page(page);
+ return NULL;
+ }
+
+ return page;
+}
+EXPORT_SYMBOL_FOR_KVM(tdx_alloc_control_page);
+
+/*
+ * Free a page that was gifted to the TDX-Module for use as a control/S-EPT
+ * page. After this, the page is no longer protected by TDX.
+ */
+void tdx_free_control_page(struct page *page)
+{
+ if (!page)
+ return;
+
+ tdx_pamt_put(page_to_pfn(page));
+ __free_page(page);
+}
+EXPORT_SYMBOL_FOR_KVM(tdx_free_control_page);
+
#ifdef CONFIG_KEXEC_CORE
void tdx_cpu_flush_cache_for_kexec(void)
{
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index dde219c823b41..8c39dde347cc2 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -46,6 +46,8 @@
#define TDH_PHYMEM_PAGE_WBINVD 41
#define TDH_VP_WR 43
#define TDH_SYS_CONFIG 45
+#define TDH_PHYMEM_PAMT_ADD 58
+#define TDH_PHYMEM_PAMT_REMOVE 59
/*
* SEAMCALL leaf:
--
2.54.0
^ permalink raw reply related
* [PATCH v6 00/11] Dynamic PAMT
From: Rick Edgecombe @ 2026-05-26 2:35 UTC (permalink / raw)
To: bp, dave.hansen, hpa, kas, kvm, linux-coco, linux-doc,
linux-kernel, mingo, nik.borisov, pbonzini, seanjc, tglx,
vannapurve, x86, chao.gao, yan.y.zhao, kai.huang
Cc: rick.p.edgecombe
Hi,
This is next revision of Dynamic PAMT TDX series, which I’m calling v6 in
order to differentiate it from Sean’s giant MMU refactor/DPAMT/Huge-page
series which he called v5 [0]. But things are not quite linear, because
that v5 didn’t include the feedback from v4 [1].
So this version is the conflict resolution of:
1. Comments on Dynamic PAMT v4
2. Sean changes in Dynamic PAMT v4 -> Sean Mega v5
3. Feedback to Sean’s v5
For Dynamic PAMT background, please refer to [2].
This series is pretty mature at this point, however with 2 pre-req series
still on the list (more on that below under "Base"), I can't ask for it to
be merged at this point. So I'm hoping to collect some Acks and RB's in
the meantime and then it can have a smooth path once those other series
land. Please especially consider any reviewabiliy concerns on the tip side
that can be ironed out in the meantime.
Changes
=======
Sean’s mega v5
--------------
This had a bunch of MMU refactor work, which did:
a. TDX MMU refactor that generally pushed more TDX knowledge into TDX.c
out of the core MMU. This covered the needs of both DPAMT and huge
pages.
b. Redid the solution for installing DPAMT backing for the pages the MMU
uses for the S-EPT operations.
c. Some huge page changes that I’ll skip here.
(a) has been split into another series [3]. After long discussions on v5,
the changes for (b) were rolled back to the original solution in v4.
Sean’s v5 included him trying to do Kai’s idea and running into trouble,
then a second new idea which also was found to have issues on review of
v5. By my count we have had at least 4 or 5 ideas by smart people that led
us back to the same solution of keeping a cache of pages and adjusting the
DPAMT right before give the page to the TDX module. I, again, think that
we should either accept the current solution or get started on going back
to change the arch in order to make it more workable for this problem.
Dropping Non-Required Changes
-----------------------------
In the interest of finally clearing these patches, I dropped everything I
could out of the series.
The most significant thing dropped was the optimization around the
refcount allocation. It is a good thing to drop because it is not required
to make Dynamic PAMT useful as a memory optimization. And there is room
for debate on how far to optimize the last little bit of memory usage.
To recap, the kernel implementation keeps a kernel side refcount for each
2MB of the physical memory. The non-optimized version just uses a single
vmalloc to cover the range from 0 to max_pfn. In the worst case this is
8GB of memory. The optimization tried to not allocate refcounts for the
sparse ranges that didn't have any RAM.
For a simple small server with mostly physical contiguous RAM and no CXL
complications, the basic implementation should be close to optimal anyway.
And for big servers, an 8GB allocation is going to have less impact. In
the end Dynamic PAMT *is* an optimization that we will force on as a
good default option. Even with all the optimizations we could throw at it,
if the system is 100% TDs, Dynamic PAMT could come out slightly behind. So
judgment on good defaults is needed regardless.
Consider a couple simple examples of TDX enabled, but no TDs, and the
non-optimized refcount solution:
Machine PAMT (GB) DPAMT (GB) Savings/(Loss)
256GB (max_pfn at 256GB) 1.02 0.01 100x
256GB (max_pfn at maxpa) 1.02 8.01 (8x)
2TB (max_pfn at 2TB) 8.19 0.08 99x
2TB (max_pfn at maxpa) 8.19 8.08 1x
The weird server loses a little bit, but not nearly as much as the normal
ones gain. Still enough benefit in general to make Dynamic PAMT a
worthwhile default setting. So let's start with the simplest solution,
which is an improvement in most cases. And then separate out the refcount
optimization discussion for later.
Besides that, I dropped the error cleanups. As I was implementing the last
discussion, I found it a bit awkward in some places. Also I noticed that
Dave did not fully agree to that proposal either. So it's a continual
source of style controversy and we can separate it out from the Dynamic
PAMT work.
I did not drop the optimization that uses the refcounts to avoid taking
the global lock in tdx_pamt_get/put() because I considered it critical for
making Dynamic PAMT default on. It is more about avoid regressing KVM EPT
violation contention, and not about squeezing out more memory savings from
Dynamic PAMT.
Regarding whether we could strip more out of the series if we made this a
boot time kernel parameter. I think it's possible to drop "x86/virt/tdx:
Allocate reference counters for PAMT memory" and "x86/virt/tdx: Allocate
reference counters for PAMT memory" and still have something that is
functional. I didn't go that route for this revision because making the
feature optional seemed like too much of a divergence from past discussion.
But it is an option if this series seems like too much to digest at once.
AI use in this revision
=======================
While AI enhanced development is still relatively new to the kernel world,
I wanted to share a bit about how this series was generated. For both
consideration in reviewing, and also maybe people might find it
interesting. This was my first time using AI for serious kernel work, so
it was kind of a micromanaged evaluation type use. I used an opus model
with a dump of the many mail threads and a description of how they were
related. Since the previous discussion was pretty disordered, I had it try
to catch any feedback that was missed or conflicted for each patch. And it
caught a few that I had missed. I also used it to turn some of the
feedback into code changes, and to heavily scrutinize the concurrency
logic in tdx_pamt_get/put(). I used it to suggest some log changes too,
but had to edit most of those pretty heavily. Lastly, I used the Chris
Meson and Sashiko review prompts to review the series, which generated a
few changes. All this experimentation generated quite a few Assisted-by
tags, which now feels kinda excessive...
Base
====
This is based on v2 of the MMU refactor series Yan posted a few weeks ago
[3], which is itself based on the struct page to pfn conversion series[4].
A full stack branch can be found here: [5].
Testing
=======
This series was tested in the usual suite, but also with the optimization
patch removed.
[0] https://lore.kernel.org/kvm/20260129011517.3545883-1-seanjc@google.com/
[1] https://lore.kernel.org/kvm/20251121005125.417831-1-rick.p.edgecombe@intel.com/
[2] https://lore.kernel.org/kvm/20250918232224.2202592-1-rick.p.edgecombe@intel.com/
[3] https://lore.kernel.org/kvm/20260509075201.4077-1-yan.y.zhao@intel.com/
[4] https://lore.kernel.org/kvm/20260430014852.24183-1-yan.y.zhao@intel.com/
[5] https://github.com/intel-staging/tdx/tree/dpamt_v6
Kiryl Shutsemau (9):
x86/virt/tdx: Allocate page bitmap for Dynamic PAMT
x86/virt/tdx: Add tdx_alloc/free_control_page() helpers
x86/virt/tdx: Allocate ref counts for Dynamic PAMT memory
x86/virt/tdx: Handle concurrent callers in tdx_pamt_get/put()
x86/virt/tdx: Optimize tdx_pamt_get/put()
KVM: TDX: Allocate PAMT memory for TD and vCPU control structures
KVM: TDX: Get/put PAMT pages when (un)mapping private memory
x86/virt/tdx: Enable Dynamic PAMT
Documentation/x86: Add documentation for TDX's Dynamic PAMT
Rick Edgecombe (2):
x86/virt/tdx: Simplify tdmr_get_pamt_sz()
x86/tdx: Add APIs to support Dynamic PAMT ops from KVM's fault path
Documentation/arch/x86/tdx.rst | 22 +
arch/x86/include/asm/kvm-x86-ops.h | 1 +
arch/x86/include/asm/kvm_host.h | 2 +
arch/x86/include/asm/tdx.h | 38 ++
arch/x86/include/asm/tdx_global_metadata.h | 3 +
arch/x86/kvm/mmu/mmu.c | 4 +
arch/x86/kvm/vmx/tdx.c | 100 +++--
arch/x86/kvm/vmx/tdx.h | 2 +
arch/x86/virt/vmx/tdx/tdx.c | 445 +++++++++++++++++---
arch/x86/virt/vmx/tdx/tdx.h | 5 +-
arch/x86/virt/vmx/tdx/tdx_global_metadata.c | 21 +-
11 files changed, 544 insertions(+), 99 deletions(-)
--
2.54.0
^ permalink raw reply
* [PATCH v6 04/11] x86/virt/tdx: Allocate ref counts for Dynamic PAMT memory
From: Rick Edgecombe @ 2026-05-26 2:35 UTC (permalink / raw)
To: bp, dave.hansen, hpa, kas, kvm, linux-coco, linux-doc,
linux-kernel, mingo, nik.borisov, pbonzini, seanjc, tglx,
vannapurve, x86, chao.gao, yan.y.zhao, kai.huang
Cc: rick.p.edgecombe, Kirill A. Shutemov
In-Reply-To: <20260526023515.288829-1-rick.p.edgecombe@intel.com>
From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
The PAMT memory holds metadata for all possible TDX protected memory. Each
physical address range is covered by PAMT entries at three levels (1GB,
2MB, 4KB). With Dynamic PAMT, the 4KB range of PAMT is allocated on
demand. The kernel supplies the TDX module with page pairs to store the
4KB entries, which cover 2MB of host physical memory. The kernel must
provide this page pair before using pages from the range for TDX. If this
is not done, SEAMCALLs that give the pages to be protected by the TDX module
will fail.
Allocate reference counters for every 2MB range to track TDX memory usage.
This can be used to handle concurrent get/put callers, in order to
accurately determine when the dynamic 4KB level of Dynamic PAMT needs to
be allocated and when it can be freed.
This allocation will currently consume 2 MB for every 1 TB of address
space from 0 to max_pfn. The allocation size will depend on how the RAM is
physically laid out. In a worst case scenario where the entire 52-bit
address space is covered this would be 8GB. Then the DPAMT refcount
allocations could hypothetically cause the savings from Dynamic PAMT to go
negative on exotic platforms with sparse, small amounts of memory.
Future changes could reduce this refcount overhead to be only allocating
refcounts for physical ranges that contain memory that TDX can use.
However, this is left for future work.
Assisted-by: Sashiko:claude-opus-4-6 GitHub Copilot:claude-opus-4-6 Sashiko:claude-opus-4-6
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Co-developed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
---
v6:
- Remove confusing reference to allocating PAMT memory in
pamt_refcounts comment. (Yan)
- Rename "metadata" function names that really deal with refcounts, as
metadata already has a different meaning in TDX.
- Move tdx_find_pamt_refcount() to this patch to aid in reviewability
v4:
- Log typo (Binbin)
- round correctly when computing PAMT refcount size (Binbin)
- Zero refcount vmalloc allocation (Note: This got replaced in
optimization patch with a zero-ed allocation, but this showed up in
testing with the optimization patches removed. Since it's fixed
before this code is exercised, it's not a bisectability issue, but fix
it anyway.)
v3:
- Split out lazily populate optimization to next patch (Dave)
- Add comment around pamt_refcounts (Dave)
- Improve log
---
arch/x86/virt/vmx/tdx/tdx.c | 54 ++++++++++++++++++++++++++++++++++++-
1 file changed, 53 insertions(+), 1 deletion(-)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 9e0812d87ab06..6658a6be6697c 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -30,6 +30,7 @@
#include <linux/suspend.h>
#include <linux/syscore_ops.h>
#include <linux/idr.h>
+#include <linux/vmalloc.h>
#include <asm/page.h>
#include <asm/special_insns.h>
#include <asm/msr-index.h>
@@ -52,6 +53,14 @@ static DEFINE_PER_CPU(bool, tdx_lp_initialized);
static struct tdmr_info_list tdx_tdmr_list;
+/*
+ * On a machine with Dynamic PAMT, the kernel maintains a reference counter
+ * for every 2M range. The counter indicates how many users there are for
+ * the PAMT memory of the 2M range. The kernel allocates PAMT refcounts at
+ * initialization.
+ */
+static atomic_t *pamt_refcounts;
+
/* All TDX-usable memory regions. Protected by mem_hotplug_lock. */
static LIST_HEAD(tdx_memlist);
@@ -254,6 +263,43 @@ static struct syscore tdx_syscore = {
.ops = &tdx_syscore_ops,
};
+/*
+ * Allocate PAMT reference counters for all physical memory.
+ *
+ * It consumes 2MiB for every 1TiB of physical memory.
+ */
+static int init_pamt_refcounts(void)
+{
+ size_t size = DIV_ROUND_UP(max_pfn, PTRS_PER_PTE) * sizeof(*pamt_refcounts);
+
+ if (!tdx_supports_dynamic_pamt(&tdx_sysinfo))
+ return 0;
+
+ pamt_refcounts = __vmalloc(size, GFP_KERNEL | __GFP_ZERO);
+ if (!pamt_refcounts)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void free_pamt_refcounts(void)
+{
+ if (!tdx_supports_dynamic_pamt(&tdx_sysinfo))
+ return;
+
+ vfree(pamt_refcounts);
+ pamt_refcounts = NULL;
+}
+
+/* Find PAMT refcount for a given physical address */
+static atomic_t * __maybe_unused tdx_find_pamt_refcount(unsigned long pfn)
+{
+ /* Find which PMD a PFN is in. */
+ unsigned long index = pfn >> (PMD_SHIFT - PAGE_SHIFT);
+
+ return &pamt_refcounts[index];
+}
+
/*
* Add a memory region as a TDX memory block. The caller must make sure
* all memory regions are added in address ascending order and don't
@@ -1151,10 +1197,14 @@ static __init int init_tdx_module(void)
*/
get_online_mems();
- ret = build_tdx_memlist(&tdx_memlist);
+ ret = init_pamt_refcounts();
if (ret)
goto out_put_tdxmem;
+ ret = build_tdx_memlist(&tdx_memlist);
+ if (ret)
+ goto err_free_pamt_refcounts;
+
/* Allocate enough space for constructing TDMRs */
ret = alloc_tdmr_list(&tdx_tdmr_list, &tdx_sysinfo.tdmr);
if (ret)
@@ -1204,6 +1254,8 @@ static __init int init_tdx_module(void)
free_tdmr_list(&tdx_tdmr_list);
err_free_tdxmem:
free_tdx_memlist(&tdx_memlist);
+err_free_pamt_refcounts:
+ free_pamt_refcounts();
goto out_put_tdxmem;
}
--
2.54.0
^ permalink raw reply related
* [PATCH v6 05/11] x86/virt/tdx: Handle concurrent callers in tdx_pamt_get/put()
From: Rick Edgecombe @ 2026-05-26 2:35 UTC (permalink / raw)
To: bp, dave.hansen, hpa, kas, kvm, linux-coco, linux-doc,
linux-kernel, mingo, nik.borisov, pbonzini, seanjc, tglx,
vannapurve, x86, chao.gao, yan.y.zhao, kai.huang
Cc: rick.p.edgecombe, Kirill A. Shutemov
In-Reply-To: <20260526023515.288829-1-rick.p.edgecombe@intel.com>
From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
tdx_pamt_get()/tdx_pamt_put() unconditionally add or remove Dynamic PAMT
backing for the 2MB region covering the passed pfn. However, multiple
callers can concurrently operate on 4KB pages that fall within the same
2MB region. When this happens only one Dynamic PAMT page pair needs to be
installed to cover the 2MB range. And when one page is freed, the Dynamic
PAMT backing cannot be freed until all pages in the range are no longer in
use. Make the helpers handle these races internally.
Use the per-2MB refcounts from previous changes to track how many 4KB
pages are in use within each region. Gate the actual Dynamic PAMT add and
remove on refcount transitions (0->1 and 1->0). Serialize the refcount
check and SEAMCALL with a global spinlock so the read-decide-act sequence
is atomic. This also avoids TDX module BUSY errors, as Dynamic PAMT add
and remove SEAMCALLs take an internal TDX module locks at 2MB granularity,
so simultaneous attempts on the same region would conflict.
The lock is global and heavyweight. Use simple conditional logic to keep
correctness obvious. This will be optimized in a later change.
Assisted-by: GitHub Copilot:claude-opus-4-6 Claude:claude-opus-4-7
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Co-developed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
---
v6:
- Split from "x86/virt/tdx: Add tdx_alloc/free_control_page() helpers"
- Return 0 instead of ret to be clearer (Binbin)
- Clarify log (Nikolay)
- Justify why the patch is not optimized in response to comments by
(Nikolay)
- Move tdx_find_pamt_refcount() to faciliate patch re-order
- Adjustments from dropping error helper patches
- Log tweaks
---
arch/x86/virt/vmx/tdx/tdx.c | 72 ++++++++++++++++++++++++++++---------
1 file changed, 56 insertions(+), 16 deletions(-)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 6658a6be6697c..50333eb96efa6 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -2043,10 +2043,14 @@ static u64 tdh_phymem_pamt_remove(kvm_pfn_t pfn, struct page **pamt_pages)
return 0;
}
-/* Allocate PAMT memory for the given page */
+/* Serializes adding/removing PAMT memory */
+static DEFINE_SPINLOCK(pamt_lock);
+
+/* Bump PAMT refcount for the given page and allocate PAMT memory if needed */
static int tdx_pamt_get(kvm_pfn_t pfn)
{
struct page *pamt_pages[TDX_DPAMT_ENTRY_PAGE_CNT];
+ atomic_t *pamt_refcount;
u64 tdx_status;
int ret;
@@ -2057,10 +2061,26 @@ static int tdx_pamt_get(kvm_pfn_t pfn)
if (ret)
return ret;
- tdx_status = tdh_phymem_pamt_add(pfn, pamt_pages);
- if (tdx_status != TDX_SUCCESS) {
- ret = -EIO;
- goto out_free;
+ pamt_refcount = tdx_find_pamt_refcount(pfn);
+
+ scoped_guard(spinlock, &pamt_lock) {
+ /*
+ * If the pamt page is already added (i.e. refcount >= 1),
+ * then just increment the refcount.
+ */
+ if (atomic_read(pamt_refcount)) {
+ atomic_inc(pamt_refcount);
+ goto out_free;
+ }
+
+ /* Try to add the pamt page and take the refcount 0->1. */
+ tdx_status = tdh_phymem_pamt_add(pfn, pamt_pages);
+ if (WARN_ON_ONCE(tdx_status != TDX_SUCCESS)) {
+ ret = -EIO;
+ goto out_free;
+ }
+
+ atomic_set(pamt_refcount, 1);
}
return 0;
@@ -2069,26 +2089,46 @@ static int tdx_pamt_get(kvm_pfn_t pfn)
return ret;
}
-/* Free PAMT memory for the given page */
+/*
+ * Drop PAMT refcount for the given page and free PAMT memory if it is no
+ * longer needed.
+ */
static void tdx_pamt_put(kvm_pfn_t pfn)
{
struct page *pamt_pages[TDX_DPAMT_ENTRY_PAGE_CNT] = {};
+ atomic_t *pamt_refcount;
u64 tdx_status;
if (!tdx_supports_dynamic_pamt(&tdx_sysinfo))
return;
- tdx_status = tdh_phymem_pamt_remove(pfn, pamt_pages);
+ pamt_refcount = tdx_find_pamt_refcount(pfn);
- /*
- * Don't free pamt_pages as it could hold garbage when
- * tdh_phymem_pamt_remove() fails. Don't panic/BUG_ON(), as
- * there is no risk of data corruption, but do yell loudly as
- * failure indicates a kernel bug, memory is being leaked, and
- * the dangling PAMT entry may cause future operations to fail.
- */
- if (WARN_ON_ONCE(tdx_status != TDX_SUCCESS))
- return;
+ scoped_guard(spinlock, &pamt_lock) {
+ /*
+ * If the there are more than 1 references on the pamt page,
+ * don't remove it yet. Just decrement the refcount.
+ */
+ if (atomic_read(pamt_refcount) > 1) {
+ atomic_dec(pamt_refcount);
+ return;
+ }
+
+ /* Try to remove the pamt page and take the refcount 1->0. */
+ tdx_status = tdh_phymem_pamt_remove(pfn, pamt_pages);
+
+ /*
+ * Don't free pamt_pages as it could hold garbage when
+ * tdh_phymem_pamt_remove() fails. Don't panic/BUG_ON(), as
+ * there is no risk of data corruption, but do yell loudly as
+ * failure indicates a kernel bug, memory is being leaked, and
+ * the dangling PAMT entry may cause future operations to fail.
+ */
+ if (WARN_ON_ONCE(tdx_status != TDX_SUCCESS))
+ return;
+
+ atomic_set(pamt_refcount, 0);
+ }
free_pamt_array(pamt_pages);
}
--
2.54.0
^ permalink raw reply related
* [PATCH v6 06/11] x86/virt/tdx: Optimize tdx_pamt_get/put()
From: Rick Edgecombe @ 2026-05-26 2:35 UTC (permalink / raw)
To: bp, dave.hansen, hpa, kas, kvm, linux-coco, linux-doc,
linux-kernel, mingo, nik.borisov, pbonzini, seanjc, tglx,
vannapurve, x86, chao.gao, yan.y.zhao, kai.huang
Cc: rick.p.edgecombe, Kirill A. Shutemov
In-Reply-To: <20260526023515.288829-1-rick.p.edgecombe@intel.com>
From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
The Dynamic PAMT get/put helpers use a global spinlock to serialize all
refcount updates and SEAMCALL invocations. This gives correct behavior for
concurrent callers, but leads to contention. It is especially bad from the
KVM side, which is designed to allow faulting in EPT under a shared lock.
With the global spinlock, not only is the lock an exclusive one, but it is
for all TDs instead of just a single one.
But taking the global lock each time is actually unnecessary. Only the 0->1
and 1->0 refcount transitions actually need the lock (to pair with
SEAMCALLs that actually add and remove with the Dynamic PAMT pages). The
common case of incrementing or decrementing a non-zero refcount can be
done locklessly.
So create a fast and slow path. Check the refcount outside the lock and
only take it for the slowpath (0->1 and 1->0 transitions).
On the put side make the refcount adjustment and lock taking atomic so if
a 'get' happens between them, it doesn't cause the Dynamic PAMT to be
freed incorrectly. On the get side there is no technique for doing the
refcount adjustment and lock atomically, so check the refcount again
inside the lock.
Assisted-by: GitHub Copilot:claude-opus-4-6
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Co-developed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
---
v6:
- Fix "tdx_pamt_add()" typo to "tdx_pamt_get()" in lost-race comment
- Fix error path bug: set ret = -EIO and use WARN_ON_ONCE() instead of
pr_err() for unexpected PAMT.ADD failures (Sean)
- Use "set the refcount 0->1" wording to match atomic_set() usage
- Wrap comments to 80 columns
- Switch to atomic_dec_and_lock() and remove handling of races that are
no longer needed as a result. Adjust comments as appropriate. (Dave)
- Adjustments from dropping error helper patches
v4:
- Use atomic_set() in the HPA_RANGE_NOT_FREE case (Kiryl)
- Log, comment typos (Binbin)
- Move PAMT page allocation after refcount check in tdx_pamt_get() to
avoid an alloc/free in the common path.
v3:
- Split out optimization from “x86/virt/tdx: Add tdx_alloc/free_page() helpers”
- Remove edge case handling that I could not find a reason for
- Write log
---
arch/x86/virt/vmx/tdx/tdx.c | 102 +++++++++++++++++++++---------------
1 file changed, 61 insertions(+), 41 deletions(-)
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 50333eb96efa6..c41c632a4cdf2 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -2057,32 +2057,50 @@ static int tdx_pamt_get(kvm_pfn_t pfn)
if (!tdx_supports_dynamic_pamt(&tdx_sysinfo))
return 0;
+ pamt_refcount = tdx_find_pamt_refcount(pfn);
+
+ /*
+ * If the pamt page is already added (i.e. refcount >= 1),
+ * then just increment the refcount.
+ */
+ if (atomic_inc_not_zero(pamt_refcount))
+ return 0;
+
ret = alloc_pamt_array(pamt_pages);
if (ret)
return ret;
- pamt_refcount = tdx_find_pamt_refcount(pfn);
+ spin_lock(&pamt_lock);
- scoped_guard(spinlock, &pamt_lock) {
- /*
- * If the pamt page is already added (i.e. refcount >= 1),
- * then just increment the refcount.
- */
- if (atomic_read(pamt_refcount)) {
- atomic_inc(pamt_refcount);
- goto out_free;
- }
-
- /* Try to add the pamt page and take the refcount 0->1. */
- tdx_status = tdh_phymem_pamt_add(pfn, pamt_pages);
- if (WARN_ON_ONCE(tdx_status != TDX_SUCCESS)) {
- ret = -EIO;
- goto out_free;
- }
-
- atomic_set(pamt_refcount, 1);
+ /*
+ * Unlike tdx_pamt_put() which uses atomic_dec_and_lock() to
+ * atomically handle the 1->0 transition, the get side has no
+ * equivalent combined primitive for 0->1. Recheck under the
+ * lock since another get may have already done the 0->1
+ * transition after both saw atomic_inc_not_zero() fail.
+ */
+ if (atomic_read(pamt_refcount)) {
+ atomic_inc(pamt_refcount);
+ spin_unlock(&pamt_lock);
+ goto out_free;
}
+ tdx_status = tdh_phymem_pamt_add(pfn, pamt_pages);
+ if (tdx_status == TDX_SUCCESS) {
+ /*
+ * The refcount is zero, and this locked path is the
+ * only way to increase it from 0->1.
+ */
+ atomic_set(pamt_refcount, 1);
+ } else {
+ WARN_ON_ONCE(1);
+ ret = -EIO;
+ spin_unlock(&pamt_lock);
+ goto out_free;
+ }
+
+ spin_unlock(&pamt_lock);
+
return 0;
out_free:
free_pamt_array(pamt_pages);
@@ -2104,32 +2122,34 @@ static void tdx_pamt_put(kvm_pfn_t pfn)
pamt_refcount = tdx_find_pamt_refcount(pfn);
- scoped_guard(spinlock, &pamt_lock) {
+ /*
+ * If there is more than 1 reference on the pamt page, don't
+ * remove it yet. Just decrement the refcount.
+ */
+ if (!atomic_dec_and_lock(pamt_refcount, &pamt_lock))
+ return;
+
+ tdx_status = tdh_phymem_pamt_remove(pfn, pamt_pages);
+
+ /*
+ * Don't free pamt_pages as it could hold garbage when
+ * tdh_phymem_pamt_remove() fails. Don't panic/BUG_ON(), as
+ * there is no risk of data corruption, but do yell loudly as
+ * failure indicates a kernel bug, memory is being leaked, and
+ * the dangling PAMT entry may cause future operations to fail.
+ */
+ if (WARN_ON_ONCE(tdx_status != TDX_SUCCESS)) {
/*
- * If the there are more than 1 references on the pamt page,
- * don't remove it yet. Just decrement the refcount.
+ * atomic_dec_and_lock() already decremented it to 0,
+ * but the PAMT entry still exists since REMOVE failed.
*/
- if (atomic_read(pamt_refcount) > 1) {
- atomic_dec(pamt_refcount);
- return;
- }
-
- /* Try to remove the pamt page and take the refcount 1->0. */
- tdx_status = tdh_phymem_pamt_remove(pfn, pamt_pages);
-
- /*
- * Don't free pamt_pages as it could hold garbage when
- * tdh_phymem_pamt_remove() fails. Don't panic/BUG_ON(), as
- * there is no risk of data corruption, but do yell loudly as
- * failure indicates a kernel bug, memory is being leaked, and
- * the dangling PAMT entry may cause future operations to fail.
- */
- if (WARN_ON_ONCE(tdx_status != TDX_SUCCESS))
- return;
-
- atomic_set(pamt_refcount, 0);
+ atomic_set(pamt_refcount, 1);
+ spin_unlock(&pamt_lock);
+ return;
}
+ spin_unlock(&pamt_lock);
+
free_pamt_array(pamt_pages);
}
--
2.54.0
^ permalink raw reply related
* [PATCH v6 07/11] KVM: TDX: Allocate PAMT memory for TD and vCPU control structures
From: Rick Edgecombe @ 2026-05-26 2:35 UTC (permalink / raw)
To: bp, dave.hansen, hpa, kas, kvm, linux-coco, linux-doc,
linux-kernel, mingo, nik.borisov, pbonzini, seanjc, tglx,
vannapurve, x86, chao.gao, yan.y.zhao, kai.huang
Cc: rick.p.edgecombe, Kirill A. Shutemov
In-Reply-To: <20260526023515.288829-1-rick.p.edgecombe@intel.com>
From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Use control page helpers for allocating and freeing TD control structures,
such these operations can work for Dynamic PAMT.
The TDX module tracks some state for each page of physical memory that it
might use. It calls this state the PAMT. It includes separate state for
each page size a physical page could be utilized at within the TDX module
(1GB, 2MB, 4KB). In Dynamic PAMT, only the 4KB page size state is
allocated dynamically. So the kernel must install PAMT backing for each 4KB
page before gifting it to the TDX module, and tear it down after the page
is reclaimed.
TD-scoped control pages (TDR, TDCS) and vCPU-scoped control pages (TDVPR,
TDCX) are all handed to the TDX module at 4KB page size and are therefore
subject to this requirement. Replace the raw alloc_page()/__free_page()
calls for these pages with tdx_alloc/free_control_page().
Switching between special Dynamic PAMT operations or normal page
alloc/free operations is handled internally in
tdx_alloc/free_control_page(). So don't check for Dynamic PAMT around these
calls. Just call them unconditionally. Similarly, drop the NULL checks
before freeing, as tdx_free_control_page() handles NULL internally.
No functional change intended when Dynamic PAMT is not in use.
Assisted-by: GitHub Copilot:claude-opus-4-6 Claude:claude-opus-4-7
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
[sean: handle alloc+free+reclaim in one patch]
Co-developed-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
[Rick: enhance log]
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
---
arch/x86/kvm/vmx/tdx.c | 35 ++++++++++++++---------------------
1 file changed, 14 insertions(+), 21 deletions(-)
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 2539107e0ad3d..3e67e2471ffe3 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -362,7 +362,7 @@ static void tdx_reclaim_control_page(struct page *ctrl_page)
if (tdx_reclaim_page(ctrl_page))
return;
- __free_page(ctrl_page);
+ tdx_free_control_page(ctrl_page);
}
struct tdx_flush_vp_arg {
@@ -599,7 +599,7 @@ static void tdx_reclaim_td_control_pages(struct kvm *kvm)
tdx_quirk_reset_paddr(page_to_phys(kvm_tdx->td.tdr_page), PAGE_SIZE);
- __free_page(kvm_tdx->td.tdr_page);
+ tdx_free_control_page(kvm_tdx->td.tdr_page);
kvm_tdx->td.tdr_page = NULL;
}
@@ -2444,7 +2444,7 @@ static int __tdx_td_init(struct kvm *kvm, struct td_params *td_params,
ret = -ENOMEM;
- tdr_page = alloc_page(GFP_KERNEL_ACCOUNT);
+ tdr_page = tdx_alloc_control_page();
if (!tdr_page)
goto free_hkid;
@@ -2458,7 +2458,7 @@ static int __tdx_td_init(struct kvm *kvm, struct td_params *td_params,
goto free_tdr;
for (i = 0; i < kvm_tdx->td.tdcs_nr_pages; i++) {
- tdcs_pages[i] = alloc_page(GFP_KERNEL_ACCOUNT);
+ tdcs_pages[i] = tdx_alloc_control_page();
if (!tdcs_pages[i])
goto free_tdcs;
}
@@ -2576,10 +2576,8 @@ static int __tdx_td_init(struct kvm *kvm, struct td_params *td_params,
teardown:
/* Only free pages not yet added, so start at 'i' */
for (; i < kvm_tdx->td.tdcs_nr_pages; i++) {
- if (tdcs_pages[i]) {
- __free_page(tdcs_pages[i]);
- tdcs_pages[i] = NULL;
- }
+ tdx_free_control_page(tdcs_pages[i]);
+ tdcs_pages[i] = NULL;
}
if (!kvm_tdx->td.tdcs_pages)
kfree(tdcs_pages);
@@ -2594,16 +2592,13 @@ static int __tdx_td_init(struct kvm *kvm, struct td_params *td_params,
free_cpumask_var(packages);
free_tdcs:
- for (i = 0; i < kvm_tdx->td.tdcs_nr_pages; i++) {
- if (tdcs_pages[i])
- __free_page(tdcs_pages[i]);
- }
+ for (i = 0; i < kvm_tdx->td.tdcs_nr_pages; i++)
+ tdx_free_control_page(tdcs_pages[i]);
kfree(tdcs_pages);
kvm_tdx->td.tdcs_pages = NULL;
free_tdr:
- if (tdr_page)
- __free_page(tdr_page);
+ tdx_free_control_page(tdr_page);
kvm_tdx->td.tdr_page = NULL;
free_hkid:
@@ -2933,7 +2928,7 @@ static int tdx_td_vcpu_init(struct kvm_vcpu *vcpu, u64 vcpu_rcx)
int ret, i;
u64 err;
- page = alloc_page(GFP_KERNEL_ACCOUNT);
+ page = tdx_alloc_control_page();
if (!page)
return -ENOMEM;
tdx->vp.tdvpr_page = page;
@@ -2953,7 +2948,7 @@ static int tdx_td_vcpu_init(struct kvm_vcpu *vcpu, u64 vcpu_rcx)
}
for (i = 0; i < kvm_tdx->td.tdcx_nr_pages; i++) {
- page = alloc_page(GFP_KERNEL_ACCOUNT);
+ page = tdx_alloc_control_page();
if (!page) {
ret = -ENOMEM;
goto free_tdcx;
@@ -2975,7 +2970,7 @@ static int tdx_td_vcpu_init(struct kvm_vcpu *vcpu, u64 vcpu_rcx)
* method, but the rest are freed here.
*/
for (; i < kvm_tdx->td.tdcx_nr_pages; i++) {
- __free_page(tdx->vp.tdcx_pages[i]);
+ tdx_free_control_page(tdx->vp.tdcx_pages[i]);
tdx->vp.tdcx_pages[i] = NULL;
}
return -EIO;
@@ -3003,16 +2998,14 @@ static int tdx_td_vcpu_init(struct kvm_vcpu *vcpu, u64 vcpu_rcx)
free_tdcx:
for (i = 0; i < kvm_tdx->td.tdcx_nr_pages; i++) {
- if (tdx->vp.tdcx_pages[i])
- __free_page(tdx->vp.tdcx_pages[i]);
+ tdx_free_control_page(tdx->vp.tdcx_pages[i]);
tdx->vp.tdcx_pages[i] = NULL;
}
kfree(tdx->vp.tdcx_pages);
tdx->vp.tdcx_pages = NULL;
free_tdvpr:
- if (tdx->vp.tdvpr_page)
- __free_page(tdx->vp.tdvpr_page);
+ tdx_free_control_page(tdx->vp.tdvpr_page);
tdx->vp.tdvpr_page = NULL;
tdx->vp.tdvpr_pa = 0;
--
2.54.0
^ permalink raw reply related
* [PATCH v6 10/11] x86/virt/tdx: Enable Dynamic PAMT
From: Rick Edgecombe @ 2026-05-26 2:35 UTC (permalink / raw)
To: bp, dave.hansen, hpa, kas, kvm, linux-coco, linux-doc,
linux-kernel, mingo, nik.borisov, pbonzini, seanjc, tglx,
vannapurve, x86, chao.gao, yan.y.zhao, kai.huang
Cc: rick.p.edgecombe, Kirill A. Shutemov
In-Reply-To: <20260526023515.288829-1-rick.p.edgecombe@intel.com>
From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
The Physical Address Metadata Table (PAMT) holds TDX metadata for
physical memory and must be allocated by the kernel during TDX module
initialization. Dynamic PAMT is a TDX module feature that can reduce this
memory use by allocating part of the PAMT dynamically.
All pieces are in place to Enable Dynamic PAMT if it is supported.
Determine if the TDX module supports it by checking the 'features0' bit
exposed by the TDX module.
The TDX module also exposes information about whether the *system* (and
not the module) supports Dynamic PAMT.
The TDX module documentation describes how PAMT works internally. To allow
the last level to be dynamically allocated, it uses a 3 level tree
structure, not unlike page tables. Like page tables, it has a maximum
address space that it can cover. This address space can be covered in 48
bits. If the host physical address space is higher than this, than the
TDX module can't guarantee the tree will be able to cover the TDX memory.
The TDX module exposes this system support via metadata stating the
minimum number of HKIDs that need to be available in order for Dynamic
PAMT to be usable. The reasoning appears to be that more HKIDs can shrink
the "real" addressable physical address bits enough to make the 48 bit
Dynamic PAMT limit workable on high physical address width HW. However,
the docs also clearly explain the 48 bit limit and how this fits into the
Dymamic PAMT tree constraints.
The handy x86_phys_bits value is already read and adjusted for keyid bits.
So just compare that against 48 instead of reading more metadata and
burdening the code with the more tenuous connection to minimum HKID bits.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Co-developed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
---
v6:
- After Nikolai pointed out that the TDX docs actually have the Dynamic
PAMT pages-per-2MB region fixed at 2 instead of variable sized, I
checked over the docs more closely looking for anything else that might
have been missed. Spotted this 48 bit physical address bit check in the
docs, so added it.
---
arch/x86/include/asm/tdx.h | 11 ++++++++++-
arch/x86/virt/vmx/tdx/tdx.c | 11 +++++++++--
arch/x86/virt/vmx/tdx/tdx.h | 3 ---
3 files changed, 19 insertions(+), 6 deletions(-)
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index 191da84bbf2a1..187014686df3e 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -33,6 +33,10 @@
#define TDX_SUCCESS 0ULL
#define TDX_RND_NO_ENTROPY 0x8000020300000000ULL
+/* Bit definitions of TDX_FEATURES0 metadata field */
+#define TDX_FEATURES0_NO_RBP_MOD BIT_ULL(18)
+#define TDX_FEATURES0_DYNAMIC_PAMT BIT_ULL(36)
+
#ifndef __ASSEMBLER__
#include <uapi/asm/mce.h>
@@ -152,7 +156,12 @@ const struct tdx_sys_info *tdx_get_sysinfo(void);
static inline bool tdx_supports_dynamic_pamt(const struct tdx_sys_info *sysinfo)
{
- return false; /* To be enabled when kernel is ready */
+ /*
+ * The TDX Module's internal Dynamic PAMT tree structure can't
+ * handle physical addresses with more than 48 bits.
+ */
+ return sysinfo->features.tdx_features0 & TDX_FEATURES0_DYNAMIC_PAMT &&
+ boot_cpu_data.x86_phys_bits <= 48;
}
/* Simple structure for pre-allocating Dynamic PAMT pages outside of locks. */
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 3544794fb092a..75140511571bf 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1028,8 +1028,9 @@ static __init int construct_tdmrs(struct list_head *tmb_list,
return ret;
}
-static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
- u64 global_keyid)
+#define TDX_SYS_CONFIG_DYNAMIC_PAMT BIT(16)
+
+static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, u64 global_keyid)
{
struct tdx_module_args args = {};
u64 *tdmr_pa_array;
@@ -1056,6 +1057,12 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
args.rcx = __pa(tdmr_pa_array);
args.rdx = tdmr_list->nr_consumed_tdmrs;
args.r8 = global_keyid;
+
+ if (tdx_supports_dynamic_pamt(&tdx_sysinfo)) {
+ pr_info("Enable Dynamic PAMT\n");
+ args.r8 |= TDX_SYS_CONFIG_DYNAMIC_PAMT;
+ }
+
ret = seamcall_prerr(TDH_SYS_CONFIG, &args);
/* Free the array as it is not required anymore. */
diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
index 8c39dde347cc2..68a68468fbeb6 100644
--- a/arch/x86/virt/vmx/tdx/tdx.h
+++ b/arch/x86/virt/vmx/tdx/tdx.h
@@ -86,9 +86,6 @@ struct tdmr_info {
DECLARE_FLEX_ARRAY(struct tdmr_reserved_area, reserved_areas);
} __packed __aligned(TDMR_INFO_ALIGNMENT);
-/* Bit definitions of TDX_FEATURES0 metadata field */
-#define TDX_FEATURES0_NO_RBP_MOD BIT(18)
-
/*
* Do not put any hardware-defined TDX structure representations below
* this comment!
--
2.54.0
^ permalink raw reply related
* [PATCH v6 09/11] KVM: TDX: Get/put PAMT pages when (un)mapping private memory
From: Rick Edgecombe @ 2026-05-26 2:35 UTC (permalink / raw)
To: bp, dave.hansen, hpa, kas, kvm, linux-coco, linux-doc,
linux-kernel, mingo, nik.borisov, pbonzini, seanjc, tglx,
vannapurve, x86, chao.gao, yan.y.zhao, kai.huang
Cc: rick.p.edgecombe, Kirill A. Shutemov
In-Reply-To: <20260526023515.288829-1-rick.p.edgecombe@intel.com>
From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Add Dynamic PAMT support to KVM's S-EPT MMU by "getting" a PAMT page when
adding guest memory (PAGE.ADD or PAGE.AUG), and "putting" the page when
removing guest memory (PAGE.REMOVE).
To access the per-vCPU PAMT caches without plumbing @vcpu throughout the
TDP MMU, begrudgingly use kvm_get_running_vcpu() to get the vCPU, and bug
the VM if KVM attempts to set an S-EPT leaf without an active vCPU. KVM
only supports creating _new_ mappings in page (pre)fault paths, all of
which require an active vCPU.
The PAMT memory holds metadata for TDX-protected memory. With Dynamic
PAMT, PAMT_4K is allocated on demand. The kernel supplies the TDX module
with a few pages that cover 2M of host physical memory.
Releases are balanced via tdx_pamt_put(): every control-page free goes
through tdx_free_control_page(), and guest data pages are put directly on
the successful tdh_mem_page_remove() path and in the
tdx_mem_page_add/aug() error path.
Assisted-by: Sashiko:claude-opus-4-6 GitHub Copilot:claude-opus-4-6 Claude:claude-opus-4-7
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Co-developed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
---
v6:
- Don't have topup op take a min param (Yan, Sean)
- Make log match style of the rest of the series
- Adjustments from dropping error helper patches
---
arch/x86/include/asm/kvm-x86-ops.h | 1 +
arch/x86/include/asm/kvm_host.h | 2 +
arch/x86/kvm/mmu/mmu.c | 4 ++
arch/x86/kvm/vmx/tdx.c | 65 ++++++++++++++++++++++++++----
arch/x86/kvm/vmx/tdx.h | 2 +
5 files changed, 66 insertions(+), 8 deletions(-)
diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h
index 10ccf6ea9d9a2..320f1d30edacc 100644
--- a/arch/x86/include/asm/kvm-x86-ops.h
+++ b/arch/x86/include/asm/kvm-x86-ops.h
@@ -97,6 +97,7 @@ KVM_X86_OP_OPTIONAL_RET0(get_mt_mask)
KVM_X86_OP(load_mmu_pgd)
KVM_X86_OP_OPTIONAL_RET0(set_external_spte)
KVM_X86_OP_OPTIONAL(free_external_spt)
+KVM_X86_OP_OPTIONAL_RET0(topup_external_cache)
KVM_X86_OP(has_wbinvd_exit)
KVM_X86_OP(get_l2_tsc_offset)
KVM_X86_OP(get_l2_tsc_multiplier)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 6b28dd387bc61..bfe92e993a212 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1898,6 +1898,8 @@ struct kvm_x86_ops {
/* Update external page tables for page table about to be freed. */
void (*free_external_spt)(struct kvm *kvm, struct kvm_mmu_page *sp);
+ int (*topup_external_cache)(struct kvm_vcpu *vcpu, int min_nr_spts);
+
bool (*has_wbinvd_exit)(void);
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 892246204435c..2a48fc7fccc11 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -607,6 +607,10 @@ static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu, bool maybe_indirect)
PT64_ROOT_MAX_LEVEL);
if (r)
return r;
+
+ r = kvm_x86_call(topup_external_cache)(vcpu, PT64_ROOT_MAX_LEVEL);
+ if (r)
+ return r;
}
r = kvm_mmu_topup_memory_cache(&vcpu->arch.mmu_shadow_page_cache,
PT64_ROOT_MAX_LEVEL);
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 3e67e2471ffe3..ee073cacafbec 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -685,6 +685,8 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu)
if (!irqchip_split(vcpu->kvm))
return -EINVAL;
+ tdx_init_pamt_cache(&tdx->pamt_cache);
+
fpstate_set_confidential(&vcpu->arch.guest_fpu);
vcpu->arch.apic->guest_apic_protected = true;
INIT_LIST_HEAD(&tdx->vt.pi_wakeup_list);
@@ -870,6 +872,8 @@ void tdx_vcpu_free(struct kvm_vcpu *vcpu)
struct vcpu_tdx *tdx = to_tdx(vcpu);
int i;
+ tdx_free_pamt_cache(&tdx->pamt_cache);
+
if (vcpu->cpu != -1) {
KVM_BUG_ON(tdx->state == VCPU_TD_STATE_INITIALIZED, vcpu->kvm);
tdx_flush_vp_on_cpu(vcpu);
@@ -1611,6 +1615,16 @@ void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level)
td_vmcs_write64(to_tdx(vcpu), SHARED_EPT_POINTER, root_hpa);
}
+static int tdx_topup_external_pamt_cache(struct kvm_vcpu *vcpu, int min_nr_spts)
+{
+ /*
+ * Don't cover the root SPT, but cover a possible 4KB private
+ * page in addition to the SPTs. So -1 to exclude the root
+ * SPT, and +1 for the guest page cancel out.
+ */
+ return tdx_topup_pamt_cache(&to_tdx(vcpu)->pamt_cache, min_nr_spts);
+}
+
static int tdx_mem_page_add(struct kvm *kvm, gfn_t gfn, enum pg_level level,
kvm_pfn_t pfn)
{
@@ -1669,16 +1683,29 @@ static struct page *tdx_spte_to_sept_pt(struct kvm *kvm, gfn_t gfn,
static int tdx_sept_map_nonleaf_spte(struct kvm *kvm, gfn_t gfn,
enum pg_level level, u64 new_spte)
{
+ struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+ struct vcpu_tdx *tdx = to_tdx(vcpu);
gpa_t gpa = gfn_to_gpa(gfn);
u64 err, entry, level_state;
struct page *sept_pt;
+ int ret;
+
+ if (KVM_BUG_ON(!vcpu, kvm))
+ return -EIO;
sept_pt = tdx_spte_to_sept_pt(kvm, gfn, new_spte, level);
if (!sept_pt)
return -EIO;
+ ret = tdx_pamt_get(page_to_pfn(sept_pt), &tdx->pamt_cache);
+ if (ret)
+ return ret;
+
err = tdh_mem_sept_add(&to_kvm_tdx(kvm)->td, gpa, level, sept_pt,
&entry, &level_state);
+ if (err)
+ tdx_pamt_put(page_to_pfn(sept_pt));
+
if (unlikely(tdx_operand_busy(err)))
return -EBUSY;
@@ -1691,8 +1718,14 @@ static int tdx_sept_map_nonleaf_spte(struct kvm *kvm, gfn_t gfn,
static int tdx_sept_map_leaf_spte(struct kvm *kvm, gfn_t gfn, enum pg_level level,
u64 new_spte)
{
+ struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
kvm_pfn_t pfn = spte_to_pfn(new_spte);
+ struct vcpu_tdx *tdx = to_tdx(vcpu);
+ int ret;
+
+ if (KVM_BUG_ON(!vcpu, kvm))
+ return -EIO;
/* TODO: handle large pages. */
if (KVM_BUG_ON(level != PG_LEVEL_4K, kvm))
@@ -1700,6 +1733,10 @@ static int tdx_sept_map_leaf_spte(struct kvm *kvm, gfn_t gfn, enum pg_level leve
WARN_ON_ONCE((new_spte & VMX_EPT_RWX_MASK) != VMX_EPT_RWX_MASK);
+ ret = tdx_pamt_get(pfn, &tdx->pamt_cache);
+ if (ret)
+ return ret;
+
/*
* Ensure pre_fault_allowed is read by kvm_arch_vcpu_pre_fault_memory()
* before kvm_tdx->state. Userspace must not be allowed to pre-fault
@@ -1712,10 +1749,15 @@ static int tdx_sept_map_leaf_spte(struct kvm *kvm, gfn_t gfn, enum pg_level leve
* If the TD isn't finalized/runnable, then userspace is initializing
* the VM image via KVM_TDX_INIT_MEM_REGION; ADD the page to the TD.
*/
- if (unlikely(kvm_tdx->state != TD_STATE_RUNNABLE))
- return tdx_mem_page_add(kvm, gfn, level, pfn);
+ if (likely(kvm_tdx->state == TD_STATE_RUNNABLE))
+ ret = tdx_mem_page_aug(kvm, gfn, level, pfn);
+ else
+ ret = tdx_mem_page_add(kvm, gfn, level, pfn);
- return tdx_mem_page_aug(kvm, gfn, level, pfn);
+ if (ret)
+ tdx_pamt_put(pfn);
+
+ return ret;
}
/*
@@ -1812,6 +1854,7 @@ static int tdx_sept_remove_leaf_spte(struct kvm *kvm, gfn_t gfn,
return -EIO;
tdx_quirk_reset_paddr(PFN_PHYS(pfn), PAGE_SIZE);
+ tdx_pamt_put(pfn);
return 0;
}
@@ -1855,6 +1898,8 @@ static int tdx_sept_set_private_spte(struct kvm *kvm, gfn_t gfn, u64 old_spte,
*/
static void tdx_sept_free_private_spt(struct kvm *kvm, struct kvm_mmu_page *sp)
{
+ struct page *sept_pt = virt_to_page(sp->external_spt);
+
/*
* KVM doesn't (yet) zap page table pages in mirror page table while
* TD is active, though guest pages mapped in mirror page table could be
@@ -1868,15 +1913,15 @@ static void tdx_sept_free_private_spt(struct kvm *kvm, struct kvm_mmu_page *sp)
* the page to prevent the kernel from accessing the encrypted page.
*/
if (KVM_BUG_ON(is_hkid_assigned(to_kvm_tdx(kvm)), kvm) ||
- tdx_reclaim_page(virt_to_page(sp->external_spt)))
+ tdx_reclaim_page(sept_pt))
goto out;
/*
- * Immediately free the S-EPT page because RCU-time free is unnecessary
- * after TDH.PHYMEM.PAGE.RECLAIM ensures there are no outstanding
- * readers.
+ * Immediately free the S-EPT page as the TDX subsystem doesn't support
+ * freeing pages from RCU callbacks, and more importantly because
+ * TDH.PHYMEM.PAGE.RECLAIM ensures there are no outstanding readers.
*/
- free_page((unsigned long)sp->external_spt);
+ tdx_free_control_page(sept_pt);
out:
sp->external_spt = NULL;
}
@@ -3468,6 +3513,10 @@ int __init tdx_hardware_setup(void)
vt_x86_ops.set_external_spte = tdx_sept_set_private_spte;
vt_x86_ops.free_external_spt = tdx_sept_free_private_spt;
+
+ if (tdx_supports_dynamic_pamt(tdx_sysinfo))
+ vt_x86_ops.topup_external_cache = tdx_topup_external_pamt_cache;
+
vt_x86_ops.protected_apic_has_interrupt = tdx_protected_apic_has_interrupt;
return 0;
diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h
index b5cd2ffb303e5..47334a5a74eab 100644
--- a/arch/x86/kvm/vmx/tdx.h
+++ b/arch/x86/kvm/vmx/tdx.h
@@ -73,6 +73,8 @@ struct vcpu_tdx {
u64 map_gpa_next;
u64 map_gpa_end;
+
+ struct tdx_pamt_cache pamt_cache;
};
void tdh_vp_rd_failed(struct vcpu_tdx *tdx, char *uclass, u32 field, u64 err);
--
2.54.0
^ permalink raw reply related
* [PATCH v6 08/11] x86/tdx: Add APIs to support Dynamic PAMT ops from KVM's fault path
From: Rick Edgecombe @ 2026-05-26 2:35 UTC (permalink / raw)
To: bp, dave.hansen, hpa, kas, kvm, linux-coco, linux-doc,
linux-kernel, mingo, nik.borisov, pbonzini, seanjc, tglx,
vannapurve, x86, chao.gao, yan.y.zhao, kai.huang
Cc: rick.p.edgecombe
In-Reply-To: <20260526023515.288829-1-rick.p.edgecombe@intel.com>
When handling an EPT violation, KVM holds a spinlock while manipulating
the EPT. Before entering the spinlock it doesn't know how many EPT page
tables will need to be installed or whether a huge page will be used. For
this reason it allocates a worst case number of page tables that it might
need as part of servicing the EPT violation.
Under Dynamic PAMT these pre-allocated pages will potentially need to have
Dynamic PAMT backing pages installed for them. KVM already has helpers to
manage topping up page caches before taking the MMU lock, but they cannot be
passed from KVM to arch/x86 code.
The problem of how and when to install the DPAMT backing pages for the
pages given to the TDX module during the fault path has had a lot of
design attempts.
- Extracting KVM's MMU caches requires too much inlined code added to
headers.
- A few varieties of installing Dynamic PAMT backing when allocating the
S-EPT page tables. [0][1]
- Using mempool_t to transfer the pages between KVM and arch/x86 doesn't
work because it is the component is designed more around maintaining a
pool of pages, rather than topping up a continually drained cache.
So don't do these as they all had various problems. Instead just create a
small simple data structure to use for handing a pre-allocated list of
pages between KVM and arch/x86 code. Model this on KVM's existing MMU
memory caches.
Add a tdx_pamt_cache arg to tdx_pamt_get() so it can draw pages from a
cache when needed. Not all DPAMT page installations will happen under
spinlock, for example control pages. So have tdx_pamt_get() maintain the
existing behavior of allocating from the page allocator when NULL is
passed for the struct tdx_pamt_cache arg. This prevents excess allocations
for cases where it can be avoided.
Export the new helpers for KVM.
Assisted-by: GitHub Copilot:claude-opus-4-6 Claude:claude-opus-4-7
Co-developed-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://lore.kernel.org/kvm/de05853257e9cc66998101943f78a4b7e6e3d741.camel@intel.com/ [0]
Link: https://lore.kernel.org/kvm/aYprxnSHKHUtk7pt@google.com/ [1]
---
v6:
- Filled out log from Sean's series
---
arch/x86/include/asm/tdx.h | 17 ++++++++++
arch/x86/virt/vmx/tdx/tdx.c | 65 +++++++++++++++++++++++++++++++++----
2 files changed, 76 insertions(+), 6 deletions(-)
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index 74e75db5728c7..191da84bbf2a1 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -155,6 +155,23 @@ static inline bool tdx_supports_dynamic_pamt(const struct tdx_sys_info *sysinfo)
return false; /* To be enabled when kernel is ready */
}
+/* Simple structure for pre-allocating Dynamic PAMT pages outside of locks. */
+struct tdx_pamt_cache {
+ struct list_head page_list;
+ int cnt;
+};
+
+static inline void tdx_init_pamt_cache(struct tdx_pamt_cache *cache)
+{
+ INIT_LIST_HEAD(&cache->page_list);
+ cache->cnt = 0;
+}
+
+void tdx_free_pamt_cache(struct tdx_pamt_cache *cache);
+int tdx_topup_pamt_cache(struct tdx_pamt_cache *cache, unsigned long npages);
+int tdx_pamt_get(kvm_pfn_t pfn, struct tdx_pamt_cache *cache);
+void tdx_pamt_put(kvm_pfn_t pfn);
+
int tdx_guest_keyid_alloc(void);
u32 tdx_get_nr_guest_keyids(void);
void tdx_guest_keyid_free(unsigned int keyid);
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index c41c632a4cdf2..3544794fb092a 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1971,12 +1971,33 @@ u64 tdh_phymem_page_wbinvd_hkid(u64 hkid, kvm_pfn_t pfn)
}
EXPORT_SYMBOL_FOR_KVM(tdh_phymem_page_wbinvd_hkid);
-static int alloc_pamt_array(struct page **pamt_pages)
+static struct page *tdx_alloc_page_pamt_cache(struct tdx_pamt_cache *cache)
+{
+ struct page *page;
+
+ page = list_first_entry_or_null(&cache->page_list, struct page, lru);
+ if (page) {
+ list_del(&page->lru);
+ cache->cnt--;
+ }
+
+ return page;
+}
+
+static struct page *alloc_dpamt_page(struct tdx_pamt_cache *cache)
+{
+ if (cache)
+ return tdx_alloc_page_pamt_cache(cache);
+
+ return alloc_page(GFP_KERNEL_ACCOUNT);
+}
+
+static int alloc_pamt_array(struct page **pamt_pages, struct tdx_pamt_cache *cache)
{
int i, j;
for (i = 0; i < TDX_DPAMT_ENTRY_PAGE_CNT; i++) {
- pamt_pages[i] = alloc_page(GFP_KERNEL_ACCOUNT);
+ pamt_pages[i] = alloc_dpamt_page(cache);
if (!pamt_pages[i])
goto err;
}
@@ -2047,7 +2068,7 @@ static u64 tdh_phymem_pamt_remove(kvm_pfn_t pfn, struct page **pamt_pages)
static DEFINE_SPINLOCK(pamt_lock);
/* Bump PAMT refcount for the given page and allocate PAMT memory if needed */
-static int tdx_pamt_get(kvm_pfn_t pfn)
+int tdx_pamt_get(kvm_pfn_t pfn, struct tdx_pamt_cache *cache)
{
struct page *pamt_pages[TDX_DPAMT_ENTRY_PAGE_CNT];
atomic_t *pamt_refcount;
@@ -2066,7 +2087,7 @@ static int tdx_pamt_get(kvm_pfn_t pfn)
if (atomic_inc_not_zero(pamt_refcount))
return 0;
- ret = alloc_pamt_array(pamt_pages);
+ ret = alloc_pamt_array(pamt_pages, cache);
if (ret)
return ret;
@@ -2106,12 +2127,13 @@ static int tdx_pamt_get(kvm_pfn_t pfn)
free_pamt_array(pamt_pages);
return ret;
}
+EXPORT_SYMBOL_FOR_KVM(tdx_pamt_get);
/*
* Drop PAMT refcount for the given page and free PAMT memory if it is no
* longer needed.
*/
-static void tdx_pamt_put(kvm_pfn_t pfn)
+void tdx_pamt_put(kvm_pfn_t pfn)
{
struct page *pamt_pages[TDX_DPAMT_ENTRY_PAGE_CNT] = {};
atomic_t *pamt_refcount;
@@ -2152,6 +2174,37 @@ static void tdx_pamt_put(kvm_pfn_t pfn)
free_pamt_array(pamt_pages);
}
+EXPORT_SYMBOL_FOR_KVM(tdx_pamt_put);
+
+void tdx_free_pamt_cache(struct tdx_pamt_cache *cache)
+{
+ struct page *page;
+
+ while ((page = tdx_alloc_page_pamt_cache(cache)))
+ __free_page(page);
+}
+EXPORT_SYMBOL_FOR_KVM(tdx_free_pamt_cache);
+
+int tdx_topup_pamt_cache(struct tdx_pamt_cache *cache, unsigned long npages)
+{
+ if (WARN_ON_ONCE(!tdx_supports_dynamic_pamt(&tdx_sysinfo)))
+ return 0;
+
+ npages *= TDX_DPAMT_ENTRY_PAGE_CNT;
+
+ while (cache->cnt < npages) {
+ struct page *page = alloc_page(GFP_KERNEL_ACCOUNT);
+
+ if (!page)
+ return -ENOMEM;
+
+ list_add(&page->lru, &cache->page_list);
+ cache->cnt++;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_FOR_KVM(tdx_topup_pamt_cache);
/*
* Return a page that can be gifted to the TDX-Module for use as a "control"
@@ -2167,7 +2220,7 @@ struct page *tdx_alloc_control_page(void)
if (!page)
return NULL;
- if (tdx_pamt_get(page_to_pfn(page))) {
+ if (tdx_pamt_get(page_to_pfn(page), NULL)) {
__free_page(page);
return NULL;
}
--
2.54.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox