Linux Confidential Computing Development
 help / color / mirror / Atom feed
From: "Edgecombe, Rick P" <rick.p.edgecombe@intel.com>
To: "linux-coco@lists.linux.dev" <linux-coco@lists.linux.dev>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"yilun.xu@linux.intel.com" <yilun.xu@linux.intel.com>,
	"x86@kernel.org" <x86@kernel.org>
Cc: "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"Li, Xiaoyao" <xiaoyao.li@intel.com>,
	"Hansen, Dave" <dave.hansen@intel.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
	"Hunter, Adrian" <adrian.hunter@intel.com>,
	"tony.lindgren@linux.intel.com" <tony.lindgren@linux.intel.com>,
	"kas@kernel.org" <kas@kernel.org>,
	"Xu, Yilun" <yilun.xu@intel.com>,
	"artem.bityutskiy@linux.intel.com"
	<artem.bityutskiy@linux.intel.com>,
	"nik.borisov@suse.com" <nik.borisov@suse.com>,
	"Mehta, Sohil" <sohil.mehta@intel.com>,
	"Duan, Zhenzhong" <zhenzhong.duan@intel.com>,
	"Gao, Chao" <chao.gao@intel.com>,
	"Fang, Peter" <peter.fang@intel.com>,
	"Maloor, Kishen" <kishen.maloor@intel.com>
Subject: Re: [PATCH v5 3/6] x86/virt/tdx: Add extra memory to TDX module for the extensions
Date: Wed, 18 Nov 2026 22:18:39 +0000	[thread overview]
Message-ID: <f9c89d6da82cd737bcb7d72eb6ca5f5bf121be2b.camel@intel.com> (raw)
In-Reply-To: <20261111-tdx-module-ext-v5-3-c74f764abc3d@linux.intel.com>

On Wed, 2026-11-11 at 11:48 +0800, Xu Yilun wrote:
> TDX module extensions need memory for their execution environment to
> serve SEAMCALL leafs. The TDX architecture implements the extensions
> in such a way that they use the memory outside of SEAM range, so the
> kernel should add the memory upfront at initialization time.
> 
> Introduce a new memory adding process backed by a new SEAMCALL leaf
> TDH.EXT.MEM.ADD. The kernel queries TDX module how much memory needed,
> allocates it, adds it to the module, and never gets it back. Note that
> the process requires Dynamic PAMT backing.

The process requires it, or the memory does?

>  A subsequent patch adds the
> support. This doesn't break bisectability cause the extensions can't be
                                            ^because?
> enabled yet without a feature that uses them.

Not clear whether the TDX module needs a new feature or the kernel does.

> 
> The TDX module accepts the memory in the form of an HPA array. This
> array is passed via a single 64-bit SEAMCALL leaf parameter, which
> encodes two values: the PFN of the container page holding the array, and
> the number of entries in the array. Create a helper to encode this
> format and name it after the TDX module term: HPA_LIST_INFO.
> 
> TDX module extensions consume tens of megabytes memory that will never
> be returned to the host. Use contiguous page allocation to isolate these
> large blocks entirely. This is a simple way to avoid permanent memory
> fragmentation: requiring the full size to be contiguous is more
> expensive than needed but should be good during the boot time. Print the
> allocation amount on TDX module extensions initialization for
> visibility.
> 
> The TDX module provides a metadata field "required_pages" to indicate
> the required memory size. This metadata is meaningful only after the
> add-on feature configuration. Don't read and cache the metadata in
> tdx_sysinfo. Instead read it right before allocating memory for the
> extensions, when it is guaranteed to be valid.
> 
> The original changelog was written by hand. AI was used to help correct
> grammatical errors and polish individual sentences for better
> readability.
> 
> Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
> ---
> An alternative solution is to use a loop that gives memory on a memory
> error code - TDX_EXT_MEMORY_POOL_REQUIRED, add one page per iteration
> until TDH.EXT.INIT succeeds. Something like:
> 
> 	do {
> 		ret = tdh_sys_init();
> 		if (ret == TDX_EXT_MEMORY_POOL_REQUIRED)
> 			tdh_ext_mem_add(); //single page
> 	} while (ret == TDX_EXT_MEMORY_POOL_REQUIRED);
> 
> This approach is slightly simpler as we don't have to query the module
> for the total memory, no memory pre-allocation or segmentation math.
> But allocating a single 4K page per iteration may cause permanent memory
> fragmentation.
> 
> v5:
>  - Add the DPAMT & bisectability note
>  - Shorten the "required_pages field" paragraph a little bit
>  - Collect Reviewed-by tags
>  - Better clarify the AI usage
> 
> v4:
>  - Rename memory_pool_required_pages to required_pages (Rick)
>  - Rename local var sysinfo_ext to ext
>  - Collect Reviewed-by tags
> 
> v3:
>  - Merge remaining pre-check operations from the previous patch (Rick)
>  - Don't save the metadata in tdx_sysinfo (Rick)
>  - Tweak the code comment for memory_pool_required_pages == 0 (Chao)
>  - Re-phrase the code comment for struct tdx_hpa_list (Tony)
>  - Put the error handling comments on top of memory add loop, to avoid
>    moving them around when DPAMT support is added in the following
>    patch.
>  - Print memory size before adding memory; it won't be freed afterward,
>    whether the operation succeeds or fails.
>  - Collect Reviewed-by tags
> 
> v2:
>  - Add code comments for container page allocation (Kiryl)
>  - Adjust comments/changelog for contiguous allocation of extensions
>    memory (Kiryl)
>  - s/PFN array/HPA array in commit log (Kiryl)
>  - State the alternative solution (Rick)
>  - Add code comment for struct tdx_hpa_list
> 
> v1:
>  - Fix return value for SEAMCALL wrappers (Chao)
>  - Print SEAMCALL error code for SEAMCALL wrappers (Xiaoyao)
>  - Rename local vars to make the ext memory adding loop clear (Rick)
>  - Remove input parameters for tdx_ext_mem_setup() (Kevin)
>  - Add a Macro for tdh_hpa_list size.
>  - Change the SEAMALL wrapper parameter type,
>    struct page *hpa_list => struct tdx_hpa_list *hpa_list
>  - changelog & code comments
> ---
>  arch/x86/include/asm/tdx.h                  |   1 +
>  arch/x86/include/asm/tdx_global_metadata.h  |   4 +
>  arch/x86/virt/vmx/tdx/tdx.h                 |   1 +
>  arch/x86/virt/vmx/tdx/tdx.c                 | 138
> ++++++++++++++++++++++++++++
>  arch/x86/virt/vmx/tdx/tdx_global_metadata.c |  14 +++
>  5 files changed, 158 insertions(+)
> 
> diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
> index 1fcfaed515fe..89e7628796b0 100644
> --- a/arch/x86/include/asm/tdx.h
> +++ b/arch/x86/include/asm/tdx.h
> @@ -37,6 +37,7 @@
>  #define TDX_FEATURES0_TD_PRESERVING	BIT_ULL(1)
>  #define TDX_FEATURES0_NO_RBP_MOD	BIT_ULL(18)
>  #define TDX_FEATURES0_DYNAMIC_PAMT	BIT_ULL(36)
> +#define TDX_FEATURES0_EXT		BIT_ULL(39)
>  
>  #ifndef __ASSEMBLER__
>  
> diff --git a/arch/x86/include/asm/tdx_global_metadata.h
> b/arch/x86/include/asm/tdx_global_metadata.h
> index 8a3cc1a2a41e..f18e63b671c2 100644
> --- a/arch/x86/include/asm/tdx_global_metadata.h
> +++ b/arch/x86/include/asm/tdx_global_metadata.h
> @@ -47,6 +47,10 @@ struct tdx_sys_info_handoff {
>  	u16 module_hv;
>  };
>  
> +struct tdx_sys_info_ext {
> +	u32 required_pages;
> +};
> +
>  struct tdx_sys_info {
>  	struct tdx_sys_info_version version;
>  	struct tdx_sys_info_features features;
> diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
> index db209541d3cd..2adc5d31d258 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.h
> +++ b/arch/x86/virt/vmx/tdx/tdx.h
> @@ -50,6 +50,7 @@
>  #define TDH_SYS_UPDATE			53
>  #define TDH_PHYMEM_PAMT_ADD		58
>  #define TDH_PHYMEM_PAMT_REMOVE		59
> +#define TDH_EXT_MEM_ADD			61
>  #define TDH_SYS_DISABLE			69
>  
>  /* TDX page types */
> diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> index 3bfe78b4b538..865d8089d343 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.c
> +++ b/arch/x86/virt/vmx/tdx/tdx.c
> @@ -1210,6 +1210,140 @@ static __init int init_tdmrs(struct tdmr_info_list
> *tdmr_list)
>  	return 0;
>  }
>  
> +#define TDX_HPA_LIST_MAX_NR_PAGES	(PAGE_SIZE / sizeof(u64))
> +
> +/*
> + * TDX module in-memory ABI to specify the memory being added to the TDX
> + * module. An array of HPAs, each element points to a 4K page.
> + */
> +struct tdx_hpa_list {
> +	u64 phys[TDX_HPA_LIST_MAX_NR_PAGES];
> +};
> +
> +static_assert(sizeof(struct tdx_hpa_list) == PAGE_SIZE);
> +
> +#define HPA_LIST_INFO_FIRST_ENTRY	GENMASK_U64(11, 3)
> +#define HPA_LIST_INFO_PFN		GENMASK_U64(51, 12)
> +#define HPA_LIST_INFO_LAST_ENTRY	GENMASK_U64(63, 55)
> +
> +static __init u64 to_hpa_list_info(struct tdx_hpa_list *hpa_list,
> +				   unsigned int nr_pages)
> +{
> +	return FIELD_PREP(HPA_LIST_INFO_FIRST_ENTRY, 0) |
> +	       FIELD_PREP(HPA_LIST_INFO_PFN, PFN_DOWN(__pa(hpa_list))) |
> +	       FIELD_PREP(HPA_LIST_INFO_LAST_ENTRY, nr_pages - 1);
> +}
> +
> +static __init int tdx_ext_mem_add(struct tdx_hpa_list *hpa_list,
> +				  unsigned int nr_pages)
> +{
> +	struct tdx_module_args args = {
> +		.rcx = to_hpa_list_info(hpa_list, nr_pages),
> +	};
> +	u64 ret;
> +
> +	do {
> +		/*
> +		 * The TDX module overwrites RCX to track progress when this
> +		 * SEAMCALL leaf is interrupted. Use seamcall_ret() to save
> and
> +		 * pass the updated value back on retry.
> +		 */
> +		ret = seamcall_ret(TDH_EXT_MEM_ADD, &args);
> +	} while (ret == TDX_INTERRUPTED_RESUMABLE);
> +
> +	if (ret != TDX_SUCCESS) {
> +		pr_err("TDH.EXT.MEM.ADD failed: 0x%016llx\n", ret);
> +		return -EIO;
> +	}
> +
> +	return 0;
> +}
> +
> +static __init int tdx_ext_mem_setup(void)
> +{
> +	struct tdx_hpa_list *hpa_list;
> +	struct tdx_sys_info_ext ext;
> +	unsigned int added_pages;
> +	struct page *page;
> +	int ret;
> +
> +	ret = get_tdx_sys_info_ext(&ext);
> +	if (ret)
> +		return ret;
> +
> +	/*
> +	 * Skip the memory setup if no memory is required. This may happen
> when
> +	 * no add-on features requiring TDX module extensions are configured
> +	 * via TDH.SYS.CONFIG.
> +	 */
> +	if (!ext.required_pages)
> +		return 0;
> +
> +	/*
> +	 * Allocate the container page for the HPA_LIST. tdx_hpa_list is
> +	 * guaranteed to be page-sized by static_assert(), so kzalloc()
> +	 * guarantees the page alignment.
> +	 */
> +	hpa_list = kzalloc_obj(*hpa_list);
> +	if (!hpa_list)
> +		return -ENOMEM;
> +
> +	/*
> +	 * Memory for TDX module extensions is never reclaimed and can be
> tens
> +	 * of megabytes. Allocating a physically contiguous chunk is a simple
> +	 * way to avoid permanent memory fragmentation: requiring the full
> size
> +	 * to be contiguous is more expensive than needed but should be good
> +	 * during the boot time.
> +	 */
> +	page = alloc_contig_pages(ext.required_pages, GFP_KERNEL,
> +				  numa_mem_id(), &node_online_map);
> +	if (!page) {
> +		ret = -ENOMEM;
> +		goto out_free_hpa_list;
> +	}
> +
> +	/* Print the amount so users know the cost. */
> +	pr_info("%lu KB allocated for TDX module extensions\n",
> +		ext.required_pages * PAGE_SIZE / 1024);

I wonder if this line is absolutely required for a minimal set?

> +
> +	/*
> +	 * Following TDX operations shouldn't fail, and if they do, things
> are
> +	 * broken enough that complex error handling isn't worth it.
> +	 * Intentionally leak all pages on failure, including un-added pages.
> +	 */
> +	added_pages = 0;
> +	while (added_pages < ext.required_pages) {
> +		unsigned int chunk_pages = min(ext.required_pages -
> added_pages,
> +					       TDX_HPA_LIST_MAX_NR_PAGES);
> +		struct page *chunk = page + added_pages;
> +		unsigned int i;
> +
> +		for (i = 0; i < chunk_pages; i++)
> +			hpa_list->phys[i] = page_to_phys(chunk + i);
> +
> +		ret = tdx_ext_mem_add(hpa_list, chunk_pages);
> +		if (ret) {
> +			WARN(1, "TDX module rejected memory for extensions,
> stranded all pages\n");
> +			break;
> +		}
> +
> +		added_pages += chunk_pages;
> +	}
> +
> +out_free_hpa_list:
> +	kfree(hpa_list);
> +
> +	return ret;
> +}
> +
> +static __init int init_tdx_module_extensions(void)
> +{
> +	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
> +		return 0;
> +
> +	return tdx_ext_mem_setup();
> +}
> +
>  static __init int init_tdx_module(void)
>  {
>  	int ret;
> @@ -1268,6 +1402,10 @@ static __init int init_tdx_module(void)
>  	if (ret)
>  		goto err_reset_pamts;
>  
> +	ret = init_tdx_module_extensions();
> +	if (ret)
> +		goto err_reset_pamts;
> +
>  	pr_info("%lu KB allocated for PAMT\n",
> tdmrs_count_pamt_kb(&tdx_tdmr_list));
>  
>  out_put_tdxmem:
> diff --git a/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
> b/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
> index 98ebf17aab1c..9ab4d021017f 100644
> --- a/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
> +++ b/arch/x86/virt/vmx/tdx/tdx_global_metadata.c
> @@ -125,6 +125,20 @@ static int get_tdx_sys_info_handoff(struct
> tdx_sys_info_handoff *sysinfo_handoff
>  	return 0;
>  }
>  
> +static __init int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)
> +{
> +	int ret;
> +	u64 val;
> +
> +	ret = read_sys_metadata_field(0x3100000200000000, &val);
> +	if (ret)
> +		return ret;
> +
> +	sysinfo_ext->required_pages = val;
> +
> +	return 0;
> +}
> +
>  static __init int get_tdx_sys_info(struct tdx_sys_info *sysinfo)
>  {
>  	int ret = 0;
> 


  parent reply	other threads:[~2026-11-18 22:18 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-11  3:48 [PATCH v5 0/6] Enable TDX module extensions Xu Yilun
2026-11-11  3:48 ` [PATCH v5 1/6] x86/virt/tdx: Move TDH.SYS.CONFIG operations into a wrapper Xu Yilun
2026-11-13 15:06   ` Nikolay Borisov
2026-11-18 18:51   ` Edgecombe, Rick P
2026-11-11  3:48 ` [PATCH v5 2/6] x86/virt/tdx: Configure add-on features on TDX module init Xu Yilun
2026-11-11  3:48 ` [PATCH v5 3/6] x86/virt/tdx: Add extra memory to TDX module for the extensions Xu Yilun
2026-11-17  3:06   ` Alexey Kardashevskiy
2026-11-18  2:32     ` Xu Yilun
2026-11-18 22:18   ` Edgecombe, Rick P [this message]
2026-11-11  3:48 ` [PATCH v5 4/6] x86/virt/tdx: Support DPAMT when adding memory " Xu Yilun
2026-11-11  3:48 ` [PATCH v5 5/6] x86/virt/tdx: Make TDX module initialize " Xu Yilun
2026-11-13 16:13   ` Nikolay Borisov
2026-11-18 22:44   ` Edgecombe, Rick P
2026-11-11  3:48 ` [PATCH v5 6/6] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update Xu Yilun

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f9c89d6da82cd737bcb7d72eb6ca5f5bf121be2b.camel@intel.com \
    --to=rick.p.edgecombe@intel.com \
    --cc=adrian.hunter@intel.com \
    --cc=artem.bityutskiy@linux.intel.com \
    --cc=baolu.lu@linux.intel.com \
    --cc=chao.gao@intel.com \
    --cc=dave.hansen@intel.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=kas@kernel.org \
    --cc=kishen.maloor@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-coco@lists.linux.dev \
    --cc=linux-kernel@vger.kernel.org \
    --cc=nik.borisov@suse.com \
    --cc=peter.fang@intel.com \
    --cc=sohil.mehta@intel.com \
    --cc=tony.lindgren@linux.intel.com \
    --cc=x86@kernel.org \
    --cc=xiaoyao.li@intel.com \
    --cc=yilun.xu@intel.com \
    --cc=yilun.xu@linux.intel.com \
    --cc=zhenzhong.duan@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox