* [PATCH v3 2/5] x86/virt/tdx: Pull kexec cache flush logic into arch/x86
From: Vishal Verma @ 2026-04-02 6:32 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, Vishal Verma, Chao Gao, Kai Huang
In-Reply-To: <20260402-fuller_tdx_kexec_support-v3-0-34438d7094bf@intel.com>
From: Rick Edgecombe <rick.p.edgecombe@intel.com>
KVM tries to take care of some required cache flushing earlier in the
kexec path in order to be kind to some long standing races that can occur
later in the operation. Until recently, VMXOFF was handled within KVM.
Since VMX being enabled is required to make a SEAMCALL, it had the best
per-cpu scoped operation to plug the flushing into. So it is kicked off
from there.
This early kexec cache flushing in KVM happens via a syscore shutdown
callback. Now that VMX enablement control has moved to arch/x86, which has
grown its own syscore shutdown callback, it no longer make sense for it to
live in KVM. It fits better with the TDX enablement managing code.
In addition, future changes will add a SEAMCALL that happens immediately
before VMXOFF, which means the cache flush in KVM will be too late to
flush the cache before the last SEAMCALL. So move it to the newly added TDX
arch/x86 syscore shutdown handler.
Since tdx_cpu_flush_cache_for_kexec() is no longer needed by KVM, make it
static and remove the export. Since it is also not part of an operation
spread across disparate components, remove the redundant comments and
verbose naming.
In the existing KVM based code, CPU offline also funnels through
tdx_cpu_flush_cache_for_kexec(). Add an explicit WBINVD in
tdx_offline_cpu() as well, even though it may be redundant with WBINVD
done elsewhere during CPU offline (e.g. hlt_play_dead()). This avoids
relying on fragile code ordering for cache coherency safety.
[Vishal: add explicit WBINVD in tdx_offline_cpu()]
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
Reviewed-by: Chao Gao <chao.gao@intel.com>
Acked-by: Kai Huang <kai.huang@intel.com>
Acked-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Acked-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/tdx.h | 6 ------
arch/x86/kvm/vmx/tdx.c | 10 ----------
arch/x86/virt/vmx/tdx/tdx.c | 46 ++++++++++++++++++++++++++-------------------
3 files changed, 27 insertions(+), 35 deletions(-)
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index a149740b24e8..bf83a974a0d5 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -226,11 +226,5 @@ static inline const char *tdx_dump_mce_info(struct mce *m) { return NULL; }
static inline const struct tdx_sys_info *tdx_get_sysinfo(void) { return NULL; }
#endif /* CONFIG_INTEL_TDX_HOST */
-#ifdef CONFIG_KEXEC_CORE
-void tdx_cpu_flush_cache_for_kexec(void);
-#else
-static inline void tdx_cpu_flush_cache_for_kexec(void) { }
-#endif
-
#endif /* !__ASSEMBLER__ */
#endif /* _ASM_X86_TDX_H */
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index b7264b533feb..50a5cfdbd33e 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -440,16 +440,6 @@ void tdx_disable_virtualization_cpu(void)
tdx_flush_vp(&arg);
}
local_irq_restore(flags);
-
- /*
- * Flush cache now if kexec is possible: this is necessary to avoid
- * having dirty private memory cachelines when the new kernel boots,
- * but WBINVD is a relatively expensive operation and doing it during
- * kexec can exacerbate races in native_stop_other_cpus(). Do it
- * now, since this is a safe moment and there is going to be no more
- * TDX activity on this CPU from this point on.
- */
- tdx_cpu_flush_cache_for_kexec();
}
#define TDX_SEAMCALL_RETRIES 10000
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index cb9b3210ab71..1b2d854ba664 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -184,6 +184,17 @@ static int tdx_online_cpu(unsigned int cpu)
return ret;
}
+static void tdx_cpu_flush_cache(void)
+{
+ lockdep_assert_preemption_disabled();
+
+ if (!this_cpu_read(cache_state_incoherent))
+ return;
+
+ wbinvd();
+ this_cpu_write(cache_state_incoherent, false);
+}
+
static int tdx_offline_cpu(unsigned int cpu)
{
int i;
@@ -220,12 +231,28 @@ static int tdx_offline_cpu(unsigned int cpu)
return -EBUSY;
done:
+ /*
+ * Flush cache on the CPU going offline to ensure no dirty
+ * cachelines of TDX private memory remain. This may be
+ * redundant with WBINVD done elsewhere during CPU offline
+ * (e.g. hlt_play_dead()), but do it explicitly for safety.
+ */
+ tdx_cpu_flush_cache();
x86_virt_put_ref(X86_FEATURE_VMX);
return 0;
}
static void tdx_shutdown_cpu(void *ign)
{
+ /*
+ * Flush cache in preparation for kexec - this is necessary to avoid
+ * having dirty private memory cachelines when the new kernel boots,
+ * but WBINVD is a relatively expensive operation and doing it during
+ * kexec can exacerbate races in native_stop_other_cpus(). Do it
+ * now, since this is a safe moment and there is going to be no more
+ * TDX activity on this CPU from this point on.
+ */
+ tdx_cpu_flush_cache();
x86_virt_put_ref(X86_FEATURE_VMX);
}
@@ -1920,22 +1947,3 @@ u64 tdh_phymem_page_wbinvd_hkid(u64 hkid, struct page *page)
return seamcall(TDH_PHYMEM_PAGE_WBINVD, &args);
}
EXPORT_SYMBOL_FOR_KVM(tdh_phymem_page_wbinvd_hkid);
-
-#ifdef CONFIG_KEXEC_CORE
-void tdx_cpu_flush_cache_for_kexec(void)
-{
- lockdep_assert_preemption_disabled();
-
- if (!this_cpu_read(cache_state_incoherent))
- return;
-
- /*
- * Private memory cachelines need to be clean at the time of
- * kexec. Write them back now, as the caller promises that
- * there should be no more SEAMCALLs on this CPU.
- */
- wbinvd();
- this_cpu_write(cache_state_incoherent, false);
-}
-EXPORT_SYMBOL_FOR_KVM(tdx_cpu_flush_cache_for_kexec);
-#endif
--
2.53.0
^ permalink raw reply related
* [PATCH v3 1/5] x86/tdx: Move TDX architectural error codes into <asm/shared/tdx_errno.h>
From: Vishal Verma @ 2026-04-02 6:32 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, Vishal Verma, Chao Gao,
Kiryl Shutsemau, Kiryl Shutsemau
In-Reply-To: <20260402-fuller_tdx_kexec_support-v3-0-34438d7094bf@intel.com>
From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Today there are two separate locations where TDX error codes are defined:
arch/x86/include/asm/tdx.h
arch/x86/kvm/vmx/tdx_errno.h
They have some overlap that is already defined similarly. Reduce the
duplication by unifying the architectural error codes at:
asm/shared/tdx_errno.h
...and update the headers that contained the duplicated definitions to
include the new unified header.
"asm/shared" is used for sharing TDX code between the early compressed
code and the normal kernel code. While the compressed code for the guest
doesn't use these error code header definitions today, it does make the
types of calls that return the values they define. So place the defines in
"shared" location so that it can, but leave such cleanups for future
changes.
[Rick: enhance log]
[Vishal: reduce to a simple move of architectural defines only]
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
Reviewed-by: Chao Gao <chao.gao@intel.com>
---
arch/x86/include/asm/shared/tdx.h | 1 +
arch/x86/{kvm/vmx => include/asm/shared}/tdx_errno.h | 7 +++----
arch/x86/kvm/vmx/tdx.h | 1 -
3 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/arch/x86/include/asm/shared/tdx.h b/arch/x86/include/asm/shared/tdx.h
index 8bc074c8d7c6..6a1646fc2b2f 100644
--- a/arch/x86/include/asm/shared/tdx.h
+++ b/arch/x86/include/asm/shared/tdx.h
@@ -4,6 +4,7 @@
#include <linux/bits.h>
#include <linux/types.h>
+#include <asm/shared/tdx_errno.h>
#define TDX_HYPERCALL_STANDARD 0
diff --git a/arch/x86/kvm/vmx/tdx_errno.h b/arch/x86/include/asm/shared/tdx_errno.h
similarity index 92%
rename from arch/x86/kvm/vmx/tdx_errno.h
rename to arch/x86/include/asm/shared/tdx_errno.h
index 6ff4672c4181..3c1e8ce716e3 100644
--- a/arch/x86/kvm/vmx/tdx_errno.h
+++ b/arch/x86/include/asm/shared/tdx_errno.h
@@ -1,8 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0 */
/* architectural status code for SEAMCALL */
-
-#ifndef __KVM_X86_TDX_ERRNO_H
-#define __KVM_X86_TDX_ERRNO_H
+#ifndef _ASM_X86_SHARED_TDX_ERRNO_H
+#define _ASM_X86_SHARED_TDX_ERRNO_H
#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL
@@ -37,4 +36,4 @@
#define TDX_OPERAND_ID_SEPT 0x92
#define TDX_OPERAND_ID_TD_EPOCH 0xa9
-#endif /* __KVM_X86_TDX_ERRNO_H */
+#endif /* _ASM_X86_SHARED_TDX_ERRNO_H */
diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h
index b5cd2ffb303e..ac8323a68b16 100644
--- a/arch/x86/kvm/vmx/tdx.h
+++ b/arch/x86/kvm/vmx/tdx.h
@@ -3,7 +3,6 @@
#define __KVM_X86_VMX_TDX_H
#include "tdx_arch.h"
-#include "tdx_errno.h"
#ifdef CONFIG_KVM_INTEL_TDX
#include "common.h"
--
2.53.0
^ permalink raw reply related
* [PATCH v3 0/5] Fuller TDX kexec support
From: Vishal Verma @ 2026-04-02 6:32 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Kiryl Shutsemau, Rick Edgecombe,
Sean Christopherson, Paolo Bonzini
Cc: linux-kernel, linux-coco, kvm, Vishal Verma, Chao Gao, Kai Huang,
Kiryl Shutsemau
Hi,
This series adds a couple of cool things -
1. Allow kexec and kdump on systems with the partial write errata
2. Allow using TDX in the second (kexec'ed) kernel
It has been waiting for VMXON refactor to land because the implementation
is much cleaner on top of that.
KVM folks, just a few deletions on your side and the long discussed moving
of tdx_errno.h. Tip folks and reviewers, the changes here are pretty small.
Optimistically, I'm hoping we can iterate this quickly and see it off the
list in the next few weeks.
Background
==========
Some early TDX-capable platforms have an erratum where a partial write
to TDX private memory can cause a machine check on a subsequent read.
Currently, kexec and kdump are disabled on these platforms because the
new (or kdump) kernel may trip over these, causing a machine check.
Future TDX modules will support TDH.SYS.DISABLE SEAMCALL, which disables
the TDX module and reclaims all memory resources allocated to TDX, and
cleans up any poison. After this SEAMCALL succeeds, the new kernel
can also re-initialize the TDX module from scratch via the normal bring-up
sequence.
It is probably worth mentioning that this is a different kind of cleanup
than the WBINVD stuff that was the cause of all the fuss in the earlier
kexec enabling. The WBINVD is flushing private keyid cachelines so they
are not later written back over the new kernels memory. It needs to happen
after the last SEAMCALL that might have produced them. So this new
SEAMCALL is for something else, but also needs to be slotted with respect
to WBINVD.
Implementation
==============
The series adds:
1. A pre-requisite patch to move TDX error code definitions to a
shared location so that TDX_INTERRUPTED_RESUMABLE etc. are
accessible from arch/x86/virt/vmx/tdx/. This comes from the Dynamic
PAMT series [0], but is also needed by some other series, and can
benefit them all from an early merge. This has now also been reduced
to a simple header move, since the new additions in the original
patch aren't needed here.
2. A preparatory patch to move some straggling stuff into arch/x86 in the
wake of the VMXON series. As noted in the discussion in v2, this
may need coordination with Kai's patch [1] as they will cause a
conflict depending on which is merged first.
3. A tdx_sys_disable() helper that wraps calls TDH.SYS.DISABLE with a
retry loop to handle TDX_INTERRUPTED_RESUMABLE.
4. Integration into the kexec path: Remove the check for partial write
errata platforms as this is addressed by the SEAMCALL clearing any
poisoned memory locations. Call tdx_sys_disable() in tdx_shutdown
which is called via syscore ops in the kexec path. Call
tdx_sys_disable() in native_machine_crash_shutdown() to cover the
crash (kdump) path.
5. A patch to update (remove) the kexec section in TDX docs.
Testing
=======
The implementation is based on the draft TDX module spec available at
[2], and was smoke tested with an engineering build of the TDX module
that supports the new SEAMCALL. The new kernel was able to initialize
the TDX module successfully:
kvm: exiting hardware virtualization
kexec_core: Starting new kernel
Linux version 7.0.0-rc2-g0077f702b21c...
...
virt/tdx: 1034220 KB allocated for PAMT
virt/tdx: TDX-Module initialized
All the other TDX CI tests pass, and some other scenarios that were
manually tested and also pass or behave as expected:
- Running on a completely non-TDX system
- Running on a TDX capable system with an old module
- Running on a TDX capable system where the module hasn't been
initialized
During development further testing was done by mocking up the new
SEAMCALL to introduce delays and exercise the retry loops, combined with
kexec, kdump, reboot and shutdown flows.
Base
====
This series is based on the vmxon branch Sean pushed to kvm_x86,
kvm-x86-vmxon-2026.03.05.
[0]: https://lore.kernel.org/kvm/20260129011517.3545883-11-seanjc@google.com/
[1]: https://lore.kernel.org/lkml/20260312100009.924136-1-kai.huang@intel.com/
[2]: https://cdrdv2.intel.com/v1/dl/getContent/871617
---
Changes in v3:
- Various: Collect tags (Kai, Chao, Kiryl, Sean)
- Patch 1: Collapse to a simple header move, drop new defines, and
non-architectural software defines. (Sean)
- Patch 2: Add WBINVD in the tdx_offline_cpu() path (Sean, Dave)
- Add a paragraph to the seamcall wrapper commit message talking about
printed errors for seamcall failures (Rick)
- Patch 2: Reword comment in tdx_shutdown_cpu (Chao)
- Patch 3: Reword comment about the TDX_SYS_BUSY error case (Chao)
- Patch 3: Formatting fixes in comments (Kiryl)
- Link to v2: https://patch.msgid.link/20260323-fuller_tdx_kexec_support-v2-0-87a36409e051@intel.com
Changes in v2:
- Use patch 1 from the DPAMT series with other feedback (Kai)
- Fix commit message typo (s/adjust_/adjust /)
- In patch 2, drop "too late to be helpful" in favor of something more
explicit (Kai)
- Fix commit message typo in patch 2 (s/both/bother/)
- In patch 2, add a bit about dropping the TDX specific WBINVD (Kai)
- Reword some commit logs to use the imperative mood (Chao)
- Kai raised offline that TDH.SYS.DISABLE can return TDX_SYS_BUSY too.
In theory this could happen if another SEAMCALL happens concurrently,
however that contention should be short lived. Update the loop to
continue on a TDX_SYS_BUSY error code too. (Kai)
- Patch 3: Add a print for SEAMCALL errors reported by the TDX module
(excluding SW errors like #UD and #GP) (Kiryl)
- Patch 3: Add a sentence to the log about skipping enumeration for the
new SEAMCALL (Kiryl)
- Adjust the patch 4 subject (Chao)
- Add a new patch to update the docs (Chao)
- Smoke test with TDX module engineering build with the new SEAMCALL.
Kiryl Shutsemau (1):
x86/tdx: Move TDX architectural error codes into <asm/shared/tdx_errno.h>
Rick Edgecombe (2):
x86/virt/tdx: Pull kexec cache flush logic into arch/x86
x86/virt/tdx: Remove kexec docs
Vishal Verma (2):
x86/virt/tdx: Add SEAMCALL wrapper for TDH.SYS.DISABLE
x86/tdx: Disable the TDX module during kexec and kdump
Documentation/arch/x86/tdx.rst | 7 -------
arch/x86/include/asm/shared/tdx.h | 1 +
arch/x86/{kvm/vmx => include/asm/shared}/tdx_errno.h | 8 +++----
arch/x86/include/asm/tdx.h | 9 +++-----
arch/x86/kvm/vmx/tdx.h | 1 -
arch/x86/virt/vmx/tdx/tdx.h | 1 +
arch/x86/kernel/crash.c | 2 ++
arch/x86/kernel/machine_kexec_64.c | 16 --------------
arch/x86/kvm/vmx/tdx.c | 10 ---------
arch/x86/virt/vmx/tdx/tdx.c | 64 +++++++++++++++++++++++++++++++++++++++++++++-----------
10 files changed, 63 insertions(+), 56 deletions(-)
--
2.53.0
---
Kiryl Shutsemau (1):
x86/tdx: Move TDX architectural error codes into <asm/shared/tdx_errno.h>
Rick Edgecombe (2):
x86/virt/tdx: Pull kexec cache flush logic into arch/x86
x86/virt/tdx: Remove kexec docs
Vishal Verma (2):
x86/virt/tdx: Add SEAMCALL wrapper for TDH.SYS.DISABLE
x86/tdx: Disable the TDX module during kexec and kdump
Documentation/arch/x86/tdx.rst | 7 ---
arch/x86/include/asm/shared/tdx.h | 1 +
.../{kvm/vmx => include/asm/shared}/tdx_errno.h | 8 +--
arch/x86/include/asm/tdx.h | 9 +--
arch/x86/kvm/vmx/tdx.h | 1 -
arch/x86/virt/vmx/tdx/tdx.h | 1 +
arch/x86/kernel/crash.c | 2 +
arch/x86/kernel/machine_kexec_64.c | 16 ------
arch/x86/kvm/vmx/tdx.c | 10 ----
arch/x86/virt/vmx/tdx/tdx.c | 64 ++++++++++++++++++----
10 files changed, 63 insertions(+), 56 deletions(-)
---
base-commit: f630de1f8d70d7e29e12bc25dc63f9c5f771dc59
change-id: 20260317-fuller_tdx_kexec_support-bc79694678be
Best regards,
--
Vishal Verma <vishal.l.verma@intel.com>
^ permalink raw reply
* Re: [PATCH v5 0/2] dma-buf: heaps: system: add an option to allocate explicitly shared/decrypted memory
From: Marek Szyprowski @ 2026-04-02 5:35 UTC (permalink / raw)
To: Sumit Semwal, Jiri Pirko
Cc: dri-devel, linaro-mm-sig, iommu, linux-media, benjamin.gaignard,
Brian.Starkey, jstultz, tjmercier, christian.koenig, robin.murphy,
jgg, leon, sean.anderson, ptesarik, catalin.marinas, aneesh.kumar,
suzuki.poulose, steven.price, thomas.lendacky, john.allen,
ashish.kalra, suravee.suthikulpanit, linux-coco
In-Reply-To: <CAO_48GEUXpcFBiyJAMgTcGPSq56-mZ0qnO3FrFRM2LoGd8W6HA@mail.gmail.com>
On 02.04.2026 06:41, Sumit Semwal wrote:
> On Thu, 26 Mar 2026 at 00:53, Jiri Pirko <jiri@resnulli.us> wrote:
>> From: Jiri Pirko <jiri@nvidia.com>
>>
>> Confidential computing (CoCo) VMs/guests, such as AMD SEV and Intel TDX,
>> run with private/encrypted memory which creates a challenge
>> for devices that do not support DMA to it (no TDISP support).
>>
>> For kernel-only DMA operations, swiotlb bounce buffering provides a
>> transparent solution by copying data through shared memory.
>> However, the only way to get this memory into userspace is via the DMA
>> API's dma_alloc_pages()/dma_mmap_pages() type interfaces which limits
>> the use of the memory to a single DMA device, and is incompatible with
>> pin_user_pages().
>>
>> These limitations are particularly problematic for the RDMA subsystem
>> which makes heavy use of pin_user_pages() and expects flexible memory
>> usage between many different DMA devices.
>>
>> This patch series enables userspace to explicitly request shared
>> (decrypted) memory allocations from new dma-buf system_cc_shared heap.
>> Userspace can mmap this memory and pass the dma-buf fd to other
>> existing importers such as RDMA or DRM devices to access the
>> memory. The DMA API is improved to allow the dma heap exporter to DMA
>> map the shared memory to each importing device.
> Thank you for the patch series, it looks good to me.
>
> Marek, if you are ok, please could you take it through your tree, with my
> Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
I've applied both patches to dma-mapping-for-next. Thanks!
Best regards
--
Marek Szyprowski, PhD
Samsung R&D Institute Poland
^ permalink raw reply
* Re: [PATCH v5 0/2] dma-buf: heaps: system: add an option to allocate explicitly shared/decrypted memory
From: Sumit Semwal @ 2026-04-02 4:41 UTC (permalink / raw)
To: Jiri Pirko
Cc: dri-devel, linaro-mm-sig, iommu, linux-media, benjamin.gaignard,
Brian.Starkey, jstultz, tjmercier, christian.koenig, m.szyprowski,
robin.murphy, jgg, leon, sean.anderson, ptesarik, catalin.marinas,
aneesh.kumar, suzuki.poulose, steven.price, thomas.lendacky,
john.allen, ashish.kalra, suravee.suthikulpanit, linux-coco
In-Reply-To: <20260325192352.437608-1-jiri@resnulli.us>
Hello Jiri,
On Thu, 26 Mar 2026 at 00:53, Jiri Pirko <jiri@resnulli.us> wrote:
>
> From: Jiri Pirko <jiri@nvidia.com>
>
> Confidential computing (CoCo) VMs/guests, such as AMD SEV and Intel TDX,
> run with private/encrypted memory which creates a challenge
> for devices that do not support DMA to it (no TDISP support).
>
> For kernel-only DMA operations, swiotlb bounce buffering provides a
> transparent solution by copying data through shared memory.
> However, the only way to get this memory into userspace is via the DMA
> API's dma_alloc_pages()/dma_mmap_pages() type interfaces which limits
> the use of the memory to a single DMA device, and is incompatible with
> pin_user_pages().
>
> These limitations are particularly problematic for the RDMA subsystem
> which makes heavy use of pin_user_pages() and expects flexible memory
> usage between many different DMA devices.
>
> This patch series enables userspace to explicitly request shared
> (decrypted) memory allocations from new dma-buf system_cc_shared heap.
> Userspace can mmap this memory and pass the dma-buf fd to other
> existing importers such as RDMA or DRM devices to access the
> memory. The DMA API is improved to allow the dma heap exporter to DMA
> map the shared memory to each importing device.
Thank you for the patch series, it looks good to me.
Marek, if you are ok, please could you take it through your tree, with my
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Best,
Sumit.
>
> Based on dma-mapping-for-next e7442a68cd1ee797b585f045d348781e9c0dde0d
>
> Jiri Pirko (2):
> dma-mapping: introduce DMA_ATTR_CC_SHARED for shared memory
> dma-buf: heaps: system: add system_cc_shared heap for explicitly
> shared memory
>
> drivers/dma-buf/heaps/system_heap.c | 103 ++++++++++++++++++++++++++--
> include/linux/dma-mapping.h | 10 +++
> include/trace/events/dma.h | 3 +-
> kernel/dma/direct.h | 14 +++-
> kernel/dma/mapping.c | 13 +++-
> 5 files changed, 132 insertions(+), 11 deletions(-)
>
> --
> 2.51.1
>
^ permalink raw reply
* Re: [PATCH v2 08/31] x86/virt/tdx: Configure TDX Module with optional TDX Connect feature
From: Huang, Kai @ 2026-04-02 1:06 UTC (permalink / raw)
To: Williams, Dan J, linux-pci@vger.kernel.org,
linux-coco@lists.linux.dev, Hansen, Dave, Edgecombe, Rick P,
yilun.xu@linux.intel.com, x86@kernel.org
Cc: Gao, Chao, Xu, Yilun, dave.hansen@linux.intel.com, kas@kernel.org,
baolu.lu@linux.intel.com, Li, Xiaoyao,
linux-kernel@vger.kernel.org, Verma, Vishal L, Jiang, Dave,
kvm@vger.kernel.org, Duan, Zhenzhong
In-Reply-To: <5183437e-bbe8-4453-9b03-9bef2e025c0a@intel.com>
On Wed, 2026-04-01 at 17:48 -0700, Dave Hansen wrote:
> On 4/1/26 17:40, Huang, Kai wrote:
> ...
> > They might not care about losing ~50M memory, though, but that's a different
> > story, and it could be more in the future.
>
> If anyone _does_ care about 50MB of memory, I expect they'll speak up.
> Until they do, could we please err on the side of least complexity?
Yeah agreed.
>
> I'm not even sure the Kconfig is worth it to be honest.
I am not sure either.
My main comment is to make the code more friendly to opt-in more features,
actually.
^ permalink raw reply
* Re: [PATCH v2 08/31] x86/virt/tdx: Configure TDX Module with optional TDX Connect feature
From: Dave Hansen @ 2026-04-02 0:48 UTC (permalink / raw)
To: Huang, Kai, Williams, Dan J, linux-pci@vger.kernel.org,
linux-coco@lists.linux.dev, Edgecombe, Rick P,
yilun.xu@linux.intel.com, x86@kernel.org
Cc: Gao, Chao, Xu, Yilun, dave.hansen@linux.intel.com,
baolu.lu@linux.intel.com, kas@kernel.org, Li, Xiaoyao,
linux-kernel@vger.kernel.org, Verma, Vishal L, Jiang, Dave,
kvm@vger.kernel.org, Duan, Zhenzhong
In-Reply-To: <847eb26878cb06afb47f4ecdb0bfe8ab7e432e8a.camel@intel.com>
On 4/1/26 17:40, Huang, Kai wrote:
...
> They might not care about losing ~50M memory, though, but that's a different
> story, and it could be more in the future.
If anyone _does_ care about 50MB of memory, I expect they'll speak up.
Until they do, could we please err on the side of least complexity?
I'm not even sure the Kconfig is worth it to be honest.
^ permalink raw reply
* Re: [PATCH v2 08/31] x86/virt/tdx: Configure TDX Module with optional TDX Connect feature
From: Huang, Kai @ 2026-04-02 0:40 UTC (permalink / raw)
To: Williams, Dan J, linux-pci@vger.kernel.org,
linux-coco@lists.linux.dev, Edgecombe, Rick P,
yilun.xu@linux.intel.com, x86@kernel.org
Cc: Gao, Chao, Xu, Yilun, dave.hansen@linux.intel.com,
baolu.lu@linux.intel.com, kas@kernel.org, Li, Xiaoyao,
linux-kernel@vger.kernel.org, Verma, Vishal L, Jiang, Dave,
kvm@vger.kernel.org, Duan, Zhenzhong
In-Reply-To: <8c4dfadaf73ed175a959ab16335a8d90266453b5.camel@intel.com>
On Wed, 2026-04-01 at 23:53 +0000, Edgecombe, Rick P wrote:
> On Wed, 2026-04-01 at 23:42 +0000, Huang, Kai wrote:
> > Maybe a more generic comment:
> >
> > I don't quite like hard-coding opt-in TDX_FEATURES0_TDXCONNECT inside
> > config_tdx_module(), especially currently we just unconditionally opt it in
> > if the module support this feature.
> >
> > Initializing TDX Connect (and other features via TDX Module Extensions)
> > consumes more memory. It would be better if we can choose to opt-in when
> > the kernel has enabled TDX Connect (or any other feature via TDX module
> > Extensions) in the Kconfig.
>
> Better how? TDX uses a lot of memory. There are many possible optimizations to
> reduce this. Why focus on this one? Do we think any TDX users would actually
> reconfigure there kernel for this reason?
>
> I mean, I don't actually know how much memory this is, but to me the reasoning
> doesn't seem in balance with the wider TDX situation.
In another patch, Yilun said:
For now, TDX Module Extensions consume relatively large amount of
memory (~50MB).
Distros tend to enable all I assume. I guess the CSPs tend to enable all as
well, but I am not sure whether CSPs can also choose to only enable basic
TDX w/o other features like runtime update, TDX Connect etc, depending on
how they want to "sell" TDX VMs.
E.g., I assume a TD without GPU passthrough could be cheaper the one which
has, right? Can the CSPs host such TDs on dedicated machine pools? Can
they choose to disable TDX Connect on these machines?
They might not care about losing ~50M memory, though, but that's a different
story, and it could be more in the future.
My thinking is, this series actually introduced a new "TDX_CONNNECT"
Kconfig, so why not only consume the memory when it's on?
At last, just my 2cents, I kinda overall don't agree we always assume
"everything will be on" but neglect avoiding unnecessary code/cost that is
not reachable when some option is off. That would defeat the purpose of
having Kconfig option.
^ permalink raw reply
* Re: [PATCH v2 05/31] x86/virt/tdx: Extend tdx_page_array to support IOMMU_MT
From: Huang, Kai @ 2026-04-02 0:05 UTC (permalink / raw)
To: Williams, Dan J, linux-pci@vger.kernel.org,
linux-coco@lists.linux.dev, yilun.xu@linux.intel.com,
x86@kernel.org
Cc: Gao, Chao, Edgecombe, Rick P, Xu, Yilun, Jiang, Dave,
dave.hansen@linux.intel.com, baolu.lu@linux.intel.com,
Duan, Zhenzhong, kas@kernel.org, Verma, Vishal L, Li, Xiaoyao,
kvm@vger.kernel.org, linux-kernel@vger.kernel.org
In-Reply-To: <20260327160132.2946114-6-yilun.xu@linux.intel.com>
On Sat, 2026-03-28 at 00:01 +0800, Xu Yilun wrote:
> IOMMU_MT is another TDX Module defined structure similar to HPA_ARRAY_T
> and HPA_LIST_INFO. The difference is it requires multi-order contiguous
> pages for some entries. It adds an additional NUM_PAGES field for every
> multi-order page entry.
>
> Add a dedicated allocation helper for IOMMU_MT. Fortunately put_page()
> works well for both single pages and multi-order folios, simplifying the
> cleanup logic for all allocation methods.
Well I guess you can have a 'free_fn' to free the pages you allocated via
'alloc_fn'? Will this simplify the code and at least keep tdx_page_array
implementation cleaner?
It's strange that you only have a 'alloc_fn' but doesn't have a 'free_fn'
anyway.
^ permalink raw reply
* Re: [PATCH v2 08/31] x86/virt/tdx: Configure TDX Module with optional TDX Connect feature
From: Edgecombe, Rick P @ 2026-04-01 23:53 UTC (permalink / raw)
To: Williams, Dan J, linux-pci@vger.kernel.org,
linux-coco@lists.linux.dev, Huang, Kai, yilun.xu@linux.intel.com,
x86@kernel.org
Cc: Gao, Chao, Xu, Yilun, Jiang, Dave, baolu.lu@linux.intel.com,
kas@kernel.org, dave.hansen@linux.intel.com, Li, Xiaoyao,
Verma, Vishal L, Duan, Zhenzhong, kvm@vger.kernel.org,
linux-kernel@vger.kernel.org
In-Reply-To: <6a28710b65d1a291213d28a8297477bf0287df7d.camel@intel.com>
On Wed, 2026-04-01 at 23:42 +0000, Huang, Kai wrote:
> Maybe a more generic comment:
>
> I don't quite like hard-coding opt-in TDX_FEATURES0_TDXCONNECT inside
> config_tdx_module(), especially currently we just unconditionally opt it in
> if the module support this feature.
>
> Initializing TDX Connect (and other features via TDX Module Extensions)
> consumes more memory. It would be better if we can choose to opt-in when
> the kernel has enabled TDX Connect (or any other feature via TDX module
> Extensions) in the Kconfig.
Better how? TDX uses a lot of memory. There are many possible optimizations to
reduce this. Why focus on this one? Do we think any TDX users would actually
reconfigure there kernel for this reason?
I mean, I don't actually know how much memory this is, but to me the reasoning
doesn't seem in balance with the wider TDX situation.
^ permalink raw reply
* Re: [PATCH v2 08/31] x86/virt/tdx: Configure TDX Module with optional TDX Connect feature
From: Huang, Kai @ 2026-04-01 23:42 UTC (permalink / raw)
To: Williams, Dan J, linux-pci@vger.kernel.org,
linux-coco@lists.linux.dev, yilun.xu@linux.intel.com,
x86@kernel.org
Cc: Gao, Chao, Edgecombe, Rick P, Xu, Yilun, Jiang, Dave,
dave.hansen@linux.intel.com, baolu.lu@linux.intel.com,
Duan, Zhenzhong, kas@kernel.org, Verma, Vishal L, Li, Xiaoyao,
kvm@vger.kernel.org, linux-kernel@vger.kernel.org
In-Reply-To: <20260327160132.2946114-9-yilun.xu@linux.intel.com>
> static int config_tdx_module(struct tdmr_info_list *tdmr_list, u64 global_keyid)
> {
> struct tdx_module_args args = {};
> + u64 seamcall_fn = TDH_SYS_CONFIG_V0;
> u64 *tdmr_pa_array;
> size_t array_sz;
> int i, ret;
> @@ -1377,7 +1378,15 @@ static int config_tdx_module(struct tdmr_info_list *tdmr_list, u64 global_keyid)
> args.rcx = __pa(tdmr_pa_array);
> args.rdx = tdmr_list->nr_consumed_tdmrs;
> args.r8 = global_keyid;
> - ret = seamcall_prerr(TDH_SYS_CONFIG, &args);
> +
> + if (tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_TDXCONNECT) {
> + args.r9 |= TDX_FEATURES0_TDXCONNECT;
> + args.r11 = ktime_get_real_seconds();
> + /* These parameters requires version >= 1 */
> + seamcall_fn = TDH_SYS_CONFIG;
> + }
> +
> + ret = seamcall_prerr(seamcall_fn, &args);
>
> /* Free the array as it is not required anymore. */
> kfree(tdmr_pa_array);
> @@ -1537,6 +1546,11 @@ static int init_tdx_module(void)
> if (ret)
> goto err_free_pamts;
>
> + /* configuration to tdx module may change tdx_sysinfo, update it */
> + ret = get_tdx_sys_info(&tdx_sysinfo);
> + if (ret)
> + goto err_reset_pamts;
> +
> /* Config the key of global KeyID on all packages */
> ret = config_global_keyid();
> if (ret)
Maybe a more generic comment:
I don't quite like hard-coding opt-in TDX_FEATURES0_TDXCONNECT inside
config_tdx_module(), especially currently we just unconditionally opt it in
if the module support this feature.
Initializing TDX Connect (and other features via TDX Module Extensions)
consumes more memory. It would be better if we can choose to opt-in when
the kernel has enabled TDX Connect (or any other feature via TDX module
Extensions) in the Kconfig.
Unfortunately we need to opt-in all these features together during module
initialization, so we cannot make tdx_enable() to accept the additional
features to enable, and in each in-kernel TDX user, call tdx_enable() with
the new feature that that TDX user concerns.
But I think it makes sense to have a dedicated place to calculate all opt-in
features. E.g., assuming we eventually are going to support TDX Connect and
live migration:
static u64 get_ext_features_tdx_connect(struct tdx_sys_info * sysinfo)
{
if (!IS_ENABLED(TDX_CONNECT))
return 0;
return sysinfo->features.tdx_features0 & TDX_FEATURES0_TDXCONNECT ?
TDX_FEATURES0_TDXCONNECT : 0;
}
static u64 get_ext_features_live_migration(struct tdx_sys_info *sysinfo)
{
u64 mig_features = TDX_FEATURES0_NRX | TDX_FEATURES0_NON_BLOCKING;
if (!IS_ENABLED(TDX_LIVE_MIGRATION))
return 0;
return sysinfo->features.tdx_features0 & mig_features;
}
static u64 calculate_ext_features(struct tdx_sys_info *sysinfo)
{
u64 ext_features = 0;
ext_features |= get_ext_features_tdx_connect(sysinfo);
ext_features |= get_ext_features_live_migration(sysinfo);
return ext_features;
}
int init_tdx_module()
{
u64 ext_features = calculate_ext_features(&tdx_sysinfo);
ret = config_tdx_module(&tdx_tdmr_list, &tdx_global_keyid,
ext_features);
/* do other initializations like TDH.SYS.KEY.CONFIG */
...
/*
* TDX Module Extension features must be initialized
* after TDH.SYS.KEY.CONFIG.
*/
if (ext_features)
ret = init_tdx_ext();
...
}
One nasty thing is per public spec R11 of TDH.SYS.CONFIG needs to be RTC if
TDX_CONNECT is on, so we still need some special handing in
config_tdx_module():
if (ext_features & TDX_RFEAURES0_TDXCONNECT)
args.r11 = ktime_get_real_seconds();
But I think this is acceptable.
^ permalink raw reply
* Re: [PATCH v2 06/31] x86/virt/tdx: Read global metadata for TDX Module Extensions/Connect
From: Huang, Kai @ 2026-04-01 21:36 UTC (permalink / raw)
To: Williams, Dan J, linux-pci@vger.kernel.org,
linux-coco@lists.linux.dev, yilun.xu@linux.intel.com,
x86@kernel.org
Cc: Gao, Chao, Edgecombe, Rick P, Xu, Yilun, Jiang, Dave,
dave.hansen@linux.intel.com, baolu.lu@linux.intel.com,
Duan, Zhenzhong, kas@kernel.org, Verma, Vishal L, Li, Xiaoyao,
kvm@vger.kernel.org, linux-kernel@vger.kernel.org
In-Reply-To: <20260327160132.2946114-7-yilun.xu@linux.intel.com>
On Sat, 2026-03-28 at 00:01 +0800, Xu Yilun wrote:
> Add reading of the global metadata for TDX Module Extensions & TDX
> Connect. Add them in a batch as TDX Connect is currently the only user
> of TDX Module Extensions and no way to initialize TDX Module Extensions
> without firstly enabling TDX Connect.
>
> TDX Module Extensions & TDX Connect are optional features enumerated by
> TDX_FEATURES0. Check the TDX_FEATURES0 before reading these metadata to
> avoid failing the whole TDX initialization.
Maybe it's better to split this patch into two, one to read generic "TDX
Module Extension" related global metadata, and the other to read TDX Connect
specific ones?
They are logically two separate things anyway. And there are other features
also need to enable TDX Module Extensions (e.g., NRX for migration), and we
can just reuse the generic metadata patch from this series.
^ permalink raw reply
* Re: [PATCH v2 2/5] x86/virt/tdx: Pull kexec cache flush logic into arch/x86
From: Dave Hansen @ 2026-04-01 18:30 UTC (permalink / raw)
To: Sean Christopherson, H. Peter Anvin
Cc: Rick P Edgecombe, Vishal L Verma, Kai Huang, bp@alien8.de,
x86@kernel.org, kas@kernel.org, mingo@redhat.com,
linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com,
tglx@kernel.org, pbonzini@redhat.com, linux-coco@lists.linux.dev,
kvm@vger.kernel.org
In-Reply-To: <ac1f9KntTv0SaktS@google.com>
On 4/1/26 11:12, Sean Christopherson wrote:
> Sorry, "costly" wasn't the right word. I know WBINVD super
> expensive, but unless someone cares deeply about the latency of
> offlining a CPU after its down TDX stuff, the "cost" is effectively
> zero.
I once increased the CPU online/offline latency once and got nastygrams
from folks. IIRC, I added a synchronize_rcu() which incurs way more
latency than WBINVD, but folks _do_ care about CPU online/offline
latency surprisingly.
In this case, though, I'm happy to add the WBINVD for simplicity and
wait for a possible repeat of the torches an pitchforks.
^ permalink raw reply
* Re: [PATCH v2 2/5] x86/virt/tdx: Pull kexec cache flush logic into arch/x86
From: Sean Christopherson @ 2026-04-01 18:12 UTC (permalink / raw)
To: H. Peter Anvin
Cc: Dave Hansen, Rick P Edgecombe, Vishal L Verma, Kai Huang,
bp@alien8.de, x86@kernel.org, kas@kernel.org, mingo@redhat.com,
linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com,
tglx@kernel.org, pbonzini@redhat.com, linux-coco@lists.linux.dev,
kvm@vger.kernel.org
In-Reply-To: <76F29857-47D8-470B-9F4D-DD98D8755EB0@zytor.com>
On Wed, Apr 01, 2026, H. Peter Anvin wrote:
> On April 1, 2026 8:03:02 AM PDT, Dave Hansen <dave.hansen@intel.com> wrote:
> >On 3/31/26 16:04, Sean Christopherson wrote:
> >> But unless the WBINVD is actually costly, why bother getting fancy?
> >
> >WBINVD might be the most expensive single instruction in the whole ISA.
> >
> >That said, I'd much rather have a potentially unnecessary WBINVD than
> >miss one. The thing I'd be worried about would be something wonky like:
> >
> > 1. CPU offline does WBINVD
> > 2. Some other TDX call gets made, dirties caches again
> > 3. tdx_offline_cpu() skips WBINVD
> >
> >So, let's just do both for now: Do WBINVD in tdx_offline_cpu() and
> >comment that it might be redundant with other things in the CPU offline
> >procedure.
> >
> >This really needs to be solved with infrastructure and keeping data
> >about the reasons for needing WBINVD, not relying on code ordering or
> >fragile semantics.
>
> It is, *by far*, the most expensive *uninterruptible* instruction in the ISA.
> REP string instructions can of course be arbitrarily long, but are
> interruptible and so don't really count.
>
> Some MSRs used during very early (pre-OS) initialization might be even slower
> on some implementations, but that's not visible to Linux and no workload of
> any kind is running.
Sorry, "costly" wasn't the right word. I know WBINVD super expensive, but unless
someone cares deeply about the latency of offlining a CPU after its down TDX stuff,
the "cost" is effectively zero.
^ permalink raw reply
* Re: [PATCH v2 2/5] x86/virt/tdx: Pull kexec cache flush logic into arch/x86
From: H. Peter Anvin @ 2026-04-01 17:42 UTC (permalink / raw)
To: Dave Hansen, Sean Christopherson, Rick P Edgecombe
Cc: Vishal L Verma, Kai Huang, bp@alien8.de, x86@kernel.org,
kas@kernel.org, mingo@redhat.com, linux-kernel@vger.kernel.org,
dave.hansen@linux.intel.com, tglx@kernel.org, pbonzini@redhat.com,
linux-coco@lists.linux.dev, kvm@vger.kernel.org
In-Reply-To: <830f1e46-0fb7-4756-827b-c8f46af24374@intel.com>
On April 1, 2026 8:03:02 AM PDT, Dave Hansen <dave.hansen@intel.com> wrote:
>On 3/31/26 16:04, Sean Christopherson wrote:
>> But unless the WBINVD is actually costly, why bother getting fancy?
>
>WBINVD might be the most expensive single instruction in the whole ISA.
>
>That said, I'd much rather have a potentially unnecessary WBINVD than
>miss one. The thing I'd be worried about would be something wonky like:
>
> 1. CPU offline does WBINVD
> 2. Some other TDX call gets made, dirties caches again
> 3. tdx_offline_cpu() skips WBINVD
>
>So, let's just do both for now: Do WBINVD in tdx_offline_cpu() and
>comment that it might be redundant with other things in the CPU offline
>procedure.
>
>This really needs to be solved with infrastructure and keeping data
>about the reasons for needing WBINVD, not relying on code ordering or
>fragile semantics.
It is, *by far*, the most expensive *uninterruptible* instruction in the ISA. REP string instructions can of course be arbitrarily long, but are interruptible and so don't really count.
Some MSRs used during very early (pre-OS) initialization might be even slower on some implementations, but that's not visible to Linux and no workload of any kind is running.
^ permalink raw reply
* Re: [PATCH v3 3/6] x86/sev: Add support to perform RMP optimizations asynchronously
From: Dave Hansen @ 2026-04-01 16:10 UTC (permalink / raw)
To: Kalra, Ashish, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <a1a5fa97-b067-4145-a4dc-fbb04fe9c720@amd.com>
On 4/1/26 08:47, Kalra, Ashish wrote:
> For programming the RMPOPT_BASE MSR performance is not really that
> important as it is for issuing the RMPOPT instruction on only thread
> per core, and as we are programming the RMPOPT_BASE MSRs on all CPUs/
> threads to the same (starting) physical address to support all RAM
> up-to 2TB for RMP optimizations, therefore, i don't think it is that
> critical to implement wrmsrq_on_cpumask() and instead we can
> continue to program the RMPOPT_BASE MSR on all CPUs (threads).
I don't mean to be a grammar pedant. But, man, that's hard to parse when
written as a single sentence.
I'm also not quite sure what the resistance is to going and adding the
precise function that is needed:
int wrmsrq_on_cpus(const struct cpumask *mask, u32 msr_no, u64 q)
{
int err;
struct msr_info rv;
memset(&rv, 0, sizeof(rv));
rv.msr_no = msr_no;
rv.reg.q = q;
err = smp_call_function_many(mask, __wrmsr_on_cpu, &rv, 1);
return err;
}
EXPORT_SYMBOL(wrmsrq_on_cpus);
It's just wrmsrq_on_cpu(), replace the 'cpu' with a cpumask and
s/_single/_many/. I think. Unless I'm missing something.
^ permalink raw reply
* Re: [PATCH 2/2] x86/tdx: Accept hotplugged memory before online
From: Edgecombe, Rick P @ 2026-04-01 15:49 UTC (permalink / raw)
To: pbonzini@redhat.com, prsampat@amd.com
Cc: dave.hansen@linux.intel.com, marcandre.lureau@redhat.com,
kas@kernel.org, bp@alien8.de, linux-kernel@vger.kernel.org,
mingo@redhat.com, x86@kernel.org, Qiang, Chenyi, tglx@kernel.org,
hpa@zytor.com, kvm@vger.kernel.org, linux-coco@lists.linux.dev
In-Reply-To: <49e8b24d836c1883e83ad72d1ab279f9e3eb7455.camel@intel.com>
On Wed, 2026-04-01 at 08:39 -0700, Edgecombe, Rick P wrote:
> > Would it not be better to have this information in the unaccepted
> > bitmap which we could explicitly query to accept/unaccept?
>
> It makes me think about shared memory too. Should the unplug event
> also signal the host to reset the memory to private? If the VMM is
> actually not adjusting the guest mapping for a unplug/re-plug then
> the memory would come back as shared.
>
> But it really starts to feel like work the host should be doing.
Although if memory was able to be unplugged, the Linux guest would have
reset the memory to private when it was done with the memory. But from
a interface perspective, it seems weird to require the guest to signal
this because private/shared state is ultimately controlled by the host.
Unlike accept state where the guest gets some say.
^ permalink raw reply
* Re: [PATCH v3 3/6] x86/sev: Add support to perform RMP optimizations asynchronously
From: Kalra, Ashish @ 2026-04-01 15:47 UTC (permalink / raw)
To: Dave Hansen, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <e2e38a91-ef8c-41d0-9373-0fdb6f246847@amd.com>
Hello Dave,
On 3/30/2026 7:46 PM, Kalra, Ashish wrote:
>
> On 3/30/2026 6:22 PM, Dave Hansen wrote:
>>
>>> static __init void configure_and_enable_rmpopt(void)
>>> {
>>> phys_addr_t pa_start = ALIGN_DOWN(PFN_PHYS(min_low_pfn), SZ_1G);
>>> @@ -499,6 +582,37 @@ static __init void configure_and_enable_rmpopt(void)
>>> */
>>> for_each_online_cpu(cpu)
>>> wrmsrq_on_cpu(cpu, MSR_AMD64_RMPOPT_BASE, rmpopt_base);
>>
>> What is the scope of MSR_AMD64_RMPOPT_BASE? Can you have it enabled on
>> one thread and not the other? Could they be different values both for
>> enabling and the rmpopt_base value?
>>
>> If it's not per-thread, then why is it being initialized for each thread?
>>
>
> Only one logical thread per core needs to set RMPOPT_BASE MSR as it is per-core,
> so i will use the "primary_threads_cpumask" here to use it for programming this
> MSR.
>
> Just another reason, to set the "primary_threads_cpumask" here in this function
> and then re-use it for the RMPOPT worker.
>
Coming back to this ...
For using the "primary_thread_cpumask" i will need to use something like
on_each_cpu_mask() similar to what i was doing in v2.
In v2, i was programming the RMPOPT_BASE MSR using on_each_cpu_mask(),
that required using a callback function to do the WRMSR:
+static void __configure_rmpopt(void *val)
+{
+ u64 rmpopt_base = ((u64)val & PUD_MASK) | MSR_AMD64_RMPOPT_ENABLE;
+
+ wrmsrq(MSR_AMD64_RMPOPT_BASE, rmpopt_base);
+}
+
+ on_each_cpu_mask(cpu_online_mask, __configure_rmpopt, (void *)pa_start, true);
But, that required using the (void *) casting, which you objected to and you
suggested the use of for_each_online_cpu() and wrmsrq_on_cpu(), and i has replied
that i need to do it (only) once on each thread per core, and that's why i may need
to use on_each_cpu_mask() and then you had suggested that if you *need* performance
then i can implement/add something like wrmsrq_on_cpumask().
For programming the RMPOPT_BASE MSR performance is not really that important as
it is for issuing the RMPOPT instruction on only thread per core, and as we are
programming the RMPOPT_BASE MSRs on all CPUs/threads to the same (starting) physical
address to support all RAM up-to 2TB for RMP optimizations, therefore, i don't
think it is that critical to implement wrmsrq_on_cpumask() and instead we can continue
to program the RMPOPT_BASE MSR on all CPUs (threads).
Thanks,
Ashish
^ permalink raw reply
* Re: [PATCH 2/2] x86/tdx: Accept hotplugged memory before online
From: Edgecombe, Rick P @ 2026-04-01 15:37 UTC (permalink / raw)
To: pbonzini@redhat.com, prsampat@amd.com
Cc: dave.hansen@linux.intel.com, marcandre.lureau@redhat.com,
kas@kernel.org, bp@alien8.de, linux-kernel@vger.kernel.org,
mingo@redhat.com, x86@kernel.org, Qiang, Chenyi, tglx@kernel.org,
hpa@zytor.com, kvm@vger.kernel.org, linux-coco@lists.linux.dev
In-Reply-To: <cab5371d-e0f4-42b7-bae9-2c7f981b26b2@amd.com>
On Mon, 2026-03-30 at 11:10 -0400, Pratik R. Sampat wrote:
> SNP likely has an analogous issue too.
> Failing to switch states on remove will cause that RMP entry to
> remain validated. A malicious hypervisor could then remap this GPA to
> another HPA which would put this in the Guest-Invalid state. On re-
> hotplug if we ignore errors suggested by Patch 1 (in our case that'd
> be PVALIDATE_FAIL_NOUPDATE error likely), we could have two RMP
> entries for the same GPA and both being validated. This is dangerous
> because hypervisor could swap these at will.
Oh, I was just wondering if we could just zero the page on accept
failure for the case of already accepted. Handle the issue internally
and actually go back to something like patch 1. Will it work for SNP?
>
> Would it not be better to have this information in the unaccepted
> bitmap which we could explicitly query to accept/unaccept?
It makes me think about shared memory too. Should the unplug event also
signal the host to reset the memory to private? If the VMM is actually
not adjusting the guest mapping for a unplug/re-plug then the memory
would come back as shared.
But it really starts to feel like work the host should be doing.
>
> For ACPI hardware-style hotplug I was working with the UEFI side on a
> POC to reflect SRAT hotplug windows in UEFI_UNACCEPTED_MEMORY using
> EFI_MEMORY_HOT_PLUGGABLE attribute and working to modify that spec.
> I’m less sure what this description for virtio-mem would look like
> and if it'd be possible to do this early-boot.
^ permalink raw reply
* Re: [PATCH v2 2/5] x86/virt/tdx: Pull kexec cache flush logic into arch/x86
From: Dave Hansen @ 2026-04-01 15:03 UTC (permalink / raw)
To: Sean Christopherson, Rick P Edgecombe
Cc: Vishal L Verma, Kai Huang, bp@alien8.de, x86@kernel.org,
kas@kernel.org, hpa@zytor.com, mingo@redhat.com,
linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com,
tglx@kernel.org, pbonzini@redhat.com, linux-coco@lists.linux.dev,
kvm@vger.kernel.org
In-Reply-To: <acxTBwaw6_xYSShf@google.com>
On 3/31/26 16:04, Sean Christopherson wrote:
> But unless the WBINVD is actually costly, why bother getting fancy?
WBINVD might be the most expensive single instruction in the whole ISA.
That said, I'd much rather have a potentially unnecessary WBINVD than
miss one. The thing I'd be worried about would be something wonky like:
1. CPU offline does WBINVD
2. Some other TDX call gets made, dirties caches again
3. tdx_offline_cpu() skips WBINVD
So, let's just do both for now: Do WBINVD in tdx_offline_cpu() and
comment that it might be redundant with other things in the CPU offline
procedure.
This really needs to be solved with infrastructure and keeping data
about the reasons for needing WBINVD, not relying on code ordering or
fragile semantics.
^ permalink raw reply
* Re: [PATCH v2 3/5] x86/virt/tdx: Add SEAMCALL wrapper for TDH.SYS.DISABLE
From: Dave Hansen @ 2026-04-01 14:24 UTC (permalink / raw)
To: Edgecombe, Rick P, Verma, Vishal L, kas@kernel.org
Cc: seanjc@google.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com,
mingo@redhat.com, linux-kernel@vger.kernel.org,
dave.hansen@linux.intel.com, tglx@kernel.org, pbonzini@redhat.com,
linux-coco@lists.linux.dev, kvm@vger.kernel.org
In-Reply-To: <944b3fe5fa8955319030a7dfc0ea164bb0266e68.camel@intel.com>
On 3/31/26 14:36, Edgecombe, Rick P wrote:
> On Tue, 2026-03-31 at 18:22 +0000, Verma, Vishal L wrote:
>>> I guess the actual behaviour is dependant on the return code. It is
>>> obviously going to be the case for TDX_SUCCESS. And from the discussion,
>>> I guess that's true for TDX_SYS_BUSY and TDX_INTERRUPTED_RESUMABLE.
>>>
>>> What about other cases? The spec draft also lists TDX_SYS_NOT_READY and
>>> TDX_SYS_SHUTDOWN.
>> I think these are safe too - TDX_SYS_SHUTDOWN means the module has
>> already been shutdown, which this seamcall would've done, so things
>> should be in the same state either way.
>>
>> TDX_SYS_NOT_READY means the module hasn't been initialized yet. This
>> seamcall should just exit, and the module is already blocking any
>> seamcall that need the module to be initialized. The seamcalls to
>> initialize the module will be allowed, as they are after a sys_disable
>> call anyway.
> Should the seamcall return success in the case where it would return
> TDX_SYS_NOT_READY? It is in basically a reset state right? The errors we care
> about are actual errors (TDX_SW_ERROR), so it makes no difference to the code in
> the patch. But it might be a nicer API for the seamcall?
The problem is that the module doesn't have *a* reset state.
TDX_SYS_NOT_READY gets returned before the module is initialized and
initialization is a long, arduous process.
For instance, I believe the module stays "not ready" in the middle of
giving it PAMT memory and a keyID and all that jazz.
TDX_SYS_NOT_READY is a way of saying it can't easily *make* it to the
actual reset state that TDH.SYS.DISABLE wants it to be in.
It's arguable that the module should be made more resilient to stop
returning TDX_SYS_NOT_READY. But it's not as simple as just changing a
return code in the module.
I'm OK with it continuing to return TDX_SYS_NOT_READY for now. I think
it's a useful indicator. Maybe the kernel can't do much with it, but
it's a little window into what went wrong.
^ permalink raw reply
* Re: [PATCH v2 11/31] x86/virt/tdx: Make TDX Module initialize Extensions
From: Huang, Kai @ 2026-04-01 11:42 UTC (permalink / raw)
To: yilun.xu@linux.intel.com, Edgecombe, Rick P
Cc: Gao, Chao, Xu, Yilun, x86@kernel.org, kas@kernel.org,
baolu.lu@linux.intel.com, dave.hansen@linux.intel.com,
Li, Xiaoyao, Williams, Dan J, Jiang, Dave,
linux-pci@vger.kernel.org, linux-coco@lists.linux.dev,
linux-kernel@vger.kernel.org, Duan, Zhenzhong, Verma, Vishal L,
kvm@vger.kernel.org
In-Reply-To: <acvhD/XeCbB4ea0C@yilunxu-OptiPlex-7050>
On Tue, 2026-03-31 at 22:58 +0800, Xu Yilun wrote:
> > > + /*
> > > + * ext_required == 0 means no need to call TDH.EXT.INIT, the Extensions
> > > + * are already working.
> >
> > How does this scenario happen exactly? And why not check it above at the
> > beginning? Before the allocation, so it doesn't need to free.
> >
> > Is there a scenario where the memory needs to be given, but the extension is
> > already inited?
>
> mm.. you are right. It leads to something absurd.
>
> I checked with TDX Module team again. The correct understanding is:
>
> - TDX_FEATURES0_EXT bit shows Extensions is supported.
> - optional feature bits are selected on TDH_SYS_CONFIG
> - If one of the optional feature (e.g. TDX CONNECT) requires Extention,
> memory_pool_required_pages > 0 && ext_required == 1. Otherwise no
> need to initialize Extension.
>
> So yes, I should check memory_pool_required_pages && ext_required at the
> beginning.
My understanding is different:
Per spec, the 'EXT_REQUIRED' global metadata just means "Return true if the
TDH.EXT.INIT is required to be called", so I think, architecturally, it's
possible that one particular feature only requires additional memory pool
but doesn't explicitly need to call TDH.EXT.INIT. Or some feature may not
require any additional memory pool but needs TDH.EXT.INIT. Or require both
(such as TDX Connect I presume).
We can safely assume 2) and 3) are not required if no feature is configured
in 1) (backward compatibility). But when there is, I think we can just:
1) If 'MEMORY_POOL_REQUIRED_PAGES' is not zero, do TDH.EXT.MEM.ADD
2) If 'EXT_REQUIRED' is true, do TDH.EXT.INIT
^ permalink raw reply
* Re: [PATCH v13 12/48] arm64: RMI: Basic infrastructure for creating a realm.
From: Steven Price @ 2026-04-01 10:54 UTC (permalink / raw)
To: Wei-Lin Chang, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <xpl4tvzm22ruavicwzgpcw7dsh5mtclhsp5tnkkyybgrxiwptj@f4tmkuyutitp>
On 21/03/2026 16:34, Wei-Lin Chang wrote:
> On Wed, Mar 18, 2026 at 03:53:36PM +0000, Steven Price wrote:
>> Introduce the skeleton functions for creating and destroying a realm.
>> The IPA size requested is checked against what the RMM supports.
>>
>> The actual work of constructing the realm will be added in future
>> patches.
>>
>> Signed-off-by: Steven Price <steven.price@arm.com>
>> ---
>> Changes since v12:
>> * Drop the RMM_PAGE_{SHIFT,SIZE} defines - the RMM is now configured to
>> be the same as the host's page size.
>> * Rework delegate/undelegate functions to use the new RMI range based
>> operations.
>> Changes since v11:
>> * Major rework to drop the realm configuration and make the
>> construction of realms implicit rather than driven by the VMM
>> directly.
>> * The code to create RDs, handle VMIDs etc is moved to later patches.
>> Changes since v10:
>> * Rename from RME to RMI.
>> * Move the stage2 cleanup to a later patch.
>> Changes since v9:
>> * Avoid walking the stage 2 page tables when destroying the realm -
>> the real ones are not accessible to the non-secure world, and the RMM
>> may leave junk in the physical pages when returning them.
>> * Fix an error path in realm_create_rd() to actually return an error value.
>> Changes since v8:
>> * Fix free_delegated_granule() to not call kvm_account_pgtable_pages();
>> a separate wrapper will be introduced in a later patch to deal with
>> RTTs.
>> * Minor code cleanups following review.
>> Changes since v7:
>> * Minor code cleanup following Gavin's review.
>> Changes since v6:
>> * Separate RMM RTT calculations from host PAGE_SIZE. This allows the
>> host page size to be larger than 4k while still communicating with an
>> RMM which uses 4k granules.
>> Changes since v5:
>> * Introduce free_delegated_granule() to replace many
>> undelegate/free_page() instances and centralise the comment on
>> leaking when the undelegate fails.
>> * Several other minor improvements suggested by reviews - thanks for
>> the feedback!
>> Changes since v2:
>> * Improved commit description.
>> * Improved return failures for rmi_check_version().
>> * Clear contents of PGD after it has been undelegated in case the RMM
>> left stale data.
>> * Minor changes to reflect changes in previous patches.
>> ---
>> arch/arm64/include/asm/kvm_emulate.h | 5 ++
>> arch/arm64/include/asm/kvm_rmi.h | 16 +++++
>> arch/arm64/kvm/arm.c | 12 ++++
>> arch/arm64/kvm/mmu.c | 11 +++-
>> arch/arm64/kvm/rmi.c | 88 ++++++++++++++++++++++++++++
>> 5 files changed, 129 insertions(+), 3 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
>> index f38b50151ce8..39310d9b4e16 100644
>> --- a/arch/arm64/include/asm/kvm_emulate.h
>> +++ b/arch/arm64/include/asm/kvm_emulate.h
>> @@ -701,6 +701,11 @@ static inline enum realm_state kvm_realm_state(struct kvm *kvm)
>> return READ_ONCE(kvm->arch.realm.state);
>> }
>>
>> +static inline bool kvm_realm_is_created(struct kvm *kvm)
>> +{
>> + return kvm_is_realm(kvm) && kvm_realm_state(kvm) != REALM_STATE_NONE;
>> +}
>> +
>> static inline bool vcpu_is_rec(struct kvm_vcpu *vcpu)
>> {
>> return false;
>> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
>> index 3506f50b05cd..0ada525af18f 100644
>> --- a/arch/arm64/include/asm/kvm_rmi.h
>> +++ b/arch/arm64/include/asm/kvm_rmi.h
>> @@ -6,6 +6,8 @@
>> #ifndef __ASM_KVM_RMI_H
>> #define __ASM_KVM_RMI_H
>>
>> +#include <asm/rmi_smc.h>
>> +
>> /**
>> * enum realm_state - State of a Realm
>> */
>> @@ -46,11 +48,25 @@ enum realm_state {
>> * struct realm - Additional per VM data for a Realm
>> *
>> * @state: The lifetime state machine for the realm
>> + * @rd: Kernel mapping of the Realm Descriptor (RD)
>> + * @params: Parameters for the RMI_REALM_CREATE command
>> + * @num_aux: The number of auxiliary pages required by the RMM
>> + * @ia_bits: Number of valid Input Address bits in the IPA
>> */
>> struct realm {
>> enum realm_state state;
>> +
>> + void *rd;
>> + struct realm_params *params;
>> +
>> + unsigned long num_aux;
>> + unsigned int ia_bits;
>> };
>>
>> void kvm_init_rmi(void);
>> +u32 kvm_realm_ipa_limit(void);
>> +
>> +int kvm_init_realm_vm(struct kvm *kvm);
>> +void kvm_destroy_realm(struct kvm *kvm);
>>
>> #endif /* __ASM_KVM_RMI_H */
>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
>> index 274d7866efdc..9b17bdfaf0c2 100644
>> --- a/arch/arm64/kvm/arm.c
>> +++ b/arch/arm64/kvm/arm.c
>> @@ -253,6 +253,13 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
>>
>> bitmap_zero(kvm->arch.vcpu_features, KVM_VCPU_MAX_FEATURES);
>>
>> + /* Initialise the realm bits after the generic bits are enabled */
>> + if (kvm_is_realm(kvm)) {
>> + ret = kvm_init_realm_vm(kvm);
>> + if (ret)
>> + goto err_free_cpumask;
>> + }
>> +
>> return 0;
>>
>> err_free_cpumask:
>> @@ -312,6 +319,8 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
>> kvm_unshare_hyp(kvm, kvm + 1);
>>
>> kvm_arm_teardown_hypercalls(kvm);
>> + if (kvm_is_realm(kvm))
>> + kvm_destroy_realm(kvm);
>> }
>>
>> static bool kvm_has_full_ptr_auth(void)
>> @@ -473,6 +482,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
>> else
>> r = kvm_supports_cacheable_pfnmap();
>> break;
>> + case KVM_CAP_ARM_RMI:
>> + r = static_key_enabled(&kvm_rmi_is_available);
>> + break;
>>
>> default:
>> r = 0;
>> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
>> index 070a01e53fcb..d6094b60c4ce 100644
>> --- a/arch/arm64/kvm/mmu.c
>> +++ b/arch/arm64/kvm/mmu.c
>> @@ -872,12 +872,16 @@ static struct kvm_pgtable_mm_ops kvm_s2_mm_ops = {
>> .icache_inval_pou = invalidate_icache_guest_page,
>> };
>>
>> -static int kvm_init_ipa_range(struct kvm_s2_mmu *mmu, unsigned long type)
>> +static int kvm_init_ipa_range(struct kvm *kvm,
>> + struct kvm_s2_mmu *mmu, unsigned long type)
>> {
>> u32 kvm_ipa_limit = get_kvm_ipa_limit();
>> u64 mmfr0, mmfr1;
>> u32 phys_shift;
>>
>> + if (kvm_is_realm(kvm))
>> + kvm_ipa_limit = kvm_realm_ipa_limit();
>> +
>> if (type & ~KVM_VM_TYPE_ARM_IPA_SIZE_MASK)
>> return -EINVAL;
>>
>> @@ -974,7 +978,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
>> return -EINVAL;
>> }
>>
>> - err = kvm_init_ipa_range(mmu, type);
>> + err = kvm_init_ipa_range(kvm, mmu, type);
>> if (err)
>> return err;
>>
>> @@ -1113,7 +1117,8 @@ void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
>> write_unlock(&kvm->mmu_lock);
>>
>> if (pgt) {
>> - kvm_stage2_destroy(pgt);
>> + if (!kvm_is_realm(kvm))
>> + kvm_stage2_destroy(pgt);
>
> Hi,
>
> Question:
> Since kvm_stage2_destroy() is only called for non-realm VMs, then where
> does the root level RTT pages get freed?
> After searching for a while I feel like it is missed, but I am not
> certain.
You're absolutely right. I do have a bug fix locally for this:
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 9cfb8c434aa5..3e55c1ff046c 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1148,6 +1148,8 @@ void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
if (pgt) {
if (!kvm_is_realm(kvm))
kvm_stage2_destroy(pgt);
+ else
+ kvm_pgtable_stage2_destroy_pgd(pgt);
kfree(pgt);
}
}
The issue here is that we don't want to talk the tables (because they
will have been scrubbed by the RMM), but I apparently forgot to add the
call to free the actual memory.
Thanks,
Steve
> Thanks,
> Wei-Lin Chang
>
>> kfree(pgt);
>> }
>> }
>> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
>> index 80aedc85e94a..700b8c935d29 100644
>> --- a/arch/arm64/kvm/rmi.c
>> +++ b/arch/arm64/kvm/rmi.c
>> @@ -6,6 +6,8 @@
>> #include <linux/kvm_host.h>
>> #include <linux/memblock.h>
>>
>> +#include <asm/kvm_emulate.h>
>> +#include <asm/kvm_mmu.h>
>> #include <asm/kvm_pgtable.h>
>> #include <asm/rmi_cmds.h>
>> #include <asm/virt.h>
>> @@ -182,6 +184,92 @@ static int rmi_init_metadata(void)
>> return 0;
>> }
>>
>> +u32 kvm_realm_ipa_limit(void)
>> +{
>> + return u64_get_bits(rmm_feat_reg0, RMI_FEATURE_REGISTER_0_S2SZ);
>> +}
>> +
>> +static int undelegate_range(phys_addr_t phys, unsigned long size)
>> +{
>> + unsigned long ret;
>> + unsigned long top = phys + size;
>> + unsigned long out_top;
>> +
>> + while (phys < top) {
>> + ret = rmi_granule_range_undelegate(phys, top, &out_top);
>> + if (ret == RMI_SUCCESS)
>> + phys = out_top;
>> + else if (ret != RMI_BUSY && ret != RMI_BLOCKED)
>> + return ret;
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +static int undelegate_page(phys_addr_t phys)
>> +{
>> + return undelegate_range(phys, PAGE_SIZE);
>> +}
>> +
>> +static int free_delegated_page(phys_addr_t phys)
>> +{
>> + if (WARN_ON(undelegate_page(phys))) {
>> + /* Undelegate failed: leak the page */
>> + return -EBUSY;
>> + }
>> +
>> + free_page((unsigned long)phys_to_virt(phys));
>> +
>> + return 0;
>> +}
>> +
>> +void kvm_destroy_realm(struct kvm *kvm)
>> +{
>> + struct realm *realm = &kvm->arch.realm;
>> + size_t pgd_size = kvm_pgtable_stage2_pgd_size(kvm->arch.mmu.vtcr);
>> +
>> + write_lock(&kvm->mmu_lock);
>> + kvm_stage2_unmap_range(&kvm->arch.mmu, 0,
>> + BIT(realm->ia_bits - 1), true);
>> + write_unlock(&kvm->mmu_lock);
>> +
>> + if (realm->params) {
>> + free_page((unsigned long)realm->params);
>> + realm->params = NULL;
>> + }
>> +
>> + if (!kvm_realm_is_created(kvm))
>> + return;
>> +
>> + WRITE_ONCE(realm->state, REALM_STATE_DYING);
>> +
>> + if (realm->rd) {
>> + phys_addr_t rd_phys = virt_to_phys(realm->rd);
>> +
>> + if (WARN_ON(rmi_realm_destroy(rd_phys)))
>> + return;
>> + free_delegated_page(rd_phys);
>> + realm->rd = NULL;
>> + }
>> +
>> + if (WARN_ON(undelegate_range(kvm->arch.mmu.pgd_phys, pgd_size)))
>> + return;
>> +
>> + WRITE_ONCE(realm->state, REALM_STATE_DEAD);
>> +
>> + /* Now that the Realm is destroyed, free the entry level RTTs */
>> + kvm_free_stage2_pgd(&kvm->arch.mmu);
>> +}
>> +
>> +int kvm_init_realm_vm(struct kvm *kvm)
>> +{
>> + kvm->arch.realm.params = (void *)get_zeroed_page(GFP_KERNEL);
>> +
>> + if (!kvm->arch.realm.params)
>> + return -ENOMEM;
>> + return 0;
>> +}
>> +
>> static int rmm_check_features(void)
>> {
>> if (kvm_lpa2_is_enabled() && !rmi_has_feature(RMI_FEATURE_REGISTER_0_LPA2)) {
>> --
>> 2.43.0
>>
^ permalink raw reply related
* Re: [PATCH v2 08/31] x86/virt/tdx: Configure TDX Module with optional TDX Connect feature
From: Huang, Kai @ 2026-04-01 10:13 UTC (permalink / raw)
To: Williams, Dan J, linux-pci@vger.kernel.org,
linux-coco@lists.linux.dev, yilun.xu@linux.intel.com,
x86@kernel.org
Cc: Gao, Chao, Edgecombe, Rick P, Xu, Yilun, Jiang, Dave,
dave.hansen@linux.intel.com, baolu.lu@linux.intel.com,
Duan, Zhenzhong, kas@kernel.org, Verma, Vishal L, Li, Xiaoyao,
kvm@vger.kernel.org, linux-kernel@vger.kernel.org
In-Reply-To: <20260327160132.2946114-9-yilun.xu@linux.intel.com>
>
> TDX Module updates global metadata when optional features are enabled.
> Host should update the cached tdx_sysinfo to reflect these changes.
>
>
[...]
> static int config_tdx_module(struct tdmr_info_list *tdmr_list, u64 global_keyid)
> {
> struct tdx_module_args args = {};
> + u64 seamcall_fn = TDH_SYS_CONFIG_V0;
> u64 *tdmr_pa_array;
> size_t array_sz;
> int i, ret;
> @@ -1377,7 +1378,15 @@ static int config_tdx_module(struct tdmr_info_list *tdmr_list, u64 global_keyid)
> args.rcx = __pa(tdmr_pa_array);
> args.rdx = tdmr_list->nr_consumed_tdmrs;
> args.r8 = global_keyid;
> - ret = seamcall_prerr(TDH_SYS_CONFIG, &args);
> +
> + if (tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_TDXCONNECT) {
> + args.r9 |= TDX_FEATURES0_TDXCONNECT;
> + args.r11 = ktime_get_real_seconds();
> + /* These parameters requires version >= 1 */
> + seamcall_fn = TDH_SYS_CONFIG;
> + }
> +
> + ret = seamcall_prerr(seamcall_fn, &args);
>
> /* Free the array as it is not required anymore. */
> kfree(tdmr_pa_array);
> @@ -1537,6 +1546,11 @@ static int init_tdx_module(void)
> if (ret)
> goto err_free_pamts;
>
> + /* configuration to tdx module may change tdx_sysinfo, update it */
> + ret = get_tdx_sys_info(&tdx_sysinfo);
> + if (ret)
> + goto err_reset_pamts;
> +
How about put this into config_tdx_module()?
In this way you can only update global metadata when there's new feature
being opted in, and at the meantime, avoid making init_tdx_module() more
complicated.
^ permalink raw reply
* Re: [PATCH v2 3/5] x86/virt/tdx: Add SEAMCALL wrapper for TDH.SYS.DISABLE
From: Kiryl Shutsemau @ 2026-04-01 9:26 UTC (permalink / raw)
To: Edgecombe, Rick P
Cc: Verma, Vishal L, seanjc@google.com, bp@alien8.de, x86@kernel.org,
hpa@zytor.com, mingo@redhat.com, linux-kernel@vger.kernel.org,
dave.hansen@linux.intel.com, tglx@kernel.org, pbonzini@redhat.com,
linux-coco@lists.linux.dev, kvm@vger.kernel.org
In-Reply-To: <944b3fe5fa8955319030a7dfc0ea164bb0266e68.camel@intel.com>
On Tue, Mar 31, 2026 at 09:36:03PM +0000, Edgecombe, Rick P wrote:
> On Tue, 2026-03-31 at 18:22 +0000, Verma, Vishal L wrote:
> > >
> > > I guess the actual behaviour is dependant on the return code. It is
> > > obviously going to be the case for TDX_SUCCESS. And from the discussion,
> > > I guess that's true for TDX_SYS_BUSY and TDX_INTERRUPTED_RESUMABLE.
> > >
> > > What about other cases? The spec draft also lists TDX_SYS_NOT_READY and
> > > TDX_SYS_SHUTDOWN.
> >
> > I think these are safe too - TDX_SYS_SHUTDOWN means the module has
> > already been shutdown, which this seamcall would've done, so things
> > should be in the same state either way.
> >
> > TDX_SYS_NOT_READY means the module hasn't been initialized yet. This
> > seamcall should just exit, and the module is already blocking any
> > seamcall that need the module to be initialized. The seamcalls to
> > initialize the module will be allowed, as they are after a sys_disable
> > call anyway.
>
> Should the seamcall return success in the case where it would return
> TDX_SYS_NOT_READY? It is in basically a reset state right? The errors we care
> about are actual errors (TDX_SW_ERROR), so it makes no difference to the code in
> the patch. But it might be a nicer API for the seamcall?
I am not sure. TDX_SYS_NOT_READY can be useful as might indicate
mismatch of system state understanding between kernel and TDX module.
> > > I wounder if it can affect the kernel. Consider the case when kexec
> > > (crash kernel start) happens due to crash on TDX module.
> > >
> > > Will we be able to shutdown TDX module cleanly and make kexec safe?
> >
> > Hm -are the semantics for what happens if there is a crash in the
> > module defined?
I meant kernel crash around/before TDX module initialization. Sorry for
confusion.
> > I think Linux should expect that sys_disable should
> > either start doing its shutdown work, or exit with one of the other
> > defined exit statuses. Anything else would be considered a module bug.
>
> We often have the question come up about how much we should to guard against
> bugs in the TDX module. I tend to also think we should not do defensive
> programming, same as we do for the kernel. If it's easy to handle something or
> emit a warning it's nice, but otherwise the solution for such cases should be to
> fix the TDX module bug.
>
> But for the kdump case, we don't actually need sys disable to succeed. The kdump
> kernel will not load the TDX module.
AFAIK, it is possible to start a normal kernel after kdump is done with
kexec (requires memmap= tricks). And the normal kernel might want to use
TDX again.
Not sure if it is done in practice. I would rather go full reboot path
after crash.
> And as for the errata, this already needs a
> special situation to be a problem. But even if it happens, I'd think better to
> try to the kdump. Not sure what the fix would be for that scenario, even if we
> allowed for a large complexity budget. So best effort seems good.
>
> Does it seem reasonable?
I am probably too picky here. We want to start from make basic kexec
functionality to work for start.
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox