* [PATCH 1/5] dma-mapping: avoid random addr value print out on error path
From: Jiri Pirko @ 2026-02-09 15:38 UTC (permalink / raw)
To: dri-devel, linaro-mm-sig, iommu, linux-media
Cc: sumit.semwal, benjamin.gaignard, Brian.Starkey, jstultz,
tjmercier, christian.koenig, m.szyprowski, robin.murphy, jgg,
leon, sean.anderson, ptesarik, catalin.marinas, aneesh.kumar,
suzuki.poulose, steven.price, thomas.lendacky, john.allen,
ashish.kalra, suravee.suthikulpanit, linux-coco
In-Reply-To: <20260209153809.250835-1-jiri@resnulli.us>
From: Jiri Pirko <jiri@nvidia.com>
dma_addr is unitialized in dma_direct_map_phys() when swiotlb is forced
and DMA_ATTR_MMIO is set which leads to random value print out in
warning. Fix that by just returning DMA_MAPPING_ERROR.
Fixes: e53d29f957b3 ("dma-mapping: convert dma_direct_*map_page to be phys_addr_t based")
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
kernel/dma/direct.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h
index da2fadf45bcd..62f0d9d0ba02 100644
--- a/kernel/dma/direct.h
+++ b/kernel/dma/direct.h
@@ -88,7 +88,7 @@ static inline dma_addr_t dma_direct_map_phys(struct device *dev,
if (is_swiotlb_force_bounce(dev)) {
if (attrs & DMA_ATTR_MMIO)
- goto err_overflow;
+ return DMA_MAPPING_ERROR;
return swiotlb_map(dev, phys, size, dir, attrs);
}
--
2.51.1
^ permalink raw reply related
* [PATCH 0/5] dma-buf: heaps: system: add an option to allocate explicitly decrypted memory
From: Jiri Pirko @ 2026-02-09 15:38 UTC (permalink / raw)
To: dri-devel, linaro-mm-sig, iommu, linux-media
Cc: sumit.semwal, benjamin.gaignard, Brian.Starkey, jstultz,
tjmercier, christian.koenig, m.szyprowski, robin.murphy, jgg,
leon, sean.anderson, ptesarik, catalin.marinas, aneesh.kumar,
suzuki.poulose, steven.price, thomas.lendacky, john.allen,
ashish.kalra, suravee.suthikulpanit, linux-coco
From: Jiri Pirko <jiri@nvidia.com>
Confidential computing (CoCo) VMs/guests, such as AMD SEV and Intel TDX,
run with encrypted/protected memory which creates a challenge
for devices that do not support DMA to it (no TDISP support).
For kernel-only DMA operations, swiotlb bounce buffering provides a
transparent solution by copying data through decrypted memory.
However, the only way to get this memory into userspace is via the DMA
API's dma_alloc_pages()/dma_mmap_pages() type interfaces which limits
the use of the memory to a single DMA device, and is incompatible with
pin_user_pages().
These limitations are particularly problematic for the RDMA subsystem
which makes heavy use of pin_user_pages() and expects flexible memory
usage between many different DMA devices.
This patch series enables userspace to explicitly request decrypted
(shared) memory allocations from the dma-buf system heap.
Userspace can mmap this memory and pass the dma-buf fd to other
existing importers such as RDMA or DRM devices to access the
memory. The DMA API is improved to allow the dma heap exporter to DMA
map the shared memory to each importing device.
Jiri Pirko (5):
dma-mapping: avoid random addr value print out on error path
dma-mapping: introduce DMA_ATTR_CC_DECRYPTED for pre-decrypted memory
dma-buf: heaps: use designated initializer for exp_info
dma-buf: heaps: allow heap to specify valid heap flags
dma-buf: heaps: system: add an option to allocate explicitly decrypted
memory
drivers/dma-buf/dma-heap.c | 5 +-
drivers/dma-buf/heaps/cma_heap.c | 7 ++-
drivers/dma-buf/heaps/system_heap.c | 96 ++++++++++++++++++++++++++---
include/linux/dma-heap.h | 3 +
include/linux/dma-mapping.h | 7 +++
include/trace/events/dma.h | 3 +-
include/uapi/linux/dma-heap.h | 12 +++-
kernel/dma/direct.h | 14 ++++-
8 files changed, 128 insertions(+), 19 deletions(-)
--
2.51.1
^ permalink raw reply
* [PATCH v2] KVM: x86: synthesize CPUID bits only if CPU capability is set
From: Carlos López @ 2026-02-09 15:31 UTC (permalink / raw)
To: seanjc, bp, kvm
Cc: linux-coco, jmattson, binbin.wu, Carlos López, Paolo Bonzini,
Thomas Gleixner, Ingo Molnar, Dave Hansen,
maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT), H. Peter Anvin,
open list:X86 ARCHITECTURE (32-BIT AND 64-BIT)
KVM incorrectly synthesizes CPUID bits for KVM-only leaves, as the
following branch in kvm_cpu_cap_init() is never taken:
if (leaf < NCAPINTS)
kvm_cpu_caps[leaf] &= kernel_cpu_caps[leaf];
This means that bits set via SYNTHESIZED_F() for KVM-only leaves are
unconditionally set. This for example can cause issues for SEV-SNP
guests running on Family 19h CPUs, as TSA_SQ_NO and TSA_L1_NO are
always enabled by KVM in 80000021[ECX]. When userspace issues a
SNP_LAUNCH_UPDATE command to update the CPUID page for the guest, SNP
firmware will explicitly reject the command if the page sets sets these
bits on vulnerable CPUs.
To fix this, check in SYNTHESIZED_F() that the corresponding X86
capability is set before adding it to to kvm_cpu_cap_features.
Fixes: 31272abd5974 ("KVM: SVM: Advertise TSA CPUID bits to guests")
Link: https://lore.kernel.org/all/20260208164233.30405-1-clopez@suse.de/
Signed-off-by: Carlos López <clopez@suse.de>
---
v2: fix SYNTHESIZED_F() instead of using SCATTERED_F() for TSA bits
arch/x86/kvm/cpuid.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 88a5426674a1..5f41924987c7 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -770,7 +770,10 @@ do { \
#define SYNTHESIZED_F(name) \
({ \
kvm_cpu_cap_synthesized |= feature_bit(name); \
- F(name); \
+ \
+ BUILD_BUG_ON(X86_FEATURE_##name >= MAX_CPU_FEATURES); \
+ if (boot_cpu_has(X86_FEATURE_##name)) \
+ F(name); \
})
/*
--
2.51.0
^ permalink raw reply related
* Re: [PATCH v2 0/2] x86/virt/tdx: Print TDX module version to dmesg
From: Kiryl Shutsemau @ 2026-02-09 10:46 UTC (permalink / raw)
To: Dave Hansen
Cc: Verma, Vishal L, dave.hansen, kvm@vger.kernel.org,
linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org,
Gao, Chao, Edgecombe, Rick P, Huang, Kai, bp@alien8.de,
mingo@redhat.com, Williams, Dan J, tglx@linutronix.de,
hpa@zytor.com, x86@kernel.org
In-Reply-To: <2235a1cf-7ccf-4134-80b5-8056537c6d33@intel.com>
On Fri, Feb 06, 2026 at 07:10:58AM -0800, Dave Hansen wrote:
> On 2/6/26 03:39, Kiryl Shutsemau wrote:
> >> Hi Kiryl, just wanted to check on the plan for this, I didn't see it
> >> merged in tip.git x86/tdx (or any other tip branch). Were you planning
> >> to take it through x86/tdx? Can I help with anything to move it along?
> > I guess it has to wait after the merge window at this point.
> >
> > Dave, could you queue this after -rc1 is tagged?
>
> Sure.
>
> Is there any other TDX stuff that needs to get picked up at the same
> time that's been languishing?
Nothing on my side.
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [RFC PATCH v5 20/45] KVM: x86/mmu: Allocate/free S-EPT pages using tdx_{alloc,free}_control_page()
From: Huang, Kai @ 2026-02-09 10:41 UTC (permalink / raw)
To: seanjc@google.com, Zhao, Yan Y
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Li, Xiaoyao,
dave.hansen@linux.intel.com, kas@kernel.org,
binbin.wu@linux.intel.com, mingo@redhat.com, pbonzini@redhat.com,
ackerleytng@google.com, linux-kernel@vger.kernel.org,
Yamahata, Isaku, sagis@google.com, tglx@kernel.org,
Edgecombe, Rick P, bp@alien8.de, Annapurve, Vishal,
x86@kernel.org
In-Reply-To: <aYYCOiMvWfSJR1AL@google.com>
>
> Option #2 would be to immediately free the page in tdx_sept_reclaim_private_sp(),
> so that pages that freed via handle_removed_pt() don't defer freeing the S-EPT
> page table (which, IIUC, is safe since the TDX-Module forces TLB flushes and exits).
>
> I really, really don't like this option (if it even works).
>
> diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> index ae7b9beb3249..4726011ad624 100644
> --- a/arch/x86/kvm/vmx/tdx.c
> +++ b/arch/x86/kvm/vmx/tdx.c
> @@ -2014,7 +2014,15 @@ static void tdx_sept_reclaim_private_sp(struct kvm *kvm, gfn_t gfn,
> */
> if (KVM_BUG_ON(is_hkid_assigned(to_kvm_tdx(kvm)), kvm) ||
> tdx_reclaim_page(virt_to_page(sp->external_spt)))
> - sp->external_spt = NULL;
> + goto out;
> +
> + /*
> + * Immediately free the control page, as the TDX subsystem doesn't
> + * support freeing pages from RCU callbacks.
> + */
> + tdx_free_control_page((unsigned long)sp->external_spt);
> +out:
> + sp->external_spt = NULL;
> }
>
> void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode,
I don't think this is so bad, given we already have a bunch of
is_mirror_sp(sp)
kvm_x86_call(xx_external_spt)(..);
in TDP MMU code?
I suppose this won't make a lot of difference, but does below make you
slightly happier?
diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
index 3181406c5e0b..3588265098a8 100644
--- a/arch/x86/kvm/mmu/tdp_mmu.c
+++ b/arch/x86/kvm/mmu/tdp_mmu.c
@@ -55,8 +55,7 @@ void kvm_mmu_uninit_tdp_mmu(struct kvm *kvm)
static void tdp_mmu_free_sp(struct kvm_mmu_page *sp)
{
- if (sp->external_spt)
- kvm_x86_call(free_external_sp)((unsigned long)sp-
>external_spt);
+ WARN_ON_ONCE(sp->external_spt);
free_page((unsigned long)sp->spt);
kmem_cache_free(mmu_page_header_cache, sp);
}
@@ -457,8 +456,17 @@ static void handle_removed_pt(struct kvm *kvm,
tdp_ptep_t pt, bool shared)
old_spte, FROZEN_SPTE, level, shared);
}
- if (is_mirror_sp(sp))
+ if (is_mirror_sp(sp)) {
kvm_x86_call(reclaim_external_sp)(kvm, base_gfn, sp);
+ /*
+ * Immediately free the control page, as the TDX subsystem
doesn't
+ * support freeing pages from RCU callbacks.
+ */
+ if (sp->external_spt) {
+ kvm_x86_call(free_external_sp)((unsigned long)sp-
>external_spt);
+ sp->external_spt = NULL;
+ }
+ }
call_rcu(&sp->rcu_head, tdp_mmu_free_sp_rcu_callback);
}
^ permalink raw reply related
* Re: [RFC PATCH v5 22/45] KVM: TDX: Get/put PAMT pages when (un)mapping private memory
From: Huang, Kai @ 2026-02-09 10:33 UTC (permalink / raw)
To: seanjc@google.com, Edgecombe, Rick P
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Li, Xiaoyao,
Zhao, Yan Y, dave.hansen@linux.intel.com,
linux-kernel@vger.kernel.org, kas@kernel.org,
binbin.wu@linux.intel.com, pbonzini@redhat.com, mingo@redhat.com,
Yamahata, Isaku, ackerleytng@google.com, tglx@kernel.org,
Annapurve, Vishal, sagis@google.com, bp@alien8.de, x86@kernel.org
In-Reply-To: <aYZ2qft-akOYwkOk@google.com>
On Fri, 2026-02-06 at 15:18 -0800, Sean Christopherson wrote:
> On Fri, Feb 06, 2026, Rick P Edgecombe wrote:
> > On Fri, 2026-02-06 at 08:03 -0800, Sean Christopherson wrote:
> > > > If this external cache is for PAMT pages allocation for guest pages only,
> > > > here
> > > > the min count should be 1 instead of PT64_ROOT_MAX_LEVEL?
> > >
> > > Oh! Right. Hmm, with that in mind, it seems like topup_external_cache()
> > > isn't
> > > quite the right interace. It's not at all clear that, unlike the other
> > > caches,
> > > the DPAMT cache isn't tied to the page tables, it's tied to the physical
> > > memory
> > > being mapped into the guest.
> > >
> > > At the very least, it seems like we should drop the @min parameter?
> > >
> > > int (*topup_external_cache)(struct kvm *kvm, struct kvm_vcpu *vcpu);
> > >
> > > Though if someone has a name that better captures what the cache is used for,
> > > without bleeding too many details into common x86...
> >
> > From the TDX perspective we have 4 types of pages that are needed to service
> > faults:
> > 1. "Control pages" (i.e. external page tables themselves)
> > 2. Private guest memory pages
> > 3. DPAMT backing pages for control pages
> > 4. DPAMT backing pages for private pages
> >
> > (3) is totally hidden now, but we need a hook to allocate (4). But from core
> > MMU's perspective we hide the existence of DPAMT backing pages. So we don't want
> > to leak that concept.
>
> Heh, there is no way around that. Common KVM needs to know that the cache is
> tied to mapping a page into the guest, otherwise the parameters don't make any
> sense whatsoever. All we can do is minimize the bleeding.
Actually, maybe we can even get rid of the DPAMT cache for the actual
private pages w/o introducing new field to 'kvm_mmu_page':
The point is:
Once we know the PFN and the actual mapping level, we can know whether we
need DPAMT pages for that PFN. If we can know outside of MMU lock, then
we can call tdx_pamt_get(PFN) directly w/o needing the "cache".
In the fault path, we already know the PFN after kvm_mmu_faultin_pfn(),
which is outside of MMU lock.
What we still don't know is the actual mapping level, which is currently
done in kvm_tdp_mmu_map() via kvm_mmu_hugepage_adjust().
However I don't see why we cannot move kvm_mmu_hugepage_adjust() out of it
to, e.g., right after kvm_mmu_faultin_pfn()?
If we can do this, then AFAICT we can just do:
r = kvm_x86_call(prepare_pfn)(vcpu, fault, pfn);
in which we can just call tdx_pamt_get(pfn) based on the mapping level?
Similar can be done for kvm_tdp_mmu_map_private_pfn() which already takes
the 'pfn' as parameter.
For the split path, we obviously can also know the 'pfn' from the huge SPTE.
I kinda do wish we could get rid of the new 'struct tdx_pamt_cache' pool if
possible.
Anything I missed?
^ permalink raw reply
* Re: [RFC PATCH v5 20/45] KVM: x86/mmu: Allocate/free S-EPT pages using tdx_{alloc,free}_control_page()
From: Yan Zhao @ 2026-02-09 9:25 UTC (permalink / raw)
To: Sean Christopherson
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Paolo Bonzini, linux-kernel, linux-coco, kvm,
Kai Huang, Rick Edgecombe, Vishal Annapurve, Ackerley Tng,
Sagi Shahar, Binbin Wu, Xiaoyao Li, Isaku Yamahata
In-Reply-To: <aYYCOiMvWfSJR1AL@google.com>
On Fri, Feb 06, 2026 at 07:01:14AM -0800, Sean Christopherson wrote:
> > (1) alloc
> > tdx_alloc_control_page
> > __tdx_alloc_control_page
> > __tdx_pamt_get
> > spin_lock(&pamt_lock) ==> under process context
> > spin_unlock(&pamt_lock)
> >
> > (2) free
> > tdp_mmu_free_sp_rcu_callback
> > tdp_mmu_free_sp
> > kvm_x86_call(free_external_sp)
> > tdx_free_control_page
> > __tdx_free_control_page
> > __tdx_pamt_put
> > spin_lock(&pamt_lock) ==> under softirq context
> > spin_unlock(&pamt_lock)
> >
> > So, invoking __tdx_pamt_put() in the RCU callback triggers deadlock warning
> > (see the bottom for details).
>
> Hrm. I can think of two options. Option #1 would be to use a raw spinlock and
> disable IRQs:
>
> diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> index 823ec092b4e4..6348085d7dcb 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.c
> +++ b/arch/x86/virt/vmx/tdx/tdx.c
> @@ -2246,7 +2246,7 @@ static u64 tdh_phymem_pamt_remove(u64 pfn, u64 *pamt_pa_array)
> }
>
> /* Serializes adding/removing PAMT memory */
> -static DEFINE_SPINLOCK(pamt_lock);
> +static DEFINE_RAW_SPINLOCK(pamt_lock);
>
> /* Bump PAMT refcount for the given page and allocate PAMT memory if needed */
> int __tdx_pamt_get(u64 pfn, struct tdx_pamt_cache *cache)
> @@ -2272,7 +2272,7 @@ int __tdx_pamt_get(u64 pfn, struct tdx_pamt_cache *cache)
> if (ret)
> goto out_free;
>
> - scoped_guard(spinlock, &pamt_lock) {
> + scoped_guard(raw_spinlock_irqsave, &pamt_lock) {
> /*
> * Lost race to other tdx_pamt_add(). Other task has already allocated
> * PAMT memory for the HPA.
> @@ -2348,7 +2348,7 @@ void __tdx_pamt_put(u64 pfn)
> if (!atomic_dec_and_test(pamt_refcount))
> return;
>
> - scoped_guard(spinlock, &pamt_lock) {
> + scoped_guard(raw_spinlock_irqsave, &pamt_lock) {
> /* Lost race with tdx_pamt_get(). */
> if (atomic_read(pamt_refcount))
> return;
This option can get rid of the warning.
However, given the pamt_lock is a global lock, which may be acquired even in the
softirq context, not sure if this irq disabled version is good.
For your reference, I measured some test data by concurrently launching and
destroying 4 TDs for 3 rounds:
t0 ---------------------
scoped_guard(spinlock, &pamt_lock) { |->T1=t1-t0 |
t1 ---------- |
... |
t2 ---------- |->T3=t4-t0
tdh_phymem_pamt_add/remove() |->T2=t3-t2 |
t3 ---------- |
... |
t4 ---------------------
}
(1) for __tdx_pamt_get()
avg us min us max us
------|---------------------------
T1 | 4 0 69
T2 | 4 2 18
T3 | 10 3 83
(2) for__tdx_pamt_put()
avg us min us max us
------|---------------------------
T1 | 0 0 5
T2 | 2 1 11
T3 | 3 2 15
> Option #2 would be to immediately free the page in tdx_sept_reclaim_private_sp(),
> so that pages that freed via handle_removed_pt() don't defer freeing the S-EPT
> page table (which, IIUC, is safe since the TDX-Module forces TLB flushes and exits).
>
> I really, really don't like this option (if it even works).
I don't like its asymmetry with tdx_sept_link_private_spt().
However, do you think it would be good to have the PAMT pages of the sept pages
allocated from (*topup_private_mapping_cache) [1]?
private_mapping could also include non-leaf mappings.
So, we could invoke tdx_pamt_get() in tdx_sept_link_private_spt() for symmetry.
[1] https://lore.kernel.org/all/aYZ2qft-akOYwkOk@google.com/
> diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> index ae7b9beb3249..4726011ad624 100644
> --- a/arch/x86/kvm/vmx/tdx.c
> +++ b/arch/x86/kvm/vmx/tdx.c
> @@ -2014,7 +2014,15 @@ static void tdx_sept_reclaim_private_sp(struct kvm *kvm, gfn_t gfn,
> */
> if (KVM_BUG_ON(is_hkid_assigned(to_kvm_tdx(kvm)), kvm) ||
> tdx_reclaim_page(virt_to_page(sp->external_spt)))
> - sp->external_spt = NULL;
> + goto out;
> +
> + /*
> + * Immediately free the control page, as the TDX subsystem doesn't
> + * support freeing pages from RCU callbacks.
> + */
> + tdx_free_control_page((unsigned long)sp->external_spt);
> +out:
> + sp->external_spt = NULL;
> }
>
> void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode,
> --
^ permalink raw reply
* Re: [RFC PATCH v5 22/45] KVM: TDX: Get/put PAMT pages when (un)mapping private memory
From: Edgecombe, Rick P @ 2026-02-06 23:19 UTC (permalink / raw)
To: seanjc@google.com
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Huang, Kai,
Li, Xiaoyao, Zhao, Yan Y, dave.hansen@linux.intel.com,
linux-kernel@vger.kernel.org, kas@kernel.org,
binbin.wu@linux.intel.com, pbonzini@redhat.com, mingo@redhat.com,
Yamahata, Isaku, ackerleytng@google.com, tglx@kernel.org,
sagis@google.com, bp@alien8.de, Annapurve, Vishal, x86@kernel.org
In-Reply-To: <aYZ2qft-akOYwkOk@google.com>
On Fri, 2026-02-06 at 15:18 -0800, Sean Christopherson wrote:
> How about?
>
> (*topup_private_mapping_cache)
>
> Because it's not just "private memory" it's specifically the mapping. E.g. for
> the hugepage split case, the primary memory is already assigned and mapped into
> the guest, but a topup is still needed because KVM is creating a new/different
> mapping.
Sure.
^ permalink raw reply
* Re: [RFC PATCH v5 22/45] KVM: TDX: Get/put PAMT pages when (un)mapping private memory
From: Sean Christopherson @ 2026-02-06 23:18 UTC (permalink / raw)
To: Rick P Edgecombe
Cc: Yan Y Zhao, kvm@vger.kernel.org, linux-coco@lists.linux.dev,
Xiaoyao Li, Kai Huang, dave.hansen@linux.intel.com,
kas@kernel.org, binbin.wu@linux.intel.com, mingo@redhat.com,
pbonzini@redhat.com, ackerleytng@google.com,
linux-kernel@vger.kernel.org, Isaku Yamahata, sagis@google.com,
tglx@kernel.org, bp@alien8.de, Vishal Annapurve, x86@kernel.org
In-Reply-To: <b3ad6d9cce83681f548b35881ebad0c5bb4fed23.camel@intel.com>
On Fri, Feb 06, 2026, Rick P Edgecombe wrote:
> On Fri, 2026-02-06 at 08:03 -0800, Sean Christopherson wrote:
> > > If this external cache is for PAMT pages allocation for guest pages only,
> > > here
> > > the min count should be 1 instead of PT64_ROOT_MAX_LEVEL?
> >
> > Oh! Right. Hmm, with that in mind, it seems like topup_external_cache()
> > isn't
> > quite the right interace. It's not at all clear that, unlike the other
> > caches,
> > the DPAMT cache isn't tied to the page tables, it's tied to the physical
> > memory
> > being mapped into the guest.
> >
> > At the very least, it seems like we should drop the @min parameter?
> >
> > int (*topup_external_cache)(struct kvm *kvm, struct kvm_vcpu *vcpu);
> >
> > Though if someone has a name that better captures what the cache is used for,
> > without bleeding too many details into common x86...
>
> From the TDX perspective we have 4 types of pages that are needed to service
> faults:
> 1. "Control pages" (i.e. external page tables themselves)
> 2. Private guest memory pages
> 3. DPAMT backing pages for control pages
> 4. DPAMT backing pages for private pages
>
> (3) is totally hidden now, but we need a hook to allocate (4). But from core
> MMU's perspective we hide the existence of DPAMT backing pages. So we don't want
> to leak that concept.
Heh, there is no way around that. Common KVM needs to know that the cache is
tied to mapping a page into the guest, otherwise the parameters don't make any
sense whatsoever. All we can do is minimize the bleeding.
> The page we need is kind of like something to "prepare" the private page before
> installing it. It actually isn't that related to the mirror/external concept. So
> if we separate it from "external" and make it about installing private guest
> memory, it fits better conceptually I think. But it could be a bit confusing for
> other types of VMs who have to trace to see if anything special is happening
> inside the op for their private memory. In that case it could be:
>
> (*topup_private_mem_prepare_cache)(struct kvm_vcpu *vcpu)
topup + prepare is redundant and confusing.
> The core MMU doesn't know about DPAMT backing pages, but it does know about the
> set_external_spte op that consumes this cache. So how about the slightly
> misleading:
>
> (*topup_set_external_spte_cache)(struct kvm_vcpu *vcpu)
I really, really, want to avoid "SPTE", because the cache isn't for the SPTE in
any way, it's for the memory that's _pointed_ at by the SPTE. And the confusion
is exactly what prompted this thread: I forgot that it's not every SPTE in the
chain that needs DPAMT backing, it's only the page that's being mapped into the
guest.
How about?
(*topup_private_mapping_cache)
Because it's not just "private memory" it's specifically the mapping. E.g. for
the hugepage split case, the primary memory is already assigned and mapped into
the guest, but a topup is still needed because KVM is creating a new/different
mapping.
> It is easier for other VM types to ignore, and not that semantically wrong from
> what is happening on the TDX side.
^ permalink raw reply
* [PATCH v3 2/2] KVM: SEV: Restrict userspace return codes for KVM_HC_MAP_GPA_RANGE
From: Sagi Shahar @ 2026-02-06 22:28 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Dave Hansen, Kiryl Shutsemau,
Rick Edgecombe
Cc: Thomas Gleixner, Borislav Petkov, H. Peter Anvin, Michael Roth,
Tom Lendacky, x86, kvm, linux-kernel, linux-coco, Sagi Shahar
In-Reply-To: <20260206222829.3758171-1-sagis@google.com>
To align with the updated TDX api that allows userspace to request
that guests retry MAP_GPA operations, make sure that userspace is only
returning EINVAL or EAGAIN as possible error codes.
Signed-off-by: Sagi Shahar <sagis@google.com>
---
arch/x86/kvm/svm/sev.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index f59c65abe3cf..5f78e4c3eb5d 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -3722,9 +3722,13 @@ static int snp_rmptable_psmash(kvm_pfn_t pfn)
static int snp_complete_psc_msr(struct kvm_vcpu *vcpu)
{
+ u64 hypercall_ret = READ_ONCE(vcpu->run->hypercall.ret);
struct vcpu_svm *svm = to_svm(vcpu);
- if (vcpu->run->hypercall.ret)
+ if (!kvm_is_valid_map_gpa_range_ret(hypercall_ret))
+ return -EINVAL;
+
+ if (hypercall_ret)
set_ghcb_msr(svm, GHCB_MSR_PSC_RESP_ERROR);
else
set_ghcb_msr(svm, GHCB_MSR_PSC_RESP);
@@ -3815,10 +3819,14 @@ static void __snp_complete_one_psc(struct vcpu_svm *svm)
static int snp_complete_one_psc(struct kvm_vcpu *vcpu)
{
+ u64 hypercall_ret = READ_ONCE(vcpu->run->hypercall.ret);
struct vcpu_svm *svm = to_svm(vcpu);
struct psc_buffer *psc = svm->sev_es.ghcb_sa;
- if (vcpu->run->hypercall.ret) {
+ if (!kvm_is_valid_map_gpa_range_ret(hypercall_ret))
+ return -EINVAL;
+
+ if (hypercall_ret) {
snp_complete_psc(svm, VMGEXIT_PSC_ERROR_GENERIC);
return 1; /* resume guest */
}
--
2.53.0.rc2.204.g2597b5adb4-goog
^ permalink raw reply related
* [PATCH v3 1/2] KVM: TDX: Allow userspace to return errors to guest for MAPGPA
From: Sagi Shahar @ 2026-02-06 22:28 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Dave Hansen, Kiryl Shutsemau,
Rick Edgecombe
Cc: Thomas Gleixner, Borislav Petkov, H. Peter Anvin, Michael Roth,
Tom Lendacky, x86, kvm, linux-kernel, linux-coco,
Vishal Annapurve, Sagi Shahar
In-Reply-To: <20260206222829.3758171-1-sagis@google.com>
From: Vishal Annapurve <vannapurve@google.com>
MAPGPA request from TDX VMs gets split into chunks by KVM using a loop
of userspace exits until the complete range is handled.
In some cases userspace VMM might decide to break the MAPGPA operation
and continue it later. For example: in the case of intrahost migration
userspace might decide to continue the MAPGPA operation after the
migration is completed.
Allow userspace to signal to TDX guests that the MAPGPA operation should
be retried the next time the guest is scheduled.
This is potentially a breaking change since if userspace sets
hypercall.ret to a value other than EBUSY or EINVAL an EINVAL error code
will be returned to userspace. As of now QEMU never sets hypercall.ret
to a non-zero value after handling KVM_EXIT_HYPERCALL so this change
should be safe.
Signed-off-by: Vishal Annapurve <vannapurve@google.com>
Co-developed-by: Sagi Shahar <sagis@google.com>
Signed-off-by: Sagi Shahar <sagis@google.com>
---
Documentation/virt/kvm/api.rst | 3 +++
arch/x86/kvm/vmx/tdx.c | 15 +++++++++++++--
arch/x86/kvm/x86.h | 6 ++++++
3 files changed, 22 insertions(+), 2 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 01a3abef8abb..9978cd9d897e 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -8679,6 +8679,9 @@ block sizes is exposed in KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES as a
This capability, if enabled, will cause KVM to exit to userspace
with KVM_EXIT_HYPERCALL exit reason to process some hypercalls.
+Userspace may fail the hypercall by setting hypercall.ret to EINVAL
+or may request the hypercall to be retried the next time the guest run
+by setting hypercall.ret to EAGAIN.
Calling KVM_CHECK_EXTENSION for this capability will return a bitmask
of hypercalls that can be configured to exit to userspace.
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 2d7a4d52ccfb..056a44b9d78b 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1186,10 +1186,21 @@ static void __tdx_map_gpa(struct vcpu_tdx *tdx);
static int tdx_complete_vmcall_map_gpa(struct kvm_vcpu *vcpu)
{
+ u64 hypercall_ret = READ_ONCE(vcpu->run->hypercall.ret);
struct vcpu_tdx *tdx = to_tdx(vcpu);
- if (vcpu->run->hypercall.ret) {
- tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND);
+ if (hypercall_ret) {
+ if (hypercall_ret == EAGAIN) {
+ tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_RETRY);
+ } else if (vcpu->run->hypercall.ret == EINVAL) {
+ tdvmcall_set_return_code(
+ vcpu, TDVMCALL_STATUS_INVALID_OPERAND);
+ } else {
+ WARN_ON_ONCE(
+ kvm_is_valid_map_gpa_range_ret(hypercall_ret));
+ return -EINVAL;
+ }
+
tdx->vp_enter_args.r11 = tdx->map_gpa_next;
return 1;
}
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index fdab0ad49098..3d464d12423a 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -706,6 +706,12 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size,
unsigned int port, void *data, unsigned int count,
int in);
+static inline bool kvm_is_valid_map_gpa_range_ret(u64 hypercall_ret)
+{
+ return !hypercall_ret || hypercall_ret == EINVAL ||
+ hypercall_ret == EAGAIN;
+}
+
static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr)
{
return kvm->arch.hypercall_exit_enabled & BIT(hc_nr);
--
2.53.0.rc2.204.g2597b5adb4-goog
^ permalink raw reply related
* [PATCH v3 0/2] Extend KVM_HC_MAP_GPA_RANGE api to allow retry
From: Sagi Shahar @ 2026-02-06 22:28 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Dave Hansen, Kiryl Shutsemau,
Rick Edgecombe
Cc: Thomas Gleixner, Borislav Petkov, H. Peter Anvin, Michael Roth,
Tom Lendacky, x86, kvm, linux-kernel, linux-coco, Sagi Shahar
In some cases, userspace might decide to split MAP_GPA requests and
retry them the next time the guest runs. One common case is MAP_GPA
requests received right before intrahost migration when userspace
might decide to complete the request after the migration is complete
to reduce blackout time.
This is v3 of the series, v1[1] and v2[2] were posted as standalone
patches.
Changes from v2:
* Rebased on top of v6.19-rc8.
* Updated documentation.
* Restricted SNP error codes to match TDX restrictions.
[1] https://lore.kernel.org/kvm/20260114003015.1386066-1-sagis@google.com/
[2] https://lore.kernel.org/lkml/20260115225238.2837449-1-sagis@google.com/
Sagi Shahar (1):
KVM: SEV: Restrict userspace return codes for KVM_HC_MAP_GPA_RANGE
Vishal Annapurve (1):
KVM: TDX: Allow userspace to return errors to guest for MAPGPA
Documentation/virt/kvm/api.rst | 3 +++
arch/x86/kvm/svm/sev.c | 12 ++++++++++--
arch/x86/kvm/vmx/tdx.c | 15 +++++++++++++--
arch/x86/kvm/x86.h | 6 ++++++
4 files changed, 32 insertions(+), 4 deletions(-)
--
2.53.0.rc2.204.g2597b5adb4-goog
^ permalink raw reply
* [PATCH] crypto: ccp - allow callers to use HV-Fixed page API when SEV is disabled
From: Ashish Kalra @ 2026-02-06 21:26 UTC (permalink / raw)
To: thomas.lendacky, john.allen, herbert, davem, bp
Cc: linux-crypto, linux-kernel, linux-coco
From: Ashish Kalra <ashish.kalra@amd.com>
When SEV is disabled, the HV-Fixed page allocation call fails, which in
turn causes SFS initialization to fail.
Fix the HV-Fixed API so callers (for example, SFS) can use it even when
SEV is disabled by performing normal page allocation and freeing.
Fixes: e09701dcdd9c ("crypto: ccp - Add new HV-Fixed page allocation/free API")
Cc: stable@vger.kernel.org
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
drivers/crypto/ccp/sev-dev.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/drivers/crypto/ccp/sev-dev.c b/drivers/crypto/ccp/sev-dev.c
index 1cdadddb744e..0d90b5f6a454 100644
--- a/drivers/crypto/ccp/sev-dev.c
+++ b/drivers/crypto/ccp/sev-dev.c
@@ -1105,15 +1105,12 @@ struct page *snp_alloc_hv_fixed_pages(unsigned int num_2mb_pages)
{
struct psp_device *psp_master = psp_get_master_device();
struct snp_hv_fixed_pages_entry *entry;
- struct sev_device *sev;
unsigned int order;
struct page *page;
- if (!psp_master || !psp_master->sev_data)
+ if (!psp_master)
return NULL;
- sev = psp_master->sev_data;
-
order = get_order(PMD_SIZE * num_2mb_pages);
/*
@@ -1126,7 +1123,8 @@ struct page *snp_alloc_hv_fixed_pages(unsigned int num_2mb_pages)
* This API uses SNP_INIT_EX to transition allocated pages to HV_Fixed
* page state, fail if SNP is already initialized.
*/
- if (sev->snp_initialized)
+ if (psp_master->sev_data &&
+ ((struct sev_device *)psp_master->sev_data)->snp_initialized)
return NULL;
/* Re-use freed pages that match the request */
@@ -1162,7 +1160,7 @@ void snp_free_hv_fixed_pages(struct page *page)
struct psp_device *psp_master = psp_get_master_device();
struct snp_hv_fixed_pages_entry *entry, *nentry;
- if (!psp_master || !psp_master->sev_data)
+ if (!psp_master)
return;
/*
--
2.34.1
^ permalink raw reply related
* Re: [RFC PATCH v5 22/45] KVM: TDX: Get/put PAMT pages when (un)mapping private memory
From: Edgecombe, Rick P @ 2026-02-06 19:27 UTC (permalink / raw)
To: seanjc@google.com, Zhao, Yan Y
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Li, Xiaoyao,
Huang, Kai, dave.hansen@linux.intel.com, kas@kernel.org,
binbin.wu@linux.intel.com, mingo@redhat.com, pbonzini@redhat.com,
ackerleytng@google.com, linux-kernel@vger.kernel.org,
Yamahata, Isaku, sagis@google.com, tglx@kernel.org, bp@alien8.de,
Annapurve, Vishal, x86@kernel.org
In-Reply-To: <aYYQ7Vx95ZrsqwCv@google.com>
On Fri, 2026-02-06 at 08:03 -0800, Sean Christopherson wrote:
> > If this external cache is for PAMT pages allocation for guest pages only,
> > here
> > the min count should be 1 instead of PT64_ROOT_MAX_LEVEL?
>
> Oh! Right. Hmm, with that in mind, it seems like topup_external_cache()
> isn't
> quite the right interace. It's not at all clear that, unlike the other
> caches,
> the DPAMT cache isn't tied to the page tables, it's tied to the physical
> memory
> being mapped into the guest.
>
> At the very least, it seems like we should drop the @min parameter?
>
> int (*topup_external_cache)(struct kvm *kvm, struct kvm_vcpu *vcpu);
>
> Though if someone has a name that better captures what the cache is used for,
> without bleeding too many details into common x86...
From the TDX perspective we have 4 types of pages that are needed to service
faults:
1. "Control pages" (i.e. external page tables themselves)
2. Private guest memory pages
3. DPAMT backing pages for control pages
4. DPAMT backing pages for private pages
(3) is totally hidden now, but we need a hook to allocate (4). But from core
MMU's perspective we hide the existence of DPAMT backing pages. So we don't want
to leak that concept.
The page we need is kind of like something to "prepare" the private page before
installing it. It actually isn't that related to the mirror/external concept. So
if we separate it from "external" and make it about installing private guest
memory, it fits better conceptually I think. But it could be a bit confusing for
other types of VMs who have to trace to see if anything special is happening
inside the op for their private memory. In that case it could be:
(*topup_private_mem_prepare_cache)(struct kvm_vcpu *vcpu)
The core MMU doesn't know about DPAMT backing pages, but it does know about the
set_external_spte op that consumes this cache. So how about the slightly
misleading:
(*topup_set_external_spte_cache)(struct kvm_vcpu *vcpu)
It is easier for other VM types to ignore, and not that semantically wrong from
what is happening on the TDX side.
^ permalink raw reply
* Re: [PATCH v2 0/2] x86/virt/tdx: Print TDX module version to dmesg
From: Edgecombe, Rick P @ 2026-02-06 18:35 UTC (permalink / raw)
To: Li, Xiaoyao, Hansen, Dave, kas@kernel.org, Verma, Vishal L,
dave.hansen@linux.intel.com
Cc: Gao, Chao, x86@kernel.org, bp@alien8.de, Huang, Kai,
hpa@zytor.com, mingo@redhat.com, Williams, Dan J,
tglx@linutronix.de, kvm@vger.kernel.org,
linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org
In-Reply-To: <2235a1cf-7ccf-4134-80b5-8056537c6d33@intel.com>
On Fri, 2026-02-06 at 07:10 -0800, Dave Hansen wrote:
> Is there any other TDX stuff that needs to get picked up at the same
> time that's been languishing?
Xiaoyao is going to send a rebase of this after RC1:
https://lore.kernel.org/kvm/20250715091312.563773-1-xiaoyao.li@intel.com/
It has pretty wide agreement, and ack's from the KVM side. Also, we have
internal branches that are carrying forms of it. So if we merge it now we can
have less dependencies later.
^ permalink raw reply
* Re: [RFC PATCH v5 08/45] KVM: x86/mmu: Propagate mirror SPTE removal to S-EPT in handle_changed_spte()
From: Sean Christopherson @ 2026-02-06 17:41 UTC (permalink / raw)
To: Yan Zhao
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Paolo Bonzini, linux-kernel, linux-coco, kvm,
Kai Huang, Rick Edgecombe, Vishal Annapurve, Ackerley Tng,
Sagi Shahar, Binbin Wu, Xiaoyao Li, Isaku Yamahata
In-Reply-To: <aYVPN5M7QQwu/r/n@yzhao56-desk.sh.intel.com>
On Fri, Feb 06, 2026, Yan Zhao wrote:
> On Thu, Feb 05, 2026 at 02:33:16PM -0800, Sean Christopherson wrote:
> > On Thu, Feb 05, 2026, Yan Zhao wrote:
> > > > > > if (was_present && !was_leaf &&
> > > > > > (is_leaf || !is_present || WARN_ON_ONCE(pfn_changed)))
> > > > > > handle_removed_pt(kvm, spte_to_child_pt(old_spte, level), shared);
> > > > > > + else if (was_leaf && is_mirror_sptep(sptep) && !is_leaf)
> > > > > Should we check !is_present instead of !is_leaf?
> > > > > e.g. a transition from a present leaf entry to a present non-leaf entry could
> > > > > also trigger this if case.
> > > >
> > > > No, the !is_leaf check is very intentional. At this point in the series, S-EPT
> > > > doesn't support hugepages. If KVM manages to install a leaf SPTE and replaces
> > > > that SPTE with a non-leaf SPTE, then we absolutely want the KVM_BUG_ON() in
> > > > tdx_sept_remove_private_spte() to fire:
> > > >
> > > > /* TODO: handle large pages. */
> > > > if (KVM_BUG_ON(level != PG_LEVEL_4K, kvm))
> > > > return -EIO;
> > > But the op is named remove_external_spte().
> > > And the check of "level != PG_LEVEL_4K" is for removing large leaf entries.
> >
> > I agree that the naming at this point in the series is unfortunate, but I don't
> > see it as outright wrong. That the TDP MMU could theoretically replace the leaf
> > SPTE with a non-leaf SPTE doesn't change the fact that the old leaf SPTE *is*
> > being removed.
> Hmm, I can't agree with that. But I won't insist if you think it's ok :)
If the code is read through a TDX lens, then I agree, it's seems wrong. Because
then you *know* that TDX doesn't support back-to-back remove()=>add() operations
to handle a page split.
But from a TDP MMU perspective, this is entirely logical (ignoring that
link_external_spt() is gone at this point in the series).
else if (was_leaf && is_mirror_sptep(sptep) && !is_leaf) {
kvm_x86_call(remove_external_spte)(kvm, gfn, level, old_spte);
/*
* Link the new page table if a hugepage is being split, i.e.
* if a leaf SPTE is being replaced with a non-leaf SPTE.
*/
if (is_present)
kvm_x86_call(link_external_spt)(kvm, gfn, level, ...);
}
And that is *exactly* why I want to get rid of the hyper-specific kvm_x86_ops
hooks. They bleed all kinds of implementation details all over the TDP MMU, which
makes it difficult to read and understand the relevant TDP MMU code if you don't
already know the TDX rules. And I absolutely do not want to effectively require
others to understand TDX's rules to be able to make changes to the TDP MMU.
> > > Relying on the TDX code's lockdep_assert_held_write() for warning seems less
> > > clear than having an explicit check here.
> >
> > ...that's TDX's responsibility to enforce, and I don't see any justification for
> > something more than a lockdep assertion. As I've said elsewhere, several times,
> My concern is that handle_changed_spte() can be invoked by callers other than
> tdp_mmu_set_spte(). e.g.,
>
> tdp_mmu_set_spte_atomic
> | __tdp_mmu_set_spte_atomic
> | | kvm_x86_call(set_external_spte)
> | handle_changed_spte
> | kvm_x86_call(set_external_spte)
>
> When !is_frozen_spte(new_spte) and was_leaf, set_external_spte() may be invoked
> twice for splitting under shared mmu_lock.
But we don't support that yet. I was going to punt dealing with that to the future,
but now that I've looked at it, I honestly think the problem is that I didn't go
far enough with the cleanup. I shouldn't have hedged.
What I said in "KVM: TDX: Drop kvm_x86_ops.link_external_spt(), use .set_external_spte()
for all":
: Ideally, KVM would provide a single pair of hooks to set S-EPT entries,
: one hook for setting SPTEs under write-lock and another for settings SPTEs
: under read-lock (e.g. to ensure the entire operation is "atomic", to allow
: for failure, etc.). Sadly, TDX's requirement that all child S-EPT entries
: are removed before the parent makes that impractical: the TDP MMU
: deliberately prunes non-leaf SPTEs and _then_ processes its children, thus
: making it quite important for the TDP MMU to differentiate between zapping
: leaf and non-leaf S-EPT entries.
isn't quite right. Ideally, KVM would provide *one* hook to set S-EPT entries.
Because the API to actually *set* the external SPTE shouldn't be any different
for read-lock vs. write-lost. I.e. the child S-EPT case remains the sole
exception. More below.
> Therefore, I think it would be better to check for !shared and only invoke
> set_external_spte() when !shared.
No. I am dead set against conditionally invoking set_external_spte() based on
shared vs. exclusive. The rules for how SPTE updates are forwarded to TDX, and
how TDX reacts to those updates, need to be defined purely on state transitions,
i.e. on old_spte vs. new_spte.
Actually, even that isn't a strong enough statement. *All* updates need to be
forwarded to TDX, and then TDX can react as appropriate.
> BTW: in the patch log, you mentioned that
>
> : Invoke .remove_external_spte() in handle_changed_spte() as appropriate
> : instead of relying on callers to do the right thing. Relying on callers
> : to invoke .remove_external_spte() is confusing and brittle, e.g. subtly
> : relies tdp_mmu_set_spte_atomic() never removing SPTEs, and removing an
> : S-EPT entry in tdp_mmu_set_spte() is bizarre (yeah, the VM is bugged so
> : it doesn't matter in practice, but it's still weird).
>
> However, when tdp_mmu_set_spte_atomic() removes SPTEs in the future, it will
> still need to follow the sequence in __tdp_mmu_set_spte_atomic() :
> 1. freeze, 2. set_external_spte(). 3. set the new_spte 4. handle_changed_spte().
>
> So, do you think we should leave set_external_spte() in tdp_mmu_set_spte() for
> exclusive mmu_lock scenarios instead of moving it to handle_changed_spte()?
Nope, as above, 100% the opposite. Over ~3 patches, e.g.
1. Drop the KVM_BUG_ON()s or move them to TDX
2. Morph the !is_frozen_spte() check into a KVM_MMU_WARN_ON()
3. Rework the code to rely on __handle_changed_spte() to propagate writes to S-EPT
That way, the _only_ path that updates external SPTEs is this:
if (was_present && !was_leaf &&
(is_leaf || !is_present || WARN_ON_ONCE(pfn_changed)))
handle_removed_pt(kvm, spte_to_child_pt(old_spte, level), shared);
else if (is_mirror_sptep(sptep))
return kvm_x86_call(set_external_spte)(kvm, gfn, old_spte,
new_spte, level);
which is fully aligned with handle_changed_spte()'s role for !mirror roots: it
exists to react to changes (the sole exception to that being aging SPTEs, which
is a special case).
Compile-tested only.
---
arch/x86/kvm/mmu/tdp_mmu.c | 118 ++++++++++++++++++-------------------
1 file changed, 59 insertions(+), 59 deletions(-)
diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
index 847f2fcb6740..33a321aedac0 100644
--- a/arch/x86/kvm/mmu/tdp_mmu.c
+++ b/arch/x86/kvm/mmu/tdp_mmu.c
@@ -464,7 +464,7 @@ static void handle_removed_pt(struct kvm *kvm, tdp_ptep_t pt, bool shared)
}
/**
- * handle_changed_spte - handle bookkeeping associated with an SPTE change
+ * __handle_changed_spte - handle bookkeeping associated with an SPTE change
* @kvm: kvm instance
* @as_id: the address space of the paging structure the SPTE was a part of
* @sptep: pointer to the SPTE
@@ -480,9 +480,9 @@ static void handle_removed_pt(struct kvm *kvm, tdp_ptep_t pt, bool shared)
* dirty logging updates are handled in common code, not here (see make_spte()
* and fast_pf_fix_direct_spte()).
*/
-static void handle_changed_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep,
- gfn_t gfn, u64 old_spte, u64 new_spte,
- int level, bool shared)
+static int __handle_changed_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep,
+ gfn_t gfn, u64 old_spte, u64 new_spte,
+ int level, bool shared)
{
bool was_present = is_shadow_present_pte(old_spte);
bool is_present = is_shadow_present_pte(new_spte);
@@ -518,7 +518,7 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep,
}
if (old_spte == new_spte)
- return;
+ return 0;
trace_kvm_tdp_mmu_spte_changed(as_id, gfn, level, old_spte, new_spte);
@@ -547,7 +547,7 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep,
"a temporary frozen SPTE.\n"
"as_id: %d gfn: %llx old_spte: %llx new_spte: %llx level: %d",
as_id, gfn, old_spte, new_spte, level);
- return;
+ return 0;
}
if (is_leaf != was_leaf)
@@ -559,30 +559,31 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep,
* SPTE being converted to a hugepage (leaf) or being zapped. Shadow
* pages are kernel allocations and should never be migrated.
*
- * When modifying leaf entries in mirrored page tables, propagate the
- * changes to the external SPTE. Bug the VM on failure, as callers
- * aren't prepared to handle errors, e.g. due to lock contention in the
- * TDX-Module. Note, changes to non-leaf mirror SPTEs are handled by
- * handle_removed_pt() (the TDX-Module requires that child entries are
- * removed before the parent SPTE), and changes to non-present mirror
- * SPTEs are handled by __tdp_mmu_set_spte_atomic() (KVM needs to set
- * the external SPTE while the mirror SPTE is frozen so that installing
- * a new SPTE is effectively an atomic operation).
+ * When modifying leaf entries in mirrored page tables, propagate all
+ * changes to the external SPTE.
*/
if (was_present && !was_leaf &&
(is_leaf || !is_present || WARN_ON_ONCE(pfn_changed)))
handle_removed_pt(kvm, spte_to_child_pt(old_spte, level), shared);
- else if (was_leaf && is_mirror_sptep(sptep))
- KVM_BUG_ON(kvm_x86_call(set_external_spte)(kvm, gfn, old_spte,
- new_spte, level), kvm);
+ else if (is_mirror_sptep(sptep))
+ return kvm_x86_call(set_external_spte)(kvm, gfn, old_spte,
+ new_spte, level);
+
+ return 0;
+}
+
+static void handle_changed_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep,
+ gfn_t gfn, u64 old_spte, u64 new_spte,
+ int level, bool shared)
+{
+ KVM_BUG_ON(__handle_changed_spte(kvm, as_id, sptep, gfn, old_spte,
+ new_spte, level, shared), kvm);
}
static inline int __must_check __tdp_mmu_set_spte_atomic(struct kvm *kvm,
struct tdp_iter *iter,
u64 new_spte)
{
- u64 *raw_sptep = rcu_dereference(iter->sptep);
-
/*
* The caller is responsible for ensuring the old SPTE is not a FROZEN
* SPTE. KVM should never attempt to zap or manipulate a FROZEN SPTE,
@@ -591,40 +592,6 @@ static inline int __must_check __tdp_mmu_set_spte_atomic(struct kvm *kvm,
*/
WARN_ON_ONCE(iter->yielded || is_frozen_spte(iter->old_spte));
- if (is_mirror_sptep(iter->sptep) && !is_frozen_spte(new_spte)) {
- int ret;
-
- /*
- * KVM doesn't currently support zapping or splitting mirror
- * SPTEs while holding mmu_lock for read.
- */
- if (KVM_BUG_ON(is_shadow_present_pte(iter->old_spte), kvm) ||
- KVM_BUG_ON(!is_shadow_present_pte(new_spte), kvm))
- return -EBUSY;
-
- /*
- * Temporarily freeze the SPTE until the external PTE operation
- * has completed, e.g. so that concurrent faults don't attempt
- * to install a child PTE in the external page table before the
- * parent PTE has been written.
- */
- if (!try_cmpxchg64(raw_sptep, &iter->old_spte, FROZEN_SPTE))
- return -EBUSY;
-
- /*
- * Update the external PTE. On success, set the mirror SPTE to
- * the desired value. On failure, restore the old SPTE so that
- * the SPTE isn't frozen in perpetuity.
- */
- ret = kvm_x86_call(set_external_spte)(kvm, iter->gfn, iter->old_spte,
- new_spte, iter->level);
- if (ret)
- __kvm_tdp_mmu_write_spte(iter->sptep, iter->old_spte);
- else
- __kvm_tdp_mmu_write_spte(iter->sptep, new_spte);
- return ret;
- }
-
/*
* Note, fast_pf_fix_direct_spte() can also modify TDP MMU SPTEs and
* does not hold the mmu_lock. On failure, i.e. if a different logical
@@ -632,7 +599,7 @@ static inline int __must_check __tdp_mmu_set_spte_atomic(struct kvm *kvm,
* the current value, so the caller operates on fresh data, e.g. if it
* retries tdp_mmu_set_spte_atomic()
*/
- if (!try_cmpxchg64(raw_sptep, &iter->old_spte, new_spte))
+ if (!try_cmpxchg64(rcu_dereference(iter->sptep), &iter->old_spte, new_spte))
return -EBUSY;
return 0;
@@ -663,14 +630,44 @@ static inline int __must_check tdp_mmu_set_spte_atomic(struct kvm *kvm,
lockdep_assert_held_read(&kvm->mmu_lock);
- ret = __tdp_mmu_set_spte_atomic(kvm, iter, new_spte);
+ /* KVM should never freeze SPTEs using higher level APIs. */
+ KVM_MMU_WARN_ON(is_frozen_spte(new_spte));
+
+ /*
+ * Temporarily freeze the SPTE until the external PTE operation has
+ * completed (unless the new SPTE itself will be frozen), e.g. so that
+ * concurrent faults don't attempt to install a child PTE in the
+ * external page table before the parent PTE has been written, or try
+ * to re-install a page table before the old one was removed.
+ */
+ if (is_mirror_sptep(iter->sptep))
+ ret = __tdp_mmu_set_spte_atomic(kvm, iter, FROZEN_SPTE);
+ else
+ ret = __tdp_mmu_set_spte_atomic(kvm, iter, new_spte);
if (ret)
return ret;
- handle_changed_spte(kvm, iter->as_id, iter->sptep, iter->gfn,
- iter->old_spte, new_spte, iter->level, true);
+ ret = __handle_changed_spte(kvm, iter->as_id, iter->sptep, iter->gfn,
+ iter->old_spte, new_spte, iter->level, true);
- return 0;
+ /*
+ * Unfreeze the mirror SPTE. If updating the external SPTE failed,
+ * restore the old SPTE so that the SPTE isn't frozen in perpetuity,
+ * otherwise set the mirror SPTE to the new desired value.
+ */
+ if (is_mirror_sptep(iter->sptep)) {
+ if (ret)
+ __kvm_tdp_mmu_write_spte(iter->sptep, iter->old_spte);
+ else
+ __kvm_tdp_mmu_write_spte(iter->sptep, new_spte);
+ } else {
+ /*
+ * Bug the VM if handling the change failed, as failure is only
+ * allowed if KVM couldn't update the external SPTE.
+ */
+ KVM_BUG_ON(ret, kvm);
+ }
+ return ret;
}
/*
@@ -1325,6 +1322,9 @@ static void kvm_tdp_mmu_age_spte(struct kvm *kvm, struct tdp_iter *iter)
{
u64 new_spte;
+ if (WARN_ON_ONCE(is_mirror_sptep(iter->sptep)))
+ return;
+
if (spte_ad_enabled(iter->old_spte)) {
iter->old_spte = tdp_mmu_clear_spte_bits_atomic(iter->sptep,
shadow_accessed_mask);
base-commit: c86be62d601ede14cbad1d0fb5af9b67d4172342
--
^ permalink raw reply related
* Re: [PATCH v3 10/26] coco/tdx-host: Implement FW_UPLOAD sysfs ABI for TDX Module updates
From: Xing, Cedric @ 2026-02-06 17:15 UTC (permalink / raw)
To: Chao Gao, linux-coco, linux-kernel, kvm, x86
Cc: reinette.chatre, ira.weiny, kai.huang, dan.j.williams, yilun.xu,
sagis, vannapurve, paulmck, nik.borisov, zhenzhong.duan, seanjc,
rick.p.edgecombe, kas, dave.hansen, vishal.l.verma, Farrah Chen,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin
In-Reply-To: <20260123145645.90444-11-chao.gao@intel.com>
On 1/23/2026 8:55 AM, Chao Gao wrote:
[...]
> +
> +static void seamldr_init(struct device *dev)
> +{
> + const struct tdx_sys_info *tdx_sysinfo = tdx_get_sysinfo();
> + int ret;
> +
> + if (WARN_ON_ONCE(!tdx_sysinfo))
> + return;
> +
> + if (!IS_ENABLED(CONFIG_INTEL_TDX_MODULE_UPDATE))
> + return;
> +
> + if (!tdx_supports_runtime_update(tdx_sysinfo))
> + pr_info("Current TDX Module cannot be updated. Consider BIOS updates\n");
> +
> + tdx_fwl = firmware_upload_register(THIS_MODULE, dev, "seamldr_upload",
I can't speak for others but the name "seamldr_upload" here doesn't look intuitive to me. Given this
FW node will show up in /sys/class/firmware/, I'd name it "tdx_module" to indicate to the user
clearly that this is for updating the TDX module.
-Cedric
^ permalink raw reply
* Re: [RFC PATCH v5 37/45] KVM: x86/tdp_mmu: Alloc external_spt page for mirror page table splitting
From: Sean Christopherson @ 2026-02-06 16:09 UTC (permalink / raw)
To: Yan Zhao
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Paolo Bonzini, linux-kernel, linux-coco, kvm,
Kai Huang, Rick Edgecombe, Vishal Annapurve, Ackerley Tng,
Sagi Shahar, Binbin Wu, Xiaoyao Li, Isaku Yamahata
In-Reply-To: <aYW9UaK7tePxDuyh@yzhao56-desk.sh.intel.com>
On Fri, Feb 06, 2026, Yan Zhao wrote:
> On Wed, Jan 28, 2026 at 05:15:09PM -0800, Sean Christopherson wrote:
> > From: Isaku Yamahata <isaku.yamahata@intel.com>
> >
> > Enhance tdp_mmu_alloc_sp_for_split() to allocate a page table page for the
> > external page table for splitting the mirror page table.
> >
> > Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> > Co-developed-by: Yan Zhao <yan.y.zhao@intel.com>
> > Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
> > [sean: use kvm_x86_ops.alloc_external_sp()]
> > Signed-off-by: Sean Christopherson <seanjc@google.com>
> > ---
> > arch/x86/kvm/mmu/tdp_mmu.c | 13 +++++++++++--
> > 1 file changed, 11 insertions(+), 2 deletions(-)
> >
> > diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
> > index 3b0da898824a..4f5b80f0ca03 100644
> > --- a/arch/x86/kvm/mmu/tdp_mmu.c
> > +++ b/arch/x86/kvm/mmu/tdp_mmu.c
> > @@ -1447,7 +1447,7 @@ bool kvm_tdp_mmu_wrprot_slot(struct kvm *kvm,
> > return spte_set;
> > }
> >
> > -static struct kvm_mmu_page *tdp_mmu_alloc_sp_for_split(void)
> > +static struct kvm_mmu_page *tdp_mmu_alloc_sp_for_split(struct tdp_iter *iter)
> > {
> > struct kvm_mmu_page *sp;
> >
> > @@ -1461,6 +1461,15 @@ static struct kvm_mmu_page *tdp_mmu_alloc_sp_for_split(void)
> > return NULL;
> > }
> >
> > + if (is_mirror_sptep(iter->sptep)) {
> tdp_mmu_alloc_sp_for_split() is invoked in tdp_mmu_split_huge_pages_root() after
> rcu_read_unlock() is called.
>
> So, it's incorrect to invoke is_mirror_sptep() which internally contains
> rcu_dereference(), resulting in "WARNING: suspicious RCU usage".
Ah, now I see why the previous code pass in a bool. I don't love passing a bool,
but passing @iter is outright dangerous, so I guess this?
diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
index a32192c35099..4d92c0d19d7c 100644
--- a/arch/x86/kvm/mmu/tdp_mmu.c
+++ b/arch/x86/kvm/mmu/tdp_mmu.c
@@ -1448,7 +1448,7 @@ bool kvm_tdp_mmu_wrprot_slot(struct kvm *kvm,
}
static struct kvm_mmu_page *tdp_mmu_alloc_sp_for_split(struct kvm *kvm,
- struct tdp_iter *iter)
+ bool is_mirror_sp)
{
struct kvm_mmu_page *sp;
@@ -1460,7 +1460,7 @@ static struct kvm_mmu_page *tdp_mmu_alloc_sp_for_split(struct kvm *kvm,
if (!sp->spt)
goto err_spt;
- if (is_mirror_sptep(iter->sptep)) {
+ if (is_mirror_sp) {
sp->external_spt = (void *)kvm_x86_call(alloc_external_sp)(GFP_KERNEL_ACCOUNT);
if (!sp->external_spt)
goto err_external_spt;
@@ -1525,6 +1525,7 @@ static int tdp_mmu_split_huge_pages_root(struct kvm *kvm,
gfn_t start, gfn_t end,
int target_level, bool shared)
{
+ const bool is_mirror_root = is_mirror_sp(root);
struct kvm_mmu_page *sp = NULL;
struct tdp_iter iter;
@@ -1557,7 +1558,7 @@ static int tdp_mmu_split_huge_pages_root(struct kvm *kvm,
else
write_unlock(&kvm->mmu_lock);
- sp = tdp_mmu_alloc_sp_for_split(kvm, &iter);
+ sp = tdp_mmu_alloc_sp_for_split(kvm, is_mirror_root);
if (shared)
read_lock(&kvm->mmu_lock);
^ permalink raw reply related
* Re: [RFC PATCH v5 22/45] KVM: TDX: Get/put PAMT pages when (un)mapping private memory
From: Sean Christopherson @ 2026-02-06 16:03 UTC (permalink / raw)
To: Yan Zhao
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Paolo Bonzini, linux-kernel, linux-coco, kvm,
Kai Huang, Rick Edgecombe, Vishal Annapurve, Ackerley Tng,
Sagi Shahar, Binbin Wu, Xiaoyao Li, Isaku Yamahata
In-Reply-To: <aYXAdJV8rvWn4EQf@yzhao56-desk.sh.intel.com>
On Fri, Feb 06, 2026, Yan Zhao wrote:
> On Wed, Jan 28, 2026 at 05:14:54PM -0800, Sean Christopherson wrote:
> > diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> > index 6e84dbc89e79..a6e4ab76b1b2 100644
> > --- a/arch/x86/include/asm/kvm_host.h
> > +++ b/arch/x86/include/asm/kvm_host.h
> > @@ -1863,6 +1863,7 @@ struct kvm_x86_ops {
> > struct kvm_mmu_page *sp);
> > void (*remove_external_spte)(struct kvm *kvm, gfn_t gfn, enum pg_level level,
> > u64 mirror_spte);
> > + int (*topup_external_cache)(struct kvm_vcpu *vcpu, int min);
> >
> >
> > bool (*has_wbinvd_exit)(void);
> > diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> > index 9b5a6861e2a4..4ecbf216d96f 100644
> > --- a/arch/x86/kvm/mmu/mmu.c
> > +++ b/arch/x86/kvm/mmu/mmu.c
> > @@ -605,6 +605,10 @@ static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu, bool maybe_indirect)
> > PT64_ROOT_MAX_LEVEL);
> > if (r)
> > return r;
> > +
> > + r = kvm_x86_call(topup_external_cache)(vcpu, PT64_ROOT_MAX_LEVEL);
> If this external cache is for PAMT pages allocation for guest pages only, here
> the min count should be 1 instead of PT64_ROOT_MAX_LEVEL?
Oh! Right. Hmm, with that in mind, it seems like topup_external_cache() isn't
quite the right interace. It's not at all clear that, unlike the other caches,
the DPAMT cache isn't tied to the page tables, it's tied to the physical memory
being mapped into the guest.
At the very least, it seems like we should drop the @min parameter?
int (*topup_external_cache)(struct kvm *kvm, struct kvm_vcpu *vcpu);
Though if someone has a name that better captures what the cache is used for,
without bleeding too many details into common x86...
^ permalink raw reply
* Re: [PATCH v2 0/2] x86/virt/tdx: Print TDX module version to dmesg
From: Dave Hansen @ 2026-02-06 15:10 UTC (permalink / raw)
To: Kiryl Shutsemau, Verma, Vishal L, dave.hansen
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev,
linux-kernel@vger.kernel.org, Gao, Chao, Edgecombe, Rick P,
Huang, Kai, bp@alien8.de, mingo@redhat.com, Williams, Dan J,
tglx@linutronix.de, hpa@zytor.com, x86@kernel.org
In-Reply-To: <aYXSd8B00OtKZcAU@thinkstation>
On 2/6/26 03:39, Kiryl Shutsemau wrote:
>> Hi Kiryl, just wanted to check on the plan for this, I didn't see it
>> merged in tip.git x86/tdx (or any other tip branch). Were you planning
>> to take it through x86/tdx? Can I help with anything to move it along?
> I guess it has to wait after the merge window at this point.
>
> Dave, could you queue this after -rc1 is tagged?
Sure.
Is there any other TDX stuff that needs to get picked up at the same
time that's been languishing?
^ permalink raw reply
* Re: [RFC PATCH v5 20/45] KVM: x86/mmu: Allocate/free S-EPT pages using tdx_{alloc,free}_control_page()
From: Sean Christopherson @ 2026-02-06 15:01 UTC (permalink / raw)
To: Yan Zhao
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Paolo Bonzini, linux-kernel, linux-coco, kvm,
Kai Huang, Rick Edgecombe, Vishal Annapurve, Ackerley Tng,
Sagi Shahar, Binbin Wu, Xiaoyao Li, Isaku Yamahata
In-Reply-To: <aYW5CbUvZrLogsWF@yzhao56-desk.sh.intel.com>
On Fri, Feb 06, 2026, Yan Zhao wrote:
> > diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
> > index 18764dbc97ea..01e3e4f4baa5 100644
> > --- a/arch/x86/kvm/mmu/tdp_mmu.c
> > +++ b/arch/x86/kvm/mmu/tdp_mmu.c
> > @@ -55,7 +55,8 @@ void kvm_mmu_uninit_tdp_mmu(struct kvm *kvm)
> >
> > static void tdp_mmu_free_sp(struct kvm_mmu_page *sp)
> > {
> > - free_page((unsigned long)sp->external_spt);
> > + if (sp->external_spt)
> > + kvm_x86_call(free_external_sp)((unsigned long)sp->external_spt);
> > free_page((unsigned long)sp->spt);
> > kmem_cache_free(mmu_page_header_cache, sp);
> > }
> Strictly speaking, external_spt is not a control page. Its alloc/free are
> different from normal control pages managed by TDX's code.
Yeah, I called that out in the changelog. I'm definitley not wedded to
tdx_{alloc,free}_control_page(), but I am very much against tdx_{alloc,free}_page().
(arguably S-EPT pages aren't "control" pages, but they're not guest pages either)
> (1) alloc
> tdx_alloc_control_page
> __tdx_alloc_control_page
> __tdx_pamt_get
> spin_lock(&pamt_lock) ==> under process context
> spin_unlock(&pamt_lock)
>
> (2) free
> tdp_mmu_free_sp_rcu_callback
> tdp_mmu_free_sp
> kvm_x86_call(free_external_sp)
> tdx_free_control_page
> __tdx_free_control_page
> __tdx_pamt_put
> spin_lock(&pamt_lock) ==> under softirq context
> spin_unlock(&pamt_lock)
>
> So, invoking __tdx_pamt_put() in the RCU callback triggers deadlock warning
> (see the bottom for details).
Hrm. I can think of two options. Option #1 would be to use a raw spinlock and
disable IRQs:
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 823ec092b4e4..6348085d7dcb 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -2246,7 +2246,7 @@ static u64 tdh_phymem_pamt_remove(u64 pfn, u64 *pamt_pa_array)
}
/* Serializes adding/removing PAMT memory */
-static DEFINE_SPINLOCK(pamt_lock);
+static DEFINE_RAW_SPINLOCK(pamt_lock);
/* Bump PAMT refcount for the given page and allocate PAMT memory if needed */
int __tdx_pamt_get(u64 pfn, struct tdx_pamt_cache *cache)
@@ -2272,7 +2272,7 @@ int __tdx_pamt_get(u64 pfn, struct tdx_pamt_cache *cache)
if (ret)
goto out_free;
- scoped_guard(spinlock, &pamt_lock) {
+ scoped_guard(raw_spinlock_irqsave, &pamt_lock) {
/*
* Lost race to other tdx_pamt_add(). Other task has already allocated
* PAMT memory for the HPA.
@@ -2348,7 +2348,7 @@ void __tdx_pamt_put(u64 pfn)
if (!atomic_dec_and_test(pamt_refcount))
return;
- scoped_guard(spinlock, &pamt_lock) {
+ scoped_guard(raw_spinlock_irqsave, &pamt_lock) {
/* Lost race with tdx_pamt_get(). */
if (atomic_read(pamt_refcount))
return;
--
Option #2 would be to immediately free the page in tdx_sept_reclaim_private_sp(),
so that pages that freed via handle_removed_pt() don't defer freeing the S-EPT
page table (which, IIUC, is safe since the TDX-Module forces TLB flushes and exits).
I really, really don't like this option (if it even works).
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index ae7b9beb3249..4726011ad624 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -2014,7 +2014,15 @@ static void tdx_sept_reclaim_private_sp(struct kvm *kvm, gfn_t gfn,
*/
if (KVM_BUG_ON(is_hkid_assigned(to_kvm_tdx(kvm)), kvm) ||
tdx_reclaim_page(virt_to_page(sp->external_spt)))
- sp->external_spt = NULL;
+ goto out;
+
+ /*
+ * Immediately free the control page, as the TDX subsystem doesn't
+ * support freeing pages from RCU callbacks.
+ */
+ tdx_free_control_page((unsigned long)sp->external_spt);
+out:
+ sp->external_spt = NULL;
}
void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode,
--
^ permalink raw reply related
* Re: [RFC PATCH v5 44/45] KVM: x86/mmu: Add support for splitting S-EPT hugepages on conversion
From: Sean Christopherson @ 2026-02-06 14:46 UTC (permalink / raw)
To: Yan Zhao
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Paolo Bonzini, linux-kernel, linux-coco, kvm,
Kai Huang, Rick Edgecombe, Vishal Annapurve, Ackerley Tng,
Sagi Shahar, Binbin Wu, Xiaoyao Li, Isaku Yamahata
In-Reply-To: <aYW+6WzOmCAvNRIH@yzhao56-desk.sh.intel.com>
On Fri, Feb 06, 2026, Yan Zhao wrote:
> On Wed, Jan 28, 2026 at 05:15:16PM -0800, Sean Christopherson wrote:
> > Add support for splitting S-EPT hugepages in preparation for converting a
> > subset of a hugepage to be shared, as KVM must precisely zap/remove S-EPT
> > entries to avoid clobbering guest memory (the lifetime of guest private
> > memory is tied to the S-EPT). I.e. KVM needs to first split a hugepage so
> > that only the to-be-converted small pages can be zapped.
> >
> > To avoid unnecessary work, e.g. if only the tail/end page of massive region
> > isn't aligned to the conversion, explicitly detect unaligned head and tail
> > pages relative to the max page size support by KVM, i.e. head/tail pages
> > that will undergo partial conversion.
> >
> > To support splitting an S-EPT hugepage without a vCPU, add a per-VM PAMT
> > cache, along with a mutex to guard the cache. Using a mutex, e.g. versus
> > a spinlock, is important at it allows KVM to allocate memory *without*
> > dropping the lock, i.e. so that the PAMT cache can be topped-up as needed
> > without needed to juggle arch.tdp_mmu_external_cache_lock.
> >
> > Signed-off-by: Sean Christopherson <seanjc@google.com>
> > ---
> > arch/x86/include/asm/kvm_host.h | 8 +++-
> > arch/x86/kvm/mmu/mmu.c | 2 +-
> > arch/x86/kvm/mmu/tdp_mmu.c | 72 +++++++++++++++++++++++++++++++--
> > arch/x86/kvm/vmx/tdx.c | 34 +++++++++++++---
> > arch/x86/kvm/vmx/tdx.h | 2 +
> > 5 files changed, 107 insertions(+), 11 deletions(-)
> >
> > diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> > index 385f1cf32d70..54dea90a53dc 100644
> > --- a/arch/x86/include/asm/kvm_host.h
> > +++ b/arch/x86/include/asm/kvm_host.h
> > @@ -1563,6 +1563,12 @@ struct kvm_arch {
> > * the code to do so.
> > */
> > spinlock_t tdp_mmu_pages_lock;
> > +
> > + /*
> > + * Protect the per-VM cache of pre-allocate pages used to populate the
> > + * Dynamic PAMT when splitting S-EPT huge pages without a vCPU.
> > + */
> > + struct mutex tdp_mmu_external_cache_lock;
> Missing "spin_lock_init(&kvm->arch.tdp_mmu_external_cache_lock);" in
> kvm_mmu_init_tdp_mmu().
>
> Will check the patch you replied next week.
It has the same bug. FWIW, I found and fixed the bug on our internal branch, but I
either missed the fixup when synchronizing back to the upstream branch, or I found
the issue after posting.
@@ -634,6 +642,7 @@ int tdx_vm_init(struct kvm *kvm)
struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
tdx_init_pamt_cache(&kvm_tdx->pamt_cache);
+ mutex_init(&kvm_tdx->pamt_cache_lock);
kvm->arch.has_protected_state = true;
/*
^ permalink raw reply
* Re: [PATCH v4 1/2] mm/memory_hotplug: Add support to accept memory during hot-add
From: Kiryl Shutsemau @ 2026-02-06 12:03 UTC (permalink / raw)
To: David Hildenbrand (Arm)
Cc: Pratik R. Sampat, linux-mm, linux-coco, x86, linux-kernel, tglx,
mingo, bp, dave.hansen, ardb, akpm, osalvador, thomas.lendacky,
michael.roth
In-Reply-To: <fc60cc71-80eb-4058-8d0a-dbd19a5c10ef@kernel.org>
On Thu, Feb 05, 2026 at 06:29:08PM +0100, David Hildenbrand (Arm) wrote:
> > Ideally, we want to know on boot:
> >
> > - what memory ranges are unaccepted - we have it;
> > - what memory range can be removed or added after boot - we don't have it
>
> The SRAT describes memory ranges where we can see hotplug memory. Is that
> too late? We calculate max_possible_pfn based on that.
The cleanest way would be to declare the ranges in EFI memory map, not
SRAT. It should be doable.
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v2 0/2] x86/virt/tdx: Print TDX module version to dmesg
From: Kiryl Shutsemau @ 2026-02-06 11:39 UTC (permalink / raw)
To: Verma, Vishal L, dave.hansen
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev,
linux-kernel@vger.kernel.org, Gao, Chao, Edgecombe, Rick P,
Huang, Kai, bp@alien8.de, mingo@redhat.com, Williams, Dan J,
tglx@linutronix.de, hpa@zytor.com, x86@kernel.org
In-Reply-To: <6d8d37740459963e6fd7f16a890a837b34ebdf17.camel@intel.com>
On Thu, Feb 05, 2026 at 07:05:39PM +0000, Verma, Vishal L wrote:
> On Fri, 2026-01-09 at 12:14 -0700, Vishal Verma wrote:
> > === Problem & Solution ===
> >
> > Currently, there is neither an ABI, nor any other way to determine from
> > the host system, what version of the TDX module is running. A sysfs ABI
> > for this has been proposed in [1], but it may need additional discussion.
> >
> > Many/most TDX developers already carry patches like this in their
> > development branches. It can be tricky to know which TDX module is
> > actually loaded on a system, and so this functionality has been needed
> > regularly for development and processing bug reports. Hence, it is
> > prudent to break out the patches to retrieve and print the TDX module
> > version, as those parts are very straightforward, and get some level of
> > debugability and traceability for TDX host systems.
> >
> > === Dependencies ===
> >
> > None. This is based on v6.19-rc4, and applies cleanly to tip.git.
> >
> > === Patch details ===
> >
> > Patch 1 is a prerequisite that adds the infrastructure to retrieve the
> > TDX module version from its global metadata. This was originally posted in [2].
> >
> > Patch 2 is based on a patch from Kai Huang [3], and prints the version to
> > dmesg during init.
> >
> > === Testing ===
> >
> > This has passed the usual suite of tests, including successful 0day
> > builds, KVM Unit tests, KVM selftests, a TD creation smoke test, and
> > selected KVM tests from the Avocado test suite.
> >
> > [1]: https://lore.kernel.org/all/20260105074350.98564-1-chao.gao@intel.com/
> > [2]: https://lore.kernel.org/all/20260105074350.98564-2-chao.gao@intel.com/
> > [3]: https://lore.kernel.org/all/57eaa1b17429315f8b5207774307f3c1dd40cf37.1730118186.git.kai.huang@intel.com/
> >
> > Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
>
> Hi Kiryl, just wanted to check on the plan for this, I didn't see it
> merged in tip.git x86/tdx (or any other tip branch). Were you planning
> to take it through x86/tdx? Can I help with anything to move it along?
I guess it has to wait after the merge window at this point.
Dave, could you queue this after -rc1 is tagged?
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [RFC PATCH v5 22/45] KVM: TDX: Get/put PAMT pages when (un)mapping private memory
From: Yan Zhao @ 2026-02-06 10:20 UTC (permalink / raw)
To: Sean Christopherson
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
Kiryl Shutsemau, Paolo Bonzini, linux-kernel, linux-coco, kvm,
Kai Huang, Rick Edgecombe, Vishal Annapurve, Ackerley Tng,
Sagi Shahar, Binbin Wu, Xiaoyao Li, Isaku Yamahata
In-Reply-To: <20260129011517.3545883-23-seanjc@google.com>
On Wed, Jan 28, 2026 at 05:14:54PM -0800, Sean Christopherson wrote:
> From: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>
> Add Dynamic PAMT support to KVM's S-EPT MMU by "getting" a PAMT page when
> adding guest memory (PAGE.ADD or PAGE.AUG), and "putting" the page when
> removing guest memory (PAGE.REMOVE).
>
> To access the per-vCPU PAMT caches without plumbing @vcpu throughout the
> TDP MMU, begrudginly use kvm_get_running_vcpu() to get the vCPU, and bug
> the VM If KVM attempts to set an S-EPT without an active vCPU. KVM only
> supports creating _new_ mappings in page (pre)fault paths, all of which
> require an active vCPU.
>
> The PAMT memory holds metadata for TDX-protected memory. With Dynamic
> PAMT, PAMT_4K is allocated on demand. The kernel supplies the TDX module
> with a few pages that cover 2M of host physical memory.
>
> PAMT memory can be reclaimed when the last user is gone. It can happen
> in a few code paths:
>
> - On TDH.PHYMEM.PAGE.RECLAIM in tdx_reclaim_td_control_pages() and
> tdx_reclaim_page().
>
> - On TDH.MEM.PAGE.REMOVE in tdx_sept_drop_private_spte().
>
> - In tdx_sept_zap_private_spte() for pages that were in the queue to be
> added with TDH.MEM.PAGE.ADD, but it never happened due to an error.
>
> - In tdx_sept_free_private_spt() for SEPT pages;
>
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> [Minor log tweak]
> Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
> Co-developed-by: Sean Christopherson <seanjc@google.com>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
> arch/x86/include/asm/kvm-x86-ops.h | 1 +
> arch/x86/include/asm/kvm_host.h | 1 +
> arch/x86/kvm/mmu/mmu.c | 4 +++
> arch/x86/kvm/vmx/tdx.c | 44 ++++++++++++++++++++++++++----
> arch/x86/kvm/vmx/tdx.h | 2 ++
> 5 files changed, 47 insertions(+), 5 deletions(-)
>
> diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h
> index 17dddada69fc..394dc29483a7 100644
> --- a/arch/x86/include/asm/kvm-x86-ops.h
> +++ b/arch/x86/include/asm/kvm-x86-ops.h
> @@ -99,6 +99,7 @@ KVM_X86_OP_OPTIONAL(free_external_sp)
> KVM_X86_OP_OPTIONAL_RET0(set_external_spte)
> KVM_X86_OP_OPTIONAL(remove_external_spte)
> KVM_X86_OP_OPTIONAL(reclaim_external_sp)
> +KVM_X86_OP_OPTIONAL_RET0(topup_external_cache)
> KVM_X86_OP(has_wbinvd_exit)
> KVM_X86_OP(get_l2_tsc_offset)
> KVM_X86_OP(get_l2_tsc_multiplier)
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 6e84dbc89e79..a6e4ab76b1b2 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1863,6 +1863,7 @@ struct kvm_x86_ops {
> struct kvm_mmu_page *sp);
> void (*remove_external_spte)(struct kvm *kvm, gfn_t gfn, enum pg_level level,
> u64 mirror_spte);
> + int (*topup_external_cache)(struct kvm_vcpu *vcpu, int min);
>
>
> bool (*has_wbinvd_exit)(void);
> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> index 9b5a6861e2a4..4ecbf216d96f 100644
> --- a/arch/x86/kvm/mmu/mmu.c
> +++ b/arch/x86/kvm/mmu/mmu.c
> @@ -605,6 +605,10 @@ static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu, bool maybe_indirect)
> PT64_ROOT_MAX_LEVEL);
> if (r)
> return r;
> +
> + r = kvm_x86_call(topup_external_cache)(vcpu, PT64_ROOT_MAX_LEVEL);
If this external cache is for PAMT pages allocation for guest pages only, here
the min count should be 1 instead of PT64_ROOT_MAX_LEVEL?
> + if (r)
> + return r;
> }
> r = kvm_mmu_topup_memory_cache(&vcpu->arch.mmu_shadow_page_cache,
> PT64_ROOT_MAX_LEVEL);
...
> void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode,
> @@ -3614,5 +3641,12 @@ void __init tdx_hardware_setup(void)
> vt_x86_ops.set_external_spte = tdx_sept_set_private_spte;
> vt_x86_ops.reclaim_external_sp = tdx_sept_reclaim_private_sp;
> vt_x86_ops.remove_external_spte = tdx_sept_remove_private_spte;
> +
> + /*
> + * FIXME: Wire up the PAMT hook iff DPAMT is supported, once VMXON is
> + * moved out of KVM and tdx_bringup() is folded into here.
> + */
> + vt_x86_ops.topup_external_cache = tdx_topup_external_pamt_cache;
> +
> vt_x86_ops.protected_apic_has_interrupt = tdx_protected_apic_has_interrupt;
> }
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox