* Re: [PATCH v5 09/22] x86/virt/seamldr: Introduce skeleton for TDX module updates
From: Kiryl Shutsemau @ 2026-03-19 13:15 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260315135920.354657-10-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:29AM -0700, Chao Gao wrote:
> TDX module updates require careful synchronization with other TDX
> operations. The requirements are (#1/#2 reflect current behavior that
> must be preserved):
>
> 1. SEAMCALLs need to be callable from both process and IRQ contexts.
> 2. SEAMCALLs need to be able to run concurrently across CPUs
> 3. During updates, only update-related SEAMCALLs are permitted; all
> other SEAMCALLs shouldn't be called.
> 4. During updates, all online CPUs must participate in the update work.
>
> No single lock primitive satisfies all requirements. For instance,
> rwlock_t handles #1/#2 but fails #4: CPUs spinning with IRQs disabled
> cannot be directed to perform update work.
>
> Use stop_machine() as it is the only well-understood mechanism that can
> meet all requirements.
>
> And TDX module updates consist of several steps (See Intel® Trust Domain
> Extensions (Intel® TDX) Module Base Architecture Specification, Revision
> 348549-007, Chapter 4.5 "TD-Preserving TDX module Update"). Ordering
> requirements between steps mandate lockstep synchronization across all
> CPUs.
>
> multi_cpu_stop() is a good example of performing a multi-step task in
> lockstep. But it doesn't synchronize steps within the callback function
> it takes. So, implement one based on its pattern to establish the
> skeleton for TDX module updates. Specifically, add a global state
> machine where each state represents a step in the update flow. The state
> advances only after all CPUs acknowledge completing their work in the
> current state. This acknowledgment mechanism is what ensures lockstep
> execution.
>
> Potential alternative to stop_machine()
> =======================================
> An alternative approach is to lock all KVM entry points and kick all
> vCPUs. Here, KVM entry points refer to KVM VM/vCPU ioctl entry points,
> implemented in KVM common code (virt/kvm). Adding a locking mechanism
> there would affect all architectures KVM supports. And to lock only TDX
> vCPUs, new logic would be needed to identify TDX vCPUs, which the KVM
> common code currently lacks. This would add significant complexity and
> maintenance overhead to KVM for this TDX-specific use case.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
One question below.
> +static struct {
> + enum module_update_state state;
> + int thread_ack;
> + /*
> + * Protect update_data. Raw spinlock as it will be acquired from
> + * interrupt-disabled contexts.
> + */
> + raw_spinlock_t lock;
> +} update_data = {
> + .lock = __RAW_SPIN_LOCK_UNLOCKED(update_data.lock)
> +};
multi_stop_cpu() used atomic_t for thread_ack insead of spinlock.
Any particular reason you took different direction?
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v5 08/22] x86/virt/seamldr: Allocate and populate a module update request
From: Kiryl Shutsemau @ 2026-03-19 12:58 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <abukxFDD3g0sKYXP@intel.com>
On Thu, Mar 19, 2026 at 03:24:52PM +0800, Chao Gao wrote:
> On Wed, Mar 18, 2026 at 10:58:44AM +0000, Kiryl Shutsemau wrote:
> >On Wed, Mar 18, 2026 at 04:50:34PM +0800, Chao Gao wrote:
> >> >> + ptr = sig;
> >> >> + for (i = 0; i < sig_size / SZ_4K; i++) {
> >> >> + /*
> >> >> + * Don't assume @sig is page-aligned although it is 4KB-aligned.
> >> >> + * Always add the in-page offset to get the physical address.
> >> >> + */
> >> >
> >> >I don't follow this. If @sig is 4k aligned in VA, it is page aligned.
> >>
> >> Dan's concern was that PAGE_SIZE is not guaranteed to be 4096.
> >>
> >> I agree that PAGE_SIZE is 4K on x86 today. But to address that concern, I saw
> >> two options:
> >>
> >> 1. Add WARN_ON_ONCE(PAGE_SIZE != SZ_4K), or
> >> 2. Handle it as in the code above.
> >>
> >> I didn't find existing code using option 1 in x86, so I chose option 2.
> >
> >Please, make it clear in the comment that you are talking about PAGE_SIZE != SZ_4K.
>
> Sure. How about:
>
> /*
> * @sig is 4KB-aligned, but that does not imply PAGE_SIZE
> * alignment when PAGE_SIZE != SZ_4K. Always include the
> * in-page offset.
> */
Ack.
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH 1/2] x86/virt/tdx: Use PFN directly for mapping guest private memory
From: Kiryl Shutsemau @ 2026-03-19 12:57 UTC (permalink / raw)
To: Yan Zhao
Cc: seanjc, pbonzini, dave.hansen, tglx, mingo, bp, x86, linux-kernel,
kvm, linux-coco, kai.huang, rick.p.edgecombe, yilun.xu,
vannapurve, ackerleytng, sagis, binbin.wu, xiaoyao.li,
isaku.yamahata
In-Reply-To: <abvlOcnY4assk8DW@yzhao56-desk.sh.intel.com>
On Thu, Mar 19, 2026 at 07:59:53PM +0800, Yan Zhao wrote:
> On Thu, Mar 19, 2026 at 10:39:14AM +0000, Kiryl Shutsemau wrote:
> > On Thu, Mar 19, 2026 at 08:57:03AM +0800, Yan Zhao wrote:
> > > @@ -1639,16 +1644,17 @@ u64 tdh_vp_addcx(struct tdx_vp *vp, struct page *tdcx_page)
> > > }
> > > EXPORT_SYMBOL_FOR_KVM(tdh_vp_addcx);
> > >
> > > -u64 tdh_mem_page_aug(struct tdx_td *td, u64 gpa, int level, struct page *page, u64 *ext_err1, u64 *ext_err2)
> > > +u64 tdh_mem_page_aug(struct tdx_td *td, u64 gpa, int level, kvm_pfn_t pfn,
> > > + u64 *ext_err1, u64 *ext_err2)
> > > {
> > > struct tdx_module_args args = {
> > > .rcx = gpa | level,
> > > .rdx = tdx_tdr_pa(td),
> > > - .r8 = page_to_phys(page),
> > > + .r8 = PFN_PHYS(pfn),
> > > };
> > > u64 ret;
> > >
> > > - tdx_clflush_page(page);
> > > + tdx_clflush_pfn(pfn);
> >
> > This is pre-existing problem, but shouldn't we respect @level here?
> > Flush size need to take page size into account.
> Hmm, flush size is fixed to PAGE_SIZE, because this series is based on the
> upstream code where huge page is not supported, so there's
> "if (KVM_BUG_ON(level != PG_LEVEL_4K, kvm))" in KVM.
>
> Though tdh_mem_page_aug() is an API, it is currently only exported to KVM and
> uses type kvm_pfn_t. So, is it still acceptable to assume flush size to be
> PAGE_SIZE? Honoring level will soon be introduced by huge page patches.
It caught my eye because previously size to flush was passed down to
tdx_clflush_page() in the struct page (although never used there).
With switching to pfn, we give up this information and it has to be
passed separately. It would be easy to miss that in huge page patches,
if we don't pass down level here.
>
> If you think it needs to be fixed before huge page series, what about fixing it
> in a separate cleanup patch? IMO, it would be better placed after Sean's cleanup
> patch [1], so we can use page_level_size() instead of inventing the wheel.
I am okay with a separate patch.
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH 1/2] x86/virt/tdx: Use PFN directly for mapping guest private memory
From: Yan Zhao @ 2026-03-19 12:14 UTC (permalink / raw)
To: Kiryl Shutsemau, seanjc, pbonzini, dave.hansen, tglx, mingo, bp,
x86, linux-kernel, kvm, linux-coco, kai.huang, rick.p.edgecombe,
yilun.xu, vannapurve, ackerleytng, sagis, binbin.wu, xiaoyao.li,
isaku.yamahata
In-Reply-To: <abvlOcnY4assk8DW@yzhao56-desk.sh.intel.com>
On Thu, Mar 19, 2026 at 07:59:53PM +0800, Yan Zhao wrote:
> On Thu, Mar 19, 2026 at 10:39:14AM +0000, Kiryl Shutsemau wrote:
> > On Thu, Mar 19, 2026 at 08:57:03AM +0800, Yan Zhao wrote:
> > > @@ -1639,16 +1644,17 @@ u64 tdh_vp_addcx(struct tdx_vp *vp, struct page *tdcx_page)
> > > }
> > > EXPORT_SYMBOL_FOR_KVM(tdh_vp_addcx);
> > >
> > > -u64 tdh_mem_page_aug(struct tdx_td *td, u64 gpa, int level, struct page *page, u64 *ext_err1, u64 *ext_err2)
> > > +u64 tdh_mem_page_aug(struct tdx_td *td, u64 gpa, int level, kvm_pfn_t pfn,
> > > + u64 *ext_err1, u64 *ext_err2)
> > > {
> > > struct tdx_module_args args = {
> > > .rcx = gpa | level,
> > > .rdx = tdx_tdr_pa(td),
> > > - .r8 = page_to_phys(page),
> > > + .r8 = PFN_PHYS(pfn),
> > > };
> > > u64 ret;
> > >
> > > - tdx_clflush_page(page);
> > > + tdx_clflush_pfn(pfn);
> >
> > This is pre-existing problem, but shouldn't we respect @level here?
> > Flush size need to take page size into account.
> Hmm, flush size is fixed to PAGE_SIZE, because this series is based on the
> upstream code where huge page is not supported, so there's
> "if (KVM_BUG_ON(level != PG_LEVEL_4K, kvm))" in KVM.
>
> Though tdh_mem_page_aug() is an API, it is currently only exported to KVM and
> uses type kvm_pfn_t. So, is it still acceptable to assume flush size to be
> PAGE_SIZE? Honoring level will soon be introduced by huge page patches.
>
> If you think it needs to be fixed before huge page series, what about fixing it
> in a separate cleanup patch? IMO, it would be better placed after Sean's cleanup
> patch [1], so we can use page_level_size() instead of inventing the wheel.
BTW, the cleanup patch would then essentially look like the one in the huge page
series [2]...
[2] https://lore.kernel.org/kvm/20260129011517.3545883-27-seanjc@google.com/
So, if a cleanup patch before the huge page series is required, maybe just
adding WARN_ON_ONCE(level != PG_LEVEL_4K) in that patch?
> [1] https://lore.kernel.org/kvm/20260129011517.3545883-2-seanjc@google.com/
^ permalink raw reply
* Re: [PATCH v13 07/48] arm64: RMI: Check for RMI support at KVM init
From: Steven Price @ 2026-03-19 12:47 UTC (permalink / raw)
To: Suzuki K Poulose, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Zenghui Yu, linux-arm-kernel, linux-kernel,
Joey Gouly, Alexandru Elisei, Christoffer Dall, Fuad Tabba,
linux-coco, Ganapatrao Kulkarni, Gavin Shan, Shanker Donthineni,
Alper Gun, Aneesh Kumar K . V, Emi Kisanuki, Vishal Annapurve
In-Reply-To: <8fd56e84-821a-47c7-8735-928057a7a653@arm.com>
On 19/03/2026 10:38, Suzuki K Poulose wrote:
> On 18/03/2026 15:53, Steven Price wrote:
>> Query the RMI version number and check if it is a compatible version. A
>> static key is also provided to signal that a supported RMM is available.
>>
>> Functions are provided to query if a VM or VCPU is a realm (or rec)
>> which currently will always return false.
>>
>> Later patches make use of struct realm and the states as the ioctls
>> interfaces are added to support realm and REC creation and destruction.
>>
>> Signed-off-by: Steven Price <steven.price@arm.com>
>> ---
>> Changes since v12:
>> * Drop check for 4k page size.
>> Changes since v11:
>> * Reword slightly the comments on the realm states.
>> Changes since v10:
>> * kvm_is_realm() no longer has a NULL check.
>> * Rename from "rme" to "rmi" when referring to the RMM interface.
>> * Check for RME (hardware) support before probing for RMI support.
>> Changes since v8:
>> * No need to guard kvm_init_rme() behind 'in_hyp_mode'.
>> Changes since v6:
>> * Improved message for an unsupported RMI ABI version.
>> Changes since v5:
>> * Reword "unsupported" message from "host supports" to "we want" to
>> clarify that 'we' are the 'host'.
>> Changes since v2:
>> * Drop return value from kvm_init_rme(), it was always 0.
>> * Rely on the RMM return value to identify whether the RSI ABI is
>> compatible.
>> ---
>> arch/arm64/include/asm/kvm_emulate.h | 18 +++++++++
>> arch/arm64/include/asm/kvm_host.h | 4 ++
>> arch/arm64/include/asm/kvm_rmi.h | 56 +++++++++++++++++++++++++++
>> arch/arm64/include/asm/virt.h | 1 +
>> arch/arm64/kernel/cpufeature.c | 1 +
>> arch/arm64/kvm/Makefile | 2 +-
>> arch/arm64/kvm/arm.c | 5 +++
>> arch/arm64/kvm/rmi.c | 57 ++++++++++++++++++++++++++++
>> 8 files changed, 143 insertions(+), 1 deletion(-)
>> create mode 100644 arch/arm64/include/asm/kvm_rmi.h
>> create mode 100644 arch/arm64/kvm/rmi.c
>>
>> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/
>> include/asm/kvm_emulate.h
>> index 5bf3d7e1d92c..f38b50151ce8 100644
>> --- a/arch/arm64/include/asm/kvm_emulate.h
>> +++ b/arch/arm64/include/asm/kvm_emulate.h
>> @@ -688,4 +688,22 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu
>> *vcpu)
>> vcpu->arch.hcrx_el2 |= HCRX_EL2_EnASR;
>> }
>> }
>
>
> --8>---
>
>> +
>> +static inline bool kvm_is_realm(struct kvm *kvm)
>> +{
>> + if (static_branch_unlikely(&kvm_rmi_is_available))
>> + return kvm->arch.is_realm;
>> + return false;
>> +}
>> +
>> +static inline enum realm_state kvm_realm_state(struct kvm *kvm)
>> +{
>> + return READ_ONCE(kvm->arch.realm.state);
>> +}
>> +
>> +static inline bool vcpu_is_rec(struct kvm_vcpu *vcpu)
>> +{
>> + return false;
>> +}
>> +
>> #endif /* __ARM64_KVM_EMULATE_H__ */
>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/
>> asm/kvm_host.h
>> index 5d5a3bbdb95e..9267a2f2d65b 100644
>> --- a/arch/arm64/include/asm/kvm_host.h
>> +++ b/arch/arm64/include/asm/kvm_host.h
>> @@ -27,6 +27,7 @@
>> #include <asm/fpsimd.h>
>> #include <asm/kvm.h>
>> #include <asm/kvm_asm.h>
>> +#include <asm/kvm_rmi.h>
>> #include <asm/vncr_mapping.h>
>> #define __KVM_HAVE_ARCH_INTC_INITIALIZED
>> @@ -405,6 +406,9 @@ struct kvm_arch {
>> * the associated pKVM instance in the hypervisor.
>> */
>> struct kvm_protected_vm pkvm;
>> +
>> + bool is_realm;
>> + struct realm realm;
>> };
>> struct kvm_vcpu_fault_info {
>> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/
>> asm/kvm_rmi.h
>> new file mode 100644
>> index 000000000000..3506f50b05cd
>> --- /dev/null
>> +++ b/arch/arm64/include/asm/kvm_rmi.h
>> @@ -0,0 +1,56 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + * Copyright (C) 2023-2025 ARM Ltd.
>> + */
>> +
>> +#ifndef __ASM_KVM_RMI_H
>> +#define __ASM_KVM_RMI_H
>> +
>> +/**
>> + * enum realm_state - State of a Realm
>> + */
>> +enum realm_state {
>> + /**
>> + * @REALM_STATE_NONE:
>> + * Realm has not yet been created. rmi_realm_create() has not
>> + * yet been called.
>> + */
>> + REALM_STATE_NONE,
>> + /**
>> + * @REALM_STATE_NEW:
>> + * Realm is under construction, rmi_realm_create() has been
>> + * called, but it is not yet activated. Pages may be populated.
>> + */
>> + REALM_STATE_NEW,
>> + /**
>> + * @REALM_STATE_ACTIVE:
>> + * Realm has been created and is eligible for execution with
>> + * rmi_rec_enter(). Pages may no longer be populated with
>> + * rmi_data_create().
>> + */
>> + REALM_STATE_ACTIVE,
>> + /**
>> + * @REALM_STATE_DYING:
>> + * Realm is in the process of being destroyed or has already
>> been
>> + * destroyed.
>> + */
>> + REALM_STATE_DYING,
>> + /**
>> + * @REALM_STATE_DEAD:
>> + * Realm has been destroyed.
>> + */
>> + REALM_STATE_DEAD
>> +};
>> +
>> +/**
>> + * struct realm - Additional per VM data for a Realm
>> + *
>> + * @state: The lifetime state machine for the realm
>> + */
>> +struct realm {
>> + enum realm_state state;
>> +};
>> +
>
>
> --8<---
>
> Minor nit: The above looks out of place in this patch. Could we
> move it to where this may be actually used ?
Yes, good point. This can be moved to patch 12 quite easily. I think
originally I'd needed this earlier on, but the code's moved on.
Thanks,
Steve
> Rest looks good to me.
>
>
>> +void kvm_init_rmi(void);
>> +
>> +#endif /* __ASM_KVM_RMI_H */
>> diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/
>> virt.h
>> index b51ab6840f9c..dc9b2899e0b2 100644
>> --- a/arch/arm64/include/asm/virt.h
>> +++ b/arch/arm64/include/asm/virt.h
>> @@ -87,6 +87,7 @@ void __hyp_reset_vectors(void);
>> bool is_kvm_arm_initialised(void);
>> DECLARE_STATIC_KEY_FALSE(kvm_protected_mode_initialized);
>> +DECLARE_STATIC_KEY_FALSE(kvm_rmi_is_available);
>> static inline bool is_pkvm_initialized(void)
>> {
>> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/
>> cpufeature.c
>> index c31f8e17732a..ddf7e57f23e8 100644
>> --- a/arch/arm64/kernel/cpufeature.c
>> +++ b/arch/arm64/kernel/cpufeature.c
>> @@ -289,6 +289,7 @@ static const struct arm64_ftr_bits
>> ftr_id_aa64isar3[] = {
>> static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
>> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
>> ID_AA64PFR0_EL1_CSV3_SHIFT, 4, 0),
>> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
>> ID_AA64PFR0_EL1_CSV2_SHIFT, 4, 0),
>> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
>> ID_AA64PFR0_EL1_RME_SHIFT, 4, 0),
>> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE,
>> ID_AA64PFR0_EL1_DIT_SHIFT, 4, 0),
>> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
>> ID_AA64PFR0_EL1_AMU_SHIFT, 4, 0),
>> ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE,
>> ID_AA64PFR0_EL1_MPAM_SHIFT, 4, 0),
>> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> index 3ebc0570345c..e17c4077d8e7 100644
>> --- a/arch/arm64/kvm/Makefile
>> +++ b/arch/arm64/kvm/Makefile
>> @@ -16,7 +16,7 @@ CFLAGS_handle_exit.o += -Wno-override-init
>> kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \
>> inject_fault.o va_layout.o handle_exit.o config.o \
>> guest.o debug.o reset.o sys_regs.o stacktrace.o \
>> - vgic-sys-reg-v3.o fpsimd.o pkvm.o \
>> + vgic-sys-reg-v3.o fpsimd.o pkvm.o rmi.o \
>> arch_timer.o trng.o vmid.o emulate-nested.o nested.o at.o \
>> vgic/vgic.o vgic/vgic-init.o \
>> vgic/vgic-irqfd.o vgic/vgic-v2.o \
>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
>> index 29f0326f7e00..274d7866efdc 100644
>> --- a/arch/arm64/kvm/arm.c
>> +++ b/arch/arm64/kvm/arm.c
>> @@ -39,6 +39,7 @@
>> #include <asm/kvm_nested.h>
>> #include <asm/kvm_pkvm.h>
>> #include <asm/kvm_ptrauth.h>
>> +#include <asm/kvm_rmi.h>
>> #include <asm/sections.h>
>> #include <asm/stacktrace/nvhe.h>
>> @@ -104,6 +105,8 @@ long kvm_get_cap_for_kvm_ioctl(unsigned int
>> ioctl, long *ext)
>> return -EINVAL;
>> }
>> +DEFINE_STATIC_KEY_FALSE(kvm_rmi_is_available);
>> +
>> DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector);
>> DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_base);
>> @@ -2921,6 +2924,8 @@ static __init int kvm_arm_init(void)
>> in_hyp_mode = is_kernel_in_hyp_mode();
>> + kvm_init_rmi();
>> +
>> if (cpus_have_final_cap(ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE) ||
>> cpus_have_final_cap(ARM64_WORKAROUND_1508412))
>> kvm_info("Guests without required CPU erratum workarounds
>> can deadlock system!\n" \
>> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
>> new file mode 100644
>> index 000000000000..fac151580c01
>> --- /dev/null
>> +++ b/arch/arm64/kvm/rmi.c
>> @@ -0,0 +1,57 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Copyright (C) 2023-2025 ARM Ltd.
>> + */
>> +
>> +#include <linux/kvm_host.h>
>> +
>> +#include <asm/rmi_cmds.h>
>> +#include <asm/virt.h>
>> +
>> +static int rmi_check_version(void)
>> +{
>> + struct arm_smccc_res res;
>> + unsigned short version_major, version_minor;
>> + unsigned long host_version = RMI_ABI_VERSION(RMI_ABI_MAJOR_VERSION,
>> + RMI_ABI_MINOR_VERSION);
>> + unsigned long aa64pfr0 =
>> read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
>> +
>> + /* If RME isn't supported, then RMI can't be */
>> + if (cpuid_feature_extract_unsigned_field(aa64pfr0,
>> ID_AA64PFR0_EL1_RME_SHIFT) == 0)
>> + return -ENXIO;
>> +
>> + arm_smccc_1_1_invoke(SMC_RMI_VERSION, host_version, &res);
>> +
>> + if (res.a0 == SMCCC_RET_NOT_SUPPORTED)
>> + return -ENXIO;
>> +
>> + version_major = RMI_ABI_VERSION_GET_MAJOR(res.a1);
>> + version_minor = RMI_ABI_VERSION_GET_MINOR(res.a1);
>> +
>> + if (res.a0 != RMI_SUCCESS) {
>> + unsigned short high_version_major, high_version_minor;
>> +
>> + high_version_major = RMI_ABI_VERSION_GET_MAJOR(res.a2);
>> + high_version_minor = RMI_ABI_VERSION_GET_MINOR(res.a2);
>> +
>> + kvm_err("Unsupported RMI ABI (v%d.%d - v%d.%d) we want v%d.
>> %d\n",
>> + version_major, version_minor,
>> + high_version_major, high_version_minor,
>> + RMI_ABI_MAJOR_VERSION,
>> + RMI_ABI_MINOR_VERSION);
>> + return -ENXIO;
>> + }
>> +
>> + kvm_info("RMI ABI version %d.%d\n", version_major, version_minor);
>> +
>> + return 0;
>> +}
>> +
>> +void kvm_init_rmi(void)
>> +{
>> + /* Continue without realm support if we can't agree on a version */
>> + if (rmi_check_version())
>> + return;
>> +
>> + /* Future patch will enable static branch kvm_rmi_is_available */
>> +}
>
^ permalink raw reply
* Re: [PATCH 1/2] x86/virt/tdx: Use PFN directly for mapping guest private memory
From: Yan Zhao @ 2026-03-19 11:59 UTC (permalink / raw)
To: Kiryl Shutsemau
Cc: seanjc, pbonzini, dave.hansen, tglx, mingo, bp, x86, linux-kernel,
kvm, linux-coco, kai.huang, rick.p.edgecombe, yilun.xu,
vannapurve, ackerleytng, sagis, binbin.wu, xiaoyao.li,
isaku.yamahata
In-Reply-To: <abvR2A19S4ZtpAli@thinkstation>
On Thu, Mar 19, 2026 at 10:39:14AM +0000, Kiryl Shutsemau wrote:
> On Thu, Mar 19, 2026 at 08:57:03AM +0800, Yan Zhao wrote:
> > @@ -1639,16 +1644,17 @@ u64 tdh_vp_addcx(struct tdx_vp *vp, struct page *tdcx_page)
> > }
> > EXPORT_SYMBOL_FOR_KVM(tdh_vp_addcx);
> >
> > -u64 tdh_mem_page_aug(struct tdx_td *td, u64 gpa, int level, struct page *page, u64 *ext_err1, u64 *ext_err2)
> > +u64 tdh_mem_page_aug(struct tdx_td *td, u64 gpa, int level, kvm_pfn_t pfn,
> > + u64 *ext_err1, u64 *ext_err2)
> > {
> > struct tdx_module_args args = {
> > .rcx = gpa | level,
> > .rdx = tdx_tdr_pa(td),
> > - .r8 = page_to_phys(page),
> > + .r8 = PFN_PHYS(pfn),
> > };
> > u64 ret;
> >
> > - tdx_clflush_page(page);
> > + tdx_clflush_pfn(pfn);
>
> This is pre-existing problem, but shouldn't we respect @level here?
> Flush size need to take page size into account.
Hmm, flush size is fixed to PAGE_SIZE, because this series is based on the
upstream code where huge page is not supported, so there's
"if (KVM_BUG_ON(level != PG_LEVEL_4K, kvm))" in KVM.
Though tdh_mem_page_aug() is an API, it is currently only exported to KVM and
uses type kvm_pfn_t. So, is it still acceptable to assume flush size to be
PAGE_SIZE? Honoring level will soon be introduced by huge page patches.
If you think it needs to be fixed before huge page series, what about fixing it
in a separate cleanup patch? IMO, it would be better placed after Sean's cleanup
patch [1], so we can use page_level_size() instead of inventing the wheel.
[1] https://lore.kernel.org/kvm/20260129011517.3545883-2-seanjc@google.com/
^ permalink raw reply
* Re: [PATCH 2/2] x86/virt/tdx: Use PFN directly for unmapping guest private memory
From: Kiryl Shutsemau @ 2026-03-19 10:48 UTC (permalink / raw)
To: Yan Zhao
Cc: seanjc, pbonzini, dave.hansen, tglx, mingo, bp, x86, linux-kernel,
kvm, linux-coco, kai.huang, rick.p.edgecombe, yilun.xu,
vannapurve, ackerleytng, sagis, binbin.wu, xiaoyao.li,
isaku.yamahata
In-Reply-To: <20260319005808.9013-1-yan.y.zhao@intel.com>
On Thu, Mar 19, 2026 at 08:58:08AM +0800, Yan Zhao wrote:
> @@ -1817,11 +1817,11 @@ static void tdx_sept_remove_private_spte(struct kvm *kvm, gfn_t gfn,
> if (TDX_BUG_ON_2(err, TDH_MEM_PAGE_REMOVE, entry, level_state, kvm))
> return;
>
> - err = tdh_phymem_page_wbinvd_hkid((u16)kvm_tdx->hkid, page);
> + err = tdh_phymem_page_wbinvd_hkid((u16)kvm_tdx->hkid, pfn);
> if (TDX_BUG_ON(err, TDH_PHYMEM_PAGE_WBINVD, kvm))
> return;
>
> - tdx_quirk_reset_page(page);
> + tdx_quirk_reset_page(pfn);
> }
>
> void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode,
The same problem. @level is ignored.
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v13 10/48] arm64: RMI: Ensure that the RMM has GPT entries for memory
From: Suzuki K Poulose @ 2026-03-19 10:41 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Zenghui Yu, linux-arm-kernel, linux-kernel,
Joey Gouly, Alexandru Elisei, Christoffer Dall, Fuad Tabba,
linux-coco, Ganapatrao Kulkarni, Gavin Shan, Shanker Donthineni,
Alper Gun, Aneesh Kumar K . V, Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20260318155413.793430-11-steven.price@arm.com>
On 18/03/2026 15:53, Steven Price wrote:
> The RMM may not be tracking all the memory of the system at boot. Create
> the necessary tracking state and GPTs within the RMM so that all boot
> memory can be delegated to the RMM as needed during runtime.
>
> Note: support is currently missing for SROs which means that if the RMM
> needs memory donating this will fail (and render CCA unusable in Linux).
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> New patch for v13
> ---
> arch/arm64/kvm/rmi.c | 89 ++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 89 insertions(+)
>
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> index 9590dff9a2c1..80aedc85e94a 100644
> --- a/arch/arm64/kvm/rmi.c
> +++ b/arch/arm64/kvm/rmi.c
> @@ -4,6 +4,7 @@
> */
>
> #include <linux/kvm_host.h>
> +#include <linux/memblock.h>
>
> #include <asm/kvm_pgtable.h>
> #include <asm/rmi_cmds.h>
> @@ -56,6 +57,18 @@ static int rmi_check_version(void)
> return 0;
> }
>
> +/*
> + * These are the 'default' sizes when passing 0 as the tracking_region_size.
This is a little bit vague. Should we explicitly mention :
"For now we set the tracking_region_size to 0 for RMI_RMM_CONFIG_SET()"
> + * TODO: Support other granule sizes
nit: s/granule/Tracking/
Suzuki
> + */
> +#ifdef CONFIG_PAGE_SIZE_4KB
> +#define RMM_GRANULE_TRACKING_SIZE SZ_1G
> +#elif defined(CONFIG_PAGE_SIZE_16KB)
> +#define RMM_GRANULE_TRACKING_SIZE SZ_32M
> +#elif defined(CONFIG_PAGE_SIZE_64KB)
> +#define RMM_GRANULE_TRACKING_SIZE SZ_512M
> +#endif
> +
> static int rmi_configure(void)
> {
> struct rmm_config *config __free(free_page) = NULL;
> @@ -95,6 +108,80 @@ static int rmi_configure(void)
> return 0;
> }
>
> +static int rmi_verify_memory_tracking(phys_addr_t start, phys_addr_t end)
> +{
> + start = ALIGN_DOWN(start, RMM_GRANULE_TRACKING_SIZE);
> + end = ALIGN(end, RMM_GRANULE_TRACKING_SIZE);
> +
> + while (start < end) {
> + unsigned long ret, category, state;
> +
> + ret = rmi_granule_tracking_get(start, &category, &state);
> + if (ret != RMI_SUCCESS ||
> + state != RMI_TRACKING_FINE ||
> + category != RMI_MEM_CATEGORY_CONVENTIONAL) {
> + /* TODO: Set granule tracking in this case */
> + kvm_err("Granule tracking for region isn't fine/conventional: %llx",
> + start);
> + return -ENODEV;
> + }
> + start += RMM_GRANULE_TRACKING_SIZE;
> + }
> +
> + return 0;
> +}
> +
> +static unsigned long rmi_l0gpt_size(void)
> +{
> + return 1UL << (30 + FIELD_GET(RMI_FEATURE_REGISTER_1_L0GPTSZ,
> + rmm_feat_reg1));
> +}
> +
> +static int rmi_create_gpts(phys_addr_t start, phys_addr_t end)
> +{
> + unsigned long l0gpt_sz = rmi_l0gpt_size();
> +
> + start = ALIGN_DOWN(start, l0gpt_sz);
> + end = ALIGN(end, l0gpt_sz);
> +
> + while (start < end) {
> + int ret = rmi_gpt_l1_create(start);
> +
> + if (ret && ret != RMI_ERROR_GPT) {
> + /*
> + * FIXME: Handle SRO so that memory can be donated for
> + * the tables.
> + */
> + kvm_err("GPT Level1 table missing for %llx\n", start);
> + return -ENOMEM;
> + }
> + start += l0gpt_sz;
> + }
> +
> + return 0;
> +}
> +
> +static int rmi_init_metadata(void)
> +{
> + phys_addr_t start, end;
> + const struct memblock_region *r;
> +
> + for_each_mem_region(r) {
> + int ret;
> +
> + start = memblock_region_memory_base_pfn(r) << PAGE_SHIFT;
> + end = memblock_region_memory_end_pfn(r) << PAGE_SHIFT;
> + ret = rmi_verify_memory_tracking(start, end);
> + if (ret)
> + return ret;
> + ret = rmi_create_gpts(start, end);
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> static int rmm_check_features(void)
> {
> if (kvm_lpa2_is_enabled() && !rmi_has_feature(RMI_FEATURE_REGISTER_0_LPA2)) {
> @@ -120,6 +207,8 @@ void kvm_init_rmi(void)
> return;
> if (rmi_configure())
> return;
> + if (rmi_init_metadata())
> + return;
>
> /* Future patch will enable static branch kvm_rmi_is_available */
> }
^ permalink raw reply
* Re: [PATCH v13 07/48] arm64: RMI: Check for RMI support at KVM init
From: Suzuki K Poulose @ 2026-03-19 10:38 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Zenghui Yu, linux-arm-kernel, linux-kernel,
Joey Gouly, Alexandru Elisei, Christoffer Dall, Fuad Tabba,
linux-coco, Ganapatrao Kulkarni, Gavin Shan, Shanker Donthineni,
Alper Gun, Aneesh Kumar K . V, Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20260318155413.793430-8-steven.price@arm.com>
On 18/03/2026 15:53, Steven Price wrote:
> Query the RMI version number and check if it is a compatible version. A
> static key is also provided to signal that a supported RMM is available.
>
> Functions are provided to query if a VM or VCPU is a realm (or rec)
> which currently will always return false.
>
> Later patches make use of struct realm and the states as the ioctls
> interfaces are added to support realm and REC creation and destruction.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v12:
> * Drop check for 4k page size.
> Changes since v11:
> * Reword slightly the comments on the realm states.
> Changes since v10:
> * kvm_is_realm() no longer has a NULL check.
> * Rename from "rme" to "rmi" when referring to the RMM interface.
> * Check for RME (hardware) support before probing for RMI support.
> Changes since v8:
> * No need to guard kvm_init_rme() behind 'in_hyp_mode'.
> Changes since v6:
> * Improved message for an unsupported RMI ABI version.
> Changes since v5:
> * Reword "unsupported" message from "host supports" to "we want" to
> clarify that 'we' are the 'host'.
> Changes since v2:
> * Drop return value from kvm_init_rme(), it was always 0.
> * Rely on the RMM return value to identify whether the RSI ABI is
> compatible.
> ---
> arch/arm64/include/asm/kvm_emulate.h | 18 +++++++++
> arch/arm64/include/asm/kvm_host.h | 4 ++
> arch/arm64/include/asm/kvm_rmi.h | 56 +++++++++++++++++++++++++++
> arch/arm64/include/asm/virt.h | 1 +
> arch/arm64/kernel/cpufeature.c | 1 +
> arch/arm64/kvm/Makefile | 2 +-
> arch/arm64/kvm/arm.c | 5 +++
> arch/arm64/kvm/rmi.c | 57 ++++++++++++++++++++++++++++
> 8 files changed, 143 insertions(+), 1 deletion(-)
> create mode 100644 arch/arm64/include/asm/kvm_rmi.h
> create mode 100644 arch/arm64/kvm/rmi.c
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 5bf3d7e1d92c..f38b50151ce8 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -688,4 +688,22 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
> vcpu->arch.hcrx_el2 |= HCRX_EL2_EnASR;
> }
> }
--8>---
> +
> +static inline bool kvm_is_realm(struct kvm *kvm)
> +{
> + if (static_branch_unlikely(&kvm_rmi_is_available))
> + return kvm->arch.is_realm;
> + return false;
> +}
> +
> +static inline enum realm_state kvm_realm_state(struct kvm *kvm)
> +{
> + return READ_ONCE(kvm->arch.realm.state);
> +}
> +
> +static inline bool vcpu_is_rec(struct kvm_vcpu *vcpu)
> +{
> + return false;
> +}
> +
> #endif /* __ARM64_KVM_EMULATE_H__ */
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 5d5a3bbdb95e..9267a2f2d65b 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -27,6 +27,7 @@
> #include <asm/fpsimd.h>
> #include <asm/kvm.h>
> #include <asm/kvm_asm.h>
> +#include <asm/kvm_rmi.h>
> #include <asm/vncr_mapping.h>
>
> #define __KVM_HAVE_ARCH_INTC_INITIALIZED
> @@ -405,6 +406,9 @@ struct kvm_arch {
> * the associated pKVM instance in the hypervisor.
> */
> struct kvm_protected_vm pkvm;
> +
> + bool is_realm;
> + struct realm realm;
> };
>
> struct kvm_vcpu_fault_info {
> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
> new file mode 100644
> index 000000000000..3506f50b05cd
> --- /dev/null
> +++ b/arch/arm64/include/asm/kvm_rmi.h
> @@ -0,0 +1,56 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2023-2025 ARM Ltd.
> + */
> +
> +#ifndef __ASM_KVM_RMI_H
> +#define __ASM_KVM_RMI_H
> +
> +/**
> + * enum realm_state - State of a Realm
> + */
> +enum realm_state {
> + /**
> + * @REALM_STATE_NONE:
> + * Realm has not yet been created. rmi_realm_create() has not
> + * yet been called.
> + */
> + REALM_STATE_NONE,
> + /**
> + * @REALM_STATE_NEW:
> + * Realm is under construction, rmi_realm_create() has been
> + * called, but it is not yet activated. Pages may be populated.
> + */
> + REALM_STATE_NEW,
> + /**
> + * @REALM_STATE_ACTIVE:
> + * Realm has been created and is eligible for execution with
> + * rmi_rec_enter(). Pages may no longer be populated with
> + * rmi_data_create().
> + */
> + REALM_STATE_ACTIVE,
> + /**
> + * @REALM_STATE_DYING:
> + * Realm is in the process of being destroyed or has already been
> + * destroyed.
> + */
> + REALM_STATE_DYING,
> + /**
> + * @REALM_STATE_DEAD:
> + * Realm has been destroyed.
> + */
> + REALM_STATE_DEAD
> +};
> +
> +/**
> + * struct realm - Additional per VM data for a Realm
> + *
> + * @state: The lifetime state machine for the realm
> + */
> +struct realm {
> + enum realm_state state;
> +};
> +
--8<---
Minor nit: The above looks out of place in this patch. Could we
move it to where this may be actually used ?
Rest looks good to me.
> +void kvm_init_rmi(void);
> +
> +#endif /* __ASM_KVM_RMI_H */
> diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
> index b51ab6840f9c..dc9b2899e0b2 100644
> --- a/arch/arm64/include/asm/virt.h
> +++ b/arch/arm64/include/asm/virt.h
> @@ -87,6 +87,7 @@ void __hyp_reset_vectors(void);
> bool is_kvm_arm_initialised(void);
>
> DECLARE_STATIC_KEY_FALSE(kvm_protected_mode_initialized);
> +DECLARE_STATIC_KEY_FALSE(kvm_rmi_is_available);
>
> static inline bool is_pkvm_initialized(void)
> {
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index c31f8e17732a..ddf7e57f23e8 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -289,6 +289,7 @@ static const struct arm64_ftr_bits ftr_id_aa64isar3[] = {
> static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_CSV3_SHIFT, 4, 0),
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_CSV2_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_RME_SHIFT, 4, 0),
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_DIT_SHIFT, 4, 0),
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_AMU_SHIFT, 4, 0),
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_MPAM_SHIFT, 4, 0),
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index 3ebc0570345c..e17c4077d8e7 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -16,7 +16,7 @@ CFLAGS_handle_exit.o += -Wno-override-init
> kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \
> inject_fault.o va_layout.o handle_exit.o config.o \
> guest.o debug.o reset.o sys_regs.o stacktrace.o \
> - vgic-sys-reg-v3.o fpsimd.o pkvm.o \
> + vgic-sys-reg-v3.o fpsimd.o pkvm.o rmi.o \
> arch_timer.o trng.o vmid.o emulate-nested.o nested.o at.o \
> vgic/vgic.o vgic/vgic-init.o \
> vgic/vgic-irqfd.o vgic/vgic-v2.o \
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 29f0326f7e00..274d7866efdc 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -39,6 +39,7 @@
> #include <asm/kvm_nested.h>
> #include <asm/kvm_pkvm.h>
> #include <asm/kvm_ptrauth.h>
> +#include <asm/kvm_rmi.h>
> #include <asm/sections.h>
> #include <asm/stacktrace/nvhe.h>
>
> @@ -104,6 +105,8 @@ long kvm_get_cap_for_kvm_ioctl(unsigned int ioctl, long *ext)
> return -EINVAL;
> }
>
> +DEFINE_STATIC_KEY_FALSE(kvm_rmi_is_available);
> +
> DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector);
>
> DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_base);
> @@ -2921,6 +2924,8 @@ static __init int kvm_arm_init(void)
>
> in_hyp_mode = is_kernel_in_hyp_mode();
>
> + kvm_init_rmi();
> +
> if (cpus_have_final_cap(ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE) ||
> cpus_have_final_cap(ARM64_WORKAROUND_1508412))
> kvm_info("Guests without required CPU erratum workarounds can deadlock system!\n" \
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> new file mode 100644
> index 000000000000..fac151580c01
> --- /dev/null
> +++ b/arch/arm64/kvm/rmi.c
> @@ -0,0 +1,57 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2023-2025 ARM Ltd.
> + */
> +
> +#include <linux/kvm_host.h>
> +
> +#include <asm/rmi_cmds.h>
> +#include <asm/virt.h>
> +
> +static int rmi_check_version(void)
> +{
> + struct arm_smccc_res res;
> + unsigned short version_major, version_minor;
> + unsigned long host_version = RMI_ABI_VERSION(RMI_ABI_MAJOR_VERSION,
> + RMI_ABI_MINOR_VERSION);
> + unsigned long aa64pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
> +
> + /* If RME isn't supported, then RMI can't be */
> + if (cpuid_feature_extract_unsigned_field(aa64pfr0, ID_AA64PFR0_EL1_RME_SHIFT) == 0)
> + return -ENXIO;
> +
> + arm_smccc_1_1_invoke(SMC_RMI_VERSION, host_version, &res);
> +
> + if (res.a0 == SMCCC_RET_NOT_SUPPORTED)
> + return -ENXIO;
> +
> + version_major = RMI_ABI_VERSION_GET_MAJOR(res.a1);
> + version_minor = RMI_ABI_VERSION_GET_MINOR(res.a1);
> +
> + if (res.a0 != RMI_SUCCESS) {
> + unsigned short high_version_major, high_version_minor;
> +
> + high_version_major = RMI_ABI_VERSION_GET_MAJOR(res.a2);
> + high_version_minor = RMI_ABI_VERSION_GET_MINOR(res.a2);
> +
> + kvm_err("Unsupported RMI ABI (v%d.%d - v%d.%d) we want v%d.%d\n",
> + version_major, version_minor,
> + high_version_major, high_version_minor,
> + RMI_ABI_MAJOR_VERSION,
> + RMI_ABI_MINOR_VERSION);
> + return -ENXIO;
> + }
> +
> + kvm_info("RMI ABI version %d.%d\n", version_major, version_minor);
> +
> + return 0;
> +}
> +
> +void kvm_init_rmi(void)
> +{
> + /* Continue without realm support if we can't agree on a version */
> + if (rmi_check_version())
> + return;
> +
> + /* Future patch will enable static branch kvm_rmi_is_available */
> +}
^ permalink raw reply
* Re: [PATCH 1/2] x86/virt/tdx: Use PFN directly for mapping guest private memory
From: Kiryl Shutsemau @ 2026-03-19 10:39 UTC (permalink / raw)
To: Yan Zhao
Cc: seanjc, pbonzini, dave.hansen, tglx, mingo, bp, x86, linux-kernel,
kvm, linux-coco, kai.huang, rick.p.edgecombe, yilun.xu,
vannapurve, ackerleytng, sagis, binbin.wu, xiaoyao.li,
isaku.yamahata
In-Reply-To: <20260319005703.8983-1-yan.y.zhao@intel.com>
On Thu, Mar 19, 2026 at 08:57:03AM +0800, Yan Zhao wrote:
> @@ -1639,16 +1644,17 @@ u64 tdh_vp_addcx(struct tdx_vp *vp, struct page *tdcx_page)
> }
> EXPORT_SYMBOL_FOR_KVM(tdh_vp_addcx);
>
> -u64 tdh_mem_page_aug(struct tdx_td *td, u64 gpa, int level, struct page *page, u64 *ext_err1, u64 *ext_err2)
> +u64 tdh_mem_page_aug(struct tdx_td *td, u64 gpa, int level, kvm_pfn_t pfn,
> + u64 *ext_err1, u64 *ext_err2)
> {
> struct tdx_module_args args = {
> .rcx = gpa | level,
> .rdx = tdx_tdr_pa(td),
> - .r8 = page_to_phys(page),
> + .r8 = PFN_PHYS(pfn),
> };
> u64 ret;
>
> - tdx_clflush_page(page);
> + tdx_clflush_pfn(pfn);
This is pre-existing problem, but shouldn't we respect @level here?
Flush size need to take page size into account.
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v13 10/48] arm64: RMI: Ensure that the RMM has GPT entries for memory
From: Suzuki K Poulose @ 2026-03-19 10:31 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Zenghui Yu, linux-arm-kernel, linux-kernel,
Joey Gouly, Alexandru Elisei, Christoffer Dall, Fuad Tabba,
linux-coco, Ganapatrao Kulkarni, Gavin Shan, Shanker Donthineni,
Alper Gun, Aneesh Kumar K . V, Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20260318155413.793430-11-steven.price@arm.com>
Hi Steven
On 18/03/2026 15:53, Steven Price wrote:
> The RMM may not be tracking all the memory of the system at boot. Create
Looks good to me. Please find some suggestions below.
May be add a bit more context here :
RMM maintains the state of all the granules in the System to make sure
that the host is abiding by the rules. This state can be maintained at
different granularity - per PAGE (TRACKING_FINE) or per region (COARSE),
where the "region size" depends on the underlying "RMI_GRANULE_SIZE".
The state of the "tracked area" must be the same. This implies, we may
need to have "FINE" tracking for DRAM, so that we can start delegating
PAGEs. For now, we only support RMM with statically carved out memory
for tracking FINE granularity for the tracking regions. We will extend
the support for modifying the TRACKING region in the future.
Similarly, the firmware may create L0 GPT entries describing the total
address space (think of this as Block mappings in the page tables). But
if we change the "PAS" of a granule in the block mapping, we may need
to create L1 tables to track the PAS at the finer granularity. For now
we only support a system where the L1 GPTs are created at boot time
and dynamic GPT support will be added later.
> the necessary tracking state and GPTs within the RMM so that all boot
> memory can be delegated to the RMM as needed during runtime.
>
> Note: support is currently missing for SROs which means that if the RMM
> needs memory donating this will fail (and render CCA unusable in Linux).
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> New patch for v13
> ---
> arch/arm64/kvm/rmi.c | 89 ++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 89 insertions(+)
>
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> index 9590dff9a2c1..80aedc85e94a 100644
> --- a/arch/arm64/kvm/rmi.c
> +++ b/arch/arm64/kvm/rmi.c
> @@ -4,6 +4,7 @@
> */
>
> #include <linux/kvm_host.h>
> +#include <linux/memblock.h>
>
> #include <asm/kvm_pgtable.h>
> #include <asm/rmi_cmds.h>
> @@ -56,6 +57,18 @@ static int rmi_check_version(void)
> return 0;
> }
>
> +/*
> + * These are the 'default' sizes when passing 0 as the tracking_region_size.
> + * TODO: Support other granule sizes
> + */
> +#ifdef CONFIG_PAGE_SIZE_4KB
> +#define RMM_GRANULE_TRACKING_SIZE SZ_1G
> +#elif defined(CONFIG_PAGE_SIZE_16KB)
> +#define RMM_GRANULE_TRACKING_SIZE SZ_32M
> +#elif defined(CONFIG_PAGE_SIZE_64KB)
> +#define RMM_GRANULE_TRACKING_SIZE SZ_512M
> +#endif
> +
Probably this should be made a Kconfig option, like the VA_BITS we have
today for each page size.
> static int rmi_configure(void)
> {
> struct rmm_config *config __free(free_page) = NULL;
> @@ -95,6 +108,80 @@ static int rmi_configure(void)
> return 0;
> }
>
> +static int rmi_verify_memory_tracking(phys_addr_t start, phys_addr_t end)
Could we add a comment what we are trying to do here ?
/*
* Make sure the area is tracked by RMM at FINE granularity.
* We do not support changing the TRACKING yet. This will
* be added in the future.
*/
> +{
> + start = ALIGN_DOWN(start, RMM_GRANULE_TRACKING_SIZE);
> + end = ALIGN(end, RMM_GRANULE_TRACKING_SIZE);
> +
> + while (start < end) {
> + unsigned long ret, category, state;
> +
> + ret = rmi_granule_tracking_get(start, &category, &state);
> + if (ret != RMI_SUCCESS ||
> + state != RMI_TRACKING_FINE ||
> + category != RMI_MEM_CATEGORY_CONVENTIONAL) {
> + /* TODO: Set granule tracking in this case */
> + kvm_err("Granule tracking for region isn't fine/conventional: %llx",
> + start);
> + return -ENODEV;
> + }
> + start += RMM_GRANULE_TRACKING_SIZE;
> + }
> +
> + return 0;
> +}
> +
> +static unsigned long rmi_l0gpt_size(void)
> +{
> + return 1UL << (30 + FIELD_GET(RMI_FEATURE_REGISTER_1_L0GPTSZ,
> + rmm_feat_reg1));
> +}
> +
> +static int rmi_create_gpts(phys_addr_t start, phys_addr_t end)
> +{
> + unsigned long l0gpt_sz = rmi_l0gpt_size();
> +
> + start = ALIGN_DOWN(start, l0gpt_sz);
> + end = ALIGN(end, l0gpt_sz);
> +
> + while (start < end) {
> + int ret = rmi_gpt_l1_create(start);
How about adding a comment here explaining why we look for RMI_ERROR_GPT ?
>
/*
* Make sure the L1 GPT tables are created for the region.
* RMI_ERROR_GPT indicates the L1 table exists.
*/
+
> + if (ret && ret != RMI_ERROR_GPT) {
> + /*
> + * FIXME: Handle SRO so that memory can be donated for
> + * the tables.
> + */
> + kvm_err("GPT Level1 table missing for %llx\n", start);
> + return -ENOMEM;
> + }
> + start += l0gpt_sz;
> + }
> +
> + return 0;
> +}
> +
> +static int rmi_init_metadata(void)
> +{
> + phys_addr_t start, end;
> + const struct memblock_region *r;
> +
> + for_each_mem_region(r) {
> + int ret;
> +
> + start = memblock_region_memory_base_pfn(r) << PAGE_SHIFT;
> + end = memblock_region_memory_end_pfn(r) << PAGE_SHIFT;
> + ret = rmi_verify_memory_tracking(start, end);
> + if (ret)
> + return ret;
> + ret = rmi_create_gpts(start, end);
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> static int rmm_check_features(void)
> {
> if (kvm_lpa2_is_enabled() && !rmi_has_feature(RMI_FEATURE_REGISTER_0_LPA2)) {
> @@ -120,6 +207,8 @@ void kvm_init_rmi(void)
> return;
> if (rmi_configure())
> return;
> + if (rmi_init_metadata())
> + return;
>
> /* Future patch will enable static branch kvm_rmi_is_available */
> }
^ permalink raw reply
* Re: [PATCH v13 37/48] arm64: RMI: Prevent Device mappings for Realms
From: Joey Gouly @ 2026-03-19 10:27 UTC (permalink / raw)
To: Steven Price
Cc: kvm, kvmarm, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20260318155413.793430-38-steven.price@arm.com>
On Wed, Mar 18, 2026 at 03:54:01PM +0000, Steven Price wrote:
> Physical device assignment is not supported by RMM v1.0, so it
But we're targetting 2.0 now!
I guess just change it to something about device support being a later feature.
Thanks,
Joey
> doesn't make much sense to allow device mappings within the realm.
> Prevent them when the guest is a realm.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes from v6:
> * Fix the check in user_mem_abort() to prevent all pages that are not
> guest_memfd() from being mapped into the protected half of the IPA.
> Changes from v5:
> * Also prevent accesses in user_mem_abort()
> ---
> arch/arm64/kvm/mmu.c | 13 +++++++++++++
> 1 file changed, 13 insertions(+)
>
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index ad1300f366df..7d7caab8f573 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -1222,6 +1222,10 @@ int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
> if (is_protected_kvm_enabled())
> return -EPERM;
>
> + /* We don't support mapping special pages into a Realm */
> + if (kvm_is_realm(kvm))
> + return -EPERM;
> +
> size += offset_in_page(guest_ipa);
> guest_ipa &= PAGE_MASK;
>
> @@ -1965,6 +1969,15 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> return 1;
> }
>
> + /*
> + * For now we shouldn't be hitting protected addresses because they are
> + * handled in private_memslot_fault(). In the future this check may be
> + * relaxed to support e.g. protected devices.
> + */
> + if (vcpu_is_rec(vcpu) &&
> + kvm_gpa_from_fault(kvm, fault_ipa) == fault_ipa)
> + return -EINVAL;
> +
> if (nested)
> adjust_nested_fault_perms(nested, &prot, &writable);
>
> --
> 2.43.0
>
>
^ permalink raw reply
* Re: [PATCH v13 36/48] arm64: RMI: Always use 4k pages for realms
From: Joey Gouly @ 2026-03-19 10:24 UTC (permalink / raw)
To: Steven Price
Cc: kvm, kvmarm, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20260318155413.793430-37-steven.price@arm.com>
Hi,
On Wed, Mar 18, 2026 at 03:54:00PM +0000, Steven Price wrote:
> Guest_memfd doesn't yet natively support huge pages, and there are
> currently difficulties for a VMM to manage huge pages efficiently so for
> now always split up mappings to PTE (4k).
>
> The two issues that need progressing before supporting huge pages for
> realms are:
>
> 1. guest_memfd needs to be able to allocate from an appropriate
> allocator which can provide huge pages.
>
> 2. The VMM needs to be able to repurpose private memory for a shared
> mapping when the guest VM requests memory is transitioned. Because
> this can happen at a 4k granularity it isn't possible to
> free/reallocate while huge pages are in use. Allowing the VMM to
> mmap() the shared portion of a huge page would allow the huge page
> to be recreated when the memory is unshared and made protected again.
>
> These two issues are not specific to realms and don't affect the realm
> API, so for now just break everything down to 4k pages in the RMM
> controlled stage 2. Future work can add huge page support without
> changing the uAPI.
The commit title/message mention 4K, but should probably say PAGE_SIZE or
something now that RMM isn't fixed to 4K.
Thanks,
Joey
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> Reviewed-by: Gavin Shan <gshan@redhat.com>
> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> ---
> Changes since v7:
> * Rewritten commit message
> ---
> arch/arm64/kvm/mmu.c | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 73c18c2861a2..ad1300f366df 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -1761,11 +1761,14 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> write_fault = kvm_is_write_fault(vcpu);
>
> /*
> - * Realms cannot map protected pages read-only
> + * Realms cannot map protected pages read-only, also force PTE mappings
> + * for Realms.
> * FIXME: It should be possible to map unprotected pages read-only
> */
> - if (vcpu_is_rec(vcpu))
> + if (vcpu_is_rec(vcpu)) {
> write_fault = true;
> + force_pte = true;
> + }
>
> exec_fault = kvm_vcpu_trap_is_exec_fault(vcpu);
> VM_WARN_ON_ONCE(write_fault && exec_fault);
> --
> 2.43.0
>
>
^ permalink raw reply
* Re: [PATCH 2/2] x86/virt/tdx: Use PFN directly for unmapping guest private memory
From: Yan Zhao @ 2026-03-19 8:56 UTC (permalink / raw)
To: Xiaoyao Li
Cc: seanjc, pbonzini, dave.hansen, tglx, mingo, bp, kas, x86,
linux-kernel, kvm, linux-coco, kai.huang, rick.p.edgecombe,
yilun.xu, vannapurve, ackerleytng, sagis, binbin.wu,
isaku.yamahata
In-Reply-To: <b74acb4b-4658-4113-9fce-9faf972975c4@intel.com>
On Thu, Mar 19, 2026 at 04:56:10PM +0800, Xiaoyao Li wrote:
> On 3/19/2026 2:45 PM, Yan Zhao wrote:
> > On Thu, Mar 19, 2026 at 11:20:48AM +0800, Xiaoyao Li wrote:
> > > On 3/19/2026 8:58 AM, Yan Zhao wrote:
> > > > From: Sean Christopherson <seanjc@google.com>
> [...]
> > > > diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> > > > index a9dd75190c67..2f9d07ad1a9a 100644
> > > > --- a/arch/x86/virt/vmx/tdx/tdx.c
> > > > +++ b/arch/x86/virt/vmx/tdx/tdx.c
> > > > @@ -730,9 +730,9 @@ static void tdx_quirk_reset_paddr(unsigned long base, unsigned long size)
> > > > mb();
> > > > }
> > > > -void tdx_quirk_reset_page(struct page *page)
> > > > +void tdx_quirk_reset_page(kvm_pfn_t pfn)
> > >
> > > So why keep the function tdx_quirk_reset_page() but expect passing in the
> > > kvm_pfn_t? It looks werid that the name indicates to reset a page but what
> > > gets passed in is a pfn.
> > I thought about introducing tdx_quirk_reset_pfn(). But considering
> > tdx_quirk_reset_pfn() has to be an exported API, I'm reluctant to do that.
> >
> > Given that even with tdx_quirk_reset_pfn(), it still expects TDX convertible
> > RAM, I think having tdx_quirk_reset_page() to take pfn is still acceptable.
> >
> > We just don't want KVM to do pfn --> struct page --> pfn conversions.
>
> Only tdx_sept_remove_private_spte() is doing such conversions. While
> tdx_reclaim_page() and tdx_reclaim_td_control_pages() already have the
> struct page natively.
Unlike requiring KVM to call pfn_to_page() before invoking guest private memory
related APIs, Having tdx_reclaim_page() and tdx_reclaim_td_control_pages() to
call page_to_pfn() does not impose unnecessary assumptions of how KVM allocates
memory. So, I think it's fine for them to invoke tdx_quirk_reset_page() which
takes PFN as input.
> So why not considering option 2?
>
> 2. keep tdx_quirk_reset_page() as-is for the cases of
> tdx_reclaim_page() and tdx_reclaim_td_control_pages() that have the
> struct page. But only change tdx_sept_remove_private_spte() to use
> tdx_quirk_reset_paddr() directly.
>
> It will need export tdx_quirk_reset_paddr() for KVM. I think it will be OK?
I don't think it's necessary. But if we have to export an extra API, IMHO,
tdx_quirk_reset_pfn() is better than tdx_quirk_reset_paddr(). Otherwise,
why not only expose tdx_quirk_reset_paddr()?
^ permalink raw reply
* Re: [PATCH 2/2] x86/virt/tdx: Use PFN directly for unmapping guest private memory
From: Xiaoyao Li @ 2026-03-19 8:56 UTC (permalink / raw)
To: Yan Zhao
Cc: seanjc, pbonzini, dave.hansen, tglx, mingo, bp, kas, x86,
linux-kernel, kvm, linux-coco, kai.huang, rick.p.edgecombe,
yilun.xu, vannapurve, ackerleytng, sagis, binbin.wu,
isaku.yamahata
In-Reply-To: <abubmi7BjMwgnrvC@yzhao56-desk.sh.intel.com>
On 3/19/2026 2:45 PM, Yan Zhao wrote:
> On Thu, Mar 19, 2026 at 11:20:48AM +0800, Xiaoyao Li wrote:
>> On 3/19/2026 8:58 AM, Yan Zhao wrote:
>>> From: Sean Christopherson <seanjc@google.com>
[...]
>>> diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
>>> index a9dd75190c67..2f9d07ad1a9a 100644
>>> --- a/arch/x86/virt/vmx/tdx/tdx.c
>>> +++ b/arch/x86/virt/vmx/tdx/tdx.c
>>> @@ -730,9 +730,9 @@ static void tdx_quirk_reset_paddr(unsigned long base, unsigned long size)
>>> mb();
>>> }
>>> -void tdx_quirk_reset_page(struct page *page)
>>> +void tdx_quirk_reset_page(kvm_pfn_t pfn)
>>
>> So why keep the function tdx_quirk_reset_page() but expect passing in the
>> kvm_pfn_t? It looks werid that the name indicates to reset a page but what
>> gets passed in is a pfn.
> I thought about introducing tdx_quirk_reset_pfn(). But considering
> tdx_quirk_reset_pfn() has to be an exported API, I'm reluctant to do that.
>
> Given that even with tdx_quirk_reset_pfn(), it still expects TDX convertible
> RAM, I think having tdx_quirk_reset_page() to take pfn is still acceptable.
>
> We just don't want KVM to do pfn --> struct page --> pfn conversions.
Only tdx_sept_remove_private_spte() is doing such conversions. While
tdx_reclaim_page() and tdx_reclaim_td_control_pages() already have the
struct page natively.
So why not considering option 2?
2. keep tdx_quirk_reset_page() as-is for the cases of
tdx_reclaim_page() and tdx_reclaim_td_control_pages() that have the
struct page. But only change tdx_sept_remove_private_spte() to use
tdx_quirk_reset_paddr() directly.
It will need export tdx_quirk_reset_paddr() for KVM. I think it will be OK?
^ permalink raw reply
* Re: [PATCH] KVM: TDX: Fix APIC MSR ranges in tdx_has_emulated_msr()
From: Binbin Wu @ 2026-03-19 7:40 UTC (permalink / raw)
To: Dave Hansen, Rick Edgecombe, Dmytro Maluka, kvm,
Sean Christopherson, Paolo Bonzini, Isaku Yamahata
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT), H. Peter Anvin,
Kiryl Shutsemau, open list:X86 ARCHITECTURE (32-BIT AND 64-BIT),
open list:X86 TRUST DOMAIN EXTENSIONS (TDX)
In-Reply-To: <93358559-5ed1-4574-8951-24d7ea9354e4@intel.com>
On 3/19/2026 9:48 AM, Dave Hansen wrote:
> On 3/18/26 18:14, Binbin Wu wrote:
>> The bug doesn't cause problems for TDs because:
>> - These x2apic MSRs (TASKPRI, PROCPRI, EOI, ISRx, TMRx, IRRx) are virtualized by CPU,
>> when a TD accesses these MSRs, it doesn't cause #VE, thus no TDVMCALL from the TD to
>> request the emulation of these MSRs.
>> - The bug make the "false" range of APIC MSRs smaller, so it doesn't impact the result
>> for the rest of the APIC MSRs.
>
> Could we fix this up so that the code that's there is actually usable
> and testable, please?
>
tdx_has_emulated_msr() is used by KVM to decide whether to emulate a MSR access from the
TDVMCALL or just return the error code.
During an off-list discussion, Rick noted that #VE reduction could change the behavior of
accessing an MSR (e.g., from #VE to #GP or to be virtualized by the TDX module) without
KVM knowing.Because KVM lacks the full context to perfectly decide if an MSR should be
emulated, the question was raised: Can we just delete tdx_has_emulated_msr() entirely?
However, these native type x2apic MSRs are a special case. Since the TDX module owns the
APICv page, KVM cannot emulate these MSRs. If we remove tdx_has_emulated_msr(), a guest
directly issuing TDVMCALLs for these native type x2apic MSRs will trigger a silent failure,
even though this is the guest's fault.
It comes down to a tradeoff. Should we prioritize code simplicity by dropping the function,
or keep it to explicitly catch this misbehaving guest corner case?
BTW, besides the bug described by this patch, according to the latest published TDX module
ABI table, MSR IA32_X2APIC_SELF_IPI is native type, but not included in the list.
There are some MSRs, which are reserved for xAPIC MSR, not included in the list, but they
can be covered by the KVM common code.
^ permalink raw reply
* Re: [PATCH v5 08/22] x86/virt/seamldr: Allocate and populate a module update request
From: Chao Gao @ 2026-03-19 7:24 UTC (permalink / raw)
To: Kiryl Shutsemau
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <abqFJZo632DAFsl_@thinkstation>
On Wed, Mar 18, 2026 at 10:58:44AM +0000, Kiryl Shutsemau wrote:
>On Wed, Mar 18, 2026 at 04:50:34PM +0800, Chao Gao wrote:
>> >> + ptr = sig;
>> >> + for (i = 0; i < sig_size / SZ_4K; i++) {
>> >> + /*
>> >> + * Don't assume @sig is page-aligned although it is 4KB-aligned.
>> >> + * Always add the in-page offset to get the physical address.
>> >> + */
>> >
>> >I don't follow this. If @sig is 4k aligned in VA, it is page aligned.
>>
>> Dan's concern was that PAGE_SIZE is not guaranteed to be 4096.
>>
>> I agree that PAGE_SIZE is 4K on x86 today. But to address that concern, I saw
>> two options:
>>
>> 1. Add WARN_ON_ONCE(PAGE_SIZE != SZ_4K), or
>> 2. Handle it as in the code above.
>>
>> I didn't find existing code using option 1 in x86, so I chose option 2.
>
>Please, make it clear in the comment that you are talking about PAGE_SIZE != SZ_4K.
Sure. How about:
/*
* @sig is 4KB-aligned, but that does not imply PAGE_SIZE
* alignment when PAGE_SIZE != SZ_4K. Always include the
* in-page offset.
*/
^ permalink raw reply
* Re: [PATCH 2/2] x86/virt/tdx: Use PFN directly for unmapping guest private memory
From: Yan Zhao @ 2026-03-19 6:45 UTC (permalink / raw)
To: Xiaoyao Li
Cc: seanjc, pbonzini, dave.hansen, tglx, mingo, bp, kas, x86,
linux-kernel, kvm, linux-coco, kai.huang, rick.p.edgecombe,
yilun.xu, vannapurve, ackerleytng, sagis, binbin.wu,
isaku.yamahata
In-Reply-To: <623ac08e-07a7-4823-bd0a-777d8df5c128@intel.com>
On Thu, Mar 19, 2026 at 11:20:48AM +0800, Xiaoyao Li wrote:
> On 3/19/2026 8:58 AM, Yan Zhao wrote:
> > From: Sean Christopherson <seanjc@google.com>
> >
> > Remove the completely unnecessary assumptions that memory unmapped from a
> > TDX guest is backed by refcounted struct page memory.
> >
> > APIs tdh_phymem_page_wbinvd_hkid(), tdx_quirk_reset_page() are used when
> > unmapping guest private memory from S-EPT. Since mapping of guest private
> > memory places no requirements on how KVM and guest_memfd manage memory,
> > neither does guest private memory unmapping.
> >
> > Rip out the misguided struct page assumptions/constraints by having the two
> > APIs take PFN directly. This ensures that for future huge page support in
> > S-EPT, the kernel doesn't pick up even worse assumptions like "a hugepage
> > must be contained in a single folio".
> >
> > Use "kvm_pfn_t pfn" for type safety. Using this KVM type is appropriate
> > since APIs tdh_phymem_page_wbinvd_hkid() and tdx_quirk_reset_page() are
> > exported to KVM only.
> >
> > Update mk_keyed_paddr(), which is invoked by tdh_phymem_page_wbinvd_hkid(),
> > to take PFN as parameter accordingly. Opportunistically, move
> > mk_keyed_paddr() from tdx.h to tdx.c since there are no external users.
> >
> > Have tdx_reclaim_page() remain using struct page as parameter since it's
> > currently not used for removing guest private memory yet.
> >
> > [Yan: Use kvm_pfn_t, drop reclaim API param update, move mk_keyed_paddr()]
> >
> > Signed-off-by: Sean Christopherson <seanjc@google.com>
> > Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
> > ---
> > arch/x86/include/asm/tdx.h | 15 ++-------------
> > arch/x86/kvm/vmx/tdx.c | 10 +++++-----
> > arch/x86/virt/vmx/tdx/tdx.c | 16 +++++++++++-----
> > 3 files changed, 18 insertions(+), 23 deletions(-)
> >
> > diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
> > index f3f0b1872176..6ceb4cd9ff21 100644
> > --- a/arch/x86/include/asm/tdx.h
> > +++ b/arch/x86/include/asm/tdx.h
> > @@ -153,7 +153,7 @@ int tdx_guest_keyid_alloc(void);
> > u32 tdx_get_nr_guest_keyids(void);
> > void tdx_guest_keyid_free(unsigned int keyid);
> > -void tdx_quirk_reset_page(struct page *page);
> > +void tdx_quirk_reset_page(kvm_pfn_t pfn);
> > struct tdx_td {
> > /* TD root structure: */
> > @@ -177,17 +177,6 @@ struct tdx_vp {
> > struct page **tdcx_pages;
> > };
> > -static inline u64 mk_keyed_paddr(u16 hkid, struct page *page)
> > -{
> > - u64 ret;
> > -
> > - ret = page_to_phys(page);
> > - /* KeyID bits are just above the physical address bits: */
> > - ret |= (u64)hkid << boot_cpu_data.x86_phys_bits;
> > -
> > - return ret;
> > -}
> > -
> > static inline int pg_level_to_tdx_sept_level(enum pg_level level)
> > {
> > WARN_ON_ONCE(level == PG_LEVEL_NONE);
> > @@ -219,7 +208,7 @@ u64 tdh_mem_track(struct tdx_td *tdr);
> > u64 tdh_mem_page_remove(struct tdx_td *td, u64 gpa, u64 level, u64 *ext_err1, u64 *ext_err2);
> > u64 tdh_phymem_cache_wb(bool resume);
> > u64 tdh_phymem_page_wbinvd_tdr(struct tdx_td *td);
> > -u64 tdh_phymem_page_wbinvd_hkid(u64 hkid, struct page *page);
> > +u64 tdh_phymem_page_wbinvd_hkid(u64 hkid, kvm_pfn_t pfn);
> > #else
> > static inline void tdx_init(void) { }
> > static inline u32 tdx_get_nr_guest_keyids(void) { return 0; }
> > diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> > index 1f1abc5b5655..75ad3debcd84 100644
> > --- a/arch/x86/kvm/vmx/tdx.c
> > +++ b/arch/x86/kvm/vmx/tdx.c
> > @@ -343,7 +343,7 @@ static int tdx_reclaim_page(struct page *page)
> > r = __tdx_reclaim_page(page);
> > if (!r)
> > - tdx_quirk_reset_page(page);
> > + tdx_quirk_reset_page(page_to_pfn(page));
> > return r;
> > }
> > @@ -597,7 +597,7 @@ static void tdx_reclaim_td_control_pages(struct kvm *kvm)
> > if (TDX_BUG_ON(err, TDH_PHYMEM_PAGE_WBINVD, kvm))
> > return;
> > - tdx_quirk_reset_page(kvm_tdx->td.tdr_page);
> > + tdx_quirk_reset_page(page_to_pfn(kvm_tdx->td.tdr_page));
> > __free_page(kvm_tdx->td.tdr_page);
> > kvm_tdx->td.tdr_page = NULL;
> > @@ -1776,9 +1776,9 @@ static int tdx_sept_free_private_spt(struct kvm *kvm, gfn_t gfn,
> > static void tdx_sept_remove_private_spte(struct kvm *kvm, gfn_t gfn,
> > enum pg_level level, u64 mirror_spte)
> > {
> > - struct page *page = pfn_to_page(spte_to_pfn(mirror_spte));
> > int tdx_level = pg_level_to_tdx_sept_level(level);
> > struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
> > + kvm_pfn_t pfn = spte_to_pfn(mirror_spte);
> > gpa_t gpa = gfn_to_gpa(gfn);
> > u64 err, entry, level_state;
> > @@ -1817,11 +1817,11 @@ static void tdx_sept_remove_private_spte(struct kvm *kvm, gfn_t gfn,
> > if (TDX_BUG_ON_2(err, TDH_MEM_PAGE_REMOVE, entry, level_state, kvm))
> > return;
> > - err = tdh_phymem_page_wbinvd_hkid((u16)kvm_tdx->hkid, page);
> > + err = tdh_phymem_page_wbinvd_hkid((u16)kvm_tdx->hkid, pfn);
> > if (TDX_BUG_ON(err, TDH_PHYMEM_PAGE_WBINVD, kvm))
> > return;
> > - tdx_quirk_reset_page(page);
> > + tdx_quirk_reset_page(pfn);
> > }
> > void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode,
> > diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> > index a9dd75190c67..2f9d07ad1a9a 100644
> > --- a/arch/x86/virt/vmx/tdx/tdx.c
> > +++ b/arch/x86/virt/vmx/tdx/tdx.c
> > @@ -730,9 +730,9 @@ static void tdx_quirk_reset_paddr(unsigned long base, unsigned long size)
> > mb();
> > }
> > -void tdx_quirk_reset_page(struct page *page)
> > +void tdx_quirk_reset_page(kvm_pfn_t pfn)
>
> So why keep the function tdx_quirk_reset_page() but expect passing in the
> kvm_pfn_t? It looks werid that the name indicates to reset a page but what
> gets passed in is a pfn.
I thought about introducing tdx_quirk_reset_pfn(). But considering
tdx_quirk_reset_pfn() has to be an exported API, I'm reluctant to do that.
Given that even with tdx_quirk_reset_pfn(), it still expects TDX convertible
RAM, I think having tdx_quirk_reset_page() to take pfn is still acceptable.
We just don't want KVM to do pfn --> struct page --> pfn conversions.
^ permalink raw reply
* Re: [PATCH v5 04/22] x86/virt/seamldr: Introduce a wrapper for P-SEAMLDR SEAMCALLs
From: Chao Gao @ 2026-03-19 7:14 UTC (permalink / raw)
To: Xiaoyao Li
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, kas, nik.borisov, paulmck,
pbonzini, reinette.chatre, rick.p.edgecombe, sagis, seanjc,
tony.lindgren, vannapurve, vishal.l.verma, yilun.xu, Farrah Chen,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, x86,
H. Peter Anvin
In-Reply-To: <8c319f32-b088-4594-a1b6-d6a755074a1b@intel.com>
>> I didn't realize that. Thanks for the information.
>>
>> But I don't think this is a stale reference. The SDM description is high-level
>> and points to the same spec for details:
>>
>> The details of such transitions are specified in the Intel® Trust Domain CPU
>> Architectural Extensions.
>
>The old "Intel® Trust Domain CPU Architectural Extensions" spec has bugs.
>Remember
>https://lore.kernel.org/all/20251014231042.1399849-1-seanjc@google.com/ ?
>They have been corrected in SDM.
>
>I'm not trying to say it's a stale reference. I just think directing people
>to an old doc (especially with an exact revision number) can make them get
>outdated and incorrect information.
Ok. Let me drop the whole paragraph.
^ permalink raw reply
* Re: [Invitation] bi-weekly guest_memfd upstream call on 2026-03-19
From: David Hildenbrand (Arm) @ 2026-03-19 6:05 UTC (permalink / raw)
To: linux-coco@lists.linux.dev, linux-mm@kvack.org, KVM
In-Reply-To: <22d1bdf9-4d21-4252-8558-e9618a409bc5@kernel.org>
On 3/18/26 11:50, David Hildenbrand (Arm) wrote:
> Hi,
>
> Our next guest_memfd upstream call is scheduled for tomorrow, Thursday,
> 2026-03-19 at 8:00 - 9:00am (GMT-07:00) Pacific Time - Vancouver.
>
> !!! Note the GMT-08:00 -> GMT-07:00 change due to DST in the US :) !!!
As Google calendar keeps dropping all invitees from the meeting for some
reason at random points in time--I suspect because I moved the calendar
entry over from my RH account by changing the owner--I just created a
new calendar entry and sent out invitations.
Hopefully it sticks now ...
--
Cheers,
David
^ permalink raw reply
* Re: [PATCH 2/2] x86/virt/tdx: Use PFN directly for unmapping guest private memory
From: Xiaoyao Li @ 2026-03-19 3:20 UTC (permalink / raw)
To: Yan Zhao, seanjc, pbonzini, dave.hansen
Cc: tglx, mingo, bp, kas, x86, linux-kernel, kvm, linux-coco,
kai.huang, rick.p.edgecombe, yilun.xu, vannapurve, ackerleytng,
sagis, binbin.wu, isaku.yamahata
In-Reply-To: <20260319005808.9013-1-yan.y.zhao@intel.com>
On 3/19/2026 8:58 AM, Yan Zhao wrote:
> From: Sean Christopherson <seanjc@google.com>
>
> Remove the completely unnecessary assumptions that memory unmapped from a
> TDX guest is backed by refcounted struct page memory.
>
> APIs tdh_phymem_page_wbinvd_hkid(), tdx_quirk_reset_page() are used when
> unmapping guest private memory from S-EPT. Since mapping of guest private
> memory places no requirements on how KVM and guest_memfd manage memory,
> neither does guest private memory unmapping.
>
> Rip out the misguided struct page assumptions/constraints by having the two
> APIs take PFN directly. This ensures that for future huge page support in
> S-EPT, the kernel doesn't pick up even worse assumptions like "a hugepage
> must be contained in a single folio".
>
> Use "kvm_pfn_t pfn" for type safety. Using this KVM type is appropriate
> since APIs tdh_phymem_page_wbinvd_hkid() and tdx_quirk_reset_page() are
> exported to KVM only.
>
> Update mk_keyed_paddr(), which is invoked by tdh_phymem_page_wbinvd_hkid(),
> to take PFN as parameter accordingly. Opportunistically, move
> mk_keyed_paddr() from tdx.h to tdx.c since there are no external users.
>
> Have tdx_reclaim_page() remain using struct page as parameter since it's
> currently not used for removing guest private memory yet.
>
> [Yan: Use kvm_pfn_t, drop reclaim API param update, move mk_keyed_paddr()]
>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
> ---
> arch/x86/include/asm/tdx.h | 15 ++-------------
> arch/x86/kvm/vmx/tdx.c | 10 +++++-----
> arch/x86/virt/vmx/tdx/tdx.c | 16 +++++++++++-----
> 3 files changed, 18 insertions(+), 23 deletions(-)
>
> diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
> index f3f0b1872176..6ceb4cd9ff21 100644
> --- a/arch/x86/include/asm/tdx.h
> +++ b/arch/x86/include/asm/tdx.h
> @@ -153,7 +153,7 @@ int tdx_guest_keyid_alloc(void);
> u32 tdx_get_nr_guest_keyids(void);
> void tdx_guest_keyid_free(unsigned int keyid);
>
> -void tdx_quirk_reset_page(struct page *page);
> +void tdx_quirk_reset_page(kvm_pfn_t pfn);
>
> struct tdx_td {
> /* TD root structure: */
> @@ -177,17 +177,6 @@ struct tdx_vp {
> struct page **tdcx_pages;
> };
>
> -static inline u64 mk_keyed_paddr(u16 hkid, struct page *page)
> -{
> - u64 ret;
> -
> - ret = page_to_phys(page);
> - /* KeyID bits are just above the physical address bits: */
> - ret |= (u64)hkid << boot_cpu_data.x86_phys_bits;
> -
> - return ret;
> -}
> -
> static inline int pg_level_to_tdx_sept_level(enum pg_level level)
> {
> WARN_ON_ONCE(level == PG_LEVEL_NONE);
> @@ -219,7 +208,7 @@ u64 tdh_mem_track(struct tdx_td *tdr);
> u64 tdh_mem_page_remove(struct tdx_td *td, u64 gpa, u64 level, u64 *ext_err1, u64 *ext_err2);
> u64 tdh_phymem_cache_wb(bool resume);
> u64 tdh_phymem_page_wbinvd_tdr(struct tdx_td *td);
> -u64 tdh_phymem_page_wbinvd_hkid(u64 hkid, struct page *page);
> +u64 tdh_phymem_page_wbinvd_hkid(u64 hkid, kvm_pfn_t pfn);
> #else
> static inline void tdx_init(void) { }
> static inline u32 tdx_get_nr_guest_keyids(void) { return 0; }
> diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> index 1f1abc5b5655..75ad3debcd84 100644
> --- a/arch/x86/kvm/vmx/tdx.c
> +++ b/arch/x86/kvm/vmx/tdx.c
> @@ -343,7 +343,7 @@ static int tdx_reclaim_page(struct page *page)
>
> r = __tdx_reclaim_page(page);
> if (!r)
> - tdx_quirk_reset_page(page);
> + tdx_quirk_reset_page(page_to_pfn(page));
> return r;
> }
>
> @@ -597,7 +597,7 @@ static void tdx_reclaim_td_control_pages(struct kvm *kvm)
> if (TDX_BUG_ON(err, TDH_PHYMEM_PAGE_WBINVD, kvm))
> return;
>
> - tdx_quirk_reset_page(kvm_tdx->td.tdr_page);
> + tdx_quirk_reset_page(page_to_pfn(kvm_tdx->td.tdr_page));
>
> __free_page(kvm_tdx->td.tdr_page);
> kvm_tdx->td.tdr_page = NULL;
> @@ -1776,9 +1776,9 @@ static int tdx_sept_free_private_spt(struct kvm *kvm, gfn_t gfn,
> static void tdx_sept_remove_private_spte(struct kvm *kvm, gfn_t gfn,
> enum pg_level level, u64 mirror_spte)
> {
> - struct page *page = pfn_to_page(spte_to_pfn(mirror_spte));
> int tdx_level = pg_level_to_tdx_sept_level(level);
> struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
> + kvm_pfn_t pfn = spte_to_pfn(mirror_spte);
> gpa_t gpa = gfn_to_gpa(gfn);
> u64 err, entry, level_state;
>
> @@ -1817,11 +1817,11 @@ static void tdx_sept_remove_private_spte(struct kvm *kvm, gfn_t gfn,
> if (TDX_BUG_ON_2(err, TDH_MEM_PAGE_REMOVE, entry, level_state, kvm))
> return;
>
> - err = tdh_phymem_page_wbinvd_hkid((u16)kvm_tdx->hkid, page);
> + err = tdh_phymem_page_wbinvd_hkid((u16)kvm_tdx->hkid, pfn);
> if (TDX_BUG_ON(err, TDH_PHYMEM_PAGE_WBINVD, kvm))
> return;
>
> - tdx_quirk_reset_page(page);
> + tdx_quirk_reset_page(pfn);
> }
>
> void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode,
> diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> index a9dd75190c67..2f9d07ad1a9a 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.c
> +++ b/arch/x86/virt/vmx/tdx/tdx.c
> @@ -730,9 +730,9 @@ static void tdx_quirk_reset_paddr(unsigned long base, unsigned long size)
> mb();
> }
>
> -void tdx_quirk_reset_page(struct page *page)
> +void tdx_quirk_reset_page(kvm_pfn_t pfn)
So why keep the function tdx_quirk_reset_page() but expect passing in
the kvm_pfn_t? It looks werid that the name indicates to reset a page
but what gets passed in is a pfn.
I think we have 2 options:
1. Drop helper tdx_quirk_reset_page() and use tdx_quirk_reset_paddr()
directly.
2. keep tdx_quirk_reset_page() as-is for the cases of tdx_reclaim_page()
and tdx_reclaim_td_control_pages() that have the struct page. But only
change tdx_sept_remove_private_spte() to use tdx_quirk_reset_paddr()
directly.
^ permalink raw reply
* Re: [PATCH v5 04/22] x86/virt/seamldr: Introduce a wrapper for P-SEAMLDR SEAMCALLs
From: Xiaoyao Li @ 2026-03-19 3:02 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, kas, nik.borisov, paulmck,
pbonzini, reinette.chatre, rick.p.edgecombe, sagis, seanjc,
tony.lindgren, vannapurve, vishal.l.verma, yilun.xu, Farrah Chen,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, x86,
H. Peter Anvin
In-Reply-To: <abpxvVk9XrEU6ssJ@intel.com>
On 3/18/2026 5:34 PM, Chao Gao wrote:
> On Wed, Mar 18, 2026 at 03:13:06PM +0800, Xiaoyao Li wrote:
>> On 3/15/2026 9:58 PM, Chao Gao wrote:
>>> The TDX architecture uses the "SEAMCALL" instruction to communicate with
>>> SEAM mode software. Right now, the only SEAM mode software that the kernel
>>> communicates with is the TDX module. But, there is actually another
>>> component that runs in SEAM mode but it is separate from the TDX module:
>>> the persistent SEAM loader or "P-SEAMLDR". Right now, the only component
>>> that communicates with it is the BIOS which loads the TDX module itself at
>>> boot. But, to support updating the TDX module, the kernel now needs to be
>>> able to talk to it.
>>>
>>> P-SEAMLDR SEAMCALLs differ from TDX module SEAMCALLs in areas such as
>>> concurrency requirements. Add a P-SEAMLDR wrapper to handle these
>>> differences and prepare for implementing concrete functions.
>>>
>>> Note that unlike P-SEAMLDR, there is also a non-persistent SEAM loader
>>> ("NP-SEAMLDR"). This is an authenticated code module (ACM) that is not
>>> callable at runtime. Only BIOS launches it to load P-SEAMLDR at boot;
>>> the kernel does not need to interact with it for runtime update.
>>>
>>> For details of P-SEAMLDR SEAMCALLs, see Intel® Trust Domain CPU
>>> Architectural Extensions, Revision 343754-002, Chapter 2.3 "INSTRUCTION
>>> SET REFERENCE".
>>
>> SDM started to contain SEAMCALL definitions. How about just dropping this
>> paragraph to avoid people from reading the old doc?
>
> I didn't realize that. Thanks for the information.
>
> But I don't think this is a stale reference. The SDM description is high-level
> and points to the same spec for details:
>
> The details of such transitions are specified in the Intel® Trust Domain CPU
> Architectural Extensions.
The old "Intel® Trust Domain CPU Architectural Extensions" spec has
bugs. Remember
https://lore.kernel.org/all/20251014231042.1399849-1-seanjc@google.com/
? They have been corrected in SDM.
I'm not trying to say it's a stale reference. I just think directing
people to an old doc (especially with an exact revision number) can make
them get outdated and incorrect information.
>>
>> ...
>>
>>> +static __maybe_unused int seamldr_call(u64 fn, struct tdx_module_args *args)
>>> +{
>>> + guard(raw_spinlock)(&seamldr_lock);
>>> + return seamcall_prerr(fn, args);
>>
>> How about adding the reason of why choosing seamcall_prerr() instead of
>> seamcall_prerr_ret() in the changelog?
>
> Sure. Will add:
>
> Use seamcall_prerr() (not '_ret') because current P-SEAMLDR calls do not use
> any output registers other than RAX.
looks good to me.
^ permalink raw reply
* Re: [PATCH] KVM: TDX: Fix APIC MSR ranges in tdx_has_emulated_msr()
From: Dave Hansen @ 2026-03-19 1:48 UTC (permalink / raw)
To: Binbin Wu, Dmytro Maluka, kvm, Sean Christopherson, Paolo Bonzini,
Isaku Yamahata
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT), H. Peter Anvin,
Kiryl Shutsemau, Rick Edgecombe,
open list:X86 ARCHITECTURE (32-BIT AND 64-BIT),
open list:X86 TRUST DOMAIN EXTENSIONS (TDX)
In-Reply-To: <cb99e44c-8bc9-4bab-b485-9f63a4d0b262@linux.intel.com>
On 3/18/26 18:14, Binbin Wu wrote:
> The bug doesn't cause problems for TDs because:
> - These x2apic MSRs (TASKPRI, PROCPRI, EOI, ISRx, TMRx, IRRx) are virtualized by CPU,
> when a TD accesses these MSRs, it doesn't cause #VE, thus no TDVMCALL from the TD to
> request the emulation of these MSRs.
> - The bug make the "false" range of APIC MSRs smaller, so it doesn't impact the result
> for the rest of the APIC MSRs.
Could we fix this up so that the code that's there is actually usable
and testable, please?
^ permalink raw reply
* [PATCH 2/2] x86/virt/tdx: Use PFN directly for unmapping guest private memory
From: Yan Zhao @ 2026-03-19 0:58 UTC (permalink / raw)
To: seanjc, pbonzini, dave.hansen
Cc: tglx, mingo, bp, kas, x86, linux-kernel, kvm, linux-coco,
kai.huang, rick.p.edgecombe, yan.y.zhao, yilun.xu, vannapurve,
ackerleytng, sagis, binbin.wu, xiaoyao.li, isaku.yamahata
In-Reply-To: <20260319005605.8965-1-yan.y.zhao@intel.com>
From: Sean Christopherson <seanjc@google.com>
Remove the completely unnecessary assumptions that memory unmapped from a
TDX guest is backed by refcounted struct page memory.
APIs tdh_phymem_page_wbinvd_hkid(), tdx_quirk_reset_page() are used when
unmapping guest private memory from S-EPT. Since mapping of guest private
memory places no requirements on how KVM and guest_memfd manage memory,
neither does guest private memory unmapping.
Rip out the misguided struct page assumptions/constraints by having the two
APIs take PFN directly. This ensures that for future huge page support in
S-EPT, the kernel doesn't pick up even worse assumptions like "a hugepage
must be contained in a single folio".
Use "kvm_pfn_t pfn" for type safety. Using this KVM type is appropriate
since APIs tdh_phymem_page_wbinvd_hkid() and tdx_quirk_reset_page() are
exported to KVM only.
Update mk_keyed_paddr(), which is invoked by tdh_phymem_page_wbinvd_hkid(),
to take PFN as parameter accordingly. Opportunistically, move
mk_keyed_paddr() from tdx.h to tdx.c since there are no external users.
Have tdx_reclaim_page() remain using struct page as parameter since it's
currently not used for removing guest private memory yet.
[Yan: Use kvm_pfn_t, drop reclaim API param update, move mk_keyed_paddr()]
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
---
arch/x86/include/asm/tdx.h | 15 ++-------------
arch/x86/kvm/vmx/tdx.c | 10 +++++-----
arch/x86/virt/vmx/tdx/tdx.c | 16 +++++++++++-----
3 files changed, 18 insertions(+), 23 deletions(-)
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index f3f0b1872176..6ceb4cd9ff21 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -153,7 +153,7 @@ int tdx_guest_keyid_alloc(void);
u32 tdx_get_nr_guest_keyids(void);
void tdx_guest_keyid_free(unsigned int keyid);
-void tdx_quirk_reset_page(struct page *page);
+void tdx_quirk_reset_page(kvm_pfn_t pfn);
struct tdx_td {
/* TD root structure: */
@@ -177,17 +177,6 @@ struct tdx_vp {
struct page **tdcx_pages;
};
-static inline u64 mk_keyed_paddr(u16 hkid, struct page *page)
-{
- u64 ret;
-
- ret = page_to_phys(page);
- /* KeyID bits are just above the physical address bits: */
- ret |= (u64)hkid << boot_cpu_data.x86_phys_bits;
-
- return ret;
-}
-
static inline int pg_level_to_tdx_sept_level(enum pg_level level)
{
WARN_ON_ONCE(level == PG_LEVEL_NONE);
@@ -219,7 +208,7 @@ u64 tdh_mem_track(struct tdx_td *tdr);
u64 tdh_mem_page_remove(struct tdx_td *td, u64 gpa, u64 level, u64 *ext_err1, u64 *ext_err2);
u64 tdh_phymem_cache_wb(bool resume);
u64 tdh_phymem_page_wbinvd_tdr(struct tdx_td *td);
-u64 tdh_phymem_page_wbinvd_hkid(u64 hkid, struct page *page);
+u64 tdh_phymem_page_wbinvd_hkid(u64 hkid, kvm_pfn_t pfn);
#else
static inline void tdx_init(void) { }
static inline u32 tdx_get_nr_guest_keyids(void) { return 0; }
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 1f1abc5b5655..75ad3debcd84 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -343,7 +343,7 @@ static int tdx_reclaim_page(struct page *page)
r = __tdx_reclaim_page(page);
if (!r)
- tdx_quirk_reset_page(page);
+ tdx_quirk_reset_page(page_to_pfn(page));
return r;
}
@@ -597,7 +597,7 @@ static void tdx_reclaim_td_control_pages(struct kvm *kvm)
if (TDX_BUG_ON(err, TDH_PHYMEM_PAGE_WBINVD, kvm))
return;
- tdx_quirk_reset_page(kvm_tdx->td.tdr_page);
+ tdx_quirk_reset_page(page_to_pfn(kvm_tdx->td.tdr_page));
__free_page(kvm_tdx->td.tdr_page);
kvm_tdx->td.tdr_page = NULL;
@@ -1776,9 +1776,9 @@ static int tdx_sept_free_private_spt(struct kvm *kvm, gfn_t gfn,
static void tdx_sept_remove_private_spte(struct kvm *kvm, gfn_t gfn,
enum pg_level level, u64 mirror_spte)
{
- struct page *page = pfn_to_page(spte_to_pfn(mirror_spte));
int tdx_level = pg_level_to_tdx_sept_level(level);
struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+ kvm_pfn_t pfn = spte_to_pfn(mirror_spte);
gpa_t gpa = gfn_to_gpa(gfn);
u64 err, entry, level_state;
@@ -1817,11 +1817,11 @@ static void tdx_sept_remove_private_spte(struct kvm *kvm, gfn_t gfn,
if (TDX_BUG_ON_2(err, TDH_MEM_PAGE_REMOVE, entry, level_state, kvm))
return;
- err = tdh_phymem_page_wbinvd_hkid((u16)kvm_tdx->hkid, page);
+ err = tdh_phymem_page_wbinvd_hkid((u16)kvm_tdx->hkid, pfn);
if (TDX_BUG_ON(err, TDH_PHYMEM_PAGE_WBINVD, kvm))
return;
- tdx_quirk_reset_page(page);
+ tdx_quirk_reset_page(pfn);
}
void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode,
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index a9dd75190c67..2f9d07ad1a9a 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -730,9 +730,9 @@ static void tdx_quirk_reset_paddr(unsigned long base, unsigned long size)
mb();
}
-void tdx_quirk_reset_page(struct page *page)
+void tdx_quirk_reset_page(kvm_pfn_t pfn)
{
- tdx_quirk_reset_paddr(page_to_phys(page), PAGE_SIZE);
+ tdx_quirk_reset_paddr(PFN_PHYS(pfn), PAGE_SIZE);
}
EXPORT_SYMBOL_FOR_KVM(tdx_quirk_reset_page);
@@ -1907,21 +1907,27 @@ u64 tdh_phymem_cache_wb(bool resume)
}
EXPORT_SYMBOL_FOR_KVM(tdh_phymem_cache_wb);
+static inline u64 mk_keyed_paddr(u16 hkid, kvm_pfn_t pfn)
+{
+ /* KeyID bits are just above the physical address bits. */
+ return PFN_PHYS(pfn) | ((u64)hkid << boot_cpu_data.x86_phys_bits);
+}
+
u64 tdh_phymem_page_wbinvd_tdr(struct tdx_td *td)
{
struct tdx_module_args args = {};
- args.rcx = mk_keyed_paddr(tdx_global_keyid, td->tdr_page);
+ args.rcx = mk_keyed_paddr(tdx_global_keyid, page_to_pfn(td->tdr_page));
return seamcall(TDH_PHYMEM_PAGE_WBINVD, &args);
}
EXPORT_SYMBOL_FOR_KVM(tdh_phymem_page_wbinvd_tdr);
-u64 tdh_phymem_page_wbinvd_hkid(u64 hkid, struct page *page)
+u64 tdh_phymem_page_wbinvd_hkid(u64 hkid, kvm_pfn_t pfn)
{
struct tdx_module_args args = {};
- args.rcx = mk_keyed_paddr(hkid, page);
+ args.rcx = mk_keyed_paddr(hkid, pfn);
return seamcall(TDH_PHYMEM_PAGE_WBINVD, &args);
}
--
2.43.2
^ permalink raw reply related
* [PATCH 1/2] x86/virt/tdx: Use PFN directly for mapping guest private memory
From: Yan Zhao @ 2026-03-19 0:57 UTC (permalink / raw)
To: seanjc, pbonzini, dave.hansen
Cc: tglx, mingo, bp, kas, x86, linux-kernel, kvm, linux-coco,
kai.huang, rick.p.edgecombe, yan.y.zhao, yilun.xu, vannapurve,
ackerleytng, sagis, binbin.wu, xiaoyao.li, isaku.yamahata
In-Reply-To: <20260319005605.8965-1-yan.y.zhao@intel.com>
From: Sean Christopherson <seanjc@google.com>
Remove the completely unnecessary assumption that memory mapped into a TDX
guest is backed by refcounted struct page memory. From KVM's point of view,
TDH_MEM_PAGE_ADD and TDH_MEM_PAGE_AUG are glorified writes to PTEs, so they
have no business placing requirements on how KVM and guest_memfd manage
memory.
Rip out the misguided struct page assumptions/constraints and instead have
the two SEAMCALL wrapper APIs take PFN directly. This ensures that for
future huge page support in S-EPT, the kernel doesn't pick up even worse
assumptions like "a hugepage must be contained in a single folio".
Use "kvm_pfn_t pfn" for type safety. Using this KVM type is appropriate
since APIs tdh_mem_page_add() and tdh_mem_page_aug() are exported to KVM
only.
[ Yan: Replace "u64 pfn" with "kvm_pfn_t pfn" ]
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
---
arch/x86/include/asm/tdx.h | 5 +++--
arch/x86/kvm/vmx/tdx.c | 7 +++----
arch/x86/virt/vmx/tdx/tdx.c | 20 +++++++++++++-------
3 files changed, 19 insertions(+), 13 deletions(-)
diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index a149740b24e8..f3f0b1872176 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -6,6 +6,7 @@
#include <linux/init.h>
#include <linux/bits.h>
#include <linux/mmzone.h>
+#include <linux/kvm_types.h>
#include <asm/errno.h>
#include <asm/ptrace.h>
@@ -195,10 +196,10 @@ static inline int pg_level_to_tdx_sept_level(enum pg_level level)
u64 tdh_vp_enter(struct tdx_vp *vp, struct tdx_module_args *args);
u64 tdh_mng_addcx(struct tdx_td *td, struct page *tdcs_page);
-u64 tdh_mem_page_add(struct tdx_td *td, u64 gpa, struct page *page, struct page *source, u64 *ext_err1, u64 *ext_err2);
+u64 tdh_mem_page_add(struct tdx_td *td, u64 gpa, kvm_pfn_t pfn, struct page *source, u64 *ext_err1, u64 *ext_err2);
u64 tdh_mem_sept_add(struct tdx_td *td, u64 gpa, int level, struct page *page, u64 *ext_err1, u64 *ext_err2);
u64 tdh_vp_addcx(struct tdx_vp *vp, struct page *tdcx_page);
-u64 tdh_mem_page_aug(struct tdx_td *td, u64 gpa, int level, struct page *page, u64 *ext_err1, u64 *ext_err2);
+u64 tdh_mem_page_aug(struct tdx_td *td, u64 gpa, int level, kvm_pfn_t pfn, u64 *ext_err1, u64 *ext_err2);
u64 tdh_mem_range_block(struct tdx_td *td, u64 gpa, int level, u64 *ext_err1, u64 *ext_err2);
u64 tdh_mng_key_config(struct tdx_td *td);
u64 tdh_mng_create(struct tdx_td *td, u16 hkid);
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 1e47c194af53..1f1abc5b5655 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1624,8 +1624,8 @@ static int tdx_mem_page_add(struct kvm *kvm, gfn_t gfn, enum pg_level level,
KVM_BUG_ON(!kvm_tdx->page_add_src, kvm))
return -EIO;
- err = tdh_mem_page_add(&kvm_tdx->td, gpa, pfn_to_page(pfn),
- kvm_tdx->page_add_src, &entry, &level_state);
+ err = tdh_mem_page_add(&kvm_tdx->td, gpa, pfn, kvm_tdx->page_add_src,
+ &entry, &level_state);
if (unlikely(tdx_operand_busy(err)))
return -EBUSY;
@@ -1640,12 +1640,11 @@ static int tdx_mem_page_aug(struct kvm *kvm, gfn_t gfn,
{
int tdx_level = pg_level_to_tdx_sept_level(level);
struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
- struct page *page = pfn_to_page(pfn);
gpa_t gpa = gfn_to_gpa(gfn);
u64 entry, level_state;
u64 err;
- err = tdh_mem_page_aug(&kvm_tdx->td, gpa, tdx_level, page, &entry, &level_state);
+ err = tdh_mem_page_aug(&kvm_tdx->td, gpa, tdx_level, pfn, &entry, &level_state);
if (unlikely(tdx_operand_busy(err)))
return -EBUSY;
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index cb9b3210ab71..a9dd75190c67 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -30,7 +30,6 @@
#include <linux/suspend.h>
#include <linux/syscore_ops.h>
#include <linux/idr.h>
-#include <linux/kvm_types.h>
#include <asm/page.h>
#include <asm/special_insns.h>
#include <asm/msr-index.h>
@@ -1568,6 +1567,11 @@ static void tdx_clflush_page(struct page *page)
clflush_cache_range(page_to_virt(page), PAGE_SIZE);
}
+static void tdx_clflush_pfn(kvm_pfn_t pfn)
+{
+ clflush_cache_range(__va(PFN_PHYS(pfn)), PAGE_SIZE);
+}
+
noinstr u64 tdh_vp_enter(struct tdx_vp *td, struct tdx_module_args *args)
{
args->rcx = td->tdvpr_pa;
@@ -1588,17 +1592,18 @@ u64 tdh_mng_addcx(struct tdx_td *td, struct page *tdcs_page)
}
EXPORT_SYMBOL_FOR_KVM(tdh_mng_addcx);
-u64 tdh_mem_page_add(struct tdx_td *td, u64 gpa, struct page *page, struct page *source, u64 *ext_err1, u64 *ext_err2)
+u64 tdh_mem_page_add(struct tdx_td *td, u64 gpa, kvm_pfn_t pfn, struct page *source,
+ u64 *ext_err1, u64 *ext_err2)
{
struct tdx_module_args args = {
.rcx = gpa,
.rdx = tdx_tdr_pa(td),
- .r8 = page_to_phys(page),
+ .r8 = PFN_PHYS(pfn),
.r9 = page_to_phys(source),
};
u64 ret;
- tdx_clflush_page(page);
+ tdx_clflush_pfn(pfn);
ret = seamcall_ret(TDH_MEM_PAGE_ADD, &args);
*ext_err1 = args.rcx;
@@ -1639,16 +1644,17 @@ u64 tdh_vp_addcx(struct tdx_vp *vp, struct page *tdcx_page)
}
EXPORT_SYMBOL_FOR_KVM(tdh_vp_addcx);
-u64 tdh_mem_page_aug(struct tdx_td *td, u64 gpa, int level, struct page *page, u64 *ext_err1, u64 *ext_err2)
+u64 tdh_mem_page_aug(struct tdx_td *td, u64 gpa, int level, kvm_pfn_t pfn,
+ u64 *ext_err1, u64 *ext_err2)
{
struct tdx_module_args args = {
.rcx = gpa | level,
.rdx = tdx_tdr_pa(td),
- .r8 = page_to_phys(page),
+ .r8 = PFN_PHYS(pfn),
};
u64 ret;
- tdx_clflush_page(page);
+ tdx_clflush_pfn(pfn);
ret = seamcall_ret(TDH_MEM_PAGE_AUG, &args);
*ext_err1 = args.rcx;
--
2.43.2
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox