* Re: [PATCH] sample/tsm-mr: Use SHA-2 library APIs
From: Eric Biggers @ 2026-03-19 17:32 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-crypto, Arnd Bergmann, Dan Williams, Dionna Amalie Glaze,
Cedric Xing, Andrew Morton, Zi Li, Masami Hiramatsu, Lance Yang,
Zhou Yuhang, Colin Ian King, Ard Biesheuvel, linux-coco,
linux-kernel
In-Reply-To: <dc124ea8-05b8-42d2-93ad-d265e0ecf585@app.fastmail.com>
On Wed, Mar 18, 2026 at 08:57:01PM +0100, Arnd Bergmann wrote:
> On Wed, Mar 18, 2026, at 17:42, Eric Biggers wrote:
> > Given that tsm_mr_sample has a particular set of algorithms that it
> > wants, just use the library APIs for those algorithms rather than
> > crypto_shash. This is more straightforward and a bit more efficient.
> >
> > This fixes an issue where this module failed to build due to the kconfig
> > options CRYPTO and CRYPTO_HASH not being selected. Also, even if it
> > built, crypto_alloc_shash() could fail at runtime due to the needed
> > algorithms not being available.
> >
> > The library functions simply use direct linking. So if it builds, which
> > it will due to the kconfig options being enabled, they are available.
> >
> > Fixes: f6953f1f9ec4 ("tsm-mr: Add tsm-mr sample code")
> > Fixes: 44a3873df811 ("coco/guest: Remove unneeded selection of CRYPTO")
> > Signed-off-by: Eric Biggers <ebiggers@kernel.org>
> > ---
> >
> > I'd like to take this via libcrypto-next, as that is where
> > "coco/guest: Remove unneeded selection of CRYPTO" is.
>
> Thanks for fixing this! It is indeed nicer than the fix
> I sent earlier today.
>
> Acked-by: Arnd Bergmann <arnd@arndb.de>
Thanks. Additional acks from the people owning this code (Dan, Cedric?)
would be appreciated. But since this fixes a build error and is related
to the crypto library, I went ahead and applied this to
https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git/log/?h=libcrypto-next
I also found that the build error is pre-existing, as CRYPTO_HASH was
not being selected. "coco/guest: Remove unneeded selection of CRYPTO"
just made it a bit easier to encounter, by not selecting CRYPTO either.
So I updated the second paragraph of the commit message to:
This also fixes a bug where this module failed to build if it was
enabled without CRYPTO_HASH happening to be set elsewhere in the
kconfig. (With the concurrent change to make TSM_MEASUREMENTS stop
selecting CRYPTO, this existing build error would have become easier to
encounter, as well.) Also, even if it built, crypto_alloc_shash() could
fail at runtime due to the needed algorithms not being available.
I also put this commit before "coco/guest: Remove unneeded selection of
CRYPTO" and dropped the Fixes reference to that. So now it just has:
Fixes: f6953f1f9ec4 ("tsm-mr: Add tsm-mr sample code")
- Eric
^ permalink raw reply
* Re: [PATCH 1/2] x86/virt/tdx: Use PFN directly for mapping guest private memory
From: Edgecombe, Rick P @ 2026-03-19 17:27 UTC (permalink / raw)
To: kas@kernel.org, Zhao, Yan Y
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Huang, Kai,
Li, Xiaoyao, dave.hansen@linux.intel.com,
linux-kernel@vger.kernel.org, binbin.wu@linux.intel.com,
seanjc@google.com, pbonzini@redhat.com, mingo@redhat.com,
ackerleytng@google.com, Yamahata, Isaku, sagis@google.com,
Annapurve, Vishal, bp@alien8.de, tglx@kernel.org,
yilun.xu@linux.intel.com, x86@kernel.org
In-Reply-To: <abvxiuJfK2eM_1UX@thinkstation>
On Thu, 2026-03-19 at 12:57 +0000, Kiryl Shutsemau wrote:
> > Though tdh_mem_page_aug() is an API, it is currently only exported to KVM
> > and
> > uses type kvm_pfn_t. So, is it still acceptable to assume flush size to be
> > PAGE_SIZE? Honoring level will soon be introduced by huge page patches.
>
> It caught my eye because previously size to flush was passed down to
> tdx_clflush_page() in the struct page (although never used there).
> With switching to pfn, we give up this information and it has to be
> passed separately. It would be easy to miss that in huge page patches,
> if we don't pass down level here.
>
> >
> > If you think it needs to be fixed before huge page series, what about fixing
> > it
> > in a separate cleanup patch? IMO, it would be better placed after Sean's
> > cleanup
> > patch [1], so we can use page_level_size() instead of inventing the wheel.
>
> I am okay with a separate patch.
I feel like we argued about this before. But it would be more correct to just
drop level and make it 4k only until huge pages? Otherwise we are tweaking dead
behavior.
^ permalink raw reply
* Re: [PATCH v13 12/48] arm64: RMI: Basic infrastructure for creating a realm.
From: Wei-Lin Chang @ 2026-03-19 17:17 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <20260318155413.793430-13-steven.price@arm.com>
On Wed, Mar 18, 2026 at 03:53:36PM +0000, Steven Price wrote:
> Introduce the skeleton functions for creating and destroying a realm.
> The IPA size requested is checked against what the RMM supports.
>
> The actual work of constructing the realm will be added in future
> patches.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v12:
> * Drop the RMM_PAGE_{SHIFT,SIZE} defines - the RMM is now configured to
> be the same as the host's page size.
> * Rework delegate/undelegate functions to use the new RMI range based
> operations.
> Changes since v11:
> * Major rework to drop the realm configuration and make the
> construction of realms implicit rather than driven by the VMM
> directly.
> * The code to create RDs, handle VMIDs etc is moved to later patches.
> Changes since v10:
> * Rename from RME to RMI.
> * Move the stage2 cleanup to a later patch.
> Changes since v9:
> * Avoid walking the stage 2 page tables when destroying the realm -
> the real ones are not accessible to the non-secure world, and the RMM
> may leave junk in the physical pages when returning them.
> * Fix an error path in realm_create_rd() to actually return an error value.
> Changes since v8:
> * Fix free_delegated_granule() to not call kvm_account_pgtable_pages();
> a separate wrapper will be introduced in a later patch to deal with
> RTTs.
> * Minor code cleanups following review.
> Changes since v7:
> * Minor code cleanup following Gavin's review.
> Changes since v6:
> * Separate RMM RTT calculations from host PAGE_SIZE. This allows the
> host page size to be larger than 4k while still communicating with an
> RMM which uses 4k granules.
> Changes since v5:
> * Introduce free_delegated_granule() to replace many
> undelegate/free_page() instances and centralise the comment on
> leaking when the undelegate fails.
> * Several other minor improvements suggested by reviews - thanks for
> the feedback!
> Changes since v2:
> * Improved commit description.
> * Improved return failures for rmi_check_version().
> * Clear contents of PGD after it has been undelegated in case the RMM
> left stale data.
> * Minor changes to reflect changes in previous patches.
> ---
> arch/arm64/include/asm/kvm_emulate.h | 5 ++
> arch/arm64/include/asm/kvm_rmi.h | 16 +++++
> arch/arm64/kvm/arm.c | 12 ++++
> arch/arm64/kvm/mmu.c | 11 +++-
> arch/arm64/kvm/rmi.c | 88 ++++++++++++++++++++++++++++
> 5 files changed, 129 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index f38b50151ce8..39310d9b4e16 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -701,6 +701,11 @@ static inline enum realm_state kvm_realm_state(struct kvm *kvm)
> return READ_ONCE(kvm->arch.realm.state);
> }
>
> +static inline bool kvm_realm_is_created(struct kvm *kvm)
> +{
> + return kvm_is_realm(kvm) && kvm_realm_state(kvm) != REALM_STATE_NONE;
> +}
> +
> static inline bool vcpu_is_rec(struct kvm_vcpu *vcpu)
> {
> return false;
> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
> index 3506f50b05cd..0ada525af18f 100644
> --- a/arch/arm64/include/asm/kvm_rmi.h
> +++ b/arch/arm64/include/asm/kvm_rmi.h
> @@ -6,6 +6,8 @@
> #ifndef __ASM_KVM_RMI_H
> #define __ASM_KVM_RMI_H
>
> +#include <asm/rmi_smc.h>
> +
> /**
> * enum realm_state - State of a Realm
> */
> @@ -46,11 +48,25 @@ enum realm_state {
> * struct realm - Additional per VM data for a Realm
> *
> * @state: The lifetime state machine for the realm
> + * @rd: Kernel mapping of the Realm Descriptor (RD)
> + * @params: Parameters for the RMI_REALM_CREATE command
> + * @num_aux: The number of auxiliary pages required by the RMM
> + * @ia_bits: Number of valid Input Address bits in the IPA
> */
> struct realm {
> enum realm_state state;
> +
> + void *rd;
> + struct realm_params *params;
> +
> + unsigned long num_aux;
> + unsigned int ia_bits;
> };
>
> void kvm_init_rmi(void);
> +u32 kvm_realm_ipa_limit(void);
> +
> +int kvm_init_realm_vm(struct kvm *kvm);
> +void kvm_destroy_realm(struct kvm *kvm);
Hi,
Sorry I missed one nit: perhaps call this kvm_init_realm()? So these two
look like a pair. There are also no realm_vm in other function names.
Thanks,
Wei-Lin Chang
>
> #endif /* __ASM_KVM_RMI_H */
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 274d7866efdc..9b17bdfaf0c2 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -253,6 +253,13 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
>
> bitmap_zero(kvm->arch.vcpu_features, KVM_VCPU_MAX_FEATURES);
>
> + /* Initialise the realm bits after the generic bits are enabled */
> + if (kvm_is_realm(kvm)) {
> + ret = kvm_init_realm_vm(kvm);
> + if (ret)
> + goto err_free_cpumask;
> + }
> +
> return 0;
>
> err_free_cpumask:
> @@ -312,6 +319,8 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
> kvm_unshare_hyp(kvm, kvm + 1);
>
> kvm_arm_teardown_hypercalls(kvm);
> + if (kvm_is_realm(kvm))
> + kvm_destroy_realm(kvm);
> }
>
> static bool kvm_has_full_ptr_auth(void)
> @@ -473,6 +482,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
> else
> r = kvm_supports_cacheable_pfnmap();
> break;
> + case KVM_CAP_ARM_RMI:
> + r = static_key_enabled(&kvm_rmi_is_available);
> + break;
>
> default:
> r = 0;
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 070a01e53fcb..d6094b60c4ce 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -872,12 +872,16 @@ static struct kvm_pgtable_mm_ops kvm_s2_mm_ops = {
> .icache_inval_pou = invalidate_icache_guest_page,
> };
>
> -static int kvm_init_ipa_range(struct kvm_s2_mmu *mmu, unsigned long type)
> +static int kvm_init_ipa_range(struct kvm *kvm,
> + struct kvm_s2_mmu *mmu, unsigned long type)
> {
> u32 kvm_ipa_limit = get_kvm_ipa_limit();
> u64 mmfr0, mmfr1;
> u32 phys_shift;
>
> + if (kvm_is_realm(kvm))
> + kvm_ipa_limit = kvm_realm_ipa_limit();
> +
> if (type & ~KVM_VM_TYPE_ARM_IPA_SIZE_MASK)
> return -EINVAL;
>
> @@ -974,7 +978,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
> return -EINVAL;
> }
>
> - err = kvm_init_ipa_range(mmu, type);
> + err = kvm_init_ipa_range(kvm, mmu, type);
> if (err)
> return err;
>
> @@ -1113,7 +1117,8 @@ void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
> write_unlock(&kvm->mmu_lock);
>
> if (pgt) {
> - kvm_stage2_destroy(pgt);
> + if (!kvm_is_realm(kvm))
> + kvm_stage2_destroy(pgt);
> kfree(pgt);
> }
> }
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> index 80aedc85e94a..700b8c935d29 100644
> --- a/arch/arm64/kvm/rmi.c
> +++ b/arch/arm64/kvm/rmi.c
> @@ -6,6 +6,8 @@
> #include <linux/kvm_host.h>
> #include <linux/memblock.h>
>
> +#include <asm/kvm_emulate.h>
> +#include <asm/kvm_mmu.h>
> #include <asm/kvm_pgtable.h>
> #include <asm/rmi_cmds.h>
> #include <asm/virt.h>
> @@ -182,6 +184,92 @@ static int rmi_init_metadata(void)
> return 0;
> }
>
> +u32 kvm_realm_ipa_limit(void)
> +{
> + return u64_get_bits(rmm_feat_reg0, RMI_FEATURE_REGISTER_0_S2SZ);
> +}
> +
> +static int undelegate_range(phys_addr_t phys, unsigned long size)
> +{
> + unsigned long ret;
> + unsigned long top = phys + size;
> + unsigned long out_top;
> +
> + while (phys < top) {
> + ret = rmi_granule_range_undelegate(phys, top, &out_top);
> + if (ret == RMI_SUCCESS)
> + phys = out_top;
> + else if (ret != RMI_BUSY && ret != RMI_BLOCKED)
> + return ret;
> + }
> +
> + return ret;
> +}
> +
> +static int undelegate_page(phys_addr_t phys)
> +{
> + return undelegate_range(phys, PAGE_SIZE);
> +}
> +
> +static int free_delegated_page(phys_addr_t phys)
> +{
> + if (WARN_ON(undelegate_page(phys))) {
> + /* Undelegate failed: leak the page */
> + return -EBUSY;
> + }
> +
> + free_page((unsigned long)phys_to_virt(phys));
> +
> + return 0;
> +}
> +
> +void kvm_destroy_realm(struct kvm *kvm)
> +{
> + struct realm *realm = &kvm->arch.realm;
> + size_t pgd_size = kvm_pgtable_stage2_pgd_size(kvm->arch.mmu.vtcr);
> +
> + write_lock(&kvm->mmu_lock);
> + kvm_stage2_unmap_range(&kvm->arch.mmu, 0,
> + BIT(realm->ia_bits - 1), true);
> + write_unlock(&kvm->mmu_lock);
> +
> + if (realm->params) {
> + free_page((unsigned long)realm->params);
> + realm->params = NULL;
> + }
> +
> + if (!kvm_realm_is_created(kvm))
> + return;
> +
> + WRITE_ONCE(realm->state, REALM_STATE_DYING);
> +
> + if (realm->rd) {
> + phys_addr_t rd_phys = virt_to_phys(realm->rd);
> +
> + if (WARN_ON(rmi_realm_destroy(rd_phys)))
> + return;
> + free_delegated_page(rd_phys);
> + realm->rd = NULL;
> + }
> +
> + if (WARN_ON(undelegate_range(kvm->arch.mmu.pgd_phys, pgd_size)))
> + return;
> +
> + WRITE_ONCE(realm->state, REALM_STATE_DEAD);
> +
> + /* Now that the Realm is destroyed, free the entry level RTTs */
> + kvm_free_stage2_pgd(&kvm->arch.mmu);
> +}
> +
> +int kvm_init_realm_vm(struct kvm *kvm)
> +{
> + kvm->arch.realm.params = (void *)get_zeroed_page(GFP_KERNEL);
> +
> + if (!kvm->arch.realm.params)
> + return -ENOMEM;
> + return 0;
> +}
> +
> static int rmm_check_features(void)
> {
> if (kvm_lpa2_is_enabled() && !rmi_has_feature(RMI_FEATURE_REGISTER_0_LPA2)) {
> --
> 2.43.0
>
^ permalink raw reply
* Re: [PATCH v13 07/48] arm64: RMI: Check for RMI support at KVM init
From: Steven Price @ 2026-03-19 16:42 UTC (permalink / raw)
To: Wei-Lin Chang, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <yogcawvd7444m6lbk3nz7y57zx6wdxvoxccyaddpkbniug27nw@zdihyc6xvrbx>
On 19/03/2026 16:17, Wei-Lin Chang wrote:
> On Wed, Mar 18, 2026 at 03:53:31PM +0000, Steven Price wrote:
>> Query the RMI version number and check if it is a compatible version. A
>> static key is also provided to signal that a supported RMM is available.
>>
>> Functions are provided to query if a VM or VCPU is a realm (or rec)
>> which currently will always return false.
>>
>> Later patches make use of struct realm and the states as the ioctls
>> interfaces are added to support realm and REC creation and destruction.
>>
>> Signed-off-by: Steven Price <steven.price@arm.com>
>> ---
>> Changes since v12:
>> * Drop check for 4k page size.
>> Changes since v11:
>> * Reword slightly the comments on the realm states.
>> Changes since v10:
>> * kvm_is_realm() no longer has a NULL check.
>> * Rename from "rme" to "rmi" when referring to the RMM interface.
>> * Check for RME (hardware) support before probing for RMI support.
>> Changes since v8:
>> * No need to guard kvm_init_rme() behind 'in_hyp_mode'.
>> Changes since v6:
>> * Improved message for an unsupported RMI ABI version.
>> Changes since v5:
>> * Reword "unsupported" message from "host supports" to "we want" to
>> clarify that 'we' are the 'host'.
>> Changes since v2:
>> * Drop return value from kvm_init_rme(), it was always 0.
>> * Rely on the RMM return value to identify whether the RSI ABI is
>> compatible.
>> ---
>> arch/arm64/include/asm/kvm_emulate.h | 18 +++++++++
>> arch/arm64/include/asm/kvm_host.h | 4 ++
>> arch/arm64/include/asm/kvm_rmi.h | 56 +++++++++++++++++++++++++++
>> arch/arm64/include/asm/virt.h | 1 +
>> arch/arm64/kernel/cpufeature.c | 1 +
>> arch/arm64/kvm/Makefile | 2 +-
>> arch/arm64/kvm/arm.c | 5 +++
>> arch/arm64/kvm/rmi.c | 57 ++++++++++++++++++++++++++++
>> 8 files changed, 143 insertions(+), 1 deletion(-)
>> create mode 100644 arch/arm64/include/asm/kvm_rmi.h
>> create mode 100644 arch/arm64/kvm/rmi.c
>>
>> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
>> index 5bf3d7e1d92c..f38b50151ce8 100644
>> --- a/arch/arm64/include/asm/kvm_emulate.h
>> +++ b/arch/arm64/include/asm/kvm_emulate.h
>> @@ -688,4 +688,22 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
>> vcpu->arch.hcrx_el2 |= HCRX_EL2_EnASR;
>> }
>> }
>> +
>> +static inline bool kvm_is_realm(struct kvm *kvm)
>> +{
>> + if (static_branch_unlikely(&kvm_rmi_is_available))
>> + return kvm->arch.is_realm;
>> + return false;
>> +}
>> +
>> +static inline enum realm_state kvm_realm_state(struct kvm *kvm)
>> +{
>> + return READ_ONCE(kvm->arch.realm.state);
>> +}
>> +
>> +static inline bool vcpu_is_rec(struct kvm_vcpu *vcpu)
>> +{
>> + return false;
>> +}
>> +
>
> Hi,
>
> Both kvm_vm_is_protected() and vcpu_is_protected() are in kvm_host.h, do
> you think that's a better place? Or is there a reason for this being in
> kvm_emulate.h?
I've no strong opinions, but as usual there are complications with the
header include order.
kvm_vm_is_protected/vcpu_is_protected are defined in asm/kvm_host.h, but
struct kvm is defined in linux/kvm_host.h (which includes
asm/kvm_host.h). The same is true for struct kvm_vcpu. This means that
we can't have a static inline function using the kvm pointer (because
the struct definition hasn't been reached). The 'solution' in the case
of kvm_vm_is_protected/vcpu_is_protected is to use a macro - which works
but has drawbacks (e.g. lack of type checking).
I'll move them if there's a strong feeling they are in the wrong place,
but to me it feels more like the macros in asm/kvm_host.h are just a
hack to get around them being in the wrong place.
Thanks,
Steve
> Thanks,
> Wei-Lin Chang
>
>> #endif /* __ARM64_KVM_EMULATE_H__ */
>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
>> index 5d5a3bbdb95e..9267a2f2d65b 100644
>> --- a/arch/arm64/include/asm/kvm_host.h
>> +++ b/arch/arm64/include/asm/kvm_host.h
>> @@ -27,6 +27,7 @@
>> #include <asm/fpsimd.h>
>> #include <asm/kvm.h>
>> #include <asm/kvm_asm.h>
>> +#include <asm/kvm_rmi.h>
>> #include <asm/vncr_mapping.h>
>>
>> #define __KVM_HAVE_ARCH_INTC_INITIALIZED
>> @@ -405,6 +406,9 @@ struct kvm_arch {
>> * the associated pKVM instance in the hypervisor.
>> */
>> struct kvm_protected_vm pkvm;
>> +
>> + bool is_realm;
>> + struct realm realm;
>> };
>>
>> struct kvm_vcpu_fault_info {
>> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
>> new file mode 100644
>> index 000000000000..3506f50b05cd
>> --- /dev/null
>> +++ b/arch/arm64/include/asm/kvm_rmi.h
>> @@ -0,0 +1,56 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + * Copyright (C) 2023-2025 ARM Ltd.
>> + */
>> +
>> +#ifndef __ASM_KVM_RMI_H
>> +#define __ASM_KVM_RMI_H
>> +
>> +/**
>> + * enum realm_state - State of a Realm
>> + */
>> +enum realm_state {
>> + /**
>> + * @REALM_STATE_NONE:
>> + * Realm has not yet been created. rmi_realm_create() has not
>> + * yet been called.
>> + */
>> + REALM_STATE_NONE,
>> + /**
>> + * @REALM_STATE_NEW:
>> + * Realm is under construction, rmi_realm_create() has been
>> + * called, but it is not yet activated. Pages may be populated.
>> + */
>> + REALM_STATE_NEW,
>> + /**
>> + * @REALM_STATE_ACTIVE:
>> + * Realm has been created and is eligible for execution with
>> + * rmi_rec_enter(). Pages may no longer be populated with
>> + * rmi_data_create().
>> + */
>> + REALM_STATE_ACTIVE,
>> + /**
>> + * @REALM_STATE_DYING:
>> + * Realm is in the process of being destroyed or has already been
>> + * destroyed.
>> + */
>> + REALM_STATE_DYING,
>> + /**
>> + * @REALM_STATE_DEAD:
>> + * Realm has been destroyed.
>> + */
>> + REALM_STATE_DEAD
>> +};
>> +
>> +/**
>> + * struct realm - Additional per VM data for a Realm
>> + *
>> + * @state: The lifetime state machine for the realm
>> + */
>> +struct realm {
>> + enum realm_state state;
>> +};
>> +
>> +void kvm_init_rmi(void);
>> +
>> +#endif /* __ASM_KVM_RMI_H */
>> diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
>> index b51ab6840f9c..dc9b2899e0b2 100644
>> --- a/arch/arm64/include/asm/virt.h
>> +++ b/arch/arm64/include/asm/virt.h
>> @@ -87,6 +87,7 @@ void __hyp_reset_vectors(void);
>> bool is_kvm_arm_initialised(void);
>>
>> DECLARE_STATIC_KEY_FALSE(kvm_protected_mode_initialized);
>> +DECLARE_STATIC_KEY_FALSE(kvm_rmi_is_available);
>>
>> static inline bool is_pkvm_initialized(void)
>> {
>> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
>> index c31f8e17732a..ddf7e57f23e8 100644
>> --- a/arch/arm64/kernel/cpufeature.c
>> +++ b/arch/arm64/kernel/cpufeature.c
>> @@ -289,6 +289,7 @@ static const struct arm64_ftr_bits ftr_id_aa64isar3[] = {
>> static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
>> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_CSV3_SHIFT, 4, 0),
>> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_CSV2_SHIFT, 4, 0),
>> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_RME_SHIFT, 4, 0),
>> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_DIT_SHIFT, 4, 0),
>> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_AMU_SHIFT, 4, 0),
>> ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_MPAM_SHIFT, 4, 0),
>> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> index 3ebc0570345c..e17c4077d8e7 100644
>> --- a/arch/arm64/kvm/Makefile
>> +++ b/arch/arm64/kvm/Makefile
>> @@ -16,7 +16,7 @@ CFLAGS_handle_exit.o += -Wno-override-init
>> kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \
>> inject_fault.o va_layout.o handle_exit.o config.o \
>> guest.o debug.o reset.o sys_regs.o stacktrace.o \
>> - vgic-sys-reg-v3.o fpsimd.o pkvm.o \
>> + vgic-sys-reg-v3.o fpsimd.o pkvm.o rmi.o \
>> arch_timer.o trng.o vmid.o emulate-nested.o nested.o at.o \
>> vgic/vgic.o vgic/vgic-init.o \
>> vgic/vgic-irqfd.o vgic/vgic-v2.o \
>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
>> index 29f0326f7e00..274d7866efdc 100644
>> --- a/arch/arm64/kvm/arm.c
>> +++ b/arch/arm64/kvm/arm.c
>> @@ -39,6 +39,7 @@
>> #include <asm/kvm_nested.h>
>> #include <asm/kvm_pkvm.h>
>> #include <asm/kvm_ptrauth.h>
>> +#include <asm/kvm_rmi.h>
>> #include <asm/sections.h>
>> #include <asm/stacktrace/nvhe.h>
>>
>> @@ -104,6 +105,8 @@ long kvm_get_cap_for_kvm_ioctl(unsigned int ioctl, long *ext)
>> return -EINVAL;
>> }
>>
>> +DEFINE_STATIC_KEY_FALSE(kvm_rmi_is_available);
>> +
>> DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector);
>>
>> DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_base);
>> @@ -2921,6 +2924,8 @@ static __init int kvm_arm_init(void)
>>
>> in_hyp_mode = is_kernel_in_hyp_mode();
>>
>> + kvm_init_rmi();
>> +
>> if (cpus_have_final_cap(ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE) ||
>> cpus_have_final_cap(ARM64_WORKAROUND_1508412))
>> kvm_info("Guests without required CPU erratum workarounds can deadlock system!\n" \
>> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
>> new file mode 100644
>> index 000000000000..fac151580c01
>> --- /dev/null
>> +++ b/arch/arm64/kvm/rmi.c
>> @@ -0,0 +1,57 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Copyright (C) 2023-2025 ARM Ltd.
>> + */
>> +
>> +#include <linux/kvm_host.h>
>> +
>> +#include <asm/rmi_cmds.h>
>> +#include <asm/virt.h>
>> +
>> +static int rmi_check_version(void)
>> +{
>> + struct arm_smccc_res res;
>> + unsigned short version_major, version_minor;
>> + unsigned long host_version = RMI_ABI_VERSION(RMI_ABI_MAJOR_VERSION,
>> + RMI_ABI_MINOR_VERSION);
>> + unsigned long aa64pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
>> +
>> + /* If RME isn't supported, then RMI can't be */
>> + if (cpuid_feature_extract_unsigned_field(aa64pfr0, ID_AA64PFR0_EL1_RME_SHIFT) == 0)
>> + return -ENXIO;
>> +
>> + arm_smccc_1_1_invoke(SMC_RMI_VERSION, host_version, &res);
>> +
>> + if (res.a0 == SMCCC_RET_NOT_SUPPORTED)
>> + return -ENXIO;
>> +
>> + version_major = RMI_ABI_VERSION_GET_MAJOR(res.a1);
>> + version_minor = RMI_ABI_VERSION_GET_MINOR(res.a1);
>> +
>> + if (res.a0 != RMI_SUCCESS) {
>> + unsigned short high_version_major, high_version_minor;
>> +
>> + high_version_major = RMI_ABI_VERSION_GET_MAJOR(res.a2);
>> + high_version_minor = RMI_ABI_VERSION_GET_MINOR(res.a2);
>> +
>> + kvm_err("Unsupported RMI ABI (v%d.%d - v%d.%d) we want v%d.%d\n",
>> + version_major, version_minor,
>> + high_version_major, high_version_minor,
>> + RMI_ABI_MAJOR_VERSION,
>> + RMI_ABI_MINOR_VERSION);
>> + return -ENXIO;
>> + }
>> +
>> + kvm_info("RMI ABI version %d.%d\n", version_major, version_minor);
>> +
>> + return 0;
>> +}
>> +
>> +void kvm_init_rmi(void)
>> +{
>> + /* Continue without realm support if we can't agree on a version */
>> + if (rmi_check_version())
>> + return;
>> +
>> + /* Future patch will enable static branch kvm_rmi_is_available */
>> +}
>> --
>> 2.43.0
>>
^ permalink raw reply
* Re: [PATCH v13 12/48] arm64: RMI: Basic infrastructure for creating a realm.
From: Steven Price @ 2026-03-19 16:24 UTC (permalink / raw)
To: Wei-Lin Chang, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <lt5yozoeeqld5laooopzjwngqlso2hdpjcwkkyagbcg4vedkd6@potsbanqeb4h>
On 19/03/2026 16:11, Wei-Lin Chang wrote:
> On Wed, Mar 18, 2026 at 03:53:36PM +0000, Steven Price wrote:
>> Introduce the skeleton functions for creating and destroying a realm.
>> The IPA size requested is checked against what the RMM supports.
>>
>> The actual work of constructing the realm will be added in future
>> patches.
>>
>> Signed-off-by: Steven Price <steven.price@arm.com>
>> ---
>> Changes since v12:
>> * Drop the RMM_PAGE_{SHIFT,SIZE} defines - the RMM is now configured to
>> be the same as the host's page size.
>> * Rework delegate/undelegate functions to use the new RMI range based
>> operations.
>> Changes since v11:
>> * Major rework to drop the realm configuration and make the
>> construction of realms implicit rather than driven by the VMM
>> directly.
>> * The code to create RDs, handle VMIDs etc is moved to later patches.
>> Changes since v10:
>> * Rename from RME to RMI.
>> * Move the stage2 cleanup to a later patch.
>> Changes since v9:
>> * Avoid walking the stage 2 page tables when destroying the realm -
>> the real ones are not accessible to the non-secure world, and the RMM
>> may leave junk in the physical pages when returning them.
>> * Fix an error path in realm_create_rd() to actually return an error value.
>> Changes since v8:
>> * Fix free_delegated_granule() to not call kvm_account_pgtable_pages();
>> a separate wrapper will be introduced in a later patch to deal with
>> RTTs.
>> * Minor code cleanups following review.
>> Changes since v7:
>> * Minor code cleanup following Gavin's review.
>> Changes since v6:
>> * Separate RMM RTT calculations from host PAGE_SIZE. This allows the
>> host page size to be larger than 4k while still communicating with an
>> RMM which uses 4k granules.
>> Changes since v5:
>> * Introduce free_delegated_granule() to replace many
>> undelegate/free_page() instances and centralise the comment on
>> leaking when the undelegate fails.
>> * Several other minor improvements suggested by reviews - thanks for
>> the feedback!
>> Changes since v2:
>> * Improved commit description.
>> * Improved return failures for rmi_check_version().
>> * Clear contents of PGD after it has been undelegated in case the RMM
>> left stale data.
>> * Minor changes to reflect changes in previous patches.
>> ---
>> arch/arm64/include/asm/kvm_emulate.h | 5 ++
>> arch/arm64/include/asm/kvm_rmi.h | 16 +++++
>> arch/arm64/kvm/arm.c | 12 ++++
>> arch/arm64/kvm/mmu.c | 11 +++-
>> arch/arm64/kvm/rmi.c | 88 ++++++++++++++++++++++++++++
>> 5 files changed, 129 insertions(+), 3 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
>> index f38b50151ce8..39310d9b4e16 100644
>> --- a/arch/arm64/include/asm/kvm_emulate.h
>> +++ b/arch/arm64/include/asm/kvm_emulate.h
>> @@ -701,6 +701,11 @@ static inline enum realm_state kvm_realm_state(struct kvm *kvm)
>> return READ_ONCE(kvm->arch.realm.state);
>> }
>>
>> +static inline bool kvm_realm_is_created(struct kvm *kvm)
>> +{
>> + return kvm_is_realm(kvm) && kvm_realm_state(kvm) != REALM_STATE_NONE;
>> +}
>> +
>> static inline bool vcpu_is_rec(struct kvm_vcpu *vcpu)
>> {
>> return false;
>> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
>> index 3506f50b05cd..0ada525af18f 100644
>> --- a/arch/arm64/include/asm/kvm_rmi.h
>> +++ b/arch/arm64/include/asm/kvm_rmi.h
>> @@ -6,6 +6,8 @@
>> #ifndef __ASM_KVM_RMI_H
>> #define __ASM_KVM_RMI_H
>>
>> +#include <asm/rmi_smc.h>
>> +
>> /**
>> * enum realm_state - State of a Realm
>> */
>> @@ -46,11 +48,25 @@ enum realm_state {
>> * struct realm - Additional per VM data for a Realm
>> *
>> * @state: The lifetime state machine for the realm
>> + * @rd: Kernel mapping of the Realm Descriptor (RD)
>> + * @params: Parameters for the RMI_REALM_CREATE command
>> + * @num_aux: The number of auxiliary pages required by the RMM
>> + * @ia_bits: Number of valid Input Address bits in the IPA
>> */
>> struct realm {
>> enum realm_state state;
>> +
>> + void *rd;
>> + struct realm_params *params;
>> +
>> + unsigned long num_aux;
>> + unsigned int ia_bits;
>> };
>>
>> void kvm_init_rmi(void);
>> +u32 kvm_realm_ipa_limit(void);
>> +
>> +int kvm_init_realm_vm(struct kvm *kvm);
>> +void kvm_destroy_realm(struct kvm *kvm);
>>
>> #endif /* __ASM_KVM_RMI_H */
>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
>> index 274d7866efdc..9b17bdfaf0c2 100644
>> --- a/arch/arm64/kvm/arm.c
>> +++ b/arch/arm64/kvm/arm.c
>> @@ -253,6 +253,13 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
>>
>> bitmap_zero(kvm->arch.vcpu_features, KVM_VCPU_MAX_FEATURES);
>>
>> + /* Initialise the realm bits after the generic bits are enabled */
>> + if (kvm_is_realm(kvm)) {
>> + ret = kvm_init_realm_vm(kvm);
>> + if (ret)
>> + goto err_free_cpumask;
>> + }
>> +
>> return 0;
>>
>> err_free_cpumask:
>> @@ -312,6 +319,8 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
>> kvm_unshare_hyp(kvm, kvm + 1);
>>
>> kvm_arm_teardown_hypercalls(kvm);
>> + if (kvm_is_realm(kvm))
>> + kvm_destroy_realm(kvm);
>> }
>>
>> static bool kvm_has_full_ptr_auth(void)
>> @@ -473,6 +482,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
>> else
>> r = kvm_supports_cacheable_pfnmap();
>> break;
>> + case KVM_CAP_ARM_RMI:
>> + r = static_key_enabled(&kvm_rmi_is_available);
>> + break;
>>
>> default:
>> r = 0;
>> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
>> index 070a01e53fcb..d6094b60c4ce 100644
>> --- a/arch/arm64/kvm/mmu.c
>> +++ b/arch/arm64/kvm/mmu.c
>> @@ -872,12 +872,16 @@ static struct kvm_pgtable_mm_ops kvm_s2_mm_ops = {
>> .icache_inval_pou = invalidate_icache_guest_page,
>> };
>>
>> -static int kvm_init_ipa_range(struct kvm_s2_mmu *mmu, unsigned long type)
>> +static int kvm_init_ipa_range(struct kvm *kvm,
>> + struct kvm_s2_mmu *mmu, unsigned long type)
>> {
>> u32 kvm_ipa_limit = get_kvm_ipa_limit();
>> u64 mmfr0, mmfr1;
>> u32 phys_shift;
>>
>> + if (kvm_is_realm(kvm))
>> + kvm_ipa_limit = kvm_realm_ipa_limit();
>> +
>
> Hi,
>
> I believe we can do:
>
> struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
>
> to avoid introducing the extra argument.
> But in order for that to work we'll have to initialize mmu->arch
> earlier:
>
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 8c5d259810b2..e98da7bde9a0 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -974,6 +974,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
> return -EINVAL;
> }
>
> + mmu->arch = &kvm->arch;
> err = kvm_init_ipa_range(mmu, type);
> if (err)
> return err;
> @@ -982,7 +983,6 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
> if (!pgt)
> return -ENOMEM;
>
> - mmu->arch = &kvm->arch;
> err = KVM_PGT_FN(kvm_pgtable_stage2_init)(pgt, mmu, &kvm_s2_mm_ops);
> if (err)
> goto out_free_pgtable;
Nice - I hadn't noticed that I could just pull the mmu->arch assignment
up and avoid that extra argument.
Thanks,
Steve
>
> Thanks,
> Wei-Lin Chang
>
>> if (type & ~KVM_VM_TYPE_ARM_IPA_SIZE_MASK)
>> return -EINVAL;
>>
>> @@ -974,7 +978,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
>> return -EINVAL;
>> }
>>
>> - err = kvm_init_ipa_range(mmu, type);
>> + err = kvm_init_ipa_range(kvm, mmu, type);
>> if (err)
>> return err;
>>
>> @@ -1113,7 +1117,8 @@ void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
>> write_unlock(&kvm->mmu_lock);
>>
>> if (pgt) {
>> - kvm_stage2_destroy(pgt);
>> + if (!kvm_is_realm(kvm))
>> + kvm_stage2_destroy(pgt);
>> kfree(pgt);
>> }
>> }
>> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
>> index 80aedc85e94a..700b8c935d29 100644
>> --- a/arch/arm64/kvm/rmi.c
>> +++ b/arch/arm64/kvm/rmi.c
>> @@ -6,6 +6,8 @@
>> #include <linux/kvm_host.h>
>> #include <linux/memblock.h>
>>
>> +#include <asm/kvm_emulate.h>
>> +#include <asm/kvm_mmu.h>
>> #include <asm/kvm_pgtable.h>
>> #include <asm/rmi_cmds.h>
>> #include <asm/virt.h>
>> @@ -182,6 +184,92 @@ static int rmi_init_metadata(void)
>> return 0;
>> }
>>
>> +u32 kvm_realm_ipa_limit(void)
>> +{
>> + return u64_get_bits(rmm_feat_reg0, RMI_FEATURE_REGISTER_0_S2SZ);
>> +}
>> +
>> +static int undelegate_range(phys_addr_t phys, unsigned long size)
>> +{
>> + unsigned long ret;
>> + unsigned long top = phys + size;
>> + unsigned long out_top;
>> +
>> + while (phys < top) {
>> + ret = rmi_granule_range_undelegate(phys, top, &out_top);
>> + if (ret == RMI_SUCCESS)
>> + phys = out_top;
>> + else if (ret != RMI_BUSY && ret != RMI_BLOCKED)
>> + return ret;
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +static int undelegate_page(phys_addr_t phys)
>> +{
>> + return undelegate_range(phys, PAGE_SIZE);
>> +}
>> +
>> +static int free_delegated_page(phys_addr_t phys)
>> +{
>> + if (WARN_ON(undelegate_page(phys))) {
>> + /* Undelegate failed: leak the page */
>> + return -EBUSY;
>> + }
>> +
>> + free_page((unsigned long)phys_to_virt(phys));
>> +
>> + return 0;
>> +}
>> +
>> +void kvm_destroy_realm(struct kvm *kvm)
>> +{
>> + struct realm *realm = &kvm->arch.realm;
>> + size_t pgd_size = kvm_pgtable_stage2_pgd_size(kvm->arch.mmu.vtcr);
>> +
>> + write_lock(&kvm->mmu_lock);
>> + kvm_stage2_unmap_range(&kvm->arch.mmu, 0,
>> + BIT(realm->ia_bits - 1), true);
>> + write_unlock(&kvm->mmu_lock);
>> +
>> + if (realm->params) {
>> + free_page((unsigned long)realm->params);
>> + realm->params = NULL;
>> + }
>> +
>> + if (!kvm_realm_is_created(kvm))
>> + return;
>> +
>> + WRITE_ONCE(realm->state, REALM_STATE_DYING);
>> +
>> + if (realm->rd) {
>> + phys_addr_t rd_phys = virt_to_phys(realm->rd);
>> +
>> + if (WARN_ON(rmi_realm_destroy(rd_phys)))
>> + return;
>> + free_delegated_page(rd_phys);
>> + realm->rd = NULL;
>> + }
>> +
>> + if (WARN_ON(undelegate_range(kvm->arch.mmu.pgd_phys, pgd_size)))
>> + return;
>> +
>> + WRITE_ONCE(realm->state, REALM_STATE_DEAD);
>> +
>> + /* Now that the Realm is destroyed, free the entry level RTTs */
>> + kvm_free_stage2_pgd(&kvm->arch.mmu);
>> +}
>> +
>> +int kvm_init_realm_vm(struct kvm *kvm)
>> +{
>> + kvm->arch.realm.params = (void *)get_zeroed_page(GFP_KERNEL);
>> +
>> + if (!kvm->arch.realm.params)
>> + return -ENOMEM;
>> + return 0;
>> +}
>> +
>> static int rmm_check_features(void)
>> {
>> if (kvm_lpa2_is_enabled() && !rmi_has_feature(RMI_FEATURE_REGISTER_0_LPA2)) {
>> --
>> 2.43.0
>>
^ permalink raw reply
* Re: [PATCH v13 07/48] arm64: RMI: Check for RMI support at KVM init
From: Wei-Lin Chang @ 2026-03-19 16:17 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <20260318155413.793430-8-steven.price@arm.com>
On Wed, Mar 18, 2026 at 03:53:31PM +0000, Steven Price wrote:
> Query the RMI version number and check if it is a compatible version. A
> static key is also provided to signal that a supported RMM is available.
>
> Functions are provided to query if a VM or VCPU is a realm (or rec)
> which currently will always return false.
>
> Later patches make use of struct realm and the states as the ioctls
> interfaces are added to support realm and REC creation and destruction.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v12:
> * Drop check for 4k page size.
> Changes since v11:
> * Reword slightly the comments on the realm states.
> Changes since v10:
> * kvm_is_realm() no longer has a NULL check.
> * Rename from "rme" to "rmi" when referring to the RMM interface.
> * Check for RME (hardware) support before probing for RMI support.
> Changes since v8:
> * No need to guard kvm_init_rme() behind 'in_hyp_mode'.
> Changes since v6:
> * Improved message for an unsupported RMI ABI version.
> Changes since v5:
> * Reword "unsupported" message from "host supports" to "we want" to
> clarify that 'we' are the 'host'.
> Changes since v2:
> * Drop return value from kvm_init_rme(), it was always 0.
> * Rely on the RMM return value to identify whether the RSI ABI is
> compatible.
> ---
> arch/arm64/include/asm/kvm_emulate.h | 18 +++++++++
> arch/arm64/include/asm/kvm_host.h | 4 ++
> arch/arm64/include/asm/kvm_rmi.h | 56 +++++++++++++++++++++++++++
> arch/arm64/include/asm/virt.h | 1 +
> arch/arm64/kernel/cpufeature.c | 1 +
> arch/arm64/kvm/Makefile | 2 +-
> arch/arm64/kvm/arm.c | 5 +++
> arch/arm64/kvm/rmi.c | 57 ++++++++++++++++++++++++++++
> 8 files changed, 143 insertions(+), 1 deletion(-)
> create mode 100644 arch/arm64/include/asm/kvm_rmi.h
> create mode 100644 arch/arm64/kvm/rmi.c
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 5bf3d7e1d92c..f38b50151ce8 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -688,4 +688,22 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
> vcpu->arch.hcrx_el2 |= HCRX_EL2_EnASR;
> }
> }
> +
> +static inline bool kvm_is_realm(struct kvm *kvm)
> +{
> + if (static_branch_unlikely(&kvm_rmi_is_available))
> + return kvm->arch.is_realm;
> + return false;
> +}
> +
> +static inline enum realm_state kvm_realm_state(struct kvm *kvm)
> +{
> + return READ_ONCE(kvm->arch.realm.state);
> +}
> +
> +static inline bool vcpu_is_rec(struct kvm_vcpu *vcpu)
> +{
> + return false;
> +}
> +
Hi,
Both kvm_vm_is_protected() and vcpu_is_protected() are in kvm_host.h, do
you think that's a better place? Or is there a reason for this being in
kvm_emulate.h?
Thanks,
Wei-Lin Chang
> #endif /* __ARM64_KVM_EMULATE_H__ */
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 5d5a3bbdb95e..9267a2f2d65b 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -27,6 +27,7 @@
> #include <asm/fpsimd.h>
> #include <asm/kvm.h>
> #include <asm/kvm_asm.h>
> +#include <asm/kvm_rmi.h>
> #include <asm/vncr_mapping.h>
>
> #define __KVM_HAVE_ARCH_INTC_INITIALIZED
> @@ -405,6 +406,9 @@ struct kvm_arch {
> * the associated pKVM instance in the hypervisor.
> */
> struct kvm_protected_vm pkvm;
> +
> + bool is_realm;
> + struct realm realm;
> };
>
> struct kvm_vcpu_fault_info {
> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
> new file mode 100644
> index 000000000000..3506f50b05cd
> --- /dev/null
> +++ b/arch/arm64/include/asm/kvm_rmi.h
> @@ -0,0 +1,56 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2023-2025 ARM Ltd.
> + */
> +
> +#ifndef __ASM_KVM_RMI_H
> +#define __ASM_KVM_RMI_H
> +
> +/**
> + * enum realm_state - State of a Realm
> + */
> +enum realm_state {
> + /**
> + * @REALM_STATE_NONE:
> + * Realm has not yet been created. rmi_realm_create() has not
> + * yet been called.
> + */
> + REALM_STATE_NONE,
> + /**
> + * @REALM_STATE_NEW:
> + * Realm is under construction, rmi_realm_create() has been
> + * called, but it is not yet activated. Pages may be populated.
> + */
> + REALM_STATE_NEW,
> + /**
> + * @REALM_STATE_ACTIVE:
> + * Realm has been created and is eligible for execution with
> + * rmi_rec_enter(). Pages may no longer be populated with
> + * rmi_data_create().
> + */
> + REALM_STATE_ACTIVE,
> + /**
> + * @REALM_STATE_DYING:
> + * Realm is in the process of being destroyed or has already been
> + * destroyed.
> + */
> + REALM_STATE_DYING,
> + /**
> + * @REALM_STATE_DEAD:
> + * Realm has been destroyed.
> + */
> + REALM_STATE_DEAD
> +};
> +
> +/**
> + * struct realm - Additional per VM data for a Realm
> + *
> + * @state: The lifetime state machine for the realm
> + */
> +struct realm {
> + enum realm_state state;
> +};
> +
> +void kvm_init_rmi(void);
> +
> +#endif /* __ASM_KVM_RMI_H */
> diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
> index b51ab6840f9c..dc9b2899e0b2 100644
> --- a/arch/arm64/include/asm/virt.h
> +++ b/arch/arm64/include/asm/virt.h
> @@ -87,6 +87,7 @@ void __hyp_reset_vectors(void);
> bool is_kvm_arm_initialised(void);
>
> DECLARE_STATIC_KEY_FALSE(kvm_protected_mode_initialized);
> +DECLARE_STATIC_KEY_FALSE(kvm_rmi_is_available);
>
> static inline bool is_pkvm_initialized(void)
> {
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index c31f8e17732a..ddf7e57f23e8 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -289,6 +289,7 @@ static const struct arm64_ftr_bits ftr_id_aa64isar3[] = {
> static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_CSV3_SHIFT, 4, 0),
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_CSV2_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_RME_SHIFT, 4, 0),
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_DIT_SHIFT, 4, 0),
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_AMU_SHIFT, 4, 0),
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_MPAM_SHIFT, 4, 0),
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index 3ebc0570345c..e17c4077d8e7 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -16,7 +16,7 @@ CFLAGS_handle_exit.o += -Wno-override-init
> kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \
> inject_fault.o va_layout.o handle_exit.o config.o \
> guest.o debug.o reset.o sys_regs.o stacktrace.o \
> - vgic-sys-reg-v3.o fpsimd.o pkvm.o \
> + vgic-sys-reg-v3.o fpsimd.o pkvm.o rmi.o \
> arch_timer.o trng.o vmid.o emulate-nested.o nested.o at.o \
> vgic/vgic.o vgic/vgic-init.o \
> vgic/vgic-irqfd.o vgic/vgic-v2.o \
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 29f0326f7e00..274d7866efdc 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -39,6 +39,7 @@
> #include <asm/kvm_nested.h>
> #include <asm/kvm_pkvm.h>
> #include <asm/kvm_ptrauth.h>
> +#include <asm/kvm_rmi.h>
> #include <asm/sections.h>
> #include <asm/stacktrace/nvhe.h>
>
> @@ -104,6 +105,8 @@ long kvm_get_cap_for_kvm_ioctl(unsigned int ioctl, long *ext)
> return -EINVAL;
> }
>
> +DEFINE_STATIC_KEY_FALSE(kvm_rmi_is_available);
> +
> DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector);
>
> DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_base);
> @@ -2921,6 +2924,8 @@ static __init int kvm_arm_init(void)
>
> in_hyp_mode = is_kernel_in_hyp_mode();
>
> + kvm_init_rmi();
> +
> if (cpus_have_final_cap(ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE) ||
> cpus_have_final_cap(ARM64_WORKAROUND_1508412))
> kvm_info("Guests without required CPU erratum workarounds can deadlock system!\n" \
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> new file mode 100644
> index 000000000000..fac151580c01
> --- /dev/null
> +++ b/arch/arm64/kvm/rmi.c
> @@ -0,0 +1,57 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2023-2025 ARM Ltd.
> + */
> +
> +#include <linux/kvm_host.h>
> +
> +#include <asm/rmi_cmds.h>
> +#include <asm/virt.h>
> +
> +static int rmi_check_version(void)
> +{
> + struct arm_smccc_res res;
> + unsigned short version_major, version_minor;
> + unsigned long host_version = RMI_ABI_VERSION(RMI_ABI_MAJOR_VERSION,
> + RMI_ABI_MINOR_VERSION);
> + unsigned long aa64pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
> +
> + /* If RME isn't supported, then RMI can't be */
> + if (cpuid_feature_extract_unsigned_field(aa64pfr0, ID_AA64PFR0_EL1_RME_SHIFT) == 0)
> + return -ENXIO;
> +
> + arm_smccc_1_1_invoke(SMC_RMI_VERSION, host_version, &res);
> +
> + if (res.a0 == SMCCC_RET_NOT_SUPPORTED)
> + return -ENXIO;
> +
> + version_major = RMI_ABI_VERSION_GET_MAJOR(res.a1);
> + version_minor = RMI_ABI_VERSION_GET_MINOR(res.a1);
> +
> + if (res.a0 != RMI_SUCCESS) {
> + unsigned short high_version_major, high_version_minor;
> +
> + high_version_major = RMI_ABI_VERSION_GET_MAJOR(res.a2);
> + high_version_minor = RMI_ABI_VERSION_GET_MINOR(res.a2);
> +
> + kvm_err("Unsupported RMI ABI (v%d.%d - v%d.%d) we want v%d.%d\n",
> + version_major, version_minor,
> + high_version_major, high_version_minor,
> + RMI_ABI_MAJOR_VERSION,
> + RMI_ABI_MINOR_VERSION);
> + return -ENXIO;
> + }
> +
> + kvm_info("RMI ABI version %d.%d\n", version_major, version_minor);
> +
> + return 0;
> +}
> +
> +void kvm_init_rmi(void)
> +{
> + /* Continue without realm support if we can't agree on a version */
> + if (rmi_check_version())
> + return;
> +
> + /* Future patch will enable static branch kvm_rmi_is_available */
> +}
> --
> 2.43.0
>
^ permalink raw reply
* Re: [PATCH v13 12/48] arm64: RMI: Basic infrastructure for creating a realm.
From: Wei-Lin Chang @ 2026-03-19 16:11 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <20260318155413.793430-13-steven.price@arm.com>
On Wed, Mar 18, 2026 at 03:53:36PM +0000, Steven Price wrote:
> Introduce the skeleton functions for creating and destroying a realm.
> The IPA size requested is checked against what the RMM supports.
>
> The actual work of constructing the realm will be added in future
> patches.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v12:
> * Drop the RMM_PAGE_{SHIFT,SIZE} defines - the RMM is now configured to
> be the same as the host's page size.
> * Rework delegate/undelegate functions to use the new RMI range based
> operations.
> Changes since v11:
> * Major rework to drop the realm configuration and make the
> construction of realms implicit rather than driven by the VMM
> directly.
> * The code to create RDs, handle VMIDs etc is moved to later patches.
> Changes since v10:
> * Rename from RME to RMI.
> * Move the stage2 cleanup to a later patch.
> Changes since v9:
> * Avoid walking the stage 2 page tables when destroying the realm -
> the real ones are not accessible to the non-secure world, and the RMM
> may leave junk in the physical pages when returning them.
> * Fix an error path in realm_create_rd() to actually return an error value.
> Changes since v8:
> * Fix free_delegated_granule() to not call kvm_account_pgtable_pages();
> a separate wrapper will be introduced in a later patch to deal with
> RTTs.
> * Minor code cleanups following review.
> Changes since v7:
> * Minor code cleanup following Gavin's review.
> Changes since v6:
> * Separate RMM RTT calculations from host PAGE_SIZE. This allows the
> host page size to be larger than 4k while still communicating with an
> RMM which uses 4k granules.
> Changes since v5:
> * Introduce free_delegated_granule() to replace many
> undelegate/free_page() instances and centralise the comment on
> leaking when the undelegate fails.
> * Several other minor improvements suggested by reviews - thanks for
> the feedback!
> Changes since v2:
> * Improved commit description.
> * Improved return failures for rmi_check_version().
> * Clear contents of PGD after it has been undelegated in case the RMM
> left stale data.
> * Minor changes to reflect changes in previous patches.
> ---
> arch/arm64/include/asm/kvm_emulate.h | 5 ++
> arch/arm64/include/asm/kvm_rmi.h | 16 +++++
> arch/arm64/kvm/arm.c | 12 ++++
> arch/arm64/kvm/mmu.c | 11 +++-
> arch/arm64/kvm/rmi.c | 88 ++++++++++++++++++++++++++++
> 5 files changed, 129 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index f38b50151ce8..39310d9b4e16 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -701,6 +701,11 @@ static inline enum realm_state kvm_realm_state(struct kvm *kvm)
> return READ_ONCE(kvm->arch.realm.state);
> }
>
> +static inline bool kvm_realm_is_created(struct kvm *kvm)
> +{
> + return kvm_is_realm(kvm) && kvm_realm_state(kvm) != REALM_STATE_NONE;
> +}
> +
> static inline bool vcpu_is_rec(struct kvm_vcpu *vcpu)
> {
> return false;
> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
> index 3506f50b05cd..0ada525af18f 100644
> --- a/arch/arm64/include/asm/kvm_rmi.h
> +++ b/arch/arm64/include/asm/kvm_rmi.h
> @@ -6,6 +6,8 @@
> #ifndef __ASM_KVM_RMI_H
> #define __ASM_KVM_RMI_H
>
> +#include <asm/rmi_smc.h>
> +
> /**
> * enum realm_state - State of a Realm
> */
> @@ -46,11 +48,25 @@ enum realm_state {
> * struct realm - Additional per VM data for a Realm
> *
> * @state: The lifetime state machine for the realm
> + * @rd: Kernel mapping of the Realm Descriptor (RD)
> + * @params: Parameters for the RMI_REALM_CREATE command
> + * @num_aux: The number of auxiliary pages required by the RMM
> + * @ia_bits: Number of valid Input Address bits in the IPA
> */
> struct realm {
> enum realm_state state;
> +
> + void *rd;
> + struct realm_params *params;
> +
> + unsigned long num_aux;
> + unsigned int ia_bits;
> };
>
> void kvm_init_rmi(void);
> +u32 kvm_realm_ipa_limit(void);
> +
> +int kvm_init_realm_vm(struct kvm *kvm);
> +void kvm_destroy_realm(struct kvm *kvm);
>
> #endif /* __ASM_KVM_RMI_H */
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 274d7866efdc..9b17bdfaf0c2 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -253,6 +253,13 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
>
> bitmap_zero(kvm->arch.vcpu_features, KVM_VCPU_MAX_FEATURES);
>
> + /* Initialise the realm bits after the generic bits are enabled */
> + if (kvm_is_realm(kvm)) {
> + ret = kvm_init_realm_vm(kvm);
> + if (ret)
> + goto err_free_cpumask;
> + }
> +
> return 0;
>
> err_free_cpumask:
> @@ -312,6 +319,8 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
> kvm_unshare_hyp(kvm, kvm + 1);
>
> kvm_arm_teardown_hypercalls(kvm);
> + if (kvm_is_realm(kvm))
> + kvm_destroy_realm(kvm);
> }
>
> static bool kvm_has_full_ptr_auth(void)
> @@ -473,6 +482,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
> else
> r = kvm_supports_cacheable_pfnmap();
> break;
> + case KVM_CAP_ARM_RMI:
> + r = static_key_enabled(&kvm_rmi_is_available);
> + break;
>
> default:
> r = 0;
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 070a01e53fcb..d6094b60c4ce 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -872,12 +872,16 @@ static struct kvm_pgtable_mm_ops kvm_s2_mm_ops = {
> .icache_inval_pou = invalidate_icache_guest_page,
> };
>
> -static int kvm_init_ipa_range(struct kvm_s2_mmu *mmu, unsigned long type)
> +static int kvm_init_ipa_range(struct kvm *kvm,
> + struct kvm_s2_mmu *mmu, unsigned long type)
> {
> u32 kvm_ipa_limit = get_kvm_ipa_limit();
> u64 mmfr0, mmfr1;
> u32 phys_shift;
>
> + if (kvm_is_realm(kvm))
> + kvm_ipa_limit = kvm_realm_ipa_limit();
> +
Hi,
I believe we can do:
struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
to avoid introducing the extra argument.
But in order for that to work we'll have to initialize mmu->arch
earlier:
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 8c5d259810b2..e98da7bde9a0 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -974,6 +974,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
return -EINVAL;
}
+ mmu->arch = &kvm->arch;
err = kvm_init_ipa_range(mmu, type);
if (err)
return err;
@@ -982,7 +983,6 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
if (!pgt)
return -ENOMEM;
- mmu->arch = &kvm->arch;
err = KVM_PGT_FN(kvm_pgtable_stage2_init)(pgt, mmu, &kvm_s2_mm_ops);
if (err)
goto out_free_pgtable;
Thanks,
Wei-Lin Chang
> if (type & ~KVM_VM_TYPE_ARM_IPA_SIZE_MASK)
> return -EINVAL;
>
> @@ -974,7 +978,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
> return -EINVAL;
> }
>
> - err = kvm_init_ipa_range(mmu, type);
> + err = kvm_init_ipa_range(kvm, mmu, type);
> if (err)
> return err;
>
> @@ -1113,7 +1117,8 @@ void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
> write_unlock(&kvm->mmu_lock);
>
> if (pgt) {
> - kvm_stage2_destroy(pgt);
> + if (!kvm_is_realm(kvm))
> + kvm_stage2_destroy(pgt);
> kfree(pgt);
> }
> }
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> index 80aedc85e94a..700b8c935d29 100644
> --- a/arch/arm64/kvm/rmi.c
> +++ b/arch/arm64/kvm/rmi.c
> @@ -6,6 +6,8 @@
> #include <linux/kvm_host.h>
> #include <linux/memblock.h>
>
> +#include <asm/kvm_emulate.h>
> +#include <asm/kvm_mmu.h>
> #include <asm/kvm_pgtable.h>
> #include <asm/rmi_cmds.h>
> #include <asm/virt.h>
> @@ -182,6 +184,92 @@ static int rmi_init_metadata(void)
> return 0;
> }
>
> +u32 kvm_realm_ipa_limit(void)
> +{
> + return u64_get_bits(rmm_feat_reg0, RMI_FEATURE_REGISTER_0_S2SZ);
> +}
> +
> +static int undelegate_range(phys_addr_t phys, unsigned long size)
> +{
> + unsigned long ret;
> + unsigned long top = phys + size;
> + unsigned long out_top;
> +
> + while (phys < top) {
> + ret = rmi_granule_range_undelegate(phys, top, &out_top);
> + if (ret == RMI_SUCCESS)
> + phys = out_top;
> + else if (ret != RMI_BUSY && ret != RMI_BLOCKED)
> + return ret;
> + }
> +
> + return ret;
> +}
> +
> +static int undelegate_page(phys_addr_t phys)
> +{
> + return undelegate_range(phys, PAGE_SIZE);
> +}
> +
> +static int free_delegated_page(phys_addr_t phys)
> +{
> + if (WARN_ON(undelegate_page(phys))) {
> + /* Undelegate failed: leak the page */
> + return -EBUSY;
> + }
> +
> + free_page((unsigned long)phys_to_virt(phys));
> +
> + return 0;
> +}
> +
> +void kvm_destroy_realm(struct kvm *kvm)
> +{
> + struct realm *realm = &kvm->arch.realm;
> + size_t pgd_size = kvm_pgtable_stage2_pgd_size(kvm->arch.mmu.vtcr);
> +
> + write_lock(&kvm->mmu_lock);
> + kvm_stage2_unmap_range(&kvm->arch.mmu, 0,
> + BIT(realm->ia_bits - 1), true);
> + write_unlock(&kvm->mmu_lock);
> +
> + if (realm->params) {
> + free_page((unsigned long)realm->params);
> + realm->params = NULL;
> + }
> +
> + if (!kvm_realm_is_created(kvm))
> + return;
> +
> + WRITE_ONCE(realm->state, REALM_STATE_DYING);
> +
> + if (realm->rd) {
> + phys_addr_t rd_phys = virt_to_phys(realm->rd);
> +
> + if (WARN_ON(rmi_realm_destroy(rd_phys)))
> + return;
> + free_delegated_page(rd_phys);
> + realm->rd = NULL;
> + }
> +
> + if (WARN_ON(undelegate_range(kvm->arch.mmu.pgd_phys, pgd_size)))
> + return;
> +
> + WRITE_ONCE(realm->state, REALM_STATE_DEAD);
> +
> + /* Now that the Realm is destroyed, free the entry level RTTs */
> + kvm_free_stage2_pgd(&kvm->arch.mmu);
> +}
> +
> +int kvm_init_realm_vm(struct kvm *kvm)
> +{
> + kvm->arch.realm.params = (void *)get_zeroed_page(GFP_KERNEL);
> +
> + if (!kvm->arch.realm.params)
> + return -ENOMEM;
> + return 0;
> +}
> +
> static int rmm_check_features(void)
> {
> if (kvm_lpa2_is_enabled() && !rmi_has_feature(RMI_FEATURE_REGISTER_0_LPA2)) {
> --
> 2.43.0
>
^ permalink raw reply related
* Re: [PATCH v13 36/48] arm64: RMI: Always use 4k pages for realms
From: Steven Price @ 2026-03-19 16:02 UTC (permalink / raw)
To: Joey Gouly
Cc: kvm, kvmarm, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20260319102454.GB3942350@e124191.cambridge.arm.com>
On 19/03/2026 10:24, Joey Gouly wrote:
> Hi,
>
> On Wed, Mar 18, 2026 at 03:54:00PM +0000, Steven Price wrote:
>> Guest_memfd doesn't yet natively support huge pages, and there are
>> currently difficulties for a VMM to manage huge pages efficiently so for
>> now always split up mappings to PTE (4k).
>>
>> The two issues that need progressing before supporting huge pages for
>> realms are:
>>
>> 1. guest_memfd needs to be able to allocate from an appropriate
>> allocator which can provide huge pages.
>>
>> 2. The VMM needs to be able to repurpose private memory for a shared
>> mapping when the guest VM requests memory is transitioned. Because
>> this can happen at a 4k granularity it isn't possible to
>> free/reallocate while huge pages are in use. Allowing the VMM to
>> mmap() the shared portion of a huge page would allow the huge page
>> to be recreated when the memory is unshared and made protected again.
>>
>> These two issues are not specific to realms and don't affect the realm
>> API, so for now just break everything down to 4k pages in the RMM
>> controlled stage 2. Future work can add huge page support without
>> changing the uAPI.
>
> The commit title/message mention 4K, but should probably say PAGE_SIZE or
> something now that RMM isn't fixed to 4K.
Indeed - this is all PAGE_SIZE not 4k any more. Also hopefully the
reasons for this patch are also going to disappear soon. (2) above isn't
really very true any more (we do support mmap() from guest_memfd).
Thanks,
Steve
> Thanks,
> Joey
>
>>
>> Signed-off-by: Steven Price <steven.price@arm.com>
>> Reviewed-by: Gavin Shan <gshan@redhat.com>
>> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>> ---
>> Changes since v7:
>> * Rewritten commit message
>> ---
>> arch/arm64/kvm/mmu.c | 7 +++++--
>> 1 file changed, 5 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
>> index 73c18c2861a2..ad1300f366df 100644
>> --- a/arch/arm64/kvm/mmu.c
>> +++ b/arch/arm64/kvm/mmu.c
>> @@ -1761,11 +1761,14 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
>> write_fault = kvm_is_write_fault(vcpu);
>>
>> /*
>> - * Realms cannot map protected pages read-only
>> + * Realms cannot map protected pages read-only, also force PTE mappings
>> + * for Realms.
>> * FIXME: It should be possible to map unprotected pages read-only
>> */
>> - if (vcpu_is_rec(vcpu))
>> + if (vcpu_is_rec(vcpu)) {
>> write_fault = true;
>> + force_pte = true;
>> + }
>>
>> exec_fault = kvm_vcpu_trap_is_exec_fault(vcpu);
>> VM_WARN_ON_ONCE(write_fault && exec_fault);
>> --
>> 2.43.0
>>
>>
^ permalink raw reply
* Re: [PATCH v2 12/19] x86, ioremap, resource: Support IORES_DESC_ENCRYPTED for encrypted PCI MMIO
From: Borislav Petkov @ 2026-03-19 15:34 UTC (permalink / raw)
To: Dan Williams
Cc: linux-coco, linux-pci, gregkh, aik, aneesh.kumar, yilun.xu,
bhelgaas, alistair23, lukas, jgg, Dave Hansen, Andy Lutomirski,
Peter Zijlstra, Thomas Gleixner, Ingo Molnar, x86, H. Peter Anvin
In-Reply-To: <20260303000207.1836586-13-dan.j.williams@intel.com>
On Mon, Mar 02, 2026 at 04:02:00PM -0800, Dan Williams wrote:
> PCIe Trusted Execution Environment Device Interface Security Protocol
> (TDISP) arranges for a PCI device to support encrypted MMIO. In support of
> that capability, ioremap() needs a mechanism to detect when a PCI device
> has been dynamically transitioned into this secure state and enforce
> encrypted MMIO mappings.
>
> Teach ioremap() about a new IORES_DESC_ENCRYPTED type that supplements the
> existing PCI Memory Space (MMIO) BAR resources. The proposal is that a
> resource, "PCI MMIO Encrypted", with this description type is injected by
> the PCI/TSM core for each PCI device BAR that is to be protected.
>
> Unlike the existing encryption determination which is "implied with a
> silent fallback to an unencrypted mapping", this indication is "explicit
> with an expectation that the request fails instead of fallback".
> IORES_MUST_ENCRYPT is added to manage this expectation.
"Add IORES_MUST_ENCRYPT to manage... "
Pls read section "2) Describe your changes" in
Documentation/process/submitting-patches.rst for more details. Especially the
part about ordering the code to do stuff.
> Given that "PCI MMIO Encrypted" is an additional resource in the tree, the
> IORESOURCE_BUSY flag will only be set on a descendant/child of that
> resource. That means it cannot share the same walk as the check for "System
> RAM". Add walk_iomem_res_desc() to check if any IORES_DESC_ENCRYPTED
> intersects the ioremap() range and set IORES_MUST_ENCRYPT accordingly. When
> IORES_MUST_ENCRYPT is set, the entire ioremap() range must be covered by
> IORES_DESC_ENCRYPTED.
>
> Cc: Dave Hansen <dave.hansen@linux.intel.com>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Borislav Petkov <bp@alien8.de>
> Cc: x86@kernel.org
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
> include/linux/ioport.h | 1 +
> arch/x86/mm/ioremap.c | 49 +++++++++++++++++++++++++++++++-----------
> 2 files changed, 37 insertions(+), 13 deletions(-)
>
> diff --git a/include/linux/ioport.h b/include/linux/ioport.h
> index 1c106608c514..3efd07443c47 100644
> --- a/include/linux/ioport.h
> +++ b/include/linux/ioport.h
> @@ -152,6 +152,7 @@ enum {
> enum {
> IORES_MAP_SYSTEM_RAM = BIT(0),
> IORES_MAP_ENCRYPTED = BIT(1),
> + IORES_MUST_ENCRYPT = BIT(2), /* disable transparent fallback */
> };
>
> /* helpers to define resources */
> diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
> index 12c8180ca1ba..0f300e226a9f 100644
> --- a/arch/x86/mm/ioremap.c
> +++ b/arch/x86/mm/ioremap.c
> @@ -36,6 +36,7 @@
> */
> struct ioremap_desc {
> unsigned int flags;
> + u64 encrypt_size;
> };
>
> /*
> @@ -88,23 +89,35 @@ static unsigned int __ioremap_check_ram(struct resource *res)
> }
>
> /*
> - * In a SEV guest, NONE and RESERVED should not be mapped encrypted because
> - * there the whole memory is already encrypted.
> + * In a encrypted guest, NONE and RESERVED should not be mapped encrypted
"In an encrypted... "
> + * because there the whole memory is already encrypted.
> + *
> + * For the encrypted case the entire range must agree with being mapped
> + * encrypted.
> */
> -static unsigned int __ioremap_check_encrypted(struct resource *res)
> +static unsigned int __ioremap_check_encrypted(struct ioremap_desc *desc,
> + struct resource *res)
> {
> + u32 flags = 0;
> +
> + if (res->desc == IORES_DESC_ENCRYPTED)
> + flags |= IORES_MUST_ENCRYPT;
> +
> if (!cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
> - return 0;
> + return flags;
>
> switch (res->desc) {
> case IORES_DESC_NONE:
> case IORES_DESC_RESERVED:
> break;
> + case IORES_DESC_ENCRYPTED:
> + desc->encrypt_size += resource_size(res);
> + fallthrough;
> default:
> - return IORES_MAP_ENCRYPTED;
> + flags |= IORES_MAP_ENCRYPTED;
> }
>
> - return 0;
> + return flags;
> }
This looks too convoluted. I'm sure we can state the rules for encrypted
ranges first and then can simplify it into something more understandable.
>
> /*
> @@ -134,14 +147,10 @@ static int __ioremap_collect_map_flags(struct resource *res, void *arg)
> {
> struct ioremap_desc *desc = arg;
>
> - if (!(desc->flags & IORES_MAP_SYSTEM_RAM))
> - desc->flags |= __ioremap_check_ram(res);
> -
> - if (!(desc->flags & IORES_MAP_ENCRYPTED))
> - desc->flags |= __ioremap_check_encrypted(res);
> + desc->flags |= __ioremap_check_ram(res);
> + desc->flags |= __ioremap_check_encrypted(desc, res);
>
> - return ((desc->flags & (IORES_MAP_SYSTEM_RAM | IORES_MAP_ENCRYPTED)) ==
> - (IORES_MAP_SYSTEM_RAM | IORES_MAP_ENCRYPTED));
> + return 0;
> }
This change I don't grok. Perhaps it needs to be a separate patch.
> /*
> @@ -162,6 +171,13 @@ static void __ioremap_check_mem(resource_size_t addr, unsigned long size,
> memset(desc, 0, sizeof(struct ioremap_desc));
>
> walk_mem_res(start, end, desc, __ioremap_collect_map_flags);
<---- newline here.
> + /*
> + * Encrypted MMIO may parent a driver's requested region, so it needs a
> + * separate search
> + */
> + desc->encrypt_size = 0;
> + walk_iomem_res_desc(IORES_DESC_ENCRYPTED, IORESOURCE_MEM, start, end,
> + desc, __ioremap_collect_map_flags);
Why is this here and executed unconditionally regardless of coco guest or not?
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply
* Re: [PATCH v13 13/48] kvm: arm64: Don't expose unsupported capabilities for realm guests
From: Steven Price @ 2026-03-19 15:25 UTC (permalink / raw)
To: Suzuki K Poulose, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Zenghui Yu, linux-arm-kernel, linux-kernel,
Joey Gouly, Alexandru Elisei, Christoffer Dall, Fuad Tabba,
linux-coco, Ganapatrao Kulkarni, Gavin Shan, Shanker Donthineni,
Alper Gun, Aneesh Kumar K . V, Emi Kisanuki, Vishal Annapurve
In-Reply-To: <56fb42ad-eada-40df-8c81-53cb7c0c310e@arm.com>
On 19/03/2026 14:09, Suzuki K Poulose wrote:
> On 18/03/2026 15:53, Steven Price wrote:
>> From: Suzuki K Poulose <suzuki.poulose@arm.com>
>>
>> RMM v1.0 provides no mechanism for the host to perform debug operations
>> on the guest. So limit the extensions that are visible to an allowlist
>> so that only those capabilities we can support are advertised.
>>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>> Signed-off-by: Steven Price <steven.price@arm.com>
>> ---
>> Changes since v10:
>> * Add a kvm_realm_ext_allowed() function which limits which extensions
>> are exposed to an allowlist. This removes the need for special casing
>> various extensions.
>> Changes since v7:
>> * Remove the helper functions and inline the kvm_is_realm() check with
>> a ternary operator.
>> * Rewrite the commit message to explain this patch.
>> ---
>> arch/arm64/kvm/arm.c | 22 ++++++++++++++++++++++
>> 1 file changed, 22 insertions(+)
>>
>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
>> index 9b17bdfaf0c2..ddbf080e4f55 100644
>> --- a/arch/arm64/kvm/arm.c
>> +++ b/arch/arm64/kvm/arm.c
>> @@ -357,6 +357,25 @@ static bool kvm_has_full_ptr_auth(void)
>> (apa + api + apa3) == 1);
>> }
>> +static bool kvm_realm_ext_allowed(long ext)
>> +{
>> + switch (ext) {
>> + case KVM_CAP_IRQCHIP:
>> + case KVM_CAP_ARM_PSCI:
>> + case KVM_CAP_ARM_PSCI_0_2:
>> + case KVM_CAP_NR_VCPUS:
>> + case KVM_CAP_MAX_VCPUS:
>> + case KVM_CAP_MAX_VCPU_ID:
>> + case KVM_CAP_MSI_DEVID:
>> + case KVM_CAP_ARM_VM_IPA_SIZE:
>> + case KVM_CAP_ARM_PTRAUTH_ADDRESS:
>> + case KVM_CAP_ARM_PTRAUTH_GENERIC:
>> + case KVM_CAP_ARM_RMI:
>> + return true;
>> + }
>> + return false;
>> +}
>> +
>> int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
>> {
>> int r;
>> @@ -364,6 +383,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm,
>> long ext)
>> if (is_protected_kvm_enabled() && !kvm_pkvm_ext_allowed(kvm, ext))
>> return 0;
>> + if (kvm && kvm_is_realm(kvm) && !kvm_realm_ext_allowed(ext))
>> + return 0;
>> +
>
> We need a similar check in in kvm_vm_ioctl_enable_cap() to prevent
> enabling the filtered caps ? Otherwise looks good to me.
Indeed - thanks for spotting.
Thanks,
Steve
> Suzuki
>
>> switch (ext) {
>> case KVM_CAP_IRQCHIP:
>> r = vgic_present;
>
^ permalink raw reply
* Re: [PATCH v13 10/48] arm64: RMI: Ensure that the RMM has GPT entries for memory
From: Steven Price @ 2026-03-19 15:20 UTC (permalink / raw)
To: Suzuki K Poulose, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Zenghui Yu, linux-arm-kernel, linux-kernel,
Joey Gouly, Alexandru Elisei, Christoffer Dall, Fuad Tabba,
linux-coco, Ganapatrao Kulkarni, Gavin Shan, Shanker Donthineni,
Alper Gun, Aneesh Kumar K . V, Emi Kisanuki, Vishal Annapurve
In-Reply-To: <b83d601c-c4a0-4e6b-aae0-e49f02e67e30@arm.com>
On 19/03/2026 10:31, Suzuki K Poulose wrote:
> Hi Steven
>
> On 18/03/2026 15:53, Steven Price wrote:
>> The RMM may not be tracking all the memory of the system at boot. Create
>
> Looks good to me. Please find some suggestions below.
>
>
> May be add a bit more context here :
>
> RMM maintains the state of all the granules in the System to make sure
> that the host is abiding by the rules. This state can be maintained at
> different granularity - per PAGE (TRACKING_FINE) or per region (COARSE),
> where the "region size" depends on the underlying "RMI_GRANULE_SIZE".
> The state of the "tracked area" must be the same. This implies, we may
> need to have "FINE" tracking for DRAM, so that we can start delegating
> PAGEs. For now, we only support RMM with statically carved out memory
> for tracking FINE granularity for the tracking regions. We will extend
> the support for modifying the TRACKING region in the future.
>
> Similarly, the firmware may create L0 GPT entries describing the total
> address space (think of this as Block mappings in the page tables). But
> if we change the "PAS" of a granule in the block mapping, we may need
> to create L1 tables to track the PAS at the finer granularity. For now
> we only support a system where the L1 GPTs are created at boot time
> and dynamic GPT support will be added later.
Thanks for the wording - that does indeed make things clearer. SRO
support will effectively enable the "future" items.
>> the necessary tracking state and GPTs within the RMM so that all boot
>> memory can be delegated to the RMM as needed during runtime.
>>
>> Note: support is currently missing for SROs which means that if the RMM
>> needs memory donating this will fail (and render CCA unusable in Linux).
>>
>> Signed-off-by: Steven Price <steven.price@arm.com>
>> ---
>> New patch for v13
>> ---
>> arch/arm64/kvm/rmi.c | 89 ++++++++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 89 insertions(+)
>>
>> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
>> index 9590dff9a2c1..80aedc85e94a 100644
>> --- a/arch/arm64/kvm/rmi.c
>> +++ b/arch/arm64/kvm/rmi.c
>> @@ -4,6 +4,7 @@
>> */
>> #include <linux/kvm_host.h>
>> +#include <linux/memblock.h>
>> #include <asm/kvm_pgtable.h>
>> #include <asm/rmi_cmds.h>
>> @@ -56,6 +57,18 @@ static int rmi_check_version(void)
>> return 0;
>> }
>> +/*
>> + * These are the 'default' sizes when passing 0 as the
>> tracking_region_size.
>> + * TODO: Support other granule sizes
>> + */
>> +#ifdef CONFIG_PAGE_SIZE_4KB
>> +#define RMM_GRANULE_TRACKING_SIZE SZ_1G
>> +#elif defined(CONFIG_PAGE_SIZE_16KB)
>> +#define RMM_GRANULE_TRACKING_SIZE SZ_32M
>> +#elif defined(CONFIG_PAGE_SIZE_64KB)
>> +#define RMM_GRANULE_TRACKING_SIZE SZ_512M
>> +#endif
>> +
>
> Probably this should be made a Kconfig option, like the VA_BITS we have
> today for each page size.
Yes that's probably a good option - note that for 4k page size there is
only the one option in the spec. So this is only relevant for 16K/64K.
Thanks for the other comment suggestions below (and in the other email)
- all good points.
Thanks,
Steve
>> static int rmi_configure(void)
>> {
>> struct rmm_config *config __free(free_page) = NULL;
>> @@ -95,6 +108,80 @@ static int rmi_configure(void)
>> return 0;
>> }
>> +static int rmi_verify_memory_tracking(phys_addr_t start,
>> phys_addr_t end)
>
> Could we add a comment what we are trying to do here ?
>
> /*
> * Make sure the area is tracked by RMM at FINE granularity.
> * We do not support changing the TRACKING yet. This will
> * be added in the future.
> */
>
>
>> +{
>> + start = ALIGN_DOWN(start, RMM_GRANULE_TRACKING_SIZE);
>> + end = ALIGN(end, RMM_GRANULE_TRACKING_SIZE);
>> +
>> + while (start < end) {
>> + unsigned long ret, category, state;
>> +
>> + ret = rmi_granule_tracking_get(start, &category, &state);
>> + if (ret != RMI_SUCCESS ||
>> + state != RMI_TRACKING_FINE ||
>> + category != RMI_MEM_CATEGORY_CONVENTIONAL) {
>> + /* TODO: Set granule tracking in this case */
>> + kvm_err("Granule tracking for region isn't fine/
>> conventional: %llx",
>> + start);
>> + return -ENODEV;
>> + }
>> + start += RMM_GRANULE_TRACKING_SIZE;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static unsigned long rmi_l0gpt_size(void)
>> +{
>> + return 1UL << (30 + FIELD_GET(RMI_FEATURE_REGISTER_1_L0GPTSZ,
>> + rmm_feat_reg1));
>> +}
>> +
>> +static int rmi_create_gpts(phys_addr_t start, phys_addr_t end)
>> +{
>> + unsigned long l0gpt_sz = rmi_l0gpt_size();
>> +
>> + start = ALIGN_DOWN(start, l0gpt_sz);
>> + end = ALIGN(end, l0gpt_sz);
>> +
>> + while (start < end) {
>> + int ret = rmi_gpt_l1_create(start);
>
> How about adding a comment here explaining why we look for RMI_ERROR_GPT ?
>
>
>>
> /*
> * Make sure the L1 GPT tables are created for the region.
> * RMI_ERROR_GPT indicates the L1 table exists.
> */
> +
>> + if (ret && ret != RMI_ERROR_GPT) {
>
>
>> + /*
>> + * FIXME: Handle SRO so that memory can be donated for
>> + * the tables.
>> + */
>> + kvm_err("GPT Level1 table missing for %llx\n", start);
>> + return -ENOMEM;
>> + }
>> + start += l0gpt_sz;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int rmi_init_metadata(void)
>> +{
>> + phys_addr_t start, end;
>> + const struct memblock_region *r;
>> +
>> + for_each_mem_region(r) {
>> + int ret;
>> +
>> + start = memblock_region_memory_base_pfn(r) << PAGE_SHIFT;
>> + end = memblock_region_memory_end_pfn(r) << PAGE_SHIFT;
>> + ret = rmi_verify_memory_tracking(start, end);
>> + if (ret)
>> + return ret;
>> + ret = rmi_create_gpts(start, end);
>> + if (ret)
>> + return ret;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> static int rmm_check_features(void)
>> {
>> if (kvm_lpa2_is_enabled() && !
>> rmi_has_feature(RMI_FEATURE_REGISTER_0_LPA2)) {
>> @@ -120,6 +207,8 @@ void kvm_init_rmi(void)
>> return;
>> if (rmi_configure())
>> return;
>> + if (rmi_init_metadata())
>> + return;
>> /* Future patch will enable static branch kvm_rmi_is_available */
>> }
>
^ permalink raw reply
* Re: [PATCH v13 13/48] kvm: arm64: Don't expose unsupported capabilities for realm guests
From: Suzuki K Poulose @ 2026-03-19 14:09 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Zenghui Yu, linux-arm-kernel, linux-kernel,
Joey Gouly, Alexandru Elisei, Christoffer Dall, Fuad Tabba,
linux-coco, Ganapatrao Kulkarni, Gavin Shan, Shanker Donthineni,
Alper Gun, Aneesh Kumar K . V, Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20260318155413.793430-14-steven.price@arm.com>
On 18/03/2026 15:53, Steven Price wrote:
> From: Suzuki K Poulose <suzuki.poulose@arm.com>
>
> RMM v1.0 provides no mechanism for the host to perform debug operations
> on the guest. So limit the extensions that are visible to an allowlist
> so that only those capabilities we can support are advertised.
>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v10:
> * Add a kvm_realm_ext_allowed() function which limits which extensions
> are exposed to an allowlist. This removes the need for special casing
> various extensions.
> Changes since v7:
> * Remove the helper functions and inline the kvm_is_realm() check with
> a ternary operator.
> * Rewrite the commit message to explain this patch.
> ---
> arch/arm64/kvm/arm.c | 22 ++++++++++++++++++++++
> 1 file changed, 22 insertions(+)
>
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 9b17bdfaf0c2..ddbf080e4f55 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -357,6 +357,25 @@ static bool kvm_has_full_ptr_auth(void)
> (apa + api + apa3) == 1);
> }
>
> +static bool kvm_realm_ext_allowed(long ext)
> +{
> + switch (ext) {
> + case KVM_CAP_IRQCHIP:
> + case KVM_CAP_ARM_PSCI:
> + case KVM_CAP_ARM_PSCI_0_2:
> + case KVM_CAP_NR_VCPUS:
> + case KVM_CAP_MAX_VCPUS:
> + case KVM_CAP_MAX_VCPU_ID:
> + case KVM_CAP_MSI_DEVID:
> + case KVM_CAP_ARM_VM_IPA_SIZE:
> + case KVM_CAP_ARM_PTRAUTH_ADDRESS:
> + case KVM_CAP_ARM_PTRAUTH_GENERIC:
> + case KVM_CAP_ARM_RMI:
> + return true;
> + }
> + return false;
> +}
> +
> int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
> {
> int r;
> @@ -364,6 +383,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
> if (is_protected_kvm_enabled() && !kvm_pkvm_ext_allowed(kvm, ext))
> return 0;
>
> + if (kvm && kvm_is_realm(kvm) && !kvm_realm_ext_allowed(ext))
> + return 0;
> +
We need a similar check in in kvm_vm_ioctl_enable_cap() to prevent
enabling the filtered caps ? Otherwise looks good to me.
Suzuki
> switch (ext) {
> case KVM_CAP_IRQCHIP:
> r = vgic_present;
^ permalink raw reply
* Re: [PATCH v5 22/22] x86/virt/seamldr: Log TDX module update failures
From: Kiryl Shutsemau @ 2026-03-19 14:06 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260315135920.354657-23-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:42AM -0700, Chao Gao wrote:
> diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
> index e6b7af410c54..4e1ad06506cc 100644
> --- a/arch/x86/virt/vmx/tdx/seamldr.c
> +++ b/arch/x86/virt/vmx/tdx/seamldr.c
> @@ -222,6 +222,11 @@ static void ack_state(void)
> set_target_state(update_data.state + 1);
> }
>
> +static void print_update_failure_message(void)
> +{
> + pr_err_once("update failed, SEAMCALLs will report failure until TDs killed\n");
Is it useful to indicate which step has failed?
>
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v5 21/22] x86/virt/tdx: Document TDX module update
From: Kiryl Shutsemau @ 2026-03-19 14:05 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-doc, linux-coco, kvm, binbin.wu,
dan.j.williams, dave.hansen, ira.weiny, kai.huang, nik.borisov,
paulmck, pbonzini, reinette.chatre, rick.p.edgecombe, sagis,
seanjc, tony.lindgren, vannapurve, vishal.l.verma, yilun.xu,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, x86,
H. Peter Anvin, Jonathan Corbet, Shuah Khan
In-Reply-To: <20260315135920.354657-22-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:41AM -0700, Chao Gao wrote:
> Document TDX module update as a subsection of "TDX Host Kernel Support" to
> provide background information and cover key points that developers and
> users may need to know, for example:
>
> - update is done in stop_machine() context
> - update instructions and results
> - update policy and tooling
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v5 20/22] coco/tdx-host: Document TDX module update compatibility criteria
From: Kiryl Shutsemau @ 2026-03-19 14:04 UTC (permalink / raw)
To: Chao Gao
Cc: x86, linux-coco, kvm, linux-kernel, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu
In-Reply-To: <20260315135920.354657-21-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:40AM -0700, Chao Gao wrote:
> The TDX module update protocol facilitates compatible runtime updates.
>
> Document the compatibility criteria and indicators of various update
> failures, including violations of the compatibility criteria.
>
> Note that runtime TDX module updates are an "update at your own risk"
> operation; userspace must enforce all of the above compatibility
> criteria.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v5 19/22] x86/virt/tdx: Enable TDX module runtime updates
From: Kiryl Shutsemau @ 2026-03-19 14:03 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260315135920.354657-20-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:39AM -0700, Chao Gao wrote:
> All pieces of TDX module runtime updates are in place. Enable it if it
> is supported.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v5 18/22] coco/tdx-host: Don't expose P-SEAMLDR features on CPUs with erratum
From: Kiryl Shutsemau @ 2026-03-19 14:02 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, kvm, linux-coco, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260315135920.354657-19-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:38AM -0700, Chao Gao wrote:
> Some TDX-capable CPUs have an erratum, as documented in Intel® Trust
> Domain CPU Architectural Extensions (May 2021 edition) Chapter 2.3:
>
> SEAMRET from the P-SEAMLDR clears the current VMCS structure pointed
> to by the current-VMCS pointer. A VMM that invokes the P-SEAMLDR using
> SEAMCALL must reload the current-VMCS, if required, using the VMPTRLD
> instruction.
>
> Clearing the current VMCS behind KVM's back will break KVM.
>
> This erratum is not present when IA32_VMX_BASIC[60] is set. Add a CPU
> bug bit for this erratum and refuse to expose P-SEAMLDR features (e.g.,
> TDX module updates) on affected CPUs.
>
> == Alternatives ==
> Two workarounds were considered but both were rejected:
>
> 1. Save/restore the current VMCS around P-SEAMLDR calls. This produces ugly
> assembly code [1] and doesn't play well with #MCE or #NMI if they
> need to use the current VMCS.
>
> 2. Move KVM's VMCS tracking logic to the TDX core code, which would break
> the boundary between KVM and the TDX core code [2].
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Reviewed-by: Kai Huang <kai.huang@intel.com>
> Link: https://lore.kernel.org/kvm/fedb3192-e68c-423c-93b2-a4dc2f964148@intel.com/ # [1]
> Link: https://lore.kernel.org/kvm/aYIXFmT-676oN6j0@google.com/ # [2]
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v5 17/22] x86/virt/tdx: Avoid updates during update-sensitive operations
From: Kiryl Shutsemau @ 2026-03-19 14:00 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260315135920.354657-18-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:37AM -0700, Chao Gao wrote:
> diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
> index b3a7301e77c6..4c4f7acd4044 100644
> --- a/arch/x86/include/asm/tdx.h
> +++ b/arch/x86/include/asm/tdx.h
> @@ -26,11 +26,18 @@
> #define TDX_SEAMCALL_GP (TDX_SW_ERROR | X86_TRAP_GP)
> #define TDX_SEAMCALL_UD (TDX_SW_ERROR | X86_TRAP_UD)
>
> +#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL
> +
> /*
> * TDX module SEAMCALL leaf function error codes
> */
> -#define TDX_SUCCESS 0ULL
> -#define TDX_RND_NO_ENTROPY 0x8000020300000000ULL
> +#define TDX_SUCCESS 0ULL
> +#define TDX_RND_NO_ENTROPY 0x8000020300000000ULL
> +#define TDX_UPDATE_COMPAT_SENSITIVE 0x8000051200000000ULL
This competes with other patchset[1].
[1] https://lore.kernel.org/all/20260307010358.819645-1-rick.p.edgecombe@intel.com
> @@ -1189,9 +1192,21 @@ int tdx_module_shutdown(void)
> * modules as new modules likely have higher handoff version.
> */
> args.rcx = tdx_sysinfo.handoff.module_hv;
> - ret = seamcall_prerr(TDH_SYS_SHUTDOWN, &args);
> - if (ret)
> - return ret;
> +
> + if (tdx_supports_update_compatibility(&tdx_sysinfo))
> + args.rcx |= TDX_SYS_SHUTDOWN_AVOID_COMPAT_SENSITIVE;
Hm. So what happens if the module doesn't support it? We just ignore
problem?
Maybe we should just block updates on such modules?
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v5 16/22] x86/virt/tdx: Update tdx_sysinfo and check features post-update
From: Kiryl Shutsemau @ 2026-03-19 13:43 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260315135920.354657-17-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:36AM -0700, Chao Gao wrote:
> tdx_sysinfo contains all metadata of the active TDX module, including
> versions, supported features, and TDMR/TDCS/TDVPS information etc. These
> values may change over updates. Blindly refreshing the entire tdx_sysinfo
> could disrupt running software, as it may subtly rely on the previous state
> unless proven otherwise.
>
> Adopt a conservative approach, like microcode updates, by only refreshing
> version information that does not affect functionality, while ignoring
> all other changes. This is acceptable as new modules are required to
> maintain backward compatibility.
>
> Any updates to metadata beyond versions should be justified and reviewed on
> a case-by-case basis.
>
> Note that preallocating a tdx_sys_info buffer before updates is to avoid
> having to handle -ENOMEM when updating tdx_sysinfo after a successful
> update.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
> Reviewed-by: Kai Huang <kai.huang@intel.com>
> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v5 15/22] x86/virt/tdx: Restore TDX module state
From: Kiryl Shutsemau @ 2026-03-19 13:37 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260315135920.354657-16-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:35AM -0700, Chao Gao wrote:
> TDX module state was packed as handoff data during module shutdown. After
> per-CPU initialization, the new module can restore TDX module state from
> handoff data to preserve running TDs.
>
> Once the restoration is done, the TDX module update is complete, which
> means the new module is ready to handle requests from the host and guests.
>
> Implement the new TDH.SYS.UPDATE SEAMCALL to restore TDX module state
> and invoke it on one CPU since it only needs to be called once.
>
> Note that Intel® Trust Domain Extensions (Intel® TDX) Module Base
> Architecture Specification, Revision 348549-007, Chapter 4.5.5 states:
>
> If TDH.SYS.UPDATE returns an error, then the host VMM can continue
> with the non-update sequence (TDH.SYS.CONFIG, 15 TDH.SYS.KEY.CONFIG
> etc.). In this case all existing TDs are lost. Alternatively, the host
> VMM can request the P-SEAMLDR to update to another TDX module. If that
> update is successful, existing TDs are preserved
>
> Don't implement the two alternatives due to their complexity and unclear
> benefits.
Make it clear which option you've took.
>
> Also note that the location and the format of handoff data is defined by
> the TDX module. The new module knows where to get handoff data and how
> to parse it. The kernel doesn't need to provide its location, format etc.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
> Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v5 14/22] x86/virt/seamldr: Do TDX per-CPU initialization after updates
From: Kiryl Shutsemau @ 2026-03-19 13:33 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260315135920.354657-15-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:34AM -0700, Chao Gao wrote:
> After installing the new TDX module, each CPU needs to be initialized
> again to make the CPU ready to run any other SEAMCALLs. So, call
> tdx_cpu_enable() on all CPUs.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
> Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v5 13/22] x86/virt/seamldr: Install a new TDX module
From: Kiryl Shutsemau @ 2026-03-19 13:32 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260315135920.354657-14-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:33AM -0700, Chao Gao wrote:
> @@ -225,6 +227,7 @@ static void ack_state(void)
> static int do_seamldr_install_module(void *seamldr_params)
> {
> enum module_update_state newstate, curstate = MODULE_UPDATE_START;
> + struct tdx_module_args args = {};
> int cpu = smp_processor_id();
> bool primary;
> int ret = 0;
> @@ -243,6 +246,10 @@ static int do_seamldr_install_module(void *seamldr_params)
> if (primary)
> ret = tdx_module_shutdown();
> break;
> + case MODULE_UPDATE_CPU_INSTALL:
> + args.rcx = __pa(seamldr_params);
> + ret = seamldr_call(P_SEAMLDR_INSTALL, &args);
I think it should be in a helper alongside with the tdx_module_args.
> + break;
> default:
> break;
> }
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v5 11/22] x86/virt/seamldr: Shut down the current TDX module
From: Kiryl Shutsemau @ 2026-03-19 13:24 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260315135920.354657-12-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:31AM -0700, Chao Gao wrote:
> The first step of TDX module updates is shutting down the current TDX
> Module. This step also packs state information that needs to be
> preserved across updates as handoff data, which will be consumed by the
> updated module. The handoff data is stored internally in the SEAM range
> and is hidden from the kernel.
>
> To ensure a successful update, the new module must be able to consume
> the handoff data generated by the old module. Since handoff data layout
> may change between modules, the handoff data is versioned. Each module
> has a native handoff version and provides backward support for several
> older versions.
>
> The complete handoff versioning protocol is complex as it supports both
> module upgrades and downgrades. See details in Intel® Trust Domain
> Extensions (Intel® TDX) Module Base Architecture Specification, Revision
> 348549-007, Chapter 4.5.3 "Handoff Versioning".
>
> Ideally, the kernel needs to retrieve the handoff versions supported by
> the current module and the new module and select a version supported by
> both. But, since this implementation chooses to only support module
> upgrades, simply request the current module to generate handoff data
> using its highest supported version, expecting that the new module will
> likely support it.
>
> Note that only one CPU needs to call the TDX module's shutdown API.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
> Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Nit is below.
> @@ -100,6 +100,17 @@ static int get_tdx_sys_info_td_conf(struct tdx_sys_info_td_conf *sysinfo_td_conf
> return ret;
> }
>
> +static int get_tdx_sys_info_handoff(struct tdx_sys_info_handoff *sysinfo_handoff)
> +{
> + int ret = 0;
> + u64 val;
> +
> + if (!ret && !(ret = read_sys_metadata_field(0x8900000100000000, &val)))
!ret check is redundant as well as the ret initialization above.
> + sysinfo_handoff->module_hv = val;
> +
> + return ret;
> +}
> +
> static int get_tdx_sys_info(struct tdx_sys_info *sysinfo)
> {
> int ret = 0;
> @@ -116,5 +127,12 @@ static int get_tdx_sys_info(struct tdx_sys_info *sysinfo)
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v5 10/22] x86/virt/seamldr: Abort updates if errors occurred midway
From: Kiryl Shutsemau @ 2026-03-19 13:19 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260315135920.354657-11-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:30AM -0700, Chao Gao wrote:
> diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
> index 978fcca92128..e195703398e7 100644
> --- a/arch/x86/virt/vmx/tdx/seamldr.c
> +++ b/arch/x86/virt/vmx/tdx/seamldr.c
> @@ -190,6 +190,7 @@ enum module_update_state {
> static struct {
> enum module_update_state state;
> int thread_ack;
> + int failed;
bool is enough, right?
> /*
> * Protect update_data. Raw spinlock as it will be acquired from
> * interrupt-disabled contexts.
> @@ -237,12 +238,17 @@ static int do_seamldr_install_module(void *seamldr_params)
> break;
> }
>
> - ack_state();
> + if (ret) {
> + scoped_guard(raw_spinlock, &update_data.lock)
> + update_data.failed++;
I don't see a reason in spin lock here. Just WRITE_ONCE() would work
fine.
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
* Re: [PATCH v5 09/22] x86/virt/seamldr: Introduce skeleton for TDX module updates
From: Kiryl Shutsemau @ 2026-03-19 13:15 UTC (permalink / raw)
To: Chao Gao
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <20260315135920.354657-10-chao.gao@intel.com>
On Sun, Mar 15, 2026 at 06:58:29AM -0700, Chao Gao wrote:
> TDX module updates require careful synchronization with other TDX
> operations. The requirements are (#1/#2 reflect current behavior that
> must be preserved):
>
> 1. SEAMCALLs need to be callable from both process and IRQ contexts.
> 2. SEAMCALLs need to be able to run concurrently across CPUs
> 3. During updates, only update-related SEAMCALLs are permitted; all
> other SEAMCALLs shouldn't be called.
> 4. During updates, all online CPUs must participate in the update work.
>
> No single lock primitive satisfies all requirements. For instance,
> rwlock_t handles #1/#2 but fails #4: CPUs spinning with IRQs disabled
> cannot be directed to perform update work.
>
> Use stop_machine() as it is the only well-understood mechanism that can
> meet all requirements.
>
> And TDX module updates consist of several steps (See Intel® Trust Domain
> Extensions (Intel® TDX) Module Base Architecture Specification, Revision
> 348549-007, Chapter 4.5 "TD-Preserving TDX module Update"). Ordering
> requirements between steps mandate lockstep synchronization across all
> CPUs.
>
> multi_cpu_stop() is a good example of performing a multi-step task in
> lockstep. But it doesn't synchronize steps within the callback function
> it takes. So, implement one based on its pattern to establish the
> skeleton for TDX module updates. Specifically, add a global state
> machine where each state represents a step in the update flow. The state
> advances only after all CPUs acknowledge completing their work in the
> current state. This acknowledgment mechanism is what ensures lockstep
> execution.
>
> Potential alternative to stop_machine()
> =======================================
> An alternative approach is to lock all KVM entry points and kick all
> vCPUs. Here, KVM entry points refer to KVM VM/vCPU ioctl entry points,
> implemented in KVM common code (virt/kvm). Adding a locking mechanism
> there would affect all architectures KVM supports. And to lock only TDX
> vCPUs, new logic would be needed to identify TDX vCPUs, which the KVM
> common code currently lacks. This would add significant complexity and
> maintenance overhead to KVM for this TDX-specific use case.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
One question below.
> +static struct {
> + enum module_update_state state;
> + int thread_ack;
> + /*
> + * Protect update_data. Raw spinlock as it will be acquired from
> + * interrupt-disabled contexts.
> + */
> + raw_spinlock_t lock;
> +} update_data = {
> + .lock = __RAW_SPIN_LOCK_UNLOCKED(update_data.lock)
> +};
multi_stop_cpu() used atomic_t for thread_ack insead of spinlock.
Any particular reason you took different direction?
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox