* Re: [PATCH] sample/tsm-mr: Use SHA-2 library APIs
From: Eric Biggers @ 2026-03-19 17:32 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-crypto, Arnd Bergmann, Dan Williams, Dionna Amalie Glaze,
Cedric Xing, Andrew Morton, Zi Li, Masami Hiramatsu, Lance Yang,
Zhou Yuhang, Colin Ian King, Ard Biesheuvel, linux-coco,
linux-kernel
In-Reply-To: <dc124ea8-05b8-42d2-93ad-d265e0ecf585@app.fastmail.com>
On Wed, Mar 18, 2026 at 08:57:01PM +0100, Arnd Bergmann wrote:
> On Wed, Mar 18, 2026, at 17:42, Eric Biggers wrote:
> > Given that tsm_mr_sample has a particular set of algorithms that it
> > wants, just use the library APIs for those algorithms rather than
> > crypto_shash. This is more straightforward and a bit more efficient.
> >
> > This fixes an issue where this module failed to build due to the kconfig
> > options CRYPTO and CRYPTO_HASH not being selected. Also, even if it
> > built, crypto_alloc_shash() could fail at runtime due to the needed
> > algorithms not being available.
> >
> > The library functions simply use direct linking. So if it builds, which
> > it will due to the kconfig options being enabled, they are available.
> >
> > Fixes: f6953f1f9ec4 ("tsm-mr: Add tsm-mr sample code")
> > Fixes: 44a3873df811 ("coco/guest: Remove unneeded selection of CRYPTO")
> > Signed-off-by: Eric Biggers <ebiggers@kernel.org>
> > ---
> >
> > I'd like to take this via libcrypto-next, as that is where
> > "coco/guest: Remove unneeded selection of CRYPTO" is.
>
> Thanks for fixing this! It is indeed nicer than the fix
> I sent earlier today.
>
> Acked-by: Arnd Bergmann <arnd@arndb.de>
Thanks. Additional acks from the people owning this code (Dan, Cedric?)
would be appreciated. But since this fixes a build error and is related
to the crypto library, I went ahead and applied this to
https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git/log/?h=libcrypto-next
I also found that the build error is pre-existing, as CRYPTO_HASH was
not being selected. "coco/guest: Remove unneeded selection of CRYPTO"
just made it a bit easier to encounter, by not selecting CRYPTO either.
So I updated the second paragraph of the commit message to:
This also fixes a bug where this module failed to build if it was
enabled without CRYPTO_HASH happening to be set elsewhere in the
kconfig. (With the concurrent change to make TSM_MEASUREMENTS stop
selecting CRYPTO, this existing build error would have become easier to
encounter, as well.) Also, even if it built, crypto_alloc_shash() could
fail at runtime due to the needed algorithms not being available.
I also put this commit before "coco/guest: Remove unneeded selection of
CRYPTO" and dropped the Fixes reference to that. So now it just has:
Fixes: f6953f1f9ec4 ("tsm-mr: Add tsm-mr sample code")
- Eric
^ permalink raw reply
* Re: [PATCH v13 15/48] arm64: RMI: RTT tear down
From: Wei-Lin Chang @ 2026-03-19 17:35 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <20260318155413.793430-16-steven.price@arm.com>
On Wed, Mar 18, 2026 at 03:53:39PM +0000, Steven Price wrote:
> The RMM owns the stage 2 page tables for a realm, and KVM must request
> that the RMM creates/destroys entries as necessary. The physical pages
> to store the page tables are delegated to the realm as required, and can
> be undelegated when no longer used.
>
> Creating new RTTs is the easy part, tearing down is a little more
> tricky. The result of realm_rtt_destroy() can be used to effectively
> walk the tree and destroy the entries (undelegating pages that were
> given to the realm).
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v12:
> * Simplify some functions now we know RMM page size is the same as the
> host's.
> Changes since v11:
> * Moved some code from earlier in the series to this one so that it's
> added when it's first used.
> Changes since v10:
> * RME->RMI rename.
> * Some code to handle freeing stage 2 PGD moved into this patch where
> it belongs.
> Changes since v9:
> * Add a comment clarifying that root level RTTs are not destroyed until
> after the RD is destroyed.
> Changes since v8:
> * Introduce free_rtt() wrapper which calls free_delegated_granule()
> followed by kvm_account_pgtable_pages(). This makes it clear where an
> RTT is being freed rather than just a delegated granule.
> Changes since v6:
> * Move rme_rtt_level_mapsize() and supporting defines from kvm_rme.h
> into rme.c as they are only used in that file.
> Changes since v5:
> * Rename some RME_xxx defines to do with page sizes as RMM_xxx - they are
> a property of the RMM specification not the RME architecture.
> Changes since v2:
> * Moved {alloc,free}_delegated_page() and ensure_spare_page() to a
> later patch when they are actually used.
> * Some simplifications now rmi_xxx() functions allow NULL as an output
> parameter.
> * Improved comments and code layout.
> ---
> arch/arm64/include/asm/kvm_rmi.h | 7 ++
> arch/arm64/kvm/mmu.c | 15 +++-
> arch/arm64/kvm/rmi.c | 145 +++++++++++++++++++++++++++++++
> 3 files changed, 166 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
> index 0ada525af18f..16a297f3091a 100644
> --- a/arch/arm64/include/asm/kvm_rmi.h
> +++ b/arch/arm64/include/asm/kvm_rmi.h
> @@ -68,5 +68,12 @@ u32 kvm_realm_ipa_limit(void);
>
> int kvm_init_realm_vm(struct kvm *kvm);
> void kvm_destroy_realm(struct kvm *kvm);
> +void kvm_realm_destroy_rtts(struct kvm *kvm);
> +
> +static inline bool kvm_realm_is_private_address(struct realm *realm,
> + unsigned long addr)
> +{
> + return !(addr & BIT(realm->ia_bits - 1));
> +}
>
> #endif /* __ASM_KVM_RMI_H */
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 9dc242c3b9c8..41152abf55b2 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -1098,10 +1098,23 @@ void stage2_unmap_vm(struct kvm *kvm)
> void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
> {
> struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
> - struct kvm_pgtable *pgt = NULL;
> + struct kvm_pgtable *pgt;
>
> write_lock(&kvm->mmu_lock);
> pgt = mmu->pgt;
> + if (kvm_is_realm(kvm) &&
> + (kvm_realm_state(kvm) != REALM_STATE_DEAD &&
> + kvm_realm_state(kvm) != REALM_STATE_NONE)) {
> + write_unlock(&kvm->mmu_lock);
> + kvm_realm_destroy_rtts(kvm);
> +
> + /*
> + * The PGD pages can be reclaimed only after the realm (RD) is
> + * destroyed. We call this again from kvm_destroy_realm() after
> + * the RD is destroyed.
> + */
> + return;
> + }
Hi,
I see that kvm_free_stage2_pgd() will be called twice:
kvm_destroy_vm()
mmu_notifier_unregister()
kvm_mmu_notifier_release()
kvm_flush_shadow_all()
kvm_arch_flush_shadow_all()
kvm_uninit_stage2_mmu()
kvm_free_stage2_pgd()
kvm_arch_destroy_vm()
kvm_destroy_realm()
kvm_free_stage2_pgd()
At the first call the realm state is REALM_STATE_ACTIVE, at the second
it is REALM_STATE_DEAD. Reading the comment added to
kvm_free_stage2_pgd() here, does it mean this function is called twice
on purpose? If so do you think it's better to extract this and create
another function instead, then use kvm_is_realm() to choose which to
run? I think it is confusing to have this function run twice for a
realm.
Thanks,
Wei-Lin Chang
> if (pgt) {
> mmu->pgd_phys = 0;
> mmu->pgt = NULL;
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> index 700b8c935d29..1fd2c18f7381 100644
> --- a/arch/arm64/kvm/rmi.c
> +++ b/arch/arm64/kvm/rmi.c
> @@ -15,6 +15,19 @@
> static unsigned long rmm_feat_reg0;
> static unsigned long rmm_feat_reg1;
>
> +#define RMM_RTT_BLOCK_LEVEL 2
> +#define RMM_RTT_MAX_LEVEL 3
> +
> +#define RMM_L2_BLOCK_SIZE PMD_SIZE
> +
> +static inline unsigned long rmi_rtt_level_mapsize(int level)
> +{
> + if (WARN_ON(level > RMM_RTT_MAX_LEVEL))
> + return PAGE_SIZE;
> +
> + return (1UL << ARM64_HW_PGTABLE_LEVEL_SHIFT(level));
> +}
> +
> static bool rmi_has_feature(unsigned long feature)
> {
> return !!u64_get_bits(rmm_feat_reg0, feature);
> @@ -189,6 +202,11 @@ u32 kvm_realm_ipa_limit(void)
> return u64_get_bits(rmm_feat_reg0, RMI_FEATURE_REGISTER_0_S2SZ);
> }
>
> +static int get_start_level(struct realm *realm)
> +{
> + return 4 - stage2_pgtable_levels(realm->ia_bits);
> +}
> +
> static int undelegate_range(phys_addr_t phys, unsigned long size)
> {
> unsigned long ret;
> @@ -223,6 +241,131 @@ static int free_delegated_page(phys_addr_t phys)
> return 0;
> }
>
> +static void free_rtt(phys_addr_t phys)
> +{
> + if (free_delegated_page(phys))
> + return;
> +
> + kvm_account_pgtable_pages(phys_to_virt(phys), -1);
> +}
> +
> +static int realm_rtt_destroy(struct realm *realm, unsigned long addr,
> + int level, phys_addr_t *rtt_granule,
> + unsigned long *next_addr)
> +{
> + unsigned long out_rtt;
> + int ret;
> +
> + ret = rmi_rtt_destroy(virt_to_phys(realm->rd), addr, level,
> + &out_rtt, next_addr);
> +
> + *rtt_granule = out_rtt;
> +
> + return ret;
> +}
> +
> +static int realm_tear_down_rtt_level(struct realm *realm, int level,
> + unsigned long start, unsigned long end)
> +{
> + ssize_t map_size;
> + unsigned long addr, next_addr;
> +
> + if (WARN_ON(level > RMM_RTT_MAX_LEVEL))
> + return -EINVAL;
> +
> + map_size = rmi_rtt_level_mapsize(level - 1);
> +
> + for (addr = start; addr < end; addr = next_addr) {
> + phys_addr_t rtt_granule;
> + int ret;
> + unsigned long align_addr = ALIGN(addr, map_size);
> +
> + next_addr = ALIGN(addr + 1, map_size);
> +
> + if (next_addr > end || align_addr != addr) {
> + /*
> + * The target range is smaller than what this level
> + * covers, recurse deeper.
> + */
> + ret = realm_tear_down_rtt_level(realm,
> + level + 1,
> + addr,
> + min(next_addr, end));
> + if (ret)
> + return ret;
> + continue;
> + }
> +
> + ret = realm_rtt_destroy(realm, addr, level,
> + &rtt_granule, &next_addr);
> +
> + switch (RMI_RETURN_STATUS(ret)) {
> + case RMI_SUCCESS:
> + free_rtt(rtt_granule);
> + break;
> + case RMI_ERROR_RTT:
> + if (next_addr > addr) {
> + /* Missing RTT, skip */
> + break;
> + }
> + /*
> + * We tear down the RTT range for the full IPA
> + * space, after everything is unmapped. Also we
> + * descend down only if we cannot tear down a
> + * top level RTT. Thus RMM must be able to walk
> + * to the requested level. e.g., a block mapping
> + * exists at L1 or L2.
> + */
> + if (WARN_ON(RMI_RETURN_INDEX(ret) != level))
> + return -EBUSY;
> + if (WARN_ON(level == RMM_RTT_MAX_LEVEL))
> + return -EBUSY;
> +
> + /*
> + * The table has active entries in it, recurse deeper
> + * and tear down the RTTs.
> + */
> + next_addr = ALIGN(addr + 1, map_size);
> + ret = realm_tear_down_rtt_level(realm,
> + level + 1,
> + addr,
> + next_addr);
> + if (ret)
> + return ret;
> + /*
> + * Now that the child RTTs are destroyed,
> + * retry at this level.
> + */
> + next_addr = addr;
> + break;
> + default:
> + WARN_ON(1);
> + return -ENXIO;
> + }
> + }
> +
> + return 0;
> +}
> +
> +static int realm_tear_down_rtt_range(struct realm *realm,
> + unsigned long start, unsigned long end)
> +{
> + /*
> + * Root level RTTs can only be destroyed after the RD is destroyed. So
> + * tear down everything below the root level
> + */
> + return realm_tear_down_rtt_level(realm, get_start_level(realm) + 1,
> + start, end);
> +}
> +
> +void kvm_realm_destroy_rtts(struct kvm *kvm)
> +{
> + struct realm *realm = &kvm->arch.realm;
> + unsigned int ia_bits = realm->ia_bits;
> +
> + WARN_ON(realm_tear_down_rtt_range(realm, 0, (1UL << ia_bits)));
> +}
> +
> void kvm_destroy_realm(struct kvm *kvm)
> {
> struct realm *realm = &kvm->arch.realm;
> @@ -246,6 +389,8 @@ void kvm_destroy_realm(struct kvm *kvm)
> if (realm->rd) {
> phys_addr_t rd_phys = virt_to_phys(realm->rd);
>
> + kvm_realm_destroy_rtts(kvm);
> +
> if (WARN_ON(rmi_realm_destroy(rd_phys)))
> return;
> free_delegated_page(rd_phys);
> --
> 2.43.0
>
^ permalink raw reply
* Re: [PATCH v13 46/48] KVM: arm64: Expose KVM_ARM_VCPU_REC to user space
From: Suzuki K Poulose @ 2026-03-19 17:36 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Zenghui Yu, linux-arm-kernel, linux-kernel,
Joey Gouly, Alexandru Elisei, Christoffer Dall, Fuad Tabba,
linux-coco, Ganapatrao Kulkarni, Gavin Shan, Shanker Donthineni,
Alper Gun, Aneesh Kumar K . V, Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20260318155413.793430-47-steven.price@arm.com>
On 18/03/2026 15:54, Steven Price wrote:
> Increment KVM_VCPU_MAX_FEATURES to expose the new capability to user
> space.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> Reviewed-by: Gavin Shan <gshan@redhat.com>
Not needed any more as we don't need the VCPU feature.
Cheers
Suzuki
> ---
> Changes since v8:
> * Since NV is now merged and enabled, this no longer conflicts with it.
> ---
> arch/arm64/include/asm/kvm_host.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 1d5fb001408c..b02f97de4436 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -40,7 +40,7 @@
>
> #define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
>
> -#define KVM_VCPU_MAX_FEATURES 9
> +#define KVM_VCPU_MAX_FEATURES 10
> #define KVM_VCPU_VALID_FEATURES (BIT(KVM_VCPU_MAX_FEATURES) - 1)
>
> #define KVM_REQ_SLEEP \
^ permalink raw reply
* Re: [PATCH] sample/tsm-mr: Use SHA-2 library APIs
From: Dan Williams @ 2026-03-19 17:51 UTC (permalink / raw)
To: Eric Biggers, Arnd Bergmann
Cc: linux-crypto, Arnd Bergmann, Dan Williams, Dionna Amalie Glaze,
Cedric Xing, Andrew Morton, Zi Li, Masami Hiramatsu, Lance Yang,
Zhou Yuhang, Colin Ian King, Ard Biesheuvel, linux-coco,
linux-kernel
In-Reply-To: <20260319173222.GA10208@quark>
Eric Biggers wrote:
> On Wed, Mar 18, 2026 at 08:57:01PM +0100, Arnd Bergmann wrote:
> > On Wed, Mar 18, 2026, at 17:42, Eric Biggers wrote:
> > > Given that tsm_mr_sample has a particular set of algorithms that it
> > > wants, just use the library APIs for those algorithms rather than
> > > crypto_shash. This is more straightforward and a bit more efficient.
> > >
> > > This fixes an issue where this module failed to build due to the kconfig
> > > options CRYPTO and CRYPTO_HASH not being selected. Also, even if it
> > > built, crypto_alloc_shash() could fail at runtime due to the needed
> > > algorithms not being available.
> > >
> > > The library functions simply use direct linking. So if it builds, which
> > > it will due to the kconfig options being enabled, they are available.
> > >
> > > Fixes: f6953f1f9ec4 ("tsm-mr: Add tsm-mr sample code")
> > > Fixes: 44a3873df811 ("coco/guest: Remove unneeded selection of CRYPTO")
> > > Signed-off-by: Eric Biggers <ebiggers@kernel.org>
> > > ---
> > >
> > > I'd like to take this via libcrypto-next, as that is where
> > > "coco/guest: Remove unneeded selection of CRYPTO" is.
> >
> > Thanks for fixing this! It is indeed nicer than the fix
> > I sent earlier today.
> >
> > Acked-by: Arnd Bergmann <arnd@arndb.de>
>
> Thanks. Additional acks from the people owning this code (Dan, Cedric?)
> would be appreciated. But since this fixes a build error and is related
> to the crypto library, I went ahead and applied this to
> https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git/log/?h=libcrypto-next
>
> I also found that the build error is pre-existing, as CRYPTO_HASH was
> not being selected. "coco/guest: Remove unneeded selection of CRYPTO"
> just made it a bit easier to encounter, by not selecting CRYPTO either.
>
> So I updated the second paragraph of the commit message to:
>
> This also fixes a bug where this module failed to build if it was
> enabled without CRYPTO_HASH happening to be set elsewhere in the
> kconfig. (With the concurrent change to make TSM_MEASUREMENTS stop
> selecting CRYPTO, this existing build error would have become easier to
> encounter, as well.) Also, even if it built, crypto_alloc_shash() could
> fail at runtime due to the needed algorithms not being available.
>
> I also put this commit before "coco/guest: Remove unneeded selection of
> CRYPTO" and dropped the Fixes reference to that. So now it just has:
>
> Fixes: f6953f1f9ec4 ("tsm-mr: Add tsm-mr sample code")
It looks good to me:
Acked-by: Dan Williams <dan.j.williams@intel.com>
Feel free to take it through your tree since I have nothing immediately
pending for tsm.git.
^ permalink raw reply
* Re: [PATCH 1/2] x86/virt/tdx: Use PFN directly for mapping guest private memory
From: Dave Hansen @ 2026-03-19 18:05 UTC (permalink / raw)
To: Yan Zhao, seanjc, pbonzini, dave.hansen
Cc: tglx, mingo, bp, kas, x86, linux-kernel, kvm, linux-coco,
kai.huang, rick.p.edgecombe, yilun.xu, vannapurve, ackerleytng,
sagis, binbin.wu, xiaoyao.li, isaku.yamahata
In-Reply-To: <20260319005703.8983-1-yan.y.zhao@intel.com>
On 3/18/26 17:57, Yan Zhao wrote:
> Remove the completely unnecessary assumption that memory mapped into a TDX
> guest is backed by refcounted struct page memory. From KVM's point of view,
> TDH_MEM_PAGE_ADD and TDH_MEM_PAGE_AUG are glorified writes to PTEs, so they
> have no business placing requirements on how KVM and guest_memfd manage
> memory.
I think this goes a bit too far.
It's one thing to say that it's more convenient for KVM to stick with
pfns because it's what KVM uses now. Or, that the goals of using 'struct
page' can be accomplished other ways. It's quite another to say what
other bits of the codebase have "business" doing.
Sean, can we tone this down a _bit_ to help guide folks in the future?
> Rip out the misguided struct page assumptions/constraints and instead have
Could we maybe tone down the editorializing a bit, please? Folks can
have honest disagreements about this stuff while not being "misguided".
> the two SEAMCALL wrapper APIs take PFN directly. This ensures that for
> future huge page support in S-EPT, the kernel doesn't pick up even worse
> assumptions like "a hugepage must be contained in a single folio".
I don't really understand what this is saying.
Is the concern that KVM might want to set up page tables for memory that
differ from how it was allocated? I'm a bit worried that this assumes
something about folios that doesn't always hold.
I think the hugetlbfs gigantic support uses folios in at least a few
spots today.
^ permalink raw reply
* Re: [PATCH v13 07/48] arm64: RMI: Check for RMI support at KVM init
From: Wei-Lin Chang @ 2026-03-19 18:05 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <20260318155413.793430-8-steven.price@arm.com>
On Wed, Mar 18, 2026 at 03:53:31PM +0000, Steven Price wrote:
> Query the RMI version number and check if it is a compatible version. A
> static key is also provided to signal that a supported RMM is available.
>
> Functions are provided to query if a VM or VCPU is a realm (or rec)
> which currently will always return false.
>
> Later patches make use of struct realm and the states as the ioctls
> interfaces are added to support realm and REC creation and destruction.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v12:
> * Drop check for 4k page size.
> Changes since v11:
> * Reword slightly the comments on the realm states.
> Changes since v10:
> * kvm_is_realm() no longer has a NULL check.
> * Rename from "rme" to "rmi" when referring to the RMM interface.
> * Check for RME (hardware) support before probing for RMI support.
> Changes since v8:
> * No need to guard kvm_init_rme() behind 'in_hyp_mode'.
> Changes since v6:
> * Improved message for an unsupported RMI ABI version.
> Changes since v5:
> * Reword "unsupported" message from "host supports" to "we want" to
> clarify that 'we' are the 'host'.
> Changes since v2:
> * Drop return value from kvm_init_rme(), it was always 0.
> * Rely on the RMM return value to identify whether the RSI ABI is
> compatible.
> ---
> arch/arm64/include/asm/kvm_emulate.h | 18 +++++++++
> arch/arm64/include/asm/kvm_host.h | 4 ++
> arch/arm64/include/asm/kvm_rmi.h | 56 +++++++++++++++++++++++++++
> arch/arm64/include/asm/virt.h | 1 +
> arch/arm64/kernel/cpufeature.c | 1 +
> arch/arm64/kvm/Makefile | 2 +-
> arch/arm64/kvm/arm.c | 5 +++
> arch/arm64/kvm/rmi.c | 57 ++++++++++++++++++++++++++++
> 8 files changed, 143 insertions(+), 1 deletion(-)
> create mode 100644 arch/arm64/include/asm/kvm_rmi.h
> create mode 100644 arch/arm64/kvm/rmi.c
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 5bf3d7e1d92c..f38b50151ce8 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -688,4 +688,22 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
> vcpu->arch.hcrx_el2 |= HCRX_EL2_EnASR;
> }
> }
> +
> +static inline bool kvm_is_realm(struct kvm *kvm)
> +{
> + if (static_branch_unlikely(&kvm_rmi_is_available))
> + return kvm->arch.is_realm;
> + return false;
> +}
> +
> +static inline enum realm_state kvm_realm_state(struct kvm *kvm)
> +{
> + return READ_ONCE(kvm->arch.realm.state);
> +}
Hi,
Do you think it would be helpful to have a write version of this?
That way we can search for the write version to see all the locations of
realm state changes, instead of having to search through all the
WRITE_ONCE()'s.
Thanks,
Wei-Lin Chang
> +
> +static inline bool vcpu_is_rec(struct kvm_vcpu *vcpu)
> +{
> + return false;
> +}
> +
> #endif /* __ARM64_KVM_EMULATE_H__ */
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 5d5a3bbdb95e..9267a2f2d65b 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -27,6 +27,7 @@
> #include <asm/fpsimd.h>
> #include <asm/kvm.h>
> #include <asm/kvm_asm.h>
> +#include <asm/kvm_rmi.h>
> #include <asm/vncr_mapping.h>
>
> #define __KVM_HAVE_ARCH_INTC_INITIALIZED
> @@ -405,6 +406,9 @@ struct kvm_arch {
> * the associated pKVM instance in the hypervisor.
> */
> struct kvm_protected_vm pkvm;
> +
> + bool is_realm;
> + struct realm realm;
> };
>
> struct kvm_vcpu_fault_info {
> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
> new file mode 100644
> index 000000000000..3506f50b05cd
> --- /dev/null
> +++ b/arch/arm64/include/asm/kvm_rmi.h
> @@ -0,0 +1,56 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2023-2025 ARM Ltd.
> + */
> +
> +#ifndef __ASM_KVM_RMI_H
> +#define __ASM_KVM_RMI_H
> +
> +/**
> + * enum realm_state - State of a Realm
> + */
> +enum realm_state {
> + /**
> + * @REALM_STATE_NONE:
> + * Realm has not yet been created. rmi_realm_create() has not
> + * yet been called.
> + */
> + REALM_STATE_NONE,
> + /**
> + * @REALM_STATE_NEW:
> + * Realm is under construction, rmi_realm_create() has been
> + * called, but it is not yet activated. Pages may be populated.
> + */
> + REALM_STATE_NEW,
> + /**
> + * @REALM_STATE_ACTIVE:
> + * Realm has been created and is eligible for execution with
> + * rmi_rec_enter(). Pages may no longer be populated with
> + * rmi_data_create().
> + */
> + REALM_STATE_ACTIVE,
> + /**
> + * @REALM_STATE_DYING:
> + * Realm is in the process of being destroyed or has already been
> + * destroyed.
> + */
> + REALM_STATE_DYING,
> + /**
> + * @REALM_STATE_DEAD:
> + * Realm has been destroyed.
> + */
> + REALM_STATE_DEAD
> +};
> +
> +/**
> + * struct realm - Additional per VM data for a Realm
> + *
> + * @state: The lifetime state machine for the realm
> + */
> +struct realm {
> + enum realm_state state;
> +};
> +
> +void kvm_init_rmi(void);
> +
> +#endif /* __ASM_KVM_RMI_H */
> diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
> index b51ab6840f9c..dc9b2899e0b2 100644
> --- a/arch/arm64/include/asm/virt.h
> +++ b/arch/arm64/include/asm/virt.h
> @@ -87,6 +87,7 @@ void __hyp_reset_vectors(void);
> bool is_kvm_arm_initialised(void);
>
> DECLARE_STATIC_KEY_FALSE(kvm_protected_mode_initialized);
> +DECLARE_STATIC_KEY_FALSE(kvm_rmi_is_available);
>
> static inline bool is_pkvm_initialized(void)
> {
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index c31f8e17732a..ddf7e57f23e8 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -289,6 +289,7 @@ static const struct arm64_ftr_bits ftr_id_aa64isar3[] = {
> static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_CSV3_SHIFT, 4, 0),
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_CSV2_SHIFT, 4, 0),
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_RME_SHIFT, 4, 0),
> ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_DIT_SHIFT, 4, 0),
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_AMU_SHIFT, 4, 0),
> ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_MPAM_SHIFT, 4, 0),
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index 3ebc0570345c..e17c4077d8e7 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -16,7 +16,7 @@ CFLAGS_handle_exit.o += -Wno-override-init
> kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \
> inject_fault.o va_layout.o handle_exit.o config.o \
> guest.o debug.o reset.o sys_regs.o stacktrace.o \
> - vgic-sys-reg-v3.o fpsimd.o pkvm.o \
> + vgic-sys-reg-v3.o fpsimd.o pkvm.o rmi.o \
> arch_timer.o trng.o vmid.o emulate-nested.o nested.o at.o \
> vgic/vgic.o vgic/vgic-init.o \
> vgic/vgic-irqfd.o vgic/vgic-v2.o \
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 29f0326f7e00..274d7866efdc 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -39,6 +39,7 @@
> #include <asm/kvm_nested.h>
> #include <asm/kvm_pkvm.h>
> #include <asm/kvm_ptrauth.h>
> +#include <asm/kvm_rmi.h>
> #include <asm/sections.h>
> #include <asm/stacktrace/nvhe.h>
>
> @@ -104,6 +105,8 @@ long kvm_get_cap_for_kvm_ioctl(unsigned int ioctl, long *ext)
> return -EINVAL;
> }
>
> +DEFINE_STATIC_KEY_FALSE(kvm_rmi_is_available);
> +
> DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector);
>
> DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_base);
> @@ -2921,6 +2924,8 @@ static __init int kvm_arm_init(void)
>
> in_hyp_mode = is_kernel_in_hyp_mode();
>
> + kvm_init_rmi();
> +
> if (cpus_have_final_cap(ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE) ||
> cpus_have_final_cap(ARM64_WORKAROUND_1508412))
> kvm_info("Guests without required CPU erratum workarounds can deadlock system!\n" \
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> new file mode 100644
> index 000000000000..fac151580c01
> --- /dev/null
> +++ b/arch/arm64/kvm/rmi.c
> @@ -0,0 +1,57 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2023-2025 ARM Ltd.
> + */
> +
> +#include <linux/kvm_host.h>
> +
> +#include <asm/rmi_cmds.h>
> +#include <asm/virt.h>
> +
> +static int rmi_check_version(void)
> +{
> + struct arm_smccc_res res;
> + unsigned short version_major, version_minor;
> + unsigned long host_version = RMI_ABI_VERSION(RMI_ABI_MAJOR_VERSION,
> + RMI_ABI_MINOR_VERSION);
> + unsigned long aa64pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
> +
> + /* If RME isn't supported, then RMI can't be */
> + if (cpuid_feature_extract_unsigned_field(aa64pfr0, ID_AA64PFR0_EL1_RME_SHIFT) == 0)
> + return -ENXIO;
> +
> + arm_smccc_1_1_invoke(SMC_RMI_VERSION, host_version, &res);
> +
> + if (res.a0 == SMCCC_RET_NOT_SUPPORTED)
> + return -ENXIO;
> +
> + version_major = RMI_ABI_VERSION_GET_MAJOR(res.a1);
> + version_minor = RMI_ABI_VERSION_GET_MINOR(res.a1);
> +
> + if (res.a0 != RMI_SUCCESS) {
> + unsigned short high_version_major, high_version_minor;
> +
> + high_version_major = RMI_ABI_VERSION_GET_MAJOR(res.a2);
> + high_version_minor = RMI_ABI_VERSION_GET_MINOR(res.a2);
> +
> + kvm_err("Unsupported RMI ABI (v%d.%d - v%d.%d) we want v%d.%d\n",
> + version_major, version_minor,
> + high_version_major, high_version_minor,
> + RMI_ABI_MAJOR_VERSION,
> + RMI_ABI_MINOR_VERSION);
> + return -ENXIO;
> + }
> +
> + kvm_info("RMI ABI version %d.%d\n", version_major, version_minor);
> +
> + return 0;
> +}
> +
> +void kvm_init_rmi(void)
> +{
> + /* Continue without realm support if we can't agree on a version */
> + if (rmi_check_version())
> + return;
> +
> + /* Future patch will enable static branch kvm_rmi_is_available */
> +}
> --
> 2.43.0
>
^ permalink raw reply
* Re: [PATCH v13 17/48] arm64: RMI: Allocate/free RECs to match vCPUs
From: Wei-Lin Chang @ 2026-03-19 18:10 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <20260318155413.793430-18-steven.price@arm.com>
On Wed, Mar 18, 2026 at 03:53:41PM +0000, Steven Price wrote:
> The RMM maintains a data structure known as the Realm Execution Context
> (or REC). It is similar to struct kvm_vcpu and tracks the state of the
> virtual CPUs. KVM must delegate memory and request the structures are
> created when vCPUs are created, and suitably tear down on destruction.
>
> RECs must also be supplied with addition pages - auxiliary (or AUX)
> granules - for storing the larger registers state (e.g. for SVE). The
> number of AUX granules for a REC depends on the parameters with which
> the Realm was created - the RMM makes this information available via the
> RMI_REC_AUX_COUNT call performed after creating the Realm Descriptor (RD).
>
> Note that only some of register state for the REC can be set by KVM, the
> rest is defined by the RMM (zeroed). The register state then cannot be
> changed by KVM after the REC is created (except when the guest
> explicitly requests this e.g. by performing a PSCI call).
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v12:
> * Use the new range-based delegation RMI.
> Changes since v11:
> * Remove the KVM_ARM_VCPU_REC feature. User space no longer needs to
> configure each VCPU separately, RECs are created on the first VCPU
> run of the guest.
> Changes since v9:
> * Size the aux_pages array according to the PAGE_SIZE of the host.
> Changes since v7:
> * Add comment explaining the aux_pages array.
> * Rename "undeleted_failed" variable to "should_free" to avoid a
> confusing double negative.
> Changes since v6:
> * Avoid reporting the KVM_ARM_VCPU_REC feature if the guest isn't a
> realm guest.
> * Support host page size being larger than RMM's granule size when
> allocating/freeing aux granules.
> Changes since v5:
> * Separate the concept of vcpu_is_rec() and
> kvm_arm_vcpu_rec_finalized() by using the KVM_ARM_VCPU_REC feature as
> the indication that the VCPU is a REC.
> Changes since v2:
> * Free rec->run earlier in kvm_destroy_realm() and adapt to previous patches.
> ---
> arch/arm64/include/asm/kvm_emulate.h | 2 +-
> arch/arm64/include/asm/kvm_host.h | 3 +
> arch/arm64/include/asm/kvm_rmi.h | 21 +++
> arch/arm64/kvm/arm.c | 10 +-
> arch/arm64/kvm/reset.c | 1 +
> arch/arm64/kvm/rmi.c | 196 +++++++++++++++++++++++++++
> 6 files changed, 230 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 39310d9b4e16..d194d91fbc2a 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -708,7 +708,7 @@ static inline bool kvm_realm_is_created(struct kvm *kvm)
>
> static inline bool vcpu_is_rec(struct kvm_vcpu *vcpu)
> {
> - return false;
> + return kvm_is_realm(vcpu->kvm);
> }
>
> #endif /* __ARM64_KVM_EMULATE_H__ */
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 9267a2f2d65b..64304848aad4 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -924,6 +924,9 @@ struct kvm_vcpu_arch {
>
> /* Per-vcpu TLB for VNCR_EL2 -- NULL when !NV */
> struct vncr_tlb *vncr_tlb;
> +
> + /* Realm meta data */
> + struct realm_rec rec;
> };
>
> /*
> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
> index 6c13847480f7..4e2c61e71a38 100644
> --- a/arch/arm64/include/asm/kvm_rmi.h
> +++ b/arch/arm64/include/asm/kvm_rmi.h
> @@ -63,6 +63,26 @@ struct realm {
> unsigned int ia_bits;
> };
>
> +/**
> + * struct realm_rec - Additional per VCPU data for a Realm
> + *
> + * @mpidr: MPIDR (Multiprocessor Affinity Register) value to identify this VCPU
> + * @rec_page: Kernel VA of the RMM's private page for this REC
> + * @aux_pages: Additional pages private to the RMM for this REC
> + * @run: Kernel VA of the RmiRecRun structure shared with the RMM
> + */
> +struct realm_rec {
> + unsigned long mpidr;
> + void *rec_page;
> + /*
> + * REC_PARAMS_AUX_GRANULES is the maximum number of 4K granules that
> + * the RMM can require. The array is sized to be large enough for the
> + * maximum number of host sized pages that could be required.
> + */
> + struct page *aux_pages[(REC_PARAMS_AUX_GRANULES * SZ_4K) >> PAGE_SHIFT];
> + struct rec_run *run;
> +};
> +
> void kvm_init_rmi(void);
> u32 kvm_realm_ipa_limit(void);
>
> @@ -70,6 +90,7 @@ int kvm_init_realm_vm(struct kvm *kvm);
> int kvm_activate_realm(struct kvm *kvm);
> void kvm_destroy_realm(struct kvm *kvm);
> void kvm_realm_destroy_rtts(struct kvm *kvm);
> +void kvm_destroy_rec(struct kvm_vcpu *vcpu);
>
> static inline bool kvm_realm_is_private_address(struct realm *realm,
> unsigned long addr)
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index c8e51ed009c0..8c50ebd9fba0 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -575,6 +575,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
> /* Force users to call KVM_ARM_VCPU_INIT */
> vcpu_clear_flag(vcpu, VCPU_INITIALIZED);
>
> + vcpu->arch.rec.mpidr = INVALID_HWID;
> +
> vcpu->arch.mmu_page_cache.gfp_zero = __GFP_ZERO;
>
> /* Set up the timer */
> @@ -1549,7 +1551,7 @@ int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_level,
> return -EINVAL;
> }
>
> -static unsigned long system_supported_vcpu_features(void)
> +static unsigned long system_supported_vcpu_features(struct kvm *kvm)
> {
> unsigned long features = KVM_VCPU_VALID_FEATURES;
>
> @@ -1587,7 +1589,7 @@ static int kvm_vcpu_init_check_features(struct kvm_vcpu *vcpu,
> return -ENOENT;
> }
>
> - if (features & ~system_supported_vcpu_features())
> + if (features & ~system_supported_vcpu_features(vcpu->kvm))
Hi,
Are these two hunks superfluous?
Thanks,
Wei-Lin Chang
> return -EINVAL;
>
> /*
> @@ -1609,6 +1611,10 @@ static int kvm_vcpu_init_check_features(struct kvm_vcpu *vcpu,
> if (test_bit(KVM_ARM_VCPU_HAS_EL2, &features))
> return -EINVAL;
>
> + /* Realms are incompatible with AArch32 */
> + if (vcpu_is_rec(vcpu))
> + return -EINVAL;
> +
> return 0;
> }
>
> diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
> index 959532422d3a..4bbf58892928 100644
> --- a/arch/arm64/kvm/reset.c
> +++ b/arch/arm64/kvm/reset.c
> @@ -161,6 +161,7 @@ void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu)
> free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
> kfree(vcpu->arch.vncr_tlb);
> kfree(vcpu->arch.ccsidr);
> + kvm_destroy_rec(vcpu);
> }
>
> static void kvm_vcpu_reset_sve(struct kvm_vcpu *vcpu)
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> index 937fababf960..6daf14c4b413 100644
> --- a/arch/arm64/kvm/rmi.c
> +++ b/arch/arm64/kvm/rmi.c
> @@ -207,6 +207,28 @@ static int get_start_level(struct realm *realm)
> return 4 - stage2_pgtable_levels(realm->ia_bits);
> }
>
> +static int delegate_range(phys_addr_t phys, unsigned long size)
> +{
> + unsigned long ret;
> + unsigned long top = phys + size;
> + unsigned long out_top;
> +
> + while (phys < top) {
> + ret = rmi_granule_range_delegate(phys, top, &out_top);
> + if (ret == RMI_SUCCESS)
> + phys = out_top;
> + else if (ret != RMI_BUSY && ret != RMI_BLOCKED)
> + return ret;
> + }
> +
> + return ret;
> +}
> +
> +static int delegate_page(phys_addr_t phys)
> +{
> + return delegate_range(phys, PAGE_SIZE);
> +}
> +
> static int undelegate_range(phys_addr_t phys, unsigned long size)
> {
> unsigned long ret;
> @@ -372,9 +394,177 @@ static int realm_ensure_created(struct kvm *kvm)
> return -ENXIO;
> }
>
> +static void free_rec_aux(struct page **aux_pages,
> + unsigned int num_aux)
> +{
> + unsigned int i;
> + unsigned int page_count = 0;
> +
> + for (i = 0; i < num_aux; i++) {
> + struct page *aux_page = aux_pages[page_count++];
> + phys_addr_t aux_page_phys = page_to_phys(aux_page);
> +
> + if (!WARN_ON(undelegate_page(aux_page_phys)))
> + __free_page(aux_page);
> + aux_page_phys += PAGE_SIZE;
> + }
> +}
> +
> +static int alloc_rec_aux(struct page **aux_pages,
> + u64 *aux_phys_pages,
> + unsigned int num_aux)
> +{
> + struct page *aux_page;
> + unsigned int i;
> + int ret;
> +
> + for (i = 0; i < num_aux; i++) {
> + phys_addr_t aux_page_phys;
> +
> + aux_page = alloc_page(GFP_KERNEL);
> + if (!aux_page) {
> + ret = -ENOMEM;
> + goto out_err;
> + }
> +
> + aux_page_phys = page_to_phys(aux_page);
> + if (delegate_page(aux_page_phys)) {
> + ret = -ENXIO;
> + goto err_undelegate;
> + }
> + aux_phys_pages[i] = aux_page_phys;
> + aux_pages[i] = aux_page;
> + }
> +
> + return 0;
> +err_undelegate:
> + while (i > 0) {
> + i--;
> + if (WARN_ON(undelegate_page(aux_phys_pages[i]))) {
> + /* Leak the page if the undelegate fails */
> + goto out_err;
> + }
> + }
> + __free_page(aux_page);
> +out_err:
> + free_rec_aux(aux_pages, i);
> + return ret;
> +}
> +
> +static int kvm_create_rec(struct kvm_vcpu *vcpu)
> +{
> + struct user_pt_regs *vcpu_regs = vcpu_gp_regs(vcpu);
> + unsigned long mpidr = kvm_vcpu_get_mpidr_aff(vcpu);
> + struct realm *realm = &vcpu->kvm->arch.realm;
> + struct realm_rec *rec = &vcpu->arch.rec;
> + unsigned long rec_page_phys;
> + struct rec_params *params;
> + int r, i;
> +
> + if (rec->run)
> + return -EBUSY;
> +
> + /*
> + * The RMM will report PSCI v1.0 to Realms and the KVM_ARM_VCPU_PSCI_0_2
> + * flag covers v0.2 and onwards.
> + */
> + if (!vcpu_has_feature(vcpu, KVM_ARM_VCPU_PSCI_0_2))
> + return -EINVAL;
> +
> + BUILD_BUG_ON(sizeof(*params) > PAGE_SIZE);
> + BUILD_BUG_ON(sizeof(*rec->run) > PAGE_SIZE);
> +
> + params = (struct rec_params *)get_zeroed_page(GFP_KERNEL);
> + rec->rec_page = (void *)__get_free_page(GFP_KERNEL);
> + rec->run = (void *)get_zeroed_page(GFP_KERNEL);
> + if (!params || !rec->rec_page || !rec->run) {
> + r = -ENOMEM;
> + goto out_free_pages;
> + }
> +
> + for (i = 0; i < ARRAY_SIZE(params->gprs); i++)
> + params->gprs[i] = vcpu_regs->regs[i];
> +
> + params->pc = vcpu_regs->pc;
> +
> + if (vcpu->vcpu_id == 0)
> + params->flags |= REC_PARAMS_FLAG_RUNNABLE;
> +
> + rec_page_phys = virt_to_phys(rec->rec_page);
> +
> + if (delegate_page(rec_page_phys)) {
> + r = -ENXIO;
> + goto out_free_pages;
> + }
> +
> + r = alloc_rec_aux(rec->aux_pages, params->aux, realm->num_aux);
> + if (r)
> + goto out_undelegate_rmm_rec;
> +
> + params->num_rec_aux = realm->num_aux;
> + params->mpidr = mpidr;
> +
> + if (rmi_rec_create(virt_to_phys(realm->rd),
> + rec_page_phys,
> + virt_to_phys(params))) {
> + r = -ENXIO;
> + goto out_free_rec_aux;
> + }
> +
> + rec->mpidr = mpidr;
> +
> + free_page((unsigned long)params);
> + return 0;
> +
> +out_free_rec_aux:
> + free_rec_aux(rec->aux_pages, realm->num_aux);
> +out_undelegate_rmm_rec:
> + if (WARN_ON(undelegate_page(rec_page_phys)))
> + rec->rec_page = NULL;
> +out_free_pages:
> + free_page((unsigned long)rec->run);
> + free_page((unsigned long)rec->rec_page);
> + free_page((unsigned long)params);
> + rec->run = NULL;
> + return r;
> +}
> +
> +void kvm_destroy_rec(struct kvm_vcpu *vcpu)
> +{
> + struct realm *realm = &vcpu->kvm->arch.realm;
> + struct realm_rec *rec = &vcpu->arch.rec;
> + unsigned long rec_page_phys;
> +
> + if (!vcpu_is_rec(vcpu))
> + return;
> +
> + if (!rec->run) {
> + /* Nothing to do if the VCPU hasn't been finalized */
> + return;
> + }
> +
> + free_page((unsigned long)rec->run);
> +
> + rec_page_phys = virt_to_phys(rec->rec_page);
> +
> + /*
> + * The REC and any AUX pages cannot be reclaimed until the REC is
> + * destroyed. So if the REC destroy fails then the REC page and any AUX
> + * pages will be leaked.
> + */
> + if (WARN_ON(rmi_rec_destroy(rec_page_phys)))
> + return;
> +
> + free_rec_aux(rec->aux_pages, realm->num_aux);
> +
> + free_delegated_page(rec_page_phys);
> +}
> +
> int kvm_activate_realm(struct kvm *kvm)
> {
> struct realm *realm = &kvm->arch.realm;
> + struct kvm_vcpu *vcpu;
> + unsigned long i;
> int ret;
>
> if (kvm_realm_state(kvm) >= REALM_STATE_ACTIVE)
> @@ -397,6 +587,12 @@ int kvm_activate_realm(struct kvm *kvm)
> /* Mark state as dead in case we fail */
> WRITE_ONCE(realm->state, REALM_STATE_DEAD);
>
> + kvm_for_each_vcpu(i, vcpu, kvm) {
> + ret = kvm_create_rec(vcpu);
> + if (ret)
> + return ret;
> + }
> +
> ret = rmi_realm_activate(virt_to_phys(realm->rd));
> if (ret)
> return -ENXIO;
> --
> 2.43.0
>
^ permalink raw reply
* Re: [PATCH v13 26/48] arm64: RMI: Create the realm descriptor
From: Wei-Lin Chang @ 2026-03-19 18:25 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <20260318155413.793430-27-steven.price@arm.com>
On Wed, Mar 18, 2026 at 03:53:50PM +0000, Steven Price wrote:
> Creating a realm involves first creating a realm descriptor (RD). This
> involves passing the configuration information to the RMM. Do this as
> part of realm_ensure_created() so that the realm is created when it is
> first needed.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v12:
> * Since RMM page size is now equal to the host's page size various
> calculations are simplified.
> * Switch to using range based APIs to delegate/undelegate.
> * VMID handling is now handled entirely by the RMM.
> ---
> arch/arm64/kvm/rmi.c | 94 +++++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 92 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> index 38349c7b34f4..d5fee203824b 100644
> --- a/arch/arm64/kvm/rmi.c
> +++ b/arch/arm64/kvm/rmi.c
> @@ -649,6 +649,83 @@ static void realm_unmap_shared_range(struct kvm *kvm,
> start, end);
> }
>
> +static int realm_create_rd(struct kvm *kvm)
> +{
> + struct realm *realm = &kvm->arch.realm;
> + struct realm_params *params = realm->params;
> + void *rd = NULL;
> + phys_addr_t rd_phys, params_phys;
> + size_t pgd_size = kvm_pgtable_stage2_pgd_size(kvm->arch.mmu.vtcr);
> + int i, r;
> +
> + realm->ia_bits = VTCR_EL2_IPA(kvm->arch.mmu.vtcr);
> +
> + if (WARN_ON(realm->rd || !realm->params))
> + return -EEXIST;
> +
> + rd = (void *)__get_free_page(GFP_KERNEL);
Hi,
Should this be GFP_KERNEL_ACCOUNT?
> + if (!rd)
> + return -ENOMEM;
> +
> + rd_phys = virt_to_phys(rd);
> + if (delegate_page(rd_phys)) {
> + r = -ENXIO;
> + goto free_rd;
> + }
> +
> + if (delegate_range(kvm->arch.mmu.pgd_phys, pgd_size)) {
> + r = -ENXIO;
> + goto out_undelegate_tables;
> + }
> +
> + params->s2sz = VTCR_EL2_IPA(kvm->arch.mmu.vtcr);
> + params->rtt_level_start = get_start_level(realm);
> + params->rtt_num_start = pgd_size / PAGE_SIZE;
> + params->rtt_base = kvm->arch.mmu.pgd_phys;
> +
> + if (kvm->arch.arm_pmu) {
> + params->pmu_num_ctrs = kvm->arch.nr_pmu_counters;
> + params->flags |= RMI_REALM_PARAM_FLAG_PMU;
> + }
> +
> + if (kvm_lpa2_is_enabled())
> + params->flags |= RMI_REALM_PARAM_FLAG_LPA2;
> +
> + params_phys = virt_to_phys(params);
> +
> + if (rmi_realm_create(rd_phys, params_phys)) {
> + r = -ENXIO;
> + goto out_undelegate_tables;
> + }
> +
> + if (WARN_ON(rmi_rec_aux_count(rd_phys, &realm->num_aux))) {
> + WARN_ON(rmi_realm_destroy(rd_phys));
> + r = -ENXIO;
> + goto out_undelegate_tables;
> + }
> +
> + realm->rd = rd;
> + WRITE_ONCE(realm->state, REALM_STATE_NEW);
> + /* The realm is up, free the parameters. */
> + free_page((unsigned long)realm->params);
> + realm->params = NULL;
> +
> + return 0;
> +
> +out_undelegate_tables:
> + if (WARN_ON(undelegate_range(kvm->arch.mmu.pgd_phys, i))) {
> + /* Leak the pages if they cannot be returned */
> + kvm->arch.mmu.pgt = NULL;
Did you mean kvm->arch.mmu.pgd_phys = NULL; ?
Thanks,
Wei-Lin Chang
> + }
> + if (WARN_ON(undelegate_page(rd_phys))) {
> + /* Leak the page if it isn't returned */
> + return r;
> + }
> +free_rd:
> + free_page((unsigned long)rd);
> + return r;
> +}
> +
> static void realm_unmap_private_range(struct kvm *kvm,
> unsigned long start,
> unsigned long end,
> @@ -893,8 +970,21 @@ static int realm_init_ipa_state(struct kvm *kvm,
>
> static int realm_ensure_created(struct kvm *kvm)
> {
> - /* Provided in later patch */
> - return -ENXIO;
> + int ret;
> +
> + switch (kvm_realm_state(kvm)) {
> + case REALM_STATE_NONE:
> + break;
> + case REALM_STATE_NEW:
> + return 0;
> + case REALM_STATE_DEAD:
> + return -ENXIO;
> + default:
> + return -EBUSY;
> + }
> +
> + ret = realm_create_rd(kvm);
> + return ret;
> }
>
> static int set_ripas_of_protected_regions(struct kvm *kvm)
> --
> 2.43.0
>
^ permalink raw reply
* Re: [PATCH v13 27/48] arm64: RMI: Runtime faulting of memory
From: Wei-Lin Chang @ 2026-03-19 18:41 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <20260318155413.793430-28-steven.price@arm.com>
On Wed, Mar 18, 2026 at 03:53:51PM +0000, Steven Price wrote:
> At runtime if the realm guest accesses memory which hasn't yet been
> mapped then KVM needs to either populate the region or fault the guest.
>
> For memory in the lower (protected) region of IPA a fresh page is
> provided to the RMM which will zero the contents. For memory in the
> upper (shared) region of IPA, the memory from the memslot is mapped
> into the realm VM non secure.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v12:
> * Switch to RMM v2.0 range based APIs.
> Changes since v11:
> * Adapt to upstream changes.
> Changes since v10:
> * RME->RMI renaming.
> * Adapt to upstream gmem changes.
> Changes since v9:
> * Fix call to kvm_stage2_unmap_range() in kvm_free_stage2_pgd() to set
> may_block to avoid stall warnings.
> * Minor coding style fixes.
> Changes since v8:
> * Propagate the may_block flag.
> * Minor comments and coding style changes.
> Changes since v7:
> * Remove redundant WARN_ONs for realm_create_rtt_levels() - it will
> internally WARN when necessary.
> Changes since v6:
> * Handle PAGE_SIZE being larger than RMM granule size.
> * Some minor renaming following review comments.
> Changes since v5:
> * Reduce use of struct page in preparation for supporting the RMM
> having a different page size to the host.
> * Handle a race when delegating a page where another CPU has faulted on
> a the same page (and already delegated the physical page) but not yet
> mapped it. In this case simply return to the guest to either use the
> mapping from the other CPU (or refault if the race is lost).
> * The changes to populate_par_region() are moved into the previous
> patch where they belong.
> Changes since v4:
> * Code cleanup following review feedback.
> * Drop the PTE_SHARED bit when creating unprotected page table entries.
> This is now set by the RMM and the host has no control of it and the
> spec requires the bit to be set to zero.
> Changes since v2:
> * Avoid leaking memory if failing to map it in the realm.
> * Correctly mask RTT based on LPA2 flag (see rtt_get_phys()).
> * Adapt to changes in previous patches.
> ---
> arch/arm64/include/asm/kvm_emulate.h | 8 ++
> arch/arm64/include/asm/kvm_rmi.h | 12 ++
> arch/arm64/kvm/mmu.c | 139 ++++++++++++++++--
> arch/arm64/kvm/rmi.c | 206 +++++++++++++++++++++++++++
> 4 files changed, 351 insertions(+), 14 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index d194d91fbc2a..0734c4a65174 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -706,6 +706,14 @@ static inline bool kvm_realm_is_created(struct kvm *kvm)
> return kvm_is_realm(kvm) && kvm_realm_state(kvm) != REALM_STATE_NONE;
> }
>
> +static inline gpa_t kvm_gpa_from_fault(struct kvm *kvm, phys_addr_t ipa)
I think we should use gpa_t as the type for ipa.
> +{
> + if (!kvm_is_realm(kvm))
> + return ipa;
> +
> + return ipa & ~BIT(kvm->arch.realm.ia_bits - 1);
> +}
> +
> static inline bool vcpu_is_rec(struct kvm_vcpu *vcpu)
> {
> return kvm_is_realm(vcpu->kvm);
> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
> index bf663bb240c4..38208be3c602 100644
> --- a/arch/arm64/include/asm/kvm_rmi.h
> +++ b/arch/arm64/include/asm/kvm_rmi.h
> @@ -6,6 +6,7 @@
> #ifndef __ASM_KVM_RMI_H
> #define __ASM_KVM_RMI_H
>
> +#include <asm/kvm_pgtable.h>
> #include <asm/rmi_smc.h>
>
> /**
> @@ -105,6 +106,17 @@ void kvm_realm_unmap_range(struct kvm *kvm,
> unsigned long size,
> bool unmap_private,
> bool may_block);
> +int realm_map_protected(struct kvm *kvm,
> + unsigned long base_ipa,
> + kvm_pfn_t pfn,
> + unsigned long size,
> + struct kvm_mmu_memory_cache *memcache);
> +int realm_map_non_secure(struct realm *realm,
> + unsigned long ipa,
> + kvm_pfn_t pfn,
> + unsigned long size,
> + enum kvm_pgtable_prot prot,
> + struct kvm_mmu_memory_cache *memcache);
>
> static inline bool kvm_realm_is_private_address(struct realm *realm,
> unsigned long addr)
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index bad93938acdb..73c18c2861a2 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -334,8 +334,15 @@ static void __unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64
>
> lockdep_assert_held_write(&kvm->mmu_lock);
> WARN_ON(size & ~PAGE_MASK);
> - WARN_ON(stage2_apply_range(mmu, start, end, KVM_PGT_FN(kvm_pgtable_stage2_unmap),
> - may_block));
> +
> + if (kvm_is_realm(kvm)) {
> + kvm_realm_unmap_range(kvm, start, size, !only_shared,
> + may_block);
> + } else {
> + WARN_ON(stage2_apply_range(mmu, start, end,
> + KVM_PGT_FN(kvm_pgtable_stage2_unmap),
> + may_block));
> + }
> }
>
> void kvm_stage2_unmap_range(struct kvm_s2_mmu *mmu, phys_addr_t start,
> @@ -355,7 +362,10 @@ static void stage2_flush_memslot(struct kvm *kvm,
> phys_addr_t addr = memslot->base_gfn << PAGE_SHIFT;
> phys_addr_t end = addr + PAGE_SIZE * memslot->npages;
>
> - kvm_stage2_flush_range(&kvm->arch.mmu, addr, end);
> + if (kvm_is_realm(kvm))
> + kvm_realm_unmap_range(kvm, addr, end - addr, false, true);
> + else
> + kvm_stage2_flush_range(&kvm->arch.mmu, addr, end);
> }
>
> /**
> @@ -1081,6 +1091,10 @@ void stage2_unmap_vm(struct kvm *kvm)
> struct kvm_memory_slot *memslot;
> int idx, bkt;
>
> + /* For realms this is handled by the RMM so nothing to do here */
> + if (kvm_is_realm(kvm))
> + return;
> +
> idx = srcu_read_lock(&kvm->srcu);
> mmap_read_lock(current->mm);
> write_lock(&kvm->mmu_lock);
> @@ -1106,6 +1120,9 @@ void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
> if (kvm_is_realm(kvm) &&
> (kvm_realm_state(kvm) != REALM_STATE_DEAD &&
> kvm_realm_state(kvm) != REALM_STATE_NONE)) {
> + struct realm *realm = &kvm->arch.realm;
> +
> + kvm_stage2_unmap_range(mmu, 0, BIT(realm->ia_bits - 1), true);
> write_unlock(&kvm->mmu_lock);
> kvm_realm_destroy_rtts(kvm);
>
> @@ -1516,6 +1533,29 @@ static bool kvm_vma_mte_allowed(struct vm_area_struct *vma)
> return vma->vm_flags & VM_MTE_ALLOWED;
> }
>
> +static int realm_map_ipa(struct kvm *kvm, phys_addr_t ipa,
> + kvm_pfn_t pfn, unsigned long map_size,
> + enum kvm_pgtable_prot prot,
> + struct kvm_mmu_memory_cache *memcache)
> +{
> + struct realm *realm = &kvm->arch.realm;
> +
> + /*
> + * Write permission is required for now even though it's possible to
> + * map unprotected pages (granules) as read-only. It's impossible to
> + * map protected pages (granules) as read-only.
> + */
> + if (WARN_ON(!(prot & KVM_PGTABLE_PROT_W)))
> + return -EFAULT;
> +
> + ipa = ALIGN_DOWN(ipa, PAGE_SIZE);
> + if (!kvm_realm_is_private_address(realm, ipa))
> + return realm_map_non_secure(realm, ipa, pfn, map_size, prot,
> + memcache);
> +
> + return realm_map_protected(kvm, ipa, pfn, map_size, memcache);
> +}
> +
> static bool kvm_vma_is_cacheable(struct vm_area_struct *vma)
> {
> switch (FIELD_GET(PTE_ATTRINDX_MASK, pgprot_val(vma->vm_page_prot))) {
> @@ -1588,6 +1628,7 @@ static int gmem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_SHARED;
> enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
> struct kvm_pgtable *pgt = vcpu->arch.hw_mmu->pgt;
> + gpa_t gpa = kvm_gpa_from_fault(vcpu->kvm, fault_ipa);
> unsigned long mmu_seq;
> struct page *page;
> struct kvm *kvm = vcpu->kvm;
> @@ -1596,6 +1637,29 @@ static int gmem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> gfn_t gfn;
> int ret;
>
> + if (kvm_is_realm(vcpu->kvm)) {
> + /* check for memory attribute mismatch */
> + bool is_priv_gfn = kvm_mem_is_private(kvm, gpa >> PAGE_SHIFT);
> + /*
> + * For Realms, the shared address is an alias of the private
> + * PA with the top bit set. Thus is the fault address matches
Hi,
Thus is -> Thus if.
> + * the GPA then it is the private alias.
> + */
> + bool is_priv_fault = (gpa == fault_ipa);
> +
> + if (is_priv_gfn != is_priv_fault) {
> + kvm_prepare_memory_fault_exit(vcpu, gpa, PAGE_SIZE,
> + kvm_is_write_fault(vcpu),
> + false,
> + is_priv_fault);
> + /*
> + * KVM_EXIT_MEMORY_FAULT requires an return code of
> + * -EFAULT, see the API documentation
> + */
> + return -EFAULT;
> + }
> + }
> +
> ret = prepare_mmu_memcache(vcpu, true, &memcache);
> if (ret)
> return ret;
> @@ -1603,7 +1667,7 @@ static int gmem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> if (nested)
> gfn = kvm_s2_trans_output(nested) >> PAGE_SHIFT;
> else
> - gfn = fault_ipa >> PAGE_SHIFT;
> + gfn = gpa >> PAGE_SHIFT;
>
> write_fault = kvm_is_write_fault(vcpu);
> exec_fault = kvm_vcpu_trap_is_exec_fault(vcpu);
> @@ -1616,7 +1680,7 @@ static int gmem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
>
> ret = kvm_gmem_get_pfn(kvm, memslot, gfn, &pfn, &page, NULL);
> if (ret) {
> - kvm_prepare_memory_fault_exit(vcpu, fault_ipa, PAGE_SIZE,
> + kvm_prepare_memory_fault_exit(vcpu, gpa, PAGE_SIZE,
> write_fault, exec_fault, false);
> return ret;
> }
> @@ -1638,15 +1702,25 @@ static int gmem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> kvm_fault_lock(kvm);
> if (mmu_invalidate_retry(kvm, mmu_seq)) {
> ret = -EAGAIN;
> - goto out_unlock;
> + goto out_release_page;
> + }
> +
> + if (kvm_is_realm(kvm)) {
> + ret = realm_map_ipa(kvm, fault_ipa, pfn,
> + PAGE_SIZE, KVM_PGTABLE_PROT_W, memcache);
> + /* if successful don't release the page */
> + if (!ret)
> + goto out_unlock;
> + goto out_release_page;
> }
>
> ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, fault_ipa, PAGE_SIZE,
> __pfn_to_phys(pfn), prot,
> memcache, flags);
>
> -out_unlock:
> +out_release_page:
> kvm_release_faultin_page(kvm, page, !!ret, writable);
> +out_unlock:
> kvm_fault_unlock(kvm);
>
> if (writable && !ret)
> @@ -1685,6 +1759,14 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> if (fault_is_perm)
> fault_granule = kvm_vcpu_trap_get_perm_fault_granule(vcpu);
> write_fault = kvm_is_write_fault(vcpu);
> +
> + /*
> + * Realms cannot map protected pages read-only
> + * FIXME: It should be possible to map unprotected pages read-only
> + */
> + if (vcpu_is_rec(vcpu))
> + write_fault = true;
> +
> exec_fault = kvm_vcpu_trap_is_exec_fault(vcpu);
> VM_WARN_ON_ONCE(write_fault && exec_fault);
>
> @@ -1779,7 +1861,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> ipa &= ~(vma_pagesize - 1);
> }
>
> - gfn = ipa >> PAGE_SHIFT;
> + gfn = kvm_gpa_from_fault(kvm, ipa) >> PAGE_SHIFT;
> mte_allowed = kvm_vma_mte_allowed(vma);
>
> vfio_allow_any_uc = vma->vm_flags & VM_ALLOW_ANY_UNCACHED;
> @@ -1855,6 +1937,15 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> if (exec_fault && s2_force_noncacheable)
> ret = -ENOEXEC;
>
> + /*
> + * For now we shouldn't be hitting protected addresses because they are
> + * handled in gmem_abort(). In the future this check may be relaxed to
> + * support e.g. protected devices.
> + */
> + if (!ret && vcpu_is_rec(vcpu) &&
> + kvm_gpa_from_fault(kvm, fault_ipa) == fault_ipa)
Maybe use !shared_ipa_fault() here?
Thanks,
Wei-Lin Chang
> + ret = -EINVAL;
> +
> if (ret) {
> kvm_release_page_unused(page);
> return ret;
> @@ -1939,6 +2030,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> */
> prot &= ~KVM_NV_GUEST_MAP_SZ;
> ret = KVM_PGT_FN(kvm_pgtable_stage2_relax_perms)(pgt, fault_ipa, prot, flags);
> + } else if (kvm_is_realm(kvm)) {
> + ret = realm_map_ipa(kvm, fault_ipa, pfn, vma_pagesize,
> + prot, memcache);
> } else {
> ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, fault_ipa, vma_pagesize,
> __pfn_to_phys(pfn), prot,
> @@ -2049,6 +2143,13 @@ int kvm_handle_guest_sea(struct kvm_vcpu *vcpu)
> return 0;
> }
>
> +static bool shared_ipa_fault(struct kvm *kvm, phys_addr_t fault_ipa)
> +{
> + gpa_t gpa = kvm_gpa_from_fault(kvm, fault_ipa);
> +
> + return (gpa != fault_ipa);
> +}
> +
> /**
> * kvm_handle_guest_abort - handles all 2nd stage aborts
> * @vcpu: the VCPU pointer
> @@ -2159,8 +2260,9 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
> nested = &nested_trans;
> }
>
> - gfn = ipa >> PAGE_SHIFT;
> + gfn = kvm_gpa_from_fault(vcpu->kvm, ipa) >> PAGE_SHIFT;
> memslot = gfn_to_memslot(vcpu->kvm, gfn);
> +
> hva = gfn_to_hva_memslot_prot(memslot, gfn, &writable);
> write_fault = kvm_is_write_fault(vcpu);
> if (kvm_is_error_hva(hva) || (write_fault && !writable)) {
> @@ -2203,7 +2305,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
> * of the page size.
> */
> ipa |= FAR_TO_FIPA_OFFSET(kvm_vcpu_get_hfar(vcpu));
> - ret = io_mem_abort(vcpu, ipa);
> + ret = io_mem_abort(vcpu, kvm_gpa_from_fault(vcpu->kvm, ipa));
> goto out_unlock;
> }
>
> @@ -2219,7 +2321,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
> VM_WARN_ON_ONCE(kvm_vcpu_trap_is_permission_fault(vcpu) &&
> !write_fault && !kvm_vcpu_trap_is_exec_fault(vcpu));
>
> - if (kvm_slot_has_gmem(memslot))
> + if (kvm_slot_has_gmem(memslot) && !shared_ipa_fault(vcpu->kvm, fault_ipa))
> ret = gmem_abort(vcpu, fault_ipa, nested, memslot,
> esr_fsc_is_permission_fault(esr));
> else
> @@ -2256,6 +2358,10 @@ bool kvm_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
> if (!kvm->arch.mmu.pgt)
> return false;
>
> + /* We don't support aging for Realms */
> + if (kvm_is_realm(kvm))
> + return true;
> +
> return KVM_PGT_FN(kvm_pgtable_stage2_test_clear_young)(kvm->arch.mmu.pgt,
> range->start << PAGE_SHIFT,
> size, true);
> @@ -2272,6 +2378,10 @@ bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
> if (!kvm->arch.mmu.pgt)
> return false;
>
> + /* We don't support aging for Realms */
> + if (kvm_is_realm(kvm))
> + return true;
> +
> return KVM_PGT_FN(kvm_pgtable_stage2_test_clear_young)(kvm->arch.mmu.pgt,
> range->start << PAGE_SHIFT,
> size, false);
> @@ -2438,10 +2548,11 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
> return -EFAULT;
>
> /*
> - * Only support guest_memfd backed memslots with mappable memory, since
> - * there aren't any CoCo VMs that support only private memory on arm64.
> + * Only support guest_memfd backed memslots with mappable memory,
> + * unless the guest is a CCA realm guest.
> */
> - if (kvm_slot_has_gmem(new) && !kvm_memslot_is_gmem_only(new))
> + if (kvm_slot_has_gmem(new) && !kvm_memslot_is_gmem_only(new) &&
> + !kvm_is_realm(kvm))
> return -EINVAL;
>
> hva = new->userspace_addr;
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> index d5fee203824b..30292814b1ec 100644
> --- a/arch/arm64/kvm/rmi.c
> +++ b/arch/arm64/kvm/rmi.c
> @@ -837,6 +837,212 @@ static int realm_create_protected_data_page(struct kvm *kvm,
> return ret;
> }
>
> +static int fold_rtt(struct realm *realm, unsigned long addr, int level)
> +{
> + phys_addr_t rtt_addr;
> + int ret;
> +
> + ret = realm_rtt_fold(realm, addr, level, &rtt_addr);
> + if (ret)
> + return ret;
> +
> + free_rtt(rtt_addr);
> +
> + return 0;
> +}
> +
> +static unsigned long addr_range_desc(unsigned long phys, unsigned long size)
> +{
> + unsigned long out = 0;
> +
> + switch (size) {
> + case P4D_SIZE:
> + out = 0 | (1 << 2);
> + break;
> + case PUD_SIZE:
> + out = 1 | (1 << 2);
> + break;
> + case PMD_SIZE:
> + out = 2 | (1 << 2);
> + break;
> + case PAGE_SIZE:
> + out = 3 | (1 << 2);
> + break;
> + default:
> + /*
> + * Only support mapping at the page level granulatity when
> + * it's an unusual length. This should get us back onto a larger
> + * block size for the subsequent mappings.
> + */
> + out = 3 | ((MIN(size >> PAGE_SHIFT, PTRS_PER_PTE - 1)) << 2);
> + break;
> + }
> +
> + WARN_ON(phys & ~PAGE_MASK);
> +
> + out |= phys & PAGE_MASK;
> +
> + return out;
> +}
> +
> +int realm_map_protected(struct kvm *kvm,
> + unsigned long ipa,
> + kvm_pfn_t pfn,
> + unsigned long map_size,
> + struct kvm_mmu_memory_cache *memcache)
> +{
> + struct realm *realm = &kvm->arch.realm;
> + phys_addr_t phys = __pfn_to_phys(pfn);
> + phys_addr_t rd = virt_to_phys(realm->rd);
> + unsigned long base_ipa = ipa;
> + unsigned long ipa_top = ipa + map_size;
> + int map_level = IS_ALIGNED(map_size, RMM_L2_BLOCK_SIZE) ?
> + RMM_RTT_BLOCK_LEVEL : RMM_RTT_MAX_LEVEL;
> + int ret = 0;
> +
> + if (WARN_ON(!IS_ALIGNED(map_size, PAGE_SIZE) ||
> + !IS_ALIGNED(ipa, map_size)))
> + return -EINVAL;
> +
> + if (map_level < RMM_RTT_MAX_LEVEL) {
> + /*
> + * A temporary RTT is needed during the map, precreate it,
> + * however if there is an error (e.g. missing parent tables)
> + * this will be handled below.
> + */
> + realm_create_rtt_levels(realm, ipa, map_level,
> + RMM_RTT_MAX_LEVEL, memcache);
> + }
> +
> + if (delegate_range(phys, map_size)) {
> + /*
> + * It's likely we raced with another VCPU on the same
> + * fault. Assume the other VCPU has handled the fault
> + * and return to the guest.
> + */
> + return 0;
> + }
> +
> + while (ipa < ipa_top) {
> + unsigned long flags = RMI_ADDR_TYPE_SINGLE;
> + unsigned long range_desc = addr_range_desc(phys, ipa_top - ipa);
> + unsigned long out_top;
> +
> + ret = rmi_rtt_data_map(rd, ipa, ipa_top, flags, range_desc,
> + &out_top);
> +
> + if (RMI_RETURN_STATUS(ret) == RMI_ERROR_RTT) {
> + /* Create missing RTTs and retry */
> + int level = RMI_RETURN_INDEX(ret);
> +
> + WARN_ON(level == RMM_RTT_MAX_LEVEL);
> + ret = realm_create_rtt_levels(realm, ipa, level,
> + RMM_RTT_MAX_LEVEL,
> + memcache);
> + if (ret)
> + goto err_undelegate;
> +
> + ret = rmi_rtt_data_map(rd, ipa, ipa_top, flags,
> + range_desc, &out_top);
> + }
> +
> + if (WARN_ON(ret))
> + goto err_undelegate;
> +
> + phys += out_top - ipa;
> + ipa = out_top;
> + }
> +
> + if (map_size == RMM_L2_BLOCK_SIZE) {
> + ret = fold_rtt(realm, base_ipa, map_level + 1);
> + if (WARN_ON(ret))
> + goto err;
> + }
> +
> + return 0;
> +
> +err_undelegate:
> + if (WARN_ON(undelegate_range(phys, map_size))) {
> + /* Page can't be returned to NS world so is lost */
> + get_page(phys_to_page(phys));
> + }
> +err:
> + realm_unmap_private_range(kvm, base_ipa, ipa, true);
> + return -ENXIO;
> +}
> +
> +int realm_map_non_secure(struct realm *realm,
> + unsigned long ipa,
> + kvm_pfn_t pfn,
> + unsigned long size,
> + enum kvm_pgtable_prot prot,
> + struct kvm_mmu_memory_cache *memcache)
> +{
> + unsigned long attr;
> + phys_addr_t rd = virt_to_phys(realm->rd);
> + phys_addr_t phys = __pfn_to_phys(pfn);
> + unsigned long offset;
> + /* TODO: Support block mappings */
> + int map_level = RMM_RTT_MAX_LEVEL;
> + int map_size = rmi_rtt_level_mapsize(map_level);
> + int ret = 0;
> +
> + if (WARN_ON(!IS_ALIGNED(size, PAGE_SIZE) ||
> + !IS_ALIGNED(ipa, size)))
> + return -EINVAL;
> +
> + switch (prot & (KVM_PGTABLE_PROT_DEVICE | KVM_PGTABLE_PROT_NORMAL_NC)) {
> + case KVM_PGTABLE_PROT_DEVICE | KVM_PGTABLE_PROT_NORMAL_NC:
> + return -EINVAL;
> + case KVM_PGTABLE_PROT_DEVICE:
> + attr = PTE_S2_MEMATTR(MT_S2_FWB_DEVICE_nGnRE);
> + break;
> + case KVM_PGTABLE_PROT_NORMAL_NC:
> + attr = PTE_S2_MEMATTR(MT_S2_FWB_NORMAL_NC);
> + break;
> + default:
> + attr = PTE_S2_MEMATTR(MT_S2_FWB_NORMAL);
> + }
> +
> + for (offset = 0; offset < size; offset += map_size) {
> + /*
> + * realm_map_ipa() enforces that the memory is writable,
> + * so for now we permit both read and write.
> + */
> + unsigned long desc = kvm_phys_to_pte(phys) | attr |
> + KVM_PTE_LEAF_ATTR_LO_S2_S2AP_R |
> + KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W;
> + ret = rmi_rtt_map_unprotected(rd, ipa, map_level, desc);
> +
> + if (RMI_RETURN_STATUS(ret) == RMI_ERROR_RTT) {
> + /* Create missing RTTs and retry */
> + int level = RMI_RETURN_INDEX(ret);
> +
> + ret = realm_create_rtt_levels(realm, ipa, level,
> + map_level, memcache);
> + if (ret)
> + return -ENXIO;
> +
> + ret = rmi_rtt_map_unprotected(rd, ipa, map_level, desc);
> + }
> + /*
> + * RMI_ERROR_RTT can be reported for two reasons: either the
> + * RTT tables are not there, or there is an RTTE already
> + * present for the address. The above call to create RTTs
> + * handles the first case, and in the second case this
> + * indicates that another thread has already populated the RTTE
> + * for us, so we can ignore the error and continue.
> + */
> + if (ret && RMI_RETURN_STATUS(ret) != RMI_ERROR_RTT)
> + return -ENXIO;
> +
> + ipa += map_size;
> + phys += map_size;
> + }
> +
> + return 0;
> +}
> +
> static int populate_region_cb(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
> struct page *src_page, void *opaque)
> {
> --
> 2.43.0
>
^ permalink raw reply
* Re: [PATCH 2/2] x86/virt/tdx: Use PFN directly for unmapping guest private memory
From: Edgecombe, Rick P @ 2026-03-19 18:44 UTC (permalink / raw)
To: Li, Xiaoyao, Zhao, Yan Y
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Huang, Kai,
dave.hansen@linux.intel.com, kas@kernel.org, seanjc@google.com,
mingo@redhat.com, pbonzini@redhat.com, binbin.wu@linux.intel.com,
ackerleytng@google.com, linux-kernel@vger.kernel.org,
Yamahata, Isaku, sagis@google.com, Annapurve, Vishal,
bp@alien8.de, tglx@kernel.org, yilun.xu@linux.intel.com,
x86@kernel.org
In-Reply-To: <b74acb4b-4658-4113-9fce-9faf972975c4@intel.com>
On Thu, 2026-03-19 at 16:56 +0800, Xiaoyao Li wrote:
> > > > }
> > > > -void tdx_quirk_reset_page(struct page *page)
> > > > +void tdx_quirk_reset_page(kvm_pfn_t pfn)
> > >
> > > So why keep the function tdx_quirk_reset_page() but expect passing in the
> > > kvm_pfn_t? It looks werid that the name indicates to reset a page but what
> > > gets passed in is a pfn.
The kernel has APIs that take non-struct page arg forms and operate on a "page".
For example free_page(), clear_page(), etc. So keeping the "_page" name seems
not too horrible to me.
> > I thought about introducing tdx_quirk_reset_pfn(). But considering
> > tdx_quirk_reset_pfn() has to be an exported API, I'm reluctant to do that.
Yea exporting two functions that do the same thing doesn't seem the right
balance.
> >
> > Given that even with tdx_quirk_reset_pfn(), it still expects TDX convertible
> > RAM, I think having tdx_quirk_reset_page() to take pfn is still acceptable.
> >
> > We just don't want KVM to do pfn --> struct page --> pfn conversions.
We can assume struct pages have pfn's pretty safely. So pfn->page, and
especially allocated from far away code, is the cleanup target here.
>
> Only tdx_sept_remove_private_spte() is doing such conversions. While
> tdx_reclaim_page() and tdx_reclaim_td_control_pages() already have the
> struct page natively.
>
> So why not considering option 2?
>
> 2. keep tdx_quirk_reset_page() as-is for the cases of
> tdx_reclaim_page() and tdx_reclaim_td_control_pages() that have the
> struct page. But only change tdx_sept_remove_private_spte() to use
> tdx_quirk_reset_paddr() directly.
>
> It will need export tdx_quirk_reset_paddr() for KVM. I think it will be OK?
Exporting tdx_quirk_reset_paddr() seems reasonable, except then we have pfn, PA
and struct page across the API. It increases the asymmetry.
We did discuss converting the whole API over to PFN for symmetry. It could
eliminate the control page and guest memory differences.
But this way seems like a more manageable step that addresses the biggest issue.
If we don't want to do a massive cleanup, there will be some stuff left for the
future.
^ permalink raw reply
* Re: [PATCH v13 37/48] arm64: RMI: Prevent Device mappings for Realms
From: Wei-Lin Chang @ 2026-03-19 18:46 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <20260318155413.793430-38-steven.price@arm.com>
On Wed, Mar 18, 2026 at 03:54:01PM +0000, Steven Price wrote:
> Physical device assignment is not supported by RMM v1.0, so it
> doesn't make much sense to allow device mappings within the realm.
> Prevent them when the guest is a realm.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes from v6:
> * Fix the check in user_mem_abort() to prevent all pages that are not
> guest_memfd() from being mapped into the protected half of the IPA.
> Changes from v5:
> * Also prevent accesses in user_mem_abort()
> ---
> arch/arm64/kvm/mmu.c | 13 +++++++++++++
> 1 file changed, 13 insertions(+)
>
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index ad1300f366df..7d7caab8f573 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -1222,6 +1222,10 @@ int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
> if (is_protected_kvm_enabled())
> return -EPERM;
>
> + /* We don't support mapping special pages into a Realm */
> + if (kvm_is_realm(kvm))
> + return -EPERM;
> +
> size += offset_in_page(guest_ipa);
> guest_ipa &= PAGE_MASK;
>
> @@ -1965,6 +1969,15 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> return 1;
> }
>
> + /*
> + * For now we shouldn't be hitting protected addresses because they are
> + * handled in private_memslot_fault(). In the future this check may be
Hi,
What is private_memslot_fault()? I don't see it anywhere in the series &
upstream.
> + * relaxed to support e.g. protected devices.
> + */
> + if (vcpu_is_rec(vcpu) &&
> + kvm_gpa_from_fault(kvm, fault_ipa) == fault_ipa)
> + return -EINVAL;
> +
Additionally, there is a hunk almost identical to this one here in added
in patch 27.
Thanks,
Wei-Lin Chang
> if (nested)
> adjust_nested_fault_perms(nested, &prot, &writable);
>
> --
> 2.43.0
>
^ permalink raw reply
* Re: [PATCH v13 39/48] arm64: RMI: Propagate number of breakpoints and watchpoints to userspace
From: Wei-Lin Chang @ 2026-03-19 18:50 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Jean-Philippe Brucker, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20260318155413.793430-40-steven.price@arm.com>
On Wed, Mar 18, 2026 at 03:54:03PM +0000, Steven Price wrote:
> From: Jean-Philippe Brucker <jean-philippe@linaro.org>
>
> The RMM describes the maximum number of BPs/WPs available to the guest
> in the Feature Register 0. Propagate those numbers into ID_AA64DFR0_EL1,
> which is visible to userspace. A VMM needs this information in order to
> set up realm parameters.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> Signed-off-by: Steven Price <steven.price@arm.com>
> Reviewed-by: Gavin Shan <gshan@redhat.com>
> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> Reviewed-by: Joey Gouly <joey.gouly@arm.com>
> ---
> arch/arm64/include/asm/kvm_rmi.h | 2 ++
> arch/arm64/kvm/rmi.c | 22 ++++++++++++++++++++++
> arch/arm64/kvm/sys_regs.c | 2 +-
> 3 files changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
> index 17bb7e2a2aa0..8fb526764c30 100644
> --- a/arch/arm64/include/asm/kvm_rmi.h
> +++ b/arch/arm64/include/asm/kvm_rmi.h
> @@ -87,6 +87,8 @@ struct realm_rec {
> void kvm_init_rmi(void);
> u32 kvm_realm_ipa_limit(void);
>
> +u64 kvm_realm_reset_id_aa64dfr0_el1(const struct kvm_vcpu *vcpu, u64 val);
> +
> bool kvm_rmi_supports_sve(void);
> bool kvm_rmi_supports_pmu(void);
>
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> index 8dc090da6e5f..01519d934d3a 100644
> --- a/arch/arm64/kvm/rmi.c
> +++ b/arch/arm64/kvm/rmi.c
> @@ -212,6 +212,28 @@ u32 kvm_realm_ipa_limit(void)
> return u64_get_bits(rmm_feat_reg0, RMI_FEATURE_REGISTER_0_S2SZ);
> }
>
> +u64 kvm_realm_reset_id_aa64dfr0_el1(const struct kvm_vcpu *vcpu, u64 val)
> +{
> + u32 bps = u64_get_bits(rmm_feat_reg0, RMI_FEATURE_REGISTER_0_NUM_BPS);
> + u32 wps = u64_get_bits(rmm_feat_reg0, RMI_FEATURE_REGISTER_0_NUM_WPS);
> + u32 ctx_cmps;
> +
> + if (!kvm_is_realm(vcpu->kvm))
> + return val;
> +
> + /* Ensure CTX_CMPs is still valid */
> + ctx_cmps = FIELD_GET(ID_AA64DFR0_EL1_CTX_CMPs, val);
> + ctx_cmps = min(bps, ctx_cmps);
> +
> + val &= ~(ID_AA64DFR0_EL1_BRPs_MASK | ID_AA64DFR0_EL1_WRPs_MASK |
> + ID_AA64DFR0_EL1_CTX_CMPs);
> + val |= FIELD_PREP(ID_AA64DFR0_EL1_BRPs_MASK, bps) |
> + FIELD_PREP(ID_AA64DFR0_EL1_WRPs_MASK, wps) |
> + FIELD_PREP(ID_AA64DFR0_EL1_CTX_CMPs, ctx_cmps);
> +
> + return val;
> +}
> +
> static int get_start_level(struct realm *realm)
> {
> return 4 - stage2_pgtable_levels(realm->ia_bits);
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 46f5e2ab3e2c..83b5c36f43bf 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -2043,7 +2043,7 @@ static u64 sanitise_id_aa64dfr0_el1(const struct kvm_vcpu *vcpu, u64 val)
> /* Hide BRBE from guests */
> val &= ~ID_AA64DFR0_EL1_BRBE_MASK;
>
> - return val;
> + return kvm_realm_reset_id_aa64dfr0_el1(vcpu, val);
Hi,
Nit:
In other places we condition on kvm_is_realm() to separate
realm/non-realm paths but here everyone goes into kvm_realm_*, do you
think it's more consistent to move the kvm_is_realm() check out of this
function?
Thanks,
Wei-Lin Chang
> }
>
> /*
> --
> 2.43.0
>
^ permalink raw reply
* Re: [PATCH v13 45/48] arm64: RMI: Provide accurate register list
From: Wei-Lin Chang @ 2026-03-19 18:53 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Jean-Philippe Brucker, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20260318155413.793430-46-steven.price@arm.com>
On Wed, Mar 18, 2026 at 03:54:09PM +0000, Steven Price wrote:
> From: Jean-Philippe Brucker <jean-philippe@linaro.org>
>
> Userspace can set a few registers with KVM_SET_ONE_REG (9 GP registers
> at runtime, and 3 system registers during initialization). Update the
> register list returned by KVM_GET_REG_LIST.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v11:
> * Reworked due to upstream changes.
> Changes since v8:
> * Minor type changes following review.
> Changes since v7:
> * Reworked on upstream changes.
> ---
> arch/arm64/kvm/guest.c | 6 ++++++
> arch/arm64/kvm/hypercalls.c | 4 ++--
> arch/arm64/kvm/sys_regs.c | 29 +++++++++++++++++++++++------
> 3 files changed, 31 insertions(+), 8 deletions(-)
>
> diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
> index 2c4db2d1a6ca..23fdb2ee8a61 100644
> --- a/arch/arm64/kvm/guest.c
> +++ b/arch/arm64/kvm/guest.c
> @@ -620,6 +620,9 @@ static unsigned long num_sve_regs(const struct kvm_vcpu *vcpu)
> if (!kvm_arm_vcpu_sve_finalized(vcpu))
> return 1; /* KVM_REG_ARM64_SVE_VLS */
>
> + if (kvm_is_realm(vcpu->kvm))
> + return 1; /* KVM_REG_ARM64_SVE_VLS */
> +
> return slices * (SVE_NUM_PREGS + SVE_NUM_ZREGS + 1 /* FFR */)
> + 1; /* KVM_REG_ARM64_SVE_VLS */
> }
> @@ -647,6 +650,9 @@ static int copy_sve_reg_indices(const struct kvm_vcpu *vcpu,
> if (!kvm_arm_vcpu_sve_finalized(vcpu))
> return num_regs;
>
> + if (kvm_is_realm(vcpu->kvm))
> + return num_regs;
> +
> for (i = 0; i < slices; i++) {
> for (n = 0; n < SVE_NUM_ZREGS; n++) {
> reg = KVM_REG_ARM64_SVE_ZREG(n, i);
> diff --git a/arch/arm64/kvm/hypercalls.c b/arch/arm64/kvm/hypercalls.c
> index 58c5fe7d7572..70ac7971416c 100644
> --- a/arch/arm64/kvm/hypercalls.c
> +++ b/arch/arm64/kvm/hypercalls.c
> @@ -414,14 +414,14 @@ void kvm_arm_teardown_hypercalls(struct kvm *kvm)
>
> int kvm_arm_get_fw_num_regs(struct kvm_vcpu *vcpu)
> {
> - return ARRAY_SIZE(kvm_arm_fw_reg_ids);
> + return kvm_is_realm(vcpu->kvm) ? 0 : ARRAY_SIZE(kvm_arm_fw_reg_ids);
> }
>
> int kvm_arm_copy_fw_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
> {
> int i;
>
> - for (i = 0; i < ARRAY_SIZE(kvm_arm_fw_reg_ids); i++) {
> + for (i = 0; i < kvm_arm_get_fw_num_regs(vcpu); i++) {
> if (put_user(kvm_arm_fw_reg_ids[i], uindices++))
> return -EFAULT;
> }
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index ebb428b861f5..088d900b9c3a 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -5436,18 +5436,18 @@ int kvm_arm_sys_reg_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg
> sys_reg_descs, ARRAY_SIZE(sys_reg_descs));
> }
>
> -static unsigned int num_demux_regs(void)
> +static inline unsigned int num_demux_regs(struct kvm_vcpu *vcpu)
> {
> - return CSSELR_MAX;
> + return kvm_is_realm(vcpu->kvm) ? 0 : CSSELR_MAX;
> }
>
> -static int write_demux_regids(u64 __user *uindices)
> +static int write_demux_regids(struct kvm_vcpu *vcpu, u64 __user *uindices)
> {
> u64 val = KVM_REG_ARM64 | KVM_REG_SIZE_U32 | KVM_REG_ARM_DEMUX;
> unsigned int i;
>
> val |= KVM_REG_ARM_DEMUX_ID_CCSIDR;
> - for (i = 0; i < CSSELR_MAX; i++) {
> + for (i = 0; i < num_demux_regs(vcpu); i++) {
> if (put_user(val | i, uindices))
> return -EFAULT;
> uindices++;
> @@ -5491,11 +5491,28 @@ static bool copy_reg_to_user(const struct sys_reg_desc *reg, u64 __user **uind)
> return true;
> }
>
> +static inline bool kvm_realm_sys_reg_hidden_user(const struct kvm_vcpu *vcpu,
> + u64 reg)
> +{
> + if (!kvm_is_realm(vcpu->kvm))
> + return false;
> +
> + switch (reg) {
> + case SYS_ID_AA64DFR0_EL1:
> + case SYS_PMCR_EL0:
> + return false;
> + }
> + return true;
> +}
> +
> static int walk_one_sys_reg(const struct kvm_vcpu *vcpu,
> const struct sys_reg_desc *rd,
> u64 __user **uind,
> unsigned int *total)
> {
> + if (kvm_realm_sys_reg_hidden_user(vcpu, reg_to_encoding(rd)))
Hi,
Same as my comment for patch 39, I would suggest moving the
kvm_is_realm() check out of this function.
Thanks,
Wei-Lin Chang
> + return 0;
> +
> /*
> * Ignore registers we trap but don't save,
> * and for which no custom user accessor is provided.
> @@ -5533,7 +5550,7 @@ static int walk_sys_regs(struct kvm_vcpu *vcpu, u64 __user *uind)
>
> unsigned long kvm_arm_num_sys_reg_descs(struct kvm_vcpu *vcpu)
> {
> - return num_demux_regs()
> + return num_demux_regs(vcpu)
> + walk_sys_regs(vcpu, (u64 __user *)NULL);
> }
>
> @@ -5546,7 +5563,7 @@ int kvm_arm_copy_sys_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
> return err;
> uindices += err;
>
> - return write_demux_regids(uindices);
> + return write_demux_regids(vcpu, uindices);
> }
>
> #define KVM_ARM_FEATURE_ID_RANGE_INDEX(r) \
> --
> 2.43.0
>
^ permalink raw reply
* Re: [PATCH v13 23/48] KVM: arm64: Expose support for private memory
From: Wei-Lin Chang @ 2026-03-19 19:01 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Suzuki K Poulose, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve
In-Reply-To: <20260318155413.793430-24-steven.price@arm.com>
On Wed, Mar 18, 2026 at 03:53:47PM +0000, Steven Price wrote:
> Select KVM_GENERIC_MEMORY_ATTRIBUTES and provide the necessary support
> functions.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v12:
> * Only define kvm_arch_has_private_mem() when
> CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES is set to avoid build issues
> when KVM is disabled.
> Changes since v10:
> * KVM_GENERIC_PRIVATE_MEM replacd with KVM_GENERIC_MEMORY_ATTRIBUTES.
> Changes since v9:
> * Drop the #ifdef CONFIG_KVM_PRIVATE_MEM guard from the definition of
> kvm_arch_has_private_mem()
> Changes since v2:
> * Switch kvm_arch_has_private_mem() to a macro to avoid overhead of a
> function call.
> * Guard definitions of kvm_arch_{pre,post}_set_memory_attributes() with
> #ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES.
> * Early out in kvm_arch_post_set_memory_attributes() if the WARN_ON
> should trigger.
> ---
> arch/arm64/include/asm/kvm_host.h | 4 ++++
> arch/arm64/kvm/Kconfig | 1 +
> arch/arm64/kvm/mmu.c | 24 ++++++++++++++++++++++++
> 3 files changed, 29 insertions(+)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 64304848aad4..1efea996f474 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -1486,6 +1486,10 @@ struct kvm *kvm_arch_alloc_vm(void);
>
> #define vcpu_is_protected(vcpu) kvm_vm_is_protected((vcpu)->kvm)
>
> +#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
> +#define kvm_arch_has_private_mem(kvm) ((kvm)->arch.is_realm)
> +#endif
> +
> int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature);
> bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu);
>
> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> index 4f803fd1c99a..1cac6dfc0972 100644
> --- a/arch/arm64/kvm/Kconfig
> +++ b/arch/arm64/kvm/Kconfig
> @@ -38,6 +38,7 @@ menuconfig KVM
> select SCHED_INFO
> select GUEST_PERF_EVENTS if PERF_EVENTS
> select KVM_GUEST_MEMFD
> + select KVM_GENERIC_MEMORY_ATTRIBUTES
Hi,
I believe we should also add this:
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index bfa0ab343081..13722f876dcd 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -6365,7 +6365,7 @@ Returns -EINVAL if called on a protected VM.
-------------------------------
:Capability: KVM_CAP_MEMORY_ATTRIBUTES
-:Architectures: x86
+:Architectures: x86, arm64
:Type: vm ioctl
:Parameters: struct kvm_memory_attributes (in)
:Returns: 0 on success, <0 on error
Thanks,
Wei-Lin Chang
> help
> Support hosting virtualized guest machines.
>
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index b705ad6c6c8b..bad93938acdb 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -2494,6 +2494,30 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
> return ret;
> }
>
> +#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
> +bool kvm_arch_pre_set_memory_attributes(struct kvm *kvm,
> + struct kvm_gfn_range *range)
> +{
> + WARN_ON_ONCE(!kvm_arch_has_private_mem(kvm));
> + return false;
> +}
> +
> +bool kvm_arch_post_set_memory_attributes(struct kvm *kvm,
> + struct kvm_gfn_range *range)
> +{
> + if (WARN_ON_ONCE(!kvm_arch_has_private_mem(kvm)))
> + return false;
> +
> + if (range->arg.attributes & KVM_MEMORY_ATTRIBUTE_PRIVATE)
> + range->attr_filter = KVM_FILTER_SHARED;
> + else
> + range->attr_filter = KVM_FILTER_PRIVATE;
> + kvm_unmap_gfn_range(kvm, range);
> +
> + return false;
> +}
> +#endif
> +
> void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot)
> {
> }
> --
> 2.43.0
>
^ permalink raw reply related
* Re: [PATCH] KVM: TDX: Fix APIC MSR ranges in tdx_has_emulated_msr()
From: Edgecombe, Rick P @ 2026-03-19 19:33 UTC (permalink / raw)
To: dmaluka@chromium.org, kvm@vger.kernel.org, pbonzini@redhat.com,
Hansen, Dave, seanjc@google.com, binbin.wu@linux.intel.com,
Yamahata, Isaku
Cc: bp@alien8.de, x86@kernel.org, kas@kernel.org, hpa@zytor.com,
linux-kernel@vger.kernel.org, mingo@redhat.com,
dave.hansen@linux.intel.com, tglx@kernel.org,
linux-coco@lists.linux.dev
In-Reply-To: <ee8d7ea3-0ece-43cf-ab07-6954df4c01ad@linux.intel.com>
On Thu, 2026-03-19 at 15:40 +0800, Binbin Wu wrote:
> tdx_has_emulated_msr() is used by KVM to decide whether to emulate a MSR access from the
> TDVMCALL or just return the error code.
>
> During an off-list discussion, Rick noted that #VE reduction could change the behavior of
> accessing an MSR (e.g., from #VE to #GP or to be virtualized by the TDX module) without
> KVM knowing.Because KVM lacks the full context to perfectly decide if an MSR should be
> emulated, the question was raised: Can we just delete tdx_has_emulated_msr() entirely?
>
> However, these native type x2apic MSRs are a special case. Since the TDX module owns the
> APICv page, KVM cannot emulate these MSRs. If we remove tdx_has_emulated_msr(), a guest
> directly issuing TDVMCALLs for these native type x2apic MSRs will trigger a silent failure,
> even though this is the guest's fault.
>
> It comes down to a tradeoff. Should we prioritize code simplicity by dropping the function,
> or keep it to explicitly catch this misbehaving guest corner case?
I think from KVM's perspective it doesn't want to help the guest behave
correctly. So we can ignore that I think. But it does really care to not define
any specific guest ABI that it has to maintain. So tdx_has_emulated_msr() has
some value there. And even more, it wants to not allow the guest to hurt the
host.
On the latter point, another problem with deleting tdx_has_emulated_msr() is the
current code path skips the checks done in the other MSR paths. So we would need
to call some appropriate higher up MSR helper to protect the host? And that
wades into the CPUID bit consistency issues.
So maybe... could we do a more limited version of the deletion where we allow
all the APIC MSRs through? We'd have to check that it won't cause problems.
Failing that, we should maybe just explicitly list the ones TDX supports rather
than the current way we define the APIC ones. As you mention below, it's not
correct in other ways too so it could be more robust.
>
>
> BTW, besides the bug described by this patch, according to the latest published TDX module
> ABI table, MSR IA32_X2APIC_SELF_IPI is native type, but not included in the list.
> There are some MSRs, which are reserved for xAPIC MSR, not included in the list, but they
> can be covered by the KVM common code.
^ permalink raw reply
* Re: [PATCH v13 00/48] arm64: Support for Arm CCA in KVM
From: Mathieu Poirier @ 2026-03-19 23:02 UTC (permalink / raw)
To: Steven Price
Cc: kvm, kvmarm, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20260318155413.793430-1-steven.price@arm.com>
Good day,
On Wed, Mar 18, 2026 at 03:53:24PM +0000, Steven Price wrote:
> This series adds support for running protected VMs using KVM under the
> Arm Confidential Compute Architecture (CCA).
>
> New major version number! This now targets RMM v2.0-bet0[1]. And unlike
> for Linux this represents a significant change.
>
> RMM v2.0 brings with it the ability to configure the RMM to have the
> same page size as the host (so no more RMM_PAGE_SIZE and dealing with
> granules being different from host pages). It also introduces range
> based APIs for many operations which should be more efficient and
> simplifies the code in places.
>
> The handling of the GIC has changed, so the system registers are used to
> pass the GIC state rather than memory. This means fewer changes to the
> KVM code as it looks much like a normal VM in this respect.
>
> And of course the new uAPI introduced in the previous v12 posting is
> retained so that also remains simplified compared to earlier postings.
>
> The RMM support for v2.0 is still early and so this series includes a
> few hacks to ease the integration. Of note are that there are some RMM
> v1.0 SMCs added to paper over areas where the RMM implementation isn't
> quite ready for v2.0, and "SROs" (see below) are deferred to the final
> patch in the series.
>
> The PMU in RMM v2.0 requires more handling on the RMM-side (and
> therefore simplifies the implementation on Linux), but this isn't quite
> ready yet. The Linux side is implemented (but untested).
>
> PSCI still requires the VMM to provide the "target" REC for operations
> that affect another vCPU. This is likely to change in a future version
> of the specification. There's also a desire to force PSCI to be handled
> in the VMM for realm guests - this isn't implemented yet as I'm waiting
> for the dust to settle on the RMM interface first.
>
> Stateful RMI Operations
> -----------------------
>
> The RMM v2.0 spec brings a new concept of Stateful RMI Operations (SROs)
> which allow the RMM to complete an operation over several SMC calls and
> requesting/returning memory to the host. This has the benefit of
> allowing interrupts to be handled in the middle of an operation (by
> returning to the host to handle the interrupt without completing the
> operation) and enables the RMM to dynamically allocate memory for
> internal tracking purposes. One example of this is RMI_REC_CREATE no
> longer needs "auxiliary granules" provided upfront but can request the
> memory needed during the RMI_REC_CREATE operation.
>
> There are a fairly large number of operations that are defined as SROs
> in the specification, but current both Linux and RMM only have support
> for RMI_REC_CREATE and RMI_REC_DESTROY. There a number of TODOs/FIXMEs
> in the code where support is missing.
>
> Given the early stage support for this, the SRO handling is all confined
> to the final patch. This patch can be dropped to return to a pre-SRO
> state (albeit a mixture of RMM v1.0 and v2.0 APIs) for testing purposes.
>
> A future posting will reorder the series to move the generic SRO support
> to an early patch and will implement the proper support for this in all
> RMI SMCs.
>
> One aspect of SROs which is not yet well captured is that in some
> circumstances the Linux kernel will need to call an SRO call in a
> context where memory allocation is restricted (e.g. because a spinlock
> is held). In this case the intention is that the SRO will be cancelled,
> the spinlock dropped so the memory allocation can be completed, and then
> the SRO restarted (obviously after rechecking the state that the
> spinlock was protecting). For this reason the code stores the memory
> allocations within a struct rmi_sro_state object - see the final patch
> for more details.
>
> This series is based on v7.0-rc1. It is also available as a git
> repository:
>
> https://gitlab.arm.com/linux-arm/linux-cca cca-host/v13
>
> Work in progress changes for kvmtool are available from the git
> repository below:
>
> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/v11
>
> Note that the kvmtool code has been tidied up (thanks to Suzuki) and
> this involves a minor change in flags. The "--restricted_mem" flag is no
> longer recognised (or necessary).
>
> The TF-RMM has not yet merged the RMMv2.0 support, so you will need to
> use the following branch:
>
> https://git.trustedfirmware.org/TF-RMM/tf-rmm.git topics/rmm-v2.0-poc
This RMM version is expecting a RMM EL3 interface version of at least 2.0. Do
you have a TF-A to use with it?
Thanks,
Mathieu
>
> [1] https://developer.arm.com/documentation/den0137/2-0bet0/
>
> Jean-Philippe Brucker (7):
> arm64: RMI: Propagate number of breakpoints and watchpoints to
> userspace
> arm64: RMI: Set breakpoint parameters through SET_ONE_REG
> arm64: RMI: Initialize PMCR.N with number counter supported by RMM
> arm64: RMI: Propagate max SVE vector length from RMM
> arm64: RMI: Configure max SVE vector length for a Realm
> arm64: RMI: Provide register list for unfinalized RMI RECs
> arm64: RMI: Provide accurate register list
>
> Joey Gouly (2):
> arm64: RMI: allow userspace to inject aborts
> arm64: RMI: support RSI_HOST_CALL
>
> Steven Price (36):
> kvm: arm64: Avoid including linux/kvm_host.h in kvm_pgtable.h
> arm64: RME: Handle Granule Protection Faults (GPFs)
> arm64: RMI: Add SMC definitions for calling the RMM
> arm64: RMI: Temporarily add SMCs from RMM v1.0 spec
> arm64: RMI: Add wrappers for RMI calls
> arm64: RMI: Check for RMI support at KVM init
> arm64: RMI: Configure the RMM with the host's page size
> arm64: RMI: Check for LPA2 support
> arm64: RMI: Ensure that the RMM has GPT entries for memory
> arm64: RMI: Define the user ABI
> arm64: RMI: Basic infrastructure for creating a realm.
> KVM: arm64: Allow passing machine type in KVM creation
> arm64: RMI: RTT tear down
> arm64: RMI: Activate realm on first VCPU run
> arm64: RMI: Allocate/free RECs to match vCPUs
> arm64: RMI: Support for the VGIC in realms
> KVM: arm64: Support timers in realm RECs
> arm64: RMI: Handle realm enter/exit
> arm64: RMI: Handle RMI_EXIT_RIPAS_CHANGE
> KVM: arm64: Handle realm MMIO emulation
> KVM: arm64: Expose support for private memory
> arm64: RMI: Allow populating initial contents
> arm64: RMI: Set RIPAS of initial memslots
> arm64: RMI: Create the realm descriptor
> arm64: RMI: Runtime faulting of memory
> KVM: arm64: Handle realm VCPU load
> KVM: arm64: Validate register access for a Realm VM
> KVM: arm64: Handle Realm PSCI requests
> KVM: arm64: WARN on injected undef exceptions
> arm64: Don't expose stolen time for realm guests
> arm64: RMI: Always use 4k pages for realms
> arm64: RMI: Prevent Device mappings for Realms
> arm64: RMI: Enable PMU support with a realm guest
> KVM: arm64: Expose KVM_ARM_VCPU_REC to user space
> arm64: RMI: Enable realms to be created
> [WIP] arm64: RMI: Add support for SRO
>
> Suzuki K Poulose (3):
> kvm: arm64: Include kvm_emulate.h in kvm/arm_psci.h
> kvm: arm64: Don't expose unsupported capabilities for realm guests
> arm64: RMI: Allow checking SVE on VM instance
>
> Documentation/virt/kvm/api.rst | 86 +-
> arch/arm64/include/asm/kvm_emulate.h | 31 +
> arch/arm64/include/asm/kvm_host.h | 15 +-
> arch/arm64/include/asm/kvm_pgtable.h | 5 +-
> arch/arm64/include/asm/kvm_pkvm.h | 2 +-
> arch/arm64/include/asm/kvm_rmi.h | 129 ++
> arch/arm64/include/asm/rmi_cmds.h | 692 +++++++++
> arch/arm64/include/asm/rmi_smc.h | 430 ++++++
> arch/arm64/include/asm/virt.h | 1 +
> arch/arm64/kernel/cpufeature.c | 1 +
> arch/arm64/kvm/Kconfig | 2 +
> arch/arm64/kvm/Makefile | 2 +-
> arch/arm64/kvm/arch_timer.c | 28 +-
> arch/arm64/kvm/arm.c | 178 ++-
> arch/arm64/kvm/guest.c | 95 +-
> arch/arm64/kvm/hyp/pgtable.c | 1 +
> arch/arm64/kvm/hypercalls.c | 4 +-
> arch/arm64/kvm/inject_fault.c | 5 +-
> arch/arm64/kvm/mmio.c | 16 +-
> arch/arm64/kvm/mmu.c | 214 ++-
> arch/arm64/kvm/pmu-emul.c | 6 +
> arch/arm64/kvm/psci.c | 30 +
> arch/arm64/kvm/reset.c | 13 +-
> arch/arm64/kvm/rmi-exit.c | 207 +++
> arch/arm64/kvm/rmi.c | 1948 ++++++++++++++++++++++++++
> arch/arm64/kvm/sys_regs.c | 53 +-
> arch/arm64/kvm/vgic/vgic-init.c | 2 +-
> arch/arm64/mm/fault.c | 28 +-
> include/kvm/arm_arch_timer.h | 2 +
> include/kvm/arm_pmu.h | 4 +
> include/kvm/arm_psci.h | 2 +
> include/uapi/linux/kvm.h | 41 +-
> 32 files changed, 4176 insertions(+), 97 deletions(-)
> create mode 100644 arch/arm64/include/asm/kvm_rmi.h
> create mode 100644 arch/arm64/include/asm/rmi_cmds.h
> create mode 100644 arch/arm64/include/asm/rmi_smc.h
> create mode 100644 arch/arm64/kvm/rmi-exit.c
> create mode 100644 arch/arm64/kvm/rmi.c
>
> --
> 2.43.0
>
>
^ permalink raw reply
* Re: [PATCH v2 08/19] PCI/TSM: Add "evidence" support
From: Dan Williams @ 2026-03-20 2:50 UTC (permalink / raw)
To: Jakub Kicinski, Dan Williams
Cc: linux-coco, linux-pci, gregkh, aik, aneesh.kumar, yilun.xu,
bhelgaas, alistair23, lukas, jgg, Donald Hunter
In-Reply-To: <20260318170014.6650d2bf@kernel.org>
Jakub Kicinski wrote:
[..]
> > > The main strength of Netlink is "do" commands with multiple optional
> > > attrs.
> >
> > Yes, that is attractive and saves a pile of bug prone ioctl handling.
> >
> > The gap I need to fill first though is a uAPI that allows for large
> > blobs to be fetched after being regenerated / reformatted besed on some
> > input attributes.
> >
> > "Multi message netlink attributes" while inventive, feels less awkward
> > and more future proof than a sysfs binary attribute scheme to do the
> > same.
>
> Alright, so to make this more Netlink-y you can either:
> - delete the F_MULTI and replicate other attrs in each message and
> add an offset attr; this will make each message in the dump more
> standalone.
> - keep the F_MULTI but object_val has to be a multi-attr, and then
> we have to teach YNL to correctly append the attrs.
>
> Former is definitely less work. Latter could end up being cleaner
> but there are some unknowns so hard to tell for sure; more plumbing.
Makes sense, and not yet sure which one is lower maintenance burden
long term. I will play with it a bit.
I am leaning to the first option more for the fact that it puts the
burden of being strange squarely on the implementation that wants it. If
this discussion attracts more large blob dumpers then the second option.
Thanks for spending some attention here.
^ permalink raw reply
* Re: [PATCH v5 17/22] x86/virt/tdx: Avoid updates during update-sensitive operations
From: Chao Gao @ 2026-03-20 8:00 UTC (permalink / raw)
To: Kiryl Shutsemau
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <abwAmw_JfpWScKGO@thinkstation>
On Thu, Mar 19, 2026 at 02:00:00PM +0000, Kiryl Shutsemau wrote:
>On Sun, Mar 15, 2026 at 06:58:37AM -0700, Chao Gao wrote:
>> diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
>> index b3a7301e77c6..4c4f7acd4044 100644
>> --- a/arch/x86/include/asm/tdx.h
>> +++ b/arch/x86/include/asm/tdx.h
>> @@ -26,11 +26,18 @@
>> #define TDX_SEAMCALL_GP (TDX_SW_ERROR | X86_TRAP_GP)
>> #define TDX_SEAMCALL_UD (TDX_SW_ERROR | X86_TRAP_UD)
>>
>> +#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL
>> +
>> /*
>> * TDX module SEAMCALL leaf function error codes
>> */
>> -#define TDX_SUCCESS 0ULL
>> -#define TDX_RND_NO_ENTROPY 0x8000020300000000ULL
>> +#define TDX_SUCCESS 0ULL
>> +#define TDX_RND_NO_ENTROPY 0x8000020300000000ULL
>> +#define TDX_UPDATE_COMPAT_SENSITIVE 0x8000051200000000ULL
>
>This competes with other patchset[1].
>
>[1] https://lore.kernel.org/all/20260307010358.819645-1-rick.p.edgecombe@intel.com
Got it. But I suppose the conflicts can be addressed when maintainers merge
the two series.
Dave asked me to remove all (false) dependencies to make this series move
reviewable since v3. So I think we don't need to rebase onto that patchset.
>
>> @@ -1189,9 +1192,21 @@ int tdx_module_shutdown(void)
>> * modules as new modules likely have higher handoff version.
>> */
>> args.rcx = tdx_sysinfo.handoff.module_hv;
>> - ret = seamcall_prerr(TDH_SYS_SHUTDOWN, &args);
>> - if (ret)
>> - return ret;
>> +
>> + if (tdx_supports_update_compatibility(&tdx_sysinfo))
>> + args.rcx |= TDX_SYS_SHUTDOWN_AVOID_COMPAT_SENSITIVE;
>
>Hm. So what happens if the module doesn't support it? We just ignore
>problem?
Yes. The kernel ignores the problem and leaves the decision (whether to
update modules) to userspace.
>
>Maybe we should just block updates on such modules?
Kai made the same suggestion, but Dan rejected it [1][2]. Dan's position is
to avoid kernel complexity and let userspace handle the check, updating at
their own risk.
I've prepared a patch for the userspace tool [3] to check if the feature is
supported and will push it.
[1]: https://lore.kernel.org/kvm/699fe97dc212f_2f4a100b@dwillia2-mobl4.notmuch/
[2]: https://lore.kernel.org/kvm/69a0c3d24310_1cc5100d1@dwillia2-mobl4.notmuch/
[3]: https://github.com/intel/confidential-computing.tdx.tdx-module.binaries/blob/main/version_select_and_load.py
^ permalink raw reply
* Re: [PATCH v5 09/22] x86/virt/seamldr: Introduce skeleton for TDX module updates
From: Chao Gao @ 2026-03-20 8:10 UTC (permalink / raw)
To: Kiryl Shutsemau
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <abv2PWj4jBwm6zKO@thinkstation>
>> +static struct {
>> + enum module_update_state state;
>> + int thread_ack;
>> + /*
>> + * Protect update_data. Raw spinlock as it will be acquired from
>> + * interrupt-disabled contexts.
>> + */
>> + raw_spinlock_t lock;
>> +} update_data = {
>> + .lock = __RAW_SPIN_LOCK_UNLOCKED(update_data.lock)
>> +};
>
>multi_stop_cpu() used atomic_t for thread_ack insead of spinlock.
>Any particular reason you took different direction?
I switched from atomic_t to spinlock in v5 after Dave complained [*] that using
atomic_t and memory barriers is "overly complicated".
Testing both approaches showed no scalability issues, so I chose spinlock for
better readability.
*: https://lore.kernel.org/kvm/abP%2F2nPh9qkElV4L@intel.com/
^ permalink raw reply
* Re: [PATCH v5 10/22] x86/virt/seamldr: Abort updates if errors occurred midway
From: Chao Gao @ 2026-03-20 8:12 UTC (permalink / raw)
To: Kiryl Shutsemau
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <abv3Jxn5QPFGjq9g@thinkstation>
On Thu, Mar 19, 2026 at 01:19:35PM +0000, Kiryl Shutsemau wrote:
>On Sun, Mar 15, 2026 at 06:58:30AM -0700, Chao Gao wrote:
>> diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
>> index 978fcca92128..e195703398e7 100644
>> --- a/arch/x86/virt/vmx/tdx/seamldr.c
>> +++ b/arch/x86/virt/vmx/tdx/seamldr.c
>> @@ -190,6 +190,7 @@ enum module_update_state {
>> static struct {
>> enum module_update_state state;
>> int thread_ack;
>> + int failed;
>
>bool is enough, right?
Yes.
>
>> /*
>> * Protect update_data. Raw spinlock as it will be acquired from
>> * interrupt-disabled contexts.
>> @@ -237,12 +238,17 @@ static int do_seamldr_install_module(void *seamldr_params)
>> break;
>> }
>>
>> - ack_state();
>> + if (ret) {
>> + scoped_guard(raw_spinlock, &update_data.lock)
>> + update_data.failed++;
>
>I don't see a reason in spin lock here. Just WRITE_ONCE() would work
>fine.
Sure. Will use a bool and WRITE_ONCE() here.
^ permalink raw reply
* Re: [PATCH v5 11/22] x86/virt/seamldr: Shut down the current TDX module
From: Chao Gao @ 2026-03-20 8:21 UTC (permalink / raw)
To: Kiryl Shutsemau
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <abv4OWfqDOOLR9XT@thinkstation>
>> +static int get_tdx_sys_info_handoff(struct tdx_sys_info_handoff *sysinfo_handoff)
>> +{
>> + int ret = 0;
>> + u64 val;
>> +
>> + if (!ret && !(ret = read_sys_metadata_field(0x8900000100000000, &val)))
>
>!ret check is redundant as well as the ret initialization above.
Ok. Will remove them:
static int get_tdx_sys_info_handoff(struct tdx_sys_info_handoff *sysinfo_handoff)
{
int ret;
u64 val;
if (!(ret = read_sys_metadata_field(0x8900000100000000, &val)))
sysinfo_handoff->module_hv = val;
return ret;
}
^ permalink raw reply
* Re: [PATCH v5 22/22] x86/virt/seamldr: Log TDX module update failures
From: Chao Gao @ 2026-03-20 8:31 UTC (permalink / raw)
To: Kiryl Shutsemau
Cc: linux-kernel, linux-coco, kvm, binbin.wu, dan.j.williams,
dave.hansen, ira.weiny, kai.huang, nik.borisov, paulmck, pbonzini,
reinette.chatre, rick.p.edgecombe, sagis, seanjc, tony.lindgren,
vannapurve, vishal.l.verma, yilun.xu, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, x86, H. Peter Anvin
In-Reply-To: <abwCz6HsMopm2wFF@thinkstation>
On Thu, Mar 19, 2026 at 02:06:36PM +0000, Kiryl Shutsemau wrote:
>On Sun, Mar 15, 2026 at 06:58:42AM -0700, Chao Gao wrote:
>> diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
>> index e6b7af410c54..4e1ad06506cc 100644
>> --- a/arch/x86/virt/vmx/tdx/seamldr.c
>> +++ b/arch/x86/virt/vmx/tdx/seamldr.c
>> @@ -222,6 +222,11 @@ static void ack_state(void)
>> set_target_state(update_data.state + 1);
>> }
>>
>> +static void print_update_failure_message(void)
>> +{
>> + pr_err_once("update failed, SEAMCALLs will report failure until TDs killed\n");
>
>Is it useful to indicate which step has failed?
For now, all failures should be from SEAMCALL failures. The kernel already
prints debug info (SEAMCALL leaf number, error code etc.), so it is quite easy
to find out which specific SEAMCALL failed (then the failing step should be
clear). So I think there is no need to print step information in this log.
^ permalink raw reply
* Re: [PATCH v13 15/48] arm64: RMI: RTT tear down
From: Suzuki K Poulose @ 2026-03-20 10:37 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Zenghui Yu, linux-arm-kernel, linux-kernel,
Joey Gouly, Alexandru Elisei, Christoffer Dall, Fuad Tabba,
linux-coco, Ganapatrao Kulkarni, Gavin Shan, Shanker Donthineni,
Alper Gun, Aneesh Kumar K . V, Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20260318155413.793430-16-steven.price@arm.com>
On 18/03/2026 15:53, Steven Price wrote:
> The RMM owns the stage 2 page tables for a realm, and KVM must request
> that the RMM creates/destroys entries as necessary. The physical pages
> to store the page tables are delegated to the realm as required, and can
> be undelegated when no longer used.
>
> Creating new RTTs is the easy part, tearing down is a little more
> tricky. The result of realm_rtt_destroy() can be used to effectively
> walk the tree and destroy the entries (undelegating pages that were
> given to the realm).
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v12:
> * Simplify some functions now we know RMM page size is the same as the
> host's.
> Changes since v11:
> * Moved some code from earlier in the series to this one so that it's
> added when it's first used.
> Changes since v10:
> * RME->RMI rename.
> * Some code to handle freeing stage 2 PGD moved into this patch where
> it belongs.
> Changes since v9:
> * Add a comment clarifying that root level RTTs are not destroyed until
> after the RD is destroyed.
> Changes since v8:
> * Introduce free_rtt() wrapper which calls free_delegated_granule()
> followed by kvm_account_pgtable_pages(). This makes it clear where an
> RTT is being freed rather than just a delegated granule.
> Changes since v6:
> * Move rme_rtt_level_mapsize() and supporting defines from kvm_rme.h
> into rme.c as they are only used in that file.
> Changes since v5:
> * Rename some RME_xxx defines to do with page sizes as RMM_xxx - they are
> a property of the RMM specification not the RME architecture.
> Changes since v2:
> * Moved {alloc,free}_delegated_page() and ensure_spare_page() to a
> later patch when they are actually used.
> * Some simplifications now rmi_xxx() functions allow NULL as an output
> parameter.
> * Improved comments and code layout.
> ---
> arch/arm64/include/asm/kvm_rmi.h | 7 ++
> arch/arm64/kvm/mmu.c | 15 +++-
> arch/arm64/kvm/rmi.c | 145 +++++++++++++++++++++++++++++++
> 3 files changed, 166 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
> index 0ada525af18f..16a297f3091a 100644
> --- a/arch/arm64/include/asm/kvm_rmi.h
> +++ b/arch/arm64/include/asm/kvm_rmi.h
> @@ -68,5 +68,12 @@ u32 kvm_realm_ipa_limit(void);
>
> int kvm_init_realm_vm(struct kvm *kvm);
> void kvm_destroy_realm(struct kvm *kvm);
> +void kvm_realm_destroy_rtts(struct kvm *kvm);
> +
> +static inline bool kvm_realm_is_private_address(struct realm *realm,
> + unsigned long addr)
> +{
> + return !(addr & BIT(realm->ia_bits - 1));
> +}
>
> #endif /* __ASM_KVM_RMI_H */
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 9dc242c3b9c8..41152abf55b2 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -1098,10 +1098,23 @@ void stage2_unmap_vm(struct kvm *kvm)
> void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
> {
> struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
> - struct kvm_pgtable *pgt = NULL;
> + struct kvm_pgtable *pgt;
>
> write_lock(&kvm->mmu_lock);
> pgt = mmu->pgt;
> + if (kvm_is_realm(kvm) &&
> + (kvm_realm_state(kvm) != REALM_STATE_DEAD &&
> + kvm_realm_state(kvm) != REALM_STATE_NONE)) {
> + write_unlock(&kvm->mmu_lock);
> + kvm_realm_destroy_rtts(kvm);
> +
> + /*
> + * The PGD pages can be reclaimed only after the realm (RD) is
> + * destroyed. We call this again from kvm_destroy_realm() after
> + * the RD is destroyed.
> + */
> + return;
> + }
> if (pgt) {
> mmu->pgd_phys = 0;
> mmu->pgt = NULL;
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> index 700b8c935d29..1fd2c18f7381 100644
> --- a/arch/arm64/kvm/rmi.c
> +++ b/arch/arm64/kvm/rmi.c
> @@ -15,6 +15,19 @@
> static unsigned long rmm_feat_reg0;
> static unsigned long rmm_feat_reg1;
>
-->
> +#define RMM_RTT_BLOCK_LEVEL 2
...
> +
> +#define RMM_L2_BLOCK_SIZE PMD_SIZE
<--
Unused ? Even better we could use PMD_SIZE directly if at all we need
it, as we are using PAGE_SIZE
minor nit: Also, may be we can have a generic name for the
RMM_RTT_MAX_LEVEL ? This applies to all page tables ?
I see we have KVM_PGTALBE_LAST_LEVEL, may be we could use that ?
> +
> +static inline unsigned long rmi_rtt_level_mapsize(int level)
> +{
> + if (WARN_ON(level > RMM_RTT_MAX_LEVEL))
> + return PAGE_SIZE;
> +
> + return (1UL << ARM64_HW_PGTABLE_LEVEL_SHIFT(level));
> +}
> +
> static bool rmi_has_feature(unsigned long feature)
> {
> return !!u64_get_bits(rmm_feat_reg0, feature);
> @@ -189,6 +202,11 @@ u32 kvm_realm_ipa_limit(void)
> return u64_get_bits(rmm_feat_reg0, RMI_FEATURE_REGISTER_0_S2SZ);
> }
>
> +static int get_start_level(struct realm *realm)
> +{
> + return 4 - stage2_pgtable_levels(realm->ia_bits);
> +}
> +
> static int undelegate_range(phys_addr_t phys, unsigned long size)
> {
> unsigned long ret;
> @@ -223,6 +241,131 @@ static int free_delegated_page(phys_addr_t phys)
> return 0;
> }
>
> +static void free_rtt(phys_addr_t phys)
> +{
> + if (free_delegated_page(phys))
> + return;
> +
> + kvm_account_pgtable_pages(phys_to_virt(phys), -1);
> +}
How about a comment here for the function below ?
Something like :
/*
* realm_rtt_destroy: Destroy an RTT at @level for @addr.
*
* Returns - Result of the RMI_RTT_DESTROY call, additionally :
* @out_rtt : RTT granule, if the RTT was destroyed.
* @next_addr: IPA corresponding to the next possible valid Table entry
* we can target.
*/
> +
> +static int realm_rtt_destroy(struct realm *realm, unsigned long addr,
> + int level, phys_addr_t *rtt_granule,
> + unsigned long *next_addr)
> +{
> + unsigned long out_rtt;
> + int ret;
> +
> + ret = rmi_rtt_destroy(virt_to_phys(realm->rd), addr, level,
> + &out_rtt, next_addr);
> +
> + *rtt_granule = out_rtt;
> +
> + return ret;
> +}
> +
> +static int realm_tear_down_rtt_level(struct realm *realm, int level,
> + unsigned long start, unsigned long end)
> +{
> + ssize_t map_size;
> + unsigned long addr, next_addr;
> +
> + if (WARN_ON(level > RMM_RTT_MAX_LEVEL))
> + return -EINVAL;
> +
> + map_size = rmi_rtt_level_mapsize(level - 1);
> +
> + for (addr = start; addr < end; addr = next_addr) {
> + phys_addr_t rtt_granule;
> + int ret;
> + unsigned long align_addr = ALIGN(addr, map_size);
> +
> + next_addr = ALIGN(addr + 1, map_size);
> +
> + if (next_addr > end || align_addr != addr) {
> + /*
> + * The target range is smaller than what this level
> + * covers, recurse deeper.
> + */
> + ret = realm_tear_down_rtt_level(realm,
> + level + 1,
> + addr,
> + min(next_addr, end));
> + if (ret)
> + return ret;
> + continue;
> + }
> +
> + ret = realm_rtt_destroy(realm, addr, level,
> + &rtt_granule, &next_addr);
> +
> + switch (RMI_RETURN_STATUS(ret)) {
> + case RMI_SUCCESS:
> + free_rtt(rtt_granule);
> + break;
> + case RMI_ERROR_RTT:
> + if (next_addr > addr) {
> + /* Missing RTT, skip */
> + break;
> + }
> + /*
> + * We tear down the RTT range for the full IPA
> + * space, after everything is unmapped. Also we
> + * descend down only if we cannot tear down a
> + * top level RTT. Thus RMM must be able to walk
> + * to the requested level. e.g., a block mapping
> + * exists at L1 or L2.
> + */
> + if (WARN_ON(RMI_RETURN_INDEX(ret) != level))
> + return -EBUSY;
> + if (WARN_ON(level == RMM_RTT_MAX_LEVEL))
> + return -EBUSY;
> +
> + /*
> + * The table has active entries in it, recurse deeper
> + * and tear down the RTTs.
> + */
> + next_addr = ALIGN(addr + 1, map_size);
> + ret = realm_tear_down_rtt_level(realm,
> + level + 1,
> + addr,
> + next_addr);
> + if (ret)
> + return ret;
> + /*
> + * Now that the child RTTs are destroyed,
> + * retry at this level.
> + */
> + next_addr = addr;
> + break;
> + default:
> + WARN_ON(1);
> + return -ENXIO;
> + }
> + }
> +
> + return 0;
> +}
> +
> +static int realm_tear_down_rtt_range(struct realm *realm,
> + unsigned long start, unsigned long end)
> +{
> + /*
> + * Root level RTTs can only be destroyed after the RD is destroyed. So
> + * tear down everything below the root level
> + */
> + return realm_tear_down_rtt_level(realm, get_start_level(realm) + 1,
> + start, end);
> +}
> +
> +void kvm_realm_destroy_rtts(struct kvm *kvm)
> +{
> + struct realm *realm = &kvm->arch.realm;
> + unsigned int ia_bits = realm->ia_bits;
> +
> + WARN_ON(realm_tear_down_rtt_range(realm, 0, (1UL << ia_bits)));
AFAICS, we already WARN_ON() in all the cases where the
realm_tear_down_rtt_range() fails, so may be we can skip this
WARN_ON here ?
Suzuki
> +}
> +
> void kvm_destroy_realm(struct kvm *kvm)
> {
> struct realm *realm = &kvm->arch.realm;
> @@ -246,6 +389,8 @@ void kvm_destroy_realm(struct kvm *kvm)
> if (realm->rd) {
> phys_addr_t rd_phys = virt_to_phys(realm->rd);
>
> + kvm_realm_destroy_rtts(kvm);
> +
> if (WARN_ON(rmi_realm_destroy(rd_phys)))
> return;
> free_delegated_page(rd_phys);
^ permalink raw reply
* Re: [PATCH v13 21/48] arm64: RMI: Handle RMI_EXIT_RIPAS_CHANGE
From: Suzuki K Poulose @ 2026-03-20 11:15 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Catalin Marinas, Marc Zyngier, Will Deacon, James Morse,
Oliver Upton, Zenghui Yu, linux-arm-kernel, linux-kernel,
Joey Gouly, Alexandru Elisei, Christoffer Dall, Fuad Tabba,
linux-coco, Ganapatrao Kulkarni, Gavin Shan, Shanker Donthineni,
Alper Gun, Aneesh Kumar K . V, Emi Kisanuki, Vishal Annapurve
In-Reply-To: <20260318155413.793430-22-steven.price@arm.com>
Hi Steven
On 18/03/2026 15:53, Steven Price wrote:
> The guest can request that a region of it's protected address space is
> switched between RIPAS_RAM and RIPAS_EMPTY (and back) using
> RSI_IPA_STATE_SET. This causes a guest exit with the
> RMI_EXIT_RIPAS_CHANGE code. We treat this as a request to convert a
> protected region to unprotected (or back), exiting to the VMM to make
> the necessary changes to the guest_memfd and memslot mappings. On the
> next entry the RIPAS changes are committed by making RMI_RTT_SET_RIPAS
> calls.
>
> The VMM may wish to reject the RIPAS change requested by the guest. For
> now it can only do this by no longer scheduling the VCPU as we don't
> currently have a usecase for returning that rejection to the guest, but
> by postponing the RMI_RTT_SET_RIPAS changes to entry we leave the door
> open for adding a new ioctl in the future for this purpose.
I have been thinking about this. Today we do a KVM_MEMORY_FAULT_EXIT
to the VMM to handle the request. The other option is to make this
a KVM_EXIT_HYPERCALL with SMC_RSI_SET_RIPAS. But this would leak RSI
implementation to the VMM. The advantage is that the VMM can provide
a clear response RSI_ACCEPT vs RSI_REJECT (including accepting a partial
range) and KVM can satisfy the RMI_RTT_SET_RIPAS.
We may end up doing something similar for Device assignment too, where
the VMM gets a chance to reject any inconsistent device mappings.
Like you mentioned, the VMM can stop the Realm today as an alternate
approach.
Suzuki
>
> There's a FIXME for the case where the RMM rejects a RIPAS change when
> (a portion of) the region. The current RMM implementation isn't spec
> compliant in this case, this should be fixed in a later release.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> Changes since v12:
> * Switch to the new RMM v2.0 RMI_RTT_DATA_UNMAP which can unmap an
> address range.
> Changes since v11:
> * Combine the "Allow VMM to set RIPAS" patch into this one to avoid
> adding functions before they are used.
> * Drop the CAP for setting RIPAS and adapt to changes from previous
> patches.
> Changes since v10:
> * Add comment explaining the assignment of rec->run->exit.ripas_base in
> kvm_complete_ripas_change().
> Changes since v8:
> * Make use of ripas_change() from a previous patch to implement
> realm_set_ipa_state().
> * Update exit.ripas_base after a RIPAS change so that, if instead of
> entering the guest we exit to user space, we don't attempt to repeat
> the RIPAS change (triggering an error from the RMM).
> Changes since v7:
> * Rework the loop in realm_set_ipa_state() to make it clear when the
> 'next' output value of rmi_rtt_set_ripas() is used.
> New patch for v7: The code was previously split awkwardly between two
> other patches.
> ---
> arch/arm64/include/asm/kvm_rmi.h | 6 +
> arch/arm64/kvm/mmu.c | 8 +-
> arch/arm64/kvm/rmi.c | 459 +++++++++++++++++++++++++++++++
> 3 files changed, 470 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
> index 7bec3a3976e7..46b0cbe6c202 100644
> --- a/arch/arm64/include/asm/kvm_rmi.h
> +++ b/arch/arm64/include/asm/kvm_rmi.h
> @@ -96,6 +96,12 @@ int kvm_rec_enter(struct kvm_vcpu *vcpu);
> int kvm_rec_pre_enter(struct kvm_vcpu *vcpu);
> int handle_rec_exit(struct kvm_vcpu *vcpu, int rec_run_status);
>
> +void kvm_realm_unmap_range(struct kvm *kvm,
> + unsigned long ipa,
> + unsigned long size,
> + bool unmap_private,
> + bool may_block);
> +
> static inline bool kvm_realm_is_private_address(struct realm *realm,
> unsigned long addr)
> {
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 41152abf55b2..b705ad6c6c8b 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -319,6 +319,7 @@ static void invalidate_icache_guest_page(void *va, size_t size)
> * @start: The intermediate physical base address of the range to unmap
> * @size: The size of the area to unmap
> * @may_block: Whether or not we are permitted to block
> + * @only_shared: If true then protected mappings should not be unmapped
> *
> * Clear a range of stage-2 mappings, lowering the various ref-counts. Must
> * be called while holding mmu_lock (unless for freeing the stage2 pgd before
> @@ -326,7 +327,7 @@ static void invalidate_icache_guest_page(void *va, size_t size)
> * with things behind our backs.
> */
> static void __unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size,
> - bool may_block)
> + bool may_block, bool only_shared)
> {
> struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
> phys_addr_t end = start + size;
> @@ -340,7 +341,7 @@ static void __unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64
> void kvm_stage2_unmap_range(struct kvm_s2_mmu *mmu, phys_addr_t start,
> u64 size, bool may_block)
> {
> - __unmap_stage2_range(mmu, start, size, may_block);
> + __unmap_stage2_range(mmu, start, size, may_block, false);
> }
>
> void kvm_stage2_flush_range(struct kvm_s2_mmu *mmu, phys_addr_t addr, phys_addr_t end)
> @@ -2241,7 +2242,8 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range)
>
> __unmap_stage2_range(&kvm->arch.mmu, range->start << PAGE_SHIFT,
> (range->end - range->start) << PAGE_SHIFT,
> - range->may_block);
> + range->may_block,
> + !(range->attr_filter & KVM_FILTER_PRIVATE));
>
> kvm_nested_s2_unmap(kvm, range->may_block);
> return false;
> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
> index ee8aab098117..13eed6f0b9eb 100644
> --- a/arch/arm64/kvm/rmi.c
> +++ b/arch/arm64/kvm/rmi.c
> @@ -251,6 +251,88 @@ static int undelegate_page(phys_addr_t phys)
> return undelegate_range(phys, PAGE_SIZE);
> }
>
> +static int find_map_level(struct realm *realm,
> + unsigned long start,
> + unsigned long end)
> +{
> + int level = RMM_RTT_MAX_LEVEL;
> +
> + while (level > get_start_level(realm)) {
> + unsigned long map_size = rmi_rtt_level_mapsize(level - 1);
> +
> + if (!IS_ALIGNED(start, map_size) ||
> + (start + map_size) > end)
> + break;
> +
> + level--;
> + }
> +
> + return level;
> +}
> +
> +static unsigned long level_to_size(int level)
> +{
> + switch (level) {
> + case 0:
> + return PAGE_SIZE;
> + case 1:
> + return PMD_SIZE;
> + case 2:
> + return PUD_SIZE;
> + case 3:
> + return P4D_SIZE;
> + }
> + WARN_ON(1);
> + return 0;
> +}
> +
> +static int undelegate_range_desc(unsigned long desc)
> +{
> + unsigned long size = level_to_size(RMI_ADDR_RANGE_SIZE(desc));
> + unsigned long count = RMI_ADDR_RANGE_COUNT(desc);
> + unsigned long addr = RMI_ADDR_RANGE_ADDR(desc);
> + unsigned long state = RMI_ADDR_RANGE_STATE(desc);
> +
> + if (state == RMI_OP_MEM_UNDELEGATED)
> + return 0;
> +
> + return undelegate_range(addr, size * count);
> +}
> +
> +static phys_addr_t alloc_delegated_granule(struct kvm_mmu_memory_cache *mc)
> +{
> + phys_addr_t phys;
> + void *virt;
> +
> + if (mc) {
> + virt = kvm_mmu_memory_cache_alloc(mc);
> + } else {
> + virt = (void *)__get_free_page(GFP_ATOMIC | __GFP_ZERO |
> + __GFP_ACCOUNT);
> + }
> +
> + if (!virt)
> + return PHYS_ADDR_MAX;
> +
> + phys = virt_to_phys(virt);
> + if (delegate_page(phys)) {
> + free_page((unsigned long)virt);
> + return PHYS_ADDR_MAX;
> + }
> +
> + return phys;
> +}
> +
> +static phys_addr_t alloc_rtt(struct kvm_mmu_memory_cache *mc)
> +{
> + phys_addr_t phys = alloc_delegated_granule(mc);
> +
> + if (phys != PHYS_ADDR_MAX)
> + kvm_account_pgtable_pages(phys_to_virt(phys), 1);
> +
> + return phys;
> +}
> +
> static int free_delegated_page(phys_addr_t phys)
> {
> if (WARN_ON(undelegate_page(phys))) {
> @@ -271,6 +353,32 @@ static void free_rtt(phys_addr_t phys)
> kvm_account_pgtable_pages(phys_to_virt(phys), -1);
> }
>
> +static int realm_rtt_create(struct realm *realm,
> + unsigned long addr,
> + int level,
> + phys_addr_t phys)
> +{
> + addr = ALIGN_DOWN(addr, rmi_rtt_level_mapsize(level - 1));
> + return rmi_rtt_create(virt_to_phys(realm->rd), phys, addr, level);
> +}
> +
> +static int realm_rtt_fold(struct realm *realm,
> + unsigned long addr,
> + int level,
> + phys_addr_t *rtt_granule)
> +{
> + unsigned long out_rtt;
> + int ret;
> +
> + addr = ALIGN_DOWN(addr, rmi_rtt_level_mapsize(level - 1));
> + ret = rmi_rtt_fold(virt_to_phys(realm->rd), addr, level, &out_rtt);
> +
> + if (rtt_granule)
> + *rtt_granule = out_rtt;
> +
> + return ret;
> +}
> +
> static int realm_rtt_destroy(struct realm *realm, unsigned long addr,
> int level, phys_addr_t *rtt_granule,
> unsigned long *next_addr)
> @@ -286,6 +394,38 @@ static int realm_rtt_destroy(struct realm *realm, unsigned long addr,
> return ret;
> }
>
> +static int realm_create_rtt_levels(struct realm *realm,
> + unsigned long ipa,
> + int level,
> + int max_level,
> + struct kvm_mmu_memory_cache *mc)
> +{
> + while (level++ < max_level) {
> + phys_addr_t rtt = alloc_rtt(mc);
> + int ret;
> +
> + if (rtt == PHYS_ADDR_MAX)
> + return -ENOMEM;
> +
> + ret = realm_rtt_create(realm, ipa, level, rtt);
> + if (RMI_RETURN_STATUS(ret) == RMI_ERROR_RTT &&
> + RMI_RETURN_INDEX(ret) == level - 1) {
> + /* The RTT already exists, continue */
> + free_rtt(rtt);
> + continue;
> + }
> +
> + if (ret) {
> + WARN(1, "Failed to create RTT at level %d: %d\n",
> + level, ret);
> + free_rtt(rtt);
> + return -ENXIO;
> + }
> + }
> +
> + return 0;
> +}
> +
> static int realm_tear_down_rtt_level(struct realm *realm, int level,
> unsigned long start, unsigned long end)
> {
> @@ -380,6 +520,62 @@ static int realm_tear_down_rtt_range(struct realm *realm,
> start, end);
> }
>
> +/*
> + * Returns 0 on successful fold, a negative value on error, a positive value if
> + * we were not able to fold all tables at this level.
> + */
> +static int realm_fold_rtt_level(struct realm *realm, int level,
> + unsigned long start, unsigned long end)
> +{
> + int not_folded = 0;
> + ssize_t map_size;
> + unsigned long addr, next_addr;
> +
> + if (WARN_ON(level > RMM_RTT_MAX_LEVEL))
> + return -EINVAL;
> +
> + map_size = rmi_rtt_level_mapsize(level - 1);
> +
> + for (addr = start; addr < end; addr = next_addr) {
> + phys_addr_t rtt_granule;
> + int ret;
> + unsigned long align_addr = ALIGN(addr, map_size);
> +
> + next_addr = ALIGN(addr + 1, map_size);
> +
> + ret = realm_rtt_fold(realm, align_addr, level, &rtt_granule);
> +
> + switch (RMI_RETURN_STATUS(ret)) {
> + case RMI_SUCCESS:
> + free_rtt(rtt_granule);
> + break;
> + case RMI_ERROR_RTT:
> + if (level == RMM_RTT_MAX_LEVEL ||
> + RMI_RETURN_INDEX(ret) < level) {
> + not_folded++;
> + break;
> + }
> + /* Recurse a level deeper */
> + ret = realm_fold_rtt_level(realm,
> + level + 1,
> + addr,
> + next_addr);
> + if (ret < 0) {
> + return ret;
> + } else if (ret == 0) {
> + /* Try again at this level */
> + next_addr = addr;
> + }
> + break;
> + default:
> + WARN_ON(1);
> + return -ENXIO;
> + }
> + }
> +
> + return not_folded;
> +}
> +
> void kvm_realm_destroy_rtts(struct kvm *kvm)
> {
> struct realm *realm = &kvm->arch.realm;
> @@ -388,12 +584,272 @@ void kvm_realm_destroy_rtts(struct kvm *kvm)
> WARN_ON(realm_tear_down_rtt_range(realm, 0, (1UL << ia_bits)));
> }
>
> +static void realm_unmap_shared_range(struct kvm *kvm,
> + int level,
> + unsigned long start,
> + unsigned long end,
> + bool may_block)
> +{
> + struct realm *realm = &kvm->arch.realm;
> + unsigned long rd = virt_to_phys(realm->rd);
> + ssize_t map_size = rmi_rtt_level_mapsize(level);
> + unsigned long next_addr, addr;
> + unsigned long shared_bit = BIT(realm->ia_bits - 1);
> +
> + if (WARN_ON(level > RMM_RTT_MAX_LEVEL))
> + return;
> +
> + start |= shared_bit;
> + end |= shared_bit;
> +
> + for (addr = start; addr < end; addr = next_addr) {
> + unsigned long align_addr = ALIGN(addr, map_size);
> + int ret;
> +
> + next_addr = ALIGN(addr + 1, map_size);
> +
> + if (align_addr != addr || next_addr > end) {
> + /* Need to recurse deeper */
> + if (addr < align_addr)
> + next_addr = align_addr;
> + realm_unmap_shared_range(kvm, level + 1, addr,
> + min(next_addr, end),
> + may_block);
> + continue;
> + }
> +
> + ret = rmi_rtt_unmap_unprotected(rd, addr, level, &next_addr);
> + switch (RMI_RETURN_STATUS(ret)) {
> + case RMI_SUCCESS:
> + break;
> + case RMI_ERROR_RTT:
> + if (next_addr == addr) {
> + /*
> + * There's a mapping here, but it's not a block
> + * mapping, so reset next_addr to the next block
> + * boundary and recurse to clear out the pages
> + * one level deeper.
> + */
> + next_addr = ALIGN(addr + 1, map_size);
> + realm_unmap_shared_range(kvm, level + 1, addr,
> + next_addr,
> + may_block);
> + }
> + break;
> + default:
> + WARN_ON(1);
> + return;
> + }
> +
> + if (may_block)
> + cond_resched_rwlock_write(&kvm->mmu_lock);
> + }
> +
> + realm_fold_rtt_level(realm, get_start_level(realm) + 1,
> + start, end);
> +}
> +
> +static void realm_unmap_private_range(struct kvm *kvm,
> + unsigned long start,
> + unsigned long end,
> + bool may_block)
> +{
> + struct realm *realm = &kvm->arch.realm;
> + unsigned long rd = virt_to_phys(realm->rd);
> + unsigned long next_addr, addr;
> + int ret;
> +
> + for (addr = start; addr < end; addr = next_addr) {
> + unsigned long out_range;
> + unsigned long flags = RMI_ADDR_TYPE_SINGLE;
> + /* TODO: Optimise using RMI_ADDR_TYPE_LIST */
> +
> +retry:
> + ret = rmi_rtt_data_unmap(rd, addr, end, flags, 0,
> + &next_addr, &out_range, NULL);
> +
> + if (RMI_RETURN_STATUS(ret) == RMI_ERROR_RTT) {
> + phys_addr_t rtt;
> +
> + if (next_addr > addr)
> + continue; /* UNASSIGNED */
> +
> + rtt = alloc_rtt(NULL);
> + if (WARN_ON(rtt == PHYS_ADDR_MAX))
> + return;
> + ret = realm_rtt_create(realm, addr,
> + RMI_RETURN_INDEX(ret) + 1, rtt);
> + if (WARN_ON(ret)) {
> + free_rtt(rtt);
> + return;
> + }
> + goto retry;
> + } else if (WARN_ON(ret)) {
> + continue;
> + }
> +
> + ret = undelegate_range_desc(out_range);
> + if (WARN_ON(ret))
> + break;
> +
> + if (may_block)
> + cond_resched_rwlock_write(&kvm->mmu_lock);
> + }
> +
> + realm_fold_rtt_level(realm, get_start_level(realm) + 1,
> + start, end);
> +}
> +
> +void kvm_realm_unmap_range(struct kvm *kvm, unsigned long start,
> + unsigned long size, bool unmap_private,
> + bool may_block)
> +{
> + unsigned long end = start + size;
> + struct realm *realm = &kvm->arch.realm;
> +
> + if (!kvm_realm_is_created(kvm))
> + return;
> +
> + end = min(BIT(realm->ia_bits - 1), end);
> +
> + realm_unmap_shared_range(kvm, find_map_level(realm, start, end),
> + start, end, may_block);
> + if (unmap_private)
> + realm_unmap_private_range(kvm, start, end, may_block);
> +}
> +
> +enum ripas_action {
> + RIPAS_INIT,
> + RIPAS_SET,
> +};
> +
> +static int ripas_change(struct kvm *kvm,
> + struct kvm_vcpu *vcpu,
> + unsigned long ipa,
> + unsigned long end,
> + enum ripas_action action,
> + unsigned long *top_ipa)
> +{
> + struct realm *realm = &kvm->arch.realm;
> + phys_addr_t rd_phys = virt_to_phys(realm->rd);
> + phys_addr_t rec_phys;
> + struct kvm_mmu_memory_cache *memcache = NULL;
> + int ret = 0;
> +
> + if (vcpu) {
> + rec_phys = virt_to_phys(vcpu->arch.rec.rec_page);
> + memcache = &vcpu->arch.mmu_page_cache;
> +
> + WARN_ON(action != RIPAS_SET);
> + } else {
> + WARN_ON(action != RIPAS_INIT);
> + }
> +
> + while (ipa < end) {
> + unsigned long next = ~0;
> +
> + switch (action) {
> + case RIPAS_INIT:
> + ret = rmi_rtt_init_ripas(rd_phys, ipa, end, &next);
> + break;
> + case RIPAS_SET:
> + ret = rmi_rtt_set_ripas(rd_phys, rec_phys, ipa, end,
> + &next);
> + break;
> + }
> +
> + switch (RMI_RETURN_STATUS(ret)) {
> + case RMI_SUCCESS:
> + ipa = next;
> + break;
> + case RMI_ERROR_RTT: {
> + int err_level = RMI_RETURN_INDEX(ret);
> + int level = find_map_level(realm, ipa, end);
> +
> + if (err_level >= level) {
> + /* FIXME: Ugly hack to skip regions which are
> + * already RIPAS_RAM
> + */
> + ipa += PAGE_SIZE;
> + break;
> + return -EINVAL;
> + }
> +
> + ret = realm_create_rtt_levels(realm, ipa, err_level,
> + level, memcache);
> + if (ret)
> + return ret;
> + /* Retry with the RTT levels in place */
> + break;
> + }
> + default:
> + WARN_ON(1);
> + return -ENXIO;
> + }
> + }
> +
> + if (top_ipa)
> + *top_ipa = ipa;
> +
> + return 0;
> +}
> +
> +static int realm_set_ipa_state(struct kvm_vcpu *vcpu,
> + unsigned long start,
> + unsigned long end,
> + unsigned long ripas,
> + unsigned long *top_ipa)
> +{
> + struct kvm *kvm = vcpu->kvm;
> + int ret = ripas_change(kvm, vcpu, start, end, RIPAS_SET, top_ipa);
> +
> + if (ripas == RMI_EMPTY && *top_ipa != start)
> + realm_unmap_private_range(kvm, start, *top_ipa, false);
> +
> + return ret;
> +}
> +
> static int realm_ensure_created(struct kvm *kvm)
> {
> /* Provided in later patch */
> return -ENXIO;
> }
>
> +static void kvm_complete_ripas_change(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = vcpu->kvm;
> + struct realm_rec *rec = &vcpu->arch.rec;
> + unsigned long base = rec->run->exit.ripas_base;
> + unsigned long top = rec->run->exit.ripas_top;
> + unsigned long ripas = rec->run->exit.ripas_value;
> + unsigned long top_ipa;
> + int ret;
> +
> + do {
> + kvm_mmu_topup_memory_cache(&vcpu->arch.mmu_page_cache,
> + kvm_mmu_cache_min_pages(vcpu->arch.hw_mmu));
> + write_lock(&kvm->mmu_lock);
> + ret = realm_set_ipa_state(vcpu, base, top, ripas, &top_ipa);
> + write_unlock(&kvm->mmu_lock);
> +
> + if (WARN_RATELIMIT(ret && ret != -ENOMEM,
> + "Unable to satisfy RIPAS_CHANGE for %#lx - %#lx, ripas: %#lx\n",
> + base, top, ripas))
> + break;
> +
> + base = top_ipa;
> + } while (base < top);
> +
> + /*
> + * If this function is called again before the REC_ENTER call then
> + * avoid calling realm_set_ipa_state() again by changing to the value
> + * of ripas_base for the part that has already been covered. The RMM
> + * ignores the contains of the rec_exit structure so this doesn't
> + * affect the RMM.
> + */
> + rec->run->exit.ripas_base = base;
> +}
> +
> /*
> * kvm_rec_pre_enter - Complete operations before entering a REC
> *
> @@ -419,6 +875,9 @@ int kvm_rec_pre_enter(struct kvm_vcpu *vcpu)
> for (int i = 0; i < REC_RUN_GPRS; i++)
> rec->run->enter.gprs[i] = vcpu_get_reg(vcpu, i);
> break;
> + case RMI_EXIT_RIPAS_CHANGE:
> + kvm_complete_ripas_change(vcpu);
> + break;
> }
>
> return 1;
^ permalink raw reply
* Re: [PATCH 1/2] x86/virt/tdx: Use PFN directly for mapping guest private memory
From: Kiryl Shutsemau @ 2026-03-20 12:59 UTC (permalink / raw)
To: Edgecombe, Rick P
Cc: Zhao, Yan Y, kvm@vger.kernel.org, linux-coco@lists.linux.dev,
Huang, Kai, Li, Xiaoyao, dave.hansen@linux.intel.com,
linux-kernel@vger.kernel.org, binbin.wu@linux.intel.com,
seanjc@google.com, pbonzini@redhat.com, mingo@redhat.com,
ackerleytng@google.com, Yamahata, Isaku, sagis@google.com,
Annapurve, Vishal, bp@alien8.de, tglx@kernel.org,
yilun.xu@linux.intel.com, x86@kernel.org
In-Reply-To: <e3acc801e4e939a0a4794ba777e72f26b27af0d2.camel@intel.com>
On Thu, Mar 19, 2026 at 05:27:48PM +0000, Edgecombe, Rick P wrote:
> On Thu, 2026-03-19 at 12:57 +0000, Kiryl Shutsemau wrote:
> > > Though tdh_mem_page_aug() is an API, it is currently only exported to KVM
> > > and
> > > uses type kvm_pfn_t. So, is it still acceptable to assume flush size to be
> > > PAGE_SIZE? Honoring level will soon be introduced by huge page patches.
> >
> > It caught my eye because previously size to flush was passed down to
> > tdx_clflush_page() in the struct page (although never used there).
> > With switching to pfn, we give up this information and it has to be
> > passed separately. It would be easy to miss that in huge page patches,
> > if we don't pass down level here.
> >
> > >
> > > If you think it needs to be fixed before huge page series, what about fixing
> > > it
> > > in a separate cleanup patch? IMO, it would be better placed after Sean's
> > > cleanup
> > > patch [1], so we can use page_level_size() instead of inventing the wheel.
> >
> > I am okay with a separate patch.
>
> I feel like we argued about this before. But it would be more correct to just
> drop level and make it 4k only until huge pages? Otherwise we are tweaking dead
> behavior.
I guess. But you add one more thing on the list to remember when adding
huge page support. This kind of stuff is easy to miss.
--
Kiryl Shutsemau / Kirill A. Shutemov
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox