* [PATCH v15 16/37] KVM: arm64: CCA: Handle realm MMIO emulation
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
MMIO emulation for a realm cannot be done directly with the VM's
registers as they are protected from the host. However, for emulatable
data aborts, the RMM uses GPRS[0] to provide the read/written value.
We can transfer this from/to the equivalent VCPU's register entry and
then depend on the generic MMIO handling code in KVM.
For a MMIO read, the value is placed in the shared RecExit structure
during kvm_handle_mmio_return() rather than in the VCPU's register
entry.
Signed-off-by: Steven Price <steven.price@arm.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
Changes since v7:
* New comment for rec_exit_sync_dabt() explaining the call to
vcpu_set_reg().
Changes since v5:
* Inject SEA to the guest is an emulatable MMIO access triggers a data
abort.
* kvm_handle_mmio_return() - disable kvm_incr_pc() for a REC (as the PC
isn't under the host's control) and move the REC_ENTER_EMULATED_MMIO
flag setting to this location (as that tells the RMM to skip the
instruction).
---
arch/arm64/kvm/inject_fault.c | 4 +++-
arch/arm64/kvm/mmio.c | 16 ++++++++++++----
arch/arm64/kvm/rmi-exit.c | 15 +++++++++++++++
3 files changed, 30 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
index 89982bd3345f..6492397b73d7 100644
--- a/arch/arm64/kvm/inject_fault.c
+++ b/arch/arm64/kvm/inject_fault.c
@@ -228,7 +228,9 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt, u32 addr)
static void __kvm_inject_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr)
{
- if (vcpu_el1_is_32bit(vcpu))
+ if (unlikely(vcpu_is_rec(vcpu)))
+ vcpu->arch.rec.run->enter.flags |= REC_ENTER_FLAG_INJECT_SEA;
+ else if (vcpu_el1_is_32bit(vcpu))
inject_abt32(vcpu, iabt, addr);
else
inject_abt64(vcpu, iabt, addr);
diff --git a/arch/arm64/kvm/mmio.c b/arch/arm64/kvm/mmio.c
index e2285ed8c91d..a8c125205695 100644
--- a/arch/arm64/kvm/mmio.c
+++ b/arch/arm64/kvm/mmio.c
@@ -6,6 +6,7 @@
#include <linux/kvm_host.h>
#include <asm/kvm_emulate.h>
+#include <linux/arm-smccc-rmi.h>
#include <trace/events/kvm.h>
#include "trace.h"
@@ -138,14 +139,21 @@ int kvm_handle_mmio_return(struct kvm_vcpu *vcpu)
trace_kvm_mmio(KVM_TRACE_MMIO_READ, len, run->mmio.phys_addr,
&data);
data = vcpu_data_host_to_guest(vcpu, data, len);
- vcpu_set_reg(vcpu, kvm_vcpu_dabt_get_rd(vcpu), data);
+
+ if (vcpu_is_rec(vcpu))
+ vcpu->arch.rec.run->enter.gprs[0] = data;
+ else
+ vcpu_set_reg(vcpu, kvm_vcpu_dabt_get_rd(vcpu), data);
}
/*
* The MMIO instruction is emulated and should not be re-executed
* in the guest.
*/
- kvm_incr_pc(vcpu);
+ if (vcpu_is_rec(vcpu))
+ vcpu->arch.rec.run->enter.flags |= REC_ENTER_FLAG_EMULATED_MMIO;
+ else
+ kvm_incr_pc(vcpu);
return 1;
}
@@ -167,14 +175,14 @@ int io_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)
* No valid syndrome? Ask userspace for help if it has
* volunteered to do so, and bail out otherwise.
*
- * In the protected VM case, there isn't much userspace can do
+ * In the protected/realm VM case, there isn't much userspace can do
* though, so directly deliver an exception to the guest.
*/
if (!kvm_vcpu_dabt_isvalid(vcpu)) {
trace_kvm_mmio_nisv(*vcpu_pc(vcpu), esr,
kvm_vcpu_get_hfar(vcpu), fault_ipa);
- if (vcpu_is_protected(vcpu))
+ if (vcpu_is_protected(vcpu) || vcpu_is_rec(vcpu))
return kvm_inject_sea_dabt(vcpu, kvm_vcpu_get_hfar(vcpu));
if (test_bit(KVM_ARCH_FLAG_RETURN_NISV_IO_ABORT_TO_USER,
diff --git a/arch/arm64/kvm/rmi-exit.c b/arch/arm64/kvm/rmi-exit.c
index 973250563d7b..c668cbfa971a 100644
--- a/arch/arm64/kvm/rmi-exit.c
+++ b/arch/arm64/kvm/rmi-exit.c
@@ -23,6 +23,21 @@ static int rec_exit_reason_notimpl(struct kvm_vcpu *vcpu)
static int rec_exit_sync_dabt(struct kvm_vcpu *vcpu)
{
+ struct realm_rec *rec = &vcpu->arch.rec;
+
+ /*
+ * In the case of a write, copy over gprs[0] to the target GPR,
+ * preparing to handle MMIO write fault. The content to be written has
+ * been saved to gprs[0] by the RMM (even if another register was used
+ * by the guest). In the case of normal memory access this is redundant
+ * (the guest will replay the instruction), but the overhead is
+ * minimal.
+ */
+ if (kvm_vcpu_dabt_iswrite(vcpu) && kvm_vcpu_dabt_isvalid(vcpu)) {
+ vcpu_set_reg(vcpu, kvm_vcpu_dabt_get_rd(vcpu),
+ rec->run->exit.gprs[0]);
+ }
+
return kvm_handle_guest_abort(vcpu);
}
--
2.43.0
^ permalink raw reply related
* [PATCH v15 15/37] KVM: arm64: CCA: Handle RMI_EXIT_RIPAS_CHANGE
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
The guest can request that a region of its protected address space is
switched between RIPAS_RAM and RIPAS_EMPTY (and back) using
RSI_IPA_STATE_SET. This causes a guest exit with the
RMI_EXIT_RIPAS_CHANGE code. We treat this as a request to convert a
protected region to unprotected (or back), exiting to the VMM to make
the necessary changes to the guest_memfd and memslot mappings. On the
next entry the RIPAS changes are committed by making RMI_RTT_SET_RIPAS
calls.
The VMM may wish to reject the RIPAS change requested by the guest. For
now it can only do this by no longer scheduling the VCPU as we don't
currently have a usecase for returning that rejection to the guest, but
by postponing the RMI_RTT_SET_RIPAS changes to entry we leave the door
open for adding a new ioctl in the future for this purpose.
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v14:
* Use addition rather than bitwise OR for adding the shared_bit in
realm_unmap_shared_range(), this handles the case where the region
includes the last address (which means 'end' already has the bit
set).
Changes since v13:
* Switch to the new RMI_RTT_UNPROT_UNMAP range-based API.
* Drop ugly hack for RMM bug which errored when the RIPAS was already
set to the desired value.
Changes since v12:
* Switch to the new RMM v2.0 RMI_RTT_DATA_UNMAP which can unmap an
address range.
Changes since v11:
* Combine the "Allow VMM to set RIPAS" patch into this one to avoid
adding functions before they are used.
* Drop the CAP for setting RIPAS and adapt to changes from previous
patches.
Changes since v10:
* Add comment explaining the assignment of rec->run->exit.ripas_base in
kvm_complete_ripas_change().
Changes since v8:
* Make use of ripas_change() from a previous patch to implement
realm_set_ipa_state().
* Update exit.ripas_base after a RIPAS change so that, if instead of
entering the guest we exit to user space, we don't attempt to repeat
the RIPAS change (triggering an error from the RMM).
Changes since v7:
* Rework the loop in realm_set_ipa_state() to make it clear when the
'next' output value of rmi_rtt_set_ripas() is used.
New patch for v7: The code was previously split awkwardly between two
other patches.
---
arch/arm64/include/asm/kvm_rmi.h | 6 +
arch/arm64/kvm/mmu.c | 8 +-
arch/arm64/kvm/rmi.c | 457 +++++++++++++++++++++++++++++++
3 files changed, 468 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
index b1e4cf0f6803..5461c49bea4d 100644
--- a/arch/arm64/include/asm/kvm_rmi.h
+++ b/arch/arm64/include/asm/kvm_rmi.h
@@ -104,6 +104,12 @@ int kvm_rec_enter(struct kvm_vcpu *vcpu);
int kvm_rec_pre_enter(struct kvm_vcpu *vcpu);
int handle_rec_exit(struct kvm_vcpu *vcpu, int rec_run_status);
+void kvm_realm_unmap_range(struct kvm *kvm,
+ unsigned long ipa,
+ unsigned long size,
+ bool unmap_private,
+ bool may_block);
+
static inline bool kvm_realm_is_private_address(struct realm *realm,
unsigned long addr)
{
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index cd06881c1497..dcc2ab08d0e4 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -319,6 +319,7 @@ static void invalidate_icache_guest_page(void *va, size_t size)
* @start: The intermediate physical base address of the range to unmap
* @size: The size of the area to unmap
* @may_block: Whether or not we are permitted to block
+ * @only_shared: If true then protected mappings should not be unmapped
*
* Clear a range of stage-2 mappings, lowering the various ref-counts. Must
* be called while holding mmu_lock (unless for freeing the stage2 pgd before
@@ -326,7 +327,7 @@ static void invalidate_icache_guest_page(void *va, size_t size)
* with things behind our backs.
*/
static void __unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size,
- bool may_block)
+ bool may_block, bool only_shared)
{
struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
phys_addr_t end = start + size;
@@ -343,7 +344,7 @@ void kvm_stage2_unmap_range(struct kvm_s2_mmu *mmu, phys_addr_t start,
if (kvm_vm_is_protected(kvm_s2_mmu_to_kvm(mmu)))
return;
- __unmap_stage2_range(mmu, start, size, may_block);
+ __unmap_stage2_range(mmu, start, size, may_block, false);
}
void kvm_stage2_flush_range(struct kvm_s2_mmu *mmu, phys_addr_t addr, phys_addr_t end)
@@ -2467,7 +2468,8 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range)
__unmap_stage2_range(&kvm->arch.mmu, range->start << PAGE_SHIFT,
(range->end - range->start) << PAGE_SHIFT,
- range->may_block);
+ range->may_block,
+ !(range->attr_filter & KVM_FILTER_PRIVATE));
kvm_nested_s2_unmap(kvm, range->may_block);
return false;
diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
index 430b82c02d15..1ef676eed172 100644
--- a/arch/arm64/kvm/rmi.c
+++ b/arch/arm64/kvm/rmi.c
@@ -34,6 +34,83 @@ static int get_start_level(struct realm *realm)
return 4 - stage2_pgtable_levels(realm->ia_bits);
}
+static int find_map_level(struct realm *realm,
+ unsigned long start,
+ unsigned long end)
+{
+ int level = KVM_PGTABLE_LAST_LEVEL;
+
+ while (level > get_start_level(realm)) {
+ unsigned long map_size = rmi_rtt_level_mapsize(level - 1);
+
+ if (!IS_ALIGNED(start, map_size) ||
+ (start + map_size) > end)
+ break;
+
+ level--;
+ }
+
+ return level;
+}
+
+static unsigned long rmi_range_entry_size(int rmi_size)
+{
+ if (WARN_ON(rmi_size > KVM_PGTABLE_LAST_LEVEL))
+ return 0;
+
+ return kvm_granule_size(KVM_PGTABLE_LAST_LEVEL - rmi_size);
+}
+
+static int undelegate_range_desc(unsigned long desc)
+{
+ unsigned long size = rmi_range_entry_size(RMI_ADDR_RANGE_SIZE(desc));
+ unsigned long count = RMI_ADDR_RANGE_COUNT(desc);
+ unsigned long addr = RMI_ADDR_RANGE_ADDR(desc);
+ unsigned long state = RMI_ADDR_RANGE_STATE(desc);
+
+ if (state == RMI_OP_MEM_UNDELEGATED)
+ return 0;
+
+ if (size * count == 0)
+ return 0;
+
+ return rmi_undelegate_range(addr, size * count);
+}
+
+static phys_addr_t alloc_delegated_granule(struct kvm_mmu_memory_cache *mc)
+{
+ phys_addr_t phys;
+ void *virt;
+
+ if (mc) {
+ virt = kvm_mmu_memory_cache_alloc(mc);
+ } else {
+ virt = (void *)__get_free_page(GFP_ATOMIC | __GFP_ZERO |
+ __GFP_ACCOUNT);
+ }
+
+ if (!virt)
+ return PHYS_ADDR_MAX;
+
+ phys = virt_to_phys(virt);
+ if (rmi_delegate_page(phys)) {
+ free_page((unsigned long)virt);
+ return PHYS_ADDR_MAX;
+ }
+
+ return phys;
+}
+
+static phys_addr_t alloc_rtt(struct kvm_mmu_memory_cache *mc)
+{
+ phys_addr_t phys = alloc_delegated_granule(mc);
+
+ if (phys != PHYS_ADDR_MAX)
+ kvm_account_pgtable_pages(phys_to_virt(phys), 1);
+
+ return phys;
+}
+
static void free_rtt(phys_addr_t phys)
{
if (free_delegated_page(phys))
@@ -42,6 +119,32 @@ static void free_rtt(phys_addr_t phys)
kvm_account_pgtable_pages(phys_to_virt(phys), -1);
}
+static int realm_rtt_create(struct realm *realm,
+ unsigned long addr,
+ int level,
+ phys_addr_t phys)
+{
+ addr = ALIGN_DOWN(addr, rmi_rtt_level_mapsize(level - 1));
+ return rmi_rtt_create(virt_to_phys(realm->rd), phys, addr, level);
+}
+
+static int realm_rtt_fold(struct realm *realm,
+ unsigned long addr,
+ int level,
+ phys_addr_t *rtt_granule)
+{
+ unsigned long out_rtt;
+ int ret;
+
+ addr = ALIGN_DOWN(addr, rmi_rtt_level_mapsize(level - 1));
+ ret = rmi_rtt_fold(virt_to_phys(realm->rd), addr, level, &out_rtt);
+
+ if (rtt_granule)
+ *rtt_granule = out_rtt;
+
+ return ret;
+}
+
/*
* realm_rtt_destroy - Destroy an RTT at @level for @addr.
*
@@ -65,6 +168,38 @@ static int realm_rtt_destroy(struct realm *realm, unsigned long addr,
return ret;
}
+static int realm_create_rtt_levels(struct realm *realm,
+ unsigned long ipa,
+ int level,
+ int max_level,
+ struct kvm_mmu_memory_cache *mc)
+{
+ while (level++ < max_level) {
+ phys_addr_t rtt = alloc_rtt(mc);
+ int ret;
+
+ if (rtt == PHYS_ADDR_MAX)
+ return -ENOMEM;
+
+ ret = realm_rtt_create(realm, ipa, level, rtt);
+ if (RMI_RETURN_STATUS(ret) == RMI_ERROR_RTT &&
+ RMI_RETURN_INDEX(ret) == level - 1) {
+ /* The RTT already exists, continue */
+ free_rtt(rtt);
+ continue;
+ }
+
+ if (ret) {
+ WARN(1, "Failed to create RTT at level %d: %d\n",
+ level, ret);
+ free_rtt(rtt);
+ return -ENXIO;
+ }
+ }
+
+ return 0;
+}
+
static int realm_tear_down_rtt_level(struct realm *realm, int level,
unsigned long start, unsigned long end)
{
@@ -159,6 +294,62 @@ static int realm_tear_down_rtt_range(struct realm *realm,
start, end);
}
+/*
+ * Returns 0 on successful fold, a negative value on error, a positive value if
+ * we were not able to fold all tables at this level.
+ */
+static int realm_fold_rtt_level(struct realm *realm, int level,
+ unsigned long start, unsigned long end)
+{
+ int not_folded = 0;
+ ssize_t map_size;
+ unsigned long addr, next_addr;
+
+ if (WARN_ON(level > KVM_PGTABLE_LAST_LEVEL))
+ return -EINVAL;
+
+ map_size = rmi_rtt_level_mapsize(level - 1);
+
+ for (addr = start; addr < end; addr = next_addr) {
+ phys_addr_t rtt_granule;
+ int ret;
+ unsigned long align_addr = ALIGN(addr, map_size);
+
+ next_addr = ALIGN(addr + 1, map_size);
+
+ ret = realm_rtt_fold(realm, align_addr, level, &rtt_granule);
+
+ switch (RMI_RETURN_STATUS(ret)) {
+ case RMI_SUCCESS:
+ free_rtt(rtt_granule);
+ break;
+ case RMI_ERROR_RTT:
+ if (level == KVM_PGTABLE_LAST_LEVEL ||
+ RMI_RETURN_INDEX(ret) < level) {
+ not_folded++;
+ break;
+ }
+ /* Recurse a level deeper */
+ ret = realm_fold_rtt_level(realm,
+ level + 1,
+ addr,
+ next_addr);
+ if (ret < 0) {
+ return ret;
+ } else if (ret == 0) {
+ /* Try again at this level */
+ next_addr = addr;
+ }
+ break;
+ default:
+ WARN_ON(1);
+ return -ENXIO;
+ }
+ }
+
+ return not_folded;
+}
+
static int realm_destroy_rtts(struct kvm *kvm)
{
struct realm *realm = &kvm->arch.realm;
@@ -203,6 +394,269 @@ int kvm_realm_teardown_stage2(struct kvm *kvm)
return realm_destroy_rtts(kvm);
}
+static void realm_unmap_shared_range(struct kvm *kvm,
+ unsigned long start,
+ unsigned long end,
+ bool may_block)
+{
+ struct realm *realm = &kvm->arch.realm;
+ unsigned long rd = virt_to_phys(realm->rd);
+ unsigned long next_addr, addr;
+ unsigned long shared_bit = BIT(realm->ia_bits - 1);
+
+ if (start >= end)
+ return;
+
+ start += shared_bit;
+ end += shared_bit;
+
+ for (addr = start; addr < end; addr = next_addr) {
+ int ret;
+
+ ret = rmi_rtt_unprot_unmap(rd, addr, end, RMI_ADDR_TYPE_NONE,
+ 0, &next_addr, NULL, NULL);
+ switch (RMI_RETURN_STATUS(ret)) {
+ case RMI_SUCCESS:
+ break;
+ case RMI_ERROR_RTT: {
+ int err_level = RMI_RETURN_INDEX(ret);
+ int level = find_map_level(realm, addr, end);
+
+ if (err_level >= level) {
+ /* Nothing present, so skip */
+ next_addr = addr + rmi_rtt_level_mapsize(err_level);
+ break;
+ }
+
+ ret = realm_create_rtt_levels(realm, addr, err_level,
+ level, NULL);
+ if (WARN_ON(ret))
+ return;
+ /* Retry with the RTT levels in place */
+ next_addr = addr;
+ break;
+ }
+ default:
+ WARN_ON(1);
+ return;
+ }
+
+ if (may_block)
+ cond_resched_rwlock_write(&kvm->mmu_lock);
+ }
+
+ realm_fold_rtt_level(realm, get_start_level(realm) + 1,
+ start, end);
+}
+
+static void realm_unmap_private_range(struct kvm *kvm,
+ unsigned long start,
+ unsigned long end,
+ bool may_block)
+{
+ struct realm *realm = &kvm->arch.realm;
+ unsigned long rd = virt_to_phys(realm->rd);
+ unsigned long next_addr, addr;
+ int ret;
+
+ for (addr = start; addr < end; addr = next_addr) {
+ unsigned long out_range;
+ unsigned long flags = RMI_ADDR_TYPE_SINGLE;
+ /* TODO: Optimise using RMI_ADDR_TYPE_LIST */
+
+retry:
+ ret = rmi_rtt_data_unmap(rd, addr, end, flags, 0,
+ &next_addr, &out_range, NULL);
+
+ if (RMI_RETURN_STATUS(ret) == RMI_ERROR_RTT) {
+ phys_addr_t rtt;
+
+ if (next_addr > addr)
+ continue; /* UNASSIGNED */
+
+ rtt = alloc_rtt(NULL);
+ if (WARN_ON(rtt == PHYS_ADDR_MAX))
+ return;
+ ret = realm_rtt_create(realm, addr,
+ RMI_RETURN_INDEX(ret) + 1, rtt);
+ if (WARN_ON(ret)) {
+ free_rtt(rtt);
+ return;
+ }
+ goto retry;
+ } else if (WARN_ON(ret)) {
+ continue;
+ }
+
+ ret = undelegate_range_desc(out_range);
+ if (WARN_ON(ret))
+ break;
+
+ if (may_block)
+ cond_resched_rwlock_write(&kvm->mmu_lock);
+ }
+
+ realm_fold_rtt_level(realm, get_start_level(realm) + 1,
+ start, end);
+}
+
+void kvm_realm_unmap_range(struct kvm *kvm, unsigned long start,
+ unsigned long size, bool unmap_private,
+ bool may_block)
+{
+ unsigned long end = start + size;
+ struct realm *realm = &kvm->arch.realm;
+
+ if (!kvm_realm_is_created(kvm))
+ return;
+
+ end = min(BIT(realm->ia_bits - 1), end);
+
+ realm_unmap_shared_range(kvm, start, end, may_block);
+ if (unmap_private)
+ realm_unmap_private_range(kvm, start, end, may_block);
+}
+
+enum ripas_action {
+ RIPAS_INIT,
+ RIPAS_SET,
+};
+
+static int ripas_change(struct kvm *kvm,
+ struct kvm_vcpu *vcpu,
+ unsigned long ipa,
+ unsigned long end,
+ enum ripas_action action,
+ unsigned long *top_ipa)
+{
+ struct realm *realm = &kvm->arch.realm;
+ phys_addr_t rd_phys = virt_to_phys(realm->rd);
+ phys_addr_t rec_phys;
+ struct kvm_mmu_memory_cache *memcache = NULL;
+ int ret = 0;
+
+ if (vcpu) {
+ rec_phys = virt_to_phys(vcpu->arch.rec.rec_page);
+ memcache = &vcpu->arch.mmu_page_cache;
+
+ WARN_ON(action != RIPAS_SET);
+ } else {
+ WARN_ON(action != RIPAS_INIT);
+ }
+
+ while (ipa < end) {
+ unsigned long next = ~0;
+
+ switch (action) {
+ case RIPAS_INIT:
+ ret = rmi_rtt_init_ripas(rd_phys, ipa, end, &next);
+ break;
+ case RIPAS_SET:
+ ret = rmi_rtt_set_ripas(rd_phys, rec_phys, ipa, end,
+ &next);
+ break;
+ }
+
+ switch (RMI_RETURN_STATUS(ret)) {
+ case RMI_SUCCESS:
+ ipa = next;
+ break;
+ case RMI_ERROR_RTT: {
+ int err_level = RMI_RETURN_INDEX(ret);
+ int level = find_map_level(realm, ipa, end);
+
+ /*
+ * If the operation failed at deeper level than
+ * what is required for the address range, this
+ * implies encountering an unexpected entry,
+ * (e.g., RIPAS_DESTROYED), which the RMM prevents
+ * us from modifying. This is only applicable for
+ * RMI_RTT_INIT_RIPAS. All the other requests
+ * are generated by the Realm and thus RMM should
+ * be able to allow the transition.
+ */
+ if (action == RIPAS_INIT && WARN_ON_ONCE(err_level >= level))
+ return -ENXIO;
+
+ ret = realm_create_rtt_levels(realm, ipa, err_level,
+ level, memcache);
+ if (ret)
+ goto out;
+ /* Retry with the RTT levels in place */
+ break;
+ }
+ default:
+ WARN_ON(1);
+ ret = -ENXIO;
+ goto out;
+ }
+ }
+
+out:
+ if (top_ipa)
+ *top_ipa = ipa;
+
+ return ret;
+}
+
+static int realm_set_ipa_state(struct kvm_vcpu *vcpu,
+ unsigned long start,
+ unsigned long end,
+ unsigned long ripas,
+ unsigned long *top_ipa)
+{
+ struct kvm *kvm = vcpu->kvm;
+ int ret = ripas_change(kvm, vcpu, start, end, RIPAS_SET, top_ipa);
+
+ if (!ret && ripas == RMI_EMPTY && *top_ipa != start)
+ realm_unmap_private_range(kvm, start, *top_ipa, false);
+
+ return ret;
+}
+
+static void kvm_complete_ripas_change(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ struct realm_rec *rec = &vcpu->arch.rec;
+ unsigned long base = rec->run->exit.ripas_base;
+ unsigned long top = rec->run->exit.ripas_top;
+ unsigned long ripas = rec->run->exit.ripas_value;
+ unsigned long top_ipa = base;
+ int ret;
+
+ do {
+ kvm_mmu_topup_memory_cache(&vcpu->arch.mmu_page_cache,
+ kvm_mmu_cache_min_pages(vcpu->arch.hw_mmu));
+ write_lock(&kvm->mmu_lock);
+ ret = realm_set_ipa_state(vcpu, base, top, ripas, &top_ipa);
+ write_unlock(&kvm->mmu_lock);
+
+ if (ret == -ENOMEM) {
+ /* If no progress, then stop */
+ if (top_ipa == base)
+ break;
+ base = top_ipa;
+ continue;
+ }
+
+ if (WARN_RATELIMIT(ret,
+ "Unable to satisfy RIPAS_CHANGE for %#lx - %#lx, ripas: %#lx\n",
+ base, top, ripas))
+ break;
+
+ base = top_ipa;
+ } while (base < top);
+
+ /*
+ * If this function is called again before the REC_ENTER call then
+ * avoid calling realm_set_ipa_state() again by changing to the value
+ * of ripas_base for the part that has already been covered. The RMM
+ * ignores the contains of the rec_exit structure so this doesn't
+ * affect the RMM.
+ */
+ rec->run->exit.ripas_base = base;
+}
+
/*
* kvm_rec_pre_enter - Complete operations before entering a REC
*
@@ -227,6 +681,9 @@ int kvm_rec_pre_enter(struct kvm_vcpu *vcpu)
for (int i = 0; i < REC_RUN_GPRS; i++)
rec->run->enter.gprs[i] = vcpu_get_reg(vcpu, i);
break;
+ case RMI_EXIT_RIPAS_CHANGE:
+ kvm_complete_ripas_change(vcpu);
+ break;
}
return 1;
--
2.43.0
^ permalink raw reply related
* [PATCH v15 14/37] KVM: arm64: CCA: Handle realm enter/exit
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
Entering a realm is done using a SMC call to the RMM. On exit the
exit-codes need to be handled slightly differently to the normal KVM
path so define our own functions for realm enter/exit and hook them
in if the guest is a realm guest.
Signed-off-by: Steven Price <steven.price@arm.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
---
Changes since v13:
* The RMM is now required to provide an ESR value with the correct
information to emulate MMIO, so we no longer need to hardcode 0s in
rec_exit_sys_reg().
* The PSCI changes mean that there is a potential race when turning on
a VCPU which can cause a RMI_ERROR_REC return. Exit to user space
with -EAGAIN in this case.
Changes since v12:
* Call guest_state_{enter,exit}_irqoff() around rmi_rec_enter().
* Add handling of the IRQ exception case where IRQs need to be briefly
enabled before exiting guest timing.
Changes since v8:
* Introduce kvm_rec_pre_enter() called before entering an atomic
section to handle operations that might require memory allocation
(specifically completing a RIPAS change introduced in a later patch).
* Updates to align with upstream changes to hpfar_el2 which now (ab)uses
HPFAR_EL2_NS as a valid flag.
* Fix exit reason when racing with PSCI shutdown to return
KVM_EXIT_SHUTDOWN rather than KVM_EXIT_UNKNOWN.
Changes since v7:
* A return of 0 from kvm_handle_sys_reg() doesn't mean the register has
been read (although that can never happen in the current code). Tidy
up the condition to handle any future refactoring.
Changes since v6:
* Use vcpu_err() rather than pr_err/kvm_err when there is an associated
vcpu to the error.
* Return -EFAULT for KVM_EXIT_MEMORY_FAULT as per the documentation for
this exit type.
* Split code handling a RIPAS change triggered by the guest to the
following patch.
Changes since v5:
* For a RIPAS_CHANGE request from the guest perform the actual RIPAS
change on next entry rather than immediately on the exit. This allows
the VMM to 'reject' a RIPAS change by refusing to continue
scheduling.
Changes since v4:
* Rename handle_rme_exit() to handle_rec_exit()
* Move the loop to copy registers into the REC enter structure from the
to rec_exit_handlers callbacks to kvm_rec_enter(). This fixes a bug
where the handler exits to user space and user space wants to modify
the GPRS.
* Some code rearrangement in rec_exit_ripas_change().
Changes since v2:
* realm_set_ipa_state() now provides an output parameter for the
top_iap that was changed. Use this to signal the VMM with the correct
range that has been transitioned.
* Adapt to previous patch changes.
---
arch/arm64/include/asm/kvm_rmi.h | 4 +
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/arm.c | 25 ++++-
arch/arm64/kvm/rmi-exit.c | 182 +++++++++++++++++++++++++++++++
arch/arm64/kvm/rmi.c | 42 +++++++
5 files changed, 249 insertions(+), 6 deletions(-)
create mode 100644 arch/arm64/kvm/rmi-exit.c
diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
index ccd09d55c69f..b1e4cf0f6803 100644
--- a/arch/arm64/include/asm/kvm_rmi.h
+++ b/arch/arm64/include/asm/kvm_rmi.h
@@ -100,6 +100,10 @@ void kvm_destroy_realm(struct kvm *kvm);
int kvm_realm_teardown_stage2(struct kvm *kvm);
void kvm_destroy_rec(struct kvm_vcpu *vcpu);
+int kvm_rec_enter(struct kvm_vcpu *vcpu);
+int kvm_rec_pre_enter(struct kvm_vcpu *vcpu);
+int handle_rec_exit(struct kvm_vcpu *vcpu, int rec_run_status);
+
static inline bool kvm_realm_is_private_address(struct realm *realm,
unsigned long addr)
{
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index ed3cf30eb06e..4a2d52fdb6a2 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -16,7 +16,7 @@ CFLAGS_handle_exit.o += -Wno-override-init
kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \
inject_fault.o va_layout.o handle_exit.o config.o \
guest.o debug.o reset.o sys_regs.o stacktrace.o \
- vgic-sys-reg-v3.o fpsimd.o pkvm.o rmi.o \
+ vgic-sys-reg-v3.o fpsimd.o pkvm.o rmi.o rmi-exit.o \
arch_timer.o trng.o vmid.o emulate-nested.o nested.o at.o \
vgic/vgic.o vgic/vgic-init.o \
vgic/vgic-irqfd.o vgic/vgic-v2.o \
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 2959a1451232..534d33b7c67a 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1341,6 +1341,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
if (ret > 0)
ret = check_vcpu_requests(vcpu);
+ if (ret > 0 && vcpu_is_rec(vcpu))
+ ret = kvm_rec_pre_enter(vcpu);
+
/*
* Preparing the interrupts to be injected also
* involves poking the GIC, which must be done in a
@@ -1388,7 +1391,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
trace_kvm_entry(*vcpu_pc(vcpu));
guest_timing_enter_irqoff();
- ret = kvm_arm_vcpu_enter_exit(vcpu);
+ if (vcpu_is_rec(vcpu))
+ ret = kvm_rec_enter(vcpu);
+ else
+ ret = kvm_arm_vcpu_enter_exit(vcpu);
vcpu->mode = OUTSIDE_GUEST_MODE;
vcpu->stat.exits++;
@@ -1434,7 +1440,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
* context synchronization event) is necessary to ensure that
* pending interrupts are taken.
*/
- if (ARM_EXCEPTION_CODE(ret) == ARM_EXCEPTION_IRQ) {
+ if ((!vcpu_is_rec(vcpu) && ARM_EXCEPTION_CODE(ret) == ARM_EXCEPTION_IRQ) ||
+ (vcpu_is_rec(vcpu) && vcpu->arch.rec.run->exit.exit_reason == RMI_EXIT_IRQ)) {
local_irq_enable();
isb();
local_irq_disable();
@@ -1446,8 +1453,13 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
trace_kvm_exit(ret, kvm_vcpu_trap_get_class(vcpu), *vcpu_pc(vcpu));
- /* Exit types that need handling before we can be preempted */
- handle_exit_early(vcpu, ret);
+ if (!vcpu_is_rec(vcpu)) {
+ /*
+ * Exit types that need handling before we can be
+ * preempted
+ */
+ handle_exit_early(vcpu, ret);
+ }
kvm_nested_sync_hwstate(vcpu);
@@ -1472,7 +1484,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
ret = ARM_EXCEPTION_IL;
}
- ret = handle_exit(vcpu, ret);
+ if (vcpu_is_rec(vcpu))
+ ret = handle_rec_exit(vcpu, ret);
+ else
+ ret = handle_exit(vcpu, ret);
}
if (unlikely(!irqchip_in_kernel(vcpu->kvm)))
diff --git a/arch/arm64/kvm/rmi-exit.c b/arch/arm64/kvm/rmi-exit.c
new file mode 100644
index 000000000000..973250563d7b
--- /dev/null
+++ b/arch/arm64/kvm/rmi-exit.c
@@ -0,0 +1,182 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2023-2026 ARM Ltd.
+ */
+
+#include <linux/kvm_host.h>
+#include <kvm/arm_hypercalls.h>
+#include <kvm/arm_psci.h>
+
+#include <linux/arm-smccc-rmi.h>
+#include <asm/kvm_emulate.h>
+#include <asm/kvm_rmi.h>
+#include <asm/kvm_mmu.h>
+
+typedef int (*exit_handler_fn)(struct kvm_vcpu *vcpu);
+
+static int rec_exit_reason_notimpl(struct kvm_vcpu *vcpu)
+{
+ vcpu_err(vcpu, "Unhandled exit reason from realm (ESR: %#llx)\n",
+ kvm_vcpu_get_esr(vcpu));
+ return -ENXIO;
+}
+
+static int rec_exit_sync_dabt(struct kvm_vcpu *vcpu)
+{
+ return kvm_handle_guest_abort(vcpu);
+}
+
+static int rec_exit_sync_iabt(struct kvm_vcpu *vcpu)
+{
+ vcpu_err(vcpu, "Unhandled instruction abort (ESR: %#llx).\n",
+ kvm_vcpu_get_esr(vcpu));
+ return -ENXIO;
+}
+
+static int rec_exit_sys_reg(struct kvm_vcpu *vcpu)
+{
+ struct realm_rec *rec = &vcpu->arch.rec;
+ unsigned long esr = kvm_vcpu_get_esr(vcpu);
+ int rt = kvm_vcpu_sys_get_rt(vcpu);
+ bool is_write = (esr & ESR_ELx_SYS64_ISS_DIR_MASK) == ESR_ELx_SYS64_ISS_DIR_WRITE;
+ int ret;
+
+ if (is_write)
+ vcpu_set_reg(vcpu, rt, rec->run->exit.gprs[rt]);
+
+ ret = kvm_handle_sys_reg(vcpu);
+ if (!is_write)
+ rec->run->enter.gprs[rt] = vcpu_get_reg(vcpu, rt);
+
+ return ret;
+}
+
+static exit_handler_fn rec_exit_handlers[] = {
+ [0 ... ESR_ELx_EC_MAX] = rec_exit_reason_notimpl,
+ [ESR_ELx_EC_SYS64] = rec_exit_sys_reg,
+ [ESR_ELx_EC_DABT_LOW] = rec_exit_sync_dabt,
+ [ESR_ELx_EC_IABT_LOW] = rec_exit_sync_iabt
+};
+
+static int rec_exit_psci(struct kvm_vcpu *vcpu)
+{
+ struct realm_rec *rec = &vcpu->arch.rec;
+ int i;
+
+ for (i = 0; i < REC_RUN_GPRS; i++)
+ vcpu_set_reg(vcpu, i, rec->run->exit.gprs[i]);
+
+ return kvm_smccc_call_handler(vcpu);
+}
+
+static int rec_exit_ripas_change(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ struct realm *realm = &kvm->arch.realm;
+ struct realm_rec *rec = &vcpu->arch.rec;
+ unsigned long base = rec->run->exit.ripas_base;
+ unsigned long top = rec->run->exit.ripas_top;
+ unsigned long ripas = rec->run->exit.ripas_value;
+
+ if (!kvm_realm_is_private_address(realm, base) ||
+ !kvm_realm_is_private_address(realm, top - 1)) {
+ vcpu_err(vcpu, "Invalid RIPAS_CHANGE for %#lx - %#lx, ripas: %#lx\n",
+ base, top, ripas);
+ /* Set RMI_REJECT bit */
+ rec->run->enter.flags = REC_ENTER_FLAG_RIPAS_RESPONSE;
+ return -EINVAL;
+ }
+
+ /* Exit to VMM, the actual RIPAS change is done on next entry */
+ kvm_prepare_memory_fault_exit(vcpu, base, top - base, false, false,
+ ripas == RMI_RAM);
+
+ /*
+ * KVM_EXIT_MEMORY_FAULT requires an return code of -EFAULT, see the
+ * API documentation
+ */
+ return -EFAULT;
+}
+
+static void update_arch_timer_irq_lines(struct kvm_vcpu *vcpu)
+{
+ struct realm_rec *rec = &vcpu->arch.rec;
+
+ __vcpu_assign_sys_reg(vcpu, CNTV_CTL_EL0, rec->run->exit.cntv_ctl);
+ __vcpu_assign_sys_reg(vcpu, CNTV_CVAL_EL0, rec->run->exit.cntv_cval);
+ __vcpu_assign_sys_reg(vcpu, CNTP_CTL_EL0, rec->run->exit.cntp_ctl);
+ __vcpu_assign_sys_reg(vcpu, CNTP_CVAL_EL0, rec->run->exit.cntp_cval);
+
+ kvm_realm_timers_update(vcpu);
+}
+
+/*
+ * Return > 0 to return to guest, < 0 on error, 0 (and set exit_reason) on
+ * proper exit to userspace.
+ */
+int handle_rec_exit(struct kvm_vcpu *vcpu, int rec_run_ret)
+{
+ struct realm_rec *rec = &vcpu->arch.rec;
+ u8 esr_ec = ESR_ELx_EC(rec->run->exit.esr);
+ unsigned long status, index;
+
+ status = RMI_RETURN_STATUS(rec_run_ret);
+ index = RMI_RETURN_INDEX(rec_run_ret);
+
+ /*
+ * If a PSCI_SYSTEM_OFF request raced with a vcpu executing, we might
+ * see the following status code and index indicating an attempt to run
+ * a REC when the RD state is SYSTEM_OFF. In this case, we just need to
+ * return to user space which can deal with the system event or will try
+ * to run the KVM VCPU again, at which point we will no longer attempt
+ * to enter the Realm because we will have a sleep request pending on
+ * the VCPU as a result of KVM's PSCI handling.
+ */
+ if (status == RMI_ERROR_REALM) {
+ vcpu->run->exit_reason = KVM_EXIT_SHUTDOWN;
+ return 0;
+ }
+
+ /*
+ * If a VCPU has been turned on, but the REC state hasn't been updated
+ * we may experience RMI_ERROR_REC. Exit to the userspace with -EAGAIN
+ * for a retry.
+ */
+ if (status == RMI_ERROR_REC)
+ return -EAGAIN;
+ if (rec_run_ret)
+ return -ENXIO;
+
+ vcpu->arch.fault.esr_el2 = rec->run->exit.esr;
+ vcpu->arch.fault.far_el2 = rec->run->exit.far;
+ /* HPFAR_EL2 is only valid for RMI_EXIT_SYNC */
+ vcpu->arch.fault.hpfar_el2 = 0;
+
+ update_arch_timer_irq_lines(vcpu);
+
+ /* Reset the emulation flags for the next run of the REC */
+ rec->run->enter.flags = 0;
+
+ switch (rec->run->exit.exit_reason) {
+ case RMI_EXIT_SYNC:
+ /*
+ * HPFAR_EL2_NS is hijacked to indicate a valid HPFAR value,
+ * see __get_fault_info()
+ */
+ vcpu->arch.fault.hpfar_el2 = rec->run->exit.hpfar | HPFAR_EL2_NS;
+ return rec_exit_handlers[esr_ec](vcpu);
+ case RMI_EXIT_IRQ:
+ case RMI_EXIT_FIQ:
+ case RMI_EXIT_SERROR:
+ return 1;
+ case RMI_EXIT_PSCI:
+ return rec_exit_psci(vcpu);
+ case RMI_EXIT_RIPAS_CHANGE:
+ return rec_exit_ripas_change(vcpu);
+ }
+
+ kvm_pr_unimpl("Unsupported exit reason: %u\n",
+ rec->run->exit.exit_reason);
+ vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
+ return 0;
+}
diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
index aa97ad5fd7e9..430b82c02d15 100644
--- a/arch/arm64/kvm/rmi.c
+++ b/arch/arm64/kvm/rmi.c
@@ -203,6 +203,48 @@ int kvm_realm_teardown_stage2(struct kvm *kvm)
return realm_destroy_rtts(kvm);
}
+/*
+ * kvm_rec_pre_enter - Complete operations before entering a REC
+ *
+ * Some operations require work to be completed before entering a realm. That
+ * work may require memory allocation so cannot be done in the kvm_rec_enter()
+ * call.
+ *
+ * Return: 1 if we should enter the guest
+ * 0 if we should exit to userspace
+ * < 0 if we should exit to userspace, where the return value indicates
+ * an error
+ */
+int kvm_rec_pre_enter(struct kvm_vcpu *vcpu)
+{
+ struct realm_rec *rec = &vcpu->arch.rec;
+
+ if (kvm_realm_state(vcpu->kvm) != REALM_STATE_ACTIVE)
+ return -EINVAL;
+
+ switch (rec->run->exit.exit_reason) {
+ case RMI_EXIT_HOST_CALL:
+ for (int i = 0; i < REC_RUN_GPRS; i++)
+ rec->run->enter.gprs[i] = vcpu_get_reg(vcpu, i);
+ break;
+ }
+
+ return 1;
+}
+
+int noinstr kvm_rec_enter(struct kvm_vcpu *vcpu)
+{
+ struct realm_rec *rec = &vcpu->arch.rec;
+ int ret;
+
+ guest_state_enter_irqoff();
+ ret = rmi_rec_enter(virt_to_phys(rec->rec_page),
+ virt_to_phys(rec->run));
+ guest_state_exit_irqoff();
+
+ return ret;
+}
+
static int __maybe_unused kvm_create_rec(struct kvm_vcpu *vcpu)
{
struct user_pt_regs *vcpu_regs = vcpu_gp_regs(vcpu);
--
2.43.0
^ permalink raw reply related
* [PATCH v15 13/37] KVM: arm64: CCA: Support timers in realm RECs
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
The RMM keeps track of the timer while the realm REC is running, but on
exit to the normal world KVM is responsible for handling the timers.
A later patch adds the support for propagating the timer values from the
exit data structure and calling kvm_realm_timers_update().
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v14:
* Special case in kvm_timer_vcpu_load()/kvm_timer_vcpu_put() the timer
handling.
Changes since v12:
* Adapt to upstream changes.
Changes since v11:
* Drop the kvm_is_realm() check from timer_set_offset(). We already
ensure that the offset is 0 when calling the function.
Changes since v10:
* KVM_CAP_COUNTER_OFFSET is now already hidden by a previous patch.
Changes since v9:
* No need to move the call to kvm_timer_unblocking() in
kvm_timer_vcpu_load().
Changes since v7:
* Hide KVM_CAP_COUNTER_OFFSET for realm guests.
---
arch/arm64/kvm/arch_timer.c | 38 +++++++++++++++++++++++++++++++++---
include/kvm/arm_arch_timer.h | 2 ++
2 files changed, 37 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 4155fe89b58a..fdd68f1f5b7b 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -482,6 +482,20 @@ static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,
timer_ctx);
}
+void kvm_realm_timers_update(struct kvm_vcpu *vcpu)
+{
+ struct arch_timer_cpu *arch_timer = &vcpu->arch.timer_cpu;
+ int i;
+
+ for (i = 0; i < NR_KVM_EL0_TIMERS; i++) {
+ struct arch_timer_context *timer = &arch_timer->timers[i];
+ bool status = timer_get_ctl(timer) & ARCH_TIMER_CTRL_IT_STAT;
+ bool level = kvm_timer_enabled(timer) && status;
+
+ kvm_timer_update_irq(vcpu, level, timer);
+ }
+}
+
/* Only called for a fully emulated timer */
static void timer_emulate(struct arch_timer_context *ctx)
{
@@ -888,6 +902,11 @@ void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
if (unlikely(!timer->enabled))
return;
+ if (vcpu_is_rec(vcpu)) {
+ kvm_timer_unblocking(vcpu);
+ return;
+ }
+
get_timer_map(vcpu, &map);
if (static_branch_likely(&has_gic_active_state)) {
@@ -923,6 +942,12 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)
if (unlikely(!timer->enabled))
return;
+ if (vcpu_is_rec(vcpu)) {
+ if (kvm_vcpu_is_blocking(vcpu))
+ kvm_timer_blocking(vcpu);
+ return;
+ }
+
get_timer_map(vcpu, &map);
timer_save_state(map.direct_vtimer);
@@ -1073,7 +1098,7 @@ static void timer_context_init(struct kvm_vcpu *vcpu, int timerid)
ctxt->timer_id = timerid;
- if (!kvm_vm_is_protected(vcpu->kvm)) {
+ if (!kvm_vm_is_protected(vcpu->kvm) && !vcpu_is_rec(vcpu)) {
if (timerid == TIMER_VTIMER)
ctxt->offset.vm_offset = &kvm->arch.timer_data.voffset;
else
@@ -1104,7 +1129,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
timer_context_init(vcpu, i);
/* Synchronize offsets across timers of a VM if not already provided */
- if (!vcpu_is_protected(vcpu) &&
+ if (!vcpu_is_protected(vcpu) && !vcpu_is_rec(vcpu) &&
!test_bit(KVM_ARCH_FLAG_VM_COUNTER_OFFSET, &vcpu->kvm->arch.flags)) {
timer_set_offset(vcpu_vtimer(vcpu), kvm_phys_timer_read());
timer_set_offset(vcpu_ptimer(vcpu), 0);
@@ -1600,6 +1625,13 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
return -EINVAL;
}
+ /*
+ * We don't use mapped IRQs for Realms because the RMI doesn't allow
+ * us setting the LR.HW bit in the VGIC.
+ */
+ if (vcpu_is_rec(vcpu))
+ return 0;
+
get_timer_map(vcpu, &map);
ops = vgic_is_v5(vcpu->kvm) ? &arch_timer_irq_ops_vgic_v5 :
@@ -1729,7 +1761,7 @@ int kvm_vm_ioctl_set_counter_offset(struct kvm *kvm,
if (offset->reserved)
return -EINVAL;
- if (kvm_vm_is_protected(kvm))
+ if (kvm_vm_is_protected(kvm) || kvm_is_realm(kvm))
return -EINVAL;
mutex_lock(&kvm->lock);
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 15a4f97f8105..31e48d5dbc31 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -108,6 +108,8 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr);
int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr);
int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr);
+void kvm_realm_timers_update(struct kvm_vcpu *vcpu);
+
u64 kvm_phys_timer_read(void);
void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu);
--
2.43.0
^ permalink raw reply related
* [PATCH v15 12/37] KVM: arm64: CCA: Support the VGIC in realms
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
The RMM provides emulation of a VGIC to the realm guest. With RMM v2.0
the registers are passed in the system registers so this works similar
to a normal guest, but kvm_arch_vcpu_put() need reordering to early out,
and realm guests don't support GICv2 even if the host does.
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes from v12:
* GIC registers are now passed in the system registers rather than via
rec_entry/rec_exit which removes most of the changes.
Changes from v11:
* Minor changes to align with the previous patches. Note that the VGIC
handling will change with RMM v2.0.
Changes from v10:
* Make sure we sync the VGIC v4 state, and only populate valid lrs from
the list.
Changes from v9:
* Copy gicv3_vmcr from the RMM at the same time as gicv3_hcr rather
than having to handle that as a special case.
Changes from v8:
* Propagate gicv3_hcr to from the RMM.
Changes from v5:
* Handle RMM providing fewer GIC LRs than the hardware supports.
---
arch/arm64/kvm/arm.c | 11 ++++++++---
arch/arm64/kvm/vgic/vgic-init.c | 2 +-
2 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 38fac98cd6a4..2959a1451232 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -791,19 +791,24 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
kvm_call_hyp_nvhe(__pkvm_vcpu_put);
}
+ kvm_timer_vcpu_put(vcpu);
+ kvm_vgic_put(vcpu);
kvm_vcpu_put_debug(vcpu);
+
+ vcpu->cpu = -1;
+
+ if (vcpu_is_rec(vcpu))
+ return;
+
kvm_arch_vcpu_put_fp(vcpu);
if (has_vhe())
kvm_vcpu_put_vhe(vcpu);
- kvm_timer_vcpu_put(vcpu);
- kvm_vgic_put(vcpu);
kvm_vcpu_pmu_restore_host(vcpu);
if (vcpu_has_nv(vcpu))
kvm_vcpu_put_hw_mmu(vcpu);
kvm_arm_vmid_clear_active();
vcpu_clear_on_unsupported_cpu(vcpu);
- vcpu->cpu = -1;
}
static void __kvm_arm_vcpu_power_off(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-init.c
index 907057881b26..fd77db35ef02 100644
--- a/arch/arm64/kvm/vgic/vgic-init.c
+++ b/arch/arm64/kvm/vgic/vgic-init.c
@@ -81,7 +81,7 @@ int kvm_vgic_create(struct kvm *kvm, u32 type)
* the proper checks already.
*/
if (type == KVM_DEV_TYPE_ARM_VGIC_V2 &&
- !kvm_vgic_global_state.can_emulate_gicv2)
+ (!kvm_vgic_global_state.can_emulate_gicv2 || kvm_is_realm(kvm)))
return -ENODEV;
/*
--
2.43.0
^ permalink raw reply related
* [PATCH v15 11/37] KVM: arm64: CCA: Allocate and free RECs to match vCPUs
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
The RMM maintains a data structure known as the Realm Execution Context
(or REC). It is similar to struct kvm_vcpu and tracks the state of the
virtual CPUs. KVM must delegate memory and request the structures are
created when vCPUs are created, and suitably tear down on destruction.
RECs may require additional pages (e.g. for storing larger register
state for SVE). The RMM can request extra pages for this purpose using
the Stateful RMI Operations (SRO) functionality to request pages during
REC creation. These pages are then passed back to the host from the RMM
('reclaimed') when the REC is destroyed. The kernel tracking object
(struct rmi_sro_state) is stored in the realm_rec structure to avoid
memory allocation during the destruction path.
Note that only some of register state for the REC can be set by KVM, the
rest is defined by the RMM (zeroed). The register state then cannot be
changed by KVM after the REC is created (except when the guest
explicitly requests this e.g. by performing a PSCI call).
Note that the function kvm_create_rec() is unused at this point, a
future patch will add the call and remove the __maybe_unused attribute.
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v14:
* Handle partial REC creation better by NULLing rec->run, rec->rec_page
and rec->sro when freeing.
Changes since v13:
* Support SRO for REC creation/destruction instead of auxiliary
granules.
Changes since v12:
* Use the new range-based delegation RMI.
Changes since v11:
* Remove the KVM_ARM_VCPU_REC feature. User space no longer needs to
configure each VCPU separately, RECs are created on the first VCPU
run of the guest.
Changes since v9:
* Size the aux_pages array according to the PAGE_SIZE of the host.
Changes since v7:
* Add comment explaining the aux_pages array.
* Rename "undeleted_failed" variable to "should_free" to avoid a
confusing double negative.
Changes since v6:
* Avoid reporting the KVM_ARM_VCPU_REC feature if the guest isn't a
realm guest.
* Support host page size being larger than RMM's granule size when
allocating/freeing aux granules.
Changes since v5:
* Separate the concept of vcpu_is_rec() and
kvm_arm_vcpu_rec_finalized() by using the KVM_ARM_VCPU_REC feature as
the indication that the VCPU is a REC.
Changes since v2:
* Free rec->run earlier in kvm_destroy_realm() and adapt to previous patches.
---
arch/arm64/include/asm/kvm_emulate.h | 5 ++
arch/arm64/include/asm/kvm_host.h | 3 +
arch/arm64/include/asm/kvm_rmi.h | 16 +++++
arch/arm64/kvm/arm.c | 6 ++
arch/arm64/kvm/reset.c | 1 +
arch/arm64/kvm/rmi.c | 100 +++++++++++++++++++++++++++
6 files changed, 131 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index e26d6755279f..2e69fe494716 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -712,4 +712,9 @@ static inline bool kvm_realm_is_created(struct kvm *kvm)
return kvm_is_realm(kvm) && kvm_realm_state(kvm) != REALM_STATE_NONE;
}
+static inline bool vcpu_is_rec(const struct kvm_vcpu *vcpu)
+{
+ return kvm_is_realm(vcpu->kvm);
+}
+
#endif /* __ARM64_KVM_EMULATE_H__ */
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 1a5e15040111..9b46b39ed11e 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -949,6 +949,9 @@ struct kvm_vcpu_arch {
/* Hyp-readable copy of kvm_vcpu::pid */
pid_t pid;
+
+ /* Realm meta data */
+ struct realm_rec rec;
};
/*
diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
index 31ec856561c6..ccd09d55c69f 100644
--- a/arch/arm64/include/asm/kvm_rmi.h
+++ b/arch/arm64/include/asm/kvm_rmi.h
@@ -77,12 +77,28 @@ struct realm {
bool rtts_destroyed;
};
+/**
+ * struct realm_rec - Additional per VCPU data for a Realm
+ *
+ * @mpidr: MPIDR (Multiprocessor Affinity Register) value to identify this VCPU
+ * @rec_page: Kernel VA of the RMM's private page for this REC
+ * @run: Kernel VA of the RmiRecRun structure shared with the RMM
+ * @sro: A preallocated SRO state context
+ */
+struct realm_rec {
+ unsigned long mpidr;
+ void *rec_page;
+ struct rec_run *run;
+ struct rmi_sro_state *sro;
+};
+
void kvm_init_rmi(void);
u32 kvm_rmm_ipa_limit(void);
int kvm_init_realm(struct kvm *kvm);
void kvm_destroy_realm(struct kvm *kvm);
int kvm_realm_teardown_stage2(struct kvm *kvm);
+void kvm_destroy_rec(struct kvm_vcpu *vcpu);
static inline bool kvm_realm_is_private_address(struct realm *realm,
unsigned long addr)
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 9881fd6c511c..38fac98cd6a4 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -589,6 +589,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
/* Force users to call KVM_ARM_VCPU_INIT */
vcpu_clear_flag(vcpu, VCPU_INITIALIZED);
+ vcpu->arch.rec.mpidr = INVALID_HWID;
+
vcpu->arch.mmu_page_cache.gfp_zero = __GFP_ZERO;
/* Set up the timer */
@@ -1668,6 +1670,10 @@ static int kvm_vcpu_init_check_features(struct kvm_vcpu *vcpu,
if (test_bit(KVM_ARM_VCPU_HAS_EL2, &features))
return -EINVAL;
+ /* Realms are incompatible with AArch32 */
+ if (vcpu_is_rec(vcpu))
+ return -EINVAL;
+
return 0;
}
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index b963fd975aac..c18cdca7d125 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -161,6 +161,7 @@ void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu)
free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
kfree(vcpu->arch.vncr_tlb);
kfree(vcpu->arch.ccsidr);
+ kvm_destroy_rec(vcpu);
}
static void kvm_vcpu_reset_sve(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
index db32448a0fc1..aa97ad5fd7e9 100644
--- a/arch/arm64/kvm/rmi.c
+++ b/arch/arm64/kvm/rmi.c
@@ -203,6 +203,106 @@ int kvm_realm_teardown_stage2(struct kvm *kvm)
return realm_destroy_rtts(kvm);
}
+static int __maybe_unused kvm_create_rec(struct kvm_vcpu *vcpu)
+{
+ struct user_pt_regs *vcpu_regs = vcpu_gp_regs(vcpu);
+ unsigned long mpidr = kvm_vcpu_get_mpidr_aff(vcpu);
+ struct realm *realm = &vcpu->kvm->arch.realm;
+ struct realm_rec *rec = &vcpu->arch.rec;
+ unsigned long rec_page_phys;
+ struct rec_params *params;
+ int r, i;
+
+ if (rec->run)
+ return -EBUSY;
+
+ /*
+ * The RMM will report PSCI v1.0 to Realms and the KVM_ARM_VCPU_PSCI_0_2
+ * flag covers v0.2 and onwards.
+ */
+ if (!vcpu_has_feature(vcpu, KVM_ARM_VCPU_PSCI_0_2))
+ return -EINVAL;
+
+ BUILD_BUG_ON(sizeof(*params) > PAGE_SIZE);
+ BUILD_BUG_ON(sizeof(*rec->run) > PAGE_SIZE);
+
+ params = (struct rec_params *)get_zeroed_page(GFP_KERNEL);
+ rec->rec_page = (void *)__get_free_page(GFP_KERNEL);
+ rec->run = (struct rec_run *)get_zeroed_page(GFP_KERNEL);
+ rec->sro = kmalloc_obj(*rec->sro);
+ if (!params || !rec->rec_page || !rec->run || !rec->sro) {
+ r = -ENOMEM;
+ goto out_free_pages;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(params->gprs); i++)
+ params->gprs[i] = vcpu_regs->regs[i];
+
+ params->pc = vcpu_regs->pc;
+
+ if (vcpu->vcpu_id == 0)
+ params->flags |= REC_PARAMS_FLAG_RUNNABLE;
+
+ rec_page_phys = virt_to_phys(rec->rec_page);
+
+ if (rmi_delegate_page(rec_page_phys)) {
+ r = -ENXIO;
+ goto out_free_pages;
+ }
+
+ params->mpidr = mpidr;
+
+ if (rmi_rec_create(virt_to_phys(realm->rd), rec_page_phys,
+ virt_to_phys(params), rec->sro)) {
+ r = -ENXIO;
+ goto out_undelegate_rmm_rec;
+ }
+
+ rec->mpidr = mpidr;
+
+ free_page((unsigned long)params);
+ return 0;
+
+out_undelegate_rmm_rec:
+ if (WARN_ON(rmi_undelegate_page(rec_page_phys)))
+ rec->rec_page = NULL;
+out_free_pages:
+ free_page((unsigned long)rec->run);
+ free_page((unsigned long)rec->rec_page);
+ free_page((unsigned long)params);
+ kfree(rec->sro);
+ rec->run = NULL;
+ rec->rec_page = NULL;
+ rec->sro = NULL;
+ return r;
+}
+
+void kvm_destroy_rec(struct kvm_vcpu *vcpu)
+{
+ struct realm_rec *rec = &vcpu->arch.rec;
+ unsigned long rec_page_phys;
+
+ if (!vcpu_is_rec(vcpu))
+ return;
+
+ if (!rec->run) {
+ /* Nothing to do if the VCPU hasn't been finalized */
+ return;
+ }
+
+ rec_page_phys = virt_to_phys(rec->rec_page);
+
+ if (WARN_ON(rmi_rec_destroy(rec_page_phys, rec->sro)))
+ return;
+
+ free_page((unsigned long)rec->run);
+ kfree(rec->sro);
+ free_delegated_page(rec_page_phys);
+ rec->run = NULL;
+ rec->sro = NULL;
+ rec->rec_page = NULL;
+}
+
void kvm_destroy_realm(struct kvm *kvm)
{
struct realm *realm = &kvm->arch.realm;
--
2.43.0
^ permalink raw reply related
* [PATCH v15 10/37] KVM: arm64: CCA: Tear down RTTs
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
The RMM owns the stage 2 page tables for a realm, and KVM must request
that the RMM creates/destroys entries as necessary. The physical pages
to store the page tables are delegated to the realm as required, and can
be undelegated when no longer used.
Creating new RTTs is the easy part, tearing down is a little more
tricky. The result of realm_rtt_destroy() can be used to effectively
walk the tree and destroy the entries (undelegating pages that were
given to the realm).
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v14:
* Teardown is now serialized under config_lock and idempotent.
Changes since v13:
* Avoid the double call of kvm_free_stage2_pgd() by splitting the work
across that and a new function kvm_realm_uninit_stage2() which is
only called for realm guests.
Changes since v12:
* Simplify some functions now we know RMM page size is the same as the
host's.
Changes since v11:
* Moved some code from earlier in the series to this one so that it's
added when it's first used.
Changes since v10:
* RME->RMI rename.
* Some code to handle freeing stage 2 PGD moved into this patch where
it belongs.
Changes since v9:
* Add a comment clarifying that root level RTTs are not destroyed until
after the RD is destroyed.
Changes since v8:
* Introduce free_rtt() wrapper which calls free_delegated_granule()
followed by kvm_account_pgtable_pages(). This makes it clear where an
RTT is being freed rather than just a delegated granule.
Changes since v6:
* Move rme_rtt_level_mapsize() and supporting defines from kvm_rme.h
into rme.c as they are only used in that file.
Changes since v5:
* Rename some RME_xxx defines to do with page sizes as RMM_xxx - they are
a property of the RMM specification not the RME architecture.
Changes since v2:
* Moved {alloc,free}_delegated_page() and ensure_spare_page() to a
later patch when they are actually used.
* Some simplifications now rmi_xxx() functions allow NULL as an output
parameter.
* Improved comments and code layout.
---
arch/arm64/include/asm/kvm_rmi.h | 11 ++
arch/arm64/kvm/mmu.c | 17 ++-
arch/arm64/kvm/rmi.c | 196 +++++++++++++++++++++++++++++++
3 files changed, 223 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
index 5c40616aa1a1..31ec856561c6 100644
--- a/arch/arm64/include/asm/kvm_rmi.h
+++ b/arch/arm64/include/asm/kvm_rmi.h
@@ -57,6 +57,8 @@ enum realm_state {
* @sro: Preallocated SRO state context for Realm MMU operations
* @state: The lifetime state machine for the realm
* @ia_bits: Number of valid Input Address bits in the IPA
+ * @stage2_unmapped: The Realm stage-2 mappings have been removed
+ * @rtts_destroyed: The non-root RTTs have been torn down
*/
struct realm {
void *rd;
@@ -71,6 +73,8 @@ struct realm {
struct rmi_sro_state *sro;
enum realm_state state;
unsigned int ia_bits;
+ bool stage2_unmapped;
+ bool rtts_destroyed;
};
void kvm_init_rmi(void);
@@ -78,5 +82,12 @@ u32 kvm_rmm_ipa_limit(void);
int kvm_init_realm(struct kvm *kvm);
void kvm_destroy_realm(struct kvm *kvm);
+int kvm_realm_teardown_stage2(struct kvm *kvm);
+
+static inline bool kvm_realm_is_private_address(struct realm *realm,
+ unsigned long addr)
+{
+ return !(addr & BIT(realm->ia_bits - 1));
+}
#endif /* __ASM_KVM_RMI_H */
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 8b9efa8a3539..cd06881c1497 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1060,9 +1060,24 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
return err;
}
+static void kvm_realm_uninit_stage2(struct kvm_s2_mmu *mmu)
+{
+ struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
+
+ mutex_lock(&kvm->arch.config_lock);
+ if (kvm_realm_state(kvm) == REALM_STATE_ACTIVE) {
+ kvm_set_realm_state(kvm, REALM_STATE_DYING);
+ WARN_ON(kvm_realm_teardown_stage2(kvm));
+ }
+ mutex_unlock(&kvm->arch.config_lock);
+}
+
void kvm_uninit_stage2_mmu(struct kvm *kvm)
{
- kvm_free_stage2_pgd(&kvm->arch.mmu);
+ if (kvm_is_realm(kvm))
+ kvm_realm_uninit_stage2(&kvm->arch.mmu);
+ else
+ kvm_free_stage2_pgd(&kvm->arch.mmu);
kvm_mmu_free_memory_cache(&kvm->arch.mmu.split_page_cache);
}
diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
index b81c404ead9f..db32448a0fc1 100644
--- a/arch/arm64/kvm/rmi.c
+++ b/arch/arm64/kvm/rmi.c
@@ -11,6 +11,14 @@
#include <asm/rmi_cmds.h>
#include <asm/virt.h>
+static inline unsigned long rmi_rtt_level_mapsize(int level)
+{
+ if (WARN_ON(level > KVM_PGTABLE_LAST_LEVEL))
+ return PAGE_SIZE;
+
+ return (1UL << ARM64_HW_PGTABLE_LEVEL_SHIFT(level));
+}
+
static bool rmi_has_feature(int reg, unsigned long feature)
{
return !!u64_get_bits(rmi_feat_reg(reg), feature);
@@ -21,11 +29,187 @@ u32 kvm_rmm_ipa_limit(void)
return u64_get_bits(rmi_feat_reg(0), RMI_FEATURE_REGISTER_0_S2SZ);
}
+static int get_start_level(struct realm *realm)
+{
+ return 4 - stage2_pgtable_levels(realm->ia_bits);
+}
+
+static void free_rtt(phys_addr_t phys)
+{
+ if (free_delegated_page(phys))
+ return;
+
+ kvm_account_pgtable_pages(phys_to_virt(phys), -1);
+}
+
+/*
+ * realm_rtt_destroy - Destroy an RTT at @level for @addr.
+ *
+ * Returns - Result of the RMI_RTT_DESTROY call, and:
+ * @rtt_granule: RTT granule, if the RTT was destroyed.
+ * @next_addr: IPA corresponding to the next possible valid entry we
+ * can target
+ */
+static int realm_rtt_destroy(struct realm *realm, unsigned long addr,
+ int level, phys_addr_t *rtt_granule,
+ unsigned long *next_addr)
+{
+ unsigned long out_rtt;
+ int ret;
+
+ ret = rmi_rtt_destroy(virt_to_phys(realm->rd), addr, level,
+ &out_rtt, next_addr);
+
+ *rtt_granule = out_rtt;
+
+ return ret;
+}
+
+static int realm_tear_down_rtt_level(struct realm *realm, int level,
+ unsigned long start, unsigned long end)
+{
+ ssize_t map_size;
+ unsigned long addr, next_addr;
+
+ if (WARN_ON(level > KVM_PGTABLE_LAST_LEVEL))
+ return -EINVAL;
+
+ map_size = rmi_rtt_level_mapsize(level - 1);
+
+ for (addr = start; addr < end; addr = next_addr) {
+ phys_addr_t rtt_granule;
+ int ret;
+ unsigned long align_addr = ALIGN(addr, map_size);
+
+ next_addr = ALIGN(addr + 1, map_size);
+
+ if (next_addr > end || align_addr != addr) {
+ /*
+ * The target range is smaller than what this level
+ * covers, recurse deeper.
+ */
+ ret = realm_tear_down_rtt_level(realm,
+ level + 1,
+ addr,
+ min(next_addr, end));
+ if (ret)
+ return ret;
+ continue;
+ }
+
+ ret = realm_rtt_destroy(realm, addr, level,
+ &rtt_granule, &next_addr);
+
+ switch (RMI_RETURN_STATUS(ret)) {
+ case RMI_SUCCESS:
+ free_rtt(rtt_granule);
+ break;
+ case RMI_ERROR_RTT:
+ if (next_addr > addr) {
+ /* Missing RTT, skip */
+ break;
+ }
+ /*
+ * We tear down the RTT range for the full IPA
+ * space, after everything is unmapped. Also we
+ * descend down only if we cannot tear down a
+ * top level RTT. Thus RMM must be able to walk
+ * to the requested level. e.g., a block mapping
+ * exists at L1 or L2.
+ */
+ if (WARN_ON(RMI_RETURN_INDEX(ret) != level))
+ return -EBUSY;
+ if (WARN_ON(level == KVM_PGTABLE_LAST_LEVEL))
+ return -EBUSY;
+
+ /*
+ * The table has active entries in it, recurse deeper
+ * and tear down the RTTs.
+ */
+ next_addr = ALIGN(addr + 1, map_size);
+ ret = realm_tear_down_rtt_level(realm,
+ level + 1,
+ addr,
+ next_addr);
+ if (ret)
+ return ret;
+ /*
+ * Now that the child RTTs are destroyed,
+ * retry at this level.
+ */
+ next_addr = addr;
+ break;
+ default:
+ WARN_ON(1);
+ return -ENXIO;
+ }
+ }
+
+ return 0;
+}
+
+static int realm_tear_down_rtt_range(struct realm *realm,
+ unsigned long start, unsigned long end)
+{
+ /*
+ * Root level RTTs can only be destroyed after the RD is destroyed. So
+ * tear down everything below the root level
+ */
+ return realm_tear_down_rtt_level(realm, get_start_level(realm) + 1,
+ start, end);
+}
+
+static int realm_destroy_rtts(struct kvm *kvm)
+{
+ struct realm *realm = &kvm->arch.realm;
+ unsigned int ia_bits = realm->ia_bits;
+ int ret;
+
+ lockdep_assert_held(&kvm->arch.config_lock);
+
+ if (realm->rtts_destroyed)
+ return 0;
+
+ ret = realm_tear_down_rtt_range(realm, 0, (1UL << ia_bits));
+ if (ret)
+ return ret;
+
+ realm->rtts_destroyed = true;
+ return 0;
+}
+
+static void realm_unmap_stage2(struct kvm *kvm)
+{
+ struct realm *realm = &kvm->arch.realm;
+
+ lockdep_assert_held(&kvm->arch.config_lock);
+
+ if (realm->stage2_unmapped)
+ return;
+
+ write_lock(&kvm->mmu_lock);
+ kvm_stage2_unmap_range(&kvm->arch.mmu, 0,
+ BIT(realm->ia_bits - 1), true);
+ write_unlock(&kvm->mmu_lock);
+
+ realm->stage2_unmapped = true;
+}
+
+int kvm_realm_teardown_stage2(struct kvm *kvm)
+{
+ lockdep_assert_held(&kvm->arch.config_lock);
+
+ realm_unmap_stage2(kvm);
+ return realm_destroy_rtts(kvm);
+}
+
void kvm_destroy_realm(struct kvm *kvm)
{
struct realm *realm = &kvm->arch.realm;
size_t pgd_size = kvm_pgtable_stage2_pgd_size(kvm->arch.mmu.vtcr);
+ guard(mutex)(&kvm->arch.config_lock);
+
if (realm->params) {
free_page((unsigned long)realm->params);
realm->params = NULL;
@@ -39,12 +223,24 @@ void kvm_destroy_realm(struct kvm *kvm)
kvm_set_realm_state(kvm, REALM_STATE_DYING);
+ /*
+ * REALM_DESTROY requires the realm to be non-live: all RECs must have
+ * been destroyed and the root RTTs must be empty. Unmap the IPA space
+ * and destroy any non-root RTTs before tearing down the RD. The root
+ * RTT pages are still owned by the RMM at this point, so keep the KVM
+ * pgtable alive until after REALM_DESTROY and undelegation.
+ */
+ realm_unmap_stage2(kvm);
+
if (realm->rd) {
phys_addr_t rd_phys = virt_to_phys(realm->rd);
if (WARN_ON(rmi_realm_terminate(rd_phys, realm->sro)))
return;
+ if (WARN_ON(realm_destroy_rtts(kvm)))
+ return;
+
if (WARN_ON(rmi_realm_destroy(rd_phys, realm->sro)))
return;
free_delegated_page(rd_phys);
--
2.43.0
^ permalink raw reply related
* [PATCH v15 09/37] KVM: arm64: CCA: Allow passing the machine type in KVM creation
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
Previously machine type was used purely for specifying the physical
address size of the guest. Reserve the higher bits to specify an ARM
specific machine type and declare a new type 'KVM_VM_TYPE_ARM_REALM'
used to create a realm guest.
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v13:
* Rework to use the two top bits for the machine type now that pKVM has
merged and used the top bit for KVM_VM_TYPE_ARM_PROTECTED.
* Update the documentation to include KVM_VM_TYPE_ARM_PROTECTED as
well.
Changes since v9:
* Explictly set realm.state to REALM_STATE_NONE rather than rely on the
zeroing of the structure.
Changes since v7:
* Add some documentation explaining the new machine type.
Changes since v6:
* Make the check for kvm_rme_is_available more visible and report an
error code of -EPERM (instead of -EINVAL) to make it explicit that
the kernel supports RME, but the platform doesn't.
---
Documentation/virt/kvm/api.rst | 18 ++++++++++++++++--
arch/arm64/kvm/arm.c | 11 +++++++++++
include/uapi/linux/kvm.h | 7 ++++++-
3 files changed, 33 insertions(+), 3 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index b38e090ad95d..e39d146b34a3 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -181,8 +181,22 @@ flag KVM_VM_MIPS_VZ.
ARM64:
^^^^^^
-On arm64, the physical address size for a VM (IPA Size limit) is limited
-to 40bits by default. The limit can be configured if the host supports the
+On arm64, the machine type identifier is used to encode a type and the
+physical address size for the VM. The lower byte (bits[7-0]) encode the
+address size and the upper bits[30-31] encode a machine type. The machine
+types that might be available are:
+
+ ========================= ============================================
+ KVM_VM_TYPE_ARM_NORMAL A standard VM
+ KVM_VM_TYPE_ARM_REALM A "Realm" VM using the Arm Confidential
+ Compute extensions, the VM's memory is
+ protected from the host.
+ KVM_VM_TYPE_ARM_PROTECTED A "protected" VM using pKVM to isolate the
+ VM from the host.
+ ========================= ============================================
+
+The physical address size for a VM (IPA Size limit) is limited to 40bits
+by default. The limit can be configured if the host supports the
extension KVM_CAP_ARM_VM_IPA_SIZE. When supported, use
KVM_VM_TYPE_ARM_IPA_SIZE(IPA_Bits) to set the size in the machine type
identifier, where IPA_Bits is the maximum width of any physical
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 6f39831dcf5d..9881fd6c511c 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -249,6 +249,17 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
mutex_unlock(&kvm->lock);
#endif
+ if ((type & KVM_VM_TYPE_ARM_PROTECTED) &&
+ (type & KVM_VM_TYPE_ARM_REALM))
+ return -EINVAL;
+
+ if (type & KVM_VM_TYPE_ARM_REALM) {
+ if (!static_branch_unlikely(&kvm_rmi_is_available))
+ return -EINVAL;
+ kvm_set_realm_state(kvm, REALM_STATE_NONE);
+ kvm->arch.is_realm = true;
+ }
+
kvm_init_nested(kvm);
ret = kvm_share_hyp(kvm, kvm + 1);
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 0231ff174a50..adee3936d6ae 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -700,14 +700,19 @@ struct kvm_enable_cap {
* address size for the VM. Bits[7-0] are reserved for the guest
* PA size shift (i.e, log2(PA_Size)). For backward compatibility,
* value 0 implies the default IPA size, 40bits.
+ *
+ * Bits[30-31] are reserved for the VM type
*/
#define KVM_VM_TYPE_ARM_IPA_SIZE_MASK 0xffULL
#define KVM_VM_TYPE_ARM_IPA_SIZE(x) \
((x) & KVM_VM_TYPE_ARM_IPA_SIZE_MASK)
+#define KVM_VM_TYPE_ARM_NORMAL 0
+#define KVM_VM_TYPE_ARM_REALM (1UL << 30)
#define KVM_VM_TYPE_ARM_PROTECTED (1UL << 31)
#define KVM_VM_TYPE_ARM_MASK (KVM_VM_TYPE_ARM_IPA_SIZE_MASK | \
- KVM_VM_TYPE_ARM_PROTECTED)
+ KVM_VM_TYPE_ARM_PROTECTED | \
+ KVM_VM_TYPE_ARM_REALM)
/*
* ioctls for /dev/kvm fds:
--
2.43.0
^ permalink raw reply related
* [PATCH v15 08/37] KVM: arm64: CCA: Don't expose unsupported capabilities for realm guests
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Suzuki K Poulose, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi, Steven Price
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
From: Suzuki K Poulose <suzuki.poulose@arm.com>
RMM v2.0 provides no mechanism for the host to perform debug operations
on the guest. So limit the extensions that are visible to an allowlist
so that only those capabilities we can support are advertised.
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v13:
* Add missing check in kvm_vm_ioctl_enable_cap().
Changes since v10:
* Add a kvm_realm_ext_allowed() function which limits which extensions
are exposed to an allowlist. This removes the need for special casing
various extensions.
Changes since v7:
* Remove the helper functions and inline the kvm_is_realm() check with
a ternary operator.
* Rewrite the commit message to explain this patch.
---
arch/arm64/kvm/arm.c | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index c4d906861736..6f39831dcf5d 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -135,6 +135,26 @@ int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE;
}
+static bool kvm_realm_ext_allowed(long ext)
+{
+ switch (ext) {
+ case KVM_CAP_IRQCHIP:
+ case KVM_CAP_ARM_PSCI:
+ case KVM_CAP_ARM_PSCI_0_2:
+ case KVM_CAP_NR_VCPUS:
+ case KVM_CAP_MAX_VCPUS:
+ case KVM_CAP_MAX_VCPU_ID:
+ case KVM_CAP_MSI_DEVID:
+ case KVM_CAP_ARM_VM_IPA_SIZE:
+ case KVM_CAP_ARM_PTRAUTH_ADDRESS:
+ case KVM_CAP_ARM_PTRAUTH_GENERIC:
+ case KVM_CAP_ARM_RMI:
+ case KVM_CAP_SYNC_MMU:
+ return true;
+ }
+ return false;
+}
+
int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
struct kvm_enable_cap *cap)
{
@@ -146,6 +166,9 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
if (is_protected_kvm_enabled() && !kvm_pkvm_ext_allowed(kvm, cap->cap))
return -EINVAL;
+ if (kvm && kvm_is_realm(kvm) && !kvm_realm_ext_allowed(cap->cap))
+ return -EINVAL;
+
switch (cap->cap) {
case KVM_CAP_ARM_NISV_TO_USER:
r = 0;
@@ -380,6 +403,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
if (is_protected_kvm_enabled() && !kvm_pkvm_ext_allowed(kvm, ext))
return 0;
+ if (kvm && kvm_is_realm(kvm) && !kvm_realm_ext_allowed(ext))
+ return 0;
+
switch (ext) {
case KVM_CAP_IRQCHIP:
r = vgic_present;
--
2.43.0
^ permalink raw reply related
* [PATCH v15 07/37] KVM: arm64: CCA: Add basic infrastructure for creating a realm
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
Introduce the skeleton functions for creating and destroying a realm.
The IPA size requested is checked against what the RMM supports.
The actual work of constructing the realm will be added in future
patches.
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v14:
* Adapt to the RMM v2.0-bet2 realm parameter layout and SRO-based realm
creation.
Changes since v13:
* Rebased and updated to RMM-v2.0-bet1.
* Auxiliary granules have been removed in RMM-v2.0-bet1
Changes since v12:
* Drop the RMM_PAGE_{SHIFT,SIZE} defines - the RMM is now configured to
be the same as the host's page size.
* Rework delegate/undelegate functions to use the new RMI range based
operations.
Changes since v11:
* Major rework to drop the realm configuration and make the
construction of realms implicit rather than driven by the VMM
directly.
* The code to create RDs, handle VMIDs etc is moved to later patches.
Changes since v10:
* Rename from RME to RMI.
* Move the stage2 cleanup to a later patch.
Changes since v9:
* Avoid walking the stage 2 page tables when destroying the realm -
the real ones are not accessible to the non-secure world, and the RMM
may leave junk in the physical pages when returning them.
* Fix an error path in realm_create_rd() to actually return an error value.
Changes since v8:
* Fix free_delegated_granule() to not call kvm_account_pgtable_pages();
a separate wrapper will be introduced in a later patch to deal with
RTTs.
* Minor code cleanups following review.
Changes since v7:
* Minor code cleanup following Gavin's review.
Changes since v6:
* Separate RMM RTT calculations from host PAGE_SIZE. This allows the
host page size to be larger than 4k while still communicating with an
RMM which uses 4k granules.
Changes since v5:
* Introduce free_delegated_granule() to replace many
undelegate/free_page() instances and centralise the comment on
leaking when the undelegate fails.
* Several other minor improvements suggested by reviews - thanks for
the feedback!
Changes since v2:
* Improved commit description.
* Improved return failures for rmi_check_version().
* Clear contents of PGD after it has been undelegated in case the RMM
left stale data.
* Minor changes to reflect changes in previous patches.
---
arch/arm64/include/asm/kvm_emulate.h | 24 ++++++++++
arch/arm64/include/asm/kvm_rmi.h | 65 +++++++++++++++++++++++++++
arch/arm64/kvm/arm.c | 12 +++++
arch/arm64/kvm/mmu.c | 18 +++++++-
arch/arm64/kvm/rmi.c | 67 ++++++++++++++++++++++++++++
5 files changed, 184 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 5bf3d7e1d92c..e26d6755279f 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -688,4 +688,28 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
vcpu->arch.hcrx_el2 |= HCRX_EL2_EnASR;
}
}
+
+static inline bool kvm_is_realm(struct kvm *kvm)
+{
+ if (static_branch_unlikely(&kvm_rmi_is_available))
+ return kvm->arch.is_realm;
+ return false;
+}
+
+static inline enum realm_state kvm_realm_state(struct kvm *kvm)
+{
+ return READ_ONCE(kvm->arch.realm.state);
+}
+
+static inline void kvm_set_realm_state(struct kvm *kvm,
+ enum realm_state new_state)
+{
+ WRITE_ONCE(kvm->arch.realm.state, new_state);
+}
+
+static inline bool kvm_realm_is_created(struct kvm *kvm)
+{
+ return kvm_is_realm(kvm) && kvm_realm_state(kvm) != REALM_STATE_NONE;
+}
+
#endif /* __ARM64_KVM_EMULATE_H__ */
diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
index 57d24b244c95..5c40616aa1a1 100644
--- a/arch/arm64/include/asm/kvm_rmi.h
+++ b/arch/arm64/include/asm/kvm_rmi.h
@@ -6,12 +6,77 @@
#ifndef __ASM_KVM_RMI_H
#define __ASM_KVM_RMI_H
+#include <linux/arm-smccc-rmi.h>
+
+/**
+ * enum realm_state - State of a Realm
+ *
+ * Mirrors the RMM's Realm lifecycle states where they are meaningful to KVM,
+ * with REALM_STATE_DYING being a KVM-internal state used to prevent further
+ * requests while teardown is in progress. KVM does not track REALM_SYSTEM_OFF
+ * or REALM_ZOMBIE separately as they naturally lead to teardown.
+ */
+enum realm_state {
+ /**
+ * @REALM_STATE_NONE:
+ * Realm has not yet been created. rmi_realm_create() has not
+ * yet been called.
+ */
+ REALM_STATE_NONE,
+ /**
+ * @REALM_STATE_NEW:
+ * Realm is under construction, rmi_realm_create() has been
+ * called, but it is not yet activated. Pages may be populated.
+ */
+ REALM_STATE_NEW,
+ /**
+ * @REALM_STATE_ACTIVE:
+ * Realm has been created and is eligible for execution with
+ * rmi_rec_enter(). Pages may no longer be populated with
+ * rmi_data_create().
+ */
+ REALM_STATE_ACTIVE,
+ /**
+ * @REALM_STATE_DYING:
+ * Realm is in the process of being destroyed or has already been
+ * destroyed.
+ */
+ REALM_STATE_DYING,
+ /**
+ * @REALM_STATE_DEAD:
+ * Realm has been destroyed.
+ */
+ REALM_STATE_DEAD
+};
+
/**
* struct realm - Additional per VM data for a Realm
+ *
+ * @rd: Kernel mapping of the RMM-managed Realm Descriptor (RD) granule
+ * @params: Parameters for the RMI_REALM_CREATE command
+ * @sro: Preallocated SRO state context for Realm MMU operations
+ * @state: The lifetime state machine for the realm
+ * @ia_bits: Number of valid Input Address bits in the IPA
*/
struct realm {
+ void *rd;
+ struct realm_params *params;
+ /*
+ * Reused by RTT map/unmap SRO commands. Those commands are only
+ * issued from Realm stage-2 map/unmap paths while kvm->mmu_lock is
+ * held for write, including Realm fault handling where
+ * kvm_fault_lock() takes the write side, so concurrent use is
+ * serialized.
+ */
+ struct rmi_sro_state *sro;
+ enum realm_state state;
+ unsigned int ia_bits;
};
void kvm_init_rmi(void);
+u32 kvm_rmm_ipa_limit(void);
+
+int kvm_init_realm(struct kvm *kvm);
+void kvm_destroy_realm(struct kvm *kvm);
#endif /* __ASM_KVM_RMI_H */
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index b961c22fce3d..c4d906861736 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -266,6 +266,13 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
bitmap_zero(kvm->arch.vcpu_features, KVM_VCPU_MAX_FEATURES);
+ /* Initialise the realm bits after the generic bits are enabled */
+ if (kvm_is_realm(kvm)) {
+ ret = kvm_init_realm(kvm);
+ if (ret)
+ goto err_uninit_mmu;
+ }
+
return 0;
err_uninit_mmu:
@@ -328,6 +335,8 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
kvm_unshare_hyp(kvm, kvm + 1);
kvm_arm_teardown_hypercalls(kvm);
+ if (kvm_is_realm(kvm))
+ kvm_destroy_realm(kvm);
}
static bool kvm_has_full_ptr_auth(void)
@@ -488,6 +497,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
else
r = kvm_supports_cacheable_pfnmap();
break;
+ case KVM_CAP_ARM_RMI:
+ r = static_key_enabled(&kvm_rmi_is_available);
+ break;
default:
r = 0;
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 6c941aaa10c6..8b9efa8a3539 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -904,10 +904,14 @@ static struct kvm_pgtable_mm_ops kvm_s2_mm_ops = {
static int kvm_init_ipa_range(struct kvm_s2_mmu *mmu, unsigned long type)
{
+ struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
u32 kvm_ipa_limit = get_kvm_ipa_limit();
u64 mmfr0, mmfr1;
u32 phys_shift;
+ if (kvm_is_realm(kvm))
+ kvm_ipa_limit = kvm_rmm_ipa_limit();
+
phys_shift = KVM_VM_TYPE_ARM_IPA_SIZE(type);
if (is_protected_kvm_enabled()) {
phys_shift = kvm_ipa_limit;
@@ -957,9 +961,18 @@ static void stage2_destroy_range(struct kvm_pgtable *pgt, phys_addr_t addr,
static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
{
+ struct kvm *kvm = kvm_s2_mmu_to_kvm(pgt->mmu);
unsigned int ia_bits = VTCR_EL2_IPA(pgt->mmu->vtcr);
- stage2_destroy_range(pgt, 0, BIT(ia_bits));
+ /*
+ * Realm RTTs are inaccessible to the host and may contain stale data
+ * after the RMM has released them. The non-root RTTs are explicitly
+ * destroyed through RMI before the RD is destroyed; only the root PGD
+ * pages remain to be freed here.
+ */
+ if (!kvm_is_realm(kvm))
+ stage2_destroy_range(pgt, 0, BIT(ia_bits));
+
KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
}
@@ -1001,6 +1014,8 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
return -EINVAL;
}
+ mmu->arch = &kvm->arch;
+
err = kvm_init_ipa_range(mmu, type);
if (err)
return err;
@@ -1009,7 +1024,6 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
if (!pgt)
return -ENOMEM;
- mmu->arch = &kvm->arch;
err = KVM_PGT_FN(kvm_pgtable_stage2_init)(pgt, mmu, &kvm_s2_mm_ops);
if (err)
goto out_free_pgtable;
diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
index 247c4f033945..b81c404ead9f 100644
--- a/arch/arm64/kvm/rmi.c
+++ b/arch/arm64/kvm/rmi.c
@@ -5,6 +5,8 @@
#include <linux/kvm_host.h>
+#include <asm/kvm_emulate.h>
+#include <asm/kvm_mmu.h>
#include <asm/kvm_pgtable.h>
#include <asm/rmi_cmds.h>
#include <asm/virt.h>
@@ -14,6 +16,71 @@ static bool rmi_has_feature(int reg, unsigned long feature)
return !!u64_get_bits(rmi_feat_reg(reg), feature);
}
+u32 kvm_rmm_ipa_limit(void)
+{
+ return u64_get_bits(rmi_feat_reg(0), RMI_FEATURE_REGISTER_0_S2SZ);
+}
+
+void kvm_destroy_realm(struct kvm *kvm)
+{
+ struct realm *realm = &kvm->arch.realm;
+ size_t pgd_size = kvm_pgtable_stage2_pgd_size(kvm->arch.mmu.vtcr);
+
+ if (realm->params) {
+ free_page((unsigned long)realm->params);
+ realm->params = NULL;
+ }
+
+ if (!kvm_realm_is_created(kvm)) {
+ kfree(realm->sro);
+ realm->sro = NULL;
+ return;
+ }
+
+ kvm_set_realm_state(kvm, REALM_STATE_DYING);
+
+ if (realm->rd) {
+ phys_addr_t rd_phys = virt_to_phys(realm->rd);
+
+ if (WARN_ON(rmi_realm_terminate(rd_phys, realm->sro)))
+ return;
+
+ if (WARN_ON(rmi_realm_destroy(rd_phys, realm->sro)))
+ return;
+ free_delegated_page(rd_phys);
+ realm->rd = NULL;
+ }
+
+ if (WARN_ON(rmi_undelegate_range(kvm->arch.mmu.pgd_phys,
+ pgd_size)))
+ return;
+
+ kvm_set_realm_state(kvm, REALM_STATE_DEAD);
+
+ /* Now that the realm is destroyed, free the entry-level RTTs. */
+ kvm_free_stage2_pgd(&kvm->arch.mmu);
+
+ kfree(realm->sro);
+ realm->sro = NULL;
+}
+
+int kvm_init_realm(struct kvm *kvm)
+{
+ struct realm *realm = &kvm->arch.realm;
+
+ realm->params = (void *)get_zeroed_page(GFP_KERNEL_ACCOUNT);
+ realm->sro = kmalloc_obj(*realm->sro);
+ if (!realm->params || !realm->sro) {
+ free_page((unsigned long)realm->params);
+ kfree(realm->sro);
+ realm->params = NULL;
+ realm->sro = NULL;
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
static int rmm_check_features(void)
{
if (kvm_lpa2_is_enabled() &&
--
2.43.0
^ permalink raw reply related
* [PATCH v15 06/37] KVM: arm64: CCA: Define the user ABI
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
There is one CAP (KVM_CAP_ARM_RMI) which identifies the presence of CCA,
and one ioctl. The ioctl (KVM_ARM_RMI_POPULATE) is used to populate
memory during creation of the realm as this requires the RMM to copy
data from an unprotected address to the protected memory - CCA does not
support shared <-> private memory conversion where the memory contents
is preserved as this is incompatible with memory encryption.
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v13:
* KVM_ARM_VCPU_RMI_PSCI_COMPLETE removed.
* KVM_ARM_RMI_POPULATE documentation updated to reflect that the
structure is written by the kernel.
* CAP number bumped.
Changes since v12:
* Change KVM_ARM_RMI_POPULATE to update the structure with the amount
that has been progressed rather than return the number of bytes
populated.
* Describe the flag KVM_ARM_RMI_POPULATE_FLAGS_MEASURE.
* CAP number is bumped.
* NOTE: The PSCI ioctl may be removed in a future spec release.
Changes since v11:
* Completely reworked to be more implicit. Rather than having explicit
CAP operations to progress the realm construction these operations
are done when needed (on populating and on first vCPU run).
* Populate and PSCI complete are promoted to proper ioctls.
Changes since v10:
* Rename symbols from RME to RMI.
Changes since v9:
* Improvements to documentation.
* Bump the magic number for KVM_CAP_ARM_RME to avoid conflicts.
Changes since v8:
* Minor improvements to documentation following review.
* Bump the magic numbers to avoid conflicts.
Changes since v7:
* Add documentation of new ioctls
* Bump the magic numbers to avoid conflicts
Changes since v6:
* Rename some of the symbols to make their usage clearer and avoid
repetition.
Changes from v5:
* Actually expose the new VCPU capability (KVM_ARM_VCPU_REC) by bumping
KVM_VCPU_MAX_FEATURES - note this also exposes KVM_ARM_VCPU_HAS_EL2!
---
Documentation/virt/kvm/api.rst | 41 ++++++++++++++++++++++++++++++++++
include/uapi/linux/kvm.h | 13 +++++++++++
2 files changed, 54 insertions(+)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 304b9c3209ae..b38e090ad95d 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -6642,6 +6642,38 @@ if the guest_memfd memory was pinned in IOMMU page tables.
See also: :ref: `KVM_SET_MEMORY_ATTRIBUTES`.
+4.146 KVM_ARM_RMI_POPULATE
+--------------------------
+
+:Capability: KVM_CAP_ARM_RMI
+:Architectures: arm64
+:Type: vm ioctl
+:Parameters: struct kvm_arm_rmi_populate (in/out)
+:Returns: 0 on success, < 0 on error
+
+::
+
+ struct kvm_arm_rmi_populate {
+ __u64 base;
+ __u64 size;
+ __u64 source_uaddr;
+ __u32 flags;
+ __u32 reserved;
+ };
+
+Populate a region of protected address space by copying the data from the
+(non-protected) user space pointer provided into a protected region (backed by
+guestmem_fd). It implicitly sets the destination region to RIPAS RAM. This is
+only valid before any VCPUs have been run. The ioctl might not populate the
+entire region and in this case the kernel updates the fields `base`, `size` and
+`source_uaddr`. User space may have to repeatedly call it until `size` is 0 to
+populate the entire region.
+
+`flags` can be set to `KVM_ARM_RMI_POPULATE_FLAGS_MEASURE` to request that the
+populated data is hashed and added to the guest's Realm Initial Measurement
+(RIM) stored by the RMM. This can then be retrieved by the guest (using the RSI
+interface) to present to an attestation server.
+
.. _kvm_run:
5. The kvm_run structure
@@ -9025,6 +9057,15 @@ enabled, cmma can't be enabled anymore and pfmfi and the storage key
interpretation are disabled. If cmma has already been enabled or the
hpage_2g module parameter is not set to 1, -EINVAL is returned.
+7.48 KVM_CAP_ARM_RMI
+--------------------
+
+:Architectures: arm64
+:Target: VM
+:Parameters: None
+
+This capability indicates that support for CCA realms is available.
+
8. Other capabilities.
======================
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 129d6f630325..0231ff174a50 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -998,6 +998,7 @@ struct kvm_enable_cap {
#define KVM_CAP_S390_VSIE_ESAMODE 248
#define KVM_CAP_S390_HPAGE_2G 249
#define KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES 250
+#define KVM_CAP_ARM_RMI 251
struct kvm_irq_routing_irqchip {
__u32 irqchip;
@@ -1686,4 +1687,16 @@ struct kvm_pre_fault_memory {
__u64 padding[5];
};
+/* Available with KVM_CAP_ARM_RMI, only for VMs with KVM_VM_TYPE_ARM_REALM */
+#define KVM_ARM_RMI_POPULATE _IOWR(KVMIO, 0xd7, struct kvm_arm_rmi_populate)
+#define KVM_ARM_RMI_POPULATE_FLAGS_MEASURE (1 << 0)
+
+struct kvm_arm_rmi_populate {
+ __u64 base;
+ __u64 size;
+ __u64 source_uaddr;
+ __u32 flags;
+ __u32 reserved;
+};
+
#endif /* __LINUX_KVM_H */
--
2.43.0
^ permalink raw reply related
* [PATCH v15 05/37] KVM: arm64: CCA: Check for LPA2 support
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
If KVM has enabled LPA2 support then check that the RMM also supports
it. If there is a mismatch then disable support for realm guests as the
VMM may attempt to create a guest which is incompatible with the RMM.
Signed-off-by: Steven Price <steven.price@arm.com>
---
v15:
* Extend rmi_has_feature() to take the register number and check the
presence of SHA-256 support which is default.
v13:
* New patch
---
arch/arm64/kvm/rmi.c | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
index 384991d69f78..247c4f033945 100644
--- a/arch/arm64/kvm/rmi.c
+++ b/arch/arm64/kvm/rmi.c
@@ -5,9 +5,31 @@
#include <linux/kvm_host.h>
+#include <asm/kvm_pgtable.h>
#include <asm/rmi_cmds.h>
#include <asm/virt.h>
+static bool rmi_has_feature(int reg, unsigned long feature)
+{
+ return !!u64_get_bits(rmi_feat_reg(reg), feature);
+}
+
+static int rmm_check_features(void)
+{
+ if (kvm_lpa2_is_enabled() &&
+ !rmi_has_feature(0, RMI_FEATURE_REGISTER_0_LPA2)) {
+ kvm_err("RMM doesn't support LPA2\n");
+ return -ENXIO;
+ }
+
+ if (!rmi_has_feature(1, RMI_FEATURE_REGISTER_1_HASH_SHA_256)) {
+ kvm_err("RMM doesn't support SHA-256 measurements\n");
+ return -ENXIO;
+ }
+
+ return 0;
+}
+
void kvm_init_rmi(void)
{
/*
@@ -20,5 +42,8 @@ void kvm_init_rmi(void)
if (!is_rmi_available())
return;
+ if (rmm_check_features())
+ return;
+
/* Future patch will enable static branch kvm_rmi_is_available */
}
--
2.43.0
^ permalink raw reply related
* [PATCH v15 04/37] KVM: arm64: CCA: Check for RMI support at KVM init
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
Check if the RMI support is sufficient for using in KVM. Specifically, we
currently only support creating realm VMs when KVM is running in VHE mode.
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v13:
* Most of the init has been moved out of the 'kvm' directory so this is
much more basic now.
Changes since v12:
* Drop check for 4k page size.
Changes since v11:
* Reword slightly the comments on the realm states.
Changes since v10:
* kvm_is_realm() no longer has a NULL check.
* Rename from "rme" to "rmi" when referring to the RMM interface.
* Check for RME (hardware) support before probing for RMI support.
Changes since v8:
* No need to guard kvm_init_rme() behind 'in_hyp_mode'.
Changes since v6:
* Improved message for an unsupported RMI ABI version.
Changes since v5:
* Reword "unsupported" message from "host supports" to "we want" to
clarify that 'we' are the 'host'.
Changes since v2:
* Drop return value from kvm_init_rme(), it was always 0.
* Rely on the RMM return value to identify whether the RSI ABI is
compatible.
---
arch/arm64/include/asm/kvm_host.h | 4 ++++
arch/arm64/include/asm/kvm_rmi.h | 17 +++++++++++++++++
arch/arm64/include/asm/virt.h | 1 +
arch/arm64/kvm/Kconfig | 1 +
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/arm.c | 5 +++++
arch/arm64/kvm/rmi.c | 24 ++++++++++++++++++++++++
7 files changed, 53 insertions(+), 1 deletion(-)
create mode 100644 arch/arm64/include/asm/kvm_rmi.h
create mode 100644 arch/arm64/kvm/rmi.c
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index bae2c4f92ef5..1a5e15040111 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -27,6 +27,7 @@
#include <asm/fpsimd.h>
#include <asm/kvm.h>
#include <asm/kvm_asm.h>
+#include <asm/kvm_rmi.h>
#include <asm/vncr_mapping.h>
#define __KVM_HAVE_ARCH_INTC_INITIALIZED
@@ -424,6 +425,9 @@ struct kvm_arch {
/* Nested virtualization info */
struct dentry *debugfs_nv_dentry;
#endif
+
+ bool is_realm;
+ struct realm realm;
};
struct kvm_vcpu_fault_info {
diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
new file mode 100644
index 000000000000..57d24b244c95
--- /dev/null
+++ b/arch/arm64/include/asm/kvm_rmi.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2023-2026 ARM Ltd.
+ */
+
+#ifndef __ASM_KVM_RMI_H
+#define __ASM_KVM_RMI_H
+
+/**
+ * struct realm - Additional per VM data for a Realm
+ */
+struct realm {
+};
+
+void kvm_init_rmi(void);
+
+#endif /* __ASM_KVM_RMI_H */
diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
index b546703c3ab9..92cec42952f4 100644
--- a/arch/arm64/include/asm/virt.h
+++ b/arch/arm64/include/asm/virt.h
@@ -87,6 +87,7 @@ void __hyp_reset_vectors(void);
bool is_kvm_arm_initialised(void);
DECLARE_STATIC_KEY_FALSE(kvm_protected_mode_initialized);
+DECLARE_STATIC_KEY_FALSE(kvm_rmi_is_available);
static inline bool is_pkvm_initialized(void)
{
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 449154f9a485..189e8ad78b22 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -37,6 +37,7 @@ menuconfig KVM
select SCHED_INFO
select GUEST_PERF_EVENTS if PERF_EVENTS
select KVM_GUEST_MEMFD
+ select ARM_RMM
help
Support hosting virtualized guest machines.
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 59612d2f277c..ed3cf30eb06e 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -16,7 +16,7 @@ CFLAGS_handle_exit.o += -Wno-override-init
kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \
inject_fault.o va_layout.o handle_exit.o config.o \
guest.o debug.o reset.o sys_regs.o stacktrace.o \
- vgic-sys-reg-v3.o fpsimd.o pkvm.o \
+ vgic-sys-reg-v3.o fpsimd.o pkvm.o rmi.o \
arch_timer.o trng.o vmid.o emulate-nested.o nested.o at.o \
vgic/vgic.o vgic/vgic-init.o \
vgic/vgic-irqfd.o vgic/vgic-v2.o \
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 50adfff75be8..b961c22fce3d 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -42,6 +42,7 @@
#include <asm/kvm_nested.h>
#include <asm/kvm_pkvm.h>
#include <asm/kvm_ptrauth.h>
+#include <asm/kvm_rmi.h>
#include <asm/sections.h>
#include <asm/stacktrace/nvhe.h>
@@ -111,6 +112,8 @@ long kvm_get_cap_for_kvm_ioctl(unsigned int ioctl, long *ext)
return -EINVAL;
}
+DEFINE_STATIC_KEY_FALSE(kvm_rmi_is_available);
+
DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector);
DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_base);
@@ -3019,6 +3022,8 @@ static __init int kvm_arm_init(void)
in_hyp_mode = is_kernel_in_hyp_mode();
+ kvm_init_rmi();
+
if (cpus_have_final_cap(ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE) ||
cpus_have_final_cap(ARM64_WORKAROUND_1508412))
kvm_info("Guests without required CPU erratum workarounds can deadlock system!\n" \
diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
new file mode 100644
index 000000000000..384991d69f78
--- /dev/null
+++ b/arch/arm64/kvm/rmi.c
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2023-2026 ARM Ltd.
+ */
+
+#include <linux/kvm_host.h>
+
+#include <asm/rmi_cmds.h>
+#include <asm/virt.h>
+
+void kvm_init_rmi(void)
+{
+ /*
+ * TODO: Support Realm guests in nVHE mode, this will require adding
+ * EL2 stub(s) for REC entry and possibly other things.
+ */
+ if (!is_kernel_in_hyp_mode())
+ return;
+
+ if (!is_rmi_available())
+ return;
+
+ /* Future patch will enable static branch kvm_rmi_is_available */
+}
--
2.43.0
^ permalink raw reply related
* [PATCH v15 03/37] arm64: mm: Handle Granule Protection Faults (GPFs)
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
If the host attempts to access granules that have been delegated for use
in a realm these accesses will be caught and will trigger a Granule
Protection Fault (GPF).
A fault during a page walk signals a bug in the kernel and is handled by
oopsing the kernel. A non-page walk fault could be caused by user space
having access to a page which has been delegated to the kernel and will
trigger a SIGBUS to allow debugging why user space is trying to access a
delegated page.
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v10:
* Don't call arm64_notify_die() in do_gpf() but simply return 1.
Changes since v2:
* Include missing "Granule Protection Fault at level -1"
---
arch/arm64/mm/fault.c | 28 ++++++++++++++++++++++------
1 file changed, 22 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 85e23388f9bb..ea3ae0ca7dba 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -909,6 +909,22 @@ static int do_tag_check_fault(unsigned long far, unsigned long esr,
return 0;
}
+static int do_gpf_ptw(unsigned long far, unsigned long esr, struct pt_regs *regs)
+{
+ const struct fault_info *inf = esr_to_fault_info(esr);
+
+ die_kernel_fault(inf->name, far, esr, regs);
+ return 0;
+}
+
+static int do_gpf(unsigned long far, unsigned long esr, struct pt_regs *regs)
+{
+ if (!is_el1_instruction_abort(esr) && fixup_exception(regs, esr))
+ return 0;
+
+ return 1;
+}
+
static const struct fault_info fault_info[] = {
{ do_bad, SIGKILL, SI_KERNEL, "ttbr address size fault" },
{ do_bad, SIGKILL, SI_KERNEL, "level 1 address size fault" },
@@ -945,12 +961,12 @@ static const struct fault_info fault_info[] = {
{ do_bad, SIGKILL, SI_KERNEL, "unknown 32" },
{ do_alignment_fault, SIGBUS, BUS_ADRALN, "alignment fault" },
{ do_bad, SIGKILL, SI_KERNEL, "unknown 34" },
- { do_bad, SIGKILL, SI_KERNEL, "unknown 35" },
- { do_bad, SIGKILL, SI_KERNEL, "unknown 36" },
- { do_bad, SIGKILL, SI_KERNEL, "unknown 37" },
- { do_bad, SIGKILL, SI_KERNEL, "unknown 38" },
- { do_bad, SIGKILL, SI_KERNEL, "unknown 39" },
- { do_bad, SIGKILL, SI_KERNEL, "unknown 40" },
+ { do_gpf_ptw, SIGKILL, SI_KERNEL, "level -1 granule protection fault (translation table walk)" },
+ { do_gpf_ptw, SIGKILL, SI_KERNEL, "level 0 granule protection fault (translation table walk)" },
+ { do_gpf_ptw, SIGKILL, SI_KERNEL, "level 1 granule protection fault (translation table walk)" },
+ { do_gpf_ptw, SIGKILL, SI_KERNEL, "level 2 granule protection fault (translation table walk)" },
+ { do_gpf_ptw, SIGKILL, SI_KERNEL, "level 3 granule protection fault (translation table walk)" },
+ { do_gpf, SIGBUS, SI_KERNEL, "granule protection fault" },
{ do_bad, SIGKILL, SI_KERNEL, "level -1 address size fault" },
{ do_bad, SIGKILL, SI_KERNEL, "unknown 42" },
{ do_translation_fault, SIGSEGV, SEGV_MAPERR, "level -1 translation fault" },
--
2.43.0
^ permalink raw reply related
* [PATCH v15 02/37] KVM: arm64: Avoid including linux/kvm_host.h in kvm_pgtable.h
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
To avoid future include cycles, drop the linux/kvm_host.h include in
kvm_pgtable.h and include the lightweight headers required for the types and
inline helpers used there. Additionally provide a forward declaration for
struct kvm_s2_mmu as it's only used as a pointer in this file.
Both pgtable.c and kvm_pkvm.h relied on the indirect inclusion of
kvm_host.h, so make that explicit.
Signed-off-by: Steven Price <steven.price@arm.com>
---
New patch in v13
---
arch/arm64/include/asm/kvm_pgtable.h | 6 +++++-
arch/arm64/include/asm/kvm_pkvm.h | 2 +-
arch/arm64/kvm/hyp/pgtable.c | 1 +
3 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h
index 41a8687938eb..c2e4b29e605f 100644
--- a/arch/arm64/include/asm/kvm_pgtable.h
+++ b/arch/arm64/include/asm/kvm_pgtable.h
@@ -8,9 +8,13 @@
#define __ARM64_KVM_PGTABLE_H__
#include <linux/bits.h>
-#include <linux/kvm_host.h>
+#include <linux/kvm_types.h>
+#include <linux/rbtree_types.h>
+#include <linux/rcupdate.h>
#include <linux/types.h>
+struct kvm_s2_mmu;
+
#define KVM_PGTABLE_FIRST_LEVEL -1
#define KVM_PGTABLE_LAST_LEVEL 3
diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
index 74fedd9c5ff0..a0f7a699d690 100644
--- a/arch/arm64/include/asm/kvm_pkvm.h
+++ b/arch/arm64/include/asm/kvm_pkvm.h
@@ -9,7 +9,7 @@
#include <linux/arm_ffa.h>
#include <linux/memblock.h>
#include <linux/scatterlist.h>
-#include <asm/kvm_host.h>
+#include <linux/kvm_host.h>
#include <asm/kvm_pgtable.h>
/* Maximum number of VMs that can co-exist under pKVM. */
diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
index 91a7dfad6686..3d608b7e0cf0 100644
--- a/arch/arm64/kvm/hyp/pgtable.c
+++ b/arch/arm64/kvm/hyp/pgtable.c
@@ -8,6 +8,7 @@
*/
#include <linux/bitfield.h>
+#include <linux/kvm_host.h>
#include <asm/kvm_pgtable.h>
#include <asm/stage2_pgtable.h>
--
2.43.0
^ permalink raw reply related
* [PATCH v15 01/37] KVM: arm64: Include kvm_emulate.h in kvm/arm_psci.h
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Suzuki K Poulose, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Zenghui Yu, linux-arm-kernel,
linux-kernel, Joey Gouly, Alexandru Elisei, Christoffer Dall,
Fuad Tabba, linux-coco, Ganapatrao Kulkarni, Gavin Shan,
Shanker Donthineni, Alper Gun, Aneesh Kumar K . V, Emi Kisanuki,
Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi, Steven Price
In-Reply-To: <20260715142841.80544-1-steven.price@arm.com>
From: Suzuki K Poulose <suzuki.poulose@arm.com>
Fix a potential build error (like below, when asm/kvm_emulate.h gets
included after the kvm/arm_psci.h) by including the missing header file
in kvm/arm_psci.h:
./include/kvm/arm_psci.h: In function ‘kvm_psci_version’:
./include/kvm/arm_psci.h:29:13: error: implicit declaration of function
‘vcpu_has_feature’; did you mean ‘cpu_have_feature’? [-Werror=implicit-function-declaration]
29 | if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_PSCI_0_2)) {
| ^~~~~~~~~~~~~~~~
| cpu_have_feature
Reviewed-by: Gavin Shan <gshan@redhat.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Steven Price <steven.price@arm.com>
---
include/kvm/arm_psci.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h
index cbaec804eb83..38dab7add79b 100644
--- a/include/kvm/arm_psci.h
+++ b/include/kvm/arm_psci.h
@@ -10,6 +10,8 @@
#include <linux/kvm_host.h>
#include <uapi/linux/psci.h>
+#include <asm/kvm_emulate.h>
+
#define KVM_ARM_PSCI_0_1 PSCI_VERSION(0, 1)
#define KVM_ARM_PSCI_0_2 PSCI_VERSION(0, 2)
#define KVM_ARM_PSCI_1_0 PSCI_VERSION(1, 0)
--
2.43.0
^ permalink raw reply related
* [PATCH v15 00/37] arm64: Support for Arm CCA in KVM
From: Steven Price @ 2026-07-15 14:28 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
This series adds support for running protected VMs using KVM under the
Arm Confidential Compute Architecture (CCA).
It is the second part of the Arm CCA host support that was previously
posted as a single 44-patch series. The generic firmware/RMM support has
now been split into a separate 6-patch base series[0] so that it can also be
used by other work (and hopefully make reviewing a little easier). This
series applies on top of that base.
This is rebased on v7.2-rc1 and targets RMM v2.0-bet2[1].
The main changes since v14 are:
* The RMI definitions and wrappers have been updated for RMM
v2.0-bet2. One bet2 change, which moves metadata out of the
individual address range descriptors, has intentionally not been
implemented because that part of the spec is expected to be reverted.
* The generic RMI code has been split out into the preceding
firmware/RMM series. Common definitions and helpers are in
include/linux/arm-smccc-rmi.h and include/linux/arm-rmi-cmds.h, with
the KVM-specific wrappers left under arch/arm64.
* SRO handling has been reworked to match the new layout. Wrappers for
SRO-based RMI commands live with the SRO infrastructure, range
delegate/undelegate use the stateful command infrastructure, BUSY and
BLOCKED polling now calls cpu_relax(), and realm creation uses the
SRO flow.
* PSCI completion has been updated for the RMM v2.0-bet2 flow, which no
longer requires the host to pass the target REC to RMI_PSCI_COMPLETE.
* Realm creation has been adapted to the v2.0-bet2 realm parameter
layout, including explicit SHA-256 selection and SRO-based realm
creation.
* RTT teardown is now serialized under config_lock and is idempotent.
* KVM_ARM_RMI_POPULATE now takes slots_lock and config_lock higher in
the call path, adds lockdep assertions in the lower helpers, and
checks for overflow in the userspace-provided range.
* RMI_EXIT_RIPAS_CHANGE handling now uses addition rather than bitwise OR
when adding the shared bit, so ranges ending at the last address are
handled correctly.
* The RMI feature checks now support selecting the feature register to
query and validate that the default SHA-256 hash algorithm is
available.
* Patch subjects have been renamed to use firmware: arm_rmm for the
generic RMI layer and KVM: arm64: CCA for the KVM realm support.
There are also the usual rebase updates and smaller fixes.
The RMM v2.0 spec introduces Stateful RMI Operations (SROs), which allow
the RMM to complete an operation over several SMC calls while requesting
or returning memory to the host. This allows interrupts to be handled in
the middle of an operation and lets the RMM dynamically allocate memory
for internal tracking purposes. For example, RMI_REC_CREATE no longer
needs auxiliary granules to be provided up front, and can instead
request memory during the operation.
The SRO infrastructure itself is provided by the prerequisite
firmware/RMM series. This KVM series uses that infrastructure for the RMI
operations which can complete statefully, including realm creation, RMM
activation, GPT creation and range delegate/undelegate.
This series is based on v7.2-rc1 and Ackerley's guest_memfd in-place
series. It is also available as a git repository:
https://gitlab.arm.com/linux-arm/linux-cca cca-host/v15
Work in progress changes for kvmtool are available from the git
repository below:
https://gitlab.arm.com/linux-arm/kvmtool-cca cca/v13
The TF-RMM has now merged most of the RMM v2.0 support needed by this
series. The remaining PSCI changes have not yet merged; for testing those
changes are available in the following TF-RMM branch:
https://git.trustedfirmware.org/TF-RMM/tf-rmm.git topics/rmm-v2.0-poc_3
There is a kvm-unit-test branch which has been updated to support the
attestation used in RMM v2.0 available here:
https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/v4
[0] https://lore.kernel.org/r/20260715142739.80398-1-steven.price@arm.com
[1] https://developer.arm.com/documentation/den0137/2-0bet2/
Note that one bet2 change, which moves metadata out of the individual
address range descriptors, has intentionally not been implemented
because that part of the spec is expected to be reverted.
Jean-Philippe Brucker (6):
KVM: arm64: CCA: Propagate breakpoint and watchpoint counts to
userspace
KVM: arm64: CCA: Set breakpoint parameters through SET_ONE_REG
KVM: arm64: CCA: Propagate max SVE vector length from the RMM
KVM: arm64: CCA: Configure max SVE vector length for a Realm
KVM: arm64: CCA: Provide register list for unfinalized RECs
KVM: arm64: CCA: Provide an accurate register list
Joey Gouly (2):
KVM: arm64: CCA: Allow userspace to inject aborts
KVM: arm64: CCA: Support RSI_HOST_CALL
Steven Price (26):
KVM: arm64: Avoid including linux/kvm_host.h in kvm_pgtable.h
arm64: mm: Handle Granule Protection Faults (GPFs)
KVM: arm64: CCA: Check for RMI support at KVM init
KVM: arm64: CCA: Check for LPA2 support
KVM: arm64: CCA: Define the user ABI
KVM: arm64: CCA: Add basic infrastructure for creating a realm
KVM: arm64: CCA: Allow passing the machine type in KVM creation
KVM: arm64: CCA: Tear down RTTs
KVM: arm64: CCA: Allocate and free RECs to match vCPUs
KVM: arm64: CCA: Support the VGIC in realms
KVM: arm64: CCA: Support timers in realm RECs
KVM: arm64: CCA: Handle realm enter/exit
KVM: arm64: CCA: Handle RMI_EXIT_RIPAS_CHANGE
KVM: arm64: CCA: Handle realm MMIO emulation
KVM: arm64: Expose support for private memory
KVM: arm64: CCA: Create the realm descriptor
KVM: arm64: CCA: Activate realms on first vCPU run
KVM: arm64: CCA: Allow populating initial contents
KVM: arm64: CCA: Set RIPAS of initial memslots
KVM: arm64: CCA: Support runtime faulting of memory
KVM: arm64: CCA: Handle realm vCPU load
KVM: arm64: CCA: Validate register access for Realm VMs
KVM: arm64: CCA: Handle Realm PSCI requests
KVM: arm64: WARN on injected undef exceptions
KVM: arm64: CCA: Prevent Device mappings for realms
KVM: arm64: CCA: Enable realms to be created
Suzuki K Poulose (3):
KVM: arm64: Include kvm_emulate.h in kvm/arm_psci.h
KVM: arm64: CCA: Don't expose unsupported capabilities for realm
guests
KVM: arm64: CCA: Allow checking SVE on VM instance
Documentation/virt/kvm/api.rst | 61 +-
arch/arm64/include/asm/kvm_emulate.h | 37 +
arch/arm64/include/asm/kvm_host.h | 11 +-
arch/arm64/include/asm/kvm_pgtable.h | 6 +-
arch/arm64/include/asm/kvm_pkvm.h | 2 +-
arch/arm64/include/asm/kvm_rmi.h | 136 +++
arch/arm64/include/asm/virt.h | 1 +
arch/arm64/kvm/Kconfig | 2 +
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/arch_timer.c | 38 +-
arch/arm64/kvm/arm.c | 140 ++-
arch/arm64/kvm/guest.c | 93 +-
arch/arm64/kvm/hyp/pgtable.c | 1 +
arch/arm64/kvm/hypercalls.c | 4 +-
arch/arm64/kvm/inject_fault.c | 5 +-
arch/arm64/kvm/mmio.c | 16 +-
arch/arm64/kvm/mmu.c | 144 ++-
arch/arm64/kvm/psci.c | 14 +
arch/arm64/kvm/reset.c | 13 +-
arch/arm64/kvm/rmi-exit.c | 212 ++++
arch/arm64/kvm/rmi.c | 1523 ++++++++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 47 +-
arch/arm64/kvm/vgic/vgic-init.c | 2 +-
arch/arm64/mm/fault.c | 28 +-
include/kvm/arm_arch_timer.h | 2 +
include/kvm/arm_psci.h | 2 +
include/uapi/linux/kvm.h | 20 +-
27 files changed, 2466 insertions(+), 96 deletions(-)
create mode 100644 arch/arm64/include/asm/kvm_rmi.h
create mode 100644 arch/arm64/kvm/rmi-exit.c
create mode 100644 arch/arm64/kvm/rmi.c
--
2.43.0
^ permalink raw reply
* [PATCH 6/6] firmware: arm_rmm: Ensure the RMM has GPT entries for memory
From: Steven Price @ 2026-07-15 14:27 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142739.80398-1-steven.price@arm.com>
The RMM maintains the state of all the granules in the system to make
sure that the host is abiding by the rules. This state can be maintained
at different granularity, per page (TRACKING_FINE) or per region
(TRACKING_COARSE). The region size depends on the underlying
"RMI_GRANULE_SIZE". For a "coarse" region all pages in the region must
be of the same state, this implies we need to have "fine" tracking for
DRAM, so that we can delegate individual pages.
For now we only support a statically carved out memory for tracking
granules for the "fine" regions. This can be extended in the future to
allow modifying the tracking granularity and remove the need for a
static allocation.
Similarly, the firmware may create L0 GPT entries describing the total
address space. But if we change the "PAS" (Physical Address Space) of a
granule then the firmware may need to create L1 tables to track the PAS
at a finer granularity.
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v14:
* Move the implementation into drivers/firmware/arm_rmm.
Changes since v13:
* Moved out of KVM
---
drivers/firmware/arm_rmm/rmi.c | 92 ++++++++++++++++++++++++++++++++++
1 file changed, 92 insertions(+)
diff --git a/drivers/firmware/arm_rmm/rmi.c b/drivers/firmware/arm_rmm/rmi.c
index 6a9f61760fd7..d0c083bdf251 100644
--- a/drivers/firmware/arm_rmm/rmi.c
+++ b/drivers/firmware/arm_rmm/rmi.c
@@ -12,6 +12,8 @@
#include <asm/memory.h>
#include <asm/pgtable-hwdef.h>
+static bool arm64_rmi_is_available;
+
/* Currently only the first 2 registers are used by Linux */
#define RMI_FEAT_REG_COUNT 2
static __ro_after_init unsigned long rmi_feat_reg_cache[RMI_FEAT_REG_COUNT];
@@ -604,6 +606,89 @@ static int rmi_configure(void)
return 0;
}
+/*
+ * Make sure the area is tracked by RMM at FINE granularity.
+ * We do not support changing the tracking yet.
+ */
+static int rmi_verify_memory_tracking(phys_addr_t start, phys_addr_t end)
+{
+ while (start < end) {
+ unsigned long ret, category, state, next;
+
+ ret = rmi_granule_tracking_get(start, end, &category, &state, &next);
+ if (ret != RMI_SUCCESS ||
+ state != RMI_TRACKING_FINE ||
+ category != RMI_MEM_CATEGORY_CONVENTIONAL) {
+ /* TODO: Set granule tracking in this case */
+ pr_err("Granule tracking for region isn't fine/conventional: %llx\n",
+ start);
+ return -ENODEV;
+ }
+ start = next;
+ }
+
+ return 0;
+}
+
+static int rmi_create_gpts(phys_addr_t start, phys_addr_t end)
+{
+ struct rmi_sro_state *sro;
+ unsigned long l0gpt_sz;
+
+ sro = kmalloc_obj(*sro, GFP_KERNEL);
+ if (!sro)
+ return -ENOMEM;
+
+ l0gpt_sz = 1UL << (30 + FIELD_GET(RMI_FEATURE_REGISTER_1_L0GPTSZ,
+ rmi_feat_reg(1)));
+ start = ALIGN_DOWN(start, l0gpt_sz);
+ end = ALIGN(end, l0gpt_sz);
+
+ while (start < end) {
+ int ret = rmi_gpt_l1_create(start, sro, GFP_KERNEL);
+
+ /*
+ * Make sure the L1 GPT tables are created for the region.
+ * RMI_ERROR_GPT indicates the L1 table already exists.
+ */
+ if (ret != RMI_SUCCESS && RMI_RETURN_STATUS(ret) != RMI_ERROR_GPT) {
+ pr_err("GPT Level1 table missing for %llx\n", start);
+ kfree(sro);
+ return -ENOMEM;
+ }
+ start += l0gpt_sz;
+ }
+
+ kfree(sro);
+ return 0;
+}
+
+static int rmi_init_metadata(void)
+{
+ phys_addr_t start, end;
+ const struct memblock_region *r;
+
+ for_each_mem_region(r) {
+ int ret;
+
+ start = memblock_region_memory_base_pfn(r) << PAGE_SHIFT;
+ end = memblock_region_memory_end_pfn(r) << PAGE_SHIFT;
+ ret = rmi_verify_memory_tracking(start, end);
+ if (ret)
+ return ret;
+ ret = rmi_create_gpts(start, end);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+bool is_rmi_available(void)
+{
+ return arm64_rmi_is_available;
+}
+
static int __init arm64_init_rmi(void)
{
int ret;
@@ -621,6 +706,13 @@ static int __init arm64_init_rmi(void)
if (ret)
return ret;
+ ret = rmi_init_metadata();
+ if (ret)
+ return ret;
+
+ arm64_rmi_is_available = true;
+ pr_info("RMI configured");
+
return 0;
}
--
2.43.0
^ permalink raw reply related
* [PATCH 5/6] firmware: arm_rmm: Add support for SRO
From: Steven Price @ 2026-07-15 14:27 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142739.80398-1-steven.price@arm.com>
RMM v2.0 introduces the concept of "Stateful RMI Operations" (SRO). This
means that an SMC can return with an operation still in progress. The
host is expected to continue the operation until it reaches a conclusion
(either success or failure). During this process the RMM can request
additional memory ('donate') or hand memory back to the host
('reclaim'). The host can request an in progress operation is cancelled,
but still continue the operation until it has completed (otherwise the
incomplete operation may cause future RMM operations to fail).
The SRO is tracked using a struct rmi_sro_state object which keeps track
of any memory which has been allocated but not yet consumed by the RMM
or reclaimed from the RMM. This allows the memory to be reused in a
future request within the same operation. It will also permit an
operation to be done in a context where memory allocation may be
difficult (e.g. atomic context) with the option to abort the operation
and retry the memory allocation outside of the atomic context. The
memory stored in the struct rmi_sro_state object can then be reused on
the subsequent attempt.
Wrappers for SRO RMI commands are also provided here because they depend
on the rmi_sro_execute() implementation added by this patch.
Delegate/undelegate handles are also added here because they now use the
SRO/stateful command infrastructure.
Signed-off-by: Steven Price <steven.price@arm.com>
---
v15:
* Wrappers for SRO RMI functions are provided in this patch due to
their dependency on the SRO infrastructure.
* Fold the range delegate/undelegate wrappers into this patch because
they depend on the stateful command infrastructure.
* Add cpu_relax() calls when RMI_BUSY/RMI_BLOCKED is returned.
* Various fixes.
v14:
* SRO support has improved although is still not fully complete. The
infrastructure has been moved out of KVM.
---
arch/arm64/include/asm/rmi_cmds.h | 392 ++++++++++++++++++++++++
drivers/firmware/arm_rmm/rmi.c | 491 +++++++++++++++++++++++++++++-
include/linux/arm-rmi-cmds.h | 114 +++++++
3 files changed, 996 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/rmi_cmds.h b/arch/arm64/include/asm/rmi_cmds.h
index 7eb2c8d7f7fa..f13ecdb40ded 100644
--- a/arch/arm64/include/asm/rmi_cmds.h
+++ b/arch/arm64/include/asm/rmi_cmds.h
@@ -9,6 +9,102 @@
#include <linux/arm-rmi-cmds.h>
#include <linux/arm-smccc-rmi.h>
+/**
+ * rmi_rtt_data_map_init() - Create a protected mapping with data contents
+ * @rd: PA of the RD
+ * @data: PA of the target granule
+ * @ipa: IPA at which the granule will be mapped in the guest
+ * @src: PA of the source granule
+ * @flags: RMI_MEASURE_CONTENT if the contents should be measured
+ *
+ * Create a mapping from Protected IPA space to conventional memory, copying
+ * contents from a Non-secure Granule provided by the caller.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_rtt_data_map_init(unsigned long rd, unsigned long data,
+ unsigned long ipa, unsigned long src,
+ unsigned long flags)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_RTT_DATA_MAP_INIT, rd, data, ipa, src, flags
+ };
+
+ return rmi_sro_execute(®s);
+}
+
+/**
+ * rmi_rtt_data_map() - Create mappings in protected IPA with unknown contents
+ * @rd: PA of the RD
+ * @base: Base of the target IPA range
+ * @top: Top of the target IPA range
+ * @flags: Flags
+ * @oaddr: Output address set descriptor
+ * @out_top: Top address of range which was processed.
+ *
+ * Return RMI return code
+ */
+static inline int rmi_rtt_data_map(unsigned long rd,
+ unsigned long base,
+ unsigned long top,
+ unsigned long flags,
+ unsigned long oaddr,
+ unsigned long *out_top)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_RTT_DATA_MAP, rd, base, top, flags, oaddr
+ };
+ int ret;
+
+ ret = rmi_sro_execute(®s);
+
+ if (out_top)
+ *out_top = regs.a1;
+
+ return ret;
+}
+
+/**
+ * rmi_rtt_data_unmap() - Remove mappings to conventional memory
+ * @rd: PA of the RD for the target Realm
+ * @base: Base of the target IPA range
+ * @top: Top of the target IPA range
+ * @flags: Flags
+ * @oaddr: Output address set descriptor
+ * @out_top: Returns top IPA of range which has been unmapped
+ * @out_range: Output address range
+ * @out_count: Number of entries in output address list
+ *
+ * Removes mappings to convention memory with a target Protected IPA range.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_rtt_data_unmap(unsigned long rd,
+ unsigned long base,
+ unsigned long top,
+ unsigned long flags,
+ unsigned long oaddr,
+ unsigned long *out_top,
+ unsigned long *out_range,
+ unsigned long *out_count)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_RTT_DATA_UNMAP, rd, base, top, flags, oaddr
+ };
+ int ret;
+
+ ret = rmi_sro_execute(®s);
+
+ if (out_top)
+ *out_top = regs.a1;
+ if (out_range)
+ *out_range = regs.a2;
+ if (out_count)
+ *out_count = regs.a3;
+
+ return ret;
+}
+
/**
* rmi_psci_complete() - Complete pending PSCI command
* @calling_rec: PA of the calling REC
@@ -46,6 +142,98 @@ static inline int rmi_realm_activate(unsigned long rd)
return res.a0;
}
+/**
+ * rmi_realm_create() - Create a realm
+ * @rd: PA of the RD
+ * @params: PA of realm parameters
+ * @sro: Preallocated SRO context to be used
+ *
+ * Create a new realm using the given parameters.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_realm_create(unsigned long rd, unsigned long params,
+ struct rmi_sro_state *sro)
+{
+ return rmi_sro_memxfer_cmd(sro, GFP_KERNEL,
+ SMC_RMI_REALM_CREATE, rd, params);
+}
+
+/**
+ * rmi_realm_terminate() - Terminate a realm
+ * @rd: PA of the RD
+ * @sro: Preallocated SRO context to be used
+ *
+ * Terminates a realm, moving it into a ZOMBIE state
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_realm_terminate(unsigned long rd,
+ struct rmi_sro_state *sro)
+{
+ return rmi_sro_memxfer_cmd(sro, GFP_KERNEL,
+ SMC_RMI_REALM_TERMINATE, rd);
+}
+
+/**
+ * rmi_realm_destroy() - Destroy a realm
+ * @rd: PA of the RD
+ * @sro: Preallocated SRO context to be used
+ *
+ * Destroys a realm, all objects belonging to the realm must be destroyed first.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_realm_destroy(unsigned long rd,
+ struct rmi_sro_state *sro)
+{
+ return rmi_sro_memxfer_cmd(sro, GFP_KERNEL,
+ SMC_RMI_REALM_DESTROY, rd);
+}
+
+/**
+ * rmi_rec_create() - Create a REC
+ * @rd: PA of the RD
+ * @rec: PA of the target REC
+ * @params: PA of REC parameters
+ * @sro: Allocated SRO context to be used
+ *
+ * Create a REC using the parameters specified in the struct rec_params pointed
+ * to by @params.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_rec_create(unsigned long rd,
+ unsigned long rec,
+ unsigned long params,
+ struct rmi_sro_state *sro)
+{
+ int ret;
+
+ *sro = (struct rmi_sro_state){.regs = {
+ SMC_RMI_REC_CREATE, rd, rec, params
+ }};
+ ret = rmi_sro_memxfer_execute(sro, GFP_KERNEL);
+ rmi_sro_free(sro);
+
+ return ret;
+}
+
+/**
+ * rmi_rec_destroy() - Destroy a REC
+ * @rec: PA of the target REC
+ * @sro: Allocated SRO context to be used
+ *
+ * Destroys a REC. The REC must not be running.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_rec_destroy(unsigned long rec,
+ struct rmi_sro_state *sro)
+{
+ return rmi_sro_memxfer_cmd(sro, GFP_KERNEL, SMC_RMI_REC_DESTROY, rec);
+}
+
/**
* rmi_rec_enter() - Enter a REC
* @rec: PA of the target REC
@@ -64,4 +252,208 @@ static inline int rmi_rec_enter(unsigned long rec, unsigned long run_ptr)
return res.a0;
}
+/**
+ * rmi_rtt_create() - Creates an RTT
+ * @rd: PA of the RD
+ * @rtt: PA of the target RTT
+ * @ipa: Base of the IPA range described by the RTT
+ * @level: Depth of the RTT within the tree
+ *
+ * Creates an RTT (Realm Translation Table) at the specified level for the
+ * translation of the specified address within the realm.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_rtt_create(unsigned long rd, unsigned long rtt,
+ unsigned long ipa, long level)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_RTT_CREATE, rd, rtt, ipa, level
+ };
+
+ return rmi_sro_execute(®s);
+}
+
+/**
+ * rmi_rtt_destroy() - Destroy an RTT
+ * @rd: PA of the RD for the target realm
+ * @ipa: Base of the IPA range described by the RTT
+ * @level: RTT level
+ * @out_rtt: Pointer to write the PA of the RTT which was destroyed
+ * @out_top: Pointer to write the top IPA of non-live RTT entries, from entry
+ * at which the RTT walk terminated.
+ *
+ * Destroys an RTT. The RTT must be non-live, i.e. none of the entries in the
+ * table are in ASSIGNED or TABLE state.
+ *
+ * Return: RMI return code.
+ */
+static inline int rmi_rtt_destroy(unsigned long rd,
+ unsigned long ipa,
+ long level,
+ unsigned long *out_rtt,
+ unsigned long *out_top)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_RTT_DESTROY, rd, ipa, level
+ };
+ int ret = rmi_sro_execute(®s);
+
+ if (out_rtt)
+ *out_rtt = regs.a1;
+ if (out_top)
+ *out_top = regs.a2;
+
+ return ret;
+}
+
+/**
+ * rmi_rtt_fold() - Fold an RTT
+ * @rd: PA of the RD
+ * @ipa: Base of the IPA range described by the RTT
+ * @level: Depth of the RTT within the tree
+ * @out_rtt: Pointer to write the PA of the RTT which was destroyed
+ *
+ * Folds an RTT. If all entries with the RTT are 'homogeneous' the RTT can be
+ * folded into the parent and the RTT destroyed.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_rtt_fold(unsigned long rd, unsigned long ipa,
+ long level, unsigned long *out_rtt)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_RTT_FOLD, rd, ipa, level
+ };
+ int ret = rmi_sro_execute(®s);
+
+ if (out_rtt)
+ *out_rtt = regs.a1;
+
+ return ret;
+}
+
+/**
+ * rmi_rtt_init_ripas() - Set RIPAS for new realm
+ * @rd: PA of the RD
+ * @base: Base of target IPA region
+ * @top: Top of target IPA region
+ * @out_top: Top IPA of range whose RIPAS was modified
+ *
+ * Sets the RIPAS of a target IPA range to RAM, for a realm in the NEW state.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_rtt_init_ripas(unsigned long rd, unsigned long base,
+ unsigned long top, unsigned long *out_top)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_RTT_INIT_RIPAS, rd, base, top
+ };
+ int ret = rmi_sro_execute(®s);
+
+ if (out_top)
+ *out_top = regs.a1;
+
+ return ret;
+}
+
+/**
+ * rmi_rtt_unprot_map() - Map unprotected granules into a realm
+ * @rd: PA of the RD
+ * @base: Base IPA of the mapping
+ * @top: Top of the target IPA range
+ * @flags: Flags
+ * @oaddr: Output address set descriptor
+ * @out_top: Top IPA of range which has been mapped
+ *
+ * Create mappings to memory within a target unprotected IPA range.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_rtt_unprot_map(unsigned long rd,
+ unsigned long base,
+ unsigned long top,
+ unsigned long flags,
+ unsigned long oaddr,
+ unsigned long *out_top)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_RTT_UNPROT_MAP, rd, base, top, flags, oaddr
+ };
+ int ret = rmi_sro_execute(®s);
+
+ if (out_top)
+ *out_top = regs.a1;
+
+ return ret;
+}
+
+/**
+ * rmi_rtt_set_ripas() - Set RIPAS for an running realm
+ * @rd: PA of the RD
+ * @rec: PA of the REC making the request
+ * @base: Base of target IPA region
+ * @top: Top of target IPA region
+ * @out_top: Pointer to write top IPA of range whose RIPAS was modified
+ *
+ * Completes a request made by the realm to change the RIPAS of a target IPA
+ * range.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_rtt_set_ripas(unsigned long rd, unsigned long rec,
+ unsigned long base, unsigned long top,
+ unsigned long *out_top)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_RTT_SET_RIPAS, rd, rec, base, top
+ };
+ int ret = rmi_sro_execute(®s);
+
+ if (out_top)
+ *out_top = regs.a1;
+
+ return ret;
+}
+
+/**
+ * rmi_rtt_unprot_unmap() - Remove mappings within an unprotected IPA range
+ * @rd: PA of the RD
+ * @base: Base IPA of the mapping
+ * @top: Top of the target IPA range
+ * @flags: Flags
+ * @oaddr: Output address set descriptor
+ * @out_top: Top IPA which has been unmapped
+ * @out_range: Output address range
+ * @out_count: Number of entries in output address list
+ *
+ * Removes mappings to memory within a target unprotected IPA range.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_rtt_unprot_unmap(unsigned long rd,
+ unsigned long base,
+ unsigned long top,
+ unsigned long flags,
+ unsigned long oaddr,
+ unsigned long *out_top,
+ unsigned long *out_range,
+ unsigned long *out_count)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_RTT_UNPROT_UNMAP, rd, base, top, flags, oaddr
+ };
+ int ret = rmi_sro_execute(®s);
+
+ if (out_top)
+ *out_top = regs.a1;
+ if (out_range)
+ *out_range = regs.a2;
+ if (out_count)
+ *out_count = regs.a3;
+
+ return ret;
+}
+
#endif /* __ASM_RMI_CMDS_H */
diff --git a/drivers/firmware/arm_rmm/rmi.c b/drivers/firmware/arm_rmm/rmi.c
index e7ab4a7df3ca..6a9f61760fd7 100644
--- a/drivers/firmware/arm_rmm/rmi.c
+++ b/drivers/firmware/arm_rmm/rmi.c
@@ -6,6 +6,7 @@
#include <linux/cpufeature.h>
#include <linux/memblock.h>
#include <linux/arm-rmi-cmds.h>
+#include <linux/processor.h>
#include <linux/slab.h>
#include <asm/memory.h>
@@ -22,6 +23,487 @@ unsigned long rmi_feat_reg(unsigned long id)
return rmi_feat_reg_cache[id];
}
+EXPORT_SYMBOL_GPL(rmi_feat_reg);
+
+int rmi_delegate_range(phys_addr_t phys,
+ unsigned long size,
+ phys_addr_t *out_phys)
+{
+ unsigned long ret = 0;
+ unsigned long top = phys + size;
+ unsigned long out_top;
+
+ while (phys < top) {
+ ret = rmi_granule_range_delegate(phys, top, &out_top);
+ if (ret == RMI_SUCCESS)
+ phys = out_top;
+ else if (ret == RMI_BUSY || ret == RMI_BLOCKED)
+ cpu_relax();
+ else
+ break;
+ }
+
+ if (out_phys)
+ *out_phys = phys;
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(rmi_delegate_range);
+
+int rmi_undelegate_range(phys_addr_t phys,
+ unsigned long size)
+{
+ unsigned long ret = 0;
+ unsigned long top = phys + size;
+ unsigned long out_top;
+
+ while (phys < top) {
+ ret = rmi_granule_range_undelegate(phys, top, &out_top);
+ if (ret == RMI_SUCCESS)
+ phys = out_top;
+ else if (ret == RMI_BUSY || ret == RMI_BLOCKED)
+ cpu_relax();
+ else
+ break;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(rmi_undelegate_range);
+
+static unsigned long donate_req_to_size(unsigned long donatereq)
+{
+ unsigned long unit_size = RMI_DONATE_SIZE(donatereq);
+
+ return BIT(ARM64_HW_PGTABLE_LEVEL_SHIFT(3 - unit_size));
+}
+
+static void rmi_smccc_invoke(struct arm_smccc_1_2_regs *regs_in,
+ struct arm_smccc_1_2_regs *regs_out)
+{
+ struct arm_smccc_1_2_regs regs = *regs_in;
+ unsigned long status;
+
+ while (1) {
+ arm_smccc_1_2_invoke(®s, regs_out);
+ status = RMI_RETURN_STATUS(regs_out->a0);
+ if (status != RMI_BUSY && status != RMI_BLOCKED)
+ break;
+ cpu_relax();
+ }
+}
+
+static void rmi_op_continue(unsigned long sro_handle, unsigned long flags,
+ struct arm_smccc_1_2_regs *out_regs)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_OP_CONTINUE, sro_handle, flags
+ };
+
+ rmi_smccc_invoke(®s, out_regs);
+}
+
+static void rmi_op_cancel(unsigned long sro_handle,
+ struct arm_smccc_1_2_regs *out_regs)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_OP_CANCEL, sro_handle
+ };
+
+ rmi_smccc_invoke(®s, out_regs);
+}
+
+static void rmi_op_mem_donate(unsigned long sro_handle, unsigned long list_addr,
+ unsigned long list_count, unsigned long flags,
+ struct arm_smccc_1_2_regs *out_regs)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_OP_MEM_DONATE, sro_handle, list_addr, list_count, flags
+ };
+
+ rmi_smccc_invoke(®s, out_regs);
+}
+
+static void rmi_op_mem_reclaim(unsigned long sro_handle,
+ unsigned long list_addr,
+ unsigned long list_count,
+ struct arm_smccc_1_2_regs *out_regs)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_OP_MEM_RECLAIM, sro_handle, list_addr, list_count
+ };
+
+ rmi_smccc_invoke(®s, out_regs);
+}
+
+int free_delegated_page(phys_addr_t phys)
+{
+ if (WARN_ON_ONCE(rmi_undelegate_page(phys))) {
+ /* Undelegate failed: leak the page */
+ return -EBUSY;
+ }
+
+ free_page((unsigned long)phys_to_virt(phys));
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(free_delegated_page);
+
+static int rmi_sro_ensure_capacity(struct rmi_sro_state *sro,
+ unsigned long count)
+{
+ if (WARN_ON_ONCE(sro->addr_count > RMI_MAX_ADDR_LIST))
+ return -EOVERFLOW;
+
+ if (count > RMI_MAX_ADDR_LIST - sro->addr_count)
+ return -ENOSPC;
+
+ return 0;
+}
+
+static int rmi_sro_donate_contig(struct rmi_sro_state *sro,
+ unsigned long sro_handle,
+ unsigned long donatereq,
+ struct arm_smccc_1_2_regs *out_regs,
+ gfp_t gfp)
+{
+ unsigned long unit_size = RMI_DONATE_SIZE(donatereq);
+ unsigned long unit_size_bytes = donate_req_to_size(donatereq);
+ unsigned long count = RMI_DONATE_COUNT(donatereq);
+ unsigned long state = RMI_DONATE_STATE(donatereq);
+ unsigned long size = unit_size_bytes * count;
+ unsigned long addr_range;
+ int ret;
+ void *virt;
+ phys_addr_t phys;
+
+ for (int i = 0; i < sro->addr_count; i++) {
+ unsigned long entry = sro->addr_list[i];
+
+ if (RMI_ADDR_RANGE_SIZE(entry) == unit_size &&
+ RMI_ADDR_RANGE_COUNT(entry) == count &&
+ RMI_ADDR_RANGE_STATE(entry) == state &&
+ IS_ALIGNED(RMI_ADDR_RANGE_ADDR(entry), size)) {
+ sro->addr_count--;
+ swap(sro->addr_list[sro->addr_count],
+ sro->addr_list[i]);
+
+ goto out;
+ }
+ }
+
+ ret = rmi_sro_ensure_capacity(sro, 1);
+ if (ret)
+ return ret;
+
+ virt = alloc_pages_exact(size, gfp);
+ if (!virt)
+ return -ENOMEM;
+ phys = virt_to_phys(virt);
+
+ if (state == RMI_OP_MEM_DELEGATED) {
+ phys_addr_t delegated_phys;
+
+ if (rmi_delegate_range(phys, size, &delegated_phys)) {
+ if (!rmi_undelegate_range(phys, delegated_phys - phys))
+ free_pages_exact(virt, size);
+ return -ENXIO;
+ }
+ }
+
+ addr_range = phys & RMI_ADDR_RANGE_ADDR_MASK;
+ FIELD_MODIFY(RMI_ADDR_RANGE_SIZE_MASK, &addr_range, unit_size);
+ FIELD_MODIFY(RMI_ADDR_RANGE_COUNT_MASK, &addr_range, count);
+ FIELD_MODIFY(RMI_ADDR_RANGE_STATE_MASK, &addr_range, state);
+
+ sro->addr_list[sro->addr_count] = addr_range;
+
+out:
+ rmi_op_mem_donate(sro_handle,
+ virt_to_phys(&sro->addr_list[sro->addr_count]), 1,
+ 0, out_regs);
+
+ unsigned long donated_granules = out_regs->a1;
+ unsigned long donated_size = donated_granules << PAGE_SHIFT;
+
+ if (donated_granules == 0) {
+ /* No pages used by the RMM */
+ sro->addr_count++;
+ } else if (donated_size < size) {
+ phys = sro->addr_list[sro->addr_count] & RMI_ADDR_RANGE_ADDR_MASK;
+
+ /* Not all granules used by the RMM, free the remaining pages */
+ for (long i = donated_size; i < size; i += PAGE_SIZE) {
+ if (state == RMI_OP_MEM_DELEGATED)
+ free_delegated_page(phys + i);
+ else
+ __free_page(phys_to_page(phys + i));
+ }
+ }
+
+ return 0;
+}
+
+static int rmi_sro_donate_noncontig(struct rmi_sro_state *sro,
+ unsigned long sro_handle,
+ unsigned long donatereq,
+ struct arm_smccc_1_2_regs *out_regs,
+ gfp_t gfp)
+{
+ unsigned long unit_size = RMI_DONATE_SIZE(donatereq);
+ unsigned long unit_size_bytes = donate_req_to_size(donatereq);
+ unsigned long count = RMI_DONATE_COUNT(donatereq);
+ unsigned long state = RMI_DONATE_STATE(donatereq);
+ unsigned long found = 0;
+ unsigned long addr_list_start = sro->addr_count;
+ int ret;
+
+ for (int i = 0; i < addr_list_start && found < count; i++) {
+ unsigned long entry = sro->addr_list[i];
+
+ if (RMI_ADDR_RANGE_SIZE(entry) == unit_size &&
+ RMI_ADDR_RANGE_COUNT(entry) == 1 &&
+ RMI_ADDR_RANGE_STATE(entry) == state) {
+ addr_list_start--;
+ swap(sro->addr_list[addr_list_start],
+ sro->addr_list[i]);
+ found++;
+ i--;
+ }
+ }
+
+ ret = rmi_sro_ensure_capacity(sro, count - found);
+ if (ret)
+ return ret;
+
+ while (found < count) {
+ unsigned long addr_range;
+ void *virt = alloc_pages_exact(unit_size_bytes, gfp);
+ phys_addr_t phys;
+
+ if (!virt)
+ return -ENOMEM;
+
+ phys = virt_to_phys(virt);
+
+ if (state == RMI_OP_MEM_DELEGATED) {
+ phys_addr_t delegated_phys;
+
+ if (rmi_delegate_range(phys, unit_size_bytes,
+ &delegated_phys)) {
+ if (!rmi_undelegate_range(phys, delegated_phys - phys))
+ free_pages_exact(virt, unit_size_bytes);
+ return -ENXIO;
+ }
+ }
+
+ addr_range = phys & RMI_ADDR_RANGE_ADDR_MASK;
+ FIELD_MODIFY(RMI_ADDR_RANGE_SIZE_MASK, &addr_range, unit_size);
+ FIELD_MODIFY(RMI_ADDR_RANGE_COUNT_MASK, &addr_range, 1);
+ FIELD_MODIFY(RMI_ADDR_RANGE_STATE_MASK, &addr_range, state);
+
+ sro->addr_list[sro->addr_count++] = addr_range;
+ found++;
+ }
+
+ rmi_op_mem_donate(sro_handle,
+ virt_to_phys(&sro->addr_list[addr_list_start]),
+ found, 0, out_regs);
+
+ unsigned long donated_granules = out_regs->a1;
+ unsigned long granules_per_unit = unit_size_bytes >> PAGE_SHIFT;
+ unsigned long consumed_units;
+
+ /*
+ * The RMM shouldn't report more granules than we provided, but clamp
+ * just in case.
+ */
+ if (WARN_ON_ONCE(donated_granules > found * granules_per_unit))
+ donated_granules = found * granules_per_unit;
+
+ /*
+ * The RMM reports the consumed memory in terms of granules, but we
+ * track in the address lists in unit-sized ranges. So divide to get
+ * the number of (complete) consumed units.
+ */
+ consumed_units = donated_granules / granules_per_unit;
+ if (donated_granules % granules_per_unit) {
+ /*
+ * A unit has been partially consumed, the start is owned by
+ * the RMM, the tail is owned by the host
+ */
+ unsigned long entry =
+ sro->addr_list[addr_list_start + consumed_units];
+ phys_addr_t phys = RMI_ADDR_RANGE_ADDR(entry);
+ unsigned long donated_size =
+ (donated_granules % granules_per_unit) << PAGE_SHIFT;
+
+ /* Free the tail back */
+ for (unsigned long i = donated_size; i < unit_size_bytes;
+ i += PAGE_SIZE) {
+ if (state == RMI_OP_MEM_DELEGATED)
+ free_delegated_page(phys + i);
+ else
+ __free_page(phys_to_page(phys + i));
+ }
+
+ /*
+ * This unit is now fully 'consumed' (either held by the RMM or
+ * freed)
+ */
+ consumed_units++;
+ }
+
+ /* Keep just the units the RMM didn't use in addr_list */
+ for (unsigned long i = consumed_units; i < found; i++)
+ sro->addr_list[addr_list_start + i - consumed_units] =
+ sro->addr_list[addr_list_start + i];
+
+ sro->addr_count -= consumed_units;
+
+ return 0;
+}
+
+static int rmi_sro_donate(struct rmi_sro_state *sro,
+ unsigned long sro_handle,
+ unsigned long donatereq,
+ struct arm_smccc_1_2_regs *regs,
+ gfp_t gfp)
+{
+ unsigned long count = RMI_DONATE_COUNT(donatereq);
+
+ if (WARN_ON_ONCE(!count))
+ return 0;
+
+ if (RMI_DONATE_CONTIG(donatereq)) {
+ return rmi_sro_donate_contig(sro, sro_handle, donatereq,
+ regs, gfp);
+ } else {
+ return rmi_sro_donate_noncontig(sro, sro_handle, donatereq,
+ regs, gfp);
+ }
+}
+
+static int rmi_sro_reclaim(struct rmi_sro_state *sro,
+ unsigned long sro_handle,
+ struct arm_smccc_1_2_regs *out_regs)
+{
+ unsigned long capacity;
+ int ret;
+
+ ret = rmi_sro_ensure_capacity(sro, 1);
+ if (ret)
+ rmi_sro_free(sro);
+
+ capacity = RMI_MAX_ADDR_LIST - sro->addr_count;
+
+ rmi_op_mem_reclaim(sro_handle,
+ virt_to_phys(&sro->addr_list[sro->addr_count]),
+ capacity, out_regs);
+
+ if (WARN_ON_ONCE(out_regs->a1 > capacity))
+ out_regs->a1 = capacity;
+
+ sro->addr_count += out_regs->a1;
+
+ return 0;
+}
+
+void rmi_sro_free(struct rmi_sro_state *sro)
+{
+ for (int i = 0; i < sro->addr_count; i++) {
+ unsigned long entry = sro->addr_list[i];
+ unsigned long addr = RMI_ADDR_RANGE_ADDR(entry);
+ unsigned long unit_size = RMI_ADDR_RANGE_SIZE(entry);
+ unsigned long count = RMI_ADDR_RANGE_COUNT(entry);
+ unsigned long state = RMI_ADDR_RANGE_STATE(entry);
+ unsigned long size = donate_req_to_size(unit_size) * count;
+
+ if (state == RMI_OP_MEM_DELEGATED) {
+ if (WARN_ON_ONCE(rmi_undelegate_range(addr, size))) {
+ /* Leak the pages */
+ continue;
+ }
+ }
+ free_pages_exact(phys_to_virt(addr), size);
+ }
+
+ sro->addr_count = 0;
+}
+EXPORT_SYMBOL_GPL(rmi_sro_free);
+
+unsigned long rmi_sro_memxfer_execute(struct rmi_sro_state *sro, gfp_t gfp)
+{
+ unsigned long sro_handle;
+ struct arm_smccc_1_2_regs *regs = &sro->regs;
+
+ rmi_smccc_invoke(regs, regs);
+
+ sro_handle = regs->a1;
+
+ while (RMI_RETURN_STATUS(regs->a0) == RMI_INCOMPLETE) {
+ bool can_cancel = RMI_RETURN_CAN_CANCEL(regs->a0);
+ int ret = 0;
+
+ switch (RMI_RETURN_MEMREQ(regs->a0)) {
+ case RMI_OP_MEM_REQ_NONE:
+ rmi_op_continue(sro_handle, RMI_CONTINUE_KEEP_GOING,
+ regs);
+ break;
+ case RMI_OP_MEM_REQ_DONATE:
+ ret = rmi_sro_donate(sro, sro_handle, regs->a2, regs,
+ gfp);
+ break;
+ case RMI_OP_MEM_REQ_RECLAIM:
+ ret = rmi_sro_reclaim(sro, sro_handle, regs);
+ break;
+ default:
+ ret = WARN_ON_ONCE(1);
+ break;
+ }
+
+ if (ret) {
+ if (can_cancel)
+ rmi_op_cancel(sro_handle, regs);
+
+ if (WARN_ON_ONCE(RMI_RETURN_STATUS(regs->a0) != RMI_INCOMPLETE))
+ return ret;
+ }
+ }
+
+ return regs->a0;
+}
+EXPORT_SYMBOL_GPL(rmi_sro_memxfer_execute);
+
+/* For RMI commands that are stateful but not memory-transferring */
+unsigned long rmi_sro_execute(struct arm_smccc_1_2_regs *regs)
+{
+ unsigned long sro_handle;
+
+ rmi_smccc_invoke(regs, regs);
+
+ sro_handle = regs->a1;
+
+ while (RMI_RETURN_STATUS(regs->a0) == RMI_INCOMPLETE) {
+ bool can_cancel = RMI_RETURN_CAN_CANCEL(regs->a0);
+
+ switch (RMI_RETURN_MEMREQ(regs->a0)) {
+ case RMI_OP_MEM_REQ_NONE:
+ rmi_op_continue(sro_handle, RMI_CONTINUE_KEEP_GOING,
+ regs);
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ if (!can_cancel)
+ return regs->a0;
+
+ rmi_op_cancel(sro_handle, regs);
+ }
+ }
+
+ return regs->a0;
+}
+EXPORT_SYMBOL_GPL(rmi_sro_execute);
static int rmi_check_version(void)
{
@@ -80,8 +562,9 @@ static int rmi_configure(void)
{
unsigned long ret;
struct rmm_config *config __free(free_page) = (struct rmm_config *)get_zeroed_page(GFP_KERNEL);
+ struct rmi_sro_state *sro __free(kfree) = (struct rmi_sro_state *)kmalloc_obj(*sro);
- if (!config)
+ if (!config || !sro)
return -ENOMEM;
switch (PAGE_SIZE) {
@@ -112,6 +595,12 @@ static int rmi_configure(void)
return -EINVAL;
}
+ ret = rmi_rmm_activate(sro);
+ if (ret) {
+ pr_err("RMM activate failed\n");
+ return -ENXIO;
+ }
+
return 0;
}
diff --git a/include/linux/arm-rmi-cmds.h b/include/linux/arm-rmi-cmds.h
index c038c1d685fa..138983ab4e3c 100644
--- a/include/linux/arm-rmi-cmds.h
+++ b/include/linux/arm-rmi-cmds.h
@@ -19,10 +19,45 @@ struct rtt_entry {
int ripas;
};
+#define RMI_MAX_ADDR_LIST 256
+
+struct rmi_sro_state {
+ struct arm_smccc_1_2_regs regs;
+ unsigned long addr_count;
+ unsigned long addr_list[RMI_MAX_ADDR_LIST];
+};
+
unsigned long rmi_feat_reg(unsigned long id);
+int rmi_delegate_range(phys_addr_t phys, unsigned long size,
+ phys_addr_t *out_phys);
+int rmi_undelegate_range(phys_addr_t phys, unsigned long size);
+int free_delegated_page(phys_addr_t phys);
+
+static inline int rmi_delegate_page(phys_addr_t phys)
+{
+ return rmi_delegate_range(phys, PAGE_SIZE, NULL);
+}
+
+static inline int rmi_undelegate_page(phys_addr_t phys)
+{
+ return rmi_undelegate_range(phys, PAGE_SIZE);
+}
+
bool is_rmi_available(void);
+unsigned long rmi_sro_memxfer_execute(struct rmi_sro_state *sro, gfp_t gfp);
+void rmi_sro_free(struct rmi_sro_state *sro);
+unsigned long rmi_sro_execute(struct arm_smccc_1_2_regs *regs);
+
+#define rmi_sro_memxfer_cmd(sro, gfp, ...) ({ \
+ struct rmi_sro_state *__sro = (sro); \
+ *__sro = (struct rmi_sro_state){ .regs = {__VA_ARGS__} }; \
+ int __ret = rmi_sro_memxfer_execute(__sro, gfp); \
+ rmi_sro_free(__sro); \
+ __ret; \
+})
+
/**
* rmi_rmm_config_set() - Configure the RMM
* @cfg_ptr: PA of a struct rmm_config
@@ -40,6 +75,17 @@ static inline int rmi_rmm_config_set(unsigned long cfg_ptr)
return res.a0;
}
+/**
+ * rmi_rmm_activate() - Activate the RMM
+ * @sro: Preallocated SRO context to be used
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_rmm_activate(struct rmi_sro_state *sro)
+{
+ return rmi_sro_memxfer_cmd(sro, GFP_KERNEL, SMC_RMI_RMM_ACTIVATE);
+}
+
/**
* rmi_granule_tracking_get() - Get configuration of a Granule tracking region
* @start: Base PA of the tracking region
@@ -70,6 +116,21 @@ static inline int rmi_granule_tracking_get(unsigned long start,
return res.a0;
}
+/**
+ * rmi_gpt_l1_create() - Create a Level 1 GPT
+ * @addr: Base of physical address region described by the L1GPT
+ * @sro: Preallocated SRO context to be used
+ * @gfp: Allocation flags for SRO memory donation requests
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_gpt_l1_create(unsigned long addr,
+ struct rmi_sro_state *sro,
+ gfp_t gfp)
+{
+ return rmi_sro_memxfer_cmd(sro, gfp, SMC_RMI_GPT_L1_CREATE, addr);
+}
+
/**
* rmi_features() - Read feature register
* @index: Feature register index
@@ -88,4 +149,57 @@ static inline int rmi_features(unsigned long index, unsigned long *out)
return res.a0;
}
+/**
+ * rmi_granule_range_delegate() - Delegate granules
+ * @base: PA of the first granule of the range
+ * @top: PA of the first granule after the range
+ * @out_top: PA of the first granule not delegated
+ *
+ * Delegate a range of granule for use by the realm world. If the entire range
+ * was delegated then @out_top == @top, otherwise the function should be called
+ * again with @base == @out_top.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_granule_range_delegate(unsigned long base,
+ unsigned long top,
+ unsigned long *out_top)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_GRANULE_RANGE_DELEGATE, base, top
+ };
+ int ret = rmi_sro_execute(®s);
+
+ if (out_top)
+ *out_top = regs.a1;
+
+ return ret;
+}
+
+/**
+ * rmi_granule_range_undelegate() - Undelegate a range of granules
+ * @base: Base PA of the target range
+ * @top: Top PA of the target range
+ * @out_top: Returns the top PA of range whose state is undelegated
+ *
+ * Undelegate a range of granules to allow use by the normal world. Will fail if
+ * the granules are in use.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_granule_range_undelegate(unsigned long base,
+ unsigned long top,
+ unsigned long *out_top)
+{
+ struct arm_smccc_1_2_regs regs = {
+ SMC_RMI_GRANULE_RANGE_UNDELEGATE, base, top
+ };
+ int ret = rmi_sro_execute(®s);
+
+ if (out_top)
+ *out_top = regs.a1;
+
+ return ret;
+}
+
#endif
--
2.43.0
^ permalink raw reply related
* [PATCH 4/6] firmware: arm_rmm: Configure the RMM with the host's page size
From: Steven Price @ 2026-07-15 14:27 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142739.80398-1-steven.price@arm.com>
RMM v2.0 brings the ability to set the RMM's granule size. Check the
feature registers and configure the RMM so that it matches the host's
page size. This means that operations can be done with a granularity
equal to PAGE_SIZE.
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v14:
* Move the implementation into drivers/firmware/arm_rmm.
Changes since v13:
* Moved out of KVM.
---
drivers/firmware/arm_rmm/rmi.c | 43 ++++++++++++++++++++++++++++++++++
1 file changed, 43 insertions(+)
diff --git a/drivers/firmware/arm_rmm/rmi.c b/drivers/firmware/arm_rmm/rmi.c
index c4850976544f..e7ab4a7df3ca 100644
--- a/drivers/firmware/arm_rmm/rmi.c
+++ b/drivers/firmware/arm_rmm/rmi.c
@@ -76,6 +76,45 @@ static int rmi_read_features(void)
return 0;
}
+static int rmi_configure(void)
+{
+ unsigned long ret;
+ struct rmm_config *config __free(free_page) = (struct rmm_config *)get_zeroed_page(GFP_KERNEL);
+
+ if (!config)
+ return -ENOMEM;
+
+ switch (PAGE_SIZE) {
+ case SZ_4K:
+ config->rmi_granule_size = RMI_GRANULE_SIZE_4KB;
+ break;
+ case SZ_16K:
+ config->rmi_granule_size = RMI_GRANULE_SIZE_16KB;
+ break;
+ case SZ_64K:
+ config->rmi_granule_size = RMI_GRANULE_SIZE_64KB;
+ break;
+ default:
+ BUILD_BUG();
+ }
+
+ /*
+ * For now we set the tracking_region_size to 0 which is the only option
+ * for 4KB PAGE_SIZE (1GB for 4KB PAGE_SIZE, 32MB/512MB for 16KB/64KB).
+ * TODO: Support other tracking sizes via Kconfig option for other
+ * PAGE_SIZES
+ */
+ config->tracking_region_size = 0;
+
+ ret = rmi_rmm_config_set(virt_to_phys(config));
+ if (ret) {
+ pr_err("RMM config set failed\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int __init arm64_init_rmi(void)
{
int ret;
@@ -89,6 +128,10 @@ static int __init arm64_init_rmi(void)
if (ret)
return ret;
+ ret = rmi_configure();
+ if (ret)
+ return ret;
+
return 0;
}
--
2.43.0
^ permalink raw reply related
* [PATCH 3/6] firmware: arm_rmm: Check for RMI support at init
From: Steven Price @ 2026-07-15 14:27 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142739.80398-1-steven.price@arm.com>
Query the RMI version number and check if it is a compatible version.
The first two feature registers are read and exposed for future code to
use.
Signed-off-by: Steven Price <steven.price@arm.com>
---
v15:
* The code is moved again, this time into the 'firmware' directory.
v14:
* This moves the basic RMI setup into the 'kernel' directory. This is
because RMI will be used for some features outside of KVM so should
be available even if KVM isn't compiled in.
---
arch/arm64/Kconfig | 1 +
arch/arm64/kernel/cpufeature.c | 1 +
drivers/firmware/Kconfig | 1 +
drivers/firmware/Makefile | 1 +
drivers/firmware/arm_rmm/Kconfig | 22 +++++++
drivers/firmware/arm_rmm/Makefile | 2 +
drivers/firmware/arm_rmm/rmi.c | 100 ++++++++++++++++++++++++++++++
7 files changed, 128 insertions(+)
create mode 100644 drivers/firmware/arm_rmm/Kconfig
create mode 100644 drivers/firmware/arm_rmm/Makefile
create mode 100644 drivers/firmware/arm_rmm/rmi.c
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b3afe0688919..0f1956edf130 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -38,6 +38,7 @@ config ARM64
select ARCH_HAS_MEMBARRIER_SYNC_CORE
select ARCH_HAS_MEM_ENCRYPT
select ARCH_SUPPORTS_MSEAL_SYSTEM_MAPPINGS
+ select ARCH_SUPPORTS_RMM
select ARCH_HAS_NMI_SAFE_THIS_CPU_OPS
select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE
select ARCH_HAS_NONLEAF_PMD_YOUNG if ARM64_HAFT
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9a22df0c5120..4ae63c89cda9 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -293,6 +293,7 @@ static const struct arm64_ftr_bits ftr_id_aa64isar3[] = {
static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_CSV3_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_CSV2_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_RME_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_DIT_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_AMU_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_EL1_MPAM_SHIFT, 4, 0),
diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig
index 12dc70254842..55542879fbb8 100644
--- a/drivers/firmware/Kconfig
+++ b/drivers/firmware/Kconfig
@@ -301,5 +301,6 @@ source "drivers/firmware/samsung/Kconfig"
source "drivers/firmware/smccc/Kconfig"
source "drivers/firmware/tegra/Kconfig"
source "drivers/firmware/xilinx/Kconfig"
+source "drivers/firmware/arm_rmm/Kconfig"
endmenu
diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile
index 4ddec2820c96..07e8ca1789b4 100644
--- a/drivers/firmware/Makefile
+++ b/drivers/firmware/Makefile
@@ -38,3 +38,4 @@ obj-y += samsung/
obj-y += smccc/
obj-y += tegra/
obj-y += xilinx/
+obj-y += arm_rmm/
diff --git a/drivers/firmware/arm_rmm/Kconfig b/drivers/firmware/arm_rmm/Kconfig
new file mode 100644
index 000000000000..d224a72cca90
--- /dev/null
+++ b/drivers/firmware/arm_rmm/Kconfig
@@ -0,0 +1,22 @@
+
+config ARCH_SUPPORTS_RMM
+ bool
+
+config ARM_RMM
+ bool "Realm Management Monitor (RMM) Support"
+ depends on ARCH_SUPPORTS_RMM
+ default y
+ help
+ Support the Realm Management Monitor (RMM) on Arm systems that
+ implement the Realm Management Extension (RME), as defined by the
+ Arm Confidential Compute Architecture.
+
+ The RMM runs in the Realm world and provides the Realm Management
+ Interface (RMI) used by a Normal World host to create, manage and run
+ protected virtual machines called Realms. This option builds the
+ host-side RMI support used by KVM to detect a compatible RMM,
+ configure it, manage delegated memory and enable Realm guests.
+
+ Selecting this option does not by itself make Realm guests available:
+ the system must also provide RME-capable hardware and firmware with a
+ compatible RMM implementation.
diff --git a/drivers/firmware/arm_rmm/Makefile b/drivers/firmware/arm_rmm/Makefile
new file mode 100644
index 000000000000..51aef5cc04b0
--- /dev/null
+++ b/drivers/firmware/arm_rmm/Makefile
@@ -0,0 +1,2 @@
+
+obj-$(CONFIG_ARM_RMM) = rmi.o
diff --git a/drivers/firmware/arm_rmm/rmi.c b/drivers/firmware/arm_rmm/rmi.c
new file mode 100644
index 000000000000..c4850976544f
--- /dev/null
+++ b/drivers/firmware/arm_rmm/rmi.c
@@ -0,0 +1,100 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2023-2026 ARM Ltd.
+ */
+
+#include <linux/cpufeature.h>
+#include <linux/memblock.h>
+#include <linux/arm-rmi-cmds.h>
+#include <linux/slab.h>
+
+#include <asm/memory.h>
+#include <asm/pgtable-hwdef.h>
+
+/* Currently only the first 2 registers are used by Linux */
+#define RMI_FEAT_REG_COUNT 2
+static __ro_after_init unsigned long rmi_feat_reg_cache[RMI_FEAT_REG_COUNT];
+
+unsigned long rmi_feat_reg(unsigned long id)
+{
+ if (WARN_ON(id >= RMI_FEAT_REG_COUNT))
+ return 0;
+
+ return rmi_feat_reg_cache[id];
+}
+
+static int rmi_check_version(void)
+{
+ struct arm_smccc_res res;
+ unsigned short version_major, version_minor;
+ unsigned long host_version = RMI_ABI_VERSION(RMI_ABI_MAJOR_VERSION,
+ RMI_ABI_MINOR_VERSION);
+ unsigned long aa64pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
+
+ /* If RME isn't supported, then RMI can't be */
+ if (cpuid_feature_extract_unsigned_field(aa64pfr0, ID_AA64PFR0_EL1_RME_SHIFT) == 0)
+ return -ENXIO;
+
+ arm_smccc_1_1_invoke(SMC_RMI_VERSION, host_version, &res);
+
+ if (res.a0 == SMCCC_RET_NOT_SUPPORTED)
+ return -ENXIO;
+
+ version_major = RMI_ABI_VERSION_GET_MAJOR(res.a1);
+ version_minor = RMI_ABI_VERSION_GET_MINOR(res.a1);
+
+ if (res.a0 != RMI_SUCCESS) {
+ unsigned short high_version_major, high_version_minor;
+
+ high_version_major = RMI_ABI_VERSION_GET_MAJOR(res.a2);
+ high_version_minor = RMI_ABI_VERSION_GET_MINOR(res.a2);
+
+ pr_err("Unsupported RMI ABI (v%d.%d - v%d.%d) we want v%d.%d\n",
+ version_major, version_minor,
+ high_version_major, high_version_minor,
+ RMI_ABI_MAJOR_VERSION,
+ RMI_ABI_MINOR_VERSION);
+ return -ENXIO;
+ }
+
+ pr_info("RMI ABI version %d.%d\n", version_major, version_minor);
+
+ return 0;
+}
+
+static int rmi_read_features(void)
+{
+ /*
+ * Since we've negotiated a compatible version these feature registers
+ * should always be available
+ */
+ for (int i = 0; i < RMI_FEAT_REG_COUNT; i++) {
+ if (WARN_ON(rmi_features(i, &rmi_feat_reg_cache[i])))
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int __init arm64_init_rmi(void)
+{
+ int ret;
+
+ /* Continue without realm support if we can't agree on a version */
+ ret = rmi_check_version();
+ if (ret)
+ return ret;
+
+ ret = rmi_read_features();
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+/*
+ * Note arm64_init_rmi() must be called before kvm_init_rmi() otherwise KVM
+ * will not support realm guests. subsys_initcall() is called before
+ * module_init() (used for KVM) so this is OK.
+ */
+subsys_initcall(arm64_init_rmi);
--
2.43.0
^ permalink raw reply related
* [PATCH 2/6] firmware: arm_rmm: Add wrappers for direct RMI calls
From: Steven Price @ 2026-07-15 14:27 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142739.80398-1-steven.price@arm.com>
The wrappers make the call sites easier to read and deal with the
boilerplate of handling the error codes from the RMM.
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes from v14:
* Update to RMM v2.0-bet2 spec.
* Move SRO related wrappers to a later patch due to dependencies.
* Common wrappers are split into include/linux/arm-rmi-cmds.h.
Changes from v13:
* Update to RMM v2.0-bet1 spec including some SRO support (there still
some FIXMEs where SRO support is incomplete).
Changes from v12:
* Update to RMM v2.0 specification
Changes from v8:
* Switch from arm_smccc_1_2_smc() to arm_smccc_1_2_invoke() in
rmi_rtt_read_entry() for consistency.
Changes from v7:
* Minor renaming of parameters and updated comments
Changes from v5:
* Further improve comments
Changes from v4:
* Improve comments
Changes from v2:
* Make output arguments optional.
* Mask RIPAS value rmi_rtt_read_entry()
* Drop unused rmi_rtt_get_phys()
---
arch/arm64/include/asm/rmi_cmds.h | 67 +++++++++++++++++++++++
include/linux/arm-rmi-cmds.h | 91 +++++++++++++++++++++++++++++++
2 files changed, 158 insertions(+)
create mode 100644 arch/arm64/include/asm/rmi_cmds.h
create mode 100644 include/linux/arm-rmi-cmds.h
diff --git a/arch/arm64/include/asm/rmi_cmds.h b/arch/arm64/include/asm/rmi_cmds.h
new file mode 100644
index 000000000000..7eb2c8d7f7fa
--- /dev/null
+++ b/arch/arm64/include/asm/rmi_cmds.h
@@ -0,0 +1,67 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2023-2026 ARM Ltd.
+ */
+
+#ifndef __ASM_RMI_CMDS_H
+#define __ASM_RMI_CMDS_H
+
+#include <linux/arm-rmi-cmds.h>
+#include <linux/arm-smccc-rmi.h>
+
+/**
+ * rmi_psci_complete() - Complete pending PSCI command
+ * @calling_rec: PA of the calling REC
+ * @status: Status of the PSCI request
+ *
+ * Completes a pending PSCI command.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_psci_complete(unsigned long calling_rec,
+ unsigned long status)
+{
+ struct arm_smccc_res res;
+
+ arm_smccc_1_1_invoke(SMC_RMI_PSCI_COMPLETE, calling_rec, status, &res);
+
+ return res.a0;
+}
+
+/**
+ * rmi_realm_activate() - Active a realm
+ * @rd: PA of the RD
+ *
+ * Mark a realm as Active signalling that creation is complete and allowing
+ * execution of the realm.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_realm_activate(unsigned long rd)
+{
+ struct arm_smccc_res res;
+
+ arm_smccc_1_1_invoke(SMC_RMI_REALM_ACTIVATE, rd, &res);
+
+ return res.a0;
+}
+
+/**
+ * rmi_rec_enter() - Enter a REC
+ * @rec: PA of the target REC
+ * @run_ptr: PA of RecRun structure
+ *
+ * Starts (or continues) execution within a REC.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_rec_enter(unsigned long rec, unsigned long run_ptr)
+{
+ struct arm_smccc_res res;
+
+ arm_smccc_1_1_invoke(SMC_RMI_REC_ENTER, rec, run_ptr, &res);
+
+ return res.a0;
+}
+
+#endif /* __ASM_RMI_CMDS_H */
diff --git a/include/linux/arm-rmi-cmds.h b/include/linux/arm-rmi-cmds.h
new file mode 100644
index 000000000000..c038c1d685fa
--- /dev/null
+++ b/include/linux/arm-rmi-cmds.h
@@ -0,0 +1,91 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2026 ARM Ltd.
+ */
+
+#ifndef __LINUX_ARM_RMI_CMDS_H_
+#define __LINUX_ARM_RMI_CMDS_H_
+
+#include <linux/arm-smccc-rmi.h>
+#include <linux/bug.h>
+#include <linux/types.h>
+
+#include <asm/page.h>
+
+struct rtt_entry {
+ unsigned long walk_level;
+ unsigned long desc;
+ int state;
+ int ripas;
+};
+
+unsigned long rmi_feat_reg(unsigned long id);
+
+bool is_rmi_available(void);
+
+/**
+ * rmi_rmm_config_set() - Configure the RMM
+ * @cfg_ptr: PA of a struct rmm_config
+ *
+ * Sets configuration options on the RMM.
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_rmm_config_set(unsigned long cfg_ptr)
+{
+ struct arm_smccc_res res;
+
+ arm_smccc_1_1_invoke(SMC_RMI_RMM_CONFIG_SET, cfg_ptr, &res);
+
+ return res.a0;
+}
+
+/**
+ * rmi_granule_tracking_get() - Get configuration of a Granule tracking region
+ * @start: Base PA of the tracking region
+ * @end: End of the PA region
+ * @out_category: Memory category
+ * @out_state: Tracking region state
+ * @out_top: Top of the memory region
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_granule_tracking_get(unsigned long start,
+ unsigned long end,
+ unsigned long *out_category,
+ unsigned long *out_state,
+ unsigned long *out_top)
+{
+ struct arm_smccc_res res;
+
+ arm_smccc_1_1_invoke(SMC_RMI_GRANULE_TRACKING_GET, start, end, &res);
+
+ if (out_category)
+ *out_category = res.a1;
+ if (out_state)
+ *out_state = res.a2;
+ if (out_top)
+ *out_top = res.a3;
+
+ return res.a0;
+}
+
+/**
+ * rmi_features() - Read feature register
+ * @index: Feature register index
+ * @out: Feature register value is written to this pointer
+ *
+ * Return: RMI return code
+ */
+static inline int rmi_features(unsigned long index, unsigned long *out)
+{
+ struct arm_smccc_res res;
+
+ arm_smccc_1_1_invoke(SMC_RMI_FEATURES, index, &res);
+
+ if (out)
+ *out = res.a1;
+ return res.a0;
+}
+
+#endif
--
2.43.0
^ permalink raw reply related
* [PATCH 1/6] firmware: arm_rmm: Add SMC definitions for calling the RMM
From: Steven Price @ 2026-07-15 14:27 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
In-Reply-To: <20260715142739.80398-1-steven.price@arm.com>
The RMM (Realm Management Monitor) provides functionality that can be
accessed by SMC calls from the host.
The SMC definitions are based on DEN0137[1] version 2.0-bet2
[1] https://developer.arm.com/documentation/den0137/2-0bet2/
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v14:
* Updated to RMM spec v2.0-bet2 but without the changes to move
metadata out of individual address range descriptors as this is
expected to be reverted in a future spec release.
Changes since v13:
* Updated to RMM spec v2.0-bet1
Changes since v12:
* Updated to RMM spec v2.0-bet0
Changes since v9:
* Corrected size of 'ripas_value' in struct rec_exit. The spec states
this is an 8-bit type with padding afterwards (rather than a u64).
Changes since v8:
* Added RMI_PERMITTED_GICV3_HCR_BITS to define which bits the RMM
permits to be modified.
Changes since v6:
* Renamed REC_ENTER_xxx defines to include 'FLAG' to make it obvious
these are flag values.
Changes since v5:
* Sorted the SMC #defines by value.
* Renamed SMI_RxI_CALL to SMI_RMI_CALL since the macro is only used for
RMI calls.
* Renamed REC_GIC_NUM_LRS to REC_MAX_GIC_NUM_LRS since the actual
number of available list registers could be lower.
* Provided a define for the reserved fields of FeatureRegister0.
* Fix inconsistent names for padding fields.
Changes since v4:
* Update to point to final released RMM spec.
* Minor rearrangements.
Changes since v3:
* Update to match RMM spec v1.0-rel0-rc1.
Changes since v2:
* Fix specification link.
* Rename rec_entry->rec_enter to match spec.
* Fix size of pmu_ovf_status to match spec.
---
include/linux/arm-smccc-rmi.h | 499 ++++++++++++++++++++++++++++++++++
1 file changed, 499 insertions(+)
create mode 100644 include/linux/arm-smccc-rmi.h
diff --git a/include/linux/arm-smccc-rmi.h b/include/linux/arm-smccc-rmi.h
new file mode 100644
index 000000000000..c965e8867518
--- /dev/null
+++ b/include/linux/arm-smccc-rmi.h
@@ -0,0 +1,499 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2023-2026 ARM Ltd.
+ *
+ * The values and structures in this file are from the Realm Management Monitor
+ * specification (DEN0137) version 2.0-bet2:
+ * https://developer.arm.com/documentation/den0137/2-0bet2/
+ */
+
+#ifndef __LINUX_ARM_SMCCC_RMI_H_
+#define __LINUX_ARM_SMCCC_RMI_H_
+
+#include <linux/arm-smccc.h>
+#include <linux/bitfield.h>
+#include <linux/bits.h>
+#include <linux/build_bug.h>
+#include <linux/sizes.h>
+
+#include <asm/page.h>
+#include <asm/sysreg.h>
+
+#define SMC_RMI_CALL(func) \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
+ ARM_SMCCC_SMC_64, \
+ ARM_SMCCC_OWNER_STANDARD, \
+ (func))
+
+#define SMC_RMI_VERSION SMC_RMI_CALL(0x0150)
+
+#define SMC_RMI_RTT_DATA_MAP_INIT SMC_RMI_CALL(0x0153)
+
+#define SMC_RMI_REALM_ACTIVATE SMC_RMI_CALL(0x0157)
+#define SMC_RMI_REALM_CREATE SMC_RMI_CALL(0x0158)
+#define SMC_RMI_REALM_DESTROY SMC_RMI_CALL(0x0159)
+#define SMC_RMI_REC_CREATE SMC_RMI_CALL(0x015a)
+#define SMC_RMI_REC_DESTROY SMC_RMI_CALL(0x015b)
+#define SMC_RMI_REC_ENTER SMC_RMI_CALL(0x015c)
+#define SMC_RMI_RTT_CREATE SMC_RMI_CALL(0x015d)
+#define SMC_RMI_RTT_DESTROY SMC_RMI_CALL(0x015e)
+
+#define SMC_RMI_RTT_READ_ENTRY SMC_RMI_CALL(0x0161)
+
+#define SMC_RMI_RTT_DEV_VALIDATE SMC_RMI_CALL(0x0163)
+#define SMC_RMI_PSCI_COMPLETE SMC_RMI_CALL(0x0164)
+#define SMC_RMI_FEATURES SMC_RMI_CALL(0x0165)
+#define SMC_RMI_RTT_FOLD SMC_RMI_CALL(0x0166)
+
+#define SMC_RMI_RTT_INIT_RIPAS SMC_RMI_CALL(0x0168)
+#define SMC_RMI_RTT_SET_RIPAS SMC_RMI_CALL(0x0169)
+#define SMC_RMI_VSMMU_CREATE SMC_RMI_CALL(0x016a)
+#define SMC_RMI_VSMMU_DESTROY SMC_RMI_CALL(0x016b)
+
+#define SMC_RMI_RMM_CONFIG_SET SMC_RMI_CALL(0x016e)
+#define SMC_RMI_PSMMU_IRQ_NOTIFY SMC_RMI_CALL(0x016f)
+#define SMC_RMI_ATTEST_PLAT_TOKEN_REFRESH SMC_RMI_CALL(0x0170)
+
+#define SMC_RMI_PDEV_ABORT SMC_RMI_CALL(0x0174)
+#define SMC_RMI_PDEV_COMMUNICATE SMC_RMI_CALL(0x0175)
+#define SMC_RMI_PDEV_CREATE SMC_RMI_CALL(0x0176)
+#define SMC_RMI_PDEV_DESTROY SMC_RMI_CALL(0x0177)
+#define SMC_RMI_PDEV_GET_STATE SMC_RMI_CALL(0x0178)
+
+#define SMC_RMI_PDEV_STREAM_KEY_REFRESH SMC_RMI_CALL(0x017a)
+#define SMC_RMI_PDEV_SET_PUBKEY SMC_RMI_CALL(0x017b)
+#define SMC_RMI_PDEV_STOP SMC_RMI_CALL(0x017c)
+#define SMC_RMI_RTT_AUX_CREATE SMC_RMI_CALL(0x017d)
+#define SMC_RMI_RTT_AUX_DESTROY SMC_RMI_CALL(0x017e)
+#define SMC_RMI_RTT_AUX_FOLD SMC_RMI_CALL(0x017f)
+
+#define SMC_RMI_VDEV_ABORT SMC_RMI_CALL(0x0185)
+#define SMC_RMI_VDEV_COMMUNICATE SMC_RMI_CALL(0x0186)
+#define SMC_RMI_VDEV_CREATE SMC_RMI_CALL(0x0187)
+#define SMC_RMI_VDEV_DESTROY SMC_RMI_CALL(0x0188)
+#define SMC_RMI_VDEV_GET_STATE SMC_RMI_CALL(0x0189)
+#define SMC_RMI_VDEV_UNLOCK SMC_RMI_CALL(0x018a)
+#define SMC_RMI_RTT_SET_S2AP SMC_RMI_CALL(0x018b)
+
+#define SMC_RMI_VDEV_GET_INTERFACE_REPORT SMC_RMI_CALL(0x01d0)
+#define SMC_RMI_VDEV_GET_MEASUREMENTS SMC_RMI_CALL(0x01d1)
+#define SMC_RMI_VDEV_LOCK SMC_RMI_CALL(0x01d2)
+#define SMC_RMI_VDEV_START SMC_RMI_CALL(0x01d3)
+#define SMC_RMI_VDEV_P2P_BIND SMC_RMI_CALL(0x01d4)
+#define SMC_RMI_VDEV_P2P_UNBIND SMC_RMI_CALL(0x01d5)
+#define SMC_RMI_VSMMU_EVENT_HANDLE SMC_RMI_CALL(0x01d6)
+#define SMC_RMI_PSMMU_ACTIVATE SMC_RMI_CALL(0x01d7)
+#define SMC_RMI_PSMMU_DEACTIVATE SMC_RMI_CALL(0x01d8)
+
+#define SMC_RMI_PSMMU_ST_L2_CREATE SMC_RMI_CALL(0x01db)
+#define SMC_RMI_PSMMU_ST_L2_DESTROY SMC_RMI_CALL(0x01dc)
+#define SMC_RMI_DPT_L0_CREATE SMC_RMI_CALL(0x01dd)
+#define SMC_RMI_DPT_L0_DESTROY SMC_RMI_CALL(0x01de)
+#define SMC_RMI_DPT_L1_CREATE SMC_RMI_CALL(0x01df)
+#define SMC_RMI_DPT_L1_DESTROY SMC_RMI_CALL(0x01e0)
+#define SMC_RMI_GRANULE_TRACKING_GET SMC_RMI_CALL(0x01e1)
+
+#define SMC_RMI_GRANULE_TRACKING_SET SMC_RMI_CALL(0x01e3)
+#define SMC_RMI_CMEM_ADD_PDEV SMC_RMI_CALL(0x01e4)
+#define SMC_RMI_CMEM_CREATE SMC_RMI_CALL(0x01e5)
+#define SMC_RMI_CMEM_DESTROY SMC_RMI_CALL(0x01e6)
+#define SMC_RMI_CMEM_POPULATE SMC_RMI_CALL(0x01e7)
+#define SMC_RMI_CMEM_REMOVE_PDEV SMC_RMI_CALL(0x01e8)
+#define SMC_RMI_CMEM_START SMC_RMI_CALL(0x01e9)
+#define SMC_RMI_CMEM_STOP SMC_RMI_CALL(0x01ea)
+#define SMC_RMI_CMEM_UNPOPULATE SMC_RMI_CALL(0x01eb)
+#define SMC_RMI_RMM_CONFIG_GET SMC_RMI_CALL(0x01ec)
+#define SMC_RMI_PDEV_MEC_REFRESH SMC_RMI_CALL(0x01ed)
+#define SMC_RMI_RMM_STATE_GET SMC_RMI_CALL(0x01ee)
+
+#define SMC_RMI_PSMMU_EVENT_CONSUME SMC_RMI_CALL(0x01f0)
+#define SMC_RMI_GRANULE_RANGE_DELEGATE SMC_RMI_CALL(0x01f1)
+#define SMC_RMI_GRANULE_RANGE_UNDELEGATE SMC_RMI_CALL(0x01f2)
+#define SMC_RMI_GPT_L1_CREATE SMC_RMI_CALL(0x01f3)
+#define SMC_RMI_GPT_L1_DESTROY SMC_RMI_CALL(0x01f4)
+#define SMC_RMI_RTT_DATA_MAP SMC_RMI_CALL(0x01f5)
+#define SMC_RMI_RTT_DATA_UNMAP SMC_RMI_CALL(0x01f6)
+#define SMC_RMI_RTT_DEV_MAP SMC_RMI_CALL(0x01f7)
+#define SMC_RMI_RTT_DEV_UNMAP SMC_RMI_CALL(0x01f8)
+#define SMC_RMI_RTT_ARCH_DEV_MAP SMC_RMI_CALL(0x01f9)
+#define SMC_RMI_RTT_ARCH_DEV_UNMAP SMC_RMI_CALL(0x01fa)
+#define SMC_RMI_RTT_UNPROT_MAP SMC_RMI_CALL(0x01fb)
+#define SMC_RMI_RTT_UNPROT_UNMAP SMC_RMI_CALL(0x01fc)
+#define SMC_RMI_RTT_AUX_PROT_MAP SMC_RMI_CALL(0x01fd)
+#define SMC_RMI_RTT_AUX_PROT_UNMAP SMC_RMI_CALL(0x01fe)
+#define SMC_RMI_RTT_AUX_UNPROT_MAP SMC_RMI_CALL(0x01ff)
+#define SMC_RMI_RTT_AUX_UNPROT_UNMAP SMC_RMI_CALL(0x0200)
+#define SMC_RMI_REALM_TERMINATE SMC_RMI_CALL(0x0201)
+#define SMC_RMI_RMM_ACTIVATE SMC_RMI_CALL(0x0202)
+#define SMC_RMI_OP_CONTINUE SMC_RMI_CALL(0x0203)
+#define SMC_RMI_PDEV_STREAM_CONNECT SMC_RMI_CALL(0x0204)
+#define SMC_RMI_PDEV_STREAM_DISCONNECT SMC_RMI_CALL(0x0205)
+#define SMC_RMI_PDEV_STREAM_COMPLETE SMC_RMI_CALL(0x0206)
+#define SMC_RMI_PDEV_STREAM_KEY_PURGE SMC_RMI_CALL(0x0207)
+#define SMC_RMI_OP_MEM_DONATE SMC_RMI_CALL(0x0208)
+#define SMC_RMI_OP_MEM_RECLAIM SMC_RMI_CALL(0x0209)
+#define SMC_RMI_OP_CANCEL SMC_RMI_CALL(0x020a)
+#define SMC_RMI_VSMMU_FEATURES SMC_RMI_CALL(0x020b)
+#define SMC_RMI_VSMMU_CMD_GET SMC_RMI_CALL(0x020c)
+#define SMC_RMI_VSMMU_CMD_COMPLETE SMC_RMI_CALL(0x020d)
+#define SMC_RMI_PSMMU_INFO SMC_RMI_CALL(0x020e)
+
+#define RMI_ABI_MAJOR_VERSION 2
+#define RMI_ABI_MINOR_VERSION 0
+
+#define RMI_ABI_VERSION_GET_MAJOR(version) ((version) >> 16)
+#define RMI_ABI_VERSION_GET_MINOR(version) ((version) & 0xFFFF)
+#define RMI_ABI_VERSION(major, minor) (((major) << 16) | (minor))
+
+#define RMI_RETURN_STATUS_MASK (0xFFUL)
+#define RMI_RETURN_INDEX_MASK (0xFFUL << 8)
+#define RMI_RETURN_MEMREQ_MASK (0x3UL << 8)
+#define RMI_RETURN_CAN_CANCEL_MASK (0x1UL << 10)
+
+#define RMI_RETURN_STATUS(ret) FIELD_GET(RMI_RETURN_STATUS_MASK, ret)
+#define RMI_RETURN_INDEX(ret) FIELD_GET(RMI_RETURN_INDEX_MASK, ret)
+#define RMI_RETURN_MEMREQ(ret) FIELD_GET(RMI_RETURN_MEMREQ_MASK, ret)
+#define RMI_RETURN_CAN_CANCEL(ret) FIELD_GET(RMI_RETURN_CAN_CANCEL_MASK, ret)
+
+#define RMI_SUCCESS 0
+#define RMI_ERROR_INPUT 1
+#define RMI_ERROR_REALM 2
+#define RMI_ERROR_REC 3
+#define RMI_ERROR_RTT 4
+#define RMI_ERROR_NOT_SUPPORTED 5
+#define RMI_ERROR_DEVICE 6
+#define RMI_ERROR_RTT_AUX 7
+#define RMI_ERROR_PSMMU_ST 8
+#define RMI_ERROR_DPT 9
+#define RMI_BUSY 10
+#define RMI_ERROR_GLOBAL 11
+#define RMI_ERROR_TRACKING 12
+#define RMI_INCOMPLETE 13
+#define RMI_BLOCKED 14
+#define RMI_ERROR_GPT 15
+#define RMI_ERROR_GRANULE 16
+
+#define RMI_CONTINUE_KEEP_GOING 0
+#define RMI_CONTINUE_STOP 1
+
+#define RMI_OP_MEM_REQ_NONE 0
+#define RMI_OP_MEM_REQ_DONATE 1
+#define RMI_OP_MEM_REQ_RECLAIM 2
+
+#define RMI_DONATE_SIZE_MASK 3UL
+#define RMI_DONATE_COUNT_MASK GENMASK(15, 2)
+#define RMI_DONATE_CONTIG_MASK BIT(16)
+#define RMI_DONATE_STATE_MASK BIT(17)
+
+#define RMI_DONATE_SIZE(req) FIELD_GET(RMI_DONATE_SIZE_MASK, req)
+#define RMI_DONATE_COUNT(req) FIELD_GET(RMI_DONATE_COUNT_MASK, req)
+#define RMI_DONATE_CONTIG(req) FIELD_GET(RMI_DONATE_CONTIG_MASK, req)
+#define RMI_DONATE_STATE(req) FIELD_GET(RMI_DONATE_STATE_MASK, req)
+
+#define RMI_OP_MEM_DELEGATED 0
+#define RMI_OP_MEM_UNDELEGATED 1
+
+#define RMI_ADDR_TYPE_NONE 0
+#define RMI_ADDR_TYPE_SINGLE 1
+#define RMI_ADDR_TYPE_LIST 2
+
+#define RMI_ADDR_RANGE_SIZE_MASK GENMASK(1, 0)
+#define RMI_ADDR_RANGE_COUNT_MASK GENMASK(PAGE_SHIFT - 1, 2)
+#define RMI_ADDR_RANGE_ADDR_MASK (PAGE_MASK & GENMASK(51, 0))
+#define RMI_ADDR_RANGE_STATE_MASK BIT(63)
+
+#define RMI_ADDR_RANGE_SIZE(ar) (FIELD_GET(RMI_ADDR_RANGE_SIZE_MASK, \
+ (ar)))
+#define RMI_ADDR_RANGE_COUNT(ar) (FIELD_GET(RMI_ADDR_RANGE_COUNT_MASK, \
+ (ar)))
+#define RMI_ADDR_RANGE_ADDR(ar) ((ar) & RMI_ADDR_RANGE_ADDR_MASK)
+#define RMI_ADDR_RANGE_STATE(ar) (FIELD_GET(RMI_ADDR_RANGE_STATE_MASK, \
+ (ar)))
+
+enum rmi_ripas {
+ RMI_EMPTY = 0,
+ RMI_RAM = 1,
+ RMI_DESTROYED = 2,
+ RMI_DEV = 3,
+};
+
+#define RMI_NO_MEASURE_CONTENT 0
+#define RMI_MEASURE_CONTENT 1
+
+#define RMI_FEATURE_REGISTER_0_S2SZ GENMASK(7, 0)
+#define RMI_FEATURE_REGISTER_0_LPA2 BIT(8)
+#define RMI_FEATURE_REGISTER_0_SVE BIT(9)
+#define RMI_FEATURE_REGISTER_0_SVE_VL GENMASK(13, 10)
+#define RMI_FEATURE_REGISTER_0_NUM_BPS GENMASK(19, 14)
+#define RMI_FEATURE_REGISTER_0_NUM_WPS GENMASK(25, 20)
+#define RMI_FEATURE_REGISTER_0_PMU BIT(26)
+#define RMI_FEATURE_REGISTER_0_PMU_NUM_CTRS GENMASK(31, 27)
+
+#define RMI_FEATURE_REGISTER_1_RMI_GRAN_SZ_4KB BIT(0)
+#define RMI_FEATURE_REGISTER_1_RMI_GRAN_SZ_16KB BIT(1)
+#define RMI_FEATURE_REGISTER_1_RMI_GRAN_SZ_64KB BIT(2)
+#define RMI_FEATURE_REGISTER_1_HASH_SHA_256 BIT(3)
+#define RMI_FEATURE_REGISTER_1_HASH_SHA_384 BIT(4)
+#define RMI_FEATURE_REGISTER_1_HASH_SHA_512 BIT(5)
+#define RMI_FEATURE_REGISTER_1_MAX_RECS_ORDER GENMASK(9, 6)
+#define RMI_FEATURE_REGISTER_1_L0GPTSZ GENMASK(13, 10)
+#define RMI_FEATURE_REGISTER_1_PPS GENMASK(16, 14)
+
+#define RMI_FEATURE_REGISTER_2_DA BIT(0)
+#define RMI_FEATURE_REGISTER_2_DA_COH BIT(1)
+#define RMI_FEATURE_REGISTER_2_VSMMU BIT(2)
+#define RMI_FEATURE_REGISTER_2_ATS BIT(3)
+#define RMI_FEATURE_REGISTER_2_MAX_VDEVS_ORDER GENMASK(7, 4)
+#define RMI_FEATURE_REGISTER_2_VDEV_KROU BIT(8)
+#define RMI_FEATURE_REGISTER_2_NON_TEE_STREAM BIT(9)
+#define RMI_FEATURE_REGISTER_2_P2P BIT(10)
+#define RMI_FEATURE_REGISTER_2_CMEM_CXL BIT(11)
+#define RMI_FEATURE_REGISTER_2_MAX_CMEM GENMASK(19, 12)
+#define RMI_FEATURE_REGISTER_2_CMEM_TSE_REQ BIT(20)
+
+#define RMI_FEATURE_REGISTER_3_MAX_NUM_AUX_PLANES GENMASK(3, 0)
+#define RMI_FEATURE_REGISTER_3_RTT_PLANE GENMASK(5, 4)
+#define RMI_FEATURE_REGISTER_3_RTT_S2AP_INDIRECT BIT(6)
+
+#define RMI_FEATURE_REGISTER_4_MEC_COUNT GENMASK(63, 0)
+
+#define RMI_MEM_CATEGORY_CONVENTIONAL 0
+#define RMI_MEM_CATEGORY_DEV_NCOH 1
+#define RMI_MEM_CATEGORY_DEV_COH 2
+
+#define RMI_TRACKING_RESERVED 0
+#define RMI_TRACKING_NONE 1
+#define RMI_TRACKING_FINE 2
+#define RMI_TRACKING_COARSE 3
+
+#define RMI_GRANULE_SIZE_4KB 0
+#define RMI_GRANULE_SIZE_16KB 1
+#define RMI_GRANULE_SIZE_64KB 2
+
+/*
+ * Note many of these fields are smaller than u64 but all fields have u64
+ * alignment, so use u64 to ensure correct alignment.
+ */
+struct rmm_config {
+ union { /* 0x0 */
+ struct {
+ u64 tracking_region_size;
+ u64 rmi_granule_size;
+ };
+ u8 sizer[SZ_4K];
+ };
+};
+
+static_assert(sizeof(struct rmm_config) == SZ_4K);
+
+#define RMI_REALM_PARAM_FLAG_LPA2 BIT(0)
+#define RMI_REALM_PARAM_FLAG_SVE BIT(1)
+#define RMI_REALM_PARAM_FLAG_PMU BIT(2)
+#define RMI_REALM_PARAM_FLAG_DA BIT(3)
+#define RMI_REALM_PARAM_FLAG_LFA_POLICY GENMASK(6, 5)
+#define RMI_REALM_PARAM_FLAG_MEC_POLICY GENMASK(8, 7)
+
+#define RMI_HASH_SHA_256 0
+#define RMI_HASH_SHA_512 1
+#define RMI_HASH_SHA_384 2
+
+struct realm_params {
+ union { /* 0x0 */
+ struct {
+ u64 flags0;
+ u64 s2sz;
+ u64 sve_vl;
+ u64 num_bps;
+ u64 num_wps;
+ u64 pmu_num_ctrs;
+ u64 hash_algo;
+ u64 num_aux_planes;
+ };
+ u8 padding0[0x400];
+ };
+ union { /* 0x400 */
+ struct {
+ u8 rpv[64];
+ u64 ats_plane;
+ };
+ u8 padding1[0x400];
+ };
+ union { /* 0x800 */
+ struct {
+ u64 padding;
+ u64 rtt_base;
+ s64 rtt_level_start;
+ u64 rtt_num_start;
+ u64 flags1;
+ u64 aux_rtt_base[3];
+ };
+ u8 padding2[0x800];
+ };
+};
+
+static_assert(sizeof(struct realm_params) == SZ_4K);
+
+/*
+ * The number of GPRs (starting from X0) that are
+ * configured by the host when a REC is created.
+ */
+#define REC_CREATE_NR_GPRS 8
+
+#define REC_PARAMS_FLAG_RUNNABLE BIT_ULL(0)
+
+struct rec_params {
+ union { /* 0x0 */
+ u64 flags;
+ u8 padding0[0x100];
+ };
+ union { /* 0x100 */
+ u64 mpidr;
+ u8 padding1[0x100];
+ };
+ union { /* 0x200 */
+ u64 pc;
+ u8 padding2[0x100];
+ };
+ union { /* 0x300 */
+ u64 gprs[REC_CREATE_NR_GPRS];
+ u8 padding3[0xd00];
+ };
+};
+
+static_assert(sizeof(struct rec_params) == SZ_4K);
+
+#define REC_ENTER_FLAG_EMULATED_MMIO BIT(0)
+#define REC_ENTER_FLAG_INJECT_SEA BIT(1)
+#define REC_ENTER_FLAG_TRAP_WFI BIT(2)
+#define REC_ENTER_FLAG_TRAP_WFE BIT(3)
+#define REC_ENTER_FLAG_RIPAS_RESPONSE BIT(4)
+#define REC_ENTER_FLAG_S2AP_RESPONSE BIT(5)
+#define REC_ENTER_FLAG_DEV_MEM_RESPONSE BIT(6)
+#define REC_ENTER_FLAG_FORCE_P0 BIT(7)
+
+#define REC_RUN_GPRS 31
+#define REC_MAX_GIC_NUM_LRS 16
+
+#define RMI_PERMITTED_GICV3_HCR_BITS (ICH_HCR_EL2_UIE | \
+ ICH_HCR_EL2_LRENPIE | \
+ ICH_HCR_EL2_NPIE | \
+ ICH_HCR_EL2_VGrp0EIE | \
+ ICH_HCR_EL2_VGrp0DIE | \
+ ICH_HCR_EL2_VGrp1EIE | \
+ ICH_HCR_EL2_VGrp1DIE | \
+ ICH_HCR_EL2_TDIR)
+
+struct rec_enter {
+ union { /* 0x000 */
+ u64 flags;
+ u8 padding0[0x200];
+ };
+ union { /* 0x200 */
+ u64 gprs[REC_RUN_GPRS];
+ u8 padding1[0x100];
+ };
+ u8 padding3[0x500];
+};
+
+static_assert(sizeof(struct rec_enter) == SZ_2K);
+
+#define RMI_EXIT_SYNC 0x00
+#define RMI_EXIT_IRQ 0x01
+#define RMI_EXIT_FIQ 0x02
+#define RMI_EXIT_PSCI 0x03
+#define RMI_EXIT_RIPAS_CHANGE 0x04
+#define RMI_EXIT_HOST_CALL 0x05
+#define RMI_EXIT_SERROR 0x06
+#define RMI_EXIT_S2AP_CHANGE 0x07
+#define RMI_EXIT_VDEV_VALIDATE_MAPPING 0x08
+#define RMI_EXIT_VSMMU_COMMAND 0x0a
+#define RMI_EXIT_VDEV_P2P_BINDING 0x0b
+
+struct rec_exit {
+ union { /* 0x000 */
+ u8 exit_reason;
+ u8 padding0[0x100];
+ };
+ union { /* 0x100 */
+ struct {
+ u64 esr;
+ u64 far;
+ u64 hpfar;
+ u64 rtt_tree;
+ };
+ u8 padding1[0x100];
+ };
+ union { /* 0x200 */
+ u64 gprs[REC_RUN_GPRS];
+ u8 padding2[0x100];
+ };
+ union { /* 0x300 */
+ u8 padding3[0x100];
+ };
+ union { /* 0x400 */
+ struct {
+ u64 cntp_ctl;
+ u64 cntp_cval;
+ u64 cntv_ctl;
+ u64 cntv_cval;
+ };
+ u8 padding4[0x100];
+ };
+ union { /* 0x500 */
+ struct {
+ u64 ripas_base;
+ u64 ripas_top;
+ u8 ripas_value;
+ u8 padding8[15];
+ u64 s2ap_base;
+ u64 s2ap_top;
+ u64 vdev_id_1;
+ u64 vdev_id_2;
+ u64 dev_mem_base;
+ u64 dev_mem_top;
+ u64 dev_mem_pa;
+ };
+ u8 padding5[0x100];
+ };
+ union { /* 0x600 */
+ struct {
+ u16 imm;
+ u16 padding9;
+ u64 plane;
+ };
+ u8 padding6[0x100];
+ };
+ union { /* 0x700 */
+ struct {
+ u8 pmu_ovf_status;
+ u8 padding10[15];
+ u64 vsmmu;
+ };
+ u8 padding7[0x100];
+ };
+};
+
+static_assert(sizeof(struct rec_exit) == SZ_2K);
+
+struct rec_run {
+ struct rec_enter enter;
+ struct rec_exit exit;
+};
+
+static_assert(sizeof(struct rec_run) == SZ_4K);
+
+/* RMI_RTT_UNPROT_MAP_FLAGS definitions */
+#define RMI_RTT_UNPROT_MAP_FLAGS_OADDR_TYPE GENMASK(1, 0)
+#define RMI_RTT_UNPROT_MAP_FLAGS_LIST_COUNT GENMASK(15, 2)
+#define RMI_RTT_UNPROT_MAP_FLAGS_MEMATTR GENMASK(18, 16)
+#define RMI_RTT_UNPROT_MAP_FLAGS_S2AP GENMASK(22, 19)
+
+/* RMI_RTT_DATA_MAP_FLAGS definitions */
+#define RMI_RTT_PROT_MAP_FLAGS_OADDR_TYPE GENMASK(1, 0)
+#define RMI_RTT_PROT_MAP_FLAGS_LIST_COUNT GENMASK(15, 2)
+
+/* S2AP Direct Encodings, used in RMI_RTT_UNPROT_MAP_FLAGS_S2AP */
+#define RMI_S2AP_DIRECT_WRITE BIT(0)
+#define RMI_S2AP_DIRECT_READ BIT(1)
+
+#endif /* __LINUX_ARM_SMCCC_RMI_H_ */
--
2.43.0
^ permalink raw reply related
* [PATCH 0/6] firmware: arm_rmm: Add RMM v2.0 support
From: Steven Price @ 2026-07-15 14:27 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo Pieralisi
This series adds the generic firmware layer for talking to the Realm
Management Monitor (RMM), as specified by the RMM v2.0-bet2
specification[1]. It is the first part of the Arm CCA host support that
was previously posted as part of the larger KVM series.
The split allows this RMM support to be used as a base for other work,
including Aneesh's changes, without depending on the KVM Realm support
that follows in the second series.
The series adds:
* The RMI SMC definitions and direct-call wrappers.
* RMM discovery and version checks during firmware init.
* RMM host configuration, including the host page size.
* Stateful RMI Operation (SRO) infrastructure for commands which the RMM
can complete across multiple SMC calls while requesting or returning
memory to the host.
* GPT setup so that the RMM has entries for host memory.
The RMM v2.0 spec introduces Stateful RMI Operations (SROs), which allow
the RMM to complete an operation over several SMC calls while requesting
or returning memory to the host. This allows interrupts to be handled in
the middle of an operation and lets the RMM dynamically allocate memory
for internal tracking purposes. For example, RMI_REC_CREATE no longer
needs auxiliary granules to be provided up front, and can instead
request memory during the operation.
This series is based on v7.2-rc1 and Ackerley's guest_memfd in-place
series[2]. It is also available as a git repository along with the KVM
host support:
https://gitlab.arm.com/linux-arm/linux-cca cca-host/v15
The TF-RMM has now merged most of the RMM v2.0 support needed by this
series. The remaining PSCI changes have not yet merged; for testing those
changes are available in the following TF-RMM branch:
https://git.trustedfirmware.org/TF-RMM/tf-rmm.git topics/rmm-v2.0-poc_3
[1] https://developer.arm.com/documentation/den0137/2-0bet2/
Note that one bet2 change, which moves metadata out of the individual
address range descriptors, has intentionally not been implemented
because that part of the spec is expected to be reverted.
[2] https://lore.kernel.org/r/20260618-gmem-inplace-conversion-v8-0-9d2959357853@google.com
with Suzuki's fix on top:
https://lore.kernel.org/all/114e2488-97ed-4740-a8e8-1edd991f26c5@arm.com/
Steven Price (6):
firmware: arm_rmm: Add SMC definitions for calling the RMM
firmware: arm_rmm: Add wrappers for direct RMI calls
firmware: arm_rmm: Check for RMI support at init
firmware: arm_rmm: Configure the RMM with the host's page size
firmware: arm_rmm: Add support for SRO
firmware: arm_rmm: Ensure the RMM has GPT entries for memory
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/rmi_cmds.h | 459 +++++++++++++++++++
arch/arm64/kernel/cpufeature.c | 1 +
drivers/firmware/Kconfig | 1 +
drivers/firmware/Makefile | 1 +
drivers/firmware/arm_rmm/Kconfig | 22 +
drivers/firmware/arm_rmm/Makefile | 2 +
drivers/firmware/arm_rmm/rmi.c | 724 ++++++++++++++++++++++++++++++
include/linux/arm-rmi-cmds.h | 205 +++++++++
include/linux/arm-smccc-rmi.h | 499 ++++++++++++++++++++
10 files changed, 1915 insertions(+)
create mode 100644 arch/arm64/include/asm/rmi_cmds.h
create mode 100644 drivers/firmware/arm_rmm/Kconfig
create mode 100644 drivers/firmware/arm_rmm/Makefile
create mode 100644 drivers/firmware/arm_rmm/rmi.c
create mode 100644 include/linux/arm-rmi-cmds.h
create mode 100644 include/linux/arm-smccc-rmi.h
--
2.43.0
^ permalink raw reply
* Re: [PATCH v14 26/44] arm64: RMI: Allow populating initial contents
From: Steven Price @ 2026-07-15 14:06 UTC (permalink / raw)
To: Kohei Enju
Cc: kvm, kvmarm, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo.Pieralisi2
In-Reply-To: <alWzvLK1RRotK1LL@FCCLS0092175.localdomain>
On 14/07/2026 05:37, Kohei Enju wrote:
> On 05/13 14:17, Steven Price wrote:
>> The VMM needs to populate the realm with some data before starting (e.g.
>> a kernel and initrd). This is measured by the RMM and used as part of
>> the attestation later on.
>>
>> Signed-off-by: Steven Price <steven.price@arm.com>
>> ---
>> Changes since v13:
>> * Rename realm_create_protected_data_page() to realm_data_map_init().
>> Changes since v12:
>> * The ioctl now updates the structure with the amount populated rather
>> than returning this through the ioctl return code.
>> * Use the new RMM v2.0 range based RMI calls.
>> * Adapt to upstream changes in kvm_gmem_populate().
>> Changes since v11:
>> * The multiplex CAP is gone and there's a new ioctl which makes use of
>> the generic kvm_gmem_populate() functionality.
>> Changes since v7:
>> * Improve the error codes.
>> * Other minor changes from review.
>> Changes since v6:
>> * Handle host potentially having a larger page size than the RMM
>> granule.
>> * Drop historic "par" (protected address range) from
>> populate_par_region() - it doesn't exist within the current
>> architecture.
>> * Add a cond_resched() call in kvm_populate_realm().
>> Changes since v5:
>> * Refactor to use PFNs rather than tracking struct page in
>> realm_create_protected_data_page().
>> * Pull changes from a later patch (in the v5 series) for accessing
>> pages from a guest memfd.
>> * Do the populate in chunks to avoid holding locks for too long and
>> triggering RCU stall warnings.
>> ---
>> arch/arm64/include/asm/kvm_rmi.h | 4 ++
>> arch/arm64/kvm/Kconfig | 1 +
>> arch/arm64/kvm/arm.c | 13 ++++
>> arch/arm64/kvm/rmi.c | 106 +++++++++++++++++++++++++++++++
>> 4 files changed, 124 insertions(+)
>>
>> diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
>> index 007249a13dbc..a2b6bc412a22 100644
>> --- a/arch/arm64/include/asm/kvm_rmi.h
>> +++ b/arch/arm64/include/asm/kvm_rmi.h
>> @@ -88,6 +88,10 @@ int kvm_rec_enter(struct kvm_vcpu *vcpu);
>> int kvm_rec_pre_enter(struct kvm_vcpu *vcpu);
>> int handle_rec_exit(struct kvm_vcpu *vcpu, int rec_run_status);
>>
>> +struct kvm_arm_rmi_populate;
>> +
>> +int kvm_arm_rmi_populate(struct kvm *kvm,
>> + struct kvm_arm_rmi_populate *arg);
>> void kvm_realm_unmap_range(struct kvm *kvm,
>> unsigned long ipa,
>> unsigned long size,
>> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
>> index 4e16719fda22..d0cd011cf672 100644
>> --- a/arch/arm64/kvm/Kconfig
>> +++ b/arch/arm64/kvm/Kconfig
>> @@ -38,6 +38,7 @@ menuconfig KVM
>> select GUEST_PERF_EVENTS if PERF_EVENTS
>> select KVM_GUEST_MEMFD
>> select KVM_GENERIC_MEMORY_ATTRIBUTES
>> + select HAVE_KVM_ARCH_GMEM_POPULATE
>> help
>> Support hosting virtualized guest machines.
>>
>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
>> index ed88a203b892..073ba9181da9 100644
>> --- a/arch/arm64/kvm/arm.c
>> +++ b/arch/arm64/kvm/arm.c
>> @@ -2131,6 +2131,19 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
>> return -EFAULT;
>> return kvm_vm_ioctl_get_reg_writable_masks(kvm, &range);
>> }
>> + case KVM_ARM_RMI_POPULATE: {
>> + struct kvm_arm_rmi_populate req;
>> + int ret;
>> +
>> + if (!kvm_is_realm(kvm))
>> + return -ENXIO;
>> + if (copy_from_user(&req, argp, sizeof(req)))
>> + return -EFAULT;
>> + ret = kvm_arm_rmi_populate(kvm, &req);
>> + if (copy_to_user(argp, &req, sizeof(req)))
>> + return -EFAULT;
>> + return ret;
>> + }
>> default:
>> return -EINVAL;
>> }
>> diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
>> index a89873a5eb77..209087bcf399 100644
>> --- a/arch/arm64/kvm/rmi.c
>> +++ b/arch/arm64/kvm/rmi.c
>> @@ -486,6 +486,75 @@ void kvm_realm_unmap_range(struct kvm *kvm, unsigned long start,
>> realm_unmap_private_range(kvm, start, end, may_block);
>> }
>>
>> +static int realm_data_map_init(struct kvm *kvm, unsigned long ipa,
>> + kvm_pfn_t dst_pfn, kvm_pfn_t src_pfn,
>> + unsigned long flags)
>> +{
>> + struct realm *realm = &kvm->arch.realm;
>> + phys_addr_t rd = virt_to_phys(realm->rd);
>> + phys_addr_t dst_phys, src_phys;
>> + int ret;
>> +
>> + dst_phys = __pfn_to_phys(dst_pfn);
>> + src_phys = __pfn_to_phys(src_pfn);
>> +
>> + if (rmi_delegate_page(dst_phys))
>> + return -ENXIO;
>> +
>> + ret = rmi_rtt_data_map_init(rd, dst_phys, ipa, src_phys, flags);
>> + if (RMI_RETURN_STATUS(ret) == RMI_ERROR_RTT) {
>> + /* Create missing RTTs and retry */
>> + int level = RMI_RETURN_INDEX(ret);
>> +
>> + KVM_BUG_ON(level == KVM_PGTABLE_LAST_LEVEL, kvm);
>> +
>> + ret = realm_create_rtt_levels(realm, ipa, level,
>> + KVM_PGTABLE_LAST_LEVEL, NULL);
>> + if (!ret) {
>> + ret = rmi_rtt_data_map_init(rd, dst_phys, ipa, src_phys,
>> + flags);
>
> I think rmi_rtt_data_map_init() returns a raw RMI return value which is
> positive on failure. Shouldn't it be converted to -ENXIO or another
> Linux errno before returning it?
>
> Otherwise, the positive error value can propagate through
> kvm_gmem_populate() and populate_region() to kvm_arm_rmi_populate().
> There, it would be treated as the number of pages populated, even though
> the operation did not populate any pages.
Good spot.
>> + }
>> + }
>> +
>> + if (ret) {
>> + if (WARN_ON(rmi_undelegate_page(dst_phys))) {
>> + /* Undelegate failed, so we leak the page */
>> + get_page(pfn_to_page(dst_pfn));
>
> Since realm_create_rtt_levels() returns either 0 or a negative errno,
> how about preserving that error and converting any raw RMI erorr to
> -ENXIO here?
>
> return ret < 0 ? ret : -ENXIO;
Makes sense, although this needs to be:
return ret <= 0 ? ret : -ENXIO;
to handle the ret==0 case. I'll fix this in my next posting.
>> + }
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +static int populate_region_cb(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
>> + struct page *src_page, void *opaque)
>> +{
>> + unsigned long data_flags = *(unsigned long *)opaque;
>> + phys_addr_t ipa = gfn_to_gpa(gfn);
>> +
>> + if (!src_page)
>> + return -EOPNOTSUPP;
>> +
>> + return realm_data_map_init(kvm, ipa, pfn, page_to_pfn(src_page),
>> + data_flags);
>> +}
>> +
>> +static long populate_region(struct kvm *kvm,
>> + gfn_t base_gfn,
>> + unsigned long pages,
>> + u64 uaddr,
>> + unsigned long data_flags)
>> +{
>> + long ret = 0;
>> +
>> + mutex_lock(&kvm->slots_lock);
>> + ret = kvm_gmem_populate(kvm, base_gfn, u64_to_user_ptr(uaddr), pages,
>> + populate_region_cb, &data_flags);
>> + mutex_unlock(&kvm->slots_lock);
>> +
>> + return ret;
>> +}
>> +
>> enum ripas_action {
>> RIPAS_INIT,
>> RIPAS_SET,
>> @@ -574,6 +643,43 @@ static int realm_ensure_created(struct kvm *kvm)
>> return -ENXIO;
>> }
>>
>> +int kvm_arm_rmi_populate(struct kvm *kvm,
>> + struct kvm_arm_rmi_populate *args)
>> +{
>> + unsigned long data_flags = 0;
>> + unsigned long ipa_start = args->base;
>> + unsigned long ipa_end = ipa_start + args->size;
>> + long pages_populated;
>> + int ret;
>> +
>> + if (args->reserved ||
>> + (args->flags & ~KVM_ARM_RMI_POPULATE_FLAGS_MEASURE) ||
>> + !IS_ALIGNED(ipa_start, PAGE_SIZE) ||
>> + !IS_ALIGNED(ipa_end, PAGE_SIZE) ||
>> + !IS_ALIGNED(args->source_uaddr, PAGE_SIZE))
>> + return -EINVAL;
>> +
>> + ret = realm_ensure_created(kvm);
>> + if (ret)
>> + return ret;
>
> If I understand correctly, concurrent KVM_ARM_RMI_POPULATE calls can
> race with each other in realm_ensure_created().
>
> realm_ensure_created() calls realm_create_rd() when kvm_realm_state() ==
> REALM_STATE_NONE. Since kvm->arch.config_lock is not held here, multiple
> threads could enter realm_create_rd() concurrently.
>
> KVM_ARM_RMI_POPULATE can also race with KVM_RUN through
> kvm_activate_realm(). kvm_activate_realm() takes kvm->arch.config_lock
> while checking the Realm state, calling realm_ensure_created(), and
> updating the state, whereas KVM_ARM_RMI_POPULATE does not.
>
> How about serializing Realm creation here as follows?
> scoped_guard(mutex, &kvm->arch.config_lock) {
> ret = realm_ensure_created(kvm);
> if (ret)
> return ret;
> }
>
> Does that make sense?
Yes this is a bug I'd already caught and is due to the fixed in my next
posting - similar to what you suggest the config_lock is now held when
realm_ensure_created() is called (and I've put a lockdep_assert_held()
in realm_ensure_created).
Thanks,
Steve
>> +
>> + if (args->flags & KVM_ARM_RMI_POPULATE_FLAGS_MEASURE)
>> + data_flags |= RMI_MEASURE_CONTENT;
>> +
>> + pages_populated = populate_region(kvm, gpa_to_gfn(ipa_start),
>> + args->size >> PAGE_SHIFT,
>> + args->source_uaddr, data_flags);
>> +
>> + if (pages_populated < 0)
>> + return pages_populated;
>> +
>> + args->size -= pages_populated << PAGE_SHIFT;
>> + args->source_uaddr += pages_populated << PAGE_SHIFT;
>> + args->base += pages_populated << PAGE_SHIFT;
>> +
>> + return 0;
>> +}
>> +
>> static void kvm_complete_ripas_change(struct kvm_vcpu *vcpu)
>> {
>> struct kvm *kvm = vcpu->kvm;
>> --
>> 2.43.0
>>
>>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox