[PATCH v17 00/20] KVM: arm64: CCA: Add basic plumbing for Realms

[PATCH v17 00/20] KVM: arm64: CCA: Add basic plumbing for Realms

[Suzuki K Poulose]

This series is a trimmed down version of the Arm CCA KVM support, previously
posted here [0]. Since the last version, we have tried to split the entire
series into the following chunks, while also addressing the review comments on
that version.

 1) Base RMM RMI support under drivers/firmware/arm_rmm -> [1]
 2) Linux Host support for handling GPFs - [2]
 3) NEW: Enlighten KVM arm64 about the different VM types and use call backs for
   the VM type, rather than spilling the is_this_type_of_vm() everywhere. This is
   not complete yet, but it is largely functional. Adds VCPU and Stage2 MMU related
   callbacks with support for the existing VM types. There are other places where
   we may be able to abstract, but those need careful performance evaluations to
   make sure they are fit (e.g., vcpu_run)
   Also adds classification of "Confidential" VMs (which includes Protected VM
   and Realms), which allows us to handle common themes without having to do
   things like :
     if (kvm_vm_is_protected() || kvm_vm_is_realm()),
     instead:
     if (kvm_vm_is_confidential())
   The S2 MMU abstraction layer is kept at the end of this series. The Realm S2
   related implementations cannot be added meaningfully without the RMI commands.
 4) Bare minimal Realm VM support without the actual functionality to run a Realm.
   This would help the maintainers to review the series in smaller chunks. This
   doesn't depend on [1] and can be independently merged, without being "functional".
 5) Core implementation of the RMI driver for KVM and actual enablement of the
    Realm support. This depends on (1), (2) and the guest-memfd-in-place
    conversion series v12 from Ackerley. This is available here at the integration
    branch [3]

This series is comprised of (3) and (4) above.

The integration branch has been tested with the following components:
  tf-RMM:   main branch (commit 5e6e2acd) compliant to RMM-v2.0-beta3 [4]
  kvmtool: git@git.gitlab.arm.com:linux-arm/kvmtool-cca.git cca/kvm-v17

[0] Arm CCA KVM Support v16 : https://lore.kernel.org/all/20260803134403.80630-1-steven.price@arm.com
[1] Linux firmware RMI https://lore.kernel.org/all/20260907095942.1140734-1-suzuki.poulose@arm.com
[2] Linux GPF Host https://lore.kernel.org/all/20260907162204.1479401-1-suzuki.poulose@arm.com
[3] https://git.gitlab.arm.com/linux-arm/linux-cca/ cca/cca-host/kvm-v17/integration
[4] https://support.arm.com/documentation/den0137/2-0bet3/

Jean-Philippe Brucker (2):
  KVM: arm64: CCA: Provide register list for unfinalized RECs
  KVM: arm64: CCA: Provide an accurate register list

Steven Price (4):
  KVM: arm64: Avoid including linux/kvm_host.h in kvm_pgtable.h
  KVM: arm64: CCA: Introduce Realms
  KVM: arm64: CCA: Support timers in realm RECs
  KVM: arm64: CCA: WARN on injected undef exceptions

Suzuki K Poulose (14):
  KVM: arm64: Include kvm_emulate.h in kvm/arm_psci.h
  KVM: arm64: Track the type of VM in kvm_arch
  KVM: arm64: Refactor the vcpu_load to allow for VM specific callbacks
  KVM: arm64: Add vcpu load/put call backs for flavors
  KVM: arm64: CCA: Add a new mode for supporting Realm guests
  KVM: arm64: coco:  Add a helper to check if a VM is confidential
    compute guest
  KVM: arm64: coco: arch_timer: Prevent timer offset configuration
  KVM: arm64: coco: Disable Steal time accounting for coco guests
  KVM: arm64: coco: Don't handle MMIO with no ISV
  KVM: arm64: CCA: Add VCPU load/put for Realms
  KVM: arm64: CCA: Don't expose unsupported capabilities for realm
    guests
  KVM: arm64: Reuse kvm_stage2_unmap_range in kvm_unmap_gfn_range
  KVM: arm64: Add VM specific callback for S2 MMU operations
  KVM: arm64: Abstract out memory abort handling

 .../admin-guide/kernel-parameters.txt         |   3 +
 arch/arm64/include/asm/kvm_emulate.h          |  16 ++
 arch/arm64/include/asm/kvm_host.h             |  58 +++-
 arch/arm64/include/asm/kvm_pgtable.h          |   6 +-
 arch/arm64/include/asm/kvm_pkvm.h             |   2 +-
 arch/arm64/include/asm/kvm_rmi.h              |  61 ++++
 arch/arm64/include/asm/virt.h                 |   1 +
 arch/arm64/kvm/Makefile                       |   2 +-
 arch/arm64/kvm/arch_timer.c                   |  28 +-
 arch/arm64/kvm/arm.c                          | 267 +++++++++++++++---
 arch/arm64/kvm/guest.c                        |  16 +-
 arch/arm64/kvm/hyp/nvhe/pkvm.c                |   2 +-
 arch/arm64/kvm/hyp/pgtable.c                  |   1 +
 arch/arm64/kvm/hypercalls.c                   |   4 +-
 arch/arm64/kvm/inject_fault.c                 |   1 +
 arch/arm64/kvm/mmio.c                         |   4 +-
 arch/arm64/kvm/mmu.c                          | 173 +++++++++---
 arch/arm64/kvm/pkvm.c                         |   1 -
 arch/arm64/kvm/pvtime.c                       |  10 +-
 arch/arm64/kvm/rmi.c                          |  18 ++
 arch/arm64/kvm/sys_regs.c                     |  27 +-
 include/kvm/arm_psci.h                        |   2 +
 22 files changed, 581 insertions(+), 122 deletions(-)
 create mode 100644 arch/arm64/include/asm/kvm_rmi.h
 create mode 100644 arch/arm64/kvm/rmi.c

-- 
2.43.0

[PATCH v17 01/20] KVM: arm64: Include kvm_emulate.h in kvm/arm_psci.h

[Suzuki K Poulose]

Fix a potential build error (like below, when asm/kvm_emulate.h gets
included after the kvm/arm_psci.h) by including the missing header file
in kvm/arm_psci.h:

./include/kvm/arm_psci.h: In function ‘kvm_psci_version’:
./include/kvm/arm_psci.h:29:13: error: implicit declaration of function
   ‘vcpu_has_feature’; did you mean ‘cpu_have_feature’? [-Werror=implicit-function-declaration]
   29 |         if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_PSCI_0_2)) {
	         |             ^~~~~~~~~~~~~~~~
			       |             cpu_have_feature

Reviewed-by: Gavin Shan <gshan@redhat.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Steven Price <steven.price@arm.com>
---
 include/kvm/arm_psci.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h
index f86a006d67136..06c20612e9e7d 100644
--- a/include/kvm/arm_psci.h
+++ b/include/kvm/arm_psci.h
@@ -10,6 +10,8 @@
 #include <linux/kvm_host.h>
 #include <uapi/linux/psci.h>
 
+#include <asm/kvm_emulate.h>
+
 #define KVM_ARM_PSCI_0_1	PSCI_VERSION(0, 1)
 #define KVM_ARM_PSCI_0_2	PSCI_VERSION(0, 2)
 #define KVM_ARM_PSCI_1_0	PSCI_VERSION(1, 0)
-- 
2.43.0

[PATCH v17 02/20] KVM: arm64: Avoid including linux/kvm_host.h in kvm_pgtable.h

[Suzuki K Poulose]

From: Steven Price <steven.price@arm.com>

To avoid future include cycles, drop the linux/kvm_host.h include in
kvm_pgtable.h and include the lightweight headers required for the types
and inline helpers used there. Additionally provide a forward
declaration for struct kvm_s2_mmu as it's only used as a pointer in this
file.

Both pgtable.c and kvm_pkvm.h relied on the indirect inclusion of
kvm_host.h, so make that explicit.

Signed-off-by: Steven Price <steven.price@arm.com>
---
New patch in v13
---
 arch/arm64/include/asm/kvm_pgtable.h | 6 +++++-
 arch/arm64/include/asm/kvm_pkvm.h    | 2 +-
 arch/arm64/kvm/hyp/pgtable.c         | 1 +
 3 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h
index 41a8687938eb6..c2e4b29e605fc 100644
--- a/arch/arm64/include/asm/kvm_pgtable.h
+++ b/arch/arm64/include/asm/kvm_pgtable.h
@@ -8,9 +8,13 @@
 #define __ARM64_KVM_PGTABLE_H__
 
 #include <linux/bits.h>
-#include <linux/kvm_host.h>
+#include <linux/kvm_types.h>
+#include <linux/rbtree_types.h>
+#include <linux/rcupdate.h>
 #include <linux/types.h>
 
+struct kvm_s2_mmu;
+
 #define KVM_PGTABLE_FIRST_LEVEL		-1
 #define KVM_PGTABLE_LAST_LEVEL		3
 
diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
index beea00e693a0a..54a618d887fa4 100644
--- a/arch/arm64/include/asm/kvm_pkvm.h
+++ b/arch/arm64/include/asm/kvm_pkvm.h
@@ -7,9 +7,9 @@
 #define __ARM64_KVM_PKVM_H__
 
 #include <linux/arm_ffa.h>
+#include <linux/kvm_host.h>
 #include <linux/memblock.h>
 #include <linux/scatterlist.h>
-#include <asm/kvm_host.h>
 #include <asm/kvm_pgtable.h>
 
 /* Maximum number of VMs that can co-exist under pKVM. */
diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
index b74dd5ce1efd3..f48253b9d88b5 100644
--- a/arch/arm64/kvm/hyp/pgtable.c
+++ b/arch/arm64/kvm/hyp/pgtable.c
@@ -8,6 +8,7 @@
  */
 
 #include <linux/bitfield.h>
+#include <linux/kvm_host.h>
 #include <asm/kvm_pgtable.h>
 #include <asm/stage2_pgtable.h>
 
-- 
2.43.0

[PATCH v17 03/20] KVM: arm64: Track the type of VM in kvm_arch

[Suzuki K Poulose]

KVM arm64 has different types of VMs with all the different modes in which the
hypervisor code can be run. e.g., VHE, nVHE, PKVM etc. Then there is protected
VM and normal VMs with PKVM. We might soon add other types, e.g., Arm CCA Realm.
So in an effort to make the handling of these different types of VMs a bit more
friendlier to the eyes, add a VM flavor to the kvm_arch and we could then add
handlers for different operations based on the VM type.

Keep the flavor initialisation at the beginning to allow for the detection
early enough and fail out on any unsupported requests. (e.g., protected on !PKVM)

With that, use the vm_flavor to detect if a VM is protected VM on PKVM.

Based on a patch by Marc Zyngier

Suggested-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_host.h | 12 ++++++++++--
 arch/arm64/kvm/arm.c              | 27 ++++++++++++++++++++++++---
 arch/arm64/kvm/hyp/nvhe/pkvm.c    |  2 +-
 arch/arm64/kvm/pkvm.c             |  1 -
 4 files changed, 35 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 27fe0cd5b2d7a..d0dccc9ad6aa8 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -257,7 +257,6 @@ struct kvm_protected_vm {
 	pkvm_handle_t handle;
 	struct kvm_hyp_memcache teardown_mc;
 	struct kvm_hyp_memcache stage2_teardown_mc;
-	bool is_protected;
 	bool is_created;
 
 	/*
@@ -306,9 +305,18 @@ enum fgt_group_id {
 	__NR_FGT_GROUP_IDS__
 };
 
+enum kvm_arm_vm_flavor {
+	VM_NVHE,
+	VM_VHE,
+	VM_PKVM,		/* Normal guests on PKVM */
+	VM_PROTECTED_PKVM,	/* Protected VM */
+	VM_FLAVOR_MAX,
+};
+
 struct kvm_arch {
 	struct kvm_s2_mmu mmu;
 
+	enum kvm_arm_vm_flavor vm_flavor;
 	/*
 	 * Fine-Grained UNDEF, mimicking the FGT layout defined by the
 	 * architecture. We track them globally, as we present the
@@ -1504,7 +1512,7 @@ struct kvm *kvm_arch_alloc_vm(void);
 
 #define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS_RANGE
 
-#define kvm_vm_is_protected(kvm)	(is_protected_kvm_enabled() && (kvm)->arch.pkvm.is_protected)
+#define kvm_vm_is_protected(kvm)	((kvm)->arch.vm_flavor == VM_PROTECTED_PKVM)
 
 #define vcpu_is_protected(vcpu)		kvm_vm_is_protected((vcpu)->kvm)
 
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 8b080804bc90b..67f1ff9bc4fbe 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -214,6 +214,26 @@ static int kvm_arm_default_max_vcpus(void)
 	return vgic_present ? kvm_vgic_get_max_vcpus() : KVM_MAX_VCPUS;
 }
 
+static int kvm_init_vm_flavor(struct kvm *kvm, unsigned long type)
+{
+	bool protected = type & KVM_VM_TYPE_ARM_PROTECTED;
+
+	if (is_protected_kvm_enabled()) {
+		if (protected)
+			kvm->arch.vm_flavor = VM_PROTECTED_PKVM;
+		else
+			kvm->arch.vm_flavor = VM_PKVM;
+	} else if (protected) {
+		return -EINVAL;
+	} else if (has_vhe()) {
+		kvm->arch.vm_flavor = VM_VHE;
+	} else {
+		kvm->arch.vm_flavor = VM_NVHE;
+	}
+
+	return 0;
+}
+
 /**
  * kvm_arch_init_vm - initializes a VM data structure
  * @kvm:	pointer to the KVM struct
@@ -236,6 +256,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	mutex_unlock(&kvm->lock);
 #endif
 
+	ret = kvm_init_vm_flavor(kvm, type);
+	if (ret)
+		return ret;
+
 	kvm_init_nested(kvm);
 
 	ret = kvm_share_hyp(kvm, kvm + 1);
@@ -260,9 +284,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 		ret = pkvm_init_host_vm(kvm, type);
 		if (ret)
 			goto err_uninit_mmu;
-	} else if (type & KVM_VM_TYPE_ARM_PROTECTED) {
-		ret = -EINVAL;
-		goto err_uninit_mmu;
 	}
 
 	kvm_vgic_early_init(kvm);
diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
index 459bd9eb7e4bc..09961c0056f2b 100644
--- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
+++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
@@ -432,7 +432,7 @@ static void init_pkvm_hyp_vm(struct kvm *host_kvm, struct pkvm_hyp_vm *hyp_vm,
 
 	hyp_vm->host_kvm = host_kvm;
 	hyp_vm->kvm.created_vcpus = nr_vcpus;
-	hyp_vm->kvm.arch.pkvm.is_protected = READ_ONCE(host_kvm->arch.pkvm.is_protected);
+	hyp_vm->kvm.arch.vm_flavor = READ_ONCE(host_kvm->arch.vm_flavor);
 	hyp_vm->kvm.arch.flags = 0;
 	pkvm_init_features_from_host(hyp_vm, host_kvm);
 
diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
index 8e4c6e4bec123..3948fa46f4d75 100644
--- a/arch/arm64/kvm/pkvm.c
+++ b/arch/arm64/kvm/pkvm.c
@@ -240,7 +240,6 @@ int pkvm_init_host_vm(struct kvm *kvm, unsigned long type)
 		return ret;
 
 	kvm->arch.pkvm.handle = ret;
-	kvm->arch.pkvm.is_protected = protected;
 	if (protected) {
 		pr_warn_once("kvm: protected VMs are experimental and for development only, tainting kernel\n");
 		add_taint(TAINT_USER, LOCKDEP_STILL_OK);
-- 
2.43.0

[PATCH v17 04/20] KVM: arm64: Refactor the vcpu_load to allow for VM specific callbacks

[Suzuki K Poulose]

Based on a work by Marc Zyngier

To keep the VCPU load/put handling cleaner with the different kinds of VM types,
we are about to introduce VM specific callbacks to do just the right thing.
In preparation for that, make some refactoring to add the change easier.

No functional changes intended.

Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/kvm/arm.c | 46 +++++++++++++++++++++++++++-----------------
 1 file changed, 28 insertions(+), 18 deletions(-)

diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 67f1ff9bc4fbe..51fc651267157 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -683,14 +683,11 @@ static bool kvm_vcpu_should_clear_twe(struct kvm_vcpu *vcpu)
 	return single_task_running();
 }
 
-void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
+static void vcpu_prepare_mmu(struct kvm_vcpu *vcpu)
 {
 	struct kvm_s2_mmu *mmu;
 	int *last_ran;
 
-	if (is_protected_kvm_enabled())
-		goto nommu;
-
 	if (vcpu_has_nv(vcpu))
 		kvm_vcpu_load_hw_mmu(vcpu);
 
@@ -720,10 +717,33 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 		kvm_call_hyp(__kvm_flush_cpu_context, mmu);
 		*last_ran = vcpu->vcpu_idx;
 	}
+}
+
+static void vcpu_set_wfx_traps(struct kvm_vcpu *vcpu)
+{
+	if (kvm_vcpu_should_clear_twe(vcpu))
+		vcpu->arch.hcr_el2 &= ~HCR_TWE;
+	else
+		vcpu->arch.hcr_el2 |= HCR_TWE;
+
+	if (kvm_vcpu_should_clear_twi(vcpu))
+		vcpu->arch.hcr_el2 &= ~HCR_TWI;
+	else
+		vcpu->arch.hcr_el2 |= HCR_TWI;
+}
+
+static void vcpu_load_pvtime(struct kvm_vcpu *vcpu)
+{
+	if (kvm_arm_is_pvtime_enabled(&vcpu->arch))
+		kvm_make_request(KVM_REQ_RECORD_STEAL, vcpu);
+}
+
+void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
+{
+	if (!is_protected_kvm_enabled())
+		vcpu_prepare_mmu(vcpu);
 
-nommu:
 	vcpu->cpu = cpu;
-
 	/*
 	 * The timer must be loaded before the vgic to correctly set up physical
 	 * interrupt deactivation in nested state (e.g. timer interrupt).
@@ -736,19 +756,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 		kvm_vcpu_load_vhe(vcpu);
 	kvm_arch_vcpu_load_fp(vcpu);
 	kvm_vcpu_pmu_restore_guest(vcpu);
-	if (kvm_arm_is_pvtime_enabled(&vcpu->arch))
-		kvm_make_request(KVM_REQ_RECORD_STEAL, vcpu);
-
-	if (kvm_vcpu_should_clear_twe(vcpu))
-		vcpu->arch.hcr_el2 &= ~HCR_TWE;
-	else
-		vcpu->arch.hcr_el2 |= HCR_TWE;
-
-	if (kvm_vcpu_should_clear_twi(vcpu))
-		vcpu->arch.hcr_el2 &= ~HCR_TWI;
-	else
-		vcpu->arch.hcr_el2 |= HCR_TWI;
 
+	vcpu_load_pvtime(vcpu);
+	vcpu_set_wfx_traps(vcpu);
 	vcpu_set_pauth_traps(vcpu);
 
 	if (is_protected_kvm_enabled()) {
-- 
2.43.0

[PATCH v17 05/20] KVM: arm64: Add vcpu load/put call backs for flavors

[Suzuki K Poulose]

Add VM flavor specific handlers for VCPU load/put, in an effort to make it
easier to follow the code.

Based on a patch by Marc Zyngier

Suggested-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_host.h |   6 ++
 arch/arm64/kvm/arm.c              | 156 ++++++++++++++++++++++--------
 2 files changed, 123 insertions(+), 39 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index d0dccc9ad6aa8..b2e99c5cb1cd3 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -150,6 +150,11 @@ struct kvm_vmid {
 	atomic64_t id;
 };
 
+struct kvm_vcpu_ops {
+	void (*vcpu_load)(struct kvm_vcpu *vcpu, int cpu);
+	void (*vcpu_put)(struct kvm_vcpu *vcpu);
+};
+
 struct kvm_s2_mmu {
 	struct kvm_vmid vmid;
 
@@ -854,6 +859,7 @@ struct vncr_tlb;
 
 struct kvm_vcpu_arch {
 	struct kvm_cpu_context ctxt;
+	const struct kvm_vcpu_ops *vcpu_ops;
 
 	/*
 	 * Guest floating point state
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 51fc651267157..9af3bbb2f8c24 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -74,6 +74,8 @@ struct kvm_ioctl_cap_map {
 	long ext;
 };
 
+static const struct kvm_vcpu_ops *arm64_vcpu_ops[VM_FLAVOR_MAX];
+
 /* Make KVM_CAP_NR_VCPUS the reference for features we always supported */
 #define KVM_CAP_ARM_BASIC	KVM_CAP_NR_VCPUS
 
@@ -569,6 +571,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
 	mutex_unlock(&vcpu->mutex);
 #endif
 
+	vcpu->arch.vcpu_ops = arm64_vcpu_ops[vcpu->kvm->arch.vm_flavor];
+
 	/* Force users to call KVM_ARM_VCPU_INIT */
 	vcpu_clear_flag(vcpu, VCPU_INITIALIZED);
 
@@ -738,36 +742,72 @@ static void vcpu_load_pvtime(struct kvm_vcpu *vcpu)
 		kvm_make_request(KVM_REQ_RECORD_STEAL, vcpu);
 }
 
+static void vhe_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
+{
+	vcpu_prepare_mmu(vcpu);
+	/*
+	 * The timer must be loaded before the vgic to correctly set up physical
+	 * interrupt deactivation in nested state (e.g. timer interrupt).
+	 */
+	kvm_timer_vcpu_load(vcpu);
+	kvm_vgic_load(vcpu);
+	kvm_vcpu_load_debug(vcpu);
+	kvm_vcpu_load_fgt(vcpu);
+	kvm_vcpu_load_vhe(vcpu);
+	kvm_arch_vcpu_load_fp(vcpu);
+	kvm_vcpu_pmu_restore_guest(vcpu);
+
+	vcpu_load_pvtime(vcpu);
+	vcpu_set_wfx_traps(vcpu);
+	vcpu_set_pauth_traps(vcpu);
+}
+
+static void nvhe_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
+{
+	vcpu_prepare_mmu(vcpu);
+	/*
+	 * The timer must be loaded before the vgic to correctly set up physical
+	 * interrupt deactivation in nested state (e.g. timer interrupt).
+	 */
+	kvm_timer_vcpu_load(vcpu);
+	kvm_vgic_load(vcpu);
+	kvm_vcpu_load_debug(vcpu);
+	kvm_vcpu_load_fgt(vcpu);
+	kvm_arch_vcpu_load_fp(vcpu);
+	kvm_vcpu_pmu_restore_guest(vcpu);
+
+	vcpu_load_pvtime(vcpu);
+	vcpu_set_wfx_traps(vcpu);
+	vcpu_set_pauth_traps(vcpu);
+}
+
+static void pkvm_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
+{
+	/*
+	 * The timer must be loaded before the vgic to correctly set up physical
+	 * interrupt deactivation in nested state (e.g. timer interrupt).
+	 */
+	kvm_timer_vcpu_load(vcpu);
+	kvm_vgic_load(vcpu);
+	kvm_vcpu_load_debug(vcpu);
+	kvm_vcpu_load_fgt(vcpu);
+	kvm_arch_vcpu_load_fp(vcpu);
+	kvm_vcpu_pmu_restore_guest(vcpu);
+
+	vcpu_load_pvtime(vcpu);
+	vcpu_set_wfx_traps(vcpu);
+
+	kvm_call_hyp_nvhe(__pkvm_vcpu_load,
+			  vcpu->kvm->arch.pkvm.handle,
+			  vcpu->vcpu_idx, vcpu->arch.hcr_el2);
+	kvm_call_hyp(__vgic_v3_restore_vmcr_aprs,
+		     &vcpu->arch.vgic_cpu.vgic_v3);
+}
+
 void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 {
-	if (!is_protected_kvm_enabled())
-		vcpu_prepare_mmu(vcpu);
-
 	vcpu->cpu = cpu;
-	/*
-	 * The timer must be loaded before the vgic to correctly set up physical
-	 * interrupt deactivation in nested state (e.g. timer interrupt).
-	 */
-	kvm_timer_vcpu_load(vcpu);
-	kvm_vgic_load(vcpu);
-	kvm_vcpu_load_debug(vcpu);
-	kvm_vcpu_load_fgt(vcpu);
-	if (has_vhe())
-		kvm_vcpu_load_vhe(vcpu);
-	kvm_arch_vcpu_load_fp(vcpu);
-	kvm_vcpu_pmu_restore_guest(vcpu);
-
-	vcpu_load_pvtime(vcpu);
-	vcpu_set_wfx_traps(vcpu);
-	vcpu_set_pauth_traps(vcpu);
-
-	if (is_protected_kvm_enabled()) {
-		kvm_call_hyp_nvhe(__pkvm_vcpu_load,
-				  vcpu->kvm->arch.pkvm.handle,
-				  vcpu->vcpu_idx, vcpu->arch.hcr_el2);
-		kvm_call_hyp(__vgic_v3_restore_vmcr_aprs,
-			     &vcpu->arch.vgic_cpu.vgic_v3);
-	}
+	vcpu->arch.vcpu_ops->vcpu_load(vcpu, cpu);
 
 	if (!cpumask_test_cpu(cpu, vcpu->kvm->arch.supported_cpus))
 		vcpu_set_on_unsupported_cpu(vcpu);
@@ -775,28 +815,44 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 	vcpu->arch.pid = pid_nr(vcpu->pid);
 }
 
-void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
+static void vhe_vcpu_put(struct kvm_vcpu *vcpu)
 {
-	if (is_protected_kvm_enabled()) {
-		kvm_call_hyp(__vgic_v3_save_aprs, &vcpu->arch.vgic_cpu.vgic_v3);
-		kvm_call_hyp_nvhe(__pkvm_vcpu_put);
-
-		/* __pkvm_vcpu_put implies a sync of the state */
-		if (!kvm_vm_is_protected(vcpu->kvm))
-			vcpu_set_flag(vcpu, PKVM_HOST_STATE_DIRTY);
-	}
-
 	kvm_vcpu_put_debug(vcpu);
 	kvm_arch_vcpu_put_fp(vcpu);
-	if (has_vhe())
-		kvm_vcpu_put_vhe(vcpu);
+	kvm_vcpu_put_vhe(vcpu);
 	kvm_timer_vcpu_put(vcpu);
 	kvm_vgic_put(vcpu);
 	kvm_vcpu_pmu_restore_host(vcpu);
 	if (vcpu_has_nv(vcpu))
 		kvm_vcpu_put_hw_mmu(vcpu);
 	kvm_arm_vmid_clear_active();
+}
 
+static void nvhe_vcpu_put(struct kvm_vcpu *vcpu)
+{
+	kvm_vcpu_put_debug(vcpu);
+	kvm_arch_vcpu_put_fp(vcpu);
+	kvm_timer_vcpu_put(vcpu);
+	kvm_vgic_put(vcpu);
+	kvm_vcpu_pmu_restore_host(vcpu);
+	kvm_arm_vmid_clear_active();
+}
+
+static void pkvm_vcpu_put(struct kvm_vcpu *vcpu)
+{
+	kvm_call_hyp(__vgic_v3_save_aprs, &vcpu->arch.vgic_cpu.vgic_v3);
+	kvm_call_hyp_nvhe(__pkvm_vcpu_put);
+
+	/* __pkvm_vcpu_put implies a sync of the state */
+	if (!kvm_vm_is_protected(vcpu->kvm))
+		vcpu_set_flag(vcpu, PKVM_HOST_STATE_DIRTY);
+
+	nvhe_vcpu_put(vcpu);
+}
+
+void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
+{
+	vcpu->arch.vcpu_ops->vcpu_put(vcpu);
 	vcpu_clear_on_unsupported_cpu(vcpu);
 	vcpu->cpu = -1;
 }
@@ -2136,6 +2192,28 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
 	}
 }
 
+static const struct kvm_vcpu_ops vhe_vcpu_ops = {
+	.vcpu_load = vhe_vcpu_load,
+	.vcpu_put = vhe_vcpu_put,
+};
+
+static const struct kvm_vcpu_ops nvhe_vcpu_ops = {
+	.vcpu_load = nvhe_vcpu_load,
+	.vcpu_put = nvhe_vcpu_put,
+};
+
+static const struct kvm_vcpu_ops pkvm_vcpu_ops = {
+	.vcpu_load = pkvm_vcpu_load,
+	.vcpu_put = pkvm_vcpu_put,
+};
+
+static const struct kvm_vcpu_ops *arm64_vcpu_ops[] = {
+	[VM_VHE] = &vhe_vcpu_ops,
+	[VM_NVHE] = &nvhe_vcpu_ops,
+	[VM_PKVM] = &pkvm_vcpu_ops,
+	[VM_PROTECTED_PKVM] = &pkvm_vcpu_ops,
+};
+
 static unsigned long nvhe_percpu_size(void)
 {
 	return (unsigned long)CHOOSE_NVHE_SYM(__per_cpu_end) -
-- 
2.43.0

[PATCH v17 06/20] KVM: arm64: CCA: Add a new mode for supporting Realm guests

[Suzuki K Poulose]

Add an explicit mode to support Arm CCA guests.

Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 Documentation/admin-guide/kernel-parameters.txt | 3 +++
 arch/arm64/include/asm/kvm_host.h               | 1 +
 arch/arm64/kvm/arm.c                            | 5 +++++
 3 files changed, 9 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 68647ff4bdd24..1afe3df3b923e 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -3256,6 +3256,9 @@ Kernel parameters
 			nested: VHE-based mode with support for nested
 				virtualization. Requires at least ARMv8.4
 				hardware (with FEAT_NV2).
+			rmm: Support for running confidential guests in Realm
+			     world using RMM, as defined by Arm Confidential
+			     Compute Architecture (CCA)
 
 			Defaults to VHE/nVHE based on hardware support. Setting
 			mode to "protected" will disable kexec and hibernation
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index b2e99c5cb1cd3..31ae9d8d8e92f 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -69,6 +69,7 @@ enum kvm_mode {
 	KVM_MODE_DEFAULT,
 	KVM_MODE_PROTECTED,
 	KVM_MODE_NV,
+	KVM_MODE_RMM,
 	KVM_MODE_NONE,
 };
 #ifdef CONFIG_KVM
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 9af3bbb2f8c24..50f0adfadab38 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -3264,6 +3264,11 @@ static int __init early_kvm_mode_cfg(char *arg)
 		return 0;
 	}
 
+	if (strcmp(arg, "rmm") == 0 && !WARN_ON(!is_kernel_in_hyp_mode())) {
+		kvm_mode = KVM_MODE_RMM;
+		return 0;
+	}
+
 	return -EINVAL;
 }
 early_param("kvm-arm.mode", early_kvm_mode_cfg);
-- 
2.43.0

[PATCH v17 07/20] KVM: arm64: CCA: Introduce Realms

[Suzuki K Poulose]

From: Steven Price <steven.price@arm.com>

Add foundational work for supporting Realms.
 - Add a new VM flavor.
 - At KVM init, check if the KVM can support Realms (though not functional yet)
   and will be advertised by static key kvm_rmi_is_available. This will be
   turned on in a later patches, once we have all the bits and pieces ready.
   For now check if we are blessed with KVM_MODE_RMM.
 - Add realm specific tracking in kvm_arch. Since Realm and protected pKVM
   states are mutually exclusive, move them into a union.

Signed-off-by: Steven Price <steven.price@arm.com>
Co-Developed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
Changes since v16:
 * Share mutually exclusive pKVM and Realm per-VM storage in a union.
 * Move to the new VM flavor infrastructure, split bits out. Trimmed down
 * Move in Realm state and basic boiler plates
Changes since v13:
 * Most of the init has been moved out of the 'kvm' directory so this is
   much more basic now.
Changes since v12:
 * Drop check for 4k page size.
Changes since v11:
 * Reword slightly the comments on the realm states.
Changes since v10:
 * kvm_is_realm() no longer has a NULL check.
 * Rename from "rme" to "rmi" when referring to the RMM interface.
 * Check for RME (hardware) support before probing for RMI support.
Changes since v8:
 * No need to guard kvm_init_rme() behind 'in_hyp_mode'.
Changes since v6:
 * Improved message for an unsupported RMI ABI version.
Changes since v5:
 * Reword "unsupported" message from "host supports" to "we want" to
   clarify that 'we' are the 'host'.
Changes since v2:
 * Drop return value from kvm_init_rme(), it was always 0.
 * Rely on the RMM return value to identify whether the RSI ABI is
   compatible.
---
 arch/arm64/include/asm/kvm_emulate.h | 16 ++++++++
 arch/arm64/include/asm/kvm_host.h    | 17 +++++---
 arch/arm64/include/asm/kvm_rmi.h     | 61 ++++++++++++++++++++++++++++
 arch/arm64/include/asm/virt.h        |  1 +
 arch/arm64/kvm/Makefile              |  2 +-
 arch/arm64/kvm/arm.c                 |  5 +++
 arch/arm64/kvm/rmi.c                 | 18 ++++++++
 7 files changed, 114 insertions(+), 6 deletions(-)
 create mode 100644 arch/arm64/include/asm/kvm_rmi.h
 create mode 100644 arch/arm64/kvm/rmi.c

diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index a3c1928bdf743..d360a8b05b8bf 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -793,4 +793,20 @@ static inline void kvm_reset_vcpu_psci(struct kvm_vcpu *vcpu,
 	vcpu_set_reg(vcpu, 0, reset_state->r0);
 }
 
+static inline enum realm_state kvm_realm_state(struct kvm *kvm)
+{
+	return READ_ONCE(kvm->arch.realm.state);
+}
+
+static inline void kvm_set_realm_state(struct kvm *kvm,
+				       enum realm_state new_state)
+{
+	WRITE_ONCE(kvm->arch.realm.state, new_state);
+}
+
+static inline bool kvm_realm_is_created(struct kvm *kvm)
+{
+	return kvm_vm_is_realm(kvm) && kvm_realm_state(kvm) != REALM_STATE_NONE;
+}
+
 #endif /* __ARM64_KVM_EMULATE_H__ */
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 31ae9d8d8e92f..824a3383409c8 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -27,6 +27,7 @@
 #include <asm/fpsimd.h>
 #include <asm/kvm.h>
 #include <asm/kvm_asm.h>
+#include <asm/kvm_rmi.h>
 #include <asm/vncr_mapping.h>
 
 #define __KVM_HAVE_ARCH_INTC_INITIALIZED
@@ -316,6 +317,7 @@ enum kvm_arm_vm_flavor {
 	VM_VHE,
 	VM_PKVM,		/* Normal guests on PKVM */
 	VM_PROTECTED_PKVM,	/* Protected VM */
+	VM_REALM,		/* CCA */
 	VM_FLAVOR_MAX,
 };
 
@@ -431,11 +433,14 @@ struct kvm_arch {
 	/* Count the number of VNCR_EL2 TLBs */
 	atomic_t vncr_tlb_count;
 
-	/*
-	 * For an untrusted host VM, 'pkvm.handle' is used to lookup
-	 * the associated pKVM instance in the hypervisor.
-	 */
-	struct kvm_protected_vm pkvm;
+	union {
+		/*
+		 * For an untrusted host VM, 'pkvm.handle' is used to lookup
+		 * the associated pKVM instance in the hypervisor.
+		 */
+		struct kvm_protected_vm pkvm;
+		struct realm realm;
+	};
 
 #ifdef CONFIG_PTDUMP_STAGE2_DEBUGFS
 	/* Nested virtualization info */
@@ -1520,8 +1525,10 @@ struct kvm *kvm_arch_alloc_vm(void);
 #define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS_RANGE
 
 #define kvm_vm_is_protected(kvm)	((kvm)->arch.vm_flavor == VM_PROTECTED_PKVM)
+#define kvm_vm_is_realm(kvm)		((kvm)->arch.vm_flavor == VM_REALM)
 
 #define vcpu_is_protected(vcpu)		kvm_vm_is_protected((vcpu)->kvm)
+#define vcpu_is_rec(vcpu)		kvm_vm_is_realm((vcpu)->kvm)
 
 int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature);
 bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h
new file mode 100644
index 0000000000000..44f5c75a27b5b
--- /dev/null
+++ b/arch/arm64/include/asm/kvm_rmi.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2023-2026 ARM Ltd.
+ */
+
+#ifndef __ASM_KVM_RMI_H
+#define __ASM_KVM_RMI_H
+
+/**
+ * enum realm_state - State of a Realm
+ *
+ * Mirrors the RMM's Realm lifecycle states where they are meaningful to KVM,
+ * with REALM_STATE_DYING being a KVM-internal state used to prevent further
+ * requests while teardown is in progress. KVM does not track REALM_SYSTEM_OFF
+ * or REALM_ZOMBIE separately as they naturally lead to teardown.
+ */
+enum realm_state {
+	/**
+	 * @REALM_STATE_NONE:
+	 *      Realm has not yet been created. rmi_realm_create() has not
+	 *      yet been called.
+	 */
+	REALM_STATE_NONE,
+	/**
+	 * @REALM_STATE_NEW:
+	 *      Realm is under construction, rmi_realm_create() has been
+	 *      called, but it is not yet activated. Pages may be populated.
+	 */
+	REALM_STATE_NEW,
+	/**
+	 * @REALM_STATE_ACTIVE:
+	 *      Realm has been created and is eligible for execution with
+	 *      rmi_rec_enter(). Pages may no longer be populated with
+	 *      rmi_data_create().
+	 */
+	REALM_STATE_ACTIVE,
+	/**
+	 * @REALM_STATE_DYING:
+	 *      Realm is in the process of being destroyed or has already been
+	 *      destroyed.
+	 */
+	REALM_STATE_DYING,
+	/**
+	 * @REALM_STATE_DEAD:
+	 *      Realm has been destroyed.
+	 */
+	REALM_STATE_DEAD
+};
+
+/**
+ * struct realm - Additional per VM data for a Realm
+ *
+ * @state: The lifetime state machine for the realm
+ */
+struct realm {
+	enum realm_state state;
+};
+
+void kvm_init_rmi(void);
+
+#endif /* __ASM_KVM_RMI_H */
diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
index b546703c3ab9a..92cec42952f42 100644
--- a/arch/arm64/include/asm/virt.h
+++ b/arch/arm64/include/asm/virt.h
@@ -87,6 +87,7 @@ void __hyp_reset_vectors(void);
 bool is_kvm_arm_initialised(void);
 
 DECLARE_STATIC_KEY_FALSE(kvm_protected_mode_initialized);
+DECLARE_STATIC_KEY_FALSE(kvm_rmi_is_available);
 
 static inline bool is_pkvm_initialized(void)
 {
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 59612d2f277c1..ed3cf30eb06e7 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -16,7 +16,7 @@ CFLAGS_handle_exit.o += -Wno-override-init
 kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \
 	 inject_fault.o va_layout.o handle_exit.o config.o \
 	 guest.o debug.o reset.o sys_regs.o stacktrace.o \
-	 vgic-sys-reg-v3.o fpsimd.o pkvm.o \
+	 vgic-sys-reg-v3.o fpsimd.o pkvm.o rmi.o \
 	 arch_timer.o trng.o vmid.o emulate-nested.o nested.o at.o \
 	 vgic/vgic.o vgic/vgic-init.o \
 	 vgic/vgic-irqfd.o vgic/vgic-v2.o \
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 50f0adfadab38..2ecd92156f7cc 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -42,6 +42,7 @@
 #include <asm/kvm_nested.h>
 #include <asm/kvm_pkvm.h>
 #include <asm/kvm_ptrauth.h>
+#include <asm/kvm_rmi.h>
 #include <asm/sections.h>
 #include <asm/stacktrace/nvhe.h>
 
@@ -113,6 +114,8 @@ long kvm_get_cap_for_kvm_ioctl(unsigned int ioctl, long *ext)
 	return -EINVAL;
 }
 
+DEFINE_STATIC_KEY_FALSE(kvm_rmi_is_available);
+
 DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector);
 
 DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_base);
@@ -3159,6 +3162,8 @@ static __init int kvm_arm_init(void)
 
 	in_hyp_mode = is_kernel_in_hyp_mode();
 
+	kvm_init_rmi();
+
 	if (cpus_have_final_cap(ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE) ||
 	    cpus_have_final_cap(ARM64_WORKAROUND_1508412))
 		kvm_info("Guests without required CPU erratum workarounds can deadlock system!\n" \
diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c
new file mode 100644
index 0000000000000..5ecc8b3498698
--- /dev/null
+++ b/arch/arm64/kvm/rmi.c
@@ -0,0 +1,18 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2023-2026 ARM Ltd.
+ */
+
+#include <linux/kvm_host.h>
+
+#include <asm/virt.h>
+
+void kvm_init_rmi(void)
+{
+	if (kvm_get_mode() != KVM_MODE_RMM)
+		return;
+
+	/* TODO: Check if the RMI is available */
+
+	/* Future patch will enable static branch kvm_rmi_is_available */
+}
-- 
2.43.0

[PATCH v17 08/20] KVM: arm64: coco: Add a helper to check if a VM is confidential compute guest

[Suzuki K Poulose]

Both protected VMs and Realms are additionally secured by the trusted hypervisor
guarantees. Add a helper to detect if a given VM is a coco guest. Base this off
the ordering of the VM flavor type.

Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_host.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 824a3383409c8..020aef5a1b698 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -316,6 +316,7 @@ enum kvm_arm_vm_flavor {
 	VM_NVHE,
 	VM_VHE,
 	VM_PKVM,		/* Normal guests on PKVM */
+	MARKER(__VM_CONFIDENTIAL),
 	VM_PROTECTED_PKVM,	/* Protected VM */
 	VM_REALM,		/* CCA */
 	VM_FLAVOR_MAX,
@@ -1524,9 +1525,11 @@ struct kvm *kvm_arch_alloc_vm(void);
 
 #define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS_RANGE
 
+#define kvm_vm_is_confidential(kvm)	((kvm)->arch.vm_flavor >= __VM_CONFIDENTIAL)
 #define kvm_vm_is_protected(kvm)	((kvm)->arch.vm_flavor == VM_PROTECTED_PKVM)
 #define kvm_vm_is_realm(kvm)		((kvm)->arch.vm_flavor == VM_REALM)
 
+#define vcpu_is_confidential(vcpu)	kvm_vm_is_confidential((vcpu)->kvm)
 #define vcpu_is_protected(vcpu)		kvm_vm_is_protected((vcpu)->kvm)
 #define vcpu_is_rec(vcpu)		kvm_vm_is_realm((vcpu)->kvm)
 
-- 
2.43.0

[PATCH v17 09/20] KVM: arm64: coco: arch_timer: Prevent timer offset configuration

[Suzuki K Poulose]

Confidential guests, including protected PKVM guests and Arm CCA Realms, always
runs with 0 timer offset. Prevent the configuration for them.

Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/kvm/arch_timer.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 6ac3321f4c575..eba84cab95d01 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -1079,7 +1079,7 @@ static void timer_context_init(struct kvm_vcpu *vcpu, int timerid)
 
 	ctxt->timer_id = timerid;
 
-	if (!kvm_vm_is_protected(vcpu->kvm)) {
+	if (!vcpu_is_confidential(vcpu)) {
 		if (timerid == TIMER_VTIMER)
 			ctxt->offset.vm_offset = &kvm->arch.timer_data.voffset;
 		else
@@ -1110,7 +1110,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
 		timer_context_init(vcpu, i);
 
 	/* Synchronize offsets across timers of a VM if not already provided */
-	if (!vcpu_is_protected(vcpu) &&
+	if (!vcpu_is_confidential(vcpu) &&
 	    !test_bit(KVM_ARCH_FLAG_VM_COUNTER_OFFSET, &vcpu->kvm->arch.flags)) {
 		timer_set_offset(vcpu_vtimer(vcpu), kvm_phys_timer_read());
 		timer_set_offset(vcpu_ptimer(vcpu), 0);
@@ -1736,7 +1736,7 @@ int kvm_vm_ioctl_set_counter_offset(struct kvm *kvm,
 	if (offset->reserved)
 		return -EINVAL;
 
-	if (kvm_vm_is_protected(kvm))
+	if (kvm_vm_is_confidential(kvm))
 		return -EINVAL;
 
 	mutex_lock(&kvm->lock);
-- 
2.43.0

[PATCH v17 10/20] KVM: arm64: coco: Disable Steal time accounting for coco guests

[Suzuki K Poulose]

PVTIME support is advertised by KVM_CAP_STEAL_TIME, which doesn't take into
account the kvm instance. Even with that, a VMM could skip the CAP check and
proceed to configure the PVTIME as we don't do further check on the DEVICE_CTRL.
Tighten this up by passing the KVM instance around wherever possible and
catch things early

Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_host.h |  2 +-
 arch/arm64/kvm/arm.c              |  2 +-
 arch/arm64/kvm/pvtime.c           | 10 +++++-----
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 020aef5a1b698..0f258f16fdf81 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -1367,7 +1367,7 @@ long kvm_hypercall_pv_features(struct kvm_vcpu *vcpu);
 gpa_t kvm_init_stolen_time(struct kvm_vcpu *vcpu);
 void kvm_update_stolen_time(struct kvm_vcpu *vcpu);
 
-bool kvm_arm_pvtime_supported(void);
+bool kvm_arm_pvtime_supported(struct kvm *kvm);
 int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
 			    struct kvm_device_attr *attr);
 int kvm_arm_pvtime_get_attr(struct kvm_vcpu *vcpu,
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 2ecd92156f7cc..b5e0893ea16f1 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -473,7 +473,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 		r = system_supports_mte();
 		break;
 	case KVM_CAP_STEAL_TIME:
-		r = kvm_arm_pvtime_supported();
+		r = kvm_arm_pvtime_supported(kvm);
 		break;
 	case KVM_CAP_ARM_EL1_32BIT:
 		r = cpus_have_final_cap(ARM64_HAS_32BIT_EL1);
diff --git a/arch/arm64/kvm/pvtime.c b/arch/arm64/kvm/pvtime.c
index 4ceabaa4c30bd..41fbfd92cd292 100644
--- a/arch/arm64/kvm/pvtime.c
+++ b/arch/arm64/kvm/pvtime.c
@@ -67,9 +67,9 @@ gpa_t kvm_init_stolen_time(struct kvm_vcpu *vcpu)
 	return base;
 }
 
-bool kvm_arm_pvtime_supported(void)
+bool kvm_arm_pvtime_supported(struct kvm *kvm)
 {
-	return !!sched_info_on();
+	return !!sched_info_on() && (!kvm || !kvm_vm_is_confidential(kvm));
 }
 
 int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
@@ -81,7 +81,7 @@ int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
 	int ret = 0;
 	int idx;
 
-	if (!kvm_arm_pvtime_supported() ||
+	if (!kvm_arm_pvtime_supported(kvm) ||
 	    attr->attr != KVM_ARM_VCPU_PVTIME_IPA)
 		return -ENXIO;
 
@@ -110,7 +110,7 @@ int kvm_arm_pvtime_get_attr(struct kvm_vcpu *vcpu,
 	u64 __user *user = (u64 __user *)attr->addr;
 	u64 ipa;
 
-	if (!kvm_arm_pvtime_supported() ||
+	if (!kvm_arm_pvtime_supported(vcpu->kvm) ||
 	    attr->attr != KVM_ARM_VCPU_PVTIME_IPA)
 		return -ENXIO;
 
@@ -126,7 +126,7 @@ int kvm_arm_pvtime_has_attr(struct kvm_vcpu *vcpu,
 {
 	switch (attr->attr) {
 	case KVM_ARM_VCPU_PVTIME_IPA:
-		if (kvm_arm_pvtime_supported())
+		if (kvm_arm_pvtime_supported(vcpu->kvm))
 			return 0;
 	}
 	return -ENXIO;
-- 
2.43.0

[PATCH v17 11/20] KVM: arm64: coco: Don't handle MMIO with no ISV

[Suzuki K Poulose]

Generalize the MMIO with no ISV handling to confidential guests.

Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/kvm/mmio.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/mmio.c b/arch/arm64/kvm/mmio.c
index d1c3a352d5a22..bbcf52f01e912 100644
--- a/arch/arm64/kvm/mmio.c
+++ b/arch/arm64/kvm/mmio.c
@@ -168,14 +168,14 @@ int io_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)
 	 * No valid syndrome? Ask userspace for help if it has
 	 * volunteered to do so, and bail out otherwise.
 	 *
-	 * In the protected VM case, there isn't much userspace can do
+	 * In the protected/Realm VM case, there isn't much userspace can do
 	 * though, so directly deliver an exception to the guest.
 	 */
 	if (!kvm_vcpu_dabt_isvalid(vcpu)) {
 		trace_kvm_mmio_nisv(*vcpu_pc(vcpu), esr,
 				    kvm_vcpu_get_hfar(vcpu), fault_ipa);
 
-		if (vcpu_is_protected(vcpu))
+		if (vcpu_is_confidential(vcpu))
 			return kvm_inject_sea_dabt(vcpu, kvm_vcpu_get_hfar(vcpu));
 
 		if (test_bit(KVM_ARCH_FLAG_RETURN_NISV_IO_ABORT_TO_USER,
-- 
2.43.0

[PATCH v17 12/20] KVM: arm64: CCA: Support timers in realm RECs

[Suzuki K Poulose]

From: Steven Price <steven.price@arm.com>

The RMM keeps track of the timer while the realm REC is running, but on
exit to the normal world KVM is responsible for handling the timers.

A later patch adds the support for propagating the timer values from the
exit data structure and makeing sure the values are in sync for KVM.

Also, RMM doesn't support injecting virtual interrupts backed by Physical
interrupts. So, use the existing software resampling mechanims for Realm
timer interrupts.

Signed-off-by: Steven Price <steven.price@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/kvm/arch_timer.c | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index eba84cab95d01..3eddf44f7b1dd 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -56,11 +56,25 @@ static unsigned long kvm_arch_timer_get_irq_flags(void)
 	return kvm_vgic_global_state.no_hw_deactivation ? VGIC_IRQ_SW_RESAMPLE : 0;
 }
 
+static unsigned long kvm_realm_timer_get_irq_flags(void)
+{
+	/*
+	 * RMI_REC_ENTER rejects LRs with the HW bit set, so use the existing
+	 * software resampling mechanism for Realm timer interrupts.
+	 */
+	return VGIC_IRQ_SW_RESAMPLE;
+}
+
 static const struct irq_ops arch_timer_irq_ops = {
 	.get_flags	 = kvm_arch_timer_get_irq_flags,
 	.get_input_level = kvm_arch_timer_get_input_level,
 };
 
+static const struct irq_ops realm_timer_irq_ops = {
+	.get_flags	 = kvm_realm_timer_get_irq_flags,
+	.get_input_level = kvm_arch_timer_get_input_level,
+};
+
 static const struct irq_ops arch_timer_irq_ops_vgic_v5 = {
 	.get_input_level = kvm_arch_timer_get_input_level,
 	.queue_irq_unlock = vgic_v5_ppi_queue_irq_unlock,
@@ -1609,8 +1623,12 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
 
 	get_timer_map(vcpu, &map);
 
-	ops = vgic_is_v5(vcpu->kvm) ? &arch_timer_irq_ops_vgic_v5 :
-				      &arch_timer_irq_ops;
+	if (vcpu_is_rec(vcpu))
+		ops = &realm_timer_irq_ops;
+	else if (vgic_is_v5(vcpu->kvm))
+		ops = &arch_timer_irq_ops_vgic_v5;
+	else
+		ops = &arch_timer_irq_ops;
 
 	for (int i = 0; i < nr_timers(vcpu); i++)
 		kvm_vgic_set_irq_ops(vcpu, timer_irq(vcpu_get_timer(vcpu, i)), ops);
-- 
2.43.0

[PATCH v17 13/20] KVM: arm64: CCA: Add VCPU load/put for Realms

[Suzuki K Poulose]

RMM controls the VCPU settings and most are hidden from the KVM, except for the
VGIC and timer bits.

A later patch would add syncing the VCPU state into the Realm REC related SMC
parameters.

Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/kvm/arm.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index b5e0893ea16f1..3dfe0e03fab7a 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -807,6 +807,13 @@ static void pkvm_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 		     &vcpu->arch.vgic_cpu.vgic_v3);
 }
 
+static void realm_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
+{
+	kvm_timer_vcpu_load(vcpu);
+	kvm_vgic_load(vcpu);
+	vcpu_set_wfx_traps(vcpu);
+}
+
 void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 {
 	vcpu->cpu = cpu;
@@ -853,6 +860,12 @@ static void pkvm_vcpu_put(struct kvm_vcpu *vcpu)
 	nvhe_vcpu_put(vcpu);
 }
 
+static void realm_vcpu_put(struct kvm_vcpu *vcpu)
+{
+	kvm_timer_vcpu_put(vcpu);
+	kvm_vgic_put(vcpu);
+}
+
 void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
 {
 	vcpu->arch.vcpu_ops->vcpu_put(vcpu);
@@ -2210,11 +2223,17 @@ static const struct kvm_vcpu_ops pkvm_vcpu_ops = {
 	.vcpu_put = pkvm_vcpu_put,
 };
 
+static const struct kvm_vcpu_ops realm_vcpu_ops = {
+	.vcpu_load = realm_vcpu_load,
+	.vcpu_put = realm_vcpu_put,
+};
+
 static const struct kvm_vcpu_ops *arm64_vcpu_ops[] = {
 	[VM_VHE] = &vhe_vcpu_ops,
 	[VM_NVHE] = &nvhe_vcpu_ops,
 	[VM_PKVM] = &pkvm_vcpu_ops,
 	[VM_PROTECTED_PKVM] = &pkvm_vcpu_ops,
+	[VM_REALM] = &realm_vcpu_ops,
 };
 
 static unsigned long nvhe_percpu_size(void)
-- 
2.43.0

[PATCH v17 14/20] KVM: arm64: CCA: Don't expose unsupported capabilities for realm guests

[Suzuki K Poulose]

RMM v2.0 provides no mechanism for the host to perform debug operations
on the guest. So limit the extensions that are visible to an allowlist
so that only those capabilities we can support are advertised.

Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v13:
 * Add missing check in kvm_vm_ioctl_enable_cap().
Changes since v10:
 * Add a kvm_realm_ext_allowed() function which limits which extensions
   are exposed to an allowlist. This removes the need for special casing
   various extensions.
Changes since v7:
 * Remove the helper functions and inline the kvm_is_realm() check with
   a ternary operator.
 * Rewrite the commit message to explain this patch.
---
 arch/arm64/kvm/arm.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 3dfe0e03fab7a..fbb4d6651ab51 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -137,6 +137,25 @@ int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
 	return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE;
 }
 
+static bool kvm_realm_ext_allowed(long ext)
+{
+	switch (ext) {
+	case KVM_CAP_IRQCHIP:
+	case KVM_CAP_ARM_PSCI:
+	case KVM_CAP_ARM_PSCI_0_2:
+	case KVM_CAP_NR_VCPUS:
+	case KVM_CAP_MAX_VCPUS:
+	case KVM_CAP_MAX_VCPU_ID:
+	case KVM_CAP_MSI_DEVID:
+	case KVM_CAP_ARM_VM_IPA_SIZE:
+	case KVM_CAP_ARM_PTRAUTH_ADDRESS:
+	case KVM_CAP_ARM_PTRAUTH_GENERIC:
+	case KVM_CAP_SYNC_MMU:
+		return true;
+	}
+	return false;
+}
+
 int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 			    struct kvm_enable_cap *cap)
 {
@@ -148,6 +167,9 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 	if (is_protected_kvm_enabled() && !kvm_pkvm_ext_allowed(kvm, cap->cap))
 		return -EINVAL;
 
+	if (kvm && kvm_vm_is_realm(kvm) && !kvm_realm_ext_allowed(cap->cap))
+		return -EINVAL;
+
 	switch (cap->cap) {
 	case KVM_CAP_ARM_NISV_TO_USER:
 		r = 0;
@@ -407,6 +429,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	if (is_protected_kvm_enabled() && !kvm_pkvm_ext_allowed(kvm, ext))
 		return 0;
 
+	if (kvm && kvm_vm_is_realm(kvm) && !kvm_realm_ext_allowed(ext))
+		return 0;
+
 	switch (ext) {
 	case KVM_CAP_IRQCHIP:
 		r = vgic_present;
-- 
2.43.0

[PATCH v17 15/20] KVM: arm64: CCA: WARN on injected undef exceptions

[Suzuki K Poulose]

From: Steven Price <steven.price@arm.com>

The RMM doesn't allow injection of a undefined exception into a realm
guest. Add a WARN to catch if this ever happens.

Signed-off-by: Steven Price <steven.price@arm.com>
---
Changes since v15:
 * Switch to KVM_BUG() to mark the VM as bugged as well.
Changes since v6:
 * if (x) WARN(1, ...) makes no sense, just WARN(x, ...)!
---
 arch/arm64/kvm/inject_fault.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
index d6c4fc16f8795..d61a3ff04fabb 100644
--- a/arch/arm64/kvm/inject_fault.c
+++ b/arch/arm64/kvm/inject_fault.c
@@ -317,6 +317,7 @@ void kvm_inject_size_fault(struct kvm_vcpu *vcpu)
  */
 void kvm_inject_undefined(struct kvm_vcpu *vcpu)
 {
+	KVM_BUG(vcpu_is_rec(vcpu), vcpu->kvm, "Unexpected undefined exception injection to REC");
 	if (vcpu_el1_is_32bit(vcpu))
 		inject_undef32(vcpu);
 	else
-- 
2.43.0

[PATCH v17 16/20] KVM: arm64: CCA: Provide register list for unfinalized RECs

[Suzuki K Poulose]

From: Jean-Philippe Brucker <jean-philippe@linaro.org>

KVM_GET_REG_LIST should not be called before SVE is finalized. The ioctl
handler currently returns -EPERM in this case. But because it uses
kvm_arm_vcpu_is_finalized(), it now also rejects the call for
unfinalized REC even though finalizing the REC can only be done late,
after Realm descriptor creation.

Move the check to copy_sve_reg_indices(). One adverse side effect of
this change is that a KVM_GET_REG_LIST call that only probes for the
array size will now succeed even if SVE is not finalized, but that seems
harmless since the following KVM_GET_REG_LIST with the full array will
fail.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Signed-off-by: Steven Price <steven.price@arm.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/kvm/arm.c   | 14 +++++++++++++-
 arch/arm64/kvm/guest.c | 10 +++++-----
 2 files changed, 18 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index fbb4d6651ab51..3fe5a4f77738e 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1980,6 +1980,18 @@ static int kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,
 	return __kvm_arm_vcpu_set_events(vcpu, events);
 }
 
+/*
+ * Realm VCPUs can be finalized after the Realm descriptor is created.
+ * Allow the register list for RECs before the VCPUs are finalized, but
+ * after the SVE is finalized.
+ */
+static bool kvm_arm_vcpu_reg_list_allowed(struct kvm_vcpu *vcpu)
+{
+	if (kvm_arm_vcpu_is_finalized(vcpu))
+		return true;
+	return vcpu_is_rec(vcpu);
+}
+
 long kvm_arch_vcpu_ioctl(struct file *filp,
 			 unsigned int ioctl, unsigned long arg)
 {
@@ -2035,7 +2047,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 			break;
 
 		r = -EPERM;
-		if (!kvm_arm_vcpu_is_finalized(vcpu))
+		if (!kvm_arm_vcpu_reg_list_allowed(vcpu))
 			break;
 
 		r = -EFAULT;
diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
index b01d6622b8720..c3ca369882273 100644
--- a/arch/arm64/kvm/guest.c
+++ b/arch/arm64/kvm/guest.c
@@ -598,8 +598,8 @@ static unsigned long num_sve_regs(const struct kvm_vcpu *vcpu)
 	if (!vcpu_has_sve(vcpu))
 		return 0;
 
-	/* Policed by KVM_GET_REG_LIST: */
-	WARN_ON(!kvm_arm_vcpu_sve_finalized(vcpu));
+	if (!kvm_arm_vcpu_sve_finalized(vcpu))
+		return 1; /* KVM_REG_ARM64_SVE_VLS */
 
 	return slices * (SVE_NUM_PREGS + SVE_NUM_ZREGS + 1 /* FFR */)
 		+ 1; /* KVM_REG_ARM64_SVE_VLS */
@@ -616,9 +616,6 @@ static int copy_sve_reg_indices(const struct kvm_vcpu *vcpu,
 	if (!vcpu_has_sve(vcpu))
 		return 0;
 
-	/* Policed by KVM_GET_REG_LIST: */
-	WARN_ON(!kvm_arm_vcpu_sve_finalized(vcpu));
-
 	/*
 	 * Enumerate this first, so that userspace can save/restore in
 	 * the order reported by KVM_GET_REG_LIST:
@@ -628,6 +625,9 @@ static int copy_sve_reg_indices(const struct kvm_vcpu *vcpu,
 		return -EFAULT;
 	++num_regs;
 
+	if (!kvm_arm_vcpu_sve_finalized(vcpu))
+		return num_regs;
+
 	for (i = 0; i < slices; i++) {
 		for (n = 0; n < SVE_NUM_ZREGS; n++) {
 			reg = KVM_REG_ARM64_SVE_ZREG(n, i);
-- 
2.43.0

[PATCH v17 17/20] KVM: arm64: CCA: Provide an accurate register list

[Suzuki K Poulose]

From: Jean-Philippe Brucker <jean-philippe@linaro.org>

Userspace can set a few registers with KVM_SET_ONE_REG (9 GP registers
at runtime, and 3 system registers during initialization). Update the
register list returned by KVM_GET_REG_LIST.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Signed-off-by: Steven Price <steven.price@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/kvm/guest.c      |  6 ++++++
 arch/arm64/kvm/hypercalls.c |  4 ++--
 arch/arm64/kvm/sys_regs.c   | 27 +++++++++++++++++++++------
 3 files changed, 29 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
index c3ca369882273..6bec73d1ee050 100644
--- a/arch/arm64/kvm/guest.c
+++ b/arch/arm64/kvm/guest.c
@@ -601,6 +601,9 @@ static unsigned long num_sve_regs(const struct kvm_vcpu *vcpu)
 	if (!kvm_arm_vcpu_sve_finalized(vcpu))
 		return 1; /* KVM_REG_ARM64_SVE_VLS */
 
+	if (kvm_vm_is_realm(vcpu->kvm))
+		return 1; /* KVM_REG_ARM64_SVE_VLS */
+
 	return slices * (SVE_NUM_PREGS + SVE_NUM_ZREGS + 1 /* FFR */)
 		+ 1; /* KVM_REG_ARM64_SVE_VLS */
 }
@@ -628,6 +631,9 @@ static int copy_sve_reg_indices(const struct kvm_vcpu *vcpu,
 	if (!kvm_arm_vcpu_sve_finalized(vcpu))
 		return num_regs;
 
+	if (kvm_vm_is_realm(vcpu->kvm))
+		return num_regs;
+
 	for (i = 0; i < slices; i++) {
 		for (n = 0; n < SVE_NUM_ZREGS; n++) {
 			reg = KVM_REG_ARM64_SVE_ZREG(n, i);
diff --git a/arch/arm64/kvm/hypercalls.c b/arch/arm64/kvm/hypercalls.c
index b11b8821c9fbc..2b1e6fdeb4d5c 100644
--- a/arch/arm64/kvm/hypercalls.c
+++ b/arch/arm64/kvm/hypercalls.c
@@ -414,14 +414,14 @@ void kvm_arm_teardown_hypercalls(struct kvm *kvm)
 
 int kvm_arm_get_fw_num_regs(struct kvm_vcpu *vcpu)
 {
-	return ARRAY_SIZE(kvm_arm_fw_reg_ids);
+	return vcpu_is_rec(vcpu) ? 0 : ARRAY_SIZE(kvm_arm_fw_reg_ids);
 }
 
 int kvm_arm_copy_fw_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
 {
 	int i;
 
-	for (i = 0; i < ARRAY_SIZE(kvm_arm_fw_reg_ids); i++) {
+	for (i = 0; i < kvm_arm_get_fw_num_regs(vcpu); i++) {
 		if (put_user(kvm_arm_fw_reg_ids[i], uindices++))
 			return -EFAULT;
 	}
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 44aae52c473d7..a58c1fc13ea17 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -5638,18 +5638,18 @@ int kvm_arm_sys_reg_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg
 				    sys_reg_descs, ARRAY_SIZE(sys_reg_descs));
 }
 
-static unsigned int num_demux_regs(void)
+static inline unsigned int num_demux_regs(struct kvm_vcpu *vcpu)
 {
-	return CSSELR_MAX;
+	return vcpu_is_rec(vcpu) ? 0 : CSSELR_MAX;
 }
 
-static int write_demux_regids(u64 __user *uindices)
+static int write_demux_regids(struct kvm_vcpu *vcpu, u64 __user *uindices)
 {
 	u64 val = KVM_REG_ARM64 | KVM_REG_SIZE_U32 | KVM_REG_ARM_DEMUX;
 	unsigned int i;
 
 	val |= KVM_REG_ARM_DEMUX_ID_CCSIDR;
-	for (i = 0; i < CSSELR_MAX; i++) {
+	for (i = 0; i < num_demux_regs(vcpu); i++) {
 		if (put_user(val | i, uindices))
 			return -EFAULT;
 		uindices++;
@@ -5693,11 +5693,26 @@ static bool copy_reg_to_user(const struct sys_reg_desc *reg, u64 __user **uind)
 	return true;
 }
 
+static inline bool kvm_realm_sys_reg_hidden_user(const struct kvm_vcpu *vcpu,
+						 u64 reg)
+{
+	switch (reg) {
+	case SYS_ID_AA64DFR0_EL1:
+	case SYS_PMCR_EL0:
+		return false;
+	}
+	return true;
+}
+
 static int walk_one_sys_reg(const struct kvm_vcpu *vcpu,
 			    const struct sys_reg_desc *rd,
 			    u64 __user **uind,
 			    unsigned int *total)
 {
+	if (vcpu_is_rec(vcpu) &&
+	    kvm_realm_sys_reg_hidden_user(vcpu, reg_to_encoding(rd)))
+		return 0;
+
 	/*
 	 * Ignore registers we trap but don't save,
 	 * and for which no custom user accessor is provided.
@@ -5735,7 +5750,7 @@ static int walk_sys_regs(struct kvm_vcpu *vcpu, u64 __user *uind)
 
 unsigned long kvm_arm_num_sys_reg_descs(struct kvm_vcpu *vcpu)
 {
-	return num_demux_regs()
+	return num_demux_regs(vcpu)
 		+ walk_sys_regs(vcpu, (u64 __user *)NULL);
 }
 
@@ -5748,7 +5763,7 @@ int kvm_arm_copy_sys_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
 		return err;
 	uindices += err;
 
-	return write_demux_regids(uindices);
+	return write_demux_regids(vcpu, uindices);
 }
 
 #define KVM_ARM_FEATURE_ID_RANGE_INDEX(r)			\
-- 
2.43.0

[PATCH v17 18/20] KVM: arm64: Reuse kvm_stage2_unmap_range in kvm_unmap_gfn_range

[Suzuki K Poulose]

In preparation for adding VM specific backends for stage2 operation,
switch to kvm_stage2_unmap_range() instead of __unmap_stage2_range()
from the kvm_unmap_gfn_range(). Drop the bail out check for protected
VMs and defer that to the one in kvm_stage2_unmap_range(). Later we
would replace the logic in kvm_stage2_unmap_range() with VM specific
backends.

No functional changes intended.

Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/kvm/mmu.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 9ba86450fe4af..67852acf7a6f8 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -2436,12 +2436,12 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
 
 bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range)
 {
-	if (!kvm->arch.mmu.pgt || kvm_vm_is_protected(kvm))
+	if (!kvm->arch.mmu.pgt)
 		return false;
 
-	__unmap_stage2_range(&kvm->arch.mmu, range->start << PAGE_SHIFT,
-			     (range->end - range->start) << PAGE_SHIFT,
-			     range->may_block);
+	kvm_stage2_unmap_range(&kvm->arch.mmu, range->start << PAGE_SHIFT,
+				(range->end - range->start) << PAGE_SHIFT,
+				range->may_block);
 
 	kvm_nested_s2_unmap(kvm, range->may_block);
 	return false;
-- 
2.43.0

[PATCH v17 19/20] KVM: arm64: Add VM specific callback for S2 MMU operations

[Suzuki K Poulose]

Add VM type specific S2 MMU operation backends which can be initialized per VM
flavor, to keep the handling cleaner.

Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_host.h |  15 ++++
 arch/arm64/kvm/mmu.c              | 129 +++++++++++++++++++++++++-----
 2 files changed, 124 insertions(+), 20 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 0f258f16fdf81..e3c308b6e319f 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -157,6 +157,19 @@ struct kvm_vcpu_ops {
 	void (*vcpu_put)(struct kvm_vcpu *vcpu);
 };
 
+struct kvm_gfn_range;
+
+struct kvm_vm_s2_ops {
+	bool (*vm_age_gfn)(struct kvm *kvm, struct kvm_gfn_range *range);
+	bool (*vm_test_age_gfn)(struct kvm *kvm, struct kvm_gfn_range *range);
+	int (*vm_flush_remote_tlbs)(struct kvm *kvm);
+	int (*vm_flush_remote_tlbs_range)(struct kvm *kvm, gfn_t gfn,
+					  u64 nr_pages);
+	void (*vm_stage2_unmap_range)(struct kvm_s2_mmu *mmu,
+				      phys_addr_t start, u64 size,
+				      bool may_block);
+};
+
 struct kvm_s2_mmu {
 	struct kvm_vmid vmid;
 
@@ -335,6 +348,8 @@ struct kvm_arch {
 	 */
 	u64 fgu[__NR_FGT_GROUP_IDS__];
 
+	const struct kvm_vm_s2_ops *vm_s2_ops;
+
 	/*
 	 * Stage 2 paging state for VMs with nested S2 using a virtual
 	 * VMID.
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 67852acf7a6f8..e78a56fb73696 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -37,6 +37,10 @@ static unsigned long __ro_after_init io_map_base;
 
 #define KVM_PGT_FN(fn)		(!is_protected_kvm_enabled() ? fn : p ## fn)
 
+static const struct kvm_vm_s2_ops protected_pkvm_s2_ops;
+static const struct kvm_vm_s2_ops unprotected_pkvm_s2_ops;
+static const struct kvm_vm_s2_ops kvm_default_vm_s2_ops;
+
 static phys_addr_t __stage2_range_addr_end(phys_addr_t addr, phys_addr_t end,
 					   phys_addr_t size)
 {
@@ -166,6 +170,18 @@ static bool memslot_is_logging(struct kvm_memory_slot *memslot)
 	return memslot->dirty_bitmap && !(memslot->flags & KVM_MEM_READONLY);
 }
 
+static int pkvm_flush_remote_tlbs(struct kvm *kvm)
+{
+	kvm_call_hyp_nvhe(__pkvm_tlb_flush_vmid, kvm->arch.pkvm.handle);
+	return 0;
+}
+
+static int kvm_vm_flush_remote_tlbs(struct kvm *kvm)
+{
+	kvm_call_hyp(__kvm_tlb_flush_vmid, &kvm->arch.mmu);
+	return 0;
+}
+
 /**
  * kvm_arch_flush_remote_tlbs() - flush all VM TLB entries for v7/8
  * @kvm:	pointer to kvm structure.
@@ -174,26 +190,36 @@ static bool memslot_is_logging(struct kvm_memory_slot *memslot)
  */
 int kvm_arch_flush_remote_tlbs(struct kvm *kvm)
 {
-	if (is_protected_kvm_enabled())
-		kvm_call_hyp_nvhe(__pkvm_tlb_flush_vmid, kvm->arch.pkvm.handle);
-	else
-		kvm_call_hyp(__kvm_tlb_flush_vmid, &kvm->arch.mmu);
-	return 0;
+	if (!kvm->arch.vm_s2_ops->vm_flush_remote_tlbs)
+		return 0;
+	return kvm->arch.vm_s2_ops->vm_flush_remote_tlbs(kvm);
 }
 
-int kvm_arch_flush_remote_tlbs_range(struct kvm *kvm,
-				      gfn_t gfn, u64 nr_pages)
+static int pkvm_flush_remote_tlbs_range(struct kvm *kvm,
+					gfn_t gfn, u64 nr_pages)
+{
+	return pkvm_flush_remote_tlbs(kvm);
+}
+
+static int kvm_vm_flush_remote_tlbs_range(struct kvm *kvm,
+					 gfn_t gfn, u64 nr_pages)
 {
 	u64 size = nr_pages << PAGE_SHIFT;
 	u64 addr = gfn << PAGE_SHIFT;
 
-	if (is_protected_kvm_enabled())
-		kvm_call_hyp_nvhe(__pkvm_tlb_flush_vmid, kvm->arch.pkvm.handle);
-	else
-		kvm_tlb_flush_vmid_range(&kvm->arch.mmu, addr, size);
+	kvm_tlb_flush_vmid_range(&kvm->arch.mmu, addr, size);
 	return 0;
 }
 
+int kvm_arch_flush_remote_tlbs_range(struct kvm *kvm,
+				     gfn_t gfn, u64 nr_pages)
+{
+	if (!kvm->arch.vm_s2_ops->vm_flush_remote_tlbs_range)
+		return 0;
+
+	return kvm->arch.vm_s2_ops->vm_flush_remote_tlbs_range(kvm, gfn, nr_pages);
+}
+
 static void *stage2_memcache_zalloc_page(void *arg)
 {
 	struct kvm_mmu_memory_cache *mc = arg;
@@ -337,13 +363,20 @@ static void __unmap_stage2_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64
 				   may_block));
 }
 
+static void kvm_vm_stage2_unmap_range(struct kvm_s2_mmu *mmu,
+				      phys_addr_t start,
+				      u64 size, bool may_block)
+{
+	__unmap_stage2_range(mmu, start, size, may_block);
+}
+
 void kvm_stage2_unmap_range(struct kvm_s2_mmu *mmu, phys_addr_t start,
 			    u64 size, bool may_block)
 {
-	if (kvm_vm_is_protected(kvm_s2_mmu_to_kvm(mmu)))
-		return;
+	struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
 
-	__unmap_stage2_range(mmu, start, size, may_block);
+	if (kvm->arch.vm_s2_ops->vm_stage2_unmap_range)
+		kvm->arch.vm_s2_ops->vm_stage2_unmap_range(mmu, start, size, may_block);
 }
 
 void kvm_stage2_flush_range(struct kvm_s2_mmu *mmu, phys_addr_t addr, phys_addr_t end)
@@ -963,6 +996,18 @@ static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
 	KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
 }
 
+static const struct kvm_vm_s2_ops *arm64_vm_s2_ops[VM_FLAVOR_MAX] = {
+	[VM_VHE] = &kvm_default_vm_s2_ops,
+	[VM_NVHE] = &kvm_default_vm_s2_ops,
+	[VM_PKVM] = &unprotected_pkvm_s2_ops,
+	[VM_PROTECTED_PKVM] = &protected_pkvm_s2_ops,
+};
+
+static void kvm_vm_init_vm_s2_ops(struct kvm *kvm, unsigned long type)
+{
+	kvm->arch.vm_s2_ops = arm64_vm_s2_ops[kvm->arch.vm_flavor];
+}
+
 /**
  * kvm_init_stage2_mmu - Initialise a S2 MMU structure
  * @kvm:	The pointer to the KVM structure
@@ -983,6 +1028,9 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
 	int cpu, err;
 	struct kvm_pgtable *pgt;
 
+	/* Initialize the VM ops for the VM instance for the first time */
+	if (mmu == &kvm->arch.mmu)
+		kvm_vm_init_vm_s2_ops(kvm, type);
 	/*
 	 * If we already have our page tables in place, and that the
 	 * MMU context is the canonical one, we have a bug somewhere,
@@ -2447,27 +2495,32 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range)
 	return false;
 }
 
-bool kvm_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
+static bool kvm_vm_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
 {
 	u64 size = (range->end - range->start) << PAGE_SHIFT;
 
-	if (!kvm->arch.mmu.pgt || kvm_vm_is_protected(kvm))
-		return false;
-
 	return KVM_PGT_FN(kvm_pgtable_stage2_test_clear_young)(kvm->arch.mmu.pgt,
 						   range->start << PAGE_SHIFT,
 						   size, true);
+}
+
+bool kvm_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
+{
+	if (!kvm->arch.mmu.pgt || !kvm->arch.vm_s2_ops->vm_age_gfn)
+		return false;
+
+	return kvm->arch.vm_s2_ops->vm_age_gfn(kvm, range);
 	/*
 	 * TODO: Handle nested_mmu structures here using the reverse mapping in
 	 * a later version of patch series.
 	 */
 }
 
-bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
+static bool kvm_vm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
 {
 	u64 size = (range->end - range->start) << PAGE_SHIFT;
 
-	if (!kvm->arch.mmu.pgt || kvm_vm_is_protected(kvm))
+	if (!kvm->arch.mmu.pgt)
 		return false;
 
 	return KVM_PGT_FN(kvm_pgtable_stage2_test_clear_young)(kvm->arch.mmu.pgt,
@@ -2475,6 +2528,15 @@ bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
 						   size, false);
 }
 
+bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
+{
+
+	if (!kvm->arch.mmu.pgt || !kvm->arch.vm_s2_ops->vm_test_age_gfn)
+		return false;
+
+	return kvm->arch.vm_s2_ops->vm_test_age_gfn(kvm, range);
+}
+
 phys_addr_t kvm_mmu_get_httbr(void)
 {
 	return __pa(hyp_pgtable->pgd);
@@ -2796,3 +2858,30 @@ void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled)
 
 	trace_kvm_toggle_cache(*vcpu_pc(vcpu), was_enabled, now_enabled);
 }
+
+static const struct kvm_vm_s2_ops protected_vm_s2_ops = {
+	.vm_flush_remote_tlbs		= pkvm_flush_remote_tlbs,
+	.vm_flush_remote_tlbs_range	= pkvm_flush_remote_tlbs_range,
+	/*
+	 * Not supported for Protected VMs under pKVM
+	 * .vm_age_gfn
+	 * .vm_test_age_gfn
+	 * .vm_stage2_unmap_range
+	 */
+};
+
+static const struct kvm_vm_s2_ops pkvm_vm_s2_ops = {
+	.vm_flush_remote_tlbs		= pkvm_flush_remote_tlbs,
+	.vm_flush_remote_tlbs_range	= pkvm_flush_remote_tlbs_range,
+	.vm_age_gfn			= kvm_vm_age_gfn,
+	.vm_test_age_gfn		= kvm_vm_test_age_gfn,
+	.vm_stage2_unmap_range		= kvm_vm_stage2_unmap_range,
+};
+
+static const struct kvm_vm_s2_ops kvm_default_vm_s2_ops = {
+	.vm_flush_remote_tlbs		= kvm_vm_flush_remote_tlbs,
+	.vm_flush_remote_tlbs_range	= kvm_vm_flush_remote_tlbs_range,
+	.vm_age_gfn			= kvm_vm_age_gfn,
+	.vm_test_age_gfn		= kvm_vm_test_age_gfn,
+	.vm_stage2_unmap_range		= kvm_vm_stage2_unmap_range,
+};
-- 
2.43.0

[PATCH v17 20/20] KVM: arm64: Abstract out memory abort handling

[Suzuki K Poulose]

Move the memory abort handling under VM specific s2 operation.

Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_host.h |  2 ++
 arch/arm64/kvm/mmu.c              | 36 +++++++++++++++++++------------
 2 files changed, 24 insertions(+), 14 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index e3c308b6e319f..fbfd85dc999ef 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -158,6 +158,7 @@ struct kvm_vcpu_ops {
 };
 
 struct kvm_gfn_range;
+struct kvm_s2_fault_desc;
 
 struct kvm_vm_s2_ops {
 	bool (*vm_age_gfn)(struct kvm *kvm, struct kvm_gfn_range *range);
@@ -168,6 +169,7 @@ struct kvm_vm_s2_ops {
 	void (*vm_stage2_unmap_range)(struct kvm_s2_mmu *mmu,
 				      phys_addr_t start, u64 size,
 				      bool may_block);
+	int (*vm_mem_abort)(const struct kvm_s2_fault_desc *s2fd);
 };
 
 struct kvm_s2_mmu {
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index e78a56fb73696..5e7cdbe310fc8 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1753,7 +1753,7 @@ struct kvm_s2_fault_vma_info {
 	bool		map_non_cacheable;
 };
 
-static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd)
+static int protected_vm_mem_abort(const struct kvm_s2_fault_desc *s2fd)
 {
 	unsigned int flags = FOLL_HWPOISON | FOLL_LONGTERM | FOLL_WRITE;
 	struct kvm_vcpu *vcpu = s2fd->vcpu;
@@ -2191,6 +2191,22 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)
 	return kvm_s2_fault_map(s2fd, &s2vi, prot, memcache);
 }
 
+static int kvm_vm_mem_abort(const struct kvm_s2_fault_desc *s2fd)
+{
+	int ret;
+	struct kvm_vcpu *vcpu = s2fd->vcpu;
+
+	VM_WARN_ON_ONCE(kvm_vcpu_trap_is_permission_fault(vcpu) &&
+			!kvm_is_write_fault(vcpu) &&
+			!kvm_vcpu_trap_is_exec_fault(vcpu));
+
+	if (kvm_slot_has_gmem(s2fd->memslot))
+		ret = gmem_abort(s2fd);
+	else
+		ret = user_mem_abort(s2fd);
+	return ret;
+}
+
 /* Resolve the access fault by making the page young again. */
 static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)
 {
@@ -2298,6 +2314,7 @@ int kvm_handle_guest_sea(struct kvm_vcpu *vcpu)
 int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
 {
 	struct kvm_s2_trans nested_trans, *nested = NULL;
+	struct kvm *kvm = vcpu->kvm;
 	unsigned long esr;
 	phys_addr_t fault_ipa; /* The address we faulted on */
 	phys_addr_t ipa; /* Always the IPA in the L1 guest phys space */
@@ -2459,19 +2476,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
 		.hva		= hva,
 	};
 
-	if (kvm_vm_is_protected(vcpu->kvm)) {
-		ret = pkvm_mem_abort(&s2fd);
-	} else {
-		VM_WARN_ON_ONCE(kvm_vcpu_trap_is_permission_fault(vcpu) &&
-				!write_fault &&
-				!kvm_vcpu_trap_is_exec_fault(vcpu));
-
-		if (kvm_slot_has_gmem(memslot))
-			ret = gmem_abort(&s2fd);
-		else
-			ret = user_mem_abort(&s2fd);
-	}
-
+	ret = kvm->arch.vm_s2_ops->vm_mem_abort(&s2fd);
 	if (ret == 0)
 		ret = 1;
 out:
@@ -2868,6 +2873,7 @@ static const struct kvm_vm_s2_ops protected_vm_s2_ops = {
 	 * .vm_test_age_gfn
 	 * .vm_stage2_unmap_range
 	 */
+	.vm_mem_abort			= protected_vm_mem_abort,
 };
 
 static const struct kvm_vm_s2_ops pkvm_vm_s2_ops = {
@@ -2876,6 +2882,7 @@ static const struct kvm_vm_s2_ops pkvm_vm_s2_ops = {
 	.vm_age_gfn			= kvm_vm_age_gfn,
 	.vm_test_age_gfn		= kvm_vm_test_age_gfn,
 	.vm_stage2_unmap_range		= kvm_vm_stage2_unmap_range,
+	.vm_mem_abort			= kvm_vm_mem_abort,
 };
 
 static const struct kvm_vm_s2_ops kvm_default_vm_s2_ops = {
@@ -2884,4 +2891,5 @@ static const struct kvm_vm_s2_ops kvm_default_vm_s2_ops = {
 	.vm_age_gfn			= kvm_vm_age_gfn,
 	.vm_test_age_gfn		= kvm_vm_test_age_gfn,
 	.vm_stage2_unmap_range		= kvm_vm_stage2_unmap_range,
+	.vm_mem_abort			= kvm_vm_mem_abort,
 };
-- 
2.43.0

Re: [PATCH v17 09/20] KVM: arm64: coco: arch_timer: Prevent timer offset configuration

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Unprivileged host userspace can trigger a kernel WARN (and potential host panic) by setting SYS_CNTVCT_EL0 or SYS_CNTPCT_EL0 via KVM_SET_ONE_REG for a confidential VM.
--

commit 563a7eb94c0d2b010c530d4a42b1717738d5106a
Author: Suzuki K Poulose <suzuki.poulose@arm.com>

KVM: arm64: coco: arch_timer: Prevent timer offset configuration
    
Confidential guests, including protected PKVM guests and Arm CCA Realms, always
runs with 0 timer offset. Prevent the configuration for them.

> diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
> index 6ac3321f4c575..eba84cab95d01 100644
> --- a/arch/arm64/kvm/arch_timer.c
> +++ b/arch/arm64/kvm/arch_timer.c
> @@ -1079,7 +1079,7 @@ static void timer_context_init(struct kvm_vcpu *vcpu, int timerid)
>  
>  	ctxt->timer_id = timerid;
>  
> -	if (!kvm_vm_is_protected(vcpu->kvm)) {
> +	if (!vcpu_is_confidential(vcpu)) {
>  		if (timerid == TIMER_VTIMER)
>  			ctxt->offset.vm_offset = &kvm->arch.timer_data.voffset;
>  		else

[ ... ]

> @@ -1110,7 +1110,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
>  		timer_context_init(vcpu, i);
>  
>  	/* Synchronize offsets across timers of a VM if not already provided */
> -	if (!vcpu_is_protected(vcpu) &&
> +	if (!vcpu_is_confidential(vcpu) &&
>  	    !test_bit(KVM_ARCH_FLAG_VM_COUNTER_OFFSET, &vcpu->kvm->arch.flags)) {
>  		timer_set_offset(vcpu_vtimer(vcpu), kvm_phys_timer_read());
>  		timer_set_offset(vcpu_ptimer(vcpu), 0);

[Severity: High]
This is a pre-existing issue, but does this configuration allow unprivileged 
host userspace to trigger a kernel WARN?

For confidential VMs, ctxt->offset.vm_offset is initialized to NULL in 
timer_context_init(), and KVM_ARCH_FLAG_VM_COUNTER_OFFSET is skipped 
here in kvm_timer_vcpu_init().

However, if userspace calls KVM_SET_ONE_REG for SYS_CNTVCT_EL0, 
arch_timer_set_user() handles it like this:

arch/arm64/kvm/sys_regs.c:arch_timer_set_user() {
    ...
    case SYS_CNTVCT_EL0:
        if (!test_bit(KVM_ARCH_FLAG_VM_COUNTER_OFFSET, &vcpu->kvm->arch.flags))
            timer_set_offset(vcpu_vtimer(vcpu), kvm_phys_timer_read() - val);
        return 0;
    ...
}

Since the flag is clear for confidential VMs, this will unconditionally invoke
timer_set_offset() with a userspace-controlled offset. 

In timer_set_offset():

include/kvm/arm_arch_timer.h:timer_set_offset() {
    ...
    if (!ctxt->offset.vm_offset) {
        WARN(offset, "timer %d\n", arch_timer_ctx_index(ctxt));
        return;
    }
    ...
}

Because vm_offset is NULL, this triggers the WARN if the computed offset is
non-zero. With panic_on_warn enabled (common in cloud environments), could
this allow unprivileged host userspace to panic the host?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260908162223.1683432-1-suzuki.poulose@arm.com?part=9

Re: [PATCH v17 14/20] KVM: arm64: CCA: Don't expose unsupported capabilities for realm guests

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Redundant NULL check on the 'kvm' pointer in kvm_vm_ioctl_enable_cap().
--

commit 849adae1adab6cdadc3f93c498689cfc52591e9d
Author: Suzuki K Poulose <suzuki.poulose@arm.com>

KVM: arm64: CCA: Don't expose unsupported capabilities for realm guests

This commit restricts the extensions visible to realm guests to an
allowlist, as RMM v2.0 provides no mechanism for the host to perform
debug operations on the guest.

> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 3dfe0e03fab7a..fbb4d6651ab51 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
[ ... ]
> @@ -148,6 +167,9 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
>  	if (is_protected_kvm_enabled() && !kvm_pkvm_ext_allowed(kvm, cap->cap))
>  		return -EINVAL;
>  
> +	if (kvm && kvm_vm_is_realm(kvm) && !kvm_realm_ext_allowed(cap->cap))

[Severity: Low]
Is the kvm && check strictly necessary in kvm_vm_ioctl_enable_cap()? 

Since this function is only called as a VM-level ioctl (dispatched from
kvm_vm_ioctl() where the pointer is extracted from the file descriptor's
private data), kvm is structurally guaranteed to be non-NULL in this
context.

While kvm_vm_ioctl_check_extension() handles global /dev/kvm calls with a
NULL kvm pointer, this function does not appear to need the same check.

> +		return -EINVAL;
> +
>  	switch (cap->cap) {
>  	case KVM_CAP_ARM_NISV_TO_USER:
>  		r = 0;
[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260908162223.1683432-1-suzuki.poulose@arm.com?part=14

Re: [PATCH v17 12/20] KVM: arm64: CCA: Support timers in realm RECs

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Realm VM timers are blackholed on GICv5 hosts due to unconditional physical interrupt masking combined with disabled direct injection.
--

commit 759ada4556cf9518a1b30ec3610b7b7833129bd9
Author: Steven Price <steven.price@arm.com>

KVM: arm64: CCA: Support timers in realm RECs

This commit adds support for handling timers when exiting a realm REC to the
normal world. It implements realm_timer_irq_ops which relies on the existing
software resampling mechanism for Realm timer interrupts.

> diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
> index eba84cab95d01..3eddf44f7b1dd 100644
> --- a/arch/arm64/kvm/arch_timer.c
> +++ b/arch/arm64/kvm/arch_timer.c
[ ... ]
> @@ -1609,8 +1623,12 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
>  
>  	get_timer_map(vcpu, &map);
>  
> -	ops = vgic_is_v5(vcpu->kvm) ? &arch_timer_irq_ops_vgic_v5 :
> -				      &arch_timer_irq_ops;
> +	if (vcpu_is_rec(vcpu))
> +		ops = &realm_timer_irq_ops;
> +	else if (vgic_is_v5(vcpu->kvm))
> +		ops = &arch_timer_irq_ops_vgic_v5;
> +	else
> +		ops = &arch_timer_irq_ops;
>  
>  	for (int i = 0; i < nr_timers(vcpu); i++)
>  		kvm_vgic_set_irq_ops(vcpu, timer_irq(vcpu_get_timer(vcpu, i)), ops);

[Severity: High]
Will this cause Realm VM timers to be lost on a GICv5 host?

When a Realm VM is created on a GICv5 host, vcpu_is_rec(vcpu) takes precedence,
assigning realm_timer_irq_ops. However, this struct lacks the
.set_direct_injection callback used for hardware direct injection.

At the same time, because vgic_is_v5(vcpu->kvm) is still true for the host,
other timer paths will unconditionally assume direct injection is active. For
example, kvm_timer_update_irq() skips software injection:

kvm_timer_update_irq() {
        ...
        /* Skip injecting on GICv5 for directly injected (DVI'd) timers */
        if (vgic_is_v5(vcpu->kvm)) {
                ...
                if (map.direct_ptimer == timer_ctx ||
                    map.direct_vtimer == timer_ctx)
                        return;
        }
}

Additionally, kvm_timer_vcpu_load_gic() unconditionally masks the physical
timer interrupt for GICv5 setups:

kvm_timer_vcpu_load_gic() {
        ...
        phys_active |= pending;
        phys_active |= vgic_is_v5(vcpu->kvm);

        set_timer_irq_phys_active(ctx, phys_active);
}

Since hardware injection is not configured by the Realm ops and software
injection is bypassed due to the GICv5 checks, does this completely blackhole
timer interrupts for Realm VMs on GICv5 hardware?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260908162223.1683432-1-suzuki.poulose@arm.com?part=12

Re: [PATCH v17 16/20] KVM: arm64: CCA: Provide register list for unfinalized RECs

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Mismatch between KVM_GET_REG_LIST implementation for RECs and its commit message / comment.
--

commit ce52f6b2727531d94e14139ee2966392538b6218
Author: Jean-Philippe Brucker <jpb@kernel.org>

KVM: arm64: CCA: Provide register list for unfinalized RECs

This commit modifies the KVM_GET_REG_LIST ioctl handler to allow fetching
the register list for Realm Execution Contexts (RECs) before the VCPUs are
fully finalized, provided SVE is finalized. It adjusts the size probing and
array copying logic to prevent -EPERM errors during late REC finalization.

> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
[ ... ]
> @@ -1980,6 +1980,18 @@ static int kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,
>  	return __kvm_arm_vcpu_set_events(vcpu, events);
>  }
>  
> +/*
> + * Realm VCPUs can be finalized after the Realm descriptor is created.
> + * Allow the register list for RECs before the VCPUs are finalized, but
> + * after the SVE is finalized.
> + */
> +static bool kvm_arm_vcpu_reg_list_allowed(struct kvm_vcpu *vcpu)
> +{
> +	if (kvm_arm_vcpu_is_finalized(vcpu))
> +		return true;
> +	return vcpu_is_rec(vcpu);
> +}

[Severity: Medium]
Does this logic in kvm_arm_vcpu_reg_list_allowed() unconditionally allow
RECs, bypassing the SVE finalization check? The comment explicitly states
"but after the SVE is finalized," yet if the VCPU is a REC, this path
returns true regardless of the SVE finalization status.

> diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
> --- a/arch/arm64/kvm/guest.c
> +++ b/arch/arm64/kvm/guest.c
[ ... ]
> @@ -616,9 +616,6 @@ static int copy_sve_reg_indices(const struct kvm_vcpu *vcpu,
>  	if (!vcpu_has_sve(vcpu))
>  		return 0;
>  
> -	/* Policed by KVM_GET_REG_LIST: */
> -	WARN_ON(!kvm_arm_vcpu_sve_finalized(vcpu));
> -
>  	/*
>  	 * Enumerate this first, so that userspace can save/restore in
>  	 * the order reported by KVM_GET_REG_LIST:
> @@ -628,6 +625,9 @@ static int copy_sve_reg_indices(const struct kvm_vcpu *vcpu,
>  		return -EFAULT;
>  	++num_regs;
>  
> +	if (!kvm_arm_vcpu_sve_finalized(vcpu))
> +		return num_regs;
> +

[Severity: Medium]
Does this return a positive success code (num_regs) rather than an error?

The commit message states that "a KVM_GET_REG_LIST call that only probes
for the array size will now succeed even if SVE is not finalized, but that
seems harmless since the following KVM_GET_REG_LIST with the full array
will fail." 

Returning a positive number of registers here in copy_sve_reg_indices()
causes the full array fetch to succeed silently instead of failing as the
commit message claims.

>  	for (i = 0; i < slices; i++) {
>  		for (n = 0; n < SVE_NUM_ZREGS; n++) {
>  			reg = KVM_REG_ARM64_SVE_ZREG(n, i);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260908162223.1683432-1-suzuki.poulose@arm.com?part=16

Re: [PATCH v17 19/20] KVM: arm64: Add VM specific callback for S2 MMU operations

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Critical] Mismatch between forward declarations and definitions of pKVM S2 MMU operations structs causes KVM to silently skip critical memory management and TLB flush operations.
- [High] `kvm_arch_flush_remote_tlbs_range` incorrectly returns 0 when the callback is NULL, bypassing the full TLB flush fallback mechanism.
- [Low] The `type` parameter in `kvm_vm_init_vm_s2_ops` is entirely unused.
--

commit 31d5dcbadc8339bf6a74dc7f2dc04dbc627760d7
Author: Suzuki K Poulose <suzuki.poulose@arm.com>

KVM: arm64: Add VM specific callback for S2 MMU operations

Add VM type specific S2 MMU operation backends which can be initialized per VM
flavor, to keep the handling cleaner.

> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 67852acf7a6f8..e78a56fb73696 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
[ ... ]
> @@ -174,26 +190,36 @@ static bool memslot_is_logging(struct kvm_memory_slot *memslot)
[ ... ]
> +int kvm_arch_flush_remote_tlbs_range(struct kvm *kvm,
> +				     gfn_t gfn, u64 nr_pages)
> +{
> +	if (!kvm->arch.vm_s2_ops->vm_flush_remote_tlbs_range)
> +		return 0;

[Severity: High]
Should this return an error code like -EOPNOTSUPP instead of 0?

By returning 0 (success) when the callback is NULL, the core KVM
implementation in kvm_flush_remote_tlbs_range() will assume the flush
was successful and skip the fallback to a full TLB flush
(kvm_flush_remote_tlbs). This regression can lead to stale TLB entries
if a VM flavor explicitly omits the range callback.

> +
> +	return kvm->arch.vm_s2_ops->vm_flush_remote_tlbs_range(kvm, gfn, nr_pages);
> +}
[ ... ]
> @@ -963,6 +996,18 @@ static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
>  	KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
>  }
>  
> +static const struct kvm_vm_s2_ops *arm64_vm_s2_ops[VM_FLAVOR_MAX] = {
> +	[VM_VHE] = &kvm_default_vm_s2_ops,
> +	[VM_NVHE] = &kvm_default_vm_s2_ops,
> +	[VM_PKVM] = &unprotected_pkvm_s2_ops,
> +	[VM_PROTECTED_PKVM] = &protected_pkvm_s2_ops,
> +};

[Severity: Critical]
Is there a mismatch in the struct names for the pKVM operations?

This array uses unprotected_pkvm_s2_ops and protected_pkvm_s2_ops,
but the actual populated structs defined at the end of this file are
named pkvm_vm_s2_ops and protected_vm_s2_ops.

Because of this mismatch, the array points to the zero-initialized
tentative definitions instead of the populated ones, meaning all function
pointers for pKVM guests will be silently NULL. This skips operations like
stage-2 unmapping, which could lead to stale TLBs and unmapped host memory
access.

> +
> +static void kvm_vm_init_vm_s2_ops(struct kvm *kvm, unsigned long type)
> +{
> +	kvm->arch.vm_s2_ops = arm64_vm_s2_ops[kvm->arch.vm_flavor];
> +}

[Severity: Low]
Is the type parameter intentionally ignored here?

The function is passed type from kvm_init_stage2_mmu() but directly queries
kvm->arch.vm_flavor instead, leaving the parameter entirely unused.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260908162223.1683432-1-suzuki.poulose@arm.com?part=19

Re: [PATCH v17 17/20] KVM: arm64: CCA: Provide an accurate register list

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] KVM_GET_REG_LIST fails to restrict core registers to the 9 GP registers for Realm VMs, exposing all core registers instead.
- [Medium] Realm VM system register restrictions are only applied to KVM_GET_REG_LIST, allowing userspace to bypass them via KVM_SET_ONE_REG and KVM_GET_ONE_REG.
--

commit 1a82d37b0064cc3f1fa34c6757432fa355d160a3
Author: Jean-Philippe Brucker <jpb@kernel.org>

KVM: arm64: CCA: Provide an accurate register list
    
Userspace can set a few registers with KVM_SET_ONE_REG (9 GP registers
at runtime, and 3 system registers during initialization). Update the
register list returned by KVM_GET_REG_LIST.

> Userspace can set a few registers with KVM_SET_ONE_REG (9 GP registers
> at runtime, and 3 system registers during initialization). Update the
> register list returned by KVM_GET_REG_LIST.

[Severity: Medium]
Does KVM_GET_REG_LIST successfully restrict core registers to just the 9 GP
registers for Realm VMs?

Looking at copy_core_reg_indices(), it loops over the entire kvm_regs array
without checking vcpu_is_rec(vcpu):

arch/arm64/kvm/guest.c:copy_core_reg_indices() {
	for (i = 0; i < sizeof(struct kvm_regs) / sizeof(__u32); i++) {
		u64 reg = KVM_REG_ARM64 | KVM_REG_ARM_CORE | i;
		int size = core_reg_size_from_offset(vcpu, i);

		if (size < 0)
			continue;
        ...
}

This appears to expose all core registers to the userspace array rather than
limiting it to the 9 GP registers mentioned in the commit message.

[ ... ]

> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 44aae52c473d7..a58c1fc13ea17 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c

[ ... ]

> @@ -5693,11 +5693,26 @@ static bool copy_reg_to_user(const struct sys_reg_desc *reg, u64 __user **uind)
>  	return true;
>  }
>  
> +static inline bool kvm_realm_sys_reg_hidden_user(const struct kvm_vcpu *vcpu,
> +						 u64 reg)
> +{
> +	switch (reg) {
> +	case SYS_ID_AA64DFR0_EL1:
> +	case SYS_PMCR_EL0:
> +		return false;
> +	}
> +	return true;
> +}
> +
>  static int walk_one_sys_reg(const struct kvm_vcpu *vcpu,
>  			    const struct sys_reg_desc *rd,
>  			    u64 __user **uind,
>  			    unsigned int *total)
>  {
> +	if (vcpu_is_rec(vcpu) &&
> +	    kvm_realm_sys_reg_hidden_user(vcpu, reg_to_encoding(rd)))
> +		return 0;
> +

[Severity: Medium]
Are the Realm system register restrictions bypassed by KVM_SET_ONE_REG and
KVM_GET_ONE_REG?

While this patch correctly hides the registers from KVM_GET_REG_LIST by
checking kvm_realm_sys_reg_hidden_user() in walk_one_sys_reg(), it doesn't
appear to add a similar check to the ioctl handlers for setting or getting
individual registers.

For example, in kvm_sys_reg_set_user(), the code only checks sysreg_hidden():

arch/arm64/kvm/sys_regs.c:kvm_sys_reg_set_user() {
    ...
	r = id_to_sys_reg_desc(vcpu, id, table, num);
	if (!r || sysreg_hidden(vcpu, r))
		return -ENOENT;
    ...
}

The getter similarly fails to enforce the Realm system register restrictions.
Does this allow userspace to directly read or overwrite hidden system
registers on a Realm VM if they know the register ID, bypassing the CCA
register protection boundary?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260908162223.1683432-1-suzuki.poulose@arm.com?part=17

Re: [PATCH v17 12/20] KVM: arm64: CCA: Support timers in realm RECs

[Suzuki K Poulose]

On 08/09/2026 17:56, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] Realm VM timers are blackholed on GICv5 hosts due to unconditional physical interrupt masking combined with disabled direct injection.
> --
> 
> commit 759ada4556cf9518a1b30ec3610b7b7833129bd9
> Author: Steven Price <steven.price@arm.com>
> 
> KVM: arm64: CCA: Support timers in realm RECs
> 
> This commit adds support for handling timers when exiting a realm REC to the
> normal world. It implements realm_timer_irq_ops which relies on the existing
> software resampling mechanism for Realm timer interrupts.
> 
>> diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
>> index eba84cab95d01..3eddf44f7b1dd 100644
>> --- a/arch/arm64/kvm/arch_timer.c
>> +++ b/arch/arm64/kvm/arch_timer.c
> [ ... ]
>> @@ -1609,8 +1623,12 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
>>   
>>   	get_timer_map(vcpu, &map);
>>   
>> -	ops = vgic_is_v5(vcpu->kvm) ? &arch_timer_irq_ops_vgic_v5 :
>> -				      &arch_timer_irq_ops;
>> +	if (vcpu_is_rec(vcpu))
>> +		ops = &realm_timer_irq_ops;
>> +	else if (vgic_is_v5(vcpu->kvm))
>> +		ops = &arch_timer_irq_ops_vgic_v5;
>> +	else
>> +		ops = &arch_timer_irq_ops;
>>   
>>   	for (int i = 0; i < nr_timers(vcpu); i++)
>>   		kvm_vgic_set_irq_ops(vcpu, timer_irq(vcpu_get_timer(vcpu, i)), ops);
> 
> [Severity: High]
> Will this cause Realm VM timers to be lost on a GICv5 host?
> 
> When a Realm VM is created on a GICv5 host, vcpu_is_rec(vcpu) takes precedence,
> assigning realm_timer_irq_ops. However, this struct lacks the
> .set_direct_injection callback used for hardware direct injection.
> 
> At the same time, because vgic_is_v5(vcpu->kvm) is still true for the host,
> other timer paths will unconditionally assume direct injection is active. For
> example, kvm_timer_update_irq() skips software injection:
> 
> kvm_timer_update_irq() {
>          ...
>          /* Skip injecting on GICv5 for directly injected (DVI'd) timers */
>          if (vgic_is_v5(vcpu->kvm)) {
>                  ...
>                  if (map.direct_ptimer == timer_ctx ||
>                      map.direct_vtimer == timer_ctx)
>                          return;
>          }
> }
> 
> Additionally, kvm_timer_vcpu_load_gic() unconditionally masks the physical
> timer interrupt for GICv5 setups:
> 
> kvm_timer_vcpu_load_gic() {
>          ...
>          phys_active |= pending;
>          phys_active |= vgic_is_v5(vcpu->kvm);
> 
>          set_timer_irq_phys_active(ctx, phys_active);
> }
> 
> Since hardware injection is not configured by the Realm ops and software
> injection is bypassed due to the GICv5 checks, does this completely blackhole
> timer interrupts for Realm VMs on GICv5 hardware?
> 

CCA doesn't support VGIC_V5. We only support VGIC_V3. I have added the 
following patch to make sure this is mandated:



---8>---

KVM: arm64: CCA: Mandate VGIC_V3 for Realms

Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>

diff --git a/arch/arm64/kvm/vgic/vgic-init.c 
b/arch/arm64/kvm/vgic/vgic-init.c
index 4012df6002ea6..452c085073f3e 100644
--- a/arch/arm64/kvm/vgic/vgic-init.c
+++ b/arch/arm64/kvm/vgic/vgic-init.c
@@ -84,6 +84,9 @@ int kvm_vgic_create(struct kvm *kvm, u32 type)
                 !kvm_vgic_global_state.can_emulate_gicv2)
                 return -ENODEV;

+       /* Realms only support VGIC_V3 */
+       if (kvm_vm_is_realm(kvm) && type != KVM_DEV_TYPE_ARM_VGIC_V3)
+               return -ENODEV;
         /*
          * Ensure mutual exclusion with vCPU creation and any vCPU 
ioctls by:

Suzuki

Re: [PATCH v17 19/20] KVM: arm64: Add VM specific callback for S2 MMU operations

[Suzuki K Poulose]

On 08/09/2026 17:59, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
> - [Critical] Mismatch between forward declarations and definitions of pKVM S2 MMU operations structs causes KVM to silently skip critical memory management and TLB flush operations.
> - [High] `kvm_arch_flush_remote_tlbs_range` incorrectly returns 0 when the callback is NULL, bypassing the full TLB flush fallback mechanism.
> - [Low] The `type` parameter in `kvm_vm_init_vm_s2_ops` is entirely unused.
> --
> 
> commit 31d5dcbadc8339bf6a74dc7f2dc04dbc627760d7
> Author: Suzuki K Poulose <suzuki.poulose@arm.com>
> 
> KVM: arm64: Add VM specific callback for S2 MMU operations
> 
> Add VM type specific S2 MMU operation backends which can be initialized per VM
> flavor, to keep the handling cleaner.
> 
>> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
>> index 67852acf7a6f8..e78a56fb73696 100644
>> --- a/arch/arm64/kvm/mmu.c
>> +++ b/arch/arm64/kvm/mmu.c
> [ ... ]
>> @@ -174,26 +190,36 @@ static bool memslot_is_logging(struct kvm_memory_slot *memslot)
> [ ... ]
>> +int kvm_arch_flush_remote_tlbs_range(struct kvm *kvm,
>> +				     gfn_t gfn, u64 nr_pages)
>> +{
>> +	if (!kvm->arch.vm_s2_ops->vm_flush_remote_tlbs_range)
>> +		return 0;
> 
> [Severity: High]
> Should this return an error code like -EOPNOTSUPP instead of 0?
> 
> By returning 0 (success) when the callback is NULL, the core KVM
> implementation in kvm_flush_remote_tlbs_range() will assume the flush
> was successful and skip the fallback to a full TLB flush
> (kvm_flush_remote_tlbs). This regression can lead to stale TLB entries
> if a VM flavor explicitly omits the range callback.
> 
>> +
>> +	return kvm->arch.vm_s2_ops->vm_flush_remote_tlbs_range(kvm, gfn, nr_pages);
>> +}
> [ ... ]
>> @@ -963,6 +996,18 @@ static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
>>   	KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
>>   }
>>   
>> +static const struct kvm_vm_s2_ops *arm64_vm_s2_ops[VM_FLAVOR_MAX] = {
>> +	[VM_VHE] = &kvm_default_vm_s2_ops,
>> +	[VM_NVHE] = &kvm_default_vm_s2_ops,
>> +	[VM_PKVM] = &unprotected_pkvm_s2_ops,
>> +	[VM_PROTECTED_PKVM] = &protected_pkvm_s2_ops,
>> +};
> 
> [Severity: Critical]
> Is there a mismatch in the struct names for the pKVM operations?
> 
> This array uses unprotected_pkvm_s2_ops and protected_pkvm_s2_ops,
> but the actual populated structs defined at the end of this file are
> named pkvm_vm_s2_ops and protected_vm_s2_ops.
> 
> Because of this mismatch, the array points to the zero-initialized
> tentative definitions instead of the populated ones, meaning all function
> pointers for pKVM guests will be silently NULL. This skips operations like
> stage-2 unmapping, which could lead to stale TLBs and unmapped host memory
> access.
> 

Thanks, that is indeed a terrible mistake from my end. I have now fixed 
this up cleanly.

>> +
>> +static void kvm_vm_init_vm_s2_ops(struct kvm *kvm, unsigned long type)
>> +{
>> +	kvm->arch.vm_s2_ops = arm64_vm_s2_ops[kvm->arch.vm_flavor];
>> +}
> 
> [Severity: Low]
> Is the type parameter intentionally ignored here?

I will drop this, left over from the rebase :-(

> 
> The function is passed type from kvm_init_stage2_mmu() but directly queries
> kvm->arch.vm_flavor instead, leaving the parameter entirely unused.
> 

Cheers
Suzuki

Re: [PATCH v17 06/20] KVM: arm64: CCA: Add a new mode for supporting Realm guests

[Kohei Enju]

Hi Suzuki,

On 09/08 17:22, Suzuki K Poulose wrote:
> Add an explicit mode to support Arm CCA guests.
> 
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> ---
>  Documentation/admin-guide/kernel-parameters.txt | 3 +++
>  arch/arm64/include/asm/kvm_host.h               | 1 +
>  arch/arm64/kvm/arm.c                            | 5 +++++
>  3 files changed, 9 insertions(+)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 68647ff4bdd24..1afe3df3b923e 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -3256,6 +3256,9 @@ Kernel parameters
>  			nested: VHE-based mode with support for nested
>  				virtualization. Requires at least ARMv8.4
>  				hardware (with FEAT_NV2).
> +			rmm: Support for running confidential guests in Realm
> +			     world using RMM, as defined by Arm Confidential
> +			     Compute Architecture (CCA)
>  
>  			Defaults to VHE/nVHE based on hardware support. Setting
>  			mode to "protected" will disable kexec and hibernation
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index b2e99c5cb1cd3..31ae9d8d8e92f 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -69,6 +69,7 @@ enum kvm_mode {
>  	KVM_MODE_DEFAULT,
>  	KVM_MODE_PROTECTED,
>  	KVM_MODE_NV,
> +	KVM_MODE_RMM,
>  	KVM_MODE_NONE,
>  };
>  #ifdef CONFIG_KVM
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 9af3bbb2f8c24..50f0adfadab38 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -3264,6 +3264,11 @@ static int __init early_kvm_mode_cfg(char *arg)
>  		return 0;
>  	}
>  
> +	if (strcmp(arg, "rmm") == 0 && !WARN_ON(!is_kernel_in_hyp_mode())) {
> +		kvm_mode = KVM_MODE_RMM;
> +		return 0;
> +	}

Does this make NV support and CCA Realm VM support mutually exclusive?

Until the v16 series, we could run Realm VMs with:
  'kvm-arm.mode=nested'

As far as the AArch64 architecture is concerned, there does not seem to
be a reason to make them mutually exclusive. I am therefore wondering
whether this restriction is intentional.

Thanks,
Kohei

> +
>  	return -EINVAL;
>  }
>  early_param("kvm-arm.mode", early_kvm_mode_cfg);
> -- 
> 2.43.0
> 

Re: [PATCH v17 06/20] KVM: arm64: CCA: Add a new mode for supporting Realm guests

[Marc Zyngier]

On Wed, 09 Sep 2026 04:26:41 +0100,
Kohei Enju <enju.kohei@fujitsu.com> wrote:
> 
> Hi Suzuki,
> 
> On 09/08 17:22, Suzuki K Poulose wrote:
> > Add an explicit mode to support Arm CCA guests.
> > 
> > Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> > ---
> >  Documentation/admin-guide/kernel-parameters.txt | 3 +++
> >  arch/arm64/include/asm/kvm_host.h               | 1 +
> >  arch/arm64/kvm/arm.c                            | 5 +++++
> >  3 files changed, 9 insertions(+)
> > 
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 68647ff4bdd24..1afe3df3b923e 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -3256,6 +3256,9 @@ Kernel parameters
> >  			nested: VHE-based mode with support for nested
> >  				virtualization. Requires at least ARMv8.4
> >  				hardware (with FEAT_NV2).
> > +			rmm: Support for running confidential guests in Realm
> > +			     world using RMM, as defined by Arm Confidential
> > +			     Compute Architecture (CCA)
> >  
> >  			Defaults to VHE/nVHE based on hardware support. Setting
> >  			mode to "protected" will disable kexec and hibernation
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index b2e99c5cb1cd3..31ae9d8d8e92f 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -69,6 +69,7 @@ enum kvm_mode {
> >  	KVM_MODE_DEFAULT,
> >  	KVM_MODE_PROTECTED,
> >  	KVM_MODE_NV,
> > +	KVM_MODE_RMM,
> >  	KVM_MODE_NONE,
> >  };
> >  #ifdef CONFIG_KVM
> > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> > index 9af3bbb2f8c24..50f0adfadab38 100644
> > --- a/arch/arm64/kvm/arm.c
> > +++ b/arch/arm64/kvm/arm.c
> > @@ -3264,6 +3264,11 @@ static int __init early_kvm_mode_cfg(char *arg)
> >  		return 0;
> >  	}
> >  
> > +	if (strcmp(arg, "rmm") == 0 && !WARN_ON(!is_kernel_in_hyp_mode())) {
> > +		kvm_mode = KVM_MODE_RMM;
> > +		return 0;
> > +	}
> 
> Does this make NV support and CCA Realm VM support mutually exclusive?
> 
> Until the v16 series, we could run Realm VMs with:
>   'kvm-arm.mode=nested'
> 
> As far as the AArch64 architecture is concerned, there does not seem to
> be a reason to make them mutually exclusive. I am therefore wondering
> whether this restriction is intentional.

I have a hack somewhere that will allow this, but this requires to
change the above to be a capability mask rather than a simple
enumeration. I'll see if I can spin that as a separate series.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH v17 01/20] KVM: arm64: Include kvm_emulate.h in kvm/arm_psci.h

[Fuad Tabba]

On Tue, 8 Sept 2026 at 17:22, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>
> Fix a potential build error (like below, when asm/kvm_emulate.h gets
> included after the kvm/arm_psci.h) by including the missing header file
> in kvm/arm_psci.h:
>
> ./include/kvm/arm_psci.h: In function ‘kvm_psci_version’:
> ./include/kvm/arm_psci.h:29:13: error: implicit declaration of function
>    ‘vcpu_has_feature’; did you mean ‘cpu_have_feature’? [-Werror=implicit-function-declaration]
>    29 |         if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_PSCI_0_2)) {
>                  |             ^~~~~~~~~~~~~~~~
>                                |             cpu_have_feature
>
> Reviewed-by: Gavin Shan <gshan@redhat.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> Signed-off-by: Steven Price <steven.price@arm.com>

Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Cheers,
/fuad
> ---
>  include/kvm/arm_psci.h | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h
> index f86a006d67136..06c20612e9e7d 100644
> --- a/include/kvm/arm_psci.h
> +++ b/include/kvm/arm_psci.h
> @@ -10,6 +10,8 @@
>  #include <linux/kvm_host.h>
>  #include <uapi/linux/psci.h>
>
> +#include <asm/kvm_emulate.h>
> +
>  #define KVM_ARM_PSCI_0_1       PSCI_VERSION(0, 1)
>  #define KVM_ARM_PSCI_0_2       PSCI_VERSION(0, 2)
>  #define KVM_ARM_PSCI_1_0       PSCI_VERSION(1, 0)
> --
> 2.43.0
>

Re: [PATCH v17 02/20] KVM: arm64: Avoid including linux/kvm_host.h in kvm_pgtable.h

[Fuad Tabba]

On Tue, 8 Sept 2026 at 17:22, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>
> From: Steven Price <steven.price@arm.com>
>
> To avoid future include cycles, drop the linux/kvm_host.h include in
> kvm_pgtable.h and include the lightweight headers required for the types
> and inline helpers used there. Additionally provide a forward
> declaration for struct kvm_s2_mmu as it's only used as a pointer in this
> file.
>
> Both pgtable.c and kvm_pkvm.h relied on the indirect inclusion of
> kvm_host.h, so make that explicit.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> New patch in v13

Is this meant to be here (the "New in patch 13")?

Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Cheers,
/fuad


> ---
>  arch/arm64/include/asm/kvm_pgtable.h | 6 +++++-
>  arch/arm64/include/asm/kvm_pkvm.h    | 2 +-
>  arch/arm64/kvm/hyp/pgtable.c         | 1 +
>  3 files changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h
> index 41a8687938eb6..c2e4b29e605fc 100644
> --- a/arch/arm64/include/asm/kvm_pgtable.h
> +++ b/arch/arm64/include/asm/kvm_pgtable.h
> @@ -8,9 +8,13 @@
>  #define __ARM64_KVM_PGTABLE_H__
>
>  #include <linux/bits.h>
> -#include <linux/kvm_host.h>
> +#include <linux/kvm_types.h>
> +#include <linux/rbtree_types.h>
> +#include <linux/rcupdate.h>
>  #include <linux/types.h>
>
> +struct kvm_s2_mmu;
> +
>  #define KVM_PGTABLE_FIRST_LEVEL                -1
>  #define KVM_PGTABLE_LAST_LEVEL         3
>
> diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
> index beea00e693a0a..54a618d887fa4 100644
> --- a/arch/arm64/include/asm/kvm_pkvm.h
> +++ b/arch/arm64/include/asm/kvm_pkvm.h
> @@ -7,9 +7,9 @@
>  #define __ARM64_KVM_PKVM_H__
>
>  #include <linux/arm_ffa.h>
> +#include <linux/kvm_host.h>
>  #include <linux/memblock.h>
>  #include <linux/scatterlist.h>
> -#include <asm/kvm_host.h>
>  #include <asm/kvm_pgtable.h>
>
>  /* Maximum number of VMs that can co-exist under pKVM. */
> diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
> index b74dd5ce1efd3..f48253b9d88b5 100644
> --- a/arch/arm64/kvm/hyp/pgtable.c
> +++ b/arch/arm64/kvm/hyp/pgtable.c
> @@ -8,6 +8,7 @@
>   */
>
>  #include <linux/bitfield.h>
> +#include <linux/kvm_host.h>
>  #include <asm/kvm_pgtable.h>
>  #include <asm/stage2_pgtable.h>
>
> --
> 2.43.0
>

Re: [PATCH v17 02/20] KVM: arm64: Avoid including linux/kvm_host.h in kvm_pgtable.h

[Suzuki K Poulose]

On 09/09/2026 12:22, Fuad Tabba wrote:
> On Tue, 8 Sept 2026 at 17:22, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>>
>> From: Steven Price <steven.price@arm.com>
>>
>> To avoid future include cycles, drop the linux/kvm_host.h include in
>> kvm_pgtable.h and include the lightweight headers required for the types
>> and inline helpers used there. Additionally provide a forward
>> declaration for struct kvm_s2_mmu as it's only used as a pointer in this
>> file.
>>
>> Both pgtable.c and kvm_pkvm.h relied on the indirect inclusion of
>> kvm_host.h, so make that explicit.
>>
>> Signed-off-by: Steven Price <steven.price@arm.com>
>> ---
>> New patch in v13
> 
> Is this meant to be here (the "New in patch 13")?

Just trying to carry the legacy, but yes, you're right. I could remove
it. v13, is so ancient now :-)


> 
> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Cheers
Suzuki


> 
> Cheers,
> /fuad
> 
> 
>> ---
>>   arch/arm64/include/asm/kvm_pgtable.h | 6 +++++-
>>   arch/arm64/include/asm/kvm_pkvm.h    | 2 +-
>>   arch/arm64/kvm/hyp/pgtable.c         | 1 +
>>   3 files changed, 7 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h
>> index 41a8687938eb6..c2e4b29e605fc 100644
>> --- a/arch/arm64/include/asm/kvm_pgtable.h
>> +++ b/arch/arm64/include/asm/kvm_pgtable.h
>> @@ -8,9 +8,13 @@
>>   #define __ARM64_KVM_PGTABLE_H__
>>
>>   #include <linux/bits.h>
>> -#include <linux/kvm_host.h>
>> +#include <linux/kvm_types.h>
>> +#include <linux/rbtree_types.h>
>> +#include <linux/rcupdate.h>
>>   #include <linux/types.h>
>>
>> +struct kvm_s2_mmu;
>> +
>>   #define KVM_PGTABLE_FIRST_LEVEL                -1
>>   #define KVM_PGTABLE_LAST_LEVEL         3
>>
>> diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
>> index beea00e693a0a..54a618d887fa4 100644
>> --- a/arch/arm64/include/asm/kvm_pkvm.h
>> +++ b/arch/arm64/include/asm/kvm_pkvm.h
>> @@ -7,9 +7,9 @@
>>   #define __ARM64_KVM_PKVM_H__
>>
>>   #include <linux/arm_ffa.h>
>> +#include <linux/kvm_host.h>
>>   #include <linux/memblock.h>
>>   #include <linux/scatterlist.h>
>> -#include <asm/kvm_host.h>
>>   #include <asm/kvm_pgtable.h>
>>
>>   /* Maximum number of VMs that can co-exist under pKVM. */
>> diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
>> index b74dd5ce1efd3..f48253b9d88b5 100644
>> --- a/arch/arm64/kvm/hyp/pgtable.c
>> +++ b/arch/arm64/kvm/hyp/pgtable.c
>> @@ -8,6 +8,7 @@
>>    */
>>
>>   #include <linux/bitfield.h>
>> +#include <linux/kvm_host.h>
>>   #include <asm/kvm_pgtable.h>
>>   #include <asm/stage2_pgtable.h>
>>
>> --
>> 2.43.0
>>

Re: [PATCH v17 03/20] KVM: arm64: Track the type of VM in kvm_arch

[Fuad Tabba]

Hi Suzuki,

On Tue, 8 Sept 2026 at 17:23, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>
> KVM arm64 has different types of VMs with all the different modes in which the
> hypervisor code can be run. e.g., VHE, nVHE, PKVM etc. Then there is protected
> VM and normal VMs with PKVM. We might soon add other types, e.g., Arm CCA Realm.
> So in an effort to make the handling of these different types of VMs a bit more
> friendlier to the eyes, add a VM flavor to the kvm_arch and we could then add

nit: "friendlier"

> handlers for different operations based on the VM type.
>
> Keep the flavor initialisation at the beginning to allow for the detection
> early enough and fail out on any unsupported requests. (e.g., protected on !PKVM)

In the commit msg and later in the code (not in the enum though), can
you please spell pKVM with a lower-case p?

Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Cheers,
/fuad

Re: [PATCH v17 03/20] KVM: arm64: Track the type of VM in kvm_arch

[Suzuki K Poulose]

On 09/09/2026 12:28, Fuad Tabba wrote:
> Hi Suzuki,
> 
> On Tue, 8 Sept 2026 at 17:23, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>>
>> KVM arm64 has different types of VMs with all the different modes in which the
>> hypervisor code can be run. e.g., VHE, nVHE, PKVM etc. Then there is protected
>> VM and normal VMs with PKVM. We might soon add other types, e.g., Arm CCA Realm.
>> So in an effort to make the handling of these different types of VMs a bit more
>> friendlier to the eyes, add a VM flavor to the kvm_arch and we could then add
> 
> nit: "friendlier"
> 
>> handlers for different operations based on the VM type.
>>
>> Keep the flavor initialisation at the beginning to allow for the detection
>> early enough and fail out on any unsupported requests. (e.g., protected on !PKVM)
> 
> In the commit msg and later in the code (not in the enum though), can
> you please spell pKVM with a lower-case p?

Ack.

> 
> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Cheers
Suzuki

Re: [PATCH v17 10/20] KVM: arm64: coco: Disable Steal time accounting for coco guests

[Fuad Tabba]

Hi Suzuki,

On Tue, 8 Sept 2026 at 17:23, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>
> PVTIME support is advertised by KVM_CAP_STEAL_TIME, which doesn't take into
> account the kvm instance. Even with that, a VMM could skip the CAP check and
> proceed to configure the PVTIME as we don't do further check on the DEVICE_CTRL.
> Tighten this up by passing the KVM instance around wherever possible and
> catch things early
>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>

...

> -bool kvm_arm_pvtime_supported(void)
> +bool kvm_arm_pvtime_supported(struct kvm *kvm)
>  {
> -       return !!sched_info_on();
> +       return !!sched_info_on() && (!kvm || !kvm_vm_is_confidential(kvm));
>  }

I think it would be better to leave KVM_CAP_STEAL_TIME reported and
instead reject the configuration for these VMs, returning -EPERM from
the KVM_ARM_VCPU_PVTIME_CTRL attr. A capability query should report
what the kernel supports, not whether a given VM may use it; gating it
per-VM here is also inconsistent, since the !kvm clause leaves the
system-fd cap at 1 while the VM-fd cap reads 0. And -EPERM says that
steal-time exists but isn't permitted for a confidential VM, whereas
-ENXIO reads as "no such feature".

Cheers,
/fuad

>
>  int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
> @@ -81,7 +81,7 @@ int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
>         int ret = 0;
>         int idx;
>
> -       if (!kvm_arm_pvtime_supported() ||
> +       if (!kvm_arm_pvtime_supported(kvm) ||
>             attr->attr != KVM_ARM_VCPU_PVTIME_IPA)
>                 return -ENXIO;
>
> @@ -110,7 +110,7 @@ int kvm_arm_pvtime_get_attr(struct kvm_vcpu *vcpu,
>         u64 __user *user = (u64 __user *)attr->addr;
>         u64 ipa;
>
> -       if (!kvm_arm_pvtime_supported() ||
> +       if (!kvm_arm_pvtime_supported(vcpu->kvm) ||
>             attr->attr != KVM_ARM_VCPU_PVTIME_IPA)
>                 return -ENXIO;
>
> @@ -126,7 +126,7 @@ int kvm_arm_pvtime_has_attr(struct kvm_vcpu *vcpu,
>  {
>         switch (attr->attr) {
>         case KVM_ARM_VCPU_PVTIME_IPA:
> -               if (kvm_arm_pvtime_supported())
> +               if (kvm_arm_pvtime_supported(vcpu->kvm))
>                         return 0;
>         }
>         return -ENXIO;
> --
> 2.43.0
>

Re: [PATCH v17 18/20] KVM: arm64: Reuse kvm_stage2_unmap_range in kvm_unmap_gfn_range

[Fuad Tabba]

On Tue, 8 Sept 2026 at 17:23, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>
> In preparation for adding VM specific backends for stage2 operation,
> switch to kvm_stage2_unmap_range() instead of __unmap_stage2_range()
> from the kvm_unmap_gfn_range(). Drop the bail out check for protected
> VMs and defer that to the one in kvm_stage2_unmap_range(). Later we
> would replace the logic in kvm_stage2_unmap_range() with VM specific
> backends.
>
> No functional changes intended.
>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>

Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Cheers,
/fuad


> ---
>  arch/arm64/kvm/mmu.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 9ba86450fe4af..67852acf7a6f8 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -2436,12 +2436,12 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
>
>  bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range)
>  {
> -       if (!kvm->arch.mmu.pgt || kvm_vm_is_protected(kvm))
> +       if (!kvm->arch.mmu.pgt)
>                 return false;
>
> -       __unmap_stage2_range(&kvm->arch.mmu, range->start << PAGE_SHIFT,
> -                            (range->end - range->start) << PAGE_SHIFT,
> -                            range->may_block);
> +       kvm_stage2_unmap_range(&kvm->arch.mmu, range->start << PAGE_SHIFT,
> +                               (range->end - range->start) << PAGE_SHIFT,
> +                               range->may_block);
>
>         kvm_nested_s2_unmap(kvm, range->may_block);
>         return false;
> --
> 2.43.0
>

Re: [PATCH v17 10/20] KVM: arm64: coco: Disable Steal time accounting for coco guests

[Suzuki K Poulose]

On 09/09/2026 12:45, Fuad Tabba wrote:
> Hi Suzuki,
> 
> On Tue, 8 Sept 2026 at 17:23, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>>
>> PVTIME support is advertised by KVM_CAP_STEAL_TIME, which doesn't take into
>> account the kvm instance. Even with that, a VMM could skip the CAP check and
>> proceed to configure the PVTIME as we don't do further check on the DEVICE_CTRL.
>> Tighten this up by passing the KVM instance around wherever possible and
>> catch things early
>>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> 
> ...
> 
>> -bool kvm_arm_pvtime_supported(void)
>> +bool kvm_arm_pvtime_supported(struct kvm *kvm)
>>   {
>> -       return !!sched_info_on();
>> +       return !!sched_info_on() && (!kvm || !kvm_vm_is_confidential(kvm));
>>   }
> 
> I think it would be better to leave KVM_CAP_STEAL_TIME reported and
> instead reject the configuration for these VMs, returning -EPERM from
> the KVM_ARM_VCPU_PVTIME_CTRL attr. A capability query should report
> what the kernel supports, not whether a given VM may use it; gating it
> per-VM here is also inconsistent, since the !kvm clause leaves the
> system-fd cap at 1 while the VM-fd cap reads 0. And -EPERM says that


But isn't that we want from a VMM perspective ? i.e., enable PVTIME if
it is supported for the given VM (type). Rather than PVTIME is supported
by the KVM, so I can go enable it by default. But, we endup getting a
failure while we enable it. I encountered this with kvmtool, where
CAP_STEAL_TIME is reported true and even the HAS_DEVICE_ATTR goes
through fine, but SET_DEVICE_ATTR seemed to fail.

Isn't it better to allow the VM to decide if the capability is supported
for the given VM type, like we are doing for other capabilities ?

> steal-time exists but isn't permitted for a confidential VM, whereas
> -ENXIO reads as "no such feature".

Cheers
Suzuki


> 
> Cheers,
> /fuad
> 
>>
>>   int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
>> @@ -81,7 +81,7 @@ int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
>>          int ret = 0;
>>          int idx;
>>
>> -       if (!kvm_arm_pvtime_supported() ||
>> +       if (!kvm_arm_pvtime_supported(kvm) ||
>>              attr->attr != KVM_ARM_VCPU_PVTIME_IPA)
>>                  return -ENXIO;
>>
>> @@ -110,7 +110,7 @@ int kvm_arm_pvtime_get_attr(struct kvm_vcpu *vcpu,
>>          u64 __user *user = (u64 __user *)attr->addr;
>>          u64 ipa;
>>
>> -       if (!kvm_arm_pvtime_supported() ||
>> +       if (!kvm_arm_pvtime_supported(vcpu->kvm) ||
>>              attr->attr != KVM_ARM_VCPU_PVTIME_IPA)
>>                  return -ENXIO;
>>
>> @@ -126,7 +126,7 @@ int kvm_arm_pvtime_has_attr(struct kvm_vcpu *vcpu,
>>   {
>>          switch (attr->attr) {
>>          case KVM_ARM_VCPU_PVTIME_IPA:
>> -               if (kvm_arm_pvtime_supported())
>> +               if (kvm_arm_pvtime_supported(vcpu->kvm))
>>                          return 0;
>>          }
>>          return -ENXIO;
>> --
>> 2.43.0
>>

Re: [PATCH v17 10/20] KVM: arm64: coco: Disable Steal time accounting for coco guests

[Fuad Tabba]

Hi Suzuki,

On Wed, 9 Sept 2026 at 12:52, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
...

> > I think it would be better to leave KVM_CAP_STEAL_TIME reported and
> > instead reject the configuration for these VMs, returning -EPERM from
> > the KVM_ARM_VCPU_PVTIME_CTRL attr. A capability query should report
> > what the kernel supports, not whether a given VM may use it; gating it
> > per-VM here is also inconsistent, since the !kvm clause leaves the
> > system-fd cap at 1 while the VM-fd cap reads 0. And -EPERM says that
>
>
> But isn't that we want from a VMM perspective ? i.e., enable PVTIME if
> it is supported for the given VM (type). Rather than PVTIME is supported
> by the KVM, so I can go enable it by default. But, we endup getting a
> failure while we enable it. I encountered this with kvmtool, where
> CAP_STEAL_TIME is reported true and even the HAS_DEVICE_ATTR goes
> through fine, but SET_DEVICE_ATTR seemed to fail.
>
> Isn't it better to allow the VM to decide if the capability is supported
> for the given VM type, like we are doing for other capabilities ?

You're right, per-VM capability reporting. My wires got crossed from
the pKVM series and the kvmtool fixes. However, I'd still return
-EPERM rather than -ENXIO from the attribute. The capability exists,
it's just not permitted for a confidential VM, and -EPERM says exactly
that, whereas -ENXIO reads as "no such feature".

Cheers,
/fuad

>
> > steal-time exists but isn't permitted for a confidential VM, whereas
> > -ENXIO reads as "no such feature".
>
> Cheers
> Suzuki
>
>
> >
> > Cheers,
> > /fuad
> >
> >>
> >>   int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
> >> @@ -81,7 +81,7 @@ int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
> >>          int ret = 0;
> >>          int idx;
> >>
> >> -       if (!kvm_arm_pvtime_supported() ||
> >> +       if (!kvm_arm_pvtime_supported(kvm) ||
> >>              attr->attr != KVM_ARM_VCPU_PVTIME_IPA)
> >>                  return -ENXIO;
> >>
> >> @@ -110,7 +110,7 @@ int kvm_arm_pvtime_get_attr(struct kvm_vcpu *vcpu,
> >>          u64 __user *user = (u64 __user *)attr->addr;
> >>          u64 ipa;
> >>
> >> -       if (!kvm_arm_pvtime_supported() ||
> >> +       if (!kvm_arm_pvtime_supported(vcpu->kvm) ||
> >>              attr->attr != KVM_ARM_VCPU_PVTIME_IPA)
> >>                  return -ENXIO;
> >>
> >> @@ -126,7 +126,7 @@ int kvm_arm_pvtime_has_attr(struct kvm_vcpu *vcpu,
> >>   {
> >>          switch (attr->attr) {
> >>          case KVM_ARM_VCPU_PVTIME_IPA:
> >> -               if (kvm_arm_pvtime_supported())
> >> +               if (kvm_arm_pvtime_supported(vcpu->kvm))
> >>                          return 0;
> >>          }
> >>          return -ENXIO;
> >> --
> >> 2.43.0
> >>
>

Re: [PATCH v17 00/20] KVM: arm64: CCA: Add basic plumbing for Realms

[Fuad Tabba]

Hi Suzuki,

On Tue, 8 Sept 2026 at 17:22, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
...
>  3) NEW: Enlighten KVM arm64 about the different VM types and use call backs for
>    the VM type, rather than spilling the is_this_type_of_vm() everywhere. This is
>    not complete yet, but it is largely functional. Adds VCPU and Stage2 MMU related
>    callbacks with support for the existing VM types.

A design thought on the vm_flavor dispatch tables. They're
hand-maintained arrays indexed by enum, and a gap fails silently at
runtime, not at build: arm64_vm_s2_ops[] has no VM_REALM row, so a
realm VM would get NULL ops.

How about something like declaring it unsized with
BUILD_BUG_ON(ARRAY_SIZE(arm64_vm_s2_ops) != VM_FLAVOR_MAX), as
host_hcall does? A non-NULL check of the selected ops' mandatory
callbacks at VM init would catch the s2_ops naming typo Sashiko
caught, where the rows point at zeroed forward-declared structs.

Cheers,
/fuad

Re: [PATCH v17 00/20] KVM: arm64: CCA: Add basic plumbing for Realms

[Suzuki K Poulose]

On 09/09/2026 14:18, Fuad Tabba wrote:
> Hi Suzuki,
> 
> On Tue, 8 Sept 2026 at 17:22, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
> ...
>>   3) NEW: Enlighten KVM arm64 about the different VM types and use call backs for
>>     the VM type, rather than spilling the is_this_type_of_vm() everywhere. This is
>>     not complete yet, but it is largely functional. Adds VCPU and Stage2 MMU related
>>     callbacks with support for the existing VM types.
> 
> A design thought on the vm_flavor dispatch tables. They're
> hand-maintained arrays indexed by enum, and a gap fails silently at
> runtime, not at build: arm64_vm_s2_ops[] has no VM_REALM row, so a
> realm VM would get NULL ops.
> 
> How about something like declaring it unsized with
> BUILD_BUG_ON(ARRAY_SIZE(arm64_vm_s2_ops) != VM_FLAVOR_MAX), as
> host_hcall does? A non-NULL check of the selected ops' mandatory
> callbacks at VM init would catch the s2_ops naming typo Sashiko
> caught, where the rows point at zeroed forward-declared structs.

Agreed, I will add that. The VM_REALM S2 ops will come in the next part,
as we can't have anything meaningful without the RMI backends.

Thanks for the review !

Cheers
Suzuki

Re: [PATCH v17 03/20] KVM: arm64: Track the type of VM in kvm_arch

[Gavin Shan]

Hi Suzuki,

On 9/9/26 2:22 AM, Suzuki K Poulose wrote:
> KVM arm64 has different types of VMs with all the different modes in which the
> hypervisor code can be run. e.g., VHE, nVHE, PKVM etc. Then there is protected
> VM and normal VMs with PKVM. We might soon add other types, e.g., Arm CCA Realm.
> So in an effort to make the handling of these different types of VMs a bit more
> friendlier to the eyes, add a VM flavor to the kvm_arch and we could then add
> handlers for different operations based on the VM type.
> 
> Keep the flavor initialisation at the beginning to allow for the detection
> early enough and fail out on any unsupported requests. (e.g., protected on !PKVM)
> 
> With that, use the vm_flavor to detect if a VM is protected VM on PKVM.
> 
> Based on a patch by Marc Zyngier
> 
> Suggested-by: Marc Zyngier <maz@kernel.org>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> ---
>   arch/arm64/include/asm/kvm_host.h | 12 ++++++++++--
>   arch/arm64/kvm/arm.c              | 27 ++++++++++++++++++++++++---
>   arch/arm64/kvm/hyp/nvhe/pkvm.c    |  2 +-
>   arch/arm64/kvm/pkvm.c             |  1 -
>   4 files changed, 35 insertions(+), 7 deletions(-)
> 

Some nitpicks below. In either way:

Reviewed-by: Gavin Shan <gshan@redhat.com>

> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 27fe0cd5b2d7a..d0dccc9ad6aa8 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -257,7 +257,6 @@ struct kvm_protected_vm {
>   	pkvm_handle_t handle;
>   	struct kvm_hyp_memcache teardown_mc;
>   	struct kvm_hyp_memcache stage2_teardown_mc;
> -	bool is_protected;
>   	bool is_created;
>   
>   	/*
> @@ -306,9 +305,18 @@ enum fgt_group_id {
>   	__NR_FGT_GROUP_IDS__
>   };
>   
> +enum kvm_arm_vm_flavor {
> +	VM_NVHE,
> +	VM_VHE,
> +	VM_PKVM,		/* Normal guests on PKVM */
> +	VM_PROTECTED_PKVM,	/* Protected VM */
> +	VM_FLAVOR_MAX,
> +};
> +
>   struct kvm_arch {
>   	struct kvm_s2_mmu mmu;
>   
> +	enum kvm_arm_vm_flavor vm_flavor;
>   	/*
>   	 * Fine-Grained UNDEF, mimicking the FGT layout defined by the
>   	 * architecture. We track them globally, as we present the
> @@ -1504,7 +1512,7 @@ struct kvm *kvm_arch_alloc_vm(void);
>   
>   #define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS_RANGE
>   
> -#define kvm_vm_is_protected(kvm)	(is_protected_kvm_enabled() && (kvm)->arch.pkvm.is_protected)
> +#define kvm_vm_is_protected(kvm)	((kvm)->arch.vm_flavor == VM_PROTECTED_PKVM)
>   

Here we may introduce one more helper for generic use cases. It's going to
be used in kvm_arch_init_vm().

#define kvm_vm_flavor_eq(kvm, flavor)	((kvm)->arch.vm_flaor == (flavor))
#define kvm_vm_is_protected(kvm)	kvm_vm_flavor_eq(kvm, VM_PROTECTED_PKVM)

>   #define vcpu_is_protected(vcpu)		kvm_vm_is_protected((vcpu)->kvm)
>   
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 8b080804bc90b..67f1ff9bc4fbe 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -214,6 +214,26 @@ static int kvm_arm_default_max_vcpus(void)
>   	return vgic_present ? kvm_vgic_get_max_vcpus() : KVM_MAX_VCPUS;
>   }
>   
> +static int kvm_init_vm_flavor(struct kvm *kvm, unsigned long type)
> +{
> +	bool protected = type & KVM_VM_TYPE_ARM_PROTECTED;
> +
> +	if (is_protected_kvm_enabled()) {
> +		if (protected)
> +			kvm->arch.vm_flavor = VM_PROTECTED_PKVM;
> +		else
> +			kvm->arch.vm_flavor = VM_PKVM;
> +	} else if (protected) {
> +		return -EINVAL;
> +	} else if (has_vhe()) {
> +		kvm->arch.vm_flavor = VM_VHE;
> +	} else {
> +		kvm->arch.vm_flavor = VM_NVHE;
> +	}
> +
> +	return 0;
> +}
> +
>   /**
>    * kvm_arch_init_vm - initializes a VM data structure
>    * @kvm:	pointer to the KVM struct
> @@ -236,6 +256,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
>   	mutex_unlock(&kvm->lock);
>   #endif
>   
> +	ret = kvm_init_vm_flavor(kvm, type);
> +	if (ret)
> +		return ret;
> +
>   	kvm_init_nested(kvm);
>   
>   	ret = kvm_share_hyp(kvm, kvm + 1);
> @@ -260,9 +284,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
>   		ret = pkvm_init_host_vm(kvm, type);
>   		if (ret)
>   			goto err_uninit_mmu;
> -	} else if (type & KVM_VM_TYPE_ARM_PROTECTED) {
> -		ret = -EINVAL;
> -		goto err_uninit_mmu;
>   	}
>   
>   	kvm_vgic_early_init(kvm);

We needn't depend on is_protected_kvm_enabled() to call pkvm_init_host_vm() because
the output of is_protected_kvm_enabled() has been updated to kvm->arch.vm_flavor
by previous call kvm_init_vm_flavor().

	if (kvm_vm_is_protected(kvm) || kvm_vm_flavor_eq(kvm, VM_PKVM)) {
		ret = pkvm_init_host_vm(kvm, type);
		if (ret)
			goto err_unit_mmu;
	}

> diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
> index 459bd9eb7e4bc..09961c0056f2b 100644
> --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
> +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
> @@ -432,7 +432,7 @@ static void init_pkvm_hyp_vm(struct kvm *host_kvm, struct pkvm_hyp_vm *hyp_vm,
>   
>   	hyp_vm->host_kvm = host_kvm;
>   	hyp_vm->kvm.created_vcpus = nr_vcpus;
> -	hyp_vm->kvm.arch.pkvm.is_protected = READ_ONCE(host_kvm->arch.pkvm.is_protected);
> +	hyp_vm->kvm.arch.vm_flavor = READ_ONCE(host_kvm->arch.vm_flavor);
>   	hyp_vm->kvm.arch.flags = 0;
>   	pkvm_init_features_from_host(hyp_vm, host_kvm);
>   
> diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
> index 8e4c6e4bec123..3948fa46f4d75 100644
> --- a/arch/arm64/kvm/pkvm.c
> +++ b/arch/arm64/kvm/pkvm.c
> @@ -240,7 +240,6 @@ int pkvm_init_host_vm(struct kvm *kvm, unsigned long type)
>   		return ret;
>   
>   	kvm->arch.pkvm.handle = ret;
> -	kvm->arch.pkvm.is_protected = protected;
>   	if (protected) {
>   		pr_warn_once("kvm: protected VMs are experimental and for development only, tainting kernel\n");
>   		add_taint(TAINT_USER, LOCKDEP_STILL_OK);

In pkvm_init_host_vm(), the local variable 'protected' is used for once
and can be removed by:

	if (kvm_vm_is_protected(kvm)) {
		pr_warn_once(...);
		add_taint(...);
	}

Thanks,
Gavin

Re: [PATCH v17 02/20] KVM: arm64: Avoid including linux/kvm_host.h in kvm_pgtable.h

[Gavin Shan]

On 9/9/26 2:22 AM, Suzuki K Poulose wrote:
> From: Steven Price <steven.price@arm.com>
> 
> To avoid future include cycles, drop the linux/kvm_host.h include in
> kvm_pgtable.h and include the lightweight headers required for the types
> and inline helpers used there. Additionally provide a forward
> declaration for struct kvm_s2_mmu as it's only used as a pointer in this
> file.
> 
> Both pgtable.c and kvm_pkvm.h relied on the indirect inclusion of
> kvm_host.h, so make that explicit.
> 
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> New patch in v13
> ---
>   arch/arm64/include/asm/kvm_pgtable.h | 6 +++++-
>   arch/arm64/include/asm/kvm_pkvm.h    | 2 +-
>   arch/arm64/kvm/hyp/pgtable.c         | 1 +
>   3 files changed, 7 insertions(+), 2 deletions(-)
> 
Reviewed-by: Gavin Shan <gshan@redhat.com>

Re: [PATCH v17 04/20] KVM: arm64: Refactor the vcpu_load to allow for VM specific callbacks

[Gavin Shan]

Hi Suzuki,

On 9/9/26 2:22 AM, Suzuki K Poulose wrote:
> Based on a work by Marc Zyngier
> 
> To keep the VCPU load/put handling cleaner with the different kinds of VM types,
> we are about to introduce VM specific callbacks to do just the right thing.
> In preparation for that, make some refactoring to add the change easier.
> 
> No functional changes intended.
> 
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> ---
>   arch/arm64/kvm/arm.c | 46 +++++++++++++++++++++++++++-----------------
>   1 file changed, 28 insertions(+), 18 deletions(-)
> 

Nitpicks below, none of them are introduced by this patch itself. In either way:

Reviewed-by: Gavin Shan <gshan@redhat.com>

> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 67f1ff9bc4fbe..51fc651267157 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -683,14 +683,11 @@ static bool kvm_vcpu_should_clear_twe(struct kvm_vcpu *vcpu)
>   	return single_task_running();
>   }
>   
> -void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
> +static void vcpu_prepare_mmu(struct kvm_vcpu *vcpu)
>   {
>   	struct kvm_s2_mmu *mmu;
>   	int *last_ran;
>   
> -	if (is_protected_kvm_enabled())
> -		goto nommu;
> -
>   	if (vcpu_has_nv(vcpu))
>   		kvm_vcpu_load_hw_mmu(vcpu);
>   
> @@ -720,10 +717,33 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
>   		kvm_call_hyp(__kvm_flush_cpu_context, mmu);
>   		*last_ran = vcpu->vcpu_idx;
>   	}
> +}
> +
> +static void vcpu_set_wfx_traps(struct kvm_vcpu *vcpu)
> +{
> +	if (kvm_vcpu_should_clear_twe(vcpu))
> +		vcpu->arch.hcr_el2 &= ~HCR_TWE;
> +	else
> +		vcpu->arch.hcr_el2 |= HCR_TWE;
> +
> +	if (kvm_vcpu_should_clear_twi(vcpu))
> +		vcpu->arch.hcr_el2 &= ~HCR_TWI;
> +	else
> +		vcpu->arch.hcr_el2 |= HCR_TWI;
> +}
> +

It seems FIELD_MODIFY() can be applied here if no visible constraints exist:

	FIELD_MODIFY(HCR_TWE, vcpu->arch.hcr_el2, !kvm_vcpu_should_clear_twe(vcpu));
	FIELD_MODIFY(HCR_TWI, vcpu->arch.hcr_el2, !kvm_vcpu_should_clear_twi(vcpu));

> +static void vcpu_load_pvtime(struct kvm_vcpu *vcpu)
> +{
> +	if (kvm_arm_is_pvtime_enabled(&vcpu->arch))
> +		kvm_make_request(KVM_REQ_RECORD_STEAL, vcpu);
> +}
> +
> +void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
> +{
> +	if (!is_protected_kvm_enabled())
> +		vcpu_prepare_mmu(vcpu);
>  

I think we probably try to depend on kvm->arch.vm_flavor instead
of !is_protected_kvm_enabled().

	struct kvm *kvm = vcpu->kvm;

	if (kvm_vm_is_protected(kvm) || kvm_vm_flavor_eq(kvm, VM_PKVM))
		vcpu_prepare_mmu(vcpu);
  
> -nommu:
>   	vcpu->cpu = cpu;
> -
>   	/*
>   	 * The timer must be loaded before the vgic to correctly set up physical
>   	 * interrupt deactivation in nested state (e.g. timer interrupt).
> @@ -736,19 +756,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
>   		kvm_vcpu_load_vhe(vcpu);
>   	kvm_arch_vcpu_load_fp(vcpu);
>   	kvm_vcpu_pmu_restore_guest(vcpu);
> -	if (kvm_arm_is_pvtime_enabled(&vcpu->arch))
> -		kvm_make_request(KVM_REQ_RECORD_STEAL, vcpu);
> -
> -	if (kvm_vcpu_should_clear_twe(vcpu))
> -		vcpu->arch.hcr_el2 &= ~HCR_TWE;
> -	else
> -		vcpu->arch.hcr_el2 |= HCR_TWE;
> -
> -	if (kvm_vcpu_should_clear_twi(vcpu))
> -		vcpu->arch.hcr_el2 &= ~HCR_TWI;
> -	else
> -		vcpu->arch.hcr_el2 |= HCR_TWI;
>   
> +	vcpu_load_pvtime(vcpu);
> +	vcpu_set_wfx_traps(vcpu);
>   	vcpu_set_pauth_traps(vcpu);
>   
>   	if (is_protected_kvm_enabled()) {

Thanks,
Gavin

Re: [PATCH v17 06/20] KVM: arm64: CCA: Add a new mode for supporting Realm guests

[Kohei Enju]

Hi Marc,

On 09/09 11:48, Marc Zyngier wrote:
> On Wed, 09 Sep 2026 04:26:41 +0100,
> Kohei Enju <enju.kohei@fujitsu.com> wrote:
> > 
> > Hi Suzuki,
> > 
> > On 09/08 17:22, Suzuki K Poulose wrote:
> > > Add an explicit mode to support Arm CCA guests.
> > > 
> > > Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> > > ---
> > >  Documentation/admin-guide/kernel-parameters.txt | 3 +++
> > >  arch/arm64/include/asm/kvm_host.h               | 1 +
> > >  arch/arm64/kvm/arm.c                            | 5 +++++
> > >  3 files changed, 9 insertions(+)
> > > 
> > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > index 68647ff4bdd24..1afe3df3b923e 100644
> > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > @@ -3256,6 +3256,9 @@ Kernel parameters
> > >  			nested: VHE-based mode with support for nested
> > >  				virtualization. Requires at least ARMv8.4
> > >  				hardware (with FEAT_NV2).
> > > +			rmm: Support for running confidential guests in Realm
> > > +			     world using RMM, as defined by Arm Confidential
> > > +			     Compute Architecture (CCA)
> > >  
> > >  			Defaults to VHE/nVHE based on hardware support. Setting
> > >  			mode to "protected" will disable kexec and hibernation
> > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > > index b2e99c5cb1cd3..31ae9d8d8e92f 100644
> > > --- a/arch/arm64/include/asm/kvm_host.h
> > > +++ b/arch/arm64/include/asm/kvm_host.h
> > > @@ -69,6 +69,7 @@ enum kvm_mode {
> > >  	KVM_MODE_DEFAULT,
> > >  	KVM_MODE_PROTECTED,
> > >  	KVM_MODE_NV,
> > > +	KVM_MODE_RMM,
> > >  	KVM_MODE_NONE,
> > >  };
> > >  #ifdef CONFIG_KVM
> > > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> > > index 9af3bbb2f8c24..50f0adfadab38 100644
> > > --- a/arch/arm64/kvm/arm.c
> > > +++ b/arch/arm64/kvm/arm.c
> > > @@ -3264,6 +3264,11 @@ static int __init early_kvm_mode_cfg(char *arg)
> > >  		return 0;
> > >  	}
> > >  
> > > +	if (strcmp(arg, "rmm") == 0 && !WARN_ON(!is_kernel_in_hyp_mode())) {
> > > +		kvm_mode = KVM_MODE_RMM;
> > > +		return 0;
> > > +	}
> > 
> > Does this make NV support and CCA Realm VM support mutually exclusive?
> > 
> > Until the v16 series, we could run Realm VMs with:
> >   'kvm-arm.mode=nested'
> > 
> > As far as the AArch64 architecture is concerned, there does not seem to
> > be a reason to make them mutually exclusive. I am therefore wondering
> > whether this restriction is intentional.
> 
> I have a hack somewhere that will allow this, but this requires to
> change the above to be a capability mask rather than a simple
> enumeration. I'll see if I can spin that as a separate series.

Ack. Thank you for the clarification.
Looking forward to seeing the series.

Thanks,
Kohei

> 
> 	M.
> 
> -- 
> Without deviation from the norm, progress is not possible.

Re: [PATCH v17 05/20] KVM: arm64: Add vcpu load/put call backs for flavors

[Gavin Shan]

Hi Suzuki,

On 9/9/26 2:22 AM, Suzuki K Poulose wrote:
> Add VM flavor specific handlers for VCPU load/put, in an effort to make it
> easier to follow the code.
> 
> Based on a patch by Marc Zyngier
> 
> Suggested-by: Marc Zyngier <maz@kernel.org>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> ---
>   arch/arm64/include/asm/kvm_host.h |   6 ++
>   arch/arm64/kvm/arm.c              | 156 ++++++++++++++++++++++--------
>   2 files changed, 123 insertions(+), 39 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index d0dccc9ad6aa8..b2e99c5cb1cd3 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -150,6 +150,11 @@ struct kvm_vmid {
>   	atomic64_t id;
>   };
>   
> +struct kvm_vcpu_ops {
> +	void (*vcpu_load)(struct kvm_vcpu *vcpu, int cpu);
> +	void (*vcpu_put)(struct kvm_vcpu *vcpu);
> +};
> +

struct kvm_vcpu::cpu has been set to 'cpu' before vcpu_load() is called in
kvm_arch_vcpu_load(), so we needn't explicitly pass @cpu to vcpu_load()?

>   struct kvm_s2_mmu {
>   	struct kvm_vmid vmid;
>   
> @@ -854,6 +859,7 @@ struct vncr_tlb;
>   
>   struct kvm_vcpu_arch {
>   	struct kvm_cpu_context ctxt;
> +	const struct kvm_vcpu_ops *vcpu_ops;
>   

I think this would be a field of 'struct kvm_arch' if all vCPUs inside a VM
have same vCPU operations.

>   	/*
>   	 * Guest floating point state
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 51fc651267157..9af3bbb2f8c24 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -74,6 +74,8 @@ struct kvm_ioctl_cap_map {
>   	long ext;
>   };
>   
> +static const struct kvm_vcpu_ops *arm64_vcpu_ops[VM_FLAVOR_MAX];
> +
>   /* Make KVM_CAP_NR_VCPUS the reference for features we always supported */
>   #define KVM_CAP_ARM_BASIC	KVM_CAP_NR_VCPUS
>   
> @@ -569,6 +571,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
>   	mutex_unlock(&vcpu->mutex);
>   #endif
>   
> +	vcpu->arch.vcpu_ops = arm64_vcpu_ops[vcpu->kvm->arch.vm_flavor];
> +
>   	/* Force users to call KVM_ARM_VCPU_INIT */
>   	vcpu_clear_flag(vcpu, VCPU_INITIALIZED);
>   
> @@ -738,36 +742,72 @@ static void vcpu_load_pvtime(struct kvm_vcpu *vcpu)
>   		kvm_make_request(KVM_REQ_RECORD_STEAL, vcpu);
>   }
>   
> +static void vhe_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
> +{
> +	vcpu_prepare_mmu(vcpu);
> +	/*
> +	 * The timer must be loaded before the vgic to correctly set up physical
> +	 * interrupt deactivation in nested state (e.g. timer interrupt).
> +	 */
> +	kvm_timer_vcpu_load(vcpu);
> +	kvm_vgic_load(vcpu);
> +	kvm_vcpu_load_debug(vcpu);
> +	kvm_vcpu_load_fgt(vcpu);
> +	kvm_vcpu_load_vhe(vcpu);
> +	kvm_arch_vcpu_load_fp(vcpu);
> +	kvm_vcpu_pmu_restore_guest(vcpu);
> +
> +	vcpu_load_pvtime(vcpu);
> +	vcpu_set_wfx_traps(vcpu);
> +	vcpu_set_pauth_traps(vcpu);
> +}
> +
> +static void nvhe_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
> +{
> +	vcpu_prepare_mmu(vcpu);
> +	/*
> +	 * The timer must be loaded before the vgic to correctly set up physical
> +	 * interrupt deactivation in nested state (e.g. timer interrupt).
> +	 */
> +	kvm_timer_vcpu_load(vcpu);
> +	kvm_vgic_load(vcpu);
> +	kvm_vcpu_load_debug(vcpu);
> +	kvm_vcpu_load_fgt(vcpu);
> +	kvm_arch_vcpu_load_fp(vcpu);
> +	kvm_vcpu_pmu_restore_guest(vcpu);
> +
> +	vcpu_load_pvtime(vcpu);
> +	vcpu_set_wfx_traps(vcpu);
> +	vcpu_set_pauth_traps(vcpu);
> +}
> +
> +static void pkvm_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
> +{
> +	/*
> +	 * The timer must be loaded before the vgic to correctly set up physical
> +	 * interrupt deactivation in nested state (e.g. timer interrupt).
> +	 */
> +	kvm_timer_vcpu_load(vcpu);
> +	kvm_vgic_load(vcpu);
> +	kvm_vcpu_load_debug(vcpu);
> +	kvm_vcpu_load_fgt(vcpu);
> +	kvm_arch_vcpu_load_fp(vcpu);
> +	kvm_vcpu_pmu_restore_guest(vcpu);
> +
> +	vcpu_load_pvtime(vcpu);
> +	vcpu_set_wfx_traps(vcpu);

It seems vcpu_set_pauth_traps(vcpu) has been missed here?

> +
> +	kvm_call_hyp_nvhe(__pkvm_vcpu_load,
> +			  vcpu->kvm->arch.pkvm.handle,
> +			  vcpu->vcpu_idx, vcpu->arch.hcr_el2);
> +	kvm_call_hyp(__vgic_v3_restore_vmcr_aprs,
> +		     &vcpu->arch.vgic_cpu.vgic_v3);
> +}
> +
>   void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
>   {
> -	if (!is_protected_kvm_enabled())
> -		vcpu_prepare_mmu(vcpu);
> -
>   	vcpu->cpu = cpu;
> -	/*
> -	 * The timer must be loaded before the vgic to correctly set up physical
> -	 * interrupt deactivation in nested state (e.g. timer interrupt).
> -	 */
> -	kvm_timer_vcpu_load(vcpu);
> -	kvm_vgic_load(vcpu);
> -	kvm_vcpu_load_debug(vcpu);
> -	kvm_vcpu_load_fgt(vcpu);
> -	if (has_vhe())
> -		kvm_vcpu_load_vhe(vcpu);
> -	kvm_arch_vcpu_load_fp(vcpu);
> -	kvm_vcpu_pmu_restore_guest(vcpu);
> -
> -	vcpu_load_pvtime(vcpu);
> -	vcpu_set_wfx_traps(vcpu);
> -	vcpu_set_pauth_traps(vcpu);
> -
> -	if (is_protected_kvm_enabled()) {
> -		kvm_call_hyp_nvhe(__pkvm_vcpu_load,
> -				  vcpu->kvm->arch.pkvm.handle,
> -				  vcpu->vcpu_idx, vcpu->arch.hcr_el2);
> -		kvm_call_hyp(__vgic_v3_restore_vmcr_aprs,
> -			     &vcpu->arch.vgic_cpu.vgic_v3);
> -	}
> +	vcpu->arch.vcpu_ops->vcpu_load(vcpu, cpu);
>   
>   	if (!cpumask_test_cpu(cpu, vcpu->kvm->arch.supported_cpus))
>   		vcpu_set_on_unsupported_cpu(vcpu);
> @@ -775,28 +815,44 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
>   	vcpu->arch.pid = pid_nr(vcpu->pid);
>   }
>   
> -void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
> +static void vhe_vcpu_put(struct kvm_vcpu *vcpu)
>   {
> -	if (is_protected_kvm_enabled()) {
> -		kvm_call_hyp(__vgic_v3_save_aprs, &vcpu->arch.vgic_cpu.vgic_v3);
> -		kvm_call_hyp_nvhe(__pkvm_vcpu_put);
> -
> -		/* __pkvm_vcpu_put implies a sync of the state */
> -		if (!kvm_vm_is_protected(vcpu->kvm))
> -			vcpu_set_flag(vcpu, PKVM_HOST_STATE_DIRTY);
> -	}
> -
>   	kvm_vcpu_put_debug(vcpu);
>   	kvm_arch_vcpu_put_fp(vcpu);
> -	if (has_vhe())
> -		kvm_vcpu_put_vhe(vcpu);
> +	kvm_vcpu_put_vhe(vcpu);
>   	kvm_timer_vcpu_put(vcpu);
>   	kvm_vgic_put(vcpu);
>   	kvm_vcpu_pmu_restore_host(vcpu);
>   	if (vcpu_has_nv(vcpu))
>   		kvm_vcpu_put_hw_mmu(vcpu);
>   	kvm_arm_vmid_clear_active();
> +}
>   
> +static void nvhe_vcpu_put(struct kvm_vcpu *vcpu)
> +{
> +	kvm_vcpu_put_debug(vcpu);
> +	kvm_arch_vcpu_put_fp(vcpu);
> +	kvm_timer_vcpu_put(vcpu);
> +	kvm_vgic_put(vcpu);
> +	kvm_vcpu_pmu_restore_host(vcpu);
> +	kvm_arm_vmid_clear_active();
> +}
> +
> +static void pkvm_vcpu_put(struct kvm_vcpu *vcpu)
> +{
> +	kvm_call_hyp(__vgic_v3_save_aprs, &vcpu->arch.vgic_cpu.vgic_v3);
> +	kvm_call_hyp_nvhe(__pkvm_vcpu_put);
> +
> +	/* __pkvm_vcpu_put implies a sync of the state */
> +	if (!kvm_vm_is_protected(vcpu->kvm))
> +		vcpu_set_flag(vcpu, PKVM_HOST_STATE_DIRTY);
> +
> +	nvhe_vcpu_put(vcpu);
> +}
> +
> +void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
> +{
> +	vcpu->arch.vcpu_ops->vcpu_put(vcpu);
>   	vcpu_clear_on_unsupported_cpu(vcpu);
>   	vcpu->cpu = -1;
>   }
> @@ -2136,6 +2192,28 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
>   	}
>   }
>   
> +static const struct kvm_vcpu_ops vhe_vcpu_ops = {
> +	.vcpu_load = vhe_vcpu_load,
> +	.vcpu_put = vhe_vcpu_put,
> +};
> +
> +static const struct kvm_vcpu_ops nvhe_vcpu_ops = {
> +	.vcpu_load = nvhe_vcpu_load,
> +	.vcpu_put = nvhe_vcpu_put,
> +};
> +
> +static const struct kvm_vcpu_ops pkvm_vcpu_ops = {
> +	.vcpu_load = pkvm_vcpu_load,
> +	.vcpu_put = pkvm_vcpu_put,
> +};
> +
> +static const struct kvm_vcpu_ops *arm64_vcpu_ops[] = {
> +	[VM_VHE] = &vhe_vcpu_ops,
> +	[VM_NVHE] = &nvhe_vcpu_ops,
> +	[VM_PKVM] = &pkvm_vcpu_ops,
> +	[VM_PROTECTED_PKVM] = &pkvm_vcpu_ops,
> +};
> +

We may avoid the variables {vhe, nvhe, pkvm}_vcpu_ops which are used for once:

static const struct kvm_vcpu_ops kvm_vcpu_ops[] = {
	[VM_VHE]            = { vhe_vcpu_load,  vhe_vcpu_put  },
	[VM_NVHE]           = { nvhe_vcpu_load, nvhe_vcpu_put },
	[VM_PKVM]           = { pkvm_vcpu_load, pkvm_vcpu_put },
	[VM_PROTECTED_PKVM] = { pkvm_vcpu_load, pkvm_vcpu_put },
};

>   static unsigned long nvhe_percpu_size(void)
>   {
>   	return (unsigned long)CHOOSE_NVHE_SYM(__per_cpu_end) -

Thanks,
Gavin

Re: [PATCH v17 06/20] KVM: arm64: CCA: Add a new mode for supporting Realm guests

[Gavin Shan]

Hi Suzuki,

On 9/9/26 2:22 AM, Suzuki K Poulose wrote:
> Add an explicit mode to support Arm CCA guests.
> 
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> ---
>   Documentation/admin-guide/kernel-parameters.txt | 3 +++
>   arch/arm64/include/asm/kvm_host.h               | 1 +
>   arch/arm64/kvm/arm.c                            | 5 +++++
>   3 files changed, 9 insertions(+)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 68647ff4bdd24..1afe3df3b923e 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -3256,6 +3256,9 @@ Kernel parameters
>   			nested: VHE-based mode with support for nested
>   				virtualization. Requires at least ARMv8.4
>   				hardware (with FEAT_NV2).
> +			rmm: Support for running confidential guests in Realm
> +			     world using RMM, as defined by Arm Confidential
> +			     Compute Architecture (CCA)
> 

I'm not sure, but 'realm' would be more appropriate name. rmm (Real Management Monitor) is
the term limited to the firmware to me. Besides, we have been using 'realm' broadly in this
series like kvm_vm_is_realm(), kvm_realm_state() and so on.

   
>   			Defaults to VHE/nVHE based on hardware support. Setting
>   			mode to "protected" will disable kexec and hibernation
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index b2e99c5cb1cd3..31ae9d8d8e92f 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -69,6 +69,7 @@ enum kvm_mode {
>   	KVM_MODE_DEFAULT,
>   	KVM_MODE_PROTECTED,
>   	KVM_MODE_NV,
> +	KVM_MODE_RMM,
>   	KVM_MODE_NONE,
>   };
>   #ifdef CONFIG_KVM
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 9af3bbb2f8c24..50f0adfadab38 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -3264,6 +3264,11 @@ static int __init early_kvm_mode_cfg(char *arg)
>   		return 0;
>   	}
>   
> +	if (strcmp(arg, "rmm") == 0 && !WARN_ON(!is_kernel_in_hyp_mode())) {
> +		kvm_mode = KVM_MODE_RMM;
> +		return 0;
> +	}
> +
>   	return -EINVAL;
>   }
>   early_param("kvm-arm.mode", early_kvm_mode_cfg);

Thanks,
Gavin

Re: [PATCH v17 03/20] KVM: arm64: Track the type of VM in kvm_arch

[Suzuki K Poulose]

On 10/09/2026 04:39, Gavin Shan wrote:
> Hi Suzuki,
> 
> On 9/9/26 2:22 AM, Suzuki K Poulose wrote:
>> KVM arm64 has different types of VMs with all the different modes in 
>> which the
>> hypervisor code can be run. e.g., VHE, nVHE, PKVM etc. Then there is 
>> protected
>> VM and normal VMs with PKVM. We might soon add other types, e.g., Arm 
>> CCA Realm.
>> So in an effort to make the handling of these different types of VMs a 
>> bit more
>> friendlier to the eyes, add a VM flavor to the kvm_arch and we could 
>> then add
>> handlers for different operations based on the VM type.
>>
>> Keep the flavor initialisation at the beginning to allow for the 
>> detection
>> early enough and fail out on any unsupported requests. (e.g., 
>> protected on !PKVM)
>>
>> With that, use the vm_flavor to detect if a VM is protected VM on PKVM.
>>
>> Based on a patch by Marc Zyngier
>>
>> Suggested-by: Marc Zyngier <maz@kernel.org>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>> ---
>>   arch/arm64/include/asm/kvm_host.h | 12 ++++++++++--
>>   arch/arm64/kvm/arm.c              | 27 ++++++++++++++++++++++++---
>>   arch/arm64/kvm/hyp/nvhe/pkvm.c    |  2 +-
>>   arch/arm64/kvm/pkvm.c             |  1 -
>>   4 files changed, 35 insertions(+), 7 deletions(-)
>>
> 
> Some nitpicks below. In either way:
> 
> Reviewed-by: Gavin Shan <gshan@redhat.com>
> 
>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/ 
>> asm/kvm_host.h
>> index 27fe0cd5b2d7a..d0dccc9ad6aa8 100644
>> --- a/arch/arm64/include/asm/kvm_host.h
>> +++ b/arch/arm64/include/asm/kvm_host.h
>> @@ -257,7 +257,6 @@ struct kvm_protected_vm {
>>       pkvm_handle_t handle;
>>       struct kvm_hyp_memcache teardown_mc;
>>       struct kvm_hyp_memcache stage2_teardown_mc;
>> -    bool is_protected;
>>       bool is_created;
>>       /*
>> @@ -306,9 +305,18 @@ enum fgt_group_id {
>>       __NR_FGT_GROUP_IDS__
>>   };
>> +enum kvm_arm_vm_flavor {
>> +    VM_NVHE,
>> +    VM_VHE,
>> +    VM_PKVM,        /* Normal guests on PKVM */
>> +    VM_PROTECTED_PKVM,    /* Protected VM */
>> +    VM_FLAVOR_MAX,
>> +};
>> +
>>   struct kvm_arch {
>>       struct kvm_s2_mmu mmu;
>> +    enum kvm_arm_vm_flavor vm_flavor;
>>       /*
>>        * Fine-Grained UNDEF, mimicking the FGT layout defined by the
>>        * architecture. We track them globally, as we present the
>> @@ -1504,7 +1512,7 @@ struct kvm *kvm_arch_alloc_vm(void);
>>   #define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS_RANGE
>> -#define kvm_vm_is_protected(kvm)    (is_protected_kvm_enabled() && 
>> (kvm)->arch.pkvm.is_protected)
>> +#define kvm_vm_is_protected(kvm)    ((kvm)->arch.vm_flavor == 
>> VM_PROTECTED_PKVM)
> 
> Here we may introduce one more helper for generic use cases. It's going to
> be used in kvm_arch_init_vm().
> 
> #define kvm_vm_flavor_eq(kvm, flavor)    ((kvm)->arch.vm_flaor == (flavor))
> #define kvm_vm_is_protected(kvm)    kvm_vm_flavor_eq(kvm, 
> VM_PROTECTED_PKVM)

see below.

> 
>>   #define vcpu_is_protected(vcpu)        kvm_vm_is_protected((vcpu)->kvm)
>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
>> index 8b080804bc90b..67f1ff9bc4fbe 100644
>> --- a/arch/arm64/kvm/arm.c
>> +++ b/arch/arm64/kvm/arm.c
>> @@ -214,6 +214,26 @@ static int kvm_arm_default_max_vcpus(void)
>>       return vgic_present ? kvm_vgic_get_max_vcpus() : KVM_MAX_VCPUS;
>>   }
>> +static int kvm_init_vm_flavor(struct kvm *kvm, unsigned long type)
>> +{
>> +    bool protected = type & KVM_VM_TYPE_ARM_PROTECTED;
>> +
>> +    if (is_protected_kvm_enabled()) {
>> +        if (protected)
>> +            kvm->arch.vm_flavor = VM_PROTECTED_PKVM;
>> +        else
>> +            kvm->arch.vm_flavor = VM_PKVM;
>> +    } else if (protected) {
>> +        return -EINVAL;
>> +    } else if (has_vhe()) {
>> +        kvm->arch.vm_flavor = VM_VHE;
>> +    } else {
>> +        kvm->arch.vm_flavor = VM_NVHE;
>> +    }
>> +
>> +    return 0;
>> +}
>> +
>>   /**
>>    * kvm_arch_init_vm - initializes a VM data structure
>>    * @kvm:    pointer to the KVM struct
>> @@ -236,6 +256,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned 
>> long type)
>>       mutex_unlock(&kvm->lock);
>>   #endif
>> +    ret = kvm_init_vm_flavor(kvm, type);
>> +    if (ret)
>> +        return ret;
>> +
>>       kvm_init_nested(kvm);
>>       ret = kvm_share_hyp(kvm, kvm + 1);
>> @@ -260,9 +284,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned 
>> long type)
>>           ret = pkvm_init_host_vm(kvm, type);
>>           if (ret)
>>               goto err_uninit_mmu;
>> -    } else if (type & KVM_VM_TYPE_ARM_PROTECTED) {
>> -        ret = -EINVAL;
>> -        goto err_uninit_mmu;
>>       }
>>       kvm_vgic_early_init(kvm);
> 
> We needn't depend on is_protected_kvm_enabled() to call 
> pkvm_init_host_vm() because
> the output of is_protected_kvm_enabled() has been updated to kvm- 
>  >arch.vm_flavor
> by previous call kvm_init_vm_flavor().

TBH, thats a faster way to check if we are running on PKVM. I could
add something linke kvm_vm_has_pkvm(kvm) which covers both VM_PKVM and
VM_PROTECTED_PKVM.

> 
>      if (kvm_vm_is_protected(kvm) || kvm_vm_flavor_eq(kvm, VM_PKVM)) {
>          ret = pkvm_init_host_vm(kvm, type);
>          if (ret)
>              goto err_unit_mmu;
>      }

And I can drop the type fron pkvm_init_host_vm with suggestion below.

Cheers
Suzuki

> 
>> diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/ 
>> pkvm.c
>> index 459bd9eb7e4bc..09961c0056f2b 100644
>> --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
>> +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
>> @@ -432,7 +432,7 @@ static void init_pkvm_hyp_vm(struct kvm *host_kvm, 
>> struct pkvm_hyp_vm *hyp_vm,
>>       hyp_vm->host_kvm = host_kvm;
>>       hyp_vm->kvm.created_vcpus = nr_vcpus;
>> -    hyp_vm->kvm.arch.pkvm.is_protected = READ_ONCE(host_kvm- 
>> >arch.pkvm.is_protected);
>> +    hyp_vm->kvm.arch.vm_flavor = READ_ONCE(host_kvm->arch.vm_flavor);
>>       hyp_vm->kvm.arch.flags = 0;
>>       pkvm_init_features_from_host(hyp_vm, host_kvm);
>> diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
>> index 8e4c6e4bec123..3948fa46f4d75 100644
>> --- a/arch/arm64/kvm/pkvm.c
>> +++ b/arch/arm64/kvm/pkvm.c
>> @@ -240,7 +240,6 @@ int pkvm_init_host_vm(struct kvm *kvm, unsigned 
>> long type)
>>           return ret;
>>       kvm->arch.pkvm.handle = ret;
>> -    kvm->arch.pkvm.is_protected = protected;
>>       if (protected) {
>>           pr_warn_once("kvm: protected VMs are experimental and for 
>> development only, tainting kernel\n");
>>           add_taint(TAINT_USER, LOCKDEP_STILL_OK);
> 
> In pkvm_init_host_vm(), the local variable 'protected' is used for once
> and can be removed by:
> 
>      if (kvm_vm_is_protected(kvm)) {
>          pr_warn_once(...);
>          add_taint(...);
>      }
> 
> Thanks,
> Gavin
> 

Re: [PATCH v17 05/20] KVM: arm64: Add vcpu load/put call backs for flavors

[Suzuki K Poulose]

Hi Gavin

On 10/09/2026 06:33, Gavin Shan wrote:
> Hi Suzuki,
> 
> On 9/9/26 2:22 AM, Suzuki K Poulose wrote:
>> Add VM flavor specific handlers for VCPU load/put, in an effort to 
>> make it
>> easier to follow the code.
>>
>> Based on a patch by Marc Zyngier
>>
>> Suggested-by: Marc Zyngier <maz@kernel.org>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>> ---
>>   arch/arm64/include/asm/kvm_host.h |   6 ++
>>   arch/arm64/kvm/arm.c              | 156 ++++++++++++++++++++++--------
>>   2 files changed, 123 insertions(+), 39 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/ 
>> asm/kvm_host.h
>> index d0dccc9ad6aa8..b2e99c5cb1cd3 100644
>> --- a/arch/arm64/include/asm/kvm_host.h
>> +++ b/arch/arm64/include/asm/kvm_host.h
>> @@ -150,6 +150,11 @@ struct kvm_vmid {
>>       atomic64_t id;
>>   };
>> +struct kvm_vcpu_ops {
>> +    void (*vcpu_load)(struct kvm_vcpu *vcpu, int cpu);
>> +    void (*vcpu_put)(struct kvm_vcpu *vcpu);
>> +};
>> +
> 
> struct kvm_vcpu::cpu has been set to 'cpu' before vcpu_load() is called in
> kvm_arch_vcpu_load(), so we needn't explicitly pass @cpu to vcpu_load()?

Ack

> 
>>   struct kvm_s2_mmu {
>>       struct kvm_vmid vmid;
>> @@ -854,6 +859,7 @@ struct vncr_tlb;
>>   struct kvm_vcpu_arch {
>>       struct kvm_cpu_context ctxt;
>> +    const struct kvm_vcpu_ops *vcpu_ops;
> 
> I think this would be a field of 'struct kvm_arch' if all vCPUs inside a VM
> have same vCPU operations.

This was done to avoid the long list of pointer chasing :

vcpu->kvm->arch.vcpu_ops->

Given this is vcpu specific ops and a single pointer per vcpu, we can 
make that easier to read.


> 
>>       /*
>>        * Guest floating point state
>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
>> index 51fc651267157..9af3bbb2f8c24 100644
>> --- a/arch/arm64/kvm/arm.c
>> +++ b/arch/arm64/kvm/arm.c
>> @@ -74,6 +74,8 @@ struct kvm_ioctl_cap_map {
>>       long ext;
>>   };
>> +static const struct kvm_vcpu_ops *arm64_vcpu_ops[VM_FLAVOR_MAX];
>> +
>>   /* Make KVM_CAP_NR_VCPUS the reference for features we always 
>> supported */
>>   #define KVM_CAP_ARM_BASIC    KVM_CAP_NR_VCPUS
>> @@ -569,6 +571,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
>>       mutex_unlock(&vcpu->mutex);
>>   #endif
>> +    vcpu->arch.vcpu_ops = arm64_vcpu_ops[vcpu->kvm->arch.vm_flavor];
>> +
>>       /* Force users to call KVM_ARM_VCPU_INIT */
>>       vcpu_clear_flag(vcpu, VCPU_INITIALIZED);
>> @@ -738,36 +742,72 @@ static void vcpu_load_pvtime(struct kvm_vcpu *vcpu)
>>           kvm_make_request(KVM_REQ_RECORD_STEAL, vcpu);
>>   }
>> +static void vhe_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
>> +{
>> +    vcpu_prepare_mmu(vcpu);
>> +    /*
>> +     * The timer must be loaded before the vgic to correctly set up 
>> physical
>> +     * interrupt deactivation in nested state (e.g. timer interrupt).
>> +     */
>> +    kvm_timer_vcpu_load(vcpu);
>> +    kvm_vgic_load(vcpu);
>> +    kvm_vcpu_load_debug(vcpu);
>> +    kvm_vcpu_load_fgt(vcpu);
>> +    kvm_vcpu_load_vhe(vcpu);
>> +    kvm_arch_vcpu_load_fp(vcpu);
>> +    kvm_vcpu_pmu_restore_guest(vcpu);
>> +
>> +    vcpu_load_pvtime(vcpu);
>> +    vcpu_set_wfx_traps(vcpu);
>> +    vcpu_set_pauth_traps(vcpu);
>> +}
>> +
>> +static void nvhe_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
>> +{
>> +    vcpu_prepare_mmu(vcpu);
>> +    /*
>> +     * The timer must be loaded before the vgic to correctly set up 
>> physical
>> +     * interrupt deactivation in nested state (e.g. timer interrupt).
>> +     */
>> +    kvm_timer_vcpu_load(vcpu);
>> +    kvm_vgic_load(vcpu);
>> +    kvm_vcpu_load_debug(vcpu);
>> +    kvm_vcpu_load_fgt(vcpu);
>> +    kvm_arch_vcpu_load_fp(vcpu);
>> +    kvm_vcpu_pmu_restore_guest(vcpu);
>> +
>> +    vcpu_load_pvtime(vcpu);
>> +    vcpu_set_wfx_traps(vcpu);
>> +    vcpu_set_pauth_traps(vcpu);
>> +}
>> +
>> +static void pkvm_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
>> +{
>> +    /*
>> +     * The timer must be loaded before the vgic to correctly set up 
>> physical
>> +     * interrupt deactivation in nested state (e.g. timer interrupt).
>> +     */
>> +    kvm_timer_vcpu_load(vcpu);
>> +    kvm_vgic_load(vcpu);
>> +    kvm_vcpu_load_debug(vcpu);
>> +    kvm_vcpu_load_fgt(vcpu);
>> +    kvm_arch_vcpu_load_fp(vcpu);
>> +    kvm_vcpu_pmu_restore_guest(vcpu);
>> +
>> +    vcpu_load_pvtime(vcpu);
>> +    vcpu_set_wfx_traps(vcpu);
> 
> It seems vcpu_set_pauth_traps(vcpu) has been missed here?

No, if you see the vcpu_set_pauth_traps(), we do nothing for 
is_protected_kvm_enabled(). So skipped that explicitly here.
I didn't remove that condition from vcpu_set_pauth_traps()
just to keep it safer.

> 
>> +
>> +    kvm_call_hyp_nvhe(__pkvm_vcpu_load,
>> +              vcpu->kvm->arch.pkvm.handle,
>> +              vcpu->vcpu_idx, vcpu->arch.hcr_el2);
>> +    kvm_call_hyp(__vgic_v3_restore_vmcr_aprs,
>> +             &vcpu->arch.vgic_cpu.vgic_v3);
>> +}
>> +

...

>> +static const struct kvm_vcpu_ops vhe_vcpu_ops = {
>> +    .vcpu_load = vhe_vcpu_load,
>> +    .vcpu_put = vhe_vcpu_put,
>> +};
>> +
>> +static const struct kvm_vcpu_ops nvhe_vcpu_ops = {
>> +    .vcpu_load = nvhe_vcpu_load,
>> +    .vcpu_put = nvhe_vcpu_put,
>> +};
>> +
>> +static const struct kvm_vcpu_ops pkvm_vcpu_ops = {
>> +    .vcpu_load = pkvm_vcpu_load,
>> +    .vcpu_put = pkvm_vcpu_put,
>> +};
>> +
>> +static const struct kvm_vcpu_ops *arm64_vcpu_ops[] = {
>> +    [VM_VHE] = &vhe_vcpu_ops,
>> +    [VM_NVHE] = &nvhe_vcpu_ops,
>> +    [VM_PKVM] = &pkvm_vcpu_ops,
>> +    [VM_PROTECTED_PKVM] = &pkvm_vcpu_ops,
>> +};
>> +
> 
> We may avoid the variables {vhe, nvhe, pkvm}_vcpu_ops which are used for 
> once:
> 
> static const struct kvm_vcpu_ops kvm_vcpu_ops[] = {
>      [VM_VHE]            = { vhe_vcpu_load,  vhe_vcpu_put  },
>      [VM_NVHE]           = { nvhe_vcpu_load, nvhe_vcpu_put },
>      [VM_PKVM]           = { pkvm_vcpu_load, pkvm_vcpu_put },
>      [VM_PROTECTED_PKVM] = { pkvm_vcpu_load, pkvm_vcpu_put },
> };

The structure might grow in the future (I have a vcpu_run callback in 
testing). So it does make sense to use the variables to keep it tidy.

I will add checks to make sure that the fields are initialised properly.


Cheers
Suzuki

> 
>>   static unsigned long nvhe_percpu_size(void)
>>   {
>>       return (unsigned long)CHOOSE_NVHE_SYM(__per_cpu_end) -
> 
> Thanks,
> Gavin
> 

Re: [PATCH v17 06/20] KVM: arm64: CCA: Add a new mode for supporting Realm guests

[Suzuki K Poulose]

On 10/09/2026 06:53, Gavin Shan wrote:
> Hi Suzuki,
> 
> On 9/9/26 2:22 AM, Suzuki K Poulose wrote:
>> Add an explicit mode to support Arm CCA guests.
>>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>> ---
>>   Documentation/admin-guide/kernel-parameters.txt | 3 +++
>>   arch/arm64/include/asm/kvm_host.h               | 1 +
>>   arch/arm64/kvm/arm.c                            | 5 +++++
>>   3 files changed, 9 insertions(+)
>>
>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/ 
>> Documentation/admin-guide/kernel-parameters.txt
>> index 68647ff4bdd24..1afe3df3b923e 100644
>> --- a/Documentation/admin-guide/kernel-parameters.txt
>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>> @@ -3256,6 +3256,9 @@ Kernel parameters
>>               nested: VHE-based mode with support for nested
>>                   virtualization. Requires at least ARMv8.4
>>                   hardware (with FEAT_NV2).
>> +            rmm: Support for running confidential guests in Realm
>> +                 world using RMM, as defined by Arm Confidential
>> +                 Compute Architecture (CCA)
>>
> 
> I'm not sure, but 'realm' would be more appropriate name. rmm (Real 
> Management Monitor) is
> the term limited to the firmware to me. Besides, we have been using 
> 'realm' broadly in this
> series like kvm_vm_is_realm(), kvm_realm_state() and so on.

May be. But we use the "mode" of the hypervisor in question here, 
mostly. e.g., nvhe, vhe, protected etc. So, I went with RMM (in favor
of CCA). Does that make sense ? Also, we rely on firmware/arm_rmm for
the support. So, it kind of make sense to keep it "rmm" ?

Cheers
Suzuki

Re: [PATCH v17 06/20] KVM: arm64: CCA: Add a new mode for supporting Realm guests

[Gavin Shan]

On 9/10/26 6:43 PM, Suzuki K Poulose wrote:
> On 10/09/2026 06:53, Gavin Shan wrote:
>> Hi Suzuki,
>>
>> On 9/9/26 2:22 AM, Suzuki K Poulose wrote:
>>> Add an explicit mode to support Arm CCA guests.
>>>
>>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>>> ---
>>>   Documentation/admin-guide/kernel-parameters.txt | 3 +++
>>>   arch/arm64/include/asm/kvm_host.h               | 1 +
>>>   arch/arm64/kvm/arm.c                            | 5 +++++
>>>   3 files changed, 9 insertions(+)
>>>
>>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/ Documentation/admin-guide/kernel-parameters.txt
>>> index 68647ff4bdd24..1afe3df3b923e 100644
>>> --- a/Documentation/admin-guide/kernel-parameters.txt
>>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>>> @@ -3256,6 +3256,9 @@ Kernel parameters
>>>               nested: VHE-based mode with support for nested
>>>                   virtualization. Requires at least ARMv8.4
>>>                   hardware (with FEAT_NV2).
>>> +            rmm: Support for running confidential guests in Realm
>>> +                 world using RMM, as defined by Arm Confidential
>>> +                 Compute Architecture (CCA)
>>>
>>
>> I'm not sure, but 'realm' would be more appropriate name. rmm (Real Management Monitor) is
>> the term limited to the firmware to me. Besides, we have been using 'realm' broadly in this
>> series like kvm_vm_is_realm(), kvm_realm_state() and so on.
> 
> May be. But we use the "mode" of the hypervisor in question here, mostly. e.g., nvhe, vhe, protected etc. So, I went with RMM (in favor
> of CCA). Does that make sense ? Also, we rely on firmware/arm_rmm for
> the support. So, it kind of make sense to keep it "rmm" ?
> 

Ok, lets keep 'rmm' unless somebody else has better suggestion.

> Cheers
> Suzuki
> 

Thanks,
Gavin

Re: [PATCH v17 04/20] KVM: arm64: Refactor the vcpu_load to allow for VM specific callbacks

[Suzuki K Poulose]

On 10/09/2026 05:00, Gavin Shan wrote:
> Hi Suzuki,
> 
> On 9/9/26 2:22 AM, Suzuki K Poulose wrote:
>> Based on a work by Marc Zyngier
>>
>> To keep the VCPU load/put handling cleaner with the different kinds of 
>> VM types,
>> we are about to introduce VM specific callbacks to do just the right 
>> thing.
>> In preparation for that, make some refactoring to add the change easier.
>>
>> No functional changes intended.
>>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>> ---
>>   arch/arm64/kvm/arm.c | 46 +++++++++++++++++++++++++++-----------------
>>   1 file changed, 28 insertions(+), 18 deletions(-)
>>
> 
> Nitpicks below, none of them are introduced by this patch itself. In 
> either way:
> 
> Reviewed-by: Gavin Shan <gshan@redhat.com>

Thank you !

> 
>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
>> index 67f1ff9bc4fbe..51fc651267157 100644
>> --- a/arch/arm64/kvm/arm.c
>> +++ b/arch/arm64/kvm/arm.c
>> @@ -683,14 +683,11 @@ static bool kvm_vcpu_should_clear_twe(struct 
>> kvm_vcpu *vcpu)
>>       return single_task_running();
>>   }
>> -void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
>> +static void vcpu_prepare_mmu(struct kvm_vcpu *vcpu)
>>   {
>>       struct kvm_s2_mmu *mmu;
>>       int *last_ran;
>> -    if (is_protected_kvm_enabled())
>> -        goto nommu;
>> -
>>       if (vcpu_has_nv(vcpu))
>>           kvm_vcpu_load_hw_mmu(vcpu);
>> @@ -720,10 +717,33 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, 
>> int cpu)
>>           kvm_call_hyp(__kvm_flush_cpu_context, mmu);
>>           *last_ran = vcpu->vcpu_idx;
>>       }
>> +}
>> +
>> +static void vcpu_set_wfx_traps(struct kvm_vcpu *vcpu)
>> +{
>> +    if (kvm_vcpu_should_clear_twe(vcpu))
>> +        vcpu->arch.hcr_el2 &= ~HCR_TWE;
>> +    else
>> +        vcpu->arch.hcr_el2 |= HCR_TWE;
>> +
>> +    if (kvm_vcpu_should_clear_twi(vcpu))
>> +        vcpu->arch.hcr_el2 &= ~HCR_TWI;
>> +    else
>> +        vcpu->arch.hcr_el2 |= HCR_TWI;
>> +}
>> +
> 
> It seems FIELD_MODIFY() can be applied here if no visible constraints 
> exist:
> 
>      FIELD_MODIFY(HCR_TWE, vcpu->arch.hcr_el2, ! 
> kvm_vcpu_should_clear_twe(vcpu));
>      FIELD_MODIFY(HCR_TWI, vcpu->arch.hcr_el2, ! 
> kvm_vcpu_should_clear_twi(vcpu));

I would leave this out for now. Can be a separate patch

> 
>> +static void vcpu_load_pvtime(struct kvm_vcpu *vcpu)
>> +{
>> +    if (kvm_arm_is_pvtime_enabled(&vcpu->arch))
>> +        kvm_make_request(KVM_REQ_RECORD_STEAL, vcpu);
>> +}
>> +
>> +void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
>> +{
>> +    if (!is_protected_kvm_enabled())
>> +        vcpu_prepare_mmu(vcpu);
>>
> 
> I think we probably try to depend on kvm->arch.vm_flavor instead
> of !is_protected_kvm_enabled().
> 
>      struct kvm *kvm = vcpu->kvm;
> 
>      if (kvm_vm_is_protected(kvm) || kvm_vm_flavor_eq(kvm, VM_PKVM))
>          vcpu_prepare_mmu(vcpu);

This will be removed as part of the call back in the next patch.

Cheers
Suzuki

Re: [PATCH v17 10/20] KVM: arm64: coco: Disable Steal time accounting for coco guests

[Suzuki K Poulose]

Hi Fuad

On 09/09/2026 13:23, Fuad Tabba wrote:
> Hi Suzuki,
> 
> On Wed, 9 Sept 2026 at 12:52, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
> ...
> 
>>> I think it would be better to leave KVM_CAP_STEAL_TIME reported and
>>> instead reject the configuration for these VMs, returning -EPERM from
>>> the KVM_ARM_VCPU_PVTIME_CTRL attr. A capability query should report
>>> what the kernel supports, not whether a given VM may use it; gating it
>>> per-VM here is also inconsistent, since the !kvm clause leaves the
>>> system-fd cap at 1 while the VM-fd cap reads 0. And -EPERM says that
>>
>>
>> But isn't that we want from a VMM perspective ? i.e., enable PVTIME if
>> it is supported for the given VM (type). Rather than PVTIME is supported
>> by the KVM, so I can go enable it by default. But, we endup getting a
>> failure while we enable it. I encountered this with kvmtool, where
>> CAP_STEAL_TIME is reported true and even the HAS_DEVICE_ATTR goes
>> through fine, but SET_DEVICE_ATTR seemed to fail.
>>
>> Isn't it better to allow the VM to decide if the capability is supported
>> for the given VM type, like we are doing for other capabilities ?
> 
> You're right, per-VM capability reporting. My wires got crossed from
> the pKVM series and the kvmtool fixes. However, I'd still return
> -EPERM rather than -ENXIO from the attribute. The capability exists,

Just to make sure I understand the picture:

For coco VMs:

  - check extension on coco KVM, KVM_CAP_STEAL_TIME => 0

And a follow up DEVICE_HAS_ATTR => -EPERM ?

But we tell the VMM, look you don't have the capabilit on this VM.
But if you try to set the attribute isn't it more appropriate to
say:
  -ENXIO - No you don't have that feature on this VM (-ENXIO)
rather than
  -EPERM - No, I won't let you do that.

Remember we are doing only in the "VM" context. It is a minor nit
but I wanted to make sure we understand what we are thinking.

Cheers
Suzuki

> it's just not permitted for a confidential VM, and -EPERM says exactly
> that, whereas -ENXIO reads as "no such feature".
> 
> Cheers,
> /fuad
> 
>>
>>> steal-time exists but isn't permitted for a confidential VM, whereas
>>> -ENXIO reads as "no such feature".
>>
>> Cheers
>> Suzuki
>>
>>
>>>
>>> Cheers,
>>> /fuad
>>>
>>>>
>>>>    int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
>>>> @@ -81,7 +81,7 @@ int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
>>>>           int ret = 0;
>>>>           int idx;
>>>>
>>>> -       if (!kvm_arm_pvtime_supported() ||
>>>> +       if (!kvm_arm_pvtime_supported(kvm) ||
>>>>               attr->attr != KVM_ARM_VCPU_PVTIME_IPA)
>>>>                   return -ENXIO;
>>>>
>>>> @@ -110,7 +110,7 @@ int kvm_arm_pvtime_get_attr(struct kvm_vcpu *vcpu,
>>>>           u64 __user *user = (u64 __user *)attr->addr;
>>>>           u64 ipa;
>>>>
>>>> -       if (!kvm_arm_pvtime_supported() ||
>>>> +       if (!kvm_arm_pvtime_supported(vcpu->kvm) ||
>>>>               attr->attr != KVM_ARM_VCPU_PVTIME_IPA)
>>>>                   return -ENXIO;
>>>>
>>>> @@ -126,7 +126,7 @@ int kvm_arm_pvtime_has_attr(struct kvm_vcpu *vcpu,
>>>>    {
>>>>           switch (attr->attr) {
>>>>           case KVM_ARM_VCPU_PVTIME_IPA:
>>>> -               if (kvm_arm_pvtime_supported())
>>>> +               if (kvm_arm_pvtime_supported(vcpu->kvm))
>>>>                           return 0;
>>>>           }
>>>>           return -ENXIO;
>>>> --
>>>> 2.43.0
>>>>
>>

Re: [PATCH v17 16/20] KVM: arm64: CCA: Provide register list for unfinalized RECs

[Suzuki K Poulose]

On 08/09/2026 17:57, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Medium] Mismatch between KVM_GET_REG_LIST implementation for RECs and its commit message / comment.
> --
> 
> commit ce52f6b2727531d94e14139ee2966392538b6218
> Author: Jean-Philippe Brucker <jpb@kernel.org>
> 
> KVM: arm64: CCA: Provide register list for unfinalized RECs
> 
> This commit modifies the KVM_GET_REG_LIST ioctl handler to allow fetching
> the register list for Realm Execution Contexts (RECs) before the VCPUs are
> fully finalized, provided SVE is finalized. It adjusts the size probing and
> array copying logic to prevent -EPERM errors during late REC finalization.
> 
>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
>> --- a/arch/arm64/kvm/arm.c
>> +++ b/arch/arm64/kvm/arm.c
> [ ... ]
>> @@ -1980,6 +1980,18 @@ static int kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,
>>   	return __kvm_arm_vcpu_set_events(vcpu, events);
>>   }
>>   
>> +/*
>> + * Realm VCPUs can be finalized after the Realm descriptor is created.
>> + * Allow the register list for RECs before the VCPUs are finalized, but
>> + * after the SVE is finalized.
>> + */
>> +static bool kvm_arm_vcpu_reg_list_allowed(struct kvm_vcpu *vcpu)
>> +{
>> +	if (kvm_arm_vcpu_is_finalized(vcpu))
>> +		return true;
>> +	return vcpu_is_rec(vcpu);
>> +}
> 
> [Severity: Medium]
> Does this logic in kvm_arm_vcpu_reg_list_allowed() unconditionally allow
> RECs, bypassing the SVE finalization check? The comment explicitly states
> "but after the SVE is finalized," yet if the VCPU is a REC, this path
> returns true regardless of the SVE finalization status.

I have fixed the commit description as below to make it clear.

> 
>> diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
>> --- a/arch/arm64/kvm/guest.c
>> +++ b/arch/arm64/kvm/guest.c
> [ ... ]
>> @@ -616,9 +616,6 @@ static int copy_sve_reg_indices(const struct kvm_vcpu *vcpu,
>>   	if (!vcpu_has_sve(vcpu))
>>   		return 0;
>>   
>> -	/* Policed by KVM_GET_REG_LIST: */
>> -	WARN_ON(!kvm_arm_vcpu_sve_finalized(vcpu));
>> -
>>   	/*
>>   	 * Enumerate this first, so that userspace can save/restore in
>>   	 * the order reported by KVM_GET_REG_LIST:
>> @@ -628,6 +625,9 @@ static int copy_sve_reg_indices(const struct kvm_vcpu *vcpu,
>>   		return -EFAULT;
>>   	++num_regs;
>>   
>> +	if (!kvm_arm_vcpu_sve_finalized(vcpu))
>> +		return num_regs;
>> +
> 
> [Severity: Medium]
> Does this return a positive success code (num_regs) rather than an error?
> 
> The commit message states that "a KVM_GET_REG_LIST call that only probes
> for the array size will now succeed even if SVE is not finalized, but that
> seems harmless since the following KVM_GET_REG_LIST with the full array
> will fail."
> 
> Returning a positive number of registers here in copy_sve_reg_indices()
> causes the full array fetch to succeed silently instead of failing as the
> commit message claims.

I have massaged the commit description as below, as the current one
looks so outdated and vague.

---8>---

KVM: arm64: CCA: Expose SVE VL register before REC finalization 


Userspace must configure the SVE vector length before the Realm is 
created (as it is part of the parameter for Realm creation), but the
Realm VCPUs cannot be finalized until after the Realm Descriptor has
been created.

KVM_GET_REG_LIST currently rejects the unfinalized VCPUs, which prevents
the userspace from discovering and configuring the VLs for  the Realm.

Allow KVM_GET_REG_LIST for unfinalized RECs and make the SVE register 

enumeration handle the unfinalized case explicitly. i.e., only expose
KVM_REG_ARM64_SVE_VLS before SVE is finalized.

One adverse side effect of this change is that a KVM_GET_REG_LIST call
that only probes for the array size will now succeed even if SVE is not
finalized, but that seems harmless since the following KVM_GET_REG_LIST
with the full array will fail.


Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Signed-off-by: Steven Price <steven.price@arm.com>
Reviewed-by: Gavin Shan <gshan@redhat.com> 

Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>



Suzuki

> 
>>   	for (i = 0; i < slices; i++) {
>>   		for (n = 0; n < SVE_NUM_ZREGS; n++) {
>>   			reg = KVM_REG_ARM64_SVE_ZREG(n, i);
> 

Re: [PATCH v17 17/20] KVM: arm64: CCA: Provide an accurate register list

[Suzuki K Poulose]

On 08/09/2026 18:00, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> - [Medium] KVM_GET_REG_LIST fails to restrict core registers to the 9 GP registers for Realm VMs, exposing all core registers instead.
> - [Medium] Realm VM system register restrictions are only applied to KVM_GET_REG_LIST, allowing userspace to bypass them via KVM_SET_ONE_REG and KVM_GET_ONE_REG.
> --
> 
> commit 1a82d37b0064cc3f1fa34c6757432fa355d160a3
> Author: Jean-Philippe Brucker <jpb@kernel.org>
> 
> KVM: arm64: CCA: Provide an accurate register list
>      
> Userspace can set a few registers with KVM_SET_ONE_REG (9 GP registers
> at runtime, and 3 system registers during initialization). Update the
> register list returned by KVM_GET_REG_LIST.
> 
>> Userspace can set a few registers with KVM_SET_ONE_REG (9 GP registers
>> at runtime, and 3 system registers during initialization). Update the
>> register list returned by KVM_GET_REG_LIST.
> 
> [Severity: Medium]
> Does KVM_GET_REG_LIST successfully restrict core registers to just the 9 GP
> registers for Realm VMs?
> 
> Looking at copy_core_reg_indices(), it loops over the entire kvm_regs array
> without checking vcpu_is_rec(vcpu):
> 
> arch/arm64/kvm/guest.c:copy_core_reg_indices() {
> 	for (i = 0; i < sizeof(struct kvm_regs) / sizeof(__u32); i++) {
> 		u64 reg = KVM_REG_ARM64 | KVM_REG_ARM_CORE | i;
> 		int size = core_reg_size_from_offset(vcpu, i);
> 
> 		if (size < 0)
> 			continue;
>          ...
> }
> 
> This appears to expose all core registers to the userspace array rather than
> limiting it to the 9 GP registers mentioned in the commit message.

That is correct, but a later patch actually restricts this for set_reg()
calls. I will pull that into this series and will also fixup the commit
description, potentially folding that one here.

> 
> [ ... ]
> 
>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
>> index 44aae52c473d7..a58c1fc13ea17 100644
>> --- a/arch/arm64/kvm/sys_regs.c
>> +++ b/arch/arm64/kvm/sys_regs.c
> 
> [ ... ]
> 
>> @@ -5693,11 +5693,26 @@ static bool copy_reg_to_user(const struct sys_reg_desc *reg, u64 __user **uind)
>>   	return true;
>>   }
>>   
>> +static inline bool kvm_realm_sys_reg_hidden_user(const struct kvm_vcpu *vcpu,
>> +						 u64 reg)
>> +{
>> +	switch (reg) {
>> +	case SYS_ID_AA64DFR0_EL1:
>> +	case SYS_PMCR_EL0:
>> +		return false;
>> +	}
>> +	return true;
>> +}
>> +
>>   static int walk_one_sys_reg(const struct kvm_vcpu *vcpu,
>>   			    const struct sys_reg_desc *rd,
>>   			    u64 __user **uind,
>>   			    unsigned int *total)
>>   {
>> +	if (vcpu_is_rec(vcpu) &&
>> +	    kvm_realm_sys_reg_hidden_user(vcpu, reg_to_encoding(rd)))
>> +		return 0;
>> +
> 
> [Severity: Medium]
> Are the Realm system register restrictions bypassed by KVM_SET_ONE_REG and
> KVM_GET_ONE_REG?
> 
> While this patch correctly hides the registers from KVM_GET_REG_LIST by
> checking kvm_realm_sys_reg_hidden_user() in walk_one_sys_reg(), it doesn't
> appear to add a similar check to the ioctl handlers for setting or getting
> individual registers.
> 
> For example, in kvm_sys_reg_set_user(), the code only checks sysreg_hidden():

As above, this is in a separate patch, I will fix this in next version.


Suzuki

Re: [PATCH v17 14/20] KVM: arm64: CCA: Don't expose unsupported capabilities for realm guests

[Suzuki K Poulose]

On 08/09/2026 17:52, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Low] Redundant NULL check on the 'kvm' pointer in kvm_vm_ioctl_enable_cap().
> --
> 
> commit 849adae1adab6cdadc3f93c498689cfc52591e9d
> Author: Suzuki K Poulose <suzuki.poulose@arm.com>
> 
> KVM: arm64: CCA: Don't expose unsupported capabilities for realm guests
> 
> This commit restricts the extensions visible to realm guests to an
> allowlist, as RMM v2.0 provides no mechanism for the host to perform
> debug operations on the guest.
> 
>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
>> index 3dfe0e03fab7a..fbb4d6651ab51 100644
>> --- a/arch/arm64/kvm/arm.c
>> +++ b/arch/arm64/kvm/arm.c
> [ ... ]
>> @@ -148,6 +167,9 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
>>   	if (is_protected_kvm_enabled() && !kvm_pkvm_ext_allowed(kvm, cap->cap))
>>   		return -EINVAL;
>>   
>> +	if (kvm && kvm_vm_is_realm(kvm) && !kvm_realm_ext_allowed(cap->cap))
> 
> [Severity: Low]
> Is the kvm && check strictly necessary in kvm_vm_ioctl_enable_cap()?
> 
> Since this function is only called as a VM-level ioctl (dispatched from
> kvm_vm_ioctl() where the pointer is extracted from the file descriptor's
> private data), kvm is structurally guaranteed to be non-NULL in this
> context.
> 
> While kvm_vm_ioctl_check_extension() handles global /dev/kvm calls with a
> NULL kvm pointer, this function does not appear to need the same check.

Ack, I have addressed this now.

Suzuki

Re: [PATCH v17 09/20] KVM: arm64: coco: arch_timer: Prevent timer offset configuration

[Suzuki K Poulose]

On 08/09/2026 17:46, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> 
> Pre-existing issues:
> - [High] Unprivileged host userspace can trigger a kernel WARN (and potential host panic) by setting SYS_CNTVCT_EL0 or SYS_CNTPCT_EL0 via KVM_SET_ONE_REG for a confidential VM.
> --
> 
> commit 563a7eb94c0d2b010c530d4a42b1717738d5106a
> Author: Suzuki K Poulose <suzuki.poulose@arm.com>
> 
> KVM: arm64: coco: arch_timer: Prevent timer offset configuration
>      
> Confidential guests, including protected PKVM guests and Arm CCA Realms, always
> runs with 0 timer offset. Prevent the configuration for them.
> 
>> diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
>> index 6ac3321f4c575..eba84cab95d01 100644
>> --- a/arch/arm64/kvm/arch_timer.c
>> +++ b/arch/arm64/kvm/arch_timer.c
>> @@ -1079,7 +1079,7 @@ static void timer_context_init(struct kvm_vcpu *vcpu, int timerid)
>>   
>>   	ctxt->timer_id = timerid;
>>   
>> -	if (!kvm_vm_is_protected(vcpu->kvm)) {
>> +	if (!vcpu_is_confidential(vcpu)) {
>>   		if (timerid == TIMER_VTIMER)
>>   			ctxt->offset.vm_offset = &kvm->arch.timer_data.voffset;
>>   		else
> 
> [ ... ]
> 
>> @@ -1110,7 +1110,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
>>   		timer_context_init(vcpu, i);
>>   
>>   	/* Synchronize offsets across timers of a VM if not already provided */
>> -	if (!vcpu_is_protected(vcpu) &&
>> +	if (!vcpu_is_confidential(vcpu) &&
>>   	    !test_bit(KVM_ARCH_FLAG_VM_COUNTER_OFFSET, &vcpu->kvm->arch.flags)) {
>>   		timer_set_offset(vcpu_vtimer(vcpu), kvm_phys_timer_read());
>>   		timer_set_offset(vcpu_ptimer(vcpu), 0);
> 
> [Severity: High]
> This is a pre-existing issue, but does this configuration allow unprivileged
> host userspace to trigger a kernel WARN?
> 
> For confidential VMs, ctxt->offset.vm_offset is initialized to NULL in
> timer_context_init(), and KVM_ARCH_FLAG_VM_COUNTER_OFFSET is skipped
> here in kvm_timer_vcpu_init().
> 
> However, if userspace calls KVM_SET_ONE_REG for SYS_CNTVCT_EL0,
> arch_timer_set_user() handles it like this:
> 
> arch/arm64/kvm/sys_regs.c:arch_timer_set_user() {
>      ...
>      case SYS_CNTVCT_EL0:
>          if (!test_bit(KVM_ARCH_FLAG_VM_COUNTER_OFFSET, &vcpu->kvm->arch.flags))
>              timer_set_offset(vcpu_vtimer(vcpu), kvm_phys_timer_read() - val);
>          return 0;
>      ...
> }
> 
> Since the flag is clear for confidential VMs, this will unconditionally invoke
> timer_set_offset() with a userspace-controlled offset.
> 
> In timer_set_offset():
> 
> include/kvm/arm_arch_timer.h:timer_set_offset() {
>      ...
>      if (!ctxt->offset.vm_offset) {
>          WARN(offset, "timer %d\n", arch_timer_ctx_index(ctxt));
>          return;
>      }
>      ...
> }
> 
> Because vm_offset is NULL, this triggers the WARN if the computed offset is
> non-zero. With panic_on_warn enabled (common in cloud environments), could
> this allow unprivileged host userspace to panic the host?

I have added the following patch to the series to fix this :

KVM: arm64: pKVM: Prevent setting CNTVCT_EL0/CNTPCT_EL0 for
  protected VMs

Protected VMs run doesn't allow setting offsets for virtual and phyiscal 
counters.
However we don't prevent the userspace from trying to write to the 
CNTVCT/CNTPCT
registers. This would lead to KVM triggering a WARN() in 
timer_set_offset() as
the vm_offset pointer is set to NULL. Prevent this by denying the writes to
them for protected VMs.

Reported by Sashiko here
  
https://lore.kernel.org/all/20260908164641.416911F00A3A@smtp.kernel.org

Fixes: f7d05ee84a6a ("KVM: arm64: Prevent host from managing timer 
offsets for protected VMs")
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
  arch/arm64/kvm/sys_regs.c | 4 ++++
  1 file changed, 4 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 44aae52c473d7..ba44873799b23 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1787,10 +1787,14 @@ static int arch_timer_set_user(struct kvm_vcpu 
*vcpu,
                 val &= ~ARCH_TIMER_CTRL_IT_STAT;
                 break;
         case SYS_CNTVCT_EL0:
+               if (vcpu_is_protected(vcpu))
+                       return -EPERM;
                 if (!test_bit(KVM_ARCH_FLAG_VM_COUNTER_OFFSET, 
&vcpu->kvm->arch.flags))
                         timer_set_offset(vcpu_vtimer(vcpu), 
kvm_phys_timer_read() - val);
                 return 0;
         case SYS_CNTPCT_EL0:
+               if (vcpu_is_protected(vcpu))
+                       return -EPERM;
                 if (!test_bit(KVM_ARCH_FLAG_VM_COUNTER_OFFSET, 
&vcpu->kvm->arch.flags))
                         timer_set_offset(vcpu_ptimer(vcpu), 
kvm_phys_timer_read() - val);
                 return 0;

Re: [PATCH v17 10/20] KVM: arm64: coco: Disable Steal time accounting for coco guests

[Fuad Tabba]

Hi Suzuki,

On Thu, 10 Sept 2026 at 11:28, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
...
> Just to make sure I understand the picture:
>
> For coco VMs:
>
>   - check extension on coco KVM, KVM_CAP_STEAL_TIME => 0
>
> And a follow up DEVICE_HAS_ATTR => -EPERM ?
>
> But we tell the VMM, look you don't have the capabilit on this VM.
> But if you try to set the attribute isn't it more appropriate to
> say:
>   -ENXIO - No you don't have that feature on this VM (-ENXIO)
> rather than
>   -EPERM - No, I won't let you do that.
>
> Remember we are doing only in the "VM" context. It is a minor nit
> but I wanted to make sure we understand what we are thinking.

-ENXIO it is. I went back to api.rst for KVM_SET_DEVICE_ATTR: ENXIO is
for a group or attribute that's "unknown/unsupported for this device",
and EPERM for one that "cannot (currently) be accessed this way", a
state or access-mode refusal, so a VM type that doesn't have the
feature is the former. KVM_HAS_DEVICE_ATTR only documents ENXIO at
all. Your patch is good as it is.

Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Cheers,
/fuad



>
> Cheers
> Suzuki
>
> > it's just not permitted for a confidential VM, and -EPERM says exactly
> > that, whereas -ENXIO reads as "no such feature".
> >
> > Cheers,
> > /fuad
> >
> >>
> >>> steal-time exists but isn't permitted for a confidential VM, whereas
> >>> -ENXIO reads as "no such feature".
> >>
> >> Cheers
> >> Suzuki
> >>
> >>
> >>>
> >>> Cheers,
> >>> /fuad
> >>>
> >>>>
> >>>>    int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
> >>>> @@ -81,7 +81,7 @@ int kvm_arm_pvtime_set_attr(struct kvm_vcpu *vcpu,
> >>>>           int ret = 0;
> >>>>           int idx;
> >>>>
> >>>> -       if (!kvm_arm_pvtime_supported() ||
> >>>> +       if (!kvm_arm_pvtime_supported(kvm) ||
> >>>>               attr->attr != KVM_ARM_VCPU_PVTIME_IPA)
> >>>>                   return -ENXIO;
> >>>>
> >>>> @@ -110,7 +110,7 @@ int kvm_arm_pvtime_get_attr(struct kvm_vcpu *vcpu,
> >>>>           u64 __user *user = (u64 __user *)attr->addr;
> >>>>           u64 ipa;
> >>>>
> >>>> -       if (!kvm_arm_pvtime_supported() ||
> >>>> +       if (!kvm_arm_pvtime_supported(vcpu->kvm) ||
> >>>>               attr->attr != KVM_ARM_VCPU_PVTIME_IPA)
> >>>>                   return -ENXIO;
> >>>>
> >>>> @@ -126,7 +126,7 @@ int kvm_arm_pvtime_has_attr(struct kvm_vcpu *vcpu,
> >>>>    {
> >>>>           switch (attr->attr) {
> >>>>           case KVM_ARM_VCPU_PVTIME_IPA:
> >>>> -               if (kvm_arm_pvtime_supported())
> >>>> +               if (kvm_arm_pvtime_supported(vcpu->kvm))
> >>>>                           return 0;
> >>>>           }
> >>>>           return -ENXIO;
> >>>> --
> >>>> 2.43.0
> >>>>
> >>
>

Re: [PATCH v17 10/20] KVM: arm64: coco: Disable Steal time accounting for coco guests

[Suzuki K Poulose]

On 10/09/2026 13:42, Fuad Tabba wrote:
> Hi Suzuki,
> 
> On Thu, 10 Sept 2026 at 11:28, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
> ...
>> Just to make sure I understand the picture:
>>
>> For coco VMs:
>>
>>    - check extension on coco KVM, KVM_CAP_STEAL_TIME => 0
>>
>> And a follow up DEVICE_HAS_ATTR => -EPERM ?
>>
>> But we tell the VMM, look you don't have the capabilit on this VM.
>> But if you try to set the attribute isn't it more appropriate to
>> say:
>>    -ENXIO - No you don't have that feature on this VM (-ENXIO)
>> rather than
>>    -EPERM - No, I won't let you do that.
>>
>> Remember we are doing only in the "VM" context. It is a minor nit
>> but I wanted to make sure we understand what we are thinking.
> 
> -ENXIO it is. I went back to api.rst for KVM_SET_DEVICE_ATTR: ENXIO is
> for a group or attribute that's "unknown/unsupported for this device",
> and EPERM for one that "cannot (currently) be accessed this way", a
> state or access-mode refusal, so a VM type that doesn't have the
> feature is the former. KVM_HAS_DEVICE_ATTR only documents ENXIO at
> all. Your patch is good as it is.
> 
> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Cheers,

Suzuki