[PATCH kvmtool 0/5] Fix diagnostics and capability probes for protected VMs

[PATCH kvmtool 0/5] Fix diagnostics and capability probes for protected VMs

[Fuad Tabba]

Hi folks,

These are the kvmtool changes that go with the pKVM series confining a
protected VM's vCPU state to EL2 [1].

That series stops the host reading a protected vCPU's registers once it
has run, and stops it setting the PVTIME attribute. kvmtool dies on
both. Its arm64 register-dump helpers call die() when KVM_GET_ONE_REG
fails, which turns a diagnostic into a VMM abort.
kvm_cpu__setup_pvtime() runs into the new -EPERM and fails vCPU init.

The PVTIME failure is a probe on the wrong fd. kvmtool asks the global
/dev/kvm fd whether steal-time is available, and that fd advertises it
for a protected VM, which cannot use it.
The last patch fixes the same wrong-fd probe for the counter offset,
where "lkvm run --protected --counter-offset" already fails on Linux
7.3-rc1, without the kernel series.
The same conversion went in for the vCPU feature probes in July [2].

Patches 2 and 3 are not specific to protected VMs. They fix the
exit-reason table on the same panic path as patch 1. The lookup ran off
the end of it, and the table had stopped naming anything KVM added after
KVM_EXIT_INTERNAL_ERROR, which on arm64 includes KVM_EXIT_SYSTEM_EVENT
and KVM_EXIT_ARM_NISV.

Patch 1's path needs a protected VM that is already running: "lkvm debug
-d" against one kills the VMM without this series, and prints the errno
with it.

Based on kvmtool master (f67bc0b).

Cheers,
/fuad

[1] https://lore.kernel.org/all/20260831163421.272420-1-fuad.tabba@linux.dev/
[2] https://lore.kernel.org/all/20260714110329.12113-1-fuad.tabba@linux.dev/

Fuad Tabba (5):
  arm64: Do not abort on register-dump failures
  kvm: Bound-check the exit-reason string lookup
  kvm: Name every exit reason the UAPI header defines
  arm64: Query steal-time support on the VM fd
  arm64: Query counter-offset support on the VM fd

 arm64/kvm-cpu.c   | 36 ++++++++++++++++++++++++------------
 arm64/kvm.c       |  2 +-
 arm64/pvtime.c    |  4 ++--
 builtin-run.c     |  2 +-
 include/kvm/kvm.h |  2 +-
 kvm.c             | 37 ++++++++++++++++++++++++++++++++++---
 6 files changed, 63 insertions(+), 20 deletions(-)


base-commit: f67bc0bdae9433a9cfd05e65ea2c1bb6102566d9
-- 
2.39.5

[PATCH kvmtool 1/5] arm64: Do not abort on register-dump failures

[Fuad Tabba]

kvm_cpu__show_registers() and kvm_cpu__show_code() read the vCPU's core
registers with KVM_GET_ONE_REG, and die() if the ioctl fails. Both are
diagnostics. They run from the KVM_EXIT_DEBUG case in kvm_cpu__start(),
from the "lkvm debug -d" dump path in handle_sigusr1(), and from the
panic dump in kvm_cpu_thread().

Once a protected vCPU has run, its registers belong to the guest and
KVM_GET_ONE_REG returns -EPERM. Dying on that turns a diagnostic into a
VMM abort.

Report that the register state is unavailable, with the errno, and
return instead of dying.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arm64/kvm-cpu.c | 36 ++++++++++++++++++++++++------------
 1 file changed, 24 insertions(+), 12 deletions(-)

diff --git a/arm64/kvm-cpu.c b/arm64/kvm-cpu.c
index 3aa7684..c58d61f 100644
--- a/arm64/kvm-cpu.c
+++ b/arm64/kvm-cpu.c
@@ -476,15 +476,19 @@ void kvm_cpu__show_code(struct kvm_cpu *vcpu)
 
 	dprintf(debug_fd, "\n*pc:\n");
 	reg.id = ARM64_CORE_REG(regs.pc);
-	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
-		die("KVM_GET_ONE_REG failed (show_code @ PC)");
+	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
+		pr_err("register state unavailable (pc): %s", strerror(errno));
+		return;
+	}
 
 	kvm__dump_mem(vcpu->kvm, data, 32, debug_fd);
 
 	dprintf(debug_fd, "\n*lr:\n");
 	reg.id = ARM64_CORE_REG(regs.regs[30]);
-	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
-		die("KVM_GET_ONE_REG failed (show_code @ LR)");
+	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
+		pr_err("register state unavailable (lr): %s", strerror(errno));
+		return;
+	}
 
 	kvm__dump_mem(vcpu->kvm, data, 32, debug_fd);
 }
@@ -499,23 +503,31 @@ void kvm_cpu__show_registers(struct kvm_cpu *vcpu)
 	dprintf(debug_fd, "\n Registers:\n");
 
 	reg.id		= ARM64_CORE_REG(regs.pc);
-	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
-		die("KVM_GET_ONE_REG failed (pc)");
+	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
+		pr_err("register state unavailable (pc): %s", strerror(errno));
+		return;
+	}
 	dprintf(debug_fd, " PC:    0x%lx\n", data);
 
 	reg.id		= ARM64_CORE_REG(regs.pstate);
-	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
-		die("KVM_GET_ONE_REG failed (pstate)");
+	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
+		pr_err("register state unavailable (pstate): %s", strerror(errno));
+		return;
+	}
 	dprintf(debug_fd, " PSTATE:    0x%lx\n", data);
 
 	reg.id		= ARM64_CORE_REG(sp_el1);
-	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
-		die("KVM_GET_ONE_REG failed (sp_el1)");
+	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
+		pr_err("register state unavailable (sp_el1): %s", strerror(errno));
+		return;
+	}
 	dprintf(debug_fd, " SP_EL1:    0x%lx\n", data);
 
 	reg.id		= ARM64_CORE_REG(regs.regs[30]);
-	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
-		die("KVM_GET_ONE_REG failed (lr)");
+	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
+		pr_err("register state unavailable (lr): %s", strerror(errno));
+		return;
+	}
 	dprintf(debug_fd, " LR:    0x%lx\n", data);
 }
 
-- 
2.39.5

[PATCH kvmtool 2/5] kvm: Bound-check the exit-reason string lookup

[Fuad Tabba]

kvm_cpu_thread()'s panic path uses the kernel's exit_reason to index
kvm_exit_reasons[] without checking it against the array's size. The
table does not cover every exit reason KVM can return, so a reason past
its last entry reads out of bounds. KVM_EXIT_SYSTEM_EVENT and
KVM_EXIT_ARM_NISV are both past it.

Move the table behind kvm__exit_reason_str(), which checks the index
first. The array now has a single caller inside kvm.c, so make it
static const.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 builtin-run.c     |  2 +-
 include/kvm/kvm.h |  2 +-
 kvm.c             | 10 +++++++++-
 3 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/builtin-run.c b/builtin-run.c
index 81f255f..f36d3e2 100644
--- a/builtin-run.c
+++ b/builtin-run.c
@@ -298,7 +298,7 @@ static void *kvm_cpu_thread(void *arg)
 panic_kvm:
 	pr_err("KVM exit reason: %u (\"%s\")",
 		current_kvm_cpu->kvm_run->exit_reason,
-		kvm_exit_reasons[current_kvm_cpu->kvm_run->exit_reason]);
+		kvm__exit_reason_str(current_kvm_cpu->kvm_run->exit_reason));
 
 	if (current_kvm_cpu->kvm_run->exit_reason == KVM_EXIT_UNKNOWN) {
 		pr_err("KVM exit code: %llu",
diff --git a/include/kvm/kvm.h b/include/kvm/kvm.h
index a9376b6..8619070 100644
--- a/include/kvm/kvm.h
+++ b/include/kvm/kvm.h
@@ -255,7 +255,7 @@ int kvm__for_each_mem_bank(struct kvm *kvm, enum kvm_mem_type type,
  */
 void kvm__dump_mem(struct kvm *kvm, unsigned long addr, unsigned long size, int debug_fd);
 
-extern const char *kvm_exit_reasons[];
+const char *kvm__exit_reason_str(__u32 exit_reason);
 
 static inline bool host_ptr_in_ram(struct kvm *kvm, void *p)
 {
diff --git a/kvm.c b/kvm.c
index 96583f9..b416f6f 100644
--- a/kvm.c
+++ b/kvm.c
@@ -33,7 +33,7 @@
 
 #define DEFINE_KVM_EXIT_REASON(reason) [reason] = #reason
 
-const char *kvm_exit_reasons[] = {
+static const char * const kvm_exit_reasons[] = {
 	DEFINE_KVM_EXIT_REASON(KVM_EXIT_UNKNOWN),
 	DEFINE_KVM_EXIT_REASON(KVM_EXIT_EXCEPTION),
 	DEFINE_KVM_EXIT_REASON(KVM_EXIT_IO),
@@ -57,6 +57,14 @@ const char *kvm_exit_reasons[] = {
 #endif
 };
 
+const char *kvm__exit_reason_str(__u32 exit_reason)
+{
+	if (exit_reason >= ARRAY_SIZE(kvm_exit_reasons))
+		return "UNKNOWN";
+
+	return kvm_exit_reasons[exit_reason];
+}
+
 static int pause_event;
 static DEFINE_MUTEX(pause_lock);
 static struct kvm_cpu *pause_req_cpu;
-- 
2.39.5

[PATCH kvmtool 3/5] kvm: Name every exit reason the UAPI header defines

[Fuad Tabba]

kvm_exit_reasons[] stops at KVM_EXIT_INTERNAL_ERROR, so the panic path
prints "UNKNOWN" for every reason KVM has gained since. On arm64 that
covers KVM_EXIT_SYSTEM_EVENT and KVM_EXIT_ARM_NISV, and the newer
KVM_EXIT_MEMORY_FAULT, KVM_EXIT_ARM_SEA and KVM_EXIT_ARM_LDST64B.

Name the rest of the reasons the synced uapi header defines, up to
KVM_EXIT_SNP_REQ_CERTS. KVM_EXIT_PAPR_HCALL also loses its CONFIG_PPC64
guard, which the build never defines, so that entry was missing from
every configuration. The neighbouring PPC reasons are unconditional
already.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 kvm.c | 27 +++++++++++++++++++++++++--
 1 file changed, 25 insertions(+), 2 deletions(-)

diff --git a/kvm.c b/kvm.c
index b416f6f..828da64 100644
--- a/kvm.c
+++ b/kvm.c
@@ -52,9 +52,32 @@ static const char * const kvm_exit_reasons[] = {
 	DEFINE_KVM_EXIT_REASON(KVM_EXIT_DCR),
 	DEFINE_KVM_EXIT_REASON(KVM_EXIT_NMI),
 	DEFINE_KVM_EXIT_REASON(KVM_EXIT_INTERNAL_ERROR),
-#ifdef CONFIG_PPC64
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_OSI),
 	DEFINE_KVM_EXIT_REASON(KVM_EXIT_PAPR_HCALL),
-#endif
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_S390_UCONTROL),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_WATCHDOG),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_S390_TSCH),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_EPR),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_SYSTEM_EVENT),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_S390_STSI),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_IOAPIC_EOI),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_HYPERV),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_ARM_NISV),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_X86_RDMSR),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_X86_WRMSR),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_DIRTY_RING_FULL),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_AP_RESET_HOLD),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_X86_BUS_LOCK),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_XEN),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_RISCV_SBI),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_RISCV_CSR),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_NOTIFY),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_LOONGARCH_IOCSR),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_MEMORY_FAULT),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_TDX),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_ARM_SEA),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_ARM_LDST64B),
+	DEFINE_KVM_EXIT_REASON(KVM_EXIT_SNP_REQ_CERTS),
 };
 
 const char *kvm__exit_reason_str(__u32 exit_reason)
-- 
2.39.5

[PATCH kvmtool 4/5] arm64: Query steal-time support on the VM fd

[Fuad Tabba]

kvm_cpu__setup_pvtime() probes KVM_CAP_STEAL_TIME with
kvm__supports_extension(), which issues KVM_CHECK_EXTENSION on the
global /dev/kvm fd. That reports the host's raw capabilities, unaware of
any per-VM restrictions.

pKVM does not offer steal-time to a protected VM, and reflects that on
the VM fd alone. kvmtool goes on to KVM_HAS_DEVICE_ATTR, the kernel
refuses the PVTIME attribute with -EPERM, and kvm_cpu__arch_init() dies:

  Fatal: Unable to initialise vcpu

Query it on the VM fd via kvm__supports_vm_extension(), as
commit 84464ba0246b ("arm64: Query per-VM capabilities when selecting
vCPU features") did for the vCPU feature probes. kvmtool then takes the
no_pvtime path it already has when steal-time is unsupported.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arm64/pvtime.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arm64/pvtime.c b/arm64/pvtime.c
index 2933ac7..839aa8a 100644
--- a/arm64/pvtime.c
+++ b/arm64/pvtime.c
@@ -58,8 +58,8 @@ int kvm_cpu__setup_pvtime(struct kvm_cpu *vcpu)
 	if (kvm_cfg->no_pvtime)
 		return 0;
 
-	has_stolen_time = kvm__supports_extension(vcpu->kvm,
-						  KVM_CAP_STEAL_TIME);
+	has_stolen_time = kvm__supports_vm_extension(vcpu->kvm,
+						     KVM_CAP_STEAL_TIME);
 	if (!has_stolen_time) {
 		kvm_cfg->no_pvtime = true;
 		return 0;
-- 
2.39.5

[PATCH kvmtool 5/5] arm64: Query counter-offset support on the VM fd

[Fuad Tabba]

kvm__arch_set_counter_offset() probes KVM_CAP_COUNTER_OFFSET with
kvm__supports_extension(), which issues KVM_CHECK_EXTENSION on the
global /dev/kvm fd. That reports the host's raw capabilities, unaware of
any per-VM restrictions.

pKVM does not offer the counter offset to a protected VM, and reflects
that on the VM fd alone. kvmtool's own check passes, and the
KVM_ARM_SET_COUNTER_OFFSET that follows is refused by
commit b12b3b04f6ba ("KVM: arm64: Check whether a VM IOCTL is allowed
in pKVM"):

  KVM_ARM_SET_COUNTER_OFFSET: Invalid argument

Query it on the VM fd via kvm__supports_vm_extension(), so
--counter-offset on a protected VM fails with kvmtool's own message:

  Fatal: No support for global counter offset

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arm64/kvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arm64/kvm.c b/arm64/kvm.c
index 36b3284..0bb3535 100644
--- a/arm64/kvm.c
+++ b/arm64/kvm.c
@@ -155,7 +155,7 @@ static void kvm__arch_set_counter_offset(struct kvm *kvm)
 	if (!kvm->cfg.arch.counter_offset)
 		return;
 
-	if (!kvm__supports_extension(kvm, KVM_CAP_COUNTER_OFFSET))
+	if (!kvm__supports_vm_extension(kvm, KVM_CAP_COUNTER_OFFSET))
 		die("No support for global counter offset");
 
 	if (ioctl(kvm->vm_fd, KVM_ARM_SET_COUNTER_OFFSET, &offset))
-- 
2.39.5

Re: [PATCH kvmtool 4/5] arm64: Query steal-time support on the VM fd

[Suzuki K Poulose]

On 31/08/2026 20:24, Fuad Tabba wrote:
> kvm_cpu__setup_pvtime() probes KVM_CAP_STEAL_TIME with
> kvm__supports_extension(), which issues KVM_CHECK_EXTENSION on the
> global /dev/kvm fd. That reports the host's raw capabilities, unaware of
> any per-VM restrictions.
> 
> pKVM does not offer steal-time to a protected VM, and reflects that on
> the VM fd alone. kvmtool goes on to KVM_HAS_DEVICE_ATTR, the kernel
> refuses the PVTIME attribute with -EPERM, and kvm_cpu__arch_init() dies:
> 
>    Fatal: Unable to initialise vcpu
> 
> Query it on the VM fd via kvm__supports_vm_extension(), as
> commit 84464ba0246b ("arm64: Query per-VM capabilities when selecting
> vCPU features") did for the vCPU feature probes. kvmtool then takes the
> no_pvtime path it already has when steal-time is unsupported.
> 
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>

> ---
>   arm64/pvtime.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/arm64/pvtime.c b/arm64/pvtime.c
> index 2933ac7..839aa8a 100644
> --- a/arm64/pvtime.c
> +++ b/arm64/pvtime.c
> @@ -58,8 +58,8 @@ int kvm_cpu__setup_pvtime(struct kvm_cpu *vcpu)
>   	if (kvm_cfg->no_pvtime)
>   		return 0;
>   
> -	has_stolen_time = kvm__supports_extension(vcpu->kvm,
> -						  KVM_CAP_STEAL_TIME);
> +	has_stolen_time = kvm__supports_vm_extension(vcpu->kvm,
> +						     KVM_CAP_STEAL_TIME);
>   	if (!has_stolen_time) {
>   		kvm_cfg->no_pvtime = true;
>   		return 0;

Re: [PATCH kvmtool 5/5] arm64: Query counter-offset support on the VM fd

[Suzuki K Poulose]

On 31/08/2026 20:24, Fuad Tabba wrote:
> kvm__arch_set_counter_offset() probes KVM_CAP_COUNTER_OFFSET with
> kvm__supports_extension(), which issues KVM_CHECK_EXTENSION on the
> global /dev/kvm fd. That reports the host's raw capabilities, unaware of
> any per-VM restrictions.
> 
> pKVM does not offer the counter offset to a protected VM, and reflects
> that on the VM fd alone. kvmtool's own check passes, and the
> KVM_ARM_SET_COUNTER_OFFSET that follows is refused by
> commit b12b3b04f6ba ("KVM: arm64: Check whether a VM IOCTL is allowed
> in pKVM"):
> 
>    KVM_ARM_SET_COUNTER_OFFSET: Invalid argument
> 
> Query it on the VM fd via kvm__supports_vm_extension(), so
> --counter-offset on a protected VM fails with kvmtool's own message:
> 
>    Fatal: No support for global counter offset
> 
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>


> ---
>   arm64/kvm.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arm64/kvm.c b/arm64/kvm.c
> index 36b3284..0bb3535 100644
> --- a/arm64/kvm.c
> +++ b/arm64/kvm.c
> @@ -155,7 +155,7 @@ static void kvm__arch_set_counter_offset(struct kvm *kvm)
>   	if (!kvm->cfg.arch.counter_offset)
>   		return;
>   
> -	if (!kvm__supports_extension(kvm, KVM_CAP_COUNTER_OFFSET))
> +	if (!kvm__supports_vm_extension(kvm, KVM_CAP_COUNTER_OFFSET))
>   		die("No support for global counter offset");
>   
>   	if (ioctl(kvm->vm_fd, KVM_ARM_SET_COUNTER_OFFSET, &offset))

Re: [PATCH kvmtool 1/5] arm64: Do not abort on register-dump failures

[Suzuki K Poulose]

On 31/08/2026 20:24, Fuad Tabba wrote:
> kvm_cpu__show_registers() and kvm_cpu__show_code() read the vCPU's core
> registers with KVM_GET_ONE_REG, and die() if the ioctl fails. Both are
> diagnostics. They run from the KVM_EXIT_DEBUG case in kvm_cpu__start(),
> from the "lkvm debug -d" dump path in handle_sigusr1(), and from the
> panic dump in kvm_cpu_thread().
> 
> Once a protected vCPU has run, its registers belong to the guest and
> KVM_GET_ONE_REG returns -EPERM. Dying on that turns a diagnostic into a
> VMM abort.

Marc once suggested that these could always succeed with junk values for
protected/Realm VMs. Do you think that is an option for pKVM ?


If we go with this approach :

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>


> 
> Report that the register state is unavailable, with the errno, and
> return instead of dying.
> 
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
> ---
>   arm64/kvm-cpu.c | 36 ++++++++++++++++++++++++------------
>   1 file changed, 24 insertions(+), 12 deletions(-)
> 
> diff --git a/arm64/kvm-cpu.c b/arm64/kvm-cpu.c
> index 3aa7684..c58d61f 100644
> --- a/arm64/kvm-cpu.c
> +++ b/arm64/kvm-cpu.c
> @@ -476,15 +476,19 @@ void kvm_cpu__show_code(struct kvm_cpu *vcpu)
>   
>   	dprintf(debug_fd, "\n*pc:\n");
>   	reg.id = ARM64_CORE_REG(regs.pc);
> -	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> -		die("KVM_GET_ONE_REG failed (show_code @ PC)");
> +	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> +		pr_err("register state unavailable (pc): %s", strerror(errno));
> +		return;
> +	}

>   
>   	kvm__dump_mem(vcpu->kvm, data, 32, debug_fd);
>   
>   	dprintf(debug_fd, "\n*lr:\n");
>   	reg.id = ARM64_CORE_REG(regs.regs[30]);
> -	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> -		die("KVM_GET_ONE_REG failed (show_code @ LR)");
> +	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> +		pr_err("register state unavailable (lr): %s", strerror(errno));
> +		return;
> +	}
>   
>   	kvm__dump_mem(vcpu->kvm, data, 32, debug_fd);
>   }
> @@ -499,23 +503,31 @@ void kvm_cpu__show_registers(struct kvm_cpu *vcpu)
>   	dprintf(debug_fd, "\n Registers:\n");
>   
>   	reg.id		= ARM64_CORE_REG(regs.pc);
> -	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> -		die("KVM_GET_ONE_REG failed (pc)");
> +	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> +		pr_err("register state unavailable (pc): %s", strerror(errno));
> +		return;
> +	}
>   	dprintf(debug_fd, " PC:    0x%lx\n", data);
>   
>   	reg.id		= ARM64_CORE_REG(regs.pstate);
> -	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> -		die("KVM_GET_ONE_REG failed (pstate)");
> +	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> +		pr_err("register state unavailable (pstate): %s", strerror(errno));
> +		return;
> +	}
>   	dprintf(debug_fd, " PSTATE:    0x%lx\n", data);
>   
>   	reg.id		= ARM64_CORE_REG(sp_el1);
> -	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> -		die("KVM_GET_ONE_REG failed (sp_el1)");
> +	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> +		pr_err("register state unavailable (sp_el1): %s", strerror(errno));
> +		return;
> +	}
>   	dprintf(debug_fd, " SP_EL1:    0x%lx\n", data);
>   
>   	reg.id		= ARM64_CORE_REG(regs.regs[30]);
> -	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> -		die("KVM_GET_ONE_REG failed (lr)");
> +	if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> +		pr_err("register state unavailable (lr): %s", strerror(errno));
> +		return;
> +	}
>   	dprintf(debug_fd, " LR:    0x%lx\n", data);
>   }
>   

Re: [PATCH kvmtool 2/5] kvm: Bound-check the exit-reason string lookup

[Suzuki K Poulose]

On 31/08/2026 20:24, Fuad Tabba wrote:
> kvm_cpu_thread()'s panic path uses the kernel's exit_reason to index
> kvm_exit_reasons[] without checking it against the array's size. The
> table does not cover every exit reason KVM can return, so a reason past
> its last entry reads out of bounds. KVM_EXIT_SYSTEM_EVENT and
> KVM_EXIT_ARM_NISV are both past it.
> 
> Move the table behind kvm__exit_reason_str(), which checks the index
> first. The array now has a single caller inside kvm.c, so make it
> static const.
> 
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>

> ---
>   builtin-run.c     |  2 +-
>   include/kvm/kvm.h |  2 +-
>   kvm.c             | 10 +++++++++-
>   3 files changed, 11 insertions(+), 3 deletions(-)
> 
> diff --git a/builtin-run.c b/builtin-run.c
> index 81f255f..f36d3e2 100644
> --- a/builtin-run.c
> +++ b/builtin-run.c
> @@ -298,7 +298,7 @@ static void *kvm_cpu_thread(void *arg)
>   panic_kvm:
>   	pr_err("KVM exit reason: %u (\"%s\")",
>   		current_kvm_cpu->kvm_run->exit_reason,
> -		kvm_exit_reasons[current_kvm_cpu->kvm_run->exit_reason]);
> +		kvm__exit_reason_str(current_kvm_cpu->kvm_run->exit_reason));
>   
>   	if (current_kvm_cpu->kvm_run->exit_reason == KVM_EXIT_UNKNOWN) {
>   		pr_err("KVM exit code: %llu",
> diff --git a/include/kvm/kvm.h b/include/kvm/kvm.h
> index a9376b6..8619070 100644
> --- a/include/kvm/kvm.h
> +++ b/include/kvm/kvm.h
> @@ -255,7 +255,7 @@ int kvm__for_each_mem_bank(struct kvm *kvm, enum kvm_mem_type type,
>    */
>   void kvm__dump_mem(struct kvm *kvm, unsigned long addr, unsigned long size, int debug_fd);
>   
> -extern const char *kvm_exit_reasons[];
> +const char *kvm__exit_reason_str(__u32 exit_reason);
>   
>   static inline bool host_ptr_in_ram(struct kvm *kvm, void *p)
>   {
> diff --git a/kvm.c b/kvm.c
> index 96583f9..b416f6f 100644
> --- a/kvm.c
> +++ b/kvm.c
> @@ -33,7 +33,7 @@
>   
>   #define DEFINE_KVM_EXIT_REASON(reason) [reason] = #reason
>   
> -const char *kvm_exit_reasons[] = {
> +static const char * const kvm_exit_reasons[] = {
>   	DEFINE_KVM_EXIT_REASON(KVM_EXIT_UNKNOWN),
>   	DEFINE_KVM_EXIT_REASON(KVM_EXIT_EXCEPTION),
>   	DEFINE_KVM_EXIT_REASON(KVM_EXIT_IO),
> @@ -57,6 +57,14 @@ const char *kvm_exit_reasons[] = {
>   #endif
>   };
>   
> +const char *kvm__exit_reason_str(__u32 exit_reason)
> +{
> +	if (exit_reason >= ARRAY_SIZE(kvm_exit_reasons))
> +		return "UNKNOWN";
> +
> +	return kvm_exit_reasons[exit_reason];
> +}
> +
>   static int pause_event;
>   static DEFINE_MUTEX(pause_lock);
>   static struct kvm_cpu *pause_req_cpu;

Re: [PATCH kvmtool 3/5] kvm: Name every exit reason the UAPI header defines

[Suzuki K Poulose]

On 31/08/2026 20:24, Fuad Tabba wrote:
> kvm_exit_reasons[] stops at KVM_EXIT_INTERNAL_ERROR, so the panic path
> prints "UNKNOWN" for every reason KVM has gained since. On arm64 that
> covers KVM_EXIT_SYSTEM_EVENT and KVM_EXIT_ARM_NISV, and the newer
> KVM_EXIT_MEMORY_FAULT, KVM_EXIT_ARM_SEA and KVM_EXIT_ARM_LDST64B.
> 
> Name the rest of the reasons the synced uapi header defines, up to
> KVM_EXIT_SNP_REQ_CERTS. KVM_EXIT_PAPR_HCALL also loses its CONFIG_PPC64
> guard, which the build never defines, so that entry was missing from
> every configuration. The neighbouring PPC reasons are unconditional
> already.
> 
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>

Re: [PATCH kvmtool 1/5] arm64: Do not abort on register-dump failures

[Fuad Tabba]

Hi Suzuki,

On Sat, 12 Sept 2026 at 08:33, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>
> On 31/08/2026 20:24, Fuad Tabba wrote:
> > kvm_cpu__show_registers() and kvm_cpu__show_code() read the vCPU's core
> > registers with KVM_GET_ONE_REG, and die() if the ioctl fails. Both are
> > diagnostics. They run from the KVM_EXIT_DEBUG case in kvm_cpu__start(),
> > from the "lkvm debug -d" dump path in handle_sigusr1(), and from the
> > panic dump in kvm_cpu_thread().
> >
> > Once a protected vCPU has run, its registers belong to the guest and
> > KVM_GET_ONE_REG returns -EPERM. Dying on that turns a diagnostic into a
> > VMM abort.
>
> Marc once suggested that these could always succeed with junk values for
> protected/Realm VMs. Do you think that is an option for pKVM ?

It's possible, the host copy is still there, but I'd rather not. Once
the vCPU has run that copy is a mix rather than junk: the boot state
the VMM wrote, plus what the exit handlers copy out (the PSTATE mode,
x0 for an MMIO write, x0 to x2 for a forwarded PSCI call). A dump
would show a live PSTATE beside the PC from boot, and the VMM has no
way to distinguish them. With the error it prints that the state is
unavailable, which is what this patch does.

It's also what the tree does for protected state elsewhere: s390
returns -EINVAL from KVM_GET/SET_ONE_REG on a protected VM, and x86
does the same from the register ioctls for the SEV-ES VM type, SNP and
TDX. Legacy SEV-ES is the one place that succeeds silently, kept for
backwards compatibility, and the commit that added the errors
(517987e3fb19) describes that as a problem. Sean's rationale for TDX:
KVM can't provide sane data, so it's userspace's job not to ask for it
[1].

>
> If we go with this approach :
>
> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>

Thanks, I'll add the tag. :)

Cheers,
/fuad

[1] https://lore.kernel.org/all/482264f17fa0652faad9bd5364d652d11cb2ecb8.1625186503.git.isaku.yamahata@intel.com/

>
>
> >
> > Report that the register state is unavailable, with the errno, and
> > return instead of dying.
> >
> > Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
> > ---
> >   arm64/kvm-cpu.c | 36 ++++++++++++++++++++++++------------
> >   1 file changed, 24 insertions(+), 12 deletions(-)
> >
> > diff --git a/arm64/kvm-cpu.c b/arm64/kvm-cpu.c
> > index 3aa7684..c58d61f 100644
> > --- a/arm64/kvm-cpu.c
> > +++ b/arm64/kvm-cpu.c
> > @@ -476,15 +476,19 @@ void kvm_cpu__show_code(struct kvm_cpu *vcpu)
> >
> >       dprintf(debug_fd, "\n*pc:\n");
> >       reg.id = ARM64_CORE_REG(regs.pc);
> > -     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> > -             die("KVM_GET_ONE_REG failed (show_code @ PC)");
> > +     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> > +             pr_err("register state unavailable (pc): %s", strerror(errno));
> > +             return;
> > +     }
>
> >
> >       kvm__dump_mem(vcpu->kvm, data, 32, debug_fd);
> >
> >       dprintf(debug_fd, "\n*lr:\n");
> >       reg.id = ARM64_CORE_REG(regs.regs[30]);
> > -     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> > -             die("KVM_GET_ONE_REG failed (show_code @ LR)");
> > +     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> > +             pr_err("register state unavailable (lr): %s", strerror(errno));
> > +             return;
> > +     }
> >
> >       kvm__dump_mem(vcpu->kvm, data, 32, debug_fd);
> >   }
> > @@ -499,23 +503,31 @@ void kvm_cpu__show_registers(struct kvm_cpu *vcpu)
> >       dprintf(debug_fd, "\n Registers:\n");
> >
> >       reg.id          = ARM64_CORE_REG(regs.pc);
> > -     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> > -             die("KVM_GET_ONE_REG failed (pc)");
> > +     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> > +             pr_err("register state unavailable (pc): %s", strerror(errno));
> > +             return;
> > +     }
> >       dprintf(debug_fd, " PC:    0x%lx\n", data);
> >
> >       reg.id          = ARM64_CORE_REG(regs.pstate);
> > -     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> > -             die("KVM_GET_ONE_REG failed (pstate)");
> > +     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> > +             pr_err("register state unavailable (pstate): %s", strerror(errno));
> > +             return;
> > +     }
> >       dprintf(debug_fd, " PSTATE:    0x%lx\n", data);
> >
> >       reg.id          = ARM64_CORE_REG(sp_el1);
> > -     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> > -             die("KVM_GET_ONE_REG failed (sp_el1)");
> > +     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> > +             pr_err("register state unavailable (sp_el1): %s", strerror(errno));
> > +             return;
> > +     }
> >       dprintf(debug_fd, " SP_EL1:    0x%lx\n", data);
> >
> >       reg.id          = ARM64_CORE_REG(regs.regs[30]);
> > -     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> > -             die("KVM_GET_ONE_REG failed (lr)");
> > +     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> > +             pr_err("register state unavailable (lr): %s", strerror(errno));
> > +             return;
> > +     }
> >       dprintf(debug_fd, " LR:    0x%lx\n", data);
> >   }
> >
>

Re: [PATCH kvmtool 1/5] arm64: Do not abort on register-dump failures

[Fuad Tabba]

On Sat, 12 Sept 2026 at 14:54, Fuad Tabba <fuad.tabba@linux.dev> wrote:
>
> Hi Suzuki,
>
> On Sat, 12 Sept 2026 at 08:33, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
> >
> > On 31/08/2026 20:24, Fuad Tabba wrote:
> > > kvm_cpu__show_registers() and kvm_cpu__show_code() read the vCPU's core
> > > registers with KVM_GET_ONE_REG, and die() if the ioctl fails. Both are
> > > diagnostics. They run from the KVM_EXIT_DEBUG case in kvm_cpu__start(),
> > > from the "lkvm debug -d" dump path in handle_sigusr1(), and from the
> > > panic dump in kvm_cpu_thread().
> > >
> > > Once a protected vCPU has run, its registers belong to the guest and
> > > KVM_GET_ONE_REG returns -EPERM. Dying on that turns a diagnostic into a
> > > VMM abort.
> >
> > Marc once suggested that these could always succeed with junk values for
> > protected/Realm VMs. Do you think that is an option for pKVM ?
>
> It's possible, the host copy is still there, but I'd rather not. Once
> the vCPU has run that copy is a mix rather than junk: the boot state
> the VMM wrote, plus what the exit handlers copy out (the PSTATE mode,
> x0 for an MMIO write, x0 to x2 for a forwarded PSCI call). A dump
> would show a live PSTATE beside the PC from boot, and the VMM has no
> way to distinguish them. With the error it prints that the state is
> unavailable, which is what this patch does.
>
> It's also what the tree does for protected state elsewhere: s390
> returns -EINVAL from KVM_GET/SET_ONE_REG on a protected VM, and x86
> does the same from the register ioctls for the SEV-ES VM type, SNP and
> TDX. Legacy SEV-ES is the one place that succeeds silently, kept for
> backwards compatibility, and the commit that added the errors
> (517987e3fb19) describes that as a problem. Sean's rationale for TDX:
> KVM can't provide sane data, so it's userspace's job not to ask for it
> [1].
>
> >
> > If we go with this approach :
> >
> > Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>
> Thanks, I'll add the tag. :)

Unless I misunderstood what "this" refers to exactly, please shout if
that's the case, or continue the discussion :)

/fuad

>
> Cheers,
> /fuad
>
> [1] https://lore.kernel.org/all/482264f17fa0652faad9bd5364d652d11cb2ecb8.1625186503.git.isaku.yamahata@intel.com/
>
> >
> >
> > >
> > > Report that the register state is unavailable, with the errno, and
> > > return instead of dying.
> > >
> > > Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
> > > ---
> > >   arm64/kvm-cpu.c | 36 ++++++++++++++++++++++++------------
> > >   1 file changed, 24 insertions(+), 12 deletions(-)
> > >
> > > diff --git a/arm64/kvm-cpu.c b/arm64/kvm-cpu.c
> > > index 3aa7684..c58d61f 100644
> > > --- a/arm64/kvm-cpu.c
> > > +++ b/arm64/kvm-cpu.c
> > > @@ -476,15 +476,19 @@ void kvm_cpu__show_code(struct kvm_cpu *vcpu)
> > >
> > >       dprintf(debug_fd, "\n*pc:\n");
> > >       reg.id = ARM64_CORE_REG(regs.pc);
> > > -     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> > > -             die("KVM_GET_ONE_REG failed (show_code @ PC)");
> > > +     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> > > +             pr_err("register state unavailable (pc): %s", strerror(errno));
> > > +             return;
> > > +     }
> >
> > >
> > >       kvm__dump_mem(vcpu->kvm, data, 32, debug_fd);
> > >
> > >       dprintf(debug_fd, "\n*lr:\n");
> > >       reg.id = ARM64_CORE_REG(regs.regs[30]);
> > > -     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> > > -             die("KVM_GET_ONE_REG failed (show_code @ LR)");
> > > +     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> > > +             pr_err("register state unavailable (lr): %s", strerror(errno));
> > > +             return;
> > > +     }
> > >
> > >       kvm__dump_mem(vcpu->kvm, data, 32, debug_fd);
> > >   }
> > > @@ -499,23 +503,31 @@ void kvm_cpu__show_registers(struct kvm_cpu *vcpu)
> > >       dprintf(debug_fd, "\n Registers:\n");
> > >
> > >       reg.id          = ARM64_CORE_REG(regs.pc);
> > > -     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> > > -             die("KVM_GET_ONE_REG failed (pc)");
> > > +     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> > > +             pr_err("register state unavailable (pc): %s", strerror(errno));
> > > +             return;
> > > +     }
> > >       dprintf(debug_fd, " PC:    0x%lx\n", data);
> > >
> > >       reg.id          = ARM64_CORE_REG(regs.pstate);
> > > -     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> > > -             die("KVM_GET_ONE_REG failed (pstate)");
> > > +     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> > > +             pr_err("register state unavailable (pstate): %s", strerror(errno));
> > > +             return;
> > > +     }
> > >       dprintf(debug_fd, " PSTATE:    0x%lx\n", data);
> > >
> > >       reg.id          = ARM64_CORE_REG(sp_el1);
> > > -     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> > > -             die("KVM_GET_ONE_REG failed (sp_el1)");
> > > +     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> > > +             pr_err("register state unavailable (sp_el1): %s", strerror(errno));
> > > +             return;
> > > +     }
> > >       dprintf(debug_fd, " SP_EL1:    0x%lx\n", data);
> > >
> > >       reg.id          = ARM64_CORE_REG(regs.regs[30]);
> > > -     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0)
> > > -             die("KVM_GET_ONE_REG failed (lr)");
> > > +     if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, &reg) < 0) {
> > > +             pr_err("register state unavailable (lr): %s", strerror(errno));
> > > +             return;
> > > +     }
> > >       dprintf(debug_fd, " LR:    0x%lx\n", data);
> > >   }
> > >
> >

Re: [PATCH kvmtool 1/5] arm64: Do not abort on register-dump failures

[Suzuki K Poulose]

On 12/09/2026 15:02, Fuad Tabba wrote:
> On Sat, 12 Sept 2026 at 14:54, Fuad Tabba <fuad.tabba@linux.dev> wrote:
>>
>> Hi Suzuki,
>>
>> On Sat, 12 Sept 2026 at 08:33, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
>>>
>>> On 31/08/2026 20:24, Fuad Tabba wrote:
>>>> kvm_cpu__show_registers() and kvm_cpu__show_code() read the vCPU's core
>>>> registers with KVM_GET_ONE_REG, and die() if the ioctl fails. Both are
>>>> diagnostics. They run from the KVM_EXIT_DEBUG case in kvm_cpu__start(),
>>>> from the "lkvm debug -d" dump path in handle_sigusr1(), and from the
>>>> panic dump in kvm_cpu_thread().
>>>>
>>>> Once a protected vCPU has run, its registers belong to the guest and
>>>> KVM_GET_ONE_REG returns -EPERM. Dying on that turns a diagnostic into a
>>>> VMM abort.
>>>
>>> Marc once suggested that these could always succeed with junk values for
>>> protected/Realm VMs. Do you think that is an option for pKVM ?
>>
>> It's possible, the host copy is still there, but I'd rather not. Once
>> the vCPU has run that copy is a mix rather than junk: the boot state
>> the VMM wrote, plus what the exit handlers copy out (the PSTATE mode,
>> x0 for an MMIO write, x0 to x2 for a forwarded PSCI call). A dump
>> would show a live PSTATE beside the PC from boot, and the VMM has no
>> way to distinguish them. With the error it prints that the state is
>> unavailable, which is what this patch does.
>>
>> It's also what the tree does for protected state elsewhere: s390
>> returns -EINVAL from KVM_GET/SET_ONE_REG on a protected VM, and x86
>> does the same from the register ioctls for the SEV-ES VM type, SNP and
>> TDX. Legacy SEV-ES is the one place that succeeds silently, kept for
>> backwards compatibility, and the commit that added the errors
>> (517987e3fb19) describes that as a problem. Sean's rationale for TDX:
>> KVM can't provide sane data, so it's userspace's job not to ask for it
>> [1].
>>

Thanks, I am aware of this. I will leave the decision to Marc.
For the record, for CCA we use GET_ONE_REG/SET_ONE_REG to configure
the SVE vector length, PMU Counter and the Debug HW BPRs and watch
points.
>>>
>>> If we go with this approach :
>>>
>>> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>>
>> Thanks, I'll add the tag. :)
> 
> Unless I misunderstood what "this" refers to exactly, please shout if
> that's the case, or continue the discussion :)

You got it right. I meant if we are happy to go with approach in the
patch.