[PATCH v2 0/3] KVM: arm64: Fixes for timers and pKVM

[PATCH v2 0/3] KVM: arm64: Fixes for timers and pKVM

[Mostafa Saleh]

What started as a  small patch ended up as a 3 patch series thanks
to Sashiko.

First patch from Marc to consolidate the offset calculation,
follow up patches fix issues with non-protected VM and timer
offset and protected VM running with broken CNTVOFF_EL2.

The patches have all the information so I will keep this short.

v1: https://lore.kernel.org/all/20260806150105.4010701-1-smostafa@google.com/
Main changes:
- Add comment based on Fuad review
- Add prefix patches that also fix the non-protected hang properly.

Marc Zyngier (1):
  KVM: arm64: Make timer_get_offset() work in all contexts

Mostafa Saleh (2):
  KVM: arm64: Fix timer offsets for non-protected VMs
  KVM: arm64: Fix hvhe and broken CNTVOFF_EL2

 arch/arm64/kvm/hyp/include/hyp/switch.h | 15 +----------
 arch/arm64/kvm/hyp/nvhe/pkvm.c          | 14 ++++++++++
 arch/arm64/kvm/hyp/nvhe/timer-sr.c      | 10 ++++----
 include/kvm/arm_arch_timer.h            | 34 +++++++++++++++----------
 4 files changed, 41 insertions(+), 32 deletions(-)

-- 
2.55.0.654.g21b8a5bc05-goog

[PATCH v2 1/3] KVM: arm64: Make timer_get_offset() work in all contexts

[Mostafa Saleh]

From: Marc Zyngier <maz@kernel.org>

We currently have two implementations of get_timer offset(), one
in arm_arch_timer.h, and another one in switch.h.

These two only differ by a pair of kern_hyp_va(), which seems a
pretty weak reason to open-code it.

Turn this function into a macro to avoid the include dependency hell
on kern_hyp_va(), and make it work correctly in all contexts.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Mostafa Saleh <smostafa@google.com>
---
 arch/arm64/kvm/hyp/include/hyp/switch.h | 15 +----------
 include/kvm/arm_arch_timer.h            | 34 +++++++++++++++----------
 2 files changed, 22 insertions(+), 27 deletions(-)

diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index 4bf624a49591..2aceda749641 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -706,22 +706,9 @@ static inline bool handle_tx2_tvm(struct kvm_vcpu *vcpu)
 	return true;
 }
 
-/* Open-coded version of timer_get_offset() to allow for kern_hyp_va() */
-static inline u64 hyp_timer_get_offset(struct arch_timer_context *ctxt)
-{
-	u64 offset = 0;
-
-	if (ctxt->offset.vm_offset)
-		offset += *kern_hyp_va(ctxt->offset.vm_offset);
-	if (ctxt->offset.vcpu_offset)
-		offset += *kern_hyp_va(ctxt->offset.vcpu_offset);
-
-	return offset;
-}
-
 static inline u64 compute_counter_value(struct arch_timer_context *ctxt)
 {
-	return arch_timer_read_cntpct_el0() - hyp_timer_get_offset(ctxt);
+	return arch_timer_read_cntpct_el0() - timer_get_offset(ctxt);
 }
 
 static bool kvm_handle_cntxct(struct kvm_vcpu *vcpu)
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 725023ddc792..f3f0a79647cd 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -163,20 +163,28 @@ static inline bool has_cntpoff(void)
 	return (has_vhe() && cpus_have_final_cap(ARM64_HAS_ECV_CNTPOFF));
 }
 
-static inline u64 timer_get_offset(struct arch_timer_context *ctxt)
-{
-	u64 offset = 0;
+#ifdef __KVM_NVHE_HYPERVISOR__
+#define KERN_HYP_VA(x)		kern_hyp_va(x)
+#else
+#define KERN_HYP_VA(x)		x
+#endif
 
-	if (!ctxt)
-		return 0;
-
-	if (ctxt->offset.vm_offset)
-		offset += *ctxt->offset.vm_offset;
-	if (ctxt->offset.vcpu_offset)
-		offset += *ctxt->offset.vcpu_offset;
-
-	return offset;
-}
+#define timer_get_offset(ctxt)						\
+	({								\
+		struct arch_timer_context *__ctxt = (ctxt);		\
+		u64 off = 0;						\
+									\
+		if (__ctxt) {						\
+			struct arch_timer_offset *ato = &__ctxt->offset;\
+									\
+			if (ato->vm_offset)				\
+				off += *KERN_HYP_VA(ato->vm_offset);	\
+			if (ato->vcpu_offset)				\
+				off += *KERN_HYP_VA(ato->vcpu_offset);	\
+		}							\
+									\
+		off;							\
+	})
 
 static inline void timer_set_offset(struct arch_timer_context *ctxt, u64 offset)
 {
-- 
2.55.0.654.g21b8a5bc05-goog

[PATCH v2 2/3] KVM: arm64: Fix timer offsets for non-protected VMs

[Mostafa Saleh]

With pKVM, protected VMs always have offset of zero.
However, timer offsets for non-protected guests fail to take effect
for two reasons:
1) In __timer_enable_traps(), enabling of traps check for
   is_protected_kvm_enabled() rather than vcpu_is_protected(vcpu)

2) The vcpu timer offsets were never initialised and kept as NULL.

This is problematic for cases when the timer is trapped in the
hypervisor as the with the case of broken CNTVOFF_EL2, which leads
to the hypervisor and host using different offsets and causing VM
hangs.

This can be confirmed by running the arch_timer selftest which fails:
./arch_timer -o 100000000
Random seed: 0x6b8b4567
Guest assert failed,  vcpu 0; stage; 3; iter: 0
==== Test Assertion Failure ====
  arm64/arch_timer.c:137: config_iter + 1 == irq_iter
  pid=310 tid=312 errno=4 - Interrupted system call
Guest assert failed,  vcpu 3; stage; 3; iter: 0
Guest assert failed,  vcpu 1; stage; 3; iter: 0
==== Test Assertion Failure ====
  arm64/arch_timer.c:137: config_iter + 1 == irq_iter
  pid=310 tid=313 errno=4 - Interrupted system call
Guest assert failed,  vcpu 2; stage; 3; iter: 0
==== Test Assertion Failure ====
  arm64/arch_timer.c:137: config_iter + 1 == irq_iter
  pid=310 tid=314 errno=4 - Interrupted system call
[...]

After the fix:

./arch_timer -o 100000000
Random seed: 0x6b8b4567
PASS(vCPU-1).
PASS(vCPU-3).
PASS(vCPU-0).
PASS(vCPU-2)

Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: cb0c272acebd ("KVM: arm64: Initialize the hypervisor's VM state at EL2")
Signed-off-by: Mostafa Saleh <smostafa@google.com>
---
 arch/arm64/kvm/hyp/nvhe/pkvm.c     | 14 ++++++++++++++
 arch/arm64/kvm/hyp/nvhe/timer-sr.c |  6 +++---
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
index 24d6f164129a..89f3d5fb55ca 100644
--- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
+++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
@@ -529,6 +529,20 @@ static int init_pkvm_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu,
 	hyp_vcpu->vcpu.arch.cflags = READ_ONCE(host_vcpu->arch.cflags);
 	hyp_vcpu->vcpu.arch.mp_state.mp_state = KVM_MP_STATE_STOPPED;
 
+	if (!pkvm_hyp_vcpu_is_protected(hyp_vcpu)) {
+		/*
+		 * Timer offsets are pointing to the untrusted KVM copy,
+		 * which is pinned in __pkvm_init_vm() for the VM life time.
+		 * It is worth noting that hyp_vm->host_kvm points to an EL2
+		 * linear map address and timer_get_offset() will use
+		 * kern_hyp_va() which is safe as it is idempotent.
+		 */
+		vcpu_vtimer(&hyp_vcpu->vcpu)->offset.vm_offset =
+			&hyp_vm->host_kvm->arch.timer_data.voffset;
+		vcpu_ptimer(&hyp_vcpu->vcpu)->offset.vm_offset =
+			&hyp_vm->host_kvm->arch.timer_data.poffset;
+	}
+
 	ret = pkvm_vcpu_init_sysregs(hyp_vcpu);
 	if (ret)
 		goto done;
diff --git a/arch/arm64/kvm/hyp/nvhe/timer-sr.c b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
index ff176f4ce7de..51b4f5010b66 100644
--- a/arch/arm64/kvm/hyp/nvhe/timer-sr.c
+++ b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
@@ -45,11 +45,11 @@ void __timer_enable_traps(struct kvm_vcpu *vcpu)
 	/*
 	 * Disallow physical timer access for the guest
 	 * Physical counter access is allowed if no offset is enforced
-	 * or running protected (we don't offset anything in this case).
+	 * or running a protected VM (we don't offset anything in this case).
 	 */
 	clr = CNTHCTL_EL1PCEN;
-	if (is_protected_kvm_enabled() ||
-	    !kern_hyp_va(vcpu->kvm)->arch.timer_data.poffset)
+	if (vcpu_is_protected(vcpu) ||
+	    !timer_get_offset(vcpu_ptimer(vcpu)))
 		set |= CNTHCTL_EL1PCTEN;
 	else
 		clr |= CNTHCTL_EL1PCTEN;
-- 
2.55.0.654.g21b8a5bc05-goog

Re: [PATCH v2 2/3] KVM: arm64: Fix timer offsets for non-protected VMs

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing Context Synchronization Event (`isb()`) after writing to control-plane system register `cnthctl_el2`.
--

--- Patch [2]: [PATCH v2 2/3] KVM: arm64: Fix timer offsets for non-protected VMs ---
commit 0ed46cb1bdd2dc0d545a0cd2b0b6f500a46a254f
Author: Mostafa Saleh <smostafa@google.com>

KVM: arm64: Fix timer offsets for non-protected VMs

This commit fixes an issue where timer offsets for non-protected guests
were not correctly applied in pKVM. It corrects the protected VM check
in __timer_enable_traps() and initializes the previously null timer offsets.

> diff --git a/arch/arm64/kvm/hyp/nvhe/timer-sr.c b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
> index ff176f4ce7deb..51b4f5010b660 100644
> --- a/arch/arm64/kvm/hyp/nvhe/timer-sr.c
> +++ b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
> @@ -45,11 +45,11 @@ void __timer_enable_traps(struct kvm_vcpu *vcpu)
[ ... ]
>  	clr = CNTHCTL_EL1PCEN;
> -	if (is_protected_kvm_enabled() ||
> -	    !kern_hyp_va(vcpu->kvm)->arch.timer_data.poffset)
> +	if (vcpu_is_protected(vcpu) ||
> +	    !timer_get_offset(vcpu_ptimer(vcpu)))
>  		set |= CNTHCTL_EL1PCTEN;
>  	else
>  		clr |= CNTHCTL_EL1PCTEN;

[Severity: High]
This is a pre-existing issue, but do __timer_enable_traps() and
__timer_disable_traps() in arch/arm64/kvm/hyp/nvhe/timer-sr.c need an
explicit isb() after writing to cnthctl_el2?

Looking at the end of __timer_enable_traps(), it updates cnthctl_el2 and
returns:

arch/arm64/kvm/hyp/nvhe/timer-sr.c:__timer_enable_traps() {
    ...
    sysreg_clear_set(cnthctl_el2, clr, set);
}

The same pattern exists in __timer_disable_traps().

Because sysreg_clear_set() does not embed a Context Synchronization Event,
the CPU executes the subsequent RET instruction (and any caller instructions
before an eventual isb() or ERET) against an architecturally undefined
pipeline state.

Could this missing immediate context synchronization event after a
control-plane system register write lead to unexpected traps or memory
safety violations?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260808085824.732659-1-smostafa@google.com?part=2

[PATCH v2 3/3] KVM: arm64: Fix hvhe and broken CNTVOFF_EL2

[Mostafa Saleh]

When running on a setup affected with broken CNTVOFF_EL2
(has_broken_cntvoff())

Booting with VHE or protected mode(nvhe) (id_aa64mmfr1.vh=0
and arm64_sw.hvhe=0) works fine.

However launching a protected VM with protected hvhe mode panics the
guest kernel:

[    0.000000] Internal error: Oops - Undefined instruction: 0000000000000000 [#1]  SMP
[    0.000000] Modules linked in:
[    0.000000] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 7.2.0-rc3-g05f75bd71e0e-dirty #29 PREEMPT
[    0.000000] Hardware name: linux,dummy-virt (DT)
[    0.000000] pstate: 000003c5 (nzcv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[    0.000000] pc : arch_timer_shutdown_virt+0x4/0x1c
[    0.000000] lr : arch_timer_starting_cpu+0x1c4/0x2d4
[    0.000000] sp : ffffa6bd9a193c00
[    0.000000] x29: ffffa6bd9a193c20 x28: ffffa6bd9a1bcf88 x27: 0000000000000000
[    0.000000] x26: ffff00001be70dd8 x25: ffffa6bd99d85000 x24: ffffa6bd99d85ee4
[    0.000000] x23: ffffa6bd99d85000 x22: ffffa6bd9a1499c0 x21: ffffa6bd9a1ab900
[    0.000000] x20: 00ffffffffffffff x19: ffff00001be8b600 x18: 000000000000028c
[    0.000000] x17: 00000000510f0010 x16: 00000000510f0010 x15: 00000000500f0000
[    0.000000] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000018
[    0.000000] x11: ffffa6bd9a8ac000 x10: 0000000000f0000f x9 : ffffffffffffffff
[    0.000000] x8 : ffffa6bd98822e18 x7 : 0070752d65746174 x6 : 00111ff76e007261
[    0.000000] x5 : ffffa6bd9ad68078 x4 : 0000000000000000 x3 : ffffa6bd98822a0c
[    0.000000] x2 : 0000000000000073 x1 : 0000000000000001 x0 : ffff00001be8b600
[    0.000000] Call trace:
[    0.000000]  arch_timer_shutdown_virt+0x4/0x1c (P)
[    0.000000]  cpuhp_invoke_callback+0x11c/0x280
[    0.000000]  cpuhp_issue_call+0x1e8/0x224
[    0.000000]  __cpuhp_setup_state_cpuslocked+0x1d8/0x2b8
[    0.000000]  __cpuhp_setup_state+0x50/0x74
[    0.000000]  arch_timer_register+0xc0/0x148
[    0.000000]  arch_timer_of_init+0x148/0x170
[    0.000000]  timer_probe+0x74/0x124
[    0.000000]  time_init+0x18/0x58
[    0.000000]  start_kernel+0x1c0/0x3ac
[    0.000000]  __primary_switched+0x88/0x90
[    0.000000] Code: c80b7d2a 35ffffab 17ffffeb d503245f (d53be328)

The workaround avoids setting non-zero CNTVOFF_EL2 and trapping the
virtual counter to emulate the offset.
In the VHE path (timer_set_traps()), traps are only enabled when the
guest actually has a non-zero virtual timer offset.
However, __timer_enable_traps() in hyp/nvhe/timer-sr.c unconditionally
set CNTHCTL_EL1TVT and CNTHCTL_EL1TVCT whenever has_broken_cntvoff()
was true.

Which causes 2 issues:
1) Protected VMs: kvm_handle_pvm_sysreg() does not find "cntv_ctl_el0"
in pvm_sys_reg_descs and injects undefined instruction exceptions.

2) non-protected guests are trapped all the time even with offset of
zero.

Fix this by adding a check in __timer_enable_traps() similar to the one in
timer_set_traps()

Fixes: 0bc9a9e85fcf ("KVM: arm64: Work around x1e's CNTVOFF_EL2 bogosity")
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Signed-off-by: Mostafa Saleh <smostafa@google.com>
---
 arch/arm64/kvm/hyp/nvhe/timer-sr.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/hyp/nvhe/timer-sr.c b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
index 51b4f5010b66..993065716913 100644
--- a/arch/arm64/kvm/hyp/nvhe/timer-sr.c
+++ b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
@@ -61,9 +61,9 @@ void __timer_enable_traps(struct kvm_vcpu *vcpu)
 
 	/*
 	 * Trap the virtual counter/timer if we have a broken cntvoff
-	 * implementation.
+	 * implementation and non zero offset as in timer_set_traps()
 	 */
-	if (has_broken_cntvoff())
+	if (has_broken_cntvoff() && timer_get_offset(vcpu_vtimer(vcpu)))
 		set |= CNTHCTL_EL1TVT | CNTHCTL_EL1TVCT;
 
 	sysreg_clear_set(cnthctl_el2, clr, set);
-- 
2.55.0.654.g21b8a5bc05-goog

Re: [PATCH v2 3/3] KVM: arm64: Fix hvhe and broken CNTVOFF_EL2

[Mostafa Saleh]

On Sat, Aug 8, 2026 at 9:58 AM Mostafa Saleh <smostafa@google.com> wrote:
>
> When running on a setup affected with broken CNTVOFF_EL2
> (has_broken_cntvoff())
>
> Booting with VHE or protected mode(nvhe) (id_aa64mmfr1.vh=0
> and arm64_sw.hvhe=0) works fine.
>
> However launching a protected VM with protected hvhe mode panics the
> guest kernel:
>
> [    0.000000] Internal error: Oops - Undefined instruction: 0000000000000000 [#1]  SMP
> [    0.000000] Modules linked in:
> [    0.000000] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 7.2.0-rc3-g05f75bd71e0e-dirty #29 PREEMPT
> [    0.000000] Hardware name: linux,dummy-virt (DT)
> [    0.000000] pstate: 000003c5 (nzcv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [    0.000000] pc : arch_timer_shutdown_virt+0x4/0x1c
> [    0.000000] lr : arch_timer_starting_cpu+0x1c4/0x2d4
> [    0.000000] sp : ffffa6bd9a193c00
> [    0.000000] x29: ffffa6bd9a193c20 x28: ffffa6bd9a1bcf88 x27: 0000000000000000
> [    0.000000] x26: ffff00001be70dd8 x25: ffffa6bd99d85000 x24: ffffa6bd99d85ee4
> [    0.000000] x23: ffffa6bd99d85000 x22: ffffa6bd9a1499c0 x21: ffffa6bd9a1ab900
> [    0.000000] x20: 00ffffffffffffff x19: ffff00001be8b600 x18: 000000000000028c
> [    0.000000] x17: 00000000510f0010 x16: 00000000510f0010 x15: 00000000500f0000
> [    0.000000] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000018
> [    0.000000] x11: ffffa6bd9a8ac000 x10: 0000000000f0000f x9 : ffffffffffffffff
> [    0.000000] x8 : ffffa6bd98822e18 x7 : 0070752d65746174 x6 : 00111ff76e007261
> [    0.000000] x5 : ffffa6bd9ad68078 x4 : 0000000000000000 x3 : ffffa6bd98822a0c
> [    0.000000] x2 : 0000000000000073 x1 : 0000000000000001 x0 : ffff00001be8b600
> [    0.000000] Call trace:
> [    0.000000]  arch_timer_shutdown_virt+0x4/0x1c (P)
> [    0.000000]  cpuhp_invoke_callback+0x11c/0x280
> [    0.000000]  cpuhp_issue_call+0x1e8/0x224
> [    0.000000]  __cpuhp_setup_state_cpuslocked+0x1d8/0x2b8
> [    0.000000]  __cpuhp_setup_state+0x50/0x74
> [    0.000000]  arch_timer_register+0xc0/0x148
> [    0.000000]  arch_timer_of_init+0x148/0x170
> [    0.000000]  timer_probe+0x74/0x124
> [    0.000000]  time_init+0x18/0x58
> [    0.000000]  start_kernel+0x1c0/0x3ac
> [    0.000000]  __primary_switched+0x88/0x90
> [    0.000000] Code: c80b7d2a 35ffffab 17ffffeb d503245f (d53be328)
>
> The workaround avoids setting non-zero CNTVOFF_EL2 and trapping the
> virtual counter to emulate the offset.
> In the VHE path (timer_set_traps()), traps are only enabled when the
> guest actually has a non-zero virtual timer offset.
> However, __timer_enable_traps() in hyp/nvhe/timer-sr.c unconditionally
> set CNTHCTL_EL1TVT and CNTHCTL_EL1TVCT whenever has_broken_cntvoff()
> was true.
>
> Which causes 2 issues:
> 1) Protected VMs: kvm_handle_pvm_sysreg() does not find "cntv_ctl_el0"
> in pvm_sys_reg_descs and injects undefined instruction exceptions.
>
> 2) non-protected guests are trapped all the time even with offset of
> zero.
>
> Fix this by adding a check in __timer_enable_traps() similar to the one in
> timer_set_traps()
>
> Fixes: 0bc9a9e85fcf ("KVM: arm64: Work around x1e's CNTVOFF_EL2 bogosity")
> Signed-off-by: Marc Zyngier <maz@kernel.org>

For some reason, when I used b4 to pull the first version, it added
Marc's `Signed-off-by` which was not intended.
Probably it got confused by the patch in the reply.

Thanks,
Mostafa

> Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
> Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
> Signed-off-by: Mostafa Saleh <smostafa@google.com>
> ---
>  arch/arm64/kvm/hyp/nvhe/timer-sr.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/nvhe/timer-sr.c b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
> index 51b4f5010b66..993065716913 100644
> --- a/arch/arm64/kvm/hyp/nvhe/timer-sr.c
> +++ b/arch/arm64/kvm/hyp/nvhe/timer-sr.c
> @@ -61,9 +61,9 @@ void __timer_enable_traps(struct kvm_vcpu *vcpu)
>
>         /*
>          * Trap the virtual counter/timer if we have a broken cntvoff
> -        * implementation.
> +        * implementation and non zero offset as in timer_set_traps()
>          */
> -       if (has_broken_cntvoff())
> +       if (has_broken_cntvoff() && timer_get_offset(vcpu_vtimer(vcpu)))
>                 set |= CNTHCTL_EL1TVT | CNTHCTL_EL1TVCT;
>
>         sysreg_clear_set(cnthctl_el2, clr, set);
> --
> 2.55.0.654.g21b8a5bc05-goog
>

Re: [PATCH v2 0/3] KVM: arm64: Fixes for timers and pKVM

[Oliver Upton]

On Sat, 08 Aug 2026 08:58:21 +0000, Mostafa Saleh wrote:
> What started as a  small patch ended up as a 3 patch series thanks
> to Sashiko.
> 
> First patch from Marc to consolidate the offset calculation,
> follow up patches fix issues with non-protected VM and timer
> offset and protected VM running with broken CNTVOFF_EL2.
> 
> [...]

Dropped the unintended SOB in patch 3 you mentioned.

Applied to next, thanks!

[1/3] KVM: arm64: Make timer_get_offset() work in all contexts
      https://git.kernel.org/kvmarm/kvmarm/c/2858600ecd01
[2/3] KVM: arm64: Fix timer offsets for non-protected VMs
      https://git.kernel.org/kvmarm/kvmarm/c/47d3eef780e3
[3/3] KVM: arm64: Fix hvhe and broken CNTVOFF_EL2
      https://git.kernel.org/kvmarm/kvmarm/c/2e813a6e8ebe

--
Best,
Oliver

Re: [PATCH v2 0/3] KVM: arm64: Fixes for timers and pKVM

[Mostafa Saleh]

On Sat, Aug 8, 2026 at 7:44 PM Oliver Upton <oupton@kernel.org> wrote:
>
> On Sat, 08 Aug 2026 08:58:21 +0000, Mostafa Saleh wrote:
> > What started as a  small patch ended up as a 3 patch series thanks
> > to Sashiko.
> >
> > First patch from Marc to consolidate the offset calculation,
> > follow up patches fix issues with non-protected VM and timer
> > offset and protected VM running with broken CNTVOFF_EL2.
> >
> > [...]
>
> Dropped the unintended SOB in patch 3 you mentioned.
>
> Applied to next, thanks!
>
> [1/3] KVM: arm64: Make timer_get_offset() work in all contexts
>       https://git.kernel.org/kvmarm/kvmarm/c/2858600ecd01
> [2/3] KVM: arm64: Fix timer offsets for non-protected VMs
>       https://git.kernel.org/kvmarm/kvmarm/c/47d3eef780e3
> [3/3] KVM: arm64: Fix hvhe and broken CNTVOFF_EL2
>       https://git.kernel.org/kvmarm/kvmarm/c/2e813a6e8ebe
>

Thanks Oliver! I believe there is one more bug. I'm not sure where the
bug is or if it relates to the broken timers.
Before those patches I could not boot a protected VM because of the
panic, now after booting protected VMs, I sometimes get a system
reset.
I confirmed that cntvoff_el2 does not get written to non-zero, I also
removed the sysreg write completely (rely on xzr value at init) so my
guess is that the HW might be allergic to more than just non-zero
values in cntvoff_el2.
I do not have issues with non-protected VMs anymore.

Thanks,
Mostafa




> --
> Best,
> Oliver

Re: [PATCH v2 0/3] KVM: arm64: Fixes for timers and pKVM

[Marc Zyngier]

On Tue, 11 Aug 2026 13:17:25 +0100,
Mostafa Saleh <smostafa@google.com> wrote:
> 
> On Sat, Aug 8, 2026 at 7:44 PM Oliver Upton <oupton@kernel.org> wrote:
> >
> > On Sat, 08 Aug 2026 08:58:21 +0000, Mostafa Saleh wrote:
> > > What started as a  small patch ended up as a 3 patch series thanks
> > > to Sashiko.
> > >
> > > First patch from Marc to consolidate the offset calculation,
> > > follow up patches fix issues with non-protected VM and timer
> > > offset and protected VM running with broken CNTVOFF_EL2.
> > >
> > > [...]
> >
> > Dropped the unintended SOB in patch 3 you mentioned.
> >
> > Applied to next, thanks!
> >
> > [1/3] KVM: arm64: Make timer_get_offset() work in all contexts
> >       https://git.kernel.org/kvmarm/kvmarm/c/2858600ecd01
> > [2/3] KVM: arm64: Fix timer offsets for non-protected VMs
> >       https://git.kernel.org/kvmarm/kvmarm/c/47d3eef780e3
> > [3/3] KVM: arm64: Fix hvhe and broken CNTVOFF_EL2
> >       https://git.kernel.org/kvmarm/kvmarm/c/2e813a6e8ebe
> >
> 
> Thanks Oliver! I believe there is one more bug. I'm not sure where the
> bug is or if it relates to the broken timers.
> Before those patches I could not boot a protected VM because of the
> panic, now after booting protected VMs, I sometimes get a system
> reset.

On this quality HW, this is usually an indication that you are taking
an exception in a tight loop.

> I confirmed that cntvoff_el2 does not get written to non-zero, I also
> removed the sysreg write completely (rely on xzr value at init) so my
> guess is that the HW might be allergic to more than just non-zero
> values in cntvoff_el2.

Is that in hVHE mode? Can you trap the access and route it to the
existing handling code?

> I do not have issues with non-protected VMs anymore.

Do these run with an offset or not?

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH v2 0/3] KVM: arm64: Fixes for timers and pKVM

[Mostafa Saleh]

On Tue, Aug 11, 2026 at 02:18:15PM +0100, Marc Zyngier wrote:
> On Tue, 11 Aug 2026 13:17:25 +0100,
> Mostafa Saleh <smostafa@google.com> wrote:
> > 
> > On Sat, Aug 8, 2026 at 7:44 PM Oliver Upton <oupton@kernel.org> wrote:
> > >
> > > On Sat, 08 Aug 2026 08:58:21 +0000, Mostafa Saleh wrote:
> > > > What started as a  small patch ended up as a 3 patch series thanks
> > > > to Sashiko.
> > > >
> > > > First patch from Marc to consolidate the offset calculation,
> > > > follow up patches fix issues with non-protected VM and timer
> > > > offset and protected VM running with broken CNTVOFF_EL2.
> > > >
> > > > [...]
> > >
> > > Dropped the unintended SOB in patch 3 you mentioned.
> > >
> > > Applied to next, thanks!
> > >
> > > [1/3] KVM: arm64: Make timer_get_offset() work in all contexts
> > >       https://git.kernel.org/kvmarm/kvmarm/c/2858600ecd01
> > > [2/3] KVM: arm64: Fix timer offsets for non-protected VMs
> > >       https://git.kernel.org/kvmarm/kvmarm/c/47d3eef780e3
> > > [3/3] KVM: arm64: Fix hvhe and broken CNTVOFF_EL2
> > >       https://git.kernel.org/kvmarm/kvmarm/c/2e813a6e8ebe
> > >
> > 
> > Thanks Oliver! I believe there is one more bug. I'm not sure where the
> > bug is or if it relates to the broken timers.
> > Before those patches I could not boot a protected VM because of the
> > panic, now after booting protected VMs, I sometimes get a system
> > reset.
> 
> On this quality HW, this is usually an indication that you are taking
> an exception in a tight loop.
> 

I tried to add a poor man exception storm detection in the kernel
handlers (el1h_64_sync_handler and __gic_handle_irq), but it didn’t trigger
before the reset.

My hunch would be that there are some paths in protected VMs that takes
too long that the watchdog fires. I saw the resets mostly at either
userspace boot or VM teardown, otherwise the VM seems functional.

I will collect timestamps from hypervisor entry/exit and check
how large are those.

> > I confirmed that cntvoff_el2 does not get written to non-zero, I also
> > removed the sysreg write completely (rely on xzr value at init) so my
> > guess is that the HW might be allergic to more than just non-zero
> > values in cntvoff_el2.
> 
> Is that in hVHE mode? Can you trap the access and route it to the
> existing handling code?
> 

Yes, only hVHE. Protected nVHE works fine.

One interesting observation is that when starting a VM with a single
vcpu I don’t see the reset anymore compared to 4 cpus before.

Enabling traps for timer unconditionally for protected VMs still has
the same issue.

> > I do not have issues with non-protected VMs anymore.
> 
> Do these run with an offset or not?

Yes, they have the offset set from kvm_timer_vcpu_init() with
kvm_phys_timer_read().

Thanks,
Mostafa

> 
> 	M.
> 
> -- 
> Without deviation from the norm, progress is not possible.

Re: [PATCH v2 0/3] KVM: arm64: Fixes for timers and pKVM

[Mostafa Saleh]

On Tue, Aug 11, 2026 at 6:22 PM Mostafa Saleh <smostafa@google.com> wrote:
>
> On Tue, Aug 11, 2026 at 02:18:15PM +0100, Marc Zyngier wrote:
> > On Tue, 11 Aug 2026 13:17:25 +0100,
> > Mostafa Saleh <smostafa@google.com> wrote:
> > >
> > > On Sat, Aug 8, 2026 at 7:44 PM Oliver Upton <oupton@kernel.org> wrote:
> > > >
> > > > On Sat, 08 Aug 2026 08:58:21 +0000, Mostafa Saleh wrote:
> > > > > What started as a  small patch ended up as a 3 patch series thanks
> > > > > to Sashiko.
> > > > >
> > > > > First patch from Marc to consolidate the offset calculation,
> > > > > follow up patches fix issues with non-protected VM and timer
> > > > > offset and protected VM running with broken CNTVOFF_EL2.
> > > > >
> > > > > [...]
> > > >
> > > > Dropped the unintended SOB in patch 3 you mentioned.
> > > >
> > > > Applied to next, thanks!
> > > >
> > > > [1/3] KVM: arm64: Make timer_get_offset() work in all contexts
> > > >       https://git.kernel.org/kvmarm/kvmarm/c/2858600ecd01
> > > > [2/3] KVM: arm64: Fix timer offsets for non-protected VMs
> > > >       https://git.kernel.org/kvmarm/kvmarm/c/47d3eef780e3
> > > > [3/3] KVM: arm64: Fix hvhe and broken CNTVOFF_EL2
> > > >       https://git.kernel.org/kvmarm/kvmarm/c/2e813a6e8ebe
> > > >
> > >
> > > Thanks Oliver! I believe there is one more bug. I'm not sure where the
> > > bug is or if it relates to the broken timers.
> > > Before those patches I could not boot a protected VM because of the
> > > panic, now after booting protected VMs, I sometimes get a system
> > > reset.
> >
> > On this quality HW, this is usually an indication that you are taking
> > an exception in a tight loop.
> >
>
> I tried to add a poor man exception storm detection in the kernel
> handlers (el1h_64_sync_handler and __gic_handle_irq), but it didn’t trigger
> before the reset.
>
> My hunch would be that there are some paths in protected VMs that takes
> too long that the watchdog fires. I saw the resets mostly at either
> userspace boot or VM teardown, otherwise the VM seems functional.
>
> I will collect timestamps from hypervisor entry/exit and check
> how large are those.
>

Max numbers I've seen doesn't exceed 2-3 ms, compared  to SMCs which
can take almost a second.
I'm not sure if the watchdog is the right conclusion, but without any
clue from the firmware logs, I am lost.

Also, correcting myself, I now see it happened with a single vCPU
although that took longer to reproduce.

Thanks,
Mostafa

> > > I confirmed that cntvoff_el2 does not get written to non-zero, I also
> > > removed the sysreg write completely (rely on xzr value at init) so my
> > > guess is that the HW might be allergic to more than just non-zero
> > > values in cntvoff_el2.
> >
> > Is that in hVHE mode? Can you trap the access and route it to the
> > existing handling code?
> >
>
> Yes, only hVHE. Protected nVHE works fine.
>
> One interesting observation is that when starting a VM with a single
> vcpu I don’t see the reset anymore compared to 4 cpus before.
>
> Enabling traps for timer unconditionally for protected VMs still has
> the same issue.
>
> > > I do not have issues with non-protected VMs anymore.
> >
> > Do these run with an offset or not?
>
> Yes, they have the offset set from kvm_timer_vcpu_init() with
> kvm_phys_timer_read().
>
> Thanks,
> Mostafa
>
> >
> >       M.
> >
> > --
> > Without deviation from the norm, progress is not possible.