[PATCH] KVM: arm64: timers: Use CNTHCTL_EL2 when setting non-CNTKCTL_EL1 bits

[PATCH] KVM: arm64: timers: Use CNTHCTL_EL2 when setting non-CNTKCTL_EL1 bits

[Marc Zyngier]

It recently appeared that, whien running VHE, there is a notable
difference between using CNTKCTL_EL1 and CNTHCTL_EL2, despite what
the architecture documents:

- When accessed from EL2, bits [19:18] and [16:10] same bits have
  the same assignment as CNTHCTL_EL2
- When accessed from EL1, bits [19:18] and [16:10] are RES0

It is all OK, until you factor in NV, where the EL2 guest runs at EL1.
In this configuration, CNTKCTL_EL11 doesn't trap, nor ends up in
the VNCR page. This means that any write from the guest affecting
CNTHCTL_EL2 using CNTKCTL_EL1 ends up losing some state. Not good.

The fix it obvious: don't use CNTKCTL_EL1 if you want to change bits
that are not part of the EL1 definition of CNTKCTL_EL1, and use
CNTHCTL_EL2 instead. This doesn't change anything for a bare-metal OS,
and fixes it when running under NV. The NV hypervisor will itself
have to work harder to merge the two accessors.

Note that there is a pending update to the architecture to address
this issue by making the affected bits UNKNOWN when CNTKCTL_EL1 is
user from EL2 with VHE enabled.

Fixes: c605ee245097 ("KVM: arm64: timers: Allow physical offset without CNTPOFF_EL2")
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org # v6.4
---
 arch/arm64/kvm/arch_timer.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 0696732fa38c..6dcdae4d38cb 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -827,8 +827,8 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)
 	assign_clear_set_bit(tpt, CNTHCTL_EL1PCEN << 10, set, clr);
 	assign_clear_set_bit(tpc, CNTHCTL_EL1PCTEN << 10, set, clr);
 
-	/* This only happens on VHE, so use the CNTKCTL_EL1 accessor */
-	sysreg_clear_set(cntkctl_el1, clr, set);
+	/* This only happens on VHE, so use the CNTHCTL_EL2 accessor. */
+	sysreg_clear_set(cnthctl_el2, clr, set);
 }
 
 void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
@@ -1563,7 +1563,7 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
 void kvm_timer_init_vhe(void)
 {
 	if (cpus_have_final_cap(ARM64_HAS_ECV_CNTPOFF))
-		sysreg_clear_set(cntkctl_el1, 0, CNTHCTL_ECV);
+		sysreg_clear_set(cnthctl_el2, 0, CNTHCTL_ECV);
 }
 
 int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
-- 
2.34.1

Re: [PATCH] KVM: arm64: timers: Use CNTHCTL_EL2 when setting non-CNTKCTL_EL1 bits

[Oliver Upton]

On Tue, Jun 27, 2023 at 03:05:57PM +0100, Marc Zyngier wrote:
> It recently appeared that, whien running VHE, there is a notable
> difference between using CNTKCTL_EL1 and CNTHCTL_EL2, despite what
> the architecture documents:
> 
> - When accessed from EL2, bits [19:18] and [16:10] same bits have
>   the same assignment as CNTHCTL_EL2
> - When accessed from EL1, bits [19:18] and [16:10] are RES0
> 
> It is all OK, until you factor in NV, where the EL2 guest runs at EL1.
> In this configuration, CNTKCTL_EL11 doesn't trap, nor ends up in
> the VNCR page. This means that any write from the guest affecting
> CNTHCTL_EL2 using CNTKCTL_EL1 ends up losing some state. Not good.
> 
> The fix it obvious: don't use CNTKCTL_EL1 if you want to change bits
> that are not part of the EL1 definition of CNTKCTL_EL1, and use
> CNTHCTL_EL2 instead. This doesn't change anything for a bare-metal OS,
> and fixes it when running under NV. The NV hypervisor will itself
> have to work harder to merge the two accessors.
> 
> Note that there is a pending update to the architecture to address
> this issue by making the affected bits UNKNOWN when CNTKCTL_EL1 is
> user from EL2 with VHE enabled.
> 
> Fixes: c605ee245097 ("KVM: arm64: timers: Allow physical offset without CNTPOFF_EL2")
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> Cc: stable@vger.kernel.org # v6.4

Looks good. I'll probably open a fixes branch around -rc1 and pick this
patch up then.

-- 
Thanks,
Oliver

Re: [PATCH] KVM: arm64: timers: Use CNTHCTL_EL2 when setting non-CNTKCTL_EL1 bits

[Marc Zyngier]

On Tue, 27 Jun 2023 17:36:16 +0100,
Oliver Upton <oliver.upton@linux.dev> wrote:
> 
> On Tue, Jun 27, 2023 at 03:05:57PM +0100, Marc Zyngier wrote:
> > It recently appeared that, whien running VHE, there is a notable
> > difference between using CNTKCTL_EL1 and CNTHCTL_EL2, despite what
> > the architecture documents:
> > 
> > - When accessed from EL2, bits [19:18] and [16:10] same bits have
> >   the same assignment as CNTHCTL_EL2
> > - When accessed from EL1, bits [19:18] and [16:10] are RES0
> > 
> > It is all OK, until you factor in NV, where the EL2 guest runs at EL1.
> > In this configuration, CNTKCTL_EL11 doesn't trap, nor ends up in
> > the VNCR page. This means that any write from the guest affecting
> > CNTHCTL_EL2 using CNTKCTL_EL1 ends up losing some state. Not good.
> > 
> > The fix it obvious: don't use CNTKCTL_EL1 if you want to change bits
> > that are not part of the EL1 definition of CNTKCTL_EL1, and use
> > CNTHCTL_EL2 instead. This doesn't change anything for a bare-metal OS,
> > and fixes it when running under NV. The NV hypervisor will itself
> > have to work harder to merge the two accessors.
> > 
> > Note that there is a pending update to the architecture to address
> > this issue by making the affected bits UNKNOWN when CNTKCTL_EL1 is
> > user from EL2 with VHE enabled.
> > 
> > Fixes: c605ee245097 ("KVM: arm64: timers: Allow physical offset without CNTPOFF_EL2")
> > Signed-off-by: Marc Zyngier <maz@kernel.org>
> > Cc: stable@vger.kernel.org # v6.4
> 
> Looks good. I'll probably open a fixes branch around -rc1 and pick this
> patch up then.

Awesome, thanks.

(/me goes back to encoding FGT handling by hand... :-/)

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH] KVM: arm64: timers: Use CNTHCTL_EL2 when setting non-CNTKCTL_EL1 bits

[Eric Auger]

Hi Marc,
On 6/27/23 16:05, Marc Zyngier wrote:
> It recently appeared that, whien running VHE, there is a notable
when
> difference between using CNTKCTL_EL1 and CNTHCTL_EL2, despite what
> the architecture documents:
> 
> - When accessed from EL2, bits [19:18] and [16:10] same bits have
s/same bits/of CNTKCTL_EL1/ ?
>   the same assignment as CNTHCTL_EL2
> - When accessed from EL1, bits [19:18] and [16:10] are RES0
s/bits/the same bits/
> 
> It is all OK, until you factor in NV, where the EL2 guest runs at EL1.
> In this configuration, CNTKCTL_EL11 doesn't trap, nor ends up in
s/CNTKCTL_EL11/CNTKCTL_EL1
> the VNCR page. This means that any write from the guest affecting
> CNTHCTL_EL2 using CNTKCTL_EL1 ends up losing some state. Not good.
> 
> The fix it obvious: don't use CNTKCTL_EL1 if you want to change bits
> that are not part of the EL1 definition of CNTKCTL_EL1, and use
> CNTHCTL_EL2 instead. This doesn't change anything for a bare-metal OS,
> and fixes it when running under NV. The NV hypervisor will itself
> have to work harder to merge the two accessors.
> 
> Note that there is a pending update to the architecture to address
> this issue by making the affected bits UNKNOWN when CNTKCTL_EL1 is
> user from EL2 with VHE enabled.
used
> 
> Fixes: c605ee245097 ("KVM: arm64: timers: Allow physical offset without CNTPOFF_EL2")
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> Cc: stable@vger.kernel.org # v6.4
> ---
>  arch/arm64/kvm/arch_timer.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
> index 0696732fa38c..6dcdae4d38cb 100644
> --- a/arch/arm64/kvm/arch_timer.c
> +++ b/arch/arm64/kvm/arch_timer.c
> @@ -827,8 +827,8 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)
>  	assign_clear_set_bit(tpt, CNTHCTL_EL1PCEN << 10, set, clr);
>  	assign_clear_set_bit(tpc, CNTHCTL_EL1PCTEN << 10, set, clr);
>  
> -	/* This only happens on VHE, so use the CNTKCTL_EL1 accessor */
> -	sysreg_clear_set(cntkctl_el1, clr, set);
> +	/* This only happens on VHE, so use the CNTHCTL_EL2 accessor. */
> +	sysreg_clear_set(cnthctl_el2, clr, set);
>  }
>  
>  void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
> @@ -1563,7 +1563,7 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
>  void kvm_timer_init_vhe(void)
>  {
>  	if (cpus_have_final_cap(ARM64_HAS_ECV_CNTPOFF))
> -		sysreg_clear_set(cntkctl_el1, 0, CNTHCTL_ECV);
> +		sysreg_clear_set(cnthctl_el2, 0, CNTHCTL_ECV);
>  }
>  
>  int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric

Re: [PATCH] KVM: arm64: timers: Use CNTHCTL_EL2 when setting non-CNTKCTL_EL1 bits

[Oliver Upton]

On Tue, 27 Jun 2023 15:05:57 +0100, Marc Zyngier wrote:
> It recently appeared that, whien running VHE, there is a notable
> difference between using CNTKCTL_EL1 and CNTHCTL_EL2, despite what
> the architecture documents:
> 
> - When accessed from EL2, bits [19:18] and [16:10] same bits have
>   the same assignment as CNTHCTL_EL2
> - When accessed from EL1, bits [19:18] and [16:10] are RES0
> 
> [...]

Applied to kvmarm/fixes, thanks!

[1/1] KVM: arm64: timers: Use CNTHCTL_EL2 when setting non-CNTKCTL_EL1 bits
      https://git.kernel.org/kvmarm/kvmarm/c/fe769e6c1f80

--
Best,
Oliver

Re: [PATCH] KVM: arm64: timers: Use CNTHCTL_EL2 when setting non-CNTKCTL_EL1 bits

[Oliver Upton]

On Tue, Jul 11, 2023 at 12:35:00PM +0200, Eric Auger wrote:
> Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks for reviewing this Eric. I addressed your comments and picked up
your R-b when applying Marc's patch.

-- 
Thanks,
Oliver