[QUESTION] arch_counter_register() restricts CNTPT access when booted in EL1, even if EL2 is supported

[QUESTION] arch_counter_register() restricts CNTPT access when booted in EL1, even if EL2 is supported

[Youngmin Nam]

[-- Attachment #1: Type: text/plain, Size: 2293 bytes --]

Hi arm arch timer experts,

While reviewing the arm_arch_timer code in Linux 6.12,
I noticed that the function arch_counter_register() restricts the use of the physical counter (cntpct_el0)
on systems where the kernel is running in EL1, even if EL2 is supported and cntpct_el0 is accessible.

In our case:
- We are not using pKVM.
- The kernel is booted in EL1.
- We disabled VIRT_PPI and explicitly selected PHYS_NONSECURE_PPI for the timer refering to below code.

static enum arch_timer_ppi_nr __init arch_timer_select_ppi(void)
{
	if (is_kernel_in_hyp_mode())
		return ARCH_TIMER_HYP_PPI;

	if (!is_hyp_mode_available() && arch_timer_ppi[ARCH_TIMER_VIRT_PPI])
		return ARCH_TIMER_VIRT_PPI;

	if (IS_ENABLED(CONFIG_ARM64))
		return ARCH_TIMER_PHYS_NONSECURE_PPI;

	return ARCH_TIMER_PHYS_SECURE_PPI;
}

Despite this, the `arch_counter_register()` logic selects `cntvct_el0` unconditionally
due to the `!is_hyp_mode_available()` condition.

static void __init arch_counter_register(unsigned type)
{
	u64 (*scr)(void);
	u64 start_count;
	int width;

	/* Register the CP15 based counter if we have one */
	if (type & ARCH_TIMER_TYPE_CP15) {
		u64 (*rd)(void);

		if ((IS_ENABLED(CONFIG_ARM64) && !is_hyp_mode_available()) ||
		    arch_timer_uses_ppi == ARCH_TIMER_VIRT_PPI) {
			if (arch_timer_counter_has_wa()) {
				rd = arch_counter_get_cntvct_stable;
				scr = raw_counter_get_cntvct_stable;
			} else {
				rd = arch_counter_get_cntvct;
				scr = arch_counter_get_cntvct;
			}
		} else {
			if (arch_timer_counter_has_wa()) {
				rd = arch_counter_get_cntpct_stable;
				scr = raw_counter_get_cntpct_stable;
			} else {
				rd = arch_counter_get_cntpct;
				scr = arch_counter_get_cntpct;
			}
		}

As I understand it, `is_hyp_mode_available()` checks whether the kernel booted into EL2
— not whether EL2 is *supported* by the hardware.

Therefore, even on systems where EL2 exists and `cntpct_el0` is accessible from EL1,
the kernel still forces the use of `cntvct_el0` if the boot EL is EL1.

Is this restriction to `cntvct_el0` in EL1 an architectural requirement,
or simply a conservative default to avoid possible traps on some systems?

If the hardware clearly supports EL2 and allows CNTPT access from EL1, could this restriction be relaxed?

Any guidance would be appreciated.

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [QUESTION] arch_counter_register() restricts CNTPT access when booted in EL1, even if EL2 is supported

[Marc Zyngier]

On Fri, 16 May 2025 07:53:58 +0100,
Youngmin Nam <youngmin.nam@samsung.com> wrote:
> 
> [1  <text/plain; utf-8 (8bit)>]
> Hi arm arch timer experts,
> 
> While reviewing the arm_arch_timer code in Linux 6.12,
> I noticed that the function arch_counter_register() restricts the
> use of the physical counter (cntpct_el0) on systems where the kernel
> is running in EL1, even if EL2 is supported and cntpct_el0 is
> accessible.
> 
> In our case:
> - We are not using pKVM.
> - The kernel is booted in EL1.
> - We disabled VIRT_PPI and explicitly selected PHYS_NONSECURE_PPI for the timer refering to below code.

That's not legal. The architecture guarantees that there is a virtual
timer and a physical timer. No ifs, no buts.

[...]

> As I understand it, `is_hyp_mode_available()` checks whether the
> kernel booted into EL2 — not whether EL2 is *supported* by the
> hardware.
> 
> Therefore, even on systems where EL2 exists and `cntpct_el0` is
> accessible from EL1, the kernel still forces the use of `cntvct_el0`
> if the boot EL is EL1.

Yes, because it isn't architecturally valid to not have a virtual
timer. This isn't about EL2 being present of not. The switch to the
physical timer is purely an optimisation for KVM so that it doesn't
have to switch the virtual timer back and forth when running a guest,
as the virtual timer is the most likely used timer.

> Is this restriction to `cntvct_el0` in EL1 an architectural
> requirement, or simply a conservative default to avoid possible
> traps on some systems?

Both. Crucially, it isn't possible to trap the virtual timer on some
older implementations.

> If the hardware clearly supports EL2 and allows CNTPT access from
> EL1, could this restriction be relaxed?

Absolutely not. Having the virtual timer is a hard requirement from
both the architecture *and* Linux. Feel free to emulate it from EL2 if
you want (and can trap it).

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [QUESTION] arch_counter_register() restricts CNTPT access when booted in EL1, even if EL2 is supported

[Youngmin Nam]

[-- Attachment #1: Type: text/plain, Size: 2824 bytes --]

On Fri, May 16, 2025 at 10:28:56AM +0100, Marc Zyngier wrote:
> On Fri, 16 May 2025 07:53:58 +0100,
> Youngmin Nam <youngmin.nam@samsung.com> wrote:
> > 
> > [1  <text/plain; utf-8 (8bit)>]
> > Hi arm arch timer experts,
> > 
> > While reviewing the arm_arch_timer code in Linux 6.12,
> > I noticed that the function arch_counter_register() restricts the
> > use of the physical counter (cntpct_el0) on systems where the kernel
> > is running in EL1, even if EL2 is supported and cntpct_el0 is
> > accessible.
> > 
> > In our case:
> > - We are not using pKVM.
> > - The kernel is booted in EL1.
> > - We disabled VIRT_PPI and explicitly selected PHYS_NONSECURE_PPI for the timer refering to below code.
> 
> That's not legal. The architecture guarantees that there is a virtual
> timer and a physical timer. No ifs, no buts.
> 
> [...]
> 
> > As I understand it, `is_hyp_mode_available()` checks whether the
> > kernel booted into EL2 — not whether EL2 is *supported* by the
> > hardware.
> > 
> > Therefore, even on systems where EL2 exists and `cntpct_el0` is
> > accessible from EL1, the kernel still forces the use of `cntvct_el0`
> > if the boot EL is EL1.
> 
> Yes, because it isn't architecturally valid to not have a virtual
> timer. This isn't about EL2 being present of not. The switch to the
> physical timer is purely an optimisation for KVM so that it doesn't
> have to switch the virtual timer back and forth when running a guest,
> as the virtual timer is the most likely used timer.
> 

Thanks for the clarification.

As a follow-up question:

We are working on a system that uses a vendor-specific hypervisor instead of KVM.
In this setup, we also want to optimize timer virtualization overhead and are considering using
the physical timer (CNTPT) in the host context for performance reasons, just like KVM does.

Would it be acceptable (from the upstream kernel's perspective) to make a similar switch
to the physical timer in this case ?

Or is this kind of optimization strictly tied to KVM's internal behavior
and not something the kernel is expected to support generically?

Any guidance or recommendation would be appreciated.

Best regards,
Youngmin

> > Is this restriction to `cntvct_el0` in EL1 an architectural
> > requirement, or simply a conservative default to avoid possible
> > traps on some systems?
> 
> Both. Crucially, it isn't possible to trap the virtual timer on some
> older implementations.
> 
> > If the hardware clearly supports EL2 and allows CNTPT access from
> > EL1, could this restriction be relaxed?
> 
> Absolutely not. Having the virtual timer is a hard requirement from
> both the architecture *and* Linux. Feel free to emulate it from EL2 if
> you want (and can trap it).
> 
> Thanks,
> 
> 	M.
> 
> -- 
> Without deviation from the norm, progress is not possible.
> 

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [QUESTION] arch_counter_register() restricts CNTPT access when booted in EL1, even if EL2 is supported

[Marc Zyngier]

On Mon, 19 May 2025 02:43:49 +0100,
Youngmin Nam <youngmin.nam@samsung.com> wrote:
> 
> [1  <text/plain; utf-8 (8bit)>]
> On Fri, May 16, 2025 at 10:28:56AM +0100, Marc Zyngier wrote:
> > On Fri, 16 May 2025 07:53:58 +0100,
> > Youngmin Nam <youngmin.nam@samsung.com> wrote:
> > > 
> > > [1  <text/plain; utf-8 (8bit)>]
> > > Hi arm arch timer experts,
> > > 
> > > While reviewing the arm_arch_timer code in Linux 6.12,
> > > I noticed that the function arch_counter_register() restricts the
> > > use of the physical counter (cntpct_el0) on systems where the kernel
> > > is running in EL1, even if EL2 is supported and cntpct_el0 is
> > > accessible.
> > > 
> > > In our case:
> > > - We are not using pKVM.
> > > - The kernel is booted in EL1.
> > > - We disabled VIRT_PPI and explicitly selected PHYS_NONSECURE_PPI for the timer refering to below code.
> > 
> > That's not legal. The architecture guarantees that there is a virtual
> > timer and a physical timer. No ifs, no buts.
> > 
> > [...]
> > 
> > > As I understand it, `is_hyp_mode_available()` checks whether the
> > > kernel booted into EL2 — not whether EL2 is *supported* by the
> > > hardware.
> > > 
> > > Therefore, even on systems where EL2 exists and `cntpct_el0` is
> > > accessible from EL1, the kernel still forces the use of `cntvct_el0`
> > > if the boot EL is EL1.
> > 
> > Yes, because it isn't architecturally valid to not have a virtual
> > timer. This isn't about EL2 being present of not. The switch to the
> > physical timer is purely an optimisation for KVM so that it doesn't
> > have to switch the virtual timer back and forth when running a guest,
> > as the virtual timer is the most likely used timer.
> > 
> 
> Thanks for the clarification.
> 
> As a follow-up question:
> 
> We are working on a system that uses a vendor-specific hypervisor instead of KVM.
> In this setup, we also want to optimize timer virtualization overhead and are considering using
> the physical timer (CNTPT) in the host context for performance reasons, just like KVM does.
> 
> Would it be acceptable (from the upstream kernel's perspective) to make a similar switch
> to the physical timer in this case ?

No. Your hypervisor already has *two* private timers it can freely
make use of (virtual and physical EL2 timers), and doesn't need to
encroach on something that a guest (be it Linux or any other guest)
relies on.

The alternative is to trap and emulate the EL1 timer for the guest so
that it *appears* to be functional. But that's obviously bad from a
performance perspective.

> Or is this kind of optimization strictly tied to KVM's internal behavior
> and not something the kernel is expected to support generically?

It is purely Linux/KVM specific, and only works because we own both
side of that equation, meaning we can enforce whatever is required to
make the two work together. This obviously isn't possible with third
party software. Look at it from a different point of view: how would
you make this work with, say, Windows? or MacOS?

On the bright side, the architecture already gives you everything you
need to implement your hypervisor. Just use it correctly.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [QUESTION] arch_counter_register() restricts CNTPT access when booted in EL1, even if EL2 is supported

[Youngmin Nam]

[-- Attachment #1: Type: text/plain, Size: 3692 bytes --]

On Mon, May 19, 2025 at 08:12:24AM +0100, Marc Zyngier wrote:
> On Mon, 19 May 2025 02:43:49 +0100,
> Youngmin Nam <youngmin.nam@samsung.com> wrote:
> > 
> > [1  <text/plain; utf-8 (8bit)>]
> > On Fri, May 16, 2025 at 10:28:56AM +0100, Marc Zyngier wrote:
> > > On Fri, 16 May 2025 07:53:58 +0100,
> > > Youngmin Nam <youngmin.nam@samsung.com> wrote:
> > > > 
> > > > [1  <text/plain; utf-8 (8bit)>]
> > > > Hi arm arch timer experts,
> > > > 
> > > > While reviewing the arm_arch_timer code in Linux 6.12,
> > > > I noticed that the function arch_counter_register() restricts the
> > > > use of the physical counter (cntpct_el0) on systems where the kernel
> > > > is running in EL1, even if EL2 is supported and cntpct_el0 is
> > > > accessible.
> > > > 
> > > > In our case:
> > > > - We are not using pKVM.
> > > > - The kernel is booted in EL1.
> > > > - We disabled VIRT_PPI and explicitly selected PHYS_NONSECURE_PPI for the timer refering to below code.
> > > 
> > > That's not legal. The architecture guarantees that there is a virtual
> > > timer and a physical timer. No ifs, no buts.
> > > 
> > > [...]
> > > 
> > > > As I understand it, `is_hyp_mode_available()` checks whether the
> > > > kernel booted into EL2 — not whether EL2 is *supported* by the
> > > > hardware.
> > > > 
> > > > Therefore, even on systems where EL2 exists and `cntpct_el0` is
> > > > accessible from EL1, the kernel still forces the use of `cntvct_el0`
> > > > if the boot EL is EL1.
> > > 
> > > Yes, because it isn't architecturally valid to not have a virtual
> > > timer. This isn't about EL2 being present of not. The switch to the
> > > physical timer is purely an optimisation for KVM so that it doesn't
> > > have to switch the virtual timer back and forth when running a guest,
> > > as the virtual timer is the most likely used timer.
> > > 
> > 
> > Thanks for the clarification.
> > 
> > As a follow-up question:
> > 
> > We are working on a system that uses a vendor-specific hypervisor instead of KVM.
> > In this setup, we also want to optimize timer virtualization overhead and are considering using
> > the physical timer (CNTPT) in the host context for performance reasons, just like KVM does.
> > 
> > Would it be acceptable (from the upstream kernel's perspective) to make a similar switch
> > to the physical timer in this case ?
> 
> No. Your hypervisor already has *two* private timers it can freely
> make use of (virtual and physical EL2 timers), and doesn't need to
> encroach on something that a guest (be it Linux or any other guest)
> relies on.
> 
> The alternative is to trap and emulate the EL1 timer for the guest so
> that it *appears* to be functional. But that's obviously bad from a
> performance perspective.
> 
> > Or is this kind of optimization strictly tied to KVM's internal behavior
> > and not something the kernel is expected to support generically?
> 
> It is purely Linux/KVM specific, and only works because we own both
> side of that equation, meaning we can enforce whatever is required to
> make the two work together. This obviously isn't possible with third
> party software. Look at it from a different point of view: how would
> you make this work with, say, Windows? or MacOS?
> 
> On the bright side, the architecture already gives you everything you
> need to implement your hypervisor. Just use it correctly.
> 
> Thanks,
> 
> 	M.

Hi Marc,

Thank you very much for the detailed explanation and your time.
Your clarification about the architectural intent and KVM-specific behavior
was really helpful and made things much clearer on our side.

Best regards,
Youngmin

> 
> -- 
> Without deviation from the norm, progress is not possible.
> 

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]