[PATCH] arm64: Document requirements for fine grained traps at boot

[PATCH] arm64: Document requirements for fine grained traps at boot

[Mark Brown]

The arm64 FEAT_FGT extension introduces a set of traps to EL2 for accesses
to small sets of registers and instructions from EL1 and EL0.  Currently
Linux makes no use of this feature, explicitly document that it should
be disabled when entering the kernel at EL2 (as is the architectural
default) to help avoid surprises.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 Documentation/arm64/booting.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/Documentation/arm64/booting.rst b/Documentation/arm64/booting.rst
index 7552dbc1cc54..1efc2d3023bb 100644
--- a/Documentation/arm64/booting.rst
+++ b/Documentation/arm64/booting.rst
@@ -270,6 +270,13 @@ Before jumping into the kernel, the following conditions must be met:
       having 0b1 set for the corresponding bit for each of the auxiliary
       counters present.
 
+  For CPUs with Fine Grained Traps (FEAT_FGT) extension present:
+
+  - If the kernel is entered at EL2:
+
+    - HAFGRTR_EL2, HDFGWTR_EL2, HDFGRTR_EL2, HFGWTR_EL2, HFGRTR_EL2 and
+      HFGITR_EL2 must be initialised to 0.
+
 The requirements described above for CPU mode, caches, MMUs, architected
 timers, coherency and system registers apply to all CPUs.  All CPUs must
 enter the kernel in the same exception level.
-- 
2.20.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH] arm64: Document requirements for fine grained traps at boot

[Catalin Marinas]

On Fri, Mar 12, 2021 at 03:49:17PM +0000, Mark Brown wrote:
> The arm64 FEAT_FGT extension introduces a set of traps to EL2 for accesses
> to small sets of registers and instructions from EL1 and EL0.  Currently
> Linux makes no use of this feature, explicitly document that it should
> be disabled when entering the kernel at EL2 (as is the architectural
> default) to help avoid surprises.
> 
> Signed-off-by: Mark Brown <broonie@kernel.org>
> ---
>  Documentation/arm64/booting.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/Documentation/arm64/booting.rst b/Documentation/arm64/booting.rst
> index 7552dbc1cc54..1efc2d3023bb 100644
> --- a/Documentation/arm64/booting.rst
> +++ b/Documentation/arm64/booting.rst
> @@ -270,6 +270,13 @@ Before jumping into the kernel, the following conditions must be met:
>        having 0b1 set for the corresponding bit for each of the auxiliary
>        counters present.
>  
> +  For CPUs with Fine Grained Traps (FEAT_FGT) extension present:
> +
> +  - If the kernel is entered at EL2:
> +
> +    - HAFGRTR_EL2, HDFGWTR_EL2, HDFGRTR_EL2, HFGWTR_EL2, HFGRTR_EL2 and
> +      HFGITR_EL2 must be initialised to 0.

While this requirement is correct, documenting such individual registers
doesn't scales well. You may run a 5 year old kernel on a newer CPU and
we can't predict which control registers have been added and what
side-effect they have. The architecture, at least for the above
registers, states that if warm reset to EL2, their value is 0. I think
the EL3 firmware (which is normally up to date with the CPU it is
running on) should follow the ARM ARM reset values. There are probably
EL1 registers with similar requirements (I haven't checked).

Can we instead have a broad statement regarding any EL1/EL2 registers
that they should be either rest to 0 or to the architectural (warm)
reset value as per the ARM ARM? Or something like any feature must be
disabled by default at the EL1/EL2 control registers level and this
would imply the fine-grained traps.

We currently have this statement:

  All writable architected system registers at the exception level where
  the kernel image will be entered must be initialised by software at a
  higher exception level to prevent execution in an UNKNOWN state.

The "prevent execution in an UNKNOWN state" needs to be clearer. The
above should also include exception levels _below_ the one where the
kernel is entered. It doesn't help if KVM is old and has no clue of new
EL1-specific registers.

Adding Mark R, I think he looked at some of these in the past.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH] arm64: Document requirements for fine grained traps at boot

[Will Deacon]

On Fri, Mar 26, 2021 at 11:55:41AM +0000, Catalin Marinas wrote:
> On Fri, Mar 12, 2021 at 03:49:17PM +0000, Mark Brown wrote:
> > The arm64 FEAT_FGT extension introduces a set of traps to EL2 for accesses
> > to small sets of registers and instructions from EL1 and EL0.  Currently
> > Linux makes no use of this feature, explicitly document that it should
> > be disabled when entering the kernel at EL2 (as is the architectural
> > default) to help avoid surprises.
> > 
> > Signed-off-by: Mark Brown <broonie@kernel.org>
> > ---
> >  Documentation/arm64/booting.rst | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/Documentation/arm64/booting.rst b/Documentation/arm64/booting.rst
> > index 7552dbc1cc54..1efc2d3023bb 100644
> > --- a/Documentation/arm64/booting.rst
> > +++ b/Documentation/arm64/booting.rst
> > @@ -270,6 +270,13 @@ Before jumping into the kernel, the following conditions must be met:
> >        having 0b1 set for the corresponding bit for each of the auxiliary
> >        counters present.
> >  
> > +  For CPUs with Fine Grained Traps (FEAT_FGT) extension present:
> > +
> > +  - If the kernel is entered at EL2:
> > +
> > +    - HAFGRTR_EL2, HDFGWTR_EL2, HDFGRTR_EL2, HFGWTR_EL2, HFGRTR_EL2 and
> > +      HFGITR_EL2 must be initialised to 0.
> 
> While this requirement is correct, documenting such individual registers
> doesn't scales well. You may run a 5 year old kernel on a newer CPU and
> we can't predict which control registers have been added and what
> side-effect they have. The architecture, at least for the above
> registers, states that if warm reset to EL2, their value is 0. I think
> the EL3 firmware (which is normally up to date with the CPU it is
> running on) should follow the ARM ARM reset values. There are probably
> EL1 registers with similar requirements (I haven't checked).

One thing I don't understand about the registers listed here is that we're
requiring firmware to initialise them when the kernel is entered at EL2. But
they're *_EL2 registers, so why can't the kernel initialise them itself? The
fewer dependencies on firmware, the better.

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH] arm64: Document requirements for fine grained traps at boot

[Catalin Marinas]

On Mon, Mar 29, 2021 at 11:31:27AM +0100, Will Deacon wrote:
> On Fri, Mar 26, 2021 at 11:55:41AM +0000, Catalin Marinas wrote:
> > On Fri, Mar 12, 2021 at 03:49:17PM +0000, Mark Brown wrote:
> > > The arm64 FEAT_FGT extension introduces a set of traps to EL2 for accesses
> > > to small sets of registers and instructions from EL1 and EL0.  Currently
> > > Linux makes no use of this feature, explicitly document that it should
> > > be disabled when entering the kernel at EL2 (as is the architectural
> > > default) to help avoid surprises.
> > > 
> > > Signed-off-by: Mark Brown <broonie@kernel.org>
> > > ---
> > >  Documentation/arm64/booting.rst | 7 +++++++
> > >  1 file changed, 7 insertions(+)
> > > 
> > > diff --git a/Documentation/arm64/booting.rst b/Documentation/arm64/booting.rst
> > > index 7552dbc1cc54..1efc2d3023bb 100644
> > > --- a/Documentation/arm64/booting.rst
> > > +++ b/Documentation/arm64/booting.rst
> > > @@ -270,6 +270,13 @@ Before jumping into the kernel, the following conditions must be met:
> > >        having 0b1 set for the corresponding bit for each of the auxiliary
> > >        counters present.
> > >  
> > > +  For CPUs with Fine Grained Traps (FEAT_FGT) extension present:
> > > +
> > > +  - If the kernel is entered at EL2:
> > > +
> > > +    - HAFGRTR_EL2, HDFGWTR_EL2, HDFGRTR_EL2, HFGWTR_EL2, HFGRTR_EL2 and
> > > +      HFGITR_EL2 must be initialised to 0.
> > 
> > While this requirement is correct, documenting such individual registers
> > doesn't scales well. You may run a 5 year old kernel on a newer CPU and
> > we can't predict which control registers have been added and what
> > side-effect they have. The architecture, at least for the above
> > registers, states that if warm reset to EL2, their value is 0. I think
> > the EL3 firmware (which is normally up to date with the CPU it is
> > running on) should follow the ARM ARM reset values. There are probably
> > EL1 registers with similar requirements (I haven't checked).
> 
> One thing I don't understand about the registers listed here is that we're
> requiring firmware to initialise them when the kernel is entered at EL2. But
> they're *_EL2 registers, so why can't the kernel initialise them itself? The
> fewer dependencies on firmware, the better.

W.r.t. these specific registers, you are right, we'd better initialise
them in the kernel rather than documenting. But in the general case,
there may be new registers an old kernel is not aware of. Here we expect
at least the EL3 firmware to be up to date with the CPU it is running
on, hence a broad statement that such registers need to be initialised
(unless we have one already).

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH] arm64: Document requirements for fine grained traps at boot

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 873 bytes --]

On Mon, Mar 29, 2021 at 11:31:27AM +0100, Will Deacon wrote:

> One thing I don't understand about the registers listed here is that we're
> requiring firmware to initialise them when the kernel is entered at EL2. But
> they're *_EL2 registers, so why can't the kernel initialise them itself? The
> fewer dependencies on firmware, the better.

Well, there's nothing stopping us doing both - that's the most robust
thing.  Relying on the kernel alone to do this has the old kernels on
new systems issue, and there's always going to be some section of code
running before we manage to go through and enumerate features then reset
the system registers even if we run that code very early.  With
something like traps it seems especially relevant, it's more of a "could
affect anything" kind of thing than something where we can initialize
the feature before we start using it.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH] arm64: Document requirements for fine grained traps at boot

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 1736 bytes --]

On Fri, Mar 26, 2021 at 11:55:41AM +0000, Catalin Marinas wrote:
> On Fri, Mar 12, 2021 at 03:49:17PM +0000, Mark Brown wrote:

> > +    - HAFGRTR_EL2, HDFGWTR_EL2, HDFGRTR_EL2, HFGWTR_EL2, HFGRTR_EL2 and
> > +      HFGITR_EL2 must be initialised to 0.

> While this requirement is correct, documenting such individual registers
> doesn't scales well. You may run a 5 year old kernel on a newer CPU and
> we can't predict which control registers have been added and what
> side-effect they have. The architecture, at least for the above

I'm not a huge fan of writing things this way either, I'd inferred that
this sort of minimal and specific thing was the idiom desired so was
trying to follow.

> Can we instead have a broad statement regarding any EL1/EL2 registers
> that they should be either rest to 0 or to the architectural (warm)
> reset value as per the ARM ARM? Or something like any feature must be
> disabled by default at the EL1/EL2 control registers level and this
> would imply the fine-grained traps.

> We currently have this statement:

>   All writable architected system registers at the exception level where
>   the kernel image will be entered must be initialised by software at a
>   higher exception level to prevent execution in an UNKNOWN state.

> The "prevent execution in an UNKNOWN state" needs to be clearer. The
> above should also include exception levels _below_ the one where the
> kernel is entered. It doesn't help if KVM is old and has no clue of new
> EL1-specific registers.

Right, I think both those things are needed to cover the general case.
The exception levels bit is hopefully straightforward but wording for
the clarification bit is more tricky - I'll try to come up with
something.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel