[PATCH for-8.1 0/3] target/arm: Fix ptw bugs introduced by FEAT_RME changes

[PATCH for-8.1 0/3] target/arm: Fix ptw bugs introduced by FEAT_RME changes

[Peter Maydell]

This patchset fixes a couple of bugs in the arm ptw code that were
inadvertently introduced by the recent changes to support FEAT_RME.
One is with the handling of debug accesses in Secure state with
EL2 disabled, reported by Jean-Philippe. The other is one I noticed
while reading the code where if VSTCR.SA/VTCR.NSA require the
output of a two-stage translation to be squashed to NS then we
don't also set the f.attrs.space accordingly.

I also think we need to refactor the ptw code to stop passing bool
in_secure around (both directly as function arguments and in the
S1Translate struct) and instead always use an ARMSecuritySpace.
However I don't think that will affect any use cases that don't
turn on the (experimental) FEAT_RME support, so it's not so
urgent as this bugfix.

Patch 1 is just documenting the meaning of some of the S1Translate
struct fields, since I've now had to think through this twice and
hope to avoid doing so a third time :-)

thanks
-- PMM

Peter Maydell (3):
  target/arm/ptw.c: Add comments to S1Translate struct fields
  target/arm: Fix S1_ptw_translate() debug path
  target/arm/ptw.c: Account for FEAT_RME when applying {N}SW,SA bits

 target/arm/ptw.c | 90 ++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 80 insertions(+), 10 deletions(-)

-- 
2.34.1

[PATCH for-8.1 1/3] target/arm/ptw.c: Add comments to S1Translate struct fields

[Peter Maydell]

Add comments to the in_* fields in the S1Translate struct
that explain what they're doing.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
I figured some of this out when writing commit fcc0b0418fff,
and then I found I'd forgotten it all when I was trying
to fix this new bug. So this time I'm writing this down :-)
---
 target/arm/ptw.c | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 9aaff1546a6..21749375f97 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -19,10 +19,50 @@
 #endif
 
 typedef struct S1Translate {
+    /*
+     * in_mmu_idx : specifies which TTBR, TCR, etc to use for the walk.
+     * Together with in_space, specifies the architectural translation regime.
+     */
     ARMMMUIdx in_mmu_idx;
+    /*
+     * in_ptw_idx: specifies which mmuidx to use for the actual
+     * page table descriptor load operations. This will be one of the
+     * ARMMMUIdx_Stage2* or one of the ARMMMUIdx_Phys_* indexes.
+     * If a Secure ptw is "downgraded" to NonSecure by an NSTable bit,
+     * this field is updated accordingly.
+     */
     ARMMMUIdx in_ptw_idx;
+    /*
+     * in_space: the security space for this walk. This plus
+     * the in_mmu_idx specify the architectural translation regime.
+     * If a Secure ptw is "downgraded" to NonSecure by an NSTable bit,
+     * this field is updated accordingly.
+     *
+     * Note that the security space for the in_ptw_idx may be different
+     * from that for the in_mmu_idx. We do not need to explicitly track
+     * the in_ptw_idx security space because:
+     *  - if the in_ptw_idx is an ARMMMUIdx_Phys_* then the mmuidx
+     *    itself specifies the security space
+     *  - if the in_ptw_idx is an ARMMMUIdx_Stage2* then the security
+     *    space used for ptw reads is the same as that of the security
+     *    space of the stage 1 translation for all cases except where
+     *    stage 1 is Secure; in that case the only possibilities for
+     *    the ptw read are Secure and NonSecure, and the in_ptw_idx
+     *    value being Stage2 vs Stage2_S distinguishes those.
+     */
     ARMSecuritySpace in_space;
+    /*
+     * in_secure: whether the translation regime is a Secure one.
+     * This is always equal to arm_space_is_secure(in_space).
+     * If a Secure ptw is "downgraded" to NonSecure by an NSTable bit,
+     * this field is updated accordingly.
+     */
     bool in_secure;
+    /*
+     * in_debug: is this a QEMU debug access (gdbstub, etc)? Debug
+     * accesses will not update the guest page table access flags
+     * and will not change the state of the softmmu TLBs.
+     */
     bool in_debug;
     /*
      * If this is stage 2 of a stage 1+2 page table walk, then this must
-- 
2.34.1

Re: [PATCH for-8.1 1/3] target/arm/ptw.c: Add comments to S1Translate struct fields

[Richard Henderson]

On 7/10/23 16:21, Peter Maydell wrote:
> Add comments to the in_* fields in the S1Translate struct
> that explain what they're doing.
> 
> Signed-off-by: Peter Maydell<peter.maydell@linaro.org>
> ---
> I figured some of this out when writing commit fcc0b0418fff,
> and then I found I'd forgotten it all when I was trying
> to fix this new bug. So this time I'm writing this down :-)
> ---
>   target/arm/ptw.c | 40 ++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 40 insertions(+)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

[PATCH for-8.1 2/3] target/arm: Fix S1_ptw_translate() debug path

[Peter Maydell]

In commit XXX we rearranged the logic in S1_ptw_translate() so that
the debug-access "call get_phys_addr_*" codepath is used both when S1
is doing ptw reads from stage 2 and when it is doing ptw reads from
physical memory.  However, we didn't update the calculation of
s2ptw->in_space and s2ptw->in_secure to account for the "ptw reads
from physical memory" case.  This meant that debug accesses when in
Secure state broke.

Create a new function S2_security_space() which returns the
correct security space to use for the ptw load, and use it to
determine the correct .in_secure and .in_space fields for the
stage 2 lookup for the ptw load.

Reported-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Fixes: fe4a5472ccd6 ("target/arm: Use get_phys_addr_with_struct in S1_ptw_translate")
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/ptw.c | 37 ++++++++++++++++++++++++++++++++-----
 1 file changed, 32 insertions(+), 5 deletions(-)

diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 21749375f97..c0b9cee5843 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -485,11 +485,39 @@ static bool S2_attrs_are_device(uint64_t hcr, uint8_t attrs)
     }
 }
 
+static ARMSecuritySpace S2_security_space(ARMSecuritySpace s1_space,
+                                          ARMMMUIdx s2_mmu_idx)
+{
+    /*
+     * Return the security space to use for stage 2 when doing
+     * the S1 page table descriptor load.
+     */
+    if (regime_is_stage2(s2_mmu_idx)) {
+        /*
+         * The security space for ptw reads is almost always the same
+         * as that of the security space of the stage 1 translation.
+         * The only exception is when stage 1 is Secure; in that case
+         * the ptw read might be to the Secure or the NonSecure space
+         * (but never Realm or Root), and the s2_mmu_idx tells us which.
+         * Root translations are always single-stage.
+         */
+        if (s1_space == ARMSS_Secure) {
+            return arm_secure_to_space(s2_mmu_idx == ARMMMUIdx_Stage2_S);
+        } else {
+            assert(s2_mmu_idx != ARMMMUIdx_Stage2_S);
+            assert(s1_space != ARMSS_Root);
+            return s1_space;
+        }
+    } else {
+        /* ptw loads are from phys: the mmu idx itself says which space */
+        return arm_phys_to_space(s2_mmu_idx);
+    }
+}
+
 /* Translate a S1 pagetable walk through S2 if needed.  */
 static bool S1_ptw_translate(CPUARMState *env, S1Translate *ptw,
                              hwaddr addr, ARMMMUFaultInfo *fi)
 {
-    ARMSecuritySpace space = ptw->in_space;
     bool is_secure = ptw->in_secure;
     ARMMMUIdx mmu_idx = ptw->in_mmu_idx;
     ARMMMUIdx s2_mmu_idx = ptw->in_ptw_idx;
@@ -502,13 +530,12 @@ static bool S1_ptw_translate(CPUARMState *env, S1Translate *ptw,
          * From gdbstub, do not use softmmu so that we don't modify the
          * state of the cpu at all, including softmmu tlb contents.
          */
+        ARMSecuritySpace s2_space = S2_security_space(ptw->in_space, s2_mmu_idx);
         S1Translate s2ptw = {
             .in_mmu_idx = s2_mmu_idx,
             .in_ptw_idx = ptw_idx_for_stage_2(env, s2_mmu_idx),
-            .in_secure = s2_mmu_idx == ARMMMUIdx_Stage2_S,
-            .in_space = (s2_mmu_idx == ARMMMUIdx_Stage2_S ? ARMSS_Secure
-                         : space == ARMSS_Realm ? ARMSS_Realm
-                         : ARMSS_NonSecure),
+            .in_secure = arm_space_is_secure(s2_space),
+            .in_space = s2_space,
             .in_debug = true,
         };
         GetPhysAddrResult s2 = { };
-- 
2.34.1

Re: [PATCH for-8.1 2/3] target/arm: Fix S1_ptw_translate() debug path

[Peter Maydell]

On Mon, 10 Jul 2023 at 16:21, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> In commit XXX we rearranged the logic in S1_ptw_translate() so that

Should read "commit fe4a5472ccd6" -- I put in the Fixes:
tag below but forgot to update the placeholder in the
commit message text...

> the debug-access "call get_phys_addr_*" codepath is used both when S1
> is doing ptw reads from stage 2 and when it is doing ptw reads from
> physical memory.  However, we didn't update the calculation of
> s2ptw->in_space and s2ptw->in_secure to account for the "ptw reads
> from physical memory" case.  This meant that debug accesses when in
> Secure state broke.
>
> Create a new function S2_security_space() which returns the
> correct security space to use for the ptw load, and use it to
> determine the correct .in_secure and .in_space fields for the
> stage 2 lookup for the ptw load.
>
> Reported-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> Fixes: fe4a5472ccd6 ("target/arm: Use get_phys_addr_with_struct in S1_ptw_translate")
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

-- PMM

Re: [PATCH for-8.1 2/3] target/arm: Fix S1_ptw_translate() debug path

[Jean-Philippe Brucker]

On Mon, Jul 10, 2023 at 04:21:29PM +0100, Peter Maydell wrote:
> In commit XXX we rearranged the logic in S1_ptw_translate() so that
> the debug-access "call get_phys_addr_*" codepath is used both when S1
> is doing ptw reads from stage 2 and when it is doing ptw reads from
> physical memory.  However, we didn't update the calculation of
> s2ptw->in_space and s2ptw->in_secure to account for the "ptw reads
> from physical memory" case.  This meant that debug accesses when in
> Secure state broke.
> 
> Create a new function S2_security_space() which returns the
> correct security space to use for the ptw load, and use it to
> determine the correct .in_secure and .in_space fields for the
> stage 2 lookup for the ptw load.
> 
> Reported-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> Fixes: fe4a5472ccd6 ("target/arm: Use get_phys_addr_with_struct in S1_ptw_translate")
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Thanks, this fixes tf-a boot with semihosting

Tested-by: Jean-Philippe Brucker <jean-philippe@linaro.org>

> ---
>  target/arm/ptw.c | 37 ++++++++++++++++++++++++++++++++-----
>  1 file changed, 32 insertions(+), 5 deletions(-)
> 
> diff --git a/target/arm/ptw.c b/target/arm/ptw.c
> index 21749375f97..c0b9cee5843 100644
> --- a/target/arm/ptw.c
> +++ b/target/arm/ptw.c
> @@ -485,11 +485,39 @@ static bool S2_attrs_are_device(uint64_t hcr, uint8_t attrs)
>      }
>  }
>  
> +static ARMSecuritySpace S2_security_space(ARMSecuritySpace s1_space,
> +                                          ARMMMUIdx s2_mmu_idx)
> +{
> +    /*
> +     * Return the security space to use for stage 2 when doing
> +     * the S1 page table descriptor load.
> +     */
> +    if (regime_is_stage2(s2_mmu_idx)) {
> +        /*
> +         * The security space for ptw reads is almost always the same
> +         * as that of the security space of the stage 1 translation.
> +         * The only exception is when stage 1 is Secure; in that case
> +         * the ptw read might be to the Secure or the NonSecure space
> +         * (but never Realm or Root), and the s2_mmu_idx tells us which.
> +         * Root translations are always single-stage.
> +         */
> +        if (s1_space == ARMSS_Secure) {
> +            return arm_secure_to_space(s2_mmu_idx == ARMMMUIdx_Stage2_S);
> +        } else {
> +            assert(s2_mmu_idx != ARMMMUIdx_Stage2_S);
> +            assert(s1_space != ARMSS_Root);
> +            return s1_space;
> +        }
> +    } else {
> +        /* ptw loads are from phys: the mmu idx itself says which space */
> +        return arm_phys_to_space(s2_mmu_idx);
> +    }
> +}
> +
>  /* Translate a S1 pagetable walk through S2 if needed.  */
>  static bool S1_ptw_translate(CPUARMState *env, S1Translate *ptw,
>                               hwaddr addr, ARMMMUFaultInfo *fi)
>  {
> -    ARMSecuritySpace space = ptw->in_space;
>      bool is_secure = ptw->in_secure;
>      ARMMMUIdx mmu_idx = ptw->in_mmu_idx;
>      ARMMMUIdx s2_mmu_idx = ptw->in_ptw_idx;
> @@ -502,13 +530,12 @@ static bool S1_ptw_translate(CPUARMState *env, S1Translate *ptw,
>           * From gdbstub, do not use softmmu so that we don't modify the
>           * state of the cpu at all, including softmmu tlb contents.
>           */
> +        ARMSecuritySpace s2_space = S2_security_space(ptw->in_space, s2_mmu_idx);
>          S1Translate s2ptw = {
>              .in_mmu_idx = s2_mmu_idx,
>              .in_ptw_idx = ptw_idx_for_stage_2(env, s2_mmu_idx),
> -            .in_secure = s2_mmu_idx == ARMMMUIdx_Stage2_S,
> -            .in_space = (s2_mmu_idx == ARMMMUIdx_Stage2_S ? ARMSS_Secure
> -                         : space == ARMSS_Realm ? ARMSS_Realm
> -                         : ARMSS_NonSecure),
> +            .in_secure = arm_space_is_secure(s2_space),
> +            .in_space = s2_space,
>              .in_debug = true,
>          };
>          GetPhysAddrResult s2 = { };
> -- 
> 2.34.1
> 

Re: [PATCH for-8.1 2/3] target/arm: Fix S1_ptw_translate() debug path

[Richard Henderson]

On 7/10/23 16:21, Peter Maydell wrote:
> In commit XXX we rearranged the logic in S1_ptw_translate() so that
> the debug-access "call get_phys_addr_*" codepath is used both when S1
> is doing ptw reads from stage 2 and when it is doing ptw reads from
> physical memory.  However, we didn't update the calculation of
> s2ptw->in_space and s2ptw->in_secure to account for the "ptw reads
> from physical memory" case.  This meant that debug accesses when in
> Secure state broke.
> 
> Create a new function S2_security_space() which returns the
> correct security space to use for the ptw load, and use it to
> determine the correct .in_secure and .in_space fields for the
> stage 2 lookup for the ptw load.
> 
> Reported-by: Jean-Philippe Brucker<jean-philippe@linaro.org>
> Fixes: fe4a5472ccd6 ("target/arm: Use get_phys_addr_with_struct in S1_ptw_translate")
> Signed-off-by: Peter Maydell<peter.maydell@linaro.org>
> ---
>   target/arm/ptw.c | 37 ++++++++++++++++++++++++++++++++-----
>   1 file changed, 32 insertions(+), 5 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

[PATCH for-8.1 3/3] target/arm/ptw.c: Account for FEAT_RME when applying {N}SW, SA bits

[Peter Maydell]

In get_phys_addr_twostage() the code that applies the effects of
VSTCR.{SA,SW} and VTCR.{NSA,NSW} only updates result->f.attrs.secure.
Now we also have f.attrs.space for FEAT_RME, we need to keep the two
in sync.

These bits only have an effect for Secure space translations, not
for Root, so use the input in_space field to determine whether to
apply them rather than the input is_secure. This doesn't actually
make a difference because Root translations are never two-stage,
but it's a little clearer.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
I noticed this while reading through the ptw code...
---
 target/arm/ptw.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index c0b9cee5843..8f94100c61f 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -3118,6 +3118,7 @@ static bool get_phys_addr_twostage(CPUARMState *env, S1Translate *ptw,
     hwaddr ipa;
     int s1_prot, s1_lgpgsz;
     bool is_secure = ptw->in_secure;
+    ARMSecuritySpace in_space = ptw->in_space;
     bool ret, ipa_secure;
     ARMCacheAttrs cacheattrs1;
     ARMSecuritySpace ipa_space;
@@ -3200,11 +3201,13 @@ static bool get_phys_addr_twostage(CPUARMState *env, S1Translate *ptw,
      * Check if IPA translates to secure or non-secure PA space.
      * Note that VSTCR overrides VTCR and {N}SW overrides {N}SA.
      */
-    result->f.attrs.secure =
-        (is_secure
-         && !(env->cp15.vstcr_el2 & (VSTCR_SA | VSTCR_SW))
-         && (ipa_secure
-             || !(env->cp15.vtcr_el2 & (VTCR_NSA | VTCR_NSW))));
+    if (in_space == ARMSS_Secure) {
+        result->f.attrs.secure =
+            !(env->cp15.vstcr_el2 & (VSTCR_SA | VSTCR_SW))
+            && (ipa_secure
+                || !(env->cp15.vtcr_el2 & (VTCR_NSA | VTCR_NSW)));
+        result->f.attrs.space = arm_secure_to_space(result->f.attrs.secure);
+    }
 
     return false;
 }
-- 
2.34.1

Re: [PATCH for-8.1 3/3] target/arm/ptw.c: Account for FEAT_RME when applying {N}SW,SA bits

[Richard Henderson]

On 7/10/23 16:21, Peter Maydell wrote:
> In get_phys_addr_twostage() the code that applies the effects of
> VSTCR.{SA,SW} and VTCR.{NSA,NSW} only updates result->f.attrs.secure.
> Now we also have f.attrs.space for FEAT_RME, we need to keep the two
> in sync.
> 
> These bits only have an effect for Secure space translations, not
> for Root, so use the input in_space field to determine whether to
> apply them rather than the input is_secure. This doesn't actually
> make a difference because Root translations are never two-stage,
> but it's a little clearer.
> 
> Signed-off-by: Peter Maydell<peter.maydell@linaro.org>
> ---
> I noticed this while reading through the ptw code...
> ---
>   target/arm/ptw.c | 13 ++++++++-----
>   1 file changed, 8 insertions(+), 5 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~