* [PATCH v3 01/42] target/arm: Split s2walk_secure from ipa_secure in get_phys_addr
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-06 14:27 ` Peter Maydell
2022-10-01 16:22 ` [PATCH v3 02/42] target/arm: Add is_secure parameter to get_phys_addr_lpae Richard Henderson
` (41 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
The starting security state comes with the translation regime,
not the current state of arm_is_secure_below_el3().
Create a new local variable, s2walk_secure, which does not need
to be written back to result->attrs.secure -- we compute that
value later, after the S2 walk is complete.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
v3: Do not modify ipa_secure, per review.
---
target/arm/ptw.c | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 2ddfc028ab..b8c494ad9f 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -2298,7 +2298,7 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
hwaddr ipa;
int s1_prot;
int ret;
- bool ipa_secure;
+ bool ipa_secure, s2walk_secure;
ARMCacheAttrs cacheattrs1;
ARMMMUIdx s2_mmu_idx;
bool is_el0;
@@ -2313,17 +2313,17 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
ipa = result->phys;
ipa_secure = result->attrs.secure;
- if (arm_is_secure_below_el3(env)) {
- if (ipa_secure) {
- result->attrs.secure = !(env->cp15.vstcr_el2 & VSTCR_SW);
- } else {
- result->attrs.secure = !(env->cp15.vtcr_el2 & VTCR_NSW);
- }
+ if (is_secure) {
+ /* Select TCR based on the NS bit from the S1 walk. */
+ s2walk_secure = !(ipa_secure
+ ? env->cp15.vstcr_el2 & VSTCR_SW
+ : env->cp15.vtcr_el2 & VTCR_NSW);
} else {
assert(!ipa_secure);
+ s2walk_secure = false;
}
- s2_mmu_idx = (result->attrs.secure
+ s2_mmu_idx = (s2walk_secure
? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2);
is_el0 = mmu_idx == ARMMMUIdx_E10_0 || mmu_idx == ARMMMUIdx_SE10_0;
@@ -2366,7 +2366,7 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
result->cacheattrs);
/* Check if IPA translates to secure or non-secure PA space. */
- if (arm_is_secure_below_el3(env)) {
+ if (is_secure) {
if (ipa_secure) {
result->attrs.secure =
!(env->cp15.vstcr_el2 & (VSTCR_SA | VSTCR_SW));
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 01/42] target/arm: Split s2walk_secure from ipa_secure in get_phys_addr
2022-10-01 16:22 ` [PATCH v3 01/42] target/arm: Split s2walk_secure from ipa_secure in get_phys_addr Richard Henderson
@ 2022-10-06 14:27 ` Peter Maydell
2022-10-06 15:10 ` Richard Henderson
0 siblings, 1 reply; 90+ messages in thread
From: Peter Maydell @ 2022-10-06 14:27 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:24, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> The starting security state comes with the translation regime,
> not the current state of arm_is_secure_below_el3().
>
> Create a new local variable, s2walk_secure, which does not need
> to be written back to result->attrs.secure -- we compute that
> value later, after the S2 walk is complete.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> v3: Do not modify ipa_secure, per review.
> ---
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
I did find myself wondering if we should explicitly set
result->attrs.secure = false;
in an else-branch of the last "if (is_secure)", though.
At the moment we rely on get_phys_addr_lpae() for the stage2
doing that for us, I think. Having the code here always set
result->attrs.secure before the 'return 0' avoids having to think
about that...
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v3 01/42] target/arm: Split s2walk_secure from ipa_secure in get_phys_addr
2022-10-06 14:27 ` Peter Maydell
@ 2022-10-06 15:10 ` Richard Henderson
2022-10-06 15:22 ` Peter Maydell
0 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-06 15:10 UTC (permalink / raw)
To: Peter Maydell; +Cc: qemu-devel, qemu-arm
On 10/6/22 07:27, Peter Maydell wrote:
> On Sat, 1 Oct 2022 at 17:24, Richard Henderson
> <richard.henderson@linaro.org> wrote:
>>
>> The starting security state comes with the translation regime,
>> not the current state of arm_is_secure_below_el3().
>>
>> Create a new local variable, s2walk_secure, which does not need
>> to be written back to result->attrs.secure -- we compute that
>> value later, after the S2 walk is complete.
>>
>> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
>> ---
>> v3: Do not modify ipa_secure, per review.
>> ---
>
> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
>
> I did find myself wondering if we should explicitly set
> result->attrs.secure = false;
> in an else-branch of the last "if (is_secure)", though.
> At the moment we rely on get_phys_addr_lpae() for the stage2
> doing that for us, I think. Having the code here always set
> result->attrs.secure before the 'return 0' avoids having to think
> about that...
Yes, we're currently (and predating this patch set) relying on the attrs structure being
cleared to start. But I can certainly add the assignment if you like.
r~
^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v3 01/42] target/arm: Split s2walk_secure from ipa_secure in get_phys_addr
2022-10-06 15:10 ` Richard Henderson
@ 2022-10-06 15:22 ` Peter Maydell
2022-10-06 18:20 ` Richard Henderson
0 siblings, 1 reply; 90+ messages in thread
From: Peter Maydell @ 2022-10-06 15:22 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Thu, 6 Oct 2022 at 16:10, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 10/6/22 07:27, Peter Maydell wrote:
> > On Sat, 1 Oct 2022 at 17:24, Richard Henderson
> > <richard.henderson@linaro.org> wrote:
> >>
> >> The starting security state comes with the translation regime,
> >> not the current state of arm_is_secure_below_el3().
> >>
> >> Create a new local variable, s2walk_secure, which does not need
> >> to be written back to result->attrs.secure -- we compute that
> >> value later, after the S2 walk is complete.
> >>
> >> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> >> ---
> >> v3: Do not modify ipa_secure, per review.
> >> ---
> >
> > Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
> >
> > I did find myself wondering if we should explicitly set
> > result->attrs.secure = false;
> > in an else-branch of the last "if (is_secure)", though.
> > At the moment we rely on get_phys_addr_lpae() for the stage2
> > doing that for us, I think. Having the code here always set
> > result->attrs.secure before the 'return 0' avoids having to think
> > about that...
>
> Yes, we're currently (and predating this patch set) relying on the attrs structure being
> cleared to start. But I can certainly add the assignment if you like.
Yeah, cleared-at-start is fine. But here we're also relying on
the stage 2 call to get_phys_addr_lpae() not setting it to 1,
because we pass that the same 'result' pointer, not a fresh one.
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v3 01/42] target/arm: Split s2walk_secure from ipa_secure in get_phys_addr
2022-10-06 15:22 ` Peter Maydell
@ 2022-10-06 18:20 ` Richard Henderson
2022-10-06 18:55 ` Peter Maydell
0 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-06 18:20 UTC (permalink / raw)
To: Peter Maydell; +Cc: qemu-devel, qemu-arm
On 10/6/22 08:22, Peter Maydell wrote:
> Yeah, cleared-at-start is fine. But here we're also relying on
> the stage 2 call to get_phys_addr_lpae() not setting it to 1,
> because we pass that the same 'result' pointer, not a fresh one.
I clear it first: that patch is already merged:
memset(result, 0, sizeof(*result));
ret = get_phys_addr_lpae(env, ipa, access_type, s2_mmu_idx,
is_el0, result, fi);
r~
^ permalink raw reply [flat|nested] 90+ messages in thread* Re: [PATCH v3 01/42] target/arm: Split s2walk_secure from ipa_secure in get_phys_addr
2022-10-06 18:20 ` Richard Henderson
@ 2022-10-06 18:55 ` Peter Maydell
2022-10-06 20:58 ` Richard Henderson
0 siblings, 1 reply; 90+ messages in thread
From: Peter Maydell @ 2022-10-06 18:55 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Thu, 6 Oct 2022 at 19:20, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 10/6/22 08:22, Peter Maydell wrote:
> > Yeah, cleared-at-start is fine. But here we're also relying on
> > the stage 2 call to get_phys_addr_lpae() not setting it to 1,
> > because we pass that the same 'result' pointer, not a fresh one.
>
> I clear it first: that patch is already merged:
>
> memset(result, 0, sizeof(*result));
>
>
>
> ret = get_phys_addr_lpae(env, ipa, access_type, s2_mmu_idx,
>
> is_el0, result, fi);
Yes, but that doesn't help if this ^^^ get_phys_addr_lpae()
call sets result->attrs.secure = true.
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v3 01/42] target/arm: Split s2walk_secure from ipa_secure in get_phys_addr
2022-10-06 18:55 ` Peter Maydell
@ 2022-10-06 20:58 ` Richard Henderson
2022-10-07 13:50 ` Peter Maydell
0 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-06 20:58 UTC (permalink / raw)
To: Peter Maydell; +Cc: qemu-devel, qemu-arm
On 10/6/22 11:55, Peter Maydell wrote:
> On Thu, 6 Oct 2022 at 19:20, Richard Henderson
> <richard.henderson@linaro.org> wrote:
>>
>> On 10/6/22 08:22, Peter Maydell wrote:
>>> Yeah, cleared-at-start is fine. But here we're also relying on
>>> the stage 2 call to get_phys_addr_lpae() not setting it to 1,
>>> because we pass that the same 'result' pointer, not a fresh one.
>>
>> I clear it first: that patch is already merged:
>>
>> memset(result, 0, sizeof(*result));
>> ret = get_phys_addr_lpae(env, ipa, access_type, s2_mmu_idx,
>> is_el0, result, fi);
>
> Yes, but that doesn't help if this ^^^ get_phys_addr_lpae()
> call sets result->attrs.secure = true.
Ok, sure, let's make the write to .secure be unconditional.
I've split this out into a new patch 2 for clarity.
r~
^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v3 01/42] target/arm: Split s2walk_secure from ipa_secure in get_phys_addr
2022-10-06 20:58 ` Richard Henderson
@ 2022-10-07 13:50 ` Peter Maydell
0 siblings, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 13:50 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Thu, 6 Oct 2022 at 21:58, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 10/6/22 11:55, Peter Maydell wrote:
> > On Thu, 6 Oct 2022 at 19:20, Richard Henderson
> > <richard.henderson@linaro.org> wrote:
> >>
> >> On 10/6/22 08:22, Peter Maydell wrote:
> >>> Yeah, cleared-at-start is fine. But here we're also relying on
> >>> the stage 2 call to get_phys_addr_lpae() not setting it to 1,
> >>> because we pass that the same 'result' pointer, not a fresh one.
> >>
> >> I clear it first: that patch is already merged:
> >>
> >> memset(result, 0, sizeof(*result));
> >> ret = get_phys_addr_lpae(env, ipa, access_type, s2_mmu_idx,
> >> is_el0, result, fi);
> >
> > Yes, but that doesn't help if this ^^^ get_phys_addr_lpae()
> > call sets result->attrs.secure = true.
>
> Ok, sure, let's make the write to .secure be unconditional.
> I've split this out into a new patch 2 for clarity.
If you can send that extra patch out, I can take it plus
1..20 from this series into target-arm.next, so your next revision
of this series can be smaller.
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 02/42] target/arm: Add is_secure parameter to get_phys_addr_lpae
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
2022-10-01 16:22 ` [PATCH v3 01/42] target/arm: Split s2walk_secure from ipa_secure in get_phys_addr Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 03/42] target/arm: Fix S2 disabled check in S1_ptw_translate Richard Henderson
` (40 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Alex Bennée
Remove the use of regime_is_secure from get_phys_addr_lpae,
using the new parameter instead.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
v3: Update to use s2walk_secure.
---
target/arm/ptw.c | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index b8c494ad9f..b7c999ffce 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -16,8 +16,8 @@
static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
- bool s1_is_el0, GetPhysAddrResult *result,
- ARMMMUFaultInfo *fi)
+ bool is_secure, bool s1_is_el0,
+ GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
__attribute__((nonnull));
/* This mapping is common between ID_AA64MMFR0.PARANGE and TCR_ELx.{I}PS. */
@@ -207,8 +207,8 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
GetPhysAddrResult s2 = {};
int ret;
- ret = get_phys_addr_lpae(env, addr, MMU_DATA_LOAD, s2_mmu_idx, false,
- &s2, fi);
+ ret = get_phys_addr_lpae(env, addr, MMU_DATA_LOAD, s2_mmu_idx,
+ *is_secure, false, &s2, fi);
if (ret) {
assert(fi->type != ARMFault_None);
fi->s2addr = addr;
@@ -965,8 +965,8 @@ static bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level,
*/
static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
- bool s1_is_el0, GetPhysAddrResult *result,
- ARMMMUFaultInfo *fi)
+ bool is_secure, bool s1_is_el0,
+ GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
{
ARMCPU *cpu = env_archcpu(env);
/* Read an LPAE long-descriptor translation table. */
@@ -1183,7 +1183,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
* remain non-secure. We implement this by just ORing in the NSTable/NS
* bits at each step.
*/
- tableattrs = regime_is_secure(env, mmu_idx) ? 0 : (1 << 4);
+ tableattrs = is_secure ? 0 : (1 << 4);
for (;;) {
uint64_t descriptor;
bool nstable;
@@ -2337,7 +2337,7 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
memset(result, 0, sizeof(*result));
ret = get_phys_addr_lpae(env, ipa, access_type, s2_mmu_idx,
- is_el0, result, fi);
+ s2walk_secure, is_el0, result, fi);
fi->s2addr = ipa;
/* Combine the S1 and S2 perms. */
@@ -2505,8 +2505,8 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
}
if (regime_using_lpae_format(env, mmu_idx)) {
- return get_phys_addr_lpae(env, address, access_type, mmu_idx, false,
- result, fi);
+ return get_phys_addr_lpae(env, address, access_type, mmu_idx,
+ is_secure, false, result, fi);
} else if (regime_sctlr(env, mmu_idx) & SCTLR_XP) {
return get_phys_addr_v6(env, address, access_type, mmu_idx,
is_secure, result, fi);
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 03/42] target/arm: Fix S2 disabled check in S1_ptw_translate
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
2022-10-01 16:22 ` [PATCH v3 01/42] target/arm: Split s2walk_secure from ipa_secure in get_phys_addr Richard Henderson
2022-10-01 16:22 ` [PATCH v3 02/42] target/arm: Add is_secure parameter to get_phys_addr_lpae Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-06 14:28 ` Peter Maydell
2022-10-01 16:22 ` [PATCH v3 04/42] target/arm: Add is_secure parameter to regime_translation_disabled Richard Henderson
` (39 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Pass the correct stage2 mmu_idx to regime_translation_disabled,
which we computed afterward.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
v3: Move earlier in the patch set.
---
target/arm/ptw.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index b7c999ffce..5192418c0e 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -200,10 +200,10 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
hwaddr addr, bool *is_secure,
ARMMMUFaultInfo *fi)
{
+ ARMMMUIdx s2_mmu_idx = *is_secure ? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2;
+
if (arm_mmu_idx_is_stage1_of_2(mmu_idx) &&
- !regime_translation_disabled(env, ARMMMUIdx_Stage2)) {
- ARMMMUIdx s2_mmu_idx = *is_secure ? ARMMMUIdx_Stage2_S
- : ARMMMUIdx_Stage2;
+ !regime_translation_disabled(env, s2_mmu_idx)) {
GetPhysAddrResult s2 = {};
int ret;
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 04/42] target/arm: Add is_secure parameter to regime_translation_disabled
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (2 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 03/42] target/arm: Fix S2 disabled check in S1_ptw_translate Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 05/42] target/arm: Split out get_phys_addr_with_secure Richard Henderson
` (38 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell, Alex Bennée
Remove the use of regime_is_secure from regime_translation_disabled,
using the new parameter instead.
This fixes a bug in S1_ptw_translate and get_phys_addr where we had
passed ARMMMUIdx_Stage2 and not ARMMMUIdx_Stage2_S to determine if
Stage2 is disabled, affecting FEAT_SEL2.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 20 +++++++++++---------
1 file changed, 11 insertions(+), 9 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 5192418c0e..f9b7c316d0 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -131,12 +131,13 @@ static uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn)
}
/* Return true if the specified stage of address translation is disabled */
-static bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx)
+static bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx,
+ bool is_secure)
{
uint64_t hcr_el2;
if (arm_feature(env, ARM_FEATURE_M)) {
- switch (env->v7m.mpu_ctrl[regime_is_secure(env, mmu_idx)] &
+ switch (env->v7m.mpu_ctrl[is_secure] &
(R_V7M_MPU_CTRL_ENABLE_MASK | R_V7M_MPU_CTRL_HFNMIENA_MASK)) {
case R_V7M_MPU_CTRL_ENABLE_MASK:
/* Enabled, but not for HardFault and NMI */
@@ -163,7 +164,7 @@ static bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx)
if (hcr_el2 & HCR_TGE) {
/* TGE means that NS EL0/1 act as if SCTLR_EL1.M is zero */
- if (!regime_is_secure(env, mmu_idx) && regime_el(env, mmu_idx) == 1) {
+ if (!is_secure && regime_el(env, mmu_idx) == 1) {
return true;
}
}
@@ -203,7 +204,7 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
ARMMMUIdx s2_mmu_idx = *is_secure ? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2;
if (arm_mmu_idx_is_stage1_of_2(mmu_idx) &&
- !regime_translation_disabled(env, s2_mmu_idx)) {
+ !regime_translation_disabled(env, s2_mmu_idx, *is_secure)) {
GetPhysAddrResult s2 = {};
int ret;
@@ -1357,7 +1358,7 @@ static bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
uint32_t base;
bool is_user = regime_is_user(env, mmu_idx);
- if (regime_translation_disabled(env, mmu_idx)) {
+ if (regime_translation_disabled(env, mmu_idx, is_secure)) {
/* MPU disabled. */
result->phys = address;
result->prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
@@ -1521,7 +1522,7 @@ static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
result->page_size = TARGET_PAGE_SIZE;
result->prot = 0;
- if (regime_translation_disabled(env, mmu_idx) ||
+ if (regime_translation_disabled(env, mmu_idx, secure) ||
m_is_ppb_region(env, address)) {
/*
* MPU disabled or M profile PPB access: use default memory map.
@@ -1733,7 +1734,7 @@ bool pmsav8_mpu_lookup(CPUARMState *env, uint32_t address,
* are done in arm_v7m_load_vector(), which always does a direct
* read using address_space_ldl(), rather than going via this function.
*/
- if (regime_translation_disabled(env, mmu_idx)) { /* MPU disabled */
+ if (regime_translation_disabled(env, mmu_idx, secure)) { /* MPU disabled */
hit = true;
} else if (m_is_ppb_region(env, address)) {
hit = true;
@@ -2307,7 +2308,8 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
result, fi);
/* If S1 fails or S2 is disabled, return early. */
- if (ret || regime_translation_disabled(env, ARMMMUIdx_Stage2)) {
+ if (ret || regime_translation_disabled(env, ARMMMUIdx_Stage2,
+ is_secure)) {
return ret;
}
@@ -2438,7 +2440,7 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
/* Definitely a real MMU, not an MPU */
- if (regime_translation_disabled(env, mmu_idx)) {
+ if (regime_translation_disabled(env, mmu_idx, is_secure)) {
uint64_t hcr;
uint8_t memattr;
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 05/42] target/arm: Split out get_phys_addr_with_secure
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (3 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 04/42] target/arm: Add is_secure parameter to regime_translation_disabled Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 06/42] target/arm: Add is_secure parameter to v7m_read_half_insn Richard Henderson
` (37 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell
Retain the existing get_phys_addr interface using the security
state derived from mmu_idx. Move the kerneldoc comments to the
header file where they belong.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
v3: Move the kerneldoc to internals.h
---
target/arm/internals.h | 40 ++++++++++++++++++++++++++++++++++++++
target/arm/ptw.c | 44 ++++++++++++++----------------------------
2 files changed, 55 insertions(+), 29 deletions(-)
diff --git a/target/arm/internals.h b/target/arm/internals.h
index 307a596505..3524d11dc5 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -1145,6 +1145,46 @@ typedef struct GetPhysAddrResult {
ARMCacheAttrs cacheattrs;
} GetPhysAddrResult;
+/**
+ * get_phys_addr_with_secure: get the physical address for a virtual address
+ * @env: CPUARMState
+ * @address: virtual address to get physical address for
+ * @access_type: 0 for read, 1 for write, 2 for execute
+ * @mmu_idx: MMU index indicating required translation regime
+ * @is_secure: security state for the access
+ * @result: set on translation success.
+ * @fi: set to fault info if the translation fails
+ *
+ * Find the physical address corresponding to the given virtual address,
+ * by doing a translation table walk on MMU based systems or using the
+ * MPU state on MPU based systems.
+ *
+ * Returns false if the translation was successful. Otherwise, phys_ptr, attrs,
+ * prot and page_size may not be filled in, and the populated fsr value provides
+ * information on why the translation aborted, in the format of a
+ * DFSR/IFSR fault register, with the following caveats:
+ * * we honour the short vs long DFSR format differences.
+ * * the WnR bit is never set (the caller must do this).
+ * * for PSMAv5 based systems we don't bother to return a full FSR format
+ * value.
+ */
+bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
+ MMUAccessType access_type,
+ ARMMMUIdx mmu_idx, bool is_secure,
+ GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
+ __attribute__((nonnull));
+
+/**
+ * get_phys_addr: get the physical address for a virtual address
+ * @env: CPUARMState
+ * @address: virtual address to get physical address for
+ * @access_type: 0 for read, 1 for write, 2 for execute
+ * @mmu_idx: MMU index indicating required translation regime
+ * @result: set on translation success.
+ * @fi: set to fault info if the translation fails
+ *
+ * Similarly, but use the security regime of @mmu_idx.
+ */
bool get_phys_addr(CPUARMState *env, target_ulong address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index f9b7c316d0..542111f99e 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -2260,35 +2260,12 @@ static ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
return ret;
}
-/**
- * get_phys_addr - get the physical address for this virtual address
- *
- * Find the physical address corresponding to the given virtual address,
- * by doing a translation table walk on MMU based systems or using the
- * MPU state on MPU based systems.
- *
- * Returns false if the translation was successful. Otherwise, phys_ptr, attrs,
- * prot and page_size may not be filled in, and the populated fsr value provides
- * information on why the translation aborted, in the format of a
- * DFSR/IFSR fault register, with the following caveats:
- * * we honour the short vs long DFSR format differences.
- * * the WnR bit is never set (the caller must do this).
- * * for PSMAv5 based systems we don't bother to return a full FSR format
- * value.
- *
- * @env: CPUARMState
- * @address: virtual address to get physical address for
- * @access_type: 0 for read, 1 for write, 2 for execute
- * @mmu_idx: MMU index indicating required translation regime
- * @result: set on translation success.
- * @fi: set to fault info if the translation fails
- */
-bool get_phys_addr(CPUARMState *env, target_ulong address,
- MMUAccessType access_type, ARMMMUIdx mmu_idx,
- GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
+bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
+ MMUAccessType access_type, ARMMMUIdx mmu_idx,
+ bool is_secure, GetPhysAddrResult *result,
+ ARMMMUFaultInfo *fi)
{
ARMMMUIdx s1_mmu_idx = stage_1_mmu_idx(mmu_idx);
- bool is_secure = regime_is_secure(env, mmu_idx);
if (mmu_idx != s1_mmu_idx) {
/*
@@ -2304,8 +2281,8 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
ARMMMUIdx s2_mmu_idx;
bool is_el0;
- ret = get_phys_addr(env, address, access_type, s1_mmu_idx,
- result, fi);
+ ret = get_phys_addr_with_secure(env, address, access_type,
+ s1_mmu_idx, is_secure, result, fi);
/* If S1 fails or S2 is disabled, return early. */
if (ret || regime_translation_disabled(env, ARMMMUIdx_Stage2,
@@ -2518,6 +2495,15 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
}
}
+bool get_phys_addr(CPUARMState *env, target_ulong address,
+ MMUAccessType access_type, ARMMMUIdx mmu_idx,
+ GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
+{
+ return get_phys_addr_with_secure(env, address, access_type, mmu_idx,
+ regime_is_secure(env, mmu_idx),
+ result, fi);
+}
+
hwaddr arm_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,
MemTxAttrs *attrs)
{
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 06/42] target/arm: Add is_secure parameter to v7m_read_half_insn
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (4 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 05/42] target/arm: Split out get_phys_addr_with_secure Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 07/42] target/arm: Add TBFLAG_M32.SECURE Richard Henderson
` (36 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell, Alex Bennée
Remove the use of regime_is_secure from v7m_read_half_insn, using
the new parameter instead.
As it happens, both callers pass true, propagated from the argument
to arm_v7m_mmu_idx_for_secstate which created the mmu_idx argument,
but that is a detail of v7m_handle_execute_nsc we need not expose
to the callee.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/m_helper.c | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/target/arm/m_helper.c b/target/arm/m_helper.c
index 5ee4ee15b3..203ba411f6 100644
--- a/target/arm/m_helper.c
+++ b/target/arm/m_helper.c
@@ -1981,7 +1981,7 @@ static bool do_v7m_function_return(ARMCPU *cpu)
return true;
}
-static bool v7m_read_half_insn(ARMCPU *cpu, ARMMMUIdx mmu_idx,
+static bool v7m_read_half_insn(ARMCPU *cpu, ARMMMUIdx mmu_idx, bool secure,
uint32_t addr, uint16_t *insn)
{
/*
@@ -2003,8 +2003,7 @@ static bool v7m_read_half_insn(ARMCPU *cpu, ARMMMUIdx mmu_idx,
ARMMMUFaultInfo fi = {};
MemTxResult txres;
- v8m_security_lookup(env, addr, MMU_INST_FETCH, mmu_idx,
- regime_is_secure(env, mmu_idx), &sattrs);
+ v8m_security_lookup(env, addr, MMU_INST_FETCH, mmu_idx, secure, &sattrs);
if (!sattrs.nsc || sattrs.ns) {
/*
* This must be the second half of the insn, and it straddles a
@@ -2109,7 +2108,7 @@ static bool v7m_handle_execute_nsc(ARMCPU *cpu)
/* We want to do the MPU lookup as secure; work out what mmu_idx that is */
mmu_idx = arm_v7m_mmu_idx_for_secstate(env, true);
- if (!v7m_read_half_insn(cpu, mmu_idx, env->regs[15], &insn)) {
+ if (!v7m_read_half_insn(cpu, mmu_idx, true, env->regs[15], &insn)) {
return false;
}
@@ -2125,7 +2124,7 @@ static bool v7m_handle_execute_nsc(ARMCPU *cpu)
goto gen_invep;
}
- if (!v7m_read_half_insn(cpu, mmu_idx, env->regs[15] + 2, &insn)) {
+ if (!v7m_read_half_insn(cpu, mmu_idx, true, env->regs[15] + 2, &insn)) {
return false;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 07/42] target/arm: Add TBFLAG_M32.SECURE
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (5 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 06/42] target/arm: Add is_secure parameter to v7m_read_half_insn Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 08/42] target/arm: Merge regime_is_secure into get_phys_addr Richard Henderson
` (35 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell
Remove the use of regime_is_secure from arm_tr_init_disas_context.
Instead, provide the value of v8m_secure directly from tb_flags.
Rather than use regime_is_secure, use the env->v7m.secure directly,
as per arm_mmu_idx_el.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/cpu.h | 2 ++
target/arm/helper.c | 4 ++++
target/arm/translate.c | 3 +--
3 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 33cdbc0143..790328c598 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -3197,6 +3197,8 @@ FIELD(TBFLAG_M32, NEW_FP_CTXT_NEEDED, 3, 1) /* Not cached. */
FIELD(TBFLAG_M32, FPCCR_S_WRONG, 4, 1) /* Not cached. */
/* Set if MVE insns are definitely not predicated by VPR or LTPSIZE */
FIELD(TBFLAG_M32, MVE_NO_PRED, 5, 1) /* Not cached. */
+/* Set if in secure mode */
+FIELD(TBFLAG_M32, SECURE, 6, 1)
/*
* Bit usage when in AArch64 state
diff --git a/target/arm/helper.c b/target/arm/helper.c
index b5dac651e7..772218f0d2 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10935,6 +10935,10 @@ static CPUARMTBFlags rebuild_hflags_m32(CPUARMState *env, int fp_el,
DP_TBFLAG_M32(flags, STACKCHECK, 1);
}
+ if (arm_feature(env, ARM_FEATURE_M_SECURITY) && env->v7m.secure) {
+ DP_TBFLAG_M32(flags, SECURE, 1);
+ }
+
return rebuild_hflags_common_32(env, fp_el, mmu_idx, flags);
}
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 5aaccbbf71..ac647e0262 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -9351,8 +9351,7 @@ static void arm_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs)
dc->vfp_enabled = 1;
dc->be_data = MO_TE;
dc->v7m_handler_mode = EX_TBFLAG_M32(tb_flags, HANDLER);
- dc->v8m_secure = arm_feature(env, ARM_FEATURE_M_SECURITY) &&
- regime_is_secure(env, dc->mmu_idx);
+ dc->v8m_secure = EX_TBFLAG_M32(tb_flags, SECURE);
dc->v8m_stackcheck = EX_TBFLAG_M32(tb_flags, STACKCHECK);
dc->v8m_fpccr_s_wrong = EX_TBFLAG_M32(tb_flags, FPCCR_S_WRONG);
dc->v7m_new_fp_ctxt_needed =
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 08/42] target/arm: Merge regime_is_secure into get_phys_addr
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (6 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 07/42] target/arm: Add TBFLAG_M32.SECURE Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 09/42] target/arm: Add is_secure parameter to do_ats_write Richard Henderson
` (34 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell
This is the last use of regime_is_secure; remove it
entirely before changing the layout of ARMMMUIdx.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/internals.h | 42 ----------------------------------------
target/arm/ptw.c | 44 ++++++++++++++++++++++++++++++++++++++++--
2 files changed, 42 insertions(+), 44 deletions(-)
diff --git a/target/arm/internals.h b/target/arm/internals.h
index 3524d11dc5..14428730d4 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -670,48 +670,6 @@ static inline bool regime_has_2_ranges(ARMMMUIdx mmu_idx)
}
}
-/* Return true if this address translation regime is secure */
-static inline bool regime_is_secure(CPUARMState *env, ARMMMUIdx mmu_idx)
-{
- switch (mmu_idx) {
- case ARMMMUIdx_E10_0:
- case ARMMMUIdx_E10_1:
- case ARMMMUIdx_E10_1_PAN:
- case ARMMMUIdx_E20_0:
- case ARMMMUIdx_E20_2:
- case ARMMMUIdx_E20_2_PAN:
- case ARMMMUIdx_Stage1_E0:
- case ARMMMUIdx_Stage1_E1:
- case ARMMMUIdx_Stage1_E1_PAN:
- case ARMMMUIdx_E2:
- case ARMMMUIdx_Stage2:
- case ARMMMUIdx_MPrivNegPri:
- case ARMMMUIdx_MUserNegPri:
- case ARMMMUIdx_MPriv:
- case ARMMMUIdx_MUser:
- return false;
- case ARMMMUIdx_SE3:
- case ARMMMUIdx_SE10_0:
- case ARMMMUIdx_SE10_1:
- case ARMMMUIdx_SE10_1_PAN:
- case ARMMMUIdx_SE20_0:
- case ARMMMUIdx_SE20_2:
- case ARMMMUIdx_SE20_2_PAN:
- case ARMMMUIdx_Stage1_SE0:
- case ARMMMUIdx_Stage1_SE1:
- case ARMMMUIdx_Stage1_SE1_PAN:
- case ARMMMUIdx_SE2:
- case ARMMMUIdx_Stage2_S:
- case ARMMMUIdx_MSPrivNegPri:
- case ARMMMUIdx_MSUserNegPri:
- case ARMMMUIdx_MSPriv:
- case ARMMMUIdx_MSUser:
- return true;
- default:
- g_assert_not_reached();
- }
-}
-
static inline bool regime_is_pan(CPUARMState *env, ARMMMUIdx mmu_idx)
{
switch (mmu_idx) {
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 542111f99e..9454ee9df5 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -2499,9 +2499,49 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
{
+ bool is_secure;
+
+ switch (mmu_idx) {
+ case ARMMMUIdx_E10_0:
+ case ARMMMUIdx_E10_1:
+ case ARMMMUIdx_E10_1_PAN:
+ case ARMMMUIdx_E20_0:
+ case ARMMMUIdx_E20_2:
+ case ARMMMUIdx_E20_2_PAN:
+ case ARMMMUIdx_Stage1_E0:
+ case ARMMMUIdx_Stage1_E1:
+ case ARMMMUIdx_Stage1_E1_PAN:
+ case ARMMMUIdx_E2:
+ case ARMMMUIdx_Stage2:
+ case ARMMMUIdx_MPrivNegPri:
+ case ARMMMUIdx_MUserNegPri:
+ case ARMMMUIdx_MPriv:
+ case ARMMMUIdx_MUser:
+ is_secure = false;
+ break;
+ case ARMMMUIdx_SE3:
+ case ARMMMUIdx_SE10_0:
+ case ARMMMUIdx_SE10_1:
+ case ARMMMUIdx_SE10_1_PAN:
+ case ARMMMUIdx_SE20_0:
+ case ARMMMUIdx_SE20_2:
+ case ARMMMUIdx_SE20_2_PAN:
+ case ARMMMUIdx_Stage1_SE0:
+ case ARMMMUIdx_Stage1_SE1:
+ case ARMMMUIdx_Stage1_SE1_PAN:
+ case ARMMMUIdx_SE2:
+ case ARMMMUIdx_Stage2_S:
+ case ARMMMUIdx_MSPrivNegPri:
+ case ARMMMUIdx_MSUserNegPri:
+ case ARMMMUIdx_MSPriv:
+ case ARMMMUIdx_MSUser:
+ is_secure = true;
+ break;
+ default:
+ g_assert_not_reached();
+ }
return get_phys_addr_with_secure(env, address, access_type, mmu_idx,
- regime_is_secure(env, mmu_idx),
- result, fi);
+ is_secure, result, fi);
}
hwaddr arm_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 09/42] target/arm: Add is_secure parameter to do_ats_write
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (7 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 08/42] target/arm: Merge regime_is_secure into get_phys_addr Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 10/42] target/arm: Fold secure and non-secure a-profile mmu indexes Richard Henderson
` (33 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell
Use get_phys_addr_with_secure directly. For a-profile, this is the
one place where the value of is_secure may not equal arm_is_secure(env).
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/helper.c | 19 ++++++++++++++-----
1 file changed, 14 insertions(+), 5 deletions(-)
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 772218f0d2..3adeb4cab4 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -3188,7 +3188,8 @@ static CPAccessResult ats_access(CPUARMState *env, const ARMCPRegInfo *ri,
#ifdef CONFIG_TCG
static uint64_t do_ats_write(CPUARMState *env, uint64_t value,
- MMUAccessType access_type, ARMMMUIdx mmu_idx)
+ MMUAccessType access_type, ARMMMUIdx mmu_idx,
+ bool is_secure)
{
bool ret;
uint64_t par64;
@@ -3196,7 +3197,8 @@ static uint64_t do_ats_write(CPUARMState *env, uint64_t value,
ARMMMUFaultInfo fi = {};
GetPhysAddrResult res = {};
- ret = get_phys_addr(env, value, access_type, mmu_idx, &res, &fi);
+ ret = get_phys_addr_with_secure(env, value, access_type, mmu_idx,
+ is_secure, &res, &fi);
/*
* ATS operations only do S1 or S1+S2 translations, so we never
@@ -3368,6 +3370,7 @@ static void ats_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
switch (el) {
case 3:
mmu_idx = ARMMMUIdx_SE3;
+ secure = true;
break;
case 2:
g_assert(!secure); /* ARMv8.4-SecEL2 is 64-bit only */
@@ -3389,6 +3392,7 @@ static void ats_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
switch (el) {
case 3:
mmu_idx = ARMMMUIdx_SE10_0;
+ secure = true;
break;
case 2:
g_assert(!secure); /* ARMv8.4-SecEL2 is 64-bit only */
@@ -3404,16 +3408,18 @@ static void ats_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
case 4:
/* stage 1+2 NonSecure PL1: ATS12NSOPR, ATS12NSOPW */
mmu_idx = ARMMMUIdx_E10_1;
+ secure = false;
break;
case 6:
/* stage 1+2 NonSecure PL0: ATS12NSOUR, ATS12NSOUW */
mmu_idx = ARMMMUIdx_E10_0;
+ secure = false;
break;
default:
g_assert_not_reached();
}
- par64 = do_ats_write(env, value, access_type, mmu_idx);
+ par64 = do_ats_write(env, value, access_type, mmu_idx, secure);
A32_BANKED_CURRENT_REG_SET(env, par, par64);
#else
@@ -3429,7 +3435,8 @@ static void ats1h_write(CPUARMState *env, const ARMCPRegInfo *ri,
MMUAccessType access_type = ri->opc2 & 1 ? MMU_DATA_STORE : MMU_DATA_LOAD;
uint64_t par64;
- par64 = do_ats_write(env, value, access_type, ARMMMUIdx_E2);
+ /* There is no SecureEL2 for AArch32. */
+ par64 = do_ats_write(env, value, access_type, ARMMMUIdx_E2, false);
A32_BANKED_CURRENT_REG_SET(env, par, par64);
#else
@@ -3472,6 +3479,7 @@ static void ats_write64(CPUARMState *env, const ARMCPRegInfo *ri,
break;
case 6: /* AT S1E3R, AT S1E3W */
mmu_idx = ARMMMUIdx_SE3;
+ secure = true;
break;
default:
g_assert_not_reached();
@@ -3490,7 +3498,8 @@ static void ats_write64(CPUARMState *env, const ARMCPRegInfo *ri,
g_assert_not_reached();
}
- env->cp15.par_el[1] = do_ats_write(env, value, access_type, mmu_idx);
+ env->cp15.par_el[1] = do_ats_write(env, value, access_type,
+ mmu_idx, secure);
#else
/* Handled by hardware accelerator. */
g_assert_not_reached();
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 10/42] target/arm: Fold secure and non-secure a-profile mmu indexes
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (8 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 09/42] target/arm: Add is_secure parameter to do_ats_write Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 11/42] target/arm: Reorg regime_translation_disabled Richard Henderson
` (32 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell
For a-profile aarch64, which does not bank system registers, it takes
quite a lot of code to switch between security states. In the process,
registers such as TCR_EL{1,2} must be swapped, which in itself requires
the flushing of softmmu tlbs. Therefore it doesn't buy us anything to
separate tlbs by security state.
Retain the distinction between Stage2 and Stage2_S.
This will be important as we implement FEAT_RME, and do not wish to
add a third set of mmu indexes for Realm state.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/cpu-param.h | 2 +-
target/arm/cpu.h | 72 +++++++------------
target/arm/internals.h | 31 +-------
target/arm/helper.c | 144 +++++++++++++------------------------
target/arm/ptw.c | 25 ++-----
target/arm/translate-a64.c | 8 ---
target/arm/translate.c | 6 +-
7 files changed, 85 insertions(+), 203 deletions(-)
diff --git a/target/arm/cpu-param.h b/target/arm/cpu-param.h
index 68ffb12427..08681828ac 100644
--- a/target/arm/cpu-param.h
+++ b/target/arm/cpu-param.h
@@ -32,6 +32,6 @@
# define TARGET_PAGE_BITS_MIN 10
#endif
-#define NB_MMU_MODES 15
+#define NB_MMU_MODES 8
#endif
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 790328c598..6475dc0cfd 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -2878,26 +2878,27 @@ bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync);
* table over and over.
* 6. we need separate EL1/EL2 mmu_idx for handling the Privileged Access
* Never (PAN) bit within PSTATE.
+ * 7. we fold together the secure and non-secure regimes for A-profile,
+ * because there are no banked system registers for aarch64, so the
+ * process of switching between secure and non-secure is
+ * already heavyweight.
*
* This gives us the following list of cases:
*
- * NS EL0 EL1&0 stage 1+2 (aka NS PL0)
- * NS EL1 EL1&0 stage 1+2 (aka NS PL1)
- * NS EL1 EL1&0 stage 1+2 +PAN
- * NS EL0 EL2&0
- * NS EL2 EL2&0
- * NS EL2 EL2&0 +PAN
- * NS EL2 (aka NS PL2)
- * S EL0 EL1&0 (aka S PL0)
- * S EL1 EL1&0 (not used if EL3 is 32 bit)
- * S EL1 EL1&0 +PAN
- * S EL3 (aka S PL1)
+ * EL0 EL1&0 stage 1+2 (aka NS PL0)
+ * EL1 EL1&0 stage 1+2 (aka NS PL1)
+ * EL1 EL1&0 stage 1+2 +PAN
+ * EL0 EL2&0
+ * EL2 EL2&0
+ * EL2 EL2&0 +PAN
+ * EL2 (aka NS PL2)
+ * EL3 (aka S PL1)
*
- * for a total of 11 different mmu_idx.
+ * for a total of 8 different mmu_idx.
*
* R profile CPUs have an MPU, but can use the same set of MMU indexes
- * as A profile. They only need to distinguish NS EL0 and NS EL1 (and
- * NS EL2 if we ever model a Cortex-R52).
+ * as A profile. They only need to distinguish EL0 and EL1 (and
+ * EL2 if we ever model a Cortex-R52).
*
* M profile CPUs are rather different as they do not have a true MMU.
* They have the following different MMU indexes:
@@ -2936,9 +2937,6 @@ bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync);
#define ARM_MMU_IDX_NOTLB 0x20 /* does not have a TLB */
#define ARM_MMU_IDX_M 0x40 /* M profile */
-/* Meanings of the bits for A profile mmu idx values */
-#define ARM_MMU_IDX_A_NS 0x8
-
/* Meanings of the bits for M profile mmu idx values */
#define ARM_MMU_IDX_M_PRIV 0x1
#define ARM_MMU_IDX_M_NEGPRI 0x2
@@ -2952,22 +2950,14 @@ typedef enum ARMMMUIdx {
/*
* A-profile.
*/
- ARMMMUIdx_SE10_0 = 0 | ARM_MMU_IDX_A,
- ARMMMUIdx_SE20_0 = 1 | ARM_MMU_IDX_A,
- ARMMMUIdx_SE10_1 = 2 | ARM_MMU_IDX_A,
- ARMMMUIdx_SE20_2 = 3 | ARM_MMU_IDX_A,
- ARMMMUIdx_SE10_1_PAN = 4 | ARM_MMU_IDX_A,
- ARMMMUIdx_SE20_2_PAN = 5 | ARM_MMU_IDX_A,
- ARMMMUIdx_SE2 = 6 | ARM_MMU_IDX_A,
- ARMMMUIdx_SE3 = 7 | ARM_MMU_IDX_A,
-
- ARMMMUIdx_E10_0 = ARMMMUIdx_SE10_0 | ARM_MMU_IDX_A_NS,
- ARMMMUIdx_E20_0 = ARMMMUIdx_SE20_0 | ARM_MMU_IDX_A_NS,
- ARMMMUIdx_E10_1 = ARMMMUIdx_SE10_1 | ARM_MMU_IDX_A_NS,
- ARMMMUIdx_E20_2 = ARMMMUIdx_SE20_2 | ARM_MMU_IDX_A_NS,
- ARMMMUIdx_E10_1_PAN = ARMMMUIdx_SE10_1_PAN | ARM_MMU_IDX_A_NS,
- ARMMMUIdx_E20_2_PAN = ARMMMUIdx_SE20_2_PAN | ARM_MMU_IDX_A_NS,
- ARMMMUIdx_E2 = ARMMMUIdx_SE2 | ARM_MMU_IDX_A_NS,
+ ARMMMUIdx_E10_0 = 0 | ARM_MMU_IDX_A,
+ ARMMMUIdx_E20_0 = 1 | ARM_MMU_IDX_A,
+ ARMMMUIdx_E10_1 = 2 | ARM_MMU_IDX_A,
+ ARMMMUIdx_E20_2 = 3 | ARM_MMU_IDX_A,
+ ARMMMUIdx_E10_1_PAN = 4 | ARM_MMU_IDX_A,
+ ARMMMUIdx_E20_2_PAN = 5 | ARM_MMU_IDX_A,
+ ARMMMUIdx_E2 = 6 | ARM_MMU_IDX_A,
+ ARMMMUIdx_E3 = 7 | ARM_MMU_IDX_A,
/*
* These are not allocated TLBs and are used only for AT system
@@ -2976,9 +2966,6 @@ typedef enum ARMMMUIdx {
ARMMMUIdx_Stage1_E0 = 0 | ARM_MMU_IDX_NOTLB,
ARMMMUIdx_Stage1_E1 = 1 | ARM_MMU_IDX_NOTLB,
ARMMMUIdx_Stage1_E1_PAN = 2 | ARM_MMU_IDX_NOTLB,
- ARMMMUIdx_Stage1_SE0 = 3 | ARM_MMU_IDX_NOTLB,
- ARMMMUIdx_Stage1_SE1 = 4 | ARM_MMU_IDX_NOTLB,
- ARMMMUIdx_Stage1_SE1_PAN = 5 | ARM_MMU_IDX_NOTLB,
/*
* Not allocated a TLB: used only for second stage of an S12 page
* table walk, or for descriptor loads during first stage of an S1
@@ -2986,8 +2973,8 @@ typedef enum ARMMMUIdx {
* then various TLB flush insns which currently are no-ops or flush
* only stage 1 MMU indexes will need to change to flush stage 2.
*/
- ARMMMUIdx_Stage2 = 6 | ARM_MMU_IDX_NOTLB,
- ARMMMUIdx_Stage2_S = 7 | ARM_MMU_IDX_NOTLB,
+ ARMMMUIdx_Stage2 = 3 | ARM_MMU_IDX_NOTLB,
+ ARMMMUIdx_Stage2_S = 4 | ARM_MMU_IDX_NOTLB,
/*
* M-profile.
@@ -3017,14 +3004,7 @@ typedef enum ARMMMUIdxBit {
TO_CORE_BIT(E2),
TO_CORE_BIT(E20_2),
TO_CORE_BIT(E20_2_PAN),
- TO_CORE_BIT(SE10_0),
- TO_CORE_BIT(SE20_0),
- TO_CORE_BIT(SE10_1),
- TO_CORE_BIT(SE20_2),
- TO_CORE_BIT(SE10_1_PAN),
- TO_CORE_BIT(SE20_2_PAN),
- TO_CORE_BIT(SE2),
- TO_CORE_BIT(SE3),
+ TO_CORE_BIT(E3),
TO_CORE_BIT(MUser),
TO_CORE_BIT(MPriv),
diff --git a/target/arm/internals.h b/target/arm/internals.h
index 14428730d4..b509d70851 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -649,21 +649,12 @@ static inline bool regime_has_2_ranges(ARMMMUIdx mmu_idx)
case ARMMMUIdx_Stage1_E0:
case ARMMMUIdx_Stage1_E1:
case ARMMMUIdx_Stage1_E1_PAN:
- case ARMMMUIdx_Stage1_SE0:
- case ARMMMUIdx_Stage1_SE1:
- case ARMMMUIdx_Stage1_SE1_PAN:
case ARMMMUIdx_E10_0:
case ARMMMUIdx_E10_1:
case ARMMMUIdx_E10_1_PAN:
case ARMMMUIdx_E20_0:
case ARMMMUIdx_E20_2:
case ARMMMUIdx_E20_2_PAN:
- case ARMMMUIdx_SE10_0:
- case ARMMMUIdx_SE10_1:
- case ARMMMUIdx_SE10_1_PAN:
- case ARMMMUIdx_SE20_0:
- case ARMMMUIdx_SE20_2:
- case ARMMMUIdx_SE20_2_PAN:
return true;
default:
return false;
@@ -674,11 +665,8 @@ static inline bool regime_is_pan(CPUARMState *env, ARMMMUIdx mmu_idx)
{
switch (mmu_idx) {
case ARMMMUIdx_Stage1_E1_PAN:
- case ARMMMUIdx_Stage1_SE1_PAN:
case ARMMMUIdx_E10_1_PAN:
case ARMMMUIdx_E20_2_PAN:
- case ARMMMUIdx_SE10_1_PAN:
- case ARMMMUIdx_SE20_2_PAN:
return true;
default:
return false;
@@ -689,30 +677,20 @@ static inline bool regime_is_pan(CPUARMState *env, ARMMMUIdx mmu_idx)
static inline uint32_t regime_el(CPUARMState *env, ARMMMUIdx mmu_idx)
{
switch (mmu_idx) {
- case ARMMMUIdx_SE20_0:
- case ARMMMUIdx_SE20_2:
- case ARMMMUIdx_SE20_2_PAN:
case ARMMMUIdx_E20_0:
case ARMMMUIdx_E20_2:
case ARMMMUIdx_E20_2_PAN:
case ARMMMUIdx_Stage2:
case ARMMMUIdx_Stage2_S:
- case ARMMMUIdx_SE2:
case ARMMMUIdx_E2:
return 2;
- case ARMMMUIdx_SE3:
+ case ARMMMUIdx_E3:
return 3;
- case ARMMMUIdx_SE10_0:
- case ARMMMUIdx_Stage1_SE0:
- return arm_el_is_aa64(env, 3) ? 1 : 3;
- case ARMMMUIdx_SE10_1:
- case ARMMMUIdx_SE10_1_PAN:
+ case ARMMMUIdx_E10_0:
case ARMMMUIdx_Stage1_E0:
+ return arm_el_is_aa64(env, 3) || !arm_is_secure_below_el3(env) ? 1 : 3;
case ARMMMUIdx_Stage1_E1:
case ARMMMUIdx_Stage1_E1_PAN:
- case ARMMMUIdx_Stage1_SE1:
- case ARMMMUIdx_Stage1_SE1_PAN:
- case ARMMMUIdx_E10_0:
case ARMMMUIdx_E10_1:
case ARMMMUIdx_E10_1_PAN:
case ARMMMUIdx_MPrivNegPri:
@@ -954,9 +932,6 @@ static inline bool arm_mmu_idx_is_stage1_of_2(ARMMMUIdx mmu_idx)
case ARMMMUIdx_Stage1_E0:
case ARMMMUIdx_Stage1_E1:
case ARMMMUIdx_Stage1_E1_PAN:
- case ARMMMUIdx_Stage1_SE0:
- case ARMMMUIdx_Stage1_SE1:
- case ARMMMUIdx_Stage1_SE1_PAN:
return true;
default:
return false;
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 3adeb4cab4..0fd0c73092 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -1754,6 +1754,7 @@ static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
/* Begin with base v8.0 state. */
uint32_t valid_mask = 0x3fff;
ARMCPU *cpu = env_archcpu(env);
+ uint64_t changed;
/*
* Because SCR_EL3 is the "real" cpreg and SCR is the alias, reset always
@@ -1813,7 +1814,22 @@ static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
/* Clear all-context RES0 bits. */
value &= valid_mask;
- raw_write(env, ri, value);
+ changed = env->cp15.scr_el3 ^ value;
+ env->cp15.scr_el3 = value;
+
+ /*
+ * If SCR_EL3.NS changes, i.e. arm_is_secure_below_el3, then
+ * we must invalidate all TLBs below EL3.
+ */
+ if (changed & SCR_NS) {
+ tlb_flush_by_mmuidx(env_cpu(env), (ARMMMUIdxBit_E10_0 |
+ ARMMMUIdxBit_E20_0 |
+ ARMMMUIdxBit_E10_1 |
+ ARMMMUIdxBit_E20_2 |
+ ARMMMUIdxBit_E10_1_PAN |
+ ARMMMUIdxBit_E20_2_PAN |
+ ARMMMUIdxBit_E2));
+ }
}
static void scr_reset(CPUARMState *env, const ARMCPRegInfo *ri)
@@ -2644,9 +2660,6 @@ static int gt_phys_redir_timeridx(CPUARMState *env)
case ARMMMUIdx_E20_0:
case ARMMMUIdx_E20_2:
case ARMMMUIdx_E20_2_PAN:
- case ARMMMUIdx_SE20_0:
- case ARMMMUIdx_SE20_2:
- case ARMMMUIdx_SE20_2_PAN:
return GTIMER_HYP;
default:
return GTIMER_PHYS;
@@ -2659,9 +2672,6 @@ static int gt_virt_redir_timeridx(CPUARMState *env)
case ARMMMUIdx_E20_0:
case ARMMMUIdx_E20_2:
case ARMMMUIdx_E20_2_PAN:
- case ARMMMUIdx_SE20_0:
- case ARMMMUIdx_SE20_2:
- case ARMMMUIdx_SE20_2_PAN:
return GTIMER_HYPVIRT;
default:
return GTIMER_VIRT;
@@ -3369,7 +3379,7 @@ static void ats_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
/* stage 1 current state PL1: ATS1CPR, ATS1CPW, ATS1CPRP, ATS1CPWP */
switch (el) {
case 3:
- mmu_idx = ARMMMUIdx_SE3;
+ mmu_idx = ARMMMUIdx_E3;
secure = true;
break;
case 2:
@@ -3377,10 +3387,9 @@ static void ats_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
/* fall through */
case 1:
if (ri->crm == 9 && (env->uncached_cpsr & CPSR_PAN)) {
- mmu_idx = (secure ? ARMMMUIdx_Stage1_SE1_PAN
- : ARMMMUIdx_Stage1_E1_PAN);
+ mmu_idx = ARMMMUIdx_Stage1_E1_PAN;
} else {
- mmu_idx = secure ? ARMMMUIdx_Stage1_SE1 : ARMMMUIdx_Stage1_E1;
+ mmu_idx = ARMMMUIdx_Stage1_E1;
}
break;
default:
@@ -3391,7 +3400,7 @@ static void ats_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
/* stage 1 current state PL0: ATS1CUR, ATS1CUW */
switch (el) {
case 3:
- mmu_idx = ARMMMUIdx_SE10_0;
+ mmu_idx = ARMMMUIdx_E10_0;
secure = true;
break;
case 2:
@@ -3399,7 +3408,7 @@ static void ats_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
mmu_idx = ARMMMUIdx_Stage1_E0;
break;
case 1:
- mmu_idx = secure ? ARMMMUIdx_Stage1_SE0 : ARMMMUIdx_Stage1_E0;
+ mmu_idx = ARMMMUIdx_Stage1_E0;
break;
default:
g_assert_not_reached();
@@ -3468,17 +3477,16 @@ static void ats_write64(CPUARMState *env, const ARMCPRegInfo *ri,
switch (ri->opc1) {
case 0: /* AT S1E1R, AT S1E1W, AT S1E1RP, AT S1E1WP */
if (ri->crm == 9 && (env->pstate & PSTATE_PAN)) {
- mmu_idx = (secure ? ARMMMUIdx_Stage1_SE1_PAN
- : ARMMMUIdx_Stage1_E1_PAN);
+ mmu_idx = ARMMMUIdx_Stage1_E1_PAN;
} else {
- mmu_idx = secure ? ARMMMUIdx_Stage1_SE1 : ARMMMUIdx_Stage1_E1;
+ mmu_idx = ARMMMUIdx_Stage1_E1;
}
break;
case 4: /* AT S1E2R, AT S1E2W */
- mmu_idx = secure ? ARMMMUIdx_SE2 : ARMMMUIdx_E2;
+ mmu_idx = ARMMMUIdx_E2;
break;
case 6: /* AT S1E3R, AT S1E3W */
- mmu_idx = ARMMMUIdx_SE3;
+ mmu_idx = ARMMMUIdx_E3;
secure = true;
break;
default:
@@ -3486,13 +3494,13 @@ static void ats_write64(CPUARMState *env, const ARMCPRegInfo *ri,
}
break;
case 2: /* AT S1E0R, AT S1E0W */
- mmu_idx = secure ? ARMMMUIdx_Stage1_SE0 : ARMMMUIdx_Stage1_E0;
+ mmu_idx = ARMMMUIdx_Stage1_E0;
break;
case 4: /* AT S12E1R, AT S12E1W */
- mmu_idx = secure ? ARMMMUIdx_SE10_1 : ARMMMUIdx_E10_1;
+ mmu_idx = ARMMMUIdx_E10_1;
break;
case 6: /* AT S12E0R, AT S12E0W */
- mmu_idx = secure ? ARMMMUIdx_SE10_0 : ARMMMUIdx_E10_0;
+ mmu_idx = ARMMMUIdx_E10_0;
break;
default:
g_assert_not_reached();
@@ -3762,11 +3770,6 @@ static void vmsa_tcr_ttbr_el2_write(CPUARMState *env, const ARMCPRegInfo *ri,
uint16_t mask = ARMMMUIdxBit_E20_2 |
ARMMMUIdxBit_E20_2_PAN |
ARMMMUIdxBit_E20_0;
-
- if (arm_is_secure_below_el3(env)) {
- mask >>= ARM_MMU_IDX_A_NS;
- }
-
tlb_flush_by_mmuidx(env_cpu(env), mask);
}
raw_write(env, ri, value);
@@ -3786,11 +3789,6 @@ static void vttbr_write(CPUARMState *env, const ARMCPRegInfo *ri,
uint16_t mask = ARMMMUIdxBit_E10_1 |
ARMMMUIdxBit_E10_1_PAN |
ARMMMUIdxBit_E10_0;
-
- if (arm_is_secure_below_el3(env)) {
- mask >>= ARM_MMU_IDX_A_NS;
- }
-
tlb_flush_by_mmuidx(cs, mask);
raw_write(env, ri, value);
}
@@ -4261,11 +4259,6 @@ static int vae1_tlbmask(CPUARMState *env)
ARMMMUIdxBit_E10_1_PAN |
ARMMMUIdxBit_E10_0;
}
-
- if (arm_is_secure_below_el3(env)) {
- mask >>= ARM_MMU_IDX_A_NS;
- }
-
return mask;
}
@@ -4292,10 +4285,6 @@ static int vae1_tlbbits(CPUARMState *env, uint64_t addr)
mmu_idx = ARMMMUIdx_E10_0;
}
- if (arm_is_secure_below_el3(env)) {
- mmu_idx &= ~ARM_MMU_IDX_A_NS;
- }
-
return tlbbits_for_regime(env, mmu_idx, addr);
}
@@ -4328,30 +4317,17 @@ static int alle1_tlbmask(CPUARMState *env)
* stage 2 translations, whereas most other scopes only invalidate
* stage 1 translations.
*/
- if (arm_is_secure_below_el3(env)) {
- return ARMMMUIdxBit_SE10_1 |
- ARMMMUIdxBit_SE10_1_PAN |
- ARMMMUIdxBit_SE10_0;
- } else {
- return ARMMMUIdxBit_E10_1 |
- ARMMMUIdxBit_E10_1_PAN |
- ARMMMUIdxBit_E10_0;
- }
+ return (ARMMMUIdxBit_E10_1 |
+ ARMMMUIdxBit_E10_1_PAN |
+ ARMMMUIdxBit_E10_0);
}
static int e2_tlbmask(CPUARMState *env)
{
- if (arm_is_secure_below_el3(env)) {
- return ARMMMUIdxBit_SE20_0 |
- ARMMMUIdxBit_SE20_2 |
- ARMMMUIdxBit_SE20_2_PAN |
- ARMMMUIdxBit_SE2;
- } else {
- return ARMMMUIdxBit_E20_0 |
- ARMMMUIdxBit_E20_2 |
- ARMMMUIdxBit_E20_2_PAN |
- ARMMMUIdxBit_E2;
- }
+ return (ARMMMUIdxBit_E20_0 |
+ ARMMMUIdxBit_E20_2 |
+ ARMMMUIdxBit_E20_2_PAN |
+ ARMMMUIdxBit_E2);
}
static void tlbi_aa64_alle1_write(CPUARMState *env, const ARMCPRegInfo *ri,
@@ -4378,7 +4354,7 @@ static void tlbi_aa64_alle3_write(CPUARMState *env, const ARMCPRegInfo *ri,
ARMCPU *cpu = env_archcpu(env);
CPUState *cs = CPU(cpu);
- tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_SE3);
+ tlb_flush_by_mmuidx(cs, ARMMMUIdxBit_E3);
}
static void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri,
@@ -4404,7 +4380,7 @@ static void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri,
{
CPUState *cs = env_cpu(env);
- tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_SE3);
+ tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUIdxBit_E3);
}
static void tlbi_aa64_vae2_write(CPUARMState *env, const ARMCPRegInfo *ri,
@@ -4432,7 +4408,7 @@ static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri,
CPUState *cs = CPU(cpu);
uint64_t pageaddr = sextract64(value << 12, 0, 56);
- tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_SE3);
+ tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdxBit_E3);
}
static void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri,
@@ -4471,12 +4447,10 @@ static void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri,
{
CPUState *cs = env_cpu(env);
uint64_t pageaddr = sextract64(value << 12, 0, 56);
- bool secure = arm_is_secure_below_el3(env);
- int mask = secure ? ARMMMUIdxBit_SE2 : ARMMMUIdxBit_E2;
- int bits = tlbbits_for_regime(env, secure ? ARMMMUIdx_SE2 : ARMMMUIdx_E2,
- pageaddr);
+ int bits = tlbbits_for_regime(env, ARMMMUIdx_E2, pageaddr);
- tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr, mask, bits);
+ tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr,
+ ARMMMUIdxBit_E2, bits);
}
static void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri,
@@ -4484,10 +4458,10 @@ static void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri,
{
CPUState *cs = env_cpu(env);
uint64_t pageaddr = sextract64(value << 12, 0, 56);
- int bits = tlbbits_for_regime(env, ARMMMUIdx_SE3, pageaddr);
+ int bits = tlbbits_for_regime(env, ARMMMUIdx_E3, pageaddr);
tlb_flush_page_bits_by_mmuidx_all_cpus_synced(cs, pageaddr,
- ARMMMUIdxBit_SE3, bits);
+ ARMMMUIdxBit_E3, bits);
}
#ifdef TARGET_AARCH64
@@ -4593,8 +4567,7 @@ static void tlbi_aa64_rvae1is_write(CPUARMState *env,
static int vae2_tlbmask(CPUARMState *env)
{
- return (arm_is_secure_below_el3(env)
- ? ARMMMUIdxBit_SE2 : ARMMMUIdxBit_E2);
+ return ARMMMUIdxBit_E2;
}
static void tlbi_aa64_rvae2_write(CPUARMState *env,
@@ -4640,8 +4613,7 @@ static void tlbi_aa64_rvae3_write(CPUARMState *env,
* flush-last-level-only.
*/
- do_rvae_write(env, value, ARMMMUIdxBit_SE3,
- tlb_force_broadcast(env));
+ do_rvae_write(env, value, ARMMMUIdxBit_E3, tlb_force_broadcast(env));
}
static void tlbi_aa64_rvae3is_write(CPUARMState *env,
@@ -4655,7 +4627,7 @@ static void tlbi_aa64_rvae3is_write(CPUARMState *env,
* flush-last-level-only or inner/outer specific flushes.
*/
- do_rvae_write(env, value, ARMMMUIdxBit_SE3, true);
+ do_rvae_write(env, value, ARMMMUIdxBit_E3, true);
}
#endif
@@ -10258,8 +10230,7 @@ uint64_t arm_sctlr(CPUARMState *env, int el)
/* Only EL0 needs to be adjusted for EL1&0 or EL2&0. */
if (el == 0) {
ARMMMUIdx mmu_idx = arm_mmu_idx_el(env, 0);
- el = (mmu_idx == ARMMMUIdx_E20_0 || mmu_idx == ARMMMUIdx_SE20_0)
- ? 2 : 1;
+ el = mmu_idx == ARMMMUIdx_E20_0 ? 2 : 1;
}
return env->cp15.sctlr_el[el];
}
@@ -10803,22 +10774,15 @@ int arm_mmu_idx_to_el(ARMMMUIdx mmu_idx)
switch (mmu_idx) {
case ARMMMUIdx_E10_0:
case ARMMMUIdx_E20_0:
- case ARMMMUIdx_SE10_0:
- case ARMMMUIdx_SE20_0:
return 0;
case ARMMMUIdx_E10_1:
case ARMMMUIdx_E10_1_PAN:
- case ARMMMUIdx_SE10_1:
- case ARMMMUIdx_SE10_1_PAN:
return 1;
case ARMMMUIdx_E2:
case ARMMMUIdx_E20_2:
case ARMMMUIdx_E20_2_PAN:
- case ARMMMUIdx_SE2:
- case ARMMMUIdx_SE20_2:
- case ARMMMUIdx_SE20_2_PAN:
return 2;
- case ARMMMUIdx_SE3:
+ case ARMMMUIdx_E3:
return 3;
default:
g_assert_not_reached();
@@ -10871,15 +10835,11 @@ ARMMMUIdx arm_mmu_idx_el(CPUARMState *env, int el)
}
break;
case 3:
- return ARMMMUIdx_SE3;
+ return ARMMMUIdx_E3;
default:
g_assert_not_reached();
}
- if (arm_is_secure_below_el3(env)) {
- idx &= ~ARM_MMU_IDX_A_NS;
- }
-
return idx;
}
@@ -11082,15 +11042,11 @@ static CPUARMTBFlags rebuild_hflags_a64(CPUARMState *env, int el, int fp_el,
switch (mmu_idx) {
case ARMMMUIdx_E10_1:
case ARMMMUIdx_E10_1_PAN:
- case ARMMMUIdx_SE10_1:
- case ARMMMUIdx_SE10_1_PAN:
/* TODO: ARMv8.3-NV */
DP_TBFLAG_A64(flags, UNPRIV, 1);
break;
case ARMMMUIdx_E20_2:
case ARMMMUIdx_E20_2_PAN:
- case ARMMMUIdx_SE20_2:
- case ARMMMUIdx_SE20_2_PAN:
/*
* Note that EL20_2 is gated by HCR_EL2.E2H == 1, but EL20_0 is
* gated by HCR_EL2.<E2H,TGE> == '11', and so is LDTR.
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 9454ee9df5..9be11f1673 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -65,12 +65,6 @@ unsigned int arm_pamax(ARMCPU *cpu)
ARMMMUIdx stage_1_mmu_idx(ARMMMUIdx mmu_idx)
{
switch (mmu_idx) {
- case ARMMMUIdx_SE10_0:
- return ARMMMUIdx_Stage1_SE0;
- case ARMMMUIdx_SE10_1:
- return ARMMMUIdx_Stage1_SE1;
- case ARMMMUIdx_SE10_1_PAN:
- return ARMMMUIdx_Stage1_SE1_PAN;
case ARMMMUIdx_E10_0:
return ARMMMUIdx_Stage1_E0;
case ARMMMUIdx_E10_1:
@@ -95,11 +89,8 @@ static bool regime_translation_big_endian(CPUARMState *env, ARMMMUIdx mmu_idx)
static bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx)
{
switch (mmu_idx) {
- case ARMMMUIdx_SE10_0:
case ARMMMUIdx_E20_0:
- case ARMMMUIdx_SE20_0:
case ARMMMUIdx_Stage1_E0:
- case ARMMMUIdx_Stage1_SE0:
case ARMMMUIdx_MUser:
case ARMMMUIdx_MSUser:
case ARMMMUIdx_MUserNegPri:
@@ -2304,7 +2295,7 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
s2_mmu_idx = (s2walk_secure
? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2);
- is_el0 = mmu_idx == ARMMMUIdx_E10_0 || mmu_idx == ARMMMUIdx_SE10_0;
+ is_el0 = mmu_idx == ARMMMUIdx_E10_0;
/*
* S1 is done, now do S2 translation.
@@ -2512,6 +2503,8 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
case ARMMMUIdx_Stage1_E1:
case ARMMMUIdx_Stage1_E1_PAN:
case ARMMMUIdx_E2:
+ is_secure = arm_is_secure_below_el3(env);
+ break;
case ARMMMUIdx_Stage2:
case ARMMMUIdx_MPrivNegPri:
case ARMMMUIdx_MUserNegPri:
@@ -2519,17 +2512,7 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
case ARMMMUIdx_MUser:
is_secure = false;
break;
- case ARMMMUIdx_SE3:
- case ARMMMUIdx_SE10_0:
- case ARMMMUIdx_SE10_1:
- case ARMMMUIdx_SE10_1_PAN:
- case ARMMMUIdx_SE20_0:
- case ARMMMUIdx_SE20_2:
- case ARMMMUIdx_SE20_2_PAN:
- case ARMMMUIdx_Stage1_SE0:
- case ARMMMUIdx_Stage1_SE1:
- case ARMMMUIdx_Stage1_SE1_PAN:
- case ARMMMUIdx_SE2:
+ case ARMMMUIdx_E3:
case ARMMMUIdx_Stage2_S:
case ARMMMUIdx_MSPrivNegPri:
case ARMMMUIdx_MSUserNegPri:
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 78b2d91ed4..5b67375f4e 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -111,14 +111,6 @@ static int get_a64_user_mem_index(DisasContext *s)
case ARMMMUIdx_E20_2_PAN:
useridx = ARMMMUIdx_E20_0;
break;
- case ARMMMUIdx_SE10_1:
- case ARMMMUIdx_SE10_1_PAN:
- useridx = ARMMMUIdx_SE10_0;
- break;
- case ARMMMUIdx_SE20_2:
- case ARMMMUIdx_SE20_2_PAN:
- useridx = ARMMMUIdx_SE20_0;
- break;
default:
g_assert_not_reached();
}
diff --git a/target/arm/translate.c b/target/arm/translate.c
index ac647e0262..2f72afe019 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -237,16 +237,12 @@ static inline int get_a32_user_mem_index(DisasContext *s)
* otherwise, access as if at PL0.
*/
switch (s->mmu_idx) {
+ case ARMMMUIdx_E3:
case ARMMMUIdx_E2: /* this one is UNPREDICTABLE */
case ARMMMUIdx_E10_0:
case ARMMMUIdx_E10_1:
case ARMMMUIdx_E10_1_PAN:
return arm_to_core_mmu_idx(ARMMMUIdx_E10_0);
- case ARMMMUIdx_SE3:
- case ARMMMUIdx_SE10_0:
- case ARMMMUIdx_SE10_1:
- case ARMMMUIdx_SE10_1_PAN:
- return arm_to_core_mmu_idx(ARMMMUIdx_SE10_0);
case ARMMMUIdx_MUser:
case ARMMMUIdx_MPriv:
return arm_to_core_mmu_idx(ARMMMUIdx_MUser);
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 11/42] target/arm: Reorg regime_translation_disabled
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (9 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 10/42] target/arm: Fold secure and non-secure a-profile mmu indexes Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 12/42] target/arm: Drop secure check for HCR.TGE vs SCTLR_EL1.M Richard Henderson
` (31 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell
Use a switch on mmu_idx for the a-profile indexes, instead of
three different if's vs regime_el and arm_mmu_idx_is_stage1_of_2.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 32 +++++++++++++++++++++++++-------
1 file changed, 25 insertions(+), 7 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 9be11f1673..2875ea881c 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -148,21 +148,39 @@ static bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx,
hcr_el2 = arm_hcr_el2_eff(env);
- if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
+ switch (mmu_idx) {
+ case ARMMMUIdx_Stage2:
+ case ARMMMUIdx_Stage2_S:
/* HCR.DC means HCR.VM behaves as 1 */
return (hcr_el2 & (HCR_DC | HCR_VM)) == 0;
- }
- if (hcr_el2 & HCR_TGE) {
+ case ARMMMUIdx_E10_0:
+ case ARMMMUIdx_E10_1:
+ case ARMMMUIdx_E10_1_PAN:
/* TGE means that NS EL0/1 act as if SCTLR_EL1.M is zero */
- if (!is_secure && regime_el(env, mmu_idx) == 1) {
+ if (!is_secure && (hcr_el2 & HCR_TGE)) {
return true;
}
- }
+ break;
- if ((hcr_el2 & HCR_DC) && arm_mmu_idx_is_stage1_of_2(mmu_idx)) {
+ case ARMMMUIdx_Stage1_E0:
+ case ARMMMUIdx_Stage1_E1:
+ case ARMMMUIdx_Stage1_E1_PAN:
/* HCR.DC means SCTLR_EL1.M behaves as 0 */
- return true;
+ if (hcr_el2 & HCR_DC) {
+ return true;
+ }
+ break;
+
+ case ARMMMUIdx_E20_0:
+ case ARMMMUIdx_E20_2:
+ case ARMMMUIdx_E20_2_PAN:
+ case ARMMMUIdx_E2:
+ case ARMMMUIdx_E3:
+ break;
+
+ default:
+ g_assert_not_reached();
}
return (regime_sctlr(env, mmu_idx) & SCTLR_M) == 0;
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 12/42] target/arm: Drop secure check for HCR.TGE vs SCTLR_EL1.M
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (10 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 11/42] target/arm: Reorg regime_translation_disabled Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 13/42] target/arm: Introduce arm_hcr_el2_eff_secstate Richard Henderson
` (30 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell
The effect of TGE does not only apply to non-secure state,
now that Secure EL2 exists.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 2875ea881c..1ea29bec58 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -157,8 +157,8 @@ static bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx,
case ARMMMUIdx_E10_0:
case ARMMMUIdx_E10_1:
case ARMMMUIdx_E10_1_PAN:
- /* TGE means that NS EL0/1 act as if SCTLR_EL1.M is zero */
- if (!is_secure && (hcr_el2 & HCR_TGE)) {
+ /* TGE means that EL0/1 act as if SCTLR_EL1.M is zero */
+ if (hcr_el2 & HCR_TGE) {
return true;
}
break;
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 13/42] target/arm: Introduce arm_hcr_el2_eff_secstate
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (11 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 12/42] target/arm: Drop secure check for HCR.TGE vs SCTLR_EL1.M Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 14/42] target/arm: Hoist read of *is_secure in S1_ptw_translate Richard Henderson
` (29 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell
For page walking, we may require HCR for a security state
that is not "current".
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/cpu.h | 20 +++++++++++++-------
target/arm/helper.c | 11 ++++++++---
2 files changed, 21 insertions(+), 10 deletions(-)
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 6475dc0cfd..0f82f4aa1d 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -2406,15 +2406,15 @@ static inline bool arm_is_secure(CPUARMState *env)
* Return true if the current security state has AArch64 EL2 or AArch32 Hyp.
* This corresponds to the pseudocode EL2Enabled()
*/
+static inline bool arm_is_el2_enabled_secstate(CPUARMState *env, bool secure)
+{
+ return arm_feature(env, ARM_FEATURE_EL2)
+ && (!secure || (env->cp15.scr_el3 & SCR_EEL2));
+}
+
static inline bool arm_is_el2_enabled(CPUARMState *env)
{
- if (arm_feature(env, ARM_FEATURE_EL2)) {
- if (arm_is_secure_below_el3(env)) {
- return (env->cp15.scr_el3 & SCR_EEL2) != 0;
- }
- return true;
- }
- return false;
+ return arm_is_el2_enabled_secstate(env, arm_is_secure_below_el3(env));
}
#else
@@ -2428,6 +2428,11 @@ static inline bool arm_is_secure(CPUARMState *env)
return false;
}
+static inline bool arm_is_el2_enabled_secstate(CPUARMState *env, bool secure)
+{
+ return false;
+}
+
static inline bool arm_is_el2_enabled(CPUARMState *env)
{
return false;
@@ -2440,6 +2445,7 @@ static inline bool arm_is_el2_enabled(CPUARMState *env)
* "for all purposes other than a direct read or write access of HCR_EL2."
* Not included here is HCR_RW.
*/
+uint64_t arm_hcr_el2_eff_secstate(CPUARMState *env, bool secure);
uint64_t arm_hcr_el2_eff(CPUARMState *env);
uint64_t arm_hcrx_el2_eff(CPUARMState *env);
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 0fd0c73092..4eec22b1f8 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -5216,15 +5216,15 @@ static void hcr_writelow(CPUARMState *env, const ARMCPRegInfo *ri,
}
/*
- * Return the effective value of HCR_EL2.
+ * Return the effective value of HCR_EL2, at the given security state.
* Bits that are not included here:
* RW (read from SCR_EL3.RW as needed)
*/
-uint64_t arm_hcr_el2_eff(CPUARMState *env)
+uint64_t arm_hcr_el2_eff_secstate(CPUARMState *env, bool secure)
{
uint64_t ret = env->cp15.hcr_el2;
- if (!arm_is_el2_enabled(env)) {
+ if (!arm_is_el2_enabled_secstate(env, secure)) {
/*
* "This register has no effect if EL2 is not enabled in the
* current Security state". This is ARMv8.4-SecEL2 speak for
@@ -5283,6 +5283,11 @@ uint64_t arm_hcr_el2_eff(CPUARMState *env)
return ret;
}
+uint64_t arm_hcr_el2_eff(CPUARMState *env)
+{
+ return arm_hcr_el2_eff_secstate(env, arm_is_secure_below_el3(env));
+}
+
/*
* Corresponds to ARM pseudocode function ELIsInHost().
*/
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 14/42] target/arm: Hoist read of *is_secure in S1_ptw_translate
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (12 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 13/42] target/arm: Introduce arm_hcr_el2_eff_secstate Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 15/42] target/arm: Remove env argument from combined_attrs_fwb Richard Henderson
` (28 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell
Rename the argument to is_secure_ptr, and introduce a
local variable is_secure with the value. We only write
back to the pointer toward the end of the function.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 22 ++++++++++++----------
1 file changed, 12 insertions(+), 10 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 1ea29bec58..cb072792a2 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -207,24 +207,25 @@ static bool ptw_attrs_are_device(CPUARMState *env, ARMCacheAttrs cacheattrs)
/* Translate a S1 pagetable walk through S2 if needed. */
static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
- hwaddr addr, bool *is_secure,
+ hwaddr addr, bool *is_secure_ptr,
ARMMMUFaultInfo *fi)
{
- ARMMMUIdx s2_mmu_idx = *is_secure ? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2;
+ bool is_secure = *is_secure_ptr;
+ ARMMMUIdx s2_mmu_idx = is_secure ? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2;
if (arm_mmu_idx_is_stage1_of_2(mmu_idx) &&
- !regime_translation_disabled(env, s2_mmu_idx, *is_secure)) {
+ !regime_translation_disabled(env, s2_mmu_idx, is_secure)) {
GetPhysAddrResult s2 = {};
int ret;
ret = get_phys_addr_lpae(env, addr, MMU_DATA_LOAD, s2_mmu_idx,
- *is_secure, false, &s2, fi);
+ is_secure, false, &s2, fi);
if (ret) {
assert(fi->type != ARMFault_None);
fi->s2addr = addr;
fi->stage2 = true;
fi->s1ptw = true;
- fi->s1ns = !*is_secure;
+ fi->s1ns = !is_secure;
return ~0;
}
if ((arm_hcr_el2_eff(env) & HCR_PTW) &&
@@ -237,19 +238,20 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
fi->s2addr = addr;
fi->stage2 = true;
fi->s1ptw = true;
- fi->s1ns = !*is_secure;
+ fi->s1ns = !is_secure;
return ~0;
}
if (arm_is_secure_below_el3(env)) {
/* Check if page table walk is to secure or non-secure PA space. */
- if (*is_secure) {
- *is_secure = !(env->cp15.vstcr_el2 & VSTCR_SW);
+ if (is_secure) {
+ is_secure = !(env->cp15.vstcr_el2 & VSTCR_SW);
} else {
- *is_secure = !(env->cp15.vtcr_el2 & VTCR_NSW);
+ is_secure = !(env->cp15.vtcr_el2 & VTCR_NSW);
}
+ *is_secure_ptr = is_secure;
} else {
- assert(!*is_secure);
+ assert(!is_secure);
}
addr = s2.phys;
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 15/42] target/arm: Remove env argument from combined_attrs_fwb
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (13 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 14/42] target/arm: Hoist read of *is_secure in S1_ptw_translate Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-06 14:30 ` Peter Maydell
2022-10-01 16:22 ` [PATCH v3 16/42] target/arm: Pass HCR to attribute subroutines Richard Henderson
` (27 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
This value is unused.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index cb072792a2..2f0eeee161 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -2172,8 +2172,7 @@ static uint8_t force_cacheattr_nibble_wb(uint8_t attr)
* s1 and s2 for the HCR_EL2.FWB == 1 case, returning the
* combined attributes in MAIR_EL1 format.
*/
-static uint8_t combined_attrs_fwb(CPUARMState *env,
- ARMCacheAttrs s1, ARMCacheAttrs s2)
+static uint8_t combined_attrs_fwb(ARMCacheAttrs s1, ARMCacheAttrs s2)
{
switch (s2.attrs) {
case 7:
@@ -2246,7 +2245,7 @@ static ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
/* Combine memory type and cacheability attributes */
if (arm_hcr_el2_eff(env) & HCR_FWB) {
- ret.attrs = combined_attrs_fwb(env, s1, s2);
+ ret.attrs = combined_attrs_fwb(s1, s2);
} else {
ret.attrs = combined_attrs_nofwb(env, s1, s2);
}
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 16/42] target/arm: Pass HCR to attribute subroutines.
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (14 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 15/42] target/arm: Remove env argument from combined_attrs_fwb Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 17/42] target/arm: Fix ATS12NSO* from S PL1 Richard Henderson
` (26 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell
These subroutines did not need ENV for anything except
retrieving the effective value of HCR anyway.
We have computed the effective value of HCR in the callers,
and this will be especially important for interpreting HCR
in a non-current security state.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 30 +++++++++++++++++-------------
1 file changed, 17 insertions(+), 13 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 2f0eeee161..a0dce9c313 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -186,7 +186,7 @@ static bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx,
return (regime_sctlr(env, mmu_idx) & SCTLR_M) == 0;
}
-static bool ptw_attrs_are_device(CPUARMState *env, ARMCacheAttrs cacheattrs)
+static bool ptw_attrs_are_device(uint64_t hcr, ARMCacheAttrs cacheattrs)
{
/*
* For an S1 page table walk, the stage 1 attributes are always
@@ -198,7 +198,7 @@ static bool ptw_attrs_are_device(CPUARMState *env, ARMCacheAttrs cacheattrs)
* when cacheattrs.attrs bit [2] is 0.
*/
assert(cacheattrs.is_s2_format);
- if (arm_hcr_el2_eff(env) & HCR_FWB) {
+ if (hcr & HCR_FWB) {
return (cacheattrs.attrs & 0x4) == 0;
} else {
return (cacheattrs.attrs & 0xc) == 0;
@@ -216,6 +216,7 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
if (arm_mmu_idx_is_stage1_of_2(mmu_idx) &&
!regime_translation_disabled(env, s2_mmu_idx, is_secure)) {
GetPhysAddrResult s2 = {};
+ uint64_t hcr;
int ret;
ret = get_phys_addr_lpae(env, addr, MMU_DATA_LOAD, s2_mmu_idx,
@@ -228,8 +229,9 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
fi->s1ns = !is_secure;
return ~0;
}
- if ((arm_hcr_el2_eff(env) & HCR_PTW) &&
- ptw_attrs_are_device(env, s2.cacheattrs)) {
+
+ hcr = arm_hcr_el2_eff(env);
+ if ((hcr & HCR_PTW) && ptw_attrs_are_device(hcr, s2.cacheattrs)) {
/*
* PTW set and S1 walk touched S2 Device memory:
* generate Permission fault.
@@ -2059,14 +2061,14 @@ static bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
* ref: shared/translation/attrs/S2AttrDecode()
* .../S2ConvertAttrsHints()
*/
-static uint8_t convert_stage2_attrs(CPUARMState *env, uint8_t s2attrs)
+static uint8_t convert_stage2_attrs(uint64_t hcr, uint8_t s2attrs)
{
uint8_t hiattr = extract32(s2attrs, 2, 2);
uint8_t loattr = extract32(s2attrs, 0, 2);
uint8_t hihint = 0, lohint = 0;
if (hiattr != 0) { /* normal memory */
- if (arm_hcr_el2_eff(env) & HCR_CD) { /* cache disabled */
+ if (hcr & HCR_CD) { /* cache disabled */
hiattr = loattr = 1; /* non-cacheable */
} else {
if (hiattr != 1) { /* Write-through or write-back */
@@ -2112,12 +2114,12 @@ static uint8_t combine_cacheattr_nibble(uint8_t s1, uint8_t s2)
* s1 and s2 for the HCR_EL2.FWB == 0 case, returning the
* combined attributes in MAIR_EL1 format.
*/
-static uint8_t combined_attrs_nofwb(CPUARMState *env,
+static uint8_t combined_attrs_nofwb(uint64_t hcr,
ARMCacheAttrs s1, ARMCacheAttrs s2)
{
uint8_t s1lo, s2lo, s1hi, s2hi, s2_mair_attrs, ret_attrs;
- s2_mair_attrs = convert_stage2_attrs(env, s2.attrs);
+ s2_mair_attrs = convert_stage2_attrs(hcr, s2.attrs);
s1lo = extract32(s1.attrs, 0, 4);
s2lo = extract32(s2_mair_attrs, 0, 4);
@@ -2217,7 +2219,7 @@ static uint8_t combined_attrs_fwb(ARMCacheAttrs s1, ARMCacheAttrs s2)
* @s1: Attributes from stage 1 walk
* @s2: Attributes from stage 2 walk
*/
-static ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
+static ARMCacheAttrs combine_cacheattrs(uint64_t hcr,
ARMCacheAttrs s1, ARMCacheAttrs s2)
{
ARMCacheAttrs ret;
@@ -2244,10 +2246,10 @@ static ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
}
/* Combine memory type and cacheability attributes */
- if (arm_hcr_el2_eff(env) & HCR_FWB) {
+ if (hcr & HCR_FWB) {
ret.attrs = combined_attrs_fwb(s1, s2);
} else {
- ret.attrs = combined_attrs_nofwb(env, s1, s2);
+ ret.attrs = combined_attrs_nofwb(hcr, s1, s2);
}
/*
@@ -2290,6 +2292,7 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
ARMCacheAttrs cacheattrs1;
ARMMMUIdx s2_mmu_idx;
bool is_el0;
+ uint64_t hcr;
ret = get_phys_addr_with_secure(env, address, access_type,
s1_mmu_idx, is_secure, result, fi);
@@ -2338,7 +2341,8 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
}
/* Combine the S1 and S2 cache attributes. */
- if (arm_hcr_el2_eff(env) & HCR_DC) {
+ hcr = arm_hcr_el2_eff(env);
+ if (hcr & HCR_DC) {
/*
* HCR.DC forces the first stage attributes to
* Normal Non-Shareable,
@@ -2351,7 +2355,7 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
}
cacheattrs1.shareability = 0;
}
- result->cacheattrs = combine_cacheattrs(env, cacheattrs1,
+ result->cacheattrs = combine_cacheattrs(hcr, cacheattrs1,
result->cacheattrs);
/* Check if IPA translates to secure or non-secure PA space. */
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 17/42] target/arm: Fix ATS12NSO* from S PL1
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (15 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 16/42] target/arm: Pass HCR to attribute subroutines Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 18/42] target/arm: Split out get_phys_addr_disabled Richard Henderson
` (25 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell
Use arm_hcr_el2_eff_secstate instead of arm_hcr_el2_eff, so
that we use is_secure instead of the current security state.
These AT* operations have been broken since arm_hcr_el2_eff
gained a check for "el2 enabled" for Secure EL2.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index a0dce9c313..7bf79779da 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -146,7 +146,7 @@ static bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx,
}
}
- hcr_el2 = arm_hcr_el2_eff(env);
+ hcr_el2 = arm_hcr_el2_eff_secstate(env, is_secure);
switch (mmu_idx) {
case ARMMMUIdx_Stage2:
@@ -230,7 +230,7 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
return ~0;
}
- hcr = arm_hcr_el2_eff(env);
+ hcr = arm_hcr_el2_eff_secstate(env, is_secure);
if ((hcr & HCR_PTW) && ptw_attrs_are_device(hcr, s2.cacheattrs)) {
/*
* PTW set and S1 walk touched S2 Device memory:
@@ -2341,7 +2341,7 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
}
/* Combine the S1 and S2 cache attributes. */
- hcr = arm_hcr_el2_eff(env);
+ hcr = arm_hcr_el2_eff_secstate(env, is_secure);
if (hcr & HCR_DC) {
/*
* HCR.DC forces the first stage attributes to
@@ -2474,7 +2474,7 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
result->page_size = TARGET_PAGE_SIZE;
/* Fill in cacheattr a-la AArch64.TranslateAddressS1Off. */
- hcr = arm_hcr_el2_eff(env);
+ hcr = arm_hcr_el2_eff_secstate(env, is_secure);
result->cacheattrs.shareability = 0;
result->cacheattrs.is_s2_format = false;
if (hcr & HCR_DC) {
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 18/42] target/arm: Split out get_phys_addr_disabled
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (16 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 17/42] target/arm: Fix ATS12NSO* from S PL1 Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-01 16:22 ` [PATCH v3 19/42] target/arm: Fix cacheattr in get_phys_addr_disabled Richard Henderson
` (24 subsequent siblings)
42 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Peter Maydell
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 138 +++++++++++++++++++++++++----------------------
1 file changed, 74 insertions(+), 64 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 7bf79779da..e494a9de67 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -2272,6 +2272,78 @@ static ARMCacheAttrs combine_cacheattrs(uint64_t hcr,
return ret;
}
+/*
+ * MMU disabled. S1 addresses within aa64 translation regimes are
+ * still checked for bounds -- see AArch64.S1DisabledOutput().
+ */
+static bool get_phys_addr_disabled(CPUARMState *env, target_ulong address,
+ MMUAccessType access_type,
+ ARMMMUIdx mmu_idx, bool is_secure,
+ GetPhysAddrResult *result,
+ ARMMMUFaultInfo *fi)
+{
+ uint64_t hcr;
+ uint8_t memattr;
+
+ if (mmu_idx != ARMMMUIdx_Stage2 && mmu_idx != ARMMMUIdx_Stage2_S) {
+ int r_el = regime_el(env, mmu_idx);
+ if (arm_el_is_aa64(env, r_el)) {
+ int pamax = arm_pamax(env_archcpu(env));
+ uint64_t tcr = env->cp15.tcr_el[r_el];
+ int addrtop, tbi;
+
+ tbi = aa64_va_parameter_tbi(tcr, mmu_idx);
+ if (access_type == MMU_INST_FETCH) {
+ tbi &= ~aa64_va_parameter_tbid(tcr, mmu_idx);
+ }
+ tbi = (tbi >> extract64(address, 55, 1)) & 1;
+ addrtop = (tbi ? 55 : 63);
+
+ if (extract64(address, pamax, addrtop - pamax + 1) != 0) {
+ fi->type = ARMFault_AddressSize;
+ fi->level = 0;
+ fi->stage2 = false;
+ return 1;
+ }
+
+ /*
+ * When TBI is disabled, we've just validated that all of the
+ * bits above PAMax are zero, so logically we only need to
+ * clear the top byte for TBI. But it's clearer to follow
+ * the pseudocode set of addrdesc.paddress.
+ */
+ address = extract64(address, 0, 52);
+ }
+ }
+
+ result->phys = address;
+ result->prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
+ result->page_size = TARGET_PAGE_SIZE;
+
+ /* Fill in cacheattr a-la AArch64.TranslateAddressS1Off. */
+ hcr = arm_hcr_el2_eff_secstate(env, is_secure);
+ result->cacheattrs.shareability = 0;
+ result->cacheattrs.is_s2_format = false;
+ if (hcr & HCR_DC) {
+ if (hcr & HCR_DCT) {
+ memattr = 0xf0; /* Tagged, Normal, WB, RWA */
+ } else {
+ memattr = 0xff; /* Normal, WB, RWA */
+ }
+ } else if (access_type == MMU_INST_FETCH) {
+ if (regime_sctlr(env, mmu_idx) & SCTLR_I) {
+ memattr = 0xee; /* Normal, WT, RA, NT */
+ } else {
+ memattr = 0x44; /* Normal, NC, No */
+ }
+ result->cacheattrs.shareability = 2; /* outer sharable */
+ } else {
+ memattr = 0x00; /* Device, nGnRnE */
+ }
+ result->cacheattrs.attrs = memattr;
+ return 0;
+}
+
bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
bool is_secure, GetPhysAddrResult *result,
@@ -2432,71 +2504,9 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
/* Definitely a real MMU, not an MPU */
if (regime_translation_disabled(env, mmu_idx, is_secure)) {
- uint64_t hcr;
- uint8_t memattr;
-
- /*
- * MMU disabled. S1 addresses within aa64 translation regimes are
- * still checked for bounds -- see AArch64.TranslateAddressS1Off.
- */
- if (mmu_idx != ARMMMUIdx_Stage2 && mmu_idx != ARMMMUIdx_Stage2_S) {
- int r_el = regime_el(env, mmu_idx);
- if (arm_el_is_aa64(env, r_el)) {
- int pamax = arm_pamax(env_archcpu(env));
- uint64_t tcr = env->cp15.tcr_el[r_el];
- int addrtop, tbi;
-
- tbi = aa64_va_parameter_tbi(tcr, mmu_idx);
- if (access_type == MMU_INST_FETCH) {
- tbi &= ~aa64_va_parameter_tbid(tcr, mmu_idx);
- }
- tbi = (tbi >> extract64(address, 55, 1)) & 1;
- addrtop = (tbi ? 55 : 63);
-
- if (extract64(address, pamax, addrtop - pamax + 1) != 0) {
- fi->type = ARMFault_AddressSize;
- fi->level = 0;
- fi->stage2 = false;
- return 1;
- }
-
- /*
- * When TBI is disabled, we've just validated that all of the
- * bits above PAMax are zero, so logically we only need to
- * clear the top byte for TBI. But it's clearer to follow
- * the pseudocode set of addrdesc.paddress.
- */
- address = extract64(address, 0, 52);
- }
- }
- result->phys = address;
- result->prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
- result->page_size = TARGET_PAGE_SIZE;
-
- /* Fill in cacheattr a-la AArch64.TranslateAddressS1Off. */
- hcr = arm_hcr_el2_eff_secstate(env, is_secure);
- result->cacheattrs.shareability = 0;
- result->cacheattrs.is_s2_format = false;
- if (hcr & HCR_DC) {
- if (hcr & HCR_DCT) {
- memattr = 0xf0; /* Tagged, Normal, WB, RWA */
- } else {
- memattr = 0xff; /* Normal, WB, RWA */
- }
- } else if (access_type == MMU_INST_FETCH) {
- if (regime_sctlr(env, mmu_idx) & SCTLR_I) {
- memattr = 0xee; /* Normal, WT, RA, NT */
- } else {
- memattr = 0x44; /* Normal, NC, No */
- }
- result->cacheattrs.shareability = 2; /* outer sharable */
- } else {
- memattr = 0x00; /* Device, nGnRnE */
- }
- result->cacheattrs.attrs = memattr;
- return 0;
+ return get_phys_addr_disabled(env, address, access_type, mmu_idx,
+ is_secure, result, fi);
}
-
if (regime_using_lpae_format(env, mmu_idx)) {
return get_phys_addr_lpae(env, address, access_type, mmu_idx,
is_secure, false, result, fi);
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 19/42] target/arm: Fix cacheattr in get_phys_addr_disabled
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (17 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 18/42] target/arm: Split out get_phys_addr_disabled Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-06 14:33 ` Peter Maydell
2022-10-01 16:22 ` [PATCH v3 20/42] target/arm: Use tlb_set_page_full Richard Henderson
` (23 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Do not apply memattr or shareability for Stage2 translations.
Make sure to apply HCR_{DC,DCT} only to Regime_EL10, per the
pseudocode in AArch64.S1DisabledOutput.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
v3: Do not use a switch or a goto.
---
target/arm/ptw.c | 48 +++++++++++++++++++++++++-----------------------
1 file changed, 25 insertions(+), 23 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index e494a9de67..8d27a98a42 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -2282,11 +2282,12 @@ static bool get_phys_addr_disabled(CPUARMState *env, target_ulong address,
GetPhysAddrResult *result,
ARMMMUFaultInfo *fi)
{
- uint64_t hcr;
- uint8_t memattr;
+ uint8_t memattr = 0x00; /* Device nGnRnE */
+ uint8_t shareability = 0; /* non-sharable */
if (mmu_idx != ARMMMUIdx_Stage2 && mmu_idx != ARMMMUIdx_Stage2_S) {
int r_el = regime_el(env, mmu_idx);
+
if (arm_el_is_aa64(env, r_el)) {
int pamax = arm_pamax(env_archcpu(env));
uint64_t tcr = env->cp15.tcr_el[r_el];
@@ -2314,32 +2315,33 @@ static bool get_phys_addr_disabled(CPUARMState *env, target_ulong address,
*/
address = extract64(address, 0, 52);
}
+
+ /* Fill in cacheattr a-la AArch64.TranslateAddressS1Off. */
+ if (r_el == 1) {
+ uint64_t hcr = arm_hcr_el2_eff_secstate(env, is_secure);
+ if (hcr & HCR_DC) {
+ if (hcr & HCR_DCT) {
+ memattr = 0xf0; /* Tagged, Normal, WB, RWA */
+ } else {
+ memattr = 0xff; /* Normal, WB, RWA */
+ }
+ }
+ }
+ if (memattr == 0 && access_type == MMU_INST_FETCH) {
+ if (regime_sctlr(env, mmu_idx) & SCTLR_I) {
+ memattr = 0xee; /* Normal, WT, RA, NT */
+ } else {
+ memattr = 0x44; /* Normal, NC, No */
+ }
+ shareability = 2; /* outer sharable */
+ }
+ result->cacheattrs.is_s2_format = false;
}
result->phys = address;
result->prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
result->page_size = TARGET_PAGE_SIZE;
-
- /* Fill in cacheattr a-la AArch64.TranslateAddressS1Off. */
- hcr = arm_hcr_el2_eff_secstate(env, is_secure);
- result->cacheattrs.shareability = 0;
- result->cacheattrs.is_s2_format = false;
- if (hcr & HCR_DC) {
- if (hcr & HCR_DCT) {
- memattr = 0xf0; /* Tagged, Normal, WB, RWA */
- } else {
- memattr = 0xff; /* Normal, WB, RWA */
- }
- } else if (access_type == MMU_INST_FETCH) {
- if (regime_sctlr(env, mmu_idx) & SCTLR_I) {
- memattr = 0xee; /* Normal, WT, RA, NT */
- } else {
- memattr = 0x44; /* Normal, NC, No */
- }
- result->cacheattrs.shareability = 2; /* outer sharable */
- } else {
- memattr = 0x00; /* Device, nGnRnE */
- }
+ result->cacheattrs.shareability = shareability;
result->cacheattrs.attrs = memattr;
return 0;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 20/42] target/arm: Use tlb_set_page_full
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (18 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 19/42] target/arm: Fix cacheattr in get_phys_addr_disabled Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-06 14:36 ` Peter Maydell
2022-10-01 16:22 ` [PATCH v3 21/42] target/arm: Enable TARGET_PAGE_ENTRY_EXTRA Richard Henderson
` (22 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Adjust GetPhysAddrResult to fill in CPUTLBEntryFull,
so that it may be passed directly to tlb_set_page_full.
The change is large, but mostly mechanical. The major
non-mechanical change is page_size -> lg_page_size.
Most of the time this is obvious, and is related to
TARGET_PAGE_BITS.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/internals.h | 5 +-
target/arm/helper.c | 12 +--
target/arm/m_helper.c | 20 ++---
target/arm/ptw.c | 181 ++++++++++++++++++++--------------------
target/arm/tlb_helper.c | 9 +-
5 files changed, 112 insertions(+), 115 deletions(-)
diff --git a/target/arm/internals.h b/target/arm/internals.h
index b509d70851..fd17aee459 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -1071,10 +1071,7 @@ typedef struct ARMCacheAttrs {
/* Fields that are valid upon success. */
typedef struct GetPhysAddrResult {
- hwaddr phys;
- target_ulong page_size;
- int prot;
- MemTxAttrs attrs;
+ CPUTLBEntryFull f;
ARMCacheAttrs cacheattrs;
} GetPhysAddrResult;
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 4eec22b1f8..6fe85c6642 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -3320,8 +3320,8 @@ static uint64_t do_ats_write(CPUARMState *env, uint64_t value,
/* Create a 64-bit PAR */
par64 = (1 << 11); /* LPAE bit always set */
if (!ret) {
- par64 |= res.phys & ~0xfffULL;
- if (!res.attrs.secure) {
+ par64 |= res.f.phys_addr & ~0xfffULL;
+ if (!res.f.attrs.secure) {
par64 |= (1 << 9); /* NS */
}
par64 |= (uint64_t)res.cacheattrs.attrs << 56; /* ATTR */
@@ -3345,13 +3345,13 @@ static uint64_t do_ats_write(CPUARMState *env, uint64_t value,
*/
if (!ret) {
/* We do not set any attribute bits in the PAR */
- if (res.page_size == (1 << 24)
+ if (res.f.lg_page_size == 24
&& arm_feature(env, ARM_FEATURE_V7)) {
- par64 = (res.phys & 0xff000000) | (1 << 1);
+ par64 = (res.f.phys_addr & 0xff000000) | (1 << 1);
} else {
- par64 = res.phys & 0xfffff000;
+ par64 = res.f.phys_addr & 0xfffff000;
}
- if (!res.attrs.secure) {
+ if (!res.f.attrs.secure) {
par64 |= (1 << 9); /* NS */
}
} else {
diff --git a/target/arm/m_helper.c b/target/arm/m_helper.c
index 203ba411f6..355cd4d60a 100644
--- a/target/arm/m_helper.c
+++ b/target/arm/m_helper.c
@@ -223,8 +223,8 @@ static bool v7m_stack_write(ARMCPU *cpu, uint32_t addr, uint32_t value,
}
goto pend_fault;
}
- address_space_stl_le(arm_addressspace(cs, res.attrs), res.phys, value,
- res.attrs, &txres);
+ address_space_stl_le(arm_addressspace(cs, res.f.attrs), res.f.phys_addr,
+ value, res.f.attrs, &txres);
if (txres != MEMTX_OK) {
/* BusFault trying to write the data */
if (mode == STACK_LAZYFP) {
@@ -298,8 +298,8 @@ static bool v7m_stack_read(ARMCPU *cpu, uint32_t *dest, uint32_t addr,
goto pend_fault;
}
- value = address_space_ldl(arm_addressspace(cs, res.attrs), res.phys,
- res.attrs, &txres);
+ value = address_space_ldl(arm_addressspace(cs, res.f.attrs),
+ res.f.phys_addr, res.f.attrs, &txres);
if (txres != MEMTX_OK) {
/* BusFault trying to read the data */
qemu_log_mask(CPU_LOG_INT, "...BusFault with BFSR.UNSTKERR\n");
@@ -2022,8 +2022,8 @@ static bool v7m_read_half_insn(ARMCPU *cpu, ARMMMUIdx mmu_idx, bool secure,
qemu_log_mask(CPU_LOG_INT, "...really MemManage with CFSR.IACCVIOL\n");
return false;
}
- *insn = address_space_lduw_le(arm_addressspace(cs, res.attrs), res.phys,
- res.attrs, &txres);
+ *insn = address_space_lduw_le(arm_addressspace(cs, res.f.attrs),
+ res.f.phys_addr, res.f.attrs, &txres);
if (txres != MEMTX_OK) {
env->v7m.cfsr[M_REG_NS] |= R_V7M_CFSR_IBUSERR_MASK;
armv7m_nvic_set_pending(env->nvic, ARMV7M_EXCP_BUS, false);
@@ -2069,8 +2069,8 @@ static bool v7m_read_sg_stack_word(ARMCPU *cpu, ARMMMUIdx mmu_idx,
}
return false;
}
- value = address_space_ldl(arm_addressspace(cs, res.attrs), res.phys,
- res.attrs, &txres);
+ value = address_space_ldl(arm_addressspace(cs, res.f.attrs),
+ res.f.phys_addr, res.f.attrs, &txres);
if (txres != MEMTX_OK) {
/* BusFault trying to read the data */
qemu_log_mask(CPU_LOG_INT,
@@ -2817,8 +2817,8 @@ uint32_t HELPER(v7m_tt)(CPUARMState *env, uint32_t addr, uint32_t op)
} else {
mrvalid = true;
}
- r = res.prot & PAGE_READ;
- rw = res.prot & PAGE_WRITE;
+ r = res.f.prot & PAGE_READ;
+ rw = res.f.prot & PAGE_WRITE;
} else {
r = false;
rw = false;
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 8d27a98a42..1bc194ffa1 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -256,7 +256,7 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
assert(!is_secure);
}
- addr = s2.phys;
+ addr = s2.f.phys_addr;
}
return addr;
}
@@ -476,7 +476,7 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
/* 1Mb section. */
phys_addr = (desc & 0xfff00000) | (address & 0x000fffff);
ap = (desc >> 10) & 3;
- result->page_size = 1024 * 1024;
+ result->f.lg_page_size = 20; /* 1MB */
} else {
/* Lookup l2 entry. */
if (type == 1) {
@@ -497,12 +497,12 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
case 1: /* 64k page. */
phys_addr = (desc & 0xffff0000) | (address & 0xffff);
ap = (desc >> (4 + ((address >> 13) & 6))) & 3;
- result->page_size = 0x10000;
+ result->f.lg_page_size = 16;
break;
case 2: /* 4k page. */
phys_addr = (desc & 0xfffff000) | (address & 0xfff);
ap = (desc >> (4 + ((address >> 9) & 6))) & 3;
- result->page_size = 0x1000;
+ result->f.lg_page_size = 12;
break;
case 3: /* 1k page, or ARMv6/XScale "extended small (4k) page" */
if (type == 1) {
@@ -510,7 +510,7 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
if (arm_feature(env, ARM_FEATURE_XSCALE)
|| arm_feature(env, ARM_FEATURE_V6)) {
phys_addr = (desc & 0xfffff000) | (address & 0xfff);
- result->page_size = 0x1000;
+ result->f.lg_page_size = 12;
} else {
/*
* UNPREDICTABLE in ARMv5; we choose to take a
@@ -521,7 +521,7 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
}
} else {
phys_addr = (desc & 0xfffffc00) | (address & 0x3ff);
- result->page_size = 0x400;
+ result->f.lg_page_size = 10;
}
ap = (desc >> 4) & 3;
break;
@@ -530,14 +530,14 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
g_assert_not_reached();
}
}
- result->prot = ap_to_rw_prot(env, mmu_idx, ap, domain_prot);
- result->prot |= result->prot ? PAGE_EXEC : 0;
- if (!(result->prot & (1 << access_type))) {
+ result->f.prot = ap_to_rw_prot(env, mmu_idx, ap, domain_prot);
+ result->f.prot |= result->f.prot ? PAGE_EXEC : 0;
+ if (!(result->f.prot & (1 << access_type))) {
/* Access permission fault. */
fi->type = ARMFault_Permission;
goto do_fault;
}
- result->phys = phys_addr;
+ result->f.phys_addr = phys_addr;
return false;
do_fault:
fi->domain = domain;
@@ -607,11 +607,11 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
phys_addr = (desc & 0xff000000) | (address & 0x00ffffff);
phys_addr |= (uint64_t)extract32(desc, 20, 4) << 32;
phys_addr |= (uint64_t)extract32(desc, 5, 4) << 36;
- result->page_size = 0x1000000;
+ result->f.lg_page_size = 24; /* 16MB */
} else {
/* Section. */
phys_addr = (desc & 0xfff00000) | (address & 0x000fffff);
- result->page_size = 0x100000;
+ result->f.lg_page_size = 20; /* 1MB */
}
ap = ((desc >> 10) & 3) | ((desc >> 13) & 4);
xn = desc & (1 << 4);
@@ -636,12 +636,12 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
case 1: /* 64k page. */
phys_addr = (desc & 0xffff0000) | (address & 0xffff);
xn = desc & (1 << 15);
- result->page_size = 0x10000;
+ result->f.lg_page_size = 16;
break;
case 2: case 3: /* 4k page. */
phys_addr = (desc & 0xfffff000) | (address & 0xfff);
xn = desc & 1;
- result->page_size = 0x1000;
+ result->f.lg_page_size = 12;
break;
default:
/* Never happens, but compiler isn't smart enough to tell. */
@@ -649,7 +649,7 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
}
}
if (domain_prot == 3) {
- result->prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
+ result->f.prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
} else {
if (pxn && !regime_is_user(env, mmu_idx)) {
xn = 1;
@@ -667,14 +667,14 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
fi->type = ARMFault_AccessFlag;
goto do_fault;
}
- result->prot = simple_ap_to_rw_prot(env, mmu_idx, ap >> 1);
+ result->f.prot = simple_ap_to_rw_prot(env, mmu_idx, ap >> 1);
} else {
- result->prot = ap_to_rw_prot(env, mmu_idx, ap, domain_prot);
+ result->f.prot = ap_to_rw_prot(env, mmu_idx, ap, domain_prot);
}
- if (result->prot && !xn) {
- result->prot |= PAGE_EXEC;
+ if (result->f.prot && !xn) {
+ result->f.prot |= PAGE_EXEC;
}
- if (!(result->prot & (1 << access_type))) {
+ if (!(result->f.prot & (1 << access_type))) {
/* Access permission fault. */
fi->type = ARMFault_Permission;
goto do_fault;
@@ -685,9 +685,9 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
* the CPU doesn't support TZ or this is a non-secure translation
* regime, because the attribute will already be non-secure.
*/
- result->attrs.secure = false;
+ result->f.attrs.secure = false;
}
- result->phys = phys_addr;
+ result->f.phys_addr = phys_addr;
return false;
do_fault:
fi->domain = domain;
@@ -1298,16 +1298,16 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
ns = mmu_idx == ARMMMUIdx_Stage2;
xn = extract32(attrs, 11, 2);
- result->prot = get_S2prot(env, ap, xn, s1_is_el0);
+ result->f.prot = get_S2prot(env, ap, xn, s1_is_el0);
} else {
ns = extract32(attrs, 3, 1);
xn = extract32(attrs, 12, 1);
pxn = extract32(attrs, 11, 1);
- result->prot = get_S1prot(env, mmu_idx, aarch64, ap, ns, xn, pxn);
+ result->f.prot = get_S1prot(env, mmu_idx, aarch64, ap, ns, xn, pxn);
}
fault_type = ARMFault_Permission;
- if (!(result->prot & (1 << access_type))) {
+ if (!(result->f.prot & (1 << access_type))) {
goto do_fault;
}
@@ -1317,11 +1317,11 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
* the CPU doesn't support TZ or this is a non-secure translation
* regime, because the attribute will already be non-secure.
*/
- result->attrs.secure = false;
+ result->f.attrs.secure = false;
}
/* When in aarch64 mode, and BTI is enabled, remember GP in the IOTLB. */
if (aarch64 && guarded && cpu_isar_feature(aa64_bti, cpu)) {
- arm_tlb_bti_gp(&result->attrs) = true;
+ arm_tlb_bti_gp(&result->f.attrs) = true;
}
if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
@@ -1347,8 +1347,8 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
result->cacheattrs.shareability = extract32(attrs, 6, 2);
}
- result->phys = descaddr;
- result->page_size = page_size;
+ result->f.phys_addr = descaddr;
+ result->f.lg_page_size = ctz64(page_size);
return false;
do_fault:
@@ -1373,12 +1373,12 @@ static bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
if (regime_translation_disabled(env, mmu_idx, is_secure)) {
/* MPU disabled. */
- result->phys = address;
- result->prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
+ result->f.phys_addr = address;
+ result->f.prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
return false;
}
- result->phys = address;
+ result->f.phys_addr = address;
for (n = 7; n >= 0; n--) {
base = env->cp15.c6_region[n];
if ((base & 1) == 0) {
@@ -1414,16 +1414,16 @@ static bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
fi->level = 1;
return true;
}
- result->prot = PAGE_READ | PAGE_WRITE;
+ result->f.prot = PAGE_READ | PAGE_WRITE;
break;
case 2:
- result->prot = PAGE_READ;
+ result->f.prot = PAGE_READ;
if (!is_user) {
- result->prot |= PAGE_WRITE;
+ result->f.prot |= PAGE_WRITE;
}
break;
case 3:
- result->prot = PAGE_READ | PAGE_WRITE;
+ result->f.prot = PAGE_READ | PAGE_WRITE;
break;
case 5:
if (is_user) {
@@ -1431,10 +1431,10 @@ static bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
fi->level = 1;
return true;
}
- result->prot = PAGE_READ;
+ result->f.prot = PAGE_READ;
break;
case 6:
- result->prot = PAGE_READ;
+ result->f.prot = PAGE_READ;
break;
default:
/* Bad permission. */
@@ -1442,12 +1442,12 @@ static bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
fi->level = 1;
return true;
}
- result->prot |= PAGE_EXEC;
+ result->f.prot |= PAGE_EXEC;
return false;
}
static void get_phys_addr_pmsav7_default(CPUARMState *env, ARMMMUIdx mmu_idx,
- int32_t address, int *prot)
+ int32_t address, uint8_t *prot)
{
if (!arm_feature(env, ARM_FEATURE_M)) {
*prot = PAGE_READ | PAGE_WRITE;
@@ -1531,9 +1531,9 @@ static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
int n;
bool is_user = regime_is_user(env, mmu_idx);
- result->phys = address;
- result->page_size = TARGET_PAGE_SIZE;
- result->prot = 0;
+ result->f.phys_addr = address;
+ result->f.lg_page_size = TARGET_PAGE_BITS;
+ result->f.prot = 0;
if (regime_translation_disabled(env, mmu_idx, secure) ||
m_is_ppb_region(env, address)) {
@@ -1545,7 +1545,7 @@ static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
* which always does a direct read using address_space_ldl(), rather
* than going via this function, so we don't need to check that here.
*/
- get_phys_addr_pmsav7_default(env, mmu_idx, address, &result->prot);
+ get_phys_addr_pmsav7_default(env, mmu_idx, address, &result->f.prot);
} else { /* MPU enabled */
for (n = (int)cpu->pmsav7_dregion - 1; n >= 0; n--) {
/* region search */
@@ -1587,7 +1587,7 @@ static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
if (ranges_overlap(base, rmask,
address & TARGET_PAGE_MASK,
TARGET_PAGE_SIZE)) {
- result->page_size = 1;
+ result->f.lg_page_size = 0;
}
continue;
}
@@ -1625,7 +1625,7 @@ static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
continue;
}
if (rsize < TARGET_PAGE_BITS) {
- result->page_size = 1 << rsize;
+ result->f.lg_page_size = rsize;
}
break;
}
@@ -1636,7 +1636,8 @@ static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
fi->type = ARMFault_Background;
return true;
}
- get_phys_addr_pmsav7_default(env, mmu_idx, address, &result->prot);
+ get_phys_addr_pmsav7_default(env, mmu_idx, address,
+ &result->f.prot);
} else { /* a MPU hit! */
uint32_t ap = extract32(env->pmsav7.dracr[n], 8, 3);
uint32_t xn = extract32(env->pmsav7.dracr[n], 12, 1);
@@ -1653,16 +1654,16 @@ static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
case 5:
break; /* no access */
case 3:
- result->prot |= PAGE_WRITE;
+ result->f.prot |= PAGE_WRITE;
/* fall through */
case 2:
case 6:
- result->prot |= PAGE_READ | PAGE_EXEC;
+ result->f.prot |= PAGE_READ | PAGE_EXEC;
break;
case 7:
/* for v7M, same as 6; for R profile a reserved value */
if (arm_feature(env, ARM_FEATURE_M)) {
- result->prot |= PAGE_READ | PAGE_EXEC;
+ result->f.prot |= PAGE_READ | PAGE_EXEC;
break;
}
/* fall through */
@@ -1678,16 +1679,16 @@ static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
case 1:
case 2:
case 3:
- result->prot |= PAGE_WRITE;
+ result->f.prot |= PAGE_WRITE;
/* fall through */
case 5:
case 6:
- result->prot |= PAGE_READ | PAGE_EXEC;
+ result->f.prot |= PAGE_READ | PAGE_EXEC;
break;
case 7:
/* for v7M, same as 6; for R profile a reserved value */
if (arm_feature(env, ARM_FEATURE_M)) {
- result->prot |= PAGE_READ | PAGE_EXEC;
+ result->f.prot |= PAGE_READ | PAGE_EXEC;
break;
}
/* fall through */
@@ -1700,14 +1701,14 @@ static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
/* execute never */
if (xn) {
- result->prot &= ~PAGE_EXEC;
+ result->f.prot &= ~PAGE_EXEC;
}
}
}
fi->type = ARMFault_Permission;
fi->level = 1;
- return !(result->prot & (1 << access_type));
+ return !(result->f.prot & (1 << access_type));
}
bool pmsav8_mpu_lookup(CPUARMState *env, uint32_t address,
@@ -1733,9 +1734,9 @@ bool pmsav8_mpu_lookup(CPUARMState *env, uint32_t address,
uint32_t addr_page_base = address & TARGET_PAGE_MASK;
uint32_t addr_page_limit = addr_page_base + (TARGET_PAGE_SIZE - 1);
- result->page_size = TARGET_PAGE_SIZE;
- result->phys = address;
- result->prot = 0;
+ result->f.lg_page_size = TARGET_PAGE_BITS;
+ result->f.phys_addr = address;
+ result->f.prot = 0;
if (mregion) {
*mregion = -1;
}
@@ -1785,13 +1786,13 @@ bool pmsav8_mpu_lookup(CPUARMState *env, uint32_t address,
ranges_overlap(base, limit - base + 1,
addr_page_base,
TARGET_PAGE_SIZE)) {
- result->page_size = 1;
+ result->f.lg_page_size = 0;
}
continue;
}
if (base > addr_page_base || limit < addr_page_limit) {
- result->page_size = 1;
+ result->f.lg_page_size = 0;
}
if (matchregion != -1) {
@@ -1817,7 +1818,7 @@ bool pmsav8_mpu_lookup(CPUARMState *env, uint32_t address,
if (matchregion == -1) {
/* hit using the background region */
- get_phys_addr_pmsav7_default(env, mmu_idx, address, &result->prot);
+ get_phys_addr_pmsav7_default(env, mmu_idx, address, &result->f.prot);
} else {
uint32_t ap = extract32(env->pmsav8.rbar[secure][matchregion], 1, 2);
uint32_t xn = extract32(env->pmsav8.rbar[secure][matchregion], 0, 1);
@@ -1832,9 +1833,9 @@ bool pmsav8_mpu_lookup(CPUARMState *env, uint32_t address,
xn = 1;
}
- result->prot = simple_ap_to_rw_prot(env, mmu_idx, ap);
- if (result->prot && !xn && !(pxn && !is_user)) {
- result->prot |= PAGE_EXEC;
+ result->f.prot = simple_ap_to_rw_prot(env, mmu_idx, ap);
+ if (result->f.prot && !xn && !(pxn && !is_user)) {
+ result->f.prot |= PAGE_EXEC;
}
/*
* We don't need to look the attribute up in the MAIR0/MAIR1
@@ -1847,7 +1848,7 @@ bool pmsav8_mpu_lookup(CPUARMState *env, uint32_t address,
fi->type = ARMFault_Permission;
fi->level = 1;
- return !(result->prot & (1 << access_type));
+ return !(result->f.prot & (1 << access_type));
}
static bool v8m_is_sau_exempt(CPUARMState *env,
@@ -2011,9 +2012,9 @@ static bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
} else {
fi->type = ARMFault_QEMU_SFault;
}
- result->page_size = sattrs.subpage ? 1 : TARGET_PAGE_SIZE;
- result->phys = address;
- result->prot = 0;
+ result->f.lg_page_size = sattrs.subpage ? 0 : TARGET_PAGE_BITS;
+ result->f.phys_addr = address;
+ result->f.prot = 0;
return true;
}
} else {
@@ -2023,7 +2024,7 @@ static bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
* might downgrade a secure access to nonsecure.
*/
if (sattrs.ns) {
- result->attrs.secure = false;
+ result->f.attrs.secure = false;
} else if (!secure) {
/*
* NS access to S memory must fault.
@@ -2036,9 +2037,9 @@ static bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
* for M_FAKE_FSR_SFAULT in arm_v7m_cpu_do_interrupt().
*/
fi->type = ARMFault_QEMU_SFault;
- result->page_size = sattrs.subpage ? 1 : TARGET_PAGE_SIZE;
- result->phys = address;
- result->prot = 0;
+ result->f.lg_page_size = sattrs.subpage ? 0 : TARGET_PAGE_BITS;
+ result->f.phys_addr = address;
+ result->f.prot = 0;
return true;
}
}
@@ -2047,7 +2048,7 @@ static bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
ret = pmsav8_mpu_lookup(env, address, access_type, mmu_idx, secure,
result, fi, NULL);
if (sattrs.subpage) {
- result->page_size = 1;
+ result->f.lg_page_size = 0;
}
return ret;
}
@@ -2338,9 +2339,9 @@ static bool get_phys_addr_disabled(CPUARMState *env, target_ulong address,
result->cacheattrs.is_s2_format = false;
}
- result->phys = address;
- result->prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
- result->page_size = TARGET_PAGE_SIZE;
+ result->f.phys_addr = address;
+ result->f.prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
+ result->f.lg_page_size = TARGET_PAGE_BITS;
result->cacheattrs.shareability = shareability;
result->cacheattrs.attrs = memattr;
return 0;
@@ -2377,8 +2378,8 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
return ret;
}
- ipa = result->phys;
- ipa_secure = result->attrs.secure;
+ ipa = result->f.phys_addr;
+ ipa_secure = result->f.attrs.secure;
if (is_secure) {
/* Select TCR based on the NS bit from the S1 walk. */
s2walk_secure = !(ipa_secure
@@ -2398,7 +2399,7 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
* Save the stage1 results so that we may merge
* prot and cacheattrs later.
*/
- s1_prot = result->prot;
+ s1_prot = result->f.prot;
cacheattrs1 = result->cacheattrs;
memset(result, 0, sizeof(*result));
@@ -2407,7 +2408,7 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
fi->s2addr = ipa;
/* Combine the S1 and S2 perms. */
- result->prot &= s1_prot;
+ result->f.prot &= s1_prot;
/* If S2 fails, return early. */
if (ret) {
@@ -2435,10 +2436,10 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
/* Check if IPA translates to secure or non-secure PA space. */
if (is_secure) {
if (ipa_secure) {
- result->attrs.secure =
+ result->f.attrs.secure =
!(env->cp15.vstcr_el2 & (VSTCR_SA | VSTCR_SW));
} else {
- result->attrs.secure =
+ result->f.attrs.secure =
!((env->cp15.vtcr_el2 & (VTCR_NSA | VTCR_NSW))
|| (env->cp15.vstcr_el2 & (VSTCR_SA | VSTCR_SW)));
}
@@ -2457,8 +2458,8 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
* cannot upgrade an non-secure translation regime's attributes
* to secure.
*/
- result->attrs.secure = is_secure;
- result->attrs.user = regime_is_user(env, mmu_idx);
+ result->f.attrs.secure = is_secure;
+ result->f.attrs.user = regime_is_user(env, mmu_idx);
/*
* Fast Context Switch Extension. This doesn't exist at all in v8.
@@ -2475,7 +2476,7 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
if (arm_feature(env, ARM_FEATURE_PMSA)) {
bool ret;
- result->page_size = TARGET_PAGE_SIZE;
+ result->f.lg_page_size = TARGET_PAGE_BITS;
if (arm_feature(env, ARM_FEATURE_V8)) {
/* PMSAv8 */
@@ -2496,9 +2497,9 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
(access_type == MMU_DATA_STORE ? "writing" : "execute"),
(uint32_t)address, mmu_idx,
ret ? "Miss" : "Hit",
- result->prot & PAGE_READ ? 'r' : '-',
- result->prot & PAGE_WRITE ? 'w' : '-',
- result->prot & PAGE_EXEC ? 'x' : '-');
+ result->f.prot & PAGE_READ ? 'r' : '-',
+ result->f.prot & PAGE_WRITE ? 'w' : '-',
+ result->f.prot & PAGE_EXEC ? 'x' : '-');
return ret;
}
@@ -2573,10 +2574,10 @@ hwaddr arm_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,
bool ret;
ret = get_phys_addr(env, addr, MMU_DATA_LOAD, mmu_idx, &res, &fi);
- *attrs = res.attrs;
+ *attrs = res.f.attrs;
if (ret) {
return -1;
}
- return res.phys;
+ return res.f.phys_addr;
}
diff --git a/target/arm/tlb_helper.c b/target/arm/tlb_helper.c
index ad225b1cb2..49601394ec 100644
--- a/target/arm/tlb_helper.c
+++ b/target/arm/tlb_helper.c
@@ -227,17 +227,16 @@ bool arm_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
* target page size are handled specially, so for those we
* pass in the exact addresses.
*/
- if (res.page_size >= TARGET_PAGE_SIZE) {
- res.phys &= TARGET_PAGE_MASK;
+ if (res.f.lg_page_size >= TARGET_PAGE_BITS) {
+ res.f.phys_addr &= TARGET_PAGE_MASK;
address &= TARGET_PAGE_MASK;
}
/* Notice and record tagged memory. */
if (cpu_isar_feature(aa64_mte, cpu) && res.cacheattrs.attrs == 0xf0) {
- arm_tlb_mte_tagged(&res.attrs) = true;
+ arm_tlb_mte_tagged(&res.f.attrs) = true;
}
- tlb_set_page_with_attrs(cs, address, res.phys, res.attrs,
- res.prot, mmu_idx, res.page_size);
+ tlb_set_page_full(cs, mmu_idx, address, &res.f);
return true;
} else if (probe) {
return false;
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 20/42] target/arm: Use tlb_set_page_full
2022-10-01 16:22 ` [PATCH v3 20/42] target/arm: Use tlb_set_page_full Richard Henderson
@ 2022-10-06 14:36 ` Peter Maydell
0 siblings, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-06 14:36 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:44, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Adjust GetPhysAddrResult to fill in CPUTLBEntryFull,
> so that it may be passed directly to tlb_set_page_full.
>
> The change is large, but mostly mechanical. The major
> non-mechanical change is page_size -> lg_page_size.
> Most of the time this is obvious, and is related to
> TARGET_PAGE_BITS.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 21/42] target/arm: Enable TARGET_PAGE_ENTRY_EXTRA
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (19 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 20/42] target/arm: Use tlb_set_page_full Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-06 14:44 ` Peter Maydell
2022-10-01 16:22 ` [PATCH v3 22/42] target/arm: Use probe_access_full for MTE Richard Henderson
` (21 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Copy attrs and sharability, into the TLB. This will eventually
be used by S1_ptw_translate to report stage1 translation failures,
and by do_ats_write to fill in PAR_EL1.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/cpu-param.h | 8 ++++++++
target/arm/tlb_helper.c | 3 +++
2 files changed, 11 insertions(+)
diff --git a/target/arm/cpu-param.h b/target/arm/cpu-param.h
index 08681828ac..118ca0e5c0 100644
--- a/target/arm/cpu-param.h
+++ b/target/arm/cpu-param.h
@@ -30,6 +30,14 @@
*/
# define TARGET_PAGE_BITS_VARY
# define TARGET_PAGE_BITS_MIN 10
+
+/*
+ * Cache the attrs and sharability fields from the page table entry.
+ */
+# define TARGET_PAGE_ENTRY_EXTRA \
+ uint8_t pte_attrs; \
+ uint8_t shareability;
+
#endif
#define NB_MMU_MODES 8
diff --git a/target/arm/tlb_helper.c b/target/arm/tlb_helper.c
index 49601394ec..353edbeb1d 100644
--- a/target/arm/tlb_helper.c
+++ b/target/arm/tlb_helper.c
@@ -236,6 +236,9 @@ bool arm_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
arm_tlb_mte_tagged(&res.f.attrs) = true;
}
+ res.f.pte_attrs = res.cacheattrs.attrs;
+ res.f.shareability = res.cacheattrs.shareability;
+
tlb_set_page_full(cs, mmu_idx, address, &res.f);
return true;
} else if (probe) {
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 21/42] target/arm: Enable TARGET_PAGE_ENTRY_EXTRA
2022-10-01 16:22 ` [PATCH v3 21/42] target/arm: Enable TARGET_PAGE_ENTRY_EXTRA Richard Henderson
@ 2022-10-06 14:44 ` Peter Maydell
0 siblings, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-06 14:44 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:54, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Copy attrs and sharability, into the TLB. This will eventually
"shareability" is how the Arm ARM spells it.
> be used by S1_ptw_translate to report stage1 translation failures,
> and by do_ats_write to fill in PAR_EL1.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> target/arm/cpu-param.h | 8 ++++++++
> target/arm/tlb_helper.c | 3 +++
> 2 files changed, 11 insertions(+)
>
> diff --git a/target/arm/cpu-param.h b/target/arm/cpu-param.h
> index 08681828ac..118ca0e5c0 100644
> --- a/target/arm/cpu-param.h
> +++ b/target/arm/cpu-param.h
> @@ -30,6 +30,14 @@
> */
> # define TARGET_PAGE_BITS_VARY
> # define TARGET_PAGE_BITS_MIN 10
> +
> +/*
> + * Cache the attrs and sharability fields from the page table entry.
"shareability".
We should document the format of these fields, similarly to
how the ARMCacheAttrs struct does. In particular, do we guarantee
at the point we fill this in that pte_attrs is in the MAIR_EL1
8-bit format and never the S2 descriptor-bits format?
> + */
> +# define TARGET_PAGE_ENTRY_EXTRA \
> + uint8_t pte_attrs; \
> + uint8_t shareability;
> +
> #endif
>
> #define NB_MMU_MODES 8
> diff --git a/target/arm/tlb_helper.c b/target/arm/tlb_helper.c
> index 49601394ec..353edbeb1d 100644
> --- a/target/arm/tlb_helper.c
> +++ b/target/arm/tlb_helper.c
> @@ -236,6 +236,9 @@ bool arm_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
> arm_tlb_mte_tagged(&res.f.attrs) = true;
> }
>
> + res.f.pte_attrs = res.cacheattrs.attrs;
> + res.f.shareability = res.cacheattrs.shareability;
> +
> tlb_set_page_full(cs, mmu_idx, address, &res.f);
> return true;
> } else if (probe) {
> --
Otherwise
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 22/42] target/arm: Use probe_access_full for MTE
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (20 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 21/42] target/arm: Enable TARGET_PAGE_ENTRY_EXTRA Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-06 14:52 ` Peter Maydell
2022-10-01 16:22 ` [PATCH v3 23/42] target/arm: Use probe_access_full for BTI Richard Henderson
` (20 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
The CPUTLBEntryFull structure now stores the original pte attributes, as
well as the physical address. Therefore, we no longer need a separate
bit in MemTxAttrs, nor do we need to walk the tree of memory regions.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/cpu.h | 1 -
target/arm/sve_ldst_internal.h | 1 +
target/arm/mte_helper.c | 61 +++++++++-------------------------
target/arm/sve_helper.c | 54 ++++++++++--------------------
target/arm/tlb_helper.c | 4 ---
5 files changed, 35 insertions(+), 86 deletions(-)
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 0f82f4aa1d..2694a93894 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -3394,7 +3394,6 @@ static inline MemTxAttrs *typecheck_memtxattrs(MemTxAttrs *x)
* generic target bits directly.
*/
#define arm_tlb_bti_gp(x) (typecheck_memtxattrs(x)->target_tlb_bit0)
-#define arm_tlb_mte_tagged(x) (typecheck_memtxattrs(x)->target_tlb_bit1)
/*
* AArch64 usage of the PAGE_TARGET_* bits for linux-user.
diff --git a/target/arm/sve_ldst_internal.h b/target/arm/sve_ldst_internal.h
index b5c473fc48..4f159ec4ad 100644
--- a/target/arm/sve_ldst_internal.h
+++ b/target/arm/sve_ldst_internal.h
@@ -134,6 +134,7 @@ typedef struct {
void *host;
int flags;
MemTxAttrs attrs;
+ bool tagged;
} SVEHostPage;
bool sve_probe_page(SVEHostPage *info, bool nofault, CPUARMState *env,
diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c
index fdd23ab3f8..a81c4a3318 100644
--- a/target/arm/mte_helper.c
+++ b/target/arm/mte_helper.c
@@ -105,10 +105,9 @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
TARGET_PAGE_BITS - LOG2_TAG_GRANULE - 1);
return tags + index;
#else
- uintptr_t index;
CPUTLBEntryFull *full;
+ MemTxAttrs attrs;
int in_page, flags;
- ram_addr_t ptr_ra;
hwaddr ptr_paddr, tag_paddr, xlat;
MemoryRegion *mr;
ARMASIdx tag_asi;
@@ -124,30 +123,12 @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
* valid. Indicate to probe_access_flags no-fault, then assert that
* we received a valid page.
*/
- flags = probe_access_flags(env, ptr, ptr_access, ptr_mmu_idx,
- ra == 0, &host, ra);
+ flags = probe_access_full(env, ptr, ptr_access, ptr_mmu_idx,
+ ra == 0, &host, &full, ra);
assert(!(flags & TLB_INVALID_MASK));
- /*
- * Find the CPUTLBEntryFull for ptr. This *must* be present in the TLB
- * because we just found the mapping.
- * TODO: Perhaps there should be a cputlb helper that returns a
- * matching tlb entry + iotlb entry.
- */
- index = tlb_index(env, ptr_mmu_idx, ptr);
-# ifdef CONFIG_DEBUG_TCG
- {
- CPUTLBEntry *entry = tlb_entry(env, ptr_mmu_idx, ptr);
- target_ulong comparator = (ptr_access == MMU_DATA_LOAD
- ? entry->addr_read
- : tlb_addr_write(entry));
- g_assert(tlb_hit(comparator, ptr));
- }
-# endif
- full = &env_tlb(env)->d[ptr_mmu_idx].fulltlb[index];
-
/* If the virtual page MemAttr != Tagged, access unchecked. */
- if (!arm_tlb_mte_tagged(&full->attrs)) {
+ if (full->pte_attrs != 0xf0) {
return NULL;
}
@@ -162,6 +143,13 @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
return NULL;
}
+ /*
+ * Remember these values across the second lookup below,
+ * which may invalidate this pointer via tlb resize.
+ */
+ ptr_paddr = full->phys_addr;
+ attrs = full->attrs;
+
/*
* The Normal memory access can extend to the next page. E.g. a single
* 8-byte access to the last byte of a page will check only the last
@@ -170,9 +158,8 @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
*/
in_page = -(ptr | TARGET_PAGE_MASK);
if (unlikely(ptr_size > in_page)) {
- void *ignore;
- flags |= probe_access_flags(env, ptr + in_page, ptr_access,
- ptr_mmu_idx, ra == 0, &ignore, ra);
+ flags |= probe_access_full(env, ptr + in_page, ptr_access,
+ ptr_mmu_idx, ra == 0, &host, &full, ra);
assert(!(flags & TLB_INVALID_MASK));
}
@@ -180,33 +167,17 @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
if (unlikely(flags & TLB_WATCHPOINT)) {
int wp = ptr_access == MMU_DATA_LOAD ? BP_MEM_READ : BP_MEM_WRITE;
assert(ra != 0);
- cpu_check_watchpoint(env_cpu(env), ptr, ptr_size,
- full->attrs, wp, ra);
+ cpu_check_watchpoint(env_cpu(env), ptr, ptr_size, attrs, wp, ra);
}
- /*
- * Find the physical address within the normal mem space.
- * The memory region lookup must succeed because TLB_MMIO was
- * not set in the cputlb lookup above.
- */
- mr = memory_region_from_host(host, &ptr_ra);
- tcg_debug_assert(mr != NULL);
- tcg_debug_assert(memory_region_is_ram(mr));
- ptr_paddr = ptr_ra;
- do {
- ptr_paddr += mr->addr;
- mr = mr->container;
- } while (mr);
-
/* Convert to the physical address in tag space. */
tag_paddr = ptr_paddr >> (LOG2_TAG_GRANULE + 1);
/* Look up the address in tag space. */
- tag_asi = full->attrs.secure ? ARMASIdx_TagS : ARMASIdx_TagNS;
+ tag_asi = attrs.secure ? ARMASIdx_TagS : ARMASIdx_TagNS;
tag_as = cpu_get_address_space(env_cpu(env), tag_asi);
mr = address_space_translate(tag_as, tag_paddr, &xlat, NULL,
- tag_access == MMU_DATA_STORE,
- full->attrs);
+ tag_access == MMU_DATA_STORE, attrs);
/*
* Note that @mr will never be NULL. If there is nothing in the address
diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
index 9cae8fd352..3d0d2987cd 100644
--- a/target/arm/sve_helper.c
+++ b/target/arm/sve_helper.c
@@ -5351,8 +5351,19 @@ bool sve_probe_page(SVEHostPage *info, bool nofault, CPUARMState *env,
*/
addr = useronly_clean_ptr(addr);
+#ifdef CONFIG_USER_ONLY
flags = probe_access_flags(env, addr, access_type, mmu_idx, nofault,
&info->host, retaddr);
+ memset(&info->attrs, 0, sizeof(info->attrs));
+ /* Require both ANON and MTE; see allocation_tag_mem(). */
+ info->tagged = (flags & PAGE_ANON) && (flags & PAGE_MTE);
+#else
+ CPUTLBEntryFull *full;
+ flags = probe_access_full(env, addr, access_type, mmu_idx, nofault,
+ &info->host, &full, retaddr);
+ info->attrs = full->attrs;
+ info->tagged = full->pte_attrs == 0xf0;
+#endif
info->flags = flags;
if (flags & TLB_INVALID_MASK) {
@@ -5362,33 +5373,6 @@ bool sve_probe_page(SVEHostPage *info, bool nofault, CPUARMState *env,
/* Ensure that info->host[] is relative to addr, not addr + mem_off. */
info->host -= mem_off;
-
-#ifdef CONFIG_USER_ONLY
- memset(&info->attrs, 0, sizeof(info->attrs));
- /* Require both MAP_ANON and PROT_MTE -- see allocation_tag_mem. */
- arm_tlb_mte_tagged(&info->attrs) =
- (flags & PAGE_ANON) && (flags & PAGE_MTE);
-#else
- /*
- * Find the iotlbentry for addr and return the transaction attributes.
- * This *must* be present in the TLB because we just found the mapping.
- */
- {
- uintptr_t index = tlb_index(env, mmu_idx, addr);
-
-# ifdef CONFIG_DEBUG_TCG
- CPUTLBEntry *entry = tlb_entry(env, mmu_idx, addr);
- target_ulong comparator = (access_type == MMU_DATA_LOAD
- ? entry->addr_read
- : tlb_addr_write(entry));
- g_assert(tlb_hit(comparator, addr));
-# endif
-
- CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
- info->attrs = full->attrs;
- }
-#endif
-
return true;
}
@@ -5617,7 +5601,7 @@ void sve_cont_ldst_mte_check(SVEContLdSt *info, CPUARMState *env,
intptr_t mem_off, reg_off, reg_last;
/* Process the page only if MemAttr == Tagged. */
- if (arm_tlb_mte_tagged(&info->page[0].attrs)) {
+ if (info->page[0].tagged) {
mem_off = info->mem_off_first[0];
reg_off = info->reg_off_first[0];
reg_last = info->reg_off_split;
@@ -5638,7 +5622,7 @@ void sve_cont_ldst_mte_check(SVEContLdSt *info, CPUARMState *env,
}
mem_off = info->mem_off_first[1];
- if (mem_off >= 0 && arm_tlb_mte_tagged(&info->page[1].attrs)) {
+ if (mem_off >= 0 && info->page[1].tagged) {
reg_off = info->reg_off_first[1];
reg_last = info->reg_off_last[1];
@@ -6017,7 +6001,7 @@ void sve_ldnfff1_r(CPUARMState *env, void *vg, const target_ulong addr,
* Disable MTE checking if the Tagged bit is not set. Since TBI must
* be set within MTEDESC for MTE, !mtedesc => !mte_active.
*/
- if (!arm_tlb_mte_tagged(&info.page[0].attrs)) {
+ if (!info.page[0].tagged) {
mtedesc = 0;
}
@@ -6568,7 +6552,7 @@ void sve_ld1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm,
cpu_check_watchpoint(env_cpu(env), addr, msize,
info.attrs, BP_MEM_READ, retaddr);
}
- if (mtedesc && arm_tlb_mte_tagged(&info.attrs)) {
+ if (mtedesc && info.tagged) {
mte_check(env, mtedesc, addr, retaddr);
}
if (unlikely(info.flags & TLB_MMIO)) {
@@ -6585,7 +6569,7 @@ void sve_ld1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm,
msize, info.attrs,
BP_MEM_READ, retaddr);
}
- if (mtedesc && arm_tlb_mte_tagged(&info.attrs)) {
+ if (mtedesc && info.tagged) {
mte_check(env, mtedesc, addr, retaddr);
}
tlb_fn(env, &scratch, reg_off, addr, retaddr);
@@ -6786,9 +6770,7 @@ void sve_ldff1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm,
(env_cpu(env), addr, msize) & BP_MEM_READ)) {
goto fault;
}
- if (mtedesc &&
- arm_tlb_mte_tagged(&info.attrs) &&
- !mte_probe(env, mtedesc, addr)) {
+ if (mtedesc && info.tagged && !mte_probe(env, mtedesc, addr)) {
goto fault;
}
@@ -6974,7 +6956,7 @@ void sve_st1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm,
info.attrs, BP_MEM_WRITE, retaddr);
}
- if (mtedesc && arm_tlb_mte_tagged(&info.attrs)) {
+ if (mtedesc && info.tagged) {
mte_check(env, mtedesc, addr, retaddr);
}
}
diff --git a/target/arm/tlb_helper.c b/target/arm/tlb_helper.c
index 353edbeb1d..3462a6ea14 100644
--- a/target/arm/tlb_helper.c
+++ b/target/arm/tlb_helper.c
@@ -231,10 +231,6 @@ bool arm_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
res.f.phys_addr &= TARGET_PAGE_MASK;
address &= TARGET_PAGE_MASK;
}
- /* Notice and record tagged memory. */
- if (cpu_isar_feature(aa64_mte, cpu) && res.cacheattrs.attrs == 0xf0) {
- arm_tlb_mte_tagged(&res.f.attrs) = true;
- }
res.f.pte_attrs = res.cacheattrs.attrs;
res.f.shareability = res.cacheattrs.shareability;
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 22/42] target/arm: Use probe_access_full for MTE
2022-10-01 16:22 ` [PATCH v3 22/42] target/arm: Use probe_access_full for MTE Richard Henderson
@ 2022-10-06 14:52 ` Peter Maydell
0 siblings, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-06 14:52 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:50, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> The CPUTLBEntryFull structure now stores the original pte attributes, as
> well as the physical address. Therefore, we no longer need a separate
> bit in MemTxAttrs, nor do we need to walk the tree of memory regions.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> @@ -162,6 +143,13 @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
> return NULL;
> }
>
> + /*
> + * Remember these values across the second lookup below,
> + * which may invalidate this pointer via tlb resize.
> + */
> + ptr_paddr = full->phys_addr;
> + attrs = full->attrs;
Can we set full = NULL here to catch any accidental attempts to use
the pointer after this point?
Otherwise
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 23/42] target/arm: Use probe_access_full for BTI
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (21 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 22/42] target/arm: Use probe_access_full for MTE Richard Henderson
@ 2022-10-01 16:22 ` Richard Henderson
2022-10-06 14:57 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 24/42] target/arm: Add ARMMMUIdx_Phys_{S,NS} Richard Henderson
` (19 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Add a field to TARGET_PAGE_ENTRY_EXTRA to hold the guarded bit.
In is_guarded_page, use probe_access_full instead of just guessing
that the tlb entry is still present. Also handles the FIXME about
executing from device memory.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/cpu-param.h | 8 ++++----
target/arm/cpu.h | 13 -------------
target/arm/internals.h | 1 +
target/arm/ptw.c | 7 ++++---
target/arm/translate-a64.c | 22 ++++++++--------------
5 files changed, 17 insertions(+), 34 deletions(-)
diff --git a/target/arm/cpu-param.h b/target/arm/cpu-param.h
index 118ca0e5c0..689a9645dc 100644
--- a/target/arm/cpu-param.h
+++ b/target/arm/cpu-param.h
@@ -32,12 +32,12 @@
# define TARGET_PAGE_BITS_MIN 10
/*
- * Cache the attrs and sharability fields from the page table entry.
+ * Cache the attrs, sharability, and gp fields from the page table entry.
*/
# define TARGET_PAGE_ENTRY_EXTRA \
- uint8_t pte_attrs; \
- uint8_t shareability;
-
+ uint8_t pte_attrs; \
+ uint8_t shareability; \
+ bool guarded;
#endif
#define NB_MMU_MODES 8
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 2694a93894..c8cad2ef7c 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -3382,19 +3382,6 @@ static inline uint64_t *aa64_vfp_qreg(CPUARMState *env, unsigned regno)
/* Shared between translate-sve.c and sve_helper.c. */
extern const uint64_t pred_esz_masks[5];
-/* Helper for the macros below, validating the argument type. */
-static inline MemTxAttrs *typecheck_memtxattrs(MemTxAttrs *x)
-{
- return x;
-}
-
-/*
- * Lvalue macros for ARM TLB bits that we must cache in the TCG TLB.
- * Using these should be a bit more self-documenting than using the
- * generic target bits directly.
- */
-#define arm_tlb_bti_gp(x) (typecheck_memtxattrs(x)->target_tlb_bit0)
-
/*
* AArch64 usage of the PAGE_TARGET_* bits for linux-user.
* Note that with the Linux kernel, PROT_MTE may not be cleared by mprotect
diff --git a/target/arm/internals.h b/target/arm/internals.h
index fd17aee459..a50189e2e4 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -1067,6 +1067,7 @@ typedef struct ARMCacheAttrs {
unsigned int attrs:8;
unsigned int shareability:2; /* as in the SH field of the VMSAv8-64 PTEs */
bool is_s2_format:1;
+ bool guarded:1; /* guarded bit of the v8-64 PTE */
} ARMCacheAttrs;
/* Fields that are valid upon success. */
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 1bc194ffa1..ccfef2caca 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -1319,9 +1319,10 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
*/
result->f.attrs.secure = false;
}
- /* When in aarch64 mode, and BTI is enabled, remember GP in the IOTLB. */
- if (aarch64 && guarded && cpu_isar_feature(aa64_bti, cpu)) {
- arm_tlb_bti_gp(&result->f.attrs) = true;
+
+ /* When in aarch64 mode, and BTI is enabled, remember GP in the TLB. */
+ if (aarch64 && cpu_isar_feature(aa64_bti, cpu)) {
+ result->f.guarded = guarded;
}
if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 5b67375f4e..22802d1d2f 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -14601,22 +14601,16 @@ static bool is_guarded_page(CPUARMState *env, DisasContext *s)
#ifdef CONFIG_USER_ONLY
return page_get_flags(addr) & PAGE_BTI;
#else
+ CPUTLBEntryFull *full;
+ void *host;
int mmu_idx = arm_to_core_mmu_idx(s->mmu_idx);
- unsigned int index = tlb_index(env, mmu_idx, addr);
- CPUTLBEntry *entry = tlb_entry(env, mmu_idx, addr);
+ int flags;
- /*
- * We test this immediately after reading an insn, which means
- * that any normal page must be in the TLB. The only exception
- * would be for executing from flash or device memory, which
- * does not retain the TLB entry.
- *
- * FIXME: Assume false for those, for now. We could use
- * arm_cpu_get_phys_page_attrs_debug to re-read the page
- * table entry even for that case.
- */
- return (tlb_hit(entry->addr_code, addr) &&
- arm_tlb_bti_gp(&env_tlb(env)->d[mmu_idx].fulltlb[index].attrs));
+ flags = probe_access_full(env, addr, MMU_INST_FETCH, mmu_idx,
+ false, &host, &full, 0);
+ assert(!(flags & TLB_INVALID_MASK));
+
+ return full->guarded;
#endif
}
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 23/42] target/arm: Use probe_access_full for BTI
2022-10-01 16:22 ` [PATCH v3 23/42] target/arm: Use probe_access_full for BTI Richard Henderson
@ 2022-10-06 14:57 ` Peter Maydell
2022-10-06 18:53 ` Richard Henderson
0 siblings, 1 reply; 90+ messages in thread
From: Peter Maydell @ 2022-10-06 14:57 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:38, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Add a field to TARGET_PAGE_ENTRY_EXTRA to hold the guarded bit.
> In is_guarded_page, use probe_access_full instead of just guessing
> that the tlb entry is still present. Also handles the FIXME about
> executing from device memory.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> target/arm/cpu-param.h | 8 ++++----
> target/arm/cpu.h | 13 -------------
> target/arm/internals.h | 1 +
> target/arm/ptw.c | 7 ++++---
> target/arm/translate-a64.c | 22 ++++++++--------------
> 5 files changed, 17 insertions(+), 34 deletions(-)
>
> diff --git a/target/arm/cpu-param.h b/target/arm/cpu-param.h
> index 118ca0e5c0..689a9645dc 100644
> --- a/target/arm/cpu-param.h
> +++ b/target/arm/cpu-param.h
> @@ -32,12 +32,12 @@
> # define TARGET_PAGE_BITS_MIN 10
>
> /*
> - * Cache the attrs and sharability fields from the page table entry.
> + * Cache the attrs, sharability, and gp fields from the page table entry.
> */
> # define TARGET_PAGE_ENTRY_EXTRA \
> - uint8_t pte_attrs; \
> - uint8_t shareability;
> -
> + uint8_t pte_attrs; \
> + uint8_t shareability; \
> + bool guarded;
I notice this now brings this very close to just having an ARMCacheAttrs
struct in it (in fact it's going to be one byte bigger than the ARMCachettrs).
But it's probably better to keep them separate since we care a lot more
about keeping the TLB entry small I suppose.
> diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
> index 5b67375f4e..22802d1d2f 100644
> --- a/target/arm/translate-a64.c
> +++ b/target/arm/translate-a64.c
> @@ -14601,22 +14601,16 @@ static bool is_guarded_page(CPUARMState *env, DisasContext *s)
> #ifdef CONFIG_USER_ONLY
> return page_get_flags(addr) & PAGE_BTI;
> #else
> + CPUTLBEntryFull *full;
> + void *host;
> int mmu_idx = arm_to_core_mmu_idx(s->mmu_idx);
> - unsigned int index = tlb_index(env, mmu_idx, addr);
> - CPUTLBEntry *entry = tlb_entry(env, mmu_idx, addr);
> + int flags;
>
> - /*
> - * We test this immediately after reading an insn, which means
> - * that any normal page must be in the TLB. The only exception
> - * would be for executing from flash or device memory, which
> - * does not retain the TLB entry.
> - *
> - * FIXME: Assume false for those, for now. We could use
> - * arm_cpu_get_phys_page_attrs_debug to re-read the page
> - * table entry even for that case.
> - */
I think we should keep at least some of this comment: the part
about the reason we can assert that probe_access_full() doesn't
return TLB_INVALID being that we tested immediately after the
insn read is still true, right?
> - return (tlb_hit(entry->addr_code, addr) &&
> - arm_tlb_bti_gp(&env_tlb(env)->d[mmu_idx].fulltlb[index].attrs));
> + flags = probe_access_full(env, addr, MMU_INST_FETCH, mmu_idx,
> + false, &host, &full, 0);
> + assert(!(flags & TLB_INVALID_MASK));
> +
> + return full->guarded;
> #endif
> }
Otherwise
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v3 23/42] target/arm: Use probe_access_full for BTI
2022-10-06 14:57 ` Peter Maydell
@ 2022-10-06 18:53 ` Richard Henderson
0 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-06 18:53 UTC (permalink / raw)
To: Peter Maydell; +Cc: qemu-devel, qemu-arm
On 10/6/22 07:57, Peter Maydell wrote:
> On Sat, 1 Oct 2022 at 17:38, Richard Henderson
> <richard.henderson@linaro.org> wrote:
>>
>> Add a field to TARGET_PAGE_ENTRY_EXTRA to hold the guarded bit.
>> In is_guarded_page, use probe_access_full instead of just guessing
>> that the tlb entry is still present. Also handles the FIXME about
>> executing from device memory.
>>
>> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
>> ---
>> target/arm/cpu-param.h | 8 ++++----
>> target/arm/cpu.h | 13 -------------
>> target/arm/internals.h | 1 +
>> target/arm/ptw.c | 7 ++++---
>> target/arm/translate-a64.c | 22 ++++++++--------------
>> 5 files changed, 17 insertions(+), 34 deletions(-)
>>
>> diff --git a/target/arm/cpu-param.h b/target/arm/cpu-param.h
>> index 118ca0e5c0..689a9645dc 100644
>> --- a/target/arm/cpu-param.h
>> +++ b/target/arm/cpu-param.h
>> @@ -32,12 +32,12 @@
>> # define TARGET_PAGE_BITS_MIN 10
>>
>> /*
>> - * Cache the attrs and sharability fields from the page table entry.
>> + * Cache the attrs, sharability, and gp fields from the page table entry.
>> */
>> # define TARGET_PAGE_ENTRY_EXTRA \
>> - uint8_t pte_attrs; \
>> - uint8_t shareability;
>> -
>> + uint8_t pte_attrs; \
>> + uint8_t shareability; \
>> + bool guarded;
>
> I notice this now brings this very close to just having an ARMCacheAttrs
> struct in it (in fact it's going to be one byte bigger than the ARMCachettrs).
> But it's probably better to keep them separate since we care a lot more
> about keeping the TLB entry small I suppose.
I kept them as separate fields like this for simplicity. Since CPUTLBEntryFull is 4 or
8-byte aligned (depending on the host), the structure still has 1 or 5 bytes of padding
after the addition of this bool.
>> - /*
>> - * We test this immediately after reading an insn, which means
>> - * that any normal page must be in the TLB. The only exception
>> - * would be for executing from flash or device memory, which
>> - * does not retain the TLB entry.
>> - *
>> - * FIXME: Assume false for those, for now. We could use
>> - * arm_cpu_get_phys_page_attrs_debug to re-read the page
>> - * table entry even for that case.
>> - */
>
> I think we should keep at least some of this comment: the part
> about the reason we can assert that probe_access_full() doesn't
> return TLB_INVALID being that we tested immediately after the
> insn read is still true, right?
Yes.
r~
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 24/42] target/arm: Add ARMMMUIdx_Phys_{S,NS}
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (22 preceding siblings ...)
2022-10-01 16:22 ` [PATCH v3 23/42] target/arm: Use probe_access_full for BTI Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-06 15:29 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 25/42] target/arm: Move ARMMMUIdx_Stage2 to a real tlb mmu_idx Richard Henderson
` (18 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Not yet used, but add mmu indexes for 1-1 mapping
to physical addresses.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/cpu-param.h | 2 +-
target/arm/cpu.h | 7 ++++++-
target/arm/ptw.c | 19 +++++++++++++++++--
3 files changed, 24 insertions(+), 4 deletions(-)
diff --git a/target/arm/cpu-param.h b/target/arm/cpu-param.h
index 689a9645dc..98bd9e435e 100644
--- a/target/arm/cpu-param.h
+++ b/target/arm/cpu-param.h
@@ -40,6 +40,6 @@
bool guarded;
#endif
-#define NB_MMU_MODES 8
+#define NB_MMU_MODES 10
#endif
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index c8cad2ef7c..0effa85c56 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -2899,8 +2899,9 @@ bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync);
* EL2 EL2&0 +PAN
* EL2 (aka NS PL2)
* EL3 (aka S PL1)
+ * Physical (NS & S)
*
- * for a total of 8 different mmu_idx.
+ * for a total of 10 different mmu_idx.
*
* R profile CPUs have an MPU, but can use the same set of MMU indexes
* as A profile. They only need to distinguish EL0 and EL1 (and
@@ -2965,6 +2966,10 @@ typedef enum ARMMMUIdx {
ARMMMUIdx_E2 = 6 | ARM_MMU_IDX_A,
ARMMMUIdx_E3 = 7 | ARM_MMU_IDX_A,
+ /* TLBs with 1-1 mapping to the physical address spaces. */
+ ARMMMUIdx_Phys_NS = 8 | ARM_MMU_IDX_A,
+ ARMMMUIdx_Phys_S = 9 | ARM_MMU_IDX_A,
+
/*
* These are not allocated TLBs and are used only for AT system
* instructions or for the first stage of an S12 page table walk.
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index ccfef2caca..05dcacf45b 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -179,6 +179,11 @@ static bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx,
case ARMMMUIdx_E3:
break;
+ case ARMMMUIdx_Phys_NS:
+ case ARMMMUIdx_Phys_S:
+ /* No translation for physical address spaces. */
+ return true;
+
default:
g_assert_not_reached();
}
@@ -2286,10 +2291,17 @@ static bool get_phys_addr_disabled(CPUARMState *env, target_ulong address,
{
uint8_t memattr = 0x00; /* Device nGnRnE */
uint8_t shareability = 0; /* non-sharable */
+ int r_el;
- if (mmu_idx != ARMMMUIdx_Stage2 && mmu_idx != ARMMMUIdx_Stage2_S) {
- int r_el = regime_el(env, mmu_idx);
+ switch (mmu_idx) {
+ case ARMMMUIdx_Stage2:
+ case ARMMMUIdx_Stage2_S:
+ case ARMMMUIdx_Phys_NS:
+ case ARMMMUIdx_Phys_S:
+ break;
+ default:
+ r_el = regime_el(env, mmu_idx);
if (arm_el_is_aa64(env, r_el)) {
int pamax = arm_pamax(env_archcpu(env));
uint64_t tcr = env->cp15.tcr_el[r_el];
@@ -2338,6 +2350,7 @@ static bool get_phys_addr_disabled(CPUARMState *env, target_ulong address,
shareability = 2; /* outer sharable */
}
result->cacheattrs.is_s2_format = false;
+ break;
}
result->f.phys_addr = address;
@@ -2543,6 +2556,7 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
is_secure = arm_is_secure_below_el3(env);
break;
case ARMMMUIdx_Stage2:
+ case ARMMMUIdx_Phys_NS:
case ARMMMUIdx_MPrivNegPri:
case ARMMMUIdx_MUserNegPri:
case ARMMMUIdx_MPriv:
@@ -2551,6 +2565,7 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
break;
case ARMMMUIdx_E3:
case ARMMMUIdx_Stage2_S:
+ case ARMMMUIdx_Phys_S:
case ARMMMUIdx_MSPrivNegPri:
case ARMMMUIdx_MSUserNegPri:
case ARMMMUIdx_MSPriv:
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 25/42] target/arm: Move ARMMMUIdx_Stage2 to a real tlb mmu_idx
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (23 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 24/42] target/arm: Add ARMMMUIdx_Phys_{S,NS} Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-06 15:46 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 26/42] target/arm: Plumb debug into S1_ptw_translate Richard Henderson
` (17 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
We had been marking this ARM_MMU_IDX_NOTLB, move it to a real tlb.
Flush the tlb when invalidating stage 1+2 translations.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/cpu-param.h | 2 +-
target/arm/cpu.h | 23 +++++++++++++----------
target/arm/helper.c | 4 +++-
3 files changed, 17 insertions(+), 12 deletions(-)
diff --git a/target/arm/cpu-param.h b/target/arm/cpu-param.h
index 98bd9e435e..283618f601 100644
--- a/target/arm/cpu-param.h
+++ b/target/arm/cpu-param.h
@@ -40,6 +40,6 @@
bool guarded;
#endif
-#define NB_MMU_MODES 10
+#define NB_MMU_MODES 12
#endif
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 0effa85c56..732c0c00ac 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -2900,8 +2900,9 @@ bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync);
* EL2 (aka NS PL2)
* EL3 (aka S PL1)
* Physical (NS & S)
+ * Stage2 (NS & S)
*
- * for a total of 10 different mmu_idx.
+ * for a total of 12 different mmu_idx.
*
* R profile CPUs have an MPU, but can use the same set of MMU indexes
* as A profile. They only need to distinguish EL0 and EL1 (and
@@ -2970,6 +2971,15 @@ typedef enum ARMMMUIdx {
ARMMMUIdx_Phys_NS = 8 | ARM_MMU_IDX_A,
ARMMMUIdx_Phys_S = 9 | ARM_MMU_IDX_A,
+ /*
+ * Used for second stage of an S12 page table walk, or for descriptor
+ * loads during first stage of an S1 page table walk. Note that both
+ * are in use simultaneously for SecureEL2: the security state for
+ * the S2 ptw is selected by the NS bit from the S1 ptw.
+ */
+ ARMMMUIdx_Stage2 = 10 | ARM_MMU_IDX_A,
+ ARMMMUIdx_Stage2_S = 11 | ARM_MMU_IDX_A,
+
/*
* These are not allocated TLBs and are used only for AT system
* instructions or for the first stage of an S12 page table walk.
@@ -2977,15 +2987,6 @@ typedef enum ARMMMUIdx {
ARMMMUIdx_Stage1_E0 = 0 | ARM_MMU_IDX_NOTLB,
ARMMMUIdx_Stage1_E1 = 1 | ARM_MMU_IDX_NOTLB,
ARMMMUIdx_Stage1_E1_PAN = 2 | ARM_MMU_IDX_NOTLB,
- /*
- * Not allocated a TLB: used only for second stage of an S12 page
- * table walk, or for descriptor loads during first stage of an S1
- * page table walk. Note that if we ever want to have a TLB for this
- * then various TLB flush insns which currently are no-ops or flush
- * only stage 1 MMU indexes will need to change to flush stage 2.
- */
- ARMMMUIdx_Stage2 = 3 | ARM_MMU_IDX_NOTLB,
- ARMMMUIdx_Stage2_S = 4 | ARM_MMU_IDX_NOTLB,
/*
* M-profile.
@@ -3016,6 +3017,8 @@ typedef enum ARMMMUIdxBit {
TO_CORE_BIT(E20_2),
TO_CORE_BIT(E20_2_PAN),
TO_CORE_BIT(E3),
+ TO_CORE_BIT(Stage2),
+ TO_CORE_BIT(Stage2_S),
TO_CORE_BIT(MUser),
TO_CORE_BIT(MPriv),
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 6fe85c6642..19a03eb200 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -4319,7 +4319,9 @@ static int alle1_tlbmask(CPUARMState *env)
*/
return (ARMMMUIdxBit_E10_1 |
ARMMMUIdxBit_E10_1_PAN |
- ARMMMUIdxBit_E10_0);
+ ARMMMUIdxBit_E10_0 |
+ ARMMMUIdxBit_Stage2 |
+ ARMMMUIdxBit_Stage2_S);
}
static int e2_tlbmask(CPUARMState *env)
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 25/42] target/arm: Move ARMMMUIdx_Stage2 to a real tlb mmu_idx
2022-10-01 16:23 ` [PATCH v3 25/42] target/arm: Move ARMMMUIdx_Stage2 to a real tlb mmu_idx Richard Henderson
@ 2022-10-06 15:46 ` Peter Maydell
2022-10-06 19:21 ` Richard Henderson
0 siblings, 1 reply; 90+ messages in thread
From: Peter Maydell @ 2022-10-06 15:46 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:39, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> We had been marking this ARM_MMU_IDX_NOTLB, move it to a real tlb.
> Flush the tlb when invalidating stage 1+2 translations.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> @@ -2977,15 +2987,6 @@ typedef enum ARMMMUIdx {
> ARMMMUIdx_Stage1_E0 = 0 | ARM_MMU_IDX_NOTLB,
> ARMMMUIdx_Stage1_E1 = 1 | ARM_MMU_IDX_NOTLB,
> ARMMMUIdx_Stage1_E1_PAN = 2 | ARM_MMU_IDX_NOTLB,
> - /*
> - * Not allocated a TLB: used only for second stage of an S12 page
> - * table walk, or for descriptor loads during first stage of an S1
> - * page table walk. Note that if we ever want to have a TLB for this
> - * then various TLB flush insns which currently are no-ops or flush
> - * only stage 1 MMU indexes will need to change to flush stage 2.
> - */
> - ARMMMUIdx_Stage2 = 3 | ARM_MMU_IDX_NOTLB,
> - ARMMMUIdx_Stage2_S = 4 | ARM_MMU_IDX_NOTLB,
>
> /*
> * M-profile.
> @@ -3016,6 +3017,8 @@ typedef enum ARMMMUIdxBit {
> TO_CORE_BIT(E20_2),
> TO_CORE_BIT(E20_2_PAN),
> TO_CORE_BIT(E3),
> + TO_CORE_BIT(Stage2),
> + TO_CORE_BIT(Stage2_S),
>
> TO_CORE_BIT(MUser),
> TO_CORE_BIT(MPriv),
> diff --git a/target/arm/helper.c b/target/arm/helper.c
> index 6fe85c6642..19a03eb200 100644
> --- a/target/arm/helper.c
> +++ b/target/arm/helper.c
> @@ -4319,7 +4319,9 @@ static int alle1_tlbmask(CPUARMState *env)
> */
> return (ARMMMUIdxBit_E10_1 |
> ARMMMUIdxBit_E10_1_PAN |
> - ARMMMUIdxBit_E10_0);
> + ARMMMUIdxBit_E10_0 |
> + ARMMMUIdxBit_Stage2 |
> + ARMMMUIdxBit_Stage2_S);
> }
This isn't sufficient. As the comment notes, you also need to change
all the TLBI ops for S2 invalidates which we currently implement as
ARM_CP_NOP so they now flush the stage 2 TLB. I think that searching
helper.c for 'IPAS2' probably finds you all of them.
alle1_tlbmask() is also only used for the aarch64 TLBI ops -- the
aarch32 ones are tlbiall_nsnh_write() and tlbiall_nsnh_is_write(),
I think, and those also now need to flush stage 2.
VMID writes also now need to flush the stage 2 TLB as well as
the combined s1&2 TLB -- see vttbr_write().
Side note, looks like we didn't update vttbr_write() to know about
the EL2&0 MMU indexes ?
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 26/42] target/arm: Plumb debug into S1_ptw_translate
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (24 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 25/42] target/arm: Move ARMMMUIdx_Stage2 to a real tlb mmu_idx Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-06 15:54 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 27/42] target/arm: Use softmmu tlbs for page table walking Richard Henderson
` (16 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Before using softmmu page tables for the ptw, plumb down
a debug parameter so that we can query page table entries
from gdbstub without modifying cpu state.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 66 +++++++++++++++++++++++++++++-------------------
1 file changed, 40 insertions(+), 26 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 05dcacf45b..45adb9d5a9 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -16,7 +16,7 @@
static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
- bool is_secure, bool s1_is_el0,
+ bool is_secure, bool s1_is_el0, bool debug,
GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
__attribute__((nonnull));
@@ -212,7 +212,7 @@ static bool ptw_attrs_are_device(uint64_t hcr, ARMCacheAttrs cacheattrs)
/* Translate a S1 pagetable walk through S2 if needed. */
static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
- hwaddr addr, bool *is_secure_ptr,
+ hwaddr addr, bool *is_secure_ptr, bool debug,
ARMMMUFaultInfo *fi)
{
bool is_secure = *is_secure_ptr;
@@ -225,7 +225,7 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
int ret;
ret = get_phys_addr_lpae(env, addr, MMU_DATA_LOAD, s2_mmu_idx,
- is_secure, false, &s2, fi);
+ is_secure, false, debug, &s2, fi);
if (ret) {
assert(fi->type != ARMFault_None);
fi->s2addr = addr;
@@ -268,7 +268,7 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
/* All loads done in the course of a page table walk go through here. */
static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
- ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi)
+ ARMMMUIdx mmu_idx, bool debug, ARMMMUFaultInfo *fi)
{
CPUState *cs = env_cpu(env);
MemTxAttrs attrs = {};
@@ -276,7 +276,7 @@ static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
AddressSpace *as;
uint32_t data;
- addr = S1_ptw_translate(env, mmu_idx, addr, &is_secure, fi);
+ addr = S1_ptw_translate(env, mmu_idx, addr, &is_secure, debug, fi);
attrs.secure = is_secure;
as = arm_addressspace(cs, attrs);
if (fi->s1ptw) {
@@ -296,7 +296,7 @@ static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
}
static uint64_t arm_ldq_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
- ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi)
+ ARMMMUIdx mmu_idx, bool debug, ARMMMUFaultInfo *fi)
{
CPUState *cs = env_cpu(env);
MemTxAttrs attrs = {};
@@ -304,7 +304,7 @@ static uint64_t arm_ldq_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
AddressSpace *as;
uint64_t data;
- addr = S1_ptw_translate(env, mmu_idx, addr, &is_secure, fi);
+ addr = S1_ptw_translate(env, mmu_idx, addr, &is_secure, debug, fi);
attrs.secure = is_secure;
as = arm_addressspace(cs, attrs);
if (fi->s1ptw) {
@@ -433,8 +433,8 @@ static int simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
- bool is_secure, GetPhysAddrResult *result,
- ARMMMUFaultInfo *fi)
+ bool is_secure, bool debug,
+ GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
{
int level = 1;
uint32_t table;
@@ -453,7 +453,7 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
fi->type = ARMFault_Translation;
goto do_fault;
}
- desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, fi);
+ desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, debug, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -491,7 +491,7 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
/* Fine pagetable. */
table = (desc & 0xfffff000) | ((address >> 8) & 0xffc);
}
- desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, fi);
+ desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, debug, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -552,8 +552,8 @@ do_fault:
static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
- bool is_secure, GetPhysAddrResult *result,
- ARMMMUFaultInfo *fi)
+ bool is_secure, bool debug,
+ GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
{
ARMCPU *cpu = env_archcpu(env);
int level = 1;
@@ -576,7 +576,7 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
fi->type = ARMFault_Translation;
goto do_fault;
}
- desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, fi);
+ desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, debug, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -629,7 +629,7 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
ns = extract32(desc, 3, 1);
/* Lookup l2 entry. */
table = (desc & 0xfffffc00) | ((address >> 10) & 0x3fc);
- desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, fi);
+ desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, debug, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -984,7 +984,7 @@ static bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level,
*/
static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
- bool is_secure, bool s1_is_el0,
+ bool is_secure, bool s1_is_el0, bool debug,
GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
{
ARMCPU *cpu = env_archcpu(env);
@@ -1210,7 +1210,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
descaddr |= (address >> (stride * (4 - level))) & indexmask;
descaddr &= ~7ULL;
nstable = extract32(tableattrs, 4, 1);
- descriptor = arm_ldq_ptw(env, descaddr, !nstable, mmu_idx, fi);
+ descriptor = arm_ldq_ptw(env, descaddr, !nstable, mmu_idx, debug, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -2361,10 +2361,13 @@ static bool get_phys_addr_disabled(CPUARMState *env, target_ulong address,
return 0;
}
-bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
- MMUAccessType access_type, ARMMMUIdx mmu_idx,
- bool is_secure, GetPhysAddrResult *result,
- ARMMMUFaultInfo *fi)
+static bool get_phys_addr_with_secure_debug(CPUARMState *env,
+ target_ulong address,
+ MMUAccessType access_type,
+ ARMMMUIdx mmu_idx,
+ bool is_secure, bool debug,
+ GetPhysAddrResult *result,
+ ARMMMUFaultInfo *fi)
{
ARMMMUIdx s1_mmu_idx = stage_1_mmu_idx(mmu_idx);
@@ -2418,7 +2421,7 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
memset(result, 0, sizeof(*result));
ret = get_phys_addr_lpae(env, ipa, access_type, s2_mmu_idx,
- s2walk_secure, is_el0, result, fi);
+ s2walk_secure, is_el0, debug, result, fi);
fi->s2addr = ipa;
/* Combine the S1 and S2 perms. */
@@ -2526,16 +2529,25 @@ bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
}
if (regime_using_lpae_format(env, mmu_idx)) {
return get_phys_addr_lpae(env, address, access_type, mmu_idx,
- is_secure, false, result, fi);
+ is_secure, false, debug, result, fi);
} else if (regime_sctlr(env, mmu_idx) & SCTLR_XP) {
return get_phys_addr_v6(env, address, access_type, mmu_idx,
- is_secure, result, fi);
+ is_secure, debug, result, fi);
} else {
return get_phys_addr_v5(env, address, access_type, mmu_idx,
- is_secure, result, fi);
+ is_secure, debug, result, fi);
}
}
+bool get_phys_addr_with_secure(CPUARMState *env, target_ulong address,
+ MMUAccessType access_type, ARMMMUIdx mmu_idx,
+ bool is_secure, GetPhysAddrResult *result,
+ ARMMMUFaultInfo *fi)
+{
+ return get_phys_addr_with_secure_debug(env, address, access_type, mmu_idx,
+ is_secure, false, result, fi);
+}
+
bool get_phys_addr(CPUARMState *env, target_ulong address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
@@ -2587,9 +2599,11 @@ hwaddr arm_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,
GetPhysAddrResult res = {};
ARMMMUFaultInfo fi = {};
ARMMMUIdx mmu_idx = arm_mmu_idx(env);
+ bool is_secure = arm_is_secure(env);
bool ret;
- ret = get_phys_addr(env, addr, MMU_DATA_LOAD, mmu_idx, &res, &fi);
+ ret = get_phys_addr_with_secure_debug(env, addr, MMU_DATA_LOAD, mmu_idx,
+ is_secure, true, &res, &fi);
*attrs = res.f.attrs;
if (ret) {
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 26/42] target/arm: Plumb debug into S1_ptw_translate
2022-10-01 16:23 ` [PATCH v3 26/42] target/arm: Plumb debug into S1_ptw_translate Richard Henderson
@ 2022-10-06 15:54 ` Peter Maydell
0 siblings, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-06 15:54 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:50, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Before using softmmu page tables for the ptw, plumb down
> a debug parameter so that we can query page table entries
> from gdbstub without modifying cpu state.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> +static bool get_phys_addr_with_secure_debug(CPUARMState *env,
> + target_ulong address,
> + MMUAccessType access_type,
> + ARMMMUIdx mmu_idx,
> + bool is_secure, bool debug,
> + GetPhysAddrResult *result,
> + ARMMMUFaultInfo *fi)
I'm not too enthused about this function name, because
Secure Debug is an entirely unrelated architecture feature, but
I can't immediately think of a better one, so
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 27/42] target/arm: Use softmmu tlbs for page table walking
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (25 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 26/42] target/arm: Plumb debug into S1_ptw_translate Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 9:01 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 28/42] target/arm: Split out get_phys_addr_twostage Richard Henderson
` (15 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
So far, limit the change to S1_ptw_translate, arm_ldl_ptw, and
arm_ldq_ptw. Use probe_access_full to find the host address,
and if so use a host load. If the probe fails, we've got our
fault info already. On the off chance that page tables are not
in RAM, continue to use the address_space_ld* functions.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/cpu.h | 5 +
target/arm/ptw.c | 207 ++++++++++++++++++++++++++--------------
target/arm/tlb_helper.c | 17 +++-
3 files changed, 155 insertions(+), 74 deletions(-)
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 732c0c00ac..7108568685 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -225,6 +225,8 @@ typedef struct CPUARMTBFlags {
target_ulong flags2;
} CPUARMTBFlags;
+typedef struct ARMMMUFaultInfo ARMMMUFaultInfo;
+
typedef struct CPUArchState {
/* Regs for current mode. */
uint32_t regs[16];
@@ -715,6 +717,9 @@ typedef struct CPUArchState {
struct CPUBreakpoint *cpu_breakpoint[16];
struct CPUWatchpoint *cpu_watchpoint[16];
+ /* Optional fault info across tlb lookup. */
+ ARMMMUFaultInfo *tlb_fi;
+
/* Fields up to this point are cleared by a CPU reset */
struct {} end_reset_fields;
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 45adb9d5a9..ba496c3421 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -9,6 +9,7 @@
#include "qemu/osdep.h"
#include "qemu/log.h"
#include "qemu/range.h"
+#include "exec/exec-all.h"
#include "cpu.h"
#include "internals.h"
#include "idau.h"
@@ -191,7 +192,7 @@ static bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx,
return (regime_sctlr(env, mmu_idx) & SCTLR_M) == 0;
}
-static bool ptw_attrs_are_device(uint64_t hcr, ARMCacheAttrs cacheattrs)
+static bool S2_attrs_are_device(uint64_t hcr, uint8_t attrs)
{
/*
* For an S1 page table walk, the stage 1 attributes are always
@@ -202,41 +203,72 @@ static bool ptw_attrs_are_device(uint64_t hcr, ARMCacheAttrs cacheattrs)
* With HCR_EL2.FWB == 1 this is when descriptor bit [4] is 0, ie
* when cacheattrs.attrs bit [2] is 0.
*/
- assert(cacheattrs.is_s2_format);
if (hcr & HCR_FWB) {
- return (cacheattrs.attrs & 0x4) == 0;
+ return (attrs & 0x4) == 0;
} else {
- return (cacheattrs.attrs & 0xc) == 0;
+ return (attrs & 0xc) == 0;
}
}
/* Translate a S1 pagetable walk through S2 if needed. */
-static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
- hwaddr addr, bool *is_secure_ptr, bool debug,
- ARMMMUFaultInfo *fi)
+static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx, hwaddr addr,
+ bool *is_secure_ptr, void **hphys, hwaddr *gphys,
+ bool debug, ARMMMUFaultInfo *fi)
{
bool is_secure = *is_secure_ptr;
ARMMMUIdx s2_mmu_idx = is_secure ? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2;
+ bool s2_phys = false;
+ uint8_t pte_attrs;
+ bool pte_secure;
- if (arm_mmu_idx_is_stage1_of_2(mmu_idx) &&
- !regime_translation_disabled(env, s2_mmu_idx, is_secure)) {
- GetPhysAddrResult s2 = {};
- uint64_t hcr;
- int ret;
+ if (!arm_mmu_idx_is_stage1_of_2(mmu_idx)
+ || regime_translation_disabled(env, s2_mmu_idx, is_secure)) {
+ s2_mmu_idx = is_secure ? ARMMMUIdx_Phys_S : ARMMMUIdx_Phys_NS;
+ s2_phys = true;
+ }
- ret = get_phys_addr_lpae(env, addr, MMU_DATA_LOAD, s2_mmu_idx,
- is_secure, false, debug, &s2, fi);
- if (ret) {
- assert(fi->type != ARMFault_None);
- fi->s2addr = addr;
- fi->stage2 = true;
- fi->s1ptw = true;
- fi->s1ns = !is_secure;
- return ~0;
+ if (unlikely(debug)) {
+ /*
+ * From gdbstub, do not use softmmu so that we don't modify the
+ * state of the cpu at all, including softmmu tlb contents.
+ */
+ if (s2_phys) {
+ *gphys = addr;
+ pte_attrs = 0;
+ pte_secure = is_secure;
+ } else {
+ GetPhysAddrResult s2 = { };
+ if (!get_phys_addr_lpae(env, addr, MMU_DATA_LOAD, s2_mmu_idx,
+ is_secure, false, debug, &s2, fi)) {
+ goto fail;
+ }
+ *gphys = s2.f.phys_addr;
+ pte_attrs = s2.cacheattrs.attrs;
+ pte_secure = s2.f.attrs.secure;
}
+ *hphys = NULL;
+ } else {
+ CPUTLBEntryFull *full;
+ int flags;
- hcr = arm_hcr_el2_eff_secstate(env, is_secure);
- if ((hcr & HCR_PTW) && ptw_attrs_are_device(hcr, s2.cacheattrs)) {
+ env->tlb_fi = fi;
+ flags = probe_access_full(env, addr, MMU_DATA_LOAD,
+ arm_to_core_mmu_idx(s2_mmu_idx),
+ true, hphys, &full, 0);
+ env->tlb_fi = NULL;
+
+ if (unlikely(flags & TLB_INVALID_MASK)) {
+ goto fail;
+ }
+ *gphys = full->phys_addr;
+ pte_attrs = full->pte_attrs;
+ pte_secure = full->attrs.secure;
+ }
+
+ if (!s2_phys) {
+ uint64_t hcr = arm_hcr_el2_eff_secstate(env, is_secure);
+
+ if ((hcr & HCR_PTW) && S2_attrs_are_device(hcr, pte_attrs)) {
/*
* PTW set and S1 walk touched S2 Device memory:
* generate Permission fault.
@@ -246,24 +278,25 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
fi->stage2 = true;
fi->s1ptw = true;
fi->s1ns = !is_secure;
- return ~0;
+ return false;
}
-
- if (arm_is_secure_below_el3(env)) {
- /* Check if page table walk is to secure or non-secure PA space. */
- if (is_secure) {
- is_secure = !(env->cp15.vstcr_el2 & VSTCR_SW);
- } else {
- is_secure = !(env->cp15.vtcr_el2 & VTCR_NSW);
- }
- *is_secure_ptr = is_secure;
- } else {
- assert(!is_secure);
- }
-
- addr = s2.f.phys_addr;
}
- return addr;
+
+ if (is_secure) {
+ /* Check if page table walk is to secure or non-secure PA space. */
+ *is_secure_ptr = !(pte_secure
+ ? env->cp15.vstcr_el2 & VSTCR_SW
+ : env->cp15.vtcr_el2 & VTCR_NSW);
+ }
+ return true;
+
+ fail:
+ assert(fi->type != ARMFault_None);
+ fi->s2addr = addr;
+ fi->stage2 = true;
+ fi->s1ptw = true;
+ fi->s1ns = !is_secure;
+ return false;
}
/* All loads done in the course of a page table walk go through here. */
@@ -271,56 +304,88 @@ static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
ARMMMUIdx mmu_idx, bool debug, ARMMMUFaultInfo *fi)
{
CPUState *cs = env_cpu(env);
- MemTxAttrs attrs = {};
- MemTxResult result = MEMTX_OK;
- AddressSpace *as;
+ void *hphys;
+ hwaddr gphys;
uint32_t data;
+ bool be;
- addr = S1_ptw_translate(env, mmu_idx, addr, &is_secure, debug, fi);
- attrs.secure = is_secure;
- as = arm_addressspace(cs, attrs);
- if (fi->s1ptw) {
+ if (!S1_ptw_translate(env, mmu_idx, addr, &is_secure,
+ &hphys, &gphys, debug, fi)) {
+ /* Failure. */
+ assert(fi->s1ptw);
return 0;
}
- if (regime_translation_big_endian(env, mmu_idx)) {
- data = address_space_ldl_be(as, addr, attrs, &result);
+
+ be = regime_translation_big_endian(env, mmu_idx);
+ if (likely(hphys)) {
+ /* Page tables are in RAM, and we have the host address. */
+ if (be) {
+ data = ldl_be_p(hphys);
+ } else {
+ data = ldl_le_p(hphys);
+ }
} else {
- data = address_space_ldl_le(as, addr, attrs, &result);
+ /* Page tables are in MMIO. */
+ MemTxAttrs attrs = { .secure = is_secure };
+ AddressSpace *as = arm_addressspace(cs, attrs);
+ MemTxResult result = MEMTX_OK;
+
+ if (be) {
+ data = address_space_ldl_be(as, gphys, attrs, &result);
+ } else {
+ data = address_space_ldl_le(as, gphys, attrs, &result);
+ }
+ if (unlikely(result != MEMTX_OK)) {
+ fi->type = ARMFault_SyncExternalOnWalk;
+ fi->ea = arm_extabort_type(result);
+ return 0;
+ }
}
- if (result == MEMTX_OK) {
- return data;
- }
- fi->type = ARMFault_SyncExternalOnWalk;
- fi->ea = arm_extabort_type(result);
- return 0;
+ return data;
}
static uint64_t arm_ldq_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
ARMMMUIdx mmu_idx, bool debug, ARMMMUFaultInfo *fi)
{
CPUState *cs = env_cpu(env);
- MemTxAttrs attrs = {};
- MemTxResult result = MEMTX_OK;
- AddressSpace *as;
+ void *hphys;
+ hwaddr gphys;
uint64_t data;
+ bool be;
- addr = S1_ptw_translate(env, mmu_idx, addr, &is_secure, debug, fi);
- attrs.secure = is_secure;
- as = arm_addressspace(cs, attrs);
- if (fi->s1ptw) {
+ if (!S1_ptw_translate(env, mmu_idx, addr, &is_secure,
+ &hphys, &gphys, debug, fi)) {
+ /* Failure. */
+ assert(fi->s1ptw);
return 0;
}
- if (regime_translation_big_endian(env, mmu_idx)) {
- data = address_space_ldq_be(as, addr, attrs, &result);
+
+ be = regime_translation_big_endian(env, mmu_idx);
+ if (likely(hphys)) {
+ /* Page tables are in RAM, and we have the host address. */
+ if (be) {
+ data = ldq_be_p(hphys);
+ } else {
+ data = ldq_le_p(hphys);
+ }
} else {
- data = address_space_ldq_le(as, addr, attrs, &result);
+ /* Page tables are in MMIO. */
+ MemTxAttrs attrs = { .secure = is_secure };
+ AddressSpace *as = arm_addressspace(cs, attrs);
+ MemTxResult result = MEMTX_OK;
+
+ if (be) {
+ data = address_space_ldq_be(as, gphys, attrs, &result);
+ } else {
+ data = address_space_ldq_le(as, gphys, attrs, &result);
+ }
+ if (unlikely(result != MEMTX_OK)) {
+ fi->type = ARMFault_SyncExternalOnWalk;
+ fi->ea = arm_extabort_type(result);
+ return 0;
+ }
}
- if (result == MEMTX_OK) {
- return data;
- }
- fi->type = ARMFault_SyncExternalOnWalk;
- fi->ea = arm_extabort_type(result);
- return 0;
+ return data;
}
static bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx,
diff --git a/target/arm/tlb_helper.c b/target/arm/tlb_helper.c
index 3462a6ea14..69b0dc69df 100644
--- a/target/arm/tlb_helper.c
+++ b/target/arm/tlb_helper.c
@@ -208,10 +208,21 @@ bool arm_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
bool probe, uintptr_t retaddr)
{
ARMCPU *cpu = ARM_CPU(cs);
- ARMMMUFaultInfo fi = {};
GetPhysAddrResult res = {};
+ ARMMMUFaultInfo local_fi, *fi;
int ret;
+ /*
+ * Allow S1_ptw_translate to see any fault generated here.
+ * Since this may recurse, read and clear.
+ */
+ fi = cpu->env.tlb_fi;
+ if (fi) {
+ cpu->env.tlb_fi = NULL;
+ } else {
+ fi = memset(&local_fi, 0, sizeof(local_fi));
+ }
+
/*
* Walk the page table and (if the mapping exists) add the page
* to the TLB. On success, return true. Otherwise, if probing,
@@ -220,7 +231,7 @@ bool arm_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
*/
ret = get_phys_addr(&cpu->env, address, access_type,
core_to_arm_mmu_idx(&cpu->env, mmu_idx),
- &res, &fi);
+ &res, fi);
if (likely(!ret)) {
/*
* Map a single [sub]page. Regions smaller than our declared
@@ -242,7 +253,7 @@ bool arm_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
} else {
/* now we have a real cpu fault */
cpu_restore_state(cs, retaddr, true);
- arm_deliver_fault(cpu, address, access_type, mmu_idx, &fi);
+ arm_deliver_fault(cpu, address, access_type, mmu_idx, fi);
}
}
#else
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 27/42] target/arm: Use softmmu tlbs for page table walking
2022-10-01 16:23 ` [PATCH v3 27/42] target/arm: Use softmmu tlbs for page table walking Richard Henderson
@ 2022-10-07 9:01 ` Peter Maydell
2022-10-07 15:27 ` Richard Henderson
0 siblings, 1 reply; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 9:01 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:52, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> So far, limit the change to S1_ptw_translate, arm_ldl_ptw, and
> arm_ldq_ptw. Use probe_access_full to find the host address,
> and if so use a host load. If the probe fails, we've got our
> fault info already. On the off chance that page tables are not
> in RAM, continue to use the address_space_ld* functions.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> -static bool ptw_attrs_are_device(uint64_t hcr, ARMCacheAttrs cacheattrs)
> +static bool S2_attrs_are_device(uint64_t hcr, uint8_t attrs)
> {
> /*
> * For an S1 page table walk, the stage 1 attributes are always
> @@ -202,41 +203,72 @@ static bool ptw_attrs_are_device(uint64_t hcr, ARMCacheAttrs cacheattrs)
> * With HCR_EL2.FWB == 1 this is when descriptor bit [4] is 0, ie
> * when cacheattrs.attrs bit [2] is 0.
> */
> - assert(cacheattrs.is_s2_format);
> if (hcr & HCR_FWB) {
> - return (cacheattrs.attrs & 0x4) == 0;
> + return (attrs & 0x4) == 0;
> } else {
> - return (cacheattrs.attrs & 0xc) == 0;
> + return (attrs & 0xc) == 0;
> }
The upcoming v8R support has its stage 2 attributes in the MAIR
format, so it might be a little awkward to assume the v8A-stage-2
format here rather than being able to add the "if !is_s2_format"
condition. I guess we'll deal with that when we get to it...
> + env->tlb_fi = fi;
> + flags = probe_access_full(env, addr, MMU_DATA_LOAD,
> + arm_to_core_mmu_idx(s2_mmu_idx),
> + true, hphys, &full, 0);
> + env->tlb_fi = NULL;
> +
> + if (unlikely(flags & TLB_INVALID_MASK)) {
> + goto fail;
> + }
> + *gphys = full->phys_addr;
> + pte_attrs = full->pte_attrs;
> + pte_secure = full->attrs.secure;
> + }
> +
> --- a/target/arm/tlb_helper.c
> +++ b/target/arm/tlb_helper.c
> @@ -208,10 +208,21 @@ bool arm_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
> bool probe, uintptr_t retaddr)
> {
> ARMCPU *cpu = ARM_CPU(cs);
> - ARMMMUFaultInfo fi = {};
> GetPhysAddrResult res = {};
> + ARMMMUFaultInfo local_fi, *fi;
> int ret;
>
> + /*
> + * Allow S1_ptw_translate to see any fault generated here.
> + * Since this may recurse, read and clear.
> + */
> + fi = cpu->env.tlb_fi;
> + if (fi) {
> + cpu->env.tlb_fi = NULL;
> + } else {
> + fi = memset(&local_fi, 0, sizeof(local_fi));
> + }
This makes two architectures now that want to do "call a probe_access
function, and get information that's known in the architecture-specific
tlb_fill function", and need to do it via this awkward "have tlb_fill
know that it should stash the info away in the CPU state struct somewhere"
trick (the other being s390 tlb_fill_exc/tlb_fill_tec). But I don't
really have a better idea, so
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread* Re: [PATCH v3 27/42] target/arm: Use softmmu tlbs for page table walking
2022-10-07 9:01 ` Peter Maydell
@ 2022-10-07 15:27 ` Richard Henderson
2022-10-07 16:08 ` Peter Maydell
0 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-07 15:27 UTC (permalink / raw)
To: Peter Maydell; +Cc: qemu-devel, qemu-arm
On 10/7/22 02:01, Peter Maydell wrote:
> The upcoming v8R support has its stage 2 attributes in the MAIR
> format, so it might be a little awkward to assume the v8A-stage-2
> format here rather than being able to add the "if !is_s2_format"
> condition. I guess we'll deal with that when we get to it...
Ah. I had wondered whether it would be better to convert the result here, so that we
always have the MAIR format. I decided against it within the scope of this patch set
because it meant that I kept the existing s1+s2 attribute merging logic unchanged.
>> + /*
>> + * Allow S1_ptw_translate to see any fault generated here.
>> + * Since this may recurse, read and clear.
>> + */
>> + fi = cpu->env.tlb_fi;
>> + if (fi) {
>> + cpu->env.tlb_fi = NULL;
>> + } else {
>> + fi = memset(&local_fi, 0, sizeof(local_fi));
>> + }
>
> This makes two architectures now that want to do "call a probe_access
> function, and get information that's known in the architecture-specific
> tlb_fill function", and need to do it via this awkward "have tlb_fill
> know that it should stash the info away in the CPU state struct somewhere"
> trick (the other being s390 tlb_fill_exc/tlb_fill_tec). But I don't
> really have a better idea...
A better idea would be most welcome, if anyone has one... :-)
r~
^ permalink raw reply [flat|nested] 90+ messages in thread* Re: [PATCH v3 27/42] target/arm: Use softmmu tlbs for page table walking
2022-10-07 15:27 ` Richard Henderson
@ 2022-10-07 16:08 ` Peter Maydell
0 siblings, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 16:08 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Fri, 7 Oct 2022 at 16:27, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 10/7/22 02:01, Peter Maydell wrote:
> > The upcoming v8R support has its stage 2 attributes in the MAIR
> > format, so it might be a little awkward to assume the v8A-stage-2
> > format here rather than being able to add the "if !is_s2_format"
> > condition. I guess we'll deal with that when we get to it...
>
> Ah. I had wondered whether it would be better to convert the result here, so that we
> always have the MAIR format. I decided against it within the scope of this patch set
> because it meant that I kept the existing s1+s2 attribute merging logic unchanged.
Unfortunately, for A-profile you can't just convert the s2 attrs
to MAIR format, because their interpretation depends on the
s1 attr values in the FWB case. So you have to keep the s2
attrs as raw until they get to the point of combination.
(v8R doesn't have any equivalent of FWB.)
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 28/42] target/arm: Split out get_phys_addr_twostage
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (26 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 27/42] target/arm: Use softmmu tlbs for page table walking Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-06 16:00 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 29/42] target/arm: Use bool consistently for get_phys_addr subroutines Richard Henderson
` (14 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 196 +++++++++++++++++++++++++----------------------
1 file changed, 106 insertions(+), 90 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index ba496c3421..3f5733a237 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -21,6 +21,15 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
__attribute__((nonnull));
+static bool get_phys_addr_with_secure_debug(CPUARMState *env,
+ target_ulong address,
+ MMUAccessType access_type,
+ ARMMMUIdx mmu_idx,
+ bool is_secure, bool debug,
+ GetPhysAddrResult *result,
+ ARMMMUFaultInfo *fi)
+ __attribute__((nonnull));
+
/* This mapping is common between ID_AA64MMFR0.PARANGE and TCR_ELx.{I}PS. */
static const uint8_t pamax_map[] = {
[0] = 32,
@@ -2426,6 +2435,98 @@ static bool get_phys_addr_disabled(CPUARMState *env, target_ulong address,
return 0;
}
+static bool get_phys_addr_twostage(CPUARMState *env, target_ulong address,
+ MMUAccessType access_type,
+ ARMMMUIdx s1_mmu_idx,
+ bool is_secure, bool debug,
+ GetPhysAddrResult *result,
+ ARMMMUFaultInfo *fi)
+{
+ hwaddr ipa;
+ int s1_prot;
+ int ret;
+ bool ipa_secure, s2walk_secure;
+ ARMCacheAttrs cacheattrs1;
+ ARMMMUIdx s2_mmu_idx;
+ bool is_el0;
+ uint64_t hcr;
+
+ ret = get_phys_addr_with_secure_debug(env, address, access_type,
+ s1_mmu_idx, is_secure, debug,
+ result, fi);
+
+ /* If S1 fails or S2 is disabled, return early. */
+ if (ret || regime_translation_disabled(env, ARMMMUIdx_Stage2, is_secure)) {
+ return ret;
+ }
+
+ ipa = result->f.phys_addr;
+ ipa_secure = result->f.attrs.secure;
+ if (is_secure) {
+ /* Select TCR based on the NS bit from the S1 walk. */
+ s2walk_secure = !(ipa_secure
+ ? env->cp15.vstcr_el2 & VSTCR_SW
+ : env->cp15.vtcr_el2 & VTCR_NSW);
+ } else {
+ assert(!ipa_secure);
+ s2walk_secure = false;
+ }
+
+ s2_mmu_idx = (s2walk_secure ? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2);
+ is_el0 = s1_mmu_idx == ARMMMUIdx_Stage1_E0;
+
+ /*
+ * S1 is done, now do S2 translation.
+ * Save the stage1 results so that we may merge prot and cacheattrs later.
+ */
+ s1_prot = result->f.prot;
+ cacheattrs1 = result->cacheattrs;
+ memset(result, 0, sizeof(*result));
+
+ ret = get_phys_addr_lpae(env, ipa, access_type, s2_mmu_idx,
+ s2walk_secure, is_el0, debug, result, fi);
+ fi->s2addr = ipa;
+
+ /* Combine the S1 and S2 perms. */
+ result->f.prot &= s1_prot;
+
+ /* If S2 fails, return early. */
+ if (ret) {
+ return ret;
+ }
+
+ /* Combine the S1 and S2 cache attributes. */
+ hcr = arm_hcr_el2_eff_secstate(env, is_secure);
+ if (hcr & HCR_DC) {
+ /*
+ * HCR.DC forces the first stage attributes to
+ * Normal Non-Shareable,
+ * Inner Write-Back Read-Allocate Write-Allocate,
+ * Outer Write-Back Read-Allocate Write-Allocate.
+ * Do not overwrite Tagged within attrs.
+ */
+ if (cacheattrs1.attrs != 0xf0) {
+ cacheattrs1.attrs = 0xff;
+ }
+ cacheattrs1.shareability = 0;
+ }
+ result->cacheattrs = combine_cacheattrs(hcr, cacheattrs1,
+ result->cacheattrs);
+
+ /* Check if IPA translates to secure or non-secure PA space. */
+ if (is_secure) {
+ if (ipa_secure) {
+ result->f.attrs.secure =
+ !(env->cp15.vstcr_el2 & (VSTCR_SA | VSTCR_SW));
+ } else {
+ result->f.attrs.secure =
+ !((env->cp15.vtcr_el2 & (VTCR_NSA | VTCR_NSW))
+ || (env->cp15.vstcr_el2 & (VSTCR_SA | VSTCR_SW)));
+ }
+ }
+ return 0;
+}
+
static bool get_phys_addr_with_secure_debug(CPUARMState *env,
target_ulong address,
MMUAccessType access_type,
@@ -2442,97 +2543,12 @@ static bool get_phys_addr_with_secure_debug(CPUARMState *env,
* translations if mmu_idx is a two-stage regime.
*/
if (arm_feature(env, ARM_FEATURE_EL2)) {
- hwaddr ipa;
- int s1_prot;
- int ret;
- bool ipa_secure, s2walk_secure;
- ARMCacheAttrs cacheattrs1;
- ARMMMUIdx s2_mmu_idx;
- bool is_el0;
- uint64_t hcr;
-
- ret = get_phys_addr_with_secure(env, address, access_type,
- s1_mmu_idx, is_secure, result, fi);
-
- /* If S1 fails or S2 is disabled, return early. */
- if (ret || regime_translation_disabled(env, ARMMMUIdx_Stage2,
- is_secure)) {
- return ret;
- }
-
- ipa = result->f.phys_addr;
- ipa_secure = result->f.attrs.secure;
- if (is_secure) {
- /* Select TCR based on the NS bit from the S1 walk. */
- s2walk_secure = !(ipa_secure
- ? env->cp15.vstcr_el2 & VSTCR_SW
- : env->cp15.vtcr_el2 & VTCR_NSW);
- } else {
- assert(!ipa_secure);
- s2walk_secure = false;
- }
-
- s2_mmu_idx = (s2walk_secure
- ? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2);
- is_el0 = mmu_idx == ARMMMUIdx_E10_0;
-
- /*
- * S1 is done, now do S2 translation.
- * Save the stage1 results so that we may merge
- * prot and cacheattrs later.
- */
- s1_prot = result->f.prot;
- cacheattrs1 = result->cacheattrs;
- memset(result, 0, sizeof(*result));
-
- ret = get_phys_addr_lpae(env, ipa, access_type, s2_mmu_idx,
- s2walk_secure, is_el0, debug, result, fi);
- fi->s2addr = ipa;
-
- /* Combine the S1 and S2 perms. */
- result->f.prot &= s1_prot;
-
- /* If S2 fails, return early. */
- if (ret) {
- return ret;
- }
-
- /* Combine the S1 and S2 cache attributes. */
- hcr = arm_hcr_el2_eff_secstate(env, is_secure);
- if (hcr & HCR_DC) {
- /*
- * HCR.DC forces the first stage attributes to
- * Normal Non-Shareable,
- * Inner Write-Back Read-Allocate Write-Allocate,
- * Outer Write-Back Read-Allocate Write-Allocate.
- * Do not overwrite Tagged within attrs.
- */
- if (cacheattrs1.attrs != 0xf0) {
- cacheattrs1.attrs = 0xff;
- }
- cacheattrs1.shareability = 0;
- }
- result->cacheattrs = combine_cacheattrs(hcr, cacheattrs1,
- result->cacheattrs);
-
- /* Check if IPA translates to secure or non-secure PA space. */
- if (is_secure) {
- if (ipa_secure) {
- result->f.attrs.secure =
- !(env->cp15.vstcr_el2 & (VSTCR_SA | VSTCR_SW));
- } else {
- result->f.attrs.secure =
- !((env->cp15.vtcr_el2 & (VTCR_NSA | VTCR_NSW))
- || (env->cp15.vstcr_el2 & (VSTCR_SA | VSTCR_SW)));
- }
- }
- return 0;
- } else {
- /*
- * For non-EL2 CPUs a stage1+stage2 translation is just stage 1.
- */
- mmu_idx = stage_1_mmu_idx(mmu_idx);
+ return get_phys_addr_twostage(env, address, access_type,
+ s1_mmu_idx, is_secure, debug,
+ result, fi);
}
+ /* For non-EL2 CPUs a stage1+stage2 translation is just stage 1. */
+ mmu_idx = s1_mmu_idx;
}
/*
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 29/42] target/arm: Use bool consistently for get_phys_addr subroutines
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (27 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 28/42] target/arm: Split out get_phys_addr_twostage Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-06 16:01 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 30/42] target/arm: Add ptw_idx argument to S1_ptw_translate Richard Henderson
` (13 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
The return type of the functions is already bool, but in a few
instances we used an integer type with the return statement.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 3f5733a237..445382ab03 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -2432,7 +2432,7 @@ static bool get_phys_addr_disabled(CPUARMState *env, target_ulong address,
result->f.lg_page_size = TARGET_PAGE_BITS;
result->cacheattrs.shareability = shareability;
result->cacheattrs.attrs = memattr;
- return 0;
+ return false;
}
static bool get_phys_addr_twostage(CPUARMState *env, target_ulong address,
@@ -2444,8 +2444,7 @@ static bool get_phys_addr_twostage(CPUARMState *env, target_ulong address,
{
hwaddr ipa;
int s1_prot;
- int ret;
- bool ipa_secure, s2walk_secure;
+ bool ret, ipa_secure, s2walk_secure;
ARMCacheAttrs cacheattrs1;
ARMMMUIdx s2_mmu_idx;
bool is_el0;
@@ -2524,7 +2523,7 @@ static bool get_phys_addr_twostage(CPUARMState *env, target_ulong address,
|| (env->cp15.vstcr_el2 & (VSTCR_SA | VSTCR_SW)));
}
}
- return 0;
+ return false;
}
static bool get_phys_addr_with_secure_debug(CPUARMState *env,
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 30/42] target/arm: Add ptw_idx argument to S1_ptw_translate
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (28 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 29/42] target/arm: Use bool consistently for get_phys_addr subroutines Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 9:19 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 31/42] target/arm: Add isar predicates for FEAT_HAFDBS Richard Henderson
` (12 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Hoist the computation of the mmu_idx for the ptw up to
get_phys_addr_with_secure_debug and get_phys_addr_twostage.
This removes the duplicate check for stage2 disabled
from the middle of the walk, performing it only once.
Pass ptw_idx through get_phys_addr_{v5,v6,lpae} and arm_{ldl,ldq}_ptw.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 104 ++++++++++++++++++++++++++++++++---------------
1 file changed, 71 insertions(+), 33 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 445382ab03..7a77bea2c7 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -17,7 +17,8 @@
static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
- bool is_secure, bool s1_is_el0, bool debug,
+ ARMMMUIdx ptw_idx, bool is_secure,
+ bool s1_is_el0, bool debug,
GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
__attribute__((nonnull));
@@ -220,21 +221,16 @@ static bool S2_attrs_are_device(uint64_t hcr, uint8_t attrs)
}
/* Translate a S1 pagetable walk through S2 if needed. */
-static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx, hwaddr addr,
+static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
+ ARMMMUIdx s2_mmu_idx, hwaddr addr,
bool *is_secure_ptr, void **hphys, hwaddr *gphys,
bool debug, ARMMMUFaultInfo *fi)
{
bool is_secure = *is_secure_ptr;
- ARMMMUIdx s2_mmu_idx = is_secure ? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2;
- bool s2_phys = false;
uint8_t pte_attrs;
- bool pte_secure;
+ bool s2_phys, pte_secure;
- if (!arm_mmu_idx_is_stage1_of_2(mmu_idx)
- || regime_translation_disabled(env, s2_mmu_idx, is_secure)) {
- s2_mmu_idx = is_secure ? ARMMMUIdx_Phys_S : ARMMMUIdx_Phys_NS;
- s2_phys = true;
- }
+ s2_phys = s2_mmu_idx == ARMMMUIdx_Phys_S || s2_mmu_idx == ARMMMUIdx_Phys_NS;
if (unlikely(debug)) {
/*
@@ -247,8 +243,12 @@ static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx, hwaddr addr,
pte_secure = is_secure;
} else {
GetPhysAddrResult s2 = { };
+ ARMMMUIdx phys_idx = (is_secure ? ARMMMUIdx_Phys_S
+ : ARMMMUIdx_Phys_NS);
+
if (!get_phys_addr_lpae(env, addr, MMU_DATA_LOAD, s2_mmu_idx,
- is_secure, false, debug, &s2, fi)) {
+ phys_idx, is_secure, false, debug,
+ &s2, fi)) {
goto fail;
}
*gphys = s2.f.phys_addr;
@@ -310,7 +310,8 @@ static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx, hwaddr addr,
/* All loads done in the course of a page table walk go through here. */
static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
- ARMMMUIdx mmu_idx, bool debug, ARMMMUFaultInfo *fi)
+ ARMMMUIdx mmu_idx, ARMMMUIdx ptw_idx,
+ bool debug, ARMMMUFaultInfo *fi)
{
CPUState *cs = env_cpu(env);
void *hphys;
@@ -318,7 +319,7 @@ static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
uint32_t data;
bool be;
- if (!S1_ptw_translate(env, mmu_idx, addr, &is_secure,
+ if (!S1_ptw_translate(env, mmu_idx, ptw_idx, addr, &is_secure,
&hphys, &gphys, debug, fi)) {
/* Failure. */
assert(fi->s1ptw);
@@ -354,7 +355,8 @@ static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
}
static uint64_t arm_ldq_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
- ARMMMUIdx mmu_idx, bool debug, ARMMMUFaultInfo *fi)
+ ARMMMUIdx mmu_idx, ARMMMUIdx ptw_idx,
+ bool debug, ARMMMUFaultInfo *fi)
{
CPUState *cs = env_cpu(env);
void *hphys;
@@ -362,7 +364,7 @@ static uint64_t arm_ldq_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
uint64_t data;
bool be;
- if (!S1_ptw_translate(env, mmu_idx, addr, &is_secure,
+ if (!S1_ptw_translate(env, mmu_idx, ptw_idx, addr, &is_secure,
&hphys, &gphys, debug, fi)) {
/* Failure. */
assert(fi->s1ptw);
@@ -507,7 +509,7 @@ static int simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
- bool is_secure, bool debug,
+ ARMMMUIdx ptw_idx, bool is_secure, bool debug,
GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
{
int level = 1;
@@ -527,7 +529,7 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
fi->type = ARMFault_Translation;
goto do_fault;
}
- desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, debug, fi);
+ desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, ptw_idx, debug, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -565,7 +567,7 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
/* Fine pagetable. */
table = (desc & 0xfffff000) | ((address >> 8) & 0xffc);
}
- desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, debug, fi);
+ desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, ptw_idx, debug, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -626,7 +628,7 @@ do_fault:
static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
- bool is_secure, bool debug,
+ ARMMMUIdx ptw_idx, bool is_secure, bool debug,
GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
{
ARMCPU *cpu = env_archcpu(env);
@@ -650,7 +652,7 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
fi->type = ARMFault_Translation;
goto do_fault;
}
- desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, debug, fi);
+ desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, ptw_idx, debug, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -703,7 +705,7 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
ns = extract32(desc, 3, 1);
/* Lookup l2 entry. */
table = (desc & 0xfffffc00) | ((address >> 10) & 0x3fc);
- desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, debug, fi);
+ desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, ptw_idx, debug, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -1058,7 +1060,8 @@ static bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level,
*/
static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
MMUAccessType access_type, ARMMMUIdx mmu_idx,
- bool is_secure, bool s1_is_el0, bool debug,
+ ARMMMUIdx ptw_idx, bool is_secure,
+ bool s1_is_el0, bool debug,
GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
{
ARMCPU *cpu = env_archcpu(env);
@@ -1284,7 +1287,8 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
descaddr |= (address >> (stride * (4 - level))) & indexmask;
descaddr &= ~7ULL;
nstable = extract32(tableattrs, 4, 1);
- descriptor = arm_ldq_ptw(env, descaddr, !nstable, mmu_idx, debug, fi);
+ descriptor = arm_ldq_ptw(env, descaddr, !nstable, mmu_idx,
+ ptw_idx, debug, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -2446,7 +2450,7 @@ static bool get_phys_addr_twostage(CPUARMState *env, target_ulong address,
int s1_prot;
bool ret, ipa_secure, s2walk_secure;
ARMCacheAttrs cacheattrs1;
- ARMMMUIdx s2_mmu_idx;
+ ARMMMUIdx s2_mmu_idx, s2_ptw_idx;
bool is_el0;
uint64_t hcr;
@@ -2471,7 +2475,13 @@ static bool get_phys_addr_twostage(CPUARMState *env, target_ulong address,
s2walk_secure = false;
}
- s2_mmu_idx = (s2walk_secure ? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2);
+ if (s2walk_secure) {
+ s2_mmu_idx = ARMMMUIdx_Stage2_S;
+ s2_ptw_idx = ARMMMUIdx_Phys_S;
+ } else {
+ s2_mmu_idx = ARMMMUIdx_Stage2;
+ s2_ptw_idx = ARMMMUIdx_Phys_NS;
+ }
is_el0 = s1_mmu_idx == ARMMMUIdx_Stage1_E0;
/*
@@ -2482,7 +2492,7 @@ static bool get_phys_addr_twostage(CPUARMState *env, target_ulong address,
cacheattrs1 = result->cacheattrs;
memset(result, 0, sizeof(*result));
- ret = get_phys_addr_lpae(env, ipa, access_type, s2_mmu_idx,
+ ret = get_phys_addr_lpae(env, ipa, access_type, s2_mmu_idx, s2_ptw_idx,
s2walk_secure, is_el0, debug, result, fi);
fi->s2addr = ipa;
@@ -2534,9 +2544,32 @@ static bool get_phys_addr_with_secure_debug(CPUARMState *env,
GetPhysAddrResult *result,
ARMMMUFaultInfo *fi)
{
- ARMMMUIdx s1_mmu_idx = stage_1_mmu_idx(mmu_idx);
+ ARMMMUIdx s1_mmu_idx, ptw_idx;
- if (mmu_idx != s1_mmu_idx) {
+ switch (mmu_idx) {
+ case ARMMMUIdx_Phys_S:
+ case ARMMMUIdx_Phys_NS:
+ do_disabled:
+ /* Checking Phys early avoids special casing later vs regime_el. */
+ return get_phys_addr_disabled(env, address, access_type, mmu_idx,
+ is_secure, result, fi);
+
+ case ARMMMUIdx_Stage1_E0:
+ case ARMMMUIdx_Stage1_E1:
+ case ARMMMUIdx_Stage1_E1_PAN:
+ /* First stage lookup uses second stage for ptw. */
+ ptw_idx = is_secure ? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2;
+ break;
+
+ case ARMMMUIdx_E10_0:
+ s1_mmu_idx = ARMMMUIdx_Stage1_E0;
+ goto do_twostage;
+ case ARMMMUIdx_E10_1:
+ s1_mmu_idx = ARMMMUIdx_Stage1_E1;
+ goto do_twostage;
+ case ARMMMUIdx_E10_1_PAN:
+ s1_mmu_idx = ARMMMUIdx_Stage1_E1_PAN;
+ do_twostage:
/*
* Call ourselves recursively to do the stage 1 and then stage 2
* translations if mmu_idx is a two-stage regime.
@@ -2548,6 +2581,12 @@ static bool get_phys_addr_with_secure_debug(CPUARMState *env,
}
/* For non-EL2 CPUs a stage1+stage2 translation is just stage 1. */
mmu_idx = s1_mmu_idx;
+ /* fall through */
+
+ default:
+ /* Single stage and second stage uses physical for ptw. */
+ ptw_idx = is_secure ? ARMMMUIdx_Phys_S : ARMMMUIdx_Phys_NS;
+ break;
}
/*
@@ -2604,18 +2643,17 @@ static bool get_phys_addr_with_secure_debug(CPUARMState *env,
/* Definitely a real MMU, not an MPU */
if (regime_translation_disabled(env, mmu_idx, is_secure)) {
- return get_phys_addr_disabled(env, address, access_type, mmu_idx,
- is_secure, result, fi);
+ goto do_disabled;
}
if (regime_using_lpae_format(env, mmu_idx)) {
return get_phys_addr_lpae(env, address, access_type, mmu_idx,
- is_secure, false, debug, result, fi);
+ ptw_idx, is_secure, false, debug, result, fi);
} else if (regime_sctlr(env, mmu_idx) & SCTLR_XP) {
return get_phys_addr_v6(env, address, access_type, mmu_idx,
- is_secure, debug, result, fi);
+ ptw_idx, is_secure, debug, result, fi);
} else {
return get_phys_addr_v5(env, address, access_type, mmu_idx,
- is_secure, debug, result, fi);
+ ptw_idx, is_secure, debug, result, fi);
}
}
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 30/42] target/arm: Add ptw_idx argument to S1_ptw_translate
2022-10-01 16:23 ` [PATCH v3 30/42] target/arm: Add ptw_idx argument to S1_ptw_translate Richard Henderson
@ 2022-10-07 9:19 ` Peter Maydell
2022-10-07 15:34 ` Richard Henderson
0 siblings, 1 reply; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 9:19 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:42, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Hoist the computation of the mmu_idx for the ptw up to
> get_phys_addr_with_secure_debug and get_phys_addr_twostage.
> This removes the duplicate check for stage2 disabled
> from the middle of the walk, performing it only once.
>
> Pass ptw_idx through get_phys_addr_{v5,v6,lpae} and arm_{ldl,ldq}_ptw.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> /* Translate a S1 pagetable walk through S2 if needed. */
> -static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx, hwaddr addr,
> +static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
> + ARMMMUIdx s2_mmu_idx, hwaddr addr,
> bool *is_secure_ptr, void **hphys, hwaddr *gphys,
> bool debug, ARMMMUFaultInfo *fi)
> {
> bool is_secure = *is_secure_ptr;
> - ARMMMUIdx s2_mmu_idx = is_secure ? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2;
> - bool s2_phys = false;
I don't think this works, because the s2_mmu_idx is not necessarily
the same through the whole of a page table walk. See the comment in
get_phys_addr_lpae():
/*
* Secure accesses start with the page table in secure memory and
* can be downgraded to non-secure at any step. Non-secure accesses
* remain non-secure. We implement this by just ORing in the NSTable/NS
* bits at each step.
*/
Currently get_phys_addr_lpae() updates the nstable bit in tableattrs and
passes that to arm_ldq_ptw() for each level of the page tables, which in
turn causes S1_ptw_translate() to select ARMMMUIdx_Stage2_S or ARMMMUIdx_Stage2.
Alternatively, maybe our existing behaviour is a bug -- but then we need
to separate out the bug fix from the refactoring patch.
> @@ -2604,18 +2643,17 @@ static bool get_phys_addr_with_secure_debug(CPUARMState *env,
> /* Definitely a real MMU, not an MPU */
>
> if (regime_translation_disabled(env, mmu_idx, is_secure)) {
> - return get_phys_addr_disabled(env, address, access_type, mmu_idx,
> - is_secure, result, fi);
> + goto do_disabled;
> }
I'd prefer to avoid this goto back up into the middle of an unrelated
switch statement.
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread* Re: [PATCH v3 30/42] target/arm: Add ptw_idx argument to S1_ptw_translate
2022-10-07 9:19 ` Peter Maydell
@ 2022-10-07 15:34 ` Richard Henderson
0 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-07 15:34 UTC (permalink / raw)
To: Peter Maydell; +Cc: qemu-devel, qemu-arm
On 10/7/22 02:19, Peter Maydell wrote:
> I don't think this works, because the s2_mmu_idx is not necessarily
> the same through the whole of a page table walk. See the comment in
> get_phys_addr_lpae():
> /*
> * Secure accesses start with the page table in secure memory and
> * can be downgraded to non-secure at any step. Non-secure accesses
> * remain non-secure. We implement this by just ORing in the NSTable/NS
> * bits at each step.
> */
>
> Currently get_phys_addr_lpae() updates the nstable bit in tableattrs and
> passes that to arm_ldq_ptw() for each level of the page tables, which in
> turn causes S1_ptw_translate() to select ARMMMUIdx_Stage2_S or ARMMMUIdx_Stage2.
Ouch. I had missed this subtlety.
We could play lsb games with the mmu_idx itself, knowing that we have either
ARMMMUIdx_{Stage2,Phys}_S and generate ARMMMUIdx_{Stage2,Phys}. I'll have another good
long look at this.
>> if (regime_translation_disabled(env, mmu_idx, is_secure)) {
>> - return get_phys_addr_disabled(env, address, access_type, mmu_idx,
>> - is_secure, result, fi);
>> + goto do_disabled;
>> }
>
> I'd prefer to avoid this goto back up into the middle of an unrelated
> switch statement.
Oops, I guess I missed this one when I went back through to eliminate the gotos.
r~
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 31/42] target/arm: Add isar predicates for FEAT_HAFDBS
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (29 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 30/42] target/arm: Add ptw_idx argument to S1_ptw_translate Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 9:21 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 32/42] target/arm: Extract HA and HD in aa64_va_parameters Richard Henderson
` (11 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
The MMFR1 field may indicate support for hardware update of
access flag alone, or access flag and dirty bit.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/cpu.h | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 7108568685..e499a84850 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -4100,6 +4100,16 @@ static inline bool isar_feature_aa64_lva(const ARMISARegisters *id)
return FIELD_EX64(id->id_aa64mmfr2, ID_AA64MMFR2, VARANGE) != 0;
}
+static inline bool isar_feature_aa64_hafs(const ARMISARegisters *id)
+{
+ return FIELD_EX64(id->id_aa64mmfr1, ID_AA64MMFR1, HAFDBS) != 0;
+}
+
+static inline bool isar_feature_aa64_hdbs(const ARMISARegisters *id)
+{
+ return FIELD_EX64(id->id_aa64mmfr1, ID_AA64MMFR1, HAFDBS) >= 2;
+}
+
static inline bool isar_feature_aa64_tts2uxn(const ARMISARegisters *id)
{
return FIELD_EX64(id->id_aa64mmfr1, ID_AA64MMFR1, XNX) != 0;
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 32/42] target/arm: Extract HA and HD in aa64_va_parameters
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (30 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 31/42] target/arm: Add isar predicates for FEAT_HAFDBS Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 9:24 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 33/42] target/arm: Split out S1TranslateResult type Richard Henderson
` (10 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/internals.h | 2 ++
target/arm/helper.c | 8 +++++++-
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/target/arm/internals.h b/target/arm/internals.h
index a50189e2e4..e95b6b1b8f 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -1014,6 +1014,8 @@ typedef struct ARMVAParameters {
bool using64k : 1;
bool tsz_oob : 1; /* tsz has been clamped to legal range */
bool ds : 1;
+ bool ha : 1;
+ bool hd : 1;
} ARMVAParameters;
ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 19a03eb200..70ae3816b9 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10280,7 +10280,7 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
ARMMMUIdx mmu_idx, bool data)
{
uint64_t tcr = regime_tcr(env, mmu_idx);
- bool epd, hpd, using16k, using64k, tsz_oob, ds;
+ bool epd, hpd, using16k, using64k, tsz_oob, ds, ha, hd;
int select, tsz, tbi, max_tsz, min_tsz, ps, sh;
ARMCPU *cpu = env_archcpu(env);
@@ -10298,6 +10298,8 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
epd = false;
sh = extract32(tcr, 12, 2);
ps = extract32(tcr, 16, 3);
+ ha = extract32(tcr, 21, 1) && cpu_isar_feature(aa64_hafs, cpu);
+ hd = extract32(tcr, 22, 1) && cpu_isar_feature(aa64_hdbs, cpu);
ds = extract64(tcr, 32, 1);
} else {
/*
@@ -10322,6 +10324,8 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
hpd = extract64(tcr, 42, 1);
}
ps = extract64(tcr, 32, 3);
+ ha = extract64(tcr, 39, 1) && cpu_isar_feature(aa64_hafs, cpu);
+ hd = extract64(tcr, 40, 1) && cpu_isar_feature(aa64_hdbs, cpu);
ds = extract64(tcr, 59, 1);
}
@@ -10393,6 +10397,8 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
.using64k = using64k,
.tsz_oob = tsz_oob,
.ds = ds,
+ .ha = ha,
+ .hd = ha & hd,
};
}
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 32/42] target/arm: Extract HA and HD in aa64_va_parameters
2022-10-01 16:23 ` [PATCH v3 32/42] target/arm: Extract HA and HD in aa64_va_parameters Richard Henderson
@ 2022-10-07 9:24 ` Peter Maydell
2022-10-07 15:37 ` Richard Henderson
0 siblings, 1 reply; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 9:24 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:42, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> target/arm/internals.h | 2 ++
> target/arm/helper.c | 8 +++++++-
> 2 files changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/target/arm/internals.h b/target/arm/internals.h
> index a50189e2e4..e95b6b1b8f 100644
> --- a/target/arm/internals.h
> +++ b/target/arm/internals.h
> @@ -1014,6 +1014,8 @@ typedef struct ARMVAParameters {
> bool using64k : 1;
> bool tsz_oob : 1; /* tsz has been clamped to legal range */
> bool ds : 1;
> + bool ha : 1;
> + bool hd : 1;
> } ARMVAParameters;
>
> ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
> diff --git a/target/arm/helper.c b/target/arm/helper.c
> index 19a03eb200..70ae3816b9 100644
> --- a/target/arm/helper.c
> +++ b/target/arm/helper.c
> @@ -10280,7 +10280,7 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
> ARMMMUIdx mmu_idx, bool data)
> {
> uint64_t tcr = regime_tcr(env, mmu_idx);
> - bool epd, hpd, using16k, using64k, tsz_oob, ds;
> + bool epd, hpd, using16k, using64k, tsz_oob, ds, ha, hd;
> int select, tsz, tbi, max_tsz, min_tsz, ps, sh;
> ARMCPU *cpu = env_archcpu(env);
>
> @@ -10298,6 +10298,8 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
> epd = false;
> sh = extract32(tcr, 12, 2);
> ps = extract32(tcr, 16, 3);
> + ha = extract32(tcr, 21, 1) && cpu_isar_feature(aa64_hafs, cpu);
> + hd = extract32(tcr, 22, 1) && cpu_isar_feature(aa64_hdbs, cpu);
> ds = extract64(tcr, 32, 1);
> } else {
> /*
> @@ -10322,6 +10324,8 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
> hpd = extract64(tcr, 42, 1);
> }
> ps = extract64(tcr, 32, 3);
> + ha = extract64(tcr, 39, 1) && cpu_isar_feature(aa64_hafs, cpu);
> + hd = extract64(tcr, 40, 1) && cpu_isar_feature(aa64_hdbs, cpu);
> ds = extract64(tcr, 59, 1);
> }
>
> @@ -10393,6 +10397,8 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
> .using64k = using64k,
> .tsz_oob = tsz_oob,
> .ds = ds,
> + .ha = ha,
> + .hd = ha & hd,
This is a bitwise operation on two bools, should be && ?
Otherwise
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread* Re: [PATCH v3 32/42] target/arm: Extract HA and HD in aa64_va_parameters
2022-10-07 9:24 ` Peter Maydell
@ 2022-10-07 15:37 ` Richard Henderson
2022-10-07 16:11 ` Peter Maydell
0 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-07 15:37 UTC (permalink / raw)
To: Peter Maydell; +Cc: qemu-devel, qemu-arm
On 10/7/22 02:24, Peter Maydell wrote:
>> + .ha = ha,
>> + .hd = ha & hd,
>
> This is a bitwise operation on two bools, should be && ?
Bitwise works fine, but I can use boolean if you like.
I'd be surprised (and filing a missed optimization bug) if the compiler treated these two
operations differently in this case (simple bool operands with no side effects).
r~
^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v3 32/42] target/arm: Extract HA and HD in aa64_va_parameters
2022-10-07 15:37 ` Richard Henderson
@ 2022-10-07 16:11 ` Peter Maydell
2022-10-07 16:13 ` Richard Henderson
0 siblings, 1 reply; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 16:11 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Fri, 7 Oct 2022 at 16:37, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 10/7/22 02:24, Peter Maydell wrote:
> >> + .ha = ha,
> >> + .hd = ha & hd,
> >
> > This is a bitwise operation on two bools, should be && ?
>
> Bitwise works fine, but I can use boolean if you like.
>
> I'd be surprised (and filing a missed optimization bug) if the compiler treated these two
> operations differently in this case (simple bool operands with no side effects).
The different treatment I would expect would be that in the '&'
case it warns you about using a bitwise operation on a boolean :-)
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v3 32/42] target/arm: Extract HA and HD in aa64_va_parameters
2022-10-07 16:11 ` Peter Maydell
@ 2022-10-07 16:13 ` Richard Henderson
2022-10-07 16:23 ` Peter Maydell
0 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-07 16:13 UTC (permalink / raw)
To: Peter Maydell; +Cc: qemu-devel, qemu-arm
On 10/7/22 09:11, Peter Maydell wrote:
> On Fri, 7 Oct 2022 at 16:37, Richard Henderson
> <richard.henderson@linaro.org> wrote:
>>
>> On 10/7/22 02:24, Peter Maydell wrote:
>>>> + .ha = ha,
>>>> + .hd = ha & hd,
>>>
>>> This is a bitwise operation on two bools, should be && ?
>>
>> Bitwise works fine, but I can use boolean if you like.
>>
>> I'd be surprised (and filing a missed optimization bug) if the compiler treated these two
>> operations differently in this case (simple bool operands with no side effects).
>
> The different treatment I would expect would be that in the '&'
> case it warns you about using a bitwise operation on a boolean :-)
Oh, well, no compiler should ever do that, because bool implicitly converts to int for any
arithmetic, just like char.
r~
^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v3 32/42] target/arm: Extract HA and HD in aa64_va_parameters
2022-10-07 16:13 ` Richard Henderson
@ 2022-10-07 16:23 ` Peter Maydell
0 siblings, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 16:23 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Fri, 7 Oct 2022 at 17:13, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 10/7/22 09:11, Peter Maydell wrote:
> > On Fri, 7 Oct 2022 at 16:37, Richard Henderson
> > <richard.henderson@linaro.org> wrote:
> >>
> >> On 10/7/22 02:24, Peter Maydell wrote:
> >>>> + .ha = ha,
> >>>> + .hd = ha & hd,
> >>>
> >>> This is a bitwise operation on two bools, should be && ?
> >>
> >> Bitwise works fine, but I can use boolean if you like.
> >>
> >> I'd be surprised (and filing a missed optimization bug) if the compiler treated these two
> >> operations differently in this case (simple bool operands with no side effects).
> >
> > The different treatment I would expect would be that in the '&'
> > case it warns you about using a bitwise operation on a boolean :-)
>
> Oh, well, no compiler should ever do that, because bool implicitly converts to int for any
> arithmetic, just like char.
Yeah, but -Wbool-operation is there to catch bugs where the bitwise
operation was unintended and the wrong behaviour.
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 33/42] target/arm: Split out S1TranslateResult type
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (31 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 32/42] target/arm: Extract HA and HD in aa64_va_parameters Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 9:27 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 34/42] target/arm: Move be test for regime into S1TranslateResult Richard Henderson
` (9 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Consolidate the results of S1_ptw_translate in one struct.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 70 +++++++++++++++++++++++++-----------------------
1 file changed, 36 insertions(+), 34 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 7a77bea2c7..99ad894180 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -220,13 +220,18 @@ static bool S2_attrs_are_device(uint64_t hcr, uint8_t attrs)
}
}
+typedef struct {
+ bool is_secure;
+ void *hphys;
+ hwaddr gphys;
+} S1TranslateResult;
+
/* Translate a S1 pagetable walk through S2 if needed. */
static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
ARMMMUIdx s2_mmu_idx, hwaddr addr,
- bool *is_secure_ptr, void **hphys, hwaddr *gphys,
- bool debug, ARMMMUFaultInfo *fi)
+ bool is_secure, bool debug,
+ S1TranslateResult *res, ARMMMUFaultInfo *fi)
{
- bool is_secure = *is_secure_ptr;
uint8_t pte_attrs;
bool s2_phys, pte_secure;
@@ -238,7 +243,7 @@ static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
* state of the cpu at all, including softmmu tlb contents.
*/
if (s2_phys) {
- *gphys = addr;
+ res->gphys = addr;
pte_attrs = 0;
pte_secure = is_secure;
} else {
@@ -251,11 +256,11 @@ static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
&s2, fi)) {
goto fail;
}
- *gphys = s2.f.phys_addr;
+ res->gphys = s2.f.phys_addr;
pte_attrs = s2.cacheattrs.attrs;
pte_secure = s2.f.attrs.secure;
}
- *hphys = NULL;
+ res->hphys = NULL;
} else {
CPUTLBEntryFull *full;
int flags;
@@ -263,13 +268,13 @@ static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
env->tlb_fi = fi;
flags = probe_access_full(env, addr, MMU_DATA_LOAD,
arm_to_core_mmu_idx(s2_mmu_idx),
- true, hphys, &full, 0);
+ true, &res->hphys, &full, 0);
env->tlb_fi = NULL;
if (unlikely(flags & TLB_INVALID_MASK)) {
goto fail;
}
- *gphys = full->phys_addr;
+ res->gphys = full->phys_addr;
pte_attrs = full->pte_attrs;
pte_secure = full->attrs.secure;
}
@@ -291,12 +296,11 @@ static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
}
}
- if (is_secure) {
- /* Check if page table walk is to secure or non-secure PA space. */
- *is_secure_ptr = !(pte_secure
- ? env->cp15.vstcr_el2 & VSTCR_SW
- : env->cp15.vtcr_el2 & VTCR_NSW);
- }
+ /* Check if page table walk is to secure or non-secure PA space. */
+ res->is_secure = (is_secure &&
+ !(pte_secure
+ ? env->cp15.vstcr_el2 & VSTCR_SW
+ : env->cp15.vtcr_el2 & VTCR_NSW));
return true;
fail:
@@ -314,36 +318,35 @@ static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
bool debug, ARMMMUFaultInfo *fi)
{
CPUState *cs = env_cpu(env);
- void *hphys;
- hwaddr gphys;
+ S1TranslateResult s1;
uint32_t data;
bool be;
- if (!S1_ptw_translate(env, mmu_idx, ptw_idx, addr, &is_secure,
- &hphys, &gphys, debug, fi)) {
+ if (!S1_ptw_translate(env, mmu_idx, ptw_idx, addr, is_secure,
+ debug, &s1, fi)) {
/* Failure. */
assert(fi->s1ptw);
return 0;
}
be = regime_translation_big_endian(env, mmu_idx);
- if (likely(hphys)) {
+ if (likely(s1.hphys)) {
/* Page tables are in RAM, and we have the host address. */
if (be) {
- data = ldl_be_p(hphys);
+ data = ldl_be_p(s1.hphys);
} else {
- data = ldl_le_p(hphys);
+ data = ldl_le_p(s1.hphys);
}
} else {
/* Page tables are in MMIO. */
- MemTxAttrs attrs = { .secure = is_secure };
+ MemTxAttrs attrs = { .secure = s1.is_secure };
AddressSpace *as = arm_addressspace(cs, attrs);
MemTxResult result = MEMTX_OK;
if (be) {
- data = address_space_ldl_be(as, gphys, attrs, &result);
+ data = address_space_ldl_be(as, s1.gphys, attrs, &result);
} else {
- data = address_space_ldl_le(as, gphys, attrs, &result);
+ data = address_space_ldl_le(as, s1.gphys, attrs, &result);
}
if (unlikely(result != MEMTX_OK)) {
fi->type = ARMFault_SyncExternalOnWalk;
@@ -359,36 +362,35 @@ static uint64_t arm_ldq_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
bool debug, ARMMMUFaultInfo *fi)
{
CPUState *cs = env_cpu(env);
- void *hphys;
- hwaddr gphys;
+ S1TranslateResult s1;
uint64_t data;
bool be;
- if (!S1_ptw_translate(env, mmu_idx, ptw_idx, addr, &is_secure,
- &hphys, &gphys, debug, fi)) {
+ if (!S1_ptw_translate(env, mmu_idx, ptw_idx, addr, is_secure,
+ debug, &s1, fi)) {
/* Failure. */
assert(fi->s1ptw);
return 0;
}
be = regime_translation_big_endian(env, mmu_idx);
- if (likely(hphys)) {
+ if (likely(s1.hphys)) {
/* Page tables are in RAM, and we have the host address. */
if (be) {
- data = ldq_be_p(hphys);
+ data = ldq_be_p(s1.hphys);
} else {
- data = ldq_le_p(hphys);
+ data = ldq_le_p(s1.hphys);
}
} else {
/* Page tables are in MMIO. */
- MemTxAttrs attrs = { .secure = is_secure };
+ MemTxAttrs attrs = { .secure = s1.is_secure };
AddressSpace *as = arm_addressspace(cs, attrs);
MemTxResult result = MEMTX_OK;
if (be) {
- data = address_space_ldq_be(as, gphys, attrs, &result);
+ data = address_space_ldq_be(as, s1.gphys, attrs, &result);
} else {
- data = address_space_ldq_le(as, gphys, attrs, &result);
+ data = address_space_ldq_le(as, s1.gphys, attrs, &result);
}
if (unlikely(result != MEMTX_OK)) {
fi->type = ARMFault_SyncExternalOnWalk;
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 33/42] target/arm: Split out S1TranslateResult type
2022-10-01 16:23 ` [PATCH v3 33/42] target/arm: Split out S1TranslateResult type Richard Henderson
@ 2022-10-07 9:27 ` Peter Maydell
0 siblings, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 9:27 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:56, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Consolidate the results of S1_ptw_translate in one struct.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> target/arm/ptw.c | 70 +++++++++++++++++++++++++-----------------------
> 1 file changed, 36 insertions(+), 34 deletions(-)
>
> diff --git a/target/arm/ptw.c b/target/arm/ptw.c
> index 7a77bea2c7..99ad894180 100644
> --- a/target/arm/ptw.c
> +++ b/target/arm/ptw.c
> @@ -220,13 +220,18 @@ static bool S2_attrs_are_device(uint64_t hcr, uint8_t attrs)
> }
> }
>
> +typedef struct {
> + bool is_secure;
> + void *hphys;
> + hwaddr gphys;
> +} S1TranslateResult;
Ah, I was wondering whether to suggest this for the previous patch
that added the hphys and gphys arguments :-)
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 34/42] target/arm: Move be test for regime into S1TranslateResult
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (32 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 33/42] target/arm: Split out S1TranslateResult type Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 9:29 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 35/42] target/arm: Move S1_ptw_translate outside arm_ld[lq]_ptw Richard Henderson
` (8 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Hoist this test out of arm_ld[lq]_ptw into S1_ptw_translate.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 99ad894180..d356b0b22d 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -222,6 +222,7 @@ static bool S2_attrs_are_device(uint64_t hcr, uint8_t attrs)
typedef struct {
bool is_secure;
+ bool be;
void *hphys;
hwaddr gphys;
} S1TranslateResult;
@@ -301,6 +302,7 @@ static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
!(pte_secure
? env->cp15.vstcr_el2 & VSTCR_SW
: env->cp15.vtcr_el2 & VTCR_NSW));
+ res->be = regime_translation_big_endian(env, mmu_idx);
return true;
fail:
@@ -320,7 +322,6 @@ static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
CPUState *cs = env_cpu(env);
S1TranslateResult s1;
uint32_t data;
- bool be;
if (!S1_ptw_translate(env, mmu_idx, ptw_idx, addr, is_secure,
debug, &s1, fi)) {
@@ -329,10 +330,9 @@ static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
return 0;
}
- be = regime_translation_big_endian(env, mmu_idx);
if (likely(s1.hphys)) {
/* Page tables are in RAM, and we have the host address. */
- if (be) {
+ if (s1.be) {
data = ldl_be_p(s1.hphys);
} else {
data = ldl_le_p(s1.hphys);
@@ -343,7 +343,7 @@ static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
AddressSpace *as = arm_addressspace(cs, attrs);
MemTxResult result = MEMTX_OK;
- if (be) {
+ if (s1.be) {
data = address_space_ldl_be(as, s1.gphys, attrs, &result);
} else {
data = address_space_ldl_le(as, s1.gphys, attrs, &result);
@@ -364,7 +364,6 @@ static uint64_t arm_ldq_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
CPUState *cs = env_cpu(env);
S1TranslateResult s1;
uint64_t data;
- bool be;
if (!S1_ptw_translate(env, mmu_idx, ptw_idx, addr, is_secure,
debug, &s1, fi)) {
@@ -373,10 +372,9 @@ static uint64_t arm_ldq_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
return 0;
}
- be = regime_translation_big_endian(env, mmu_idx);
if (likely(s1.hphys)) {
/* Page tables are in RAM, and we have the host address. */
- if (be) {
+ if (s1.be) {
data = ldq_be_p(s1.hphys);
} else {
data = ldq_le_p(s1.hphys);
@@ -387,7 +385,7 @@ static uint64_t arm_ldq_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
AddressSpace *as = arm_addressspace(cs, attrs);
MemTxResult result = MEMTX_OK;
- if (be) {
+ if (s1.be) {
data = address_space_ldq_be(as, s1.gphys, attrs, &result);
} else {
data = address_space_ldq_le(as, s1.gphys, attrs, &result);
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 35/42] target/arm: Move S1_ptw_translate outside arm_ld[lq]_ptw
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (33 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 34/42] target/arm: Move be test for regime into S1TranslateResult Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 9:33 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 36/42] target/arm: Add ARMFault_UnsuppAtomicUpdate Richard Henderson
` (7 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Separate S1 translation from the actual lookup.
Will enable lpae hardware updates.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 92 +++++++++++++++++++++++++-----------------------
1 file changed, 48 insertions(+), 44 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index d356b0b22d..84b55b640b 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -315,38 +315,29 @@ static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
}
/* All loads done in the course of a page table walk go through here. */
-static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
- ARMMMUIdx mmu_idx, ARMMMUIdx ptw_idx,
- bool debug, ARMMMUFaultInfo *fi)
+static uint32_t arm_ldl_ptw(CPUARMState *env, const S1TranslateResult *s1,
+ ARMMMUFaultInfo *fi)
{
CPUState *cs = env_cpu(env);
- S1TranslateResult s1;
uint32_t data;
- if (!S1_ptw_translate(env, mmu_idx, ptw_idx, addr, is_secure,
- debug, &s1, fi)) {
- /* Failure. */
- assert(fi->s1ptw);
- return 0;
- }
-
- if (likely(s1.hphys)) {
+ if (likely(s1->hphys)) {
/* Page tables are in RAM, and we have the host address. */
- if (s1.be) {
- data = ldl_be_p(s1.hphys);
+ if (s1->be) {
+ data = ldl_be_p(s1->hphys);
} else {
- data = ldl_le_p(s1.hphys);
+ data = ldl_le_p(s1->hphys);
}
} else {
/* Page tables are in MMIO. */
- MemTxAttrs attrs = { .secure = s1.is_secure };
+ MemTxAttrs attrs = { .secure = s1->is_secure };
AddressSpace *as = arm_addressspace(cs, attrs);
MemTxResult result = MEMTX_OK;
- if (s1.be) {
- data = address_space_ldl_be(as, s1.gphys, attrs, &result);
+ if (s1->be) {
+ data = address_space_ldl_be(as, s1->gphys, attrs, &result);
} else {
- data = address_space_ldl_le(as, s1.gphys, attrs, &result);
+ data = address_space_ldl_le(as, s1->gphys, attrs, &result);
}
if (unlikely(result != MEMTX_OK)) {
fi->type = ARMFault_SyncExternalOnWalk;
@@ -357,38 +348,29 @@ static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
return data;
}
-static uint64_t arm_ldq_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
- ARMMMUIdx mmu_idx, ARMMMUIdx ptw_idx,
- bool debug, ARMMMUFaultInfo *fi)
+static uint64_t arm_ldq_ptw(CPUARMState *env, const S1TranslateResult *s1,
+ ARMMMUFaultInfo *fi)
{
CPUState *cs = env_cpu(env);
- S1TranslateResult s1;
uint64_t data;
- if (!S1_ptw_translate(env, mmu_idx, ptw_idx, addr, is_secure,
- debug, &s1, fi)) {
- /* Failure. */
- assert(fi->s1ptw);
- return 0;
- }
-
- if (likely(s1.hphys)) {
+ if (likely(s1->hphys)) {
/* Page tables are in RAM, and we have the host address. */
- if (s1.be) {
- data = ldq_be_p(s1.hphys);
+ if (s1->be) {
+ data = ldq_be_p(s1->hphys);
} else {
- data = ldq_le_p(s1.hphys);
+ data = ldq_le_p(s1->hphys);
}
} else {
/* Page tables are in MMIO. */
- MemTxAttrs attrs = { .secure = s1.is_secure };
+ MemTxAttrs attrs = { .secure = s1->is_secure };
AddressSpace *as = arm_addressspace(cs, attrs);
MemTxResult result = MEMTX_OK;
- if (s1.be) {
- data = address_space_ldq_be(as, s1.gphys, attrs, &result);
+ if (s1->be) {
+ data = address_space_ldq_be(as, s1->gphys, attrs, &result);
} else {
- data = address_space_ldq_le(as, s1.gphys, attrs, &result);
+ data = address_space_ldq_le(as, s1->gphys, attrs, &result);
}
if (unlikely(result != MEMTX_OK)) {
fi->type = ARMFault_SyncExternalOnWalk;
@@ -520,6 +502,7 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
int domain = 0;
int domain_prot;
hwaddr phys_addr;
+ S1TranslateResult s1;
uint32_t dacr;
/* Pagetable walk. */
@@ -529,7 +512,11 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
fi->type = ARMFault_Translation;
goto do_fault;
}
- desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, ptw_idx, debug, fi);
+ if (!S1_ptw_translate(env, mmu_idx, ptw_idx, table,
+ is_secure, debug, &s1, fi)) {
+ goto do_fault;
+ }
+ desc = arm_ldl_ptw(env, &s1, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -567,7 +554,11 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
/* Fine pagetable. */
table = (desc & 0xfffff000) | ((address >> 8) & 0xffc);
}
- desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, ptw_idx, debug, fi);
+ if (!S1_ptw_translate(env, mmu_idx, ptw_idx, table,
+ is_secure, debug, &s1, fi)) {
+ goto do_fault;
+ }
+ desc = arm_ldl_ptw(env, &s1, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -643,6 +634,7 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
int domain_prot;
hwaddr phys_addr;
uint32_t dacr;
+ S1TranslateResult s1;
bool ns;
/* Pagetable walk. */
@@ -652,7 +644,11 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
fi->type = ARMFault_Translation;
goto do_fault;
}
- desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, ptw_idx, debug, fi);
+ if (!S1_ptw_translate(env, mmu_idx, ptw_idx, table,
+ is_secure, debug, &s1, fi)) {
+ goto do_fault;
+ }
+ desc = arm_ldl_ptw(env, &s1, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -705,7 +701,11 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
ns = extract32(desc, 3, 1);
/* Lookup l2 entry. */
table = (desc & 0xfffffc00) | ((address >> 10) & 0x3fc);
- desc = arm_ldl_ptw(env, table, is_secure, mmu_idx, ptw_idx, debug, fi);
+ if (!S1_ptw_translate(env, mmu_idx, ptw_idx, table,
+ is_secure, debug, &s1, fi)) {
+ goto do_fault;
+ }
+ desc = arm_ldl_ptw(env, &s1, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
@@ -1281,14 +1281,18 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
*/
tableattrs = is_secure ? 0 : (1 << 4);
for (;;) {
+ S1TranslateResult s1;
uint64_t descriptor;
bool nstable;
descaddr |= (address >> (stride * (4 - level))) & indexmask;
descaddr &= ~7ULL;
nstable = extract32(tableattrs, 4, 1);
- descriptor = arm_ldq_ptw(env, descaddr, !nstable, mmu_idx,
- ptw_idx, debug, fi);
+ if (!S1_ptw_translate(env, mmu_idx, ptw_idx, descaddr,
+ !nstable, debug, &s1, fi)) {
+ goto do_fault;
+ }
+ descriptor = arm_ldq_ptw(env, &s1, fi);
if (fi->type != ARMFault_None) {
goto do_fault;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 36/42] target/arm: Add ARMFault_UnsuppAtomicUpdate
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (34 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 35/42] target/arm: Move S1_ptw_translate outside arm_ld[lq]_ptw Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 9:36 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 37/42] target/arm: Remove loop from get_phys_addr_lpae Richard Henderson
` (6 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
This fault type is to be used with FEAT_HAFDBS when
the guest enables hw updates, but places the tables
in memory where atomic updates are unsupported.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/internals.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/target/arm/internals.h b/target/arm/internals.h
index e95b6b1b8f..4a2b1ec31c 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -338,6 +338,7 @@ typedef enum ARMFaultType {
ARMFault_AsyncExternal,
ARMFault_Debug,
ARMFault_TLBConflict,
+ ARMFault_UnsuppAtomicUpdate,
ARMFault_Lockdown,
ARMFault_Exclusive,
ARMFault_ICacheMaint,
@@ -524,6 +525,9 @@ static inline uint32_t arm_fi_to_lfsc(ARMMMUFaultInfo *fi)
case ARMFault_TLBConflict:
fsc = 0x30;
break;
+ case ARMFault_UnsuppAtomicUpdate:
+ fsc = 0x31;
+ break;
case ARMFault_Lockdown:
fsc = 0x34;
break;
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 37/42] target/arm: Remove loop from get_phys_addr_lpae
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (35 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 36/42] target/arm: Add ARMFault_UnsuppAtomicUpdate Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 10:20 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 38/42] target/arm: Fix fault reporting in get_phys_addr_lpae Richard Henderson
` (5 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
The unconditional loop was used both to iterate over levels
and to control parsing of attributes. Use an explicit goto
in both cases.
While this appears less clean for iterating over levels, we
will need to jump back into the middle of this loop for
atomic updates, which is even uglier.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 176 +++++++++++++++++++++++------------------------
1 file changed, 88 insertions(+), 88 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 84b55b640b..e6b385a8b1 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -1082,6 +1082,9 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
uint64_t descaddrmask;
bool aarch64 = arm_el_is_aa64(env, el);
bool guarded = false;
+ S1TranslateResult s1;
+ uint64_t descriptor;
+ bool nstable;
/* TODO: This code does not support shareability levels. */
if (aarch64) {
@@ -1280,96 +1283,93 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
* bits at each step.
*/
tableattrs = is_secure ? 0 : (1 << 4);
- for (;;) {
- S1TranslateResult s1;
- uint64_t descriptor;
- bool nstable;
- descaddr |= (address >> (stride * (4 - level))) & indexmask;
- descaddr &= ~7ULL;
- nstable = extract32(tableattrs, 4, 1);
- if (!S1_ptw_translate(env, mmu_idx, ptw_idx, descaddr,
- !nstable, debug, &s1, fi)) {
- goto do_fault;
- }
- descriptor = arm_ldq_ptw(env, &s1, fi);
- if (fi->type != ARMFault_None) {
- goto do_fault;
- }
-
- if (!(descriptor & 1) ||
- (!(descriptor & 2) && (level == 3))) {
- /* Invalid, or the Reserved level 3 encoding */
- goto do_fault;
- }
-
- descaddr = descriptor & descaddrmask;
-
- /*
- * For FEAT_LPA and PS=6, bits [51:48] of descaddr are in [15:12]
- * of descriptor. For FEAT_LPA2 and effective DS, bits [51:50] of
- * descaddr are in [9:8]. Otherwise, if descaddr is out of range,
- * raise AddressSizeFault.
- */
- if (outputsize > 48) {
- if (param.ds) {
- descaddr |= extract64(descriptor, 8, 2) << 50;
- } else {
- descaddr |= extract64(descriptor, 12, 4) << 48;
- }
- } else if (descaddr >> outputsize) {
- fault_type = ARMFault_AddressSize;
- goto do_fault;
- }
-
- if ((descriptor & 2) && (level < 3)) {
- /*
- * Table entry. The top five bits are attributes which may
- * propagate down through lower levels of the table (and
- * which are all arranged so that 0 means "no effect", so
- * we can gather them up by ORing in the bits at each level).
- */
- tableattrs |= extract64(descriptor, 59, 5);
- level++;
- indexmask = indexmask_grainsize;
- continue;
- }
- /*
- * Block entry at level 1 or 2, or page entry at level 3.
- * These are basically the same thing, although the number
- * of bits we pull in from the vaddr varies. Note that although
- * descaddrmask masks enough of the low bits of the descriptor
- * to give a correct page or table address, the address field
- * in a block descriptor is smaller; so we need to explicitly
- * clear the lower bits here before ORing in the low vaddr bits.
- */
- page_size = (1ULL << ((stride * (4 - level)) + 3));
- descaddr &= ~(hwaddr)(page_size - 1);
- descaddr |= (address & (page_size - 1));
- /* Extract attributes from the descriptor */
- attrs = extract64(descriptor, 2, 10)
- | (extract64(descriptor, 52, 12) << 10);
-
- if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
- /* Stage 2 table descriptors do not include any attribute fields */
- break;
- }
- /* Merge in attributes from table descriptors */
- attrs |= nstable << 3; /* NS */
- guarded = extract64(descriptor, 50, 1); /* GP */
- if (param.hpd) {
- /* HPD disables all the table attributes except NSTable. */
- break;
- }
- attrs |= extract32(tableattrs, 0, 2) << 11; /* XN, PXN */
- /*
- * The sense of AP[1] vs APTable[0] is reversed, as APTable[0] == 1
- * means "force PL1 access only", which means forcing AP[1] to 0.
- */
- attrs &= ~(extract32(tableattrs, 2, 1) << 4); /* !APT[0] => AP[1] */
- attrs |= extract32(tableattrs, 3, 1) << 5; /* APT[1] => AP[2] */
- break;
+ next_level:
+ descaddr |= (address >> (stride * (4 - level))) & indexmask;
+ descaddr &= ~7ULL;
+ nstable = extract32(tableattrs, 4, 1);
+ if (!S1_ptw_translate(env, mmu_idx, ptw_idx, descaddr,
+ !nstable, debug, &s1, fi)) {
+ goto do_fault;
}
+ descriptor = arm_ldq_ptw(env, &s1, fi);
+ if (fi->type != ARMFault_None) {
+ goto do_fault;
+ }
+
+ if (!(descriptor & 1) || (!(descriptor & 2) && (level == 3))) {
+ /* Invalid, or the Reserved level 3 encoding */
+ goto do_fault;
+ }
+
+ descaddr = descriptor & descaddrmask;
+
+ /*
+ * For FEAT_LPA and PS=6, bits [51:48] of descaddr are in [15:12]
+ * of descriptor. For FEAT_LPA2 and effective DS, bits [51:50] of
+ * descaddr are in [9:8]. Otherwise, if descaddr is out of range,
+ * raise AddressSizeFault.
+ */
+ if (outputsize > 48) {
+ if (param.ds) {
+ descaddr |= extract64(descriptor, 8, 2) << 50;
+ } else {
+ descaddr |= extract64(descriptor, 12, 4) << 48;
+ }
+ } else if (descaddr >> outputsize) {
+ fault_type = ARMFault_AddressSize;
+ goto do_fault;
+ }
+
+ if ((descriptor & 2) && (level < 3)) {
+ /*
+ * Table entry. The top five bits are attributes which may
+ * propagate down through lower levels of the table (and
+ * which are all arranged so that 0 means "no effect", so
+ * we can gather them up by ORing in the bits at each level).
+ */
+ tableattrs |= extract64(descriptor, 59, 5);
+ level++;
+ indexmask = indexmask_grainsize;
+ goto next_level;
+ }
+
+ /*
+ * Block entry at level 1 or 2, or page entry at level 3.
+ * These are basically the same thing, although the number
+ * of bits we pull in from the vaddr varies. Note that although
+ * descaddrmask masks enough of the low bits of the descriptor
+ * to give a correct page or table address, the address field
+ * in a block descriptor is smaller; so we need to explicitly
+ * clear the lower bits here before ORing in the low vaddr bits.
+ */
+ page_size = (1ULL << ((stride * (4 - level)) + 3));
+ descaddr &= ~(page_size - 1);
+ descaddr |= (address & (page_size - 1));
+ /* Extract attributes from the descriptor */
+ attrs = extract64(descriptor, 2, 10)
+ | (extract64(descriptor, 52, 12) << 10);
+
+ if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
+ /* Stage 2 table descriptors do not include any attribute fields */
+ goto skip_attrs;
+ }
+ /* Merge in attributes from table descriptors */
+ attrs |= nstable << 3; /* NS */
+ guarded = extract64(descriptor, 50, 1); /* GP */
+ if (param.hpd) {
+ /* HPD disables all the table attributes except NSTable. */
+ goto skip_attrs;
+ }
+ attrs |= extract32(tableattrs, 0, 2) << 11; /* XN, PXN */
+ /*
+ * The sense of AP[1] vs APTable[0] is reversed, as APTable[0] == 1
+ * means "force PL1 access only", which means forcing AP[1] to 0.
+ */
+ attrs &= ~(extract32(tableattrs, 2, 1) << 4); /* !APT[0] => AP[1] */
+ attrs |= extract32(tableattrs, 3, 1) << 5; /* APT[1] => AP[2] */
+ skip_attrs:
+
/*
* Here descaddr is the final physical address, and attributes
* are all in attrs.
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 37/42] target/arm: Remove loop from get_phys_addr_lpae
2022-10-01 16:23 ` [PATCH v3 37/42] target/arm: Remove loop from get_phys_addr_lpae Richard Henderson
@ 2022-10-07 10:20 ` Peter Maydell
0 siblings, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 10:20 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 18:02, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> The unconditional loop was used both to iterate over levels
> and to control parsing of attributes. Use an explicit goto
> in both cases.
>
> While this appears less clean for iterating over levels, we
> will need to jump back into the middle of this loop for
> atomic updates, which is even uglier.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> target/arm/ptw.c | 176 +++++++++++++++++++++++------------------------
> 1 file changed, 88 insertions(+), 88 deletions(-)
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 38/42] target/arm: Fix fault reporting in get_phys_addr_lpae
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (36 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 37/42] target/arm: Remove loop from get_phys_addr_lpae Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 10:26 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 39/42] target/arm: Don't shift attrs " Richard Henderson
` (4 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Always overriding fi->type was incorrect, as we would not properly
propagate the fault type from S1_ptw_translate, or arm_ldq_ptw.
Simplify things by providing a new label for a translation fault.
For other faults, store into fi directly.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 31 +++++++++++++------------------
1 file changed, 13 insertions(+), 18 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index e6b385a8b1..01a27b30fb 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -1065,8 +1065,6 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
GetPhysAddrResult *result, ARMMMUFaultInfo *fi)
{
ARMCPU *cpu = env_archcpu(env);
- /* Read an LPAE long-descriptor translation table. */
- ARMFaultType fault_type = ARMFault_Translation;
uint32_t level;
ARMVAParameters param;
uint64_t ttbr;
@@ -1104,8 +1102,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
* so our choice is to always raise the fault.
*/
if (param.tsz_oob) {
- fault_type = ARMFault_Translation;
- goto do_fault;
+ goto do_translation_fault;
}
addrsize = 64 - 8 * param.tbi;
@@ -1142,8 +1139,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
addrsize - inputsize);
if (-top_bits != param.select) {
/* The gap between the two regions is a Translation fault */
- fault_type = ARMFault_Translation;
- goto do_fault;
+ goto do_translation_fault;
}
}
@@ -1175,7 +1171,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
* Translation table walk disabled => Translation fault on TLB miss
* Note: This is always 0 on 64-bit EL2 and EL3.
*/
- goto do_fault;
+ goto do_translation_fault;
}
if (mmu_idx != ARMMMUIdx_Stage2 && mmu_idx != ARMMMUIdx_Stage2_S) {
@@ -1206,8 +1202,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
if (param.ds && stride == 9 && sl2) {
if (sl0 != 0) {
level = 0;
- fault_type = ARMFault_Translation;
- goto do_fault;
+ goto do_translation_fault;
}
startlevel = -1;
} else if (!aarch64 || stride == 9) {
@@ -1226,8 +1221,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
ok = check_s2_mmu_setup(cpu, aarch64, startlevel,
inputsize, stride, outputsize);
if (!ok) {
- fault_type = ARMFault_Translation;
- goto do_fault;
+ goto do_translation_fault;
}
level = startlevel;
}
@@ -1249,7 +1243,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
descaddr |= extract64(ttbr, 2, 4) << 48;
} else if (descaddr >> outputsize) {
level = 0;
- fault_type = ARMFault_AddressSize;
+ fi->type = ARMFault_AddressSize;
goto do_fault;
}
@@ -1299,7 +1293,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
if (!(descriptor & 1) || (!(descriptor & 2) && (level == 3))) {
/* Invalid, or the Reserved level 3 encoding */
- goto do_fault;
+ goto do_translation_fault;
}
descaddr = descriptor & descaddrmask;
@@ -1317,7 +1311,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
descaddr |= extract64(descriptor, 12, 4) << 48;
}
} else if (descaddr >> outputsize) {
- fault_type = ARMFault_AddressSize;
+ fi->type = ARMFault_AddressSize;
goto do_fault;
}
@@ -1374,9 +1368,9 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
* Here descaddr is the final physical address, and attributes
* are all in attrs.
*/
- fault_type = ARMFault_AccessFlag;
if ((attrs & (1 << 8)) == 0) {
/* Access flag */
+ fi->type = ARMFault_AccessFlag;
goto do_fault;
}
@@ -1393,8 +1387,8 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
result->f.prot = get_S1prot(env, mmu_idx, aarch64, ap, ns, xn, pxn);
}
- fault_type = ARMFault_Permission;
if (!(result->f.prot & (1 << access_type))) {
+ fi->type = ARMFault_Permission;
goto do_fault;
}
@@ -1439,8 +1433,9 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
result->f.lg_page_size = ctz64(page_size);
return false;
-do_fault:
- fi->type = fault_type;
+ do_translation_fault:
+ fi->type = ARMFault_Translation;
+ do_fault:
fi->level = level;
/* Tag the error as S2 for failed S1 PTW at S2 or ordinary S2. */
fi->stage2 = fi->s1ptw || (mmu_idx == ARMMMUIdx_Stage2 ||
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 38/42] target/arm: Fix fault reporting in get_phys_addr_lpae
2022-10-01 16:23 ` [PATCH v3 38/42] target/arm: Fix fault reporting in get_phys_addr_lpae Richard Henderson
@ 2022-10-07 10:26 ` Peter Maydell
0 siblings, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 10:26 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:48, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Always overriding fi->type was incorrect, as we would not properly
> propagate the fault type from S1_ptw_translate, or arm_ldq_ptw.
> Simplify things by providing a new label for a translation fault.
> For other faults, store into fi directly.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> target/arm/ptw.c | 31 +++++++++++++------------------
> 1 file changed, 13 insertions(+), 18 deletions(-)
>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 39/42] target/arm: Don't shift attrs in get_phys_addr_lpae
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (37 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 38/42] target/arm: Fix fault reporting in get_phys_addr_lpae Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 10:35 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 40/42] target/arm: Consider GP an attribute " Richard Henderson
` (3 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Leave the upper and lower attributes in the place they originate
from in the descriptor. Shifting them around is confusing, since
one cannot read the bit numbers out of the manual. Also, new
attributes have been added which would alter the shifts.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 31 +++++++++++++++----------------
1 file changed, 15 insertions(+), 16 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 01a27b30fb..c68fd73617 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -1071,7 +1071,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
hwaddr descaddr, indexmask, indexmask_grainsize;
uint32_t tableattrs;
target_ulong page_size;
- uint32_t attrs;
+ uint64_t attrs;
int32_t stride;
int addrsize, inputsize, outputsize;
uint64_t tcr = regime_tcr(env, mmu_idx);
@@ -1341,49 +1341,48 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
descaddr &= ~(page_size - 1);
descaddr |= (address & (page_size - 1));
/* Extract attributes from the descriptor */
- attrs = extract64(descriptor, 2, 10)
- | (extract64(descriptor, 52, 12) << 10);
+ attrs = descriptor & (MAKE_64BIT_MASK(2, 10) | MAKE_64BIT_MASK(52, 12));
if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
/* Stage 2 table descriptors do not include any attribute fields */
goto skip_attrs;
}
/* Merge in attributes from table descriptors */
- attrs |= nstable << 3; /* NS */
+ attrs |= nstable << 5; /* NS */
guarded = extract64(descriptor, 50, 1); /* GP */
if (param.hpd) {
/* HPD disables all the table attributes except NSTable. */
goto skip_attrs;
}
- attrs |= extract32(tableattrs, 0, 2) << 11; /* XN, PXN */
+ attrs |= extract64(tableattrs, 0, 2) << 53; /* XN, PXN */
/*
* The sense of AP[1] vs APTable[0] is reversed, as APTable[0] == 1
* means "force PL1 access only", which means forcing AP[1] to 0.
*/
- attrs &= ~(extract32(tableattrs, 2, 1) << 4); /* !APT[0] => AP[1] */
- attrs |= extract32(tableattrs, 3, 1) << 5; /* APT[1] => AP[2] */
+ attrs &= ~(extract64(tableattrs, 2, 1) << 6); /* !APT[0] => AP[1] */
+ attrs |= extract32(tableattrs, 3, 1) << 7; /* APT[1] => AP[2] */
skip_attrs:
/*
* Here descaddr is the final physical address, and attributes
* are all in attrs.
*/
- if ((attrs & (1 << 8)) == 0) {
+ if ((attrs & (1 << 10)) == 0) {
/* Access flag */
fi->type = ARMFault_AccessFlag;
goto do_fault;
}
- ap = extract32(attrs, 4, 2);
+ ap = extract32(attrs, 6, 2);
if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
ns = mmu_idx == ARMMMUIdx_Stage2;
- xn = extract32(attrs, 11, 2);
+ xn = extract64(attrs, 54, 2);
result->f.prot = get_S2prot(env, ap, xn, s1_is_el0);
} else {
- ns = extract32(attrs, 3, 1);
- xn = extract32(attrs, 12, 1);
- pxn = extract32(attrs, 11, 1);
+ ns = extract32(attrs, 5, 1);
+ xn = extract64(attrs, 54, 1);
+ pxn = extract64(attrs, 53, 1);
result->f.prot = get_S1prot(env, mmu_idx, aarch64, ap, ns, xn, pxn);
}
@@ -1408,10 +1407,10 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
result->cacheattrs.is_s2_format = true;
- result->cacheattrs.attrs = extract32(attrs, 0, 4);
+ result->cacheattrs.attrs = extract32(attrs, 2, 4);
} else {
/* Index into MAIR registers for cache attributes */
- uint8_t attrindx = extract32(attrs, 0, 3);
+ uint8_t attrindx = extract32(attrs, 2, 3);
uint64_t mair = env->cp15.mair_el[regime_el(env, mmu_idx)];
assert(attrindx <= 7);
result->cacheattrs.is_s2_format = false;
@@ -1426,7 +1425,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
if (param.ds) {
result->cacheattrs.shareability = param.sh;
} else {
- result->cacheattrs.shareability = extract32(attrs, 6, 2);
+ result->cacheattrs.shareability = extract32(attrs, 8, 2);
}
result->f.phys_addr = descaddr;
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 39/42] target/arm: Don't shift attrs in get_phys_addr_lpae
2022-10-01 16:23 ` [PATCH v3 39/42] target/arm: Don't shift attrs " Richard Henderson
@ 2022-10-07 10:35 ` Peter Maydell
0 siblings, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 10:35 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:49, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Leave the upper and lower attributes in the place they originate
> from in the descriptor. Shifting them around is confusing, since
> one cannot read the bit numbers out of the manual. Also, new
> attributes have been added which would alter the shifts.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> target/arm/ptw.c | 31 +++++++++++++++----------------
> 1 file changed, 15 insertions(+), 16 deletions(-)
>
> diff --git a/target/arm/ptw.c b/target/arm/ptw.c
> index 01a27b30fb..c68fd73617 100644
> --- a/target/arm/ptw.c
> +++ b/target/arm/ptw.c
> @@ -1071,7 +1071,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
> hwaddr descaddr, indexmask, indexmask_grainsize;
> uint32_t tableattrs;
> target_ulong page_size;
> - uint32_t attrs;
> + uint64_t attrs;
> int32_t stride;
> int addrsize, inputsize, outputsize;
> uint64_t tcr = regime_tcr(env, mmu_idx);
> @@ -1341,49 +1341,48 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
> descaddr &= ~(page_size - 1);
> descaddr |= (address & (page_size - 1));
> /* Extract attributes from the descriptor */
> - attrs = extract64(descriptor, 2, 10)
> - | (extract64(descriptor, 52, 12) << 10);
> + attrs = descriptor & (MAKE_64BIT_MASK(2, 10) | MAKE_64BIT_MASK(52, 12));
So bit positions for the lower-part bits need to have 2 added,
and bit positions for upper-part bits need to have 42 added.
> if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
> ns = mmu_idx == ARMMMUIdx_Stage2;
> - xn = extract32(attrs, 11, 2);
> + xn = extract64(attrs, 54, 2);
...so this one looks wrong, should be 53 ?
> result->f.prot = get_S2prot(env, ap, xn, s1_is_el0);
> } else {
> - ns = extract32(attrs, 3, 1);
> - xn = extract32(attrs, 12, 1);
> - pxn = extract32(attrs, 11, 1);
> + ns = extract32(attrs, 5, 1);
> + xn = extract64(attrs, 54, 1);
> + pxn = extract64(attrs, 53, 1);
Compare here where bit 11 in the old code is 53 in the new.
> result->f.prot = get_S1prot(env, mmu_idx, aarch64, ap, ns, xn, pxn);
> }
>
Otherwise
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 40/42] target/arm: Consider GP an attribute in get_phys_addr_lpae
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (38 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 39/42] target/arm: Don't shift attrs " Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 10:40 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 41/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (2 subsequent siblings)
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Both GP and DBM are in the upper attribute block.
Extend the computation of attrs to include them,
then simplify the setting of guarded.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index c68fd73617..45734b0d28 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -1079,7 +1079,6 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
uint32_t el = regime_el(env, mmu_idx);
uint64_t descaddrmask;
bool aarch64 = arm_el_is_aa64(env, el);
- bool guarded = false;
S1TranslateResult s1;
uint64_t descriptor;
bool nstable;
@@ -1341,7 +1340,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
descaddr &= ~(page_size - 1);
descaddr |= (address & (page_size - 1));
/* Extract attributes from the descriptor */
- attrs = descriptor & (MAKE_64BIT_MASK(2, 10) | MAKE_64BIT_MASK(52, 12));
+ attrs = descriptor & (MAKE_64BIT_MASK(2, 10) | MAKE_64BIT_MASK(50, 14));
if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
/* Stage 2 table descriptors do not include any attribute fields */
@@ -1349,7 +1348,6 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
}
/* Merge in attributes from table descriptors */
attrs |= nstable << 5; /* NS */
- guarded = extract64(descriptor, 50, 1); /* GP */
if (param.hpd) {
/* HPD disables all the table attributes except NSTable. */
goto skip_attrs;
@@ -1402,7 +1400,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
/* When in aarch64 mode, and BTI is enabled, remember GP in the TLB. */
if (aarch64 && cpu_isar_feature(aa64_bti, cpu)) {
- result->f.guarded = guarded;
+ result->f.guarded = extract64(attrs, 50, 1); /* GP */
}
if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* [PATCH v3 41/42] target/arm: Implement FEAT_HAFDBS
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (39 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 40/42] target/arm: Consider GP an attribute " Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 13:47 ` Peter Maydell
2022-10-01 16:23 ` [PATCH v3 42/42] target/arm: Use the max page size in a 2-stage ptw Richard Henderson
2022-10-10 13:10 ` [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Peter Maydell
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm
Perform the atomic update for hardware management of the access flag
and the dirty bit.
A limitation of the implementation so far is that the page table
itself must already be writable, i.e. the dirty bit for the stage2
page table must already be set, i.e. we cannot set both dirty bits
at the same time.
This is allowed because it is CONSTRAINED UNPREDICTABLE whether any
atomic update happens at all. The implementation is allowed to simply
fall back on software update at any time.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
docs/system/arm/emulation.rst | 1 +
target/arm/cpu64.c | 1 +
target/arm/ptw.c | 119 ++++++++++++++++++++++++++++++++--
3 files changed, 115 insertions(+), 6 deletions(-)
diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index be7bbffe59..c3582d075e 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -31,6 +31,7 @@ the following architecture extensions:
- FEAT_FRINTTS (Floating-point to integer instructions)
- FEAT_FlagM (Flag manipulation instructions v2)
- FEAT_FlagM2 (Enhancements to flag manipulation instructions)
+- FEAT_HAFDBS (Hardware management of the access flag and dirty bit state)
- FEAT_HCX (Support for the HCRX_EL2 register)
- FEAT_HPDS (Hierarchical permission disables)
- FEAT_I8MM (AArch64 Int8 matrix multiplication instructions)
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index e6314e86d2..b064dc7964 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -1116,6 +1116,7 @@ static void aarch64_max_initfn(Object *obj)
cpu->isar.id_aa64mmfr0 = t;
t = cpu->isar.id_aa64mmfr1;
+ t = FIELD_DP64(t, ID_AA64MMFR1, HAFDBS, 2); /* FEAT_HAFDBS */
t = FIELD_DP64(t, ID_AA64MMFR1, VMIDBITS, 2); /* FEAT_VMID16 */
t = FIELD_DP64(t, ID_AA64MMFR1, VH, 1); /* FEAT_VHE */
t = FIELD_DP64(t, ID_AA64MMFR1, HPDS, 1); /* FEAT_HPDS */
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 45734b0d28..14ab56d1b5 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -223,6 +223,7 @@ static bool S2_attrs_are_device(uint64_t hcr, uint8_t attrs)
typedef struct {
bool is_secure;
bool be;
+ bool rw;
void *hphys;
hwaddr gphys;
} S1TranslateResult;
@@ -261,7 +262,8 @@ static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
pte_attrs = s2.cacheattrs.attrs;
pte_secure = s2.f.attrs.secure;
}
- res->hphys = NULL;
+ res->hphys = NULL; /* force slow path */
+ res->rw = false; /* debug never modifies */
} else {
CPUTLBEntryFull *full;
int flags;
@@ -276,6 +278,7 @@ static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
goto fail;
}
res->gphys = full->phys_addr;
+ res->rw = full->prot & PAGE_WRITE;
pte_attrs = full->pte_attrs;
pte_secure = full->attrs.secure;
}
@@ -381,6 +384,56 @@ static uint64_t arm_ldq_ptw(CPUARMState *env, const S1TranslateResult *s1,
return data;
}
+static uint64_t arm_casq_ptw(CPUARMState *env, uint64_t old_val,
+ uint64_t new_val, const S1TranslateResult *s1,
+ ARMMMUFaultInfo *fi)
+{
+ uint64_t cur_val;
+
+ if (unlikely(!s1->hphys)) {
+ fi->type = ARMFault_UnsuppAtomicUpdate;
+ fi->s1ptw = true;
+ return 0;
+ }
+
+#ifndef CONFIG_ATOMIC64
+ /*
+ * We can't support the atomic operation on the host. We should be
+ * running in round-robin mode though, which means that we would only
+ * race with dma i/o.
+ */
+ qemu_mutex_lock_iothread();
+ if (s1->be) {
+ cur_val = ldq_be_p(s1->hphys);
+ if (cur_val == old_val) {
+ stq_be_p(s1->hphys, new_val);
+ }
+ } else {
+ cur_val = ldq_le_p(s1->hphys);
+ if (cur_val == old_val) {
+ stq_le_p(s1->hphys, new_val);
+ }
+ }
+ qemu_mutex_unlock_iothread();
+#else
+ if (s1->be) {
+ old_val = cpu_to_be64(old_val);
+ new_val = cpu_to_be64(new_val);
+ cur_val = qatomic_cmpxchg__nocheck((uint64_t *)s1->hphys,
+ old_val, new_val);
+ cur_val = be64_to_cpu(cur_val);
+ } else {
+ old_val = cpu_to_le64(old_val);
+ new_val = cpu_to_le64(new_val);
+ cur_val = qatomic_cmpxchg__nocheck((uint64_t *)s1->hphys,
+ old_val, new_val);
+ cur_val = le64_to_cpu(cur_val);
+ }
+#endif
+
+ return cur_val;
+}
+
static bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx,
uint32_t *table, uint32_t address)
{
@@ -1290,6 +1343,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
goto do_fault;
}
+ restart_atomic_update:
if (!(descriptor & 1) || (!(descriptor & 2) && (level == 3))) {
/* Invalid, or the Reserved level 3 encoding */
goto do_translation_fault;
@@ -1365,10 +1419,28 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
* Here descaddr is the final physical address, and attributes
* are all in attrs.
*/
- if ((attrs & (1 << 10)) == 0) {
+ if ((attrs & (1 << 10)) == 0 && !debug) {
/* Access flag */
- fi->type = ARMFault_AccessFlag;
- goto do_fault;
+ uint64_t new_des, old_des;
+
+ /*
+ * If HA is disabled, or if the pte is not writable,
+ * pass on the access fault to software.
+ */
+ if (!param.ha || !s1.rw) {
+ fi->type = ARMFault_AccessFlag;
+ goto do_fault;
+ }
+
+ old_des = descriptor;
+ new_des = descriptor | (1 << 10); /* AF */
+ descriptor = arm_casq_ptw(env, old_des, new_des, &s1, fi);
+ if (fi->type != ARMFault_None) {
+ goto do_fault;
+ }
+ if (old_des != descriptor) {
+ goto restart_atomic_update;
+ }
}
ap = extract32(attrs, 6, 2);
@@ -1385,8 +1457,43 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
}
if (!(result->f.prot & (1 << access_type))) {
- fi->type = ARMFault_Permission;
- goto do_fault;
+ uint64_t new_des, old_des;
+
+ /* Writes may set dirty if DBM attribute is set. */
+ if (!param.hd
+ || access_type != MMU_DATA_STORE
+ || !extract64(attrs, 51, 1) /* DBM */
+ || !s1.rw) {
+ fi->type = ARMFault_Permission;
+ goto do_fault;
+ }
+
+ old_des = descriptor;
+ if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
+ new_des = descriptor | (1ull << 7); /* S2AP[1] */
+ } else {
+ new_des = descriptor & ~(1ull << 7); /* AP[2] */
+ }
+
+ /*
+ * If the descriptor didn't change, then attributes weren't the
+ * reason for the permission fault, so deliver it.
+ */
+ if (old_des == new_des) {
+ fi->type = ARMFault_Permission;
+ goto do_fault;
+ }
+
+ descriptor = arm_casq_ptw(env, old_des, new_des, &s1, fi);
+ if (fi->type != ARMFault_None) {
+ goto do_fault;
+ }
+ if (old_des != descriptor) {
+ goto restart_atomic_update;
+ }
+
+ /* Success: the page is now writable. */
+ result->f.prot |= 1 << MMU_DATA_STORE;
}
if (ns) {
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 41/42] target/arm: Implement FEAT_HAFDBS
2022-10-01 16:23 ` [PATCH v3 41/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
@ 2022-10-07 13:47 ` Peter Maydell
2022-10-07 16:04 ` Peter Maydell
2022-10-07 16:45 ` Richard Henderson
0 siblings, 2 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 13:47 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 18:04, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Perform the atomic update for hardware management of the access flag
> and the dirty bit.
>
> A limitation of the implementation so far is that the page table
> itself must already be writable, i.e. the dirty bit for the stage2
> page table must already be set, i.e. we cannot set both dirty bits
> at the same time.
>
> This is allowed because it is CONSTRAINED UNPREDICTABLE whether any
> atomic update happens at all. The implementation is allowed to simply
> fall back on software update at any time.
I can't see where in the Arm ARM this is stated.
In any case, if HA is set you can't simply return ARMFault_AccessFlag
without breaking the bit in D5.4.12 which says
"When hardware updates of the Access flag are enabled for a stage of
translation an address translation instruction that uses that stage
of translation will not report that the address will give rise to
an Access flag fault in the PAR"
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> docs/system/arm/emulation.rst | 1 +
> target/arm/cpu64.c | 1 +
> target/arm/ptw.c | 119 ++++++++++++++++++++++++++++++++--
> 3 files changed, 115 insertions(+), 6 deletions(-)
>
> diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
> index be7bbffe59..c3582d075e 100644
> --- a/docs/system/arm/emulation.rst
> +++ b/docs/system/arm/emulation.rst
> @@ -31,6 +31,7 @@ the following architecture extensions:
> - FEAT_FRINTTS (Floating-point to integer instructions)
> - FEAT_FlagM (Flag manipulation instructions v2)
> - FEAT_FlagM2 (Enhancements to flag manipulation instructions)
> +- FEAT_HAFDBS (Hardware management of the access flag and dirty bit state)
> - FEAT_HCX (Support for the HCRX_EL2 register)
> - FEAT_HPDS (Hierarchical permission disables)
> - FEAT_I8MM (AArch64 Int8 matrix multiplication instructions)
> diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
> index e6314e86d2..b064dc7964 100644
> --- a/target/arm/cpu64.c
> +++ b/target/arm/cpu64.c
> @@ -1116,6 +1116,7 @@ static void aarch64_max_initfn(Object *obj)
> cpu->isar.id_aa64mmfr0 = t;
>
> t = cpu->isar.id_aa64mmfr1;
> + t = FIELD_DP64(t, ID_AA64MMFR1, HAFDBS, 2); /* FEAT_HAFDBS */
I think we should split the access flag update and the
dirty-bit update into separate commits. It might be useful
for bisection purposes later, and it looks like they're pretty
cleanly separable. (Though if you look at my last comment in this
email, maybe not quite so clean as in the code as you have it here.)
> t = FIELD_DP64(t, ID_AA64MMFR1, VMIDBITS, 2); /* FEAT_VMID16 */
> t = FIELD_DP64(t, ID_AA64MMFR1, VH, 1); /* FEAT_VHE */
> t = FIELD_DP64(t, ID_AA64MMFR1, HPDS, 1); /* FEAT_HPDS */
> diff --git a/target/arm/ptw.c b/target/arm/ptw.c
> index 45734b0d28..14ab56d1b5 100644
> --- a/target/arm/ptw.c
> +++ b/target/arm/ptw.c
> @@ -223,6 +223,7 @@ static bool S2_attrs_are_device(uint64_t hcr, uint8_t attrs)
> typedef struct {
> bool is_secure;
> bool be;
> + bool rw;
> void *hphys;
> hwaddr gphys;
> } S1TranslateResult;
> @@ -261,7 +262,8 @@ static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
> pte_attrs = s2.cacheattrs.attrs;
> pte_secure = s2.f.attrs.secure;
> }
> - res->hphys = NULL;
> + res->hphys = NULL; /* force slow path */
> + res->rw = false; /* debug never modifies */
> } else {
> CPUTLBEntryFull *full;
> int flags;
> @@ -276,6 +278,7 @@ static bool S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
> goto fail;
> }
> res->gphys = full->phys_addr;
> + res->rw = full->prot & PAGE_WRITE;
> pte_attrs = full->pte_attrs;
> pte_secure = full->attrs.secure;
> }
> @@ -381,6 +384,56 @@ static uint64_t arm_ldq_ptw(CPUARMState *env, const S1TranslateResult *s1,
> return data;
> }
>
> +static uint64_t arm_casq_ptw(CPUARMState *env, uint64_t old_val,
> + uint64_t new_val, const S1TranslateResult *s1,
> + ARMMMUFaultInfo *fi)
> +{
> + uint64_t cur_val;
> +
> + if (unlikely(!s1->hphys)) {
> + fi->type = ARMFault_UnsuppAtomicUpdate;
> + fi->s1ptw = true;
> + return 0;
> + }
> +
> +#ifndef CONFIG_ATOMIC64
> + /*
> + * We can't support the atomic operation on the host. We should be
> + * running in round-robin mode though, which means that we would only
> + * race with dma i/o.
> + */
> + qemu_mutex_lock_iothread();
Are there definitely no code paths where we might try to do
a page table walk with the iothread already locked ?
> + if (s1->be) {
> + cur_val = ldq_be_p(s1->hphys);
> + if (cur_val == old_val) {
> + stq_be_p(s1->hphys, new_val);
> + }
> + } else {
> + cur_val = ldq_le_p(s1->hphys);
> + if (cur_val == old_val) {
> + stq_le_p(s1->hphys, new_val);
> + }
> + }
> + qemu_mutex_unlock_iothread();
> +#else
> + if (s1->be) {
> + old_val = cpu_to_be64(old_val);
> + new_val = cpu_to_be64(new_val);
> + cur_val = qatomic_cmpxchg__nocheck((uint64_t *)s1->hphys,
> + old_val, new_val);
> + cur_val = be64_to_cpu(cur_val);
> + } else {
> + old_val = cpu_to_le64(old_val);
> + new_val = cpu_to_le64(new_val);
> + cur_val = qatomic_cmpxchg__nocheck((uint64_t *)s1->hphys,
> + old_val, new_val);
> + cur_val = le64_to_cpu(cur_val);
> + }
> +#endif
> +
> + return cur_val;
> +}
> +
> static bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx,
> uint32_t *table, uint32_t address)
> {
> @@ -1290,6 +1343,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
> goto do_fault;
> }
>
> + restart_atomic_update:
> if (!(descriptor & 1) || (!(descriptor & 2) && (level == 3))) {
> /* Invalid, or the Reserved level 3 encoding */
> goto do_translation_fault;
> @@ -1365,10 +1419,28 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
> * Here descaddr is the final physical address, and attributes
> * are all in attrs.
> */
> - if ((attrs & (1 << 10)) == 0) {
> + if ((attrs & (1 << 10)) == 0 && !debug) {
> /* Access flag */
> - fi->type = ARMFault_AccessFlag;
> - goto do_fault;
> + uint64_t new_des, old_des;
> +
> + /*
> + * If HA is disabled, or if the pte is not writable,
> + * pass on the access fault to software.
> + */
> + if (!param.ha || !s1.rw) {
> + fi->type = ARMFault_AccessFlag;
> + goto do_fault;
> + }
> +
> + old_des = descriptor;
> + new_des = descriptor | (1 << 10); /* AF */
> + descriptor = arm_casq_ptw(env, old_des, new_des, &s1, fi);
> + if (fi->type != ARMFault_None) {
> + goto do_fault;
> + }
> + if (old_des != descriptor) {
> + goto restart_atomic_update;
Do we really need to go all the way back to restart_atomic_update?
My reading of the Arm ARM is that we just need the "write the Access bit"
to be atomic, not that everything we do with the descriptor must be atomic.
Though the pseudocode does do a complete loop back like this, so I guess
that's the 'safe' interpretation. Hmm...
> + }
> }
>
> ap = extract32(attrs, 6, 2);
> @@ -1385,8 +1457,43 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
> }
>
> if (!(result->f.prot & (1 << access_type))) {
> - fi->type = ARMFault_Permission;
> - goto do_fault;
> + uint64_t new_des, old_des;
> +
> + /* Writes may set dirty if DBM attribute is set. */
> + if (!param.hd
> + || access_type != MMU_DATA_STORE
> + || !extract64(attrs, 51, 1) /* DBM */
> + || !s1.rw) {
> + fi->type = ARMFault_Permission;
> + goto do_fault;
> + }
> +
> + old_des = descriptor;
> + if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
> + new_des = descriptor | (1ull << 7); /* S2AP[1] */
> + } else {
> + new_des = descriptor & ~(1ull << 7); /* AP[2] */
> + }
If we update the prot bits, we need to also re-calculate the exec bit,
I think, because the execute-never stuff depends on whether the page is
writeable. Alternatively you can do it the way the pseudocode does and
pre-figure-out the final permission bits value (see AArch64.S1ApplyOutputPerms()
and AArch64.S2ApplyOutputPerms()) so you only need to calculate the
exec bit once.
> +
> + /*
> + * If the descriptor didn't change, then attributes weren't the
> + * reason for the permission fault, so deliver it.
> + */
> + if (old_des == new_des) {
> + fi->type = ARMFault_Permission;
> + goto do_fault;
> + }
> +
> + descriptor = arm_casq_ptw(env, old_des, new_des, &s1, fi);
Are we allowed to do the access and dirty bit updates with separate
atomic accesses? This bit in D5.4.12 suggests the expectation is
it will be a single atomic rmw to update the whole thing:
"For a Block or Page Translation Table descriptor for which the AF
bit is 0, the DBM bit is 1, and either the value of the stage 1
AP[2] bit is 1 or the value of the stage 2 S2AP[1] bit is 0, both
AF can be set to 1, and either AP[2] set to 0 or S2AP[1] set to 1,
in a single atomic read-modify-write operation, as a result of an
attempted write to a memory location that uses the translation
table entry."
(though it depends how you interpret that "can"...)
> + if (fi->type != ARMFault_None) {
> + goto do_fault;
> + }
> + if (old_des != descriptor) {
> + goto restart_atomic_update;
> + }
> +
> + /* Success: the page is now writable. */
> + result->f.prot |= 1 << MMU_DATA_STORE;
> }
>
> if (ns) {
> --
> 2.34.1
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread* Re: [PATCH v3 41/42] target/arm: Implement FEAT_HAFDBS
2022-10-07 13:47 ` Peter Maydell
@ 2022-10-07 16:04 ` Peter Maydell
2022-10-07 16:45 ` Richard Henderson
1 sibling, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 16:04 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Fri, 7 Oct 2022 at 14:47, Peter Maydell <peter.maydell@linaro.org> wrote:
> Do we really need to go all the way back to restart_atomic_update?
> Are we allowed to do the access and dirty bit updates with separate
> atomic accesses?
I've just discovered that the latest revision of the Arm ARM (rev I.a)
is clearer on this -- R_SGJBL and I_YZSVV clearly say that we need to
go back to restart_atomic_update for dirty bit updates, and it's a
reasonable assumption that this is true also for atomic updates.
And R_PRHKD says you're permitted to do everything in one rmw,
which clearly implies that you're permitted also not to. And if
you restart the descriptor handling it's architecturally not
distinguishable whether you did one rmw or two. So both of these
are fine the way you have them in your patch.
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v3 41/42] target/arm: Implement FEAT_HAFDBS
2022-10-07 13:47 ` Peter Maydell
2022-10-07 16:04 ` Peter Maydell
@ 2022-10-07 16:45 ` Richard Henderson
2022-10-07 16:50 ` Peter Maydell
1 sibling, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-07 16:45 UTC (permalink / raw)
To: Peter Maydell; +Cc: qemu-devel, qemu-arm
On 10/7/22 06:47, Peter Maydell wrote:
> On Sat, 1 Oct 2022 at 18:04, Richard Henderson
> <richard.henderson@linaro.org> wrote:
>>
>> Perform the atomic update for hardware management of the access flag
>> and the dirty bit.
>>
>> A limitation of the implementation so far is that the page table
>> itself must already be writable, i.e. the dirty bit for the stage2
>> page table must already be set, i.e. we cannot set both dirty bits
>> at the same time.
>>
>> This is allowed because it is CONSTRAINED UNPREDICTABLE whether any
>> atomic update happens at all. The implementation is allowed to simply
>> fall back on software update at any time.
>
> I can't see where in the Arm ARM this is stated.
>
> In any case, if HA is set you can't simply return ARMFault_AccessFlag
> without breaking the bit in D5.4.12 which says
> "When hardware updates of the Access flag are enabled for a stage of
> translation an address translation instruction that uses that stage
> of translation will not report that the address will give rise to
> an Access flag fault in the PAR"
I think this may have been loose (or incorrect) reading of what has become R_QSPMS in I_a,
via "access generates ... a Permission fault", due to the pte page being read-only, due to
the dirty bit being clear.
However, having performed the same atomic update conversion for x86, I can see now that I
merely need to perform the lookup for s1 ptw with MMU_DATA_STORE rather than MMU_DATA_LOAD
in order for the s1 pte page to have its own dirty bit processed and become writable.
>> + t = FIELD_DP64(t, ID_AA64MMFR1, HAFDBS, 2); /* FEAT_HAFDBS */
>
> I think we should split the access flag update and the
> dirty-bit update into separate commits. It might be useful
> for bisection purposes later, and it looks like they're pretty
> cleanly separable. (Though if you look at my last comment in this
> email, maybe not quite so clean as in the code as you have it here.)
Shouldn't be too hard to split. I'll try, at least.
>> +static uint64_t arm_casq_ptw(CPUARMState *env, uint64_t old_val,
>> + uint64_t new_val, const S1TranslateResult *s1,
>> + ARMMMUFaultInfo *fi)
>> +{
>> + uint64_t cur_val;
>> +
>> + if (unlikely(!s1->hphys)) {
>> + fi->type = ARMFault_UnsuppAtomicUpdate;
>> + fi->s1ptw = true;
>> + return 0;
>> + }
>> +
>> +#ifndef CONFIG_ATOMIC64
>> + /*
>> + * We can't support the atomic operation on the host. We should be
>> + * running in round-robin mode though, which means that we would only
>> + * race with dma i/o.
>> + */
>> + qemu_mutex_lock_iothread();
>
> Are there definitely no code paths where we might try to do
> a page table walk with the iothread already locked ?
I'll double-check, but another possibility is to simply perform the atomic operation on
the low 32-bits, where both AF and DB are located. Another trick I learned from x86...
>> + old_des = descriptor;
>> + if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
>> + new_des = descriptor | (1ull << 7); /* S2AP[1] */
>> + } else {
>> + new_des = descriptor & ~(1ull << 7); /* AP[2] */
>> + }
>
> If we update the prot bits, we need to also re-calculate the exec bit,
> I think, because the execute-never stuff depends on whether the page is
> writeable. Alternatively you can do it the way the pseudocode does and
> pre-figure-out the final permission bits value (see AArch64.S1ApplyOutputPerms()
> and AArch64.S2ApplyOutputPerms()) so you only need to calculate the
> exec bit once.
Good catch.
r~
^ permalink raw reply [flat|nested] 90+ messages in thread* Re: [PATCH v3 41/42] target/arm: Implement FEAT_HAFDBS
2022-10-07 16:45 ` Richard Henderson
@ 2022-10-07 16:50 ` Peter Maydell
2022-10-07 17:35 ` Richard Henderson
0 siblings, 1 reply; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 16:50 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Fri, 7 Oct 2022 at 17:45, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 10/7/22 06:47, Peter Maydell wrote:
> > Are there definitely no code paths where we might try to do
> > a page table walk with the iothread already locked ?
>
> I'll double-check, but another possibility is to simply perform the atomic operation on
> the low 32-bits, where both AF and DB are located. Another trick I learned from x86...
Doesn't that cause a problem where we don't detect that some other
CPU wrote to the high 32 bits of the descriptor ? We're supposed to
be using those high 32 bits, not the ones we have in hand...
If we do need the iothread lock, we could do it the way that
io_readx() does, I guess, where we track whether we needed to
lock it or not.
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v3 41/42] target/arm: Implement FEAT_HAFDBS
2022-10-07 16:50 ` Peter Maydell
@ 2022-10-07 17:35 ` Richard Henderson
0 siblings, 0 replies; 90+ messages in thread
From: Richard Henderson @ 2022-10-07 17:35 UTC (permalink / raw)
To: Peter Maydell; +Cc: qemu-devel, qemu-arm
On 10/7/22 09:50, Peter Maydell wrote:
> On Fri, 7 Oct 2022 at 17:45, Richard Henderson
> <richard.henderson@linaro.org> wrote:
>>
>> On 10/7/22 06:47, Peter Maydell wrote:
>>> Are there definitely no code paths where we might try to do
>>> a page table walk with the iothread already locked ?
>>
>> I'll double-check, but another possibility is to simply perform the atomic operation on
>> the low 32-bits, where both AF and DB are located. Another trick I learned from x86...
>
> Doesn't that cause a problem where we don't detect that some other
> CPU wrote to the high 32 bits of the descriptor ? We're supposed to
> be using those high 32 bits, not the ones we have in hand...
Hmm, yes. Which now makes me wonder if the x86 case is in fact buggy...
> If we do need the iothread lock, we could do it the way that
> io_readx() does, I guess, where we track whether we needed to
> lock it or not.
yes.
r~
^ permalink raw reply [flat|nested] 90+ messages in thread
* [PATCH v3 42/42] target/arm: Use the max page size in a 2-stage ptw
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (40 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 41/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
@ 2022-10-01 16:23 ` Richard Henderson
2022-10-07 10:42 ` Peter Maydell
2022-10-10 13:10 ` [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Peter Maydell
42 siblings, 1 reply; 90+ messages in thread
From: Richard Henderson @ 2022-10-01 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-arm, Marc Zyngier
We had only been reporting the stage2 page size. This causes
problems if stage1 is using a larger page size (16k, 2M, etc),
but stage2 is using a smaller page size, because cputlb does
not set large_page_{addr,mask} properly.
Fix by using the max of the two page sizes.
Reported-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/ptw.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 14ab56d1b5..985a5703c3 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -2550,7 +2550,7 @@ static bool get_phys_addr_twostage(CPUARMState *env, target_ulong address,
ARMMMUFaultInfo *fi)
{
hwaddr ipa;
- int s1_prot;
+ int s1_prot, s1_lgpgsz;
bool ret, ipa_secure, s2walk_secure;
ARMCacheAttrs cacheattrs1;
ARMMMUIdx s2_mmu_idx, s2_ptw_idx;
@@ -2592,6 +2592,7 @@ static bool get_phys_addr_twostage(CPUARMState *env, target_ulong address,
* Save the stage1 results so that we may merge prot and cacheattrs later.
*/
s1_prot = result->f.prot;
+ s1_lgpgsz = result->f.lg_page_size;
cacheattrs1 = result->cacheattrs;
memset(result, 0, sizeof(*result));
@@ -2607,6 +2608,14 @@ static bool get_phys_addr_twostage(CPUARMState *env, target_ulong address,
return ret;
}
+ /*
+ * Use the maximum of the S1 & S2 page size, so that invalidation
+ * of pages > TARGET_PAGE_SIZE works correctly.
+ */
+ if (result->f.lg_page_size < s1_lgpgsz) {
+ result->f.lg_page_size = s1_lgpgsz;
+ }
+
/* Combine the S1 and S2 cache attributes. */
hcr = arm_hcr_el2_eff_secstate(env, is_secure);
if (hcr & HCR_DC) {
--
2.34.1
^ permalink raw reply related [flat|nested] 90+ messages in thread* Re: [PATCH v3 42/42] target/arm: Use the max page size in a 2-stage ptw
2022-10-01 16:23 ` [PATCH v3 42/42] target/arm: Use the max page size in a 2-stage ptw Richard Henderson
@ 2022-10-07 10:42 ` Peter Maydell
0 siblings, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-07 10:42 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm, Marc Zyngier
On Sat, 1 Oct 2022 at 18:04, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> We had only been reporting the stage2 page size. This causes
> problems if stage1 is using a larger page size (16k, 2M, etc),
> but stage2 is using a smaller page size, because cputlb does
> not set large_page_{addr,mask} properly.
>
> Fix by using the max of the two page sizes.
>
> Reported-by: Marc Zyngier <maz@kernel.org>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread
* Re: [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS
2022-10-01 16:22 [PATCH v3 00/42] target/arm: Implement FEAT_HAFDBS Richard Henderson
` (41 preceding siblings ...)
2022-10-01 16:23 ` [PATCH v3 42/42] target/arm: Use the max page size in a 2-stage ptw Richard Henderson
@ 2022-10-10 13:10 ` Peter Maydell
42 siblings, 0 replies; 90+ messages in thread
From: Peter Maydell @ 2022-10-10 13:10 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, qemu-arm
On Sat, 1 Oct 2022 at 17:24, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> This is a major reorg to arm page table walking. While the result
> here is "merely" Hardware-assited Access Flag and Dirty Bit Setting
> (HAFDBS), the ultimate goal is the Realm Management Extension (RME).
> RME "recommends" that HAFDBS be implemented (I_CSLWZ).
>
I've taken patches 1-20 plus the extra patch 1.5 into
target-arm.next (last patch taken "Use tlb_set_page_full").
thanks
-- PMM
^ permalink raw reply [flat|nested] 90+ messages in thread