[PATCH v2 0/2] Disable 32-bit EL0 for Apple A10(X), T2

[PATCH v2 0/2] Disable 32-bit EL0 for Apple A10(X), T2

[Nick Chan]

Hi,

Apple's A10(X), T2 SoCs consists of pairs of performance and efficiency
cores. However, only one of the core types may be active at a given time,
and to software, it appears as logical cores that could switch between
P-mode and E-mode, depending on the p-state.

Unforunately, only the performance cores can execute 32-bit EL0. To
software, this results in logical cores that lose ability to execute
32-bit EL0 when the p-state is below a certain value.

Since these CPU cores only supported 16K pages, many AArch32
executables will not run anyways. This series disables 32-bit EL0 for
these SoCs.

Changes since v1:
  - Drop #ifdef CONFIG_ARCH_APPLE, the code to disable NV1 on M2 does
    not use it either.

  - Added comment to explain why 32-bit EL0 have to be disabled.

v1: https://lore.kernel.org/asahi/20240906171449.324354-1-towinchenmi@gmail.com

Nick Chan
---

Nick Chan (2):
  arm64: cputype: Add CPU types for A7-A11, T2 SoCs
  arm64: cpufeature: Pretend that Apple A10 family does not support
    32-bit EL0

 arch/arm64/include/asm/cputype.h | 42 +++++++++++++++++++++++---------
 arch/arm64/kernel/cpufeature.c   | 27 ++++++++++++++++++++
 2 files changed, 57 insertions(+), 12 deletions(-)


base-commit: 9aaeb87ce1e966169a57f53a02ba05b30880ffb8
-- 
2.46.0

[PATCH v2 1/2] arm64: cputype: Add CPU types for A7-A11, T2 SoCs

[Nick Chan]

A10(X), T2 types will be used soon, and the rest are added for
documentation purposes.

The A9 is made in two different fabs and those have different part
numbers, and the TSMC cores are also used in A9X, so it cannot use
the usual naming scheme.

The A10(X), T2 performance/efficiency core pairs appears as single
logical cores to software, so both the performance and efficiency
core codenames needs to be included.

Signed-off-by: Nick Chan <towinchenmi@gmail.com>
---
 arch/arm64/include/asm/cputype.h | 42 +++++++++++++++++++++++---------
 1 file changed, 30 insertions(+), 12 deletions(-)

diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index 5a7dfeb8e8eb..f1720158a54f 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -129,18 +129,27 @@
 
 #define HISI_CPU_PART_TSV110		0xD01
 
-#define APPLE_CPU_PART_M1_ICESTORM	0x022
-#define APPLE_CPU_PART_M1_FIRESTORM	0x023
-#define APPLE_CPU_PART_M1_ICESTORM_PRO	0x024
-#define APPLE_CPU_PART_M1_FIRESTORM_PRO	0x025
-#define APPLE_CPU_PART_M1_ICESTORM_MAX	0x028
-#define APPLE_CPU_PART_M1_FIRESTORM_MAX	0x029
-#define APPLE_CPU_PART_M2_BLIZZARD	0x032
-#define APPLE_CPU_PART_M2_AVALANCHE	0x033
-#define APPLE_CPU_PART_M2_BLIZZARD_PRO	0x034
-#define APPLE_CPU_PART_M2_AVALANCHE_PRO	0x035
-#define APPLE_CPU_PART_M2_BLIZZARD_MAX	0x038
-#define APPLE_CPU_PART_M2_AVALANCHE_MAX	0x039
+#define APPLE_CPU_PART_A7_CYCLONE		0x1
+#define APPLE_CPU_PART_A8_TYPHOON		0x2
+#define APPLE_CPU_PART_A8X_TYPHOON		0x3
+#define APPLE_CPU_PART_SAMSUNG_TWISTER		0x4 /* Used in Samsung A9 */
+#define APPLE_CPU_PART_TSMC_TWISTER		0x5 /* Used in TSMC A9 and A9X */
+#define APPLE_CPU_PART_A10_T2_HURRICANE_ZEPHYR	0x6
+#define APPLE_CPU_PART_A10X_HURRICANE_ZEPHYR	0x7
+#define APPLE_CPU_PART_A11_MONSOON		0x8
+#define APPLE_CPU_PART_A11_MISTRAL		0x9
+#define APPLE_CPU_PART_M1_ICESTORM		0x022
+#define APPLE_CPU_PART_M1_FIRESTORM		0x023
+#define APPLE_CPU_PART_M1_ICESTORM_PRO		0x024
+#define APPLE_CPU_PART_M1_FIRESTORM_PRO		0x025
+#define APPLE_CPU_PART_M1_ICESTORM_MAX		0x028
+#define APPLE_CPU_PART_M1_FIRESTORM_MAX		0x029
+#define APPLE_CPU_PART_M2_BLIZZARD		0x032
+#define APPLE_CPU_PART_M2_AVALANCHE		0x033
+#define APPLE_CPU_PART_M2_BLIZZARD_PRO		0x034
+#define APPLE_CPU_PART_M2_AVALANCHE_PRO		0x035
+#define APPLE_CPU_PART_M2_BLIZZARD_MAX		0x038
+#define APPLE_CPU_PART_M2_AVALANCHE_MAX		0x039
 
 #define AMPERE_CPU_PART_AMPERE1		0xAC3
 #define AMPERE_CPU_PART_AMPERE1A	0xAC4
@@ -200,6 +209,15 @@
 #define MIDR_NVIDIA_CARMEL MIDR_CPU_MODEL(ARM_CPU_IMP_NVIDIA, NVIDIA_CPU_PART_CARMEL)
 #define MIDR_FUJITSU_A64FX MIDR_CPU_MODEL(ARM_CPU_IMP_FUJITSU, FUJITSU_CPU_PART_A64FX)
 #define MIDR_HISI_TSV110 MIDR_CPU_MODEL(ARM_CPU_IMP_HISI, HISI_CPU_PART_TSV110)
+#define MIDR_APPLE_A7_CYCLONE MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_A7_CYCLONE)
+#define MIDR_APPLE_A8_TYPHOON MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_A8_TYPHOON)
+#define MIDR_APPLE_A8X_TYPHOON MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_A8X_TYPHOON)
+#define MIDR_APPLE_SAMSUNG_TWISTER MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_SAMSUNG_TWISTER)
+#define MIDR_APPLE_TSMC_TWISTER MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_TSMC_TWISTER)
+#define MIDR_APPLE_A10_T2_HURRICANE_ZEPHYR MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_A10_T2_HURRICANE_ZEPHYR)
+#define MIDR_APPLE_A10X_HURRICANE_ZEPHYR MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_A10X_HURRICANE_ZEPHYR)
+#define MIDR_APPLE_A11_MONSOON MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_A11_MONSOON)
+#define MIDR_APPLE_A11_MISTRAL MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_A11_MISTRAL)
 #define MIDR_APPLE_M1_ICESTORM MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M1_ICESTORM)
 #define MIDR_APPLE_M1_FIRESTORM MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M1_FIRESTORM)
 #define MIDR_APPLE_M1_ICESTORM_PRO MIDR_CPU_MODEL(ARM_CPU_IMP_APPLE, APPLE_CPU_PART_M1_ICESTORM_PRO)
-- 
2.46.0

[PATCH v2 2/2] arm64: cpufeature: Pretend that Apple A10 family does not support 32-bit EL0

[Nick Chan]

The Apple A10 family consists of physical performance and efficiency
cores, and only one of them can be active at a given time depending on
the current p-state. However, only the performance cores can execute
32-bit EL0. This results in logical cores that can only execute 32-bit
EL0 in high p-states.

Trying to support 32-bit EL0 on a CPU that can only execute it in certain
states is a bad idea. The A10 family only supports 16KB page size anyway
so many AArch32 executables won't run anyways. Pretend that it does not
support 32-bit EL0 at all.

Signed-off-by: Nick Chan <towinchenmi@gmail.com>
---
 arch/arm64/kernel/cpufeature.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 718728a85430..386698f42172 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -3529,6 +3529,31 @@ void __init setup_boot_cpu_features(void)
 	setup_boot_cpu_capabilities();
 }
 
+static void __init bad_aarch32_el0_fixup(void)
+{
+	static const struct midr_range bad_aarch32_el0[] = {
+		MIDR_ALL_VERSIONS(MIDR_APPLE_A10_T2_HURRICANE_ZEPHYR),
+		MIDR_ALL_VERSIONS(MIDR_APPLE_A10X_HURRICANE_ZEPHYR),
+		{}
+	};
+
+	/*
+	 * The Apple A10 family can only execute 32-bit EL0 when in high
+	 * p-states. Pretend it does not support 32-bit EL0.
+	 */
+	if (is_midr_in_range_list(read_cpuid_id(), bad_aarch32_el0)) {
+		struct arm64_ftr_reg *regp;
+
+		regp = get_arm64_ftr_reg(SYS_ID_AA64PFR0_EL1);
+		if (!regp)
+			return;
+		u64 val = (regp->sys_val & ~ID_AA64PFR0_EL1_EL0_MASK)
+		  | ID_AA64PFR0_EL1_EL0_IMP;
+
+		update_cpu_ftr_reg(regp, val);
+	}
+}
+
 static void __init setup_system_capabilities(void)
 {
 	/*
@@ -3562,6 +3587,8 @@ static void __init setup_system_capabilities(void)
 
 void __init setup_system_features(void)
 {
+	bad_aarch32_el0_fixup();
+
 	setup_system_capabilities();
 
 	kpti_install_ng_mappings();
-- 
2.46.0

Re: [PATCH v2 2/2] arm64: cpufeature: Pretend that Apple A10 family does not support 32-bit EL0

[Nick Chan]

On 9/9/2024 17:10, Nick Chan wrote:
> The Apple A10 family consists of physical performance and efficiency
> cores, and only one of them can be active at a given time depending on
> the current p-state. However, only the performance cores can execute
> 32-bit EL0. This results in logical cores that can only execute 32-bit
> EL0 in high p-states.
Further research shows that the MPIDR_EL1 values between the two core
types are different. And whether the two core type have any extra
differences is anyone's guess right now. So far, nothing seems to break
horribly without special workarounds for the MPIDR value (with cpufreq
enabled downstream) as:
1. There are no KVM, GIC, ACPI, PSCI or cpuidle
2. All CPUs switch P-mode and E-mode together

However, all of this is broken enough that this piece of code should go
into arch/arm64/kernel/cpu_errata.c, and also generate a
TAINT_CPU_OUT_OF_SPEC for these cursed CPUs.

> 
> Trying to support 32-bit EL0 on a CPU that can only execute it in certain
> states is a bad idea. The A10 family only supports 16KB page size anyway
> so many AArch32 executables won't run anyways. Pretend that it does not
> support 32-bit EL0 at all.
> 
> Signed-off-by: Nick Chan <towinchenmi@gmail.com>
> ---
>  arch/arm64/kernel/cpufeature.c | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
> 
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 718728a85430..386698f42172 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -3529,6 +3529,31 @@ void __init setup_boot_cpu_features(void)
>  	setup_boot_cpu_capabilities();
>  }
>  
> +static void __init bad_aarch32_el0_fixup(void)
> +{
> +	static const struct midr_range bad_aarch32_el0[] = {
> +		MIDR_ALL_VERSIONS(MIDR_APPLE_A10_T2_HURRICANE_ZEPHYR),
> +		MIDR_ALL_VERSIONS(MIDR_APPLE_A10X_HURRICANE_ZEPHYR),
> +		{}
> +	};
> +
> +	/*
> +	 * The Apple A10 family can only execute 32-bit EL0 when in high
> +	 * p-states. Pretend it does not support 32-bit EL0.
> +	 */
> +	if (is_midr_in_range_list(read_cpuid_id(), bad_aarch32_el0)) {
> +		struct arm64_ftr_reg *regp;
> +
> +		regp = get_arm64_ftr_reg(SYS_ID_AA64PFR0_EL1);
> +		if (!regp)
> +			return;
> +		u64 val = (regp->sys_val & ~ID_AA64PFR0_EL1_EL0_MASK)
> +		  | ID_AA64PFR0_EL1_EL0_IMP;
> +
> +		update_cpu_ftr_reg(regp, val);
> +	}
> +}
> +
>  static void __init setup_system_capabilities(void)
>  {
>  	/*
> @@ -3562,6 +3587,8 @@ static void __init setup_system_capabilities(void)
>  
>  void __init setup_system_features(void)
>  {
> +	bad_aarch32_el0_fixup();
> +
>  	setup_system_capabilities();
>  
>  	kpti_install_ng_mappings();

Nick Chan

Re: [PATCH v2 2/2] arm64: cpufeature: Pretend that Apple A10 family does not support 32-bit EL0

[Catalin Marinas]

On Mon, Sep 16, 2024 at 09:41:12PM +0800, Nick Chan wrote:
> On 9/9/2024 17:10, Nick Chan wrote:
> > The Apple A10 family consists of physical performance and efficiency
> > cores, and only one of them can be active at a given time depending on
> > the current p-state. However, only the performance cores can execute
> > 32-bit EL0. This results in logical cores that can only execute 32-bit
> > EL0 in high p-states.
> 
> Further research shows that the MPIDR_EL1 values between the two core
> types are different. And whether the two core type have any extra
> differences is anyone's guess right now. So far, nothing seems to break
> horribly without special workarounds for the MPIDR value (with cpufreq
> enabled downstream) as:
> 1. There are no KVM, GIC, ACPI, PSCI or cpuidle
> 2. All CPUs switch P-mode and E-mode together
> 
> However, all of this is broken enough that this piece of code should go
> into arch/arm64/kernel/cpu_errata.c, and also generate a
> TAINT_CPU_OUT_OF_SPEC for these cursed CPUs.

I wouldn't carry any additional logic in the kernel for such
configuration (long time ago Arm had something similar, the big.LITTLE
switcher, but the CPUs were fairly similar from a feature perspective).

> > Trying to support 32-bit EL0 on a CPU that can only execute it in certain
> > states is a bad idea. The A10 family only supports 16KB page size anyway
> > so many AArch32 executables won't run anyways. Pretend that it does not
> > support 32-bit EL0 at all.

CONFIG_COMPAT depends on ARM64_4K_PAGES || EXPERT. Do we really need
these patches in case one enables EXPERT and tries to run 32-bit
binaries that never ran on 16K pages before?

-- 
Catalin

Re: [PATCH v2 2/2] arm64: cpufeature: Pretend that Apple A10 family does not support 32-bit EL0

[Nick Chan]

Catalin Marinas 於 2024/9/16 晚上11:30 寫道:
> On Mon, Sep 16, 2024 at 09:41:12PM +0800, Nick Chan wrote:
>> On 9/9/2024 17:10, Nick Chan wrote:
>>> The Apple A10 family consists of physical performance and efficiency
>>> cores, and only one of them can be active at a given time depending on
>>> the current p-state. However, only the performance cores can execute
>>> 32-bit EL0. This results in logical cores that can only execute 32-bit
>>> EL0 in high p-states.
>>
>> Further research shows that the MPIDR_EL1 values between the two core
>> types are different. And whether the two core type have any extra
>> differences is anyone's guess right now. So far, nothing seems to break
>> horribly without special workarounds for the MPIDR value (with cpufreq
>> enabled downstream) as:
>> 1. There are no KVM, GIC, ACPI, PSCI or cpuidle
>> 2. All CPUs switch P-mode and E-mode together
>>
>> However, all of this is broken enough that this piece of code should go
>> into arch/arm64/kernel/cpu_errata.c, and also generate a
>> TAINT_CPU_OUT_OF_SPEC for these cursed CPUs.
> 
> I wouldn't carry any additional logic in the kernel for such
> configuration (long time ago Arm had something similar, the big.LITTLE
> switcher, but the CPUs were fairly similar from a feature perspective).
This is fine from a functionality perspective, currently nothing that
accesses MPIDR after boot is used on A10(X). However, it does not sound
right either to not note that the kernel is running on a cursed CPU.

> 
>>> Trying to support 32-bit EL0 on a CPU that can only execute it in certain
>>> states is a bad idea. The A10 family only supports 16KB page size anyway
>>> so many AArch32 executables won't run anyways. Pretend that it does not
>>> support 32-bit EL0 at all.
> 
> CONFIG_COMPAT depends on ARM64_4K_PAGES || EXPERT. Do we really need
> these patches in case one enables EXPERT and tries to run 32-bit
> binaries that never ran on 16K pages before?
The worst thing that can happen is the 32-bit process crashes with illegal
instruction, the kernel will still be fine.

> 

Nick Chan

Re: [PATCH v2 2/2] arm64: cpufeature: Pretend that Apple A10 family does not support 32-bit EL0

[Catalin Marinas]

On Tue, Sep 17, 2024 at 12:00:10AM +0800, Nick Chan wrote:
> Catalin Marinas 於 2024/9/16 晚上11:30 寫道:
> > On Mon, Sep 16, 2024 at 09:41:12PM +0800, Nick Chan wrote:
> >> On 9/9/2024 17:10, Nick Chan wrote:
> >>> Trying to support 32-bit EL0 on a CPU that can only execute it in certain
> >>> states is a bad idea. The A10 family only supports 16KB page size anyway
> >>> so many AArch32 executables won't run anyways. Pretend that it does not
> >>> support 32-bit EL0 at all.
> > 
> > CONFIG_COMPAT depends on ARM64_4K_PAGES || EXPERT. Do we really need
> > these patches in case one enables EXPERT and tries to run 32-bit
> > binaries that never ran on 16K pages before?
> 
> The worst thing that can happen is the 32-bit process crashes with illegal
> instruction, the kernel will still be fine.

Yes, but that's not my point. By default you do not get CONFIG_COMPAT
enabled with CONFIG_ARM64_16K_PAGES. So these patches are not really
necessary (unless one enabled EXPERT and turns on CONFIG_COMPAT but
these are specialist cases that I don't care about).

-- 
Catalin