* [PATCH v3 01/21] arm64: capabilities: Move errata work around check on boot CPU
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 01/22] arm64: capabilities: Update prototype for enable call back Suzuki K Poulose
` (42 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
We trigger CPU errata work around check on the boot CPU from
smp_prepare_boot_cpu() to make sure that we run the checks only
after the CPU feature infrastructure is initialised. While this
is correct, we can also do this from init_cpu_features() which
initilises the infrastructure, and is called only on the
Boot CPU. This helps to consolidate the CPU capability handling
to cpufeature.c. No functional changes.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 6 ++++++
arch/arm64/kernel/smp.c | 6 ------
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index dc7efed09135..5c08d886b53d 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -545,6 +545,12 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
init_cpu_ftr_reg(SYS_ZCR_EL1, info->reg_zcr);
sve_init_vq_map();
}
+
+ /*
+ * Run the errata work around checks on the boot CPU, once we have
+ * initialised the cpu feature infrastructure.
+ */
+ update_cpu_errata_workarounds();
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 3b8ad7be9c33..5cef11450183 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -448,12 +448,6 @@ void __init smp_prepare_boot_cpu(void)
jump_label_init();
cpuinfo_store_boot_cpu();
save_boot_cpu_run_el();
- /*
- * Run the errata work around checks on the boot CPU, once we have
- * initialised the cpu feature infrastructure from
- * cpuinfo_store_boot_cpu() above.
- */
- update_cpu_errata_workarounds();
}
static u64 __init of_get_cpu_mpidr(struct device_node *dn)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 01/22] arm64: capabilities: Update prototype for enable call back
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 01/21] arm64: capabilities: Move errata work around check on boot CPU Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 02/21] arm64: capabilities: Move errata processing code Suzuki K Poulose
` (41 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
From: Dave Martin <dave.martin@arm.com>
We issue the enable() call back for all CPU hwcaps capabilities
available on the system, on all the CPUs. So far we have ignored
the argument passed to the call back, which had a prototype to
accept a "void *" for use with on_each_cpu() and later with
stop_machine(). However, with commit 0a0d111d40fd1
("arm64: cpufeature: Pass capability structure to ->enable callback"),
there are some users of the argument who wants the matching capability
struct pointer where there are multiple matching criteria for a single
capability. Clean up the declaration of the call back to make it clear.
1) Renamed to cpu_enable(), to imply taking necessary actions on the
called CPU for the entry.
2) Pass const pointer to the capability, to allow the call back to
check the entry. (e.,g to check if any action is needed on the CPU)
3) We don't care about the result of the call back, turning this to
a void.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Andre Przywara <andre.przywara@arm.com>
Cc: James Morse <james.morse@arm.com>
Acked-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Julien Thierry <julien.thierry@arm.com>
Signed-off-by: Dave Martin <dave.martin@arm.com>
[
Rebased to for-next/core converting more users, rename call back,
drop call back results
]
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 7 ++++-
arch/arm64/include/asm/fpsimd.h | 4 ++-
arch/arm64/include/asm/processor.h | 7 ++---
arch/arm64/kernel/cpu_errata.c | 51 +++++++++++++++++--------------------
arch/arm64/kernel/cpufeature.c | 41 ++++++++++++++++-------------
arch/arm64/kernel/fpsimd.c | 5 ++--
arch/arm64/kernel/traps.c | 4 +--
arch/arm64/mm/fault.c | 3 +--
8 files changed, 65 insertions(+), 57 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index ac67cfc2585a..f46eb7d1625f 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -97,7 +97,12 @@ struct arm64_cpu_capabilities {
u16 capability;
int def_scope; /* default scope */
bool (*matches)(const struct arm64_cpu_capabilities *caps, int scope);
- int (*enable)(void *); /* Called on all active CPUs */
+ /*
+ * Take the appropriate actions to enable this capability for this CPU.
+ * For each successfully booted CPU, this method is called for each
+ * globally detected capability.
+ */
+ void (*cpu_enable)(const struct arm64_cpu_capabilities *cap);
union {
struct { /* To be used for erratum handling only */
u32 midr_model;
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 8857a0f0d0f7..7623762f7fa6 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -83,7 +83,9 @@ extern void sve_save_state(void *state, u32 *pfpsr);
extern void sve_load_state(void const *state, u32 const *pfpsr,
unsigned long vq_minus_1);
extern unsigned int sve_get_vl(void);
-extern int sve_kernel_enable(void *);
+
+struct arm64_cpu_capabilities;
+extern void sve_kernel_enable(const struct arm64_cpu_capabilities *__unused);
extern int __ro_after_init sve_max_vl;
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index dfefbf3fd9e3..60fdf1d59722 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -37,6 +37,7 @@
#include <linux/string.h>
#include <asm/alternative.h>
+#include <asm/cpufeature.h>
#include <asm/fpsimd.h>
#include <asm/hw_breakpoint.h>
#include <asm/lse.h>
@@ -217,9 +218,9 @@ static inline void spin_lock_prefetch(const void *ptr)
#endif
-int cpu_enable_pan(void *__unused);
-int cpu_enable_cache_maint_trap(void *__unused);
-int cpu_clear_disr(void *__unused);
+void cpu_enable_pan(const struct arm64_cpu_capabilities *__unused);
+void cpu_enable_cache_maint_trap(const struct arm64_cpu_capabilities *__unused);
+void cpu_clear_disr(const struct arm64_cpu_capabilities *__unused);
/* Userspace interface for PR_SVE_{SET,GET}_VL prctl()s: */
#define SVE_SET_VL(arg) sve_set_current_vl(arg)
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 07823595b7f0..51f969b7e770 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -53,11 +53,11 @@ has_mismatched_cache_line_size(const struct arm64_cpu_capabilities *entry,
(arm64_ftr_reg_ctrel0.sys_val & arm64_ftr_reg_ctrel0.strict_mask);
}
-static int cpu_enable_trap_ctr_access(void *__unused)
+static void
+cpu_enable_trap_ctr_access(const struct arm64_cpu_capabilities *__unused)
{
/* Clear SCTLR_EL1.UCT */
config_sctlr_el1(SCTLR_EL1_UCT, 0);
- return 0;
}
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
@@ -161,25 +161,25 @@ static void call_hvc_arch_workaround_1(void)
arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL);
}
-static int enable_smccc_arch_workaround_1(void *data)
+static void
+enable_smccc_arch_workaround_1(const struct arm64_cpu_capabilities *entry)
{
- const struct arm64_cpu_capabilities *entry = data;
bp_hardening_cb_t cb;
void *smccc_start, *smccc_end;
struct arm_smccc_res res;
if (!entry->matches(entry, SCOPE_LOCAL_CPU))
- return 0;
+ return;
if (psci_ops.smccc_version == SMCCC_VERSION_1_0)
- return 0;
+ return;
switch (psci_ops.conduit) {
case PSCI_CONDUIT_HVC:
arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
ARM_SMCCC_ARCH_WORKAROUND_1, &res);
if (res.a0)
- return 0;
+ return;
cb = call_hvc_arch_workaround_1;
smccc_start = __smccc_workaround_1_hvc_start;
smccc_end = __smccc_workaround_1_hvc_end;
@@ -189,19 +189,19 @@ static int enable_smccc_arch_workaround_1(void *data)
arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
ARM_SMCCC_ARCH_WORKAROUND_1, &res);
if (res.a0)
- return 0;
+ return;
cb = call_smc_arch_workaround_1;
smccc_start = __smccc_workaround_1_smc_start;
smccc_end = __smccc_workaround_1_smc_end;
break;
default:
- return 0;
+ return;
}
install_bp_hardening_cb(entry, cb, smccc_start, smccc_end);
- return 0;
+ return;
}
static void qcom_link_stack_sanitization(void)
@@ -216,15 +216,12 @@ static void qcom_link_stack_sanitization(void)
: "=&r" (tmp));
}
-static int qcom_enable_link_stack_sanitization(void *data)
+static void
+qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
{
- const struct arm64_cpu_capabilities *entry = data;
-
install_bp_hardening_cb(entry, qcom_link_stack_sanitization,
__qcom_hyp_sanitize_link_stack_start,
__qcom_hyp_sanitize_link_stack_end);
-
- return 0;
}
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
@@ -251,7 +248,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "ARM errata 826319, 827319, 824069",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x02),
- .enable = cpu_enable_cache_maint_trap,
+ .cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_819472
@@ -260,7 +257,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "ARM errata 819472",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x01),
- .enable = cpu_enable_cache_maint_trap,
+ .cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_832075
@@ -341,7 +338,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.capability = ARM64_MISMATCHED_CACHE_LINE_SIZE,
.matches = has_mismatched_cache_line_size,
.def_scope = SCOPE_LOCAL_CPU,
- .enable = cpu_enable_trap_ctr_access,
+ .cpu_enable = cpu_enable_trap_ctr_access,
},
#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003
{
@@ -380,27 +377,27 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
- .enable = qcom_enable_link_stack_sanitization,
+ .cpu_enable = qcom_enable_link_stack_sanitization,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
@@ -409,12 +406,12 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
#endif
{
@@ -432,8 +429,8 @@ void verify_local_cpu_errata_workarounds(void)
for (; caps->matches; caps++) {
if (cpus_have_cap(caps->capability)) {
- if (caps->enable)
- caps->enable((void *)caps);
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
} else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
pr_crit("CPU%d: Requires work around for %s, not detected"
" at boot time\n",
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 0ff8ab79352c..dc7efed09135 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -891,7 +891,8 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
ID_AA64PFR0_CSV3_SHIFT);
}
-static int kpti_install_ng_mappings(void *__unused)
+static void
+kpti_install_ng_mappings(const struct arm64_cpu_capabilities *__unused)
{
typedef void (kpti_remap_fn)(int, int, phys_addr_t);
extern kpti_remap_fn idmap_kpti_install_ng_mappings;
@@ -901,7 +902,7 @@ static int kpti_install_ng_mappings(void *__unused)
int cpu = smp_processor_id();
if (kpti_applied)
- return 0;
+ return;
remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
@@ -912,7 +913,7 @@ static int kpti_install_ng_mappings(void *__unused)
if (!cpu)
kpti_applied = true;
- return 0;
+ return;
}
static int __init parse_kpti(char *str)
@@ -929,7 +930,7 @@ static int __init parse_kpti(char *str)
__setup("kpti=", parse_kpti);
#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */
-static int cpu_copy_el2regs(void *__unused)
+static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused)
{
/*
* Copy register values that aren't redirected by hardware.
@@ -941,8 +942,6 @@ static int cpu_copy_el2regs(void *__unused)
*/
if (!alternatives_applied)
write_sysreg(read_sysreg(tpidr_el1), tpidr_el2);
-
- return 0;
}
static const struct arm64_cpu_capabilities arm64_features[] = {
@@ -966,7 +965,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.field_pos = ID_AA64MMFR1_PAN_SHIFT,
.sign = FTR_UNSIGNED,
.min_field_value = 1,
- .enable = cpu_enable_pan,
+ .cpu_enable = cpu_enable_pan,
},
#endif /* CONFIG_ARM64_PAN */
#if defined(CONFIG_AS_LSE) && defined(CONFIG_ARM64_LSE_ATOMICS)
@@ -1014,7 +1013,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.capability = ARM64_HAS_VIRT_HOST_EXTN,
.def_scope = SCOPE_SYSTEM,
.matches = runs_at_el2,
- .enable = cpu_copy_el2regs,
+ .cpu_enable = cpu_copy_el2regs,
},
{
.desc = "32-bit EL0 Support",
@@ -1038,7 +1037,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
.def_scope = SCOPE_SYSTEM,
.matches = unmap_kernel_at_el0,
- .enable = kpti_install_ng_mappings,
+ .cpu_enable = kpti_install_ng_mappings,
},
#endif
{
@@ -1069,7 +1068,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.field_pos = ID_AA64PFR0_SVE_SHIFT,
.min_field_value = ID_AA64PFR0_SVE,
.matches = has_cpuid_feature,
- .enable = sve_kernel_enable,
+ .cpu_enable = sve_kernel_enable,
},
#endif /* CONFIG_ARM64_SVE */
#ifdef CONFIG_ARM64_RAS_EXTN
@@ -1082,7 +1081,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.sign = FTR_UNSIGNED,
.field_pos = ID_AA64PFR0_RAS_SHIFT,
.min_field_value = ID_AA64PFR0_RAS_V1,
- .enable = cpu_clear_disr,
+ .cpu_enable = cpu_clear_disr,
},
#endif /* CONFIG_ARM64_RAS_EXTN */
{},
@@ -1226,6 +1225,14 @@ void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
}
}
+static int __enable_cpu_capability(void *arg)
+{
+ const struct arm64_cpu_capabilities *cap = arg;
+
+ cap->cpu_enable(cap);
+ return 0;
+}
+
/*
* Run through the enabled capabilities and enable() it on all active
* CPUs
@@ -1241,14 +1248,14 @@ void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
/* Ensure cpus_have_const_cap(num) works */
static_branch_enable(&cpu_hwcap_keys[num]);
- if (caps->enable) {
+ if (caps->cpu_enable) {
/*
* Use stop_machine() as it schedules the work allowing
* us to modify PSTATE, instead of on_each_cpu() which
* uses an IPI, giving us a PSTATE that disappears when
* we return.
*/
- stop_machine(caps->enable, (void *)caps, cpu_online_mask);
+ stop_machine(__enable_cpu_capability, (void *)caps, cpu_online_mask);
}
}
}
@@ -1291,8 +1298,8 @@ verify_local_cpu_features(const struct arm64_cpu_capabilities *caps_list)
smp_processor_id(), caps->desc);
cpu_die_early();
}
- if (caps->enable)
- caps->enable((void *)caps);
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
}
}
@@ -1515,10 +1522,8 @@ static int __init enable_mrs_emulation(void)
core_initcall(enable_mrs_emulation);
-int cpu_clear_disr(void *__unused)
+void cpu_clear_disr(const struct arm64_cpu_capabilities *__unused)
{
/* Firmware may have left a deferred SError in this register. */
write_sysreg_s(0, SYS_DISR_EL1);
-
- return 0;
}
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 55fb544072f6..35c563536b0c 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -40,6 +40,7 @@
#include <linux/sysctl.h>
#include <asm/fpsimd.h>
+#include <asm/cpufeature.h>
#include <asm/cputype.h>
#include <asm/simd.h>
#include <asm/sigcontext.h>
@@ -757,12 +758,10 @@ static void __init sve_efi_setup(void)
* Enable SVE for EL1.
* Intended for use by the cpufeatures code during CPU boot.
*/
-int sve_kernel_enable(void *__always_unused p)
+void sve_kernel_enable(const struct arm64_cpu_capabilities *__always_unused p)
{
write_sysreg(read_sysreg(CPACR_EL1) | CPACR_EL1_ZEN_EL1EN, CPACR_EL1);
isb();
-
- return 0;
}
void __init sve_setup(void)
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index bbb0fde2780e..a2f6492b4ea3 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -38,6 +38,7 @@
#include <asm/atomic.h>
#include <asm/bug.h>
+#include <asm/cpufeature.h>
#include <asm/daifflags.h>
#include <asm/debug-monitors.h>
#include <asm/esr.h>
@@ -374,10 +375,9 @@ asmlinkage void __exception do_undefinstr(struct pt_regs *regs)
force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
}
-int cpu_enable_cache_maint_trap(void *__unused)
+void cpu_enable_cache_maint_trap(const struct arm64_cpu_capabilities *__unused)
{
config_sctlr_el1(SCTLR_EL1_UCI, 0);
- return 0;
}
#define __user_cache_maint(insn, address, res) \
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 8fd141233dec..af781bba7cc9 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -828,7 +828,7 @@ asmlinkage int __exception do_debug_exception(unsigned long addr,
NOKPROBE_SYMBOL(do_debug_exception);
#ifdef CONFIG_ARM64_PAN
-int cpu_enable_pan(void *__unused)
+void cpu_enable_pan(const struct arm64_cpu_capabilities *__unused)
{
/*
* We modify PSTATE. This won't work from irq context as the PSTATE
@@ -838,6 +838,5 @@ int cpu_enable_pan(void *__unused)
config_sctlr_el1(SCTLR_EL1_SPAN, 0);
asm(SET_PSTATE_PAN(1));
- return 0;
}
#endif /* CONFIG_ARM64_PAN */
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 02/21] arm64: capabilities: Move errata processing code
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 01/21] arm64: capabilities: Move errata work around check on boot CPU Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 01/22] arm64: capabilities: Update prototype for enable call back Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 02/22] arm64: capabilities: Move errata work around check on boot CPU Suzuki K Poulose
` (40 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
We have errata work around processing code in cpu_errata.c,
which calls back into helpers defined in cpufeature.c. Now
that we are going to make the handling of capabilities
generic, by adding the information to each capability,
move the errata work around specific processing code.
No functional changes.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 7 -------
arch/arm64/kernel/cpu_errata.c | 33 ------------------------------
arch/arm64/kernel/cpufeature.c | 40 +++++++++++++++++++++++++++++++++++--
3 files changed, 38 insertions(+), 42 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index f46eb7d1625f..7925e40c6ded 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -238,15 +238,8 @@ static inline bool id_aa64pfr0_sve(u64 pfr0)
}
void __init setup_cpu_features(void);
-
-void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- const char *info);
-void enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps);
void check_local_cpu_capabilities(void);
-void update_cpu_errata_workarounds(void);
-void __init enable_errata_workarounds(void);
-void verify_local_cpu_errata_workarounds(void);
u64 read_sanitised_ftr_reg(u32 id);
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 51f969b7e770..0186788437a7 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -417,36 +417,3 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
}
};
-
-/*
- * The CPU Errata work arounds are detected and applied at boot time
- * and the related information is freed soon after. If the new CPU requires
- * an errata not detected at boot, fail this CPU.
- */
-void verify_local_cpu_errata_workarounds(void)
-{
- const struct arm64_cpu_capabilities *caps = arm64_errata;
-
- for (; caps->matches; caps++) {
- if (cpus_have_cap(caps->capability)) {
- if (caps->cpu_enable)
- caps->cpu_enable(caps);
- } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
- pr_crit("CPU%d: Requires work around for %s, not detected"
- " at boot time\n",
- smp_processor_id(),
- caps->desc ? : "an erratum");
- cpu_die_early();
- }
- }
-}
-
-void update_cpu_errata_workarounds(void)
-{
- update_cpu_capabilities(arm64_errata, "enabling workaround for");
-}
-
-void __init enable_errata_workarounds(void)
-{
- enable_cpu_capabilities(arm64_errata);
-}
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 5c08d886b53d..359fd6b4d159 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -503,6 +503,9 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
reg->user_mask = user_mask;
}
+extern const struct arm64_cpu_capabilities arm64_errata[];
+static void update_cpu_errata_workarounds(void);
+
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
/* Before we start using the tables, make sure it is sorted */
@@ -1218,7 +1221,7 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
return false;
}
-void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
const char *info)
{
for (; caps->matches; caps++) {
@@ -1243,7 +1246,7 @@ static int __enable_cpu_capability(void *arg)
* Run through the enabled capabilities and enable() it on all active
* CPUs
*/
-void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
+static void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
{
for (; caps->matches; caps++) {
unsigned int num = caps->capability;
@@ -1326,6 +1329,39 @@ static void verify_sve_features(void)
/* Add checks on other ZCR bits here if necessary */
}
+/*
+ * The CPU Errata work arounds are detected and applied at boot time
+ * and the related information is freed soon after. If the new CPU requires
+ * an errata not detected at boot, fail this CPU.
+ */
+static void verify_local_cpu_errata_workarounds(void)
+{
+ const struct arm64_cpu_capabilities *caps = arm64_errata;
+
+ for (; caps->matches; caps++) {
+ if (cpus_have_cap(caps->capability)) {
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
+ } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
+ pr_crit("CPU%d: Requires work around for %s, not detected"
+ " at boot time\n",
+ smp_processor_id(),
+ caps->desc ? : "an erratum");
+ cpu_die_early();
+ }
+ }
+}
+
+static void update_cpu_errata_workarounds(void)
+{
+ update_cpu_capabilities(arm64_errata, "enabling workaround for");
+}
+
+static void __init enable_errata_workarounds(void)
+{
+ enable_cpu_capabilities(arm64_errata);
+}
+
/*
* Run through the enabled system capabilities and enable() it on this CPU.
* The capabilities were decided based on the available CPUs at the boot time.
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 02/22] arm64: capabilities: Move errata work around check on boot CPU
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (2 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 02/21] arm64: capabilities: Move errata processing code Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 03/22] arm64: capabilities: Move errata processing code Suzuki K Poulose
` (39 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
We trigger CPU errata work around check on the boot CPU from
smp_prepare_boot_cpu() to make sure that we run the checks only
after the CPU feature infrastructure is initialised. While this
is correct, we can also do this from init_cpu_features() which
initilises the infrastructure, and is called only on the
Boot CPU. This helps to consolidate the CPU capability handling
to cpufeature.c. No functional changes.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 6 ++++++
arch/arm64/kernel/smp.c | 6 ------
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index dc7efed09135..5c08d886b53d 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -545,6 +545,12 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
init_cpu_ftr_reg(SYS_ZCR_EL1, info->reg_zcr);
sve_init_vq_map();
}
+
+ /*
+ * Run the errata work around checks on the boot CPU, once we have
+ * initialised the cpu feature infrastructure.
+ */
+ update_cpu_errata_workarounds();
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 3b8ad7be9c33..5cef11450183 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -448,12 +448,6 @@ void __init smp_prepare_boot_cpu(void)
jump_label_init();
cpuinfo_store_boot_cpu();
save_boot_cpu_run_el();
- /*
- * Run the errata work around checks on the boot CPU, once we have
- * initialised the cpu feature infrastructure from
- * cpuinfo_store_boot_cpu() above.
- */
- update_cpu_errata_workarounds();
}
static u64 __init of_get_cpu_mpidr(struct device_node *dn)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 03/22] arm64: capabilities: Move errata processing code
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (3 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 02/22] arm64: capabilities: Move errata work around check on boot CPU Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 03/21] arm64: capabilities: Prepare for fine grained capabilities Suzuki K Poulose
` (38 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
We have errata work around processing code in cpu_errata.c,
which calls back into helpers defined in cpufeature.c. Now
that we are going to make the handling of capabilities
generic, by adding the information to each capability,
move the errata work around specific processing code.
No functional changes.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 7 -------
arch/arm64/kernel/cpu_errata.c | 33 ------------------------------
arch/arm64/kernel/cpufeature.c | 40 +++++++++++++++++++++++++++++++++++--
3 files changed, 38 insertions(+), 42 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index f46eb7d1625f..7925e40c6ded 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -238,15 +238,8 @@ static inline bool id_aa64pfr0_sve(u64 pfr0)
}
void __init setup_cpu_features(void);
-
-void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- const char *info);
-void enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps);
void check_local_cpu_capabilities(void);
-void update_cpu_errata_workarounds(void);
-void __init enable_errata_workarounds(void);
-void verify_local_cpu_errata_workarounds(void);
u64 read_sanitised_ftr_reg(u32 id);
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 51f969b7e770..0186788437a7 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -417,36 +417,3 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
}
};
-
-/*
- * The CPU Errata work arounds are detected and applied at boot time
- * and the related information is freed soon after. If the new CPU requires
- * an errata not detected at boot, fail this CPU.
- */
-void verify_local_cpu_errata_workarounds(void)
-{
- const struct arm64_cpu_capabilities *caps = arm64_errata;
-
- for (; caps->matches; caps++) {
- if (cpus_have_cap(caps->capability)) {
- if (caps->cpu_enable)
- caps->cpu_enable(caps);
- } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
- pr_crit("CPU%d: Requires work around for %s, not detected"
- " at boot time\n",
- smp_processor_id(),
- caps->desc ? : "an erratum");
- cpu_die_early();
- }
- }
-}
-
-void update_cpu_errata_workarounds(void)
-{
- update_cpu_capabilities(arm64_errata, "enabling workaround for");
-}
-
-void __init enable_errata_workarounds(void)
-{
- enable_cpu_capabilities(arm64_errata);
-}
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 5c08d886b53d..359fd6b4d159 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -503,6 +503,9 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
reg->user_mask = user_mask;
}
+extern const struct arm64_cpu_capabilities arm64_errata[];
+static void update_cpu_errata_workarounds(void);
+
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
/* Before we start using the tables, make sure it is sorted */
@@ -1218,7 +1221,7 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
return false;
}
-void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
const char *info)
{
for (; caps->matches; caps++) {
@@ -1243,7 +1246,7 @@ static int __enable_cpu_capability(void *arg)
* Run through the enabled capabilities and enable() it on all active
* CPUs
*/
-void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
+static void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
{
for (; caps->matches; caps++) {
unsigned int num = caps->capability;
@@ -1326,6 +1329,39 @@ static void verify_sve_features(void)
/* Add checks on other ZCR bits here if necessary */
}
+/*
+ * The CPU Errata work arounds are detected and applied at boot time
+ * and the related information is freed soon after. If the new CPU requires
+ * an errata not detected at boot, fail this CPU.
+ */
+static void verify_local_cpu_errata_workarounds(void)
+{
+ const struct arm64_cpu_capabilities *caps = arm64_errata;
+
+ for (; caps->matches; caps++) {
+ if (cpus_have_cap(caps->capability)) {
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
+ } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
+ pr_crit("CPU%d: Requires work around for %s, not detected"
+ " at boot time\n",
+ smp_processor_id(),
+ caps->desc ? : "an erratum");
+ cpu_die_early();
+ }
+ }
+}
+
+static void update_cpu_errata_workarounds(void)
+{
+ update_cpu_capabilities(arm64_errata, "enabling workaround for");
+}
+
+static void __init enable_errata_workarounds(void)
+{
+ enable_cpu_capabilities(arm64_errata);
+}
+
/*
* Run through the enabled system capabilities and enable() it on this CPU.
* The capabilities were decided based on the available CPUs at the boot time.
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 03/21] arm64: capabilities: Prepare for fine grained capabilities
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (4 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 03/22] arm64: capabilities: Move errata processing code Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 04/21] arm64: capabilities: Add flags to handle the conflicts on late CPU Suzuki K Poulose
` (37 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
We use arm64_cpu_capabilities to represent CPU ELF HWCAPs exposed
to the userspace and the CPU hwcaps used by the kernel, which
include cpu features and CPU errata work arounds. Capabilities
have some properties that decide how they should be treated :
1) Detection, i.e scope : A cap could be "detected" either :
- if it is present on at least one CPU (SCOPE_LOCAL_CPU)
Or
- if it is present on all the CPUs (SCOPE_SYSTEM)
2) When is it enabled ? - A cap is treated as "enabled" when the
system takes some action based on whether the capability is detected or
not. e.g, setting some control register, patching the kernel code.
Right now, we treat all caps are enabled at boot-time, after all
the CPUs are brought up by the kernel. But there are certain caps,
which are enabled early during the boot (e.g, VHE, GIC_CPUIF for NMI)
and kernel starts using them, even before the secondary CPUs are brought
up. We would need a way to describe this for each capability.
3) Conflict on a late CPU - When a CPU is brought up, it is checked
against the caps that are known to be enabled on the system (via
verify_local_cpu_capabilities()). Based on the state of the capability
on the CPU vs. that of System we could have the following combinations
of conflict.
x-----------------------------x
| Type | System | Late CPU |
------------------------------|
| a | y | n |
------------------------------|
| b | n | y |
x-----------------------------x
Case (a) is not permitted for caps which are system features, which the
system expects all the CPUs to have (e.g VHE). While (a) is ignored for
all errata work arounds. However, there could be exceptions to the plain
filtering approach. e.g, KPTI is an optional feature for a late CPU as
long as the system already enables it.
Case (b) is not permitted for errata work arounds which requires some
work around, which cannot be delayed. And we ignore (b) for features. Here,
yet again, KPTI is an exception, where if a late CPU needs KPTI we are too
late to enable it (because we change the allocation of ASIDs etc).
So this calls for a lot more fine grained behavior for each capability.
And if we define all the attributes to control their behavior properly,
we may be able to use a single table for the CPU hwcaps (which cover
errata and features, not the ELF HWCAPs). This is a prepartory step
to get there. More bits would be added for the properties listed above.
We are going to use a bit-mask to encode all the properties of a capabilities.
This patch encodes the "SCOPE" of the capability.
As such there is no change in how the capabilities are treated.
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 99 ++++++++++++++++++++++++++++++++++---
arch/arm64/kernel/cpu_errata.c | 8 +--
arch/arm64/kernel/cpufeature.c | 38 +++++++-------
3 files changed, 116 insertions(+), 29 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 7925e40c6ded..a2b708b9a6e3 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -86,16 +86,98 @@ struct arm64_ftr_reg {
extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
-/* scope of capability check */
-enum {
- SCOPE_SYSTEM,
- SCOPE_LOCAL_CPU,
-};
+/*
+ * CPU capabilities:
+ *
+ * We use arm64_cpu_capabilities to represent system features, errata work
+ * arounds (both used internally by kernel and tracked in cpu_hwcaps) and
+ * ELF HWCAPs (which are exposed to user).
+ *
+ * To support systems with heterogeneous CPUs, we need to make sure that we
+ * detect the capabilities correctly on the system and take appropriate
+ * measures to ensure there are not incompatibilities.
+ *
+ * This comment tries to explain how we treat the capabilities.
+ * Each capability has the following list of attributes :
+ *
+ * 1) Scope of Detection : The system detects a given capability by performing
+ * some checks at runtime. This could be, e.g, checking the value of a field
+ * in CPU ID feature register or checking the cpu model. The capability
+ * provides a call back ( @matches() ) to perform the check.
+ * Scope defines how the checks should be performed. There are two cases:
+ *
+ * a) SCOPE_LOCAL_CPU: check all the CPUs and "detect" if at least one
+ * matches. This implies, we have to run the check on all the booting
+ * CPUs, until the system decides that state of the capability is finalised.
+ * (See section 2 below)
+ * Or
+ * b) SCOPE_SYSTEM: check all the CPUs and "detect" if all the CPUs matches.
+ * This implies, we run the check only once, when the system decides to
+ * finalise the state of the capability. If the capability relies on a
+ * field in one of the CPU ID feature registers, we use the sanitised
+ * value of the register from the CPU feature infrastructure to make
+ * the decision.
+ * The process of detection is usually denoted by "update" capability state
+ * in the code.
+ *
+ * 2) Finalise the state : The kernel should finalise the state of a capability
+ * at some point during its execution and take necessary actions if any. Usually,
+ * this is done, after all the boot-time enabled CPUs are brought up by the
+ * kernel, so that it can make better decision based on the available set
+ * of CPUs. However, there are some special cases, where the action is taken
+ * during the early boot by the primary boot CPU. (e.g, running the kernel at
+ * EL2 with Virtualisation Host Extensions). The kernel usually disallows
+ * any changes to the state of a capability once it finalises the capability
+ * and takes any action, as it may be impossible to execute the actions safely.
+ * A CPU brought up after a capability is "finalised" is referred to as "Late CPU"
+ * w.r.t the capability. e.g, all secondary CPUs are treated "late CPUs" for
+ * capabilities determined by the boot CPU.
+ *
+ * 3) Verification: When a CPU is brought online (e.g, by user or by the kernel),
+ * the kernel should make sure that it is safe to use the CPU, by verifying
+ * that the CPU is compliant with the state of the capabilities finalised
+ * already. This happens via :
+ *
+ * secondary_start_kernel()-> check_local_cpu_capabilities()
+ *
+ * As explained in (2) above, capabilities could be finalised at different
+ * points in the execution. Each CPU is verified against the "finalised"
+ * capabilities and if there is a conflict, the kernel takes an action, based
+ * on the severity (e.g, a CPU could be prevented from booting or cause a
+ * kernel panic). The CPU is allowed to "affect" the state of the capability,
+ * if it has not been finalised already.
+ *
+ * 4) Action: As mentioned in (2), the kernel can take an action for each detected
+ * capability, on all CPUs on the system. Appropriate actions include, turning
+ * on an architectural feature, modifying the control registers (e.g, SCTLR,
+ * TCR etc.) or patching the kernel via alternatives. The kernel patching is
+ * batched and performed at later point. The actions are alwayss initiated only
+ * after the capability is finalised. This is usally denoted by "enabling" the
+ * capability. The actions are initiated as follows :
+ * a) Action is triggered on all online CPUs, after the capability is
+ * finalised, invoked within the stop_machine() context from
+ * enable_cpu_capabilitie().
+ *
+ * b) Any late CPU, brought up after (1), the action is triggered via:
+ * check_local_cpu_capabilities() -> verify_local_cpu_capabilities()
+ *
+ */
+
+
+/* Decide how the capability is detected. On a local CPU vs System wide */
+#define ARM64_CPUCAP_SCOPE_LOCAL_CPU ((u16)BIT(0))
+#define ARM64_CPUCAP_SCOPE_SYSTEM ((u16)BIT(1))
+#define ARM64_CPUCAP_SCOPE_MASK \
+ (ARM64_CPUCAP_SCOPE_SYSTEM | \
+ ARM64_CPUCAP_SCOPE_LOCAL_CPU)
+
+#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
+#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
- int def_scope; /* default scope */
+ u16 type;
bool (*matches)(const struct arm64_cpu_capabilities *caps, int scope);
/*
* Take the appropriate actions to enable this capability for this CPU.
@@ -120,6 +202,11 @@ struct arm64_cpu_capabilities {
};
};
+static inline int cpucap_default_scope(const struct arm64_cpu_capabilities *cap)
+{
+ return cap->type & ARM64_CPUCAP_SCOPE_MASK;
+}
+
extern DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
extern struct static_key_false arm64_const_caps_ready;
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 0186788437a7..3d3893bbc2dc 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -226,14 +226,14 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
#define MIDR_RANGE(model, min, max) \
- .def_scope = SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = min, \
.midr_range_max = max
#define MIDR_ALL_VERSIONS(model) \
- .def_scope = SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = 0, \
@@ -337,7 +337,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "Mismatched cache line size",
.capability = ARM64_MISMATCHED_CACHE_LINE_SIZE,
.matches = has_mismatched_cache_line_size,
- .def_scope = SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
.cpu_enable = cpu_enable_trap_ctr_access,
},
#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003
@@ -351,7 +351,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Kryo erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
- .def_scope = SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
.midr_model = MIDR_QCOM_KRYO,
.matches = is_kryo_midr,
},
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 359fd6b4d159..9065a5974a3b 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -957,7 +957,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "GIC system register CPU interface",
.capability = ARM64_HAS_SYSREG_GIC_CPUIF,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_useable_gicv3_cpuif,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.field_pos = ID_AA64PFR0_GIC_SHIFT,
@@ -968,7 +968,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Privileged Access Never",
.capability = ARM64_HAS_PAN,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR1_EL1,
.field_pos = ID_AA64MMFR1_PAN_SHIFT,
@@ -981,7 +981,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "LSE atomic instructions",
.capability = ARM64_HAS_LSE_ATOMICS,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR0_EL1,
.field_pos = ID_AA64ISAR0_ATOMICS_SHIFT,
@@ -992,14 +992,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Software prefetching using PRFM",
.capability = ARM64_HAS_NO_HW_PREFETCH,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_no_hw_prefetch,
},
#ifdef CONFIG_ARM64_UAO
{
.desc = "User Access Override",
.capability = ARM64_HAS_UAO,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR2_EL1,
.field_pos = ID_AA64MMFR2_UAO_SHIFT,
@@ -1013,21 +1013,21 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_PAN
{
.capability = ARM64_ALT_PAN_NOT_UAO,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = cpufeature_pan_not_uao,
},
#endif /* CONFIG_ARM64_PAN */
{
.desc = "Virtualization Host Extensions",
.capability = ARM64_HAS_VIRT_HOST_EXTN,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
{
.desc = "32-bit EL0 Support",
.capability = ARM64_HAS_32BIT_EL0,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1037,14 +1037,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Reduced HYP mapping offset",
.capability = ARM64_HYP_OFFSET_LOW,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = hyp_offset_low,
},
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
{
.desc = "Kernel page table isolation (KPTI)",
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = unmap_kernel_at_el0,
.cpu_enable = kpti_install_ng_mappings,
},
@@ -1052,7 +1052,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
/* FP/SIMD is not implemented */
.capability = ARM64_HAS_NO_FPSIMD,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.min_field_value = 0,
.matches = has_no_fpsimd,
},
@@ -1060,7 +1060,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Data cache clean to Point of Persistence",
.capability = ARM64_HAS_DCPOP,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR1_EL1,
.field_pos = ID_AA64ISAR1_DPB_SHIFT,
@@ -1070,8 +1070,8 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_SVE
{
.desc = "Scalable Vector Extension",
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.capability = ARM64_SVE,
- .def_scope = SCOPE_SYSTEM,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
.field_pos = ID_AA64PFR0_SVE_SHIFT,
@@ -1084,7 +1084,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "RAS Extension Support",
.capability = ARM64_HAS_RAS_EXTN,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1096,16 +1096,16 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{},
};
-#define HWCAP_CAP(reg, field, s, min_value, type, cap) \
+#define HWCAP_CAP(reg, field, s, min_value, cap_type, cap) \
{ \
.desc = #cap, \
- .def_scope = SCOPE_SYSTEM, \
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM, \
.matches = has_cpuid_feature, \
.sys_reg = reg, \
.field_pos = field, \
.sign = s, \
.min_field_value = min_value, \
- .hwcap_type = type, \
+ .hwcap_type = cap_type, \
.hwcap = cap, \
}
@@ -1198,7 +1198,7 @@ static void __init setup_elf_hwcaps(const struct arm64_cpu_capabilities *hwcaps)
/* We support emulation of accesses to CPU ID feature registers */
elf_hwcap |= HWCAP_CPUID;
for (; hwcaps->matches; hwcaps++)
- if (hwcaps->matches(hwcaps, hwcaps->def_scope))
+ if (hwcaps->matches(hwcaps, cpucap_default_scope(hwcaps)))
cap_set_elf_hwcap(hwcaps);
}
@@ -1225,7 +1225,7 @@ static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
const char *info)
{
for (; caps->matches; caps++) {
- if (!caps->matches(caps, caps->def_scope))
+ if (!caps->matches(caps, cpucap_default_scope(caps)))
continue;
if (!cpus_have_cap(caps->capability) && caps->desc)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 04/21] arm64: capabilities: Add flags to handle the conflicts on late CPU
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (5 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 03/21] arm64: capabilities: Prepare for fine grained capabilities Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 04/22] arm64: capabilities: Prepare for fine grained capabilities Suzuki K Poulose
` (36 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
When a CPU is brought up, it is checked against the caps that are
known to be enabled on the system (via verify_local_cpu_capabilities()).
Based on the state of the capability on the CPU vs. that of System we
could have the following combinations of conflict.
x-----------------------------x
| Type | System | Late CPU |
|-----------------------------|
| a | y | n |
|-----------------------------|
| b | n | y |
x-----------------------------x
Case (a) is not permitted for caps which are system features, which the
system expects all the CPUs to have (e.g VHE). While (a) is ignored for
all errata work arounds. However, there could be exceptions to the plain
filtering approach. e.g, KPTI is an optional feature for a late CPU as
long as the system already enables it.
Case (b) is not permitted for errata work arounds that cannot be activated
after the kernel has finished booting.And we ignore (b) for features. Here,
yet again, KPTI is an exception, where if a late CPU needs KPTI we are too
late to enable it (because we change the allocation of ASIDs etc).
Add two different flags to indicate how the conflict should be handled.
ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU - CPUs may have the capability
ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU - CPUs may not have the cappability.
Now that we have the flags to describe the behavior of the errata and
the features, as we treat them, define types for ERRATUM and FEATURE.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 70 ++++++++++++++++++++++++++++++++++++-
arch/arm64/kernel/cpu_errata.c | 8 ++---
arch/arm64/kernel/cpufeature.c | 30 ++++++++--------
3 files changed, 88 insertions(+), 20 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index a2b708b9a6e3..5f58623a4599 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -145,7 +145,8 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* capabilities and if there is a conflict, the kernel takes an action, based
* on the severity (e.g, a CPU could be prevented from booting or cause a
* kernel panic). The CPU is allowed to "affect" the state of the capability,
- * if it has not been finalised already.
+ * if it has not been finalised already. See section 5 for more details on
+ * conflicts.
*
* 4) Action: As mentioned in (2), the kernel can take an action for each detected
* capability, on all CPUs on the system. Appropriate actions include, turning
@@ -161,6 +162,34 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* b) Any late CPU, brought up after (1), the action is triggered via:
* check_local_cpu_capabilities() -> verify_local_cpu_capabilities()
*
+ * 5) Conflicts: Based on the state of the capability on a late CPU vs. the system
+ * state, we could have the following combinations :
+ *
+ * x-----------------------------x
+ * | Type | System | Late CPU |
+ * |-----------------------------|
+ * | a | y | n |
+ * |-----------------------------|
+ * | b | n | y |
+ * x-----------------------------x
+ *
+ * Two separate flag bits are defined to indicate whether each kind of conflict
+ * can be allowed:
+ * ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU - Case(a) is allowed
+ * ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU - Case(b) is allowed
+ *
+ * Case (a) is not permitted for a capability that the system requires
+ * all CPUs to have in order for the capability to be enabled. This is
+ * typical for capabilities that represent enhanced functionality.
+ *
+ * Case (b) is not permitted for a capability that must be enabled during
+ * boot if any CPU in the system requires it in order to run safely.
+ * This is typical for erratum work arounds that cannot be enabled after the
+ * corresponding capability is finalised.
+ *
+ * In some non-typical cases either both (a) and (b), or neither, should be
+ * permitted. This can be described by including neither or both flags in the
+ * capability's type field.
*/
@@ -174,6 +203,33 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
+/*
+ * Is it permitted for a late CPU to have this capability when system
+ * hasn't already enabled it ?
+ */
+#define ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU ((u16)BIT(4))
+/* Is it safe for a late CPU to miss this capability when system has it */
+#define ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU ((u16)BIT(5))
+
+/*
+ * CPU errata workarounds that need to be enabled at boot time if one or more
+ * CPUs in the system requires it. When one of these capabilities has been
+ * enabled, it is safe to allow any CPU to boot that doesn't require the
+ * workaround. However, it is not safe if a "late" CPU requires a workaround
+ * and the system hasn't enabled it already.
+ */
+#define ARM64_CPUCAP_LOCAL_CPU_ERRATUM \
+ (ARM64_CPUCAP_SCOPE_LOCAL_CPU | ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
+/*
+ * CPU feature detected at boot time based on system-wide value of a feature.
+ * It is safe for a late CPU to have this feature even though the system hasn't
+ * enabled it, although the featuer will not be used by Linux in this case.
+ * If the system has enabled this feature already, then every late CPU must
+ * have it.
+ */
+#define ARM64_CPUCAP_SYSTEM_FEATURE \
+ (ARM64_CPUCAP_SCOPE_SYSTEM | ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
+
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
@@ -207,6 +263,18 @@ static inline int cpucap_default_scope(const struct arm64_cpu_capabilities *cap)
return cap->type & ARM64_CPUCAP_SCOPE_MASK;
}
+static inline bool
+cpucap_late_cpu_optional(const struct arm64_cpu_capabilities *cap)
+{
+ return !!(cap->type & ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU);
+}
+
+static inline bool
+cpucap_late_cpu_permitted(const struct arm64_cpu_capabilities *cap)
+{
+ return !!(cap->type & ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU);
+}
+
extern DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
extern struct static_key_false arm64_const_caps_ready;
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 3d3893bbc2dc..8d2d3cf819a7 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -226,14 +226,14 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
#define MIDR_RANGE(model, min, max) \
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = min, \
.midr_range_max = max
#define MIDR_ALL_VERSIONS(model) \
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = 0, \
@@ -337,7 +337,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "Mismatched cache line size",
.capability = ARM64_MISMATCHED_CACHE_LINE_SIZE,
.matches = has_mismatched_cache_line_size,
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
.cpu_enable = cpu_enable_trap_ctr_access,
},
#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003
@@ -351,7 +351,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Kryo erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
.midr_model = MIDR_QCOM_KRYO,
.matches = is_kryo_midr,
},
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9065a5974a3b..7df82c9daeb0 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -957,7 +957,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "GIC system register CPU interface",
.capability = ARM64_HAS_SYSREG_GIC_CPUIF,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_useable_gicv3_cpuif,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.field_pos = ID_AA64PFR0_GIC_SHIFT,
@@ -968,7 +968,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Privileged Access Never",
.capability = ARM64_HAS_PAN,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR1_EL1,
.field_pos = ID_AA64MMFR1_PAN_SHIFT,
@@ -981,7 +981,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "LSE atomic instructions",
.capability = ARM64_HAS_LSE_ATOMICS,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR0_EL1,
.field_pos = ID_AA64ISAR0_ATOMICS_SHIFT,
@@ -992,14 +992,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Software prefetching using PRFM",
.capability = ARM64_HAS_NO_HW_PREFETCH,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_no_hw_prefetch,
},
#ifdef CONFIG_ARM64_UAO
{
.desc = "User Access Override",
.capability = ARM64_HAS_UAO,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR2_EL1,
.field_pos = ID_AA64MMFR2_UAO_SHIFT,
@@ -1013,21 +1013,21 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_PAN
{
.capability = ARM64_ALT_PAN_NOT_UAO,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = cpufeature_pan_not_uao,
},
#endif /* CONFIG_ARM64_PAN */
{
.desc = "Virtualization Host Extensions",
.capability = ARM64_HAS_VIRT_HOST_EXTN,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
{
.desc = "32-bit EL0 Support",
.capability = ARM64_HAS_32BIT_EL0,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1037,14 +1037,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Reduced HYP mapping offset",
.capability = ARM64_HYP_OFFSET_LOW,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = hyp_offset_low,
},
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
{
.desc = "Kernel page table isolation (KPTI)",
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = unmap_kernel_at_el0,
.cpu_enable = kpti_install_ng_mappings,
},
@@ -1052,7 +1052,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
/* FP/SIMD is not implemented */
.capability = ARM64_HAS_NO_FPSIMD,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.min_field_value = 0,
.matches = has_no_fpsimd,
},
@@ -1060,7 +1060,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Data cache clean to Point of Persistence",
.capability = ARM64_HAS_DCPOP,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR1_EL1,
.field_pos = ID_AA64ISAR1_DPB_SHIFT,
@@ -1070,7 +1070,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_SVE
{
.desc = "Scalable Vector Extension",
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.capability = ARM64_SVE,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1084,7 +1084,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "RAS Extension Support",
.capability = ARM64_HAS_RAS_EXTN,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1099,7 +1099,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#define HWCAP_CAP(reg, field, s, min_value, cap_type, cap) \
{ \
.desc = #cap, \
- .type = ARM64_CPUCAP_SCOPE_SYSTEM, \
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE, \
.matches = has_cpuid_feature, \
.sys_reg = reg, \
.field_pos = field, \
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 04/22] arm64: capabilities: Prepare for fine grained capabilities
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (6 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 04/21] arm64: capabilities: Add flags to handle the conflicts on late CPU Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 05/22] arm64: capabilities: Add flags to handle the conflicts on late CPU Suzuki K Poulose
` (35 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
We use arm64_cpu_capabilities to represent CPU ELF HWCAPs exposed
to the userspace and the CPU hwcaps used by the kernel, which
include cpu features and CPU errata work arounds. Capabilities
have some properties that decide how they should be treated :
1) Detection, i.e scope : A cap could be "detected" either :
- if it is present on at least one CPU (SCOPE_LOCAL_CPU)
Or
- if it is present on all the CPUs (SCOPE_SYSTEM)
2) When is it enabled ? - A cap is treated as "enabled" when the
system takes some action based on whether the capability is detected or
not. e.g, setting some control register, patching the kernel code.
Right now, we treat all caps are enabled at boot-time, after all
the CPUs are brought up by the kernel. But there are certain caps,
which are enabled early during the boot (e.g, VHE, GIC_CPUIF for NMI)
and kernel starts using them, even before the secondary CPUs are brought
up. We would need a way to describe this for each capability.
3) Conflict on a late CPU - When a CPU is brought up, it is checked
against the caps that are known to be enabled on the system (via
verify_local_cpu_capabilities()). Based on the state of the capability
on the CPU vs. that of System we could have the following combinations
of conflict.
x-----------------------------x
| Type | System | Late CPU |
------------------------------|
| a | y | n |
------------------------------|
| b | n | y |
x-----------------------------x
Case (a) is not permitted for caps which are system features, which the
system expects all the CPUs to have (e.g VHE). While (a) is ignored for
all errata work arounds. However, there could be exceptions to the plain
filtering approach. e.g, KPTI is an optional feature for a late CPU as
long as the system already enables it.
Case (b) is not permitted for errata work arounds which requires some
work around, which cannot be delayed. And we ignore (b) for features. Here,
yet again, KPTI is an exception, where if a late CPU needs KPTI we are too
late to enable it (because we change the allocation of ASIDs etc).
So this calls for a lot more fine grained behavior for each capability.
And if we define all the attributes to control their behavior properly,
we may be able to use a single table for the CPU hwcaps (which cover
errata and features, not the ELF HWCAPs). This is a prepartory step
to get there. More bits would be added for the properties listed above.
We are going to use a bit-mask to encode all the properties of a capabilities.
This patch encodes the "SCOPE" of the capability.
As such there is no change in how the capabilities are treated.
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 99 ++++++++++++++++++++++++++++++++++---
arch/arm64/kernel/cpu_errata.c | 8 +--
arch/arm64/kernel/cpufeature.c | 38 +++++++-------
3 files changed, 116 insertions(+), 29 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 7925e40c6ded..a2b708b9a6e3 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -86,16 +86,98 @@ struct arm64_ftr_reg {
extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
-/* scope of capability check */
-enum {
- SCOPE_SYSTEM,
- SCOPE_LOCAL_CPU,
-};
+/*
+ * CPU capabilities:
+ *
+ * We use arm64_cpu_capabilities to represent system features, errata work
+ * arounds (both used internally by kernel and tracked in cpu_hwcaps) and
+ * ELF HWCAPs (which are exposed to user).
+ *
+ * To support systems with heterogeneous CPUs, we need to make sure that we
+ * detect the capabilities correctly on the system and take appropriate
+ * measures to ensure there are not incompatibilities.
+ *
+ * This comment tries to explain how we treat the capabilities.
+ * Each capability has the following list of attributes :
+ *
+ * 1) Scope of Detection : The system detects a given capability by performing
+ * some checks at runtime. This could be, e.g, checking the value of a field
+ * in CPU ID feature register or checking the cpu model. The capability
+ * provides a call back ( @matches() ) to perform the check.
+ * Scope defines how the checks should be performed. There are two cases:
+ *
+ * a) SCOPE_LOCAL_CPU: check all the CPUs and "detect" if at least one
+ * matches. This implies, we have to run the check on all the booting
+ * CPUs, until the system decides that state of the capability is finalised.
+ * (See section 2 below)
+ * Or
+ * b) SCOPE_SYSTEM: check all the CPUs and "detect" if all the CPUs matches.
+ * This implies, we run the check only once, when the system decides to
+ * finalise the state of the capability. If the capability relies on a
+ * field in one of the CPU ID feature registers, we use the sanitised
+ * value of the register from the CPU feature infrastructure to make
+ * the decision.
+ * The process of detection is usually denoted by "update" capability state
+ * in the code.
+ *
+ * 2) Finalise the state : The kernel should finalise the state of a capability
+ * at some point during its execution and take necessary actions if any. Usually,
+ * this is done, after all the boot-time enabled CPUs are brought up by the
+ * kernel, so that it can make better decision based on the available set
+ * of CPUs. However, there are some special cases, where the action is taken
+ * during the early boot by the primary boot CPU. (e.g, running the kernel at
+ * EL2 with Virtualisation Host Extensions). The kernel usually disallows
+ * any changes to the state of a capability once it finalises the capability
+ * and takes any action, as it may be impossible to execute the actions safely.
+ * A CPU brought up after a capability is "finalised" is referred to as "Late CPU"
+ * w.r.t the capability. e.g, all secondary CPUs are treated "late CPUs" for
+ * capabilities determined by the boot CPU.
+ *
+ * 3) Verification: When a CPU is brought online (e.g, by user or by the kernel),
+ * the kernel should make sure that it is safe to use the CPU, by verifying
+ * that the CPU is compliant with the state of the capabilities finalised
+ * already. This happens via :
+ *
+ * secondary_start_kernel()-> check_local_cpu_capabilities()
+ *
+ * As explained in (2) above, capabilities could be finalised at different
+ * points in the execution. Each CPU is verified against the "finalised"
+ * capabilities and if there is a conflict, the kernel takes an action, based
+ * on the severity (e.g, a CPU could be prevented from booting or cause a
+ * kernel panic). The CPU is allowed to "affect" the state of the capability,
+ * if it has not been finalised already.
+ *
+ * 4) Action: As mentioned in (2), the kernel can take an action for each detected
+ * capability, on all CPUs on the system. Appropriate actions include, turning
+ * on an architectural feature, modifying the control registers (e.g, SCTLR,
+ * TCR etc.) or patching the kernel via alternatives. The kernel patching is
+ * batched and performed at later point. The actions are alwayss initiated only
+ * after the capability is finalised. This is usally denoted by "enabling" the
+ * capability. The actions are initiated as follows :
+ * a) Action is triggered on all online CPUs, after the capability is
+ * finalised, invoked within the stop_machine() context from
+ * enable_cpu_capabilitie().
+ *
+ * b) Any late CPU, brought up after (1), the action is triggered via:
+ * check_local_cpu_capabilities() -> verify_local_cpu_capabilities()
+ *
+ */
+
+
+/* Decide how the capability is detected. On a local CPU vs System wide */
+#define ARM64_CPUCAP_SCOPE_LOCAL_CPU ((u16)BIT(0))
+#define ARM64_CPUCAP_SCOPE_SYSTEM ((u16)BIT(1))
+#define ARM64_CPUCAP_SCOPE_MASK \
+ (ARM64_CPUCAP_SCOPE_SYSTEM | \
+ ARM64_CPUCAP_SCOPE_LOCAL_CPU)
+
+#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
+#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
- int def_scope; /* default scope */
+ u16 type;
bool (*matches)(const struct arm64_cpu_capabilities *caps, int scope);
/*
* Take the appropriate actions to enable this capability for this CPU.
@@ -120,6 +202,11 @@ struct arm64_cpu_capabilities {
};
};
+static inline int cpucap_default_scope(const struct arm64_cpu_capabilities *cap)
+{
+ return cap->type & ARM64_CPUCAP_SCOPE_MASK;
+}
+
extern DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
extern struct static_key_false arm64_const_caps_ready;
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 0186788437a7..3d3893bbc2dc 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -226,14 +226,14 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
#define MIDR_RANGE(model, min, max) \
- .def_scope = SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = min, \
.midr_range_max = max
#define MIDR_ALL_VERSIONS(model) \
- .def_scope = SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = 0, \
@@ -337,7 +337,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "Mismatched cache line size",
.capability = ARM64_MISMATCHED_CACHE_LINE_SIZE,
.matches = has_mismatched_cache_line_size,
- .def_scope = SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
.cpu_enable = cpu_enable_trap_ctr_access,
},
#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003
@@ -351,7 +351,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Kryo erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
- .def_scope = SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
.midr_model = MIDR_QCOM_KRYO,
.matches = is_kryo_midr,
},
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 359fd6b4d159..9065a5974a3b 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -957,7 +957,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "GIC system register CPU interface",
.capability = ARM64_HAS_SYSREG_GIC_CPUIF,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_useable_gicv3_cpuif,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.field_pos = ID_AA64PFR0_GIC_SHIFT,
@@ -968,7 +968,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Privileged Access Never",
.capability = ARM64_HAS_PAN,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR1_EL1,
.field_pos = ID_AA64MMFR1_PAN_SHIFT,
@@ -981,7 +981,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "LSE atomic instructions",
.capability = ARM64_HAS_LSE_ATOMICS,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR0_EL1,
.field_pos = ID_AA64ISAR0_ATOMICS_SHIFT,
@@ -992,14 +992,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Software prefetching using PRFM",
.capability = ARM64_HAS_NO_HW_PREFETCH,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_no_hw_prefetch,
},
#ifdef CONFIG_ARM64_UAO
{
.desc = "User Access Override",
.capability = ARM64_HAS_UAO,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR2_EL1,
.field_pos = ID_AA64MMFR2_UAO_SHIFT,
@@ -1013,21 +1013,21 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_PAN
{
.capability = ARM64_ALT_PAN_NOT_UAO,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = cpufeature_pan_not_uao,
},
#endif /* CONFIG_ARM64_PAN */
{
.desc = "Virtualization Host Extensions",
.capability = ARM64_HAS_VIRT_HOST_EXTN,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
{
.desc = "32-bit EL0 Support",
.capability = ARM64_HAS_32BIT_EL0,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1037,14 +1037,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Reduced HYP mapping offset",
.capability = ARM64_HYP_OFFSET_LOW,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = hyp_offset_low,
},
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
{
.desc = "Kernel page table isolation (KPTI)",
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = unmap_kernel_at_el0,
.cpu_enable = kpti_install_ng_mappings,
},
@@ -1052,7 +1052,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
/* FP/SIMD is not implemented */
.capability = ARM64_HAS_NO_FPSIMD,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.min_field_value = 0,
.matches = has_no_fpsimd,
},
@@ -1060,7 +1060,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Data cache clean to Point of Persistence",
.capability = ARM64_HAS_DCPOP,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR1_EL1,
.field_pos = ID_AA64ISAR1_DPB_SHIFT,
@@ -1070,8 +1070,8 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_SVE
{
.desc = "Scalable Vector Extension",
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.capability = ARM64_SVE,
- .def_scope = SCOPE_SYSTEM,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
.field_pos = ID_AA64PFR0_SVE_SHIFT,
@@ -1084,7 +1084,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "RAS Extension Support",
.capability = ARM64_HAS_RAS_EXTN,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1096,16 +1096,16 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{},
};
-#define HWCAP_CAP(reg, field, s, min_value, type, cap) \
+#define HWCAP_CAP(reg, field, s, min_value, cap_type, cap) \
{ \
.desc = #cap, \
- .def_scope = SCOPE_SYSTEM, \
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM, \
.matches = has_cpuid_feature, \
.sys_reg = reg, \
.field_pos = field, \
.sign = s, \
.min_field_value = min_value, \
- .hwcap_type = type, \
+ .hwcap_type = cap_type, \
.hwcap = cap, \
}
@@ -1198,7 +1198,7 @@ static void __init setup_elf_hwcaps(const struct arm64_cpu_capabilities *hwcaps)
/* We support emulation of accesses to CPU ID feature registers */
elf_hwcap |= HWCAP_CPUID;
for (; hwcaps->matches; hwcaps++)
- if (hwcaps->matches(hwcaps, hwcaps->def_scope))
+ if (hwcaps->matches(hwcaps, cpucap_default_scope(hwcaps)))
cap_set_elf_hwcap(hwcaps);
}
@@ -1225,7 +1225,7 @@ static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
const char *info)
{
for (; caps->matches; caps++) {
- if (!caps->matches(caps, caps->def_scope))
+ if (!caps->matches(caps, cpucap_default_scope(caps)))
continue;
if (!cpus_have_cap(caps->capability) && caps->desc)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 05/22] arm64: capabilities: Add flags to handle the conflicts on late CPU
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (7 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 04/22] arm64: capabilities: Prepare for fine grained capabilities Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 05/21] arm64: capabilities: Unify the verification Suzuki K Poulose
` (34 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
When a CPU is brought up, it is checked against the caps that are
known to be enabled on the system (via verify_local_cpu_capabilities()).
Based on the state of the capability on the CPU vs. that of System we
could have the following combinations of conflict.
x-----------------------------x
| Type | System | Late CPU |
|-----------------------------|
| a | y | n |
|-----------------------------|
| b | n | y |
x-----------------------------x
Case (a) is not permitted for caps which are system features, which the
system expects all the CPUs to have (e.g VHE). While (a) is ignored for
all errata work arounds. However, there could be exceptions to the plain
filtering approach. e.g, KPTI is an optional feature for a late CPU as
long as the system already enables it.
Case (b) is not permitted for errata work arounds that cannot be activated
after the kernel has finished booting.And we ignore (b) for features. Here,
yet again, KPTI is an exception, where if a late CPU needs KPTI we are too
late to enable it (because we change the allocation of ASIDs etc).
Add two different flags to indicate how the conflict should be handled.
ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU - CPUs may have the capability
ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU - CPUs may not have the cappability.
Now that we have the flags to describe the behavior of the errata and
the features, as we treat them, define types for ERRATUM and FEATURE.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 70 ++++++++++++++++++++++++++++++++++++-
arch/arm64/kernel/cpu_errata.c | 8 ++---
arch/arm64/kernel/cpufeature.c | 30 ++++++++--------
3 files changed, 88 insertions(+), 20 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index a2b708b9a6e3..5f58623a4599 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -145,7 +145,8 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* capabilities and if there is a conflict, the kernel takes an action, based
* on the severity (e.g, a CPU could be prevented from booting or cause a
* kernel panic). The CPU is allowed to "affect" the state of the capability,
- * if it has not been finalised already.
+ * if it has not been finalised already. See section 5 for more details on
+ * conflicts.
*
* 4) Action: As mentioned in (2), the kernel can take an action for each detected
* capability, on all CPUs on the system. Appropriate actions include, turning
@@ -161,6 +162,34 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* b) Any late CPU, brought up after (1), the action is triggered via:
* check_local_cpu_capabilities() -> verify_local_cpu_capabilities()
*
+ * 5) Conflicts: Based on the state of the capability on a late CPU vs. the system
+ * state, we could have the following combinations :
+ *
+ * x-----------------------------x
+ * | Type | System | Late CPU |
+ * |-----------------------------|
+ * | a | y | n |
+ * |-----------------------------|
+ * | b | n | y |
+ * x-----------------------------x
+ *
+ * Two separate flag bits are defined to indicate whether each kind of conflict
+ * can be allowed:
+ * ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU - Case(a) is allowed
+ * ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU - Case(b) is allowed
+ *
+ * Case (a) is not permitted for a capability that the system requires
+ * all CPUs to have in order for the capability to be enabled. This is
+ * typical for capabilities that represent enhanced functionality.
+ *
+ * Case (b) is not permitted for a capability that must be enabled during
+ * boot if any CPU in the system requires it in order to run safely.
+ * This is typical for erratum work arounds that cannot be enabled after the
+ * corresponding capability is finalised.
+ *
+ * In some non-typical cases either both (a) and (b), or neither, should be
+ * permitted. This can be described by including neither or both flags in the
+ * capability's type field.
*/
@@ -174,6 +203,33 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
+/*
+ * Is it permitted for a late CPU to have this capability when system
+ * hasn't already enabled it ?
+ */
+#define ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU ((u16)BIT(4))
+/* Is it safe for a late CPU to miss this capability when system has it */
+#define ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU ((u16)BIT(5))
+
+/*
+ * CPU errata workarounds that need to be enabled at boot time if one or more
+ * CPUs in the system requires it. When one of these capabilities has been
+ * enabled, it is safe to allow any CPU to boot that doesn't require the
+ * workaround. However, it is not safe if a "late" CPU requires a workaround
+ * and the system hasn't enabled it already.
+ */
+#define ARM64_CPUCAP_LOCAL_CPU_ERRATUM \
+ (ARM64_CPUCAP_SCOPE_LOCAL_CPU | ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
+/*
+ * CPU feature detected at boot time based on system-wide value of a feature.
+ * It is safe for a late CPU to have this feature even though the system hasn't
+ * enabled it, although the featuer will not be used by Linux in this case.
+ * If the system has enabled this feature already, then every late CPU must
+ * have it.
+ */
+#define ARM64_CPUCAP_SYSTEM_FEATURE \
+ (ARM64_CPUCAP_SCOPE_SYSTEM | ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
+
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
@@ -207,6 +263,18 @@ static inline int cpucap_default_scope(const struct arm64_cpu_capabilities *cap)
return cap->type & ARM64_CPUCAP_SCOPE_MASK;
}
+static inline bool
+cpucap_late_cpu_optional(const struct arm64_cpu_capabilities *cap)
+{
+ return !!(cap->type & ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU);
+}
+
+static inline bool
+cpucap_late_cpu_permitted(const struct arm64_cpu_capabilities *cap)
+{
+ return !!(cap->type & ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU);
+}
+
extern DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
extern struct static_key_false arm64_const_caps_ready;
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 3d3893bbc2dc..8d2d3cf819a7 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -226,14 +226,14 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
#define MIDR_RANGE(model, min, max) \
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = min, \
.midr_range_max = max
#define MIDR_ALL_VERSIONS(model) \
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = 0, \
@@ -337,7 +337,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "Mismatched cache line size",
.capability = ARM64_MISMATCHED_CACHE_LINE_SIZE,
.matches = has_mismatched_cache_line_size,
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
.cpu_enable = cpu_enable_trap_ctr_access,
},
#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003
@@ -351,7 +351,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Kryo erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
.midr_model = MIDR_QCOM_KRYO,
.matches = is_kryo_midr,
},
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9065a5974a3b..7df82c9daeb0 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -957,7 +957,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "GIC system register CPU interface",
.capability = ARM64_HAS_SYSREG_GIC_CPUIF,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_useable_gicv3_cpuif,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.field_pos = ID_AA64PFR0_GIC_SHIFT,
@@ -968,7 +968,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Privileged Access Never",
.capability = ARM64_HAS_PAN,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR1_EL1,
.field_pos = ID_AA64MMFR1_PAN_SHIFT,
@@ -981,7 +981,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "LSE atomic instructions",
.capability = ARM64_HAS_LSE_ATOMICS,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR0_EL1,
.field_pos = ID_AA64ISAR0_ATOMICS_SHIFT,
@@ -992,14 +992,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Software prefetching using PRFM",
.capability = ARM64_HAS_NO_HW_PREFETCH,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_no_hw_prefetch,
},
#ifdef CONFIG_ARM64_UAO
{
.desc = "User Access Override",
.capability = ARM64_HAS_UAO,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR2_EL1,
.field_pos = ID_AA64MMFR2_UAO_SHIFT,
@@ -1013,21 +1013,21 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_PAN
{
.capability = ARM64_ALT_PAN_NOT_UAO,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = cpufeature_pan_not_uao,
},
#endif /* CONFIG_ARM64_PAN */
{
.desc = "Virtualization Host Extensions",
.capability = ARM64_HAS_VIRT_HOST_EXTN,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
{
.desc = "32-bit EL0 Support",
.capability = ARM64_HAS_32BIT_EL0,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1037,14 +1037,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Reduced HYP mapping offset",
.capability = ARM64_HYP_OFFSET_LOW,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = hyp_offset_low,
},
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
{
.desc = "Kernel page table isolation (KPTI)",
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = unmap_kernel_at_el0,
.cpu_enable = kpti_install_ng_mappings,
},
@@ -1052,7 +1052,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
/* FP/SIMD is not implemented */
.capability = ARM64_HAS_NO_FPSIMD,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.min_field_value = 0,
.matches = has_no_fpsimd,
},
@@ -1060,7 +1060,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Data cache clean to Point of Persistence",
.capability = ARM64_HAS_DCPOP,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR1_EL1,
.field_pos = ID_AA64ISAR1_DPB_SHIFT,
@@ -1070,7 +1070,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_SVE
{
.desc = "Scalable Vector Extension",
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.capability = ARM64_SVE,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1084,7 +1084,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "RAS Extension Support",
.capability = ARM64_HAS_RAS_EXTN,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1099,7 +1099,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#define HWCAP_CAP(reg, field, s, min_value, cap_type, cap) \
{ \
.desc = #cap, \
- .type = ARM64_CPUCAP_SCOPE_SYSTEM, \
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE, \
.matches = has_cpuid_feature, \
.sys_reg = reg, \
.field_pos = field, \
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 05/21] arm64: capabilities: Unify the verification
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (8 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 05/22] arm64: capabilities: Add flags to handle the conflicts on late CPU Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 06/21] arm64: capabilities: Filter the entries based on a given mask Suzuki K Poulose
` (33 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
Now that each capability describes how to treat the conflicts
of CPU cap state vs System wide cap state, we can unify the
verification logic to a single place.
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 89 ++++++++++++++++++++++++++----------------
1 file changed, 56 insertions(+), 33 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 7df82c9daeb0..48ed6d71094f 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1269,6 +1269,56 @@ static void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *
}
}
+/*
+ * Run through the list of capabilities to check for conflicts.
+ * If the system has already detected a capability, take necessary
+ * action on this CPU.
+ *
+ * Returns "false" on conflicts.
+ */
+static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list)
+{
+ bool cpu_has_cap, system_has_cap;
+ const struct arm64_cpu_capabilities *caps = caps_list;
+
+ for (; caps->matches; caps++) {
+ cpu_has_cap = __this_cpu_has_cap(caps_list, caps->capability);
+ system_has_cap = cpus_have_cap(caps->capability);
+
+ if (system_has_cap) {
+ /*
+ * Check if the new CPU misses an advertised feature, which is not
+ * safe to miss.
+ */
+ if (!cpu_has_cap && !cpucap_late_cpu_optional(caps))
+ break;
+ /*
+ * We have to issue cpu_enable() irrespective of whether the CPU
+ * has it or not, as it is enabeld system wide. It is upto
+ * the call back to take appropriate action on this CPU.
+ */
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
+ } else {
+ /*
+ * Check if the CPU has this capability if it isn't safe to
+ * have when the system doesn't.
+ */
+ if (cpu_has_cap && !cpucap_late_cpu_permitted(caps))
+ break;
+ }
+ }
+
+ if (caps->matches) {
+ pr_crit("CPU%d: Detected conflict for capability %d (%s), System: %d, CPU: %d\n",
+ smp_processor_id(), caps->capability,
+ caps->desc, system_has_cap, cpu_has_cap);
+ return false;
+ }
+
+ return true;
+}
+
/*
* Check for CPU features that are used in early boot
* based on the Boot CPU value.
@@ -1291,25 +1341,10 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps)
}
}
-static void
-verify_local_cpu_features(const struct arm64_cpu_capabilities *caps_list)
+static void verify_local_cpu_features(void)
{
- const struct arm64_cpu_capabilities *caps = caps_list;
- for (; caps->matches; caps++) {
- if (!cpus_have_cap(caps->capability))
- continue;
- /*
- * If the new CPU misses an advertised feature, we cannot proceed
- * further, park the cpu.
- */
- if (!__this_cpu_has_cap(caps_list, caps->capability)) {
- pr_crit("CPU%d: missing feature: %s\n",
- smp_processor_id(), caps->desc);
- cpu_die_early();
- }
- if (caps->cpu_enable)
- caps->cpu_enable(caps);
- }
+ if (!__verify_local_cpu_caps(arm64_features))
+ cpu_die_early();
}
static void verify_sve_features(void)
@@ -1336,20 +1371,8 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_errata_workarounds(void)
{
- const struct arm64_cpu_capabilities *caps = arm64_errata;
-
- for (; caps->matches; caps++) {
- if (cpus_have_cap(caps->capability)) {
- if (caps->cpu_enable)
- caps->cpu_enable(caps);
- } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
- pr_crit("CPU%d: Requires work around for %s, not detected"
- " at boot time\n",
- smp_processor_id(),
- caps->desc ? : "an erratum");
- cpu_die_early();
- }
- }
+ if (!__verify_local_cpu_caps(arm64_errata))
+ cpu_die_early();
}
static void update_cpu_errata_workarounds(void)
@@ -1373,7 +1396,7 @@ static void __init enable_errata_workarounds(void)
static void verify_local_cpu_capabilities(void)
{
verify_local_cpu_errata_workarounds();
- verify_local_cpu_features(arm64_features);
+ verify_local_cpu_features();
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 06/21] arm64: capabilities: Filter the entries based on a given mask
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (9 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 05/21] arm64: capabilities: Unify the verification Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 06/22] arm64: capabilities: Unify the verification Suzuki K Poulose
` (32 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
While processing the list of capabilities, it is useful to
filter out some of the entries based on the given mask for the
scope of the capabilities to allow better control. This can be
used later for handling LOCAL vs SYSTEM wide capabilities and more.
All capabilities should have their scope set to either LOCAL_CPU or
SYSTEM. No functional/flow change.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
Changes since v2:
- Use SCOPE_ALL instead of ARM64_CPUCAP_SCOPE_ALL in the code
---
arch/arm64/include/asm/cpufeature.h | 1 +
arch/arm64/kernel/cpufeature.c | 33 ++++++++++++++++++++++-----------
2 files changed, 23 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 5f58623a4599..6e354a98bc24 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -202,6 +202,7 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
+#define SCOPE_ALL ARM64_CPUCAP_SCOPE_MASK
/*
* Is it permitted for a late CPU to have this capability when system
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 48ed6d71094f..3447821bec5a 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1222,10 +1222,12 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
}
static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- const char *info)
+ u16 scope_mask, const char *info)
{
+ scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
- if (!caps->matches(caps, cpucap_default_scope(caps)))
+ if (!(caps->type & scope_mask) ||
+ !caps->matches(caps, cpucap_default_scope(caps)))
continue;
if (!cpus_have_cap(caps->capability) && caps->desc)
@@ -1246,12 +1248,14 @@ static int __enable_cpu_capability(void *arg)
* Run through the enabled capabilities and enable() it on all active
* CPUs
*/
-static void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
+static void __init
+enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_mask)
{
+ scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
unsigned int num = caps->capability;
- if (!cpus_have_cap(num))
+ if (!(caps->type & scope_mask) || !cpus_have_cap(num))
continue;
/* Ensure cpus_have_const_cap(num) works */
@@ -1276,12 +1280,17 @@ static void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *
*
* Returns "false" on conflicts.
*/
-static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list)
+static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list,
+ u16 scope_mask)
{
bool cpu_has_cap, system_has_cap;
const struct arm64_cpu_capabilities *caps = caps_list;
+ scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
+ if (!(caps->type & scope_mask))
+ continue;
+
cpu_has_cap = __this_cpu_has_cap(caps_list, caps->capability);
system_has_cap = cpus_have_cap(caps->capability);
@@ -1343,7 +1352,7 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps)
static void verify_local_cpu_features(void)
{
- if (!__verify_local_cpu_caps(arm64_features))
+ if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
cpu_die_early();
}
@@ -1371,18 +1380,19 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_errata_workarounds(void)
{
- if (!__verify_local_cpu_caps(arm64_errata))
+ if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
cpu_die_early();
}
static void update_cpu_errata_workarounds(void)
{
- update_cpu_capabilities(arm64_errata, "enabling workaround for");
+ update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ "enabling workaround for");
}
static void __init enable_errata_workarounds(void)
{
- enable_cpu_capabilities(arm64_errata);
+ enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
}
/*
@@ -1431,8 +1441,9 @@ void check_local_cpu_capabilities(void)
static void __init setup_feature_capabilities(void)
{
- update_cpu_capabilities(arm64_features, "detected feature:");
- enable_cpu_capabilities(arm64_features);
+ update_cpu_capabilities(arm64_features, SCOPE_ALL,
+ "detected feature:");
+ enable_cpu_capabilities(arm64_features, SCOPE_ALL);
}
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 06/22] arm64: capabilities: Unify the verification
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (10 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 06/21] arm64: capabilities: Filter the entries based on a given mask Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 07/22] arm64: capabilities: Filter the entries based on a given mask Suzuki K Poulose
` (31 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
Now that each capability describes how to treat the conflicts
of CPU cap state vs System wide cap state, we can unify the
verification logic to a single place.
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 89 ++++++++++++++++++++++++++----------------
1 file changed, 56 insertions(+), 33 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 7df82c9daeb0..48ed6d71094f 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1269,6 +1269,56 @@ static void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *
}
}
+/*
+ * Run through the list of capabilities to check for conflicts.
+ * If the system has already detected a capability, take necessary
+ * action on this CPU.
+ *
+ * Returns "false" on conflicts.
+ */
+static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list)
+{
+ bool cpu_has_cap, system_has_cap;
+ const struct arm64_cpu_capabilities *caps = caps_list;
+
+ for (; caps->matches; caps++) {
+ cpu_has_cap = __this_cpu_has_cap(caps_list, caps->capability);
+ system_has_cap = cpus_have_cap(caps->capability);
+
+ if (system_has_cap) {
+ /*
+ * Check if the new CPU misses an advertised feature, which is not
+ * safe to miss.
+ */
+ if (!cpu_has_cap && !cpucap_late_cpu_optional(caps))
+ break;
+ /*
+ * We have to issue cpu_enable() irrespective of whether the CPU
+ * has it or not, as it is enabeld system wide. It is upto
+ * the call back to take appropriate action on this CPU.
+ */
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
+ } else {
+ /*
+ * Check if the CPU has this capability if it isn't safe to
+ * have when the system doesn't.
+ */
+ if (cpu_has_cap && !cpucap_late_cpu_permitted(caps))
+ break;
+ }
+ }
+
+ if (caps->matches) {
+ pr_crit("CPU%d: Detected conflict for capability %d (%s), System: %d, CPU: %d\n",
+ smp_processor_id(), caps->capability,
+ caps->desc, system_has_cap, cpu_has_cap);
+ return false;
+ }
+
+ return true;
+}
+
/*
* Check for CPU features that are used in early boot
* based on the Boot CPU value.
@@ -1291,25 +1341,10 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps)
}
}
-static void
-verify_local_cpu_features(const struct arm64_cpu_capabilities *caps_list)
+static void verify_local_cpu_features(void)
{
- const struct arm64_cpu_capabilities *caps = caps_list;
- for (; caps->matches; caps++) {
- if (!cpus_have_cap(caps->capability))
- continue;
- /*
- * If the new CPU misses an advertised feature, we cannot proceed
- * further, park the cpu.
- */
- if (!__this_cpu_has_cap(caps_list, caps->capability)) {
- pr_crit("CPU%d: missing feature: %s\n",
- smp_processor_id(), caps->desc);
- cpu_die_early();
- }
- if (caps->cpu_enable)
- caps->cpu_enable(caps);
- }
+ if (!__verify_local_cpu_caps(arm64_features))
+ cpu_die_early();
}
static void verify_sve_features(void)
@@ -1336,20 +1371,8 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_errata_workarounds(void)
{
- const struct arm64_cpu_capabilities *caps = arm64_errata;
-
- for (; caps->matches; caps++) {
- if (cpus_have_cap(caps->capability)) {
- if (caps->cpu_enable)
- caps->cpu_enable(caps);
- } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
- pr_crit("CPU%d: Requires work around for %s, not detected"
- " at boot time\n",
- smp_processor_id(),
- caps->desc ? : "an erratum");
- cpu_die_early();
- }
- }
+ if (!__verify_local_cpu_caps(arm64_errata))
+ cpu_die_early();
}
static void update_cpu_errata_workarounds(void)
@@ -1373,7 +1396,7 @@ static void __init enable_errata_workarounds(void)
static void verify_local_cpu_capabilities(void)
{
verify_local_cpu_errata_workarounds();
- verify_local_cpu_features(arm64_features);
+ verify_local_cpu_features();
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 07/22] arm64: capabilities: Filter the entries based on a given mask
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (11 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 06/22] arm64: capabilities: Unify the verification Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 07/21] arm64: capabilities: Prepare for grouping features and errata work arounds Suzuki K Poulose
` (30 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
While processing the list of capabilities, it is useful to
filter out some of the entries based on the given mask for the
scope of the capabilities to allow better control. This can be
used later for handling LOCAL vs SYSTEM wide capabilities and more.
All capabilities should have their scope set to either LOCAL_CPU or
SYSTEM. No functional/flow change.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
Changes since v2:
- Use SCOPE_ALL instead of ARM64_CPUCAP_SCOPE_ALL in the code
---
arch/arm64/include/asm/cpufeature.h | 1 +
arch/arm64/kernel/cpufeature.c | 33 ++++++++++++++++++++++-----------
2 files changed, 23 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 5f58623a4599..6e354a98bc24 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -202,6 +202,7 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
+#define SCOPE_ALL ARM64_CPUCAP_SCOPE_MASK
/*
* Is it permitted for a late CPU to have this capability when system
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 48ed6d71094f..3447821bec5a 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1222,10 +1222,12 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
}
static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- const char *info)
+ u16 scope_mask, const char *info)
{
+ scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
- if (!caps->matches(caps, cpucap_default_scope(caps)))
+ if (!(caps->type & scope_mask) ||
+ !caps->matches(caps, cpucap_default_scope(caps)))
continue;
if (!cpus_have_cap(caps->capability) && caps->desc)
@@ -1246,12 +1248,14 @@ static int __enable_cpu_capability(void *arg)
* Run through the enabled capabilities and enable() it on all active
* CPUs
*/
-static void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
+static void __init
+enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_mask)
{
+ scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
unsigned int num = caps->capability;
- if (!cpus_have_cap(num))
+ if (!(caps->type & scope_mask) || !cpus_have_cap(num))
continue;
/* Ensure cpus_have_const_cap(num) works */
@@ -1276,12 +1280,17 @@ static void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *
*
* Returns "false" on conflicts.
*/
-static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list)
+static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list,
+ u16 scope_mask)
{
bool cpu_has_cap, system_has_cap;
const struct arm64_cpu_capabilities *caps = caps_list;
+ scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
+ if (!(caps->type & scope_mask))
+ continue;
+
cpu_has_cap = __this_cpu_has_cap(caps_list, caps->capability);
system_has_cap = cpus_have_cap(caps->capability);
@@ -1343,7 +1352,7 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps)
static void verify_local_cpu_features(void)
{
- if (!__verify_local_cpu_caps(arm64_features))
+ if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
cpu_die_early();
}
@@ -1371,18 +1380,19 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_errata_workarounds(void)
{
- if (!__verify_local_cpu_caps(arm64_errata))
+ if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
cpu_die_early();
}
static void update_cpu_errata_workarounds(void)
{
- update_cpu_capabilities(arm64_errata, "enabling workaround for");
+ update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ "enabling workaround for");
}
static void __init enable_errata_workarounds(void)
{
- enable_cpu_capabilities(arm64_errata);
+ enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
}
/*
@@ -1431,8 +1441,9 @@ void check_local_cpu_capabilities(void)
static void __init setup_feature_capabilities(void)
{
- update_cpu_capabilities(arm64_features, "detected feature:");
- enable_cpu_capabilities(arm64_features);
+ update_cpu_capabilities(arm64_features, SCOPE_ALL,
+ "detected feature:");
+ enable_cpu_capabilities(arm64_features, SCOPE_ALL);
}
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 07/21] arm64: capabilities: Prepare for grouping features and errata work arounds
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (12 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 07/22] arm64: capabilities: Filter the entries based on a given mask Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 08/22] " Suzuki K Poulose
` (29 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
We are about to group the handling of all capabilities (features
and errata workarounds). This patch open codes the wrapper routines
to make it easier to merge the handling.
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 60 +++++++++++++-----------------------------
1 file changed, 19 insertions(+), 41 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 3447821bec5a..9f52678a4df1 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -504,7 +504,8 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
-static void update_cpu_errata_workarounds(void);
+static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+ u16 scope_mask, const char *info);
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
@@ -553,7 +554,8 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
* Run the errata work around checks on the boot CPU, once we have
* initialised the cpu feature infrastructure.
*/
- update_cpu_errata_workarounds();
+ update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ "enabling workaround for");
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1350,12 +1352,6 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps)
}
}
-static void verify_local_cpu_features(void)
-{
- if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
- cpu_die_early();
-}
-
static void verify_sve_features(void)
{
u64 safe_zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
@@ -1373,27 +1369,6 @@ static void verify_sve_features(void)
/* Add checks on other ZCR bits here if necessary */
}
-/*
- * The CPU Errata work arounds are detected and applied at boot time
- * and the related information is freed soon after. If the new CPU requires
- * an errata not detected at boot, fail this CPU.
- */
-static void verify_local_cpu_errata_workarounds(void)
-{
- if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
- cpu_die_early();
-}
-
-static void update_cpu_errata_workarounds(void)
-{
- update_cpu_capabilities(arm64_errata, SCOPE_ALL,
- "enabling workaround for");
-}
-
-static void __init enable_errata_workarounds(void)
-{
- enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
-}
/*
* Run through the enabled system capabilities and enable() it on this CPU.
@@ -1405,8 +1380,15 @@ static void __init enable_errata_workarounds(void)
*/
static void verify_local_cpu_capabilities(void)
{
- verify_local_cpu_errata_workarounds();
- verify_local_cpu_features();
+ /*
+ * The CPU Errata work arounds are detected and applied at boot time
+ * and the related information is freed soon after. If the new CPU
+ * requires an errata not detected at boot, fail this CPU.
+ */
+ if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
+ cpu_die_early();
+ if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
+ cpu_die_early();
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
@@ -1434,18 +1416,12 @@ void check_local_cpu_capabilities(void)
* advertised capabilities.
*/
if (!sys_caps_initialised)
- update_cpu_errata_workarounds();
+ update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ "enabling workaround for");
else
verify_local_cpu_capabilities();
}
-static void __init setup_feature_capabilities(void)
-{
- update_cpu_capabilities(arm64_features, SCOPE_ALL,
- "detected feature:");
- enable_cpu_capabilities(arm64_features, SCOPE_ALL);
-}
-
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
EXPORT_SYMBOL(arm64_const_caps_ready);
@@ -1468,8 +1444,10 @@ void __init setup_cpu_features(void)
int cls;
/* Set the CPU feature capabilies */
- setup_feature_capabilities();
- enable_errata_workarounds();
+ update_cpu_capabilities(arm64_features, SCOPE_ALL,
+ "detected feature:");
+ enable_cpu_capabilities(arm64_features, SCOPE_ALL);
+ enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
mark_const_caps_ready();
setup_elf_hwcaps(arm64_elf_hwcaps);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 08/22] arm64: capabilities: Prepare for grouping features and errata work arounds
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (13 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 07/21] arm64: capabilities: Prepare for grouping features and errata work arounds Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 18:11 ` Dave Martin
2018-02-09 17:54 ` [PATCH v3 08/21] arm64: capabilities: Split the processing of " Suzuki K Poulose
` (28 subsequent siblings)
43 siblings, 1 reply; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
We are about to group the handling of all capabilities (features
and errata workarounds). This patch open codes the wrapper routines
to make it easier to merge the handling.
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 60 +++++++++++++-----------------------------
1 file changed, 19 insertions(+), 41 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 3447821bec5a..9f52678a4df1 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -504,7 +504,8 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
-static void update_cpu_errata_workarounds(void);
+static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+ u16 scope_mask, const char *info);
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
@@ -553,7 +554,8 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
* Run the errata work around checks on the boot CPU, once we have
* initialised the cpu feature infrastructure.
*/
- update_cpu_errata_workarounds();
+ update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ "enabling workaround for");
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1350,12 +1352,6 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps)
}
}
-static void verify_local_cpu_features(void)
-{
- if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
- cpu_die_early();
-}
-
static void verify_sve_features(void)
{
u64 safe_zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
@@ -1373,27 +1369,6 @@ static void verify_sve_features(void)
/* Add checks on other ZCR bits here if necessary */
}
-/*
- * The CPU Errata work arounds are detected and applied at boot time
- * and the related information is freed soon after. If the new CPU requires
- * an errata not detected at boot, fail this CPU.
- */
-static void verify_local_cpu_errata_workarounds(void)
-{
- if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
- cpu_die_early();
-}
-
-static void update_cpu_errata_workarounds(void)
-{
- update_cpu_capabilities(arm64_errata, SCOPE_ALL,
- "enabling workaround for");
-}
-
-static void __init enable_errata_workarounds(void)
-{
- enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
-}
/*
* Run through the enabled system capabilities and enable() it on this CPU.
@@ -1405,8 +1380,15 @@ static void __init enable_errata_workarounds(void)
*/
static void verify_local_cpu_capabilities(void)
{
- verify_local_cpu_errata_workarounds();
- verify_local_cpu_features();
+ /*
+ * The CPU Errata work arounds are detected and applied at boot time
+ * and the related information is freed soon after. If the new CPU
+ * requires an errata not detected at boot, fail this CPU.
+ */
+ if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
+ cpu_die_early();
+ if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
+ cpu_die_early();
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
@@ -1434,18 +1416,12 @@ void check_local_cpu_capabilities(void)
* advertised capabilities.
*/
if (!sys_caps_initialised)
- update_cpu_errata_workarounds();
+ update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ "enabling workaround for");
else
verify_local_cpu_capabilities();
}
-static void __init setup_feature_capabilities(void)
-{
- update_cpu_capabilities(arm64_features, SCOPE_ALL,
- "detected feature:");
- enable_cpu_capabilities(arm64_features, SCOPE_ALL);
-}
-
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
EXPORT_SYMBOL(arm64_const_caps_ready);
@@ -1468,8 +1444,10 @@ void __init setup_cpu_features(void)
int cls;
/* Set the CPU feature capabilies */
- setup_feature_capabilities();
- enable_errata_workarounds();
+ update_cpu_capabilities(arm64_features, SCOPE_ALL,
+ "detected feature:");
+ enable_cpu_capabilities(arm64_features, SCOPE_ALL);
+ enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
mark_const_caps_ready();
setup_elf_hwcaps(arm64_elf_hwcaps);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 08/22] arm64: capabilities: Prepare for grouping features and errata work arounds
2018-02-09 17:54 ` [PATCH v3 08/22] " Suzuki K Poulose
@ 2018-02-09 18:11 ` Dave Martin
0 siblings, 0 replies; 62+ messages in thread
From: Dave Martin @ 2018-02-09 18:11 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Feb 09, 2018 at 05:54:45PM +0000, Suzuki K Poulose wrote:
> We are about to group the handling of all capabilities (features
> and errata workarounds). This patch open codes the wrapper routines
> to make it easier to merge the handling.
>
> Cc: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
This seems to make sense as a no-functional-change bit of refactoring.
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
> ---
> arch/arm64/kernel/cpufeature.c | 60 +++++++++++++-----------------------------
> 1 file changed, 19 insertions(+), 41 deletions(-)
>
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 3447821bec5a..9f52678a4df1 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -504,7 +504,8 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
> }
>
> extern const struct arm64_cpu_capabilities arm64_errata[];
> -static void update_cpu_errata_workarounds(void);
> +static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
> + u16 scope_mask, const char *info);
>
> void __init init_cpu_features(struct cpuinfo_arm64 *info)
> {
> @@ -553,7 +554,8 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
> * Run the errata work around checks on the boot CPU, once we have
> * initialised the cpu feature infrastructure.
> */
> - update_cpu_errata_workarounds();
> + update_cpu_capabilities(arm64_errata, SCOPE_ALL,
> + "enabling workaround for");
> }
>
> static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
> @@ -1350,12 +1352,6 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps)
> }
> }
>
> -static void verify_local_cpu_features(void)
> -{
> - if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
> - cpu_die_early();
> -}
> -
> static void verify_sve_features(void)
> {
> u64 safe_zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
> @@ -1373,27 +1369,6 @@ static void verify_sve_features(void)
> /* Add checks on other ZCR bits here if necessary */
> }
>
> -/*
> - * The CPU Errata work arounds are detected and applied at boot time
> - * and the related information is freed soon after. If the new CPU requires
> - * an errata not detected at boot, fail this CPU.
> - */
> -static void verify_local_cpu_errata_workarounds(void)
> -{
> - if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
> - cpu_die_early();
> -}
> -
> -static void update_cpu_errata_workarounds(void)
> -{
> - update_cpu_capabilities(arm64_errata, SCOPE_ALL,
> - "enabling workaround for");
> -}
> -
> -static void __init enable_errata_workarounds(void)
> -{
> - enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
> -}
>
> /*
> * Run through the enabled system capabilities and enable() it on this CPU.
> @@ -1405,8 +1380,15 @@ static void __init enable_errata_workarounds(void)
> */
> static void verify_local_cpu_capabilities(void)
> {
> - verify_local_cpu_errata_workarounds();
> - verify_local_cpu_features();
> + /*
> + * The CPU Errata work arounds are detected and applied at boot time
> + * and the related information is freed soon after. If the new CPU
> + * requires an errata not detected at boot, fail this CPU.
> + */
> + if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
> + cpu_die_early();
> + if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
> + cpu_die_early();
> verify_local_elf_hwcaps(arm64_elf_hwcaps);
>
> if (system_supports_32bit_el0())
> @@ -1434,18 +1416,12 @@ void check_local_cpu_capabilities(void)
> * advertised capabilities.
> */
> if (!sys_caps_initialised)
> - update_cpu_errata_workarounds();
> + update_cpu_capabilities(arm64_errata, SCOPE_ALL,
> + "enabling workaround for");
> else
> verify_local_cpu_capabilities();
> }
>
> -static void __init setup_feature_capabilities(void)
> -{
> - update_cpu_capabilities(arm64_features, SCOPE_ALL,
> - "detected feature:");
> - enable_cpu_capabilities(arm64_features, SCOPE_ALL);
> -}
> -
> DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
> EXPORT_SYMBOL(arm64_const_caps_ready);
>
> @@ -1468,8 +1444,10 @@ void __init setup_cpu_features(void)
> int cls;
>
> /* Set the CPU feature capabilies */
> - setup_feature_capabilities();
> - enable_errata_workarounds();
> + update_cpu_capabilities(arm64_features, SCOPE_ALL,
> + "detected feature:");
> + enable_cpu_capabilities(arm64_features, SCOPE_ALL);
> + enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
> mark_const_caps_ready();
> setup_elf_hwcaps(arm64_elf_hwcaps);
>
> --
> 2.14.3
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v3 08/21] arm64: capabilities: Split the processing of errata work arounds
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (14 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 08/22] " Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 09/21] arm64: capabilities: Allow features based on local CPU scope Suzuki K Poulose
` (27 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
Right now we run through the errata workarounds check on all boot
active CPUs, with SCOPE_ALL. This doesn't help with detecting the
errata's with a SYSTEM_SCOPE. While nobody uses it, let us clean
this up in preparation for merging capability handling.
So, we run the checks with SCOPE_LOCAL_CPU on all CPUs and SCOPE_SYSTEM
checks are run only once after all the boot time CPUs are active.
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9f52678a4df1..a6e124063e21 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -554,7 +554,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
* Run the errata work around checks on the boot CPU, once we have
* initialised the cpu feature infrastructure.
*/
- update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
}
@@ -1416,7 +1416,7 @@ void check_local_cpu_capabilities(void)
* advertised capabilities.
*/
if (!sys_caps_initialised)
- update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
else
verify_local_cpu_capabilities();
@@ -1446,6 +1446,8 @@ void __init setup_cpu_features(void)
/* Set the CPU feature capabilies */
update_cpu_capabilities(arm64_features, SCOPE_ALL,
"detected feature:");
+ update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
+ "enabling workaround for");
enable_cpu_capabilities(arm64_features, SCOPE_ALL);
enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
mark_const_caps_ready();
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 09/21] arm64: capabilities: Allow features based on local CPU scope
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (15 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 08/21] arm64: capabilities: Split the processing of " Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 09/22] arm64: capabilities: Split the processing of errata work arounds Suzuki K Poulose
` (26 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
So far we have treated the feature capabilities as system wide
and this doesn't help with features that could be detected on
one or more CPUs (e.g, KPTI, Software prefetch). This patch
splits the feature detection to two phases :
1) Local CPU features are checked on all boot time active CPUs.
2) System wide features are checked only once after all CPUs are
active.
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index a6e124063e21..45bac0c4ae91 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -551,11 +551,13 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
}
/*
- * Run the errata work around checks on the boot CPU, once we have
- * initialised the cpu feature infrastructure.
+ * Run the errata work around and local feature checks on the
+ * boot CPU, once we have initialised the cpu feature infrastructure.
*/
update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
+ update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
+ "detected feature:");
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1411,15 +1413,18 @@ void check_local_cpu_capabilities(void)
/*
* If we haven't finalised the system capabilities, this CPU gets
- * a chance to update the errata work arounds.
+ * a chance to update the errata work arounds and local features.
* Otherwise, this CPU should verify that it has all the system
* advertised capabilities.
*/
- if (!sys_caps_initialised)
+ if (!sys_caps_initialised) {
update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
- else
+ update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
+ "detected feature:");
+ } else {
verify_local_cpu_capabilities();
+ }
}
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
@@ -1444,7 +1449,7 @@ void __init setup_cpu_features(void)
int cls;
/* Set the CPU feature capabilies */
- update_cpu_capabilities(arm64_features, SCOPE_ALL,
+ update_cpu_capabilities(arm64_features, SCOPE_SYSTEM,
"detected feature:");
update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
"enabling workaround for");
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 09/22] arm64: capabilities: Split the processing of errata work arounds
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (16 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 09/21] arm64: capabilities: Allow features based on local CPU scope Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 18:18 ` Dave Martin
2018-02-09 17:54 ` [PATCH v3 10/22] arm64: capabilities: Allow features based on local CPU scope Suzuki K Poulose
` (25 subsequent siblings)
43 siblings, 1 reply; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
Right now we run through the errata workarounds check on all boot
active CPUs, with SCOPE_ALL. This doesn't help with detecting the
errata's with a SYSTEM_SCOPE. While nobody uses it, let us clean
this up in preparation for merging capability handling.
So, we run the checks with SCOPE_LOCAL_CPU on all CPUs and SCOPE_SYSTEM
checks are run only once after all the boot time CPUs are active.
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9f52678a4df1..a6e124063e21 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -554,7 +554,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
* Run the errata work around checks on the boot CPU, once we have
* initialised the cpu feature infrastructure.
*/
- update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
}
@@ -1416,7 +1416,7 @@ void check_local_cpu_capabilities(void)
* advertised capabilities.
*/
if (!sys_caps_initialised)
- update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
else
verify_local_cpu_capabilities();
@@ -1446,6 +1446,8 @@ void __init setup_cpu_features(void)
/* Set the CPU feature capabilies */
update_cpu_capabilities(arm64_features, SCOPE_ALL,
"detected feature:");
+ update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
+ "enabling workaround for");
enable_cpu_capabilities(arm64_features, SCOPE_ALL);
enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
mark_const_caps_ready();
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 09/22] arm64: capabilities: Split the processing of errata work arounds
2018-02-09 17:54 ` [PATCH v3 09/22] arm64: capabilities: Split the processing of errata work arounds Suzuki K Poulose
@ 2018-02-09 18:18 ` Dave Martin
2018-03-08 12:24 ` Suzuki K Poulose
0 siblings, 1 reply; 62+ messages in thread
From: Dave Martin @ 2018-02-09 18:18 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Feb 09, 2018 at 05:54:48PM +0000, Suzuki K Poulose wrote:
> Right now we run through the errata workarounds check on all boot
> active CPUs, with SCOPE_ALL. This doesn't help with detecting the
> errata's with a SYSTEM_SCOPE. While nobody uses it, let us clean
> this up in preparation for merging capability handling.
errata; and SCOPE_SYSTEM (if I'm being pedantic)
To make it clearer this this is adding support for new behaviour rather
than changing old behaviour, perhaps change "This doesn't help ...
While nobody uses it ..." to something like:
"This wouldn't help for detecting erratum workarounds with SCOPE_SYSTEM.
There are none yet, but we plan to introduce some: let us clean this up
so that such workarounds can be detected and enabled correctly."
>
> So, we run the checks with SCOPE_LOCAL_CPU on all CPUs and SCOPE_SYSTEM
> checks are run only once after all the boot time CPUs are active.
>
> Cc: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Otherwise,
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
> ---
> arch/arm64/kernel/cpufeature.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 9f52678a4df1..a6e124063e21 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -554,7 +554,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
> * Run the errata work around checks on the boot CPU, once we have
> * initialised the cpu feature infrastructure.
> */
> - update_cpu_capabilities(arm64_errata, SCOPE_ALL,
> + update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
> "enabling workaround for");
> }
>
> @@ -1416,7 +1416,7 @@ void check_local_cpu_capabilities(void)
> * advertised capabilities.
> */
> if (!sys_caps_initialised)
> - update_cpu_capabilities(arm64_errata, SCOPE_ALL,
> + update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
> "enabling workaround for");
> else
> verify_local_cpu_capabilities();
> @@ -1446,6 +1446,8 @@ void __init setup_cpu_features(void)
> /* Set the CPU feature capabilies */
> update_cpu_capabilities(arm64_features, SCOPE_ALL,
> "detected feature:");
> + update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
> + "enabling workaround for");
> enable_cpu_capabilities(arm64_features, SCOPE_ALL);
> enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
> mark_const_caps_ready();
> --
> 2.14.3
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v3 09/22] arm64: capabilities: Split the processing of errata work arounds
2018-02-09 18:18 ` Dave Martin
@ 2018-03-08 12:24 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-08 12:24 UTC (permalink / raw)
To: linux-arm-kernel
On 09/02/18 18:18, Dave Martin wrote:
> On Fri, Feb 09, 2018 at 05:54:48PM +0000, Suzuki K Poulose wrote:
>> Right now we run through the errata workarounds check on all boot
>> active CPUs, with SCOPE_ALL. This doesn't help with detecting the
>> errata's with a SYSTEM_SCOPE. While nobody uses it, let us clean
>> this up in preparation for merging capability handling.
>
> errata; and SCOPE_SYSTEM (if I'm being pedantic)
>
> To make it clearer this this is adding support for new behaviour rather
> than changing old behaviour, perhaps change "This doesn't help ...
> While nobody uses it ..." to something like:
>
> "This wouldn't help for detecting erratum workarounds with SCOPE_SYSTEM.
> There are none yet, but we plan to introduce some: let us clean this up
> so that such workarounds can be detected and enabled correctly."
Fixed
>>
>> So, we run the checks with SCOPE_LOCAL_CPU on all CPUs and SCOPE_SYSTEM
>> checks are run only once after all the boot time CPUs are active.
>>
>> Cc: Dave Martin <dave.martin@arm.com>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>
> Otherwise,
>
> Reviewed-by: Dave Martin <Dave.Martin@arm.com>
Thanks
Suzuki
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v3 10/22] arm64: capabilities: Allow features based on local CPU scope
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (17 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 09/22] arm64: capabilities: Split the processing of errata work arounds Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 18:22 ` Dave Martin
2018-02-09 17:54 ` [PATCH v3 10/21] arm64: capabilities: Group handling of features and errata workarounds Suzuki K Poulose
` (24 subsequent siblings)
43 siblings, 1 reply; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
So far we have treated the feature capabilities as system wide
and this doesn't help with features that could be detected on
one or more CPUs (e.g, KPTI, Software prefetch). This patch
splits the feature detection to two phases :
1) Local CPU features are checked on all boot time active CPUs.
2) System wide features are checked only once after all CPUs are
active.
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index a6e124063e21..45bac0c4ae91 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -551,11 +551,13 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
}
/*
- * Run the errata work around checks on the boot CPU, once we have
- * initialised the cpu feature infrastructure.
+ * Run the errata work around and local feature checks on the
+ * boot CPU, once we have initialised the cpu feature infrastructure.
*/
update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
+ update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
+ "detected feature:");
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1411,15 +1413,18 @@ void check_local_cpu_capabilities(void)
/*
* If we haven't finalised the system capabilities, this CPU gets
- * a chance to update the errata work arounds.
+ * a chance to update the errata work arounds and local features.
* Otherwise, this CPU should verify that it has all the system
* advertised capabilities.
*/
- if (!sys_caps_initialised)
+ if (!sys_caps_initialised) {
update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
- else
+ update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
+ "detected feature:");
+ } else {
verify_local_cpu_capabilities();
+ }
}
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
@@ -1444,7 +1449,7 @@ void __init setup_cpu_features(void)
int cls;
/* Set the CPU feature capabilies */
- update_cpu_capabilities(arm64_features, SCOPE_ALL,
+ update_cpu_capabilities(arm64_features, SCOPE_SYSTEM,
"detected feature:");
update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
"enabling workaround for");
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 10/22] arm64: capabilities: Allow features based on local CPU scope
2018-02-09 17:54 ` [PATCH v3 10/22] arm64: capabilities: Allow features based on local CPU scope Suzuki K Poulose
@ 2018-02-09 18:22 ` Dave Martin
2018-03-08 12:14 ` Suzuki K Poulose
0 siblings, 1 reply; 62+ messages in thread
From: Dave Martin @ 2018-02-09 18:22 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Feb 09, 2018 at 05:54:49PM +0000, Suzuki K Poulose wrote:
> So far we have treated the feature capabilities as system wide
> and this doesn't help with features that could be detected on
doesn't -> wouldn't (since no such features exist yet)
detected to "detected locally" (maybe)
> one or more CPUs (e.g, KPTI, Software prefetch). This patch
> splits the feature detection to two phases :
>
> 1) Local CPU features are checked on all boot time active CPUs.
> 2) System wide features are checked only once after all CPUs are
> active.
>
> Cc: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Otherwise,
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
> ---
> arch/arm64/kernel/cpufeature.c | 17 +++++++++++------
> 1 file changed, 11 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index a6e124063e21..45bac0c4ae91 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -551,11 +551,13 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
> }
>
> /*
> - * Run the errata work around checks on the boot CPU, once we have
> - * initialised the cpu feature infrastructure.
> + * Run the errata work around and local feature checks on the
> + * boot CPU, once we have initialised the cpu feature infrastructure.
> */
> update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
> "enabling workaround for");
> + update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
> + "detected feature:");
> }
>
> static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
> @@ -1411,15 +1413,18 @@ void check_local_cpu_capabilities(void)
>
> /*
> * If we haven't finalised the system capabilities, this CPU gets
> - * a chance to update the errata work arounds.
> + * a chance to update the errata work arounds and local features.
> * Otherwise, this CPU should verify that it has all the system
> * advertised capabilities.
> */
> - if (!sys_caps_initialised)
> + if (!sys_caps_initialised) {
> update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
> "enabling workaround for");
> - else
> + update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
> + "detected feature:");
> + } else {
> verify_local_cpu_capabilities();
> + }
> }
>
> DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
> @@ -1444,7 +1449,7 @@ void __init setup_cpu_features(void)
> int cls;
>
> /* Set the CPU feature capabilies */
> - update_cpu_capabilities(arm64_features, SCOPE_ALL,
> + update_cpu_capabilities(arm64_features, SCOPE_SYSTEM,
> "detected feature:");
> update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
> "enabling workaround for");
> --
> 2.14.3
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 62+ messages in thread* [PATCH v3 10/22] arm64: capabilities: Allow features based on local CPU scope
2018-02-09 18:22 ` Dave Martin
@ 2018-03-08 12:14 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-08 12:14 UTC (permalink / raw)
To: linux-arm-kernel
On 09/02/18 18:22, Dave Martin wrote:
> On Fri, Feb 09, 2018 at 05:54:49PM +0000, Suzuki K Poulose wrote:
>> So far we have treated the feature capabilities as system wide
>> and this doesn't help with features that could be detected on
>
> doesn't -> wouldn't (since no such features exist yet)
>
> detected to "detected locally" (maybe)
>
Sure, fixed.
>> one or more CPUs (e.g, KPTI, Software prefetch). This patch
>> splits the feature detection to two phases :
>>
>> 1) Local CPU features are checked on all boot time active CPUs.
>> 2) System wide features are checked only once after all CPUs are
>> active.
>>
>> Cc: Dave Martin <dave.martin@arm.com>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>
> Otherwise,
>
> Reviewed-by: Dave Martin <Dave.Martin@arm.com>
>
Thanks,
Suzuki
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v3 10/21] arm64: capabilities: Group handling of features and errata workarounds
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (18 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 10/22] arm64: capabilities: Allow features based on local CPU scope Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 11/22] " Suzuki K Poulose
` (23 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
Now that the features and errata workarounds have the same
rules and flow, group the handling of the tables.
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 72 ++++++++++++++++++++++++------------------
1 file changed, 41 insertions(+), 31 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 45bac0c4ae91..9b111d560b30 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -504,8 +504,7 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
-static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- u16 scope_mask, const char *info);
+static void update_cpu_capabilities(u16 scope_mask);
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
@@ -554,10 +553,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
* Run the errata work around and local feature checks on the
* boot CPU, once we have initialised the cpu feature infrastructure.
*/
- update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
- "enabling workaround for");
- update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
- "detected feature:");
+ update_cpu_capabilities(SCOPE_LOCAL_CPU);
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1225,7 +1221,7 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
return false;
}
-static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+static void __update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
u16 scope_mask, const char *info)
{
scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
@@ -1240,6 +1236,14 @@ static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
}
}
+static void update_cpu_capabilities(u16 scope_mask)
+{
+ __update_cpu_capabilities(arm64_features, scope_mask,
+ "detected feature:");
+ __update_cpu_capabilities(arm64_errata, scope_mask,
+ "enabling workaround for");
+}
+
static int __enable_cpu_capability(void *arg)
{
const struct arm64_cpu_capabilities *cap = arg;
@@ -1253,7 +1257,7 @@ static int __enable_cpu_capability(void *arg)
* CPUs
*/
static void __init
-enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_mask)
+__enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_mask)
{
scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
@@ -1277,6 +1281,12 @@ enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_mas
}
}
+static void __init enable_cpu_capabilities(u16 scope_mask)
+{
+ __enable_cpu_capabilities(arm64_features, scope_mask);
+ __enable_cpu_capabilities(arm64_errata, scope_mask);
+}
+
/*
* Run through the list of capabilities to check for conflicts.
* If the system has already detected a capability, take necessary
@@ -1332,6 +1342,12 @@ static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_li
return true;
}
+static bool verify_local_cpu_caps(u16 scope_mask)
+{
+ return __verify_local_cpu_caps(arm64_errata, scope_mask) &&
+ __verify_local_cpu_caps(arm64_features, scope_mask);
+}
+
/*
* Check for CPU features that are used in early boot
* based on the Boot CPU value.
@@ -1371,7 +1387,6 @@ static void verify_sve_features(void)
/* Add checks on other ZCR bits here if necessary */
}
-
/*
* Run through the enabled system capabilities and enable() it on this CPU.
* The capabilities were decided based on the available CPUs at the boot time.
@@ -1382,15 +1397,9 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_capabilities(void)
{
- /*
- * The CPU Errata work arounds are detected and applied at boot time
- * and the related information is freed soon after. If the new CPU
- * requires an errata not detected at boot, fail this CPU.
- */
- if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
- cpu_die_early();
- if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
+ if (!verify_local_cpu_caps(SCOPE_ALL))
cpu_die_early();
+
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
@@ -1417,14 +1426,10 @@ void check_local_cpu_capabilities(void)
* Otherwise, this CPU should verify that it has all the system
* advertised capabilities.
*/
- if (!sys_caps_initialised) {
- update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
- "enabling workaround for");
- update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
- "detected feature:");
- } else {
+ if (!sys_caps_initialised)
+ update_cpu_capabilities(SCOPE_LOCAL_CPU);
+ else
verify_local_cpu_capabilities();
- }
}
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
@@ -1443,18 +1448,23 @@ bool this_cpu_has_cap(unsigned int cap)
__this_cpu_has_cap(arm64_errata, cap));
}
+static void __init setup_system_capabilities(void)
+{
+ /*
+ * We have finalised the system-wide safe feature
+ * registers, finalise the capabilities that depend
+ * on it. Also enable all the available capabilities.
+ */
+ update_cpu_capabilities(SCOPE_SYSTEM);
+ enable_cpu_capabilities(SCOPE_ALL);
+}
+
void __init setup_cpu_features(void)
{
u32 cwg;
int cls;
- /* Set the CPU feature capabilies */
- update_cpu_capabilities(arm64_features, SCOPE_SYSTEM,
- "detected feature:");
- update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
- "enabling workaround for");
- enable_cpu_capabilities(arm64_features, SCOPE_ALL);
- enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
+ setup_system_capabilities();
mark_const_caps_ready();
setup_elf_hwcaps(arm64_elf_hwcaps);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 11/22] arm64: capabilities: Group handling of features and errata workarounds
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (19 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 10/21] arm64: capabilities: Group handling of features and errata workarounds Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 18:25 ` Dave Martin
2018-02-09 17:54 ` [PATCH v3 11/21] arm64: capabilities: Introduce weak features based on local CPU Suzuki K Poulose
` (22 subsequent siblings)
43 siblings, 1 reply; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
Now that the features and errata workarounds have the same
rules and flow, group the handling of the tables.
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 72 ++++++++++++++++++++++++------------------
1 file changed, 41 insertions(+), 31 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 45bac0c4ae91..9b111d560b30 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -504,8 +504,7 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
-static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- u16 scope_mask, const char *info);
+static void update_cpu_capabilities(u16 scope_mask);
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
@@ -554,10 +553,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
* Run the errata work around and local feature checks on the
* boot CPU, once we have initialised the cpu feature infrastructure.
*/
- update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
- "enabling workaround for");
- update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
- "detected feature:");
+ update_cpu_capabilities(SCOPE_LOCAL_CPU);
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1225,7 +1221,7 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
return false;
}
-static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+static void __update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
u16 scope_mask, const char *info)
{
scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
@@ -1240,6 +1236,14 @@ static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
}
}
+static void update_cpu_capabilities(u16 scope_mask)
+{
+ __update_cpu_capabilities(arm64_features, scope_mask,
+ "detected feature:");
+ __update_cpu_capabilities(arm64_errata, scope_mask,
+ "enabling workaround for");
+}
+
static int __enable_cpu_capability(void *arg)
{
const struct arm64_cpu_capabilities *cap = arg;
@@ -1253,7 +1257,7 @@ static int __enable_cpu_capability(void *arg)
* CPUs
*/
static void __init
-enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_mask)
+__enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_mask)
{
scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
@@ -1277,6 +1281,12 @@ enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_mas
}
}
+static void __init enable_cpu_capabilities(u16 scope_mask)
+{
+ __enable_cpu_capabilities(arm64_features, scope_mask);
+ __enable_cpu_capabilities(arm64_errata, scope_mask);
+}
+
/*
* Run through the list of capabilities to check for conflicts.
* If the system has already detected a capability, take necessary
@@ -1332,6 +1342,12 @@ static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_li
return true;
}
+static bool verify_local_cpu_caps(u16 scope_mask)
+{
+ return __verify_local_cpu_caps(arm64_errata, scope_mask) &&
+ __verify_local_cpu_caps(arm64_features, scope_mask);
+}
+
/*
* Check for CPU features that are used in early boot
* based on the Boot CPU value.
@@ -1371,7 +1387,6 @@ static void verify_sve_features(void)
/* Add checks on other ZCR bits here if necessary */
}
-
/*
* Run through the enabled system capabilities and enable() it on this CPU.
* The capabilities were decided based on the available CPUs at the boot time.
@@ -1382,15 +1397,9 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_capabilities(void)
{
- /*
- * The CPU Errata work arounds are detected and applied at boot time
- * and the related information is freed soon after. If the new CPU
- * requires an errata not detected at boot, fail this CPU.
- */
- if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
- cpu_die_early();
- if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
+ if (!verify_local_cpu_caps(SCOPE_ALL))
cpu_die_early();
+
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
@@ -1417,14 +1426,10 @@ void check_local_cpu_capabilities(void)
* Otherwise, this CPU should verify that it has all the system
* advertised capabilities.
*/
- if (!sys_caps_initialised) {
- update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
- "enabling workaround for");
- update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
- "detected feature:");
- } else {
+ if (!sys_caps_initialised)
+ update_cpu_capabilities(SCOPE_LOCAL_CPU);
+ else
verify_local_cpu_capabilities();
- }
}
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
@@ -1443,18 +1448,23 @@ bool this_cpu_has_cap(unsigned int cap)
__this_cpu_has_cap(arm64_errata, cap));
}
+static void __init setup_system_capabilities(void)
+{
+ /*
+ * We have finalised the system-wide safe feature
+ * registers, finalise the capabilities that depend
+ * on it. Also enable all the available capabilities.
+ */
+ update_cpu_capabilities(SCOPE_SYSTEM);
+ enable_cpu_capabilities(SCOPE_ALL);
+}
+
void __init setup_cpu_features(void)
{
u32 cwg;
int cls;
- /* Set the CPU feature capabilies */
- update_cpu_capabilities(arm64_features, SCOPE_SYSTEM,
- "detected feature:");
- update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
- "enabling workaround for");
- enable_cpu_capabilities(arm64_features, SCOPE_ALL);
- enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
+ setup_system_capabilities();
mark_const_caps_ready();
setup_elf_hwcaps(arm64_elf_hwcaps);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 11/22] arm64: capabilities: Group handling of features and errata workarounds
2018-02-09 17:54 ` [PATCH v3 11/22] " Suzuki K Poulose
@ 2018-02-09 18:25 ` Dave Martin
0 siblings, 0 replies; 62+ messages in thread
From: Dave Martin @ 2018-02-09 18:25 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Feb 09, 2018 at 05:54:51PM +0000, Suzuki K Poulose wrote:
> Now that the features and errata workarounds have the same
> rules and flow, group the handling of the tables.
>
> Cc: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
The naming gets a bit confusing, particularly with
setup_system_capabilities() which enables for SCOPE_ALL, but I can't
think of an obviously better name.
Otherwise the refactoring looks sensible, so FWIW,
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
> ---
> arch/arm64/kernel/cpufeature.c | 72 ++++++++++++++++++++++++------------------
> 1 file changed, 41 insertions(+), 31 deletions(-)
>
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 45bac0c4ae91..9b111d560b30 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -504,8 +504,7 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
> }
>
> extern const struct arm64_cpu_capabilities arm64_errata[];
> -static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
> - u16 scope_mask, const char *info);
> +static void update_cpu_capabilities(u16 scope_mask);
>
> void __init init_cpu_features(struct cpuinfo_arm64 *info)
> {
> @@ -554,10 +553,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
> * Run the errata work around and local feature checks on the
> * boot CPU, once we have initialised the cpu feature infrastructure.
> */
> - update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
> - "enabling workaround for");
> - update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
> - "detected feature:");
> + update_cpu_capabilities(SCOPE_LOCAL_CPU);
> }
>
> static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
> @@ -1225,7 +1221,7 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
> return false;
> }
>
> -static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
> +static void __update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
> u16 scope_mask, const char *info)
> {
> scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
> @@ -1240,6 +1236,14 @@ static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
> }
> }
>
> +static void update_cpu_capabilities(u16 scope_mask)
> +{
> + __update_cpu_capabilities(arm64_features, scope_mask,
> + "detected feature:");
> + __update_cpu_capabilities(arm64_errata, scope_mask,
> + "enabling workaround for");
> +}
> +
> static int __enable_cpu_capability(void *arg)
> {
> const struct arm64_cpu_capabilities *cap = arg;
> @@ -1253,7 +1257,7 @@ static int __enable_cpu_capability(void *arg)
> * CPUs
> */
> static void __init
> -enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_mask)
> +__enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_mask)
> {
> scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
> for (; caps->matches; caps++) {
> @@ -1277,6 +1281,12 @@ enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_mas
> }
> }
>
> +static void __init enable_cpu_capabilities(u16 scope_mask)
> +{
> + __enable_cpu_capabilities(arm64_features, scope_mask);
> + __enable_cpu_capabilities(arm64_errata, scope_mask);
> +}
> +
> /*
> * Run through the list of capabilities to check for conflicts.
> * If the system has already detected a capability, take necessary
> @@ -1332,6 +1342,12 @@ static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_li
> return true;
> }
>
> +static bool verify_local_cpu_caps(u16 scope_mask)
> +{
> + return __verify_local_cpu_caps(arm64_errata, scope_mask) &&
> + __verify_local_cpu_caps(arm64_features, scope_mask);
> +}
> +
> /*
> * Check for CPU features that are used in early boot
> * based on the Boot CPU value.
> @@ -1371,7 +1387,6 @@ static void verify_sve_features(void)
> /* Add checks on other ZCR bits here if necessary */
> }
>
> -
> /*
> * Run through the enabled system capabilities and enable() it on this CPU.
> * The capabilities were decided based on the available CPUs at the boot time.
> @@ -1382,15 +1397,9 @@ static void verify_sve_features(void)
> */
> static void verify_local_cpu_capabilities(void)
> {
> - /*
> - * The CPU Errata work arounds are detected and applied at boot time
> - * and the related information is freed soon after. If the new CPU
> - * requires an errata not detected at boot, fail this CPU.
> - */
> - if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
> - cpu_die_early();
> - if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
> + if (!verify_local_cpu_caps(SCOPE_ALL))
> cpu_die_early();
> +
> verify_local_elf_hwcaps(arm64_elf_hwcaps);
>
> if (system_supports_32bit_el0())
> @@ -1417,14 +1426,10 @@ void check_local_cpu_capabilities(void)
> * Otherwise, this CPU should verify that it has all the system
> * advertised capabilities.
> */
> - if (!sys_caps_initialised) {
> - update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
> - "enabling workaround for");
> - update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
> - "detected feature:");
> - } else {
> + if (!sys_caps_initialised)
> + update_cpu_capabilities(SCOPE_LOCAL_CPU);
> + else
> verify_local_cpu_capabilities();
> - }
> }
>
> DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
> @@ -1443,18 +1448,23 @@ bool this_cpu_has_cap(unsigned int cap)
> __this_cpu_has_cap(arm64_errata, cap));
> }
>
> +static void __init setup_system_capabilities(void)
> +{
> + /*
> + * We have finalised the system-wide safe feature
> + * registers, finalise the capabilities that depend
> + * on it. Also enable all the available capabilities.
> + */
> + update_cpu_capabilities(SCOPE_SYSTEM);
> + enable_cpu_capabilities(SCOPE_ALL);
> +}
> +
> void __init setup_cpu_features(void)
> {
> u32 cwg;
> int cls;
>
> - /* Set the CPU feature capabilies */
> - update_cpu_capabilities(arm64_features, SCOPE_SYSTEM,
> - "detected feature:");
> - update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
> - "enabling workaround for");
> - enable_cpu_capabilities(arm64_features, SCOPE_ALL);
> - enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
> + setup_system_capabilities();
> mark_const_caps_ready();
> setup_elf_hwcaps(arm64_elf_hwcaps);
>
> --
> 2.14.3
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v3 11/21] arm64: capabilities: Introduce weak features based on local CPU
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (20 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 11/22] " Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 12/22] " Suzuki K Poulose
` (21 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
Now that we have the flexibility of defining system features based
on individual CPUs, introduce CPU feature type that can be detected
on a local SCOPE and ignores the conflict on late CPUs. This is
applicable for ARM64_HAS_NO_HW_PREFETCH, where it is fine for
the system to have CPUs without hardware prefetch turning up
later. We only suffer a performance penalty, nothing fatal.
Cc: Will Deacon <will.deacon@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 8 ++++++++
arch/arm64/kernel/cpufeature.c | 2 +-
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 6e354a98bc24..49c4ae331ca7 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -230,6 +230,14 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
*/
#define ARM64_CPUCAP_SYSTEM_FEATURE \
(ARM64_CPUCAP_SCOPE_SYSTEM | ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
+/*
+ * CPU feature detected at boot time based on feature of one or more CPUs.
+ * All possible conflicts for a late CPU are ignored.
+ */
+#define ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE \
+ (ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
+ ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU | \
+ ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
struct arm64_cpu_capabilities {
const char *desc;
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9b111d560b30..373173bc3ab3 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -992,7 +992,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Software prefetching using PRFM",
.capability = ARM64_HAS_NO_HW_PREFETCH,
- .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE,
.matches = has_no_hw_prefetch,
},
#ifdef CONFIG_ARM64_UAO
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 12/22] arm64: capabilities: Introduce weak features based on local CPU
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (21 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 11/21] arm64: capabilities: Introduce weak features based on local CPU Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 12/21] arm64: capabilities: Restrict KPTI detection to boot-time CPUs Suzuki K Poulose
` (20 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
Now that we have the flexibility of defining system features based
on individual CPUs, introduce CPU feature type that can be detected
on a local SCOPE and ignores the conflict on late CPUs. This is
applicable for ARM64_HAS_NO_HW_PREFETCH, where it is fine for
the system to have CPUs without hardware prefetch turning up
later. We only suffer a performance penalty, nothing fatal.
Cc: Will Deacon <will.deacon@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 8 ++++++++
arch/arm64/kernel/cpufeature.c | 2 +-
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 6e354a98bc24..49c4ae331ca7 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -230,6 +230,14 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
*/
#define ARM64_CPUCAP_SYSTEM_FEATURE \
(ARM64_CPUCAP_SCOPE_SYSTEM | ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
+/*
+ * CPU feature detected at boot time based on feature of one or more CPUs.
+ * All possible conflicts for a late CPU are ignored.
+ */
+#define ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE \
+ (ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
+ ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU | \
+ ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
struct arm64_cpu_capabilities {
const char *desc;
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9b111d560b30..373173bc3ab3 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -992,7 +992,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Software prefetching using PRFM",
.capability = ARM64_HAS_NO_HW_PREFETCH,
- .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE,
.matches = has_no_hw_prefetch,
},
#ifdef CONFIG_ARM64_UAO
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 12/21] arm64: capabilities: Restrict KPTI detection to boot-time CPUs
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (22 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 12/22] " Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 13/21] arm64: capabilities: Add support for features enabled early Suzuki K Poulose
` (19 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
KPTI is treated as a system wide feature and is only detected if all
the CPUs in the sysetm needs the defense, unless it is forced via kernel
command line. This leaves a system with a mix of CPUs with and without
the defense vulnerable. Also, if a late CPU needs KPTI but KPTI was not
activated at boot time, the CPU is currently allowed to boot, which is a
potential security vulnerability.
This patch ensures that the KPTI is turned on if at least one CPU detects
the capability (i.e, change scope to SCOPE_LOCAL_CPU). Also rejetcs a late
CPU, if it requires the defense, when the system hasn't enabled it,
Cc: Will Deacon <will.deacon@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 9 +++++++++
arch/arm64/kernel/cpufeature.c | 16 +++++++++++-----
2 files changed, 20 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 49c4ae331ca7..383c69c95f23 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -239,6 +239,15 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU | \
ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
+/*
+ * CPU feature detected at boot time, on one or more CPUs. A late CPU
+ * is not allowed to have the capability when the system doesn't have it.
+ * It is Ok for a late CPU to miss the feature.
+ */
+#define ARM64_CPUCAP_BOOT_RESTRICTED_CPU_LOCAL_FEATURE \
+ (ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
+ ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
+
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 373173bc3ab3..a15be4db6bf4 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -862,10 +862,9 @@ static bool has_no_fpsimd(const struct arm64_cpu_capabilities *entry, int __unus
static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */
static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
- int __unused)
+ int scope)
{
char const *str = "command line option";
- u64 pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
/*
* For reasons that aren't entirely clear, enabling KPTI on Cavium
@@ -896,8 +895,7 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
}
/* Defer to CPU feature registers */
- return !cpuid_feature_extract_unsigned_field(pfr0,
- ID_AA64PFR0_CSV3_SHIFT);
+ return !has_cpuid_feature(entry, scope);
}
static void
@@ -1044,7 +1042,15 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Kernel page table isolation (KPTI)",
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
- .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .type = ARM64_CPUCAP_BOOT_RESTRICTED_CPU_LOCAL_FEATURE,
+ /*
+ * The ID feature fields below are used to indicate that
+ * the CPU doesn't need KPTI. See unmap_kernel_at_el0 for
+ * more details.
+ */
+ .sys_reg = SYS_ID_AA64PFR0_EL1,
+ .field_pos = ID_AA64PFR0_CSV3_SHIFT,
+ .min_field_value = 1,
.matches = unmap_kernel_at_el0,
.cpu_enable = kpti_install_ng_mappings,
},
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 13/21] arm64: capabilities: Add support for features enabled early
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (23 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 12/21] arm64: capabilities: Restrict KPTI detection to boot-time CPUs Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 13/22] arm64: capabilities: Restrict KPTI detection to boot-time CPUs Suzuki K Poulose
` (18 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
The kernel detects and uses some of the features based on the boot
CPU and expects that all the following CPUs conform to it. e.g,
with VHE and the boot CPU running at EL2, the kernel decides to
keep the kernel running at EL2. If another CPU is brought up without
this capability, we use custom hooks (via check_early_cpu_features())
to handle it. To handle such capabilities add support for detecting
and enabling capabilities based on the boot CPU.
A bit is added to indicate if the capability should be detected
early on the boot CPU. The infrastructure then ensures that such
capabilities are probed and "enabled" early on in the boot CPU
and, enabled on the subsequent CPUs.
Cc: Julien Thierry <julien.thierry@arm.com>
Cc: Dave Martin <dave.martin@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 46 +++++++++++++++++++++++++++++-------
arch/arm64/kernel/cpufeature.c | 47 +++++++++++++++++++++++++++----------
2 files changed, 72 insertions(+), 21 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 383c69c95f23..5f56a8342065 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -104,7 +104,7 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* some checks at runtime. This could be, e.g, checking the value of a field
* in CPU ID feature register or checking the cpu model. The capability
* provides a call back ( @matches() ) to perform the check.
- * Scope defines how the checks should be performed. There are two cases:
+ * Scope defines how the checks should be performed. There are three cases:
*
* a) SCOPE_LOCAL_CPU: check all the CPUs and "detect" if at least one
* matches. This implies, we have to run the check on all the booting
@@ -117,6 +117,11 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* field in one of the CPU ID feature registers, we use the sanitised
* value of the register from the CPU feature infrastructure to make
* the decision.
+ * Or
+ * c) SCOPE_BOOT_CPU: Check only on the primary boot CPU to detect the feature.
+ * This category is for features that are "finalised" (or used) by the kernel
+ * very early even before the SMP cpus are brought up.
+ *
* The process of detection is usually denoted by "update" capability state
* in the code.
*
@@ -133,6 +138,11 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* w.r.t the capability. e.g, all secondary CPUs are treated "late CPUs" for
* capabilities determined by the boot CPU.
*
+ * At the moment there are two passes of finalising the capabilities.
+ * a) Boot CPU scope capabilities - Finalised by primary boot CPU via
+ * setup_boot_cpu_capabilities().
+ * b) Everything except (a) - Run via setup_system_capabilities().
+ *
* 3) Verification: When a CPU is brought online (e.g, by user or by the kernel),
* the kernel should make sure that it is safe to use the CPU, by verifying
* that the CPU is compliant with the state of the capabilities finalised
@@ -141,12 +151,19 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* secondary_start_kernel()-> check_local_cpu_capabilities()
*
* As explained in (2) above, capabilities could be finalised at different
- * points in the execution. Each CPU is verified against the "finalised"
- * capabilities and if there is a conflict, the kernel takes an action, based
- * on the severity (e.g, a CPU could be prevented from booting or cause a
- * kernel panic). The CPU is allowed to "affect" the state of the capability,
- * if it has not been finalised already. See section 5 for more details on
- * conflicts.
+ * points in the execution. Each newly booted CPU is verified against those
+ * capabilities that have been finalised by the time it boots.
+ *
+ * a) SCOPE_BOOT_CPU : All CPUs are verified against the capability except
+ * for the primary boot CPU.
+ *
+ * b) SCOPE_LOCAL_CPU, SCOPE_SYSTEM: All CPUs hotplugged on by the user
+ * after the kernel boot are verified against the capability.
+ *
+ * If there is a conflict, the kernel takes an action, based on the severity
+ * (e.g, a CPU could be prevented from booting or cause a kernel panic).
+ * The CPU is allowed to "affect" the state of the capability, if it has not
+ * been finalised already. See section 5 for more details on conflicts.
*
* 4) Action: As mentioned in (2), the kernel can take an action for each detected
* capability, on all CPUs on the system. Appropriate actions include, turning
@@ -193,15 +210,26 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
*/
-/* Decide how the capability is detected. On a local CPU vs System wide */
+/*
+ * Decide how the capability is detected.
+ * On any local CPU vs System wide vs the primary boot CPU
+ */
#define ARM64_CPUCAP_SCOPE_LOCAL_CPU ((u16)BIT(0))
#define ARM64_CPUCAP_SCOPE_SYSTEM ((u16)BIT(1))
+/*
+ * The capabilitiy is detected on the Boot CPU and is used by kernel
+ * during early boot. i.e, the capability should be "detected" and "enabled"
+ * as early as possibly on all booting CPUs.
+ */
+#define ARM64_CPUCAP_SCOPE_BOOT_CPU ((u16)BIT(2))
#define ARM64_CPUCAP_SCOPE_MASK \
(ARM64_CPUCAP_SCOPE_SYSTEM | \
- ARM64_CPUCAP_SCOPE_LOCAL_CPU)
+ ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
+ ARM64_CPUCAP_SCOPE_BOOT_CPU)
#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
+#define SCOPE_BOOT_CPU ARM64_CPUCAP_SCOPE_BOOT_CPU
#define SCOPE_ALL ARM64_CPUCAP_SCOPE_MASK
/*
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index a15be4db6bf4..7625e2962e2b 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -504,7 +504,7 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
-static void update_cpu_capabilities(u16 scope_mask);
+static void __init setup_boot_cpu_capabilities(void);
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
@@ -550,10 +550,10 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
}
/*
- * Run the errata work around and local feature checks on the
- * boot CPU, once we have initialised the cpu feature infrastructure.
+ * Detect and enable early CPU capabilities based on the boot CPU,
+ * after we have initialised the CPU feature infrastructure.
*/
- update_cpu_capabilities(SCOPE_LOCAL_CPU);
+ setup_boot_cpu_capabilities();
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1277,12 +1277,21 @@ __enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_m
if (caps->cpu_enable) {
/*
- * Use stop_machine() as it schedules the work allowing
- * us to modify PSTATE, instead of on_each_cpu() which
- * uses an IPI, giving us a PSTATE that disappears when
- * we return.
+ * If we are dealing with a boot CPU capability, we
+ * have to enable this only on the Boot CPU, where it
+ * is detected. All the secondaries enable it via
+ * check_local_cpu_capabilities().
+ *
+ * Otherwise, use stop_machine() as it schedules the
+ * work allowing us to modify PSTATE, instead of
+ * on_each_cpu() which uses an IPI, giving us a PSTATE
+ * that disappears when we return.
*/
- stop_machine(__enable_cpu_capability, (void *)caps, cpu_online_mask);
+ if (scope_mask & SCOPE_BOOT_CPU)
+ caps->cpu_enable(caps);
+ else
+ stop_machine(__enable_cpu_capability,
+ (void *)caps, cpu_online_mask);
}
}
}
@@ -1362,6 +1371,12 @@ static void check_early_cpu_features(void)
{
verify_cpu_run_el();
verify_cpu_asid_bits();
+ /*
+ * Early features are used by the kernel already. If there
+ * is a conflict, we cannot proceed further.
+ */
+ if (!verify_local_cpu_caps(SCOPE_BOOT_CPU))
+ cpu_panic_kernel();
}
static void
@@ -1403,9 +1418,8 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_capabilities(void)
{
- if (!verify_local_cpu_caps(SCOPE_ALL))
+ if (!verify_local_cpu_caps(SCOPE_ALL & ~SCOPE_BOOT_CPU))
cpu_die_early();
-
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
@@ -1438,6 +1452,14 @@ void check_local_cpu_capabilities(void)
verify_local_cpu_capabilities();
}
+static void __init setup_boot_cpu_capabilities(void)
+{
+ /* Detect capabilities with either SCOPE_BOOT_CPU or SCOPE_LOCAL_CPU */
+ update_cpu_capabilities(SCOPE_BOOT_CPU | SCOPE_LOCAL_CPU);
+ /* Enable the SCOPE_BOOT_CPU capabilities alone right away */
+ enable_cpu_capabilities(SCOPE_BOOT_CPU);
+}
+
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
EXPORT_SYMBOL(arm64_const_caps_ready);
@@ -1462,7 +1484,8 @@ static void __init setup_system_capabilities(void)
* on it. Also enable all the available capabilities.
*/
update_cpu_capabilities(SCOPE_SYSTEM);
- enable_cpu_capabilities(SCOPE_ALL);
+ /* Enable all the available capabilities, which are not already enabled. */
+ enable_cpu_capabilities(SCOPE_ALL & ~SCOPE_BOOT_CPU);
}
void __init setup_cpu_features(void)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 13/22] arm64: capabilities: Restrict KPTI detection to boot-time CPUs
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (24 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 13/21] arm64: capabilities: Add support for features enabled early Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 14/22] arm64: capabilities: Add support for features enabled early Suzuki K Poulose
` (17 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
KPTI is treated as a system wide feature and is only detected if all
the CPUs in the sysetm needs the defense, unless it is forced via kernel
command line. This leaves a system with a mix of CPUs with and without
the defense vulnerable. Also, if a late CPU needs KPTI but KPTI was not
activated at boot time, the CPU is currently allowed to boot, which is a
potential security vulnerability.
This patch ensures that the KPTI is turned on if at least one CPU detects
the capability (i.e, change scope to SCOPE_LOCAL_CPU). Also rejetcs a late
CPU, if it requires the defense, when the system hasn't enabled it,
Cc: Will Deacon <will.deacon@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 9 +++++++++
arch/arm64/kernel/cpufeature.c | 16 +++++++++++-----
2 files changed, 20 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 49c4ae331ca7..383c69c95f23 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -239,6 +239,15 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU | \
ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
+/*
+ * CPU feature detected at boot time, on one or more CPUs. A late CPU
+ * is not allowed to have the capability when the system doesn't have it.
+ * It is Ok for a late CPU to miss the feature.
+ */
+#define ARM64_CPUCAP_BOOT_RESTRICTED_CPU_LOCAL_FEATURE \
+ (ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
+ ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
+
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 373173bc3ab3..a15be4db6bf4 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -862,10 +862,9 @@ static bool has_no_fpsimd(const struct arm64_cpu_capabilities *entry, int __unus
static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */
static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
- int __unused)
+ int scope)
{
char const *str = "command line option";
- u64 pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
/*
* For reasons that aren't entirely clear, enabling KPTI on Cavium
@@ -896,8 +895,7 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
}
/* Defer to CPU feature registers */
- return !cpuid_feature_extract_unsigned_field(pfr0,
- ID_AA64PFR0_CSV3_SHIFT);
+ return !has_cpuid_feature(entry, scope);
}
static void
@@ -1044,7 +1042,15 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Kernel page table isolation (KPTI)",
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
- .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .type = ARM64_CPUCAP_BOOT_RESTRICTED_CPU_LOCAL_FEATURE,
+ /*
+ * The ID feature fields below are used to indicate that
+ * the CPU doesn't need KPTI. See unmap_kernel_at_el0 for
+ * more details.
+ */
+ .sys_reg = SYS_ID_AA64PFR0_EL1,
+ .field_pos = ID_AA64PFR0_CSV3_SHIFT,
+ .min_field_value = 1,
.matches = unmap_kernel_at_el0,
.cpu_enable = kpti_install_ng_mappings,
},
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 14/22] arm64: capabilities: Add support for features enabled early
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (25 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 13/22] arm64: capabilities: Restrict KPTI detection to boot-time CPUs Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-12 17:17 ` Dave Martin
2018-02-09 17:54 ` [PATCH v3 14/21] arm64: capabilities: Change scope of VHE to Boot CPU feature Suzuki K Poulose
` (16 subsequent siblings)
43 siblings, 1 reply; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
The kernel detects and uses some of the features based on the boot
CPU and expects that all the following CPUs conform to it. e.g,
with VHE and the boot CPU running at EL2, the kernel decides to
keep the kernel running at EL2. If another CPU is brought up without
this capability, we use custom hooks (via check_early_cpu_features())
to handle it. To handle such capabilities add support for detecting
and enabling capabilities based on the boot CPU.
A bit is added to indicate if the capability should be detected
early on the boot CPU. The infrastructure then ensures that such
capabilities are probed and "enabled" early on in the boot CPU
and, enabled on the subsequent CPUs.
Cc: Julien Thierry <julien.thierry@arm.com>
Cc: Dave Martin <dave.martin@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 46 +++++++++++++++++++++++++++++-------
arch/arm64/kernel/cpufeature.c | 47 +++++++++++++++++++++++++++----------
2 files changed, 72 insertions(+), 21 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 383c69c95f23..5f56a8342065 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -104,7 +104,7 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* some checks at runtime. This could be, e.g, checking the value of a field
* in CPU ID feature register or checking the cpu model. The capability
* provides a call back ( @matches() ) to perform the check.
- * Scope defines how the checks should be performed. There are two cases:
+ * Scope defines how the checks should be performed. There are three cases:
*
* a) SCOPE_LOCAL_CPU: check all the CPUs and "detect" if at least one
* matches. This implies, we have to run the check on all the booting
@@ -117,6 +117,11 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* field in one of the CPU ID feature registers, we use the sanitised
* value of the register from the CPU feature infrastructure to make
* the decision.
+ * Or
+ * c) SCOPE_BOOT_CPU: Check only on the primary boot CPU to detect the feature.
+ * This category is for features that are "finalised" (or used) by the kernel
+ * very early even before the SMP cpus are brought up.
+ *
* The process of detection is usually denoted by "update" capability state
* in the code.
*
@@ -133,6 +138,11 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* w.r.t the capability. e.g, all secondary CPUs are treated "late CPUs" for
* capabilities determined by the boot CPU.
*
+ * At the moment there are two passes of finalising the capabilities.
+ * a) Boot CPU scope capabilities - Finalised by primary boot CPU via
+ * setup_boot_cpu_capabilities().
+ * b) Everything except (a) - Run via setup_system_capabilities().
+ *
* 3) Verification: When a CPU is brought online (e.g, by user or by the kernel),
* the kernel should make sure that it is safe to use the CPU, by verifying
* that the CPU is compliant with the state of the capabilities finalised
@@ -141,12 +151,19 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* secondary_start_kernel()-> check_local_cpu_capabilities()
*
* As explained in (2) above, capabilities could be finalised at different
- * points in the execution. Each CPU is verified against the "finalised"
- * capabilities and if there is a conflict, the kernel takes an action, based
- * on the severity (e.g, a CPU could be prevented from booting or cause a
- * kernel panic). The CPU is allowed to "affect" the state of the capability,
- * if it has not been finalised already. See section 5 for more details on
- * conflicts.
+ * points in the execution. Each newly booted CPU is verified against those
+ * capabilities that have been finalised by the time it boots.
+ *
+ * a) SCOPE_BOOT_CPU : All CPUs are verified against the capability except
+ * for the primary boot CPU.
+ *
+ * b) SCOPE_LOCAL_CPU, SCOPE_SYSTEM: All CPUs hotplugged on by the user
+ * after the kernel boot are verified against the capability.
+ *
+ * If there is a conflict, the kernel takes an action, based on the severity
+ * (e.g, a CPU could be prevented from booting or cause a kernel panic).
+ * The CPU is allowed to "affect" the state of the capability, if it has not
+ * been finalised already. See section 5 for more details on conflicts.
*
* 4) Action: As mentioned in (2), the kernel can take an action for each detected
* capability, on all CPUs on the system. Appropriate actions include, turning
@@ -193,15 +210,26 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
*/
-/* Decide how the capability is detected. On a local CPU vs System wide */
+/*
+ * Decide how the capability is detected.
+ * On any local CPU vs System wide vs the primary boot CPU
+ */
#define ARM64_CPUCAP_SCOPE_LOCAL_CPU ((u16)BIT(0))
#define ARM64_CPUCAP_SCOPE_SYSTEM ((u16)BIT(1))
+/*
+ * The capabilitiy is detected on the Boot CPU and is used by kernel
+ * during early boot. i.e, the capability should be "detected" and "enabled"
+ * as early as possibly on all booting CPUs.
+ */
+#define ARM64_CPUCAP_SCOPE_BOOT_CPU ((u16)BIT(2))
#define ARM64_CPUCAP_SCOPE_MASK \
(ARM64_CPUCAP_SCOPE_SYSTEM | \
- ARM64_CPUCAP_SCOPE_LOCAL_CPU)
+ ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
+ ARM64_CPUCAP_SCOPE_BOOT_CPU)
#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
+#define SCOPE_BOOT_CPU ARM64_CPUCAP_SCOPE_BOOT_CPU
#define SCOPE_ALL ARM64_CPUCAP_SCOPE_MASK
/*
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index a15be4db6bf4..7625e2962e2b 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -504,7 +504,7 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
-static void update_cpu_capabilities(u16 scope_mask);
+static void __init setup_boot_cpu_capabilities(void);
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
@@ -550,10 +550,10 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
}
/*
- * Run the errata work around and local feature checks on the
- * boot CPU, once we have initialised the cpu feature infrastructure.
+ * Detect and enable early CPU capabilities based on the boot CPU,
+ * after we have initialised the CPU feature infrastructure.
*/
- update_cpu_capabilities(SCOPE_LOCAL_CPU);
+ setup_boot_cpu_capabilities();
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1277,12 +1277,21 @@ __enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_m
if (caps->cpu_enable) {
/*
- * Use stop_machine() as it schedules the work allowing
- * us to modify PSTATE, instead of on_each_cpu() which
- * uses an IPI, giving us a PSTATE that disappears when
- * we return.
+ * If we are dealing with a boot CPU capability, we
+ * have to enable this only on the Boot CPU, where it
+ * is detected. All the secondaries enable it via
+ * check_local_cpu_capabilities().
+ *
+ * Otherwise, use stop_machine() as it schedules the
+ * work allowing us to modify PSTATE, instead of
+ * on_each_cpu() which uses an IPI, giving us a PSTATE
+ * that disappears when we return.
*/
- stop_machine(__enable_cpu_capability, (void *)caps, cpu_online_mask);
+ if (scope_mask & SCOPE_BOOT_CPU)
+ caps->cpu_enable(caps);
+ else
+ stop_machine(__enable_cpu_capability,
+ (void *)caps, cpu_online_mask);
}
}
}
@@ -1362,6 +1371,12 @@ static void check_early_cpu_features(void)
{
verify_cpu_run_el();
verify_cpu_asid_bits();
+ /*
+ * Early features are used by the kernel already. If there
+ * is a conflict, we cannot proceed further.
+ */
+ if (!verify_local_cpu_caps(SCOPE_BOOT_CPU))
+ cpu_panic_kernel();
}
static void
@@ -1403,9 +1418,8 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_capabilities(void)
{
- if (!verify_local_cpu_caps(SCOPE_ALL))
+ if (!verify_local_cpu_caps(SCOPE_ALL & ~SCOPE_BOOT_CPU))
cpu_die_early();
-
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
@@ -1438,6 +1452,14 @@ void check_local_cpu_capabilities(void)
verify_local_cpu_capabilities();
}
+static void __init setup_boot_cpu_capabilities(void)
+{
+ /* Detect capabilities with either SCOPE_BOOT_CPU or SCOPE_LOCAL_CPU */
+ update_cpu_capabilities(SCOPE_BOOT_CPU | SCOPE_LOCAL_CPU);
+ /* Enable the SCOPE_BOOT_CPU capabilities alone right away */
+ enable_cpu_capabilities(SCOPE_BOOT_CPU);
+}
+
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
EXPORT_SYMBOL(arm64_const_caps_ready);
@@ -1462,7 +1484,8 @@ static void __init setup_system_capabilities(void)
* on it. Also enable all the available capabilities.
*/
update_cpu_capabilities(SCOPE_SYSTEM);
- enable_cpu_capabilities(SCOPE_ALL);
+ /* Enable all the available capabilities, which are not already enabled. */
+ enable_cpu_capabilities(SCOPE_ALL & ~SCOPE_BOOT_CPU);
}
void __init setup_cpu_features(void)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 14/22] arm64: capabilities: Add support for features enabled early
2018-02-09 17:54 ` [PATCH v3 14/22] arm64: capabilities: Add support for features enabled early Suzuki K Poulose
@ 2018-02-12 17:17 ` Dave Martin
2018-03-07 17:42 ` Suzuki K Poulose
0 siblings, 1 reply; 62+ messages in thread
From: Dave Martin @ 2018-02-12 17:17 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Feb 09, 2018 at 05:54:57PM +0000, Suzuki K Poulose wrote:
> The kernel detects and uses some of the features based on the boot
> CPU and expects that all the following CPUs conform to it. e.g,
> with VHE and the boot CPU running at EL2, the kernel decides to
> keep the kernel running at EL2. If another CPU is brought up without
> this capability, we use custom hooks (via check_early_cpu_features())
> to handle it. To handle such capabilities add support for detecting
> and enabling capabilities based on the boot CPU.
>
> A bit is added to indicate if the capability should be detected
> early on the boot CPU. The infrastructure then ensures that such
> capabilities are probed and "enabled" early on in the boot CPU
> and, enabled on the subsequent CPUs.
>
> Cc: Julien Thierry <julien.thierry@arm.com>
> Cc: Dave Martin <dave.martin@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> ---
> arch/arm64/include/asm/cpufeature.h | 46 +++++++++++++++++++++++++++++-------
> arch/arm64/kernel/cpufeature.c | 47 +++++++++++++++++++++++++++----------
> 2 files changed, 72 insertions(+), 21 deletions(-)
>
> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> index 383c69c95f23..5f56a8342065 100644
> --- a/arch/arm64/include/asm/cpufeature.h
> +++ b/arch/arm64/include/asm/cpufeature.h
> @@ -104,7 +104,7 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
> * some checks at runtime. This could be, e.g, checking the value of a field
> * in CPU ID feature register or checking the cpu model. The capability
> * provides a call back ( @matches() ) to perform the check.
> - * Scope defines how the checks should be performed. There are two cases:
> + * Scope defines how the checks should be performed. There are three cases:
> *
> * a) SCOPE_LOCAL_CPU: check all the CPUs and "detect" if at least one
> * matches. This implies, we have to run the check on all the booting
> @@ -117,6 +117,11 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
> * field in one of the CPU ID feature registers, we use the sanitised
> * value of the register from the CPU feature infrastructure to make
> * the decision.
> + * Or
> + * c) SCOPE_BOOT_CPU: Check only on the primary boot CPU to detect the feature.
> + * This category is for features that are "finalised" (or used) by the kernel
> + * very early even before the SMP cpus are brought up.
> + *
Nit: the overlong lines bring no benefit here. Please wrap them if
possible -- but to avoid patch churn only bother for lines actually
changed/added by this patch.
> * The process of detection is usually denoted by "update" capability state
> * in the code.
> *
> @@ -133,6 +138,11 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
> * w.r.t the capability. e.g, all secondary CPUs are treated "late CPUs" for
> * capabilities determined by the boot CPU.
> *
> + * At the moment there are two passes of finalising the capabilities.
> + * a) Boot CPU scope capabilities - Finalised by primary boot CPU via
> + * setup_boot_cpu_capabilities().
> + * b) Everything except (a) - Run via setup_system_capabilities().
> + *
> * 3) Verification: When a CPU is brought online (e.g, by user or by the kernel),
> * the kernel should make sure that it is safe to use the CPU, by verifying
> * that the CPU is compliant with the state of the capabilities finalised
> @@ -141,12 +151,19 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
> * secondary_start_kernel()-> check_local_cpu_capabilities()
> *
> * As explained in (2) above, capabilities could be finalised at different
> - * points in the execution. Each CPU is verified against the "finalised"
> - * capabilities and if there is a conflict, the kernel takes an action, based
> - * on the severity (e.g, a CPU could be prevented from booting or cause a
> - * kernel panic). The CPU is allowed to "affect" the state of the capability,
> - * if it has not been finalised already. See section 5 for more details on
> - * conflicts.
> + * points in the execution. Each newly booted CPU is verified against those
> + * capabilities that have been finalised by the time it boots.
> + *
> + * a) SCOPE_BOOT_CPU : All CPUs are verified against the capability except
> + * for the primary boot CPU.
> + *
> + * b) SCOPE_LOCAL_CPU, SCOPE_SYSTEM: All CPUs hotplugged on by the user
> + * after the kernel boot are verified against the capability.
> + *
> + * If there is a conflict, the kernel takes an action, based on the severity
> + * (e.g, a CPU could be prevented from booting or cause a kernel panic).
> + * The CPU is allowed to "affect" the state of the capability, if it has not
> + * been finalised already. See section 5 for more details on conflicts.
> *
> * 4) Action: As mentioned in (2), the kernel can take an action for each detected
> * capability, on all CPUs on the system. Appropriate actions include, turning
> @@ -193,15 +210,26 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
> */
>
>
> -/* Decide how the capability is detected. On a local CPU vs System wide */
> +/*
> + * Decide how the capability is detected.
> + * On any local CPU vs System wide vs the primary boot CPU
> + */
> #define ARM64_CPUCAP_SCOPE_LOCAL_CPU ((u16)BIT(0))
> #define ARM64_CPUCAP_SCOPE_SYSTEM ((u16)BIT(1))
> +/*
> + * The capabilitiy is detected on the Boot CPU and is used by kernel
> + * during early boot. i.e, the capability should be "detected" and "enabled"
> + * as early as possibly on all booting CPUs.
> + */
> +#define ARM64_CPUCAP_SCOPE_BOOT_CPU ((u16)BIT(2))
> #define ARM64_CPUCAP_SCOPE_MASK \
> (ARM64_CPUCAP_SCOPE_SYSTEM | \
> - ARM64_CPUCAP_SCOPE_LOCAL_CPU)
> + ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
> + ARM64_CPUCAP_SCOPE_BOOT_CPU)
>
> #define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
> #define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
> +#define SCOPE_BOOT_CPU ARM64_CPUCAP_SCOPE_BOOT_CPU
> #define SCOPE_ALL ARM64_CPUCAP_SCOPE_MASK
>
> /*
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index a15be4db6bf4..7625e2962e2b 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -504,7 +504,7 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
> }
>
> extern const struct arm64_cpu_capabilities arm64_errata[];
> -static void update_cpu_capabilities(u16 scope_mask);
> +static void __init setup_boot_cpu_capabilities(void);
>
> void __init init_cpu_features(struct cpuinfo_arm64 *info)
> {
> @@ -550,10 +550,10 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
> }
>
> /*
> - * Run the errata work around and local feature checks on the
> - * boot CPU, once we have initialised the cpu feature infrastructure.
> + * Detect and enable early CPU capabilities based on the boot CPU,
> + * after we have initialised the CPU feature infrastructure.
> */
> - update_cpu_capabilities(SCOPE_LOCAL_CPU);
> + setup_boot_cpu_capabilities();
> }
>
> static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
> @@ -1277,12 +1277,21 @@ __enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_m
>
> if (caps->cpu_enable) {
> /*
> - * Use stop_machine() as it schedules the work allowing
> - * us to modify PSTATE, instead of on_each_cpu() which
> - * uses an IPI, giving us a PSTATE that disappears when
> - * we return.
> + * If we are dealing with a boot CPU capability, we
> + * have to enable this only on the Boot CPU, where it
> + * is detected. All the secondaries enable it via
> + * check_local_cpu_capabilities().
I found this confusing to read, because it's not 100% clear whether the
"If we are dealing with a boot CPU capability" applies to the second
sentence as well.
Maybe this would be clearer as:
"Capabilities with SCOPE_BOOT_CPU are finalised before any secondary
CPU boots. Thus, each secondary will enable the capability as
appropriate via check_local_cpu_capabilities(). The only exception is
the boot CPU, for which the capability must be enabled here. This
approach avoids costly stop_machine() calls for this case."
Thoughts?
> + *
> + * Otherwise, use stop_machine() as it schedules the
> + * work allowing us to modify PSTATE, instead of
> + * on_each_cpu() which uses an IPI, giving us a PSTATE
> + * that disappears when we return.
> */
> - stop_machine(__enable_cpu_capability, (void *)caps, cpu_online_mask);
> + if (scope_mask & SCOPE_BOOT_CPU)
> + caps->cpu_enable(caps);
> + else
> + stop_machine(__enable_cpu_capability,
> + (void *)caps, cpu_online_mask);
> }
> }
> }
> @@ -1362,6 +1371,12 @@ static void check_early_cpu_features(void)
> {
> verify_cpu_run_el();
> verify_cpu_asid_bits();
> + /*
> + * Early features are used by the kernel already. If there
> + * is a conflict, we cannot proceed further.
> + */
> + if (!verify_local_cpu_caps(SCOPE_BOOT_CPU))
> + cpu_panic_kernel();
> }
>
> static void
> @@ -1403,9 +1418,8 @@ static void verify_sve_features(void)
> */
> static void verify_local_cpu_capabilities(void)
> {
Nit: Maybe add a comment saying where SCOPE_BOOT_CPU capabilities are
checked.
> - if (!verify_local_cpu_caps(SCOPE_ALL))
> + if (!verify_local_cpu_caps(SCOPE_ALL & ~SCOPE_BOOT_CPU))
> cpu_die_early();
> -
Nit: keep blank line?
Otherwise it looks like the if() falls through, where really
cpu_die_early() does not return.
> verify_local_elf_hwcaps(arm64_elf_hwcaps);
>
> if (system_supports_32bit_el0())
> @@ -1438,6 +1452,14 @@ void check_local_cpu_capabilities(void)
> verify_local_cpu_capabilities();
> }
>
> +static void __init setup_boot_cpu_capabilities(void)
> +{
> + /* Detect capabilities with either SCOPE_BOOT_CPU or SCOPE_LOCAL_CPU */
> + update_cpu_capabilities(SCOPE_BOOT_CPU | SCOPE_LOCAL_CPU);
> + /* Enable the SCOPE_BOOT_CPU capabilities alone right away */
> + enable_cpu_capabilities(SCOPE_BOOT_CPU);
> +}
> +
> DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
> EXPORT_SYMBOL(arm64_const_caps_ready);
>
> @@ -1462,7 +1484,8 @@ static void __init setup_system_capabilities(void)
> * on it. Also enable all the available capabilities.
> */
> update_cpu_capabilities(SCOPE_SYSTEM);
> - enable_cpu_capabilities(SCOPE_ALL);
> + /* Enable all the available capabilities, which are not already enabled. */
> + enable_cpu_capabilities(SCOPE_ALL & ~SCOPE_BOOT_CPU);
> }
>
> void __init setup_cpu_features(void)
> --
> 2.14.3
With fair consideration given to the nits above:
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
^ permalink raw reply [flat|nested] 62+ messages in thread* [PATCH v3 14/22] arm64: capabilities: Add support for features enabled early
2018-02-12 17:17 ` Dave Martin
@ 2018-03-07 17:42 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-07 17:42 UTC (permalink / raw)
To: linux-arm-kernel
On 12/02/18 17:17, Dave Martin wrote:
> On Fri, Feb 09, 2018 at 05:54:57PM +0000, Suzuki K Poulose wrote:
>> The kernel detects and uses some of the features based on the boot
>> CPU and expects that all the following CPUs conform to it. e.g,
>> with VHE and the boot CPU running at EL2, the kernel decides to
>> keep the kernel running at EL2. If another CPU is brought up without
>> this capability, we use custom hooks (via check_early_cpu_features())
>> to handle it. To handle such capabilities add support for detecting
>> and enabling capabilities based on the boot CPU.
>>
>> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
>> index 383c69c95f23..5f56a8342065 100644
>> --- a/arch/arm64/include/asm/cpufeature.h
>> +++ b/arch/arm64/include/asm/cpufeature.h
>> @@ -104,7 +104,7 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
>> * some checks at runtime. This could be, e.g, checking the value of a field
>> * in CPU ID feature register or checking the cpu model. The capability
>> * provides a call back ( @matches() ) to perform the check.
>> - * Scope defines how the checks should be performed. There are two cases:
>> + * Scope defines how the checks should be performed. There are three cases:
>> *
>> * a) SCOPE_LOCAL_CPU: check all the CPUs and "detect" if at least one
>> * matches. This implies, we have to run the check on all the booting
>> @@ -117,6 +117,11 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
>> * field in one of the CPU ID feature registers, we use the sanitised
>> * value of the register from the CPU feature infrastructure to make
>> * the decision.
>> + * Or
>> + * c) SCOPE_BOOT_CPU: Check only on the primary boot CPU to detect the feature.
>> + * This category is for features that are "finalised" (or used) by the kernel
>> + * very early even before the SMP cpus are brought up.
>> + *
>
> Nit: the overlong lines bring no benefit here. Please wrap them if
> possible -- but to avoid patch churn only bother for lines actually
> changed/added by this patch.
Sure
>> static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
>> @@ -1277,12 +1277,21 @@ __enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps, u16 scope_m
>>
>> if (caps->cpu_enable) {
>> /*
>> - * Use stop_machine() as it schedules the work allowing
>> - * us to modify PSTATE, instead of on_each_cpu() which
>> - * uses an IPI, giving us a PSTATE that disappears when
>> - * we return.
>> + * If we are dealing with a boot CPU capability, we
>> + * have to enable this only on the Boot CPU, where it
>> + * is detected. All the secondaries enable it via
>> + * check_local_cpu_capabilities().
>
> I found this confusing to read, because it's not 100% clear whether the
> "If we are dealing with a boot CPU capability" applies to the second
> sentence as well.
>
> Maybe this would be clearer as:
>
> "Capabilities with SCOPE_BOOT_CPU are finalised before any secondary
> CPU boots. Thus, each secondary will enable the capability as
> appropriate via check_local_cpu_capabilities(). The only exception is
> the boot CPU, for which the capability must be enabled here. This
> approach avoids costly stop_machine() calls for this case."
>
> Thoughts?
Definitely better, will change it.
>> @@ -1362,6 +1371,12 @@ static void check_early_cpu_features(void)
>> {
>> verify_cpu_run_el();
>> verify_cpu_asid_bits();
>> + /*
>> + * Early features are used by the kernel already. If there
>> + * is a conflict, we cannot proceed further.
>> + */
>> + if (!verify_local_cpu_caps(SCOPE_BOOT_CPU))
>> + cpu_panic_kernel();
>> }
>>
>> static void
>> @@ -1403,9 +1418,8 @@ static void verify_sve_features(void)
>> */
>> static void verify_local_cpu_capabilities(void)
>> {
>
> Nit: Maybe add a comment saying where SCOPE_BOOT_CPU capabilities are
> checked.
Ok
>
>> - if (!verify_local_cpu_caps(SCOPE_ALL))
>> + if (!verify_local_cpu_caps(SCOPE_ALL & ~SCOPE_BOOT_CPU))
>> cpu_die_early();
>> -
>
> Nit: keep blank line?
>
> Otherwise it looks like the if() falls through, where really
> cpu_die_early() does not return.
>
ok
>> void __init setup_cpu_features(void)
>> --
>> 2.14.3
>
> With fair consideration given to the nits above:
>
> Reviewed-by: Dave Martin <Dave.Martin@arm.com>
Cheers
Suzuki
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v3 14/21] arm64: capabilities: Change scope of VHE to Boot CPU feature
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (26 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 14/22] arm64: capabilities: Add support for features enabled early Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-09 17:54 ` [PATCH v3 15/22] " Suzuki K Poulose
` (15 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
We expect all CPUs to be running at the same EL inside the kernel
with or without VHE enabled and we have strict checks to ensure
that any mismatch triggers a kernel panic. If VHE is enabled,
we use the feature based on the boot CPU and all other CPUs
should follow. This makes it a perfect candidate for a cpability
based on the boot CPU, which should be matched by all the CPUs
(both when is ON and OFF). This saves us some not-so-pretty
hooks and special code, just for verifying the conflict.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Dave Martin <dave.martin@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 7 +++++++
arch/arm64/include/asm/virt.h | 6 ------
arch/arm64/kernel/cpufeature.c | 5 +++--
arch/arm64/kernel/smp.c | 38 -------------------------------------
4 files changed, 10 insertions(+), 46 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 5f56a8342065..dfce93f79ae7 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -276,6 +276,13 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
(ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
+/*
+ * Critical CPU feature used early in the boot based on the boot CPU.
+ * The feature should be matched by all booting CPU (both miss and hit
+ * cases).
+ */
+#define ARM64_CPUCAP_CRITICAL_BOOT_CPU_FEATURE ARM64_CPUCAP_SCOPE_BOOT_CPU
+
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
index c5f89442785c..9d1e24e030b3 100644
--- a/arch/arm64/include/asm/virt.h
+++ b/arch/arm64/include/asm/virt.h
@@ -102,12 +102,6 @@ static inline bool has_vhe(void)
return false;
}
-#ifdef CONFIG_ARM64_VHE
-extern void verify_cpu_run_el(void);
-#else
-static inline void verify_cpu_run_el(void) {}
-#endif
-
#endif /* __ASSEMBLY__ */
#endif /* ! __ASM__VIRT_H */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 7625e2962e2b..f66e66c79916 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1016,11 +1016,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
},
#endif /* CONFIG_ARM64_PAN */
{
+#ifdef CONFIG_ARM64_VHE
.desc = "Virtualization Host Extensions",
.capability = ARM64_HAS_VIRT_HOST_EXTN,
- .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .type = ARM64_CPUCAP_CRITICAL_BOOT_CPU_FEATURE,
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
+#endif
},
{
.desc = "32-bit EL0 Support",
@@ -1369,7 +1371,6 @@ static bool verify_local_cpu_caps(u16 scope_mask)
*/
static void check_early_cpu_features(void)
{
- verify_cpu_run_el();
verify_cpu_asid_bits();
/*
* Early features are used by the kernel already. If there
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 5cef11450183..f3e2e3aec0b0 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -85,43 +85,6 @@ enum ipi_msg_type {
IPI_WAKEUP
};
-#ifdef CONFIG_ARM64_VHE
-
-/* Whether the boot CPU is running in HYP mode or not*/
-static bool boot_cpu_hyp_mode;
-
-static inline void save_boot_cpu_run_el(void)
-{
- boot_cpu_hyp_mode = is_kernel_in_hyp_mode();
-}
-
-static inline bool is_boot_cpu_in_hyp_mode(void)
-{
- return boot_cpu_hyp_mode;
-}
-
-/*
- * Verify that a secondary CPU is running the kernel at the same
- * EL as that of the boot CPU.
- */
-void verify_cpu_run_el(void)
-{
- bool in_el2 = is_kernel_in_hyp_mode();
- bool boot_cpu_el2 = is_boot_cpu_in_hyp_mode();
-
- if (in_el2 ^ boot_cpu_el2) {
- pr_crit("CPU%d: mismatched Exception Level(EL%d) with boot CPU(EL%d)\n",
- smp_processor_id(),
- in_el2 ? 2 : 1,
- boot_cpu_el2 ? 2 : 1);
- cpu_panic_kernel();
- }
-}
-
-#else
-static inline void save_boot_cpu_run_el(void) {}
-#endif
-
#ifdef CONFIG_HOTPLUG_CPU
static int op_cpu_kill(unsigned int cpu);
#else
@@ -447,7 +410,6 @@ void __init smp_prepare_boot_cpu(void)
*/
jump_label_init();
cpuinfo_store_boot_cpu();
- save_boot_cpu_run_el();
}
static u64 __init of_get_cpu_mpidr(struct device_node *dn)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 15/22] arm64: capabilities: Change scope of VHE to Boot CPU feature
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (27 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 14/21] arm64: capabilities: Change scope of VHE to Boot CPU feature Suzuki K Poulose
@ 2018-02-09 17:54 ` Suzuki K Poulose
2018-02-12 17:17 ` Dave Martin
2018-02-09 17:55 ` [PATCH v3 15/21] arm64: capabilities: Clean up midr range helpers Suzuki K Poulose
` (14 subsequent siblings)
43 siblings, 1 reply; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:54 UTC (permalink / raw)
To: linux-arm-kernel
We expect all CPUs to be running at the same EL inside the kernel
with or without VHE enabled and we have strict checks to ensure
that any mismatch triggers a kernel panic. If VHE is enabled,
we use the feature based on the boot CPU and all other CPUs
should follow. This makes it a perfect candidate for a cpability
based on the boot CPU, which should be matched by all the CPUs
(both when is ON and OFF). This saves us some not-so-pretty
hooks and special code, just for verifying the conflict.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Dave Martin <dave.martin@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 7 +++++++
arch/arm64/include/asm/virt.h | 6 ------
arch/arm64/kernel/cpufeature.c | 5 +++--
arch/arm64/kernel/smp.c | 38 -------------------------------------
4 files changed, 10 insertions(+), 46 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 5f56a8342065..dfce93f79ae7 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -276,6 +276,13 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
(ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
+/*
+ * Critical CPU feature used early in the boot based on the boot CPU.
+ * The feature should be matched by all booting CPU (both miss and hit
+ * cases).
+ */
+#define ARM64_CPUCAP_CRITICAL_BOOT_CPU_FEATURE ARM64_CPUCAP_SCOPE_BOOT_CPU
+
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
index c5f89442785c..9d1e24e030b3 100644
--- a/arch/arm64/include/asm/virt.h
+++ b/arch/arm64/include/asm/virt.h
@@ -102,12 +102,6 @@ static inline bool has_vhe(void)
return false;
}
-#ifdef CONFIG_ARM64_VHE
-extern void verify_cpu_run_el(void);
-#else
-static inline void verify_cpu_run_el(void) {}
-#endif
-
#endif /* __ASSEMBLY__ */
#endif /* ! __ASM__VIRT_H */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 7625e2962e2b..f66e66c79916 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1016,11 +1016,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
},
#endif /* CONFIG_ARM64_PAN */
{
+#ifdef CONFIG_ARM64_VHE
.desc = "Virtualization Host Extensions",
.capability = ARM64_HAS_VIRT_HOST_EXTN,
- .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .type = ARM64_CPUCAP_CRITICAL_BOOT_CPU_FEATURE,
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
+#endif
},
{
.desc = "32-bit EL0 Support",
@@ -1369,7 +1371,6 @@ static bool verify_local_cpu_caps(u16 scope_mask)
*/
static void check_early_cpu_features(void)
{
- verify_cpu_run_el();
verify_cpu_asid_bits();
/*
* Early features are used by the kernel already. If there
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 5cef11450183..f3e2e3aec0b0 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -85,43 +85,6 @@ enum ipi_msg_type {
IPI_WAKEUP
};
-#ifdef CONFIG_ARM64_VHE
-
-/* Whether the boot CPU is running in HYP mode or not*/
-static bool boot_cpu_hyp_mode;
-
-static inline void save_boot_cpu_run_el(void)
-{
- boot_cpu_hyp_mode = is_kernel_in_hyp_mode();
-}
-
-static inline bool is_boot_cpu_in_hyp_mode(void)
-{
- return boot_cpu_hyp_mode;
-}
-
-/*
- * Verify that a secondary CPU is running the kernel at the same
- * EL as that of the boot CPU.
- */
-void verify_cpu_run_el(void)
-{
- bool in_el2 = is_kernel_in_hyp_mode();
- bool boot_cpu_el2 = is_boot_cpu_in_hyp_mode();
-
- if (in_el2 ^ boot_cpu_el2) {
- pr_crit("CPU%d: mismatched Exception Level(EL%d) with boot CPU(EL%d)\n",
- smp_processor_id(),
- in_el2 ? 2 : 1,
- boot_cpu_el2 ? 2 : 1);
- cpu_panic_kernel();
- }
-}
-
-#else
-static inline void save_boot_cpu_run_el(void) {}
-#endif
-
#ifdef CONFIG_HOTPLUG_CPU
static int op_cpu_kill(unsigned int cpu);
#else
@@ -447,7 +410,6 @@ void __init smp_prepare_boot_cpu(void)
*/
jump_label_init();
cpuinfo_store_boot_cpu();
- save_boot_cpu_run_el();
}
static u64 __init of_get_cpu_mpidr(struct device_node *dn)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 15/22] arm64: capabilities: Change scope of VHE to Boot CPU feature
2018-02-09 17:54 ` [PATCH v3 15/22] " Suzuki K Poulose
@ 2018-02-12 17:17 ` Dave Martin
2018-03-08 12:10 ` Suzuki K Poulose
0 siblings, 1 reply; 62+ messages in thread
From: Dave Martin @ 2018-02-12 17:17 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Feb 09, 2018 at 05:54:59PM +0000, Suzuki K Poulose wrote:
> We expect all CPUs to be running at the same EL inside the kernel
> with or without VHE enabled and we have strict checks to ensure
> that any mismatch triggers a kernel panic. If VHE is enabled,
> we use the feature based on the boot CPU and all other CPUs
> should follow. This makes it a perfect candidate for a cpability
capability
> based on the boot CPU, which should be matched by all the CPUs
> (both when is ON and OFF). This saves us some not-so-pretty
> hooks and special code, just for verifying the conflict.
>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Dave Martin <dave.martin@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> ---
> arch/arm64/include/asm/cpufeature.h | 7 +++++++
> arch/arm64/include/asm/virt.h | 6 ------
> arch/arm64/kernel/cpufeature.c | 5 +++--
> arch/arm64/kernel/smp.c | 38 -------------------------------------
> 4 files changed, 10 insertions(+), 46 deletions(-)
>
> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> index 5f56a8342065..dfce93f79ae7 100644
> --- a/arch/arm64/include/asm/cpufeature.h
> +++ b/arch/arm64/include/asm/cpufeature.h
> @@ -276,6 +276,13 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
> (ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
> ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
>
> +/*
> + * Critical CPU feature used early in the boot based on the boot CPU.
> + * The feature should be matched by all booting CPU (both miss and hit
> + * cases).
> + */
> +#define ARM64_CPUCAP_CRITICAL_BOOT_CPU_FEATURE ARM64_CPUCAP_SCOPE_BOOT_CPU
> +
Nit: would it be consistent with the uses we already have for the word
"strict" to use that word here? i.e.,
ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE.
Or do you think that would be more confusing?
Otherwise, "critical" sounds a bit like we depend on the capability
being available.
If "strict" doesn't fit though and no other option suggests itself,
it's probably not worth changing this.
> struct arm64_cpu_capabilities {
> const char *desc;
> u16 capability;
> diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
> index c5f89442785c..9d1e24e030b3 100644
> --- a/arch/arm64/include/asm/virt.h
> +++ b/arch/arm64/include/asm/virt.h
> @@ -102,12 +102,6 @@ static inline bool has_vhe(void)
> return false;
> }
>
> -#ifdef CONFIG_ARM64_VHE
> -extern void verify_cpu_run_el(void);
> -#else
> -static inline void verify_cpu_run_el(void) {}
> -#endif
> -
> #endif /* __ASSEMBLY__ */
>
> #endif /* ! __ASM__VIRT_H */
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 7625e2962e2b..f66e66c79916 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -1016,11 +1016,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
> },
> #endif /* CONFIG_ARM64_PAN */
> {
> +#ifdef CONFIG_ARM64_VHE
> .desc = "Virtualization Host Extensions",
> .capability = ARM64_HAS_VIRT_HOST_EXTN,
> - .type = ARM64_CPUCAP_SYSTEM_FEATURE,
> + .type = ARM64_CPUCAP_CRITICAL_BOOT_CPU_FEATURE,
> .matches = runs_at_el2,
> .cpu_enable = cpu_copy_el2regs,
> +#endif
Shouldn't the #ifdef...#endif be outside the { ... },?
Otherwise this yields an empty block that will truncate the list in the
CONFIG_ARM64_VHE case...
Removal of this block for !CONFIG_ARM64_VHE is a change rather than just
refactoring, so the commit message should explain it.
[...]
Cheers
---Dave
^ permalink raw reply [flat|nested] 62+ messages in thread* [PATCH v3 15/22] arm64: capabilities: Change scope of VHE to Boot CPU feature
2018-02-12 17:17 ` Dave Martin
@ 2018-03-08 12:10 ` Suzuki K Poulose
2018-03-08 13:43 ` Dave Martin
0 siblings, 1 reply; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-08 12:10 UTC (permalink / raw)
To: linux-arm-kernel
On 12/02/18 17:17, Dave Martin wrote:
> On Fri, Feb 09, 2018 at 05:54:59PM +0000, Suzuki K Poulose wrote:
>> We expect all CPUs to be running at the same EL inside the kernel
>> with or without VHE enabled and we have strict checks to ensure
>> that any mismatch triggers a kernel panic. If VHE is enabled,
>> we use the feature based on the boot CPU and all other CPUs
>> should follow. This makes it a perfect candidate for a cpability
>
> capability
>
>> based on the boot CPU, which should be matched by all the CPUs
>> (both when is ON and OFF). This saves us some not-so-pretty
>> hooks and special code, just for verifying the conflict.
>>
>> Cc: Marc Zyngier <marc.zyngier@arm.com>
>> Cc: Dave Martin <dave.martin@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>> ---
>> arch/arm64/include/asm/cpufeature.h | 7 +++++++
>> arch/arm64/include/asm/virt.h | 6 ------
>> arch/arm64/kernel/cpufeature.c | 5 +++--
>> arch/arm64/kernel/smp.c | 38 -------------------------------------
>> 4 files changed, 10 insertions(+), 46 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
>> index 5f56a8342065..dfce93f79ae7 100644
>> --- a/arch/arm64/include/asm/cpufeature.h
>> +++ b/arch/arm64/include/asm/cpufeature.h
>> @@ -276,6 +276,13 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
>> (ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
>> ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
>>
>> +/*
>> + * Critical CPU feature used early in the boot based on the boot CPU.
>> + * The feature should be matched by all booting CPU (both miss and hit
>> + * cases).
>> + */
>> +#define ARM64_CPUCAP_CRITICAL_BOOT_CPU_FEATURE ARM64_CPUCAP_SCOPE_BOOT_CPU
>> +
>
> Nit: would it be consistent with the uses we already have for the word
> "strict" to use that word here? i.e.,
> ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE.
> Or do you think that would be more confusing?
We don't use the "STRICT" tag anymore. Moreover, I used CRITICAL to indicate
that it is special in a way that all the "late" CPUs (in this case all
secondaries) should match the "state" of the capability (i.e, both ON and OFF)
as that of the boot CPU. I am OK to change it to STRICT.
>
> Otherwise, "critical" sounds a bit like we depend on the capability
> being available.
>
> If "strict" doesn't fit though and no other option suggests itself,
> it's probably not worth changing this.
>
>> struct arm64_cpu_capabilities {
>> const char *desc;
>> u16 capability;
>> diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
>> index c5f89442785c..9d1e24e030b3 100644
>> --- a/arch/arm64/include/asm/virt.h
>> +++ b/arch/arm64/include/asm/virt.h
>> @@ -102,12 +102,6 @@ static inline bool has_vhe(void)
>> return false;
>> }
>>
>> -#ifdef CONFIG_ARM64_VHE
>> -extern void verify_cpu_run_el(void);
>> -#else
>> -static inline void verify_cpu_run_el(void) {}
>> -#endif
>> -
>> #endif /* __ASSEMBLY__ */
>>
>> #endif /* ! __ASM__VIRT_H */
>> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
>> index 7625e2962e2b..f66e66c79916 100644
>> --- a/arch/arm64/kernel/cpufeature.c
>> +++ b/arch/arm64/kernel/cpufeature.c
>> @@ -1016,11 +1016,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
>> },
>> #endif /* CONFIG_ARM64_PAN */
>> {
>> +#ifdef CONFIG_ARM64_VHE
>> .desc = "Virtualization Host Extensions",
>> .capability = ARM64_HAS_VIRT_HOST_EXTN,
>> - .type = ARM64_CPUCAP_SYSTEM_FEATURE,
>> + .type = ARM64_CPUCAP_CRITICAL_BOOT_CPU_FEATURE,
>> .matches = runs_at_el2,
>> .cpu_enable = cpu_copy_el2regs,
>> +#endif
>
> Shouldn't the #ifdef...#endif be outside the { ... },?
>
> Otherwise this yields an empty block that will truncate the list in the
> CONFIG_ARM64_VHE case...
Good catch. You're right, I will fix it.
>
>
> Removal of this block for !CONFIG_ARM64_VHE is a change rather than just
> refactoring, so the commit message should explain it.
Ok.
Cheers
Suzuki
^ permalink raw reply [flat|nested] 62+ messages in thread* [PATCH v3 15/22] arm64: capabilities: Change scope of VHE to Boot CPU feature
2018-03-08 12:10 ` Suzuki K Poulose
@ 2018-03-08 13:43 ` Dave Martin
0 siblings, 0 replies; 62+ messages in thread
From: Dave Martin @ 2018-03-08 13:43 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Mar 08, 2018 at 12:10:22PM +0000, Suzuki K Poulose wrote:
> On 12/02/18 17:17, Dave Martin wrote:
> >On Fri, Feb 09, 2018 at 05:54:59PM +0000, Suzuki K Poulose wrote:
> >>We expect all CPUs to be running at the same EL inside the kernel
> >>with or without VHE enabled and we have strict checks to ensure
> >>that any mismatch triggers a kernel panic. If VHE is enabled,
> >>we use the feature based on the boot CPU and all other CPUs
> >>should follow. This makes it a perfect candidate for a cpability
> >
> >capability
> >
> >>based on the boot CPU, which should be matched by all the CPUs
> >>(both when is ON and OFF). This saves us some not-so-pretty
> >>hooks and special code, just for verifying the conflict.
> >>
> >>Cc: Marc Zyngier <marc.zyngier@arm.com>
> >>Cc: Dave Martin <dave.martin@arm.com>
> >>Cc: Will Deacon <will.deacon@arm.com>
> >>Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> >>---
> >> arch/arm64/include/asm/cpufeature.h | 7 +++++++
> >> arch/arm64/include/asm/virt.h | 6 ------
> >> arch/arm64/kernel/cpufeature.c | 5 +++--
> >> arch/arm64/kernel/smp.c | 38 -------------------------------------
> >> 4 files changed, 10 insertions(+), 46 deletions(-)
> >>
> >>diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> >>index 5f56a8342065..dfce93f79ae7 100644
> >>--- a/arch/arm64/include/asm/cpufeature.h
> >>+++ b/arch/arm64/include/asm/cpufeature.h
> >>@@ -276,6 +276,13 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
> >> (ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
> >> ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
> >>+/*
> >>+ * Critical CPU feature used early in the boot based on the boot CPU.
> >>+ * The feature should be matched by all booting CPU (both miss and hit
> >>+ * cases).
> >>+ */
> >>+#define ARM64_CPUCAP_CRITICAL_BOOT_CPU_FEATURE ARM64_CPUCAP_SCOPE_BOOT_CPU
> >>+
> >
> >Nit: would it be consistent with the uses we already have for the word
> >"strict" to use that word here? i.e.,
> >ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE.
> >Or do you think that would be more confusing?
>
> We don't use the "STRICT" tag anymore. Moreover, I used CRITICAL to indicate
> that it is special in a way that all the "late" CPUs (in this case all
> secondaries) should match the "state" of the capability (i.e, both ON and OFF)
> as that of the boot CPU. I am OK to change it to STRICT.
OK, I think so long as the definition is clear the precise name doesn't
matter too much.
I suggest deleting "critical" from the comment though, since that
suggests a circular definition. The meaning seems clear(er) without
it.
[...]
Cheers
---Dave
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v3 15/21] arm64: capabilities: Clean up midr range helpers
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (28 preceding siblings ...)
2018-02-09 17:54 ` [PATCH v3 15/22] " Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-09 17:55 ` [PATCH v3 16/21] arm64: Add helpers for checking CPU MIDR against a range Suzuki K Poulose
` (13 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
We are about to introduce generic MIDR range helpers. Clean
up the existing helpers in erratum handling, preparing them
to use generic version.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpu_errata.c | 100 +++++++++++++++++++++++------------------
1 file changed, 56 insertions(+), 44 deletions(-)
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 8d2d3cf819a7..22ea61141596 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -225,20 +225,35 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
}
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
-#define MIDR_RANGE(model, min, max) \
- .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
- .matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = min, \
- .midr_range_max = max
+#define CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
+ .matches = is_affected_midr_range, \
+ .midr_model = model, \
+ .midr_range_min = MIDR_CPU_VAR_REV(v_min, r_min), \
+ .midr_range_max = MIDR_CPU_VAR_REV(v_max, r_max)
-#define MIDR_ALL_VERSIONS(model) \
- .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
- .matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = 0, \
+#define CAP_MIDR_ALL_VERSIONS(model) \
+ .matches = is_affected_midr_range, \
+ .midr_model = model, \
+ .midr_range_min = MIDR_CPU_VAR_REV(0, 0), \
.midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
+#define ERRATA_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
+ CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max)
+
+/* Errata affecting a range of revisions of given model variant */
+#define ERRATA_MIDR_REV_RANGE(m, var, r_min, r_max) \
+ ERRATA_MIDR_RANGE(m, var, r_min, var, r_max)
+
+/* Errata affecting a single variant/revision of a model */
+#define ERRATA_MIDR_REV(model, var, rev) \
+ ERRATA_MIDR_RANGE(model, var, rev, var, rev)
+
+/* Errata affecting all variants/revisions of a given a model */
+#define ERRATA_MIDR_ALL_VERSIONS(model) \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
+ CAP_MIDR_ALL_VERSIONS(model)
+
const struct arm64_cpu_capabilities arm64_errata[] = {
#if defined(CONFIG_ARM64_ERRATUM_826319) || \
defined(CONFIG_ARM64_ERRATUM_827319) || \
@@ -247,7 +262,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[012] */
.desc = "ARM errata 826319, 827319, 824069",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x02),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 2),
.cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
@@ -256,7 +271,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[01] */
.desc = "ARM errata 819472",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x01),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 1),
.cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
@@ -265,9 +280,9 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A57 r0p0 - r1p2 */
.desc = "ARM erratum 832075",
.capability = ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE,
- MIDR_RANGE(MIDR_CORTEX_A57,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(1, 2)),
+ ERRATA_MIDR_RANGE(MIDR_CORTEX_A57,
+ 0, 0,
+ 1, 2),
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_834220
@@ -275,9 +290,9 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A57 r0p0 - r1p2 */
.desc = "ARM erratum 834220",
.capability = ARM64_WORKAROUND_834220,
- MIDR_RANGE(MIDR_CORTEX_A57,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(1, 2)),
+ ERRATA_MIDR_RANGE(MIDR_CORTEX_A57,
+ 0, 0,
+ 1, 2),
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_845719
@@ -285,7 +300,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[01234] */
.desc = "ARM erratum 845719",
.capability = ARM64_WORKAROUND_845719,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x04),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 4),
},
#endif
#ifdef CONFIG_CAVIUM_ERRATUM_23154
@@ -293,7 +308,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cavium ThunderX, pass 1.x */
.desc = "Cavium erratum 23154",
.capability = ARM64_WORKAROUND_CAVIUM_23154,
- MIDR_RANGE(MIDR_THUNDERX, 0x00, 0x01),
+ ERRATA_MIDR_REV_RANGE(MIDR_THUNDERX, 0, 0, 1),
},
#endif
#ifdef CONFIG_CAVIUM_ERRATUM_27456
@@ -301,15 +316,15 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cavium ThunderX, T88 pass 1.x - 2.1 */
.desc = "Cavium erratum 27456",
.capability = ARM64_WORKAROUND_CAVIUM_27456,
- MIDR_RANGE(MIDR_THUNDERX,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(1, 1)),
+ ERRATA_MIDR_RANGE(MIDR_THUNDERX,
+ 0, 0,
+ 1, 1),
},
{
/* Cavium ThunderX, T81 pass 1.0 */
.desc = "Cavium erratum 27456",
.capability = ARM64_WORKAROUND_CAVIUM_27456,
- MIDR_RANGE(MIDR_THUNDERX_81XX, 0x00, 0x00),
+ ERRATA_MIDR_REV(MIDR_THUNDERX_81XX, 0, 0),
},
#endif
#ifdef CONFIG_CAVIUM_ERRATUM_30115
@@ -317,20 +332,21 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cavium ThunderX, T88 pass 1.x - 2.2 */
.desc = "Cavium erratum 30115",
.capability = ARM64_WORKAROUND_CAVIUM_30115,
- MIDR_RANGE(MIDR_THUNDERX, 0x00,
- (1 << MIDR_VARIANT_SHIFT) | 2),
+ ERRATA_MIDR_RANGE(MIDR_THUNDERX,
+ 0, 0,
+ 1, 2),
},
{
/* Cavium ThunderX, T81 pass 1.0 - 1.2 */
.desc = "Cavium erratum 30115",
.capability = ARM64_WORKAROUND_CAVIUM_30115,
- MIDR_RANGE(MIDR_THUNDERX_81XX, 0x00, 0x02),
+ ERRATA_MIDR_REV_RANGE(MIDR_THUNDERX_81XX, 0, 0, 2),
},
{
/* Cavium ThunderX, T83 pass 1.0 */
.desc = "Cavium erratum 30115",
.capability = ARM64_WORKAROUND_CAVIUM_30115,
- MIDR_RANGE(MIDR_THUNDERX_83XX, 0x00, 0x00),
+ ERRATA_MIDR_REV(MIDR_THUNDERX_83XX, 0, 0),
},
#endif
{
@@ -344,9 +360,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Falkor erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
- MIDR_RANGE(MIDR_QCOM_FALKOR_V1,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(0, 0)),
+ ERRATA_MIDR_REV(MIDR_QCOM_FALKOR_V1, 0, 0),
},
{
.desc = "Qualcomm Technologies Kryo erratum 1003",
@@ -360,9 +374,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Falkor erratum 1009",
.capability = ARM64_WORKAROUND_REPEAT_TLBI,
- MIDR_RANGE(MIDR_QCOM_FALKOR_V1,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(0, 0)),
+ ERRATA_MIDR_REV(MIDR_QCOM_FALKOR_V1, 0, 0),
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_858921
@@ -370,47 +382,47 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A73 all versions */
.desc = "ARM erratum 858921",
.capability = ARM64_WORKAROUND_858921,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
},
#endif
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
.cpu_enable = qcom_enable_link_stack_sanitization,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
- MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
.cpu_enable = enable_smccc_arch_workaround_1,
},
#endif
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 16/21] arm64: Add helpers for checking CPU MIDR against a range
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (29 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 15/21] arm64: capabilities: Clean up midr range helpers Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-09 17:55 ` [PATCH v3 16/22] arm64: capabilities: Clean up midr range helpers Suzuki K Poulose
` (12 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
Add helpers for checking if the given CPU midr falls in a range
of variants/revisions for a given model.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 7 ++-----
arch/arm64/include/asm/cputype.h | 30 ++++++++++++++++++++++++++++++
arch/arm64/kernel/cpu_errata.c | 16 +++++-----------
3 files changed, 37 insertions(+), 16 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index dfce93f79ae7..e47f39f62440 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -10,6 +10,7 @@
#define __ASM_CPUFEATURE_H
#include <asm/cpucaps.h>
+#include <asm/cputype.h>
#include <asm/fpsimd.h>
#include <asm/hwcap.h>
#include <asm/sigcontext.h>
@@ -295,11 +296,7 @@ struct arm64_cpu_capabilities {
*/
void (*cpu_enable)(const struct arm64_cpu_capabilities *cap);
union {
- struct { /* To be used for erratum handling only */
- u32 midr_model;
- u32 midr_range_min, midr_range_max;
- };
-
+ struct midr_range midr_range; /* To be used for erratum handling only */
struct { /* Feature register checking */
u32 sys_reg;
u8 field_pos;
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index 2f8d39ed9c2e..6c45acb6ef38 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -115,6 +115,36 @@
#define read_cpuid(reg) read_sysreg_s(SYS_ ## reg)
+/*
+ * Represent a range of MIDR values for a given CPU model and a
+ * range of variant/revision values.
+ *
+ * @model - CPU model as defined by MIDR_CPU_MODEL
+ * @rv_min - Minimum value for the revision/variant as defined by
+ * MIDR_CPU_VAR_REV
+ * @rv_max - Maximum value for the variant/revision for the range.
+ */
+struct midr_range {
+ u32 model;
+ u32 rv_min;
+ u32 rv_max;
+};
+
+#define MIDR_RANGE(m, v_min, r_min, v_max, r_max) \
+ { \
+ .model = m, \
+ .rv_min = MIDR_CPU_VAR_REV(v_min, r_min), \
+ .rv_max = MIDR_CPU_VAR_REV(v_max, r_max), \
+ }
+
+#define MIDR_ALL_VERSIONS(m) MIDR_RANGE(m, 0, 0, 0xf, 0xf)
+
+static inline bool is_midr_in_range(u32 midr, struct midr_range const *range)
+{
+ return MIDR_IS_CPU_MODEL_RANGE(midr, range->model,
+ range->rv_min, range->rv_max);
+}
+
/*
* The CPU ID never changes at run time, so we might as well tell the
* compiler that it's constant. Use this function to read the CPU ID
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 22ea61141596..5f903e37a267 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -25,9 +25,7 @@ static bool __maybe_unused
is_affected_midr_range(const struct arm64_cpu_capabilities *entry, int scope)
{
WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
- return MIDR_IS_CPU_MODEL_RANGE(read_cpuid_id(), entry->midr_model,
- entry->midr_range_min,
- entry->midr_range_max);
+ return is_midr_in_range(read_cpuid_id(), &entry->midr_range);
}
static bool __maybe_unused
@@ -41,7 +39,7 @@ is_kryo_midr(const struct arm64_cpu_capabilities *entry, int scope)
model &= MIDR_IMPLEMENTOR_MASK | (0xf00 << MIDR_PARTNUM_SHIFT) |
MIDR_ARCHITECTURE_MASK;
- return model == entry->midr_model;
+ return model == entry->midr_range.model;
}
static bool
@@ -227,15 +225,11 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
#define CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
.matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = MIDR_CPU_VAR_REV(v_min, r_min), \
- .midr_range_max = MIDR_CPU_VAR_REV(v_max, r_max)
+ .midr_range = MIDR_RANGE(model, v_min, r_min, v_max, r_max)
#define CAP_MIDR_ALL_VERSIONS(model) \
.matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = MIDR_CPU_VAR_REV(0, 0), \
- .midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
+ .midr_range = MIDR_ALL_VERSIONS(model)
#define ERRATA_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
@@ -366,7 +360,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "Qualcomm Technologies Kryo erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
- .midr_model = MIDR_QCOM_KRYO,
+ .midr_range.model = MIDR_QCOM_KRYO,
.matches = is_kryo_midr,
},
#endif
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 16/22] arm64: capabilities: Clean up midr range helpers
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (30 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 16/21] arm64: Add helpers for checking CPU MIDR against a range Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-09 17:55 ` [PATCH v3 17/22] arm64: Add helpers for checking CPU MIDR against a range Suzuki K Poulose
` (11 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
We are about to introduce generic MIDR range helpers. Clean
up the existing helpers in erratum handling, preparing them
to use generic version.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpu_errata.c | 100 +++++++++++++++++++++++------------------
1 file changed, 56 insertions(+), 44 deletions(-)
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 8d2d3cf819a7..22ea61141596 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -225,20 +225,35 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
}
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
-#define MIDR_RANGE(model, min, max) \
- .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
- .matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = min, \
- .midr_range_max = max
+#define CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
+ .matches = is_affected_midr_range, \
+ .midr_model = model, \
+ .midr_range_min = MIDR_CPU_VAR_REV(v_min, r_min), \
+ .midr_range_max = MIDR_CPU_VAR_REV(v_max, r_max)
-#define MIDR_ALL_VERSIONS(model) \
- .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
- .matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = 0, \
+#define CAP_MIDR_ALL_VERSIONS(model) \
+ .matches = is_affected_midr_range, \
+ .midr_model = model, \
+ .midr_range_min = MIDR_CPU_VAR_REV(0, 0), \
.midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
+#define ERRATA_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
+ CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max)
+
+/* Errata affecting a range of revisions of given model variant */
+#define ERRATA_MIDR_REV_RANGE(m, var, r_min, r_max) \
+ ERRATA_MIDR_RANGE(m, var, r_min, var, r_max)
+
+/* Errata affecting a single variant/revision of a model */
+#define ERRATA_MIDR_REV(model, var, rev) \
+ ERRATA_MIDR_RANGE(model, var, rev, var, rev)
+
+/* Errata affecting all variants/revisions of a given a model */
+#define ERRATA_MIDR_ALL_VERSIONS(model) \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
+ CAP_MIDR_ALL_VERSIONS(model)
+
const struct arm64_cpu_capabilities arm64_errata[] = {
#if defined(CONFIG_ARM64_ERRATUM_826319) || \
defined(CONFIG_ARM64_ERRATUM_827319) || \
@@ -247,7 +262,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[012] */
.desc = "ARM errata 826319, 827319, 824069",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x02),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 2),
.cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
@@ -256,7 +271,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[01] */
.desc = "ARM errata 819472",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x01),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 1),
.cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
@@ -265,9 +280,9 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A57 r0p0 - r1p2 */
.desc = "ARM erratum 832075",
.capability = ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE,
- MIDR_RANGE(MIDR_CORTEX_A57,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(1, 2)),
+ ERRATA_MIDR_RANGE(MIDR_CORTEX_A57,
+ 0, 0,
+ 1, 2),
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_834220
@@ -275,9 +290,9 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A57 r0p0 - r1p2 */
.desc = "ARM erratum 834220",
.capability = ARM64_WORKAROUND_834220,
- MIDR_RANGE(MIDR_CORTEX_A57,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(1, 2)),
+ ERRATA_MIDR_RANGE(MIDR_CORTEX_A57,
+ 0, 0,
+ 1, 2),
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_845719
@@ -285,7 +300,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[01234] */
.desc = "ARM erratum 845719",
.capability = ARM64_WORKAROUND_845719,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x04),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 4),
},
#endif
#ifdef CONFIG_CAVIUM_ERRATUM_23154
@@ -293,7 +308,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cavium ThunderX, pass 1.x */
.desc = "Cavium erratum 23154",
.capability = ARM64_WORKAROUND_CAVIUM_23154,
- MIDR_RANGE(MIDR_THUNDERX, 0x00, 0x01),
+ ERRATA_MIDR_REV_RANGE(MIDR_THUNDERX, 0, 0, 1),
},
#endif
#ifdef CONFIG_CAVIUM_ERRATUM_27456
@@ -301,15 +316,15 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cavium ThunderX, T88 pass 1.x - 2.1 */
.desc = "Cavium erratum 27456",
.capability = ARM64_WORKAROUND_CAVIUM_27456,
- MIDR_RANGE(MIDR_THUNDERX,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(1, 1)),
+ ERRATA_MIDR_RANGE(MIDR_THUNDERX,
+ 0, 0,
+ 1, 1),
},
{
/* Cavium ThunderX, T81 pass 1.0 */
.desc = "Cavium erratum 27456",
.capability = ARM64_WORKAROUND_CAVIUM_27456,
- MIDR_RANGE(MIDR_THUNDERX_81XX, 0x00, 0x00),
+ ERRATA_MIDR_REV(MIDR_THUNDERX_81XX, 0, 0),
},
#endif
#ifdef CONFIG_CAVIUM_ERRATUM_30115
@@ -317,20 +332,21 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cavium ThunderX, T88 pass 1.x - 2.2 */
.desc = "Cavium erratum 30115",
.capability = ARM64_WORKAROUND_CAVIUM_30115,
- MIDR_RANGE(MIDR_THUNDERX, 0x00,
- (1 << MIDR_VARIANT_SHIFT) | 2),
+ ERRATA_MIDR_RANGE(MIDR_THUNDERX,
+ 0, 0,
+ 1, 2),
},
{
/* Cavium ThunderX, T81 pass 1.0 - 1.2 */
.desc = "Cavium erratum 30115",
.capability = ARM64_WORKAROUND_CAVIUM_30115,
- MIDR_RANGE(MIDR_THUNDERX_81XX, 0x00, 0x02),
+ ERRATA_MIDR_REV_RANGE(MIDR_THUNDERX_81XX, 0, 0, 2),
},
{
/* Cavium ThunderX, T83 pass 1.0 */
.desc = "Cavium erratum 30115",
.capability = ARM64_WORKAROUND_CAVIUM_30115,
- MIDR_RANGE(MIDR_THUNDERX_83XX, 0x00, 0x00),
+ ERRATA_MIDR_REV(MIDR_THUNDERX_83XX, 0, 0),
},
#endif
{
@@ -344,9 +360,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Falkor erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
- MIDR_RANGE(MIDR_QCOM_FALKOR_V1,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(0, 0)),
+ ERRATA_MIDR_REV(MIDR_QCOM_FALKOR_V1, 0, 0),
},
{
.desc = "Qualcomm Technologies Kryo erratum 1003",
@@ -360,9 +374,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Falkor erratum 1009",
.capability = ARM64_WORKAROUND_REPEAT_TLBI,
- MIDR_RANGE(MIDR_QCOM_FALKOR_V1,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(0, 0)),
+ ERRATA_MIDR_REV(MIDR_QCOM_FALKOR_V1, 0, 0),
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_858921
@@ -370,47 +382,47 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A73 all versions */
.desc = "ARM erratum 858921",
.capability = ARM64_WORKAROUND_858921,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
},
#endif
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
.cpu_enable = qcom_enable_link_stack_sanitization,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
- MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
.cpu_enable = enable_smccc_arch_workaround_1,
},
#endif
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 17/22] arm64: Add helpers for checking CPU MIDR against a range
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (31 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 16/22] arm64: capabilities: Clean up midr range helpers Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-09 17:55 ` [PATCH v3 17/21] arm64: capabilities: Add support for checks based on a list of MIDRs Suzuki K Poulose
` (10 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
Add helpers for checking if the given CPU midr falls in a range
of variants/revisions for a given model.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 7 ++-----
arch/arm64/include/asm/cputype.h | 30 ++++++++++++++++++++++++++++++
arch/arm64/kernel/cpu_errata.c | 16 +++++-----------
3 files changed, 37 insertions(+), 16 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index dfce93f79ae7..e47f39f62440 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -10,6 +10,7 @@
#define __ASM_CPUFEATURE_H
#include <asm/cpucaps.h>
+#include <asm/cputype.h>
#include <asm/fpsimd.h>
#include <asm/hwcap.h>
#include <asm/sigcontext.h>
@@ -295,11 +296,7 @@ struct arm64_cpu_capabilities {
*/
void (*cpu_enable)(const struct arm64_cpu_capabilities *cap);
union {
- struct { /* To be used for erratum handling only */
- u32 midr_model;
- u32 midr_range_min, midr_range_max;
- };
-
+ struct midr_range midr_range; /* To be used for erratum handling only */
struct { /* Feature register checking */
u32 sys_reg;
u8 field_pos;
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index 2f8d39ed9c2e..6c45acb6ef38 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -115,6 +115,36 @@
#define read_cpuid(reg) read_sysreg_s(SYS_ ## reg)
+/*
+ * Represent a range of MIDR values for a given CPU model and a
+ * range of variant/revision values.
+ *
+ * @model - CPU model as defined by MIDR_CPU_MODEL
+ * @rv_min - Minimum value for the revision/variant as defined by
+ * MIDR_CPU_VAR_REV
+ * @rv_max - Maximum value for the variant/revision for the range.
+ */
+struct midr_range {
+ u32 model;
+ u32 rv_min;
+ u32 rv_max;
+};
+
+#define MIDR_RANGE(m, v_min, r_min, v_max, r_max) \
+ { \
+ .model = m, \
+ .rv_min = MIDR_CPU_VAR_REV(v_min, r_min), \
+ .rv_max = MIDR_CPU_VAR_REV(v_max, r_max), \
+ }
+
+#define MIDR_ALL_VERSIONS(m) MIDR_RANGE(m, 0, 0, 0xf, 0xf)
+
+static inline bool is_midr_in_range(u32 midr, struct midr_range const *range)
+{
+ return MIDR_IS_CPU_MODEL_RANGE(midr, range->model,
+ range->rv_min, range->rv_max);
+}
+
/*
* The CPU ID never changes at run time, so we might as well tell the
* compiler that it's constant. Use this function to read the CPU ID
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 22ea61141596..5f903e37a267 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -25,9 +25,7 @@ static bool __maybe_unused
is_affected_midr_range(const struct arm64_cpu_capabilities *entry, int scope)
{
WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
- return MIDR_IS_CPU_MODEL_RANGE(read_cpuid_id(), entry->midr_model,
- entry->midr_range_min,
- entry->midr_range_max);
+ return is_midr_in_range(read_cpuid_id(), &entry->midr_range);
}
static bool __maybe_unused
@@ -41,7 +39,7 @@ is_kryo_midr(const struct arm64_cpu_capabilities *entry, int scope)
model &= MIDR_IMPLEMENTOR_MASK | (0xf00 << MIDR_PARTNUM_SHIFT) |
MIDR_ARCHITECTURE_MASK;
- return model == entry->midr_model;
+ return model == entry->midr_range.model;
}
static bool
@@ -227,15 +225,11 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
#define CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
.matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = MIDR_CPU_VAR_REV(v_min, r_min), \
- .midr_range_max = MIDR_CPU_VAR_REV(v_max, r_max)
+ .midr_range = MIDR_RANGE(model, v_min, r_min, v_max, r_max)
#define CAP_MIDR_ALL_VERSIONS(model) \
.matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = MIDR_CPU_VAR_REV(0, 0), \
- .midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
+ .midr_range = MIDR_ALL_VERSIONS(model)
#define ERRATA_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
@@ -366,7 +360,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "Qualcomm Technologies Kryo erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
- .midr_model = MIDR_QCOM_KRYO,
+ .midr_range.model = MIDR_QCOM_KRYO,
.matches = is_kryo_midr,
},
#endif
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 17/21] arm64: capabilities: Add support for checks based on a list of MIDRs
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (32 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 17/22] arm64: Add helpers for checking CPU MIDR against a range Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-09 17:55 ` [PATCH v3 18/22] " Suzuki K Poulose
` (9 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
Add helpers for detecting an errata on list of midr ranges
of affected CPUs, with the same work around.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 1 +
arch/arm64/include/asm/cputype.h | 8 +++++
arch/arm64/kernel/cpu_errata.c | 61 +++++++++++++++++++++----------------
arch/arm64/kernel/cpufeature.c | 10 +++---
4 files changed, 50 insertions(+), 30 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index e47f39f62440..3ab1c3422f14 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -297,6 +297,7 @@ struct arm64_cpu_capabilities {
void (*cpu_enable)(const struct arm64_cpu_capabilities *cap);
union {
struct midr_range midr_range; /* To be used for erratum handling only */
+ const struct midr_range *midr_range_list;
struct { /* Feature register checking */
u32 sys_reg;
u8 field_pos;
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index 6c45acb6ef38..26bc5b302060 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -145,6 +145,14 @@ static inline bool is_midr_in_range(u32 midr, struct midr_range const *range)
range->rv_min, range->rv_max);
}
+static inline bool is_midr_in_range_list(u32 midr, struct midr_range const *ranges)
+{
+ while (ranges->model)
+ if (is_midr_in_range(midr, ranges++))
+ return true;
+ return false;
+}
+
/*
* The CPU ID never changes at run time, so we might as well tell the
* compiler that it's constant. Use this function to read the CPU ID
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 5f903e37a267..a602a3049404 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -28,6 +28,13 @@ is_affected_midr_range(const struct arm64_cpu_capabilities *entry, int scope)
return is_midr_in_range(read_cpuid_id(), &entry->midr_range);
}
+static bool __maybe_unused
+is_affected_midr_range_list(const struct arm64_cpu_capabilities *entry, int scope)
+{
+ WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
+ return is_midr_in_range_list(read_cpuid_id(), entry->midr_range_list);
+}
+
static bool __maybe_unused
is_kryo_midr(const struct arm64_cpu_capabilities *entry, int scope)
{
@@ -235,6 +242,10 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max)
+#define CAP_MIDR_RANGE_LIST(list) \
+ .matches = is_affected_midr_range_list, \
+ .midr_range_list = list
+
/* Errata affecting a range of revisions of given model variant */
#define ERRATA_MIDR_REV_RANGE(m, var, r_min, r_max) \
ERRATA_MIDR_RANGE(m, var, r_min, var, r_max)
@@ -248,6 +259,29 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
CAP_MIDR_ALL_VERSIONS(model)
+/* Errata affecting a list of midr ranges, with same work around */
+#define ERRATA_MIDR_RANGE_LIST(midr_list) \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
+ CAP_MIDR_RANGE_LIST(midr_list)
+
+#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
+
+/*
+ * List of CPUs where we need to issue a psci call to
+ * harden the branch predictor.
+ */
+static const struct midr_range arm64_bp_harden_psci_cpus[] = {
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
+ MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
+ MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
+ {},
+};
+
+#endif
+
const struct arm64_cpu_capabilities arm64_errata[] = {
#if defined(CONFIG_ARM64_ERRATUM_826319) || \
defined(CONFIG_ARM64_ERRATUM_827319) || \
@@ -382,22 +416,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
+ ERRATA_MIDR_RANGE_LIST(arm64_bp_harden_psci_cpus),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
@@ -409,16 +428,6 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
},
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
#endif
{
}
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index f66e66c79916..9eb9e9570468 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -864,6 +864,11 @@ static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */
static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
int scope)
{
+ /* List of CPUs that are not vulnerable and don't need KPTI */
+ static const struct midr_range kpti_safe_list[] = {
+ MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
+ MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
+ };
char const *str = "command line option";
/*
@@ -888,11 +893,8 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
return true;
/* Don't force KPTI for CPUs that are not vulnerable */
- switch (read_cpuid_id() & MIDR_CPU_MODEL_MASK) {
- case MIDR_CAVIUM_THUNDERX2:
- case MIDR_BRCM_VULCAN:
+ if (is_midr_in_range_list(read_cpuid_id(), kpti_safe_list))
return false;
- }
/* Defer to CPU feature registers */
return !has_cpuid_feature(entry, scope);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 18/22] arm64: capabilities: Add support for checks based on a list of MIDRs
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (33 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 17/21] arm64: capabilities: Add support for checks based on a list of MIDRs Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-09 17:55 ` [PATCH v3 18/21] arm64: capabilities: Handle shared entries Suzuki K Poulose
` (8 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
Add helpers for detecting an errata on list of midr ranges
of affected CPUs, with the same work around.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 1 +
arch/arm64/include/asm/cputype.h | 8 +++++
arch/arm64/kernel/cpu_errata.c | 61 +++++++++++++++++++++----------------
arch/arm64/kernel/cpufeature.c | 10 +++---
4 files changed, 50 insertions(+), 30 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index e47f39f62440..3ab1c3422f14 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -297,6 +297,7 @@ struct arm64_cpu_capabilities {
void (*cpu_enable)(const struct arm64_cpu_capabilities *cap);
union {
struct midr_range midr_range; /* To be used for erratum handling only */
+ const struct midr_range *midr_range_list;
struct { /* Feature register checking */
u32 sys_reg;
u8 field_pos;
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index 6c45acb6ef38..26bc5b302060 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -145,6 +145,14 @@ static inline bool is_midr_in_range(u32 midr, struct midr_range const *range)
range->rv_min, range->rv_max);
}
+static inline bool is_midr_in_range_list(u32 midr, struct midr_range const *ranges)
+{
+ while (ranges->model)
+ if (is_midr_in_range(midr, ranges++))
+ return true;
+ return false;
+}
+
/*
* The CPU ID never changes at run time, so we might as well tell the
* compiler that it's constant. Use this function to read the CPU ID
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 5f903e37a267..a602a3049404 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -28,6 +28,13 @@ is_affected_midr_range(const struct arm64_cpu_capabilities *entry, int scope)
return is_midr_in_range(read_cpuid_id(), &entry->midr_range);
}
+static bool __maybe_unused
+is_affected_midr_range_list(const struct arm64_cpu_capabilities *entry, int scope)
+{
+ WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
+ return is_midr_in_range_list(read_cpuid_id(), entry->midr_range_list);
+}
+
static bool __maybe_unused
is_kryo_midr(const struct arm64_cpu_capabilities *entry, int scope)
{
@@ -235,6 +242,10 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max)
+#define CAP_MIDR_RANGE_LIST(list) \
+ .matches = is_affected_midr_range_list, \
+ .midr_range_list = list
+
/* Errata affecting a range of revisions of given model variant */
#define ERRATA_MIDR_REV_RANGE(m, var, r_min, r_max) \
ERRATA_MIDR_RANGE(m, var, r_min, var, r_max)
@@ -248,6 +259,29 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
CAP_MIDR_ALL_VERSIONS(model)
+/* Errata affecting a list of midr ranges, with same work around */
+#define ERRATA_MIDR_RANGE_LIST(midr_list) \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
+ CAP_MIDR_RANGE_LIST(midr_list)
+
+#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
+
+/*
+ * List of CPUs where we need to issue a psci call to
+ * harden the branch predictor.
+ */
+static const struct midr_range arm64_bp_harden_psci_cpus[] = {
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
+ MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
+ MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
+ {},
+};
+
+#endif
+
const struct arm64_cpu_capabilities arm64_errata[] = {
#if defined(CONFIG_ARM64_ERRATUM_826319) || \
defined(CONFIG_ARM64_ERRATUM_827319) || \
@@ -382,22 +416,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
+ ERRATA_MIDR_RANGE_LIST(arm64_bp_harden_psci_cpus),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
@@ -409,16 +428,6 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
},
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
#endif
{
}
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index f66e66c79916..9eb9e9570468 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -864,6 +864,11 @@ static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */
static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
int scope)
{
+ /* List of CPUs that are not vulnerable and don't need KPTI */
+ static const struct midr_range kpti_safe_list[] = {
+ MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
+ MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
+ };
char const *str = "command line option";
/*
@@ -888,11 +893,8 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
return true;
/* Don't force KPTI for CPUs that are not vulnerable */
- switch (read_cpuid_id() & MIDR_CPU_MODEL_MASK) {
- case MIDR_CAVIUM_THUNDERX2:
- case MIDR_BRCM_VULCAN:
+ if (is_midr_in_range_list(read_cpuid_id(), kpti_safe_list))
return false;
- }
/* Defer to CPU feature registers */
return !has_cpuid_feature(entry, scope);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 18/21] arm64: capabilities: Handle shared entries
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (34 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 18/22] " Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-09 17:55 ` [PATCH v3 19/21] arm64: Add MIDR encoding for Arm Cortex-A55 and Cortex-A35 Suzuki K Poulose
` (7 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
Some capabilities have different criteria for detection and associated
actions based on the matching criteria, even though they all share the
same capability bit. So far we have used multiple entries with the same
capability bit to handle this. This is prone to errors, as the
cpu_enable is invoked for each entry, irrespective of whether the
detection rule applies to the CPU or not. And also this complicates
other helpers, e.g, __this_cpu_has_cap.
This patch adds a wrapper entry to cover all the possible variations
of a capability by maintaining list of matches + cpu_enable callbacks.
To avoid complicating the prototypes for the "matches()", we use
arm64_cpu_capabilities maintain the list and we ignore all the other
fields except the matches & cpu_enable.
This ensures :
1) The capabilitiy is set when at least one of the entry detects
2) Action is only taken for the entries that "matches".
This avoids explicit checks in the cpu_enable() take some action.
The only constraint here is that, all the entries should have the
same "type" (i.e, scope and conflict rules).
If a cpu_enable() method is associated with multiple matches for a
single capability, care should be taken that either the match criteria
are mutually exclusive, or that the method is robust against being
called multiple times.
This also reverts the changes introduced by commit 67948af41f2e6818ed
("arm64: capabilities: Handle duplicate entries for a capability").
Cc: Dave Martin <dave.martin@arm.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 11 ++++++++
arch/arm64/kernel/cpu_errata.c | 53 ++++++++++++++++++++++++++++++++-----
arch/arm64/kernel/cpufeature.c | 10 +++----
3 files changed, 61 insertions(+), 13 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 3ab1c3422f14..074537acc08b 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -306,6 +306,17 @@ struct arm64_cpu_capabilities {
bool sign;
unsigned long hwcap;
};
+ /*
+ * A list of "matches/cpu_enable" pair for the same "capability"
+ * of the same "type" as described by the parent. All the
+ * fields, except "matches"/"cpu_enable" are ignored in the list.
+ * The cpu_enable is invoked only if the corresponding entry
+ * "matches()". However, if a cpu_enable() method is associated
+ * with multiple matches, care should be taken that either the
+ * match criteria are mutually exclusive, or that the method is
+ * robust against being called multiple times.
+ */
+ const struct arm64_cpu_capabilities *cap_list;
};
};
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index a602a3049404..902d281ea26f 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -264,6 +264,36 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
CAP_MIDR_RANGE_LIST(midr_list)
+/*
+ * Generic helper for handling capabilties with multiple (match,enable) pairs
+ * of call backs, sharing the same capability bit.
+ * Iterate over each entry to see if at least one matches.
+ */
+static bool multi_entry_cap_matches(const struct arm64_cpu_capabilities *entry,
+ int scope)
+{
+ const struct arm64_cpu_capabilities *caps = entry->cap_list;
+
+ for (; caps->matches; caps++)
+ if (caps->matches(caps, scope))
+ return true;
+ return false;
+}
+
+/*
+ * Take appropriate action for all matching entries in the shared capability
+ * entry.
+ */
+static void multi_entry_cap_cpu_enable(const struct arm64_cpu_capabilities *entry)
+{
+ const struct arm64_cpu_capabilities *caps = entry->cap_list;
+
+ for (; caps->matches; caps++)
+ if (caps->matches(caps, SCOPE_LOCAL_CPU) &&
+ caps->cpu_enable)
+ caps->cpu_enable(caps);
+}
+
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
/*
@@ -280,6 +310,18 @@ static const struct midr_range arm64_bp_harden_psci_cpus[] = {
{},
};
+static const struct arm64_cpu_capabilities arm64_bp_harden_list[] = {
+ {
+ CAP_MIDR_RANGE_LIST(arm64_bp_harden_psci_cpus),
+ .cpu_enable = enable_smccc_arch_workaround_1,
+ },
+ {
+ CAP_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ .cpu_enable = qcom_enable_link_stack_sanitization,
+ },
+ {},
+};
+
#endif
const struct arm64_cpu_capabilities arm64_errata[] = {
@@ -416,13 +458,10 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_RANGE_LIST(arm64_bp_harden_psci_cpus),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
- .cpu_enable = qcom_enable_link_stack_sanitization,
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
+ .matches = multi_entry_cap_matches,
+ .cpu_enable = multi_entry_cap_cpu_enable,
+ .cap_list = arm64_bp_harden_list,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9eb9e9570468..d8663822c604 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1225,9 +1225,8 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
return false;
for (caps = cap_array; caps->matches; caps++)
- if (caps->capability == cap &&
- caps->matches(caps, SCOPE_LOCAL_CPU))
- return true;
+ if (caps->capability == cap)
+ return caps->matches(caps, SCOPE_LOCAL_CPU);
return false;
}
@@ -1313,18 +1312,17 @@ static void __init enable_cpu_capabilities(u16 scope_mask)
*
* Returns "false" on conflicts.
*/
-static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list,
+static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps,
u16 scope_mask)
{
bool cpu_has_cap, system_has_cap;
- const struct arm64_cpu_capabilities *caps = caps_list;
scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
if (!(caps->type & scope_mask))
continue;
- cpu_has_cap = __this_cpu_has_cap(caps_list, caps->capability);
+ cpu_has_cap = caps->matches(caps, SCOPE_LOCAL_CPU);
system_has_cap = cpus_have_cap(caps->capability);
if (system_has_cap) {
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 19/21] arm64: Add MIDR encoding for Arm Cortex-A55 and Cortex-A35
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (35 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 18/21] arm64: capabilities: Handle shared entries Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-09 17:55 ` [PATCH v3 19/22] arm64: capabilities: Handle shared entries Suzuki K Poulose
` (6 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
Update the MIDR encodings for the Cortex-A55 and Cortex-A35
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cputype.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index 26bc5b302060..308507e99430 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -83,6 +83,8 @@
#define ARM_CPU_PART_CORTEX_A53 0xD03
#define ARM_CPU_PART_CORTEX_A73 0xD09
#define ARM_CPU_PART_CORTEX_A75 0xD0A
+#define ARM_CPU_PART_CORTEX_A35 0xD04
+#define ARM_CPU_PART_CORTEX_A55 0xD05
#define APM_CPU_PART_POTENZA 0x000
@@ -101,6 +103,8 @@
#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72)
#define MIDR_CORTEX_A73 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A73)
#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75)
+#define MIDR_CORTEX_A35 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A35)
+#define MIDR_CORTEX_A55 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A55)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 19/22] arm64: capabilities: Handle shared entries
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (36 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 19/21] arm64: Add MIDR encoding for Arm Cortex-A55 and Cortex-A35 Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-12 17:17 ` Dave Martin
2018-02-09 17:55 ` [PATCH v3 20/22] arm64: Add MIDR encoding for Arm Cortex-A55 and Cortex-A35 Suzuki K Poulose
` (5 subsequent siblings)
43 siblings, 1 reply; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
Some capabilities have different criteria for detection and associated
actions based on the matching criteria, even though they all share the
same capability bit. So far we have used multiple entries with the same
capability bit to handle this. This is prone to errors, as the
cpu_enable is invoked for each entry, irrespective of whether the
detection rule applies to the CPU or not. And also this complicates
other helpers, e.g, __this_cpu_has_cap.
This patch adds a wrapper entry to cover all the possible variations
of a capability by maintaining list of matches + cpu_enable callbacks.
To avoid complicating the prototypes for the "matches()", we use
arm64_cpu_capabilities maintain the list and we ignore all the other
fields except the matches & cpu_enable.
This ensures :
1) The capabilitiy is set when at least one of the entry detects
2) Action is only taken for the entries that "matches".
This avoids explicit checks in the cpu_enable() take some action.
The only constraint here is that, all the entries should have the
same "type" (i.e, scope and conflict rules).
If a cpu_enable() method is associated with multiple matches for a
single capability, care should be taken that either the match criteria
are mutually exclusive, or that the method is robust against being
called multiple times.
This also reverts the changes introduced by commit 67948af41f2e6818ed
("arm64: capabilities: Handle duplicate entries for a capability").
Cc: Dave Martin <dave.martin@arm.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 11 ++++++++
arch/arm64/kernel/cpu_errata.c | 53 ++++++++++++++++++++++++++++++++-----
arch/arm64/kernel/cpufeature.c | 10 +++----
3 files changed, 61 insertions(+), 13 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 3ab1c3422f14..074537acc08b 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -306,6 +306,17 @@ struct arm64_cpu_capabilities {
bool sign;
unsigned long hwcap;
};
+ /*
+ * A list of "matches/cpu_enable" pair for the same "capability"
+ * of the same "type" as described by the parent. All the
+ * fields, except "matches"/"cpu_enable" are ignored in the list.
+ * The cpu_enable is invoked only if the corresponding entry
+ * "matches()". However, if a cpu_enable() method is associated
+ * with multiple matches, care should be taken that either the
+ * match criteria are mutually exclusive, or that the method is
+ * robust against being called multiple times.
+ */
+ const struct arm64_cpu_capabilities *cap_list;
};
};
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index a602a3049404..902d281ea26f 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -264,6 +264,36 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
CAP_MIDR_RANGE_LIST(midr_list)
+/*
+ * Generic helper for handling capabilties with multiple (match,enable) pairs
+ * of call backs, sharing the same capability bit.
+ * Iterate over each entry to see if at least one matches.
+ */
+static bool multi_entry_cap_matches(const struct arm64_cpu_capabilities *entry,
+ int scope)
+{
+ const struct arm64_cpu_capabilities *caps = entry->cap_list;
+
+ for (; caps->matches; caps++)
+ if (caps->matches(caps, scope))
+ return true;
+ return false;
+}
+
+/*
+ * Take appropriate action for all matching entries in the shared capability
+ * entry.
+ */
+static void multi_entry_cap_cpu_enable(const struct arm64_cpu_capabilities *entry)
+{
+ const struct arm64_cpu_capabilities *caps = entry->cap_list;
+
+ for (; caps->matches; caps++)
+ if (caps->matches(caps, SCOPE_LOCAL_CPU) &&
+ caps->cpu_enable)
+ caps->cpu_enable(caps);
+}
+
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
/*
@@ -280,6 +310,18 @@ static const struct midr_range arm64_bp_harden_psci_cpus[] = {
{},
};
+static const struct arm64_cpu_capabilities arm64_bp_harden_list[] = {
+ {
+ CAP_MIDR_RANGE_LIST(arm64_bp_harden_psci_cpus),
+ .cpu_enable = enable_smccc_arch_workaround_1,
+ },
+ {
+ CAP_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ .cpu_enable = qcom_enable_link_stack_sanitization,
+ },
+ {},
+};
+
#endif
const struct arm64_cpu_capabilities arm64_errata[] = {
@@ -416,13 +458,10 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_RANGE_LIST(arm64_bp_harden_psci_cpus),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
- .cpu_enable = qcom_enable_link_stack_sanitization,
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
+ .matches = multi_entry_cap_matches,
+ .cpu_enable = multi_entry_cap_cpu_enable,
+ .cap_list = arm64_bp_harden_list,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9eb9e9570468..d8663822c604 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1225,9 +1225,8 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
return false;
for (caps = cap_array; caps->matches; caps++)
- if (caps->capability == cap &&
- caps->matches(caps, SCOPE_LOCAL_CPU))
- return true;
+ if (caps->capability == cap)
+ return caps->matches(caps, SCOPE_LOCAL_CPU);
return false;
}
@@ -1313,18 +1312,17 @@ static void __init enable_cpu_capabilities(u16 scope_mask)
*
* Returns "false" on conflicts.
*/
-static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list,
+static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps,
u16 scope_mask)
{
bool cpu_has_cap, system_has_cap;
- const struct arm64_cpu_capabilities *caps = caps_list;
scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
if (!(caps->type & scope_mask))
continue;
- cpu_has_cap = __this_cpu_has_cap(caps_list, caps->capability);
+ cpu_has_cap = caps->matches(caps, SCOPE_LOCAL_CPU);
system_has_cap = cpus_have_cap(caps->capability);
if (system_has_cap) {
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 19/22] arm64: capabilities: Handle shared entries
2018-02-09 17:55 ` [PATCH v3 19/22] arm64: capabilities: Handle shared entries Suzuki K Poulose
@ 2018-02-12 17:17 ` Dave Martin
2018-03-08 12:16 ` Suzuki K Poulose
0 siblings, 1 reply; 62+ messages in thread
From: Dave Martin @ 2018-02-12 17:17 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Feb 09, 2018 at 05:55:08PM +0000, Suzuki K Poulose wrote:
> Some capabilities have different criteria for detection and associated
> actions based on the matching criteria, even though they all share the
> same capability bit. So far we have used multiple entries with the same
> capability bit to handle this. This is prone to errors, as the
> cpu_enable is invoked for each entry, irrespective of whether the
> detection rule applies to the CPU or not. And also this complicates
> other helpers, e.g, __this_cpu_has_cap.
>
> This patch adds a wrapper entry to cover all the possible variations
> of a capability by maintaining list of matches + cpu_enable callbacks.
> To avoid complicating the prototypes for the "matches()", we use
> arm64_cpu_capabilities maintain the list and we ignore all the other
> fields except the matches & cpu_enable.
>
> This ensures :
>
> 1) The capabilitiy is set when at least one of the entry detects
> 2) Action is only taken for the entries that "matches".
>
> This avoids explicit checks in the cpu_enable() take some action.
> The only constraint here is that, all the entries should have the
> same "type" (i.e, scope and conflict rules).
>
> If a cpu_enable() method is associated with multiple matches for a
> single capability, care should be taken that either the match criteria
> are mutually exclusive, or that the method is robust against being
> called multiple times.
>
> This also reverts the changes introduced by commit 67948af41f2e6818ed
> ("arm64: capabilities: Handle duplicate entries for a capability").
>
> Cc: Dave Martin <dave.martin@arm.com>
> Cc: Robin Murphy <robin.murphy@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> ---
> arch/arm64/include/asm/cpufeature.h | 11 ++++++++
> arch/arm64/kernel/cpu_errata.c | 53 ++++++++++++++++++++++++++++++++-----
> arch/arm64/kernel/cpufeature.c | 10 +++----
> 3 files changed, 61 insertions(+), 13 deletions(-)
>
> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> index 3ab1c3422f14..074537acc08b 100644
> --- a/arch/arm64/include/asm/cpufeature.h
> +++ b/arch/arm64/include/asm/cpufeature.h
> @@ -306,6 +306,17 @@ struct arm64_cpu_capabilities {
> bool sign;
> unsigned long hwcap;
> };
> + /*
> + * A list of "matches/cpu_enable" pair for the same "capability"
> + * of the same "type" as described by the parent. All the
> + * fields, except "matches"/"cpu_enable" are ignored in the list.
Nit: This is not quite true: other fields may be needed, for use by the
matches() method -- for example, if matches == has_cpuid_feature, then
sys_reg, field_pos etc. will be used.
To keep things simple, maybe say "Only matches(), cpu_enable() and
fields relevant to these methods are significant in the list." ?
> + * The cpu_enable is invoked only if the corresponding entry
> + * "matches()". However, if a cpu_enable() method is associated
> + * with multiple matches, care should be taken that either the
> + * match criteria are mutually exclusive, or that the method is
> + * robust against being called multiple times.
> + */
> + const struct arm64_cpu_capabilities *cap_list;
Nit: this is not really a list of capabilities, as noted above.
Can we call it something like "match_list"?
> };
> };
>
> diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
> index a602a3049404..902d281ea26f 100644
> --- a/arch/arm64/kernel/cpu_errata.c
> +++ b/arch/arm64/kernel/cpu_errata.c
> @@ -264,6 +264,36 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
> .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
> CAP_MIDR_RANGE_LIST(midr_list)
>
> +/*
> + * Generic helper for handling capabilties with multiple (match,enable) pairs
> + * of call backs, sharing the same capability bit.
> + * Iterate over each entry to see if at least one matches.
> + */
> +static bool multi_entry_cap_matches(const struct arm64_cpu_capabilities *entry,
> + int scope)
> +{
> + const struct arm64_cpu_capabilities *caps = entry->cap_list;
> +
> + for (; caps->matches; caps++)
> + if (caps->matches(caps, scope))
> + return true;
Nit: add blank line?
> + return false;
> +}
> +
> +/*
> + * Take appropriate action for all matching entries in the shared capability
> + * entry.
> + */
> +static void multi_entry_cap_cpu_enable(const struct arm64_cpu_capabilities *entry)
> +{
> + const struct arm64_cpu_capabilities *caps = entry->cap_list;
> +
> + for (; caps->matches; caps++)
Nit: can we move the initialiser into the for(); so
for (entry->cap_list; caps->matches; [...]
IMHO it's more readable to avoid empty expressions in for() unless
there's a good reason.
> + if (caps->matches(caps, SCOPE_LOCAL_CPU) &&
> + caps->cpu_enable)
> + caps->cpu_enable(caps);
> +}
> +
> #ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
>
> /*
> @@ -280,6 +310,18 @@ static const struct midr_range arm64_bp_harden_psci_cpus[] = {
> {},
> };
>
> +static const struct arm64_cpu_capabilities arm64_bp_harden_list[] = {
> + {
> + CAP_MIDR_RANGE_LIST(arm64_bp_harden_psci_cpus),
> + .cpu_enable = enable_smccc_arch_workaround_1,
> + },
> + {
> + CAP_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
> + .cpu_enable = qcom_enable_link_stack_sanitization,
> + },
> + {},
> +};
> +
> #endif
>
> const struct arm64_cpu_capabilities arm64_errata[] = {
> @@ -416,13 +458,10 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
> #ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
> {
> .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
> - ERRATA_MIDR_RANGE_LIST(arm64_bp_harden_psci_cpus),
> - .cpu_enable = enable_smccc_arch_workaround_1,
> - },
> - {
> - .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
> - ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
> - .cpu_enable = qcom_enable_link_stack_sanitization,
> + .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
> + .matches = multi_entry_cap_matches,
> + .cpu_enable = multi_entry_cap_cpu_enable,
> + .cap_list = arm64_bp_harden_list,
> },
> {
> .capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 9eb9e9570468..d8663822c604 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -1225,9 +1225,8 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
> return false;
>
> for (caps = cap_array; caps->matches; caps++)
> - if (caps->capability == cap &&
> - caps->matches(caps, SCOPE_LOCAL_CPU))
> - return true;
> + if (caps->capability == cap)
> + return caps->matches(caps, SCOPE_LOCAL_CPU);
Nit: add blank line?
> return false;
> }
>
> @@ -1313,18 +1312,17 @@ static void __init enable_cpu_capabilities(u16 scope_mask)
> *
> * Returns "false" on conflicts.
> */
> -static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list,
> +static bool __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps,
> u16 scope_mask)
> {
> bool cpu_has_cap, system_has_cap;
> - const struct arm64_cpu_capabilities *caps = caps_list;
>
> scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
> for (; caps->matches; caps++) {
> if (!(caps->type & scope_mask))
> continue;
>
> - cpu_has_cap = __this_cpu_has_cap(caps_list, caps->capability);
> + cpu_has_cap = caps->matches(caps, SCOPE_LOCAL_CPU);
> system_has_cap = cpus_have_cap(caps->capability);
>
> if (system_has_cap) {
> --
> 2.14.3
>
With fair consideration to the nits above,
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
^ permalink raw reply [flat|nested] 62+ messages in thread* [PATCH v3 19/22] arm64: capabilities: Handle shared entries
2018-02-12 17:17 ` Dave Martin
@ 2018-03-08 12:16 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-08 12:16 UTC (permalink / raw)
To: linux-arm-kernel
On 12/02/18 17:17, Dave Martin wrote:
> On Fri, Feb 09, 2018 at 05:55:08PM +0000, Suzuki K Poulose wrote:
>> Some capabilities have different criteria for detection and associated
>> actions based on the matching criteria, even though they all share the
>> same capability bit. So far we have used multiple entries with the same
>> capability bit to handle this. This is prone to errors, as the
>> cpu_enable is invoked for each entry, irrespective of whether the
>> detection rule applies to the CPU or not. And also this complicates
>> other helpers, e.g, __this_cpu_has_cap.
>>
>> This patch adds a wrapper entry to cover all the possible variations
>> of a capability by maintaining list of matches + cpu_enable callbacks.
>> To avoid complicating the prototypes for the "matches()", we use
>> arm64_cpu_capabilities maintain the list and we ignore all the other
>> fields except the matches & cpu_enable.
>>
>> This ensures :
>>
>> 1) The capabilitiy is set when at least one of the entry detects
>> 2) Action is only taken for the entries that "matches".
>>
>> This avoids explicit checks in the cpu_enable() take some action.
>> The only constraint here is that, all the entries should have the
>> same "type" (i.e, scope and conflict rules).
>>
>> If a cpu_enable() method is associated with multiple matches for a
>> single capability, care should be taken that either the match criteria
>> are mutually exclusive, or that the method is robust against being
>> called multiple times.
>>
>> This also reverts the changes introduced by commit 67948af41f2e6818ed
>> ("arm64: capabilities: Handle duplicate entries for a capability").
>>
>> Cc: Dave Martin <dave.martin@arm.com>
>> Cc: Robin Murphy <robin.murphy@arm.com>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>> ---
>> arch/arm64/include/asm/cpufeature.h | 11 ++++++++
>> arch/arm64/kernel/cpu_errata.c | 53 ++++++++++++++++++++++++++++++++-----
>> arch/arm64/kernel/cpufeature.c | 10 +++----
>> 3 files changed, 61 insertions(+), 13 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
>> index 3ab1c3422f14..074537acc08b 100644
>> --- a/arch/arm64/include/asm/cpufeature.h
>> +++ b/arch/arm64/include/asm/cpufeature.h
>> @@ -306,6 +306,17 @@ struct arm64_cpu_capabilities {
>> bool sign;
>> unsigned long hwcap;
>> };
>> + /*
>> + * A list of "matches/cpu_enable" pair for the same "capability"
>> + * of the same "type" as described by the parent. All the
>> + * fields, except "matches"/"cpu_enable" are ignored in the list.
>
> Nit: This is not quite true: other fields may be needed, for use by the
> matches() method -- for example, if matches == has_cpuid_feature, then
> sys_reg, field_pos etc. will be used.
>
> To keep things simple, maybe say "Only matches(), cpu_enable() and
> fields relevant to these methods are significant in the list." ?
Will fix it.
>
>> + * The cpu_enable is invoked only if the corresponding entry
>> + * "matches()". However, if a cpu_enable() method is associated
>> + * with multiple matches, care should be taken that either the
>> + * match criteria are mutually exclusive, or that the method is
>> + * robust against being called multiple times.
>> + */
>> + const struct arm64_cpu_capabilities *cap_list;
>
> Nit: this is not really a list of capabilities, as noted above.
>
> Can we call it something like "match_list"?
Ok.
>
>> };
>> };
>>
>> diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
>> index a602a3049404..902d281ea26f 100644
>> --- a/arch/arm64/kernel/cpu_errata.c
>> +++ b/arch/arm64/kernel/cpu_errata.c
>> @@ -264,6 +264,36 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
>> .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
>> CAP_MIDR_RANGE_LIST(midr_list)
>>
>> +/*
>> + * Generic helper for handling capabilties with multiple (match,enable) pairs
>> + * of call backs, sharing the same capability bit.
>> + * Iterate over each entry to see if at least one matches.
>> + */
>> +static bool multi_entry_cap_matches(const struct arm64_cpu_capabilities *entry,
>> + int scope)
>> +{
>> + const struct arm64_cpu_capabilities *caps = entry->cap_list;
>> +
>> + for (; caps->matches; caps++)
>> + if (caps->matches(caps, scope))
>> + return true;
>
> Nit: add blank line?
>> + return false;
>> +}
>> +
>> +/*
>> + * Take appropriate action for all matching entries in the shared capability
>> + * entry.
>> + */
>> +static void multi_entry_cap_cpu_enable(const struct arm64_cpu_capabilities *entry)
>> +{
>> + const struct arm64_cpu_capabilities *caps = entry->cap_list;
>> +
>> + for (; caps->matches; caps++)
>
> Nit: can we move the initialiser into the for(); so
>
> for (entry->cap_list; caps->matches; [...]
>
> IMHO it's more readable to avoid empty expressions in for() unless
> there's a good reason.
>
>> + if (caps->matches(caps, SCOPE_LOCAL_CPU) &&
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
>> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
>> index 9eb9e9570468..d8663822c604 100644
>> --- a/arch/arm64/kernel/cpufeature.c
>> +++ b/arch/arm64/kernel/cpufeature.c
>> @@ -1225,9 +1225,8 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
>> return false;
>>
>> for (caps = cap_array; caps->matches; caps++)
>> - if (caps->capability == cap &&
>> - caps->matches(caps, SCOPE_LOCAL_CPU))
>> - return true;
>> + if (caps->capability == cap)
>> + return caps->matches(caps, SCOPE_LOCAL_CPU);
>
> Nit: add blank line?
>
>> return false;
>
> With fair consideration to the nits above,
Will do
>
> Reviewed-by: Dave Martin <Dave.Martin@arm.com>
Thanks
Suzuki
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v3 20/22] arm64: Add MIDR encoding for Arm Cortex-A55 and Cortex-A35
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (37 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 19/22] arm64: capabilities: Handle shared entries Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-09 17:55 ` [PATCH v3 20/21] arm64: Delay enabling hardware DBM feature Suzuki K Poulose
` (4 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
Update the MIDR encodings for the Cortex-A55 and Cortex-A35
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cputype.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index 26bc5b302060..308507e99430 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -83,6 +83,8 @@
#define ARM_CPU_PART_CORTEX_A53 0xD03
#define ARM_CPU_PART_CORTEX_A73 0xD09
#define ARM_CPU_PART_CORTEX_A75 0xD0A
+#define ARM_CPU_PART_CORTEX_A35 0xD04
+#define ARM_CPU_PART_CORTEX_A55 0xD05
#define APM_CPU_PART_POTENZA 0x000
@@ -101,6 +103,8 @@
#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72)
#define MIDR_CORTEX_A73 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A73)
#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75)
+#define MIDR_CORTEX_A35 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A35)
+#define MIDR_CORTEX_A55 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A55)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 20/21] arm64: Delay enabling hardware DBM feature
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (38 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 20/22] arm64: Add MIDR encoding for Arm Cortex-A55 and Cortex-A35 Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-09 17:55 ` [PATCH v3 21/21] arm64: Add work around for Arm Cortex-A55 Erratum 1024718 Suzuki K Poulose
` (3 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
We enable hardware DBM bit in a capable CPU, very early in the
boot via __cpu_setup. This doesn't give us a flexibility of
optionally disable the feature, as the clearing the bit
is a bit costly as the TLB can cache the settings. Instead,
we delay enabling the feature until the CPU is brought up
into the kernel. We use the feature capability mechanism
to handle it.
The hardware DBM is a non-conflicting feature. i.e, the kernel
can safely run with a mix of CPUs with some using the feature
and the others don't. So, it is safe for a late CPU to have
this capability and enable it, even if the active CPUs don't.
To get this handled properly by the infrastructure, we
unconditionally set the capability and only enable it
on CPUs which really have the feature.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
Changes since v2:
- Hide the description of the DBM capability to not mislead
the user that the capability is detected while none of the
CPUs actuall use it.
---
arch/arm64/include/asm/cpucaps.h | 3 ++-
arch/arm64/kernel/cpufeature.c | 53 ++++++++++++++++++++++++++++++++++++++++
arch/arm64/mm/proc.S | 13 +++++-----
3 files changed, 61 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index bb263820de13..8df80cc828ac 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -45,7 +45,8 @@
#define ARM64_HARDEN_BRANCH_PREDICTOR 24
#define ARM64_HARDEN_BP_POST_GUEST_EXIT 25
#define ARM64_HAS_RAS_EXTN 26
+#define ARM64_HW_DBM 27
-#define ARM64_NCAPS 27
+#define ARM64_NCAPS 28
#endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index d8663822c604..72431d983d87 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -939,6 +939,32 @@ static int __init parse_kpti(char *str)
__setup("kpti=", parse_kpti);
#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */
+/*
+ * Dummy matches call back for capabilities that are always
+ * turned ON.
+ */
+static bool cpu_cap_always_on(const struct arm64_cpu_capabilities *__unused,
+ int __unused_scope)
+{
+ return true;
+}
+
+#ifdef CONFIG_ARM64_HW_AFDBM
+static inline void __cpu_enable_hw_dbm(void)
+{
+ u64 tcr = read_sysreg(tcr_el1) | TCR_HD;
+
+ write_sysreg(tcr, tcr_el1);
+ isb();
+}
+
+static void cpu_enable_hw_dbm(struct arm64_cpu_capabilities const *cap)
+{
+ if (has_cpuid_feature(cap, SCOPE_LOCAL_CPU))
+ __cpu_enable_hw_dbm();
+}
+#endif
+
static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused)
{
/*
@@ -1103,6 +1129,33 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.cpu_enable = cpu_clear_disr,
},
#endif /* CONFIG_ARM64_RAS_EXTN */
+#ifdef CONFIG_ARM64_HW_AFDBM
+ {
+ /*
+ * Since we turn this on always, we don't want the user to
+ * think that the feature is available when it may not be.
+ * So hide the description.
+ *
+ * .desc = "Hardware pagetable Dirty Bit Management",
+ *
+ */
+ .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE,
+ .capability = ARM64_HW_DBM,
+ .sys_reg = SYS_ID_AA64MMFR1_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64MMFR1_HADBS_SHIFT,
+ .min_field_value = 2,
+ /*
+ * DBM is a non-conflicting feature. i.e, the kernel can safely
+ * run a mix of CPUs with and without the feature. So, we
+ * unconditionally enable the capability to allow any late CPU
+ * to use the feature. We only enable the control bits on the
+ * CPU, if it actually supports.
+ */
+ .matches = cpu_cap_always_on,
+ .cpu_enable = cpu_enable_hw_dbm,
+ },
+#endif
{},
};
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 71baed7e592a..ba9b69aa7f32 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -437,16 +437,15 @@ ENTRY(__cpu_setup)
tcr_compute_pa_size x10, #TCR_IPS_SHIFT, x5, x6
#ifdef CONFIG_ARM64_HW_AFDBM
/*
- * Hardware update of the Access and Dirty bits.
+ * Enable hardware update of the Access Flags bit.
+ * Hardware dirty bit management is enabled later,
+ * via capabilities.
*/
mrs x9, ID_AA64MMFR1_EL1
and x9, x9, #0xf
- cbz x9, 2f
- cmp x9, #2
- b.lt 1f
- orr x10, x10, #TCR_HD // hardware Dirty flag update
-1: orr x10, x10, #TCR_HA // hardware Access flag update
-2:
+ cbz x9, 1f
+ orr x10, x10, #TCR_HA // hardware Access flag update
+1:
#endif /* CONFIG_ARM64_HW_AFDBM */
msr tcr_el1, x10
ret // return to head.S
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 21/21] arm64: Add work around for Arm Cortex-A55 Erratum 1024718
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (39 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 20/21] arm64: Delay enabling hardware DBM feature Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-09 17:55 ` [PATCH v3 21/22] arm64: Delay enabling hardware DBM feature Suzuki K Poulose
` (2 subsequent siblings)
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
Some variants of the Arm Cortex-55 cores (r0p0, r0p1, r1p0) suffer
from an erratum 1024718, which causes incorrect updates when DBM/AP
bits in a page table entry is modified without a break-before-make
sequence. The work around is to skip enabling the hardware DBM feature
on the affected cores. The hardware Access Flag management features
is not affected. There are some other cores suffering from this
errata, which could be added to the midr_list to trigger the work
around.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: ckadabi at codeaurora.org
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
Documentation/arm64/silicon-errata.txt | 1 +
arch/arm64/Kconfig | 14 ++++++++++++++
arch/arm64/kernel/cpufeature.c | 14 +++++++++++++-
3 files changed, 28 insertions(+), 1 deletion(-)
diff --git a/Documentation/arm64/silicon-errata.txt b/Documentation/arm64/silicon-errata.txt
index c1d520de6dfe..3b2f2dd82225 100644
--- a/Documentation/arm64/silicon-errata.txt
+++ b/Documentation/arm64/silicon-errata.txt
@@ -55,6 +55,7 @@ stable kernels.
| ARM | Cortex-A57 | #834220 | ARM64_ERRATUM_834220 |
| ARM | Cortex-A72 | #853709 | N/A |
| ARM | Cortex-A73 | #858921 | ARM64_ERRATUM_858921 |
+| ARM | Cortex-A55 | #1024718 | ARM64_ERRATUM_1024718 |
| ARM | MMU-500 | #841119,#826419 | N/A |
| | | | |
| Cavium | ThunderX ITS | #22375, #24313 | CAVIUM_ERRATUM_22375 |
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b488076d63c2..85fcbd1bddac 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -461,6 +461,20 @@ config ARM64_ERRATUM_843419
If unsure, say Y.
+config ARM64_ERRATUM_1024718
+ bool "Cortex-A55: 1024718: Update of DBM/AP bits without break before make might result in incorrect update"
+ default y
+ help
+ This option adds work around for Arm Cortex-A55 Erratum 1024718.
+
+ Affected Cortex-A55 cores (r0p0, r0p1, r1p0) could cause incorrect
+ update of the hardware dirty bit when the DBM/AP bits are updated
+ without a break-before-make. The work around is to disable the usage
+ of hardware DBM locally on the affected cores. CPUs not affected by
+ erratum will continue to use the feature.
+
+ If unsure, say Y.
+
config CAVIUM_ERRATUM_22375
bool "Cavium erratum 22375, 24313"
default y
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 72431d983d87..1eb925d52922 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -958,9 +958,21 @@ static inline void __cpu_enable_hw_dbm(void)
isb();
}
+static bool cpu_has_erratum_1024718(void)
+{
+ static const struct midr_range cpus[] = {
+ MIDR_RANGE(MIDR_CORTEX_A55, 0, 0, 1, 0), // A55 r0p0 -r1p0
+ {},
+ };
+
+ return IS_ENABLED(CONFIG_ARM64_ERRATUM_1024718) &&
+ is_midr_in_range_list(read_cpuid_id(), cpus);
+}
+
static void cpu_enable_hw_dbm(struct arm64_cpu_capabilities const *cap)
{
- if (has_cpuid_feature(cap, SCOPE_LOCAL_CPU))
+ if (has_cpuid_feature(cap, SCOPE_LOCAL_CPU) &&
+ !cpu_has_erratum_1024718())
__cpu_enable_hw_dbm();
}
#endif
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 21/22] arm64: Delay enabling hardware DBM feature
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (40 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 21/21] arm64: Add work around for Arm Cortex-A55 Erratum 1024718 Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-09 18:58 ` Dave Martin
2018-02-09 17:55 ` [PATCH v3 22/22] arm64: Add work around for Arm Cortex-A55 Erratum 1024718 Suzuki K Poulose
2018-02-09 18:04 ` [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
43 siblings, 1 reply; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
We enable hardware DBM bit in a capable CPU, very early in the
boot via __cpu_setup. This doesn't give us a flexibility of
optionally disable the feature, as the clearing the bit
is a bit costly as the TLB can cache the settings. Instead,
we delay enabling the feature until the CPU is brought up
into the kernel. We use the feature capability mechanism
to handle it.
The hardware DBM is a non-conflicting feature. i.e, the kernel
can safely run with a mix of CPUs with some using the feature
and the others don't. So, it is safe for a late CPU to have
this capability and enable it, even if the active CPUs don't.
To get this handled properly by the infrastructure, we
unconditionally set the capability and only enable it
on CPUs which really have the feature. Also, we print the
feature detection from the "matches" call back to make sure
we don't mislead the user when none of the CPUs could use the
feature.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
Changes since V2
- Print the feature detection message only when at least one CPU
is actually using it.
---
arch/arm64/include/asm/cpucaps.h | 3 +-
arch/arm64/kernel/cpufeature.c | 71 ++++++++++++++++++++++++++++++++++++++++
arch/arm64/mm/proc.S | 13 ++++----
3 files changed, 79 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index bb263820de13..8df80cc828ac 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -45,7 +45,8 @@
#define ARM64_HARDEN_BRANCH_PREDICTOR 24
#define ARM64_HARDEN_BP_POST_GUEST_EXIT 25
#define ARM64_HAS_RAS_EXTN 26
+#define ARM64_HW_DBM 27
-#define ARM64_NCAPS 27
+#define ARM64_NCAPS 28
#endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index d8663822c604..a96a1f94f427 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -939,6 +939,57 @@ static int __init parse_kpti(char *str)
__setup("kpti=", parse_kpti);
#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */
+#ifdef CONFIG_ARM64_HW_AFDBM
+static inline void __cpu_enable_hw_dbm(void)
+{
+ u64 tcr = read_sysreg(tcr_el1) | TCR_HD;
+
+ write_sysreg(tcr, tcr_el1);
+ isb();
+}
+
+static bool cpu_can_use_dbm(const struct arm64_cpu_capabilities *cap)
+{
+ return has_cpuid_feature(cap, SCOPE_LOCAL_CPU);
+}
+
+static void cpu_enable_hw_dbm(struct arm64_cpu_capabilities const *cap)
+{
+ if (cpu_can_use_dbm(cap))
+ __cpu_enable_hw_dbm();
+}
+
+static bool has_hw_dbm(const struct arm64_cpu_capabilities *cap,
+ int __unused)
+{
+ static bool detected = false;
+ /*
+ * DBM is a non-conflicting feature. i.e, the kernel can safely
+ * run a mix of CPUs with and without the feature. So, we
+ * unconditionally enable the capability to allow any late CPU
+ * to use the feature. We only enable the control bits on the
+ * CPU, if it actually supports.
+ *
+ * We have to make sure we print the "feature" detection only
+ * when at least one CPU actually uses it. So check if this CPU
+ * can actually use it and print the message exactly once.
+ *
+ * This is safe as all CPUs (including secondary CPUs - due to the
+ * LOCAL_CPU scope - and the hotplugged CPUs - via verification)
+ * goes through the "matches" check exactly once. Also if a CPU
+ * matches the criteria, it is guaranteed that the CPU will turn
+ * the DBM on, as the capability is unconditionally enabled.
+ */
+ if (!detected && cpu_can_use_dbm(cap)) {
+ detected = true;
+ pr_info("detected feature: Hardware dirty bit management\n");
+ }
+
+ return true;
+}
+
+#endif
+
static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused)
{
/*
@@ -1103,6 +1154,26 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.cpu_enable = cpu_clear_disr,
},
#endif /* CONFIG_ARM64_RAS_EXTN */
+#ifdef CONFIG_ARM64_HW_AFDBM
+ {
+ /*
+ * Since we turn this on always, we don't want the user to
+ * think that the feature is available when it may not be.
+ * So hide the description.
+ *
+ * .desc = "Hardware pagetable Dirty Bit Management",
+ *
+ */
+ .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE,
+ .capability = ARM64_HW_DBM,
+ .sys_reg = SYS_ID_AA64MMFR1_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64MMFR1_HADBS_SHIFT,
+ .min_field_value = 2,
+ .matches = has_hw_dbm,
+ .cpu_enable = cpu_enable_hw_dbm,
+ },
+#endif
{},
};
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 71baed7e592a..ba9b69aa7f32 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -437,16 +437,15 @@ ENTRY(__cpu_setup)
tcr_compute_pa_size x10, #TCR_IPS_SHIFT, x5, x6
#ifdef CONFIG_ARM64_HW_AFDBM
/*
- * Hardware update of the Access and Dirty bits.
+ * Enable hardware update of the Access Flags bit.
+ * Hardware dirty bit management is enabled later,
+ * via capabilities.
*/
mrs x9, ID_AA64MMFR1_EL1
and x9, x9, #0xf
- cbz x9, 2f
- cmp x9, #2
- b.lt 1f
- orr x10, x10, #TCR_HD // hardware Dirty flag update
-1: orr x10, x10, #TCR_HA // hardware Access flag update
-2:
+ cbz x9, 1f
+ orr x10, x10, #TCR_HA // hardware Access flag update
+1:
#endif /* CONFIG_ARM64_HW_AFDBM */
msr tcr_el1, x10
ret // return to head.S
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 21/22] arm64: Delay enabling hardware DBM feature
2018-02-09 17:55 ` [PATCH v3 21/22] arm64: Delay enabling hardware DBM feature Suzuki K Poulose
@ 2018-02-09 18:58 ` Dave Martin
2018-03-07 17:39 ` Suzuki K Poulose
0 siblings, 1 reply; 62+ messages in thread
From: Dave Martin @ 2018-02-09 18:58 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Feb 09, 2018 at 05:55:12PM +0000, Suzuki K Poulose wrote:
> We enable hardware DBM bit in a capable CPU, very early in the
> boot via __cpu_setup. This doesn't give us a flexibility of
> optionally disable the feature, as the clearing the bit
> is a bit costly as the TLB can cache the settings. Instead,
> we delay enabling the feature until the CPU is brought up
> into the kernel. We use the feature capability mechanism
> to handle it.
>
> The hardware DBM is a non-conflicting feature. i.e, the kernel
> can safely run with a mix of CPUs with some using the feature
> and the others don't. So, it is safe for a late CPU to have
> this capability and enable it, even if the active CPUs don't.
>
> To get this handled properly by the infrastructure, we
> unconditionally set the capability and only enable it
> on CPUs which really have the feature. Also, we print the
> feature detection from the "matches" call back to make sure
> we don't mislead the user when none of the CPUs could use the
> feature.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> ---
> Changes since V2
> - Print the feature detection message only when at least one CPU
> is actually using it.
> ---
> arch/arm64/include/asm/cpucaps.h | 3 +-
> arch/arm64/kernel/cpufeature.c | 71 ++++++++++++++++++++++++++++++++++++++++
> arch/arm64/mm/proc.S | 13 ++++----
> 3 files changed, 79 insertions(+), 8 deletions(-)
>
> diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
> index bb263820de13..8df80cc828ac 100644
> --- a/arch/arm64/include/asm/cpucaps.h
> +++ b/arch/arm64/include/asm/cpucaps.h
> @@ -45,7 +45,8 @@
> #define ARM64_HARDEN_BRANCH_PREDICTOR 24
> #define ARM64_HARDEN_BP_POST_GUEST_EXIT 25
> #define ARM64_HAS_RAS_EXTN 26
> +#define ARM64_HW_DBM 27
>
> -#define ARM64_NCAPS 27
> +#define ARM64_NCAPS 28
>
> #endif /* __ASM_CPUCAPS_H */
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index d8663822c604..a96a1f94f427 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -939,6 +939,57 @@ static int __init parse_kpti(char *str)
> __setup("kpti=", parse_kpti);
> #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */
>
> +#ifdef CONFIG_ARM64_HW_AFDBM
> +static inline void __cpu_enable_hw_dbm(void)
> +{
> + u64 tcr = read_sysreg(tcr_el1) | TCR_HD;
> +
> + write_sysreg(tcr, tcr_el1);
> + isb();
> +}
> +
> +static bool cpu_can_use_dbm(const struct arm64_cpu_capabilities *cap)
> +{
> + return has_cpuid_feature(cap, SCOPE_LOCAL_CPU);
> +}
> +
> +static void cpu_enable_hw_dbm(struct arm64_cpu_capabilities const *cap)
> +{
> + if (cpu_can_use_dbm(cap))
> + __cpu_enable_hw_dbm();
> +}
> +
> +static bool has_hw_dbm(const struct arm64_cpu_capabilities *cap,
> + int __unused)
> +{
> + static bool detected = false;
> + /*
> + * DBM is a non-conflicting feature. i.e, the kernel can safely
> + * run a mix of CPUs with and without the feature. So, we
> + * unconditionally enable the capability to allow any late CPU
> + * to use the feature. We only enable the control bits on the
> + * CPU, if it actually supports.
> + *
> + * We have to make sure we print the "feature" detection only
> + * when at least one CPU actually uses it. So check if this CPU
> + * can actually use it and print the message exactly once.
> + *
> + * This is safe as all CPUs (including secondary CPUs - due to the
> + * LOCAL_CPU scope - and the hotplugged CPUs - via verification)
> + * goes through the "matches" check exactly once. Also if a CPU
> + * matches the criteria, it is guaranteed that the CPU will turn
> + * the DBM on, as the capability is unconditionally enabled.
> + */
> + if (!detected && cpu_can_use_dbm(cap)) {
> + detected = true;
> + pr_info("detected feature: Hardware dirty bit management\n");
> + }
Can we just do
if (cpu_can_use_dbm(cap))
pr_info_once(...);
Then we can get rid of "detected".
[...]
Cheers
---Dave
^ permalink raw reply [flat|nested] 62+ messages in thread* [PATCH v3 21/22] arm64: Delay enabling hardware DBM feature
2018-02-09 18:58 ` Dave Martin
@ 2018-03-07 17:39 ` Suzuki K Poulose
2018-03-08 13:53 ` Dave Martin
0 siblings, 1 reply; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-07 17:39 UTC (permalink / raw)
To: linux-arm-kernel
On 09/02/18 18:58, Dave Martin wrote:
> On Fri, Feb 09, 2018 at 05:55:12PM +0000, Suzuki K Poulose wrote:
>> We enable hardware DBM bit in a capable CPU, very early in the
>> boot via __cpu_setup. This doesn't give us a flexibility of
>> optionally disable the feature, as the clearing the bit
>> is a bit costly as the TLB can cache the settings. Instead,
>> we delay enabling the feature until the CPU is brought up
>> into the kernel. We use the feature capability mechanism
>> to handle it.
>>
>> The hardware DBM is a non-conflicting feature. i.e, the kernel
>> can safely run with a mix of CPUs with some using the feature
>> and the others don't. So, it is safe for a late CPU to have
>> this capability and enable it, even if the active CPUs don't.
>>
>> To get this handled properly by the infrastructure, we
>> unconditionally set the capability and only enable it
>> on CPUs which really have the feature. Also, we print the
>> feature detection from the "matches" call back to make sure
>> we don't mislead the user when none of the CPUs could use the
>> feature.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Dave Martin <dave.martin@arm.com>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>> ---
>> Changes since V2
>> - Print the feature detection message only when at least one CPU
>> is actually using it.
>> +static bool has_hw_dbm(const struct arm64_cpu_capabilities *cap,
>> + int __unused)
>> +{
>> + static bool detected = false;
>> + /*
>> + * DBM is a non-conflicting feature. i.e, the kernel can safely
>> + * run a mix of CPUs with and without the feature. So, we
>> + * unconditionally enable the capability to allow any late CPU
>> + * to use the feature. We only enable the control bits on the
>> + * CPU, if it actually supports.
>> + *
>> + * We have to make sure we print the "feature" detection only
>> + * when at least one CPU actually uses it. So check if this CPU
>> + * can actually use it and print the message exactly once.
>> + *
>> + * This is safe as all CPUs (including secondary CPUs - due to the
>> + * LOCAL_CPU scope - and the hotplugged CPUs - via verification)
>> + * goes through the "matches" check exactly once. Also if a CPU
>> + * matches the criteria, it is guaranteed that the CPU will turn
>> + * the DBM on, as the capability is unconditionally enabled.
>> + */
>> + if (!detected && cpu_can_use_dbm(cap)) {
>> + detected = true;
>> + pr_info("detected feature: Hardware dirty bit management\n");
>> + }
>
> Can we just do
>
> if (cpu_can_use_dbm(cap))
> pr_info_once(...);
>
> Then we can get rid of "detected".
The reason for open coding is the cost of cpu_can_use_dbm() with
addition of black listed CPUs in the next patch in the series.
Cheers
Suzuki
^ permalink raw reply [flat|nested] 62+ messages in thread* [PATCH v3 21/22] arm64: Delay enabling hardware DBM feature
2018-03-07 17:39 ` Suzuki K Poulose
@ 2018-03-08 13:53 ` Dave Martin
2018-03-08 13:55 ` Mark Rutland
0 siblings, 1 reply; 62+ messages in thread
From: Dave Martin @ 2018-03-08 13:53 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Mar 07, 2018 at 05:39:09PM +0000, Suzuki K Poulose wrote:
> On 09/02/18 18:58, Dave Martin wrote:
> >On Fri, Feb 09, 2018 at 05:55:12PM +0000, Suzuki K Poulose wrote:
> >>We enable hardware DBM bit in a capable CPU, very early in the
> >>boot via __cpu_setup. This doesn't give us a flexibility of
> >>optionally disable the feature, as the clearing the bit
> >>is a bit costly as the TLB can cache the settings. Instead,
> >>we delay enabling the feature until the CPU is brought up
> >>into the kernel. We use the feature capability mechanism
> >>to handle it.
> >>
> >>The hardware DBM is a non-conflicting feature. i.e, the kernel
> >>can safely run with a mix of CPUs with some using the feature
> >>and the others don't. So, it is safe for a late CPU to have
> >>this capability and enable it, even if the active CPUs don't.
> >>
> >>To get this handled properly by the infrastructure, we
> >>unconditionally set the capability and only enable it
> >>on CPUs which really have the feature. Also, we print the
> >>feature detection from the "matches" call back to make sure
> >>we don't mislead the user when none of the CPUs could use the
> >>feature.
> >>
> >>Cc: Catalin Marinas <catalin.marinas@arm.com>
> >>Cc: Dave Martin <dave.martin@arm.com>
> >>Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> >>---
> >>Changes since V2
> >> - Print the feature detection message only when at least one CPU
> >> is actually using it.
>
>
> >>+static bool has_hw_dbm(const struct arm64_cpu_capabilities *cap,
> >>+ int __unused)
> >>+{
> >>+ static bool detected = false;
> >>+ /*
> >>+ * DBM is a non-conflicting feature. i.e, the kernel can safely
> >>+ * run a mix of CPUs with and without the feature. So, we
> >>+ * unconditionally enable the capability to allow any late CPU
> >>+ * to use the feature. We only enable the control bits on the
> >>+ * CPU, if it actually supports.
> >>+ *
> >>+ * We have to make sure we print the "feature" detection only
> >>+ * when at least one CPU actually uses it. So check if this CPU
> >>+ * can actually use it and print the message exactly once.
> >>+ *
> >>+ * This is safe as all CPUs (including secondary CPUs - due to the
> >>+ * LOCAL_CPU scope - and the hotplugged CPUs - via verification)
> >>+ * goes through the "matches" check exactly once. Also if a CPU
> >>+ * matches the criteria, it is guaranteed that the CPU will turn
> >>+ * the DBM on, as the capability is unconditionally enabled.
> >>+ */
> >>+ if (!detected && cpu_can_use_dbm(cap)) {
> >>+ detected = true;
> >>+ pr_info("detected feature: Hardware dirty bit management\n");
> >>+ }
> >
> >Can we just do
> >
> > if (cpu_can_use_dbm(cap))
> > pr_info_once(...);
> >
> >Then we can get rid of "detected".
>
> The reason for open coding is the cost of cpu_can_use_dbm() with
> addition of black listed CPUs in the next patch in the series.
Oh, I see. Yes, that makes sense.
There's obvious raciness here, but I guess pr_info_once() doesn't defend
against that either. In practice, we don't race booting of two CPUs
against each other IIUC.
If you really like you could make detected __read_mostly like
printk_once() does, but that's no big deal if this is not on a hot path
(and probably not even then).
Cheers
---Dave
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v3 22/22] arm64: Add work around for Arm Cortex-A55 Erratum 1024718
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (41 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 21/22] arm64: Delay enabling hardware DBM feature Suzuki K Poulose
@ 2018-02-09 17:55 ` Suzuki K Poulose
2018-02-09 18:04 ` [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 17:55 UTC (permalink / raw)
To: linux-arm-kernel
Some variants of the Arm Cortex-55 cores (r0p0, r0p1, r1p0) suffer
from an erratum 1024718, which causes incorrect updates when DBM/AP
bits in a page table entry is modified without a break-before-make
sequence. The work around is to skip enabling the hardware DBM feature
on the affected cores. The hardware Access Flag management features
is not affected. There are some other cores suffering from this
errata, which could be added to the midr_list to trigger the work
around.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: ckadabi at codeaurora.org
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
Documentation/arm64/silicon-errata.txt | 1 +
arch/arm64/Kconfig | 14 ++++++++++++++
arch/arm64/kernel/cpufeature.c | 14 +++++++++++++-
3 files changed, 28 insertions(+), 1 deletion(-)
diff --git a/Documentation/arm64/silicon-errata.txt b/Documentation/arm64/silicon-errata.txt
index c1d520de6dfe..3b2f2dd82225 100644
--- a/Documentation/arm64/silicon-errata.txt
+++ b/Documentation/arm64/silicon-errata.txt
@@ -55,6 +55,7 @@ stable kernels.
| ARM | Cortex-A57 | #834220 | ARM64_ERRATUM_834220 |
| ARM | Cortex-A72 | #853709 | N/A |
| ARM | Cortex-A73 | #858921 | ARM64_ERRATUM_858921 |
+| ARM | Cortex-A55 | #1024718 | ARM64_ERRATUM_1024718 |
| ARM | MMU-500 | #841119,#826419 | N/A |
| | | | |
| Cavium | ThunderX ITS | #22375, #24313 | CAVIUM_ERRATUM_22375 |
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b488076d63c2..85fcbd1bddac 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -461,6 +461,20 @@ config ARM64_ERRATUM_843419
If unsure, say Y.
+config ARM64_ERRATUM_1024718
+ bool "Cortex-A55: 1024718: Update of DBM/AP bits without break before make might result in incorrect update"
+ default y
+ help
+ This option adds work around for Arm Cortex-A55 Erratum 1024718.
+
+ Affected Cortex-A55 cores (r0p0, r0p1, r1p0) could cause incorrect
+ update of the hardware dirty bit when the DBM/AP bits are updated
+ without a break-before-make. The work around is to disable the usage
+ of hardware DBM locally on the affected cores. CPUs not affected by
+ erratum will continue to use the feature.
+
+ If unsure, say Y.
+
config CAVIUM_ERRATUM_22375
bool "Cavium erratum 22375, 24313"
default y
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index a96a1f94f427..360209733396 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -948,9 +948,21 @@ static inline void __cpu_enable_hw_dbm(void)
isb();
}
+static bool cpu_has_erratum_1024718(void)
+{
+ static const struct midr_range cpus[] = {
+ MIDR_RANGE(MIDR_CORTEX_A55, 0, 0, 1, 0), // A55 r0p0 -r1p0
+ {},
+ };
+
+ return IS_ENABLED(CONFIG_ARM64_ERRATUM_1024718) &&
+ is_midr_in_range_list(read_cpuid_id(), cpus);
+}
+
static bool cpu_can_use_dbm(const struct arm64_cpu_capabilities *cap)
{
- return has_cpuid_feature(cap, SCOPE_LOCAL_CPU);
+ return has_cpuid_feature(cap, SCOPE_LOCAL_CPU) &&
+ !cpu_has_erratum_1024718();
}
static void cpu_enable_hw_dbm(struct arm64_cpu_capabilities const *cap)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH v3 00/21] arm64: Rework cpu capabilities handling
2018-02-09 17:54 [PATCH v3 00/21] arm64: Rework cpu capabilities handling Suzuki K Poulose
` (42 preceding siblings ...)
2018-02-09 17:55 ` [PATCH v3 22/22] arm64: Add work around for Arm Cortex-A55 Erratum 1024718 Suzuki K Poulose
@ 2018-02-09 18:04 ` Suzuki K Poulose
43 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-02-09 18:04 UTC (permalink / raw)
To: linux-arm-kernel
On 09/02/18 17:54, Suzuki K Poulose wrote:
> This series reworks the arm64 CPU capabilities handling (which
> manages the system features and errata). The current infrastructure
> doesn't allow fine control for handling different features or errata.
> There is one rule for features and another rule for errata.
>
> * Features are checked only once, after all the boot time CPUs are
> activated. Any new CPU that is brought up is prevented from booting
> if it misses a feature already established. If the new CPU has a
> feature not enabled already, it is allowed to boot.
>
> * Errata checks are performed on all the CPUs and any new CPU is
> OK to miss the capability. However if a late CPU requires a work around,
> then we fail the CPU.
>
> This doesn't always apply to some features. e.g, KPTI is a security
> feature which should be applied when at least one CPU needs it. So,
> the tests should be performed on all the booting CPUs individually.
> Also, if a CPU that needs this security feature is brought up later,
> when the system has not enabled it, the CPU can boot making the system
> insecure. Another exception is the hardware DBM for page tables. The
> kernel can safely run with a mix of CPUs that have the feature turned
> on and off. This again causes problem when a new CPU is brought up
> which may not have the feature, which is killed.
>
> Also there are other features like, GICV3 system register access,
> which now need to be enabled very early based on the boot CPU to
> allow the use of Priority handling to implement NMI.
>
> This calls for finer level of control per capability and the series
> implements the same by defining how to deal with a conflict of a
> given capability on a CPU with that of the system level state. It
> also consolidates the handling of features and errata into generic
> helpers. The table of features and errata are left as they are to
> allow easier look up for a given type.
>
> The series also gets rid of duplicate entries for a single capability
> by introducing a wrapper entry which takes care of managing a list
> of entries with distinct matches/enable pair.
>
> We also cleans up the MIDR range handling and cleans up some of the
> errata checks where the entries were duplicated for checking different
> CPU models. Finally it also implements a work around for Arm Cortex-A55
> erratum 1024718 based on the new infrastructure.
>
> Applies on arm64 for-next/core. The tree is available at :
>
> git://linux-arm.org/linux-skp.git cpu-cap-rework/v3
Apologies for the mess... obviously I messed up my folder.
If anyone needs the patches for testing, the branch above is sane.
Apologies..
Suzuki
^ permalink raw reply [flat|nested] 62+ messages in thread