* [PATCH ARM64 v4.4 V3 21/44] arm64: cpufeature: Pass capability structure to ->enable callback
From: Viresh Kumar @ 2019-08-29 11:34 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Will Deacon <will.deacon@arm.com>
commit 0a0d111d40fd1dc588cc590fab6b55d86ddc71d3 upstream.
In order to invoke the CPU capability ->matches callback from the ->enable
callback for applying local-CPU workarounds, we need a handle on the
capability structure.
This patch passes a pointer to the capability structure to the ->enable
callback.
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
[ v4.4: Use &caps[i] instead as caps isn't incremented ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/kernel/cpufeature.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index c74df3ca000e..474b34243521 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -832,7 +832,7 @@ void enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
* uses an IPI, giving us a PSTATE that disappears when
* we return.
*/
- stop_machine(caps[i].enable, NULL, cpu_online_mask);
+ stop_machine(caps[i].enable, (void *)&caps[i], cpu_online_mask);
}
#ifdef CONFIG_HOTPLUG_CPU
@@ -881,7 +881,7 @@ static void verify_local_cpu_capabilities(void)
cpu_die_early();
}
if (caps[i].enable)
- caps[i].enable(NULL);
+ caps[i].enable((void *)&caps[i]);
}
for (i = 0, caps = arm64_hwcaps; caps[i].matches; i++) {
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 20/44] arm64: Run enable method for errata work arounds on late CPUs
From: Viresh Kumar @ 2019-08-29 11:34 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Suzuki K Poulose <suzuki.poulose@arm.com>
commit 55b35d070c2534dfb714b883f3c3ae05d02032da upstream.
When a CPU is brought up after we have finalised the system
wide capabilities (i.e, features and errata), we make sure the
new CPU doesn't need a new errata work around which has not been
detected already. However we don't run enable() method on the new
CPU for the errata work arounds already detected. This could
cause the new CPU running without potential work arounds.
It is upto the "enable()" method to decide if this CPU should
do something about the errata.
Fixes: commit 6a6efbb45b7d95c84 ("arm64: Verify CPU errata work arounds on hotplugged CPU")
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Andre Przywara <andre.przywara@arm.com>
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/kernel/cpu_errata.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index d9f095439011..047f1da59cb1 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -125,15 +125,18 @@ void verify_local_cpu_errata(void)
{
const struct arm64_cpu_capabilities *caps = arm64_errata;
- for (; caps->matches; caps++)
- if (!cpus_have_cap(caps->capability) &&
- caps->matches(caps, SCOPE_LOCAL_CPU)) {
+ for (; caps->matches; caps++) {
+ if (cpus_have_cap(caps->capability)) {
+ if (caps->enable)
+ caps->enable((void *)caps);
+ } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
pr_crit("CPU%d: Requires work around for %s, not detected"
" at boot time\n",
smp_processor_id(),
caps->desc ? : "an erratum");
cpu_die_early();
}
+ }
}
void check_local_cpu_errata(void)
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 19/44] arm64: Rearrange CPU errata workaround checks
From: Viresh Kumar @ 2019-08-29 11:34 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Suzuki K Poulose <suzuki.poulose@arm.com>
commit c47a1900ad710fd2c97127e2ba19da1df79cf733 upstream.
Right now we run through the work around checks on a CPU
from __cpuinfo_store_cpu. There are some problems with that:
1) We initialise the system wide CPU feature registers only after the
Boot CPU updates its cpuinfo. Now, if a work around depends on the
variance of a CPU ID feature (e.g, check for Cache Line size mismatch),
we have no way of performing it cleanly for the boot CPU.
2) It is out of place, invoked from __cpuinfo_store_cpu() in cpuinfo.c. It
is not an obvious place for that.
This patch rearranges the CPU specific capability(aka work around) checks.
1) At the moment we use verify_local_cpu_capabilities() to check if a new
CPU has all the system advertised features. Use this for the secondary CPUs
to perform the work around check. For that we rename
verify_local_cpu_capabilities() => check_local_cpu_capabilities()
which:
If the system wide capabilities haven't been initialised (i.e, the CPU
is activated at the boot), update the system wide detected work arounds.
Otherwise (i.e a CPU hotplugged in later) verify that this CPU conforms to the
system wide capabilities.
2) Boot CPU updates the work arounds from smp_prepare_boot_cpu() after we have
initialised the system wide CPU feature values.
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Andre Przywara <andre.przywara@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/cpufeature.h | 8 +-------
arch/arm64/kernel/cpufeature.c | 23 +++++++++++++++--------
arch/arm64/kernel/cpuinfo.c | 2 --
arch/arm64/kernel/smp.c | 8 +++++++-
4 files changed, 23 insertions(+), 18 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 0267bab6ac18..1bc51f8835e5 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -177,13 +177,7 @@ void __init enable_errata_workarounds(void);
void verify_local_cpu_errata(void);
-#ifdef CONFIG_HOTPLUG_CPU
-void verify_local_cpu_capabilities(void);
-#else
-static inline void verify_local_cpu_capabilities(void)
-{
-}
-#endif
+void check_local_cpu_capabilities(void);
u64 read_system_reg(u32 id);
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 7773bea6927e..c74df3ca000e 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -860,18 +860,11 @@ static inline void set_sys_caps_initialised(void)
* cannot do anything to fix it up and could cause unexpected failures. So
* we park the CPU.
*/
-void verify_local_cpu_capabilities(void)
+static void verify_local_cpu_capabilities(void)
{
int i;
const struct arm64_cpu_capabilities *caps;
- /*
- * If we haven't computed the system capabilities, there is nothing
- * to verify.
- */
- if (!sys_caps_initialised)
- return;
-
verify_local_cpu_errata();
caps = arm64_features;
@@ -902,6 +895,20 @@ void verify_local_cpu_capabilities(void)
}
}
+void check_local_cpu_capabilities(void)
+{
+ /*
+ * If we haven't finalised the system capabilities, this CPU gets
+ * a chance to update the errata work arounds.
+ * Otherwise, this CPU should verify that it has all the system
+ * advertised capabilities.
+ */
+ if (!sys_caps_initialised)
+ check_local_cpu_errata();
+ else
+ verify_local_cpu_capabilities();
+}
+
#else /* !CONFIG_HOTPLUG_CPU */
static inline void set_sys_caps_initialised(void)
diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c
index 0166cfbc866c..13e659fda04a 100644
--- a/arch/arm64/kernel/cpuinfo.c
+++ b/arch/arm64/kernel/cpuinfo.c
@@ -239,8 +239,6 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info)
info->reg_mvfr2 = read_cpuid(MVFR2_EL1);
cpuinfo_detect_icache_policy(info);
-
- check_local_cpu_errata();
}
void cpuinfo_store_cpu(void)
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 23e8ae0c6305..02b76bb78d59 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -161,7 +161,7 @@ asmlinkage notrace void secondary_start_kernel(void)
* this CPU ticks all of those. If it doesn't, the CPU will
* fail to come online.
*/
- verify_local_cpu_capabilities();
+ check_local_cpu_capabilities();
if (cpu_ops[cpu]->cpu_postboot)
cpu_ops[cpu]->cpu_postboot();
@@ -357,6 +357,12 @@ void __init smp_prepare_boot_cpu(void)
{
set_my_cpu_offset(per_cpu_offset(smp_processor_id()));
cpuinfo_store_boot_cpu();
+ /*
+ * Run the errata work around checks on the boot CPU, once we have
+ * initialised the cpu feature infrastructure from
+ * cpuinfo_store_boot_cpu() above.
+ */
+ check_local_cpu_errata();
}
static u64 __init of_get_cpu_mpidr(struct device_node *dn)
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 18/44] arm64: errata: Calling enable functions for CPU errata too
From: Viresh Kumar @ 2019-08-29 11:34 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Andre Przywara <andre.przywara@arm.com>
commit 8e2318521bf5837dae093413f81292b59d49d030 upstream.
Currently we call the (optional) enable function for CPU _features_
only. As CPU _errata_ descriptions share the same data structure and
having an enable function is useful for errata as well (for instance
to set bits in SCTLR), lets call it when enumerating erratas too.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/cpufeature.h | 2 ++
arch/arm64/kernel/cpu_errata.c | 5 +++++
arch/arm64/kernel/cpufeature.c | 3 ++-
3 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index dd1aab8e52aa..0267bab6ac18 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -171,7 +171,9 @@ void __init setup_cpu_features(void);
void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
const char *info);
+void enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps);
void check_local_cpu_errata(void);
+void __init enable_errata_workarounds(void);
void verify_local_cpu_errata(void);
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index a3567881c01b..d9f095439011 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -140,3 +140,8 @@ void check_local_cpu_errata(void)
{
update_cpu_capabilities(arm64_errata, "enabling workaround for");
}
+
+void __init enable_errata_workarounds(void)
+{
+ enable_cpu_capabilities(arm64_errata);
+}
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9a4b638b1c18..7773bea6927e 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -820,7 +820,7 @@ void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
* Run through the enabled capabilities and enable() it on all active
* CPUs
*/
-static void enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
+void enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
{
int i;
@@ -923,6 +923,7 @@ void __init setup_cpu_features(void)
/* Set the CPU feature capabilies */
setup_feature_capabilities();
+ enable_errata_workarounds();
setup_cpu_hwcaps();
/* Advertise that we have computed the system capabilities */
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 17/44] arm64: Verify CPU errata work arounds on hotplugged CPU
From: Viresh Kumar @ 2019-08-29 11:34 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Suzuki K Poulose <suzuki.poulose@arm.com>
commit 6a6efbb45b7d95c84840010095367eb06a64f342 upstream.
CPU Errata work arounds are detected and applied to the
kernel code at boot time and the data is then freed up.
If a new hotplugged CPU requires a work around which
was not applied at boot time, there is nothing we can
do but simply fail the booting.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
[ Viresh: Resolved rebase conflict ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/cpufeature.h | 2 ++
arch/arm64/kernel/cpu_errata.c | 20 ++++++++++++++++++++
arch/arm64/kernel/cpufeature.c | 2 ++
3 files changed, 24 insertions(+)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 4c31e14c0f0e..dd1aab8e52aa 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -173,6 +173,8 @@ void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
const char *info);
void check_local_cpu_errata(void);
+void verify_local_cpu_errata(void);
+
#ifdef CONFIG_HOTPLUG_CPU
void verify_local_cpu_capabilities(void);
#else
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 0971d80d3623..a3567881c01b 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -116,6 +116,26 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
}
};
+/*
+ * The CPU Errata work arounds are detected and applied at boot time
+ * and the related information is freed soon after. If the new CPU requires
+ * an errata not detected at boot, fail this CPU.
+ */
+void verify_local_cpu_errata(void)
+{
+ const struct arm64_cpu_capabilities *caps = arm64_errata;
+
+ for (; caps->matches; caps++)
+ if (!cpus_have_cap(caps->capability) &&
+ caps->matches(caps, SCOPE_LOCAL_CPU)) {
+ pr_crit("CPU%d: Requires work around for %s, not detected"
+ " at boot time\n",
+ smp_processor_id(),
+ caps->desc ? : "an erratum");
+ cpu_die_early();
+ }
+}
+
void check_local_cpu_errata(void)
{
update_cpu_capabilities(arm64_errata, "enabling workaround for");
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index a0273cd8be51..9a4b638b1c18 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -872,6 +872,8 @@ void verify_local_cpu_capabilities(void)
if (!sys_caps_initialised)
return;
+ verify_local_cpu_errata();
+
caps = arm64_features;
for (i = 0; caps[i].matches; i++) {
if (!cpus_have_cap(caps[i].capability) || !caps[i].sys_reg)
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 16/44] arm64: Move cpu_die_early to smp.c
From: Viresh Kumar @ 2019-08-29 11:34 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Suzuki K Poulose <suzuki.poulose@arm.com>
Commit fce6361fe9b0caeba0c05b7d72ceda406f8780df upstream.
This patch moves cpu_die_early to smp.c, where it fits better.
No functional changes, except for adding the necessary checks
for CONFIG_HOTPLUG_CPU.
Cc: Mark Rutland <mark.rutland@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/smp.h | 1 +
arch/arm64/kernel/cpufeature.c | 19 -------------------
arch/arm64/kernel/smp.c | 22 ++++++++++++++++++++++
3 files changed, 23 insertions(+), 19 deletions(-)
diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h
index 53b53a9b3e5a..32e75ee21d5e 100644
--- a/arch/arm64/include/asm/smp.h
+++ b/arch/arm64/include/asm/smp.h
@@ -68,6 +68,7 @@ extern int __cpu_disable(void);
extern void __cpu_die(unsigned int cpu);
extern void cpu_die(void);
+extern void cpu_die_early(void);
static inline void cpu_park_loop(void)
{
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 4adf18307568..a0273cd8be51 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -852,25 +852,6 @@ static inline void set_sys_caps_initialised(void)
sys_caps_initialised = true;
}
-/*
- * Kill the calling secondary CPU, early in bringup before it is turned
- * online.
- */
-void cpu_die_early(void)
-{
- int cpu = smp_processor_id();
-
- pr_crit("CPU%d: will not boot\n", cpu);
-
- /* Mark this CPU absent */
- set_cpu_present(cpu, 0);
-
- /* Check if we can park ourselves */
- if (cpu_ops[cpu] && cpu_ops[cpu]->cpu_die)
- cpu_ops[cpu]->cpu_die(cpu);
- cpu_park_loop();
-}
-
/*
* Run through the enabled system capabilities and enable() it on this CPU.
* The capabilities were decided based on the available CPUs at the boot time.
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 03c0946b79d2..23e8ae0c6305 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -312,6 +312,28 @@ void cpu_die(void)
}
#endif
+/*
+ * Kill the calling secondary CPU, early in bringup before it is turned
+ * online.
+ */
+void cpu_die_early(void)
+{
+ int cpu = smp_processor_id();
+
+ pr_crit("CPU%d: will not boot\n", cpu);
+
+ /* Mark this CPU absent */
+ set_cpu_present(cpu, 0);
+
+#ifdef CONFIG_HOTPLUG_CPU
+ /* Check if we can park ourselves */
+ if (cpu_ops[cpu] && cpu_ops[cpu]->cpu_die)
+ cpu_ops[cpu]->cpu_die(cpu);
+#endif
+
+ cpu_park_loop();
+}
+
static void __init hyp_mode_check(void)
{
if (is_hyp_mode_available())
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 15/44] arm64: Add a helper for parking CPUs in a loop
From: Viresh Kumar @ 2019-08-29 11:34 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Suzuki K Poulose <suzuki.poulose@arm.com>
Commit c4bc34d20273db698c51951a1951dba0a722e162 upstream.
Adds a routine which can be used to park CPUs (spinning in kernel)
when they can't be killed.
Cc: Mark Rutland <mark.rutland@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/smp.h | 8 ++++++++
arch/arm64/kernel/cpufeature.c | 5 +----
2 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h
index d9c3d6a6100a..53b53a9b3e5a 100644
--- a/arch/arm64/include/asm/smp.h
+++ b/arch/arm64/include/asm/smp.h
@@ -69,4 +69,12 @@ extern int __cpu_disable(void);
extern void __cpu_die(unsigned int cpu);
extern void cpu_die(void);
+static inline void cpu_park_loop(void)
+{
+ for (;;) {
+ wfe();
+ wfi();
+ }
+}
+
#endif /* ifndef __ASM_SMP_H */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index b7f01bf47988..4adf18307568 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -868,10 +868,7 @@ void cpu_die_early(void)
/* Check if we can park ourselves */
if (cpu_ops[cpu] && cpu_ops[cpu]->cpu_die)
cpu_ops[cpu]->cpu_die(cpu);
- asm(
- "1: wfe\n"
- " wfi\n"
- " b 1b");
+ cpu_park_loop();
}
/*
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 14/44] arm64: Introduce cpu_die_early
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Suzuki K Poulose <suzuki.poulose@arm.com>
commit ee02a15919cf86c004142edaa05b43f7ff10edf0 upstream.
Or in other words, make fail_incapable_cpu() reusable.
We use fail_incapable_cpu() to kill a secondary CPU early during the
bringup, which doesn't have the system advertised capabilities.
This patch makes the routine more generic, to kill a secondary
booting CPU, getting rid of the dependency on capability struct.
This can be used by checks which are not necessarily attached to
a capability struct (e.g, cpu ASIDBits).
In that process, renames the function to cpu_die_early() to better
match its functionality. This will be moved to arch/arm64/kernel/smp.c
later.
Cc: Mark Rutland <mark.rutland@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/kernel/cpufeature.c | 24 +++++++++++++++---------
1 file changed, 15 insertions(+), 9 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index d0c82bc02de4..b7f01bf47988 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -853,15 +853,15 @@ static inline void set_sys_caps_initialised(void)
}
/*
- * Park the CPU which doesn't have the capability as advertised
- * by the system.
+ * Kill the calling secondary CPU, early in bringup before it is turned
+ * online.
*/
-static void fail_incapable_cpu(char *cap_type,
- const struct arm64_cpu_capabilities *cap)
+void cpu_die_early(void)
{
int cpu = smp_processor_id();
- pr_crit("CPU%d: missing %s : %s\n", cpu, cap_type, cap->desc);
+ pr_crit("CPU%d: will not boot\n", cpu);
+
/* Mark this CPU absent */
set_cpu_present(cpu, 0);
@@ -902,8 +902,11 @@ void verify_local_cpu_capabilities(void)
* If the new CPU misses an advertised feature, we cannot proceed
* further, park the cpu.
*/
- if (!feature_matches(__raw_read_system_reg(caps[i].sys_reg), &caps[i]))
- fail_incapable_cpu("arm64_features", &caps[i]);
+ if (!feature_matches(__raw_read_system_reg(caps[i].sys_reg), &caps[i])) {
+ pr_crit("CPU%d: missing feature: %s\n",
+ smp_processor_id(), caps[i].desc);
+ cpu_die_early();
+ }
if (caps[i].enable)
caps[i].enable(NULL);
}
@@ -911,8 +914,11 @@ void verify_local_cpu_capabilities(void)
for (i = 0, caps = arm64_hwcaps; caps[i].matches; i++) {
if (!cpus_have_hwcap(&caps[i]))
continue;
- if (!feature_matches(__raw_read_system_reg(caps[i].sys_reg), &caps[i]))
- fail_incapable_cpu("arm64_hwcaps", &caps[i]);
+ if (!feature_matches(__raw_read_system_reg(caps[i].sys_reg), &caps[i])) {
+ pr_crit("CPU%d: missing HWCAP: %s\n",
+ smp_processor_id(), caps[i].desc);
+ cpu_die_early();
+ }
}
}
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 13/44] arm64: cpufeature: Add scope for capability check
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Suzuki K Poulose <suzuki.poulose@arm.com>
commit 92406f0cc9e3d5cc77bf3de6d68c9c2373dcd701 upstream.
Add scope parameter to the arm64_cpu_capabilities::matches(), so that
this can be reused for checking the capability on a given CPU vs the
system wide. The system uses the default scope associated with the
capability for initialising the CPU_HWCAPs and ELF_HWCAPs.
Cc: James Morse <james.morse@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Andre Przywara <andre.przywara@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
[ v4.4: Changes made according to 4.4 codebase ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/cpufeature.h | 9 ++-
arch/arm64/kernel/cpu_errata.c | 5 +-
arch/arm64/kernel/cpufeature.c | 105 +++++++++++++++-------------
3 files changed, 70 insertions(+), 49 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index ad83c245781c..4c31e14c0f0e 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -74,10 +74,17 @@ struct arm64_ftr_reg {
struct arm64_ftr_bits *ftr_bits;
};
+/* scope of capability check */
+enum {
+ SCOPE_SYSTEM,
+ SCOPE_LOCAL_CPU,
+};
+
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
- bool (*matches)(const struct arm64_cpu_capabilities *);
+ int def_scope; /* default scope */
+ bool (*matches)(const struct arm64_cpu_capabilities *caps, int scope);
int (*enable)(void *); /* Called on all active CPUs */
union {
struct { /* To be used for erratum handling only */
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index a3e846a28b05..0971d80d3623 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -29,10 +29,12 @@
MIDR_ARCHITECTURE_MASK)
static bool __maybe_unused
-is_affected_midr_range(const struct arm64_cpu_capabilities *entry)
+is_affected_midr_range(const struct arm64_cpu_capabilities *entry, int scope)
{
u32 midr = read_cpuid_id();
+ WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
+
if ((midr & CPU_MODEL_MASK) != entry->midr_model)
return false;
@@ -42,6 +44,7 @@ is_affected_midr_range(const struct arm64_cpu_capabilities *entry)
}
#define MIDR_RANGE(model, min, max) \
+ .def_scope = SCOPE_LOCAL_CPU, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = min, \
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index bdb4cd9ffccf..d0c82bc02de4 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -589,6 +589,48 @@ u64 read_system_reg(u32 id)
return regp->sys_val;
}
+/*
+ * __raw_read_system_reg() - Used by a STARTING cpu before cpuinfo is populated.
+ * Read the system register on the current CPU
+ */
+static u64 __raw_read_system_reg(u32 sys_id)
+{
+ switch (sys_id) {
+ case SYS_ID_PFR0_EL1: return (u64)read_cpuid(ID_PFR0_EL1);
+ case SYS_ID_PFR1_EL1: return (u64)read_cpuid(ID_PFR1_EL1);
+ case SYS_ID_DFR0_EL1: return (u64)read_cpuid(ID_DFR0_EL1);
+ case SYS_ID_MMFR0_EL1: return (u64)read_cpuid(ID_MMFR0_EL1);
+ case SYS_ID_MMFR1_EL1: return (u64)read_cpuid(ID_MMFR1_EL1);
+ case SYS_ID_MMFR2_EL1: return (u64)read_cpuid(ID_MMFR2_EL1);
+ case SYS_ID_MMFR3_EL1: return (u64)read_cpuid(ID_MMFR3_EL1);
+ case SYS_ID_ISAR0_EL1: return (u64)read_cpuid(ID_ISAR0_EL1);
+ case SYS_ID_ISAR1_EL1: return (u64)read_cpuid(ID_ISAR1_EL1);
+ case SYS_ID_ISAR2_EL1: return (u64)read_cpuid(ID_ISAR2_EL1);
+ case SYS_ID_ISAR3_EL1: return (u64)read_cpuid(ID_ISAR3_EL1);
+ case SYS_ID_ISAR4_EL1: return (u64)read_cpuid(ID_ISAR4_EL1);
+ case SYS_ID_ISAR5_EL1: return (u64)read_cpuid(ID_ISAR4_EL1);
+ case SYS_MVFR0_EL1: return (u64)read_cpuid(MVFR0_EL1);
+ case SYS_MVFR1_EL1: return (u64)read_cpuid(MVFR1_EL1);
+ case SYS_MVFR2_EL1: return (u64)read_cpuid(MVFR2_EL1);
+
+ case SYS_ID_AA64PFR0_EL1: return (u64)read_cpuid(ID_AA64PFR0_EL1);
+ case SYS_ID_AA64PFR1_EL1: return (u64)read_cpuid(ID_AA64PFR0_EL1);
+ case SYS_ID_AA64DFR0_EL1: return (u64)read_cpuid(ID_AA64DFR0_EL1);
+ case SYS_ID_AA64DFR1_EL1: return (u64)read_cpuid(ID_AA64DFR0_EL1);
+ case SYS_ID_AA64MMFR0_EL1: return (u64)read_cpuid(ID_AA64MMFR0_EL1);
+ case SYS_ID_AA64MMFR1_EL1: return (u64)read_cpuid(ID_AA64MMFR1_EL1);
+ case SYS_ID_AA64ISAR0_EL1: return (u64)read_cpuid(ID_AA64ISAR0_EL1);
+ case SYS_ID_AA64ISAR1_EL1: return (u64)read_cpuid(ID_AA64ISAR1_EL1);
+
+ case SYS_CNTFRQ_EL0: return (u64)read_cpuid(CNTFRQ_EL0);
+ case SYS_CTR_EL0: return (u64)read_cpuid(CTR_EL0);
+ case SYS_DCZID_EL0: return (u64)read_cpuid(DCZID_EL0);
+ default:
+ BUG();
+ return 0;
+ }
+}
+
#include <linux/irqchip/arm-gic-v3.h>
static bool
@@ -600,19 +642,24 @@ feature_matches(u64 reg, const struct arm64_cpu_capabilities *entry)
}
static bool
-has_cpuid_feature(const struct arm64_cpu_capabilities *entry)
+has_cpuid_feature(const struct arm64_cpu_capabilities *entry, int scope)
{
u64 val;
- val = read_system_reg(entry->sys_reg);
+ WARN_ON(scope == SCOPE_LOCAL_CPU && preemptible());
+ if (scope == SCOPE_SYSTEM)
+ val = read_system_reg(entry->sys_reg);
+ else
+ val = __raw_read_system_reg(entry->sys_reg);
+
return feature_matches(val, entry);
}
-static bool has_useable_gicv3_cpuif(const struct arm64_cpu_capabilities *entry)
+static bool has_useable_gicv3_cpuif(const struct arm64_cpu_capabilities *entry, int scope)
{
bool has_sre;
- if (!has_cpuid_feature(entry))
+ if (!has_cpuid_feature(entry, scope))
return false;
has_sre = gic_enable_sre();
@@ -627,6 +674,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "GIC system register CPU interface",
.capability = ARM64_HAS_SYSREG_GIC_CPUIF,
+ .def_scope = SCOPE_SYSTEM,
.matches = has_useable_gicv3_cpuif,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.field_pos = ID_AA64PFR0_GIC_SHIFT,
@@ -636,6 +684,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Privileged Access Never",
.capability = ARM64_HAS_PAN,
+ .def_scope = SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR1_EL1,
.field_pos = ID_AA64MMFR1_PAN_SHIFT,
@@ -647,6 +696,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "LSE atomic instructions",
.capability = ARM64_HAS_LSE_ATOMICS,
+ .def_scope = SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR0_EL1,
.field_pos = ID_AA64ISAR0_ATOMICS_SHIFT,
@@ -656,6 +706,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "32-bit EL0 Support",
.capability = ARM64_HAS_32BIT_EL0,
+ .def_scope = SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.field_pos = ID_AA64PFR0_EL0_SHIFT,
@@ -667,6 +718,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#define HWCAP_CAP(reg, field, min_value, type, cap) \
{ \
.desc = #cap, \
+ .def_scope = SCOPE_SYSTEM, \
.matches = has_cpuid_feature, \
.sys_reg = reg, \
.field_pos = field, \
@@ -745,7 +797,7 @@ static void setup_cpu_hwcaps(void)
const struct arm64_cpu_capabilities *hwcaps = arm64_hwcaps;
for (i = 0; hwcaps[i].matches; i++)
- if (hwcaps[i].matches(&hwcaps[i]))
+ if (hwcaps[i].matches(&hwcaps[i], hwcaps[i].def_scope))
cap_set_hwcap(&hwcaps[i]);
}
@@ -755,7 +807,7 @@ void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
int i;
for (i = 0; caps[i].matches; i++) {
- if (!caps[i].matches(&caps[i]))
+ if (!caps[i].matches(&caps[i], caps[i].def_scope))
continue;
if (!cpus_have_cap(caps[i].capability) && caps[i].desc)
@@ -800,47 +852,6 @@ static inline void set_sys_caps_initialised(void)
sys_caps_initialised = true;
}
-/*
- * __raw_read_system_reg() - Used by a STARTING cpu before cpuinfo is populated.
- */
-static u64 __raw_read_system_reg(u32 sys_id)
-{
- switch (sys_id) {
- case SYS_ID_PFR0_EL1: return (u64)read_cpuid(ID_PFR0_EL1);
- case SYS_ID_PFR1_EL1: return (u64)read_cpuid(ID_PFR1_EL1);
- case SYS_ID_DFR0_EL1: return (u64)read_cpuid(ID_DFR0_EL1);
- case SYS_ID_MMFR0_EL1: return (u64)read_cpuid(ID_MMFR0_EL1);
- case SYS_ID_MMFR1_EL1: return (u64)read_cpuid(ID_MMFR1_EL1);
- case SYS_ID_MMFR2_EL1: return (u64)read_cpuid(ID_MMFR2_EL1);
- case SYS_ID_MMFR3_EL1: return (u64)read_cpuid(ID_MMFR3_EL1);
- case SYS_ID_ISAR0_EL1: return (u64)read_cpuid(ID_ISAR0_EL1);
- case SYS_ID_ISAR1_EL1: return (u64)read_cpuid(ID_ISAR1_EL1);
- case SYS_ID_ISAR2_EL1: return (u64)read_cpuid(ID_ISAR2_EL1);
- case SYS_ID_ISAR3_EL1: return (u64)read_cpuid(ID_ISAR3_EL1);
- case SYS_ID_ISAR4_EL1: return (u64)read_cpuid(ID_ISAR4_EL1);
- case SYS_ID_ISAR5_EL1: return (u64)read_cpuid(ID_ISAR4_EL1);
- case SYS_MVFR0_EL1: return (u64)read_cpuid(MVFR0_EL1);
- case SYS_MVFR1_EL1: return (u64)read_cpuid(MVFR1_EL1);
- case SYS_MVFR2_EL1: return (u64)read_cpuid(MVFR2_EL1);
-
- case SYS_ID_AA64PFR0_EL1: return (u64)read_cpuid(ID_AA64PFR0_EL1);
- case SYS_ID_AA64PFR1_EL1: return (u64)read_cpuid(ID_AA64PFR0_EL1);
- case SYS_ID_AA64DFR0_EL1: return (u64)read_cpuid(ID_AA64DFR0_EL1);
- case SYS_ID_AA64DFR1_EL1: return (u64)read_cpuid(ID_AA64DFR0_EL1);
- case SYS_ID_AA64MMFR0_EL1: return (u64)read_cpuid(ID_AA64MMFR0_EL1);
- case SYS_ID_AA64MMFR1_EL1: return (u64)read_cpuid(ID_AA64MMFR1_EL1);
- case SYS_ID_AA64ISAR0_EL1: return (u64)read_cpuid(ID_AA64ISAR0_EL1);
- case SYS_ID_AA64ISAR1_EL1: return (u64)read_cpuid(ID_AA64ISAR1_EL1);
-
- case SYS_CNTFRQ_EL0: return (u64)read_cpuid(CNTFRQ_EL0);
- case SYS_CTR_EL0: return (u64)read_cpuid(CTR_EL0);
- case SYS_DCZID_EL0: return (u64)read_cpuid(DCZID_EL0);
- default:
- BUG();
- return 0;
- }
-}
-
/*
* Park the CPU which doesn't have the capability as advertised
* by the system.
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 12/44] arm64: cpufeature: Test 'matches' pointer to find the end of the list
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: James Morse <james.morse@arm.com>
commit 644c2ae198412c956700e55a2acf80b2541f6aa5 upstream.
CPU feature code uses the desc field as a test to find the end of the list,
this means every entry must have a description. This generates noise for
entries in the list that aren't really features, but combinations of them.
e.g.
> CPU features: detected feature: Privileged Access Never
> CPU features: detected feature: PAN and not UAO
These combination features are needed for corner cases with alternatives,
where cpu features interact.
Change all walkers of the arm64_features[] and arm64_hwcaps[] lists to test
'matches' not 'desc', and only print 'desc' if it is non-NULL.
Signed-off-by: James Morse <james.morse@arm.com>
Reviewed-by : Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/kernel/cpufeature.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index c1eddc07d996..bdb4cd9ffccf 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -744,7 +744,7 @@ static void setup_cpu_hwcaps(void)
int i;
const struct arm64_cpu_capabilities *hwcaps = arm64_hwcaps;
- for (i = 0; hwcaps[i].desc; i++)
+ for (i = 0; hwcaps[i].matches; i++)
if (hwcaps[i].matches(&hwcaps[i]))
cap_set_hwcap(&hwcaps[i]);
}
@@ -754,11 +754,11 @@ void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
{
int i;
- for (i = 0; caps[i].desc; i++) {
+ for (i = 0; caps[i].matches; i++) {
if (!caps[i].matches(&caps[i]))
continue;
- if (!cpus_have_cap(caps[i].capability))
+ if (!cpus_have_cap(caps[i].capability) && caps[i].desc)
pr_info("%s %s\n", info, caps[i].desc);
cpus_set_cap(caps[i].capability);
}
@@ -772,7 +772,7 @@ static void enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
{
int i;
- for (i = 0; caps[i].desc; i++)
+ for (i = 0; caps[i].matches; i++)
if (caps[i].enable && cpus_have_cap(caps[i].capability))
/*
* Use stop_machine() as it schedules the work allowing
@@ -884,7 +884,7 @@ void verify_local_cpu_capabilities(void)
return;
caps = arm64_features;
- for (i = 0; caps[i].desc; i++) {
+ for (i = 0; caps[i].matches; i++) {
if (!cpus_have_cap(caps[i].capability) || !caps[i].sys_reg)
continue;
/*
@@ -897,7 +897,7 @@ void verify_local_cpu_capabilities(void)
caps[i].enable(NULL);
}
- for (i = 0, caps = arm64_hwcaps; caps[i].desc; i++) {
+ for (i = 0, caps = arm64_hwcaps; caps[i].matches; i++) {
if (!cpus_have_hwcap(&caps[i]))
continue;
if (!feature_matches(__raw_read_system_reg(caps[i].sys_reg), &caps[i]))
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 11/44] arm64: uaccess: Mask __user pointers for __arch_{clear, copy_*}_user
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Will Deacon <will.deacon@arm.com>
commit f71c2ffcb20dd8626880747557014bb9a61eb90e upstream.
Like we've done for get_user and put_user, ensure that user pointers
are masked before invoking the underlying __arch_{clear,copy_*}_user
operations.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
[ v4.4: fixup for v4.4 style uaccess primitives ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/uaccess.h | 18 ++++++++++--------
arch/arm64/kernel/arm64ksyms.c | 4 ++--
arch/arm64/lib/clear_user.S | 6 +++---
arch/arm64/lib/copy_in_user.S | 4 ++--
4 files changed, 17 insertions(+), 15 deletions(-)
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index 693a0d784534..f2f5a152f372 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -303,19 +303,18 @@ do { \
extern unsigned long __must_check __arch_copy_from_user(void *to, const void __user *from, unsigned long n);
extern unsigned long __must_check __arch_copy_to_user(void __user *to, const void *from, unsigned long n);
-extern unsigned long __must_check __copy_in_user(void __user *to, const void __user *from, unsigned long n);
-extern unsigned long __must_check __clear_user(void __user *addr, unsigned long n);
+extern unsigned long __must_check __arch_copy_in_user(void __user *to, const void __user *from, unsigned long n);
static inline unsigned long __must_check __copy_from_user(void *to, const void __user *from, unsigned long n)
{
kasan_check_write(to, n);
- return __arch_copy_from_user(to, from, n);
+ return __arch_copy_from_user(to, __uaccess_mask_ptr(from), n);
}
static inline unsigned long __must_check __copy_to_user(void __user *to, const void *from, unsigned long n)
{
kasan_check_read(from, n);
- return __arch_copy_to_user(to, from, n);
+ return __arch_copy_to_user(__uaccess_mask_ptr(to), from, n);
}
static inline unsigned long __must_check copy_from_user(void *to, const void __user *from, unsigned long n)
@@ -338,22 +337,25 @@ static inline unsigned long __must_check copy_to_user(void __user *to, const voi
return n;
}
-static inline unsigned long __must_check copy_in_user(void __user *to, const void __user *from, unsigned long n)
+static inline unsigned long __must_check __copy_in_user(void __user *to, const void __user *from, unsigned long n)
{
if (access_ok(VERIFY_READ, from, n) && access_ok(VERIFY_WRITE, to, n))
- n = __copy_in_user(to, from, n);
+ n = __arch_copy_in_user(__uaccess_mask_ptr(to), __uaccess_mask_ptr(from), n);
return n;
}
+#define copy_in_user __copy_in_user
#define __copy_to_user_inatomic __copy_to_user
#define __copy_from_user_inatomic __copy_from_user
-static inline unsigned long __must_check clear_user(void __user *to, unsigned long n)
+extern unsigned long __must_check __arch_clear_user(void __user *to, unsigned long n);
+static inline unsigned long __must_check __clear_user(void __user *to, unsigned long n)
{
if (access_ok(VERIFY_WRITE, to, n))
- n = __clear_user(__uaccess_mask_ptr(to), n);
+ n = __arch_clear_user(__uaccess_mask_ptr(to), n);
return n;
}
+#define clear_user __clear_user
extern long strncpy_from_user(char *dest, const char __user *src, long count);
diff --git a/arch/arm64/kernel/arm64ksyms.c b/arch/arm64/kernel/arm64ksyms.c
index c654df05b7d7..abe4e0984dbb 100644
--- a/arch/arm64/kernel/arm64ksyms.c
+++ b/arch/arm64/kernel/arm64ksyms.c
@@ -35,8 +35,8 @@ EXPORT_SYMBOL(clear_page);
/* user mem (segment) */
EXPORT_SYMBOL(__arch_copy_from_user);
EXPORT_SYMBOL(__arch_copy_to_user);
-EXPORT_SYMBOL(__clear_user);
-EXPORT_SYMBOL(__copy_in_user);
+EXPORT_SYMBOL(__arch_clear_user);
+EXPORT_SYMBOL(__arch_copy_in_user);
/* physical memory */
EXPORT_SYMBOL(memstart_addr);
diff --git a/arch/arm64/lib/clear_user.S b/arch/arm64/lib/clear_user.S
index a9723c71c52b..fc6bb0f83511 100644
--- a/arch/arm64/lib/clear_user.S
+++ b/arch/arm64/lib/clear_user.S
@@ -24,7 +24,7 @@
.text
-/* Prototype: int __clear_user(void *addr, size_t sz)
+/* Prototype: int __arch_clear_user(void *addr, size_t sz)
* Purpose : clear some user memory
* Params : addr - user memory address to clear
* : sz - number of bytes to clear
@@ -32,7 +32,7 @@
*
* Alignment fixed up by hardware.
*/
-ENTRY(__clear_user)
+ENTRY(__arch_clear_user)
ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_HAS_PAN, \
CONFIG_ARM64_PAN)
mov x2, x1 // save the size for fixup return
@@ -57,7 +57,7 @@ USER(9f, strb wzr, [x0] )
ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_HAS_PAN, \
CONFIG_ARM64_PAN)
ret
-ENDPROC(__clear_user)
+ENDPROC(__arch_clear_user)
.section .fixup,"ax"
.align 2
diff --git a/arch/arm64/lib/copy_in_user.S b/arch/arm64/lib/copy_in_user.S
index 81c8fc93c100..0219aa85b3cc 100644
--- a/arch/arm64/lib/copy_in_user.S
+++ b/arch/arm64/lib/copy_in_user.S
@@ -67,7 +67,7 @@
.endm
end .req x5
-ENTRY(__copy_in_user)
+ENTRY(__arch_copy_in_user)
ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_HAS_PAN, \
CONFIG_ARM64_PAN)
add end, x0, x2
@@ -76,7 +76,7 @@ ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_HAS_PAN, \
CONFIG_ARM64_PAN)
mov x0, #0
ret
-ENDPROC(__copy_in_user)
+ENDPROC(__arch_copy_in_user)
.section .fixup,"ax"
.align 2
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 10/44] arm64: kasan: instrument user memory access API
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Yang Shi <yang.shi@linaro.org>
commit bffe1baff5d57521b0c41b6997c41ff1993e9818 upstream.
The upstream commit 1771c6e1a567ea0ba2cccc0a4ffe68a1419fd8ef
("x86/kasan: instrument user memory access API") added KASAN instrument to
x86 user memory access API, so added such instrument to ARM64 too.
Define __copy_to/from_user in C in order to add kasan_check_read/write call,
rename assembly implementation to __arch_copy_to/from_user.
Tested by test_kasan module.
Acked-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Tested-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Yang Shi <yang.shi@linaro.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/uaccess.h | 25 +++++++++++++++++++++----
arch/arm64/kernel/arm64ksyms.c | 4 ++--
arch/arm64/lib/copy_from_user.S | 4 ++--
arch/arm64/lib/copy_to_user.S | 4 ++--
4 files changed, 27 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index a34324436ce1..693a0d784534 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -22,6 +22,7 @@
* User space memory access functions
*/
#include <linux/bitops.h>
+#include <linux/kasan-checks.h>
#include <linux/string.h>
#include <linux/thread_info.h>
@@ -300,15 +301,29 @@ do { \
#define put_user __put_user
-extern unsigned long __must_check __copy_from_user(void *to, const void __user *from, unsigned long n);
-extern unsigned long __must_check __copy_to_user(void __user *to, const void *from, unsigned long n);
+extern unsigned long __must_check __arch_copy_from_user(void *to, const void __user *from, unsigned long n);
+extern unsigned long __must_check __arch_copy_to_user(void __user *to, const void *from, unsigned long n);
extern unsigned long __must_check __copy_in_user(void __user *to, const void __user *from, unsigned long n);
extern unsigned long __must_check __clear_user(void __user *addr, unsigned long n);
+static inline unsigned long __must_check __copy_from_user(void *to, const void __user *from, unsigned long n)
+{
+ kasan_check_write(to, n);
+ return __arch_copy_from_user(to, from, n);
+}
+
+static inline unsigned long __must_check __copy_to_user(void __user *to, const void *from, unsigned long n)
+{
+ kasan_check_read(from, n);
+ return __arch_copy_to_user(to, from, n);
+}
+
static inline unsigned long __must_check copy_from_user(void *to, const void __user *from, unsigned long n)
{
+ kasan_check_write(to, n);
+
if (access_ok(VERIFY_READ, from, n))
- n = __copy_from_user(to, from, n);
+ n = __arch_copy_from_user(to, from, n);
else /* security hole - plug it */
memset(to, 0, n);
return n;
@@ -316,8 +331,10 @@ static inline unsigned long __must_check copy_from_user(void *to, const void __u
static inline unsigned long __must_check copy_to_user(void __user *to, const void *from, unsigned long n)
{
+ kasan_check_read(from, n);
+
if (access_ok(VERIFY_WRITE, to, n))
- n = __copy_to_user(to, from, n);
+ n = __arch_copy_to_user(to, from, n);
return n;
}
diff --git a/arch/arm64/kernel/arm64ksyms.c b/arch/arm64/kernel/arm64ksyms.c
index 3b6d8cc9dfe0..c654df05b7d7 100644
--- a/arch/arm64/kernel/arm64ksyms.c
+++ b/arch/arm64/kernel/arm64ksyms.c
@@ -33,8 +33,8 @@ EXPORT_SYMBOL(copy_page);
EXPORT_SYMBOL(clear_page);
/* user mem (segment) */
-EXPORT_SYMBOL(__copy_from_user);
-EXPORT_SYMBOL(__copy_to_user);
+EXPORT_SYMBOL(__arch_copy_from_user);
+EXPORT_SYMBOL(__arch_copy_to_user);
EXPORT_SYMBOL(__clear_user);
EXPORT_SYMBOL(__copy_in_user);
diff --git a/arch/arm64/lib/copy_from_user.S b/arch/arm64/lib/copy_from_user.S
index 4699cd74f87e..281e75db899a 100644
--- a/arch/arm64/lib/copy_from_user.S
+++ b/arch/arm64/lib/copy_from_user.S
@@ -66,7 +66,7 @@
.endm
end .req x5
-ENTRY(__copy_from_user)
+ENTRY(__arch_copy_from_user)
ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_HAS_PAN, \
CONFIG_ARM64_PAN)
add end, x0, x2
@@ -75,7 +75,7 @@ ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_HAS_PAN, \
CONFIG_ARM64_PAN)
mov x0, #0 // Nothing to copy
ret
-ENDPROC(__copy_from_user)
+ENDPROC(__arch_copy_from_user)
.section .fixup,"ax"
.align 2
diff --git a/arch/arm64/lib/copy_to_user.S b/arch/arm64/lib/copy_to_user.S
index 7512bbbc07ac..db4d187de61f 100644
--- a/arch/arm64/lib/copy_to_user.S
+++ b/arch/arm64/lib/copy_to_user.S
@@ -65,7 +65,7 @@
.endm
end .req x5
-ENTRY(__copy_to_user)
+ENTRY(__arch_copy_to_user)
ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_HAS_PAN, \
CONFIG_ARM64_PAN)
add end, x0, x2
@@ -74,7 +74,7 @@ ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_HAS_PAN, \
CONFIG_ARM64_PAN)
mov x0, #0
ret
-ENDPROC(__copy_to_user)
+ENDPROC(__arch_copy_to_user)
.section .fixup,"ax"
.align 2
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 09/44] mm/kasan: add API to check memory regions
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Andrey Ryabinin <aryabinin@virtuozzo.com>
commit 64f8ebaf115bcddc4aaa902f981c57ba6506bc42 upstream.
Memory access coded in an assembly won't be seen by KASAN as a compiler
can instrument only C code. Add kasan_check_[read,write]() API which is
going to be used to check a certain memory range.
Link: http://lkml.kernel.org/r/1462538722-1574-3-git-send-email-aryabinin@virtuozzo.com
Signed-off-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Acked-by: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
[ v4.4: Fixed MAINTAINERS conflict and added whole kasan entry. Drop 4th
argument to check_memory_region(). ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
MAINTAINERS | 14 ++++++++++++++
include/linux/kasan-checks.h | 12 ++++++++++++
mm/kasan/kasan.c | 12 ++++++++++++
3 files changed, 38 insertions(+)
create mode 100644 include/linux/kasan-checks.h
diff --git a/MAINTAINERS b/MAINTAINERS
index f4d4a5544dc1..2a8826732967 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5982,6 +5982,20 @@ S: Maintained
F: Documentation/hwmon/k8temp
F: drivers/hwmon/k8temp.c
+KASAN
+M: Andrey Ryabinin <aryabinin@virtuozzo.com>
+R: Alexander Potapenko <glider@google.com>
+R: Dmitry Vyukov <dvyukov@google.com>
+L: kasan-dev@googlegroups.com
+S: Maintained
+F: arch/*/include/asm/kasan.h
+F: arch/*/mm/kasan_init*
+F: Documentation/kasan.txt
+F: include/linux/kasan*.h
+F: lib/test_kasan.c
+F: mm/kasan/
+F: scripts/Makefile.kasan
+
KCONFIG
M: "Yann E. MORIN" <yann.morin.1998@free.fr>
L: linux-kbuild@vger.kernel.org
diff --git a/include/linux/kasan-checks.h b/include/linux/kasan-checks.h
new file mode 100644
index 000000000000..b7f8aced7870
--- /dev/null
+++ b/include/linux/kasan-checks.h
@@ -0,0 +1,12 @@
+#ifndef _LINUX_KASAN_CHECKS_H
+#define _LINUX_KASAN_CHECKS_H
+
+#ifdef CONFIG_KASAN
+void kasan_check_read(const void *p, unsigned int size);
+void kasan_check_write(const void *p, unsigned int size);
+#else
+static inline void kasan_check_read(const void *p, unsigned int size) { }
+static inline void kasan_check_write(const void *p, unsigned int size) { }
+#endif
+
+#endif
diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c
index b7397b459960..1cdcab0c976a 100644
--- a/mm/kasan/kasan.c
+++ b/mm/kasan/kasan.c
@@ -274,6 +274,18 @@ static __always_inline void check_memory_region(unsigned long addr,
void __asan_loadN(unsigned long addr, size_t size);
void __asan_storeN(unsigned long addr, size_t size);
+void kasan_check_read(const void *p, unsigned int size)
+{
+ check_memory_region((unsigned long)p, size, false);
+}
+EXPORT_SYMBOL(kasan_check_read);
+
+void kasan_check_write(const void *p, unsigned int size)
+{
+ check_memory_region((unsigned long)p, size, true);
+}
+EXPORT_SYMBOL(kasan_check_write);
+
#undef memset
void *memset(void *addr, int c, size_t len)
{
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 08/44] arm64: uaccess: Don't bother eliding access_ok checks in __{get, put}_user
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Will Deacon <will.deacon@arm.com>
commit 84624087dd7e3b482b7b11c170ebc1f329b3a218 upstream.
access_ok isn't an expensive operation once the addr_limit for the current
thread has been loaded into the cache. Given that the initial access_ok
check preceding a sequence of __{get,put}_user operations will take
the brunt of the miss, we can make the __* variants identical to the
full-fat versions, which brings with it the benefits of address masking.
The likely cost in these sequences will be from toggling PAN/UAO, which
we can address later by implementing the *_unsafe versions.
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
[ v4.4: Fixed conflicts around {__get_user|__put_user}_unaligned macros ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/uaccess.h | 62 ++++++++++++++++++--------------
1 file changed, 36 insertions(+), 26 deletions(-)
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index fc11c50af558..a34324436ce1 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -200,30 +200,35 @@ do { \
CONFIG_ARM64_PAN)); \
} while (0)
-#define __get_user(x, ptr) \
+#define __get_user_check(x, ptr, err) \
({ \
- int __gu_err = 0; \
- __get_user_err((x), (ptr), __gu_err); \
- __gu_err; \
+ __typeof__(*(ptr)) __user *__p = (ptr); \
+ might_fault(); \
+ if (access_ok(VERIFY_READ, __p, sizeof(*__p))) { \
+ __p = uaccess_mask_ptr(__p); \
+ __get_user_err((x), __p, (err)); \
+ } else { \
+ (x) = 0; (err) = -EFAULT; \
+ } \
})
#define __get_user_error(x, ptr, err) \
({ \
- __get_user_err((x), (ptr), (err)); \
+ __get_user_check((x), (ptr), (err)); \
(void)0; \
})
-#define __get_user_unaligned __get_user
-
-#define get_user(x, ptr) \
+#define __get_user(x, ptr) \
({ \
- __typeof__(*(ptr)) __user *__p = (ptr); \
- might_fault(); \
- access_ok(VERIFY_READ, __p, sizeof(*__p)) ? \
- __p = uaccess_mask_ptr(__p), __get_user((x), __p) : \
- ((x) = 0, -EFAULT); \
+ int __gu_err = 0; \
+ __get_user_check((x), (ptr), __gu_err); \
+ __gu_err; \
})
+#define __get_user_unaligned __get_user
+
+#define get_user __get_user
+
#define __put_user_asm(instr, reg, x, addr, err) \
asm volatile( \
"1: " instr " " reg "1, [%2]\n" \
@@ -266,30 +271,35 @@ do { \
CONFIG_ARM64_PAN)); \
} while (0)
-#define __put_user(x, ptr) \
+#define __put_user_check(x, ptr, err) \
({ \
- int __pu_err = 0; \
- __put_user_err((x), (ptr), __pu_err); \
- __pu_err; \
+ __typeof__(*(ptr)) __user *__p = (ptr); \
+ might_fault(); \
+ if (access_ok(VERIFY_WRITE, __p, sizeof(*__p))) { \
+ __p = uaccess_mask_ptr(__p); \
+ __put_user_err((x), __p, (err)); \
+ } else { \
+ (err) = -EFAULT; \
+ } \
})
#define __put_user_error(x, ptr, err) \
({ \
- __put_user_err((x), (ptr), (err)); \
+ __put_user_check((x), (ptr), (err)); \
(void)0; \
})
-#define __put_user_unaligned __put_user
-
-#define put_user(x, ptr) \
+#define __put_user(x, ptr) \
({ \
- __typeof__(*(ptr)) __user *__p = (ptr); \
- might_fault(); \
- access_ok(VERIFY_WRITE, __p, sizeof(*__p)) ? \
- __p = uaccess_mask_ptr(__p), __put_user((x), __p) : \
- -EFAULT; \
+ int __pu_err = 0; \
+ __put_user_check((x), (ptr), __pu_err); \
+ __pu_err; \
})
+#define __put_user_unaligned __put_user
+
+#define put_user __put_user
+
extern unsigned long __must_check __copy_from_user(void *to, const void __user *from, unsigned long n);
extern unsigned long __must_check __copy_to_user(void __user *to, const void *from, unsigned long n);
extern unsigned long __must_check __copy_in_user(void __user *to, const void __user *from, unsigned long n);
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 07/44] arm64: uaccess: Prevent speculative use of the current addr_limit
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Will Deacon <will.deacon@arm.com>
commit c2f0ad4fc089cff81cef6a13d04b399980ecbfcc upstream.
A mispredicted conditional call to set_fs could result in the wrong
addr_limit being forwarded under speculation to a subsequent access_ok
check, potentially forming part of a spectre-v1 attack using uaccess
routines.
This patch prevents this forwarding from taking place, but putting heavy
barriers in set_fs after writing the addr_limit.
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/uaccess.h | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index 75363d723262..fc11c50af558 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -62,6 +62,13 @@ extern int fixup_exception(struct pt_regs *regs);
static inline void set_fs(mm_segment_t fs)
{
current_thread_info()->addr_limit = fs;
+
+ /*
+ * Prevent a mispredicted conditional call to set_fs from forwarding
+ * the wrong address limit to access_ok under speculation.
+ */
+ dsb(nsh);
+ isb();
}
#define segment_eq(a, b) ((a) == (b))
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 06/44] arm64: entry: Ensure branch through syscall table is bounded under speculation
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Will Deacon <will.deacon@arm.com>
commit 6314d90e64936c584f300a52ef173603fb2461b5 upstream.
In a similar manner to array_index_mask_nospec, this patch introduces an
assembly macro (mask_nospec64) which can be used to bound a value under
speculation. This macro is then used to ensure that the indirect branch
through the syscall table is bounded under speculation, with out-of-range
addresses speculating as calls to sys_io_setup (0).
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
[ v4.4: use existing scno & sc_nr definitions ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/assembler.h | 11 +++++++++++
arch/arm64/kernel/entry.S | 1 +
2 files changed, 12 insertions(+)
diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index 683c2875278f..2b30363a3a89 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -102,6 +102,17 @@
hint #20
.endm
+/*
+ * Sanitise a 64-bit bounded index wrt speculation, returning zero if out
+ * of bounds.
+ */
+ .macro mask_nospec64, idx, limit, tmp
+ sub \tmp, \idx, \limit
+ bic \tmp, \tmp, \idx
+ and \idx, \idx, \tmp, asr #63
+ csdb
+ .endm
+
#define USER(l, x...) \
9999: x; \
.section __ex_table,"a"; \
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 4c5013b09dcb..e6aec982dea9 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -697,6 +697,7 @@ el0_svc_naked: // compat entry point
b.ne __sys_trace
cmp scno, sc_nr // check upper syscall limit
b.hs ni_sys
+ mask_nospec64 scno, sc_nr, x19 // enforce bounds for syscall number
ldr x16, [stbl, scno, lsl #3] // address in the syscall table
blr x16 // call sys_* routine
b ret_fast_syscall
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 05/44] arm64: Use pointer masking to limit uaccess speculation
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Robin Murphy <robin.murphy@arm.com>
commit 4d8efc2d5ee4c9ccfeb29ee8afd47a8660d0c0ce upstream.
Similarly to x86, mitigate speculation past an access_ok() check by
masking the pointer against the address limit before use.
Even if we don't expect speculative writes per se, it is plausible that
a CPU may still speculate at least as far as fetching a cache line for
writing, hence we also harden put_user() and clear_user() for peace of
mind.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/uaccess.h | 26 +++++++++++++++++++++++---
1 file changed, 23 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index c625cc5531fc..75363d723262 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -121,6 +121,26 @@ static inline unsigned long __range_ok(unsigned long addr, unsigned long size)
#define access_ok(type, addr, size) __range_ok((unsigned long)(addr), size)
#define user_addr_max get_fs
+/*
+ * Sanitise a uaccess pointer such that it becomes NULL if above the
+ * current addr_limit.
+ */
+#define uaccess_mask_ptr(ptr) (__typeof__(ptr))__uaccess_mask_ptr(ptr)
+static inline void __user *__uaccess_mask_ptr(const void __user *ptr)
+{
+ void __user *safe_ptr;
+
+ asm volatile(
+ " bics xzr, %1, %2\n"
+ " csel %0, %1, xzr, eq\n"
+ : "=&r" (safe_ptr)
+ : "r" (ptr), "r" (current_thread_info()->addr_limit)
+ : "cc");
+
+ csdb();
+ return safe_ptr;
+}
+
/*
* The "__xxx" versions of the user access functions do not verify the address
* space - it must have been done previously with a separate "access_ok()"
@@ -193,7 +213,7 @@ do { \
__typeof__(*(ptr)) __user *__p = (ptr); \
might_fault(); \
access_ok(VERIFY_READ, __p, sizeof(*__p)) ? \
- __get_user((x), __p) : \
+ __p = uaccess_mask_ptr(__p), __get_user((x), __p) : \
((x) = 0, -EFAULT); \
})
@@ -259,7 +279,7 @@ do { \
__typeof__(*(ptr)) __user *__p = (ptr); \
might_fault(); \
access_ok(VERIFY_WRITE, __p, sizeof(*__p)) ? \
- __put_user((x), __p) : \
+ __p = uaccess_mask_ptr(__p), __put_user((x), __p) : \
-EFAULT; \
})
@@ -297,7 +317,7 @@ static inline unsigned long __must_check copy_in_user(void __user *to, const voi
static inline unsigned long __must_check clear_user(void __user *to, unsigned long n)
{
if (access_ok(VERIFY_WRITE, to, n))
- n = __clear_user(to, n);
+ n = __clear_user(__uaccess_mask_ptr(to), n);
return n;
}
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 04/44] arm64: Make USER_DS an inclusive limit
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Robin Murphy <robin.murphy@arm.com>
commit 51369e398d0d33e8f524314e672b07e8cf870e79 upstream.
Currently, USER_DS represents an exclusive limit while KERNEL_DS is
inclusive. In order to do some clever trickery for speculation-safe
masking, we need them both to behave equivalently - there aren't enough
bits to make KERNEL_DS exclusive, so we have precisely one option. This
also happens to correct a longstanding false negative for a range
ending on the very top byte of kernel memory.
Mark Rutland points out that we've actually got the semantics of
addresses vs. segments muddled up in most of the places we need to
amend, so shuffle the {USER,KERNEL}_DS definitions around such that we
can correct those properly instead of just pasting "-1"s everywhere.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
[ 4.4: Dropped changes from fault.c and fixed minor rebase conflict ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/processor.h | 3 ++
arch/arm64/include/asm/uaccess.h | 45 +++++++++++++++++-------------
arch/arm64/kernel/entry.S | 4 +--
3 files changed, 31 insertions(+), 21 deletions(-)
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 75d9ef6c457c..ff1449c25bf4 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -21,6 +21,9 @@
#define TASK_SIZE_64 (UL(1) << VA_BITS)
+#define KERNEL_DS UL(-1)
+#define USER_DS (TASK_SIZE_64 - 1)
+
#ifndef __ASSEMBLY__
/*
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index 829fa6d3e561..c625cc5531fc 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -56,10 +56,7 @@ struct exception_table_entry
extern int fixup_exception(struct pt_regs *regs);
-#define KERNEL_DS (-1UL)
#define get_ds() (KERNEL_DS)
-
-#define USER_DS TASK_SIZE_64
#define get_fs() (current_thread_info()->addr_limit)
static inline void set_fs(mm_segment_t fs)
@@ -87,22 +84,32 @@ static inline void set_fs(mm_segment_t fs)
* Returns 1 if the range is valid, 0 otherwise.
*
* This is equivalent to the following test:
- * (u65)addr + (u65)size <= current->addr_limit
- *
- * This needs 65-bit arithmetic.
+ * (u65)addr + (u65)size <= (u65)current->addr_limit + 1
*/
-#define __range_ok(addr, size) \
-({ \
- unsigned long __addr = (unsigned long __force)(addr); \
- unsigned long flag, roksum; \
- __chk_user_ptr(addr); \
- asm("adds %1, %1, %3; ccmp %1, %4, #2, cc; cset %0, ls" \
- : "=&r" (flag), "=&r" (roksum) \
- : "1" (__addr), "Ir" (size), \
- "r" (current_thread_info()->addr_limit) \
- : "cc"); \
- flag; \
-})
+static inline unsigned long __range_ok(unsigned long addr, unsigned long size)
+{
+ unsigned long limit = current_thread_info()->addr_limit;
+
+ __chk_user_ptr(addr);
+ asm volatile(
+ // A + B <= C + 1 for all A,B,C, in four easy steps:
+ // 1: X = A + B; X' = X % 2^64
+ " adds %0, %0, %2\n"
+ // 2: Set C = 0 if X > 2^64, to guarantee X' > C in step 4
+ " csel %1, xzr, %1, hi\n"
+ // 3: Set X' = ~0 if X >= 2^64. For X == 2^64, this decrements X'
+ // to compensate for the carry flag being set in step 4. For
+ // X > 2^64, X' merely has to remain nonzero, which it does.
+ " csinv %0, %0, xzr, cc\n"
+ // 4: For X < 2^64, this gives us X' - C - 1 <= 0, where the -1
+ // comes from the carry in being clear. Otherwise, we are
+ // testing X' - C == 0, subject to the previous adjustments.
+ " sbcs xzr, %0, %1\n"
+ " cset %0, ls\n"
+ : "+r" (addr), "+r" (limit) : "Ir" (size) : "cc");
+
+ return addr;
+}
/*
* When dealing with data aborts, watchpoints, or instruction traps we may end
@@ -111,7 +118,7 @@ static inline void set_fs(mm_segment_t fs)
*/
#define untagged_addr(addr) sign_extend64(addr, 55)
-#define access_ok(type, addr, size) __range_ok(addr, size)
+#define access_ok(type, addr, size) __range_ok((unsigned long)(addr), size)
#define user_addr_max get_fs
/*
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index c849be9231bb..4c5013b09dcb 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -96,10 +96,10 @@
.else
add x21, sp, #S_FRAME_SIZE
get_thread_info tsk
- /* Save the task's original addr_limit and set USER_DS (TASK_SIZE_64) */
+ /* Save the task's original addr_limit and set USER_DS */
ldr x20, [tsk, #TI_ADDR_LIMIT]
str x20, [sp, #S_ORIG_ADDR_LIMIT]
- mov x20, #TASK_SIZE_64
+ mov x20, #USER_DS
str x20, [tsk, #TI_ADDR_LIMIT]
.endif /* \el == 0 */
mrs x22, elr_el1
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 03/44] arm64: move TASK_* definitions to <asm/processor.h>
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Yury Norov <ynorov@caviumnetworks.com>
commit eef94a3d09aab437c8c254de942d8b1aa76455e2 upstream.
ILP32 series [1] introduces the dependency on <asm/is_compat.h> for
TASK_SIZE macro. Which in turn requires <asm/thread_info.h>, and
<asm/thread_info.h> include <asm/memory.h>, giving a circular dependency,
because TASK_SIZE is currently located in <asm/memory.h>.
In other architectures, TASK_SIZE is defined in <asm/processor.h>, and
moving TASK_SIZE there fixes the problem.
Discussion: https://patchwork.kernel.org/patch/9929107/
[1] https://github.com/norov/linux/tree/ilp32-next
CC: Will Deacon <will.deacon@arm.com>
CC: Laura Abbott <labbott@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Suggested-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Yury Norov <ynorov@caviumnetworks.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/memory.h | 15 ---------------
arch/arm64/include/asm/processor.h | 21 +++++++++++++++++++++
arch/arm64/kernel/entry.S | 2 +-
3 files changed, 22 insertions(+), 16 deletions(-)
diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index b42b930cc19a..959a1e9188fe 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -43,8 +43,6 @@
* (VA_BITS - 1))
* VA_BITS - the maximum number of bits for virtual addresses.
* VA_START - the first kernel virtual address.
- * TASK_SIZE - the maximum size of a user space task.
- * TASK_UNMAPPED_BASE - the lower boundary of the mmap VM area.
* The module space lives between the addresses given by TASK_SIZE
* and PAGE_OFFSET - it must be within 128MB of the kernel text.
*/
@@ -58,19 +56,6 @@
#define PCI_IO_END (MODULES_VADDR - SZ_2M)
#define PCI_IO_START (PCI_IO_END - PCI_IO_SIZE)
#define FIXADDR_TOP (PCI_IO_START - SZ_2M)
-#define TASK_SIZE_64 (UL(1) << VA_BITS)
-
-#ifdef CONFIG_COMPAT
-#define TASK_SIZE_32 UL(0x100000000)
-#define TASK_SIZE (test_thread_flag(TIF_32BIT) ? \
- TASK_SIZE_32 : TASK_SIZE_64)
-#define TASK_SIZE_OF(tsk) (test_tsk_thread_flag(tsk, TIF_32BIT) ? \
- TASK_SIZE_32 : TASK_SIZE_64)
-#else
-#define TASK_SIZE TASK_SIZE_64
-#endif /* CONFIG_COMPAT */
-
-#define TASK_UNMAPPED_BASE (PAGE_ALIGN(TASK_SIZE / 4))
/*
* Physical vs virtual RAM address space conversion. These are
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index d08559528927..75d9ef6c457c 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -19,6 +19,10 @@
#ifndef __ASM_PROCESSOR_H
#define __ASM_PROCESSOR_H
+#define TASK_SIZE_64 (UL(1) << VA_BITS)
+
+#ifndef __ASSEMBLY__
+
/*
* Default implementation of macro that returns current
* instruction pointer ("program counter").
@@ -36,6 +40,22 @@
#include <asm/types.h>
#ifdef __KERNEL__
+/*
+ * TASK_SIZE - the maximum size of a user space task.
+ * TASK_UNMAPPED_BASE - the lower boundary of the mmap VM area.
+ */
+#ifdef CONFIG_COMPAT
+#define TASK_SIZE_32 UL(0x100000000)
+#define TASK_SIZE (test_thread_flag(TIF_32BIT) ? \
+ TASK_SIZE_32 : TASK_SIZE_64)
+#define TASK_SIZE_OF(tsk) (test_tsk_thread_flag(tsk, TIF_32BIT) ? \
+ TASK_SIZE_32 : TASK_SIZE_64)
+#else
+#define TASK_SIZE TASK_SIZE_64
+#endif /* CONFIG_COMPAT */
+
+#define TASK_UNMAPPED_BASE (PAGE_ALIGN(TASK_SIZE / 4))
+
#define STACK_TOP_MAX TASK_SIZE_64
#ifdef CONFIG_COMPAT
#define AARCH32_VECTORS_BASE 0xffff0000
@@ -188,4 +208,5 @@ static inline void spin_lock_prefetch(const void *x)
int cpu_enable_pan(void *__unused);
+#endif /* __ASSEMBLY__ */
#endif /* __ASM_PROCESSOR_H */
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 586326981769..c849be9231bb 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -27,7 +27,7 @@
#include <asm/cpufeature.h>
#include <asm/errno.h>
#include <asm/esr.h>
-#include <asm/memory.h>
+#include <asm/processor.h>
#include <asm/thread_info.h>
#include <asm/asm-uaccess.h>
#include <asm/unistd.h>
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 02/44] arm64: Implement array_index_mask_nospec()
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Robin Murphy <robin.murphy@arm.com>
commit 022620eed3d0bc4bf2027326f599f5ad71c2ea3f upstream.
Provide an optimised, assembly implementation of array_index_mask_nospec()
for arm64 so that the compiler is not in a position to transform the code
in ways which affect its ability to inhibit speculation (e.g. by introducing
conditional branches).
This is similar to the sequence used by x86, modulo architectural differences
in the carry/borrow flags.
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/barrier.h | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h
index 574486634c62..7c25e3e11b6d 100644
--- a/arch/arm64/include/asm/barrier.h
+++ b/arch/arm64/include/asm/barrier.h
@@ -37,6 +37,27 @@
#define dma_rmb() dmb(oshld)
#define dma_wmb() dmb(oshst)
+/*
+ * Generate a mask for array_index__nospec() that is ~0UL when 0 <= idx < sz
+ * and 0 otherwise.
+ */
+#define array_index_mask_nospec array_index_mask_nospec
+static inline unsigned long array_index_mask_nospec(unsigned long idx,
+ unsigned long sz)
+{
+ unsigned long mask;
+
+ asm volatile(
+ " cmp %1, %2\n"
+ " sbc %0, xzr, xzr\n"
+ : "=r" (mask)
+ : "r" (idx), "Ir" (sz)
+ : "cc");
+
+ csdb();
+ return mask;
+}
+
#define smp_mb() dmb(ish)
#define smp_rmb() dmb(ishld)
#define smp_wmb() dmb(ishst)
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 01/44] arm64: barrier: Add CSDB macros to control data-value prediction
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
In-Reply-To: <cover.1567077734.git.viresh.kumar@linaro.org>
From: Will Deacon <will.deacon@arm.com>
commit 669474e772b952b14f4de4845a1558fd4c0414a4 upstream.
For CPUs capable of data value prediction, CSDB waits for any outstanding
predictions to architecturally resolve before allowing speculative execution
to continue. Provide macros to expose it to the arch code.
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
arch/arm64/include/asm/assembler.h | 7 +++++++
arch/arm64/include/asm/barrier.h | 2 ++
2 files changed, 9 insertions(+)
diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index f68abb17aa4b..683c2875278f 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -95,6 +95,13 @@
dmb \opt
.endm
+/*
+ * Value prediction barrier
+ */
+ .macro csdb
+ hint #20
+ .endm
+
#define USER(l, x...) \
9999: x; \
.section __ex_table,"a"; \
diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h
index f2d2c0bbe21b..574486634c62 100644
--- a/arch/arm64/include/asm/barrier.h
+++ b/arch/arm64/include/asm/barrier.h
@@ -28,6 +28,8 @@
#define dmb(opt) asm volatile("dmb " #opt : : : "memory")
#define dsb(opt) asm volatile("dsb " #opt : : : "memory")
+#define csdb() asm volatile("hint #20" : : : "memory")
+
#define mb() dsb(sy)
#define rmb() dsb(ld)
#define wmb() dsb(st)
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH ARM64 v4.4 V3 00/44] V4.4 backport of arm64 Spectre patches
From: Viresh Kumar @ 2019-08-29 11:33 UTC (permalink / raw)
To: stable, Julien Thierry, Mark Rutland
Cc: Marc Zyngier, Viresh Kumar, Will Deacon, mark.brown,
Catalin Marinas, Russell King, linux-arm-kernel
Hello,
This series backports arm64 spectre patches to v4.4 stable kernel. I
have started this backport with Mark Rutland's backport of Spectre to
4.9 [1] and tried applying the upstream version of them over 4.4 and
resolved conflicts by checking how they have been resolved in 4.9.
The KVM changes are mostly dropped as the KVM code in v4.4 is quite
different and it makes backport more complex. This was suggested by the
ARM team.
I had to pick few extra upstream patches to avoid conflicts and to make
things work:
mm/kasan: add API to check memory regions
arm64: kasan: instrument user memory access API
arm64: cpufeature: Add scope for capability check
arm64: cputype info for Broadcom Vulcan
arm64: cputype: Add MIDR values for Cavium ThunderX2 CPUs
ARM: 8478/2: arm/arm64: add arm-smccc
arm64: cpufeature: Test 'matches' pointer to find the end of the list
arm64: Introduce cpu_die_early
arm64: Move cpu_die_early to smp.c
arm64: Verify CPU errata work arounds on hotplugged CPU
arm64: errata: Calling enable functions for CPU errata too
arm64: Rearrange CPU errata workaround checks
arm64: Add a helper for parking CPUs in a loop
I also had to drop few patches as they weren't getting applied properly
due to missing files/features or they were KVM related:
arm64: cpufeature: __this_cpu_has_cap() shouldn't stop early
arm64: KVM: Use per-CPU vector when BP hardening is enabled
arm64: KVM: Make PSCI_VERSION a fast path
mm: Introduce lm_alias
arm64: KVM: Increment PC after handling an SMC trap
arm/arm64: KVM: Consolidate the PSCI include files
arm/arm64: KVM: Add PSCI_VERSION helper
arm/arm64: KVM: Add smccc accessors to PSCI code
arm/arm64: KVM: Implement PSCI 1.0 support
arm/arm64: KVM: Turn kvm_psci_version into a static inline
arm64: KVM: Add SMCCC_ARCH_WORKAROUND_1 fast handling
The patches in this series are pushed here [2].
This is tested on Hikey board (octa A53) and I verified that BP
hardening code is getting hit for CPUs (had to hack a bit and enable
BP hardening support for A53 for this).
V2->V3:
- Included an extra patch and used the helper introduced by it:
arm64: Add a helper for parking CPUs in a loop
- Droped few extra blank lines added by mistake
- Avoid overlapping with SVE2 bits
V1->V2:
- Rebased over 4.4.184 (was 4.4.180 earlier).
- Fixed an build issue with CONFIG_KASAN (Julien).
- Dropped few patches, mostly KVM stuff (Julien):
arm64: remove duplicate macro __KERNEL__ check
mm: Introduce lm_alias
arm64: KVM: Increment PC after handling an SMC trap
arm/arm64: KVM: Consolidate the PSCI include files
arm/arm64: KVM: Add PSCI_VERSION helper
arm/arm64: KVM: Add smccc accessors to PSCI code
arm/arm64: KVM: Implement PSCI 1.0 support
arm/arm64: KVM: Turn kvm_psci_version into a static inline
arm64: KVM: Add SMCCC_ARCH_WORKAROUND_1 fast handling
- Added few patches to fix issues reported by Julien:
arm64: cpufeature: Test 'matches' pointer to find the end of the list
arm64: Introduce cpu_die_early
arm64: Move cpu_die_early to smp.c
arm64: Verify CPU errata work arounds on hotplugged CPU
arm64: errata: Calling enable functions for CPU errata too
arm64: Rearrange CPU errata workaround checks
--
viresh
[1] https://patches.linaro.org/cover/133195/ with top commit in 4.9 stable tree:
a3b292fe0560 arm64: futex: Mask __user pointers prior to dereference
[2] https://git.kernel.org/pub/scm/linux/kernel/git/vireshk/linux.git stable/arm64/v4.4.y/spectre
-------------------------8<-------------------------
Andre Przywara (1):
arm64: errata: Calling enable functions for CPU errata too
Andrey Ryabinin (1):
mm/kasan: add API to check memory regions
Catalin Marinas (1):
arm64: Factor out TTBR0_EL1 post-update workaround into a specific asm
macro
James Morse (1):
arm64: cpufeature: Test 'matches' pointer to find the end of the list
Jayachandran C (3):
arm64: cputype info for Broadcom Vulcan
arm64: cputype: Add MIDR values for Cavium ThunderX2 CPUs
arm64: Branch predictor hardening for Cavium ThunderX2
Jens Wiklander (1):
ARM: 8478/2: arm/arm64: add arm-smccc
Marc Zyngier (11):
arm64: Move post_ttbr_update_workaround to C code
arm64: Move BP hardening to check_and_switch_context
arm64: cpu_errata: Allow an erratum to be match for all revisions of a
core
arm/arm64: KVM: Advertise SMCCC v1.1
arm64: KVM: Report SMCCC_ARCH_WORKAROUND_1 BP hardening support
firmware/psci: Expose PSCI conduit
firmware/psci: Expose SMCCC version through psci_ops
arm/arm64: smccc: Make function identifiers an unsigned quantity
arm/arm64: smccc: Implement SMCCC v1.1 inline primitive
arm64: Add ARM_SMCCC_ARCH_WORKAROUND_1 BP hardening support
arm64: Kill PSCI_GET_VERSION as a variant-2 workaround
Robin Murphy (3):
arm64: Implement array_index_mask_nospec()
arm64: Make USER_DS an inclusive limit
arm64: Use pointer masking to limit uaccess speculation
Suzuki K Poulose (7):
arm64: cpufeature: Add scope for capability check
arm64: Introduce cpu_die_early
arm64: Add a helper for parking CPUs in a loop
arm64: Move cpu_die_early to smp.c
arm64: Verify CPU errata work arounds on hotplugged CPU
arm64: Rearrange CPU errata workaround checks
arm64: Run enable method for errata work arounds on late CPUs
Will Deacon (13):
arm64: barrier: Add CSDB macros to control data-value prediction
arm64: entry: Ensure branch through syscall table is bounded under
speculation
arm64: uaccess: Prevent speculative use of the current addr_limit
arm64: uaccess: Don't bother eliding access_ok checks in __{get,
put}_user
arm64: uaccess: Mask __user pointers for __arch_{clear, copy_*}_user
arm64: cpufeature: Pass capability structure to ->enable callback
drivers/firmware: Expose psci_get_version through psci_ops structure
arm64: Add skeleton to harden the branch predictor against aliasing
attacks
arm64: entry: Apply BP hardening for high-priority synchronous
exceptions
arm64: entry: Apply BP hardening for suspicious interrupts from EL0
arm64: cputype: Add missing MIDR values for Cortex-A72 and Cortex-A75
arm64: Implement branch predictor hardening for affected Cortex-A CPUs
arm64: futex: Mask __user pointers prior to dereference
Yang Shi (1):
arm64: kasan: instrument user memory access API
Yury Norov (1):
arm64: move TASK_* definitions to <asm/processor.h>
MAINTAINERS | 14 ++
arch/arm64/Kconfig | 17 ++
arch/arm64/include/asm/assembler.h | 18 ++
arch/arm64/include/asm/barrier.h | 23 +++
arch/arm64/include/asm/cpufeature.h | 24 ++-
arch/arm64/include/asm/cputype.h | 12 ++
arch/arm64/include/asm/futex.h | 9 +-
arch/arm64/include/asm/memory.h | 15 --
arch/arm64/include/asm/mmu.h | 39 ++++
arch/arm64/include/asm/processor.h | 24 +++
arch/arm64/include/asm/smp.h | 9 +
arch/arm64/include/asm/sysreg.h | 2 +
arch/arm64/include/asm/uaccess.h | 173 ++++++++++++------
arch/arm64/kernel/Makefile | 5 +
arch/arm64/kernel/arm64ksyms.c | 8 +-
arch/arm64/kernel/bpi.S | 75 ++++++++
arch/arm64/kernel/cpu_errata.c | 213 +++++++++++++++++++++-
arch/arm64/kernel/cpufeature.c | 186 +++++++++----------
arch/arm64/kernel/cpuinfo.c | 2 -
arch/arm64/kernel/entry.S | 26 ++-
arch/arm64/kernel/smp.c | 30 +++-
arch/arm64/lib/clear_user.S | 6 +-
arch/arm64/lib/copy_from_user.S | 4 +-
arch/arm64/lib/copy_in_user.S | 4 +-
arch/arm64/lib/copy_to_user.S | 4 +-
arch/arm64/mm/context.c | 12 ++
arch/arm64/mm/fault.c | 31 ++++
arch/arm64/mm/proc.S | 12 +-
drivers/firmware/Kconfig | 3 +
drivers/firmware/psci.c | 58 +++++-
include/linux/arm-smccc.h | 267 ++++++++++++++++++++++++++++
include/linux/kasan-checks.h | 12 ++
include/linux/psci.h | 14 ++
mm/kasan/kasan.c | 12 ++
34 files changed, 1150 insertions(+), 213 deletions(-)
create mode 100644 arch/arm64/kernel/bpi.S
create mode 100644 include/linux/arm-smccc.h
create mode 100644 include/linux/kasan-checks.h
--
2.21.0.rc0.269.g1a574e7a288b
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply
* Re: [PATCH v1 1/1] ARM: dts: rockchip: set crypto default disabled on rk3288
From: Elon Zhang @ 2019-08-29 11:31 UTC (permalink / raw)
To: Heiko Stuebner
Cc: mark.rutland, devicetree, linux-kernel, linux-rockchip, robh+dt,
linux-arm-kernel
In-Reply-To: <4806912.UyKsYhR33o@phil>
Hi Heiko,
On 8/27/2019 22:28, Heiko Stuebner wrote:
> Hi,
>
> Am Dienstag, 27. August 2019, 09:14:39 CEST schrieb Elon Zhang:
>> Not every board needs to enable crypto node, so the node should
>> be set default disabled in rk3288.dtsi and enabled in specific
>> board dts file.
> Can you give a bit more rationale here? There would need to be a very
> specific reason because of the following:
>
> The crypto module is not wired to some board-specific components,
> so its usability does not depend on the specific board at all.
> Instead every board can just use it out of the box and the devicetree
> is supposed to describe the hardware and is _not_ meant as a space
> for user configuration.
Right for almost all normal hardware modules but the crypto module was
designed
for secure world. As a result, the crypto module will become
inaccessible for linux
kernel if secure world enable it.
We plan to enable the crypto module in secure world so we should set
crypto module
default disabled in linux kernel.
>
> So in fact the status property should probably go away completely from
> the crypto node, as it's usable out of the box in all cases.
>
>
> Heiko
>
>
>
>> Signed-off-by: Elon Zhang <zhangzj@rock-chips.com>
>> ---
>> arch/arm/boot/dts/rk3288.dtsi | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/arch/arm/boot/dts/rk3288.dtsi b/arch/arm/boot/dts/rk3288.dtsi
>> index cc893e154fe5..d509aa24177c 100644
>> --- a/arch/arm/boot/dts/rk3288.dtsi
>> +++ b/arch/arm/boot/dts/rk3288.dtsi
>> @@ -984,7 +984,7 @@
>> clock-names = "aclk", "hclk", "sclk", "apb_pclk";
>> resets = <&cru SRST_CRYPTO>;
>> reset-names = "crypto-rst";
>> - status = "okay";
>> + status = "disabled";
>> };
>>
>> iep_mmu: iommu@ff900800 {
>>
>
>
>
>
>
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply
* [PATCH v4 07/14] drm/mxsfb: Signal mode changed when bpp changed
From: Robert Chiras @ 2019-08-29 11:30 UTC (permalink / raw)
To: Guido Günther, Marek Vasut, Stefan Agner, David Airlie,
Daniel Vetter, Rob Herring, Mark Rutland, Shawn Guo, Sascha Hauer,
Fabio Estevam
Cc: devicetree, linux-kernel, dri-devel, NXP Linux Team,
Pengutronix Kernel Team, linux-arm-kernel
In-Reply-To: <1567078215-31601-1-git-send-email-robert.chiras@nxp.com>
From: Mirela Rabulea <mirela.rabulea@nxp.com>
Add mxsfb_atomic_helper_check to signal mode changed when bpp changed.
This will trigger the execution of disable/enable on
a modeset with different bpp than the current one.
Signed-off-by: Mirela Rabulea <mirela.rabulea@nxp.com>
Signed-off-by: Robert Chiras <robert.chiras@nxp.com>
Tested-by: Guido Günther <agx@sigxcpu.org>
---
drivers/gpu/drm/mxsfb/mxsfb_drv.c | 45 ++++++++++++++++++++++++++++++++++++++-
1 file changed, 44 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/mxsfb/mxsfb_drv.c b/drivers/gpu/drm/mxsfb/mxsfb_drv.c
index 23027a9..f95ba63 100644
--- a/drivers/gpu/drm/mxsfb/mxsfb_drv.c
+++ b/drivers/gpu/drm/mxsfb/mxsfb_drv.c
@@ -26,6 +26,7 @@
#include <drm/drm_drv.h>
#include <drm/drm_fb_cma_helper.h>
#include <drm/drm_fb_helper.h>
+#include <drm/drm_fourcc.h>
#include <drm/drm_gem_cma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_irq.h>
@@ -105,9 +106,51 @@ void mxsfb_disable_axi_clk(struct mxsfb_drm_private *mxsfb)
clk_disable_unprepare(mxsfb->clk_axi);
}
+/**
+ * mxsfb_atomic_helper_check - validate state object
+ * @dev: DRM device
+ * @state: the driver state object
+ *
+ * On top of the drm imlementation drm_atomic_helper_check,
+ * check if the bpp is changed, if so, signal mode_changed,
+ * this will trigger disable/enable
+ *
+ * RETURNS:
+ * Zero for success or -errno
+ */
+static int mxsfb_atomic_helper_check(struct drm_device *dev,
+ struct drm_atomic_state *state)
+{
+ struct drm_crtc *crtc;
+ struct drm_crtc_state *new_state;
+ int i, ret;
+
+ ret = drm_atomic_helper_check(dev, state);
+ if (ret)
+ return ret;
+
+ for_each_new_crtc_in_state(state, crtc, new_state, i) {
+ struct drm_plane_state *primary_state;
+ int old_bpp = 0;
+ int new_bpp = 0;
+
+ if (!crtc->primary || !crtc->primary->old_fb)
+ continue;
+ primary_state =
+ drm_atomic_get_plane_state(state, crtc->primary);
+ if (!primary_state || !primary_state->fb)
+ continue;
+ old_bpp = crtc->primary->old_fb->format->depth;
+ new_bpp = primary_state->fb->format->depth;
+ if (old_bpp != new_bpp)
+ new_state->mode_changed = true;
+ }
+ return ret;
+}
+
static const struct drm_mode_config_funcs mxsfb_mode_config_funcs = {
.fb_create = drm_gem_fb_create,
- .atomic_check = drm_atomic_helper_check,
+ .atomic_check = mxsfb_atomic_helper_check,
.atomic_commit = drm_atomic_helper_commit,
};
--
2.7.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH v4 10/14] drm/mxsfb: Update mxsfb to support LCD reset
From: Robert Chiras @ 2019-08-29 11:30 UTC (permalink / raw)
To: Guido Günther, Marek Vasut, Stefan Agner, David Airlie,
Daniel Vetter, Rob Herring, Mark Rutland, Shawn Guo, Sascha Hauer,
Fabio Estevam
Cc: devicetree, linux-kernel, dri-devel, NXP Linux Team,
Pengutronix Kernel Team, linux-arm-kernel
In-Reply-To: <1567078215-31601-1-git-send-email-robert.chiras@nxp.com>
The eLCDIF controller has control pin for the external LCD reset pin.
Add support for it and assert this pin in enable and de-assert it in
disable.
Signed-off-by: Robert Chiras <robert.chiras@nxp.com>
Tested-by: Guido Günther <agx@sigxcpu.org>
---
drivers/gpu/drm/mxsfb/mxsfb_crtc.c | 14 ++++++++++----
drivers/gpu/drm/mxsfb/mxsfb_regs.h | 2 ++
2 files changed, 12 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/mxsfb/mxsfb_crtc.c b/drivers/gpu/drm/mxsfb/mxsfb_crtc.c
index 1be29f5..a4ba368 100644
--- a/drivers/gpu/drm/mxsfb/mxsfb_crtc.c
+++ b/drivers/gpu/drm/mxsfb/mxsfb_crtc.c
@@ -224,9 +224,12 @@ static void mxsfb_enable_controller(struct mxsfb_drm_private *mxsfb)
clk_prepare_enable(mxsfb->clk_disp_axi);
clk_prepare_enable(mxsfb->clk);
- if (mxsfb->devdata->ipversion >= 4)
+ if (mxsfb->devdata->ipversion >= 4) {
writel(CTRL2_OUTSTANDING_REQS(REQ_16),
mxsfb->base + LCDC_V4_CTRL2 + REG_SET);
+ /* Assert LCD Reset bit */
+ writel(CTRL2_LCD_RESET, mxsfb->base + LCDC_V4_CTRL2 + REG_SET);
+ }
/* If it was disabled, re-enable the mode again */
writel(CTRL_DOTCLK_MODE, mxsfb->base + LCDC_CTRL + REG_SET);
@@ -244,11 +247,14 @@ static void mxsfb_disable_controller(struct mxsfb_drm_private *mxsfb)
{
u32 reg;
- if (mxsfb->devdata->ipversion >= 4)
+ writel(CTRL_RUN, mxsfb->base + LCDC_CTRL + REG_CLR);
+
+ if (mxsfb->devdata->ipversion >= 4) {
writel(CTRL2_OUTSTANDING_REQS(0x7),
mxsfb->base + LCDC_V4_CTRL2 + REG_CLR);
-
- writel(CTRL_RUN, mxsfb->base + LCDC_CTRL + REG_CLR);
+ /* De-assert LCD Reset bit */
+ writel(CTRL2_LCD_RESET, mxsfb->base + LCDC_V4_CTRL2 + REG_CLR);
+ }
/*
* Even if we disable the controller here, it will still continue
diff --git a/drivers/gpu/drm/mxsfb/mxsfb_regs.h b/drivers/gpu/drm/mxsfb/mxsfb_regs.h
index dc4daa0..0f63ba1 100644
--- a/drivers/gpu/drm/mxsfb/mxsfb_regs.h
+++ b/drivers/gpu/drm/mxsfb/mxsfb_regs.h
@@ -108,6 +108,8 @@
#define CTRL2_LINE_PATTERN_BGR 5
#define CTRL2_LINE_PATTERN_CLR 7
+#define CTRL2_LCD_RESET BIT(0)
+
#define TRANSFER_COUNT_SET_VCOUNT(x) REG_PUT((x), 31, 16)
#define TRANSFER_COUNT_GET_VCOUNT(x) REG_GET((x), 31, 16)
#define TRANSFER_COUNT_SET_HCOUNT(x) REG_PUT((x), 15, 0)
--
2.7.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox