* [PATCH v11 08/19] arm64: fpsimd: Eliminate task->mm checks
From: Dave Martin @ 2018-05-25 9:52 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180525090257.GB54529@C02W217FHV2R.local>
On Fri, May 25, 2018 at 11:02:57AM +0200, Christoffer Dall wrote:
> On Thu, May 24, 2018 at 05:56:37PM +0100, Dave Martin wrote:
> > Currently the FPSIMD handling code uses the condition task->mm ==
> > NULL as a hint that task has no FPSIMD register context.
> >
> > The ->mm check is only there to filter out tasks that cannot
> > possibly have FPSIMD context loaded, for optimisation purposes.
> > Also, TIF_FOREIGN_FPSTATE must always be checked anyway before
> > saving FPSIMD context back to memory. For these reasons, the ->mm
> > checks are not useful, providing that TIF_FOREIGN_FPSTATE is
> > maintained in a consistent way for all threads.
> >
> > The context switch logic is already deliberately optimised to defer
> > reloads of the regs until ret_to_user (or sigreturn as a special
> > case), and save them only if they have been previously loaded.
> > These paths are the only places where the wrong_task and wrong_cpu
> > conditions can be made false, by calling fpsimd_bind_task_to_cpu().
> > Kernel threads by definition never reach these paths. As a result,
> > the wrong_task and wrong_cpu tests in fpsimd_thread_switch() will
> > always yield true for kernel threads.
> >
> > This patch removes the redundant checks and special-case code, ensuring that TIF_FOREIGN_FPSTATE is set whenever a kernel thread is scheduled in, and ensures that this flag is set for the init
> > task. The fpsimd_flush_task_state() call already present in
> > copy_thread() ensures the same for any new task.
>
> nit: formatting still funny, but you shouldn't respin just for that.
Ah jeez... it was a long day.
I guess this isn't the end of the world, but I'll fix this in my branch
and point Marc at it.
> >
> > With TIF_FOREIGN_FPSTATE always set for kernel threads, this patch
> > ensures that no extra context save work is added for kernel
> > threads, and eliminates the redundant context saving that may
> > currently occur for kernel threads that have acquired an mm via
> > use_mm().
>
> Reviewed-by: Christoffer Dall <christoffer.dall@arm.com>
[...]
Thanks
---Dave
^ permalink raw reply
* [PATCH V2 3/4] clk: add managed version of clk_bulk_get_all
From: A.s. Dong @ 2018-05-25 9:51 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <201803231452.YBWTInPr%fengguang.wu@intel.com>
> -----Original Message-----
> From: kbuild test robot [mailto:lkp at intel.com]
> Sent: Friday, March 23, 2018 2:49 PM
> To: A.s. Dong <aisheng.dong@nxp.com>
> Cc: kbuild-all at 01.org; linux-clk at vger.kernel.org; linux-
> kernel at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> mturquette at baylibre.com; hdegoede at redhat.com;
> b.zolnierkie at samsung.com; linux at armlinux.org.uk; linux-
> fbdev at vger.kernel.org; dl-linux-imx <linux-imx@nxp.com>;
> sboyd at kernel.org; A.s. Dong <aisheng.dong@nxp.com>; Stephen Boyd
> <sboyd@codeaurora.org>
> Subject: Re: [PATCH V2 3/4] clk: add managed version of clk_bulk_get_all
>
[...]
> All errors (new ones prefixed by >>):
>
> In file included from include/linux/cpufreq.h:14:0,
> from arch/powerpc/platforms/cell/cpufreq_spudemand.c:23:
> include/linux/clk.h:679:1: error: expected identifier or '(' before '{' token
> {
> ^
> >> include/linux/clk.h:677:32: error: 'devm_clk_bulk_get_all' declared
> >> 'static' but never defined [-Werror=unused-function]
> static inline int __must_check devm_clk_bulk_get_all(struct device *dev,
> ^~~~~~~~~~~~~~~~~~~~~
> cc1: all warnings being treated as errors
>
> vim +677 include/linux/clk.h
>
> 676
> > 677 static inline int __must_check devm_clk_bulk_get_all(struct device
> *dev,
> 678 struct
> clk_bulk_data **clks);
Thanks for the reporting.
Caused by the extra unneeded ';' at the end of the line.
Will fix.
Regards
Dong Aisheng
> > 679 {
> 680
> 681 return 0;
> 682 }
> 683
>
> ---
> 0-DAY kernel test infrastructure Open Source Technology Center
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist
> s.01.org%2Fpipermail%2Fkbuild-
> all&data=02%7C01%7Caisheng.dong%40nxp.com%7Ce9e35f96c61a4c46d1790
> 8d5908a4d46%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6365738
> 46071200287&sdata=yIWEjcpE2tyJP%2BmsMPqn%2F83TCsxw64J%2Ba5ue73
> qGnN0%3D&reserved=0 Intel Corporation
^ permalink raw reply
* [PATCH V2 2/4] clk: add new APIs to operate on all available clocks
From: A.s. Dong @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <152182421754.178046.7553203878044979411@swboyd.mtv.corp.google.com>
> -----Original Message-----
> From: Stephen Boyd [mailto:sboyd at kernel.org]
> Sent: Saturday, March 24, 2018 12:57 AM
> To: A.s. Dong <aisheng.dong@nxp.com>; linux-clk at vger.kernel.org
> Cc: linux-kernel at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> mturquette at baylibre.com; hdegoede at redhat.com;
> b.zolnierkie at samsung.com; linux at armlinux.org.uk; linux-
> fbdev at vger.kernel.org; dl-linux-imx <linux-imx@nxp.com>; A.s. Dong
> <aisheng.dong@nxp.com>; Stephen Boyd <sboyd@codeaurora.org>;
> Masahiro Yamada <yamada.masahiro@socionext.com>
> Subject: Re: [PATCH V2 2/4] clk: add new APIs to operate on all available
> clocks
>
> Quoting Dong Aisheng (2018-03-20 20:19:49)
> > @@ -50,6 +52,38 @@ static int __must_check of_clk_bulk_get(struct
> device_node *np, int num_clks,
> > return ret;
> > }
> > EXPORT_SYMBOL(of_clk_bulk_get);
> > +
> > +static int __must_check of_clk_bulk_get_all(struct device_node *np,
> > + struct clk_bulk_data
> > +**clks) {
> > + struct clk_bulk_data *clk_bulk;
> > + int num_clks;
> > + int ret;
> > +
> > + num_clks = of_clk_get_parent_count(np);
> > + if (!num_clks)
> > + return 0;
> > +
> > + clk_bulk = kcalloc(num_clks, sizeof(*clk_bulk), GFP_KERNEL);
>
> Can be kmalloc_array? of_clk_bulk_get() already clears things out
> appropriately.
Yes, indeed. Will use kmalloc_array instead.
>
> > + if (!clk_bulk)
> > + return -ENOMEM;
> > +
> > + ret = of_clk_bulk_get(np, num_clks, clk_bulk);
> > + if (ret) {
> > + kfree(clk_bulk);
> > + return ret;
> > + }
> > +
> > + *clks = clk_bulk;
> > +
> > + return num_clks;
> > +}
> > +#else
> > +static int __must_check of_clk_bulk_get_all(struct device_node *np,
> > + struct clk_bulk_data
> > +**clks) {
> > + return -ENOENT;
> > +}
> > #endif
>
> This else can probably be dropped too.
>
Right.
> >
> > void clk_bulk_put(int num_clks, struct clk_bulk_data *clks) @@ -90,6
> > +124,29 @@ int __must_check clk_bulk_get(struct device *dev, int
> > num_clks, } EXPORT_SYMBOL(clk_bulk_get);
> >
> > +void clk_bulk_put_all(int num_clks, struct clk_bulk_data *clks) {
> > + if (IS_ERR_OR_NULL(clks))
> > + return;
> > +
> > + clk_bulk_put(num_clks, clks);
> > +
> > + kfree(clks);
> > +}
> > +EXPORT_SYMBOL(clk_bulk_put_all);
> > +
> > +int __must_check clk_bulk_get_all(struct device *dev,
> > + struct clk_bulk_data **clks) {
> > + struct device_node *np = dev_of_node(dev);
> > +
> > + if (!np)
> > + return 0;
> > +
> > + return of_clk_bulk_get_all(np, clks); }
> > +EXPORT_SYMBOL(clk_bulk_get_all);
>
> Looks better!
Thanks
Regards
Dong Aisheng
^ permalink raw reply
* [PATCH v4 26/26] irqchip/gic-v3: Allow interrupts to be set as pseudo-NMI
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
Provide a way to set a GICv3 interrupt as pseudo-NMI. The interrupt
must not be enabled when setting/clearing the NMI status of the interrupt.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
drivers/irqchip/irq-gic-v3.c | 54 ++++++++++++++++++++++++++++++++++++++++++++
include/linux/interrupt.h | 1 +
2 files changed, 55 insertions(+)
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index fa23d12..cea1000 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -305,6 +305,43 @@ static void handle_percpu_devid_nmi(struct irq_desc *desc)
chip->irq_eoi(&desc->irq_data);
}
+static int gic_irq_set_irqchip_prio(struct irq_data *d, bool val)
+{
+ u8 prio;
+ irq_flow_handler_t handler;
+
+ if (gic_peek_irq(d, GICD_ISENABLER)) {
+ pr_err("Cannot set NMI property of enabled IRQ %u\n", d->irq);
+ return -EPERM;
+ }
+
+ if (val) {
+ prio = GICD_INT_NMI_PRI;
+
+ if (gic_irq(d) < 32)
+ handler = handle_percpu_devid_nmi;
+ else
+ handler = handle_fasteoi_nmi;
+ } else {
+ prio = GICD_INT_DEF_PRI;
+
+ if (gic_irq(d) < 32)
+ handler = handle_percpu_devid_irq;
+ else
+ handler = handle_fasteoi_irq;
+ }
+
+ /*
+ * Already in a locked context for the desc from calling
+ * irq_set_irq_chip_state.
+ * It should be safe to simply modify the handler.
+ */
+ irq_to_desc(d->irq)->handle_irq = handler;
+ gic_set_irq_prio(gic_irq(d), gic_dist_base(d), prio);
+
+ return 0;
+}
+
static int gic_irq_set_irqchip_state(struct irq_data *d,
enum irqchip_irq_state which, bool val)
{
@@ -326,6 +363,16 @@ static int gic_irq_set_irqchip_state(struct irq_data *d,
reg = val ? GICD_ICENABLER : GICD_ISENABLER;
break;
+ case IRQCHIP_STATE_NMI:
+ if (gic_supports_nmi()) {
+ return gic_irq_set_irqchip_prio(d, val);
+ } else if (val) {
+ pr_warn("Failed to set IRQ %u as NMI, NMIs are unsupported\n",
+ gic_irq(d));
+ return -EINVAL;
+ }
+ return 0;
+
default:
return -EINVAL;
}
@@ -353,6 +400,13 @@ static int gic_irq_get_irqchip_state(struct irq_data *d,
*val = !gic_peek_irq(d, GICD_ISENABLER);
break;
+ case IRQCHIP_STATE_NMI:
+ if (!gic_supports_nmi())
+ return -EINVAL;
+ *val = (gic_get_irq_prio(gic_irq(d), gic_dist_base(d)) ==
+ GICD_INT_NMI_PRI);
+ break;
+
default:
return -EINVAL;
}
diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
index 5426627..02c794f 100644
--- a/include/linux/interrupt.h
+++ b/include/linux/interrupt.h
@@ -419,6 +419,7 @@ enum irqchip_irq_state {
IRQCHIP_STATE_ACTIVE, /* Is interrupt in progress? */
IRQCHIP_STATE_MASKED, /* Is interrupt masked? */
IRQCHIP_STATE_LINE_LEVEL, /* Is IRQ line high? */
+ IRQCHIP_STATE_NMI, /* Is IRQ an NMI? */
};
extern int irq_get_irqchip_state(unsigned int irq, enum irqchip_irq_state which,
--
1.9.1
^ permalink raw reply related
* [PATCH v4 25/26] irqchip/gic-v3: Provide NMI handlers
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
Provide irqchip handlers that are NMI safe for PPIs and SPIs.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
drivers/irqchip/irq-gic-v3.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 44 insertions(+)
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 4be5996..fa23d12 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -34,6 +34,8 @@
#include <linux/irqchip/arm-gic-v3.h>
#include <linux/irqchip/irq-partition-percpu.h>
+#include <trace/events/irq.h>
+
#include <asm/cputype.h>
#include <asm/exception.h>
#include <asm/smp_plat.h>
@@ -261,6 +263,48 @@ static inline bool gic_supports_nmi(void)
&& static_branch_likely(&have_non_secure_prio_view);
}
+/*
+ * Chip flow handler for SPIs set as NMI
+ */
+static void handle_fasteoi_nmi(struct irq_desc *desc)
+{
+ struct irq_chip *chip = irq_desc_get_chip(desc);
+ struct irqaction *action = desc->action;
+ unsigned int irq = irq_desc_get_irq(desc);
+ irqreturn_t res;
+
+ if (chip->irq_ack)
+ chip->irq_ack(&desc->irq_data);
+
+ trace_irq_handler_entry(irq, action);
+ res = action->handler(irq, action->dev_id);
+ trace_irq_handler_exit(irq, action, res);
+
+ if (chip->irq_eoi)
+ chip->irq_eoi(&desc->irq_data);
+}
+
+/*
+ * Chip flow handler for PPIs set as NMI
+ */
+static void handle_percpu_devid_nmi(struct irq_desc *desc)
+{
+ struct irq_chip *chip = irq_desc_get_chip(desc);
+ struct irqaction *action = desc->action;
+ unsigned int irq = irq_desc_get_irq(desc);
+ irqreturn_t res;
+
+ if (chip->irq_ack)
+ chip->irq_ack(&desc->irq_data);
+
+ trace_irq_handler_entry(irq, action);
+ res = action->handler(irq, raw_cpu_ptr(action->percpu_dev_id));
+ trace_irq_handler_exit(irq, action, res);
+
+ if (chip->irq_eoi)
+ chip->irq_eoi(&desc->irq_data);
+}
+
static int gic_irq_set_irqchip_state(struct irq_data *d,
enum irqchip_irq_state which, bool val)
{
--
1.9.1
^ permalink raw reply related
* [PATCH v4 24/26] irqchip/gic-v3: Add base support for pseudo-NMI
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
Provide a higher priority to be used for pseudo-NMIs. When such an
interrupt is received, enter the NMI state and prevent other NMIs to
be raised.
When returning from a pseudo-NMI, skip preemption and tracing if the
interrupted context has interrupts disabled.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm/include/asm/arch_gicv3.h | 6 ++++++
arch/arm64/include/asm/arch_gicv3.h | 6 ++++++
arch/arm64/kernel/entry.S | 43 +++++++++++++++++++++++++++++++++++++
drivers/irqchip/irq-gic-v3.c | 41 +++++++++++++++++++++++++++++++++++
4 files changed, 96 insertions(+)
diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index b39d620..1ed0476 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -374,5 +374,11 @@ static inline void gic_start_pmr_masking(void)
WARN_ON(true);
}
+static inline void gic_set_nmi_active(void)
+{
+ /* Should not get called */
+ WARN_ON(true);
+}
+
#endif /* !__ASSEMBLY__ */
#endif /* !__ASM_ARCH_GICV3_H */
diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h
index 23c88ac0..3196cf1 100644
--- a/arch/arm64/include/asm/arch_gicv3.h
+++ b/arch/arm64/include/asm/arch_gicv3.h
@@ -166,5 +166,11 @@ static inline void gic_start_pmr_masking(void)
asm volatile ("msr daifclr, #2" : : : "memory");
}
+/* Notify an NMI is active, blocking other NMIs */
+static inline void gic_set_nmi_active(void)
+{
+ asm volatile ("msr daifset, #2" : : : "memory");
+}
+
#endif /* __ASSEMBLY__ */
#endif /* __ASM_ARCH_GICV3_H */
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index f56f27e..0d0c829 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -391,6 +391,16 @@ alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0
mov sp, x19
.endm
+ /* Should be checked on return from irq handlers */
+ .macro branch_if_was_nmi, tmp, target
+ alternative_if ARM64_HAS_IRQ_PRIO_MASKING
+ mrs \tmp, daif
+ alternative_else
+ mov \tmp, #0
+ alternative_endif
+ tbnz \tmp, #7, \target // Exiting an NMI
+ .endm
+
/*
* These are the registers used in the syscall handler, and allow us to
* have in theory up to 7 arguments to a function - x0 to x6.
@@ -611,12 +621,30 @@ ENDPROC(el1_sync)
el1_irq:
kernel_entry 1
enable_da_f
+
#ifdef CONFIG_TRACE_IRQFLAGS
+#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS
+ ldr x20, [sp, #S_PMR_SAVE]
+ /* Irqs were disabled, don't trace */
+ tbz x20, ICC_PMR_EL1_EN_SHIFT, 1f
+#endif
bl trace_hardirqs_off
+1:
#endif
irq_handler
+#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS
+ /*
+ * Irqs were disabled, we have an nmi.
+ * We might have interrupted a context with interrupt disabled that set
+ * NEED_RESCHED flag.
+ * Skip preemption and irq tracing if needed.
+ */
+ tbz x20, ICC_PMR_EL1_EN_SHIFT, untraced_irq_exit
+ branch_if_was_nmi x0, skip_preempt
+#endif
+
#ifdef CONFIG_PREEMPT
ldr w24, [tsk, #TSK_TI_PREEMPT] // get preempt count
cbnz w24, 1f // preempt count != 0
@@ -625,9 +653,13 @@ el1_irq:
bl el1_preempt
1:
#endif
+
+skip_preempt:
#ifdef CONFIG_TRACE_IRQFLAGS
bl trace_hardirqs_on
#endif
+
+untraced_irq_exit:
kernel_exit 1
ENDPROC(el1_irq)
@@ -858,6 +890,9 @@ el0_irq_naked:
#ifdef CONFIG_TRACE_IRQFLAGS
bl trace_hardirqs_on
#endif
+
+ branch_if_was_nmi x2, nmi_ret_to_user
+
b ret_to_user
ENDPROC(el0_irq)
@@ -1353,3 +1388,11 @@ alternative_else_nop_endif
ENDPROC(__sdei_asm_handler)
NOKPROBE(__sdei_asm_handler)
#endif /* CONFIG_ARM_SDE_INTERFACE */
+
+/*
+ * NMI return path to EL0
+ */
+nmi_ret_to_user:
+ ldr x1, [tsk, #TSK_TI_FLAGS]
+ b finish_ret_to_user
+ENDPROC(nmi_ret_to_user)
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index b144f73..4be5996 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -41,6 +41,8 @@
#include "irq-gic-common.h"
+#define GICD_INT_NMI_PRI 0xa0
+
struct redist_region {
void __iomem *redist_base;
phys_addr_t phys_base;
@@ -253,6 +255,12 @@ static inline bool arch_uses_gic_prios(void)
return IS_ENABLED(CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS);
}
+static inline bool gic_supports_nmi(void)
+{
+ return arch_uses_gic_prios()
+ && static_branch_likely(&have_non_secure_prio_view);
+}
+
static int gic_irq_set_irqchip_state(struct irq_data *d,
enum irqchip_irq_state which, bool val)
{
@@ -371,6 +379,20 @@ static u64 gic_mpidr_to_affinity(unsigned long mpidr)
return aff;
}
+static void do_handle_nmi(unsigned int hwirq, struct pt_regs *regs)
+{
+ struct pt_regs *old_regs = set_irq_regs(regs);
+ unsigned int irq;
+
+ nmi_enter();
+
+ irq = irq_find_mapping(gic_data.domain, hwirq);
+ generic_handle_irq(irq);
+
+ nmi_exit();
+ set_irq_regs(old_regs);
+}
+
static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs)
{
u32 irqnr;
@@ -386,6 +408,23 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
if (likely(irqnr > 15 && irqnr < 1020) || irqnr >= 8192) {
int err;
+ if (gic_supports_nmi()
+ && unlikely(gic_read_rpr() == GICD_INT_NMI_PRI)) {
+ /*
+ * We need to prevent other NMIs to occur even after a
+ * priority drop.
+ * We keep I flag set until cpsr is restored from
+ * kernel_exit.
+ */
+ gic_set_nmi_active();
+
+ if (static_branch_likely(&supports_deactivate_key))
+ gic_write_eoir(irqnr);
+
+ do_handle_nmi(irqnr, regs);
+ return;
+ }
+
if (static_branch_likely(&supports_deactivate_key))
gic_write_eoir(irqnr);
else if (!arch_uses_gic_prios())
@@ -1183,6 +1222,8 @@ static int __init gic_init_bases(void __iomem *dist_base,
if (arch_uses_gic_prios()) {
if (!gic_has_group0() || gic_dist_security_disabled())
static_branch_enable(&have_non_secure_prio_view);
+ else
+ pr_warn("SCR_EL3.FIQ set, cannot enable use of pseudo-NMIs\n");
}
return 0;
--
1.9.1
^ permalink raw reply related
* [PATCH v4 23/26] irqchip/gic: Add functions to access irq priorities
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
Add accessors to the GIC distributor/redistributors priority registers.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
drivers/irqchip/irq-gic-common.c | 10 ++++++++++
drivers/irqchip/irq-gic-common.h | 2 ++
2 files changed, 12 insertions(+)
diff --git a/drivers/irqchip/irq-gic-common.c b/drivers/irqchip/irq-gic-common.c
index 01e673c..910746f 100644
--- a/drivers/irqchip/irq-gic-common.c
+++ b/drivers/irqchip/irq-gic-common.c
@@ -98,6 +98,16 @@ int gic_configure_irq(unsigned int irq, unsigned int type,
return ret;
}
+void gic_set_irq_prio(unsigned int irq, void __iomem *base, u8 prio)
+{
+ writeb_relaxed(prio, base + GIC_DIST_PRI + irq);
+}
+
+u8 gic_get_irq_prio(unsigned int irq, void __iomem *base)
+{
+ return readb_relaxed(base + GIC_DIST_PRI + irq);
+}
+
void gic_dist_config(void __iomem *base, int gic_irqs,
void (*sync_access)(void))
{
diff --git a/drivers/irqchip/irq-gic-common.h b/drivers/irqchip/irq-gic-common.h
index 3919cd7..1586dbd 100644
--- a/drivers/irqchip/irq-gic-common.h
+++ b/drivers/irqchip/irq-gic-common.h
@@ -35,6 +35,8 @@ void gic_dist_config(void __iomem *base, int gic_irqs,
void gic_cpu_config(void __iomem *base, void (*sync_access)(void));
void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,
void *data);
+void gic_set_irq_prio(unsigned int irq, void __iomem *base, u8 prio);
+u8 gic_get_irq_prio(unsigned int irq, void __iomem *base);
void gic_set_kvm_info(const struct gic_kvm_info *info);
--
1.9.1
^ permalink raw reply related
* [PATCH v4 22/26] arm64: Detect current view of GIC priorities
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
The values non secure EL1 needs to use for PMR and RPR registers depends on
the value of SCR_EL3.FIQ.
The values non secure EL1 sees from the distributor and redistributor
depend on whether security is enabled for the GIC or not.
Figure out what values we are dealing with to know if the values we use for
PMR and RPR match the priority values that have been configured in the
distributor and redistributors.
Also, add firmware requirements related to SCR_EL3.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
Documentation/arm64/booting.txt | 5 +++++
drivers/irqchip/irq-gic-v3.c | 42 +++++++++++++++++++++++++++++++++++------
2 files changed, 41 insertions(+), 6 deletions(-)
diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
index 8d0df62..e387938 100644
--- a/Documentation/arm64/booting.txt
+++ b/Documentation/arm64/booting.txt
@@ -188,6 +188,11 @@ Before jumping into the kernel, the following conditions must be met:
the kernel image will be entered must be initialised by software at a
higher exception level to prevent execution in an UNKNOWN state.
+ - SCR_EL3.FIQ must have the same value across all CPUs the kernel is
+ executing on.
+ - The value of SCR_EL3.FIQ must be the same as the one present at boot
+ time whenever the kernel is executing.
+
For systems with a GICv3 interrupt controller to be used in v3 mode:
- If EL3 is present:
ICC_SRE_EL3.Enable (bit 3) must be initialiased to 0b1.
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 2fd0440..b144f73 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -63,6 +63,28 @@ struct gic_chip_data {
static struct gic_chip_data gic_data __read_mostly;
static DEFINE_STATIC_KEY_TRUE(supports_deactivate_key);
+/*
+ * The behaviours of RPR and PMR registers differ depending on the value of
+ * SCR_EL3.FIQ, and the behaviour of non-secure priority registers of the
+ * distributor and redistributors depends on whether security is enabled in the
+ * GIC.
+ *
+ * When security is enabled, non-secure priority values from the (re)distributor
+ * are presented to the GIC CPUIF as follow:
+ * (GIC_(R)DIST_PRI[irq] >> 1) | 0x80;
+ *
+ * If SCR_EL3.FIQ == 1, the values writen to/read from PMR and RPR at non-secure
+ * EL1 are subject to a similar operation thus matching the priorities presented
+ * from the (re)distributor when security is enabled.
+ *
+ * see GICv3/GICv4 Architecture Specification (IHI0069D):
+ * - section 4.8.1 Non-secure accesses to register fields for Secure interrupt
+ * priorities.
+ * - Figure 4-7 Secure read of the priority field for a Non-secure Group 1
+ * interrupt.
+ */
+DEFINE_STATIC_KEY_FALSE(have_non_secure_prio_view);
+
static struct gic_kvm_info gic_v3_kvm_info;
static DEFINE_PER_CPU(bool, has_rss);
@@ -573,6 +595,12 @@ static void gic_update_vlpi_properties(void)
!gic_data.rdists.has_direct_lpi ? "no " : "");
}
+/* Check whether it's single security state view */
+static inline bool gic_dist_security_disabled(void)
+{
+ return readl_relaxed(gic_data.dist_base + GICD_CTLR) & GICD_CTLR_DS;
+}
+
static void gic_cpu_sys_reg_init(void)
{
int i, cpu = smp_processor_id();
@@ -598,6 +626,9 @@ static void gic_cpu_sys_reg_init(void)
/* Set priority mask register */
if (!arch_uses_gic_prios())
write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
+ else if (static_branch_likely(&have_non_secure_prio_view) && group0)
+ /* Mismatch configuration with boot CPU */
+ WARN_ON(group0 && !gic_dist_security_disabled());
/*
* Some firmwares hand over to the kernel with the BPR changed from
@@ -850,12 +881,6 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
#endif
#ifdef CONFIG_CPU_PM
-/* Check whether it's single security state view */
-static bool gic_dist_security_disabled(void)
-{
- return readl_relaxed(gic_data.dist_base + GICD_CTLR) & GICD_CTLR_DS;
-}
-
static int gic_cpu_pm_notifier(struct notifier_block *self,
unsigned long cmd, void *v)
{
@@ -1155,6 +1180,11 @@ static int __init gic_init_bases(void __iomem *dist_base,
gic_cpu_init();
gic_cpu_pm_init();
+ if (arch_uses_gic_prios()) {
+ if (!gic_has_group0() || gic_dist_security_disabled())
+ static_branch_enable(&have_non_secure_prio_view);
+ }
+
return 0;
out_free:
--
1.9.1
^ permalink raw reply related
* [PATCH v4 21/26] arm64: Add build option for IRQ masking via priority
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
Provide a build option to enable using GICv3 priorities to enable/disable
interrupts.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/Kconfig | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index eb2cf49..ab214b9 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -938,6 +938,21 @@ config HARDEN_EL2_VECTORS
If unsure, say Y.
+config USE_ICC_SYSREGS_FOR_IRQFLAGS
+ bool "Use ICC system registers for IRQ masking"
+ select CONFIG_ARM_GIC_V3
+ help
+ Using the ICC system registers for IRQ masking makes it possible
+ to simulate NMI on ARM64 systems. This allows several interesting
+ features (especially debug features) to be used on these systems.
+
+ Say Y here to implement IRQ masking using ICC system
+ registers when the GIC System Registers are available. The changes
+ are applied dynamically using the alternatives system so it is safe
+ to enable this option on systems with older interrupt controllers.
+
+ If unsure, say N
+
menuconfig ARMV8_DEPRECATED
bool "Emulate deprecated/obsolete ARMv8 instructions"
depends on COMPAT
--
1.9.1
^ permalink raw reply related
* [PATCH v4 20/26] arm64: Switch to PMR masking when starting CPUs
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
Once the boot CPU has been prepared or a new secondary CPU has been
brought up, use ICC_PMR_EL1 to mask interrupts on that CPU and clear
PSR.I bit.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/include/asm/irqflags.h | 3 +++
arch/arm64/kernel/head.S | 35 +++++++++++++++++++++++++++++++++++
arch/arm64/kernel/smp.c | 5 +++++
3 files changed, 43 insertions(+)
diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
index 193cfd0..d31e9b6 100644
--- a/arch/arm64/include/asm/irqflags.h
+++ b/arch/arm64/include/asm/irqflags.h
@@ -153,5 +153,8 @@ static inline int arch_irqs_disabled_flags(unsigned long flags)
return (ARCH_FLAGS_GET_DAIF(flags) & (PSR_I_BIT)) |
!(ARCH_FLAGS_GET_PMR(flags) & ICC_PMR_EL1_EN_BIT);
}
+
+void maybe_switch_to_sysreg_gic_cpuif(void);
+
#endif
#endif
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index b085306..ba73690 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -648,6 +648,41 @@ set_cpu_boot_mode_flag:
ENDPROC(set_cpu_boot_mode_flag)
/*
+ * void maybe_switch_to_sysreg_gic_cpuif(void)
+ *
+ * Enable interrupt controller system register access if this feature
+ * has been detected by the alternatives system.
+ *
+ * Before we jump into generic code we must enable interrupt controller system
+ * register access because this is required by the irqflags macros. We must
+ * also mask interrupts at the PMR and unmask them within the PSR. That leaves
+ * us set up and ready for the kernel to make its first call to
+ * arch_local_irq_enable().
+ *
+ */
+ENTRY(maybe_switch_to_sysreg_gic_cpuif)
+alternative_if_not ARM64_HAS_IRQ_PRIO_MASKING
+ b 1f
+alternative_else
+ mrs_s x0, SYS_ICC_SRE_EL1
+alternative_endif
+ orr x0, x0, #1
+ msr_s SYS_ICC_SRE_EL1, x0 // Set ICC_SRE_EL1.SRE==1
+ isb // Make sure SRE is now set
+ mrs x0, daif
+ tbz x0, #7, no_mask_pmr // Are interrupts on?
+ mov x0, ICC_PMR_EL1_MASKED
+ msr_s SYS_ICC_PMR_EL1, x0 // Prepare for unmask of I bit
+ msr daifclr, #2 // Clear the I bit
+ b 1f
+no_mask_pmr:
+ mov x0, ICC_PMR_EL1_UNMASKED
+ msr_s SYS_ICC_PMR_EL1, x0
+1:
+ ret
+ENDPROC(maybe_switch_to_sysreg_gic_cpuif)
+
+/*
* These values are written with the MMU off, but read with the MMU on.
* Writers will invalidate the corresponding address, discarding up to a
* 'Cache Writeback Granule' (CWG) worth of data. The linker script ensures
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index b7fb909..3f39d8c 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -185,6 +185,8 @@ asmlinkage void secondary_start_kernel(void)
struct mm_struct *mm = &init_mm;
unsigned int cpu;
+ maybe_switch_to_sysreg_gic_cpuif();
+
cpu = task_cpu(current);
set_my_cpu_offset(per_cpu_offset(cpu));
@@ -417,6 +419,9 @@ void __init smp_prepare_boot_cpu(void)
* and/or scheduling is enabled.
*/
apply_boot_alternatives();
+
+ /* Conditionally switch to GIC PMR for interrupt masking */
+ maybe_switch_to_sysreg_gic_cpuif();
}
static u64 __init of_get_cpu_mpidr(struct device_node *dn)
--
1.9.1
^ permalink raw reply related
* [PATCH v4 19/26] irqchip/gic-v3: Switch to PMR masking after IRQ acknowledge
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
After an interrupt has been acknowledged, mask the IRQ priority through
PMR and clear PSR.I bit, allowing higher priority interrupts to be
received during interrupt handling.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm/include/asm/arch_gicv3.h | 6 ++++++
arch/arm64/include/asm/arch_gicv3.h | 6 ++++++
drivers/irqchip/irq-gic-v3.c | 8 +++++++-
3 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index 58d5d3e..b39d620 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -368,5 +368,11 @@ static inline bool gic_prio_masking_enabled(void)
return false;
}
+static inline void gic_start_pmr_masking(void)
+{
+ /* Should not get called */
+ WARN_ON(true);
+}
+
#endif /* !__ASSEMBLY__ */
#endif /* !__ASM_ARCH_GICV3_H */
diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h
index 98b09db..23c88ac0 100644
--- a/arch/arm64/include/asm/arch_gicv3.h
+++ b/arch/arm64/include/asm/arch_gicv3.h
@@ -160,5 +160,11 @@ static inline bool gic_prio_masking_enabled(void)
return cpus_have_const_cap(ARM64_HAS_IRQ_PRIO_MASKING);
}
+static inline void gic_start_pmr_masking(void)
+{
+ gic_write_pmr(ICC_PMR_EL1_MASKED);
+ asm volatile ("msr daifclr, #2" : : : "memory");
+}
+
#endif /* __ASSEMBLY__ */
#endif /* __ASM_ARCH_GICV3_H */
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index fc477e2..2fd0440 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -355,12 +355,18 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
irqnr = gic_read_iar();
+ if (arch_uses_gic_prios()) {
+ isb();
+ /* Masking IRQs earlier would prevent to ack the current interrupt */
+ gic_start_pmr_masking();
+ }
+
if (likely(irqnr > 15 && irqnr < 1020) || irqnr >= 8192) {
int err;
if (static_branch_likely(&supports_deactivate_key))
gic_write_eoir(irqnr);
- else
+ else if (!arch_uses_gic_prios())
isb();
err = handle_domain_irq(gic_data.domain, irqnr, regs);
--
1.9.1
^ permalink raw reply related
* [PATCH v4 18/26] irqchip/gic-v3: Do not overwrite PMR value
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
If the architecture is using ICC_PMR_EL1 to mask IRQs, do not overwrite
that value.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
drivers/irqchip/irq-gic-v3.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 13d283d..fc477e2 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -226,6 +226,11 @@ static void gic_unmask_irq(struct irq_data *d)
gic_poke_irq(d, GICD_ISENABLER);
}
+static inline bool arch_uses_gic_prios(void)
+{
+ return IS_ENABLED(CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS);
+}
+
static int gic_irq_set_irqchip_state(struct irq_data *d,
enum irqchip_irq_state which, bool val)
{
@@ -404,6 +409,9 @@ static u32 gic_get_pribits(void)
static bool gic_has_group0(void)
{
u32 val;
+ u32 old_pmr;
+
+ old_pmr = gic_read_pmr();
/*
* Let's find out if Group0 is under control of EL3 or not by
@@ -419,6 +427,8 @@ static bool gic_has_group0(void)
gic_write_pmr(BIT(8 - gic_get_pribits()));
val = gic_read_pmr();
+ gic_write_pmr(old_pmr);
+
return val != 0;
}
@@ -580,7 +590,8 @@ static void gic_cpu_sys_reg_init(void)
group0 = gic_has_group0();
/* Set priority mask register */
- write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
+ if (!arch_uses_gic_prios())
+ write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
/*
* Some firmwares hand over to the kernel with the BPR changed from
--
1.9.1
^ permalink raw reply related
* [PATCH v4 17/26] irqchip/gic-v3: Factor group0 detection into functions
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
The code to detect whether Linux has access to group0 interrupts can
prove useful in other parts of the driver.
Provide a separate function to do this.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
drivers/irqchip/irq-gic-v3.c | 55 +++++++++++++++++++++++++++++---------------
1 file changed, 36 insertions(+), 19 deletions(-)
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 0c58db3..13d283d 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -389,6 +389,39 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
}
}
+static u32 gic_get_pribits(void)
+{
+ u32 pribits;
+
+ pribits = gic_read_ctlr();
+ pribits &= ICC_CTLR_EL1_PRI_BITS_MASK;
+ pribits >>= ICC_CTLR_EL1_PRI_BITS_SHIFT;
+ pribits++;
+
+ return pribits;
+}
+
+static bool gic_has_group0(void)
+{
+ u32 val;
+
+ /*
+ * Let's find out if Group0 is under control of EL3 or not by
+ * setting the highest possible, non-zero priority in PMR.
+ *
+ * If SCR_EL3.FIQ is set, the priority gets shifted down in
+ * order for the CPU interface to set bit 7, and keep the
+ * actual priority in the non-secure range. In the process, it
+ * looses the least significant bit and the actual priority
+ * becomes 0x80. Reading it back returns 0, indicating that
+ * we're don't have access to Group0.
+ */
+ gic_write_pmr(BIT(8 - gic_get_pribits()));
+ val = gic_read_pmr();
+
+ return val != 0;
+}
+
static void __init gic_dist_init(void)
{
unsigned int i;
@@ -530,7 +563,7 @@ static void gic_cpu_sys_reg_init(void)
u64 mpidr = cpu_logical_map(cpu);
u64 need_rss = MPIDR_RS(mpidr);
bool group0;
- u32 val, pribits;
+ u32 pribits;
/*
* Need to check that the SRE bit has actually been set. If
@@ -542,25 +575,9 @@ static void gic_cpu_sys_reg_init(void)
if (!gic_enable_sre())
pr_err("GIC: unable to set SRE (disabled at EL2), panic ahead\n");
- pribits = gic_read_ctlr();
- pribits &= ICC_CTLR_EL1_PRI_BITS_MASK;
- pribits >>= ICC_CTLR_EL1_PRI_BITS_SHIFT;
- pribits++;
+ pribits = gic_get_pribits();
- /*
- * Let's find out if Group0 is under control of EL3 or not by
- * setting the highest possible, non-zero priority in PMR.
- *
- * If SCR_EL3.FIQ is set, the priority gets shifted down in
- * order for the CPU interface to set bit 7, and keep the
- * actual priority in the non-secure range. In the process, it
- * looses the least significant bit and the actual priority
- * becomes 0x80. Reading it back returns 0, indicating that
- * we're don't have access to Group0.
- */
- write_gicreg(BIT(8 - pribits), ICC_PMR_EL1);
- val = read_gicreg(ICC_PMR_EL1);
- group0 = val != 0;
+ group0 = gic_has_group0();
/* Set priority mask register */
write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
--
1.9.1
^ permalink raw reply related
* [PATCH v4 16/26] arm64: daifflags: Include PMR in daifflags restore operations
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
The addition of PMR should not bypass the semantics of daifflags.
When DA_F are set, I bit is also set as no interrupts (even of higher
priority) is allowed.
When DA_F are cleared, I bit is cleared and interrupt enabling/disabling
goes through ICC_PMR_EL1.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: James Morse <james.morse@arm.com>
---
arch/arm64/include/asm/daifflags.h | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h
index 8d91f22..3ae8139 100644
--- a/arch/arm64/include/asm/daifflags.h
+++ b/arch/arm64/include/asm/daifflags.h
@@ -18,8 +18,17 @@
#include <linux/irqflags.h>
-#define DAIF_PROCCTX 0
-#define DAIF_PROCCTX_NOIRQ PSR_I_BIT
+#include <asm/arch_gicv3.h>
+
+#define DAIF_PROCCTX \
+ (gic_prio_masking_enabled() ? \
+ MAKE_ARCH_FLAGS(0, ICC_PMR_EL1_UNMASKED) : \
+ 0)
+
+#define DAIF_PROCCTX_NOIRQ \
+ (gic_prio_masking_enabled() ? \
+ MAKE_ARCH_FLAGS(0, ICC_PMR_EL1_MASKED) : \
+ PSR_I_BIT)
/* mask/save/unmask/restore all exceptions, including interrupts. */
static inline void local_daif_mask(void)
@@ -51,6 +60,10 @@ static inline void local_daif_unmask(void)
:
:
: "memory");
+
+ /* Unmask IRQs in PMR if needed */
+ if (gic_prio_masking_enabled())
+ arch_local_irq_enable();
}
static inline void local_daif_restore(unsigned long flags)
--
1.9.1
^ permalink raw reply related
* [PATCH v4 15/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
Instead disabling interrupts by setting the PSR.I bit, use a priority
higher than the one used for interrupts to mask them via PMR.
The value chosen for PMR to enable/disable interrupts encodes the status
of interrupts on a single bit. This information is stored in the irqflags
values used when saving/restoring IRQ status.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Oleg Nesterov <oleg@redhat.com>
---
arch/arm64/include/asm/assembler.h | 17 ++++++-
arch/arm64/include/asm/efi.h | 3 +-
arch/arm64/include/asm/irqflags.h | 97 ++++++++++++++++++++++++++++++--------
arch/arm64/include/asm/ptrace.h | 10 ++--
arch/arm64/kernel/entry.S | 6 +--
5 files changed, 104 insertions(+), 29 deletions(-)
diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index 0bcc98d..0b2dcfd 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -23,6 +23,7 @@
#ifndef __ASM_ASSEMBLER_H
#define __ASM_ASSEMBLER_H
+#include <asm/alternative.h>
#include <asm/asm-offsets.h>
#include <asm/cpufeature.h>
#include <asm/debug-monitors.h>
@@ -62,12 +63,24 @@
/*
* Enable and disable interrupts.
*/
- .macro disable_irq
+ .macro disable_irq, tmp
+ mov \tmp, #ICC_PMR_EL1_MASKED
+alternative_if_not ARM64_HAS_IRQ_PRIO_MASKING
msr daifset, #2
+alternative_else
+ msr_s SYS_ICC_PMR_EL1, \tmp
+alternative_endif
.endm
- .macro enable_irq
+ .macro enable_irq, tmp
+ mov \tmp, #ICC_PMR_EL1_UNMASKED
+alternative_if_not ARM64_HAS_IRQ_PRIO_MASKING
msr daifclr, #2
+ nop
+alternative_else
+ msr_s SYS_ICC_PMR_EL1, \tmp
+ dsb sy
+alternative_endif
.endm
.macro save_and_disable_irq, flags
diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
index 192d791..a4e0730 100644
--- a/arch/arm64/include/asm/efi.h
+++ b/arch/arm64/include/asm/efi.h
@@ -42,7 +42,8 @@
efi_status_t __efi_rt_asm_wrapper(void *, const char *, ...);
-#define ARCH_EFI_IRQ_FLAGS_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT)
+#define ARCH_EFI_IRQ_FLAGS_MASK \
+ (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT | ARCH_FLAG_PMR_EN)
/* arch specific definitions used by the stub code */
diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
index 24692ed..193cfd0 100644
--- a/arch/arm64/include/asm/irqflags.h
+++ b/arch/arm64/include/asm/irqflags.h
@@ -18,7 +18,27 @@
#ifdef __KERNEL__
+#include <asm/alternative.h>
+#include <asm/cpufeature.h>
#include <asm/ptrace.h>
+#include <asm/sysreg.h>
+
+
+/*
+ * When ICC_PMR_EL1 is used for interrupt masking, only the bit indicating
+ * whether the normal interrupts are masked is kept along with the daif
+ * flags.
+ */
+#define ARCH_FLAG_PMR_EN 0x1
+
+#define MAKE_ARCH_FLAGS(daif, pmr) \
+ ((daif) | (((pmr) >> ICC_PMR_EL1_EN_SHIFT) & ARCH_FLAG_PMR_EN))
+
+#define ARCH_FLAGS_GET_PMR(flags) \
+ ((((flags) & ARCH_FLAG_PMR_EN) << ICC_PMR_EL1_EN_SHIFT) \
+ | ICC_PMR_EL1_MASKED)
+
+#define ARCH_FLAGS_GET_DAIF(flags) ((flags) & ~ARCH_FLAG_PMR_EN)
/*
* Aarch64 has flags for masking: Debug, Asynchronous (serror), Interrupts and
@@ -38,31 +58,50 @@
*/
static inline unsigned long arch_local_irq_save(void)
{
- unsigned long flags;
- asm volatile(
+ unsigned long flags, masked = ICC_PMR_EL1_MASKED;
+ unsigned long pmr = 0;
+
+ asm volatile(ALTERNATIVE(
"mrs %0, daif // arch_local_irq_save\n"
- "msr daifset, #2"
- : "=r" (flags)
- :
+ "msr daifset, #2\n"
+ "mov %1, #" __stringify(ICC_PMR_EL1_UNMASKED),
+ /* --- */
+ "mrs %0, daif\n"
+ "mrs_s %1, " __stringify(SYS_ICC_PMR_EL1) "\n"
+ "msr_s " __stringify(SYS_ICC_PMR_EL1) ", %2",
+ ARM64_HAS_IRQ_PRIO_MASKING)
+ : "=&r" (flags), "=&r" (pmr)
+ : "r" (masked)
: "memory");
- return flags;
+
+ return MAKE_ARCH_FLAGS(flags, pmr);
}
static inline void arch_local_irq_enable(void)
{
- asm volatile(
- "msr daifclr, #2 // arch_local_irq_enable"
- :
+ unsigned long unmasked = ICC_PMR_EL1_UNMASKED;
+
+ asm volatile(ALTERNATIVE(
+ "msr daifclr, #2 // arch_local_irq_enable\n"
+ "nop",
+ "msr_s " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
+ "dsb sy",
+ ARM64_HAS_IRQ_PRIO_MASKING)
:
+ : "r" (unmasked)
: "memory");
}
static inline void arch_local_irq_disable(void)
{
- asm volatile(
- "msr daifset, #2 // arch_local_irq_disable"
- :
+ unsigned long masked = ICC_PMR_EL1_MASKED;
+
+ asm volatile(ALTERNATIVE(
+ "msr daifset, #2 // arch_local_irq_disable",
+ "msr_s " __stringify(SYS_ICC_PMR_EL1) ",%0",
+ ARM64_HAS_IRQ_PRIO_MASKING)
:
+ : "r" (masked)
: "memory");
}
@@ -72,12 +111,19 @@ static inline void arch_local_irq_disable(void)
static inline unsigned long arch_local_save_flags(void)
{
unsigned long flags;
- asm volatile(
- "mrs %0, daif // arch_local_save_flags"
- : "=r" (flags)
+ unsigned long pmr = 0;
+
+ asm volatile(ALTERNATIVE(
+ "mrs %0, daif // arch_local_save_flags\n"
+ "mov %1, #" __stringify(ICC_PMR_EL1_UNMASKED),
+ "mrs %0, daif\n"
+ "mrs_s %1, " __stringify(SYS_ICC_PMR_EL1),
+ ARM64_HAS_IRQ_PRIO_MASKING)
+ : "=r" (flags), "=r" (pmr)
:
: "memory");
- return flags;
+
+ return MAKE_ARCH_FLAGS(flags, pmr);
}
/*
@@ -85,16 +131,27 @@ static inline unsigned long arch_local_save_flags(void)
*/
static inline void arch_local_irq_restore(unsigned long flags)
{
- asm volatile(
- "msr daif, %0 // arch_local_irq_restore"
+ unsigned long pmr = ARCH_FLAGS_GET_PMR(flags);
+
+ flags = ARCH_FLAGS_GET_DAIF(flags);
+
+ asm volatile(ALTERNATIVE(
+ "msr daif, %0 // arch_local_irq_restore\n"
+ "nop\n"
+ "nop",
+ "msr daif, %0\n"
+ "msr_s " __stringify(SYS_ICC_PMR_EL1) ",%1\n"
+ "dsb sy",
+ ARM64_HAS_IRQ_PRIO_MASKING)
:
- : "r" (flags)
+ : "r" (flags), "r" (pmr)
: "memory");
}
static inline int arch_irqs_disabled_flags(unsigned long flags)
{
- return flags & PSR_I_BIT;
+ return (ARCH_FLAGS_GET_DAIF(flags) & (PSR_I_BIT)) |
+ !(ARCH_FLAGS_GET_PMR(flags) & ICC_PMR_EL1_EN_BIT);
}
#endif
#endif
diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h
index e87aef7..3ec58a4 100644
--- a/arch/arm64/include/asm/ptrace.h
+++ b/arch/arm64/include/asm/ptrace.h
@@ -25,8 +25,11 @@
#define CurrentEL_EL1 (1 << 2)
#define CurrentEL_EL2 (2 << 2)
-/* PMR value use to unmask interrupts */
+/* PMR values used to mask/unmask interrupts */
#define ICC_PMR_EL1_UNMASKED 0xf0
+#define ICC_PMR_EL1_EN_SHIFT 6
+#define ICC_PMR_EL1_EN_BIT (1 << ICC_PMR_EL1_EN_SHIFT) // PMR IRQ enable
+#define ICC_PMR_EL1_MASKED (ICC_PMR_EL1_UNMASKED ^ ICC_PMR_EL1_EN_BIT)
/* AArch32-specific ptrace requests */
#define COMPAT_PTRACE_GETREGS 12
@@ -174,8 +177,9 @@ static inline void forget_syscall(struct pt_regs *regs)
#define processor_mode(regs) \
((regs)->pstate & PSR_MODE_MASK)
-#define interrupts_enabled(regs) \
- (!((regs)->pstate & PSR_I_BIT))
+#define interrupts_enabled(regs) \
+ ((!((regs)->pstate & PSR_I_BIT)) && \
+ ((regs)->pmr_save & ICC_PMR_EL1_EN_BIT))
#define fast_interrupts_enabled(regs) \
(!((regs)->pstate & PSR_F_BIT))
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 78e4ff4..f56f27e 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -888,7 +888,7 @@ ENDPROC(el0_error)
* and this includes saving x0 back into the kernel stack.
*/
ret_fast_syscall:
- disable_irq // disable interrupts
+ disable_irq x21 // disable interrupts
str x0, [sp, #S_X0] // returned x0
ldr x1, [tsk, #TSK_TI_FLAGS] // re-check for syscall tracing
and x2, x1, #_TIF_SYSCALL_WORK
@@ -898,7 +898,7 @@ ret_fast_syscall:
enable_step_tsk x1, x2
kernel_exit 0
ret_fast_syscall_trace:
- enable_irq // enable interrupts
+ enable_irq x0 // enable interrupts
b __sys_trace_return_skipped // we already saved x0
/*
@@ -916,7 +916,7 @@ work_pending:
* "slow" syscall return path.
*/
ret_to_user:
- disable_irq // disable interrupts
+ disable_irq x21 // disable interrupts
ldr x1, [tsk, #TSK_TI_FLAGS]
and x2, x1, #_TIF_WORK_MASK
cbnz x2, work_pending
--
1.9.1
^ permalink raw reply related
* [PATCH v4 14/26] arm64: kvm: Unmask PMR before entering guest
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
Interrupts masked by ICC_PMR_EL1 will not be signaled to the CPU. This
means that hypervisor will not receive masked interrupts while running a
guest.
Avoid this by making sure ICC_PMR_EL1 is unmasked when we enter a guest.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Christoffer Dall <christoffer.dall@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: kvmarm at lists.cs.columbia.edu
---
arch/arm64/include/asm/kvm_host.h | 12 ++++++++++++
arch/arm64/kvm/hyp/switch.c | 17 +++++++++++++++++
2 files changed, 29 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 469de8a..bfd9e5b 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -24,6 +24,7 @@
#include <linux/types.h>
#include <linux/kvm_types.h>
+#include <asm/arch_gicv3.h>
#include <asm/cpufeature.h>
#include <asm/daifflags.h>
#include <asm/fpsimd.h>
@@ -433,6 +434,17 @@ static inline void kvm_fpsimd_flush_cpu_state(void)
static inline void kvm_arm_vhe_guest_enter(void)
{
local_daif_mask();
+
+ /*
+ * Having IRQs masked via PMR when entering the guest means the GIC
+ * will not signal the CPU of interrupts of lower priority, and the
+ * only way to get out will be via guest exceptions.
+ * Naturally, we want to avoid this.
+ */
+ if (gic_prio_masking_enabled()) {
+ gic_write_pmr(ICC_PMR_EL1_UNMASKED);
+ dsb(sy);
+ }
}
static inline void kvm_arm_vhe_guest_exit(void)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index d964523..f00a04c 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -21,6 +21,7 @@
#include <kvm/arm_psci.h>
+#include <asm/arch_gicv3.h>
#include <asm/kvm_asm.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
@@ -442,6 +443,19 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu)
struct kvm_cpu_context *guest_ctxt;
bool fp_enabled;
u64 exit_code;
+ u32 host_pmr = ICC_PMR_EL1_UNMASKED;
+
+ /*
+ * Having IRQs masked via PMR when entering the guest means the GIC
+ * will not signal the CPU of interrupts of lower priority, and the
+ * only way to get out will be via guest exceptions.
+ * Naturally, we want to avoid this.
+ */
+ if (gic_prio_masking_enabled()) {
+ host_pmr = gic_read_pmr();
+ gic_write_pmr(ICC_PMR_EL1_UNMASKED);
+ dsb(sy);
+ }
vcpu = kern_hyp_va(vcpu);
@@ -496,6 +510,9 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu)
*/
__debug_switch_to_host(vcpu);
+ if (gic_prio_masking_enabled())
+ gic_write_pmr(host_pmr);
+
return exit_code;
}
--
1.9.1
^ permalink raw reply related
* [PATCH v4 13/26] arm/arm64: gic-v3: Add helper functions to manage IRQ priorities
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
Add a function to check if priority masking is supported and accessors
for PMR/RPR.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm/include/asm/arch_gicv3.h | 21 +++++++++++++++++++++
arch/arm64/include/asm/arch_gicv3.h | 20 ++++++++++++++++++++
2 files changed, 41 insertions(+)
diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index 0bd5307..58d5d3e 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -34,6 +34,7 @@
#define ICC_SRE __ACCESS_CP15(c12, 0, c12, 5)
#define ICC_IGRPEN1 __ACCESS_CP15(c12, 0, c12, 7)
#define ICC_BPR1 __ACCESS_CP15(c12, 0, c12, 3)
+#define ICC_RPR __ACCESS_CP15(c12, 0, c11, 3)
#define __ICC_AP0Rx(x) __ACCESS_CP15(c12, 0, c8, 4 | x)
#define ICC_AP0R0 __ICC_AP0Rx(0)
@@ -245,6 +246,21 @@ static inline void gic_write_bpr1(u32 val)
write_sysreg(val, ICC_BPR1);
}
+static inline u32 gic_read_pmr(void)
+{
+ return read_sysreg(ICC_PMR);
+}
+
+static inline void gic_write_pmr(u32 val)
+{
+ write_sysreg(val, ICC_PMR);
+}
+
+static inline u32 gic_read_rpr(void)
+{
+ return read_sysreg(ICC_RPR);
+}
+
/*
* Even in 32bit systems that use LPAE, there is no guarantee that the I/O
* interface provides true 64bit atomic accesses, so using strd/ldrd doesn't
@@ -347,5 +363,10 @@ static inline void gits_write_vpendbaser(u64 val, void * __iomem addr)
#define gits_read_vpendbaser(c) __gic_readq_nonatomic(c)
+static inline bool gic_prio_masking_enabled(void)
+{
+ return false;
+}
+
#endif /* !__ASSEMBLY__ */
#endif /* !__ASM_ARCH_GICV3_H */
diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h
index e278f94..98b09db 100644
--- a/arch/arm64/include/asm/arch_gicv3.h
+++ b/arch/arm64/include/asm/arch_gicv3.h
@@ -114,6 +114,21 @@ static inline void gic_write_bpr1(u32 val)
write_sysreg_s(val, SYS_ICC_BPR1_EL1);
}
+static inline u32 gic_read_pmr(void)
+{
+ return read_sysreg_s(SYS_ICC_PMR_EL1);
+}
+
+static inline void gic_write_pmr(u32 val)
+{
+ write_sysreg_s(val, SYS_ICC_PMR_EL1);
+}
+
+static inline u32 gic_read_rpr(void)
+{
+ return read_sysreg_s(SYS_ICC_RPR_EL1);
+}
+
#define gic_read_typer(c) readq_relaxed(c)
#define gic_write_irouter(v, c) writeq_relaxed(v, c)
#define gic_read_lpir(c) readq_relaxed(c)
@@ -140,5 +155,10 @@ static inline void gic_write_bpr1(u32 val)
#define gits_write_vpendbaser(v, c) writeq_relaxed(v, c)
#define gits_read_vpendbaser(c) readq_relaxed(c)
+static inline bool gic_prio_masking_enabled(void)
+{
+ return cpus_have_const_cap(ARM64_HAS_IRQ_PRIO_MASKING);
+}
+
#endif /* __ASSEMBLY__ */
#endif /* __ASM_ARCH_GICV3_H */
--
1.9.1
^ permalink raw reply related
* [PATCH v4 12/26] arm64: Unmask PMR before going idle
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
CPU does not received signals for interrupts with a priority masked by
ICC_PMR_EL1. This means the CPU might not come back from a WFI
instruction.
Make sure ICC_PMR_EL1 does not mask interrupts when doing a WFI.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/mm/proc.S | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 5f9a73a..e7cafbf 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -20,6 +20,7 @@
#include <linux/init.h>
#include <linux/linkage.h>
+#include <linux/irqchip/arm-gic-v3.h>
#include <asm/assembler.h>
#include <asm/asm-offsets.h>
#include <asm/hwcap.h>
@@ -53,10 +54,27 @@
* cpu_do_idle()
*
* Idle the processor (wait for interrupt).
+ *
+ * If the CPU supports priority masking we must do additional work to
+ * ensure that interrupts are not masked at the PMR (because the core will
+ * not wake up if we block the wake up signal in the interrupt controller).
*/
ENTRY(cpu_do_idle)
+alternative_if_not ARM64_HAS_IRQ_PRIO_MASKING
+ dsb sy // WFI may enter a low-power mode
+ wfi
+ ret
+alternative_else
+ mrs x0, daif // save I bit
+ msr daifset, #2 // set I bit
+ mrs_s x1, SYS_ICC_PMR_EL1 // save PMR
+alternative_endif
+ mov x2, #ICC_PMR_EL1_UNMASKED
+ msr_s SYS_ICC_PMR_EL1, x2 // unmask at PMR
dsb sy // WFI may enter a low-power mode
wfi
+ msr_s SYS_ICC_PMR_EL1, x1 // restore PMR
+ msr daif, x0 // restore I bit
ret
ENDPROC(cpu_do_idle)
--
1.9.1
^ permalink raw reply related
* [PATCH v4 11/26] arm64: Make PMR part of task context
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
If ICC_PMR_EL1 is used to mask interrupts, its value should be
saved/restored whenever a task is context switched out/in or
gets an exception.
Add PMR to the registers to save in the pt_regs struct upon kernel entry,
and restore it before ERET. Also, initialize it to a sane value when
creating new tasks.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/include/asm/processor.h | 1 +
arch/arm64/include/asm/ptrace.h | 5 ++++-
arch/arm64/kernel/asm-offsets.c | 1 +
arch/arm64/kernel/entry.S | 16 ++++++++++++++++
arch/arm64/kernel/process.c | 2 ++
5 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 7675989..40bd0e1 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -163,6 +163,7 @@ static inline void start_thread_common(struct pt_regs *regs, unsigned long pc)
memset(regs, 0, sizeof(*regs));
forget_syscall(regs);
regs->pc = pc;
+ regs->pmr_save = ICC_PMR_EL1_UNMASKED;
}
static inline void start_thread(struct pt_regs *regs, unsigned long pc,
diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h
index 6069d66..e87aef7 100644
--- a/arch/arm64/include/asm/ptrace.h
+++ b/arch/arm64/include/asm/ptrace.h
@@ -25,6 +25,9 @@
#define CurrentEL_EL1 (1 << 2)
#define CurrentEL_EL2 (2 << 2)
+/* PMR value use to unmask interrupts */
+#define ICC_PMR_EL1_UNMASKED 0xf0
+
/* AArch32-specific ptrace requests */
#define COMPAT_PTRACE_GETREGS 12
#define COMPAT_PTRACE_SETREGS 13
@@ -136,7 +139,7 @@ struct pt_regs {
#endif
u64 orig_addr_limit;
- u64 unused; // maintain 16 byte alignment
+ u64 pmr_save;
u64 stackframe[2];
};
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 5bdda65..1f6a0a9 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -78,6 +78,7 @@ int main(void)
DEFINE(S_ORIG_X0, offsetof(struct pt_regs, orig_x0));
DEFINE(S_SYSCALLNO, offsetof(struct pt_regs, syscallno));
DEFINE(S_ORIG_ADDR_LIMIT, offsetof(struct pt_regs, orig_addr_limit));
+ DEFINE(S_PMR_SAVE, offsetof(struct pt_regs, pmr_save));
DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
BLANK();
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 20252d5..78e4ff4 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -230,6 +230,15 @@ alternative_else_nop_endif
msr sp_el0, tsk
.endif
+ /* Save pmr */
+alternative_if ARM64_HAS_IRQ_PRIO_MASKING
+ mrs_s x20, SYS_ICC_PMR_EL1
+alternative_else
+ /* Keep a sane value in the task context */
+ mov x20, ICC_PMR_EL1_UNMASKED
+alternative_endif
+ str x20, [sp, #S_PMR_SAVE]
+
/*
* Registers that may be useful after this macro is invoked:
*
@@ -250,6 +259,13 @@ alternative_else_nop_endif
/* No need to restore UAO, it will be restored from SPSR_EL1 */
.endif
+ /* Restore pmr */
+alternative_if ARM64_HAS_IRQ_PRIO_MASKING
+ ldr x20, [sp, #S_PMR_SAVE]
+ msr_s SYS_ICC_PMR_EL1, x20
+ dsb sy
+alternative_else_nop_endif
+
ldp x21, x22, [sp, #S_PC] // load ELR, SPSR
.if \el == 0
ct_user_enter
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index f08a2ed..fbc9886 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -230,6 +230,7 @@ void __show_regs(struct pt_regs *regs)
}
printk("sp : %016llx\n", sp);
+ printk("pmr_save: %08llx\n", regs->pmr_save);
i = top_reg;
@@ -355,6 +356,7 @@ int copy_thread(unsigned long clone_flags, unsigned long stack_start,
} else {
memset(childregs, 0, sizeof(struct pt_regs));
childregs->pstate = PSR_MODE_EL1h;
+ childregs->pmr_save = ICC_PMR_EL1_UNMASKED;
if (IS_ENABLED(CONFIG_ARM64_UAO) &&
cpus_have_const_cap(ARM64_HAS_UAO))
childregs->pstate |= PSR_UAO_BIT;
--
1.9.1
^ permalink raw reply related
* [PATCH v4 10/26] arm64: Delay daif masking for user return
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
Masking daif flags is done very early before returning to EL0.
Only toggle the interrupt masking while in the vector entry and mask daif
once in kernel_exit.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: James Morse <james.morse@arm.com>
---
arch/arm64/kernel/entry.S | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index ec2ee72..20252d5 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -240,9 +240,9 @@ alternative_else_nop_endif
.endm
.macro kernel_exit, el
- .if \el != 0
disable_daif
+ .if \el != 0
/* Restore the task's original addr_limit. */
ldr x20, [sp, #S_ORIG_ADDR_LIMIT]
str x20, [tsk, #TSK_TI_ADDR_LIMIT]
@@ -872,7 +872,7 @@ ENDPROC(el0_error)
* and this includes saving x0 back into the kernel stack.
*/
ret_fast_syscall:
- disable_daif
+ disable_irq // disable interrupts
str x0, [sp, #S_X0] // returned x0
ldr x1, [tsk, #TSK_TI_FLAGS] // re-check for syscall tracing
and x2, x1, #_TIF_SYSCALL_WORK
@@ -882,7 +882,7 @@ ret_fast_syscall:
enable_step_tsk x1, x2
kernel_exit 0
ret_fast_syscall_trace:
- enable_daif
+ enable_irq // enable interrupts
b __sys_trace_return_skipped // we already saved x0
/*
@@ -900,7 +900,7 @@ work_pending:
* "slow" syscall return path.
*/
ret_to_user:
- disable_daif
+ disable_irq // disable interrupts
ldr x1, [tsk, #TSK_TI_FLAGS]
and x2, x1, #_TIF_WORK_MASK
cbnz x2, work_pending
--
1.9.1
^ permalink raw reply related
* [PATCH v4 09/26] arm64: Use daifflag_restore after bp_hardening
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
For EL0 entries requiring bp_hardening, daif status is kept at
DAIF_PROCCTX_NOIRQ until after hardening has been done. Then interrupts
are enabled through local_irq_enable().
Before using local_irq_* functions, daifflags should be properly restored
to a state where IRQs are enabled.
Enable IRQs by restoring DAIF_PROCCTX state after bp hardening.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: James Morse <james.morse@arm.com>
---
arch/arm64/mm/fault.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
Note:
This doesn't introduce a real change in behaviour, but once PMR
is used for interrupt masking, PSR.I bit needs to be cleared and
local_irq_enable won't do that anymore.
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 4165485..7a18634 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -37,6 +37,7 @@
#include <asm/cmpxchg.h>
#include <asm/cpufeature.h>
#include <asm/exception.h>
+#include <asm/daifflags.h>
#include <asm/debug-monitors.h>
#include <asm/esr.h>
#include <asm/sysreg.h>
@@ -712,7 +713,7 @@ asmlinkage void __exception do_el0_ia_bp_hardening(unsigned long addr,
if (addr > TASK_SIZE)
arm64_apply_bp_hardening();
- local_irq_enable();
+ local_daif_restore(DAIF_PROCCTX);
do_mem_abort(addr, esr, regs);
}
@@ -726,7 +727,7 @@ asmlinkage void __exception do_sp_pc_abort(unsigned long addr,
if (user_mode(regs)) {
if (instruction_pointer(regs) > TASK_SIZE)
arm64_apply_bp_hardening();
- local_irq_enable();
+ local_daif_restore(DAIF_PROCCTX);
}
info.si_signo = SIGBUS;
--
1.9.1
^ permalink raw reply related
* [PATCH v4 08/26] arm64: daifflags: Use irqflags functions for daifflags
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
Some of the work done in daifflags save/restore is already provided
by irqflags functions. Daifflags should always be a superset of irqflags
(it handles irq status + status of other flags). Modifying behaviour of
irqflags should alter the behaviour of daifflags.
Use irqflags_save/restore functions for the corresponding daifflags
operation.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: James Morse <james.morse@arm.com>
---
arch/arm64/include/asm/daifflags.h | 15 +++++----------
1 file changed, 5 insertions(+), 10 deletions(-)
Note:
A latter patch changes the behaviour of arch_local_flags but the daif
flag still need to follow the semantics of arch_local_flags.
diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h
index 22e4c83..8d91f22 100644
--- a/arch/arm64/include/asm/daifflags.h
+++ b/arch/arm64/include/asm/daifflags.h
@@ -36,11 +36,8 @@ static inline unsigned long local_daif_save(void)
{
unsigned long flags;
- asm volatile(
- "mrs %0, daif // local_daif_save\n"
- : "=r" (flags)
- :
- : "memory");
+ flags = arch_local_save_flags();
+
local_daif_mask();
return flags;
@@ -60,11 +57,9 @@ static inline void local_daif_restore(unsigned long flags)
{
if (!arch_irqs_disabled_flags(flags))
trace_hardirqs_on();
- asm volatile(
- "msr daif, %0 // local_daif_restore"
- :
- : "r" (flags)
- : "memory");
+
+ arch_local_irq_restore(flags);
+
if (arch_irqs_disabled_flags(flags))
trace_hardirqs_off();
}
--
1.9.1
^ permalink raw reply related
* [PATCH v4 07/26] irqchip/gic-v3: Remove acknowledge loop
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
Multiple interrupts pending for a CPU is actually rare. Doing an
acknowledge loop does not give much better performance or even can
deteriorate them.
Do not loop when an interrupt has been acknowledged, just return
from interrupt and wait for another one to be raised.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
drivers/irqchip/irq-gic-v3.c | 65 +++++++++++++++++++++-----------------------
1 file changed, 31 insertions(+), 34 deletions(-)
Note:
The loop pattern also discourages from using PMR for IRQ masking because
PMR needs to be unmasked to acknowledge interrupts.
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index e5d1014..0c58db3 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -348,48 +348,45 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
{
u32 irqnr;
- do {
- irqnr = gic_read_iar();
+ irqnr = gic_read_iar();
- if (likely(irqnr > 15 && irqnr < 1020) || irqnr >= 8192) {
- int err;
+ if (likely(irqnr > 15 && irqnr < 1020) || irqnr >= 8192) {
+ int err;
- if (static_branch_likely(&supports_deactivate_key))
+ if (static_branch_likely(&supports_deactivate_key))
+ gic_write_eoir(irqnr);
+ else
+ isb();
+
+ err = handle_domain_irq(gic_data.domain, irqnr, regs);
+ if (err) {
+ WARN_ONCE(true, "Unexpected interrupt received!\n");
+ if (static_branch_likely(&supports_deactivate_key)) {
+ if (irqnr < 8192)
+ gic_write_dir(irqnr);
+ } else {
gic_write_eoir(irqnr);
- else
- isb();
-
- err = handle_domain_irq(gic_data.domain, irqnr, regs);
- if (err) {
- WARN_ONCE(true, "Unexpected interrupt received!\n");
- if (static_branch_likely(&supports_deactivate_key)) {
- if (irqnr < 8192)
- gic_write_dir(irqnr);
- } else {
- gic_write_eoir(irqnr);
- }
}
- continue;
}
- if (irqnr < 16) {
- gic_write_eoir(irqnr);
- if (static_branch_likely(&supports_deactivate_key))
- gic_write_dir(irqnr);
+ return;
+ }
+ if (irqnr < 16) {
+ gic_write_eoir(irqnr);
+ if (static_branch_likely(&supports_deactivate_key))
+ gic_write_dir(irqnr);
#ifdef CONFIG_SMP
- /*
- * Unlike GICv2, we don't need an smp_rmb() here.
- * The control dependency from gic_read_iar to
- * the ISB in gic_write_eoir is enough to ensure
- * that any shared data read by handle_IPI will
- * be read after the ACK.
- */
- handle_IPI(irqnr, regs);
+ /*
+ * Unlike GICv2, we don't need an smp_rmb() here.
+ * The control dependency from gic_read_iar to
+ * the ISB in gic_write_eoir is enough to ensure
+ * that any shared data read by handle_IPI will
+ * be read after the ACK.
+ */
+ handle_IPI(irqnr, regs);
#else
- WARN_ONCE(true, "Unexpected SGI received!\n");
+ WARN_ONCE(true, "Unexpected SGI received!\n");
#endif
- continue;
- }
- } while (irqnr != ICC_IAR1_EL1_SPURIOUS);
+ }
}
static void __init gic_dist_init(void)
--
1.9.1
^ permalink raw reply related
* [PATCH v3 2/3] arm: shmobile: Add the R9A06G032 SMP enabler driver
From: Geert Uytterhoeven @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527157834-7747-3-git-send-email-michel.pollet@bp.renesas.com>
Hi Michel,
On Thu, May 24, 2018 at 12:30 PM, Michel Pollet
<michel.pollet@bp.renesas.com> wrote:
> The Renesas R9A06G032 second CA7 is parked in a ROM pen at boot time, it
> requires a special enable method to get it started.
>
> Signed-off-by: Michel Pollet <michel.pollet@bp.renesas.com>
Thanks for your patch!
> arch/arm/mach-shmobile/smp-r9a06g032.c | 85 ++++++++++++++++++++++++++++++++++
I think you can safely call this driver smp-rzn1d.c, or smp-rzn1.c.
Source files are not covered by the stable DT ABI, and can be reordered
later at will.
I expect you will just add more CPU_METHOD_OF_DECLARE() lines later
(perhaps with a little bit of extra code to handle deviations).
> --- /dev/null
> +++ b/arch/arm/mach-shmobile/smp-r9a06g032.c
> @@ -0,0 +1,85 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * RZ/N1D Second CA7 enabler.
> + *
> + * Copyright (C) 2018 Renesas Electronics Europe Limited
> + *
> + * Michel Pollet <michel.pollet@bp.renesas.com>, <buserror@gmail.com>
> + * Derived from action,s500-smp
> + */
> +
> +#include <linux/delay.h>
Do you need this?
> +#include <asm/cacheflush.h>
> +#include <asm/smp_plat.h>
> +#include <asm/smp_scu.h>
Do you need these?
> +static void __init rzn1_smp_prepare_cpus(unsigned int max_cpus)
> +{
> + struct device_node *dn;
> + int ret;
> + u32 bootaddr;
> + pr_info("CPU#1: cpu-release-addr %08x\n", (u32)bootaddr);
The cast is not needed.
> +
> + cpu_bootaddr = ioremap(bootaddr, sizeof(bootaddr));
> + if (!cpu_bootaddr)
> + pr_err("CPU#1: cpu-release-addr map failed\n");
If this fails, that is basically an OOM condition, which will scream anyway.
So I think you should drop the error message.
> +}
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert at linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* [PATCH v4 06/26] irqchip/gic: Lower priority of GIC interrupts
From: Julien Thierry @ 2018-05-25 9:49 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com>
The current value used for IRQ priorities is high among the
non-secure interrupt priority values.
Lower the default priority of interrupts so there is more flexibility
to define higher priority interrupts.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
include/linux/irqchip/arm-gic-common.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/irqchip/arm-gic-common.h b/include/linux/irqchip/arm-gic-common.h
index 9a1a479..2c9a4b3 100644
--- a/include/linux/irqchip/arm-gic-common.h
+++ b/include/linux/irqchip/arm-gic-common.h
@@ -13,7 +13,7 @@
#include <linux/types.h>
#include <linux/ioport.h>
-#define GICD_INT_DEF_PRI 0xa0
+#define GICD_INT_DEF_PRI 0xc0
#define GICD_INT_DEF_PRI_X4 ((GICD_INT_DEF_PRI << 24) |\
(GICD_INT_DEF_PRI << 16) |\
(GICD_INT_DEF_PRI << 8) |\
--
1.9.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox