[PATCH V2] arm: mvebu: Improve the SMP support of the interrupt controller

[PATCH V2] arm: mvebu: Improve the SMP support of the interrupt controller

[Gregory CLEMENT]

This patch makes the interrupt controller driver more SMP aware for
the Armada XP SoCs. It adds the support for the per-CPU irq. It also
adds the implementation for the set_affinity hook.

Patch initialy wrote by Yehuda Yitschak and reworked by Gregory
CLEMENT.

Signed-off-by: Yehuda Yitschak <yehuday@marvell.com>
Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 arch/arm/boot/dts/armada-xp.dtsi        |    2 +-
 arch/arm/mach-mvebu/irq-armada-370-xp.c |   52 +++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+), 1 deletion(-)

diff --git a/arch/arm/boot/dts/armada-xp.dtsi b/arch/arm/boot/dts/armada-xp.dtsi
index 45a567c..088d280 100644
--- a/arch/arm/boot/dts/armada-xp.dtsi
+++ b/arch/arm/boot/dts/armada-xp.dtsi
@@ -23,7 +23,7 @@
 	compatible = "marvell,armadaxp", "marvell,armada-370-xp";
 
 	mpic: interrupt-controller at d0020000 {
-	      reg = <0xd0020a00 0x1d0>,
+	      reg = <0xd0020a00 0x2d0>,
 		    <0xd0021070 0x58>;
 	};
 
diff --git a/arch/arm/mach-mvebu/irq-armada-370-xp.c b/arch/arm/mach-mvebu/irq-armada-370-xp.c
index 549b684..737abd7 100644
--- a/arch/arm/mach-mvebu/irq-armada-370-xp.c
+++ b/arch/arm/mach-mvebu/irq-armada-370-xp.c
@@ -33,6 +33,7 @@
 #define ARMADA_370_XP_INT_CONTROL		(0x00)
 #define ARMADA_370_XP_INT_SET_ENABLE_OFFS	(0x30)
 #define ARMADA_370_XP_INT_CLEAR_ENABLE_OFFS	(0x34)
+#define ARMADA_370_XP_INT_SOURCE_CTL(irq)	(0x100 + irq*4)
 
 #define ARMADA_370_XP_CPU_INTACK_OFFS		(0x44)
 
@@ -40,28 +41,79 @@
 #define ARMADA_370_XP_IN_DRBEL_MSK_OFFS          (0xc)
 #define ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS        (0x8)
 
+#define ARMADA_370_XP_MAX_PER_CPU_IRQS		(28)
+
 #define ACTIVE_DOORBELLS			(8)
 
+static DEFINE_RAW_SPINLOCK(irq_controller_lock);
+
 static void __iomem *per_cpu_int_base;
 static void __iomem *main_int_base;
 static struct irq_domain *armada_370_xp_mpic_domain;
 
+/*
+ * In SMP mode:
+ * For shared global interrupts, mask/unmask global enable bit
+ * For CPU interrtups, mask/unmask the calling CPU's bit
+ */
 static void armada_370_xp_irq_mask(struct irq_data *d)
 {
+#ifdef CONFIG_SMP
+	irq_hw_number_t hwirq = irqd_to_hwirq(d);
+
+	if (hwirq > ARMADA_370_XP_MAX_PER_CPU_IRQS)
+		writel(hwirq, main_int_base +
+				ARMADA_370_XP_INT_CLEAR_ENABLE_OFFS);
+	else
+		writel(hwirq, per_cpu_int_base +
+				ARMADA_370_XP_INT_SET_MASK_OFFS);
+#else
 	writel(irqd_to_hwirq(d),
 	       per_cpu_int_base + ARMADA_370_XP_INT_SET_MASK_OFFS);
+#endif
 }
 
 static void armada_370_xp_irq_unmask(struct irq_data *d)
 {
+#ifdef CONFIG_SMP
+	irq_hw_number_t hwirq = irqd_to_hwirq(d);
+
+	if (hwirq > ARMADA_370_XP_MAX_PER_CPU_IRQS)
+		writel(hwirq, main_int_base +
+				ARMADA_370_XP_INT_SET_ENABLE_OFFS);
+	else
+		writel(hwirq, per_cpu_int_base +
+				ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+#else
 	writel(irqd_to_hwirq(d),
 	       per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+#endif
 }
 
 #ifdef CONFIG_SMP
 static int armada_xp_set_affinity(struct irq_data *d,
 				  const struct cpumask *mask_val, bool force)
 {
+	unsigned long reg;
+	unsigned long new_mask = 0;
+	unsigned long online_mask = 0;
+	irq_hw_number_t hwirq = irqd_to_hwirq(d);
+	int cpu;
+
+	for_each_cpu(cpu, cpu_online_mask)
+		online_mask |= 1 << cpu_logical_map(cpu);
+
+	for_each_cpu(cpu, mask_val)
+		new_mask |= 1 << cpu_logical_map(cpu);
+
+	raw_spin_lock(&irq_controller_lock);
+
+	reg = readl(main_int_base + ARMADA_370_XP_INT_SOURCE_CTL(hwirq));
+	reg = (reg & (~online_mask)) | new_mask;
+	writel(reg, main_int_base + ARMADA_370_XP_INT_SOURCE_CTL(hwirq));
+
+	raw_spin_unlock(&irq_controller_lock);
+
 	return 0;
 }
 #endif
-- 
1.7.9.5

[PATCH V2] arm: mvebu: Improve the SMP support of the interrupt controller

[Jason Cooper]

On Wed, Dec 05, 2012 at 09:43:23PM +0100, Gregory CLEMENT wrote:
> This patch makes the interrupt controller driver more SMP aware for
> the Armada XP SoCs. It adds the support for the per-CPU irq. It also
> adds the implementation for the set_affinity hook.
> 
> Patch initialy wrote by Yehuda Yitschak and reworked by Gregory
> CLEMENT.
> 
> Signed-off-by: Yehuda Yitschak <yehuday@marvell.com>
> Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
> ---
>  arch/arm/boot/dts/armada-xp.dtsi        |    2 +-
>  arch/arm/mach-mvebu/irq-armada-370-xp.c |   52 +++++++++++++++++++++++++++++++
>  2 files changed, 53 insertions(+), 1 deletion(-)

Applied to mvebu/boards

thx,

Jason.

[PATCH V2] arm: mvebu: Improve the SMP support of the interrupt controller

[Gregory CLEMENT]

On 01/07/2013 06:51 PM, Jason Cooper wrote:
> On Wed, Dec 05, 2012 at 09:43:23PM +0100, Gregory CLEMENT wrote:
>> This patch makes the interrupt controller driver more SMP aware for
>> the Armada XP SoCs. It adds the support for the per-CPU irq. It also
>> adds the implementation for the set_affinity hook.
>>
>> Patch initialy wrote by Yehuda Yitschak and reworked by Gregory
>> CLEMENT.
>>
>> Signed-off-by: Yehuda Yitschak <yehuday@marvell.com>
>> Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
>> ---
>>  arch/arm/boot/dts/armada-xp.dtsi        |    2 +-
>>  arch/arm/mach-mvebu/irq-armada-370-xp.c |   52 +++++++++++++++++++++++++++++++
>>  2 files changed, 53 insertions(+), 1 deletion(-)
> 
> Applied to mvebu/boards

Hi Jason,
thanks for taking this patch. Unfortunately it won't be enough. Indeed
after speaking with Marvell engineers we realized that the MPIC didn't
work as we expected. As it is written in the following patch: the MPIC,
unlike the GIC, allows several CPUs to acknowledge the same global
interrupt.

I joined the patch to this email, you can apply it on top of your
mvebu/boards or you can squash it with the previous patch.

Thanks,

Gregory
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-arm-mvebu-Fix-interrupt-handling-in-SMP-mode.patch
Type: text/x-diff
Size: 2305 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130107/c54c135b/attachment.bin>

[PATCH V2] arm: mvebu: Improve the SMP support of the interrupt controller

[Jason Cooper]

On Mon, Jan 07, 2013 at 09:48:13PM +0100, Gregory CLEMENT wrote:
> On 01/07/2013 06:51 PM, Jason Cooper wrote:
> > On Wed, Dec 05, 2012 at 09:43:23PM +0100, Gregory CLEMENT wrote:
> >> This patch makes the interrupt controller driver more SMP aware for
> >> the Armada XP SoCs. It adds the support for the per-CPU irq. It also
> >> adds the implementation for the set_affinity hook.
> >>
> >> Patch initialy wrote by Yehuda Yitschak and reworked by Gregory
> >> CLEMENT.
> >>
> >> Signed-off-by: Yehuda Yitschak <yehuday@marvell.com>
> >> Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
> >> ---
> >>  arch/arm/boot/dts/armada-xp.dtsi        |    2 +-
> >>  arch/arm/mach-mvebu/irq-armada-370-xp.c |   52 +++++++++++++++++++++++++++++++
> >>  2 files changed, 53 insertions(+), 1 deletion(-)
> > 
> > Applied to mvebu/boards
> 
> Hi Jason,
> thanks for taking this patch. Unfortunately it won't be enough. Indeed
> after speaking with Marvell engineers we realized that the MPIC didn't
> work as we expected. As it is written in the following patch: the MPIC,
> unlike the GIC, allows several CPUs to acknowledge the same global
> interrupt.
> 
> I joined the patch to this email, you can apply it on top of your
> mvebu/boards or you can squash it with the previous patch.

Ok, I'll let this sit for a few days and barring any comments, I'll
squash it in with the first patch.  If you could provide some
Tested-by's, that would be appreciated.

thx,

Jason.

[PATCH V2] arm: mvebu: Improve the SMP support of the interrupt controller

[Gregory CLEMENT]

On 01/07/2013 10:03 PM, Jason Cooper wrote:
> On Mon, Jan 07, 2013 at 09:48:13PM +0100, Gregory CLEMENT wrote:
>> On 01/07/2013 06:51 PM, Jason Cooper wrote:
>>> On Wed, Dec 05, 2012 at 09:43:23PM +0100, Gregory CLEMENT wrote:
>>>> This patch makes the interrupt controller driver more SMP aware for
>>>> the Armada XP SoCs. It adds the support for the per-CPU irq. It also
>>>> adds the implementation for the set_affinity hook.
>>>>
>>>> Patch initialy wrote by Yehuda Yitschak and reworked by Gregory
>>>> CLEMENT.
>>>>
>>>> Signed-off-by: Yehuda Yitschak <yehuday@marvell.com>
>>>> Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
>>>> ---
>>>>  arch/arm/boot/dts/armada-xp.dtsi        |    2 +-
>>>>  arch/arm/mach-mvebu/irq-armada-370-xp.c |   52 +++++++++++++++++++++++++++++++
>>>>  2 files changed, 53 insertions(+), 1 deletion(-)
>>>
>>> Applied to mvebu/boards
>>
>> Hi Jason,
>> thanks for taking this patch. Unfortunately it won't be enough. Indeed
>> after speaking with Marvell engineers we realized that the MPIC didn't
>> work as we expected. As it is written in the following patch: the MPIC,
>> unlike the GIC, allows several CPUs to acknowledge the same global
>> interrupt.
>>
>> I joined the patch to this email, you can apply it on top of your
>> mvebu/boards or you can squash it with the previous patch.
> 
> Ok, I'll let this sit for a few days and barring any comments, I'll
> squash it in with the first patch.  If you could provide some
> Tested-by's, that would be appreciated.

I know that Thomas used it, so we can expect his official Tested-by
very soon :)

> 
> thx,
> 
> Jason.
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 


-- 
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[PATCH V2] arm: mvebu: Improve the SMP support of the interrupt controller

[Yehuda Yitschak]

Well, I also tested this patch, so...

Tested-by: Yehuda Yitschak <yehuday@marvell.com>

> -----Original Message-----
> From: Gregory CLEMENT [mailto:gregory.clement at free-electrons.com]
> Sent: Monday, January 07, 2013 11:08 PM
> To: Jason Cooper
> Cc: Thomas Petazzoni; Andrew Lunn; Yehuda Yitschak; Jani Monoses; Ike
> Pan; Tawfik Bayouk; Arnd Bergmann; Nicolas Pitre; Dan Frazier; Chris Van
> Hoof; David Marlin; Eran Ben-Avi; Nadav Haklai; Maen Suleiman; Lior
> Amsalem; Shadi Ammouri; Jon Masters; Olof Johansson; Leif Lindholm; linux-
> arm-kernel at lists.infradead.org; Sebastian Hesselbarth
> Subject: Re: [PATCH V2] arm: mvebu: Improve the SMP support of the
> interrupt controller
> 
> On 01/07/2013 10:03 PM, Jason Cooper wrote:
> > On Mon, Jan 07, 2013 at 09:48:13PM +0100, Gregory CLEMENT wrote:
> >> On 01/07/2013 06:51 PM, Jason Cooper wrote:
> >>> On Wed, Dec 05, 2012 at 09:43:23PM +0100, Gregory CLEMENT wrote:
> >>>> This patch makes the interrupt controller driver more SMP aware for
> >>>> the Armada XP SoCs. It adds the support for the per-CPU irq. It
> >>>> also adds the implementation for the set_affinity hook.
> >>>>
> >>>> Patch initialy wrote by Yehuda Yitschak and reworked by Gregory
> >>>> CLEMENT.
> >>>>
> >>>> Signed-off-by: Yehuda Yitschak <yehuday@marvell.com>
> >>>> Signed-off-by: Gregory CLEMENT <gregory.clement@free-
> electrons.com>
> >>>> ---
> >>>>  arch/arm/boot/dts/armada-xp.dtsi        |    2 +-
> >>>>  arch/arm/mach-mvebu/irq-armada-370-xp.c |   52
> +++++++++++++++++++++++++++++++
> >>>>  2 files changed, 53 insertions(+), 1 deletion(-)
> >>>
> >>> Applied to mvebu/boards
> >>
> >> Hi Jason,
> >> thanks for taking this patch. Unfortunately it won't be enough.
> >> Indeed after speaking with Marvell engineers we realized that the
> >> MPIC didn't work as we expected. As it is written in the following
> >> patch: the MPIC, unlike the GIC, allows several CPUs to acknowledge
> >> the same global interrupt.
> >>
> >> I joined the patch to this email, you can apply it on top of your
> >> mvebu/boards or you can squash it with the previous patch.
> >
> > Ok, I'll let this sit for a few days and barring any comments, I'll
> > squash it in with the first patch.  If you could provide some
> > Tested-by's, that would be appreciated.
> 
> I know that Thomas used it, so we can expect his official Tested-by very soon
> :)
> 
> >
> > thx,
> >
> > Jason.
> >
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> >
> 
> 
> --
> Gregory Clement, Free Electrons
> Kernel, drivers, real-time and embedded Linux development, consulting,
> training and support.
> http://free-electrons.com

[PATCH V2] arm: mvebu: Improve the SMP support of the interrupt controller

[Jason Cooper]

On Tue, Jan 08, 2013 at 09:15:57AM +0200, Yehuda Yitschak wrote:
> Well, I also tested this patch, so...
> 
> Tested-by: Yehuda Yitschak <yehuday@marvell.com>

Thanks, Yehuda.

Jason.

[PATCH V2] arm: mvebu: Improve the SMP support of the interrupt controller

[Jason Cooper]

On Mon, Jan 07, 2013 at 09:48:13PM +0100, Gregory CLEMENT wrote:
> On 01/07/2013 06:51 PM, Jason Cooper wrote:
> > On Wed, Dec 05, 2012 at 09:43:23PM +0100, Gregory CLEMENT wrote:
> >> This patch makes the interrupt controller driver more SMP aware for
> >> the Armada XP SoCs. It adds the support for the per-CPU irq. It also
> >> adds the implementation for the set_affinity hook.
> >>
> >> Patch initialy wrote by Yehuda Yitschak and reworked by Gregory
> >> CLEMENT.
> >>
> >> Signed-off-by: Yehuda Yitschak <yehuday@marvell.com>
> >> Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
> >> ---
> >>  arch/arm/boot/dts/armada-xp.dtsi        |    2 +-
> >>  arch/arm/mach-mvebu/irq-armada-370-xp.c |   52 +++++++++++++++++++++++++++++++
> >>  2 files changed, 53 insertions(+), 1 deletion(-)
> > 
> > Applied to mvebu/boards
> 
> Hi Jason,
> thanks for taking this patch. Unfortunately it won't be enough. Indeed
> after speaking with Marvell engineers we realized that the MPIC didn't
> work as we expected. As it is written in the following patch: the MPIC,
> unlike the GIC, allows several CPUs to acknowledge the same global
> interrupt.
> 
> I joined the patch to this email, you can apply it on top of your
> mvebu/boards or you can squash it with the previous patch.
> 
> Thanks,
> 
> Gregory

> From e3d36240832be914034bb3fb55dcf65c5a9f01df Mon Sep 17 00:00:00 2001
> From: Yehuda Yitschak <yehuday@marvell.com>
> Date: Thu, 27 Dec 2012 13:03:36 +0200
> Subject: [PATCH] arm: mvebu: Fix interrupt handling in SMP mode
> 
> The Multi Processor interrupt Controller (MPIC), unlike the GIC,
> allows several CPUs to acknowledge the same global interrupt.
> 
> This patch introduces changes to avoid this situation by forbidding
> multi-CPU interrupt affinity. This is done by setting the default
> interrupt affinity to the boot CPU and rejecting the affinity requests
> to more than one CPU.
> 
> Signed-off-by: Yehuda Yitschak <yehuday@marvell.com>
> Acked-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
> ---
>  arch/arm/mach-mvebu/irq-armada-370-xp.c |   26 +++++++++++++++++++++++---
>  1 file changed, 23 insertions(+), 3 deletions(-)

Applied to mvebu/boards

thx,

Jason.

[PATCH V2] arm: mvebu: Improve the SMP support of the interrupt controller

[Jason Cooper]

On Wed, Dec 05, 2012 at 09:43:23PM +0100, Gregory CLEMENT wrote:
> This patch makes the interrupt controller driver more SMP aware for
> the Armada XP SoCs. It adds the support for the per-CPU irq. It also
> adds the implementation for the set_affinity hook.
> 
> Patch initialy wrote by Yehuda Yitschak and reworked by Gregory
> CLEMENT.
> 
> Signed-off-by: Yehuda Yitschak <yehuday@marvell.com>
> Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
> ---
>  arch/arm/boot/dts/armada-xp.dtsi        |    2 +-
>  arch/arm/mach-mvebu/irq-armada-370-xp.c |   52 +++++++++++++++++++++++++++++++
>  2 files changed, 53 insertions(+), 1 deletion(-)

reapplied to mvebu/boards ;-)

squashed in Yehuda's fix.

thx,

Jason.