[PATCH 0/9] Clean up SMP IPI support

[PATCH 0/9] Clean up SMP IPI support

[Russell King - ARM Linux]

This series is aimed at cleaning up our SMP IPI support.  As has been
recently pointed out, we have a spinlock in percpu data, which is not
very nice.

If we switch to using the GIC SGIs, one for each IPI function, then
we don't need the mask of ipi bits indicating what work needs to be
done.  This allows us to get rid of the troublesome spinlock in
send_ipi_message().

However, we end up with slightly more expense on the IPI receiver side,
as they no longer have a word to read to process a set of pending IPIs.
This isn't much of a down-side as you hardly end up with masses of
pending IPIs.  Modern Cortex A9 platforms usually only clock up the
reschedule IPI.

This leaves us with just ipi_count in the ipi per-cpu data structure.
Other platforms put this in the global irq_stat array, and break this
down into individual IPIs.  So, let's do the same.

One down-side of this patchset is that platforms which use SGI 1 to
wake their secondary CPUs will receive an "Unknown IPI message" kernel
critical message at boot time - as SGI 1 is no longer recognised as a
Linux IPI interrupt.  We _could_ make this 'IPI_BOOT', something which
is only used for booting and therefore simply ignored.

[PATCH 1/9] ARM: remove obsolete smp_cross_call_done()

[Russell King - ARM Linux]

smp_cross_call_done() was removed long ago (see 78d236c - remove useless
smp_cross_call_done()).  Remove those which have been subsequently
merged.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
---
 arch/arm/mach-tegra/include/mach/smp.h |    7 -------
 1 files changed, 0 insertions(+), 7 deletions(-)

diff --git a/arch/arm/mach-tegra/include/mach/smp.h b/arch/arm/mach-tegra/include/mach/smp.h
index e4a34a3..d3cd265 100644
--- a/arch/arm/mach-tegra/include/mach/smp.h
+++ b/arch/arm/mach-tegra/include/mach/smp.h
@@ -12,11 +12,4 @@ static inline void smp_cross_call(const struct cpumask *mask)
 	gic_raise_softirq(mask, 1);
 }
 
-/*
- * Do nothing on MPcore.
- */
-static inline void smp_cross_call_done(cpumask_t callmap)
-{
-}
-
 #endif
-- 
1.6.2.5

[PATCH 2/9] ARM: smp: pass an ipi number to smp_cross_call()

[Russell King - ARM Linux]

This allows us to use smp_cross_call() to trigger a number of different
software generated interrupts, rather than combining them all on one
SGI.  Recover the SGI number via do_IPI.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
---
 arch/arm/include/asm/smp.h                |    4 ++--
 arch/arm/kernel/entry-armv.S              |    2 +-
 arch/arm/kernel/smp.c                     |   10 ++--------
 arch/arm/mach-msm/include/mach/smp.h      |    4 ++--
 arch/arm/mach-omap2/omap-smp.c            |    2 +-
 arch/arm/mach-realview/include/mach/smp.h |    4 ++--
 arch/arm/mach-realview/platsmp.c          |    2 +-
 arch/arm/mach-s5pv310/include/mach/smp.h  |    4 ++--
 arch/arm/mach-s5pv310/platsmp.c           |    2 +-
 arch/arm/mach-tegra/include/mach/smp.h    |    4 ++--
 arch/arm/mach-ux500/include/mach/smp.h    |    4 ++--
 arch/arm/mach-ux500/platsmp.c             |    2 +-
 arch/arm/mach-vexpress/include/mach/smp.h |    4 ++--
 arch/arm/mach-vexpress/platsmp.c          |    2 +-
 arch/arm/plat-omap/include/plat/smp.h     |    4 ++--
 15 files changed, 24 insertions(+), 30 deletions(-)

diff --git a/arch/arm/include/asm/smp.h b/arch/arm/include/asm/smp.h
index 3d05190..da7e7ca 100644
--- a/arch/arm/include/asm/smp.h
+++ b/arch/arm/include/asm/smp.h
@@ -38,7 +38,7 @@ extern void show_ipi_list(struct seq_file *p);
 /*
  * Called from assembly code, this handles an IPI.
  */
-asmlinkage void do_IPI(struct pt_regs *regs);
+asmlinkage void do_IPI(int ipinr, struct pt_regs *regs);
 
 /*
  * Setup the set of possible CPUs (via set_cpu_possible)
@@ -53,7 +53,7 @@ extern void smp_store_cpu_info(unsigned int cpuid);
 /*
  * Raise an IPI cross call on CPUs in callmap.
  */
-extern void smp_cross_call(const struct cpumask *mask);
+extern void smp_cross_call(const struct cpumask *mask, int ipi);
 
 /*
  * Boot a secondary CPU, and assign it the specified idle task.
diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S
index c09e357..955cf5f 100644
--- a/arch/arm/kernel/entry-armv.S
+++ b/arch/arm/kernel/entry-armv.S
@@ -48,7 +48,7 @@
 	 */
 	ALT_SMP(test_for_ipi r0, r6, r5, lr)
 	ALT_UP_B(9997f)
-	movne	r0, sp
+	movne	r1, sp
 	adrne	lr, BSYM(1b)
 	bne	do_IPI
 
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index 8c19595..7a236db 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -404,7 +404,7 @@ static void send_ipi_message(const struct cpumask *mask, enum ipi_msg_type msg)
 	/*
 	 * Call the platform specific cross-CPU call function.
 	 */
-	smp_cross_call(mask);
+	smp_cross_call(mask, 1);
 
 	local_irq_restore(flags);
 }
@@ -537,14 +537,8 @@ static void ipi_cpu_stop(unsigned int cpu)
 
 /*
  * Main handler for inter-processor interrupts
- *
- * For ARM, the ipimask now only identifies a single
- * category of IPI (Bit 1 IPIs have been replaced by a
- * different mechanism):
- *
- *  Bit 0 - Inter-processor function call
  */
-asmlinkage void __exception do_IPI(struct pt_regs *regs)
+asmlinkage void __exception do_IPI(int ipinr, struct pt_regs *regs)
 {
 	unsigned int cpu = smp_processor_id();
 	struct ipi_data *ipi = &per_cpu(ipi_data, cpu);
diff --git a/arch/arm/mach-msm/include/mach/smp.h b/arch/arm/mach-msm/include/mach/smp.h
index 3ff7bf5..a95f7b9 100644
--- a/arch/arm/mach-msm/include/mach/smp.h
+++ b/arch/arm/mach-msm/include/mach/smp.h
@@ -31,9 +31,9 @@
 
 #include <asm/hardware/gic.h>
 
-static inline void smp_cross_call(const struct cpumask *mask)
+static inline void smp_cross_call(const struct cpumask *mask, int ipi)
 {
-	gic_raise_softirq(mask, 1);
+	gic_raise_softirq(mask, ipi);
 }
 
 #endif
diff --git a/arch/arm/mach-omap2/omap-smp.c b/arch/arm/mach-omap2/omap-smp.c
index 9e9f70e..56a8bce 100644
--- a/arch/arm/mach-omap2/omap-smp.c
+++ b/arch/arm/mach-omap2/omap-smp.c
@@ -76,7 +76,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
 	omap_modify_auxcoreboot0(0x200, 0xfffffdff);
 	flush_cache_all();
 	smp_wmb();
-	smp_cross_call(cpumask_of(cpu));
+	smp_cross_call(cpumask_of(cpu), 1);
 
 	/*
 	 * Now the secondary core is starting up let it run its
diff --git a/arch/arm/mach-realview/include/mach/smp.h b/arch/arm/mach-realview/include/mach/smp.h
index d3cd265..d1aa704 100644
--- a/arch/arm/mach-realview/include/mach/smp.h
+++ b/arch/arm/mach-realview/include/mach/smp.h
@@ -7,9 +7,9 @@
 /*
  * We use IRQ1 as the IPI
  */
-static inline void smp_cross_call(const struct cpumask *mask)
+static inline void smp_cross_call(const struct cpumask *mask, int ipi)
 {
-	gic_raise_softirq(mask, 1);
+	gic_raise_softirq(mask, ipi);
 }
 
 #endif
diff --git a/arch/arm/mach-realview/platsmp.c b/arch/arm/mach-realview/platsmp.c
index 0092658..af3d909 100644
--- a/arch/arm/mach-realview/platsmp.c
+++ b/arch/arm/mach-realview/platsmp.c
@@ -116,7 +116,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
 	 * Use smp_cross_call() for this, since there's little
 	 * point duplicating the code here
 	 */
-	smp_cross_call(cpumask_of(cpu));
+	smp_cross_call(cpumask_of(cpu), 1);
 
 	timeout = jiffies + (1 * HZ);
 	while (time_before(jiffies, timeout)) {
diff --git a/arch/arm/mach-s5pv310/include/mach/smp.h b/arch/arm/mach-s5pv310/include/mach/smp.h
index b7ec252..2897747 100644
--- a/arch/arm/mach-s5pv310/include/mach/smp.h
+++ b/arch/arm/mach-s5pv310/include/mach/smp.h
@@ -14,9 +14,9 @@ extern void __iomem *gic_cpu_base_addr;
 /*
  * We use IRQ1 as the IPI
  */
-static inline void smp_cross_call(const struct cpumask *mask)
+static inline void smp_cross_call(const struct cpumask *mask, int ipi)
 {
-	gic_raise_softirq(mask, 1);
+	gic_raise_softirq(mask, ipi);
 }
 
 #endif
diff --git a/arch/arm/mach-s5pv310/platsmp.c b/arch/arm/mach-s5pv310/platsmp.c
index d357c19..d474426 100644
--- a/arch/arm/mach-s5pv310/platsmp.c
+++ b/arch/arm/mach-s5pv310/platsmp.c
@@ -97,7 +97,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
 	 * the boot monitor to read the system wide flags register,
 	 * and branch to the address found there.
 	 */
-	smp_cross_call(cpumask_of(cpu));
+	smp_cross_call(cpumask_of(cpu), 1);
 
 	timeout = jiffies + (1 * HZ);
 	while (time_before(jiffies, timeout)) {
diff --git a/arch/arm/mach-tegra/include/mach/smp.h b/arch/arm/mach-tegra/include/mach/smp.h
index d3cd265..d1aa704 100644
--- a/arch/arm/mach-tegra/include/mach/smp.h
+++ b/arch/arm/mach-tegra/include/mach/smp.h
@@ -7,9 +7,9 @@
 /*
  * We use IRQ1 as the IPI
  */
-static inline void smp_cross_call(const struct cpumask *mask)
+static inline void smp_cross_call(const struct cpumask *mask, int ipi)
 {
-	gic_raise_softirq(mask, 1);
+	gic_raise_softirq(mask, ipi);
 }
 
 #endif
diff --git a/arch/arm/mach-ux500/include/mach/smp.h b/arch/arm/mach-ux500/include/mach/smp.h
index 197e841..bd57c50 100644
--- a/arch/arm/mach-ux500/include/mach/smp.h
+++ b/arch/arm/mach-ux500/include/mach/smp.h
@@ -18,8 +18,8 @@ extern void u8500_secondary_startup(void);
 /*
  * We use IRQ1 as the IPI
  */
-static inline void smp_cross_call(const struct cpumask *mask)
+static inline void smp_cross_call(const struct cpumask *mask, int ipi)
 {
-	gic_raise_softirq(mask, 1);
+	gic_raise_softirq(mask, ipi);
 }
 #endif
diff --git a/arch/arm/mach-ux500/platsmp.c b/arch/arm/mach-ux500/platsmp.c
index 9e4c678..b8987bd 100644
--- a/arch/arm/mach-ux500/platsmp.c
+++ b/arch/arm/mach-ux500/platsmp.c
@@ -78,7 +78,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
 	__cpuc_flush_dcache_area((void *)&pen_release, sizeof(pen_release));
 	outer_clean_range(__pa(&pen_release), __pa(&pen_release) + 1);
 
-	smp_cross_call(cpumask_of(cpu));
+	smp_cross_call(cpumask_of(cpu), 1);
 
 	timeout = jiffies + (1 * HZ);
 	while (time_before(jiffies, timeout)) {
diff --git a/arch/arm/mach-vexpress/include/mach/smp.h b/arch/arm/mach-vexpress/include/mach/smp.h
index 5a6da4f..721be0f 100644
--- a/arch/arm/mach-vexpress/include/mach/smp.h
+++ b/arch/arm/mach-vexpress/include/mach/smp.h
@@ -7,8 +7,8 @@
 /*
  * We use IRQ1 as the IPI
  */
-static inline void smp_cross_call(const struct cpumask *mask)
+static inline void smp_cross_call(const struct cpumask *mask, int ipi)
 {
-	gic_raise_softirq(mask, 1);
+	gic_raise_softirq(mask, ipi);
 }
 #endif
diff --git a/arch/arm/mach-vexpress/platsmp.c b/arch/arm/mach-vexpress/platsmp.c
index 6709706..276f916 100644
--- a/arch/arm/mach-vexpress/platsmp.c
+++ b/arch/arm/mach-vexpress/platsmp.c
@@ -92,7 +92,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
 	 * the boot monitor to read the system wide flags register,
 	 * and branch to the address found there.
 	 */
-	smp_cross_call(cpumask_of(cpu));
+	smp_cross_call(cpumask_of(cpu), 1);
 
 	timeout = jiffies + (1 * HZ);
 	while (time_before(jiffies, timeout)) {
diff --git a/arch/arm/plat-omap/include/plat/smp.h b/arch/arm/plat-omap/include/plat/smp.h
index ecd6a48..e5541e5 100644
--- a/arch/arm/plat-omap/include/plat/smp.h
+++ b/arch/arm/plat-omap/include/plat/smp.h
@@ -29,9 +29,9 @@ extern u32 omap_read_auxcoreboot0(void);
 /*
  * We use Soft IRQ1 as the IPI
  */
-static inline void smp_cross_call(const struct cpumask *mask)
+static inline void smp_cross_call(const struct cpumask *mask, int ipi)
 {
-	gic_raise_softirq(mask, 1);
+	gic_raise_softirq(mask, ipi);
 }
 
 #endif
-- 
1.6.2.5

[PATCH 3/9] ARM: smp: avoid using bitmasks and locks for IPIs, use hardware instead

[Russell King - ARM Linux]

Avoid using bitmasks and locks in the percpu area for IPIs, and instead
use individual software generated interrupts to identify the reason for
the IPI.  This avoids the problems of having spinlocks in the percpu
area.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
---
 arch/arm/kernel/smp.c |   87 ++++++++++++++----------------------------------
 1 files changed, 26 insertions(+), 61 deletions(-)

diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index 7a236db..78d55c6 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -48,20 +48,15 @@ struct secondary_data secondary_data;
 
 /*
  * structures for inter-processor calls
- * - A collection of single bit ipi messages.
  */
 struct ipi_data {
-	spinlock_t lock;
 	unsigned long ipi_count;
-	unsigned long bits;
 };
 
-static DEFINE_PER_CPU(struct ipi_data, ipi_data) = {
-	.lock	= SPIN_LOCK_UNLOCKED,
-};
+static DEFINE_PER_CPU(struct ipi_data, ipi_data);
 
 enum ipi_msg_type {
-	IPI_TIMER,
+	IPI_TIMER = 2,
 	IPI_RESCHEDULE,
 	IPI_CALL_FUNC,
 	IPI_CALL_FUNC_SINGLE,
@@ -389,22 +384,13 @@ void __init smp_prepare_boot_cpu(void)
 static void send_ipi_message(const struct cpumask *mask, enum ipi_msg_type msg)
 {
 	unsigned long flags;
-	unsigned int cpu;
 
 	local_irq_save(flags);
 
-	for_each_cpu(cpu, mask) {
-		struct ipi_data *ipi = &per_cpu(ipi_data, cpu);
-
-		spin_lock(&ipi->lock);
-		ipi->bits |= 1 << msg;
-		spin_unlock(&ipi->lock);
-	}
-
 	/*
 	 * Call the platform specific cross-CPU call function.
 	 */
-	smp_cross_call(mask, 1);
+	smp_cross_call(mask, msg);
 
 	local_irq_restore(flags);
 }
@@ -546,56 +532,35 @@ asmlinkage void __exception do_IPI(int ipinr, struct pt_regs *regs)
 
 	ipi->ipi_count++;
 
-	for (;;) {
-		unsigned long msgs;
-
-		spin_lock(&ipi->lock);
-		msgs = ipi->bits;
-		ipi->bits = 0;
-		spin_unlock(&ipi->lock);
-
-		if (!msgs)
-			break;
-
-		do {
-			unsigned nextmsg;
-
-			nextmsg = msgs & -msgs;
-			msgs &= ~nextmsg;
-			nextmsg = ffz(~nextmsg);
-
-			switch (nextmsg) {
-			case IPI_TIMER:
-				ipi_timer();
-				break;
+	switch (ipinr) {
+	case IPI_TIMER:
+		ipi_timer();
+		break;
 
-			case IPI_RESCHEDULE:
-				/*
-				 * nothing more to do - eveything is
-				 * done on the interrupt return path
-				 */
-				break;
+	case IPI_RESCHEDULE:
+		/*
+		 * nothing more to do - eveything is
+		 * done on the interrupt return path
+		 */
+		break;
 
-			case IPI_CALL_FUNC:
-				generic_smp_call_function_interrupt();
-				break;
+	case IPI_CALL_FUNC:
+		generic_smp_call_function_interrupt();
+		break;
 
-			case IPI_CALL_FUNC_SINGLE:
-				generic_smp_call_function_single_interrupt();
-				break;
+	case IPI_CALL_FUNC_SINGLE:
+		generic_smp_call_function_single_interrupt();
+		break;
 
-			case IPI_CPU_STOP:
-				ipi_cpu_stop(cpu);
-				break;
+	case IPI_CPU_STOP:
+		ipi_cpu_stop(cpu);
+		break;
 
-			default:
-				printk(KERN_CRIT "CPU%u: Unknown IPI message 0x%x\n",
-				       cpu, nextmsg);
-				break;
-			}
-		} while (msgs);
+	default:
+		printk(KERN_CRIT "CPU%u: Unknown IPI message 0x%x\n",
+		       cpu, ipinr);
+		break;
 	}
-
 	set_irq_regs(old_regs);
 }
 
-- 
1.6.2.5

[PATCH 4/9] ARM: include local timer irq stats only when local timers configured

[Russell King - ARM Linux]

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
---
 arch/arm/include/asm/hardirq.h |    2 ++
 arch/arm/kernel/irq.c          |    2 ++
 arch/arm/kernel/smp.c          |   24 ++++++++++++------------
 3 files changed, 16 insertions(+), 12 deletions(-)

diff --git a/arch/arm/include/asm/hardirq.h b/arch/arm/include/asm/hardirq.h
index 6d7485a..522fb23 100644
--- a/arch/arm/include/asm/hardirq.h
+++ b/arch/arm/include/asm/hardirq.h
@@ -7,7 +7,9 @@
 
 typedef struct {
 	unsigned int __softirq_pending;
+#ifdef CONFIG_LOCAL_TIMERS
 	unsigned int local_timer_irqs;
+#endif
 } ____cacheline_aligned irq_cpustat_t;
 
 #include <linux/irq_cpustat.h>	/* Standard mappings for irq_cpustat_t above */
diff --git a/arch/arm/kernel/irq.c b/arch/arm/kernel/irq.c
index 36ad3be..ea29721 100644
--- a/arch/arm/kernel/irq.c
+++ b/arch/arm/kernel/irq.c
@@ -93,6 +93,8 @@ unlock:
 #endif
 #ifdef CONFIG_SMP
 		show_ipi_list(p);
+#endif
+#ifdef CONFIG_LOCAL_TIMERS
 		show_local_irqs(p);
 #endif
 		seq_printf(p, "Err: %10lu\n", irq_err_count);
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index 78d55c6..b58b6c4 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -417,18 +417,6 @@ void show_ipi_list(struct seq_file *p)
 	seq_putc(p, '\n');
 }
 
-void show_local_irqs(struct seq_file *p)
-{
-	unsigned int cpu;
-
-	seq_printf(p, "LOC: ");
-
-	for_each_present_cpu(cpu)
-		seq_printf(p, "%10u ", irq_stat[cpu].local_timer_irqs);
-
-	seq_putc(p, '\n');
-}
-
 /*
  * Timer (local or broadcast) support
  */
@@ -455,6 +443,18 @@ asmlinkage void __exception do_local_timer(struct pt_regs *regs)
 
 	set_irq_regs(old_regs);
 }
+
+void show_local_irqs(struct seq_file *p)
+{
+	unsigned int cpu;
+
+	seq_printf(p, "LOC: ");
+
+	for_each_present_cpu(cpu)
+		seq_printf(p, "%10u ", irq_stat[cpu].local_timer_irqs);
+
+	seq_putc(p, '\n');
+}
 #endif
 
 #ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
-- 
1.6.2.5

[PATCH 5/9] ARM: smp: provide accessors for irq_stat data

[Russell King - ARM Linux]

Provide __inc_irq_stat() and __get_irq_stat() to increment and
read the irq stat counters.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
---
 arch/arm/include/asm/hardirq.h |    3 +++
 arch/arm/kernel/smp.c          |    4 ++--
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/arch/arm/include/asm/hardirq.h b/arch/arm/include/asm/hardirq.h
index 522fb23..923eaa1 100644
--- a/arch/arm/include/asm/hardirq.h
+++ b/arch/arm/include/asm/hardirq.h
@@ -14,6 +14,9 @@ typedef struct {
 
 #include <linux/irq_cpustat.h>	/* Standard mappings for irq_cpustat_t above */
 
+#define __inc_irq_stat(cpu, member)	__IRQ_STAT(cpu, member)++
+#define __get_irq_stat(cpu, member)	__IRQ_STAT(cpu, member)
+
 #if NR_IRQS > 512
 #define HARDIRQ_BITS	10
 #elif NR_IRQS > 256
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index b58b6c4..f276336 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -437,7 +437,7 @@ asmlinkage void __exception do_local_timer(struct pt_regs *regs)
 	int cpu = smp_processor_id();
 
 	if (local_timer_ack()) {
-		irq_stat[cpu].local_timer_irqs++;
+		__inc_irq_stat(cpu, local_timer_irqs);
 		ipi_timer();
 	}
 
@@ -451,7 +451,7 @@ void show_local_irqs(struct seq_file *p)
 	seq_printf(p, "LOC: ");
 
 	for_each_present_cpu(cpu)
-		seq_printf(p, "%10u ", irq_stat[cpu].local_timer_irqs);
+		seq_printf(p, "%10u ", __get_irq_stat(cpu, local_timer_irqs));
 
 	seq_putc(p, '\n');
 }
-- 
1.6.2.5

[PATCH 6/9] ARM: smp: move ipi_count into irq_stat structure

[Russell King - ARM Linux]

Move the ipi_count into irq_stat, which allows the ipi_data structure
to be entirely removed.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
---
 arch/arm/include/asm/hardirq.h |    3 +++
 arch/arm/kernel/smp.c          |   14 ++------------
 2 files changed, 5 insertions(+), 12 deletions(-)

diff --git a/arch/arm/include/asm/hardirq.h b/arch/arm/include/asm/hardirq.h
index 923eaa1..824b08a 100644
--- a/arch/arm/include/asm/hardirq.h
+++ b/arch/arm/include/asm/hardirq.h
@@ -10,6 +10,9 @@ typedef struct {
 #ifdef CONFIG_LOCAL_TIMERS
 	unsigned int local_timer_irqs;
 #endif
+#ifdef CONFIG_SMP
+	unsigned int ipi_irqs;
+#endif
 } ____cacheline_aligned irq_cpustat_t;
 
 #include <linux/irq_cpustat.h>	/* Standard mappings for irq_cpustat_t above */
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index f276336..edb8fd5 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -46,15 +46,6 @@
  */
 struct secondary_data secondary_data;
 
-/*
- * structures for inter-processor calls
- */
-struct ipi_data {
-	unsigned long ipi_count;
-};
-
-static DEFINE_PER_CPU(struct ipi_data, ipi_data);
-
 enum ipi_msg_type {
 	IPI_TIMER = 2,
 	IPI_RESCHEDULE,
@@ -412,7 +403,7 @@ void show_ipi_list(struct seq_file *p)
 	seq_puts(p, "IPI:");
 
 	for_each_present_cpu(cpu)
-		seq_printf(p, " %10lu", per_cpu(ipi_data, cpu).ipi_count);
+		seq_printf(p, " %10u", __get_irq_stat(cpu, ipi_irqs));
 
 	seq_putc(p, '\n');
 }
@@ -527,10 +518,9 @@ static void ipi_cpu_stop(unsigned int cpu)
 asmlinkage void __exception do_IPI(int ipinr, struct pt_regs *regs)
 {
 	unsigned int cpu = smp_processor_id();
-	struct ipi_data *ipi = &per_cpu(ipi_data, cpu);
 	struct pt_regs *old_regs = set_irq_regs(regs);
 
-	ipi->ipi_count++;
+	__inc_irq_stat(cpu, ipi_irqs);
 
 	switch (ipinr) {
 	case IPI_TIMER:
-- 
1.6.2.5

[PATCH 7/9] ARM: fix /proc/interrupts formatting

[Russell King - ARM Linux]

As per x86, align the initial column according to how many IRQs we
have.  Also, provide an english explaination for the 'LOC:' and
'IPI:' lines.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
---
 arch/arm/include/asm/mach/irq.h |    2 +-
 arch/arm/include/asm/smp.h      |    4 ++--
 arch/arm/kernel/fiq.c           |    5 +++--
 arch/arm/kernel/irq.c           |   16 ++++++++++------
 arch/arm/kernel/smp.c           |   14 +++++++-------
 5 files changed, 23 insertions(+), 18 deletions(-)

diff --git a/arch/arm/include/asm/mach/irq.h b/arch/arm/include/asm/mach/irq.h
index ce3eee9..2bc47fb 100644
--- a/arch/arm/include/asm/mach/irq.h
+++ b/arch/arm/include/asm/mach/irq.h
@@ -20,7 +20,7 @@ struct seq_file;
 extern unsigned int arch_nr_irqs;
 extern void (*init_arch_irq)(void);
 extern void init_FIQ(void);
-extern int show_fiq_list(struct seq_file *, void *);
+extern int show_fiq_list(struct seq_file *, int);
 
 /*
  * This is for easy migration, but should be changed in the source
diff --git a/arch/arm/include/asm/smp.h b/arch/arm/include/asm/smp.h
index da7e7ca..f93d0a6 100644
--- a/arch/arm/include/asm/smp.h
+++ b/arch/arm/include/asm/smp.h
@@ -33,7 +33,7 @@ struct seq_file;
 /*
  * generate IPI list text
  */
-extern void show_ipi_list(struct seq_file *p);
+extern void show_ipi_list(struct seq_file *, int);
 
 /*
  * Called from assembly code, this handles an IPI.
@@ -97,6 +97,6 @@ extern void arch_send_call_function_ipi_mask(const struct cpumask *mask);
 /*
  * show local interrupt info
  */
-extern void show_local_irqs(struct seq_file *);
+extern void show_local_irqs(struct seq_file *, int);
 
 #endif /* ifndef __ASM_ARM_SMP_H */
diff --git a/arch/arm/kernel/fiq.c b/arch/arm/kernel/fiq.c
index 6ff7919..47837b8 100644
--- a/arch/arm/kernel/fiq.c
+++ b/arch/arm/kernel/fiq.c
@@ -67,10 +67,11 @@ static struct fiq_handler default_owner = {
 
 static struct fiq_handler *current_fiq = &default_owner;
 
-int show_fiq_list(struct seq_file *p, void *v)
+int show_fiq_list(struct seq_file *p, int prec)
 {
 	if (current_fiq != &default_owner)
-		seq_printf(p, "FIQ:              %s\n", current_fiq->name);
+		seq_printf(p, "%*s:              %s\n", prec, "FIQ",
+			current_fiq->name);
 
 	return 0;
 }
diff --git a/arch/arm/kernel/irq.c b/arch/arm/kernel/irq.c
index ea29721..4e7a7d2 100644
--- a/arch/arm/kernel/irq.c
+++ b/arch/arm/kernel/irq.c
@@ -57,11 +57,15 @@ int show_interrupts(struct seq_file *p, void *v)
 	struct irq_desc *desc;
 	struct irqaction * action;
 	unsigned long flags;
+	int prec, n;
+
+	for (prec = 3, n = 1000; prec < 10 && n <= nr_irqs; prec++)
+		n *= 10;
 
 	if (i == 0) {
 		char cpuname[12];
 
-		seq_printf(p, "    ");
+		seq_printf(p, "%*s ", prec, "");
 		for_each_present_cpu(cpu) {
 			sprintf(cpuname, "CPU%d", cpu);
 			seq_printf(p, " %10s", cpuname);
@@ -76,7 +80,7 @@ int show_interrupts(struct seq_file *p, void *v)
 		if (!action)
 			goto unlock;
 
-		seq_printf(p, "%3d: ", i);
+		seq_printf(p, "%*d: ", prec, i);
 		for_each_present_cpu(cpu)
 			seq_printf(p, "%10u ", kstat_irqs_cpu(i, cpu));
 		seq_printf(p, " %10s", desc->chip->name ? : "-");
@@ -89,15 +93,15 @@ unlock:
 		raw_spin_unlock_irqrestore(&desc->lock, flags);
 	} else if (i == nr_irqs) {
 #ifdef CONFIG_FIQ
-		show_fiq_list(p, v);
+		show_fiq_list(p, prec);
 #endif
 #ifdef CONFIG_SMP
-		show_ipi_list(p);
+		show_ipi_list(p, prec);
 #endif
 #ifdef CONFIG_LOCAL_TIMERS
-		show_local_irqs(p);
+		show_local_irqs(p, prec);
 #endif
-		seq_printf(p, "Err: %10lu\n", irq_err_count);
+		seq_printf(p, "%*s: %10lu\n", prec, "Err", irq_err_count);
 	}
 	return 0;
 }
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index edb8fd5..d0e36e3 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -396,16 +396,16 @@ void arch_send_call_function_single_ipi(int cpu)
 	send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC_SINGLE);
 }
 
-void show_ipi_list(struct seq_file *p)
+void show_ipi_list(struct seq_file *p, int prec)
 {
 	unsigned int cpu;
 
-	seq_puts(p, "IPI:");
+	seq_printf(p, "%*s: ", prec, "IPI");
 
 	for_each_present_cpu(cpu)
-		seq_printf(p, " %10u", __get_irq_stat(cpu, ipi_irqs));
+		seq_printf(p, "%10u ", __get_irq_stat(cpu, ipi_irqs));
 
-	seq_putc(p, '\n');
+	seq_printf(p, " Inter-processor interrupts\n");
 }
 
 /*
@@ -435,16 +435,16 @@ asmlinkage void __exception do_local_timer(struct pt_regs *regs)
 	set_irq_regs(old_regs);
 }
 
-void show_local_irqs(struct seq_file *p)
+void show_local_irqs(struct seq_file *p, int prec)
 {
 	unsigned int cpu;
 
-	seq_printf(p, "LOC: ");
+	seq_printf(p, "%*s: ", prec, "LOC");
 
 	for_each_present_cpu(cpu)
 		seq_printf(p, "%10u ", __get_irq_stat(cpu, local_timer_irqs));
 
-	seq_putc(p, '\n');
+	seq_printf(p, " Local timer interrupts\n");
 }
 #endif
 
-- 
1.6.2.5

[PATCH 8/9] ARM: smp: provide individual IPI interrupt statistics

[Russell King - ARM Linux]

This separates out the individual IPI interrupt counts from the
total IPI count, which allows better visibility of what IPIs are
being used for.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
---
 arch/arm/include/asm/hardirq.h |    4 +++-
 arch/arm/kernel/irq.c          |    5 +++++
 arch/arm/kernel/smp.c          |   25 +++++++++++++++++++------
 3 files changed, 27 insertions(+), 7 deletions(-)

diff --git a/arch/arm/include/asm/hardirq.h b/arch/arm/include/asm/hardirq.h
index 824b08a..c52e250 100644
--- a/arch/arm/include/asm/hardirq.h
+++ b/arch/arm/include/asm/hardirq.h
@@ -5,13 +5,15 @@
 #include <linux/threads.h>
 #include <asm/irq.h>
 
+#define NR_IPI	5
+
 typedef struct {
 	unsigned int __softirq_pending;
 #ifdef CONFIG_LOCAL_TIMERS
 	unsigned int local_timer_irqs;
 #endif
 #ifdef CONFIG_SMP
-	unsigned int ipi_irqs;
+	unsigned int ipi_irqs[NR_IPI];
 #endif
 } ____cacheline_aligned irq_cpustat_t;
 
diff --git a/arch/arm/kernel/irq.c b/arch/arm/kernel/irq.c
index 4e7a7d2..6276f01 100644
--- a/arch/arm/kernel/irq.c
+++ b/arch/arm/kernel/irq.c
@@ -62,6 +62,11 @@ int show_interrupts(struct seq_file *p, void *v)
 	for (prec = 3, n = 1000; prec < 10 && n <= nr_irqs; prec++)
 		n *= 10;
 
+#ifdef CONFIG_SMP
+	if (prec < 4)
+		prec = 4;
+#endif
+
 	if (i == 0) {
 		char cpuname[12];
 
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index d0e36e3..c088485 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -396,16 +396,28 @@ void arch_send_call_function_single_ipi(int cpu)
 	send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC_SINGLE);
 }
 
+static const char *ipi_types[NR_IPI] = {
+#define S(x,s)	[x - IPI_TIMER] = s
+	S(IPI_TIMER, "Timer broadcast interrupts"),
+	S(IPI_RESCHEDULE, "Rescheduling interrupts"),
+	S(IPI_CALL_FUNC, "Function call interrupts"),
+	S(IPI_CALL_FUNC_SINGLE, "Single function call interrupts"),
+	S(IPI_CPU_STOP, "CPU stop interrupts"),
+};
+
 void show_ipi_list(struct seq_file *p, int prec)
 {
-	unsigned int cpu;
+	unsigned int cpu, i;
 
-	seq_printf(p, "%*s: ", prec, "IPI");
+	for (i = 0; i < NR_IPI; i++) {
+		seq_printf(p, "%*s%u: ", prec - 1, "IPI", i);
 
-	for_each_present_cpu(cpu)
-		seq_printf(p, "%10u ", __get_irq_stat(cpu, ipi_irqs));
+		for_each_present_cpu(cpu)
+			seq_printf(p, "%10u ",
+				   __get_irq_stat(cpu, ipi_irqs[i]));
 
-	seq_printf(p, " Inter-processor interrupts\n");
+		seq_printf(p, " %s\n", ipi_types[i]);
+	}
 }
 
 /*
@@ -520,7 +532,8 @@ asmlinkage void __exception do_IPI(int ipinr, struct pt_regs *regs)
 	unsigned int cpu = smp_processor_id();
 	struct pt_regs *old_regs = set_irq_regs(regs);
 
-	__inc_irq_stat(cpu, ipi_irqs);
+	if (ipinr >= IPI_TIMER && ipinr < IPI_TIMER + NR_IPI)
+		__inc_irq_stat(cpu, ipi_irqs[ipinr - IPI_TIMER]);
 
 	switch (ipinr) {
 	case IPI_TIMER:
-- 
1.6.2.5

[PATCH 9/9] ARM: smp: collect IPI and local timer IRQs for /proc/stat

[Russell King - ARM Linux]

The IPI and local timer interrupts weren't being properly accounted
for in /proc/stat.  Collect them from the irq_stat structure, and
return their sum.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
---
 arch/arm/include/asm/hardirq.h |    8 ++++++++
 arch/arm/kernel/smp.c          |   15 +++++++++++++++
 2 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/arch/arm/include/asm/hardirq.h b/arch/arm/include/asm/hardirq.h
index c52e250..89ad180 100644
--- a/arch/arm/include/asm/hardirq.h
+++ b/arch/arm/include/asm/hardirq.h
@@ -22,6 +22,14 @@ typedef struct {
 #define __inc_irq_stat(cpu, member)	__IRQ_STAT(cpu, member)++
 #define __get_irq_stat(cpu, member)	__IRQ_STAT(cpu, member)
 
+#ifdef CONFIG_SMP
+u64 smp_irq_stat_cpu(unsigned int cpu);
+#else
+#define smp_irq_stat_cpu(cpu)	0
+#endif
+
+#define arch_irq_stat_cpu	smp_irq_stat_cpu
+
 #if NR_IRQS > 512
 #define HARDIRQ_BITS	10
 #elif NR_IRQS > 256
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index c088485..5d0dc16 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -420,6 +420,21 @@ void show_ipi_list(struct seq_file *p, int prec)
 	}
 }
 
+u64 smp_irq_stat_cpu(unsigned int cpu)
+{
+	u64 sum = 0;
+	int i;
+
+	for (i = 0; i < NR_IPI; i++)
+		sum += __get_irq_stat(cpu, ipi_irqs[i]);
+
+#ifdef CONFIG_LOCAL_TIMERS
+	sum += __get_irq_stat(cpu, local_timer_irqs);
+#endif
+
+	return sum;
+}
+
 /*
  * Timer (local or broadcast) support
  */
-- 
1.6.2.5

[PATCH 2/9] ARM: smp: pass an ipi number to smp_cross_call()

[Catalin Marinas]

On Mon, 2010-11-15 at 17:58 +0000, Russell King - ARM Linux wrote:
> --- a/arch/arm/mach-msm/include/mach/smp.h
> +++ b/arch/arm/mach-msm/include/mach/smp.h
> @@ -31,9 +31,9 @@
> 
>  #include <asm/hardware/gic.h>
> 
> -static inline void smp_cross_call(const struct cpumask *mask)
> +static inline void smp_cross_call(const struct cpumask *mask, int ipi)
>  {
> -       gic_raise_softirq(mask, 1);
> +       gic_raise_softirq(mask, ipi);
>  }

BTW, why don't we make smp_cross_call() generic, it looks like all
platforms define the same thing.

Unless we get SMP systems in the future without a GIC.

-- 
Catalin

[PATCH 3/9] ARM: smp: avoid using bitmasks and locks for IPIs, use hardware instead

[Catalin Marinas]

On Mon, 2010-11-15 at 17:58 +0000, Russell King - ARM Linux wrote:
> Avoid using bitmasks and locks in the percpu area for IPIs, and instead
> use individual software generated interrupts to identify the reason for
> the IPI.  This avoids the problems of having spinlocks in the percpu
> area.
> 
> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
> ---
>  arch/arm/kernel/smp.c |   87 ++++++++++++++----------------------------------
>  1 files changed, 26 insertions(+), 61 deletions(-)
> 
> diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
> index 7a236db..78d55c6 100644
> --- a/arch/arm/kernel/smp.c
> +++ b/arch/arm/kernel/smp.c
[...]
>  enum ipi_msg_type {
> -       IPI_TIMER,
> +       IPI_TIMER = 2,
>         IPI_RESCHEDULE,
>         IPI_CALL_FUNC,
>         IPI_CALL_FUNC_SINGLE,

Maybe a comment here on why this starts at 2. People may wonder in the
future.

-- 
Catalin

[PATCH 0/9] Clean up SMP IPI support

[Catalin Marinas]

On Mon, 2010-11-15 at 17:55 +0000, Russell King - ARM Linux wrote:
> This series is aimed at cleaning up our SMP IPI support.  As has been
> recently pointed out, we have a spinlock in percpu data, which is not
> very nice.
> 
> If we switch to using the GIC SGIs, one for each IPI function, then
> we don't need the mask of ipi bits indicating what work needs to be
> done.  This allows us to get rid of the troublesome spinlock in
> send_ipi_message().

FWIW, the patches look fine to me (though I have tested them):

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>

[PATCH 0/9] Clean up SMP IPI support

[Catalin Marinas]

On Wed, 2010-11-17 at 17:07 +0000, Catalin Marinas wrote:
> On Mon, 2010-11-15 at 17:55 +0000, Russell King - ARM Linux wrote:
> > This series is aimed at cleaning up our SMP IPI support.  As has been
> > recently pointed out, we have a spinlock in percpu data, which is not
> > very nice.
> > 
> > If we switch to using the GIC SGIs, one for each IPI function, then
> > we don't need the mask of ipi bits indicating what work needs to be
> > done.  This allows us to get rid of the troublesome spinlock in
> > send_ipi_message().
> 
> FWIW, the patches look fine to me (though I have tested them):
                                              ^^^^
I meant I have *not* tested them.

-- 
Catalin

[PATCH 3/9] ARM: smp: avoid using bitmasks and locks for IPIs, use hardware instead

[Russell King - ARM Linux]

On Wed, Nov 17, 2010 at 05:06:39PM +0000, Catalin Marinas wrote:
> On Mon, 2010-11-15 at 17:58 +0000, Russell King - ARM Linux wrote:
> > Avoid using bitmasks and locks in the percpu area for IPIs, and instead
> > use individual software generated interrupts to identify the reason for
> > the IPI.  This avoids the problems of having spinlocks in the percpu
> > area.
> > 
> > Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
> > ---
> >  arch/arm/kernel/smp.c |   87 ++++++++++++++----------------------------------
> >  1 files changed, 26 insertions(+), 61 deletions(-)
> > 
> > diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
> > index 7a236db..78d55c6 100644
> > --- a/arch/arm/kernel/smp.c
> > +++ b/arch/arm/kernel/smp.c
> [...]
> >  enum ipi_msg_type {
> > -       IPI_TIMER,
> > +       IPI_TIMER = 2,
> >         IPI_RESCHEDULE,
> >         IPI_CALL_FUNC,
> >         IPI_CALL_FUNC_SINGLE,
> 
> Maybe a comment here on why this starts at 2. People may wonder in the
> future.

I would like to get rid of the 'starts at 2' but we can only do this once
we're sure that the use of SGI1 to wake up the CPUs is not something that
is a hard requirement for boot loaders.

I've not yet tested whether Realview and Versatile Express are fine with
a different SGI number.

[PATCH 0/9] Clean up SMP IPI support

[Russell King - ARM Linux]

On Wed, Nov 17, 2010 at 05:17:28PM +0000, Catalin Marinas wrote:
> On Wed, 2010-11-17 at 17:07 +0000, Catalin Marinas wrote:
> > On Mon, 2010-11-15 at 17:55 +0000, Russell King - ARM Linux wrote:
> > > This series is aimed at cleaning up our SMP IPI support.  As has been
> > > recently pointed out, we have a spinlock in percpu data, which is not
> > > very nice.
> > > 
> > > If we switch to using the GIC SGIs, one for each IPI function, then
> > > we don't need the mask of ipi bits indicating what work needs to be
> > > done.  This allows us to get rid of the troublesome spinlock in
> > > send_ipi_message().
> > 
> > FWIW, the patches look fine to me (though I have tested them):
>                                               ^^^^
> I meant I have *not* tested them.

They work fine on my Versatile Express, but I've not checked the
Realview EB yet.

[PATCH 3/9] ARM: smp: avoid using bitmasks and locks for IPIs, use hardware instead

[Catalin Marinas]

On Wed, 2010-11-17 at 17:19 +0000, Russell King - ARM Linux wrote:
> On Wed, Nov 17, 2010 at 05:06:39PM +0000, Catalin Marinas wrote:
> > On Mon, 2010-11-15 at 17:58 +0000, Russell King - ARM Linux wrote:
> > > Avoid using bitmasks and locks in the percpu area for IPIs, and instead
> > > use individual software generated interrupts to identify the reason for
> > > the IPI.  This avoids the problems of having spinlocks in the percpu
> > > area.
> > >
> > > Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
> > > ---
> > >  arch/arm/kernel/smp.c |   87 ++++++++++++++----------------------------------
> > >  1 files changed, 26 insertions(+), 61 deletions(-)
> > >
> > > diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
> > > index 7a236db..78d55c6 100644
> > > --- a/arch/arm/kernel/smp.c
> > > +++ b/arch/arm/kernel/smp.c
> > [...]
> > >  enum ipi_msg_type {
> > > -       IPI_TIMER,
> > > +       IPI_TIMER = 2,
> > >         IPI_RESCHEDULE,
> > >         IPI_CALL_FUNC,
> > >         IPI_CALL_FUNC_SINGLE,
> >
> > Maybe a comment here on why this starts at 2. People may wonder in the
> > future.
> 
> I would like to get rid of the 'starts at 2' but we can only do this once
> we're sure that the use of SGI1 to wake up the CPUs is not something that
> is a hard requirement for boot loaders.
> 
> I've not yet tested whether Realview and Versatile Express are fine with
> a different SGI number.

AFAIK, the ARM boot monitor doesn't care about the IPI number, it just
needs something to take it out of the WFI.

BTW, in boot_secondary() we could issue a SEV as well from the primary
CPU. That's mainly useful when you don't have a full boot monitor to set
up the GIC and it uses WFE in a loop (I have some simple wrapper for
this, mainly used on software models).

-- 
Catalin