[RFC PATCH] ARM: keep __my_cpu_offset consistent with generic one

[RFC PATCH] ARM: keep __my_cpu_offset consistent with generic one

[Ming Lei]

Commit 14318efb(ARM: 7587/1: implement optimized percpu variable access)
introduces arm's __my_cpu_offset to optimize percpu vaiable access,
which really works well on hackbench test, but may cause __my_cpu_offset
to return garbage value before it is initialized in cpu_init() called
by setup_arch, so accessing a percpu variable before setup_arch may cause
kernel hang. But the generic__my_cpu_offset always returns zero before
percpu area is brought up.

So the patch tries to clear __my_cpu_offset on boot CPU early
to avoid boot hang.

At least now percpu variable is accessed by lockdep before
setup_arch(), and enabling CONFIG_LOCK_STAT or CONFIG_DEBUG_LOCKDEP
can trigger kernel hang.

Cc: Tejun Heo <tj@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Rob Herring <rob.herring@calxeda.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Nicolas Pitre <nico@linaro.org>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
 arch/arm/kernel/setup.c |    7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 3f6cbb2..bc7ee4b 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -439,6 +439,13 @@ void __init smp_setup_processor_id(void)
 	for (i = 1; i < nr_cpu_ids; ++i)
 		cpu_logical_map(i) = i == cpu ? 0 : i;
 
+	/*
+	 * clear __my_cpu_offset on boot CPU early to avoid hang
+	 * caused by accessing percpu variable before percpu area is
+	 * brought up
+	 */
+	set_my_cpu_offset(0);
+
 	printk(KERN_INFO "Booting Linux on physical CPU 0x%x\n", mpidr);
 }
 
-- 
1.7.9.5

[RFC PATCH] ARM: keep __my_cpu_offset consistent with generic one

[Will Deacon]

Hello,

On Thu, Mar 07, 2013 at 02:47:48AM +0000, Ming Lei wrote:
> Commit 14318efb(ARM: 7587/1: implement optimized percpu variable access)
> introduces arm's __my_cpu_offset to optimize percpu vaiable access,
> which really works well on hackbench test, but may cause __my_cpu_offset
> to return garbage value before it is initialized in cpu_init() called
> by setup_arch, so accessing a percpu variable before setup_arch may cause
> kernel hang. But the generic__my_cpu_offset always returns zero before
> percpu area is brought up.
> 
> So the patch tries to clear __my_cpu_offset on boot CPU early
> to avoid boot hang.
> 
> At least now percpu variable is accessed by lockdep before
> setup_arch(), and enabling CONFIG_LOCK_STAT or CONFIG_DEBUG_LOCKDEP
> can trigger kernel hang.

For the record, can you include a backtrace or something please? The
description makes it sounds like a caller bug, so it would be good to
document a valid user of per-cpu before cpu_init().

Will

[RFC PATCH] ARM: keep __my_cpu_offset consistent with generic one

[Ming Lei]

Hi,

On Thu, Mar 7, 2013 at 11:44 AM, Will Deacon <will.deacon@arm.com> wrote:
>
> For the record, can you include a backtrace or something please? The

lock_release_holdtime() will access percpu variable, so every lock
release may trigger the hang if CONFIG_LOCK_STAT is enabled,
and there are many lock acquire/release before setup_arch.

No detailed backtrace since only early_printk can work at that time.

> description makes it sounds like a caller bug, so it would be good to
> document a valid user of per-cpu before cpu_init().

OK, I will add the document: lockdep will use percpu variable
before cpu_init() called inside setup_arch().

Thanks,
-- 
Ming Lei