[PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"

[PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"

[Santosh Shilimkar]

This patch fixes the preempt leak in the cpuidle path invoked from
cpu-hotplug
log:
BUG: scheduling while atomic: swapper/0/0x00000002
<d>Modules linked in:
Backtrace:
[<c0024f90>] (dump_backtrace+0x0/0x110) from [<c01762c4>] (dump_stack+0x18/0x1c)
 r7:c0218cc4 r6:c0341f00 r5:c7836000 r4:00000000
[<c01762ac>] (dump_stack+0x0/0x1c) from [<c003d94c>] (__schedule_bug+0x60/0x70)
[<c003d8ec>] (__schedule_bug+0x0/0x70) from [<c0176914>] (schedule+0x98/0x7b8)
 r5:c7836000 r4:c7836000
[<c017687c>] (schedule+0x0/0x7b8) from [<c00228c4>] (cpu_idle+0xb4/0xd4)
# [<c0022810>] (cpu_idle+0x0/0xd4) from [<c01744e8>] (secondary_start_kernel+0xe0/0xf0)
 r5:c7836000 r4:c020a220
[<c0174408>] (secondary_start_kernel+0x0/0xf0) from [<c0031150>] (set_cpu_power_state+0x40/0x44)
 r7:c0218cc4 r6:00000001 r5:00000001 r4:c7836000
Backtrace aborted due to bad frame pointer <c7837fbc>

Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
---
 arch/arm/kernel/smp.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index 57162af..2a42d22 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -226,6 +226,7 @@ void __ref cpu_die(void)
 	 * actual CPU shutdown procedure is at least platform (if not
 	 * CPU) specific
 	 */
+	preempt_enable_no_resched();
 	platform_cpu_die(cpu);
 
 	/*
-- 
1.6.0.4

[PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"

[Russell King - ARM Linux]

On Tue, Mar 16, 2010 at 06:42:29PM +0530, Santosh Shilimkar wrote:
> This patch fixes the preempt leak in the cpuidle path invoked from
> cpu-hotplug
> log:
> BUG: scheduling while atomic: swapper/0/0x00000002
> <d>Modules linked in:
> Backtrace:
> [<c0024f90>] (dump_backtrace+0x0/0x110) from [<c01762c4>] (dump_stack+0x18/0x1c)
>  r7:c0218cc4 r6:c0341f00 r5:c7836000 r4:00000000
> [<c01762ac>] (dump_stack+0x0/0x1c) from [<c003d94c>] (__schedule_bug+0x60/0x70)
> [<c003d8ec>] (__schedule_bug+0x0/0x70) from [<c0176914>] (schedule+0x98/0x7b8)
>  r5:c7836000 r4:c7836000
> [<c017687c>] (schedule+0x0/0x7b8) from [<c00228c4>] (cpu_idle+0xb4/0xd4)
> # [<c0022810>] (cpu_idle+0x0/0xd4) from [<c01744e8>] (secondary_start_kernel+0xe0/0xf0)
>  r5:c7836000 r4:c020a220
> [<c0174408>] (secondary_start_kernel+0x0/0xf0) from [<c0031150>] (set_cpu_power_state+0x40/0x44)
>  r7:c0218cc4 r6:00000001 r5:00000001 r4:c7836000
> Backtrace aborted due to bad frame pointer <c7837fbc>

Can you describe the conditions which leads up to this please?

[PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"

[Shilimkar, Santosh]

> -----Original Message-----
> From: Russell King - ARM Linux [mailto:linux at arm.linux.org.uk]
> Sent: Wednesday, March 17, 2010 2:20 AM
> To: Shilimkar, Santosh
> Cc: linux-arm-kernel at lists.infradead.org; Catalin Marinas
> Subject: Re: [PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"
> 
> On Tue, Mar 16, 2010 at 06:42:29PM +0530, Santosh Shilimkar wrote:
> > This patch fixes the preempt leak in the cpuidle path invoked from
> > cpu-hotplug
> > log:
> > BUG: scheduling while atomic: swapper/0/0x00000002
> > <d>Modules linked in:
> > Backtrace:
> > [<c0024f90>] (dump_backtrace+0x0/0x110) from [<c01762c4>] (dump_stack+0x18/0x1c)
> >  r7:c0218cc4 r6:c0341f00 r5:c7836000 r4:00000000
> > [<c01762ac>] (dump_stack+0x0/0x1c) from [<c003d94c>] (__schedule_bug+0x60/0x70)
> > [<c003d8ec>] (__schedule_bug+0x0/0x70) from [<c0176914>] (schedule+0x98/0x7b8)
> >  r5:c7836000 r4:c7836000
> > [<c017687c>] (schedule+0x0/0x7b8) from [<c00228c4>] (cpu_idle+0xb4/0xd4)
> > # [<c0022810>] (cpu_idle+0x0/0xd4) from [<c01744e8>] (secondary_start_kernel+0xe0/0xf0)
> >  r5:c7836000 r4:c020a220
> > [<c0174408>] (secondary_start_kernel+0x0/0xf0) from [<c0031150>] (set_cpu_power_state+0x40/0x44)
> >  r7:c0218cc4 r6:00000001 r5:00000001 r4:c7836000
> > Backtrace aborted due to bad frame pointer <c7837fbc>
> 
> Can you describe the conditions which leads up to this please?

On SMP kernel with PREEMPT enabled, if you try to offline the CPU using hot-plug and then
bring it back(online), you will get this dump. Since CPU is taken out from idle thread, the
process shown is 2.

Regards,
Santosh

[PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"

[Russell King - ARM Linux]

On Wed, Mar 17, 2010 at 10:51:44AM +0530, Shilimkar, Santosh wrote:
> On SMP kernel with PREEMPT enabled, if you try to offline the CPU using hot-plug and then
> bring it back(online), you will get this dump. Since CPU is taken out from idle thread, the
> process shown is 2.

Hmm, I don't think this is how we should be solving this problem.
x86 calls init_idle() on the idle task when it's re-used, and this
resets the preempt count amongst other things.  So maybe we want to
do this instead in the __cpu_up path:

	if (!idle) {
		idle = fork_idle(cpu);
		if (IS_ERR(idle)) {
			printk(KERN_ERR "CPU%u: fork() failed\n", cpu);
			return PTR_ERR(idle);
		}
		ci->idle = idle;
-	}
+	} else {
+		init_idle(idle);
+	}

[PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"

[Shilimkar, Santosh]

> -----Original Message-----
> From: Russell King - ARM Linux [mailto:linux at arm.linux.org.uk]
> Sent: Friday, March 19, 2010 5:43 PM
> To: Shilimkar, Santosh
> Cc: linux-arm-kernel at lists.infradead.org; Catalin Marinas
> Subject: Re: [PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"
> 
> On Wed, Mar 17, 2010 at 10:51:44AM +0530, Shilimkar, Santosh wrote:
> > On SMP kernel with PREEMPT enabled, if you try to offline the CPU using hot-plug and then
> > bring it back(online), you will get this dump. Since CPU is taken out from idle thread, the
> > process shown is 2.
> 
> Hmm, I don't think this is how we should be solving this problem.
> x86 calls init_idle() on the idle task when it's re-used, and this
> resets the preempt count amongst other things.  So maybe we want to
> do this instead in the __cpu_up path:
> 
> 	if (!idle) {
> 		idle = fork_idle(cpu);
> 		if (IS_ERR(idle)) {
> 			printk(KERN_ERR "CPU%u: fork() failed\n", cpu);
> 			return PTR_ERR(idle);
> 		}
> 		ci->idle = idle;
> -	}
> +	} else {
> +		init_idle(idle);
> +	}
This doesn't fix the dump. Will have look at this in more details on the same lines
as you suggested.

- Santosh

[PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"

[Sundar R IYER]

Hi Santosh/Russell,


>Hmm, I don't think this is how we should be solving this problem.

>x86 calls init_idle() on the idle task when it's re-used, and this

>resets the preempt count amongst other things.  So maybe we want to

>do this instead in the __cpu_up path:

The following patch fixes the dump messages at my end during re-onlining the offline CPU.

diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index 577543f..ee1d608 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -86,6 +86,9 @@ int __cpuinit __cpu_up(unsigned int cpu)
                                                return PTR_ERR(idle);
                                }
                                ci->idle = idle;
+              } else {
+                              /* fix any preempt leaks whilst going down */
+                              init_idle(idle, cpu);
                }

Please let me know if this is alright.

Thanx!
Sundar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20100416/51e730a4/attachment.htm>

[PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"

[Sundar R IYER]

Hi Santosh/Russell,

>Hmm, I don't think this is how we should be solving this problem.
>x86 calls init_idle() on the idle task when it's re-used, and this
>resets the preempt count amongst other things.  So maybe we want to
>do this instead in the __cpu_up path:

The following patch fixes the dump messages at my end during re-onlining the offline CPU.

diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index 577543f..ee1d608 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -86,6 +86,9 @@ int __cpuinit __cpu_up(unsigned int cpu)
                                                return PTR_ERR(idle);
                                }
                                ci->idle = idle;
+              } else {
+                              /* fix any preempt leaks whilst going down */
+                              init_idle(idle, cpu);
                }

Please let me know if this is alright.

Thanx!
Sundar

[PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"

[Shilimkar, Santosh]

> -----Original Message-----
> From: linux-arm-kernel-bounces at lists.infradead.org [mailto:linux-arm-kernel-
> bounces at lists.infradead.org] On Behalf Of Sundar R IYER
> Sent: Friday, April 16, 2010 7:14 PM
> To: linux-arm-kernel at lists.infradead.org
> Subject: Re : [PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"
> 
> Hi Santosh/Russell,
> 
> >Hmm, I don't think this is how we should be solving this problem.
> >x86 calls init_idle() on the idle task when it's re-used, and this
> >resets the preempt count amongst other things.  So maybe we want to
> >do this instead in the __cpu_up path:
> 
> The following patch fixes the dump messages at my end during re-onlining the offline CPU.
> 
> diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
> index 577543f..ee1d608 100644
> --- a/arch/arm/kernel/smp.c
> +++ b/arch/arm/kernel/smp.c
> @@ -86,6 +86,9 @@ int __cpuinit __cpu_up(unsigned int cpu)
>                                                 return PTR_ERR(idle);
>                                 }
>                                 ci->idle = idle;
> +              } else {
> +                              /* fix any preempt leaks whilst going down */
> +                              init_idle(idle, cpu);
>                 }
> 
Actually I found there is while the CPU is going down. If you fix that, both paths
- going down and coming up - gets fixed.

Regards,
Santosh

[PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"

[Shilimkar, Santosh]

> -----Original Message-----
> From: linux-arm-kernel-bounces at lists.infradead.org [mailto:linux-arm-kernel-
> bounces at lists.infradead.org] On Behalf Of Shilimkar, Santosh
> Sent: Friday, April 16, 2010 7:31 PM
> To: Sundar R IYER; linux-arm-kernel at lists.infradead.org
> Subject: RE: Re : [PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"
> 
> > -----Original Message-----
> > From: linux-arm-kernel-bounces at lists.infradead.org [mailto:linux-arm-kernel-
> > bounces at lists.infradead.org] On Behalf Of Sundar R IYER
> > Sent: Friday, April 16, 2010 7:14 PM
> > To: linux-arm-kernel at lists.infradead.org
> > Subject: Re : [PATCH] ARM: Fix "BUG: scheduling while atomic: swapper/0/0x00000002"
> >
> > Hi Santosh/Russell,
> >
> > >Hmm, I don't think this is how we should be solving this problem.
> > >x86 calls init_idle() on the idle task when it's re-used, and this
> > >resets the preempt count amongst other things.  So maybe we want to
> > >do this instead in the __cpu_up path:
> >
> > The following patch fixes the dump messages at my end during re-onlining the offline CPU.
> >
> > diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
> > index 577543f..ee1d608 100644
> > --- a/arch/arm/kernel/smp.c
> > +++ b/arch/arm/kernel/smp.c
> > @@ -86,6 +86,9 @@ int __cpuinit __cpu_up(unsigned int cpu)
> >                                                 return PTR_ERR(idle);
> >                                 }
> >                                 ci->idle = idle;
> > +              } else {
> > +                              /* fix any preempt leaks whilst going down */
> > +                              init_idle(idle, cpu);
> >                 }
> >
> Actually I found there is while the CPU is going down. If you fix that, both paths
> - going down and coming up - gets fixed.
> 
Will check this again and confirm.

Regards,
Santosh