[BUG] 2.6.25-rc5-mm1 kernel panic with "Exception: 501 " on powerpc

[BUG] 2.6.25-rc5-mm1 kernel panic with "Exception: 501 " on powerpc

[Kamalesh Babulal]

Hi Andrew,

The 2.6.25-rc5-mm1 kernel panics while bootup on powerpc

returning from prom_init
Unable to handle kernel paging request for data at address 0x00000000
Faulting instruction address: 0xc00000000000d5dc
cpu 0x0: Vector: 300 (Data Access) at [c0000000007636e0]
    pc: c00000000000d5dc: .do_IRQ+0x74/0x1f4
    lr: c00000000000d5a8: .do_IRQ+0x40/0x1f4
    sp: c000000000763960
   msr: 8000000000001032
   dar: 0
 dsisr: 40000000
  current = 0xc000000000688e60
  paca    = 0xc000000000689900
    pid   = 0, comm = swapper
enter ? for help
[c000000000763a00] c000000000004c24 hardware_interrupt_entry+0x24/0x28
--- Exception: 501 (Hardware Interrupt) at c0000000006021b0 .free_bootmem_core+0x94/0xcc
[link register   ] c00000000060373c .free_bootmem_with_active_regions+0x78/0xb8
[c000000000763cf0] c000000000602610 .init_bootmem_core+0x5c/0xfc (unreliable)
[c000000000763d80] c0000000005eb68c .do_init_bootmem+0x964/0xaf0
[c000000000763e50] c0000000005e03b0 .setup_arch+0x1a4/0x218
[c000000000763ee0] c0000000005d76bc .start_kernel+0xe8/0x424
[c000000000763f90] c000000000008590 .start_here_common+0x60/0xd0

-- 
Thanks & Regards,
Kamalesh Babulal,
Linux Technology Center,
IBM, ISTL.

Re: [BUG] 2.6.25-rc5-mm1 kernel panic with "Exception: 501 " on powerpc

[Andrew Morton]

On Wed, 12 Mar 2008 18:25:37 +0530 Kamalesh Babulal <kamalesh@linux.vnet.ibm.com> wrote:

> Hi Andrew,
> 
> The 2.6.25-rc5-mm1 kernel panics while bootup on powerpc
> 
> returning from prom_init
> Unable to handle kernel paging request for data at address 0x00000000
> Faulting instruction address: 0xc00000000000d5dc
> cpu 0x0: Vector: 300 (Data Access) at [c0000000007636e0]
>     pc: c00000000000d5dc: .do_IRQ+0x74/0x1f4
>     lr: c00000000000d5a8: .do_IRQ+0x40/0x1f4
>     sp: c000000000763960
>    msr: 8000000000001032
>    dar: 0
>  dsisr: 40000000
>   current = 0xc000000000688e60
>   paca    = 0xc000000000689900
>     pid   = 0, comm = swapper
> enter ? for help
> [c000000000763a00] c000000000004c24 hardware_interrupt_entry+0x24/0x28
> --- Exception: 501 (Hardware Interrupt) at c0000000006021b0 .free_bootmem_core+0x94/0xcc
> [link register   ] c00000000060373c .free_bootmem_with_active_regions+0x78/0xb8
> [c000000000763cf0] c000000000602610 .init_bootmem_core+0x5c/0xfc (unreliable)
> [c000000000763d80] c0000000005eb68c .do_init_bootmem+0x964/0xaf0
> [c000000000763e50] c0000000005e03b0 .setup_arch+0x1a4/0x218
> [c000000000763ee0] c0000000005d76bc .start_kernel+0xe8/0x424
> [c000000000763f90] c000000000008590 .start_here_common+0x60/0xd0
> 

Beats me.  Maybe we're still enabling interrupts too early.  But the new
semaphore code got fixed (didn't it?)

Re: [BUG] 2.6.25-rc5-mm1 kernel panic with "Exception: 501 " on powerpc

[Matthew Wilcox]

On Wed, Mar 12, 2008 at 10:46:45AM -0700, Andrew Morton wrote:
> On Wed, 12 Mar 2008 18:25:37 +0530 Kamalesh Babulal <kamalesh@linux.vnet.ibm.com> wrote:
> Beats me.  Maybe we're still enabling interrupts too early.  But the new
> semaphore code got fixed (didn't it?)

On the 7th, according to my records.  Easy to check -- look in
kernel/semaphore.c and see whether down() is using spin_lock_irqsave
(good) or spin_lock_irq (bad).

-- 
Intel are signing my paycheques ... these opinions are still mine
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."

Re: [BUG] 2.6.25-rc5-mm1 kernel panic with "Exception: 501 " on powerpc

[Badari Pulavarty]

On Wed, 2008-03-12 at 18:25 +0530, Kamalesh Babulal wrote:
> Hi Andrew,
> 
> The 2.6.25-rc5-mm1 kernel panics while bootup on powerpc
> 
> returning from prom_init
> Unable to handle kernel paging request for data at address 0x00000000
> Faulting instruction address: 0xc00000000000d5dc
> cpu 0x0: Vector: 300 (Data Access) at [c0000000007636e0]
>     pc: c00000000000d5dc: .do_IRQ+0x74/0x1f4
>     lr: c00000000000d5a8: .do_IRQ+0x40/0x1f4
>     sp: c000000000763960
>    msr: 8000000000001032
>    dar: 0
>  dsisr: 40000000
>   current = 0xc000000000688e60
>   paca    = 0xc000000000689900
>     pid   = 0, comm = swapper
> enter ? for help
> [c000000000763a00] c000000000004c24 hardware_interrupt_entry+0x24/0x28
> --- Exception: 501 (Hardware Interrupt) at c0000000006021b0 .free_bootmem_core+0x94/0xcc
> [link register   ] c00000000060373c .free_bootmem_with_active_regions+0x78/0xb8
> [c000000000763cf0] c000000000602610 .init_bootmem_core+0x5c/0xfc (unreliable)
> [c000000000763d80] c0000000005eb68c .do_init_bootmem+0x964/0xaf0
> [c000000000763e50] c0000000005e03b0 .setup_arch+0x1a4/0x218
> [c000000000763ee0] c0000000005d76bc .start_kernel+0xe8/0x424
> [c000000000763f90] c000000000008590 .start_here_common+0x60/0xd0

Is this only on one machine ? happens all the time ?

I ran into similar issues on rc3-mm1. rc5-mm1 seems to be working fine
for me on ppc64.

Thanks,
Badari

Re: [BUG] 2.6.25-rc5-mm1 kernel panic with "Exception: 501 " on powerpc

[Benjamin Herrenschmidt]

> Beats me.  Maybe we're still enabling interrupts too early.  But the new
> semaphore code got fixed (didn't it?)

Won't lockdep/irqtrace warn if that happens ? You don't yet have the
lockdep patches for ppc64 (I'm still trying to find out why they break
iSeries) but it should warn of such a spurrious IRQ enable on other
archs too... At least, from a quick look at the code, it -seems- that it
does have such a test.

Cheers,
Ben.

Re: [BUG] 2.6.25-rc5-mm1 kernel panic with "Exception: 501 " on powerpc

[Michael Ellerman]

[-- Attachment #1: Type: text/plain, Size: 934 bytes --]

On Wed, 2008-03-12 at 11:51 -0600, Matthew Wilcox wrote:
> On Wed, Mar 12, 2008 at 10:46:45AM -0700, Andrew Morton wrote:
> > On Wed, 12 Mar 2008 18:25:37 +0530 Kamalesh Babulal <kamalesh@linux.vnet.ibm.com> wrote:
> > Beats me.  Maybe we're still enabling interrupts too early.  But the new
> > semaphore code got fixed (didn't it?)
> 
> On the 7th, according to my records.  Easy to check -- look in
> kernel/semaphore.c and see whether down() is using spin_lock_irqsave
> (good) or spin_lock_irq (bad).

down() looks OK, but there's still a spin_lock_irq() in __down_common(),
although I don't know if it makes sense for us to be in __down() at that
stage.

cheers

-- 
Michael Ellerman
OzLabs, IBM Australia Development Lab

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [BUG] 2.6.25-rc5-mm1 kernel panic with "Exception: 501 " on powerpc

[Matthew Wilcox]

On Thu, Mar 13, 2008 at 09:26:09AM +1100, Michael Ellerman wrote:
> down() looks OK, but there's still a spin_lock_irq() in __down_common(),
> although I don't know if it makes sense for us to be in __down() at that
> stage.

The spin_lock_irq in __down_common is correct.  We're going to schedule(),
so we spin_unlock_irq() to save us passing the flags into the helper
function.  If we had interrupts disabled on entry, there's an Aieee
for that.

-- 
Intel are signing my paycheques ... these opinions are still mine
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."

Re: [BUG] 2.6.25-rc5-mm1 kernel panic with "Exception: 501 " on powerpc

[Kamalesh Babulal]

Matthew Wilcox wrote:
> On Thu, Mar 13, 2008 at 09:26:09AM +1100, Michael Ellerman wrote:
>> down() looks OK, but there's still a spin_lock_irq() in __down_common(),
>> although I don't know if it makes sense for us to be in __down() at that
>> stage.
> 
> The spin_lock_irq in __down_common is correct.  We're going to schedule(),
> so we spin_unlock_irq() to save us passing the flags into the helper
> function.  If we had interrupts disabled on entry, there's an Aieee
> for that.
> 

Hi All,

Sorry for all the noise made :-(, something wrong in the test setup from my end, 
the kernel was 2.6.25-rc3-mm1 not 2.6.25-rc5-mm1. This bug is not seen in the
2.6.25-rc5-mm1 kernel.

-- 
Thanks & Regards,
Kamalesh Babulal,