Re: 2.6.24-rc3-mm1 - Kernel Panic on IO-APIC

Kamalesh Babulal
2007-11-21T09:44:22Z

Andrew Morton wrote:
> On Wed, 21 Nov 2007 14:52:26 +0530 Kamalesh Babulal <kamalesh@linux.vnet.ibm.com> wrote:
>
>> Andrew Morton wrote:
>>> On Wed, 21 Nov 2007 11:41:23 +0530 Kamalesh Babulal <kamalesh@linux.vnet.ibm.com> wrote:
>>>
>>>> Hi Andrew,
>>>>
>>>> Kernel panic's across different architectures like powerpc, x86_64,
>>> powerpc complains about IO-APICs??
>>>
>>>> Dentry cache hash table entries: 8388608 (order: 14, 67108864 bytes)
>>>> Inode-cache hash table entries: 4194304 (order: 13, 33554432 bytes)
>>>> Mount-cache hash table entries: 256
>>>> SMP alternatives: switching to UP code
>>>> ACPI: Core revision 20070126
>>>> ..MP-BIOS bug: 8254 timer not connected to IO-APIC
>>>> Kernel panic - not syncing: IO-APIC + timer doesn't work! Try using the 'noapic' kernel parameter
>>> ACPI or x86 breakage, I guess.
>>>
>>> Did 'noapic' work?
>> Hi Andrew,
>>
>> Passing noapic works,
>
> OK.
>
>> but the kernel oops's
>>
>> [ 97.161103] Unable to handle kernel NULL pointer dereference at 0000000000000009 RIP:
>> [ 97.193973] [<ffffffff802341df>] cpu_to_allnodes_group+0x69/0x7c
>> [ 97.245359] PGD 0
>> [ 97.257611] Oops: 0000 [1] SMP
>> [ 97.276638] last sysfs file:
>> [ 97.294417] CPU 0
>> [ 97.306620] Modules linked in:
>> [ 97.325066] Pid: 1, comm: swapper Not tainted 2.6.24-rc3-mm1 #1
>> [ 97.360514] RIP: 0010:[<ffffffff802341df>] [<ffffffff802341df>] cpu_to_allnodes_group+0x69/0x7c
>> [ 97.413287] RSP: 0000:ffff81012fabb650 EFLAGS: 00010286
>> [ 97.445363] RAX: ffffffff809bb060 RBX: ffff81012fabb650 RCX: 00000000000000ff
>> [ 97.488378] RDX: 0000000000000001 RSI: 000000000000013e RDI: 0000000000000100
>> [ 97.531413] RBP: ffff81012fabb680 R08: ffff81012fa88180 R09: 0000000000000000
>> [ 97.574428] R10: 0000000000000000 R11: 0000000000000000 R12: ffff810001005f50
>> [ 97.617394] R13: 0000000000000000 R14: ffff81012fa88180 R15: ffff810001005f40
>> [ 97.660421] FS: 0000000000000000(0000) GS:ffffffff806c3000(0000) knlGS:0000000000000000
>> [ 97.709327] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
>> [ 97.743995] CR2: 0000000000000009 CR3: 0000000000201000 CR4: 00000000000006a0
>> [ 97.787021] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> [ 97.830053] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> [ 97.873036] Process swapper (pid: 1, threadinfo FFFF81012FABA000, task FFFF81012FAB8040)
>> [ 97.921993] Stack: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 97.971056] ffff810001005f40 ffff81012fabb700 ffff81012fabbdf0 ffffffff80235487
>> [ 98.016420] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 98.060324] Call Trace:
>> [ 98.076657] [<ffffffff80235487>] build_sched_domains+0x1e1/0xc19
>> [ 98.113383] [<ffffffff8025072a>] __kernel_text_address+0x22/0x30
>> [ 98.150173] [<ffffffff8025b127>] check_chain_key+0x9c/0x15f
>> [ 98.184355] [<ffffffff8025d544>] mark_lock+0x3b/0x5b3
>> [ 98.215406] [<ffffffff8025db06>] mark_held_locks+0x4a/0x6a
>> [ 98.249027] [<ffffffff80284f4a>] get_page_from_freelist+0x42a/0x77d
>> [ 98.287362] [<ffffffff8025dda1>] trace_hardirqs_on+0x198/0x1c3
>> [ 98.323123] [<ffffffff8028527a>] get_page_from_freelist+0x75a/0x77d
>> [ 98.361429] [<ffffffff8025d544>] mark_lock+0x3b/0x5b3
>> [ 98.392427] [<ffffffff8025b127>] check_chain_key+0x9c/0x15f
>> [ 98.426621] [<ffffffff8034d01a>] number+0x115/0x21f
>> [ 98.456594] [<ffffffff8025072a>] __kernel_text_address+0x22/0x30
>> [ 98.493362] [<ffffffff8020d80c>] dump_trace+0x248/0x25d
>> [ 98.525493] [<ffffffff8025b127>] check_chain_key+0x9c/0x15f
>> [ 98.559678] [<ffffffff8025f03f>] __lock_acquire+0xdee/0xf06
>> [ 98.593868] [<ffffffff8025b127>] check_chain_key+0x9c/0x15f
>> [ 98.628038] [<ffffffff8025b127>] check_chain_key+0x9c/0x15f
>> [ 98.662225] [<ffffffff8025b127>] check_chain_key+0x9c/0x15f
>> [ 98.696370] [<ffffffff8025f03f>] __lock_acquire+0xdee/0xf06
>> [ 98.730563] [<ffffffff8025b127>] check_chain_key+0x9c/0x15f
>> [ 98.764689] [<ffffffff8025d544>] mark_lock+0x3b/0x5b3
>> [ 98.795767] [<ffffffff8025db06>] mark_held_locks+0x4a/0x6a
>> [ 98.829432] [<ffffffff8034d01a>] number+0x115/0x21f
>> [ 98.859460] [<ffffffff804ca267>] kprobe_flush_task+0x63/0xa9
>> [ 98.894166] [<ffffffff8034ddbd>] vsnprintf+0x58f/0x5d5
>> [ 98.925739] [<ffffffff8034de6b>] sprintf+0x68/0x6a
>> [ 98.955257] [<ffffffff8025f1c9>] lock_acquire+0x72/0xe0
>> [ 98.987363] [<ffffffff8025ca45>] lock_acquired+0x57/0x1d4
>> [ 99.020446] [<ffffffff8025f430>] lock_release+0x67/0x21a
>> [ 99.053079] [<ffffffff8025b127>] check_chain_key+0x9c/0x15f
>> [ 99.087261] [<ffffffff8025d544>] mark_lock+0x3b/0x5b3
>> [ 99.118328] [<ffffffff8025d544>] mark_lock+0x3b/0x5b3
>> [ 99.149394] [<ffffffff80236330>] arch_init_sched_domains+0x27/0x69
>> [ 99.187217] [<ffffffff802a5a5f>] dbg_redzone2+0x2a/0x52
>> [ 99.219320] [<ffffffff802a656b>] cache_alloc_debugcheck_after+0x16e/0x1cb
>> [ 99.260779] [<ffffffff802a8083>] kmem_cache_alloc+0x15e/0x182
>> [ 99.295944] [<ffffffff80236365>] arch_init_sched_domains+0x5c/0x69
>> [ 99.333768] [<ffffffff8098e501>] sched_init_smp+0x27/0x113
>> [ 99.367400] [<ffffffff8034ff35>] __bitmap_weight+0x78/0x8d
>> [ 99.401090] [<ffffffff8097e631>] kernel_init+0x12d/0x315
>> [ 99.433718] [<ffffffff804c6f57>] _spin_unlock_irq+0x2b/0x30
>> [ 99.467842] [<ffffffff8025dda1>] trace_hardirqs_on+0x198/0x1c3
>> [ 99.503534] [<ffffffff8025dda1>] trace_hardirqs_on+0x198/0x1c3
>> [ 99.539251] [<ffffffff8020d1b8>] child_rip+0xa/0x12
>> [ 99.569234] [<ffffffff8020c8cf>] restore_args+0x0/0x30
>> [ 99.600845] [<ffffffff8097e504>] kernel_init+0x0/0x315
>> [ 99.632426] [<ffffffff8020d1ae>] child_rip+0x0/0x12
>> [ 99.662455]
>> [ 99.671637] INFO: lockdep is turned off.
>> [ 99.695385]
>> [ 99.695385] Code: 48 03 42 08 49 89 04 24 48 83 c4 20 89 c8 5b 41 5c c9 c3 55
>> [ 99.750603] RIP [<ffffffff802341df>] cpu_to_allnodes_group+0x69/0x7c
>> [ 99.789632] RSP <ffff81012fabb650>
>
> urgh, mess. Enabling frame pointers might help here.
>
> But we're cc'ing the right guy ;)
>

The kernel was compiled with frame pointers enabled.
--
Thanks & Regards,
Kamalesh Babulal,
Linux Technology Center,
IBM, ISTL.