-next March 3: Boot failure on x86 (Oops)

Sachin Sant
2010-03-03T15:44:29Z

[-- Attachment #1: Type: text/plain, Size: 2039 bytes --]

Today's next failed to boot on x86 box with following trace

Unpacking initramfs...
Freeing initrd memory: 10584k freed
BUG: unable to handle kernel NULL pointer dereference at (null)
IP: [<c01a6b12>] pcpu_alloc+0x1cb/0x75e
*pdpt = 00000000005dd001 *pde = 0000000000000000
Oops: 0000 [#1] SMP
last sysfs file:
Modules linked in:

Pid: 1, comm: swapper Not tainted 2.6.33-autotest-next-20100303 #1 /eserver xSeries 235 -[86717AX]-
EIP: 0060:[<c01a6b12>] EFLAGS: 00010002 CPU: 1
EIP is at pcpu_alloc+0x1cb/0x75e
EAX: 00000000 EBX: c05c4100 ECX: cccccccc EDX: 00000000
ESI: 000000b0 EDI: 00000005 EBP: f5c69fa8 ESP: f5c69f2c
DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
Process swapper (pid: 1, ti=f5c68000 task=f5c66ce0 task.ti=f5c68000)
Stack:
00000000 00000005 61746164 0a383d29 5f656400 61746164 00000004 000000b0
<0> 000a3036 00000286 61637061 29656863 0a36313d c0579100 c058da30 f5c69f94
<0> c01310be c05c41cc 00000028 c058da30 f5c69fa4 c0116a73 c0492091 c0492089
Call Trace:
[<c058da30>] ? crash_save_vmcoreinfo_init+0x0/0x31a
[<c01310be>] ? log_buf_kexec_setup+0x3f/0x67
[<c058da30>] ? crash_save_vmcoreinfo_init+0x0/0x31a
[<c0116a73>] ? arch_crash_save_vmcoreinfo+0x37/0x3c
[<c058d9ff>] ? crash_notes_memory_init+0x0/0x31
[<c01a70be>] ? __alloc_percpu+0xa/0xc
[<c058da11>] ? crash_notes_memory_init+0x12/0x31
[<c0101139>] ? do_one_initcall+0x4c/0x131
[<c057b352>] ? kernel_init+0x127/0x178
[<c057b22b>] ? kernel_init+0x0/0x178
[<c0102df6>] ? kernel_thread_helper+0x6/0x10
Code: 45 a8 e9 65 ff ff ff 8b 4d 9c 8b 55 a0 8b 45 84 e8 31 fa ff ff 85 c0 89 45 a4 0f 89 fd 00 00 00 8b 45 84 8b 00 89 45 84 8b 55 84 <8b> 02 0f 18 00 90 8b 45 cc 03 05 a0 9b 57 c0 39 c2 0f 85 67 ff
EIP: [<c01a6b12>] pcpu_alloc+0x1cb/0x75e SS:ESP 0068:f5c69f2c
CR2: 0000000000000000
---[ end trace a7919e7f17c0a725 ]---

x86_64 boots fine. Have attached dmesg log.

Thanks
-Sachin

--

---------------------------------
Sachin Sant
IBM Linux Technology Center
India Systems and Technology Labs
Bangalore, India
---------------------------------

[-- Attachment #2: next-0303-log --]
[-- Type: text/plain, Size: 13585 bytes --]

Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.33-autotest-next-20100303 (root@x235b) (gcc version 4.3.2 [gcc-4_3-branch revision 141291] (SUSE Linux) ) #1 SMP Wed Mar 3 13:56:30 IST 2010
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009c000 (usable)
BIOS-e820: 000000000009c000 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000005ffd8740 (usable)
BIOS-e820: 000000005ffd8740 - 000000005ffe0000 (ACPI data)
BIOS-e820: 000000005ffe0000 - 0000000060000000 (reserved)
BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
Notice: NX (Execute Disable) protection missing in CPU or disabled in BIOS!
DMI 2.3 present.
last_pfn = 0x5ffd8 max_arch_pfn = 0x1000000
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
found SMP MP-table at [c009c140] 9c140
init_memory_mapping: 0000000000000000-00000000371fe000
RAMDISK: 37599000 - 37fef152
Allocated new RAMDISK: 0077b000 - 011d1152
Move RAMDISK from 0000000037599000 - 0000000037fef151 to 0077b000 - 011d1151
ACPI: RSDP 000fdfc0 00014 (v00 IBM )
ACPI: RSDT 5ffdff80 00030 (v01 IBM SERONYXP 00001000 IBM 45444F43)
ACPI: FACP 5ffdff00 00074 (v01 IBM SERONYXP 00001000 IBM 45444F43)
ACPI: DSDT 5ffd8740 075D2 (v01 IBM SERGEODE 00001000 MSFT 0100000B)
ACPI: FACS 5ffdfe00 00040
ACPI: APIC 5ffdfe40 00092 (v01 IBM SERONYXP 00001000 IBM 45444F43)
ACPI: ASF! 5ffdfd80 0004B (v16 IBM SERONYXP 00000001 IBM 45444F43)
Node: 0, start_pfn: 0, end_pfn: 5ffd8
Reserving total of e00 pages for numa KVA remap
kva_start_pfn ~ 36200 max_low_pfn ~ 371fe
max_pfn = 5ffd8
653MB HIGHMEM available.
881MB LOWMEM available.
mapped low ram: 0 - 371fe000
low ram: 0 - 371fe000
Zone PFN ranges:
DMA 0x00000001 -> 0x00001000
Normal 0x00001000 -> 0x000371fe
HighMem 0x000371fe -> 0x0005ffd8
Movable zone start PFN for each node
early_node_map[3] active PFN ranges
0: 0x00000001 -> 0x0000009c
0: 0x00000100 -> 0x0005f000
0: 0x0005fe00 -> 0x0005ffd8
Using APIC driver default
ACPI: PM-Timer IO Port: 0x488
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x06] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x07] enabled)
ACPI: LAPIC_NMI (acpi_id[0x00] dfl dfl lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x06] dfl dfl lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] dfl dfl lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x07] dfl dfl lint[0x1])
ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
ACPI: IOAPIC (id[0x0c] address[0xfec02000] gsi_base[32])
IOAPIC[2]: apic_id 12, version 17, address 0xfec02000, GSI 32-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
Using ACPI (MADT) for SMP configuration information
SMP: Allowing 4 CPUs, 0 hotplug CPUs
PM: Registered nosave memory: 000000000009c000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000e0000
PM: Registered nosave memory: 00000000000e0000 - 0000000000100000
Allocating PCI resources starting at 60000000 (gap: 60000000:9ec00000)
setup_percpu: NR_CPUS:128 nr_cpumask_bits:128 nr_cpu_ids:4 nr_node_ids:8
PERCPU: Embedded 13 pages/cpu @c1200000 s29140 r0 d24108 u524288
pcpu-alloc: s29140 r0 d24108 u524288 alloc=1*2097152
pcpu-alloc: [0] 0 1 2 3
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 386419
Policy zone: HighMem
Kernel command line: root=/dev/sda2 console=tty0 console=ttyS0,9600n1 IDENT=1267607421
PID hash table entries: 4096 (order: 2, 16384 bytes)
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
allocated 7863500 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
Subtract (48 early reservations)
#1 [0000001000 - 0000002000] EX TRAMPOLINE
#2 [0000100000 - 00007731c4] TEXT DATA BSS
#3 [0000774000 - 000077a1d8] BRK
#4 [000009c000 - 000009c140] BIOS reserved
#5 [000009c140 - 000009c150] MP-table mpf
#6 [000009c150 - 000009d0a0] BIOS reserved
#7 [000009d22c - 0000100000] BIOS reserved
#8 [000009d0a0 - 000009d22c] MP-table mpc
#9 [0000002000 - 0000003000] TRAMPOLINE
#10 [0000003000 - 0000007000] ACPI WAKEUP
#11 [0000007000 - 000000e000] PGTABLE
#12 [000077b000 - 00011d1152] NEW RAMDISK
#13 [005f000000 - 005fe00000] KVA RAM
#14 [0036200000 - 0037000000] KVA PG
#15 [00011d2000 - 00011d3000] BOOTMEM
#16 [00011d1180 - 00011d1184] BOOTMEM
#17 [00011d11c0 - 00011d1280] BOOTMEM
#18 [00011d1280 - 00011d1324] BOOTMEM
#19 [00011d3000 - 00011d6000] BOOTMEM
#20 [00011d1340 - 00011d13bc] BOOTMEM
#21 [00011d6000 - 00011d9000] BOOTMEM
#22 [00011d13c0 - 00011d144d] BOOTMEM
#23 [00011d1480 - 00011d15a0] BOOTMEM
#24 [00011d15c0 - 00011d1600] BOOTMEM
#25 [00011d1600 - 00011d1640] BOOTMEM
#26 [00011d1640 - 00011d1680] BOOTMEM
#27 [00011d1680 - 00011d16c0] BOOTMEM
#28 [00011d16c0 - 00011d1700] BOOTMEM
#29 [00011d1700 - 00011d1740] BOOTMEM
#30 [00011d1740 - 00011d1780] BOOTMEM
#31 [00011d1780 - 00011d1790] BOOTMEM
#32 [00011d17c0 - 00011d1802] BOOTMEM
#33 [00011d1840 - 00011d1882] BOOTMEM
#34 [0001200000 - 000120d000] BOOTMEM
#35 [0001280000 - 000128d000] BOOTMEM
#36 [0001300000 - 000130d000] BOOTMEM
#37 [0001380000 - 000138d000] BOOTMEM
#38 [00011d18c0 - 00011d18c4] BOOTMEM
#39 [00011d1900 - 00011d1904] BOOTMEM
#40 [00011d1940 - 00011d1950] BOOTMEM
#41 [00011d1980 - 00011d1990] BOOTMEM
#42 [00011d19c0 - 00011d1a58] BOOTMEM
#43 [00011d1a80 - 00011d1ab8] BOOTMEM
#44 [00011d9000 - 00011dd000] BOOTMEM
#45 [000138d000 - 000140d000] BOOTMEM
#46 [000120d000 - 000124d000] BOOTMEM
#47 [000140d000 - 0001b8cccc] BOOTMEM
Initializing HighMem for node 0 (000371fe:0005ffd8)
Memory: 1517652k/1572704k available (2744k kernel code, 40312k reserved, 1842k data, 356k init, 655208k highmem)
virtual kernel memory layout:
fixmap : 0xff5b6000 - 0xfffff000 (10532 kB)
pkmap : 0xff200000 - 0xff400000 (2048 kB)
vmalloc : 0xf79fe000 - 0xff1fe000 ( 120 MB)
lowmem : 0xc0000000 - 0xf71fe000 ( 881 MB)
.init : 0xc057b000 - 0xc05d4000 ( 356 kB)
.data : 0xc03ae3e8 - 0xc057ad88 (1842 kB)
.text : 0xc0100000 - 0xc03ae3e8 (2744 kB)
Checking if this processor honours the WP bit even in supervisor mode...Ok.
Hierarchical RCU implementation.
RCU-based detection of stalled CPUs is enabled.
NR_IRQS:2304
Console: colour VGA+ 80x25
console [tty0] enabled
console [ttyS0] enabled
Fast TSC calibration using PIT
Detected 2793.676 MHz processor.
Calibrating delay loop (skipped), value calculated using timer frequency.. 5587.35 BogoMIPS (lpj=11174704)
Security Framework initialized
SELinux: Disabled at boot.
Mount-cache hash table entries: 512
Initializing cgroup subsys ns
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
mce: CPU supports 4 MCE banks
CPU0: Thermal monitoring enabled (TM1)
Performance Events: no PMU driver, software events only.
Checking 'hlt' instruction... OK.
ACPI: Core revision 20100121
Enabling APIC mode: Flat. Using 3 I/O APICs
Mapping cpu 0 to node 0
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: Intel(R) Xeon(TM) CPU 2.80GHz stepping 07
Booting Node 0, Processors #1
Initializing CPU#1
Mapping cpu 1 to node 0
#2
Initializing CPU#2
Mapping cpu 2 to node 0
#3 Ok.
Initializing CPU#3
Mapping cpu 3 to node 0
Brought up 4 CPUs
Total of 4 processors activated (22352.10 BogoMIPS).
devtmpfs: initialized
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: PCI BIOS revision 2.10 entry at 0xfd7bc, last bus=10
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: Interpreter enabled
ACPI: (supports S0 S4 S5)
ACPI: Using IOAPIC for interrupt routing
PCI: Ignoring host bridge windows from ACPI; if necessary, use "pci=use_crs" and report a bug
ACPI: PCI Root Bridge [PCI0] (0000:00)
ACPI: PCI Root Bridge [PCI1] (0000:02)
ACPI: PCI Root Bridge [PCI2] (0000:05)
ACPI: PCI Root Bridge [PCI3] (0000:07)
ACPI: PCI Root Bridge [PCI4] (0000:09)
vgaarb: device added: PCI:0000:00:09.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: loaded
PCI: Using ACPI for IRQ routing
Switching to clocksource tsc
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 17 devices
ACPI: ACPI bus type pnp unregistered
PnPBIOS: Disabled by ACPI PNP
system 00:01: [io 0x0900-0x093f] has been reserved
system 00:01: [io 0x0510-0x0517] has been reserved
system 00:01: [io 0x0504-0x0507] has been reserved
system 00:01: [io 0x0500-0x0503] has been reserved
system 00:01: [io 0x0520-0x053f] has been reserved
system 00:01: [io 0x0420-0x0427] has been reserved
system 00:01: [io 0x0460-0x0461] has been reserved
system 00:0c: [io 0x01ec-0x01ef] has been reserved
system 00:0c: [io 0x0400-0x04fe] could not be reserved
system 00:0c: [io 0x0600] has been reserved
system 00:0c: [io 0x0800-0x080f] has been reserved
system 00:0c: [io 0x0c00-0x0cfe] could not be reserved
system 00:0c: [io 0x0f50-0x0f58] has been reserved
system 00:0c: [mem 0xfec00000-0xffffffff] could not be reserved
pci 0000:00:09.0: BAR 6: assigned [mem 0x60000000-0x6001ffff pref]
pci 0000:05:03.0: BAR 6: assigned [mem 0x60020000-0x60027fff pref]
NET: Registered protocol family 2
IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
UDP hash table entries: 512 (order: 2, 16384 bytes)
UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
NET: Registered protocol family 1
Unpacking initramfs...
Freeing initrd memory: 10584k freed
BUG: unable to handle kernel NULL pointer dereference at (null)
IP: [<c01a6b12>] pcpu_alloc+0x1cb/0x75e
*pdpt = 00000000005dd001 *pde = 0000000000000000
Oops: 0000 [#1] SMP
last sysfs file:
Modules linked in:

Pid: 1, comm: swapper Not tainted 2.6.33-autotest-next-20100303 #1 /eserver xSeries 235 -[86717AX]-
EIP: 0060:[<c01a6b12>] EFLAGS: 00010002 CPU: 1
EIP is at pcpu_alloc+0x1cb/0x75e
EAX: 00000000 EBX: c05c4100 ECX: cccccccc EDX: 00000000
ESI: 000000b0 EDI: 00000005 EBP: f5c69fa8 ESP: f5c69f2c
DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
Process swapper (pid: 1, ti=f5c68000 task=f5c66ce0 task.ti=f5c68000)
Stack:
00000000 00000005 61746164 0a383d29 5f656400 61746164 00000004 000000b0
<0> 000a3036 00000286 61637061 29656863 0a36313d c0579100 c058da30 f5c69f94
<0> c01310be c05c41cc 00000028 c058da30 f5c69fa4 c0116a73 c0492091 c0492089
Call Trace:
[<c058da30>] ? crash_save_vmcoreinfo_init+0x0/0x31a
[<c01310be>] ? log_buf_kexec_setup+0x3f/0x67
[<c058da30>] ? crash_save_vmcoreinfo_init+0x0/0x31a
[<c0116a73>] ? arch_crash_save_vmcoreinfo+0x37/0x3c
[<c058d9ff>] ? crash_notes_memory_init+0x0/0x31
[<c01a70be>] ? __alloc_percpu+0xa/0xc
[<c058da11>] ? crash_notes_memory_init+0x12/0x31
[<c0101139>] ? do_one_initcall+0x4c/0x131
[<c057b352>] ? kernel_init+0x127/0x178
[<c057b22b>] ? kernel_init+0x0/0x178
[<c0102df6>] ? kernel_thread_helper+0x6/0x10
Code: 45 a8 e9 65 ff ff ff 8b 4d 9c 8b 55 a0 8b 45 84 e8 31 fa ff ff 85 c0 89 45 a4 0f 89 fd 00 00 00 8b 45 84 8b 00 89 45 84 8b 55 84 <8b> 02 0f 18 00 90 8b 45 cc 03 05 a0 9b 57 c0 39 c2 0f 85 67 ff
EIP: [<c01a6b12>] pcpu_alloc+0x1cb/0x75e SS:ESP 0068:f5c69f2c
CR2: 0000000000000000
---[ end trace a7919e7f17c0a725 ]---
Kernel panic - not syncing: Attempted to kill init!
Pid: 1, comm: swapper Tainted: G D 2.6.33-autotest-next-20100303 #1
Call Trace:
[<c03a72e2>] ? printk+0xf/0x15
[<c03a7227>] panic+0x39/0xe5
[<c0132dae>] do_exit+0x5c/0x5e3
[<c0130a7c>] ? kmsg_dump+0xe4/0xf8
[<c012fddb>] ? oops_exit+0x2a/0x2f
[<c03aa5a1>] oops_end+0x93/0x9b
[<c0119f8e>] no_context+0x13f/0x149
[<c0289edf>] ? number+0x11f/0x1da
[<c011a0f8>] __bad_area_nosemaphore+0x160/0x168
[<c028a4b3>] ? string+0x33/0x81
[<c028b3f1>] ? vsnprintf+0x4fe/0x6d1
[<c011a10d>] bad_area_nosemaphore+0xd/0x10
[<c03abf1c>] do_page_fault+0x199/0x2f7
[<c03abd83>] ? do_page_fault+0x0/0x2f7
[<c03a9c86>] error_code+0x66/0x70
[<c015007b>] ? sys_futex+0x93/0xff
[<c03abd83>] ? do_page_fault+0x0/0x2f7
[<c01a6b12>] ? pcpu_alloc+0x1cb/0x75e
[<c058da30>] ? crash_save_vmcoreinfo_init+0x0/0x31a
[<c01310be>] ? log_buf_kexec_setup+0x3f/0x67
[<c058da30>] ? crash_save_vmcoreinfo_init+0x0/0x31a
[<c0116a73>] ? arch_crash_save_vmcoreinfo+0x37/0x3c
[<c058d9ff>] ? crash_notes_memory_init+0x0/0x31
[<c01a70be>] __alloc_percpu+0xa/0xc
[<c058da11>] crash_notes_memory_init+0x12/0x31
[<c0101139>] do_one_initcall+0x4c/0x131
[<c057b352>] kernel_init+0x127/0x178
[<c057b22b>] ? kernel_init+0x0/0x178
[<c0102df6>] kernel_thread_helper+0x6/0x10