kernel expection generated with 2.6.21-mm1 kernel boot up

kernel expection generated with 2.6.21-mm1 kernel boot up

[Kamalesh Babulal]

[-- Attachment #1: Type: text/plain, Size: 1857 bytes --]

Hi,

I tried compiling and booting up with 2.6.21-mm1 kernel
on the following machine

Architecture      :    PPC64
CPU Type           :    POWER5 (gr)
Machine Type    :    CHRP IBM,9117-570
Base OS              :    Fedora Core 5

the kernel enters the xmon state after an exception as following
Red Hat nash version 5.0.32 starting

Mounting proc filesystem

Mounting sysfs filesystem

Creating /dev

Creating initial device nodes

Setting up hotplug.

Creating block device nodes.

Loading jbd.ko module

Unable to handle kernel paging request for data at address 0x0083a680

Faulting instruction address: 0xc0000000000cc5e4

cpu 0x1: Vector: 300 (Data Access) at [c000000005373320]

    pc: c0000000000cc5e4: .slab_free+0x6c/0x424

    lr: c00000000002fa70: .pgtable_free_tlb+0xd0/0x140

    sp: c0000000053735a0

   msr: 8000000000009032

   dar: 83a680

 dsisr: 40000000

  current = 0xc000000009408790

  paca    = 0xc0000000004b2700

    pid   = 295, comm = init

enter ? for help

[c000000005373650] c00000000002fa70 .pgtable_free_tlb+0xd0/0x140

[c0000000053736f0] c0000000000b0f40 .free_pgd_range+0x23c/0x3c4

[c0000000053737f0] c0000000000b13cc .free_pgtables+0x100/0x134

[c0000000053738a0] c0000000000b29f0 .exit_mmap+0xcc/0x15c

[c000000005373950] c000000000057808 .mmput+0x78/0x170

[c0000000053739f0] c0000000000d9c38 .flush_old_exec+0x750/0xa74

[c000000005373ad0] c000000000012ef0 .load_elf_binary+0x5f8/0x1a18

[c000000005373c30] c0000000000d8a2c .search_binary_handler+0x124/0x304

[c000000005373ce0] c00000000011728c .compat_do_execve+0x1d0/0x328

[c000000005373d90] c000000000016428 .compat_sys_execve+0x74/0xf8

[c000000005373e30] c00000000000872c syscall_exit+0x0/0x40

--- Exception: c01 (System Call) at 00000000100d9358

SP (ff8b79e0) is in userspace

1:mon>

Attached and console log and config file.

Thanks,
Kamalesh Babulal.

[-- Attachment #2: boot_message --]
[-- Type: text/plain, Size: 5661 bytes --]

Linux version 2.6.21-mm1-autokern1 (root@gekko-lp2.ltc.austin.ibm.com) (gcc version 4.1.0 20060304 (Red Hat 4.1.0-3)) #1 SMP Mon May 7 00:13:58 CDT 2007

[boot]0012 Setup Arch

Node 0 active with no memory

EEH: PCI Enhanced I/O Error Handling Enabled

PPC64 nvram contains 7168 bytes

Zone PFN ranges:

  DMA             0 ->  1048576

  Normal    1048576 ->  1048576

Movable zone start PFN for each node

early_node_map[1] active PFN ranges

    1:        0 ->  1048576

Could not find start_pfn for node 0

Node 0 active with no memory

Node 0 active with no memory

Node 0 active with no memory

Node 0 active with no memory

Node 0 active with no memory

Node 0 active with no memory

Node 0 active with no memory

Node 0 active with no memory

Node 0 active with no memory

Node 0 active with no memory

Node 0 active with no memory

Node 0 active with no memory

[boot]0015 Setup Done

Built 2 zonelists, mobility grouping on.  Total pages: 1034240

Kernel command line: ro console=hvc0 autobench_args: root=/dev/sda7 ABAT:1178515939 

[boot]0020 XICS Init

[boot]0021 XICS Done

PID hash table entries: 4096 (order: 12, 32768 bytes)

Console: colour dummy device 80x25

console handover: boot [udbg0] -> real [hvc0]

Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)

Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)

freeing bootmem node 1

Memory: 4114076k/4194304k available (4728k kernel code, 80228k reserved, 892k data, 577k bss, 260k init)

SLUB: Genslabs=18, HWalign=128, Order=0-4, MinObjects=16, Processors=4, Nodes=16

Security Framework v1.0.0 initialized

SELinux:  Initializing.

selinux_register_security:  Registering secondary module capability

Capability LSM initialized as secondary

Mount-cache hash table entries: 256

Processor 1 found.

Processor 2 found.

Processor 3 found.

Brought up 4 CPUs

migration_cost=4,3

NET: Registered protocol family 16

IOMMU table initialized, virtual merging enabled

usbcore: registered new interface driver usbfs

usbcore: registered new interface driver hub

usbcore: registered new device driver usb

NET: Registered protocol family 2

IP route cache hash table entries: 131072 (order: 8, 1048576 bytes)

TCP established hash table entries: 524288 (order: 11, 12582912 bytes)

TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)

TCP: Hash tables configured (established 524288 bind 65536)

TCP reno registered

checking if image is initramfs... it is

Freeing initrd memory: 1300k freed

scan-log-dump not implemented on this system

audit: initializing netlink socket (disabled)

audit(1178516185.644:1): initialized

Total HugeTLB memory allocated, 0

VFS: Disk quotas dquot_6.5.1

Dquot-cache hash table entries: 512 (order 0, 4096 bytes)

JFS: nTxBlock = 8192, nTxLock = 65536

io scheduler noop registered

io scheduler anticipatory registered (default)

io scheduler deadline registered

io scheduler cfq registered

pci_hotplug: PCI Hot Plug PCI Core version: 0.5

rpaphp: RPA HOT Plug PCI Controller Driver version: 0.1

rpaphp: Slot [0000:00:02.4](PCI location=U7879.001.DQD0T7T-P1-C5) registered

Generic RTC Driver v1.07

[drm] Initialized drm 1.1.0 20060810

Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled

RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize

Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2

ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx

ide-floppy driver 0.99.newide

usbmon: debugfs is not available

mice: PS/2 mouse device common for all mice

async_tx: api initialized (sync-only)

xor: measuring software checksumming speed

   8regs     :  6091.000 MB/sec

   8regs_prefetch:  5169.000 MB/sec

   32regs    :  6744.000 MB/sec

   32regs_prefetch:  4828.000 MB/sec

xor: using function: 32regs (6744.000 MB/sec)

usbcore: registered new interface driver hiddev

usbcore: registered new interface driver usbhid

drivers/hid/usbhid/hid-core.c: v2.6:USB HID core driver

TCP cubic registered

Initializing XFRM netlink socket

NET: Registered protocol family 1

NET: Registered protocol family 17

Freeing unused kernel memory: 260k freed

Red Hat nash version 5.0.32 starting

Mounting proc filesystem

Mounting sysfs filesystem

Creating /dev

Creating initial device nodes

Setting up hotplug.

Creating block device nodes.

Loading jbd.ko module

Unable to handle kernel paging request for data at address 0x0083a680

Faulting instruction address: 0xc0000000000cc5e4

cpu 0x1: Vector: 300 (Data Access) at [c000000005373320]

    pc: c0000000000cc5e4: .slab_free+0x6c/0x424

    lr: c00000000002fa70: .pgtable_free_tlb+0xd0/0x140

    sp: c0000000053735a0

   msr: 8000000000009032

   dar: 83a680

 dsisr: 40000000

  current = 0xc000000009408790

  paca    = 0xc0000000004b2700

    pid   = 295, comm = init

enter ? for help

[c000000005373650] c00000000002fa70 .pgtable_free_tlb+0xd0/0x140

[c0000000053736f0] c0000000000b0f40 .free_pgd_range+0x23c/0x3c4

[c0000000053737f0] c0000000000b13cc .free_pgtables+0x100/0x134

[c0000000053738a0] c0000000000b29f0 .exit_mmap+0xcc/0x15c

[c000000005373950] c000000000057808 .mmput+0x78/0x170

[c0000000053739f0] c0000000000d9c38 .flush_old_exec+0x750/0xa74

[c000000005373ad0] c000000000012ef0 .load_elf_binary+0x5f8/0x1a18

[c000000005373c30] c0000000000d8a2c .search_binary_handler+0x124/0x304

[c000000005373ce0] c00000000011728c .compat_do_execve+0x1d0/0x328

[c000000005373d90] c000000000016428 .compat_sys_execve+0x74/0xf8

[c000000005373e30] c00000000000872c syscall_exit+0x0/0x40

--- Exception: c01 (System Call) at 00000000100d9358

SP (ff8b79e0) is in userspace

1:mon>

[-- Attachment #3: dotconfig.bz2 --]
[-- Type: application/x-bzip, Size: 12371 bytes --]

Re: kernel expection generated with 2.6.21-mm1 kernel boot up

[Michael Ellerman]

[-- Attachment #1: Type: text/plain, Size: 775 bytes --]

On Mon, 2007-05-07 at 16:30 +0530, Kamalesh Babulal wrote:
> Hi,
> 
> I tried compiling and booting up with 2.6.21-mm1 kernel
> on the following machine
> 
> Architecture      :    PPC64
> CPU Type           :    POWER5 (gr)
> Machine Type    :    CHRP IBM,9117-570
> Base OS              :    Fedora Core 5
> 

> [boot]0012 Setup Arch
> 
> Node 0 active with no memory

This looks bad.

Can you attach a tarball of /proc/device-tree from the system (booted
with a different kernel obviously).

cheers

-- 
Michael Ellerman
OzLabs, IBM Australia Development Lab

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: kernel expection generated with 2.6.21-mm1 kernel boot up

[Kamalesh Babulal]

[-- Attachment #1: Type: text/plain, Size: 824 bytes --]

***********************
Warning: Your file, device-tress.tar.bz, contains more than 32 files after decompression and cannot be scanned.
***********************


Michael Ellerman wrote:
> On Mon, 2007-05-07 at 16:30 +0530, Kamalesh Babulal wrote:
>   
>> Hi,
>>
>> I tried compiling and booting up with 2.6.21-mm1 kernel
>> on the following machine
>>
>> Architecture      :    PPC64
>> CPU Type           :    POWER5 (gr)
>> Machine Type    :    CHRP IBM,9117-570
>> Base OS              :    Fedora Core 5
>>
>>     
>
>   
>> [boot]0012 Setup Arch
>>
>> Node 0 active with no memory
>>     
>
> This looks bad.
>
> Can you attach a tarball of /proc/device-tree from the system (booted
> with a different kernel obviously).
>
> cheers
>
>   
Find the tarball of the /proc/device-tree attached.

Thanks,
Kamalesh Babulal.


[-- Attachment #2: device-tress.tar.bz --]
[-- Type: application/x-bzip, Size: 12769 bytes --]

Re: kernel expection generated with 2.6.21-mm1 kernel boot up

[Srinivasa Ds]

Kamalesh Babulal wrote:
> 
> 
> Michael Ellerman wrote:
>> On Mon, 2007-05-07 at 16:30 +0530, Kamalesh Babulal wrote:
>>  
>>> Hi,
>>>
>>> I tried compiling and booting up with 2.6.21-mm1 kernel
>>> on the following machine
>>>
>>> Architecture      :    PPC64
>>> CPU Type           :    POWER5 (gr)
>>> Machine Type    :    CHRP IBM,9117-570
>>> Base OS              :    Fedora Core 5
>>>
>>>     
>>
>>  

Looks like problem is in SLUB layer, I unmarked SLUB(CONFIG_SLUB)
option and now system is booting.

Thanks
 Srinivasa DS

Re: kernel expection generated with 2.6.21-mm1 kernel boot up

[Linas Vepstas]

On Tue, May 08, 2007 at 05:34:44PM +0530, Srinivasa Ds wrote:
> 
> Looks like problem is in SLUB layer, I unmarked SLUB(CONFIG_SLUB)
> option and now system is booting.

Its been repored by others (and I can reproduce it),
its a problem with a bad interaction with SLUB and pg_tables
A patch flew by recetnly that would not use slub for pg_tables, 
but I don't know if that's the permanent fix.

--linas

Re: kernel expection generated with 2.6.21-mm1 kernel boot up

[Benjamin Herrenschmidt]

On Tue, 2007-05-08 at 11:23 -0500, Linas Vepstas wrote:
> On Tue, May 08, 2007 at 05:34:44PM +0530, Srinivasa Ds wrote:
> > 
> > Looks like problem is in SLUB layer, I unmarked SLUB(CONFIG_SLUB)
> > option and now system is booting.
> 
> Its been repored by others (and I can reproduce it),
> its a problem with a bad interaction with SLUB and pg_tables
> A patch flew by recetnly that would not use slub for pg_tables, 
> but I don't know if that's the permanent fix.

Working on it... we'll be using __get_free_page() for the lowest level
of page tables.

Ben.