2.6.15rc6: ide oops+panic

2.6.15rc6: ide oops+panic

[Andreas Steinmetz]

[-- Attachment #1: Type: text/plain, Size: 171 bytes --]

Attached are boot messages and panic captured via serial console, as
well as the system config.
-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de

[-- Attachment #2: top.log --]
[-- Type: text/plain, Size: 22731 bytes --]

Bootdata ok (command line is BOOT_IMAGE=2.6.15rc6s root=301 video=radeonfb:off console=ttyS0,115200 console=tty0)
Linux version 2.6.15-rc6top (root@gringo) (gcc version 3.4.4) #1 SMP PREEMPT Thu Dec 22 14:23:34 CET 2005
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000bfff0000 (usable)
 BIOS-e820: 00000000bfff0000 - 00000000bffff000 (ACPI data)
 BIOS-e820: 00000000bffff000 - 00000000c0000000 (ACPI NVS)
 BIOS-e820: 00000000ff780000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000140000000 (usable)
SRAT: PXM 0 -> APIC 0 -> Node 0
SRAT: PXM 1 -> APIC 1 -> Node 1
ACPI: [SRAT:0x00] ignored 2 entries of 4 found
SRAT: Node 0 PXM 0 100000-80000000
SRAT: Node 1 PXM 1 80000000-c0000000
SRAT: Node 1 PXM 1 80000000-140000000
SRAT: Node 0 PXM 0 0-80000000
Bootmem setup node 0 0000000000000000-0000000080000000
Bootmem setup node 1 0000000080000000-0000000140000000
ACPI: PM-Timer IO Port: 0x5008
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Processor #1 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x82] disabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x83] disabled)
ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x03] address[0xfe9ff000] gsi_base[24])
IOAPIC[1]: apic_id 3, version 17, address 0xfe9ff000, GSI 24-27
ACPI: IOAPIC (id[0x04] address[0xfe9fe000] gsi_base[28])
IOAPIC[2]: apic_id 4, version 17, address 0xfe9fe000, GSI 28-31
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
Setting APIC routing to flat
ACPI: HPET id: 0x102282a0 base: 0xfec01000
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at c4000000 (gap: c0000000:3f780000)
Checking aperture...
CPU 0: aperture @ f0000000 size 128 MB
CPU 1: aperture @ f0000000 size 128 MB
Built 2 zonelists
Kernel command line: BOOT_IMAGE=2.6.15rc6s root=301 video=radeonfb:off console=ttyS0,115200 console=tty0
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 131072 bytes)
time.c: Using 14.318180 MHz HPET timer.
time.c: Detected 1992.254 MHz processor.
Console: colour VGA+ 80x50
Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
Memory: 4102684k/5242880k available (4188k kernel code, 91168k reserved, 1774k data, 280k init)
Calibrating delay using timer specific routine.. 3988.45 BogoMIPS (lpj=1994226)
Security Framework v1.0.0 initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
selinux_register_security:  Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 0(1) -> Node 0 -> Core 0
mtrr: v2.0 (20020519)
Using local APIC timer interrupts.
Detected 12.451 MHz APIC timer.
Booting processor 1/2 APIC 0x1
Initializing CPU#1
Calibrating delay using timer specific routine.. 3983.84 BogoMIPS (lpj=1991921)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 1(1) -> Node 1 -> Core 0
AMD Opteron(tm) Processor 246 stepping 08
CPU 1: Syncing TSC to CPU 0.
CPU 1: synchronized TSC with CPU 0 (last diff -4 cycles, maxerr 1073 cycles)
Brought up 2 CPUs
time.c: Using HPET based timekeeping.
testing NMI watchdog ... OK.
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: Using configuration type 1
ACPI: Subsystem revision 20050902
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
ACPI: PCI Root Bridge [PCIB] (0000:04)
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 *9 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 *5 6 7 9 10 11 12 14 15)
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI init
pnp: PnP ACPI: found 14 devices
SCSI subsystem initialized
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
hpet0: at MMIO 0xfec01000 (virtual 0xffffffffff5fe000), IRQs 2, 8, 0
hpet0: 3 32-bit timers, 14318180 Hz
agpgart: Detected AMD 8151 AGP Bridge rev B2
agpgart: AGP aperture is 128M @ 0xf0000000
PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture
pnp: 00:09: ioport range 0x680-0x6ff has been reserved
pnp: 00:09: ioport range 0x295-0x296 has been reserved
PCI: Bridge: 0000:00:06.0
  IO window: 6000-7fff
  MEM window: fe200000-fe3fffff
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:0a.0
  IO window: 8000-9fff
  MEM window: fe400000-fe4fffff
  PREFETCH window: cde00000-cdefffff
PCI: Bridge: 0000:00:0b.0
  IO window: a000-afff
  MEM window: fe500000-fe8fffff
  PREFETCH window: cdf00000-cdffffff
PCI: Bridge: 0000:04:01.0
  IO window: c000-cfff
  MEM window: fea00000-feafffff
  PREFETCH window: ce100000-ee0fffff
IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak Exp $
audit: initializing netlink socket (disabled)
audit(1135374123.884:1): initialized
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
SELinux:  Registering netfilter hooks
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered
PCI: MSI quirk detected. pci_msi_quirk set.
PCI: MSI quirk detected. pci_msi_quirk set.
ACPI: Power Button (FF) [PWRF]
ACPI: Power Button (CM) [PWRB]
ACPI: Processor [CPU1] (supports 8 throttling states)
lp: driver loaded but no devices found
Real Time Clock Driver v1.12
Non-volatile memory driver v1.2
hw_random: AMD768 system management I/O registers at 0x5000.
hw_random hardware driver 1.0.0 loaded
Linux agpgart interface v0.101 (c) Dave Jones
[drm] Initialized drm 1.0.0 20040925
GSI 16 sharing vector 0xA9 and IRQ 16
ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI 16 (level, low) -> IRQ 16
[drm] Initialized radeon 1.19.0 20050911 on minor 0: 
PNP: PS/2 Controller [PNP0303:PS2K,PNP0f03:PS2M] at 0x60,0x64 irq 1,12
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:06: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:07: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
loop: loaded (max 8 devices)
pktcdvd: v0.2.0a 2004-07-14 Jens Axboe (axboe@suse.de) and petero2@telia.com
Intel(R) PRO/1000 Network Driver - version 6.1.16-k2-NAPI
Copyright (c) 1999-2005 Intel Corporation.
GSI 17 sharing vector 0xB1 and IRQ 17
ACPI: PCI Interrupt 0000:03:03.0[A] -> GSI 28 (level, low) -> IRQ 17
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
GSI 18 sharing vector 0xB9 and IRQ 18
ACPI: PCI Interrupt 0000:03:03.1[B] -> GSI 29 (level, low) -> IRQ 18
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
tg3.c:v3.45 (Dec 13, 2005)
GSI 19 sharing vector 0xC1 and IRQ 19
ACPI: PCI Interrupt 0000:02:09.0[A] -> GSI 24 (level, low) -> IRQ 19
eth2: Tigon3 [partno(BCM95703A30) rev 1002 PHY(5703)] (PCI:33MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:28:25:e3
eth2: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1] 
eth2: dma_rwctrl[763f0000]
PPP generic driver version 2.4.2
PPP Deflate Compression module registered
PPP BSD Compression module registered
NET: Registered protocol family 24
SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256).
CSLIP: code copyright 1989 Regents of the University of California.
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
AMD8111: IDE controller at PCI slot 0000:00:07.1
AMD8111: chipset revision 3
AMD8111: not 100% native mode: will probe irqs later
AMD8111: 0000:00:07.1 (rev 03) UDMA133 controller
    ide0: BM-DMA at 0xffa0-0xffa7, BIOS settings: hda:DMA, hdb:pio
    ide1: BM-DMA at 0xffa8-0xffaf, BIOS settings: hdc:DMA, hdd:pio
hda: WDC WD2500JB-32FUA0, ATA DISK drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hdc: WDC WD2500JB-32FUA0, ATA DISK drive
ide1 at 0x170-0x177,0x376 on irq 15
PDC20268: IDE controller at PCI slot 0000:02:08.0
GSI 20 sharing vector 0xC9 and IRQ 20
ACPI: PCI Interrupt 0000:02:08.0[A] -> GSI 27 (level, low) -> IRQ 20
PDC20268: chipset revision 2
PDC20268: ROM enabled at 0xfe4e0000
PDC20268: 100% native mode on irq 20
    ide2: BM-DMA at 0x8800-0x8807, BIOS settings: hde:pio, hdf:pio
    ide3: BM-DMA at 0x8808-0x880f, BIOS settings: hdg:pio, hdh:pio
hde: WDC WD2500JB-32FUA0, ATA DISK drive
ide2 at 0x9800-0x9807,0x9402 on irq 20
hdg: PLEXTOR DVDR PX-708A, ATAPI CD/DVD-ROM drive
ide3 at 0x9000-0x9007,0x8c02 on irq 20
hda: max request size: 1024KiB
hda: 488397168 sectors (250059 MB) w/8192KiB Cache, CHS=30401/255/63, UDMA(100)
hda: cache flushes supported
 hda: hda1 hda2 hda3 hda4 < hda5 hda6 hda7 >
hdc: max request size: 1024KiB
hdc: 488397168 sectors (250059 MB) w/8192KiB Cache, CHS=30401/255/63, UDMA(100)
hdc: cache flushes supported
 hdc: hdc1 hdc2 hdc3 hdc4 < hdc5 hdc6 hdc7 >
hde: max request size: 1024KiB
hde: 488397168 sectors (250059 MB) w/8192KiB Cache, CHS=30401/255/63, UDMA(100)
hde: cache flushes supported
 hde: hde1 hde2 hde3 hde4 < hde5 hde6 hde7 >
hdg: ATAPI 40X DVD-ROM DVD-R CD-R/RW drive, 2048kB Cache
Uniform CD-ROM driver Revision: 3.20
st: Version 20050830, fixed bufsize 32768, s/g segs 256
ohci1394: $Rev: 1313 $ Ben Collins <bcollins@debian.org>
GSI 21 sharing vector 0xD1 and IRQ 21
ACPI: PCI Interrupt 0000:01:0c.0[A] -> GSI 19 (level, low) -> IRQ 21
ohci1394: fw-host0: OHCI-1394 1.1 (PCI): IRQ=[21]  MMIO=[fe3fd000-fe3fd7ff]  Max Packet=[2048]
video1394: Installed video1394 module
ieee1394: raw1394: /dev/raw1394 device initialized
sbp2: $Rev: 1306 $ Ben Collins <bcollins@debian.org>
ieee1394: sbp2: Driver forced to serialize I/O (serialize_io=1)
ieee1394: sbp2: Try serialize_io=0 for better performance
usbmon: debugfs is not available
GSI 22 sharing vector 0xD9 and IRQ 22
ACPI: PCI Interrupt 0000:01:0a.2[C] -> GSI 18 (level, low) -> IRQ 22
ehci_hcd 0000:01:0a.2: EHCI Host Controller
ehci_hcd 0000:01:0a.2: new USB bus registered, assigned bus number 1
ehci_hcd 0000:01:0a.2: irq 22, io mem 0xfe3fdc00
ehci_hcd 0000:01:0a.2: USB 2.0 started, EHCI 0.95, driver 10 Dec 2004
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ACPI: PCI Interrupt 0000:01:00.0[D] -> GSI 19 (level, low) -> IRQ 21
ohci_hcd 0000:01:00.0: OHCI Host Controller
ohci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 2
ohci_hcd 0000:01:00.0: irq 21, io mem 0xfe3fe000
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 3 ports detected
ACPI: PCI Interrupt 0000:01:00.1[D] -> GSI 19 (level, low) -> IRQ 21
ohci_hcd 0000:01:00.1: OHCI Host Controller
ohci_hcd 0000:01:00.1: new USB bus registered, assigned bus number 3
ohci_hcd 0000:01:00.1: irq 21, io mem 0xfe3ff000
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 3 ports detected
ACPI: PCI Interrupt 0000:01:0a.0[A] -> GSI 16 (level, low) -> IRQ 16
ohci_hcd 0000:01:0a.0: OHCI Host Controller
ohci_hcd 0000:01:0a.0: new USB bus registered, assigned bus number 4
ohci_hcd 0000:01:0a.0: irq 16, io mem 0xfe3fb000
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 3 ports detected
hub 4-0:1.0: over-current change on port 1
GSI 23 sharing vector 0xE1 and IRQ 23
ACPI: PCI Interrupt 0000:01:0a.1[B] -> GSI 17 (level, low) -> IRQ 23
ohci_hcd 0000:01:0a.1: OHCI Host Controller
ohci_hcd 0000:01:0a.1: new USB bus registered, assigned bus number 5
ohci_hcd 0000:01:0a.1: irq 23, io mem 0xfe3fc000
hub 5-0:1.0: USB hub found
hub 5-0:1.0: 2 ports detected
usb 4-1: new full speed USB device using ohci_hcd and address 2
hub 4-0:1.0: over-current change on port 2
USB Universal Host Controller Interface driver v2.3
Initializing USB Mass Storage driver...
hub 4-0:1.0: over-current change on port 3
scsi0 : SCSI emulation for USB Mass Storage devices
hub 5-0:1.0: over-current change on port 1
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
mice: PS/2 mouse device common for all mice
md: raid5 personality registered as nr 4
raid5: automatically using best checksumming function: generic_sse
   generic_sse:  6072.000 MB/sec
raid5: using function: generic_sse (6072.000 MB/sec)
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
device-mapper: 4.4.0-ioctl (2005-01-12) initialised: dm-devel@redhat.com
Advanced Linux Sound Architecture Driver Version 1.0.10rc3 (Mon Nov 07 13:30:21 2005 UTC).
ACPI: PCI Interrupt 0000:00:07.5[B] -> GSI 17 (level, low) -> IRQ 23
input: AT Translated Set 2 keyboard as /class/input/input0
intel8x0_measure_ac97_clock: measured 51379 usecs
intel8x0: clocking to 48000
ALSA device list:
  #0: AMD AMD8111 with AD1981B at 0xb800, irq 23
u32 classifier
    OLD policer on 
NET: Registered protocol family 2
IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
ip_conntrack version 2.4 (8192 buckets, 65536 max) - 288 bytes per conntrack
ip_tables: (C) 2000-2002 Netfilter core team
hub 5-0:1.0: over-current change on port 2
ipt_recent v0.3.1: Stephen Frost <sfrost@snowman.net>.  http://snowman.net/projects/ipt_recent/
arp_tables: (C) 2002 David S. Miller
TCP bic registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
NET: Registered protocol family 15
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
input: ImExPS/2 Generic Explorer Mouse as /class/input/input1
md: Autodetecting RAID arrays.
md: autorun ...
md: considering hde6 ...
md:  adding hde6 ...
md: hde3 has different UUID to hde6
md:  adding hdc6 ...
md: hdc3 has different UUID to hde6
md:  adding hda6 ...
md: hda3 has different UUID to hde6
md: created md1
md: bind<hda6>
md: bind<hdc6>
md: bind<hde6>
md: running: <hde6><hdc6><hda6>
raid5: device hde6 operational as raid disk 2
raid5: device hdc6 operational as raid disk 1
raid5: device hda6 operational as raid disk 0
raid5: allocated 3212kB for md1
raid5: raid level 5 set md1 active with 3 out of 3 devices, algorithm 0
RAID5 conf printout:
 --- rd:3 wd:3 fd:0
 disk 0, o:1, dev:hda6
 disk 1, o:1, dev:hdc6
 disk 2, o:1, dev:hde6
md: considering hde3 ...
md:  adding hde3 ...
md:  adding hdc3 ...
md:  adding hda3 ...
md: created md0
md: bind<hda3>
md: bind<hdc3>
md: bind<hde3>
md: running: <hde3><hdc3><hda3>
raid5: device hde3 operational as raid disk 2
raid5: device hdc3 operational as raid disk 1
raid5: device hda3 operational as raid disk 0
raid5: allocated 3212kB for md0
raid5: raid level 5 set md0 active with 3 out of 3 devices, algorithm 0
RAID5 conf printout:
 --- rd:3 wd:3 fd:0
 disk 0, o:1, dev:hda3
 disk 1, o:1, dev:hdc3
 disk 2, o:1, dev:hde3
md: ... autorun DONE.
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 280k freed
----------- [cut here ] --------- [please bite here ] ---------
Kernel BUG at drivers/ide/ide-io.c:63
invalid operand: 0000 [1] PREEMPT SMP 
CPU 0 
Modules linked in: snd_usb_audio snd_usb_lib snd_rawmidi snd_hwdep
Pid: 0, comm: swapper Not tainted 2.6.15-rc6top #1
RIP: 0010:[<ffffffff8038e995>] <ffffffff8038e995>{__ide_end_request+53}
RSP: 0018:ffffffff806d67d0  EFLAGS: 00010046
RAX: 0000000001000479 RBX: ffff810002f7b490 RCX: 0000000000000008
RDX: 0000000000000000 RSI: ffff810002f7b490 RDI: ffffffff80730d98
RBP: 0000000000000000 R08: 000000003b9aca00 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000064 R12: ffffffff80730d98
R13: 0000000000000008 R14: 0000000000000001 R15: ffffffff80730c80
FS:  00002aaaab1aaae0(0000) GS:ffffffff80751800(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00002aaaaafbb7b0 CR3: 000000007e0e0000 CR4: 00000000000006e0
Process swapper (pid: 0, threadinfo ffffffff8075a000, task ffffffff805d36c0)
Stack: ffff810002f7b490 ffff81013feeacd0 ffff810002f5a550 0000000000000050 
       0000000000000050 ffffffff80299aae 0000000000000246 ffff81013feeacd0 
       ffffffff80730d98 ffffffff8038f0f2 
Call Trace: <IRQ> <ffffffff80299aae>{blk_pre_flush_end_io+110}
       <ffffffff8038f0f2>{ide_end_drive_cmd+946} <ffffffff8038f6f0>{drive_cmd_intr+288}
       <ffffffff8038f5d0>{drive_cmd_intr+0} <ffffffff80390850>{ide_intr+400}
       <ffffffff8015c7bc>{handle_IRQ_event+44} <ffffffff8015c8a1>{__do_IRQ+177}
       <ffffffff80110c9f>{do_IRQ+47} <ffffffff8010e2d8>{ret_from_intr+0}
       <ffffffff80390ad0>{ide_outb+0} <ffffffff80390ad6>{ide_outb+6}
       <ffffffff803995b1>{ide_do_rw_disk+529} <ffffffff8049a036>{ip_rcv+1302}
       <ffffffff80120000>{__ioremap+512} <ffffffff80390160>{ide_do_request+1648}
       <ffffffff8038f0f2>{ide_end_drive_cmd+946} <ffffffff80390895>{ide_intr+469}
       <ffffffff8015c7bc>{handle_IRQ_event+44} <ffffffff8015c8a1>{__do_IRQ+177}
       <ffffffff80110c9f>{do_IRQ+47} <ffffffff8010e2d8>{ret_from_intr+0}
        <EOI> <ffffffff8010ebf1>{kernel_thread+129} <ffffffff80390ad0>{ide_outb+0}
       <ffffffff8010bd96>{default_idle+54} <ffffffff8010c012>{cpu_idle+98}
       <ffffffff8075c835>{start_kernel+485} <ffffffff8075c294>{_sinittext+660}
       

Code: 0f 0b 68 61 85 57 80 c2 3f 00 90 a8 02 74 0c 85 ed 7f 08 44 
RIP <ffffffff8038e995>{__ide_end_request+53} RSP <ffffffff806d67d0>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 NMI Watchdog detected LOCKUP on CPU 0
CPU 0 
Modules linked in: snd_usb_audio snd_usb_lib snd_rawmidi snd_hwdep
Pid: 0, comm: swapper Not tainted 2.6.15-rc6top #1
RIP: 0010:[<ffffffff801198f4>] <ffffffff801198f4>{__smp_call_function+116}
RSP: 0018:ffffffff806d6488  EFLAGS: 00000097
RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000002
RDX: 0000ffff0000ffff RSI: 0000000000000000 RDI: ffffffff807560d8
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff80119a40
R13: 0000000000000000 R14: ffffffff805526dd R15: ffffffff80730c80
FS:  00002aaaab1aaae0(0000) GS:ffffffff80751800(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00002aaaaafbb7b0 CR3: 000000007e0e0000 CR4: 00000000000006e0
Process swapper (pid: 0, threadinfo ffffffff8075a000, task ffffffff805d36c0)
Stack: ffffffff80119a40 0000000000000000 ffffffff00000000 ffffffff00000000 
       0000000000000400 0000000000000000 0000000000000000 ffffffff805d36c0 
       000000000000000b ffffffff80119aa0 
Call Trace: <IRQ> <ffffffff80119a40>{smp_really_stop_cpu+0} <ffffffff80119aa0>{smp_send_stop+64}
       <ffffffff80135ca1>{panic+225} <ffffffff8010f543>{show_stack+211}
       <ffffffff8010f639>{show_registers+233} <ffffffff8013869f>{do_exit+143}
       <ffffffff80315327>{do_unblank_screen+119} <ffffffff8010f8f1>{die+81}
       <ffffffff8010fd41>{do_invalid_op+145} <ffffffff8038e995>{__ide_end_request+53}
       <ffffffff80136a1d>{printk+141} <ffffffff804d2b54>{ip_refrag+36}
       <ffffffff8010eaa1>{error_exit+0} <ffffffff8038e995>{__ide_end_request+53}
       <ffffffff80299aae>{blk_pre_flush_end_io+110} <ffffffff8038f0f2>{ide_end_drive_cmd+946}
       <ffffffff8038f6f0>{drive_cmd_intr+288} <ffffffff8038f5d0>{drive_cmd_intr+0}
       <ffffffff80390850>{ide_intr+400} <ffffffff8015c7bc>{handle_IRQ_event+44}
       <ffffffff8015c8a1>{__do_IRQ+177} <ffffffff80110c9f>{do_IRQ+47}
       <ffffffff8010e2d8>{ret_from_intr+0} <ffffffff80390ad0>{ide_outb+0}
       <ffffffff80390ad6>{ide_outb+6} <ffffffff803995b1>{ide_do_rw_disk+529}
       <ffffffff8049a036>{ip_rcv+1302} <ffffffff80120000>{__ioremap+512}
       <ffffffff80390160>{ide_do_request+1648} <ffffffff8038f0f2>{ide_end_drive_cmd+946}
       <ffffffff80390895>{ide_intr+469} <ffffffff8015c7bc>{handle_IRQ_event+44}
       <ffffffff8015c8a1>{__do_IRQ+177} <ffffffff80110c9f>{do_IRQ+47}
       <ffffffff8010e2d8>{ret_from_intr+0}  <EOI> <ffffffff8010ebf1>{kernel_thread+129}
       <ffffffff80390ad0>{ide_outb+0} <ffffffff8010bd96>{default_idle+54}
       <ffffffff8010c012>{cpu_idle+98} <ffffffff8075c835>{start_kernel+485}
       <ffffffff8075c294>{_sinittext+660} 

Code: 39 d8 74 08 f3 90 eb f4 66 66 66 90 85 ed 74 0e 8b 44 24 14 
console shuts up ...
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 NMI Watchdog detected LOCKUP on CPU 1
CPU 1 
Modules linked in: snd_usb_audio snd_usb_lib snd_rawmidi snd_hwdep
Pid: 0, comm: swapper Not tainted 2.6.15-rc6top #1
RIP: 0010:[<ffffffff80513975>] <ffffffff80513975>{_spin_lock_irqsave+117}
RSP: 0018:ffff81013fc5fed8  EFLAGS: 00000002
RAX: 0000000000000000 RBX: ffffffff80755e40 RCX: 0000000000000001
RDX: ffff810002ca3e28 RSI: 0000000000000000 RDI: ffffffff80755e40
RBP: 0000000000000000 R08: ffff810002ca2000 R09: 0000000000000000
R10: 0000000000000002 R11: 0000000000005023 R12: ffff810002ca3e28
R13: 000000000000000f R14: 0000000000000000 R15: ffff810002ca3e28
FS:  00002aaaab0146d0(0000) GS:ffffffff80751880(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00000000004810b0 CR3: 000000013e467000 CR4: 00000000000006e0
Process swapper (pid: 0, threadinfo ffff810002ca2000, task ffff810002c9c080)
Stack: 0000000000000046 ffff810002f67b40 ffff810002f6f400 ffffffff803906ef 
       ffff81013fc5fef8 ffff81013fc5fef8 ffff810002f67b40 0000000000000000 
       ffff810002ca3e28 000000000000000f 
Call Trace: <IRQ> <ffffffff803906ef>{ide_intr+47} <ffffffff8015c7bc>{handle_IRQ_event+44}
       <ffffffff8015c8a1>{__do_IRQ+177} <ffffffff80110c9f>{do_IRQ+47}
       <ffffffff8010e2d8>{ret_from_intr+0}  <EOI> <ffffffff8010bd96>{default_idle+54}
       <ffffffff8010c012>{cpu_idle+98} <ffffffff80767ff9>{start_secondary+1257}
       

Code: f3 90 8b 03 85 c0 7e f1 65 48 8b 04 25 10 00 00 00 ff 80 44 
console shuts up ...
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 

[-- Attachment #3: config.top.gz --]
[-- Type: application/x-gunzip, Size: 9674 bytes --]

Re: 2.6.15rc6: ide oops+panic

[Bartlomiej Zolnierkiewicz]

Hi,

On 12/22/05, Andreas Steinmetz <ast@domdv.de> wrote:
> Attached are boot messages and panic captured via serial console, as
> well as the system config.

Driver OOPS-es on handling write barrier request (on finishing pre-flush)
because REQ_STARTED flag is not set in __ide_end_request()
but I don't see how this can happen, maybe something has changed
in the block layer...  Does 2.6.14 work for you?

Does mounting ext3 with "barrier=0" option workaround the problem?

Thanks,
Bartlomiej

Re: 2.6.15rc6: ide oops+panic

[Andrew Morton]

Andreas Steinmetz <ast@domdv.de> wrote:
>
> Attached are boot messages and panic captured via serial console, as
> well as the system config.
> 
> ...
>
> VFS: Mounted root (ext3 filesystem) readonly.
> Freeing unused kernel memory: 280k freed
> ----------- [cut here ] --------- [please bite here ] ---------
> Kernel BUG at drivers/ide/ide-io.c:63
> invalid operand: 0000 [1] PREEMPT SMP 
> CPU 0 
> Modules linked in: snd_usb_audio snd_usb_lib snd_rawmidi snd_hwdep
> Pid: 0, comm: swapper Not tainted 2.6.15-rc6top #1
> RIP: 0010:[<ffffffff8038e995>] <ffffffff8038e995>{__ide_end_request+53}
> RSP: 0018:ffffffff806d67d0  EFLAGS: 00010046
> RAX: 0000000001000479 RBX: ffff810002f7b490 RCX: 0000000000000008
> RDX: 0000000000000000 RSI: ffff810002f7b490 RDI: ffffffff80730d98
> RBP: 0000000000000000 R08: 000000003b9aca00 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000064 R12: ffffffff80730d98
> R13: 0000000000000008 R14: 0000000000000001 R15: ffffffff80730c80
> FS:  00002aaaab1aaae0(0000) GS:ffffffff80751800(0000) knlGS:0000000000000000
> CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
> CR2: 00002aaaaafbb7b0 CR3: 000000007e0e0000 CR4: 00000000000006e0
> Process swapper (pid: 0, threadinfo ffffffff8075a000, task ffffffff805d36c0)
> Stack: ffff810002f7b490 ffff81013feeacd0 ffff810002f5a550 0000000000000050 
>        0000000000000050 ffffffff80299aae 0000000000000246 ffff81013feeacd0 
>        ffffffff80730d98 ffffffff8038f0f2 
> Call Trace: <IRQ> <ffffffff80299aae>{blk_pre_flush_end_io+110}
>        <ffffffff8038f0f2>{ide_end_drive_cmd+946} <ffffffff8038f6f0>{drive_cmd_intr+288}
>        <ffffffff8038f5d0>{drive_cmd_intr+0} <ffffffff80390850>{ide_intr+400}
>        <ffffffff8015c7bc>{handle_IRQ_event+44} <ffffffff8015c8a1>{__do_IRQ+177}
>        <ffffffff80110c9f>{do_IRQ+47} <ffffffff8010e2d8>{ret_from_intr+0}
>        <ffffffff80390ad0>{ide_outb+0} <ffffffff80390ad6>{ide_outb+6}
>        <ffffffff803995b1>{ide_do_rw_disk+529} <ffffffff8049a036>{ip_rcv+1302}
>        <ffffffff80120000>{__ioremap+512} <ffffffff80390160>{ide_do_request+1648}
>        <ffffffff8038f0f2>{ide_end_drive_cmd+946} <ffffffff80390895>{ide_intr+469}
>        <ffffffff8015c7bc>{handle_IRQ_event+44} <ffffffff8015c8a1>{__do_IRQ+177}
>        <ffffffff80110c9f>{do_IRQ+47} <ffffffff8010e2d8>{ret_from_intr+0}
>         <EOI> <ffffffff8010ebf1>{kernel_thread+129} <ffffffff80390ad0>{ide_outb+0}
>        <ffffffff8010bd96>{default_idle+54} <ffffffff8010c012>{cpu_idle+98}
>        <ffffffff8075c835>{start_kernel+485} <ffffffff8075c294>{_sinittext+660}
>        
> 
> Code: 0f 0b 68 61 85 57 80 c2 3f 00 90 a8 02 74 0c 85 ed 7f 08 44 
> RIP <ffffffff8038e995>{__ide_end_request+53} RSP <ffffffff806d67d0>

Thanks.  Are you able to identify the most-recent kernel version which
didn't do this?

Re: 2.6.15rc6: ide oops+panic

[Andreas Steinmetz]

Bartlomiej Zolnierkiewicz wrote:
> Driver OOPS-es on handling write barrier request (on finishing pre-flush)
> because REQ_STARTED flag is not set in __ide_end_request()
> but I don't see how this can happen, maybe something has changed
> in the block layer...  Does 2.6.14 work for you?
> 
> Does mounting ext3 with "barrier=0" option workaround the problem?
> 

Testing the workaround now. I'll know for sure if the system survives
for the next few days.

-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de

Re: 2.6.15rc6: ide oops+panic

[Andreas Steinmetz]

Andrew Morton wrote:
> 
> Thanks.  Are you able to identify the most-recent kernel version which
> didn't do this?
> 

I'm trying currently Bartlomiej's workaround as I need some image
analysis results on this machine which will take a few days.

>From the back of my head:

2.6.13 seemed to run fine

2.6.14.2 which I tried just lately caused freezes which seem similar to
2.6.15rc6 (unfortunately without serial console, so no logs)

I can do further tests in a few days, if the workaround helps, otherwise
sooner.
-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de

Re: 2.6.15rc6: ide oops+panic

[Andreas Steinmetz]

Andrew Morton wrote:
> Thanks.  Are you able to identify the most-recent kernel version which
> didn't do this?
> 

Bartlomiej's workaround (mount with "barrier=0") doesn't seem to help to
workaround the problem. I had one BUG/oops/panic (the same as reported)
after 96 hours of uptime and another one after only a few minutes of uptime.

Some more info on the disk setup (all partitions including root):

hda(1)/hdc(1)/hde(2) <-> 2x md raid5 <-> dm-crypt <-> lvm2 <-> ext3

(1) AMD8111
(2) PDC20268

Swap is set up as:

hda(1)/hdc(1)/hde(2) <-> dm-crypt <-> swap

(1) AMD8111
(2) PDC20268

I could start trying to find out where the problem started but will have
to start with 2.6.13 as I had other problems with earlier kernels.
Please let me know if you want me to go through this as each test run
will probably take a few days.

BTW: To prevent bit error speculations, just before initially reporting
the problem I did run >200 hours of memtest with no errors and all
memory is ECC.

If it helps, here is some information from /proc/interrupts:

           CPU0       CPU1
  0:     357569    1104266    IO-APIC-edge  timer
  1:          1         35    IO-APIC-edge  i8042
  8:          0          1    IO-APIC-edge  rtc
  9:        240       2288   IO-APIC-level  acpi
 12:          1         92    IO-APIC-edge  i8042
 14:       4509     115633    IO-APIC-edge  ide0
 15:       1822     117156    IO-APIC-edge  ide1
 16:          1        195   IO-APIC-level  ohci_hcd:usb4
 19:        875          2   IO-APIC-level  eth0
 20:       1376     115699   IO-APIC-level  ide2, ide3
 21:          0          3   IO-APIC-level  ohci1394, ohci_hcd:usb2,
ohci_hcd:usb3
 22:          0          4   IO-APIC-level  ehci_hcd:usb1
 23:          0          0   IO-APIC-level  ohci_hcd:usb5, AMD AMD8111
NMI:        221        161
LOC:    1460699    1461160
ERR:          0
MIS:          0

-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de

Re: 2.6.15rc6: ide oops+panic

[Lee Revell]

On Tue, 2005-12-27 at 00:17 +0100, Andreas Steinmetz wrote:
> Andrew Morton wrote:
> > Thanks.  Are you able to identify the most-recent kernel version which
> > didn't do this?
> > 
> 
> Bartlomiej's workaround (mount with "barrier=0") doesn't seem to help to
> workaround the problem. I had one BUG/oops/panic (the same as reported)
> after 96 hours of uptime and another one after only a few minutes of uptime.
> 
> Some more info on the disk setup (all partitions including root):
> 
> hda(1)/hdc(1)/hde(2) <-> 2x md raid5 <-> dm-crypt <-> lvm2 <-> ext3

Stack overflow?

Lee

Re: 2.6.15rc6: ide oops+panic

[Andreas Steinmetz]

Lee Revell wrote:
> On Tue, 2005-12-27 at 00:17 +0100, Andreas Steinmetz wrote:
> 
>>Andrew Morton wrote:
>>
>>>Thanks.  Are you able to identify the most-recent kernel version which
>>>didn't do this?
>>>
>>
>>Bartlomiej's workaround (mount with "barrier=0") doesn't seem to help to
>>workaround the problem. I had one BUG/oops/panic (the same as reported)
>>after 96 hours of uptime and another one after only a few minutes of uptime.
>>
>>Some more info on the disk setup (all partitions including root):
>>
>>hda(1)/hdc(1)/hde(2) <-> 2x md raid5 <-> dm-crypt <-> lvm2 <-> ext3
> 
> 
> Stack overflow?

Hasn't x86_64 8k stacks?

-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de

Re: 2.6.15rc6: ide oops+panic

[Andreas Steinmetz]

Andrew Morton wrote:

Ping

<rude>
Do you want me to track this further or shall I place my hardware in the
waste paper bin (Welcome to the pleasure dome of the new development
method)?
</rude>

Some more hint: It seems the system is stable as long as networking
(tg3) is not touched. I'll switch to e1000 for a while and see what
happens. Activating only one processor (booting with maxcpus=1) or
mounting with "barrier=0" doesn't help.

PS:
I'm dearly in need of a four way system based on the followup revision
of my board and two dual core Opterons which is delayed until this
Bug/Ooops/Panic is resolved. Can't develop anymore (due to CPU power
constraints and financial limits). Bad.
-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de