From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Georg C. F. Greve" Subject: PROBLEM: Kernel 2.6.10 crashing repeatedly and hard Date: Thu, 30 Dec 2004 01:31:37 +0100 Message-ID: Mime-Version: 1.0 Content-Type: multipart/signed; boundary="20041230013137+010063835142-11532450-112591835"; micalg=pgp-sha1; protocol="application/pgp-signature" Return-path: Sender: linux-raid-owner@vger.kernel.org To: linux-kernel@vger.kernel.org Cc: dm-crypt@saout.de List-Id: linux-raid.ids --20041230013137+010063835142-11532450-112591835 Content-Type: multipart/mixed; boundary="20041230013137+0100848599494631431-50327522" --20041230013137+0100848599494631431-50327522 Hi all, I've been moving things on my server to use software RAID5 of the LVM, trying out to use the device mapper (dm-crypt) on top of that with an ext3 filesystem and have seen repeated hard crashes. The machine was entirely dead. Since the machine was quite stable for a couple of days with 2.6.10 before moving to the software RAID setup, my suspicion is that it is related to either LVM, DM, EXT3 or any combination of the three. This is what I could preserve in output from the crashes: --20041230013137+0100848599494631431-50327522 Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename=crash1 Content-Transfer-Encoding: quoted-printable Call Trace: [] cache_flusharray+0x41/0xb0 [] kmem_cache_free+0x38/0x40 [] free_buffer_head+0x1f/0x70 [] try_to_free_buffers+0x59/0x90 [] journal_try_to_free_buffers+0xbd/0x130 [] ext3_releasepage+0x30/0x60 [] try_to_release_page+0x39/0x50 [] shrink_list+0x35f/0x440 [] shrink_cache+0x187/0x430 [] mb_cache_shrink_fn+0x167/0x170 [] shrink_slab+0x82/0x1b0 [] shrink_zone+0xb2/0xe0 [] balance_pgdat+0x24e/0x2d0 [] kswapd+0xdc/0x100 [] autoremove_wake_function+0x0/0x40 [] ret_from_fork+0x6/0x14 [] autoremove_wake_function+0x0/0x40 [] kswapd+0x0/0x100 [] kernel_thread_helper+0x5/0x18 Code: 7e 8d 46 38 89 04 24 8b 44 24 1c 8b 15 10 c0 53 c0 8b 0c b8 8d 81 00 = 00 00 40 c1 e8 0c c1 e0 05 8b 5c 02 1c 8b 53 04 8b =C3=9F3 89 02 <89> 50 04 8b 4= 3 0c c7 03 00 01 10 00 29 c1 c7 43 04 00 02 20 00 <6>note: kswapd0[196] exited with preempt_count 1 --20041230013137+0100848599494631431-50327522 Content-Disposition: attachment; filename=crash2 [] alloc_page_buffers+0x1e/0x90 [] create_empty_buffers+0x18/0x90 [] __block_prepare_write+0x373/0x3c0 [] block_prepare_write+0x20/0x30 [] ext3_get_block+0x0/0x70 [] ext3_prepare_write+0x58/0x110 [] ext3_get_block+0x0/0x70 [] generic_file_buffered_write+0x19f/0x600 [] autoremove_wake_function+0x0/0x40 [] __generic_file_aio_write_nolock+0x37/0x90 [] generic_file_aio_write_nolock+0x37/0x90 [] generic_file_aio_write+0x60/0xc0 [] ext3_file_write+0x2a/0xa0 [] do_sync_write+0xab/0xe0 [] wait_on_page_writeback_range+0x74/0x120 [] autoremove_wake_function+0x0/0x40 [] ext3_sync_file+0xb7/0xc0 [] vfs_write+0x8c/0xd0 [] sys_write+0x3d/0x70 [] syscall_call+0x7/0xb Code: 14 42 25 ff ff 00 00 89 51 10 8b 3c 24 66 8b 04 47 66 89 41 14 8b 44 24 24 3b 50 58 73 06 4e 83 fe ff 75 b5 8b 51 04 8b 01 89 02 <89> 50 04 c7 01 00 01 10 00 c7 41 04 00 02 20 00 66 83 79 14 ff <6>note: mythbackend[16084] exited with preempt_count 1 --20041230013137+0100848599494631431-50327522 Content-Disposition: attachment; filename=crash3 EFLAGS: 00010002 (2.6.10) EIP is at free_block+0x45/0xd0 eax: 46484849 ebx: df2b1000 ecx: df2b1050 edx: df2ab000 esi: c183cd80 edi: 00000001 ebp: 00000018 esp: c188fef8 ds: 007b es: 007b ss: 0068 Process events/0 (pid: 6, threadinfo:c188e000 task=c185ca20) Stack: c183cdb8 c1858810 c1858800 00000018 c183cd80 c0141724 c183cd80 c1858810 00000018 c183ccb8 c183cd80 00000002 c183cce0 c01417c6 c183cd80 c1858800 00000000 c183ccb8 c183ce10 00000003 c170fc20 c183b000 c170fc24 00000000 Call Trace: [] drain_array_locked+0x54&0x80 [] cache_reap+0x75/0x1e0 [] worker_thread+0x197/0x230 [] cache_reap+0x0/0x1e0 [] default_wake_function+0x0/0x20 [] default_wake_function+0x0/0x20 [] worker_thread+0x0/0x230 [] kthread+0xa7/0xb0 [] kthread+0x0/0xb0 [] kernel_thread_helper+0x5/0x18 Code: 7e 8d 46 38 89 04 24 8b 44 24 1c 8b 15 10 c0 53 c0 8b 0c b8 8d 81 00 00 00 40 c1 e8 0c c1 e0 05 8b 5c 02 1c 8b 53 04 8b 03 89 02 <89> 50 04 8b 43 0c c7 03 00 01 10 00 29 c1 c7 43 04 00 02 20 00 <6>note: events/0[6] exited with preempt_count 1 --20041230013137+0100848599494631431-50327522 All of them have in common the notice of some process having "exited with preempt_count 1" and all of them happened within three hours -- this is the first time that a mainline kernel has been behaving so consistently unstable for me, in fact. The machine is a P4 Xeon 2.8GHz running Debian GNU/Linux (sarge) and here are the lspci and lsusb -vvv output and the Kernel configuration file: --20041230013137+0100848599494631431-50327522 Content-Disposition: attachment; filename=lspci.vvv 0000:00:00.0 Host bridge: Intel Corp. 82875P/E7210 Memory Controller Hub (rev 02) Subsystem: ASUSTeK Computer Inc.: Unknown device 80f6 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- SERR- TAbort- SERR- Reset- FastB2B- 0000:00:1d.0 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #1 (rev 02) (prog-if 00 [UHCI]) Subsystem: ASUSTeK Computer Inc. P4P800 Mainboard Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- Reset- FastB2B- 0000:00:1f.0 ISA bridge: Intel Corp. 82801EB/ER (ICH5/ICH5R) LPC Interface Bridge (rev 02) Control: I/O+ Mem+ BusMaster+ SpecCycle+ MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- SERR- TAbort- SERR- Region 1: I/O ports at Region 2: I/O ports at Region 3: I/O ports at Region 4: I/O ports at fc00 [size=16] Region 5: Memory at 40000000 (32-bit, non-prefetchable) [size=1K] 0000:00:1f.2 IDE interface: Intel Corp. 82801EB (ICH5) SATA Controller (rev 02) (prog-if 8f [Master SecP SecO PriP PriO]) Subsystem: ASUSTeK Computer Inc.: Unknown device 80a6 Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- Free Software Foundation Europe (http://fsfeurope.org) GNU Business Network (http://mailman.gnubiz.org) Brave GNU World (http://brave-gnu-world.org) --20041230013137+0100848599494631431-50327522-- --20041230013137+010063835142-11532450-112591835 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.3.92 (GNU/Linux) iD8DBQBB00xrbvivwoZXSsoRAr5iAJ4h0lJ52xnrzKTMDQsA4GmBtXx+rgCgsf61 pBzqiLg6wRoOHm1QO4oLl5M= =i1v1 -----END PGP SIGNATURE----- --20041230013137+010063835142-11532450-112591835--