From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754122Ab2DBCoB (ORCPT ); Sun, 1 Apr 2012 22:44:01 -0400 Received: from cantor2.suse.de ([195.135.220.15]:47296 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753857Ab2DBCn7 (ORCPT ); Sun, 1 Apr 2012 22:43:59 -0400 Date: Mon, 2 Apr 2012 12:43:42 +1000 From: NeilBrown To: =?utf-8?Q?Pawe=C5=82?= Sikora Cc: linux-kernel@vger.kernel.org, gregkh@linuxfoundation.org, arekm@pld-linux.org Subject: Re: [PING] Re: [3.2.2] tasks blocked during matrix auto checking. Message-ID: <20120402124342.5b6b7cbc@notabene.brown> In-Reply-To: <1404353.BmjcZtneqB@localhost> References: <2156863.jdbU24e2Kr@pawels> <20120207094751.51e5321a@notabene.brown> <4142598.MA7Zy4QWhf@vmx> <1404353.BmjcZtneqB@localhost> X-Mailer: Claws Mail 3.7.10 (GTK+ 2.24.7; x86_64-suse-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/IuPO4JhdR2XfdJRGZOfDJ=e"; protocol="application/pgp-signature" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org --Sig_/IuPO4JhdR2XfdJRGZOfDJ=e Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Sun, 01 Apr 2012 14:49:03 +0200 Pawe=C5=82 Sikora wrote: > On Tuesday 07 of February 2012 06:58:39 Pawe=C5=82 Sikora wrote: > > On Tuesday 07 of February 2012 09:47:51 NeilBrown wrote: > > > On Mon, 06 Feb 2012 13:02 +0100 Pawel Sikora = wrote: > > >=20 > > > > Hi, > > > >=20 > > > > on heavy loaded opterons i've noticed some blocked tasks during mat= rix auto checking. > > > > is it a known issue? > > >=20 > > > No.... maybe not too surprising though. > > > The data-check will pause to let other IO through, but if there is lo= ts of IO > > > queued up it could cause some longish delays... > > >=20 > > > 2 minutes does seem a bit long though, so maybe there is a bug somewh= ere.=20 > > >=20 > > > And had 3 consecutive timeouts, so that makes it 6 minutes which real= ly is > > > too long. > > >=20 > > > What sort of array was this? RAID1? RAID5 ?? > >=20 > > ~810GB in raid-10: > >=20 > > md2 : active raid10 sda4[0] sdd4[3] sdc4[2] sdb4[1] > > 849514496 blocks super 1.2 512K chunks 2 far-copies [4/4] [UUUU] >=20 > any news? 3.2.13 still blocks my systems for few minutes. I don't know that I can help unfortunately. There seems be be traces below from both ext3 and ext4. Is that right? What are you using on md0 and md2? Are there other arrays on the same devices? What sort of speed does the data-check report when it is working? The check of 810Gig takes 8329 seconds so that should average 100M/sec. 50M/sec for each device. So the data check seems to be going flat-out. What happens if you reduce /proc/sys/dev/raid/speed_limit_max down to 50000 ? NeilBrown >=20 > [316118.098442] md: data-check of RAID array md0 > [316118.098550] md: minimum _guaranteed_ speed: 1000 KB/sec/disk. > [316118.098651] md: using maximum available idle IO bandwidth (but not mo= re than 200000 KB/sec) for data-check. > [316118.098808] md: using 128k window, over a total of 8000256k. > [316118.105344] md: delaying data-check of md2 until md0 has finished (th= ey share one or more physical units) > [316238.881071] md: md0: data-check done. > [316238.899775] md: data-check of RAID array md2 > [316238.899884] md: minimum _guaranteed_ speed: 1000 KB/sec/disk. > [316238.899985] md: using maximum available idle IO bandwidth (but not mo= re than 200000 KB/sec) for data-check. > [316238.900142] md: using 128k window, over a total of 849514496k. > [318603.174883] INFO: task kjournald:1544 blocked for more than 120 secon= ds. > [318603.174994] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disabl= es this message. > [318603.175128] kjournald D 0000000105b02cce 0 1544 2 0x0= 0000000 > [318603.175351] ffff880404d45cd0 0000000000000046 0000000000000000 ffff8= 808057bb270 > [318603.175712] ffff880405853d40 ffff880404d45fd8 ffff880404d45fd8 ffff8= 80404d45fd8 > [318603.176068] ffff880405cbeb30 ffff880405853d40 ffff880404d45ce0 00000= 001057bb028 > [318603.176423] Call Trace: > [318603.176523] [] schedule+0x3a/0x50 > [318603.176646] [] journal_commit_transaction+0x156/0x= 13d0 [jbd] > [318603.176783] [] ? _raw_spin_lock_irqsave+0x51/0x60 > [318603.176893] [] ? lock_timer_base.isra.28+0x33/0x70 > [318603.177001] [] ? trace_hardirqs_on+0xd/0x10 > [318603.177107] [] ? __init_waitqueue_head+0x60/0x60 > [318603.177213] [] ? try_to_del_timer_sync+0x130/0x130 > [318603.177324] [] kjournald+0xeb/0x250 [jbd] > [318603.177428] [] ? __init_waitqueue_head+0x60/0x60 > [318603.177537] [] ? commit_timeout+0x10/0x10 [jbd] > [318603.177643] [] kthread+0xa2/0xb0 > [318603.177745] [] ? trace_hardirqs_on+0xd/0x10 > [318603.177853] [] kernel_thread_helper+0x4/0x10 > [318603.177957] [] ? retint_restore_args+0x13/0x13 > [318603.178064] [] ? __init_kthread_worker+0x70/0x70 > [318603.178192] [] ? gs_change+0x13/0x13 > [318603.178292] INFO: lockdep is turned off. > [318603.178389] INFO: task jbd2/md1-8:2146 blocked for more than 120 seco= nds. > [318603.178495] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disabl= es this message. > [318603.178628] jbd2/md1-8 D 0000000105b026fc 0 2146 2 0x0= 0000000 > [318603.178844] ffff880405681c90 0000000000000046 0000000000000000 ffff8= 81004673a70 > [318603.179196] ffff880405f99ea0 ffff880405681fd8 ffff880405681fd8 ffff8= 80405681fd8 > [318603.179547] ffff880405d50f50 ffff880405f99ea0 ffff880405681ca0 00000= 00104673828 > [318603.179902] Call Trace: > [318603.179995] [] schedule+0x3a/0x50 > [318603.180099] [] jbd2_journal_commit_transaction+0x1= 9c/0x15f0 [jbd2] > [318603.180236] [] ? _raw_spin_lock_irqsave+0x51/0x60 > [318603.181768] [] ? trace_hardirqs_on+0xd/0x10 > [318603.181871] [] ? try_to_del_timer_sync+0x8d/0x130 > [318603.181977] [] ? __init_waitqueue_head+0x60/0x60 > [318603.182083] [] ? try_to_del_timer_sync+0x130/0x130 > [318603.182191] [] kjournald2+0xbf/0x220 [jbd2] > [318603.182294] [] ? __init_waitqueue_head+0x60/0x60 > [318603.182402] [] ? commit_timeout+0x10/0x10 [jbd2] > [318603.182506] [] kthread+0xa2/0xb0 > [318603.182605] [] ? trace_hardirqs_on+0xd/0x10 > [318603.182709] [] kernel_thread_helper+0x4/0x10 > [318603.182814] [] ? retint_restore_args+0x13/0x13 > [318603.182918] [] ? __init_kthread_worker+0x70/0x70 > [318603.183023] [] ? gs_change+0x13/0x13 > [318603.183123] INFO: lockdep is turned off. > [318603.183224] INFO: task nscd:3084 blocked for more than 120 seconds. > [318603.183328] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disabl= es this message. > [318603.183462] nscd D 0000000105b07655 0 3084 1 0x0= 0000000 > [318603.183685] ffff880404f01df8 0000000000000046 ffff880400000000 00000= 00000000246 > [318603.184049] ffff880405845be0 ffff880404f01fd8 ffff880404f01fd8 ffff8= 80404f01fd8 > [318603.184414] ffff880405d08000 ffff880405845be0 ffff881004cc7a20 00000= 00104cc7a20 > [318603.184776] Call Trace: > [318603.184880] [] schedule+0x3a/0x50 > [318603.184985] [] rwsem_down_failed_common+0xb5/0x150 > [318603.185094] [] rwsem_down_write_failed+0x13/0x20 > [318603.185202] [] call_rwsem_down_write_failed+0x13/0= x20 > [318603.185310] [] ? down_write+0x4d/0x60 > [318603.185415] [] ? sys_mmap_pgoff+0x125/0x230 > [318603.185519] [] sys_mmap_pgoff+0x125/0x230 > [318603.185627] [] ? trace_hardirqs_on_thunk+0x3a/0x3f > [318603.185737] [] sys_mmap+0x1d/0x20 > [318603.185840] [] system_call_fastpath+0x16/0x1b > [318603.185943] INFO: lockdep is turned off. > [318603.186040] INFO: task nscd:3086 blocked for more than 120 seconds. > [318603.186143] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disabl= es this message. > [318603.186283] nscd D 0000000105b03039 0 3086 1 0x0= 0000000 > [318603.186502] ffff880403d9bdf8 0000000000000046 ffff880400000000 00000= 00000000246 > [318603.186858] ffff880405846b30 ffff880403d9bfd8 ffff880403d9bfd8 ffff8= 80403d9bfd8 > [318603.187212] ffff880405cbeb30 ffff880405846b30 ffff881004cc7a20 00000= 00104cc7a20 > [318603.187565] Call Trace: > [318603.187656] [] schedule+0x3a/0x50 > [318603.187756] [] rwsem_down_failed_common+0xb5/0x150 > [318603.187861] [] rwsem_down_write_failed+0x13/0x20 > [318603.187965] [] call_rwsem_down_write_failed+0x13/0= x20 > [318603.188070] [] ? down_write+0x4d/0x60 > [318603.188184] [] ? sys_mmap_pgoff+0x125/0x230 > [318603.188289] [] sys_mmap_pgoff+0x125/0x230 > [318603.188391] [] ? trace_hardirqs_on_thunk+0x3a/0x3f > [318603.188496] [] sys_mmap+0x1d/0x20 > [318603.188597] [] system_call_fastpath+0x16/0x1b > [318603.188700] INFO: lockdep is turned off. > [318603.188797] INFO: task nscd:3087 blocked for more than 120 seconds. > [318603.188901] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disabl= es this message. > [318603.189035] nscd D 0000000105b025c8 0 3087 1 0x0= 0000000 > [318603.189256] ffff880402ddb8b8 0000000000000046 ffff880400000000 00000= 00000000046 > [318603.189615] ffff880405840000 ffff880402ddbfd8 ffff880402ddbfd8 ffff8= 80402ddbfd8 > [318603.189973] ffff880405d08f50 ffff880405840000 ffff880402ddb8b8 00000= 0018108c4a8 > [318603.190332] Call Trace: > [318603.190426] [] ? __wait_on_buffer+0x30/0x30 > [318603.190530] [] schedule+0x3a/0x50 > [318603.190631] [] io_schedule+0x8a/0xd0 > [318603.190733] [] sleep_on_buffer+0x9/0x10 > [318603.190836] [] __wait_on_bit_lock+0x52/0xb0 > [318603.190940] [] ? __wait_on_buffer+0x30/0x30 > [318603.191044] [] out_of_line_wait_on_bit_lock+0x77/0= x90 > [318603.191156] [] ? autoremove_wake_function+0x40/0x40 > [318603.191264] [] __lock_buffer+0x2e/0x30 > [318603.191372] [] do_get_write_access+0x49c/0x500 [jb= d] > [318603.191492] [] ? __find_get_block+0xc6/0x220 > [318603.191597] [] ? __getblk+0x27/0x280 > [318603.191706] [] ? ext3_dirty_inode+0x59/0x90 [ext3] > [318603.191815] [] journal_get_write_access+0x2b/0x50 = [jbd] > [318603.191930] [] __ext3_journal_get_write_access+0x2= 8/0x60 [ext3] > [318603.192071] [] ext3_reserve_inode_write+0x73/0xa0 = [ext3] > [318603.192183] [] ext3_mark_inode_dirty+0x3f/0xc0 [ex= t3] > [318603.192294] [] ? journal_start+0x117/0x190 [jbd] > [318603.192405] [] ext3_dirty_inode+0x59/0x90 [ext3] > [318603.192511] [] __mark_inode_dirty+0x37/0x240 > [318603.192615] [] file_update_time+0xcf/0x130 > [318603.192717] [] ? __set_page_dirty_buffers+0xc4/0xe0 > [318603.192824] [] do_wp_page+0x39c/0x6c0 > [318603.192928] [] handle_pte_fault+0x26d/0xa00 > [318603.193033] [] ? mem_cgroup_count_vm_event+0x16/0x= b0 > [318603.193141] [] handle_mm_fault+0x1d8/0x2e0 > [318603.193245] [] do_page_fault+0x145/0x500 > [318603.193348] [] ? trace_hardirqs_off_thunk+0x3a/0x3c > [318603.193455] [] page_fault+0x25/0x30 > [318603.193558] INFO: lockdep is turned off. > [318603.193667] INFO: task flush-9:1:30825 blocked for more than 120 seco= nds. > [318603.193772] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disabl= es this message. > [318603.193908] flush-9:1 D 0000000105b02afd 0 30825 2 0x0= 0000000 > [318603.194128] ffff880401593840 0000000000000046 0000000000000000 ffff8= 81004673908 > [318603.194493] ffff8804015deb30 ffff880401593fd8 ffff880401593fd8 ffff8= 80401593fd8 > [318603.194866] ffff880405c92df0 ffff8804015deb30 ffff880401593850 00000= 00104673828 > [318603.195232] Call Trace: > [318603.195326] [] schedule+0x3a/0x50 > [318603.195432] [] start_this_handle+0x2b5/0x570 [jbd2] > [318603.195541] [] ? static_obj+0x5a/0x70 > [318603.195645] [] ? __init_waitqueue_head+0x60/0x60 > [318603.195754] [] jbd2__journal_start+0x122/0x1a0 [jb= d2] > [318603.195867] [] jbd2_journal_start+0xe/0x10 [jbd2] > [318603.195990] [] ext4_journal_start_sb+0x7e/0x1d0 [e= xt4] > [318603.196107] [] ? ext4_da_writepages+0x314/0x620 [e= xt4] > [318603.196222] [] ? ext4_meta_trans_blocks+0xa5/0xb0 = [ext4] > [318603.196337] [] ext4_da_writepages+0x314/0x620 [ext= 4] > [318603.196446] [] ? writeback_single_inode+0x1c0/0x3d0 > [318603.196554] [] ? writeback_single_inode+0x1c0/0x3d0 > [318603.196663] [] do_writepages+0x1f/0x40 > [318603.196767] [] writeback_single_inode+0x13d/0x3d0 > [318603.196874] [] writeback_sb_inodes+0x1a8/0x260 > [318603.196979] [] __writeback_inodes_wb+0x96/0xc0 > [318603.197084] [] wb_writeback+0x26b/0x320 > [318603.197187] [] ? lock_timer_base.isra.28+0x33/0x70 > [318603.197294] [] ? get_nr_dirty_inodes+0x52/0x80 > [318603.197399] [] wb_check_old_data_flush+0x97/0xa0 > [318603.197506] [] wb_do_writeback+0x141/0x1c0 > [318603.197610] [] bdi_writeback_thread+0x83/0x2a0 > [318603.197715] [] ? wb_do_writeback+0x1c0/0x1c0 > [318603.197818] [] kthread+0xa2/0xb0 > [318603.197918] [] ? trace_hardirqs_on+0xd/0x10 > [318603.198022] [] kernel_thread_helper+0x4/0x10 > [318603.198128] [] ? finish_task_switch+0x77/0x110 > [318603.198245] [] ? _raw_spin_unlock_irq+0x2b/0x40 > [318603.198350] [] ? retint_restore_args+0x13/0x13 > [318603.198459] [] ? __init_kthread_worker+0x70/0x70 > [318603.198565] [] ? gs_change+0x13/0x13 > [318603.198665] INFO: lockdep is turned off. > [318603.198768] INFO: task map:25641 blocked for more than 120 seconds. > [318603.198873] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disabl= es this message. > [318603.199009] map D 0000000105b024e0 0 25641 25638 0x0= 0020000 > [318603.199234] ffff880872c93998 0000000000000046 ffff880800000000 00000= 00000000046 > [318603.199594] ffff88084fadeb30 ffff880872c93fd8 ffff880872c93fd8 ffff8= 80872c93fd8 > [318603.199953] ffff880405d0cc90 ffff88084fadeb30 ffff880872c93958 00000= 00172c93958 > [318603.200312] Call Trace: > [318603.200407] [] ? __lock_page+0x70/0x70 > [318603.200510] [] schedule+0x3a/0x50 > [318603.200611] [] io_schedule+0x8a/0xd0 > [318603.200714] [] sleep_on_page+0x9/0x10 > [318603.200816] [] __wait_on_bit+0x57/0x80 > [318603.200919] [] wait_on_page_bit+0x70/0x80 > [318603.201021] [] ? autoremove_wake_function+0x40/0x40 > [318603.201135] [] grab_cache_page_write_begin+0x90/0x= e0 > [318603.201242] [] ? trace_hardirqs_on+0xd/0x10 > [318603.201355] [] ext4_da_write_begin+0x98/0x210 [ext= 4] > [318603.201463] [] ? __mark_inode_dirty+0x13b/0x240 > [318603.201581] [] generic_file_buffered_write+0x112/0= x290 > [318603.201687] [] ? _raw_spin_unlock+0x26/0x30 > [318603.201793] [] __generic_file_aio_write+0x229/0x440 > [318603.201899] [] generic_file_aio_write+0x6f/0xe0 > [318603.202005] [] ? task_rq_lock+0x56/0xa0 > [318603.202118] [] ext4_file_write+0xbf/0x270 [ext4] > [318603.202226] [] ? task_rq_lock+0x56/0xa0 > [318603.202330] [] do_sync_write+0xca/0x110 > [318603.202433] [] ? might_fault+0x40/0x90 > [318603.202537] [] ? security_file_permission+0x27/0xb0 > [318603.202644] [] ? rw_verify_area+0x5c/0xf0 > [318603.202748] [] vfs_write+0xae/0x180 > [318603.202849] [] sys_write+0x45/0x90 > [318603.202951] [] cstar_dispatch+0x7/0x33 > [318603.203051] INFO: lockdep is turned off. > [318603.204635] INFO: task map:26039 blocked for more than 120 seconds. > [318603.204738] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disabl= es this message. > [318603.204884] map D 0000000105b026a2 0 26039 26036 0x0= 0020000 > [318603.205104] ffff880435c57868 0000000000000046 ffff880400000000 00000= 00000000046 > [318603.205462] ffff880805085be0 ffff880435c57fd8 ffff880435c57fd8 ffff8= 80435c57fd8 > [318603.205822] ffff880405d56b30 ffff880805085be0 ffff880435c57828 00000= 00135c57828 > [318603.206179] Call Trace: > [318603.206272] [] ? __wait_on_buffer+0x30/0x30 > [318603.206376] [] schedule+0x3a/0x50 > [318603.206475] [] io_schedule+0x8a/0xd0 > [318603.206576] [] sleep_on_buffer+0x9/0x10 > [318603.206678] [] __wait_on_bit_lock+0x52/0xb0 > [318603.206783] [] ? _raw_spin_unlock+0x26/0x30 > [318603.206887] [] ? __wait_on_buffer+0x30/0x30 > [318603.206990] [] out_of_line_wait_on_bit_lock+0x77/0= x90 > [318603.207096] [] ? autoremove_wake_function+0x40/0x40 > [318603.207200] [] __lock_buffer+0x2e/0x30 > [318603.207304] [] do_get_write_access+0x4ab/0x530 [jb= d2] > [318603.207408] [] ? __getblk+0x27/0x280 > [318603.207511] [] ? start_this_handle+0x503/0x570 [jb= d2] > [318603.207620] [] ? start_this_handle+0x4b1/0x570 [jb= d2] > [318603.207731] [] jbd2_journal_get_write_access+0x2b/= 0x50 [jbd2] > [318603.207874] [] __ext4_journal_get_write_access+0x3= 9/0x80 [ext4] > [318603.208016] [] ext4_reserve_inode_write+0x78/0xa0 = [ext4] > [318603.208131] [] ext4_mark_inode_dirty+0x4e/0x210 [e= xt4] > [318603.208259] [] ? ext4_dirty_inode+0x37/0x60 [ext4] > [318603.208376] [] ext4_dirty_inode+0x37/0x60 [ext4] > [318603.208487] [] __mark_inode_dirty+0x37/0x240 > [318603.208593] [] file_update_time+0xcf/0x130 > [318603.208698] [] __generic_file_aio_write+0x1f8/0x440 > [318603.208810] [] generic_file_aio_write+0x6f/0xe0 > [318603.208916] [] ? task_rq_lock+0x56/0xa0 > [318603.209026] [] ext4_file_write+0xbf/0x270 [ext4] > [318603.209135] [] ? fsnotify+0x82/0x2e0 > [318603.209238] [] do_sync_write+0xca/0x110 > [318603.209342] [] ? security_file_permission+0x27/0xb0 > [318603.209451] [] ? rw_verify_area+0x5c/0xf0 > [318603.209559] [] vfs_write+0xae/0x180 > [318603.209663] [] sys_write+0x45/0x90 > [318603.209765] [] cstar_dispatch+0x7/0x33 > [318603.209867] INFO: lockdep is turned off. > [318603.209964] INFO: task map:26075 blocked for more than 120 seconds. > [318603.210069] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disabl= es this message. > [318603.210203] map D 0000000105b0291b 0 26075 26072 0x0= 0020000 > [318603.210424] ffff880c02f0d9d8 0000000000000046 0000000000000000 ffff8= 81004673908 > [318603.210782] ffff880c03739ea0 ffff880c02f0dfd8 ffff880c02f0dfd8 ffff8= 80c02f0dfd8 > [318603.211140] ffff880405d0eb30 ffff880c03739ea0 ffff880c02f0d9e8 00000= 00104673828 > [318603.211510] Call Trace: > [318603.211606] [] schedule+0x3a/0x50 > [318603.211713] [] start_this_handle+0x2b5/0x570 [jbd2] > [318603.211822] [] ? static_obj+0x5a/0x70 > [318603.211926] [] ? __init_waitqueue_head+0x60/0x60 > [318603.212034] [] jbd2__journal_start+0x122/0x1a0 [jb= d2] > [318603.212144] [] jbd2_journal_start+0xe/0x10 [jbd2] > [318603.212259] [] ext4_journal_start_sb+0x7e/0x1d0 [e= xt4] > [318603.212374] [] ? ext4_dirty_inode+0x21/0x60 [ext4] > [318603.212482] [] ? __mem_cgroup_try_charge+0x2b8/0x5= d0 > [318603.212596] [] ext4_dirty_inode+0x21/0x60 [ext4] > [318603.212702] [] __mark_inode_dirty+0x37/0x240 > [318603.212807] [] file_update_time+0xcf/0x130 > [318603.212911] [] __generic_file_aio_write+0x1f8/0x440 > [318603.213018] [] generic_file_aio_write+0x6f/0xe0 > [318603.213123] [] ? task_rq_lock+0x56/0xa0 > [318603.213232] [] ext4_file_write+0xbf/0x270 [ext4] > [318603.213337] [] ? task_rq_lock+0x56/0xa0 > [318603.213440] [] do_sync_write+0xca/0x110 > [318603.213542] [] ? might_fault+0x40/0x90 > [318603.213645] [] ? security_file_permission+0x27/0xb0 > [318603.213752] [] ? rw_verify_area+0x5c/0xf0 > [318603.213855] [] vfs_write+0xae/0x180 > [318603.213958] [] sys_write+0x45/0x90 > [318603.214063] [] cstar_dispatch+0x7/0x33 > [318603.214164] INFO: lockdep is turned off. > [318603.214262] INFO: task map:26081 blocked for more than 120 seconds. > [318603.214367] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disabl= es this message. > [318603.214507] map D 0000000105b043b2 0 26081 26078 0x0= 0020000 > [318603.214731] ffff880c043af9d8 0000000000000046 0000000000000000 ffff8= 81004673908 > [318603.215112] ffff880c051bcc90 ffff880c043affd8 ffff880c043affd8 ffff8= 80c043affd8 > [318603.215481] ffff880405cbcc90 ffff880c051bcc90 ffff880c043af9e8 00000= 00104673828 > [318603.215845] Call Trace: > [318603.215937] [] schedule+0x3a/0x50 > [318603.216041] [] start_this_handle+0x2b5/0x570 [jbd2] > [318603.216155] [] ? static_obj+0x5a/0x70 > [318603.216258] [] ? __init_waitqueue_head+0x60/0x60 > [318603.216365] [] jbd2__journal_start+0x122/0x1a0 [jb= d2] > [318603.216475] [] jbd2_journal_start+0xe/0x10 [jbd2] > [318603.216590] [] ext4_journal_start_sb+0x7e/0x1d0 [e= xt4] > [318603.216703] [] ? ext4_dirty_inode+0x21/0x60 [ext4] > [318603.216808] [] ? __mem_cgroup_try_charge+0x2b8/0x5= d0 > [318603.216924] [] ext4_dirty_inode+0x21/0x60 [ext4] > [318603.217030] [] __mark_inode_dirty+0x37/0x240 > [318603.217135] [] file_update_time+0xcf/0x130 > [318603.217239] [] __generic_file_aio_write+0x1f8/0x440 > [318603.217346] [] generic_file_aio_write+0x6f/0xe0 > [318603.217452] [] ? task_rq_lock+0x56/0xa0 > [318603.217562] [] ext4_file_write+0xbf/0x270 [ext4] > [318603.217670] [] ? task_rq_lock+0x56/0xa0 > [318603.217773] [] do_sync_write+0xca/0x110 > [318603.217878] [] ? might_fault+0x40/0x90 > [318603.217981] [] ? security_file_permission+0x27/0xb0 > [318603.218088] [] ? rw_verify_area+0x5c/0xf0 > [318603.218207] [] vfs_write+0xae/0x180 > [318603.218311] [] sys_write+0x45/0x90 > [318603.218414] [] cstar_dispatch+0x7/0x33 > [318603.218515] INFO: lockdep is turned off. > [324567.341610] md: md2: data-check done. >=20 > $ cat /proc/mdstat > Personalities : [raid1] [raid0] [raid10] > md2 : active raid10 sda4[0] sdd4[3] sdc4[2] sdb4[1] > 849514496 blocks super 1.2 512K chunks 2 far-copies [4/4] [UUUU] >=20 > md1 : active raid0 sda3[0] sdd3[3] sdc3[2] sdb3[1] > 2047993856 blocks super 1.2 512k chunks >=20 > md0 : active raid1 sda1[0] sdd1[3] sdc1[2] sdb1[1] > 8000256 blocks [4/4] [UUUU] --Sig_/IuPO4JhdR2XfdJRGZOfDJ=e Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) iQIVAwUBT3kSXjnsnt1WYoG5AQL0oA//e/3zqTTqTztJ1j+QDkX5deZB+bgH0aAl 7QNM4U2k0hcn70lxlGVywV87SdOm2LGCZQV8edPqeQOxLzsC7qQZO9JInw6ddeBr iSfnB/uJeXhjc9/k8Qx9UkRC3C6GvhJb20luexHA19Q+uvt88N565iQR6NvVdwK7 qMiwhiN9mVPLg3aksvTIYr4Gl3CWTYI/vUMOknGmrj1qDBBi2itXfSpAzgIpuTlm SvCbAaZRKvWWAU5bFxGmVlRkOfrOvl14A2x0ys5qOB3Lj5ixAFy/pPGYGt1ZvAyH 4KpR7fLqYfSR2bqiYY1p+3wQJyRHzr1r1aGZap7FIO+jcn/SbJ3iA8GDiMuVnY/+ yO+kPUNfoh5jSEBPU2ztQQsUUVl1Lyrs5cX8MTHT9M3Eisj9aHwBNQ10xRWzQzL5 akcWyUzK7g76XQ/XWfwpSlRD8nTFESVX+guSQjEeNImiEghxzmGHiuQRNOv4dqmq ssabvNPl/dbFD1EtJrPdmuB/lpPoKp8/k/AjxKldFFt2VMzOgmh8z+y0eyFhBfJ7 3dWLJTohvJ2KFP13CCTGuhra8tWdJ8swPYON1m+3W8F4jQ/5/QB4/CXplRSS8luD nCWWrZcHCmejnGqui5JHWY2poUJTLLGuMpP9Yqv0V2f7ni6TGcMJUI7DY0zb7dJG SQysnB0G6h0= =V2Yc -----END PGP SIGNATURE----- --Sig_/IuPO4JhdR2XfdJRGZOfDJ=e--