From: Jim Rees <rees@umich.edu>
To: Peng Tao <bergwolf@gmail.com>
Cc: linux-nfs@vger.kernel.org, peter honeyman <honey@citi.umich.edu>
Subject: Re: client hang in lock_flocks()
Date: Sun, 3 Apr 2011 11:12:29 -0400 [thread overview]
Message-ID: <20110403151229.GA13399@merit.edu> (raw)
In-Reply-To: <BANLkTinFftt0OiLYmY3cipXTtx1uTYoJPg@mail.gmail.com>
Peng Tao wrote:
Would you please send the complete trace log, or maybe open a bugzilla
entry and attach it there?
Here is everything that was reported to me. I can ask for rpc info and a
tcpdump if that would be useful.
Note that the second set of traces seems to be a different problem but I've
included it in case they are related.
Mar 24 17:21:08 r38-13-78 kernel: INFO: task iozone:2657 blocked for more than 120 seconds.
Mar 24 17:21:08 r38-13-78 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 24 17:21:08 r38-13-78 kernel: iozone D ffff88024ac55f70 0 2657 1 0x00000080
Mar 24 17:21:08 r38-13-78 kernel: ffff88024cc65c50 0000000000000082 0000000000000000 0000000000000000
Mar 24 17:21:08 r38-13-78 kernel: ffff88024cc64010 ffff88024cc65fd8 00000000000136c0 ffff88024ac55bc0
Mar 24 17:21:08 r38-13-78 kernel: ffff88024ac55f78 ffff88024ac55f70 00000000000136c0 00000000000136c0
Mar 24 17:21:08 r38-13-78 kernel: Call Trace:
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffff81139c1d>] ? lock_flocks+0x10/0x12
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffff8143e287>] rwsem_down_failed_common+0xd3/0x105
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffff8143e2e0>] rwsem_down_read_failed+0x12/0x14
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffff8120a5b4>] call_rwsem_down_read_failed+0x14/0x30
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffff8143d9c7>] ? down_read+0x32/0x36
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffffa03ddca6>] nfs4_proc_lock+0x224/0x342 [nfs]
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffffa03c39cb>] do_setlk+0x61/0xd5 [nfs]
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffffa03c3c66>] nfs_lock+0x170/0x17f [nfs]
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffff8113b4dc>] vfs_lock_file+0x24/0x38
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffff8113b823>] fcntl_setlk+0x15e/0x2a6
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffff811058bb>] ? fsnotify_access+0x5d/0x65
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffff81112dc6>] sys_fcntl+0x30e/0x4af
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffff8143eed0>] ? do_device_not_available+0x9/0xb
Mar 24 17:21:08 r38-13-78 kernel: [<ffffffff8100aa52>] system_call_fastpath+0x16/0x1b
Mar 24 17:23:08 r38-13-78 kernel: INFO: task iozone:2657 blocked for more than 120 seconds.
Mar 24 17:23:08 r38-13-78 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 24 17:23:08 r38-13-78 kernel: iozone D ffff88024ac55f70 0 2657 1 0x00000080
Mar 24 17:23:08 r38-13-78 kernel: ffff88024cc65c50 0000000000000082 0000000000000000 0000000000000000
Mar 24 17:23:08 r38-13-78 kernel: ffff88024cc64010 ffff88024cc65fd8 00000000000136c0 ffff88024ac55bc0
Mar 24 17:23:08 r38-13-78 kernel: ffff88024ac55f78 ffff88024ac55f70 00000000000136c0 00000000000136c0
Mar 24 17:23:08 r38-13-78 kernel: Call Trace:
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffff81139c1d>] ? lock_flocks+0x10/0x12
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffff8143e287>] rwsem_down_failed_common+0xd3/0x105
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffff8143e2e0>] rwsem_down_read_failed+0x12/0x14
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffff8120a5b4>] call_rwsem_down_read_failed+0x14/0x30
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffff8143d9c7>] ? down_read+0x32/0x36
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffffa03ddca6>] nfs4_proc_lock+0x224/0x342 [nfs]
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffffa03c39cb>] do_setlk+0x61/0xd5 [nfs]
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffffa03c3c66>] nfs_lock+0x170/0x17f [nfs]
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffff8113b4dc>] vfs_lock_file+0x24/0x38
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffff8113b823>] fcntl_setlk+0x15e/0x2a6
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffff811058bb>] ? fsnotify_access+0x5d/0x65
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffff81112dc6>] sys_fcntl+0x30e/0x4af
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffff8143eed0>] ? do_device_not_available+0x9/0xb
Mar 24 17:23:08 r38-13-78 kernel: [<ffffffff8100aa52>] system_call_fastpath+0x16/0x1b
Mar 24 17:25:08 r38-13-78 kernel: INFO: task iozone:2657 blocked for more than 120 seconds.
Mar 24 17:25:08 r38-13-78 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 24 17:25:08 r38-13-78 kernel: iozone D ffff88024ac55f70 0 2657 1 0x00000080
Mar 24 17:25:08 r38-13-78 kernel: ffff88024cc65c50 0000000000000082 0000000000000000 0000000000000000
Mar 24 17:25:08 r38-13-78 kernel: ffff88024cc64010 ffff88024cc65fd8 00000000000136c0 ffff88024ac55bc0
Mar 24 17:25:08 r38-13-78 kernel: ffff88024ac55f78 ffff88024ac55f70 00000000000136c0 00000000000136c0
Mar 24 17:25:08 r38-13-78 kernel: Call Trace:
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffff81139c1d>] ? lock_flocks+0x10/0x12
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffff8143e287>] rwsem_down_failed_common+0xd3/0x105
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffff8143e2e0>] rwsem_down_read_failed+0x12/0x14
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffff8120a5b4>] call_rwsem_down_read_failed+0x14/0x30
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffff8143d9c7>] ? down_read+0x32/0x36
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffffa03ddca6>] nfs4_proc_lock+0x224/0x342 [nfs]
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffffa03c39cb>] do_setlk+0x61/0xd5 [nfs]
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffffa03c3c66>] nfs_lock+0x170/0x17f [nfs]
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffff8113b4dc>] vfs_lock_file+0x24/0x38
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffff8113b823>] fcntl_setlk+0x15e/0x2a6
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffff811058bb>] ? fsnotify_access+0x5d/0x65
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffff81112dc6>] sys_fcntl+0x30e/0x4af
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffff8143eed0>] ? do_device_not_available+0x9/0xb
Mar 24 17:25:08 r38-13-78 kernel: [<ffffffff8100aa52>] system_call_fastpath+0x16/0x1b
**************************
Mar 24 17:48:18 r38-13-78 kernel: INFO: task iozone:2718 blocked for more than 120 seconds.
Mar 24 17:48:18 r38-13-78 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 24 17:48:18 r38-13-78 kernel: iozone D ffff88024b72b190 0 2718 1 0x00000080
Mar 24 17:48:18 r38-13-78 kernel: ffff88024b8cfb08 0000000000000082 ffff88024b8cfa88 ffffffffa03f4d00
Mar 24 17:48:18 r38-13-78 kernel: ffff88024b8ce010 ffff88024b8cffd8 00000000000136c0 ffff88024b72ade0
Mar 24 17:48:18 r38-13-78 kernel: ffff88024b72b198 ffff88024b72b190 00000000000136c0 00000000000136c0
Mar 24 17:48:18 r38-13-78 kernel: Call Trace:
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff810c0fab>] ? sync_page+0x0/0x48
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff8143cd96>] io_schedule+0x6b/0xae
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff810c0fef>] sync_page+0x44/0x48
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff8143d2e3>] __wait_on_bit+0x43/0x76
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff810c114b>] wait_on_page_bit+0x6d/0x74
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff81067d3b>] ? wake_bit_function+0x0/0x2e
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff810c95f0>] ? pagevec_lookup_tag+0x20/0x29
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff810c145d>] filemap_fdatawait_range+0x9f/0x179
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff810c15e3>] filemap_write_and_wait_range+0x3e/0x51
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff81126f65>] vfs_fsync_range+0x30/0x75
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff81126fe6>] generic_write_sync+0x3c/0x3e
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff810c1a53>] generic_file_aio_write+0x83/0xa1
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffffa03c6ef4>] nfs_file_write+0xd4/0x169 [nfs]
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff81105c26>] do_sync_write+0xc6/0x103
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff81067d02>] ? wake_up_bit+0x20/0x25
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff810f88cd>] ? slab_pre_alloc_hook+0x28/0x2c
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff811c800a>] ? security_file_permission+0x29/0x2e
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff811065b6>] vfs_write+0xa9/0x105
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff811066cb>] sys_write+0x45/0x6c
Mar 24 17:48:18 r38-13-78 kernel: [<ffffffff8100aa52>] system_call_fastpath+0x16/0x1b
Mar 24 17:49:44 r38-13-78 ntpd[1161]: 0.0.0.0 c612 02 freq_set kernel 37.872 PPM
Mar 24 17:49:44 r38-13-78 ntpd[1161]: 0.0.0.0 c615 05 clock_sync
Mar 24 17:50:18 r38-13-78 kernel: INFO: task iozone:2718 blocked for more than 120 seconds.
Mar 24 17:50:18 r38-13-78 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 24 17:50:18 r38-13-78 kernel: iozone D ffff88024b72b190 0 2718 1 0x00000080
Mar 24 17:50:18 r38-13-78 kernel: ffff88024b8cfb08 0000000000000082 ffff88024b8cfa88 ffffffffa03f4d00
Mar 24 17:50:18 r38-13-78 kernel: ffff88024b8ce010 ffff88024b8cffd8 00000000000136c0 ffff88024b72ade0
Mar 24 17:50:18 r38-13-78 kernel: ffff88024b72b198 ffff88024b72b190 00000000000136c0 00000000000136c0
Mar 24 17:50:18 r38-13-78 kernel: Call Trace:
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff810c0fab>] ? sync_page+0x0/0x48
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff8143cd96>] io_schedule+0x6b/0xae
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff810c0fef>] sync_page+0x44/0x48
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff8143d2e3>] __wait_on_bit+0x43/0x76
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff810c114b>] wait_on_page_bit+0x6d/0x74
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff81067d3b>] ? wake_bit_function+0x0/0x2e
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff810c95f0>] ? pagevec_lookup_tag+0x20/0x29
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff810c145d>] filemap_fdatawait_range+0x9f/0x179
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff810c15e3>] filemap_write_and_wait_range+0x3e/0x51
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff81126f65>] vfs_fsync_range+0x30/0x75
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff81126fe6>] generic_write_sync+0x3c/0x3e
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff810c1a53>] generic_file_aio_write+0x83/0xa1
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffffa03c6ef4>] nfs_file_write+0xd4/0x169 [nfs]
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff81105c26>] do_sync_write+0xc6/0x103
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff81067d02>] ? wake_up_bit+0x20/0x25
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff810f88cd>] ? slab_pre_alloc_hook+0x28/0x2c
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff811c800a>] ? security_file_permission+0x29/0x2e
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff811065b6>] vfs_write+0xa9/0x105
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff811066cb>] sys_write+0x45/0x6c
Mar 24 17:50:18 r38-13-78 kernel: [<ffffffff8100aa52>] system_call_fastpath+0x16/0x1b
Mar 24 17:52:18 r38-13-78 kernel: INFO: task iozone:2718 blocked for more than 120 seconds.
Mar 24 17:52:18 r38-13-78 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 24 17:52:18 r38-13-78 kernel: iozone D ffff88024b72b190 0 2718 1 0x00000080
Mar 24 17:52:18 r38-13-78 kernel: ffff88024b8cfb08 0000000000000082 ffff88024b8cfa88 ffffffffa03f4d00
Mar 24 17:52:18 r38-13-78 kernel: ffff88024b8ce010 ffff88024b8cffd8 00000000000136c0 ffff88024b72ade0
Mar 24 17:52:18 r38-13-78 kernel: ffff88024b72b198 ffff88024b72b190 00000000000136c0 00000000000136c0
Mar 24 17:52:18 r38-13-78 kernel: Call Trace:
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff810c0fab>] ? sync_page+0x0/0x48
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff8143cd96>] io_schedule+0x6b/0xae
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff810c0fef>] sync_page+0x44/0x48
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff8143d2e3>] __wait_on_bit+0x43/0x76
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff810c114b>] wait_on_page_bit+0x6d/0x74
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff81067d3b>] ? wake_bit_function+0x0/0x2e
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff810c95f0>] ? pagevec_lookup_tag+0x20/0x29
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff810c145d>] filemap_fdatawait_range+0x9f/0x179
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff810c15e3>] filemap_write_and_wait_range+0x3e/0x51
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff81126f65>] vfs_fsync_range+0x30/0x75
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff81126fe6>] generic_write_sync+0x3c/0x3e
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff810c1a53>] generic_file_aio_write+0x83/0xa1
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffffa03c6ef4>] nfs_file_write+0xd4/0x169 [nfs]
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff81105c26>] do_sync_write+0xc6/0x103
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff81067d02>] ? wake_up_bit+0x20/0x25
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff810f88cd>] ? slab_pre_alloc_hook+0x28/0x2c
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff811c800a>] ? security_file_permission+0x29/0x2e
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff811065b6>] vfs_write+0xa9/0x105
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff811066cb>] sys_write+0x45/0x6c
Mar 24 17:52:18 r38-13-78 kernel: [<ffffffff8100aa52>] system_call_fastpath+0x16/0x1b
**************************
Mar 30 23:47:07 r38-13-78 xinetd[1144]: START: shell pid=2604 from=::ffff:10.244.76.80
Mar 30 23:47:07 r38-13-78 rshd[2606]: root@r38-15-80 as root: cmd='/opt/iozone/bin/iozone -+s -t 1 -r 4 -s 4 -+c r38-15-80 '
Mar 30 23:47:07 r38-13-78 xinetd[1144]: EXIT: shell status=0 pid=2604 duration=0(sec)
Mar 30 23:47:12 r38-13-78 xinetd[1144]: START: shell pid=2625 from=::ffff:10.244.76.80
Mar 30 23:47:12 r38-13-78 rshd[2626]: root@r38-15-80 as root: cmd='/opt/iozone/bin/iozone -+s -t 1 -r 4 -s 4 -+c r38-15-80 '
Mar 30 23:47:12 r38-13-78 xinetd[1144]: EXIT: shell status=0 pid=2625 duration=0(sec)
Mar 30 23:47:49 r38-13-78 xinetd[1144]: START: shell pid=2648 from=::ffff:10.244.76.80
Mar 30 23:47:49 r38-13-78 rshd[2649]: root@r38-15-80 as root: cmd='/opt/iozone/bin/iozone -+s -t 1 -r 4 -s 4 -+c r38-15-80 '
Mar 30 23:47:49 r38-13-78 xinetd[1144]: EXIT: shell status=0 pid=2648 duration=0(sec)
Mar 30 23:47:54 r38-13-78 xinetd[1144]: START: shell pid=2669 from=::ffff:10.244.76.80
Mar 30 23:47:54 r38-13-78 rshd[2670]: root@r38-15-80 as root: cmd='/opt/iozone/bin/iozone -+s -t 1 -r 4 -s 4 -+c r38-15-80 '
Mar 30 23:47:54 r38-13-78 xinetd[1144]: EXIT: shell status=0 pid=2669 duration=0(sec)
Mar 30 23:51:26 r38-13-78 kernel: INFO: task iozone:2666 blocked for more than 120 seconds.
Mar 30 23:51:26 r38-13-78 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 30 23:51:26 r38-13-78 kernel: iozone D ffff88024cdf1aa0 0 2666 1 0x00000080
Mar 30 23:51:26 r38-13-78 kernel: ffff880246be7c50 0000000000000082 0000000000000000 0000000000000000
Mar 30 23:51:26 r38-13-78 kernel: ffff880246be6010 ffff880246be7fd8 00000000000136c0 ffff88024cdf16f0
Mar 30 23:51:26 r38-13-78 kernel: ffff88024cdf1aa8 ffff88024cdf1aa0 00000000000136c0 00000000000136c0
Mar 30 23:51:26 r38-13-78 kernel: Call Trace:
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffff8113978a>] ? lock_flocks+0x10/0x12
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffff8143dc37>] rwsem_down_failed_common+0xd3/0x105
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffff8143dc90>] rwsem_down_read_failed+0x12/0x14
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffff81209f54>] call_rwsem_down_read_failed+0x14/0x30
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffff8143d377>] ? down_read+0x32/0x36
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffffa03dbd20>] nfs4_proc_lock+0x224/0x342 [nfs]
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffffa03c19af>] do_setlk+0x61/0xd5 [nfs]
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffffa03c1c4a>] nfs_lock+0x170/0x17f [nfs]
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffff8113af72>] vfs_lock_file+0x24/0x38
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffff8113b2b9>] fcntl_setlk+0x15e/0x2a6
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffff811056d7>] ? fsnotify_access+0x5d/0x65
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffff81112ad6>] sys_fcntl+0x30e/0x4af
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffff8143e890>] ? do_device_not_available+0x9/0xb
Mar 30 23:51:26 r38-13-78 kernel: [<ffffffff8100aa52>] system_call_fastpath+0x16/0x1b
Mar 30 23:52:54 r38-13-78 ntpd[1153]: 0.0.0.0 c612 02 freq_set kernel 37.606 PPM
Mar 30 23:52:54 r38-13-78 ntpd[1153]: 0.0.0.0 c615 05 clock_sync
Mar 30 23:53:26 r38-13-78 kernel: INFO: task iozone:2666 blocked for more than 120 seconds.
Mar 30 23:53:26 r38-13-78 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 30 23:53:26 r38-13-78 kernel: iozone D ffff88024cdf1aa0 0 2666 1 0x00000080
Mar 30 23:53:26 r38-13-78 kernel: ffff880246be7c50 0000000000000082 0000000000000000 0000000000000000
Mar 30 23:53:26 r38-13-78 kernel: ffff880246be6010 ffff880246be7fd8 00000000000136c0 ffff88024cdf16f0
Mar 30 23:53:26 r38-13-78 kernel: ffff88024cdf1aa8 ffff88024cdf1aa0 00000000000136c0 00000000000136c0
Mar 30 23:53:26 r38-13-78 kernel: Call Trace:
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffff8113978a>] ? lock_flocks+0x10/0x12
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffff8143dc37>] rwsem_down_failed_common+0xd3/0x105
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffff8143dc90>] rwsem_down_read_failed+0x12/0x14
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffff81209f54>] call_rwsem_down_read_failed+0x14/0x30
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffff8143d377>] ? down_read+0x32/0x36
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffffa03dbd20>] nfs4_proc_lock+0x224/0x342 [nfs]
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffffa03c19af>] do_setlk+0x61/0xd5 [nfs]
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffffa03c1c4a>] nfs_lock+0x170/0x17f [nfs]
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffff8113af72>] vfs_lock_file+0x24/0x38
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffff8113b2b9>] fcntl_setlk+0x15e/0x2a6
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffff811056d7>] ? fsnotify_access+0x5d/0x65
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffff81112ad6>] sys_fcntl+0x30e/0x4af
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffff8143e890>] ? do_device_not_available+0x9/0xb
Mar 30 23:53:26 r38-13-78 kernel: [<ffffffff8100aa52>] system_call_fastpath+0x16/0x1b
Mar 30 23:55:26 r38-13-78 kernel: INFO: task iozone:2666 blocked for more than 120 seconds.
next prev parent reply other threads:[~2011-04-03 15:12 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-03-31 16:52 client hang in lock_flocks() Jim Rees
[not found] ` <20110331165258.GA3048-8f4Pc2RrbJmHXe+LvDLADg@public.gmane.org>
2011-04-03 1:37 ` Peng Tao
2011-04-03 15:12 ` Jim Rees [this message]
2011-04-03 17:02 ` Peng Tao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110403151229.GA13399@merit.edu \
--to=rees@umich.edu \
--cc=bergwolf@gmail.com \
--cc=honey@citi.umich.edu \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.