From: Sev Binello <sev@bnl.gov>
To: linux-kernel@vger.kernel.org
Subject: Many processes stuck in blk_congestion_wait
Date: Wed, 23 Jan 2008 17:19:14 -0500 [thread overview]
Message-ID: <4797BD62.1070608@bnl.gov> (raw)
We are experiencing numerous hangs/slow downs on our WS4 U5 nfs client
machines (not seen on WS3).
Many processes end up in a D state for long periods of time
and are in blk_congestion_wait().
Time in this state can vary,
not sure what triggers them to come out, some don't
But time in this state can be quite lengthy.
Machines are running 2.6.9-55.0.9.ELsmp
Below is a sample trace, they all look pretty much the same....
Any clues how to debug this ?
Thanks
-Sev
--
Sev Binello
Brookhaven National Laboratory
Upton, New York
631-344-5647
sev@bnl.gov
Jan 23 12:39:27 acnmcr4p kernel: RampEditor D C02D6EB4 1288
21855 1 21857 22139 (NOTLB)
Jan 23 12:39:27 acnmcr4p kernel: f6e5ec94 00200086 65cc8b98 c02d6eb4
f56a9430 e3923830 00200200 00000000 Jan 23 12:39:27 acnmcr4p kernel:
c35c4860 00000000 c35bc780 c35bbde0 00000000 00000000 0d1d4140 001553b9
Jan 23 12:39:27 acnmcr4p kernel:e3923830 e8fd93f0 e8fd955c 00000000
00200246 65cef381 65cef381 f6e5ed04 Jan 23 12:39:27 acnmcr4p kernel:
Call Trace:
Jan 23 12:39:27 acnmcr4p kernel: [<c02d6eb4>] common_interrupt+0x18/0x20
Jan 23 12:39:27 acnmcr4p kernel: [<c02d4c5a>] schedule_timeout+0x139/0x154
Jan 23 12:39:27 acnmcr4p kernel: [<c012a73a>] process_timeout+0x0/0x5
Jan 23 12:39:27 acnmcr4p kernel: [<c02d4b17>]
io_schedule_timeout+0x26/0x30
Jan 23 12:39:27 acnmcr4p kernel: [<c02267f8>]
blk_congestion_wait+0x64/0x78
Jan 23 12:39:27 acnmcr4p kernel: [<c012052d>]
autoremove_wake_function+0x0/0x2d
Jan 23 12:39:27 acnmcr4p kernel: [<c0145038>]
get_writeback_state+0x30/0x35
Jan 23 12:39:27 acnmcr4p kernel: [<c012052d>]
autoremove_wake_function+0x0/0x2d
Jan 23 12:39:27 acnmcr4p kernel: [<c0145291>]
balance_dirty_pages+0xbe/0x11c
Jan 23 12:39:27 acnmcr4p kernel: [<f8f86105>]
nfs_commit_write+0x43/0x72 [nfs]
Jan 23 12:39:27 acnmcr4p kernel: [<c0142517>]
generic_file_buffered_write+0x41f/0x501
Jan 23 12:39:27 acnmcr4p kernel: [<c01ac299>] inode_has_perm+0x4c/0x54
Jan 23 12:39:27 acnmcr4p kernel: [<c011dd34>] move_tasks+0x19d/0x202
Jan 23 12:39:27 acnmcr4p kernel: [<c0142982>]
__generic_file_aio_write_nolock+0x389/0x3b7
Jan 23 12:39:27 acnmcr4p kernel: [<c01429e9>]
generic_file_aio_write_nolock+0x39/0x7f
Jan 23 12:39:27 acnmcr4p kernel: [<c0142bd3>]
generic_file_aio_write+0x72/0xc6
Jan 23 12:39:27 acnmcr4p kernel: [<f8f86212>] nfs_file_write+0xde/0xf9
[nfs]
Jan 23 12:39:27 acnmcr4p kernel: [<c015b5b8>] do_sync_write+0x9e/0xcb
Jan 23 12:39:27 acnmcr4p kernel: [<c016be58>] poll_freewait+0x33/0x38
Jan 23 12:39:27 acnmcr4p kernel: [<c01adf3a>]
selinux_file_permission+0x117/0x120
Jan 23 12:39:27 acnmcr4p kernel: [<c012052d>]
autoremove_wake_function+0x0/0x2d
Jan 23 12:39:27 acnmcr4p kernel: [<c015b69b>] vfs_write+0xb6/0xe2
Jan 23 12:39:27 acnmcr4p kernel: [<c015b765>] sys_write+0x3c/0x62
Jan 23 12:39:27 acnmcr4p kernel: [<c02d64db>] syscall_call+0x7/0xb
Jan 23 12:39:27 acnmcr4p kernel: [<c02d007b>]
unix_dgram_sendmsg+0x23c/0x45d
next reply other threads:[~2008-01-23 22:31 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-01-23 22:19 Sev Binello [this message]
[not found] ` <E1JJKTe-0001Ha-Mr@localhost.localdomain>
2008-01-28 3:16 ` Many processes stuck in blk_congestion_wait Fengguang Wu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4797BD62.1070608@bnl.gov \
--to=sev@bnl.gov \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox