From: maximilian attems <max-U9r9yeDMy7A@public.gmane.org>
To: Trond.Myklebust@netapp.com
Cc: Andrew Dixie <andrew-0FSrDVjkvKm9koe0gwxAeg@public.gmane.org>,
linux-nfs@vger.kernel.org
Subject: (fwd) nfs hang on 2.6.24
Date: Tue, 5 Feb 2008 10:01:32 +0100 [thread overview]
Message-ID: <20080205090132.GA8286@stro.at> (raw)
please take a look
----- Forwarded message from Andrew Dixie <andrew-0FSrDVjkvKm9koe0gwxAeg@public.gmane.org> -----
Date: Tue, 5 Feb 2008 12:57:37 +1300 (NZDT)
Subject: nfs hang on 2.6.24
From: Andrew Dixie <andrew-0FSrDVjkvKm9koe0gwxAeg@public.gmane.org>
To: debian-kernel@lists.debian.org
I've recently updated to 2.6.24 as recommended on bug 463508.
But the new kernel now has nfs hanging problems.
A few times now it has got into a situation such that any process that
touches an nfs mount hangs.
The machine has both nfs3 and nfs4 mounts. My suspicion is that it relates
to putting a heavy load on both nfs3 and nfs4 mounts at the same time.
The machine for the moment is still hung so let me know if there are
further debugging steps I can perform.
There are many other processes but these are the unique call stacks that
involve nfs related functions.
=======================
nfsv4-delegre S 00000282 0 17156 2
df8dcd30 00000046 f8c5f813 00000282 f35d4a60 00000000 93c19d62
000008a4
df8dce98 c180a940 00000000 f35d4b10 0000a296 00000000 f669be00
f8ce9b74
f8c5f08b 00000246 e6a59f34 00000000 e6a59f3c c18003b0 f8c62e85
c02bc97f
Call Trace:
[<f8c5f813>] xprt_timer+0x0/0x6f [sunrpc]
[<f8ce9b74>] nfs4_xdr_enc_delegreturn+0x0/0x84 [nfs]
[<f8c5f08b>] xprt_release_xprt+0x39/0x66 [sunrpc]
[<f8c62e85>] rpc_wait_bit_interruptible+0x1a/0x1f [sunrpc]
[<c02bc97f>] __wait_on_bit+0x33/0x58
[<f8c62e6b>] rpc_wait_bit_interruptible+0x0/0x1f [sunrpc]
[<f8c62e6b>] rpc_wait_bit_interruptible+0x0/0x1f [sunrpc]
[<c02bca07>] out_of_line_wait_on_bit+0x63/0x6b
[<c013545e>] wake_bit_function+0x0/0x3c
[<f8c62e19>] __rpc_wait_for_completion_task+0x32/0x39 [sunrpc]
[<f8ce1352>] nfs4_wait_for_completion_rpc_task+0x1b/0x2f [nfs]
[<f8ce2336>] nfs4_proc_delegreturn+0x116/0x172 [nfs]
[<f8ced370>] nfs_do_return_delegation+0xf/0x1d [nfs]
[<f8ced88a>] recall_thread+0xad/0xc0 [nfs]
[<f8ced7dd>] recall_thread+0x0/0xc0 [nfs]
[<c0104b0f>] kernel_thread_helper+0x7/0x10
=======================
mysqld S f704db38 0 3156 3110
df848130 00000082 00000002 f704db38 f704db30 00000000 6369898c
0000052b
df848298 c1822940 00000003 0163574b f704db5c f7c86000 000000ff
00000000
00000000 00000000 f704db5c 016359d3 f7459140 000001f4 c02bc84e
00000001
Call Trace:
[<c02bc84e>] schedule_timeout+0x70/0x8d
[<c01048c3>] common_interrupt+0x23/0x28
[<c012c3e2>] process_timeout+0x0/0x5
[<c02bc849>] schedule_timeout+0x6b/0x8d
[<c0183476>] do_select+0x365/0x3bc
[<c0183a60>] __pollwait+0x0/0xac
[<c011e069>] enqueue_entity+0x2b/0x3d
[<c0115343>] apic_wait_icr_idle+0xe/0x15
[<c011e091>] enqueue_task_fair+0x16/0x24
[<c011d647>] enqueue_task+0x52/0x5d
[<c011de8a>] resched_task+0x52/0x54
[<c011f445>] try_to_wake_up+0x2b8/0x2c2
[<c013543e>] autoremove_wake_function+0x15/0x35
[<c011d482>] __wake_up_common+0x32/0x5c
[<c011eeb8>] __wake_up+0x32/0x42
[<c013540e>] __wake_up_bit+0x2e/0x33
[<f8c630f5>] __rpc_do_wake_up_task+0x1fb/0x218 [sunrpc]
[<f8c6003a>] xdr_partial_copy_from_skb+0x32/0x171 [sunrpc]
[<f8c63148>] rpc_wake_up_task+0x36/0x4a [sunrpc]
[<f8c61cfd>] xs_tcp_data_recv+0x3d3/0x407 [sunrpc]
[<f8c6028f>] xdr_skb_read_bits+0x0/0x35 [sunrpc]
[<c0259bb5>] skb_release_all+0xa3/0xfa
[<c028484c>] tcp_read_sock+0x15e/0x16a
[<f8c6192a>] xs_tcp_data_recv+0x0/0x407 [sunrpc]
[<f8c624a8>] xs_tcp_data_ready+0x55/0x61 [sunrpc]
[<c012c8ec>] mod_timer+0x19/0x36
[<c02564e7>] sk_reset_timer+0xc/0x16
[<c028ba16>] tcp_rcv_established+0x3ae/0x63c
[<c0290ceb>] tcp_v4_do_rcv+0x2b/0x343
[<f8a593a5>] ip_vs_in+0xa5/0x220 [ip_vs]
[<f8a59300>] ip_vs_in+0x0/0x220 [ip_vs]
[<c0293000>] tcp_v4_rcv+0x80e/0x882
[<c0183750>] core_sys_select+0x283/0x2a0
[<c027a934>] ip_local_deliver_finish+0x114/0x1b7
[<c0259c6b>] __alloc_skb+0x49/0xf7
[<c0259bb5>] skb_release_all+0xa3/0xfa
[<f8833e62>] e1000_alloc_rx_buffers+0x1cb/0x295 [e1000]
[<c02594b1>] __kfree_skb+0x8/0x61
[<f88301f9>] e1000_unmap_and_free_tx_resource+0x1b/0x23 [e1000]
[<f8831c09>] e1000_clean_tx_irq+0xbe/0x2c3 [e1000]
[<f883480e>] e1000_clean_rx_irq+0x419/0x449 [e1000]
[<f88343f5>] e1000_clean_rx_irq+0x0/0x449 [e1000]
[<f8832045>] e1000_clean+0x1e9/0x213 [e1000]
[<c025fe81>] net_rx_action+0x9f/0x198
[<c0183bb0>] sys_select+0xa4/0x187
[<c0103e5e>] sysenter_past_esp+0x6b/0xa1
=======================
sshd S f74fdc80 0 17157 3065
df813830 00000086 00000002 f74fdc80 f74fdc78 00000000 f52ac680
f8c634bf
df813998 c1822940 00000003 015ba1ef 00000000 f7d6fcd0 000000ff
00000000
00000000 00000000 f74fdcc0 00000000 f74fdcc8 c1800ce0 f8c62e85
c02bc97f
Call Trace:
[<f8c634bf>] rpc_sleep_on+0x21/0x221 [sunrpc]
[<f8c62e85>] rpc_wait_bit_interruptible+0x1a/0x1f [sunrpc]
[<c02bc97f>] __wait_on_bit+0x33/0x58
[<f8c62e6b>] rpc_wait_bit_interruptible+0x0/0x1f [sunrpc]
[<f8c62e6b>] rpc_wait_bit_interruptible+0x0/0x1f [sunrpc]
[<c02bca07>] out_of_line_wait_on_bit+0x63/0x6b
[<c013545e>] wake_bit_function+0x0/0x3c
[<f8c62e19>] __rpc_wait_for_completion_task+0x32/0x39 [sunrpc]
[<f8ce1352>] nfs4_wait_for_completion_rpc_task+0x1b/0x2f [nfs]
[<f8ce3812>] _nfs4_proc_open+0x75/0x1b4 [nfs]
[<f8ce3e0c>] nfs4_do_open+0x112/0x23b [nfs]
[<f8ce50d3>] nfs4_open_revalidate+0x5e/0x11e [nfs]
[<f8cd25c1>] nfs_open_revalidate+0xc7/0x187 [nfs]
[<c017e8c4>] do_lookup+0x101/0x140
[<c01803f0>] __link_path_walk+0x744/0xb4b
[<c018083b>] link_path_walk+0x44/0xb3
[<c0176b59>] get_unused_fd_flags+0x4d/0xba
[<c0180b23>] do_path_lookup+0x162/0x1c4
[<c01793cc>] get_empty_filp+0x95/0x152
[<c0181488>] __path_lookup_intent_open+0x45/0x75
[<c0181527>] path_lookup_open+0x20/0x25
[<c0181606>] open_namei+0x72/0x558
[<c0176e00>] do_filp_open+0x25/0x39
[<c0176b59>] get_unused_fd_flags+0x4d/0xba
[<c0176e58>] do_sys_open+0x44/0xc0
[<c0176f0d>] sys_open+0x1c/0x1e
[<c0103e5e>] sysenter_past_esp+0x6b/0xa1
=======================
sshd S f52e4800 0 17199 3065
df8dd8f0 00000082 f8c63859 f52e4800 f669be00 c02bd82e 8660c63b
000008ce
df8dda58 c180a940 00000000 c01868b0 0000a690 00000000 c7387ce4
f7699b80
f7699b80 00000246 e7379cc0 00000000 e7379cc8 c1802c60 f8c62e85
c02bc97f
Call Trace:
[<f8c63859>] rpc_wake_up_next+0x12c/0x136 [sunrpc]
[<c02bd82e>] _spin_lock_bh+0x8/0x18
[<c01868b0>] dput+0x15/0xdc
[<f8c62e85>] rpc_wait_bit_interruptible+0x1a/0x1f [sunrpc]
[<c02bc97f>] __wait_on_bit+0x33/0x58
[<f8c62e6b>] rpc_wait_bit_interruptible+0x0/0x1f [sunrpc]
[<f8c62e6b>] rpc_wait_bit_interruptible+0x0/0x1f [sunrpc]
[<c02bca07>] out_of_line_wait_on_bit+0x63/0x6b
[<c013545e>] wake_bit_function+0x0/0x3c
[<f8c62e19>] __rpc_wait_for_completion_task+0x32/0x39 [sunrpc]
[<f8ce1352>] nfs4_wait_for_completion_rpc_task+0x1b/0x2f [nfs]
[<f8ce3812>] _nfs4_proc_open+0x75/0x1b4 [nfs]
[<f8ce3e0c>] nfs4_do_open+0x112/0x23b [nfs]
[<f8ce50d3>] nfs4_open_revalidate+0x5e/0x11e [nfs]
[<f8cd25c1>] nfs_open_revalidate+0xc7/0x187 [nfs]
[<c017e8c4>] do_lookup+0x101/0x140
[<c01803f0>] __link_path_walk+0x744/0xb4b
[<f8ce2021>] nfs4_proc_getattr+0x31/0x3e [nfs]
[<f8cd459b>] __nfs_revalidate_inode+0x269/0x276 [nfs]
[<c018083b>] link_path_walk+0x44/0xb3
[<c0176b59>] get_unused_fd_flags+0x4d/0xba
[<c0180b23>] do_path_lookup+0x162/0x1c4
[<c01793cc>] get_empty_filp+0x95/0x152
[<c0181488>] __path_lookup_intent_open+0x45/0x75
[<c0181527>] path_lookup_open+0x20/0x25
[<c0181606>] open_namei+0x72/0x558
[<c0176e00>] do_filp_open+0x25/0x39
[<c0176b59>] get_unused_fd_flags+0x4d/0xba
[<c0176e58>] do_sys_open+0x44/0xc0
[<c0176f0d>] sys_open+0x1c/0x1e
[<c0103e5e>] sysenter_past_esp+0x6b/0xa1
=======================
sshd S f5203bf4 0 17252 3065
f7cd2db0 00000086 00000002 f5203bf4 f5203bec 00000000 ffffffff
f76742c0
f7cd2f18 c1822940 00000003 015e25d4 015e2abf c017554d 000000ff
00000000
00000000 00000000 f5203c34 00000000 f5203c3c c1802f30 f8c62e85
c02bc97f
Call Trace:
[<c017554d>] cache_alloc_refill+0x58/0x477
[<f8c62e85>] rpc_wait_bit_interruptible+0x1a/0x1f [sunrpc]
[<c02bc97f>] __wait_on_bit+0x33/0x58
[<f8c62e6b>] rpc_wait_bit_interruptible+0x0/0x1f [sunrpc]
[<f8c62e6b>] rpc_wait_bit_interruptible+0x0/0x1f [sunrpc]
[<c02bca07>] out_of_line_wait_on_bit+0x63/0x6b
[<c013545e>] wake_bit_function+0x0/0x3c
[<f8c632cb>] __rpc_execute+0xeb/0x231 [sunrpc]
[<f8c62ac7>] rpc_set_active+0x3a/0x56 [sunrpc]
[<f8c5dc5e>] rpc_do_run_task+0x76/0x8f [sunrpc]
[<f8c5dcf8>] rpc_call_sync+0x21/0x39 [sunrpc]
[<f8ce2501>] nfs4_proc_access+0x129/0x19e [nfs]
[<c0176b59>] get_unused_fd_flags+0x4d/0xba
[<c0176a8e>] fd_install+0x1b/0x41
[<c025c25c>] scm_detach_fds+0xf4/0x12c
[<c02b0fe5>] unix_stream_recvmsg+0x48d/0x4bd
[<c0259c6b>] __alloc_skb+0x49/0xf7
[<f8cd0d83>] nfs_do_access+0x133/0x2ab [nfs]
[<f8c63fb0>] rpcauth_lookupcred+0x65/0x8a [sunrpc]
[<f8cd0fc0>] nfs_permission+0xc5/0x134 [nfs]
[<c01868b0>] dput+0x15/0xdc
[<c017e6c1>] __follow_mount+0x1e/0x60
[<f8cd0efb>] nfs_permission+0x0/0x134 [nfs]
[<c017e5bc>] permission+0xa3/0xef
[<c017fdc6>] __link_path_walk+0x11a/0xb4b
[<c015a9cd>] find_lock_page+0x19/0x7f
[<c018083b>] link_path_walk+0x44/0xb3
[<c0259157>] skb_dequeue+0x39/0x3f
[<c0180b23>] do_path_lookup+0x162/0x1c4
[<c017fae8>] getname+0x59/0xad
[<c01812f4>] __user_walk_fd+0x2f/0x40
[<c017b33f>] vfs_stat_fd+0x19/0x40
[<c0259157>] skb_dequeue+0x39/0x3f
[<c017b41b>] sys_stat64+0xf/0x23
[<c012fe3f>] set_current_groups+0x14d/0x159
[<c01bd9cf>] security_capable+0x9/0xa
[<c012aff8>] __capable+0x8/0x1b
[<c0131894>] sys_setresuid+0x163/0x18e
[<c01bdc8f>] security_task_post_setuid+0x17/0x1a
[<c0103e5e>] sysenter_past_esp+0x6b/0xa1
--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
----- End forwarded message -----
next reply other threads:[~2008-02-05 9:01 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-02-05 9:01 maximilian attems [this message]
[not found] ` <20080205090132.GA8286-U9r9yeDMy7A@public.gmane.org>
2008-02-05 22:02 ` (fwd) nfs hang on 2.6.24 Trond Myklebust
2008-02-06 6:24 ` Andrew Dixie
2008-02-06 15:00 ` Trond Myklebust
[not found] ` <1202310021.12647.6.camel-rJ7iovZKK19ZJLDQqaL3InhyD016LWXt@public.gmane.org>
2008-02-06 15:07 ` J. Bruce Fields
2008-02-06 15:15 ` Trond Myklebust
[not found] ` <1202310924.12647.24.camel-rJ7iovZKK19ZJLDQqaL3InhyD016LWXt@public.gmane.org>
2008-02-06 17:23 ` J. Bruce Fields
2008-02-06 17:52 ` Trond Myklebust
[not found] ` <1202320337.14889.18.camel-rJ7iovZKK19ZJLDQqaL3InhyD016LWXt@public.gmane.org>
2008-02-06 18:31 ` J. Bruce Fields
2008-02-06 21:19 ` Andrew Dixie
[not found] ` <37673.203.167.214.129.1202332746.squirrel-pmwrj2wvkLORZ0GbGNPwb6VXKuFTiq87@public.gmane.org>
2008-02-06 21:45 ` J. Bruce Fields
2008-02-06 22:40 ` Andrew Dixie
[not found] ` <55598.203.167.214.129.1202337638.squirrel-pmwrj2wvkLORZ0GbGNPwb6VXKuFTiq87@public.gmane.org>
2008-02-06 22:58 ` Trond Myklebust
[not found] ` <1202338699.8549.42.camel-rJ7iovZKK19ZJLDQqaL3InhyD016LWXt@public.gmane.org>
2008-02-08 0:05 ` Andrew Dixie
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20080205090132.GA8286@stro.at \
--to=max-u9r9yedmy7a@public.gmane.org \
--cc=Trond.Myklebust@netapp.com \
--cc=andrew-0FSrDVjkvKm9koe0gwxAeg@public.gmane.org \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.