From: Gertjan Oude Lohuis <gertjan-DW70C6hi67U@public.gmane.org>
To: linux-nfs@vger.kernel.org
Subject: Kernel (2.6.24) crash on nfsd (BUG: soft lockup)
Date: Tue, 26 Feb 2008 16:48:34 +0100 [thread overview]
Message-ID: <47C434D2.80601@byte.nl> (raw)
[-- Attachment #1: Type: text/plain, Size: 722 bytes --]
Hi!
One of our fileservers went down pretty hard yesterday. We recently
upgraded the kernel to 2.6.24 because we suffered from the lockd-lockup
with our previous kernel (2.6.18).
The server stopped responding completely to any requests (nfs, ssh,
ping) and every few seconds a stacktrace was dumped on the console. The
stacktraces hint at nfsd (Pid: 2716, comm: nfsd Not tainted
(2.6.24.2-fwsh-byte #2) and various nfs-functions in the trace). I
attached some of them to this message.
Do these stacktraces seem familiar to anyone? I couldn't find any
similar crashes with google.
--
Met vriendelijke groet,
Gertjan Oude Lohuis
Byte Internet
W www.byte.nl
E support-DW70C6hi67U@public.gmane.org
F 020 6255 922
[-- Attachment #2: stacktrace.txt --]
[-- Type: text/plain, Size: 4973 bytes --]
BUG: soft lockup - CPU#0 stuck for 11s! [nfsd:2716]
Pid: 2716, comm: nfsd Not tainted (2.6.24.2-fwsh-byte #2)
EIP: 0060:[<c0140967>] EFLAGS: 00000286 CPU: 0
EIP is at find_get_pages_contig+0x67/0x73
EAX: 00000000 EBX: 00000001 ECX: c25cc520 EDX: c25cc520
ESI: 00000078 EDI: ca2fbdbc EBP: 00000001 ESP: dffb5c6c
DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
CR0: 8005003b CR2: b7f5d000 CR3: 1fc45000 CR4: 000006f0
DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: ffff0ff0 DR7: 00000400
[<c017c49c>] __generic_file_splice_read+0xa2/0x41e
[<c0113b11>] sched_slice+0x15/0x6f
[<c0131291>] getnstimeofday+0x31/0x105
[<c0134301>] clockevents_program_event+0xbf/0x134
[<c012ef49>] ktime_get_ts+0x15/0x47
[<c01231ea>] run_timer_softirq+0x30/0x184
[<c012a893>] __rcu_process_callbacks+0x76/0xbb
[<c011f979>] tasklet_action+0x53/0x93
[<c011f754>] __do_softirq+0xba/0xcf
[<c010e20e>] smp_apic_timer_interrupt+0x2c/0x35
[<c01032e0>] apic_timer_interrupt+0x28/0x30
[<c017c88d>] generic_file_splice_read+0x75/0xc9
[<c017d083>] do_splice_to+0x6e/0x90
[<c017d144>] splice_direct_to_actor+0x9f/0x166
[<f8f2cf72>] nfsd_direct_splice_actor+0x0/0xa [nfsd]
[<c017c818>] generic_file_splice_read+0x0/0xc9
[<f8f2d309>] nfsd_vfs_read+0x38d/0x3b1 [nfsd]
[<f8f2b3b8>] nfsd_acceptable+0x0/0xd1 [nfsd]
[<c016014f>] dentry_open+0x34/0x64
[<f8f2d73c>] nfsd_read+0xee/0xfb [nfsd]
[<f8f33b8b>] nfsd3_proc_read+0xfe/0x186 [nfsd]
[<f8f354cb>] nfs3svc_decode_readargs+0x0/0xeb [nfsd]
[<f8f29855>] nfsd_dispatch+0xc5/0x1ac [nfsd]
[<c0445ab2>] svcauth_unix_set_client+0x116/0x165
[<c0441ad1>] svc_process+0x4e9/0x6b4
[<c01168e2>] default_wake_function+0x0/0x8
[<f8f2963d>] nfsd+0x16a/0x290 [nfsd]
[<f8f294d3>] nfsd+0x0/0x290 [nfsd]
[<c0103463>] kernel_thread_helper+0x7/0x10
=======================
BUG: soft lockup - CPU#0 stuck for 11s! [nfsd:2716]
Pid: 2716, comm: nfsd Not tainted (2.6.24.2-fwsh-byte #2)
EIP: 0060:[<c0140967>] EFLAGS: 00000286 CPU: 0
EIP is at find_get_pages_contig+0x67/0x73
EAX: 00000000 EBX: 00000001 ECX: c25cc520 EDX: c25cc520
ESI: 00000078 EDI: ca2fbdbc EBP: 00000001 ESP: dffb5c6c
DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
CR0: 8005003b CR2: b7f5d000 CR3: 1fc45000 CR4: 000006f0
DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: ffff0ff0 DR7: 00000400
[<c017c49c>] __generic_file_splice_read+0xa2/0x41e
[<c0132efc>] clocksource_get_next+0x3a/0x40
[<c0113b11>] sched_slice+0x15/0x6f
[<c0131291>] getnstimeofday+0x31/0x105
[<c0134301>] clockevents_program_event+0xbf/0x134
[<c012ef49>] ktime_get_ts+0x15/0x47
[<c01231ea>] run_timer_softirq+0x30/0x184
[<c012a893>] __rcu_process_callbacks+0x76/0xbb
[<c011f979>] tasklet_action+0x53/0x93
[<c011f754>] __do_softirq+0xba/0xcf
[<c010e20e>] smp_apic_timer_interrupt+0x2c/0x35
[<c01032e0>] apic_timer_interrupt+0x28/0x30
[<c017007b>] locks_show+0x0/0x67
[<c017c88d>] generic_file_splice_read+0x75/0xc9
[<c017d083>] do_splice_to+0x6e/0x90
[<c017d144>] splice_direct_to_actor+0x9f/0x166
[<f8f2cf72>] nfsd_direct_splice_actor+0x0/0xa [nfsd]
[<c017c818>] generic_file_splice_read+0x0/0xc9
[<f8f2d309>] nfsd_vfs_read+0x38d/0x3b1 [nfsd]
[<f8f2b3b8>] nfsd_acceptable+0x0/0xd1 [nfsd]
[<c016014f>] dentry_open+0x34/0x64
[<f8f2d73c>] nfsd_read+0xee/0xfb [nfsd]
[<f8f33b8b>] nfsd3_proc_read+0xfe/0x186 [nfsd]
[<f8f354cb>] nfs3svc_decode_readargs+0x0/0xeb [nfsd]
[<f8f29855>] nfsd_dispatch+0xc5/0x1ac [nfsd]
[<c0445ab2>] svcauth_unix_set_client+0x116/0x165
[<c0441ad1>] svc_process+0x4e9/0x6b4
[<c01168e2>] default_wake_function+0x0/0x8
[<f8f2963d>] nfsd+0x16a/0x290 [nfsd]
[<f8f294d3>] nfsd+0x0/0x290 [nfsd]
[<c0103463>] kernel_thread_helper+0x7/0x10
=======================
BUG: soft lockup - CPU#0 stuck for 11s! [nfsd:2716]
Pid: 2716, comm: nfsd Not tainted (2.6.24.2-fwsh-byte #2)
EIP: 0060:[<c017c88f>] EFLAGS: 00000246 CPU: 0
EIP is at generic_file_splice_read+0x77/0xc9
EAX: 00000000 EBX: 00000000 ECX: 00000001 EDX: 00000000
ESI: 00000000 EDI: 00000000 EBP: 00001000 ESP: dffb5df0
DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
CR0: 8005003b CR2: b7f5d000 CR3: 1fc45000 CR4: 000006f0
DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: ffff0ff0 DR7: 00000400
[<c017d083>] do_splice_to+0x6e/0x90
[<c017d144>] splice_direct_to_actor+0x9f/0x166
[<f8f2cf72>] nfsd_direct_splice_actor+0x0/0xa [nfsd]
[<c017c818>] generic_file_splice_read+0x0/0xc9
[<f8f2d309>] nfsd_vfs_read+0x38d/0x3b1 [nfsd]
[<f8f2b3b8>] nfsd_acceptable+0x0/0xd1 [nfsd]
[<c016014f>] dentry_open+0x34/0x64
[<f8f2d73c>] nfsd_read+0xee/0xfb [nfsd]
[<f8f33b8b>] nfsd3_proc_read+0xfe/0x186 [nfsd]
[<f8f354cb>] nfs3svc_decode_readargs+0x0/0xeb [nfsd]
[<f8f29855>] nfsd_dispatch+0xc5/0x1ac [nfsd]
[<c0445ab2>] svcauth_unix_set_client+0x116/0x165
[<c0441ad1>] svc_process+0x4e9/0x6b4
[<c01168e2>] default_wake_function+0x0/0x8
[<f8f2963d>] nfsd+0x16a/0x290 [nfsd]
[<f8f294d3>] nfsd+0x0/0x290 [nfsd]
[<c0103463>] kernel_thread_helper+0x7/0x10
=======================
next reply other threads:[~2008-02-26 16:13 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-02-26 15:48 Gertjan Oude Lohuis [this message]
[not found] ` <47C434D2.80601-DW70C6hi67U@public.gmane.org>
2008-02-27 6:46 ` Kernel (2.6.24) crash on nfsd (BUG: soft lockup) Gertjan Oude Lohuis
[not found] ` <47C50754.5030107-DW70C6hi67U@public.gmane.org>
2008-02-27 7:01 ` Gertjan Oude Lohuis
[not found] ` <47C50ABB.8050700-DW70C6hi67U@public.gmane.org>
2008-02-28 10:56 ` Kernel 2.6.23.17 crash (Was: Kernel (2.6.24) crash on nfsd (BUG: soft lockup)) Allard Hoeve
[not found] ` <Pine.LNX.4.62.0802281153040.31013-FHjt3+7qfYHBZBx2VKNGNcSTQT6m/s+e@public.gmane.org>
2008-03-01 16:39 ` J. Bruce Fields
2008-03-01 17:03 ` Jens Axboe
2008-03-05 10:25 ` Gertjan Oude Lohuis
2008-02-28 11:08 ` Kernel (2.6.24) crash on nfsd (BUG: soft lockup) Gertjan Oude Lohuis
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=47C434D2.80601@byte.nl \
--to=gertjan-dw70c6hi67u@public.gmane.org \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox