From: Gertjan Oude Lohuis <gertjan-DW70C6hi67U@public.gmane.org>
To: linux-nfs@vger.kernel.org
Subject: Kernel (2.6.24) crash on nfsd (BUG: soft lockup)
Date: Tue, 26 Feb 2008 16:48:34 +0100 [thread overview]
Message-ID: <47C434D2.80601@byte.nl> (raw)
[-- Attachment #1: Type: text/plain, Size: 722 bytes --]
Hi!
One of our fileservers went down pretty hard yesterday. We recently
upgraded the kernel to 2.6.24 because we suffered from the lockd-lockup
with our previous kernel (2.6.18).
The server stopped responding completely to any requests (nfs, ssh,
ping) and every few seconds a stacktrace was dumped on the console. The
stacktraces hint at nfsd (Pid: 2716, comm: nfsd Not tainted
(2.6.24.2-fwsh-byte #2) and various nfs-functions in the trace). I
attached some of them to this message.
Do these stacktraces seem familiar to anyone? I couldn't find any
similar crashes with google.
--
Met vriendelijke groet,
Gertjan Oude Lohuis
Byte Internet
W www.byte.nl
E support-DW70C6hi67U@public.gmane.org
F 020 6255 922
[-- Attachment #2: stacktrace.txt --]
[-- Type: text/plain, Size: 4973 bytes --]
BUG: soft lockup - CPU#0 stuck for 11s! [nfsd:2716]
Pid: 2716, comm: nfsd Not tainted (2.6.24.2-fwsh-byte #2)
EIP: 0060:[<c0140967>] EFLAGS: 00000286 CPU: 0
EIP is at find_get_pages_contig+0x67/0x73
EAX: 00000000 EBX: 00000001 ECX: c25cc520 EDX: c25cc520
ESI: 00000078 EDI: ca2fbdbc EBP: 00000001 ESP: dffb5c6c
DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
CR0: 8005003b CR2: b7f5d000 CR3: 1fc45000 CR4: 000006f0
DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: ffff0ff0 DR7: 00000400
[<c017c49c>] __generic_file_splice_read+0xa2/0x41e
[<c0113b11>] sched_slice+0x15/0x6f
[<c0131291>] getnstimeofday+0x31/0x105
[<c0134301>] clockevents_program_event+0xbf/0x134
[<c012ef49>] ktime_get_ts+0x15/0x47
[<c01231ea>] run_timer_softirq+0x30/0x184
[<c012a893>] __rcu_process_callbacks+0x76/0xbb
[<c011f979>] tasklet_action+0x53/0x93
[<c011f754>] __do_softirq+0xba/0xcf
[<c010e20e>] smp_apic_timer_interrupt+0x2c/0x35
[<c01032e0>] apic_timer_interrupt+0x28/0x30
[<c017c88d>] generic_file_splice_read+0x75/0xc9
[<c017d083>] do_splice_to+0x6e/0x90
[<c017d144>] splice_direct_to_actor+0x9f/0x166
[<f8f2cf72>] nfsd_direct_splice_actor+0x0/0xa [nfsd]
[<c017c818>] generic_file_splice_read+0x0/0xc9
[<f8f2d309>] nfsd_vfs_read+0x38d/0x3b1 [nfsd]
[<f8f2b3b8>] nfsd_acceptable+0x0/0xd1 [nfsd]
[<c016014f>] dentry_open+0x34/0x64
[<f8f2d73c>] nfsd_read+0xee/0xfb [nfsd]
[<f8f33b8b>] nfsd3_proc_read+0xfe/0x186 [nfsd]
[<f8f354cb>] nfs3svc_decode_readargs+0x0/0xeb [nfsd]
[<f8f29855>] nfsd_dispatch+0xc5/0x1ac [nfsd]
[<c0445ab2>] svcauth_unix_set_client+0x116/0x165
[<c0441ad1>] svc_process+0x4e9/0x6b4
[<c01168e2>] default_wake_function+0x0/0x8
[<f8f2963d>] nfsd+0x16a/0x290 [nfsd]
[<f8f294d3>] nfsd+0x0/0x290 [nfsd]
[<c0103463>] kernel_thread_helper+0x7/0x10
=======================
BUG: soft lockup - CPU#0 stuck for 11s! [nfsd:2716]
Pid: 2716, comm: nfsd Not tainted (2.6.24.2-fwsh-byte #2)
EIP: 0060:[<c0140967>] EFLAGS: 00000286 CPU: 0
EIP is at find_get_pages_contig+0x67/0x73
EAX: 00000000 EBX: 00000001 ECX: c25cc520 EDX: c25cc520
ESI: 00000078 EDI: ca2fbdbc EBP: 00000001 ESP: dffb5c6c
DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
CR0: 8005003b CR2: b7f5d000 CR3: 1fc45000 CR4: 000006f0
DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: ffff0ff0 DR7: 00000400
[<c017c49c>] __generic_file_splice_read+0xa2/0x41e
[<c0132efc>] clocksource_get_next+0x3a/0x40
[<c0113b11>] sched_slice+0x15/0x6f
[<c0131291>] getnstimeofday+0x31/0x105
[<c0134301>] clockevents_program_event+0xbf/0x134
[<c012ef49>] ktime_get_ts+0x15/0x47
[<c01231ea>] run_timer_softirq+0x30/0x184
[<c012a893>] __rcu_process_callbacks+0x76/0xbb
[<c011f979>] tasklet_action+0x53/0x93
[<c011f754>] __do_softirq+0xba/0xcf
[<c010e20e>] smp_apic_timer_interrupt+0x2c/0x35
[<c01032e0>] apic_timer_interrupt+0x28/0x30
[<c017007b>] locks_show+0x0/0x67
[<c017c88d>] generic_file_splice_read+0x75/0xc9
[<c017d083>] do_splice_to+0x6e/0x90
[<c017d144>] splice_direct_to_actor+0x9f/0x166
[<f8f2cf72>] nfsd_direct_splice_actor+0x0/0xa [nfsd]
[<c017c818>] generic_file_splice_read+0x0/0xc9
[<f8f2d309>] nfsd_vfs_read+0x38d/0x3b1 [nfsd]
[<f8f2b3b8>] nfsd_acceptable+0x0/0xd1 [nfsd]
[<c016014f>] dentry_open+0x34/0x64
[<f8f2d73c>] nfsd_read+0xee/0xfb [nfsd]
[<f8f33b8b>] nfsd3_proc_read+0xfe/0x186 [nfsd]
[<f8f354cb>] nfs3svc_decode_readargs+0x0/0xeb [nfsd]
[<f8f29855>] nfsd_dispatch+0xc5/0x1ac [nfsd]
[<c0445ab2>] svcauth_unix_set_client+0x116/0x165
[<c0441ad1>] svc_process+0x4e9/0x6b4
[<c01168e2>] default_wake_function+0x0/0x8
[<f8f2963d>] nfsd+0x16a/0x290 [nfsd]
[<f8f294d3>] nfsd+0x0/0x290 [nfsd]
[<c0103463>] kernel_thread_helper+0x7/0x10
=======================
BUG: soft lockup - CPU#0 stuck for 11s! [nfsd:2716]
Pid: 2716, comm: nfsd Not tainted (2.6.24.2-fwsh-byte #2)
EIP: 0060:[<c017c88f>] EFLAGS: 00000246 CPU: 0
EIP is at generic_file_splice_read+0x77/0xc9
EAX: 00000000 EBX: 00000000 ECX: 00000001 EDX: 00000000
ESI: 00000000 EDI: 00000000 EBP: 00001000 ESP: dffb5df0
DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
CR0: 8005003b CR2: b7f5d000 CR3: 1fc45000 CR4: 000006f0
DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: ffff0ff0 DR7: 00000400
[<c017d083>] do_splice_to+0x6e/0x90
[<c017d144>] splice_direct_to_actor+0x9f/0x166
[<f8f2cf72>] nfsd_direct_splice_actor+0x0/0xa [nfsd]
[<c017c818>] generic_file_splice_read+0x0/0xc9
[<f8f2d309>] nfsd_vfs_read+0x38d/0x3b1 [nfsd]
[<f8f2b3b8>] nfsd_acceptable+0x0/0xd1 [nfsd]
[<c016014f>] dentry_open+0x34/0x64
[<f8f2d73c>] nfsd_read+0xee/0xfb [nfsd]
[<f8f33b8b>] nfsd3_proc_read+0xfe/0x186 [nfsd]
[<f8f354cb>] nfs3svc_decode_readargs+0x0/0xeb [nfsd]
[<f8f29855>] nfsd_dispatch+0xc5/0x1ac [nfsd]
[<c0445ab2>] svcauth_unix_set_client+0x116/0x165
[<c0441ad1>] svc_process+0x4e9/0x6b4
[<c01168e2>] default_wake_function+0x0/0x8
[<f8f2963d>] nfsd+0x16a/0x290 [nfsd]
[<f8f294d3>] nfsd+0x0/0x290 [nfsd]
[<c0103463>] kernel_thread_helper+0x7/0x10
=======================
next reply other threads:[~2008-02-26 16:13 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-02-26 15:48 Gertjan Oude Lohuis [this message]
[not found] ` <47C434D2.80601-DW70C6hi67U@public.gmane.org>
2008-02-27 6:46 ` Kernel (2.6.24) crash on nfsd (BUG: soft lockup) Gertjan Oude Lohuis
[not found] ` <47C50754.5030107-DW70C6hi67U@public.gmane.org>
2008-02-27 7:01 ` Gertjan Oude Lohuis
[not found] ` <47C50ABB.8050700-DW70C6hi67U@public.gmane.org>
2008-02-28 10:56 ` Kernel 2.6.23.17 crash (Was: Kernel (2.6.24) crash on nfsd (BUG: soft lockup)) Allard Hoeve
[not found] ` <Pine.LNX.4.62.0802281153040.31013-FHjt3+7qfYHBZBx2VKNGNcSTQT6m/s+e@public.gmane.org>
2008-03-01 16:39 ` J. Bruce Fields
2008-03-01 17:03 ` Jens Axboe
2008-03-05 10:25 ` Gertjan Oude Lohuis
2008-02-28 11:08 ` Kernel (2.6.24) crash on nfsd (BUG: soft lockup) Gertjan Oude Lohuis
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=47C434D2.80601@byte.nl \
--to=gertjan-dw70c6hi67u@public.gmane.org \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.