From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>
Cc: viro@ZenIV.linux.org.uk, linux-kernel@vger.kernel.org,
hugh@veritas.com, jmorris@namei.org, akpm@linux-foundation.org
Subject: Re: [2.6.30-rc1] RCU detected CPU 1 stall
Date: Mon, 13 Apr 2009 12:55:04 -0700 [thread overview]
Message-ID: <20090413195504.GF6983@linux.vnet.ibm.com> (raw)
In-Reply-To: <200904130048.n3D0mw1f077050@www262.sakura.ne.jp>
On Mon, Apr 13, 2009 at 09:48:58AM +0900, Tetsuo Handa wrote:
> Hello.
>
> Paul E. McKenney wrote:
> > Is this reproducible?
> Not always, but it is reproducible.
Does it reproduce well enough to bisect?
> Al Viro wrote:
> > I'd really love to see results of repeated alt-sysrq-p/alt-sysrq-l, just
> > to see where was it actually spinning.
> Below is sysrq message.
> Maybe something related to khelper's current->mm == NULL warning problem.
> Full log is at http://I-love.SAKURA.ne.jp/tmp/dmesg-2.6.30-rc1-200904130930.txt .
I have to defer to the mm guys on this one.
Thanx, Paul
> [ 47.412519] SysRq : Show Regs
> [ 47.413986]
> [ 47.414584] Pid: 3655, comm: khelper Tainted: G W (2.6.30-rc1 #1) VMware Virtual Platform
> [ 47.415804] EIP: 0060:[<c0379c3d>] EFLAGS: 00000293 CPU: 0
> [ 47.415804] EIP is at __get_user_4+0x11/0x17
> [ 47.415804] EAX: f7150003 EBX: f7150000 ECX: 00000000 EDX: f6744000
> [ 47.415804] ESI: 000007b8 EDI: 7fffffff EBP: f6744f20 ESP: f6744f10
> [ 47.415804] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> [ 47.415804] CR0: 8005003b CR2: f7150000 CR3: 3599a000 CR4: 000006d0
> [ 47.415804] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> [ 47.415804] DR6: ffff0ff0 DR7: 00000400
> [ 47.415804] Call Trace:
> [ 47.415804] [<c0225f4e>] ? count+0x3e/0xb0
> [ 47.415804] [<c0228581>] do_execve+0x621/0x890
> [ 47.415804] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 47.415804] [<c010237e>] sys_execve+0x5e/0xb0
> [ 47.415804] [<c0103d19>] syscall_call+0x7/0xb
> [ 47.415804] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 47.415804] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 47.415804] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 47.415804] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 52.551469] SysRq : Show backtrace of all active CPUs
> [ 52.551803] CPU0:
> [ 52.551803]
> [ 52.551803] Pid: 3655, comm: khelper Tainted: G W (2.6.30-rc1 #1) VMware Virtual Platform
> [ 52.551803] EIP: 0060:[<c0379c3d>] EFLAGS: 00000293 CPU: 0
> [ 52.551803] EIP is at __get_user_4+0x11/0x17
> [ 52.551803] EAX: f7150003 EBX: f7150000 ECX: 00000000 EDX: f6744000
> [ 52.551803] ESI: 000007b8 EDI: 7fffffff EBP: f6744f20 ESP: f6744f10
> [ 52.551803] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> [ 52.551803] CR0: 8005003b CR2: f7150000 CR3: 3599a000 CR4: 000006d0
> [ 52.551803] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> [ 52.551803] DR6: ffff0ff0 DR7: 00000400
> [ 52.551803] Call Trace:
> [ 52.551803] [<c0225f4e>] ? count+0x3e/0xb0
> [ 52.551803] [<c0228581>] do_execve+0x621/0x890
> [ 52.551803] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 52.551803] [<c010237e>] sys_execve+0x5e/0xb0
> [ 52.551803] [<c0103d19>] syscall_call+0x7/0xb
> [ 52.551803] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 52.551803] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 52.551803] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 52.551803] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 60.956967] SysRq : Show Regs
> [ 60.958362]
> [ 60.958896] Pid: 3655, comm: khelper Tainted: G W (2.6.30-rc1 #1) VMware Virtual Platform
> [ 60.959802] EIP: 0060:[<c0379c3d>] EFLAGS: 00000293 CPU: 0
> [ 60.959802] EIP is at __get_user_4+0x11/0x17
> [ 60.959802] EAX: f7150003 EBX: f7150000 ECX: 00000000 EDX: f6744000
> [ 60.959802] ESI: 000007b8 EDI: 7fffffff EBP: f6744f20 ESP: f6744f10
> [ 60.959802] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> [ 60.959802] CR0: 8005003b CR2: f7150000 CR3: 3599a000 CR4: 000006d0
> [ 60.959802] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> [ 60.959802] DR6: ffff0ff0 DR7: 00000400
> [ 60.959802] Call Trace:
> [ 60.959802] [<c0225f4e>] ? count+0x3e/0xb0
> [ 60.959802] [<c0228581>] do_execve+0x621/0x890
> [ 60.959802] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 60.959802] [<c010237e>] sys_execve+0x5e/0xb0
> [ 60.959802] [<c0103d19>] syscall_call+0x7/0xb
> [ 60.959802] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 60.959802] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 60.959802] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 60.959802] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 69.014489] SysRq : Show backtrace of all active CPUs
> [ 69.015802] CPU0:
> [ 69.015802]
> [ 69.015802] Pid: 3655, comm: khelper Tainted: G W (2.6.30-rc1 #1) VMware Virtual Platform
> [ 69.015802] EIP: 0060:[<c0379c3d>] EFLAGS: 00000293 CPU: 0
> [ 69.015802] EIP is at __get_user_4+0x11/0x17
> [ 69.015802] EAX: f7150003 EBX: f7150000 ECX: 00000000 EDX: f6744000
> [ 69.015802] ESI: 000007b8 EDI: 7fffffff EBP: f6744f20 ESP: f6744f10
> [ 69.015802] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> [ 69.015802] CR0: 8005003b CR2: f7150000 CR3: 3599a000 CR4: 000006d0
> [ 69.015802] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> [ 69.015802] DR6: ffff0ff0 DR7: 00000400
> [ 69.015802] Call Trace:
> [ 69.015802] [<c0225f4e>] ? count+0x3e/0xb0
> [ 69.015802] [<c0228581>] do_execve+0x621/0x890
> [ 69.015802] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 69.015802] [<c010237e>] sys_execve+0x5e/0xb0
> [ 69.015802] [<c0103d19>] syscall_call+0x7/0xb
> [ 69.015802] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 69.015802] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 69.015802] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 69.015802] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 73.695801] INFO: RCU detected CPU 0 stall (t=4294910558/10000 jiffies)
> [ 73.695801] Pid: 3655, comm: khelper Tainted: G W 2.6.30-rc1 #1
> [ 73.695801] Call Trace:
> [ 73.695801] [<c015949d>] ? printk+0x1d/0x30
> [ 73.695801] [<c01b38b5>] print_cpu_stall+0x45/0xa0
> [ 73.695801] [<c01b39f8>] check_cpu_stall+0xe8/0x190
> [ 73.695801] [<c01ad59d>] ? get_timestamp+0xd/0x20
> [ 73.695801] [<c01b43ab>] __rcu_pending+0x1b/0x160
> [ 73.695801] [<c01b451e>] rcu_pending+0x2e/0x70
> [ 73.695801] [<c01692ab>] update_process_times+0x3b/0x80
> [ 73.695801] [<c018bcf0>] tick_periodic+0x40/0x90
> [ 73.695801] [<c018bd5e>] tick_handle_periodic+0x1e/0xa0
> [ 73.695801] [<c01045e0>] ? apic_timer_interrupt+0x28/0x34
> [ 73.695801] [<c012215f>] local_apic_timer_interrupt+0x6f/0x80
> [ 73.695801] [<c0161184>] ? irq_enter+0x14/0x60
> [ 73.695801] [<c05e7d83>] smp_apic_timer_interrupt+0x33/0x42
> [ 73.695801] [<c01045e7>] apic_timer_interrupt+0x2f/0x34
> [ 73.695801] [<c0379c3d>] ? __get_user_4+0x11/0x17
> [ 73.695801] [<c0225f4e>] ? count+0x3e/0xb0
> [ 73.695801] [<c0228581>] do_execve+0x621/0x890
> [ 73.695801] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 73.695801] [<c010237e>] sys_execve+0x5e/0xb0
> [ 73.695801] [<c0103d19>] syscall_call+0x7/0xb
> [ 73.695801] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 73.695801] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 73.695801] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 73.695801] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 79.029887] SysRq : Show Regs
> [ 79.030975]
> [ 79.031578] Pid: 3655, comm: khelper Tainted: G W (2.6.30-rc1 #1) VMware Virtual Platform
> [ 79.031800] EIP: 0060:[<c0379c3d>] EFLAGS: 00000293 CPU: 0
> [ 79.031800] EIP is at __get_user_4+0x11/0x17
> [ 79.031800] EAX: f7150003 EBX: f7150000 ECX: 00000000 EDX: f6744000
> [ 79.031800] ESI: 000007b8 EDI: 7fffffff EBP: f6744f20 ESP: f6744f10
> [ 79.031800] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> [ 79.031800] CR0: 8005003b CR2: f7150000 CR3: 3599a000 CR4: 000006d0
> [ 79.031800] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> [ 79.031800] DR6: ffff0ff0 DR7: 00000400
> [ 79.031800] Call Trace:
> [ 79.031800] [<c0225f4e>] ? count+0x3e/0xb0
> [ 79.031800] [<c0228581>] do_execve+0x621/0x890
> [ 79.031800] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 79.031800] [<c010237e>] sys_execve+0x5e/0xb0
> [ 79.031800] [<c0103d19>] syscall_call+0x7/0xb
> [ 79.031800] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 79.031800] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 79.031800] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 79.031800] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 87.925907] SysRq : Show backtrace of all active CPUs
> [ 87.927799] CPU0:
> [ 87.927799]
> [ 87.927799] Pid: 3655, comm: khelper Tainted: G W (2.6.30-rc1 #1) VMware Virtual Platform
> [ 87.927799] EIP: 0060:[<c0379c3d>] EFLAGS: 00000293 CPU: 0
> [ 87.927799] EIP is at __get_user_4+0x11/0x17
> [ 87.927799] EAX: f7150003 EBX: f7150000 ECX: 00000000 EDX: f6744000
> [ 87.927799] ESI: 000007b8 EDI: 7fffffff EBP: f6744f20 ESP: f6744f10
> [ 87.927799] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> [ 87.927799] CR0: 8005003b CR2: f7150000 CR3: 3599a000 CR4: 000006d0
> [ 87.927799] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> [ 87.927799] DR6: ffff0ff0 DR7: 00000400
> [ 87.927799] Call Trace:
> [ 87.927799] [<c0225f4e>] ? count+0x3e/0xb0
> [ 87.927799] [<c0228581>] do_execve+0x621/0x890
> [ 87.927799] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 87.927799] [<c010237e>] sys_execve+0x5e/0xb0
> [ 87.927799] [<c0103d19>] syscall_call+0x7/0xb
> [ 87.927799] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 87.927799] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 87.927799] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 87.927799] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 98.785592] BUG: soft lockup - CPU#1 stuck for 61s! [khelper:3657]
> [ 98.785592] Modules linked in: ipv6 pcnet32 crc32
> [ 98.785592] irq event stamp: 0
> [ 98.785592] hardirqs last enabled at (0): [<(null)>] (null)
> [ 98.785592] hardirqs last disabled at (0): [<c0155bcb>] copy_process+0x2ab/0x11a0
> [ 98.785592] softirqs last enabled at (0): [<c0155bcb>] copy_process+0x2ab/0x11a0
> [ 98.785592] softirqs last disabled at (0): [<(null)>] (null)
> [ 98.785592]
>
> Regards.
next prev parent reply other threads:[~2009-04-13 19:55 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-04-08 0:57 [2.6.30-rc1] RCU detected CPU 1 stall Tetsuo Handa
2009-04-10 14:22 ` Paul E. McKenney
2009-04-10 15:03 ` Al Viro
2009-04-10 15:32 ` Paul E. McKenney
2009-04-10 21:08 ` Tetsuo Handa
2009-04-10 23:12 ` Paul E. McKenney
2009-04-10 23:39 ` Al Viro
2009-04-11 0:48 ` Paul E. McKenney
2009-04-13 0:48 ` Tetsuo Handa
2009-04-13 19:55 ` Paul E. McKenney [this message]
2009-04-13 21:31 ` Tetsuo Handa
2009-04-13 22:47 ` Paul E. McKenney
2009-04-14 17:11 ` Hugh Dickins
2009-04-15 3:28 ` Tetsuo Handa
2009-04-15 20:34 ` Hugh Dickins
2009-04-16 0:46 ` Tetsuo Handa
2009-04-16 5:15 ` Hugh Dickins
2009-04-16 5:35 ` Tetsuo Handa
2009-04-16 20:37 ` Greg KH
2009-04-16 20:55 ` Please revert kobject_uevent UMH_NO_WAIT Hugh Dickins
2009-04-16 20:58 ` [PATCH] mm: pass correct mm when growing stack Hugh Dickins
2009-04-16 21:24 ` Please revert kobject_uevent UMH_NO_WAIT Greg KH
2009-04-16 21:30 ` Arjan van de Ven
2009-04-16 21:36 ` Will Newton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20090413195504.GF6983@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=hugh@veritas.com \
--cc=jmorris@namei.org \
--cc=linux-kernel@vger.kernel.org \
--cc=penguin-kernel@i-love.sakura.ne.jp \
--cc=viro@ZenIV.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox