From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>
Cc: viro@ZenIV.linux.org.uk, linux-kernel@vger.kernel.org,
hugh@veritas.com, jmorris@namei.org, akpm@linux-foundation.org
Subject: Re: [2.6.30-rc1] RCU detected CPU 1 stall
Date: Mon, 13 Apr 2009 12:55:04 -0700 [thread overview]
Message-ID: <20090413195504.GF6983@linux.vnet.ibm.com> (raw)
In-Reply-To: <200904130048.n3D0mw1f077050@www262.sakura.ne.jp>
On Mon, Apr 13, 2009 at 09:48:58AM +0900, Tetsuo Handa wrote:
> Hello.
>
> Paul E. McKenney wrote:
> > Is this reproducible?
> Not always, but it is reproducible.
Does it reproduce well enough to bisect?
> Al Viro wrote:
> > I'd really love to see results of repeated alt-sysrq-p/alt-sysrq-l, just
> > to see where was it actually spinning.
> Below is sysrq message.
> Maybe something related to khelper's current->mm == NULL warning problem.
> Full log is at http://I-love.SAKURA.ne.jp/tmp/dmesg-2.6.30-rc1-200904130930.txt .
I have to defer to the mm guys on this one.
Thanx, Paul
> [ 47.412519] SysRq : Show Regs
> [ 47.413986]
> [ 47.414584] Pid: 3655, comm: khelper Tainted: G W (2.6.30-rc1 #1) VMware Virtual Platform
> [ 47.415804] EIP: 0060:[<c0379c3d>] EFLAGS: 00000293 CPU: 0
> [ 47.415804] EIP is at __get_user_4+0x11/0x17
> [ 47.415804] EAX: f7150003 EBX: f7150000 ECX: 00000000 EDX: f6744000
> [ 47.415804] ESI: 000007b8 EDI: 7fffffff EBP: f6744f20 ESP: f6744f10
> [ 47.415804] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> [ 47.415804] CR0: 8005003b CR2: f7150000 CR3: 3599a000 CR4: 000006d0
> [ 47.415804] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> [ 47.415804] DR6: ffff0ff0 DR7: 00000400
> [ 47.415804] Call Trace:
> [ 47.415804] [<c0225f4e>] ? count+0x3e/0xb0
> [ 47.415804] [<c0228581>] do_execve+0x621/0x890
> [ 47.415804] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 47.415804] [<c010237e>] sys_execve+0x5e/0xb0
> [ 47.415804] [<c0103d19>] syscall_call+0x7/0xb
> [ 47.415804] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 47.415804] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 47.415804] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 47.415804] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 52.551469] SysRq : Show backtrace of all active CPUs
> [ 52.551803] CPU0:
> [ 52.551803]
> [ 52.551803] Pid: 3655, comm: khelper Tainted: G W (2.6.30-rc1 #1) VMware Virtual Platform
> [ 52.551803] EIP: 0060:[<c0379c3d>] EFLAGS: 00000293 CPU: 0
> [ 52.551803] EIP is at __get_user_4+0x11/0x17
> [ 52.551803] EAX: f7150003 EBX: f7150000 ECX: 00000000 EDX: f6744000
> [ 52.551803] ESI: 000007b8 EDI: 7fffffff EBP: f6744f20 ESP: f6744f10
> [ 52.551803] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> [ 52.551803] CR0: 8005003b CR2: f7150000 CR3: 3599a000 CR4: 000006d0
> [ 52.551803] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> [ 52.551803] DR6: ffff0ff0 DR7: 00000400
> [ 52.551803] Call Trace:
> [ 52.551803] [<c0225f4e>] ? count+0x3e/0xb0
> [ 52.551803] [<c0228581>] do_execve+0x621/0x890
> [ 52.551803] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 52.551803] [<c010237e>] sys_execve+0x5e/0xb0
> [ 52.551803] [<c0103d19>] syscall_call+0x7/0xb
> [ 52.551803] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 52.551803] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 52.551803] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 52.551803] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 60.956967] SysRq : Show Regs
> [ 60.958362]
> [ 60.958896] Pid: 3655, comm: khelper Tainted: G W (2.6.30-rc1 #1) VMware Virtual Platform
> [ 60.959802] EIP: 0060:[<c0379c3d>] EFLAGS: 00000293 CPU: 0
> [ 60.959802] EIP is at __get_user_4+0x11/0x17
> [ 60.959802] EAX: f7150003 EBX: f7150000 ECX: 00000000 EDX: f6744000
> [ 60.959802] ESI: 000007b8 EDI: 7fffffff EBP: f6744f20 ESP: f6744f10
> [ 60.959802] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> [ 60.959802] CR0: 8005003b CR2: f7150000 CR3: 3599a000 CR4: 000006d0
> [ 60.959802] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> [ 60.959802] DR6: ffff0ff0 DR7: 00000400
> [ 60.959802] Call Trace:
> [ 60.959802] [<c0225f4e>] ? count+0x3e/0xb0
> [ 60.959802] [<c0228581>] do_execve+0x621/0x890
> [ 60.959802] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 60.959802] [<c010237e>] sys_execve+0x5e/0xb0
> [ 60.959802] [<c0103d19>] syscall_call+0x7/0xb
> [ 60.959802] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 60.959802] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 60.959802] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 60.959802] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 69.014489] SysRq : Show backtrace of all active CPUs
> [ 69.015802] CPU0:
> [ 69.015802]
> [ 69.015802] Pid: 3655, comm: khelper Tainted: G W (2.6.30-rc1 #1) VMware Virtual Platform
> [ 69.015802] EIP: 0060:[<c0379c3d>] EFLAGS: 00000293 CPU: 0
> [ 69.015802] EIP is at __get_user_4+0x11/0x17
> [ 69.015802] EAX: f7150003 EBX: f7150000 ECX: 00000000 EDX: f6744000
> [ 69.015802] ESI: 000007b8 EDI: 7fffffff EBP: f6744f20 ESP: f6744f10
> [ 69.015802] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> [ 69.015802] CR0: 8005003b CR2: f7150000 CR3: 3599a000 CR4: 000006d0
> [ 69.015802] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> [ 69.015802] DR6: ffff0ff0 DR7: 00000400
> [ 69.015802] Call Trace:
> [ 69.015802] [<c0225f4e>] ? count+0x3e/0xb0
> [ 69.015802] [<c0228581>] do_execve+0x621/0x890
> [ 69.015802] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 69.015802] [<c010237e>] sys_execve+0x5e/0xb0
> [ 69.015802] [<c0103d19>] syscall_call+0x7/0xb
> [ 69.015802] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 69.015802] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 69.015802] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 69.015802] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 73.695801] INFO: RCU detected CPU 0 stall (t=4294910558/10000 jiffies)
> [ 73.695801] Pid: 3655, comm: khelper Tainted: G W 2.6.30-rc1 #1
> [ 73.695801] Call Trace:
> [ 73.695801] [<c015949d>] ? printk+0x1d/0x30
> [ 73.695801] [<c01b38b5>] print_cpu_stall+0x45/0xa0
> [ 73.695801] [<c01b39f8>] check_cpu_stall+0xe8/0x190
> [ 73.695801] [<c01ad59d>] ? get_timestamp+0xd/0x20
> [ 73.695801] [<c01b43ab>] __rcu_pending+0x1b/0x160
> [ 73.695801] [<c01b451e>] rcu_pending+0x2e/0x70
> [ 73.695801] [<c01692ab>] update_process_times+0x3b/0x80
> [ 73.695801] [<c018bcf0>] tick_periodic+0x40/0x90
> [ 73.695801] [<c018bd5e>] tick_handle_periodic+0x1e/0xa0
> [ 73.695801] [<c01045e0>] ? apic_timer_interrupt+0x28/0x34
> [ 73.695801] [<c012215f>] local_apic_timer_interrupt+0x6f/0x80
> [ 73.695801] [<c0161184>] ? irq_enter+0x14/0x60
> [ 73.695801] [<c05e7d83>] smp_apic_timer_interrupt+0x33/0x42
> [ 73.695801] [<c01045e7>] apic_timer_interrupt+0x2f/0x34
> [ 73.695801] [<c0379c3d>] ? __get_user_4+0x11/0x17
> [ 73.695801] [<c0225f4e>] ? count+0x3e/0xb0
> [ 73.695801] [<c0228581>] do_execve+0x621/0x890
> [ 73.695801] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 73.695801] [<c010237e>] sys_execve+0x5e/0xb0
> [ 73.695801] [<c0103d19>] syscall_call+0x7/0xb
> [ 73.695801] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 73.695801] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 73.695801] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 73.695801] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 79.029887] SysRq : Show Regs
> [ 79.030975]
> [ 79.031578] Pid: 3655, comm: khelper Tainted: G W (2.6.30-rc1 #1) VMware Virtual Platform
> [ 79.031800] EIP: 0060:[<c0379c3d>] EFLAGS: 00000293 CPU: 0
> [ 79.031800] EIP is at __get_user_4+0x11/0x17
> [ 79.031800] EAX: f7150003 EBX: f7150000 ECX: 00000000 EDX: f6744000
> [ 79.031800] ESI: 000007b8 EDI: 7fffffff EBP: f6744f20 ESP: f6744f10
> [ 79.031800] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> [ 79.031800] CR0: 8005003b CR2: f7150000 CR3: 3599a000 CR4: 000006d0
> [ 79.031800] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> [ 79.031800] DR6: ffff0ff0 DR7: 00000400
> [ 79.031800] Call Trace:
> [ 79.031800] [<c0225f4e>] ? count+0x3e/0xb0
> [ 79.031800] [<c0228581>] do_execve+0x621/0x890
> [ 79.031800] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 79.031800] [<c010237e>] sys_execve+0x5e/0xb0
> [ 79.031800] [<c0103d19>] syscall_call+0x7/0xb
> [ 79.031800] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 79.031800] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 79.031800] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 79.031800] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 87.925907] SysRq : Show backtrace of all active CPUs
> [ 87.927799] CPU0:
> [ 87.927799]
> [ 87.927799] Pid: 3655, comm: khelper Tainted: G W (2.6.30-rc1 #1) VMware Virtual Platform
> [ 87.927799] EIP: 0060:[<c0379c3d>] EFLAGS: 00000293 CPU: 0
> [ 87.927799] EIP is at __get_user_4+0x11/0x17
> [ 87.927799] EAX: f7150003 EBX: f7150000 ECX: 00000000 EDX: f6744000
> [ 87.927799] ESI: 000007b8 EDI: 7fffffff EBP: f6744f20 ESP: f6744f10
> [ 87.927799] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> [ 87.927799] CR0: 8005003b CR2: f7150000 CR3: 3599a000 CR4: 000006d0
> [ 87.927799] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> [ 87.927799] DR6: ffff0ff0 DR7: 00000400
> [ 87.927799] Call Trace:
> [ 87.927799] [<c0225f4e>] ? count+0x3e/0xb0
> [ 87.927799] [<c0228581>] do_execve+0x621/0x890
> [ 87.927799] [<c022bd8b>] ? getname+0x6b/0xa0
> [ 87.927799] [<c010237e>] sys_execve+0x5e/0xb0
> [ 87.927799] [<c0103d19>] syscall_call+0x7/0xb
> [ 87.927799] [<c010aee4>] ? kernel_execve+0x24/0x30
> [ 87.927799] [<c0172b6f>] ? ____call_usermodehelper+0xff/0x170
> [ 87.927799] [<c0172a70>] ? ____call_usermodehelper+0x0/0x170
> [ 87.927799] [<c0104707>] ? kernel_thread_helper+0x7/0x10
> [ 98.785592] BUG: soft lockup - CPU#1 stuck for 61s! [khelper:3657]
> [ 98.785592] Modules linked in: ipv6 pcnet32 crc32
> [ 98.785592] irq event stamp: 0
> [ 98.785592] hardirqs last enabled at (0): [<(null)>] (null)
> [ 98.785592] hardirqs last disabled at (0): [<c0155bcb>] copy_process+0x2ab/0x11a0
> [ 98.785592] softirqs last enabled at (0): [<c0155bcb>] copy_process+0x2ab/0x11a0
> [ 98.785592] softirqs last disabled at (0): [<(null)>] (null)
> [ 98.785592]
>
> Regards.
next prev parent reply other threads:[~2009-04-13 19:55 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-04-08 0:57 [2.6.30-rc1] RCU detected CPU 1 stall Tetsuo Handa
2009-04-10 14:22 ` Paul E. McKenney
2009-04-10 15:03 ` Al Viro
2009-04-10 15:32 ` Paul E. McKenney
2009-04-10 21:08 ` Tetsuo Handa
2009-04-10 23:12 ` Paul E. McKenney
2009-04-10 23:39 ` Al Viro
2009-04-11 0:48 ` Paul E. McKenney
2009-04-13 0:48 ` Tetsuo Handa
2009-04-13 19:55 ` Paul E. McKenney [this message]
2009-04-13 21:31 ` Tetsuo Handa
2009-04-13 22:47 ` Paul E. McKenney
2009-04-14 17:11 ` Hugh Dickins
2009-04-15 3:28 ` Tetsuo Handa
2009-04-15 20:34 ` Hugh Dickins
2009-04-16 0:46 ` Tetsuo Handa
2009-04-16 5:15 ` Hugh Dickins
2009-04-16 5:35 ` Tetsuo Handa
2009-04-16 20:37 ` Greg KH
2009-04-16 20:55 ` Please revert kobject_uevent UMH_NO_WAIT Hugh Dickins
2009-04-16 20:58 ` [PATCH] mm: pass correct mm when growing stack Hugh Dickins
2009-04-16 21:24 ` Please revert kobject_uevent UMH_NO_WAIT Greg KH
2009-04-16 21:30 ` Arjan van de Ven
2009-04-16 21:36 ` Will Newton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20090413195504.GF6983@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=hugh@veritas.com \
--cc=jmorris@namei.org \
--cc=linux-kernel@vger.kernel.org \
--cc=penguin-kernel@i-love.sakura.ne.jp \
--cc=viro@ZenIV.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.