From: Vivek Goyal <vgoyal@redhat.com>
To: Michel Lespinasse <walken@google.com>
Cc: linux kernel mailing list <linux-kernel@vger.kernel.org>,
Hugh Dickins <hughd@google.com>, Rik van Riel <riel@redhat.com>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: Re: 3.9-rc5: Encountedred INFO: rcu_sched self-detected stall on CPU due to 09a9f1d27
Date: Mon, 15 Apr 2013 13:34:24 -0400 [thread overview]
Message-ID: <20130415173424.GB31868@redhat.com> (raw)
In-Reply-To: <20130415163552.GA31868@redhat.com>
On Mon, Apr 15, 2013 at 12:35:52PM -0400, Vivek Goyal wrote:
[..]
> > My first guess would be that mmap_sem is held during exec, so you
> > can't have __mm_populate() try holding it recursively.
>
> I think it is not mmap_sem as even with VM_LOCKED, we take mmap_sem
> and things are fine.
>
> So things work till 3.8 and break in 3.8-rc1 (with both VM_LOCKED and
> VM_POPULATE specifed). I will do git bisect and try to figure out which
> is first commit which has the issue.
Ok, following seems to be first bad commit.
commit bebeb3d68b24bb4132d452c5707fe321208bcbcd
Author: Michel Lespinasse <walken@google.com>
Date: Fri Feb 22 16:32:37 2013 -0800
mm: introduce mm_populate() for populating new vmas
I have locked down /sbin/kexec. And I get following traceback after a while.
Thanks
Vivek
[ 93.130001] INFO: rcu_sched self-detected stall on CPU[ 93.131007]
INFO: rcu_sched detected stalls on CPUs/tasks: { 2} (detected by 3,
t=60002 jiffies, g=2786, c=2785, q=10480)
[ 93.131016] Task dump for CPU 2:
[ 93.131018] kexec R running task 0 4861 2982
0x00000088
[ 93.131022] ffff88013252bc68 ffffffff8109ab12 00000000ffffffff
0000000000000000
[ 93.131026] ffff88013252bc58 ffff88013a43c180 ffffffff81881edc
00007feed583221a
[ 93.131028] 00007feed5832000 0000000000000000 ffff88013a43c828
ffff88013a43c180
[ 93.131032] Call Trace:
[ 93.131032] [<ffffffff8109ab12>] ? mark_held_locks+0x82/0x130
[ 93.131040] [<ffffffff81881edc>] ? retint_restore_args+0xe/0xe
[ 93.131045] [<ffffffff8109ab12>] ? mark_held_locks+0x82/0x130
[ 93.131047] [<ffffffff81881edc>] ? retint_restore_args+0xe/0xe
[ 93.131049] [<ffffffff8109acc5>] ?
trace_hardirqs_on_caller+0x105/0x190
[ 93.131051] [<ffffffff8132913e>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[ 93.131055] [<ffffffff81881edc>] ? retint_restore_args+0xe/0xe
[ 93.131058] [<ffffffff81127898>] ? __get_user_pages+0x1d8/0x540
[ 93.131061] [<ffffffff81129fb7>] ? __mlock_vma_pages_range+0x67/0x70
[ 93.131064] [<ffffffff8112a721>] ? __mm_populate+0x71/0x150
[ 93.131066] [<ffffffff8111d01f>] ? vm_mmap_pgoff+0xdf/0xf0
[ 93.131071] [<ffffffff8112c839>] ? sys_mmap_pgoff+0xa9/0x180
[ 93.131073] [<ffffffff8109acc5>] ?
trace_hardirqs_on_caller+0x105/0x190
[ 93.131076] [<ffffffff8132913e>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[ 93.131078] [<ffffffff81006972>] ? sys_mmap+0x22/0x30
[ 93.131081] [<ffffffff8188a342>] ? system_call_fastpath+0x16/0x1b
[ 93.130001] { 2} (t=60153 jiffies g=2786 c=2785 q=10480)
[ 93.130001] Pid: 4861, comm: kexec Not tainted 3.8.0+ #270
[ 93.130001] Call Trace:
[ 93.130001] <IRQ> [<ffffffff810ce69a>]
rcu_check_callbacks+0x21a/0x760
[ 93.130001] [<ffffffff810d22ec>] ? acct_account_cputime+0x1c/0x20
[ 93.130001] [<ffffffff81077570>] ? account_system_time+0x160/0x230
[ 93.130001] [<ffffffff8104f388>] update_process_times+0x48/0x80
[ 93.130001] [<ffffffff810934ad>] tick_sched_handle+0x3d/0x50
[ 93.130001] [<ffffffff810936b5>] tick_sched_timer+0x45/0x70
[ 93.130001] [<ffffffff81067661>] __run_hrtimer+0x81/0x220
[ 93.130001] [<ffffffff81093670>] ? tick_nohz_handler+0xa0/0xa0
[ 93.130001] [<ffffffff8108ce7c>] ? ktime_get_update_offsets+0x4c/0xd0
[ 93.130001] [<ffffffff81067fb7>] hrtimer_interrupt+0xf7/0x250
[ 93.130001] [<ffffffff8188bdb9>] smp_apic_timer_interrupt+0x69/0x99
[ 93.130001] [<ffffffff8188af6f>] apic_timer_interrupt+0x6f/0x80
[ 93.130001] <EOI> [<ffffffff81881edc>] ? retint_restore_args+0xe/0xe
[ 93.130001] [<ffffffff81127898>] ? __get_user_pages+0x1d8/0x540
[ 93.130001] [<ffffffff81129fb7>] __mlock_vma_pages_range+0x67/0x70
[ 93.130001] [<ffffffff8112a721>] __mm_populate+0x71/0x150
[ 93.130001] [<ffffffff8111d01f>] vm_mmap_pgoff+0xdf/0xf0
[ 93.130001] [<ffffffff8112c839>] sys_mmap_pgoff+0xa9/0x180
[ 93.130001] [<ffffffff8109acc5>] ?
trace_hardirqs_on_caller+0x105/0x190
[ 93.130001] [<ffffffff8132913e>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[ 93.130001] [<ffffffff81006972>] sys_mmap+0x22/0x30
[ 93.130001] [<ffffffff8188a342>] system_call_fastpath+0x16/0x1b
next prev parent reply other threads:[~2013-04-15 17:35 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-04-12 18:13 3.9-rc5: Encountedred INFO: rcu_sched self-detected stall on CPU due to 09a9f1d27 Vivek Goyal
2013-04-13 2:12 ` Michel Lespinasse
2013-04-13 2:31 ` Michel Lespinasse
2013-04-15 16:35 ` Vivek Goyal
2013-04-15 17:34 ` Vivek Goyal [this message]
2013-04-15 17:59 ` Vivek Goyal
2013-04-15 19:13 ` Vivek Goyal
2013-04-15 21:47 ` Hugh Dickins
2013-04-16 0:21 ` Michel Lespinasse
2013-04-16 1:27 ` Hugh Dickins
2013-04-29 8:57 ` Michel Lespinasse
2013-04-29 13:29 ` Vivek Goyal
2013-05-20 20:23 ` Vivek Goyal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130415173424.GB31868@redhat.com \
--to=vgoyal@redhat.com \
--cc=hughd@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
--cc=riel@redhat.com \
--cc=walken@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox