From: Sasha Levin <sasha.levin@oracle.com>
To: Davidlohr Bueso <dave@stgolabs.net>, Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
LKML <linux-kernel@vger.kernel.org>,
Dave Jones <davej@codemonkey.org.uk>,
jason.low2@hp.com, Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: sched: softlockups in multi_cpu_stop
Date: Fri, 06 Mar 2015 16:59:02 -0500 [thread overview]
Message-ID: <54FA2326.3000300@oracle.com> (raw)
In-Reply-To: <54F9EBCA.1060300@oracle.com>
On 03/06/2015 01:02 PM, Sasha Levin wrote:
> I can go redo that again if you suspect that that commit is not the cause.
I took a closer look at the logs, and I'm seeing hangs that begin this way
as well:
[ 2298.020237] NMI watchdog: BUG: soft lockup - CPU#19 stuck for 23s! [trinity-c19:839]
[ 2298.020237] Modules linked in:
[ 2298.020237] CPU: 19 PID: 839 Comm: trinity-c19 Not tainted 4.0.0-rc2-next-20150306-sasha-00056-g61886e8 #2005
[ 2298.020237] task: ffff880278d62000 ti: ffff880254fe8000 task.ti: ffff880254fe8000
[ 2298.020237] RIP: 0010:[<ffffffffa442702f>] [<ffffffffa442702f>] __rcu_read_unlock+0x9f/0x130
[ 2298.020237] RSP: 0000:ffff880254fefbd8 EFLAGS: 00000207
[ 2298.020237] RAX: dffffc0000000000 RBX: ffff880254fe8000 RCX: 1ffff1004a9fd002
[ 2298.020237] RDX: 1ffff1004f1ac4e2 RSI: ffff8802c3ff6000 RDI: ffff880278d62714
[ 2298.020237] RBP: ffff880254fefbe8 R08: ffff880362b2e080 R09: ffffffff00000001
[ 2298.020237] R10: ffff880362b2e140 R11: ffffea000e253800 R12: 0000000000000a3e
[ 2298.020237] R13: ffff880278d62cb0 R14: ffffed014e1e4899 R15: 0034c1c55efd9eff
[ 2298.020237] FS: 00007f183b9c3700(0000) GS:ffff880375200000(0000) knlGS:0000000000000000
[ 2298.020237] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 2298.020237] CR2: 0000000000bc8fe8 CR3: 0000000259a0a000 CR4: 00000000000007a0
[ 2298.020237] Stack:
[ 2298.020237] ffff8802c3ff6000 ffff880126cfdc28 ffff880254fefc38 ffffffffa43fcb75
[ 2298.020237] ffff8802c3ff6000 ffff8802c3ff6000 ffff880278d62714 ffff880126cfdc48
[ 2298.020237] ffff8802c3ff6000 ffff880126cfdc44 ffff880126cfdc28 ffff880254fefd78
[ 2298.020237] Call Trace:
[ 2298.020237] [<ffffffffa43fcb75>] rwsem_spin_on_owner+0x165/0x250
[ 2298.020237] [<ffffffffae92a67f>] rwsem_down_write_failed+0x22f/0x750
[ 2298.020237] [<ffffffffae92a450>] ? rwsem_down_read_failed+0x260/0x260
[ 2298.020237] [<ffffffffa438fc31>] ? get_parent_ip+0x11/0x50
[ 2298.020237] [<ffffffffa438fd76>] ? preempt_count_add+0x106/0x160
[ 2298.020237] [<ffffffffa5f77c77>] ? debug_smp_processor_id+0x17/0x20
[ 2298.020237] [<ffffffffa47128c0>] ? cmpxchg_double_slab.isra.25+0x210/0x240
[ 2298.020237] [<ffffffffa47119df>] ? free_debug_processing+0x19f/0x320
[ 2298.020237] [<ffffffffa5f4da33>] call_rwsem_down_write_failed+0x13/0x20
[ 2298.020237] [<ffffffffae9296a9>] ? down_write+0x29/0x70
[ 2298.020237] [<ffffffffa46acd32>] validate_mm+0xa2/0x910
[ 2298.020237] [<ffffffffa46b5fd1>] do_munmap+0x421/0xf50
[ 2298.020237] [<ffffffffa41627d0>] ? send_sigtrap+0x1e0/0x1e0
[ 2298.020237] [<ffffffffa46b6b5f>] vm_munmap+0x5f/0x80
[ 2298.020237] [<ffffffffa46b9562>] SyS_munmap+0x22/0x30
[ 2298.020237] [<ffffffffae92e60d>] system_call_fastpath+0x16/0x1b
[ 2298.020237] Code: 02 84 c0 74 04 3c 03 7e 7c c7 83 10 07 00 00 00 00 00 80 48 8d bb 14 07 00 00 48 b8 00 00 00 00 00 fc ff df 48 89 fa 48 c1 ea 03 <0f> b6 14 02 48 89 f8 83 e0 07 83 c0 01 38 d0 7c 04 84 d2 75 52
So it seems that we end up spinning for quite a while?
Thanks,
Sasha
next prev parent reply other threads:[~2015-03-06 21:59 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-03-02 7:45 sched: softlockups in multi_cpu_stop Sasha Levin
[not found] ` <CAMiJ5CVWvUhGK=MWYB_CTNs901p=jsT4i5gkWTaHih7qdQdkFQ@mail.gmail.com>
2015-03-04 5:44 ` Rafael David Tinoco
2015-03-06 11:27 ` Sasha Levin
2015-03-06 12:32 ` Ingo Molnar
2015-03-06 14:34 ` Rafael David Tinoco
2015-03-06 14:45 ` Sasha Levin
2015-03-06 15:46 ` Sasha Levin
2015-03-06 17:19 ` Davidlohr Bueso
2015-03-06 18:02 ` Sasha Levin
2015-03-06 21:59 ` Sasha Levin [this message]
2015-03-06 18:57 ` Jason Low
2015-03-06 19:05 ` Linus Torvalds
2015-03-06 19:20 ` Davidlohr Bueso
2015-03-06 19:32 ` Linus Torvalds
2015-03-06 19:45 ` Davidlohr Bueso
2015-03-06 19:55 ` Davidlohr Bueso
2015-03-06 20:00 ` Davidlohr Bueso
2015-03-06 21:42 ` Linus Torvalds
2015-03-06 19:29 ` Jason Low
2015-03-06 21:12 ` Jason Low
2015-03-06 21:24 ` Linus Torvalds
2015-03-07 1:53 ` Jason Low
2015-03-06 22:15 ` Davidlohr Bueso
2015-03-07 1:55 ` Ming Lei
2015-03-07 2:07 ` Davidlohr Bueso
2015-03-07 2:10 ` Ming Lei
2015-03-07 2:26 ` Davidlohr Bueso
2015-03-07 2:29 ` Davidlohr Bueso
2015-03-07 2:55 ` Ming Lei
2015-03-07 3:10 ` Davidlohr Bueso
2015-03-07 3:19 ` Ming Lei
2015-03-07 3:41 ` Davidlohr Bueso
2015-03-07 2:56 ` Jason Low
2015-03-07 3:08 ` Ming Lei
2015-03-07 3:10 ` Davidlohr Bueso
2015-03-07 3:17 ` Jason Low
2015-03-07 3:39 ` Ming Lei
2015-03-07 3:53 ` Jason Low
2015-03-07 1:58 ` Jason Low
2015-03-07 4:31 ` Jason Low
2015-03-07 4:44 ` Davidlohr Bueso
2015-03-07 6:45 ` Jason Low
2015-03-07 5:54 ` Ming Lei
2015-03-07 6:57 ` Jason Low
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=54FA2326.3000300@oracle.com \
--to=sasha.levin@oracle.com \
--cc=dave@stgolabs.net \
--cc=davej@codemonkey.org.uk \
--cc=jason.low2@hp.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox