From: Dave Jones <davej@redhat.com>
To: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Linux Kernel <linux-kernel@vger.kernel.org>
Subject: Re: rcu_prempt stalls / lockup
Date: Tue, 1 Apr 2014 11:08:49 -0400 [thread overview]
Message-ID: <20140401150849.GA14757@redhat.com> (raw)
In-Reply-To: <20140401004801.GQ4284@linux.vnet.ibm.com>
On Mon, Mar 31, 2014 at 05:48:01PM -0700, Paul E. McKenney wrote:
> Alternatively, I suppose a wakeup could be getting lost. The main change
> related to that this merge window was ffa83fb565fb, which eliminated
> idle wakeups from RCU in the CONFIG_RCU_NOCB_CPU_ALL=y case.
>
> So, could you please try reverting ffa83fb565fb?
Did a run overnight with that reverted, no luck..
[11869.580161] INFO: rcu_preempt detected stalls on CPUs/tasks:
[11869.581245] (detected by 0, t=6502 jiffies, g=449892, c=449891, q=0)
[11869.582047] INFO: Stall ended before state dump start
[12064.517461] INFO: rcu_preempt detected stalls on CPUs/tasks:
[12064.518539] (detected by 0, t=26007 jiffies, g=449892, c=449891, q=0)
[12064.519347] INFO: Stall ended before state dump start
[12259.454767] INFO: rcu_preempt detected stalls on CPUs/tasks:
[12259.455818] (detected by 0, t=45512 jiffies, g=449892, c=449891, q=0)
[12259.456602] INFO: Stall ended before state dump start
[12454.392112] INFO: rcu_preempt detected stalls on CPUs/tasks:
[12454.393131] (detected by 0, t=65017 jiffies, g=449892, c=449891, q=0)
[12454.393942] INFO: Stall ended before state dump start
[12649.329446] INFO: rcu_preempt detected stalls on CPUs/tasks:
[12649.330446] (detected by 0, t=84522 jiffies, g=449892, c=449891, q=0)
[12649.331267] INFO: Stall ended before state dump start
[12844.266776] INFO: rcu_preempt detected stalls on CPUs/tasks:
[12844.267695] (detected by 0, t=104027 jiffies, g=449892, c=449891, q=0)
[12844.268571] INFO: Stall ended before state dump start
[13039.204102] INFO: rcu_preempt detected stalls on CPUs/tasks:
[13039.205119] (detected by 0, t=123532 jiffies, g=449892, c=449891, q=0)
[13039.205871] INFO: Stall ended before state dump start
[13234.141424] INFO: rcu_preempt detected stalls on CPUs/tasks:
[13234.141424] INFO: rcu_preempt detected stalls on CPUs/tasks:
[13234.142448] (detected by 0, t=143037 jiffies, g=449892, c=449891, q=0)
[13234.143339] INFO: Stall ended before state dump start
[13429.078730] INFO: rcu_preempt detected stalls on CPUs/tasks:
[13429.079771] (detected by 0, t=162542 jiffies, g=449892, c=449891, q=0)
[13429.080591] INFO: Stall ended before state dump start
[13624.016058] INFO: rcu_preempt detected stalls on CPUs/tasks:
[13624.017116] (detected by 0, t=182047 jiffies, g=449892, c=449891, q=0)
[13624.018033] INFO: Stall ended before state dump start
[13818.953370] INFO: rcu_preempt detected stalls on CPUs/tasks:
[13818.954437] (detected by 0, t=201552 jiffies, g=449892, c=449891, q=0)
[13818.955334] INFO: Stall ended before state dump start
[14013.890682] INFO: rcu_preempt detected stalls on CPUs/tasks:
[14013.891761] (detected by 0, t=221057 jiffies, g=449892, c=449891, q=0)
[14013.892691] INFO: Stall ended before state dump start
[14208.827993] INFO: rcu_preempt detected stalls on CPUs/tasks:
[14208.829088] (detected by 0, t=240562 jiffies, g=449892, c=449891, q=0)
[14208.829977] INFO: Stall ended before state dump start
[14403.765300] INFO: rcu_preempt detected stalls on CPUs/tasks:
[14403.766405] (detected by 0, t=260067 jiffies, g=449892, c=449891, q=0)
and on and on..
so kernel space still works like before, but userspace is locked up.
> If that doesn't work, I will need to put together some diagnostic patches.
> Starting with the one below.
bugger, I forgot to apply this last night. I'll do it on the next run.
Dave
next prev parent reply other threads:[~2014-04-01 15:09 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-03-31 23:02 rcu_prempt stalls / lockup Dave Jones
2014-03-31 23:22 ` Paul E. McKenney
2014-03-31 23:35 ` Dave Jones
2014-04-01 0:48 ` Paul E. McKenney
2014-04-01 15:08 ` Dave Jones [this message]
2014-04-01 15:30 ` Paul E. McKenney
2014-04-01 17:22 ` Dave Jones
2014-04-01 17:55 ` Paul E. McKenney
2014-04-01 18:04 ` Dave Jones
2014-04-01 18:32 ` Paul E. McKenney
2014-04-01 22:16 ` Dave Jones
2014-04-01 23:18 ` Paul E. McKenney
2014-04-01 23:31 ` Dave Jones
2014-04-01 23:57 ` Paul E. McKenney
2014-04-02 0:07 ` Dave Jones
2014-04-02 16:20 ` Paul E. McKenney
2014-04-02 16:23 ` Dave Jones
2014-04-02 22:48 ` Dave Jones
2014-04-03 20:01 ` Dave Jones
2014-04-03 20:46 ` Paul E. McKenney
2014-04-03 21:44 ` Dave Jones
2014-04-03 22:37 ` Dave Jones
2014-04-04 17:06 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140401150849.GA14757@redhat.com \
--to=davej@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).