From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Nicholas Piggin <npiggin@gmail.com>
Cc: Abdul Haleem <abdhalee@linux.vnet.ibm.com>,
sachinp <sachinp@linux.vnet.ibm.com>,
Stephen Rothwell <sfr@canb.auug.org.au>,
Paul McKenney <Paul.McKenney@us.ibm.com>,
linux-kernel <linux-kernel@vger.kernel.org>,
linux-next <linux-next@vger.kernel.org>,
paulus@samba.org, linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Subject: Re: cpus stalls detected few hours after booting next kernel
Date: Fri, 30 Jun 2017 11:15:11 -0700 [thread overview]
Message-ID: <20170630181511.GC2393@linux.vnet.ibm.com> (raw)
In-Reply-To: <20170630172802.6d4ddca2@roar.ozlabs.ibm.com>
On Fri, Jun 30, 2017 at 05:28:02PM +1000, Nicholas Piggin wrote:
> On Fri, 30 Jun 2017 10:52:18 +0530
> Abdul Haleem <abdhalee@linux.vnet.ibm.com> wrote:
>
> > On Fri, 2017-06-30 at 00:45 +1000, Nicholas Piggin wrote:
> > > On Thu, 29 Jun 2017 20:23:05 +1000
> > > Nicholas Piggin <npiggin@gmail.com> wrote:
> > >
> > > > On Thu, 29 Jun 2017 19:36:14 +1000
> > > > Nicholas Piggin <npiggin@gmail.com> wrote:
> > >
> > > > > I don't *think* the replay-wakeup-interrupt patch is directly involved, but
> > > > > it's likely to be one of the idle patches.
> > >
> > > Okay this turned out to be misconfigured sleep states I added for the
> > > simulator, sorry for the false alarm.
> > >
> > > > Although you have this in the backtrace. I wonder if that's a stuck
> > > > lock in rcu_process_callbacks?
> > >
> > > So this spinlock becomes top of the list of suspects. Can you try
> > > enabling lockdep and try to reproduce it?
> >
> > Yes, recreated again with CONFIG_LOCKDEP=y & CONFIG_DEBUG_LOCKDEP=y set.
> > I do not see any difference in trace messages with and without LOCKDEP
> > enabled.
> >
> > Please find the attached log file.
>
> Can you get an rcu_invoke_callback event trace that Paul suggested?
>
> Does this bug show up with just the powerpc next branch?
And I must say that those RCU CPU stall warnings are spectacular!
Did you by chance boot with a small subset of the CPUs online, and
bring the rest online later on?
Thanx, Paul
WARNING: multiple messages have this Message-ID (diff)
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Nicholas Piggin <npiggin@gmail.com>
Cc: Abdul Haleem <abdhalee@linux.vnet.ibm.com>,
sachinp <sachinp@linux.vnet.ibm.com>,
Stephen Rothwell <sfr@canb.auug.org.au>,
Paul McKenney <Paul.McKenney@us.ibm.com>,
linux-kernel <linux-kernel@vger.kernel.org>,
linux-next <linux-next@vger.kernel.org>,
paulus@samba.org, linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Subject: Re: [linux-next] cpus stalls detected few hours after booting next kernel
Date: Fri, 30 Jun 2017 11:15:11 -0700 [thread overview]
Message-ID: <20170630181511.GC2393@linux.vnet.ibm.com> (raw)
In-Reply-To: <20170630172802.6d4ddca2@roar.ozlabs.ibm.com>
On Fri, Jun 30, 2017 at 05:28:02PM +1000, Nicholas Piggin wrote:
> On Fri, 30 Jun 2017 10:52:18 +0530
> Abdul Haleem <abdhalee@linux.vnet.ibm.com> wrote:
>
> > On Fri, 2017-06-30 at 00:45 +1000, Nicholas Piggin wrote:
> > > On Thu, 29 Jun 2017 20:23:05 +1000
> > > Nicholas Piggin <npiggin@gmail.com> wrote:
> > >
> > > > On Thu, 29 Jun 2017 19:36:14 +1000
> > > > Nicholas Piggin <npiggin@gmail.com> wrote:
> > >
> > > > > I don't *think* the replay-wakeup-interrupt patch is directly involved, but
> > > > > it's likely to be one of the idle patches.
> > >
> > > Okay this turned out to be misconfigured sleep states I added for the
> > > simulator, sorry for the false alarm.
> > >
> > > > Although you have this in the backtrace. I wonder if that's a stuck
> > > > lock in rcu_process_callbacks?
> > >
> > > So this spinlock becomes top of the list of suspects. Can you try
> > > enabling lockdep and try to reproduce it?
> >
> > Yes, recreated again with CONFIG_LOCKDEP=y & CONFIG_DEBUG_LOCKDEP=y set.
> > I do not see any difference in trace messages with and without LOCKDEP
> > enabled.
> >
> > Please find the attached log file.
>
> Can you get an rcu_invoke_callback event trace that Paul suggested?
>
> Does this bug show up with just the powerpc next branch?
And I must say that those RCU CPU stall warnings are spectacular!
Did you by chance boot with a small subset of the CPUs online, and
bring the rest online later on?
Thanx, Paul
next prev parent reply other threads:[~2017-06-30 18:15 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-06-29 9:21 [linux-next] cpus stalls detected few hours after booting next kernel Abdul Haleem
2017-06-29 9:36 ` Nicholas Piggin
2017-06-29 9:36 ` [linux-next] " Nicholas Piggin
2017-06-29 10:23 ` Nicholas Piggin
2017-06-29 10:23 ` [linux-next] " Nicholas Piggin
2017-06-29 14:45 ` Nicholas Piggin
2017-06-29 14:45 ` [linux-next] " Nicholas Piggin
2017-06-29 15:47 ` Paul McKenney
2017-06-29 15:47 ` Paul McKenney
2017-06-30 5:22 ` Abdul Haleem
2017-06-30 7:28 ` Nicholas Piggin
2017-06-30 7:28 ` [linux-next] " Nicholas Piggin
2017-06-30 18:15 ` Paul E. McKenney [this message]
2017-06-30 18:15 ` Paul E. McKenney
2017-07-04 14:14 ` Abdul Haleem
2017-07-04 23:31 ` Paul E. McKenney
2017-07-04 23:31 ` [linux-next] " Paul E. McKenney
2017-07-07 11:06 ` Abdul Haleem
2017-07-25 5:19 ` Abdul Haleem
2017-07-25 5:19 ` [linux-next] " Abdul Haleem
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170630181511.GC2393@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=Paul.McKenney@us.ibm.com \
--cc=abdhalee@linux.vnet.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-next@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=npiggin@gmail.com \
--cc=paulus@samba.org \
--cc=sachinp@linux.vnet.ibm.com \
--cc=sfr@canb.auug.org.au \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.