All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Tony Lindgren <tony@atomide.com>, Paul Walmsley <paul@pwsan.com>,
	"Bruce, Becky" <bbruce@ti.com>,
	"Paul E. McKenney" <paul.mckenney@linaro.org>,
	"<linux-kernel@vger.kernel.org>" <linux-kernel@vger.kernel.org>,
	"<linux-omap@vger.kernel.org>" <linux-omap@vger.kernel.org>,
	"<linux-arm-kernel@lists.infradead.org>"
	<linux-arm-kernel@lists.infradead.org>,
	"Hilman, Kevin" <khilman@ti.com>,
	"Shilimkar, Santosh" <santosh.shilimkar@ti.com>,
	"Hunter, Jon" <jon-hunter@ti.com>,
	"<snijsure@grid-net.com>" <snijsure@grid-net.com>
Subject: Re: rcu self-detected stall messages on OMAP3, 4 boards
Date: Sat, 22 Sep 2012 09:00:31 -0700	[thread overview]
Message-ID: <20120922160031.GC2934@linux.vnet.ibm.com> (raw)
In-Reply-To: <CAFTL4hy5wO9CL5_1e8U7L3Tihm_HYD6uAsda3ue-zDKFjir2Xg@mail.gmail.com>

On Sat, Sep 22, 2012 at 05:45:12PM +0200, Frederic Weisbecker wrote:
> 2012/9/22 Paul E. McKenney <paulmck@linux.vnet.ibm.com>:
> > On Fri, Sep 21, 2012 at 01:31:49PM -0700, Tony Lindgren wrote:
> >> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> [120921 12:58]:
> >> >
> >> > Just to make sure I understand the combinations:
> >> >
> >> > o   All stalls have happened when running a minimal userspace.
> >> > o   CONFIG_NO_HZ=n suppresses the stalls.
> >> > o   CONFIG_RCU_FAST_NO_HZ (which depends on CONFIG_NO_HZ=y) has
> >> >     no observable effect on the stalls.
> >>
> >> The reason why you may need minimal userspace is to cut down
> >> the number of timers waking up the system with NO_HZ.
> >> Booting with init=/bin/sh might also do the trick for that.
> >
> > Good point!  This does make for a very quiet system, but does not
> > reproduce the problem under kvm, even after waiting for four minutes.
> > I will leave it for more time, but it looks like I really might need to
> > ask Linaro for remote access to a Panda.
> 
> I have one. I'm currently installing Ubuntu on it and I'll try to
> manage to build
> a kernel and reproduce the issue.
> 
> I'll give more news soon.

Thank you!

My bet is that you have to have a userspace that is so small that it
registers only a few (but at least one!) RCU callback at boot time,
then never registers any callbacks ever again.  I have coded up a
crude test case, using Tony Lindgren's suggestion of "init=/bin/sh",
but I appear to have inadvertently fixed this bug in current -rcu
(git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git,
branch rcu/next).

But I have been wrong a few times already on this particular bug...

							Thanx, Paul

WARNING: multiple messages have this Message-ID (diff)
From: paulmck@linux.vnet.ibm.com (Paul E. McKenney)
To: linux-arm-kernel@lists.infradead.org
Subject: rcu self-detected stall messages on OMAP3, 4 boards
Date: Sat, 22 Sep 2012 09:00:31 -0700	[thread overview]
Message-ID: <20120922160031.GC2934@linux.vnet.ibm.com> (raw)
In-Reply-To: <CAFTL4hy5wO9CL5_1e8U7L3Tihm_HYD6uAsda3ue-zDKFjir2Xg@mail.gmail.com>

On Sat, Sep 22, 2012 at 05:45:12PM +0200, Frederic Weisbecker wrote:
> 2012/9/22 Paul E. McKenney <paulmck@linux.vnet.ibm.com>:
> > On Fri, Sep 21, 2012 at 01:31:49PM -0700, Tony Lindgren wrote:
> >> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> [120921 12:58]:
> >> >
> >> > Just to make sure I understand the combinations:
> >> >
> >> > o   All stalls have happened when running a minimal userspace.
> >> > o   CONFIG_NO_HZ=n suppresses the stalls.
> >> > o   CONFIG_RCU_FAST_NO_HZ (which depends on CONFIG_NO_HZ=y) has
> >> >     no observable effect on the stalls.
> >>
> >> The reason why you may need minimal userspace is to cut down
> >> the number of timers waking up the system with NO_HZ.
> >> Booting with init=/bin/sh might also do the trick for that.
> >
> > Good point!  This does make for a very quiet system, but does not
> > reproduce the problem under kvm, even after waiting for four minutes.
> > I will leave it for more time, but it looks like I really might need to
> > ask Linaro for remote access to a Panda.
> 
> I have one. I'm currently installing Ubuntu on it and I'll try to
> manage to build
> a kernel and reproduce the issue.
> 
> I'll give more news soon.

Thank you!

My bet is that you have to have a userspace that is so small that it
registers only a few (but at least one!) RCU callback at boot time,
then never registers any callbacks ever again.  I have coded up a
crude test case, using Tony Lindgren's suggestion of "init=/bin/sh",
but I appear to have inadvertently fixed this bug in current -rcu
(git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git,
branch rcu/next).

But I have been wrong a few times already on this particular bug...

							Thanx, Paul

  reply	other threads:[~2012-09-22 16:00 UTC|newest]

Thread overview: 101+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-09-12 22:51 rcu self-detected stall messages on OMAP3, 4 boards Paul Walmsley
2012-09-12 22:51 ` Paul Walmsley
2012-09-13  1:12 ` Paul E. McKenney
2012-09-13  1:12   ` Paul E. McKenney
2012-09-13 18:52   ` Paul Walmsley
2012-09-13 18:52     ` Paul Walmsley
2012-09-20  0:03     ` Paul E. McKenney
2012-09-20  0:03       ` Paul E. McKenney
2012-09-20  0:03       ` Paul E. McKenney
2012-09-20  7:56       ` Paul Walmsley
2012-09-20  7:56         ` Paul Walmsley
2012-09-20 15:03         ` Bruce, Becky
2012-09-20 15:03           ` Bruce, Becky
2012-09-20 21:49         ` Bruce, Becky
2012-09-20 21:49           ` Bruce, Becky
2012-09-20 22:01           ` Paul E. McKenney
2012-09-20 22:01             ` Paul E. McKenney
2012-09-20 22:01             ` Paul E. McKenney
2012-09-20 22:47             ` Paul Walmsley
2012-09-20 22:47               ` Paul Walmsley
2012-09-20 23:21               ` Paul E. McKenney
2012-09-20 23:21                 ` Paul E. McKenney
2012-09-20 23:21                 ` Paul E. McKenney
2012-09-21 18:08                 ` Paul Walmsley
2012-09-21 18:08                   ` Paul Walmsley
2012-09-21 18:58                   ` Paul E. McKenney
2012-09-21 18:58                     ` Paul E. McKenney
2012-09-21 19:11                     ` Paul Walmsley
2012-09-21 19:11                       ` Paul Walmsley
2012-09-21 19:57                       ` Paul E. McKenney
2012-09-21 19:57                         ` Paul E. McKenney
2012-09-21 20:31                         ` Tony Lindgren
2012-09-21 20:31                           ` Tony Lindgren
2012-09-21 22:03                           ` Paul E. McKenney
2012-09-21 22:03                             ` Paul E. McKenney
2012-09-22 15:45                             ` Frederic Weisbecker
2012-09-22 15:45                               ` Frederic Weisbecker
2012-09-22 16:00                               ` Paul E. McKenney [this message]
2012-09-22 16:00                                 ` Paul E. McKenney
2012-09-21 22:12                         ` Paul E. McKenney
2012-09-21 22:12                           ` Paul E. McKenney
2012-09-22 18:42                         ` Paul Walmsley
2012-09-22 18:42                           ` Paul Walmsley
2012-09-22 20:10                           ` Paul E. McKenney
2012-09-22 20:10                             ` Paul E. McKenney
2012-09-22 21:59                             ` Paul E. McKenney
2012-09-22 21:59                               ` Paul E. McKenney
2012-09-22 22:25                               ` Paul Walmsley
2012-09-22 22:25                                 ` Paul Walmsley
2012-09-22 23:11                                 ` Paul E. McKenney
2012-09-22 23:11                                   ` Paul E. McKenney
2012-09-22 23:11                                   ` Paul E. McKenney
2012-09-23  7:55                                   ` Paul Walmsley
2012-09-23  7:55                                     ` Paul Walmsley
2012-09-23  7:55                                     ` Paul Walmsley
2012-09-23 12:11                                     ` Paul E. McKenney
2012-09-23 12:11                                       ` Paul E. McKenney
2012-09-23 12:11                                       ` Paul E. McKenney
2012-09-23  1:42                                 ` Paul Walmsley
2012-09-23  1:42                                   ` Paul Walmsley
2012-09-23  1:56                                   ` Paul E. McKenney
2012-09-23  1:56                                     ` Paul E. McKenney
2012-09-23  1:56                                     ` Paul E. McKenney
2012-09-23  2:01                                     ` Paul Walmsley
2012-09-23  2:01                                       ` Paul Walmsley
2012-09-24  9:41                               ` Shilimkar, Santosh
2012-09-24  9:41                                 ` Shilimkar, Santosh
2012-09-24 13:18                                 ` Paul E. McKenney
2012-09-24 13:18                                   ` Paul E. McKenney
2012-10-01  8:55                               ` Linus Walleij
2012-10-01  8:55                                 ` Linus Walleij
2012-10-01 13:28                                 ` Paul E. McKenney
2012-10-01 13:28                                   ` Paul E. McKenney
2012-09-21 18:59                   ` Paul Walmsley
2012-09-21 18:59                     ` Paul Walmsley
2012-09-21 17:47               ` Paul Walmsley
2012-09-21 17:47                 ` Paul Walmsley
2012-09-21 17:51                 ` Paul Walmsley
2012-09-21 17:51                   ` Paul Walmsley
2012-09-21 21:20                 ` Paul E. McKenney
2012-09-21 21:20                   ` Paul E. McKenney
2012-09-21 21:20                   ` Paul E. McKenney
2012-09-21 22:41                   ` Paul Walmsley
2012-09-21 22:41                     ` Paul Walmsley
2012-09-22  0:05                     ` Paul E. McKenney
2012-09-22  0:05                       ` Paul E. McKenney
2012-09-22 18:16                       ` Paul Walmsley
2012-09-22 18:16                         ` Paul Walmsley
2012-09-22 18:16                         ` Paul Walmsley
2012-09-22 19:52                         ` Paul E. McKenney
2012-09-22 19:52                           ` Paul E. McKenney
2012-09-22 19:52                           ` Paul E. McKenney
2012-09-22 22:20                           ` Paul Walmsley
2012-09-22 22:20                             ` Paul Walmsley
2012-09-22 22:20                             ` Paul Walmsley
2012-09-22 23:17                             ` Paul E. McKenney
2012-09-22 23:17                               ` Paul E. McKenney
2012-09-24 21:54                               ` Paul Walmsley
2012-09-24 21:54                                 ` Paul Walmsley
2012-09-24 22:00                                 ` Paul E. McKenney
2012-09-24 22:00                                   ` Paul E. McKenney

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20120922160031.GC2934@linux.vnet.ibm.com \
    --to=paulmck@linux.vnet.ibm.com \
    --cc=bbruce@ti.com \
    --cc=fweisbec@gmail.com \
    --cc=jon-hunter@ti.com \
    --cc=khilman@ti.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-omap@vger.kernel.org \
    --cc=paul.mckenney@linaro.org \
    --cc=paul@pwsan.com \
    --cc=santosh.shilimkar@ti.com \
    --cc=snijsure@grid-net.com \
    --cc=tony@atomide.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.