From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: linux-kernel@vger.kernel.org
Subject: Re: [PATCH] rcu: restore correct batch limiting
Date: Thu, 18 Oct 2012 04:58:59 -0700 [thread overview]
Message-ID: <20121018115859.GH2518@linux.vnet.ibm.com> (raw)
In-Reply-To: <1350458052.26103.31.camel@edumazet-glaptop>
On Wed, Oct 17, 2012 at 09:14:12AM +0200, Eric Dumazet wrote:
> From: Eric Dumazet <edumazet@google.com>
>
> Commit 29c00b4a1d9e27 (rcu: Add event-tracing for RCU callback
> invocation) added a regression in rcu_do_batch()
>
> Under stress, RCU is supposed to allow to process all items in queue,
> instead of a batch of 10 items (blimit), but an integer overflow makes
> the effective limit being 1.
>
> So RCU cannot recover and machine eventually crash because of OOM.
>
> Using long instead of int is not really needed, convert everything
> to integers.
<facepalm>
Good catch!!!
The reason for favoring long over int is that there are a few systems out
there with terabytes of main memory. In addition, there have been a few
bugs over the past few years that could result in RCU CPU stalls of more
than a minute. This makes it impossible to rule out the possibility of
a billion callbacks appearing on one CPU.
So, does the following patch fix things, or a I still confused?
Thanx, Paul
------------------------------------------------------------------------
rcu: Fix batch-limit size problem
Commit 29c00b4a1d9e27 (rcu: Add event-tracing for RCU callback
invocation) added a regression in rcu_do_batch()
Under stress, RCU is supposed to allow to process all items in queue,
instead of a batch of 10 items (blimit), but an integer overflow makes
the effective limit being 1. So, unless there is frequent idle periods
(during which RCU ignores batch limits), RCU can be forced into a
state where it cannot keep up with the callback-generation rate,
eventually resulting in OOM.
This commit therefore converts a few variables in rcu_do_batch() from
int to long to fix this problem.
Reported-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index e36d085..e056e1e 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -1823,7 +1823,8 @@ static void rcu_do_batch(struct rcu_state *rsp, struct rcu_data *rdp)
{
unsigned long flags;
struct rcu_head *next, *list, **tail;
- int bl, count, count_lazy, i;
+ long bl, count, count_lazy;
+ int i;
/* If no callbacks are ready, just return.*/
if (!cpu_has_callbacks_ready_to_invoke(rdp)) {
next prev parent reply other threads:[~2012-10-18 11:59 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-10-17 7:14 [PATCH] rcu: restore correct batch limiting Eric Dumazet
2012-10-17 13:18 ` Hillf Danton
2012-10-17 14:01 ` Eric Dumazet
2012-10-18 11:58 ` Paul E. McKenney [this message]
2012-10-18 12:44 ` Eric Dumazet
2012-10-18 15:25 ` Paul E. McKenney
2012-10-18 16:10 ` Eric Dumazet
2012-10-18 16:35 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20121018115859.GH2518@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=eric.dumazet@gmail.com \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox