From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751955Ab0KYJ1P (ORCPT ); Thu, 25 Nov 2010 04:27:15 -0500 Received: from mail-fx0-f46.google.com ([209.85.161.46]:41168 "EHLO mail-fx0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751146Ab0KYJ1N (ORCPT ); Thu, 25 Nov 2010 04:27:13 -0500 DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=cKv0iSLhqYasABxRJ/O5j3KA44kGjllhd72GHsMn4sI2O/HFyoNJ1bs/iGoig/X4Mt sVIOJeW1dlwMAr/QotBkjhWjz39SfTLQ4TyTJ8bfLGRgjXCmpszrpxdF0qALIXUqEHPt 6r5OltAZIdDnmmrJyEukTVArzo46rk8MClY4I= Date: Thu, 25 Nov 2010 10:27:08 +0100 From: Frederic Weisbecker To: Lai Jiangshan Cc: "Paul E. McKenney" , LKML , Thomas Gleixner , Peter Zijlstra , Steven Rostedt , Ingo Molnar Subject: Re: [PATCH 0/2] rcu: Fix series of spurious RCU softirqs Message-ID: <20101125092706.GH2538@nowhere> References: <1290558673-23580-1-git-send-crap-fweisbec@gmail.com> <4CEDDB2A.2020807@cn.fujitsu.com> <20101125073857.GB2538@nowhere> <4CEE1FD2.7040707@cn.fujitsu.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4CEE1FD2.7040707@cn.fujitsu.com> User-Agent: Mutt/1.5.20 (2009-06-14) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Nov 25, 2010 at 04:35:30PM +0800, Lai Jiangshan wrote: > On 11/25/2010 03:38 PM, Frederic Weisbecker wrote: > > On Thu, Nov 25, 2010 at 11:42:34AM +0800, Lai Jiangshan wrote: > >> On 11/24/2010 08:31 AM, Frederic Weisbecker wrote: > >>> Hi, > >>> > >>> I've observed some not so unfrequent series of spurious rcu > >>> softirqs, sometimes happening at each ticks for a random > >>> while. > >>> > >>> These patches aims at fixing them. > >>> > >>> Thanks. > >>> > >>> Frederic Weisbecker (2): > >>> rcu: Don't chase unnecessary quiescent states after extended grace periods > >>> rcu: Stop checking quiescent states after grace period completion from remote > >>> > >> > >> If we ensure rdp->gpnum >= rdp->completed is always true, the problems as > >> you described will not be existed. Or maybe I misunderstand you. > >> > >> rdp->gpnum >= rdp->completed is a very important guarantee I think. > >> (In my RCURING, it is guaranteed.) I'm afraid there are some other > >> problems still hidden if it is not guaranteed. > >> > >> so I recommend: (code is better than words) > >> > >> diff --git a/kernel/rcutree.c b/kernel/rcutree.c > >> index d5bc439..af4e87a 100644 > >> --- a/kernel/rcutree.c > >> +++ b/kernel/rcutree.c > >> @@ -648,6 +648,13 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat > >> > >> /* Remember that we saw this grace-period completion. */ > >> rdp->completed = rnp->completed; > >> + > >> + /* Ensure ->gpnum >= ->completed after NO_HZ */ > >> + if (unlikely(rnp->completed - rdp->gpnum > 0 > >> + || rdp->gpnum - rnp->gpnum > 0)) { > >> + rdp->gpnum = rnp->completed; > >> + rdp->qs_pending = 0; > > > > > > That's an alternative to my first patch yeah. > > Since rdp->gpnum >= rdp->completed is guaranteed. > your second patch is not needed, the problem is also fixed. > > if rnp->gpnum == rnp->completed, rcu_report_qs_rdp() will not be called. > it is because rdp->qs_pending == 0 when rnp->gpnum == rnp->completed. Aaah... > > And if rdp->gpnum >= rdp->completed > > must be a guarantee outside the rnp lock, then it's certainly better because > > the lock is relaxed between rcu_process_gp_end() and note_new_gpnum(), and > > both values are async in this lockless frame. > > > > But perhaps this shouldn't touch rdp->qs_pending: > > if rdp->gpnum == rnp->completed, it means we don't need a qs for rdp->gpnum, > it is completed. so we must set rdp->qs_pending = 0; > > when we really need a qs, rdp->qs_pending will be fixed in note_new_gp_new(). Ok that makes all sense now! I'm just not sure about your check above. (rdp->gpnum - rnp->gpnum > 0) can never happen, right? Also perhaps we should set rdp->qs_pending = 0 only if rnp->completed == rdp->completed? Which in the end would be: /* Remember that we saw this grace-period completion. */ rdp->completed = rnp->completed; + if (rdp->gpnum < rdp->completed) + rdp->gpnum = rdp->completed; + + if (rdp->gpnum == rdp->completed) + rdp->qs_pending = 0; And then if there is a new grace period to handle, this will be done through note_new_pgnum(). Hm?