Re: [PATCH -rcu dev 3/3] RFC: rcu/tree: Read dynticks_nmi_nesting in advance

["Paul E. McKenney" <paulmck@linux.ibm.com>]

On Mon, Aug 19, 2019 at 10:22:08AM -0400, Joel Fernandes wrote:
> On Mon, Aug 19, 2019 at 02:59:08PM +0200, Frederic Weisbecker wrote:
> > On Fri, Aug 16, 2019 at 09:52:42AM -0700, Paul E. McKenney wrote:
> > > On Fri, Aug 16, 2019 at 12:24:04PM -0400, Joel Fernandes wrote:
> > > > On Thu, Aug 15, 2019 at 10:53:11PM -0400, Joel Fernandes (Google) wrote:
> > > > > I really cannot explain this patch, but without it, the "else if" block
> > > > > just doesn't execute thus causing the tick's dep mask to not be set and
> > > > > causes the tick to be turned off.
> > > > > 
> > > > > I tried various _ONCE() macros but the only thing that works is this
> > > > > patch.
> > > > > 
> > > > > Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> > > > > ---
> > > > >  kernel/rcu/tree.c | 3 ++-
> > > > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> > > > > index 856d3c9f1955..ac6bcf7614d7 100644
> > > > > --- a/kernel/rcu/tree.c
> > > > > +++ b/kernel/rcu/tree.c
> > > > > @@ -802,6 +802,7 @@ static __always_inline void rcu_nmi_enter_common(bool irq)
> > > > >  {
> > > > >  	struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
> > > > >  	long incby = 2;
> > > > > +	int dnn = rdp->dynticks_nmi_nesting;
> > > > 
> > > > I believe the accidental sign extension / conversion from long to int was
> > > > giving me an illusion since things started working well. Changing the 'int
> > > > dnn' to 'long dnn' gives similar behavior as without this patch! At least I
> > > > know now. Please feel free to ignore this particular RFC patch while I debug
> > > > this more (over the weekend or early next week). The first 2 patches are
> > > > good, just ignore this one.
> > > 
> > > Ah, good point on the type!  So you were ending up with zero due to the
> > > low-order 32 bits of DYNTICK_IRQ_NONIDLE being zero, correct?  If so,
> > > the "!rdp->dynticks_nmi_nesting" instead needs to be something like
> > > "rdp->dynticks_nmi_nesting == DYNTICK_IRQ_NONIDLE", which sounds like
> > > it is actually worse then the earlier comparison against the constant 2.
> > > 
> > > Sounds like I should revert the -rcu commit 805a16eaefc3 ("rcu: Force
> > > nohz_full tick on upon irq enter instead of exit").
> > 
> > I can't find that patch so all I can say so far is that its title doesn't
> > inspire me much. Do you still need that change for some reason?
> 
> No we don't need it. Paul's dev branch fixed it by checking DYNTICK_IRQ_NONIDLE:
> https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git/commit/?h=dev&id=227482fd4f3ede0502b586da28a59971dfbac0b0

Ah, so you have tested reverting this?  If so, thank you very much!

							Thanx, Paul