Re: [PATCH] net: allow netdev_wait_allrefs() to run faster

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Sat, Oct 24, 2009 at 04:24:27PM +0200, Eric Dumazet wrote:
> Paul E. McKenney a écrit :
> > On Sat, Oct 24, 2009 at 10:49:55AM +0200, Eric Dumazet wrote:
> >>
> >> On my dev machine, a synchronize_rcu() lasts between 2 an 12 ms
> > 
> > That sounds like the right range, depending on what else is happening
> > on the machine at the time.
> > 
> > The synchronize_rcu_expedited() primitive would run in the 10s-100s
> > of microseconds.  It involves a pair of wakeups and a pair of context
> > switches on each CPU.
> 
> Hmm... I'll make some experiments Monday and post results, but it seems very
> promising.

I should hasten to add that synchronize_rcu_expedited() goes fast for
TREE_RCU but not yet for TREE_PREEMPT_RCU (where it maps safely but
slowly to synchronize_rcu()).

> Do you think the "on_each_cpu(flush_backlog, dev, 1);"
> we perform right before calling netdev_wait_allrefs() could be changed
> somehow to speedup rcu callbacks ? Maybe we ould avoid sending IPI twice to
> cpus ?

This is an interesting possibility, and might fit in with some of the
changes that I am thinking about to reduce OS jitter for the heavy-duty
numerical-computing guys.

In the meantime, you could try doing the following from flush_backlog():

	local_irq_save(flags);
	rcu_check_callbacks(smp_processor_id(), 0);
	local_irq_restore(flags);

This would emulate a much-faster HZ value, but only for RCU.  This works
better in TREE_RCU than it does in TREE_PREEMPT_RCU at the moment (on my
todo list!).  In older kernels, this should also work for CLASSIC_RCU.
Of course, in TINY_RCU, synchronize_rcu() is a no-op anyway.  ;-)

And just to be clear, synchronize_rcu_expedited() currently just does
wakeups, not explicit IPIs.

							Thanx, Paul