Re: [PATCH 2/2] smp_call_function: use rwlocks on queues rather than rcu

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Mon, Aug 25, 2008 at 12:31:31PM +0200, Peter Zijlstra wrote:
> On Fri, 2008-08-22 at 13:53 -0700, Paul E. McKenney wrote:
> > On Fri, Aug 22, 2008 at 03:03:13PM -0500, Christoph Lameter wrote:
> > > Paul E. McKenney wrote:
> > > 
> > > > I was indeed thinking in terms of the free from RCU being specially marked.
> > > 
> > > Isnt there some way to shorten the rcu periods significantly? Critical
> > > sections do not take that long after all.
> > 
> > In theory, yes.  However, the shorter the grace period, the greater the
> > per-update overhead of grace-period detection -- the general approach
> > is to use a per-CPU high-resolution timer to force RCU grace period
> > processing every 100 microseconds or so.  
> 
> You could of course also drive the rcu state machine from
> rcu_read_unlock().

True, and Jim Houston implemented something similar to this some years
back: http://marc.theaimsgroup.com/?l=linux-kernel&m=109387402400673&w=2

This of course greatly increases rcu_read_unlock() overhead.  But
perhaps it is a good implementation for the workloads that Christoph is
thinking of.

> > Also, by definition, the RCU
> > grace period can be no shorter than the longest active RCU read-side
> > critical section.  Nevertheless, I have designed my current hierarchical
> > RCU patch with expedited grace periods in mind, though more for the
> > purpose of reducing latency of long strings of operations that involve
> > synchronize_rcu() than for cache locality.
> 
> Another thing that could be done is more often force a grace period by
> flipping the counters.

Yep.  That is exactly what I was getting at with the high-resolution
timer point above.  This seems to be a reasonable compromise, as it
allows someone to specify how quickly the grace periods happen
dynamically.

But I am not sure that this gets the grace periods to go fast enough to
cover Christoph's use case -- he seems to be in a "faster is better"
space rather than in an "at least this fast" space.  Still, it would
likely help in some important cases.

							Thanx, Paul