Re: [RFC&PATCH] Alternative RCU implementation

[Jim Houston <jim.houston@comcast.net>]

On Mon, 2004-08-30 at 14:52, Paul E. McKenney wrote: 
> On Mon, Aug 30, 2004 at 01:13:41PM -0400, Jim Houston wrote:
> > I know that I'm questioning one of your design goals for RCU by adding
> > overhead to the read-side.  I have read everything I could find on RCU.
> > My belief is that the cost of the xchg() instruction is small 
> > compared to the cache benifit of freeing memory more quickly.
> > I think it's more interesting to look at the impact of the xchg() at the
> > level of an entire system call.  Adding 30 nanoseconds to a open/close
> > path that tasks 3 microseconds seems reasonable.  It is hard to measure
> > the benefit of reusing the a dcache entry more quickly.
> 
> Hello, Jim,
> 
> The other thing to keep in mind is that reducing the grace-period
> duration increases the per-access overhead, since each grace period
> incurs a cost.  So there is a balance that needs to be struck between
> overflowing memory with a too-long grace period and incurring too
> much overhead with a too-short grace period.
> 
> How does the rest of the kernel work with all interrupts to
> a particular CPU shut off?  For example, how do you timeslice?
> 
> 						Thanx, Paul
> 
> PS.  My concerns with some aspects of your design aside, your
>      getting a significant change to the RCU infrastructure to
>      work reasonably well is quite impressive!

Hi Paul,

I have two module parameters in the patch which can be used to
tune how often grace periods are started.  They can be set at boot
time as follows:

rcupdate.rcu_max_count=#
	The per-cpu count of queued requests at which to
	start a new batch.  Patch defaults to 256.

rcupdate.rcu_max_time=#
	Timeout value in jiffies at which to start a batch. 
	Defaults to HZ/10.

I picked the defaults to start batches with similar frequency to
the existing code.

I tested a dual processor with rcupdate.rcu_max_count=0. This
will start a grace period for every call_rcu(). I ran
my rename test this way and it worked suprisingly well.

I maintain a nxtbatch value which lets me check if the grace period
for the entries in the nxt list has started or perhaps already
completed.  I check this in call_rcu() and avoid mixing batches.
Any requests queued before the batch was started will be completed
at the end of the grace period.  Unless a very small rcu_max_count
value is used, there is likely to be some delay between completing
a grace period and needing to start another.


> How does the rest of the kernel work with all interrupts to
> a particular CPU shut off?  For example, how do you timeslice?

It's a balancing act.  In some cases we just document the
missing functionality.  If the local timer is disabled on a cpu,
all processes are SCHED_FIFO.  In the case of Posix timers, we
move timers to honor the procesor shielding an the process affinity.

Jim Houston