Re: RCU latency regression in 2.6.16-rc1

["Paul E. McKenney" <paulmck@us.ibm.com>]

On Tue, Jan 24, 2006 at 03:15:27AM -0500, Lee Revell wrote:
> On Tue, 2006-01-24 at 09:13 +0100, Ingo Molnar wrote:
> > * Lee Revell <rlrevell@joe-job.com> wrote:
> > 
> > > On Tue, 2006-01-24 at 09:01 +0100, Ingo Molnar wrote:
> > > > * Lee Revell <rlrevell@joe-job.com> wrote:
> > > > 
> > > > > On Tue, 2006-01-24 at 08:56 +0100, Ingo Molnar wrote:
> > > > > > * Lee Revell <rlrevell@joe-job.com> wrote:
> > > > > > 
> > > > > > > I ported the latency tracer to 2.6.16 and got this 13ms latency within 
> > > > > > > a few hours.  This is a regression from 2.6.15.
> > > > > > > 
> > > > > > > It appears that RCU can invoke ipv4_dst_destroy thousands of times in 
> > > > > > > a single batch.
> > > > > > 
> > > > > > could you try the PREEMPT_RCU patch below?
> > > > > 
> > > > > Sure.  If it works do you see this making it in 2.6.16?  Otherwise we 
> > > > > still would have a regression...
> > > > 
> > > > nope, that likely wont make v2.6.16, which is frozen already.
> > > > 
> > > 
> > > How about just lowering maxbatch to 1000?
> > 
> > does that fix the latency for you? I think "maxbatch=1000" should work 
> > as a boot parameter too.
> > 
> 
> Have not tested yet but it appears that will reduce it substantially:
> 
> $ grep "dst_destroy (dst_rcu_free)" /proc/latency_trace | wc -l
> 3027
> 
> This implies the latency would be reduced to ~4ms, still not great but
> it will be overshadowed by rt_run_flush/rt_garbage_collect.

The other patch to try would be Dipankar Sarma's patch at:

	http://marc.theaimsgroup.com/?l=linux-kernel&m=113657112726596&w=2

This patch was primarily designed to reduce memory overhead, but given
that it tends to reduce batch size, it should also reduce latency.

							Thanx, Paul