Re: hackbench regression with kernel 2.6.32-rc1

["Zhang, Yanmin" <yanmin_zhang@linux.intel.com>]

On Thu, 2009-10-29 at 06:46 +0100, Mike Galbraith wrote:
> On Thu, 2009-10-29 at 08:50 +0800, Zhang, Yanmin wrote:
> > On Wed, 2009-10-28 at 15:22 +0100, Mike Galbraith wrote:
> > > On Wed, 2009-10-28 at 17:29 +0800, Zhang, Yanmin wrote:
> > > > 	-Mike
> > > > I'm investigating 5% tbench regression on Nehalem machine. perf_counter shows
> > > > select_task_rq_fair consumes about 5% cpu time with 2.6.32-rc1 while it consumes
> > > > less than 0.5% with 2.6.31.
> > > > 
> > > > Patch c88d5910890 has comments to explain it, but I still can't understand why
> > > > to add complicated balance logic when selecting task rq.
> > > > 
> > > > I will check which section in function select_task_rq_fair consumes so much time.
> > > 
> > > Turn off SD_WAKE_BALANCE as it was called in rc1.  See commit 182a85f.
> > I run testing against 2.6.32-rc1 which already includes the patch.
> 
> Duh, I checked the wrong tree.
> 
> SD_PREFER_LOCAL is still on in rc1 though (double checks;), so you'll go
> through the power saving code until you reach a domain containing both
> waker's cpu and wakee's previous cpu even if that code already found
> that a higher domain wasn't overloaded.  Looks to me like that block
> wants a want_sd && qualifier. 
> 
> Even it you turn SD_PREFER_LOCAL off, you can still hit the overhead if
> SD_POWERSAVINGS_BALANCE is set, so I'd make sure both are off and see if
> that's the source (likely, since the rest is already off).
Yes. SD_POWERSAVINGS_BALANCE is disabled by default. I applied Peter's patch which
turning SD_PREFER_LOCAL off for MC and cpu domain and it doesn't help.
perf counter shows select_task_rq_fair still consumes about 5% cpu time. Eventually,
I found for_each_cpu in for_each_domain consumes the 5% cpu time, because Peter's
patch doesn't turn off SD_PREFER_LOCAL for node domain.
I turned it off for node domain against the latest tips tree and tbench regression
disappears on a Nehalem machine and becomes about 2% on another one.

Can we turn it off for node domain by default?