Re: hackbench regression with kernel 2.6.32-rc1

[Mike Galbraith <efault@gmx.de>]

On Tue, 2009-10-13 at 11:39 +0200, Mike Galbraith wrote:
> On Tue, 2009-10-13 at 11:12 +0800, Zhang, Yanmin wrote:
> 
> > NEXT_BUDDY has no help on volanoMark and tbench.
> 
> Vmark is mostly about preemption and affinity.  Increases in wakeup
> preemption or load balancing will bring it down.  The affinity bit
> applies heavily to mysql+oltp too, though it loves wakeup preemption.
> 
> test test test...
> 
> My conclusion for results _here_ is load balancing changes are harming
> cache wise.  GENTLE_FAIR_SLEEPERS harms short term fairness, but despite
> GENTLE_FAIR_SLEEPERS, we're still wakeup preempting too much, likely
> doing more cache harm.
> 
> (even for the desktop, overly aggressive wakeup preemption can do harm)
> 
> I'd suggest trying the settings below.

Except don't bother tweaking min_granularity.  Further testing showed
that's fine.  So turn off GENTLE_FAIR_SLEEPERS, and bump
wakeup_granularity up a bit.

Note: don't bump it further than the short term fairness goal
(sched_latency, or half of that if gentle is enabled), or you won't have
any wakeup preemption, which is deadly for many loads. 

> vmark 
> 
>  92143 stock settings
>  96396 -SD_BALANCE_NEWIDLE
>  99403 -SD_WAKE_BALANCE
> 121821 min_granularity+wakeup_granularity *= 2
> 128795 NO_WAKEUP_PREEMPT (and back on, just checking fairness)
>  97193 NO_FAIR_SLEEPERS (vmark likes fairness, max out fairness)
> 107519 FAIR_SLEEPERS NO_GENTLE_FAIR_SLEEPERS (over-preempt again, so..)
> 123721 min_granularity+wakeup_granularity *= 2
> 131290 NO_WAKEUP_PREEMPT (and back on, just checking fairness)
> 123464 NEXT_BUDDY (no effect)
> vs stock 1.339
> 
> tbench 8 with these settings.
> 
> 752.249 MB/sec 8 procs
> 747.010 MB/sec 8 procs NO_NEXT_BUDDY
> 753.177 MB/sec 8 procs min_granularity+wakeup_granularity /= 2
> 749.518 MB/sec 8 procs GENTLE_FAIR_SLEEPERS
> 753.051 MB/sec 8 procs min_granularity+wakeup_granularity /= 2
> 734.772 MB/sec 8 procs +SD_WAKE_BALANCE
> 733.683 MB/sec 8 procs +SD_BALANCE_NEWIDLE (we are at stock)
> vs stock 1.025
> 
> Turns off netfilter, stock settings
> 
> 903.304 MB/sec 8 procs
> 900.656 MB/sec 8 procs -SD_BALANCE_NEWIDLE
> 928.914 MB/sec 8 procs -SD_WAKE_BALANCE
> 930.591 MB/sec 8 procs min_granularity+wakeup_granularity *= 2
> 926.836 MB/sec 8 procs NO_GENTLE_FAIR_SLEEPERS
> 931.148 MB/sec 8 procs min_granularity+wakeup_granularity *= 2
> vs stock 1.030
> 
> vmark
> 
> 146264
> 116559 stock
> vs stock 1.254