Re: -mm seems significanty slower than mainline on kernbench

[Martin Bligh <mbligh@google.com>]

Andrew Morton wrote:
> Martin Bligh <mbligh@google.com> wrote:
> 
>>OK, I fixed the graphs so you can actually read them now ;-)
> 
> They're cute.

Thanks. Hopefully we can also make them useful ;-)

Have finally got some time to play, and am working on a "push" 
comparison model that'll send you email pro-actively.

>>http://test.kernel.org/perf/kernbench.elm3b6.png (x86_64 4x)
>>http://test.kernel.org/perf/kernbench.moe.png (NUMA-Q)
>>http://test.kernel.org/perf/kernbench.elm3b132.png (4x SMP ia32)
>>
>>Both seems significantly slower on -mm (mm is green line)
> 
> 
> Well, 1% slower.  -mm has permanent not-for-linus debug things, some of
> which are expected to have a performance impact.  I don't know whether
> they'd have a 1% impact though.

OK, fair enough. Can I turn them off somehow to check? it's 10% on 
NUMA-Q. The good news is that it's stayed in -mm for long time, so ...
am praying.

Is cool to have extra debug stuff. It does make it harder to check perf 
though ... if we can do runs both with and without, it'd be ideal, I 
guess. I'd like to be able to spot perf degredations before they hit 
mainline.

>>If I look at diffprofile between 2.6.15 and 2.6.15-mm1, it just looks
>>like we have lots more idle time.
> 
> 
> Yes, we do.   It'd be useful to test -git7..

Will do. it does all of them.

>>You got strange scheduler changes in
>>there, that you've been carrying for a long time (2.6.14-mm1 at least)? 
>>or HZ piddling? See to be mainly getting much more idle time.
> 
> Yes, there are CPU scheduler changes, although much fewer than usual. 
> Ingo, any suggestions as to a culprit?

I'd truncated all -mm info in the filtering before 2.6.14 .. am putting 
it back so we can see clearly ... done. Look again.

Seems to have gone wrong between 2.6.14-rc1-mm1 and 2.6.14-rc2-mm1 ?
See http://test.kernel.org/perf/kernbench.moe.png for clearest effect.

M.