Volanomark regress on ia64

Volanomark regress on ia64

[Chen, Kenneth W]

[-- Attachment #1: Type: text/plain, Size: 799 bytes --]

Take a look at http://kernel-perf.sourceforge.net/results.machine_id=26.html

First chart with volanomark benchmark.  It took ~ 11% performance regression
on 2.6.13-git9 and 2.6.14-rc1.

It is not those two particular kernel version have the problem per se.  It
would've shown on 2.6.13-rc1 when default hz rate was switched to 250.  Last
week, I happened to audit the system and made the hz switch and the problem
showed up.

Hz=1000  cpu idle is about 54%, the raw throughput is 74935 msgs/sec
Hz=250   cpu idle is about 58%, the raw throughput is 67246 msgs/sec

I think it has something to do with the scheduler load balance.  While I'm
looking into schedstat and etc.  Posting here for everyone to pour over the
issue.

vmstat/readprofle/schedstat result with 1K hz and 250 hz attached.

- Ken

[-- Attachment #2: volano.tar.gz --]
[-- Type: application/x-gzip, Size: 21607 bytes --]

Re: Volanomark regress on ia64

[Nick Piggin]

Chen, Kenneth W wrote:
> Take a look at http://kernel-perf.sourceforge.net/results.machine_id&.html
> 
> First chart with volanomark benchmark.  It took ~ 11% performance regression
> on 2.6.13-git9 and 2.6.14-rc1.
> 
> It is not those two particular kernel version have the problem per se.  It
> would've shown on 2.6.13-rc1 when default hz rate was switched to 250.  Last
> week, I happened to audit the system and made the hz switch and the problem
> showed up.
> 
> Hz\x1000  cpu idle is about 54%, the raw throughput is 74935 msgs/sec
> Hz%0   cpu idle is about 58%, the raw throughput is 67246 msgs/sec
> 
> I think it has something to do with the scheduler load balance.  While I'm
> looking into schedstat and etc.  Posting here for everyone to pour over the
> issue.
> 
> vmstat/readprofle/schedstat result with 1K hz and 250 hz attached.
> 

Hi Ken,
Made any progress yet?

It looks like the problem is confined to ia64? Are the x86-64 machines
also running with HZ set to 250?

I would look at differences between SD_CPU_INIT for ia64 versus the
generic definition. cache_hot_time and cache_nice_tries by the looks,
both those could cause increased idle time.

Nick

-- 
SUSE Labs, Novell Inc.

Send instant messages to your online friends http://au.messenger.yahoo.com 

Re: Volanomark regress on ia64

[John Hawkes]

I am also seeing a degradation of 16% @48p and 9% @32p for my favorite AIM7
workload on ia64 (SGI SN/Altix) between 2.6.13-rc6 and 2.6.14-rc1.  I haven't
peeked beneath the covers to analyze why it's happening.  I've still been
gathering performance numbers.

John Hawkes

RE: Volanomark regress on ia64

[Chen, Kenneth W]

Nick Piggin wrote on Sunday, September 18, 2005 3:57 AM
> It looks like the problem is confined to ia64? Are the x86-64 machines
> also running with HZ set to 250?

Yes, it appears to be confined to ia64.  X86-64 machines are running at
250 Hz and there is no perf. regression there.


> I would look at differences between SD_CPU_INIT for ia64 versus the
> generic definition. cache_hot_time and cache_nice_tries by the looks,
> both those could cause increased idle time.

Thanks for the suggestion.  My first experiment was in those area as
well, in particular changing newidle_idx and idle_idx.  I have not yet
come up any concrete resolution.

- Ken