Re: sched : performance regression 24% between 4.4rc4 and 4.3 kernel

[Mike Galbraith <umgwanakikbuti@gmail.com>]

(it's always a good idea to CC subsystem maintainers when reporting) 

On Fri, 2015-12-11 at 15:17 +0100, Jirka Hladky wrote:
> Hello,
> 
> we are doing performance testing of the new kernel scheduler (commit
> 53528695ff6d8b77011bc818407c13e30914a946). In most cases we see
> performance improvements compared to 4.3 kernel with the exception of
> stream benchmark when running on 4 NUMA node server.
> 
> When we run 4 stream benchmark processes on 4 NUMA node server and we
> compare the total performance we see drop about 24% compared to 4.3
> kernel. This is caused by the fact that 2 stream benchmarks are
> running on the same NUMA node while 1 NUMA node does not run any
> stream benchmark. With kernel 4.3, load is distributed evenly among
> all 4 NUMA nodes. When two stream benchmarks are running on the same
> NUMA node then the runtime is almost twice as long compared to one
> stream bench running on one NUMA node. See log files [1] bellow.
> 
> Please see the graph comparing stream benchmark results between
> kernel
> 4.3 and 4.4rc4 (for legend see [2] bellow).
> https://jhladky.fedorapeople.org/sched_stream_kernel_4.3vs4.4rc4/Stre
> am_benchmark_on_4_NUMA_node_server_4.3vs4.4rc4_kernel.png
> 
> Could you please help us to identify the root cause of this
> regression? We don't have the skills to fix the problem ourselves but
> we will be more than happy to test any proposed patch for this issue.

A bisection doesn't require any special skills, but may give busy
maintainers a single change to eyeball vs the entire lot. 

	-Mike