same application has different performance with kernel 2.6.32/34/36

same application has different performance with kernel 2.6.32/34/36

[zhou rui]

hi
my test application add a packet hook via  dev_add_pack(&prot_hook);
then copy the packet to a shared memory between userspace and kernel

tests wereperformed at same machine(DL380 G7, 12cores, intel X520 10G
nic),same traffic,same nic driver(ixgbe 3.3.9)

kernel 2.6.32.12 (SUSE SLES11,SP1)
APPlication cpu usage: 5-6%


kernel updated to  2.6.34.7
APPlication cpu usage: 19-25%

kernel 2.6.36.4 (enable/disable RPS)
APPlication cpu usage 35-40%


before I installing profile tool to find the root cause, any idea about it?
'top' result is not accurate  in older kernel? or any kernel
module/feature impact this?

Re: same application has different performance with kernel 2.6.32/34/36

[Stephen Hemminger]

On Fri, 29 Apr 2011 22:46:04 +0800
zhou rui <zhourui.cn@gmail.com> wrote:

> hi
> my test application add a packet hook via  dev_add_pack(&prot_hook);
> then copy the packet to a shared memory between userspace and kernel
> 
> tests wereperformed at same machine(DL380 G7, 12cores, intel X520 10G
> nic),same traffic,same nic driver(ixgbe 3.3.9)
> 
> kernel 2.6.32.12 (SUSE SLES11,SP1)
> APPlication cpu usage: 5-6%
> 
> 
> kernel updated to  2.6.34.7
> APPlication cpu usage: 19-25%
> 
> kernel 2.6.36.4 (enable/disable RPS)
> APPlication cpu usage 35-40%
> 
> 
> before I installing profile tool to find the root cause, any idea about it?
> 'top' result is not accurate  in older kernel? or any kernel
> module/feature impact this?

The report is appreciated, but since your application is not part of the main
kernel, developers are going to be unable to provide much direct help.
If it is closed source, go away we won't help with closed kernel components.
If it is open source, post a link where people can download it.

This is the kind of problem that requires effort and running multiple
times and using kernel bisection can show which change had the most impact.
It could be driver, scheduling or the change to packet steering.