Re: scheduler performance issue with irq load in 2.6.27

["Chris Friesen" <cfriesen@nortel.com>]

Karl Pickett wrote:
> Running a server work load here.  Two e1000e nics, irqs set to cpu 2
> and cpu 3 by irqbalanced.  One proxy process that has 2 worker
> threads.
> Kernel 2.6.27.21-170.ns6.fc10.i686, (fedora 10 rebuilt with cut down config).
> 
> With no affinity settings.  Note how cpu1 is almost completely idle.
> Mbps = 730
> Cpu0  :  8.3%us, 14.7%sy,  0.0%ni, 74.7%id,  0.0%wa,  0.0%hi,  2.3%si,  0.0%st
> Cpu1  :  0.7%us,  2.7%sy,  0.0%ni, 96.3%id,  0.0%wa,  0.0%hi,  0.3%si,  0.0%st
> Cpu2  : 10.7%us, 25.1%sy,  0.0%ni, 13.7%id,  0.0%wa,  6.0%hi, 44.5%si,  0.0%st
> Cpu3  : 15.0%us, 28.3%sy,  0.0%ni, 23.0%id,  0.0%wa,  4.3%hi, 29.3%si,  0.0%st
> 
> 
> Setting the proxy threads affinity to cpu 0-1.  Note the cpu usage is
> more even and the box
> can handle more speed.
> Mbps = 850
> Cpu0  : 15.0%us, 36.3%sy,  0.0%ni, 41.7%id,  0.0%wa,  0.0%hi,  7.0%si,  0.0%st
> Cpu1  : 14.7%us, 38.7%sy,  0.0%ni, 39.0%id,  0.0%wa,  0.0%hi,  7.7%si,  0.0%st
> Cpu2  :  0.0%us,  0.0%sy,  0.0%ni, 38.2%id,  0.0%wa,  4.7%hi, 57.1%si,  0.0%st
> Cpu3  :  0.0%us,  0.3%sy,  0.0%ni, 61.8%id,  0.0%wa,  2.0%hi, 35.9%si,  0.0%st
> 
> I'm not seeing any thing I can tune scheduler wise here.


/proc/sys/kernel/sched_domain/cpuX/domainY/imbalance_pct

If you set that to something like 110 or so, it will increase the 
likelihood of balancing across cpus.

You may need to enable a config option, can't remember.

Chris