Re: bond + tc regression ?

[Stephen Hemminger <shemminger@vyatta.com>]

On Wed, 6 May 2009 02:50:08 +0300
Vladimir Ivashchenko <hazard@francoudi.com> wrote:

> On Tue, May 05, 2009 at 08:50:26PM +0200, Eric Dumazet wrote:
> 
> > > I have tried with IRQs bound to one CPU per NIC. Same result.
> > 
> > Did you check "grep eth /proc/interrupts" that your affinities setup 
> > were indeed taken into account ?
> > 
> > You should use same CPU for eth0 and eth2 (bond0),
> > 
> > and another CPU for eth1 and eth3 (bond1)
> 
> Ok, the best result is when assign all IRQs to the same CPU. Zero drops.
> 
> When I bind slaves of bond interfaces to the same CPU, I start to get 
> some drops, but much less than before. I didn't play with combinations.
> 
> My problem is, after applying your accounting patch below, one of my 
> HTB servers reports only 30-40% CPU idle on one of the cores. That won't 
> take me for very long, load balancing across cores is needed.
> 
> Is there any way at least to balance individual NICs on per core basis?
> 

The user level irqbalance program is a good place to start:
  http://www.irqbalance.org/
But it doesn't yet no how to handle multi-queue devices, and it seems
to not handle NUMA (like SMP Nehalam) perfectly.