Re: irq load balancing

Chris Snook
2007-09-13T21:30:55Z

Venkat Subbiah wrote:
> Since most network devices have a single status register for both
> receiver and transmit (and errors and the like), which needs a lock to
> protect access, you will likely end up with serious thrashing of moving
> the lock between cpus.
>> Any ways to measure the trashing of locks?
>
> Since most network devices have a single status register for both
> receiver and transmit (and errors and the like)
>> These register accesses will be mostly within the irq handler which I
> plan on keeping on the same processor. The network driver is actually
> tg3. Will looks closely into the driver.

Why are you trying to do this, anyway? This is a classic example of fairness
hurting both performance and efficiency. Unbalanced distribution of a single
IRQ gives superior performance. There are cases when this is a worthwhile
tradeoff, but the network stack is not one of them. In the HPC world, people
generally want to squeeze maximum performance out of CPU/cache/RAM so they just
accept the imbalance because it performs better than balancing it, and
irqbalance can keep things fair over longer intervals if that's important. In
the realtime world, people generally bind everything they can to one or two
CPUs, and bind their realtime applications to the remaining ones to minimize
contention.

Distributing your network interrupts in a round-robin fashion will make your
computer do exactly one thing faster: heat up the room.

-- Chris