Re: CONFIG_IRQBALANCE for AMD64?

[Andi Kleen <ak@muc.de>]

On Fri, May 28, 2004 at 03:05:48PM -0700, Nakajima, Jun wrote:
> >At least the AMD chipsets found in most Opteron boxes need software
> >balancing too.
> 
> Actually lowest priority delivery works on P4 and AMD (I did not tested
> it on AMD, though), if we _update_ TPR. But I don't recommend that,

True. I remember there was even a patch for that from James C. long ago.
I considered at one point to add it to x86-64, but ended up 
not doing anything and just recommending irqbalanced.

[I didn't test it neither, so no guarantee TPR really works on AMD]

> instead we should implement the similar or optimized behavior in
> software because "soft TPR" can be more efficient and scalable. And I
> think this is something in my mind, and I think the kernel should do it.

I'm not convinced of that. At least the current i386 implemention
is basically a kernel thread that wakes up regularly, reads some
statistics and then updates the APIC.

There is not much reason this cannot be done in user space, and
user space has the advantage that more advanced heuristics (which
I'm sure will be there) can be more easily implemented.

And handling all interrupts at CPU #0 during early boot up is 
not really an issue.

An kernel implementation may make sense when you're doing something
really dynamic: e.g. not just a timer, but dynamically redirecting
network interrupts to the CPU the process who will read from the
socket runs on. Obviously it would need kernel support for that, since
user space could not keep up with such a high sampling rate. But that's
future research work (if it can be even done generically at all)
and I don't see it on the radar screen anytime soon. We first need to solve
the NUMA scheduling problem, which is already hard enough ;-) 

And for the simpler heuristics that don't need real time updates
user space is probably better.

-Andi