Re: next-20081030: voyager compile busted

[James Bottomley <James.Bottomley@HansenPartnership.com>]

On Thu, 2008-10-30 at 23:40 +0100, Ingo Molnar wrote: 
> * H. Peter Anvin <hpa@zytor.com> wrote:
> 
> > James Bottomley wrote:
> > > 
> > > Yes ... been having IRC conversations about that.  We'd need to use
> > > runtime patching to fix the performance regressions virtualisation has
> > > been causing us first ... but then we could use it for voyager.
> > 
> > I thought we already were, at least to some degree (the call sites 
> > are way too big and way bigger than they need to be, so we end up 
> > with a lot of NOPs.  I proposed a solution to Jeremy at Kernel 
> > Summit, but he basically said "I don't want to maintain that, I 
> > don't care about hardware performance", which is understandable but 
> > highly unfortunate.)
> 
> in practice the function pointer overhead is almost unmeasurable for 
> things like IPIs which are rather expensive to issue. In fact it's 
> probably more expensive on Voyager as it does not utilize the local 
> APIC for SMP messaging (which is pretty much the only 
> performance-sensitive thing here). It uses its own glue logic it 
> appears, which is almost certainly behind the system bus.

Heh ... don't be so sure:  The VIC uses an array of up to 8 standard
8255 IRQ lines, which is slow.  The QIC uses CPI trigger via cache line
interference, which is far faster than APIC bus based systems (but is
fairly similar to the way SAPIC systems work ... almost a decade after
the original QIC system).

> James, how many cycles do typical SMP ops take on Voyager?

To be honest, I'm not too bothered about that.  What bothers me is that
if a normal system performance goes down because of the function pointer
replacement (remember all x86 SMP systems will see an increase in
function pointer usage because of this approach), then it probably
wasn't worth it.

However, the only way to be sure is to try it and get the benchmarks.

James