Re: [PATCH][2.5][RFC] Using xAPIC apic address space on !Summit

[James Cleverdon <jamesclv@us.ibm.com>]

On Thursday 12 December 2002 09:53 pm, Steffen Persvold wrote:
> On Thu, 12 Dec 2002, James Cleverdon wrote:
> > On Thursday 12 December 2002 07:26 pm, Zwane Mwaikambo wrote:
> > > On Thu, 12 Dec 2002, Nakajima, Jun wrote:
> > > > BTW, we are working on a xAPIC patch that supports more than 8 CPUs
> > > > in a generic fashion (don't use hardcode OEM checking). We already
> > > > tested it on two OEM systems with 16 CPUs.
> > > > - It uses clustered mode. We don't want to use physical mode because
> > > > it does not support lowest priority delivery mode.
> > >
> > > Wouldn't that only be for all including self? Or is the documentation
> > > incorrect?
> > >
> > > Thanks,
> > > 	Zwane
> >
> > I'm not sure I understand your question.  Lowest Priority delivery mode
> > only works with logical interrupts.  (I've tried it with physical intrs. 
> > It fails miserably.)  The "all including self" and "all excluding self"
> > destination shorthands don't do lowest priority arbitration.  They always
> > deliver the interrupt to the CPUs mentioned in the shortand.
> >
> > Lowest priority delivery mode isn't _too_ useful in Linux yet.  It would
> > be nice to preferentially target idle CPUs with interrupts in real time. 
> > That means changing each CPU's Task Priority Register (TPR) to represent
> > how busy it is.  I've got some patches to do that, but haven't posted
> > them as anything more than a RFC.
>
> Hmm, I though the APIC routing patch found in the LSE project
> (http://sourceforge.net/projects/lse/) did this already. Atleast I've
> tested this patch on a couple of Dual E7500 Xeon boxes (kernel 2.4.20) and
> it distributes interrupts nicely.
>
> However with the patch enabled, the interrupt latency on for example the
> Intel GbE 82544GC devices increased a fraction with this patch (a
> microsecond or two).
>
> Regards,

Sure, Dave Olien's patch adjusted the TPR.  However, he wrote that for the 
classic APIC; it does most of the priority adjustments in the lower nibble of 
the TPR's value.  xAPIC routing is done via HW in the PCI-to-host bridge 
chips.  There they keep a copy of each CPU's TPR value in eight XTPR 
registers for lowest priority interrupt routing -- but only the TPR's upper 
nibble.  So, Dave's patch is less useful on xAPIC systems.

I came up with something simpler.   Just 2 lines added to idle_cpu() and 
do_IRQ respectively.  It's a hack but it seemed useful.

Interesting that it would be a microsecond slower.  Maybe that's the time it 
takes to adjust the TPR.  One more reason to keep those adjustments as simple 
as possible.

-- 
James Cleverdon
IBM xSeries Linux Solutions
{jamesclv(Unix, preferred), cleverdj(Notes)} at us dot ibm dot com