"No handler for vector" patches don't work on some systems

"No handler for vector" patches don't work on some systems

[Chuck Ebbert]

So far I've tried the simple "survive having no handler
for a vector" patch and the preliminary 3-patch series
that was in -mm for a while, and neither work on the
Dell PowerEdge 29xx and 19xx systems. These servers
have the Intel 5000X chipset with the 6700PXH PCI Hub
with dual independent PCI-X busses, each with its own
I/OxAPIC with 24 interrupts. The fixes do work on
"simple" systems but not on these high-end ones.

"No handler for vector" patches don't work on some systems

[Chuck Ebbert]

[sorry for the dup: this time to the right recipient]

So far I've tried the simple "survive having no handler
for a vector" patch and the preliminary 3-patch series
that was in -mm for a while, and neither work on the
Dell PowerEdge 29xx and 19xx systems. These servers
have the Intel 5000X chipset with the 6700PXH PCI Hub
with dual independent PCI-X busses, each with its own
I/OxAPIC with 24 interrupts. The fixes do work on
"simple" systems but not on these high-end ones.

Re: "No handler for vector" patches don't work on some systems

[Eric W. Biederman]

Chuck Ebbert <cebbert@redhat.com> writes:

> [sorry for the dup: this time to the right recipient]
>
> So far I've tried the simple "survive having no handler
> for a vector" patch and the preliminary 3-patch series
> that was in -mm for a while, and neither work on the
> Dell PowerEdge 29xx and 19xx systems. These servers
> have the Intel 5000X chipset with the 6700PXH PCI Hub
> with dual independent PCI-X busses, each with its own
> I/OxAPIC with 24 interrupts. The fixes do work on
> "simple" systems but not on these high-end ones.

Ok thanks for the report.  It sounds like there is another cause
for the problem in the Dell case.

The simple patch drops the interrupt handler but acknowledges the
hardware so if the driver can survive missing an interrupt we
should be ok.  With level triggered interrupts this should pretty
much be guaranteed as after the acknowledgement the unhandled
interrupt will be refired.

One of my internal test systems had a 6700PXH PCI hub (at least I
think that was the part) the E7520 chipset.  So I don't think it is
just a matter of the hardware.  Although I do recall Intel having an
errata out on that class of hardware for occasionally reordering
interrupt messages with the end of interrupt coming before the 
interrupt message itself.  Causing various things to get confused.
It would not surprise me if we were tickling some errata like that.

I would very much like to know if what I merged linus's tree helps.
It is a little more conservative, than my earlier patches.  I need
a way to reproduce this or to work closely with someone who is, because
this sounds like it has a different cause and I need to start with
that assumption.

Eric

Re: "No handler for vector" patches don't work on some systems

[Chuck Ebbert]

Eric W. Biederman wrote:
> Chuck Ebbert <cebbert@redhat.com> writes:
>>
>> So far I've tried the simple "survive having no handler
>> for a vector" patch and the preliminary 3-patch series
>> that was in -mm for a while, and neither work on the
>> Dell PowerEdge 29xx and 19xx systems. These servers
>> have the Intel 5000X chipset with the 6700PXH PCI Hub
>> with dual independent PCI-X busses, each with its own
>> I/OxAPIC with 24 interrupts. The fixes do work on
>> "simple" systems but not on these high-end ones.
> 
> 
> I would very much like to know if what I merged linus's tree helps.
> It is a little more conservative, than my earlier patches.  I need
> a way to reproduce this or to work closely with someone who is, because
> this sounds like it has a different cause and I need to start with
> that assumption.

Was that merged or is it still in -mm? The last thing I see in
arch/x86_64/irq.c is:

  	[PATCH] x86-64: survive having no irq mapping for a vector

And we tried that one.

Re: "No handler for vector" patches don't work on some systems

[Eric W. Biederman]

Chuck Ebbert <cebbert@redhat.com> writes:

> Eric W. Biederman wrote:
>> Chuck Ebbert <cebbert@redhat.com> writes:
>>>
>>> So far I've tried the simple "survive having no handler
>>> for a vector" patch and the preliminary 3-patch series
>>> that was in -mm for a while, and neither work on the
>>> Dell PowerEdge 29xx and 19xx systems. These servers
>>> have the Intel 5000X chipset with the 6700PXH PCI Hub
>>> with dual independent PCI-X busses, each with its own
>>> I/OxAPIC with 24 interrupts. The fixes do work on
>>> "simple" systems but not on these high-end ones.
>> 
>> 
>> I would very much like to know if what I merged linus's tree helps.
>> It is a little more conservative, than my earlier patches.  I need
>> a way to reproduce this or to work closely with someone who is, because
>> this sounds like it has a different cause and I need to start with
>> that assumption.
>
> Was that merged or is it still in -mm? The last thing I see in
> arch/x86_64/irq.c is:
>
>   	[PATCH] x86-64: survive having no irq mapping for a vector
>
> And we tried that one.

Look in arch/x86_64/io_apic.c. That is where most of the work happened.
If you can extract that patch series for a backport more power to you.

Eric

commit 610142927b5bc149da92b03c7ab08b8b5f205b74
Author: Eric W. Biederman <ebiederm@xmission.com>
Date:   Fri Feb 23 04:40:58 2007 -0700

    [PATCH] x86_64 irq: Safely cleanup an irq after moving it.
    
    The problem:  After moving an interrupt when is it safe to teardown
    the data structures for receiving the interrupt at the old location?
    
    With a normal pci device it is possible to issue a read to a device
    to flush all posted writes.  This does not work for the oldest ioapics
    because they are on a 3-wire apic bus which is a completely different
    data path.  For some more modern ioapics when everything is using
    front side bus delivery you can flush interrupts by simply issuing a
    read to the ioapic.  For other modern ioapics emperical testing has
    shown that this does not work.
    
    So it appears the only reliable way to know the last of the irqs from an
    ioapic have been received from before the ioapic was reprogrammed is to
    received the first irq from the ioapic from after it was reprogrammed.
    
    Once we know the last irq message has been received from an ioapic
    into a local apic we then need to know that irq message has been
    processed through the local apics.
    
    Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>