Xtalk bridge IRQs

Xtalk bridge IRQs

[Stanislaw Skowronek]

Hello!

I've got a problem with Xtalk-PCI bridge IRQs. The IOC3 should send an
interrupt before transmitting a packet. I don't know if it sends it or
not, but for sure it does not arrive to me. The power button interrupt,
which is also routed via the bridge, works OK, so the IRQ router part of
the bridge is correctly services. However, I can't get any PCI interrupts.

Stanislaw Skowronek

--<=>--
  Paranoid: one who is truly in touch with reality.

Re: Xtalk bridge IRQs

[Ralf Baechle]

On Sat, Apr 24, 2004 at 07:14:32PM +0200, Stanislaw Skowronek wrote:

> I've got a problem with Xtalk-PCI bridge IRQs. The IOC3 should send an
> interrupt before transmitting a packet. I don't know if it sends it or
> not, but for sure it does not arrive to me. The power button interrupt,
> which is also routed via the bridge, works OK, so the IRQ router part of
> the bridge is correctly services. However, I can't get any PCI interrupts.

The way that PCI interrupts work on IP27 is: A device's INTA pin is
connected to the BRIDGE ASIC.  When a device's interrupt line is asserted,
the bridge ASIC will store a value which consists of 0x100 | intnum into
another register; the address of that other register can be configured per
device in the BRIDGE chip.  The value is usuall the xtalk address of another
register in the HUB ASIC.  For HUB the value 0x100 | intnum would mean to
set bit intnum in the interrupt pending register.  The BRIDGE chip
can also be configured to send an interrupt clear packet if the PCI
interrupt is deasserted again; it's a good idea to have this enabled since
otherwise writing race-free interrupt handlers is a PITA.  The HUB
chip then asserts one of the CPU interrupt lines if a bit is set in the
interrupt pending register and mask0/mask1 register for the CPU.

For a single node system this could be slightly simplified - no idea if
SGI did that for the HEART design.  Anyway, in case of an Origin you'd
have to chase the interrupt through the various stages of processing:

 - Interupt sent from the device?
 - Interrupt asserted at BRIDGE?
 - Interrupt bit set in HUB chip?
 - Interrupt enabled in HUB mask register?
 - Interrupt pending in the CPU's cause register?
 - Interrupt bit set and EXL and ERL both clear in CPU status register?

I hope that's somehow applicable to the IP30 ...

  Ralf

Re: Xtalk bridge IRQs

[Stanislaw Skowronek]

> The way that PCI interrupts work on IP27 is: A device's INTA pin is
> connected to the BRIDGE ASIC.

Only INTAs? Does each INTA of a device have a dedicated pin on the bridge
(I am not sure, but it seems so)?

> When a device's interrupt line is asserted, the bridge ASIC will store
> a value which consists of 0x100 | intnum into another register; the
> address of that other register can be configured per device in the
> BRIDGE chip.  The value is usuall the xtalk address of another
> register in the HUB ASIC.  For HUB the value 0x100 | intnum would mean
> to set bit intnum in the interrupt pending register.

OK, I figured that out from the IRIX headers. Apparently, HEART does it
the same way, only the register is at 0x80 and not 0x90.

> The BRIDGE chip can also be configured to send an interrupt clear
> packet if the PCI interrupt is deasserted again; it's a good idea to
> have this enabled since otherwise writing race-free interrupt handlers
> is a PITA.  The HUB chip then asserts one of the CPU interrupt lines
> if a bit is set in the interrupt pending register and mask0/mask1
> register for the CPU.

It is very similar to the HEART, then. But I didn't know about clear
packets.

> For a single node system this could be slightly simplified - no idea if
> SGI did that for the HEART design.  Anyway, in case of an Origin you'd
> have to chase the interrupt through the various stages of processing:
>  - Interupt sent from the device?

Well, I don't know... Should I take out my logic analyzer?

>  - Interrupt asserted at BRIDGE?

I don't know this, either.

>  - Interrupt bit set in HUB chip?

Well, if anything is asserted at the bridge, all the following steps are
correctly executed. It's a problem around the IOC. Could you please tell
me, how are IOC packets transmitted? What is the relation of this to the
interrupts?

I have tried to run NFSRoot, but the machine sends no packets, even though
it believes firmly that it sends (i.e. no error messages).

> I hope that's somehow applicable to the IP30 ...

So do I :)

Stanislaw Skowronek

Re: Xtalk bridge IRQs

[Ralf Baechle]

On Sat, Apr 24, 2004 at 08:34:10PM +0200, Stanislaw Skowronek wrote:

> > The way that PCI interrupts work on IP27 is: A device's INTA pin is
> > connected to the BRIDGE ASIC.
> 
> Only INTAs? Does each INTA of a device have a dedicated pin on the bridge
> (I am not sure, but it seems so)?

If I recall right the INTA-INTD pins of each devices are logically or'ed
together.  Since IOC3 only uses INTA this simple fact doesn't really matter
to us anyway ...

> > The BRIDGE chip can also be configured to send an interrupt clear
> > packet if the PCI interrupt is deasserted again; it's a good idea to
> > have this enabled since otherwise writing race-free interrupt handlers
> > is a PITA.  The HUB chip then asserts one of the CPU interrupt lines
> > if a bit is set in the interrupt pending register and mask0/mask1
> > register for the CPU.
> 
> It is very similar to the HEART, then. But I didn't know about clear
> packets.

If BRIDGE isn't configured to send clear packets the interrupt needs to be
clear manually by a processor write to the HUB's register.  Leaving that
to the hardware is just simpler.

> > For a single node system this could be slightly simplified - no idea if
> > SGI did that for the HEART design.  Anyway, in case of an Origin you'd
> > have to chase the interrupt through the various stages of processing:
> >  - Interupt sent from the device?
> 
> Well, I don't know... Should I take out my logic analyzer?

Let's wait with the big guns :)

> >  - Interrupt asserted at BRIDGE?
> 
> I don't know this, either.

You can verify by reading the pending register.

> >  - Interrupt bit set in HUB chip?
> 
> Well, if anything is asserted at the bridge, all the following steps are
> correctly executed. It's a problem around the IOC. Could you please tell
> me, how are IOC packets transmitted? What is the relation of this to the
> interrupts?
> 
> I have tried to run NFSRoot, but the machine sends no packets, even though
> it believes firmly that it sends (i.e. no error messages).

IOC3 is basically your average mid-90s 100BaseT NIC.  If you know some other
NIC like for example a PCnet32, they're basically similar in their
fundamental concepts such as RX/TX rings.

An IOC3 TX ring consists of entries which each are 128 bytes long.  Each
contains a few status etc. bits, a few bytes that could be used to store a
small packet (ioc3-eth.c uses this) and two 64-bit pointers to actual
packet data.  IOC3 has the ugly constraint of not supporting transmit of
packets that cross a 16kB page boundary.  In that case of such a packet we
use the pointer to the second fragment, otherwise only the first.

IP27 support 32-bit mapped DMA which Linux doesn't support yet, so only
64-bit PCI devices will work atm.  For those the upper 16 bit specify extra
attributes which you'll find in include/asm-mips/pci/bridge.h which again
is just a copy of the IRIX <PCI/bridge.h>.

I think the way we use those bits should also be aplicable to IP30 but I'm
not sure never having seen any material on the machine.  You may want to
try to figure out what value the firmware uses when configuring the IOC3
for netbooting.

Does link negotiation and reading the MAC address from the NIC (Number In
a Can, that coin battery like Dallas chip near the IOC3) work?  At least
link negotiation would have to work before you'd have any chance to see
any packets.

  Ralf