Bamboo PCI interrupt issues

Bamboo PCI interrupt issues

[Hollis Blanchard]

I'm having two problems with PCI interrupts as described in bamboo.dts.
Here is are the properties in question:

	/* Bamboo has all 4 IRQ pins tied together per slot */
	interrupt-map-mask = <f800 0 0 0>;
	interrupt-map = <
		/* IDSEL 1 */
		0800 0 0 0 &UIC0 1c 8

		/* IDSEL 2 */
		1000 0 0 0 &UIC0 1b 8

		/* IDSEL 3 */
		1800 0 0 0 &UIC0 1a 8

		/* IDSEL 4 */
		2000 0 0 0 &UIC0 19 8
	>;


First, the 440EP[1] and Bamboo[2] user manuals indicate that PCI IRQ 0-3
-> board IRQ 2-5 -> UIC IRQ 25-28. However, the device tree has that
reversed, so PCI IRQ 0 appears as UIC IRQ 28 (0x1c).

Second, the sensitivity seems to be wrong. All these interrupts have the
sensitivity encoded as 8, which means "high to low edge" in the OpenPIC
binding. Now, 440EP has a UIC, rather than an OpenPIC, but there is no
UIC binding AFAICS.

When I change the 8 to a 4 ("active high level"), I see the proper
values in the UIC polarity register, and PCI interrupts start working in
KVM.

Is anybody using Bamboo PCI support right now? Does it actually work?

[1]
https://www.amcc.com/MyAMCC/retrieveDocument/PowerPC/440EP/PPC440EP_UM2000.pdf
[2] Seems to have been deleted from the web. Thanks, AMCC.

-- 
Hollis Blanchard
IBM Linux Technology Center

Re: Bamboo PCI interrupt issues

[David Gibson]

On Mon, Mar 03, 2008 at 06:02:33PM -0600, Hollis Blanchard wrote:
> I'm having two problems with PCI interrupts as described in bamboo.dts.
> Here is are the properties in question:
> 
> 	/* Bamboo has all 4 IRQ pins tied together per slot */
> 	interrupt-map-mask = <f800 0 0 0>;
> 	interrupt-map = <
> 		/* IDSEL 1 */
> 		0800 0 0 0 &UIC0 1c 8
> 
> 		/* IDSEL 2 */
> 		1000 0 0 0 &UIC0 1b 8
> 
> 		/* IDSEL 3 */
> 		1800 0 0 0 &UIC0 1a 8
> 
> 		/* IDSEL 4 */
> 		2000 0 0 0 &UIC0 19 8
> 	>;
> 
> 
> First, the 440EP[1] and Bamboo[2] user manuals indicate that PCI IRQ 0-3
> -> board IRQ 2-5 -> UIC IRQ 25-28. However, the device tree has that
> reversed, so PCI IRQ 0 appears as UIC IRQ 28 (0x1c).
> 
> Second, the sensitivity seems to be wrong. All these interrupts have the
> sensitivity encoded as 8, which means "high to low edge" in the OpenPIC
> binding. Now, 440EP has a UIC, rather than an OpenPIC, but there is no
> UIC binding AFAICS.

Uh.. there's no binding written down, it's just encoded into uic.c.
But UIC doesn't use OpenPIC sensitivity encoding.  Like FSL's IPIC, it
uses Linux IRQ_TYPE values from include/linux/irq.h which makes 8
"level sensitive, active-low".

> When I change the 8 to a 4 ("active high level"), I see the proper
> values in the UIC polarity register, and PCI interrupts start working in
> KVM.
> 
> Is anybody using Bamboo PCI support right now? Does it actually work?
> 
> [1]
> https://www.amcc.com/MyAMCC/retrieveDocument/PowerPC/440EP/PPC440EP_UM2000.pdf
> [2] Seems to have been deleted from the web. Thanks, AMCC.
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: Bamboo PCI interrupt issues

[Benjamin Herrenschmidt]

On Tue, 2008-03-04 at 11:59 +1100, David Gibson wrote:
> 
> Uh.. there's no binding written down, it's just encoded into uic.c.
> But UIC doesn't use OpenPIC sensitivity encoding.  Like FSL's IPIC, it
> uses Linux IRQ_TYPE values from include/linux/irq.h which makes 8
> "level sensitive, active-low".

On a related note: aren't we taking a risk here of seeing those values
change in linux ?

Ben.

Re: Bamboo PCI interrupt issues

[David Gibson]

On Tue, Mar 04, 2008 at 12:42:47PM +1100, Benjamin Herrenschmidt wrote:
> 
> On Tue, 2008-03-04 at 11:59 +1100, David Gibson wrote:
> > 
> > Uh.. there's no binding written down, it's just encoded into uic.c.
> > But UIC doesn't use OpenPIC sensitivity encoding.  Like FSL's IPIC, it
> > uses Linux IRQ_TYPE values from include/linux/irq.h which makes 8
> > "level sensitive, active-low".
> 
> On a related note: aren't we taking a risk here of seeing those values
> change in linux ?

We've discussed this before.  If that happens, the binding must remain
on the old values.  It means the driver will then need a translation
which it doesn't now, but we can deal with it.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: Bamboo PCI interrupt issues

[Segher Boessenkool]

>>> Uh.. there's no binding written down, it's just encoded into uic.c.
>>> But UIC doesn't use OpenPIC sensitivity encoding.  Like FSL's IPIC, 
>>> it
>>> uses Linux IRQ_TYPE values from include/linux/irq.h which makes 8
>>> "level sensitive, active-low".
>>
>> On a related note: aren't we taking a risk here of seeing those values
>> change in linux ?
>
> We've discussed this before.  If that happens, the binding must remain
> on the old values.  It means the driver will then need a translation
> which it doesn't now, but we can deal with it.

It also means it should be written down in the binding _already_.
Come on, how much work is that?


Segher

Re: Bamboo PCI interrupt issues

[David Gibson]

On Tue, Mar 04, 2008 at 03:07:50AM +0100, Segher Boessenkool wrote:
> >>> Uh.. there's no binding written down, it's just encoded into uic.c.
> >>> But UIC doesn't use OpenPIC sensitivity encoding.  Like FSL's IPIC, 
> >>> it
> >>> uses Linux IRQ_TYPE values from include/linux/irq.h which makes 8
> >>> "level sensitive, active-low".
> >>
> >> On a related note: aren't we taking a risk here of seeing those values
> >> change in linux ?
> >
> > We've discussed this before.  If that happens, the binding must remain
> > on the old values.  It means the driver will then need a translation
> > which it doesn't now, but we can deal with it.
> 
> It also means it should be written down in the binding _already_.

Well, yes, there should be, but isn't, a written binding for this,
amongst many other things.

> Come on, how much work is that?

Greater than zero.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: Bamboo PCI interrupt issues

[Josh Boyer]

On Mon, 03 Mar 2008 18:02:33 -0600
Hollis Blanchard <hollisb@us.ibm.com> wrote:

> I'm having two problems with PCI interrupts as described in bamboo.dts.
> Here is are the properties in question:
> 
> 	/* Bamboo has all 4 IRQ pins tied together per slot */
> 	interrupt-map-mask = <f800 0 0 0>;
> 	interrupt-map = <
> 		/* IDSEL 1 */
> 		0800 0 0 0 &UIC0 1c 8
> 
> 		/* IDSEL 2 */
> 		1000 0 0 0 &UIC0 1b 8
> 
> 		/* IDSEL 3 */
> 		1800 0 0 0 &UIC0 1a 8
> 
> 		/* IDSEL 4 */
> 		2000 0 0 0 &UIC0 19 8
> 	>;
> 
> 
> First, the 440EP[1] and Bamboo[2] user manuals indicate that PCI IRQ 0-3
> -> board IRQ 2-5 -> UIC IRQ 25-28. However, the device tree has that
> reversed, so PCI IRQ 0 appears as UIC IRQ 28 (0x1c).

Actually, the device tree is right.  I got annoyed with myself for not
knowing how this works so I went and figured it out.

2000 0 0 0 is device #4.  According to the specs, device #4 has AD(14)
asserted during type 0 configuration.  Looking at the board schematics,
PCI slot 0 has it's IDSEL line tied to AD(14).  So:

dev #4 -> PCI 0 -> board IRQ 2 -> UIC IRQ 25.

which is exactly what the device tree has.

> Second, the sensitivity seems to be wrong. All these interrupts have the
> sensitivity encoded as 8, which means "high to low edge" in the OpenPIC
> binding. Now, 440EP has a UIC, rather than an OpenPIC, but there is no
> UIC binding AFAICS.

There isn't.  It uses the sense numbers from linux/irq.h.  Which means
8 is level, low.  This matches exactly what the board manual says for
IRQ2-5 on page 69.

> When I change the 8 to a 4 ("active high level"), I see the proper
> values in the UIC polarity register, and PCI interrupts start working in
> KVM.

That's odd.

> Is anybody using Bamboo PCI support right now? Does it actually work?

I plugged in an old 3Com ethernet card tonight.  Slot 0.  It was
assigned dev #4 IRQ 25.  Using the device tree as-is, I could see
interrupts happening in /proc/interrupts but ethernet traffic failed.

Then I changed the sense level to 4 as you suggested, and my card hung
hard on the first ethernet traffic.  I've no idea if we're dealing with
a crappy card or a crappy driver but the device tree seems to be
working ok.  If I can find a different card to test with I will.

Ben, do you have any input here?

josh

Re: Bamboo PCI interrupt issues

[Stefan Roese]

On Tuesday 04 March 2008, Josh Boyer wrote:
> > Is anybody using Bamboo PCI support right now? Does it actually work?
>
> I plugged in an old 3Com ethernet card tonight.  Slot 0.  It was
> assigned dev #4 IRQ 25.  Using the device tree as-is, I could see
> interrupts happening in /proc/interrupts but ethernet traffic failed.
>
> Then I changed the sense level to 4 as you suggested, and my card hung
> hard on the first ethernet traffic.

Using '8' is correct. PCI interrupts are *always* level sensitive and active 
low.

> I've no idea if we're dealing with 
> a crappy card or a crappy driver but the device tree seems to be
> working ok.  If I can find a different card to test with I will.

One thing always worth to check on 4xx IRQ problems is, if the external IRQ 
pins are configured correctly for IRQ usage. Most of the times, the external 
IRQ's are shared with other peripheral pins and/or GPIO pins. This 
configuration is done in the GPIO core (and sometimes SDR PFCx registers). 
This should be done correctly by the bootloader but sometimes the 
configuration is wrong. I have to admit that I probably never tested PCI on 
Bamboo. Just a thought.

Best regards,
Stefan

Re: Bamboo PCI interrupt issues

[Benjamin Herrenschmidt]

On Tue, 2008-03-04 at 07:15 +0100, Stefan Roese wrote:
> 
> Using '8' is correct. PCI interrupts are *always* level sensitive and
> active 
> low.

Unless you use one of those strange bridges that stick not gates on the
PCI IRQ inputs :-) But I don't think that's the case on the 440EP.

Ben.

Re: Bamboo PCI interrupt issues

[Segher Boessenkool]

>> Using '8' is correct. PCI interrupts are *always* level sensitive and
>> active
>> low.
>
> Unless you use one of those strange bridges that stick not gates on the
> PCI IRQ inputs :-) But I don't think that's the case on the 440EP.

More generally, the target interrupt descriptors (sense values, in
particular) in a device tree interrupt map describe the interrupts as
seen on the target interrupt controller, *not* as seen on this (source)
interrupt domain.  This should be obvious, but since the source 
interrupt
descriptor for PCI doesn't have a sense value (it's always level low,
after all), it can be confusing.  Well, interrupts always are confusing 
:-)


Segher

Re: Bamboo PCI interrupt issues

[Benjamin Herrenschmidt]

On Tue, 2008-03-04 at 21:39 +0100, Segher Boessenkool wrote:
> >> Using '8' is correct. PCI interrupts are *always* level sensitive and
> >> active
> >> low.
> >
> > Unless you use one of those strange bridges that stick not gates on the
> > PCI IRQ inputs :-) But I don't think that's the case on the 440EP.
> 
> More generally, the target interrupt descriptors (sense values, in
> particular) in a device tree interrupt map describe the interrupts as
> seen on the target interrupt controller, *not* as seen on this (source)
> interrupt domain.  This should be obvious, but since the source 
> interrupt
> descriptor for PCI doesn't have a sense value (it's always level low,
> after all), it can be confusing.  Well, interrupts always are confusing 
> :-)

Sure. But if your stupid bridge sticks a not gate between the PIRQ input
and the UIC (interrupt controller), effectively, the UIC sees a reversed
polarity. Thus you need to put in your interrupt map a reversed polarity
information for the UIC interrupt specifiers.

Ben.

Re: Bamboo PCI interrupt issues

[Segher Boessenkool]

>> More generally, the target interrupt descriptors (sense values, in
>> particular) in a device tree interrupt map describe the interrupts as
>> seen on the target interrupt controller, *not* as seen on this 
>> (source)
>> interrupt domain.  This should be obvious, but since the source
>> interrupt
>> descriptor for PCI doesn't have a sense value (it's always level low,
>> after all), it can be confusing.  Well, interrupts always are 
>> confusing
>> :-)
>
> Sure. But if your stupid bridge sticks a not gate between the PIRQ 
> input
> and the UIC (interrupt controller), effectively, the UIC sees a 
> reversed
> polarity. Thus you need to put in your interrupt map a reversed 
> polarity
> information for the UIC interrupt specifiers.

That's what I said, isn't it?  :-)


Segher

Re: Bamboo PCI interrupt issues

[Benjamin Herrenschmidt]

On Tue, 2008-03-04 at 21:59 +0100, Segher Boessenkool wrote:
> >> More generally, the target interrupt descriptors (sense values, in
> >> particular) in a device tree interrupt map describe the interrupts as
> >> seen on the target interrupt controller, *not* as seen on this 
> >> (source)
> >> interrupt domain.  This should be obvious, but since the source
> >> interrupt
> >> descriptor for PCI doesn't have a sense value (it's always level low,
> >> after all), it can be confusing.  Well, interrupts always are 
> >> confusing
> >> :-)
> >
> > Sure. But if your stupid bridge sticks a not gate between the PIRQ 
> > input
> > and the UIC (interrupt controller), effectively, the UIC sees a 
> > reversed
> > polarity. Thus you need to put in your interrupt map a reversed 
> > polarity
> > information for the UIC interrupt specifiers.
> 
> That's what I said, isn't it?  :-)

Maybe, I wasn't sure I decrypted you properly :-)

Ben.

Re: Bamboo PCI interrupt issues

[Benjamin Herrenschmidt]

On Mon, 2008-03-03 at 21:37 -0600, Josh Boyer wrote:
> I plugged in an old 3Com ethernet card tonight.  Slot 0.  It was
> assigned dev #4 IRQ 25.  Using the device tree as-is, I could see
> interrupts happening in /proc/interrupts but ethernet traffic failed.
> 
> Then I changed the sense level to 4 as you suggested, and my card hung
> hard on the first ethernet traffic.  I've no idea if we're dealing
> with
> a crappy card or a crappy driver but the device tree seems to be
> working ok.  If I can find a different card to test with I will.
> 
> Ben, do you have any input here?

Other than bamboo has the weirdest combination of FPGA/CPLD/DIP switches
that I could never figure out if PCI was clocked properly ?

That might just be the problem :-)

I do remember having issues now that we talk about it. Though not
specifically what they were.

Ben.