ppc4xx: u-boot sees PCIe endpoint, linux does not.

ppc4xx: u-boot sees PCIe endpoint, linux does not.

[Leon Woestenberg]

Hello,

AMCC PPC460EX canyonlands board with an FPGA PCIe end point:

u-boot sees the end point, but Linux does not:

U-Boot 1.3.3-00249-ga524e11 (Jun 30 2008 - 16:05:51)
CPU:   AMCC PowerPC 460EX Rev. A at 800 MHz (PLB=200, OPB=100, EBC=100 MHz)
<...>
Board: Canyonlands - AMCC PPC460EX Evaluation Board, 2*PCIe, Rev. 16
<...>
PCIE1: successfully set as root-complex
        02  00  2071  2071  00ff  00


Now, if I re-program the end-point FPGA during the u-boot boot
time-out, Linux will recognize the end-point.

Any takers on what I should start looking for?

Regards,
-- 
Leon

Re: ppc4xx: u-boot sees PCIe endpoint, linux does not.

[Benjamin Herrenschmidt]

On Fri, 2008-11-28 at 13:50 +0100, Leon Woestenberg wrote:
> Hello,
> 
> AMCC PPC460EX canyonlands board with an FPGA PCIe end point:
> 
> u-boot sees the end point, but Linux does not:
> 
> U-Boot 1.3.3-00249-ga524e11 (Jun 30 2008 - 16:05:51)
> CPU:   AMCC PowerPC 460EX Rev. A at 800 MHz (PLB=200, OPB=100, EBC=100 MHz)
> <...>
> Board: Canyonlands - AMCC PPC460EX Evaluation Board, 2*PCIe, Rev. 16
> <...>
> PCIE1: successfully set as root-complex
>         02  00  2071  2071  00ff  00
> 
> 
> Now, if I re-program the end-point FPGA during the u-boot boot
> time-out, Linux will recognize the end-point.
> 
> Any takers on what I should start looking for?

It's possible that either the reset in between goes bonkers or something
else causes your FPGA to stop responding. It looks like a programming
problem with the FPGA to me.

Ben.

Re: ppc4xx: u-boot sees PCIe endpoint, linux does not.

[Leon Woestenberg]

Hello all,

On Mon, Dec 1, 2008 at 9:12 AM, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> On Fri, 2008-11-28 at 13:50 +0100, Leon Woestenberg wrote:
>>
>> AMCC PPC460EX canyonlands board with an FPGA PCIe end point:
>>
>> u-boot sees the end point, but Linux does not:
>>
>> U-Boot 1.3.3-00249-ga524e11 (Jun 30 2008 - 16:05:51)
>> CPU:   AMCC PowerPC 460EX Rev. A at 800 MHz (PLB=200, OPB=100, EBC=100 MHz)
>> <...>
>> Board: Canyonlands - AMCC PPC460EX Evaluation Board, 2*PCIe, Rev. 16
>> <...>
>> PCIE1: successfully set as root-complex
>>         02  00  2071  2071  00ff  00
>>
>> Now, if I re-program the end-point FPGA during the u-boot boot
>> time-out, Linux will recognize the end-point.
>>
> It's possible that either the reset in between goes bonkers or something
> else causes your FPGA to stop responding. It looks like a programming
> problem with the FPGA to me.
>
I have verified that the end point does not receive any kind of reset.

Also, this problem only happens on the Canyonlands board; on x86 and
powerpc MPC8315E it remains properly working after soft/hard resets,
u-boot init etc.

Could it be u-boot overwrites a too large payload into the config
space or something similar, which makes subsequent accesses fail?

Regards,
-- 
Leon

Re: [U-Boot] ppc4xx: u-boot sees PCIe endpoint, linux does not.

[Stefan Roese]

On Monday 01 December 2008, Leon Woestenberg wrote:
> >> Now, if I re-program the end-point FPGA during the u-boot boot
> >> time-out, Linux will recognize the end-point.
> >
> > It's possible that either the reset in between goes bonkers or something
> > else causes your FPGA to stop responding. It looks like a programming
> > problem with the FPGA to me.
>
> I have verified that the end point does not receive any kind of reset.
>
> Also, this problem only happens on the Canyonlands board; on x86 and
> powerpc MPC8315E it remains properly working after soft/hard resets,
> u-boot init etc.

This could be because only the 4xx Linux PCI(e) driver really resets the 
endpoint (PHY reset). But you seem to have analyzed this already.

> Could it be u-boot overwrites a too large payload into the config
> space or something similar, which makes subsequent accesses fail?

Not sure. I suggest that you disable the PCI(e) support in U-Boot to see if 
Linux behaves differently on a non-pre-initialized endpoint.

Best regards,
Stefan

=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office@denx.de
=====================================================================

Re: [U-Boot] ppc4xx: u-boot sees PCIe endpoint, linux does not.

[Leon Woestenberg]

Hello Stefan,

On Mon, Dec 1, 2008 at 8:46 PM, Stefan Roese <sr@denx.de> wrote:
> On Monday 01 December 2008, Leon Woestenberg wrote:
>> >> Now, if I re-program the end-point FPGA during the u-boot boot
>> >> time-out, Linux will recognize the end-point.
>> >
>> > It's possible that either the reset in between goes bonkers or something
>> > else causes your FPGA to stop responding. It looks like a programming
>> > problem with the FPGA to me.
>>
>> I have verified that the end point does not receive any kind of reset.
>>
>> Also, this problem only happens on the Canyonlands board; on x86 and
>> powerpc MPC8315E it remains properly working after soft/hard resets,
>> u-boot init etc.
>
> This could be because only the 4xx Linux PCI(e) driver really resets the
> endpoint (PHY reset). But you seem to have analyzed this already.
>

Some progress:

Using au-boot  GIT checkout of 9-2-2009 (one month old) I now have
different behaviour:
u-boot does report a link, but no longer the PCIe vendor id:

CPU:   AMCC PowerPC 460EX Rev. A at 800 MHz (PLB=200, OPB=100, EBC=100 MHz)
       Security/Kasumi support
       Bootstrap Option H - Boot ROM Location I2C (Addr 0x52)
       Internal PCI arbiter disabled
       32 kB I-Cache 32 kB D-Cache
Board: Canyonlands - AMCC PPC460EX Evaluation Board, 2*PCIe, Rev. 16
I2C:   ready
DTT:   1 is 44 C
DRAM:  512 MB (ECC not enabled, 400 MHz, CL3)
FLASH: 64 MB
NAND:  128 MiB
PCI:   Bus Dev VenId DevId Class Int
PCIE0: link is not up.
PCIE0: initialization as root-complex failed
PCIE1: successfully set as root-complex

Linux now correctly recognizes the device.
The FPGA with PCIe end point now also survives both a hard reset
(reset button) and soft reset (shutdown -r now in Linux).

Regards,
-- 
Leon