PCI-PCI bridge scanning broken on 460EX

PCI-PCI bridge scanning broken on 460EX

[Felix Radensky]

Hi,

I'm running linux-2.6.33-rc2 on Canyonlands board. When PLX 6254 
transparent PCI-PCI
bridge is plugged into PCI slot the kernel simply resets the board 
without printing anything
to console. Without PLX bridge kernel boots fine.

I've tracked down the problem to the following code in pci_scan_bridge() 
in drivers/pci/probe.c:

if (pcibios_assign_all_busses() || broken)
                /* Temporarily disable forwarding of the
                   configuration cycles on all bridges in
                   this bus segment to avoid possible
                   conflicts in the second pass between two
                   bridges programmed with overlapping
                   bus ranges. */
                pci_write_config_dword(dev, PCI_PRIMARY_BUS,
                               buses & ~0xffffff);

If test for broken is removed, kernel boots fine, detects the bridge, but
does not detect the device behind the bridge. The same device plugged
directly into PCI slot is detected correctly.

To remind you, tests for broken were added by commit 
a1c19894b786f10c76ac40e93c6b5d70c9b946d2,
and were intended to solve device detection problem behind PCI-E 
switches, as discussed in this thread:
http://lists.ozlabs.org/pipermail/linuxppc-dev/2008-October/063939.html

I'd appreciate any help in resolving this issue. Below is dmesg output 
with DEBUG enabled
and problematic test for broken removed.

Using PowerPC 44x Platform machine description
Linux version 2.6.33-rc2 (felix@felix-laptop.lan) (gcc version 4.2.2) 
#10 Mon De
c 28 12:21:25 IST 2009
Found legacy serial port 0 for /plb/opb/serial@ef600300
  mem=4ef600300, taddr=4ef600300, irq=0, clk=7407407, speed=0
Found legacy serial port 1 for /plb/opb/serial@ef600400
  mem=4ef600400, taddr=4ef600400, irq=0, clk=7407407, speed=0
Found legacy serial port 2 for /plb/opb/serial@ef600500
  mem=4ef600500, taddr=4ef600500, irq=0, clk=7407407, speed=0
Found legacy serial port 3 for /plb/opb/serial@ef600600
  mem=4ef600600, taddr=4ef600600, irq=0, clk=7407407, speed=0
Top of RAM: 0x20000000, Total RAM: 0x20000000
Memory hole size: 0MB
Zone PFN ranges:
  DMA      0x00000000 -> 0x00020000
  Normal   0x00020000 -> 0x00020000
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
    0: 0x00000000 -> 0x00020000
On node 0 totalpages: 131072
free_area_init_node: node 0, pgdat c026ba48, node_mem_map c0289000
  DMA zone: 1024 pages used for memmap
  DMA zone: 0 pages reserved
  DMA zone: 130048 pages, LIFO batch:31
MMU: Allocated 1088 bytes of context maps for 255 contexts
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 130048
Kernel command line: root=/dev/nfs rw 
nfsroot=10.0.0.10:/opt/eldk/ppc_4xx ip=10.
0.0.30:10.0.0.10::255.0.0.0:canyonlands:eth0:off panic=1 
console=ttyS0,115200 de
bug
PID hash table entries: 2048 (order: 1, 8192 bytes)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 516864k/524288k available (2416k kernel code, 7140k reserved, 
100k data,
 72k bss, 128k init)
Kernel virtual memory layout:
  * 0xfffdf000..0xfffff000  : fixmap
  * 0xfde00000..0xfe000000  : consistent mem
  * 0xfde00000..0xfde00000  : early ioremap
  * 0xe1000000..0xfde00000  : vmalloc & ioremap
SLUB: Genslabs=11, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Hierarchical RCU implementation.
NR_IRQS:512 nr_irqs:512
UIC0 (32 IRQ sources) at DCR 0xc0
UIC1 (32 IRQ sources) at DCR 0xd0
  alloc irq_desc for 30 on node 0
  alloc kstat_irqs on node 0
irq: irq 30 on host /interrupt-controller0 mapped to virtual irq 30
UIC2 (32 IRQ sources) at DCR 0xe0
  alloc irq_desc for 16 on node 0
  alloc kstat_irqs on node 0
irq: irq 10 on host /interrupt-controller0 mapped to virtual irq 16
UIC3 (32 IRQ sources) at DCR 0xf0
  alloc irq_desc for 17 on node 0
  alloc kstat_irqs on node 0
irq: irq 16 on host /interrupt-controller0 mapped to virtual irq 17
time_init: decrementer frequency = 600.000007 MHz
time_init: processor frequency   = 600.000007 MHz
clocksource: timebase mult[6aaaab] shift[22] registered
clockevent: decrementer mult[999999b7] shift[32] cpu[0]
Mount-cache hash table entries: 512
NET: Registered protocol family 16
  alloc irq_desc for 18 on node 0
  alloc kstat_irqs on node 0
irq: irq 11 on host /interrupt-controller1 mapped to virtual irq 18
256k L2-cache enabled
PCI host bridge /plb/pci@c0ec00000 (primary) ranges:
 MEM 0x0000000d80000000..0x0000000dffffffff -> 0x0000000080000000
 MEM 0x0000000c0ee00000..0x0000000c0eefffff -> 0x0000000000000000
  IO 0x0000000c08000000..0x0000000c0800ffff -> 0x0000000000000000
 Removing ISA hole at 0x0000000c0ee00000
4xx PCI DMA offset set to 0x00000000
/plb/pci@c0ec00000: Legacy ISA memory support enabled
PCI: Probing PCI hardware
pci_bus 0000:00: scanning bus
pci 0000:00:06.0: found [3388:0020] class 000604 header type 01
pci 0000:00:06.0: supports D1 D2
pci 0000:00:06.0: PME# supported from D0 D1 D2 D3hot
pci 0000:00:06.0: PME# disabled
pci_bus 0000:00: fixups for bus
pci 0000:00:06.0: scanning behind bridge, config 000000, pass 0
pci 0000:00:06.0: bus configuration invalid, reconfiguring
pci 0000:00:06.0: scanning behind bridge, config 000000, pass 1
pci_bus 0000:01: scanning bus
pci 0000:01:06.0: found [3388:0020] class 000604 header type 01
pci 0000:01:06.0: supports D1 D2
pci 0000:01:06.0: PME# supported from D0 D1 D2 D3hot
pci 0000:01:06.0: PME# disabled
pci_bus 0000:01: fixups for bus
pci 0000:00:06.0: PCI bridge to [bus 01-ff]
pci 0000:00:06.0:   bridge window [io  0x0000-0x0fff]
pci 0000:00:06.0:   bridge window [mem 0x00000000-0x000fffff]
pci 0000:00:06.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
pci 0000:01:06.0: scanning behind bridge, config ff0100, pass 0
pci 0000:01:06.0: bus configuration invalid, reconfiguring
pci 0000:01:06.0: scanning behind bridge, config ff0100, pass 1
pci_bus 0000:01: bus scan returning with max=01
pci_bus 0000:00: bus scan returning with max=01
pci 0000:00:06.0: disabling bridge window [io  0x0000-0x0fff] to [bus 
01-01] (un
used)
pci 0000:00:06.0: disabling bridge window [mem 0xd00000000-0xd000fffff 
pref] to
[bus 01-01] (unused)
pci 0000:00:06.0: disabling bridge window [mem 0xd00000000-0xd000fffff] 
to [bus
01-01] (unused)
pci 0000:00:06.0: PCI bridge to [bus 01-01]
pci 0000:00:06.0:   bridge window [io  disabled]
pci 0000:00:06.0:   bridge window [mem disabled]
pci 0000:00:06.0:   bridge window [mem pref disabled]
pci_bus 0000:00: resource 0 [io  0x0000-0xffff]
pci_bus 0000:00: resource 1 [mem 0xd80000000-0xdffffffff]
pci_bus 0000:01: resource 0 [??? 0-4095 flags 0x0]
pci_bus 0000:01: resource 1 [??? 55834574848-55835623423 flags 0x0]
pci_bus 0000:01: resource 2 [??? 55834574848-55835623423 flags 0x0]

Thanks a lot.

Felix.

Re: PCI-PCI bridge scanning broken on 460EX

[Benjamin Herrenschmidt]

On Mon, 2009-12-28 at 12:51 +0200, Felix Radensky wrote:
> Hi,
> 
> I'm running linux-2.6.33-rc2 on Canyonlands board. When PLX 6254 
> transparent PCI-PCI
> bridge is plugged into PCI slot the kernel simply resets the board 
> without printing anything
> to console. Without PLX bridge kernel boots fine.

Sorry for the late reply...

> I've tracked down the problem to the following code in pci_scan_bridge() 
> in drivers/pci/probe.c:
> 
> if (pcibios_assign_all_busses() || broken)
>                 /* Temporarily disable forwarding of the
>                    configuration cycles on all bridges in
>                    this bus segment to avoid possible
>                    conflicts in the second pass between two
>                    bridges programmed with overlapping
>                    bus ranges. */
>                 pci_write_config_dword(dev, PCI_PRIMARY_BUS,
>                                buses & ~0xffffff);
> 
> If test for broken is removed, kernel boots fine, detects the bridge, but
> does not detect the device behind the bridge. The same device plugged
> directly into PCI slot is detected correctly.

So we would have a similar mismatch between the initial setup and the
kernel...  However, I don't quite see yet why the kernel trying to fix
it up breaks things, that will need a bit more debugging here...

Can you give it a quick try with adding something like :

 ppc_pci_add_flags(PPC_PCI_REASSIGN_ALL_BUS);

Near the end of ppc4xx_pci.c ? It looks like another case of reset
not actually resetting bridges (are we not properly doing a fundamental
reset ? Stefan what's your take there ?)

The above will cause busses to be re-assigned which is risky because it
will allow the kernel to assign numbers beyond the limits of what
ppc4xx_pci.c supports (see my comments in the thread you quotes).

The good thing is that we now have a working fixmap infrastructure, so
we could/should just move ppc4xx_pci.c to use that, and just always
re-assign busses.

> To remind you, tests for broken were added by commit 
> a1c19894b786f10c76ac40e93c6b5d70c9b946d2,
> and were intended to solve device detection problem behind PCI-E 
> switches, as discussed in this thread:
> http://lists.ozlabs.org/pipermail/linuxppc-dev/2008-October/063939.html

> PCI: Probing PCI hardware
> pci_bus 0000:00: scanning bus
> pci 0000:00:06.0: found [3388:0020] class 000604 header type 01
> pci 0000:00:06.0: supports D1 D2
> pci 0000:00:06.0: PME# supported from D0 D1 D2 D3hot
> pci 0000:00:06.0: PME# disabled
> pci_bus 0000:00: fixups for bus
> pci 0000:00:06.0: scanning behind bridge, config 000000, pass 0
> pci 0000:00:06.0: bus configuration invalid, reconfiguring

Ok so we hit a P2P bridge whose primary, secondary and subordinate bus
numbers are all 0, which is clearly unconfigured. I think this is the
root complex bridge

> pci 0000:00:06.0: scanning behind bridge, config 000000, pass 1

Now this is when the bus should be reconfigured (pass 1). Sadly the code
doesn't print much debug.

Also from that point, it should renumber things and work... 

> pci_bus 0000:01: scanning bus

Which it does to some extent. It assigned bus number 1 to it afaik so we
now start looking below the RC bridge:

> pci 0000:01:06.0: found [3388:0020] class 000604 header type 01

Hrm... class PCI bridge, vendor 3388 device 0020, is that your PLX ?
It's not the right vendor ID but maybe that's configurable by our OEM or
something...

> pci 0000:01:06.0: supports D1 D2
> pci 0000:01:06.0: PME# supported from D0 D1 D2 D3hot
> pci 0000:01:06.0: PME# disabled
> pci_bus 0000:01: fixups for bus
> pci 0000:00:06.0: PCI bridge to [bus 01-ff]
> pci 0000:00:06.0:   bridge window [io  0x0000-0x0fff]
> pci 0000:00:06.0:   bridge window [mem 0x00000000-0x000fffff]
> pci 0000:00:06.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
> pci 0000:01:06.0: scanning behind bridge, config ff0100, pass 0

Allright, that's where it gets interesting. It tries to scan behind the
bridge. It gets something it doesn't like. IE, it gets a secondary bus
number of 1 (what the heck ? I wonder what your firmware does) which
Linux is not happy about and decides to renumber it.

> pci 0000:01:06.0: bus configuration invalid, reconfiguring

Now, that's where Linux should have written 000000 to the register,
which is what you commented out.

> pci 0000:01:06.0: scanning behind bridge, config ff0100, pass 1
> pci_bus 0000:01: bus scan returning with max=01
> pci_bus 0000:00: bus scan returning with max=01

Because of that commenting out, it doesn't see the config as 000000 and
thus doesn't re-assign a bus number in pass 1, so from there you can't
see what's behind the bus.

So we have two things here:

 - It seems like the writing of 000000 to the register in pass 0 is
causing your crash. Can you verify that ? IE. Can you verify that it's
indeed crashing on this specific statement:

pci_write_config_dword(dev, PCI_PRIMARY_BUS,
                               buses & ~0xffffff);

When writing to the bridge, and that this seems to be causing a hard
reboot of the system ?

It might be useful to ask AMCC how that is possible in HW, ie what kind
of signal can be causing that. IE, even if the bridge is causing a PCIe
error, that should not cause a reboot ... right ?

 - You can test a quick hack workaround which consists of changing:

	/* Check if setup is sensible at all */
-	if (!pass &&
-	if (1 &&
	    ((buses & 0xff) != bus->number || ((buses >> 8) & 0xff) <= bus->number)) {
		dev_dbg(&dev->dev, "bus configuration invalid, reconfiguring\n");
		broken = 1;
	}

In -addition- to your commenting out of the broken test. This will cause the
second pass to go through the re-assign code path despite the fact that you
have not written 000000 to the bus numbers.

Cheers,
Ben.

Re: PCI-PCI bridge scanning broken on 460EX

[Felix Radensky]

Hi, Ben

Adding Feng Kan from AMCC to CC.

Benjamin Herrenschmidt wrote:
> On Mon, 2009-12-28 at 12:51 +0200, Felix Radensky wrote:
>   
>> Hi,
>>
>> I'm running linux-2.6.33-rc2 on Canyonlands board. When PLX 6254 
>> transparent PCI-PCI
>> bridge is plugged into PCI slot the kernel simply resets the board 
>> without printing anything
>> to console. Without PLX bridge kernel boots fine.
>>     
>
> Sorry for the late reply...
>   

No need to apologize, I appreciate you help very much.

>   
>> I've tracked down the problem to the following code in pci_scan_bridge() 
>> in drivers/pci/probe.c:
>>
>> if (pcibios_assign_all_busses() || broken)
>>                 /* Temporarily disable forwarding of the
>>                    configuration cycles on all bridges in
>>                    this bus segment to avoid possible
>>                    conflicts in the second pass between two
>>                    bridges programmed with overlapping
>>                    bus ranges. */
>>                 pci_write_config_dword(dev, PCI_PRIMARY_BUS,
>>                                buses & ~0xffffff);
>>
>> If test for broken is removed, kernel boots fine, detects the bridge, but
>> does not detect the device behind the bridge. The same device plugged
>> directly into PCI slot is detected correctly.
>>     
>
> So we would have a similar mismatch between the initial setup and the
> kernel...  However, I don't quite see yet why the kernel trying to fix
> it up breaks things, that will need a bit more debugging here...
>
> Can you give it a quick try with adding something like :
>
>  ppc_pci_add_flags(PPC_PCI_REASSIGN_ALL_BUS);
>
> Near the end of ppc4xx_pci.c ? It looks like another case of reset
> not actually resetting bridges (are we not properly doing a fundamental
> reset ? Stefan what's your take there ?)
>
> The above will cause busses to be re-assigned which is risky because it
> will allow the kernel to assign numbers beyond the limits of what
> ppc4xx_pci.c supports (see my comments in the thread you quotes).
>
> The good thing is that we now have a working fixmap infrastructure, so
> we could/should just move ppc4xx_pci.c to use that, and just always
> re-assign busses.
>
>   
>> To remind you, tests for broken were added by commit 
>> a1c19894b786f10c76ac40e93c6b5d70c9b946d2,
>> and were intended to solve device detection problem behind PCI-E 
>> switches, as discussed in this thread:
>> http://lists.ozlabs.org/pipermail/linuxppc-dev/2008-October/063939.html
>>     
>
>   
>> PCI: Probing PCI hardware
>> pci_bus 0000:00: scanning bus
>> pci 0000:00:06.0: found [3388:0020] class 000604 header type 01
>> pci 0000:00:06.0: supports D1 D2
>> pci 0000:00:06.0: PME# supported from D0 D1 D2 D3hot
>> pci 0000:00:06.0: PME# disabled
>> pci_bus 0000:00: fixups for bus
>> pci 0000:00:06.0: scanning behind bridge, config 000000, pass 0
>> pci 0000:00:06.0: bus configuration invalid, reconfiguring
>>     
>
> Ok so we hit a P2P bridge whose primary, secondary and subordinate bus
> numbers are all 0, which is clearly unconfigured. I think this is the
> root complex bridge
>
>   
>> pci 0000:00:06.0: scanning behind bridge, config 000000, pass 1
>>     
>
> Now this is when the bus should be reconfigured (pass 1). Sadly the code
> doesn't print much debug.
>
> Also from that point, it should renumber things and work... 
>
>   
>> pci_bus 0000:01: scanning bus
>>     
>
> Which it does to some extent. It assigned bus number 1 to it afaik so we
> now start looking below the RC bridge:
>
>   
>> pci 0000:01:06.0: found [3388:0020] class 000604 header type 01
>>     
>
> Hrm... class PCI bridge, vendor 3388 device 0020, is that your PLX ?
> It's not the right vendor ID but maybe that's configurable by our OEM or
> something...
>   

The bridge and its evaluation board were manufactured by HiNT, later 
purchased by PLX.
3388:0020 is HiNT HB6 Universal PCI-PCI bridge in transparent mode.

>   
>> pci 0000:01:06.0: supports D1 D2
>> pci 0000:01:06.0: PME# supported from D0 D1 D2 D3hot
>> pci 0000:01:06.0: PME# disabled
>> pci_bus 0000:01: fixups for bus
>> pci 0000:00:06.0: PCI bridge to [bus 01-ff]
>> pci 0000:00:06.0:   bridge window [io  0x0000-0x0fff]
>> pci 0000:00:06.0:   bridge window [mem 0x00000000-0x000fffff]
>> pci 0000:00:06.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
>> pci 0000:01:06.0: scanning behind bridge, config ff0100, pass 0
>>     
>
> Allright, that's where it gets interesting. It tries to scan behind the
> bridge. It gets something it doesn't like. IE, it gets a secondary bus
> number of 1 (what the heck ? I wonder what your firmware does) which
> Linux is not happy about and decides to renumber it.
>   

U-boot has problems with this bridge as well, so I had to completely 
disable PCI
support in u-boot to get linux running.
>   
>> pci 0000:01:06.0: bus configuration invalid, reconfiguring
>>     
>
> Now, that's where Linux should have written 000000 to the register,
> which is what you commented out.
>
>   
>> pci 0000:01:06.0: scanning behind bridge, config ff0100, pass 1
>> pci_bus 0000:01: bus scan returning with max=01
>> pci_bus 0000:00: bus scan returning with max=01
>>     
>
> Because of that commenting out, it doesn't see the config as 000000 and
> thus doesn't re-assign a bus number in pass 1, so from there you can't
> see what's behind the bus.
>
> So we have two things here:
>
>  - It seems like the writing of 000000 to the register in pass 0 is
> causing your crash. Can you verify that ? IE. Can you verify that it's
> indeed crashing on this specific statement:
>
> pci_write_config_dword(dev, PCI_PRIMARY_BUS,
>                                buses & ~0xffffff);
>
> When writing to the bridge, and that this seems to be causing a hard
> reboot of the system ?
>   

Yes, this particular statement causes hard reboot. With original broken 
tests restored
and writing to bridge commented out the system boots. If writing to 
bridge happens
I get hard reset.

> It might be useful to ask AMCC how that is possible in HW, ie what kind
> of signal can be causing that. IE, even if the bridge is causing a PCIe
> error, that should not cause a reboot ... right ?
>   

Feng, can you please comment on this ?
>  - You can test a quick hack workaround which consists of changing:
>
> 	/* Check if setup is sensible at all */
> -	if (!pass &&
> -	if (1 &&
> 	    ((buses & 0xff) != bus->number || ((buses >> 8) & 0xff) <= bus->number)) {
> 		dev_dbg(&dev->dev, "bus configuration invalid, reconfiguring\n");
> 		broken = 1;
> 	}
>
> In -addition- to your commenting out of the broken test. This will cause the
> second pass to go through the re-assign code path despite the fact that you
> have not written 000000 to the bus numbers.
>   

With this change and commented out broken test I still get hard reset.

I didn't try adding ppc_pci_add_flags(PPC_PCI_REASSIGN_ALL_BUS)
If you still want me to try this, please let me know. Should I leave broken
tests enabled in that case ?

Thanks a lot for your help.

Felix.

Re: PCI-PCI bridge scanning broken on 460EX

[Felix Radensky]

Hi, Ben

Felix Radensky wrote:
> Hi, Ben
>
> Adding Feng Kan from AMCC to CC.
>
> Benjamin Herrenschmidt wrote:
>> On Mon, 2009-12-28 at 12:51 +0200, Felix Radensky wrote:
>>  
>>> Hi,
>>>
>>> I'm running linux-2.6.33-rc2 on Canyonlands board. When PLX 6254 
>>> transparent PCI-PCI
>>> bridge is plugged into PCI slot the kernel simply resets the board 
>>> without printing anything
>>> to console. Without PLX bridge kernel boots fine.
>>>     
>>
>> Sorry for the late reply...
>>   
>
> No need to apologize, I appreciate you help very much.
>
>>  
>>> I've tracked down the problem to the following code in 
>>> pci_scan_bridge() in drivers/pci/probe.c:
>>>
>>> if (pcibios_assign_all_busses() || broken)
>>>                 /* Temporarily disable forwarding of the
>>>                    configuration cycles on all bridges in
>>>                    this bus segment to avoid possible
>>>                    conflicts in the second pass between two
>>>                    bridges programmed with overlapping
>>>                    bus ranges. */
>>>                 pci_write_config_dword(dev, PCI_PRIMARY_BUS,
>>>                                buses & ~0xffffff);
>>>
>>> If test for broken is removed, kernel boots fine, detects the 
>>> bridge, but
>>> does not detect the device behind the bridge. The same device plugged
>>> directly into PCI slot is detected correctly.
>>>     
>>
>> So we would have a similar mismatch between the initial setup and the
>> kernel...  However, I don't quite see yet why the kernel trying to fix
>> it up breaks things, that will need a bit more debugging here...
>>
>> Can you give it a quick try with adding something like :
>>
>>  ppc_pci_add_flags(PPC_PCI_REASSIGN_ALL_BUS);
>>
>> Near the end of ppc4xx_pci.c ? It looks like another case of reset
>> not actually resetting bridges (are we not properly doing a fundamental
>> reset ? Stefan what's your take there ?)
>>
>> The above will cause busses to be re-assigned which is risky because it
>> will allow the kernel to assign numbers beyond the limits of what
>> ppc4xx_pci.c supports (see my comments in the thread you quotes).
>>
>> The good thing is that we now have a working fixmap infrastructure, so
>> we could/should just move ppc4xx_pci.c to use that, and just always
>> re-assign busses.
>>
>>  
>>> To remind you, tests for broken were added by commit 
>>> a1c19894b786f10c76ac40e93c6b5d70c9b946d2,
>>> and were intended to solve device detection problem behind PCI-E 
>>> switches, as discussed in this thread:
>>> http://lists.ozlabs.org/pipermail/linuxppc-dev/2008-October/063939.html
>>>     
>>
>>  
>>> PCI: Probing PCI hardware
>>> pci_bus 0000:00: scanning bus
>>> pci 0000:00:06.0: found [3388:0020] class 000604 header type 01
>>> pci 0000:00:06.0: supports D1 D2
>>> pci 0000:00:06.0: PME# supported from D0 D1 D2 D3hot
>>> pci 0000:00:06.0: PME# disabled
>>> pci_bus 0000:00: fixups for bus
>>> pci 0000:00:06.0: scanning behind bridge, config 000000, pass 0
>>> pci 0000:00:06.0: bus configuration invalid, reconfiguring
>>>     
>>
>> Ok so we hit a P2P bridge whose primary, secondary and subordinate bus
>> numbers are all 0, which is clearly unconfigured. I think this is the
>> root complex bridge
>>
>>  
>>> pci 0000:00:06.0: scanning behind bridge, config 000000, pass 1
>>>     
>>
>> Now this is when the bus should be reconfigured (pass 1). Sadly the code
>> doesn't print much debug.
>>
>> Also from that point, it should renumber things and work...
>>  
>>> pci_bus 0000:01: scanning bus
>>>     
>>
>> Which it does to some extent. It assigned bus number 1 to it afaik so we
>> now start looking below the RC bridge:
>>
>>  
>>> pci 0000:01:06.0: found [3388:0020] class 000604 header type 01
>>>     
>>
>> Hrm... class PCI bridge, vendor 3388 device 0020, is that your PLX ?
>> It's not the right vendor ID but maybe that's configurable by our OEM or
>> something...
>>   
>
> The bridge and its evaluation board were manufactured by HiNT, later 
> purchased by PLX.
> 3388:0020 is HiNT HB6 Universal PCI-PCI bridge in transparent mode.
>
>>  
>>> pci 0000:01:06.0: supports D1 D2
>>> pci 0000:01:06.0: PME# supported from D0 D1 D2 D3hot
>>> pci 0000:01:06.0: PME# disabled
>>> pci_bus 0000:01: fixups for bus
>>> pci 0000:00:06.0: PCI bridge to [bus 01-ff]
>>> pci 0000:00:06.0:   bridge window [io  0x0000-0x0fff]
>>> pci 0000:00:06.0:   bridge window [mem 0x00000000-0x000fffff]
>>> pci 0000:00:06.0:   bridge window [mem 0x00000000-0x000fffff 64bit 
>>> pref]
>>> pci 0000:01:06.0: scanning behind bridge, config ff0100, pass 0
>>>     
>>
>> Allright, that's where it gets interesting. It tries to scan behind the
>> bridge. It gets something it doesn't like. IE, it gets a secondary bus
>> number of 1 (what the heck ? I wonder what your firmware does) which
>> Linux is not happy about and decides to renumber it.
>>   
>
> U-boot has problems with this bridge as well, so I had to completely 
> disable PCI
> support in u-boot to get linux running.
>>  
>>> pci 0000:01:06.0: bus configuration invalid, reconfiguring
>>>     
>>
>> Now, that's where Linux should have written 000000 to the register,
>> which is what you commented out.
>>
>>  
>>> pci 0000:01:06.0: scanning behind bridge, config ff0100, pass 1
>>> pci_bus 0000:01: bus scan returning with max=01
>>> pci_bus 0000:00: bus scan returning with max=01
>>>     
>>
>> Because of that commenting out, it doesn't see the config as 000000 and
>> thus doesn't re-assign a bus number in pass 1, so from there you can't
>> see what's behind the bus.
>>
>> So we have two things here:
>>
>>  - It seems like the writing of 000000 to the register in pass 0 is
>> causing your crash. Can you verify that ? IE. Can you verify that it's
>> indeed crashing on this specific statement:
>>
>> pci_write_config_dword(dev, PCI_PRIMARY_BUS,
>>                                buses & ~0xffffff);
>>
>> When writing to the bridge, and that this seems to be causing a hard
>> reboot of the system ?
>>   
>
> Yes, this particular statement causes hard reboot. With original 
> broken tests restored
> and writing to bridge commented out the system boots. If writing to 
> bridge happens
> I get hard reset.
>
>> It might be useful to ask AMCC how that is possible in HW, ie what kind
>> of signal can be causing that. IE, even if the bridge is causing a PCIe
>> error, that should not cause a reboot ... right ?
>>   
>
> Feng, can you please comment on this ?
>>  - You can test a quick hack workaround which consists of changing:
>>
>>     /* Check if setup is sensible at all */
>> -    if (!pass &&
>> -    if (1 &&
>>         ((buses & 0xff) != bus->number || ((buses >> 8) & 0xff) <= 
>> bus->number)) {
>>         dev_dbg(&dev->dev, "bus configuration invalid, 
>> reconfiguring\n");
>>         broken = 1;
>>     }
>>
>> In -addition- to your commenting out of the broken test. This will 
>> cause the
>> second pass to go through the re-assign code path despite the fact 
>> that you
>> have not written 000000 to the bus numbers.
>>   
>
> With this change and commented out broken test I still get hard reset.
>
> I didn't try adding ppc_pci_add_flags(PPC_PCI_REASSIGN_ALL_BUS)
> If you still want me to try this, please let me know. Should I leave 
> broken
> tests enabled in that case ?
>
> Thanks a lot for your help.
>
> Felix.
I now have a custom board with 460EX and the same PLX bridge, running 
2.6.23-rc3
Things look better here, as u-boot is now able to properly detect PLX 
and device behind
it, but kernel still has problems. First, I'm still getting hard reset on

pci_write_config_dword(dev, PCI_PRIMARY_BUS,
                               buses & ~0xffffff);

If this line is removed, PLX is detected twice, see below. I also get 
hard reset
if pass test is modified as you requested and broken test removed.

Any ideas how to fix this ? I was suspecting PLX evaluation board, but
PLX on our custom board seems to be OK, so it looks like kernel needs 
fixing.

PCI: Probing PCI hardware
pci_bus 0000:00: scanning bus
pci 0000:00:02.0: found [3388:0020] class 000604 header type 01
pci 0000:00:02.0: calling pcibios_fixup_resources+0x0/0xf4
pci 0000:00:02.0: calling fixup_ppc4xx_pci_bridge+0x0/0x154
pci 0000:00:02.0: calling quirk_resource_alignment+0x0/0x200
pci 0000:00:02.0: supports D1 D2
pci 0000:00:02.0: PME# supported from D0 D1 D2 D3hot
pci 0000:00:02.0: PME# disabled
pci_bus 0000:00: fixups for bus
pci 0000:00:02.0: scanning behind bridge, config 010100, pass 0
pci_bus 0000:01: scanning bus
pci 0000:01:02.0: found [3388:0020] class 000604 header type 01
pci 0000:01:02.0: calling pcibios_fixup_resources+0x0/0xf4
pci 0000:01:02.0: calling fixup_ppc4xx_pci_bridge+0x0/0x154
pci 0000:01:02.0: calling quirk_resource_alignment+0x0/0x200
pci 0000:01:02.0: supports D1 D2
pci 0000:01:02.0: PME# supported from D0 D1 D2 D3hot
pci 0000:01:02.0: PME# disabled
pci_bus 0000:01: fixups for bus
pci 0000:00:02.0: PCI bridge to [bus 01-01]
pci 0000:01:02.0: scanning behind bridge, config 010100, pass 0
pci 0000:01:02.0: bus configuration invalid, reconfiguring
pci 0000:01:02.0: scanning behind bridge, config 010100, pass 1
pci_bus 0000:01: bus scan returning with max=01
pci 0000:00:02.0: scanning behind bridge, config 010100, pass 1
pci_bus 0000:00: bus scan returning with max=01
pci 0000:00:02.0: PCI bridge to [bus 01-01]
pci 0000:00:02.0:   bridge window [io  disabled]
pci 0000:00:02.0:   bridge window [mem disabled]
pci 0000:00:02.0:   bridge window [mem pref disabled]
pci_bus 0000:00: resource 0 [io  0x0000-0xffff]
pci_bus 0000:00: resource 1 [mem 0xd80000000-0xdffffffff]

Thanks.

Felix.

Re: PCI-PCI bridge scanning broken on 460EX

[Benjamin Herrenschmidt]

On Sun, 2010-01-10 at 14:56 +0200, Felix Radensky wrote:

> I now have a custom board with 460EX and the same PLX bridge, running 
> 2.6.23-rc3
> Things look better here, as u-boot is now able to properly detect PLX 
> and device behind
> it, but kernel still has problems. First, I'm still getting hard reset on
> 
> pci_write_config_dword(dev, PCI_PRIMARY_BUS,
>                                buses & ~0xffffff);
> 
> If this line is removed, PLX is detected twice, see below. I also get 
> hard reset
> if pass test is modified as you requested and broken test removed.
> 
> Any ideas how to fix this ? I was suspecting PLX evaluation board, but
> PLX on our custom board seems to be OK, so it looks like kernel needs 
> fixing.

I have no idea no. It looks like something is wrong with the PLX bridge
but again, I don't know why that would cause the 460EX to hard reset
like that, unless some of the PCIe error handling of the 460 has been
configured to cause such a reset on some kind of errors (which it
shouldn't at least not in host mode).

Can you try instead of writing all the bus number related registers in
one single dword write above, writing them byte by byte ? Which one is
causing the reset ? Does it reset whatever the value you write there
is ?

It looks like something is causing a hard reset as soon as you try to
configure the PLX bridge and without configuring it properly I fail to
see how you'll get things working.

Cheers,
Ben.

Re: PCI-PCI bridge scanning broken on 460EX

[Felix Radensky]

Benjamin Herrenschmidt wrote:
> On Sun, 2010-01-10 at 14:56 +0200, Felix Radensky wrote:
>
>   
>> I now have a custom board with 460EX and the same PLX bridge, running 
>> 2.6.23-rc3
>> Things look better here, as u-boot is now able to properly detect PLX 
>> and device behind
>> it, but kernel still has problems. First, I'm still getting hard reset on
>>
>> pci_write_config_dword(dev, PCI_PRIMARY_BUS,
>>                                buses & ~0xffffff);
>>
>> If this line is removed, PLX is detected twice, see below. I also get 
>> hard reset
>> if pass test is modified as you requested and broken test removed.
>>
>> Any ideas how to fix this ? I was suspecting PLX evaluation board, but
>> PLX on our custom board seems to be OK, so it looks like kernel needs 
>> fixing.
>>     
>
> I have no idea no. It looks like something is wrong with the PLX bridge
> but again, I don't know why that would cause the 460EX to hard reset
> like that, unless some of the PCIe error handling of the 460 has been
> configured to cause such a reset on some kind of errors (which it
> shouldn't at least not in host mode).
>
> Can you try instead of writing all the bus number related registers in
> one single dword write above, writing them byte by byte ? Which one is
> causing the reset ? Does it reset whatever the value you write there
> is ?
>
> It looks like something is causing a hard reset as soon as you try to
> configure the PLX bridge and without configuring it properly I fail to
> see how you'll get things working.
>
>   

OK, I'll try writing byte by byte. The funny thing is the u-boot also 
writes the
same value to PCI_PRIMARY_BUS register and it doesn't cause reset.

Thanks a lot.

Felix.

Re: PCI-PCI bridge scanning broken on 460EX

[Benjamin Herrenschmidt]

> OK, I'll try writing byte by byte. The funny thing is the u-boot also 
> writes the
> same value to PCI_PRIMARY_BUS register and it doesn't cause reset.

Maybe the bridge doesn't want to be programmed more than once on these
registers ? In any case, that's very very fishy.... I wonder if the
bridge is causing a PCI reset -upstream- (which would really be a weird
thing to do) and the 460 is turning that into a system reset ? Check if
there are ways to control how the 460 reacts to PCI resets...

In any case, it looks like a fucked up bridge to me. I don't suppose
you've seen anything in the bridge data sheet or errata sheet that could
explain what it's doing ?

Cheers,
Ben.

RE: PCI-PCI bridge scanning broken on 460EX

[Stef van Os]

Hello=20Felix,

I=20had=20a=20problem=20similar=20to=20this=20on=20the=20440GX,=20the=20P=
CI=20code=20was=20not
sending=20type=201=20transactions=20when=20scanning=20behind=20bridges.=
=20Perhaps=20you
could=20try=20this:

Index:=20linux/arch/powerpc/sysdev/ppc4xx_pci.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
---=20linux/arch/powerpc/sysdev/ppc4xx_pci.c=20=20=20=20=20=20(revision=
=2026)
+++=20linux/arch/powerpc/sysdev/ppc4xx_pci.c=20=20=20=20=20=20(revision=
=2027)
@@=20-569,7=20+569,7=20@@
=20=20=20=20=20=20=20=20hose->last_busno=20=3D=20bus_range=20?=20bus_rang=
e[1]=20:=200xff;

=20=20=20=20=20=20=20=20/*=20Setup=20config=20space=20*/
-=20=20=20=20=20=20=20setup_indirect_pci(hose,=20rsrc_cfg.start,=20rsrc_c=
fg.start=20+=200x4,
0);
+=20=20=20=20=20=20=20setup_indirect_pci(hose,=20rsrc_cfg.start,=20rsrc_c=
fg.start=20+=200x4,
PPC_INDIRECT_TYPE_SET_CFG_TYPE);

=20=20=20=20=20=20=20=20/*=20Disable=20all=20windows=20*/
=20=20=20=20=20=20=20=20writel(0,=20reg=20+=20PCIX0_POM0SA);



With=20kind=20regards=20/=20Met=20vriendelijke=20groet,

Stef=20van=20Os

Prodrive=20B.V.=20


-----Original=20Message-----
From:=20linuxppc-dev-bounces+stef.van.os=3Dprodrive.nl@lists.ozlabs.org
[mailto:linuxppc-dev-bounces+stef.van.os=3Dprodrive.nl@lists.ozlabs.org]
On=20Behalf=20Of=20Benjamin=20Herrenschmidt
Sent:=20zondag=2010=20januari=202010=2022:32
To:=20Felix=20Radensky
Cc:=20linuxppc-dev@ozlabs.org;=20Stefan=20Roese;=20Feng=20Kan
Subject:=20Re:=20PCI-PCI=20bridge=20scanning=20broken=20on=20460EX


>=20OK,=20I'll=20try=20writing=20byte=20by=20byte.=20The=20funny=20thing=
=20is=20the=20u-boot=20also=20
>=20writes=20the=20same=20value=20to=20PCI_PRIMARY_BUS=20register=20and=
=20it=20doesn't=20cause

>=20reset.

Maybe=20the=20bridge=20doesn't=20want=20to=20be=20programmed=20more=20tha=
n=20once=20on=20these
registers=20?=20In=20any=20case,=20that's=20very=20very=20fishy....=20I=
=20wonder=20if=20the
bridge=20is=20causing=20a=20PCI=20reset=20-upstream-=20(which=20would=20r=
eally=20be=20a=20weird
thing=20to=20do)=20and=20the=20460=20is=20turning=20that=20into=20a=20sys=
tem=20reset=20?=20Check=20if
there=20are=20ways=20to=20control=20how=20the=20460=20reacts=20to=20PCI=
=20resets...

In=20any=20case,=20it=20looks=20like=20a=20fucked=20up=20bridge=20to=20me=
.=20I=20don't=20suppose
you've=20seen=20anything=20in=20the=20bridge=20data=20sheet=20or=20errata=
=20sheet=20that=20could
explain=20what=20it's=20doing=20?

Cheers,
Ben.


_______________________________________________
Linuxppc-dev=20mailing=20list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Disclaimer:=20The=20information=20contained=20in=20this=20email,=20includ=
ing=20any=20attachments=20is=20
confidential=20and=20is=20for=20the=20sole=20use=20of=20the=20intended=20=
recipient(s).=20Any=20unauthorized=20
review,=20use,=20disclosure=20or=20distribution=20is=20prohibited.=20If=
=20you=20are=20not=20the=20intended=20
recipient,=20please=20notify=20the=20sender=20immediately=20by=20replying=
=20to=20this=20message=20and=20
destroy=20all=20copies=20of=20this=20message=20and=20any=20attachments.

Re: PCI-PCI bridge scanning broken on 460EX

[Felix Radensky]

Hi Stef,

Stef van Os wrote:
> Hello Felix,
>
> I had a problem similar to this on the 440GX, the PCI code was not
> sending type 1 transactions when scanning behind bridges. Perhaps you
> could try this:
>
> Index: linux/arch/powerpc/sysdev/ppc4xx_pci.c
> ===================================================================
> --- linux/arch/powerpc/sysdev/ppc4xx_pci.c      (revision 26)
> +++ linux/arch/powerpc/sysdev/ppc4xx_pci.c      (revision 27)
> @@ -569,7 +569,7 @@
>         hose->last_busno = bus_range ? bus_range[1] : 0xff;
>
>         /* Setup config space */
> -       setup_indirect_pci(hose, rsrc_cfg.start, rsrc_cfg.start + 0x4,
> 0);
> +       setup_indirect_pci(hose, rsrc_cfg.start, rsrc_cfg.start + 0x4,
> PPC_INDIRECT_TYPE_SET_CFG_TYPE);
>
>         /* Disable all windows */
>         writel(0, reg + PCIX0_POM0SA);
>
>
>
> With kind regards / Met vriendelijke groet,
>
> Stef van Os
>
> Prodrive B.V. 
>
>
>   

I think you patch is a valid one, and should be applied, but 
unfortunately it doesn't fix by problem.
BTW, in u-boot transaction type bit is set correctly.

Felix.

Re: PCI-PCI bridge scanning broken on 460EX

[Felix Radensky]

Hi Stef

Felix Radensky wrote:
> Hi Stef,
>
> Stef van Os wrote:
>> Hello Felix,
>>
>> I had a problem similar to this on the 440GX, the PCI code was not
>> sending type 1 transactions when scanning behind bridges. Perhaps you
>> could try this:
>>
>> Index: linux/arch/powerpc/sysdev/ppc4xx_pci.c
>> ===================================================================
>> --- linux/arch/powerpc/sysdev/ppc4xx_pci.c      (revision 26)
>> +++ linux/arch/powerpc/sysdev/ppc4xx_pci.c      (revision 27)
>> @@ -569,7 +569,7 @@
>>         hose->last_busno = bus_range ? bus_range[1] : 0xff;
>>
>>         /* Setup config space */
>> -       setup_indirect_pci(hose, rsrc_cfg.start, rsrc_cfg.start + 0x4,
>> 0);
>> +       setup_indirect_pci(hose, rsrc_cfg.start, rsrc_cfg.start + 0x4,
>> PPC_INDIRECT_TYPE_SET_CFG_TYPE);
>>
>>         /* Disable all windows */
>>         writel(0, reg + PCIX0_POM0SA);
>>
>>
>>
>> With kind regards / Met vriendelijke groet,
>>
>> Stef van Os
>>
>> Prodrive B.V.
>>
>>   
>
> I think you patch is a valid one, and should be applied, but 
> unfortunately it doesn't fix by problem.
> BTW, in u-boot transaction type bit is set correctly.
>
>

It seems I was wrong. I've manually applied the patch at the wrong 
place. After patching the correct function
I'm not getting hard resets any more, which is a great improvement ! 
Thanks a lot, I really appreciate your help !

Unfortunately not all problems are gone. PLX is now identified 
correctly, but device behind it is not detected,
although u-boot detects it correctly. See below.

PCI: Probing PCI hardware
pci_bus 0000:00: scanning bus
pci 0000:00:02.0: found [3388:0020] class 000604 header type 01
pci 0000:00:02.0: calling pcibios_fixup_resources+0x0/0xf4
pci 0000:00:02.0: calling fixup_ppc4xx_pci_bridge+0x0/0x154
pci 0000:00:02.0: calling quirk_resource_alignment+0x0/0x200
pci 0000:00:02.0: supports D1 D2
pci 0000:00:02.0: PME# supported from D0 D1 D2 D3hot
pci 0000:00:02.0: PME# disabled
pci_bus 0000:00: fixups for bus
pci 0000:00:02.0: scanning behind bridge, config 010100, pass 0
pci_bus 0000:01: scanning bus
pci 0000:01:02.0: found [3388:0020] class 000604 header type 01
pci 0000:01:02.0: calling pcibios_fixup_resources+0x0/0xf4
pci 0000:01:02.0: calling fixup_ppc4xx_pci_bridge+0x0/0x154
pci 0000:01:02.0: calling quirk_resource_alignment+0x0/0x200
pci 0000:01:02.0: supports D1 D2
pci 0000:01:02.0: PME# supported from D0 D1 D2 D3hot
pci 0000:01:02.0: PME# disabled
pci_bus 0000:01: fixups for bus
pci 0000:00:02.0: PCI bridge to [bus 01-01]
pci 0000:00:02.0:   bridge window [mem 0x80000000-0x8c0fffff]
pci 0000:01:02.0: scanning behind bridge, config 010100, pass 0
pci 0000:01:02.0: bus configuration invalid, reconfiguring
pci 0000:01:02.0: scanning behind bridge, config 010100, pass 1
pci_bus 0000:01: bus scan returning with max=01
pci 0000:00:02.0: scanning behind bridge, config 010100, pass 1
pci_bus 0000:00: bus scan returning with max=01
pci 0000:00:02.0: disabling bridge window [mem 0xd80000000-0xd8c0fffff] 
to [bus 01-01] (unused)
pci 0000:00:02.0: PCI bridge to [bus 01-01]
pci 0000:00:02.0:   bridge window [io  disabled]
pci 0000:00:02.0:   bridge window [mem disabled]
pci 0000:00:02.0:   bridge window [mem pref disabled]
pci_bus 0000:00: resource 0 [io  0x0000-0xffff]
pci_bus 0000:00: resource 1 [mem 0xd80000000-0xdffffffff]
pci_bus 0000:01: resource 1 [??? 57982058496-58184433663 flags 0x0]

Any ideas what could be wrong now ?

Thanks a lot.

Felix.

Re: PCI-PCI bridge scanning broken on 460EX

[Benjamin Herrenschmidt]

> It seems I was wrong. I've manually applied the patch at the wrong 
> place. After patching the correct function
> I'm not getting hard resets any more, which is a great improvement ! 
> Thanks a lot, I really appreciate your help !

This is somewhat funny... I wonder how it would have managed to find
anything behind the root complex P2P bridge with broken type 1 cycles...
very very strange.

> Unfortunately not all problems are gone. PLX is now identified 
> correctly, but device behind it is not detected,
> although u-boot detects it correctly. See below.

You have removed all your changes to that code right ?

Also the log still looks weird:

> pci 0000:01:02.0: scanning behind bridge, config 010100, pass 0
> pci 0000:01:02.0: bus configuration invalid, reconfiguring
> pci 0000:01:02.0: scanning behind bridge, config 010100, pass 1
> pci_bus 0000:01: bus scan returning with max=01
> pci 0000:00:02.0: scanning behind bridge, config 010100, pass 1
> pci_bus 0000:00: bus scan returning with max=01

Unless you left some experimental changes in, the above isn't right, the
"config" value should have changed due to the write of ~ffffff to it

If the code running is indeed unmodified from upsteam, can you try with
just that change:

        /* Check if setup is sensible at all */
-        if (!pass &&
-            ((buses & 0xff) != bus->number || ((buses >> 8) & 0xff) <= bus->number)) {
+        if (((buses & 0xff) != bus->number || ((buses >> 8) & 0xff) <= bus->number)) {
                 dev_dbg(&dev->dev, "bus configuration invalid, reconfiguring\n");
                 broken = 1;
         }

(IE remove the check for !pass)

Cheers,
Ben.

> pci 0000:00:02.0: disabling bridge window [mem 0xd80000000-0xd8c0fffff] 
> to [bus 01-01] (unused)
> pci 0000:00:02.0: PCI bridge to [bus 01-01]
> pci 0000:00:02.0:   bridge window [io  disabled]
> pci 0000:00:02.0:   bridge window [mem disabled]
> pci 0000:00:02.0:   bridge window [mem pref disabled]
> pci_bus 0000:00: resource 0 [io  0x0000-0xffff]
> pci_bus 0000:00: resource 1 [mem 0xd80000000-0xdffffffff]
> pci_bus 0000:01: resource 1 [??? 57982058496-58184433663 flags 0x0]
> 
> Any ideas what could be wrong now ?
> 
> Thanks a lot.
> 
> Felix.
> 

Re: PCI-PCI bridge scanning broken on 460EX

[Felix Radensky]

Hi, Ben

Benjamin Herrenschmidt wrote:
>> It seems I was wrong. I've manually applied the patch at the wrong 
>> place. After patching the correct function
>> I'm not getting hard resets any more, which is a great improvement ! 
>> Thanks a lot, I really appreciate your help !
>>     
>
> This is somewhat funny... I wonder how it would have managed to find
> anything behind the root complex P2P bridge with broken type 1 cycles...
> very very strange.
>   

Maybe because the bus behind root P2P bridge is bus 0, and type 1 cycles are
needed for bus numbers greater than 0. That's what 460EX manual says.
>   
>> Unfortunately not all problems are gone. PLX is now identified 
>> correctly, but device behind it is not detected,
>> although u-boot detects it correctly. See below.
>>     
>
> You have removed all your changes to that code right ?
>   

Yes, I've removed all experimental changes.

> Also the log still looks weird:
>
>   
>> pci 0000:01:02.0: scanning behind bridge, config 010100, pass 0
>> pci 0000:01:02.0: bus configuration invalid, reconfiguring
>> pci 0000:01:02.0: scanning behind bridge, config 010100, pass 1
>> pci_bus 0000:01: bus scan returning with max=01
>> pci 0000:00:02.0: scanning behind bridge, config 010100, pass 1
>> pci_bus 0000:00: bus scan returning with max=01
>>     
>
> Unless you left some experimental changes in, the above isn't right, the
> "config" value should have changed due to the write of ~ffffff to it
>   

You are correct, the log is from older version, without Stef's fix. I 
don't have access to a
system with devices behind PLX, and the guy who did the testing used 
wrong kernel.
I'll make sure he uses the correct one and get back to you. Maybe 
everything works after all :)
I'm really sorry for confusion.

Felix.

Re: PCI-PCI bridge scanning broken on 460EX

[Benjamin Herrenschmidt]

On Tue, 2010-01-12 at 00:48 +0200, Felix Radensky wrote:
> 
> Maybe because the bus behind root P2P bridge is bus 0, and type 1
> cycles are
> needed for bus numbers greater than 0. That's what 460EX manual says.

Well, no... the bus behind the root P2P is bus 1 ... the root P2P itself
is on bus 0... but then, it's some trick in the way they implemented it
I suppose.

> You are correct, the log is from older version, without Stef's fix. I 
> don't have access to a
> system with devices behind PLX, and the guy who did the testing used 
> wrong kernel.
> I'll make sure he uses the correct one and get back to you. Maybe 
> everything works after all :)
> I'm really sorry for confusion.

No worries :-) Feel free to send a proper patch to fix that problem
upstream too !

Cheers,
Ben.

Re: PCI-PCI bridge scanning broken on 460EX

[Felix Radensky]

Hi Ben

Benjamin Herrenschmidt wrote:
> On Tue, 2010-01-12 at 00:48 +0200, Felix Radensky wrote:
>   
>> Maybe because the bus behind root P2P bridge is bus 0, and type 1
>> cycles are
>> needed for bus numbers greater than 0. That's what 460EX manual says.
>>     
>
> Well, no... the bus behind the root P2P is bus 1 ... the root P2P itself
> is on bus 0... but then, it's some trick in the way they implemented it
> I suppose.
>
>   
>> You are correct, the log is from older version, without Stef's fix. I 
>> don't have access to a
>> system with devices behind PLX, and the guy who did the testing used 
>> wrong kernel.
>> I'll make sure he uses the correct one and get back to you. Maybe 
>> everything works after all :)
>> I'm really sorry for confusion.
>>     
>
> No worries :-) Feel free to send a proper patch to fix that problem
> upstream too !
>
>   
The kernel with Stef's fix works fine, and recognizes both PLX and 
device behind it.
Stef,  do want to provide a patch for upstream kernel, or do you want me 
to do that on
your behalf ?

Thanks a lot everyone for you help !

Felix.
Felix.

RE: PCI-PCI bridge scanning broken on 460EX

[Stef van Os]

Hello=20Felix,

Glad=20to=20know=20this=20is=20working=20for=20you!

I'll=20try=20to=20send=20out=20a=20patch=20later=20today.=20The=20same=20=
change=20should=20also=20be
applied=20to=20the=20"ppc4xx_probe_pci_bridge"=20function,=20as=20it=20al=
so
initialises=20PCI=20without=20type=201=20transactions.

With=20kind=20regards=20/=20Met=20vriendelijke=20groet,

Stef=20van=20Os

Prodrive=20B.V.=20


-----Original=20Message-----
From:=20Felix=20Radensky=20[mailto:felix@embedded-sol.com]=20
Sent:=20dinsdag=2012=20januari=202010=2012:03
To:=20Benjamin=20Herrenschmidt
Cc:=20Stef=20van=20Os;=20Stefan=20Roese;=20Feng=20Kan;=20linuxppc-dev@ozl=
abs.org
Subject:=20Re:=20PCI-PCI=20bridge=20scanning=20broken=20on=20460EX

Hi=20Ben

Benjamin=20Herrenschmidt=20wrote:
>=20On=20Tue,=202010-01-12=20at=2000:48=20+0200,=20Felix=20Radensky=20wro=
te:
>=20=20=20
>>=20Maybe=20because=20the=20bus=20behind=20root=20P2P=20bridge=20is=20bu=
s=200,=20and=20type=201=20
>>=20cycles=20are=20needed=20for=20bus=20numbers=20greater=20than=200.=20=
That's=20what=20460EX=20
>>=20manual=20says.
>>=20=20=20=20=20
>
>=20Well,=20no...=20the=20bus=20behind=20the=20root=20P2P=20is=20bus=201=
=20...=20the=20root=20P2P=20
>=20itself=20is=20on=20bus=200...=20but=20then,=20it's=20some=20trick=20i=
n=20the=20way=20they=20
>=20implemented=20it=20I=20suppose.
>
>=20=20=20
>>=20You=20are=20correct,=20the=20log=20is=20from=20older=20version,=20wi=
thout=20Stef's=20fix.=20I

>>=20don't=20have=20access=20to=20a=20system=20with=20devices=20behind=20=
PLX,=20and=20the=20guy=20
>>=20who=20did=20the=20testing=20used=20wrong=20kernel.
>>=20I'll=20make=20sure=20he=20uses=20the=20correct=20one=20and=20get=20b=
ack=20to=20you.=20Maybe=20
>>=20everything=20works=20after=20all=20:)=20I'm=20really=20sorry=20for=
=20confusion.
>>=20=20=20=20=20
>
>=20No=20worries=20:-)=20Feel=20free=20to=20send=20a=20proper=20patch=20t=
o=20fix=20that=20problem=20
>=20upstream=20too=20!
>
>=20=20=20
The=20kernel=20with=20Stef's=20fix=20works=20fine,=20and=20recognizes=20b=
oth=20PLX=20and
device=20behind=20it.
Stef,=20=20do=20want=20to=20provide=20a=20patch=20for=20upstream=20kernel=
,=20or=20do=20you=20want=20me
to=20do=20that=20on=20your=20behalf=20?

Thanks=20a=20lot=20everyone=20for=20you=20help=20!

Felix.
Felix.

Disclaimer:=20The=20information=20contained=20in=20this=20email,=20includ=
ing=20any=20attachments=20is=20
confidential=20and=20is=20for=20the=20sole=20use=20of=20the=20intended=20=
recipient(s).=20Any=20unauthorized=20
review,=20use,=20disclosure=20or=20distribution=20is=20prohibited.=20If=
=20you=20are=20not=20the=20intended=20
recipient,=20please=20notify=20the=20sender=20immediately=20by=20replying=
=20to=20this=20message=20and=20
destroy=20all=20copies=20of=20this=20message=20and=20any=20attachments.