Re: [PATCH v7 01/10] PCI: host: rcar: Add Renesas R-Car PCIe driver

[Bjorn Helgaas <bhelgaas@google.com>]

[+cc Srikanth, Michal, Grant because we're having the same discussion
about root bus numbers in the Xilinx host bridge driver]

On Thu, May 1, 2014 at 3:50 AM, Phil Edworthy <phil.edworthy@renesas.com> wrote:

> When would the PCI core change the root bus number to something other than set in sys->busnr?

The PCI core never changes the root bus number of a host bridge, but
it's important for the core to know exactly what bus numbers are
behind each host bridge so we can correctly assign bus numbers, MMIO,
and I/O port resources.

The reason is that when we enumerate a PCI-to-PCI bridge, we have to
know whether bus numbers are available for the secondary bus.  For
example, assume two host bridges configured (or wired) as follows:

  A leads to [bus 00-1f], with aperture [mem 0x80000000-0x8fffffff]
  B leads to [bus 20-ff], with aperture [mem 0x90000000-0x9fffffff]

Further, assume a PCI-to-PCI bridge at 1f:00.0.  If the host bridge
driver doesn't tell us the [bus 00-1f] range for host bridge A, the
PCI core will assume [bus 00-ff].  When the core enumerates the
1f:00.0 bridge, it will assign bus number 20 to its secondary bus.
Now it enumerates devices on bus 20.  The core expects this to be done
via host bridge A, but enumeration uses ID-routed config transactions,
so they are claimed by host bridge B instead.  If we find a device
20:00.0, the core thinks it is below A, but it is really below B.

Now we assign resources to 20:00.0, and since we think it's below A,
we assign them from the [mem 0x80000000-0x8fffffff] aperture.  This
obviously doesn't work, because when the driver performs MMIO
accesses, which are address-routed, they are claimed by host bridge A,
not B, so the 20:00.0 device never sees them.

If your host bridge really claims ALL bus numbers (00-ff), that's
fine, and you can tell the PCI core that.  But if you have several
such bridges, they should be placed in separate domains.

This is probably more detail than you wanted, and maybe you don't ever
expect to have multiple host bridges or complicated hierarchies of
devices.  But the PCI core has to deal with these issues on other
systems, so it's important to get these details right.

Bjorn