Re: [PATCH v2 07/27] PCI: Add software-emulated host bridge

[Jason Gunthorpe <jgunthorpe@obsidianresearch.com>]

On Mon, Jan 28, 2013 at 07:40:16PM -0700, Bjorn Helgaas wrote:
> On Mon, Jan 28, 2013 at 3:09 PM, Jason Gunthorpe
> <jgunthorpe@obsidianresearch.com> wrote:
> > On Mon, Jan 28, 2013 at 03:03:55PM -0700, Stephen Warren wrote:
> >> On 01/28/2013 01:18 PM, Arnd Bergmann wrote:
> >> > On Monday 28 January 2013, Thomas Petazzoni wrote:
> >> >> From: Thierry Reding <thierry.reding@avionic-design.de>
> >> >>
> >> >> [Thomas Petazzoni:
> >> >>  - Simplify capabilities handling.
> >> >>  - Move to a separate file.
> >> >>  - Fix mask used when writing a 4 bytes value.]
> >> >>
> >> >> Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de>
> >> >> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> >> >
> >> > Not even a description why this is needed?
> >> >
> >> > This patch (together with patch 8) seems like the most controversial
> >> > one of the series, so you should better provide a really good reason
> >> > why we would emulate something in software rather than using whatever
> >> > hardware is there.
> >>
> >> At least on Tegra, there is no HW that exposes PCI configuration
> >> registers for the host bridge itself. Only the root ports have exposed
> >> PCI configuration registers. There was some debate re: whether a host
> >> bridge device needed to exist or not. This patch makes such a device
> >> exist if it's required.
> 
> Host bridges are not actually PCI devices on any architecture.  The
> upstream side of a host bridge is by definition not on a PCI bus.  On
> some architectures, it *looks* like the host bridge is a PCI device
> because it responds to PCI config accesses and you can get to

Sure, you can't discover domains through any standard means, but once
you have found a domain (notably a way to issue config transactions)
then the PCI-E standard actually does place requirements on what
config transactions should return:
 - 0:00.0 is a host bridge config space.
 - 0:XX.X will be one of:
   - A root complex internal function, with some restrictions this
     is basically a PCI end device
   - A PCI-PCI bridge with various mandatory capability headers.
     One of these must show up for every physical PCI-E link
     on the root complex.
This collection of stuff on bus 0 is called the 'root complex'. This
is new in PCI-E, PCI-X and PCI didn't have such requirements.

SOC vendors are taking various liberties with their PCI-E implementations.
 - nvidia followed the standard but did not include the host bridge
   at 0:00.0
 - Marvell ignored everything about the root complex config space
   behavior :)

There are two patch sets in this subject, one for nvidia tegra and one
for Marvell, both presenting to Linux a view of the HW that matches
what the PCI-E spec describes - specifically that there is one domain,
and each PCI-E link/controller shows up as a PCI-PCI bridge on bus 0.

In this model, there is no 'host bridge aperture' hardware, each PCI-E
link has a dedicated aperture and control of that aperture is through
the PCI-PCI bridge window registers, again as PCI-E specifies.

Jason