Re: [PATCH] arm64: Add architecture support for PCI

[Jason Gunthorpe <jgunthorpe@obsidianresearch.com>]

On Tue, Feb 04, 2014 at 10:44:52AM +0100, Arnd Bergmann wrote:

>   Now I want to integrate the EHCI into my SoC and not waste one
>   of my precious PCIe root ports, so I have to create another PCI
>   domain with its own ECAM compliant config space to put it into.
>   Fortunately SBSA lets me add an arbitrary number of PCI domains,
>   as long as they are all strictly compliant. To software it will

Just to touch on this for others who might be reading..

IMHO any simple SOC that requires multiple domains is *broken*. A
single domain covers all reasonable needs until you get up to
mega-scale NUMA systems, encouraging people to design with multiple
domains only complicates the kernel :(

SOC internal peripherals should all show up in the bus 0 config space
of the only domain and SOC PCI-E physical ports should show up on bus
0 as PCI-PCI bridges. This is all covered in the PCI-E specs regarding
the root complex.

Generally I would expect the internal peripherals to still be
internally connected with AXI, but also connected through the ECAM
space for configuration, control, power management and address
assignment.

> 2. all address windows are set up by the boot loader, we only
>   need to know the location (IMHO this should be the
>   preferred way to do things regardless of SBSA).

Linux does a full address map re-assignment on boot, IIRC. You need
more magics to inhibit that if your BAR's and bridge windows don't
work.

Hot plug is a whole other thing..

> it's possible that the designware based ones get point 4 right.

The designware one's also appear to be re-purposed end point cores, so
their config handling is somewhat bonkers. Tegra got theirs sort of
close because they re-used knowledge/IP from their x86 south bridges -
but even then they didn't really implement ECAM properly for an ARM
environment.

Since config space is where everyone to date has fallen down, I think
the SBSA would have been wise to list dword by dword what a typical
ECAM config space should look like.

Jason