Re: [PATCH v9 1/4] pci:host: APM X-Gene PCIe host controller driver

[Jason Gunthorpe <jgunthorpe@obsidianresearch.com>]

On Mon, Sep 22, 2014 at 04:09:03PM -0600, Bjorn Helgaas wrote:
> On Mon, Sep 22, 2014 at 3:33 PM, Tanmay Inamdar <tinamdar@apm.com> wrote:
> 
> >>> +static void xgene_pcie_fixup_bridge(struct pci_dev *dev)
> >>> +{
> >>> +     int i;
> >>> +
> >>> +     /* Hide the PCI host BARs from the kernel as their content doesn't
> >>> +      * fit well in the resource management
> >>> +      */
> >>
> >> This needs a better explanation than "doesn't fit well."
> >>
> >> I *think* you're probably talking about something similar to the MVEBU
> >> devices mentioned here:
> >> http://lkml.kernel.org/r/CAErSpo56jB1Bf2JtYCGKXZBZqRF1jXFxGmeewPX_e6vSXueGyA@mail.gmail.com
> >>
> >> where the device can be configured as either an endpoint or a root port,
> >> and the endpoint BARs are still visible when configured as a root port.
> >>
> >
> > It is true that X-Gene PCIe port can be configured as EP, however the
> > the FIXUP is required not because of the BARs are still visible when
> > configured as EP in past. X-Gene PCIe port, when configured as RC,
> > uses BAR0-BAR1 of RC's configuration space as inbound BARs. Entire DDR
> > region is mapped in these BARs so that it is accessible for EP devices
> > for DMA. So if the fixup is not done during enumeration, whole
> > outbound memory resource space gets assigned to RC and nothing is left
> > EP devices.
> 
> I'm not familiar with the concept of an "inbound BAR"; at least I'm
> not aware of anything like this in the PCI specs.  I think it might
> reduce confusion if we avoided calling them "BARs".  They happen to be
> at the same addresses where real BARs would be, but they certainly
> don't behave like real BARs.

So what does the lspci look like?

If x-gene has an otherwise correct bridge config space and only the
0/1 BARs have a non-standard meaning, that I'd agree with Bjorn, hide
all access to these registers from the kernel (and tell your HW crew
to read the specs, because it is very clear what those registers are
supposed to do) .

If x-gene doesn't even have a bridge config space, then there are much
bigger problems, and these corrupt BARs are just the start.

Jason