Question about a quirky (DesignWare) PCIe RC in Allwinner H6

[icenowy@aosc.io (Icenowy Zheng)]

? 2018-03-05?? 15:47 -0600?Bjorn Helgaas???
> On Mon, Mar 05, 2018 at 10:50:58PM +0800, Icenowy Zheng wrote:
> > Hi everyone,
> > 
> > I'm trying to implement a driver for the quirky (DW) PCIe RC in the
> > Allwinner H6 SoC.
> > 
> > The quirk is that only the "dbi" space is always mapped, but at the
> > same time only 64KiB of other spaces (config, downstream IO and
> > non-
> > prefetchable memory) are accessible. To access a certain address
> > the
> > high 16-bit of the address (all bus addresses in H6 SoC are 32-bit
> > despite the CPU is 64-bit) needs to be written into the
> > PCIE_ADDR_PAGE_CFG register (a vendor-defined register in DBI
> > space).
> > So the access to these spaces cannot be processed correctly with
> > just
> > readl/writel, as the existing code does.
> > 
> > Is it possible to workaround this in the PCI subsystem of Linux?
> > 
> > (I have thought a workaround that only maps the current accessible
> > 64KiB with the MMU, and when accessing the non-accessible part,
> > catch
> > the page fault and re-setup the map to the new 64KiB page. But
> > surely
> > it will kill the performance.)
> 
> If you only need to write to PCIE_ADDR_PAGE_CFG for *config*
> accesses,
> this should be easy.  All config accesses go through wrappers
> (pci_read_config_word(), etc) and several host bridge drivers have to
> update a register with parts of the config address, e.g.,
> advk_pcie_rd_conf(), mvebu_pcie_hw_rd_conf().
> 
> It sounds like you also need to update PCIE_ADDR_PAGE_CFG for I/O
> port
> accesses.  I/O port accesses do go through a wrapper (inb(), etc), so
> it might be possible in principle to intercept these, although much
> more difficult than config accesses, because the config accesses are
> already set up with per-host bridge hooks, while the I/O port
> accessors are mostly per-arch.
> 
> But if you have to update PCIE_ADDR_PAGE_CFG for memory accesses,
> that's really a problem because drivers are allowed to essentially
> read a memory BAR, convert that bus address to a CPU physical memory
> address, ioremap() it, then directly access the PCIe memory with
> loads
> and stores.

Unfortunately this is the case. All above (config, IO, memory) needs
PCIE_ADDR_PAGE_CFG update.

> 
> Bjorn