Question about a quirky (DesignWare) PCIe RC in Allwinner H6

Question about a quirky (DesignWare) PCIe RC in Allwinner H6

[Icenowy Zheng]

Hi everyone,

I'm trying to implement a driver for the quirky (DW) PCIe RC in the
Allwinner H6 SoC.

The quirk is that only the "dbi" space is always mapped, but at the
same time only 64KiB of other spaces (config, downstream IO and non-
prefetchable memory) are accessible. To access a certain address the
high 16-bit of the address (all bus addresses in H6 SoC are 32-bit
despite the CPU is 64-bit) needs to be written into the
PCIE_ADDR_PAGE_CFG register (a vendor-defined register in DBI space).
So the access to these spaces cannot be processed correctly with just
readl/writel, as the existing code does.

Is it possible to workaround this in the PCI subsystem of Linux?

(I have thought a workaround that only maps the current accessible
64KiB with the MMU, and when accessing the non-accessible part, catch
the page fault and re-setup the map to the new 64KiB page. But surely
it will kill the performance.)

Thanks,
Icenowy

Question about a quirky (DesignWare) PCIe RC in Allwinner H6

[Bjorn Helgaas]

On Mon, Mar 05, 2018 at 10:50:58PM +0800, Icenowy Zheng wrote:
> Hi everyone,
> 
> I'm trying to implement a driver for the quirky (DW) PCIe RC in the
> Allwinner H6 SoC.
> 
> The quirk is that only the "dbi" space is always mapped, but at the
> same time only 64KiB of other spaces (config, downstream IO and non-
> prefetchable memory) are accessible. To access a certain address the
> high 16-bit of the address (all bus addresses in H6 SoC are 32-bit
> despite the CPU is 64-bit) needs to be written into the
> PCIE_ADDR_PAGE_CFG register (a vendor-defined register in DBI space).
> So the access to these spaces cannot be processed correctly with just
> readl/writel, as the existing code does.
> 
> Is it possible to workaround this in the PCI subsystem of Linux?
> 
> (I have thought a workaround that only maps the current accessible
> 64KiB with the MMU, and when accessing the non-accessible part, catch
> the page fault and re-setup the map to the new 64KiB page. But surely
> it will kill the performance.)

If you only need to write to PCIE_ADDR_PAGE_CFG for *config* accesses,
this should be easy.  All config accesses go through wrappers
(pci_read_config_word(), etc) and several host bridge drivers have to
update a register with parts of the config address, e.g.,
advk_pcie_rd_conf(), mvebu_pcie_hw_rd_conf().

It sounds like you also need to update PCIE_ADDR_PAGE_CFG for I/O port
accesses.  I/O port accesses do go through a wrapper (inb(), etc), so
it might be possible in principle to intercept these, although much
more difficult than config accesses, because the config accesses are
already set up with per-host bridge hooks, while the I/O port
accessors are mostly per-arch.

But if you have to update PCIE_ADDR_PAGE_CFG for memory accesses,
that's really a problem because drivers are allowed to essentially
read a memory BAR, convert that bus address to a CPU physical memory
address, ioremap() it, then directly access the PCIe memory with loads
and stores.

Bjorn

Question about a quirky (DesignWare) PCIe RC in Allwinner H6

[Icenowy Zheng]

? 2018-03-05?? 15:47 -0600?Bjorn Helgaas???
> On Mon, Mar 05, 2018 at 10:50:58PM +0800, Icenowy Zheng wrote:
> > Hi everyone,
> > 
> > I'm trying to implement a driver for the quirky (DW) PCIe RC in the
> > Allwinner H6 SoC.
> > 
> > The quirk is that only the "dbi" space is always mapped, but at the
> > same time only 64KiB of other spaces (config, downstream IO and
> > non-
> > prefetchable memory) are accessible. To access a certain address
> > the
> > high 16-bit of the address (all bus addresses in H6 SoC are 32-bit
> > despite the CPU is 64-bit) needs to be written into the
> > PCIE_ADDR_PAGE_CFG register (a vendor-defined register in DBI
> > space).
> > So the access to these spaces cannot be processed correctly with
> > just
> > readl/writel, as the existing code does.
> > 
> > Is it possible to workaround this in the PCI subsystem of Linux?
> > 
> > (I have thought a workaround that only maps the current accessible
> > 64KiB with the MMU, and when accessing the non-accessible part,
> > catch
> > the page fault and re-setup the map to the new 64KiB page. But
> > surely
> > it will kill the performance.)
> 
> If you only need to write to PCIE_ADDR_PAGE_CFG for *config*
> accesses,
> this should be easy.  All config accesses go through wrappers
> (pci_read_config_word(), etc) and several host bridge drivers have to
> update a register with parts of the config address, e.g.,
> advk_pcie_rd_conf(), mvebu_pcie_hw_rd_conf().
> 
> It sounds like you also need to update PCIE_ADDR_PAGE_CFG for I/O
> port
> accesses.  I/O port accesses do go through a wrapper (inb(), etc), so
> it might be possible in principle to intercept these, although much
> more difficult than config accesses, because the config accesses are
> already set up with per-host bridge hooks, while the I/O port
> accessors are mostly per-arch.
> 
> But if you have to update PCIE_ADDR_PAGE_CFG for memory accesses,
> that's really a problem because drivers are allowed to essentially
> read a memory BAR, convert that bus address to a CPU physical memory
> address, ioremap() it, then directly access the PCIe memory with
> loads
> and stores.

Unfortunately this is the case. All above (config, IO, memory) needs
PCIE_ADDR_PAGE_CFG update.

> 
> Bjorn

Question about a quirky (DesignWare) PCIe RC in Allwinner H6

[Bjorn Helgaas]

[+cc Marc]

On Tue, Mar 06, 2018 at 11:57:03AM +0800, Icenowy Zheng wrote:
> ? 2018-03-05?? 15:47 -0600?Bjorn Helgaas???
> > On Mon, Mar 05, 2018 at 10:50:58PM +0800, Icenowy Zheng wrote:
> > > Hi everyone,
> > > 
> > > I'm trying to implement a driver for the quirky (DW) PCIe RC in the
> > > Allwinner H6 SoC.
> > > 
> > > The quirk is that only the "dbi" space is always mapped, but at the
> > > same time only 64KiB of other spaces (config, downstream IO and
> > > non-
> > > prefetchable memory) are accessible. To access a certain address
> > > the
> > > high 16-bit of the address (all bus addresses in H6 SoC are 32-bit
> > > despite the CPU is 64-bit) needs to be written into the
> > > PCIE_ADDR_PAGE_CFG register (a vendor-defined register in DBI
> > > space).
> > > So the access to these spaces cannot be processed correctly with
> > > just
> > > readl/writel, as the existing code does.
> > > 
> > > Is it possible to workaround this in the PCI subsystem of Linux?
> > > 
> > > (I have thought a workaround that only maps the current accessible
> > > 64KiB with the MMU, and when accessing the non-accessible part,
> > > catch
> > > the page fault and re-setup the map to the new 64KiB page. But
> > > surely
> > > it will kill the performance.)
> > 
> > If you only need to write to PCIE_ADDR_PAGE_CFG for *config*
> > accesses,
> > this should be easy.  All config accesses go through wrappers
> > (pci_read_config_word(), etc) and several host bridge drivers have to
> > update a register with parts of the config address, e.g.,
> > advk_pcie_rd_conf(), mvebu_pcie_hw_rd_conf().
> > 
> > It sounds like you also need to update PCIE_ADDR_PAGE_CFG for I/O
> > port
> > accesses.  I/O port accesses do go through a wrapper (inb(), etc), so
> > it might be possible in principle to intercept these, although much
> > more difficult than config accesses, because the config accesses are
> > already set up with per-host bridge hooks, while the I/O port
> > accessors are mostly per-arch.
> > 
> > But if you have to update PCIE_ADDR_PAGE_CFG for memory accesses,
> > that's really a problem because drivers are allowed to essentially
> > read a memory BAR, convert that bus address to a CPU physical memory
> > address, ioremap() it, then directly access the PCIe memory with
> > loads
> > and stores.
> 
> Unfortunately this is the case. All above (config, IO, memory) needs
> PCIE_ADDR_PAGE_CFG update.

tango_pcie_probe() has a similar issue.  We don't have a good solution
there, so we just live with the fact that the platform will be
unreliable.

Bjorn

Question about a quirky (DesignWare) PCIe RC in Allwinner H6

[Icenowy Zheng]

? 2018-03-06?? 17:38 -0600?Bjorn Helgaas???
> [+cc Marc]
> 
> On Tue, Mar 06, 2018 at 11:57:03AM +0800, Icenowy Zheng wrote:
> > ? 2018-03-05?? 15:47 -0600?Bjorn Helgaas???
> > > On Mon, Mar 05, 2018 at 10:50:58PM +0800, Icenowy Zheng wrote:
> > > > Hi everyone,
> > > > 
> > > > I'm trying to implement a driver for the quirky (DW) PCIe RC in
> > > > the
> > > > Allwinner H6 SoC.
> > > > 
> > > > The quirk is that only the "dbi" space is always mapped, but at
> > > > the
> > > > same time only 64KiB of other spaces (config, downstream IO and
> > > > non-
> > > > prefetchable memory) are accessible. To access a certain
> > > > address
> > > > the
> > > > high 16-bit of the address (all bus addresses in H6 SoC are 32-
> > > > bit
> > > > despite the CPU is 64-bit) needs to be written into the
> > > > PCIE_ADDR_PAGE_CFG register (a vendor-defined register in DBI
> > > > space).
> > > > So the access to these spaces cannot be processed correctly
> > > > with
> > > > just
> > > > readl/writel, as the existing code does.
> > > > 
> > > > Is it possible to workaround this in the PCI subsystem of
> > > > Linux?
> > > > 
> > > > (I have thought a workaround that only maps the current
> > > > accessible
> > > > 64KiB with the MMU, and when accessing the non-accessible part,
> > > > catch
> > > > the page fault and re-setup the map to the new 64KiB page. But
> > > > surely
> > > > it will kill the performance.)
> > > 
> > > If you only need to write to PCIE_ADDR_PAGE_CFG for *config*
> > > accesses,
> > > this should be easy.  All config accesses go through wrappers
> > > (pci_read_config_word(), etc) and several host bridge drivers
> > > have to
> > > update a register with parts of the config address, e.g.,
> > > advk_pcie_rd_conf(), mvebu_pcie_hw_rd_conf().
> > > 
> > > It sounds like you also need to update PCIE_ADDR_PAGE_CFG for I/O
> > > port
> > > accesses.  I/O port accesses do go through a wrapper (inb(),
> > > etc), so
> > > it might be possible in principle to intercept these, although
> > > much
> > > more difficult than config accesses, because the config accesses
> > > are
> > > already set up with per-host bridge hooks, while the I/O port
> > > accessors are mostly per-arch.
> > > 
> > > But if you have to update PCIE_ADDR_PAGE_CFG for memory accesses,
> > > that's really a problem because drivers are allowed to
> > > essentially
> > > read a memory BAR, convert that bus address to a CPU physical
> > > memory
> > > address, ioremap() it, then directly access the PCIe memory with
> > > loads
> > > and stores.
> > 
> > Unfortunately this is the case. All above (config, IO, memory)
> > needs
> > PCIE_ADDR_PAGE_CFG update.
> 
> tango_pcie_probe() has a similar issue.  We don't have a good
> solution
> there, so we just live with the fact that the platform will be
> unreliable.

It's still less quirky than Allwinner H6 PCIe. It's only a config/MMIO
mux on tango; however on H6 PCIe both config space and MMIO space are
splitted to many pages. So on H6 if no solution is worked out, it will
not be unreliable -- it will be unusable instead.

> 
> Bjorn

Question about a quirky (DesignWare) PCIe RC in Allwinner H6

[Marc Gonzalez]

On 08/03/2018 06:48, Icenowy Zheng wrote:

> I'm trying to implement a driver for the quirky (DW) PCIe RC in the
> Allwinner H6 SoC.
> 
> The quirk is that only the "dbi" space is always mapped, but at the 
> same time only 64KiB of other spaces (config, downstream IO and non- 
> prefetchable memory) are accessible. To access a certain address the 
> high 16-bit of the address (all bus addresses in H6 SoC are 32- bit 
> despite the CPU is 64-bit) needs to be written into the 
> PCIE_ADDR_PAGE_CFG register (a vendor-defined register in DBI 
> space). So the access to these spaces cannot be processed correctly 
> with just readl/writel, as the existing code does.
> 
> Is it possible to workaround this in the PCI subsystem of Linux?

I didn't think anyone would come up with something more broken
than tango's PCIe host bridge...

It's hard to understand what's going on inside the minds of these
HW devs. I mean, if the steps required are

	WRITE MAGIC CONFIG REG
	READ/WRITE ADDRESS REG

then the whole operation is clearly not atomic, and bad things happen
when 2+ operations race.

Do they think in terms of single core systems with non-multitasking OS?
Probably not.

Maybe they think it is not a problem to wrap the operation using a
mutex? I'll confess I have no idea how bad that is for performance.
However, that is not an option in Linux, because mem space accesses
are just plain mmio accesses, and it's not possible to rewrite the
drivers, even as an out-of-tree patch.

I suppose it could be possible to make the first write "magic" in the
sense that it could "lock" the bus until the second access is performed?
Sort of like an implicit HW mutex. Actually, tango has explicit HW
mutexes that work along these lines.

> (I have thought a workaround that only maps the current accessible 
> 64KiB with the MMU, and when accessing the non-accessible part, 
> catch the page fault and re-setup the map to the new 64KiB page. But 
> surely it will kill the performance.)

The implementation might be non-trivial, as well.

> [tango is] still less quirky than Allwinner H6 PCIe. It's only a
> config/MMIO mux on tango; however on H6 PCIe both config space and
> MMIO space are splitted to many pages. So on H6 if no solution is
> worked out, it will not be unreliable -- it will be unusable
> instead.

I have only limited experience, but it seems that many useful PCIe
adapters require only very little mem address space.

For example, the USB3 PCIe adapter I tested my implementation with
requires only 8 KB.

01:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller (rev 03) (prog-if 30 [XHCI])
        Flags: fast devsel
        Memory at 50400000 (64-bit, non-prefetchable) [size=8K]
        Capabilities: [50] Power Management version 3
        Capabilities: [70] MSI: Enable- Count=1/8 Maskable- 64bit+
        Capabilities: [90] MSI-X: Enable- Count=8 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [150] Latency Tolerance Reporting


Same thing for the following WiFi adapter.

01:00.0 Network controller: Intel Corporation Wireless 7260 (rev bb)
        Subsystem: Intel Corporation Dual Band Wireless-AC 7260
        Flags: fast devsel, IRQ 22
        Memory at 50400000 (64-bit, non-prefetchable) [disabled] [size=8K]
        Capabilities: [c8] Power Management version 3
        Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [40] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Device Serial Number 48-51-b7-ff-ff-84-69-26
        Capabilities: [14c] Latency Tolerance Reporting
        Capabilities: [154] Vendor Specific Information: ID=cafe Rev=1 Len=014 <?>


Maybe you can decide that you will support only 64 KB?

Regards.

Question about a quirky (DesignWare) PCIe RC in Allwinner H6

[Marc Gonzalez]

On 07/03/2018 00:38, Bjorn Helgaas wrote:

> [+cc Marc]

Thanks for the CC.

Please update your address books, by changing
marc_gonzalez at sigmadesigns.com to marc.w.gonzalez at free.fr
if you wish to contact me, as Sigma Designs has started the
the liquidation process.

In related news, I have received the following notice.

> Unfortunately, the PCI-SIG has expired Sigma Designs, Inc's
> membership.  The annual membership fee of $4,000 US was due upon your
> renewal date of February 20, 2018.  Several reminders were sent to
> enable Sigma Designs, Inc to renew their membership, however, the
> PCI-SIG did not receive your company's renewal payment.  Please
> understand that the use of your vendor ID number is conditioned upon
> your maintaining an active membership in the PCI-SIG.
> 
> The PCI-SIG does reserve the right to re-assign or revoke any vendor
> ID previously assigned to former members.  Since we have not received
> your renewal payment, the PCI-SIG must request that you cease and
> desist all future use of your vendor ID number on devices
> manufactured by your company.

So Long, and Thanks for All the Fish :-)

Regards.

Question about a quirky (DesignWare) PCIe RC in Allwinner H6

[Icenowy Zheng]

? 2018-03-08?? 12:55 +0100?Marc Gonzalez???
> On 08/03/2018 06:48, Icenowy Zheng wrote:
> 
> > I'm trying to implement a driver for the quirky (DW) PCIe RC in the
> > Allwinner H6 SoC.
> > 
> > The quirk is that only the "dbi" space is always mapped, but at
> > the 
> > same time only 64KiB of other spaces (config, downstream IO and
> > non- 
> > prefetchable memory) are accessible. To access a certain address
> > the 
> > high 16-bit of the address (all bus addresses in H6 SoC are 32-
> > bit 
> > despite the CPU is 64-bit) needs to be written into the 
> > PCIE_ADDR_PAGE_CFG register (a vendor-defined register in DBI 
> > space). So the access to these spaces cannot be processed
> > correctly 
> > with just readl/writel, as the existing code does.
> > 
> > Is it possible to workaround this in the PCI subsystem of Linux?
> 
> I didn't think anyone would come up with something more broken
> than tango's PCIe host bridge...
> 
> It's hard to understand what's going on inside the minds of these
> HW devs. I mean, if the steps required are
> 
> 	WRITE MAGIC CONFIG REG
> 	READ/WRITE ADDRESS REG
> 
> then the whole operation is clearly not atomic, and bad things happen
> when 2+ operations race.
> 
> Do they think in terms of single core systems with non-multitasking
> OS?
> Probably not.
> 
> Maybe they think it is not a problem to wrap the operation using a
> mutex? I'll confess I have no idea how bad that is for performance.
> However, that is not an option in Linux, because mem space accesses
> are just plain mmio accesses, and it's not possible to rewrite the
> drivers, even as an out-of-tree patch.
> 
> I suppose it could be possible to make the first write "magic" in the
> sense that it could "lock" the bus until the second access is
> performed?
> Sort of like an implicit HW mutex. Actually, tango has explicit HW
> mutexes that work along these lines.

Unfortunately sun50i-h6 doesn't have it.

> 
> > (I have thought a workaround that only maps the current accessible 
> > 64KiB with the MMU, and when accessing the non-accessible part, 
> > catch the page fault and re-setup the map to the new 64KiB page.
> > But 
> > surely it will kill the performance.)
> 
> The implementation might be non-trivial, as well.

It might be possible for set up a hypervisor to do it. (Although it's
surely an abuse of the HYP/EL2 mode)

> 
> > [tango is] still less quirky than Allwinner H6 PCIe. It's only a
> > config/MMIO mux on tango; however on H6 PCIe both config space and
> > MMIO space are splitted to many pages. So on H6 if no solution is
> > worked out, it will not be unreliable -- it will be unusable
> > instead.
> 
> I have only limited experience, but it seems that many useful PCIe
> adapters require only very little mem address space.

But there's also many that require bigger memory address space than
64K.

e.g. the Atheros AR9462 802.11n adapter needs 512K non-prefetchable
memory.

> 
> For example, the USB3 PCIe adapter I tested my implementation with
> requires only 8 KB.
> 
> 01:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0
> Host Controller (rev 03) (prog-if 30 [XHCI])
>         Flags: fast devsel
>         Memory at 50400000 (64-bit, non-prefetchable) [size=8K]
>         Capabilities: [50] Power Management version 3
>         Capabilities: [70] MSI: Enable- Count=1/8 Maskable- 64bit+
>         Capabilities: [90] MSI-X: Enable- Count=8 Masked-
>         Capabilities: [a0] Express Endpoint, MSI 00
>         Capabilities: [100] Advanced Error Reporting
>         Capabilities: [150] Latency Tolerance Reporting
> 
> 
> Same thing for the following WiFi adapter.
> 
> 01:00.0 Network controller: Intel Corporation Wireless 7260 (rev bb)
>         Subsystem: Intel Corporation Dual Band Wireless-AC 7260
>         Flags: fast devsel, IRQ 22
>         Memory at 50400000 (64-bit, non-prefetchable) [disabled]
> [size=8K]
>         Capabilities: [c8] Power Management version 3
>         Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
>         Capabilities: [40] Express Endpoint, MSI 00
>         Capabilities: [100] Advanced Error Reporting
>         Capabilities: [140] Device Serial Number 48-51-b7-ff-ff-84-
> 69-26
>         Capabilities: [14c] Latency Tolerance Reporting
>         Capabilities: [154] Vendor Specific Information: ID=cafe
> Rev=1 Len=014 <?>
> 
> 
> Maybe you can decide that you will support only 64 KB?

Despite support of some devices will fail, I still think this is a good
idea. It makes the problem much simpler -- it will become a mux among
config space, IO space and 64KB non-prefetchable memory.

If a PCIe-fixing hypervisor is done it can simply skip the kernel glue,
but let the kernel configure it with ECAM mode.

> 
> Regards.
> 

Question about a quirky (DesignWare) PCIe RC in Allwinner H6

[Icenowy Zheng]

? 2018-03-08?? 21:18 +0800?Icenowy Zheng???
> ? 2018-03-08?? 12:55 +0100?Marc Gonzalez???
> > On 08/03/2018 06:48, Icenowy Zheng wrote:
> > 
> > > I'm trying to implement a driver for the quirky (DW) PCIe RC in
> > > the
> > > Allwinner H6 SoC.
> > > 
> > > The quirk is that only the "dbi" space is always mapped, but at
> > > the 
> > > same time only 64KiB of other spaces (config, downstream IO and
> > > non- 
> > > prefetchable memory) are accessible. To access a certain address
> > > the 
> > > high 16-bit of the address (all bus addresses in H6 SoC are 32-
> > > bit 
> > > despite the CPU is 64-bit) needs to be written into the 
> > > PCIE_ADDR_PAGE_CFG register (a vendor-defined register in DBI 
> > > space). So the access to these spaces cannot be processed
> > > correctly 
> > > with just readl/writel, as the existing code does.
> > > 
> > > Is it possible to workaround this in the PCI subsystem of Linux?
> > 
> > I didn't think anyone would come up with something more broken
> > than tango's PCIe host bridge...
> > 
> > It's hard to understand what's going on inside the minds of these
> > HW devs. I mean, if the steps required are
> > 
> > 	WRITE MAGIC CONFIG REG
> > 	READ/WRITE ADDRESS REG
> > 
> > then the whole operation is clearly not atomic, and bad things
> > happen
> > when 2+ operations race.
> > 
> > Do they think in terms of single core systems with non-multitasking
> > OS?
> > Probably not.
> > 
> > Maybe they think it is not a problem to wrap the operation using a
> > mutex? I'll confess I have no idea how bad that is for performance.
> > However, that is not an option in Linux, because mem space accesses
> > are just plain mmio accesses, and it's not possible to rewrite the
> > drivers, even as an out-of-tree patch.
> > 
> > I suppose it could be possible to make the first write "magic" in
> > the
> > sense that it could "lock" the bus until the second access is
> > performed?
> > Sort of like an implicit HW mutex. Actually, tango has explicit HW
> > mutexes that work along these lines.
> 
> Unfortunately sun50i-h6 doesn't have it.
> 
> > 
> > > (I have thought a workaround that only maps the current
> > > accessible 
> > > 64KiB with the MMU, and when accessing the non-accessible part, 
> > > catch the page fault and re-setup the map to the new 64KiB page.
> > > But 
> > > surely it will kill the performance.)
> > 
> > The implementation might be non-trivial, as well.
> 
> It might be possible for set up a hypervisor to do it. (Although it's
> surely an abuse of the HYP/EL2 mode)
> 
> > 
> > > [tango is] still less quirky than Allwinner H6 PCIe. It's only a
> > > config/MMIO mux on tango; however on H6 PCIe both config space
> > > and
> > > MMIO space are splitted to many pages. So on H6 if no solution is
> > > worked out, it will not be unreliable -- it will be unusable
> > > instead.
> > 
> > I have only limited experience, but it seems that many useful PCIe
> > adapters require only very little mem address space.
> 
> But there's also many that require bigger memory address space than
> 64K.
> 
> e.g. the Atheros AR9462 802.11n adapter needs 512K non-prefetchable
> memory.
> 
> > 
> > For example, the USB3 PCIe adapter I tested my implementation with
> > requires only 8 KB.
> > 
> > 01:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0
> > Host Controller (rev 03) (prog-if 30 [XHCI])
> >         Flags: fast devsel
> >         Memory at 50400000 (64-bit, non-prefetchable) [size=8K]
> >         Capabilities: [50] Power Management version 3
> >         Capabilities: [70] MSI: Enable- Count=1/8 Maskable- 64bit+
> >         Capabilities: [90] MSI-X: Enable- Count=8 Masked-
> >         Capabilities: [a0] Express Endpoint, MSI 00
> >         Capabilities: [100] Advanced Error Reporting
> >         Capabilities: [150] Latency Tolerance Reporting
> > 
> > 
> > Same thing for the following WiFi adapter.
> > 
> > 01:00.0 Network controller: Intel Corporation Wireless 7260 (rev
> > bb)
> >         Subsystem: Intel Corporation Dual Band Wireless-AC 7260
> >         Flags: fast devsel, IRQ 22
> >         Memory at 50400000 (64-bit, non-prefetchable) [disabled]
> > [size=8K]
> >         Capabilities: [c8] Power Management version 3
> >         Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
> >         Capabilities: [40] Express Endpoint, MSI 00
> >         Capabilities: [100] Advanced Error Reporting
> >         Capabilities: [140] Device Serial Number 48-51-b7-ff-ff-84-
> > 69-26
> >         Capabilities: [14c] Latency Tolerance Reporting
> >         Capabilities: [154] Vendor Specific Information: ID=cafe
> > Rev=1 Len=014 <?>
> > 
> > 
> > Maybe you can decide that you will support only 64 KB?
> 
> Despite support of some devices will fail, I still think this is a
> good
> idea. It makes the problem much simpler -- it will become a mux among
> config space, IO space and 64KB non-prefetchable memory.

However the DW PCIe RC core claims 1MiB non-prefetchable memory...

> 
> If a PCIe-fixing hypervisor is done it can simply skip the kernel
> glue,
> but let the kernel configure it with ECAM mode.
> 
> > 
> > Regards.
> > 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel