Re: [PATCH] x86/pci: make pci_mem_start to be aligned only -v4

[Jesse Barnes <jbarnes@virtuousgeek.org>]

On Tue, 21 Apr 2009 02:33:05 +0400
Ivan Kokshaysky <ink@jurassic.park.msu.ru> wrote:
> x86 pci: first cut on 64-bit resource allocation
> 
> I believe that we should consider PCI memory above 4G as yet another
> type of address space. This actually makes sense, as even accesses to
> that memory are physically different - Dual Address Cycle (DAC) vs.
> 32-bit Single Address Cycle (SAC).
> 
> So, platform that can deal with 64-bit allocations would set up an
> additional root bus resource and mark it with IORESOURCE_MEM64 flag.
> 
> The main problem here is how the kernel would detect that hardware can
> actually access a DAC memory (I know for a fact that a lot of Intel
> chipsets cannot access MMIO >4G, even though subordinate p2p bridges
> are 64-bit capable).
> On the other hand, there are PCI devices with 64-bit BARs that do not
> work properly being placed above 4G boundary. For example, some
> radeon cards have 64-bit BAR for video RAM, but allocating that BAR in
> the DAC area doesn't work for various reasons, like video-BIOS
> limitations or drivers not taking into account that GPU is 32-bit.
> 
> So moving stuff into MEM64 area should be considered as generally
> unsafe operation, and the best default policy is to not enable MEM64
> resource unless we find that BIOS has allocated something there.
> At the same time, MEM64 can be easily enabled/disabled based on host
> bridge PCI IDs, kernel command line options and so on.

This sounds like reasonable default behavior given the variety of
chipsets and device quirks out there.  This does muddy up the arch vs.
generic code just a little bit more though; iirc mips & ia64 use full
64 bit ranges for their current IORESOURCE_MEM types (hm now that I've
checked it appears ia64 has changed a bit here, but still other
arches should probably get cleaned up to use the new 64 bit type at
some point).

> Here is a basic implementation of the above for x86. I think it's
> reasonably good starting point for PCI64 work - the next step would
> be to teach pci_bus_alloc_resource() about IORESOURCE_MEM64: logic is
> similar to prefetch vs non-prefetch case - MEM can hold MEM64
> resource, but not vice versa. And eventually bridge sizing code will
> be updated for reasonable 64-bit allocations (it's a non-trivial
> task, though).
> 
> This patch alone should fix cardbus >4G allocations and similar
> nonsense.
> 
> Signed-off-by: Ivan Kokshaysky <ink@jurassic.park.msu.ru>

Nice.  Any cleanups to existing arch code could be done at the same
time as the updates to the bus allocation.

Anyone care to send me some tested-by lines?

Thanks,
-- 
Jesse Barnes, Intel Open Source Technology Center