Re: [PATCH 3/3] tile pci: enable IOMMU to support DMA for legacy devices

[Chris Metcalf <cmetcalf@tilera.com>]

On 7/18/2012 12:50 PM, Bjorn Helgaas wrote:
> On Wed, Jul 18, 2012 at 10:15 AM, Chris Metcalf <cmetcalf@tilera.com> wrote:
>> On 7/13/2012 1:25 PM, Bjorn Helgaas wrote:
>>> On Fri, Jul 13, 2012 at 11:52:11AM -0400, Chris Metcalf wrote:
>>>> We use the same pci_iomem_resource for different domains or host
>>>> bridges, but the MMIO apertures for each bridge do not overlap because
>>>> non-overlapping resource ranges are allocated for each domains.
>>> You should not use the same pci_iomem_resource for different host bridges
>>> because that tells the PCI core that everything in pci_iomem_resource is
>>> available for devices under every host bridge, which I doubt is the case.
>>>
>>> The fact that your firmware assigns non-overlapping resources is good and
>>> works now, but if the kernel ever needs to allocate resources itself,
>> Actually, we were not using any firmware. It was indeed the kernel which
>> allocates resources from the shared pci_iomem_resource.
> Wow.  I wonder how that managed to work.  Is there some information
> that would have helped the PCI core do the right allocations?  Or
> maybe the host bridges forward everything they receive to PCI,
> regardless of address, and any given MMIO address is only routed to
> one of the host bridges because of the routing info in the page
> tables?

Since each host bridge contains non-overlapping ranges in its bridge config
header, ioremap() locates the right host bridge for the target PCI resource
address and programs the host bridge info into the MMIO mapping. The end
result is the MMIO address is routed to the right host bridge. On Tile
processors, different host bridges are like separate IO devices, in
completely separate domains.

> I guess in that case, the "apertures" would basically be
> defined by the page tables, not by the host bridges.  But that still
> doesn't explain how we would assign non-overlapping ranges to each
> domain.

Since all domains share the single resource, allocate_resource() "allocate
empty slot in the resource tree", giving non-overlapping ranges to each
devices.

Just to confirm, I'm assuming I'll ask Linus to pull this code out of my
tile tree when the merge window opens, right?  Would you like me to add
your name to the commit as acked or reviewed?  Thanks!

-- 
Chris Metcalf, Tilera Corp.
http://www.tilera.com