Re: [PATCH] cleanup: Add 'struct dev' in the TTM layer to be passed in for DMA API calls.

[Paul Mundt <lethal@linux-sh.org>]

On Tue, Mar 08, 2011 at 09:52:54PM +0100, Thomas Hellstrom wrote:
> Is passing a struct device to the DMA api really *strictly* necessary?
> 
Yes.

> I'd like to avoid that at all cost, since we don't want pages that are 
> backing buffer objects
> (coherent pages) to be associated with a specific device.
> 
And what do you do when coherent memory for a given device can only be
obtained from that bus or that device alone?

This API permits for cases where system memory is non-coherent or you are
handling transactions across busses lacking coherency protocols between
each other and so on. There are plenty of cases where coherent DMA
buffers for PCI devices need to be allocated within the context of the
PCI bus and no other coherent memory is possible. More complex examples
exist as well. There are MFDs where DMA transactions are only coherent
with regards to local SRAM that in turn is mapped in and out of an 8051
microcontroller's address space, and so on. The kernel deals with all of
these sorts of cases by way of that dev pointer in the DMA API, and any
attempt to move beyond that scope makes non-portable assumptions about
the nature of coherent memory pools.

> The reason for this is that we probably soon will want to move ttm 
> buffer objects between devices, and that should ideally be a simple 
> operation: If the memory type the buffer object currently resides in is 
> not shared between two devices, then move it out to system memory and 
> change its struct bo_device pointer.
> 
> If pages are associated with a specific device, this will become much 
> harder. Basically we need to change backing pages and copy all the data.
> 
That won't work if a device has consistent buffers locally but system
memory is non-coherent. A not too uncommon situation for embedded
platforms with a PCI bus hanging off of the CPU local bus.