Re: PCI DMA to small buffers on cache-incoherent arch

[David Brownell <david-b@pacbell.net>]

> From: Roland Dreier <roland@topspin.com>
> Date: 10 Jun 2002 21:39:27 -0700
> 
>        David> How about allocating struct something using pci_pool?
>    
>    The problem is the driver can't safely touch field1 or field2 near the
>    DMA (it might pull the cache line back in too soon, or dirty the cache
>    line and have it written back on top of DMA'ed data)
> 
> Oh duh, I see, then go to making the thing a pointer to the
> dma area.

If it's going to be a pointer, it might as well be normal kmalloc data,
avoiding obfuscation of the memory model used by USB device drivers.
(They don't have to worry about DMA addresses, which IMO is a feature.)

Seems like there are two basic options for how to handle all this.

(a) Expose the dma/cache interaction to device drivers, so
     they can manage (and prevent) bad ones like the cacheline
     sharing scenarios.

(b) Require drivers to provide I/O buffers that are slab/page/...
     pointers from some API that prevents such problems.

I think (a) is more or less a variant of the existing alignment
requirements that get embedded in ABIs, but which still show up
in code that's very close to the hardware.  Many device driver
folk are familiar with them, or performance tuners, but not so
many folk doing USB device drivers would be.  (They are thankfully
far from host controller hardware and its quirks!)

Personally I'd avoid (b) since I like to allocate memory all at
once in most cases ... lots fewer error cases to cope with (or,
more typically _not_ cope with :).

One question is whether to support only one of them, or allow both.
In either case the DMA-mapping.txt will need to touch on the issue.

- Dave