Re: PCI DMA to small buffers on cache-incoherent arch

David Brownell
2002-06-12T14:13:07Z

> Please don't limit the API design to PCI ;)
>
> When I say "PCI pools" think "struct device pools" because that
> will what it will be in the end.
>
> That is Linus's long range plan, everything you see as pci_*
> will become dev_*.

I think the guts of it could happen in the 2.5 series soonish
though, so it'd only be "long range" in that quaint Wall Street
sense of "finishes sometime after next quarter". :)

Just to put some flesh on the discussion ... here's the rough
shape of what I was thinking, as more wood for the fire:

- Associated with a "struct device" would be some kind of
memory allocator/mapper object that would expose every (?)
primitive in DMA-mapping.txt, but not PCI-specific. That
would basically be a vtable, using the device for state.

struct dev_dma_whatsit {
/* page-level "consistent" allocators
* should they continue to be nonblocking?
*/
void *alloc_consistent(struct device *dev,
size_t *size, dma_addr_t *dma);
void free_consistent(struct device *dev,
size_t size, void *addr, dma_addr_t dma);

/* "streaming mapping" calls
* return BAD_DMA_ADDR on error
*/
dma_addr_t map_single(struct device *dev,
void *addr, size_t size, int dir);
void unmap_single(struct device *dev,
dma_addr_t, size_t size, int dir);
void sync_single(struct device *dev,
dma_addr_t, size_t size, int dir);

/* Also: dma masks, page and scatterlist
* map/unmap/sync, dma64_addr_t, ...
*/
};

Those would be designed so they could be shared between
devices, invisibly to code talking to a struct device.

Example: all USB devices connected to a given bus would
normally delegate to the PCI device underlying that bus.
(Except for non-PCI host controllers, of course!)

And the PCI versions of those methods have rather obvious
implementations ... :)

- There'd be dev_*() routines that in many cases just call
directly to those methods, or failed cleanly if the whatsit
wasn't set up ("used the default allocator"), or omitted
key methods. Many could be inlinable functions:

dev_alloc_consistent(...)
dev_free_consistent(...)
dev_map_single(...)
... etc

In some cases they'd mostly be layered over those primitives:

/* kmalloc analogue */
void *dev_kmalloc(struct device *dev,
size_t size, int mem_flags,
dma_addr_t *dma);
void dev_kfree (struct device *dev, void *mem,
dma_addr_t dma);

/* pci_pool/kmem_cache_t analogues */
struct dev_pool *dev_pool_create(const char *name,
struct device *dev, size_t size,
size_t allocation, int mem_flags);
void dev_pool_destroy(struct dev_pool *pool);

void *dev_pool_alloc(struct dev_pool *pool,
int mem_flags, dma_addr_t *dma);
void dev_pool_free(struct dev_pool *pool,
void *mem, dma_addr_t dma);

- Some of the other driver APIs would want to change to
leverage these primitives. They might want to provide
wrappers to make the dev_*() calls given the subsystem
device API.

Again using USB for an, one _possible_ change would be
to make urb->transfer_buffer be a dma_addr_t ... where
today it's a void*. (Discussions of such details belong
on the linux-usb-devel list, for the most part.) That'd
be a pretty major change to the USB API, since it'd force
device drivers to manage DMA mappings. While usb-storage
might benefit from sglist support, other drivers might
not see much of a win.

OK, that's enough trouble for this morning... ;)

- Dave