Re: DMA descriptor alignment

[Andi Kleen <andi@firstfloor.org>]

Larry Finger <Larry.Finger@lwfinger.net> writes:

> For those variants of BCM43xx cards that use 64-bit DMA, there is a requirement that all descriptor
> rings must be aligned on an 8K boundary and must fit within an 8K page. On the x86_64 architecture
> where the page size is 4K, I was getting addresses like 0x67AF000 when using dma_alloc_coherent
> calls.

Normally x86-64 dma_alloc_coherent calls the buddy allocator which gives
you always naturally aligned blocks. But there is a fallback calling
into swiotlb and swiotlb uses best fit allocation which only guarantees
single page alignment. That is probably what you're seeing.

My dma zone rework would remove that fallback case and should make it work.

> From the description of the dma_pool_create and dma_pool_allocate routines, I thought they
> would fix my problems; however, even with a dma_pool_create(name, dev, 8192, 8192, 8192) call, I'm
> still getting 4K rather than 8K alignment, which results in DMA errors.

They cannot give you more alignment than the underlying allocator.

> Is there a bug in these routines, am I using them incorrectly, or do I have a misunderstanding of
> what it takes to get this kind of alignment?

My suggestion as a short term workaround would be to first allocate 
8K using dma_alloc_coherent and if that has the wrong alignment get 16K 
and align yourself. When the driver is loaded later that might be unreliable,
but near boot or with enough free memory using order 2 should usually work.
 
With the dma zone rework that could be removed later.

-Andi