Re: DMA descriptor alignment

[Larry Finger <larry.finger@lwfinger.net>]

Andi Kleen wrote:
> Larry Finger <Larry.Finger@lwfinger.net> writes:
> 
>> For those variants of BCM43xx cards that use 64-bit DMA, there is a requirement that all descriptor
>> rings must be aligned on an 8K boundary and must fit within an 8K page. On the x86_64 architecture
>> where the page size is 4K, I was getting addresses like 0x67AF000 when using dma_alloc_coherent
>> calls.
> 
> Normally x86-64 dma_alloc_coherent calls the buddy allocator which gives
> you always naturally aligned blocks. But there is a fallback calling
> into swiotlb and swiotlb uses best fit allocation which only guarantees
> single page alignment. That is probably what you're seeing.
> 
> My dma zone rework would remove that fallback case and should make it work.
> 
>> From the description of the dma_pool_create and dma_pool_allocate routines, I thought they
>> would fix my problems; however, even with a dma_pool_create(name, dev, 8192, 8192, 8192) call, I'm
>> still getting 4K rather than 8K alignment, which results in DMA errors.
> 
> They cannot give you more alignment than the underlying allocator.
> 
>> Is there a bug in these routines, am I using them incorrectly, or do I have a misunderstanding of
>> what it takes to get this kind of alignment?
> 
> My suggestion as a short term workaround would be to first allocate 
> 8K using dma_alloc_coherent and if that has the wrong alignment get 16K 
> and align yourself. When the driver is loaded later that might be unreliable,
> but near boot or with enough free memory using order 2 should usually work.
>  
> With the dma zone rework that could be removed later.

I will be most interested in the dma zone rework; however, I now have the descriptors properly
allocated merely by asking for 8K, just as I had thought it should work. I'm not quite sure what was
wrong before, but I added a test to make certain that the alignment is OK just in case the problem
comes back.

Thanks for your response,

Larry