Re: [LKML] Re: [PATCH v3] ad7877: keep dma rx buffers in seperate cache lines

[Nick Piggin <npiggin@suse.de>]

On Tue, May 11, 2010 at 07:07:35PM -0700, Marc Gauthier wrote:
> Mike Frysinger wrote:
> > lets look at the cacheline sizes for arches that dont set
> > ARCH_KMALLOC_MINALIGN to L1_CACHE_BYTES:
> >  - alplha - 32 or 64
> >  - frv - 32 or 64
> >  - blackfin - 32
> >  - parisc - 32 or 64
> >  - mn10300 - 16
> >  - s390 - 256
> >  - score - 16
> >  - sparc - 32
> >  - xtensa - 16 or 32
> >
> > assuming alpha and s390 handle cache coherency in hardware, it looks
> > to me like the proposed assumption (kmalloc returns cachealigned
> > pointers when cache management is in software) does not hold true.
> >
> > so should these other arches also be setting ARCH_KMALLOC_MINALIGN to
> > L1_CACHE_BYTES ?
> 
> IMHO, yes.  It just makes sense to avoid false-sharing issues, not to
> allocate unrelated blocks in the same cache line.
> 
> Also as it turns out (hope he doesn't me telling), Christian Zankel
> recently found a bug that was fixed exactly that way, by setting
> ARCH_KMALLOC_MINALIGN to L1_CACHE_BYTES for the Xtensa architecture.
> (Too recent to have percolated to mainline.)
> 
> A lot of the above might be cache line aligned (?).

I don't think it's necessarily a good idea. MINALIGN is an enforced
minimum alignment and the allocator has no leeway in reducing this.
In a UP system, or in a memory constrained system, it might be a better
idea to pack objects more tightly, for example.

If we allow drivers to assume kmalloc is cacheline aligned, it will be
(practically) impossible to revert this because it would require driver
audits.

So whenever strengthening API guarantees like this, it is better to be
very careful and conservative. Probably even introducing a new API with
the stronger semantics (even if it is just a wrapper in the case where
KMALLOC_MINALIGNED *is* cacheline sized).

I think adding to the DMA API would be a better idea. If the arch knows
that kmalloc is suitable for the job directly, it can be used. Drivers
can use the new interface, and kmalloc doesn't get saddled with
alignment requirements.