Re: [Regression] b1a000d3b8ec ("block: relax direct io memory alignment")

[Ming Lei <ming.lei@redhat.com>]

On Wed, Oct 23, 2024 at 08:12:33AM +0200, Christoph Hellwig wrote:
> On Tue, Oct 22, 2024 at 11:24:31AM +0100, Catalin Marinas wrote:
> > > > We should not allow smaller than cache line alignment on architectures
> > > > that are not cache coherent indeed.
> > 
> > Even on architectures that are not fully coherent, the coherency is a
> > property of the device. You may need to somehow pass this information in
> > struct queue_limits if you want it to be optimal.
> 
> Well, devices set the queue limits.  So this would be a fix in the
> drivers that set the queue limits.  SCSI already does this in the
> midlayer code,

I guess it isn't true:

[linux]# cat /sys/block/sda/queue/dma_alignment
3

> so the main places to fix are nvme und ublk.
> 
> I cant take care of nvme by copying the scsi pattern.
> 
> > That said, the DMA debug code also uses the static L1_CACHE_SHIFT and it
> > will trigger the warning anyway. Some discussion around the DMA API
> > debug came up during the small ARCH_KMALLOC_MINALIGN changes (don't
> > remember it was in private with Robin or on the list). Now kmalloc() can
> > return a small buffer (less than a cache line) that won't be bounced if
> > the device is coherent (see dma_kmalloc_safe()) but the DMA API debug
> > code only checks for direction == DMA_TO_DEVICE, not
> > dev_is_dma_coherent(). For arm64 I did not want to disable small
> > ARCH_KMALLOC_MINALIGN if CONFIG_DMA_API_DEBUG is enabled as this would
> > skew the testing by forcing all allocations to be ARCH_DMA_MINALIGN
> > aligned.
> > 
> > Maybe I'm missing something in those checks but I'm surprised that the
> > DMA API debug code doesn't complain about small kmalloc() buffers on x86
> > (which never had any bouncing for this specific case since it's fully
> > coherent). I suspect people just don't enable DMA debugging on x86 for
> > such devices (typically USB drivers have this issue).
> 
> I don't think there's too many of these indeed.

Usually it is assumed that it is safe to DMA over kmalloc() buffer...

thanks,
Ming