Re: dma_opt_mapping_size returns way too low sizes when using IOMMU

[John Garry <john.g.garry@oracle.com>]

On 17/08/2026 09:36, Christoph Hellwig wrote:
> Hi all,
> 
> I got reports that NVMe devices were arbitrarily limited to 128kiB
> transfers in recent kernel. 

How recent a kernel? This NVMe and DMA mapping code has not changed in 
years as far as I know.

> It turns out that this only happens when
> using an IOMMU and is caused by iommu_dma_opt_mapping_size returning
> iova_rcache_range(), which is defined as follows:
> 
> unsigned long iova_rcache_range(void)
> {
> 	return PAGE_SIZE << (IOVA_RANGE_CACHE_MAX_SIZE - 1);
> }
> 
> and thus indeed hardcodes a 128kiB return on 4k page size architectures.
> 
> Both the NVMe performance numbers and common sense suggest that this
> is NOT the optimal DMA mapping granularity.  Can we pick a saner value
> for iommu_dma_opt_mapping_size that does not restrict common I/O sizes?

Note that SCSI does not use iommu_dma_opt_mapping_size() for clamping 
max HW sectors, but instead sets opt size / max sectors from this value 
(so it is not a hard limit there). Could we consider similar for NVMe?

The reason for which we have iommu_dma_opt_mapping_size() is that 
performance can go through the floor we can't use the rcache for getting 
the IOVA, i.e. we need to always alloc and dealloc an IOVA from the RB 
tree for each mapping, and this can greatly reduce performance when the 
IOVA space fills.

If we increase IOVA_RANGE_CACHE_MAX_SIZE, then we just get caching of 
larger IOVAs and I am not sure that is a great idea.