Re: [BUG?] vfio/pci: VA alignment sensitivity of VFIO_IOMMU_MAP_DMA which target MMIO

[Jason Gunthorpe <jgg@nvidia.com>]

On Thu, May 29, 2025 at 02:44:14PM -0700, Alex Mastro wrote:

> We are wondering the following:
> - Is all of the above expected behavior, and usage of VFIO?
> - Is there an expected minimum alignment greater than 4K (our system page size)
>   for non-MAP_FIXED mmap on a VFIO device fd?
> - Was there an unintended regression to our use-case in between 6.9 and 6.13?

I think this is something we have missed. VFIO should automatically
align the VMA's address if not MAP_FIXED, otherwise it can't use the
efficient huge page sizes anymore. qemu uses MAP_FIXED so we've left
out the non-qemu users from this performance optimization.

To fix it, the flow from the mm side is something like what
shmem_get_unmapped_area() does. VFIO would probably want to align all
BAR's to their size.

Which seems to me probably wants some refactoring and a core helper
'mm_get_aligned_unmapped_area()'..

I think if you are mmaping a huge huge BAR it is not surprising that
it will take a huge amount of time to write out all of the 4K
PTEs. The stalls on old kernels should probably be addressed by having
cond_resched() inside the remap_pfnmap().

Jason