Re: Xe performance regression with recent IOMMU changes

[Leon Romanovsky <leon@kernel.org>]

On Wed, Jan 21, 2026 at 11:36:47PM -0800, Matthew Brost wrote:
> On Thu, Jan 22, 2026 at 09:29:13AM +0200, Leon Romanovsky wrote:
> > On Wed, Jan 21, 2026 at 10:15:14PM -0800, Matthew Brost wrote:
> > > On Wed, Jan 21, 2026 at 02:04:49PM -0400, Jason Gunthorpe wrote:
> > > > On Wed, Jan 21, 2026 at 09:11:35AM -0400, Jason Gunthorpe wrote:
> > > > > On Wed, Jan 21, 2026 at 02:02:16PM +0100, Francois Dugast wrote:
> > > > > > I am reporting a slowdown in Xe caused by a couple of IOMMU changes. It
> > > > > > can be observed during DMA mappings/unmappings required to issue copies
> > > > > > between system memory and the device, when handling GPU faults. Not sure
> > > > > > how other use cases or vendors are affected but below is the impact on
> > > > > > execution times for BMG:
> > > > > > 
> > > > > > Before changes:
> > > > > >   4KB
> > > > > >     drm_pagemap_migrate_map_pages: 0.4 us
> > > > > >     drm_pagemap_migrate_unmap_pages: 0.4 us
> > > > > >   64KB
> > > > > >     drm_pagemap_migrate_map_pages: 2.5 us
> > > > > >     drm_pagemap_migrate_unmap_pages: 3.5 us
> > > > > >   2MB
> > > > > >     drm_pagemap_migrate_map_pages: 88 us
> > > > > >     drm_pagemap_migrate_unmap_pages: 108 us
> > > > > > 
> > > > > > After changes:
> > > > > >   4KB
> > > > > >     drm_pagemap_migrate_map_pages: 0.7 us
> > > > > >     drm_pagemap_migrate_unmap_pages: 0.7 us
> > > > > >   64KB
> > > > > >     drm_pagemap_migrate_map_pages: 3.5 us
> > > > > >     drm_pagemap_migrate_unmap_pages: 10.5 us
> > > > > >   2MB
> > > > > >     drm_pagemap_migrate_map_pages: 102 us
> > > > > >     drm_pagemap_migrate_unmap_pages: 330 us
> > > > > 
> > > > > I posted some more optimizations for these cases, it should reduce the
> > > > > numbers.
> > > > > 
> > > 
> > > We can try those — link? I believe I know the series, but just to make
> > > sure we’re on the same page.
> > > 
> > > > > This is the opposite of the benchmark numbers I ran which showed
> > > > > significant gains as the page count and sizes increased.
> > > > > 
> > > > > But something weird is going on to see a 3x increase in unmap, that
> > > > > shouldn't be just algorithm overhead. That almost seems like
> > > > > additional IOTLB invalidation overhead or something else going wrong.
> > > > > 
> > > > > Is this from a system with the VT-d cache flushing requirement? That
> > > > > logic changed around too and could have this kind of big impact.
> > > > 
> > > > Oh looking at the code a bit you've got pretty much the slowest
> > > > possible thing you can do here:
> > > 
> > > This was a fairly common pattern prior to Leon’s series, I believe. The
> > > cross-references show this pattern appearing frequently in the kernel
> > > [1]. I do agree with the point below that, with Leon’s changes applied,
> > > this could be refactored into an IOVA alloc/link/unlink/free flow, which
> > > would work better (also 2M device pages reduces the common 2M case to a
> > > mute point).
> > > 
> > > But that’s not what we’re discussing here. We’re talking about a
> > > regression introduced in the dma-mapping API for x86, which in my view
> > > is unacceptable for a kernel release. So IMO we should revert those
> > > changes [2].
> > > 
> > > [1] https://elixir.bootlin.com/linux/v6.18.6/A/ident/dma_unmap_page
> > 
> > I think this comparison is unfair. The previous behavior was bad for
> > everyone, while the current issue affects only the specific
> > drm_pagemap_migrate_unmap_pages() flow. Cases where the performance of
> > dma_unmap_page() in non-direct mode matters are extremely rare.
> > 
> 
> I don’t think you can reason about this without extensive testing across
> multiple platforms. Nor is it fair to say - sorry we slowed down your
> existing code, good luck.

It is not what I said. I only pointed to the specific point that loop
over dma_unmap_page() is universally performance critical.

Thanks