Re: [PATCH v4] drm/i915/dmabuf: avoid global wbinvd on dma-buf import

[Matthew Auld <matthew.auld@intel.com>]

On 02/09/2026 20:31, Prabhakaran, Krishna wrote:
> From: Krishna Prabhakaran <krishna.prabhakaran@intel.com>
> 
> When i915 needs to make an imported dma-buf coherent for GPU access on
> non-LLC platforms, or for objects that bypass LLC, it currently calls
> wbinvd_on_all_cpus(). get_pages() runs whenever an imported buffer is
> pinned, so this triggers a whole-cache write-back and invalidate,
> broadcast by IPI to every CPU, on every execbuf submission involving an
> imported dma-buf. That stalls the entire machine for milliseconds and
> starves latency-sensitive work on unrelated cores (e.g. USB isochronous
> audio serviced on the VMM's main thread).
> 
> Flush only the pages that actually need it instead:
> 
>   - If we imported one of our own dma-bufs, the backing object is
>     normally struct-page backed once migrated to SMEM, so flush it
>     directly with drm_clflush_sg(), exactly as we flush our other
>     objects. This also avoids re-entering the exporter through
>     dma_buf_vmap(), which would recurse into i915_gem_object_pin_map()
>     on the source object we already hold locked (and fails the
>     igt_dmabuf_import_same_driver selftest with -EBUSY). SMEM does not
>     guarantee the struct-page flag though: phys objects clear it and it
>     is dropped transiently during TTM moves, and drm_clflush_sg() walks
>     the sgt with sg_page(), so guard the direct flush with
>     i915_gem_object_has_struct_page() and fall back to wbinvd when it is
>     not set.
> 
>   - For a foreign dma-buf the sg_table is not guaranteed to be backed by
>     struct pages, and the importer has no way to tell, so drm_clflush_sg()
>     cannot be used. vmap the buffer and flush that virtual range with
>     drm_clflush_virt_range() instead: x86 uses PIPT caches, so flushing
>     one virtual alias evicts the cache lines for every alias of the same
>     physical pages. The dma_resv lock required by dma_buf_vmap() is
>     already held here via the imported object.
> 
> Fall back to wbinvd only when the buffer cannot be vmapped or is backed
> by I/O memory, where there is no CPU-side range to clflush.
> 
> Fixes: a035154da45d ("drm/i915/dmabuf: add paranoid flush-on-acquire")
> Signed-off-by: Krishna Prabhakaran <krishna.prabhakaran@intel.com>

Reviewed-by: Matthew Auld <matthew.auld@intel.com>

> ---
> v4: Guard the own-dma-buf direct drm_clflush_sg() with
>      i915_gem_object_has_struct_page() and fall back to wbinvd when the
>      SMEM object is not struct-page backed (phys objects, transient TTM
>      moves); sg_page() would otherwise walk bogus pages. (review feedback)
> v3: Drop the wbinvd fallback inside the own-dma-buf branch; the exporter is
>      migrated to SMEM on attach, so flushing device memory made no sense.
>      (Matthew Auld)
> v2: Special-case our own dma-bufs with drm_clflush_sg() instead of
>      dma_buf_vmap(), which recursed into i915_gem_object_pin_map() on the
>      already-locked source object and failed igt_dmabuf_import_same_driver
>      with -EBUSY; foreign dma-bufs use vmap + drm_clflush_virt_range().
> v1: https://patchwork.freedesktop.org/series/171760/
>   drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c | 62 +++++++++++++++++++---
>   1 file changed, 54 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> index b43d34c7d641..d1695171019c 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> @@ -10,6 +10,8 @@
>   
>   #include <asm/smp.h>
>   
> +#include <drm/drm_cache.h>
> +
>   #include "gem/i915_gem_dmabuf.h"
>   #include "i915_drv.h"
>   #include "i915_gem_object.h"
> @@ -249,16 +251,60 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
>   	 * DG1 is special here since it still snoops transactions even with
>   	 * CACHE_NONE. This is not the case with other HAS_SNOOP platforms. We
>   	 * might need to revisit this as we add new discrete platforms.
> -	 *
> -	 * XXX: Consider doing a vmap flush or something, where possible.
> -	 * Currently we just do a heavy handed wbinvd_on_all_cpus() here since
> -	 * the underlying sg_table might not even point to struct pages, so we
> -	 * can't just call drm_clflush_sg or similar, like we do elsewhere in
> -	 * the driver.
>   	 */
>   	if (i915_gem_object_can_bypass_llc(obj) ||
> -	    (!HAS_LLC(i915) && !IS_DG1(i915)))
> -		wbinvd_on_all_cpus();
> +	    (!HAS_LLC(i915) && !IS_DG1(i915))) {
> +		struct dma_buf *dma_buf = obj->base.import_attach->dmabuf;
> +
> +		if (dma_buf->ops == &i915_dmabuf_ops) {
> +			struct drm_i915_gem_object *dma_obj =
> +				dma_buf_to_obj(dma_buf);
> +
> +			/*
> +			 * We imported one of our own dma-bufs. The exporter is
> +			 * migrated to SMEM on attach, so it is normally struct-page
> +			 * backed and we can flush it directly, the same way we
> +			 * flush our other objects. This also avoids re-entering
> +			 * the exporter through dma_buf_vmap(), which would
> +			 * recurse into i915_gem_object_pin_map() on the source
> +			 * object we already hold locked.
> +			 *
> +			 * SMEM does not guarantee the STRUCT_PAGE flag though: phys
> +			 * objects clear it and it is dropped transiently during TTM
> +			 * moves. drm_clflush_sg() walks the sgt with sg_page(), so
> +			 * without struct pages those pointers are bogus; fall back to
> +			 * wbinvd in that case (not the vmap path, which would recurse
> +			 * on the already-locked source object).
> +			 */
> +			if (i915_gem_object_has_struct_page(dma_obj))
> +				drm_clflush_sg(dma_obj->mm.pages);
> +			else
> +				wbinvd_on_all_cpus();
> +		} else {
> +			struct iosys_map map;
> +
> +			/*
> +			 * A foreign sg_table is not guaranteed to be backed by
> +			 * struct pages, so we cannot use drm_clflush_sg(). vmap
> +			 * the buffer and flush the virtual range instead; x86
> +			 * uses PIPT caches, so flushing one alias evicts the
> +			 * lines for every alias of the same physical pages.
> +			 *
> +			 * We already hold the dma_resv lock via the imported
> +			 * obj, so use the locked dma_buf_vmap() variant.
> +			 */
> +			if (!dma_buf_vmap(dma_buf, &map)) {
> +				if (!map.is_iomem)
> +					drm_clflush_virt_range(map.vaddr,
> +							       obj->base.size);
> +				else
> +					wbinvd_on_all_cpus();
> +				dma_buf_vunmap(dma_buf, &map);
> +			} else {
> +				wbinvd_on_all_cpus();
> +			}
> +		}
> +	}
>   
>   	__i915_gem_object_set_pages(obj, sgt);
>