Re: [PATCH] drm/xe/dma_buf: stop relying on placement in unmap

[Matthew Brost <matthew.brost@intel.com>]

On Thu, Apr 03, 2025 at 03:56:38PM +0100, Matthew Auld wrote:
> On 03/04/2025 15:07, Matthew Auld wrote:
> > The is_vram() is checking the current placement, however if we consider
> > exported VRAM with dynamic dma-buf, it looks possible for the xe driver
> > to async evict the memory, notifying the importer, however importer does
> > not have to call unmap_attachment() immediately, but rather just as
> > "soon as possible", like when the dma-resv idles. Following from this we
> > would then pipeline the move, attaching the fence to the manager, and
> > then update the current placement. But when the unmap_attachment() runs
> > at some later point we might see that is_vram() is now false, and take
> > the complete wrong path when dma-unmapping the sg, leading to
> > explosions.
> > 
> > To fix this rather make a note in the attachment if the sg was
> > originally mapping vram or tt pages.
> > 
> > Link: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/4563
> > Fixes: dd08ebf6c352 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
> > Signed-off-by: Matthew Auld <matthew.auld@intel.com>
> > Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > Cc: Matthew Brost <matthew.brost@intel.com>
> > Cc: <stable@vger.kernel.org> # v6.8+
> > ---
> >   drivers/gpu/drm/xe/xe_dma_buf.c | 23 ++++++++++++++++++++---
> >   1 file changed, 20 insertions(+), 3 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/xe/xe_dma_buf.c b/drivers/gpu/drm/xe/xe_dma_buf.c
> > index f67803e15a0e..b71058e26820 100644
> > --- a/drivers/gpu/drm/xe/xe_dma_buf.c
> > +++ b/drivers/gpu/drm/xe/xe_dma_buf.c
> > @@ -22,6 +22,22 @@
> >   MODULE_IMPORT_NS("DMA_BUF");
> > +/**
> > + * struct xe_sg_info - Track the exported sg info
> > + */
> > +struct xe_sg_info {
> > +	/** @is_vram: True if this sg is mapping VRAM. */
> > +	bool is_vram;
> > +};
> > +
> > +static struct xe_sg_info tt_sg_info = {
> > +	.is_vram = false,
> > +};
> > +
> > +static struct xe_sg_info vram_sg_info = {
> > +	.is_vram = true,
> > +};
> > +
> >   static int xe_dma_buf_attach(struct dma_buf *dmabuf,
> >   			     struct dma_buf_attachment *attach)
> >   {
> > @@ -118,6 +134,7 @@ static struct sg_table *xe_dma_buf_map(struct dma_buf_attachment *attach,
> >   		if (dma_map_sgtable(attach->dev, sgt, dir,
> >   				    DMA_ATTR_SKIP_CPU_SYNC))
> >   			goto error_free;
> > +		attach->priv = &tt_sg_info;
> >   		break;
> >   	case XE_PL_VRAM0:
> > @@ -128,6 +145,7 @@ static struct sg_table *xe_dma_buf_map(struct dma_buf_attachment *attach,
> >   					      dir, &sgt);
> >   		if (r)
> >   			return ERR_PTR(r);
> > +		attach->priv = &vram_sg_info;
> 
> Maybe we need to subclass the sg itself? It looks possible to call map
> again, before the unmap, and you might get different memory if you had mixed
> placement bo...
> 

I think that would be a better idea but drm_prime_pages_to_sg allocates
the table so I think we'd a bit DRM rework to decouple the allocation
from the implementation. Likewise xe_ttm_vram_mgr_alloc_sgt for too.

Matt

> >   		break;
> >   	default:
> >   		return ERR_PTR(-EINVAL);
> > @@ -145,10 +163,9 @@ static void xe_dma_buf_unmap(struct dma_buf_attachment *attach,
> >   			     struct sg_table *sgt,
> >   			     enum dma_data_direction dir)
> >   {
> > -	struct dma_buf *dma_buf = attach->dmabuf;
> > -	struct xe_bo *bo = gem_to_xe_bo(dma_buf->priv);
> > +	struct xe_sg_info *sg_info = attach->priv;
> > -	if (!xe_bo_is_vram(bo)) {
> > +	if (!sg_info->is_vram) {
> >   		dma_unmap_sgtable(attach->dev, sgt, dir, 0);
> >   		sg_free_table(sgt);
> >   		kfree(sgt);
>