Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Christian König via Virtualization]

Hi Dmitry,

First of all please separate out this patch from the rest of the series, 
since this is a complex separate structural change.

Am 27.05.22 um 01:50 schrieb Dmitry Osipenko:
> All dma-bufs have dma-reservation lock that allows drivers to perform
> exclusive operations over shared dma-bufs. Today's dma-buf API has
> incomplete locking specification, which creates dead lock situation
> for dma-buf importers and exporters that don't coordinate theirs locks.

Well please drop that sentence. The locking specifications are actually 
very well defined, it's just that some drivers are a bit broken 
regarding them.

What you do here is rather moving all the non-dynamic drivers over to 
the dynamic locking specification (which is really nice to have).

I have tried this before and failed because catching all the locks in 
the right code paths are very tricky. So expect some fallout from this 
and make sure the kernel test robot and CI systems are clean.

> This patch introduces new locking convention for dma-buf users. From now
> on all dma-buf importers are responsible for holding dma-buf reservation
> lock around operations performed over dma-bufs.
>
> This patch implements the new dma-buf locking convention by:
>
>    1. Making dma-buf API functions to take the reservation lock.
>
>    2. Adding new locked variants of the dma-buf API functions for drivers
>       that need to manage imported dma-bufs under the held lock.

Instead of adding new locked variants please mark all variants which 
expect to be called without a lock with an _unlocked postfix.

This should make it easier to remove those in a follow up patch set and 
then fully move the locking into the importer.

>    3. Converting all drivers to the new locking scheme.

I have strong doubts that you got all of them. At least radeon and 
nouveau should grab the reservation lock in their ->attach callbacks 
somehow.

>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>   drivers/gpu/drm/drm_client.c                  |   4 +-
>   drivers/gpu/drm/drm_gem.c                     |  33 +++
>   drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
>   drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
>   drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
>   drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
>   .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
>   .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
>   .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
>   include/drm/drm_gem.h                         |   3 +
>   include/linux/dma-buf.h                       |  14 +-
>   13 files changed, 241 insertions(+), 159 deletions(-)
>
> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
> index 32f55640890c..64a9909ccfa2 100644
> --- a/drivers/dma-buf/dma-buf.c
> +++ b/drivers/dma-buf/dma-buf.c
> @@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
>   	file->f_mode |= FMODE_LSEEK;
>   	dmabuf->file = file;
>   
> -	mutex_init(&dmabuf->lock);

Please make removing dmabuf->lock a separate change.

Regards,
Christian.

>   	INIT_LIST_HEAD(&dmabuf->attachments);
>   
>   	mutex_lock(&db_list.lock);
> @@ -737,14 +736,14 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   	attach->importer_ops = importer_ops;
>   	attach->importer_priv = importer_priv;  3. Converting all drivers to the new locking scheme.
>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>   drivers/gpu/drm/drm_client.c                  |   4 +-
>
>   
> +	dma_resv_lock(dmabuf->resv, NULL);
> +
>   	if (dmabuf->ops->attach) {
>   		ret = dmabuf->ops->attach(dmabuf, attach);
>   		if (ret)
>   			goto err_attach;
>   	}
> -	dma_resv_lock(dmabuf->resv, NULL);
>   	list_add(&attach->node, &dmabuf->attachments);
> -	dma_resv_unlock(dmabuf->resv);
>   
>   	/* When either the importer or the exporter can't handle dynamic
>   	 * mappings we cache the mapping here to avoid issues with the
> @@ -755,7 +754,6 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   		struct sg_table *sgt;
>   
>   		if (dma_buf_is_dynamic(attach->dmabuf)) {
> -			dma_resv_lock(attach->dmabuf->resv, NULL);
>   			ret = dmabuf->ops->pin(attach);
>   			if (ret)
>   				goto err_unlock;
> @@ -768,15 +766,16 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   			ret = PTR_ERR(sgt);
>   			goto err_unpin;
>   		}
> -		if (dma_buf_is_dynamic(attach->dmabuf))
> -			dma_resv_unlock(attach->dmabuf->resv);
>   		attach->sgt = sgt;
>   		attach->dir = DMA_BIDIRECTIONAL;
>   	}
>   
> +	dma_resv_unlock(dmabuf->resv);
> +
>   	return attach;
>   
>   err_attach:
> +	dma_resv_unlock(attach->dmabuf->resv);
>   	kfree(attach);
>   	return ERR_PTR(ret);
>   
> @@ -785,10 +784,10 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   		dmabuf->ops->unpin(attach);
>   
>   err_unlock:
> -	if (dma_buf_is_dynamic(attach->dmabuf))
> -		dma_resv_unlock(attach->dmabuf->resv);
> +	dma_resv_unlock(dmabuf->resv);
>   
>   	dma_buf_detach(dmabuf, attach);
> +
>   	return ERR_PTR(ret);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_dynamic_attach, DMA_BUF);
> @@ -832,24 +831,23 @@ void dma_buf_detach(struct dma_buf *dmabuf, struct dma_buf_attachment *attach)
>   	if (WARN_ON(!dmabuf || !attach))
>   		return;
>   
> -	if (attach->sgt) {
> -		if (dma_buf_is_dynamic(attach->dmabuf))
> -			dma_resv_lock(attach->dmabuf->resv, NULL);
> +	if (WARN_ON(dmabuf != attach->dmabuf))
> +		return;
>   
> +	dma_resv_lock(dmabuf->resv, NULL);
> +
> +	if (attach->sgt) {
>   		__unmap_dma_buf(attach, attach->sgt, attach->dir);
>   
> -		if (dma_buf_is_dynamic(attach->dmabuf)) {
> +		if (dma_buf_is_dynamic(attach->dmabuf))
>   			dmabuf->ops->unpin(attach);
> -			dma_resv_unlock(attach->dmabuf->resv);
> -		}
>   	}
>   
> -	dma_resv_lock(dmabuf->resv, NULL);
>   	list_del(&attach->node);
> -	dma_resv_unlock(dmabuf->resv);
>   	if (dmabuf->ops->detach)
>   		dmabuf->ops->detach(dmabuf, attach);
>   
> +	dma_resv_unlock(dmabuf->resv);
>   	kfree(attach);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_detach, DMA_BUF);
> @@ -906,28 +904,18 @@ void dma_buf_unpin(struct dma_buf_attachment *attach)
>   EXPORT_SYMBOL_NS_GPL(dma_buf_unpin, DMA_BUF);
>   
>   /**
> - * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
> + * dma_buf_map_attachment_locked - Returns the scatterlist table of the attachment;
>    * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
>    * dma_buf_ops.
>    * @attach:	[in]	attachment whose scatterlist is to be returned
>    * @direction:	[in]	direction of DMA transfer
>    *
> - * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
> - * on error. May return -EINTR if it is interrupted by a signal.
> - *
> - * On success, the DMA addresses and lengths in the returned scatterlist are
> - * PAGE_SIZE aligned.
> - *
> - * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
> - * the underlying backing storage is pinned for as long as a mapping exists,
> - * therefore users/importers should not hold onto a mapping for undue amounts of
> - * time.
> + * Locked variant of dma_buf_map_attachment().
>    *
> - * Important: Dynamic importers must wait for the exclusive fence of the struct
> - * dma_resv attached to the DMA-BUF first.
> + * Caller is responsible for holding dmabuf's reservation lock.
>    */
> -struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
> -					enum dma_data_direction direction)
> +struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *attach,
> +					       enum dma_data_direction direction)
>   {
>   	struct sg_table *sg_table;
>   	int r;
> @@ -937,8 +925,7 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   	if (WARN_ON(!attach || !attach->dmabuf))
>   		return ERR_PTR(-EINVAL);
>   
> -	if (dma_buf_attachment_is_dynamic(attach))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> +	dma_resv_assert_held(attach->dmabuf->resv);
>   
>   	if (attach->sgt) {
>   		/*
> @@ -953,7 +940,6 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   	}
>   
>   	if (dma_buf_is_dynamic(attach->dmabuf)) {
> -		dma_resv_assert_held(attach->dmabuf->resv);
>   		if (!IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY)) {
>   			r = attach->dmabuf->ops->pin(attach);
>   			if (r)
> @@ -993,42 +979,101 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   #endif /* CONFIG_DMA_API_DEBUG */
>   	return sg_table;
>   }
> -EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
> +EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment_locked, DMA_BUF);
>   
>   /**
> - * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
> - * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
> + * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
> + * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
>    * dma_buf_ops.
> - * @attach:	[in]	attachment to unmap buffer from
> - * @sg_table:	[in]	scatterlist info of the buffer to unmap
> - * @direction:  [in]    direction of DMA transfer
> + * @attach:	[in]	attachment whose scatterlist is to be returned
> + * @direction:	[in]	direction of DMA transfer
>    *
> - * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
> + * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
> + * on error. May return -EINTR if it is interrupted by a signal.
> + *
> + * On success, the DMA addresses and lengths in the returned scatterlist are
> + * PAGE_SIZE aligned.
> + *
> + * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
> + * the underlying backing storage is pinned for as long as a mapping exists,
> + * therefore users/importers should not hold onto a mapping for undue amounts of
> + * time.
> + *
> + * Important: Dynamic importers must wait for the exclusive fence of the struct
> + * dma_resv attached to the DMA-BUF first.
>    */
> -void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
> -				struct sg_table *sg_table,
> +struct sg_table *
> +dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   				enum dma_data_direction direction)
>   {
> +	struct sg_table *sg_table;
> +
>   	might_sleep();
>   
> -	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
> -		return;
> +	if (WARN_ON(!attach || !attach->dmabuf))
> +		return ERR_PTR(-EINVAL);
> +
> +	dma_resv_lock(attach->dmabuf->resv, NULL);
> +	sg_table = dma_buf_map_attachment_locked(attach, direction);
> +	dma_resv_unlock(attach->dmabuf->resv);
>   
> -	if (dma_buf_attachment_is_dynamic(attach))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> +	return sg_table;
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
> +
> +/**
> + * dma_buf_unmap_attachment_locked - Returns the scatterlist table of the attachment;
> + * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
> + * dma_buf_ops.
> + * @attach:	[in]	attachment whose scatterlist is to be returned
> + * @direction:	[in]	direction of DMA transfer
> + *
> + * Locked variant of dma_buf_unmap_attachment().
> + *
> + * Caller is responsible for holding dmabuf's reservation lock.
> + */
> +void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *attach,
> +				     struct sg_table *sg_table,
> +				     enum dma_data_direction direction)
> +{
> +	might_sleep();
> +
> +	dma_resv_assert_held(attach->dmabuf->resv);
>   
>   	if (attach->sgt == sg_table)
>   		return;
>   
> -	if (dma_buf_is_dynamic(attach->dmabuf))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> -
>   	__unmap_dma_buf(attach, sg_table, direction);
>   
>   	if (dma_buf_is_dynamic(attach->dmabuf) &&
>   	    !IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY))
>   		dma_buf_unpin(attach);
>   }
> +EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment_locked, DMA_BUF);
> +
> +/**
> + * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
> + * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
> + * dma_buf_ops.
> + * @attach:	[in]	attachment to unmap buffer from
> + * @sg_table:	[in]	scatterlist info of the buffer to unmap
> + * @direction:  [in]    direction of DMA transfer
> + *
> + * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
> + */
> +void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
> +			      struct sg_table *sg_table,
> +			      enum dma_data_direction direction)
> +{
> +	might_sleep();
> +
> +	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
> +		return;
> +
> +	dma_resv_lock(attach->dmabuf->resv, NULL);
> +	dma_buf_unmap_attachment_locked(attach, sg_table, direction);
> +	dma_resv_unlock(attach->dmabuf->resv);
> +}
>   EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment, DMA_BUF);
>   
>   /**
> @@ -1224,6 +1269,31 @@ int dma_buf_end_cpu_access(struct dma_buf *dmabuf,
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
>   
> +static int dma_buf_mmap_locked(struct dma_buf *dmabuf,
> +			       struct vm_area_struct *vma,
> +			       unsigned long pgoff)
> +{
> +	dma_resv_assert_held(dmabuf->resv);
> +
> +	/* check if buffer supports mmap */
> +	if (!dmabuf->ops->mmap)
> +		return -EINVAL;
> +
> +	/* check for offset overflow */
> +	if (pgoff + vma_pages(vma) < pgoff)
> +		return -EOVERFLOW;
> +
> +	/* check for overflowing the buffer's size */
> +	if (pgoff + vma_pages(vma) >
> +	    dmabuf->size >> PAGE_SHIFT)
> +		return -EINVAL;
> +
> +	/* readjust the vma */
> +	vma_set_file(vma, dmabuf->file);
> +	vma->vm_pgoff = pgoff;
> +
> +	return dmabuf->ops->mmap(dmabuf, vma);
> +}
>   
>   /**
>    * dma_buf_mmap - Setup up a userspace mmap with the given vma
> @@ -1242,29 +1312,46 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
>   int dma_buf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma,
>   		 unsigned long pgoff)
>   {
> +	int ret;
> +
>   	if (WARN_ON(!dmabuf || !vma))
>   		return -EINVAL;
>   
> -	/* check if buffer supports mmap */
> -	if (!dmabuf->ops->mmap)
> -		return -EINVAL;
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	ret = dma_buf_mmap_locked(dmabuf, vma, pgoff);
> +	dma_resv_unlock(dmabuf->resv);
>   
> -	/* check for offset overflow */
> -	if (pgoff + vma_pages(vma) < pgoff)
> -		return -EOVERFLOW;
> +	return ret;
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>   
> -	/* check for overflowing the buffer's size */
> -	if (pgoff + vma_pages(vma) >
> -	    dmabuf->size >> PAGE_SHIFT)
> -		return -EINVAL;
> +static int dma_buf_vmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
> +{
> +	struct iosys_map ptr;
> +	int ret;
>   
> -	/* readjust the vma */
> -	vma_set_file(vma, dmabuf->file);
> -	vma->vm_pgoff = pgoff;
> +	dma_resv_assert_held(dmabuf->resv);
>   
> -	return dmabuf->ops->mmap(dmabuf, vma);
> +	if (dmabuf->vmapping_counter) {
> +		dmabuf->vmapping_counter++;
> +		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
> +		*map = dmabuf->vmap_ptr;
> +		return ret;
> +	}
> +
> +	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
> +
> +	ret = dmabuf->ops->vmap(dmabuf, &ptr);
> +	if (WARN_ON_ONCE(ret))
> +		return ret;
> +
> +	dmabuf->vmap_ptr = ptr;
> +	dmabuf->vmapping_counter = 1;
> +
> +	*map = dmabuf->vmap_ptr;
> +
> +	return 0;
>   }
> -EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>   
>   /**
>    * dma_buf_vmap - Create virtual mapping for the buffer object into kernel
> @@ -1284,8 +1371,7 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>    */
>   int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
>   {
> -	struct iosys_map ptr;
> -	int ret = 0;
> +	int ret;
>   
>   	iosys_map_clear(map);
>   
> @@ -1295,52 +1381,40 @@ int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
>   	if (!dmabuf->ops->vmap)
>   		return -EINVAL;
>   
> -	mutex_lock(&dmabuf->lock);
> -	if (dmabuf->vmapping_counter) {
> -		dmabuf->vmapping_counter++;
> -		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
> -		*map = dmabuf->vmap_ptr;
> -		goto out_unlock;
> -	}
> -
> -	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
> -
> -	ret = dmabuf->ops->vmap(dmabuf, &ptr);
> -	if (WARN_ON_ONCE(ret))
> -		goto out_unlock;
> -
> -	dmabuf->vmap_ptr = ptr;
> -	dmabuf->vmapping_counter = 1;
> -
> -	*map = dmabuf->vmap_ptr;
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	ret = dma_buf_vmap_locked(dmabuf, map);
> +	dma_resv_unlock(dmabuf->resv);
>   
> -out_unlock:
> -	mutex_unlock(&dmabuf->lock);
>   	return ret;
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_vmap, DMA_BUF);
>   
> -/**
> - * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
> - * @dmabuf:	[in]	buffer to vunmap
> - * @map:	[in]	vmap pointer to vunmap
> - */
> -void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
> +static void dma_buf_vunmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
>   {
> -	if (WARN_ON(!dmabuf))
> -		return;
> -
>   	BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
>   	BUG_ON(dmabuf->vmapping_counter == 0);
>   	BUG_ON(!iosys_map_is_equal(&dmabuf->vmap_ptr, map));
>   
> -	mutex_lock(&dmabuf->lock);
>   	if (--dmabuf->vmapping_counter == 0) {
>   		if (dmabuf->ops->vunmap)
>   			dmabuf->ops->vunmap(dmabuf, map);
>   		iosys_map_clear(&dmabuf->vmap_ptr);
>   	}
> -	mutex_unlock(&dmabuf->lock);
> +}
> +
> +/**
> + * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
> + * @dmabuf:	[in]	buffer to vunmap
> + * @map:	[in]	vmap pointer to vunmap
> + */
> +void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
> +{
> +	if (WARN_ON(!dmabuf))
> +		return;
> +
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	dma_buf_vunmap_locked(dmabuf, map);
> +	dma_resv_unlock(dmabuf->resv);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_vunmap, DMA_BUF);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> index be6f76a30ac6..b704bdf5601a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -882,7 +882,8 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev,
>   			struct sg_table *sgt;
>   
>   			attach = gtt->gobj->import_attach;
> -			sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
> +			sgt = dma_buf_map_attachment_locked(attach,
> +							    DMA_BIDIRECTIONAL);
>   			if (IS_ERR(sgt))
>   				return PTR_ERR(sgt);
>   
> @@ -1007,7 +1008,8 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev,
>   		struct dma_buf_attachment *attach;
>   
>   		attach = gtt->gobj->import_attach;
> -		dma_buf_unmap_attachment(attach, ttm->sg, DMA_BIDIRECTIONAL);
> +		dma_buf_unmap_attachment_locked(attach, ttm->sg,
> +						DMA_BIDIRECTIONAL);
>   		ttm->sg = NULL;
>   	}
>   
> diff --git a/drivers/gpu/drm/drm_client.c b/drivers/gpu/drm/drm_client.c
> index af3b7395bf69..e9a1cd310352 100644
> --- a/drivers/gpu/drm/drm_client.c
> +++ b/drivers/gpu/drm/drm_client.c
> @@ -323,7 +323,7 @@ drm_client_buffer_vmap(struct drm_client_buffer *buffer,
>   	 * fd_install step out of the driver backend hooks, to make that
>   	 * final step optional for internal users.
>   	 */
> -	ret = drm_gem_vmap(buffer->gem, map);
> +	ret = drm_gem_vmap_unlocked(buffer->gem, map);
>   	if (ret)
>   		return ret;
>   
> @@ -345,7 +345,7 @@ void drm_client_buffer_vunmap(struct drm_client_buffer *buffer)
>   {
>   	struct iosys_map *map = &buffer->map;
>   
> -	drm_gem_vunmap(buffer->gem, map);
> +	drm_gem_vunmap_unlocked(buffer->gem, map);
>   }
>   EXPORT_SYMBOL(drm_client_buffer_vunmap);
>   
> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
> index 7c0b025508e4..c61674887582 100644
> --- a/drivers/gpu/drm/drm_gem.c
> +++ b/drivers/gpu/drm/drm_gem.c
> @@ -1053,7 +1053,12 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
>   	vma->vm_ops = obj->funcs->vm_ops;
>   
>   	if (obj->funcs->mmap) {
> +		ret = dma_resv_lock_interruptible(obj->resv, NULL);
> +		if (ret)
> +			goto err_drm_gem_object_put;
> +
>   		ret = obj->funcs->mmap(obj, vma);
> +		dma_resv_unlock(obj->resv);
>   		if (ret)
>   			goto err_drm_gem_object_put;
>   		WARN_ON(!(vma->vm_flags & VM_DONTEXPAND));
> @@ -1158,6 +1163,8 @@ void drm_gem_print_info(struct drm_printer *p, unsigned int indent,
>   
>   int drm_gem_pin(struct drm_gem_object *obj)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (obj->funcs->pin)
>   		return obj->funcs->pin(obj);
>   	else
> @@ -1166,6 +1173,8 @@ int drm_gem_pin(struct drm_gem_object *obj)
>   
>   void drm_gem_unpin(struct drm_gem_object *obj)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (obj->funcs->unpin)
>   		obj->funcs->unpin(obj);
>   }
> @@ -1174,6 +1183,8 @@ int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
>   {
>   	int ret;
>   
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (!obj->funcs->vmap)
>   		return -EOPNOTSUPP;
>   
> @@ -1189,6 +1200,8 @@ EXPORT_SYMBOL(drm_gem_vmap);
>   
>   void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (iosys_map_is_null(map))
>   		return;
>   
> @@ -1200,6 +1213,26 @@ void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
>   }
>   EXPORT_SYMBOL(drm_gem_vunmap);
>   
> +int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
> +{
> +	int ret;
> +
> +	dma_resv_lock(obj->resv, NULL);
> +	ret = drm_gem_vmap(obj, map);
> +	dma_resv_unlock(obj->resv);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL(drm_gem_vmap_unlocked);
> +
> +void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
> +{
> +	dma_resv_lock(obj->resv, NULL);
> +	drm_gem_vunmap(obj, map);
> +	dma_resv_unlock(obj->resv);
> +}
> +EXPORT_SYMBOL(drm_gem_vunmap_unlocked);
> +
>   /**
>    * drm_gem_lock_reservations - Sets up the ww context and acquires
>    * the lock on an array of GEM objects.
> diff --git a/drivers/gpu/drm/drm_gem_framebuffer_helper.c b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> index f4619803acd0..a0bff53b158e 100644
> --- a/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> +++ b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> @@ -348,7 +348,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
>   			iosys_map_clear(&map[i]);
>   			continue;
>   		}
> -		ret = drm_gem_vmap(obj, &map[i]);
> +		ret = drm_gem_vmap_unlocked(obj, &map[i]);
>   		if (ret)
>   			goto err_drm_gem_vunmap;
>   	}
> @@ -370,7 +370,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
>   		obj = drm_gem_fb_get_obj(fb, i);
>   		if (!obj)
>   			continue;
> -		drm_gem_vunmap(obj, &map[i]);
> +		drm_gem_vunmap_unlocked(obj, &map[i]);
>   	}
>   	return ret;
>   }
> @@ -398,7 +398,7 @@ void drm_gem_fb_vunmap(struct drm_framebuffer *fb,
>   			continue;
>   		if (iosys_map_is_null(&map[i]))
>   			continue;
> -		drm_gem_vunmap(obj, &map[i]);
> +		drm_gem_vunmap_unlocked(obj, &map[i]);
>   	}
>   }
>   EXPORT_SYMBOL(drm_gem_fb_vunmap);
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> index f5062d0c6333..09502d490da8 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> @@ -72,7 +72,7 @@ static int i915_gem_dmabuf_vmap(struct dma_buf *dma_buf,
>   	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
>   	void *vaddr;
>   
> -	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
> +	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
>   	if (IS_ERR(vaddr))
>   		return PTR_ERR(vaddr);
>   
> @@ -241,8 +241,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
>   
>   	assert_object_held(obj);
>   
> -	pages = dma_buf_map_attachment(obj->base.import_attach,
> -				       DMA_BIDIRECTIONAL);
> +	pages = dma_buf_map_attachment_locked(obj->base.import_attach,
> +					      DMA_BIDIRECTIONAL);
>   	if (IS_ERR(pages))
>   		return PTR_ERR(pages);
>   
> @@ -270,8 +270,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
>   static void i915_gem_object_put_pages_dmabuf(struct drm_i915_gem_object *obj,
>   					     struct sg_table *pages)
>   {
> -	dma_buf_unmap_attachment(obj->base.import_attach, pages,
> -				 DMA_BIDIRECTIONAL);
> +	dma_buf_unmap_attachment_locked(obj->base.import_attach, pages,
> +					DMA_BIDIRECTIONAL);
>   }
>   
>   static const struct drm_i915_gem_object_ops i915_gem_object_dmabuf_ops = {
> diff --git a/drivers/gpu/drm/qxl/qxl_object.c b/drivers/gpu/drm/qxl/qxl_object.c
> index b42a657e4c2f..a64cd635fbc0 100644
> --- a/drivers/gpu/drm/qxl/qxl_object.c
> +++ b/drivers/gpu/drm/qxl/qxl_object.c
> @@ -168,9 +168,16 @@ int qxl_bo_vmap_locked(struct qxl_bo *bo, struct iosys_map *map)
>   		bo->map_count++;
>   		goto out;
>   	}
> -	r = ttm_bo_vmap(&bo->tbo, &bo->map);
> +
> +	r = __qxl_bo_pin(bo);
>   	if (r)
>   		return r;
> +
> +	r = ttm_bo_vmap(&bo->tbo, &bo->map);
> +	if (r) {
> +		__qxl_bo_unpin(bo);
> +		return r;
> +	}
>   	bo->map_count = 1;
>   
>   	/* TODO: Remove kptr in favor of map everywhere. */
> @@ -192,12 +199,6 @@ int qxl_bo_vmap(struct qxl_bo *bo, struct iosys_map *map)
>   	if (r)
>   		return r;
>   
> -	r = __qxl_bo_pin(bo);
> -	if (r) {
> -		qxl_bo_unreserve(bo);
> -		return r;
> -	}
> -
>   	r = qxl_bo_vmap_locked(bo, map);
>   	qxl_bo_unreserve(bo);
>   	return r;
> @@ -247,6 +248,7 @@ void qxl_bo_vunmap_locked(struct qxl_bo *bo)
>   		return;
>   	bo->kptr = NULL;
>   	ttm_bo_vunmap(&bo->tbo, &bo->map);
> +	__qxl_bo_unpin(bo);
>   }
>   
>   int qxl_bo_vunmap(struct qxl_bo *bo)
> @@ -258,7 +260,6 @@ int qxl_bo_vunmap(struct qxl_bo *bo)
>   		return r;
>   
>   	qxl_bo_vunmap_locked(bo);
> -	__qxl_bo_unpin(bo);
>   	qxl_bo_unreserve(bo);
>   	return 0;
>   }
> diff --git a/drivers/gpu/drm/qxl/qxl_prime.c b/drivers/gpu/drm/qxl/qxl_prime.c
> index 142d01415acb..9169c26357d3 100644
> --- a/drivers/gpu/drm/qxl/qxl_prime.c
> +++ b/drivers/gpu/drm/qxl/qxl_prime.c
> @@ -59,7 +59,7 @@ int qxl_gem_prime_vmap(struct drm_gem_object *obj, struct iosys_map *map)
>   	struct qxl_bo *bo = gem_to_qxl_bo(obj);
>   	int ret;
>   
> -	ret = qxl_bo_vmap(bo, map);
> +	ret = qxl_bo_vmap_locked(bo, map);
>   	if (ret < 0)
>   		return ret;
>   
> @@ -71,5 +71,5 @@ void qxl_gem_prime_vunmap(struct drm_gem_object *obj,
>   {
>   	struct qxl_bo *bo = gem_to_qxl_bo(obj);
>   
> -	qxl_bo_vunmap(bo);
> +	qxl_bo_vunmap_locked(bo);
>   }
> diff --git a/drivers/media/common/videobuf2/videobuf2-dma-contig.c b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> index 678b359717c4..617062076370 100644
> --- a/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> +++ b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> @@ -382,18 +382,12 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_dc_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -409,14 +403,11 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir,
>   			    DMA_ATTR_SKIP_CPU_SYNC)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/drivers/media/common/videobuf2/videobuf2-dma-sg.c b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> index fa69158a65b1..d2075e7078cd 100644
> --- a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> +++ b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> @@ -424,18 +424,12 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_dma_sg_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -446,14 +440,11 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
>   	/* mapping to the client with new direction */
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/drivers/media/common/videobuf2/videobuf2-vmalloc.c b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> index 948152f1596b..3d00a7f0aac1 100644
> --- a/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> +++ b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> @@ -267,18 +267,12 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_vmalloc_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -289,14 +283,11 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
>   	/* mapping to the client with new direction */
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
> index 9d7c61a122dc..0b427939f466 100644
> --- a/include/drm/drm_gem.h
> +++ b/include/drm/drm_gem.h
> @@ -410,4 +410,7 @@ void drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
>   int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
>   			    u32 handle, u64 *offset);
>   
> +int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
> +void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
> +
>   #endif /* __DRM_GEM_H__ */
> diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
> index 71731796c8c3..23698c6b1d1e 100644
> --- a/include/linux/dma-buf.h
> +++ b/include/linux/dma-buf.h
> @@ -326,15 +326,6 @@ struct dma_buf {
>   	/** @ops: dma_buf_ops associated with this buffer object. */
>   	const struct dma_buf_ops *ops;
>   
> -	/**
> -	 * @lock:
> -	 *
> -	 * Used internally to serialize list manipulation, attach/detach and
> -	 * vmap/unmap. Note that in many cases this is superseeded by
> -	 * dma_resv_lock() on @resv.
> -	 */
> -	struct mutex lock;
> -
>   	/**
>   	 * @vmapping_counter:
>   	 *
> @@ -618,6 +609,11 @@ int dma_buf_fd(struct dma_buf *dmabuf, int flags);
>   struct dma_buf *dma_buf_get(int fd);
>   void dma_buf_put(struct dma_buf *dmabuf);
>   
> +struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *,
> +					       enum dma_data_direction);
> +void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *,
> +				     struct sg_table *,
> +				     enum dma_data_direction);
>   struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *,
>   					enum dma_data_direction);
>   void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,


_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Christian König via Virtualization]

Hi Dmitry,

Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
> Hello Christian,
>
> On 5/30/22 09:50, Christian König wrote:
>> Hi Dmitry,
>>
>> First of all please separate out this patch from the rest of the series,
>> since this is a complex separate structural change.
> I assume all the patches will go via the DRM tree in the end since the
> rest of the DRM patches in this series depend on this dma-buf change.
> But I see that separation may ease reviewing of the dma-buf changes, so
> let's try it.

That sounds like you are underestimating a bit how much trouble this 
will be.

>> I have tried this before and failed because catching all the locks in
>> the right code paths are very tricky. So expect some fallout from this
>> and make sure the kernel test robot and CI systems are clean.
> Sure, I'll fix up all the reported things in the next iteration.
>
> BTW, have you ever posted yours version of the patch? Will be great if
> we could compare the changed code paths.

No, I never even finished creating it after realizing how much work it 
would be.

>>> This patch introduces new locking convention for dma-buf users. From now
>>> on all dma-buf importers are responsible for holding dma-buf reservation
>>> lock around operations performed over dma-bufs.
>>>
>>> This patch implements the new dma-buf locking convention by:
>>>
>>>     1. Making dma-buf API functions to take the reservation lock.
>>>
>>>     2. Adding new locked variants of the dma-buf API functions for drivers
>>>        that need to manage imported dma-bufs under the held lock.
>> Instead of adding new locked variants please mark all variants which
>> expect to be called without a lock with an _unlocked postfix.
>>
>> This should make it easier to remove those in a follow up patch set and
>> then fully move the locking into the importer.
> Do we really want to move all the locks to the importers? Seems the
> majority of drivers should be happy with the dma-buf helpers handling
> the locking for them.

Yes, I clearly think so.

>
>>>     3. Converting all drivers to the new locking scheme.
>> I have strong doubts that you got all of them. At least radeon and
>> nouveau should grab the reservation lock in their ->attach callbacks
>> somehow.
> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
> lock already, seems they should be okay (?)

You are looking at the wrong side. You need to fix the export code path, 
not the import ones.

See for example attach on radeon works like this 
drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.

Same for nouveau and probably a few other exporters as well. That will 
certainly cause a deadlock if you don't fix it.

I strongly suggest to do this step by step, first attach/detach and then 
the rest.

Regards,
Christian.

>
> I assume all the basics should covered in this v6. At minimum Intel,
> Tegra, Panfrost, Lima and Rockchip drivers should be good. If I missed
> something, then please let me know and I'll correct it.
>
>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>> ---
>>>    drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>>>    drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>>>    drivers/gpu/drm/drm_client.c                  |   4 +-
>>>    drivers/gpu/drm/drm_gem.c                     |  33 +++
>>>    drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
>>>    drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
>>>    drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
>>>    drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
>>>    .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
>>>    .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
>>>    .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
>>>    include/drm/drm_gem.h                         |   3 +
>>>    include/linux/dma-buf.h                       |  14 +-
>>>    13 files changed, 241 insertions(+), 159 deletions(-)
>>>
>>> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
>>> index 32f55640890c..64a9909ccfa2 100644
>>> --- a/drivers/dma-buf/dma-buf.c
>>> +++ b/drivers/dma-buf/dma-buf.c
>>> @@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct
>>> dma_buf_export_info *exp_info)
>>>        file->f_mode |= FMODE_LSEEK;
>>>        dmabuf->file = file;
>>>    -    mutex_init(&dmabuf->lock);
>> Please make removing dmabuf->lock a separate change.
> Alright
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>

So I guess I get a price for being blind since forever, because this
thing existed since at least 2013. I just stumbled over
llist_lru.[hc], a purpose built list helper for shrinkers. I think we
should try to adopt that so that our gpu shrinkers look more like
shrinkers for everything else.

Apologies for this, since I fear this might cause a bit of churn.
Hopefully it's all contained to the list manipulation code in shmem
helpers, I don't think this should leak any further.
-Daniel

> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }
> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)
> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>


-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Sun, Jun 5, 2022 at 9:47 AM Daniel Vetter <daniel@ffwll.ch> wrote:
>
> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> >
> > Introduce a common DRM SHMEM shrinker framework that allows to reduce
> > code duplication among DRM drivers by replacing theirs custom shrinker
> > implementations with the generic shrinker.
> >
> > In order to start using DRM SHMEM shrinker drivers should:
> >
> > 1. Implement new evict() shmem object callback.
> > 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> > 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
> >    activate shrinking of shmem GEMs.
> >
> > This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> > Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
> >
> > Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> > Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>
> So I guess I get a price for being blind since forever, because this
> thing existed since at least 2013. I just stumbled over
> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> should try to adopt that so that our gpu shrinkers look more like
> shrinkers for everything else.

followup from a bit of irc discussion w/ danvet about list_lru:

* It seems to be missing a way to bail out of iteration before
  nr_to_scan is hit.. which is going to be inconvenient if you
  want to allow active bos on the LRU but bail scanning once
  you encounter the first one.

* Not sure if the numa node awareness is super useful for GEM
  bos

First issue is perhaps not too hard to fix.  But maybe a better
idea is a drm_gem_lru helper type thing which is more tailored
to GEM buffers?

BR,
-R

> Apologies for this, since I fear this might cause a bit of churn.
> Hopefully it's all contained to the list manipulation code in shmem
> helpers, I don't think this should leak any further.
> -Daniel
>
> > ---
> >  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
> >  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
> >  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
> >  include/drm/drm_device.h                      |   4 +
> >  include/drm/drm_gem_shmem_helper.h            |  87 ++-
> >  5 files changed, 594 insertions(+), 49 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > index 555fe212bd98..4cd0b5913492 100644
> > --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> > +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
> >
> > +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       return (shmem->madv >= 0) && shmem->evict &&
> > +               shmem->eviction_enabled && shmem->pages_use_count &&
> > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> > +}
> > +
> > +static void
> > +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> > +
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (!gem_shrinker || obj->import_attach)
> > +               return;
> > +
> > +       mutex_lock(&gem_shrinker->lock);
> > +
> > +       if (drm_gem_shmem_is_evictable(shmem) ||
> > +           drm_gem_shmem_is_purgeable(shmem))
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> > +       else if (shmem->madv < 0)
> > +               list_del_init(&shmem->madv_list);
> > +       else if (shmem->evicted)
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> > +       else if (!shmem->pages)
> > +               list_del_init(&shmem->madv_list);
> > +       else
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +}
> > +
> >  /**
> >   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
> >   * @shmem: shmem GEM object to free
> > @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >         } else {
> >                 dma_resv_lock(shmem->base.resv, NULL);
> >
> > +               /* take out shmem GEM object from the memory shrinker */
> > +               drm_gem_shmem_madvise(shmem, -1);
> > +
> >                 WARN_ON(shmem->vmap_use_count);
> >
> >                 if (shmem->sgt) {
> > @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >                         sg_free_table(shmem->sgt);
> >                         kfree(shmem->sgt);
> >                 }
> > -               if (shmem->pages)
> > +               if (shmem->pages_use_count)
> >                         drm_gem_shmem_put_pages(shmem);
> >
> >                 WARN_ON(shmem->pages_use_count);
> > @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
> >
> > -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > +/**
> > + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> > + * @shmem: shmem GEM object
> > + *
> > + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       dma_resv_lock(shmem->base.resv, NULL);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       shmem->eviction_enabled = true;
> > +
> > +       dma_resv_unlock(shmem->base.resv);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> > +
> > +/**
> > + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> > + * @shmem: shmem GEM object
> > + *
> > + * Tell memory shrinker that this GEM can be purged. Initially purging is
> > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       dma_resv_lock(shmem->base.resv, NULL);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       shmem->purge_enabled = true;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       dma_resv_unlock(shmem->base.resv);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> > +
> > +static int
> > +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
> >  {
> >         struct drm_gem_object *obj = &shmem->base;
> >         struct page **pages;
> >
> > -       if (shmem->pages_use_count++ > 0)
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (shmem->madv < 0) {
> > +               WARN_ON(shmem->pages);
> > +               return -ENOMEM;
> > +       }
> > +
> > +       if (shmem->pages) {
> > +               WARN_ON(!shmem->evicted);
> >                 return 0;
> > +       }
> > +
> > +       if (WARN_ON(!shmem->pages_use_count))
> > +               return -EINVAL;
> >
> >         pages = drm_gem_get_pages(obj);
> >         if (IS_ERR(pages)) {
> >                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> > -               shmem->pages_use_count = 0;
> >                 return PTR_ERR(pages);
> >         }
> >
> > @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> >         return 0;
> >  }
> >
> > +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       int err;
> > +
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       if (shmem->pages_use_count++ > 0) {
> > +               err = drm_gem_shmem_swap_in(shmem);
> > +               if (err)
> > +                       goto err_zero_use;
> > +
> > +               return 0;
> > +       }
> > +
> > +       err = drm_gem_shmem_acquire_pages(shmem);
> > +       if (err)
> > +               goto err_zero_use;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +
> > +err_zero_use:
> > +       shmem->pages_use_count = 0;
> > +
> > +       return err;
> > +}
> > +
> > +static void
> > +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +
> > +       if (!shmem->pages) {
> > +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> > +               return;
> > +       }
> > +
> > +#ifdef CONFIG_X86
> > +       if (shmem->map_wc)
> > +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > +#endif
> > +
> > +       drm_gem_put_pages(obj, shmem->pages,
> > +                         shmem->pages_mark_dirty_on_put,
> > +                         shmem->pages_mark_accessed_on_put);
> > +       shmem->pages = NULL;
> > +}
> > +
> >  /*
> >   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
> >   * @shmem: shmem GEM object
> > @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> >   */
> >  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> >  {
> > -       struct drm_gem_object *obj = &shmem->base;
> > -
> >         dma_resv_assert_held(shmem->base.resv);
> >
> >         if (WARN_ON_ONCE(!shmem->pages_use_count))
> > @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> >         if (--shmem->pages_use_count > 0)
> >                 return;
> >
> > -#ifdef CONFIG_X86
> > -       if (shmem->map_wc)
> > -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > -#endif
> > +       drm_gem_shmem_release_pages(shmem);
> >
> > -       drm_gem_put_pages(obj, shmem->pages,
> > -                         shmem->pages_mark_dirty_on_put,
> > -                         shmem->pages_mark_accessed_on_put);
> > -       shmem->pages = NULL;
> > +       drm_gem_shmem_update_pages_state(shmem);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> >
> > @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> >   */
> >  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
> >  {
> > +       int ret;
> > +
> >         dma_resv_assert_held(shmem->base.resv);
> >
> >         WARN_ON(shmem->base.import_attach);
> >
> > -       return drm_gem_shmem_get_pages(shmem);
> > +       ret = drm_gem_shmem_get_pages(shmem);
> > +       if (!ret)
> > +               shmem->pages_pin_count++;
> > +
> > +       return ret;
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_pin);
> >
> > @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
> >         WARN_ON(shmem->base.import_attach);
> >
> >         drm_gem_shmem_put_pages(shmem);
> > +
> > +       shmem->pages_pin_count--;
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_unpin);
> >
> > @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> >                         return 0;
> >                 }
> >
> > -               ret = drm_gem_shmem_get_pages(shmem);
> > +               ret = drm_gem_shmem_pin(shmem);
> >                 if (ret)
> >                         goto err_zero_use;
> >
> > @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> >
> >  err_put_pages:
> >         if (!obj->import_attach)
> > -               drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_unpin(shmem);
> >  err_zero_use:
> >         shmem->vmap_use_count = 0;
> >
> > @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
> >                         return;
> >
> >                 vunmap(shmem->vaddr);
> > -               drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_unpin(shmem);
> >         }
> >
> >         shmem->vaddr = NULL;
> > @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
> >
> >         madv = shmem->madv;
> >
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> >         return (madv >= 0);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_madvise);
> >
> > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > +/**
> > + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> > + *                           hardware access to the memory.
> > + * @shmem: shmem GEM object
> > + *
> > + * This function moves shmem GEM back to memory if it was previously evicted
> > + * by the memory shrinker. The GEM is ready to use on success.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
> >  {
> >         struct drm_gem_object *obj = &shmem->base;
> > -       struct drm_device *dev = obj->dev;
> > +       struct sg_table *sgt;
> > +       int err;
> >
> >         dma_resv_assert_held(shmem->base.resv);
> >
> > -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > +       if (shmem->evicted) {
> > +               err = drm_gem_shmem_acquire_pages(shmem);
> > +               if (err)
> > +                       return err;
> > +
> > +               sgt = drm_gem_shmem_get_sg_table(shmem);
> > +               if (IS_ERR(sgt))
> > +                       return PTR_ERR(sgt);
> > +
> > +               err = dma_map_sgtable(obj->dev->dev, sgt,
> > +                                     DMA_BIDIRECTIONAL, 0);
> > +               if (err) {
> > +                       sg_free_table(sgt);
> > +                       kfree(sgt);
> > +                       return err;
> > +               }
> >
> > -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > -       sg_free_table(shmem->sgt);
> > -       kfree(shmem->sgt);
> > -       shmem->sgt = NULL;
> > +               shmem->sgt = sgt;
> > +               shmem->evicted = false;
> >
> > -       drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_update_pages_state(shmem);
> > +       }
> >
> > -       shmem->madv = -1;
> > +       if (!shmem->pages)
> > +               return -ENOMEM;
> >
> > -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > -       drm_gem_free_mmap_offset(obj);
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
> >
> > -       /* Our goal here is to return as much of the memory as
> > -        * is possible back to the system as we are called from OOM.
> > -        * To do this we must instruct the shmfs to drop all of its
> > -        * backing pages, *now*.
> > -        */
> > -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +       struct drm_device *dev = obj->dev;
> >
> > -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > +       if (shmem->evicted)
> > +               return;
> > +
> > +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > +       drm_gem_shmem_release_pages(shmem);
> > +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > +
> > +       sg_free_table(shmem->sgt);
> > +       kfree(shmem->sgt);
> > +       shmem->sgt = NULL;
> >  }
> > -EXPORT_SYMBOL(drm_gem_shmem_purge);
> >
> >  /**
> >   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> > @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
> >         vm_fault_t ret;
> >         struct page *page;
> >         pgoff_t page_offset;
> > +       bool pages_unpinned;
> > +       int err;
> >
> >         /* We don't use vmf->pgoff since that has the fake offset */
> >         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
> >
> >         dma_resv_lock(shmem->base.resv, NULL);
> >
> > -       if (page_offset >= num_pages ||
> > -           WARN_ON_ONCE(!shmem->pages) ||
> > -           shmem->madv < 0) {
> > +       /* Sanity-check that we have the pages pointer when it should present */
> > +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> > +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> > +
> > +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
> >                 ret = VM_FAULT_SIGBUS;
> >         } else {
> > +               err = drm_gem_shmem_swap_in(shmem);
> > +               if (err) {
> > +                       ret = VM_FAULT_OOM;
> > +                       goto unlock;
> > +               }
> > +
> >                 page = shmem->pages[page_offset];
> >
> >                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
> >         }
> >
> > +unlock:
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         return ret;
> > @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
> >  {
> >         struct drm_gem_object *obj = vma->vm_private_data;
> >         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> > -       int ret;
> >
> >         WARN_ON(shmem->base.import_attach);
> >
> >         dma_resv_lock(shmem->base.resv, NULL);
> > -       ret = drm_gem_shmem_get_pages(shmem);
> > -       WARN_ON_ONCE(ret != 0);
> > +
> > +       if (drm_gem_shmem_get_pages(shmem))
> > +               shmem->pages_use_count++;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         drm_gem_vm_open(vma);
> > @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
> >  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> >                               struct drm_printer *p, unsigned int indent)
> >  {
> > +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> > +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
> >         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
> >
> >         if (shmem->base.import_attach)
> > @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> >                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
> >                                   shmem->vmap_use_count);
> >
> > +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
> >         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> > +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_print_info);
> >
> > @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
> >
> >         shmem->sgt = sgt;
> >
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         return sgt;
> > @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
> >
> > +static struct drm_gem_shmem_shrinker *
> > +to_drm_shrinker(struct shrinker *shrinker)
> > +{
> > +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> > +}
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > +                                    struct shrink_control *sc)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > +       struct drm_gem_shmem_object *shmem;
> > +       unsigned long count = 0;
> > +
> > +       if (!mutex_trylock(&gem_shrinker->lock))
> > +               return 0;
> > +
> > +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > +               count += shmem->base.size;
> > +
> > +               if (count >= SHRINK_EMPTY)
> > +                       break;
> > +       }
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +
> > +       if (count >= SHRINK_EMPTY)
> > +               return SHRINK_EMPTY - 1;
> > +
> > +       return count ?: SHRINK_EMPTY;
> > +}
> > +
> > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> > +{
> > +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> > +       WARN_ON(shmem->evicted);
> > +
> > +       drm_gem_shmem_unpin_pages(shmem);
> > +
> > +       shmem->evicted = true;
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> > +
> > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +
> > +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > +
> > +       drm_gem_shmem_unpin_pages(shmem);
> > +       drm_gem_free_mmap_offset(obj);
> > +
> > +       /* Our goal here is to return as much of the memory as
> > +        * is possible back to the system as we are called from OOM.
> > +        * To do this we must instruct the shmfs to drop all of its
> > +        * backing pages, *now*.
> > +        */
> > +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > +
> > +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > +
> > +       shmem->madv = -1;
> > +       shmem->evicted = false;
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> > +                                       unsigned long nr_to_scan,
> > +                                       bool *lock_contention,
> > +                                       bool evict)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > +       struct drm_gem_shmem_object *shmem;
> > +       struct list_head still_in_list;
> > +       struct drm_gem_object *obj;
> > +       unsigned long freed = 0;
> > +       size_t page_count;
> > +       int err;
> > +
> > +       INIT_LIST_HEAD(&still_in_list);
> > +
> > +       mutex_lock(&gem_shrinker->lock);
> > +
> > +       while (freed < nr_to_scan) {
> > +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> > +                                                typeof(*shmem), madv_list);
> > +               if (!shmem)
> > +                       break;
> > +
> > +               obj = &shmem->base;
> > +               page_count = obj->size >> PAGE_SHIFT;
> > +               list_move_tail(&shmem->madv_list, &still_in_list);
> > +
> > +               if (evict) {
> > +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> > +                           get_nr_swap_pages() < page_count)
> > +                               continue;
> > +               } else {
> > +                       if (!drm_gem_shmem_is_purgeable(shmem))
> > +                               continue;
> > +               }
> > +
> > +               /*
> > +                * If it's in the process of being freed, gem_object->free()
> > +                * may be blocked on lock waiting to remove it.  So just
> > +                * skip it.
> > +                */
> > +               if (!kref_get_unless_zero(&obj->refcount))
> > +                       continue;
> > +
> > +               mutex_unlock(&gem_shrinker->lock);
> > +
> > +               /* prevent racing with job-submission code paths */
> > +               if (!dma_resv_trylock(obj->resv)) {
> > +                       *lock_contention |= true;
> > +                       goto shrinker_lock;
> > +               }
> > +
> > +               /* prevent racing with the dma-buf importing/exporting */
> > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > +                       *lock_contention |= true;
> > +                       goto resv_unlock;
> > +               }
> > +
> > +               /* check whether h/w uses this object */
> > +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> > +                       goto object_name_unlock;
> > +
> > +               /* re-check whether eviction status hasn't changed */
> > +               if (!drm_gem_shmem_is_evictable(shmem) &&
> > +                   !drm_gem_shmem_is_purgeable(shmem))
> > +                       goto object_name_unlock;
> > +
> > +               err = shmem->evict(shmem);
> > +               if (!err)
> > +                       freed += obj->size >> PAGE_SHIFT;
> > +
> > +object_name_unlock:
> > +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> > +resv_unlock:
> > +               dma_resv_unlock(obj->resv);
> > +shrinker_lock:
> > +               drm_gem_object_put(&shmem->base);
> > +               mutex_lock(&gem_shrinker->lock);
> > +       }
> > +
> > +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +
> > +       return freed;
> > +}
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> > +                                   struct shrink_control *sc)
> > +{
> > +       unsigned long nr_to_scan = sc->nr_to_scan;
> > +       bool lock_contention = false;
> > +       unsigned long freed;
> > +
> > +       /* purge as many objects as we can */
> > +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> > +                                                       &lock_contention, false);
> > +
> > +       /* evict as many objects as we can */
> > +       if (freed < nr_to_scan)
> > +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> > +                                                                nr_to_scan - freed,
> > +                                                                &lock_contention,
> > +                                                                true);
> > +
> > +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> > +}
> > +
> > +/**
> > + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> > + * @dev: DRM device
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker;
> > +       int err;
> > +
> > +       if (WARN_ON(dev->shmem_shrinker))
> > +               return -EBUSY;
> > +
> > +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> > +       if (!gem_shrinker)
> > +               return -ENOMEM;
> > +
> > +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> > +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> > +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> > +       gem_shrinker->dev = dev;
> > +
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> > +       mutex_init(&gem_shrinker->lock);
> > +
> > +       dev->shmem_shrinker = gem_shrinker;
> > +
> > +       err = register_shrinker(&gem_shrinker->base);
> > +       if (err) {
> > +               dev->shmem_shrinker = NULL;
> > +               kfree(gem_shrinker);
> > +               return err;
> > +       }
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> > +
> > +/**
> > + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> > + * @dev: DRM device
> > + */
> > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> > +
> > +       if (gem_shrinker) {
> > +               unregister_shrinker(&gem_shrinker->base);
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> > +               mutex_destroy(&gem_shrinker->lock);
> > +               dev->shmem_shrinker = NULL;
> > +               kfree(gem_shrinker);
> > +       }
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> > +
> >  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
> >  MODULE_IMPORT_NS(DMA_BUF);
> >  MODULE_LICENSE("GPL v2");
> > diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > index a4bedfeb2ec4..7cc32556f908 100644
> > --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > @@ -15,6 +15,13 @@
> >  #include "panfrost_gem.h"
> >  #include "panfrost_mmu.h"
> >
> > +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       return (shmem->madv > 0) &&
> > +               !shmem->pages_pin_count && shmem->sgt &&
> > +               !shmem->base.dma_buf && !shmem->base.import_attach;
> > +}
> > +
> >  static unsigned long
> >  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> >  {
> > @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
> >                 return 0;
> >
> >         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> > -               if (drm_gem_shmem_is_purgeable(shmem))
> > +               if (panfrost_gem_shmem_is_purgeable(shmem))
> >                         count += shmem->base.size >> PAGE_SHIFT;
> >         }
> >
> > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > index b2d93cb12ebf..81bacc7e1873 100644
> > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > @@ -89,6 +89,7 @@ struct virtio_gpu_object {
> >         uint32_t hw_res_handle;
> >         bool dumb;
> >         bool created;
> > +       bool detached;
> >         bool host3d_blob, guest_blob;
> >         uint32_t blob_mem, blob_flags;
> >
> > @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> >
> >  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
> >
> > +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> > +
> >  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> >                                uint32_t *resid);
> >  /* virtgpu_prime.c */
> > diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> > index 9923c7a6885e..929546cad894 100644
> > --- a/include/drm/drm_device.h
> > +++ b/include/drm/drm_device.h
> > @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
> >  struct drm_vma_offset_manager;
> >  struct drm_vram_mm;
> >  struct drm_fb_helper;
> > +struct drm_gem_shmem_shrinker;
> >
> >  struct inode;
> >
> > @@ -277,6 +278,9 @@ struct drm_device {
> >         /** @vram_mm: VRAM MM memory manager */
> >         struct drm_vram_mm *vram_mm;
> >
> > +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> > +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> > +
> >         /**
> >          * @switch_power_state:
> >          *
> > diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> > index 9a8983ee8abe..62c640678a91 100644
> > --- a/include/drm/drm_gem_shmem_helper.h
> > +++ b/include/drm/drm_gem_shmem_helper.h
> > @@ -6,6 +6,7 @@
> >  #include <linux/fs.h>
> >  #include <linux/mm.h>
> >  #include <linux/mutex.h>
> > +#include <linux/shrinker.h>
> >
> >  #include <drm/drm_file.h>
> >  #include <drm/drm_gem.h>
> > @@ -15,6 +16,7 @@
> >  struct dma_buf_attachment;
> >  struct drm_mode_create_dumb;
> >  struct drm_printer;
> > +struct drm_device;
> >  struct sg_table;
> >
> >  /**
> > @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
> >          */
> >         unsigned int pages_use_count;
> >
> > +       /**
> > +        * @pages_pin_count:
> > +        *
> > +        * Reference count on the pinned pages table.
> > +        * The pages can be evicted by memory shrinker
> > +        * when the count reaches zero.
> > +        */
> > +       unsigned int pages_pin_count;
> > +
> >         /**
> >          * @madv: State for madvise
> >          *
> >          * 0 is active/inuse.
> > +        * 1 is not-needed/can-be-purged
> >          * A negative value is the object is purged.
> > -        * Positive values are driver specific and not used by the helpers.
> >          */
> >         int madv;
> >
> > @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
> >          * @map_wc: map object write-combined (instead of using shmem defaults).
> >          */
> >         bool map_wc;
> > +
> > +       /**
> > +        * @eviction_enabled:
> > +        *
> > +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool eviction_enabled;
> > +
> > +       /**
> > +        * @purge_enabled:
> > +        *
> > +        * The shmem pages can be purged only if @purge_enabled is set to true.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool purge_enabled;
> > +
> > +       /**
> > +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool evicted;
> > +
> > +       /**
> > +        * @evict:
> > +        *
> > +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> > +        * GEM's DMA reservation is kept locked by the shrinker. This is
> > +        * optional callback that should be specified by drivers.
> > +        *
> > +        * Returns 0 on success, or -errno on error.
> > +        */
> > +       int (*evict)(struct drm_gem_shmem_object *shmem);
> >  };
> >
> >  #define to_drm_gem_shmem_obj(obj) \
> > @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
> >
> >  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
> >
> > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> > +
> >  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> >  {
> > -       return (shmem->madv > 0) &&
> > -               !shmem->vmap_use_count && shmem->sgt &&
> > -               !shmem->base.dma_buf && !shmem->base.import_attach;
> > +       return (shmem->madv > 0) && shmem->evict &&
> > +               shmem->purge_enabled && shmem->pages_use_count &&
> > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
> >  }
> >
> > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> > +
> > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> >
> >  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
> >  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> > @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
> >         return drm_gem_shmem_mmap(shmem, vma);
> >  }
> >
> > +/**
> > + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> > + */
> > +struct drm_gem_shmem_shrinker {
> > +       /** @base: Shrinker for purging shmem GEM objects */
> > +       struct shrinker base;
> > +
> > +       /** @lock: Protects @lru_* */
> > +       struct mutex lock;
> > +
> > +       /** @lru_pinned: List of pinned shmem GEM objects */
> > +       struct list_head lru_pinned;
> > +
> > +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> > +       struct list_head lru_evictable;
> > +
> > +       /** @lru_evicted: List of evicted shmem GEM objects */
> > +       struct list_head lru_evicted;
> > +
> > +       /** @dev: DRM device that uses this shrinker */
> > +       struct drm_device *dev;
> > +};
> > +
> > +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> > +
> >  /*
> >   * Driver ops
> >   */
> > --
> > 2.35.3
> >
>
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Sun, 5 Jun 2022 at 20:32, Rob Clark <robdclark@gmail.com> wrote:
>
> On Sun, Jun 5, 2022 at 9:47 AM Daniel Vetter <daniel@ffwll.ch> wrote:
> >
> > On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> > <dmitry.osipenko@collabora.com> wrote:
> > >
> > > Introduce a common DRM SHMEM shrinker framework that allows to reduce
> > > code duplication among DRM drivers by replacing theirs custom shrinker
> > > implementations with the generic shrinker.
> > >
> > > In order to start using DRM SHMEM shrinker drivers should:
> > >
> > > 1. Implement new evict() shmem object callback.
> > > 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> > > 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
> > >    activate shrinking of shmem GEMs.
> > >
> > > This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> > > Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
> > >
> > > Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> > > Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> >
> > So I guess I get a price for being blind since forever, because this
> > thing existed since at least 2013. I just stumbled over
> > llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> > should try to adopt that so that our gpu shrinkers look more like
> > shrinkers for everything else.
>
> followup from a bit of irc discussion w/ danvet about list_lru:
>
> * It seems to be missing a way to bail out of iteration before
>   nr_to_scan is hit.. which is going to be inconvenient if you
>   want to allow active bos on the LRU but bail scanning once
>   you encounter the first one.
>
> * Not sure if the numa node awareness is super useful for GEM
>   bos
>
> First issue is perhaps not too hard to fix.  But maybe a better
> idea is a drm_gem_lru helper type thing which is more tailored
> to GEM buffers?

Yeah I guess reusing list_lru isn't that good idea. So just
open-coding it for now, and then drm_gem_bo_lru or so if we need to
share it separately from shmem helpers with other drivers. Maybe will
be needed for ttm or so.
-Daniel

>
> BR,
> -R
>
> > Apologies for this, since I fear this might cause a bit of churn.
> > Hopefully it's all contained to the list manipulation code in shmem
> > helpers, I don't think this should leak any further.
> > -Daniel
> >
> > > ---
> > >  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
> > >  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
> > >  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
> > >  include/drm/drm_device.h                      |   4 +
> > >  include/drm/drm_gem_shmem_helper.h            |  87 ++-
> > >  5 files changed, 594 insertions(+), 49 deletions(-)
> > >
> > > diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > index 555fe212bd98..4cd0b5913492 100644
> > > --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
> > >
> > > +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       return (shmem->madv >= 0) && shmem->evict &&
> > > +               shmem->eviction_enabled && shmem->pages_use_count &&
> > > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > > +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> > > +}
> > > +
> > > +static void
> > > +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> > > +
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (!gem_shrinker || obj->import_attach)
> > > +               return;
> > > +
> > > +       mutex_lock(&gem_shrinker->lock);
> > > +
> > > +       if (drm_gem_shmem_is_evictable(shmem) ||
> > > +           drm_gem_shmem_is_purgeable(shmem))
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> > > +       else if (shmem->madv < 0)
> > > +               list_del_init(&shmem->madv_list);
> > > +       else if (shmem->evicted)
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> > > +       else if (!shmem->pages)
> > > +               list_del_init(&shmem->madv_list);
> > > +       else
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +}
> > > +
> > >  /**
> > >   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
> > >   * @shmem: shmem GEM object to free
> > > @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >         } else {
> > >                 dma_resv_lock(shmem->base.resv, NULL);
> > >
> > > +               /* take out shmem GEM object from the memory shrinker */
> > > +               drm_gem_shmem_madvise(shmem, -1);
> > > +
> > >                 WARN_ON(shmem->vmap_use_count);
> > >
> > >                 if (shmem->sgt) {
> > > @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >                         sg_free_table(shmem->sgt);
> > >                         kfree(shmem->sgt);
> > >                 }
> > > -               if (shmem->pages)
> > > +               if (shmem->pages_use_count)
> > >                         drm_gem_shmem_put_pages(shmem);
> > >
> > >                 WARN_ON(shmem->pages_use_count);
> > > @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
> > >
> > > -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > > +/**
> > > + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> > > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       dma_resv_lock(shmem->base.resv, NULL);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       shmem->eviction_enabled = true;
> > > +
> > > +       dma_resv_unlock(shmem->base.resv);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> > > +
> > > +/**
> > > + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * Tell memory shrinker that this GEM can be purged. Initially purging is
> > > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       dma_resv_lock(shmem->base.resv, NULL);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       shmem->purge_enabled = true;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       dma_resv_unlock(shmem->base.resv);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> > > +
> > > +static int
> > > +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
> > >  {
> > >         struct drm_gem_object *obj = &shmem->base;
> > >         struct page **pages;
> > >
> > > -       if (shmem->pages_use_count++ > 0)
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (shmem->madv < 0) {
> > > +               WARN_ON(shmem->pages);
> > > +               return -ENOMEM;
> > > +       }
> > > +
> > > +       if (shmem->pages) {
> > > +               WARN_ON(!shmem->evicted);
> > >                 return 0;
> > > +       }
> > > +
> > > +       if (WARN_ON(!shmem->pages_use_count))
> > > +               return -EINVAL;
> > >
> > >         pages = drm_gem_get_pages(obj);
> > >         if (IS_ERR(pages)) {
> > >                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> > > -               shmem->pages_use_count = 0;
> > >                 return PTR_ERR(pages);
> > >         }
> > >
> > > @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > >         return 0;
> > >  }
> > >
> > > +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       int err;
> > > +
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       if (shmem->pages_use_count++ > 0) {
> > > +               err = drm_gem_shmem_swap_in(shmem);
> > > +               if (err)
> > > +                       goto err_zero_use;
> > > +
> > > +               return 0;
> > > +       }
> > > +
> > > +       err = drm_gem_shmem_acquire_pages(shmem);
> > > +       if (err)
> > > +               goto err_zero_use;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +
> > > +err_zero_use:
> > > +       shmem->pages_use_count = 0;
> > > +
> > > +       return err;
> > > +}
> > > +
> > > +static void
> > > +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +
> > > +       if (!shmem->pages) {
> > > +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> > > +               return;
> > > +       }
> > > +
> > > +#ifdef CONFIG_X86
> > > +       if (shmem->map_wc)
> > > +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > > +#endif
> > > +
> > > +       drm_gem_put_pages(obj, shmem->pages,
> > > +                         shmem->pages_mark_dirty_on_put,
> > > +                         shmem->pages_mark_accessed_on_put);
> > > +       shmem->pages = NULL;
> > > +}
> > > +
> > >  /*
> > >   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
> > >   * @shmem: shmem GEM object
> > > @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > >   */
> > >  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> > >  {
> > > -       struct drm_gem_object *obj = &shmem->base;
> > > -
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > >         if (WARN_ON_ONCE(!shmem->pages_use_count))
> > > @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> > >         if (--shmem->pages_use_count > 0)
> > >                 return;
> > >
> > > -#ifdef CONFIG_X86
> > > -       if (shmem->map_wc)
> > > -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > > -#endif
> > > +       drm_gem_shmem_release_pages(shmem);
> > >
> > > -       drm_gem_put_pages(obj, shmem->pages,
> > > -                         shmem->pages_mark_dirty_on_put,
> > > -                         shmem->pages_mark_accessed_on_put);
> > > -       shmem->pages = NULL;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> > >
> > > @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> > >   */
> > >  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
> > >  {
> > > +       int ret;
> > > +
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > > -       return drm_gem_shmem_get_pages(shmem);
> > > +       ret = drm_gem_shmem_get_pages(shmem);
> > > +       if (!ret)
> > > +               shmem->pages_pin_count++;
> > > +
> > > +       return ret;
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_pin);
> > >
> > > @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > >         drm_gem_shmem_put_pages(shmem);
> > > +
> > > +       shmem->pages_pin_count--;
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_unpin);
> > >
> > > @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> > >                         return 0;
> > >                 }
> > >
> > > -               ret = drm_gem_shmem_get_pages(shmem);
> > > +               ret = drm_gem_shmem_pin(shmem);
> > >                 if (ret)
> > >                         goto err_zero_use;
> > >
> > > @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> > >
> > >  err_put_pages:
> > >         if (!obj->import_attach)
> > > -               drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_unpin(shmem);
> > >  err_zero_use:
> > >         shmem->vmap_use_count = 0;
> > >
> > > @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
> > >                         return;
> > >
> > >                 vunmap(shmem->vaddr);
> > > -               drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_unpin(shmem);
> > >         }
> > >
> > >         shmem->vaddr = NULL;
> > > @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
> > >
> > >         madv = shmem->madv;
> > >
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > >         return (madv >= 0);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_madvise);
> > >
> > > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > > +/**
> > > + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> > > + *                           hardware access to the memory.
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * This function moves shmem GEM back to memory if it was previously evicted
> > > + * by the memory shrinker. The GEM is ready to use on success.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
> > >  {
> > >         struct drm_gem_object *obj = &shmem->base;
> > > -       struct drm_device *dev = obj->dev;
> > > +       struct sg_table *sgt;
> > > +       int err;
> > >
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > > -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > > +       if (shmem->evicted) {
> > > +               err = drm_gem_shmem_acquire_pages(shmem);
> > > +               if (err)
> > > +                       return err;
> > > +
> > > +               sgt = drm_gem_shmem_get_sg_table(shmem);
> > > +               if (IS_ERR(sgt))
> > > +                       return PTR_ERR(sgt);
> > > +
> > > +               err = dma_map_sgtable(obj->dev->dev, sgt,
> > > +                                     DMA_BIDIRECTIONAL, 0);
> > > +               if (err) {
> > > +                       sg_free_table(sgt);
> > > +                       kfree(sgt);
> > > +                       return err;
> > > +               }
> > >
> > > -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > > -       sg_free_table(shmem->sgt);
> > > -       kfree(shmem->sgt);
> > > -       shmem->sgt = NULL;
> > > +               shmem->sgt = sgt;
> > > +               shmem->evicted = false;
> > >
> > > -       drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_update_pages_state(shmem);
> > > +       }
> > >
> > > -       shmem->madv = -1;
> > > +       if (!shmem->pages)
> > > +               return -ENOMEM;
> > >
> > > -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > > -       drm_gem_free_mmap_offset(obj);
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
> > >
> > > -       /* Our goal here is to return as much of the memory as
> > > -        * is possible back to the system as we are called from OOM.
> > > -        * To do this we must instruct the shmfs to drop all of its
> > > -        * backing pages, *now*.
> > > -        */
> > > -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > > +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +       struct drm_device *dev = obj->dev;
> > >
> > > -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > > +       if (shmem->evicted)
> > > +               return;
> > > +
> > > +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > > +       drm_gem_shmem_release_pages(shmem);
> > > +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > > +
> > > +       sg_free_table(shmem->sgt);
> > > +       kfree(shmem->sgt);
> > > +       shmem->sgt = NULL;
> > >  }
> > > -EXPORT_SYMBOL(drm_gem_shmem_purge);
> > >
> > >  /**
> > >   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> > > @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
> > >         vm_fault_t ret;
> > >         struct page *page;
> > >         pgoff_t page_offset;
> > > +       bool pages_unpinned;
> > > +       int err;
> > >
> > >         /* We don't use vmf->pgoff since that has the fake offset */
> > >         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
> > >
> > >         dma_resv_lock(shmem->base.resv, NULL);
> > >
> > > -       if (page_offset >= num_pages ||
> > > -           WARN_ON_ONCE(!shmem->pages) ||
> > > -           shmem->madv < 0) {
> > > +       /* Sanity-check that we have the pages pointer when it should present */
> > > +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> > > +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> > > +
> > > +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
> > >                 ret = VM_FAULT_SIGBUS;
> > >         } else {
> > > +               err = drm_gem_shmem_swap_in(shmem);
> > > +               if (err) {
> > > +                       ret = VM_FAULT_OOM;
> > > +                       goto unlock;
> > > +               }
> > > +
> > >                 page = shmem->pages[page_offset];
> > >
> > >                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
> > >         }
> > >
> > > +unlock:
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         return ret;
> > > @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
> > >  {
> > >         struct drm_gem_object *obj = vma->vm_private_data;
> > >         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> > > -       int ret;
> > >
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > >         dma_resv_lock(shmem->base.resv, NULL);
> > > -       ret = drm_gem_shmem_get_pages(shmem);
> > > -       WARN_ON_ONCE(ret != 0);
> > > +
> > > +       if (drm_gem_shmem_get_pages(shmem))
> > > +               shmem->pages_use_count++;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         drm_gem_vm_open(vma);
> > > @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
> > >  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> > >                               struct drm_printer *p, unsigned int indent)
> > >  {
> > > +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> > > +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
> > >         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
> > >
> > >         if (shmem->base.import_attach)
> > > @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> > >                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
> > >                                   shmem->vmap_use_count);
> > >
> > > +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
> > >         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> > > +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_print_info);
> > >
> > > @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
> > >
> > >         shmem->sgt = sgt;
> > >
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         return sgt;
> > > @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
> > >
> > > +static struct drm_gem_shmem_shrinker *
> > > +to_drm_shrinker(struct shrinker *shrinker)
> > > +{
> > > +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> > > +}
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > > +                                    struct shrink_control *sc)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > > +       struct drm_gem_shmem_object *shmem;
> > > +       unsigned long count = 0;
> > > +
> > > +       if (!mutex_trylock(&gem_shrinker->lock))
> > > +               return 0;
> > > +
> > > +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > > +               count += shmem->base.size;
> > > +
> > > +               if (count >= SHRINK_EMPTY)
> > > +                       break;
> > > +       }
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +       if (count >= SHRINK_EMPTY)
> > > +               return SHRINK_EMPTY - 1;
> > > +
> > > +       return count ?: SHRINK_EMPTY;
> > > +}
> > > +
> > > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> > > +       WARN_ON(shmem->evicted);
> > > +
> > > +       drm_gem_shmem_unpin_pages(shmem);
> > > +
> > > +       shmem->evicted = true;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> > > +
> > > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +
> > > +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > > +
> > > +       drm_gem_shmem_unpin_pages(shmem);
> > > +       drm_gem_free_mmap_offset(obj);
> > > +
> > > +       /* Our goal here is to return as much of the memory as
> > > +        * is possible back to the system as we are called from OOM.
> > > +        * To do this we must instruct the shmfs to drop all of its
> > > +        * backing pages, *now*.
> > > +        */
> > > +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > > +
> > > +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > > +
> > > +       shmem->madv = -1;
> > > +       shmem->evicted = false;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> > > +                                       unsigned long nr_to_scan,
> > > +                                       bool *lock_contention,
> > > +                                       bool evict)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > > +       struct drm_gem_shmem_object *shmem;
> > > +       struct list_head still_in_list;
> > > +       struct drm_gem_object *obj;
> > > +       unsigned long freed = 0;
> > > +       size_t page_count;
> > > +       int err;
> > > +
> > > +       INIT_LIST_HEAD(&still_in_list);
> > > +
> > > +       mutex_lock(&gem_shrinker->lock);
> > > +
> > > +       while (freed < nr_to_scan) {
> > > +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> > > +                                                typeof(*shmem), madv_list);
> > > +               if (!shmem)
> > > +                       break;
> > > +
> > > +               obj = &shmem->base;
> > > +               page_count = obj->size >> PAGE_SHIFT;
> > > +               list_move_tail(&shmem->madv_list, &still_in_list);
> > > +
> > > +               if (evict) {
> > > +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> > > +                           get_nr_swap_pages() < page_count)
> > > +                               continue;
> > > +               } else {
> > > +                       if (!drm_gem_shmem_is_purgeable(shmem))
> > > +                               continue;
> > > +               }
> > > +
> > > +               /*
> > > +                * If it's in the process of being freed, gem_object->free()
> > > +                * may be blocked on lock waiting to remove it.  So just
> > > +                * skip it.
> > > +                */
> > > +               if (!kref_get_unless_zero(&obj->refcount))
> > > +                       continue;
> > > +
> > > +               mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +               /* prevent racing with job-submission code paths */
> > > +               if (!dma_resv_trylock(obj->resv)) {
> > > +                       *lock_contention |= true;
> > > +                       goto shrinker_lock;
> > > +               }
> > > +
> > > +               /* prevent racing with the dma-buf importing/exporting */
> > > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > > +                       *lock_contention |= true;
> > > +                       goto resv_unlock;
> > > +               }
> > > +
> > > +               /* check whether h/w uses this object */
> > > +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> > > +                       goto object_name_unlock;
> > > +
> > > +               /* re-check whether eviction status hasn't changed */
> > > +               if (!drm_gem_shmem_is_evictable(shmem) &&
> > > +                   !drm_gem_shmem_is_purgeable(shmem))
> > > +                       goto object_name_unlock;
> > > +
> > > +               err = shmem->evict(shmem);
> > > +               if (!err)
> > > +                       freed += obj->size >> PAGE_SHIFT;
> > > +
> > > +object_name_unlock:
> > > +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> > > +resv_unlock:
> > > +               dma_resv_unlock(obj->resv);
> > > +shrinker_lock:
> > > +               drm_gem_object_put(&shmem->base);
> > > +               mutex_lock(&gem_shrinker->lock);
> > > +       }
> > > +
> > > +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +       return freed;
> > > +}
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> > > +                                   struct shrink_control *sc)
> > > +{
> > > +       unsigned long nr_to_scan = sc->nr_to_scan;
> > > +       bool lock_contention = false;
> > > +       unsigned long freed;
> > > +
> > > +       /* purge as many objects as we can */
> > > +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> > > +                                                       &lock_contention, false);
> > > +
> > > +       /* evict as many objects as we can */
> > > +       if (freed < nr_to_scan)
> > > +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> > > +                                                                nr_to_scan - freed,
> > > +                                                                &lock_contention,
> > > +                                                                true);
> > > +
> > > +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> > > +}
> > > +
> > > +/**
> > > + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> > > + * @dev: DRM device
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker;
> > > +       int err;
> > > +
> > > +       if (WARN_ON(dev->shmem_shrinker))
> > > +               return -EBUSY;
> > > +
> > > +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> > > +       if (!gem_shrinker)
> > > +               return -ENOMEM;
> > > +
> > > +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> > > +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> > > +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> > > +       gem_shrinker->dev = dev;
> > > +
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> > > +       mutex_init(&gem_shrinker->lock);
> > > +
> > > +       dev->shmem_shrinker = gem_shrinker;
> > > +
> > > +       err = register_shrinker(&gem_shrinker->base);
> > > +       if (err) {
> > > +               dev->shmem_shrinker = NULL;
> > > +               kfree(gem_shrinker);
> > > +               return err;
> > > +       }
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> > > +
> > > +/**
> > > + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> > > + * @dev: DRM device
> > > + */
> > > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> > > +
> > > +       if (gem_shrinker) {
> > > +               unregister_shrinker(&gem_shrinker->base);
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> > > +               mutex_destroy(&gem_shrinker->lock);
> > > +               dev->shmem_shrinker = NULL;
> > > +               kfree(gem_shrinker);
> > > +       }
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> > > +
> > >  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
> > >  MODULE_IMPORT_NS(DMA_BUF);
> > >  MODULE_LICENSE("GPL v2");
> > > diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > index a4bedfeb2ec4..7cc32556f908 100644
> > > --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > @@ -15,6 +15,13 @@
> > >  #include "panfrost_gem.h"
> > >  #include "panfrost_mmu.h"
> > >
> > > +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       return (shmem->madv > 0) &&
> > > +               !shmem->pages_pin_count && shmem->sgt &&
> > > +               !shmem->base.dma_buf && !shmem->base.import_attach;
> > > +}
> > > +
> > >  static unsigned long
> > >  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> > >  {
> > > @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
> > >                 return 0;
> > >
> > >         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> > > -               if (drm_gem_shmem_is_purgeable(shmem))
> > > +               if (panfrost_gem_shmem_is_purgeable(shmem))
> > >                         count += shmem->base.size >> PAGE_SHIFT;
> > >         }
> > >
> > > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > index b2d93cb12ebf..81bacc7e1873 100644
> > > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > @@ -89,6 +89,7 @@ struct virtio_gpu_object {
> > >         uint32_t hw_res_handle;
> > >         bool dumb;
> > >         bool created;
> > > +       bool detached;
> > >         bool host3d_blob, guest_blob;
> > >         uint32_t blob_mem, blob_flags;
> > >
> > > @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> > >
> > >  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
> > >
> > > +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> > > +
> > >  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> > >                                uint32_t *resid);
> > >  /* virtgpu_prime.c */
> > > diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> > > index 9923c7a6885e..929546cad894 100644
> > > --- a/include/drm/drm_device.h
> > > +++ b/include/drm/drm_device.h
> > > @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
> > >  struct drm_vma_offset_manager;
> > >  struct drm_vram_mm;
> > >  struct drm_fb_helper;
> > > +struct drm_gem_shmem_shrinker;
> > >
> > >  struct inode;
> > >
> > > @@ -277,6 +278,9 @@ struct drm_device {
> > >         /** @vram_mm: VRAM MM memory manager */
> > >         struct drm_vram_mm *vram_mm;
> > >
> > > +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> > > +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> > > +
> > >         /**
> > >          * @switch_power_state:
> > >          *
> > > diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> > > index 9a8983ee8abe..62c640678a91 100644
> > > --- a/include/drm/drm_gem_shmem_helper.h
> > > +++ b/include/drm/drm_gem_shmem_helper.h
> > > @@ -6,6 +6,7 @@
> > >  #include <linux/fs.h>
> > >  #include <linux/mm.h>
> > >  #include <linux/mutex.h>
> > > +#include <linux/shrinker.h>
> > >
> > >  #include <drm/drm_file.h>
> > >  #include <drm/drm_gem.h>
> > > @@ -15,6 +16,7 @@
> > >  struct dma_buf_attachment;
> > >  struct drm_mode_create_dumb;
> > >  struct drm_printer;
> > > +struct drm_device;
> > >  struct sg_table;
> > >
> > >  /**
> > > @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
> > >          */
> > >         unsigned int pages_use_count;
> > >
> > > +       /**
> > > +        * @pages_pin_count:
> > > +        *
> > > +        * Reference count on the pinned pages table.
> > > +        * The pages can be evicted by memory shrinker
> > > +        * when the count reaches zero.
> > > +        */
> > > +       unsigned int pages_pin_count;
> > > +
> > >         /**
> > >          * @madv: State for madvise
> > >          *
> > >          * 0 is active/inuse.
> > > +        * 1 is not-needed/can-be-purged
> > >          * A negative value is the object is purged.
> > > -        * Positive values are driver specific and not used by the helpers.
> > >          */
> > >         int madv;
> > >
> > > @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
> > >          * @map_wc: map object write-combined (instead of using shmem defaults).
> > >          */
> > >         bool map_wc;
> > > +
> > > +       /**
> > > +        * @eviction_enabled:
> > > +        *
> > > +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool eviction_enabled;
> > > +
> > > +       /**
> > > +        * @purge_enabled:
> > > +        *
> > > +        * The shmem pages can be purged only if @purge_enabled is set to true.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool purge_enabled;
> > > +
> > > +       /**
> > > +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool evicted;
> > > +
> > > +       /**
> > > +        * @evict:
> > > +        *
> > > +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> > > +        * GEM's DMA reservation is kept locked by the shrinker. This is
> > > +        * optional callback that should be specified by drivers.
> > > +        *
> > > +        * Returns 0 on success, or -errno on error.
> > > +        */
> > > +       int (*evict)(struct drm_gem_shmem_object *shmem);
> > >  };
> > >
> > >  #define to_drm_gem_shmem_obj(obj) \
> > > @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
> > >
> > >  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
> > >
> > > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> > > +
> > >  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > >  {
> > > -       return (shmem->madv > 0) &&
> > > -               !shmem->vmap_use_count && shmem->sgt &&
> > > -               !shmem->base.dma_buf && !shmem->base.import_attach;
> > > +       return (shmem->madv > 0) && shmem->evict &&
> > > +               shmem->purge_enabled && shmem->pages_use_count &&
> > > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > > +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
> > >  }
> > >
> > > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> > > +
> > > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > >
> > >  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
> > >  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> > > @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
> > >         return drm_gem_shmem_mmap(shmem, vma);
> > >  }
> > >
> > > +/**
> > > + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> > > + */
> > > +struct drm_gem_shmem_shrinker {
> > > +       /** @base: Shrinker for purging shmem GEM objects */
> > > +       struct shrinker base;
> > > +
> > > +       /** @lock: Protects @lru_* */
> > > +       struct mutex lock;
> > > +
> > > +       /** @lru_pinned: List of pinned shmem GEM objects */
> > > +       struct list_head lru_pinned;
> > > +
> > > +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> > > +       struct list_head lru_evictable;
> > > +
> > > +       /** @lru_evicted: List of evicted shmem GEM objects */
> > > +       struct list_head lru_evicted;
> > > +
> > > +       /** @dev: DRM device that uses this shrinker */
> > > +       struct drm_device *dev;
> > > +};
> > > +
> > > +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> > > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> > > +
> > >  /*
> > >   * Driver ops
> > >   */
> > > --
> > > 2.35.3
> > >
> >
> >
> > --
> > Daniel Vetter
> > Software Engineer, Intel Corporation
> > http://blog.ffwll.ch



-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Christian König]

Am 05.06.22 um 18:47 schrieb Daniel Vetter:
> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
>> Introduce a common DRM SHMEM shrinker framework that allows to reduce
>> code duplication among DRM drivers by replacing theirs custom shrinker
>> implementations with the generic shrinker.
>>
>> In order to start using DRM SHMEM shrinker drivers should:
>>
>> 1. Implement new evict() shmem object callback.
>> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
>> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>>     activate shrinking of shmem GEMs.
>>
>> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
>> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>>
>> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> So I guess I get a price for being blind since forever, because this
> thing existed since at least 2013. I just stumbled over
> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> should try to adopt that so that our gpu shrinkers look more like
> shrinkers for everything else.

What the heck are you talking about?

I can't find any llist_lru.[hc] in the linux kernel sources.

Christian.

>
> Apologies for this, since I fear this might cause a bit of churn.
> Hopefully it's all contained to the list manipulation code in shmem
> helpers, I don't think this should leak any further.
> -Daniel
>
>> ---
>>   drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>>   .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>>   drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>>   include/drm/drm_device.h                      |   4 +
>>   include/drm/drm_gem_shmem_helper.h            |  87 ++-
>>   5 files changed, 594 insertions(+), 49 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> index 555fe212bd98..4cd0b5913492 100644
>> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
>> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>>
>> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       return (shmem->madv >= 0) && shmem->evict &&
>> +               shmem->eviction_enabled && shmem->pages_use_count &&
>> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
>> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
>> +}
>> +
>> +static void
>> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
>> +
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (!gem_shrinker || obj->import_attach)
>> +               return;
>> +
>> +       mutex_lock(&gem_shrinker->lock);
>> +
>> +       if (drm_gem_shmem_is_evictable(shmem) ||
>> +           drm_gem_shmem_is_purgeable(shmem))
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
>> +       else if (shmem->madv < 0)
>> +               list_del_init(&shmem->madv_list);
>> +       else if (shmem->evicted)
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
>> +       else if (!shmem->pages)
>> +               list_del_init(&shmem->madv_list);
>> +       else
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +}
>> +
>>   /**
>>    * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>>    * @shmem: shmem GEM object to free
>> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>          } else {
>>                  dma_resv_lock(shmem->base.resv, NULL);
>>
>> +               /* take out shmem GEM object from the memory shrinker */
>> +               drm_gem_shmem_madvise(shmem, -1);
>> +
>>                  WARN_ON(shmem->vmap_use_count);
>>
>>                  if (shmem->sgt) {
>> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>                          sg_free_table(shmem->sgt);
>>                          kfree(shmem->sgt);
>>                  }
>> -               if (shmem->pages)
>> +               if (shmem->pages_use_count)
>>                          drm_gem_shmem_put_pages(shmem);
>>
>>                  WARN_ON(shmem->pages_use_count);
>> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>>
>> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>> +/**
>> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
>> + * @shmem: shmem GEM object
>> + *
>> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
>> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       dma_resv_lock(shmem->base.resv, NULL);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       shmem->eviction_enabled = true;
>> +
>> +       dma_resv_unlock(shmem->base.resv);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
>> +
>> +/**
>> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
>> + * @shmem: shmem GEM object
>> + *
>> + * Tell memory shrinker that this GEM can be purged. Initially purging is
>> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       dma_resv_lock(shmem->base.resv, NULL);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       shmem->purge_enabled = true;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       dma_resv_unlock(shmem->base.resv);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
>> +
>> +static int
>> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>>   {
>>          struct drm_gem_object *obj = &shmem->base;
>>          struct page **pages;
>>
>> -       if (shmem->pages_use_count++ > 0)
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (shmem->madv < 0) {
>> +               WARN_ON(shmem->pages);
>> +               return -ENOMEM;
>> +       }
>> +
>> +       if (shmem->pages) {
>> +               WARN_ON(!shmem->evicted);
>>                  return 0;
>> +       }
>> +
>> +       if (WARN_ON(!shmem->pages_use_count))
>> +               return -EINVAL;
>>
>>          pages = drm_gem_get_pages(obj);
>>          if (IS_ERR(pages)) {
>>                  DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
>> -               shmem->pages_use_count = 0;
>>                  return PTR_ERR(pages);
>>          }
>>
>> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>>          return 0;
>>   }
>>
>> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       int err;
>> +
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       if (shmem->pages_use_count++ > 0) {
>> +               err = drm_gem_shmem_swap_in(shmem);
>> +               if (err)
>> +                       goto err_zero_use;
>> +
>> +               return 0;
>> +       }
>> +
>> +       err = drm_gem_shmem_acquire_pages(shmem);
>> +       if (err)
>> +               goto err_zero_use;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +
>> +err_zero_use:
>> +       shmem->pages_use_count = 0;
>> +
>> +       return err;
>> +}
>> +
>> +static void
>> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +
>> +       if (!shmem->pages) {
>> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
>> +               return;
>> +       }
>> +
>> +#ifdef CONFIG_X86
>> +       if (shmem->map_wc)
>> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
>> +#endif
>> +
>> +       drm_gem_put_pages(obj, shmem->pages,
>> +                         shmem->pages_mark_dirty_on_put,
>> +                         shmem->pages_mark_accessed_on_put);
>> +       shmem->pages = NULL;
>> +}
>> +
>>   /*
>>    * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>>    * @shmem: shmem GEM object
>> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>>    */
>>   void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>>   {
>> -       struct drm_gem_object *obj = &shmem->base;
>> -
>>          dma_resv_assert_held(shmem->base.resv);
>>
>>          if (WARN_ON_ONCE(!shmem->pages_use_count))
>> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>>          if (--shmem->pages_use_count > 0)
>>                  return;
>>
>> -#ifdef CONFIG_X86
>> -       if (shmem->map_wc)
>> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
>> -#endif
>> +       drm_gem_shmem_release_pages(shmem);
>>
>> -       drm_gem_put_pages(obj, shmem->pages,
>> -                         shmem->pages_mark_dirty_on_put,
>> -                         shmem->pages_mark_accessed_on_put);
>> -       shmem->pages = NULL;
>> +       drm_gem_shmem_update_pages_state(shmem);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>>
>> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>>    */
>>   int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>>   {
>> +       int ret;
>> +
>>          dma_resv_assert_held(shmem->base.resv);
>>
>>          WARN_ON(shmem->base.import_attach);
>>
>> -       return drm_gem_shmem_get_pages(shmem);
>> +       ret = drm_gem_shmem_get_pages(shmem);
>> +       if (!ret)
>> +               shmem->pages_pin_count++;
>> +
>> +       return ret;
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_pin);
>>
>> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>>          WARN_ON(shmem->base.import_attach);
>>
>>          drm_gem_shmem_put_pages(shmem);
>> +
>> +       shmem->pages_pin_count--;
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_unpin);
>>
>> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>>                          return 0;
>>                  }
>>
>> -               ret = drm_gem_shmem_get_pages(shmem);
>> +               ret = drm_gem_shmem_pin(shmem);
>>                  if (ret)
>>                          goto err_zero_use;
>>
>> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>>
>>   err_put_pages:
>>          if (!obj->import_attach)
>> -               drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_unpin(shmem);
>>   err_zero_use:
>>          shmem->vmap_use_count = 0;
>>
>> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>>                          return;
>>
>>                  vunmap(shmem->vaddr);
>> -               drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_unpin(shmem);
>>          }
>>
>>          shmem->vaddr = NULL;
>> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>>
>>          madv = shmem->madv;
>>
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>>          return (madv >= 0);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_madvise);
>>
>> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
>> +/**
>> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
>> + *                           hardware access to the memory.
>> + * @shmem: shmem GEM object
>> + *
>> + * This function moves shmem GEM back to memory if it was previously evicted
>> + * by the memory shrinker. The GEM is ready to use on success.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>>   {
>>          struct drm_gem_object *obj = &shmem->base;
>> -       struct drm_device *dev = obj->dev;
>> +       struct sg_table *sgt;
>> +       int err;
>>
>>          dma_resv_assert_held(shmem->base.resv);
>>
>> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
>> +       if (shmem->evicted) {
>> +               err = drm_gem_shmem_acquire_pages(shmem);
>> +               if (err)
>> +                       return err;
>> +
>> +               sgt = drm_gem_shmem_get_sg_table(shmem);
>> +               if (IS_ERR(sgt))
>> +                       return PTR_ERR(sgt);
>> +
>> +               err = dma_map_sgtable(obj->dev->dev, sgt,
>> +                                     DMA_BIDIRECTIONAL, 0);
>> +               if (err) {
>> +                       sg_free_table(sgt);
>> +                       kfree(sgt);
>> +                       return err;
>> +               }
>>
>> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
>> -       sg_free_table(shmem->sgt);
>> -       kfree(shmem->sgt);
>> -       shmem->sgt = NULL;
>> +               shmem->sgt = sgt;
>> +               shmem->evicted = false;
>>
>> -       drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_update_pages_state(shmem);
>> +       }
>>
>> -       shmem->madv = -1;
>> +       if (!shmem->pages)
>> +               return -ENOMEM;
>>
>> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
>> -       drm_gem_free_mmap_offset(obj);
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>>
>> -       /* Our goal here is to return as much of the memory as
>> -        * is possible back to the system as we are called from OOM.
>> -        * To do this we must instruct the shmfs to drop all of its
>> -        * backing pages, *now*.
>> -        */
>> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
>> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +       struct drm_device *dev = obj->dev;
>>
>> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
>> +       if (shmem->evicted)
>> +               return;
>> +
>> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
>> +       drm_gem_shmem_release_pages(shmem);
>> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
>> +
>> +       sg_free_table(shmem->sgt);
>> +       kfree(shmem->sgt);
>> +       shmem->sgt = NULL;
>>   }
>> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>>
>>   /**
>>    * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
>> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>>          vm_fault_t ret;
>>          struct page *page;
>>          pgoff_t page_offset;
>> +       bool pages_unpinned;
>> +       int err;
>>
>>          /* We don't use vmf->pgoff since that has the fake offset */
>>          page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>>
>>          dma_resv_lock(shmem->base.resv, NULL);
>>
>> -       if (page_offset >= num_pages ||
>> -           WARN_ON_ONCE(!shmem->pages) ||
>> -           shmem->madv < 0) {
>> +       /* Sanity-check that we have the pages pointer when it should present */
>> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
>> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
>> +
>> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>>                  ret = VM_FAULT_SIGBUS;
>>          } else {
>> +               err = drm_gem_shmem_swap_in(shmem);
>> +               if (err) {
>> +                       ret = VM_FAULT_OOM;
>> +                       goto unlock;
>> +               }
>> +
>>                  page = shmem->pages[page_offset];
>>
>>                  ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>>          }
>>
>> +unlock:
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          return ret;
>> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>>   {
>>          struct drm_gem_object *obj = vma->vm_private_data;
>>          struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
>> -       int ret;
>>
>>          WARN_ON(shmem->base.import_attach);
>>
>>          dma_resv_lock(shmem->base.resv, NULL);
>> -       ret = drm_gem_shmem_get_pages(shmem);
>> -       WARN_ON_ONCE(ret != 0);
>> +
>> +       if (drm_gem_shmem_get_pages(shmem))
>> +               shmem->pages_use_count++;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          drm_gem_vm_open(vma);
>> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>>   void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>>                                struct drm_printer *p, unsigned int indent)
>>   {
>> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
>> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>>          drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>>
>>          if (shmem->base.import_attach)
>> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>>                  drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>>                                    shmem->vmap_use_count);
>>
>> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>>          drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
>> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_print_info);
>>
>> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>>
>>          shmem->sgt = sgt;
>>
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          return sgt;
>> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>>
>> +static struct drm_gem_shmem_shrinker *
>> +to_drm_shrinker(struct shrinker *shrinker)
>> +{
>> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
>> +}
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
>> +                                    struct shrink_control *sc)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
>> +       struct drm_gem_shmem_object *shmem;
>> +       unsigned long count = 0;
>> +
>> +       if (!mutex_trylock(&gem_shrinker->lock))
>> +               return 0;
>> +
>> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
>> +               count += shmem->base.size;
>> +
>> +               if (count >= SHRINK_EMPTY)
>> +                       break;
>> +       }
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +
>> +       if (count >= SHRINK_EMPTY)
>> +               return SHRINK_EMPTY - 1;
>> +
>> +       return count ?: SHRINK_EMPTY;
>> +}
>> +
>> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
>> +{
>> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
>> +       WARN_ON(shmem->evicted);
>> +
>> +       drm_gem_shmem_unpin_pages(shmem);
>> +
>> +       shmem->evicted = true;
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
>> +
>> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +
>> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
>> +
>> +       drm_gem_shmem_unpin_pages(shmem);
>> +       drm_gem_free_mmap_offset(obj);
>> +
>> +       /* Our goal here is to return as much of the memory as
>> +        * is possible back to the system as we are called from OOM.
>> +        * To do this we must instruct the shmfs to drop all of its
>> +        * backing pages, *now*.
>> +        */
>> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
>> +
>> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
>> +
>> +       shmem->madv = -1;
>> +       shmem->evicted = false;
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
>> +                                       unsigned long nr_to_scan,
>> +                                       bool *lock_contention,
>> +                                       bool evict)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
>> +       struct drm_gem_shmem_object *shmem;
>> +       struct list_head still_in_list;
>> +       struct drm_gem_object *obj;
>> +       unsigned long freed = 0;
>> +       size_t page_count;
>> +       int err;
>> +
>> +       INIT_LIST_HEAD(&still_in_list);
>> +
>> +       mutex_lock(&gem_shrinker->lock);
>> +
>> +       while (freed < nr_to_scan) {
>> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
>> +                                                typeof(*shmem), madv_list);
>> +               if (!shmem)
>> +                       break;
>> +
>> +               obj = &shmem->base;
>> +               page_count = obj->size >> PAGE_SHIFT;
>> +               list_move_tail(&shmem->madv_list, &still_in_list);
>> +
>> +               if (evict) {
>> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
>> +                           get_nr_swap_pages() < page_count)
>> +                               continue;
>> +               } else {
>> +                       if (!drm_gem_shmem_is_purgeable(shmem))
>> +                               continue;
>> +               }
>> +
>> +               /*
>> +                * If it's in the process of being freed, gem_object->free()
>> +                * may be blocked on lock waiting to remove it.  So just
>> +                * skip it.
>> +                */
>> +               if (!kref_get_unless_zero(&obj->refcount))
>> +                       continue;
>> +
>> +               mutex_unlock(&gem_shrinker->lock);
>> +
>> +               /* prevent racing with job-submission code paths */
>> +               if (!dma_resv_trylock(obj->resv)) {
>> +                       *lock_contention |= true;
>> +                       goto shrinker_lock;
>> +               }
>> +
>> +               /* prevent racing with the dma-buf importing/exporting */
>> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
>> +                       *lock_contention |= true;
>> +                       goto resv_unlock;
>> +               }
>> +
>> +               /* check whether h/w uses this object */
>> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
>> +                       goto object_name_unlock;
>> +
>> +               /* re-check whether eviction status hasn't changed */
>> +               if (!drm_gem_shmem_is_evictable(shmem) &&
>> +                   !drm_gem_shmem_is_purgeable(shmem))
>> +                       goto object_name_unlock;
>> +
>> +               err = shmem->evict(shmem);
>> +               if (!err)
>> +                       freed += obj->size >> PAGE_SHIFT;
>> +
>> +object_name_unlock:
>> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
>> +resv_unlock:
>> +               dma_resv_unlock(obj->resv);
>> +shrinker_lock:
>> +               drm_gem_object_put(&shmem->base);
>> +               mutex_lock(&gem_shrinker->lock);
>> +       }
>> +
>> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +
>> +       return freed;
>> +}
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
>> +                                   struct shrink_control *sc)
>> +{
>> +       unsigned long nr_to_scan = sc->nr_to_scan;
>> +       bool lock_contention = false;
>> +       unsigned long freed;
>> +
>> +       /* purge as many objects as we can */
>> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
>> +                                                       &lock_contention, false);
>> +
>> +       /* evict as many objects as we can */
>> +       if (freed < nr_to_scan)
>> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
>> +                                                                nr_to_scan - freed,
>> +                                                                &lock_contention,
>> +                                                                true);
>> +
>> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
>> +}
>> +
>> +/**
>> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
>> + * @dev: DRM device
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker;
>> +       int err;
>> +
>> +       if (WARN_ON(dev->shmem_shrinker))
>> +               return -EBUSY;
>> +
>> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
>> +       if (!gem_shrinker)
>> +               return -ENOMEM;
>> +
>> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
>> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
>> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
>> +       gem_shrinker->dev = dev;
>> +
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
>> +       mutex_init(&gem_shrinker->lock);
>> +
>> +       dev->shmem_shrinker = gem_shrinker;
>> +
>> +       err = register_shrinker(&gem_shrinker->base);
>> +       if (err) {
>> +               dev->shmem_shrinker = NULL;
>> +               kfree(gem_shrinker);
>> +               return err;
>> +       }
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
>> +
>> +/**
>> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
>> + * @dev: DRM device
>> + */
>> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
>> +
>> +       if (gem_shrinker) {
>> +               unregister_shrinker(&gem_shrinker->base);
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
>> +               mutex_destroy(&gem_shrinker->lock);
>> +               dev->shmem_shrinker = NULL;
>> +               kfree(gem_shrinker);
>> +       }
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
>> +
>>   MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>>   MODULE_IMPORT_NS(DMA_BUF);
>>   MODULE_LICENSE("GPL v2");
>> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> index a4bedfeb2ec4..7cc32556f908 100644
>> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> @@ -15,6 +15,13 @@
>>   #include "panfrost_gem.h"
>>   #include "panfrost_mmu.h"
>>
>> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       return (shmem->madv > 0) &&
>> +               !shmem->pages_pin_count && shmem->sgt &&
>> +               !shmem->base.dma_buf && !shmem->base.import_attach;
>> +}
>> +
>>   static unsigned long
>>   panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>>   {
>> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>>                  return 0;
>>
>>          list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
>> -               if (drm_gem_shmem_is_purgeable(shmem))
>> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>>                          count += shmem->base.size >> PAGE_SHIFT;
>>          }
>>
>> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
>> index b2d93cb12ebf..81bacc7e1873 100644
>> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
>> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
>> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>>          uint32_t hw_res_handle;
>>          bool dumb;
>>          bool created;
>> +       bool detached;
>>          bool host3d_blob, guest_blob;
>>          uint32_t blob_mem, blob_flags;
>>
>> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>>
>>   bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>>
>> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
>> +
>>   int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>>                                 uint32_t *resid);
>>   /* virtgpu_prime.c */
>> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
>> index 9923c7a6885e..929546cad894 100644
>> --- a/include/drm/drm_device.h
>> +++ b/include/drm/drm_device.h
>> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>>   struct drm_vma_offset_manager;
>>   struct drm_vram_mm;
>>   struct drm_fb_helper;
>> +struct drm_gem_shmem_shrinker;
>>
>>   struct inode;
>>
>> @@ -277,6 +278,9 @@ struct drm_device {
>>          /** @vram_mm: VRAM MM memory manager */
>>          struct drm_vram_mm *vram_mm;
>>
>> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
>> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
>> +
>>          /**
>>           * @switch_power_state:
>>           *
>> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
>> index 9a8983ee8abe..62c640678a91 100644
>> --- a/include/drm/drm_gem_shmem_helper.h
>> +++ b/include/drm/drm_gem_shmem_helper.h
>> @@ -6,6 +6,7 @@
>>   #include <linux/fs.h>
>>   #include <linux/mm.h>
>>   #include <linux/mutex.h>
>> +#include <linux/shrinker.h>
>>
>>   #include <drm/drm_file.h>
>>   #include <drm/drm_gem.h>
>> @@ -15,6 +16,7 @@
>>   struct dma_buf_attachment;
>>   struct drm_mode_create_dumb;
>>   struct drm_printer;
>> +struct drm_device;
>>   struct sg_table;
>>
>>   /**
>> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>>           */
>>          unsigned int pages_use_count;
>>
>> +       /**
>> +        * @pages_pin_count:
>> +        *
>> +        * Reference count on the pinned pages table.
>> +        * The pages can be evicted by memory shrinker
>> +        * when the count reaches zero.
>> +        */
>> +       unsigned int pages_pin_count;
>> +
>>          /**
>>           * @madv: State for madvise
>>           *
>>           * 0 is active/inuse.
>> +        * 1 is not-needed/can-be-purged
>>           * A negative value is the object is purged.
>> -        * Positive values are driver specific and not used by the helpers.
>>           */
>>          int madv;
>>
>> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>>           * @map_wc: map object write-combined (instead of using shmem defaults).
>>           */
>>          bool map_wc;
>> +
>> +       /**
>> +        * @eviction_enabled:
>> +        *
>> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool eviction_enabled;
>> +
>> +       /**
>> +        * @purge_enabled:
>> +        *
>> +        * The shmem pages can be purged only if @purge_enabled is set to true.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool purge_enabled;
>> +
>> +       /**
>> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool evicted;
>> +
>> +       /**
>> +        * @evict:
>> +        *
>> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
>> +        * GEM's DMA reservation is kept locked by the shrinker. This is
>> +        * optional callback that should be specified by drivers.
>> +        *
>> +        * Returns 0 on success, or -errno on error.
>> +        */
>> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>>   };
>>
>>   #define to_drm_gem_shmem_obj(obj) \
>> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>>
>>   int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>>
>> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
>> +
>>   static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>>   {
>> -       return (shmem->madv > 0) &&
>> -               !shmem->vmap_use_count && shmem->sgt &&
>> -               !shmem->base.dma_buf && !shmem->base.import_attach;
>> +       return (shmem->madv > 0) && shmem->evict &&
>> +               shmem->purge_enabled && shmem->pages_use_count &&
>> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
>> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>>   }
>>
>> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
>> +
>> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>>
>>   struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>>   struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
>> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>>          return drm_gem_shmem_mmap(shmem, vma);
>>   }
>>
>> +/**
>> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
>> + */
>> +struct drm_gem_shmem_shrinker {
>> +       /** @base: Shrinker for purging shmem GEM objects */
>> +       struct shrinker base;
>> +
>> +       /** @lock: Protects @lru_* */
>> +       struct mutex lock;
>> +
>> +       /** @lru_pinned: List of pinned shmem GEM objects */
>> +       struct list_head lru_pinned;
>> +
>> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
>> +       struct list_head lru_evictable;
>> +
>> +       /** @lru_evicted: List of evicted shmem GEM objects */
>> +       struct list_head lru_evicted;
>> +
>> +       /** @dev: DRM device that uses this shrinker */
>> +       struct drm_device *dev;
>> +};
>> +
>> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
>> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
>> +
>>   /*
>>    * Driver ops
>>    */
>> --
>> 2.35.3
>>
>


_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);

As I mentioned on other thread, count_objects, being approximate but
lockless and fast is the important thing.  Otherwise when you start
hitting the shrinker on many threads, you end up serializing them all,
even if you have no pages to return to the system at that point.

> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }

I'm not sure this is a good idea to serialize on object_name_lock.
Purgeable buffers should never be shared (imported or exported).  So
at best you are avoiding evicting and immediately swapping back in, in
a rare case, at the cost of serializing multiple threads trying to
reclaim pages in parallel.

BR,
-R

> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)
> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Mon, Jun 20, 2022 at 7:09 AM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> On 6/19/22 20:53, Rob Clark wrote:
> ...
> >> +static unsigned long
> >> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> >> +                                    struct shrink_control *sc)
> >> +{
> >> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> >> +       struct drm_gem_shmem_object *shmem;
> >> +       unsigned long count = 0;
> >> +
> >> +       if (!mutex_trylock(&gem_shrinker->lock))
> >> +               return 0;
> >> +
> >> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> >> +               count += shmem->base.size;
> >> +
> >> +               if (count >= SHRINK_EMPTY)
> >> +                       break;
> >> +       }
> >> +
> >> +       mutex_unlock(&gem_shrinker->lock);
> >
> > As I mentioned on other thread, count_objects, being approximate but
> > lockless and fast is the important thing.  Otherwise when you start
> > hitting the shrinker on many threads, you end up serializing them all,
> > even if you have no pages to return to the system at that point.
>
> Daniel's point for dropping the lockless variant was that we're already
> in trouble if we're hitting shrinker too often and extra optimizations
> won't bring much benefits to us.

At least with zram swap (which I highly recommend using even if you
are not using a physical swap file/partition), swapin/out is actually
quite fast.  And if you are leaning on zram swap to fit 8GB of chrome
browser on a 4GB device, the shrinker gets hit quite a lot.  Lower
spec (4GB RAM) chromebooks can be under constant memory pressure and
can quite easily get into a situation where you are hitting the
shrinker on many threads simultaneously.  So it is pretty important
for all shrinkers in the system (not just drm driver) to be as
concurrent as possible.  As long as you avoid serializing reclaim on
all the threads, performance can still be quite good, but if you don't
performance will fall off a cliff.

jfwiw, we are seeing pretty good results (iirc 40-70% increase in open
tab counts) with the combination of eviction + multigen LRU[1] +
sizing zram swap to be 2x physical RAM

[1] https://lwn.net/Articles/856931/

> Alright, I'll add back the lockless variant (or will use yours
> drm_gem_lru) in the next revision. The code difference is very small
> after all.
>
> ...
> >> +               /* prevent racing with the dma-buf importing/exporting */
> >> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> >> +                       *lock_contention |= true;
> >> +                       goto resv_unlock;
> >> +               }
> >
> > I'm not sure this is a good idea to serialize on object_name_lock.
> > Purgeable buffers should never be shared (imported or exported).  So
> > at best you are avoiding evicting and immediately swapping back in, in
> > a rare case, at the cost of serializing multiple threads trying to
> > reclaim pages in parallel.
>
> The object_name_lock shouldn't cause contention in practice. But objects
> are also pinned on attachment, hence maybe this lock is indeed
> unnecessary.. I'll re-check it.

I'm not worried about contention with export/import/etc, but
contention between multiple threads hitting the shrinker in parallel.
I guess since you are using trylock, it won't *block* the other
threads hitting shrinker, but they'll just end up looping in
do_shrink_slab() because they are hitting contention.

I'd have to do some experiments to see how it works out in practice,
but my gut feel is that it isn't a good idea

BR,
-R

> --
> Best regards,
> Dmitry
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Mon, Jun 20, 2022 at 08:18:04AM -0700, Rob Clark wrote:
> On Mon, Jun 20, 2022 at 7:09 AM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> >
> > On 6/19/22 20:53, Rob Clark wrote:
> > ...
> > >> +static unsigned long
> > >> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > >> +                                    struct shrink_control *sc)
> > >> +{
> > >> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > >> +       struct drm_gem_shmem_object *shmem;
> > >> +       unsigned long count = 0;
> > >> +
> > >> +       if (!mutex_trylock(&gem_shrinker->lock))
> > >> +               return 0;
> > >> +
> > >> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > >> +               count += shmem->base.size;
> > >> +
> > >> +               if (count >= SHRINK_EMPTY)
> > >> +                       break;
> > >> +       }
> > >> +
> > >> +       mutex_unlock(&gem_shrinker->lock);
> > >
> > > As I mentioned on other thread, count_objects, being approximate but
> > > lockless and fast is the important thing.  Otherwise when you start
> > > hitting the shrinker on many threads, you end up serializing them all,
> > > even if you have no pages to return to the system at that point.
> >
> > Daniel's point for dropping the lockless variant was that we're already
> > in trouble if we're hitting shrinker too often and extra optimizations
> > won't bring much benefits to us.
> 
> At least with zram swap (which I highly recommend using even if you
> are not using a physical swap file/partition), swapin/out is actually
> quite fast.  And if you are leaning on zram swap to fit 8GB of chrome
> browser on a 4GB device, the shrinker gets hit quite a lot.  Lower
> spec (4GB RAM) chromebooks can be under constant memory pressure and
> can quite easily get into a situation where you are hitting the
> shrinker on many threads simultaneously.  So it is pretty important
> for all shrinkers in the system (not just drm driver) to be as
> concurrent as possible.  As long as you avoid serializing reclaim on
> all the threads, performance can still be quite good, but if you don't
> performance will fall off a cliff.
> 
> jfwiw, we are seeing pretty good results (iirc 40-70% increase in open
> tab counts) with the combination of eviction + multigen LRU[1] +
> sizing zram swap to be 2x physical RAM
> 
> [1] https://lwn.net/Articles/856931/
> 
> > Alright, I'll add back the lockless variant (or will use yours
> > drm_gem_lru) in the next revision. The code difference is very small
> > after all.
> >
> > ...
> > >> +               /* prevent racing with the dma-buf importing/exporting */
> > >> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > >> +                       *lock_contention |= true;
> > >> +                       goto resv_unlock;
> > >> +               }
> > >
> > > I'm not sure this is a good idea to serialize on object_name_lock.
> > > Purgeable buffers should never be shared (imported or exported).  So
> > > at best you are avoiding evicting and immediately swapping back in, in
> > > a rare case, at the cost of serializing multiple threads trying to
> > > reclaim pages in parallel.
> >
> > The object_name_lock shouldn't cause contention in practice. But objects
> > are also pinned on attachment, hence maybe this lock is indeed
> > unnecessary.. I'll re-check it.
> 
> I'm not worried about contention with export/import/etc, but
> contention between multiple threads hitting the shrinker in parallel.
> I guess since you are using trylock, it won't *block* the other
> threads hitting shrinker, but they'll just end up looping in
> do_shrink_slab() because they are hitting contention.
> 
> I'd have to do some experiments to see how it works out in practice,
> but my gut feel is that it isn't a good idea

Yeah trylock on anything else than the object lock is No Good in the
shrinker. And it really shouldn't be needed, since import/export should
pin stuff as needed. Which should be protected by the dma_resv object
lock. If not, we need to fix that.

Picking a random drm-internal lock like this is definitely no good design.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Sun, Jun 19, 2022 at 10:53:03AM -0700, Rob Clark wrote:
> On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> > +       mutex_unlock(&gem_shrinker->lock);
> 
> As I mentioned on other thread, count_objects, being approximate but
> lockless and fast is the important thing.  Otherwise when you start
> hitting the shrinker on many threads, you end up serializing them all,
> even if you have no pages to return to the system at that point.

Yeah agreed, seems like I was wrong here :-) Atomic counter or something
would also be in link the the lru_list stuff.

It would be to record this in the kerneldoc for the shrinker structure
though, to make sure this is all understood.

> > +               /* prevent racing with the dma-buf importing/exporting */
> > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > +                       *lock_contention |= true;
> > +                       goto resv_unlock;
> > +               }
> 
> I'm not sure this is a good idea to serialize on object_name_lock.
> Purgeable buffers should never be shared (imported or exported).  So
> at best you are avoiding evicting and immediately swapping back in, in
> a rare case, at the cost of serializing multiple threads trying to
> reclaim pages in parallel.

Yeah this sounds really bad. Plus this is a per-device lock, and doing
those with trylock means the shrinker will fail to find shrinkable memory
way too often. We need to engineer this out somehow.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

()

On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }
> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)

oh, one other small note, both in scan_objects() and count_objects(),
you should check that get_nr_swap_pages()>0 before counting
evictable/willneed objects.  (And you probably want to keep separate
LRUs for dontneed vs willneed to accomplish that.)  At least for CrOS,
inside the VM there is no swap enabled (but instead we rely on zram
swap in the host.. plus vm-balloon to balance memory pressure between
host and guest)

BR,
-R


> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Robin Murphy]

On 2022-05-27 00:50, Dmitry Osipenko wrote:
> Hello,
> 
> This patchset introduces memory shrinker for the VirtIO-GPU DRM driver
> and adds memory purging and eviction support to VirtIO-GPU driver.
> 
> The new dma-buf locking convention is introduced here as well.
> 
> During OOM, the shrinker will release BOs that are marked as "not needed"
> by userspace using the new madvise IOCTL, it will also evict idling BOs
> to SWAP. The userspace in this case is the Mesa VirGL driver, it will mark
> the cached BOs as "not needed", allowing kernel driver to release memory
> of the cached shmem BOs on lowmem situations, preventing OOM kills.
> 
> The Panfrost driver is switched to use generic memory shrinker.

I think we still have some outstanding issues here - Alyssa reported 
some weirdness yesterday, so I just tried provoking a low-memory 
condition locally with this series applied and a few debug options 
enabled, and the results as below were... interesting.

Thanks,
Robin.

----->8-----
[   68.295951] ======================================================
[   68.295956] WARNING: possible circular locking dependency detected
[   68.295963] 5.19.0-rc3+ #400 Not tainted
[   68.295972] ------------------------------------------------------
[   68.295977] cc1/295 is trying to acquire lock:
[   68.295986] ffff000008d7f1a0 
(reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
[   68.296036]
[   68.296036] but task is already holding lock:
[   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at: 
__alloc_pages_slowpath.constprop.0+0x4d8/0x1470
[   68.296080]
[   68.296080] which lock already depends on the new lock.
[   68.296080]
[   68.296085]
[   68.296085] the existing dependency chain (in reverse order) is:
[   68.296090]
[   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
[   68.296111]        fs_reclaim_acquire+0xb8/0x150
[   68.296130]        dma_resv_lockdep+0x298/0x3fc
[   68.296148]        do_one_initcall+0xe4/0x5f8
[   68.296163]        kernel_init_freeable+0x414/0x49c
[   68.296180]        kernel_init+0x2c/0x148
[   68.296195]        ret_from_fork+0x10/0x20
[   68.296207]
[   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
[   68.296229]        __lock_acquire+0x1724/0x2398
[   68.296246]        lock_acquire+0x218/0x5b0
[   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
[   68.296277]        ww_mutex_lock+0x7c/0x4d8
[   68.296291]        drm_gem_shmem_free+0x7c/0x198
[   68.296304]        panfrost_gem_free_object+0x118/0x138
[   68.296318]        drm_gem_object_free+0x40/0x68
[   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
[   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
[   68.296368]        do_shrink_slab+0x220/0x808
[   68.296381]        shrink_slab+0x11c/0x408
[   68.296392]        shrink_node+0x6ac/0xb90
[   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
[   68.296416]        try_to_free_pages+0x1ec/0x5b0
[   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
[   68.296444]        __alloc_pages+0x4e0/0x5b8
[   68.296455]        __folio_alloc+0x24/0x60
[   68.296467]        vma_alloc_folio+0xb8/0x2f8
[   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
[   68.296498]        __handle_mm_fault+0x918/0x12a8
[   68.296513]        handle_mm_fault+0x130/0x300
[   68.296527]        do_page_fault+0x1d0/0x568
[   68.296539]        do_translation_fault+0xa0/0xb8
[   68.296551]        do_mem_abort+0x68/0xf8
[   68.296562]        el0_da+0x74/0x100
[   68.296572]        el0t_64_sync_handler+0x68/0xc0
[   68.296585]        el0t_64_sync+0x18c/0x190
[   68.296596]
[   68.296596] other info that might help us debug this:
[   68.296596]
[   68.296601]  Possible unsafe locking scenario:
[   68.296601]
[   68.296604]        CPU0                    CPU1
[   68.296608]        ----                    ----
[   68.296612]   lock(fs_reclaim);
[   68.296622] 
lock(reservation_ww_class_mutex);
[   68.296633]                                lock(fs_reclaim);
[   68.296644]   lock(reservation_ww_class_mutex);
[   68.296654]
[   68.296654]  *** DEADLOCK ***
[   68.296654]
[   68.296658] 3 locks held by cc1/295:
[   68.296666]  #0: ffff00000616e898 (&mm->mmap_lock){++++}-{3:3}, at: 
do_page_fault+0x144/0x568
[   68.296702]  #1: ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at: 
__alloc_pages_slowpath.constprop.0+0x4d8/0x1470
[   68.296740]  #2: ffff80000c1215b0 (shrinker_rwsem){++++}-{3:3}, at: 
shrink_slab+0xc0/0x408
[   68.296774]
[   68.296774] stack backtrace:
[   68.296780] CPU: 2 PID: 295 Comm: cc1 Not tainted 5.19.0-rc3+ #400
[   68.296794] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[   68.296803] Call trace:
[   68.296808]  dump_backtrace+0x1e4/0x1f0
[   68.296821]  show_stack+0x20/0x70
[   68.296832]  dump_stack_lvl+0x8c/0xb8
[   68.296849]  dump_stack+0x1c/0x38
[   68.296864]  print_circular_bug.isra.0+0x284/0x378
[   68.296881]  check_noncircular+0x1d8/0x1f8
[   68.296896]  __lock_acquire+0x1724/0x2398
[   68.296911]  lock_acquire+0x218/0x5b0
[   68.296926]  __ww_mutex_lock.constprop.0+0x158/0x2378
[   68.296942]  ww_mutex_lock+0x7c/0x4d8
[   68.296956]  drm_gem_shmem_free+0x7c/0x198
[   68.296970]  panfrost_gem_free_object+0x118/0x138
[   68.296984]  drm_gem_object_free+0x40/0x68
[   68.296999]  drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
[   68.297017]  drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
[   68.297033]  do_shrink_slab+0x220/0x808
[   68.297045]  shrink_slab+0x11c/0x408
[   68.297056]  shrink_node+0x6ac/0xb90
[   68.297068]  do_try_to_free_pages+0x1dc/0x8d0
[   68.297081]  try_to_free_pages+0x1ec/0x5b0
[   68.297094]  __alloc_pages_slowpath.constprop.0+0x528/0x1470
[   68.297110]  __alloc_pages+0x4e0/0x5b8
[   68.297122]  __folio_alloc+0x24/0x60
[   68.297134]  vma_alloc_folio+0xb8/0x2f8
[   68.297148]  alloc_zeroed_user_highpage_movable+0x58/0x68
[   68.297163]  __handle_mm_fault+0x918/0x12a8
[   68.297178]  handle_mm_fault+0x130/0x300
[   68.297193]  do_page_fault+0x1d0/0x568
[   68.297205]  do_translation_fault+0xa0/0xb8
[   68.297218]  do_mem_abort+0x68/0xf8
[   68.297229]  el0_da+0x74/0x100
[   68.297239]  el0t_64_sync_handler+0x68/0xc0
[   68.297252]  el0t_64_sync+0x18c/0x190
[   68.471812] arm-scmi firmware:scmi: timed out in resp(caller: 
scmi_power_state_set+0x11c/0x190)
[   68.501947] arm-scmi firmware:scmi: Message for 119 type 0 is not 
expected!
[   68.939686] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000915e2d34
[   69.739386] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ac77ac55
[   70.415329] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ee980c7e
[   70.987166] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ffb7ff37
[   71.914939] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000e92b26e
[   72.426987] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000c036a911
[   73.578683] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001c6fc094
[   74.090555] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000075d00f9
[   74.922709] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000005add546
[   75.434401] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000154189b
[   76.394300] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ac77ac55
[   76.906236] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ee980c7e
[   79.657234] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000f6d059fb
[   80.168831] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000061a0f6bf
[   80.808354] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000071ade02
[   81.319967] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000b0afea73
[   81.831574] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000d78f36c2
[   82.343160] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000f689397
[   83.046689] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000412c2a2f
[   83.558352] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000020e551b3
[   84.261913] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000009437aace
[   84.773576] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001c6fc094
[   85.317275] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000c036a911
[   85.829035] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000e92b26e
[   86.660555] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ac77ac55
[   87.172126] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000b940e406
[   87.875846] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001c6fc094
[   88.387443] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000009437aace
[   89.059175] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000075dadb7f
[   89.570960] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000005add546
[   90.146687] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000cba2873c
[   90.662497] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000a4beb490
[   95.392748] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000005b5fc4ec
[   95.904179] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000a17436ee
[   96.416085] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000003888d2a7
[   96.927874] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000093e04a98
[   97.439742] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000c036a911
[   97.954109] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000084c51113
[   98.467374] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000664663ce
[   98.975192] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000060f2d45c
[   99.487231] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000b29288f8
[   99.998833] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000002f07ab24
[  100.510744] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000008c15c751
[  100.511411] 
==================================================================
[  100.511419] BUG: KASAN: use-after-free in irq_work_single+0xa4/0x110
[  100.511445] Write of size 4 at addr ffff0000107f5830 by task 
glmark2-es2-drm/280
[  100.511458]
[  100.511464] CPU: 1 PID: 280 Comm: glmark2-es2-drm Not tainted 
5.19.0-rc3+ #400
[  100.511479] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[  100.511489] Call trace:
[  100.511494]  dump_backtrace+0x1e4/0x1f0
[  100.511512]  show_stack+0x20/0x70
[  100.511523]  dump_stack_lvl+0x8c/0xb8
[  100.511543]  print_report+0x16c/0x668
[  100.511559]  kasan_report+0x80/0x208
[  100.511574]  kasan_check_range+0x100/0x1b8
[  100.511590]  __kasan_check_write+0x34/0x60
[  100.511607]  irq_work_single+0xa4/0x110
[  100.511619]  irq_work_run_list+0x6c/0x88
[  100.511632]  irq_work_run+0x28/0x48
[  100.511644]  ipi_handler+0x254/0x468
[  100.511664]  handle_percpu_devid_irq+0x11c/0x518
[  100.511681]  generic_handle_domain_irq+0x50/0x70
[  100.511699]  gic_handle_irq+0xd4/0x118
[  100.511711]  call_on_irq_stack+0x2c/0x58
[  100.511725]  do_interrupt_handler+0xc0/0xc8
[  100.511741]  el1_interrupt+0x40/0x68
[  100.511754]  el1h_64_irq_handler+0x18/0x28
[  100.511767]  el1h_64_irq+0x64/0x68
[  100.511778]  irq_work_queue+0xc0/0xd8
[  100.511790]  drm_sched_entity_fini+0x2c4/0x3b0
[  100.511805]  drm_sched_entity_destroy+0x2c/0x40
[  100.511818]  panfrost_job_close+0x44/0x1c0
[  100.511833]  panfrost_postclose+0x38/0x60
[  100.511845]  drm_file_free.part.0+0x33c/0x4b8
[  100.511862]  drm_close_helper.isra.0+0xc0/0xd8
[  100.511877]  drm_release+0xe4/0x1e0
[  100.511891]  __fput+0xf8/0x390
[  100.511904]  ____fput+0x18/0x28
[  100.511917]  task_work_run+0xc4/0x1e0
[  100.511929]  do_exit+0x554/0x1168
[  100.511945]  do_group_exit+0x60/0x108
[  100.511960]  __arm64_sys_exit_group+0x34/0x38
[  100.511977]  invoke_syscall+0x64/0x180
[  100.511993]  el0_svc_common.constprop.0+0x13c/0x170
[  100.512012]  do_el0_svc+0x48/0xe8
[  100.512028]  el0_svc+0x5c/0xe0
[  100.512038]  el0t_64_sync_handler+0xb8/0xc0
[  100.512051]  el0t_64_sync+0x18c/0x190
[  100.512064]
[  100.512068] Allocated by task 280:
[  100.512075]  kasan_save_stack+0x2c/0x58
[  100.512091]  __kasan_kmalloc+0x90/0xb8
[  100.512105]  kmem_cache_alloc_trace+0x1d4/0x330
[  100.512118]  panfrost_ioctl_submit+0x100/0x630
[  100.512131]  drm_ioctl_kernel+0x160/0x250
[  100.512147]  drm_ioctl+0x36c/0x628
[  100.512161]  __arm64_sys_ioctl+0xd8/0x120
[  100.512178]  invoke_syscall+0x64/0x180
[  100.512194]  el0_svc_common.constprop.0+0x13c/0x170
[  100.512211]  do_el0_svc+0x48/0xe8
[  100.512226]  el0_svc+0x5c/0xe0
[  100.512236]  el0t_64_sync_handler+0xb8/0xc0
[  100.512248]  el0t_64_sync+0x18c/0x190
[  100.512259]
[  100.512262] Freed by task 280:
[  100.512268]  kasan_save_stack+0x2c/0x58
[  100.512283]  kasan_set_track+0x2c/0x40
[  100.512296]  kasan_set_free_info+0x28/0x50
[  100.512312]  __kasan_slab_free+0xf0/0x170
[  100.512326]  kfree+0x124/0x418
[  100.512337]  panfrost_job_cleanup+0x1f0/0x298
[  100.512350]  panfrost_job_free+0x80/0xb0
[  100.512363]  drm_sched_entity_kill_jobs_irq_work+0x80/0xa0
[  100.512377]  irq_work_single+0x88/0x110
[  100.512389]  irq_work_run_list+0x6c/0x88
[  100.512401]  irq_work_run+0x28/0x48
[  100.512413]  ipi_handler+0x254/0x468
[  100.512427]  handle_percpu_devid_irq+0x11c/0x518
[  100.512443]  generic_handle_domain_irq+0x50/0x70
[  100.512460]  gic_handle_irq+0xd4/0x118
[  100.512471]
[  100.512474] The buggy address belongs to the object at ffff0000107f5800
[  100.512474]  which belongs to the cache kmalloc-512 of size 512
[  100.512484] The buggy address is located 48 bytes inside of
[  100.512484]  512-byte region [ffff0000107f5800, ffff0000107f5a00)
[  100.512497]
[  100.512500] The buggy address belongs to the physical page:
[  100.512506] page:000000000a626feb refcount:1 mapcount:0 
mapping:0000000000000000 index:0x0 pfn:0x907f4
[  100.512520] head:000000000a626feb order:2 compound_mapcount:0 
compound_pincount:0
[  100.512530] flags: 
0xffff00000010200(slab|head|node=0|zone=0|lastcpupid=0xffff)
[  100.512556] raw: 0ffff00000010200 fffffc0000076400 dead000000000002 
ffff000000002600
[  100.512569] raw: 0000000000000000 0000000080100010 00000001ffffffff 
0000000000000000
[  100.512577] page dumped because: kasan: bad access detected
[  100.512582]
[  100.512585] Memory state around the buggy address:
[  100.512592]  ffff0000107f5700: fc fc fc fc fc fc fc fc fc fc fc fc fc 
fc fc fc
[  100.512602]  ffff0000107f5780: fc fc fc fc fc fc fc fc fc fc fc fc fc 
fc fc fc
[  100.512612] >ffff0000107f5800: fa fb fb fb fb fb fb fb fb fb fb fb fb 
fb fb fb
[  100.512619]                                      ^
[  100.512627]  ffff0000107f5880: fb fb fb fb fb fb fb fb fb fb fb fb fb 
fb fb fb
[  100.512636]  ffff0000107f5900: fb fb fb fb fb fb fb fb fb fb fb fb fb 
fb fb fb
[  100.512643] 
==================================================================
[  101.022573] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000be4b1b31
[  101.534469] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000a8ff2c8a
[  101.535981] BUG: sleeping function called from invalid context at 
kernel/locking/mutex.c:870
[  101.535994] in_atomic(): 1, irqs_disabled(): 128, non_block: 0, pid: 
280, name: glmark2-es2-drm
[  101.536006] preempt_count: 10000, expected: 0
[  101.536012] RCU nest depth: 0, expected: 0
[  101.536019] INFO: lockdep is turned off.
[  101.536023] irq event stamp: 1666508
[  101.536029] hardirqs last  enabled at (1666507): [<ffff80000997ed70>] 
exit_to_kernel_mode.isra.0+0x40/0x140
[  101.536056] hardirqs last disabled at (1666508): [<ffff800009985030>] 
__schedule+0xb38/0xea8
[  101.536076] softirqs last  enabled at (1664950): [<ffff800008010ac8>] 
__do_softirq+0x6b8/0x89c
[  101.536092] softirqs last disabled at (1664941): [<ffff8000080e4fdc>] 
irq_exit_rcu+0x27c/0x2b0
[  101.536118] CPU: 1 PID: 280 Comm: glmark2-es2-drm Tainted: G    B 
          5.19.0-rc3+ #400
[  101.536134] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[  101.536143] Call trace:
[  101.536147]  dump_backtrace+0x1e4/0x1f0
[  101.536161]  show_stack+0x20/0x70
[  101.536171]  dump_stack_lvl+0x8c/0xb8
[  101.536189]  dump_stack+0x1c/0x38
[  101.536204]  __might_resched+0x1f0/0x2b0
[  101.536220]  __might_sleep+0x74/0xd0
[  101.536234]  ww_mutex_lock+0x40/0x4d8
[  101.536249]  drm_gem_shmem_free+0x7c/0x198
[  101.536264]  panfrost_gem_free_object+0x118/0x138
[  101.536278]  drm_gem_object_free+0x40/0x68
[  101.536295]  panfrost_job_cleanup+0x1bc/0x298
[  101.536309]  panfrost_job_free+0x80/0xb0
[  101.536322]  drm_sched_entity_kill_jobs_irq_work+0x80/0xa0
[  101.536337]  irq_work_single+0x88/0x110
[  101.536351]  irq_work_run_list+0x6c/0x88
[  101.536364]  irq_work_run+0x28/0x48
[  101.536375]  ipi_handler+0x254/0x468
[  101.536392]  handle_percpu_devid_irq+0x11c/0x518
[  101.536409]  generic_handle_domain_irq+0x50/0x70
[  101.536428]  gic_handle_irq+0xd4/0x118
[  101.536439]  call_on_irq_stack+0x2c/0x58
[  101.536453]  do_interrupt_handler+0xc0/0xc8
[  101.536468]  el1_interrupt+0x40/0x68
[  101.536479]  el1h_64_irq_handler+0x18/0x28
[  101.536492]  el1h_64_irq+0x64/0x68
[  101.536503]  __asan_load8+0x30/0xd0
[  101.536519]  drm_sched_entity_fini+0x1e8/0x3b0
[  101.536532]  drm_sched_entity_destroy+0x2c/0x40
[  101.536545]  panfrost_job_close+0x44/0x1c0
[  101.536559]  panfrost_postclose+0x38/0x60
[  101.536571]  drm_file_free.part.0+0x33c/0x4b8
[  101.536586]  drm_close_helper.isra.0+0xc0/0xd8
[  101.536601]  drm_release+0xe4/0x1e0
[  101.536615]  __fput+0xf8/0x390
[  101.536628]  ____fput+0x18/0x28
[  101.536640]  task_work_run+0xc4/0x1e0
[  101.536652]  do_exit+0x554/0x1168
[  101.536667]  do_group_exit+0x60/0x108
[  101.536682]  __arm64_sys_exit_group+0x34/0x38
[  101.536698]  invoke_syscall+0x64/0x180
[  101.536714]  el0_svc_common.constprop.0+0x13c/0x170
[  101.536733]  do_el0_svc+0x48/0xe8
[  101.536748]  el0_svc+0x5c/0xe0
[  101.536759]  el0t_64_sync_handler+0xb8/0xc0
[  101.536771]  el0t_64_sync+0x18c/0x190
[  101.541928] ------------[ cut here ]------------
[  101.541934] kernel BUG at kernel/irq_work.c:235!
[  101.541944] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
[  101.541961] Modules linked in:
[  101.541978] CPU: 1 PID: 280 Comm: glmark2-es2-drm Tainted: G    B   W 
         5.19.0-rc3+ #400
[  101.541997] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[  101.542009] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS 
BTYPE=--)
[  101.542027] pc : irq_work_run_list+0x80/0x88
[  101.542044] lr : irq_work_run+0x34/0x48
[  101.542060] sp : ffff80000da37eb0
[  101.542069] x29: ffff80000da37eb0 x28: ffff000006bb0000 x27: 
ffff000006bb0008
[  101.542107] x26: ffff80000da37f20 x25: ffff8000080304d8 x24: 
0000000000000001
[  101.542142] x23: ffff80000abcd008 x22: ffff80000da37ed0 x21: 
ffff80001c0de000
[  101.542177] x20: ffff80000abcd008 x19: ffff80000abdbad0 x18: 
0000000000000000
[  101.542212] x17: 616e202c30383220 x16: 3a646970202c3020 x15: 
ffff8000082df9d0
[  101.542246] x14: ffff800008dfada8 x13: 0000000000000003 x12: 
1fffe000018b2a06
[  101.542280] x11: ffff6000018b2a06 x10: dfff800000000000 x9 : 
ffff00000c595033
[  101.542315] x8 : ffff6000018b2a07 x7 : 0000000000000001 x6 : 
00000000000000fb
[  101.542349] x5 : ffff00000c595030 x4 : 0000000000000000 x3 : 
ffff00000c595030
[  101.542382] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 
ffff000026cb9ad0
[  101.542416] Call trace:
[  101.542424]  irq_work_run_list+0x80/0x88
[  101.542441]  ipi_handler+0x254/0x468
[  101.542460]  handle_percpu_devid_irq+0x11c/0x518
[  101.542480]  generic_handle_domain_irq+0x50/0x70
[  101.542501]  gic_handle_irq+0xd4/0x118
[  101.542516]  call_on_irq_stack+0x2c/0x58
[  101.542534]  do_interrupt_handler+0xc0/0xc8
[  101.542553]  el1_interrupt+0x40/0x68
[  101.542568]  el1h_64_irq_handler+0x18/0x28
[  101.542584]  el1h_64_irq+0x64/0x68
[  101.542599]  __asan_load8+0x30/0xd0
[  101.542617]  drm_sched_entity_fini+0x1e8/0x3b0
[  101.542634]  drm_sched_entity_destroy+0x2c/0x40
[  101.542651]  panfrost_job_close+0x44/0x1c0
[  101.542669]  panfrost_postclose+0x38/0x60
[  101.542685]  drm_file_free.part.0+0x33c/0x4b8
[  101.542704]  drm_close_helper.isra.0+0xc0/0xd8
[  101.542723]  drm_release+0xe4/0x1e0
[  101.542740]  __fput+0xf8/0x390
[  101.542756]  ____fput+0x18/0x28
[  101.542773]  task_work_run+0xc4/0x1e0
[  101.542788]  do_exit+0x554/0x1168
[  101.542806]  do_group_exit+0x60/0x108
[  101.542825]  __arm64_sys_exit_group+0x34/0x38
[  101.542845]  invoke_syscall+0x64/0x180
[  101.542865]  el0_svc_common.constprop.0+0x13c/0x170
[  101.542887]  do_el0_svc+0x48/0xe8
[  101.542906]  el0_svc+0x5c/0xe0
[  101.542921]  el0t_64_sync_handler+0xb8/0xc0
[  101.542938]  el0t_64_sync+0x18c/0x190
[  101.542960] Code: a94153f3 a8c27bfd d50323bf d65f03c0 (d4210000)
[  101.542979] ---[ end trace 0000000000000000 ]---
[  101.678650] Kernel panic - not syncing: Oops - BUG: Fatal exception 
in interrupt
[  102.046301] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001da14c98
[  103.227334] SMP: stopping secondary CPUs
[  103.241055] Kernel Offset: disabled
[  103.254316] CPU features: 0x800,00184810,00001086
[  103.268904] Memory Limit: 800 MB
[  103.411625] ---[ end Kernel panic - not syncing: Oops - BUG: Fatal 
exception in interrupt ]---
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Rob Clark]

On Tue, Jun 28, 2022 at 5:51 AM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> On 6/28/22 15:31, Robin Murphy wrote:
> > ----->8-----
> > [   68.295951] ======================================================
> > [   68.295956] WARNING: possible circular locking dependency detected
> > [   68.295963] 5.19.0-rc3+ #400 Not tainted
> > [   68.295972] ------------------------------------------------------
> > [   68.295977] cc1/295 is trying to acquire lock:
> > [   68.295986] ffff000008d7f1a0
> > (reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
> > [   68.296036]
> > [   68.296036] but task is already holding lock:
> > [   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at:
> > __alloc_pages_slowpath.constprop.0+0x4d8/0x1470
> > [   68.296080]
> > [   68.296080] which lock already depends on the new lock.
> > [   68.296080]
> > [   68.296085]
> > [   68.296085] the existing dependency chain (in reverse order) is:
> > [   68.296090]
> > [   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
> > [   68.296111]        fs_reclaim_acquire+0xb8/0x150
> > [   68.296130]        dma_resv_lockdep+0x298/0x3fc
> > [   68.296148]        do_one_initcall+0xe4/0x5f8
> > [   68.296163]        kernel_init_freeable+0x414/0x49c
> > [   68.296180]        kernel_init+0x2c/0x148
> > [   68.296195]        ret_from_fork+0x10/0x20
> > [   68.296207]
> > [   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
> > [   68.296229]        __lock_acquire+0x1724/0x2398
> > [   68.296246]        lock_acquire+0x218/0x5b0
> > [   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
> > [   68.296277]        ww_mutex_lock+0x7c/0x4d8
> > [   68.296291]        drm_gem_shmem_free+0x7c/0x198
> > [   68.296304]        panfrost_gem_free_object+0x118/0x138
> > [   68.296318]        drm_gem_object_free+0x40/0x68
> > [   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
> > [   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
> > [   68.296368]        do_shrink_slab+0x220/0x808
> > [   68.296381]        shrink_slab+0x11c/0x408
> > [   68.296392]        shrink_node+0x6ac/0xb90
> > [   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
> > [   68.296416]        try_to_free_pages+0x1ec/0x5b0
> > [   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
> > [   68.296444]        __alloc_pages+0x4e0/0x5b8
> > [   68.296455]        __folio_alloc+0x24/0x60
> > [   68.296467]        vma_alloc_folio+0xb8/0x2f8
> > [   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
> > [   68.296498]        __handle_mm_fault+0x918/0x12a8
> > [   68.296513]        handle_mm_fault+0x130/0x300
> > [   68.296527]        do_page_fault+0x1d0/0x568
> > [   68.296539]        do_translation_fault+0xa0/0xb8
> > [   68.296551]        do_mem_abort+0x68/0xf8
> > [   68.296562]        el0_da+0x74/0x100
> > [   68.296572]        el0t_64_sync_handler+0x68/0xc0
> > [   68.296585]        el0t_64_sync+0x18c/0x190
> > [   68.296596]
> > [   68.296596] other info that might help us debug this:
> > [   68.296596]
> > [   68.296601]  Possible unsafe locking scenario:
> > [   68.296601]
> > [   68.296604]        CPU0                    CPU1
> > [   68.296608]        ----                    ----
> > [   68.296612]   lock(fs_reclaim);
> > [   68.296622] lock(reservation_ww_class_mutex);
> > [   68.296633]                                lock(fs_reclaim);
> > [   68.296644]   lock(reservation_ww_class_mutex);
> > [   68.296654]
> > [   68.296654]  *** DEADLOCK ***
>
> This splat could be ignored for now. I'm aware about it, although
> haven't looked closely at how to fix it since it's a kind of a lockdep
> misreporting.

The lockdep splat could be fixed with something similar to what I've
done in msm, ie. basically just not acquire the lock in the finalizer:

https://patchwork.freedesktop.org/patch/489364/

There is one gotcha to watch for, as danvet pointed out
(scan_objects() could still see the obj in the LRU before the
finalizer removes it), but if scan_objects() does the
kref_get_unless_zero() trick, it is safe.

BR,
-R
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [Linaro-mm-sig] Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Christian König via Virtualization]

Am 30.06.22 um 01:06 schrieb Dmitry Osipenko:
> On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
>> On 6/29/22 10:22, Dmitry Osipenko wrote:
>>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>>> handle imported dma-bufs properly, which results in mapping of
>>>>> something
>>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>>> hard
>>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>>> was
>>>>> imported into Tegra's DRM GEM.
>>>>>
>>>>> To fix this bug, move mapping of imported dma-bufs to
>>>>> drm_gem_mmap_obj().
>>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>>> Same comment about Fixes: as in patch 1,
>>>>> Cc: stable@vger.kernel.org
>>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>>> ---
>>>>>     drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>>     drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>>     drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>>     3 files changed, 7 insertions(+), 9 deletions(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>>> index 86d670c71286..7c0b025508e4 100644
>>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>>>> unsigned long obj_size,
>>>>>         if (obj_size < vma->vm_end - vma->vm_start)
>>>>>             return -EINVAL;
>>>>>     +    if (obj->import_attach)
>>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>>> drivers in this way, how do we ensure that user-space is not blindly
>>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>>
>>>> I was under the impression (admittedly without looking) that the few
>>>> drivers that actually called into dma_buf_mmap() had some private
>>>> user-mode driver code in place that ensured this happened.
>>> Since it's a userspace who does the mapping, then it should be a
>>> responsibility of userspace to do all the necessary syncing.
>> Sure, but nothing prohibits user-space to ignore the syncing thinking
>> "It works anyway", testing those drivers where the syncing is a NOP. And
>> when a driver that finally needs syncing is tested it's too late to fix
>> all broken user-space.
>>
>>>    I'm not
>>> sure whether anyone in userspace really needs to map imported dma-bufs
>>> in practice. Nevertheless, this use-case is broken and should be fixed
>>> by either allowing to do the mapping or prohibiting it.
>>>
>> Then I'd vote for prohibiting it, at least for now. And for the future
>> moving forward we could perhaps revisit the dma-buf need for syncing,
>> requiring those drivers that actually need it to implement emulated
>> coherent memory which can be done not too inefficiently (vmwgfx being
>> one example).
> Alright, I'll change it to prohibit the mapping. This indeed should be a
> better option.

Oh, yes please. But I would expect that some people start screaming.

Over time I've got tons of TTM patches because people illegally tried to 
mmap() imported DMA-bufs in their driver.

Anyway this is probably the right thing to do and we can work on fixing 
the fallout later on.

Regards,
Christian.


_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization