Re: [PATCH v2 01/15] drm/shmem-helper: Make pages_use_count an atomic_t

[Boris Brezillon <boris.brezillon@collabora.com>]

On Sat, 19 Aug 2023 05:13:06 +0300
Dmitry Osipenko <dmitry.osipenko@collabora.com> wrote:

> On 8/11/23 16:08, Steven Price wrote:
> > On 09/08/2023 17:53, Boris Brezillon wrote:  
> >> This way we can grab a pages ref without acquiring the resv lock when
> >> pages_use_count > 0. Need to implement asynchronous map using the  
> > 
> > NIT: s/Need/This is needed/
> >   
> >> drm_gpuva_mgr when the map/unmap operation triggers a mapping split,
> >> requiring the new left/right regions to grab an additional page ref
> >> to guarantee that the pages stay pinned when the middle section is
> >> unmapped.
> >>
> >> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> >> ---
> >>  drivers/gpu/drm/drm_gem_shmem_helper.c  | 28 +++++++++++++------------
> >>  drivers/gpu/drm/lima/lima_gem.c         |  2 +-
> >>  drivers/gpu/drm/panfrost/panfrost_mmu.c |  2 +-
> >>  include/drm/drm_gem_shmem_helper.h      |  2 +-
> >>  4 files changed, 18 insertions(+), 16 deletions(-)
> >>
> >> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> >> index a783d2245599..ca6938ea1b82 100644
> >> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> >> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> >> @@ -155,7 +155,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >>  		if (shmem->pages)
> >>  			drm_gem_shmem_put_pages(shmem);
> >>  
> >> -		drm_WARN_ON(obj->dev, shmem->pages_use_count);
> >> +		drm_WARN_ON(obj->dev, atomic_read(&shmem->pages_use_count));
> >>  
> >>  		dma_resv_unlock(shmem->base.resv);
> >>  	}
> >> @@ -172,14 +172,14 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> >>  
> >>  	dma_resv_assert_held(shmem->base.resv);
> >>  
> >> -	if (shmem->pages_use_count++ > 0)
> >> +	if (atomic_inc_return(&shmem->pages_use_count) > 1)
> >>  		return 0;
> >>  
> >>  	pages = drm_gem_get_pages(obj);
> >>  	if (IS_ERR(pages)) {
> >>  		drm_dbg_kms(obj->dev, "Failed to get pages (%ld)\n",
> >>  			    PTR_ERR(pages));
> >> -		shmem->pages_use_count = 0;
> >> +		atomic_set(&shmem->pages_use_count, 0);
> >>  		return PTR_ERR(pages);
> >>  	}
> >>  
> >> @@ -210,10 +210,10 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> >>  
> >>  	dma_resv_assert_held(shmem->base.resv);
> >>  
> >> -	if (drm_WARN_ON_ONCE(obj->dev, !shmem->pages_use_count))
> >> +	if (drm_WARN_ON_ONCE(obj->dev, !atomic_read(&shmem->pages_use_count)))
> >>  		return;
> >>  
> >> -	if (--shmem->pages_use_count > 0)
> >> +	if (atomic_dec_return(&shmem->pages_use_count) > 0)
> >>  		return;
> >>  
> >>  #ifdef CONFIG_X86
> >> @@ -263,6 +263,10 @@ int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
> >>  
> >>  	drm_WARN_ON(obj->dev, obj->import_attach);
> >>  
> >> +	/* If we are the first owner, we need to grab the lock. */
> >> +	if (atomic_inc_not_zero(&shmem->pages_use_count))
> >> +		return 0;
> >> +  
> > 
> > Unless I'm misunderstanding I think this introduces a race where two
> > threads call drm_gem_shmem_pin() at the same time:
> > 
> > Thread1				| Thread 2
> > --------------------------------+------------------------------
> > drm_gem_shmem_pin()		|
> >  - pages_use_count == 0 so not  |
> >    incremented                  |
> >  - lock taken			|
> > drm_gem_shmem_pin_locked()	|
> > drm_gem_shmem_get_pages()	|
> >  - pages_use_count incremented	|
> > <thread descheduled>            | drm_gem_shmem_pin()
> >                                 |  - pages_use_count == 1 so is it
> > 				|    incremented and returns early
> > 				|    without taking the lock
> > 				| Code tries to use shmem->pages
> > <thread rescheduled>		| and blows up
> > drm_gem_get_pages()		|
> > shmem->pages populated		|
> > lock released			|
> > 
> > I think you need to modify drm_gem_shmem_get_pages() to only increment
> > pages_use_count when shmem->pages has been populated.

Oops, didn't spot that race. Thanks for pointing it out.

> 
> This is correct, both pin() and get_pages() should use
> atomic_inc_not_zero().
> 
> Note that we shouldn't use atomic functions open-coded, there is kref
> helper for that which uses refcount_t underneath and has additional
> checks/warnings for count underflow/overflow. I'm going to post patches
> converting drm-shmem to kref around next week, Boris is aware about it
> and we should then sync shrinker/panthor patchsets to the common
> drm-shmem base.

Thanks, I'll have a look at these patches pretty soon.