Re: [RFC PATCH 02/18] drm/ttm: Add per-BO eviction tracking

["Christian König" <ckoenig.leichtzumerken@gmail.com>]

Am 25.04.24 um 21:02 schrieb Matthew Brost:
> On Thu, Apr 25, 2024 at 08:18:38AM +0200, Christian König wrote:
>> Am 24.04.24 um 18:56 schrieb Friedrich Vock:
>>> Make each buffer object aware of whether it has been evicted or not.
>> That reverts some changes we made a couple of years ago.
>>
>> In general the idea is that eviction isn't something we need to reverse in
>> TTM.
>>
>> Rather the driver gives the desired placement.
>>
>> Regards,
>> Christian.
>>
> We have added a concept similar to this in drm_gpuvm [1]. GPUVM
> maintains a list of evicted BOs and when the GPUVM is locked for
> submission it has validate vfunc which is called on each BO. If driver
> is using TTM, this is where the driver would call TTM BO validate which
> unevicts the BO. Well at least this what we do it Xe [2].
>
> The uneviction is a per VM operation not a global one. With this, a
> global eviction list does not seem correct (admittedly not through the
> entire series).

Yeah, that's exactly what I meant when I wrote that this is controlled 
by the "driver" :)

The state machine in AMDGPUs VM code is pretty much the same.

Regards,
Christian.

>
> Matt
>
> [1] https://elixir.bootlin.com/linux/v6.8.7/source/drivers/gpu/drm/drm_gpuvm.c#L86
> [2] https://elixir.bootlin.com/linux/v6.8.7/source/drivers/gpu/drm/xe/xe_vm.c#L464
>
>>> Signed-off-by: Friedrich Vock <friedrich.vock@gmx.de>
>>> ---
>>>    drivers/gpu/drm/ttm/ttm_bo.c |  1 +
>>>    include/drm/ttm/ttm_bo.h     | 11 +++++++++++
>>>    2 files changed, 12 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c
>>> index edf10618fe2b2..3968b17453569 100644
>>> --- a/drivers/gpu/drm/ttm/ttm_bo.c
>>> +++ b/drivers/gpu/drm/ttm/ttm_bo.c
>>> @@ -980,6 +980,7 @@ int ttm_bo_init_reserved(struct ttm_device *bdev, struct ttm_buffer_object *bo,
>>>    	bo->pin_count = 0;
>>>    	bo->sg = sg;
>>>    	bo->bulk_move = NULL;
>>> +	bo->evicted_type = TTM_NUM_MEM_TYPES;
>>>    	if (resv)
>>>    		bo->base.resv = resv;
>>>    	else
>>> diff --git a/include/drm/ttm/ttm_bo.h b/include/drm/ttm/ttm_bo.h
>>> index 0223a41a64b24..8a1a29c6fbc50 100644
>>> --- a/include/drm/ttm/ttm_bo.h
>>> +++ b/include/drm/ttm/ttm_bo.h
>>> @@ -121,6 +121,17 @@ struct ttm_buffer_object {
>>>    	unsigned priority;
>>>    	unsigned pin_count;
>>>
>>> +	/**
>>> +	 * @evicted_type: Memory type this BO was evicted from, if any.
>>> +	 * TTM_NUM_MEM_TYPES if this BO was not evicted.
>>> +	 */
>>> +	int evicted_type;
>>> +	/**
>>> +	 * @evicted: Entry in the evicted list for the resource manager
>>> +	 * this BO was evicted from.
>>> +	 */
>>> +	struct list_head evicted;
>>> +
>>>    	/**
>>>    	 * @delayed_delete: Work item used when we can't delete the BO
>>>    	 * immediately
>>> --
>>> 2.44.0
>>>