Re: [PATCH] drm/ttm: add minimum residency constraint for bo eviction

[Thomas Hellstrom <thomas@shipmail.org>]

On 11/29/2012 12:24 AM, Jerome Glisse wrote:
> On Wed, Nov 28, 2012 at 6:18 PM, Thomas Hellstrom <thomas@shipmail.org> wrote:
>> On 11/28/2012 04:58 PM, j.glisse@gmail.com wrote:
>>> From: Jerome Glisse <jglisse@redhat.com>
>>>
>>> This patch add a minimum residency time configurable for each memory
>>> pool (VRAM, GTT, ...). Intention is to avoid having a lot of memory
>>> eviction from VRAM up to a point where the GPU pretty much spend all
>>> it's time moving things in and out.
>>
>> This patch seems odd to me.
>>
>> It seems the net effect is to refuse evictions from VRAM and make buffers go
>> somewhere else, and that makes things faster?
>>
>> Why don't they go there in the first place instead of trying to force them
>> into VRAM,
>> when VRAM is full?
>>
>> /Thomas
> It's mostly a side effect of cs and validating with each cs, if boA is
> in cs1 and not in cs2 and boB is in cs1 but not in cs2 than boA could
> be evicted by cs2 and boB moved in, if next cs ie cs3 is like cs1 then
> boA move back again and boB is evicted, then you get cs4 which
> reference boB but not boA, boA get evicted and boB move in ... So ttm
> just spend its time doing eviction but he doing so because it's ask by
> the driver to do so. Note that what is costly there is not the bo move
> in itself but the page allocation.

Yes, this is the cause of the trashing, but that was not what I asked.

What your patch is doing is looking at the last recently used bo, to 
check if it has been
resident for at least 500ms. Otherwise it refuses eviction for *all* 
buffers of that memory type.

This means new buffers can't fit in VRAM, they need to go somewhere 
else. Perhaps TT?

So my question was. If VRAM is full, instead of starting to evict, why 
not put new buffers in TT, so that

placement(GEM_DOMAIN_VRAM) = VRAM | TT  // Prefer VRAM but allow TT 
before starting to evict.
busy_placement(GEM_DOMAIN_VRAM) = TT | VRAM // *If* we need to evict, 
prefer evicting TT, then evict VRAM)

This will more or less mimic carmack's algorithm by using TT as his "MRU 
scratch space".

And as a side note, your patch breaks
ttm_bo_force_list_clean()
which should be used at GPU memory exhaustion to avoid OOM due to 
fragmentation and for those drivers that
implement VRAM cleanup on VT switch and / or suspend / hibernation.

/Thomas