Re: TTM page pool allocator

[Jerome Glisse <glisse@freedesktop.org>]

On Wed, 2009-07-22 at 10:27 +0200, Thomas Hellström wrote:
> Jerome Glisse wrote:
> > On Thu, 2009-06-25 at 17:53 +0200, Thomas Hellström wrote:
> >   
> >> Jerome Glisse skrev:
> >>     
> >>> Hi,
> >>>
> >>> Thomas i attach a reworked page pool allocator based on Dave works,
> >>> this one should be ok with ttm cache status tracking. It definitely
> >>> helps on AGP system, now the bottleneck is in mesa vertex's dma
> >>> allocation.
> >>>
> >>> Cheers,
> >>> Jerome
> >>>   
> >>>       
> >> Hi, Jerome!
> >> In general it looks very good. Some things that need fixing:
> >>
> >> 1)  We must have a way to hand back pages. I still not quite understand 
> >> how the shrink callbacks work and whether they are applicable. Another 
> >> scheme would be to free, say 1MB when we have at least 2MB available.
> >>
> >> 2) We should avoid including AGP headers if AGP is not configured. 
> >> Either reimplement unmap_page_from_agp or map_page_into_agp or move them 
> >> out from the AGP headers. We've hade complaints before from people with 
> >> AGP free systems that the code doesn't compile.
> >>
> >> 3) Since we're allocating (and freeing) in batches we should use the 
> >> set_pages_array() interface to avoid a global tlb flush per page.
> >>
> >> 4) We could now skip the ttm_tt_populate() in ttm_tt_set_caching, since 
> >> it will always allocate cached pages and then transition them.
> >>
> >>     
> >
> > Okay 4) is bad, what happens (my brain is a bit meltdown so i might be
> > wrong) :
> > 1 - bo get allocated tt->state = unpopulated
> > 2 - bo is mapped few page are faulted tt->state = unpopulated
> > 3 - bo is cache transitioned but tt->state == unpopulated but
> >     they are page which have been touch by the cpu so we need
> >     to clflush them and transition them, 
> 
> Right.
> 
> > this never happen if
> >     we don't call ttm_tt_populate and proceed with the remaining
> >     of the cache transitioning functions
> >   
> 
> Why can't we just skip ttm_tt_populate and proceed with the remaining of 
> the cache transitioning functions? Then all pages currently allocated to 
> the TTM will be transitioned.
> 

Because not all tt->pages entry have valid page some are null and thus
we have to go through the whole table and transition page per page. I
did that and quick benchmark showed that it's faster to fully populate
and fully transition.

Cheers,
Jerome