Questions about TTM buffer object maping

Questions about TTM buffer object maping

[Jean-Sébastien Pédron]

Hello,

I'm trying to understand how TTM buffer object mapping works on Linux, 
to make this behave properly on FreeBSD.

Here's what I think I understand:

When a buffer object is mmap()'d, ttm_bo_vm_open() is called. When 
there's a page fault, the page is looked up and inserted in the VMA 
using vm_insert_mixed(). When a buffer object is munmap()'d, 
ttm_bo_vm_close() is called, which drops a reference. When the last 
reference is dropped, the buffer object is destroyed.

What's still not clear to me is how munmap() works here. After talking 
about this on IRC with some people, I think that unmap_mapping_range() 
(called by ttm_bo_unmap_virtual_locked()) is equivalent to calling 
munmap() from userland. Is that true?

When a buffer object is moved, what happens to the mapping?

In particular, I see in ttm_bo_move_accel_cleanup() that the ttm 
structure can be transferred to ghost_obj, which is destroyed shortly 
after. This ends up in ttm_put_pages() which uses __free_page(), for 
each page of the buffer object. At this stage, is the ghost object 
already munmap()'d? Or does __free_page() unmap a page implicitly (ie. 
remove it from VMA)?

Sorry if my questions are stupid, I'm rather new to memory management.

-- 
Jean-Sébastien Pédron

Re: Questions about TTM buffer object maping

[Jerome Glisse]

On Wed, Jul 10, 2013 at 8:27 PM, Jean-Sébastien Pédron
<jean-sebastien.pedron@dumbbell.fr> wrote:
> Hello,
>
> I'm trying to understand how TTM buffer object mapping works on Linux, to
> make this behave properly on FreeBSD.
>
> Here's what I think I understand:
>
> When a buffer object is mmap()'d, ttm_bo_vm_open() is called. When there's a
> page fault, the page is looked up and inserted in the VMA using
> vm_insert_mixed(). When a buffer object is munmap()'d, ttm_bo_vm_close() is
> called, which drops a reference. When the last reference is dropped, the
> buffer object is destroyed.
>
> What's still not clear to me is how munmap() works here. After talking about
> this on IRC with some people, I think that unmap_mapping_range() (called by
> ttm_bo_unmap_virtual_locked()) is equivalent to calling munmap() from
> userland. Is that true?

Yes that's true.

> When a buffer object is moved, what happens to the mapping?

unmap_mapping_range is call from ttm_bo_move (indirectly through the
helper function).

>
> In particular, I see in ttm_bo_move_accel_cleanup() that the ttm structure
> can be transferred to ghost_obj, which is destroyed shortly after. This ends
> up in ttm_put_pages() which uses __free_page(), for each page of the buffer
> object. At this stage, is the ghost object already munmap()'d? Or does
> __free_page() unmap a page implicitly (ie. remove it from VMA)?

Yes object is unmapped prior to move.

Cheers,
Jerome

>
> Sorry if my questions are stupid, I'm rather new to memory management.
>
> --
> Jean-Sébastien Pédron
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: Questions about TTM buffer object maping

[Daniel Vetter]

On Wed, Jul 10, 2013 at 09:00:33PM -0400, Jerome Glisse wrote:
> On Wed, Jul 10, 2013 at 8:27 PM, Jean-Sébastien Pédron
> <jean-sebastien.pedron@dumbbell.fr> wrote:
> > Hello,
> >
> > I'm trying to understand how TTM buffer object mapping works on Linux, to
> > make this behave properly on FreeBSD.
> >
> > Here's what I think I understand:
> >
> > When a buffer object is mmap()'d, ttm_bo_vm_open() is called. When there's a
> > page fault, the page is looked up and inserted in the VMA using
> > vm_insert_mixed(). When a buffer object is munmap()'d, ttm_bo_vm_close() is
> > called, which drops a reference. When the last reference is dropped, the
> > buffer object is destroyed.
> >
> > What's still not clear to me is how munmap() works here. After talking about
> > this on IRC with some people, I think that unmap_mapping_range() (called by
> > ttm_bo_unmap_virtual_locked()) is equivalent to calling munmap() from
> > userland. Is that true?
> 
> Yes that's true.

Afaik unmap_mapping_range only kills the ptes and doesn't remove the vma.
So not equivalent to a munmap from userspace. It simply allows us to
intercept the next access in the page fault handler and move the buffer
back into place.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: Questions about TTM buffer object maping

[Jerome Glisse]

On Thu, Jul 11, 2013 at 2:24 AM, Daniel Vetter <daniel@ffwll.ch> wrote:
> On Wed, Jul 10, 2013 at 09:00:33PM -0400, Jerome Glisse wrote:
>> On Wed, Jul 10, 2013 at 8:27 PM, Jean-Sébastien Pédron
>> <jean-sebastien.pedron@dumbbell.fr> wrote:
>> > Hello,
>> >
>> > I'm trying to understand how TTM buffer object mapping works on Linux, to
>> > make this behave properly on FreeBSD.
>> >
>> > Here's what I think I understand:
>> >
>> > When a buffer object is mmap()'d, ttm_bo_vm_open() is called. When there's a
>> > page fault, the page is looked up and inserted in the VMA using
>> > vm_insert_mixed(). When a buffer object is munmap()'d, ttm_bo_vm_close() is
>> > called, which drops a reference. When the last reference is dropped, the
>> > buffer object is destroyed.
>> >
>> > What's still not clear to me is how munmap() works here. After talking about
>> > this on IRC with some people, I think that unmap_mapping_range() (called by
>> > ttm_bo_unmap_virtual_locked()) is equivalent to calling munmap() from
>> > userland. Is that true?
>>
>> Yes that's true.
>
> Afaik unmap_mapping_range only kills the ptes and doesn't remove the vma.
> So not equivalent to a munmap from userspace. It simply allows us to
> intercept the next access in the page fault handler and move the buffer
> back into place.
> -Daniel

Yes, i was talking from a page point of view, ie page no longer have
mapping and can
be free.

Cheers,
Jerome

Re: Questions about TTM buffer object maping

[Jean-Sébastien Pédron]

Hi,

Thank you Jérôme and Daniel for your input, that's really helpful!

I have another question: in ttm_bo_mmap(), a reference to the buffer 
object is acquired at the beginning of the function. Another reference 
is acquired in ttm_bo_vm_open() (released in ttm_bo_vm_close()).

But where is the first reference released?

-- 
Jean-Sébastien Pédron

Re: Questions about TTM buffer object maping

[Jerome Glisse]

On Thu, Jul 11, 2013 at 5:43 PM, Jean-Sébastien Pédron
<jean-sebastien.pedron@dumbbell.fr> wrote:
> Hi,
>
> Thank you Jérôme and Daniel for your input, that's really helpful!
>
> I have another question: in ttm_bo_mmap(), a reference to the buffer object
> is acquired at the beginning of the function. Another reference is acquired
> in ttm_bo_vm_open() (released in ttm_bo_vm_close()).
>
> But where is the first reference released?
>
> --
> Jean-Sébastien Pédron

the ttm_bo_vm_open is not call on first time a vma is mmap, ie when
userspace do mmap it call driver
mmap callback which call ttm_bo_mmap but ttm_bo_vm_open is never call.
If the same process or
another process mmap the same area or subarea the ttm_bo_vm_open is
call. Then on each unmap
ttm_bo_vm_close is call.

Cheers,
Jerome

Re: Questions about TTM buffer object maping

[David Herrmann]

Hi

On Thu, Jul 11, 2013 at 11:43 PM, Jean-Sébastien Pédron
<jean-sebastien.pedron@dumbbell.fr> wrote:
> Hi,
>
> Thank you Jérôme and Daniel for your input, that's really helpful!
>
> I have another question: in ttm_bo_mmap(), a reference to the buffer object
> is acquired at the beginning of the function. Another reference is acquired
> in ttm_bo_vm_open() (released in ttm_bo_vm_close()).
>
> But where is the first reference released?

->vm_open() isn't called for the first mmap(), afaik (only called
during fork()s or similar). So the reference in ttm_bo_mmap() is a
replacement for the reference you take in the ->vm_open() callback.

Cheers
David

> --
> Jean-Sébastien Pédron
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: Questions about TTM buffer object maping

[Jean-Sébastien Pédron]

Le 11/07/2013 23:51, David Herrmann a écrit :
> ->vm_open() isn't called for the first mmap(), afaik (only called
> during fork()s or similar). So the reference in ttm_bo_mmap() is a
> replacement for the reference you take in the ->vm_open() callback.

So the reference is acquired either in ttm_bo_mmap() or in 
ttm_bo_vm_open(), and always released in ttm_bo_vm_close().

Thanks to both of you!

-- 
Jean-Sébastien Pédron