Re: locking user space memory in kernel

[Roland Dreier <roland@topspin.com>]

    Manfred> I think just get_user_pages() should be sufficient: the
    Manfred> pages won't be swapped out. You don't need to set
    Manfred> VM_LOCKED in vma->vm_flags to prevent the swap out. In
    Manfred> the worst case, the pte is cleared a that will cause a
    Manfred> soft page fault, but the physical address won't
    Manfred> change. Multiple get_user_pages() calls on overlapping
    Manfred> regions are ok, the page count is an atomic_t, at least
    Manfred> 24-bit large.

    Roland> There is one case that we ran into where the physical
    Roland> address can change: if a process does a fork() and then
    Roland> triggers COW.

    Manfred> You are right.  What should happen if there are
    Manfred> registered transfers during fork()?  Copy the pages
    Manfred> during the fork() syscall?

The current Mellanox InfiniBand driver goes to some trouble to mark
the memory being registered with VM_DONTCOPY.  This means the vmas
don't get copied into the child of a fork(), so the COW doesn't
happen.  However, this certainly leads to some quirks in semantics.
In particular, an application using fork() has to be careful that
registered memory doesn't share a page with something the child
process wants to use.

I don't think copying all the registered memory on fork() is feasible,
because it's going to kill performance (especially since exec() is
likely to immediately follow the fork() in the child).  Also, there
may not be enough memory around to copy everything.

Out of curiousity, what happens if I fork with pending AIO in the
current kernel?

 - Roland