Re: [openib-general] Re: [PATCH][RFC][0/4] InfiniBand userspace verbs implementation

[Timur Tabi <timur.tabi@ammasso.com>]

Troy Benjegerdes wrote:

> Someone (aka Tospin, infinicon, and Amasso) should probably post a patch
> adding '#define VM_REGISTERD 0x01000000', and some extensions to
> something like 'madvise' to set pages to be registered.
> 
> My preference is said patch will also allow a way for the kernel to
> reclaim registered memory from an application under memory pressure.

I don't know if VM_REGISTERED is a good idea or not, but it should be absolutely 
impossible for the kernel to reclaim "registered" (aka pinned) memory, no matter what. 
For RDMA services (such as Infiniband, iWARP, etc), it's normal for non-root processes to 
pin hundreds of megabytes of memory, and that memory better be locked to those physical 
pages until the application deregisters them.

If kernel really thinks it needs to unpin those pages, then at the very least it should 
kill the process, and the syslog better have a very clear message indicating why.  That 
way, the application doesn't continue thinking that everything's still going to work.  If 
those pages become unpinned, the applications are going to experience serious data corruption.