Re: [openib-general] Re: [PATCH][RFC][0/4] InfiniBand userspace verbs implementation

Timur Tabi
2005-04-21T03:08:33Z

Troy Benjegerdes wrote:

> Someone (aka Tospin, infinicon, and Amasso) should probably post a patch
> adding '#define VM_REGISTERD 0x01000000', and some extensions to
> something like 'madvise' to set pages to be registered.
>
> My preference is said patch will also allow a way for the kernel to
> reclaim registered memory from an application under memory pressure.

I don't know if VM_REGISTERED is a good idea or not, but it should be absolutely
impossible for the kernel to reclaim "registered" (aka pinned) memory, no matter what.
For RDMA services (such as Infiniband, iWARP, etc), it's normal for non-root processes to
pin hundreds of megabytes of memory, and that memory better be locked to those physical
pages until the application deregisters them.

If kernel really thinks it needs to unpin those pages, then at the very least it should
kill the process, and the syslog better have a very clear message indicating why. That
way, the application doesn't continue thinking that everything's still going to work. If
those pages become unpinned, the applications are going to experience serious data corruption.