Re: [PATCH][RFC][0/4] InfiniBand userspace verbs implementation

[Timur Tabi <timur.tabi@ammasso.com>]

Andrew Morton wrote:

> The way we expect get_user_pages() to be used is that the kernel will use
> get_user_pages() once per application I/O request.
> 
> Are you saying that RDMA clients will semi-permanently own pages which were
> pinned by get_user_pages()?  That those pages will be used for multiple
> separate I/O operations?

Yes, absolutely!

The memory buffer is allocated by the process (usually just via malloc) and 
registed/pinned by the driver.  It then stays pinned for the life of the process (typically).

> If so, then that's a significant design departure and it would be good to
> hear why it is necessary.

That's just how RMDA works.  Once the memory is pinned, if the app wants to send data to 
another node, it does two things:

1) Puts the data into its buffer
2) Sends a "work request" to the driver with (among other things) the offset and length of 
the data.

This is a time-critical operation.  It must occurs as fast as possible, which means the 
memory must have already been pinned.

-- 
Timur Tabi
Staff Software Engineer
timur.tabi@ammasso.com

One thing a Southern boy will never say is,
"I don't think duct tape will fix it."
      -- Ed Smylie, NASA engineer for Apollo 13