Re: Status of "ummunot" branch?

[Haggai Eran <haggaie-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>]

On 05/06/2013 22:05, Jason Gunthorpe wrote:
> On Wed, Jun 05, 2013 at 06:45:13PM +0000, Jeff Squyres (jsquyres) wrote:
>> What happens if you:
>>
>> a = malloc(N * page_size);
>> ibv_reg_mr(..., a, N * page_size, ...);
>> free(a);
>> // incoming RDMA arrives targeted at buffer a
> 
> Haggai should comment on this, but my impression/expectation was
> you'll get a remote protection fault/
> 
>> Or if you:
>>
>> a = malloc(N * page_size);
>> ibv_reg_mr(..., a, N * page_size, ...);
>> free(a);
>> a = malloc(N / 2 * page_size);
>> // incoming RDMA arrives targeted at buffer a that is of length (N*page_size)
> 
> again, I expect a remote protection fault.
> 
> Noting of course, both of these cases are only true if the underlying
> VM is manipulated in a way that makes the pages unmapped (eg
> mmap/munmap, not free)

That's right. If pages are unmapped and a remote operation tries to
access them the QP will be closed with a protection error.

> 
> I would also assume that attempts to RDMA write read only pages
> protection fault as well.
Right.

> Haggai: A verb to resize a registration would probably be a helpful
> step. MPI could maintain one registration that covers the sbrk
> region and one registration that covers the heap, much easier than
> searching tables and things.

That's a nice idea. Even without this verb, I think it is possible to
develop a registration cache that covers those regions though. When you
find out you have some part of your region not registered, you can
register a new, larger region that covers everything you need. For new
operations you only use the newer region. Once the previous, smaller
region is not used, you de-register it.

What do you think?

Haggai

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html