Re: [PATCH for-next 1/7] IB/core: Refactor IDR to be per-device

[Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>]

On Tue, Jan 17, 2017 at 07:08:02PM +0200, Matan Barak wrote:

> > I'm still unclear why we want the idr to be global and not per
> > /dev/uverbs0 open fd.
> 
> Assuming we keep the current separate fds model (i.e, per-device fd
> and another rdma-cm fd), what is the actual benefit here?  Memory
> wise - we'll probably consume a lot more (every idr layer is > 2K on
> a 64bit architecture), so sharing here might have some memory usage
> benefits.

Well, it makes things simpler and clearer.

- Locking is per ucontext, not per-device global, so it is potentially
  more parallel.
- Obvious that the lifetime of these objects is tied to the lifetime
  of the ucontext
- max idr value is per-context, not per system, so the tree will be
  intrinsiclly smaller. (eg consider a MPI that allocates 1 million
  objects is permanently degrading the global IDR)
- idr exhaustion is per-context not global, thus it plays nicer with
  a memory cgroup controller.
- It is very clear you have to have a ucontext fd to make any use
  of the object numbers - this is an essential property from a
  security perspective.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html