Re: [PATCH 00/25] Shared PD and MR

[Yuval Shaia <yuval.shaia@oracle.com>]

On Wed, Jul 17, 2019 at 04:55:26PM -0700, Ira Weiny wrote:
> On Wed, Jul 17, 2019 at 04:35:30PM +0300, Shamir Rabinovitch wrote:
> > On Wed, Jul 17, 2019 at 2:55 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> > >
> > > On Wed, Jul 17, 2019 at 02:09:50PM +0300, Shamir Rabinovitch wrote:
> > > > On Wed, Jul 17, 2019 at 8:09 AM Christoph Hellwig <hch@infradead.org> wrote:
> > > > >
> > > > > On Tue, Jul 16, 2019 at 09:11:35PM +0300, Shamir Rabinovitch wrote:
> > > > > > Following patch-set introduce the shared object feature.
> > > > > >
> > > > > > A shared object feature allows one process to create HW objects (currently
> > > > > > PD and MR) so that a second process can import.
> > > > >
> > > > > That sounds like a major complication, so you'd better also explain
> > > > > the use case very well.
> > > >
> > > > The main use case was that there is a server that has giant shared
> > > > memory that is shared across many processes (lots of mtts).
> > > > Each process needs the same memory registration (lots of mrs that
> > > > register same memory).
> > > > In such scenario, the HCA runs out of mtts.
> > > > To solve this problem, an single memory registration is shared across
> > > > all the process in that server saving hca mtts.
> > >
> > > Well, why not just share the entire uverbs FD then? Once the PD is
> > > shared all security is lost anyhow..
> > >
> > > This is not the model that was explained to me last year
> > >
> > > Jason
> > 
> > We do share the whole uvrbs FD (context) with the second process and
> > let that process to instantiate the PD & MR from the shared FD.
> 
> Then the first (both) process(es) should have access to the MR right?

Yes.
(Please note that since we maintain refcount then the ib_mr object will be
destroyed only when the two will destroy it).

> 
> > The instantiation include creating new uobject in the second process
> > context that points to the same ib_x HW objects.
> > The second process does not own the shared context.
> > It just use it to get access to the shared ib_x objects and then it
> > mark those & shared FD as shared.
> 
> I'm not following this?

Shamir, correct me if i'm wrong here:
So there is one ib_mr object and two uobjects that "points" to it.

> 
> > 
> > What was the expectation from "import_from_xxx" ?
> 
> ... and I don't understand this question.

Shamir?

> 
> Ira
>