RE: [PATCH V3 1/5] RDMA/core: Transport-independent access flags

Steve Wise
2015-07-14T20:54:15Z

> -----Original Message-----
> From: linux-nfs-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org [mailto:linux-nfs-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org] On Behalf Of Jason Gunthorpe
> Sent: Tuesday, July 14, 2015 3:45 PM
> To: Steve Wise
> Cc: 'Christoph Hellwig'; 'Sagi Grimberg'; 'Tom Talpey'; 'Doug Ledford'; sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org; ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org;
> roid-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org; linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; eli-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org; target-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org;
> trond.myklebust-7I+n7zu2hftEKMMhf/gKZA@public.gmane.org; bfields-uC3wQj2KruNg9hUCZPvPmw@public.gmane.org; 'Oren Duer'
> Subject: Re: [PATCH V3 1/5] RDMA/core: Transport-independent access flags
>
> On Tue, Jul 14, 2015 at 03:40:49PM -0500, Steve Wise wrote:
> > > local_dma_lkey appears to be global, it works with any PD.
> > >
> > > ib_get_dma_mr is tied to a PD, so it cannot replace local_dma_lkey at
> > > the struct device level.
> > >
> > > ib_alloc_pd is the in-kernel entry point, the UAPI calls
> > > device->alloc_pd directly.. So how about the below patch as a starting
> > > >point?
> > >
> > > (Steve the goal of step #1 would be to remove ib_get_dma_mr from ULPs
> > > Follow on patches would be to convert all ULPs to use this API change.)
>
> > I'm not seeing the benefit of adding pd->local_dma_lkey?
> > pd->device->local_dma_lkey is there for core and ULP use, and we
> > could have old drivers that don't currently have support for
> > local_dma_lkey allocate their own private pd/dma_mr (via their
> > private functions for doing this) with only LOCAL_WRITE access
> > flags, and export that lkey as the device->local_dma_lkey. Wouldn't
> > that be simpler?
>
> It would be, but AFAIK that can't work?
>
> My understanding is if you create a QP against a PD then only lkeys
> and rkeys (and local_dma_rkey) created against that PD are valid for
> use with that QP.
>
> I can't use an lkey from a PD not associated with the QP.
>
> Am I wrong on this?

Kernel users can use the local_dma_lkey for all lkey IO on all QPs (ignoring the iwarp read issue). Look at sc_dma_lkey in the
NFSRDMA server.

>
> Jason
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in