Re: [PATCH V3 1/5] RDMA/core: Transport-independent access flags

["'Christoph Hellwig'" <hch@infradead.org>]

On Tue, Jul 14, 2015 at 02:29:43PM -0600, Jason Gunthorpe wrote:
> local_dma_lkey appears to be global, it works with any PD.
> 
> ib_get_dma_mr is tied to a PD, so it cannot replace local_dma_lkey at
> the struct device level.
> 
> ib_alloc_pd is the in-kernel entry point, the UAPI calls
> device->alloc_pd directly.. So how about the below patch as a starting
> point?

This looks perfect to me.  After this we can get rid of the
ib_get_dma_mr calls outside of ib_alloc_pd, and eventuall move
setting up ->local_dma_lkey into the HW driver and kill of
ib_get_dma_mr, IB_DEVICE_LOCAL_DMA_LKEY and device->local_dma_lkey.

> >  (2) now ULD can check for IB_DEVICE_LOCAL_DMA_LKEY and use local_dma_lkey
> >      in that case, or will have to perform a per-IO MR with LOCAL and
> >      REMOTE flags if not
> 
> If we do the above all ULPs simply use pd->local_dma_lkey and we drop
> correct uses of ib_get_dma_mr completely ?

Yes, please, although:

> >  (3) remove insecure remote uses of ib_get_dma_mr from ULDs

I kept this separate as the I suspect the "optimizations" using 
ib_get_dma_mr with remote flags will warrant a separate discussion.

> >From 5a9799bf487d822daae5a8b8f3b197f1dda1db45 Mon Sep 17 00:00:00 2001
> From: Jason Gunthorpe <jgunthorpe@obsidianresearch.com>
> Date: Tue, 14 Jul 2015 14:27:37 -0600
> Subject: [PATCH] IB/core: Guarantee that a local_dma_lkey is available
> 
> Every single ULP requires a local_dma_lkey to do anything with
> a QP, so lets us ensure one exists for every PD created.
> 
> If the driver can supply a global local_dma_lkey then use that, otherwise
> ask the driver to create a local use all physical memory MR associated
> with the new PD.
> 
> Signed-off-by: Jason Gunthorpe <jgunthorpe@obsidianresearch.com>


Acked-by: Christoph Hellwig <hch@lst.de>

(especially together with patches removing all other callsites of
"ib_get_dma_mr(pd, IB_ACCESS_LOCAL_WRITE)".