Re: [PATCH V3 1/5] RDMA/core: Transport-independent access flags

[Tom Talpey <tom@talpey.com>]

On 7/8/2015 3:08 PM, Jason Gunthorpe wrote:
> The MR stuff was never really designed, the HW people provided some
> capability and the SW side just raw exposed it, thoughtlessly.

Jason, I don't disagree that the API can be improved. I have some
responses to your statements below though.

> Why is code using iWarp calling ib_get_dma_mr with
> RDMA_MRR_READ_DEST/IB_ACCESS_REMOTE_WRITE ? That is insecure.

Because the iWARP protocol requires it, which was very much an
intentional decision. It actually is not insecure, as discussed
in detail in RFC5042. However, it is different from Infiniband.

> Why on earth is NFS using frmr to create RDMA READ lkeys?

Because NFS desires to have a single code path that works for all
transports. In addition, using the IB dma_mr as an lkey means that
large I/O (common with NFS) would require multiple RDMA Read
operations, when the page list exceeded the local scatter/gather
limit of the device.

> I think when you do that, it quickly becomes clear that iWarp's
> problem is not a seemingly minor issue with different flag bits, but
> that iWarp *cannot* use local_dma_lkey as a RDMA READ local buffer.
> Using ib_get_dma_mr(IB_ACCESS_REMOTE_WRITE) is an insecure work
> around. So iWarp (and only iWarp) should take the pain of spinning up
> temporary local MRs for RDMA READ.

That is entirely the wrong layer to address this. It would prevent
reuse of MRs, and require upper layers to be aware that this was
the case - which is exactly the opposite of what you are trying
to achieve.

> This should all be hidden under a common API and it shouldn't be
> sprinkled all over the place in NFS, iSER, etc.

Are you arguing that all upper layer storage protocols take a single
common approach to memory registration? That is a very different
discussion.

Tom.