Re: [PATCH V3 1/5] RDMA/core: Transport-independent access flags

[Jason Gunthorpe <jgunthorpe@obsidianresearch.com>]

On Tue, Jul 14, 2015 at 02:58:23PM -0500, Steve Wise wrote:
> The local_dma_lkey can be used in any rdma sge that requires an
> lkey.

No, this is where iWarp doesn't follow the generic API - a local dma
lkey cannot be used with iWarp's RDMA_READ WR lkey. In effect RDMA
READ requires an *rkey* (confusingly stuck into the lkey slot) for
iWarp. (Right?)

*THAT* is really the core difference between IB and iWarp in this
area, not that the access flags are different.

(cap_rmda_read_lkey_is_rkey ?)

> domain.  But I claim for lkeys, the PD doesn't really protect
> anything since the remote peers can't use it anyway.

I agree.

To use a PD properly I'd have thought it should be created on a client
by client basis. The risk is tiny, but client X should not be able to
guess Y's RKey and then corrupt a data transfer. *Especially* on a
server if client X hasn't auth'd yet .... That is what the PD is for.

> There is confusion about lkeys and rkeys with regard to iWARP.  In
> the iWARP verbs, there is no distinction between an lkey and rkey:
> they are the same key, called a Steering Tag or STAG.  When you
> create a MR, the lkey == rkey == STAG for iwarp transports.
> Somewhat related, but really a different issue, is that SGEs that
> are the target of a read need REMOTE_WRITE access flags on their
> STAG for iWARP.

That is the clearest explanation for the iWarp difference I've seen so
far, thanks!

Christoph: The take away from all this is that on iWarp RDMA_READ
requires a restricted temporary MR to provide the lkey value in the
WC. It cannot use local_dma_lkey.

Everything else is the same between IB and iWarp: local_dma_lkey can
be used as the lkey for SEND, RECV, WRITE.

Jason