RE: [PATCH V3 1/5] RDMA/core: Transport-independent access flags

["Steve Wise" <swise@opengridcomputing.com>]

> -----Original Message-----
> From: Hefty, Sean [mailto:sean.hefty@intel.com]
> Sent: Thursday, July 23, 2015 1:53 PM
> To: Steve Wise; 'Christoph Hellwig'
> Cc: 'Sagi Grimberg'; 'Steve Wise'; 'Jason Gunthorpe'; 'Tom Talpey'; 'Doug Ledford'; sagig@mellanox.com; ogerlitz@mellanox.com;
> roid@mellanox.com; linux-rdma@vger.kernel.org; eli@mellanox.com; target-devel@vger.kernel.org; linux-nfs@vger.kernel.org;
> trond.myklebust@primarydata.com; bfields@fieldses.org; 'Oren Duer'
> Subject: RE: [PATCH V3 1/5] RDMA/core: Transport-independent access flags
> 
> > There is confusion about lkeys and rkeys with regard to iWARP.  In the
> > iWARP verbs, there is no distinction between an lkey and
> > rkey: they are the same key, called a Steering Tag or STAG.  When you
> > create a MR, the lkey == rkey == STAG for iwarp transports.
> > Somewhat related, but really a different issue, is that SGEs that are the
> > target of a read need REMOTE_WRITE access flags on their
> > STAG for iWARP.
> >
> > Clear as mud? :)
> 
> This may be a nit, but IMO, the use of the term 'rkey' versus 'stag' matters.  They convey different ways of finding a data
buffer.  For
> example, do you locate a buffer using the stag, then verify that the offset + length fits into the target buffer?  

Yes.  HW always uses the stag to locate a record that contains the stag state (valid or invalid), the access flags, the 8b key, the
va_base, length, PBL describing the host pages, etc.  HW validates all that before using the buffer.  NOTE: An stag of 0 is the
special local-dma-lkey which HW treats differently: If the stag is 0, then the address in the SGE is the bus/dma address itself and
no lookup of a MR/PBL/etc is needed.   Stag 0 can ONLY be used by kernel users and MUST never be accepted/used from an ingress
packet and MUST never be emitted on the wire in a READ or WRITE.

> Or do you locate the buffer
> by address, then verify that the key matches?
> 

This is never done.

> Consider if we allow an app to specify the rkey/stag, or reference the buffer using an offset, rather than a virtual address.
> 
> This seems to be part of the difference between an lkey and an rkey.