RE: [PATCH rdma-next 0/3] Support out of order data placement

Parav Pandit
2017-06-12T21:25:44Z

Hi Sean,

> -----Original Message-----
> From: Hefty, Sean [mailto:sean.hefty@intel.com]
> Sent: Monday, June 12, 2017 3:41 PM
> To: Parav Pandit <parav@mellanox.com>; Dalessandro, Dennis
> <dennis.dalessandro@intel.com>; Steve Wise
> <swise@opengridcomputing.com>; 'Leon Romanovsky' <leon@kernel.org>;
> 'Doug Ledford' <dledford@redhat.com>
> Cc: linux-rdma@vger.kernel.org; Idan Burstein <idanb@mellanox.com>
> Subject: RE: [PATCH rdma-next 0/3] Support out of order data placement
>
> > > For example, can sends be re-ordered by the network, such that they
> > > may be received in the opposite order
> > that they
> > > were sent?
> >
> > No. Only rdma reads and writes. Table 79 of IB spec still holds true.
> > In internal patch review I had the table, but since there was no
> > change, I just removed it.
> > I think it's worth to mention in the out_of_order.txt that it still
> > holds true. I will add this line.
> >
> > I would also document which compliant statements are not applicable
> > when such flag is set.
> > How about having responder side table, similar to table 79 in
> > Documentation/out_of_order.txt That makes it more readable for users.
> >
> > > Can rdma writes be processed out of order, such that the target data
> > > ends up with some sort of interleaving of the written data?
> > Yes.
> >
> > > Are there size limits to ordering constraints?
> > No.
>
> My questions were more examples of the level of detail that I think any API
> change needs to (eventually) be able to address. Libibverbs is supposed to
> work over iwarp as well, which has different ordering semantics already. I
> think that was part of Steve's point as to why any change is needed to the
> api.
>
> IMO, read and write ordering should be determined separately from the
> perspective of the api. This is exposing a hw specific limitation where these
> semantics are joined, and send/receive is not supported.

They are joined for simplicity. If there is established use case that I like to hear from users
and if hardware supports it, more caps can be added.
That’s why caps is 32-bit number for future addition of RD_ONLY, WR_ONLY modes.