Re: [PATCH rdma-next 0/3] Support out of order data placement

[Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>]

On Sat, Jul 22, 2017 at 07:51:33AM -0700, Tom Talpey wrote:
> Well, if the broken applications won't use the extension, and
> the existing storage protocols and applications will have to
> change both their implementation and their protocol to use it,
> who do you envision actually doing so?

The apps are not broken, it appears that this extension allows the
receiver HCA to process inbound packets without seeing earlier
packets.

This means it can really start doing things out of order, subject only
to the ability of the transmitter to halt sending (eg fence) until
acks are seen.

Since there is no way to predict what is in the data the HCA didn't
see, this would seem to allow quite a lot of out of order execution
inside the HCA.

Eg

 RDMA-W VA=0 Data=1
 RDMA-R VA=0
 RDMA-W VA=0 Data=2

What does the read return? Spec says 1, but it sounds like this
relaxed ordering could return 2.

Whta about

 RDMA-W VA=0 Data=1
 SEND WITH INVALIDATE VA=0
 RDMA-W VA=0 Data=2

Spec says the second RDMA-W must fail, but it sounds like this relaxed
ordering would allow it to happen.

So it cannot be turned on by default.. But a traditional well behaved
storage ULP will only do single access to a single VA and should be
able to safely turn something like this on.

All of these is independent to observability at the CPU..

All this means it needs to be negotiated via RDMA-CM, and making it a
common verbs option would make sense - is there a plan to do that?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html