RE: [PATCH rdma-next 0/3] Support out of order data placement

Hefty, Sean
2017-06-12T19:19:06Z

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1618 bytes --]

> I don't understand why the application has to care about this at all.
> If
> the HW wants to place things out of order. Ok go for it. Applications
> should be depending on the correct mechanism to know when it's OK to
> look at the data. Why do they care what order it arrived in?
>
> The only reason to care is if the applications wants to do something
> that is not compliant and look at parts of the data early. Perhaps you
> can explain *how* an application may benefit from out-of-order
> placement
> and likewise, why it wouldn't want to benefit.

Relaxed ordering can improve performance by allowing packets to take multiple paths from the source to destination. FWIW, the ofiwg actually discussed ordering at length, and libfabric exposes several ordering related attributes.

The application usually needs to know what sort of ordering the network is providing for correct operation. For example, can sends be re-ordered by the network, such that they may be received in the opposite order that they were sent? Can rdma writes be processed out of order, such that the target data ends up with some sort of interleaving of the written data? Are there size limits to ordering constraints?

It's hard to capture this level of detail with a single flag. There's ordering of data within a single message, as well as ordering of data between messages. I don't know how this proposal works with explicit pathing that IB defines, or path failover.

- Sean
N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±­ÙšŠ{ayº\x1dÊ‡Ú™ë,j\a­¢f£¢·hš‹»öì\x17/oSc¾™Ú³9˜uÀ¦æå‰È&jw¨®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿïêäz¹Þ–Šàþf£¢·hšˆ§~ˆmš