Re: [PATCH rdma-next 00/10] Hardware tag matching support

[Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 2160 bytes --]

On Thu, Oct 13, 2016 at 04:30:10PM -0400, Doug Ledford wrote:
> On 10/13/2016 4:02 PM, Christoph Hellwig wrote:
> > On Thu, Oct 13, 2016 at 01:56:05PM -0600, Jason Gunthorpe wrote:
> >> A bigger objection I have is that much of this new stuff is not
> >> adaquately documented.
> >>
> >> Tag matching is a great example. It looks like this introduces many new
> >> headers into the on-wire protocol. Where are these headers documented?
> >> Where is the spec for the on-wire format? Where is the implementation
> >> for rxe?
> >
> > Agreed on that part.  For pon the wire changes we need the hardware
> > standards organization to act.  IBTA for IB and RoCE and IETF for iWarp,
> > although not even having an active WG for iWarp there is a major problem
> > at the moment.
> >
>
> I think there are some fundamental questions here about what this is.
> As I understand it (and Mellanox can chime in if I'm wrong), tag
> matching already exists on the wire, it's just implemented above the
> level of the link layer transmissions.  This enables a hardware
> accelerator for something that is already done at an upper layer, so the
> implementation is mostly defined by the defacto existing use in upper
> layer applications, not by the IBTA.  I believe this could roughly be
> considered analogous to a network driver enabling a layer3 or layer4
> accelerator.  It's not a layer1 thing, so the IBTA is not appropriate to
> review it.

Yes, you are right. It is in use by most if not all major HPC players
and in this patch set, we are providing possibility to configure it and
expose supported flavours of tag matching, so they will be able to run
RHEL out of the box. There are no HW limitations on how other vendors
(let's pretend that they exist) will implement such feature.

As Sean mentioned in the other email, PSM has it and libfabric natively
supports this functionality through MXM provider.

Regarding documentation, we are in continuous process of improvements
for our submissions and will definitely put our attention to that point.

Thanks

>
> --
> Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>     GPG Key ID: 0E572FDD
>




[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]