Re: [RFC PATCH 1/5] IB/core: Add Core Capability flags to ib_device

Dave Goodell (dgoodell)
2015-05-05T20:22:21Z

On May 5, 2015, at 2:51 PM, Liran Liss <liranl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:

>> From: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org [mailto:linux-rdma-
>> owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org] On Behalf Of Hefty, Sean
>
>>> I thought USNIC_UDP had an embedded USNIC protocol header inside the
>>> UDP header. That would make it a UDP_ENCAP protocol.
>>
>> Someone from Cisco can correct me, but USNIC supports 2 protocols. Just
>> plain UDP, and a proprietary protocol that runs over Ethernet, but uses the
>> same EtherType as RoCE. I thought these could both be active on the same
>> port at the same time.
>>
>
> 'protocol' refers to what your device generates when you do a post_send() on
> some QP.
> In the RoCE case, it is IBTA transport headers + payload over UDP encapsulation
> over IP. In the USNIC case, you might want the protocol to refer to addition
> information the distinguishes this wire protocol rather than just "I am sending
> UDP packets"...

In the case that usNIC is operating in UDP mode (which is the overwhelming majority of the cases), there is absolutely no additional protocol that ends up on the wire or headers in the user buffers besides UDP/IP/Ethernet. They are 100% plain UDP packets, they just happen to be sent via OS-bypass queues instead of traveling through the kernel networking stack.

[^^^^^ there continues to be confusion about this for some reason, but I don't know why]

> In other words, I think that 'protocol' should uniquely distinguish interoperable
> peers at the Verbs API level. We are *not* trying to describe a certain header,
> but rather a stack of protocols.

Any non-UDP "protocol" that might currently be in use over usNIC is an entirely application-level protocol outside of the view of the Verbs API level.

-Dave