Re: [RFC PATCH 1/5] IB/core: Add Core Capability flags to ib_device

[Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 2551 bytes --]

On Mon, 2015-05-04 at 18:26 +0000, Hefty, Sean wrote:

> > Given that this is a per port thing, there is no assumption about a
> > device only supporting a single protocol.
> 
> Device, no, but we are assuming this per port.

At a high level, there are certain things that will be per port because
they are tied to the link layer in question (you won't have a port with
IB_SA that doesn't have IB_SA on all its queue pairs on that port).

>   I don't think this is
> true for USNIC.

I can't speak for USNIC, I haven't read the driver for it closely
enough.

>   For that matter, it's entirely possible for a RoCEv2
> device to expose UDP directly to user space, same as USNIC.

I'm not sure I'm following what you are saying here.  USNIC is unique in
that it presents a NIC to the application that it has exclusive control
over.

>   (I'd
> actually be surprised if no devices have this capability, if for
> debugging capabilities, even if for nothing else.)  What are we going
> to do if there's a device that supports both iWarp and RoCEv2?  That's
> easily doable today through software.

It's doable, but nobody is actually doing it.

> Basically, I question how protocol is defined and what it means to
> expose it as a device attribute.  Should it instead be negotiated (i.e.
> more like sockets)?  If an app needs a specific "RDMA protocol" (IB or
> iWarp) or "application protocol" (IB or iWarp or UDP or MADs?), they
> can request it.  Otherwise, the app gets assigned some protocol.  And
> the protocol should really be associated with a QP, rather than a
> port.

If you're going down to that level (which we will when the RoCEv2
patches are integrated), then it doesn't quite map to QPs.  Since a UD
QP may need to talk to both RoCEv1 and RoCEv2 hosts, it's at the AH
level for RoCE.

To a certain extent you are right.  RoCE, RoCEv2, iWARP, and USNIC could
all reside on a single port by virtue of them all sharing an Ethernet
link layer.  IB link layer (and associated attributes) and OPA link
layer are the outliers in this method of thinking in that their ports
will only support their own transport.  So if we were to actually be
forward thinking about this, then the link layer is the important bit
you need to capture on a per port basis, not the transport technology.
We would have to modify lots of the stack to support per QP or per AH
transport settings though.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]