Re: [PATCH v2 01/17] IB/Verbs: Implement new callback query_transport() for each HW

["ira.weiny" <ira.weiny@intel.com>]

On Fri, Apr 10, 2015 at 12:04:55PM -0600, Jason Gunthorpe wrote:
> On Fri, Apr 10, 2015 at 01:38:38PM -0400, ira.weiny wrote:
>  
> > 
> > I don't think that is appropriate.  You have been advocating that the checks
> > be clear as to what support we need. 
> 
> Right, but this is narrow, and we are not hiding meaning.
> 
> Look at the IPoIB ULP, and look at the hard requiments of the code,
> then translate those back to our new cap scheme. We see today's IPoIB
> will not run without:
>  - UD support
>  - IB addressing
>  - IB multicast
>  - IB SA
>  - CM (optional)
> 
> It seems perfectly correct for a ULP to say at the very start, I need
> all these caps, or I will not run (how could it run?). This is true of
> any ULP that has a hard need to use those APIs.

Having IPoIB check for all of these and fail to start if not supported is good.
But the suggestion before was to have "cap_ipoib".  I don't think we want that.

> 
> That would seem to be the very essance of the cap scheme. Declare what
> you need, not what standard you think you need.

Right.

> 
> Hiding meaning is to say 'only run on IB or OPA': WHY are we limited
> to those two?

Because only those 2 support the list of capabilities above.

> 
> > While currently the IPoIB layer does (for IB and OPA) require an SA
> > I think those checks are only appropriate when it is attempting an
> > SA query.
> 
> That doesn't make any sense unless someone also adds support for
> handling the !SA case.

Fair enough but IPoIB does need to check for SA support now as well as IB
addressing which is currently IB Link Layer (although as Doug said the Port and
Address format go hand in hand.  So I'm happy calling it whatever.)

> 
> > > It appears we have at least rocee, rocee v2 (udp?), tcp, ib and opa
> > > address and AH formats?
> > 
> > Seems that way.  But has the rocee v2 been accepted?
> 
> Don't know much about it yet, patches exist, it seems to have a
> slightly different addressing format.
> 
> > > opa would support ib addresses too I guess.
> > 
> > Yes opa address == ib addresses.  So there is no need to distinguish them.
> 
> The patches you sent showed a different LRH format for OPA (eg 32 bit
> LID), so someday we will need to know that the full 32 bit LID is
> available.
> 
> We can see how this might work in future, lets say OPAv2 *requires* the
> 32 bit LID, for that case cap_ib_address = 0 cap_opa_address = 1. If
> we don't update IPoIB and it uses the tests from above then it
> immediately, and correctly, stops running on those OPAv2 devices.

For your hypothetical case, agreed.  But I want to make it clear for those who
may be casually reading this thread that OPA addresses are IB addresses right
now.

> 
> Once patched to support cap_op_address then it will begin working
> again. That seems very sane..

Agreed.

Ira