Re: [PATCH 02/12] IB/cma: pass the port number to ib_create_qp

[Jason Gunthorpe <jgunthorpe@obsidianresearch.com>]

On Thu, Apr 28, 2016 at 09:53:52PM +0000, Hefty, Sean wrote:
> > I see several litmus tests for what kinds of ports can be combined
> > into a device (eg the 'protocol'):
> > 
> > 1) Various cap tests are the same on every port. Particularly the
> >    iWarp special behaviours we are talking about here.
> > 2) AHs are not port-specific, so the AH addressing format must be
> >    defined by the device. Thus IB and iWarp cannot be combined.
> > 3) Verbs APM must work across ports. So eg rocee and IB cannot be
> >    combined since they use a different CM process.
> > 
> > Multi-port really only exists to support APM, if APM doesn't work then
> > drivers don't need to create multi-port devices.
> 
> I don't know the details of the qlogic device, but it is entirely
> possible that it allows different protocols to share resources (PDs,
> CQs, IP addresses, etc.).  I think we need to be careful dismissing
> multi-protocol devices as silly, or restricting which protocols can
> run over which port.

This isn't dismissing them as silly, it is a pragmatic need in the
core code that everything associated with a PD have a minimum standard
of uniformity - and it is very clear that includes things like the
iwarp special cases and the particular format of the AHs.

For instance, even if a hardware device can run rocee and iwarp
concurrently over a single port, today we absolutely must have
different struct ib_devices for the same physical port to be able to
plug that into the core stack.

Fundamentally we have the wrong model for such hardware. When a PD is
created it should set the 'protocol' and select the compatible member
ports that belong to the PD. Cap tests and so forth should be done
against the PD, not a port or a device.

Fixing that is major surgery, and having cap tests to the port is not
helping clarify the current situation.

> Restricting all ports on a device to support all protocols is
> different than restricting a device to supporting a single protocol,
> and it affects more than APM.

What else is there that is cross port in verbs?

Jason