Re: mlx4: IB_EVENT_PATH_MIG not generated on path migration

[Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>]

On Wed, Feb 11, 2015 at 10:21:36AM +0100, Fabian Holler wrote:
> Hello Jason,
> 
> thanks for your answer.
> 
> On Tue, Feb 10, 2015 at 11:00:39AM -0700, Jason Gunthorpe wrote:
> > On Tue, Feb 10, 2015 at 04:56:43PM +0100, Fabian Holler wrote:
> > 
> > > Does anybody have an idea what could be wrong?
> > > Are the PATH_MIG* notifications with mlx4 drivers are working for somebody?
> > 
> > IIRC rdmacm does not forward these events.
> > 
> > There is a lot missing to support APM with RDMA CM, it doesn't do any
> > of the CM state machine operations to coordinate loading alternate
> > paths, for instance.
> 
> Alternate paths can be set directly at the QP.

APM requires a certain amount of synchronization between the local and
remote QP state, particularly if you want it to work more than
once. That is supposed to be done with CMA mads.

> The event handler specified in the struct ib_qp_init_attr * parameter of
> rdma_create_qp() is registered as QP event handler.
> rdma_create_qp() calls ib_create_qp() and sets qp->event_handler to
> ib_qp_init_attr->event_handler.

Okay, right, I was thinking of something else.

We had exactly this working a few months ago for a demo, in kernel
using Lustre and RDMA CM. I checked the patches we made and yes, the
event_handler was being properly called with IB_EVENT_PATH_MIG on
mlx4 and qib cards.

IIRC we may have needed the latest firmware on the mlx side.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html