Re: [PATCH rdma-core 0/5] Common systemd/udev based boot support

[Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>]

On Wed, Jul 26, 2017 at 09:05:03AM -0500, Steve Wise wrote:

> > I think you should change this to create the RDMA device when the
> > module is installed and the hardware is present..
> >
> 
> Not gonna happen.  cxgb4 doesn't setup the queues, rss, irq mappings, etc, until
> an interface is brought up.  So iw_cxgb4 cannot initialize and register with the
> rdma core until that happens. 

But it doesn't tear all that stuff down on ifdown?

> > Basically, it is very hard to start a RDMA daemon and not have it race
> > with something and randomly fail to start properly the more hotpluggy
> > things are.
> 
> I think these races exist today, no?  Or is this patch series introducing the
> races?   The iwpmd does not need the providers registered at the time it starts.
> It will discover new iwarp providers as they initialize. 

There are races today, particularly with systemd, but I suspect they
are slightly different.. That doesn't mean we should ignore them :)

In particular the rdma.target approach loaded all the modules, even if
there wasn't hardware support, so when the cxgb4 RDMA device is
created everything is already to go. We still have races, but they are
probably smaller

Now, we don't even plug the modules until the RDMA device appears, so
we have broader possibilities for racing..

Overall it makes it hard to fit into the ideal system configuration
where RDMA sets up before network-pre.target when a RDMA device
doesn't even plug in until after network.target.

> Can you think of other ways to address your concerns?

Maybe we should have udev trigger loading iwarp services on the cxgb4
ethernet device? At least the races then become more like what we had
before. Are other roce/iwarp devices like this?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html