Re: [PATCH 07/22] RDMA/irdma: Register an auxiliary driver and implement private channel OPs

[Leon Romanovsky <leon@kernel.org>]

On Wed, Jan 27, 2021 at 12:41:41AM +0000, Saleem, Shiraz wrote:
> > Subject: Re: [PATCH 07/22] RDMA/irdma: Register an auxiliary driver and
> > implement private channel OPs
> >
> > On Tue, Jan 26, 2021 at 12:42:16AM +0000, Saleem, Shiraz wrote:
> >
> > > I think this essentially means doing away with .open/.close piece.
> >
> > Yes, that too, and probably the FSM as well.
> >
> > > Or are you saying that is ok?  Yes we had a discussion in the past and
> > > I thought we concluded. But maybe I misunderstood.
> > >
> > > https://lore.kernel.org/linux-rdma/9DD61F30A802C4429A01CA4200E302A7DCD
> > > 4FD03@fmsmsx124.amr.corp.intel.com/
> >
> > Well, having now seen how aux bus ended up and the way it effected the
> > mlx5 driver, I am more firmly of the opinion this needs to be fixed. It is extremly
> > hard to get everything right with two different registration schemes running around.
> >
> > You never answered my question:
>
> Sorry I missed it.
> >
> > > Still, you need to be able to cope with the user unbinding your
> > > drivers in any order via sysfs. What happens to the VFs when the PF is
> > > unbound and releases whatever resources? This is where the broadcom
> > > driver ran into troubles..
> >
> > ?
>
> echo -n "ice.intel_rdma.0" > /sys/bus/auxiliary/drivers/irdma/unbind  ???
>
> That I believe will trigger a drv.remove() on the rdma PF side which require
> the rdma VFs to go down.
>
> Yes, we currently have a requirement the aux rdma PF driver remain inited at least to .probe()
> for VFs to survive.
>
> We are doing internal review, but it appears we could potentially get rid of the .open/.close callbacks.
> And its associated FSM in ice.
>
> But if we remove peer_register/unregister, how do we synchronize between say unload of the rdma driver
> and netdev driver stop accessing the priv channel iidc_peer_ops that it uses to send events to rdma?

And here we are returning to square one of intended usage of aux bus.
Your driver should be structured to have PCI core logic that will represent
physical device and many small sub-devices with their respective drivers.

ETH is another sub-device that shouldn't talk directly to the RDMA.

Thanks

>
> Shiraz
>
>