Re: [PATCH] [RFC] IB/hfi1: Fix port ordering issue in a multiport device

[Doug Ledford <dledford@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 2740 bytes --]

On Wed, 2017-01-11 at 11:10 -0700, Jason Gunthorpe wrote:
> On Tue, Jan 10, 2017 at 03:57:52PM -0800, Tadeusz Struk wrote:
> > 
> > We can fix this by enforcing the correct port order at module load
> > time.
> > To reorder the ports to match the numbering labels on the back of
> > the
> > device we need to delay registering devices with the ib_core until
> > we
> 
> Sorry, no way - this is horrifying.

Agree.

> If you need stable names for RDMA devices then you need to add proper
> infrastructure to the kernel to rename RDMA devices from user space
> via udev. ala netdev.

This has its own problems in this particular case, namely having to
ship files that know which parts are effected and then modify the
names.  Or requiring that users create all of the rename rules when
they really shouldn't be bothered with anything.  Although, the module
option to turn this fix off for existing clusters that have been wired
up wrong is just as bad as creating persistent naming rules...

> or change the ib_core to allow your driver to specify the full name
> and manage things in your driver.
> 
> No way on this insane block probe approach.

Isn't there already code in the code device layer to handle this kind
of thing?  I seem to recall backporting it from upstream to a rhel
kernel many years ago.  Lemme go look...

OK, sure, as far as the reordering stuff is concerned, all you need to
do is to make use of the EPROBE_DEFER return option to your PCI probe
routine.  That way, when you get the probe for the out of order port,
the first time you pass EPROBE_DEFER as your return, then you get the
second port, your register it with the IB layer which will make it
appear as the first port (optionally, and as I haven't tried this I
don't know if it will work or if it's necessary, you can save the
pointer to the first port's device struct off, and when you get the
second one, you can tell the driver layer to splice the second port in
front of the first port in the device child list on the parent device,
but I think this is optional and really has no lasting effect on the
outcome), then later the first port gets retried and ends up being the
second port.

So, scrap all of this hand done junk and use the provided
infrastructure as it was intended to be used.  I *really* don't want to
do a kernel module option for this either.  Do you know for a fact that
this is wired up wrong in the field and the people can't just swap
hfi1_0<->hfi1_1 in their config files and have it all work without
being recabled?

-- 
Doug Ledford <dledford@redhat.com>
    GPG KeyID: B826A3330E572FDD
   
Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]