Re: Handling multiple paths to enclosure devices?

[James Bottomley <James.Bottomley@HansenPartnership.com>]

On Fri, 2011-07-29 at 09:21 +0200, Hannes Reinecke wrote:
> On 07/29/2011 09:09 AM, James Bottomley wrote:
> > On Thu, 2011-07-28 at 13:57 -0700, Roland Dreier wrote:
> >>   >  My initial thought is that in a multi-path situation, as above, we get
> >>   >  two enclosures appearing as well (one down each path).  If we
> >>   >  incorporated the idea of topological subtrees into the identity matching
> >>   >  code, we'd end up filling each of the enclosures with the path connected
> >>   >  devices.  That seems to be an easy situation for multi-path drivers to
> >>   >  sort out and one requiring no alteration of the existing enclosure code
> >>   >  (except to do the topological subtree search).
> >>
> >> Ah, good insight: we do have two copies of the expander device, and so
> >> it would be natural to attach each disk to the expander device on the
> >> same path to that disk.
> >>
> >> I'm a bit confused by what you mean about multi-path drivers though --
> >> it would seem we would need the enclosure stuff to handle this
> >> somehow?  It seems that if I have this situation, my HBA driver (eg
> >> mpt2sas) will discover the SCSI bus, hit an enclosure, trigger loading
> >> the ses driver (which pulls in the enclosure driver), and then
> >> continue discovering disks.  And it seems this code needs to get the
> >> topology sorted by itself -- how could a multi-path driver inject
> >> itself into the symlink creation in enclosure.c?
> >
> > I merely meant that our current philosophy is to layer multi-path
> > awareness in a separate driver: dm.  There's certainly no expander
> > awareness in dm-mp at the moment, but there could be.  It should be
> > quite simple:  match the expanders to something like a dm-exp and then
> > basically use the slot information in the expander to construct dm
> > devices for each of the physicals (the rule for dm device creation would
> > be dm-exp slot matching).
> >
> Doesn't help as device-mapper is only concerned with block devices,
> what with dm devices being essentially abstract block devices.

Well, all SCSI devices are effectively block devices as far as DM sees
it because they have request queues.  It's exploiting a sick trick, I
know, but I don't think it will be too difficult to express them in the
current code.

> But yes, we should stick to our current philosophy of treating 
> multipath devices as separate trees underneath the accessing HBA.
> We should not (and, in fact, cannot) try to map the external 
> topology in sysfs, rather should stick to a logical view as seen 
> from each HBA. So the enclosure class needs to be updated
> int include the HBA number in there to avoid duplicate links.
> 
> The correct mapping and accessing these 'multipathed enclosures'
> will then be left as an exercise to the reader :-)

Agreed.  I'm agnostic as to whether we trick dm or invent something new.

James