Re: [PATCH] aic94xx: make use of the new sas_port

[Douglas Gilbert <dougg@torque.net>]

[-- Attachment #1: Type: text/plain, Size: 3112 bytes --]

Douglas Gilbert wrote:
> Alexis Bruemmer wrote:
> 
>>These patches look good and are running on our x260 and the ppc64
>>(power).  I was just wondering about a few things under the sys/class/:
>>
>>there is a sas_device, sas_end_device, and a sas_expander dir.
>>sas_device contains the same info as sas_end_device, and a sas_expander
>>can we eliminate one or the other?  Or was this a design choice?
>>
>>Also during the boot process on the x260 with an expander we see these
>>printk's:
>>sas: phy[n] matched wide port0
>>If we are going to print this then we better be sure that this is truly
>>a wide port.  It seems under the sys/class/ we have both a single
>>port0:0 and then also port-0:0:[n]-- a port[n] for each phy[n].  and
>>although each phy seems to have the same sas_address the end_device
>>attached to that phy have unique addresses.  My understanding is that a
>>wide port has multiple phys with the same sas address and are attached
>>to one end device.
> 
> 
> A wide port is an abstraction just above the link
> layer in which two or more links running between
> the same two devices have phys with the same SAS
> addresses associated with them.
> 
> Assume we have two phys on a HBA connected to two
> phys on an expander, then two other phys on the
> expander connected to a dual ported SAS disk.
> That is one wide port connection (between the
> at HBA and the expander) and two narrow
> port connections (between the expander and each
> port on the SAS disk).
> 
> This comes about because phys on a HBA share a
> SAS address, phys on a expander share a SAS address
> but phys of a dual ported SAS disk have distinct
> SAS addresses.
> [Could someone suggest an instructive diagram url
> of this as the sas drafts are a bit complex.]
> 
> You are probably aware of this distinction but my
> guess is a lot of other people aren't.
> 
> Put another way, seen from a LLD or discover process when
> multiple phys on the same device have this tuple
> <SAS_address, attached_SAS_address> equal then there is
> a wide port connection.

I don't have a diagram, but I do have some real
output. The hardware is set up as in my example
above with a LSI SAS HBA (0x500605b00006f260).
There is a twelve port expander (0x500605b000000af0)
with phy ids 3 and 4 connected to the HBA, and phy
ids 8 and 9 connected to a dual ported SAS disk.

sg_vpd shows the naa-5 addresses for /dev/sdb
and /dev/sdc . So many to choose from! The first
is the lu name, next is the port identifier and the
last two are different renderings of the target device
name. Only the target port identifiers differ (the
secondary port is usually one higher than the primary).

[Aside: if lsscsi was to show a device's wwn then I
 believe it should show the lu name (if there is one).
 Thoughts?]

Doing a SMP discover on the expander's connected phys
shows a 2 link wide port to the HBA because the
two <sas_addres, attached_sas_address> tuples match. Note
that the attached phy ids (i.e. on the HBA) are not
consecutive. There are also two narrow port links to the
primary and secondary ports of the SAS disk.

Doug Gilbert



[-- Attachment #2: wide_narrow_ex.txt --]
[-- Type: text/plain, Size: 2057 bytes --]

[root@sas ~]# lsscsi
[0:0:0:0]    disk    ATA      WDC WD360GD-00FL 33.0  /dev/sda
[4:2:0:0]    disk    SEAGATE  ST336754SS       0003  /dev/sdb
[4:2:1:0]    disk    SEAGATE  ST336754SS       0003  /dev/sdc

[root@sas ~]# sg_vpd /dev/sdb -iq
0x5000c50000520a2b
0x5000c50000520a2a
0x5000c50000520a28
naa.5000C50000520A28

[root@sas ~]# sg_vpd /dev/sdc -iq
0x5000c50000520a2b
0x5000c50000520a29
0x5000c50000520a28
naa.5000C50000520A28

[root@sas ~]# smp_discover -s 0x500605b000000af0 /dev/mptctl -b -p 3
Discover response:
  phy identifier: 3
  attached device type: end device
  negotiated physical link rate: phy enabled; 3 Gbps
  attached initiator: ssp=1 stp=1 smp=1 sata_host=0
  attached target: ssp=0 stp=0 smp=0 sata_device=0
  SAS address: 0x500605b000000af0
  attached SAS address: 0x500605b00006f260
  attached phy identifier: 0

[root@sas ~]# smp_discover -s 0x500605b000000af0 /dev/mptctl -b -p 4
Discover response:
  phy identifier: 4
  attached device type: end device
  negotiated physical link rate: phy enabled; 3 Gbps
  attached initiator: ssp=1 stp=1 smp=1 sata_host=0
  attached target: ssp=0 stp=0 smp=0 sata_device=0
  SAS address: 0x500605b000000af0
  attached SAS address: 0x500605b00006f260
  attached phy identifier: 3

[root@sas ~]# smp_discover -s 0x500605b000000af0 /dev/mptctl -b -p 8
Discover response:
  phy identifier: 8
  attached device type: end device
  negotiated physical link rate: phy enabled; 3 Gbps
  attached initiator: ssp=0 stp=0 smp=0 sata_host=0
  attached target: ssp=1 stp=0 smp=0 sata_device=0
  SAS address: 0x500605b000000af0
  attached SAS address: 0x5000c50000520a2a
  attached phy identifier: 1

[root@sas ~]# smp_discover -s 0x500605b000000af0 /dev/mptctl -b -p 9
Discover response:
  phy identifier: 9
  attached device type: end device
  negotiated physical link rate: phy enabled; 3 Gbps
  attached initiator: ssp=0 stp=0 smp=0 sata_host=0
  attached target: ssp=1 stp=0 smp=0 sata_device=0
  SAS address: 0x500605b000000af0
  attached SAS address: 0x5000c50000520a29
  attached phy identifier: 0