qla2xxx: automatically rescan removed luns.

qla2xxx: automatically rescan removed luns.

[Benjamin ESTRABAUD]

Hi,

Using the qla2xxx driver from Linux 3.10.1 (release), if a LUN from the 
target side (multiple drives exported over a LIO IBlock qla2xxx export) 
is removed at the LIO level, the initiator side will not automatically 
detect that this LUN was removed.

All commands to this LUN will fail (sg_inq, read, writes) but the 
qla2xxx will never kick the drive out.

If another drive was to be exported on the target side using the same 
lun as the previously removed drive, commands sent to the same block id 
would now work again but be sent to the new LUN.

I tried to echo "- - -" in the scsi_host sysfs "scan" file corresponding 
to the FC port. This worked to detect new LUNs but not to "clear" 
removed ones.

I tried issuing a lip_reset on both sides, but no change.

Looking through Google, I realized that, on RedHat, a /proc/scsi/qla2xxx 
virtual FS exists which allows to send a "scsi-qlascan" command to the 
driver that apparently can detect removed LUNs. Unfortunately I don't 
have this FS visible on my Linux kernel, and I made sure /proc/scsi was 
enabled.

The only way I found was to "echo 1 > /sys/block/sdX/device/delete" to 
the stale device (that corresponds to the removed LUN from the target 
side), then replace the removed LUN on the target side before finally 
issuing a "scan" command on the initiator side.

Am I missing something here? Is there a way to rescan a Qlogic FC port 
using the "qla2xxx" driver from the official, stock kernel?

Thank you very much in advance for your help!

Regards,

Ben - MPSTOR

Re: qla2xxx: automatically rescan removed luns.

[Ewan Milne]

On Mon, 2013-11-25 at 15:37 +0000, Benjamin ESTRABAUD wrote:
> Hi,
> 
> Using the qla2xxx driver from Linux 3.10.1 (release), if a LUN from the 
> target side (multiple drives exported over a LIO IBlock qla2xxx export) 
> is removed at the LIO level, the initiator side will not automatically 
> detect that this LUN was removed.
> 
> All commands to this LUN will fail (sg_inq, read, writes) but the 
> qla2xxx will never kick the drive out.
> 
> If another drive was to be exported on the target side using the same 
> lun as the previously removed drive, commands sent to the same block id 
> would now work again but be sent to the new LUN.
> 
> I tried to echo "- - -" in the scsi_host sysfs "scan" file corresponding 
> to the FC port. This worked to detect new LUNs but not to "clear" 
> removed ones.
> 
> I tried issuing a lip_reset on both sides, but no change.
> 
> Looking through Google, I realized that, on RedHat, a /proc/scsi/qla2xxx 
> virtual FS exists which allows to send a "scsi-qlascan" command to the 
> driver that apparently can detect removed LUNs. Unfortunately I don't 
> have this FS visible on my Linux kernel, and I made sure /proc/scsi was 
> enabled.
> 
> The only way I found was to "echo 1 > /sys/block/sdX/device/delete" to 
> the stale device (that corresponds to the removed LUN from the target 
> side), then replace the removed LUN on the target side before finally 
> issuing a "scan" command on the initiator side.
> 
> Am I missing something here? Is there a way to rescan a Qlogic FC port 
> using the "qla2xxx" driver from the official, stock kernel?

The kernel does not contain support for removing devices automatically
during a subsequent rescan.  You have to delete them individually.
You can, however, update the properties detected when scanning a device
(e.g. disk capacity) by scanning the individual device in sysfs.  For
SCSI devices, there is a "rescan" capability, e.g.

echo 1> /sys/devices/<pci device path>/host0/target0:0:0/0:0:0:0/rescan

*Very* recent kernels contain a udev notification mechanism when a SCSI
device reports a Unit Attention, so you can have udev rules like:

ACTION=="change", SUBSYSTEM=="scsi", \
ENV{SDEV_UA}=="INQUIRY_DATA_HAS_CHANGED", TEST=="rescan", \
ATTR{rescan}="x"

ACTION=="change", SUBSYSTEM=="scsi", \
ENV{SDEV_UA}=="CAPACITY_DATA_HAS_CHANGED", TEST=="rescan", \
ATTR{rescan}="x"

ACTION=="change", SUBSYSTEM=="scsi", \
ENV{SDEV_UA}=="THIN_PROVISIONING_SOFT_THRESHOLD_REACHED", \
TEST=="rescan", ATTR{rescan}="x"

ACTION=="change", SUBSYSTEM=="scsi", \
ENV{SDEV_UA}=="MODE_PARAMETERS_CHANGED", TEST=="rescan", \
ATTR{rescan}="x"

ACTION=="change", SUBSYSTEM=="scsi", \
ENV{SDEV_UA}=="REPORTED_LUNS_DATA_HAS_CHANGED", \
RUN+="<script to scan devices> $env{DEVPATH}"

...

A script to scan devices can use the devpath of the SCSI device
reporting the Unit Attention to perform the desired action.  If you
just want to scan the target port, something like this might work,
although it is a little clumsy...

#!/bin/sh -e

HOST_PATH1=`echo $1 | awk '{print substr($1,0,index($1,"/target")-1)}'`
HOST_PATH2="scsi_host/host"`echo $1 | awk
'{split(substr($1,index($1,"target")),a,"/"); split(a[2],b,":"); print
b[1]}'`"/scan"

SCAN_STRING=`echo $1 | awk '{split(substr($1,index($1,"target")),a,"/");
split(a[2],b,":"); print b[2],b[3],"-"}'`

echo $SCAN_STRING > "/sys/"$HOST_PATH1"/"$HOST_PATH2

Or, "rescan-scsi-bus.sh" from (I think) sg3_utils might do what you
want.  Beware of running shell scripts from udev rules, however, as
that doesn't scale well in large configurations.

-Ewan


> 
> Thank you very much in advance for your help!
> 
> Regards,
> 
> Ben - MPSTOR
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: qla2xxx: automatically rescan removed luns.

[Benjamin ESTRABAUD]

On 25/11/13 17:17, Ewan Milne wrote:
> On Mon, 2013-11-25 at 15:37 +0000, Benjamin ESTRABAUD wrote:
>> Hi,
>>
>> Using the qla2xxx driver from Linux 3.10.1 (release), if a LUN from the
>> target side (multiple drives exported over a LIO IBlock qla2xxx export)
>> is removed at the LIO level, the initiator side will not automatically
>> detect that this LUN was removed.
>>
>> All commands to this LUN will fail (sg_inq, read, writes) but the
>> qla2xxx will never kick the drive out.
>>
>> If another drive was to be exported on the target side using the same
>> lun as the previously removed drive, commands sent to the same block id
>> would now work again but be sent to the new LUN.
>>
>> I tried to echo "- - -" in the scsi_host sysfs "scan" file corresponding
>> to the FC port. This worked to detect new LUNs but not to "clear"
>> removed ones.
>>
>> I tried issuing a lip_reset on both sides, but no change.
>>
>> Looking through Google, I realized that, on RedHat, a /proc/scsi/qla2xxx
>> virtual FS exists which allows to send a "scsi-qlascan" command to the
>> driver that apparently can detect removed LUNs. Unfortunately I don't
>> have this FS visible on my Linux kernel, and I made sure /proc/scsi was
>> enabled.
>>
>> The only way I found was to "echo 1 > /sys/block/sdX/device/delete" to
>> the stale device (that corresponds to the removed LUN from the target
>> side), then replace the removed LUN on the target side before finally
>> issuing a "scan" command on the initiator side.
>>
>> Am I missing something here? Is there a way to rescan a Qlogic FC port
>> using the "qla2xxx" driver from the official, stock kernel?
>
Hi,

Thank you very much for your reply, this is much appreciated.

> The kernel does not contain support for removing devices automatically
> during a subsequent rescan.

I thought that it used to with older kernels, is that possible (was this 
a recent change of behaviour)? It does, however, remove devices which 
port or connection has gone offline (pulled SAS disk, FC target 
inaccessible because of a disconnected cable).

Looking into it, the 'echo "- - -" > /sys/class/scsi_host/hostX/scan' 
scan command does issue a request_lun command (at least in SAS, used a 
target on which I could trace incoming commands).

When ran and a new LUN is detected, Linux does the right thing and adds 
the device.

When ran and a LUN is gone, it does nothing, as you explained above.


   You have to delete them individually.
Noted.

> You can, however, update the properties detected when scanning a device
> (e.g. disk capacity) by scanning the individual device in sysfs.  For
> SCSI devices, there is a "rescan" capability, e.g.
>
> echo 1> /sys/devices/<pci device path>/host0/target0:0:0/0:0:0:0/rescan
>
I used that rescan feature that I didn't know about and it's *very* 
useful, thank you!!

> *Very* recent kernels contain a udev notification mechanism when a SCSI
> device reports a Unit Attention, so you can have udev rules like:
>
> ACTION=="change", SUBSYSTEM=="scsi", \
> ENV{SDEV_UA}=="INQUIRY_DATA_HAS_CHANGED", TEST=="rescan", \
> ATTR{rescan}="x"
>
> ACTION=="change", SUBSYSTEM=="scsi", \
> ENV{SDEV_UA}=="CAPACITY_DATA_HAS_CHANGED", TEST=="rescan", \
> ATTR{rescan}="x"
>
> ACTION=="change", SUBSYSTEM=="scsi", \
> ENV{SDEV_UA}=="THIN_PROVISIONING_SOFT_THRESHOLD_REACHED", \
> TEST=="rescan", ATTR{rescan}="x"
>
> ACTION=="change", SUBSYSTEM=="scsi", \
> ENV{SDEV_UA}=="MODE_PARAMETERS_CHANGED", TEST=="rescan", \
> ATTR{rescan}="x"
>
> ACTION=="change", SUBSYSTEM=="scsi", \
> ENV{SDEV_UA}=="REPORTED_LUNS_DATA_HAS_CHANGED", \
> RUN+="<script to scan devices> $env{DEVPATH}"
>
> ...
>
That sounded promising, unfortunately the version we use (3.10.y branch) 
doesn't have this yet.

> A script to scan devices can use the devpath of the SCSI device
> reporting the Unit Attention to perform the desired action.  If you
> just want to scan the target port, something like this might work,
> although it is a little clumsy...
>
Thank you very much for that script, that gives me a pretty good idea of 
what needs to be done here.

> #!/bin/sh -e
>
> HOST_PATH1=`echo $1 | awk '{print substr($1,0,index($1,"/target")-1)}'`
> HOST_PATH2="scsi_host/host"`echo $1 | awk
> '{split(substr($1,index($1,"target")),a,"/"); split(a[2],b,":"); print
> b[1]}'`"/scan"
>
> SCAN_STRING=`echo $1 | awk '{split(substr($1,index($1,"target")),a,"/");
> split(a[2],b,":"); print b[2],b[3],"-"}'`
>
> echo $SCAN_STRING > "/sys/"$HOST_PATH1"/"$HOST_PATH2
>
> Or, "rescan-scsi-bus.sh" from (I think) sg3_utils might do what you
> want.  Beware of running shell scripts from udev rules, however, as
> that doesn't scale well in large configurations.
>
"rescan-scsi-bus.sh" did detect new LUN, but apparently not removed 
ones. However I need to test it on a system with a compatible bash shell 
as I wasn't able to run the script without errors.

Thanks a lot again for all your help.

> -Ewan
>
>
Regards,
Ben.
>>
>> Thank you very much in advance for your help!
>>
>> Regards,
>>
>> Ben - MPSTOR
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>

Re: qla2xxx: automatically rescan removed luns.

[Douglas Gilbert]

On 13-11-26 11:06 AM, Benjamin ESTRABAUD wrote:
> On 25/11/13 17:17, Ewan Milne wrote:
>> On Mon, 2013-11-25 at 15:37 +0000, Benjamin ESTRABAUD wrote:
>>> Hi,
>>>
>>> Using the qla2xxx driver from Linux 3.10.1 (release), if a LUN from the
>>> target side (multiple drives exported over a LIO IBlock qla2xxx export)
>>> is removed at the LIO level, the initiator side will not automatically
>>> detect that this LUN was removed.
>>>
>>> All commands to this LUN will fail (sg_inq, read, writes) but the
>>> qla2xxx will never kick the drive out.
>>>
>>> If another drive was to be exported on the target side using the same
>>> lun as the previously removed drive, commands sent to the same block id
>>> would now work again but be sent to the new LUN.
>>>
>>> I tried to echo "- - -" in the scsi_host sysfs "scan" file corresponding
>>> to the FC port. This worked to detect new LUNs but not to "clear"
>>> removed ones.
>>>
>>> I tried issuing a lip_reset on both sides, but no change.
>>>
>>> Looking through Google, I realized that, on RedHat, a /proc/scsi/qla2xxx
>>> virtual FS exists which allows to send a "scsi-qlascan" command to the
>>> driver that apparently can detect removed LUNs. Unfortunately I don't
>>> have this FS visible on my Linux kernel, and I made sure /proc/scsi was
>>> enabled.
>>>
>>> The only way I found was to "echo 1 > /sys/block/sdX/device/delete" to
>>> the stale device (that corresponds to the removed LUN from the target
>>> side), then replace the removed LUN on the target side before finally
>>> issuing a "scan" command on the initiator side.
>>>
>>> Am I missing something here? Is there a way to rescan a Qlogic FC port
>>> using the "qla2xxx" driver from the official, stock kernel?
>>
> Hi,
>
> Thank you very much for your reply, this is much appreciated.
>
>> The kernel does not contain support for removing devices automatically
>> during a subsequent rescan.
>
> I thought that it used to with older kernels, is that possible (was this a
> recent change of behaviour)? It does, however, remove devices which port or
> connection has gone offline (pulled SAS disk, FC target inaccessible because of
> a disconnected cable).
>
> Looking into it, the 'echo "- - -" > /sys/class/scsi_host/hostX/scan' scan
> command does issue a request_lun command (at least in SAS, used a target on
> which I could trace incoming commands).
>
> When ran and a new LUN is detected, Linux does the right thing and adds the device.
>
> When ran and a LUN is gone, it does nothing, as you explained above.
>
>
>    You have to delete them individually.
> Noted.
>
>> You can, however, update the properties detected when scanning a device
>> (e.g. disk capacity) by scanning the individual device in sysfs.  For
>> SCSI devices, there is a "rescan" capability, e.g.
>>
>> echo 1> /sys/devices/<pci device path>/host0/target0:0:0/0:0:0:0/rescan
>>
> I used that rescan feature that I didn't know about and it's *very* useful,
> thank you!!
>
>> *Very* recent kernels contain a udev notification mechanism when a SCSI
>> device reports a Unit Attention, so you can have udev rules like:
>>
>> ACTION=="change", SUBSYSTEM=="scsi", \
>> ENV{SDEV_UA}=="INQUIRY_DATA_HAS_CHANGED", TEST=="rescan", \
>> ATTR{rescan}="x"
>>
>> ACTION=="change", SUBSYSTEM=="scsi", \
>> ENV{SDEV_UA}=="CAPACITY_DATA_HAS_CHANGED", TEST=="rescan", \
>> ATTR{rescan}="x"
>>
>> ACTION=="change", SUBSYSTEM=="scsi", \
>> ENV{SDEV_UA}=="THIN_PROVISIONING_SOFT_THRESHOLD_REACHED", \
>> TEST=="rescan", ATTR{rescan}="x"
>>
>> ACTION=="change", SUBSYSTEM=="scsi", \
>> ENV{SDEV_UA}=="MODE_PARAMETERS_CHANGED", TEST=="rescan", \
>> ATTR{rescan}="x"
>>
>> ACTION=="change", SUBSYSTEM=="scsi", \
>> ENV{SDEV_UA}=="REPORTED_LUNS_DATA_HAS_CHANGED", \
>> RUN+="<script to scan devices> $env{DEVPATH}"
>>
>> ...
>>
> That sounded promising, unfortunately the version we use (3.10.y branch) doesn't
> have this yet.
>
>> A script to scan devices can use the devpath of the SCSI device
>> reporting the Unit Attention to perform the desired action.  If you
>> just want to scan the target port, something like this might work,
>> although it is a little clumsy...
>>
> Thank you very much for that script, that gives me a pretty good idea of what
> needs to be done here.
>
>> #!/bin/sh -e
>>
>> HOST_PATH1=`echo $1 | awk '{print substr($1,0,index($1,"/target")-1)}'`
>> HOST_PATH2="scsi_host/host"`echo $1 | awk
>> '{split(substr($1,index($1,"target")),a,"/"); split(a[2],b,":"); print
>> b[1]}'`"/scan"
>>
>> SCAN_STRING=`echo $1 | awk '{split(substr($1,index($1,"target")),a,"/");
>> split(a[2],b,":"); print b[2],b[3],"-"}'`
>>
>> echo $SCAN_STRING > "/sys/"$HOST_PATH1"/"$HOST_PATH2
>>
>> Or, "rescan-scsi-bus.sh" from (I think) sg3_utils might do what you
>> want.  Beware of running shell scripts from udev rules, however, as
>> that doesn't scale well in large configurations.
>>
> "rescan-scsi-bus.sh" did detect new LUN, but apparently not removed ones.
> However I need to test it on a system with a compatible bash shell as I wasn't
> able to run the script without errors.

Did you try the rescan-scsi-bus.sh from sg3_utils v 1.37 or
earlier? The reason I ask is that a fair amount of work
was done on the rescan-scsi-bus.sh found in version 1.37
including syncing with Kurt Garloff's version 1.57 plus
patches from Hannes Reinecke and Sean Stewart.

Doug Gilbert

>> -Ewan
>>
>>
> Regards,
> Ben.
>>>
>>> Thank you very much in advance for your help!
>>>
>>> Regards,
>>>
>>> Ben - MPSTOR
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: qla2xxx: automatically rescan removed luns.

[Hannes Reinecke]

On 11/26/2013 05:26 PM, Douglas Gilbert wrote:
> On 13-11-26 11:06 AM, Benjamin ESTRABAUD wrote:
[ .. ]
>> "rescan-scsi-bus.sh" did detect new LUN, but apparently not
>> removed ones.
>> However I need to test it on a system with a compatible bash shell
>> as I wasn't
>> able to run the script without errors.
> 
> Did you try the rescan-scsi-bus.sh from sg3_utils v 1.37 or
> earlier? The reason I ask is that a fair amount of work
> was done on the rescan-scsi-bus.sh found in version 1.37
> including syncing with Kurt Garloff's version 1.57 plus
> patches from Hannes Reinecke and Sean Stewart.
> 
Plus you need to call it with '-r', otherwise it won't remove any
stale LUNs. I'm sure it's documented somewhere ...

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: qla2xxx: automatically rescan removed luns.

[Benjamin ESTRABAUD]

On 28/11/13 13:29, Hannes Reinecke wrote:
> On 11/26/2013 05:26 PM, Douglas Gilbert wrote:
>> On 13-11-26 11:06 AM, Benjamin ESTRABAUD wrote:
> [ .. ]
>>> "rescan-scsi-bus.sh" did detect new LUN, but apparently not
>>> removed ones.
>>> However I need to test it on a system with a compatible bash shell
>>> as I wasn't
>>> able to run the script without errors.
>>
>> Did you try the rescan-scsi-bus.sh from sg3_utils v 1.37 or
>> earlier? The reason I ask is that a fair amount of work
>> was done on the rescan-scsi-bus.sh found in version 1.37
>> including syncing with Kurt Garloff's version 1.57 plus
>> patches from Hannes Reinecke and Sean Stewart.
>>
> Plus you need to call it with '-r', otherwise it won't remove any
> stale LUNs. I'm sure it's documented somewhere ...
>
Hi Doug and Hannes,

I did indeed try but I was using an older version and without specifying 
"-r" so it didn't remove anything.

I looked at the latest version's "remove" code (at around line 445 from 
rescan-scsi-bus.sh) and it seems to delete disks using the scsi_device 
sysfs "delete" function (which is the function I'm using right now).

I was unable to tell how the script detects whether a drive is gone or 
has been replaced with another backend storage at the same LUN, which in 
fact turns out to be the thing I'm more interested in (since I can now 
delete stale luns).

Thanks in advance for your help!

> Cheers,
>
> Hannes
>

Regards,
Ben.