SCSI device rescan, detection of disconnected device, or switched devices.

SCSI device rescan, detection of disconnected device, or switched devices.

[Gal Rosen]

Hello all,

I know that when adding new device there is no problem;
by echo "- - -" >/sys/class/scsi_host/hostX/scan the SCSI subsystem will
recognize the new device and new /dev/sgX device will be created, but if
someone remove a device, scan will only recognize that the report luns
has changed, but it will not remove the /dev/sgX device.
If now someone add new device it will mapped to the /dev/sgX that
previously mapped to the device that just removed.

1. Why the SCSI subsystem doesn't release devices that removed ?

2. In the situation that I described above someone can switched devices
without notifying the application that use those devices. The
notification will come only when the next SCSI command will return with
unit attention saying "Power on, reset, or bus device reset occured", or
if device just removed without adding new device it will return
"Reported luns data has changed".
If I have an application that control SCSI devices using sg driver and I
would like to have the ability to change configuration online, what is
the preferred way to rescan the bus and update the application that sgX
that previously controls device Y is now controlling device Z ?
In other words, what is the best way for the application to identify
that device has been removed or changed ?

3. I am using Qlogic firmware ability to create virtual ports, and I
notice that on disconnect and then reconnect the FC cable, the sg
mapping can changed. If on module load the physical port got sg0, and on
creating vport it got sg1, now the SCSI subsystem scans the vport first
and mapped it to sg0 and the physical port gets sg1.
Is there a way to control the mapping (scanning) ?

Best regards,
Gal Rosen
Senior Software Engineer
www.storwize.com

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SCSI device rescan, detection of disconnected device, or switched devices.

[Stefan Richter]

Gal Rosen wrote:
> I know that when adding new device there is no problem;
> by echo "- - -" >/sys/class/scsi_host/hostX/scan the SCSI subsystem will
> recognize the new device and new /dev/sgX device will be created, but if
> someone remove a device, scan will only recognize that the report luns
> has changed, but it will not remove the /dev/sgX device.
> If now someone add new device it will mapped to the /dev/sgX that
> previously mapped to the device that just removed.
> 
> 1. Why the SCSI subsystem doesn't release devices that removed ?

Because somewhere someone still holds a reference on the device (for a
good reason, or due to a bug).

> 2. In the situation that I described above someone can switched devices
> without notifying the application that use those devices. The
> notification will come only when the next SCSI command will return with
> unit attention saying "Power on, reset, or bus device reset occured", or
> if device just removed without adding new device it will return
> "Reported luns data has changed".
> If I have an application that control SCSI devices using sg driver and I
> would like to have the ability to change configuration online, what is
> the preferred way to rescan the bus and update the application that sgX
> that previously controls device Y is now controlling device Z ?
> In other words, what is the best way for the application to identify
> that device has been removed or changed ?

Hmm, I too am curious what the answer to this would be.

> 3. I am using Qlogic firmware ability to create virtual ports, and I
> notice that on disconnect and then reconnect the FC cable, the sg
> mapping can changed. If on module load the physical port got sg0, and on
> creating vport it got sg1, now the SCSI subsystem scans the vport first
> and mapped it to sg0 and the physical port gets sg1.
> Is there a way to control the mapping (scanning) ?

There is currently one canonical way:  The low-level userspace (udev,
hal, or even your application) looks at the device representation in
sysfs and fetches persistent and unique properties of the target and
logical unit from there.  The target identifier and logical unit
identifier come first to mind as suitable properties.  Then the
low-level userspace can for example create symlinks.  Most distributions
contain udev scripts which do this for sd devices; see /dev/disk/by-id/.

Alas there is no standard place where to look for target identifier and
logical unit identifier.  At least at the last time when I looked for
them, the paths to respective sysfs attributes are so far implementation
details of the various transport-layer drivers or so-called transport
classes (FC, SAS, iSCSI, SBP-2...).
-- 
Stefan Richter
-=====-==--- -=== ===--
http://arcgraph.de/sr/
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SCSI device rescan, detection of disconnected device, or switched devices.

[Gal Rosen]

Hi,

I will try to make my question more clear.
I would like to put a box between the client and the storage, and to be
fully transparent against the host. All changes on the storage
(add/remove/extend devices) should be present automatically to the
client. In a direct connection between client and storage the
administrator will coordinate between the client and storage before
making any changes, in my configuration I do not want to add
administration work on my box that stands between the client and the
storage.
Addition is simple, the application on my box can once in a while scan
the SCSI BUS, and then the Linux system will create new sg devices.
In removal of a device the Linux system will not remove sg devices after
scan. In that case again I can do it from my application, such when I
get attention on SCSI command, I can decide to remove the device by
echoing to the /sys file system.

What do you think ? is this enough or there is better way ?

Thanks,
Gal.

On Mon, 2008-07-28 at 15:10 +0200, Stefan Richter wrote:
> Gal Rosen wrote:
> > I know that when adding new device there is no problem;
> > by echo "- - -" >/sys/class/scsi_host/hostX/scan the SCSI subsystem will
> > recognize the new device and new /dev/sgX device will be created, but if
> > someone remove a device, scan will only recognize that the report luns
> > has changed, but it will not remove the /dev/sgX device.
> > If now someone add new device it will mapped to the /dev/sgX that
> > previously mapped to the device that just removed.
> > 
> > 1. Why the SCSI subsystem doesn't release devices that removed ?
> 
> Because somewhere someone still holds a reference on the device (for a
> good reason, or due to a bug).
> 
> > 2. In the situation that I described above someone can switched devices
> > without notifying the application that use those devices. The
> > notification will come only when the next SCSI command will return with
> > unit attention saying "Power on, reset, or bus device reset occured", or
> > if device just removed without adding new device it will return
> > "Reported luns data has changed".
> > If I have an application that control SCSI devices using sg driver and I
> > would like to have the ability to change configuration online, what is
> > the preferred way to rescan the bus and update the application that sgX
> > that previously controls device Y is now controlling device Z ?
> > In other words, what is the best way for the application to identify
> > that device has been removed or changed ?
> 
> Hmm, I too am curious what the answer to this would be.
> 
> > 3. I am using Qlogic firmware ability to create virtual ports, and I
> > notice that on disconnect and then reconnect the FC cable, the sg
> > mapping can changed. If on module load the physical port got sg0, and on
> > creating vport it got sg1, now the SCSI subsystem scans the vport first
> > and mapped it to sg0 and the physical port gets sg1.
> > Is there a way to control the mapping (scanning) ?
> 
> There is currently one canonical way:  The low-level userspace (udev,
> hal, or even your application) looks at the device representation in
> sysfs and fetches persistent and unique properties of the target and
> logical unit from there.  The target identifier and logical unit
> identifier come first to mind as suitable properties.  Then the
> low-level userspace can for example create symlinks.  Most distributions
> contain udev scripts which do this for sd devices; see /dev/disk/by-id/.
> 
> Alas there is no standard place where to look for target identifier and
> logical unit identifier.  At least at the last time when I looked for
> them, the paths to respective sysfs attributes are so far implementation
> details of the various transport-layer drivers or so-called transport
> classes (FC, SAS, iSCSI, SBP-2...).
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SCSI device rescan, detection of disconnected device, or switched devices.

[Stefan Richter]

Gal Rosen wrote:
> I will try to make my question more clear.
> I would like to put a box between the client and the storage, and to be
> fully transparent against the host. All changes on the storage
> (add/remove/extend devices) should be present automatically to the
> client. In a direct connection between client and storage the
> administrator will coordinate between the client and storage before
> making any changes, in my configuration I do not want to add
> administration work on my box that stands between the client and the
> storage.
> Addition is simple, the application on my box can once in a while scan
> the SCSI BUS, and then the Linux system will create new sg devices.
> In removal of a device the Linux system will not remove sg devices after
> scan. In that case again I can do it from my application, such when I
> get attention on SCSI command, I can decide to remove the device by
> echoing to the /sys file system.

I really don't know how the FibreChannel drivers handle device removal.
But from what I gather from the documentation of
fc_remote_port_delete(), which the qla2xxx drivers use, the /dev/sg
devices should automatically disappear
  - after a connection loss timeout,
  - provided that userspace doesn't have the device file opened
    (among else this is manifest in the kernel in a reference count of
    the logical unit device representation),
  - provided that you manage your device files with udev.

http://lxr.linux.no/linux+v2.6.26/drivers/scsi/scsi_transport_fc.c#L2756

That would be basically just like SCSI device files are created and
destroyed on the fly for USB or FireWire devices for example.  Except
that FC has the additional configurable connection loss timeout which
hides temporary connection losses from applications.

(However, better wait for an answer of somebody who actually uses FC or
knows the drivers.  I only have experience with FireWire.)
-- 
Stefan Richter
-=====-==--- -=== =====
http://arcgraph.de/sr/

Re: SCSI device rescan, detection of disconnected device, or switched devices.

[Gal Rosen]

On Thu, 2008-07-31 at 10:34 +0200, Stefan Richter wrote:
> Gal Rosen wrote:
> > I will try to make my question more clear.
> > I would like to put a box between the client and the storage, and to be
> > fully transparent against the host. All changes on the storage
> > (add/remove/extend devices) should be present automatically to the
> > client. In a direct connection between client and storage the
> > administrator will coordinate between the client and storage before
> > making any changes, in my configuration I do not want to add
> > administration work on my box that stands between the client and the
> > storage.
> > Addition is simple, the application on my box can once in a while scan
> > the SCSI BUS, and then the Linux system will create new sg devices.
> > In removal of a device the Linux system will not remove sg devices after
> > scan. In that case again I can do it from my application, such when I
> > get attention on SCSI command, I can decide to remove the device by
> > echoing to the /sys file system.
> 
> I really don't know how the FibreChannel drivers handle device removal.
> But from what I gather from the documentation of
> fc_remote_port_delete(), which the qla2xxx drivers use, the /dev/sg
> devices should automatically disappear
>   - after a connection loss timeout,
>   - provided that userspace doesn't have the device file opened
>     (among else this is manifest in the kernel in a reference count of
>     the logical unit device representation),
>   - provided that you manage your device files with udev.
> 
> http://lxr.linux.no/linux+v2.6.26/drivers/scsi/scsi_transport_fc.c#L2756
> 

This is true; when the remote port is no longer exists, after timeout
all the sg devices will disappear, but what if I am changing the
configuration in the disk array, and remove only one logical device.
There will not be any event for that. The hotplug implementation in the
SCSI today creates events of add/remove when adding/removing HBA, or
when the FC link go down and up again, but not for additional and
removal of devices. In the linux documentation in scsi_mid_low_api.txt
it is written that "The hotplug concept may be extended to SCSI
devices". I guess that nobody implemented it because what you said
previously in this thread that someone could hold reference to the
device.
I think that I understand the concept. When the FC port is going down/up
you get interrupt, so you can remove all devices or initiate scsi scan
and add devices. But when device is added you can't get any event,
because this device is not known to the client yet (no body is working
with it), all you can do is to initiate scan every once a while. For
removing a device there is a possibility to do it automatically. The LLD
can identify that the device is not ready, and then call to the mid
layer to remove this device. It will succeed only if nobody is holding a
reference to that device.

Gal.
 
> That would be basically just like SCSI device files are created and
> destroyed on the fly for USB or FireWire devices for example.  Except
> that FC has the additional configurable connection loss timeout which
> hides temporary connection losses from applications.
> 
> (However, better wait for an answer of somebody who actually uses FC or
> knows the drivers.  I only have experience with FireWire.)

Re: SCSI device rescan, detection of disconnected device, or switched devices.

[James Bottomley]

On Thu, 2008-07-31 at 14:48 +0300, Gal Rosen wrote:
> On Thu, 2008-07-31 at 10:34 +0200, Stefan Richter wrote:
> > Gal Rosen wrote:
> > > I will try to make my question more clear.
> > > I would like to put a box between the client and the storage, and to be
> > > fully transparent against the host. All changes on the storage
> > > (add/remove/extend devices) should be present automatically to the
> > > client. In a direct connection between client and storage the
> > > administrator will coordinate between the client and storage before
> > > making any changes, in my configuration I do not want to add
> > > administration work on my box that stands between the client and the
> > > storage.
> > > Addition is simple, the application on my box can once in a while scan
> > > the SCSI BUS, and then the Linux system will create new sg devices.
> > > In removal of a device the Linux system will not remove sg devices after
> > > scan. In that case again I can do it from my application, such when I
> > > get attention on SCSI command, I can decide to remove the device by
> > > echoing to the /sys file system.
> > 
> > I really don't know how the FibreChannel drivers handle device removal.
> > But from what I gather from the documentation of
> > fc_remote_port_delete(), which the qla2xxx drivers use, the /dev/sg
> > devices should automatically disappear
> >   - after a connection loss timeout,
> >   - provided that userspace doesn't have the device file opened
> >     (among else this is manifest in the kernel in a reference count of
> >     the logical unit device representation),
> >   - provided that you manage your device files with udev.
> > 
> > http://lxr.linux.no/linux+v2.6.26/drivers/scsi/scsi_transport_fc.c#L2756
> > 
> 
> This is true; when the remote port is no longer exists, after timeout
> all the sg devices will disappear, but what if I am changing the
> configuration in the disk array, and remove only one logical device.
> There will not be any event for that. The hotplug implementation in the
> SCSI today creates events of add/remove when adding/removing HBA, or
> when the FC link go down and up again, but not for additional and
> removal of devices. In the linux documentation in scsi_mid_low_api.txt
> it is written that "The hotplug concept may be extended to SCSI
> devices". I guess that nobody implemented it because what you said
> previously in this thread that someone could hold reference to the
> device.

Oh ... you're not really talking about hotplug, which is why everyone is
confused.  Hotplug is when you add or remove something from the bus.
What you've done is reconfigure the array.

Most of the hotplug we do depends on the transport model (because what's
on the transport is changing).  Array reconfiguration has no hotplug
event because SAM-3 has no real way of passing the information
asynchronously.  The best it can do is Unit Attention/reported luns data
has changed (asc=0x3f/ascq=0xe) on the next command.

The problem is that there's no way to process the event correctly even
when we get it.  All we can do is issue another report LUNS command and
compare.  However, just because it looks like a single LUN disappeared
doesn't mean the others weren't permuted or altered in some way (which
data we cannot get).

So ... this event is manual intervention required because you have to
tell SCSI what you did to the array, it can't work it out for itself.

James

Re: SCSI device rescan, detection of disconnected device, or switched devices.

[Vladislav Bolkhovitin]

James Bottomley wrote:
> On Thu, 2008-07-31 at 14:48 +0300, Gal Rosen wrote:
>> On Thu, 2008-07-31 at 10:34 +0200, Stefan Richter wrote:
>>> Gal Rosen wrote:
>>>> I will try to make my question more clear.
>>>> I would like to put a box between the client and the storage, and to be
>>>> fully transparent against the host. All changes on the storage
>>>> (add/remove/extend devices) should be present automatically to the
>>>> client. In a direct connection between client and storage the
>>>> administrator will coordinate between the client and storage before
>>>> making any changes, in my configuration I do not want to add
>>>> administration work on my box that stands between the client and the
>>>> storage.
>>>> Addition is simple, the application on my box can once in a while scan
>>>> the SCSI BUS, and then the Linux system will create new sg devices.
>>>> In removal of a device the Linux system will not remove sg devices after
>>>> scan. In that case again I can do it from my application, such when I
>>>> get attention on SCSI command, I can decide to remove the device by
>>>> echoing to the /sys file system.
>>> I really don't know how the FibreChannel drivers handle device removal.
>>> But from what I gather from the documentation of
>>> fc_remote_port_delete(), which the qla2xxx drivers use, the /dev/sg
>>> devices should automatically disappear
>>>   - after a connection loss timeout,
>>>   - provided that userspace doesn't have the device file opened
>>>     (among else this is manifest in the kernel in a reference count of
>>>     the logical unit device representation),
>>>   - provided that you manage your device files with udev.
>>>
>>> http://lxr.linux.no/linux+v2.6.26/drivers/scsi/scsi_transport_fc.c#L2756
>>>
>> This is true; when the remote port is no longer exists, after timeout
>> all the sg devices will disappear, but what if I am changing the
>> configuration in the disk array, and remove only one logical device.
>> There will not be any event for that. The hotplug implementation in the
>> SCSI today creates events of add/remove when adding/removing HBA, or
>> when the FC link go down and up again, but not for additional and
>> removal of devices. In the linux documentation in scsi_mid_low_api.txt
>> it is written that "The hotplug concept may be extended to SCSI
>> devices". I guess that nobody implemented it because what you said
>> previously in this thread that someone could hold reference to the
>> device.
> 
> Oh ... you're not really talking about hotplug, which is why everyone is
> confused.  Hotplug is when you add or remove something from the bus.
> What you've done is reconfigure the array.
> 
> Most of the hotplug we do depends on the transport model (because what's
> on the transport is changing).  Array reconfiguration has no hotplug
> event because SAM-3 has no real way of passing the information
> asynchronously.  The best it can do is Unit Attention/reported luns data
> has changed (asc=0x3f/ascq=0xe) on the next command.
> 
> The problem is that there's no way to process the event correctly even
> when we get it.  All we can do is issue another report LUNS command and
> compare.  However, just because it looks like a single LUN disappeared
> doesn't mean the others weren't permuted or altered in some way (which
> data we cannot get).

But why don't do what's possible to do and for what there is complete 
information: add new LUN for new LUN and delete deleted LUN for reported 
luns data has changed UA? As well as get new capacity for a LU on 
capacity data has changed UA after, e.g., new space added to it?

Vlad

Re: SCSI device rescan, detection of disconnected device, or switched devices.

[Gal Rosen]

On Thu, 2008-07-31 at 09:15 -0500, James Bottomley wrote:
> Oh ... you're not really talking about hotplug, which is why everyone is
> confused.  Hotplug is when you add or remove something from the bus.
> What you've done is reconfigure the array.
> 
> Most of the hotplug we do depends on the transport model (because what's
> on the transport is changing).  Array reconfiguration has no hotplug
> event because SAM-3 has no real way of passing the information
> asynchronously.  The best it can do is Unit Attention/reported luns data
> has changed (asc=0x3f/ascq=0xe) on the next command.
> 
> The problem is that there's no way to process the event correctly even
> when we get it.  All we can do is issue another report LUNS command and
> compare.  However, just because it looks like a single LUN disappeared
> doesn't mean the others weren't permuted or altered in some way (which
> data we cannot get).
> 

Why? if I keep a mapping between sg devices and device id (Inquiry page
0x83) then I have all the data.

Gal.

> So ... this event is manual intervention required because you have to
> tell SCSI what you did to the array, it can't work it out for itself.
> 
> James
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SCSI device rescan, detection of disconnected device, or switched devices.

[James Bottomley]

On Thu, 2008-07-31 at 19:59 +0400, Vladislav Bolkhovitin wrote:
> James Bottomley wrote:
> > Most of the hotplug we do depends on the transport model (because what's
> > on the transport is changing).  Array reconfiguration has no hotplug
> > event because SAM-3 has no real way of passing the information
> > asynchronously.  The best it can do is Unit Attention/reported luns data
> > has changed (asc=0x3f/ascq=0xe) on the next command.
> > 
> > The problem is that there's no way to process the event correctly even
> > when we get it.  All we can do is issue another report LUNS command and
> > compare.  However, just because it looks like a single LUN disappeared
> > doesn't mean the others weren't permuted or altered in some way (which
> > data we cannot get).
> 
> But why don't do what's possible to do and for what there is complete 
> information: add new LUN for new LUN and delete deleted LUN for reported 
> luns data has changed UA? As well as get new capacity for a LU on 
> capacity data has changed UA after, e.g., new space added to it?

Well, feel free to submit a patch, but it's an incredible mine field.
We certainly can't do something that builds a trap for users (as in a
policy which gets it right 10% of the time and wrong 90% of the time) so
I'm strongly inclined to feel array reconfiguration is a manual process.
If you think you can do it online, we don't panic or throw a spanner in
the works, but we wait until you tell us what you did, but you'd be well
advised to do it offline.

The point is that we can't distinguish LUN added from LUN split or space
reapportioned or something ... even LUN renumbered would be seen as a
remove followed by an add.

James

Re: SCSI device rescan, detection of disconnected device, or switched devices.

[James Bottomley]

On Thu, 2008-07-31 at 19:09 +0300, Gal Rosen wrote:
> On Thu, 2008-07-31 at 09:15 -0500, James Bottomley wrote:
> > Oh ... you're not really talking about hotplug, which is why everyone is
> > confused.  Hotplug is when you add or remove something from the bus.
> > What you've done is reconfigure the array.
> > 
> > Most of the hotplug we do depends on the transport model (because what's
> > on the transport is changing).  Array reconfiguration has no hotplug
> > event because SAM-3 has no real way of passing the information
> > asynchronously.  The best it can do is Unit Attention/reported luns data
> > has changed (asc=0x3f/ascq=0xe) on the next command.
> > 
> > The problem is that there's no way to process the event correctly even
> > when we get it.  All we can do is issue another report LUNS command and
> > compare.  However, just because it looks like a single LUN disappeared
> > doesn't mean the others weren't permuted or altered in some way (which
> > data we cannot get).
> > 
> 
> Why? if I keep a mapping between sg devices and device id (Inquiry page
> 0x83) then I have all the data.

No we don't, it depends on array policy.  Most arrays when told to take
10% of space from 10 luns and create a new one won't change the WWN of
the existing 10 LUNS, so we have no idea what just happened or what we
should do about it (incidentally, this is an example of a
reconfiguration you probably wanted to do offline anyway).

James


--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: SCSI device rescan, detection of disconnected device, or switched devices.

[James.Smart]

Stefan,

You summed it up pretty well...

-- james s 

> -----Original Message-----
> From: linux-scsi-owner@vger.kernel.org 
> [mailto:linux-scsi-owner@vger.kernel.org] On Behalf Of Stefan Richter
> Sent: Thursday, July 31, 2008 4:35 AM
> To: Gal Rosen
> Cc: linux-scsi@vger.kernel.org
> Subject: Re: SCSI device rescan, detection of disconnected 
> device, or switched devices.
> 
> Gal Rosen wrote:
> > I will try to make my question more clear.
> > I would like to put a box between the client and the 
> storage, and to be
> > fully transparent against the host. All changes on the storage
> > (add/remove/extend devices) should be present automatically to the
> > client. In a direct connection between client and storage the
> > administrator will coordinate between the client and storage before
> > making any changes, in my configuration I do not want to add
> > administration work on my box that stands between the client and the
> > storage.
> > Addition is simple, the application on my box can once in a 
> while scan
> > the SCSI BUS, and then the Linux system will create new sg devices.
> > In removal of a device the Linux system will not remove sg 
> devices after
> > scan. In that case again I can do it from my application, 
> such when I
> > get attention on SCSI command, I can decide to remove the device by
> > echoing to the /sys file system.
> 
> I really don't know how the FibreChannel drivers handle 
> device removal.
> But from what I gather from the documentation of
> fc_remote_port_delete(), which the qla2xxx drivers use, the /dev/sg
> devices should automatically disappear
>   - after a connection loss timeout,
>   - provided that userspace doesn't have the device file opened
>     (among else this is manifest in the kernel in a reference count of
>     the logical unit device representation),
>   - provided that you manage your device files with udev.
> 
> http://lxr.linux.no/linux+v2.6.26/drivers/scsi/scsi_transport_
> fc.c#L2756
> 
> That would be basically just like SCSI device files are created and
> destroyed on the fly for USB or FireWire devices for example.  Except
> that FC has the additional configurable connection loss timeout which
> hides temporary connection losses from applications.
> 
> (However, better wait for an answer of somebody who actually 
> uses FC or
> knows the drivers.  I only have experience with FireWire.)
> -- 
> Stefan Richter
> -=====-==--- -=== =====
> http://arcgraph.de/sr/
> --
> To unsubscribe from this list: send the line "unsubscribe 
> linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

RE: SCSI device rescan, detection of disconnected device,or switched devices.

[James.Smart]

 

> -----Original Message-----
> This is true; when the remote port is no longer exists, after timeout
> all the sg devices will disappear, but what if I am changing the
> configuration in the disk array, and remove only one logical device.
> There will not be any event for that. The hotplug 
> implementation in the
> SCSI today creates events of add/remove when adding/removing HBA, or
> when the FC link go down and up again, but not for additional and
> removal of devices. In the linux documentation in scsi_mid_low_api.txt
> it is written that "The hotplug concept may be extended to SCSI
> devices". I guess that nobody implemented it because what you said
> previously in this thread that someone could hold reference to the
> device.
> I think that I understand the concept. When the FC port is 
> going down/up
> you get interrupt, so you can remove all devices or initiate scsi scan
> and add devices. 

True - Target-level add or remove usually generates transport events, 
which allow the transports to detect it.

SCSI-Lun-level add/remove doesn't. Well.. it could, as there is a check
condition code that can be generated to report configuration change, and
in some transports, there's Async Event Notification (AEN) -- BUT --
no one really does AEN, and not all arrays generate the Chk Condition,
and even if it did, our midlayer doesn't act on it.  And even if the 
midlayer did act on it, we would have to add device identifier
recognition
into the equation, which we don't do today, and leave it up to udev.
It's not a simple thing to add...

> But when device is added you can't get any event,
> because this device is not known to the client yet (no body is working
> with it), all you can do is to initiate scan every once a while. For
> removing a device there is a possibility to do it 
> automatically. The LLD
> can identify that the device is not ready, and then call to the mid
> layer to remove this device. It will succeed only if nobody 
> is holding a
> reference to that device.
> 
> Gal.

Yep - with the addition of the device identifier part.

-- james s

Re: SCSI device rescan, detection of disconnected device, or switched devices.

[Vladislav Bolkhovitin]

James Bottomley wrote:
> On Thu, 2008-07-31 at 19:59 +0400, Vladislav Bolkhovitin wrote:
>> James Bottomley wrote:
>>> Most of the hotplug we do depends on the transport model (because what's
>>> on the transport is changing).  Array reconfiguration has no hotplug
>>> event because SAM-3 has no real way of passing the information
>>> asynchronously.  The best it can do is Unit Attention/reported luns data
>>> has changed (asc=0x3f/ascq=0xe) on the next command.
>>>
>>> The problem is that there's no way to process the event correctly even
>>> when we get it.  All we can do is issue another report LUNS command and
>>> compare.  However, just because it looks like a single LUN disappeared
>>> doesn't mean the others weren't permuted or altered in some way (which
>>> data we cannot get).
>> But why don't do what's possible to do and for what there is complete 
>> information: add new LUN for new LUN and delete deleted LUN for reported 
>> luns data has changed UA? As well as get new capacity for a LU on 
>> capacity data has changed UA after, e.g., new space added to it?
> 
> Well, feel free to submit a patch, but it's an incredible mine field.
> We certainly can't do something that builds a trap for users (as in a
> policy which gets it right 10% of the time and wrong 90% of the time) so
> I'm strongly inclined to feel array reconfiguration is a manual process.
> If you think you can do it online, we don't panic or throw a spanner in
> the works, but we wait until you tell us what you did, but you'd be well
> advised to do it offline.
> 
> The point is that we can't distinguish LUN added from LUN split or space
> reapportioned or something ... even LUN renumbered would be seen as a
> remove followed by an add.

Sure, in cases, where it is impossible based of INQUIRY data 
unambiguously determine that, it's better don't do anything. But in 
cases if it is possible, which is true for most modern devices, it can 
be handled properly.

Vlad

RE: SCSI device rescan, detection of disconnected device,or switched devices.

[James Bottomley]

On Thu, 2008-07-31 at 21:52 +0300, Gal Rosen wrote:
> It won't change the device identification of the existing 10 LUNs, but
> it will report UA of "Power on, reset, or bus device reset occured",
> then I can send read capacity command.

Sending the same email four times doesn't help ... it likely got eaten
by linux-scsi each time because you sent a mixed text/html message ...
the lists only like plain text

But the point is that what happens is highly array dependent ... some
will change the id some won't.  There's nothing we can say for certain
without having knowledge of the array, which really belongs in policy
somewhere.

James

> 
> -----Original Message-----
> From: James Bottomley [mailto:James.Bottomley@HansenPartnership.com]
> Sent: Thu 7/31/2008 7:20 PM
> To: Gal Rosen
> Cc: Stefan Richter; linux-scsi@vger.kernel.org
> Subject: Re: SCSI device rescan, detection of disconnected device,or
> switched devices.
> 
> On Thu, 2008-07-31 at 19:09 +0300, Gal Rosen wrote:
> > On Thu, 2008-07-31 at 09:15 -0500, James Bottomley wrote:
> > > Oh ... you're not really talking about hotplug, which is why
> everyone is
> > > confused.  Hotplug is when you add or remove something from the
> bus.
> > > What you've done is reconfigure the array.
> > >
> > > Most of the hotplug we do depends on the transport model (because
> what's
> > > on the transport is changing).  Array reconfiguration has no
> hotplug
> > > event because SAM-3 has no real way of passing the information
> > > asynchronously.  The best it can do is Unit Attention/reported
> luns data
> > > has changed (asc=0x3f/ascq=0xe) on the next command.
> > >
> > > The problem is that there's no way to process the event correctly
> even
> > > when we get it.  All we can do is issue another report LUNS
> command and
> > > compare.  However, just because it looks like a single LUN
> disappeared
> > > doesn't mean the others weren't permuted or altered in some way
> (which
> > > data we cannot get).
> > >
> >
> > ?Why? if I keep a mapping between sg devices and device id (Inquiry
> page
> > 0x83) then I have all the data.
> 
> No we don't, it depends on array policy.  Most arrays when told to
> take
> 10% of space from 10 luns and create a new one won't change the WWN of
> the existing 10 LUNS, so we have no idea what just happened or what we
> should do about it (incidentally, this is an example of a
> reconfiguration you probably wanted to do offline anyway).
> 
> James
>