One question about SCSI device hotplug

One question about SCSI device hotplug

[Zhao, Forrest]

Hi, list

If I surprisingly hot-remove a SCSI disk from HBA manually 
without executing "echo "scsi remove-single-device
 <h> <b> <t <l>" > /proc/scsi/scsi", can the "hotplug"
event be notified to SCSI mid-layer or user space?

I briefly browse some code of LLDD, it seems that LLDD never
report "hotplug" event to upper-level layer. Am I right?


Thanks,
Forrest

Re: One question about SCSI device hotplug

[Matthew Wilcox]

On Tue, Mar 01, 2005 at 05:06:55PM +0800, Zhao, Forrest wrote:
> If I surprisingly hot-remove a SCSI disk from HBA manually 
> without executing "echo "scsi remove-single-device
>  <h> <b> <t <l>" > /proc/scsi/scsi", can the "hotplug"
> event be notified to SCSI mid-layer or user space?
> 
> I briefly browse some code of LLDD, it seems that LLDD never
> report "hotplug" event to upper-level layer. Am I right?

I don't think it *can* ... the SCSI protocols don't tell the host "hey,
this device has disappeared".

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

RE: One question about SCSI device hotplug

[Nagpure, Dinesh]

If you are trying to automate the process then taking a look at SCSI RAS
tools at http://scsirastools.sourceforge.net might help. I am not sure how
much are these tools in use around (any comments?). Alternatively If you are
running on a disk hot-swap capable system the enclosure service processor
might provide you information on SCSI topology change.

-dinesh
 

-----Original Message-----
From: linux-scsi-owner@vger.kernel.org
[mailto:linux-scsi-owner@vger.kernel.org]On Behalf Of Zhao, Forrest
Sent: Tuesday, March 01, 2005 4:07 AM
To: linux-scsi@vger.kernel.org
Cc: Caushik, Ramesh
Subject: One question about SCSI device hotplug


Hi, list

If I surprisingly hot-remove a SCSI disk from HBA manually 
without executing "echo "scsi remove-single-device
 <h> <b> <t <l>" > /proc/scsi/scsi", can the "hotplug"
event be notified to SCSI mid-layer or user space?

I briefly browse some code of LLDD, it seems that LLDD never
report "hotplug" event to upper-level layer. Am I right?


Thanks,
Forrest
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: One question about SCSI device hotplug

[James.Smart]

> On Tue, Mar 01, 2005 at 05:06:55PM +0800, Zhao, Forrest wrote:
> > If I surprisingly hot-remove a SCSI disk from HBA manually 
> > without executing "echo "scsi remove-single-device
> >  <h> <b> <t <l>" > /proc/scsi/scsi", can the "hotplug"
> > event be notified to SCSI mid-layer or user space?
> > 
> > I briefly browse some code of LLDD, it seems that LLDD never
> > report "hotplug" event to upper-level layer. Am I right?
> 
> I don't think it *can* ... the SCSI protocols don't tell the 
> host "hey,
> this device has disappeared".

All depends on what transport it is... In FC, we can detect when
a target is removed. In the most recent FC transport code (not
all integrated yet), it will indeed automagically invoke
scsi_remove_device() for all luns on the target.

-- james s

RE: One question about SCSI device hotplug

[Rajat Jain, Noida]

What surprises me is that if the LLDD *can't* get to know about the hot
plugging / unplugging of device, what is the use of the functions
scsi_add_device() and scsi_remove_device()??

Please note that these functions are provided by scsi mid-level driver to
LLDD to add / remove devices (Refer to
Documentation/scsi/scsi_mid_low_api.txt).

Thanks & Regards,

Rajat Jain
HCL Technologies,
A-11, Sector 16, Noida-201301.
Ph: +91-120-2510701/702/813 Extn.3159
rajatj@noida.hcltech.com
http://www.hcltech.com/ 

	"...one of the main causes of the fall of the Roman Empire was that,
lacking zero, they had no way to indicate successful termination of their C
programs."
	Robert Firth 





-----Original Message-----
From: linux-scsi-owner@vger.kernel.org
[mailto:linux-scsi-owner@vger.kernel.org] On Behalf Of Matthew Wilcox
Sent: Tuesday, March 01, 2005 8:16 PM
To: Zhao, Forrest
Cc: linux-scsi@vger.kernel.org; Caushik, Ramesh
Subject: Re: One question about SCSI device hotplug

On Tue, Mar 01, 2005 at 05:06:55PM +0800, Zhao, Forrest wrote:
> If I surprisingly hot-remove a SCSI disk from HBA manually without 
> executing "echo "scsi remove-single-device  <h> <b> <t <l>" > 
> /proc/scsi/scsi", can the "hotplug"
> event be notified to SCSI mid-layer or user space?
> 
> I briefly browse some code of LLDD, it seems that LLDD never report 
> "hotplug" event to upper-level layer. Am I right?

I don't think it *can* ... the SCSI protocols don't tell the host "hey, this
device has disappeared".

--
"Next the statesmen will invent cheap lies, putting the blame upon the
nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse to
examine any refutations of them; and thus he will by and by convince himself
that the war is just, and will thank God for the better sleep he enjoys
after this process of grotesque self-deception." -- Mark Twain
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the
body of a message to majordomo@vger.kernel.org More majordomo info at
http://vger.kernel.org/majordomo-info.html

Re: One question about SCSI device hotplug

[Matt Domsch]

On Tue, Mar 01, 2005 at 11:35:28AM -0500, James.Smart@Emulex.Com wrote:
> > On Tue, Mar 01, 2005 at 05:06:55PM +0800, Zhao, Forrest wrote:
> > > If I surprisingly hot-remove a SCSI disk from HBA manually 
> > > without executing "echo "scsi remove-single-device
> > >  <h> <b> <t <l>" > /proc/scsi/scsi", can the "hotplug"
> > > event be notified to SCSI mid-layer or user space?
> > > 
> > > I briefly browse some code of LLDD, it seems that LLDD never
> > > report "hotplug" event to upper-level layer. Am I right?
> > 
> > I don't think it *can* ... the SCSI protocols don't tell the 
> > host "hey,
> > this device has disappeared".
> 
> All depends on what transport it is... In FC, we can detect when
> a target is removed. In the most recent FC transport code (not
> all integrated yet), it will indeed automagically invoke
> scsi_remove_device() for all luns on the target.

And in the patches I posted a couple weeks ago, userspace can notify
the kernel about an added/removed device if it knows about it but the
kernel doesn't (say, a management app does the add/remove but the LLDD
doesn't get an interrupt from the firmware on such events).

There's also the slave_{alloc,configure,destroy} callbacks in the
drivers, that let the LLDDs know about a LUN creation or removal, if
you've got device state stored in the driver.

-- 
Matt Domsch
Software Architect
Dell Linux Solutions linux.dell.com & www.dell.com/linux
Linux on Dell mailing lists @ http://lists.us.dell.com

RE: One question about SCSI device hotplug

[Zhao, Forrest]

< but the LLDD doesn't get an interrupt from the firmware on such
events).

So you mean that when SCSI device is surprise-removed, HBA will not
assert an interrupt to kernel? Even LLDD doesn't know the SCSI device is
removed?

Thanks,
Forrest

Re: One question about SCSI device hotplug

[Matt Domsch]

On Wed, Mar 02, 2005 at 12:47:46PM +0800, Zhao, Forrest wrote:
> < but the LLDD doesn't get an interrupt from the firmware on such
> events).
> 
> So you mean that when SCSI device is surprise-removed, HBA will not
> assert an interrupt to kernel? Even LLDD doesn't know the SCSI device is
> removed?

Exactly.  Some controllers know, and assert events up to their LLDD.
Jame's Emulex driver apparently does, perhaps this holds true for all
FC devices.

Some firmwares (LSI megaraid for sure, Adaptec SCSI I believe) don't.

-- 
Matt Domsch
Software Architect
Dell Linux Solutions linux.dell.com & www.dell.com/linux
Linux on Dell mailing lists @ http://lists.us.dell.com

RE: One question about SCSI device hotplug

[Zhao, Forrest]

<Exactly.  Some controllers know, and assert events up to their LLDD.
<Jame's Emulex driver apparently does, perhaps this holds true for all
<FC devices.

<Some firmwares (LSI megaraid for sure, Adaptec SCSI I believe) don't.

Thank you, Matt. Then I have another question:
As we know SCSI mid-layer issue a command to LLDD by 
host->hostt->queuecommand(cmd, scsi_done); and in the meantime a 
timer is set. When the timer expires, SCSI mid-layer know the 
execution of command has failed.
My question is: when SCSI device is surprise-removed, if SCSI mid-layer
issue a command to this removed device, will mid-layer has to wait 
a timeout before it can know the execution of command failed? Or is
there
any other mechanism that LLDD can notify mid-layer that execution of
command failed without waiting for a timeout?

Thanks,
Forrest

RE: One question about SCSI device hotplug

[James.Smart]

> Thank you, Matt. Then I have another question:
> As we know SCSI mid-layer issue a command to LLDD by 
> host->hostt->queuecommand(cmd, scsi_done); and in the meantime a 
> timer is set. When the timer expires, SCSI mid-layer know the 
> execution of command has failed.
> My question is: when SCSI device is surprise-removed, if SCSI 
> mid-layer
> issue a command to this removed device, will mid-layer has to wait 
> a timeout before it can know the execution of command failed? Or is
> there
> any other mechanism that LLDD can notify mid-layer that execution of
> command failed without waiting for a timeout?

What we did in the FC transport - there's a transport level timeout at
the target level that controls how long we "insulate" the system from
the device's disappearance. When the device is first removed, the
transport has the midlayer suspend i/o (e.g block) the device, so no
i/o failures, other than timeouts on in-flight i/o's occur. As the
midlayer (for disk devices) typically retries i/o's, even the in-flight
errors don't result in an error to the application, as the retry get's
delayed due the blocked state of the device. If the device returns
within the insulation period, i/o resumes, and the system continues
happily along it's way. If the device does not return, the timeout
fires, and the device is restarted. The i/o then reaches the LLDD, who
is expected to fail the i/o immediately as the target doesn't exist.
The midlayer reacts accordingly and places the device into an offline
state.

If the device is readded, the LLDD sets the target to a good state, but
the midlayer keeps the devices in an offline state until steps are taken
to bring them back online. E.g. The admin takes whatever steps are
necessary to clean up the system for the previous failure of the device,
then brings the device online by writing the device state to running
and rescanning the device.

If multipath solutions are in place, they will want to set the
"insulation" timeout as low as possible so that access so that it's
alternate pathing can kick in as soon as possible.  The multipathing
solution, upon device re-addition, is required to take the steps to
bring the device back online.

-- james s