* [LSF/MM TOPIC] SCSI referrals support @ 2011-01-20 16:00 Hannes Reinecke 2011-01-20 16:14 ` Douglas Gilbert 0 siblings, 1 reply; 9+ messages in thread From: Hannes Reinecke @ 2011-01-20 16:00 UTC (permalink / raw) To: SCSI Mailing List Agenda topic proposal: SCSI referrals support has already been discussed at last year's LSF conference. However, the solution proposed there would not support failover and would require quite a lot of changes to multipathing. To enable failover it might be an idea to handle the LUN directly in multipathing. This would require eg: - request splitting - I/O alignment handling - SCSI unit attention handling I would be giving a short overview/presentation of the current state of the art, the shortcomings on the original proposal, and would like to invite a discussion on how to best support SCSI referrals. Cheers, Hannes -- Dr. Hannes Reinecke zSeries & Storage hare@suse.de +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [LSF/MM TOPIC] SCSI referrals support 2011-01-20 16:00 [LSF/MM TOPIC] SCSI referrals support Hannes Reinecke @ 2011-01-20 16:14 ` Douglas Gilbert 2011-01-20 16:36 ` James Bottomley 0 siblings, 1 reply; 9+ messages in thread From: Douglas Gilbert @ 2011-01-20 16:14 UTC (permalink / raw) To: Hannes Reinecke; +Cc: SCSI Mailing List On 11-01-20 05:00 PM, Hannes Reinecke wrote: > Agenda topic proposal: > > SCSI referrals support has already been discussed at last year's LSF > conference. However, the solution proposed there would not support > failover and would require quite a lot of changes to multipathing. > > To enable failover it might be an idea to handle the LUN directly in > multipathing. This would require eg: > - request splitting > - I/O alignment handling > - SCSI unit attention handling > > I would be giving a short overview/presentation of the current > state of the art, the shortcomings on the original proposal, > and would like to invite a discussion on how to best support > SCSI referrals. IMO a worrying aspect of the changes associated with SCSI referrals is that sense data can now be returned with any SCSI status (i.e. not just CHECK CONDITION). How well would the SCSI subsystem cope with that? I know that the sg driver (and probably bsg) would need changes, as would my libsgutils library used by sg3_utils. Doug Gilbert ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [LSF/MM TOPIC] SCSI referrals support 2011-01-20 16:14 ` Douglas Gilbert @ 2011-01-20 16:36 ` James Bottomley 2011-01-23 11:25 ` Boaz Harrosh 0 siblings, 1 reply; 9+ messages in thread From: James Bottomley @ 2011-01-20 16:36 UTC (permalink / raw) To: dgilbert; +Cc: Hannes Reinecke, SCSI Mailing List On Thu, 2011-01-20 at 17:14 +0100, Douglas Gilbert wrote: > On 11-01-20 05:00 PM, Hannes Reinecke wrote: > > Agenda topic proposal: > > > > SCSI referrals support has already been discussed at last year's LSF > > conference. However, the solution proposed there would not support > > failover and would require quite a lot of changes to multipathing. > > > > To enable failover it might be an idea to handle the LUN directly in > > multipathing. This would require eg: > > - request splitting > > - I/O alignment handling > > - SCSI unit attention handling > > > > I would be giving a short overview/presentation of the current > > state of the art, the shortcomings on the original proposal, > > and would like to invite a discussion on how to best support > > SCSI referrals. > > IMO a worrying aspect of the changes associated with SCSI > referrals is that sense data can now be returned with any > SCSI status (i.e. not just CHECK CONDITION). How well would > the SCSI subsystem cope with that? I know that the sg driver > (and probably bsg) would need changes, as would my libsgutils > library used by sg3_utils. Well, not necessarily. It scuppers plans to try to use the sense buffer more efficiently, but right at the moment, we can receive sense for any command. We only actually look at it on a check condition return, but that's easily updated. As I read the standard, referrals sense data on successfully completed commands is only really relevant to mp implementations, so it looks like the mid-layer should ignore it anyway (as it would now) but provide a hook for the device handlers to see it if they wanted. James ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [LSF/MM TOPIC] SCSI referrals support 2011-01-20 16:36 ` James Bottomley @ 2011-01-23 11:25 ` Boaz Harrosh 2011-01-24 8:04 ` Hannes Reinecke 0 siblings, 1 reply; 9+ messages in thread From: Boaz Harrosh @ 2011-01-23 11:25 UTC (permalink / raw) To: James Bottomley; +Cc: dgilbert, Hannes Reinecke, SCSI Mailing List On 01/20/2011 06:36 PM, James Bottomley wrote: > On Thu, 2011-01-20 at 17:14 +0100, Douglas Gilbert wrote: >> On 11-01-20 05:00 PM, Hannes Reinecke wrote: >>> Agenda topic proposal: >>> >>> SCSI referrals support has already been discussed at last year's LSF >>> conference. However, the solution proposed there would not support >>> failover and would require quite a lot of changes to multipathing. >>> >>> To enable failover it might be an idea to handle the LUN directly in >>> multipathing. This would require eg: >>> - request splitting >>> - I/O alignment handling >>> - SCSI unit attention handling >>> >>> I would be giving a short overview/presentation of the current >>> state of the art, the shortcomings on the original proposal, >>> and would like to invite a discussion on how to best support >>> SCSI referrals. >> >> IMO a worrying aspect of the changes associated with SCSI >> referrals is that sense data can now be returned with any >> SCSI status (i.e. not just CHECK CONDITION). How well would >> the SCSI subsystem cope with that? I know that the sg driver >> (and probably bsg) would need changes, as would my libsgutils >> library used by sg3_utils. > > Well, not necessarily. It scuppers plans to try to use the sense buffer > more efficiently, but right at the moment, we can receive sense for any > command. We only actually look at it on a check condition return, but > that's easily updated. > > As I read the standard, referrals sense data on successfully completed > commands is only really relevant to mp implementations, so it looks like > the mid-layer should ignore it anyway (as it would now) but provide a > hook for the device handlers to see it if they wanted. > > James > Something like: diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index eafeeda..454e562 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -722,20 +722,23 @@ void scsi_io_completion(struct scsi_cmnd *cmd, unsigned int good_bytes) sense_deferred = scsi_sense_is_deferred(&sshdr); } + if (req->sense) { + /* + * SG_IO wants current and deferred errors + */ + int len = 8 + cmd->sense_buffer[7]; + + if (unlikely(len)) { + if (len > SCSI_SENSE_BUFFERSIZE) + len = SCSI_SENSE_BUFFERSIZE; + memcpy(req->sense, cmd->sense_buffer, len); + req->sense_len = len; + } + } + if (req->cmd_type == REQ_TYPE_BLOCK_PC) { /* SG_IO ioctl from block level */ req->errors = result; if (result) { - if (sense_valid && req->sense) { - /* - * SG_IO wants current and deferred errors - */ - int len = 8 + cmd->sense_buffer[7]; - - if (len > SCSI_SENSE_BUFFERSIZE) - len = SCSI_SENSE_BUFFERSIZE; - memcpy(req->sense, cmd->sense_buffer, len); - req->sense_len = len; - } if (!sense_deferred) error = -EIO; } Then an interested user can just look at req->sense_len. The reset of the stack will ignore all that because it looks for status/error-code. I was needing something like that as well but it's to do with advance storage maintenance, that's far down the road. Thanks Boaz ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [LSF/MM TOPIC] SCSI referrals support 2011-01-23 11:25 ` Boaz Harrosh @ 2011-01-24 8:04 ` Hannes Reinecke 2011-01-24 9:17 ` Boaz Harrosh 2011-03-31 16:40 ` Bart Van Assche 0 siblings, 2 replies; 9+ messages in thread From: Hannes Reinecke @ 2011-01-24 8:04 UTC (permalink / raw) To: Boaz Harrosh; +Cc: James Bottomley, dgilbert, SCSI Mailing List On 01/23/2011 12:25 PM, Boaz Harrosh wrote: > On 01/20/2011 06:36 PM, James Bottomley wrote: >> On Thu, 2011-01-20 at 17:14 +0100, Douglas Gilbert wrote: >>> On 11-01-20 05:00 PM, Hannes Reinecke wrote: >>>> Agenda topic proposal: >>>> >>>> SCSI referrals support has already been discussed at last year's LSF >>>> conference. However, the solution proposed there would not support >>>> failover and would require quite a lot of changes to multipathing. >>>> >>>> To enable failover it might be an idea to handle the LUN directly in >>>> multipathing. This would require eg: >>>> - request splitting >>>> - I/O alignment handling >>>> - SCSI unit attention handling >>>> >>>> I would be giving a short overview/presentation of the current >>>> state of the art, the shortcomings on the original proposal, >>>> and would like to invite a discussion on how to best support >>>> SCSI referrals. >>> >>> IMO a worrying aspect of the changes associated with SCSI >>> referrals is that sense data can now be returned with any >>> SCSI status (i.e. not just CHECK CONDITION). How well would >>> the SCSI subsystem cope with that? I know that the sg driver >>> (and probably bsg) would need changes, as would my libsgutils >>> library used by sg3_utils. >> >> Well, not necessarily. It scuppers plans to try to use the sense buffer >> more efficiently, but right at the moment, we can receive sense for any >> command. We only actually look at it on a check condition return, but >> that's easily updated. >> >> As I read the standard, referrals sense data on successfully completed >> commands is only really relevant to mp implementations, so it looks like >> the mid-layer should ignore it anyway (as it would now) but provide a >> hook for the device handlers to see it if they wanted. >> >> James >> > Something like: > diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c > index eafeeda..454e562 100644 > --- a/drivers/scsi/scsi_lib.c > +++ b/drivers/scsi/scsi_lib.c > @@ -722,20 +722,23 @@ void scsi_io_completion(struct scsi_cmnd *cmd, unsigned int good_bytes) > sense_deferred = scsi_sense_is_deferred(&sshdr); > } > > + if (req->sense) { > + /* > + * SG_IO wants current and deferred errors > + */ > + int len = 8 + cmd->sense_buffer[7]; > + > + if (unlikely(len)) { > + if (len > SCSI_SENSE_BUFFERSIZE) > + len = SCSI_SENSE_BUFFERSIZE; > + memcpy(req->sense, cmd->sense_buffer, len); > + req->sense_len = len; > + } > + } > + > if (req->cmd_type == REQ_TYPE_BLOCK_PC) { /* SG_IO ioctl from block level */ > req->errors = result; > if (result) { > - if (sense_valid && req->sense) { > - /* > - * SG_IO wants current and deferred errors > - */ > - int len = 8 + cmd->sense_buffer[7]; > - > - if (len > SCSI_SENSE_BUFFERSIZE) > - len = SCSI_SENSE_BUFFERSIZE; > - memcpy(req->sense, cmd->sense_buffer, len); > - req->sense_len = len; > - } > if (!sense_deferred) > error = -EIO; > } > > Then an interested user can just look at req->sense_len. The reset of the stack will > ignore all that because it looks for status/error-code. > Ah, if it were so easy. Currently sense codes have two problems: - They are limited to 96 bytes. Anything larger than that will just be discarded (or crash with your patch above :-) - They inherit the same lifetime than the scsi command. But for any decent handling you really need to push them into some asynchronous context as you might well be within an interrupt handler here. I'm currently working on a handling framework using relayfs (basically blktrace for SCSI Unit Attention); I can be doing a short presentation at LSF if requested. Cheers, Hannes -- Dr. Hannes Reinecke zSeries & Storage hare@suse.de +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [LSF/MM TOPIC] SCSI referrals support 2011-01-24 8:04 ` Hannes Reinecke @ 2011-01-24 9:17 ` Boaz Harrosh 2011-01-26 15:54 ` Hannes Reinecke 2011-03-31 16:40 ` Bart Van Assche 1 sibling, 1 reply; 9+ messages in thread From: Boaz Harrosh @ 2011-01-24 9:17 UTC (permalink / raw) To: Hannes Reinecke; +Cc: James Bottomley, dgilbert, SCSI Mailing List On 01/24/2011 10:04 AM, Hannes Reinecke wrote: > On 01/23/2011 12:25 PM, Boaz Harrosh wrote: >> On 01/20/2011 06:36 PM, James Bottomley wrote: >>> On Thu, 2011-01-20 at 17:14 +0100, Douglas Gilbert wrote: >>>> On 11-01-20 05:00 PM, Hannes Reinecke wrote: >>>>> Agenda topic proposal: >>>>> >>>>> SCSI referrals support has already been discussed at last year's LSF >>>>> conference. However, the solution proposed there would not support >>>>> failover and would require quite a lot of changes to multipathing. >>>>> >>>>> To enable failover it might be an idea to handle the LUN directly in >>>>> multipathing. This would require eg: >>>>> - request splitting >>>>> - I/O alignment handling >>>>> - SCSI unit attention handling >>>>> >>>>> I would be giving a short overview/presentation of the current >>>>> state of the art, the shortcomings on the original proposal, >>>>> and would like to invite a discussion on how to best support >>>>> SCSI referrals. >>>> >>>> IMO a worrying aspect of the changes associated with SCSI >>>> referrals is that sense data can now be returned with any >>>> SCSI status (i.e. not just CHECK CONDITION). How well would >>>> the SCSI subsystem cope with that? I know that the sg driver >>>> (and probably bsg) would need changes, as would my libsgutils >>>> library used by sg3_utils. >>> >>> Well, not necessarily. It scuppers plans to try to use the sense buffer >>> more efficiently, but right at the moment, we can receive sense for any >>> command. We only actually look at it on a check condition return, but >>> that's easily updated. >>> >>> As I read the standard, referrals sense data on successfully completed >>> commands is only really relevant to mp implementations, so it looks like >>> the mid-layer should ignore it anyway (as it would now) but provide a >>> hook for the device handlers to see it if they wanted. >>> >>> James >>> >> Something like: >> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c >> index eafeeda..454e562 100644 >> --- a/drivers/scsi/scsi_lib.c >> +++ b/drivers/scsi/scsi_lib.c >> @@ -722,20 +722,23 @@ void scsi_io_completion(struct scsi_cmnd *cmd, unsigned int good_bytes) >> sense_deferred = scsi_sense_is_deferred(&sshdr); >> } >> >> + if (req->sense) { >> + /* >> + * SG_IO wants current and deferred errors >> + */ >> + int len = 8 + cmd->sense_buffer[7]; >> + >> + if (unlikely(len)) { >> + if (len > SCSI_SENSE_BUFFERSIZE) >> + len = SCSI_SENSE_BUFFERSIZE; See here it will not crash >> + memcpy(req->sense, cmd->sense_buffer, len); >> + req->sense_len = len; >> + } >> + } >> + >> if (req->cmd_type == REQ_TYPE_BLOCK_PC) { /* SG_IO ioctl from block level */ >> req->errors = result; >> if (result) { >> - if (sense_valid && req->sense) { >> - /* >> - * SG_IO wants current and deferred errors >> - */ >> - int len = 8 + cmd->sense_buffer[7]; >> - >> - if (len > SCSI_SENSE_BUFFERSIZE) >> - len = SCSI_SENSE_BUFFERSIZE; >> - memcpy(req->sense, cmd->sense_buffer, len); >> - req->sense_len = len; >> - } >> if (!sense_deferred) >> error = -EIO; >> } >> >> Then an interested user can just look at req->sense_len. The reset of the stack will >> ignore all that because it looks for status/error-code. >> > Ah, if it were so easy. Currently sense codes have two problems: > > - They are limited to 96 bytes. Anything larger than that will just > be discarded (or crash with your patch above :-) It will not crash. I have a patchset here that expands the request/scsi_cmnd sense_buffer support to the maximum 260 bytes supported by the std. If you want I can revive it. It is a sweep of all drivers, but a small one, nothing like the accessors changes. > - They inherit the same lifetime than the scsi command. But for any > decent handling you really need to push them into some > asynchronous context as you might well be within an interrupt > handler here. No that's fine. request->sense is a user supplied buffer that extends the life of scsi_cmnd. above code just copies from a scsi-layer dma-able buffer to the request->sense buffer. interrupt-time is fine. > I'm currently working on a handling framework using relayfs > (basically blktrace for SCSI Unit Attention); I can be doing a short > presentation at LSF if requested. > That would be interesting. Thanks > Cheers, > > Hannes Boaz ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [LSF/MM TOPIC] SCSI referrals support 2011-01-24 9:17 ` Boaz Harrosh @ 2011-01-26 15:54 ` Hannes Reinecke 0 siblings, 0 replies; 9+ messages in thread From: Hannes Reinecke @ 2011-01-26 15:54 UTC (permalink / raw) To: Boaz Harrosh; +Cc: James Bottomley, dgilbert, SCSI Mailing List On 01/24/2011 10:17 AM, Boaz Harrosh wrote: > On 01/24/2011 10:04 AM, Hannes Reinecke wrote: [ .. ] >> Ah, if it were so easy. Currently sense codes have two problems: >> >> - They are limited to 96 bytes. Anything larger than that will just >> be discarded (or crash with your patch above :-) > > It will not crash. > Oh, indeed. You are right. > I have a patchset here that expands the request/scsi_cmnd sense_buffer > support to the maximum 260 bytes supported by the std. > > If you want I can revive it. It is a sweep of all drivers, but a small > one, nothing like the accessors changes. > Please. Can you send it to me? I have a prototype working right now, and expanding the sense buffer would be a good idea here. >> - They inherit the same lifetime than the scsi command. But for any >> decent handling you really need to push them into some >> asynchronous context as you might well be within an interrupt >> handler here. > > No that's fine. request->sense is a user supplied buffer that extends > the life of scsi_cmnd. above code just copies from a scsi-layer dma-able > buffer to the request->sense buffer. interrupt-time is fine. > >> I'm currently working on a handling framework using relayfs >> (basically blktrace for SCSI Unit Attention); I can be doing a short >> presentation at LSF if requested. >> > > That would be interesting. Thanks > Ok, will then be doing it. Cheers, Hannes -- Dr. Hannes Reinecke zSeries & Storage hare@suse.de +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [LSF/MM TOPIC] SCSI referrals support 2011-01-24 8:04 ` Hannes Reinecke 2011-01-24 9:17 ` Boaz Harrosh @ 2011-03-31 16:40 ` Bart Van Assche 2011-03-31 17:56 ` Brian King 1 sibling, 1 reply; 9+ messages in thread From: Bart Van Assche @ 2011-03-31 16:40 UTC (permalink / raw) To: Hannes Reinecke Cc: Boaz Harrosh, James Bottomley, dgilbert, SCSI Mailing List On Mon, Jan 24, 2011 at 9:04 AM, Hannes Reinecke <hare@suse.de> wrote: > > Currently sense codes have two problems: > > - They are limited to 96 bytes. Anything larger than that will just > be discarded (or crash with your patch above :-) > - They inherit the same lifetime than the scsi command. But for any > decent handling you really need to push them into some > asynchronous context as you might well be within an interrupt > handler here. > > I'm currently working on a handling framework using relayfs > (basically blktrace for SCSI Unit Attention); I can be doing a short > presentation at LSF if requested. (replying to an e-mail of two months ago) There is another scenario where better unit attention support would help. Both the LIO-port and the SCST-port of the ibmvstgt driver suffer from the same race condition during reboot of the virtual I/O server (VIOS). Since the ibmvscsic driver reconnects to the ibmvscsis driver as soon as the ibmvscsis driver has been loaded the client reconnects before all LUNs have been configured in the VIOS system. So even if the target implementation generates an INQUIRY DATA HAS CHANGED unit attention condition for each LUN change, LUNs will have to be rescanned explicitly at the initiator side in order to get the initiator state in sync with the target state. Also, when using iSCSI, it would beneficial if LUN changes at the target would result in an automatic update of the LUNs reported by the initiator. This is not yet the case when using a Linux iSCSI initiator because the INQUIRY DATA HAS CHANGED unit attention condition is ignored by the Linux SCSI stack. Bart. -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [LSF/MM TOPIC] SCSI referrals support 2011-03-31 16:40 ` Bart Van Assche @ 2011-03-31 17:56 ` Brian King 0 siblings, 0 replies; 9+ messages in thread From: Brian King @ 2011-03-31 17:56 UTC (permalink / raw) To: Bart Van Assche Cc: Hannes Reinecke, Boaz Harrosh, James Bottomley, dgilbert, SCSI Mailing List On 03/31/2011 11:40 AM, Bart Van Assche wrote: > On Mon, Jan 24, 2011 at 9:04 AM, Hannes Reinecke <hare@suse.de> wrote: >> >> Currently sense codes have two problems: >> >> - They are limited to 96 bytes. Anything larger than that will just >> be discarded (or crash with your patch above :-) >> - They inherit the same lifetime than the scsi command. But for any >> decent handling you really need to push them into some >> asynchronous context as you might well be within an interrupt >> handler here. >> >> I'm currently working on a handling framework using relayfs >> (basically blktrace for SCSI Unit Attention); I can be doing a short >> presentation at LSF if requested. > > (replying to an e-mail of two months ago) > > There is another scenario where better unit attention support would > help. Both the LIO-port and the SCST-port of the ibmvstgt driver > suffer from the same race condition during reboot of the virtual I/O > server (VIOS). Since the ibmvscsic driver reconnects to the ibmvscsis > driver as soon as the ibmvscsis driver has been loaded the client > reconnects before all LUNs have been configured in the VIOS system. So > even if the target implementation generates an INQUIRY DATA HAS > CHANGED unit attention condition for each LUN change, LUNs will have > to be rescanned explicitly at the initiator side in order to get the > initiator state in sync with the target state. This sounds to me like the vscsi server drivers really need to change to only register their CRQ and not initialize their CRQ until userspace has setup all the target devices. -Brian -- Brian King Linux on Power Virtualization IBM Linux Technology Center ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2011-03-31 17:56 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2011-01-20 16:00 [LSF/MM TOPIC] SCSI referrals support Hannes Reinecke 2011-01-20 16:14 ` Douglas Gilbert 2011-01-20 16:36 ` James Bottomley 2011-01-23 11:25 ` Boaz Harrosh 2011-01-24 8:04 ` Hannes Reinecke 2011-01-24 9:17 ` Boaz Harrosh 2011-01-26 15:54 ` Hannes Reinecke 2011-03-31 16:40 ` Bart Van Assche 2011-03-31 17:56 ` Brian King
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).