[RFC] scsi_normalize_sense()

[RFC] scsi_normalize_sense()

[Douglas Gilbert]

Looking around the linux SCSI subsystem the handling
of data in the sense buffer is haphazard. Most places
ignore "deferred" errors (e.g. medium error reported
later when write caching is on) and will break horribly
if they every see the newer "descriptor" format.

Descriptor format sense data is much cleaner (amongst
other reason are that it doesn't have to carry around 20
years of baggage). See SPC-3 revision 20a section 4.5
at http://www.t10.org for more information.

The sense format a SCSI device will use is controlled
by the D_SENSE bit in the control mode page.

Here is some code which I am proposing to put into
scsi_lib.c to facilitate cleaner handling. I have been
testing it out in sg3_utils-1.08 (beta) for the last
week.  Comments??



/* This is a slightly stretched SCSI sense "descriptor" format header.
    The addition is to allow the 0x70 and 0x71 response codes. The idea
    is to place the salient data of both "fixed" and "descriptor" sense
    format into one structure to ease application processing.
    The original sense buffer should be kept around for those cases
    in which more information is required (e.g. the LBA of a MEDIUM 
ERROR). */
struct scsi_sense_descriptor_hd {
     unsigned char response_code; /* permit: 0x0, 0x70, 0x71, 0x72, 0x73 */
     unsigned char sense_key;
     unsigned char asc;
     unsigned char ascq;
     unsigned char byte4;
     unsigned char byte5;
     unsigned char byte6;
     unsigned char additional_length;
};


/* Maps the salient data from a sense buffer which is in either fixed or
    descriptor format into a structure mimicking a descriptor format
    header (i.e. the first 8 bytes).
    If zero response code returns 0. Otherwise returns 1 and if 'sdescp' is
    non-NULL then zero all fields and then set the appropriate fields in
    that structure. sdescp::additional_length is always 0 for response
    codes 0x70 and 0x71 (fixed format). */
int scsi_normalize_sense(const unsigned char * sensep,
                          int sb_len,
                          struct scsi_sense_descriptor_hd * sdescp)
{
     if (sdescp)
         memset(sdescp, 0, sizeof(struct scsi_sense_descriptor_hd));
     if ((NULL == sensep) || (0 == sb_len) || (0x70 != (0x70 & sensep[0])))
         return 0;
     if (sdescp) {
         sdescp->response_code = (0x7f & sensep[0]);
         if (sdescp->response_code >= 0x72) {  /* descriptor format */
             if (sb_len > 1)
                 sdescp->sense_key = (0xf & sensep[1]);
             if (sb_len > 2)
                 sdescp->asc = sensep[2];
             if (sb_len > 3)
                 sdescp->ascq = sensep[3];
             if (sb_len > 7)
                 sdescp->additional_length = sensep[7];
         } else {                              /* fixed format */
             if (sb_len > 2)
                 sdescp->sense_key = (0xf & sensep[2]);
             if (sb_len > 7) {
                 sb_len = (sb_len < (sensep[7] + 8)) ? sb_len :
                                                       (sensep[7] + 8);
                 if (sb_len > 12)
                     sdescp->asc = sensep[12];
                 if (sb_len > 13)
                     sdescp->ascq = sensep[13];
             }
         }
     }
     return 1;
}


Doug Gilbert

Re: [RFC] scsi_normalize_sense()

[Christoph Hellwig]

On Fri, Aug 20, 2004 at 03:43:06PM +1000, Douglas Gilbert wrote:
> Looking around the linux SCSI subsystem the handling
> of data in the sense buffer is haphazard. Most places
> ignore "deferred" errors (e.g. medium error reported
> later when write caching is on) and will break horribly
> if they every see the newer "descriptor" format.
> 
> Descriptor format sense data is much cleaner (amongst
> other reason are that it doesn't have to carry around 20
> years of baggage). See SPC-3 revision 20a section 4.5
> at http://www.t10.org for more information.
> 
> The sense format a SCSI device will use is controlled
> by the D_SENSE bit in the control mode page.
> 
> Here is some code which I am proposing to put into
> scsi_lib.c to facilitate cleaner handling. I have been
> testing it out in sg3_utils-1.08 (beta) for the last
> week.  Comments??

This looks good to me.  Before submission for the kernel side
please massage it into the canonical kernel style, though.

Re: [RFC] scsi_normalize_sense()

[Luben Tuikov]

Douglas Gilbert wrote:
> Looking around the linux SCSI subsystem the handling
> of data in the sense buffer is haphazard. Most places
> ignore "deferred" errors (e.g. medium error reported
> later when write caching is on) and will break horribly
> if they every see the newer "descriptor" format.
> 
> Descriptor format sense data is much cleaner (amongst
> other reason are that it doesn't have to carry around 20
> years of baggage). See SPC-3 revision 20a section 4.5
> at http://www.t10.org for more information.
> 
> The sense format a SCSI device will use is controlled
> by the D_SENSE bit in the control mode page.
> 
> Here is some code which I am proposing to put into
> scsi_lib.c to facilitate cleaner handling. I have been
> testing it out in sg3_utils-1.08 (beta) for the last
> week.  Comments??
> 
> 
> 
> /* This is a slightly stretched SCSI sense "descriptor" format header.
>     The addition is to allow the 0x70 and 0x71 response codes. The idea
>     is to place the salient data of both "fixed" and "descriptor" sense
>     format into one structure to ease application processing.
>     The original sense buffer should be kept around for those cases
>     in which more information is required (e.g. the LBA of a MEDIUM
> ERROR). */
> struct scsi_sense_descriptor_hd {

Since this would represend neither descriptor nor fixed
sense format, why not call it just "scsi_sense_hd"?

		Luben



>      unsigned char response_code; /* permit: 0x0, 0x70, 0x71, 0x72, 0x73 */
>      unsigned char sense_key;
>      unsigned char asc;
>      unsigned char ascq;
>      unsigned char byte4;
>      unsigned char byte5;
>      unsigned char byte6;
>      unsigned char additional_length;
> };
> 
> 
> /* Maps the salient data from a sense buffer which is in either fixed or
>     descriptor format into a structure mimicking a descriptor format
>     header (i.e. the first 8 bytes).
>     If zero response code returns 0. Otherwise returns 1 and if 'sdescp' is
>     non-NULL then zero all fields and then set the appropriate fields in
>     that structure. sdescp::additional_length is always 0 for response
>     codes 0x70 and 0x71 (fixed format). */
> int scsi_normalize_sense(const unsigned char * sensep,
>                           int sb_len,
>                           struct scsi_sense_descriptor_hd * sdescp)
> {
>      if (sdescp)
>          memset(sdescp, 0, sizeof(struct scsi_sense_descriptor_hd));
>      if ((NULL == sensep) || (0 == sb_len) || (0x70 != (0x70 & sensep[0])))
>          return 0;
>      if (sdescp) {
>          sdescp->response_code = (0x7f & sensep[0]);
>          if (sdescp->response_code >= 0x72) {  /* descriptor format */
>              if (sb_len > 1)
>                  sdescp->sense_key = (0xf & sensep[1]);
>              if (sb_len > 2)
>                  sdescp->asc = sensep[2];
>              if (sb_len > 3)
>                  sdescp->ascq = sensep[3];
>              if (sb_len > 7)
>                  sdescp->additional_length = sensep[7];
>          } else {                              /* fixed format */
>              if (sb_len > 2)
>                  sdescp->sense_key = (0xf & sensep[2]);
>              if (sb_len > 7) {
>                  sb_len = (sb_len < (sensep[7] + 8)) ? sb_len :
>                                                        (sensep[7] + 8);
>                  if (sb_len > 12)
>                      sdescp->asc = sensep[12];
>                  if (sb_len > 13)
>                      sdescp->ascq = sensep[13];
>              }
>          }
>      }
>      return 1;
> }
> 
> 
> Doug Gilbert
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [RFC] scsi_normalize_sense()

[Kai Makisara]

On Fri, 20 Aug 2004, Douglas Gilbert wrote:

> Looking around the linux SCSI subsystem the handling
> of data in the sense buffer is haphazard. Most places
> ignore "deferred" errors (e.g. medium error reported
> later when write caching is on) and will break horribly
> if they every see the newer "descriptor" format.
> 
...
> Here is some code which I am proposing to put into
> scsi_lib.c to facilitate cleaner handling. I have been
> testing it out in sg3_utils-1.08 (beta) for the last
> week.  Comments??
> 
Looks good, especially after some thinking :-) (The extra fields byte? and 
additional_length looked useless until I thought a little further into the 
future.)

In addition to using this basic data, the drivers have to parse the 
descriptors. Parsing the descriptors is device-specific but a library 
function to find a descriptor of certain type might be useful. The 
prototype could be something like this:

/* Returns a pointer to the descriptor of type desc_type in descriptor 
   format sense data or NULL if the descriptor is not found. */
unsigned char *scsi_sense_descriptor(const unsigned char *sensep,
                                     int sb_len, int desc_type)

-- 
Kai

Re: [RFC] scsi_normalize_sense()

[Douglas Gilbert]

Kai Makisara wrote:
> On Fri, 20 Aug 2004, Douglas Gilbert wrote:
> 
> 
>>Looking around the linux SCSI subsystem the handling
>>of data in the sense buffer is haphazard. Most places
>>ignore "deferred" errors (e.g. medium error reported
>>later when write caching is on) and will break horribly
>>if they every see the newer "descriptor" format.
>>
> 
> ...
> 
>>Here is some code which I am proposing to put into
>>scsi_lib.c to facilitate cleaner handling. I have been
>>testing it out in sg3_utils-1.08 (beta) for the last
>>week.  Comments??
>>
> 
> Looks good, especially after some thinking :-) (The extra fields byte? and 
> additional_length looked useless until I thought a little further into the 
> future.)

The proposed structure is the first 8 bytes of the "descriptor"
format sense data**. The addition is to squeeze in the
response codes 0x70 and 0x71 which indicate that the given
sense buffer is in "fixed" format.

Deferred errors can be detected like this:
     if (sdesc.response_code & 1)
         process_deferred_error(...);

and "descriptor" format can be detected two ways:
     if (sdesc.response_code >= 0x72)
         ucp = scsi_sense_descriptor(,,4);
or
     if (sdesc.additional_length > 0)
         ucp = scsi_sense_descriptor(,,4);

The idea is to generalize access to sense_key,
asc and ascq (and stop using error prone array
indexing) in the presence of both fixed and
descriptor format. This covers the vast majority of
sense data processing in the scsi subsystem at
the moment. As the descriptor format becomes
more common, folks should recognise the proposed
structure.

That said SSC's FILEMARK, EOM and ILI bits are
found in the "stream commands" descriptor.

> In addition to using this basic data, the drivers have to parse the 
> descriptors.

There is such a parser in sg3_utils-1.08 (beta) on the
sg website. See sg_print_sense_descriptors() in sg_err.c
It outputs to stdout so it isn't exactly what is needed.

  Parsing the descriptors is device-specific but a library
> function to find a descriptor of certain type might be useful. The 
> prototype could be something like this:
> 
> /* Returns a pointer to the descriptor of type desc_type in descriptor 
>    format sense data or NULL if the descriptor is not found. */
> unsigned char *scsi_sense_descriptor(const unsigned char *sensep,
>                                      int sb_len, int desc_type)

This looks good for finding a specific 'desc_type'.

This approach also hints at the other approach to this
problem, namely multiple accessor functions to pick up
current/deferred, sense_key, asc and ascq.


BTW I think that the SCSI subsystem gets 32 bytes
of sense data currently. That should be ok until OSDs
arrive. They have big everythings (e.g. variable length
commands and sense descriptors).


** I resisted the temptation to memcpy the "descriptor" format
    header into the proposed structure just in case an 8 byte
    array didn't align with a structure with 8 unsigned chars in
    it.

P.S. Fixed format sense descriptors can still be used with
block devices that need 16 bytes READs and WRITEs. However
if there is a MEDIUM ERROR and the broken LBA cannot fit
in 32 bits then 0 is placed in the information field (i.e.
you don't get told the broken LBA).

Doug Gilbert