Re: [linux-usb-devel] FW: USB 2.0 external hard drive problem

Re: [linux-usb-devel] FW: USB 2.0 external hard drive problem

[Alan Stern]

On Fri, 6 Feb 2004, Joerg Schilling wrote:

> What you see, is that some instance in the Linux kernel completely
> misshandles this transfer:
> 
> -	The drive seems to return only 34 bytes although it did advertise
> 	to return 36.
> 
> 	The correct behavior for Linux would be to set sg_io.resid = 2;
> 	and _not_ to return any error.

You're quite right that the residue value should have been set.  This is a
known bug in the usb-storage driver and a patch (as173) has already been
submitted, though not yet accepted, to fix it.  For Linux 2.6 that is, 
although the same fix will work just as well for 2.4.

> But as you may know, there is a Linux bug that causes the status to be 0 when
> sense data (correctly) is available. Libscg _needs_ to add a workarounf for
> this bug or nothing would work!

What bug is that?  Or rather, under what circumstances is the status equal 
to 0 when it shouldn't be?

> It is definitely a severe bug in the kernel to retrieve sense data in case
> of a DMA size error.

> -	The Linux kernel issues a REQUEST SENSE command ---> this is the first
> 	severe bug. It is completely wrong to issue a REQUEST SENSE in
> 	case of a DMA Size error.

> And there exactly is the bug (which is most likely in usb-storage but
> definitely in the Linux kernel:
> 
>       In case of _only_ a DMA size error, a request sense must not be
>       issued by the driver.
> 
> If you are the maintainer of this piece of software, please fix this.

(I'm not the maintainer; Matthew Dharm is.)

Who says it's wrong to retrieve sense information when there is an
underrun but not Check Condition?  Can you provide a reference to a
published (or draft) document that states this?

Bear in mind that there are some transports which do not provide any
status at all.  The _only_ way for a driver to determine whether an
error occurred is to get the sense data.

In the absence of any official statement to the contrary, the logical
conclusion is that whether or not the driver gets the sense data is a
low-level matter.  It should only be of interest to the driver and the
device.  Higher-level drivers and user programs should not care one way or
the other.

Alan Stern

Re: [linux-usb-devel] FW: USB 2.0 external hard drive problem

[Joerg Schilling]

>From stern@rowland.harvard.edu  Fri Feb  6 15:59:52 2004

>> But as you may know, there is a Linux bug that causes the status to be 0 when
>> sense data (correctly) is available. Libscg _needs_ to add a workarounf for
>> this bug or nothing would work!

>What bug is that?  Or rather, under what circumstances is the status equal 
>to 0 when it shouldn't be?

Sorry, this is long long ago ---> Check the cdrwite mail archives.


>> It is definitely a severe bug in the kernel to retrieve sense data in case
>> of a DMA size error.

>> -	The Linux kernel issues a REQUEST SENSE command ---> this is the first
>> 	severe bug. It is completely wrong to issue a REQUEST SENSE in
>> 	case of a DMA Size error.

>> And there exactly is the bug (which is most likely in usb-storage but
>> definitely in the Linux kernel:
>> 
>>       In case of _only_ a DMA size error, a request sense must not be
>>       issued by the driver.
>> 
>> If you are the maintainer of this piece of software, please fix this.

>(I'm not the maintainer; Matthew Dharm is.)

>Who says it's wrong to retrieve sense information when there is an
>underrun but not Check Condition?  Can you provide a reference to a
>published (or draft) document that states this?

The SCSI standard says that the sense data should retrieved when the 
CHECK CONDITION bit is set. ATAPI drives use a bit in the ATA Status
for this purpose.

Read the SCSI CAM standard to see more on this topic.

>Bear in mind that there are some transports which do not provide any
>status at all.  The _only_ way for a driver to determine whether an
>error occurred is to get the sense data.

Let us asume such transport does not exist, unless you name at least a
single transport that has no bit that means "CHECK CONDITION".


>In the absence of any official statement to the contrary, the logical
>conclusion is that whether or not the driver gets the sense data is a
>low-level matter.  It should only be of interest to the driver and the
>device.  Higher-level drivers and user programs should not care one way or
>the other.

If you retrive sense data without reason, you may "read away" a senseful
error status that is related to a different command. 


Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de		(uni)  If you don't have iso-8859-1
       schilling@fokus.fraunhofer.de	(work) chars I am J"org Schilling
 URL:  http://www.fokus.fraunhofer.de/usr/schilling ftp://ftp.berlios.de/pub/schily
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [linux-usb-devel] FW: USB 2.0 external hard drive problem

[James Bottomley]

On Fri, 2004-02-06 at 09:59, Alan Stern wrote:
> Who says it's wrong to retrieve sense information when there is an
> underrun but not Check Condition?  Can you provide a reference to a
> published (or draft) document that states this?

It's not wrong according to the SCSI standards, a REQUEST SENSE issued
to a device with no sense data will produce a NO SENSE key.

However, because of the way sense is processed in contingent allegiance
conditions (suspending the I_T_L nexus until sense is cleared) it is
dangerous to issue arbitrary request sense commands because you may
receive sense for a different command (and prevent that command from
erroring correctly).

The SCSI mid-layer definitely does not expect this behaviour.  A driver
either does not do any sense commands at all or it only issues a REQUEST
SENSE in response to a Check Condition status in order not to have the
nexus halted because of a contingent allegiance condition (simulating
ACA if you will).

James

Re: [linux-usb-devel] FW: USB 2.0 external hard drive problem

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 1731 bytes --]

On Fri, Feb 06, 2004 at 10:24:25AM -0500, James Bottomley wrote:
> On Fri, 2004-02-06 at 09:59, Alan Stern wrote:
> > Who says it's wrong to retrieve sense information when there is an
> > underrun but not Check Condition?  Can you provide a reference to a
> > published (or draft) document that states this?
> 
> It's not wrong according to the SCSI standards, a REQUEST SENSE issued
> to a device with no sense data will produce a NO SENSE key.
> 
> However, because of the way sense is processed in contingent allegiance
> conditions (suspending the I_T_L nexus until sense is cleared) it is
> dangerous to issue arbitrary request sense commands because you may
> receive sense for a different command (and prevent that command from
> erroring correctly).

You'll excuse me if I stare at you blankly after you say this?

If I'm guessing correctly, this is only an issue with multiple pending
commands (i.e. TCQ or somesuch)?

A USB device only ever has one command at a time.

> The SCSI mid-layer definitely does not expect this behaviour.  A driver
> either does not do any sense commands at all or it only issues a REQUEST
> SENSE in response to a Check Condition status in order not to have the
> nexus halted because of a contingent allegiance condition (simulating
> ACA if you will).

'Check Condition' isn't something we can directly observe.  We get 2 status
bits back which encode "good", "bad", or "you're screwed -- you are now
required to perform a reset".

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

Stef, you just got beaten by a ball of DIRT.
					-- Greg
User Friendly, 12/7/1997

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [linux-usb-devel] FW: USB 2.0 external hard drive problem

[Alan Stern]

On Fri, 6 Feb 2004, Matthew Dharm wrote:

> On Fri, Feb 06, 2004 at 10:24:25AM -0500, James Bottomley wrote:
> > On Fri, 2004-02-06 at 09:59, Alan Stern wrote:
> > > Who says it's wrong to retrieve sense information when there is an
> > > underrun but not Check Condition?  Can you provide a reference to a
> > > published (or draft) document that states this?
> > 
> > It's not wrong according to the SCSI standards, a REQUEST SENSE issued
> > to a device with no sense data will produce a NO SENSE key.
> > 
> > However, because of the way sense is processed in contingent allegiance
> > conditions (suspending the I_T_L nexus until sense is cleared) it is
> > dangerous to issue arbitrary request sense commands because you may
> > receive sense for a different command (and prevent that command from
> > erroring correctly).
> 
> You'll excuse me if I stare at you blankly after you say this?

A surfeit of jargon :-)

> If I'm guessing correctly, this is only an issue with multiple pending
> commands (i.e. TCQ or somesuch)?
> 
> A USB device only ever has one command at a time.
> 
> > The SCSI mid-layer definitely does not expect this behaviour.  A driver
> > either does not do any sense commands at all or it only issues a REQUEST
> > SENSE in response to a Check Condition status in order not to have the
> > nexus halted because of a contingent allegiance condition (simulating
> > ACA if you will).
> 
> 'Check Condition' isn't something we can directly observe.  We get 2 status
> bits back which encode "good", "bad", or "you're screwed -- you are now
> required to perform a reset".

In the absence of anything better, we're forced to assume "bad" status 
corresponds to Check Condition...

What do you think, Matt?  Should we remove the auto-sense for short 
transfers when we get "good" status?  Bearing in mind that it's 
technically legal, but other drivers or programs may not expect it?  Also 
bearing in mind that we have no choice but to auto-sense for non-IN 
transfers with the CB transport.

Maybe we should just log a KERN_DEBUG or _INFO message instead.

Alan Stern

Re: [linux-usb-devel] FW: USB 2.0 external hard drive problem

[James Bottomley]

On Mon, 2004-02-09 at 11:40, Alan Stern wrote:
> In the absence of anything better, we're forced to assume "bad" status 
> corresponds to Check Condition...
> 
> What do you think, Matt?  Should we remove the auto-sense for short 
> transfers when we get "good" status?  Bearing in mind that it's 
> technically legal, but other drivers or programs may not expect it?  Also 
> bearing in mind that we have no choice but to auto-sense for non-IN 
> transfers with the CB transport.

OK, if you want to understand what the mid-layer problem is, look at
scsi_finish_command().  You see in there we set DRIVER_SENSE if we find
any valid sense code in the sense buffer (including NO SENSE)

We will return this to the user as a sense error at various points.

The safest course, if you want to send unsolicited request sense
commands is probably to zero out the sense buffer if you get NO SENSE
back.

James

Re: [linux-usb-devel] FW: USB 2.0 external hard drive problem

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 1408 bytes --]

On Mon, Feb 09, 2004 at 11:50:13AM -0500, James Bottomley wrote:
> On Mon, 2004-02-09 at 11:40, Alan Stern wrote:
> > In the absence of anything better, we're forced to assume "bad" status 
> > corresponds to Check Condition...
> > 
> > What do you think, Matt?  Should we remove the auto-sense for short 
> > transfers when we get "good" status?  Bearing in mind that it's 
> > technically legal, but other drivers or programs may not expect it?  Also 
> > bearing in mind that we have no choice but to auto-sense for non-IN 
> > transfers with the CB transport.
> 
> OK, if you want to understand what the mid-layer problem is, look at
> scsi_finish_command().  You see in there we set DRIVER_SENSE if we find
> any valid sense code in the sense buffer (including NO SENSE)
> 
> We will return this to the user as a sense error at various points.
> 
> The safest course, if you want to send unsolicited request sense
> commands is probably to zero out the sense buffer if you get NO SENSE
> back.

This seems like the right approach to me.  It should follow the principal
of least suprise for existing users, while work around the reported issue.

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

A female who groks UNIX?  My universe is collapsing.
					-- Mike
User Friendly, 10/11/1998

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [linux-usb-devel] FW: USB 2.0 external hard drive problem

[Alan Stern]

On 9 Feb 2004, James Bottomley wrote:

> OK, if you want to understand what the mid-layer problem is, look at
> scsi_finish_command().  You see in there we set DRIVER_SENSE if we find
> any valid sense code in the sense buffer (including NO SENSE)
> 
> We will return this to the user as a sense error at various points.

I don't understand the reasoning here.  If DRIVER_SENSE is set it must 
mean the driver had some reason of its own for wanting to get the sense 
data -- presumably to find out whether or not some error occurred.  So why 
should this get passed to the user as a sense error?  Especially if the 
sense code is NO SENSE.  Logically that should indicate the driver needed 
to find out whether or not there was error, and it learned that there 
wasn't.

> The safest course, if you want to send unsolicited request sense
> commands is probably to zero out the sense buffer if you get NO SENSE
> back.

Okay, we can do that.

Alan Stern

RE: [linux-usb-devel] FW: USB 2.0 external hard drive problem

[Tony Battersby]

> Especially if the sense code is NO SENSE.  Logically that should
> indicate the driver needed to find out whether or not there was error,
> and it learned that there wasn't.

For tape drives, sense key == NO SENSE is not equivalent to no error to
report.  Often a tape drive returns a sense key of NO SENSE but with one
of the other bits (EOM, FILEMARK, ILI) set in the same byte as the sense
key.  There are also some ASC/ASCQ combinations that go with a sense key
of NO SENSE but without any of the EOM, FILEMARK, or ILI bits set.

On the other hand, some tape drives set the EOM bit in unsolicited
request sense data just to report the position of the tape, so the EOM
bit being set isn't necessarily an indicator that there was an error.
However, ASC/ASCQ should be nonzero when the EOM bit is set because of
an error.

I recommend defining "no error" to be:

(sense_data[2] & 0xaf) == 0x00 && /* Filemark 0, ignore EOM, ILI 0, NO
SENSE */
sense_data[12]  == 0x00 && /* ASC */
sense_data[13] == 0x00 /* ASCQ */

This should work for most devices that I have encountered.

Anthony J. Battersby
Cybernetics