Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[Alan Stern]

On Wed, 10 Sep 2003, Georgi Chorbadzhiyski wrote:

> Alan Stern wrote:
> > 
> > More problems with that stupid MODE-SENSE cache page!  There are so many 
> > USB storage devices that have problems with that -- I wonder if it's worth 
> > the effort to try to continue supporting it?

> > If you want a temporary fix for 2.6.0, you can do this:  Edit the 
> > routine sd_read_cache_type in the file drivers/scsi/sd.c (near line 1100).  
> > Get rid of (or #ifdef out) most of the function; just leave the last few 
> > lines where it does:
> > 
> > 		printk(KERN_ERR "%s: assuming drive cache: write through\n",
> > 		       diskname);
> > 		sdkp->WCE = 0;
> > 		sdkp->RCD = 0;
> > 
> > You might want to change the KERN_ERR to KERN_NOTICE.
> 
> Thanks a lot! That worked fine! Now the device is detected and working.

Is there any feeling about how to handle these ongoing problems with the 
mode-sense cache page?  There doesn't seem to be any general solution that 
can work with all USB storage devices.  Some hang when asked to read the 
entire page; some hang when asked to read just part of the page; some hang 
when asked to read just the page header.

What do the SCSI folk have to say about it?

Alan Stern

Re: [usb-storage] Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[Pat LaVarre]

> What do the SCSI folk have to say about it?

Anyone?

> > for 2.6.0 ... drivers/scsi/sd.c ...
> > sd_read_cache_type ...
> > Get rid of (or #ifdef out) most ...
>
> Is there any feeling about how to handle these
> ongoing problems with the mode-sense cache
> page?

Yes, much.

Maybe more feeling if you remind/teach me what breaks if we don't fetch
page 8 re Cache.  First I imagine automounters mount read-only disks rw,
and then the write cache gets errors while flushing.  I wonder what else
breaks.  Maybe thruput varies for devices that do or do not bother to
claim they have write cache.

Maybe more feeling if you can say how in/accurate page 8 re Cache
commonly is.  For example, I know of no clear & broadly adopted standard
that requires a device to declare write-behind or background defect map
updates as a form of write cache.

> There doesn't seem to be any general solution
> that can work with all USB storage devices.

I understand you to be saying we can settle for an sd solution, we don't
mind if the sd heuristic we discover then doesn't apply to pdt x05
dvd/cd.

> some hang when asked to read just the page
> header.
> ...
> some hang when asked to read just part of the
> page;
> ...
> Some hang when asked to read the entire page;

Device folk ship bugs, aye, same as host folk.

Are not two of these three failures disproportionately common?

Me, I would expect asking anything but what Windows asks would fail most
often.  Is that not so?  Do we know what Windows asks?  Do we know
if/how Win ME/ 2K/ XP differ here?  Have we matched CBWCB and
dCBWDataTransferLength in every bit?

Pat LaVarre

sg MiB writes scheduling while atomic

[Pat LaVarre]

In reply to `modprobe -r sg`, linux-2.6.0-test5 dmesg tells me:

Debug: sleeping function called from invalid context at
include/linux/rwsem.h:66
...
bad: scheduling while atomic!

Those lines appear in the log below.  The ... ellipses you see were
typed by me, sorry I don't have a more complete intact log, nor have I
yet repeated this result myself.

To provoke this log overnight I ran a shell script to repeat something
like:
pldd of=/dev/sg1 if=/dev/zero bs=1m

The "usb 1-1: sg_complete, unlink --> -19" appeared when or before an
ioctl SG_IO hung.  The Debug/ Call Trace appeared in response to a
`modprobe -r sg` that hung.  I'm confident the cable was not unplugged
because rebooting the system made sg work again.  I'm confident the
drive power was not cycled because reboot provoked only an SK ASC = x 6
29 Reset unit attention, without an SK ASC = x 6 28 GoneReady unit
attention.

[usb-storage] folk tell me some linux-scsi folk may appreciate me
mentioning this.

Pat LaVarre

...
usb-storage: Command WRITE_10 (10 bytes)
usb-storage:  2a 00 00 ce 26 00 00 02 00 00
usb-storage: Bulk Command S 0x43425355 T 0x439c9 L 1048576 F 0 Trg 0 LUN 0 CL 10
usb-storage: usb_stor_bulk_transfer_buf: xfer 31 bytes
usb-storage: Status code 0; transferred 31/31
usb-storage: -- transfer complete
usb-storage: Bulk command transfer result=0
usb-storage: usb_stor_bulk_transfer_sglist: xfer 1048576 bytes, 32 entries
usb-storage: Status code 0; transferred 1048576/1048576
usb-storage: -- transfer complete
usb-storage: Bulk data transfer result 0x0
usb-storage: Attempting to get CSW...
usb-storage: usb_stor_bulk_transfer_buf: xfer 13 bytes
usb-storage: Status code 0; transferred 13/13
usb-storage: -- transfer complete
usb-storage: Bulk status result = 0
usb-storage: Bulk Status S 0x53425355 T 0x439c9 R 0 Stat 0x0
usb-storage: scsi cmd done, result=0x0
usb-storage: *** thread sleeping.
usb-storage: queuecommand called
usb-storage: *** thread awakened.
usb-storage: Command WRITE_10 (10 bytes)
usb-storage:  2a 00 00 ce 28 00 00 02 00 00
usb-storage: Bulk Command S 0x43425355 T 0x439ca L 1048576 F 0 Trg 0 LUN 0 CL 10
usb-storage: usb_stor_bulk_transfer_buf: xfer 31 bytes
usb-storage: Status code 0; transferred 31/31
usb-storage: -- transfer complete
usb-storage: Bulk command transfer result=0
usb-storage: usb_stor_bulk_transfer_sglist: xfer 1048576 bytes, 32 entries
usb-storage: Status code 0; transferred 1048576/1048576
usb-storage: -- transfer complete
usb-storage: Bulk data transfer result 0x0
usb-storage: Attempting to get CSW...
usb-storage: usb_stor_bulk_transfer_buf: xfer 13 bytes
usb-storage: Status code 0; transferred 13/13
usb-storage: -- transfer complete
usb-storage: Bulk status result = 0
usb-storage: Bulk Status S 0x53425355 T 0x439ca R 0 Stat 0x0
usb-storage: scsi cmd done, result=0x0
usb-storage: *** thread sleeping.
usb-storage: queuecommand called
usb-storage: *** thread awakened.
usb-storage: Command WRITE_10 (10 bytes)
usb-storage:  2a 00 00 ce 2a 00 00 02 00 00
usb-storage: Bulk Command S 0x43425355 T 0x439cb L 1048576 F 0 Trg 0 LUN 0 CL 10
usb-storage: usb_stor_bulk_transfer_buf: xfer 31 bytes
usb-storage: Status code 0; transferred 31/31
usb-storage: -- transfer complete
usb-storage: Bulk command transfer result=0
usb-storage: usb_stor_bulk_transfer_sglist: xfer 1048576 bytes, 32 entries
usb 1-1: sg_complete, unlink --> -19
usb 1-1: sg_complete, unlink --> -19
usb 1-1: sg_complete, unlink --> -19
usb 1-1: sg_complete, unlink --> -19
usb 1-1: sg_complete, unlink --> -19
usb 1-1: sg_complete, unlink --> -19
usb 1-1: sg_complete, unlink --> -19
usb 1-1: sg_complete, unlink --> -19
usb 1-1: sg_complete, unlink --> -19
usb 1-1: sg_complete, unlink --> -19
usb 1-1: sg_complete, unlink --> -19
...

Debug: sleeping function called from invalid context at
include/linux/rwsem.h:66
Call Trace:
 [<c0121f16>] __might_sleep+0x5f/0x72
 [<c0280ec0>] scsi_device_put+0x3c/0xfe
 [<df9179bc>] sg_remove+0x146/0x1bd [sg]
 [<c024db2f>] class_interface_unregister+0x7c/0xc0
 [<df91a7b3>] exit_sg+0x17/0x56 [sg]
 [<c013b533>] sys_delete_module+0x18e/0x1d1
 [<c0151695>] sys_munmap+0x58/0x77
 [<c010b46d>] sysenter_past_esp+0x52/0x71
 
bad: scheduling while atomic!
Call Trace:
 [<c012018e>] schedule+0x57a/0x57f
 [<c01251d6>] printk+0x180/0x1cc
 [<c01eca67>] rwsem_down_write_failed+0xa3/0x15c
 [<c010c279>] dump_stack+0x1e/0x22
 [<c0281189>] .text.lock.scsi+0xa7/0xce
 [<df9179bc>] sg_remove+0x146/0x1bd [sg]
 [<c024db2f>] class_interface_unregister+0x7c/0xc0
 [<df91a7b3>] exit_sg+0x17/0x56 [sg]
 [<c013b533>] sys_delete_module+0x18e/0x1d1
 [<c0151695>] sys_munmap+0x58/0x77
 [<c010b46d>] sysenter_past_esp+0x52/0x71
...

Re: sg MiB writes scheduling while atomic

[Pat LaVarre]

By the way, I notice I can quickly repeat the first couple of lines of
Call Trace if I interrupt the default 0.25 MiB writes of sg_dd with
Ctrl+C:

> http://www.tldp.org/HOWTO/SCSI-Generic-HOWTO/sg3_utils.html
> Appendix A. Sg3_utils package
> variants of the Unix dd command:
> sg_dd, sgp_dd, sgq_dd and sgm_dd, ...
> ...
> http://www.torque.net/sg/
> http://www.torque.net/sg/p/sg3_utils-1.05.tgz
> $ tar -zx ...
> $ cd sg3_utils-1.05
> $ make
> $ sudo ./sg_scan -i
> $ sudo ./sg_dd of=/dev/sg1 if=/dev/zero bs=2k
> ^C

Where I quote ^C I mean I use Ctrl+C to interrupt sg i/o.  I see I
provoke such dmsg as:

...
usb-storage: Command WRITE_10 (10 bytes)
usb-storage:  2a 00 00 00 0f 80 00 00 80 00
usb-storage: Bulk Command S 0x43425355 T 0x1dfbf L 262144 F 0 Trg 0 LUN 0 CL 10
usb-storage: usb_stor_bulk_transfer_buf: xfer 31 bytes
usb-storage: Status code 0; transferred 31/31
usb-storage: -- transfer complete
usb-storage: Bulk command transfer result=0
usb-storage: usb_stor_bulk_transfer_sglist: xfer 262144 bytes, 8 entries
usb-storage: Status code 0; transferred 262144/262144
usb-storage: -- transfer complete
usb-storage: Bulk data transfer result 0x0
usb-storage: Attempting to get CSW...
usb-storage: usb_stor_bulk_transfer_buf: xfer 13 bytes
usb-storage: Status code 0; transferred 13/13
usb-storage: -- transfer complete
usb-storage: Bulk status result = 0
usb-storage: Bulk Status S 0x53425355 T 0x1dfbf R 0 Stat 0x0
usb-storage: scsi cmd done, result=0x0
usb-storage: *** thread sleeping.
usb-storage: queuecommand called
usb-storage: *** thread awakened.
usb-storage: Command WRITE_10 (10 bytes)
usb-storage:  2a 00 00 00 10 00 00 00 80 00
usb-storage: Bulk Command S 0x43425355 T 0x1dfc0 L 262144 F 0 Trg 0 LUN 0 CL 10
usb-storage: usb_stor_bulk_transfer_buf: xfer 31 bytes
usb-storage: Status code 0; transferred 31/31
usb-storage: -- transfer complete
usb-storage: Bulk command transfer result=0
usb-storage: usb_stor_bulk_transfer_sglist: xfer 262144 bytes, 8 entries
usb-storage: Status code 0; transferred 262144/262144
usb-storage: -- transfer complete
usb-storage: Bulk data transfer result 0x0
usb-storage: Attempting to get CSW...
usb-storage: usb_stor_bulk_transfer_buf: xfer 13 bytes
usb-storage: Status code 0; transferred 13/13
usb-storage: -- transfer complete
usb-storage: Bulk status result = 0
usb-storage: Bulk Status S 0x53425355 T 0x1dfc0 R 0 Stat 0x0
usb-storage: scsi cmd done, result=0x0
usb-storage: *** thread sleeping.
Debug: sleeping function called from invalid context at include/linux/rwsem.h:66
Call Trace:
 [<c0121f16>] __might_sleep+0x5f/0x72
 [<c0280ec0>] scsi_device_put+0x3c/0xfe
 [<df9263e3>] sg_cmd_done+0x19e/0x266 [sg]
 [<c0280ba6>] scsi_finish_command+0x78/0xb0
 [<c0280ad2>] scsi_softirq+0xb4/0xd9
 [<c0128c0b>] do_softirq+0xd3/0xd5
 [<c012912f>] ksoftirqd+0xbf/0xf8
 [<c0129070>] ksoftirqd+0x0/0xf8
 [<c0109245>] kernel_thread_helper+0x5/0xb

Another time:

Debug: sleeping function called from invalid context at include/linux/rwsem.h:66
Call Trace:
 [<c0121f16>] __might_sleep+0x5f/0x72
 [<c0280ec0>] scsi_device_put+0x3c/0xfe
 [<df9263e3>] sg_cmd_done+0x19e/0x266 [sg]
 [<c021fa59>] read_chan+0x0/0x98c
 [<c0280ba6>] scsi_finish_command+0x78/0xb0
 [<c0280ad2>] scsi_softirq+0xb4/0xd9
 [<c0128c0b>] do_softirq+0xd3/0xd5
 [<c011b4a0>] smp_apic_timer_interrupt+0xd8/0x140
 [<c010beae>] apic_timer_interrupt+0x1a/0x20

Re: sg MiB writes scheduling while atomic

[Douglas Gilbert]

Pat LaVarre wrote:
> By the way, I notice I can quickly repeat the first couple of lines of
> Call Trace if I interrupt the default 0.25 MiB writes of sg_dd with
> Ctrl+C:
> 
> 
>>http://www.tldp.org/HOWTO/SCSI-Generic-HOWTO/sg3_utils.html
>>Appendix A. Sg3_utils package
>>variants of the Unix dd command:
>>sg_dd, sgp_dd, sgq_dd and sgm_dd, ...
>>...
>>http://www.torque.net/sg/
>>http://www.torque.net/sg/p/sg3_utils-1.05.tgz
>>$ tar -zx ...
>>$ cd sg3_utils-1.05
>>$ make
>>$ sudo ./sg_scan -i
>>$ sudo ./sg_dd of=/dev/sg1 if=/dev/zero bs=2k
>>^C
> 
> 
> Where I quote ^C I mean I use Ctrl+C to interrupt sg i/o.  I see I
> provoke such dmsg as:

Pat,
Thanks for bringing this up.

A user process is doing IO and the user decides to kill the
process while a command is "in flight". After the user hits
Ctrl+C, sg_close() is called and the sg driver lets it go
through (i.e. doesn't hold the user process in an unkillable
wait). Since the sg driver knows a command done notification
is coming (callback to sg_cmd_done()) then it keeps all its
related instances alive (including the scsi_device instance)
and notes this situation by setting its sfp->close flag.

When the callback arrives, the respone is discarded and if
this is the last outstanding command on this device then
scsi_device_put(sdp->device) is called. scsi_device_put()
isn't designed to be called from such a context. However sg
(and all other upper level drivers I suspect) have no suitable
context to call scsi_device_put() from in this situation.

Holding the sg_close() until the response arrives is an ugly
solution. Any better suggestions?

Doug Gilbert

Re: sg MiB writes scheduling while atomic

[Matthew Wilcox]

On Tue, Sep 16, 2003 at 04:35:07PM +1000, Douglas Gilbert wrote:
> A user process is doing IO and the user decides to kill the
> process while a command is "in flight". After the user hits
> Ctrl+C, sg_close() is called and the sg driver lets it go
> through (i.e. doesn't hold the user process in an unkillable
> wait). Since the sg driver knows a command done notification
> is coming (callback to sg_cmd_done()) then it keeps all its
> related instances alive (including the scsi_device instance)
> and notes this situation by setting its sfp->close flag.
> 
> When the callback arrives, the respone is discarded and if
> this is the last outstanding command on this device then
> scsi_device_put(sdp->device) is called. scsi_device_put()
> isn't designed to be called from such a context. However sg
> (and all other upper level drivers I suspect) have no suitable
> context to call scsi_device_put() from in this situation.
> 
> Holding the sg_close() until the response arrives is an ugly
> solution. Any better suggestions?

scsi_device_put() calls down_write() so it needs to be called in process
context.  Probably the best way to do this is by calling schedule_work().
An ickier solution would be to hijack an error handling thread ;-)

-- 
"It's not Hollywood.  War is real, war is primarily not about defeat or
victory, it is about death.  I've seen thousands and thousands of dead bodies.
Do you think I want to have an academic debate on this subject?" -- Robert Fisk

Re: sg MiB writes scheduling while atomic

[Christoph Hellwig]

On Tue, Sep 16, 2003 at 12:42:38PM +0100, Matthew Wilcox wrote:
> scsi_device_put() calls down_write() so it needs to be called in process
> context.  Probably the best way to do this is by calling schedule_work().
> An ickier solution would be to hijack an error handling thread ;-)

The down_write was recently introduced and also hurts my scsi_device
refcounting patches badly.  I'll submit a patch make scsi_device_{get,put}
nonblocking again ASAP.

Re: sg MiB writes scheduling while atomic

[Pat LaVarre]

linux-2.6.0-test7 I like better.  As this thread detailed back in
mid-September, if I boot to -test4, Ctrl+C does reliably spew dmesg if
pressed during:

sg_dd of=/dev/sg0 if=/dev/zero bs=2k

-test7 with much the same .config accepts Ctrl+C without complaint.

Pat LaVarre

Re: [PATCH] mount -w of dvd+rw etc. in vanilla 2.6

[Pat LaVarre]

Seemingly linux disagrees with itself over how writable profiles are:

# /sbin/blockdev --setro /dev/scd1
# /sbin/blockdev --getro /dev/scd1
1
# dd of=/dev/scd1 if=/dev/zero bs=2K count=1
dd: opening `/dev/scd1': Read-only file system
#

So far so good, but then:

# /sbin/blockdev --setrw /dev/scd1
# /sbin/blockdev --getro /dev/scd1
0
# dd of=/dev/scd1 if=/dev/zero bs=2K count=1
dd: opening `/dev/scd1': Read-only file system
#

That is, no matter if --getro is 0 or 1, still we do not write.  The
default I see here is 0, but still I cannot write.

Pat LaVarre

Re: [PATCH] mount -w of dvd+rw etc. in vanilla 2.6

[Patrick Mansfield]

On Wed, Sep 10, 2003 at 02:08:40PM -0600, Pat LaVarre wrote:

> # /sbin/blockdev --setro /dev/scd1
> # /sbin/blockdev --getro /dev/scd1
> 1
> # dd of=/dev/scd1 if=/dev/zero bs=2K count=1
> dd: opening `/dev/scd1': Read-only file system
> #
> 
> So far so good, but then:
> 
> # /sbin/blockdev --setrw /dev/scd1
> # /sbin/blockdev --getro /dev/scd1
> 0
> # dd of=/dev/scd1 if=/dev/zero bs=2K count=1
> dd: opening `/dev/scd1': Read-only file system
> #
> 
> That is, no matter if --getro is 0 or 1, still we do not write.  The
> default I see here is 0, but still I cannot write.

Should sd.c and sr.c be calling set_device_ro (after adding a read only
block device, or changing back to read/write)?

-- Patrick Mansfield

max GiB written per boot

[Pat LaVarre]

> ... always ... an oops ... Must be fixed.

Here's another "kernel NULL pointer dereference", for we of linux-scsi
to fix, again brought to us courtesy the [usb-storage] mailing list  ...

Specifically I tried trivially repeating writes in 2.6.0-test5:

date ; sync
date ; dd of=/dev/scd1 if=/dev/zero bs=1M
date ; sync
...

Two days and five hours later, my 77th write completed normally:
dd: writing `/dev/scd1': No space left on device

But five minutes after that, my 78th write and all my following writes
completed abnormally:
dd: opening `/dev/scd1': No such device or address

Some hours after that I collected the following `tail` of a `dmesg`:

Then I rebooted.

Pat LaVarre

...
lost page write due to I/O error on sr1
SCSI error : <2 0 0 0> return code = 0x10000
end_request: I/O error, dev sr1, sector 54044664
Buffer I/O error on device sr1, logical block 6755583
lost page write due to I/O error on sr1
Unable to handle kernel NULL pointer dereference at virtual address
00000000
 printing eip:
c01e818f
*pde = 00000000
Oops: 0000 [#1]
CPU:    0
EIP:    0060:[<c01e818f>]    Not tainted
EFLAGS: 00010282
EIP is at get_kobj_path_length+0x19/0x30
eax: 00000000   ebx: 00000000   ecx: ffffffff   edx: cb0eaef4
esi: 00000015   edi: 00000000   ebp: c9eb5e88   esp: c9eb5e7c
ds: 007b   es: 007b   ss: 0068
Process dd (pid: 29851, threadinfo=c9eb4000 task=de3dace0)
Stack: c9eb4000 000000a5 d2c1b900 c9eb5ed0 c01e834f c03f5b60 c17149ac
000000a5 
       d2c1b880 c1714800 c9eb5eb8 cddfb400 d2c1b880 cddfb419 c03ba0a0
c0383d13 
       00000000 c1714988 c17149ac d36778ac c1714800 c9eb5ee8 c01e881d
c037dfa5 
Call Trace:
 [<c01e834f>] kset_hotplug+0x15e/0x2b0
 [<c01e881d>] kobject_del+0x66/0x6d
 [<c02470fc>] device_del+0x72/0x98
 [<c027b92a>] scsi_device_put+0xc9/0xe7
 [<df9088d6>] cdrom_release+0x8c/0x105 [cdrom]
 [<c0164fb6>] blkdev_put+0x1df/0x20b
 [<c015d0c3>] __fput+0x123/0x135
 [<c015b79b>] filp_close+0x57/0x81
 [<c015b846>] sys_close+0x81/0xc7
 [<c010b409>] sysenter_past_esp+0x52/0x71

Code: f2 ae f7 d1 49 8b 52 24 8d 74 31 01 85 d2 75 e7 5b 89 f0 5e 
 <6>usb 1-2: USB disconnect, address 5

unsolicited sense in 2.6.0-test5 usb-storage.ko

[Pat LaVarre]

In linux-2.6.0-test5, I see sg3_utils-1.05/sg_dd plugs 'n play more than
does sgp_dd.

A contributing factor seems to be usb/storage injecting an unsolicited
request sense injected as if it were an auto sense, despite bCSWStatus =
x00 Passed.

Specifically I tried the following, except where you see ^D in my
terminal log I actually pressed Ctrl+D:

$
$ sudo ./sg_dd of=/dev/sg1 bs=2k bpt=512
^D
0+0 records in
0+0 records out
$
$ sudo ./sgp_dd of=/dev/sg1 bs=2k bpt=512
read capacity: Driver_status=0x08 (DRIVER_SENSE,SUGGEST_OK)
Current, Sense key: No Sense
[valid=0] Info fld=0x0, Raw sense data (in hex):
  70 00 00 00 00 00 00 18 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  00 00 00 00 00 00 00 00
Unable to read capacity on /dev/sg1
Couldn't calculate count, please give one
$

CONFIG_USB_STORAGE_DEBUG=y gives contrasting dmesg traces.

For the unhappy sgp_dd we have 8 = Di < Hi = x40 = 64:

> usb-storage: queuecommand called
> usb-storage: *** thread awakened.
> usb-storage: Command READ_CAPACITY (10 bytes)
> usb-storage:  25 00 00 00 00 00 00 00 00 00
> usb-storage: Bulk Command S 0x43425355 T 0x2539c L 64 F 128 Trg 0 LUN 0 CL 10
> usb-storage: usb_stor_bulk_transfer_buf: xfer 31 bytes
> usb-storage: Status code 0; transferred 31/31
> usb-storage: -- transfer complete
> usb-storage: Bulk command transfer result=0
> usb-storage: usb_stor_bulk_transfer_buf: xfer 64 bytes
> usb-storage: Status code 0; transferred 8/64
> usb-storage: -- short transfer
> usb-storage: Bulk data transfer result 0x1
> usb-storage: Attempting to get CSW...
> usb-storage: usb_stor_bulk_transfer_buf: xfer 13 bytes
> usb-storage: Status code -32; transferred 0/13
> usb-storage: clearing endpoint halt for pipe 0xc0010280
> usb-storage: usb_stor_control_msg: rq=01 rqtype=02 value=0000 index=82 len=0
> usb-storage: usb_stor_clear_halt: result = 0
> usb-storage: Attempting to get CSW (2nd try)...
> usb-storage: usb_stor_bulk_transfer_buf: xfer 13 bytes
> usb-storage: Status code 0; transferred 13/13
> usb-storage: -- transfer complete
> usb-storage: Bulk status result = 0
> usb-storage: Bulk Status S 0x53425355 T 0x2539c R 56 Stat 0x0
> usb-storage: -- unexpectedly short transfer
> usb-storage: Issuing auto-REQUEST_SENSE
> usb-storage: Bulk Command S 0x43425355 T 0x8002539c L 18 F 128 Trg 0 LUN 0 CL 6
> usb-storage: usb_stor_bulk_transfer_buf: xfer 31 bytes
> usb-storage: Status code 0; transferred 31/31
> usb-storage: -- transfer complete
> usb-storage: Bulk command transfer result=0
> usb-storage: usb_stor_bulk_transfer_buf: xfer 18 bytes
> usb-storage: Status code 0; transferred 18/18
> usb-storage: -- transfer complete
> usb-storage: Bulk data transfer result 0x0
> usb-storage: Attempting to get CSW...
> usb-storage: usb_stor_bulk_transfer_buf: xfer 13 bytes
> usb-storage: Status code 0; transferred 13/13
> usb-storage: -- transfer complete
> usb-storage: Bulk status result = 0
> usb-storage: Bulk Status S 0x53425355 T 0x8002539c R 0 Stat 0x0
> usb-storage: -- Result from auto-sense is 0
> usb-storage: -- code: 0x70, key: 0x0, ASC: 0x0, ASCQ: 0x0
> usb-storage: (Unknown Key): (unknown ASC/ASCQ)
> usb-storage: scsi cmd done, result=0x0
> usb-storage: *** thread sleeping.

For the happy sg_dd we have Di = Hi = 8:

> usb-storage: queuecommand called
> usb-storage: *** thread awakened.
> usb-storage: Command READ_CAPACITY (10 bytes)
> usb-storage:  25 00 00 00 00 00 00 00 00 00
> usb-storage: Bulk Command S 0x43425355 T 0x2539a L 8 F 128 Trg 0 LUN 0 CL 10
> usb-storage: usb_stor_bulk_transfer_buf: xfer 31 bytes
> usb-storage: Status code 0; transferred 31/31
> usb-storage: -- transfer complete
> usb-storage: Bulk command transfer result=0
> usb-storage: usb_stor_bulk_transfer_buf: xfer 8 bytes
> usb-storage: Status code 0; transferred 8/8
> usb-storage: -- transfer complete
> usb-storage: Bulk data transfer result 0x0
> usb-storage: Attempting to get CSW...
> usb-storage: usb_stor_bulk_transfer_buf: xfer 13 bytes
> usb-storage: Status code 0; transferred 13/13
> usb-storage: -- transfer complete
> usb-storage: Bulk status result = 0
> usb-storage: Bulk Status S 0x53425355 T 0x2539a R 0 Stat 0x0
> usb-storage: scsi cmd done, result=0x0
> usb-storage: *** thread sleeping.
> usb-storage: queuecommand called
> usb-storage: *** thread awakened.
> usb-storage: Command WRITE_10 (10 bytes)
> usb-storage:  2a 00 00 00 00 00 00 00 00 00
> usb-storage: Bulk Command S 0x43425355 T 0x2539b L 0 F 0 Trg 0 LUN 0 CL 10
> usb-storage: usb_stor_bulk_transfer_buf: xfer 31 bytes
> usb-storage: Status code 0; transferred 31/31
> usb-storage: -- transfer complete
> usb-storage: Bulk command transfer result=0
> usb-storage: Attempting to get CSW...
> usb-storage: usb_stor_bulk_transfer_buf: xfer 13 bytes
> usb-storage: Status code 0; transferred 13/13
> usb-storage: -- transfer complete
> usb-storage: Bulk status result = 0
> usb-storage: Bulk Status S 0x53425355 T 0x2539b R 0 Stat 0x0
> usb-storage: scsi cmd done, result=0x0
> usb-storage: *** thread sleeping.

Pat LaVarre

Re: [usb-storage] unsolicited sense in 2.6.0-test5 usb-storage.ko

[Alan Stern]

On 10 Sep 2003, Pat LaVarre wrote:

> A contributing factor seems to be usb/storage injecting an unsolicited
> request sense injected as if it were an auto sense, despite bCSWStatus =
> x00 Passed.

> CONFIG_USB_STORAGE_DEBUG=y gives contrasting dmesg traces.
> 
> For the unhappy sgp_dd we have 8 = Di < Hi = x40 = 64:

> For the happy sg_dd we have Di = Hi = 8:

The reason for the unsolicited REQUEST-SENSE is that usb-storage expects
that certain commands will never yield a short transfer, so if they do it
must mean something has gone wrong, even if Status = 0.  READ-CAPACITY is
one of those commands.  Since sgp_dd asked for a 64-byte READ-CAPACITY, it
gets what it deserves.

Alan Stern

Re: [usb-storage] unsolicited sense in 2.6.0-test5 usb-storage.ko

[Pat LaVarre]

> The reason for the unsolicited REQUEST-SENSE ...

Help, lost me.

I'm thinking usb-storage auto sense should always and only follow
bCSWStatus = x01 Failed.  I'm thinking unsolicited auto sense should
occur only if the layers above request such an abuse - never
automagically.

I'm thinking not injecting unsolicited auto sense is a part of the
unwritten specification that "everyone knows" helps hosts get along with
devices.

No?  Where have I gone wrong?

> usb-storage expects that certain commands will
> never yield a short transfer, so if they do it
> must mean something has gone wrong,

I agree something has gone wrong, indeed I am delighted to see that
linux usb-storage detects that something has gone wrong.

> Since sgp_dd asked for a 64-byte
> READ-CAPACITY, it gets what it deserves.

Ah, here we educate me.

I should prepare and forward a patch to correct sgp_dd?

We the community now broadly accept this rule?  We say apps own the job
of keeping the bus "on the thin diagonal" i.e. asking for precisely the
expected count of data bytes?

I expect we can do better, but already I stumbled into a patch that
works (-: because I needed one :-):

--- sg3_utils-1.05/sgp_dd.c	2003-05-29 03:00:48.000000000 -0600
+++ sg3_utils/sgp_dd.c	2003-09-10 14:52:40.408420816 -0600
@@ -244,6 +244,9 @@
     io_hdr.mx_sb_len = sizeof(sense_b);
     io_hdr.dxfer_direction = SG_DXFER_FROM_DEV;
     io_hdr.dxfer_len = sizeof(rcBuff);
+#if 1 // PEL
+    io_hdr.dxfer_len = 8;
+#endif
     io_hdr.dxferp = rcBuff;
     io_hdr.cmdp = rcCmdBlk;
     io_hdr.sbp = sense_b;

Pat LaVarre

Re: [usb-storage] unsolicited sense in 2.6.0-test5 usb-storage.ko

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 1214 bytes --]

On Wed, Sep 10, 2003 at 03:24:09PM -0600, Pat LaVarre wrote:
> 
> > The reason for the unsolicited REQUEST-SENSE ...
> 
> Help, lost me.
> 
> I'm thinking usb-storage auto sense should always and only follow
> bCSWStatus = x01 Failed.  I'm thinking unsolicited auto sense should
> occur only if the layers above request such an abuse - never
> automagically.
> 
> I'm thinking not injecting unsolicited auto sense is a part of the
> unwritten specification that "everyone knows" helps hosts get along with
> devices.
> 
> No?  Where have I gone wrong?

Experience tells us that this is a flawed algorithm.

> > Since sgp_dd asked for a 64-byte
> > READ-CAPACITY, it gets what it deserves.
> 
> Ah, here we educate me.
> 
> I should prepare and forward a patch to correct sgp_dd?
> 
> We the community now broadly accept this rule?  We say apps own the job
> of keeping the bus "on the thin diagonal" i.e. asking for precisely the
> expected count of data bytes?

Yes.

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

Somebody call an exorcist!
					-- Dust Puppy
User Friendly, 5/16/1998

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [usb-storage] unsolicited sense in 2.6.0-test5 usb-storage.ko

[Pat LaVarre]

> > > The reason for the unsolicited REQUEST-SENSE ...
> > 
> > Help, lost me.
> > 
> > I'm thinking usb-storage auto sense should
> > always and only follow bCSWStatus = x01
> > Failed.  I'm thinking unsolicited auto sense
> > should occur only if the layers above request
> > such an abuse - never automagically.
> > 
> > I'm thinking not injecting unsolicited auto
> > sense is a part of the unwritten specification
> > that "everyone knows" helps hosts get along
> > with devices.
> > 
> > No?  Where have I gone wrong?
>
> Experience tells us that this is a flawed algorithm.

I don't mean to be stubborn - but I do not understand.

Are you saying we have experience of devices that work Better if we the
host assault them with unsolicited request sense?  Personally I have
precisely the opposite history of pain: I know of Scsi transports that
die if abused with an unsolicited request sense.

I know bInterfaceProtocol = x01 CB pretty much requires unsolicited
request sense.  I mean to be asking about bInterfaceProtocol = x50 BBB.

> > ... I should prepare and forward
> > a patch to correct sgp_dd? ...
> > ...
>
> Yes.

Clear now, thanks, will do.

Pat LaVarre

Re: [usb-storage] unsolicited sense in 2.6.0-test5 usb-storage.ko

[Pat LaVarre]

> > > ... I should prepare and forward
> > > a patch to correct sgp_dd? ...
> > > ...
> >
> > Yes.
> 
> Clear now, thanks, will do.

Below appears a patch to make sg3_utils sgp_dd.c as polite as sg_dd.c or
sgm_dd.c.  I know now, this patch sets Hi = Di in sgp_dd.c precisely as
sg_dd.c and sgp_dd.c do.

This fixes my actual problem, but leaves me mystified over why we
include in drivers/usb/storage/ any code path that assaults a
bInterfaceProtocol = x50 BBB device with unsolicited sense. 

Pat LaVarre

diff -ur sg3_utils-1.05/sgp_dd.c sg3_utils/sgp_dd.c
--- sg3_utils-1.05/sgp_dd.c	2003-05-29 03:00:48.000000000 -0600
+++ sg3_utils/sgp_dd.c	2003-09-11 18:05:54.013907720 -0600
@@ -59,6 +59,7 @@
 /* #define SG_DEBUG */
 
 #define SENSE_BUFF_LEN 32       /* Arbitrary, could be larger */
+#define READ_CAP_REPLY_LEN 8
 #define DEF_TIMEOUT 60000       /* 60,000 millisecs == 60 seconds */
 
 #define SGP_READ10 0x28
@@ -234,7 +235,7 @@
 {
     int res;
     unsigned char rcCmdBlk [10] = {READ_CAPACITY, 0, 0, 0, 0, 0, 0, 0,
0, 0};
-    unsigned char rcBuff[64];
+    unsigned char rcBuff[READ_CAP_REPLY_LEN];
     unsigned char sense_b[64];
     struct sg_io_hdr io_hdr;
 

Re: [usb-storage] unsolicited sense in 2.6.0-test5 usb-storage.ko

[Pat LaVarre]

Retransmitted with less line breaks.  That's me wrong thrice now in two
days, ouch sorry.  This time I kept the [usb-storage] prefix, since that
is the Subject of the google archived thread where all the replies have
appeared.

> > > ... I should prepare and forward
> > > a patch to correct sgp_dd? ...
> > > ...
> >
> > Yes.
> 
> Clear now, thanks, will do.

Below appears a patch to make sg3_utils sgp_dd.c work precisely as hard
to avoid nonzero residue as sg_dd.c and sgm_dd.c already do.

This fixes my actual problem, but leaves me mystified over why we
include in drivers/usb/storage/ any code path that assaults a
bInterfaceProtocol = x50 BBB device with unsolicited sense. 

Pat LaVarre

diff -ur sg3_utils-1.05/sgp_dd.c sg3_utils/sgp_dd.c
--- sg3_utils-1.05/sgp_dd.c	2003-05-29 03:00:48.000000000 -0600
+++ sg3_utils/sgp_dd.c	2003-09-11 18:05:54.013907720 -0600
@@ -59,6 +59,7 @@
 /* #define SG_DEBUG */
 
 #define SENSE_BUFF_LEN 32       /* Arbitrary, could be larger */
+#define READ_CAP_REPLY_LEN 8
 #define DEF_TIMEOUT 60000       /* 60,000 millisecs == 60 seconds */
 
 #define SGP_READ10 0x28
@@ -234,7 +235,7 @@
 {
     int res;
     unsigned char rcCmdBlk [10] = {READ_CAPACITY, 0, 0, 0, 0, 0, 0, 0, 0, 0};
-    unsigned char rcBuff[64];
+    unsigned char rcBuff[READ_CAP_REPLY_LEN];
     unsigned char sense_b[64];
     struct sg_io_hdr io_hdr;
 

Re: [usb-storage] unsolicited sense in 2.6.0-test5 usb-storage.ko

[Douglas Gilbert]

Matthew Dharm wrote:
> On Wed, Sep 10, 2003 at 03:24:09PM -0600, Pat LaVarre wrote:
>
 > <snip/>
> 
>>>Since sgp_dd asked for a 64-byte
>>>READ-CAPACITY, it gets what it deserves.
>>
>>Ah, here we educate me.
>>
>>I should prepare and forward a patch to correct sgp_dd?
>>
>>We the community now broadly accept this rule?  We say apps own the job
>>of keeping the bus "on the thin diagonal" i.e. asking for precisely the
>>expected count of data bytes?
> 
> 
> Yes.

There is a new (beta) version of sg3_utils-1.05.tgz on
http://www.torque.net/sg
that corrects this problem in sgp_dd.


Changelog for sg3_utils-1.05 [20030916]
   - sgp_dd: reduce READ CAPACITY response size to 8 bytes
   - utils/hxascdmp: add utility for displaying ASCII hex
   - sg_readcap: show total size in bytes, MB, GB
   - sg_logs: read log pages twice (first get response length), for
     fragile HBAs
   - sg_modes: fix core dump when corrupted response, don't print
     extra pages
   - sg_iovec.tst: added to examples, for testing sg_iovec
   - sg_map: increase sg device scanning from 128 to 256


Doug Gilbert

Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[Patrick Mansfield]

On Wed, Sep 10, 2003 at 12:23:43PM -0400, Alan Stern wrote:

> Is there any feeling about how to handle these ongoing problems with the 
> mode-sense cache page?  There doesn't seem to be any general solution that 
> can work with all USB storage devices.  Some hang when asked to read the 
> entire page; some hang when asked to read just part of the page; some hang 
> when asked to read just the page header.
> 
> What do the SCSI folk have to say about it?

There was at least this thread last April:

http://marc.theaimsgroup.com/?t=104839042700001&r=1&w=2

Matthew had a simple patch so that emulated hosts did not do any (sd.c)
mode-sense cache checks, James had a filter commands patch.

There were changes since then in how the MODE SENSE is handled (sd.c
only?).

In current 2.6 code, the scsi_host emulated flag does nothing, and can
be deleted.

I don't like per-host (or transport) flags or filters that cannot be
modified. They make some sense for hosts that process the scsi commands
(like a RAID card). If you moved a device from one transport to another,
the commands sent to the device should not change: for example, you move
an iSCSI attached device onto your local system via SPI.

IMO we ought to go with two new BFLAGS, one to block the MODE SENSE page 8
(for the cache), and one to block the MODE SENSE page 0x3f (to check the
write protect). 

A per host bflags could be added, and overwritten by per device (devinfo)
settings. I'd rather we black list the broken ones but that does not
appear to be practical. usb-storage could then set the per-host bflag to
include the two new flags.

We still end up sending some non-popular SCSI commands to all SCSI
devices.

Such flags would not stop sg from sending those commands, but that is
postive as well as negative in that you can override what the kernel is
doing via user space (sort of what Pat LaVarre experienced).

Also mentioned in the above thread:

The EVPD code has been removed, and is not an issue.

There is now code to set BLIST_INQUIRY_36 (we can still potentially hang
broken devices before it is set, but you should get a warning, and then
could modify devinfo on the boot/module command line or via proc devinfo).

-- Patrick Mansfield

Re: [usb-storage] Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[Pat LaVarre]

> http://marc.theaimsgroup.com/?t=104839042700001&r=1&w=2

Thanks, there I find 44 posts dated since 2003-03 with Subject's like:
"PATCH: exclude certain commands from emulated SCSI hosts".

By the way, Google yields a slightly different set of 44 posts:
http://groups.google.com/groups?threadm=linux.scsi.20030322193705.C17056%40one-eyed-alien.net

In both archives I think I see much sense:

> > From: Matthew Dharm <mdharm-usb () one-eyed-alien ! net> ...
> >
> > (a) don't ask for EVPD data, ...
> > (b) assume ... only 36-bytes of INQUIRY data, ...
> > (c) ... assume write-enabled for all disk-like media.
>
> From: Linus Torvalds <torvalds () transmeta ! com>
>
> suggest ... make the Linux SCSI layer
> ... use the sequences of commands that "that other OS" uses ...
> "don't use commands that haven't gotten much testing" ...

I haven't yet reviewed the rest.

Pat LaVarre

Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[Alan Stern]

On Wed, 10 Sep 2003, Patrick Mansfield wrote:

> On Wed, Sep 10, 2003 at 12:23:43PM -0400, Alan Stern wrote:
> 
> > Is there any feeling about how to handle these ongoing problems with the 
> > mode-sense cache page?  There doesn't seem to be any general solution that 
> > can work with all USB storage devices.  Some hang when asked to read the 
> > entire page; some hang when asked to read just part of the page; some hang 
> > when asked to read just the page header.
> > 
> > What do the SCSI folk have to say about it?

> I don't like per-host (or transport) flags or filters that cannot be
> modified. They make some sense for hosts that process the scsi commands
> (like a RAID card). If you moved a device from one transport to another,
> the commands sent to the device should not change: for example, you move
> an iSCSI attached device onto your local system via SPI.
> 
> IMO we ought to go with two new BFLAGS, one to block the MODE SENSE page 8
> (for the cache), and one to block the MODE SENSE page 0x3f (to check the
> write protect). 
> 
> A per host bflags could be added, and overwritten by per device (devinfo)
> settings. I'd rather we black list the broken ones but that does not
> appear to be practical. usb-storage could then set the per-host bflag to
> include the two new flags.

What do people think of just having per-device flags that the host driver
could set during the slave_configure() callback?  The point of these flags
is not to prevent bad commands from being sent to the device --
user-generated commands sent via sg should always be allowed.  Rather, the
point is to prevent sd.c from generating these commands in the first
place.  (Apparently the commands don't present a problem for sr.c.)

So for example, usb-storage could set the BFLAG to block MODE-SENSE page 8
for any disk-type device.  This isn't a perfect solution; consider an
iSCSI-attached device that is actually a usb-storage disk on some server.  
Nevertheless, this might take care of the majority of the problems we see.  
(I haven't seen any MODE-SENSE page 0x3f problems, but others have.)

Alan Stern

Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[James Bottomley]

On Thu, 2003-09-11 at 16:05, Alan Stern wrote:
> What do people think of just having per-device flags that the host driver
> could set during the slave_configure() callback?  The point of these flags
> is not to prevent bad commands from being sent to the device --
> user-generated commands sent via sg should always be allowed.  Rather, the
> point is to prevent sd.c from generating these commands in the first
> place.  (Apparently the commands don't present a problem for sr.c.)

I'm not in principle opposed to this.  That's essentially what Andries
did for the mode sense 6 vs 10 problem.

> So for example, usb-storage could set the BFLAG to block MODE-SENSE page 8
> for any disk-type device.  This isn't a perfect solution; consider an
> iSCSI-attached device that is actually a usb-storage disk on some server.  
> Nevertheless, this might take care of the majority of the problems we see.  
> (I haven't seen any MODE-SENSE page 0x3f problems, but others have.)

They can't be BLIST flags, they have to be flags in the struct
scsi_device, but they're easy to add as long as we get a definitive list
of what's necessary.

James

Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[Alan Stern]

On 11 Sep 2003, James Bottomley wrote:

> On Thu, 2003-09-11 at 16:05, Alan Stern wrote:
> > What do people think of just having per-device flags that the host driver
> > could set during the slave_configure() callback?  The point of these flags
> > is not to prevent bad commands from being sent to the device --
> > user-generated commands sent via sg should always be allowed.  Rather, the
> > point is to prevent sd.c from generating these commands in the first
> > place.  (Apparently the commands don't present a problem for sr.c.)
> 
> I'm not in principle opposed to this.  That's essentially what Andries
> did for the mode sense 6 vs 10 problem.
> 
> > So for example, usb-storage could set the BFLAG to block MODE-SENSE page 8
> > for any disk-type device.  This isn't a perfect solution; consider an
> > iSCSI-attached device that is actually a usb-storage disk on some server.  
> > Nevertheless, this might take care of the majority of the problems we see.  
> > (I haven't seen any MODE-SENSE page 0x3f problems, but others have.)
> 
> They can't be BLIST flags, they have to be flags in the struct
> scsi_device, but they're easy to add as long as we get a definitive list
> of what's necessary.

James and Matt:

Please tell us what you think of the two patches below.  The first patch 
introduces 2 bitflags: one causes sd.c to skip the mode-sense request for 
page 0x08 and the other causes it to skip the request for page(s) 0x3f.

The other patch makes usb-storage set the page-8 flag for disk-type
devices.  I didn't set the page-3f flag because I haven't noticed that
causing much problems.  If Matt wants to add it that will be easy to do;
if we end up not needing it then it will be easy to remove it from the
first patch.

Alan Stern


PATCH: (as95)
===== include/scsi/scsi_device.h 1.5 vs edited =====
--- 1.5/include/scsi/scsi_device.h	Fri Sep  5 07:48:41 2003
+++ edited/include/scsi/scsi_device.h	Fri Sep 12 16:17:08 2003
@@ -85,6 +85,8 @@
 				     * because we did a bus reset. */
 	unsigned use_10_for_rw:1; /* first try 10-byte read / write */
 	unsigned use_10_for_ms:1; /* first try 10-byte mode sense/select */
+	unsigned skip_ms_page_8:1;	/* do not MODE-SENSE for page 0x08 */
+	unsigned skip_ms_page_3f:1;	/* do not MODE-SENSE for page 0x3f */
 	unsigned no_start_on_add:1;	/* do not issue start on add */
 
 	unsigned int device_blocked;	/* Device returned QUEUE_FULL. */
===== sd.c 1.56 vs edited =====
--- 1.56/drivers/scsi/sd.c	Fri Sep  5 12:16:39 2003
+++ edited/drivers/scsi/sd.c	Fri Sep 12 16:21:28 2003
@@ -1059,6 +1059,15 @@
 	struct scsi_mode_data data;
 
 	/*
+	 * Don't try to issue mode sense for page 0x3F if the device
+	 * won't like it.
+	 */
+	if (sdkp->device->skip_ms_page_3f) {
+		printk(KERN_NOTICE "%s: assuming Write Enabled\n", diskname);
+		return;
+	}
+
+	/*
 	 * First attempt: ask for all pages (0x3F), but only 4 bytes.
 	 * We have to start carefully: some devices hang if we ask
 	 * for more than is available.
@@ -1104,6 +1113,18 @@
 	const int modepage = 0x08; /* current values, cache page */
 	struct scsi_mode_data data;
 
+
+	/*
+	 * Don't try to issue mode sense for page 0x08 if the device
+	 * won't like it.
+	 */
+	if (sdkp->device->skip_ms_page_8) {
+		printk(KERN_NOTICE "%s: assuming drive cache: write through\n",
+		       diskname);
+		sdkp->WCE = 0;
+		sdkp->RCD = 0;
+		return;
+	}
 
 	/* cautiously ask */
 	res = sd_do_mode_sense(SRpnt, dbd, modepage, buffer, 4, &data);


PATCH: (as96)
===== drivers/usb/storage/scsiglue.c 1.59 vs edited =====
--- 1.59/drivers/usb/storage/scsiglue.c	Mon Jul 28 14:29:04 2003
+++ edited/drivers/usb/storage/scsiglue.c	Fri Sep 12 16:55:26 2003
@@ -68,6 +68,12 @@
 	sdev->use_10_for_ms = 1;
 	sdev->use_10_for_rw = 1;
 
+	/* For disk-type devices, skip the mode-sense query for page 8
+	 * (the caching page).  Lots of USB Mass Storage devices crash
+	 * no matter how carefully we ask. */
+	if (sdev->type == TYPE_DISK)
+		sdev->skip_ms_page_8 = 1;
+
 	/* this is to satisify the compiler, tho I don't think the 
 	 * return code is ever checked anywhere. */
 	return 0;

Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[Pat LaVarre]

> From: Matthew Dharm <mdharm-scsi () one-eyed-alien ! net>
>
> I think allowing write to read-only media is
> the only way to go.  I don't see another way
> to get write-protect status.

Does write-protect work for dvd-ram?

How?

> From: 	Alan Stern <stern@rowland.harvard.edu>
> ...
> (Apparently the commands don't present a
> problem for sr.c.)

sr.c shows this problem by getting the wrong answer.

sr.c is getting cd->device->writeable wrong for all the seven-or-so mmc
writable profiles except dvd-ram.

Some background from me and patches from Andy Polyakov and Jens Axboe
appear in:
Subject: Re: [PATCH] mount -w of dvd+rw etc. in vanilla 2.6
http://marc.theaimsgroup.com/?t=106123318400007

> From: Matthew Dharm <mdharm-scsi () one-eyed-alien ! net>
> ... I'm going to have some code that will not
> apply the filters (any filters) if the request
> comes from sg,

More old good sense!

Following are relevant posts from me that some archives before now have
not included in this thread.

Pat LaVarre

---

Subject: Re: PATCH: exclude certain commands from emulated SCSI hosts 
Date: 2003-04-23 14:45:31 PST 
From: Pat LaVarre (ppaatt@aol.com)

> http://groups.google.com/groups?group=mlist.linux.scsi
> Subject: Re: PATCH: exclude certain commands from emulated SCSI hosts 
>
> > From: Patrick Mansfield (patmans@us.ibm.com)
> > Date: 2003-04-07 15:42:13 PST 
> >
> > Would a user ever want to send a MODE SENSE
> > to USB storage device?)
>
> From: Matthew Dharm (mdharm-scsi@one-eyed-alien.net)
> Date: 2003-04-21 17:04:05 PST  
>
> a very large number of USB devices of TYPE DISK do
> not  support MODE SENSE in any form.  Apparently,
> the 'popular' OSes don't use it.

I have some reason to believe Windows often passes thru something like:
-x "1A 00 08 00 18 00" -i x18
i.e. a ModeSense6 of Cacheing page x08 for the whole 4 byte header plus
one whole 8 byte block descriptor plus the whole xC byte standard page.

But I haven't studied this closely enough to know if that's precisely
correct, nor do I know when it gets translated to op x5A ModeSense10,
nor do I know when it is faked out in the host entirely, never reaching
the bus.

> From: Alan Stern (stern@rowland.harvard.edu)
> Date: 2003-04-22 11:21:36 PST ...
>
> a comment in drivers/storage/usb/protocol.c
> indicating that 8 is the minimum size for a
> MODE-SENSE (line 257).

Perhaps we mean:
http://lxr.linux.no/source/drivers/usb/storage/protocol.c?v=2.5.56#L274
/* again, for MODE_SENSE_10, we get the minimum (8) */

I don't know of any merely public standard that admits 8 (i.e. the whole
standard header) is a minimum.

I do know Win 2K kernel dies if a root-privileged app is so rude as to
express an x5A ModeSense10 ModeSense for less than 8 as op x1A
ModeSense6:
http://support.microsoft.com/?kbid=813908
SCSI Pass-Through Mode Sense Command May Crash the Computer

> > From: Matthew Dharm (mdharm-scsi@one-eyed-alien.net)
> > Date: 2003-04-05 11:40:24 PST 
> >
> > (1) can't filter TEST UNIT READY (opcode 0).
> > Not a big deal, but a theoretical problem.
> >
> From: James Bottomley (James.Bottomley@steeleye.com)
> Date: 2003-04-05 12:00:18 PST 
>
> TUR has been mandatory since SCSI-1, ...

A zeroed six byte CDB has been mandatory.  Other forms of TUR are less
widely supported.
...

-----Forwarded Message-----

From: p.lavarre@ieee.org
To: patmans@us.ibm.com
Cc: stern@rowland.harvard.edu, mdharm-usb@one-eyed-alien.net,
usb-storage@one-eyed-alien.net, linux-scsi@vger.kernel.org
Subject: Re: [usb-storage] Re: [linux-usb-devel] [2.6-test] Bug in
usb-storage or scsi?
Date: 11 Sep 2003 14:04:54 -0600

> http://marc.theaimsgroup.com/?t=104839042700001&r=1&w=2

Thanks, there I find 44 posts dated since 2003-03 with Subject's like:
"PATCH: exclude certain commands from emulated SCSI hosts".

By the way, Google yields [an interestingly] different set of 44 posts:
http://groups.google.com/groups?threadm=linux.scsi.20030322193705.C17056%40one-eyed-alien.net

In both archives I think I see much sense:

> > From: Matthew Dharm <mdharm-usb () one-eyed-alien ! net> ...
> >
> > (a) don't ask for EVPD data, ...
> > (b) assume ... only 36-bytes of INQUIRY data, ...
> > (c) ... assume write-enabled for all disk-like media.
>
> From: Linus Torvalds <torvalds () transmeta ! com>
>
> suggest ... make the Linux SCSI layer
> ... use the sequences of commands that "that other OS" uses ...
> "don't use commands that haven't gotten much testing" ...
...

Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[Alan Stern]

On 11 Sep 2003, Pat LaVarre wrote:

> > From: 	Alan Stern <stern@rowland.harvard.edu>
> > ...
> > (Apparently the commands don't present a
> > problem for sr.c.)
> 
> sr.c shows this problem by getting the wrong answer.
> 
> sr.c is getting cd->device->writeable wrong for all the seven-or-so mmc
> writable profiles except dvd-ram.

Getting a wrong answer isn't quite as bad as crashing the peripheral's 
firmware.  My proposed solution is meant to attack the problems of USB 
mass storage devices that crash when asked to report MODE-SENSE page 8, 
possible also when asked for page 0x3f.

Alan Stern

Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[Pat LaVarre]

> From: Alan Stern <stern@rowland.harvard.edu>
>
> the MODE SENSE page 0x3f (to check the write
> protect) ... (I haven't seen ... problems,
> but others have.)

I imagine page x3F is an sd issue only?

I remember Ansi says the mode sense header reports writability only for
pdt x00 hdd.

I remember pdt x05 dvd/cd devices misbehaving when their available x3f
AllPages data exceeded the xFF bytes that x1A MODE_SENSE can copy,
though x5A MODE_SENSE_10 worked fine.

And I often repeat, I remember clearly that the popular OS doesn't often
ask only for a four-byte x1A MODE_SENSE header, because I saw that
actually doing that crashes Win 2K kernels up thru SP3, if Windows has
concluded the device needs the host to translate to x5A MODE_SENSE_10.

Pat LaVarre

Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[Pat LaVarre]

> From: Patrick Mansfield <patmans@us.ibm.com>
> ...
> last April: ... changes since then
> in how the MODE SENSE is handled (sd.c only?)

Anybody know more?

> If you moved a device from one transport to
> another, the commands sent to the device
> should not change: for example, you move an
> iSCSI attached device onto your local system
> via SPI.

Eh?

I thought in linux we have competing cdb authors on purpose.  For
example, how sr decides if a device is writable differs from how ide-cd
decides if a device is writable.  I thought that the powers that be like
linux that way.

No?

Having multiple cdb authors necessarily means that the cdb's passed thru
one kind of transport or another do differ any time the multiple
versions of copy-edited authoring code differ.

In particular, people designing an unusual device to work with linux
have to repeat their work: once for sr, again for ide-cd, and so on.

No?

Pat LaVarre

Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[Patrick Mansfield]

On Fri, Sep 12, 2003 at 12:43:14PM -0600, Pat LaVarre wrote:
> 
> > From: Patrick Mansfield <patmans@us.ibm.com>
> > ...
> > last April: ... changes since then
> > in how the MODE SENSE is handled (sd.c only?)
> 
> Anybody know more?

The mode sense 6 vs 10 that James mentioned, search the code for
use_10_for_ms, used in scsi_lib.c, set to 0 in scsi_scan.c and set to 1 by
drivers/usb/storage/scsiglue.c.

> > If you moved a device from one transport to
> > another, the commands sent to the device
> > should not change: for example, you move an
> > iSCSI attached device onto your local system
> > via SPI.
> 
> Eh?

More specifically, if you are using some LLDD or transport X as a bridge
to another transport Y, we might set flags - maybe what commands we can
send - relative to transport X, when they might be inappropriate for
transport Y.

Examples:

If USB (usb-storage) bridged to a remote device that is actually connected
via SPI, today we will try and send 10 byte MODE SENSE commands to the
device.

If iSCSI bridged to a remote USB device, we will only send 6 byte MODE
SENSE commands.

There are also devices that work with FCP or SPI, but those generally
don't have borken devices like the USB storage ones.

> I thought in linux we have competing cdb authors on purpose.  For
> example, how sr decides if a device is writable differs from how ide-cd
> decides if a device is writable.  I thought that the powers that be like
> linux that way.
> 
> No?

> Having multiple cdb authors necessarily means that the cdb's passed thru
> one kind of transport or another do differ any time the multiple
> versions of copy-edited authoring code differ.
> 
> In particular, people designing an unusual device to work with linux
> have to repeat their work: once for sr, again for ide-cd, and so on.
> 
> No?

I don't get your point - I was talking about the transport (LLDD), not the
actual scsi device or upper level drivers.

Code used for different upper level scsi devices should not be duplicated,
I don't know ide or cd enough to comment in that area, except that there
is common cd code (struct cdrom_device_ops).

sr could probably do some of the same things as sd to figure out if a
device is writable, but we are best maintaining current algorithms (don't
have sr send new mode sense commands) to avoid borken devices.

-- Patrick Mansfield

Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[Pat LaVarre]

> I don't get your point ...
> 
> Code used for different upper level scsi
> devices should not be duplicated, ...

Re whether a pdt x05 dvd/cd device is writable, I think I see
duplication among ide-cd, ide-scsi, and sr.  I hear ide-scsi is dying,
but that ide-cd and sr plan to continue competing.

For example:

sr.c get_capabilities decides CDC_DVD_RAM and cd->device->writeable by
fetching & parsing as raw hex the 2Ah Capabilities mode page via
scsi_lib.c scsi_mode_sense of page x2a, sending op MODE_SENSE = x1a or
MODE_SENSE_10 = x5a.

ide-cd.c ide_cdrom_probe_capabilities decides CDC_DVD_RAM (also stored
as CDROM_CONFIG_FLAGS(drive)->dvd_ram) by fetching & parsing as a
asm/byteorder.h packed struct the 2Ah Capabilities mode page via
ide_cdrom_get_capabilities via cdrom_mode_sense of page
GPMODE_CAPABILITIES_PAGE = x2a, always sending op GPCMD_MODE_SENSE_10 =
x5a.

Neither yet proceeds to try op 46h GET_CONFIGURATION to discover the
other six writable mmc profiles.

> talking about the transport (LLDD),
> not the actual scsi device or upper level drivers ...

Thank you for your patience, sorry at first I did not understand how
narrowly we mean "transport" here.

> use_10_for_ms ...
> set to 0 in scsi_scan.c ...
> set to 1 by drivers/usb/storage/scsiglue.c ...
>
> transport X as a bridge to another transport Y,
> flags ... maybe what commands we can send ...
> relative to transport X,
> when they might be inappropriate for transport Y ...
> Examples ...

Clear to me now (and cool) thank you.

> I don't know ide or cd enough ...
> except ...there is common cd code (struct cdrom_device_ops).

Aye, helpful.

Pat LaVarre

Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?

[Pat LaVarre]

I see Google didn't thread my first try properly, sorry.  Here I am
again with an edited subject line.

-----Forwarded Message-----

From: Pat LaVarre <p.lavarre@ieee.org>
To: stern@rowland.harvard.edu
Cc: mdharm-usb@one-eyed-alien.net, usb-storage@one-eyed-alien.net, linux-scsi@vger.kernel.org
Subject: Re: [usb-storage] Re: [linux-usb-devel] [2.6-test] Bug in usb-storage or scsi?
Date: 10 Sep 2003 12:16:31 -0600


> What do the SCSI folk have to say about it?

Anyone?

> > for 2.6.0 ... drivers/scsi/sd.c ...
> > sd_read_cache_type ...
> > Get rid of (or #ifdef out) most ...
>
> Is there any feeling about how to handle these
> ongoing problems with the mode-sense cache
> page?

Yes, much.

Maybe more feeling if you remind/teach me what breaks if we don't fetch
page 8 re Cache.  First I imagine automounters mount read-only disks rw,
and then the write cache gets errors while flushing.  I wonder what else
breaks.  Maybe thruput varies for devices that do or do not bother to
claim they have write cache.

Maybe more feeling if you can say how in/accurate page 8 re Cache
commonly is.  For example, I know of no clear & broadly adopted standard
that requires a device to declare write-behind or background defect map
updates as a form of write cache.

> There doesn't seem to be any general solution
> that can work with all USB storage devices.

I understand you to be saying we can settle for an sd solution, we don't
mind if the sd heuristic we discover then doesn't apply to pdt x05
dvd/cd.

> some hang when asked to read just the page
> header.
> ...
> some hang when asked to read just part of the
> page;
> ...
> Some hang when asked to read the entire page;

Device folk ship bugs, aye, same as host folk.

Are not two of these three failures disproportionately common?

Me, I would expect asking anything but what Windows asks would fail most
often.  Is that not so?  Do we know what Windows asks?  Do we know
if/how Win ME/ 2K/ XP differ here?  Have we matched CBWCB and
dCBWDataTransferLength in every bit?

Pat LaVarre