Re: [linux-usb-devel] Re: USB storage problems on OHCI..

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Alan Stern]

This problem has been cropping up with many, many USB storage devices.  
They just don't handle MODE-SENSE page 8 correctly.  Some devices are okay
with a 128-byte transfer and others aren't.  Some are okay with a 64-byte
transfer and others aren't.  Some are okay transferring the actual size of
the page (as given in the header) and others aren't.  Some are okay
transferring the minimum amount we actually need (12 bytes IIRC) and
others aren't.

Although it may not directly address all the problems that Linus observed,
a solution has been floating around in the linux-scsi and usb-storage
development lists.  It looks to me like it's ready to be merged, although
no one has done so yet.  Basically, it amounts to a per-device flag that
can be inherited by default from the host template or set/cleared
specifically by the host driver, rather like the flag for using 10-byte
READ/WRITE commands.  The flag tells sd.c not to ask for page 8 but to
assume a write-through cache.  The patch is in

http://marc.theaimsgroup.com/?l=linux-scsi&m=106366112221507&w=2

At the moment, this MODE-SENSE page 8 is probably the most severe 
outstanding problem with the usb-storage driver.  I'm all in favor of the 
patch being adopted (hint, hint...).

I've been assuming that the "babble" problem arises because the device
either tries to send more than 128 bytes of mode-sense data or because it
tries to send the data and the subsequent 13-byte status in the same
packet.  Whether or not this would be detected as an error depends on the
maxpacket size, which depends on the speed of the USB bus.  (Maybe the
device even tries to send an over-sized packet.)  Of course, it's always
possible that reading page 8 just triggers a bug in the device firmware,
and from then on it's all over.

On Mon, 22 Sep 2003, Matthew Dharm wrote:

> You say this worked with UHCI 'some time ago'?  Perhaps that was before all
> this mode page 8 stuff got settled?

The MODE-SENSE page 8 stuff was added sometime during the 2.5 development 
cycle.  Before it was put in, quite likely the device would have worked 
fine with UHCI.

Alan Stern

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Christoph Hellwig]

On Mon, Sep 22, 2003 at 10:25:28AM -0400, Alan Stern wrote:
> http://marc.theaimsgroup.com/?l=linux-scsi&m=106366112221507&w=2
> 
> At the moment, this MODE-SENSE page 8 is probably the most severe 
> outstanding problem with the usb-storage driver.  I'm all in favor of the 
> patch being adopted (hint, hint...).

Patch looks mostly fine to me, but please all flags should be unsigned instead
of signed and scsi_devinfo.h needs some inclusion guards.

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Linus Torvalds]

On Mon, 22 Sep 2003, Alan Stern wrote:
>
> This problem has been cropping up with many, many USB storage devices.  

Interesting data-point: the device is a happy EHCI camper, and is totally 
able to read codepage 8 on EHCI.

However, if I put it behind a USB-1 hub on the EHCI port, I see the same 
problems I saw with OHCI.

So it is somehow related to USB-1 vs USB-2. I don't understand why the 
device would make a difference for something like mode page 8, but it 
looks like it transfers data fine for _small_ mode page requests under 
USB-1, and under USB-2 it works even for big ones.

		Linus

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Alan Stern]

On Mon, 22 Sep 2003, Christoph Hellwig wrote:

> On Mon, Sep 22, 2003 at 03:31:24PM +0100, Christoph Hellwig wrote:
> > Patch looks mostly fine to me, but please all flags should be unsigned instead
> > of signed and scsi_devinfo.h needs some inclusion guards.
> 
> Actually I think it could be made much simpler by killing the per-template
> bflags and just setting the scsi_device flags from slave_alloc or
> slave_configure.  Could you cook up a patch like that?

Interestingly, my original patch was a lot like you describe.  It can be 
found in

http://marc.theaimsgroup.com/?l=linux-scsi&m=106340167723122&w=2

Patrick Mansfield later beefed it up to the version you were looking at.

I don't care which version of the patch gets accepted, so long as
_something_ is done.  Patrick, what do you think?

Alan Stern

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Linus Torvalds]

[ Andries added to the cc ]

On Mon, 22 Sep 2003, Patrick Mansfield wrote:
> 
> I can modify the patch for that.

How about just making the sd.c layer more robust? That has worked well in 
the past, and it seems wrong to have to add more and more flags.

In particular, while hunting down the usb-1/usb-2 differences, it became
clear that all the mode sense transfers that _worked_ under usb-1 were
short. With 64 bytes than with 128 bytes it no longer babbled, and the
whole thing worked fine with 11 bytes (the minimum required to get the
caching status).

Adding some printk's showed that the initial (successful) 4-byte page read
resulted in:

 - page len=1538 (pretty obviously crap)
 - header len=8 (correct)
 - block descriptor len=0 (correct)

and the thing is, we were using the "pretty obviously crap" value for 
judging the size of the thing.

So my suggestion would be to just replace that usage (in sd.c):

	len = data.length;
	if (len > 128)
		len = 128;

with something like this instead:

	/* We need three bytes past the block descriptor length */
	len = data.header_length + data.block_descriptor_length + 3;

	/* Sanity check */
	if (len > data.len || len < 11 || len > 128)
		.. fail ..

Basically, Andries Brouwer's strategy of making sd.c more conservative has
been a very successful one in the past. Why not continue on that?

(Btw, where are the different mode sense pages documented?)

		Linus

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Linus Torvalds]

On Mon, 22 Sep 2003, David Brownell wrote:
>
> Linus Torvalds wrote:
> > Interesting data-point: the device is a happy EHCI camper, and is
> > totally able to read codepage 8 on EHCI.
> > 
> > However, if I put it behind a USB-1 hub on the EHCI port, I see the
> > same problems I saw with OHCI.
> 
> Can you be more clear?

That is with the exact same external port on the computer, but with an
added external (old) USB 1 hub in between. I have no idea how the internal
EHCI logic switching ends up working, so I'll just append the full
/proc/bus/usb/devices output for you to judge.

(this is from a kernel that actually is able to access the device
correctly, by virtue of the sd.c change I outlined in the previous email).

			Linus

----

T:  Bus=04 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=12  MxCh= 2
B:  Alloc=  0/900 us ( 0%), #Int=  0, #Iso=  0
D:  Ver= 1.10 Cls=09(hub  ) Sub=00 Prot=00 MxPS= 8 #Cfgs=  1
P:  Vendor=0000 ProdID=0000 Rev= 2.06
S:  Manufacturer=Linux 2.6.0-test5 uhci-hcd
S:  Product=UHCI Host Controller
S:  SerialNumber=0000:00:1d.2
C:* #Ifs= 1 Cfg#= 1 Atr=40 MxPwr=  0mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   2 Ivl=255ms

T:  Bus=03 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=12  MxCh= 2
B:  Alloc= 11/900 us ( 1%), #Int=  1, #Iso=  0
D:  Ver= 1.10 Cls=09(hub  ) Sub=00 Prot=00 MxPS= 8 #Cfgs=  1
P:  Vendor=0000 ProdID=0000 Rev= 2.06
S:  Manufacturer=Linux 2.6.0-test5 uhci-hcd
S:  Product=UHCI Host Controller
S:  SerialNumber=0000:00:1d.1
C:* #Ifs= 1 Cfg#= 1 Atr=40 MxPwr=  0mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   2 Ivl=255ms

T:  Bus=03 Lev=01 Prnt=01 Port=01 Cnt=01 Dev#=  2 Spd=12  MxCh= 4
D:  Ver= 1.00 Cls=09(hub  ) Sub=00 Prot=00 MxPS= 8 #Cfgs=  1
P:  Vendor=0451 ProdID=1446 Rev= 1.00
C:* #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=100mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   1 Ivl=255ms

T:  Bus=03 Lev=02 Prnt=02 Port=01 Cnt=01 Dev#=  4 Spd=12  MxCh= 0
D:  Ver= 2.00 Cls=00(>ifc ) Sub=00 Prot=00 MxPS=64 #Cfgs=  1
P:  Vendor=05e3 ProdID=0703 Rev= 0.04
S:  Product=USB TO IDE
C:* #Ifs= 1 Cfg#= 1 Atr=c0 MxPwr= 96mA
I:  If#= 0 Alt= 0 #EPs= 2 Cls=08(stor.) Sub=06 Prot=50 Driver=usb-storage
E:  Ad=81(I) Atr=02(Bulk) MxPS=  64 Ivl=0ms
E:  Ad=02(O) Atr=02(Bulk) MxPS=  64 Ivl=0ms

T:  Bus=02 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=12  MxCh= 2
B:  Alloc=  0/900 us ( 0%), #Int=  0, #Iso=  0
D:  Ver= 1.10 Cls=09(hub  ) Sub=00 Prot=00 MxPS= 8 #Cfgs=  1
P:  Vendor=0000 ProdID=0000 Rev= 2.06
S:  Manufacturer=Linux 2.6.0-test5 uhci-hcd
S:  Product=UHCI Host Controller
S:  SerialNumber=0000:00:1d.0
C:* #Ifs= 1 Cfg#= 1 Atr=40 MxPwr=  0mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   2 Ivl=255ms

T:  Bus=02 Lev=01 Prnt=01 Port=01 Cnt=01 Dev#=  2 Spd=12  MxCh= 0
D:  Ver= 1.00 Cls=00(>ifc ) Sub=00 Prot=00 MxPS= 8 #Cfgs=  1
P:  Vendor=04f9 ProdID=0100 Rev= 1.00
C:* #Ifs= 3 Cfg#= 1 Atr=40 MxPwr=  0mA
I:  If#= 0 Alt= 0 #EPs= 2 Cls=07(print) Sub=01 Prot=02 Driver=usblp
E:  Ad=01(O) Atr=02(Bulk) MxPS=  64 Ivl=0ms
E:  Ad=82(I) Atr=02(Bulk) MxPS=  16 Ivl=0ms
I:  If#= 1 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=ff Driver=(none)
E:  Ad=03(O) Atr=02(Bulk) MxPS=  16 Ivl=0ms
E:  Ad=84(I) Atr=02(Bulk) MxPS=  64 Ivl=0ms
I:  If#= 2 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=ff Driver=(none)
E:  Ad=05(O) Atr=02(Bulk) MxPS=  16 Ivl=0ms
E:  Ad=86(I) Atr=02(Bulk) MxPS=  16 Ivl=0ms

T:  Bus=01 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480 MxCh= 6
B:  Alloc=  0/800 us ( 0%), #Int=  0, #Iso=  0
D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=01 MxPS= 8 #Cfgs=  1
P:  Vendor=0000 ProdID=0000 Rev= 2.06
S:  Manufacturer=Linux 2.6.0-test5 ehci_hcd
S:  Product=EHCI Host Controller
S:  SerialNumber=0000:00:1d.7
C:* #Ifs= 1 Cfg#= 1 Atr=40 MxPwr=  0mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   2 Ivl=256ms

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Alan Stern]

On Mon, 22 Sep 2003, Patrick Mansfield wrote:

> On Mon, Sep 22, 2003 at 11:50:47AM -0400, Alan Stern wrote:
> 
> > I don't care which version of the patch gets accepted, so long as
> > _something_ is done.  Patrick, what do you think?
> 
> I would rather we can modify the flags for any broken device and override
> a host's setting for known functioning devices. But let's get some patch
> applied.

Yes.  But be sure to allow host drivers to adjust the flag within
slave_config(); slave_alloc() may be too early.  (Although since sd.c is 
the only driver that uses the flag, I guess it doesn't matter.)

> If there is a problem (per Linus' post) in the USB transport this isn't a
> real fix.

The problems that Linus and many other people are seeing aren't in the USB 
transport; they are bugs in the device firmware.  These bugs manifest as 
errors reported by the transport drivers.

Alan Stern

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Christoph Hellwig]

On Mon, Sep 22, 2003 at 08:49:30AM -0700, Patrick Mansfield wrote:
> On Mon, Sep 22, 2003 at 04:11:04PM +0100, Christoph Hellwig wrote:
> > On Mon, Sep 22, 2003 at 03:31:24PM +0100, Christoph Hellwig wrote:
> > > Patch looks mostly fine to me, but please all flags should be unsigned instead
> > > of signed and scsi_devinfo.h needs some inclusion guards.
> > 
> > Actually I think it could be made much simpler by killing the per-template
> > bflags and just setting the scsi_device flags from slave_alloc or
> > slave_configure.  Could you cook up a patch like that?
> 
> You mean add the flags to scsi_device rather than scsi_host?
> And allow setting the sdev->flags only in slave_alloc.

What I meant is not adding a blist flags member at all but rather
setting skip_ms_page_8 and skip_ms_page_3f from ->slave_alloc.

But I probably missed something obvious :)

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Patrick Mansfield]

On Mon, Sep 22, 2003 at 05:37:32PM +0100, Christoph Hellwig wrote:
> 
> What I meant is not adding a blist flags member at all but rather
> setting skip_ms_page_8 and skip_ms_page_3f from ->slave_alloc.
> 
> But I probably missed something obvious :)

The current code allows us to set or clear a given bit, but not both. So
if we set them in slave_alloc, they can't be cleared without adding
other flags or code.

-- Patrick Mansfield

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Martin Diehl]

On Mon, 22 Sep 2003, David Brownell wrote:

> > So it is somehow related to USB-1 vs USB-2. I don't understand why the 
> > device would make a difference for something like mode page 8, but it 
> > looks like it transfers data fine for _small_ mode page requests under 
> > USB-1, and under USB-2 it works even for big ones.
> 
> It could be some kind of logic in the device forgetting that to use
> the right maxpacket setting in that specific case.

Just a wild guess: If the device misbehaved on HS/FS transition and would 
send packets >64 bytes in FS mode, I think it would explain the babble 
issue:

* the 128 byte read which reported babble was the first IN transfer >64 
  byte and the babble condition was reported after receiving 64 bytes, 
  i.e. at a point where the HC has stored _less_ data than expected!

* limiting the transfer length to 64 bytes made the babble disappear

* works with EHCI because HS can handle bigger packets

Unfortunately I don't see an easy way to check the sourced packed size on 
the wire - except using a bus analyzer of course.

I'm not sure how uhci behaves with packets exceeding FS max packet size - 
if it doesn't fit into the read buffer it should report babble as well - 
but what if the buffer is big enough and it's just the packet is to big? 
IIRC the TD's actlen is 11-bit anyway. Could it be this device (mis)uses 
HS maxpacket when operating at fullspeed?

Martin

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[David Brownell]

Martin Diehl wrote:

> Unfortunately I don't see an easy way to check the sourced packed size on 
> the wire - except using a bus analyzer of course.

Right.  There are not-easy ways to do this, involving tricked out
host controller drivers usable only to debug things like this, but
I wouldn't want to go there myself!  :)

- Dave

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Christoph Hellwig]

On Mon, Sep 22, 2003 at 09:44:04AM -0700, Patrick Mansfield wrote:
> The current code allows us to set or clear a given bit, but not both. So
> if we set them in slave_alloc, they can't be cleared without adding
> other flags or code.

If we want drivers to mess with blist flags that's the more general
solution, yes.  But the blist flags really are a target thing and
I'd prefer to keep host drivers a bit away from this.  Of course
this doesn't really work for the usb case where the host driver only
deals with emulated targets that are all completly hosed.

Maybe sdev->scsi_level should recognize a new level, SCSI_TOTALLY_B0RKED
for those usb-storage devices where the vendor somehow heard of the
spec but nothing that isn't excercised by the windows drivers has
the slightest chance of working..

> 
> -- Patrick Mansfield
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
---end quoted text---

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Linus Torvalds]

On Mon, 22 Sep 2003, Patrick Mansfield wrote:
> On Mon, Sep 22, 2003 at 09:09:47AM -0700, Linus Torvalds wrote:
> > 
> > (Btw, where are the different mode sense pages documented?)
> 
> The latest SCSI 3 draft standards are online as follows, the SCSI 2 specs
> are also online at www.t10.org (.ORG!).
> 
> For block (applies to some USB mass storage) specific pages:
> 
> http://www.t10.org/ftp/t10/drafts/sbc/sbc-r08c.pdf
> 
> See page 105 section 7.1.3 of the above (table 73 the "Mode page codes").

Ok, thanks. In particular, it seems to be pointless to read anything past 
byte 20 - nothing past there is even defined.

What's the general sense of things - for a random SCSI device with bugs 
(and they all have _some_ sort of bugs, let's not just rain on USB here), 
is it saner to try to read just the bytes we need (3 bytes: page code, 
page size and the cache bits), or the full 20 bytes?

		Linus

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[David Brownell]

Linus Torvalds wrote:
> On Mon, 22 Sep 2003, David Brownell wrote:
> 
>>In this case, because it's not "EHCI + USB 2.0 hub", it's still using
>>the OHCI companion controller.  So that wasn't a new case.
> 
> 
> Ok. Here's the broken case with an added usb-2 hub in between. It is 
> indeed a bit different.

Yes.  Instead of getting 64/128 bytes transferred, with -EOVERFLOW
reported, it gets 0/128, with -EPIPE.  And the recovery got a bit
confused; it's a STALL but not from the device, it's from the hub.

It might be worth making EHCI use a different code in that case,
if this class of TT-related errors starts to be trouble.


> Again, this case worked fine with the sd.c changes, so it does seem to be 
> all related to "big" transfers out of the mode page.

Or at any rate, "big enough" to confuse the device.

Thanks for the additional test results.

- Dave

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Linus Torvalds]

On Mon, 22 Sep 2003, David Brownell wrote:
> > Again, this case worked fine with the sd.c changes, so it does seem to be 
> > all related to "big" transfers out of the mode page.
> 
> Or at any rate, "big enough" to confuse the device.

Yes. Additional testing (making the code just increase the size of the 
transfer until it fails) shows that a size of 63 still works, but a size 
of 64 bytes fails.

Actually - with a 64-byte transfer, we appear to get the 64 bytes ok, but 
the subsequent CSW status word read fails, so the 64-byte case seems to be 
the one that starts confusing the device.

If I remember right, USB-1 "big packets" are 64 bytes in size, and a 
64-byte bulk transfer would be two packets (one full-sized one, one 
zero-sized one). So this seems to support the notion that the device is 
fine as long as it can fit the whole bulk transfer in just one packet.

Which cleanly explains why EHCI "just worked". 

			Linus

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Alan Stern]

On Mon, 22 Sep 2003, Christoph Hellwig wrote:

> If we want drivers to mess with blist flags that's the more general
> solution, yes.  But the blist flags really are a target thing and
> I'd prefer to keep host drivers a bit away from this.  Of course
> this doesn't really work for the usb case where the host driver only
> deals with emulated targets that are all completly hosed.
> 
> Maybe sdev->scsi_level should recognize a new level, SCSI_TOTALLY_B0RKED
> for those usb-storage devices where the vendor somehow heard of the
> spec but nothing that isn't excercised by the windows drivers has
> the slightest chance of working..

Please be careful when setting this up.  The mode-sense pages are vitally
important for USB CD devices, and they all seem to work fine.  It's just
the disk-type devices that have been causing trouble.

Many of them seem to share the failure mode observed by Linus: when asked
to send 128 bytes of m-s page 8 they apparently forget that the connection
is only full-speed with a 64-byte packet maximum, and they try to send all
128 bytes in a single packet.  But other failure modes have cropped up as
well.

Alan Stern

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[James Bottomley]

On Mon, 2003-09-22 at 13:55, Andries.Brouwer@cwi.nl wrote:
>         if (sdkp->media_present) {
>                 sd_read_capacity(sdkp, disk->disk_name, sreq, buffer);
>                 if (sdp->removable)
>                         sd_read_write_protect_flag(sdkp, disk->disk_name,
>                                         sreq, buffer);
>                 sd_read_cache_type(sdkp, disk->disk_name, sreq, buffer);
>         }
> 
> and I suppose we could skip sd_read_cache_type() in the
> hot-pluggable case - a flag that USB storage could set.

what about just having a conservative mode for sd?  This could still
internally be a set of flags, just one single way of clearing them all
from slave configure.  For the most conservative setting, we could
probably dump spin up, read write protect, and read cache type.

James

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Linus Torvalds]

On 22 Sep 2003, James Bottomley wrote:
> 
> I think we could try 4 bytes for this (even to avoid wide residue
> problems) and see how it goes.

How about just trusting the size (and as far as I can tell from the SCSI 
specs, the size is the size _without_ the header and block descriptors), 
and capping it at 20?

If the device reports a size that is smaller than the required one (3), we 
just fail the request - instead of doing a small read and then making 
decisions on data that isn't even there, like we used to do!

(The old code would use the "size" as the read length argument, which
really looks fundamentally wrong. At the very least it should add in the
header length etc).

Can people try this attached patch? It looks bigger than it is - in order 
to do sane error handling for the "size is too small" case I re-organized 
the thing a bit. The real change is just

 - capping size at 20 (real value) instead of 128 (random crud)
 - properly add in the header and block descriptor sizes

Comments?

			Linus

-----
===== drivers/scsi/sd.c 1.134 vs edited =====
--- 1.134/drivers/scsi/sd.c	Wed Jun 25 04:47:26 2003
+++ edited/drivers/scsi/sd.c	Mon Sep 22 12:42:32 2003
@@ -1108,14 +1108,26 @@
 	/* cautiously ask */
 	res = sd_do_mode_sense(SRpnt, dbd, modepage, buffer, 4, &data);
 
-	if (scsi_status_is_good(res)) {
-		/* that went OK, now ask for the proper length */
-		len = data.length;
-		if (len > 128)
-			len = 128;
-		res = sd_do_mode_sense(SRpnt, dbd, modepage, buffer,
-				       len, &data);
-	}
+	if (!scsi_status_is_good(res))
+		goto bad_sense;
+
+	/* that went OK, now ask for the proper length */
+	len = data.length;
+
+	/*
+	 * We're only interested in the first three bytes, actually.
+	 * But the data cache page is defined for the first 20.
+	 */
+	if (len < 3)
+		goto bad_sense;
+	if (len > 20)
+		len = 20;
+
+	/* Take headers and block descriptors into account */
+	len += data.header_length + data.block_descriptor_length;
+
+	/* Get the data */
+	res = sd_do_mode_sense(SRpnt, dbd, modepage, buffer, len, &data);
 
 	if (scsi_status_is_good(res)) {
 		const char *types[] = {
@@ -1133,23 +1145,26 @@
 
 		printk(KERN_NOTICE "SCSI device %s: drive cache: %s\n",
 		       diskname, types[ct]);
+
+		return;
+	}
+
+bad_sense:
+	if ((SRpnt->sr_sense_buffer[0] & 0x70) == 0x70
+	     && (SRpnt->sr_sense_buffer[2] & 0x0f) == ILLEGAL_REQUEST
+	     /* ASC 0x24 ASCQ 0x00: Invalid field in CDB */
+	     && SRpnt->sr_sense_buffer[12] == 0x24
+	     && SRpnt->sr_sense_buffer[13] == 0x00) {
+		printk(KERN_NOTICE "%s: cache data unavailable\n",
+		       diskname);
 	} else {
-		if ((SRpnt->sr_sense_buffer[0] & 0x70) == 0x70
-		     && (SRpnt->sr_sense_buffer[2] & 0x0f) == ILLEGAL_REQUEST
-		     /* ASC 0x24 ASCQ 0x00: Invalid field in CDB */
-		     && SRpnt->sr_sense_buffer[12] == 0x24
-		     && SRpnt->sr_sense_buffer[13] == 0x00) {
-			printk(KERN_NOTICE "%s: cache data unavailable\n",
-			       diskname);
-		} else {
-			printk(KERN_ERR "%s: asking for cache data failed\n",
-			       diskname);
-		}
-		printk(KERN_ERR "%s: assuming drive cache: write through\n",
+		printk(KERN_ERR "%s: asking for cache data failed\n",
 		       diskname);
-		sdkp->WCE = 0;
-		sdkp->RCD = 0;
 	}
+	printk(KERN_ERR "%s: assuming drive cache: write through\n",
+	       diskname);
+	sdkp->WCE = 0;
+	sdkp->RCD = 0;
 }
 
 /**

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Andries.Brouwer]

    From James.Bottomley@SteelEye.com  Mon Sep 22 21:29:06 2003

    On Mon, 2003-09-22 at 13:55, Andries.Brouwer@cwi.nl wrote:
    >         if (sdkp->media_present) {
    >                 sd_read_capacity(sdkp, disk->disk_name, sreq, buffer);
    >                 if (sdp->removable)
    >                         sd_read_write_protect_flag(sdkp, disk->disk_name,
    >                                         sreq, buffer);
    >                 sd_read_cache_type(sdkp, disk->disk_name, sreq, buffer);
    >         }
    > 
    > and I suppose we could skip sd_read_cache_type() in the
    > hot-pluggable case - a flag that USB storage could set.

    what about just having a conservative mode for sd?  This could still
    internally be a set of flags, just one single way of clearing them all
    from slave configure.  For the most conservative setting, we could
    probably dump spin up, read write protect, and read cache type.

Maybe.

[By the way - reading Write Protect is meaningful and works,
for my Smart Media card readers.]

I have seen proposals around here for flags that are far too specific
(like "do not ask for mode page 8"). If we go to that level of detail
then we'll soon have fifty flags.
Black lists, and flags that describe various ways of being broken
are a bad idea in my opinion. I will not deny that they may be needed
in some cases, but they are never the preferred solution.

Also "conservative mode" sounds like a flag that describes some
way of being broken.

On the other hand "hot-pluggable" describes a positive asset,
and if we can conclude from that that it is unnecessary to ask
for mode page 8 then we achieve the same effect in a positive way.


There is another comment here.
A scsi device declares its level of scsi compliance.
Most USB storage devices are not very scsi compliant at all,
and report 0 there.
To everybody's surprise USB storage does

        US_DEBUGP("Fixing INQUIRY data to show SCSI rev 2 - was %d\n",
                  data_ptr[2] & 7);

        /* Change the SCSI revision number */
        data_ptr[2] = (data_ptr[2] & ~7) | 2;

It claims that the device is SCSI-2 compliant, even when the
device itself does not make that claim at all.


Suppose that we stop changing this compliance level.
Then getting SCSI-1 or no compliance level could be a
"conservative mode" flag.

[Of course this was done for a reason - USB storage was written
assuming the SCSI layer given. If we stop changing the SCSI level
that may require changes in the SCSI code. Probably Matt remembers
what the problems were.]

Andries

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Alan Stern]

On Mon, 22 Sep 2003 Andries.Brouwer@cwi.nl wrote:

> I have seen proposals around here for flags that are far too specific
> (like "do not ask for mode page 8"). If we go to that level of detail
> then we'll soon have fifty flags.
> Black lists, and flags that describe various ways of being broken
> are a bad idea in my opinion. I will not deny that they may be needed
> in some cases, but they are never the preferred solution.
> 
> Also "conservative mode" sounds like a flag that describes some
> way of being broken.
> 
> On the other hand "hot-pluggable" describes a positive asset,
> and if we can conclude from that that it is unnecessary to ask
> for mode page 8 then we achieve the same effect in a positive way.

Bravo! a capital plan!

(Does anybody remember _Utopia Limited_ these days?)

I will be just as pleased as the next man to see these little flags go 
away -- or rather, not arrive in the first place.  And as Andries says, 
there's no real point in trying to determine the cache type for a 
hot-pluggable device in the first place.  Furthermore, this will get rid 
of some annoying "Can't SYNCHRONIZE" messages from sd.c.

Alan Stern

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 2322 bytes --]

On Mon, Sep 22, 2003 at 09:56:38PM +0200, Andries.Brouwer@cwi.nl wrote:
> A scsi device declares its level of scsi compliance.
> Most USB storage devices are not very scsi compliant at all,
> and report 0 there.

Not exactly.  The reporting of 0, 1, 2, or something random in that field
appears to be completely unrelated to the capabilities of the device.  I've
found -zero- correlation between the revision level and how the device
functions.

Also, sampling my devices, I see a wide range of values reported -- saying
'most report 0' would be incorrect based on my data.

> To everybody's surprise USB storage does
> 
>         US_DEBUGP("Fixing INQUIRY data to show SCSI rev 2 - was %d\n",
>                   data_ptr[2] & 7);
> 
>         /* Change the SCSI revision number */
>         data_ptr[2] = (data_ptr[2] & ~7) | 2;
> 
> It claims that the device is SCSI-2 compliant, even when the
> device itself does not make that claim at all.
> 
> Suppose that we stop changing this compliance level.
> Then getting SCSI-1 or no compliance level could be a
> "conservative mode" flag.
> 
> [Of course this was done for a reason - USB storage was written
> assuming the SCSI layer given. If we stop changing the SCSI level
> that may require changes in the SCSI code. Probably Matt remembers
> what the problems were.]

It's been a couple of years since I looked at this.... but I positively
recall that I added this code to fix definate and clear problems.

One that is immediately obvious to me is that if the SCSI level is < 2, we
can't address anything beyond the 6-byte command limit.

Another that immediately comes to mind is that the display in
/proc/scsi/scsi would show version ffffffff

I've heard arguments from various people (including Jorge Shilling and some
device-vendor folks) that I shouldn't override this value... the argument I
used to counter that request was that things break completely if I don't
override the value; I don't have a record of what specifically breaks.

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

A:  The most ironic oxymoron wins ...
DP: "Microsoft Works"
A:  Uh, okay, you win.
					-- A.J. & Dust Puppy
User Friendly, 1/18/1998

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Pat LaVarre]

> From: Linus Torvalds <torvalds () osdl ! org>
> Date: 2003-09-22 19:55:30
> ...
> How about just trusting the size ...
> and capping it at 20?

Besides dramatic overestimates of mode sense additional length, I've
also seen the off-by-one slight overestimate and the high-byte-zeroed
dramatic underestimate.

> General MODE SENSE stuff is in:
> http://www.t10.org/ftp/t10/drafts/spc3/spc3r15.pdf

Yes, but t10 mmc often quietly contradicts & overrides t10 spc for pdt
x05 dvd/cd and for atapi pdt x00 hdd/rmb.  For t10 newbies wanting to
fetch the bleeding edge, latest final draft, oldest still available,
etc. I sometimes recommend the index:

http://www.t10.org/scsi-3.htm

> > A scsi device declares its level of scsi compliance.
> ...
> From: Matthew Dharm <mdharm-scsi () one-eyed-alien ! net>
> Date: 2003-09-22 20:51:59
> ...
> reporting of 0, 1, 2, or something random ...
> appears ... unrelated to the capabilities of the device ...

The pernicious idea of zeroing the scsi revision in byte 2 of op x12
inquiry forms part of how mmc quietly redefines spc.  Misapplying the
sff precursor of the pdt x05 mmc spec to pdt x00 devices is part of how
msft whql qualified atapi pdt x00 devices for sale thru oem's in the
late 1990's.

> > > > From: Alan Stern <stern () rowland ! harvard ! edu>
> > > > Date: 2003-09-22 14:25:28
> > > > ...
> > > > many, many USB storage devices.  
> > > > They just don't handle MODE-SENSE page 8 correctly.
> > >
> > > From: Linus Torvalds <torvalds () osdl ! org>
> > > Date: 2003-09-22 16:09:47
> > > ...
> > > How about just making the sd.c layer more robust? 
> >
> > From: Alan Stern <stern () rowland ! harvard ! edu>
> > Date: 2003-09-22 16:42:52
> > ...
> > variety of ways ... fail ... truly amazing.
>
> From:  Christoph Hellwig <hch () infradead ! org>
> Date: 2003-09-22 17:21:42
> ...
> for those usb-storage devices where ...
> nothing that isn't excercised by the windows drivers
> has the slightest chance of working..

In the sd world, usb hdd/rmb (pdt x00) may handle specifically the mode
sense of page 8 that win sends.

For usb bInterfaceSubClass = x06 TransparentScsi that cdb might be:
x 1A 00:08:00 18 00

For usb bInterfaceSubClass != x06 TransparentScsi that cdb might be:
x 5A 00 08:00:00:00 00 00:1C 00

usb dvd/cd (pdt x05) sell into the sr/ide-cd world instead, of course.

Tell me we care to hear specifically what some win sends to some of my
usb hdd/rmb, and I'll go collect a few sample usb bus traces.  I do NOT
know that the above cdb's are correct.  Those are just my guesses,
summing up little clues from years of messing around near here.  I hear
in particular that win 9x/me differs from win 2k/xp near hear.

> Subject:  Re: [linux-usb-devel] Re: USB storage problems on OHCI..
> From: Linus Torvalds <torvalds () osdl ! org>
> Date: 2003-09-22 17:41:47
> ...
> is it saner to try to read just the bytes we need
> (3 bytes: page code, page size and the cache bits),
> or the full 20 bytes?

I thought "everyone knew" sanest was to always fetch header + block
descriptors + whole pages?

Trouble is, that's a circular definition in that the host can't know
whether the device will volunteer an optional block descriptor or not. 
mmc says shall not, spc says maybe not but should.

We only have to care if we have to know how many Data bytes to expect to
copy In.

> From: Andries.Brouwer () cwi ! nl
> Date: 2003-09-22 18:55:22
> ...
> Similarly, USB storage devices tend to ...
> only systematically work
> if one does precisely what Windows does.

Yes, what ain't tested don't work.

> USB is hot pluggable,
> so it should not be necessary
> to send a flush cache command at shutdown.

Who has tested this rule?  I know I've seen ata hdd lose data in
response to a pin 1 reset issued without op x35 sync cache.  I've heard
same of dvd.  And loss of bus power can kill any write cache in a drive,
of course.

> From: James Bottomley <James.Bottomley () steeleye ! com>
> Date: 2003-09-22 17:55:09 ...
> ...
> I think we could try 4 bytes

We know that isn't talk like windows.  msdn.microsoft.com tells us the
win 2k kernel crashes if a root-privileged app asks for just the 4 byte
header from devices for which Win translates to x5A MODE_SENSE_10 from
x1A MODE_SENSE  Because the win 2k kernel crashes so reliably, we can
confidently guess no working win 2k apps talk that way.

> From: James Bottomley <James.Bottomley () steeleye ! com>
> Date: 2003-09-22 19:28:56
> ...
> sd? ... the most conservative ..., we could probably
> dump spin up, read write protect, and read cache type ...

Aye, although in sr/ide-cd we have a default of not writable to overcome
somehow.

Pat LaVarre

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Andries.Brouwer]

> Tell me we care to hear specifically what some win sends to some of my
> usb hdd/rmb, and I'll go collect a few sample usb bus traces.

I'll collect and publish.
(Change the subject line to "USB storage traces" or so.)
Must have some old traces somewhere myself too.

Andries

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[James Bottomley]

On Mon, 2003-09-22 at 14:56, Andries.Brouwer@cwi.nl wrote:
> I have seen proposals around here for flags that are far too specific
> (like "do not ask for mode page 8"). If we go to that level of detail
> then we'll soon have fifty flags.
> Black lists, and flags that describe various ways of being broken
> are a bad idea in my opinion. I will not deny that they may be needed
> in some cases, but they are never the preferred solution.
> 
> Also "conservative mode" sounds like a flag that describes some
> way of being broken.
> 
> On the other hand "hot-pluggable" describes a positive asset,
> and if we can conclude from that that it is unnecessary to ask
> for mode page 8 then we achieve the same effect in a positive way.

I disagree on this one.  hot-pluggable sounds like it should be set for
ever hotplug device (currently that would include firewire, which may be
iffy, and Fibre Channel, which has our highest level of SCSI compliance
and would definitely be wrong).

The design goal is that this flag makes sd assume as little SCSI
standards compliance as it can get away with while still operating the
device.

So call a spade "a spade"

My thought, by the way, is that it would be a callback that would clear
all the "extras" flags, and the slave_configure routine could
selectively turn them back on again if necessary.

James

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Andries.Brouwer]

    From James.Bottomley@SteelEye.com  Tue Sep 23 16:05:21 2003

    > Also "conservative mode" sounds like a flag that describes some
    > way of being broken.
    > 
    > On the other hand "hot-pluggable" describes a positive asset,
    > and if we can conclude from that that it is unnecessary to ask
    > for mode page 8 then we achieve the same effect in a positive way.

    I disagree on this one.  hot-pluggable sounds like it should be set for
    ever hotplug device (currently that would include firewire, which may be
    iffy, and Fibre Channel, which has our highest level of SCSI compliance
    and would definitely be wrong).

Why would it be wrong?

    The design goal is that this flag makes sd assume as little SCSI
    standards compliance as it can get away with while still operating the
    device.

No, the design goal of "hot-pluggable" is that it indicates that
the device can disappear any moment. Nothing at all about SCSI
compliance.

Pulling out a device while it is actively reading or writing
will probably break something. But if a device is hot-pluggable
it should be OK to pull it out when it has been inactive for
a second or so.

But if that is really true, then it should not be necessary
to send the device any "synchronise cache" commands when we
shut down.

And if no such commands will be sent anyway, then we need not ask
the device about its type of cache.

And if we do not ask, then we need not worry whether the device
is sufficiently compliant to answer such a question.

Andries

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[James Bottomley]

On Tue, 2003-09-23 at 09:37, Andries.Brouwer@cwi.nl wrote:
> No, the design goal of "hot-pluggable" is that it indicates that
> the device can disappear any moment. Nothing at all about SCSI
> compliance.

Actually, then, these are two issues...hotplug is being worked on
separately at the moment.

I thought the problem under discussion was devices which lacked SCSI
standards compliance.

> Pulling out a device while it is actively reading or writing
> will probably break something. But if a device is hot-pluggable
> it should be OK to pull it out when it has been inactive for
> a second or so.
> 
> But if that is really true, then it should not be necessary
> to send the device any "synchronise cache" commands when we
> shut down.

For a FC array, suprise unplug would be caveat emptor (possibly because
fibre connection transience is going to cause it to come back), but
notified unplug would still want to flush the cache on the assumption
the next action might be to power down the array.

James

Re: Re: USB storage problems on OHCI..

[Alan Stern]

On 23 Sep 2003, James Bottomley wrote:

> On Tue, 2003-09-23 at 09:37, Andries.Brouwer@cwi.nl wrote:
> 
> > Pulling out a device while it is actively reading or writing
> > will probably break something. But if a device is hot-pluggable
> > it should be OK to pull it out when it has been inactive for
> > a second or so.
> > 
> > But if that is really true, then it should not be necessary
> > to send the device any "synchronise cache" commands when we
> > shut down.
> 
> For a FC array, suprise unplug would be caveat emptor (possibly because
> fibre connection transience is going to cause it to come back), but
> notified unplug would still want to flush the cache on the assumption
> the next action might be to power down the array.

Is there any way to notify the system that you are about to unplug a 
drive?  It seems to me that the best approach is to flush the cache on an 
unmount.  People naturally assume that it's safe to unplug a device once 
it has been unmounted, and they also realize that it's unsafe to unplug a 
device containing a mounted filesystem.

That doesn't address the problem of raw device access, but perhaps 
whatever ioctl is used by blockdev --flushbufs can also flush the cache.

Is there any harm in sending a SYNCHRONIZE command to a device that
doesn't need it (write-through cache)?  Maybe doing that is less dangerous 
than trying to read mode-sense page 8 on these buggy USB devices.  
(Although I'm not aware of anyone who has tried the experiment.)

Alan Stern



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
linux-usb-devel@lists.sourceforge.net
To unsubscribe, use the last form field at:
https://lists.sourceforge.net/lists/listinfo/linux-usb-devel

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Linus Torvalds]

On Tue, 23 Sep 2003 Andries.Brouwer@cwi.nl wrote:
> 
> No, the design goal of "hot-pluggable" is that it indicates that
> the device can disappear any moment. Nothing at all about SCSI
> compliance.

You're talking past each other.

Server people think that "hot-pluggable" means "I will tell the system 
before it goes away". Normal users consider hot-pluggable to be "I can rip 
the thing out by hand".

I'm with the normal users - I think the server guys are really talking 
about "controlled shutdown and insert", not "hot-pluggable", but hey, the 
fact is, there's two kinds.

Trying to enumerate three values with one bit is counterproductive.

So we should _not_ have one bit that says "hotpluggable", we should have
separate bits for "should survive a unexpected disconnect" (_real_
hotplug) and "can be disconnected with operator help" (server
"pseudo-hotplug").

USB devices are obviously supposed to be able to just be ripped out, ie
"unexpected", real hotplug.  While FC etc tends to be "pseudo-hotplug".

			Linus

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Ruud Linders]

I tried the patch but it doesn't work for me using an USB-2 Memory stick
"DiskonKey" on an USB-2 port (with uhci_hcd & ehci_hcd loaded).

After a 3 minute time-out I get
     "SCSI device sda: drive cache: write through"
and the device starts working just fine.

Unloading the ehci_hcd module doesn't make any difference.
Only when ripping out the whole mode-sense call it works immediately :-)
so it appears the device is just unhappy about the sd_do_mode_sense.

_
Regards,
	Ruud Linders

Linus Torvalds wrote:
> On 22 Sep 2003, James Bottomley wrote:
> 
>>I think we could try 4 bytes for this (even to avoid wide residue
>>problems) and see how it goes.
> 
> 
> How about just trusting the size (and as far as I can tell from the SCSI 
> specs, the size is the size _without_ the header and block descriptors), 
> and capping it at 20?
> 
> If the device reports a size that is smaller than the required one (3), we 
> just fail the request - instead of doing a small read and then making 
> decisions on data that isn't even there, like we used to do!
> 
> (The old code would use the "size" as the read length argument, which
> really looks fundamentally wrong. At the very least it should add in the
> header length etc).
> 
> Can people try this attached patch? It looks bigger than it is - in order 
> to do sane error handling for the "size is too small" case I re-organized 
> the thing a bit. The real change is just
> 
>  - capping size at 20 (real value) instead of 128 (random crud)
>  - properly add in the header and block descriptor sizes
> 
> Comments?
> 
> 			Linus
> 
> -----
> ===== drivers/scsi/sd.c 1.134 vs edited =====
> --- 1.134/drivers/scsi/sd.c	Wed Jun 25 04:47:26 2003
> +++ edited/drivers/scsi/sd.c	Mon Sep 22 12:42:32 2003
> @@ -1108,14 +1108,26 @@
>  	/* cautiously ask */
>  	res = sd_do_mode_sense(SRpnt, dbd, modepage, buffer, 4, &data);
>  
> -	if (scsi_status_is_good(res)) {
> -		/* that went OK, now ask for the proper length */
> -		len = data.length;
> -		if (len > 128)
> -			len = 128;
> -		res = sd_do_mode_sense(SRpnt, dbd, modepage, buffer,
> -				       len, &data);
> -	}
> +	if (!scsi_status_is_good(res))
> +		goto bad_sense;
> +
> +	/* that went OK, now ask for the proper length */
> +	len = data.length;
> +
> +	/*
> +	 * We're only interested in the first three bytes, actually.
> +	 * But the data cache page is defined for the first 20.
> +	 */
> +	if (len < 3)
> +		goto bad_sense;
> +	if (len > 20)
> +		len = 20;
> +
> +	/* Take headers and block descriptors into account */
> +	len += data.header_length + data.block_descriptor_length;
> +
> +	/* Get the data */
> +	res = sd_do_mode_sense(SRpnt, dbd, modepage, buffer, len, &data);
>  
>  	if (scsi_status_is_good(res)) {
>  		const char *types[] = {
> @@ -1133,23 +1145,26 @@
>  
>  		printk(KERN_NOTICE "SCSI device %s: drive cache: %s\n",
>  		       diskname, types[ct]);
> +
> +		return;
> +	}
> +
> +bad_sense:
> +	if ((SRpnt->sr_sense_buffer[0] & 0x70) == 0x70
> +	     && (SRpnt->sr_sense_buffer[2] & 0x0f) == ILLEGAL_REQUEST
> +	     /* ASC 0x24 ASCQ 0x00: Invalid field in CDB */
> +	     && SRpnt->sr_sense_buffer[12] == 0x24
> +	     && SRpnt->sr_sense_buffer[13] == 0x00) {
> +		printk(KERN_NOTICE "%s: cache data unavailable\n",
> +		       diskname);
>  	} else {
> -		if ((SRpnt->sr_sense_buffer[0] & 0x70) == 0x70
> -		     && (SRpnt->sr_sense_buffer[2] & 0x0f) == ILLEGAL_REQUEST
> -		     /* ASC 0x24 ASCQ 0x00: Invalid field in CDB */
> -		     && SRpnt->sr_sense_buffer[12] == 0x24
> -		     && SRpnt->sr_sense_buffer[13] == 0x00) {
> -			printk(KERN_NOTICE "%s: cache data unavailable\n",
> -			       diskname);
> -		} else {
> -			printk(KERN_ERR "%s: asking for cache data failed\n",
> -			       diskname);
> -		}
> -		printk(KERN_ERR "%s: assuming drive cache: write through\n",
> +		printk(KERN_ERR "%s: asking for cache data failed\n",
>  		       diskname);
> -		sdkp->WCE = 0;
> -		sdkp->RCD = 0;
>  	}
> +	printk(KERN_ERR "%s: assuming drive cache: write through\n",
> +	       diskname);
> +	sdkp->WCE = 0;
> +	sdkp->RCD = 0;
>  }
>  
>  /**
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> linux-usb-devel@lists.sourceforge.net
> To unsubscribe, use the last form field at:
> https://lists.sourceforge.net/lists/listinfo/linux-usb-devel
> 

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Linus Torvalds]

On Tue, 23 Sep 2003, Ruud Linders wrote:
> 
> I tried the patch but it doesn't work for me using an USB-2 Memory stick
> "DiskonKey" on an USB-2 port (with uhci_hcd & ehci_hcd loaded).
> 
> After a 3 minute time-out I get
>      "SCSI device sda: drive cache: write through"
> and the device starts working just fine.

Is this different from a plain kernel _without_ the patch?

> Unloading the ehci_hcd module doesn't make any difference.
> Only when ripping out the whole mode-sense call it works immediately :-)
> so it appears the device is just unhappy about the sd_do_mode_sense.

Yes, I think we might want to mark USB devices as being write-through by 
default. That said, I think the patch is still better than what we have 
now (which just doesn't make any sense at all).

		Linus

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[James Bottomley]

On Tue, 2003-09-23 at 10:23, Alan Stern wrote: 
> Is there any way to notify the system that you are about to unplug a 
> drive?  It seems to me that the best approach is to flush the cache on an 
> unmount.  People naturally assume that it's safe to unplug a device once 
> it has been unmounted, and they also realize that it's unsafe to unplug a 
> device containing a mounted filesystem.
> 
> That doesn't address the problem of raw device access, but perhaps 
> whatever ioctl is used by blockdev --flushbufs can also flush the cache.

Well, a synchronize can be really expensive (minutes to flush on a large
array), you only really want to do it if absolutely necessary, so tying
it to something separate from normal OS operation seems like the best
thing to do. 

> Is there any harm in sending a SYNCHRONIZE command to a device that
> doesn't need it (write-through cache)?  Maybe doing that is less dangerous 
> than trying to read mode-sense page 8 on these buggy USB devices.  
> (Although I'm not aware of anyone who has tried the experiment.)

Devices that don't support mode page 8 invariably seem to react badly to
the SYNCHRONIZE command (I have a few of these SCSI devices)...Although,
usually they just send an ILLEGAL COMMAND sense.  I wouldn't like to try
the same thing with a USB device... 

James 

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Ruud Linders]

Linus Torvalds wrote:
> On Tue, 23 Sep 2003, Ruud Linders wrote:
> 
>>I tried the patch but it doesn't work for me using an USB-2 Memory stick
>>"DiskonKey" on an USB-2 port (with uhci_hcd & ehci_hcd loaded).
>>
>>After a 3 minute time-out I get
>>     "SCSI device sda: drive cache: write through"
>>and the device starts working just fine.
> 
> 
> Is this different from a plain kernel _without_ the patch?
> 

No difference.

> 
>>Unloading the ehci_hcd module doesn't make any difference.
>>Only when ripping out the whole mode-sense call it works immediately :-)
>>so it appears the device is just unhappy about the sd_do_mode_sense.
> 
> 
> Yes, I think we might want to mark USB devices as being write-through by 
> default. That said, I think the patch is still better than what we have 
> now (which just doesn't make any sense at all).
> 

Fully agree here, 2.4.2x doesn't appear to do this mode_sense checking
and works with all the usb-mass-storage devices I tried.


> 		Linus
> 
> 


_
Ruud

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Linus Torvalds]

On Wed, 24 Sep 2003, Ruud Linders wrote:
> > 
> > Is this different from a plain kernel _without_ the patch?
> 
> No difference.

Ok. I committed my version as "better than what is there now", but clearly
it's not good enough.

So we should really add code to sd_read_cache_type() to default to 
write-through for USB devices. The question is, what kind of flag do we 
want to use?

We already have the combination "removable && !lockable", which would make
a lot of sense. But I don't think USB even touches those flags, and
"removable" wrt USB is actually a bit ambiguous (in the USB unplug case,
it's actually the _controller_ that is removed, not the medium).

Suggestions? Just add a new flag like the "use_10_for_ms" flag?

		Linus

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Alan Stern]

On Wed, 24 Sep 2003, Linus Torvalds wrote:

> Ok. I committed my version as "better than what is there now", but clearly
> it's not good enough.
> 
> So we should really add code to sd_read_cache_type() to default to 
> write-through for USB devices. The question is, what kind of flag do we 
> want to use?
> 
> We already have the combination "removable && !lockable", which would make
> a lot of sense. But I don't think USB even touches those flags, and
> "removable" wrt USB is actually a bit ambiguous (in the USB unplug case,
> it's actually the _controller_ that is removed, not the medium).
> 
> Suggestions? Just add a new flag like the "use_10_for_ms" flag?

Here's a suggestion.  This patch (as115) implements the concepts brought
up by Andries and Linus.  It adds a flag "randomly_unpluggable", meaning
that a host or device might be disconnected at some unknown and
uncontrollable time.  The flag is present in the host template, the host
structure (copied from the template), and the device structure (inherited
from the host), and it can be changed at will.  (Remember that a USB mass
storage device is treated as a SCSI host -- in fact, some such devices
actually _are_ hosts.)

The only place the flag gets used is in sd.c, where it prevents
sd_read_cache_type() from being called.  And the only place the flag gets 
set is in the usb-storage host template.

Although not ideal, this is one of the simplest ways of doing what we
want.  One thing that should be added is some way to SYNCHRONIZE the cache
at the appropriate times.  Unfortunately, I don't know when those times
should be -- unmount of course, but what else in addition?  (Also I don't
know how to tell at the level of sd.c when an unmount has occurred.)  And
I don't know what the effect of a SYNCHRONIZE command will be on a USB
device that doesn't like it; only experimentation with different devices
can answer that.

On the other hand, perhaps it's not necessary to worry about this.  Any
hot-unpluggable device compatible with the Windows/MacOS mass market ought
to survive an impromptu disconnect without losing any data.

This patch applies to 2.6.0-test5-bk10.

Alan Stern


===== include/scsi/scsi_device.h 1.5 vs edited =====
--- 1.5/include/scsi/scsi_device.h	Fri Sep  5 07:48:41 2003
+++ edited/include/scsi/scsi_device.h	Fri Sep 26 11:41:48 2003
@@ -85,6 +85,7 @@
 				     * because we did a bus reset. */
 	unsigned use_10_for_rw:1; /* first try 10-byte read / write */
 	unsigned use_10_for_ms:1; /* first try 10-byte mode sense/select */
+	unsigned randomly_unpluggable:1;	/* May disappear at any time */
 	unsigned no_start_on_add:1;	/* do not issue start on add */
 
 	unsigned int device_blocked;	/* Device returned QUEUE_FULL. */
===== include/scsi/scsi_host.h 1.3 vs edited =====
--- 1.3/include/scsi/scsi_host.h	Thu Aug 14 18:37:32 2003
+++ edited/include/scsi/scsi_host.h	Fri Sep 26 11:40:51 2003
@@ -312,6 +312,12 @@
 	 */
 	unsigned emulated:1;
 
+	/*
+	 * True for hosts that may be unplugged at unpredictable times
+	 * (e.g. USB)
+	 */
+	unsigned randomly_unpluggable:1;
+
 	/* 
 	 * True if the driver wishes to use the generic block layer
 	 * tag queueing functions
@@ -431,6 +437,7 @@
 	unsigned unchecked_isa_dma:1;
 	unsigned use_clustering:1;
 	unsigned highmem_io:1;
+	unsigned randomly_unpluggable:1;
 	unsigned use_blk_tcq:1;
 
 	/*
===== drivers/scsi/hosts.c 1.33 vs edited =====
--- 1.33/drivers/scsi/hosts.c	Tue Sep  2 02:15:39 2003
+++ edited/drivers/scsi/hosts.c	Fri Sep 26 11:44:41 2003
@@ -236,6 +236,7 @@
 	shost->cmd_per_lun = sht->cmd_per_lun;
 	shost->unchecked_isa_dma = sht->unchecked_isa_dma;
 	shost->use_clustering = sht->use_clustering;
+	shost->randomly_unpluggable = sht->randomly_unpluggable;
 	shost->use_blk_tcq = sht->use_blk_tcq;
 
 	if (sht->max_host_blocked)
===== drivers/scsi/sd.c 1.56 vs edited =====
--- 1.56/drivers/scsi/sd.c	Fri Sep  5 12:16:39 2003
+++ edited/drivers/scsi/sd.c	Fri Sep 26 11:51:38 2003
@@ -1206,7 +1206,9 @@
 		if (sdp->removable)
 			sd_read_write_protect_flag(sdkp, disk->disk_name,
 					sreq, buffer);
-		sd_read_cache_type(sdkp, disk->disk_name, sreq, buffer);
+		if (!sdp->randomly_unpluggable)
+			sd_read_cache_type(sdkp, disk->disk_name, sreq,
+					buffer);
 	}
 		
 	set_capacity(disk, sdkp->capacity);
===== drivers/scsi/scsi_scan.c 1.50 vs edited =====
--- 1.50/drivers/scsi/scsi_scan.c	Fri Sep  5 12:16:39 2003
+++ edited/drivers/scsi/scsi_scan.c	Fri Sep 26 11:49:14 2003
@@ -632,6 +632,7 @@
 
 	sdev->use_10_for_rw = 1;
 	sdev->use_10_for_ms = 0;
+	sdev->randomly_unpluggable = sdev->host->randomly_unpluggable;
 
 	if(sdev->host->hostt->slave_configure)
 		sdev->host->hostt->slave_configure(sdev);
===== drivers/usb/storage/scsiglue.c 1.59 vs edited =====
--- 1.59/drivers/usb/storage/scsiglue.c	Mon Jul 28 14:29:04 2003
+++ edited/drivers/usb/storage/scsiglue.c	Fri Sep 26 12:11:10 2003
@@ -324,6 +324,9 @@
 	/* emulated HBA */
 	.emulated =			TRUE,
 
+	/* may be unplugged at any time */
+	.randomly_unpluggable = 	TRUE,
+
 	/* module management */
 	.module =			THIS_MODULE
 };

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Pat LaVarre]

> From: Linus Torvalds <torvalds () osdl ! org>
> Date: 2003-09-23 15:46:21
>
> Normal users consider hot-pluggable
> to be "I can rip the thing out by hand".

Yes.

Meanwhile, much (all except flash?) actual usb-storage now commercially
available is Not hot-pluggable in this sense.  Instead, "ata hdd lose
data in response to a pin 1 reset issued without op x35
"SYNCHRONIZE_CACHE". I've heard same of" writeable "dvd.  And loss of
bus power can kill any write cache in a drive, of course."

Pat LaVarre

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Alan Stern]

There haven't been any replies to my suggestion from a week ago

http://marc.theaimsgroup.com/?l=linux-scsi&m=106462295800726&w=2

for a way to resolve the mode-sense page 8 problem.  One way or another, I 
wish somebody would do _something_ about this, particularly before 
2.6.0-final comes out.

Alan Stern

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[James Bottomley]

On Fri, 2003-10-03 at 09:18, Alan Stern wrote:
> There haven't been any replies to my suggestion from a week ago
> 
> http://marc.theaimsgroup.com/?l=linux-scsi&m=106462295800726&w=2
> 
> for a way to resolve the mode-sense page 8 problem.  One way or another, I 
> wish somebody would do _something_ about this, particularly before 
> 2.6.0-final comes out.

Well, the patch isn't quite correct because if it's not going to probe
the cache it should set up a write through cache (or disabled cache) as
the default.

Patrick's patch

http://marc.theaimsgroup.com/?l=linux-scsi&m=106366112221507

Does get this right.

But my principle objection, as I've stated before, is that it isn't true
that we don't need to manage the caches of hot pluggable devices, fibre
channel being the counter example.

Even for devices whose model is always forced remove, knowing the cache
type is valuable information (especially as writeback cache type would
be a rather grave error).

The reason not to try to read the cache type for USB is that some of the
devices barf on receiving the command.  These type of devices are the
ones we don't want to make any attempt to manage the cache for, and I'd
like the flag name we settle on to reflect this.

However, if there's no further objections on the naming front, I'll put
Patrick's patch into the SCSI repository.

James

Re: [linux-usb-devel] Re: USB storage problems on OHCI..

[Patrick Mansfield]

On Fri, Oct 03, 2003 at 10:05:52AM -0500, James Bottomley wrote:

> Well, the patch isn't quite correct because if it's not going to probe
> the cache it should set up a write through cache (or disabled cache) as
> the default.

Alan's original patch is included in my patch.

> Patrick's patch
> 
> http://marc.theaimsgroup.com/?l=linux-scsi&m=106366112221507
> 
> Does get this right.

I am re-rolling the patch based on current 2.6, plus a couple comments
Christoph had, I'll resend to you and linux-scsi.

> But my principle objection, as I've stated before, is that it isn't true
> that we don't need to manage the caches of hot pluggable devices, fibre
> channel being the counter example.

I agree - the type of transport does not indicate the capabilties of the
device. If connectors were free and took no space (yeh never), we would
have usb, firewire, spi, and fcp available on all disk drives.

But for some reason with USB, we are more likely to have devices that
cannot handle mode sense page 8 (cache). So shove some code into the
usb-storage driver, and make it possible to set policy allowing us to
disable page 8 for any device.

Though I do not see the utility in syncing the cache, especially given the
failure to figure out the caching method. 

High end storage won't care (probably battery backed ram, redundant
storage, and separate out-of-band monitoring facilities). 

If the cache sync fails, the best we can do is notify the user, we can't
reissue the write or even tell the user what block failed to make it to
disk, the best we can do is say "your hosed".

-- Patrick Mansfield