RE: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

RE: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Pat LaVarre]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="UTF-8", Size: 2604 bytes --]

> For sd, we gauge the size of the command from the
> size of the medium:  <1Gb=> six byte, from 1Gb to
> 2Tb 10 byte, over 2Tb 16 byte
 
By now this theory we have completely disavowed?
 
I haven't seen our English yet mention that asking to
read/write more than xFF blocks triggers us to skip
past the commonly misunderstood x08/0A
Read(6)/Write(6) standard of using x00 to mean x100.
 
I think instead we resort immediately to x28/2A
Read(10)/Write(10).

In http://lxr.linux.no/source/drivers/scsi/sd.c?v=2.5.56
I see fragments that match our later English:
 
 
SRpnt->sr_device->ten = 1;
 
 
if (block > 0xffffffff) {
        SCpnt->cmnd[0] += READ_16 - READ_6;
        ...
if (((this_count > 0xff) || (block > 0x1fffff)) || SCpnt->device->ten) {
        ...
        SCpnt->cmnd[0] += READ_10 - READ_6;
 
 
case ILLEGAL_REQUEST:
        if (SCpnt->device->ten == 1) {
                if (SCpnt->cmnd[0] == READ_10 ||
                    SCpnt->cmnd[0] == WRITE_10)
                        SCpnt->device->ten = 0;
        }
 
 
Being a dramatically raw Linux newbie, I can't know if
sequencing this code to match the English is correct
or not, nor indeed if I am reading the source file we mean.
 
Click-thru ILLEGAL_REQUEST shows me three source files
agree with me thinking by that jargon we mean x05.
And more or less source files agree that by READ_6
READ_10 WRITE_10 we mean x 08 0A 2A.

Cluelessly, curiously, thankfully yours, Pat LaVarre


	-----Original Message----- 
	From: James Bottomley [mailto:James.Bottomley@SteelEye.com] 
	Sent: Mon 3/24/2003 9:56 AM 
	To: Jens Axboe 
	Cc: Linus Torvalds; Matthew Dharm; USB Developers; USB Storage List; Linux SCSI list 
	Subject: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts
	
	

	On Mon, 2003-03-24 at 10:52, Jens Axboe wrote: 
	> It's not true, ->ten is set unconditionally and we only fall back to 6 
	> byte cdb's if we see an ILLEGAL_REQUEST on a READ_10/WRITE_10. 
	> 
	> So the logic is, always assume 10-byte commands. If an incoming request 
	> cannot be addressed with 10-byte commands, use 16. 

	Sorry, that was me misreading the fallback logic in sd.c 

	Ten bytes it is unless we have problems. 

	James 


	_______________________________________________ 
	usb-storage mailing list 
	usb-storage@one-eyed-alien.net 
	http://www2.one-eyed-alien.net/mailman/listinfo/usb-storage 

N\x18¬±ù޵隊X¬²š'²ŠÞu¼“†)ä\x19ç¤Yé\¢g­¢\až’\x17š½éá¶Ú\x7fþØbžH\x1ezG(›û\x1f–)îÆë^[uëÞ–X¬¶Ë(º·\x1e~Šàzw­N‹§²æìr¸›zë\x1e¶\x17¥jË_¢¹Ÿ‰é]jØm¶›?þX¬¶Ë(º·\x1e~Šàzw­þX¬¶ÏåŠËbú?–)îÆë^[uëÞ

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Jens Axboe]

On Mon, Mar 24 2003, Pat LaVarre wrote:
> > For sd, we gauge the size of the command from the
> > size of the medium:  <1Gb=> six byte, from 1Gb to
> > 2Tb 10 byte, over 2Tb 16 byte
>  
> By now this theory we have completely disavowed?

Yes it was wrong, see earlier posting.

(BTW, top posting sucks...)

-- 
Jens Axboe



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
linux-usb-devel@lists.sourceforge.net
To unsubscribe, use the last form field at:
https://lists.sourceforge.net/lists/listinfo/linux-usb-devel

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Andries.Brouwer]

> I believe a recent patch addressed this problem.
> the patch is documented as fixing bug #510

Very good. And see, I had no time to look at all of these
Bugzilla texts, but now that I am back and look at the sequel
for this bug #510 it turns out that it was semi-fixed.
However, the author complains

=====
I confirm that your patch makes my memory stick reader work again.

However, the usb-storage module initialisation is still slow. See the attached
traces: I modprobe usb-storage at 08:50:16, and the initialisation was not
ended until 08:50:53. Under 2.4 it takes only a few (2-3) seconds to do this...
=====

and the trace shows

=====
Apr  8 11:46:10 crusoe: usb-storage: Command MODE_SENSE (6 bytes)
Apr  8 11:46:10 crusoe: usb-storage:  1a 00 3f 00 04 00
Apr  8 11:46:10 crusoe: usb-storage: usb_stor_ctrl_transfer(): rq=00 rqtype=21 value=0000 index=00 len=12
Apr  8 11:46:10 crusoe: usb-storage: Status code 0; transferred 12/12
Apr  8 11:46:10 crusoe: usb-storage: -- transfer complete
Apr  8 11:46:10 crusoe: usb-storage: Call to usb_stor_ctrl_transfer() returned 0
Apr  8 11:46:11 crusoe: usb-storage: usb_stor_bulk_transfer_buf(): xfer 4 bytes
Apr  8 11:46:11 crusoe: usb-storage: Status code -75; transferred 4/4
Apr  8 11:46:11 crusoe: usb-storage: -- unknown error
Apr  8 11:46:11 crusoe: usb-storage: CB data stage result is 0x3
Apr  8 11:46:11 crusoe: usb-storage: -- transport indicates error, resetting
Apr  8 11:46:11 crusoe: usb-storage: CB_reset() called
=====

That is, now all three bug reports talk about the same problem.

OK. Something is wrong with MODE_SENSE, that was not yet wrong
in 2.4. If we fix that, then all bug reports mentioned so far
are covered.

[James, I saw your mail, but have not yet looked at the code.]

Andries

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Andries.Brouwer]

Finally the third one is again of the kind 2.4 works, 2.5 fails.
So again, not a mysterious device but a software bug in
usb-storage and/or scsi/sd.

It looks different from the other two - the core is


Mar 30 15:42:12 crusoe kernel: usb-storage: Command TEST_UNIT_READY (6 bytes)
Mar 30 15:42:12 crusoe kernel: usb-storage:  00 00 00 00 00 00
Mar 30 15:42:12 crusoe kernel: usb-storage: usb_stor_ctrl_transfer(): rq=00 rqtype=21 value=0000 index=00 len=12
Mar 30 15:42:12 crusoe kernel: usb-storage: Status code -32; transferred 0/12
Mar 30 15:42:12 crusoe kernel: usb-storage: -- stall on control pipe
Mar 30 15:42:12 crusoe kernel: usb-storage: Call to usb_stor_ctrl_transfer() returned 3
Mar 30 15:42:12 crusoe kernel: usb-storage: -- transport indicates error, resetting
Mar 30 15:42:12 crusoe kernel: usb-storage: CB_reset() called


But I suppose nothing is wrong with the command TEST_UNIT_READY.
It is just that the device was not ready yet.
The successful 2.4 trace also has a failing TEST_UNIT_READY,
but itis followed by a successful one.

Mar 30 19:47:46 crusoe kernel: usb-storage: Command TEST_UNIT_READY (6 bytes)
Mar 30 19:47:46 crusoe kernel: usb-storage: 00 00 00 00 00 00 00 00 00 00 14 c0
Mar 30 19:47:46 crusoe kernel: usb-uhci.c: interrupt, status 2, frame# 1414
Mar 30 19:47:46 crusoe kernel: usb-storage: Call to usb_stor_control_msg() returned -32
Mar 30 19:47:46 crusoe kernel: usb-storage: -- Stall on control pipe. Clearing
Mar 30 19:47:46 crusoe kernel: usb-storage: usb_stor_clear_halt: result=0
...
Mar 30 19:47:46 crusoe kernel: usb-storage: Command TEST_UNIT_READY (6 bytes)
Mar 30 19:47:46 crusoe kernel: usb-storage: 00 00 00 00 00 00 00 00 00 00 14 c0
Mar 30 19:47:46 crusoe kernel: usb-storage: Call to usb_stor_control_msg() returned 12


So, it looks like the bad part of 2.5 is that it asks for a reset.

Andries

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Alan Stern]

On Tue, 22 Apr 2003 Andries.Brouwer@cwi.nl wrote:

> Finally the third one is again of the kind 2.4 works, 2.5 fails.
> So again, not a mysterious device but a software bug in
> usb-storage and/or scsi/sd.
> 
> It looks different from the other two - the core is
> 
> 
> Mar 30 15:42:12 crusoe kernel: usb-storage: Command TEST_UNIT_READY (6 bytes)
> Mar 30 15:42:12 crusoe kernel: usb-storage:  00 00 00 00 00 00
> Mar 30 15:42:12 crusoe kernel: usb-storage: usb_stor_ctrl_transfer(): rq=00 rqtype=21 value=0000 index=00 len=12
> Mar 30 15:42:12 crusoe kernel: usb-storage: Status code -32; transferred 0/12
> Mar 30 15:42:12 crusoe kernel: usb-storage: -- stall on control pipe
> Mar 30 15:42:12 crusoe kernel: usb-storage: Call to usb_stor_ctrl_transfer() returned 3
> Mar 30 15:42:12 crusoe kernel: usb-storage: -- transport indicates error, resetting
> Mar 30 15:42:12 crusoe kernel: usb-storage: CB_reset() called
> 
> 
> But I suppose nothing is wrong with the command TEST_UNIT_READY.
> It is just that the device was not ready yet.
> The successful 2.4 trace also has a failing TEST_UNIT_READY,
> but itis followed by a successful one.
> 
> Mar 30 19:47:46 crusoe kernel: usb-storage: Command TEST_UNIT_READY (6 bytes)
> Mar 30 19:47:46 crusoe kernel: usb-storage: 00 00 00 00 00 00 00 00 00 00 14 c0
> Mar 30 19:47:46 crusoe kernel: usb-uhci.c: interrupt, status 2, frame# 1414
> Mar 30 19:47:46 crusoe kernel: usb-storage: Call to usb_stor_control_msg() returned -32
> Mar 30 19:47:46 crusoe kernel: usb-storage: -- Stall on control pipe. Clearing
> Mar 30 19:47:46 crusoe kernel: usb-storage: usb_stor_clear_halt: result=0
> ...
> Mar 30 19:47:46 crusoe kernel: usb-storage: Command TEST_UNIT_READY (6 bytes)
> Mar 30 19:47:46 crusoe kernel: usb-storage: 00 00 00 00 00 00 00 00 00 00 14 c0
> Mar 30 19:47:46 crusoe kernel: usb-storage: Call to usb_stor_control_msg() returned 12
> 
> 
> So, it looks like the bad part of 2.5 is that it asks for a reset.
> 
> Andries

I believe a recent patch addressed this problem.  I've lost track of the 
Bugzilla bug #'s, but the patch is documented as fixing bug #510: a STALL 
on a control endpoint isn't necessarily an error; it can indicate a 
command failure.  See drivers/usb/storage/transport.c rev 1.971.70.16:
<http://linux.bkbits.net:8080/linux-2.5/related/drivers/usb/storage/transport.c?nav=index.html|src/|src/drivers|src/drivers/usb|src/drivers/usb/storage>

Alan Stern

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Andries.Brouwer]

Then we get the second bugzilla report. Interesting.
It fails in precisely the same way as the first one:


Apr  9 17:34:55 lemon: SCSI device sda: 126848 512-byte hdwr sectors (65 MB)
Apr  9 17:34:55 lemon: usb-storage: queuecommand() called
Apr  9 17:34:55 lemon: usb-storage: *** thread awakened.
Apr  9 17:34:55 lemon: usb-storage: Command MODE_SENSE (6 bytes)
Apr  9 17:34:55 lemon: usb-storage:  1a 00 3f 00 04 00
Apr  9 17:34:55 lemon: usb-storage: Invoking Mode Translation
Apr  9 17:34:55 lemon: usb-storage: usb_stor_ctrl_transfer(): rq=00 rqtype=21 value=0000 index=00 len=12
Apr  9 17:34:55 lemon: usb-storage: Status code 0; transferred 12/12
Apr  9 17:34:55 lemon: usb-storage: -- transfer complete
Apr  9 17:34:55 lemon: usb-storage: Call to usb_stor_ctrl_transfer() returned 0
Apr  9 17:34:55 lemon: usb-storage: usb_stor_bulk_transfer_buf(): xfer 4 bytes
Apr  9 17:34:55 lemon: usb-storage: Status code -75; transferred 4/4
Apr  9 17:34:55 lemon: usb-storage: -- unknown error
Apr  9 17:34:55 lemon: usb-storage: CB data stage result is 0x3
Apr  9 17:34:55 lemon: usb-storage: -- transport indicates error, resetting
Apr  9 17:34:55 lemon: usb-storage: CB_reset() called
Apr  9 17:35:11 lemon modprobe: FATAL: Module /dev/tts/64 not found. 
Apr  9 17:35:15 lemon: drivers/usb/core/message.c: usb_control/bulk_msg: timeout
Apr  9 17:35:25 lemon: usb-storage: command_abort() called
Apr  9 17:35:25 lemon: usb-storage: usb_stor_abort_transport called


But here the additional information is:
It was working fine under 2.4.* but under 2.5.66/67 it hangs.

Is there a problem with MODE_SENSE? But 2.4 also does this MODE_SENSE.
A first step is perhaps finding out what the * is for.

Which 2.4 versions are known to work correctly?

Andries

[And is there a verbose usb-storage log on a successful 2.4?]


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
linux-usb-devel@lists.sourceforge.net
To unsubscribe, use the last form field at:
https://lists.sourceforge.net/lists/listinfo/linux-usb-devel

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Mike Bursell]

[-- Attachment #1: Type: text/plain, Size: 2383 bytes --]

Hi -

I'm not sure if I'm the first or second bugzilla report (I'm 553 -
what's the other one?), but I don't have a log for 2.4.*, and I suspect
that the changes I've made to my machine since I moved to the 2.5 series
full-time will mean that I can't boot up sensibly in 2.4.* anymore. 
Anything else I could help with, though?

-Mike.


On Tue, 2003-04-22 at 18:22, Andries.Brouwer@cwi.nl wrote:
> Then we get the second bugzilla report. Interesting.
> It fails in precisely the same way as the first one:
> 
> 
> Apr  9 17:34:55 lemon: SCSI device sda: 126848 512-byte hdwr sectors (65 MB)
> Apr  9 17:34:55 lemon: usb-storage: queuecommand() called
> Apr  9 17:34:55 lemon: usb-storage: *** thread awakened.
> Apr  9 17:34:55 lemon: usb-storage: Command MODE_SENSE (6 bytes)
> Apr  9 17:34:55 lemon: usb-storage:  1a 00 3f 00 04 00
> Apr  9 17:34:55 lemon: usb-storage: Invoking Mode Translation
> Apr  9 17:34:55 lemon: usb-storage: usb_stor_ctrl_transfer(): rq=00 rqtype=21 value=0000 index=00 len=12
> Apr  9 17:34:55 lemon: usb-storage: Status code 0; transferred 12/12
> Apr  9 17:34:55 lemon: usb-storage: -- transfer complete
> Apr  9 17:34:55 lemon: usb-storage: Call to usb_stor_ctrl_transfer() returned 0
> Apr  9 17:34:55 lemon: usb-storage: usb_stor_bulk_transfer_buf(): xfer 4 bytes
> Apr  9 17:34:55 lemon: usb-storage: Status code -75; transferred 4/4
> Apr  9 17:34:55 lemon: usb-storage: -- unknown error
> Apr  9 17:34:55 lemon: usb-storage: CB data stage result is 0x3
> Apr  9 17:34:55 lemon: usb-storage: -- transport indicates error, resetting
> Apr  9 17:34:55 lemon: usb-storage: CB_reset() called
> Apr  9 17:35:11 lemon modprobe: FATAL: Module /dev/tts/64 not found. 
> Apr  9 17:35:15 lemon: drivers/usb/core/message.c: usb_control/bulk_msg: timeout
> Apr  9 17:35:25 lemon: usb-storage: command_abort() called
> Apr  9 17:35:25 lemon: usb-storage: usb_stor_abort_transport called
> 
> 
> But here the additional information is:
> It was working fine under 2.4.* but under 2.5.66/67 it hangs.
> 
> Is there a problem with MODE_SENSE? But 2.4 also does this MODE_SENSE.
> A first step is perhaps finding out what the * is for.
> 
> Which 2.4 versions are known to work correctly?
> 
> Andries
> 
> [And is there a verbose usb-storage log on a successful 2.4?]
-- 
07050 144575 mike@hingston.demon.co.uk

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[James Bottomley]

On Tue, 2003-04-22 at 12:22, Andries.Brouwer@cwi.nl wrote:
[...]

Actually, could this

> Apr  9 17:34:55 lemon: usb-storage: Invoking Mode Translation

Be the root cause?  It indicates that usb storage modified the command.

In 2.5.68 that's doing a translation from MODE SENSE(6) to MODE
SENSE(10) inside the driver.

But the odd thing in usb/storage/protocol.c is this (about line 354):

                case MODE_SENSE:
                case MODE_SELECT:
                        srb->cmd_len = 12;
                        srb->cmnd[11] = 0;
                        srb->cmnd[10] = 0;

Why is a 10 byte command given a command length of 12?

James

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 2326 bytes --]

On Tue, Apr 22, 2003 at 07:22:11PM +0200, Andries.Brouwer@cwi.nl wrote:
> Then we get the second bugzilla report. Interesting.
> It fails in precisely the same way as the first one:
> 
> Apr  9 17:34:55 lemon: SCSI device sda: 126848 512-byte hdwr sectors (65 MB)
> Apr  9 17:34:55 lemon: usb-storage: queuecommand() called
> Apr  9 17:34:55 lemon: usb-storage: *** thread awakened.
> Apr  9 17:34:55 lemon: usb-storage: Command MODE_SENSE (6 bytes)
> Apr  9 17:34:55 lemon: usb-storage:  1a 00 3f 00 04 00
> Apr  9 17:34:55 lemon: usb-storage: Invoking Mode Translation
> Apr  9 17:34:55 lemon: usb-storage: usb_stor_ctrl_transfer(): rq=00 rqtype=21 value=0000 index=00 len=12
> Apr  9 17:34:55 lemon: usb-storage: Status code 0; transferred 12/12
> Apr  9 17:34:55 lemon: usb-storage: -- transfer complete
> Apr  9 17:34:55 lemon: usb-storage: Call to usb_stor_ctrl_transfer() returned 0
> Apr  9 17:34:55 lemon: usb-storage: usb_stor_bulk_transfer_buf(): xfer 4 bytes
> Apr  9 17:34:55 lemon: usb-storage: Status code -75; transferred 4/4
> Apr  9 17:34:55 lemon: usb-storage: -- unknown error
> Apr  9 17:34:55 lemon: usb-storage: CB data stage result is 0x3
> Apr  9 17:34:55 lemon: usb-storage: -- transport indicates error, resetting
> Apr  9 17:34:55 lemon: usb-storage: CB_reset() called
> Apr  9 17:35:11 lemon modprobe: FATAL: Module /dev/tts/64 not found. 
> Apr  9 17:35:15 lemon: drivers/usb/core/message.c: usb_control/bulk_msg: timeout
> Apr  9 17:35:25 lemon: usb-storage: command_abort() called
> Apr  9 17:35:25 lemon: usb-storage: usb_stor_abort_transport called
> 
> But here the additional information is:
> It was working fine under 2.4.* but under 2.5.66/67 it hangs.
> 
> Is there a problem with MODE_SENSE? But 2.4 also does this MODE_SENSE.
> A first step is perhaps finding out what the * is for.

2.4 does a MODE_SENSE, but does it differently.

> Which 2.4 versions are known to work correctly?

That depends on the particular device in question.

> [And is there a verbose usb-storage log on a successful 2.4?]

Yes.

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

Umm, these aren't the droids you're looking for.
					-- Bill Gates
User Friendly, 11/14/1998

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Andries.Brouwer]

> The relevant bugzilla bugs are

> http://bugzilla.kernel.org/show_bug.cgi?id=211
> (Mounting a SM/CF reader does not work and does not return)

Hmm. Let me start looking at this one to see whether there is
any indication that filtering might work.

First of all, this user seems to give the wrong command.
He complains
	When I try to mount /dev/sdc /mnt/ nothing happens
and later
	mount /dev/sdb /mnt

But these cards have a partition table, so he should have
tried to mount sdc1 or sdb1.

(And even straight IDE often fails when one tries to access
ridiculous sectors. Maybe I should submit a patch for that.)

So, even though it would be nicer if this user got a good
error message, there is no sign yet that there is anything
seriously wrong, I think. The inquiry works, the device is
recognized and gets the right size.

In this situation I always say "blockdev --rereadpt /dev/sdX"
to force reading of the partition table. Afterwards one can
mount if all is well. If something is bad, the blockdev command
will hang.

Andries

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Andries.Brouwer]

I have not been following this USB and SCSI stuff very carefully,
but some time ago I tried a large number of strange USB devices,
and of course some failed, and I changed the SCSI stuff a little
so that everything worked. I think Linus took these patches,
which means that on my sample of devices 2.5 works.

Is the present discussion about 2.4, or is it about 2.5,
and in case it is the latter, can anybody describe what
devices fail, and in what way?

Andries

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 2364 bytes --]

First, we're talking about 2.5.

As for what fails... start by looking at bugzilla for bugs assigned to me.
There are about 4-5 which are all devices that choke on a command (or it's
variant) that the 'popular OSes' don't send.  There is a constant stream of
reports of similar problems on linux-usb-users and -devel.

The basic failure mode is this:  The host sends a command that the device
doesn't like (for the aforementioned reason), and the device firmware
crashes or otherwise enters an unrecoverable state.  Some of the evil
commands include various forms of INQUIRY, MODE_SENSE (to disks), READ_6,
WRITE_6, etc.

Your patches went a long way to improving things.  But, then people added
EVPD and further changed MODE_SENSE and other things.   It's pretty clear
to me that trying to change the SCSI core to not use these commands isn't
going to happen -- so I want a way to filter them.  I originally wanted a
centralized filter (i.e. usb-storage could indicate to the mid-layer not to
send the 'unwelcome' commands), but that got shot down in favor of a
centralized filter-framework which could be applied by any LLDD that wanted
to.

Or, at least, that's how I recap the situation.  I think we're at the point
where we're trying to decide what to merge.  At least I hope that's where
we are -- and I still stand by my patch (based on James' work).

The idea of a centralized framework was to allow multiple sections of code
to use this (sbp2, usb-storage, ide-scsi perhaps, and other LLDDs).

Matt

On Tue, Apr 22, 2003 at 03:09:23AM +0200, Andries.Brouwer@cwi.nl wrote:
> I have not been following this USB and SCSI stuff very carefully,
> but some time ago I tried a large number of strange USB devices,
> and of course some failed, and I changed the SCSI stuff a little
> so that everything worked. I think Linus took these patches,
> which means that on my sample of devices 2.5 works.
> 
> Is the present discussion about 2.4, or is it about 2.5,
> and in case it is the latter, can anybody describe what
> devices fail, and in what way?
> 
> Andries

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

But where are the THEMES?!  How do you expect me to use an OS without 
themes?!
					-- Stef
User Friendly, 10/9/1998

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Alan Cox]

On Maw, 2003-04-22 at 02:17, Matthew Dharm wrote:
> Your patches went a long way to improving things.  But, then people added
> EVPD and further changed MODE_SENSE and other things.   It's pretty clear
> to me that trying to change the SCSI core to not use these commands isn't
> going to happen -- so I want a way to filter them.  I originally wanted a
> centralized filter (i.e. usb-storage could indicate to the mid-layer not to
> send the 'unwelcome' commands), but that got shot down in favor of a
> centralized filter-framework which could be applied by any LLDD that wanted
> to.

You already can do this. You just need to write yourself a library
routine which converts/blocks according to the filter rules (ie
read6->read10, returns errors to fancy stuff etc).

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 1641 bytes --]

On Tue, Apr 22, 2003 at 03:31:52PM +0100, Alan Cox wrote:
> On Maw, 2003-04-22 at 02:17, Matthew Dharm wrote:
> > Your patches went a long way to improving things.  But, then people added
> > EVPD and further changed MODE_SENSE and other things.   It's pretty clear
> > to me that trying to change the SCSI core to not use these commands isn't
> > going to happen -- so I want a way to filter them.  I originally wanted a
> > centralized filter (i.e. usb-storage could indicate to the mid-layer not to
> > send the 'unwelcome' commands), but that got shot down in favor of a
> > centralized filter-framework which could be applied by any LLDD that wanted
> > to.
> 
> You already can do this. You just need to write yourself a library
> routine which converts/blocks according to the filter rules (ie
> read6->read10, returns errors to fancy stuff etc).

I thought that the entire point was to have all this in a centralized place
for use by multiple LLDDs (or anyone else for that matter).  Which is the
entire point of this discussion.

If it's not going to be accepted as a centralized filter-framework, then
I'll just add it to the usb-storage driver and leave it at that.  But I'm
pretty certain that we will keep revisiting this issue as we see more and
more development of the SCSI core to support some more unusual commands and
we see more and more emulated-SCSI busses which choke on those commands.

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

It was a new hope.
					-- Dust Puppy
User Friendly, 12/25/1998

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[James Bottomley]

On Tue, 2003-04-22 at 09:31, Alan Cox wrote:
> On Maw, 2003-04-22 at 02:17, Matthew Dharm wrote:
> > Your patches went a long way to improving things.  But, then people added
> > EVPD and further changed MODE_SENSE and other things.   It's pretty clear
> > to me that trying to change the SCSI core to not use these commands isn't
> > going to happen -- so I want a way to filter them.  I originally wanted a
> > centralized filter (i.e. usb-storage could indicate to the mid-layer not to
> > send the 'unwelcome' commands), but that got shot down in favor of a
> > centralized filter-framework which could be applied by any LLDD that wanted
> > to.
> 
> You already can do this. You just need to write yourself a library
> routine which converts/blocks according to the filter rules (ie
> read6->read10, returns errors to fancy stuff etc).


Actually, this one should already be fixed in the mid-layer.  We use
READ_10/WRITE_10 by default (device->ten flag, which is initialised to
1).  We only revert back to READ_6 if the device rejects the 10 byte
command.

We also use READ_16, but *only* if the requested block is over the
READ_10 limit (2Tb) so this should never be a problem.

We have been changing the probing ordering and command type in the
mid-layer to cope with these cases. (for instance, Andries noticed that
we could avoid annoying removable media by not probing for capacity and
cache type unless the media is actually present, not probing for write
protect unless the media is removeable).

However, we seem to have reached the point over the cache probe and the
write protect probe where there are no more ordering change tricks that
can be used.

The relevant bugzilla bugs are

http://bugzilla.kernel.org/show_bug.cgi?id=211 (Mounting a SM/CF reader
does not work and does not return)
http://bugzilla.kernel.org/show_bug.cgi?id=553 (On connecting Sony
TRV340E camera in USB mode, USB hangs)
http://bugzilla.kernel.org/show_bug.cgi?id=510 (USB storage doesn't work
on my Sony Vaio C1VE)

A lot of the problems seem to be memory stick related.

The plan is to put a filter library into scsi_lib which could be called
from the usb-storage queuecommand() to filter out the offending commands
with the correct sense rejections that the mid-layer can process and
recover from.

James

RE: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Pat LaVarre]

> > Fibre Channel, Parallel SCSI, SSA, IEEE 1394,
> > RDMA (infiniband), Internet SCSI, SAS, ...
> > ATAPI ... USB ...
>...
> SCSI thru the legacy PC printer port ...

I hear Windows contains separate binary-code-only for
each of these that it supports.

Rumour tells me, for example, that the first CDB's
seen by the device can vary between FireWire and USB,
because they don't come from a shared analogoue of
sd.c, but instead they come from code built for just
one particular bus, charged with beginning life by
distinguishing HDD from CD from DVD etc.

For example, I've heard that some FireWire HDD's
worked in Windows without supporting op x12 Inquiry.

Might be smart for Linux to give in sooner, rather
than later.  So long as we are engaged in publishing a
model of what the de facto Windows SCSI-over-whatever
standards are, we may as well include a copy the basic
structure of the original.

Pat LaVarre

P.S. Please ignore Yahoo spam that follows, if any.



__________________________________________________
Do you Yahoo!?
Yahoo! Tax Center - File online, calculators, forms, and more
http://tax.yahoo.com

RE: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Pat LaVarre]

> From: Douglas Gilbert [mailto:dougg@torque.net]
> Sent: Wed 4/9/2003 1:45 AM
>
> Back to one of my favourite subjects: replacing the
> emulated flag with a protocol identifier ...

In short:  How about removing the emulated flag
altogether?  Answered at a link somewhere, right?

At length ...

> > In particular, we need to hold on to some way of
> > letting sg pass thru ... arbitrary enough
>
> The sg policy is basically "you want, you got
> it" ... [except] a little magic inside ...
> O_RDONLY ...

Glad to find we celebrate the value of utterly
transparent pass thru.

I trust we're all aware that passing intact thru sg
doesn't guarantee passing intact thru
drivers/usb/storage/.

I remember in particular seeing byte 2 of op x12
Inquiry being helpfully changed, also I remember
hearing of op x25 Read Capacity data being helpfully
changed on occasion.

> Back to one of my favourite subjects: replacing the
> emulated flag with a protocol identifier ...

How about removing the emulated flag altogether?

I ask because I find this topic (falsely?) familiar.

Personally I've mostly done the device side of Scsi.

Host folk periodically ask me to teach the device to
identify what protocol it uses.

Trouble is, my device doesn't really know.

Is my device really a parallel SCSI device?  Or is my
device parallel SCSI behind a mostly transparent
USB/SCSI bridge?  Or is my device an ATAPI device
behind a mostly transparent FireWire/ATAPI bridge?
Or ...?

In all my (sharply limited) experience the host folk
in question are mistakenly associating with protocol a
natural and growing variation in the actual device
response to CDB's and Data.

> As for the OO encapsulation counter argument, the
> transport protocol _is_ significant for parts of
> the SCSI command set (see SAM-3 and SPC-3)
> as well as device identification/discovery, async
> error notification ....

I look forward to my education.

Maybe nowadays we should not mention SAM-3 and SPC-3
without also mentioning MMC-4, which I see redefines
elements of SPC-3 to recognise, at last at ANSI, some
of the more prominent binary incompatibilities first
propagated via SFF, such as the op x12 Inquiry data
and the op x5A ModeSense10 CDB.

> There are many other uses for a protocol
> identifier: a "bus scan" based device discovery on
> protocol 5 doesn't make much sense.

Aye, many uses.

For example, a particular mostly transparent PCI/ATAPI
South bridge might be known to scramble copies of
anything but multiples of 8 data bytes.

Where that limitation is known, we could tell the
layers above to jump thru hoops to try and avoid that
case.

Unless we'd rather code to the least common
denominator always.

> Fibre Channel, Parallel SCSI, SSA, IEEE 1394,
> RDMA (infiniband), Internet SCSI, SAS, ...
> ATAPI ... USB ...

I know some folk pass SCSI thru the legacy PC printer
port ... does that fit in here, or is that an N-th
protocol?

> possible to overspeed an ATA disk ...
> proposed filtering ...
> consensus ... not ...

Also good to hear, thanks for talking, Pat LaVarre

P.S. Please ignore Yahoo spam that follows, if any.

__________________________________________________
Do you Yahoo!?
Yahoo! Tax Center - File online, calculators, forms, and more
http://tax.yahoo.com

RE: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Pat LaVarre]

> From: Patrick Mansfield [mailto:patmans@us.ibm.com]
> Sent: Mon 4/7/2003 4:33 PM ...
>
> Besides the current b/w listed items (except for
> the BLIST_INQUIRY_36/58 flags), we have:
>
> ... too long an INQUIRY request,
> ... report back wrong INQUIRY length
> ... don't like EVPD INQUIRY

And more.  Talk like Windows means say -x "12 0 0 0 24
0" -i x24 // op x12 Inquiry for 36, nothing else, at
least until broken 64-bit motherboards force us to x20
or x28.

> Why can't usb storage conditionally modify the
> INQUIRY to request no more than 36 bytes
> (allocation length), and modify any results to be
> <= 36 (additional length)?

sd.c isn't the only client of usb storage.

In particular, we need to hold on to some way of
letting sg pass thru op x12 Inquiry arbitrary enough
to copy In the vendor-specific data out past 36 bytes.
 Or to copy In the Ans-specified data out there that
lets an SFF/ ANSI MMC compliant device report an
attempt to comply simultaneously with the somewhat
contradictory ANSI SPC.

No?

Pat LaVarre
p.lavarre@ieee.org

P.S. Please ignore the Yahoo spam that follows, if any.

__________________________________________________
Do you Yahoo!?
Yahoo! Tax Center - File online, calculators, forms, and more
http://tax.yahoo.com

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Douglas Gilbert]

Pat LaVarre wrote:
>>From: Patrick Mansfield [mailto:patmans@us.ibm.com]
>>Sent: Mon 4/7/2003 4:33 PM ...
>>
>>Besides the current b/w listed items (except for
>>the BLIST_INQUIRY_36/58 flags), we have:
>>
>>... too long an INQUIRY request,
>>... report back wrong INQUIRY length
>>... don't like EVPD INQUIRY
> 
> 
> And more.  Talk like Windows means say -x "12 0 0 0 24
> 0" -i x24 // op x12 Inquiry for 36, nothing else, at
> least until broken 64-bit motherboards force us to x20
> or x28.
> 
> 
>>Why can't usb storage conditionally modify the
>>INQUIRY to request no more than 36 bytes
>>(allocation length), and modify any results to be
>><= 36 (additional length)?
> 
> 
> sd.c isn't the only client of usb storage.
> 
> In particular, we need to hold on to some way of
> letting sg pass thru op x12 Inquiry arbitrary enough
> to copy In the vendor-specific data out past 36 bytes.
>  Or to copy In the Ans-specified data out there that
> lets an SFF/ ANSI MMC compliant device report an
> attempt to comply simultaneously with the somewhat
> contradictory ANSI SPC.
> 
> No?

Pat,
The sg policy is basically "you want, you got it" **. Andre
Hedrick found out that in some dark corner of the ATA
instruction set it was possible to overspeed an ATA disk.
He proposed filtering out that instruction but the linux-kernel
list consensus was not to do that.

Back to one of my favourite subjects: replacing the emulated
flag with a protocol identifier. Table 237 in section 8.5
(Protocol Specific parameters) of a recent SPC-3 draft
has a table of transport protocol identifiers:
   0      Fibre Channel
   1      Parallel SCSI
   2      SSA
   3      IEEE 1394
   4      RDMA (infiniband)
   5      Internet SCSI
   6      SAS
7-15     reserved

Perhaps we could borrow 15 for ATAPI and 14 for USB mass storage
and we should have current transports covered. As for the OO
encapsulation counter argument, the transport protocol _is_
significant for parts of the SCSI command set (see SAM-3 and
SPC-3) as well as device identification/discovery, async
error notification ....

So then we could associate a "quality of implementation" metric
with each protocol and program defensively within the kernel
accordingly.  For those using the pass through  interface (i.e.
sg [and the SG_IO ioctl now in the block layer]) it remains
'caveat emptor'.

There are many other uses for a protocol identifier: a "bus scan"
based device discovery on protocol 5 doesn't make much sense.


** There is a little magic inside sg's SG_IO ioctl that
only allows through non-modifying SCSI commands (e.g. INQUIRY
and READ) when a file is opened O_RDONLY.

Doug Gilbert

RE: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Pat LaVarre]

> > On Sat, Mar 22, 2003 at 11:39:05PM -0800, Linus
> > Torvalds wrote:
> >
> > > How about making the SCSI stuff pass a "common"
> > > flag (or "required") down with the command?
> > > Then, a emulated thing could just decide to
> > > punt all commands with an immediate failure if
> > > they aren't marked "required".
> >
> > > That still _allows_ the driver to implement it
> > > if it wants to, unlike your previous approach.
 
In particular, passing CDB's thru sg will mark them
Required, until/ unless we rev sg to let its client
choose to say Required or not?
 
Pat LaVarre

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 641 bytes --]

On Mon, Mar 24, 2003 at 12:16:29PM -0700, Pat LaVarre wrote:
> In particular, passing CDB's thru sg will mark them
> Required, until/ unless we rev sg to let its client
> choose to say Required or not?

That was my intent, at least.  The assumption being that a userspace
program should have done an INQUIRY for 36-bytes to determine if it knows
what it's about to do is safe.

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

Pitr!  That's brilliant!  Funded by Microsoft refunds.  What sweet irony!
					-- A.J.
User Friendly, 2/15/1999

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: PATCH: exclude certain commands from emulated SCSI hosts

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 2504 bytes --]

On Sat, Mar 22, 2003 at 08:09:57PM -0800, Linus Torvalds wrote:
> 
> On Sat, 22 Mar 2003, Matthew Dharm wrote:
> >
> > This patch changes how devices a probed on a SCSI bus if they are on an
> > emulated host.
> 
> I really think this is wrong. I'd much much rather get _rid_ of that 
> stupid "emulated" flag, instead of adding meaning to it.

Well, you could just delete it.  It's not used anywhere.

> > If a host is emulated, we (a) don't ask for EVPD data, as it's likely not
> > there, (b) assume that we have only 36-bytes of INQUIRY data, and (c) don't
> > ask disks for their cache-type (assume write-through).
> 
> ..and thus anything that wants to emulate SCSI has to never be able to be
> write-back again.

Okay, I'll admit that could be a problem.

> Now, I'd say that it is stupid to call yourself a SCSI disk if you aren't 
> one _anyway_ (the raw block device interfaces are simpler and faster), but 
> if you do, then I think it's doubly stupid to put in arbitrary 
> restrictions like this.

The problem is this:  usb-storage interfaces via SCSI, but not just for
disks.  Tape, CD, etc. are all handled with the same protocol.  Heck, ATAPI
disk/cd/tape are handled with these code paths.

> If there are known commands that devices have trouble with (whether 
> emulated or not), maybe we could have helper routines to do some default 
> filtering, and have the queuecommand function check those. But this just 
> looks ugly and hacky to me.

Well, there are lots of 'known commands' that fall in to this category.  If
we had centralized helper functions, that would be great.  But, as it
stands, right now all the low-level drivers have to do all that separately,
and badly.

Right now, we've got nothing, which leaves low-level driver folks out in
the cold.

As I see it, SCSI commands break down into two basic categories:  common
and uncommon.  Common things (basic read and write, 36-byte INQUIRY, eject,
etc.) are all fine, but the 'uncommon' things (checking cache type,
255-byte INQUIRY, etc) cause problems.  I'm trying to find a way to choke
off the problematic commands without having to write code to recognize what
is being sent (and choke it off) based on what is in the command bytes.

I'm open to suggestions.

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

It was a new hope.
					-- Dust Puppy
User Friendly, 12/25/1998

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: PATCH: exclude certain commands from emulated SCSI hosts

[Linus Torvalds]

On Sat, 22 Mar 2003, Matthew Dharm wrote:
> 
> As I see it, SCSI commands break down into two basic categories:  common
> and uncommon.  Common things (basic read and write, 36-byte INQUIRY, eject,
> etc.) are all fine, but the 'uncommon' things (checking cache type,
> 255-byte INQUIRY, etc) cause problems.  I'm trying to find a way to choke
> off the problematic commands without having to write code to recognize what
> is being sent (and choke it off) based on what is in the command bytes.
> 
> I'm open to suggestions.

How about making the SCSI stuff pass a "common" flag (or "required") down
with the command? Then, a emulated thing could just decide to punt all 
commands with an immediate failure if they aren't marked "required".

That still _allows_ the driver to implement it if it wants to, unlike your 
previous approach.

		Linus

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 841 bytes --]

On Sat, Mar 22, 2003 at 11:39:05PM -0800, Linus Torvalds wrote:
> How about making the SCSI stuff pass a "common" flag (or "required") down
> with the command? Then, a emulated thing could just decide to punt all 
> commands with an immediate failure if they aren't marked "required".
> 
> That still _allows_ the driver to implement it if it wants to, unlike your 
> previous approach.

That seems reasonable... but we need to define a standard way to
reject/fail non-common commands.  Special sense data?  Special result
code?  It doesn't really matter to me, but we need to pick something.

Do you have a preference?

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

It was a new hope.
					-- Dust Puppy
User Friendly, 12/25/1998

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [usb-storage] Re: PATCH: exclude certain commands from emulated SCSI hosts

[Douglas Gilbert]

Matthew Dharm wrote:
> On Sat, Mar 22, 2003 at 11:39:05PM -0800, Linus Torvalds wrote:
> 
>>How about making the SCSI stuff pass a "common" flag (or "required") down
>>with the command? Then, a emulated thing could just decide to punt all 
>>commands with an immediate failure if they aren't marked "required".
>>
>>That still _allows_ the driver to implement it if it wants to, unlike your 
>>previous approach.
> 
> 
> That seems reasonable... but we need to define a standard way to
> reject/fail non-common commands.  Special sense data?  Special result
> code?  It doesn't really matter to me, but we need to pick something.
> 
> Do you have a preference?

Matt,
Evidentally your driver (and/or sbp2) synthesizes INQUIRY
responses in some cases. So if you think a SCSI command
is risky then why not follow the standard and synthesize
the scsi status and auto-sense buffer. Here is a modern
rendition from a (draft) standard:

"If a device server receives a CDB containing an operation
code that is invalid or not supported, it shall return
CHECK CONDITION status with the sense key set to ILLEGAL
REQUEST and an additional sense code of INVALID COMMAND
OPERATION CODE." [SPC-3, T10/1416-D Revision 10 2002/11/10]

If it is a field within a command that is in doubt (e.g.
the enable Vital Product Data (EVPD) bit in an INQUIRY)
then the additional sense code becomes INVALID FIELD IN
CDB.

In the case of a disk the commands that must be
supported are:
    - 36 byte INQUIRY (EVPD=0, CmdDt=0)
    - READ CAPACITY
    - READ (10 byte)
    - WRITE (10 byte)

For SCSI mid level scan code and the sd
disk initialization code a flag could be added to
struct scsi_cmnd to indicate that "I can live without
knowing the answer to this command".

I always wanted the emulated flag generalized (e.g.
indicating SPI, ATAPI, iSCSI, FC, SAS, SBP-2,
USB_mass_storage). Knowledge of the SCSI transport
protocol is important for device discovery,
identification and asynchronous notification sources.
As a boolean flag, "emulated" has outlived its usefulness.
With sysfs we can now find which hardware system a
SCSI low level driver belongs  to (e.g. SCSI Parallel
Interface (SPI) adapters are either PCI or ISA).

Doug Gilbert