Re: usb storage traces

Re: usb storage traces

[Alan Stern]

Here is a digested version of a log trace contributed by Alex Sanks.  It 
was collected from a generic USB Bulk-only storage device (actually the 
g_file_storage file-backed storage gadget) attached to a host running 
Windows XP SP1.  It agrees quite well with the results of my own tests 
using USB Snoopy on a Windows 2000 host.

In this trace, the number following the command name is the command length
in decimal; following it are the command bytes in hex, then the direction
and transfer length in decimal.


INQUIRY		6: 12 00 00 00 24 00  In 36
Vendor-specific	10: 23 00 00 00 00 00 00 00 fc 00  In 252
	Provokes an Invalid Command error
REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18
	Bug in Windows: REQUEST-SENSE is really a 6-byte command

Vendor-specific	10: 23 00 00 00 00 00 00 00 fc 00  In 252
	Retry previous failed command; it still doesn't work
REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18
Vendor-specific	10: 23 00 00 00 00 00 00 00 fc 00  In 252
	Another futile retry
REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18

READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
	Provokes Unit Attention: Reset Occurred
REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18
READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
	This time it works

READ		10: 28 00 00 00 00 00 00 00 01 00  In 512
	Read the first sector
READ		10: 28 00 00 00 00 00 00 00 01 00  In 512
	Windows likes reading the first sector
READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
	It also likes reading the disk capacity

MODE-SENSE	6: 1a 00 1c 00 c0 00  In 192
	Page 1c is listed as reserved; I don't know what it is;
	provokes Invalid Field in CDB error
REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18

MODE-SENSE	6: 1a 00 3f 00 c0 00  In 192
	So page 3f should work if we request 192 bytes!

MODE-SENSE	6: 1a 00 08 00 c0 00  In 192
	And so should page 08!

MODE-SELECT	6: 15 10 00 00 18 00  Out 24
	Although not shown here, a USB Snoopy trace reveals that this
	attempts to set the WCE (write cache enable) bit in page 8;
	here it provokes Invalid Field in CDB error
REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18

MODE-SELECT	6: 15 10 00 00 18 00  Out 24
	Retry of previous failed command; it fails again
REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18

READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
	Windows really likes to get the capacity!
READ		10: 28 00 00 00 00 00 00 00 01 00  In 512
	And it really likes to read the first sector!
READ		10: 28 00 00 00 00 00 00 00 01 00  In 512
READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
READ		10: 28 00 00 00 00 00 00 00 01 00  In 512
READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
	Ommitted: this command was repeated 8 times

TEST-UNIT-READY	6: 00 00 00 00 00 00  
MODE-SENSE	6: 1a 00 00 00 0c 00  In 12
	I don't know what Windows expects to find in page 0;
	but this fails
REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18

READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
	Ommitted: this command was repeated 3 times

TEST-UNIT-READY	6: 00 00 00 00 00 00  
MODE-SENSE	6: 1a 00 00 00 0c 00  In 12
	Fails again
REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18
READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8


Some of the activity may depend on the contents of the partition table
stored in the first sector.  But it seems clear that, subject to the
unknown function of command x23 and of mode page x1c, we might be able to
work with devices that advertise themselves as Bulk-only by requesting 192
bytes from page x3f and page 8.

Does anybody know what command x23 and mode page x1c do?  Although 
nominally vendor-specific, they must be reasonably standardized if Windows 
can get away with using them on a generic device.

Unfortunately, there's nothing to stop a manufacturer from supplying their
own driver which would avoid reading those pages.  So even an apparently
normal device might react badly to these commands.  However I think it's 
still worth a try.

Alan Stern

Re: usb storage traces

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 5346 bytes --]

Did the device advertise 'generic scsi' or one of the other protocol codes?

Microsoft has published information that indicates that they 'translate'
the mode-sense commands into the 10-byte variants for everything bug
'transparent scsi' -- I think actually seeing that information would be
necessary in constructing a new algorithm for Linux.

Matt

On Thu, Dec 04, 2003 at 01:30:16PM -0500, Alan Stern wrote:
> Here is a digested version of a log trace contributed by Alex Sanks.  It 
> was collected from a generic USB Bulk-only storage device (actually the 
> g_file_storage file-backed storage gadget) attached to a host running 
> Windows XP SP1.  It agrees quite well with the results of my own tests 
> using USB Snoopy on a Windows 2000 host.
> 
> In this trace, the number following the command name is the command length
> in decimal; following it are the command bytes in hex, then the direction
> and transfer length in decimal.
> 
> 
> INQUIRY		6: 12 00 00 00 24 00  In 36
> Vendor-specific	10: 23 00 00 00 00 00 00 00 fc 00  In 252
> 	Provokes an Invalid Command error
> REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18
> 	Bug in Windows: REQUEST-SENSE is really a 6-byte command
> 
> Vendor-specific	10: 23 00 00 00 00 00 00 00 fc 00  In 252
> 	Retry previous failed command; it still doesn't work
> REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18
> Vendor-specific	10: 23 00 00 00 00 00 00 00 fc 00  In 252
> 	Another futile retry
> REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18
> 
> READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
> 	Provokes Unit Attention: Reset Occurred
> REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18
> READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
> 	This time it works
> 
> READ		10: 28 00 00 00 00 00 00 00 01 00  In 512
> 	Read the first sector
> READ		10: 28 00 00 00 00 00 00 00 01 00  In 512
> 	Windows likes reading the first sector
> READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
> 	It also likes reading the disk capacity
> 
> MODE-SENSE	6: 1a 00 1c 00 c0 00  In 192
> 	Page 1c is listed as reserved; I don't know what it is;
> 	provokes Invalid Field in CDB error
> REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18
> 
> MODE-SENSE	6: 1a 00 3f 00 c0 00  In 192
> 	So page 3f should work if we request 192 bytes!
> 
> MODE-SENSE	6: 1a 00 08 00 c0 00  In 192
> 	And so should page 08!
> 
> MODE-SELECT	6: 15 10 00 00 18 00  Out 24
> 	Although not shown here, a USB Snoopy trace reveals that this
> 	attempts to set the WCE (write cache enable) bit in page 8;
> 	here it provokes Invalid Field in CDB error
> REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18
> 
> MODE-SELECT	6: 15 10 00 00 18 00  Out 24
> 	Retry of previous failed command; it fails again
> REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18
> 
> READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
> 	Windows really likes to get the capacity!
> READ		10: 28 00 00 00 00 00 00 00 01 00  In 512
> 	And it really likes to read the first sector!
> READ		10: 28 00 00 00 00 00 00 00 01 00  In 512
> READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
> READ		10: 28 00 00 00 00 00 00 00 01 00  In 512
> READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
> 	Ommitted: this command was repeated 8 times
> 
> TEST-UNIT-READY	6: 00 00 00 00 00 00  
> MODE-SENSE	6: 1a 00 00 00 0c 00  In 12
> 	I don't know what Windows expects to find in page 0;
> 	but this fails
> REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18
> 
> READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
> 	Ommitted: this command was repeated 3 times
> 
> TEST-UNIT-READY	6: 00 00 00 00 00 00  
> MODE-SENSE	6: 1a 00 00 00 0c 00  In 12
> 	Fails again
> REQUEST-SENSE	12: 03 00 00 00 12 00 00 00 00 00 00 00  In 18
> READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
> READ-CAPACITY	10: 25 00 00 00 00 00 00 00 00 00  In 8
> 
> 
> Some of the activity may depend on the contents of the partition table
> stored in the first sector.  But it seems clear that, subject to the
> unknown function of command x23 and of mode page x1c, we might be able to
> work with devices that advertise themselves as Bulk-only by requesting 192
> bytes from page x3f and page 8.
> 
> Does anybody know what command x23 and mode page x1c do?  Although 
> nominally vendor-specific, they must be reasonably standardized if Windows 
> can get away with using them on a generic device.
> 
> Unfortunately, there's nothing to stop a manufacturer from supplying their
> own driver which would avoid reading those pages.  So even an apparently
> normal device might react badly to these commands.  However I think it's 
> still worth a try.
> 
> Alan Stern
> 
> 
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

Hey Chief.  We've figured out how to save the technical department.  We 
need to be committed.
					-- The Techs
User Friendly, 1/22/1998

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: usb storage traces

[Alan Stern]

On Thu, 4 Dec 2003, Matthew Dharm wrote:

> Did the device advertise 'generic scsi' or one of the other protocol codes?
> 
> Microsoft has published information that indicates that they 'translate'
> the mode-sense commands into the 10-byte variants for everything bug
> 'transparent scsi' -- I think actually seeing that information would be
> necessary in constructing a new algorithm for Linux.
> 
> Matt

It was generic (transparent) scsi.

Perhaps I'll try implementing some of the other protocol codes in FSG,
maybe also the "removeable media" flag.  This would also require
implementing the PREVENT-ALLOW-MEDIUM-REMOVAL, READ-FORMAT-CAPACITIES, and
START-STOP, possibly also REZERO-UNIT, SEEK, READ(12), and WRITE(12).

(1).  Am I missing anything important?

(2).  Can I afford to leave out some of those?  The last four don't look 
very useful.

As I understand it, the only real difference between the protocols is that
everything but transparent scsi and RBC requires padding the commands to
12 bytes.  Pat LaVarre says that protocols x02 (SFF-8020i = CD/DVD) and
x05 (SFF-8070i = Removeable media, floppy) are the most important, since
they are the only other ones the generic Windows USB mass storage drivers
recognize.

Alan Stern

Re: usb storage traces

[Matthew Dharm]

[-- Attachment #1: Type: text/plain, Size: 1559 bytes --]

On Thu, Dec 04, 2003 at 03:59:58PM -0500, Alan Stern wrote:
> On Thu, 4 Dec 2003, Matthew Dharm wrote:
> 
> > Did the device advertise 'generic scsi' or one of the other protocol codes?
> > 
> > Microsoft has published information that indicates that they 'translate'
> > the mode-sense commands into the 10-byte variants for everything bug
> > 'transparent scsi' -- I think actually seeing that information would be
> > necessary in constructing a new algorithm for Linux.
> 
> It was generic (transparent) scsi.
> 
> Perhaps I'll try implementing some of the other protocol codes in FSG,
> maybe also the "removeable media" flag.  This would also require
> implementing the PREVENT-ALLOW-MEDIUM-REMOVAL, READ-FORMAT-CAPACITIES, and
> START-STOP, possibly also REZERO-UNIT, SEEK, READ(12), and WRITE(12).
> 
> (1).  Am I missing anything important?

Based on the information I've seen, any one of the other protocol codes
will be fine -- you don't need to do all, just transparent SCSI and one
other.  Of course, all codes would be good for testing.

> (2).  Can I afford to leave out some of those?  The last four don't look 
> very useful.

I agree about the last four.  I can't think of anything else that might be
good to test.

Matt

-- 
Matthew Dharm                              Home: mdharm-usb@one-eyed-alien.net 
Maintainer, Linux USB Mass Storage Driver

G:  Money isn't everything, A.J.
AJ: Who convinced you of that?
G:  The Chief, at my last salary review.
					-- Mike and Greg
User Friendly, 11/3/1998

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: usb storage traces

[Pat LaVarre]

> > From: Matthew Dharm ...
> >
> > Microsoft has published information that indicates that they
> > 'translate' the mode-sense commands into the 10-byte variants for
> > everything bug 'transparent scsi' -- I think actually seeing that
> > information would be necessary in constructing a new algorithm for
> > Linux.
> ...
> As I understand it, the only real difference between the protocols is that
> everything but transparent scsi and RBC requires padding the commands to
> 12 bytes.

diff vs. bInterfaceSubClass=x06 as sketched by MSFT is as follows.

---

From: http://www.microsoft.com/whdc/hwdev/bus/usb/usbfaq.mspx
...

Usbstor.sys and Device Class Support

Q: Which device classes does usbstor.sys understand?

In Windows XP, Windows 2000, and Windows Me, usbstor.sys makes only one
distinction: 

bInterfaceSubClass == 06h 
- OR - 
bInterfaceSubClass != 06h 

The value bInterfaceSubClass == 06h means that:

Command descriptor blocks (CDBs) should not be padded to 12 bytes.

Mode Sense / Mode Select commands should not be translated from 1AH /
15h to 5AH / 55h.

Subclass 0x06 should generally be used for flash memory devices.

The value bInterfaceSubClass != 06h means that:

CDBs should be padded to 12 bytes.

Mode Sense / Mode Select commands should be translated from 1AH / 15h to
5AH / 55h.

---

Includes a plea I nutshell as { generic usb flash = x 08 06 50 }.

To make sense of this text we have to remember Linux usb-storage
"protocols" = usb.org bInterfaceSubClass != usb.org bInterfaceProtocol.

Pat LaVarre

Re: usb storage traces

[Pat LaVarre]

> Pat LaVarre says that protocols x02 (SFF-8020i = CD/DVD) and 
> x05 (SFF-8070i = Removeable media, floppy) are the most important, since
> they are the only other ones the generic Windows USB mass storage drivers
> recognize.

I mean to be paraphrasing the windows/inf/usbstor.inf quotation:

...
[Generic] 
%GenericBulkOnly.DeviceDesc%=USBSTOR_BULK, USB\Class_08&SubClass_02&Prot_50 
%GenericBulkOnly.DeviceDesc%=USBSTOR_BULK, USB\Class_08&SubClass_05&Prot_50 
%GenericBulkOnly.DeviceDesc%=USBSTOR_BULK, USB\Class_08&SubClass_06&Prot_50 

[...

>From that quotation I ignorantly guess that plugging anything other than
x 08 (02|05|06) 50 into Win XP/ 2K/ ME requires an idVendor:idProduct
match (possibly already found elsewhere in Windows e.g. elsewhere in
that same file), else installing a driver (possibly as trivial as
additional .inf text) to match the device.

Pat LaVarre

Re: usb storage traces

[Pat LaVarre]

> As I understand it, the only real difference between the protocols is
> that everything but transparent scsi and RBC requires padding the
> commands to 12 bytes.

If we dig I believe we can unearth specific binary incompatibilities
among the de jure paper definitions of bInterfaceSubClass.

Me, I invented bInterfaceSubClass = x06 in 1998 to give my device a way
of reporting that I chose to perpetuate my employer's binary-code-only
legacy of an ANSI T10 SPC interpretation, rather than abandoning that
legacy in favour of the technically binary incompatible SFF proposal
which had not yet become ANSI T10 MMC.

For example, SFF/ MMC requires the device to zero the the byte 2 Version
of op x12 Inquiry data, SCSI-2/ SPC recommends nonzero.  SFF/ MMC
requires the device to exclude block descriptors from mode sense data,
SCSI-2/ SPC recommends inclusion.

Hopefully we will find SFF/ MMC never requires something SCSI-2/ SPC
actually forbids.

Hopefully we will find we only dig up insignificant incompatibilities.

I think I remember these binary incompatibilities didn't appear
officially incorporated into ANSI T10 until bleeding-edge T10 (= MMC 4,
trails SFF) i.e. may be absent from last stable T10 (= MMC 3).

Frustrates me that clueless folk repeatedly design pointless binary
incompatibility into competing SCSI definitions.  For example, the SFF
folk could have ducked the block descriptor issue by requiring the host
to set the standard mode sense cdb bit that disables block descriptors. 
Instead the SFF folk pointlessly chose to require the host and the
device to redefine what a zero bit there means.  I've heard the latest
SFF drafts try to patch up this mess somehow.

Truth or slander I do not know, but I've heard that the RBC folk messed
up legacy mode sense, and that some FireWire folk neglected to implement
legacy op x12 Inquiry (!!!).

Once upon a time, the usb.org bInterfaceSubClass = x06 specifications
very carefully referenced no specific de jure paper.  The original
English for x06 was something like "Windows, Linux, Mac, Sun SCSI".
People lost the sense of that by voting in the replacement text
"Transparent SCSI". In the time since, people may have polluted that
accurately pure description by linking to technically less than
completely accurate texts.

So far as I know, we have no executable specification of SCSI in open
source, unless you count Linux itself, which by being designed to work
consciously includes such redefining exceptions as the
US_FL_FIX_CAPACITY of drivers/usb/storage/unusual_devs.h.

Pat LaVarre

Re: usb storage traces

[Pat LaVarre]

> > > > As before, if still now always we each launch a new thread
> > > > to reply to posts of this thread, then this thread will grow
> > > > to contain only an index of usb storage traces.
> > >
> > > Subject: Re: usb storage traces
> > Subject: Re: usb storage traces
> Subject: Re: usb storage traces

Ouch I see we've lost control of this thread.  Hopefully someone better
connected than I will volunteer ...

... to go create a web page offline to keep track of contributed
usb-storage traces.

Pat LaVarre

Re: usb storage traces

[Pat LaVarre]

> Does anybody know what ... mode page x1c do?  Although 
> nominally vendor-specific, they must be reasonably standardized if Windows 
> can get away with using them on a generic device.

I fetch my SPC guess from the web as follows.  As ever we cannot know if
MMC contradicts unless we check nearby.  To sort these clicks from most
to least applicable I logged them in reverse chronological order.

1Ch = Informational Exceptions Control

page 297 of 467 ...
7.4.5 Mode page and subpage formats and page codes ...
page 299 of 467 ...
Table 222 -- Mode page codes

SCSI Primary Commands - 3 (SPC-3)
http://www.t10.org/ftp/t10/drafts/spc3/spc3r15.pdf

SPC-3 SCSI Primary Commands - 3 ...
http://www.t10.org/drafts.htm#spc3

ANSI T10
http://www.t10.org/scsi-3.htm

Fun: Ramshackle World
http://members.aol.com/ppaatt/essay/why/

Pat LaVarre
http://members.aol.com/ppaatt/

http://www.google.com/search?q=Pat+LaVarre

Re: usb storage traces

[Pat LaVarre]

> MODE-SENSE	6: 1a 00 00 00 0c 00  In 12
> 	I don't know what Windows expects to find in page 0;
> 	but this fails

Last I checked, page 0 was vendor-specific.  I don't clearly remember if
the header before page 0 was standard.  If yes, then Win could be
legitly fetching header alone e.g. to check write-protect.

Pat LaVarre

Informational Exception mpage [was: usb storage traces]

[Douglas Gilbert]

Pat LaVarre wrote:
>>Does anybody know what ... mode page x1c do?  Although 
>>nominally vendor-specific, they must be reasonably standardized if Windows 
>>can get away with using them on a generic device.
> 
> 
> I fetch my SPC guess from the web as follows.  As ever we cannot know if
> MMC contradicts unless we check nearby.  To sort these clicks from most
> to least applicable I logged them in reverse chronological order.
> 
> 1Ch = Informational Exceptions Control

The latest drafts of MMC-3 and MMC-4 call this mode page:
"Fault/Failure Reporting Control Page". Looking at the
definition in MMC-4 it is basically the same as the
Informational Exceptions mode page defined in SPC-3 with
a few fields reserved.

Just to confuse things further there is an Informational
Exceptions _log_ page that started life as the SMART log page.
Seems as though someone in t10 is keen on the term
"Informational Exceptions" but other groups are not so
enthusiastic.

Various command sets (e.g. SSC (tapes) and MMC (cd/dvd)) "fine
tune" the defintion of mode pages. Mode page 0x1c is the
only one in which the name changes that I have noticed.
SSC also overloads this mode page but keeps the SPC-3 name.

In work I'm doing on sginfo to decode a mode page, first the
peripheral device type is obtained from an INQUIRY. Then a
peripheral device type specific list is checked; if there is
no match then the common list (from SPC-3) is checked.

There is another dimension of mode pages for different
transport protocols (e.g. FCP, SPI-4 and SAS). It is based
around the Protocol Specific lu/port pages (0x18 and 0x19
and their subpages). Pity that USB is not a SPC-3 recognised
transport (evidentally ATAPI will be added shortly).
But I digress ...

Doug Gilbert

Re: usb storage traces

[Douglas Gilbert]

Alan Stern wrote:

<snip/>

> Does anybody know what command x23 

READ FORMAT CAPACITIES (found in MMC-3 and MMC-4)

> and mode page x1c do?  Although 
> nominally vendor-specific, 

Vendor specific mode pages start at 0x20 so mode
page 0x1c (Informational Exceptions) is not vendor
specific.

<snip/>

Doug Gilbert

Re: usb storage traces

[Patrick Mansfield]

On Thu, Dec 04, 2003 at 03:59:58PM -0500, Alan Stern wrote:

> It was generic (transparent) scsi.
> 
> Perhaps I'll try implementing some of the other protocol codes in FSG,
> maybe also the "removeable media" flag.  This would also require
> implementing the PREVENT-ALLOW-MEDIUM-REMOVAL, READ-FORMAT-CAPACITIES, and
> START-STOP, possibly also REZERO-UNIT, SEEK, READ(12), and WRITE(12).

What is FSG?

-- Patrick Mansfield

Re: usb storage traces

[Alan Stern]

On 4 Dec 2003, Pat LaVarre wrote:

> > Does anybody know what ... mode page x1c do?  Although 
> > nominally vendor-specific, they must be reasonably standardized if Windows 
> > can get away with using them on a generic device.

According to my newly-located source, page x1c is the Timer and Protect
page.  It includes values for the Inactivity Time Multiplier, Disable
Media Access, and Software Write Protect.

Windows may use it to try and set the drive's spin-down time.

Incidentally, Pat, can you confirm whether bInterfaceSubClass x01 = RBC 
(Reduced Block Commands, T10 project 1240-D) uses 0-padding of commands to 
12 bytes or not?  The Linux usb-storage driver seems to think that it 
doesn't.

Alan Stern

Re: Informational Exception mpage [was: usb storage traces]

[Alan Stern]

On Fri, 5 Dec 2003, Douglas Gilbert wrote:

> The latest drafts of MMC-3 and MMC-4 call this mode page:
> "Fault/Failure Reporting Control Page". Looking at the
> definition in MMC-4 it is basically the same as the
> Informational Exceptions mode page defined in SPC-3 with
> a few fields reserved.

According to the USB Mass Storage Class - UFI Command Specification, which 
might be more applicable in this circumstance, page x1c is defined as the 
Timer and Protect page.

> Just to confuse things further there is an Informational
> Exceptions _log_ page that started life as the SMART log page.
> Seems as though someone in t10 is keen on the term
> "Informational Exceptions" but other groups are not so
> enthusiastic.
> 
> Various command sets (e.g. SSC (tapes) and MMC (cd/dvd)) "fine
> tune" the defintion of mode pages. Mode page 0x1c is the
> only one in which the name changes that I have noticed.
> SSC also overloads this mode page but keeps the SPC-3 name.

So it's hard to say whether Windows uses this page to control SMART 
logging, disk spin-down times, or something else.  I guess it's just as 
well that the devices I've looked at don't support it.

Alan Stern

Re: usb storage traces

[Alan Stern]

On Thu, 4 Dec 2003, Patrick Mansfield wrote:

> On Thu, Dec 04, 2003 at 03:59:58PM -0500, Alan Stern wrote:
> 
> > It was generic (transparent) scsi.
> > 
> > Perhaps I'll try implementing some of the other protocol codes in FSG,
> > maybe also the "removeable media" flag.  This would also require
> > implementing the PREVENT-ALLOW-MEDIUM-REMOVAL, READ-FORMAT-CAPACITIES, and
> > START-STOP, possibly also REZERO-UNIT, SEEK, READ(12), and WRITE(12).
> 
> What is FSG?
> 
> -- Patrick Mansfield

Sorry about not explaining that.  FSG is my File-backed Storage Gadget.  
It's a driver for a Linux-based USB _device_ (as opposed to _host_) that 
emulates a USB Mass Storage device, using a file or block device as 
backing storage in somewhat the same manner as the loop driver.

Since FSG is totally under our control, it's perfect for logging the 
sequence of commands issued by an arbitrary host when connected to a 
generic USB storage device.  Unfortunately, although I am keenly 
interested in seeing the results of such tests against a variety of 
Windows hosts, I don't have any USB device hardware capable of running 
Linux.  So I have to rely on tests done by the relatively few people who 
do have such hardware.

Alan Stern

Re: Informational Exception mpage [was: usb storage traces]

[Pat LaVarre]

> According to the USB Mass Storage Class - UFI Command Specification, which 
> might be more applicable in this circumstance,

Please NO!!!!!!!!!!

On this digressing point, I think all the applicable English texts
actually agree with each other and with reality.

UFI applies exclusively to usb-storage devices with the corresponding
bInterfaceSubClass.  I think I remember that is x04, not found in Win
generic x 08 (02|05|06) 50.

Personally I haven't yet had to grok UFI.  I could choose instead to
study only ANSI T10 SCSI and SFF SCSI and USB Bootable SCSI.  Each of
those have dozens of subflavours.  I don't want to have to start
studying a fourth major flavour as yet confined only to USB1FS CBI/CB
FDD, though rumoured to be coming to USB2HS BBB.

Pat LaVarre

Re: usb storage traces

[Pat LaVarre]

> > What is FSG?
> ...
> ... File-backed Storage Gadget.  
> It's a driver for a Linux-based USB _device_ (as opposed to _host_) that 
> emulates a USB Mass Storage device, using a file or block device as 
> backing storage in somewhat the same manner as the loop driver.
> ... Unfortunately, ...

Wow, our lead FSG developer has no Linux USB _device_.

Ouch.  Some sponsor with money should fix that.

> ...

Mostly I'm writing to mention, recently one of the semi-private
usb-storage ancestors of this thread also gave me:

http://www.linux-usb.org/gadget/

explained as follows.

Pat LaVarre

-----Forwarded Message-----
Subject: Re: [usb-storage] g_file_storage logs for Windows hosts
Date: 04 Dec 2003 15:59:45 -0700

Pat LaVarre wrote:

> What I lack is (a) free time and (b) a tutorial I can rapidly
> assimilate for how I fetch and deploy current fsg to h/w I own.

See www.linux-usb.org/gadget for the bare info.  Clone kernel from BK
(or just pull into your own repository), configure your kernel, build
and install with the file storage gadget; "modprobe g_file_storage ..."
as shown on that page.  That assumes you have appropriate hardware.

> Possibly the only appropriate hardware I have is a once-free-with-msdn
> "ViewSonic" "Pocket PC V37".  On that, Windows --> Settings --> System
> --> About reports "Intel PXA255" "36.45 MB".

Hardware that will run ... FSG code includes 2.4 or 2.6 systems
running Linux is less common than generic PCs or USB disks:

  - PXA 2xx hardware, which tends to be semi-custom.  www.handhelds.org
    is a good source of info.  (They don't list that ViewSonic, and
    focus on iPaqs even though Zaurii run Linux too.)

  - PCI-based hardware with a net2280 (www.netchip.com); simplest as
    a PCI card in a generic PC.  (This is the high speed option.)

  - PCI-based hardware with a Toshiba TC86c001 chip, includes some
    MIPS systems; a PCI card version needs a 3.3V slot.

  - ... can probably say something about the SuperH hardware
    that runs it; this support is 2.4-only for now.

I understand there are drivers for some other device-side USB
controllers which are in development, or otherwise not widely
circulated.  Such as some other NetChip hardware, Cypress SX2, some
other ARM-based SOCs, and more.
...

Re: usb storage traces

[David Brownell]

Pat LaVarre wrote:
>>>What is FSG?
>>
>>...
>>... File-backed Storage Gadget.  
>>It's a driver for a Linux-based USB _device_ (as opposed to _host_) that 
>>emulates a USB Mass Storage device, using a file or block device as 
>>backing storage in somewhat the same manner as the loop driver.
>>... Unfortunately, ...
> 
> 
> Wow, our lead FSG developer has no Linux USB _device_.

Well, he does have the "dummy_hcd" which lets him test
on the same machine running the USB host ... I'll leave
it to experts to contrast that with CONFIG_SCSI_DEBUG.

The driver was basically working before it hit real
hardware.


> Ouch.  Some sponsor with money should fix that.

Or Santa Claus ... ;)

- Dave

bCWBCBLength is cb length no when

[Pat LaVarre]

> Incidentally, Pat, can you confirm whether bInterfaceSubClass x01 = RBC 
> (Reduced Block Commands, T10 project 1240-D) uses 0-padding of commands to 
> 12 bytes or not?  The Linux usb-storage driver seems to think that it 
> doesn't.

I can neither concisely confirm, nor concisely deny, sorry.

Want a discussion?

I believe we're asking:

---

Is the bCWBCBLength the length of the "cdb" or is it the length of the
cdb "plus pad"?  Specifically when we see bInterfaceSubClass equal to
the hex we've chosen to name "RBC"?

---

I have nine answers:

a) So far as I know as yet there is no public specification in English
that claims one answer or the other.

b) I personally know "cdb" length = "cb" "length" = bCBWCBLength was
original intent, first sketched one afternoon in a window-free West-side
room in August 1998, inherited from the Dos Aspi tradition.

c) I agree "plus pad" design traditions have arisen to assault us.

d) I'm sorry to see the "plus pad" traditions arise, because, by
definition, a transport that cannot know cdb length cannot accurately
report whether copying the cdb out worked or not.  I see the "plus pad"
traditions as indefensible oversimplifications.  To be "conservative in
what you send", hosts should pad with something determine, like zeroes
or a copy of the cb length.  Rumour i.e. truth or slander says mfst has
been seen to write noisy pad in place of the cdb[cb_length - 1] byte.

e) msft in bus traces sends 12 byte op x03 Request Sense cdb's in auto
sense of bInterfaceSubClass x06 SCSI.  That means "to work with Windows 
we have to miscompare": we have to quietly discard bytes of data a
portion of the cdb mfst claims to be asking us to send.  Welcome to the
real world.

f) msft on the web tells us "cdb" for bInterfaceSubClass x06 T10 SPC
SCSI, else "plus pad", except this "else" only clearly includes the
by-msft-generic bInterfaceSubClass x02|05.

g) rbc scsi may be mostly a firewire thing.

h) sff scsi over ide (aka atapi) has a "plus pad" tradition we could
blame for the "plus pad" msft design choice of bInterfaceSubClass
x02|05.

i) scsi over firewire has a "plus pad" tradition, I think. 
[Specifically I argue as detailed in the e-mail of this morning quoted
in full below.]

---

I cannot conclude confidently either way.

By a+b+d I say "cdb" for RBC, as for all forms of T10 SPC SCSI over USB.

By c+e+f+g+h+i I say "plus pad" for RBC, as for all forms of SFF/ T10
MMC SCSI over USB.

I feel a+b+d >= c+e+f+g+h+i.  Perhaps only because of my personal
involvement in b.  Perhaps those sums appear less equally balanced to
you?

Pat LaVarre
-----Forwarded Message-----


From: Pat LaVarre <p.lavarre@ieee.org>
To: stds-1394@ieee.org
Subject: sbp cdb length max 12 rumoured
Date: 05 Dec 2003 08:44:54 -0700

Hi.  I think I remember scsi over firewire by design chokes over cdb
longer than 12 bytes and bus traces cannot b=distinguish between such
cdb lengths as 6, 10, and 12.

Do I remember falsely?

More specifically I think I remember normative annex b of sbp 2 copies
out the cdb via the last three quadlets of an orb, with no explicitly
redundant indication of orb length to tell us more than three quadlets
might be there, and no redundant indication of cdb length to tell us how
many of the bytes at the end of the three quadlets are noise.

I actually at first failed to remember all this detail myself without
first querying a friend face to face.

Do we remember falsely?

Anyone willing to comment?

Perhaps you can tell I long ago forgot the url's to the (final draft?)
English that would give me this answer on the web.

Thanks in advance, Pat LaVarre

P.S. I ask by way of relaying a question from a Linux USB RBC developer.

P.P.S. Once upon a time people here were interested on how impossible it
is to defeat the sometimes selected Microsoft Outlook default of
uuencoding mail.  I believe the technique I'm using here now can work on
any PC, by rebooting via DVD/ CD into Knoppix Linux in place of
Windows.  Mind you, I'm not completely confident that this e-mail is not
uuencoded, since by now I have also forgotten the url of the web service
that bounces a plain text copy of my headers and mail body to me.

Re: usb storage traces

[Pat LaVarre]

> According to my newly-located source,

Source has url?

> page x1c ... includes ... Software Write Protect.

I remember software write protect as being one of the ways in which the
SFF 8070i Compaq LS-120 manual was an inaccurate description of other
RMB PDT x00 HDD/ Flash.  Also bleeding-edge MMC has a totally different
"SWP".

> Windows may use it to try and set the drive's spin-down time.

Sorry I do not know.

Pat LaVarre

Re: usb storage traces

[Pat LaVarre]

> > Wow, our lead FSG developer has no Linux USB _device_.
> 
> Well, he does have the "dummy_hcd" which lets him test
> on the same machine running the USB host ... I'll leave
> it to experts to contrast that with CONFIG_SCSI_DEBUG.

Ah.

Does that mean I likewise can test fsg with a PC, rather than having to
divine how to burn Linux into my 'once-free-with-msdn "ViewSonic"
"Pocket PC V37" whose Windows --> Settings --> System > --> About
reports "Intel PXA255" "36.45 MB"'?

Pat LaVarre

P.S. With CONFIG_SCSI_DEBUG as yet personally I have gotten no further
than finding it in `make xconfig` and changing the default off to =m.

Re: usb storage traces

[Alan Stern]

On 5 Dec 2003, Pat LaVarre wrote:

> > > Wow, our lead FSG developer has no Linux USB _device_.
> > 
> > Well, he does have the "dummy_hcd" which lets him test
> > on the same machine running the USB host ... I'll leave
> > it to experts to contrast that with CONFIG_SCSI_DEBUG.
> 
> Ah.
> 
> Does that mean I likewise can test fsg with a PC, rather than having to
> divine how to burn Linux into my 'once-free-with-msdn "ViewSonic"
> "Pocket PC V37" whose Windows --> Settings --> System > --> About
> reports "Intel PXA255" "36.45 MB"'?

Well, sort of.  You can indeed run FSG using the dummy_hcd driver under 
Linux.  It's good enough for testing and seeing how things work.  If you 
want more detailed instructions I'll send them later.

But you can't connect to another host.  dummy_hcd is strictly a "loopback"  
adapter.  So FSG/dummy_hcd can talk to usb-storage but there's no way to
hook it up to Windows (not even using VMWare).

Alan Stern

Re: usb storage traces

[Alan Stern]

On 5 Dec 2003, Pat LaVarre wrote:

> > page x1c ... includes ... Software Write Protect.
> 
> I remember software write protect as being one of the ways in which the
> SFF 8070i Compaq LS-120 manual was an inaccurate description of other
> RMB PDT x00 HDD/ Flash.  Also bleeding-edge MMC has a totally different
> "SWP".
> 
> > Windows may use it to try and set the drive's spin-down time.
> 
> Sorry I do not know.
> 
> Pat LaVarre

Never mind.  It's not worth worrying about; I just won't support it.

Alan Stern

Re: usb storage traces

[David Brownell]

Pat LaVarre wrote:
>>>Wow, our lead FSG developer has no Linux USB _device_.
>>
>>Well, he does have the "dummy_hcd" which lets him test
>>on the same machine running the USB host ... I'll leave
>>it to experts to contrast that with CONFIG_SCSI_DEBUG.
> 
> 
> Ah.
> 
> Does that mean I likewise can test fsg with a PC...

I don't think you can test windows interop that way,
even with windows emulators.  But yes, you could test
it against usb-storage.  That whole stack is available
through the "gadget-2.6" tree ... give it a whirl!

Re: usb storage traces

[Pat LaVarre]

> You can indeed run FSG using the dummy_hcd driver under 
> Linux.  It's good enough for testing and seeing how things work.  If you 
> want more detailed instructions I'll send them later.

Yes please.

cc linux-scsi@vger.kernel.org of course would give us a google url we
can use thereafter.

> But you can't connect to another host.  dummy_hcd is strictly a "loopback"  
> adapter.  So FSG/dummy_hcd can talk to usb-storage but there's no way to
> hook it up to Windows (not even using VMWare).

True til the lk win emulators let us plug usb devices into win.  Truth
or slander, I hear Microsoft's Mac OS X Virtual PC can do this.

Pat LaVarre

Re: bCWBCBLength is cb length no when

[Alan Stern]

On 5 Dec 2003, Pat LaVarre wrote:

> > Incidentally, Pat, can you confirm whether bInterfaceSubClass x01 = RBC 
> > (Reduced Block Commands, T10 project 1240-D) uses 0-padding of commands to 
> > 12 bytes or not?  The Linux usb-storage driver seems to think that it 
> > doesn't.
> 
> I can neither concisely confirm, nor concisely deny, sorry.
> 
> I cannot conclude confidently either way.

> By a+b+d I say "cdb" for RBC, as for all forms of T10 SPC SCSI over USB.
> 
> By c+e+f+g+h+i I say "plus pad" for RBC, as for all forms of SFF/ T10
> MMC SCSI over USB.
> 
> I feel a+b+d >= c+e+f+g+h+i.  Perhaps only because of my personal
> involvement in b.  Perhaps those sums appear less equally balanced to
> you?

Since usb-storage uses the a+b+d interpretation, that's what I'm inclined 
to go with until more evidence arrives.  But to be safe, it might be best 
to accept commands either with or without padding.


> P.P.S. Once upon a time people here were interested on how impossible it
> is to defeat the sometimes selected Microsoft Outlook default of
> uuencoding mail.  I believe the technique I'm using here now can work on
> any PC, by rebooting via DVD/ CD into Knoppix Linux in place of
> Windows.  Mind you, I'm not completely confident that this e-mail is not
> uuencoded, since by now I have also forgotten the url of the web service
> that bounces a plain text copy of my headers and mail body to me.

I don't know about the original message this postscript came from, but the 
message that came to me was not uuencoded at all.

Alan Stern

Re: bCWBCBLength is cb length no when

[Pat LaVarre]

> > > Incidentally, Pat, can you confirm whether bInterfaceSubClass x01 = RBC 
> > > (Reduced Block Commands, T10 project 1240-D) uses 0-padding of commands to 
> > > 12 bytes or not?  The Linux usb-storage driver seems to think that it 
> > > doesn't.
> > 
> > I can neither concisely confirm, nor concisely deny, sorry.
> ...
> Since usb-storage uses the a+b+d interpretation, that's what I'm inclined 
> to go with until more evidence arrives.  But to be safe, it might be best 
> to accept commands either with or without padding.

Oh, aye, sorry I answered with host context in mind, certainly I agree
"be liberal in what you accept" when speaking as the device receiving
the cdb.

Since as a device we know what the cdb[0] op means, we can develop an
independent idea of how long the cdb is.  With dreary conventionality,
let's call that length Do, and say Ho = bCBWCBLength.  Then concisely we
conclude:

1) Do = Ho everybody happy.

2) Ho < Do miscompare on purpose by quietly discarding cdb bytes out,
especially if Ho is 12 and/or pad is zero (rather than the other popular
choices of xFF and of copy of cb length)

3) Do < Ho miscompare on purpose by quietly fabricating zero pad.

That's the choice I shipped as usb/atapi bridge, mostly it works.

Something like (3) is explicitly required by 99 Sep bbb: text defining
CBW says something about device shall ignore cb bits past the cb length.

Sorry to say, yes actually I have seen (3) necessary to make sense of
the misspelled -i 8 -y "25 00 00:00:00:00" meaning -i 8 -y "25 00
00:00:00:00 00 00:00 00" i.e. Read Capacity.

> ...

However, since as a device we are aggressively choosing to recover from
the soft error of Do != Ho, ...

Please can we have a way of counting soft errors e.g. please can we have
at least one dmesg per boot to mention this msft/other host design
un/consciously disregarded "be conservative in what you send".

Pat LaVarre

P.S.

> message that came to me was not uuencoded at all.

Thanks for saying, I was curious.

Reverse chronological web trails celebrating "Linux live-CD" distros
defeating annoying Windows limitations include:

http://linux.oreillynet.com/pub/a/linux/2003/11/20/knoppix.html

http://lwn.net/Articles/59504/bigpage

draft not yet connected with ftp put privilege of
http://members.aol.com/plscsi/tools/knoppix/

Re: usb storage traces

[Randy.Dunlap]

On Fri, 05 Dec 2003 09:14:21 -0800 David Brownell <david-b@pacbell.net> wrote:

| Pat LaVarre wrote:
| >>>What is FSG?
| >>
| >>...
| >>... File-backed Storage Gadget.  
| >>It's a driver for a Linux-based USB _device_ (as opposed to _host_) that 
| >>emulates a USB Mass Storage device, using a file or block device as 
| >>backing storage in somewhat the same manner as the loop driver.
| >>... Unfortunately, ...
| > 
| > 
| > Wow, our lead FSG developer has no Linux USB _device_.
| 
| Well, he does have the "dummy_hcd" which lets him test
| on the same machine running the USB host ... I'll leave
| it to experts to contrast that with CONFIG_SCSI_DEBUG.
| 
| The driver was basically working before it hit real
| hardware.
| 
| 
| > Ouch.  Some sponsor with money should fix that.
| 
| Or Santa Claus ... ;)


or http://www.linux-usb.org/beanieForm.html

(sorry about my lateness, still catching up)
--
~Randy
MOTD:  Always include version info.

Re: usb storage traces

[Pat LaVarre]

> | >>>What is FSG? ... File-backed Storage Gadget.  
> | >>It's a driver for a Linux-based USB _device_ (as opposed to _host_) that 
> | >>emulates a USB Mass Storage device, using a file or block device as 
> | >>backing storage in somewhat the same manner as the loop driver.
> | >>... Unfortunately, ...
> | > Wow, our lead FSG developer has no Linux USB _device_.
> | 
> | Well, he does have the "dummy_hcd" ...
> | > Ouch.  Some sponsor with money should fix that.
> | Or Santa Claus ... ;)
> 
> or http://www.linux-usb.org/beanieForm.html

By now I have heard another sponsor with money already did fix this.

I appreciate that link all the same, yes new to me, now there I see:

---

Slashdot Beanie Award Application Form

This form should be used to apply for part of the Slashdot Beanie Award
that the Linux USB project was awarded. Please read the following
information before submitting the form (found at the end of this page).
About the Linux-USB Purchase Request Fund

The Linux-USB Purchase Request fund (PRQ) is a limited-duration cash
distribution fund established with part of the Award made at the "2000
Slashdot Beanies", where the Linux USB community won US$... for the Most
Improved Kernel Module - see
http://slashdot.org/articles/00/02/06/1950248.shtml.

The aim of the Linux-USB PRQ is to support the Open Source development
of Linux-USB. A secondary goal is to support related projects;
especially those that are likely to assist Linux-USB, such as other
hotplug support (devices or infrastructure).

The fund was started with US$... in Sept. 2000. No additional funds are
being sought, and the PRQ will be wound up when all funds are
distributed. 
...