RE: [usb-storage] [Merging ATA passthru] on integratingSMART/ATA-Security in usb-storage driver

RE: [usb-storage] [Merging ATA passthru] on integratingSMART/ATA-Security in usb-storage driver

[Timothy Thelin]

> 
> On Fri, 2005-11-04 at 10:30 -0800, Timothy Thelin wrote:
> > And for an even more concrete example:
> > The CY7C68300B cypress bridge board (has various siblings as well
> > on their site that act very similar) implements SCSI spec 0 (ie it
> > doesn't claim to support any scsi spec).  Now usb-storage sees
> > this in inquiry, and decides to export the device as a scsi2 device
> > since based on the usb-storage devs' experience most usb devices
> > really want scsi2 cdbs.  So SCSI core thinks this is a scsi2 device
> > and procedes to mangle the cdbs as they're going through.
> 
> What happens if you prevent USB mangling the scsi_level?  I think, for
> the most part, we would handle 0 in about the same way as we handle 2.
> However, we could gate the if around the CDB[1] mangling as
> 
> if (scsi_level != SCSI_UNKNOWN && scsi_level <= SCSI_2)
> 
> which should fix your problem, I think.
> 
> James
> 

Irconically that was one of the solutions I was investigating
first.  Here is the usb mailing list discussion on it:
http://marc.theaimsgroup.com/?t=112672009700004&r=1&w=2

In summary, the idea of making the scsi stack "level 0 aware" was
suggested and rejected because usb devs want to keep promoting
the devices to level 2 for various reaons.  At the end of it
they suggested using SG_FLAG_LUN_INHIBIT so that the usb device
could stay level 2 yet vendor specific commands could get passed
through un-touched. Neither of us knew that flag didn't work
anymore, and after I wasn't able to get it to work, that
other conversation I posted previously occured to figure out
what was going on with SG_FLAG_LUN_INHIBIT.

Regards,
Tim Thelin

RE: [usb-storage] [Merging ATA passthru] on integratingSMART/ATA-Security in usb-storage driver

[James Bottomley]

On Fri, 2005-11-04 at 11:59 -0800, Timothy Thelin wrote:
> Irconically that was one of the solutions I was investigating
> first.  Here is the usb mailing list discussion on it:
> http://marc.theaimsgroup.com/?t=112672009700004&r=1&w=2

OK, I think the assertion that just passing through the level as scsi0
"won't work" is wrong.  Most of our checks for SCSI-2 are 

if (xxx->scsi_level <= SCSI_2)

which catches scsi0 as well as scsi2.  The length of commands is
actually separately settable in slave_configure, but by default we use
10 byte commands even for scsi2.

> In summary, the idea of making the scsi stack "level 0 aware" was
> suggested and rejected because usb devs want to keep promoting
> the devices to level 2 for various reaons.  At the end of it
> they suggested using SG_FLAG_LUN_INHIBIT so that the usb device
> could stay level 2 yet vendor specific commands could get passed
> through un-touched. Neither of us knew that flag didn't work
> anymore, and after I wasn't able to get it to work, that
> other conversation I posted previously occured to figure out
> what was going on with SG_FLAG_LUN_INHIBIT.

Well ... before embarking on a massive programme to put this flag back
(or implement a "I say SCSI2 but I'm really not" blacklist flag), I'd
like proof that just passing SCSI_UNKNOWN as the level really won't work
(and is unfixable).  I think it will turn out to be the simplest course
of action.

James

RE: [usb-storage] [Merging ATA passthru] on integratingSMART/ATA-Security in usb-storage driver

[Timothy Thelin]

> On Mon, 2005-11-07 at 15:59 -0500, Alan Stern wrote:
> > > That is, is usb-storage forcing scsi-2 when the device 
> tells us it is
> > > scsi-3 compliant, or is the hardware reporting devices 
> are scsi-2, yet
> > > requiring non-LUN value in cdb[1]?
> > 
> > I think we may have both.  However I don't know how this 
> Cypress chip
> > reports itself.  A system log showing the INQUIRY data would be very
> > helpful.
> 
> We were told in prior emails that it actually reports a level of zero
> (i.e. no compliance with any SCSI standard).  My original proposal was
> just not to modify the CDB[1] for this case if we could get 
> the INQUIRY
> passed through unmangled.

Here is the inquiry data on my CY7C68300B:
00-03: 00 00 00 00
04-07: 1F 00 00 00
08-11: 57 44 43 20
12-15: 57 44 34 30
16-19: 30 55 45 2D
20-23: 30 30 48 43
24-27: 54 30 20 20
28-31: 20 20 20 20
32-35: 30 30 30 30
36-39: 00 00 00 00
40-43: 00 00 00 00

Regards,
Tim Thelin

Re: [usb-storage] [Merging ATA passthru] on integrating SMART/ATA-Security in usb-storage driver

[Patrick Mansfield]

On Mon, Nov 07, 2005 at 06:54:58PM +0100, thomas schorpp wrote:
> Bus 004 Device 003: ID 04b4:6830 Cypress Semiconductor Corp. USB-2.0 IDE Adapter
> Device Descriptor:
>   bLength                18
>   bDescriptorType         1
>   bcdUSB               2.00
>   bDeviceClass            0 (Defined at Interface level)
>   bDeviceSubClass         0
>   bDeviceProtocol         0
>   bMaxPacketSize0        64
>   idVendor           0x04b4 Cypress Semiconductor Corp.
>   idProduct          0x6830 USB-2.0 IDE Adapter

So this is a USB to ATA/ATAPI bridge?

Can we trigger black listing based on the above vendor/product values?
i.e. can you check for these values in usb-storage slave_configure code?

I am ATA ignorant ... how does this thing pass multi-lun values to the ATA
side? Is it just effectively not allowed in ATA?

Per earlier email, datasheet is here:

http://www.cypress.com/portal/server.pt?space=CommunityPage&control=SetCommunity&CommunityID=209&PageID=259&fid=14&rpn=CY7C68320

On page 13 of datasheet:

Sounds like this device has special snooping of cdb[1], so if it sees 0x24
in there it looks for an embedded ATA command, and translates and sends
that to the underlying device, so we can never black list based on the
device that we are using to send the SG_IO ioctl.

Yeh, some other hardware might want lun inihibit in the future, and that
could still be added back, but IMHO black list is better.

-- Patrick Mansfield

Re: [usb-storage] [Merging ATA passthru] on integrating SMART/ATA-Security in usb-storage driver

[Patrick Mansfield]

On Mon, Nov 07, 2005 at 02:53:34PM -0500, Alan Stern wrote:
> On Mon, 7 Nov 2005, Patrick Mansfield wrote:
> > Can we trigger black listing based on the above vendor/product values?
> > i.e. can you check for these values in usb-storage slave_configure code?
> 
> Yes.  The patch below illustrates how.  You'll have to change the name of 
> the sdev flag to something sensible...
> 
> I firmly believe this is the wrong approach, however.  It's a specific 
> solution to a general problem.  I would much prefer to add a new flag to 
> struct request.

We have the specific problem of a device being reported as scsi-2
compliant when it is not.

Stepping back a bit ...

usb-storage should pass through the scsi level, we should not require
special handling (adding back inhibit lun support or a new black list
option) for compliant hardware.

So, do we have a usb-storage problem or non-compliant hardware here? I
did not find the answer in previous emails.

That is, is usb-storage forcing scsi-2 when the device tells us it is
scsi-3 compliant, or is the hardware reporting devices are scsi-2, yet
requiring non-LUN value in cdb[1]?

A method that can reliably and easily set some black list value for the
hardware in question will to lead to fewer questions in the future as to
why an SG_IO command is getting LUN values set in cdb[1].

If this is a hardware problem I'm not opposed to adding back support for
the ioctl and AFAIUI a new struct request flag.

-- Patrick Mansfield

Re: [usb-storage] [Merging ATA passthru] on integratingSMART/ATA-Security in usb-storage driver

[Pat LaVarre]

> ... is usb-storage forcing scsi-2 when the device tells us it is
> scsi-3 compliant, or is the hardware reporting devices are scsi-2, yet
> requiring non-LUN value in cdb[1]?

The very newest devices that pass info thru mask xE0 of cdb[1] may be  
reporting SCSI zero, especially when the cable is not SPI &  
especially when Peripheral Device Type (mask x1F of byte 0 of Inquiry  
data) is nonzero.

> We were told in prior emails that it actually reports a level of zero
> (i.e. no compliance with any SCSI standard)

People dispute what zero means in byte 2 of SCSI Inquiry data.  Here  
are five widely disseminated opinions:

a) T10 s1-r17b: "Version is unspecified.  (Use this code if you are  
implementing to this document - it is not published, yet.)"

b) T10 s1-r17b: "does not claim compliance to the ISO or ECMA  
versions of SCSI".

c) T10 s2-r10l.pdf: "might or might not comply to an ANSI-approved  
standard"

d) T10 s2-r10l.pdf: "does not claim compliance to the ISO or ECMA  
versions of SCSI".

e) SFF sff8020i.pdf: [mask x07] "must contain a zero to comply with  
this version of the Specification"

See that?  Often, the very same document disagrees with itself.

Nowadays, the SFF heresy has grown to where T10 simultaneous  
publishes two or more distinct definitions of Inquiry - the SPC  
definition and the MMC definition.

SPC Inquiry data bytes 58 and following now have "version descriptor"  
definitions that let a device that zeroes byte 2 say more, in  
theory.  I haven't actually seen anyone populate those fields in a  
mass market product?

> ...

I'd think that everyone should accept that trying to reclaim mask xFF  
in the last CDB byte and mask xE0 of CDB byte 1 is rude ... I'm not  
sure what rationally motivates those movements in device design that  
demand greater transparency of a pass thru SCSI host.

RE: [usb-storage] [Merging ATA passthru] on integratingSMART/ATA-Security in usb-storage driver

[Timothy Thelin]

> 
> Well, that might be a problem if it weren't for the fact that this
> LUN_INHIBIT flag was removed in 2002.  If it's taken three 
> years to find
> a device that has a problem with it, I don't really think it's a
> particularly widespread problem.  
>
> ...snip....
>
> James
> 

Just as an FYI to shed some light....

It took Western Digital 3 years to find the problem because September
(back when I started the other conversation about this topic) was the
first time we've tried to get access to a USB storage device under
Linux using a passthru CDB.  Maxtor and Seagate might be in a similar
boat (but obviously that's just speculation), and it's not like thumb
drives need ATA passthru support.

This also isn't the first time a kernel issue has been around for a
while, but nobody noticed. Fixed in 2.6.14 was a bug that existed for
(as far as I can tell) all of 2.6.x.  The kernel would oops on pio-out
HDIO_DRIVE_TASKFILE ioctls because of a null pointer.  The first
time I tried to do a pio-out command I hit it, yet nobody had
complained about it as far as I could tell.  The kernel 2.6.x series
has been out how long?  I was baffled and my only guess was that
even though the API was there for pio-out, nobody was using it.

I wonder if it's the same issue with vendor specific cdbs to USB
storage devices; nobody is really using it?

The cypress passthru CDB works fine under Windows, and since
most companies only care about Windows, they probably didn't
even try to get it to work under Linux.  For instance I wasn't
even told to support it under Linux until some people hit a 
Windows limitation they couldn't get around.

The other possibility is that for groups that do use vendor specific
CDBs and have never had this issue, they probably have CDBs that don't
stick meaningful bits where the LUN goes, so therefore it was never an
issue.  However, both the ATACB and the SAT passthru CDB have
meaningful bits there.


Regards,
Tim Thelin

RE: [usb-storage] [Merging ATA passthru] on integrating SMART/ATA-Security in usb-storage driver

[Timothy Thelin]

> 
> On Thu, Nov 03, 2005 at 11:08:07PM -0500, James Bottomley wrote:
> > On Wed, 2005-11-02 at 15:45 -0800, Matthew Dharm wrote:
> > > On Wed, Nov 02, 2005 at 02:18:52PM -0800, Timothy Thelin wrote:
> > > > 
> > > > If you had time to spare, instead of touching usb-storage,
> > > > it might be better spent resurecting SG_FLAG_LUN_INHIBIT to
> > > > stop the above behavior so that SG_IO cdbs can be passed
> > > > through untouched.
> > > > (SG_FLAG_FUN_INHIBIT was a flag SG_IO used to support a long
> > > > time ago, and I have no idea why it was dropped, but it was)
> > > 
> > > I didn't realize that had been removed.  Anyone that sends a
> > > vendor-specific command to a device needs this flag to 
> make sure it goes
> > > through unmangled.
> > > 
> > > Perhaps someone on linux-scsi can comment on why this was 
> removed and how
> > > we might get it back?
> > 
> > I've no distinct recollection of someone removing this, but if I
> > remember correctly what it used to do, it was a hack to stop us from
> > mangling SCSI-3 CDB's.  We fixed the mid-layer not to 
> require the hack
> > by only setting the CDB[1] lun field for SCSI-1 and SCSI-2 
> devices (as
> > the standards mandate).  What's the actual problem?  No 
> SCSI-1 or SCSI-2
> > device should have any vendor specific CDBs that uses these bits in
> > CDB[1].
> 
> Unfortunately, reality appears to disagree with the last 
> "should".  I've
> personally seen devices with vendor-specific commands that 
> want to control
> CDB[1] in SCSI-2.
> 
> I didn't know it was removed; I only know what Timothy Thelin 
> told me.  Can
> we get the feature back?
> 
> Matt

And for an even more concrete example:
The CY7C68300B cypress bridge board (has various siblings as well
on their site that act very similar) implements SCSI spec 0 (ie it
doesn't claim to support any scsi spec).  Now usb-storage sees
this in inquiry, and decides to export the device as a scsi2 device
since based on the usb-storage devs' experience most usb devices
really want scsi2 cdbs.  So SCSI core thinks this is a scsi2 device
and procedes to mangle the cdbs as they're going through.

The problem then is with vendor specific commands as SCSI core
mangles those as well.  The vendor specific command I had issues
with is the "ATACB" ATA passthru cdb that cypress boards understand.
CDB[1] is a signature byte that must be 0x24.  Needless to say,
that leading 2 gets masked off to a 0, and the drive aborts the
ATACB.  Now cypress could have been smarter about designing their
cdb, but they wern't, and so there really needs to be a way to tell
SCSI core "hands off the cdb".

This is also going to be an issue with USB devices that implement
SAT passthru but state they implement SCSI spec 0, as SAT passthru
CDB[1] can't afford to be mangled either.

I started this conversation a little over a month ago, but it died.
http://marc.theaimsgroup.com/?l=linux-scsi&m=112714917116713&w=2

Regards,
Tim Thelin

Re: [usb-storage] [Merging ATA passthru] on integrating SMART/ATA-Security in usb-storage driver

[Pete Zaitcev]

On Fri, 4 Nov 2005 10:30:44 -0800, "Timothy Thelin" <Timothy.Thelin@wdc.com> wrote:

> The problem then is with vendor specific commands as SCSI core
> mangles those as well.  The vendor specific command I had issues
> with is the "ATACB" ATA passthru cdb that cypress boards understand.
> CDB[1] is a signature byte that must be 0x24.  Needless to say,
> that leading 2 gets masked off to a 0, and the drive aborts the
> ATACB.  Now cypress could have been smarter about designing their
> cdb, but they wern't, and so there really needs to be a way to tell
> SCSI core "hands off the cdb".

If this issue is real, you can use ub with that device for now.

-- Pete

Re: [usb-storage] [Merging ATA passthru] on integrating SMART/ATA-Security in usb-storage driver

[thomas schorpp]

Pete Zaitcev wrote:
> On Fri, 4 Nov 2005 10:30:44 -0800, "Timothy Thelin" <Timothy.Thelin@wdc.com> wrote:
> 
> 
>>The problem then is with vendor specific commands as SCSI core
>>mangles those as well.  The vendor specific command I had issues
>>with is the "ATACB" ATA passthru cdb that cypress boards understand.
>>CDB[1] is a signature byte that must be 0x24.  Needless to say,
>>that leading 2 gets masked off to a 0, and the drive aborts the
>>ATACB.  Now cypress could have been smarter about designing their
>>cdb, but they wern't, and so there really needs to be a way to tell
>>SCSI core "hands off the cdb".
> 
> 
> If this issue is real, you can use ub with that device for now.
> 
> -- Pete
> -

hello Pete

nice idea :)

unfortunately it doesnt work with latest smartctl cvs on my usb2 drive, no smart functionality
tom1:/usr/src/sm5# ./smartctl -a -d scsi /dev/uba
smartctl version 5.34 [i686-pc-linux-gnu] Copyright (C) 2002-5 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

Device: IC35L030 AVVN07-0         Version:
Device type: disk
Local Time is: Sat Nov  5 16:49:28 2005 CET
Device does not support SMART

Error Counter logging not supported
Device does not support Self Test logging
tom1:/usr/src/sm5#

hdparm cant work(?). not implemented.

tom1:/usr/src/sm5# sdparm -alv /dev/uba
    /dev/uba: IC35L030  AVVN07-0
Read write error recovery [PS=0] mode page:
  AWRE        0  [cha: n, def:  0, sav:  0]  Automatic write reallocation enabled
  ARRE        0  [cha: n, def:  0, sav:  0]  Automatic read reallocation enabled
  TB          0  [cha: n, def:  0, sav:  0]  Transfer block
  RC          0  [cha: n, def:  0, sav:  0]  Read continuous
  EER         0  [cha: n, def:  0, sav:  0]  Enable early recover
  PER         0  [cha: n, def:  0, sav:  0]  Post error
  DTE         0  [cha: n, def:  0, sav:  0]  Data terminate on error
  DCR         0  [cha: n, def:  0, sav:  0]  Disable correction
  RRC         0  [cha: n, def:  0, sav:  0]  Read retry count
  WRC         0  [cha: n, def:  0, sav:  0]  Write retry count
  RTL         0  [cha: n, def:  0, sav:  0]  Recovery time limit (ms)
>> Format (SBC) mode page not supported
>> Rigid disk (SBC) mode page not supported
>> Verify error recovery (SBC) mode page not supported
Caching (SBC) [PS=0] mode page:
  IC          0  [cha: n, def:  0, sav:  0]  Initiator control
  ABPF        0  [cha: n, def:  0, sav:  0]  Abort pre-fetch
  CAP         0  [cha: n, def:  0, sav:  0]  Caching analysis permitted
  DISC        0  [cha: n, def:  0, sav:  0]  Discontinuity
  SIZE        0  [cha: n, def:  0, sav:  0]  Size (1->CSS valid, 0->NCS valid)
  WCE         0  [cha: n, def:  0, sav:  0]  Write cache enable
  MF          0  [cha: n, def:  0, sav:  0]  Multiplication factor
  RCD         0  [cha: n, def:  0, sav:  0]  Read cache disable
  DRRP        0  [cha: n, def:  0, sav:  0]  Demand read retension priority
  WRP         0  [cha: n, def:  0, sav:  0]  Write retension priority
  DPTL        0  [cha: n, def:  0, sav:  0]  Disable pre-fetch transfer length
  MIPF        0  [cha: n, def:  0, sav:  0]  Minimum pre-fetch
  MAPF        0  [cha: n, def:  0, sav:  0]  Maximum pre-fetch
  MAPFC       0  [cha: n, def:  0, sav:  0]  Maximum pre-fetch ceiling
 >> skipping rest as field position exceeds mode page length=12
>> Control mode page not supported
>> XOR control (SBC) mode page not supported
>> Protocol specific logical unit mode page not supported
>> Protocol specific port mode page not supported
>> Power condition mode page not supported
>> Informational exceptions control mode page not supported
>> Background control (SBC) mode subpage not supported
tom1:/usr/src/sm5#

device is in locked state, ata-security.

hmm, could someone supply me a sdparm commandline and data to test unlock (if supported), pls ?
im not through with all the specs from T10.org yet and they want money for more docs.

thx,
tom

Re: [usb-storage] [Merging ATA passthru] on integratingSMART/ATA-Security in usb-storage driver

[Pat LaVarre]

>>> The problem then is with vendor specific commands as SCSI core
>>> mangles those as well.  The vendor specific command I had issues
>>> with is the "ATACB" ATA passthru cdb that cypress boards understand.
>>> CDB[1] is a signature byte that must be 0x24.  Needless to say,
>>> that leading 2 gets masked off to a 0, and the drive aborts the
>>> ATACB.  Now cypress could have been smarter about designing their
>>> cdb, but they wern't, and so there really needs to be a way to tell
>>> SCSI core "hands off the cdb".

I may be remembering this "could have been smarter" choice was  
conscious.

This design work was done in Boise by ISD, later acquired by the  
Cypress San Diego people?  I didn't work for them, but I knew some of  
the time.  At the time people didn't want opaque pass thru to work,  
only fully transparent pass thru.  People looked around for some bits  
that most fragile-because-opaque pass thru schemes stomp on, and  
found mask xE0 in CDB byte 1.

The push against making ATA pass thru too easy came from the debates  
over whether to encode storage requests as ATA or SCSI or USB or a  
mix or all three.  At the time we were advocating SCSI & deprecating  
ATA, because ATA thru USB had reached the market first without  
allowing for RMB ATAPI thru USB.  The revolutionary purely USB  
encoding that a solution like ub.ko allows had no traction at all.

Another analogous theoretical possibility we deprecated at the same  
time was devices that simultaneously accepted both CBI & BBB  
encodings of SCSI.  Again the issue was the hosts that might have  
neglected to rapidly adopt SCSI thru BBB, and thus implicitly RMB  
support, if ATA thru CB had also been a mass-produced option.

I can't remember if the '\x24' = '$' = U.S. dollar sign pun was  
significant.

Certainly SCSI thru BBB worked well enough that the efforts to step  
further forward lost impetus.  The fact that ATA, while lacking RMB  
support, includes encodings for purportedly useful requests that SCSI  
lacks, such as SMART, gathered no significant attention.