Re: Inquiry data and emulated SG devices

Re: Inquiry data and emulated SG devices

[Robert Hancock]

(cc-ing linux-ide)

Mathieu Fluhr wrote:
> Hello all,
> 
> 
> First of all, let me introduce myself a little bit. I am the responsable
> for the development of the Nero Linux burning application. So I have
> access to all the source code of the application.
> 
> 
> Now let's go with the story: It seems that there is a slight problem in
> the libata (and also the new pata_xxx) interfaces with the data returned
> by the INQUIRY cmd since every S-ATA or IDE device is assumed to be a
> SCSI dev.
> 
> Normally, the IDE devices (physical type) can be differentied with the
> format field of the inquiry data, i.e. the last bytes (ANSI version) are
> set to 0 -> This is done and works great with USB external devices for
> example.
> 
> 
> The thing is that with S-ATA/IDE devices when using the libata/pata
> driver, the version field is (always?) set to 5, as any _real_ SCSI
> device, and thus the physical device type cannot be checked anymore.

This doesn't seem a very reliable way to identify an IDE device, as all
that 0 means is that the device does not claim conformance to any
standard. I would think it would be legitimate for an IDE device to put
a value like 5 in there as well, if it complies with SPC-4..

In the case of libata though, that appears to be due to this code in
drivers/ata/libata-scsi.c:

	/* ATAPI devices typically report zero for their SCSI version,
	 * and sometimes deviate from the spec WRT response data
	 * format.  If SCSI version is reported as zero like normal,
	 * then we make the following fixups:  1) Fake MMC-5 version,
	 * to indicate to the Linux scsi midlayer this is a modern
	 * device.  2) Ensure response data format / ATAPI information
	 * are always correct.
	 */
			if (buf[2] == 0) {
				buf[2] = 0x5;
				buf[3] = 0x32;
			}

This technically seems to go against what the SCSI/ATA Translation (SAT)
spec says regarding INQUIRY on ATAPI devices: "the SATL shall use the
ATA PACKET Command feature set to pass all INQUIRY commands and
parameter data to the ATAPI device without altering the INQUIRY
commands or the parameter data." However, it might realistically be
needed if the SCSI layer in the kernel has problems with a device
indicating it supports no SCSI version..

> 
> I cannot go so deep in details, but our burn engine is differentiating
> IDE and read-good-old-SCSI devices and handles them sometimes in a
> different way, so this differenciation is really important for us.
> 
>    -> How can I detect the real physical device type now?

I don't have a great answer off the top of my head..

-- 
Robert Hancock      Saskatoon, SK, Canada
To email, remove "nospam" from hancockr@nospamshaw.ca
Home Page: http://www.roberthancock.com/

Re: Inquiry data and emulated SG devices

[Jeff Garzik]

Robert Hancock wrote:
> This doesn't seem a very reliable way to identify an IDE device, as all
> that 0 means is that the device does not claim conformance to any
> standard. I would think it would be legitimate for an IDE device to put
> a value like 5 in there as well, if it complies with SPC-4..

Via the this-doesnt-really-matter-but-it-should-be-noted department:

According to the latest on t10.org, MMC retroactively permitted SCSI 
version to be zero, for MMC-compliant USB and ATAPI devices.


> In the case of libata though, that appears to be due to this code in
> drivers/ata/libata-scsi.c:
> 
>     /* ATAPI devices typically report zero for their SCSI version,
>      * and sometimes deviate from the spec WRT response data
>      * format.  If SCSI version is reported as zero like normal,
>      * then we make the following fixups:  1) Fake MMC-5 version,
>      * to indicate to the Linux scsi midlayer this is a modern
>      * device.  2) Ensure response data format / ATAPI information
>      * are always correct.
>      */
>             if (buf[2] == 0) {
>                 buf[2] = 0x5;
>                 buf[3] = 0x32;
>             }
> 
> This technically seems to go against what the SCSI/ATA Translation (SAT)
> spec says regarding INQUIRY on ATAPI devices: "the SATL shall use the
> ATA PACKET Command feature set to pass all INQUIRY commands and
> parameter data to the ATAPI device without altering the INQUIRY
> commands or the parameter data." However, it might realistically be
> needed if the SCSI layer in the kernel has problems with a device
> indicating it supports no SCSI version..

The above tweak is entirely software->software communication...  as the 
comment you quoted notes, it's just a signal to the SCSI midlayer.

At the moment, the SCSI midlayer assumes any device that reports scsi 
version as less than 2 is forced to SCSI version 2.  Ultimately that's 
incorrect behavior for all ATAPI devices (and later MMC revisions).

At the time, libata simply worked around this SCSI buglet in its own 
code, since that was easier than auditing all SCSI code paths to ensure 
new ATAPI/USB MMC logic does not break ancient devices.

But if someone is motivated enough to revisit this...

	Jeff

Re: Inquiry data and emulated SG devices

[Mathieu Fluhr]

On Wed, 2007-10-17 at 21:31 -0400, Jeff Garzik wrote:
> Robert Hancock wrote:
> > This doesn't seem a very reliable way to identify an IDE device, as all
> > that 0 means is that the device does not claim conformance to any
> > standard. I would think it would be legitimate for an IDE device to put
> > a value like 5 in there as well, if it complies with SPC-4..
> 
> Via the this-doesnt-really-matter-but-it-should-be-noted department:
> 
> According to the latest on t10.org, MMC retroactively permitted SCSI 
> version to be zero, for MMC-compliant USB and ATAPI devices.
> 

Quoting to the latest MtFuji draft (Section 17.7.1):
 "The ANSI Version field shall contain a non-zero value to comply with 
  this version of the Specification for a SCSI logical unit or zero for
  an ATAPI logical unit."


> > In the case of libata though, that appears to be due to this code in
> > drivers/ata/libata-scsi.c:
> > 
> >     /* ATAPI devices typically report zero for their SCSI version,
> >      * and sometimes deviate from the spec WRT response data
> >      * format.  If SCSI version is reported as zero like normal,
> >      * then we make the following fixups:  1) Fake MMC-5 version,
> >      * to indicate to the Linux scsi midlayer this is a modern
> >      * device.  2) Ensure response data format / ATAPI information
> >      * are always correct.
> >      */
> >             if (buf[2] == 0) {
> >                 buf[2] = 0x5;
> >                 buf[3] = 0x32;
> >             }
> > 

This explain a lot... But (Sorry I am not a scsi mid-layer expert) why
faking what the device outputs?

>From the application side, every device is then saw as real SCSI
devices. So from the developer point of view, this is really bad as
afterwards you have no way to optimize your code regarding the original
hardware architecture.


	Mathieu

Re: Inquiry data and emulated SG devices

[Jeff Garzik]

Mathieu Fluhr wrote:
> On Wed, 2007-10-17 at 21:31 -0400, Jeff Garzik wrote:
>> Robert Hancock wrote:
>>> This doesn't seem a very reliable way to identify an IDE device, as all
>>> that 0 means is that the device does not claim conformance to any
>>> standard. I would think it would be legitimate for an IDE device to put
>>> a value like 5 in there as well, if it complies with SPC-4..
>> Via the this-doesnt-really-matter-but-it-should-be-noted department:
>>
>> According to the latest on t10.org, MMC retroactively permitted SCSI 
>> version to be zero, for MMC-compliant USB and ATAPI devices.
>>
> 
> Quoting to the latest MtFuji draft (Section 17.7.1):
>  "The ANSI Version field shall contain a non-zero value to comply with 
>   this version of the Specification for a SCSI logical unit or zero for
>   an ATAPI logical unit."
> 
> 
>>> In the case of libata though, that appears to be due to this code in
>>> drivers/ata/libata-scsi.c:
>>>
>>>     /* ATAPI devices typically report zero for their SCSI version,
>>>      * and sometimes deviate from the spec WRT response data
>>>      * format.  If SCSI version is reported as zero like normal,
>>>      * then we make the following fixups:  1) Fake MMC-5 version,
>>>      * to indicate to the Linux scsi midlayer this is a modern
>>>      * device.  2) Ensure response data format / ATAPI information
>>>      * are always correct.
>>>      */
>>>             if (buf[2] == 0) {
>>>                 buf[2] = 0x5;
>>>                 buf[3] = 0x32;
>>>             }
>>>
> 
> This explain a lot... But (Sorry I am not a scsi mid-layer expert) why
> faking what the device outputs?

The SCSI midlayer makes a lot of "if scsi version <= 2" choices.  In the 
case of ATAPI, we do not want to force ATAPI down the path of ancient 
SCSI devices, as this disables some MMC features that modern ATAPI 
devices support.

	Jeff

Re: Inquiry data and emulated SG devices

[Mathieu Fluhr]

On Thu, 2007-10-18 at 06:06 -0400, Jeff Garzik wrote:

> The SCSI midlayer makes a lot of "if scsi version <= 2" choices.  In the 
> case of ATAPI, we do not want to force ATAPI down the path of ancient 
> SCSI devices, as this disables some MMC features that modern ATAPI 
> devices support.

If I fully understand your point, if this faking code wasn't present,
and if the inquiry data was left as it is, all modern ATAPI devices
would be considered as really old SCSI devices. Am I right?

And then why external (FireWire and USB) devices reports the inquiry
data correctly? After all they are also considered to be SCSI devices...
 -> But apparently, from what /proc/scsi/scsi outputs, the ANSI SCSI   
    revision is set to 0 (for both ieee1394 and USB devices). This 
    seems to be coherent with what the inquiry data buffer outputs.


Also another question, more from the developer point of view, you have
to know which real interface is behind a device. You might get
performance loss if the real type is not known.

For example, I send a BLANK cdb to fully blank a disc, with the 'immed'
flag not set, so that the requested operation is processed to completion
prior to returning status.
- for real SCSI devices, this is not harmfull, as the bus is not blocked
- for IDE devices, AFAIK, the bus will be blocked until the command
termination... so for a CD erased a 1x, the bus might be blocked for
74min.


Also the SCSI commands that are sent to the device depends from its
hardware bus type. Not only for the CDB length, but also for MODE
SENSE/MODE SELECT CDBs in which for example a MODE SENSE (6) would fail
on an IDE device... even if it is described in the SPC-3 standard.

As far as I saw in libata-scsi.c there a SCSI-to-ATAPI and ATAPI-to-SCSI
translator that automatically transform the command sent... Am I also
right on this point?
    
	Mathieu

Re: Inquiry data and emulated SG devices

[James Bottomley]

On Thu, 2007-10-18 at 13:57 +0200, Mathieu Fluhr wrote:
> On Thu, 2007-10-18 at 06:06 -0400, Jeff Garzik wrote:
> 
> > The SCSI midlayer makes a lot of "if scsi version <= 2" choices.  In the 
> > case of ATAPI, we do not want to force ATAPI down the path of ancient 
> > SCSI devices, as this disables some MMC features that modern ATAPI 
> > devices support.
> 
> If I fully understand your point, if this faking code wasn't present,
> and if the inquiry data was left as it is, all modern ATAPI devices
> would be considered as really old SCSI devices. Am I right?

Not really.  Firstly, a lot of ATAPI devices report the correct
compliance level and secondly, the consequence of reporting no
compliance (level 0) is that the mid layer will be very careful to use
the most minimal command set it can (the usual consequence of sending a
USB device a command it doesn't understand is to have it crash).

However, this will have no effect for at least CDs:  The sr driver is
only interested in the MMC compliance level not the SCSI compliance
level.

> And then why external (FireWire and USB) devices reports the inquiry
> data correctly? After all they are also considered to be SCSI devices...
>  -> But apparently, from what /proc/scsi/scsi outputs, the ANSI SCSI   
>     revision is set to 0 (for both ieee1394 and USB devices). This 
>     seems to be coherent with what the inquiry data buffer outputs.
> 
> 
> Also another question, more from the developer point of view, you have
> to know which real interface is behind a device. You might get
> performance loss if the real type is not known.
> 
> For example, I send a BLANK cdb to fully blank a disc, with the 'immed'
> flag not set, so that the requested operation is processed to completion
> prior to returning status.
> - for real SCSI devices, this is not harmfull, as the bus is not blocked
> - for IDE devices, AFAIK, the bus will be blocked until the command
> termination... so for a CD erased a 1x, the bus might be blocked for
> 74min.
> 
> 
> Also the SCSI commands that are sent to the device depends from its
> hardware bus type. Not only for the CDB length, but also for MODE
> SENSE/MODE SELECT CDBs in which for example a MODE SENSE (6) would fail
> on an IDE device... even if it is described in the SPC-3 standard.

Er, I think you might have a slight misunderstanding of what the mid
layer does with the scsi level.  It doesn't police incoming commands
that are user generated (the user is free to crash the device or set
their house on fire) according to the compliance level.  All it does is
try to make sure that the mid-layer transactions (which are pretty much
inquiry, mode sense and test unit ready) are all as minimal as possible.

> As far as I saw in libata-scsi.c there a SCSI-to-ATAPI and ATAPI-to-SCSI
> translator that automatically transform the command sent... Am I also
> right on this point?

The former only, I believe.

James

Re: Inquiry data and emulated SG devices

[Mathieu Fluhr]

On Thu, 2007-10-18 at 08:13 -0400, James Bottomley wrote:
> On Thu, 2007-10-18 at 13:57 +0200, Mathieu Fluhr wrote:
> > On Thu, 2007-10-18 at 06:06 -0400, Jeff Garzik wrote:
> > 
> > > The SCSI midlayer makes a lot of "if scsi version <= 2" choices.  In the 
> > > case of ATAPI, we do not want to force ATAPI down the path of ancient 
> > > SCSI devices, as this disables some MMC features that modern ATAPI 
> > > devices support.
> > 
> > If I fully understand your point, if this faking code wasn't present,
> > and if the inquiry data was left as it is, all modern ATAPI devices
> > would be considered as really old SCSI devices. Am I right?
> 
> Not really.  Firstly, a lot of ATAPI devices report the correct
> compliance level and secondly, the consequence of reporting no
> compliance (level 0) is that the mid layer will be very careful to use
> the most minimal command set it can (the usual consequence of sending a
> USB device a command it doesn't understand is to have it crash).
> 

Ok, then, what about what the standard is saying regarding the ANSI
version field? Quoting to the latest MtFuji draft (Section 17.7.1):
 "The ANSI Version field shall contain a non-zero value to comply with 
  this version of the Specification for a SCSI logical unit or zero for
  an ATAPI logical unit."

and this is exactly how we see that the logical unit is a real SCSI one
or an ATAPI one.

> However, this will have no effect for at least CDs:  The sr driver is
> only interested in the MMC compliance level not the SCSI compliance
> level.

We are not using the sr driver to perform our SCSI commands, as it is
filtering which CDB is allowed and which isn't. As we have some
vendor-specific SCSI commands (like the one for changing booktype), we
are using the sg driver to be able to use them.



> > As far as I saw in libata-scsi.c there a SCSI-to-ATAPI and ATAPI-to-SCSI
> > translator that automatically transform the command sent... Am I also
> > right on this point?
> 
> The former only, I believe.

Absolutly... but if the inquiry data buffer gets modified by the
midlayer, after that I am not able anymore to setup correctly the
commands for the _real_ interface. (once again, the blank command
example).

Mathieu

Re: Inquiry data and emulated SG devices

[James Bottomley]

On Wed, 2007-10-17 at 21:31 -0400, Jeff Garzik wrote:
> Robert Hancock wrote:
> > This doesn't seem a very reliable way to identify an IDE device, as all
> > that 0 means is that the device does not claim conformance to any
> > standard. I would think it would be legitimate for an IDE device to put
> > a value like 5 in there as well, if it complies with SPC-4..
> 
> Via the this-doesnt-really-matter-but-it-should-be-noted department:
> 
> According to the latest on t10.org, MMC retroactively permitted SCSI 
> version to be zero, for MMC-compliant USB and ATAPI devices.
> 
> 
> > In the case of libata though, that appears to be due to this code in
> > drivers/ata/libata-scsi.c:
> > 
> >     /* ATAPI devices typically report zero for their SCSI version,
> >      * and sometimes deviate from the spec WRT response data
> >      * format.  If SCSI version is reported as zero like normal,
> >      * then we make the following fixups:  1) Fake MMC-5 version,
> >      * to indicate to the Linux scsi midlayer this is a modern
> >      * device.  2) Ensure response data format / ATAPI information
> >      * are always correct.
> >      */
> >             if (buf[2] == 0) {
> >                 buf[2] = 0x5;
> >                 buf[3] = 0x32;
> >             }
> > 
> > This technically seems to go against what the SCSI/ATA Translation (SAT)
> > spec says regarding INQUIRY on ATAPI devices: "the SATL shall use the
> > ATA PACKET Command feature set to pass all INQUIRY commands and
> > parameter data to the ATAPI device without altering the INQUIRY
> > commands or the parameter data." However, it might realistically be
> > needed if the SCSI layer in the kernel has problems with a device
> > indicating it supports no SCSI version..
> 
> The above tweak is entirely software->software communication...  as the 
> comment you quoted notes, it's just a signal to the SCSI midlayer.
> 
> At the moment, the SCSI midlayer assumes any device that reports scsi 
> version as less than 2 is forced to SCSI version 2.  Ultimately that's 
> incorrect behavior for all ATAPI devices (and later MMC revisions).

Actually, no we don't.  SCSI level 0 means "no compliant standard
specified".  We're quite careful if we see this not to do anything that
might upset the device ... SCSI level 0 is a fairly usual thing to see
on USB devices, which is why we process it differently.

> At the time, libata simply worked around this SCSI buglet in its own 
> code, since that was easier than auditing all SCSI code paths to ensure 
> new ATAPI/USB MMC logic does not break ancient devices.
> 
> But if someone is motivated enough to revisit this...

Like I said, it should all be fixed ... if you try it.

James

Inquiry data and emulated SG devices

[Mathieu Fluhr]

Hello all,


First of all, let me introduce myself a little bit. I am the responsable
for the development of the Nero Linux burning application. So I have
access to all the source code of the application.


Now let's go with the story: It seems that there is a slight problem in
the libata (and also the new pata_xxx) interfaces with the data returned
by the INQUIRY cmd since every S-ATA or IDE device is assumed to be a
SCSI dev.

Normally, the IDE devices (physical type) can be differentied with the
format field of the inquiry data, i.e. the last bytes (ANSI version) are
set to 0 -> This is done and works great with USB external devices for
example.


The thing is that with S-ATA/IDE devices when using the libata/pata
driver, the version field is (always?) set to 5, as any _real_ SCSI
device, and thus the physical device type cannot be checked anymore.

I cannot go so deep in details, but our burn engine is differentiating
IDE and read-good-old-SCSI devices and handles them sometimes in a
different way, so this differenciation is really important for us.

   -> How can I detect the real physical device type now?



Best Regards,
Mathieu


-- 
*********************************************************
Mathieu Fluhr
Linux Development Team Leader

E-mail: mfluhr@nero.com


NERO - BECAUSE TECHNOLOGY COUNTS
www.nero.com

*********************************************************
Nero AG
Im Stoeckmaedle 13-15
76307 Karlsbad
Germany

Vorstand/CEO: Richard Lesser
Aufsichtsratvorsitzender/chairman of the supervisory board: Jim Corbett

Amtsgericht Mannheim HRB 362519
*********************************************************
This e-mail may contain confidential and/or
privileged information. If you are not the intended
recipient (or have received this e-mail in error)
please notify the sender immediately and destroy
this e-mail. Any unauthorised copying, disclosure
or distribution of the material in this e-mail is
strictly forbidden.
*********************************************************