READ SCSI cmd seems to fail on SATA optical devices...

READ SCSI cmd seems to fail on SATA optical devices...

[Mathieu Fluhr]

Hello,

I recently tried to burn some datas on CDs and DVD using a SATA burner
and the latest 2.6.18.2 kernel... using NeroLINUX. (It is controlling
the device by sending SCSI commands over the 'sg' driver)

The burn process works like a charm, no problems at all. But it seems
that there are some slight problems with the READ scsi cmd:
Inside our software, we have a verification routine that will make a
sector-by-sector verification to check that everything that has been
written is OK.

The problem is that, on SATA devices controlled by libata, on some big
files (like for example a 600 MB file) the READ command seems to fail
and outputs garbage (not 1 or 2 bytes diff, but the whole buffer).
 -> This problem does not come out everytime, and each time on    
    different sectors.

Please note that:
- it is not chipset dependant (tested on nforce4 and sii3114)
- it is not medium or device dependant


I debugged a little bit and found out the following:

- Our software tries to guess the bus type for each device by using
INQUIRY SCSI cmds. For SATA devices, it always returns SCSI. 
(which according to the source code is normal -> libata-scsi.c:2396)

- When I force the bus type to be IDE, our software will then send ATA
commands. In this case, everything works like a charm. No errors at all.
 

I tried to debug a little bit more inside the source code of the Linux
kernel, but I must admit that I am really not so familiar with the
code... but it would be no problem to help anyone on this issue. :)

Best Regards,
Mathieu


-- 
*********************************************************
Mathieu Fluhr
Linux Software Engineer

Nero AG
Im Stoeckmaedle 18
76307 Karlsbad
Germany

E-mail: mfluhr@nero.com

NERO - BECAUSE TECHNOLOGY COUNTS
www.nero.com

*********************************************************
This e-mail may contain confidential and/or
privileged information. If you are not the intended
recipient (or have received this e-mail in error)
please notify the sender immediately and destroy
this e-mail. Any unauthorised copying, disclosure
or distribution of the material in this e-mail is
strictly forbidden.
********************************************************* 

Re: READ SCSI cmd seems to fail on SATA optical devices...

[Phillip Susi]

Mathieu Fluhr wrote:
> Hello,
> 
> I recently tried to burn some datas on CDs and DVD using a SATA burner
> and the latest 2.6.18.2 kernel... using NeroLINUX. (It is controlling
> the device by sending SCSI commands over the 'sg' driver)
> 

Please note that the sg interface is depreciated.  It is now recommended 
that you send the CCBs directly to the normal device, i.e. /dev/hdc.

> The burn process works like a charm, no problems at all. But it seems
> that there are some slight problems with the READ scsi cmd:
> Inside our software, we have a verification routine that will make a
> sector-by-sector verification to check that everything that has been
> written is OK.
> 
> The problem is that, on SATA devices controlled by libata, on some big
> files (like for example a 600 MB file) the READ command seems to fail
> and outputs garbage (not 1 or 2 bytes diff, but the whole buffer).
>  -> This problem does not come out everytime, and each time on    
>     different sectors.
> 

I'm not sure what you mean by "on some big files" as the sg interface 
has no notion of files; you're just reading the disk sector by sector, 
as you said earlier.  Also by fail do you mean that the request returns 
an error status?  If so then the read buffer will not be valid.

Re: READ SCSI cmd seems to fail on SATA optical devices...

[Mathieu Fluhr]

On Mon, 2006-11-13 at 13:49 -0500, Phillip Susi wrote:
> Mathieu Fluhr wrote:
> > Hello,
> > 
> > I recently tried to burn some datas on CDs and DVD using a SATA burner
> > and the latest 2.6.18.2 kernel... using NeroLINUX. (It is controlling
> > the device by sending SCSI commands over the 'sg' driver)
> > 
> 
> Please note that the sg interface is depreciated.  It is now recommended 
> that you send the CCBs directly to the normal device, i.e. /dev/hdc.

Of course for native IDE devices, we are using the /dev/hdXX device, but
for SATA devices controlled by the libata, this is not possible ;)

> 
> > The burn process works like a charm, no problems at all. But it seems
> > that there are some slight problems with the READ scsi cmd:
> > Inside our software, we have a verification routine that will make a
> > sector-by-sector verification to check that everything that has been
> > written is OK.
> > 
> > The problem is that, on SATA devices controlled by libata, on some big
> > files (like for example a 600 MB file) the READ command seems to fail
> > and outputs garbage (not 1 or 2 bytes diff, but the whole buffer).
> >  -> This problem does not come out everytime, and each time on    
> >     different sectors.
> > 
> 
> I'm not sure what you mean by "on some big files" as the sg interface 
> has no notion of files; you're just reading the disk sector by sector, 
> as you said earlier.  Also by fail do you mean that the request returns 
> an error status?  If so then the read buffer will not be valid.

I said 'some big files', as the problem is not reproducable everytime.
It occurs most of the time with large files.
The request does not return an error code (no sense key I mean), and it
is working as if everything would have work fine.

> 
> 

Re: READ SCSI cmd seems to fail on SATA optical devices...

[Arjan van de Ven]

On Mon, 2006-11-13 at 19:56 +0100, Mathieu Fluhr wrote:
> On Mon, 2006-11-13 at 13:49 -0500, Phillip Susi wrote:
> > Mathieu Fluhr wrote:
> > > Hello,
> > > 
> > > I recently tried to burn some datas on CDs and DVD using a SATA burner
> > > and the latest 2.6.18.2 kernel... using NeroLINUX. (It is controlling
> > > the device by sending SCSI commands over the 'sg' driver)
> > > 
> > 
> > Please note that the sg interface is depreciated.  It is now recommended 
> > that you send the CCBs directly to the normal device, i.e. /dev/hdc.
> 
> Of course for native IDE devices, we are using the /dev/hdXX device, but
> for SATA devices controlled by the libata, this is not possible ;)

for those there is /dev/scd0 etc...

(usually nicely symlinked to /dev/cdrom)



-- 
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org

Re: READ SCSI cmd seems to fail on SATA optical devices...

[Mathieu Fluhr]

On Mon, 2006-11-13 at 20:32 +0100, Arjan van de Ven wrote:
> On Mon, 2006-11-13 at 19:56 +0100, Mathieu Fluhr wrote:
> > On Mon, 2006-11-13 at 13:49 -0500, Phillip Susi wrote:
> > > Mathieu Fluhr wrote:
> > > > Hello,
> > > > 
> > > > I recently tried to burn some datas on CDs and DVD using a SATA burner
> > > > and the latest 2.6.18.2 kernel... using NeroLINUX. (It is controlling
> > > > the device by sending SCSI commands over the 'sg' driver)
> > > > 
> > > 
> > > Please note that the sg interface is depreciated.  It is now recommended 
> > > that you send the CCBs directly to the normal device, i.e. /dev/hdc.
> > 
> > Of course for native IDE devices, we are using the /dev/hdXX device, but
> > for SATA devices controlled by the libata, this is not possible ;)
> 
> for those there is /dev/scd0 etc...
> (usually nicely symlinked to /dev/cdrom)

Hummm as we are _writing_ to devices, I think that using /dev/sgXX with
SG_IO is better no?

... and the problem is not in accessing the device itself (this is
working like a charm) but understanding why a SCSI READ(10) cmd
sometimes fails as a ATA-padded READ(10) cmd - as discribed in the Annex
A of the MMC-5 spec - ALWAYS works.
-> I would suspect somehow a synchronisation problem somehow in the
translation of SCSI to ATA command...

Another point: When I say that a READ(10) fails, it does NOT mean that
the command execution itself fails. Everything works as if the command
exectution succeeds, but the resulting buffer contains garbage (i.e. not
only 1 or 2 bytes differs)


> 
> 

Re: READ SCSI cmd seems to fail on SATA optical devices...

[Tejun Heo]

[-- Attachment #1: Type: text/plain, Size: 1446 bytes --]

Mathieu Fluhr wrote:
> On Mon, 2006-11-13 at 20:32 +0100, Arjan van de Ven wrote:
>> On Mon, 2006-11-13 at 19:56 +0100, Mathieu Fluhr wrote:
>>> On Mon, 2006-11-13 at 13:49 -0500, Phillip Susi wrote:
>>>> Mathieu Fluhr wrote:
>>>>> Hello,
>>>>>
>>>>> I recently tried to burn some datas on CDs and DVD using a SATA burner
>>>>> and the latest 2.6.18.2 kernel... using NeroLINUX. (It is controlling
>>>>> the device by sending SCSI commands over the 'sg' driver)
>>>>>
>>>> Please note that the sg interface is depreciated.  It is now recommended 
>>>> that you send the CCBs directly to the normal device, i.e. /dev/hdc.
>>> Of course for native IDE devices, we are using the /dev/hdXX device, but
>>> for SATA devices controlled by the libata, this is not possible ;)
>> for those there is /dev/scd0 etc...
>> (usually nicely symlinked to /dev/cdrom)
> 
> Hummm as we are _writing_ to devices, I think that using /dev/sgXX with
> SG_IO is better no?

The recommended way is using SG_IO to /dev/srX (or /dev/scdX).

> ... and the problem is not in accessing the device itself (this is
> working like a charm) but understanding why a SCSI READ(10) cmd
> sometimes fails as a ATA-padded READ(10) cmd - as discribed in the Annex
> A of the MMC-5 spec - ALWAYS works.
> -> I would suspect somehow a synchronisation problem somehow in the
> translation of SCSI to ATA command...

Can you try the attached patch and see if anything changes?

-- 
tejun

[-- Attachment #2: patch --]
[-- Type: text/plain, Size: 828 bytes --]

diff --git a/block/scsi_ioctl.c b/block/scsi_ioctl.c
index 2dc3264..fa82151 100644
--- a/block/scsi_ioctl.c
+++ b/block/scsi_ioctl.c
@@ -286,6 +286,7 @@ static int sg_io(struct file *file, requ
 	 * fill in request structure
 	 */
 	rq->cmd_len = hdr->cmd_len;
+	memset(rq->cmd, 0, BLK_MAX_CDB);
 	memcpy(rq->cmd, cmd, hdr->cmd_len);
 	if (sizeof(rq->cmd) != hdr->cmd_len)
 		memset(rq->cmd + hdr->cmd_len, 0, sizeof(rq->cmd) - hdr->cmd_len);
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index d2c02df..080c2ed 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -410,6 +410,7 @@ int scsi_execute_async(struct scsi_devic
 		goto free_req;
 
 	req->cmd_len = cmd_len;
+	memset(req->cmd, 0, BLK_MAX_CDB);
 	memcpy(req->cmd, cmd, req->cmd_len);
 	req->sense = sioc->sense;
 	req->sense_len = 0;

Re: READ SCSI cmd seems to fail on SATA optical devices...

[Mathieu Fluhr]

On Wed, 2006-11-15 at 02:24 +0900, Tejun Heo wrote:
> > ... and the problem is not in accessing the device itself (this is
> > working like a charm) but understanding why a SCSI READ(10) cmd
> > sometimes fails as a ATA-padded READ(10) cmd - as discribed in the
> Annex
> > A of the MMC-5 spec - ALWAYS works.
> > -> I would suspect somehow a synchronisation problem somehow in the
> > translation of SCSI to ATA command...
> 
> Can you try the attached patch and see if anything changes?
> 

The patch _seems_ to solve my problem. I am just really astonished when
I read the diff file :D. Can I expect that it will be merged to the
official kernel sources ?

Thanks a lot for your answers!

Re: READ SCSI cmd seems to fail on SATA optical devices...

[Tejun Heo]

Mathieu Fluhr wrote:
> On Wed, 2006-11-15 at 02:24 +0900, Tejun Heo wrote:
>>> ... and the problem is not in accessing the device itself (this is
>>> working like a charm) but understanding why a SCSI READ(10) cmd
>>> sometimes fails as a ATA-padded READ(10) cmd - as discribed in the
>> Annex
>>> A of the MMC-5 spec - ALWAYS works.
>>> -> I would suspect somehow a synchronisation problem somehow in the
>>> translation of SCSI to ATA command...
>> Can you try the attached patch and see if anything changes?
>>
> 
> The patch _seems_ to solve my problem. I am just really astonished when
> I read the diff file :D. Can I expect that it will be merged to the
> official kernel sources ?

It seems that some devices choke when the bytes after CDB contain 
garbage.  I seem to recall that I read somewhere ATAPI device require 
left command bytes cleared to zero but I can't find it anywhere now. 
Maybe I'm just imagining.  Anyways, yeah, I'll push it to upstream.

-- 
tejun

Re: READ SCSI cmd seems to fail on SATA optical devices...

[Phillip Susi]

Tejun Heo wrote:
>> The patch _seems_ to solve my problem. I am just really astonished when
>> I read the diff file :D. Can I expect that it will be merged to the
>> official kernel sources ?
> 
> It seems that some devices choke when the bytes after CDB contain 
> garbage.  I seem to recall that I read somewhere ATAPI device require 
> left command bytes cleared to zero but I can't find it anywhere now. 
> Maybe I'm just imagining.  Anyways, yeah, I'll push it to upstream.
> 

The original patch memsets the entire buffer only to copy over most of 
it right after.  Could you amend the patch so that it memcpys first, 
then memsets only the remainder of the buffer?  No sense wasting cpu 
cycles.

Re: READ SCSI cmd seems to fail on SATA optical devices...

[Arjan van de Ven]

On Wed, 2006-11-15 at 11:17 -0500, Phillip Susi wrote:
> Tejun Heo wrote:
> >> The patch _seems_ to solve my problem. I am just really astonished when
> >> I read the diff file :D. Can I expect that it will be merged to the
> >> official kernel sources ?
> > 
> > It seems that some devices choke when the bytes after CDB contain 
> > garbage.  I seem to recall that I read somewhere ATAPI device require 
> > left command bytes cleared to zero but I can't find it anywhere now. 
> > Maybe I'm just imagining.  Anyways, yeah, I'll push it to upstream.
> > 
> 
> The original patch memsets the entire buffer only to copy over most of 
> it right after.  Could you amend the patch so that it memcpys first, 
> then memsets only the remainder of the buffer?  No sense wasting cpu 
> cycles.

due to the funnies of how cpu caches work it might not actually be
faster though... Write Allocate and things like that are.. well fun.



-- 
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org

Re: READ SCSI cmd seems to fail on SATA optical devices...

[Alan]

On Wed, 15 Nov 2006 11:17:29 -0500
Phillip Susi <psusi@cfl.rr.com> wrote:

> The original patch memsets the entire buffer only to copy over most of 
> it right after.  Could you amend the patch so that it memcpys first, 
> then memsets only the remainder of the buffer?  No sense wasting cpu 
> cycles.

Look at the generated code - its faster to memset the entire CDB as Tejun
is doing than actually bother to think about it.

Alan

Re: READ SCSI cmd seems to fail on SATA optical devices...

[Tejun Heo]

Hello, Mattieu Fluhr.

Mathieu Fluhr wrote:
> The problem is that, on SATA devices controlled by libata, on some big
> files (like for example a 600 MB file) the READ command seems to fail
> and outputs garbage (not 1 or 2 bytes diff, but the whole buffer).
>  -> This problem does not come out everytime, and each time on    
>     different sectors.
> 
> Please note that:
> - it is not chipset dependant (tested on nforce4 and sii3114)
> - it is not medium or device dependant

Hmmm... Interesting.  So, you're reading the media by directly issuing 
commands through the sg interface, right?  Can you please try the 
followings?

* Try using /dev/srX or /dev/scdX device instead of /dev/sgX.  You can 
use the command SG_IO but the code path is different, so it will help us 
rule out sg bug.

* Perform rounds of read-verify test using standard block interface (ie. 
simply opening /dev/srX and reading it).

[--snip--]
> - When I force the bus type to be IDE, our software will then send ATA
> commands. In this case, everything works like a charm. No errors at all.

What do you mean by 'sending ATA commands'?  libata exports all devices 
as SCSI devices.  For ATA devices, you can use ATA passthrough but you 
send CDBs to ATAPI devices anyway.

Thanks.

-- 
tejun

Re: READ SCSI cmd seems to fail on SATA optical devices...

[Mathieu Fluhr]

On Tue, 2006-11-14 at 11:02 +0900, Tejun Heo wrote:
> Hello, Mattieu Fluhr.
> 
> Mathieu Fluhr wrote:
> > The problem is that, on SATA devices controlled by libata, on some big
> > files (like for example a 600 MB file) the READ command seems to fail
> > and outputs garbage (not 1 or 2 bytes diff, but the whole buffer).
> >  -> This problem does not come out everytime, and each time on    
> >     different sectors.
> > 
> > Please note that:
> > - it is not chipset dependant (tested on nforce4 and sii3114)
> > - it is not medium or device dependant
> 
> Hmmm... Interesting.  So, you're reading the media by directly issuing 
> commands through the sg interface, right?  Can you please try the 
> followings?
> 
> * Try using /dev/srX or /dev/scdX device instead of /dev/sgX.  You can 
> use the command SG_IO but the code path is different, so it will help us 
> rule out sg bug.
> 
> * Perform rounds of read-verify test using standard block interface (ie. 
> simply opening /dev/srX and reading it).
I will try this and report the results.

> 
> [--snip--]
> > - When I force the bus type to be IDE, our software will then send ATA
> > commands. In this case, everything works like a charm. No errors at all.
> 
> What do you mean by 'sending ATA commands'?  libata exports all devices 
> as SCSI devices.  For ATA devices, you can use ATA passthrough but you 
> send CDBs to ATAPI devices anyway.
Inside our code, we first guess which the bus type of every device using
an INQUIRY cmd (0x12). 
- If the device appears to be a SCSI one, then we send "pure" SCSI CDBs
- For an IDE device, we do, as described in the Annex A of the MMC-5
spec, and pad each CDBs to 12 bytes.

So in the case of a READ CDB, we send 10 bytes if the device is a SCSI
one and 12 for IDE one.


> 
> Thanks.
>