Perform uncached reads on SCSI drives?

Perform uncached reads on SCSI drives?

[Drew Winstel]

Is there a way to perform large quantities of uncached reads on a drive 
connected to a SCSI controller under Linux 2.6.x?

Here's a little about the rationale behind such a move:
Previously, we had been testing hard drives using full-capacity, 1/3-capacity, 
and random stroke using the SCSI Seek 10 command with SCSI drives and the 
WIN_SEEK command for 'true' IDE drives (i.e. drives accessed via /dev/hd[a-t]) 
as part of our process for ensuring drives meet customer performance and 
reliability requirements.  Thanks to the emergence of SATA and the resulting 
inquiries from technical minds, we have begun to test those drives using the 
same benchmarks.

This is where we run into problems.  

Since libata-scsi.c simulates (basically a no-op; see libata-scsi.c) the SEEK 
6 and SEEK 10 commands, we have attempted to switch to calling reads to the 
beginning/middle/end/wherever as necessary.  

The reads we're calling are getting cached.  If we call 100 reads--50 to the 
beginning of the drive alternating with 50 to the end--the first read will 
sometimes appear to have a realistic result, then the rest will all complete 
in the neighborhood of 10 microseconds.  

Here is a sample SG_IO header we're passing via ioctl():
interface_id = 83
dxfer_direction = -3
cmd_len = 10 
mx_sb_len = 32 
iovec_count = 0
dxfer_len = 512
dxferp = 0xbfffedc0
cmdp = 0xbfffed60 
sbp = 0x80786a8 
timeout = 20000
flags = 3
pack_id = 0
usr_ptr = 0x0
status = 0 
masked_status = 0 
msg_status = 0 
sb_len_wr = 0 
host_status = 0
driver_status = 0
resid = 0
duration = 0
info = 0

Hardware tested:
Silicon Image SiI 3512 (sata_sil driver)
Promise SATA150 TX4 (sata_promise)
Promise Ultra133 TX2 (PDC20269 chip; pata_pdc2027x driver)

Drives:
Fujitsu MHT2080BH (80 GB 2.5" SATA)
Seagate ST3200822AS (200 GB 3.5" SATA)
Seagate ST3400832AS (400 GB SATA)
Fujitsu MHT2080AH (80 GB 2.5" PATA)
Toshiba MK4019GAX (40 GB 2.5" PATA)

Kernel versions tested:
2.6.7
2.6.11 with libata-dev as of March 11

Only the large Seagate drive was tested with the Promise SATA controller (and 
vice versa).  Only the sata_sil driver was tested with 2.6.7. All other 
combinations (keeping SATA with SATA and PATA with PATA, of course) were tested 
with the same results, leading me to believe libata was to blame, unless by some
chance we're missing a magic flag in our command block.

Just to make sure this was not a libata-specific issue, I tested this same read 
sequence with a Seagate ST336706LC Cheetah and a Seagate ST973401LC Savvio 
connected to an Adaptec 29160 (aic7xxx driver) Ultra160 controller.  Lo and 
behold, this produced the same results, hence me posting here instead of 
linux-ide.  

Any help would be greatly appreciated.

Thanks,
Drew

RE: Perform uncached reads on SCSI drives?

[Drew Winstel]

It has been a busy week, I know, but does anyone have any tips?

Thanks, 

Drew

---------------------------------------------------------------------------
Is there a way to perform large quantities of uncached reads on a drive 
connected to a SCSI controller under Linux 2.6.x?

Here's a little about the rationale behind such a move:
Previously, we had been testing hard drives using full-capacity, 1/3-capacity, 
and random stroke using the SCSI Seek 10 command with SCSI drives and the 
WIN_SEEK command for 'true' IDE drives (i.e. drives accessed via /dev/hd[a-t]) 
as part of our process for ensuring drives meet customer performance and 
reliability requirements.  Thanks to the emergence of SATA and the resulting 
inquiries from technical minds, we have begun to test those drives using the 
same benchmarks.

This is where we run into problems.  

Since libata-scsi.c simulates (basically a no-op; see libata-scsi.c) the SEEK 
6 and SEEK 10 commands, we have attempted to switch to calling reads to the 
beginning/middle/end/wherever as necessary.  

The reads we're calling are getting cached.  If we call 100 reads--50 to the 
beginning of the drive alternating with 50 to the end--the first read will 
sometimes appear to have a realistic result, then the rest will all complete 
in the neighborhood of 10 microseconds.  

Here is a sample SG_IO header we're passing via ioctl():
interface_id = 83
dxfer_direction = -3
cmd_len = 10 
mx_sb_len = 32 
iovec_count = 0
dxfer_len = 512
dxferp = 0xbfffedc0
cmdp = 0xbfffed60 
sbp = 0x80786a8 
timeout = 20000
flags = 3
pack_id = 0
usr_ptr = 0x0
status = 0 
masked_status = 0 
msg_status = 0 
sb_len_wr = 0 
host_status = 0
driver_status = 0
resid = 0
duration = 0
info = 0

Hardware tested:
Silicon Image SiI 3512 (sata_sil driver)
Promise SATA150 TX4 (sata_promise)
Promise Ultra133 TX2 (PDC20269 chip; pata_pdc2027x driver)

Drives:
Fujitsu MHT2080BH (80 GB 2.5" SATA)
Seagate ST3200822AS (200 GB 3.5" SATA)
Seagate ST3400832AS (400 GB SATA)
Fujitsu MHT2080AH (80 GB 2.5" PATA)
Toshiba MK4019GAX (40 GB 2.5" PATA)

Kernel versions tested:
2.6.7
2.6.11 with libata-dev as of March 11

Only the large Seagate drive was tested with the Promise SATA controller (and 
vice versa).  Only the sata_sil driver was tested with 2.6.7. All other 
combinations (keeping SATA with SATA and PATA with PATA, of course) were tested 
with the same results, leading me to believe libata was to blame, unless by some
chance we're missing a magic flag in our command block.

Just to make sure this was not a libata-specific issue, I tested this same read 
sequence with a Seagate ST336706LC Cheetah and a Seagate ST973401LC Savvio 
connected to an Adaptec 29160 (aic7xxx driver) Ultra160 controller.  Lo and 
behold, this produced the same results, hence me posting here instead of 
linux-ide.  

Any help would be greatly appreciated.

Thanks,
Drew
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Perform uncached reads on SCSI drives?

[Patrick Mansfield]

On Thu, Apr 21, 2005 at 10:44:23AM -0500, Drew Winstel wrote:
> It has been a busy week, I know, but does anyone have any tips?

You could try setting read cache disable, I don't know if that setting
must be honored, or how ATA/SATA handle it. See mode page 8 for SCSI block
commands. 

The sg utilities sg_modes can probably set it.

Better yet, do IO to more locations on the disk, not just the very
beginning and the very end.

> The reads we're calling are getting cached.  If we call 100 reads--50 to the 
> beginning of the drive alternating with 50 to the end--the first read will 
> sometimes appear to have a realistic result, then the rest will all complete 
> in the neighborhood of 10 microseconds.  

-- Patrick Mansfield

RE: Perform uncached reads on SCSI drives?

[Drew Winstel]

Hello, Patrick, and thanks for the reply.

>You could try setting read cache disable, I don't know if that setting
>must be honored, or how ATA/SATA handle it. See mode page 8 for SCSI block
>commands. 

Hmm, that's definitely worth a shot.  I'll try it and let you know how it
works.

>The sg utilities sg_modes can probably set it.

As would the new sdparm, I imagine.  

>Better yet, do IO to more locations on the disk, not just the very
>beginning and the very end.

In our seek tests, we do (in order) full-stroke, one-third capacity, and random
seeks.  The random seeks appear to be reasonably well-behaved, although still 
lower than what we're used to seeing (read time is roughly 75% of the seek time 
on an average of four runs of 100 seeks each).  

>-- Patrick Mansfield

Drew

Re: Perform uncached reads on SCSI drives?

[Douglas Gilbert]

Drew Winstel wrote:
> Hello, Patrick, and thanks for the reply.
> 
> 
>>You could try setting read cache disable, I don't know if that setting
>>must be honored, or how ATA/SATA handle it. See mode page 8 for SCSI block
>>commands. 
> 
> 
> Hmm, that's definitely worth a shot.  I'll try it and let you know how it
> works.
> 
> 
>>The sg utilities sg_modes can probably set it.
> 
> 
> As would the new sdparm, I imagine.

Drew,
sg_modes only reads mode pages. They can be changed with
sginfo (or scsiinfo) and sg_wr_mode. The simplest is
probably:
   sdparm --set=RCD /dev/sda

or, if you want the change to survive the next disk
power cycle:
   sdparm --set=RCD --save /dev/sda

If /dev/sda is really a ATA/SATA disk then I don't
think the read cache can be disabled (judging from
the SAT document).

BTW If the sd driver sees a disk with WCE=1 and RCD=1
it outputs: "driver cache: write back, no read (daft)".
IMO the "daft" bit is itself daft. It is logically valid
(i.e. a read of a block in the write cache will cause
that block to be written to the media, then read back)
and reserves the cache exclusively for writing. When
copying one disk (or partition) to another, caching
reads seems futile.

Doug Gilbert