Re: Higher than expected disk write(2) latency

[Robert Hancock <hancockr@shaw.ca>]

Martin Lucina wrote:
> Hi,
> 
> we're getting some rather high figures for write(2) latency when testing
> synchronous writing to disk.  The test I'm running writes 2000 blocks of
> contiguous data to a raw device, using O_DIRECT and various block sizes
> down to a minimum of 512 bytes.  
> 
> The disk is a Seagate ST380817AS SATA connected to an Intel ICH7
> using ata_piix.  Write caching has been explicitly disabled on the
> drive, and there is no other activity that should affect the test
> results (all system filesystems are on a separate drive).  The system is
> running Debian etch, with a 2.6.24 kernel.
> 
> Observed results:

Well, write performance on SATA with no command queueing and write cache 
disabled is always going to be pretty atrocious, since the drive has no 
opportunity to reorder the writes at all, and it also can't receive the 
next write command until the previous one completes so that it may have 
to wait for another rotation in order to perform each write. In this 
case I don't think command queueing even helps you though, as only one 
write from the app is ever outstanding at a time. I suspect the only 
thing that would really help this workload is a RAID controller with a 
battery-backed write cache (that way those tiny O_DIRECT writes don't 
all have to hit the physical disk).