Write Performance double Read Performance

Write Performance double Read Performance

[Terrence Martin]

I have the following system(s)

Dual 2.8 Ghz Xeon 2U Supermicro Server with 2GB RAM
Qlogic QLA2310f Controller

attached to an...

Infortrend A16F with 16 300GB WD IDE PATA Drives
The array as a RAID 5 Array with 3 partitions of 1.4TB each. I have 
mounted one of the partitions to test.

The OS is....

Fedora Core 1 with the latest Fedora 2.4 Kernel (2.4.22-1.2174.nptlsmp)
I have compiled the QLA drivers from the Qlogic site and am using v6.06.10.

The problem I am noticing is that while doing an IOZone performance test 
I get nice write performance when the test gets to large files. However 
my read performance seems to be a little over half my write.   I thought 
the read should always be faster than the write?

Command line used: /opt/iozone/bin/iozone -Ra -g 4G -i 0 -i 1

Eg. Copy paste from IOZone run

KB         reclen write      rewrite     read       reread
2097152      64  154313  171266    91012    92388
2097152     128  154125  170701    89532    82991
2097152     256  151473  169734    86227    85231
2097152     512  150925  168767    84953    86313
2097152    1024  154947  175371    84576    85733
2097152    2048  154581  175161    84865    85233
2097152    4096  155754  175689    85229    86450
2097152    8192  155847  175645    85320    86052
2097152   16384  155500  175313    84817    86864

I only notice this performance drop once I get into large reads. Here 
when the file is half RAM performance is very good.

KB         reclen write      rewrite     read       reread
1048576      64  174326  236646  1571371  1722042
1048576     128  171660  236849  1552772  1699937
1048576     256  173722  239855   893958   938153
1048576     512  171075  238875   660975   683853
1048576    1024  169105  239178   651627   675901
1048576    2048  167846  240387   649875   673732
1048576    4096  168663  238126   653533   675654
1048576    8192  171406  239028   646698   669387
1048576   16384  171545  237270   647777   669695

Before I went down the road of compiling various kernels etc I thought I 
would post this to linux-scsi to see if it is just some configuration 
setting to a driver or kernel parameter that needs to be tweaked.  Of 
course I could be completely reading this wrong but bonnie++ also 
appears to give similar results and vmstat is similarly low on the read 
side.

Thanks,

Terrence

Re: Write Performance double Read Performance

[James Pearson]

Terrence Martin <tmartin@physics.ucsd.edu> wrote:
>
> I have the following system(s)
> 
> Dual 2.8 Ghz Xeon 2U Supermicro Server with 2GB RAM
> Qlogic QLA2310f Controller
> 
> attached to an...
> 
> Infortrend A16F with 16 300GB WD IDE PATA Drives
> The array as a RAID 5 Array with 3 partitions of 1.4TB each. I have 
> mounted one of the partitions to test.
> 
> The OS is....
> 
> Fedora Core 1 with the latest Fedora 2.4 Kernel (2.4.22-1.2174.nptlsmp)
> I have compiled the QLA drivers from the Qlogic site and am using v6.06.10.
> 
> The problem I am noticing is that while doing an IOZone performance test 
> I get nice write performance when the test gets to large files. However 
> my read performance seems to be a little over half my write.   I thought 
> the read should always be faster than the write?

Unfortunately I don't know why, but this is something I've been thinking
of asking as well.

I've seen similar behaviour on a 2.6.3 kernel machine (dual 2.4 Xeons,
2GB RAM) using an external U160 SCSI Chaparral hardware RAID 5 on a
aic79xx controller. I'm using XFS on the RAID.

I've just been using 'lmdd' to/from a 2Gb file - I also see writes about
twice the speed of reads (I'm using the fsync or sync option to lmdd
with the writes).

However, if I increase the readahead on the disk using blockdev from the
default of 256 to 4096, I can get read speeds to be about the same as
the writes.

However, increasing the readahead seems to help this simple streaming
case, but as yet I don't know if it will work in a general file serving
(NFS) situation.

> Before I went down the road of compiling various kernels etc I thought I 
> would post this to linux-scsi to see if it is just some configuration 
> setting to a driver or kernel parameter that needs to be tweaked.  Of 
> course I could be completely reading this wrong but bonnie++ also 
> appears to give similar results and vmstat is similarly low on the read 
> side.

I would also like to know of any optimizations that can be made.

Thanks

James Pearson