Re: 3ware + RAID5 + xfs performance

[Harry Mangalam <hjm@tacgi.com>]

I went thru the same hardware config gymnastics (confounded by bad disks, a 
bad controller, bad hotswap cages, and some SW config issues - see:
http://maillists.uci.edu/mailman/public/uci-linux/2005-June/001067.html).


The 3ware info is partially reported here:
http://maillists.uci.edu/mailman/public/uci-linux/2005-June/001134.html

and I believe I also sent a similar report to this list, but google doesn't 
show it.


Here's the results of my bonnie tests on a similar system:
8x250GB WD SD series disks on a 3ware 9500S-8, on an IWILL 2x Opteron mobo, 
4GB RAM, with XFS.

The XFS parameters are with a 64K stripe to match the RAID card; other params 
more or less vanilla.

Below are some bonnie timing results with differing filesystems (1 run with 
ext3, 3 with XFS with differnt file sizes - unwrap in an editor to compare in 
columns:

 XFS sand, 
7000M,52315,99,104402,22,32242,8,32018,60,127998,20,435.5,0,80,1724,15,+++++,
+++, 3248, 16,1718,15,+++++,+++,750, 4
ext3 sand, 7000M,40682,91, 47732,23,25432,9,38027,72,179352,27,311.1,0,80, 
416,99,+++++,+++,53250,100, 423,99,+++++,+++,560,58
 XFS sand, 
7000M,50040,98,106324,24,33046,8,31269,59,112240,17,416.0,0,80,1930,17,+++++,
+++, 3753, 17,1913,17,+++++,+++,449, 2
 XFS sand, 
15000M,51065,99,101659,24,26884,7,35344,69,141223,23,263.0,0,80,1666,14,
+++++,+++, 4565, 22,1700,16,+++++,+++,793, 4

From my reading (see URL above for resourcelist), XFS is quite bad for tiny 
files - we use it for very large files (>GB size); using XFS for this would 
generally be a bad thing.  We do NOT get tremendous performance out of it; 
but the performance is much better than with ext3 and the CPU usage is lower, 
sometimes dramtically so.  Real life experience with some benchmarks confirms 
this - we get approximately ~ the same real life thruput as we do on a IBM 
SP2 8way module with a direct attach disk.

We are now considering adding a local PVFS2 system to a small cluster for very 
fast IO under MPI




On Monday 25 July 2005 18:11, Gaspar Bakos wrote:
> Dear all,
>
> The purpose of this email is twofold:
> - to share the results of the many tests I performed with
>   a 3ware RAID card + RAID-5 + XFS, pushing for better file I/O,
> - and to initiate some brainstorming on what parameters can be tuned for
>   getting a good performance out of this hardware under 2.6.* kernels.
>
> I started all these tests because the performance was quite poor, meaning
> that the write speed was slow, the read speed was barely acceptable, and
> the system load went very high (10.0) during bonnie++ tests.
> My questions are marked below with "Q".

<clipped>
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html