Re: calculating optimal chunk size for Linux software-RAID

[Stan Hoeppner <stan@hardwarefreak.com>]

On 3/6/2014 8:06 PM, Martin T wrote:
> Am I correct that optimal chunk size is usually the size of the
> average file read/written to disk divided by number of block devices
> in RAID array storing the data? For example if the average file size
> is 1024KiB and I have four disks in RAID1, then I should choose the
> chunk size around 256KiB to get the optimal read performance? Or if I
> have two drives in RAID0, then I should choose the chunk size 512KiB
> instead? Or are there better methods/benchmarks to determine the
> optimal chunk size for software-RAID? 

You're asking the wrong question.  Storage architecture design always
begins with the workload.  The correct question is:

My workload (application mix) performs *most* IO in manner X, where X is

1.  large streaming write/read
2.  small file write/read
3.  metadata heavy

I have Y number of disk drives.  I plan to use XFS/EXT4/etc filesystem.
 What RAID level and chunk size are optimal for my workload, and how do
I properly tune my filesystem to my workload and storage stack?

> Last but not least, is there a
> good utility which could help one to measure the average I/O
> read/write size?

In flight IO size has no correlation to stripe and chunk size.  What you
need to know is how your application(s) write to the filesystem and how
your filesystem issues write IOs.  You should already know that the
former, and it's easy to determine the latter.

-- 
Stan