Re: RAID-5 streaming read performance

[David Greaves <david@dgreaves.com>]

Dan Christensen wrote:

>Ming Zhang <mingz@ele.uri.edu> writes:
>
>  
>
>>test on a production environment is too dangerous. :P
>>and many benchmark tool u can not perform as well.
>>    
>>
>
>Well, I put "production" in quotes because this is just a home mythtv
>box.  :-)  So there are plenty of times when it is idle and I can do
>benchmarks.  But I can't erase the hard drives in my tests.
>  
>
Me too.

>>LVM overhead is small, but file system overhead is hard to say.
>>    
>>
>I expected LVM overhead to be small, but in my tests it is very high.
>I plan to discuss this on the lvm mailing list after I've got the RAID
>working as well as possible, but as an example:
>
>Streaming reads using dd to /dev/null:
>
>component partitions, e.g. /dev/sda7: 58MB/s
>raid device /dev/md2:                 59MB/s
>lvm device /dev/main/media:           34MB/s
>  
>
This is not my experience.
What are the readahead settings?
I found significant variation in performance by varying the readahead at
raw, md and lvm device level

In my setup I get

component partitions, e.g. /dev/sda7: 39MB/s
raid device /dev/md2:                 31MB/s
lvm device /dev/main/media:           53MB/s

(oldish system - but note that lvm device is *much* faster)

For your entertainment you may like to try this to 'tune' your readahead
- it's OK to use so long as you're not recording:

(FYI I find that setting readahead to 0 on all devices and 4096 on the
lvm device gets me the best performance - which makes sense if you think
about it...)

#!/bin/bash
RAW_DEVS="/dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/hdb"
MD_DEVS=/dev/md0
LV_DEVS=/dev/huge_vg/huge_lv

LV_RAS="0 128 256 1024 4096 8192"
MD_RAS="0 128 256 1024 4096 8192"
RAW_RAS="0 128 256 1024 4096 8192"

function show_ra()
{
for i in $RAW_DEVS $MD_DEVS $LV_DEVS
do echo -n "$i `blockdev --getra $i`  ::  "
done
echo
}

function set_ra()
{
 RA=$1
 shift
 for dev in $@
 do
   blockdev --setra $RA $dev
 done
}

function show_performance()
{
 COUNT=4000000
 dd if=$LV_DEVS of=/dev/null count=$COUNT 2>&1 | grep seconds
}

for RAW_RA in $RAW_RAS
 do
 set_ra $RAW_RA $RAW_DEVS
 for MD_RA in $MD_RAS
   do
   set_ra $MD_RA $MD_DEVS
   for LV_RA in $LV_RAS
     do
     set_ra $LV_RA $LV_DEVS
     show_ra
     show_performance
     done
   done
 done