Mirrored volume peformance questions

Mirrored volume peformance questions

[Morad, Steve]

I have a few questions about volume mirroring performance implications.

1. I'm looking for an optimal configuration to maximize read speed while protecting the data from a disk failure (e.g. for an application that primarily does reads).  For RAID 1, does the linux raid driver perform parallel read requests across all available disks?  If, for example, I set up a RAID 1 volume with 4 disks, would the OS handle concurrently reading (different) data from all 4 disks at the same time?

2. Similarly, would a RAID10 configuration give me the same (or better) read behavior across these same disks, while providing twice the storage capacity of the above configuration?

3. Is the raid driver smart enough to keep track of underlying volume performance history or queue lengths and make read requests appropriately? For example, if for some crazy reason a 10K disk was mirrored with a 7K disk, would the OS make more read requests to the 10K disk than to the 7K disk, or would read requests be equally distributed across both disks?

Re: Mirrored volume peformance questions

[Roberto Spadim]

2011/5/3 Morad, Steve <morad@amazon.com>:
> I have a few questions about volume mirroring performance implications.
>
> 1. I'm looking for an optimal configuration to maximize read speed while protecting the data from a disk failure (e.g. for an application that primarily does reads).  For RAID 1, does the linux raid driver perform parallel read requests across all available disks?  If, for example, I set up a RAID 1 volume with 4 disks, would the OS handle concurrently reading (different) data from all 4 disks at the same time?

today read_balance algorithm is optimized for multi-thread
read algorithm preffer disk with minimal head distance (current block
- block that will be read)

you can unselect disks setting they as write-mostly

optimizations at raid1 aren´t for bigger sequencial read, they are for
multi-thread



> 2. Similarly, would a RAID10 configuration give me the same (or better) read behavior across these same disks, while providing twice the storage capacity of the above configuration?

in md world
raid1+ raid0 != raid10

raid10 can use layouts
raid1 can´t

raid10 have diferent read_balance algorithms than raid1
raid10 with far layout is better optimized for sequencial read (it´s
like raid0 stripe)
raid10 with near/offset layoute are better optimized for multthread


> 3. Is the raid driver smart enough to keep track of underlying volume performance history or queue lengths and make read requests appropriately? For example, if for some crazy reason a 10K disk was mirrored with a 7K disk, would the OS make more read requests to the 10K disk than to the 7K disk, or would read requests be equally distributed across both disks?

no
i made a new raid1.c algorithm for kernel 2.6.37, that you can change
this options, you will have 1% of speed improvement in a very high
workload of read (ok it´s not a lot but i tested three times (1 hour
non sequencial test) and 1% are seeing, it´s not a error of
benchmark), you can mix SSD and HD too

code is here:
http://www.spadim.com.br/raid1/


it´s not a md-team solution
it´s a patch, more information inside the file
to compile you must compile like anyother kernel module
make drivers/md/raid1 ....

>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mirrored volume peformance questions

[Keld Jørn Simonsen]

On Tue, May 03, 2011 at 04:52:07PM -0300, Roberto Spadim wrote:
> 2011/5/3 Morad, Steve <morad@amazon.com>:
> > I have a few questions about volume mirroring performance implications.
> >
> 
> > 2. Similarly, would a RAID10 configuration give me the same (or better) read behavior across these same disks, while providing twice the storage capacity of the above configuration?

RAID10 and RAID1 gives the same storage capacity with the same disks.

Linux MD RAID10 is actually just another way of doing raid1-like
layouts.


> in md world
> raid1+ raid0 != raid10
> 
> raid10 can use layouts
> raid1 can?t
> 
> raid10 have diferent read_balance algorithms than raid1
> raid10 with far layout is better optimized for sequencial read (it?s
> like raid0 stripe)
> raid10 with near/offset layoute are better optimized for multthread

Hmm, raid10 near, offset and far are about the same for multithread,
according to several benchmarks. Actually the far layout has significant
better random read performance than the near layout in some thests,
about 25 % better speed, and about 100 % bettter speed than raid1.

best regards
keld

Re: Mirrored volume peformance questions

[Roberto Spadim]

near/offset i think it´s the same (near) speed
i´m checking this:
https://raid.wiki.kernel.org/index.php/Performance

and some tests i did

2011/5/3 Keld Jørn Simonsen <keld@keldix.com>:
> On Tue, May 03, 2011 at 04:52:07PM -0300, Roberto Spadim wrote:
>> 2011/5/3 Morad, Steve <morad@amazon.com>:
>> > I have a few questions about volume mirroring performance implications.
>> >
>>
>> > 2. Similarly, would a RAID10 configuration give me the same (or better) read behavior across these same disks, while providing twice the storage capacity of the above configuration?
>
> RAID10 and RAID1 gives the same storage capacity with the same disks.
>
> Linux MD RAID10 is actually just another way of doing raid1-like
> layouts.
>
>
>> in md world
>> raid1+ raid0 != raid10
>>
>> raid10 can use layouts
>> raid1 can?t
>>
>> raid10 have diferent read_balance algorithms than raid1
>> raid10 with far layout is better optimized for sequencial read (it?s
>> like raid0 stripe)
>> raid10 with near/offset layoute are better optimized for multthread
>
> Hmm, raid10 near, offset and far are about the same for multithread,
> according to several benchmarks. Actually the far layout has significant
> better random read performance than the near layout in some thests,
> about 25 % better speed, and about 100 % bettter speed than raid1.
>
> best regards
> keld
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mirrored volume peformance questions

[David Brown]

On 03/05/2011 23:34, Keld Jørn Simonsen wrote:
> On Tue, May 03, 2011 at 04:52:07PM -0300, Roberto Spadim wrote:
>> 2011/5/3 Morad, Steve<morad@amazon.com>:
>>> I have a few questions about volume mirroring performance
>>> implications.
>>>
>>
>>> 2. Similarly, would a RAID10 configuration give me the same (or
>>> better) read behavior across these same disks, while providing
>>> twice the storage capacity of the above configuration?
>
> RAID10 and RAID1 gives the same storage capacity with the same
> disks.
>

Perhaps he meant a 4-drive RAID1 set?  That way reads for any data can 
be taken for any drive.

> Linux MD RAID10 is actually just another way of doing raid1-like
> layouts.
>
>
>> in md world raid1+ raid0 != raid10
>>
>> raid10 can use layouts raid1 can?t
>>
>> raid10 have diferent read_balance algorithms than raid1 raid10 with
>> far layout is better optimized for sequencial read (it?s like raid0
>> stripe) raid10 with near/offset layoute are better optimized for
>> multthread
>
> Hmm, raid10 near, offset and far are about the same for multithread,
> according to several benchmarks. Actually the far layout has
> significant better random read performance than the near layout in
> some thests, about 25 % better speed, and about 100 % bettter speed
> than raid1.
>

raid10,far is better for sequential reads - it gives better-than-raid0 
performance on average since it will do striped reads from the faster 
outer tracks.  And for multi-threaded reads, it should also be a little 
faster than other raid10 layouts (and raid1, which is much the same as 
raid10,near).  Since it prefers to get the data from the outer half, you 
get the benefits of short-stroking your disks - faster transfer speeds 
and less head movement.

The cost of raid10,far is greater head movement for writes - but that is 
not the OP's main concern.

If I interpreted correctly that the OP is considering a 4-way mirror, 
then perhaps raid10,f4 is ideal - sequential reads will be taken from 
all drives at once (in the outer quarter of the disks), and for 
multi-thread reads all requested data can be found on any of the disks.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mirrored volume peformance questions

[Keld Jørn Simonsen]

On Wed, May 04, 2011 at 09:42:40AM +0200, David Brown wrote:
> raid10,far is better for sequential reads - it gives better-than-raid0 
> performance on average since it will do striped reads from the faster 
> outer tracks.  And for multi-threaded reads, it should also be a little 
> faster than other raid10 layouts (and raid1, which is much the same as 
> raid10,near).  Since it prefers to get the data from the outer half, you 
> get the benefits of short-stroking your disks - faster transfer speeds 
> and less head movement.
> 
> The cost of raid10,far is greater head movement for writes - but that is 
> not the OP's main concern.

yes, in theory this is so. But two reasons almost eliminates this in
practice. First, the processes do not wait for completion of the IO of
writes, the processes only deliver the data to the file buffer cache of
the kernel, which then periodically flushes the data to the disk drives.
Second, the flushing of the data is ordered so that the collected data
buffers are written as much sequentially as possible to the drives.
This goes for all Linux MD RAID1/RAID10 layouts. Given that random
writes are random over the whole set of drives, for any mirrored
raid1/raid10 layout, the flushing of the data is about the same.

best regards
keld

Re: Mirrored volume peformance questions

[Roberto Spadim]

i think that a good add to today read balance could be add disk
acess_time (1/rpm, 0 for 'non rotational', 1 for unknow)
head_distance * acess_time, could allow a better select of disks or
mix of 7200/10000/15000 rpm disks
maybe it can be read from disk information at assemble time, i don't
know if disk speed (rpm) can be read from device block (some disks
display information with hdparm and smartctl)
it's not a 1000% improvement, it's 1% at high read/write load

2011/5/4 Keld Jørn Simonsen <keld@keldix.com>:
> On Wed, May 04, 2011 at 09:42:40AM +0200, David Brown wrote:
>> raid10,far is better for sequential reads - it gives better-than-raid0
>> performance on average since it will do striped reads from the faster
>> outer tracks.  And for multi-threaded reads, it should also be a little
>> faster than other raid10 layouts (and raid1, which is much the same as
>> raid10,near).  Since it prefers to get the data from the outer half, you
>> get the benefits of short-stroking your disks - faster transfer speeds
>> and less head movement.
>>
>> The cost of raid10,far is greater head movement for writes - but that is
>> not the OP's main concern.
>
> yes, in theory this is so. But two reasons almost eliminates this in
> practice. First, the processes do not wait for completion of the IO of
> writes, the processes only deliver the data to the file buffer cache of
> the kernel, which then periodically flushes the data to the disk drives.
> Second, the flushing of the data is ordered so that the collected data
> buffers are written as much sequentially as possible to the drives.
> This goes for all Linux MD RAID1/RAID10 layouts. Given that random
> writes are random over the whole set of drives, for any mirrored
> raid1/raid10 layout, the flushing of the data is about the same.
>
> best regards
> keld
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html