Re: 答复：md raid5 performace 6x SSD RAID5

[Stan Hoeppner <stan@hardwarefreak.com>]

On 11/27/2013 10:46 PM, Roman Mamedov wrote:
> On Wed, 27 Nov 2013 22:41:49 -0600
> Stan Hoeppner <stan@hardwarefreak.com> wrote:
> 
>>> when the stripe_cache_size is set 4096. 
> ...
>> 2.  A 4KB stripe cache is too small for 6 SSDs, try 8KB
> 
> The stripe cache size setting is specified not in KB, but in pages per disk,
> so a value of 4096 on x86 systems means 4096*4096*6 = 96 MB of cache for the
> whole array.

Thanks Roman for correcting me on that which I know well.  Typing a
trailing "KB" so often hard wires the brain and fingers I guess.  My KBs
were intended to be Ks.

http://www.spinics.net/lists/raid/msg42370.html
On 04/03/13 23:20, Stan Hoeppner wrote:
...
> Formula:  stripe_cache_size * 4096 bytes * drive_count = RAM usage.

To expound on the importance of this, with a handful of SSDs and a value
of 8K, throughput tends to plateau, and then slowly decreases as
stripe_cache_size is increased.  The upper bound of stripe_cache_size
gains has not yet been established because the write thread peaks a core
with only a few SSDs.  Multiple write threads and a larger quantity of
SSDs, or much faster SSDs, are needed to explore whether values of
16K-32K provide a meaningful increase in throughput, and whether this is
worth the RAM consumed.  For instance, with 12 SSDs and
stripe_cache_size of 32768:

(((32768*4096)*12)/1048576)/1000 = 1.5 GB of RAM is consumed

When Shaohua Li completes his threading patch series it may be possible
to explore this more thoroughly.

-- 
Stan