Optimal chunk size for RAID5?

Optimal chunk size for RAID5?

[Christer Solskogen]

Hi!

I'm about to create a RAID5 with three 4TB disks (ST4000VN000
  from Seagate) and I wonder what the optimal chunk size is/should be.
There are so many different views on the internet that I've almost lost 
faith in the internets.

Is there even such a thing as optimal chunk size?
Are there any other stuff I should think about?

-- 
chs

Re: Optimal chunk size for RAID5?

[Roman Mamedov]

On Sun, 22 Feb 2015 12:31:23 +0100
Christer Solskogen <christer.solskogen@gmail.com> wrote:

> There are so many different views on the internet

...and yet you're asking for some more? :)

> Is there even such a thing as optimal chunk size?

64K should be fine:
http://louwrentius.com/linux-raid-level-and-chunk-size-the-benchmarks.html

-- 
With respect,
Roman

Re: Optimal chunk size for RAID5?

[Christer Solskogen]

On 22.02.2015 13:30, Roman Mamedov wrote:
> On Sun, 22 Feb 2015 12:31:23 +0100
> Christer Solskogen <christer.solskogen@gmail.com> wrote:
>
>> There are so many different views on the internet
>
> ...and yet you're asking for some more? :)
>

Heh, yes. I see the irony in that :-) But this is pretty much as the 
source as you can get.

>> Is there even such a thing as optimal chunk size?
>
> 64K should be fine:
> http://louwrentius.com/linux-raid-level-and-chunk-size-the-benchmarks.html
>

Okay, even if this is almost 5 years old, it is still applicable?

The disks I have are 4k sectors. Is there anything special I need to 
think about or is this handled automagicly these days? (I run a pretty 
fresh distro)

-- 
chs

Re: Optimal chunk size for RAID5?

[Alireza Haghdoost]

On Sun, Feb 22, 2015 at 6:30 AM, Roman Mamedov <rm@romanrm.net> wrote:
> On Sun, 22 Feb 2015 12:31:23 +0100
> Christer Solskogen <christer.solskogen@gmail.com> wrote:
>
>> There are so many different views on the internet
>
> ...and yet you're asking for some more? :)
>
>> Is there even such a thing as optimal chunk size?
>
> 64K should be fine:
> http://louwrentius.com/linux-raid-level-and-chunk-size-the-benchmarks.html
>

I have seen that people report 64K chunk size results better
performance. However, I was not able to find why mdadm maintainers
decided to switch into 512K default chunk size a few years ago ? Was
that decision related to the write-intent bitmap overhead ?

Re: Optimal chunk size for RAID5?

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1266 bytes --]

On Sun, 22 Feb 2015 08:33:02 -0600 Alireza Haghdoost <alireza@cs.umn.edu>
wrote:

> On Sun, Feb 22, 2015 at 6:30 AM, Roman Mamedov <rm@romanrm.net> wrote:
> > On Sun, 22 Feb 2015 12:31:23 +0100
> > Christer Solskogen <christer.solskogen@gmail.com> wrote:
> >
> >> There are so many different views on the internet
> >
> > ...and yet you're asking for some more? :)
> >
> >> Is there even such a thing as optimal chunk size?
> >
> > 64K should be fine:
> > http://louwrentius.com/linux-raid-level-and-chunk-size-the-benchmarks.html
> >
> 
> I have seen that people report 64K chunk size results better
> performance. However, I was not able to find why mdadm maintainers
> decided to switch into 512K default chunk size a few years ago ? Was
> that decision related to the write-intent bitmap overhead ?

No, write-intent-bitmap sizing is completely independent from chunk sizes.

I don't remember the detail for the change, but some measurement must have
gone faster with larger chunk size.

single threaded loads tend to prefer large chunk sizes.
multi-threaded small-request random IO tends to prefer smaller chunk sizes.

There is no "Optimal" without reference to a particular work load.  Or
particular hardware.

NeilBrown



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]

Re: Optimal chunk size for RAID5?

[Christer Solskogen]

On 22.02.2015 22:53, NeilBrown wrote:

> There is no "Optimal" without reference to a particular work load.  Or
> particular hardware.
>

Do you know of such a reference? I mean, some stats that show type of 
workload / chunk size. The only one I've found is the 5 year old 
benchmark that was done ( 
http://louwrentius.com/linux-raid-level-and-chunk-size-the-benchmarks.html) 
- which shows that under benchmarking with dd that 64 is preferred.

-- 
chs

Re: Optimal chunk size for RAID5?

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1632 bytes --]

On Mon, 23 Feb 2015 02:36:18 +0100 Christer Solskogen
<christer.solskogen@gmail.com> wrote:

> On 22.02.2015 22:53, NeilBrown wrote:
> 
> > There is no "Optimal" without reference to a particular work load.  Or
> > particular hardware.
> >
> 
> Do you know of such a reference? I mean, some stats that show type of 
> workload / chunk size. The only one I've found is the 5 year old 
> benchmark that was done ( 
> http://louwrentius.com/linux-raid-level-and-chunk-size-the-benchmarks.html) 
> - which shows that under benchmarking with dd that 64 is preferred.
> 

Interesting graphs ... but when you see a big jump like they show between 64
and 128K chunk sizes for RAID5/6, that doesn't mean "64K is better" but
"something strange is happening here".  My guess is that read-ahead is
working very well for some reason.

If your actually workload is writing 10GB files with 'dd', then the graphs
might be useful.  For other workloads ... it's hard to tell.

Nothing beats performing your own tests on your own hardware with your own
choice of filesystem and getting your own results.

I did some tests myself recently (which I really want to automate and turn
into web pages etc ... one day).
For RAID5 on 4 drives I used chunk sizes of 4, 16, 64, 256, 1024 and applied
a variety of fio loads use XFS.

The only load that showed significant variation of chunk sizes was sequential
read which gets generally faster with larger chunk sizes, though for some
layouts (I tried la, ls, ra, rs) 1024k chunks were worse than 256k.

So any reference you find will probably lead you astray.

NeilBrown

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]