Re: Rationale for hardware RAID 10 su, sw values in FAQ

[Dave Chinner <david@fromorbit.com>]

On Thu, Sep 28, 2017 at 12:36:39PM +1300, Ewen McNeill wrote:
> Hi Dave,
> 
> On 27/09/17 17:33, Dave Chinner wrote:
> >On Wed, Sep 27, 2017 at 02:56:16PM +1300, Ewen McNeill wrote:
> >>As a suggestion the FAQ section [hint at reason for RAID10 sw=N/2]
> >
> >This assumes the reader understands exactly how RAID works and how
> >the different types of RAID affect IO performance. The FAQ simply
> >tries to convey how to get it right without going into the
> >complicated reasons for doing it that way.
> 
> Fair enough.
> 
> However, as a "sysadmin searching for answers" it is.... difficult
> to judge the up-to-date-ness and accuracy of "use these magic
> values" tuning information without some insight into why and thus
> whether those values (a) are still applicable with modern
> software/hardware or (b) applicable in one's own situation.
> Particularly so when one has already read N different variations on
> recommendations, all without justification, some with conflicting
> (or confused) advice.

Yeah, searching for information doesn't help these days - engines
like google are bloody terrible at finding relevant, useful answers
to technical questions these days....

> If the FAQ isn't the right place to even hint at the reasons, then
> it would at least be very helpful if the FAQ said "see this
> $REFERENCE for more information" and said $REFERENCE had some more
> "behind the scenes" information.

*nod*

> >https://git.kernel.org/pub/scm/fs/xfs/xfs-documentation.git/tree/admin/XFS_Performance_Tuning/filesystem_tunables.asciidoc
> >
> >You'll see the section about all this reads: [TODO]
> >because nobody (well, me, really) has had the time to write
> >everything down that is needed to cover this topic sufficiently...
> 
> Perhaps the perfect is the enemy of the good here.

It always is :)

> Would it help if
> I were to write up some text covering:
> 
> - key storage alignment tuning aims (avoid work amplification, eg
> RMW; avoid hot spots; increase parallelism)
> 
> - summary of physical storage technology concerns (4K sectors, seek
> latency, SSD erase blocks)
> 
> - summary of considerations for RAID levels (1 - none; 0 - hot
> spots; 5/6 RMW, hot spots; 10 - hot spots; stripe size/width)
> 
> - summary of storage virtualisation considerations on alignment (eg, LVM)
> 
> - high level advice on determining su/sw given the above, and short
> rationale for each formula
> 
> - some observations on unit confusion (sunit/su swidth/sw, values
> supplied in 512 byte sectors / kB versus reported in usually 4kB
> blocks)
> 
> I'm obviously not an expert in filesystem layout optimisation.  But
> I have many years sysadmin experience dealing with lots of types of
> storage, and can write a couple of pages of text on the "background"
> bits next time I have a free moment.  The more advanced tunables
> could remain "todo" until there's time for the perfect version...

That sounds like an excellent idea. The documentation always ends up
more useful to end users and admins when it is written by a user
rather than a developer.

> (No promises when, but "October" seems plausible if this would be useful.)

Sounds good to me.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com