md.txt, preread_bypass_threshold and raid5: corrected error

md.txt, preread_bypass_threshold and raid5: corrected error

[Justin Piszcz]

Read through md.txt from 2.6.27.5, noticed this at the end:

   preread_bypass_threshold (currently raid5 only)
       number of times a stripe requiring preread will be bypassed by
       a stripe that does not require preread.  For fairness defaults
       to 1.  Setting this to 0 disables bypass accounting and
       requires preread stripes to wait until all full-width stripe-
       writes are complete.  Valid values are 0 to stripe_cache_size.

In terms of performance for raid5/raid6 - what does this mean?

Also in md.txt there is a lot of outdated information, (e.g. you can only 
grow raid5, etc).

e.g.:

   raid_disks
      a text file with a simple number indicating the number of devices
      in a fully functional array.  If this is not yet known, the file
      will be empty.  If an array is being resized (not currently
                                                   ^^^^^^^^^^^^^^
      possible) this will contain the larger of the old and new sizes.
      ^^^^^^^^^
      Some raid level (RAID1) allow this value to be set while the
      array is active.  This will reconfigure the array.   Otherwise
      it can only be set while assembling an array.


BTW, I did read the URL on correcting sectors etc on RAID5 from yesterday, 
it was quite nice, but is there anyway that-- it can happen automatically 
like a traditional HW raid controller does during a scrubbing or must one 
perform those tasks in the document in order to make it force-reallocate 
the sectors etc?

Justin.

Re: md.txt, preread_bypass_threshold and raid5: corrected error

[Justin Piszcz]

On Wed, 12 Nov 2008, Justin Piszcz wrote:

> Read through md.txt from 2.6.27.5, noticed this at the end:
>
>  preread_bypass_threshold (currently raid5 only)
>      number of times a stripe requiring preread will be bypassed by
>      a stripe that does not require preread.  For fairness defaults
>      to 1.  Setting this to 0 disables bypass accounting and
>      requires preread stripes to wait until all full-width stripe-
>      writes are complete.  Valid values are 0 to stripe_cache_size.
>
> In terms of performance for raid5/raid6 - what does this mean?
>
> Also in md.txt there is a lot of outdated information, (e.g. you can only 
> grow raid5, etc).
>
> e.g.:
>
>  raid_disks
>     a text file with a simple number indicating the number of devices
>     in a fully functional array.  If this is not yet known, the file
>     will be empty.  If an array is being resized (not currently
>                                                  ^^^^^^^^^^^^^^
>     possible) this will contain the larger of the old and new sizes.
>     ^^^^^^^^^
>     Some raid level (RAID1) allow this value to be set while the
>     array is active.  This will reconfigure the array.   Otherwise
>     it can only be set while assembling an array.
>
>
> BTW, I did read the URL on correcting sectors etc on RAID5 from yesterday, it 
> was quite nice, but is there anyway that-- it can happen automatically like a 
> traditional HW raid controller does during a scrubbing or must one perform 
> those tasks in the document in order to make it force-reallocate the sectors 
> etc?
>
> Justin.
>

This is from Bruce on the smartmontools mailing list:

Here is a nice description of tracking down an UNC sector on a Linux 
software RAID device.  Apparently it was subsequently corrected by the 
RAID driver.

http://www.roeckx.be/journal/Finding_which_sector_belongs_to_which_file_on_a_RAID_device_.html

Re: md.txt, preread_bypass_threshold and raid5: corrected error

[Dan Williams]

On Wed, Nov 12, 2008 at 9:49 AM, Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
> Read through md.txt from 2.6.27.5, noticed this at the end:
>
>  preread_bypass_threshold (currently raid5 only)
>      number of times a stripe requiring preread will be bypassed by
>      a stripe that does not require preread.  For fairness defaults
>      to 1.  Setting this to 0 disables bypass accounting and
>      requires preread stripes to wait until all full-width stripe-
>      writes are complete.  Valid values are 0 to stripe_cache_size.
>
> In terms of performance for raid5/raid6 - what does this mean?
>

When writes are streaming do a raid5/6 array the best performance is
achieved when the code can do a full-stripe-write.  In other words, a
write that does not require reading old data to generate new parity.
By default MD tries to be fair and allow partially written stripes the
same access to the disk as fully written stripes to prevent starvation
for little writers.  However, you can sometimes eek out more streaming
performance at the cost of fairness by increasing this value (where 0
== never let a preread pass a full-stripe-write).

--
Dan

Re: md.txt, preread_bypass_threshold and raid5: corrected error

[Richard Scobie]

Dan Williams wrote:

> By default MD tries to be fair and allow partially written stripes the
> same access to the disk as fully written stripes to prevent starvation
> for little writers.  However, you can sometimes eek out more streaming
> performance at the cost of fairness by increasing this value (where 0
> == never let a preread pass a full-stripe-write).

I believe XFS optimises for full stripe writes, so perhaps this setting 
is of more benefit for other file systems?

As always I guess, real world testing is the only answer.

Regards,

Richard

Re: md.txt, preread_bypass_threshold and raid5: corrected error

[Dan Williams]

On Sat, Nov 15, 2008 at 10:52 AM, Richard Scobie <richard@sauce.co.nz> wrote:
> Dan Williams wrote:
>
>> By default MD tries to be fair and allow partially written stripes the
>> same access to the disk as fully written stripes to prevent starvation
>> for little writers.  However, you can sometimes eek out more streaming
>> performance at the cost of fairness by increasing this value (where 0
>> == never let a preread pass a full-stripe-write).
>
> I believe XFS optimises for full stripe writes, so perhaps this setting is
> of more benefit for other file systems?
>

Not really.  XFS was used when benchmarking this change [1] and was
able to measure a performance gain.

[1] http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=8b3e6cdc53b7f29f7026955d6cb6902a49322a15