On 2015-08-17 19:06, Duncan wrote:
> Austin S Hemmelgarn posted on Mon, 17 Aug 2015 07:38:13 -0400 as
> excerpted:
>
>> I've also found that BTRFS raid5/6 on top of MD RAID0 mitigates (to a
>> certain extent that is) the performance penalty of doing raid5/6 if you
>> aren't on ridiculously fast storage, probably not something that should
>> be used in production yet, but it's how I've got the near-line backups
>> setup on my home server system.
>
> As should be clear from my previous posts on the subject, I'm
> conservative enough not to be comfortable with the btrfs raid56
> implementation yet.  My recommendation has been, and remains, unless
> you're deliberately testing it in ordered to help find/report/workout
> bugs, give it a year after the nominally full implementation (3.19, so
> until 4.4), before expecting it to be reasonably as stable as the rest of
> btrfs (which itself isn't fully stable yet).
>
> But the almost-released 4.2 does seem to be past the initial nominally
> btrfs raid56 full-code bugs, and I'd call an intermediate level backup,
> with working copies in front and itself backed up in back, a reasonable
> first working (as opposed to testing) deployment.
Yeah, I've been ridiculously luck to have not hit _any_ of the raid56 
related bugs.  In fact the only issue I've had with it was a result of a 
btrfs interaction with dm-thinp (if dm-thinp isn't set to zero newly 
allocated blocks, btrfs sometimes loses it's mind during remount, which 
in turn reminds me that I meant to check if this was fixed or not).

And the deployment you suggest is ironically how I use it, I've got my 
root filesystem on btrfs raid1 across 2 SSD's, with a btrfs raid6 on top 
of LVM single volumes on a set of 4 1TB HDD's as a target for receive 
(and configured such that I can directly boot any of the backups there), 
and then store compressed, encrypted tarballs of the Sunday backups on 3 
different cloud storage services and an external 4TB HDD (It's wonderful 
how Gentoo lends itself so well to custom solutions).
>
> And yes, btrfs raid5/6 over mdraid0  would have the same general
> complementary nature as btrfs raid1/10 over mdraid0.
>
>> It may also be worth pointing out that
>> BTRFS raid6 lets you use 4 disks minimum, as opposed to most other raid6
>> implementations that (unnecessarily, as a 4 disk RAID6 is not a
>> degenerate form) require 5.
>
> 4-device raid6, btrfs and mdraid both allow that, good point.  But of
> course mdraid6 doesn't have the data integrity, only rebuild-parity.
>
Huh, I didn't know that mdraid allowed that, I know dm-raid through LVM 
doesn't (which in turn is a large part of what caused me to try btrfs 
raid56 so soon, I had been going to do btrfs raid1 on top of LVM based 
raid6).