Re: Issues while doing btrfs delete missing in raid6

["Jérôme Carretero" <cJ-ko@zougloub.eu>]

Hi Duncan,

On Mon, 20 Nov 2017 21:57:47 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:

> Jérôme Carretero posted on Mon, 20 Nov 2017 01:43:44 -0500 as
> excerpted:
> 
> > While doing a test (to evaluate drives), where I'm filling a bunch
> > of drives in RAID6, one of the disks failed in the process.
> > (System with v4.14 / ECC).  
> 
> FWIW, see raid56 status in the status page (table and below raid56
> note).
> 
> https://btrfs.wiki.kernel.org/index.php/Status
> 
> Basically, after the fixes in 4.12, it mostly works as long as things 
> don't go too badly wrong, but due to the write hole and corner cases
> such as the checksum repair failure you ran into, it's not something
> people on this list can in good conscience recommend for general use,
> because it simply lacks the reliability people tend to want raid56
> for, at least in combination with the file-integrity/checksumming
> features btrfs may be chosen for.  The two together simply aren't as
> reliable as the separate features might imply they should be, and
> there are known to be better alternatives.
> 
> Unfortunately that's likely to remain the case for awhile due to the 
> complexity of a real fix, despite the 4.12 fixes to the worst of the 
> problems.
> 
> One reasonably performant and reliable alternative, tho it's more 
> directly an alternative to btrfs raid10, where it's better performing
> due to btrfs raid10 not yet being performance optimized, is btrfs
> raid1 on top of two raid0s (mdraid0, for instance).

I normally use btrfs RAID1, but wanted to see what's new with RAID6
while "priming" some new disks. There was some click-bait on Phoronix,
the wiki page (status or https://btrfs.wiki.kernel.org/index.php/RAID56)
didn't look very up-to-date and quite vague, and not so many horror
stories on LKML...

Anyway, this was a 200-hour experiment, and apart from the failure, the
speed was low, really far from RAID1 (and there was plenty of CPU left
to compute parity), and "delete missing" was unexpectedly slow, running
at perhaps 10 MB/s average.

TL;DR: As of v4.14 RAID6 is as reliable as RAID0, but slower =)


-- 
Jérôme