Re: Oddly slow read performance with near-full largish FS

[Robert White <rwhite@pobox.com>]

On 12/19/2014 09:33 AM, Duncan wrote:
> Charles Cazabon posted on Fri, 19 Dec 2014 10:58:49 -0600 as excerpted:
>
>> This configuration is one I've been using for many years.  It's only
>> recently that I've noticed it being particularly slow with btrfs -- I
>> don't know if that's because the filesystem has filled up past some
>> critical point, or due to something else entirely.  That's why I'm
>> trying to figure this out.
>
> Not recommending at this point, just saying these are options...
>
> Btrfs raid56 mode should, I believe, be pretty close to done with the
> latest patches.  That would be 3.19, however, which isn't out yet of
> course.

Putting the encryption above the raid is a _huge_ win that he'd lose.

I've used this same layering before (though not with btrfs).

So if you write a sector in this order only one encryption event (e.g. 
"encrypt this sector") has to take place no matter what raid level is in 
place.

If you put the encryption below the raid, then a write or one sector on 
a non-degraded RAID5 requires four encryption events (two decrypts, one 
for the parity and one for the sector being overwritten; followed by two 
encryptions on the same results).

In degraded conditions the profile is much worse.


If encryptions and RAID > 0 is in use, he's better off with what he's 
got in terms of CPU and scheduling.

>
> There's also raid10, if you have enough devices or little enough data to
> do it.  That's much more mature than raid56 mode and should be about as
> mature and stable as btrfs in single-device mode, which is what you are
> using now.  But it'll require more devices than a raid56 would...
>