Re: 1 week to rebuid 4x 3TB raid10 is a long time!

[George Mitchell <george@chinilu.com>]

On 07/20/2014 02:28 PM, Bob Marley wrote:
> On 20/07/2014 21:36, Roman Mamedov wrote:
>> On Sun, 20 Jul 2014 21:15:31 +0200
>> Bob Marley <bobmarley@shiftmail.org> wrote:
>>
>>> Hi TM, are you doing other significant filesystem activity during this
>>> rebuild, especially random accesses?
>>> This can reduce performances a lot on HDDs.
>>> E.g. if you were doing strenous multithreaded random writes in the
>>> meanwhile, I could expect even less than 5MB/sec overall...
>> I believe the problem here might be that a Btrfs rebuild *is* a 
>> strenuous
>> random read (+ random-ish write) just by itself.
>>
>> Mdadm-based RAID would rebuild the array reading/writing disks in a 
>> completely
>> linear manner, and it would finish an order of magnitude faster.
>
> Now this explains a lot!
> So they would just need to be sorted?
> Sorting the files of a disk from lowest to highers block number prior 
> to starting reconstruction seems feasible. Maybe not all of them 
> together because they will be millions, but sorting them in chunks of 
> 1000 files would still produce a very significant speedup!
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
As I understand the problem, it has to do with where btrfs is in the 
overall development process.  There are a LOT of opportunities for 
optimization, but optimization cannot begin until btrfs is feature 
complete, because any work done beforehand would be wasted effort in 
that it would likely have to be repeated after being broken by feature 
enhancements.  So now it is a waiting game for completion of all the 
major features (like additional RAID levels and possible n-way options, 
etc) before optimization efforts can begin.  Once that happens we will 
likely see HUGE gains in efficiency and speed, but until then we are 
kind of stuck in this position where it "works" but leaves somewhat to 
be desired.  I think this is one reason developers often caution users 
not to expect too much from btrfs at this point.  Its just not there yet 
and it will still be some time yet before it is.