btrfs replace performance with missing drive

["Sébastien Luttringer" <seblu@seblu.net>]

[-- Attachment #1: Type: text/plain, Size: 2775 bytes --]

Hello,

I have a performance issue with «btrfs replace» with raid5 and a _missing_
device. My btrfs rely on 6x4TB HDD and the operating system is an Archlinux.

In a nutshell, I will need 23 to 46 days to replace on missing disk.

# btrfs fi sh /home
Label: 'raptor.home'  uuid: 8739c8b2-110b-44ac-8b4d-285ad06ee446
        Total devices 7 FS bytes used 14.60TiB
        devid    0 size 3.64TiB used 2.80TiB path /dev/sdf
        devid    3 size 3.64TiB used 2.97TiB path /dev/sdh
        devid    5 size 3.64TiB used 2.97TiB path /dev/sdc
        devid    6 size 3.64TiB used 2.97TiB path /dev/sdd
        devid    7 size 3.64TiB used 2.97TiB path /dev/sde
        devid    8 size 3.64TiB used 2.97TiB path /dev/sdg
        *** Some devices missing


At a disk full speed (100 MB/s) replacing the missing disk (4 TB) should take
around 8 hours. With the same disk model and same HBA card in another computer
with a mdadm/raid5, I could verify this duration could be reach.

I also tested a «btrfs replace» without a missing disk and the speed was not so
bad. Somewhere around half disk speed (50-60MB/s). Performances are under
mdadm.

But, in my case, the drive is pass away, I can't use it as source of the
replace, so I have a replace speed of 1-2MB/s ! Which mean between 23-46 days
with bad usage performance and security risk.

I tried to upgrade the kernel to the latest (4.7-rc6) but it's not better in
performance. I got some crash during replace with 4.6.0 which vanish with the
last rc.

# iostat -md  
Linux 4.7.0-rc6-seblu
(raptor.seblu.net)        14/07/2016      _x86_64_        (4 CPU)

Device:            tps    MB_read/s    MB_wrtn/s    MB_read    MB_wrtn
sdc             356,75        22,51         0,14    9132054      58427
sdd             356,27        22,51         0,14    9131612      57094
sde             361,53        22,52         0,14    9132207      57245
sdf             362,78         0,00         1,81          4     735786
sdg             357,82        22,51         0,14    9131763      58323
sdh             325,25        22,52         0,14    9132715      58355


So I have a really poor performance in rebuilding a raid5 mostly when the
replaced device is missing.
Is there a parameter to tweak of something I can do to improve the replace ? 

Regards,

-- 
Sébastien "Seblu" Luttringer
https://seblu.net | Twitter: @seblu42
GPG: 0x2072D77A

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 811 bytes --]