Re: [PATCH 0/5] btrfs: scrub: improve the scrub performance

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

On 2023/8/2 10:20, Qu Wenruo wrote:
> 
> 
> On 2023/8/2 10:15, Jani Partanen wrote:
>>
>> On 02/08/2023 4.56, Qu Wenruo wrote:
>>>
>>> So the btrfs scrub report is doing the correct report using the values
>>> from kernel.
>>>
>>> And considering the used space is around 600G, divided by 4 disks (aka,
>>> 3 data stripes + 1 parity stripes), it's not that weird, as we would got
>>> around 200G per device (parity doesn't contribute to the scrubbed 
>>> bytes).
>>>
>>> Especially considering your metadata is RAID1C4, it means we should only
>>> have more than 200G.
>>> Instead it's the old report of less than 200G doesn't seem correct.
>>>
>>> Mind to provide the output of "btrfs fi usage <mnt>" to verify my
>>> assumption?
>>>
>> btrfs fi usage /mnt/
>> Overall:
>>      Device size:                   1.16TiB
>>      Device allocated:            844.25GiB
>>      Device unallocated:          348.11GiB
>>      Device missing:                  0.00B
>>      Device slack:                    0.00B
>>      Used:                        799.86GiB
>>      Free (estimated):            289.58GiB      (min: 115.52GiB)
>>      Free (statfs, df):           289.55GiB
>>      Data ratio:                       1.33
>>      Metadata ratio:                   4.00
>>      Global reserve:              471.80MiB      (used: 0.00B)
>>      Multiple profiles:                  no
>>
>> Data,RAID5: Size:627.00GiB, Used:598.51GiB (95.46%)
>>     /dev/sdb      209.00GiB
>>     /dev/sdc      209.00GiB
>>     /dev/sdd      209.00GiB
>>     /dev/sde      209.00GiB
> 
> OK, my previous calculation is incorrect...
> 
> For each device there should be 209GiB used by RAID5 chunks, and only 
> 3/4 of them contributes to the scrubbed data bytes.
> 
> Thus there seems to be some double accounting.
> 
> Definitely needs extra digging for this situation.

Well, this turns out to be something related to the patchset.

If you don't apply the patchset, the reporting is correct.

The problem is in the last patch, which calls 
scrub_stripe_report_errors() twice, thus double accounting the values.

I'll fix it soon.

Thanks for spotting this one!
Qu

> 
> Thanks,
> Qu
> 
>>
>> Metadata,RAID1C4: Size:2.00GiB, Used:472.56MiB (23.07%)
>>     /dev/sdb        2.00GiB
>>     /dev/sdc        2.00GiB
>>     /dev/sdd        2.00GiB
>>     /dev/sde        2.00GiB
>>
>> System,RAID1C4: Size:64.00MiB, Used:64.00KiB (0.10%)
>>     /dev/sdb       64.00MiB
>>     /dev/sdc       64.00MiB
>>     /dev/sdd       64.00MiB
>>     /dev/sde       64.00MiB
>>
>> Unallocated:
>>     /dev/sdb       87.03GiB
>>     /dev/sdc       87.03GiB
>>     /dev/sdd       87.03GiB
>>
>>     /dev/sde       87.03GiB
>>
>>
>> There is 1 extra 2GB file now so thats why it show little more usage now.
>>
>>
>>> Sure, I'll CC you when refreshing the patchset, extra tests are always
>>> appreciated.
>>>
>> Sound good, thanks!
>>