Re: question to btrfs scrub

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

On 2023/7/5 23:01, Bernd Lentes wrote:
>> -----Original Message-----
>> From: Qu Wenruo <wqu@suse.com>
>> Sent: Wednesday, July 5, 2023 10:46 AM
>> To: Bernd Lentes <bernd.lentes@helmholtz-muenchen.de>; Qu Wenruo
>> <quwenruo.btrfs@gmx.com>; linux-btrfs <linux-btrfs@vger.kernel.org>
>> Subject: Re: question to btrfs scrub
>> That's why it doesn't report the csum error, and we need "--check-data-
>> csum"
>> to verify data csum.
>>>
>>> OK. I started it. But isn't that the same as "btrfs scrub" ?
>>> The man page gives a hint:
>>> --check-data-csum
>>> verify checksums of data blocks
>>> This expects that the filesystem is otherwise OK, and is basically an offline
>>> scrub that does not repair data from spare copies.
>
>> Btrfs check has way more comprehensive checks on metadata, but it by
>> default not check data csums.
>
>> Which is quite the opposite of btrfs scrub, which only checks data csum, and
>> metadata checks are very lightweight.
>
>> The main thing here is, nodatacow implies nodatacsum, thus it would not
>> generate any csum nor verify it.
>
> But aren't checksums important in case of errors ?

Checksum have two functions:

- Detect errors
   As long as it's not a false alert, then it makes sense.
   But please keep in mind that, a csum mismatch will prevent anyone
   from reading that corrupted data, whether this is the expected
   behavior really depends on your workload.

   E.g. if some archive with built-in error detect and recovery (like RAR
   files), it's definitely not a good idea to return -EIO for the whole
   block, other than reading out the corrupted data and let the software
   to handle them.

- Recover the good data from the extra mirrors
   This only works if you're using profiles with duplication (DUP,
   RAID1*, RAID10, RAID5, RAID6).
   Otherwise btrfs won't be able to recover anything.

In your case, since you're already using LVM, thus I believe the fs is
using default profiles (DUP for meta, SINGLE for data), thus there would
be no extra copy to recover from.

So there is really error detection functionality lost if go nodatasum.

> OK. I know which VM images produced checksum errors. I delete them and restore them from the backup.

You mentioned snapshots are utilized for those images, thus you have to
delete all the involved files, including ones in the snapshots.

> Then I set the attribute for the directory.
>
> OK ?
>
> Here my output from the btrfs check:
> ha-idg-1:~ # btrfs check -p --check-data-csum /dev/vg_san/lv_domains
> Opening filesystem to check...
> Checking filesystem on /dev/vg_san/lv_domains
> UUID: bbcfa007-fb2b-432a-b513-207d5df35a2a
> [1/7] checking root items                      (0:00:35 elapsed, 6900134 items checked)
> [2/7] checking extents                         (0:01:58 elapsed, 484995 items checked)
> [3/7] checking free space cache                (0:00:14 elapsed, 5184 items checked)
> [4/7] checking fs roots                        (0:02:38 elapsed, 65549 items checked)
> mirror 1 bytenr 1489997918208 csum 0x0e45a39c expected csum 0xaa326ac93 items checked)
> mirror 1 bytenr 2036881817600 csum 0x714ca3eb expected csum 0x2cf56c3f9 items checked)
> mirror 1 bytenr 2708733394944 csum 0xa5bc757d expected csum 0x7055fdf48 items checked)
> mirror 1 bytenr 2743035994112 csum 0x743f7516 expected csum 0x2f21def46 items checked)
> mirror 1 bytenr 2855677394944 csum 0x50cbb065 expected csum 0xfb923a738 items checked)
> mirror 1 bytenr 4354053398528 csum 0x62eba801 expected csum 0x879bde4a0 items checked)
> mirror 1 bytenr 4355127242752 csum 0x71594d4c expected csum 0x879bde4a1 items checked)
> mirror 1 bytenr 4359422152704 csum 0x71594d4c expected csum 0x879bde4a6 items checked)
>   ...
> mirror 1 bytenr 5227759489024 csum 0x92051821 expected csum 0xa61efb7f89 items checked)
> mirror 1 bytenr 5229552353280 csum 0x26981ed8 expected csum 0xad21497c44 items checked)
> mirror 1 bytenr 5229990215680 csum 0x27f66fb8 expected csum 0x746827b001 items checked)
> mirror 1 bytenr 5231527952384 csum 0x9cfa691f expected csum 0x2847c53493 items checked)
> mirror 1 bytenr 5233822765056 csum 0xe8072b89 expected csum 0xaf60264806 items checked)
> mirror 1 bytenr 5234632364032 csum 0xfd83365b expected csum 0x26dc10d424 items checked)
> [5/7] checking csums against data              (3:54:58 elapsed, 3236328 items checked)
> ERROR: errors found in csum tree
> [6/7] checking root refs                       (0:00:00 elapsed, 13 items checked)
> [7/7] checking quota groups skipped (not enabled on this FS)
> found 5396770611200 bytes used, error(s) found
> total csum bytes: 5263001424
> total tree bytes: 7945863168
> total fs tree bytes: 1077493760
> total extent tree bytes: 828391424
> btree space waste bytes: 1124143787
> file data blocks allocated: 127704684556288
>   referenced 8084906622976
>
> So it finds errors in the data csum, correct ?

Yes, either there are some files not deleted, or the file is snapshoted.

You'll need to resolve the file, either through scrub (and check dmesg
of the file names), or go with "btrfs inspect logical -o <bytenr like
2743035994112> <mount point>" to find the files.

Otherwise your fs is completely fine.

Thanks,
Qu

> Thanks.
>
> Bernd
>
>
>
> Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
> Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de
> Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp, Daniela Sommer (komm.) | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling
> Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671