From: Russell Haley <yumpusamongus@gmail.com>
To: Qu Wenruo <quwenruo.btrfs@gmx.com>, linux-btrfs@vger.kernel.org
Subject: Re: BTRFS corruption after conversion to block group tree
Date: Fri, 1 Dec 2023 20:45:18 -0600 [thread overview]
Message-ID: <1b32a750-d464-49f1-a288-577ee2fd473e@gmail.com> (raw)
In-Reply-To: <e20c7d59-c460-4236-9771-9cb4a3f9dfb2@gmx.com>
[-- Attachment #1: Type: text/plain, Size: 2467 bytes --]
On 12/1/23 14:09, Qu Wenruo wrote:
>
>
> On 2023/12/1 22:16, Russell Haley wrote:
>> Distro: Fedora 39
>> kernel: 6.5.12
>> btrfs-progs: v6.5.1.
>>
>> To put everyone at ease, the data seems mostly okay. The disk is
>> mountable with ro,rescue=ingorebadroots:nologreplay, and I'm pulling off
>> what I can. Furthermore, there's a good chance this is the result of a
>> marginally-unstable overclock. However, the symptoms are very similar to
>> a recent report from Alexander Duscheleit [1], so there may be a deeper
>> problem. I can provide requested metadata dumps if it would be useful,
>> and I also have a few questions.
>>
>> A few days ago I used btrfstune to convert two btrfs filesystems to
>> block-group-tree, after reading that it would reduce mount time. After
>> about 10 hours, while I was using Steam, the more heavily-used of the
>> two disks, which holds (among other things) the Steam game library,
>> experienced a transaction abort and went read-only.
>
> Any dmesg of that transaction abort? That's the most critical info here.
[80343.530191] BTRFS error (device dm-1): parent transid verify failed
on logical 31850496 mirror 1 wanted 123883 found 123907
[80343.587776] BTRFS error (device dm-1): parent transid verify failed
on logical 31850496 mirror 2 wanted 123883 found 123907
[80343.587836] BTRFS error (device dm-1): failed to run delayed ref for
logical 916815872 num_bytes 16384 type 176 action 1 ref_mod 1: -5
[80343.587844] BTRFS error (device dm-1: state A): Transaction aborted
(error -5)
[80343.587846] BTRFS: error (device dm-1: state A) in
btrfs_run_delayed_refs:2182: errno=-5 IO failure
>> 2. There's something extremely weird with the primary superblock. "btrfs
>> inspect-internal dump-super" says that superblock 0 doesn't have the
>> BLOCK_GROUP_TREE or NO_HOLES flags set, unlike superblocks 1 and 2,
>> which are identical in every field except csum ("[match]", for 0, 1,
>> and 2) and bytenr.
>
> This is not good at all, if csum of super blocks doesn't match, there
> must be something especially wrong.
Comparing to other btrfs disks I have access to, different checksum for
the backup superblocks seems normal? It does say [match] after the
checksum value. Just the value itself is different.
I dumped the 3 superblocks with btrfs inspect-internal dump-super -f and
attached them to this message. "Meld" for 3-way diff worked well for me.
Thanks for your help,
Russell Haley
[-- Attachment #2: super-0.txt --]
[-- Type: text/plain, Size: 3376 bytes --]
superblock: bytenr=65536, device=/dev/mapper/3tb_spinner
---------------------------------------------------------
csum_type 0 (crc32c)
csum_size 4
csum 0x404bfde3 [match]
bytenr 65536
flags 0x1
( WRITTEN )
magic _BHRfS_M [match]
fsid 8cf4bcd8-76c7-4494-a2c8-2578136c0aa6
metadata_uuid 00000000-0000-0000-0000-000000000000
label
generation 123887
root 875036672
sys_array_size 129
chunk_root_generation 123187
root_level 0
chunk_root 28459008
chunk_root_level 1
log_root 920076288
log_root_transid (deprecated) 0
log_root_level 0
total_bytes 3000589352960
bytes_used 1951818739712
sectorsize 4096
nodesize 16384
leafsize (deprecated) 16384
stripesize 4096
root_dir 6
num_devices 1
compat_flags 0x0
compat_ro_flags 0x3
( FREE_SPACE_TREE |
FREE_SPACE_TREE_VALID )
incompat_flags 0x171
( MIXED_BACKREF |
COMPRESS_ZSTD |
BIG_METADATA |
EXTENDED_IREF |
SKINNY_METADATA )
cache_generation 0
uuid_tree_generation 123887
dev_item.uuid ba0038ab-68b2-4646-8f30-7d40dbd1b4ab
dev_item.fsid 8cf4bcd8-76c7-4494-a2c8-2578136c0aa6 [match]
dev_item.type 0
dev_item.total_bytes 3000589352960
dev_item.bytes_used 1981078831104
dev_item.io_align 4096
dev_item.io_width 4096
dev_item.sector_size 4096
dev_item.devid 1
dev_item.dev_group 0
dev_item.seek_speed 0
dev_item.bandwidth 0
dev_item.generation 0
sys_chunk_array[2048]:
item 0 key (FIRST_CHUNK_TREE CHUNK_ITEM 22020096)
length 8388608 owner 2 stripe_len 65536 type SYSTEM|DUP
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 1
stripe 0 devid 1 offset 22020096
dev_uuid ba0038ab-68b2-4646-8f30-7d40dbd1b4ab
stripe 1 devid 1 offset 30408704
dev_uuid ba0038ab-68b2-4646-8f30-7d40dbd1b4ab
backup_roots[4]:
backup 0:
backup_tree_root: 287932416 gen: 123885 level: 0
backup_chunk_root: 28459008 gen: 123187 level: 1
backup_extent_root: 203538432 gen: 123885 level: 2
backup_fs_root: 185040896 gen: 123885 level: 3
backup_dev_root: 146063360 gen: 123884 level: 1
csum_root: 185630720 gen: 123885 level: 2
backup_total_bytes: 3000589352960
backup_bytes_used: 1951818743808
backup_num_devices: 1
backup 1:
backup_tree_root: 805404672 gen: 123886 level: 0
backup_chunk_root: 28459008 gen: 123187 level: 1
backup_extent_root: 458981376 gen: 123886 level: 2
backup_fs_root: 291143680 gen: 123886 level: 3
backup_dev_root: 146063360 gen: 123884 level: 1
csum_root: 185630720 gen: 123885 level: 2
backup_total_bytes: 3000589352960
backup_bytes_used: 1951818743808
backup_num_devices: 1
backup 2:
backup_tree_root: 875036672 gen: 123887 level: 0
backup_chunk_root: 28459008 gen: 123187 level: 1
backup_extent_root: 806912000 gen: 123887 level: 2
backup_fs_root: 805961728 gen: 123887 level: 3
backup_dev_root: 146063360 gen: 123884 level: 1
csum_root: 806600704 gen: 123887 level: 2
backup_total_bytes: 3000589352960
backup_bytes_used: 1951818739712
backup_num_devices: 1
backup 3:
backup_tree_root: 146046976 gen: 123884 level: 0
backup_chunk_root: 28459008 gen: 123187 level: 1
backup_extent_root: 44793856 gen: 123884 level: 2
backup_fs_root: 185040896 gen: 123885 level: 3
backup_dev_root: 146063360 gen: 123884 level: 1
csum_root: 36765696 gen: 123884 level: 2
backup_total_bytes: 3000589352960
backup_bytes_used: 1951779827712
backup_num_devices: 1
[-- Attachment #3: super-1.txt --]
[-- Type: text/plain, Size: 3432 bytes --]
superblock: bytenr=67108864, device=/dev/mapper/3tb_spinner
---------------------------------------------------------
csum_type 0 (crc32c)
csum_size 4
csum 0x076d5bba [match]
bytenr 67108864
flags 0x1
( WRITTEN )
magic _BHRfS_M [match]
fsid 8cf4bcd8-76c7-4494-a2c8-2578136c0aa6
metadata_uuid 00000000-0000-0000-0000-000000000000
label
generation 123917
root 1871598567424
sys_array_size 129
chunk_root_generation 123187
root_level 0
chunk_root 28459008
chunk_root_level 1
log_root 0
log_root_transid (deprecated) 0
log_root_level 0
total_bytes 3000589352960
bytes_used 1951820050432
sectorsize 4096
nodesize 16384
leafsize (deprecated) 16384
stripesize 4096
root_dir 6
num_devices 1
compat_flags 0x0
compat_ro_flags 0xb
( FREE_SPACE_TREE |
FREE_SPACE_TREE_VALID |
BLOCK_GROUP_TREE )
incompat_flags 0x371
( MIXED_BACKREF |
COMPRESS_ZSTD |
BIG_METADATA |
EXTENDED_IREF |
SKINNY_METADATA |
NO_HOLES )
cache_generation 0
uuid_tree_generation 123887
dev_item.uuid ba0038ab-68b2-4646-8f30-7d40dbd1b4ab
dev_item.fsid 8cf4bcd8-76c7-4494-a2c8-2578136c0aa6 [match]
dev_item.type 0
dev_item.total_bytes 3000589352960
dev_item.bytes_used 1981078831104
dev_item.io_align 4096
dev_item.io_width 4096
dev_item.sector_size 4096
dev_item.devid 1
dev_item.dev_group 0
dev_item.seek_speed 0
dev_item.bandwidth 0
dev_item.generation 0
sys_chunk_array[2048]:
item 0 key (FIRST_CHUNK_TREE CHUNK_ITEM 22020096)
length 8388608 owner 2 stripe_len 65536 type SYSTEM|DUP
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 1
stripe 0 devid 1 offset 22020096
dev_uuid ba0038ab-68b2-4646-8f30-7d40dbd1b4ab
stripe 1 devid 1 offset 30408704
dev_uuid ba0038ab-68b2-4646-8f30-7d40dbd1b4ab
backup_roots[4]:
backup 0:
backup_tree_root: 1871598567424 gen: 123917 level: 0
backup_chunk_root: 28459008 gen: 123187 level: 1
backup_extent_root: 30507008 gen: 123917 level: 2
backup_fs_root: 805961728 gen: 123887 level: 3
backup_dev_root: 146063360 gen: 123884 level: 1
csum_root: 806600704 gen: 123887 level: 2
backup_total_bytes: 3000589352960
backup_bytes_used: 1951820050432
backup_num_devices: 1
backup 1:
backup_tree_root: 1871599681536 gen: 123914 level: 0
backup_chunk_root: 28459008 gen: 123187 level: 1
backup_extent_root: 30457856 gen: 123914 level: 2
backup_fs_root: 805961728 gen: 123887 level: 3
backup_dev_root: 146063360 gen: 123884 level: 1
csum_root: 806600704 gen: 123887 level: 2
backup_total_bytes: 3000589352960
backup_bytes_used: 1951820050432
backup_num_devices: 1
backup 2:
backup_tree_root: 1871600844800 gen: 123915 level: 0
backup_chunk_root: 28459008 gen: 123187 level: 1
backup_extent_root: 30507008 gen: 123915 level: 2
backup_fs_root: 805961728 gen: 123887 level: 3
backup_dev_root: 146063360 gen: 123884 level: 1
csum_root: 806600704 gen: 123887 level: 2
backup_total_bytes: 3000589352960
backup_bytes_used: 1951820050432
backup_num_devices: 1
backup 3:
backup_tree_root: 1871602515968 gen: 123916 level: 0
backup_chunk_root: 28459008 gen: 123187 level: 1
backup_extent_root: 30457856 gen: 123916 level: 2
backup_fs_root: 805961728 gen: 123887 level: 3
backup_dev_root: 146063360 gen: 123884 level: 1
csum_root: 806600704 gen: 123887 level: 2
backup_total_bytes: 3000589352960
backup_bytes_used: 1951820050432
backup_num_devices: 1
[-- Attachment #4: super-2.txt --]
[-- Type: text/plain, Size: 3440 bytes --]
superblock: bytenr=274877906944, device=/dev/mapper/3tb_spinner
---------------------------------------------------------
csum_type 0 (crc32c)
csum_size 4
csum 0xfaea0d8b [match]
bytenr 274877906944
flags 0x1
( WRITTEN )
magic _BHRfS_M [match]
fsid 8cf4bcd8-76c7-4494-a2c8-2578136c0aa6
metadata_uuid 00000000-0000-0000-0000-000000000000
label
generation 123917
root 1871598567424
sys_array_size 129
chunk_root_generation 123187
root_level 0
chunk_root 28459008
chunk_root_level 1
log_root 0
log_root_transid (deprecated) 0
log_root_level 0
total_bytes 3000589352960
bytes_used 1951820050432
sectorsize 4096
nodesize 16384
leafsize (deprecated) 16384
stripesize 4096
root_dir 6
num_devices 1
compat_flags 0x0
compat_ro_flags 0xb
( FREE_SPACE_TREE |
FREE_SPACE_TREE_VALID |
BLOCK_GROUP_TREE )
incompat_flags 0x371
( MIXED_BACKREF |
COMPRESS_ZSTD |
BIG_METADATA |
EXTENDED_IREF |
SKINNY_METADATA |
NO_HOLES )
cache_generation 0
uuid_tree_generation 123887
dev_item.uuid ba0038ab-68b2-4646-8f30-7d40dbd1b4ab
dev_item.fsid 8cf4bcd8-76c7-4494-a2c8-2578136c0aa6 [match]
dev_item.type 0
dev_item.total_bytes 3000589352960
dev_item.bytes_used 1981078831104
dev_item.io_align 4096
dev_item.io_width 4096
dev_item.sector_size 4096
dev_item.devid 1
dev_item.dev_group 0
dev_item.seek_speed 0
dev_item.bandwidth 0
dev_item.generation 0
sys_chunk_array[2048]:
item 0 key (FIRST_CHUNK_TREE CHUNK_ITEM 22020096)
length 8388608 owner 2 stripe_len 65536 type SYSTEM|DUP
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 1
stripe 0 devid 1 offset 22020096
dev_uuid ba0038ab-68b2-4646-8f30-7d40dbd1b4ab
stripe 1 devid 1 offset 30408704
dev_uuid ba0038ab-68b2-4646-8f30-7d40dbd1b4ab
backup_roots[4]:
backup 0:
backup_tree_root: 1871598567424 gen: 123917 level: 0
backup_chunk_root: 28459008 gen: 123187 level: 1
backup_extent_root: 30507008 gen: 123917 level: 2
backup_fs_root: 805961728 gen: 123887 level: 3
backup_dev_root: 146063360 gen: 123884 level: 1
csum_root: 806600704 gen: 123887 level: 2
backup_total_bytes: 3000589352960
backup_bytes_used: 1951820050432
backup_num_devices: 1
backup 1:
backup_tree_root: 1871599681536 gen: 123914 level: 0
backup_chunk_root: 28459008 gen: 123187 level: 1
backup_extent_root: 30457856 gen: 123914 level: 2
backup_fs_root: 805961728 gen: 123887 level: 3
backup_dev_root: 146063360 gen: 123884 level: 1
csum_root: 806600704 gen: 123887 level: 2
backup_total_bytes: 3000589352960
backup_bytes_used: 1951820050432
backup_num_devices: 1
backup 2:
backup_tree_root: 1871600844800 gen: 123915 level: 0
backup_chunk_root: 28459008 gen: 123187 level: 1
backup_extent_root: 30507008 gen: 123915 level: 2
backup_fs_root: 805961728 gen: 123887 level: 3
backup_dev_root: 146063360 gen: 123884 level: 1
csum_root: 806600704 gen: 123887 level: 2
backup_total_bytes: 3000589352960
backup_bytes_used: 1951820050432
backup_num_devices: 1
backup 3:
backup_tree_root: 1871602515968 gen: 123916 level: 0
backup_chunk_root: 28459008 gen: 123187 level: 1
backup_extent_root: 30457856 gen: 123916 level: 2
backup_fs_root: 805961728 gen: 123887 level: 3
backup_dev_root: 146063360 gen: 123884 level: 1
csum_root: 806600704 gen: 123887 level: 2
backup_total_bytes: 3000589352960
backup_bytes_used: 1951820050432
backup_num_devices: 1
next prev parent reply other threads:[~2023-12-02 2:45 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-01 11:46 BTRFS corruption after conversion to block group tree Russell Haley
2023-12-01 20:09 ` Qu Wenruo
2023-12-02 2:45 ` Russell Haley [this message]
2023-12-02 4:13 ` Qu Wenruo
2023-12-02 4:25 ` Qu Wenruo
2023-12-02 6:13 ` Russell Haley
2023-12-02 6:43 ` Qu Wenruo
2023-12-02 8:04 ` Russell Haley
2023-12-02 9:00 ` Qu Wenruo
2023-12-03 12:05 ` Russell Haley
2023-12-03 12:16 ` Russell Haley
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1b32a750-d464-49f1-a288-577ee2fd473e@gmail.com \
--to=yumpusamongus@gmail.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=quwenruo.btrfs@gmx.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox