* BTRFS FS corruption
@ 2021-12-29 20:30 Mark Murawski
2021-12-29 23:40 ` Qu Wenruo
2021-12-30 4:46 ` Zygo Blaxell
0 siblings, 2 replies; 3+ messages in thread
From: Mark Murawski @ 2021-12-29 20:30 UTC (permalink / raw)
To: linux-btrfs
Kernel: Debian 5.10.70-1~bpo10+1 (2021-10-10)
# ls -al /mnt/var/lib/postgresql/12/main/pg_stat_tmp/
ls: cannot access
'/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
or directory
ls: cannot access
'/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
or directory
ls: cannot access
'/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
or directory
ls: cannot access
'/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
or directory
ls: cannot access
'/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
or directory
ls: cannot access
'/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
or directory
ls: cannot access
'/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
or directory
ls: cannot access
'/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
or directory
ls: cannot access
'/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
or directory
total 80
drwx------ 1 1001 1001 94 Dec 29 13:52 .
drwx------ 1 1001 1001 552 Dec 29 13:51 ..
-rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
-rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
-rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
-rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
-rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
-rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
-rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
-rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
-rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
-rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
-rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
-rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
-rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
-rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
-rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
-rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
-rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
-rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
-rw------- 1 1001 1001 1864 Dec 29 13:52 db_16391.stat
-rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
-rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
-rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
-rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
-rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
-rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
-rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
-rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
-rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
-rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
-????????? ? ? ? ? ? global.tmp
-????????? ? ? ? ? ? global.tmp
-????????? ? ? ? ? ? global.tmp
-????????? ? ? ? ? ? global.tmp
-????????? ? ? ? ? ? global.tmp
-????????? ? ? ? ? ? global.tmp
-????????? ? ? ? ? ? global.tmp
-????????? ? ? ? ? ? global.tmp
-????????? ? ? ? ? ? global.tmp
This is a dev/test vm so we can do intrusive operations on it...
Should I run btrfsck?
btrfs scrub did not show any errors/fixes and it did not fix the issue.
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: BTRFS FS corruption
2021-12-29 20:30 BTRFS FS corruption Mark Murawski
@ 2021-12-29 23:40 ` Qu Wenruo
2021-12-30 4:46 ` Zygo Blaxell
1 sibling, 0 replies; 3+ messages in thread
From: Qu Wenruo @ 2021-12-29 23:40 UTC (permalink / raw)
To: Mark Murawski, linux-btrfs
On 2021/12/30 04:30, Mark Murawski wrote:
> Kernel: Debian 5.10.70-1~bpo10+1 (2021-10-10)
>
>
> # ls -al /mnt/var/lib/postgresql/12/main/pg_stat_tmp/
> ls: cannot access
> '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
> or directory
> ls: cannot access
> '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
> or directory
> ls: cannot access
> '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
> or directory
> ls: cannot access
> '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
> or directory
> ls: cannot access
> '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
> or directory
> ls: cannot access
> '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
> or directory
> ls: cannot access
> '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
> or directory
> ls: cannot access
> '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
> or directory
> ls: cannot access
> '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp': No such file
> or directory
> total 80
> drwx------ 1 1001 1001 94 Dec 29 13:52 .
> drwx------ 1 1001 1001 552 Dec 29 13:51 ..
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 1864 Dec 29 13:52 db_16391.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
>
> This is a dev/test vm so we can do intrusive operations on it...
Dmesg please.
>
> Should I run btrfsck?
Definitely.
>
> btrfs scrub did not show any errors/fixes and it did not fix the issue.
>
Scrub won't check cross-ref, unlike btrfs-check.
Thanks,
Qu
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: BTRFS FS corruption
2021-12-29 20:30 BTRFS FS corruption Mark Murawski
2021-12-29 23:40 ` Qu Wenruo
@ 2021-12-30 4:46 ` Zygo Blaxell
1 sibling, 0 replies; 3+ messages in thread
From: Zygo Blaxell @ 2021-12-30 4:46 UTC (permalink / raw)
To: Mark Murawski; +Cc: linux-btrfs
On Wed, Dec 29, 2021 at 03:30:46PM -0500, Mark Murawski wrote:
> Kernel: Debian 5.10.70-1~bpo10+1 (2021-10-10)
I've been trying to reproduce the "ghost dirents" bug on 5.10 without
success. It's much easier to hit on 5.14 and 5.15, but apparently not
impossible on 5.10.
> # ls -al /mnt/var/lib/postgresql/12/main/pg_stat_tmp/
> ls: cannot access '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp':
> No such file or directory
> ls: cannot access '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp':
> No such file or directory
> ls: cannot access '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp':
> No such file or directory
> ls: cannot access '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp':
> No such file or directory
> ls: cannot access '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp':
> No such file or directory
> ls: cannot access '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp':
> No such file or directory
> ls: cannot access '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp':
> No such file or directory
> ls: cannot access '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp':
> No such file or directory
> ls: cannot access '/mnt/var/lib/postgresql/12/main/pg_stat_tmp/global.tmp':
> No such file or directory
> total 80
> drwx------ 1 1001 1001 94 Dec 29 13:52 .
> drwx------ 1 1001 1001 552 Dec 29 13:51 ..
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 1526 Dec 29 13:52 db_0.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 0 Dec 29 13:52 db_106964.stat
> -rw------- 1 1001 1001 1864 Dec 29 13:52 db_16391.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -rw------- 1 1001 1001 840 Dec 29 13:52 global.stat
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
> -????????? ? ? ? ? ? global.tmp
This looks like the "ghost dirents" bug I've been chasing for a while.
Some prior discussion at
https://lore.kernel.org/linux-btrfs/CAL3q7H4CYtaW_aEQSEZ_KxZ_ba3u=FmPT8VtXH+OE6FTR8oxOQ@mail.gmail.com/
>
> This is a dev/test vm so we can do intrusive operations on it...
If you can reproduce this reliably, can you apply a kernel patch:
v5.16-rc1: 9a35fc9542fa btrfs: change error handling for btrfs_delete_*_in_log
There's a link to the commit in the kernel.org thread.
> Should I run btrfsck?
Ghost dirents can be removed by the normal 'rm -rf' tool (though it will
complain with errors). You can run btrfs check to confirm that this is
the only problem on the filesystem.
> btrfs scrub did not show any errors/fixes and it did not fix the issue.
Scrub verifies metadata block csums, and there will be no errors in
csums from the ghost dirent bug case, so scrub will not detect or report
ghost dirents.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-12-30 4:46 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-12-29 20:30 BTRFS FS corruption Mark Murawski
2021-12-29 23:40 ` Qu Wenruo
2021-12-30 4:46 ` Zygo Blaxell
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.