* Multiple files with the same name in one directory @ 2021-03-10 17:15 Martin Raiber 2021-03-11 14:43 ` Filipe Manana 2021-03-11 16:47 ` Maksim Fomin 0 siblings, 2 replies; 6+ messages in thread From: Martin Raiber @ 2021-03-10 17:15 UTC (permalink / raw) To: linux-btrfs@vger.kernel.org Hi, I have this in a btrfs directory. Linux kernel 5.10.16, no errors in dmesg, no scrub errors: ls -lh total 19G -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat ... disk_config.dat gets written to using fsync rename ( write new version to disk_config.dat.new, fsync disk_config.dat.new, then rename to disk_config.dat -- it is missing the parent directory fsync). So far no negative consequences... (except that programs might get confused). echo 3 > /proc/sys/vm/drop_caches doesn't help. Regards, Martin Raiber ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Multiple files with the same name in one directory 2021-03-10 17:15 Multiple files with the same name in one directory Martin Raiber @ 2021-03-11 14:43 ` Filipe Manana 2021-03-11 18:05 ` Martin Raiber 2021-03-12 4:02 ` Zygo Blaxell 2021-03-11 16:47 ` Maksim Fomin 1 sibling, 2 replies; 6+ messages in thread From: Filipe Manana @ 2021-03-11 14:43 UTC (permalink / raw) To: Martin Raiber; +Cc: linux-btrfs@vger.kernel.org On Wed, Mar 10, 2021 at 5:18 PM Martin Raiber <martin@urbackup.org> wrote: > > Hi, > > I have this in a btrfs directory. Linux kernel 5.10.16, no errors in dmesg, no scrub errors: > > ls -lh > total 19G > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > ... > > disk_config.dat gets written to using fsync rename ( write new version to disk_config.dat.new, fsync disk_config.dat.new, then rename to disk_config.dat -- it is missing the parent directory fsync). That's interesting. I've just tried something like the following on 5.10.15 (and 5.12-rc2): create disk_config.dat sync for ((i = 0; i < 10; i++)); do create disk_config.dat.new write to disk_config.dat.new fsync disk_config.dat.new mv -f disk_config.dat.new disk_config.dat done <power fail> mount fs list directory I only get one file with the name disk_config.dat and one file with the name disk_config.dat.new. File disk_config.dat has the data written at iteration 9 and disk_config.dat.new has the data written at iteration 10 (expected). You haven't mentioned, but I suppose you had a power failure / unclean shutdown somewhere after an fsync, right? Is this something you can reproduce at will? > > So far no negative consequences... (except that programs might get confused). > > echo 3 > /proc/sys/vm/drop_caches doesn't help. > > Regards, > Martin Raiber > -- Filipe David Manana, “Whether you think you can, or you think you can't — you're right.” ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Multiple files with the same name in one directory 2021-03-11 14:43 ` Filipe Manana @ 2021-03-11 18:05 ` Martin Raiber 2021-03-11 18:12 ` Filipe Manana 2021-03-12 4:02 ` Zygo Blaxell 1 sibling, 1 reply; 6+ messages in thread From: Martin Raiber @ 2021-03-11 18:05 UTC (permalink / raw) To: fdmanana; +Cc: linux-btrfs@vger.kernel.org On 11.03.2021 15:43 Filipe Manana wrote: > On Wed, Mar 10, 2021 at 5:18 PM Martin Raiber <martin@urbackup.org> wrote: >> Hi, >> >> I have this in a btrfs directory. Linux kernel 5.10.16, no errors in dmesg, no scrub errors: >> >> ls -lh >> total 19G >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat >> ... >> >> disk_config.dat gets written to using fsync rename ( write new version to disk_config.dat.new, fsync disk_config.dat.new, then rename to disk_config.dat -- it is missing the parent directory fsync). > That's interesting. > > I've just tried something like the following on 5.10.15 (and 5.12-rc2): > > create disk_config.dat > sync > for ((i = 0; i < 10; i++)); do > create disk_config.dat.new > write to disk_config.dat.new > fsync disk_config.dat.new > mv -f disk_config.dat.new disk_config.dat > done > <power fail> > mount fs > list directory > > I only get one file with the name disk_config.dat and one file with > the name disk_config.dat.new. > File disk_config.dat has the data written at iteration 9 and > disk_config.dat.new has the data written at iteration 10 (expected). > > You haven't mentioned, but I suppose you had a power failure / unclean > shutdown somewhere after an fsync, right? > Is this something you can reproduce at will? I think I rebooted via "echo b > /proc/sysrq-trigger". But at that point it probably didn't write to disk_config.dat anymore (for more than the commit interval). I'm also not sure about the delay of me noticing those multiple files (since it doesn't cause any problems) -- can't reproduce. This is the same machine and file system with ENOSPC in btrfs_async_reclaim_metadata_space -> flush_space -> btrfs_run_delayed_refs. Could be that something went wrong with the error handling/remount-ro w.r.t. to the tree log? > >> So far no negative consequences... (except that programs might get confused). >> >> echo 3 > /proc/sys/vm/drop_caches doesn't help. >> >> Regards, >> Martin Raiber ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Multiple files with the same name in one directory 2021-03-11 18:05 ` Martin Raiber @ 2021-03-11 18:12 ` Filipe Manana 0 siblings, 0 replies; 6+ messages in thread From: Filipe Manana @ 2021-03-11 18:12 UTC (permalink / raw) To: Martin Raiber; +Cc: linux-btrfs@vger.kernel.org On Thu, Mar 11, 2021 at 6:05 PM Martin Raiber <martin@urbackup.org> wrote: > > On 11.03.2021 15:43 Filipe Manana wrote: > > On Wed, Mar 10, 2021 at 5:18 PM Martin Raiber <martin@urbackup.org> wrote: > >> Hi, > >> > >> I have this in a btrfs directory. Linux kernel 5.10.16, no errors in dmesg, no scrub errors: > >> > >> ls -lh > >> total 19G > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > >> ... > >> > >> disk_config.dat gets written to using fsync rename ( write new version to disk_config.dat.new, fsync disk_config.dat.new, then rename to disk_config.dat -- it is missing the parent directory fsync). > > That's interesting. > > > > I've just tried something like the following on 5.10.15 (and 5.12-rc2): > > > > create disk_config.dat > > sync > > for ((i = 0; i < 10; i++)); do > > create disk_config.dat.new > > write to disk_config.dat.new > > fsync disk_config.dat.new > > mv -f disk_config.dat.new disk_config.dat > > done > > <power fail> > > mount fs > > list directory > > > > I only get one file with the name disk_config.dat and one file with > > the name disk_config.dat.new. > > File disk_config.dat has the data written at iteration 9 and > > disk_config.dat.new has the data written at iteration 10 (expected). > > > > You haven't mentioned, but I suppose you had a power failure / unclean > > shutdown somewhere after an fsync, right? > > Is this something you can reproduce at will? > > I think I rebooted via "echo b > /proc/sysrq-trigger". But at that point it probably didn't write to disk_config.dat anymore (for more than the commit interval). I'm also not sure about the delay of me noticing those multiple files (since it doesn't cause any problems) -- can't reproduce. > > This is the same machine and file system with ENOSPC in btrfs_async_reclaim_metadata_space -> flush_space -> btrfs_run_delayed_refs. Could be that something went wrong with the error handling/remount-ro w.r.t. to the tree log? It could be a bug between aborting a transaction (the error handling/remount ro) and an fsync in progress yes. I'll have to look at that and see if it's possible. Thanks for the report. > > > > >> So far no negative consequences... (except that programs might get confused). > >> > >> echo 3 > /proc/sys/vm/drop_caches doesn't help. > >> > >> Regards, > >> Martin Raiber > > -- Filipe David Manana, “Whether you think you can, or you think you can't — you're right.” ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Multiple files with the same name in one directory 2021-03-11 14:43 ` Filipe Manana 2021-03-11 18:05 ` Martin Raiber @ 2021-03-12 4:02 ` Zygo Blaxell 1 sibling, 0 replies; 6+ messages in thread From: Zygo Blaxell @ 2021-03-12 4:02 UTC (permalink / raw) To: Filipe Manana; +Cc: Martin Raiber, linux-btrfs@vger.kernel.org On Thu, Mar 11, 2021 at 02:43:26PM +0000, Filipe Manana wrote: > On Wed, Mar 10, 2021 at 5:18 PM Martin Raiber <martin@urbackup.org> wrote: > > > > Hi, > > > > I have this in a btrfs directory. Linux kernel 5.10.16, no errors in dmesg, no scrub errors: > > > > ls -lh > > total 19G > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > > ... > > > > disk_config.dat gets written to using fsync rename ( write new version to disk_config.dat.new, fsync disk_config.dat.new, then rename to disk_config.dat -- it is missing the parent directory fsync). > > That's interesting. > > I've just tried something like the following on 5.10.15 (and 5.12-rc2): > > create disk_config.dat > sync > for ((i = 0; i < 10; i++)); do > create disk_config.dat.new > write to disk_config.dat.new > fsync disk_config.dat.new > mv -f disk_config.dat.new disk_config.dat > done > <power fail> > mount fs > list directory > > I only get one file with the name disk_config.dat and one file with > the name disk_config.dat.new. > File disk_config.dat has the data written at iteration 9 and > disk_config.dat.new has the data written at iteration 10 (expected). > > You haven't mentioned, but I suppose you had a power failure / unclean > shutdown somewhere after an fsync, right? > Is this something you can reproduce at will? I've seen this off and on as far back as kernel 4.4. Usually (only?) happens on machines with metadata-heavy write loads at the time of the fsync. Very easy to reproduce if you hit a transaction deadlock bug (which happened to be a lot more common a few years ago), these inodeless filenames would show up almost every time if fsync was called before rename. Somewhere between 4.9 and 4.14, it became possible to create new files with the same names as the inodeless filenames, and the bug became much less annoying (the filenames would even go away by themselves sometimes). Then the bug became harder to spot--earler versions of the bug would stop an application dead with a filename that could never be used again. Later versions didn't break anything, so they could only be found by looking for errors when running e.g. 'find' or 'rsync'. The issue never got near the top of my bug list, and until today I thought it had been quietly fixed before 5.1. I guess it's still active, or a new bug has replaced it. The last instance I recorded of this issue was found on a filesystem in late 2019, although I don't know which kernel was running at the time the inodeless filename was created (could have been anything between 4.4 and 5.0, though probably not before 4.14 because I would have been forced to burn the subvol if it happened on an earlier kernel). > > So far no negative consequences... (except that programs might get confused). > > > > echo 3 > /proc/sys/vm/drop_caches doesn't help. > > > > Regards, > > Martin Raiber > > > > > -- > Filipe David Manana, > > “Whether you think you can, or you think you can't — you're right.” ^ permalink raw reply [flat|nested] 6+ messages in thread
* Multiple files with the same name in one directory 2021-03-10 17:15 Multiple files with the same name in one directory Martin Raiber 2021-03-11 14:43 ` Filipe Manana @ 2021-03-11 16:47 ` Maksim Fomin 1 sibling, 0 replies; 6+ messages in thread From: Maksim Fomin @ 2021-03-11 16:47 UTC (permalink / raw) To: linux-btrfs@vger.kernel.org ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, 10 March 2021 г., 20:15, Martin Raiber <martin@urbackup.org> wrote: > Hi, > > I have this in a btrfs directory. Linux kernel 5.10.16, no errors in dmesg, no scrub errors: > > ls -lh > total 19G > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > -rwxr-x--- 1 root root 783 Mar 10 14:56 disk_config.dat > ... > > disk_config.dat gets written to using fsync rename ( write new version to disk_config.dat.new, fsync disk_config.dat.new, then rename to disk_config.dat -- it is missing the parent directory fsync). > > So far no negative consequences... (except that programs might get confused). > > echo 3 > /proc/sys/vm/drop_caches doesn't help. > > Regards, > Martin Raiber I can say that approx 1 month ago I also was surprised by having several files with duplicate names (listed by pcmanfm file manager and ls in console). In my case I had duplicates for several different files in one folder, not 10 copies of one file. I resolved the problem by renaming files and deleting duplicates. I decided not to report that issue because I thought I have unintentionally done something crazy. Btw, they likely occured also after power loss. Regards, Maxim Fomin ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-03-12 4:03 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-03-10 17:15 Multiple files with the same name in one directory Martin Raiber 2021-03-11 14:43 ` Filipe Manana 2021-03-11 18:05 ` Martin Raiber 2021-03-11 18:12 ` Filipe Manana 2021-03-12 4:02 ` Zygo Blaxell 2021-03-11 16:47 ` Maksim Fomin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox