From: Filipe Manana <fdmanana@kernel.org>
To: Evangelos Foutras <evangelos@foutras.com>
Cc: Ian Johnson <ian@ianjohnson.dev>, linux-btrfs@vger.kernel.org
Subject: Re: Possible readdir regression with BTRFS
Date: Sat, 9 Sep 2023 13:18:49 +0100 [thread overview]
Message-ID: <ZPxiqYCeMb6vOjw9@debian0.Home> (raw)
In-Reply-To: <00ed09b9-d60c-4605-b3b6-f4e79bf92fca@foutras.com>
On Sat, Sep 09, 2023 at 02:52:19PM +0300, Evangelos Foutras wrote:
> Hi Filipe,
>
> Please be aware that this bug might not be as harmless as it seems. I'm not
> sure if the fix you're preparing would also fix an issue we saw at Arch
> Linux but I thought I'd mention it here.
>
> We have a package repository server with 4x10 TB drives in RAID10 (btrfs
> only, no mdadm). On multiple mirrors syncing from it we have seen rsync
> occasionally delete ~4 small (<10 MB) files that get frequently updated by
> renaming temporary files into them. This only happened with 6.4.12 and went
> away after going back to 6.4.10 (the former had the commit Ian mentioned).
>
> Unfortunately I don't have a reproducer for this. I can only describe what
> our repo-add script does and how rsync behaves during problematic syncs.
>
> Our repo-add script frequently adds packages to the extra repo by doing:
>
> ln -f extra.db.tar.gz extra.db.tar.gz.old
> mv .tmp.extra.db.tar.gz extra.db.tar.gz
>
> And the same for extra.files.tar.gz:
>
> ln -f extra.files.tar.gz extra.files.tar.gz.old
> mv .tmp.extra.files.tar.gz extra.files.tar.gz
>
> While the server was running Linux 6.4.12, rsync on some mirrors would
> occasionally (3-4 times in the day) delete these files:
>
> deleting extra/os/x86_64/extra.files.tar.gz.old
> deleting extra/os/x86_64/extra.files.tar.gz
> deleting extra/os/x86_64/extra.db.tar.gz.old
> deleting extra/os/x86_64/extra.db.tar.gz
>
> Since renames are atomic, I would expect this scenario to never happen.
>
> Again, sorry for not being able to provide a proper reproducer like Ian;
> there is probably some timing interaction with how rsync does directory
> scanning and repo-add updating the directory entry during this time.
No worries, I've just sent a patchset with 2 patches:
https://lore.kernel.org/linux-btrfs/cover.1694260751.git.fdmanana@suse.com/
I've only seen your message after sending it, but I think the first patch
should fix what you are seeing.
Thanks.
>
> [1] https://gitlab.archlinux.org/pacman/pacman/-/blob/v6.0.2/scripts/repo-add.sh.in#L473
next prev parent reply other threads:[~2023-09-09 12:19 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-09 1:07 Possible readdir regression with BTRFS Ian Johnson
2023-09-09 7:27 ` Filipe Manana
2023-09-09 11:52 ` Evangelos Foutras
2023-09-09 12:18 ` Filipe Manana [this message]
2023-09-11 11:56 ` Filipe Manana
2023-09-09 12:16 ` Filipe Manana
2023-09-09 19:27 ` Ian Johnson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZPxiqYCeMb6vOjw9@debian0.Home \
--to=fdmanana@kernel.org \
--cc=evangelos@foutras.com \
--cc=ian@ianjohnson.dev \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox