Linux Btrfs filesystem development
 help / color / mirror / Atom feed
From: "koraynilay" <koray.fra@gmail.com>
To: "Amir Goldstein" <amir73il@gmail.com>,
	"koraynilay" <koray.fra@gmail.com>
Cc: "Lauri Gates" <lauri.gates@gmail.com>,
	"Dave Chinner" <dgc@kernel.org>,
	"Christian Brauner" <brauner@kernel.org>,
	"Christoph Hellwig" <hch@infradead.org>,
	<linux-xfs@vger.kernel.org>, "Carlos Maiolino" <cem@kernel.org>,
	"Christian Brauner" <christian@brauner.io>,
	"Darrick J. Wong" <djwong@kernel.org>,
	<linux-api@vger.kernel.org>, <linux-fsdevel@vger.kernel.org>,
	<linux-btrfs@vger.kernel.org>, <ocfs2-devel@lists.linux.dev>,
	<linux-nfs@vger.kernel.org>, <linux-cifs@vger.kernel.org>,
	"Theodore Tso" <tytso@mit.edu>
Subject: Re: Concurrent FIEMAP on reflinked files serialises on the AGF buffer lock
Date: Fri, 20 Nov 2026 15:12:54 +0100	[thread overview]
Message-ID: <DN03LWNSDYP5.11FTQ5KCWU2X7@gmail.com> (raw)
In-Reply-To: <CAOQ4uxjVQ9cjV=_jBjQtNg9zCuWUEJT9KNYMKV+1jwBkHen5OA@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 9665 bytes --]

On Fri Nov 20, 2026 at 6:52 AM CET, Amir Goldstein wrote:
> On Thu, Nov 19, 2026 at 11:21 PM koraynilay <koray.fra@gmail.com> wrote:
>>
>> On Thu Nov 19, 2026 at 6:04 PM CET, Amir Goldstein wrote:
>> > On Thu, Nov 19, 2026 at 12:00 PM koraynilay <koray.fra@gmail.com> wrote:
>> >>
>> >> On Thu Nov 19, 2026 at 10:59 AM CET, Lauri Gates wrote:
>> >> > On Wed, Nov 18, 2026 at 10:16PM +0200, Dave Chinner wrote:
>> >> >> I really don't care if it is exposed by statx or file_attr; checking
>> >> >> the flag through either interface will be orders of magnitude faster
>> >> >> than using FIEMAP to iterate extents on every file to find shared
>> >> >> extents. My concern with statx is coherency - it runs completely
>> >> >> unlocked and so can race with operations that add/remove the reflink
>> >> >> flag...
>> >> >
>> >> > Thanks, Dave and Christian. Given Christian's preference and the
>> >> > coherency point, I'll write the RFC against file_attr, once the
>> >> > question at the end is settled. xfs_fileattr_get() already reads the
>> >> > flags under XFS_ILOCK_SHARED, so the new flag gets that without extra
>> >> > work. The answer can still go stale right after the call, as with any
>> >> > stat result: a clone that lands after the read is missed, just as one
>> >> > that lands after FIEMAP returns is missed today.
>> >> >
>> >> > As I suggested on the 6th, the flag would take the inverted meaning,
>> >> > so that an unset flag stays the safe answer: say FS_XFLAG_NOSHARED
>> >> > ("no extents shared with another file"), read-only, in
>> >> > FS_XFLAG_RDONLY_MASK next to the case-folding flags.
>> >>
>> >> Hello, I don't have any say in the FIEMAP issue, but I wanted to let you
>> >> know that, since also a mask was mentioned and your patch would be
>> >> touching the fileattr API, there is work going[1] on for adding a
>> >> fsx_xflags_mask that filesystem use[2][3] to advertise supported flags (on
>> >> get, like stx_attributes_mask) and users use to say what they want to
>> >> change (on set).
>> >> It doesn't seem like an FS_XFLAG_NOSHARED flag would touch this new
>> >> change, but I thought it was worth pointing out, especially because this
>> >> patch would also add FS_XFLAG_PROJID and a new FS_FILE_ATTR_XFLAGS_MASK
>> >> to have a way to include both legacy and newly added xflags, so
>> >> FS_XFLAG_NOSHARED would maybe also go there (correct me if I'm wrong).
>> >>
>> >> > Christian, one question on the shape within file_attr: would you take
>> >> > the inverted flag, or would you rather keep a plain "may share" flag
>> >> > and add a mask to struct file_attr that says which xflags the
>> >> > filesystem reports?
>> >>
>> >> And if Christian chooses the mask approach, then my patch[4] becomes
>> >> relevant.
>> >> (in [4] I put the patch that will be sent in v5 of [1], since in [1]
>> >> it's merged with the ptime change)
>> >>
>> >> (For Amir who I just CC'd, [5] is the main link for this thread)
>> >
>> > Thanks for the CC and the link
>> >
>> >>
>> >> Thanks,
>> >> koraynilay
>> >>
>> >> [1]: https://lore.kernel.org/all/20261115180343.576773-1-koray.fra@gmail.com/T/#u
>> >> [2]: https://lore.kernel.org/all/CAOQ4uxj82K4z+2OiY5W1g7NyWAFOFjqqWnZSS3prytW36j1LgA@mail.gmail.com/
>> >> [3]: https://lore.kernel.org/all/CAOQ4uxiuJ=eZu28FZAUz3rpqPKqCimPxJucOnYYuBowvD09_Cw@mail.gmail.com/
>> >> [5]: https://lore.kernel.org/linux-xfs/20261102193533.93690-1-lauri.gates@gmail.com/T/#u
>> >>
>> >> [4]:
>> >> commit 0efe0de3b5cc2b767aa6e2bb9174eb05b0d818f1
>> >> Author: koraynilay <koray.fra@gmail.com>
>> >> Date:   Tue Nov 17 14:57:47 2026 +0100
>> >>
>> >>     vfs: add xflags mask to file_attr API
>> >>
>> >>     Add fa_xflags_mask to struct file_attr and equivalent fsx_xflags_mask
>> >>     to struct file_kattr.
>> >>
>> >>     This lets filesystems advertise their support for specific xflags and
>> >>     users specify only what they want to change, behaving like
>> >>     statx.stx_attributes_mask:
>> >>     - on GET (file_getattr): each filesystem fills the mask according to
>> >>       its supported xflags, e.g. FS_XFLAG_COWEXTSIZE or FS_XFLAG_EXTSIZE,
>> >>       and provides the values in the corresponding struct fields;
>> >>     - on SET (file_setattr): userspace fills the mask according to which
>> >>       xflags it wants to change, VFS validates this and preserves xflags
>> >>       that were not specified in the mask;
>> >>     - if xflags_mask == 0, use legacy full-struct replacement to keep
>> >>       compatibility with older programs.
>> >>
>> >>     Furthermore, add the missing FS_XFLAG_PROJID flag, to allow userspace to
>> >>     set the projid using file_[sg]etattr().
>> >
>> > Correction. usersapce can already set projid using file_[sg]etattr().
>> > With FS_XFLAG_PROJID in fsx_xflags_mask, chattr -p <projid>
>> > could set the projid without having to read and set all the other xflags.
>>
>> Oh right, thanks, I'll fix it.
>>
>> >>
>> >>     Finally, in order to prevent the legacy FS_IOC_FSSETXATTR from accepting
>> >>     FS_XFLAG_PROJID, add an FS_FILE_ATTR_XFLAGS_MASK to support new xflags
>> >>     and use it in fileattr_to_file_attr()/file_attr_to_fileattr() to
>> >>     validate that the xflags passed in fa_xflags and fa_xflags_mask fields
>> >>     are valid.
>> >>
>> >>     Assisted-by: Gemini:3.8-flash-high antigravity-cli-1.5.7
>> >>     Signed-off-by: koraynilay <koray.fra@gmail.com>
>> >> ---
>> >>  fs/file_attr.c           | 46 ++++++++++++++++++++++++++++++++++++++++++----
>> >>  include/linux/fileattr.h |  5 +++++
>> >>  include/uapi/linux/fs.h  |  3 +++
>> >>  3 files changed, 50 insertions(+), 4 deletions(-)
>> >>
>> >> diff --git a/fs/file_attr.c b/fs/file_attr.c
>> >> index 833e03810b24..56ea65f790f1 100644
>> >> --- a/fs/file_attr.c
>> >> +++ b/fs/file_attr.c
>> >> @@ -102,10 +102,18 @@ EXPORT_SYMBOL(vfs_fileattr_get);
>> >>  static void fileattr_to_file_attr(const struct file_kattr *fa,
>> >>                                   struct file_attr *fattr)
>> >>  {
>> >> -       __u32 mask = FS_XFLAGS_MASK;
>> >> +       /*
>> >> +        * fsx_xflags_mask contains only flags added after the
>> >> +        * file_[gs]etattr() API, so we need to tell userspace the filesystem
>> >> +        * actually supports all legacy (FS_XFLAGS_MASK and FS_XFLAG_PROJID,
>> >> +        * which lacked its own bit) and new advertised flags.
>> >> +        */
>> >> +       __u32 mask = (FS_XFLAGS_MASK | FS_XFLAG_PROJID | fa->fsx_xflags_mask) &
>> >> +                    FS_FILE_ATTR_XFLAGS_MASK;
>> >
>> > This is not exactly how I envisioned exporting fa_xflags_mask.
>>
>> This is exactly why I CC'd you :D
>>
>> > fa_xflags_mask == 0 means old kernel or unmodified fs.
>> >
>> > For a new kernel with fs that supports flags other than
>> > FS_XFLAGS_MASK it's the fs responsibility to advertise its supported
>> > flags.
>> >
>> > Something like:
>> >
>> >      __u32 mask = (fa->fsx_xflags_mask ?: FS_XFLAGS_MASK) &
>> >                               FS_FILE_ATTR_XFLAGS_MASK;
>> >
>> >      memset(fattr, 0, sizeof(struct file_attr));
>> (I'm sorry, but why memset if we assign the fields right after?)
>> >      fattr->fa_xflags = fa->fsx_xflags & mask;
>> >      fattr->fa_xflags_mask = fa->fsx_xflags_mask ? mask : 0;
>> >
>> > Then you can add filesystems that support get/set of new
>> > attributes without support for legacy attributes.
>> >
>> > And specific filesytems could opt-into the set of an individual
>> > attribute like projid by advertising
>> > fa->fsx_xflags_mask = FS_XFLAGS_MASK | FS_XFLAG_PROJID
>> > or a subset thereof.
>>
>> I agree, this is definitely better.
>>
>> > And I am still on the fence wrt should API also advertise the
>> > fsx_xflags_ro_mask in addition to the supported mask.
>> >
>> > The advantage being that it makes the interface symmetric.
>> > User can alway setattr() with the buffer they got from getattr()
>> > and instead of randomly ignoring flags in set, the fsx_ro_mask
>> > along with fsx_mask defines the behavior exactly.
>>
>> Hmm, so ro_mask would be ignored by vfs on set so users don't have to
>> manually remove the read only flags, correct?
>>
>
> Correct.

Ok.

> With non-zero xflags_mask, internal contact masks
> can be used for -EINVAL not for silent ignoring.

What do you mean by "internal contact masks"?

>> Also I think I'll try writing some basic userspace programs that use
>> this API and see how I'd expect it to be.
>>
>
> Don't write programs please contribute to chattr (e2fsprogs) and
> xfs_io (xfsprogs). The latter already support the new syscalls, the
> former not yet.

Oh no, I didn't mean full fledged programs, just simple struct ->
syscall like the set_ptime[1] I've been using for testing.

>>
>> Anyway, I think it would be better to split the _mask change into its
>> own separate patch series, so it can more easily be used by this shared
>> extent patch and it's easier to reason about the API when it's
>> disconnected from the ptime/btime change.
>
> Agree.

Nice.

Thanks,
koraynilay

> Thanks,
> Amir.

[1]:

#include "include/uapi/linux/fs.h"

int main(int argc, char *argv[]) {
	char *path = argv[1];
	long long sec = atoll(argv[2]);
	long long nsec = atoll(argv[3]);
	struct file_attr fa = {
		.fa_xflags = FS_XFLAG_PTIME,
		.fa_xflags_mask = FS_XFLAG_PTIME,
		.fa_ptime_sec = sec,
		.fa_ptime_nsec = nsec,
	};

	return syscall(__NR_file_setattr, AT_FDCWD, path, &fa, sizeof(fa), AT_SYMLINK_NOFOLLOW);
}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

      reply	other threads:[~2026-11-20 14:12 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <auudz7Iwm2MrNA-b@dread>
     [not found] ` <20261106122355.16885-1-lauri.gates@gmail.com>
2026-11-06 15:20   ` Concurrent FIEMAP on reflinked files serialises on the AGF buffer lock Christoph Hellwig
2026-11-06 20:40     ` Lauri Gates
2026-11-12 20:05       ` Christoph Hellwig
2026-11-17 16:22         ` Christian Brauner
2026-11-18 20:16           ` Dave Chinner
2026-11-19  9:59             ` Lauri Gates
2026-11-19 11:00               ` koraynilay
2026-11-19 15:32                 ` Lauri Gates
2026-11-19 17:04                 ` Amir Goldstein
2026-11-19 22:21                   ` koraynilay
2026-11-20  5:52                     ` Amir Goldstein
2026-11-20 14:12                       ` koraynilay [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DN03LWNSDYP5.11FTQ5KCWU2X7@gmail.com \
    --to=koray.fra@gmail.com \
    --cc=amir73il@gmail.com \
    --cc=brauner@kernel.org \
    --cc=cem@kernel.org \
    --cc=christian@brauner.io \
    --cc=dgc@kernel.org \
    --cc=djwong@kernel.org \
    --cc=hch@infradead.org \
    --cc=lauri.gates@gmail.com \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=linux-xfs@vger.kernel.org \
    --cc=ocfs2-devel@lists.linux.dev \
    --cc=tytso@mit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox