Linux Btrfs filesystem development
 help / color / mirror / Atom feed
From: "koraynilay" <koray.fra@gmail.com>
To: "Amir Goldstein" <amir73il@gmail.com>,
	"koraynilay" <koray.fra@gmail.com>
Cc: "Lauri Gates" <lauri.gates@gmail.com>,
	"Dave Chinner" <dgc@kernel.org>,
	"Christian Brauner" <brauner@kernel.org>,
	"Christoph Hellwig" <hch@infradead.org>,
	<linux-xfs@vger.kernel.org>, "Carlos Maiolino" <cem@kernel.org>,
	"Christian Brauner" <christian@brauner.io>,
	"Darrick J. Wong" <djwong@kernel.org>,
	<linux-api@vger.kernel.org>, <linux-fsdevel@vger.kernel.org>,
	<linux-btrfs@vger.kernel.org>, <ocfs2-devel@lists.linux.dev>,
	<linux-nfs@vger.kernel.org>, <linux-cifs@vger.kernel.org>
Subject: Re: Concurrent FIEMAP on reflinked files serialises on the AGF buffer lock
Date: Thu, 19 Nov 2026 23:21:30 +0100	[thread overview]
Message-ID: <DMZJDGMJ1RVF.32ES391JKNFGP@gmail.com> (raw)
In-Reply-To: <CAOQ4uximow25oc=3XEN1ATgQvonQaTXtDPmQufxo75tJ=e355A@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 8098 bytes --]

On Thu Nov 19, 2026 at 6:04 PM CET, Amir Goldstein wrote:
> On Thu, Nov 19, 2026 at 12:00 PM koraynilay <koray.fra@gmail.com> wrote:
>>
>> On Thu Nov 19, 2026 at 10:59 AM CET, Lauri Gates wrote:
>> > On Wed, Nov 18, 2026 at 10:16PM +0200, Dave Chinner wrote:
>> >> I really don't care if it is exposed by statx or file_attr; checking
>> >> the flag through either interface will be orders of magnitude faster
>> >> than using FIEMAP to iterate extents on every file to find shared
>> >> extents. My concern with statx is coherency - it runs completely
>> >> unlocked and so can race with operations that add/remove the reflink
>> >> flag...
>> >
>> > Thanks, Dave and Christian. Given Christian's preference and the
>> > coherency point, I'll write the RFC against file_attr, once the
>> > question at the end is settled. xfs_fileattr_get() already reads the
>> > flags under XFS_ILOCK_SHARED, so the new flag gets that without extra
>> > work. The answer can still go stale right after the call, as with any
>> > stat result: a clone that lands after the read is missed, just as one
>> > that lands after FIEMAP returns is missed today.
>> >
>> > As I suggested on the 6th, the flag would take the inverted meaning,
>> > so that an unset flag stays the safe answer: say FS_XFLAG_NOSHARED
>> > ("no extents shared with another file"), read-only, in
>> > FS_XFLAG_RDONLY_MASK next to the case-folding flags.
>>
>> Hello, I don't have any say in the FIEMAP issue, but I wanted to let you
>> know that, since also a mask was mentioned and your patch would be
>> touching the fileattr API, there is work going[1] on for adding a
>> fsx_xflags_mask that filesystem use[2][3] to advertise supported flags (on
>> get, like stx_attributes_mask) and users use to say what they want to
>> change (on set).
>> It doesn't seem like an FS_XFLAG_NOSHARED flag would touch this new
>> change, but I thought it was worth pointing out, especially because this
>> patch would also add FS_XFLAG_PROJID and a new FS_FILE_ATTR_XFLAGS_MASK
>> to have a way to include both legacy and newly added xflags, so
>> FS_XFLAG_NOSHARED would maybe also go there (correct me if I'm wrong).
>>
>> > Christian, one question on the shape within file_attr: would you take
>> > the inverted flag, or would you rather keep a plain "may share" flag
>> > and add a mask to struct file_attr that says which xflags the
>> > filesystem reports?
>>
>> And if Christian chooses the mask approach, then my patch[4] becomes
>> relevant.
>> (in [4] I put the patch that will be sent in v5 of [1], since in [1]
>> it's merged with the ptime change)
>>
>> (For Amir who I just CC'd, [5] is the main link for this thread)
>
> Thanks for the CC and the link
>
>>
>> Thanks,
>> koraynilay
>>
>> [1]: https://lore.kernel.org/all/20261115180343.576773-1-koray.fra@gmail.com/T/#u
>> [2]: https://lore.kernel.org/all/CAOQ4uxj82K4z+2OiY5W1g7NyWAFOFjqqWnZSS3prytW36j1LgA@mail.gmail.com/
>> [3]: https://lore.kernel.org/all/CAOQ4uxiuJ=eZu28FZAUz3rpqPKqCimPxJucOnYYuBowvD09_Cw@mail.gmail.com/
>> [5]: https://lore.kernel.org/linux-xfs/20261102193533.93690-1-lauri.gates@gmail.com/T/#u
>>
>> [4]:
>> commit 0efe0de3b5cc2b767aa6e2bb9174eb05b0d818f1
>> Author: koraynilay <koray.fra@gmail.com>
>> Date:   Tue Nov 17 14:57:47 2026 +0100
>>
>>     vfs: add xflags mask to file_attr API
>>
>>     Add fa_xflags_mask to struct file_attr and equivalent fsx_xflags_mask
>>     to struct file_kattr.
>>
>>     This lets filesystems advertise their support for specific xflags and
>>     users specify only what they want to change, behaving like
>>     statx.stx_attributes_mask:
>>     - on GET (file_getattr): each filesystem fills the mask according to
>>       its supported xflags, e.g. FS_XFLAG_COWEXTSIZE or FS_XFLAG_EXTSIZE,
>>       and provides the values in the corresponding struct fields;
>>     - on SET (file_setattr): userspace fills the mask according to which
>>       xflags it wants to change, VFS validates this and preserves xflags
>>       that were not specified in the mask;
>>     - if xflags_mask == 0, use legacy full-struct replacement to keep
>>       compatibility with older programs.
>>
>>     Furthermore, add the missing FS_XFLAG_PROJID flag, to allow userspace to
>>     set the projid using file_[sg]etattr().
>
> Correction. usersapce can already set projid using file_[sg]etattr().
> With FS_XFLAG_PROJID in fsx_xflags_mask, chattr -p <projid>
> could set the projid without having to read and set all the other xflags.

Oh right, thanks, I'll fix it.

>>
>>     Finally, in order to prevent the legacy FS_IOC_FSSETXATTR from accepting
>>     FS_XFLAG_PROJID, add an FS_FILE_ATTR_XFLAGS_MASK to support new xflags
>>     and use it in fileattr_to_file_attr()/file_attr_to_fileattr() to
>>     validate that the xflags passed in fa_xflags and fa_xflags_mask fields
>>     are valid.
>>
>>     Assisted-by: Gemini:3.8-flash-high antigravity-cli-1.5.7
>>     Signed-off-by: koraynilay <koray.fra@gmail.com>
>> ---
>>  fs/file_attr.c           | 46 ++++++++++++++++++++++++++++++++++++++++++----
>>  include/linux/fileattr.h |  5 +++++
>>  include/uapi/linux/fs.h  |  3 +++
>>  3 files changed, 50 insertions(+), 4 deletions(-)
>>
>> diff --git a/fs/file_attr.c b/fs/file_attr.c
>> index 833e03810b24..56ea65f790f1 100644
>> --- a/fs/file_attr.c
>> +++ b/fs/file_attr.c
>> @@ -102,10 +102,18 @@ EXPORT_SYMBOL(vfs_fileattr_get);
>>  static void fileattr_to_file_attr(const struct file_kattr *fa,
>>                                   struct file_attr *fattr)
>>  {
>> -       __u32 mask = FS_XFLAGS_MASK;
>> +       /*
>> +        * fsx_xflags_mask contains only flags added after the
>> +        * file_[gs]etattr() API, so we need to tell userspace the filesystem
>> +        * actually supports all legacy (FS_XFLAGS_MASK and FS_XFLAG_PROJID,
>> +        * which lacked its own bit) and new advertised flags.
>> +        */
>> +       __u32 mask = (FS_XFLAGS_MASK | FS_XFLAG_PROJID | fa->fsx_xflags_mask) &
>> +                    FS_FILE_ATTR_XFLAGS_MASK;
>
> This is not exactly how I envisioned exporting fa_xflags_mask.

This is exactly why I CC'd you :D

> fa_xflags_mask == 0 means old kernel or unmodified fs.
>
> For a new kernel with fs that supports flags other than
> FS_XFLAGS_MASK it's the fs responsibility to advertise its supported
> flags.
>
> Something like:
>
>      __u32 mask = (fa->fsx_xflags_mask ?: FS_XFLAGS_MASK) &
>                               FS_FILE_ATTR_XFLAGS_MASK;
>
>      memset(fattr, 0, sizeof(struct file_attr));
(I'm sorry, but why memset if we assign the fields right after?)
>      fattr->fa_xflags = fa->fsx_xflags & mask;
>      fattr->fa_xflags_mask = fa->fsx_xflags_mask ? mask : 0;
>
> Then you can add filesystems that support get/set of new
> attributes without support for legacy attributes.
>
> And specific filesytems could opt-into the set of an individual
> attribute like projid by advertising
> fa->fsx_xflags_mask = FS_XFLAGS_MASK | FS_XFLAG_PROJID
> or a subset thereof.

I agree, this is definitely better.

> And I am still on the fence wrt should API also advertise the
> fsx_xflags_ro_mask in addition to the supported mask.
>
> The advantage being that it makes the interface symmetric.
> User can alway setattr() with the buffer they got from getattr()
> and instead of randomly ignoring flags in set, the fsx_ro_mask
> along with fsx_mask defines the behavior exactly.

Hmm, so ro_mask would be ignored by vfs on set so users don't have to
manually remove the read only flags, correct?

Also I think I'll try writing some basic userspace programs that use
this API and see how I'd expect it to be.


Anyway, I think it would be better to split the _mask change into its
own separate patch series, so it can more easily be used by this shared
extent patch and it's easier to reason about the API when it's
disconnected from the ptime/btime change.

Thanks,
koraynilay

> Thanks,
> Amir.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

      reply	other threads:[~2026-11-19 22:21 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <auudz7Iwm2MrNA-b@dread>
     [not found] ` <20261106122355.16885-1-lauri.gates@gmail.com>
2026-11-06 15:20   ` Concurrent FIEMAP on reflinked files serialises on the AGF buffer lock Christoph Hellwig
2026-11-06 20:40     ` Lauri Gates
2026-11-12 20:05       ` Christoph Hellwig
2026-11-17 16:22         ` Christian Brauner
2026-11-18 20:16           ` Dave Chinner
2026-11-19  9:59             ` Lauri Gates
2026-11-19 11:00               ` koraynilay
2026-11-19 15:32                 ` Lauri Gates
2026-11-19 17:04                 ` Amir Goldstein
2026-11-19 22:21                   ` koraynilay [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DMZJDGMJ1RVF.32ES391JKNFGP@gmail.com \
    --to=koray.fra@gmail.com \
    --cc=amir73il@gmail.com \
    --cc=brauner@kernel.org \
    --cc=cem@kernel.org \
    --cc=christian@brauner.io \
    --cc=dgc@kernel.org \
    --cc=djwong@kernel.org \
    --cc=hch@infradead.org \
    --cc=lauri.gates@gmail.com \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=linux-xfs@vger.kernel.org \
    --cc=ocfs2-devel@lists.linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox