The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: Jeff Layton <jlayton@kernel.org>
To: Amir Goldstein <amir73il@gmail.com>, koraynilay <koray.fra@gmail.com>
Cc: "David Sterba" <dsterba@suse.com>,
	"Namjae Jeon" <linkinjeon@kernel.org>,
	"Sungjong Seo" <sj1557.seo@samsung.com>,
	"Alexander Viro" <viro@zeniv.linux.org.uk>,
	"Christian Brauner" <brauner@kernel.org>,
	"Trond Myklebust" <trondmy@kernel.org>,
	"Anna Schumaker" <anna@kernel.org>,
	"Chuck Lever" <cel@kernel.org>,
	"Konstantin Komarov" <almaz.alexandrovich@paragon-software.com>,
	"Paulo Alcantara" <pc@manguebit.org>,
	"Chris Mason" <mason@kernel.org>,
	"Yuezhang Mo" <yuezhang.mo@sony.com>, "Jan Kara" <jack@suse.cz>,
	NeilBrown <neil@brown.name>,
	"Olga Kornievskaia" <okorniev@redhat.com>,
	"Dai Ngo" <Dai.Ngo@oracle.com>, "Tom Talpey" <tom@talpey.com>,
	"Ronnie Sahlberg" <ronniesahlberg@gmail.com>,
	"Shyam Prasad N" <sprasad@microsoft.com>,
	"Bharath SM" <bharathsm@microsoft.com>,
	"Sergey Senozhatsky" <senozhatsky@chromium.org>,
	ChenXiaoSong <chenxiaosong@chenxiaosong.com>,
	"Darrick J . Wong" <djwong@kernel.org>,
	"Sean Smith" <defendthedisabled@gmail.com>,
	"Andrey Albershteyn" <aalbersh@kernel.org>,
	"Pedro Falcato" <pfalcato@suse.de>,
	"Pali Rohár" <pali@kernel.org>,
	linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-fsdevel@vger.kernel.org, exfat@lists.linux.dev,
	ntfs3@lists.linux.dev, linux-cifs@vger.kernel.org,
	linux-nfs@vger.kernel.org, linux-api@vger.kernel.org
Subject: Re: [RFC PATCH v4 00/10] add provenance time (ptime)
Date: Wed, 18 Nov 2026 05:45:28 -0800	[thread overview]
Message-ID: <a2889a0bfb26e329ecf2eed6d2318bfcbbb0ff2c.camel@kernel.org> (raw)
In-Reply-To: <CAOQ4uxjcvAKTaGXO0=BpbtgoEPmN7GKieVbX1ZpZR38H1Z3j9g@mail.gmail.com>

On Wed, 2026-11-18 at 09:14 +0100, Amir Goldstein wrote:
> On Wed, Nov 18, 2026 at 2:09 AM koraynilay <koray.fra@gmail.com> wrote:
> > 
> > Hello and thanks for replying,
> > 
> > On Wed Nov 18, 2026 at 12:53 AM CET, Jeff Layton wrote:
> > > On Sun, 2026-11-15 at 19:03 +0100, koraynilay wrote:
> > > > This is an initial implementation of a new timestamp to indicate the
> > > > time when a file/directory was initially created.
> > > > 
> > > > This patch series takes inspiration from the one by Sean Smith
> > > > <defendthedisabled@gmail.com> [1], but using
> > > > file_setattr()/file_getattr() as suggested by Darrick J. Wong
> > > > <djwong@kernel.org> in [2].
> > > > 
> > > > The semantics of this new timestamp should be at least these:
> > > > - it should be the same as btime on a newly created file
> > > > - it's user-settable, so it can be preserved for backups, filesystem
> > > >   conversions, archives and any similar operation
> > > > - it should be preserved every time mtime gets preserved
> > > > 
> > > > and the correspondence with the other timestamps is as follows:
> > > > 
> > > > ctime <-> btime
> > > > mtime <-> ptime
> > > > 
> > > 
> > > I'm not sure how to interpret the correspondence above.
> > 
> >   |-----\_-----------------|-----------|
> >   |       \_      change/  |   birth/  |
> >   |         \_ modification|  creation |
> >   |           \------------|-----------|
> >   | inode time |   ctime   |   btime   |
> >   |------------|-----------|-----------|
> >   | file time  |   mtime   |   ptime   |
> >   |------------|-----------|-----------|
> > 
> > ("file" as in "the concept of this file", I'd say "file content" but
> > that would be incorrect for symlinks)
> > 
> > so ctime is "inode change time", btime is "inode birth/creation time",
> > while mtime is "file content change/modification time" and ptime would
> > be "file content creation time" (same as Window's or MacOS's creation
> > time).
> > 
> > Does this make more sense?
> > 
> > > 
> > > > Examples:
> > > > 1) a .zsh_history file created in 2019, then after a few years the user
> > > >    decides to switch to another filesystem, copying the .zsh_history to
> > > >    the new home, in that case btime would have the timestamp of the copy
> > > >    and ptime would have the 2019 timestamp (just like mtime would have
> > > >    been preserved)
> > > > 2) a symlink created in 2020 from the home directory to a /path going
> > > >    through the same process as the .zsh_history, in this case it would
> > > >    also make sense to sync the ptime to the 2020 timestamp instead of
> > > >    keeping only the btime (so the inode creation time).
> > > > 
> > > > The proof-of-concept code (for testing) for stat(1) and cp(1) is
> > > > available at https://github.com/koraynilay/coreutils (master branch)
> > > > (Note: these coreutils code changes are a prototype and won't be
> > > > submitted upstream as-is, but they work as expected in simple stat and
> > > > cp -a/--preserve=ptime tests)
> > > > 
> > > > Lastly, the name "provenance" and "ptime" are not final, I personally
> > > > don't have a strong opinion on any name, but while testing with stat(1)
> > > > I noticed that "Provenance" is definitely too long for the normal
> > > > output, so another option which would fit well is "Origin time",
> > > > abbreviated as "ntime" (OrigiN time) and not as "otime", as that would
> > > > conflict with btrfs' internal naming and create confusion.
> > > > 
> > > > v2 changes:
> > > > - fix permission bypass leading to ptime update on immutable and
> > > >   append-only files (found by sashiko[3])
> > > > - normalize ptime timestamp as done in fs/attr.c:notify_change() (found
> > > >   by sashiko[3])
> > > > - fix wording in vfs patch to be like the cover-letter
> > > > 
> > > > v3 changes:
> > > > - vfs: reject setting ptime with -EOPNOTSUPP on filesystems that do not
> > > >   support it instead of silently ignoring it (found by sashiko[4])
> > > > - vfs: fix not being able to clear immutable and append-only on files
> > > >   with ptime, and allow updating immutable/append-only atomically along
> > > >   with ptime (found by sashiko[4])
> > > > - btrfs: validate on-disk ptime nanoseconds and reset to otime if
> > > >   invalid (found by sashiko[4])
> > > > - btrfs: avoid re-assigning ptime on unrelated chattr calls (found by
> > > >   sashiko[4])
> > > > - add ntfs3 and exfat support as suggested by Amir Goldstein[5], but no
> > > >   vfat, smb, or nfs4 yet as they are a bit more complex
> > > > - move the introduction of fileattr_has_fsx_mask() from the btrfs patch
> > > >   to a standalone patch in the series
> > > > - btrfs: add fileattr_[gs]et to symlink and special file operations
> > > > 
> > > > v4 changes:
> > > > - vfs: implement fsx_xflags_mask as suggested by Amir Goldstein[6];
> > > > - add SMB (ksmbd/cifs) and nfs (nfsd/nfs) support as requested in
> > > >   previous threads;
> > > > 
> > > > Note:
> > > > to apply this patch 2 previous patches[7][8] from the ksmbd/cifs
> > > > subsystem are needed.
> > > > 
> > > > New Question:
> > > > should I rename fileattr_has_fsx_mask() to something like
> > > > fileattr_has_unsupported_fsx() and keep its code as-is, or should I add
> > > > the fsx_xflags_mask & ~allowed check in it and keep the name as-is? (or
> > > > both?)
> > > > 
> > > > Thanks.
> > > > 
> > > > [1]: https://lore.kernel.org/20260405195007.1306-2-DefendTheDisabled@gmail.com
> > > > [2]: https://lore.kernel.org/20260407014129.GC6192@frogsfrogsfrogs
> > > > [3]: https://sashiko.dev/#/patchset/20261027220532.736352-1-koray.fra%40gmail.com
> > > > [4]: https://sashiko.dev/#/patchset/20261028134728.294127-1-koray.fra%40gmail.com
> > > > [5]: https://lore.kernel.org/CAOQ4uxiuy0VwZj+3M-zRqQ49WTpHZpiaQ+6MZ8z8WdLjyQM_zA@mail.gmail.com/
> > > > [6]: https://lore.kernel.org/CAOQ4uxjtuTDT63vJdYaq+RHQEQFLTksp66e46OMN99WhJz3SbA@mail.gmail.com/
> > > > [7]: https://lore.kernel.org/linux-cifs/20261112111637.184045-2-koray.fra@gmail.com/
> > > > [8]: https://lore.kernel.org/linux-cifs/20261115155308.415734-1-koray.fra@gmail.com/
> > > > 
> > > > 
> > > > koraynilay (10):
> > > >   vfs: add provenance time (ptime)
> > > >   vfs: add fileattr_has_fsx_mask()
> > > >   ntfs3: make crtime user-settable using ptime
> > > >   exfat: make crtime user-settable using ptime
> > > >   btrfs: add provenance time (ptime)
> > > >   vfs: add __vfs_fileattr_set_locked() for inode_lock()'d callers
> > > >   nfsd: add provenance time (ptime) via FATTR4_TIME_CREATE
> > > >   nfs: add provenance time (ptime) via FATTR4_TIME_CREATE
> > > >   ksmbd: add provenance time (ptime) and map it to CreationTime
> > > >   cifs: add provenance time (ptime) and map it to SMB's CreationTime
> > > > 
> > > >  fs/btrfs/btrfs_inode.h          |   4 +
> > > >  fs/btrfs/delayed-inode.c        |   6 ++
> > > >  fs/btrfs/inode.c                |  29 +++++-
> > > >  fs/btrfs/ioctl.c                |  24 ++++-
> > > >  fs/btrfs/print-tree.c           |   1 +
> > > >  fs/btrfs/tree-log.c             |   3 +
> > > >  fs/exfat/exfat_fs.h             |   2 +
> > > >  fs/exfat/file.c                 |  42 ++++++++-
> > > >  fs/exfat/namei.c                |   1 +
> > > >  fs/file_attr.c                  | 156 ++++++++++++++++++++++++++------
> > > >  fs/nfs/inode.c                  |  27 +++++-
> > > >  fs/nfs/internal.h               |   2 +
> > > >  fs/nfs/nfs4proc.c               |  56 +++++++++++-
> > > >  fs/nfs/nfs4xdr.c                |  19 ++--
> > > >  fs/nfs/symlink.c                |   1 +
> > > >  fs/nfsd/nfs4proc.c              |   9 ++
> > > >  fs/nfsd/nfs4xdr.c               |  40 +++++---
> > > >  fs/nfsd/vfs.c                   |  17 ++++
> > > >  fs/nfsd/vfs.h                   |   4 +-
> > > >  fs/nfsd/xdr4.h                  |   3 +
> > > >  fs/ntfs3/file.c                 |  20 +++-
> > > >  fs/ntfs3/frecord.c              |   8 +-
> > > >  fs/smb/client/cifsfs.c          |   8 ++
> > > >  fs/smb/client/inode.c           |   7 +-
> > > >  fs/smb/client/ioctl.c           | 109 ++++++++++++++++------
> > > >  fs/smb/server/smb2pdu.c         |  37 +++++---
> > > >  fs/smb/server/vfs.c             |  43 ++++++++-
> > > >  fs/smb/server/vfs.h             |   1 +
> > > >  fs/stat.c                       |   2 +
> > > >  include/linux/fileattr.h        |  29 +++++-
> > > >  include/linux/nfs_xdr.h         |   3 +
> > > >  include/linux/stat.h            |   1 +
> > > >  include/uapi/linux/btrfs_tree.h |   3 +-
> > > >  include/uapi/linux/fs.h         |   9 +-
> > > >  include/uapi/linux/stat.h       |   7 +-
> > > >  35 files changed, 614 insertions(+), 119 deletions(-)
> > > > 
> > > > 
> > > > base-commit: 3bc1fb9c3b20cd478c83496ab16addecb401770d
> > > > prerequisite-patch-id: 049ae0eba275b5be35a238833107e0b251a55aa2
> > > > prerequisite-patch-id: 614f0bc9411391403d4e9702bafeb31b0ec04ec1
> > > > prerequisite-patch-id: ddf636b9d33d82ea206ce265d817371350bf8858
> > > 
> > > I think this is going to be very hard to do properly, because the
> > > kernel can't really manage this on its own. Something in userland is
> > > going to have to set this value if you want to update it to something
> > > other than the original inode instantiation time (aka btime).
> > > 
> > > It really sounds like what you want is the ability to make the btime
> > > settable. Personally I think the decision to forbid that was wrong-
> > > headed and that it should have been settable from userland from the
> > > beginning. Maybe we should just revisit that decision.
> > 
> 
> Frankly, I don't think this decision was made at all.
> 
> There are only 2 types of filesystems in Linux today:
> 1. btime is constant (xfs, etx4, btrfs,...)
> 2. crtime is settable in principle but not via a standardized Linux
> API (ntfs, cifs, ...)
> 

CephFS is in this category too. I explicitly made it settable so we
could set it when restoring backups.


> AFAIK, btrfs in this series is the only filesystem which is going to
> have two deviated copies of btime and ptime.
> For all the rest, the deviation is artificial in statx.
> 
> So the questions to ask are:
> 1. Do btrfs developers prefer to store a new settable ptime or
>     making otime/btime settable with this new API?
> 2. Will any other fs in the future want to store settable ptime along
>     side the legacy immutable btime?
> 
> My take on these question is:
> - Personally I would love to see ptime support in xfs in the future and would
>   willing to contribute to this work, because I have hard evidence from the
>   performance benefits that it brings to smb server workloads
> - Judging by past experience with i_version bump interpretation on xfs,
>   I find it hard to believe that xfs developer would agree to making the
>   legacy btime settable
> 

Maybe? The main reason for that was that XFS uses i_version  as
internally as a change counter for any inode change. I doubt that it
uses btime internally like that.

The only reason I can see to _not_ make btime settable is for
forensics: maybe having the inode btime be immutable would make it
harder to disguise when an inode was instantiated, but I imagine that
would be defeatable with offline tools anyway.

If people are using the btime in this way, then it would be nice for
them to speak up and give us some details about this usage. How does
keeping it immutable help you?

> Hence the distinction in statx API is needed, but for most filesystems
> it is a pure semantic way of saying "I do/don't have settable birth time".
> 
> > Originally I just wanted to have btime settable, yes, and if btime was
> > settable I wouldn't have made this RFC PATCH either, but after reading
> > the already existing discussions about it, I can understand having an
> > "immutable inode creation time", like ctime exists, so even after making
> > btime settable, a new timestamp would be necessary anyway (e.g. for the
> > people who want btime to exist as a forensics timestamp).
> > 
> 
> Right, but every filesystem (e.g. btrfs) can still make an independent
> choice whether storing one or two copies is desired.
> The API does not limit those choices.
> 
> > > Failing that, why not just track this in an xattr of some sort, since
> > > this is really just an opaque blob once you allow it to be settable. If
> > > the xattr isn't present, then you can just use the value of the btime
> > > as-is. FWIW, I think samba does this today, so you could try to sync
> > > with its behavior here.
> > 
> > (This is copied and adapted from [2])
> > 
> > I actually have been using fully userspace xattrs[1] for a few months,
> > mostly for backups, and it "works"; but last month I wanted to merge 2
> > FSs (and have another merge I have to do), so I ran [1] on all files and
> > then compared all metadata/data between source and destination and
> > noticed there were quite a lot of files that didn't have the xattr set.
> > So I looked online and learned that xattrs only work on regular
> > files/directories, not on symlinks or anything else (and the missing
> > ones were of symlinks).
> > This is what prompted me to search more if anyone had already proposed a
> > new timestamp (instead of making btime settable) and found the email
> > thread from april linked in the cover letter, which gave me the excuse
> > to finally work on a proper patch.
> > 
> 
> I am tired of repeating my arguments (and finding my own quotes)
> about why using xattrs is a poor replacement for in-inode ptime.
> AI agents can easily find those quotes for those who are interested
> so I won't bother, I just say that smbd already uses xattr to store
> the creation time - it's just bad for performance and disk usage in
> metadata intensive workloads.
> 
> > > That seems a lot simpler than adding an entirely new inode timestamp
> > > that is always going to be a copy of the btime, except in rare cases.
> > > Having to patch all of the on-disk filesystems with a new per-inode
> > > field is a lot more work.
> > 
> > > This patch is quite simple, but changing btrfs, then ext4, xfs, etc is a
> > > mess, and inevitably you can't get these timestamps until you upgrade your
> > > filesystems.
> > 

Weird. I didn't write the above sentence.

> 
> Jeff, this is standard "how to make progress".
> I completely don't agree with this argument.
> We cannot use this argument to block progress.
> Is it already a long standing practice for userspace file servers (samba)
> to probe the host filesystem for specific features and fall back to
> alternative implementation if they are not supported.
> 
> There will be nothing new in this case, that's the only way to
> make forward progress - define the API, make support queryable,
> implement the feature in some filesystems, implement consumers, repeat.
> 
> Also for nfsd, that's the only way to ever implement settable
> FATTR4_TIME_CREATE which is the protocol allows.
> This is never going to be possible with custom xattrs.
> So I must say that I don't understand the objection from you and Chuck
> 

I'm not saying that to try and block progress. I'm just pointing out
that it is a rather heavy lift to add a new inode field. This will
require revving the on-disk format for most local filesystems, and it
will take years to roll out. If we can get away with just making the
btime settable, that seems like a much simpler way to do this.

FWIW, Omar Sandoval apparently tried to add AT_UTIME_BTIME to
utimensat() back in 2019, but it never made it in. It might be good to
look over the objections to it then to see if they can be overcome.
-- 
Jeff Layton <jlayton@kernel.org>

  parent reply	other threads:[~2026-11-18 13:45 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-15 18:03 [RFC PATCH v4 00/10] add provenance time (ptime) koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 01/10] vfs: " koraynilay
2026-11-16 20:10   ` Amir Goldstein
2026-11-16 20:53     ` koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 02/10] vfs: add fileattr_has_fsx_mask() koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 03/10] ntfs3: make crtime user-settable using ptime koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 04/10] exfat: " koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 05/10] btrfs: add provenance time (ptime) koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 06/10] vfs: add __vfs_fileattr_set_locked() for inode_lock()'d callers koraynilay
2026-11-16 13:52   ` Amir Goldstein
2026-11-16 15:55     ` koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 07/10] nfsd: add provenance time (ptime) via FATTR4_TIME_CREATE koraynilay
2026-11-17 22:01   ` Chuck Lever
2026-11-18  0:35     ` koraynilay
2026-11-18  7:44     ` Amir Goldstein
2026-11-15 18:03 ` [RFC PATCH v4 08/10] nfs: " koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 09/10] ksmbd: add provenance time (ptime) and map it to CreationTime koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 10/10] cifs: add provenance time (ptime) and map it to SMB's CreationTime koraynilay
2026-11-17 23:53 ` [RFC PATCH v4 00/10] add provenance time (ptime) Jeff Layton
2026-11-18  1:09   ` koraynilay
2026-11-18  8:14     ` Amir Goldstein
2026-11-18  9:34       ` koraynilay
2026-11-18 13:39         ` Amir Goldstein
2026-11-18 13:58           ` koraynilay
2026-11-18 13:45       ` Jeff Layton [this message]
2026-11-18 14:03         ` koraynilay
2026-11-18 14:26 ` Pali Rohár
2026-11-18 14:58   ` koraynilay
2026-11-18 15:16     ` Pali Rohár

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a2889a0bfb26e329ecf2eed6d2318bfcbbb0ff2c.camel@kernel.org \
    --to=jlayton@kernel.org \
    --cc=Dai.Ngo@oracle.com \
    --cc=aalbersh@kernel.org \
    --cc=almaz.alexandrovich@paragon-software.com \
    --cc=amir73il@gmail.com \
    --cc=anna@kernel.org \
    --cc=bharathsm@microsoft.com \
    --cc=brauner@kernel.org \
    --cc=cel@kernel.org \
    --cc=chenxiaosong@chenxiaosong.com \
    --cc=defendthedisabled@gmail.com \
    --cc=djwong@kernel.org \
    --cc=dsterba@suse.com \
    --cc=exfat@lists.linux.dev \
    --cc=jack@suse.cz \
    --cc=koray.fra@gmail.com \
    --cc=linkinjeon@kernel.org \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=mason@kernel.org \
    --cc=neil@brown.name \
    --cc=ntfs3@lists.linux.dev \
    --cc=okorniev@redhat.com \
    --cc=pali@kernel.org \
    --cc=pc@manguebit.org \
    --cc=pfalcato@suse.de \
    --cc=ronniesahlberg@gmail.com \
    --cc=senozhatsky@chromium.org \
    --cc=sj1557.seo@samsung.com \
    --cc=sprasad@microsoft.com \
    --cc=tom@talpey.com \
    --cc=trondmy@kernel.org \
    --cc=viro@zeniv.linux.org.uk \
    --cc=yuezhang.mo@sony.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox