All of lore.kernel.org
 help / color / mirror / Atom feed
From: "koraynilay" <koray.fra@gmail.com>
To: "Jeff Layton" <jlayton@kernel.org>,
	"koraynilay" <koray.fra@gmail.com>,
	"David Sterba" <dsterba@suse.com>,
	"Namjae Jeon" <linkinjeon@kernel.org>,
	"Sungjong Seo" <sj1557.seo@samsung.com>,
	"Alexander Viro" <viro@zeniv.linux.org.uk>,
	"Christian Brauner" <brauner@kernel.org>,
	"Trond Myklebust" <trondmy@kernel.org>,
	"Anna Schumaker" <anna@kernel.org>,
	"Chuck Lever" <cel@kernel.org>,
	"Konstantin Komarov" <almaz.alexandrovich@paragon-software.com>,
	"Paulo Alcantara" <pc@manguebit.org>
Cc: "Chris Mason" <mason@kernel.org>,
	"Yuezhang Mo" <yuezhang.mo@sony.com>, "Jan Kara" <jack@suse.cz>,
	NeilBrown <neil@brown.name>,
	"Olga Kornievskaia" <okorniev@redhat.com>,
	"Dai Ngo" <Dai.Ngo@oracle.com>, "Tom Talpey" <tom@talpey.com>,
	"Ronnie Sahlberg" <ronniesahlberg@gmail.com>,
	"Shyam Prasad N" <sprasad@microsoft.com>,
	"Bharath SM" <bharathsm@microsoft.com>,
	"Sergey Senozhatsky" <senozhatsky@chromium.org>,
	ChenXiaoSong <chenxiaosong@chenxiaosong.com>,
	"Darrick J . Wong" <djwong@kernel.org>,
	"Sean Smith" <defendthedisabled@gmail.com>,
	"Andrey Albershteyn" <aalbersh@kernel.org>,
	"Pedro Falcato" <pfalcato@suse.de>,
	"Amir Goldstein" <amir73il@gmail.com>,
	"Pali Rohár" <pali@kernel.org>,
	linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-fsdevel@vger.kernel.org, exfat@lists.linux.dev,
	ntfs3@lists.linux.dev, linux-cifs@vger.kernel.org,
	linux-nfs@vger.kernel.org, linux-api@vger.kernel.org
Subject: Re: [RFC PATCH v4 00/10] add provenance time (ptime)
Date: Wed, 18 Nov 2026 02:09:06 +0100	[thread overview]
Message-ID: <DMXXOPD9UDNP.2YP9ROMHFLM9E@gmail.com> (raw)
In-Reply-To: <fc53ec55d315a4b96ec1e6f250ad3763f2189755.camel@kernel.org>

[-- Attachment #1: Type: text/plain, Size: 11093 bytes --]

Hello and thanks for replying,

On Wed Nov 18, 2026 at 12:53 AM CET, Jeff Layton wrote:
> On Sun, 2026-11-15 at 19:03 +0100, koraynilay wrote:
>> This is an initial implementation of a new timestamp to indicate the
>> time when a file/directory was initially created.
>> 
>> This patch series takes inspiration from the one by Sean Smith
>> <defendthedisabled@gmail.com> [1], but using
>> file_setattr()/file_getattr() as suggested by Darrick J. Wong
>> <djwong@kernel.org> in [2].
>> 
>> The semantics of this new timestamp should be at least these:
>> - it should be the same as btime on a newly created file
>> - it's user-settable, so it can be preserved for backups, filesystem
>>   conversions, archives and any similar operation
>> - it should be preserved every time mtime gets preserved
>> 
>> and the correspondence with the other timestamps is as follows:
>> 
>> ctime <-> btime
>> mtime <-> ptime
>> 
>
> I'm not sure how to interpret the correspondence above.

  |-----\_-----------------|-----------|
  |       \_      change/  |   birth/  |
  |         \_ modification|  creation |
  |           \------------|-----------|
  | inode time |   ctime   |   btime   |
  |------------|-----------|-----------|
  | file time  |   mtime   |   ptime   |
  |------------|-----------|-----------|
  
("file" as in "the concept of this file", I'd say "file content" but
that would be incorrect for symlinks)

so ctime is "inode change time", btime is "inode birth/creation time",
while mtime is "file content change/modification time" and ptime would
be "file content creation time" (same as Window's or MacOS's creation
time).

Does this make more sense?

>
>> Examples:
>> 1) a .zsh_history file created in 2019, then after a few years the user
>>    decides to switch to another filesystem, copying the .zsh_history to
>>    the new home, in that case btime would have the timestamp of the copy
>>    and ptime would have the 2019 timestamp (just like mtime would have
>>    been preserved)
>> 2) a symlink created in 2020 from the home directory to a /path going
>>    through the same process as the .zsh_history, in this case it would
>>    also make sense to sync the ptime to the 2020 timestamp instead of
>>    keeping only the btime (so the inode creation time).
>> 
>> The proof-of-concept code (for testing) for stat(1) and cp(1) is
>> available at https://github.com/koraynilay/coreutils (master branch)
>> (Note: these coreutils code changes are a prototype and won't be
>> submitted upstream as-is, but they work as expected in simple stat and
>> cp -a/--preserve=ptime tests)
>> 
>> Lastly, the name "provenance" and "ptime" are not final, I personally
>> don't have a strong opinion on any name, but while testing with stat(1)
>> I noticed that "Provenance" is definitely too long for the normal
>> output, so another option which would fit well is "Origin time",
>> abbreviated as "ntime" (OrigiN time) and not as "otime", as that would
>> conflict with btrfs' internal naming and create confusion.
>> 
>> v2 changes:
>> - fix permission bypass leading to ptime update on immutable and
>>   append-only files (found by sashiko[3])
>> - normalize ptime timestamp as done in fs/attr.c:notify_change() (found
>>   by sashiko[3])
>> - fix wording in vfs patch to be like the cover-letter
>> 
>> v3 changes:
>> - vfs: reject setting ptime with -EOPNOTSUPP on filesystems that do not
>>   support it instead of silently ignoring it (found by sashiko[4])
>> - vfs: fix not being able to clear immutable and append-only on files
>>   with ptime, and allow updating immutable/append-only atomically along
>>   with ptime (found by sashiko[4])
>> - btrfs: validate on-disk ptime nanoseconds and reset to otime if
>>   invalid (found by sashiko[4])
>> - btrfs: avoid re-assigning ptime on unrelated chattr calls (found by
>>   sashiko[4])
>> - add ntfs3 and exfat support as suggested by Amir Goldstein[5], but no
>>   vfat, smb, or nfs4 yet as they are a bit more complex
>> - move the introduction of fileattr_has_fsx_mask() from the btrfs patch
>>   to a standalone patch in the series
>> - btrfs: add fileattr_[gs]et to symlink and special file operations
>> 
>> v4 changes:
>> - vfs: implement fsx_xflags_mask as suggested by Amir Goldstein[6];
>> - add SMB (ksmbd/cifs) and nfs (nfsd/nfs) support as requested in
>>   previous threads;
>> 
>> Note:
>> to apply this patch 2 previous patches[7][8] from the ksmbd/cifs
>> subsystem are needed.
>> 
>> New Question:
>> should I rename fileattr_has_fsx_mask() to something like
>> fileattr_has_unsupported_fsx() and keep its code as-is, or should I add
>> the fsx_xflags_mask & ~allowed check in it and keep the name as-is? (or
>> both?)
>> 
>> Thanks.
>> 
>> [1]: https://lore.kernel.org/20260405195007.1306-2-DefendTheDisabled@gmail.com
>> [2]: https://lore.kernel.org/20260407014129.GC6192@frogsfrogsfrogs
>> [3]: https://sashiko.dev/#/patchset/20261027220532.736352-1-koray.fra%40gmail.com
>> [4]: https://sashiko.dev/#/patchset/20261028134728.294127-1-koray.fra%40gmail.com
>> [5]: https://lore.kernel.org/CAOQ4uxiuy0VwZj+3M-zRqQ49WTpHZpiaQ+6MZ8z8WdLjyQM_zA@mail.gmail.com/
>> [6]: https://lore.kernel.org/CAOQ4uxjtuTDT63vJdYaq+RHQEQFLTksp66e46OMN99WhJz3SbA@mail.gmail.com/
>> [7]: https://lore.kernel.org/linux-cifs/20261112111637.184045-2-koray.fra@gmail.com/
>> [8]: https://lore.kernel.org/linux-cifs/20261115155308.415734-1-koray.fra@gmail.com/
>> 
>> 
>> koraynilay (10):
>>   vfs: add provenance time (ptime)
>>   vfs: add fileattr_has_fsx_mask()
>>   ntfs3: make crtime user-settable using ptime
>>   exfat: make crtime user-settable using ptime
>>   btrfs: add provenance time (ptime)
>>   vfs: add __vfs_fileattr_set_locked() for inode_lock()'d callers
>>   nfsd: add provenance time (ptime) via FATTR4_TIME_CREATE
>>   nfs: add provenance time (ptime) via FATTR4_TIME_CREATE
>>   ksmbd: add provenance time (ptime) and map it to CreationTime
>>   cifs: add provenance time (ptime) and map it to SMB's CreationTime
>> 
>>  fs/btrfs/btrfs_inode.h          |   4 +
>>  fs/btrfs/delayed-inode.c        |   6 ++
>>  fs/btrfs/inode.c                |  29 +++++-
>>  fs/btrfs/ioctl.c                |  24 ++++-
>>  fs/btrfs/print-tree.c           |   1 +
>>  fs/btrfs/tree-log.c             |   3 +
>>  fs/exfat/exfat_fs.h             |   2 +
>>  fs/exfat/file.c                 |  42 ++++++++-
>>  fs/exfat/namei.c                |   1 +
>>  fs/file_attr.c                  | 156 ++++++++++++++++++++++++++------
>>  fs/nfs/inode.c                  |  27 +++++-
>>  fs/nfs/internal.h               |   2 +
>>  fs/nfs/nfs4proc.c               |  56 +++++++++++-
>>  fs/nfs/nfs4xdr.c                |  19 ++--
>>  fs/nfs/symlink.c                |   1 +
>>  fs/nfsd/nfs4proc.c              |   9 ++
>>  fs/nfsd/nfs4xdr.c               |  40 +++++---
>>  fs/nfsd/vfs.c                   |  17 ++++
>>  fs/nfsd/vfs.h                   |   4 +-
>>  fs/nfsd/xdr4.h                  |   3 +
>>  fs/ntfs3/file.c                 |  20 +++-
>>  fs/ntfs3/frecord.c              |   8 +-
>>  fs/smb/client/cifsfs.c          |   8 ++
>>  fs/smb/client/inode.c           |   7 +-
>>  fs/smb/client/ioctl.c           | 109 ++++++++++++++++------
>>  fs/smb/server/smb2pdu.c         |  37 +++++---
>>  fs/smb/server/vfs.c             |  43 ++++++++-
>>  fs/smb/server/vfs.h             |   1 +
>>  fs/stat.c                       |   2 +
>>  include/linux/fileattr.h        |  29 +++++-
>>  include/linux/nfs_xdr.h         |   3 +
>>  include/linux/stat.h            |   1 +
>>  include/uapi/linux/btrfs_tree.h |   3 +-
>>  include/uapi/linux/fs.h         |   9 +-
>>  include/uapi/linux/stat.h       |   7 +-
>>  35 files changed, 614 insertions(+), 119 deletions(-)
>> 
>> 
>> base-commit: 3bc1fb9c3b20cd478c83496ab16addecb401770d
>> prerequisite-patch-id: 049ae0eba275b5be35a238833107e0b251a55aa2
>> prerequisite-patch-id: 614f0bc9411391403d4e9702bafeb31b0ec04ec1
>> prerequisite-patch-id: ddf636b9d33d82ea206ce265d817371350bf8858
>
> I think this is going to be very hard to do properly, because the
> kernel can't really manage this on its own. Something in userland is
> going to have to set this value if you want to update it to something
> other than the original inode instantiation time (aka btime).
>
> It really sounds like what you want is the ability to make the btime
> settable. Personally I think the decision to forbid that was wrong-
> headed and that it should have been settable from userland from the
> beginning. Maybe we should just revisit that decision.

Originally I just wanted to have btime settable, yes, and if btime was
settable I wouldn't have made this RFC PATCH either, but after reading
the already existing discussions about it, I can understand having an
"immutable inode creation time", like ctime exists, so even after making
btime settable, a new timestamp would be necessary anyway (e.g. for the
people who want btime to exist as a forensics timestamp).

> Failing that, why not just track this in an xattr of some sort, since
> this is really just an opaque blob once you allow it to be settable. If
> the xattr isn't present, then you can just use the value of the btime
> as-is. FWIW, I think samba does this today, so you could try to sync
> with its behavior here.

(This is copied and adapted from [2])

I actually have been using fully userspace xattrs[1] for a few months,
mostly for backups, and it "works"; but last month I wanted to merge 2
FSs (and have another merge I have to do), so I ran [1] on all files and
then compared all metadata/data between source and destination and
noticed there were quite a lot of files that didn't have the xattr set.
So I looked online and learned that xattrs only work on regular
files/directories, not on symlinks or anything else (and the missing
ones were of symlinks).
This is what prompted me to search more if anyone had already proposed a
new timestamp (instead of making btime settable) and found the email
thread from april linked in the cover letter, which gave me the excuse
to finally work on a proper patch.

> That seems a lot simpler than adding an entirely new inode timestamp
> that is always going to be a copy of the btime, except in rare cases.
> Having to patch all of the on-disk filesystems with a new per-inode
> field is a lot more work.

> This patch is quite simple, but changing btrfs, then ext4, xfs, etc is a
> mess, and inevitably you can't get these timestamps until you upgrade your
> filesystems.

So yeah while I do agree it touches a lot of code that could create
problems, IMO this is the best solution and it would complete files'
timestamps like other OSs have (and this, I think, is necessary for
both backup/archival and migrations).

Thanks,
koraynilay

[1]: https://github.com/koraynilay/btime-to-xattr
[2]: https://lore.kernel.org/DMGS4KQLY24Q.14MUTR6U93GIC@gmail.com/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

      reply	other threads:[~2026-11-18  1:09 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-15 18:03 [RFC PATCH v4 00/10] add provenance time (ptime) koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 01/10] vfs: " koraynilay
2026-11-16 20:10   ` Amir Goldstein
2026-11-16 20:53     ` koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 02/10] vfs: add fileattr_has_fsx_mask() koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 03/10] ntfs3: make crtime user-settable using ptime koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 04/10] exfat: " koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 05/10] btrfs: add provenance time (ptime) koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 06/10] vfs: add __vfs_fileattr_set_locked() for inode_lock()'d callers koraynilay
2026-11-16 13:52   ` Amir Goldstein
2026-11-16 15:55     ` koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 07/10] nfsd: add provenance time (ptime) via FATTR4_TIME_CREATE koraynilay
2026-11-17 22:01   ` Chuck Lever
2026-11-18  0:35     ` koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 08/10] nfs: " koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 09/10] ksmbd: add provenance time (ptime) and map it to CreationTime koraynilay
2026-11-15 18:03 ` [RFC PATCH v4 10/10] cifs: add provenance time (ptime) and map it to SMB's CreationTime koraynilay
2026-11-17 23:53 ` [RFC PATCH v4 00/10] add provenance time (ptime) Jeff Layton
2026-11-18  1:09   ` koraynilay [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DMXXOPD9UDNP.2YP9ROMHFLM9E@gmail.com \
    --to=koray.fra@gmail.com \
    --cc=Dai.Ngo@oracle.com \
    --cc=aalbersh@kernel.org \
    --cc=almaz.alexandrovich@paragon-software.com \
    --cc=amir73il@gmail.com \
    --cc=anna@kernel.org \
    --cc=bharathsm@microsoft.com \
    --cc=brauner@kernel.org \
    --cc=cel@kernel.org \
    --cc=chenxiaosong@chenxiaosong.com \
    --cc=defendthedisabled@gmail.com \
    --cc=djwong@kernel.org \
    --cc=dsterba@suse.com \
    --cc=exfat@lists.linux.dev \
    --cc=jack@suse.cz \
    --cc=jlayton@kernel.org \
    --cc=linkinjeon@kernel.org \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=mason@kernel.org \
    --cc=neil@brown.name \
    --cc=ntfs3@lists.linux.dev \
    --cc=okorniev@redhat.com \
    --cc=pali@kernel.org \
    --cc=pc@manguebit.org \
    --cc=pfalcato@suse.de \
    --cc=ronniesahlberg@gmail.com \
    --cc=senozhatsky@chromium.org \
    --cc=sj1557.seo@samsung.com \
    --cc=sprasad@microsoft.com \
    --cc=tom@talpey.com \
    --cc=trondmy@kernel.org \
    --cc=viro@zeniv.linux.org.uk \
    --cc=yuezhang.mo@sony.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.