A syscall for changing birth time

A syscall for changing birth time

[Artem S. Tashkinov]

Hello,

Why is it that the Linux kernel supports reading btime, but there's no
syscall to change it? At least for ext4 there's the debugfs utility, but
for other filesystems there's just nothing. And even debugfs is not a
solution, since it requires root privileges and an unmounted/mounted RO
filesystem.

Having it allows for better compatibility with Windows and BSD operating
systems and could improve backup/restore.

Best regards,
Artem

Re: A syscall for changing birth time

[Theodore Ts'o]

On Mon, Mar 10, 2025 at 07:26:00AM +0000, Artem S. Tashkinov wrote:
> 
> Why is it that the Linux kernel supports reading btime, but there's no
> syscall to change it? At least for ext4 there's the debugfs utility, but
> for other filesystems there's just nothing. And even debugfs is not a
> solution, since it requires root privileges and an unmounted/mounted RO
> filesystem.

POSIX and Single Unix Specification also doesn't provide a way to
allow userspace to set ctime (inode change time).  That's because the
definition of "change time" is defined to include the time to change
anything about the inode metadata --- including the inode timestamps.

Simply, the definition of "birth time" is about the time that the
inode was "birthed", and that's not something that you can change.
The problem is that DOS has a concept of "creation time", which seems
to mean "the time that the abstract concept of the file was created".
So if a file was created somewhere in a build farm in Redmond,
Washington, that's the time that the file should have, according to
Microsoft.  So Windows allows the "creation time" to be set to any
arbitrary file, since installers need to be able to set the "abstract
creation time".

You can debate whether "birth time" (which can't be set) or a
"abstract creation time" (which can set to any arbitrary value), is
"better" but that's why Linux doesn't support a way to set the "birth
time".

Whether you think we should bow to what Microsoft dictates probably
depends on how much you believe Windows is a legacy operating system
or not.  :-)  Personally, it's not something I really care about, and
if someone really wants to add a Windows-compatible "Creation Time",
my suggestion would be to define an extended attribute where this
could be stored.

We *could* allocate space in the on-disk inode to store this
timestamp, but since I would estimate 99.9% of deployed Linux systems
don't care about Windows compatibility, it's not a good use of
resources.  We could also add a mount option which changes the
semantics of birth time, but that adds extra complexity, and again, I
would estimate that 99.9% of Linux systems (where I include all of the
Linux deployments in Cloud VM's) don't care about Windows
compatibility in this way.

Cheers,

						- Ted

Re: A syscall for changing birth time

[Artem S. Tashkinov]

On 3/10/25 1:58 PM, Theodore Ts'o wrote:
> On Mon, Mar 10, 2025 at 07:26:00AM +0000, Artem S. Tashkinov wrote:
>>
>> Why is it that the Linux kernel supports reading btime, but there's no
>> syscall to change it? At least for ext4 there's the debugfs utility, but
>> for other filesystems there's just nothing. And even debugfs is not a
>> solution, since it requires root privileges and an unmounted/mounted RO
>> filesystem.
>
> POSIX and Single Unix Specification also doesn't provide a way to
> allow userspace to set ctime (inode change time).  That's because the
> definition of "change time" is defined to include the time to change
> anything about the inode metadata --- including the inode timestamps.
>
> Simply, the definition of "birth time" is about the time that the
> inode was "birthed", and that's not something that you can change.
> The problem is that DOS has a concept of "creation time", which seems
> to mean "the time that the abstract concept of the file was created".
> So if a file was created somewhere in a build farm in Redmond,
> Washington, that's the time that the file should have, according to
> Microsoft.  So Windows allows the "creation time" to be set to any
> arbitrary file, since installers need to be able to set the "abstract
> creation time".
>
> You can debate whether "birth time" (which can't be set) or a
> "abstract creation time" (which can set to any arbitrary value), is
> "better" but that's why Linux doesn't support a way to set the "birth
> time".
>
> Whether you think we should bow to what Microsoft dictates probably
> depends on how much you believe Windows is a legacy operating system
> or not.  :-)  Personally, it's not something I really care about, and
> if someone really wants to add a Windows-compatible "Creation Time",
> my suggestion would be to define an extended attribute where this
> could be stored.
>
> We *could* allocate space in the on-disk inode to store this
> timestamp, but since I would estimate 99.9% of deployed Linux systems
> don't care about Windows compatibility, it's not a good use of
> resources.  We could also add a mount option which changes the
> semantics of birth time, but that adds extra complexity, and again, I
> would estimate that 99.9% of Linux systems (where I include all of the
> Linux deployments in Cloud VM's) don't care about Windows
> compatibility in this way.
>
> Cheers,
>
> 						- Ted

Hello,

I'm not going to argue with your reasoning but being able to set btime
could be beneficial for backup and restore purposes/utilities.

Secondly, I really like having separate modification and creation times
for all my files.

Finally, as for POSIX not offering this feature - doesn't Linux already
have a lot of syscalls that are not found in POSIX?

So, it's not just about Windows compatibility. It's just very useful.

Regards,
Artem

Re: A syscall for changing birth time

[Theodore Ts'o]

On Mon, Mar 10, 2025 at 02:11:13PM +0000, Artem S. Tashkinov wrote:
> 
> I'm not going to argue with your reasoning but being able to set btime
> could be beneficial for backup and restore purposes/utilities.

I think the fundmental question is should we change the current
defined semantics of btime, versus adding some new "crtime" field for
those people who care about Windows compatibility.

Granted, it's only been relatively recently that userspace could even
*access* btime for a file in the a mounted file system via the
statx(2) system call.  So there might not be that many users of btime
today, so changing the semantics might not break userspace (much).

Again, the choices are:

* Change semantics of "birth time" for everyone as of some new kernel version

* Make the semantics of "birth time" be subject to chagge via a mount option

* Add the ability to optionally store a new "creation time" which as
  Windows semantics in addition to btime.  This could either be via an
  xattr, or by adding a new on-disk inode field for some file systems
  that care about Windows compatibility.

> Secondly, I really like having separate modification and creation times
> for all my files.

Personally, I don't find it all that useful, since "creation time" is
really (with apologies to Lewis Carroll):

   “When I use a timestamp,’ Humpty Dumpty said in rather a scornful tone, ‘it
    means just what I choose it to mean — neither more nor less.’

    ’The question is,’ said Alice, ‘whether you can make timestamps mean so
    many different things.’

    ’The question is,’ said Humpty Dumpty, ‘which is to be master — that’s all.”

.... combined with the fact that most file systems don't support
creation time today.

Hence I find it more useful to find some other way to do version
stamping --- and something more reliable, like a SHA1 hash like git
uses.  Or as a nother example, using an external database, such as a
rpm or dpkg installed package database, with a crypto hash to verify
whether the file has actually changed, no matter what the package
manager thinks.

But that's just my personal opinion.  I am sure that other people
might find "creation time" to be useful.

> Finally, as for POSIX not offering this feature - doesn't Linux already
> have a lot of syscalls that are not found in POSIX?

What I was trying to point out that there is precedent in POSIX not
allowing certain timestamps to be changed, because that would violate
the semantics of ctime --- and the same would apply for btime's
current semantics.

Can we forcibly change btime's semantics?  Perhaps.  But let's do so
consciously, and not blindly follow Windows' way of doing things.

Cheers,

	     	     	     	    	     	 - Ted

Re: A syscall for changing birth time

[David Sterba]

On Mon, Mar 10, 2025 at 11:37:44AM -0400, Theodore Ts'o wrote:
> On Mon, Mar 10, 2025 at 02:11:13PM +0000, Artem S. Tashkinov wrote:
> > 
> > I'm not going to argue with your reasoning but being able to set btime
> > could be beneficial for backup and restore purposes/utilities.

From my experience and use case: for archiving purposes the creation
time has a meaning and it gets lost when the files get moved to bigger
storage. Sometimes the creation is stored in the file metadata (like jpeg
or video files), but ultimately the creation time is stored in the file
inode metadata itself regardless of the format.

Another use case is for send/receive that transfers snapshots from one
btrfs filesystem to another. In protocol v2 we added the creation time
(otime) to the stream so the receiving side can read it, but cannot
write it yet.

Question about ability to change otime/btime can be found around
internet, there was attempt to make it a utimensat() operation
(https://lore.kernel.org/linux-btrfs/cover.1550136164.git.osandov@fb.com).

In that thread there's a disagreement with providing that and I
disagreed as well back then, but over the time I changed my mind also
because I found a use case for myself.  So I have a WIP to add it as
ioctl to btrfs. I understand it's a niche use case and won't attempt to
implement it on the VFS level. At least for btrfs and the send/receive
use case it's slightly more justified than in other filesystems.

Re: A syscall for changing birth time

[Theodore Ts'o]

On Tue, Mar 11, 2025 at 05:08:39PM +0100, David Sterba wrote:
> 
> From my experience and use case: for archiving purposes the creation
> time has a meaning and it gets lost when the files get moved to bigger
> storage. Sometimes the creation is stored in the file metadata (like jpeg
> or video files), but ultimately the creation time is stored in the file
> inode metadata itself regardless of the format.
> 
> Another use case is for send/receive that transfers snapshots from one
> btrfs filesystem to another. In protocol v2 we added the creation time
> (otime) to the stream so the receiving side can read it, but cannot
> write it yet.
> 
> Question about ability to change otime/btime can be found around
> internet, there was attempt to make it a utimensat() operation
> (https://lore.kernel.org/linux-btrfs/cover.1550136164.git.osandov@fb.com).

How about this as a compromise?  We can define in statx two different
timestamp fields.  One is "btime" which is considered forensic
information which is not changeable by a system call interface, and
the other is "crtime" which *can* be changed by a system call
interface.  An individual file system is free to support one, or the
other, or both of these time stamp fields.

So file systems that come from the Windows world, including ntfs,
cifs, vfat, etc. can use crtime, and it can be set via some
standardized interface --- and file systems like xfs that want to
treat it as a non-changeable timestamp can only support btime.

What I might do with ext4 is to add a file system compat flag which
will cause the file system to support either crtime or btime, and
would use the on-dik inode field for one or the other.  That way I
don't break existing file system and potential use cases that want to
use btime, and allow those users who do want to use crtime in some of
the use cases that you've described to do so.  It does mean that a
single file system can only support "btime" or "crtime", but not both,
but it has the advantage that the semantics which are offered by a
particular file system are very clear --- and we don't have to gain
consensus across the various Linux file system maintainers about what
the One True Semantic for "btime/crttime" should be.  We can just have
both.

(And of course, some file systems might only support
mtime/ctime/atime, or only support a single timestamp value, and
that's also fine.)

						- Ted

Re: A syscall for changing birth time

[David Laight]

On Mon, 10 Mar 2025 09:58:28 -0400
"Theodore Ts'o" <tytso@mit.edu> wrote:

> On Mon, Mar 10, 2025 at 07:26:00AM +0000, Artem S. Tashkinov wrote:
> > 
> > Why is it that the Linux kernel supports reading btime, but there's no
> > syscall to change it? At least for ext4 there's the debugfs utility, but
> > for other filesystems there's just nothing. And even debugfs is not a
> > solution, since it requires root privileges and an unmounted/mounted RO
> > filesystem.  
> 
> POSIX and Single Unix Specification also doesn't provide a way to
> allow userspace to set ctime (inode change time).  That's because the
> definition of "change time" is defined to include the time to change
> anything about the inode metadata --- including the inode timestamps.
> 

I'm sure that hadn't used to be the case.
But as some point the 'ctime' changed from something that was usually
the file create time (for some definition of create) to a pretty useless
time that is almost a waste of disk space.

	David

Re: A syscall for changing birth time

[Al Viro]

On Mon, Mar 10, 2025 at 10:12:43PM +0000, David Laight wrote:
> On Mon, 10 Mar 2025 09:58:28 -0400
> "Theodore Ts'o" <tytso@mit.edu> wrote:
> 
> > On Mon, Mar 10, 2025 at 07:26:00AM +0000, Artem S. Tashkinov wrote:
> > > 
> > > Why is it that the Linux kernel supports reading btime, but there's no
> > > syscall to change it? At least for ext4 there's the debugfs utility, but
> > > for other filesystems there's just nothing. And even debugfs is not a
> > > solution, since it requires root privileges and an unmounted/mounted RO
> > > filesystem.  
> > 
> > POSIX and Single Unix Specification also doesn't provide a way to
> > allow userspace to set ctime (inode change time).  That's because the
> > definition of "change time" is defined to include the time to change
> > anything about the inode metadata --- including the inode timestamps.
> > 
> 
> I'm sure that hadn't used to be the case.

You are incorrect.

> But as some point the 'ctime' changed from something that was usually
> the file create time (for some definition of create) to a pretty useless
> time that is almost a waste of disk space.

Which point would that be, considering that v6 inodes have only two timestamps
(atime and mtime), while in v7 e.g. chmod() definitely does update ctime?

sys/sys4.c:chmod():
        ip->i_flag |= ICHG;
	...
	iput(ip);
sys/iget.c:iput():
        if(ip->i_count == 1) {
		...
		iupdat(ip, &time, &time);
sys/iget.c:iupdat():
                if(ip->i_flag&ICHG)
			dp->di_ctime = time;

And filesys(5) has this:
.I Di_ctime
records the time of last modification
to the inode or to the file, and is used
to determine whether it should be dumped.

... which ought to answer the question about the intended use of that thing.
Hopefully useful mnemonics for your next attempt at UNIX101:
	A: Accessed
	M: contents Modified
	C: Changed in any way, including metadata

Re: A syscall for changing birth time

[Theodore Ts'o]

On Mon, Mar 10, 2025 at 10:12:43PM +0000, David Laight wrote:
> 
> I'm sure that hadn't used to be the case.
> But as some point the 'ctime' changed from something that was usually
> the file create time (for some definition of create) to a pretty useless
> time that is almost a waste of disk space.

ctime was "inode change time" in BSD 4.2, released in 1983.  So it's
been "change time" and not "creation time" for at least 42 years.

It might have been "creation time" back in the V7 Unix days, but I'd
gently suggest that after over four decades, arguing that we get to
change the semantics of an inode timestamp because we think know
better.... is not really justification for breaking backwards
compatibility.


I'd also note that trying to add "creation time" has all sorts of
interesting questions.  The way all pretty much all text editors work
when you save some file, at least if they are compotently implemented,
is to *NOT* truncate the existing file, and then rewrite it with the
new contents, but rather, to write the contents to "foo.c.new", then
rename "foo.c" to "foo.c.old", and then rename "foo.c.new" to "foo.c".

So pretty much all text editors that exist in Unix today will result
in the "creation time" to be pretty much the same "last moification
time" (within a few milliseconds; however long it takes to write the
file).

So without changing pretty much all userspace tools which rewrite
files, adding a "creation time" to Linux would be pretty much useless.
For example, what should git do when you run "git checkout"?  Should
itfigure out which git commit a file was first tracked in git, and use
that time stamp for the file's "creation time".  That would take a
fairly large effort for git, especially if it needs to be performant
--- and is it *really* all that useful to know when a file was first
created in the git history and to make it available as a file system
timestamp?

This really goes to my question of exactly how useful the file
creation time concept really is.  Perhaps that's why the developers at
the UC Berkley made ctime be "inode change time", I suspect when they
authored the BSD Fast File System 42 years ago.  Personally, while I
don't find "change time" to be all that useful --- I find "creation
time" an order of magnitude *more* useless.  :-)

					- Ted

Re: A syscall for changing birth time

[Al Viro]

On Tue, Mar 11, 2025 at 12:49:35AM -0400, Theodore Ts'o wrote:

> This really goes to my question of exactly how useful the file
> creation time concept really is.  Perhaps that's why the developers at
> the UC Berkley made ctime be "inode change time", I suspect when they
> authored the BSD Fast File System 42 years ago.  Personally, while I
> don't find "change time" to be all that useful --- I find "creation
> time" an order of magnitude *more* useless.  :-)

The third timestamp had been introduced in v7 and it had been "change
time" from the very beginning, with incremental backups as stated
rationale in filesys(5).  "I'm sure that" from David means "I couldn't
be arsed to check my WAG"...

Re: A syscall for changing birth time

[Theodore Ts'o]

On Tue, Mar 11, 2025 at 04:56:35AM +0000, Al Viro wrote:
> On Tue, Mar 11, 2025 at 12:49:35AM -0400, Theodore Ts'o wrote:
> 
> > This really goes to my question of exactly how useful the file
> > creation time concept really is.  Perhaps that's why the developers at
> > the UC Berkley made ctime be "inode change time", I suspect when they
> > authored the BSD Fast File System 42 years ago.  Personally, while I
> > don't find "change time" to be all that useful --- I find "creation
> > time" an order of magnitude *more* useless.  :-)
> 
> The third timestamp had been introduced in v7 and it had been "change
> time" from the very beginning, with incremental backups as stated
> rationale in filesys(5).  "I'm sure that" from David means "I couldn't
> be arsed to check my WAG"...


I actually pulled down the V7 sources and there was a comment in
/usr/sys/h/ino.h which has a comment around the on-disk inode stating
"creation time" (see below).  These comments are also there up to
3BSD, and changed to "inode change time" in the BSD 4.2 sources,
probably coincident with the BSD Fast File System implementation.

So to be fair to David, I'm guessing this is what he saw.

I still maintain that "creation time" as a concept isn't terribly
useful, and that's probably *why* historical Unix systems have used
ctime as "change time" for decades.  Whether it's 42 years or 45 years
doesn't really change my point.

  		      		      	 - Ted

struct dinode
{
	unsigned short	di_mode;     	/* mode and type of file */
	short	di_nlink;    	/* number of links to file */
	short	di_uid;      	/* owner's user id */
	short	di_gid;      	/* owner's group id */
	off_t	di_size;     	/* number of bytes in file */
	char  	di_addr[40];	/* disk block addresses */
	time_t	di_atime;   	/* time last accessed */
	time_t	di_mtime;   	/* time last modified */
	time_t	di_ctime;   	/* time created */
};

Re: A syscall for changing birth time

[Al Viro]

On Tue, Mar 11, 2025 at 01:07:25PM -0400, Theodore Ts'o wrote:

> I actually pulled down the V7 sources and there was a comment in
> /usr/sys/h/ino.h which has a comment around the on-disk inode stating
> "creation time" (see below).  These comments are also there up to
> 3BSD, and changed to "inode change time" in the BSD 4.2 sources,
> probably coincident with the BSD Fast File System implementation.

Ken Thompson is at google these days - might be interesting to ask if
there was a point between v6 and v7 where it was used as 'creation
time'.

https://www.tuhs.org/Archive/Documentation/Papers/BSTJ/bstj57-6-1905.pdf
looks like a reference to 'creation time' in Apr 78, and man/man5/filsys.5
is Jan 79 (already matching actual behaviour of released v7).

I wonder if anything of Interdata 8/32 port survives...

Re: A syscall for changing birth time

[David Laight]

On Tue, 11 Mar 2025 13:07:25 -0400
"Theodore Ts'o" <tytso@mit.edu> wrote:

> On Tue, Mar 11, 2025 at 04:56:35AM +0000, Al Viro wrote:
> > On Tue, Mar 11, 2025 at 12:49:35AM -0400, Theodore Ts'o wrote:
> >   
> > > This really goes to my question of exactly how useful the file
> > > creation time concept really is.  Perhaps that's why the developers at
> > > the UC Berkley made ctime be "inode change time", I suspect when they
> > > authored the BSD Fast File System 42 years ago.  Personally, while I
> > > don't find "change time" to be all that useful --- I find "creation
> > > time" an order of magnitude *more* useless.  :-)  
> > 
> > The third timestamp had been introduced in v7 and it had been "change
> > time" from the very beginning, with incremental backups as stated
> > rationale in filesys(5).  "I'm sure that" from David means "I couldn't
> > be arsed to check my WAG"...  
> 
> 
> I actually pulled down the V7 sources and there was a comment in
> /usr/sys/h/ino.h which has a comment around the on-disk inode stating
> "creation time" (see below).  These comments are also there up to
> 3BSD, and changed to "inode change time" in the BSD 4.2 sources,
> probably coincident with the BSD Fast File System implementation.
> 
> So to be fair to David, I'm guessing this is what he saw.

Quite likely - it was a long time ago and I didn't take an 'offsite
backup' of the sources (and I definitely have nothing that will read
the system disk from an old 68010 box).
I didn't use Unix until the mid 80s - and I think that was SVR2 rather
than anything Berkeley. Most of the systems were SVR4 - around the time
of the initial collaboration between AT&T and Sun to get SMP working
(which, IIRC, pulled some BSD code into SVR4).

> I still maintain that "creation time" as a concept isn't terribly
> useful, and that's probably *why* historical Unix systems have used
> ctime as "change time" for decades.  Whether it's 42 years or 45 years
> doesn't really change my point.

I do have half a brain cell that remembers it not quite being 'file
create' time - probably just changes to di_mode, di_uid or di_gid.

Anyway it is all old history.

	David

> 
>   		      		      	 - Ted
> 
> struct dinode
> {
> 	unsigned short	di_mode;     	/* mode and type of file */
> 	short	di_nlink;    	/* number of links to file */
> 	short	di_uid;      	/* owner's user id */
> 	short	di_gid;      	/* owner's group id */
> 	off_t	di_size;     	/* number of bytes in file */
> 	char  	di_addr[40];	/* disk block addresses */
> 	time_t	di_atime;   	/* time last accessed */
> 	time_t	di_mtime;   	/* time last modified */
> 	time_t	di_ctime;   	/* time created */
> };
>