preserving mtime

preserving mtime

[Fabrizio Pollastri]

Hi all,
is it possible to tell git to preserve the file modification time in a 
checked out copy? It is useful when managing web files, where mtime is 
tested by spiders for download decisions.
Thank you, for any help.

--
Cheers,
F. Pollastri

Re: preserving mtime

[Andreas Ericsson]

Fabrizio Pollastri wrote:
> Hi all,
> is it possible to tell git to preserve the file modification time in a 
> checked out copy? It is useful when managing web files, where mtime is 
> tested by spiders for download decisions.

No. Doing so would seriously break build-systems. If you want you can
have a post-checkout hook that sets the mtime on all the files though.
You should be able to parse the time the commit being checked out was
made from HEAD.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: preserving mtime

[Jakub Narebski]

Fabrizio Pollastri wrote:

> is it possible to tell git to preserve the file modification time in a 
> checked out copy? It is useful when managing web files, where mtime is 
> tested by spiders for download decisions.

I don't quite understand. When git checks out copy, or switch branches,
or resets it does update only files which changed.

Git does not store mtime of files in tree object.
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: preserving mtime

[Junio C Hamano]

Fabrizio Pollastri <f.pollastri@inrim.it> writes:

> is it possible to tell git to preserve the file modification time in a
> checked out copy? It is useful when managing web files, where mtime is
> tested by spiders for download decisions.

"git checkout branchA" after "git checkout branchB" would not
touch "file" if "file" are identical between two branches, so
the modification time is already preserved.

If the contents of "file" from the version you would want to
check out is different from the version you previously checked
out, and you still want to keep the old timestamp, then you are
trying to do something that a normal SCM user would actively not
want (e.g. doing so would screw up the build systems such as
"make").  Such a specialized need usually is addressed by the
build and install procedure of the application (in your case, a
website management).  Maybe your current build procedure may
blindly copy when installing:

	install: web.html
        	$(install) web.html $(dest)/var/www/web.html

but you may want to ignore certain classes of changes and avoid
re-installing to help crawlers.  You would do:

	install = ./myinstall.sh
	install: web.html
        	$(install) web.html $(dest)/var/www/web.html

and then ./myinstall.sh might look like:

	#!/bin/sh
	test -f "$2" &&
        compare-ignoring-minor-changes "$1" "$2" && exit 0
	install "$1" "$2"

Re: preserving mtime

[Erik Warendorph]

* Fabrizio Pollastri <f.pollastri@inrim.it> [2007-11-16 10:33:45 +0100]:
>
> is it possible to tell git to preserve the file modification time in a 
> checked out copy? It is useful when managing web files, where mtime is 
> tested by spiders for download decisions.

You may find the script "git-set-file-times" in the GitWiki
useful:

  ExampleScripts - GitWiki
  Setting the timestamps of the files to the commit timestamp
  of the commit which last touched them
    <http://git.or.cz/gitwiki/ExampleScripts#head-a57deb2b4ab1e2de80ab5fd3c681a6055a9d3247>

You should of course pay attention to advice about (and
against) doing stuff like this, both in the description of that
script and in other postings on this list.  But as you are
using Git to manage web files and you (probably) don't care
about build systems such as "make", you should be pretty safe.

About the script:  I think it originally was made by Eric Wong
(= normalperson) who is also on this list.  I have just made a
tiny, tiny modification to it (adding " or s/\0$//" to the
elsif test).

I've also thought about adding a --prefix option to the script.
This would enable it to be used together with git-archive,
leaving the working directory alone and affecting the files in
the directory where the archive is extracted instead.  In this
way, you would distinguish between your working directory and
your "live" directory, and the command sequence

  git archive --prefix=foo/ HEAD | (cd /var/www/ && tar xf -)
  git-set-file-times --prefix=/var/www/foo/

would be part of the build system, publishing your working
directory to your "live" directory (in this case
/var/www/foo/).

-- 
Erik Warendorph <erik@warendorph.org>

Re: preserving mtime

[Wayne Davison]

On Fri, Nov 16, 2007 at 11:15:34AM +0100, Andreas Ericsson wrote:
>> is it possible to tell git to preserve the file modification time in
>> a checked out copy?

> Fabrizio Pollastri wrote:
> No. Doing so would seriously break build-systems.

I wish that the initial clone would set the modification time to the
commit time.  It would make the intial checkout have a more accurate
representation of when a file was last changed instead of all files
being set to the clone date.  Then, files that are being updated would
get their time set as they do now.  I supposed I'll just use the handy
git-set-file-times script (mentioned in another reply) every time I do
a clone.

..wayne..

Re: preserving mtime

[Mike Hommey]

On Sat, Nov 17, 2007 at 10:22:36AM -0800, Wayne Davison wrote:
> On Fri, Nov 16, 2007 at 11:15:34AM +0100, Andreas Ericsson wrote:
> >> is it possible to tell git to preserve the file modification time in
> >> a checked out copy?
> 
> > Fabrizio Pollastri wrote:
> > No. Doing so would seriously break build-systems.
> 
> I wish that the initial clone would set the modification time to the
> commit time.  It would make the intial checkout have a more accurate
> representation of when a file was last changed instead of all files
> being set to the clone date.  Then, files that are being updated would
> get their time set as they do now.  I supposed I'll just use the handy
> git-set-file-times script (mentioned in another reply) every time I do
> a clone.

For completeness, it would make sense to do so every time you git
checkout (like, when switching branches).

Mike

Re: preserving mtime

[Martin Langhoff]

On Nov 18, 2007 9:45 PM, Mike Hommey <mh@glandium.org> wrote:
> On Sat, Nov 17, 2007 at 10:22:36AM -0800, Wayne Davison wrote:
> > I wish that the initial clone would set the modification time to the
> > commit time.  It would make the intial checkout have a more accurate
> > representation of when a file was last changed instead of all files
...
> For completeness, it would make sense to do so every time you git
> checkout (like, when switching branches).

I do hope anyone doing those things is _very_ aware that the mtime
metadata has a specific meaning -- when did this specific file in this
filesystem last change -- and is used by many tools in that sense. You
are trying to use it for something else. Lots of things will break.

Like incremental backups, for example.

So no no NO. Not recommended. Stuff will break in new and surprising
ways. It'd be trivial to write a quick script that shows the data you
want from git in Perl/Python/etc. But don't use mtime. It's used for
other stuff. Actually used for other stuff. Don't replace that data
with time data you want to see, the actual users of mtime will break.

cheers,


m

Re: preserving mtime

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1360 bytes --]

On Sun, Nov 18, 2007 at 09:45:11 +0100, Mike Hommey wrote:
> On Sat, Nov 17, 2007 at 10:22:36AM -0800, Wayne Davison wrote:
> > On Fri, Nov 16, 2007 at 11:15:34AM +0100, Andreas Ericsson wrote:
> > >> is it possible to tell git to preserve the file modification time in
> > >> a checked out copy?
> > 
> > > Fabrizio Pollastri wrote:
> > > No. Doing so would seriously break build-systems.
> > 
> > I wish that the initial clone would set the modification time to the
> > commit time.  It would make the intial checkout have a more accurate
> > representation of when a file was last changed instead of all files
> > being set to the clone date.  Then, files that are being updated would
> > get their time set as they do now.  I supposed I'll just use the handy
> > git-set-file-times script (mentioned in another reply) every time I do
> > a clone.
> 
> For completeness, it would make sense to do so every time you git
> checkout (like, when switching branches).

 - That would still screw-up make hard. You know, checking out does NOT
   delete any untracked files.

 - There is no such thing as last modification time in git. Because there is
   no file history in git. (Besides, what would be last modification time of
   a file that was last modified in two parents, for example?)

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: preserving mtime

[Robin Rosenberg]

There's an FAQ on this topic on the wiki as it pops up every now and then.

-- robin

Re: preserving mtime

[David Brown]

On Sun, Nov 18, 2007 at 10:34:55PM +1300, Martin Langhoff wrote:

>I do hope anyone doing those things is _very_ aware that the mtime
>metadata has a specific meaning -- when did this specific file in this
>filesystem last change -- and is used by many tools in that sense. You
>are trying to use it for something else. Lots of things will break.
>
>Like incremental backups, for example.

'mtime' does _not_ have the specific meaning of 'when did this specific
file last change'.  That is the 'ctime' field.  'mtime' is also updated
when a file is modified, but can be changed by the user.  Many utilities
restore mtime to older values, including tar.

Any competent backup program would use 'ctime' for incremental backups,
unless they don't mind missing a vast majority of files on many machines.

The main thing I've seen mess up backup software is if the mtime is set
into the future.

However, it will make 'make' very confusing, since it uses the mtime to
determine if files are out of date.  If moving to an older version of a
file causes the file to become older, make won't recompile.  This is
arguably a defect in make, but that is how it works.

Preserving mtime definitely should not be a default.

David

Re: preserving mtime

[Martin Langhoff]

On Nov 19, 2007 7:47 AM, David Brown <git@davidb.org> wrote:
> On Sun, Nov 18, 2007 at 10:34:55PM +1300, Martin Langhoff wrote:
>
> >I do hope anyone doing those things is _very_ aware that the mtime
> >metadata has a specific meaning -- when did this specific file in this
> >filesystem last change -- and is used by many tools in that sense. You
> >are trying to use it for something else. Lots of things will break.
> >
> >Like incremental backups, for example.
>
> 'mtime' does _not_ have the specific meaning of 'when did this specific
> file last change'.  That is the 'ctime' field.  'mtime' is also updated
> when a file is modified, but can be changed by the user.  Many utilities
> restore mtime to older values, including tar.

Hmmm. After a bit of googling I've found conflicting descriptions of
the mtime/ctime semantics (I thought - for 10 years now - that ctime
was "creation time", it is "changed time"). Some people think that
anything that updates mtime also updates ctime, and others say the
opposite.

Wikipedia says (at http://en.wikipedia.org/wiki/MAC_times and
http://en.wikipedia.org/wiki/Stat_%28Unix%29 ) that mtime is about the
content, and ctime about metadata (owner, permissions, moved inode,
etc). Changes in content "touch" mtime + ctime.

With that in mind, I think it makes sense for things like make and
amanda to read mtime as referring to a real change of that concrete
file. The abstract notion of the file having changed in the big DSCM
in the sky is useful, but putting that data in mtime messes things up.

> However, it will make 'make' very confusing, since it uses the mtime to
> determine if files are out of date.  If moving to an older version of a
> file causes the file to become older, make won't recompile.  This is
> arguably a defect in make, but that is how it works.

It's not a bug in make. mtime has a definite meaning, and make is
using that meaning. Same with amanda.

cheers,


m

Re: preserving mtime

[David Brown]

On Mon, Nov 19, 2007 at 09:36:52AM +1300, Martin Langhoff wrote:

>Hmmm. After a bit of googling I've found conflicting descriptions of
>the mtime/ctime semantics (I thought - for 10 years now - that ctime
>was "creation time", it is "changed time"). Some people think that
>anything that updates mtime also updates ctime, and others say the
>opposite.

One of them is wrong.  All modifications to the file change the ctime.
Some modifications change the mtime.  There is also a call to change the
mtime (which will touch the ctime).  It's been this way for a long time.
I think most of the confusion comes from the 'c' in ctime.

It doesn't help that the Posix spec is so hard to read on this.  Basically,
you have to look up every command that might modify a file to figure out
which time changes it is supposed to modify.

>Wikipedia says (at http://en.wikipedia.org/wiki/MAC_times and
>http://en.wikipedia.org/wiki/Stat_%28Unix%29 ) that mtime is about the
>content, and ctime about metadata (owner, permissions, moved inode,
>etc). Changes in content "touch" mtime + ctime.
>
>With that in mind, I think it makes sense for things like make and
>amanda to read mtime as referring to a real change of that concrete
>file. The abstract notion of the file having changed in the big DSCM
>in the sky is useful, but putting that data in mtime messes things up.

Backup software should _never_ look at the mtime (other than to save it).
Both GNU tar and dump use the ctime field exclusively for incremental
purposes.

Think about this:

   wget .../linux-2.4.tar.gz
   tar -xzf linux-2.4.tar.gz

I've just expanded lots of files on my machine.  Tar is going to set the
mtime to the date they were at when the tarball was made, which was
probably several years ago.  It is crucial, though, that any backup
software I run still back these files up, since they are newly added.

There are backup programs that use mtime, but they are just broken, plain
and simple.

>> However, it will make 'make' very confusing, since it uses the mtime to
>> determine if files are out of date.  If moving to an older version of a
>> file causes the file to become older, make won't recompile.  This is
>> arguably a defect in make, but that is how it works.
>
>It's not a bug in make. mtime has a definite meaning, and make is
>using that meaning. Same with amanda.

It is very much a bug (well, a feature) in make.  But the whole date
comparison model of make is completely wrong.  It should rebuild a file if
it has changed, not if it is newer.  Most make replacements do something
more intelligent (often similar to the index cache git uses).

I haven't used Amanda for a while, but it at least used to do the right
thing (using ctime).  They might have had to break things to support FAT,
but I would guess it still works on a real filesystem.

David

Re: preserving mtime

[Johannes Schindelin]

Hi,

On Sat, 17 Nov 2007, Wayne Davison wrote:

> On Fri, Nov 16, 2007 at 11:15:34AM +0100, Andreas Ericsson wrote:
> >> is it possible to tell git to preserve the file modification time in
> >> a checked out copy?
> 
> > Fabrizio Pollastri wrote:
> > No. Doing so would seriously break build-systems.
> 
> I wish that the initial clone would set the modification time to the
> commit time.

Could you stop this discussion, please?  This subject comes up every once 
in a while, and in the meantime the archives contain more sane 
explanations why this would be a bad behaviour of git than I could tell 
off of my head.

So no, this is not a sane behaviour.

If you _must_ inist on this behaviour, write your own hook, don't tell 
anybody about it, especially not when things break.

Hth,
Dscho