naive question

naive question

[Paul Mackerras]

Is there a way to check out a tree without changing the mtime of any
files that you have already checked out and which are the same as the
version you are checking out?  It seems that checkout-cache -a doesn't
overwrite any existing files, and checkout-cache -f -a overwrites all
files and gives them the current mtime.  This is a pain if you are
using make and your tree is large (like, for instance, the linux
kernel :), because it means that after a checkout-cache -f -a you get
to recompile everything.

Paul.

Re: naive question

[David Woodhouse]

On Tue, 2005-04-19 at 23:00 +1000, Paul Mackerras wrote:
> Is there a way to check out a tree without changing the mtime of any
> files that you have already checked out and which are the same as the
> version you are checking out?  It seems that checkout-cache -a doesn't
> overwrite any existing files, and checkout-cache -f -a overwrites all
> files and gives them the current mtime.  This is a pain if you are
> using make and your tree is large (like, for instance, the linux
> kernel :), because it means that after a checkout-cache -f -a you get
> to recompile everything.

Corollary: why aren't we storing mtime in the tree objects?

-- 
dwmw2

Re: naive question

[Ingo Molnar]

* David Woodhouse <dwmw2@infradead.org> wrote:

> On Tue, 2005-04-19 at 23:00 +1000, Paul Mackerras wrote:
> > Is there a way to check out a tree without changing the mtime of any
> > files that you have already checked out and which are the same as the
> > version you are checking out?  It seems that checkout-cache -a doesn't
> > overwrite any existing files, and checkout-cache -f -a overwrites all
> > files and gives them the current mtime.  This is a pain if you are
> > using make and your tree is large (like, for instance, the linux
> > kernel :), because it means that after a checkout-cache -f -a you get
> > to recompile everything.
> 
> Corollary: why aren't we storing mtime in the tree objects?

Check the "[bug] git: check-files mtime problem?" thread - i noticed 
this problem before and gave a few suggestions but the discussion got 
nowhere. But the problem is still very much present.

	Ingo

Re: naive question

[Petr Baudis]

Dear diary, on Tue, Apr 19, 2005 at 03:00:14PM CEST, I got a letter
where Paul Mackerras <paulus@samba.org> told me that...
> Is there a way to check out a tree without changing the mtime of any
> files that you have already checked out and which are the same as the
> version you are checking out?  It seems that checkout-cache -a doesn't
> overwrite any existing files, and checkout-cache -f -a overwrites all
> files and gives them the current mtime.  This is a pain if you are
> using make and your tree is large (like, for instance, the linux
> kernel :), because it means that after a checkout-cache -f -a you get
> to recompile everything.

Actually, to then get sensible show-diff output, you need to also
update-cache --refresh to compensate for the changes. I personally
really hate update-cache --refresh; sure, 0.1s with hot cache, but
easily eats 5 minutes (!) with cold cache.

I'd actually prefer, if:

(i) checkout-cache simply wouldn't touch files whose stat matches with
what is in the cache; it updates the cache with the stat informations
of touched files

(ii) read-tree would take over the stat information from the matching
files in previous cache.

This way, doing update-cache --refresh would become a rather rare event.
Stuff would become swifter, faster, less I/O bound and you would get rid
of problems as the one described above.

What do you think?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: naive question

[Linus Torvalds]

On Tue, 19 Apr 2005, Petr Baudis wrote:
> 
> I'd actually prefer, if:
> 
> (i) checkout-cache simply wouldn't touch files whose stat matches with
> what is in the cache; it updates the cache with the stat informations
> of touched files

Run "update-cache --refresh" _before_ doing the "checkout-cache", and that 
is exactly what will happen.

But yes, if you want to make checkout-cache update the stat info (Ingo 
wanted to do that too), it should be possible. The end result is a 
combination of "update-cache" and "checkout-cache", though: you'll 
effectively need to both (just in one pass).

With the current setup, you have to do

	update-cache --refresh
	checkout-cache -f -a
	update-cache --refresh

which is admittedly fairly inefficient.

The real expense right now of a merge is that we always forget all the
stat information when we do a merge (since it does a read-tree). I have a
cunning way to fix that, though, which is to make "read-tree -m" read in
the old index state like it used to, and then at the end just throw it
away except for the stat information.

		Linus

Re: naive question

[Linus Torvalds]

On Tue, 19 Apr 2005, Linus Torvalds wrote:
> 
> The real expense right now of a merge is that we always forget all the
> stat information when we do a merge (since it does a read-tree). I have a
> cunning way to fix that, though, which is to make "read-tree -m" read in
> the old index state like it used to, and then at the end just throw it
> away except for the stat information.

Ok, done. That was really the plan all along, it just got dropped in the 
excitement of trying to get the dang thing to _work_ in the first place ;)

The current version only does

	read-tree -m <orig> <branch1> <branch2>

which now reads the old stat cache information, and then applies that to 
the end result of any trivial merges in case the merge result matches the 
old file stats. It really boils down to this littel gem;

            /*
             * See if we can re-use the old CE directly?
             * That way we get the uptodate stat info.
             */
            if (path_matches(result, old) && same(result, old))
                    *result = *old;


and it seems to work fine.

HOWEVER, I'll also make it do the same for a "single-tree merge":

	read-tree -m <newtree>

so that you can basically say "read a new tree, and merge the stat 
information from the current cache".  That means that if you do a
"read-tree -m <newtree>" followed by a "checkout-cache -f -a", the 
checkout-cache only checks out the stuff that really changed.

You'll still need to do an "update-cache --refresh" for the actual new
stuff. We could make "checkout-cache" update the cache too, but I really
do prefer a "checkout-cache only reads the index, never changes it"  
world-view. It's nice to be able to have a read-only git tree.

Final note: just doing a plain "read-tree <newtree>" will still throw all
the stat info away, and you'll have to refresh it all...

		Linus