Tracking files across tree reorganizations

Tracking files across tree reorganizations

[H. Peter Anvin]

Did anything ever happen with that?

	-hpa

Re: Tracking files across tree reorganizations

[Petr Baudis]

Hah, here we go again. :-)

Dear diary, on Wed, Dec 14, 2005 at 10:15:59PM CET, I got a letter
where "H. Peter Anvin" <hpa@zytor.com> said that...
> Did anything ever happen with that?

Linus is against it.

Cogito will do it anyway ;-), when someone sends me a nice patch or when
I get to it (probably not very soon). I imagine it like this:

(a) User can explicitly note file moves / renames. We follow those notes.
Probably the most viable for recording the notes is appending them at
the tail of the commit message.

(b) If there are no notes for the given commit, we do the rename
autodetection already implemented in GIT. If it yields something,
we follow it.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: Tracking files across tree reorganizations

[H. Peter Anvin]

Petr Baudis wrote:
> Hah, here we go again. :-)
> 
> Dear diary, on Wed, Dec 14, 2005 at 10:15:59PM CET, I got a letter
> where "H. Peter Anvin" <hpa@zytor.com> said that...
> 
>>Did anything ever happen with that?
> 
> Linus is against it.
> 

I don't think so.  Linus is against the user having to explicitly record 
the moves, but we can detect the moves at the point of reorganization.

> Cogito will do it anyway ;-), when someone sends me a nice patch or when
> I get to it (probably not very soon). I imagine it like this:
> 
> (a) User can explicitly note file moves / renames. We follow those notes.
> Probably the most viable for recording the notes is appending them at
> the tail of the commit message.
> 
> (b) If there are no notes for the given commit, we do the rename
> autodetection already implemented in GIT. If it yields something,
> we follow it.

I don't see anything in Linus' posts that says (b) is unacceptable.

	-hpa

Re: Tracking files across tree reorganizations

[Petr Baudis]

Dear diary, on Thu, Dec 15, 2005 at 12:12:33AM CET, I got a letter
where "H. Peter Anvin" <hpa@zytor.com> said that...
> Petr Baudis wrote:
> >Hah, here we go again. :-)
> >
> >Dear diary, on Wed, Dec 14, 2005 at 10:15:59PM CET, I got a letter
> >where "H. Peter Anvin" <hpa@zytor.com> said that...
> >
> >>Did anything ever happen with that?
> >
> >Linus is against it.
> >
> 
> I don't think so.  Linus is against the user having to explicitly record 
> the moves, but we can detect the moves at the point of reorganization.

Linus' <Pine.LNX.4.58.0504150753440.7211@ppc970.osdl.org>:

 - you're doing the work at the wrong point for _another_ reason. You're
   freezing your (crappy) algorithm at tree creation time, and basically
   making it pointless to ever create something better later, because
   even if hardware and software improves, you've codified that "we have
   to have crappy information".

  I tend to agree with this now - although I disagree about his
performance point in the mail; but we can cache the autodetection
results out of commit objects to improve the performance, if it's
worth it (and I suspect it actually isn't, if you don't care about
the "A renamed to B and new A introduced, both in the same commit"
case).

  Note that the intent of the explicit rename recording is to really
record file reorganizations, not code refactoring. When you just
reorganize your tree, Linus' "meaningless special case" becomes very
meaningful, because at such point all the code inside the file travels.
But if you just move big chunks of code around and rename files based on
prevailing code origin or something (that's a weird thing to do but I've
seen people do it), then that's probably where the file as a whole
shouldn't be marked renamed.

  To encourage this practice, I might check the similarities of the
renamed files at the time of recording them, and print a big fat warning
to the user if it's less than say 90% or so. But I think that the huge
majority of cases where people want to rename is when they reorganize
their trees and move files as a whole, and that's why it's so useful
to support explicit renames recording.

> >Cogito will do it anyway ;-), when someone sends me a nice patch or when
> >I get to it (probably not very soon). I imagine it like this:
> >
> >(a) User can explicitly note file moves / renames. We follow those notes.
> >Probably the most viable for recording the notes is appending them at
> >the tail of the commit message.
> >
> >(b) If there are no notes for the given commit, we do the rename
> >autodetection already implemented in GIT. If it yields something,
> >we follow it.
> 
> I don't see anything in Linus' posts that says (b) is unacceptable.

  If we do it at the walk time, not commit time - I didn't emphasize
that in my previous mail while I should have.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: Tracking files across tree reorganizations

[H. Peter Anvin]

Petr Baudis wrote:
>>>
>>>(b) If there are no notes for the given commit, we do the rename
>>>autodetection already implemented in GIT. If it yields something,
>>>we follow it.
>>
>>I don't see anything in Linus' posts that says (b) is unacceptable.
> 
>   If we do it at the walk time, not commit time - I didn't emphasize
> that in my previous mail while I should have.

Exactly.  This is the right thing to do, because it means that just 
because you made the checkin with version foo, you're forever limited to 
the capabilities of version foo.

	-hpa

Re: Tracking files across tree reorganizations

[Linus Torvalds]

On Wed, 14 Dec 2005, Petr Baudis wrote:
> 
> Linus is against it.
> 
> Cogito will do it anyway ;-), when someone sends me a nice patch or when
> I get to it (probably not very soon). I imagine it like this:

I warn people that if cogito starts polluting the commit messages too 
much, I'll stop pulling from such trees.

		Linus

Re: Tracking files across tree reorganizations

[H. Peter Anvin]

Linus Torvalds wrote:
> 
> On Wed, 14 Dec 2005, Petr Baudis wrote:
> 
>>Linus is against it.
>>
>>Cogito will do it anyway ;-), when someone sends me a nice patch or when
>>I get to it (probably not very soon). I imagine it like this:
> 
> I warn people that if cogito starts polluting the commit messages too 
> much, I'll stop pulling from such trees.
> 

I agree, putting that into the commit messages sounds like a pretty bad 
thing.  If anything it should go in the commit header, possibly in the 
form of an object reference (with the object carrying the actual data.)

HOWEVER, I maintain that this is unnecessary (and, as Linus has pointed 
out several time, losing) -- we already detect renames without relying 
on commit-time metadata.  If it's too expensive to generate the metadata 
on every merge, it can be cached.

	-hpa

Re: Tracking files across tree reorganizations

[Junio C Hamano]

"H. Peter Anvin" <hpa@zytor.com> writes:

> HOWEVER, I maintain that this is unnecessary (and, as Linus has pointed 
> out several time, losing) -- we already detect renames without relying 
> on commit-time metadata.  If it's too expensive to generate the metadata 
> on every merge, it can be cached.

I agree; I was wondering why you brought it up again.

Re: Tracking files across tree reorganizations

[H. Peter Anvin]

Junio C Hamano wrote:
> "H. Peter Anvin" <hpa@zytor.com> writes:
> 
> 
>>HOWEVER, I maintain that this is unnecessary (and, as Linus has pointed 
>>out several time, losing) -- we already detect renames without relying 
>>on commit-time metadata.  If it's too expensive to generate the metadata 
>>on every merge, it can be cached.
> 
> 
> I agree; I was wondering why you brought it up again.
> 

Just because of Petr's comment about adding stuff to commit messages.

	-hpa

Re: Tracking files across tree reorganizations

[Petr Baudis]

Dear diary, on Thu, Dec 15, 2005 at 12:44:43AM CET, I got a letter
where "H. Peter Anvin" <hpa@zytor.com> said that...
> HOWEVER, I maintain that this is unnecessary (and, as Linus has pointed 
> out several time, losing) -- we already detect renames without relying 
> on commit-time metadata.  If it's too expensive to generate the metadata 
> on every merge, it can be cached.

Just for the record, I'm not convinced at all (there's probably no
point in elaborating the reasons again). I give up, though - I don't
know how to store the explicit rename information "invisibly" so that
Linus would be willing to merge commits containing it, and that would
make the whole thing pretty much pointless at least for the kernel.

I plan to revive some old patches changing large portions of cg-log
in the next few days, consequently making it trivial to add the
on-the-fly automatic renames detection to per-file cg-log.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: Tracking files across tree reorganizations

[Martin Langhoff]

On 12/15/05, Petr Baudis <pasky@suse.cz> wrote:
> in the next few days, consequently making it trivial to add the
> on-the-fly automatic renames detection to per-file cg-log.

Does it matter? Who makes a big file reorg and doesn't put "big tree
reorganization" in the commit message, and can we shoot him?

cheers,

martin

Re: Tracking files across tree reorganizations

[Martin Langhoff]

On 12/15/05, H. Peter Anvin <hpa@zytor.com> wrote:
> Did anything ever happen with that?

Perhaps I'm slow today, but git-merge -s recursive was supposed to do
it transparently (automagically). At least it was merged into git with
that excuse ;-)

Does it not work for you or am I missing something?


martin

Re: Tracking files across tree reorganizations

[H. Peter Anvin]

Martin Langhoff wrote:
> On 12/15/05, H. Peter Anvin <hpa@zytor.com> wrote:
> 
>>Did anything ever happen with that?
> 
> Perhaps I'm slow today, but git-merge -s recursive was supposed to do
> it transparently (automagically). At least it was merged into git with
> that excuse ;-)
> 
> Does it not work for you or am I missing something?
> 

I'll try it.

	-hpa