How to fork a file (git cp ?)

How to fork a file (git cp ?)

[Mikhail T.]

I need to add a new thing to our project. The thing will be similar to what 
already exists. I'd like to "derive" the new files from the existing ones -- 
without altering them and by preserving the change-history.

This is not a separate branch -- the "forked" files will co-exist. I'd call this 
git-cp (analogous to git-mv), but it does not exist...

Any ideas? Thanks!

    -mi

Re: How to fork a file (git cp ?)

[Johannes Sixt]

Am 04.05.2011 19:56, schrieb Mikhail T.:
> I need to add a new thing to our project. The thing will be similar to
> what already exists. I'd like to "derive" the new files from the
> existing ones -- without altering them and by preserving the
> change-history.






















> 
> This is not a separate branch -- the "forked" files will co-exist. I'd
> call this git-cp (analogous to git-mv), but it does not exist...
> 
> Any ideas? Thanks!
> 
>    -mi
> 

Re: How to fork a file (git cp ?)

[Johannes Sixt]

(Sorry for the botched previous post - fat fingers)

Am 04.05.2011 19:56, schrieb Mikhail T.:
> I need to add a new thing to our project. The thing will be similar to
> what already exists. I'd like to "derive" the new files from the
> existing ones -- without altering them and by preserving the
> change-history.

You cannot. Git does not have such a thing as "copy-with-preserved-history".

You just cp the file and git add it. But you will not be able to follow
a history of the file.

-- Hannes

Re: How to fork a file (git cp ?)

[Stephen Bash]

----- Original Message -----
> From: "Johannes Sixt" <j6t@kdbg.org>
> Sent: Wednesday, May 4, 2011 2:22:13 PM
> Subject: Re: How to fork a file (git cp ?)
> 
> Am 04.05.2011 19:56, schrieb Mikhail T.:
> > I need to add a new thing to our project. The thing will be similar
> > to
> > what already exists. I'd like to "derive" the new files from the
> > existing ones -- without altering them and by preserving the
> > change-history.
> 
> You cannot. Git does not have such a thing as
> "copy-with-preserved-history".
> 
> You just cp the file and git add it. But you will not be able to
> follow a history of the file.

Log (and other commands) can search for copies while traversing history:

  -C[<n>]
  --find-copies[=<n>]
  Detect copies as well as renames. See also --find-copies-harder. If n is specified, it has the same meaning as for -M<n>.

  --find-copies-harder
  For performance reasons, by default, -C option finds copies only if the original file of the copy was modified in the same changeset. This flag makes the command inspect unmodified files as candidates for the source of copy. This is a very expensive operation for large projects, so use it with caution. Giving more than one -C option has the same effect.

But as I discovered a few weeks ago, the existing merge strategies don't understand copies (recursive can follow a rename, but if two files pass rename detection, I think the one with the higher similarity index wins).

Thanks,
Stephen

Re: How to fork a file (git cp ?)

[Mikhail T.]

On 04.05.2011 14:22, Johannes Sixt wrote:
> You just cp the file and git add it. But you will not be able to follow
> a history of the file.
Thank you for the information...

So, is this something worth adding to the wishlist, or was it omitted on purpose 
(and which purpose was that, then)?

Yours,

    -mi

Re: How to fork a file (git cp ?)

[Øyvind A. Holm]

On 4 May 2011 21:17, Mikhail T. <mi+thun@aldan.algebra.com> wrote:
> On 04.05.2011 14:22, Johannes Sixt wrote:
> >
> > You just cp the file and git add it. But you will not be able to
> > follow a history of the file.
>
> Thank you for the information...
>
> So, is this something worth adding to the wishlist, or was it omitted
> on purpose (and which purpose was that, then)?

Oh yes, that was intentional. This is easily one of the most debated
"features" of Git, especially in the early days of Git when almost all
SCM systems did it "the CVS way", by tracking the history of single
files. Instead, Git tracks snapshots of the whole tree and focuses on
the whole content instead of single files. Renames are tracked by
detecting removal/adding of files, which can be detected later, for
example using "git log --follow". The reason for this is mostly speed
issues, and most of the time the history of a single file is not
interesting in a project, but changes in the file tree as a whole.

From the FAQ at <https://git.wiki.kernel.org/index.php/GitFaq>:

  Git has to interoperate with a lot of different workflows, for example
  some changes can come from patches, where rename information may not
  be available. Relying on explicit rename tracking makes it impossible
  to merge two trees that have done exactly the same thing, except one
  did it as a patch (create/delete) and one did it using some other
  heuristic.

  On a second note, tracking renames is really just a special case of
  tracking how content moves in the tree. In some cases, you may instead
  be interested in querying when a function was added or moved to a
  different file. By only relying on the ability to recreate this
  information when needed, Git aims to provide a more flexible way to
  track how your tree is changing.

  However, this does not mean that Git has no support for renames. The
  diff machinery in Git has support for automatically detecting renames,
  this is turned on by the '-M' switch to the git-diff-* family of
  commands. The rename detection machinery is used by git-log(1) and
  git-whatchanged(1), so for example, 'git log -M' will give the commit
  history with rename information. Git also supports a limited form of
  merging across renames. The two tools for assigning blame,
  git-blame(1) and git-annotate(1) both use the automatic rename
  detection code to track renames.

  As a very special case, 'git log' version 1.5.3 and later has
  '--follow' option that allows you to follow renames when given a
  single path.

  Git has a rename command git mv, but that is just for convenience. The
  effect is indistinguishable from removing the file and adding another
  with different name and the same content.

This mail from Linus explains the issue in more detail and colour:
<http://permalink.gmane.org/gmane.comp.version-control.git/217>.

Regards,
Øyvind

Re: How to fork a file (git cp ?)

[Junio C Hamano]

Johannes Sixt <j6t@kdbg.org> writes:

> Am 04.05.2011 19:56, schrieb Mikhail T.:
>> I need to add a new thing to our project. The thing will be similar to
>> what already exists. I'd like to "derive" the new files from the
>> existing ones -- without altering them and by preserving the
>> change-history.
>
> You cannot. Git does not have such a thing as "copy-with-preserved-history".

Well, if you come from the mindset that a "file" has an identity (hence
there is a distinction between "This file used to be called A and at one
point was renamed to B which is the name we see today" and "Some time ago
somebody created a file B with the same contents as A and then removed A
at the same time"), "copy" would not make much sense.  What identity does
a new file B gets when you create it by copying from A?

The same identity, or a different one?  What happens when you later
refactor the redundant part from these two files to create a common third
file C?  What identity does C have?

> You just cp the file and git add it. But you will not be able to follow
> a history of the file.

Correct.  You cannot follow a history of _the file_, as there is no such
thing.

You can still follow the history of contents, though.  If you did a
refactor like the one in the above example, "blame -L <range>" would
follow the contents just fine.  The command is a 80% satisfactory
implementation of Linus's grand vision expressed in one of the most
important message in the git mailing list archive:

    http://thread.gmane.org/gmane.comp.version-control.git/27/focus=217

Re: How to fork a file (git cp ?)

[Mikhail T.]

On 04.05.2011 17:02, Junio C Hamano wrote:
> Well, if you come from the mindset that a "file" has an identity (hence
> there is a distinction between "This file used to be called A and at one
> point was renamed to B which is the name we see today" and "Some time ago
> somebody created a file B with the same contents as A and then removed A
> at the same time"), "copy" would not make much sense.  What identity does
> a new file B gets when you create it by copying from A?
What I want is to signify something like: "This code was obtained from 
that in file A."

"copy" -- of an individual file -- makes just as much sense as "move" 
(rename).

    -mi

Re: How to fork a file (git cp ?)

[Junio C Hamano]

"Mikhail T." <mi+thun@aldan.algebra.com> writes:

> On 04.05.2011 17:02, Junio C Hamano wrote:
>> Well, if you come from the mindset that a "file" has an identity (hence
>> there is a distinction between "This file used to be called A and at one
>> point was renamed to B which is the name we see today" and "Some time ago
>> somebody created a file B with the same contents as A and then removed A
>> at the same time"), "copy" would not make much sense.  What identity does
>> a new file B gets when you create it by copying from A?
> What I want is to signify something like: "This code was obtained from
> that in file A."

I think that is what exactly "blame -C -C" gives you.

Re: How to fork a file (git cp ?)

[Piotr Krukowiecki]

On Thu, May 5, 2011 at 4:14 AM, Junio C Hamano <gitster@pobox.com> wrote:
> "Mikhail T." <mi+thun@aldan.algebra.com> writes:
>
>> On 04.05.2011 17:02, Junio C Hamano wrote:
>>> Well, if you come from the mindset that a "file" has an identity (hence
>>> there is a distinction between "This file used to be called A and at one
>>> point was renamed to B which is the name we see today" and "Some time ago
>>> somebody created a file B with the same contents as A and then removed A
>>> at the same time"), "copy" would not make much sense.  What identity does
>>> a new file B gets when you create it by copying from A?
>> What I want is to signify something like: "This code was obtained from
>> that in file A."
>
> I think that is what exactly "blame -C -C" gives you.

Maybe Mikhail wanted to say that if there's a git-mv as a shortcut for
  "cp old new ; rm old; add new"
then there should be a git-cp as a shortcut for
  "cp old new; add new"

Just for convenience (and symmetry with git-mv).

He did write:
> "copy" -- of an individual file -- makes just as much sense as "move" (rename).

-- 
Piotr Krukowiecki

Re: How to fork a file (git cp ?)

[Junio C Hamano]

Piotr Krukowiecki <piotr.krukowiecki@gmail.com> writes:

> Maybe Mikhail wanted to say that if there's a git-mv as a shortcut for
>   "cp old new ; rm old; add new"
> then there should be a git-cp as a shortcut for
>   "cp old new; add new"

Copying and then futzing with a copy is a bad discipline to begin with.
git already has a reputation of having too many commands.  I am not
thrilled about the idea of making things worse by adding an unnecessary
command, and especially one that encourages a bad workflow.

Re: How to fork a file (git cp ?)

[Piotr Krukowiecki]

On Thu, May 5, 2011 at 8:50 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Piotr Krukowiecki <piotr.krukowiecki@gmail.com> writes:
>
>> Maybe Mikhail wanted to say that if there's a git-mv as a shortcut for
>>   "cp old new ; rm old; add new"
>> then there should be a git-cp as a shortcut for
>>   "cp old new; add new"
>
> Copying and then futzing with a copy is a bad discipline to begin with.
> git already has a reputation of having too many commands.  I am not
> thrilled about the idea of making things worse by adding an unnecessary
> command, and especially one that encourages a bad workflow.

I'm not saying I want to have git-cp, but could you explain what is wrong with
"cp old new; add new" (why is that bad workflow? How come adding a copy
of a file can be a workflow at all?)


-- 
Piotr Krukowiecki

Re: How to fork a file (git cp ?)

[Mikhail T.]

On 04.05.2011 22:14, Junio C Hamano wrote:
> I think that is what exactly "blame -C -C" gives you.
For that to be useful, one has to suspect, the file was derived by 
copying something else... Simple "git log" will not suggest that -- 
unless the commit message, that adds the new copy of a file points to it...

On 05.05.2011 14:02, Piotr Krukowiecki wrote:
> Maybe Mikhail wanted to say that if there's a git-mv as a shortcut for
>    "cp old new ; rm old; add new"
git-mv preserves the old's change-history in new, so it is more than the 
above, is not it?
> then there should be a git-cp as a shortcut for
Yes...

    -mi

Re: How to fork a file (git cp ?)

[Jeff King]

On Thu, May 05, 2011 at 03:31:22PM -0400, Mikhail T. wrote:

> On 04.05.2011 22:14, Junio C Hamano wrote:
> >I think that is what exactly "blame -C -C" gives you.
> For that to be useful, one has to suspect, the file was derived by
> copying something else... Simple "git log" will not suggest that --
> unless the commit message, that adds the new copy of a file points to
> it...

I think "git log --follow" will do what you want; it uses
FIND_COPIES_HARDER.

> On 05.05.2011 14:02, Piotr Krukowiecki wrote:
> >Maybe Mikhail wanted to say that if there's a git-mv as a shortcut for
> >   "cp old new ; rm old; add new"
> git-mv preserves the old's change-history in new, so it is more than
> the above, is not it?

No. The renames are detected at the time of viewing, not at the time of
commit.

-Peff

Re: How to fork a file (git cp ?)

[Piotr Krukowiecki]

On Thu, May 5, 2011 at 9:31 PM, Mikhail T. <mi+thun@aldan.algebra.com> wrote:
> On 04.05.2011 22:14, Junio C Hamano wrote:
>>
>> I think that is what exactly "blame -C -C" gives you.
>
> For that to be useful, one has to suspect, the file was derived by copying
> something else... Simple "git log" will not suggest that -- unless the
> commit message, that adds the new copy of a file points to it...

Maybe it should be the default (performance issues?)


> On 05.05.2011 14:02, Piotr Krukowiecki wrote:
>>
>> Maybe Mikhail wanted to say that if there's a git-mv as a shortcut for
>>   "cp old new ; rm old; add new"
>
> git-mv preserves the old's change-history in new, so it is more than the
> above, is not it?

It's the same IMO: log with --follow will follow both "copies" and "renames".


BTW, I don't understand why 'status' shows renames but not copies:

$ cp f fcp && git add fcp && git status
# Changes to be committed:
#	new file:   fcp

$ mv f fmv && git add fmv && git rm f && git status
# Changes to be committed:
#	renamed:    f -> fmv

I would expect sth like "copied: f -> fcp".
Not sure what about one file copied to multiple files, but I suppose
renames have the same problem.

It should not be a problem performance-wise...


>> then there should be a git-cp as a shortcut for
>
> Yes...



-- 
Piotr Krukowiecki

Re: How to fork a file (git cp ?)

[Piotr Krukowiecki]

On Thu, May 5, 2011 at 10:01 PM, Piotr Krukowiecki
<piotr.krukowiecki@gmail.com> wrote:
> BTW, I don't understand why 'status' shows renames but not copies:
>
> $ cp f fcp && git add fcp && git status
> # Changes to be committed:
> #       new file:   fcp
>
> $ mv f fmv && git add fmv && git rm f && git status
> # Changes to be committed:
> #       renamed:    f -> fmv
>
> I would expect sth like "copied: f -> fcp".
> Not sure what about one file copied to multiple files, but I suppose
> renames have the same problem.

I mean similar problem - you can delete one or more identical files and add one
or more new files with the same content.


> It should not be a problem performance-wise...


-- 
Piotr Krukowiecki

Re: How to fork a file (git cp ?)

[Jeff King]

On Thu, May 05, 2011 at 10:01:32PM +0200, Piotr Krukowiecki wrote:

> >> I think that is what exactly "blame -C -C" gives you.
> >
> > For that to be useful, one has to suspect, the file was derived by copying
> > something else... Simple "git log" will not suggest that -- unless the
> > commit message, that adds the new copy of a file points to it...
> 
> Maybe it should be the default (performance issues?)

Performance is part of it, but also the fact that "--follow" has some
limitations. For example, you can't use it with arbitrary pathspecs. I
hope to fix that at some point. There was a GSoC proposal, but it didn't
get selected; I'm hoping to work on it myself sometime this summer.

> BTW, I don't understand why 'status' shows renames but not copies:
> 
> $ cp f fcp && git add fcp && git status
> # Changes to be committed:
> #	new file:   fcp
> 
> $ mv f fmv && git add fmv && git rm f && git status
> # Changes to be committed:
> #	renamed:    f -> fmv
> 
> I would expect sth like "copied: f -> fcp".

Yeah, we probably should do copy detection. Even weirder, we seem to do
rename detection for what's to be committed, but not for what's in the
worktree.

If you want to do a patch, the changes would go in wt-status.c, in the
functions:

  wt_status_collect_changes_worktree
  wt_status_collect_changes_index

> It should not be a problem performance-wise...

For people running "git status" manually, no. But something like
FIND_COPIES_HARDER may be expensive on a big tree for people who use
"git status" output as part of their shell prompt. So probably it should
be configurable.

-Peff

Re: How to fork a file (git cp ?)

[Pete Harlan]

On 5/5/2011 1:01 PM, Piotr Krukowiecki wrote:
> On Thu, May 5, 2011 at 9:31 PM, Mikhail T.<mi+thun@aldan.algebra.com>  wrote:
>> On 04.05.2011 22:14, Junio C Hamano wrote:
>>>
>>> I think that is what exactly "blame -C -C" gives you.
>>
>> For that to be useful, one has to suspect, the file was derived by copying
>> something else... Simple "git log" will not suggest that -- unless the
>> commit message, that adds the new copy of a file points to it...
>
> Maybe it should be the default (performance issues?)

...

> BTW, I don't understand why 'status' shows renames but not copies:

Rename detection compares new files against deleted files.  Copy 
detection compares new files against every file in the tree, which is 
usually much more costly.

--Pete

Re: How to fork a file (git cp ?)

[Junio C Hamano]

Pete Harlan <pgit@pcharlan.com> writes:

> Rename detection compares new files against deleted files.  Copy
> detection compares new files against every file in the tree, which is
> usually much more costly.

Almost correct.  Copy detection usually compares new paths with paths
modified in the same commit.  "Every file in the tree" is done only when
you ask to find copies harder.