Proposal: tell git a file has been renamed

Proposal: tell git a file has been renamed

[Jeremy Morton]

Yes, I know Linus specifically doesn't store file rename info in Git. 
The trouble is, every now and then, I'll come across a situation where 
Git doesn't successfully detect that I've renamed a file because I'm 
doing something like renaming a class at the same time.  So I'll have 
a file OldClassNameTests.cs and a NewClassNameTests.cs but a bunch of 
lines in that file have also changed from OldClassName.DoThing() to 
NewClassName.DoThing().  I can clearly see that this is a rename, but 
Git sees enough changed content that it doesn't realize it, and puts 
it in as a delete/add, losing the content history.

The standard answer for this is to rename the file in one commit, then 
make the changes.  That's fine if you know ahead of time you'll want 
to do this.  However it's a total PITA if you have a bunch of changes 
and you realize that a rename has caused this problem.  You now have 
to back out your changes to the renamed file, add the rename, commit 
it, then re-apply the changes.

Could a command be added to git that means you tell Git that counts as 
a file rename?  Git would add a marker to the staging area that the 
file has been renamed, and upon commit, would first generate an 
additional commit for each rename before generating the main commit, 
ensuring the rename operation counts as an actual rename, and the 
content's history is maintained.

-- 
Best regards,
Jeremy Morton (Jez)

RE: Proposal: tell git a file has been renamed

[rsbecker]

On Saturday, April 22, 2023 2:02 PM, Jeremy Morton wrote:
>Yes, I know Linus specifically doesn't store file rename info in Git.
>The trouble is, every now and then, I'll come across a situation where Git doesn't
>successfully detect that I've renamed a file because I'm doing something like
>renaming a class at the same time.  So I'll have a file OldClassNameTests.cs and a
>NewClassNameTests.cs but a bunch of lines in that file have also changed from
>OldClassName.DoThing() to NewClassName.DoThing().  I can clearly see that this is a
>rename, but Git sees enough changed content that it doesn't realize it, and puts it in
>as a delete/add, losing the content history.
>
>The standard answer for this is to rename the file in one commit, then make the
>changes.  That's fine if you know ahead of time you'll want to do this.  However it's a
>total PITA if you have a bunch of changes and you realize that a rename has caused
>this problem.  You now have to back out your changes to the renamed file, add the
>rename, commit it, then re-apply the changes.
>
>Could a command be added to git that means you tell Git that counts as a file
>rename?  Git would add a marker to the staging area that the file has been renamed,
>and upon commit, would first generate an additional commit for each rename before
>generating the main commit, ensuring the rename operation counts as an actual
>rename, and the content's history is maintained.

Would git mv work in your situation? You can stage changes to the original file, then use git mv. Or use git mv first. The rename shows as staged in any event.
--Randall

Re: Proposal: tell git a file has been renamed

[Jeremy Morton]

I read that git mv is basically the equivalent to deleting the old 
file, creating the new file, and adding the changes.  Isn't it?  If so 
it's gonna have the same problem as I have now.

-- 
Best regards,
Jeremy Morton (Jez)

On 22/04/2023 19:54, rsbecker@nexbridge.com wrote:
> On Saturday, April 22, 2023 2:02 PM, Jeremy Morton wrote:
>> Yes, I know Linus specifically doesn't store file rename info in Git.
>> The trouble is, every now and then, I'll come across a situation where Git doesn't
>> successfully detect that I've renamed a file because I'm doing something like
>> renaming a class at the same time.  So I'll have a file OldClassNameTests.cs and a
>> NewClassNameTests.cs but a bunch of lines in that file have also changed from
>> OldClassName.DoThing() to NewClassName.DoThing().  I can clearly see that this is a
>> rename, but Git sees enough changed content that it doesn't realize it, and puts it in
>> as a delete/add, losing the content history.
>>
>> The standard answer for this is to rename the file in one commit, then make the
>> changes.  That's fine if you know ahead of time you'll want to do this.  However it's a
>> total PITA if you have a bunch of changes and you realize that a rename has caused
>> this problem.  You now have to back out your changes to the renamed file, add the
>> rename, commit it, then re-apply the changes.
>>
>> Could a command be added to git that means you tell Git that counts as a file
>> rename?  Git would add a marker to the staging area that the file has been renamed,
>> and upon commit, would first generate an additional commit for each rename before
>> generating the main commit, ensuring the rename operation counts as an actual
>> rename, and the content's history is maintained.
> 
> Would git mv work in your situation? You can stage changes to the original file, then use git mv. Or use git mv first. The rename shows as staged in any event.
> --Randall
> 
> 

RE: Proposal: tell git a file has been renamed

[rsbecker]

No, history is preserved in the rename.

>-----Original Message-----
>From: Jeremy Morton <admin@game-point.net>
>Sent: Saturday, April 22, 2023 3:45 PM
>To: rsbecker@nexbridge.com; git@vger.kernel.org
>Subject: Re: Proposal: tell git a file has been renamed
>
>I read that git mv is basically the equivalent to deleting the old file, creating the new
>file, and adding the changes.  Isn't it?  If so it's gonna have the same problem as I
>have now.
>
>--
>Best regards,
>Jeremy Morton (Jez)
>
>On 22/04/2023 19:54, rsbecker@nexbridge.com wrote:
>> On Saturday, April 22, 2023 2:02 PM, Jeremy Morton wrote:
>>> Yes, I know Linus specifically doesn't store file rename info in Git.
>>> The trouble is, every now and then, I'll come across a situation
>>> where Git doesn't successfully detect that I've renamed a file
>>> because I'm doing something like renaming a class at the same time.
>>> So I'll have a file OldClassNameTests.cs and a NewClassNameTests.cs
>>> but a bunch of lines in that file have also changed from
>>> OldClassName.DoThing() to NewClassName.DoThing().  I can clearly see
>>> that this is a rename, but Git sees enough changed content that it
>>> doesn't realize it, and puts it in as a delete/add, losing the content history.
>>>
>>> The standard answer for this is to rename the file in one commit,
>>> then make the changes.  That's fine if you know ahead of time you'll
>>> want to do this.  However it's a total PITA if you have a bunch of
>>> changes and you realize that a rename has caused this problem.  You
>>> now have to back out your changes to the renamed file, add the rename, commit
>it, then re-apply the changes.
>>>
>>> Could a command be added to git that means you tell Git that counts
>>> as a file rename?  Git would add a marker to the staging area that
>>> the file has been renamed, and upon commit, would first generate an
>>> additional commit for each rename before generating the main commit,
>>> ensuring the rename operation counts as an actual rename, and the content's
>history is maintained.
>>
>> Would git mv work in your situation? You can stage changes to the original file,
>then use git mv. Or use git mv first. The rename shows as staged in any event.
>> --Randall
>>
>>

Re: Proposal: tell git a file has been renamed

[Jeremy Morton]

https://stackoverflow.com/a/1094392/178757

says:

git mv oldname newname

is just shorthand for:

mv oldname newname
git add newname
git rm oldname

-- 
Best regards,
Jeremy Morton (Jez)

On 22/04/2023 20:47, rsbecker@nexbridge.com wrote:
> No, history is preserved in the rename.
> 
>> -----Original Message-----
>> From: Jeremy Morton <admin@game-point.net>
>> Sent: Saturday, April 22, 2023 3:45 PM
>> To: rsbecker@nexbridge.com; git@vger.kernel.org
>> Subject: Re: Proposal: tell git a file has been renamed
>>
>> I read that git mv is basically the equivalent to deleting the old file, creating the new
>> file, and adding the changes.  Isn't it?  If so it's gonna have the same problem as I
>> have now.
>>
>> --
>> Best regards,
>> Jeremy Morton (Jez)
>>
>> On 22/04/2023 19:54, rsbecker@nexbridge.com wrote:
>>> On Saturday, April 22, 2023 2:02 PM, Jeremy Morton wrote:
>>>> Yes, I know Linus specifically doesn't store file rename info in Git.
>>>> The trouble is, every now and then, I'll come across a situation
>>>> where Git doesn't successfully detect that I've renamed a file
>>>> because I'm doing something like renaming a class at the same time.
>>>> So I'll have a file OldClassNameTests.cs and a NewClassNameTests.cs
>>>> but a bunch of lines in that file have also changed from
>>>> OldClassName.DoThing() to NewClassName.DoThing().  I can clearly see
>>>> that this is a rename, but Git sees enough changed content that it
>>>> doesn't realize it, and puts it in as a delete/add, losing the content history.
>>>>
>>>> The standard answer for this is to rename the file in one commit,
>>>> then make the changes.  That's fine if you know ahead of time you'll
>>>> want to do this.  However it's a total PITA if you have a bunch of
>>>> changes and you realize that a rename has caused this problem.  You
>>>> now have to back out your changes to the renamed file, add the rename, commit
>> it, then re-apply the changes.
>>>>
>>>> Could a command be added to git that means you tell Git that counts
>>>> as a file rename?  Git would add a marker to the staging area that
>>>> the file has been renamed, and upon commit, would first generate an
>>>> additional commit for each rename before generating the main commit,
>>>> ensuring the rename operation counts as an actual rename, and the content's
>> history is maintained.
>>>
>>> Would git mv work in your situation? You can stage changes to the original file,
>> then use git mv. Or use git mv first. The rename shows as staged in any event.
>>> --Randall
>>>
>>>
> 
> 

RE: Proposal: tell git a file has been renamed

[rsbecker]

On Saturday, April 22, 2023 3:54 PM, Jeremy Morton wrote:
>Subject: Re: Proposal: tell git a file has been renamed
>
>https://stackoverflow.com/a/1094392/178757
>
>says:
>
>git mv oldname newname
>
>is just shorthand for:
>
>mv oldname newname
>git add newname
>git rm oldname

The above stackoverflow topic is from 2009. A lot has changed since then. My test follows. Please note the status and git log contents indicating the rename.

$ mkdir test2
$ cd test2
$ git init
$ echo "Initial" > file1
$ git add file1
$ commit -m "Commit 1"
$ git mv file1 file2
$ git status
On branch master
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        renamed:    file1 -> file2
$ git commit -m "Rename"
$ git log --patch
commit 014068fcedaf361f45c356046cf513b79537f53f (HEAD -> master)
Author: Randall S. Becker <rsbecker@nexbridge.com>
Date:   Sat Apr 22 18:02:07 2023 -0400

    Rename

diff --git a/file1 b/file2
similarity index 100%
rename from file1
rename to file2

commit 235a33801b82eac67e25c57e512ca428f2d49cea
Author: Randall S. Becker <rsbecker@nexbridge.com>
Date:   Sat Apr 22 18:01:48 2023 -0400

    Commit 1

diff --git a/file1 b/file1
new file mode 100644
index 0000000..a77fa51
--- /dev/null
+++ b/file1
@@ -0,0 +1 @@
+Initial

>
>On 22/04/2023 20:47, rsbecker@nexbridge.com wrote:
>> No, history is preserved in the rename.
>>
>>> -----Original Message-----
>>> From: Jeremy Morton <admin@game-point.net>
>>> Sent: Saturday, April 22, 2023 3:45 PM
>>> To: rsbecker@nexbridge.com; git@vger.kernel.org
>>> Subject: Re: Proposal: tell git a file has been renamed
>>>
>>> I read that git mv is basically the equivalent to deleting the old
>>> file, creating the new file, and adding the changes.  Isn't it?  If
>>> so it's gonna have the same problem as I have now.
>>>
>>> --
>>> Best regards,
>>> Jeremy Morton (Jez)
>>>
>>> On 22/04/2023 19:54, rsbecker@nexbridge.com wrote:
>>>> On Saturday, April 22, 2023 2:02 PM, Jeremy Morton wrote:
>>>>> Yes, I know Linus specifically doesn't store file rename info in Git.
>>>>> The trouble is, every now and then, I'll come across a situation
>>>>> where Git doesn't successfully detect that I've renamed a file
>>>>> because I'm doing something like renaming a class at the same time.
>>>>> So I'll have a file OldClassNameTests.cs and a NewClassNameTests.cs
>>>>> but a bunch of lines in that file have also changed from
>>>>> OldClassName.DoThing() to NewClassName.DoThing().  I can clearly
>>>>> see that this is a rename, but Git sees enough changed content that
>>>>> it doesn't realize it, and puts it in as a delete/add, losing the content history.
>>>>>
>>>>> The standard answer for this is to rename the file in one commit,
>>>>> then make the changes.  That's fine if you know ahead of time
>>>>> you'll want to do this.  However it's a total PITA if you have a
>>>>> bunch of changes and you realize that a rename has caused this
>>>>> problem.  You now have to back out your changes to the renamed
>>>>> file, add the rename, commit
>>> it, then re-apply the changes.
>>>>>
>>>>> Could a command be added to git that means you tell Git that counts
>>>>> as a file rename?  Git would add a marker to the staging area that
>>>>> the file has been renamed, and upon commit, would first generate an
>>>>> additional commit for each rename before generating the main
>>>>> commit, ensuring the rename operation counts as an actual rename,
>>>>> and the content's
>>> history is maintained.
>>>>
>>>> Would git mv work in your situation? You can stage changes to the
>>>> original file,
>>> then use git mv. Or use git mv first. The rename shows as staged in any event.
>>>> --Randall
>>>>
>>>>
>>
>>

Re: Proposal: tell git a file has been renamed

[Erik Cervin Edin]

On Sat, Apr 22, 2023 at 9:57 PM Jeremy Morton <admin@game-point.net> wrote:
>
> https://stackoverflow.com/a/1094392/178757
>
> says:
>
> git mv oldname newname
>
> is just shorthand for:
>
> mv oldname newname
> git add newname
> git rm oldname

That's what I though but it's more like:
  git show :0:oldname > newname
  git add newname
  git rm --cached oldname
  mv oldname newname

Basically, a move but also preserving staged/unstaged contents. But
yes. How git handles renames can sometimes be a bit of a PITA.

On Sun, Apr 23, 2023, 12:16 AM <rsbecker@nexbridge.com> wrote:
>
> $ git add file1
> $ commit -m "Commit 1"
> $ git mv file1 file2
> $ git status
> On branch master
> Changes to be committed:
>   (use "git restore --staged <file>..." to unstage)
>         renamed:    file1 -> file2

I think your point is better illustrated
  git add file1
  commit -m "Commit 1"
  echo 'Changed' > file1
  git mv file1 file2
  git status
Which yields:
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        renamed:    file1 -> file2

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   file2

Or perhaps even:
  echo Initial > file1
  seq 1 10 >> file1 # We need a larger file to detect rename with change
  git add file1
  git commit -m 'Commit 1'
  seq -i 's/Initial/Changed/' file1
  git add file1 # Stage changes
  seq 11 20 >> file1 # Add some unstaged changes
  git mv file1 file2

This will result in a staged rename, as well as the staged change
'Initial'  -> 'Changed' with a bunch of additional unstaged lines of
numbers 11 to 20.
NB that this doesn't play nice unless the contents of file1 in HEAD
and index are similar enough (at least 50%?)

Re: Proposal: tell git a file has been renamed

[Kristoffer Haugsbakk]

On Sat, Apr 22, 2023, at 20:01, Jeremy Morton wrote:
> Could a command be added to git that means you tell Git that counts as
> a file rename?  Git would add a marker to the staging area that the
> file has been renamed, and upon commit, would first generate an
> additional commit for each rename before generating the main commit,
> ensuring the rename operation counts as an actual rename, and the
> content's history is maintained.

I don’t see the (conceptual) problem with a modification of this as a
history rewriting tool:

• Given a series of commits:
• Tool X modifies all the commits so that the default similarity index
  for tools like git-log(1) is triggered on intended file renames
• The user will be probably be prompted with a list of initial potential
  renames and then will
  • Keep the intended renames
  • Remove the not-intended renames
  • Add the additional renames

But this looks like something that a third-party tool could implement.

-- 
Kristoffer Haugsbakk

Re: Proposal: tell git a file has been renamed

[Chris Torek]

(By the way, apologies for the stuttering "gitt" typos that seem
to keep creeping into my messages here. I'm not sure what's
causing them -- I make a lot of typos but this is abnormal!)

On Sat, Apr 22, 2023 at 12:22 PM Jeremy Morton <admin@game-point.net> wrote:

> [renames are a problem]

I have long wondered if there was a way to improve this
experience myself.

(I also note that most of the followup messages up to this point
have missed the point.  It's true that you can run `git mv` and
`git commit`, but you've already said that this becomes
particularly painful when you realize that it's appropriate *after
the fact*, when you've already made intermediate commits and/or
staged changes or whatever.)

> The standard answer for this is to rename the file in one commit, then
> make the changes.  That's fine if you know ahead of time you'll want
> to do this.  However it's a total PITA if you have a bunch of changes
> and you realize that a rename has caused this problem.  You now have
> to back out your changes to the renamed file, add the rename, commit
> it, then re-apply the changes.
>
> Could a command be added to git that means you tell Git that counts as
> a file rename?  Git would add a marker to the staging area that the
> file has been renamed, and upon commit, would first generate an
> additional commit for each rename before generating the main commit,
> ensuring the rename operation counts as an actual rename, and the
> content's history is maintained.

The index *currently* has no room to store anything like this: it
is, in effect, just the proposed next commit, stored as a
flattened tree.  There are, however, extra marker records that
can be added.  So:

If `git mv` (or a new command) had a flag to say "make a special
index entry so that the next `git commit` does a double commit",
we could in fact make this work.  Alternatively, we could have a
command -- similar to `git commit --only` in effect -- that uses
the current (HEAD) commit to construct a renames-only commit, in
which 100%-identical-file matching would (in general) find the
desired renames -- and make it, perhaps also co-ordinating with
`git mv` of existing files in the index.  (I'd also like to have
`git mv --after`, in the same vein as `hg mv --after`; I long ago
wrote a cheesy script to achieve this, but it would be nice to
have a proper command.)

On top of this, it might be nice to have a standardized commit
message and/or other marker (in the commit header?) for a "rename-
only" commit, which this kind of extra-rename-commit operation
would use.  Then `git log` and `git blame` and other commands
could easily detect such commits and default to an automatic
`--follow` style following, and `git log` might be allowed to omit
the *display* of such a commit by default, by showing all the
renames as renames in the subsequent commit (though this would
presumably require an internal verification step to check for
spoofed renames that are not in fact rename-only operations).

In any case, this *idea* is easy, like many ideas.  It really
comes down to implementation.  If someone thinks this is a great
idea, someone (perhaps me) should work on *implementing* it. :-)

Chris

Re: Proposal: tell git a file has been renamed

[Jeremy Morton]

On 24/04/2023 02:43, Chris Torek wrote:>> [renames are a problem]
> 
> I have long wondered if there was a way to improve this
> experience myself.
> 
> (I also note that most of the followup messages up to this point
> have missed the point.  It's true that you can run `git mv` and
> `git commit`, but you've already said that this becomes
> particularly painful when you realize that it's appropriate *after
> the fact*, when you've already made intermediate commits and/or
> staged changes or whatever.)
> 
> The index *currently* has no room to store anything like this: it
> is, in effect, just the proposed next commit, stored as a
> flattened tree.  There are, however, extra marker records that
> can be added.  So:
> 
> If `git mv` (or a new command) had a flag to say "make a special
> index entry so that the next `git commit` does a double commit",
> we could in fact make this work.  Alternatively, we could have a
> command -- similar to `git commit --only` in effect -- that uses
> the current (HEAD) commit to construct a renames-only commit, in
> which 100%-identical-file matching would (in general) find the
> desired renames -- and make it, perhaps also co-ordinating with
> `git mv` of existing files in the index.  (I'd also like to have

I'm not sure what the utility of the --only thing would be - to detect 
renames that didn't have changed content so that all renames could be 
done in one pre-commit?

> `git mv --after`, in the same vein as `hg mv --after`; I long ago
> wrote a cheesy script to achieve this, but it would be nice to
> have a proper command.)

Huh, I just read the docs on that... does that mean hg already has 
this functionality of being able to store a "this was renamed" marker 
in its index?

> On top of this, it might be nice to have a standardized commit
> message and/or other marker (in the commit header?) for a "rename-
> only" commit, which this kind of extra-rename-commit operation
> would use.  Then `git log` and `git blame` and other commands
> could easily detect such commits and default to an automatic
> `--follow` style following, and `git log` might be allowed to omit
> the *display* of such a commit by default, by showing all the
> renames as renames in the subsequent commit (though this would
> presumably require an internal verification step to check for
> spoofed renames that are not in fact rename-only operations).

I'm surprised --follow isn't the default, actually... isn't the whole 
point of detecting renames to allow content history to be tracked back 
through renames?

Another one that jumped to mind for me is bisect.  As rename-only 
commits are liable to create broken builds, it should skip over them 
to the 'content' commit.

> In any case, this *idea* is easy, like many ideas.  It really
> comes down to implementation.  If someone thinks this is a great
> idea, someone (perhaps me) should work on *implementing* it. :-)

If you do please feel free to CC me in an email about it, it'd be good 
to know if this became available!

-- 
Best regards,
Jeremy Morton (Jez)

Re: Proposal: tell git a file has been renamed

[Chris Torek]

On Mon, Apr 24, 2023 at 3:15 AM Jeremy Morton <admin@game-point.net> wrote:
> On 24/04/2023 02:43, Chris Torek wrote:
> > ... Alternatively, we could have a
> > command -- similar to `git commit --only` in effect

> I'm not sure what the utility of the --only thing would be - to detect
> renames that didn't have changed content so that all renames could be
> done in one pre-commit?

I mentioned `git commit --only` here just to point out that `git
commit` already has the ability to make a commit without using the
current index as the new commit's source.  A "just do renames"
commit operation (in spite of other changes already made in the
current index) would need similar functionality.

Exactly how this might work, I haven't defined.

> Huh, I just read the docs on [hg mv]... does that mean hg already has
> this functionality of being able to store a "this was renamed" marker
> in its index?

Mercurial does not have an index in the first place.  The internal
structure of the Mercurial database is an append-only series of
changelongs, with files stored as deltas from the previous version
of that file (with some exceptions).  Files are listed in a manifest,
and renaming a file preserves the file's identity despite the
change of name.

(This internal format is very different from Git's.  Git is a
content-addressable file system, rather than an append-only
changelog.)

Chris

Re: Proposal: tell git a file has been renamed

[Erik Cervin Edin]

On Mon, Apr 24, 2023 at 12:21 PM Jeremy Morton <admin@game-point.net> wrote:
>
> I'm surprised --follow isn't the default, actually... isn't the whole
> point of detecting renames to allow content history to be tracked back
> through renames?

You can set log to --follow by default
  If true, git log will act as if the --follow option was used when a
single <path> is given.
  This has the same limitations as --follow, i.e. it cannot be used to
follow multiple files
  and does not work well on non-linear history.
  https://git-scm.com/docs/git-log#Documentation/git-log.txt-logfollow

The reason it's not default is probably those limitations.

>
> Another one that jumped to mind for me is bisect.  As rename-only
> commits are liable to create broken builds, it should skip over them
> to the 'content' commit.
>

I always find this to be the main dilemma.
I try to make commits as discrete changes but it's not always possible
with renames.
Sometimes, renaming a file changes it so much that the rename
detection doesn't work by default.
There are also other problems that arise when reordering commits and
changes in a feature branch.
I've found that the safest thing is to split renames out into discrete
commits and only do 100% renames.

Re: Proposal: tell git a file has been renamed

[Jeremy Morton]

On 24/04/2023 11:49, Erik Cervin Edin wrote:

> I always find this to be the main dilemma.
> I try to make commits as discrete changes but it's not always possible
> with renames.
> Sometimes, renaming a file changes it so much that the rename
> detection doesn't work by default.
> There are also other problems that arise when reordering commits and
> changes in a feature branch.
> I've found that the safest thing is to split renames out into discrete
> commits and only do 100% renames.

There's no getting away from the fact that this adds a lot of (IMHO 
unnecessary) work if you've already done a rename that git can't 
detect and have both that and a bunch of other changes sitting in the 
index.  What feels like it would be a natural resolution in these 
cases, though, is a "no, this remove/add is actually a rename" command.

-- 
Best regards,
Jeremy Morton (Jez)

Re: Proposal: tell git a file has been renamed

[Erik Cervin Edin]

On Mon, Apr 24, 2023 at 1:17 PM Jeremy Morton <admin@game-point.net> wrote:
>
> There's no getting away from the fact that this adds a lot of (IMHO
> unnecessary) work if you've already done a rename that git can't
> detect and have both that and a bunch of other changes sitting in the
> index.  What feels like it would be a natural resolution in these
> cases, though, is a "no, this remove/add is actually a rename" command.

It can definitely be both arduous and non-obvious how to deal with this.

The problem is that such a command cannot exist atm. because renames
don't exist, they are only interpreted. So the only way to achieve
this is to revert enough of the contents staged to the index such that
the rename is detected. The only way to do that in a foolproof manner
is reverting all the staged changes except the path so that the moved
file in the index is identical to the old file in HEAD.

If I understand you correctly, your point here is that it's
non-trivial to go from a state where a file has been renamed, with
enough changes staged into the index that the rename hasn't been
detected, to a state where the rename is recorded in the index?

The most straightforward method I can think of to restore the staged
changes, do the rename without changes, commit and then go about the
rest
eg:
  git restore --staged file2
  mv file2 file1
  git mv file1 file2
  git commit -m rename

Re: Proposal: tell git a file has been renamed

[Jeremy Morton]

Yep, and that suggestion is rather arduous compared to something like:

git markasrenamed file2 file1
git commit

-- 
Best regards,
Jeremy Morton (Jez)

On 24/04/2023 15:00, Erik Cervin Edin wrote:
> On Mon, Apr 24, 2023 at 1:17 PM Jeremy Morton <admin@game-point.net> wrote:
>>
>> There's no getting away from the fact that this adds a lot of (IMHO
>> unnecessary) work if you've already done a rename that git can't
>> detect and have both that and a bunch of other changes sitting in the
>> index.  What feels like it would be a natural resolution in these
>> cases, though, is a "no, this remove/add is actually a rename" command.
> 
> It can definitely be both arduous and non-obvious how to deal with this.
> 
> The problem is that such a command cannot exist atm. because renames
> don't exist, they are only interpreted. So the only way to achieve
> this is to revert enough of the contents staged to the index such that
> the rename is detected. The only way to do that in a foolproof manner
> is reverting all the staged changes except the path so that the moved
> file in the index is identical to the old file in HEAD.
> 
> If I understand you correctly, your point here is that it's
> non-trivial to go from a state where a file has been renamed, with
> enough changes staged into the index that the rename hasn't been
> detected, to a state where the rename is recorded in the index?
> 
> The most straightforward method I can think of to restore the staged
> changes, do the rename without changes, commit and then go about the
> rest
> eg:
>    git restore --staged file2
>    mv file2 file1
>    git mv file1 file2
>    git commit -m rename
> 

Re: Proposal: tell git a file has been renamed

[Felipe Contreras]

Erik Cervin Edin wrote:
> On Mon, Apr 24, 2023 at 1:17 PM Jeremy Morton <admin@game-point.net> wrote:
> >
> > There's no getting away from the fact that this adds a lot of (IMHO
> > unnecessary) work if you've already done a rename that git can't
> > detect and have both that and a bunch of other changes sitting in the
> > index.  What feels like it would be a natural resolution in these
> > cases, though, is a "no, this remove/add is actually a rename" command.
> 
> It can definitely be both arduous and non-obvious how to deal with this.
> 
> The problem is that such a command cannot exist atm. because renames
> don't exist, they are only interpreted. So the only way to achieve
> this is to revert enough of the contents staged to the index such that
> the rename is detected. The only way to do that in a foolproof manner
> is reverting all the staged changes except the path so that the moved
> file in the index is identical to the old file in HEAD.

I agree recording renames explicitely might be a good addition to git, but the
real question is how are they going to be stored in the object storage.

My guess is that it can be added in the commit object after "committer", just
add a "renames" field with all the renames, or one "rename" field per rename.
It would be backwards compatible because any field can be added this way.

How to generate these fields is a separate issue: first things first.

-- 
Felipe Contreras

Re: Proposal: tell git a file has been renamed

[Jacob Keller]

On 4/24/2023 12:25 PM, Felipe Contreras wrote:
> Erik Cervin Edin wrote:
>> On Mon, Apr 24, 2023 at 1:17 PM Jeremy Morton <admin@game-point.net> wrote:
>>>
>>> There's no getting away from the fact that this adds a lot of (IMHO
>>> unnecessary) work if you've already done a rename that git can't
>>> detect and have both that and a bunch of other changes sitting in the
>>> index.  What feels like it would be a natural resolution in these
>>> cases, though, is a "no, this remove/add is actually a rename" command.
>>
>> It can definitely be both arduous and non-obvious how to deal with this.
>>
>> The problem is that such a command cannot exist atm. because renames
>> don't exist, they are only interpreted. So the only way to achieve
>> this is to revert enough of the contents staged to the index such that
>> the rename is detected. The only way to do that in a foolproof manner
>> is reverting all the staged changes except the path so that the moved
>> file in the index is identical to the old file in HEAD.
> 
> I agree recording renames explicitely might be a good addition to git, but the
> real question is how are they going to be stored in the object storage.
> 
> My guess is that it can be added in the commit object after "committer", just
> add a "renames" field with all the renames, or one "rename" field per rename.
> It would be backwards compatible because any field can be added this way.
> 
> How to generate these fields is a separate issue: first things first.
> 

The other end of the solution space is to try to find ways to make the
rename detection logic more accurate. We wouldn't need to store such a
rename if the detection gave the correct answer in the first place.

You can already kind of do this by modifying the similarity values when
searching for renames: the -M option takes a percentage for how similar
files need to be in order for it to be a rename. The default is 50%,
meaning that the files have to stay at least 50% the same in order to be
considered a rename.

One potential in the case for renames which also must change class
names, etc could be a mechanism to say "treat it as a rename if the
content is the same after performing this replacement", where you could
specify what string is being replaced in the rename somehow.

This could easily be done for reviewing a previous change, but
specifically how to *store* this expression so you don't need to
remember such values in the future is a bigger challenge.

Re: Proposal: tell git a file has been renamed

[Felipe Contreras]

Jacob Keller wrote:
> On 4/24/2023 12:25 PM, Felipe Contreras wrote:
> > Erik Cervin Edin wrote:
> >> On Mon, Apr 24, 2023 at 1:17 PM Jeremy Morton <admin@game-point.net> wrote:
> >>>
> >>> There's no getting away from the fact that this adds a lot of (IMHO
> >>> unnecessary) work if you've already done a rename that git can't
> >>> detect and have both that and a bunch of other changes sitting in the
> >>> index.  What feels like it would be a natural resolution in these
> >>> cases, though, is a "no, this remove/add is actually a rename" command.
> >>
> >> It can definitely be both arduous and non-obvious how to deal with this.
> >>
> >> The problem is that such a command cannot exist atm. because renames
> >> don't exist, they are only interpreted. So the only way to achieve
> >> this is to revert enough of the contents staged to the index such that
> >> the rename is detected. The only way to do that in a foolproof manner
> >> is reverting all the staged changes except the path so that the moved
> >> file in the index is identical to the old file in HEAD.
> > 
> > I agree recording renames explicitely might be a good addition to git, but the
> > real question is how are they going to be stored in the object storage.
> > 
> > My guess is that it can be added in the commit object after "committer", just
> > add a "renames" field with all the renames, or one "rename" field per rename.
> > It would be backwards compatible because any field can be added this way.
> > 
> > How to generate these fields is a separate issue: first things first.
> 
> The other end of the solution space is to try to find ways to make the
> rename detection logic more accurate. We wouldn't need to store such a
> rename if the detection gave the correct answer in the first place.

We can make it more accurate, but it will never be 100%. Sometimes being
explicit is the only way.

Cheers.

-- 
Felipe Contreras

Re: Proposal: tell git a file has been renamed

[Jacob Keller]

On 4/24/2023 12:25 PM, Felipe Contreras wrote:
> Erik Cervin Edin wrote:
>> On Mon, Apr 24, 2023 at 1:17 PM Jeremy Morton <admin@game-point.net> wrote:
>>>
>>> There's no getting away from the fact that this adds a lot of (IMHO
>>> unnecessary) work if you've already done a rename that git can't
>>> detect and have both that and a bunch of other changes sitting in the
>>> index.  What feels like it would be a natural resolution in these
>>> cases, though, is a "no, this remove/add is actually a rename" command.
>>
>> It can definitely be both arduous and non-obvious how to deal with this.
>>
>> The problem is that such a command cannot exist atm. because renames
>> don't exist, they are only interpreted. So the only way to achieve
>> this is to revert enough of the contents staged to the index such that
>> the rename is detected. The only way to do that in a foolproof manner
>> is reverting all the staged changes except the path so that the moved
>> file in the index is identical to the old file in HEAD.
> 
> I agree recording renames explicitely might be a good addition to git, but the
> real question is how are they going to be stored in the object storage.
> 
> My guess is that it can be added in the commit object after "committer", just
> add a "renames" field with all the renames, or one "rename" field per rename.
> It would be backwards compatible because any field can be added this way.
> 
> How to generate these fields is a separate issue: first things first.
> 

I think such a renames field might work, but I know historically there
was a lot of resistance to "baking in" such rename logic because you
can't "fix" it later.

I know there are some big threads from early days of git discussing
this, with many objections on baking in renames for various reasons.

I think the argument can be made that there is still value in a commit
which says "I as the commiter know this is renaming the file, please
show it as such and track it as such", and I think the other use cases
(code moving from one file to another without a full rename, multiple
copies of a file getting deleted, etc) can be handled in other ways or
with an escape hatch to disable the commit rename indicator if necessary.

In the more complex cases where code is moved without a full rename,
there are tools for handling this such as pickaxe or the proposed magic
blame tool that was brought up in the discussions against static renames
being coded into the object storage. I think those are interesting but
separate problems from "I am renaming this file and some of its contents".

At commit basically is the only way this could be baked into the object
storage, since blobs don't store filenames and trees don't store any
method of difference individually. The commit is the only place to
really do it.

The alternative would be something like Junio suggested where we have a
separate rename storage database which is computed ahead of time (kind
of like commit graph storage) and which could be overridden manually.
This doesn't "bake" anything into the objects, but obviously requires
manually keeping up to date and adds some difficulty in sharing such
updates across remotes.

Re: Proposal: tell git a file has been renamed

[Junio C Hamano]

Jacob Keller <jacob.e.keller@intel.com> writes:

> The alternative would be something like Junio suggested where we have a
> separate rename storage database which is computed ahead of time (kind
> of like commit graph storage) and which could be overridden manually.
> This doesn't "bake" anything into the objects, but obviously requires
> manually keeping up to date and adds some difficulty in sharing such
> updates across remotes.

I think originally the recorded rename mechanism was dreamed up to
be a cache.  Unlike commit-graph that requires precomputation, it
would be populated incrementally when you run "git log -M" in a part
of the history that you haven't done so.  Sharing it would just be
as hard as sharing notes, once we point the rename cache with a ref,
I would think.

I cannot take credit for the idea, though.

Re: Proposal: tell git a file has been renamed

[Erik Cervin Edin]

Another example of where the current implementation isn't working great

  A - B - C

commit A makes changes to foo
commit B changes the name of foo to bar
commit C symlinks foo => bar

Rebasing the commits and reordering them

  commit B changes the name of foo to bar
  commit C symlinks foo => bar
  commit A makes changes to foo

is going to result in merge conflicts. The best way to achieve this
ordering is actually doing multiple rebases.

  commit B changes the name of foo to bar
  commit A makes changes to foo
  commit C symlinks foo => bar

followed by

  commit B changes the name of foo to bar
  commit C symlinks foo => bar
  commit A makes changes to foo

I guess the difficulty that arises from reordering commits like this
isn't unique to renames but it arguably results in a more hostile user
experience. It requires an unusual amount of discipline and
understanding of what is going on to resolve as well as how to avoid
it.

Re: Proposal: tell git a file has been renamed

[Junio C Hamano]

Jeremy Morton <admin@game-point.net> writes:

> Could a command be added to git that means you tell Git that counts as
> a file rename?

Telling Git is the easy part.  How to make it honor what it was told
earlier is harder.

I recall that there was an idea about recording similarity index
between two blobs precomputed so that the same pair of blobs do not
have to be computed over and over again, when you do "git log -M"?

An obvious way to implement this "I know HEAD^:path1 and HEAD:path2
are similar enough that they deserve to be called a rename" can be
to invalidate and then contaminate such a similarity cache with
end-user supplied data.

Re: Proposal: tell git a file has been renamed

[Junio C Hamano]

Jeremy Morton <admin@game-point.net> writes:

> The standard answer for this is to rename the file in one commit, then
> make the changes.

Oh, by the way, this is a pure myth that would unlikely be helpful
in the bigger picture.

When you rename and heavily modify the resulting new path because
you have to solve something, such a work would likely be done on the
same topic branch.  One step of it may be a pure rename, and other
steps may involve heavily changing the renamed result, or you may
update the contents in the original and the do a rename at the end,
but either way, when you integrate the end result of the whole topic
branch into the master history, what such a merge will see is that
the original file has disappeared and a new file with contents not
at all similar to the disappeared file has appeared.  "pure rename
with changes in separate commits" would have no effect when showing
such a history with "git log --first-parent -p" for a birds-eye
view.

Re: Proposal: tell git a file has been renamed

[Jeremy Morton]

But if you were doing, say, a git blame for a file, you'd be able to 
see the commits where its lines were actually last modified even 
through the rename, rather than a bunch of them being stopped at the 
"added this file (because it wasn't detected as a rename)" commit, no?

-- 
Best regards,
Jeremy Morton (Jez)

On 24/04/2023 20:41, Junio C Hamano wrote:
> Jeremy Morton <admin@game-point.net> writes:
> 
>> The standard answer for this is to rename the file in one commit, then
>> make the changes.
> 
> Oh, by the way, this is a pure myth that would unlikely be helpful
> in the bigger picture.
> 
> When you rename and heavily modify the resulting new path because
> you have to solve something, such a work would likely be done on the
> same topic branch.  One step of it may be a pure rename, and other
> steps may involve heavily changing the renamed result, or you may
> update the contents in the original and the do a rename at the end,
> but either way, when you integrate the end result of the whole topic
> branch into the master history, what such a merge will see is that
> the original file has disappeared and a new file with contents not
> at all similar to the disappeared file has appeared.  "pure rename
> with changes in separate commits" would have no effect when showing
> such a history with "git log --first-parent -p" for a birds-eye
> view.
> 
>