Finding all commits which modify a file

Finding all commits which modify a file

[Neal Groothuis]

Hello,

I'm trying to find /all/ commits that change a file in the
repository...and its proving to be trickier than I thought. :-)

The situation that we were dealing with is this:

- Person A and person B both pull from the same central repository.

- Person A makes a change to file foo.txt and bar.txt, commits, and pushes
to the central repository.

- Person B makes a similar change to bar.txt and commits it.

- Person B does a fetch and merge.  Since both A and B made changes to
bar.txt, this requires conflicts to be resolved manually.

- B reverts A's changes to foo.txt. (If B is coming from a different
revision control system, this may happen due to confusion about how merges
are handled.)

- B commits the changes.

- B makes more changes to bar.txt, commits them, and pushes to the central
repository.

At this point, A's changes to foo.txt have been undone.

Graphically:

    A1
   /  ^
  v    \
  C1   B2<-B3
  ^    /
   \  v
    B1

B1, B2, and B3 have the same version of foo.txt as C1, A1 modifies it.

Person A discovers that his changes are missing and wants to know what
happened.

git log foo.txt doesn't help; it won't even show commit A1, due to history
simplification.

git log --full-history foo.txt will show commit A1.  It still won't show
commit B2, though, which we'd also like to show (because that's where the
change to foo.txt got removed).

I would think that git log --simplify-merges foo.txt would have done what
I'd wanted, but it still does not show commit B2.   Based on what I'm
reading in the man page, I would expect the simplification to go like
this:

    A1
    | ^
    |  \
    |  B2<-B3
    |  /
    v v
    C1

(since B1 is TREESAME as C1 if we're only considering foo.txt)

    A1
    | ^
    |  \
    |  B2<-B3
    |
    v
    C1

(since C1 is an ancestor of A1)

However, the actual output only includes A1, not B2.

 - Can someone explain this, and/or
 - can someone offer a command to display all commits (including merges)
in which ANY parent is not TREESAME?

Thanks!

- Neal

Re: Finding all commits which modify a file

[Neal Kreitzinger]

On 1/20/2012 3:35 PM, Neal Groothuis wrote:
> Hello,
>
> I'm trying to find /all/ commits that change a file in the
> repository...and its proving to be trickier than I thought. :-)
>
> The situation that we were dealing with is this:
>
> - Person A and person B both pull from the same central repository.
>
> - Person A makes a change to file foo.txt and bar.txt, commits, and pushes
> to the central repository.
>
> - Person B makes a similar change to bar.txt and commits it.
>
> - Person B does a fetch and merge.  Since both A and B made changes to
> bar.txt, this requires conflicts to be resolved manually.
>
> - B reverts A's changes to foo.txt. (If B is coming from a different
> revision control system, this may happen due to confusion about how merges
> are handled.)
>
> - B commits the changes.
>
> - B makes more changes to bar.txt, commits them, and pushes to the central
> repository.
>
> At this point, A's changes to foo.txt have been undone.
>
> Graphically:
>
>      A1
>     /  ^
>    v    \
>    C1   B2<-B3
>    ^    /
>     \  v
>      B1
>
> B1, B2, and B3 have the same version of foo.txt as C1, A1 modifies it.
>
> Person A discovers that his changes are missing and wants to know what
> happened.
>
> git log foo.txt doesn't help; it won't even show commit A1, due to history
> simplification.
>
> git log --full-history foo.txt will show commit A1.  It still won't show
> commit B2, though, which we'd also like to show (because that's where the
> change to foo.txt got removed).
>
> I would think that git log --simplify-merges foo.txt would have done what
> I'd wanted, but it still does not show commit B2.   Based on what I'm
> reading in the man page, I would expect the simplification to go like
> this:
>
>      A1
>      | ^
>      |  \
>      |  B2<-B3
>      |  /
>      v v
>      C1
>
> (since B1 is TREESAME as C1 if we're only considering foo.txt)
>
>      A1
>      | ^
>      |  \
>      |  B2<-B3
>      |
>      v
>      C1
>
> (since C1 is an ancestor of A1)
>
> However, the actual output only includes A1, not B2.
>
>   - Can someone explain this, and/or
>   - can someone offer a command to display all commits (including merges)
> in which ANY parent is not TREESAME?
>
Does git-log --all help?

v/r,
neal

Re: Finding all commits which modify a file

[Neal Groothuis]

> On 1/20/2012 3:35 PM, Neal Groothuis wrote:
>> I'm trying to find /all/ commits that change a file in the
>> repository...and its proving to be trickier than I thought. :-)

On 1/21/2012 6:16 PM, Neal Kreitzinger wrote:
> Does git-log --all help?

I don't see how it would.  The commits are all reachable from HEAD, which
would seem to be the problem that --all would correct.

What I'm trying to do is find the commits in which a file differs from
that same file in any of its parents.

If I'm missing something, could you provide an example of using git-log
--all to accomplish this?

Re: Finding all commits which modify a file

[Santi Béjar]

[-- Attachment #1: Type: text/plain, Size: 3104 bytes --]

[Note: CC main authors of the code surrounding the patch]

On Mon, Jan 23, 2012 at 5:14 PM, Neal Groothuis <ngroot@lo-cal.org> wrote:
>> On 1/20/2012 3:35 PM, Neal Groothuis wrote:
>>> I'm trying to find /all/ commits that change a file in the
>>> repository...and its proving to be trickier than I thought. :-)
>
> On 1/21/2012 6:16 PM, Neal Kreitzinger wrote:
>> Does git-log --all help?
>
> I don't see how it would.  The commits are all reachable from HEAD, which
> would seem to be the problem that --all would correct.
>
> What I'm trying to do is find the commits in which a file differs from
> that same file in any of its parents.

If you add parent rewriting (--parent, --graph or see it in gitk, with
--full-history) you'll get your B2 commit as it adds commits to have a
meaningful history. But I don't think this is what you are asking for.

  You could try the following patch (sorry for the whitespace damage,
also attatched):

Subject: [PATCH/RFC] revision: merging branches with different content
is interesting in --full-history

---
 revision.c                                 |    2 +-
 t/t6016-rev-list-graph-simplify-history.sh |    1 +
 2 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/revision.c b/revision.c
index 064e351..db97250 100644
--- a/revision.c
+++ b/revision.c
@@ -492,7 +492,7 @@ static void try_to_simplify_commit(struct rev_info
*revs, struct commit *commit)
                }
                die("bad tree compare for commit %s",
sha1_to_hex(commit->object.sha1));
        }
-       if (tree_changed && !tree_same)
+       if ((tree_changed && !tree_same) || (!revs->simplify_history
&& tree_changed))
                return;
        commit->object.flags |= TREESAME;
 }
diff --git a/t/t6016-rev-list-graph-simplify-history.sh
b/t/t6016-rev-list-graph-simplify-history.sh
index f7181d1..50ffcf4 100755
--- a/t/t6016-rev-list-graph-simplify-history.sh
+++ b/t/t6016-rev-list-graph-simplify-history.sh
@@ -168,6 +168,7 @@ test_expect_success '--graph --full-history
--simplify-merges -- bar.txt' '
        echo "|\\  " >> expected &&
        echo "| * $C4" >> expected &&
        echo "* | $A5" >> expected &&
+       echo "* | $A4" >> expected &&
        echo "* | $A3" >> expected &&
        echo "|/  " >> expected &&
        echo "* $A2" >> expected &&

(I could rewrite the condition but I think it is cleaner).

This patch changes the semantics of --full-history to consider all
commits with at least one modified parent as an interesting commit
(even for merges). This is almost as enabling --parent:

git $ git rev-list --full-history HEAD Makefile | wc -l
1769

$ git rev-list --full-history --parents HEAD Makefile | wc -l
6732

git $ ./git rev-list --full-history HEAD Makefile | wc -l
6052

I think that --full-history should list these extra merges as you are
asking for the full history and a merge merging two branches with
different content is an interesting event in this case (full history).
But maybe we should just add an extra flag...

HTH,
Santi

[-- Attachment #2: 0001-revision-merging-branches-with-different-content-is-.patch --]
[-- Type: text/x-patch, Size: 1446 bytes --]

From 36a09f1a39212eb4b45268215cc9c336a7afebda Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Santi=20B=C3=A9jar?= <santi@agolina.net>
Date: Tue, 24 Jan 2012 00:46:23 +0100
Subject: [PATCH] revision: merging branches with different content is interesting in --full-history

---
 revision.c                                 |    2 +-
 t/t6016-rev-list-graph-simplify-history.sh |    1 +
 2 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/revision.c b/revision.c
index 064e351..db97250 100644
--- a/revision.c
+++ b/revision.c
@@ -492,7 +492,7 @@ static void try_to_simplify_commit(struct rev_info *revs, struct commit *commit)
 		}
 		die("bad tree compare for commit %s", sha1_to_hex(commit->object.sha1));
 	}
-	if (tree_changed && !tree_same)
+	if ((tree_changed && !tree_same) || (!revs->simplify_history && tree_changed))
 		return;
 	commit->object.flags |= TREESAME;
 }
diff --git a/t/t6016-rev-list-graph-simplify-history.sh b/t/t6016-rev-list-graph-simplify-history.sh
index f7181d1..50ffcf4 100755
--- a/t/t6016-rev-list-graph-simplify-history.sh
+++ b/t/t6016-rev-list-graph-simplify-history.sh
@@ -168,6 +168,7 @@ test_expect_success '--graph --full-history --simplify-merges -- bar.txt' '
 	echo "|\\  " >> expected &&
 	echo "| * $C4" >> expected &&
 	echo "* | $A5" >> expected &&
+	echo "* | $A4" >> expected &&
 	echo "* | $A3" >> expected &&
 	echo "|/  " >> expected &&
 	echo "* $A2" >> expected &&
-- 
1.7.4.rc3.8.gd2d4

Re: Finding all commits which modify a file

[Santi Béjar]

On Tue, Jan 24, 2012 at 1:58 AM, Santi Béjar <santi@agolina.net> wrote:
> [Note: CC main authors of the code surrounding the patch]
>
> On Mon, Jan 23, 2012 at 5:14 PM, Neal Groothuis <ngroot@lo-cal.org> wrote:
>>> On 1/20/2012 3:35 PM, Neal Groothuis wrote:
>>>> I'm trying to find /all/ commits that change a file in the
>>>> repository...and its proving to be trickier than I thought. :-)
>>
>> On 1/21/2012 6:16 PM, Neal Kreitzinger wrote:
>>> Does git-log --all help?
>>
>> I don't see how it would.  The commits are all reachable from HEAD, which
>> would seem to be the problem that --all would correct.
>>
>> What I'm trying to do is find the commits in which a file differs from
>> that same file in any of its parents.
>
> If you add parent rewriting (--parent, --graph or see it in gitk, with
> --full-history) you'll get your B2 commit as it adds commits to have a
> meaningful history. But I don't think this is what you are asking for.

Note that even if it get listed, you won't get a diff for foo.txt
because it is an evil merge as the result is not the expected (using
the three way merge) one but is equal to one of the branches.

HTH,
Santi

Re: Finding all commits which modify a file

[Linus Torvalds]

On Mon, Jan 23, 2012 at 4:58 PM, Santi Béjar <santi@agolina.net> wrote:
>
> If you add parent rewriting (--parent, --graph or see it in gitk, with
> --full-history) you'll get your B2 commit as it adds commits to have a
> meaningful history. But I don't think this is what you are asking for.
>
>  You could try the following patch (sorry for the whitespace damage,
> also attatched):
>
> Subject: [PATCH/RFC] revision: merging branches with different content
> is interesting in --full-history

The concept seems sane.

But please check the interaction with "--simplify-merges" too, just in
case. The merge simplification looks at TREESAME too, so I suspect
your change may break merge simplification.

                      Linus

Re: Finding all commits which modify a file

[Santi Béjar]

On Tue, Jan 24, 2012 at 2:15 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Mon, Jan 23, 2012 at 4:58 PM, Santi Béjar <santi@agolina.net> wrote:
>>
>> If you add parent rewriting (--parent, --graph or see it in gitk, with
>> --full-history) you'll get your B2 commit as it adds commits to have a
>> meaningful history. But I don't think this is what you are asking for.
>>
>>  You could try the following patch (sorry for the whitespace damage,
>> also attatched):
>>
>> Subject: [PATCH/RFC] revision: merging branches with different content
>> is interesting in --full-history
>
> The concept seems sane.
>
> But please check the interaction with "--simplify-merges" too, just in
> case. The merge simplification looks at TREESAME too, so I suspect
> your change may break merge simplification.

Indeed, there is a bad interaction with --simplify-merges. If you add
--simplify-merges it not only increase the number of commit but
changes them :-(

$ ./git rev-list --full-history --simplify-merges HEAD Makefile >
rev-list.simp-merges
$ ./git rev-list --full-history HEAD Makefile > rev-list.new
$ diff rev-list.new rev-list.simp-merges -u | diffstat
 rev-list.simp-merges | 1841 ++++++++++++++++++++++++++-------------------------
 1 file changed, 944 insertions(+), 897 deletions(-)

Santi

Re: Finding all commits which modify a file

[Santi Béjar]

On Tue, Jan 24, 2012 at 2:36 AM, Santi Béjar <santi@agolina.net> wrote:
> On Tue, Jan 24, 2012 at 2:15 AM, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
>> On Mon, Jan 23, 2012 at 4:58 PM, Santi Béjar <santi@agolina.net> wrote:
>>>
>>> If you add parent rewriting (--parent, --graph or see it in gitk, with
>>> --full-history) you'll get your B2 commit as it adds commits to have a
>>> meaningful history. But I don't think this is what you are asking for.
>>>
>>>  You could try the following patch (sorry for the whitespace damage,
>>> also attatched):
>>>
>>> Subject: [PATCH/RFC] revision: merging branches with different content
>>> is interesting in --full-history
>>
>> The concept seems sane.
>>
>> But please check the interaction with "--simplify-merges" too, just in
>> case. The merge simplification looks at TREESAME too, so I suspect
>> your change may break merge simplification.
>
> Indeed, there is a bad interaction with --simplify-merges. If you add
> --simplify-merges it not only increase the number of commit but
> changes them :-(
>
> $ ./git rev-list --full-history --simplify-merges HEAD Makefile >
> rev-list.simp-merges
> $ ./git rev-list --full-history HEAD Makefile > rev-list.new
> $ diff rev-list.new rev-list.simp-merges -u | diffstat
>  rev-list.simp-merges | 1841 ++++++++++++++++++++++++++-------------------------
>  1 file changed, 944 insertions(+), 897 deletions(-)

Ops, it even happens without my patch...

I think it is OK, it just redefines what is TREESAME, and use the new
meaning in:

* If after this parent rewriting, `C'` is a root or merge commit (has
  zero or >1 parents), a boundary commit, or !TREESAME, it remains.
  Otherwise, it is replaced with its only parent.

We could keep the old meaning if --simplify-merges or we could have a
flag to just change the meaning of TREESAME for merges
(--with-all-interesting-merges? I'm not good at naming flags...)

Santi

Re: Finding all commits which modify a file

[Junio C Hamano]

Santi Béjar <santi@agolina.net> writes:

> diff --git a/revision.c b/revision.c
> index 064e351..db97250 100644
> --- a/revision.c
> +++ b/revision.c
> @@ -492,7 +492,7 @@ static void try_to_simplify_commit(struct rev_info
> *revs, struct commit *commit)
>                 }
>                 die("bad tree compare for commit %s",
> sha1_to_hex(commit->object.sha1));
>         }
> -       if (tree_changed && !tree_same)
> +       if ((tree_changed && !tree_same) || (!revs->simplify_history
> && tree_changed))

Is that the same as saying this?

	if (!(tree_same && revs->simplify_history) && tree_changed)
		return;

Which reads: unless we find a parent that matches the result and we are
simplifying the history, a child different from at least one parent is
worth showing.

Which makes sort of sense at the conceptual level, but I am not sure if it
is practically useful (the same issue with --full-history that makes
complex history almost unreadable).

Re: Finding all commits which modify a file

[Neal Groothuis]

> On Mon, Jan 23, 2012 at 5:14 PM, Neal Groothuis <ngroot@lo-cal.org> wrote:
>>> On 1/20/2012 3:35 PM, Neal Groothuis wrote:
>>>> I'm trying to find /all/ commits that change a file in the
>>>> repository...and its proving to be trickier than I thought. :-)
>>
>> On 1/21/2012 6:16 PM, Neal Kreitzinger wrote:
>>> Does git-log --all help?
>>
>> I don't see how it would.  The commits are all reachable from HEAD,
>> which
>> would seem to be the problem that --all would correct.
>>
>> What I'm trying to do is find the commits in which a file differs from
>> that same file in any of its parents.
>
> If you add parent rewriting (--parent, --graph or see it in gitk, with
> --full-history) you'll get your B2 commit as it adds commits to have a
> meaningful history. But I don't think this is what you are asking for.

Correct.  If I add parent rewriting, I get all merges, even those in which
the file is not changed from either parent.

Based on what's in the man page for git log about the history
simplification algorithm, it seems that B2 should be included in the
output when I do a git log --full-history --simplify-history foo.txt, as
per the steps I noted in the original post.  Is my understanding of the
algorithm faulty?

Re: Finding all commits which modify a file

[Santi Béjar]

[ Do not cut the CC]

On Tue, Jan 24, 2012 at 5:34 PM, Neal Groothuis <ngroot@lo-cal.org> wrote:
>> On Mon, Jan 23, 2012 at 5:14 PM, Neal Groothuis <ngroot@lo-cal.org> wrote:
>>>> On 1/20/2012 3:35 PM, Neal Groothuis wrote:
>>>>> I'm trying to find /all/ commits that change a file in the
>>>>> repository...and its proving to be trickier than I thought. :-)
>>>
>>> On 1/21/2012 6:16 PM, Neal Kreitzinger wrote:
>>>> Does git-log --all help?
>>>
>>> I don't see how it would.  The commits are all reachable from HEAD,
>>> which
>>> would seem to be the problem that --all would correct.
>>>
>>> What I'm trying to do is find the commits in which a file differs from
>>> that same file in any of its parents.
>>
>> If you add parent rewriting (--parent, --graph or see it in gitk, with
>> --full-history) you'll get your B2 commit as it adds commits to have a
>> meaningful history. But I don't think this is what you are asking for.
>
> Correct.  If I add parent rewriting, I get all merges, even those in which
> the file is not changed from either parent.
>
> Based on what's in the man page for git log about the history
> simplification algorithm, it seems that B2 should be included in the
> output when I do a git log --full-history --simplify-history foo.txt, as
> per the steps I noted in the original post.  Is my understanding of the
> algorithm faulty?
>

Following your steps in the first post, B2 is excluded in the
--simplify-merge phase because it is (originally) TREESAME, even if it
is not in the rewritten history...

HTH,
Santi

Re: Finding all commits which modify a file

[Neal Groothuis]

> [ Do not cut the CC]

My apologies.

> On Tue, Jan 24, 2012 at 5:34 PM, Neal Groothuis <ngroot@lo-cal.org> wrote:
>>> On Mon, Jan 23, 2012 at 5:14 PM, Neal Groothuis <ngroot@lo-cal.org>
>>> wrote:
>>>>> On 1/20/2012 3:35 PM, Neal Groothuis wrote:
>>>>>> I'm trying to find /all/ commits that change a file in the
>>>>>> repository...and its proving to be trickier than I thought. :-)
>>>>
>>>> On 1/21/2012 6:16 PM, Neal Kreitzinger wrote:
>>>>> Does git-log --all help?
>>>>
>>>> I don't see how it would.  The commits are all reachable from HEAD,
>>>> which
>>>> would seem to be the problem that --all would correct.
>>>>
>>>> What I'm trying to do is find the commits in which a file differs from
>>>> that same file in any of its parents.
>>>
>>> If you add parent rewriting (--parent, --graph or see it in gitk, with
>>> --full-history) you'll get your B2 commit as it adds commits to have a
>>> meaningful history. But I don't think this is what you are asking for.
>>
>> Correct.  If I add parent rewriting, I get all merges, even those in
>> which
>> the file is not changed from either parent.
>>
>> Based on what's in the man page for git log about the history
>> simplification algorithm, it seems that B2 should be included in the
>> output when I do a git log --full-history --simplify-history foo.txt, as
>> per the steps I noted in the original post.  Is my understanding of the
>> algorithm faulty?
>>
>
> Following your steps in the first post, B2 is excluded in the
> --simplify-merge phase because it is (originally) TREESAME, even if it
> is not in the rewritten history...

Thanks, I see---labeling a commit as TREESAME happens before
simplification, rather than after.

In my example, that results in a simplified history where a commit in
which the contents of the specified paths change gets removed.  That seems
perverse; I would think the utility of a simplified history would be to
track down the commits in which the contents of the specified paths change
without having to consider ones in which they do not.

Is there a situation where checking for TREESAMEness before simplification
is desirable and checking after would not be?

- Neal

Re: Finding all commits which modify a file

[Junio C Hamano]

"Neal Groothuis" <ngroot@lo-cal.org> writes:

> Is there a situation where checking for TREESAMEness before simplification
> is desirable and checking after would not be?

When you do not want to see a side branch that does not contribute to the
end result at all, obviously ;-). Outside that situation, before or after
should not make a difference, I would think.

Re: Finding all commits which modify a file

[Tay Ray Chuan]

On Sat, Jan 21, 2012 at 5:35 AM, Neal Groothuis <ngroot@lo-cal.org> wrote:
> Hello,
>
> I'm trying to find /all/ commits that change a file in the
> repository...and its proving to be trickier than I thought. :-)
>
> The situation that we were dealing with is this:
>
> - Person A and person B both pull from the same central repository.
>
> - Person A makes a change to file foo.txt and bar.txt, commits, and pushes
> to the central repository.
>
> - Person B makes a similar change to bar.txt and commits it.
>
> - Person B does a fetch and merge.  Since both A and B made changes to
> bar.txt, this requires conflicts to be resolved manually.
>
> - B reverts A's changes to foo.txt. (If B is coming from a different
> revision control system, this may happen due to confusion about how merges
> are handled.)

How is this "revert" done? Was it done at the conflict resolution
level or with a git-revert invocation?

Nonetheless, either way, A's commit would be still be present in the
log history.

>[snip]
> Graphically:
>
>    A1
>   /  ^
>  v    \
>  C1   B2<-B3
>  ^    /
>   \  v
>    B1
>
> B1, B2, and B3 have the same version of foo.txt as C1, A1 modifies it.

Just to clarify, is C1 the commit that both A and B both share when
they first pull in the first step? And B2 is the merge?

-- 
Cheers,
Ray Chuan

Re: Finding all commits which modify a file

[Neal Groothuis]

> "Neal Groothuis" <ngroot@lo-cal.org> writes:
>
>> Is there a situation where checking for TREESAMEness before
>> simplification
>> is desirable and checking after would not be?
>
> When you do not want to see a side branch that does not contribute to
the end result at all, obviously ;-). Outside that situation, before or
after should not make a difference, I would think.

In that case, you wouldn't be using the --full-history flag at all, yeah?

Right now, we can see where the file gets changed (A1), we just can't see
where it gets changed back (B2).  In fact, if I run git-log --full-history
--simplify-merges foo.txt, it looks like A1 was the last thing to make
changes to foo.txt, which seems misleading to me---history has been
simplified to the point of not being true.

Re: Finding all commits which modify a file

[Junio C Hamano]

"Neal Groothuis" <ngroot@lo-cal.org> writes:

>> "Neal Groothuis" <ngroot@lo-cal.org> writes:
>>
>>> Is there a situation where checking for TREESAMEness before
>>> simplification
>>> is desirable and checking after would not be?
>>
>> When you do not want to see a side branch that does not contribute to
> the end result at all, obviously ;-). Outside that situation, before or
> after should not make a difference, I would think.
>
> In that case, you wouldn't be using the --full-history flag at all, yeah?

Yes. In case my tongue-in-cheek comment was too obscure, I was saying that
I do not think the change to TREESAME-ness check you were alluding to would
break any use case I would think of off the top of my head.

We of course might discover undesired consequences in unexpected corners
after we try your change, but I do not think we can discuss such corner
cases further without seeing a patch.