git-log --cherry-pick gives different results when using tag or tag^{}

git-log --cherry-pick gives different results when using tag or tag^{}

[Francis Moreau]

Hello,

In mykernel repository, I'm having 2 different behaviours with git-log
but I don't understand why:

Doing:

    $ git log --oneline --cherry-pick --left-right v3.4.71-1^{}...next

and

    $ git log --oneline --cherry-pick --left-right v3.4.71-1...next

give something different (where v3.4.71-1 is a tag).

The command using ^{} looks the one that gives correct result I think.

Could anybody enlight me ?

Thanks.

Re: git-log --cherry-pick gives different results when using tag or tag^{}

[Jeff King]

[+cc Junio, as the bug blames to him]

On Fri, Jan 10, 2014 at 02:15:40PM +0100, Francis Moreau wrote:

> In mykernel repository, I'm having 2 different behaviours with git-log
> but I don't understand why:
> 
> Doing:
> 
>     $ git log --oneline --cherry-pick --left-right v3.4.71-1^{}...next
> 
> and
> 
>     $ git log --oneline --cherry-pick --left-right v3.4.71-1...next
> 
> give something different (where v3.4.71-1 is a tag).
> 
> The command using ^{} looks the one that gives correct result I think.

Yeah, this looks like a bug. Here's a simple reproduction recipe:

  commit() {
    echo content >$1 &&
    git add $1 &&
    git commit -m $1
  }

  git init repo && cd repo &&
  commit one &&
  commit two &&
  sleep 1 &&
  git tag -m foo mytag &&
  git checkout -b side HEAD^ &&
  git cherry-pick mytag &&
  commit three

The sleep seems to be necessary, to give the commit and its
cherry-picked version different commit times (presumably because it
impacts the order in which we visit them during the traversal).

Running:

  git log --oneline --decorate --cherry-pick --left-right mytag^{}...HEAD

produces the expected:

  > e36cc32 (HEAD, side) three

but running it with the tag, as:

  git log --oneline --decorate --cherry-pick --left-right mytag...HEAD

yields:

  > e36cc32 (HEAD, side) three
  > 5e96f7d two
  > db92fca (tag: mytag, master) two

Not only do we get the cherry-pick wrong (we should omit both "twos"),
but we seem to erroneously count the tagged "two" as being on the
right-hand side, which it clearly is not (and which is probably why we
don't find the match via --cherry-pick).

This worked in v1.8.4, but is broken in v1.8.5. It bisects to Junio's
895c5ba (revision: do not peel tags used in range notation, 2013-09-19),
which sounds promising.

I think what is happening is that we used to apply the SYMMETRIC_LEFT
flag directly to the commit. Now we apply it to the tag, and it does not
seem to get propagated. The patch below fixes it for me, but I have no
idea if we actually need to be setting the other flags, or just
SYMMETRIC_LEFT. I also wonder if the non-symmetric two-dot case needs to
access any pointed-to commit and propagate flags in a similar way.

diff --git a/revision.c b/revision.c
index 7010aff..1d99bfc 100644
--- a/revision.c
+++ b/revision.c
@@ -1197,6 +1197,8 @@ int handle_revision_arg(const char *arg_, struct rev_info *revs, int flags, unsi
 				free_commit_list(exclude);
 
 				a_flags = flags | SYMMETRIC_LEFT;
+				a->object.flags |= a_flags;
+				b->object.flags |= flags;
 			}
 
 			a_obj->flags |= a_flags;

-Peff

Re: git-log --cherry-pick gives different results when using tag or tag^{}

[Francis Moreau]

On 01/15/2014 10:49 AM, Jeff King wrote:
> [+cc Junio, as the bug blames to him]
> 
> On Fri, Jan 10, 2014 at 02:15:40PM +0100, Francis Moreau wrote:
> 
>> In mykernel repository, I'm having 2 different behaviours with git-log
>> but I don't understand why:
>>
>> Doing:
>>
>>     $ git log --oneline --cherry-pick --left-right v3.4.71-1^{}...next
>>
>> and
>>
>>     $ git log --oneline --cherry-pick --left-right v3.4.71-1...next
>>
>> give something different (where v3.4.71-1 is a tag).
>>
>> The command using ^{} looks the one that gives correct result I think.
> 
> Yeah, this looks like a bug. Here's a simple reproduction recipe:

Thanks a lot Jeff for your good analyze.

Re: git-log --cherry-pick gives different results when using tag or tag^{}

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> [+cc Junio, as the bug blames to him]
> ...
> I think what is happening is that we used to apply the SYMMETRIC_LEFT
> flag directly to the commit. Now we apply it to the tag, and it does not
> seem to get propagated. The patch below fixes it for me, but I have no
> idea if we actually need to be setting the other flags, or just
> SYMMETRIC_LEFT. I also wonder if the non-symmetric two-dot case needs to
> access any pointed-to commit and propagate flags in a similar way.

Thanks.

Where do we pass down other flags from tags to commits?  For
example, if we do this:

	$ git log ^v1.8.5 master

we mark v1.8.5 tag as UNINTERESTING, and throw that tag (not commit
v1.8.5^0) into revs->pending.objects[].  We do the same for 'master',
which is a commit.

Later, in prepare_revision_walk(), we call handle_commit() on them,
and unwrap the tag v1.8.5 to get v1.8.5^0, and then handles that
commit object with flags obtained from the tag object.  This code
only cares about UNINTERESTING and manually propagates it.

Perhaps that code needs to propagate at least SYMMETRIC_LEFT down to
the commit object as well, no?  With your patch, the topmost level
of tag object and the eventual commit object are marked with the
flag, but if we were dealing with a tag that points at another tag
that in turn points at a commit, the intermediate tag will not be
marked with SYMMETRIC_LEFT (nor UNINTERESTING for that matter),
which may not affect the final outcome, but it somewhat feels wrong.

How about doing it this way instead (totally untested, though)?

 revision.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/revision.c b/revision.c
index a68fde6..def070e 100644
--- a/revision.c
+++ b/revision.c
@@ -276,6 +276,7 @@ static struct commit *handle_commit(struct rev_info *revs,
 				return NULL;
 			die("bad object %s", sha1_to_hex(tag->tagged->sha1));
 		}
+		object->flags |= flags;
 	}
 
 	/*
@@ -287,7 +288,6 @@ static struct commit *handle_commit(struct rev_info *revs,
 		if (parse_commit(commit) < 0)
 			die("unable to parse commit %s", name);
 		if (flags & UNINTERESTING) {
-			commit->object.flags |= UNINTERESTING;
 			mark_parents_uninteresting(commit);
 			revs->limited = 1;
 		}

revision: propagate flag bits from tags to pointees

[Junio C Hamano]

With the previous fix 895c5ba3 (revision: do not peel tags used in
range notation, 2013-09-19), handle_revision_arg() that processes
command line arguments for the "git log" family of commands no
longer directly places the object pointed by the tag in the pending
object array when it sees a tag object.  We used to place pointee
there after copying the flag bits like UNINTERESTING and
SYMMETRIC_LEFT.

This change meant that any flag that is relevant to later history
traversal must now be propagated to the pointed objects (most often
these are commits) while starting the traversal, which is partly
done by handle_commit() that is called from prepare_revision_walk().
We did propagate UNINTERESTING, but did not do so for others, most
notably SYMMETRIC_LEFT.  This caused "git log --left-right v1.0..."
(where "v1.0" is a tag) to start losing the "leftness" from the
commit the tag points at.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---

 * Comes directly on top of the faulty commit, so that we could
   backport it to 1.8.4.x series.

 revision.c               |  2 +-
 t/t6000-rev-list-misc.sh | 11 +++++++++++
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/revision.c b/revision.c
index 7010aff..6d1c8f9 100644
--- a/revision.c
+++ b/revision.c
@@ -265,6 +265,7 @@ static struct commit *handle_commit(struct rev_info *revs, struct object *object
 				return NULL;
 			die("bad object %s", sha1_to_hex(tag->tagged->sha1));
 		}
+		object->flags |= flags;
 	}
 
 	/*
@@ -276,7 +277,6 @@ static struct commit *handle_commit(struct rev_info *revs, struct object *object
 		if (parse_commit(commit) < 0)
 			die("unable to parse commit %s", name);
 		if (flags & UNINTERESTING) {
-			commit->object.flags |= UNINTERESTING;
 			mark_parents_uninteresting(commit);
 			revs->limited = 1;
 		}
diff --git a/t/t6000-rev-list-misc.sh b/t/t6000-rev-list-misc.sh
index 15e3d64..b84d6b0 100755
--- a/t/t6000-rev-list-misc.sh
+++ b/t/t6000-rev-list-misc.sh
@@ -56,4 +56,15 @@ test_expect_success 'rev-list A..B and rev-list ^A B are the same' '
 	test_cmp expect actual
 '
 
+test_expect_success 'symleft flag bit is propagated down from tag' '
+	git log --format="%m %s" --left-right v1.0...master >actual &&
+	cat >expect <<-\EOF &&
+	> two
+	> one
+	< another
+	< that
+	EOF
+	test_cmp expect actual
+'
+
 test_done

Re: git-log --cherry-pick gives different results when using tag or tag^{}

[Jeff King]

On Wed, Jan 15, 2014 at 11:57:39AM -0800, Junio C Hamano wrote:

> Where do we pass down other flags from tags to commits?  For
> example, if we do this:
> 
> 	$ git log ^v1.8.5 master
> 
> we mark v1.8.5 tag as UNINTERESTING, and throw that tag (not commit
> v1.8.5^0) into revs->pending.objects[].  We do the same for 'master',
> which is a commit.
> 
> Later, in prepare_revision_walk(), we call handle_commit() on them,
> and unwrap the tag v1.8.5 to get v1.8.5^0, and then handles that
> commit object with flags obtained from the tag object.  This code
> only cares about UNINTERESTING and manually propagates it.

Thanks for picking up this line of thought. I had some notion that the
right solution would be in propagating the flags later from the pending
tags to the commits, but I didn't quite know where to look. Knowing that
we explicitly propagate UNINTERESTING but nothing else makes what I was
seeing make a lot more sense.

> Perhaps that code needs to propagate at least SYMMETRIC_LEFT down to
> the commit object as well, no?  With your patch, the topmost level
> of tag object and the eventual commit object are marked with the
> flag, but if we were dealing with a tag that points at another tag
> that in turn points at a commit, the intermediate tag will not be
> marked with SYMMETRIC_LEFT (nor UNINTERESTING for that matter),
> which may not affect the final outcome, but it somewhat feels wrong.

Agreed. I think the lack of flags on intermediate tags has always been
that way, even before 895c5ba, and I do not know of any case where it
currently matters. But it seems like the obvious right thing to mark
those intermediate tags.

> How about doing it this way instead (totally untested, though)?

Makes sense. It also means we will propagate flags down to any
pointed-to trees and blobs. I can't think of a case where that will
matter either (and they cannot be SYMMETRIC_LEFT, as that only makes
sense for commit objects).

I do notice that when we have a tree, we explicitly propagate
UNINTERESTING to the rest of the tree. Should we be propagating all
flags instead? Again, I can't think of a reason to do so (and if it is
not UNINTERESTING, it is a non-trivial amount of time to mark all paths
in the tree).


> @@ -287,7 +288,6 @@ static struct commit *handle_commit(struct rev_info *revs,
>  		if (parse_commit(commit) < 0)
>  			die("unable to parse commit %s", name);
>  		if (flags & UNINTERESTING) {
> -			commit->object.flags |= UNINTERESTING;
>  			mark_parents_uninteresting(commit);
>  			revs->limited = 1;
>  		}

We don't need to propagate the UNINTERESTING flag here, because either:

  - "object" pointed to the commit, in which case flags comes from
    object->flags, and we already have it set

or

  - "object" was a tag, and we propagated the flags as we peeled (from
    your earlier hunk)

Makes sense. I think the "mark_blob_uninteresting" call later in the
function is now irrelevant for the same reasons. The
mark_tree_uninteresting call is not, though, because it recurses.

-Peff

Re: revision: propagate flag bits from tags to pointees

[Jeff King]

On Wed, Jan 15, 2014 at 12:26:13PM -0800, Junio C Hamano wrote:

> With the previous fix 895c5ba3 (revision: do not peel tags used in
> range notation, 2013-09-19), handle_revision_arg() that processes
> command line arguments for the "git log" family of commands no
> longer directly places the object pointed by the tag in the pending
> object array when it sees a tag object.  We used to place pointee
> there after copying the flag bits like UNINTERESTING and
> SYMMETRIC_LEFT.
> 
> This change meant that any flag that is relevant to later history
> traversal must now be propagated to the pointed objects (most often
> these are commits) while starting the traversal, which is partly
> done by handle_commit() that is called from prepare_revision_walk().
> We did propagate UNINTERESTING, but did not do so for others, most
> notably SYMMETRIC_LEFT.  This caused "git log --left-right v1.0..."
> (where "v1.0" is a tag) to start losing the "leftness" from the
> commit the tag points at.
> 
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---

Looks good to me. As per my previous mail, I _think_ you could squash
in:

diff --git a/revision.c b/revision.c
index f786b51..2db906c 100644
--- a/revision.c
+++ b/revision.c
@@ -316,13 +316,10 @@ static struct commit *handle_commit(struct rev_info *revs,
 	 * Blob object? You know the drill by now..
 	 */
 	if (object->type == OBJ_BLOB) {
-		struct blob *blob = (struct blob *)object;
 		if (!revs->blob_objects)
 			return NULL;
-		if (flags & UNINTERESTING) {
-			mark_blob_uninteresting(blob);
+		if (flags & UNINTERESTING)
 			return NULL;
-		}
 		add_pending_object(revs, object, "");
 		return NULL;
 	}

but that is not very much code reduction (and mark_blob_uninteresting is
very cheap). So it may not be worth the risk that my analysis is wrong.
:)

-Peff

Re: revision: propagate flag bits from tags to pointees

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Looks good to me. As per my previous mail, I _think_ you could squash
> in:
>
> diff --git a/revision.c b/revision.c
> index f786b51..2db906c 100644
> --- a/revision.c
> +++ b/revision.c
> @@ -316,13 +316,10 @@ static struct commit *handle_commit(struct rev_info *revs,
>  	 * Blob object? You know the drill by now..
>  	 */
>  	if (object->type == OBJ_BLOB) {
> -		struct blob *blob = (struct blob *)object;
>  		if (!revs->blob_objects)
>  			return NULL;
> -		if (flags & UNINTERESTING) {
> -			mark_blob_uninteresting(blob);
> +		if (flags & UNINTERESTING)
>  			return NULL;
> -		}
>  		add_pending_object(revs, object, "");
>  		return NULL;
>  	}
>
> but that is not very much code reduction (and mark_blob_uninteresting is
> very cheap). So it may not be worth the risk that my analysis is wrong.
> :)

Your analysis is correct, but I think the pros-and-cons of the your
squashable change boils down to the choice between:

 - leaving it in will keep similarity between tree and blob
   codepaths (both have mark_X_uninteresting(); and

 - reducing cycles by taking advantage of the explicit knowledge
   that mark_X_uninteresting() recurses for a tree while it does not
   for a blob.

But I have a suspicion that my patch may break if any codepath looks
at the current flag on the object and decides "ah, it already is
marked" and punts.

It indeed looks like mark_tree_uninteresting() does have that
property.  When an uninteresting tag directly points at a tree, if
we propagate the UNINTERESTING bit to the pointee while peeling,
wouldn't we end up calling mark_tree_uninteresting() on a tree,
whose flags already have UNINTERESTING bit set, causing it not to
recurse?

Re: revision: propagate flag bits from tags to pointees

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> But I have a suspicion that my patch may break if any codepath looks
> at the current flag on the object and decides "ah, it already is
> marked" and punts.
>
> It indeed looks like mark_tree_uninteresting() does have that
> property.  When an uninteresting tag directly points at a tree, if
> we propagate the UNINTERESTING bit to the pointee while peeling,
> wouldn't we end up calling mark_tree_uninteresting() on a tree,
> whose flags already have UNINTERESTING bit set, causing it not to
> recurse?

Extending that line of thought further, what should this do?

    git rev-list --objects ^HEAD^^{tree} HEAD^{tree} |
    git pack-object --stdin pack

It says "I am interested in the objects that is used in the tree of
HEAD, but I do not need those that already appear in HEAD^".

With the current code (with or without the fix under discussion, or
even without the faulty "do not peel tags used in range notation"),
the tree of the HEAD^ is marked in handle_revision_arg() as
UNINTERESTING when it is placed in revs->pending.objects[], and the
handle_commit() --- we should rename it to handle_pending_object()
or something, by the way --- will call mark_tree_uninteresting() on
that tree, which then would say "It is already uninteresting" and
return without marking the objects common to these two trees
uninteresting, no?

I think that is a related but separate bug that dates back to
prehistoric times, and the asymmetry between handle_commit() deals
with commits and trees should have been a clear clue that tells us
something is fishy.  It calls "mark PARENTS uninteresting", leaving
the responsibility of marking the commit itself to the caller, but
it calls mark_tree_uninteresting() whose caller is not supposed to
mark the tree itself.

Which suggest me that a right fix for this separate bug would be to
introduce mark_tree_contents_uninteresting() or something, which has
the parallel semantics to mark_parents_uninteresting().  Then
mark_blob_uninteresting() call in the function can clearly go.  Such
a change will make it clear that handle_commit() is responsible for
handling the flags for the given object, and any helper functions
called by it should not peek and stop the flag of the object itself
when deciding to recurse into the objects linked to it.