diff: --quiet does not imply --exit-code if --diff-filter is present

diff: --quiet does not imply --exit-code if --diff-filter is present

[Yasushi SHOJI]

Hi,

just noticed that quiet does not return exit code when I set
diff-filter.  at the current tip (v1.7.5.3-401-gfb674d7):

  git diff --quiet     --diff-filter=A v1.7.5 v1.7.5.1 -- t  #=> 0
  git diff --exit-code --diff-filter=A v1.7.5 v1.7.5.1 -- t  #=> 1

these two line returns different exit code.

is this a bug or a feature?

Thanks,
-- 
           yashi

Re: diff: --quiet does not imply --exit-code if --diff-filter is present

[Jeff King]

On Tue, May 31, 2011 at 07:34:39PM +0900, Yasushi SHOJI wrote:

> just noticed that quiet does not return exit code when I set
> diff-filter.  at the current tip (v1.7.5.3-401-gfb674d7):
> 
>   git diff --quiet     --diff-filter=A v1.7.5 v1.7.5.1 -- t  #=> 0
>   git diff --exit-code --diff-filter=A v1.7.5 v1.7.5.1 -- t  #=> 1
> 
> these two line returns different exit code.
> 
> is this a bug or a feature?

It's a bug.

-- >8 --
Subject: [PATCH] diff_tree: disable QUICK optimization with diff filter

We stop looking for changes early with QUICK, so our diff
queue contains only a subset of the changes. However, we
don't apply diff filters until later; it will appear at that
point as though there are no changes matching our filter,
when in reality we simply didn't keep looking for changes
long enough.

Commit 2cfe8a6 (diff --quiet: disable optimization when
--diff-filter=X is used, 2011-03-16) fixes this in some
cases by disabling the optimization when a filter is
present. However, it only tweaked run_diff_files, missing
the similar case in diff_tree. Thus the fix worked only for
diffing the working tree and index, but not between trees.

Noticed by Yasushi SHOJI.

Signed-off-by: Jeff King <peff@peff.net>
---
Grepping around, I think this was the only other missed case.

 t/t4040-whitespace-status.sh |    5 +++++
 tree-diff.c                  |    1 +
 2 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/t/t4040-whitespace-status.sh b/t/t4040-whitespace-status.sh
index abc4934..3c728a3 100755
--- a/t/t4040-whitespace-status.sh
+++ b/t/t4040-whitespace-status.sh
@@ -67,4 +67,9 @@ test_expect_success 'diff-files --diff-filter --quiet' '
 	test_must_fail git diff-files --diff-filter=M --quiet
 '
 
+test_expect_success 'diff-tree --diff-filter --quiet' '
+	git commit -a -m "worktree state" &&
+	test_must_fail git diff-tree --diff-filter=M --quiet HEAD^ HEAD
+'
+
 test_done
diff --git a/tree-diff.c b/tree-diff.c
index 7a79660..3d08f78 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -143,6 +143,7 @@ int diff_tree(struct tree_desc *t1, struct tree_desc *t2,
 
 	for (;;) {
 		if (DIFF_OPT_TST(opt, QUICK) &&
+		    !opt->filter &&
 		    DIFF_OPT_TST(opt, HAS_CHANGES))
 			break;
 		if (opt->pathspec.nr) {
-- 
1.7.5.3.7.g7dde6.dirty

Re: diff: --quiet does not imply --exit-code if --diff-filter is present

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Commit 2cfe8a6 (diff --quiet: disable optimization when
> --diff-filter=X is used, 2011-03-16) fixes this in some
> cases by disabling the optimization when a filter is
> present. However, it only tweaked run_diff_files, missing
> the similar case in diff_tree. Thus the fix worked only for
> diffing the working tree and index, but not between trees.

Thanks; a natural question is if we need the same for diff-index, then.

> diff --git a/tree-diff.c b/tree-diff.c
> index 7a79660..3d08f78 100644
> --- a/tree-diff.c
> +++ b/tree-diff.c
> @@ -143,6 +143,7 @@ int diff_tree(struct tree_desc *t1, struct tree_desc *t2,
>  
>  	for (;;) {
>  		if (DIFF_OPT_TST(opt, QUICK) &&
> +		    !opt->filter &&
>  		    DIFF_OPT_TST(opt, HAS_CHANGES))
>  			break;
>  		if (opt->pathspec.nr) {

We probably want to have a helper in diff.c that does

	int diff_can_quit_early(struct diff_options *opt)
        {
        	return (DIFF_OPT_TST(opt, QUICK) &&
                	!opt->filter &&
                        DIFF_OPT_TST(opt, HAS_CHANGES));
	}

It is possible for us to later add new diffcore transformations that need
a similar "do not stop feeding early, as results may be filtered".

Re: diff: --quiet does not imply --exit-code if --diff-filter is present

[Jeff King]

On Tue, May 31, 2011 at 08:46:29AM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > Commit 2cfe8a6 (diff --quiet: disable optimization when
> > --diff-filter=X is used, 2011-03-16) fixes this in some
> > cases by disabling the optimization when a filter is
> > present. However, it only tweaked run_diff_files, missing
> > the similar case in diff_tree. Thus the fix worked only for
> > diffing the working tree and index, but not between trees.
> 
> Thanks; a natural question is if we need the same for diff-index, then.

No, it calls straight into unpack_trees. So I think the question is
"should unpack_trees respect the QUICK optimization". I suspect it
didn't happen simply because unpack_trees is so complex, and there are
probably corner cases with merging.

> We probably want to have a helper in diff.c that does
> 
> 	int diff_can_quit_early(struct diff_options *opt)
>         {
>         	return (DIFF_OPT_TST(opt, QUICK) &&
>                 	!opt->filter &&
>                         DIFF_OPT_TST(opt, HAS_CHANGES));
> 	}
> 
> It is possible for us to later add new diffcore transformations that need
> a similar "do not stop feeding early, as results may be filtered".

Yeah, that is a good refactoring. It's more readable, and it would have
prevented this bug. :)

-Peff

Re* diff: --quiet does not imply --exit-code if --diff-filter is present

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

>> Thanks; a natural question is if we need the same for diff-index, then.
>
> No, it calls straight into unpack_trees. So I think the question is
> "should unpack_trees respect the QUICK optimization". I suspect it
> didn't happen simply because unpack_trees is so complex, and there are
> probably corner cases with merging.

Yeah, a somewhat hacky version would look like this.

-- >8 --
diff-index --quiet: learn the "stop feeding the backend early" logic

A negative return from the unpack callback function usually means unpack
failed for the entry and signals the unpack_trees() machinery to fail the
entire merge operation, immediately and there is no other way for the
callback to tell the machinery to exit early without reporting an error.

This is what we usually want to make a merge all-or-nothing operation, but
the machinery is also used for diff-index codepath by using a custom
unpack callback function. And we do sometimes want to exit early without
failing, namely when we are under --quiet and can short-cut the diff upon
finding the first difference.

Add "exiting_early" field to unpack_trees_options structure, to signal the
unpack_trees() machinery that the negative return value is not signaling
an error but an early return from the unpack_trees() machinery. As this by
definition hasn't unpacked everything, discard the resulting index just
like the failure codepath.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff-lib.c     |    7 ++++++-
 unpack-trees.c |    4 +++-
 unpack-trees.h |    1 +
 3 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/diff-lib.c b/diff-lib.c
index 9c29293..2e09500 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -433,8 +433,13 @@ static int oneway_diff(struct cache_entry **src, struct unpack_trees_options *o)
 	if (tree == o->df_conflict_entry)
 		tree = NULL;
 
-	if (ce_path_match(idx ? idx : tree, &revs->prune_data))
+	if (ce_path_match(idx ? idx : tree, &revs->prune_data)) {
 		do_oneway_diff(o, idx, tree);
+		if (diff_can_quit_early(&revs->diffopt)) {
+			o->exiting_early = 1;
+			return -1;
+		}
+	}
 
 	return 0;
 }
diff --git a/unpack-trees.c b/unpack-trees.c
index 07f8364..3a61d82 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -593,7 +593,7 @@ static int unpack_nondirectories(int n, unsigned long mask,
 static int unpack_failed(struct unpack_trees_options *o, const char *message)
 {
 	discard_index(&o->result);
-	if (!o->gently) {
+	if (!o->gently && !o->exiting_early) {
 		if (message)
 			return error("%s", message);
 		return -1;
@@ -1133,6 +1133,8 @@ return_failed:
 		display_error_msgs(o);
 	mark_all_ce_unused(o->src_index);
 	ret = unpack_failed(o, NULL);
+	if (o->exiting_early)
+		ret = 0;
 	goto done;
 }
 
diff --git a/unpack-trees.h b/unpack-trees.h
index 64f02cb..4b07683 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -47,6 +47,7 @@ struct unpack_trees_options {
 		     skip_sparse_checkout,
 		     gently,
 		     show_all_errors,
+		     exiting_early,
 		     dry_run;
 	const char *prefix;
 	int cache_bottom;

Re: Re* diff: --quiet does not imply --exit-code if --diff-filter is present

[Jeff King]

On Tue, May 31, 2011 at 10:06:44AM -0700, Junio C Hamano wrote:

> Add "exiting_early" field to unpack_trees_options structure, to signal the
> unpack_trees() machinery that the negative return value is not signaling
> an error but an early return from the unpack_trees() machinery. As this by
> definition hasn't unpacked everything, discard the resulting index just
> like the failure codepath.

Maybe it is just me, but I would think that it's a little more intuitive
to set exiting_early and then return "0" to indicate "success, but I
didn't look at everything".

I guess you did it this way to better share the discard-the-result
codepath. Maybe it wouldn't be too painful to refactor unpack_failed()
into the failed bits plus a call to a new unpack_discard() that does the
non-error bits. It may not be worth the trouble though.

-Peff

Re: Re* diff: --quiet does not imply --exit-code if --diff-filter is present

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> I guess you did it this way to better share the discard-the-result
> codepath.

No, I did it as a hack because many places already do:

	if (... sub helper function that eventually call_callback ... < 0)
        	break; // or
                return; // or
                goto fail_return; // or whatever to exit recursion and loop

and obviously it was too much pain to change everybody to also pay
attention to the new flag.

A possibly cleaner way would be to designate a single negative value that
is not -1 as "early return but not failure" without using an extra bit,
but that also needs full vetting of the existing callchain, which I didn't
want to do just to write a "it would be as little as this" patch.

Re: diff: --quiet does not imply --exit-code if --diff-filter is present

[Yasushi SHOJI]

Hi Jeff,

At Tue, 31 May 2011 11:33:56 -0400,
Jeff King wrote:
> 
> On Tue, May 31, 2011 at 07:34:39PM +0900, Yasushi SHOJI wrote:
> 
> > just noticed that quiet does not return exit code when I set
> > diff-filter.  at the current tip (v1.7.5.3-401-gfb674d7):
> > 
> >   git diff --quiet     --diff-filter=A v1.7.5 v1.7.5.1 -- t  #=> 0
> >   git diff --exit-code --diff-filter=A v1.7.5 v1.7.5.1 -- t  #=> 1
> > 
> > these two line returns different exit code.
> > 
> > is this a bug or a feature?
> 
> It's a bug.

your patch works like a charm.  Thanks.

> Commit 2cfe8a6 (diff --quiet: disable optimization when
> --diff-filter=X is used, 2011-03-16) fixes this in some
> cases by disabling the optimization when a filter is
> present. However, it only tweaked run_diff_files, missing
> the similar case in diff_tree. Thus the fix worked only for
> diffing the working tree and index, but not between trees.

oh, sorry about not realizing the commit 2cfe8a6.

regards,
-- 
           yashi