[PATCH 1/2] remerge-diff: lazily prpare temporary objdir on demand

[PATCH 1/2] remerge-diff: lazily prpare temporary objdir on demand

[Junio C Hamano]

It is error prone for each caller that sets revs.remerge_diff bit
to be responsible for preparing a temporary object directory and
rotate it into the list of alternate object stores, making it the
primary object store.

Instead, remove the code to set up and arrange the temporary object
directory from the current callers and implement it in the code that
runs remerge-diff logic.  The code to undo the futzing of the list
of alternate object store is still spread across the callers, but we
will deal with it in future steps.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---

 * So here are two patches to illustate how the code to do the lazy
   set-up and centralized tear down looks like.  The first part is
   the lazy set-up.

 builtin/diff-tree.c |  7 -------
 builtin/log.c       |  7 -------
 log-tree.c          | 16 ++++++++++++----
 3 files changed, 12 insertions(+), 18 deletions(-)

diff --git a/builtin/diff-tree.c b/builtin/diff-tree.c
index 12012ea093..af51cd29c6 100644
--- a/builtin/diff-tree.c
+++ b/builtin/diff-tree.c
@@ -168,13 +168,6 @@ int cmd_diff_tree(int argc, const char **argv, const char *prefix)
 
 	opt->diffopt.rotate_to_strict = 1;
 
-	if (opt->remerge_diff) {
-		opt->remerge_objdir = tmp_objdir_create("remerge-diff");
-		if (!opt->remerge_objdir)
-			die(_("unable to create temporary object directory"));
-		tmp_objdir_replace_primary_odb(opt->remerge_objdir, 1);
-	}
-
 	/*
 	 * NOTE!  We expect "a..b" to expand to "^a b" but it is
 	 * perfectly valid for revision range parser to yield "b ^a",
diff --git a/builtin/log.c b/builtin/log.c
index af6403cb71..227823a16e 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -494,13 +494,6 @@ static int cmd_log_walk_no_free(struct rev_info *rev)
 	int saved_nrl = 0;
 	int saved_dcctc = 0;
 
-	if (rev->remerge_diff) {
-		rev->remerge_objdir = tmp_objdir_create("remerge-diff");
-		if (!rev->remerge_objdir)
-			die(_("unable to create temporary object directory"));
-		tmp_objdir_replace_primary_odb(rev->remerge_objdir, 1);
-	}
-
 	if (rev->early_output)
 		setup_early_output();
 
diff --git a/log-tree.c b/log-tree.c
index 337b9334cd..83e648b7cd 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -1019,6 +1019,17 @@ static int do_remerge_diff(struct rev_info *opt,
 	struct strbuf parent1_desc = STRBUF_INIT;
 	struct strbuf parent2_desc = STRBUF_INIT;
 
+	/*
+	 * Lazily prepare a temporary object directory and rotate it
+	 * into the alternative object store list as the primary.
+	 */
+	if (opt->remerge_diff && !opt->remerge_objdir) {
+		opt->remerge_objdir = tmp_objdir_create("remerge-diff");
+		if (!opt->remerge_objdir)
+			return error(_("unable to create temporary object directory"));
+		tmp_objdir_replace_primary_odb(opt->remerge_objdir, 1);
+	}
+
 	/* Setup merge options */
 	init_merge_options(&o, the_repository);
 	o.show_rename_progress = 0;
@@ -1053,10 +1064,7 @@ static int do_remerge_diff(struct rev_info *opt,
 	merge_finalize(&o, &res);
 
 	/* Clean up the contents of the temporary object directory */
-	if (opt->remerge_objdir)
-		tmp_objdir_discard_objects(opt->remerge_objdir);
-	else
-		BUG("did a remerge diff without remerge_objdir?!?");
+	tmp_objdir_discard_objects(opt->remerge_objdir);
 
 	return !opt->loginfo;
 }
-- 
2.46.0-326-g1e046905fc

[PATCH 2/2] remerge-diff: clean up temporary objdir at a central place

[Junio C Hamano]

After running a diff between two things, or a series of diffs while
walking the history, the diff computation is concluded by a call to
diff_result_code() to extract the exit status of the diff machinery.

The function can work on "struct diffopt", but all the callers
historically and currently pass "struct diffopt" that is embedded in
the "struct rev_info" that is used to hold the remerge_diff bit and
the remerge_objdir variable that points at the temporary object
directory in use.

Redefine diff_result_code() to take the whole "struct rev_info" to
give it an access to these members related to remerge-diff, so that
it can get rid of the temporary object directory for any and all
callers that used the feature.  We can lose the equivalent code to
do so from the code paths for individual commands, diff-tree, diff,
and log.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 builtin/describe.c          |  2 +-
 builtin/diff-files.c        |  2 +-
 builtin/diff-index.c        |  2 +-
 builtin/diff-tree.c         |  7 +------
 builtin/diff.c              |  2 +-
 builtin/log.c               | 11 ++++-------
 builtin/stash.c             |  6 +++---
 builtin/submodule--helper.c |  2 +-
 diff-no-index.c             |  2 +-
 diff.c                      | 10 +++++++++-
 diff.h                      |  2 +-
 wt-status.c                 |  4 ++--
 12 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/builtin/describe.c b/builtin/describe.c
index d6c77a714f..a1457d59e8 100644
--- a/builtin/describe.c
+++ b/builtin/describe.c
@@ -687,7 +687,7 @@ int cmd_describe(int argc, const char **argv, const char *prefix)
 				BUG("malformed internal diff-index command line");
 			run_diff_index(&revs, 0);
 
-			if (!diff_result_code(&revs.diffopt))
+			if (!diff_result_code(&revs))
 				suffix = NULL;
 			else
 				suffix = dirty;
diff --git a/builtin/diff-files.c b/builtin/diff-files.c
index 018011f29e..dd0b76e7d5 100644
--- a/builtin/diff-files.c
+++ b/builtin/diff-files.c
@@ -82,7 +82,7 @@ int cmd_diff_files(int argc, const char **argv, const char *prefix)
 	if (repo_read_index_preload(the_repository, &rev.diffopt.pathspec, 0) < 0)
 		die_errno("repo_read_index_preload");
 	run_diff_files(&rev, options);
-	result = diff_result_code(&rev.diffopt);
+	result = diff_result_code(&rev);
 	release_revisions(&rev);
 	return result;
 }
diff --git a/builtin/diff-index.c b/builtin/diff-index.c
index 3e05260ac0..560c099524 100644
--- a/builtin/diff-index.c
+++ b/builtin/diff-index.c
@@ -71,7 +71,7 @@ int cmd_diff_index(int argc, const char **argv, const char *prefix)
 		return -1;
 	}
 	run_diff_index(&rev, option);
-	result = diff_result_code(&rev.diffopt);
+	result = diff_result_code(&rev);
 	release_revisions(&rev);
 	return result;
 }
diff --git a/builtin/diff-tree.c b/builtin/diff-tree.c
index af51cd29c6..995225ef5f 100644
--- a/builtin/diff-tree.c
+++ b/builtin/diff-tree.c
@@ -232,10 +232,5 @@ int cmd_diff_tree(int argc, const char **argv, const char *prefix)
 		diff_free(&opt->diffopt);
 	}
 
-	if (opt->remerge_diff) {
-		tmp_objdir_destroy(opt->remerge_objdir);
-		opt->remerge_objdir = NULL;
-	}
-
-	return diff_result_code(&opt->diffopt);
+	return diff_result_code(opt);
 }
diff --git a/builtin/diff.c b/builtin/diff.c
index 6e196e0c7d..a5834704aa 100644
--- a/builtin/diff.c
+++ b/builtin/diff.c
@@ -605,7 +605,7 @@ int cmd_diff(int argc, const char **argv, const char *prefix)
 		builtin_diff_combined(&rev, argc, argv,
 				      ent.objects, ent.nr,
 				      first_non_parent);
-	result = diff_result_code(&rev.diffopt);
+	result = diff_result_code(&rev);
 	if (1 < rev.diffopt.skip_stat_unmatch)
 		refresh_index_quietly();
 	release_revisions(&rev);
diff --git a/builtin/log.c b/builtin/log.c
index 227823a16e..b1a8fa73b1 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -493,6 +493,7 @@ static int cmd_log_walk_no_free(struct rev_info *rev)
 	struct commit *commit;
 	int saved_nrl = 0;
 	int saved_dcctc = 0;
+	int result;
 
 	if (rev->early_output)
 		setup_early_output();
@@ -533,16 +534,12 @@ static int cmd_log_walk_no_free(struct rev_info *rev)
 	rev->diffopt.degraded_cc_to_c = saved_dcctc;
 	rev->diffopt.needed_rename_limit = saved_nrl;
 
-	if (rev->remerge_diff) {
-		tmp_objdir_destroy(rev->remerge_objdir);
-		rev->remerge_objdir = NULL;
-	}
-
+	result = diff_result_code(rev);
 	if (rev->diffopt.output_format & DIFF_FORMAT_CHECKDIFF &&
 	    rev->diffopt.flags.check_failed) {
-		return 02;
+		result = 02;
 	}
-	return diff_result_code(&rev->diffopt);
+	return result;
 }
 
 static int cmd_log_walk(struct rev_info *rev)
diff --git a/builtin/stash.c b/builtin/stash.c
index 7fb355bff0..6dc70a6be2 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -972,7 +972,7 @@ static int show_stash(int argc, const char **argv, const char *prefix)
 	}
 	log_tree_diff_flush(&rev);
 
-	ret = diff_result_code(&rev.diffopt);
+	ret = diff_result_code(&rev);
 cleanup:
 	strvec_clear(&stash_args);
 	free_stash_info(&info);
@@ -1116,13 +1116,13 @@ static int check_changes_tracked_files(const struct pathspec *ps)
 	diff_setup_done(&rev.diffopt);
 
 	run_diff_index(&rev, DIFF_INDEX_CACHED);
-	if (diff_result_code(&rev.diffopt)) {
+	if (diff_result_code(&rev)) {
 		ret = 1;
 		goto done;
 	}
 
 	run_diff_files(&rev, 0);
-	if (diff_result_code(&rev.diffopt)) {
+	if (diff_result_code(&rev)) {
 		ret = 1;
 		goto done;
 	}
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index cc0db6336c..96d2889907 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -675,7 +675,7 @@ static void status_submodule(const char *path, const struct object_id *ce_oid,
 	setup_revisions(diff_files_args.nr, diff_files_args.v, &rev, &opt);
 	run_diff_files(&rev, 0);
 
-	if (!diff_result_code(&rev.diffopt)) {
+	if (!diff_result_code(&rev)) {
 		print_status(flags, ' ', path, ce_oid,
 			     displaypath);
 	} else if (!(flags & OPT_CACHED)) {
diff --git a/diff-no-index.c b/diff-no-index.c
index 3a8965672c..c5fb06e6d1 100644
--- a/diff-no-index.c
+++ b/diff-no-index.c
@@ -362,7 +362,7 @@ int diff_no_index(struct rev_info *revs,
 	 * The return code for --no-index imitates diff(1):
 	 * 0 = no changes, 1 = changes, else error
 	 */
-	ret = diff_result_code(&revs->diffopt);
+	ret = diff_result_code(revs);
 
 out:
 	for (i = 0; i < ARRAY_SIZE(to_free); i++)
diff --git a/diff.c b/diff.c
index ccfa1fca0d..970ea47710 100644
--- a/diff.c
+++ b/diff.c
@@ -9,6 +9,7 @@
 #include "environment.h"
 #include "gettext.h"
 #include "tempfile.h"
+#include "revision.h"
 #include "quote.h"
 #include "diff.h"
 #include "diffcore.h"
@@ -26,6 +27,7 @@
 #include "merge-ll.h"
 #include "string-list.h"
 #include "strvec.h"
+#include "tmp-objdir.h"
 #include "graph.h"
 #include "oid-array.h"
 #include "packfile.h"
@@ -7012,10 +7014,16 @@ void diffcore_std(struct diff_options *options)
 	options->found_follow = 0;
 }
 
-int diff_result_code(struct diff_options *opt)
+int diff_result_code(struct rev_info *revs)
 {
+	struct diff_options *opt = &revs->diffopt;
 	int result = 0;
 
+	if (revs->remerge_diff) {
+		tmp_objdir_destroy(revs->remerge_objdir);
+		revs->remerge_objdir = NULL;
+	}
+
 	diff_warn_rename_limit("diff.renameLimit",
 			       opt->needed_rename_limit,
 			       opt->degraded_cc_to_c);
diff --git a/diff.h b/diff.h
index 66bd8aeb29..1966ba6bb7 100644
--- a/diff.h
+++ b/diff.h
@@ -648,7 +648,7 @@ int do_diff_cache(const struct object_id *, struct diff_options *);
 int diff_flush_patch_id(struct diff_options *, struct object_id *, int);
 void flush_one_hunk(struct object_id *result, git_hash_ctx *ctx);
 
-int diff_result_code(struct diff_options *);
+int diff_result_code(struct rev_info *);
 
 int diff_no_index(struct rev_info *,
 		  int implicit_no_index, int, const char **);
diff --git a/wt-status.c b/wt-status.c
index 9e8c08003b..d1c3f3dc4d 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -2582,7 +2582,7 @@ int has_unstaged_changes(struct repository *r, int ignore_submodules)
 	rev_info.diffopt.flags.quick = 1;
 	diff_setup_done(&rev_info.diffopt);
 	run_diff_files(&rev_info, 0);
-	result = diff_result_code(&rev_info.diffopt);
+	result = diff_result_code(&rev_info);
 	release_revisions(&rev_info);
 	return result;
 }
@@ -2616,7 +2616,7 @@ int has_uncommitted_changes(struct repository *r,
 
 	diff_setup_done(&rev_info.diffopt);
 	run_diff_index(&rev_info, DIFF_INDEX_CACHED);
-	result = diff_result_code(&rev_info.diffopt);
+	result = diff_result_code(&rev_info);
 	release_revisions(&rev_info);
 	return result;
 }
-- 
2.46.0-326-g1e046905fc

Re: [PATCH 1/2] remerge-diff: lazily prpare temporary objdir on demand

[Eric Sunshine]

On Fri, Aug 9, 2024 at 6:31 PM Junio C Hamano <gitster@pobox.com> wrote:
> remerge-diff: lazily prpare temporary objdir on demand

s/prpare/prepare/

> It is error prone for each caller that sets revs.remerge_diff bit
> to be responsible for preparing a temporary object directory and
> rotate it into the list of alternate object stores, making it the
> primary object store.
>
> Instead, remove the code to set up and arrange the temporary object
> directory from the current callers and implement it in the code that
> runs remerge-diff logic.  The code to undo the futzing of the list
> of alternate object store is still spread across the callers, but we
> will deal with it in future steps.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Re: [PATCH 2/2] remerge-diff: clean up temporary objdir at a central place

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> After running a diff between two things, or a series of diffs while
> walking the history, the diff computation is concluded by a call to
> diff_result_code() to extract the exit status of the diff machinery.
>
> The function can work on "struct diffopt", but all the callers
> historically and currently pass "struct diffopt" that is embedded in
> the "struct rev_info" that is used to hold the remerge_diff bit and
> the remerge_objdir variable that points at the temporary object
> directory in use.
>
> Redefine diff_result_code() to take the whole "struct rev_info" to
> give it an access to these members related to remerge-diff, so that
> it can get rid of the temporary object directory for any and all
> callers that used the feature.  We can lose the equivalent code to
> do so from the code paths for individual commands, diff-tree, diff,
> and log.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---

I forgot to add that I am not happy with this "centralized tear
down" step, even though I am reasonably happy with the "lazy set-up"
step.  I wonder why the remerge-diff related members have to exist
in the rev_info structure in the first place, instead of being in
the diffopt structure?  Moving them to diffopt may make the end
result much more pleasant to read.

Re: [PATCH 2/2] remerge-diff: clean up temporary objdir at a central place

[Elijah Newren]

On Sun, Aug 11, 2024 at 10:17 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Junio C Hamano <gitster@pobox.com> writes:
>
> > After running a diff between two things, or a series of diffs while
> > walking the history, the diff computation is concluded by a call to
> > diff_result_code() to extract the exit status of the diff machinery.
> >
> > The function can work on "struct diffopt", but all the callers
> > historically and currently pass "struct diffopt" that is embedded in
> > the "struct rev_info" that is used to hold the remerge_diff bit and
> > the remerge_objdir variable that points at the temporary object
> > directory in use.
> >
> > Redefine diff_result_code() to take the whole "struct rev_info" to
> > give it an access to these members related to remerge-diff, so that
> > it can get rid of the temporary object directory for any and all
> > callers that used the feature.  We can lose the equivalent code to
> > do so from the code paths for individual commands, diff-tree, diff,
> > and log.
> >
> > Signed-off-by: Junio C Hamano <gitster@pobox.com>
> > ---
>
> I forgot to add that I am not happy with this "centralized tear
> down" step, even though I am reasonably happy with the "lazy set-up"
> step.  I wonder why the remerge-diff related members have to exist
> in the rev_info structure in the first place, instead of being in
> the diffopt structure?  Moving them to diffopt may make the end
> result much more pleasant to read.

I'm not sure they need to exist in the rev_info structure.  I was
probably thinking that e.g. log --remerge-diff would "need to do lots
of diffs" but I only needed the temporary store setup once, and once
in my mind matched better with rev_info.  I wasn't aware of
diff_options.no_free or anything surrounding it.  If we can do the
temporary store setup in diffopt and only do it once for all N diffs
in a `git log --remerge-diff` run, then we could move this stuff from
rev_info to diffopt.

Re: [PATCH 2/2] remerge-diff: clean up temporary objdir at a central place

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

>> I forgot to add that I am not happy with this "centralized tear
>> down" step, even though I am reasonably happy with the "lazy set-up"
>> step.  I wonder why the remerge-diff related members have to exist
>> in the rev_info structure in the first place, instead of being in
>> the diffopt structure?  Moving them to diffopt may make the end
>> result much more pleasant to read.
>
> I'm not sure they need to exist in the rev_info structure.  I was
> probably thinking that e.g. log --remerge-diff would "need to do lots
> of diffs" but I only needed the temporary store setup once, and once
> in my mind matched better with rev_info.  I wasn't aware of
> diff_options.no_free or anything surrounding it.  If we can do the
> temporary store setup in diffopt and only do it once for all N diffs
> in a `git log --remerge-diff` run, then we could move this stuff from
> rev_info to diffopt.

After staring at the data structure around this area for a while, I
think these two patches are OK, especially within the current
structure.  rev_info structure has a set of bits that are marked as
"Diff flags" (some of which are not about diff at all, though; for
example, verbose_header and no_commit_id belong more to the "Format
info" group), and those related to merges do belong there, as they
need more than "here are two tree-like things I want to compare".

So, I'll mark these two patches for 'next' now.

Thanks.