[PATCH] filter-branch: skip index read/write when possible

[Jeff King <peff@peff.net>]

On Thu, Nov 05, 2015 at 07:20:48PM -0500, Jeff King wrote:

> Here's a totally untested patch that seems to make a filter-branch like
> this on the kernel orders of magnitude faster:

Testing shows that it is indeed broken. :)

If $filter_subdir is set, it handles the index read itself, but my
earlier patch did not correctly do the write.

This one should work for all cases (unless the user does something
really strange, like expect to manipulate the index inside the
--env-filter or something, but IMHO it is insane for anyone to rely on
that working).

-- >8 --
Subject: filter-branch: skip index read/write when possible

If the user specifies an index filter but not a tree filter,
filter-branch cleverly avoids checking out the tree
entirely. But we don't do the next level of optimization: if
you have no index or tree filter, we do not need to read the
index at all.

This can greatly speed up cases where we are only changing
the commit objects (e.g., cementing a graft into place).
Here are numbers from the newly-added perf test:

  Test                  HEAD^              HEAD
  ---------------------------------------------------------------
  7000.2: noop filter   13.81(4.95+0.83)   5.43(0.42+0.43) -60.7%

Signed-off-by: Jeff King <peff@peff.net>
---
Those numbers are from git.git. The bigger your tree, the better the
speedup (I didn't run the perf test, because even the span of
HEAD~100..HEAD takes tens of minutes for each trial with the old code.
With the new it's less than 30 seconds).

 git-filter-branch.sh          | 23 +++++++++++++++++++++--
 t/perf/p7000-filter-branch.sh | 19 +++++++++++++++++++
 2 files changed, 40 insertions(+), 2 deletions(-)
 create mode 100755 t/perf/p7000-filter-branch.sh

diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 27c9c54..d61f9ba 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -306,6 +306,15 @@ then
 	start_timestamp=$(date '+%s')
 fi
 
+if test -n "$filter_index" ||
+   test -n "$filter_tree" ||
+   test -n "$filter_subdir"
+then
+	need_index=t
+else
+	need_index=
+fi
+
 while read commit parents; do
 	git_filter_branch__commit_count=$(($git_filter_branch__commit_count+1))
 
@@ -313,7 +322,10 @@ while read commit parents; do
 
 	case "$filter_subdir" in
 	"")
-		GIT_ALLOW_NULL_SHA1=1 git read-tree -i -m $commit
+		if test -n "$need_index"
+		then
+			GIT_ALLOW_NULL_SHA1=1 git read-tree -i -m $commit
+		fi
 		;;
 	*)
 		# The commit may not have the subdirectory at all
@@ -387,8 +399,15 @@ while read commit parents; do
 	} <../commit |
 		eval "$filter_msg" > ../message ||
 			die "msg filter failed: $filter_msg"
+
+	if test -n "$need_index"
+	then
+		tree=$(git write-tree)
+	else
+		tree="$commit^{tree}"
+	fi
 	workdir=$workdir @SHELL_PATH@ -c "$filter_commit" "git commit-tree" \
-		$(git write-tree) $parentstr < ../message > ../map/$commit ||
+		"$tree" $parentstr < ../message > ../map/$commit ||
 			die "could not write rewritten commit"
 done <../revs
 
diff --git a/t/perf/p7000-filter-branch.sh b/t/perf/p7000-filter-branch.sh
new file mode 100755
index 0000000..15ee5d1
--- /dev/null
+++ b/t/perf/p7000-filter-branch.sh
@@ -0,0 +1,19 @@
+#!/bin/sh
+
+test_description='performance of filter-branch'
+. ./perf-lib.sh
+
+test_perf_default_repo
+test_checkout_worktree
+
+test_expect_success 'mark bases for tests' '
+	git tag -f tip &&
+	git tag -f base HEAD~100
+'
+
+test_perf 'noop filter' '
+	git checkout --detach tip &&
+	git filter-branch -f base..HEAD
+'
+
+test_done
-- 
2.6.2.711.g30c79de