Git development
 help / color / mirror / Atom feed
From: Avery Pennarun <apenwarr@gmail.com>
To: git@vger.kernel.org
Cc: Avery Pennarun <apenwarr@gmail.com>
Subject: [PATCH 2/3] Make filter-branch --glob-filter much faster by not calling 'cat'
Date: Wed, 23 Apr 2008 15:42:36 -0400	[thread overview]
Message-ID: <1208979757-30860-2-git-send-email-apenwarr@gmail.com> (raw)
In-Reply-To: <1208979757-30860-1-git-send-email-apenwarr@gmail.com>

The main loop of munge_blobs() had to fork-exec "cat" every time through the
loop, even when a blob was already cached.  Let's use the sh builtin 'read'
instead for a huge speedup.

cd git
time git filter-branch --blob-filter 'tr a-z A-Z' HEAD~10..HEAD

(original --blob-filter)
real    3m58.569s
user    0m22.900s
sys     3m32.030s

(with 'cat' calls removed)
real	1m11.931s
user	0m8.520s
sys	1m2.900s

(with 'cat' calls removed and blob cache already filled)
real	0m19.660s
user	0m3.930s
sys	0m15.720s

Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
---
 git-filter-branch.sh |   18 +++++++++++-------
 1 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 980c431..37ac99d 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -57,16 +57,18 @@ eval "$functions"
 munge_blobs() {
 	while read mode sha1 stage path
 	do
-		if ! test -r "$workdir/../blob-cache/$sha1"
+		if ! test -r "$cachedir/$sha1"
 		then
-			new=`git cat-file blob $sha1 |
-			     eval "$filter_blob" |
-			     git hash-object -w --stdin`
-			printf $new >$workdir/../blob-cache/$sha1
+			new=$(git cat-file blob $sha1 |
+			      eval "$filter_blob" |
+			      git hash-object -w --stdin)
+			printf $new >$cachedir/$sha1
+		else
+			read new <"$cachedir/$sha1"
 		fi
 		printf "%s %s\t%s\n" \
 			"$mode" \
-			$(cat "$workdir/../blob-cache/$sha1") \
+			"$new" \
 			"$path"
 	done
 }
@@ -108,6 +110,7 @@ USAGE="[--env-filter <command>] [--tree-filter <command>] \
 [--index-filter <command>] [--parent-filter <command>] \
 [--msg-filter <command>] [--commit-filter <command>] \
 [--tag-name-filter <command>] [--subdirectory-filter <directory>] \
+[--blob-filter <command>] \
 [--original <namespace>] [-d <directory>] [-f | --force] \
 [<rev-list options>...]"
 
@@ -249,7 +252,8 @@ ret=0
 mkdir ../map || die "Could not create map/ directory"
 
 # cache rewritten blobs for blob filter
-mkdir ../blob-cache || die "Could not create blob-cache/ directory"
+cachedir="$workdir/../blob-cache"
+mkdir "$cachedir" || die "Could not create blob-cache/ directory"
 
 case "$filter_subdir" in
 "")
-- 
1.5.4.3

  reply	other threads:[~2008-04-23 19:44 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-04-23 19:42 [PATCH 1/3] Add --blob-filter option to filter-branch Avery Pennarun
2008-04-23 19:42 ` Avery Pennarun [this message]
2008-04-23 19:42   ` [PATCH 3/3] Update documentation to describe git filter-branch --blob-filter Avery Pennarun
2008-04-23 20:05 ` [PATCH 1/3] Add --blob-filter option to filter-branch Johannes Schindelin
2008-04-23 20:12   ` Avery Pennarun
2008-04-23 20:14     ` Johannes Schindelin
2008-04-23 20:18       ` [PATCH 1/3 v2] " Avery Pennarun
2008-04-23 20:22         ` Johannes Schindelin
2008-04-23 21:55         ` Jeff King
2008-04-23 22:07           ` Avery Pennarun
2008-04-24  1:33             ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1208979757-30860-2-git-send-email-apenwarr@gmail.com \
    --to=apenwarr@gmail.com \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox