PATCH] Allow git-filter-branch to process large repositories with lots of branches.

PATCH] Allow git-filter-branch to process large repositories with lots of branches.

[Lee Carver]

As noted in several forums, a recommended way to move trees between
repositories is to use git-filter-branch to revise the history for a
single tree 
(http://gbayer.com/development/moving-files-from-one-git-repository-to-anot
her-preserving-history/,
http://stackoverflow.com/questions/1365541/how-to-move-files-from-one-git-r
epo-to-another-not-a-clone-preserving-history).

However, this can lead to argument list too long errors when the original
repository has many retained branches (>6k)

/usr/local/git/libexec/git-core/git-filter-branch: line 270:
/usr/local/git/libexec/git-core/git: Argument list too long
Could not get the commits

Piping the output from git rev-parse directly into git rev-list avoids
this problem, since the rev-parse output is not processed as a command
line argument.
---
 git-filter-branch.sh | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index ac2a005..d7e0fae 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -255,8 +255,6 @@ else
 	remap_to_ancestor=t
 fi
 
-rev_args=$(git rev-parse --revs-only "$@")
-
 case "$filter_subdir" in
 "")
 	eval set -- "$(git rev-parse --sq --no-revs "$@")"
@@ -267,8 +265,9 @@ case "$filter_subdir" in
 	;;
 esac
 
+git rev-parse --revs-only "$@" | \
 git rev-list --reverse --topo-order --default HEAD \
-	--parents --simplify-merges $rev_args "$@" > ../revs ||
+	--parents --simplify-merges --stdin "$@" > ../revs ||
 	die "Could not get the commits"
 commits=$(wc -l <../revs | tr -d " ")
 
-- 
1.8.3.2

Re: PATCH] Allow git-filter-branch to process large repositories with lots of branches.

[Andreas Schwab]

Lee Carver <Lee.Carver@servicenow.com> writes:

> diff --git a/git-filter-branch.sh b/git-filter-branch.sh
> index ac2a005..d7e0fae 100755
> --- a/git-filter-branch.sh
> +++ b/git-filter-branch.sh
> @@ -255,8 +255,6 @@ else
>  	remap_to_ancestor=t
>  fi
>  
> -rev_args=$(git rev-parse --revs-only "$@")
> -
>  case "$filter_subdir" in
>  "")
>  	eval set -- "$(git rev-parse --sq --no-revs "$@")"

Here the arguments are reset.

> @@ -267,8 +265,9 @@ case "$filter_subdir" in
>  	;;
>  esac
>  
> +git rev-parse --revs-only "$@" | \

So this is using a different argument list than before.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: PATCH] Allow git-filter-branch to process large repositories with lots of branches.

[Lee Carver]

Andreas -

So that's what the "eval set -- " does!

We actually got it to work with a temporary file Š I'll send that patch in
a minute.

Thanks
   Lee

On 9/7/13 1:29 PM, "Andreas Schwab" <schwab@linux-m68k.org> wrote:

>Lee Carver <Lee.Carver@servicenow.com> writes:
>
>> diff --git a/git-filter-branch.sh b/git-filter-branch.sh
>> index ac2a005..d7e0fae 100755
>> --- a/git-filter-branch.sh
>> +++ b/git-filter-branch.sh
>> @@ -255,8 +255,6 @@ else
>>  	remap_to_ancestor=t
>>  fi
>>  
>> -rev_args=$(git rev-parse --revs-only "$@")
>> -
>>  case "$filter_subdir" in
>>  "")
>>  	eval set -- "$(git rev-parse --sq --no-revs "$@")"
>
>Here the arguments are reset.
>
>> @@ -267,8 +265,9 @@ case "$filter_subdir" in
>>  	;;
>>  esac
>>  
>> +git rev-parse --revs-only "$@" | \
>
>So this is using a different argument list than before.
>
>Andreas.
>
>-- 
>Andreas Schwab, schwab@linux-m68k.org
>GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
>"And now for something completely different."