Re: Performance issue exposed by git-filter-branch

[Jonathan Nieder <jrnieder@gmail.com>]

Jonathan Nieder wrote:
> Ken Brownfield wrote:

>> The thread titled "git and larger trees, not so fast?".
>
> Here it is[1].  Sorry to say, the improvements discussed there
> were made right away and indeed had a dramatic effect.

Of course I missed your point. :)

filter-branch --index-filter works a little like this: for
each commit:

. find the underlying tree
. read-tree: unpack that tree and all of its subtrees into
the index file.  That is, convert from a recursive structure
   /:
	COPYING
	Documentation/
	INSTALL
	Makefile
	...

   Documentation/:
	CodingGuidelines
	Makefile
	...

into a flat structure

	COPYING
	Documentation/CodingGuideLines
	Documentation/Makefile
	Documentation/RelNotes/1.5.0.txt
	...
. rm: find entries matching certain patterns and remove them
from the index file.  This takes two passes through the index:
first to find matching entries, second to write the result to
disk.
. write-tree: write new trees for the object store.  That is,
convert from a flat structure back to a recursive structure.

This is convenient, but it does not sound to me like the most
efficient way to eliminate a few subtrees from each commit.  That is
why I was suggesting a method that avoids unpacking some trees
altogether.

That said, speedups for read-tree, rm, and write-tree would certainly
be nice to have.  One project of interest to some people is to give
the index file a recursive structure, so finding the entries to remove
in the "git rm" example could be faster.