Re: Debugging git-commit slowness on a large repo

[Joshua Redstone <joshua.redstone@fb.com>]

Hi Duy,
Thanks for the documentation link.

git ls-files shows 100k files, which matches # of files in the working
tree ('find . -type f -print | wc -l').

I added a 'git read-tree HEAD' before the git-add, and a 'git write-tree'
after the add.  With that, the commit time slowed down to 8 seconds per
commit, plus 4 more seconds for the read-tree/add/write-tree ops.  The
read-tree/add/write-tree each took about a second.

As an experiment, I also tried removing the 'git read-tree' and just
having the git-write-tree.  That sped up commits to 0.6 seconds, but the
overall time for add/write-tree/commit was still 3 to 6 seconds.

For comparison, without the read-tree and write-tree, commits take about 1
second and add/commit in total takes about 2 seconds.

It surprises me that the presence of git read-tree or write-tree would
slow things down so much.

Josh

On 12/6/11 6:08 PM, "Nguyen Thai Ngoc Duy" <pclouds@gmail.com> wrote:

>On Wed, Dec 7, 2011 at 8:48 AM, Joshua Redstone <joshua.redstone@fb.com>
>wrote:
>> I tried doing a 'git read-tree HEAD' before each 'git add ; git commit'
>> iteration, and the time for git-commit jumped from about 1 second to
>>about
>> 8 seconds.  That is a pretty dramatic slowdown.  Any idea why?  I wonder
>> if that's related to the overall commit slowness.
>
>How big is your working directory? "git ls-files | wc -l" should show
>it. Try "git read-tree HEAD; git add; git write-tree" and see if the
>write-tree part takes as much time as commit. write-tree is mainly
>about cache-tree generation.
>
>> @Carlos and/or @Junio, can you point me at any docs/code to understand
>> what a tree-cache is and how it differs from the index?  I did a google
>> search for [git tree-cache index], but nothing popped out.
>
>Have a look at Documentation/technical/index-format.txt. Cache tree
>extension is near the end.
>-- 
>Duy