Re: [PATCH] Speedup recursive by flushing index only once for all entries

[fork0@t-online.de (Alex Riesen)]

Junio C Hamano, Thu, Jan 11, 2007 21:23:45 +0100:
> > That said, I think we actually have another problem entirely:
> >
> > Look at "write_cache()", Junio: isn't it leaking memory like mad?
> >
> > Shouldn't we have something like this?
> >
> > It's entirely possible that the _real_ problem with the "flush the index 
> > all the time" was that it just caused this bug: tons and tons of lost 
> > memory, causing git-merge-recursive to grow explosively (~6MB per 
> > cache flush, and a _lot_ of cache flushes), which on a 384MB machine 
> > quickly uses up memory and causes totally unnecessary swapping.
> 
> You are right -- there is absolutely no reason to retain this
> memory.  It is a serialized representation of cache-tree data
> only to be stored in the index, and no other user of this data
> exists.  Thanks for spotting this.
> 
> Writing out 6MB per every path changed in a merge would still be
> an unnecessary overhead over the one in 'next', so there is no
> reason to replace 'next' with this single liner of yours, but I
> am interested in seeing how much of the 20-minute vs 1-minute
> difference is attributable to this leak, just out of curiosity.
> 
> Alex, if you have a chance, could you apply Linus's single-liner
> on top of 'master', without either of the merge-recursive
> patches in 'next', and see what kind of numbers you would get?

With regard to speed: not noticable on the cygwin machine. The
384Mb-laptop liked it: moved into 40-50 sec range (it had real
problems (minutes) doing that merge without at least my first patch.
Because of the leak, as we now understand).

It must have been large leak, as I really have seen the memory usage
dropping down significantly.