Re: Anomaly with the new code - Re: git-svn performance

[Eric Wong <normalperson@yhbt.net>]

Hin-Tak Leung <htl10@users.sourceforge.net> wrote:
> I keep tabs of a particular svn repository over many years
> and run "git svn fetch --all" every few days. So that's the old clone.
> Since this discussion started, I made a new one with git 2.1.0 patched
> with the first two patches below, a couple of weeks ago. And I ran
> 'git svn fetch --all' on both every few days since.
> 
> I have added a few more patches, so the whole list is the 6
> below against 2.1.0. The latest fetch is really strange - the fetch against
> the new clone took almost twice as long and uses almost twice
> as much memory, vs against the old. 17 min, 800 MB vs 10 min 400MB.
> Details below. Maybe this is a performance issue about how the clones
> were made?

Memory usage seems to grow with the amount of revisions fetched,
see below.  And higher memory means slower fork() on Linux systems.

> 0001-git-svn-only-look-at-the-new-parts-of-svn-mergeinfo.patch    
> 0002-git-svn-only-look-at-the-root-path-for-svn-mergeinfo.patch   
> 0003-git-svn-reduce-check_cherry_pick-cache-overhead.patch        
> 0004-git-svn-cache-only-mergeinfo-revisions.patch                 

> 0006-git-svn-clear-global-SVN-pool-between-get_log-invoca.patch   

0006 is insufficient and incompatible with older SVN.
I pushed "git-svn: reload RA every log-window-size"
(commit dfa72fdb96befbd790f623bb2909a347176753c2) instead
which saves much more memory:

http://mid.gmane.org/20141024225352.GB31716@dcvr.yhbt.net

But there still seems to be some slow growth with many revisions
which is not mergeinfo-related.

> 0007-git-svn-remove-mergeinfo-rev-caching.patch 

I think it is also safe to remove the _rev_list memoization since
it uses a lot of memory.  The remaining caches should be tiny
(but useful, I think).