git-annotate - leaking hundrets of megabytes

git-annotate - leaking hundrets of megabytes

[Sven Herzberg]

Hey,

I just started to write a small tool that's supposed to provide a
graphical annotation view. It's supposed to display the information of
git-annotate and provide a way to browse through the history (so people
can click on links that refer to older revisions, etc.). According to
the man page of git-annotate, I looked into the incremental mode to make
sure the tool can load the information step by step.

However, to make sure the user interface doesn't block while
"git-annotate --incremental" is running, I decided to take a really slow
annotation while developing and then I saw that git-annotate leaks
*lots* of memory (at least in that use-case).

In my example, it used up to 450MB:
http://people.imendio.com/~sven/massif.18740.png

Reproduce:
1. git clone git://git.webkit.org/WebKit.git
2. cd WebKit/WebCore
3. git-annotate --incremental ChangeLog

Regards,
  Sven

PS: Please CC me as I'm not on the list.

Re: git-annotate - leaking hundrets of megabytes

[Linus Torvalds]

On Tue, 23 Oct 2007, Sven Herzberg wrote:
> 
> In my example, it used up to 450MB:
> http://people.imendio.com/~sven/massif.18740.png
> 
> Reproduce:
> 1. git clone git://git.webkit.org/WebKit.git
> 2. cd WebKit/WebCore
> 3. git-annotate --incremental ChangeLog

Hmm. I'm not seeing this at all. I do

	/usr/bin/time git-annotate --incremental ChangeLog > /dev/null

in WebCore, and it returns

	0.16user 0.01system 0:00.18elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
	0inputs+0outputs (0major+8365minor)pagefaults 0swaps

where that minor pagefault count is a good estimate of total memory used. 
8365 minor page faults implies it only ever got 32MB of memory total.

What git version do you have, or did you perhaps mean some other file or 
other config that causes this?

(It does end up using lots more memory if you ask for "-C" to see if there 
are copies from other sources, but even then I'm not seeing anything 
close to 450MB!)

			Linus

Re: git-annotate - leaking hundrets of megabytes

[Florian Weimer]

* Linus Torvalds:

> What git version do you have, or did you perhaps mean some other file or 
> other config that causes this?

I see it with Debian's 1:1.5.3.4-1 version, but also with dd8175f83c725.
In my case, it's a git-svn mirror of:

  svn://svn.debian.org/secure-testing/data/CVE/list

This is a very difficult file for git-annotate (and "svn blame") because
of its history and size.