cygwin: push/pull takes very long time

cygwin: push/pull takes very long time

[Alex Riesen]

Hi,

I didn't really notice when it happened, but since some time
push and pull takes *very* long. git-pack-objects appears to
be the major offender:
$ git pull ../src march:march
Generating pack...
Done counting 20238 objects.
Result has 12 objects.
Deltifying 12 objects.
   8% (1/12) done

It's still working, half an hour since it was started.
It can be seen that it is constantly growing and shrinking
(for about 20k,50k,80k,100k back and forth).

I did a small instrumentation to figure out how much
and how often is allocated, but it's waiting for the
git-pack-objects to finish. "Counting objects" was
slow too, with git-rev-list "pulsating" in a similar
way (most commonly allocated objects in the size
range from 16 to 256).

I continue looking for the problem (I didn't do bisect yet,
want to wait for mem profile first), but any ideas on how
to fix that are very welcome.

Re: cygwin: push/pull takes very long time

[Alex Riesen]

On 3/2/06, Alex Riesen <raa.lkml@gmail.com> wrote:
> It's still working, half an hour since it was started.
> It can be seen that it is constantly growing and shrinking
> (for about 20k,50k,80k,100k back and forth).

The distribution is (there are some git-commits and gitks
probably, here as git-read-tree?) below. The first number
after program name is lower range limit, the second - upper limit.
The number after them - counter of *alloc calls.
The strange pair 65536:99999 means very big objects, more than 1Mb.
I'll cleanup the profiling code and send it as well soon
(I had to instrument x*alloc).

      git-read-tree:16:256      127002
       git-rev-list:16:256      44775
        git-read-tree:0:16      22577
   git-pack-objects:16:256      12495
 git-pack-objects:256:4096      12303
git-pack-objects:4096:65536     6929
     git-diff-index:16:256      4381
     git-http-fetch:16:256      3535
     git-diff-index:16:256      2914
  git-write-tree:4096:65536     2876
       git-http-fetch:0:16      2111
     git-fetch-pack:16:256      1646
       git-fetch-pack:0:16      1522
     git-rev-list:256:4096      1067
                   :16:256      976
    git-read-tree:256:4096      965
   git-http-fetch:256:4096      835
                     :0:16      825
         git-ls-files:0:16      791
         git-rev-list:0:16      748
   git-fetch-pack:256:4096      698
    git-upload-pack:16:256      610
git-pack-objects:65536:1048576  594
      git-rev-parse:16:256      571
                 :256:4096      554
        git-rev-parse:0:16      492
      git-upload-pack:0:16      469
  git-upload-pack:256:4096      248
         git-cat-file:0:16      243
    git-peek-remote:16:256      242
       git-merge-base:0:16      229
     git-merge-base:16:256      208
       git-cat-file:16:256      167
    git-rev-parse:256:4096      141
      git-rev-parse:16:256      117
        git-rev-parse:0:16      116
     git-cat-file:256:4096      80
     git-pack-objects:0:16      73
       git-diff-index:0:16      65
   git-merge-base:256:4096      59
git-pack-objects:65536:99999    54
   git-current-branch:0:16      48
       git-diff-index:0:16      47
      git-diff-tree:16:256      42
   git-diff-index:256:4096      36
 git-unpack-objects:16:256      31
      git-diff-tree:16:256      30
       git-ls-files:16:256      27
   git-update-index:16:256      27
   git-diff-index:256:4096      24
     git-write-tree:16:256      23
       git-cat-file:16:256      22

Re: cygwin: push/pull takes very long time

[Alex Riesen]

Alex Riesen, Thu, Mar 02, 2006 18:09:23 +0100:
> I'll cleanup the profiling code and send it as well soon
> (I had to instrument x*alloc).

This is not exactly the same. It counts free as well, even if that is
not really interesting - there are places were there is more frees
than allocs. Probably something missed or a result coming from libc.

Also it is _not_ the code I used for windows. I had to have a global
variable for argv[0], which needs modification of all main()s, which
gets too easily out of sync.

BTW, maybe someone has an idea as to how attach valgrind to everything?
(I mean, without changing every script. Maybe modify git.c?)

Anyway, here it is, could be useful for something.

---

diff --git a/Makefile b/Makefile
index a5eb0c4..18fc802 100644
--- a/Makefile
+++ b/Makefile
@@ -206,7 +206,7 @@ LIB_OBJS = \
 	quote.o read-cache.o refs.o run-command.o \
 	server-info.o setup.o sha1_file.o sha1_name.o strbuf.o \
 	tag.o tree.o usage.o config.o environment.o ctype.o copy.o \
-	fetch-clone.o \
+	fetch-clone.o alloc.o \
 	$(DIFF_OBJS)
 
 LIBS = $(LIB_FILE)
diff --git a/alloc.c b/alloc.c
new file mode 100644
index 0000000..76727e6
--- /dev/null
+++ b/alloc.c
@@ -0,0 +1,108 @@
+/* simple allocation logging */
+#include <unistd.h>
+#include <limits.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include "git-compat-util.h"
+
+#undef free
+
+#define SHIFT 2
+static unsigned cnts[12] = {0,0,0,0, 0,0,0,0};
+static unsigned fcnt = 0;
+
+static int exit_hook = 1;
+
+static void alloc_profile()
+{
+	char argv0[PATH_MAX];
+	FILE *fp;
+	fp = fopen("/proc/self/cmdline", "r");
+	if ( !fp ) {
+		sprintf(argv0, "%d", getpid());
+	} else {
+		fread(argv0, sizeof(argv0), 1, fp);
+		fclose(fp);
+		char *s = strrchr(argv0, '/');
+		if ( s )
+			memmove(argv0, s + 1, strlen(s));
+	}
+	fp = fopen("/tmp/git-alloc", "ab");
+	if ( !fp )
+		return;
+	unsigned i, c = 0;
+	for ( i = 0; i < sizeof(cnts)/sizeof(*cnts); ++i ) {
+		if ( !cnts[i] )
+			continue;
+		fprintf(fp, "%s %u %u:%u %u times\n",
+			argv0,
+			i,
+			i ? 1 << i * SHIFT: 0,
+			1 << (i+1) * SHIFT,
+			cnts[i]);
+		c += cnts[i];
+	}
+	fprintf(fp, "%s alloc-free = %u\n", argv0, c - fcnt);
+	fclose(fp);
+}
+
+static inline void count(size_t size)
+{
+	if ( exit_hook ) {
+		exit_hook = 0;
+		atexit(alloc_profile);
+	}
+        unsigned i = 0;
+	while ( size && i < sizeof(cnts)/sizeof(*cnts) ) {
+		size >>= SHIFT;
+		if ( size )
+			++i;
+		else {
+			++cnts[i];
+			return;
+		}
+	}
+	++cnts[sizeof(cnts)/sizeof(*cnts)-1];
+}
+
+void *xmalloc(size_t size)
+{
+	void *ret = malloc(size);
+	count(size);
+	if (!ret && !size)
+		ret = malloc(1);
+	if (!ret)
+		die("Out of memory, malloc failed, %u bytes requested", size);
+	return ret;
+}
+
+void *xrealloc(void *ptr, size_t size)
+{
+	void *ret = realloc(ptr, size);
+	count(size);
+	if (!ret && !size)
+		ret = realloc(ptr, 1);
+	if (!ret)
+		die("Out of memory, realloc failed, %u bytes requested", size);
+	return ret;
+}
+
+void *xcalloc(size_t nmemb, size_t size)
+{
+	void *ret = calloc(nmemb, size);
+	count(size);
+	if (!ret && (!nmemb || !size))
+		ret = calloc(1, 1);
+	if (!ret)
+		die("Out of memory, calloc failed");
+	return ret;
+}
+
+void xfree(void *ptr)
+{
+	if ( !ptr )
+		return;
+	free(ptr);
+	++fcnt;
+}
+
diff --git a/git-compat-util.h b/git-compat-util.h
index f982b8e..ab4f855 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -73,35 +73,11 @@ extern void gitunsetenv(const char *);
 extern char *gitstrcasestr(const char *haystack, const char *needle);
 #endif
 
-static inline void *xmalloc(size_t size)
-{
-	void *ret = malloc(size);
-	if (!ret && !size)
-		ret = malloc(1);
-	if (!ret)
-		die("Out of memory, malloc failed");
-	return ret;
-}
-
-static inline void *xrealloc(void *ptr, size_t size)
-{
-	void *ret = realloc(ptr, size);
-	if (!ret && !size)
-		ret = realloc(ptr, 1);
-	if (!ret)
-		die("Out of memory, realloc failed");
-	return ret;
-}
-
-static inline void *xcalloc(size_t nmemb, size_t size)
-{
-	void *ret = calloc(nmemb, size);
-	if (!ret && (!nmemb || !size))
-		ret = calloc(1, 1);
-	if (!ret)
-		die("Out of memory, calloc failed");
-	return ret;
-}
+void *xmalloc(size_t size);
+void *xrealloc(void *ptr, size_t size);
+void *xcalloc(size_t nmemb, size_t size);
+void xfree(void *ptr);
+#define free(p) xfree(p)
 
 static inline ssize_t xread(int fd, void *buf, size_t len)
 {
diff --git a/prof-decode.sh b/prof-decode.sh
new file mode 100755
index 0000000..9c07e05
--- /dev/null
+++ b/prof-decode.sh
@@ -0,0 +1,25 @@
+#!/bin/bash
+if [ $# = 0 ]; then
+    set -- /tmp/git-alloc
+fi
+cat "$@" | perl -e '
+    while(<>) {
+        if (/(\S*) \d+ (\d+:\d+) (\d+)/) {
+		$c{$1}->{$2} += $3;
+		$c{$1}->{all} += $3
+	}
+        if (/(\S*) alloc-free = (\d+)/) { $d{$1} += $2 }
+    }
+    foreach $k (sort {$c{$b}->{all} <=> $c{$a}->{all}} grep {$c{$_}} keys %c) {
+	for ( sort {
+		if ( $a eq "all" ) { -1 }
+		elsif ( $b eq "all" ) { 1 }
+		else {
+		"$a $b" =~ /(\d+):\d+ (\d+):\d+/;
+		$1 <=> $2 }
+	    } grep { $c{$k}->{$_} } keys %{$c{$k}} ) {
+	    printf "%20s\t%20s\t%d\n", $k, $_, $c{$k}->{$_}
+	}
+	printf "%20s\t%d leaks\n\n", $k, $d{$k}
+    }
+'

Re: cygwin: push/pull takes very long time

[Christopher Faylor]

On Thu, Mar 02, 2006 at 10:54:08PM +0100, Alex Riesen wrote:
>Alex Riesen, Thu, Mar 02, 2006 18:09:23 +0100:
>>I'll cleanup the profiling code and send it as well soon (I had to
>>instrument x*alloc).
>
>This is not exactly the same.  It counts free as well, even if that is
>not really interesting - there are places were there is more frees than
>allocs.  Probably something missed or a result coming from libc.
>
>Also it is _not_ the code I used for windows.  I had to have a global
>variable for argv[0], which needs modification of all main()s, which
>gets too easily out of sync.

I wasn't following this discussion closely so maybe this is useless
information, but for Cygwin you can either use the undocumented global
__argv or you can use /proc/cmdline.  /proc/self/cmdline is going to be
pretty slow, however.

It looks like pure Windows console apps define _argv in stdlib.h also
but I've never used this and don't know if it is what it looks like.

cgf

Re: cygwin: push/pull takes very long time

[Alex Riesen]

On 3/3/06, Christopher Faylor <me@cgf.cx> wrote:
> On Thu, Mar 02, 2006 at 10:54:08PM +0100, Alex Riesen wrote:
> >Alex Riesen, Thu, Mar 02, 2006 18:09:23 +0100:
> >>I'll cleanup the profiling code and send it as well soon (I had to
> >>instrument x*alloc).
> >
> >This is not exactly the same.  It counts free as well, even if that is
> >not really interesting - there are places were there is more frees than
> >allocs.  Probably something missed or a result coming from libc.
> >
> >Also it is _not_ the code I used for windows.  I had to have a global
> >variable for argv[0], which needs modification of all main()s, which
> >gets too easily out of sync.
>
> I wasn't following this discussion closely so maybe this is useless
> information, but for Cygwin you can either use the undocumented global
> __argv or you can use /proc/cmdline.  /proc/self/cmdline is going to be
> pretty slow, however.

Oh, thanks. The speed is of no problem here: it's in atexit callback.

> It looks like pure Windows console apps define _argv in stdlib.h also
> but I've never used this and don't know if it is what it looks like.

It works as usual argv. I used "char **__argv", got the message from linker:
Info: resolving ___argv by linking to __imp____argv (auto-import)
which I almost expected.