[PATCH/RFC] Always auto-gc after calling a fast-import transport

[PATCH/RFC] Always auto-gc after calling a fast-import transport

[Stepan Kasal]

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Mon, 9 Apr 2012 13:04:35 -0500

After importing anything with fast-import, we should always let the
garbage collector do its job, since the objects are written to disk
inefficiently.

This brings down an initial import of http://selenic.com/hg from about
230 megabytes to about 14.

In the future, we may want to make this configurable on a per-remote
basis, or maybe teach fast-import about it in the first place.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
---

Hello,
another patch that waited in msysgit for 2 years.
Could anyone on the list try to reproduce the performance problem
that triggered this?

Thanks,
  Stepan

 transport-helper.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/transport-helper.c b/transport-helper.c
index a01ea47..956b800 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -14,6 +14,8 @@
 #include "refs.h"
 
 static int debug;
+/* TODO: put somewhere sensible, e.g. git_transport_options? */
+static int auto_gc = 1;
 
 struct helper_data {
 	const char *name;
@@ -519,6 +521,12 @@ static int fetch_with_import(struct transport *transport,
 		}
 	}
 	strbuf_release(&buf);
+	if (auto_gc) {
+		const char *argv_gc_auto[] = {
+			"gc", "--auto", "--quiet", NULL,
+		};
+		run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
+	}
 	return 0;
 }
 
-- 
1.9.2.msysgit.0.335.gd2a461f

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: [PATCH/RFC] Always auto-gc after calling a fast-import transport

[Felipe Contreras]

Stepan Kasal wrote:
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> Date: Mon, 9 Apr 2012 13:04:35 -0500
> 
> After importing anything with fast-import, we should always let the
> garbage collector do its job, since the objects are written to disk
> inefficiently.

I'm not sure how slow `git gc --auto` is with a totally unpacked
repository, but it guess it will be slow enough that some people might
not want to do the gc initially.

Not many people have complained about the size of the repoitory after
cloning, so I think it should be OK to let users call `git gc` when they
wish to.

-- 
Felipe Contreras

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: [PATCH/RFC] Always auto-gc after calling a fast-import transport

[Felipe Contreras]

Stepan Kasal wrote:
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> Date: Mon, 9 Apr 2012 13:04:35 -0500
> 
> After importing anything with fast-import, we should always let the
> garbage collector do its job, since the objects are written to disk
> inefficiently.
> 
> This brings down an initial import of http://selenic.com/hg from about
> 230 megabytes to about 14.
> 
> In the future, we may want to make this configurable on a per-remote
> basis, or maybe teach fast-import about it in the first place.

Actually I tested this patch and it makes no difference. Before and
after the patch the repository size is exactly the same (and so is the
run-time).

That is because after the initial import there are no loose objects,
every thing is into one big pack, so `git gc --auto` does nothing.

> Could anyone on the list try to reproduce the performance problem
> that triggered this?

I'd say this patch is not doing anything in recent versions of Git.

-- 
Felipe Contreras

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.