[PATCH] git-am: Run git gc only once and not for every patch.

[PATCH] git-am: Run git gc only once and not for every patch.

[Michael Stefaniuc]

[-- Attachment #1: Type: text/plain, Size: 939 bytes --]

With "too many unreachable loose objects" git gc --auto will always
trigger. This clutters the output of git am and thus git rebase.

The work flow of the Wine project doesn't include git merge. git rebase
is therefor used to track the origin. This will produce soon too many
loose objects for git gc --auto's taste. Pruning the repository would
"fix" it. But we tell Wine developers new to git to NOT prune as long as
they aren't confident enough with git; just as a safety net in case they
have thrown away month of work.

Signed-off-by: Michael Stefaniuc <mstefani@redhat.com>
---
 git-am.sh |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/git-am.sh b/git-am.sh
index 65c634f..5f0f241 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -469,9 +469,9 @@ do
 		"$GIT_DIR"/hooks/post-applypatch
 	fi
 
-	git gc --auto
-
 	go_next
 done
 
+git gc --auto
+
 rm -fr "$dotest"
-- 
1.5.4.rc2

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] git-am: Run git gc only once and not for every patch.

[Nicolas Pitre]

On Fri, 4 Jan 2008, Michael Stefaniuc wrote:

> With "too many unreachable loose objects" git gc --auto will always
> trigger. This clutters the output of git am and thus git rebase.
> 
> The work flow of the Wine project doesn't include git merge. git rebase
> is therefor used to track the origin. This will produce soon too many
> loose objects for git gc --auto's taste. Pruning the repository would
> "fix" it. But we tell Wine developers new to git to NOT prune as long as
> they aren't confident enough with git; just as a safety net in case they
> have thrown away month of work.

The safety is the reflog.  What it refers to doesn't get pruned.


Nicolas

Re: [PATCH] git-am: Run git gc only once and not for every patch.

[Michael Stefaniuc]

Nicolas Pitre wrote:
> On Fri, 4 Jan 2008, Michael Stefaniuc wrote:
> 
>> With "too many unreachable loose objects" git gc --auto will always
>> trigger. This clutters the output of git am and thus git rebase.
>>
>> The work flow of the Wine project doesn't include git merge. git rebase
>> is therefor used to track the origin. This will produce soon too many
>> loose objects for git gc --auto's taste. Pruning the repository would
>> "fix" it. But we tell Wine developers new to git to NOT prune as long as
>> they aren't confident enough with git; just as a safety net in case they
>> have thrown away month of work.
> 
> The safety is the reflog.  What it refers to doesn't get pruned.
Then git gc --auto should just prune too and not spam, right?
But the reflog is only there for branches that still exist; the rest is
gone. A git stash clear will also remove the reflog for the stash.

Regardless of the safety of git prune i don't see a reason why git gc
--auto needs to be called for every patch in a git-am run and not once
at the end.

bye
	michael

Re: [PATCH] git-am: Run git gc only once and not for every patch.

[Junio C Hamano]

Nicolas Pitre <nico@cam.org> writes:

> On Fri, 4 Jan 2008, Michael Stefaniuc wrote:
>
>> With "too many unreachable loose objects" git gc --auto will always
>> trigger. This clutters the output of git am and thus git rebase.
>> 
>> The work flow of the Wine project doesn't include git merge. git rebase
>> is therefor used to track the origin. This will produce soon too many
>> loose objects for git gc --auto's taste. Pruning the repository would
>> "fix" it. But we tell Wine developers new to git to NOT prune as long as
>> they aren't confident enough with git; just as a safety net in case they
>> have thrown away month of work.
>
> The safety is the reflog.  What it refers to doesn't get pruned.

What Nico said.

More importantly, ones who are not confident with git would not
be able to resurrect unreachables that are left unpruned anyway.
The unreachables are by definition not connected to anything, so
they cannot do much better than grepping through droppings left
by "fsck --lost-found", which they probably even do not know how
to do yet.

Re: [PATCH] git-am: Run git gc only once and not for every patch.

[Nicolas Pitre]

On Fri, 4 Jan 2008, Michael Stefaniuc wrote:

> Nicolas Pitre wrote:
> > On Fri, 4 Jan 2008, Michael Stefaniuc wrote:
> > 
> >> With "too many unreachable loose objects" git gc --auto will always
> >> trigger. This clutters the output of git am and thus git rebase.
> >>
> >> The work flow of the Wine project doesn't include git merge. git rebase
> >> is therefor used to track the origin. This will produce soon too many
> >> loose objects for git gc --auto's taste. Pruning the repository would
> >> "fix" it. But we tell Wine developers new to git to NOT prune as long as
> >> they aren't confident enough with git; just as a safety net in case they
> >> have thrown away month of work.
> > 
> > The safety is the reflog.  What it refers to doesn't get pruned.
> Then git gc --auto should just prune too and not spam, right?

Pruning might be dangerous if some other operation is happening 
concurrently, which is why it is not done by default.

> But the reflog is only there for branches that still exist; the rest is
> gone. A git stash clear will also remove the reflog for the stash.

But the reflog for HEAD remains nevertheless, and it contains everything 
that happened to be the current branch.

> Regardless of the safety of git prune i don't see a reason why git gc
> --auto needs to be called for every patch in a git-am run and not once
> at the end.

Indeed.


Nicolas

Re: [PATCH] git-am: Run git gc only once and not for every patch.

[Michael Stefaniuc]

Junio C Hamano wrote:
> Nicolas Pitre <nico@cam.org> writes:
> 
>> On Fri, 4 Jan 2008, Michael Stefaniuc wrote:
>>
>>> With "too many unreachable loose objects" git gc --auto will always
>>> trigger. This clutters the output of git am and thus git rebase.
>>>
>>> The work flow of the Wine project doesn't include git merge. git rebase
>>> is therefor used to track the origin. This will produce soon too many
>>> loose objects for git gc --auto's taste. Pruning the repository would
>>> "fix" it. But we tell Wine developers new to git to NOT prune as long as
>>> they aren't confident enough with git; just as a safety net in case they
>>> have thrown away month of work.
>> The safety is the reflog.  What it refers to doesn't get pruned.
> 
> What Nico said.
Not sure if you read my reply to Nico but the reflog is not there for
deleted branches. Nor for a cleared stash. Common operations where one
can loose work by mistake.

> More importantly, ones who are not confident with git would not
> be able to resurrect unreachables that are left unpruned anyway.
> The unreachables are by definition not connected to anything, so
> they cannot do much better than grepping through droppings left
> by "fsck --lost-found", which they probably even do not know how
> to do yet.
The smarter ones come and ask on irc. We helped a few people with git
problems on the Wine channels. Depending on the amount of time someone
put into his "lost" work grepping through "fsck --lost-found" might be
well worth it.

Nevertheless is there a reason why git gc needs to run after applying
each patch in git-am? Why can't it run just once at the end? git prune
is optional and there's no reason to penalize a user that doesn't feel
safe to run it by cluttering the output of git am/rebase. There is also
a time penalty: git gc --auto on a pruned tree runs so fast that it
isn't measurable but on my unpruned wine it took 1.5 seconds. Waiting
1.5 seconds per am/rebase is acceptable; wasting 1.5 seconds per patch
in the mailbox/rebase isn't that much fun if there are more than a
handful of patches to apply.

bye
	michael
-- 
Michael Stefaniuc                           Tel.: +49-711-96437-199
Consulting Communications Engineer          Fax.: +49-711-96437-111
--------------------------------------------------------------------
Reg. Adresse: Red Hat GmbH, Hauptstätter Strasse 58, 70178 Stuttgart
Handelsregister: Amtsgericht Stuttgart HRB 153243
Geschäftsführer: Brendan Lane, Charlie Peters, Michael Cunningham,
                 Werner Knoblich

Re: [PATCH] git-am: Run git gc only once and not for every patch.

[Junio C Hamano]

Michael Stefaniuc <mstefani@redhat.com> writes:

>> What Nico said.
> Not sure if you read my reply to Nico but the reflog is not there for
> deleted branches. Nor for a cleared stash. Common operations where one
> can loose work by mistake.

I am sure you read what Nico pointed out about HEAD reflog.

> Nevertheless is there a reason why git gc needs to run after applying
> each patch in git-am? Why can't it run just once at the end? git prune
> is optional and there's no reason to penalize a user that doesn't feel
> safe to run it by cluttering the output of git am/rebase. There is also
> a time penalty: git gc --auto on a pruned tree runs so fast that it
> isn't measurable but on my unpruned wine it took 1.5 seconds. Waiting
> 1.5 seconds per am/rebase is acceptable; wasting 1.5 seconds per patch
> in the mailbox/rebase isn't that much fun if there are more than a
> handful of patches to apply.

"Optional" does not mean "whether you do it or not you will see
identical behaviour".  Not pruning may slow things down but you
may still choose not to prune and suffer the penalty --- that's
your choice.  But we still produce the correct result (hopefully
;-) --- and that is what "Optional" means.

The message from "gc --auto" may serve as a coalmine canary for
you to know when to prune.

I do not think moving "gc --auto" outside the loop hurts in
practice because you are not likely to be rebasing a truly huge
series every day, but cruft can accumulate during "git am" run
and the "gc --auto" inside loop was meant to clean them up.  The
idea was taken from importers that run repack every once in a
while (e.g. cvsimport runs every 1k commits), but "gc --auto"
was designed to be much more lightweight than a full repack and
that was the reason it was placed in the loop without counting
"every N commits".

Re: [PATCH] git-am: Run git gc only once and not for every patch.

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> ...
> I do not think moving "gc --auto" outside the loop hurts in
> practice because you are not likely to be rebasing a truly huge
> series every day, but cruft can accumulate during "git am" run
> and the "gc --auto" inside loop was meant to clean them up.  The
> idea was taken from importers that run repack every once in a
> while (e.g. cvsimport runs every 1k commits), but "gc --auto"
> was designed to be much more lightweight than a full repack and
> that was the reason it was placed in the loop without counting
> "every N commits".

Having said all that, I'll take your patch as-is except that I'd
drop the later part of the commit log message that explains
Wine's practice.

Re: [PATCH] git-am: Run git gc only once and not for every patch.

[Michael Stefaniuc]

Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
>> ...
>> I do not think moving "gc --auto" outside the loop hurts in
>> practice because you are not likely to be rebasing a truly huge
>> series every day, but cruft can accumulate during "git am" run
>> and the "gc --auto" inside loop was meant to clean them up.  The
>> idea was taken from importers that run repack every once in a
>> while (e.g. cvsimport runs every 1k commits), but "gc --auto"
>> was designed to be much more lightweight than a full repack and
>> that was the reason it was placed in the loop without counting
>> "every N commits".
> 
> Having said all that, I'll take your patch as-is except that I'd
Thanks.

> drop the later part of the commit log message that explains
> Wine's practice.
Yeah, sorry about that. That should have been outside of the commit
message anyway. But forgot about it as the Wine maintainer takes only
the subject as commit message.

bye
	michael