When to repack?

When to repack?

[Paul Gardiner]

There's advantage to repacking a repository after using git-fast-import.
If that repacked repository is then git-pushed to another, is there
any point in repacking the other afterwards? I'm guessing not, but
just checking.

P.

Re: When to repack?

[Karl Hasselström]

On 2008-02-29 10:55:41 +0000, Paul Gardiner wrote:

> There's advantage to repacking a repository after using
> git-fast-import. If that repacked repository is then git-pushed to
> another, is there any point in repacking the other afterwards? I'm
> guessing not, but just checking.

No, there isn't. The pack is essentially sent unmodified over the
wire. (IIRC, packs were originally conceived as a way of reducing
transfer bandwidth, and were later adopted for on-disk storage as
well. But I might be mistaken.)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: When to repack?

[Jakub Narebski]

Paul Gardiner <osronline@glidos.net> writes:

> There's advantage to repacking a repository after using git-fast-import.
> If that repacked repository is then git-pushed to another, is there
> any point in repacking the other afterwards? I'm guessing not, but
> just checking.

Yes, there is, to concatenate packs. But this should be less urgent.

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: When to repack?

[Xavier Maillard]

   There's advantage to repacking a repository after using git-fast-import.
   If that repacked repository is then git-pushed to another, is there
   any point in repacking the other afterwards? I'm guessing not, but
   just checking.

I am not an expert but I would not do it afterwards too.


	Xavier
-- 
http://www.gnu.org
http://www.april.org
http://www.lolica.org

When to repack?

[Bill Lear]

We have a company repo used by many people throughout the day.  When/how
can I repack this?  I have come to adopt this approach:

% mv project project.pack
% cd project.pack
% GIT_DIR=. git repack -a -d
% cd ..
% mv project.pack project


Bill

Re: When to repack?

[Jakub Narebski]

[Cc: git@vger.kernel.org]

Bill Lear wrote:

> We have a company repo used by many people throughout the day.  When/how
> can I repack this?  I have come to adopt this approach:
> 
> % mv project project.pack
> % cd project.pack
> % GIT_DIR=. git repack -a -d
> % cd ..
> % mv project.pack project

Repack is safe. You can do it on live repository. It does the save under
temporary name and move to proper place on it's own.

By the way you can use "git --bare repack -a -d" instead of 
"GIT_DIR=. git repack -a -d"
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: When to repack?

[Andreas Ericsson]

Bill Lear wrote:
> We have a company repo used by many people throughout the day.  When/how
> can I repack this?  I have come to adopt this approach:
> 
> % mv project project.pack
> % cd project.pack
> % GIT_DIR=. git repack -a -d
> % cd ..
> % mv project.pack project
> 

Renaming the directory isn't necessary. The packs won't be used until they
have a .idx file. That .idx-file is written after the packfile, so any
operations on the repo will simply use the old, loose, objects before the
packing is completed.

The worst thing that can happen is that an object about to be fetched is
deleted in its loose version before the upload-pack program can open it,
but that's no worse than having the entire directory being moved out from
under it.

On a side-note, this is a grade A example of something that should typically
be done sunday night at 4am.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: When to repack?

[Johannes Schindelin]

Hi,

On Wed, 31 Jan 2007, Andreas Ericsson wrote:

> Bill Lear wrote:
> > We have a company repo used by many people throughout the day.  When/how
> > can I repack this?  I have come to adopt this approach:
> > 
> > % mv project project.pack
> > % cd project.pack
> > % GIT_DIR=. git repack -a -d
> > % cd ..
> > % mv project.pack project
> > 
> 
> Renaming the directory isn't necessary. The packs won't be used until they
> have a .idx file. That .idx-file is written after the packfile, so any
> operations on the repo will simply use the old, loose, objects before the
> packing is completed.
> 
> The worst thing that can happen is that an object about to be fetched is
> deleted in its loose version before the upload-pack program can open it,
> but that's no worse than having the entire directory being moved out from
> under it.

AFAIR this case is handled gracefully by git. If the object it is still 
accessing moves to a(nother) pack, git will still find it.

> On a side-note, this is a grade A example of something that should 
> typically be done sunday night at 4am.

Actually, I'd recommend git-gc. It does not even call git-prune anymore, 
so there is no excuse.

I even do it interactively very often, and I just love the fact that "gc" 
is so much shorter than "repack -a -d", _plus_ it also does other cleanup 
tasks.

Ciao,
Dscho

Re: When to repack?

[Shawn O. Pearce]

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> On Wed, 31 Jan 2007, Andreas Ericsson wrote:
> > Bill Lear wrote:
> > > We have a company repo used by many people throughout the day.  When/how
> > > can I repack this?  I have come to adopt this approach:
> 
> AFAIR this case is handled gracefully by git. If the object it is still 
> accessing moves to a(nother) pack, git will still find it.

No AFAIR, its definately true.  `git gc` is completely safe on a
live repository.  Run it at will.  Toss it in a cronjob.  Whatever.

What is *not* safe is `git gc --prune`.  Don't run that on an
active repository.
 
> > On a side-note, this is a grade A example of something that should 
> > typically be done sunday night at 4am.

Possibly.  Almost doesn't matter when you run it, except on very huge
repositories where the repack would take more than a few minutes.

Really, just toss something like the following in a cronjob that
runs once a week:

	#!/bin/sh
	for g in /path/to/gits/*.git
	do
	  git --git-dir="$g" gc
	done

If you want to get fancy, use the output of `git count-objects -v`:

	count: 325
	size: 2332
	in-pack: 40894
	packs: 1
	prune-packable: 0
	garbage: 0

I look for a count over 2000 or packs over 5.  If either is true,
I run gc, otherwise I skip that repository and leave it alone
that week.  And that's actually packing more frequently than I
really need to.  On any UNIX system you can probably let those go
to >5,000 or 20 and still not really see a performance problem.

In general, repacks and network transfers (or basically any
operation) takes longer as the number of loose objects increases
(that's the count field in `git count-objects -v`).  Keep below
~2000 and `git gc` times tend to be measured in just a minute or
two for even 200 MiB repositories.

-- 
Shawn.