From: Nix <nix@esperi.org.uk>
To: git@vger.kernel.org
Subject: git-pull followed by git-gc often yields *much* bigger pack than git-pull alone
Date: Wed, 12 Dec 2007 20:47:41 +0000 [thread overview]
Message-ID: <87ejdrej2q.fsf@hades.wkstn.nix> (raw)
I've seen this intermittently, but this is a particularly extreme
example. This happens to be with 1.5.3.7, but I've seen the same
symptoms back in git 1.4.x. I assumed it was something I was doing wrong
then, but I'm no longer so sure.
,----
| hades /usr/packages/poppler/poppler% ls -al .git/objects/pack
| drwxr-xr-x 2 compiler hackers 2048 2007-12-12 20:35 .
| drwxr-xr-x 9 compiler hackers 2048 2007-12-12 20:35 ..
| -r--r--r-- 1 compiler hackers 149288 2007-10-25 23:51 pack-f5a84fe3e212f9dc62a65aee617be4bf1387cb2b.idx
| -rw-r--r-- 1 compiler hackers 0 2007-12-06 14:45 pack-f5a84fe3e212f9dc62a65aee617be4bf1387cb2b.keep
| -r--r--r-- 1 compiler hackers 1610421 2007-10-25 23:51 pack-f5a84fe3e212f9dc62a65aee617be4bf1387cb2b.pack
|
| hades /usr/packages/poppler/poppler% git-count-objects
| 0 objects, 0 kilobytes
|
| hades /usr/packages/poppler/poppler% git-remote update
| Updating origin
| remote: Generating pack...
| remote: Done counting 491 objects.
| remote: Result has 351 objects.
| remote: Deltifying 351 objects.
| remote: 100% (351/351) done
| remote: Total 351, written 351 (delta 274), reused 0 (delta 0)
| * refs/remotes/origin/master: fast forward to branch 'master' of git://git.freedesktop.org/git/poppler/poppler
| old..new: ab0a6c3..f24259c
| * refs/remotes/origin/poppler-0.6: fast forward to branch 'poppler-0.6' of git://git.freedesktop.org/git/poppler/poppler
| old..new: 5c321d3..db1e360
| Auto-following refs/tags/poppler-0.6.2
| Auto-following refs/tags/poppler-0.6.3
| remote: Generating pack...
| remote: Done counting 2 objects.
| remote: Deltifying 2 objects.
| remote: /2) done
| remote: Total 2, written 2 (delta 0), reused 0 (delta 0)
| Unpacking 2 objects...
| 100% (2/2) done
| * refs/tags/poppler-0.6.2: storing tag 'poppler-0.6.2' of git://git.freedesktop.org/git/poppler/poppler
| tag: 26470b3
| * refs/tags/poppler-0.6.3: storing tag 'poppler-0.6.3' of git://git.freedesktop.org/git/poppler/poppler
| tag: fdf7e0b
|
| hades /usr/packages/poppler/poppler% ls -al .git/objects/pack
| drwxr-xr-x 2 compiler hackers 2048 2007-12-12 20:35 .
| drwxr-xr-x 6 compiler hackers 2048 2007-12-12 20:35 ..
| -r--r--r-- 1 compiler hackers 12080 2007-12-12 20:35 pack-f534c957f3ee7f3f1046dee0081fcc315d9698bb.idx
| -r--r--r-- 1 compiler hackers 678643 2007-12-12 20:35 pack-f534c957f3ee7f3f1046dee0081fcc315d9698bb.pack
| -r--r--r-- 1 compiler hackers 149288 2007-10-25 23:51 pack-f5a84fe3e212f9dc62a65aee617be4bf1387cb2b.idx
| -rw-r--r-- 1 compiler hackers 0 2007-12-06 14:45 pack-f5a84fe3e212f9dc62a65aee617be4bf1387cb2b.keep
| -r--r--r-- 1 compiler hackers 1610421 2007-10-25 23:51 pack-f5a84fe3e212f9dc62a65aee617be4bf1387cb2b.pack
`----
OK, so that's a 662Kb packfile, 351/274...
[... a couple of trivial merges, then ...]
,----
| hades /usr/packages/poppler/poppler% git-count-objects
| 5 objects, 10 kilobytes
|
| hades /usr/packages/poppler/poppler% git-gc
| Generating pack...
| Done counting 827 objects.
| Deltifying 827 objects...
| 100% (827/827) done
| Writing 827 objects...
| 100% (827/827) done
| Total 827 (delta 393), reused 695 (delta 331)
| Pack pack-5145ce9770077cee99795fc21cfcc6ea30eb6d47 created.
| Removing unused objects 100%...
| Done.
|
| hades /usr/packages/poppler/poppler% ls -al .git/objects/pack
| drwxr-xr-x 2 compiler hackers 2048 2007-12-12 20:36 .
| drwxr-xr-x 4 compiler hackers 2048 2007-12-12 20:36 ..
| -r--r--r-- 1 compiler hackers 20912 2007-12-12 20:36 pack-5145ce9770077cee99795fc21cfcc6ea30eb6d47.idx
| -r--r--r-- 1 compiler hackers 1144055 2007-12-12 20:36 pack-5145ce9770077cee99795fc21cfcc6ea30eb6d47.pack
| -r--r--r-- 1 compiler hackers 149288 2007-10-25 23:51 pack-f5a84fe3e212f9dc62a65aee617be4bf1387cb2b.idx
| -rw-r--r-- 1 compiler hackers 0 2007-12-06 14:45 pack-f5a84fe3e212f9dc62a65aee617be4bf1387cb2b.keep
| -r--r--r-- 1 compiler hackers 1610421 2007-10-25 23:51 pack-f5a84fe3e212f9dc62a65aee617be4bf1387cb2b.pack
`----
The git-gc figures are wildly different from the original server-side
packing, as is the size of the resulting packfile.
This has me scared to do git-gc, which is presumably not good if we're
supposed to get into the habit of running it whenever we have a spare
moment :)
A while back I did a git-gc on the kernel and watched as a 10Mb pack
resulting from a git-pull blew right up to 130Mb: I had to zap the .keep
file, do a complete repack, and blow a backup CD to get the space
back. (This only happens if you have .keep files lying around: if git-gc
is allowed to amalgamate all your packs, it doesn't do this.)
Possibly-relevant .gitconfig settings:
[core]
legacyheaders=false
deltabasecachelimit=32m
[pack]
window=50
aggressiveWindow=100
depth=100
[repack]
usedeltabaseoffset=true
(yes, those windows and depths are intentionally high. I was hoping that
maybe the servers were running with enormous windows... it didn't help.)
--
`The rest is a tale of post and counter-post.' --- Ian Rawlings
describes USENET
next reply other threads:[~2007-12-12 20:48 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-12-12 20:47 Nix [this message]
-- strict thread matches above, loose matches on Subject: below --
2007-12-12 21:53 git-pull followed by git-gc often yields *much* bigger pack than git-pull alone linux
2007-12-12 22:06 ` Nix
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87ejdrej2q.fsf@hades.wkstn.nix \
--to=nix@esperi.org.uk \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).