Re: Keeping unreachable objects in a separate pack instead of loose?

[Ted Ts'o <tytso@mit.edu>]

On Tue, Jun 12, 2012 at 02:25:47PM -0400, Nicolas Pitre wrote:
> > Earlier in the thread, I outlined another scheme by which you could
> > repack and avoid the duplicates. It does not require changes to git's
> > object lookup process, because it would involve manually feeding the
> > list of cruft objects to pack-objects (which will pack what you ask it,
> > regardless of whether the objects are in other packs).
> 
> That might be hard to achieve good delta compression though, as the main 
> key to sort those objects is their path name, and with unreferenced 
> objects you might not necessarily have that information.  The ability to 
> reuse pack data might mitigate this though.

Compared to loose objects, even not-so-great delta compression is
manna from heaven.  Remember what originally got me to start this
flag.  There was 4.5 megabytes worth of loose objects, that when I
created the object id list and fed the result to git pack-object, the
resulting pack was 244k.

OK, maybe the delta compression wasn't optimal.  Compared to the 4.5
megabytes of loose objects --- I'll happily settle for that!  :-)

> So the problem is really about 'git gc' creating more data on disk which 
> is counter productive for a garbage collecting task.  Maybe the trick is 
> simply not to delete any of the old pack which content was repacked into 
> a single new pack and let them age before deleting them, rather than 
> exploding a bunch of loose objects.  But then we're back to the same 
> issue I wanted to get away from i.e. identifying real cruft packs and 
> making them safely deletable.

But the old packs are huge; in my case, a full set of packs was around
16 megabytes.  Right now, git gc *increased* my disk usage by 4.5
megabytes.  If we don't delete the old backs, then git gc would
increase disk usage by 16 megabytes --- which is far, far worse.

Writing a 244k cruft pack is a soooooo much preferable.

	      	       	  	    - Ted