Re: "git-send-pack"

["H. Peter Anvin" <hpa@zytor.com>]

Linus Torvalds wrote:
> 
> On Thu, 30 Jun 2005, H. Peter Anvin wrote:
> 
>>It does that, but it only have to do that when the actual file has 
>>changed.  That's acceptable, at least for the repository sizes we're 
>>likely to deal with within the medium term.
> 
> 
> Well, realize that "incremental packs" deltify a lot worse than a "big
> pack", since pack-files don't do deltas to objects outside the pack-file.
> 
> So we'd get _some_ compression, but not as much as possible. The current
> kernel compresses down to a single 63 MB pack-file (that's with the 2.6.11
> tree too, not just the HEAD history), but without deltas it weights in at
> about 177 MB.
> 
> So a "sum of incremental packs" should be somewhere in between those two
> values, even today. For a single kernel archive.
> 
> So repository sizes aren't exactly trivial. I don't know how expensive
> that rsync hash thing is, but one thing you lose is the ability to
> hardlink objects, so if you have a few kernel repositories at some point
> it doesn't fit in the cache any more, and then the rsync will have to read
> that much pack object stuff from disk in addition to doing the hash. Ugh.
> 

The bulk of the cost in doing the hashing comes from having to read the 
file.

Well, if you grow a single pack file with appending, then you can have 
delta references to earlier objects within the same pack file.

At least at this point, we'd handle a few very large files a lot better 
than an enormous swarm of smaller ones.

In the end, it might be that the right thing to do for git on kernel.org 
is to have a single, unified object store which isn't accessible by 
anything other than git-specific protocols.  There would have to be some 
way of dealing with, for example, conflicting tags that apply to 
different repositories, though.

	-hpa