Re: Packfile can't be mapped

[Shawn Pearce <spearce@spearce.org>]

Nicolas Pitre <nico@cam.org> wrote:
> On Sun, 27 Aug 2006, Shawn Pearce wrote:
> 
> > I'm going to try to get tree deltas written to the pack sometime this
> > week. That should compact this intermediate pack down to something
> > that git-pack-objects would be able to successfully mmap into a
> > 32 bit address space.  A complete repack with no delta reuse will
> > hopefully generate a pack closer to 400 MB in size.  But I know
> > Jon would like to get that pack even smaller.  :)
> 
> One thing to consider in your code (if you didn't implement that 
> already) is to _not_ attempt any delta on any object whose size is 
> smaller than 50 bytes, and then limit the maximum delta size to 
> object_size/2 - 20 (use that for the last argument to diff-delta() and 
> store the undeltified object when diff-delta returns NULL).  This way 
> you'll avoid creating delta objects that are most likely to end up being 
> _larger_ than the undeltified object.

I haven't tried this.  Should be trivial to implement.  Thanks for
the suggestion.

> > I should point out that the input stream to fast-import was 20 GB
> > (completely decompressed revisions from RCS) plus all commit data.
> > The original CVS ,v files are around 3 GB.  An archive .tar.gz'ing
> > the ,v files is around 550 MB.  Going to only 1.7 GB without tree
> > or commit deltas is certainly pretty good.  :)
> 
> Good job indeed.  Oh and you probably should not bother trying to 
> deltify commit objects at all since that would be a waste of time.

I wasn't going to bother even trying to delta the commits.  In this
import the 200k commits isn't a very large percentage of the data.
As I'm sure you are well aware its pretty much a waste time to try
with the commits, especially with an "intermediate" pack such as
this one.

-- 
Shawn.