Re: Packfile can't be mapped

[Shawn Pearce <spearce@spearce.org>]

Shawn Pearce <spearce@spearce.org> wrote:
> I'm going to try to get tree deltas written to the pack sometime this
> week.

I was able to implement and with Jon Smirl's help debug the tree
delta code in fast-import.
 
Earlier this evening Jon sent me the following:
> git-fast-import statistics:
> ---------------------------------------------------------------------
> Alloc'd objects:    1980000 (         0 overflow  )
> Total objects:      1967527 (     41856 duplicates                  )
>       blobs  :       633842 (         0 duplicates     576219 deltas)
>       trees  :      1131208 (     41856 duplicates    1019741 deltas)
>       commits:       200921 (         0 duplicates          0 deltas)
>       tags   :         1556 (         0 duplicates          0 deltas)
> Total branches:        1600 (      2228 loads     )
>       marks:        1048576 (    200921 unique    )
>       atoms:          56803
> Memory total:         75213 KiB
>        pools:         13338 KiB
>      objects:         61875 KiB
> Pack remaps:            658
> Pack size:           895983 KiB
> Index size:           46114 KiB
> ---------------------------------------------------------------------

Compared to our last attempt:
> > Pack size:          1713200 KiB
> > Index size:           46114 KiB

This tree delta version came out pretty good.  The pack with tree
deltas is 874 MiB.  Quite a reduction in size.  fast-import takes
about 20 minutes to convert its 20 GiB input file into this 874 MiB
pack.  Producing the 20 GiB input file from the 3 GiB CVS ,v
files takes about 4 hours with Jon's modified cvs2svn.

Jon has started a `git-repack -a -f` with aggressive depth and
window sizes.  He estimated it may need another 2.5 hours to process.
Hopefully I'll hear more details tomorrow.

-- 
Shawn.