Re: Something is broken in repack

["Jon Smirl" <jonsmirl@gmail.com>]

New run using same configuration. With the addition of the more
efficient load balancing patches and delta cache accounting.

Seconds are wall clock time. They are lower since the patch made
threading better at using all four cores. I am stuck at 380-390% CPU
utilization for the git process.

complete seconds RAM
10%   60    900M (includes counting)
20%   15    900M
30%   15    900M
40%   50    1.2G
50%   80    1.3G
60%   70    1.7G
70%   140  1.8G
80%   180  2.0G
90%   280  2.2G
95%   530  2.8G - 1,420 total to here, previous was 1,983
100% 1390 2.85G
During the writing phase RAM fell to 1.6G
What is being freed in the writing phase??

I have no explanation for the change in RAM usage. Two guesses come to
mind. Memory fragmentation. Or the change in the way the work was
split up altered RAM usage.

Total CPU time was 195 minutes in 70 minutes clock time. About 70%
efficient. During the compress phase all four cores were active until
the last 90 seconds. Writing the objects took over 23 minutes CPU
bound on one core.

New pack file is: 270,594,853
Old one was: 344,543,752
It still has 828,660 objects


On 12/7/07, Jon Smirl <jonsmirl@gmail.com> wrote:
> Using this config:
> [pack]
>         threads = 4
>         deltacachesize = 256M
>         deltacachelimit = 0
>
> And the 330MB gcc pack for input
>  git repack -a -d -f  --depth=250 --window=250
>
> complete seconds RAM
> 10%  47 1GB
> 20%  29 1Gb
> 30%  24 1Gb
> 40%  18 1GB
> 50%  110 1.2GB
> 60%  85 1.4GB
> 70%  195 1.5GB
> 80%  186 2.5GB
> 90%  489 3.8GB
> 95%  800 4.8GB
> I killed it because it started swapping
>
> The mmaps are only about 400MB in this case.
> At the end the git process had 4.4GB of physical RAM allocated.
>
> Starting from a highly compressed pack greatly aggravates the problem.
> Starting with a 2GB pack of the same data my process size only grew to
> 3GB with 2GB of mmaps.
>
> --
> Jon Smirl
> jonsmirl@gmail.com
>


-- 
Jon Smirl
jonsmirl@gmail.com