Re: [PATCH 2/2] pack-objects: fix threaded load balancing

["Jon Smirl" <jonsmirl@gmail.com>]

On 12/10/07, Jon Smirl <jonsmirl@gmail.com> wrote:
> I just deleted the section looking for identical hashes.
>
> +                       while (sub_size && list[0]->hash &&
> +                              list[0]->hash == list[-1]->hash) {
> +                               list++;
> +                               sub_size--;
> +                       }
>
> Doing that allows the long chains to be split over the cores.
>
> My last 5% of objects is taking over 50% of the total CPU time in the
> repack. I think these objects are the ones from that 103,817 entry
> chain. It is also causing the explosion in RAM consumption.
>
> At the end I can only do 20 objects per clock second on four cores. It
> takes 30 clock minutes (120 CPU minutes) to do the last 3% of objects.

It's all in create_delta...

samples  %        symbol name
10344074 98.5961  create_delta
138010    1.3155  create_delta_index
4380      0.0417  find_deltas
2526      0.0241  patch_delta
776       0.0074  unpack_entry



>
> Can the chains be limited to not grow over some reasonable number, say
> 5,000? It will make the pack a little bigger but it will help a lot
> with performance.
>
> --
> Jon Smirl
> jonsmirl@gmail.com
>


-- 
Jon Smirl
jonsmirl@gmail.com