Re: cleaner/better zlib sources?

["Shawn O. Pearce" <spearce@spearce.org>]

Jeff Garzik <jeff@garzik.org> wrote:
> Although it sounds like zlib could indeed be optimized to reduce its 
> startup and shutdown overhead, I wonder if switching compression 
> algorithms to a pure Huffman or even RLE compression (with associated 
> lower startup/shutdown costs) would perform better in the face of all 
> those small objects.

As Nico already stated, for pack v4 we are probably heading in a
direction where these really small (except for blobs anyway) objects
aren't compressed at all by zlib.  They are smaller in disk space,
and are faster to reconstruct to their raw format.
 
> And another random thought, though it may be useless in this thread:  I 
> bet using a pre-built (compiled into git) static zlib dictionary for git 
> commit and tree objects might improve things a bit.

I've actually tried this with the Mozilla project.  The improvement
was under 2% on disk space usage and no runtime performance gains.
Not worth the pain involved.  We are seeing much higher disk
space improvements and much better performance gains in the pack
v4 prototype.

Oh, and that was *with* a dictionary that was customized to Mozilla.
Not a static one.  A lot of keywords in the dictionary were Mozilla
project specific, and would actually *hurt* compression for the
Linux kernel, Git, X.org, etc...

-- 
Shawn.