Re: cleaner/better zlib sources?

[Nicolas Pitre <nico@cam.org>]

On Fri, 16 Mar 2007, Linus Torvalds wrote:

> On Fri, 16 Mar 2007, Nicolas Pitre wrote:
> 
> > This is why in pack v4 there will be an alternate tree object 
> > representation which is not deflated at all.
> 
> Well, the thing is, for things that really don't compress, zlib shouldn't 
> add much of an overhead on uncompression. It *should* just end up being a 
> single "memcpy()" after you've done:
>  - check the header for size and mode ("plain data")
>  - check the adler checksum (which is *really* nice - we've found real 
>    corruption this way!).

But the thing is that with tree objects which records are 6 fairly 
random bytes we already know that compression will never be worth it 
size wise, so it is not worth it even if the header overhead was zero.  
In that case it is preferable to do without compression entirely.

> The adler32 checksumming may sound unnecessary when you already have the 
> SHA1 checksum, but the thing is, we normally don't actually *check* the 
> SHA1 except when doing a full fsck. So I actually like the fact that 
> object unpacking always checks at least the adler32 checksum at each 
> stage, which you get "for free" when you use zlib.

We still can perform adler32 on undeflated objects directly though.  But 
they need no be stored in the pack.  I'd store the adler32 checksum for 
each object in the pack index as it can be recomputed by index-pack 
(which will do the full SHA1 validation anyway).


Nicolas