Re: Huge win, compressing a window of delta runs as a unit

[Nicolas Pitre <nico@cam.org>]

On Thu, 17 Aug 2006, Jon Smirl wrote:

> On 8/17/06, Nicolas Pitre <nico@cam.org> wrote:
> > We're streaving for optimal data storage here so don't be afraid to use
> > one of the available types for an "object stream" object.  Because when
> > you think of it, the deflating of multiple objects into a single zlib
> > stream can be applied to all object types not only deltas.  If ever
> > deflating many blobs into one zlib stream is dimmed worth it then the
> > encoding will already be ready for it.  Also you can leverage existing
> > code to write headers, etc.
> 
> Here are two more case that need to be accounted for in the packs.
> 
> 1) If you zip something and it gets bigger. We need an entry that just
> stores the object without it being zipped. Zipping jpegs or mpegs will
> likely make them significantly bigger. Or does zlib like already
> detect this case and do a null compression?

zlib appears to do the right thing here.

> 2) If you delta something and the delta is bigger than the object
> being deltaed. The delta code should detect this and store the full
> object instead of the delta. Again jpegs and mpegs will trigger this.
> You may even want to say that the delta has to be smaller than 80% of
> the full object.

I'd invite you to read te try_delta() code in 
[builtin-]builtin-pack-objects.c.  You might find that we're already way 
more agressive than that with rather simple heuristics that have been 
tweaked and tuned for over a year now.  The commit logs for that file 
(before the rename to builtin-pack-objects.c) is also quite 
enlightening.


Nicolas