Re: Decompression speed: zip vs lzo

["Marco Costalba" <mcostalba@gmail.com>]

On Jan 10, 2008 7:55 AM, Marco Costalba <mcostalba@gmail.com> wrote:
>
> [1] where inflate() is called:
>
> -inflate_it() in builtin-apply.c
> -check_pack_inflate() in builtin-pack-objects.c
> -get_data() in builtin-unpack-objects.c
> -fwrite_sha1_file() in http-push.c and http-walker.c  [mmm interesting
> same function in two files, also the signature and the contents seems
> the same....]
> -unpack_entry_data() in index-pack.c
> -unpack_sha1_header(), unpack_sha1_rest(), get_size_from_delta(),
> unpack_compressed_entry, write_sha1_from_fd() in sha1_file.c
>

Looking at the git sources I have found that zip routines are
candidate for a cleaning up, as example the more or less very similar
lines of code are repeated many times in git files:

memset(&stream, 0, sizeof(stream));
deflateInit(&stream, pack_compression_level);
maxsize = deflateBound(&stream, size);
out = xmalloc(maxsize);
stream.next_out = out;
stream.avail_out = maxsize;


So what I'm planning to do to test with different algorithms is first
a cleanup work that is more or less the following

- Remove #include <zlib.h> from cache.h and substitute with #include
"compress.h"

- Add #include <zlib.h> where it is "really" intended as example archive-zip.c

- Rename inflate()/deflate() and other zlib calls with corresponding
  zlib_inflate()
  zlib_deflate()

and declared in compress.h

- Define zlib_inflate() and friends as simple wrappers to
corresponding zlib function

- Test if everything is ok (should be only code shuffling/renaming until now)

- Start cleaning up as example adding a do_deflateInit() that wraps
all the code I have reported above and that involves deflateInit()

- When compression routines are cleaned up add new functions

do_inflate(), do_deflate() instead of zlib_* ones that wrap the
compression alghorithm dispatching logic.

Dispatching could be choose in different ways going from

- compile time (at #define level)
- config (some configuration value stored in some global variable)
- dynamic (at run time, with no configuration needed, I have some
ideas on this ;-)


Comments?

Thanks
Marco