Re: [RFC] CryptoAPI & Compression

Artem B. Bityuckiy
2005-04-01T14:36:37Z

Hi Herbert,

> For the default zlib parameters (which crypto/deflate.c does not use)
> the maximum overhead is 5 bytes every 16KB plus 6 bytes. So for input
> streams less than 16KB the figure of 12 bytes is correct. However,
> in general the overhead needs to grow proportionally to the number of
> blocks in the input.

I've explored the issue a bit. According RFC-1950 and RFC-1951, zlib
stream is:

2bytes zstream header, block1, block2, ... 4bytes adler32.
Each block has one 1-byte "end-of-block" marker and a prefix of max len =
1 byte.

In our code we do zlib_deflate(stream, Z_SYNC_FLUSH), so we always flush
the output. So the final zlib_deflate(stream, Z_FINISH) requires 1 byte
for the EOB marker and 4 bytes for adler32 (5 bytes total). Thats all. If
we compress a huge buffer, then we still need to output those 5 bytes as
well. I.e, the overhead of each block *is not accumulated* ! I even need
to make the reserved space less then 12 bytes!

Anyway, don't apply that patch please, I'll send a new one.