Re: Compression Plugin

[Gregory Maxwell <gmaxwell@gmail.com>]

On 9/20/05, David Masover <ninja@slaphack.com> wrote:
> Probably lzo, which is already used for other things like network
> connections (ssh, openvpn, and so on).  The nice thing about lzo is that
> it's fast, faster than gzip or bzip2, and gets decent compression -- not
> great, but decent.  I don't usually get gzip or bzip2 to compress at
> disk speed, but then, I usually crank the compression way up, so YMMV.
> The point of using a fast algorithm is that you not only save space, but
> when you apply it to things like text files, it can actually make things
> go faster.
> 
> But I imagine it will be settable per-file.  Files can be both encrypted
> and compressed, and I think (I hope) it could be with a choice of
> crypto/compression algorithms.

I didn't know SSH supported LZO.  Rsync does though...

Gzip compression is pretty darn quick at lower levels, though
depending on the lz77 implimentaiton it can be fairly slow at higher
compression levels.

An interesting idea:  select the algo and a range of compression
levels per file, but select the actual compression level at flush time
based on some estimate of how loaded the system is.. :)
Probably not worth it even though the amount of compression and the
speed differ greatly from -1 to -9... I hope no one wastes their time
on it until the more important things are done.. but perhaps a nice
touch.