Re: worse than expected compression ratios with -o compress

Josef Bacik
2010-01-18T14:12:40Z

On Sat, Jan 16, 2010 at 11:16:50AM -0500, Jim Faulkner wrote:
>
> I have a mysql database which consists of hundreds of millions, if not
> billions of Usenet newsgroup headers. This data should be highly
> compressable, so I put the mysql data directory on a btrfs filesystem
> mounted with the compress option:
> /dev/sdi on /var/news/mysql type btrfs (rw,noatime,compress,noacl)
>
> However, I'm not seeing the kind of compression ratios that I would
> expect with this type of data. FYI, all my tests are using Linux
> 2.6.32.3. Here's my current disk usage:
> Filesystem Size Used Avail Use% Mounted on
> /dev/sdi 302G 122G 181G 41% /var/news/mysql
>
> and here's the actual size of all files:
> delta-9 mysql # pwd
> /var/news/mysql
> delta-9 mysql # du -h --max-depth=1
> 747K ./mysql
> 0 ./test
> 125G ./urd
> 125G .
> delta-9 mysql #
>
> As you can see, I am only shaving off 3 gigs out of 125 gigs worth of
> what should be very compressable data. The compressed data ends up being
> around 98% the size of the original data.
>
> To contrast, rzip can compress a database dump of this data to around 7%
> of its original size. This is an older database dump, which is why it is
> smaller. Before:
> -rw------- 1 root root 69G 2010-01-15 14:55 mysqlurdbackup.2010-01-15
> and after:
> -rw------- 1 root root 5.2G 2010-01-16 05:34 mysqlurdbackup.2010-01-15.rz
>
> Of course it took 15 hours to compress the data, and btrfs wouldn't be
> able to use rzip for compression anyway.
>
> However, I still would expect to see better compression ratios than 98%
> on such data. Are there plans to implement a better compression
> algorithm? Alternatively, is there a way to tune btrfs compression to
> achieve better ratios?
>

Currently the only compression algorithm we support is gzip, so try gzipp'ing
your database to get a better comparison. The plan is to eventually support
other compression algorithms, but currently we do not. Thanks,

Josef