From: Andrew Mahone <andrew.mahone@gmail.com>
To: linux-btrfs@vger.kernel.org
Cc: Andrew Mahone <andrew.mahone@gmail.com>
Subject: [PATCH 0/5] btrfs: lz4/lz4hc compression
Date: Sat, 23 Jun 2012 04:05:43 -0400 [thread overview]
Message-ID: <1340438748-348-1-git-send-email-andrew.mahone@gmail.com> (raw)
WARNING: This is not compatible with the previous lz4 patchset. If you're using
experimental compression that isn't in mainline kernels, be prepared to backup
and restore or decompress before upgrading, and have backups in case it eats
data (which appears not to be a problem any more, but has been during
development).
These patches add lz4 and lz4hc compression (https://code.google.com/p/lz4/) to
btrfs. Compression with lz4 is about the same in throughput and compression
ratio as lzo, but decompression is a good deal faster. In practice, this
implementation also compresses faster and better than lzo because the existing
lzo implementation breaks extents into pages that are compressed individually.
The lz4hc compressor is a slower, higher-compression variation that is
doesn't quite match zlib on compression throughput or ratio, but is much faster
at decompression.
Some quick benchmarks, writing and reading a tarball of the Silesia corpus
(http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia) in a VM on a
tmpfs-backed device. Times are in milliseconds.
method write read size
none 489 356 202.27MB
lzo 1618 632 115.62MB
zlib 6583 1370 76.07MB
lz4 1216 368 102.15MB
lz4hc 8372 337 81.38MB
And with linux kernel sources (data size is btrfs fi df data size, may be
underestimated due to inline extents).
method write read size
none 8255 9880 446.11MB
lzo 11867 9011 225.36MB
zlib 20340 10419 154.22MB
lz4 10642 8417 203.70MB
lz4hc 20270 8280 171.68MB
A large portion of the work was done by Dave Sterba, but in flattening the
commits attribution has been lost. My contribution has been in debugging some
of the cases in which it could crash or produce incorrect data, and cleaning
it up for the ML.
Andrew Mahone (5):
btrfs: lz4: import lz4/lz4hc C and header files
btrfs: lz4: add incompat flags and compression types
btrfs: lz4: add lz4_wrapper implementing btrfs compression interface
btrfs: lz4: add lz4 files in Makefile
btrfs: enable lz4/lz4hc compression
fs/btrfs/Makefile | 2 +-
fs/btrfs/compression.c | 2 +
fs/btrfs/compression.h | 2 +
fs/btrfs/ctree.h | 16 +-
fs/btrfs/disk-io.c | 4 +
fs/btrfs/ioctl.c | 5 +
fs/btrfs/lz4.c | 843 +++++++++++++++++++++++++++++++++++++++++++++++++
fs/btrfs/lz4.h | 128 ++++++++
fs/btrfs/lz4_wrapper.c | 419 ++++++++++++++++++++++++
fs/btrfs/lz4hc.c | 685 ++++++++++++++++++++++++++++++++++++++++
fs/btrfs/lz4hc.h | 58 ++++
fs/btrfs/super.c | 16 +-
12 files changed, 2174 insertions(+), 6 deletions(-)
create mode 100644 fs/btrfs/lz4.c
create mode 100644 fs/btrfs/lz4.h
create mode 100644 fs/btrfs/lz4_wrapper.c
create mode 100644 fs/btrfs/lz4hc.c
create mode 100644 fs/btrfs/lz4hc.h
--
1.7.11
next reply other threads:[~2012-06-23 8:07 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-06-23 8:05 Andrew Mahone [this message]
2012-06-23 8:05 ` [PATCH 1/5] btrfs: lz4: import lz4/lz4hc C and header files Andrew Mahone
2012-06-23 8:05 ` [PATCH 2/5] btrfs: lz4: add incompat flags and compression types Andrew Mahone
2012-06-23 20:21 ` Andrew Mahone
2012-06-23 8:05 ` [PATCH 3/5] btrfs: lz4: add lz4_wrapper implementing btrfs compression interface Andrew Mahone
2012-06-23 8:05 ` [PATCH 4/5] btrfs: lz4: add lz4 files in Makefile Andrew Mahone
2012-06-23 8:05 ` [PATCH 5/5] btrfs: enable lz4/lz4hc compression Andrew Mahone
2012-06-25 13:19 ` [PATCH 0/5] btrfs: " David Sterba
-- strict thread matches above, loose matches on Subject: below --
2012-07-07 21:06 Jan Killius
2012-07-17 11:57 ` David Sterba
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1340438748-348-1-git-send-email-andrew.mahone@gmail.com \
--to=andrew.mahone@gmail.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).