[RFC PATCH v3 0/2] Online data deduplication

[Liu Bo <bo.li.liu@oracle.com>]

NOTE: This leads to a FORMAT CHANGE, DO NOT use it on real data!

Data deduplication is a specialized data compression technique for eliminating
duplicate copies of repeating data.[1]

This patch set is also related to "Content based storage" in project ideas[2].

PATCH 1 is a hang fix when deduplication is on, but it's also useful with no
deduplication in practice use.

For more implementation details, please refer to PATCH 2.

TODO:
* a bit-to-bit comparison callback.

All comments are welcome!

[1]: http://en.wikipedia.org/wiki/Data_deduplication
[2]: https://btrfs.wiki.kernel.org/index.php/Project_ideas#Content_based_storage


v3:
  * add COMPRESS support
  * add a real ioctl to enable dedup feature
  * change the maximum allowed dedup blocksize to 128k because of compressed
    range limit
v2:
  * To avoid enlarging the file extent item's size, add another index key used
    for freeing dedup extent.
  * Freeing dedup extent is now like how we delete checksum.
  * Add support for alternative deduplicatin blocksize larger than PAGESIZE.
  * Add a mount option to set deduplication blocksize.
  * Add support for those writes that are smaller than deduplication blocksize.

=====================
HOW To turn deduplication on:

There are 2 steps you need to do before using it,
1) mount /dev/disk /mnt_of_your_btrfs -o dedup
   (or mount /dev/disk /mnt_of_your_btrfs -o dedup_bs=128K)
2) btrfs filesystem dedup-register /mnt_of_your_btrfs
=====================

Liu Bo (2):
  Btrfs: skip merge part for delayed data refs
  Btrfs: online data deduplication

 fs/btrfs/ctree.h           |   54 ++++
 fs/btrfs/delayed-ref.c     |    7 +
 fs/btrfs/disk-io.c         |   34 +++-
 fs/btrfs/extent-tree.c     |    7 +
 fs/btrfs/extent_io.c       |   27 ++-
 fs/btrfs/extent_io.h       |   15 ++
 fs/btrfs/file-item.c       |  242 ++++++++++++++++++
 fs/btrfs/inode.c           |  583 ++++++++++++++++++++++++++++++++++++++------
 fs/btrfs/ioctl.c           |   38 +++
 fs/btrfs/ordered-data.c    |   30 ++-
 fs/btrfs/ordered-data.h    |   11 +-
 fs/btrfs/super.c           |   27 ++-
 include/uapi/linux/btrfs.h |    1 +
 13 files changed, 990 insertions(+), 86 deletions(-)

-- 
1.7.7