Re: [RFC PATCH v8 00/14] Online(inband) data deduplication

[Konstantinos Skarlatos <k.skarlatos@gmail.com>]

Hello, I am trying to test your patches and they do not apply to latest 
3.12 source or 3.13 git. Am I doing something wrong?

---logs for 3.12---

Hunk #1 succeeded at 59 with fuzz 2 (offset 1 line).
patching file init/Kconfig
Hunk #1 succeeded at 1085 (offset 96 lines).
Hunk #2 succeeded at 1096 (offset 96 lines).
patching file fs/btrfs/ctree.h
Hunk #1 FAILED at 3692.
1 out of 1 hunk FAILED -- saving rejects to file fs/btrfs/ctree.h.rej
patching file fs/btrfs/extent-tree.c
Hunk #1 FAILED at 5996.
Hunk #2 FAILED at 6023.
2 out of 2 hunks FAILED -- saving rejects to file fs/btrfs/extent-tree.c.rej
patching file fs/btrfs/file-item.c
Hunk #1 FAILED at 887.
Hunk #2 succeeded at 765 with fuzz 2 (offset -151 lines).
Hunk #3 FAILED at 978.
Hunk #4 FAILED at 1061.
Hunk #5 FAILED at 1094.
4 out of 5 hunks FAILED -- saving rejects to file fs/btrfs/file-item.c.rej
patching file fs/btrfs/inode.c
Hunk #1 FAILED at 969.
Hunk #2 FAILED at 2364.
2 out of 2 hunks FAILED -- saving rejects to file fs/btrfs/inode.c.rej

---logs for 3.13---
Hunk #1 succeeded at 59 with fuzz 2 (offset 1 line).
patching file init/Kconfig
Hunk #1 succeeded at 1078 (offset 89 lines).
Hunk #2 succeeded at 1089 (offset 89 lines).
patching file fs/btrfs/ctree.h
Hunk #1 FAILED at 3692.
1 out of 1 hunk FAILED -- saving rejects to file fs/btrfs/ctree.h.rej
patching file fs/btrfs/extent-tree.c
Hunk #1 FAILED at 5996.
Hunk #2 FAILED at 6023.
2 out of 2 hunks FAILED -- saving rejects to file fs/btrfs/extent-tree.c.rej
patching file fs/btrfs/file-item.c
Hunk #1 FAILED at 887.
Hunk #2 succeeded at 768 with fuzz 2 (offset -148 lines).
Hunk #3 FAILED at 978.
Hunk #4 FAILED at 1061.
Hunk #5 FAILED at 1094.
4 out of 5 hunks FAILED -- saving rejects to file fs/btrfs/file-item.c.rej
patching file fs/btrfs/inode.c
Hunk #1 FAILED at 969.
Hunk #2 FAILED at 2364.
2 out of 2 hunks FAILED -- saving rejects to file fs/btrfs/inode.c.rej


On 30/12/2013 10:12 πμ, Liu Bo wrote:
> Hello,
>
> Here is the New Year patch bomb :-)
>
> Data deduplication is a specialized data compression technique for eliminating
> duplicate copies of repeating data.[1]
>
> This patch set is also related to "Content based storage" in project ideas[2],
> it introduces inband data deduplication for btrfs and dedup/dedupe is for short.
>
> PATCH 1 is a hang fix with deduplication on, but it's also useful without
> dedup in practice use.
>
> PATCH 2 and 3 are targetting delayed refs' scalability problems, which are
> uncovered by the dedup feature.
>
> PATCH 4 is a speed-up improvement, which is about dedup and quota.
>
> PATCH 5-8 is the preparation work for dedup implementation.
>
> PATCH 9 shows how we implement dedup feature.
>
> PATCH 10 fixes a backref walking bug with dedup.
>
> PATCH 11 fixes a free space bug of dedup extents on error handling.
>
> PATCH 12 adds the ioctl to control dedup feature.
>
> PATCH 13 fixes the metadata ENOSPC problem with dedup which has been there
> WAY TOO LONG.
>
> PATCH 14 fixes a race bug on dedup writes.
>
> And there is also a btrfs-progs patch(PATCH 15) which offers all details about
> how to control the dedup feature.
>
> I've tested this with xfstests by adding a inline dedup 'enable & on' in xfstests'
> mount and scratch_mount.
>
> TODO:
> * a bit-to-bit comparison callback.
>
> All comments are welcome!
>
>
> [1]: http://en.wikipedia.org/wiki/Data_deduplication
> [2]: https://btrfs.wiki.kernel.org/index.php/Project_ideas#Content_based_storage
>
> v8:
> - fix the race crash of dedup ref again.
> - fix the metadata ENOSPC problem with dedup.
>
> v7:
> - rebase onto the lastest btrfs
> - break a big patch into smaller ones to make reviewers happy.
> - kill mount options of dedup and use ioctl method instead.
> - fix two crash due to the special dedup ref
>
> For former patch sets:
> v6: http://thread.gmane.org/gmane.comp.file-systems.btrfs/27512
> v5: http://thread.gmane.org/gmane.comp.file-systems.btrfs/27257
> v4: http://thread.gmane.org/gmane.comp.file-systems.btrfs/25751
> v3: http://comments.gmane.org/gmane.comp.file-systems.btrfs/25433
> v2: http://comments.gmane.org/gmane.comp.file-systems.btrfs/24959
>
> Liu Bo (14):
>    Btrfs: skip merge part for delayed data refs
>    Btrfs: improve the delayed refs process in rm case
>    Btrfs: introduce a head ref rbtree
>    Btrfs: disable qgroups accounting when quata_enable is 0
>    Btrfs: introduce dedup tree and relatives
>    Btrfs: introduce dedup tree operations
>    Btrfs: introduce dedup state
>    Btrfs: make ordered extent aware of dedup
>    Btrfs: online(inband) data dedup
>    Btrfs: skip dedup reference during backref walking
>    Btrfs: don't return space for dedup extent
>    Btrfs: add ioctl of dedup control
>    Btrfs: fix dedupe 'ENOSPC' problem
>    Btrfs: fix a crash of dedup ref
>
>   fs/btrfs/backref.c           |   9 +
>   fs/btrfs/ctree.c             |   2 +-
>   fs/btrfs/ctree.h             |  86 ++++++
>   fs/btrfs/delayed-ref.c       | 161 +++++++----
>   fs/btrfs/delayed-ref.h       |   8 +
>   fs/btrfs/disk-io.c           |  40 +++
>   fs/btrfs/extent-tree.c       | 208 ++++++++++++--
>   fs/btrfs/extent_io.c         |  22 +-
>   fs/btrfs/extent_io.h         |  16 ++
>   fs/btrfs/file-item.c         | 244 +++++++++++++++++
>   fs/btrfs/inode.c             | 635 ++++++++++++++++++++++++++++++++++++++-----
>   fs/btrfs/ioctl.c             | 167 ++++++++++++
>   fs/btrfs/ordered-data.c      |  38 ++-
>   fs/btrfs/ordered-data.h      |  13 +-
>   fs/btrfs/qgroup.c            |   3 +
>   fs/btrfs/relocation.c        |   3 +
>   fs/btrfs/transaction.c       |   4 +-
>   include/trace/events/btrfs.h |   3 +-
>   include/uapi/linux/btrfs.h   |  11 +
>   19 files changed, 1501 insertions(+), 172 deletions(-)
>