From: Liu Bo <bo.li.liu@oracle.com>
To: Aurelien Jarno <aurelien@aurel32.net>
Cc: linux-btrfs@vger.kernel.org
Subject: Re: [RFC PATCH v7 00/13] Online(inband) data deduplication
Date: Wed, 23 Oct 2013 10:26:17 +0800 [thread overview]
Message-ID: <20131023022616.GB22893@localhost.localdomain> (raw)
In-Reply-To: <20131022185524.GA30512@ohm.rr44.fr>
On Tue, Oct 22, 2013 at 08:55:24PM +0200, Aurelien Jarno wrote:
> Hi,
>
> On Mon, Oct 14, 2013 at 12:59:42PM +0800, Liu Bo wrote:
> > Data deduplication is a specialized data compression technique for eliminating
> > duplicate copies of repeating data.[1]
> >
> > This patch set is also related to "Content based storage" in project ideas[2].
> >
> > PATCH 1 is a hang fix with deduplication on, but it's also useful without
> > dedup in practice use.
> >
> > PATCH 2 and 3 are targetting delayed refs' scalability problems, which are
> > uncovered by the dedup feature.
> >
> > PATCH 4 is a speed-up improvement, which is about dedup and quota.
> >
> > PATCH 5-8 is the preparation for dedup implementation.
> >
> > PATCH 9 shows how we implement dedup feature.
> >
> > PATCH 10 fixes a backref walking bug with dedup.
> >
> > PATCH 11 fixes a free space bug of dedup extents on error handling.
> >
> > PATCH 12 fixes a race bug on dedup writes.
> >
> > PATCH 13 adds the ioctl to control dedup feature.
> >
> > And there is also a btrfs-progs patch(PATCH 14) which involves all details of
> > how to control dedup feature.
> >
> > I've tested this with xfstests by adding a inline dedup 'enable & on' in xfstests'
> > mount and scratch_mount.
> >
> > TODO:
> > * a bit-to-bit comparison callback.
> >
> > All comments are welcome!
> >
>
> Thanks for this new patchset. I have tested it on top of kernel 3.12-rc6
> and it worked correctly, although I haven't used it on production
> servers given the bit-to-bit comparison callback isn't implemented yet.
Many thanks for testing this!
It's not yet proper for production server use until we solve the
metadata reservation problems(I'm working on it right now).
>
> I have a few comments on the ioctl to control the dedup feature.
> Basically it is used to enable the deduplication, to switch it on or off
> and to select the blocksize. Couldn't it be implemented as a mount
> option instead like for the other btrfs features? The dedup tree would
> be created the first time the mount option is activated, and the on/off
> would be controlled by the presence of the dedup mount flag. The
> blocksize could be specified with the value appended to the dedup
> option, for example dedup=8192.
In the previous version patch set, actually I chose to use mount options
to provide a flexible control of dedup, but as thread[1] shows, David
thinked that mount options is too heavy to use as it cannot be removed
once it's merged.
[1]: http://www.spinics.net/lists/linux-btrfs/msg27294.html
-liubo
next prev parent reply other threads:[~2013-10-23 2:26 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-10-14 4:59 [RFC PATCH v7 00/13] Online(inband) data deduplication Liu Bo
2013-10-14 4:59 ` [PATCH v7 01/13] Btrfs: skip merge part for delayed data refs Liu Bo
2013-10-14 4:59 ` [PATCH v7 02/13] Btrfs: improve the delayed refs process in rm case Liu Bo
2013-10-22 20:58 ` Josef Bacik
2013-10-14 4:59 ` [PATCH v7 03/13] Btrfs: introduce a head ref rbtree Liu Bo
2013-10-14 4:59 ` [PATCH v7 04/13] Btrfs: disable qgroups accounting when quata_enable is 0 Liu Bo
2013-12-03 17:13 ` Alex Lyakas
2013-12-03 18:58 ` Alex Lyakas
2013-12-13 5:42 ` Liu Bo
2013-12-13 6:04 ` Liu Bo
2013-10-14 4:59 ` [PATCH v7 05/13] Btrfs: introduce dedup tree and relatives Liu Bo
2013-10-14 4:59 ` [PATCH v7 06/13] Btrfs: introduce dedup tree operations Liu Bo
2013-10-14 4:59 ` [PATCH v7 07/13] Btrfs: introduce dedup state Liu Bo
2013-10-14 4:59 ` [PATCH v7 08/13] Btrfs: make ordered extent aware of dedup Liu Bo
2013-10-14 4:59 ` [PATCH v7 09/13] Btrfs: online(inband) data dedup Liu Bo
2013-10-14 4:59 ` [PATCH v7 10/13] Btrfs: skip dedup reference during backref walking Liu Bo
2013-10-14 4:59 ` [PATCH v7 11/13] Btrfs: don't return space for dedup extent Liu Bo
2013-10-14 4:59 ` [PATCH v7 12/13] Btrfs: fix a crash of dedup ref Liu Bo
2013-10-14 4:59 ` [PATCH v7 13/13] Btrfs: add ioctl of dedup control Liu Bo
2013-10-14 4:59 ` [PATCH v3] Btrfs-progs: add dedup subcommand Liu Bo
2013-10-22 18:55 ` [RFC PATCH v7 00/13] Online(inband) data deduplication Aurelien Jarno
2013-10-23 2:26 ` Liu Bo [this message]
2013-10-23 2:36 ` Liu Bo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20131023022616.GB22893@localhost.localdomain \
--to=bo.li.liu@oracle.com \
--cc=aurelien@aurel32.net \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).