From: Liu Bo <bo.li.liu@oracle.com>
To: Aurelien Jarno <aurelien@aurel32.net>
Cc: linux-btrfs@vger.kernel.org, David Sterba <dsterba@suse.cz>
Subject: Re: [RFC PATCH v7 00/13] Online(inband) data deduplication
Date: Wed, 23 Oct 2013 10:36:44 +0800 [thread overview]
Message-ID: <20131023023643.GC22893@localhost.localdomain> (raw)
In-Reply-To: <20131023022616.GB22893@localhost.localdomain>
(Cced: David)
On Wed, Oct 23, 2013 at 10:26:17AM +0800, Liu Bo wrote:
> On Tue, Oct 22, 2013 at 08:55:24PM +0200, Aurelien Jarno wrote:
> > Hi,
> >
> > On Mon, Oct 14, 2013 at 12:59:42PM +0800, Liu Bo wrote:
> > > Data deduplication is a specialized data compression technique for eliminating
> > > duplicate copies of repeating data.[1]
> > >
> > > This patch set is also related to "Content based storage" in project ideas[2].
> > >
> > > PATCH 1 is a hang fix with deduplication on, but it's also useful without
> > > dedup in practice use.
> > >
> > > PATCH 2 and 3 are targetting delayed refs' scalability problems, which are
> > > uncovered by the dedup feature.
> > >
> > > PATCH 4 is a speed-up improvement, which is about dedup and quota.
> > >
> > > PATCH 5-8 is the preparation for dedup implementation.
> > >
> > > PATCH 9 shows how we implement dedup feature.
> > >
> > > PATCH 10 fixes a backref walking bug with dedup.
> > >
> > > PATCH 11 fixes a free space bug of dedup extents on error handling.
> > >
> > > PATCH 12 fixes a race bug on dedup writes.
> > >
> > > PATCH 13 adds the ioctl to control dedup feature.
> > >
> > > And there is also a btrfs-progs patch(PATCH 14) which involves all details of
> > > how to control dedup feature.
> > >
> > > I've tested this with xfstests by adding a inline dedup 'enable & on' in xfstests'
> > > mount and scratch_mount.
> > >
> > > TODO:
> > > * a bit-to-bit comparison callback.
> > >
> > > All comments are welcome!
> > >
> >
> > Thanks for this new patchset. I have tested it on top of kernel 3.12-rc6
> > and it worked correctly, although I haven't used it on production
> > servers given the bit-to-bit comparison callback isn't implemented yet.
>
> Many thanks for testing this!
>
> It's not yet proper for production server use until we solve the
> metadata reservation problems(I'm working on it right now).
>
> >
> > I have a few comments on the ioctl to control the dedup feature.
> > Basically it is used to enable the deduplication, to switch it on or off
> > and to select the blocksize. Couldn't it be implemented as a mount
> > option instead like for the other btrfs features? The dedup tree would
> > be created the first time the mount option is activated, and the on/off
> > would be controlled by the presence of the dedup mount flag. The
> > blocksize could be specified with the value appended to the dedup
> > option, for example dedup=8192.
>
> In the previous version patch set, actually I chose to use mount options
> to provide a flexible control of dedup, but as thread[1] shows, David
> thinked that mount options is too heavy to use as it cannot be removed
> once it's merged.
>
> [1]: http://www.spinics.net/lists/linux-btrfs/msg27294.html
>
> -liubo
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
-liubo
prev parent reply other threads:[~2013-10-23 2:37 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-10-14 4:59 [RFC PATCH v7 00/13] Online(inband) data deduplication Liu Bo
2013-10-14 4:59 ` [PATCH v7 01/13] Btrfs: skip merge part for delayed data refs Liu Bo
2013-10-14 4:59 ` [PATCH v7 02/13] Btrfs: improve the delayed refs process in rm case Liu Bo
2013-10-22 20:58 ` Josef Bacik
2013-10-14 4:59 ` [PATCH v7 03/13] Btrfs: introduce a head ref rbtree Liu Bo
2013-10-14 4:59 ` [PATCH v7 04/13] Btrfs: disable qgroups accounting when quata_enable is 0 Liu Bo
2013-12-03 17:13 ` Alex Lyakas
2013-12-03 18:58 ` Alex Lyakas
2013-12-13 5:42 ` Liu Bo
2013-12-13 6:04 ` Liu Bo
2013-10-14 4:59 ` [PATCH v7 05/13] Btrfs: introduce dedup tree and relatives Liu Bo
2013-10-14 4:59 ` [PATCH v7 06/13] Btrfs: introduce dedup tree operations Liu Bo
2013-10-14 4:59 ` [PATCH v7 07/13] Btrfs: introduce dedup state Liu Bo
2013-10-14 4:59 ` [PATCH v7 08/13] Btrfs: make ordered extent aware of dedup Liu Bo
2013-10-14 4:59 ` [PATCH v7 09/13] Btrfs: online(inband) data dedup Liu Bo
2013-10-14 4:59 ` [PATCH v7 10/13] Btrfs: skip dedup reference during backref walking Liu Bo
2013-10-14 4:59 ` [PATCH v7 11/13] Btrfs: don't return space for dedup extent Liu Bo
2013-10-14 4:59 ` [PATCH v7 12/13] Btrfs: fix a crash of dedup ref Liu Bo
2013-10-14 4:59 ` [PATCH v7 13/13] Btrfs: add ioctl of dedup control Liu Bo
2013-10-14 4:59 ` [PATCH v3] Btrfs-progs: add dedup subcommand Liu Bo
2013-10-22 18:55 ` [RFC PATCH v7 00/13] Online(inband) data deduplication Aurelien Jarno
2013-10-23 2:26 ` Liu Bo
2013-10-23 2:36 ` Liu Bo [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20131023023643.GC22893@localhost.localdomain \
--to=bo.li.liu@oracle.com \
--cc=aurelien@aurel32.net \
--cc=dsterba@suse.cz \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).