From: Sander <sander@humilis.net>
To: Heinz-Josef Claes <hjclaes@web.de>
Cc: Chris Mason <chris.mason@oracle.com>,
Thomas Glanzmann <thomas@glanzmann.de>,
Edward Shishkin <edward.shishkin@gmail.com>,
Tomasz Chmielewski <mangoo@wpkg.org>,
linux-btrfs@vger.kernel.org
Subject: Re: Data Deduplication with the help of an online filesystem check
Date: Wed, 6 May 2009 17:16:26 +0200 [thread overview]
Message-ID: <20090506151625.GA28614@cumulus> (raw)
In-Reply-To: <200904281945.10274.hjclaes@web.de>
Heinz-Josef Claes wrote (ao):
> Am Dienstag, 28. April 2009 19:38:24 schrieb Chris Mason:
> > On Tue, 2009-04-28 at 19:34 +0200, Thomas Glanzmann wrote:
> > > Hello,
> > >
> > > > I wouldn't rely on crc32: it is not a strong hash,
> > > > Such deduplication can lead to various problems,
> > > > including security ones.
> > >
> > > sure thing, did you think of replacing crc32 with sha1 or md5, is this
> > > even possible (is there enough space reserved so that the change can be
> > > done without changing the filesystem layout) at the moment with btrfs?
> >
> > It is possible, there's room in the metadata for about about 4k of
> > checksum for each 4k of data. The initial btrfs code used sha256, but
> > the real limiting factor is the CPU time used.
> >
> > -chris
> >
> It's not only cpu time, it's also memory. You need 32 byte for each 4k block.
> It needs to be in RAM for performance reason.
Less so with SSD I would assume.
--
Humilis IT Services and Solutions
http://www.humilis.net
next prev parent reply other threads:[~2009-05-06 15:16 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-04-27 3:33 Data Deduplication with the help of an online filesystem check Thomas Glanzmann
2009-04-27 13:37 ` Chris Mason
2009-04-28 5:22 ` Thomas Glanzmann
2009-04-28 10:02 ` Chris Mason
2009-04-28 13:49 ` Andrey Kuzmin
2009-04-28 13:58 ` Chris Mason
2009-04-28 14:04 ` Thomas Glanzmann
2009-04-28 17:21 ` Chris Mason
2009-04-28 20:10 ` Thomas Glanzmann
2009-04-28 20:29 ` Thomas Glanzmann
2009-04-28 13:58 ` jim owens
2009-04-28 16:10 ` Anthony Roberts
2009-04-28 15:59 ` Thomas Glanzmann
2009-04-28 16:04 ` Tomasz Chmielewski
2009-04-28 17:29 ` Edward Shishkin
2009-04-28 17:34 ` Thomas Glanzmann
2009-04-28 17:38 ` Chris Mason
2009-04-28 17:43 ` Thomas Glanzmann
2009-04-28 17:45 ` Heinz-Josef Claes
2009-04-28 20:16 ` Thomas Glanzmann
2009-04-28 20:36 ` Heinz-Josef Claes
2009-04-28 20:52 ` Thomas Glanzmann
2009-04-28 20:58 ` Chris Mason
2009-04-28 21:12 ` Thomas Glanzmann
2009-04-28 21:26 ` Chris Mason
2009-04-28 22:14 ` Thomas Glanzmann
2009-04-28 23:18 ` Chris Mason
2009-04-29 12:03 ` Thomas Glanzmann
2009-04-29 13:11 ` Michael Tharp
2009-04-29 13:14 ` Chris Mason
2009-04-29 13:58 ` Thomas Glanzmann
2009-04-29 14:31 ` Chris Mason
2009-04-29 15:26 ` Thomas Glanzmann
2009-04-29 15:45 ` Chris Mason
2009-06-04 8:49 ` Thomas Glanzmann
2009-06-04 11:43 ` Chris Mason
2009-06-04 12:03 ` Thomas Glanzmann
2009-06-04 12:43 ` Chris Mason
2009-06-05 12:20 ` Tomasz Chmielewski
2009-06-05 12:50 ` Chris Mason
2009-06-05 15:35 ` Tomasz Chmielewski
2009-04-29 0:06 ` Bron Gondwana
2009-05-06 15:16 ` Sander [this message]
2009-04-28 17:32 ` Thomas Glanzmann
2009-04-28 17:41 ` Michael Tharp
2009-04-28 20:14 ` Thomas Glanzmann
2009-05-04 14:29 ` Ric Wheeler
2009-05-04 14:39 ` Tomasz Chmielewski
2009-05-04 14:45 ` Ric Wheeler
2009-05-04 15:15 ` Thomas Glanzmann
2009-05-04 16:03 ` Ric Wheeler
2009-05-04 16:16 ` Andrey Kuzmin
2009-05-04 16:24 ` Thomas Glanzmann
2009-05-04 18:06 ` Jan-Frode Myklebust
2009-05-04 19:16 ` Andrey Kuzmin
2009-05-05 8:02 ` Thomas Glanzmann
2009-05-04 16:26 ` Thomas Glanzmann
2009-05-04 19:11 ` Heinz-Josef Claes
2009-05-04 21:29 ` Dmitri Nikulin
2009-05-05 7:18 ` Heinz-Josef Claes
2009-05-24 7:27 ` Thomas Glanzmann
2009-04-28 17:23 ` Chris Mason
2009-04-28 17:37 ` Thomas Glanzmann
2009-04-28 17:43 ` Chris Mason
2009-04-28 20:15 ` Thomas Glanzmann
2009-04-28 21:19 ` Dmitri Nikulin
2009-04-28 20:24 ` Thomas Glanzmann
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20090506151625.GA28614@cumulus \
--to=sander@humilis.net \
--cc=chris.mason@oracle.com \
--cc=edward.shishkin@gmail.com \
--cc=hjclaes@web.de \
--cc=linux-btrfs@vger.kernel.org \
--cc=mangoo@wpkg.org \
--cc=thomas@glanzmann.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox