public inbox for linux-btrfs@vger.kernel.org
 help / color / mirror / Atom feed
From: Daniel J Blueman <daniel.blueman@gmail.com>
To: Linux BTRFS <linux-btrfs@vger.kernel.org>,
	Linux Kernel <linux-kernel@vger.kernel.org>
Subject: file/extent checksums for dedup/sync...
Date: Wed, 27 Jan 2010 12:10:50 +0000	[thread overview]
Message-ID: <6278d2221001270410k1493582fvccdf23bed14cc0ff@mail.gmail.com> (raw)

For purposes of data deduplication and data synchronisation, it would
be a powerful tool to expose file data checksums.

Since eg BTRFS uses the crc32c algorithm [1], it's possible to compute
the file's overall CRC from the accumulation of the CRCs from all it's
extents' CRCs.

For now, exposing this via an IOCTL may be sufficient, though any
ideas for introducing it in a more standard way? (it's a pity that
when stat64 was introduced, reserved fields weren't added)

Thanks,
  Daniel

[1] http://www.research.ibm.com/haifa/satran/ips/Vince-Luben-crc32c-01.pdf
-- 
Daniel J Blueman

             reply	other threads:[~2010-01-27 12:10 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-01-27 12:10 Daniel J Blueman [this message]
2010-01-27 12:30 ` file/extent checksums for dedup/sync Andi Kleen
2010-01-27 13:23   ` Daniel J Blueman
2010-01-27 20:15     ` Chris Mason

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6278d2221001270410k1493582fvccdf23bed14cc0ff@mail.gmail.com \
    --to=daniel.blueman@gmail.com \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox