data integrity in btrfs

data integrity in btrfs

[Jaromir Zdrazil]

Hi again,

   I know that ZFS include data integrity verification against data corruption modes using propably SHA256.

   By sketchy readings at https://btrfs.wiki.kernel.org/index.html , http://en.wikipedia.org/wiki/Btrfs and other sources I have found just that there is Sha32C used and that it should be similar to ZFS.

   How are data faults detected and repaired ni BRTFS? If the answer could be simple and precize, I would be more than happy.

Thank you! Gone to lunch ;O)

Jaromir

Re: data integrity in btrfs

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1742 bytes --]

On Fri, Dec 30, 2011 at 02:29:46PM +0100, Jaromir Zdrazil wrote:
> Hi again,
> 
>    I know that ZFS include data integrity verification against data corruption modes using propably SHA256.
> 
>    By sketchy readings at https://btrfs.wiki.kernel.org/index.html , http://en.wikipedia.org/wiki/Btrfs and other sources I have found just that there is Sha32C used and that it should be similar to ZFS.
> 
>    How are data faults detected and repaired ni BRTFS? If the answer could be simple and precize, I would be more than happy.
> 
> Thank you! Gone to lunch ;O)

   Every 4k block in btrfs is checksummed (using CRC32, so it's not
cryptographically robust against malicious modification, but should
spot most random errors).

   If you use RAID-1 or RAID-10 storage, then you get two copies of
each piece of data, stored on different devices. Each copy is
independently checksummed. When data is read, the checksum is verified
as well, and a failed checksum is logged to syslog. In this case, the
filesystem will attempt to read the other copy. If both copies are
bad, an I/O error is returned; if one of the copies is good, that data
is returned.

   With recent kernels and an up-to-date userspace, there is a feature
called scrub which will read both copies of all of the data blocks in
the filesystem and compare them to each other. If there is a mismatch
with a failed checksum, scrub will rewrite the broken block to fix it.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
       --- "Are you the man who rules the Universe?" "Well,  I ---       
                              try not to."                               

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: data integrity in btrfs

[Kai Krakow]

As long as you create your data and metadata with a mirror policy, you can 
use btrfs scrubbing to find and correct broken data blocks. I think latest 
kernels also so this "repairing" online.

It works by finding a mirrored block with correct checksum if the block in 
question has a bad checksum.

Jaromir Zdrazil <jaromir.zdrazil@email.cz> schrieb:

> Hi again,
> 
>    I know that ZFS include data integrity verification against data
>    corruption modes using propably SHA256.
> 
>    By sketchy readings at https://btrfs.wiki.kernel.org/index.html ,
>    http://en.wikipedia.org/wiki/Btrfs and other sources I have found just
>    that there is Sha32C used and that it should be similar to ZFS.
> 
>    How are data faults detected and repaired ni BRTFS? If the answer could
>    be simple and precize, I would be more than happy.
> 
> Thank you! Gone to lunch ;O)
> 
> Jaromir
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html