Re: dear developers, can we have notdatacow + checksumming, plz?

[Christoph Anton Mitterer <calestyo@scientia.net>]

[-- Attachment #1: Type: text/plain, Size: 3002 bytes --]

On Mon, 2015-12-14 at 17:42 +1100, Russell Coker wrote:
> My understanding of BTRFS is that the metadata referencing data
> blocks has the 
> checksums for those blocks, then the blocks which link to that
> metadata (EG 
> directory entries referencing file metadata) has checksums of those.
You mean basically, that all metadata is chained, right?

> For each 
> metadata block there is a new version that is eventually linked from
> a new 
> version of the tree root.
> 
> This means that the regular checksum mechanisms can't work with nocow
> data.  A 
> filesystem can have checksums just pointing to data blocks but you
> need to 
> cater for the case where a corrupt metadata block points to an old
> version of 
> a data block and matching checksum.  The way that BTRFS works with an
> entire 
> checksumed tree means that there's no possibility of pointing to an
> old 
> version of a data block.
Hmm I'm not sure whether I understand that (or better said, I'm
probably sure I don't :D).

AFAIU, the metadata is always CoWed, right? So when a nodatacow file is
written, I'd assume it's mtime was update, which already leads to
CoWing of metadata... just that now, the checksums should be written as
well.

If the metadata block is corrupt, then should that be noticed via the
csums on that?

And you said "The way that BTRFS works with an entire checksumed tree
means that there's no possibility of pointing to an old version of a
data block."... how would that work for nodatacow'ed blocks? If there
is a crash it cannot know whether it was still the old block or the new
one or any garbage in between?!


> The NetApp published research into hard drive errors indicates that
> they are 
> usually in small numbers and located in small areas of the disk.  So
> if BTRFS 
> had a nocow file with any storage method other than dup you would
> have metadata 
> and file data far enough apart that they are not likely to be hit by
> the same 
> corruption (and the same thing would apply with most Ext4 Inode
> tables and 
> data blocks).
Well put aside any such research (whose results aren't guaranteed to be
always the case)... but that's just one reason from my motivation why
I've said checksums for no-CoWed files would be great (I used the
multi-device example though, not DUP).


> I think that a file mode where there were checksums on data 
> blocks with no checksums on the metadata tree would be useful.  But
> it would 
> require a moderate amount of coding
Do you mean in general, or having this as a mode for nodatacow'ed
files?
Loosing the meta data checksumming, doesn't seem really much more
appealing than not having data checksumming :-(


> and there's lots of other things that the 
> developers are working on.
Sure, I just wanted to bring this to their attending... I already
imagined that they wouldn't drop their current work to do that, just
because me whining for it ;-)


Thanks,
Chris.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5313 bytes --]