[Lustre-devel] Integrity and corruption - can file systems be scalable?

[Nicolas Williams <Nicolas.Williams@oracle.com>]

On Fri, Jul 02, 2010 at 11:37:52PM -0400, Dmitry Zogin wrote:
> Well, the hash trees certainly help to achieve data integrity, but
> at the performance cost.

Merkle hash trees cost more CPU cycles, not more I/O.  Indeed, they
result in _less_ I/O in the case of RAID-Zn because there's no need to
read the parity unless the checksum doesn't match.  Also, how much CPU
depends on the hash function.  And HW could help if this became enough
of a problem for us.

> Eventually, the file system becomes fragmented, and moving the data
> around implies more random seeks with Merkle hash trees.

Yes, fragmentation is a problem for COW, but that has nothing to do with
Merkle trees.  But practically every modern filesystem coalesces writes
into contiguous writes on disk to reach streaming write perfmormance,
and that, like COW, results in filesystem fragmentation.

(Of course, you needn't get fragmentation if you never delete or over
write files.  You'll get some fragmentation of meta-data, but that's
much easier to garbage collect since meta-data will amount to much less
on disk than data.)

Everything we do involves trade-offs.

Nico
--