Re: How to build a big file server

[Carl-Daniel Hailfinger <c-d.hailfinger.kernel.2003@gmx.net>]

Heinz-Josef Claes wrote:
From the debian web page:
>
> http://packages.debian.org/testing/utils/storebackup.html
>
> File comparisons are done with MD5 checksums, so no changes go
> unnoticed.

If you believe the last sentence, I have a bridge to sell.

To be more exact: MD5 is a 128=2^7 bit hash. Assuming a file length of 4kB
= 2^8*4096=2^20 bits, approximately 2^(2^(20-7))= 2^8192= 10^2457
different files have the same hash.

That's right: for a given MD5 hash, there are more different files with
4kB size sharing the same hash than the count of atoms in the whole
universe. If the files are larger, it gets worse.

md5sum(1) is not diff(1). Most of the time, it will suffice as el cheapo
replacement, but for backups it's definitely horrible. You don't store
your backup tapes in the microwave, do you?


Carl-Daniel

-- 
http://www.hailfinger.org/