Re: Data Deduplication with the help of an online filesystem check

Michael Tharp
2009-04-29T13:11:36Z

Thomas Glanzmann wrote:
> Looking at this picture, when I'm going to implement the dedup code, do I also
> have to take care to spread the blocks over the different devices or is
> there already infrastructure in place that automates that process?

If you somehow had blocks duplicated exactly across two devices such
that the deduplicator discarded all the blocks from one disk and kept
all the blocks from the other disk, then there would be a problem. The
best way to solve this would be to always keep the block on the
least-full device and discard the rest, which shouldn't be too difficult
(especially since this would be happening in kernelspace), but the
cheapest solution is to randomize the choice which would be sufficient
to prevent any further imbalance from developing.

-- m. tharp