Re: Deduplication tools

[Duncan <1i5t5.duncan@cox.net>]

Marat Khalili posted on Thu, 13 Apr 2017 14:06:07 +0300 as excerpted:

> After reading this maillist for a while I became a bit more cautious
> about using various BTRFS features, so decided to ask just in case: is
> it safe to use out-of-band deduplication tools
> <https://btrfs.wiki.kernel.org/index.php/Deduplication>, and which of
> them are considered more stable/mainstream? Also, won't running these
> tools exacerbate often mentioned stability/performance problems with
> too-many-snapshots? Any first-hand experience is very welcome.

Dedupe works by reflinking, which is how snapshots work too, so yes, 
it'll trigger the same general issues.

However, while snapshots will reflink /everything/ in the snapshot, one 
could hope dedupe won't find /that/ many duplicates and thus won't reflink 
at the scale that snapshotting does.  Tho of course if it doesn't find 
at least some reasonable number of duplicates, it's hardly worth the 
trouble to run.

So I'd say it may be worth doing, but do be aware of the scale of dedupe 
reflinking.  What I'd do here is consider that and then count it as 
roughly another snapshot or two, or whatever, depending on how much 
dedupeing you actually get.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman