Re: Questions on incremental backups

[Duncan <1i5t5.duncan@cox.net>]

On Fri, 18 Jul 2014 16:55:26 +0600
Roman Mamedov <rm@romanrm.net> wrote:

> On Fri, 18 Jul 2014 10:45:37 +0000 (UTC)
> Duncan <1i5t5.duncan@cox.net> wrote:
> 
> > Russell Coker posted on Fri, 18 Jul 2014 14:35:20 +1000 as
> > excerpted:
> > 
> > > Daily snapshots work welk with kernel 3.14 and above (I had
> > > problems with 3.13 and previous). I have snapshots every 15 mins
> > > on some subvols.
> > > 
> > > Very large numbers of snapshots can cause performance problems. I
> > > suggest keeping below 1000 snapshots at this time.
> > 
> > The other caveat with btrfs snapshots is how they deal with NOCOW
> > files, the usual workaround recommended for large (Gig-ish-plus)
> > internal- rewrite-pattern files such as databases and VM images.
> 
> And "how" do they deal with them? To the best of my knowledge there
> is no "caveat" whatsoever, NOCOW and snapshots interact perfectly,
> exactly as it should be (snapshotted and then changed bits get
> COW'ed, but only once).

Yes, but the fact that NOCOW files must never-the-less be COWed anyway
on the first write to a block after a snapshot isn't exactly intuitive
to many admins, and even to many list regulars until relatively
recently.

For some time the recommendation for active large database files and
VM images (the ones I mentioned) was to make them NOCOW in ordered to
avoid extreme fragmentation, and people were still reporting extreme
fragmentation and the related performance issues even when the files
were properly NOCOWed at creation. Turned out the reason was that they
had scripted auto-snapshotting enabled, sometimes snapshotting the
files as often as once a minute!  With an active VM writing data more
or less randomly to its image equally often, NOCOW lost its
effectiveness as the snapshotting was forcing COW writes most of the
time anyway!

In the context of frequent snapshots, NOCOW is in practice broken
and doesn't do what the label would indicate, thus the caveat.

Effectively, admins can choose NOCOW XOR frequent-snapshotting, altho
the fact that snapshots stop at subvolume borders can be used as a
partial workaround, by putting NOCOW files on a dedicated partition and
not snapshotting it, exactly as I mentioned.

-- 
Duncan - No HTML messages please, as they are filtered as spam.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman