Re: Incremental backup for a raid1

[Hugo Mills <hugo@carfax.org.uk>]

[-- Attachment #1: Type: text/plain, Size: 3091 bytes --]

On Sat, Mar 15, 2014 at 12:35:30PM +0100, Michael Schuerig wrote:
> On Thursday 13 March 2014 17:29:11 George Mitchell wrote:
> > I currently use rsync to a separate drive to maintain a 
> > backup copy, but it is not integrated into the array like n-way would 
> > be, and is definitely not a perfect solution.
> 
> Could you explain how you're using rsync? I was just about to copy a 
> btrfs filesystem to another disk. That filesystem has several subvolumes 
> and about 100 snapshots overall. Owing to COW, this amounts to about 
> 1.2TB. However, I reckon that rsync doesn't know anything about COW and 
> accordingly would blow up my data immensely on the destination disk.
> 
> How do I copy a btrfs filesystem preserving its complete contents? How 
> do I update such a copy?
> 
> Yes, I want to keep the subvolume layout of the original and I want to 
> copy all snapshots. I don't think send/receive is the answer, but it's 
> likey I don't understand it well enough. I'm concerned, that a 
> send/receive-based approach is not robust against mishaps.

   send/receive is the answer, but it's going to be a bit more
complicated to manage *all* of the snapshots. (Questions -- do you
actually need them all backed up? Can you instead do incremental
backups of the "main" subvol and keep each of those independently on
the backup machine instead?)

> Consider: I want to incrementally back-up a filesystem to two external 
> disks. For this I'd have to for each subvolume keep a snapshot 
> corresponding to its state on the backup disk. If I make any mistake in 
> managing these snapshots, I can't update the external backup anymore.

   Correct (I got bitten by this last week with my fledgling backup
process). You need a place that stores the "current state" subvolumes
that's not going to be touched by anything else, and you can't clean
up any given base until you're certain that there's a good new one
available on both sides. One thing that helps here is that send
requires the snapshot being sent to be marked read-only, so it's not
possible to change it at all -- but you can delete them.

> Also, I don't understand whether send/receive would allow me to 
> copy/update a subvolume *including* its snapshots.

   Snapshots aren't owned by subvolumes. Once you've made a snapshot,
that snapshot is a fully equal partner of the subvol that it was a
snapshot of -- there is no hierarchy of ownership. This means that you
will have to send each snapshot independently.

   What send allows you to do is to specify that one or more
subvolumes on the send side can be assumed to exist on the receive
side (via -p and -c). If you do that, the stream can then use them as
clone sources (i.e. should make shared CoW copies from them, rather
than sending all the data).

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
          --- ...  one ping(1) to rule them all, and in the ---          
                         darkness bind(2) them.                          

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]