Re: is back and forth incremental send/receive supported/stable?

[Hugo Mills <hugo@carfax.org.uk>]

On Fri, Jan 29, 2021 at 08:09:49PM +0100, Christoph Anton Mitterer wrote:
> I regularly do the following with btrfs, which seems to work pretty
> stable since years:
> - having n+1 filesystems MASTER and COPY_n
> - creating snapshots on MASTER, e.g. one each month
> - incremental send/receive the new snapshot from MASTER to each of
>   COPY_n (which already have the previous snapshot)
> 
> 
> so for example:
> - MASTER has
>   - snapshot-2020-11/
>   - snapshot-2020-12/
>   and newly get's
>   - snapshot-2021-01/
> - each of COPY_n has only
>   - snapshot-2020-11/
>   - snapshot-2020-12(
> - with:
>   # btrfs send -p MASTER/snapshot-2020-12 MASTER/snapshot-2021-01  |  btrfs receive COPY_n/
>   I incrementally send the new snapshot from MASTER to each of COPY_n
>   using the already available previous snapshot as parent.
> 
> Works(TM)
> 
> 
> 
> Now I basically want to swap a MASTER with a COPY_n (e.g. because
> MASTER's HDD has started to age).
> 
> So the plan is e.g.:
> - COPY_1 becomes NEW_MASTER
> - MASTER becomes OLD_MASTER later known NEW_COPY_1
> 
> a) Can I then start e.g. in February to incrementally send/receive from
> NEW_MASTER back(!!) to OLD_MASTER?

   No.

   When you make an incremental send, you give it a reference
subvolume with -p. This subvol's UUID is sent in the send stream to
the remote side for receive.

   When receive gets told about a reference subvolume in this way, it
looks for the reference and snapshots it (RW) to use as the base to
apply the incremental on top of.

   The way it finds the reference subvol is to look for a subvol with
the "received_uuid" field matching. This field is set by the receiving
process that made it in the first place (as the result of an earlier
send).

   In your scenario with MASTER and COPY-1 swapped, you'd have to
match the received_uuid from the sending side (on old COPY-1) to the
actual UUID on old MASTER. The code doesn't do this, so you'd have to
patch send/receive to do this.

   Your best bet here is to do a new full send and then continue a new
incremental sequence based on that.

   There's a detailed and fairly formal description of this stuff that
I wrote a few years ago here:

https://www.spinics.net/lists/linux-btrfs/msg44089.html

   Hugo.

> Like:
> # btrfs send -p NEW_MASTER/snapshot-2021-01 NEW_MASTER/snapshot-2021-02  |  btrfs receive OLD_MASTER/
> 
> b) And the same from NEW_MSTER to all the other COPY_n?
> Like:
> # btrfs send -p NEW_MASTER/snapshot-2021-01 NEW_MASTER/snapshot-2021-02  |  btrfs receive COPY_n
> 
> 
> So in other words, does btrfs get, that the new parent (which is no
> longer on the OLD_MASTER but the previous COPY_1, now NEW_MASTER) is
> already present (and identical and usable) on the OLD_MASTER, now
> NEW_COPY_1, and also on the other COPY_n ?
> 
> 
> By the way, I'm talking about *precious* data, so I'd like to be really
> sure that this works... and whether it's intended to work and ideally
> have been tested.
> 
> 
> Thanks,
> Chris.
> 

-- 
Hugo Mills             | You shouldn't anthropomorphise computers. They
hugo@... carfax.org.uk | really don't like that.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |