btrfs send/receive for backup of subvolumes

btrfs send/receive for backup of subvolumes

[sri]

Hi

My query speicific to incremental backup/recovery with btrfs 
send/receive option available for btrfs subvolume backups.

say i have /b1/s1 subvolume and snapshot of s1 is /b1/snap1_s1

I run btrfs send /b1/snap_s1 | btrfs receive /backup where /backup is my 
backup btrfs file system.


Now I will take snapshot of s1 again ( after modifiying data in s1) 
which is /b1/snap2_s1

Using btrfs send -p snap1_s1 snap2_s2 and destination as /backup I can 
send incremental to backup store.

My question is is there a way I can have option recover files from 
/backup from incremental backup or full backup (i.e. from copy of 
snap2_s1 or snap1_s1)

And I do not want to make 2 copies of 1 full and 1 incremetal backup 
since if full backup is 1Gb and if there is 0.6Gb change for 
incremental, I would like to backup only difference for incremental 
backup but still would able to get option to recover files from full or 
incremental from backedup subvolume.

is there a way I can achieve this using current btrfs send/receive and 
other snapshot and tracking options?

thank you in advance.

Re: btrfs send/receive for backup of subvolumes

[Duncan]

sri posted on Tue, 05 May 2015 11:30:27 +0000 as excerpted:

> Hi
> 
> My query speicific to incremental backup/recovery with btrfs
> send/receive option available for btrfs subvolume backups.
> 
> say i have /b1/s1 subvolume and snapshot of s1 is /b1/snap1_s1
> 
> I run btrfs send /b1/snap_s1 | btrfs receive /backup where /backup is my
> backup btrfs file system.
> 
> 
> Now I will take snapshot of s1 again ( after modifiying data in s1)
> which is /b1/snap2_s1
> 
> Using btrfs send -p snap1_s1 snap2_s2 and destination as /backup I can
> send incremental to backup store.
> 
> My question is is there a way I can have option recover files from
> /backup from incremental backup or full backup (i.e. from copy of
> snap2_s1 or snap1_s1)
> 
> And I do not want to make 2 copies of 1 full and 1 incremetal backup
> since if full backup is 1Gb and if there is 0.6Gb change for
> incremental, I would like to backup only difference for incremental
> backup but still would able to get option to recover files from full or
> incremental from backedup subvolume.
> 
> is there a way I can achieve this using current btrfs send/receive and
> other snapshot and tracking options?

>From a later post it looks like you  probably figured this out on your 
own.  But in case you didn't and for others who may google this thread...

Disclaimer: While I'm a btrfs user and list regular, I'm not a btrfs dev, 
and my own use-case doesn't involve send/receive, so while the following 
is as I understand the functionality based on the wiki documentation and 
previous list discussion, I've never used send/receive myself, so if 
actual functionality appears to conflict with the below, consider me 
wrong and go with actual functionality...

AFAIK, when you btrfs receive a send, btrfs creates a subvolume 
duplicating the one being sent from the send side.

So in the scenario above, you should end up with two different snapshots 
under /backup, one for s1 and one for s2.  Due to the parent relationship 
and the fact that btrfs is copy-on-write, where extents in s1 and s2 
differ, the difference is sent (as a copy of the new extent along with 
metadata describing its place in reference to the parent) and the 
snapshots occupy the space used by both.  Where they do NOT differ, there 
is no difference to send, and the snapshots share the space, so while 
either one can be accessed, they're actually referencing the same shared 
extent and only that shared copy of that extent exists.

So as long as you keep both snapshots, you can access both, but given 1 
GiB in snap_s1 and 0.6 GiB worth of changes between it and snap_s2, the 
total space used by the two should still be only 1.6 GiB, even tho you 
can access the full 1 GiB of either snapshot.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: btrfs send/receive for backup of subvolumes

[Paul Harvey]

Disclaimer: I'm just a btrfs user, not an expert.

When dealing with a snapshot/subvolume, it should not matter how it
was created - once instantiated, the file contents should always be
the same. The resulting filesystem on the destination you've targeted
with "btrfs receive" should be usable in exactly in the same way as
the original regardless of whether it was created using an incremental
send or otherwise.

If you'd like some guarantee other than just blind faith that this is
the case, I've written a script [1] to help report sha512sums and PGP
signatures of directories. I wrote it to detect if the content of my
snapshots remain exactly (or at least mostly - see [2]) the same
regardless of what the eventual filesystem is (btrfs or not) and how
many send/receive transports had occurred.

You may find the full set of my scripts at [3] useful, the main script
can automatically snapshot all of your btrfs filesystems and each of
their subvolumes (or not, if you like). It does so by creating
snapshots under a '.snapshotz' directory in the root of each
subvolume, in the form of:
/path/to/subvol/.snapshotz/
YYYY-MM-DDTHHMMSS+hhmm (i.e. a valid
isodate). It also takes care of snapshot pruning, accumulating
measurement reports, and I've included some helper scripts to automate
transport of snapshots over ssh or between local filesystems.

Feel free to share any feedback you may have, but check the list of
bugs at [4] before you start using this code seriously.

[1] https://github.com/csirac2/snazzer/blob/master/doc/snazzer-measure.md
[2] https://bugzilla.kernel.org/show_bug.cgi?id=95201
[3] https://github.com/csirac2/snazzer
[4] https://github.com/csirac2/snazzer/issues