btrfs-send-receive vs rsync for (incremental/full) backups

btrfs-send-receive vs rsync for (incremental/full) backups

[Martin]

OK... So for backing up across a local network to a second physical host...

Is btrfs-send-receive stable enough now to be used?

How does send-receive compare to using rsync for backups?


Any comments please from those using such things?

Thanks,
Martin

Re: btrfs-send-receive vs rsync for (incremental/full) backups

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1163 bytes --]

On Fri, Dec 13, 2013 at 07:44:16PM +0000, Martin wrote:
> OK... So for backing up across a local network to a second physical host...
> 
> Is btrfs-send-receive stable enough now to be used?

   It seems to be OK for me at the moment -- I'm not running it across
a network (yet), but I am using it between different filesystems on
the same host, using pipes.

> How does send-receive compare to using rsync for backups?

   It's more fiddly to get right, and needs a bit more thought about
how to set it up sanely. I have a python library that I use to write
my backup scripts. I'll have to write it up one of these days...

> Any comments please from those using such things?

   Works for me, so far, as far as I know. Although we've got recent
reports of *something* still going wrong with systemd journal files.
No investigation of that yet, though.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
    --- In one respect at least, the Martians are a happy people: ---    
                          they have no lawyers.                          

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: btrfs-send-receive vs rsync for (incremental/full) backups

[Chris Murphy]

On Dec 13, 2013, at 1:35 PM, Hugo Mills <hugo@carfax.org.uk> wrote:

> On Fri, Dec 13, 2013 at 07:44:16PM +0000, Martin wrote:
>> OK... So for backing up across a local network to a second physical host...
>> 
>> Is btrfs-send-receive stable enough now to be used?
> 
>   It seems to be OK for me at the moment -- I'm not running it across
> a network (yet), but I am using it between different filesystems on
> the same host, using pipes.

Question. Are the source file system's checksums (and metadata) preserved in the send file? Or does the send file just contain data, and checksums are recomputed on receive? I'm curious if the send file can be stored on a non-checksumming file system, and yet have a means to verify the integrity of the files once received. Either the checksums in the send file being check during receive, or subsequent to receive completion by using scrub?

And yes I can confirm that send outputting to a file on a 2nd drive, and receiving it to yet another drive does work, and the source and destination subvolumes are the same in seemingly every way. I didn't separately rsync checksum compare the two subvolumes, however. That seems like possibly a good test. I also haven't tried doing it over a network.


Chris Murphy

Re: btrfs-send-receive vs rsync for (incremental/full) backups

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2207 bytes --]

On Fri, Dec 13, 2013 at 02:30:26PM -0700, Chris Murphy wrote:
> 
> On Dec 13, 2013, at 1:35 PM, Hugo Mills <hugo@carfax.org.uk> wrote:
> 
> > On Fri, Dec 13, 2013 at 07:44:16PM +0000, Martin wrote:
> >> OK... So for backing up across a local network to a second physical host...
> >> 
> >> Is btrfs-send-receive stable enough now to be used?
> > 
> >   It seems to be OK for me at the moment -- I'm not running it across
> > a network (yet), but I am using it between different filesystems on
> > the same host, using pipes.

> Question. Are the source file system's checksums (and metadata)
> preserved in the send file?

   I don't think checksums are transported (from what I remember of
the FAR format).

> Or does the send file just contain data, and checksums are
> recomputed on receive? I'm curious if the send file can be stored on
> a non-checksumming file system, and yet have a means to verify the
> integrity of the files once received. Either the checksums in the
> send file being check during receive, or subsequent to receive
> completion by using scrub?

   The FAR format contains a sequence of what amount to FS commands:
there are equivalents of cp, rm, chmod, chown, setfattr, cat (of data
inline in the FAR file). There's also a data type that clones an
extent from a file in another subvolume with a given UUID. Assuming
you have some way of keeping track of subvolume UUIDs on an FS that
doesn't support them, there's no reason you can't generate or unpack
FAR files on a non-checksumming (and non-CoW) FS.

> And yes I can confirm that send outputting to a file on a 2nd drive,
> and receiving it to yet another drive does work, and the source and
> destination subvolumes are the same in seemingly every way. I didn't
> separately rsync checksum compare the two subvolumes, however. That
> seems like possibly a good test. I also haven't tried doing it over
> a network.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- Once is happenstance; twice is coincidence; three times ---     
                            is enemy action.                             

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]