Which better: rsync or snapshot + rsync --delete

Which better: rsync or snapshot + rsync --delete

[Martin]

Which is 'best' or 'faster'?

Take a snapshot of an existing backup and then "rsync --delete" into
that to make a backup of some other filesystem?

Or use "rsync --link" to link a new backup tree against a previous
backup tree for the some other filesystem?

Which case does btrfs handle the better?

Would there be any problems for doing this over an nfs mount of the btrfs?


Both cases can take advantage of the raid and dedup and compression
features of btrfs. Would taking a btrfs snapshot be better than rsync
creating the hard links to unchanged files?

Any other considerations?

(There are perhaps about 5% new or changed files each time.)

Thanks,
Martin

Re: Which better: rsync or snapshot + rsync --delete

[Bart Noordervliet]

Hi Martin,

though I can't determine which is 'better' for you because I don't
know what your aims are, I can recommend 'btrfs snapshot' together
with 'rsync --inplace'. It is 'better' in the sense of disk-usage,
because only the modified parts of files take up space in your
snapshots. That only applies if you use snapshots to retain multiple
historic versions of your backups, though. The cost is more
fragmentation, though snapshot-aware autodefrag (in-kernel since at
least 3.10) might help out a little there, and the fact that your
snapshot will be in an inconsistent state until rsync finishes
successfully.

Regards,

Bart


On Sat, Aug 3, 2013 at 1:59 AM, Martin <m_btrfs@ml1.co.uk> wrote:
> Which is 'best' or 'faster'?
>
> Take a snapshot of an existing backup and then "rsync --delete" into
> that to make a backup of some other filesystem?
>
> Or use "rsync --link" to link a new backup tree against a previous
> backup tree for the some other filesystem?
>
> Which case does btrfs handle the better?
>
> Would there be any problems for doing this over an nfs mount of the btrfs?
>
>
> Both cases can take advantage of the raid and dedup and compression
> features of btrfs. Would taking a btrfs snapshot be better than rsync
> creating the hard links to unchanged files?
>
> Any other considerations?
>
> (There are perhaps about 5% new or changed files each time.)
>
> Thanks,
> Martin
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Which better: rsync or snapshot + rsync --delete

[Kai Krakow]

Martin <m_btrfs@ml1.co.uk> schrieb:

> Which is 'best' or 'faster'?
> 
> Take a snapshot of an existing backup and then "rsync --delete" into
> that to make a backup of some other filesystem?
> 
> Or use "rsync --link" to link a new backup tree against a previous
> backup tree for the some other filesystem?

I'm doing it vice-versa... I rsync --inplace --delete to a scratch area and 
only then after that finished without errors, I'm taking a snapshot of that. 
That way I always know the snapshots are in good and consistent state while 
the scratch area is just playground for in-progress backups.

Just in case you are curious, here's the gist: 
https://gist.github.com/kakra/5520370

Using inplace and no-whole-file options ensures space-efficient snapshots. I 
don't worry about fragmentation, it's only the backup drive. It doesn't 
matter if it's fragmented and thus a little bit slower, it only matters 
restore works fine.

It is using systemd for automation but feel free to call it from cron...

HTH
Kai