Re: cloning a btrfs drive with send and receive: clone is bigger than the original?

[Andrei Borzenkov <arvidjaar@gmail.com>]

10.01.2021 10:41, Cedric.dewijs@eclipso.eu пишет:
> I've tested some more.
> 
> Repeatedly sending the difference between two consecutive snapshots creates a structure on the target drive where all the snapshots share data. So 10 snapshots of 10 files of 100MB takes up 1GB, as expected.
> 
> Repeatedly sending the difference between the first snapshot and each next snapshot creates a structure on the target drive where the snapshots are independent, so they don't share any data.

How should "btrfs receive" know that in

btrfs send -p base snap1 | btrfs receive
btrfs send -p base snap2 | btrfs receive

snap1 and snap2 are related? By definition "btrfs send -p base" computes
difference to base snapshot and btrfs receive applies this difference to
replica of base snapshot. btrfs receive cannot reuse replica of snap1
because send stream does not contain any information about it.


> How can that be avoided?
> 

You can specify additional clone sources (btrfs send -p base -c snap1
snap2) but in your example the most efficient is to send delta between
two consecutive snapshots.

> Script (version that sends the difference between the first snapshot and each current snapshot):
> # cat ~/btrfs-send-test.sh 
> #!/bin/bash
> 
> btrfs subvolume delete /mnt/send/storage
> btrfs subvolume delete /mnt/send/snapshots/*
> btrfs subvolume delete /mnt/send/snapshots/
> btrfs subvolume delete /mnt/rec/diff
> btrfs subvolume delete /mnt/rec/snapshots/*
> btrfs subvolume delete /mnt/rec/snapshots/
> sync
> btrfs subvolume create /mnt/send/storage
> btrfs subvolume create /mnt/send/snapshots/
> btrfs subvolume create /mnt/rec/diff
> btrfs subvolume create /mnt/rec/snapshots
> 
> btrfs subvolume snapshot -r /mnt/send/storage/ /mnt/send/snapshots/0
> btrfs send /mnt/send/snapshots/0 | btrfs receive /mnt/rec/snapshots
> 
> onelesscounter=0
> counter=1
> while [ $counter -le 10 ]
> do
> 	dd if=/dev/urandom of=/mnt/send/storage/file$( printf %03d "$counter" ).bin bs=1M count=100
> 	md5sum /mnt/send/storage/file$( printf %03d "$counter" ).bin >> /mnt/send/storage/md5sums.txt
> 	btrfs subvolume snapshot -r /mnt/send/storage /mnt/send/snapshots/$counter
> 	btrfs send -p /mnt/send/snapshots/0 /mnt/send/snapshots/$counter -f /mnt/rec/diff/$counter
> 	#btrfs send -p /mnt/send/snapshots/$onelesscounter /mnt/send/snapshots/$counter -f /mnt/rec/diff/$counter 
> 	btrfs receive -f /mnt/rec/diff/$counter /mnt/rec/snapshots
> 	((counter++))
> 	((onelesscounter++))
> done
> echo All done
> 
> # df -h
> /dev/sda3       5.0G 1007M  3.6G  22% /mnt/send
> /dev/sdd2       932G   11G  919G   2% /mnt/rec
> 
> # ls -lh /mtn/rec/diff
> total 5.4G
> -rw------- 1 root root  101M Jan 10 09:17 1
> -rw------- 1 root root 1001M Jan 10 09:19 10
> -rw------- 1 root root  201M Jan 10 09:17 2
> -rw------- 1 root root  301M Jan 10 09:17 3
> -rw------- 1 root root  401M Jan 10 09:17 4
> -rw------- 1 root root  501M Jan 10 09:17 5
> -rw------- 1 root root  601M Jan 10 09:18 6
> -rw------- 1 root root  701M Jan 10 09:18 7
> -rw------- 1 root root  801M Jan 10 09:18 8
> -rw------- 1 root root  901M Jan 10 09:18 9
> 
> #rm /mtn/rec/diff/*
> #sync
> 
> # df -h
> /dev/sda3       5.0G 1007M  3.6G  22% /mnt/send
> /dev/sdd2       932G  5.4G  924G   1% /mnt/rec  <= all data is individually stored in the snapshots?
> 
> 
> 
> ---
> 
> Take your mailboxes with you. Free, fast and secure Mail &amp; Cloud: https://www.eclipso.eu - Time to change!
> 
>