* Copy BTRFS volume to another BTRFS volume including subvolumes and snapshots
@ 2016-10-13 21:21 Alberto Bursi
2016-10-14 11:38 ` Austin S. Hemmelgarn
0 siblings, 1 reply; 6+ messages in thread
From: Alberto Bursi @ 2016-10-13 21:21 UTC (permalink / raw)
To: linux-btrfs@vger.kernel.org
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 3416 bytes --]
Hi, I'm using OpenSUSE on a btrfs volume spanning 2 disks (set as raid1
for both metadata and data), no separate /home partition.
The distro loves to create dozens of subvolumes for various things and
makes snapshots, see:
alby@openSUSE-xeon:~> sudo btrfs subvolume list /
ID 257 gen 394 top level 5 path @
ID 258 gen 293390 top level 257 path @/.snapshots
ID 259 gen 293607 top level 258 path @/.snapshots/1/snapshot
ID 260 gen 107012 top level 257 path @/boot/grub2/i386-pc
ID 261 gen 107012 top level 257 path @/boot/grub2/x86_64-efi
ID 262 gen 293610 top level 257 path @/home
ID 263 gen 292439 top level 257 path @/opt
ID 264 gen 288726 top level 257 path @/srv
ID 265 gen 293610 top level 257 path @/tmp
ID 266 gen 292657 top level 257 path @/usr/local
ID 267 gen 104612 top level 257 path @/var/crash
ID 268 gen 133454 top level 257 path @/var/lib/libvirt/images
ID 269 gen 104612 top level 257 path @/var/lib/mailman
ID 270 gen 104612 top level 257 path @/var/lib/mariadb
ID 271 gen 292441 top level 257 path @/var/lib/mysql
ID 272 gen 104612 top level 257 path @/var/lib/named
ID 273 gen 104612 top level 257 path @/var/lib/pgsql
ID 274 gen 293608 top level 257 path @/var/log
ID 275 gen 104612 top level 257 path @/var/opt
ID 276 gen 293610 top level 257 path @/var/spool
ID 277 gen 293606 top level 257 path @/var/tmp
ID 362 gen 228259 top level 258 path @/.snapshots/56/snapshot
ID 364 gen 228259 top level 258 path @/.snapshots/57/snapshot
ID 528 gen 228259 top level 258 path @/.snapshots/110/snapshot
ID 529 gen 228259 top level 258 path @/.snapshots/111/snapshot
ID 670 gen 228259 top level 258 path @/.snapshots/240/snapshot
ID 671 gen 228259 top level 258 path @/.snapshots/241/snapshot
ID 894 gen 228283 top level 258 path @/.snapshots/438/snapshot
ID 895 gen 228283 top level 258 path @/.snapshots/439/snapshot
ID 896 gen 228283 top level 258 path @/.snapshots/440/snapshot
ID 897 gen 228283 top level 258 path @/.snapshots/441/snapshot
ID 1033 gen 288965 top level 258 path @/.snapshots/554/snapshot
ID 1034 gen 289531 top level 258 path @/.snapshots/555/snapshot
ID 1035 gen 289726 top level 258 path @/.snapshots/556/snapshot
ID 1036 gen 289729 top level 258 path @/.snapshots/557/snapshot
ID 1037 gen 290297 top level 258 path @/.snapshots/558/snapshot
ID 1038 gen 290301 top level 258 path @/.snapshots/559/snapshot
ID 1039 gen 290336 top level 258 path @/.snapshots/560/snapshot
ID 1041 gen 290338 top level 258 path @/.snapshots/562/snapshot
ID 1043 gen 292047 top level 258 path @/.snapshots/563/snapshot
ID 1044 gen 292051 top level 258 path @/.snapshots/564/snapshot
ID 1045 gen 292531 top level 258 path @/.snapshots/565/snapshot
ID 1046 gen 293153 top level 258 path @/.snapshots/566/snapshot
I'd like to be able to clone verbatim the whole volume to another
volume, for backup purposes.
Now, I think I can do that with a brutal partition clone from my
"recovery" (a debian system installed in another drive) and then doing
the procedures to deal with a lost drive.
But I'd rather prefer a clone on a live system, and without doing a
brutal clone as that will keep the same UUIDs.
I can(will) script this so even if it is a tedious process or it
involves writing a huge line in the commandline it's not an issue.
Thanks for any input.
-Alberto
ÿôèº{.nÇ+‰·Ÿ®‰†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±ý»k~ÏâžØ^n‡r¡ö¦zË\x1aëh™¨èÚ&£ûàz¿äz¹Þ—ú+€Ê+zf£¢·hšˆ§~††Ûiÿÿïêÿ‘êçz_è®\x0fæj:+v‰¨þ)ߣøm
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Copy BTRFS volume to another BTRFS volume including subvolumes and snapshots
2016-10-13 21:21 Copy BTRFS volume to another BTRFS volume including subvolumes and snapshots Alberto Bursi
@ 2016-10-14 11:38 ` Austin S. Hemmelgarn
2016-10-14 22:07 ` Alberto Bursi
0 siblings, 1 reply; 6+ messages in thread
From: Austin S. Hemmelgarn @ 2016-10-14 11:38 UTC (permalink / raw)
To: Alberto Bursi, linux-btrfs@vger.kernel.org
On 2016-10-13 17:21, Alberto Bursi wrote:
> Hi, I'm using OpenSUSE on a btrfs volume spanning 2 disks (set as raid1
> for both metadata and data), no separate /home partition.
> The distro loves to create dozens of subvolumes for various things and
> makes snapshots, see:
> alby@openSUSE-xeon:~> sudo btrfs subvolume list /
> ID 257 gen 394 top level 5 path @
> ID 258 gen 293390 top level 257 path @/.snapshots
> ID 259 gen 293607 top level 258 path @/.snapshots/1/snapshot
> ID 260 gen 107012 top level 257 path @/boot/grub2/i386-pc
> ID 261 gen 107012 top level 257 path @/boot/grub2/x86_64-efi
> ID 262 gen 293610 top level 257 path @/home
> ID 263 gen 292439 top level 257 path @/opt
> ID 264 gen 288726 top level 257 path @/srv
> ID 265 gen 293610 top level 257 path @/tmp
> ID 266 gen 292657 top level 257 path @/usr/local
> ID 267 gen 104612 top level 257 path @/var/crash
> ID 268 gen 133454 top level 257 path @/var/lib/libvirt/images
> ID 269 gen 104612 top level 257 path @/var/lib/mailman
> ID 270 gen 104612 top level 257 path @/var/lib/mariadb
> ID 271 gen 292441 top level 257 path @/var/lib/mysql
> ID 272 gen 104612 top level 257 path @/var/lib/named
> ID 273 gen 104612 top level 257 path @/var/lib/pgsql
> ID 274 gen 293608 top level 257 path @/var/log
> ID 275 gen 104612 top level 257 path @/var/opt
> ID 276 gen 293610 top level 257 path @/var/spool
> ID 277 gen 293606 top level 257 path @/var/tmp
> ID 362 gen 228259 top level 258 path @/.snapshots/56/snapshot
> ID 364 gen 228259 top level 258 path @/.snapshots/57/snapshot
> ID 528 gen 228259 top level 258 path @/.snapshots/110/snapshot
> ID 529 gen 228259 top level 258 path @/.snapshots/111/snapshot
> ID 670 gen 228259 top level 258 path @/.snapshots/240/snapshot
> ID 671 gen 228259 top level 258 path @/.snapshots/241/snapshot
> ID 894 gen 228283 top level 258 path @/.snapshots/438/snapshot
> ID 895 gen 228283 top level 258 path @/.snapshots/439/snapshot
> ID 896 gen 228283 top level 258 path @/.snapshots/440/snapshot
> ID 897 gen 228283 top level 258 path @/.snapshots/441/snapshot
> ID 1033 gen 288965 top level 258 path @/.snapshots/554/snapshot
> ID 1034 gen 289531 top level 258 path @/.snapshots/555/snapshot
> ID 1035 gen 289726 top level 258 path @/.snapshots/556/snapshot
> ID 1036 gen 289729 top level 258 path @/.snapshots/557/snapshot
> ID 1037 gen 290297 top level 258 path @/.snapshots/558/snapshot
> ID 1038 gen 290301 top level 258 path @/.snapshots/559/snapshot
> ID 1039 gen 290336 top level 258 path @/.snapshots/560/snapshot
> ID 1041 gen 290338 top level 258 path @/.snapshots/562/snapshot
> ID 1043 gen 292047 top level 258 path @/.snapshots/563/snapshot
> ID 1044 gen 292051 top level 258 path @/.snapshots/564/snapshot
> ID 1045 gen 292531 top level 258 path @/.snapshots/565/snapshot
> ID 1046 gen 293153 top level 258 path @/.snapshots/566/snapshot
>
> I'd like to be able to clone verbatim the whole volume to another
> volume, for backup purposes.
>
> Now, I think I can do that with a brutal partition clone from my
> "recovery" (a debian system installed in another drive) and then doing
> the procedures to deal with a lost drive.
>
> But I'd rather prefer a clone on a live system, and without doing a
> brutal clone as that will keep the same UUIDs.
>
> I can(will) script this so even if it is a tedious process or it
> involves writing a huge line in the commandline it's not an issue.
I'm not sure there is any way to do this on a live system. You
essentially need to either do a block level copy and change the UUID's
(which recent versions of btrfstune can do), or use send with some
manual setup to get the subvolumes across. Both options require the
filesystem to be effectively read-only, which is not something that any
modern Linux distro can reliably handle for more than a few minutes.
If I had to do this, I'd go with the block level copy, since it requires
a lot less effort, just make sure to use btrfstune to change the UUID's
when the copy is done (that may take a while itself though, since it has
to rewrite a lot of metadata).
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Copy BTRFS volume to another BTRFS volume including subvolumes and snapshots
2016-10-14 11:38 ` Austin S. Hemmelgarn
@ 2016-10-14 22:07 ` Alberto Bursi
2016-10-14 22:17 ` Chris Murphy
0 siblings, 1 reply; 6+ messages in thread
From: Alberto Bursi @ 2016-10-14 22:07 UTC (permalink / raw)
To: linux-btrfs@vger.kernel.org
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2204 bytes --]
On 10/14/2016 01:38 PM, Austin S. Hemmelgarn wrote:
> On 2016-10-13 17:21, Alberto Bursi wrote:
>> Hi, I'm using OpenSUSE on a btrfs volume spanning 2 disks (set as raid1
>> for both metadata and data), no separate /home partition.
>> -----
>> I'd like to be able to clone verbatim the whole volume to another
>> volume, for backup purposes.
>>
>> Now, I think I can do that with a brutal partition clone from my
>> "recovery" (a debian system installed in another drive) and then doing
>> the procedures to deal with a lost drive.
>>
>> But I'd rather prefer a clone on a live system, and without doing a
>> brutal clone as that will keep the same UUIDs.
>>
>> I can(will) script this so even if it is a tedious process or it
>> involves writing a huge line in the commandline it's not an issue.
> I'm not sure there is any way to do this on a live system. You
> essentially need to either do a block level copy and change the UUID's
> (which recent versions of btrfstune can do), or use send with some
> manual setup to get the subvolumes across. Both options require the
> filesystem to be effectively read-only, which is not something that any
> modern Linux distro can reliably handle for more than a few minutes.
>
> If I had to do this, I'd go with the block level copy, since it requires
> a lot less effort, just make sure to use btrfstune to change the UUID's
> when the copy is done (that may take a while itself though, since it has
> to rewrite a lot of metadata).
Heh, one of the reasons I migrated to btrfs was that I wanted to do
things like these on a live system.
With my script I can already generate any necessary folder structure and
send over all subvolumes with a bunch of btrfs send | btrfs receive with
like 5 lines of script.
I was hoping there was some neat trick with btrfs send | btrfs receive
that allowed me to send a snapshot of / to the / of the new volume.
With rsync (from a ro snapshot of /) it should be possible to use the
subvolume list as exclusion list, but I'd have rather wanted to use a
btrfs feature instead.
-Alberto
ÿôèº{.nÇ+‰·Ÿ®‰†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±ý»k~ÏâžØ^n‡r¡ö¦zË\x1aëh™¨èÚ&£ûàz¿äz¹Þ—ú+€Ê+zf£¢·hšˆ§~††Ûiÿÿïêÿ‘êçz_è®\x0fæj:+v‰¨þ)ߣøm
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Copy BTRFS volume to another BTRFS volume including subvolumes and snapshots
2016-10-14 22:07 ` Alberto Bursi
@ 2016-10-14 22:17 ` Chris Murphy
2016-10-14 22:58 ` Alberto Bursi
0 siblings, 1 reply; 6+ messages in thread
From: Chris Murphy @ 2016-10-14 22:17 UTC (permalink / raw)
To: Alberto Bursi; +Cc: linux-btrfs@vger.kernel.org
It should be -e can accept a listing of all the subvolumes you want to
send at once. And possibly an -r flag, if it existed, could
automatically populate -e. But the last time I tested -e I just got
errors.
https://bugzilla.kernel.org/show_bug.cgi?id=111221
--
Chris Murphy
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Copy BTRFS volume to another BTRFS volume including subvolumes and snapshots
2016-10-14 22:17 ` Chris Murphy
@ 2016-10-14 22:58 ` Alberto Bursi
2016-10-17 3:46 ` Andrei Borzenkov
0 siblings, 1 reply; 6+ messages in thread
From: Alberto Bursi @ 2016-10-14 22:58 UTC (permalink / raw)
To: linux-btrfs@vger.kernel.org; +Cc: linux-btrfs@vger.kernel.org
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1210 bytes --]
On 10/15/2016 12:17 AM, Chris Murphy wrote:
> It should be -e can accept a listing of all the subvolumes you want to
> send at once. And possibly an -r flag, if it existed, could
> automatically populate -e. But the last time I tested -e I just got
> errors.
>
> https://bugzilla.kernel.org/show_bug.cgi?id=111221
>
>
Not a problem (for me anyway), I can send all subvolumes already with my
script (one after another, but still automatically).
What I can't do with btrfs commands is to send over the contents of a ro
snapshot of / called for example "oldRootSnapshot", directly to
"/tmp/newroot" (which is where I have mounted the other drive/volume).
The only thing I can do is send over the subvolume as a subvolume.
So I end up with /tmp/newroot/oldRootSnapshot and inside oldRootSnapshot
I get my root, not what I wanted.
Only way I found so far is using rsync to move the contents of
oldRootSnapshot in the /tmp/newroot by setting an exclusion list for all
subvolumes, then run a deduplication with duperemove.
So, is there something I missed to do that?
-Alberto
ÿôèº{.nÇ+‰·Ÿ®‰†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±ý»k~ÏâžØ^n‡r¡ö¦zË\x1aëh™¨èÚ&£ûàz¿äz¹Þ—ú+€Ê+zf£¢·hšˆ§~††Ûiÿÿïêÿ‘êçz_è®\x0fæj:+v‰¨þ)ߣøm
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Copy BTRFS volume to another BTRFS volume including subvolumes and snapshots
2016-10-14 22:58 ` Alberto Bursi
@ 2016-10-17 3:46 ` Andrei Borzenkov
0 siblings, 0 replies; 6+ messages in thread
From: Andrei Borzenkov @ 2016-10-17 3:46 UTC (permalink / raw)
To: Alberto Bursi, linux-btrfs@vger.kernel.org
15.10.2016 01:58, Alberto Bursi пишет:
>
>
> On 10/15/2016 12:17 AM, Chris Murphy wrote:
>> It should be -e can accept a listing of all the subvolumes you want to
>> send at once. And possibly an -r flag, if it existed, could
>> automatically populate -e. But the last time I tested -e I just got
>> errors.
>>
>> https://bugzilla.kernel.org/show_bug.cgi?id=111221
>>
>>
>
> Not a problem (for me anyway), I can send all subvolumes already with my
> script (one after another, but still automatically).
>
> What I can't do with btrfs commands is to send over the contents of a ro
> snapshot of / called for example "oldRootSnapshot", directly to
> "/tmp/newroot" (which is where I have mounted the other drive/volume).
>
Somehow this is expected - it sends one subvolume to another subvolume.
I am not sure whether zfs can do it either.
But speaking about openSUSE - it does not have any real data in `/' at
all - it is just skeleton of root filesystem with a couple of
directories where actual root is in one of /.snapshots subvolumes.
> The only thing I can do is send over the subvolume as a subvolume.
> So I end up with /tmp/newroot/oldRootSnapshot and inside oldRootSnapshot
> I get my root, not what I wanted.
>
> Only way I found so far is using rsync to move the contents of
> oldRootSnapshot in the /tmp/newroot by setting an exclusion list for all
> subvolumes, then run a deduplication with duperemove.
>
> So, is there something I missed to do that?
>
> -Alberto
> N�����r��y���b�X��ǧv�^�){.n�+����{�n�߲)���w*\x1fjg���\x1e�����ݢj/���z�ޖ��2�ޙ���&�)ߡ�a��\x7f��\x1e�G���h�\x0f�j:+v���w�٥
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-10-17 3:46 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-10-13 21:21 Copy BTRFS volume to another BTRFS volume including subvolumes and snapshots Alberto Bursi
2016-10-14 11:38 ` Austin S. Hemmelgarn
2016-10-14 22:07 ` Alberto Bursi
2016-10-14 22:17 ` Chris Murphy
2016-10-14 22:58 ` Alberto Bursi
2016-10-17 3:46 ` Andrei Borzenkov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).