rollback to a snapshot

rollback to a snapshot

[Bernd Lentes]

Hi guys,

thanks for your help.

I have an Ubuntu box which I wanted to upgrade. Fortunately I made a snapshot before.
The upgrade ran only partially and now I want to go back to my snapshot.
I booted the system now with a recuse cd.
This is my setup:

root@Microknoppix:/home/knoppix# mount|grep btrfs
/dev/mapper/ubuntu--vg-ubuntu--lv on /mnt/btrfs type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)

root@Microknoppix:/home/knoppix# btrfs sub list /mnt/btrfs
ID 430 gen 1215864 top level 5 path .snapshots
ID 434 gen 1213568 top level 430 path .snapshots/06-06-2023--15:16_PRE_UPGRADE
ID 435 gen 1216086 top level 430 path .snapshots/06-06-2023
I want to go back to ID 434 or 435.

I found a lot of different approaches in the net, so I’m a bit confused.
Changing the default subvolume, moving or renaming subvolumes … A lot of examples have a subvolume @ which I don’t have.

Can you help me please ?

Bernd

Bernd Lentes

--
Bernd Lentes
System Administrator
MCD
Helmholtzzentrum München
+49 89 3187 1241
mailto:bernd.lentes@helmholtz-munich.de
https://www.helmholtz-munich.de/en/mcd


Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de
Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp, Daniela Sommer (komm.) | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling
Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671

Re: rollback to a snapshot

[Eldon]

On Tue, Jun 06, 2023 at 08:58:03PM +0000, Bernd Lentes wrote:
> Hi guys,
> 
> thanks for your help.
> 
> I have an Ubuntu box which I wanted to upgrade. Fortunately I made a snapshot before.
> The upgrade ran only partially and now I want to go back to my snapshot.
> I booted the system now with a recuse cd.
> This is my setup:
> 
> root@Microknoppix:/home/knoppix# mount|grep btrfs
> /dev/mapper/ubuntu--vg-ubuntu--lv on /mnt/btrfs type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)
> 
> root@Microknoppix:/home/knoppix# btrfs sub list /mnt/btrfs
> ID 430 gen 1215864 top level 5 path .snapshots
> ID 434 gen 1213568 top level 430 path .snapshots/06-06-2023--15:16_PRE_UPGRADE
> ID 435 gen 1216086 top level 430 path .snapshots/06-06-2023
> I want to go back to ID 434 or 435.

Hi Bernd,
I would generally agree that changing the default subvolume is the best
strategy to get the desired result. I would create a new writable
subvolume from the snapshot you want to use, in a reasonable place to
put it. I use my own idiosyncratic paths, but you can use whatever you
want. I would use a destination outside of the .snapshots directory,
maybe something like .live .
Then, do the btrfs subvolume set-default to that subvolume.

That's how I usually do my rollbacks, hope it helps with your situation.

Cheers,
Eldon

Re: rollback to a snapshot

[Andrei Borzenkov]

On 06.06.2023 23:58, Bernd Lentes wrote:
> Hi guys,
> 
> thanks for your help.
> 
> I have an Ubuntu box which I wanted to upgrade. Fortunately I made a snapshot before.
> The upgrade ran only partially and now I want to go back to my snapshot.
> I booted the system now with a recuse cd.
> This is my setup:
> 
> root@Microknoppix:/home/knoppix# mount|grep btrfs
> /dev/mapper/ubuntu--vg-ubuntu--lv on /mnt/btrfs type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)
> 
> root@Microknoppix:/home/knoppix# btrfs sub list /mnt/btrfs
> ID 430 gen 1215864 top level 5 path .snapshots
> ID 434 gen 1213568 top level 430 path .snapshots/06-06-2023--15:16_PRE_UPGRADE
> ID 435 gen 1216086 top level 430 path .snapshots/06-06-2023
> I want to go back to ID 434 or 435.
> 
> I found a lot of different approaches in the net, so I’m a bit confused.
> Changing the default subvolume, moving or renaming subvolumes … A lot of examples have a subvolume @ which I don’t have.
> 

You cannot rename or move because you cannot "rename" or "move" 
subvolume to become filesystem top level. Which is one reason why you 
should never use btrfs top level subvolume if you plan to use snapshots 
and be able to revert. Actually you should probably never use btrfs top 
level subvolume except as container for other subvolumes. Period.

You could simply rsync from snapshot to revert the content of your root. 
It would be the most simple way (it will probably increase space 
consumption slightly). Or you could boot linux with subvol=... rootfs 
option - there is no need to actually change default subvolume.

If you chose "use different subvolume as root" route, keep in mind that

- you will likely need to reinstall bootloader because paths will change.

- any subvolume below / like /.snapshots will be invisible from your 
booted system and you will need to explicitly mount it if you need to 
access its content.

- as mentioned already, do not flip read-only snapshot to read-write, 
rather create new writable clone. I would also chose some different path 
leaving /.snapshots for snapshots. It does not matter in this case (at 
least, with the information provided so far) but it is good habit to 
develop.

- after you booted new root subvolume you will have the old root content 
in top level which cannot be easily deleted because it is not subvolume 
(and it will be invisible as well). You will have to clean it up 
manually, and be careful to not delete your subvolumes doing it :)

RE: rollback to a snapshot

[Bernd Lentes]

Dear Andrei,


>-----Original Message-----
>From: Andrei Borzenkov <arvidjaar@gmail.com>
>Sent: Wednesday, June 7, 2023 6:37 AM
>To: Bernd Lentes <bernd.lentes@helmholtz-muenchen.de>; linux-
>btrfs@vger.kernel.org
>Subject: Re: rollback to a snapshot
>
>You cannot rename or move because you cannot "rename" or "move"
>subvolume to become filesystem top level. Which is one reason why you
>should never use btrfs top level subvolume if you plan to use snapshots and be
>able to revert. Actually you should probably never use btrfs top level
>subvolume except as container for other subvolumes. Period.

Could you propose a setup here which does not use top level subvolumes ?
I found it very frustrating that everyone says "BTRFS is great, you can do snapshots and easily rollback".
Because in reality rollback is not easy. How can i avoid to use top level subvolumes ?
I have some Suse boxes which seem to have a correct default setup. But when i install e.g. an Ubuntu box
i really don't know how to setup BTRFS manually.
There was a great Wiki (https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/Main_Page.html),
but now it's outdated. Is there something new ?

>You could simply rsync from snapshot to revert the content of your root.
>It would be the most simple way (it will probably increase space consumption
>slightly). Or you could boot linux with subvol=... rootfs option - there is no
>need to actually change default subvolume.
>
>If you chose "use different subvolume as root" route, keep in mind that
>
>- you will likely need to reinstall bootloader because paths will change.
>
>- any subvolume below / like /.snapshots will be invisible from your booted
>system and you will need to explicitly mount it if you need to access its
>content.
>
>- as mentioned already, do not flip read-only snapshot to read-write, rather
>create new writable clone.

Why not ?

Bernd

Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de
Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp, Daniela Sommer (komm.) | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling
Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671

Re: rollback to a snapshot

[Andrei Borzenkov]

On Wed, Jun 7, 2023 at 1:38 PM Bernd Lentes
<bernd.lentes@helmholtz-muenchen.de> wrote:
> >
> >You cannot rename or move because you cannot "rename" or "move"
> >subvolume to become filesystem top level. Which is one reason why you
> >should never use btrfs top level subvolume if you plan to use snapshots and be
> >able to revert. Actually you should probably never use btrfs top level
> >subvolume except as container for other subvolumes. Period.
>
> Could you propose a setup here which does not use top level subvolumes ?

In case I was not clear - I mean using top level btrfs root (subvolume
ID 5). Of course there is no problem using subvolume directly under
root (like /root1, /root2 etc).

> I found it very frustrating that everyone says "BTRFS is great, you can do snapshots and easily rollback".
> Because in reality rollback is not easy.

Again - you can always rsync back. Or use any other directory
synchronization tool to bring your original subvolume to the same
state.

> How can i avoid to use top level subvolumes ?

That really depends on what installer of your distribution offers and supports.

> I have some Suse boxes which seem to have a correct default setup.

They started it with the wrong one and had to change the setup exactly
due to rollback problems.

>  But when i install e.g. an Ubuntu box
> i really don't know how to setup BTRFS manually.

The question cannot be answered globally because it depends on tools
your distribution offers. The SUSE setup is correct for SUSE tools and
"foreign" tools have problems with it (like grub2 using different
rules to resolve path names on btrfs).

If you are going to do rollback manually - you set it up so that you
know how to rollback. Otherwise it should be described in your
distribution guides.

> >- as mentioned already, do not flip read-only snapshot to read-write, rather
> >create new writable clone.
>

It breaks the replication chain. You may not be using it right now,
but should you decide to use it, it is better you have the habit of
doing it right. Just look how many reports about replication problems
on this list were traced back to ro -> rw change.

And it is useful to have the original snapshot as a reference so you
always have your known good recovery point.

Re: rollback to a snapshot

[Phillip Susi]

Bernd Lentes <bernd.lentes@helmholtz-muenchen.de> writes:

> Could you propose a setup here which does not use top level subvolumes ?
> I found it very frustrating that everyone says "BTRFS is great, you can do snapshots and easily rollback".
> Because in reality rollback is not easy. How can i avoid to use top level subvolumes ?
> I have some Suse boxes which seem to have a correct default setup. But when i install e.g. an Ubuntu box
> i really don't know how to setup BTRFS manually.
> There was a great Wiki (https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/Main_Page.html),
> but now it's outdated. Is there something new ?

The last time I installed Ubuntu on btrfs, the installer automatically
created a top level subvolume named @ and actually installed the system
in that subvolume, then configured grub to tell the kernel to mount the
root btrfs volume with the flag subvol="@" so that the system would boot
normally.  Then you just mount the real root of the filesystem somewhere
else and make a snapshot of the @ subvolume.  When you want to roll
back, you just rename @ to something else, and either rename or create a
new writable snapshot from your previous snapshot and name it @, then
reboot.

RE: rollback to a snapshot

[Bernd Lentes]

>-----Original Message-----
>From: Phillip Susi <phill@thesusis.net>
>Sent: Wednesday, June 7, 2023 10:09 PM
>To: Bernd Lentes <bernd.lentes@helmholtz-muenchen.de>
>Cc: Andrei Borzenkov <arvidjaar@gmail.com>; linux-btrfs@vger.kernel.org
>Subject: Re: rollback to a snapshot

>The last time I installed Ubuntu on btrfs, the installer automatically created a
>top level subvolume named @ and actually installed the system in that
>subvolume, then configured grub to tell the kernel to mount the root btrfs
>volume with the flag subvol="@" so that the system would boot normally.
>Then you just mount the real root of the filesystem somewhere else and make
>a snapshot of the @ subvolume.  When you want to roll back, you just rename
>@ to something else, and either rename or create a new writable snapshot
>from your previous snapshot and name it @, then reboot.

Do you remember which version it was ?
You say " top level subvolume named @". You mean subvolid 5 ?
" Then you just mount the real root of the filesystem". What do you mean by "real root" ?
Does the subvolume @ already contain files or is it just a container for another subvolume ?
"you just rename @ to something else". How can i rename a subvolume ? Just rename the corresponding directory ?
" Then you just mount the real root of the filesystem somewhere else" Why ? With a bind mount ?

Bernd

Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de
Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp, Daniela Sommer (komm.) | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling
Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671

RE: rollback to a snapshot

[Nicholas D Steeves]

[-- Attachment #1: Type: text/plain, Size: 4232 bytes --]

Bernd Lentes <bernd.lentes@helmholtz-muenchen.de> writes:

>>-----Original Message-----
>>From: Phillip Susi <phill@thesusis.net>
>>Sent: Wednesday, June 7, 2023 10:09 PM
>>To: Bernd Lentes <bernd.lentes@helmholtz-muenchen.de>
>>Cc: Andrei Borzenkov <arvidjaar@gmail.com>; linux-btrfs@vger.kernel.org
>>Subject: Re: rollback to a snapshot
>
>>The last time I installed Ubuntu on btrfs, the installer automatically created a
>>top level subvolume named @ and actually installed the system in that
>>subvolume, then configured grub to tell the kernel to mount the root btrfs
>>volume with the flag subvol="@" so that the system would boot normally.
>>Then you just mount the real root of the filesystem somewhere else and make
>>a snapshot of the @ subvolume.  When you want to roll back, you just rename
>>@ to something else, and either rename or create a new writable snapshot
>>from your previous snapshot and name it @, then reboot.
>
> Do you remember which version it was ?
> You say " top level subvolume named @". You mean subvolid 5 ?
> " Then you just mount the real root of the filesystem". What do you mean by "real root" ?
> Does the subvolume @ already contain files or is it just a container for another subvolume ?
> "you just rename @ to something else". How can i rename a subvolume ? Just rename the corresponding directory ?
> " Then you just mount the real root of the filesystem somewhere else" Why ? With a bind mount ?
>
> Bernd

If I remember correctly, Ubuntu looks something like this

sudo btrfs sub list / -t
ID      gen     top level       path
--      ---     ---------       ----
347     477642  5               @
348     476215  5               @home

which is

subvolid=5
         |_ @
         |_ @home

and you shared this info in the initial thread:

> root@Microknoppix:/home/knoppix# mount|grep btrfs
> /dev/mapper/ubuntu--vg-ubuntu--lv on /mnt/btrfs type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)

> root@Microknoppix:/home/knoppix# btrfs sub list /mnt/btrfs
> ID 430 gen 1215864 top level 5 path .snapshots
> ID 434 gen 1213568 top level 430 path .snapshots/06-06-2023--15:16_PRE_UPGRADE
> ID 435 gen 1216086 top level 430 path .snapshots/06-06-2023
> I want to go back to ID 434 or 435.

So you have something like
1. subvolid=5
2.          |_ @
3.          |_ @home
4.          |_ snapshots
5.                     |_ 06-06-2023--15:16_PRE_UPGRADE
6.                     |_ 06-06-2023

subvolid=5 is always the "real root" I think Andrei Borzenkov is calling
it to mitigate the confusion resulting from potentially having used the
default subvolume feature, which in my opinion should never be used,
because it will confuse the bootloader.  A default Ubuntu installation
will have "subvol=@" in the bootloader, which means all that you need to
do is move your snapshot into place (position #2 in the above table)

So I think you're going to do something like the following: Mount the
true / (position #1) or subvolid=5 to /mnt/btrfs, then

  cd /mnt/btrfs/snapshots
  btrfs sub snap -r ../@ ./@_broken-upgrade  # note readonly "-r"
  cd ..
  # I still do a subvol sync and fi sync here
  btrfs sub del -c ./@
  # I still do a subvol sync and fi sync here
  btrfs sub snap ./snapshots/06-06-2023--15:16_PRE_UPGRADE ./@
  #             /\ note no readonly "-r"

If you stuck with Ubuntu defaults and didn't use the default subvolume
feature, then subvol=@ is still what grub will boot, and
06-06-2023--15:16_PRE_UPGRADE is now the new writable @.

I believe this is the safest method available.  The perfect snapshotting
tool would make it easier to see the relationship between snapshoted
copies of subvolumes.

Note that rootfs rollbacks won't fix potentially incompatibly upgraded
databases in @home (is that rare these days?).  There's not really any
good fix for that, other than to restore them from backup.  Note that
with this method you will lose a day's worth of everything in @ that
didn't have its own subvolume (ie logs, any VMs not in @home, etc).

For the record: If you consult the table above, the default subvolume
feature makes any subvolume become the user-apparent / of the
filesystem, and hides anything that's not beneath its tree.

'hope this helps,
Nicholas

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

RE: rollback to a snapshot

[Bernd Lentes]

>-----Original Message-----
>From: Nicholas D Steeves <nsteeves@gmail.com>
>Sent: Thursday, June 8, 2023 12:18 AM
>To: Bernd Lentes <bernd.lentes@helmholtz-muenchen.de>; Phillip Susi
><phill@thesusis.net>
>Cc: Andrei Borzenkov <arvidjaar@gmail.com>; linux-btrfs@vger.kernel.org
>Subject: RE: rollback to a snapshot
>


>If I remember correctly, Ubuntu looks something like this
>
>sudo btrfs sub list / -t
>ID      gen     top level       path
>--      ---     ---------       ----
>347     477642  5               @
>348     476215  5               @home
>
>which is
>
>subvolid=5
>         |_ @
>         |_ @home
>
>and you shared this info in the initial thread:
>
>> root@Microknoppix:/home/knoppix# mount|grep btrfs
>> /dev/mapper/ubuntu--vg-ubuntu--lv on /mnt/btrfs type btrfs
>> (rw,relatime,space_cache,subvolid=5,subvol=/)
>
>> root@Microknoppix:/home/knoppix# btrfs sub list /mnt/btrfs ID 430 gen
>> 1215864 top level 5 path .snapshots ID 434 gen 1213568 top level 430
>> path .snapshots/06-06-2023--15:16_PRE_UPGRADE
>> ID 435 gen 1216086 top level 430 path .snapshots/06-06-2023 I want to
>> go back to ID 434 or 435.
>
>So you have something like
>1. subvolid=5
>2.          |_ @
>3.          |_ @home
>4.          |_ snapshots
>5.                     |_ 06-06-2023--15:16_PRE_UPGRADE
>6.                     |_ 06-06-2023
>
>subvolid=5 is always the "real root" I think Andrei Borzenkov is calling it to
>mitigate the confusion resulting from potentially having used the default
>subvolume feature, which in my opinion should never be used, because it will
>confuse the bootloader.  A default Ubuntu installation will have "subvol=@" in
>the bootloader, which means all that you need to do is move your snapshot
>into place (position #2 in the above table)
>
>So I think you're going to do something like the following: Mount the true /
>(position #1) or subvolid=5 to /mnt/btrfs, then
>
>  cd /mnt/btrfs/snapshots
>  btrfs sub snap -r ../@ ./@_broken-upgrade  # note readonly "-r"
>  cd ..
>  # I still do a subvol sync and fi sync here
>  btrfs sub del -c ./@
>  # I still do a subvol sync and fi sync here
>  btrfs sub snap ./snapshots/06-06-2023--15:16_PRE_UPGRADE ./@
>  #             /\ note no readonly "-r"
>
>If you stuck with Ubuntu defaults and didn't use the default subvolume
>feature, then subvol=@ is still what grub will boot, and 06-06-2023--
>15:16_PRE_UPGRADE is now the new writable @.
>
>I believe this is the safest method available.  The perfect snapshotting tool
>would make it easier to see the relationship between snapshoted copies of
>subvolumes.
>
>Note that rootfs rollbacks won't fix potentially incompatibly upgraded
>databases in @home (is that rare these days?).  There's not really any good fix
>for that, other than to restore them from backup.  Note that with this method
>you will lose a day's worth of everything in @ that didn't have its own
>subvolume (ie logs, any VMs not in @home, etc).
>
>For the record: If you consult the table above, the default subvolume feature
>makes any subvolume become the user-apparent / of the filesystem, and
>hides anything that's not beneath its tree.
>
>'hope this helps,
>Nicholas

Hi Nicholas,

thanks for this thorough explanation.
I don't have these @ subvolumes.
My setup is:
root@crispor-server:~/skripte# mount|grep btrfs
/dev/mapper/ubuntu--vg-ubuntu--lv on / type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)

So i put the / directly in the top level subvolume (ID 5). From what i understand this is not recommended, right ?
Is it better to put a subvolume (maybe called @)  in the top level subvolume and then mount the root fs to @ ?

Bernd

Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de
Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp, Daniela Sommer (komm.) | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling
Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671

RE: rollback to a snapshot

[Bernd Lentes]

>-----Original Message-----
>From: Nicholas D Steeves <nsteeves@gmail.com>
>Sent: Thursday, June 8, 2023 12:18 AM
>To: Bernd Lentes <bernd.lentes@helmholtz-muenchen.de>; Phillip Susi
><phill@thesusis.net>
>Cc: Andrei Borzenkov <arvidjaar@gmail.com>; linux-btrfs@vger.kernel.org
>Subject: RE: rollback to a snapshot
>
>Bernd Lentes <bernd.lentes@helmholtz-muenchen.de> writes:
>
>If I remember correctly, Ubuntu looks something like this
>
>sudo btrfs sub list / -t
>ID      gen     top level       path
>--      ---     ---------       ----
>347     477642  5               @
>348     476215  5               @home
>
>which is
>
>subvolid=5
>         |_ @
>         |_ @home
>
>and you shared this info in the initial thread:
>
>> root@Microknoppix:/home/knoppix# mount|grep btrfs
>> /dev/mapper/ubuntu--vg-ubuntu--lv on /mnt/btrfs type btrfs
>> (rw,relatime,space_cache,subvolid=5,subvol=/)
>
>> root@Microknoppix:/home/knoppix# btrfs sub list /mnt/btrfs ID 430 gen
>> 1215864 top level 5 path .snapshots ID 434 gen 1213568 top level 430
>> path .snapshots/06-06-2023--15:16_PRE_UPGRADE
>> ID 435 gen 1216086 top level 430 path .snapshots/06-06-2023 I want to
>> go back to ID 434 or 435.
>
>So you have something like
>1. subvolid=5
>2.          |_ @
>3.          |_ @home
>4.          |_ snapshots
>5.                     |_ 06-06-2023--15:16_PRE_UPGRADE
>6.                     |_ 06-06-2023
>
>subvolid=5 is always the "real root" I think Andrei Borzenkov is calling it to
>mitigate the confusion resulting from potentially having used the default
>subvolume feature, which in my opinion should never be used, because it will
>confuse the bootloader.  A default Ubuntu installation will have "subvol=@" in
>the bootloader, which means all that you need to do is move your snapshot
>into place (position #2 in the above table)
>
>So I think you're going to do something like the following: Mount the true /
>(position #1) or subvolid=5 to /mnt/btrfs, then
>
>  cd /mnt/btrfs/snapshots
>  btrfs sub snap -r ../@ ./@_broken-upgrade  # note readonly "-r"
>  cd ..
>  # I still do a subvol sync and fi sync here
>  btrfs sub del -c ./@
>  # I still do a subvol sync and fi sync here
>  btrfs sub snap ./snapshots/06-06-2023--15:16_PRE_UPGRADE ./@
>  #             /\ note no readonly "-r"
>
>If you stuck with Ubuntu defaults and didn't use the default subvolume
>feature, then subvol=@ is still what grub will boot, and 06-06-2023--
>15:16_PRE_UPGRADE is now the new writable @.
>
>I believe this is the safest method available.  The perfect snapshotting tool
>would make it easier to see the relationship between snapshoted copies of
>subvolumes.
>
>Note that rootfs rollbacks won't fix potentially incompatibly upgraded
>databases in @home (is that rare these days?).  There's not really any good fix
>for that, other than to restore them from backup.  Note that with this method
>you will lose a day's worth of everything in @ that didn't have its own
>subvolume (ie logs, any VMs not in @home, etc).
>
>For the record: If you consult the table above, the default subvolume feature
>makes any subvolume become the user-apparent / of the filesystem, and
>hides anything that's not beneath its tree.
>

Let's have a look at the Suse way:

ha-idg-1:/mnt/domains/.snapshots/06-06-2023 # btrfs sub list /
ID 256 gen 30 top level 5 path @
ID 258 gen 122260 top level 256 path @/var
ID 259 gen 122240 top level 256 path @/usr/local
ID 260 gen 122260 top level 256 path @/tmp
ID 261 gen 41688 top level 256 path @/srv
ID 262 gen 122120 top level 256 path @/opt
ID 263 gen 41688 top level 256 path @/home
ID 264 gen 41688 top level 256 path @/boot/grub2/x86_64-efi
ID 265 gen 113176 top level 256 path @/boot/grub2/i386-pc
ID 266 gen 121824 top level 256 path @/.snapshots
ID 270 gen 41688 top level 258 path @/var/lib/machines
ID 272 gen 55 top level 266 path @/.snapshots/2/snapshot
ID 670 gen 8345 top level 266 path @/.snapshots/37/snapshot

In the top level subvolume (ID 5) they put another subvolume @ .
Under the subvolume @ they put a lot of other subvolumes.
So they don't mount a fs directly to ID 5.
I think @ should be mounted to /.

Let's have a look at the current mounts:
ha-idg-1:/mnt/domains/.snapshots/06-06-2023 # mount|grep btrfs
/dev/mapper/3600508b1001cbde9bbc03f7d66f637a7-part3 on / type btrfs (rw,relatime,space_cache,subvolid=1103,subvol=/@/.snapshots/58/snapshot)
/dev/mapper/3600508b1001cbde9bbc03f7d66f637a7-part3 on /.snapshots type btrfs (rw,relatime,space_cache,subvolid=266,subvol=/@/.snapshots)
/dev/mapper/3600508b1001cbde9bbc03f7d66f637a7-part3 on /boot/grub2/i386-pc type btrfs (rw,relatime,space_cache,subvolid=265,subvol=/@/boot/grub2/i386-pc)
/dev/mapper/3600508b1001cbde9bbc03f7d66f637a7-part3 on /boot/grub2/x86_64-efi type btrfs (rw,relatime,space_cache,subvolid=264,subvol=/@/boot/grub2/x86_64-efi)
/dev/mapper/3600508b1001cbde9bbc03f7d66f637a7-part3 on /home type btrfs (rw,relatime,space_cache,subvolid=263,subvol=/@/home)
/dev/mapper/3600508b1001cbde9bbc03f7d66f637a7-part3 on /opt type btrfs (rw,relatime,space_cache,subvolid=262,subvol=/@/opt)
/dev/mapper/3600508b1001cbde9bbc03f7d66f637a7-part3 on /srv type btrfs (rw,relatime,space_cache,subvolid=261,subvol=/@/srv)
/dev/mapper/3600508b1001cbde9bbc03f7d66f637a7-part3 on /tmp type btrfs (rw,relatime,space_cache,subvolid=260,subvol=/@/t

/ is mounted into a snapshot (i did a rollback with snapper).

Is this setup a good choice ?

Bernd

Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de
Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp, Daniela Sommer (komm.) | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling
Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671

RE: rollback to a snapshot

[Bernd Lentes]

> -----Original Message-----
> From: Nicholas D Steeves <nsteeves@gmail.com>
> Sent: Thursday, June 8, 2023 12:18 AM
> To: Bernd Lentes <bernd.lentes@helmholtz-muenchen.de>; Phillip Susi
> <phill@thesusis.net>
> Cc: Andrei Borzenkov <arvidjaar@gmail.com>; linux-btrfs@vger.kernel.org
> Subject: RE: rollback to a snapshot
>
> If I remember correctly, Ubuntu looks something like this
>
> sudo btrfs sub list / -t
> ID      gen     top level       path
> --      ---     ---------       ----
> 347     477642  5               @
> 348     476215  5               @home
>
> which is
>
> subvolid=5
>          |_ @
>          |_ @home
>
> So you have something like
> 1. subvolid=5
> 2.          |_ @
> 3.          |_ @home
> 4.          |_ snapshots
> 5.                     |_ 06-06-2023--15:16_PRE_UPGRADE
> 6.                     |_ 06-06-2023
>
> subvolid=5 is always the "real root" I think Andrei Borzenkov is calling it to
> mitigate the confusion resulting from potentially having used the default
> subvolume feature, which in my opinion should never be used, because it will
> confuse the bootloader.  A default Ubuntu installation will have "subvol=@" in the
> bootloader, which means all that you need to do is move your snapshot into place
> (position #2 in the above table)
>
> So I think you're going to do something like the following: Mount the true /
> (position #1) or subvolid=5 to /mnt/btrfs, then
>
>   cd /mnt/btrfs/snapshots
>   btrfs sub snap -r ../@ ./@_broken-upgrade  # note readonly "-r"
>   cd ..
>   # I still do a subvol sync and fi sync here
>   btrfs sub del -c ./@
>   # I still do a subvol sync and fi sync here
>   btrfs sub snap ./snapshots/06-06-2023--15:16_PRE_UPGRADE ./@
>   #             /\ note no readonly "-r"
>
Hi Nicholas,

does that mean I never can delete snapshots/06-06-2023--15:16_PRE_UPGRADE ?

Bernd

Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de
Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp, Daniela Sommer (komm.) | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling
Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671

RE: rollback to a snapshot

[Bernd Lentes]

Dear Andrei,

>-----Original Message-----
>From: Andrei Borzenkov <arvidjaar@gmail.com>
>Sent: Wednesday, June 7, 2023 6:37 AM
>To: Bernd Lentes <bernd.lentes@helmholtz-muenchen.de>; linux-
>btrfs@vger.kernel.org
>Subject: Re: rollback to a snapshot
>
>
>You cannot rename or move because you cannot "rename" or "move"
>subvolume to become filesystem top level. Which is one reason why you
>should never use btrfs top level subvolume if you plan to use snapshots and be
>able to revert. Actually you should probably never use btrfs top level
>subvolume except as container for other subvolumes. Period.
>
>You could simply rsync from snapshot to revert the content of your root.
>It would be the most simple way (it will probably increase space consumption
>slightly). Or you could boot linux with subvol=... rootfs option - there is no
>need to actually change default subvolume.
>
>If you chose "use different subvolume as root" route, keep in mind that
>
>- you will likely need to reinstall bootloader because paths will change.
>
>- any subvolume below / like /.snapshots will be invisible from your booted
>system and you will need to explicitly mount it if you need to access its
>content.
>
>- as mentioned already, do not flip read-only snapshot to read-write, rather
>create new writable clone. I would also chose some different path leaving
>/.snapshots for snapshots. It does not matter in this case (at least, with the
>information provided so far) but it is good habit to develop.
>
>- after you booted new root subvolume you will have the old root content in
>top level which cannot be easily deleted because it is not subvolume (and it
>will be invisible as well). You will have to clean it up manually, and be careful
>to not delete your subvolumes doing it :)

Another scenario:
I have some virtual machines for which i create a snapshot each night with BTRFS.
The snapshots reside in another directory. How do i revert to a snapshot? Switch the virtual machine off and just copy
the snapshot over the original ?

Bernd

Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de
Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp, Daniela Sommer (komm.) | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling
Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671

Re: rollback to a snapshot

[Andrei Borzenkov]

On 07.06.2023 13:45, Bernd Lentes wrote:
> I have some virtual machines for which i create a snapshot each night with BTRFS.
> The snapshots reside in another directory. How do i revert to a snapshot? Switch the virtual machine off and just copy
> the snapshot over the original ?
> 

It is rather unclear what you mean. VM disk images are on host btrfs and 
you create snapshot on host that contains those images? Or you have VM 
which use btrfs in their guest system?

RE: rollback to a snapshot

[Bernd Lentes]

>-----Original Message-----
>From: Andrei Borzenkov <arvidjaar@gmail.com>
>Sent: Wednesday, June 7, 2023 6:36 PM
>To: Bernd Lentes <bernd.lentes@helmholtz-muenchen.de>; linux-
>btrfs@vger.kernel.org
>Subject: Re: rollback to a snapshot

>It is rather unclear what you mean. VM disk images are on host btrfs and you
>create snapshot on host that contains those images? Or you have VM which
>use btrfs in their guest system?

VM images are on host BTRFS and i create the snapshots from them on the host.

Bernd


Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de
Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp, Daniela Sommer (komm.) | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling
Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671

Re: rollback to a snapshot

[remi]

On Wed, Jun 7, 2023, at 3:37 PM, Bernd Lentes wrote:

> VM images are on host BTRFS and i create the snapshots from them on the host.
>

So there are a couple choices on how you want to restore.

You can make a new snapshot from your 'good' snapshot.  This will roll back the entire subvolume.

mv /subvolume /subvolume-bad
btrfs sub snap /path-to-snapshot /subvolume

When you are confident that you want to delete the replaced subvolume

btrfs sub del /subvolume-bad

*Note*. you might want to first rename sobvolume-bad to something safe, like mv subvolume-bad deleteme; btrfs sub del deleteme
That way, is not not possible to accidentally hit <Enter> when you have btrfs sub del /subvolume on the command line.


Alternatively, you can just copy the disk image file from your snapshot into your active subvolume

cp -a --reflink=always /path-to-snapshot/vm.img /subvolume/vm.img

RE: rollback to a snapshot

[Bernd Lentes]

>-----Original Message-----
>From: remi@georgianit.com <remi@georgianit.com>
>Sent: Wednesday, June 7, 2023 9:46 PM
>To: Bernd Lentes <bernd.lentes@helmholtz-muenchen.de>; linux-
>btrfs@vger.kernel.org
>Subject: Re: rollback to a snapshot
>Alternatively, you can just copy the disk image file from your snapshot into
>your active subvolume
>
>cp -a --reflink=always /path-to-snapshot/vm.img /subvolume/vm.img

That's what i did.

Bernd

Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de
Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp, Daniela Sommer (komm.) | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling
Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671

RE: rollback to a snapshot

[Bernd Lentes]

>-----Original Message-----
>From: remi@georgianit.com <remi@georgianit.com>
>Sent: Wednesday, June 7, 2023 9:46 PM
>To: Bernd Lentes <bernd.lentes@helmholtz-muenchen.de>; linux-
>btrfs@vger.kernel.org
>Subject: Re: rollback to a snapshot
>Alternatively, you can just copy the disk image file from your snapshot into
>your active subvolume
>
>cp -a --reflink=always /path-to-snapshot/vm.img /subvolume/vm.img

I didn't know "--reflink". The help says:
" When --reflink[=always] is specified, perform a lightweight copy, where the
data blocks are copied only when modified.  If this is not possible the copy
fails, or if --reflink=auto is specified, fall back to a standard copy.
Use --reflink=never to ensure a standard copy is performed.".

Is that independent from the fs ? That seems to be a kind of a snapshot.
Am i right ?

Bernd

Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de
Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp, Daniela Sommer (komm.) | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling
Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671

Re: rollback to a snapshot

[Roman Mamedov]

On Wed, 7 Jun 2023 20:14:27 +0000
Bernd Lentes <bernd.lentes@helmholtz-muenchen.de> wrote:

> I didn't know "--reflink". The help says:
> " When --reflink[=always] is specified, perform a lightweight copy, where the
> data blocks are copied only when modified.  If this is not possible the copy
> fails, or if --reflink=auto is specified, fall back to a standard copy.
> Use --reflink=never to ensure a standard copy is performed.".
> 
> Is that independent from the fs ? That seems to be a kind of a snapshot.
> Am i right ?

Not all filesystems support reflinks.

Btrfs and XFS support it, but for XFS this has to be enabled at FS creation
time (IIRC recent mkfs enables it by default). Ext4, ReiserFS and JFS do not.

For XFS it is a way to have the best of the both worlds: have a filesystem
that is well regarded for performance, but also gain a degree of the CoW
ability that Btrfs is liked for, e.g. allowing instant snapshots of VM images,
including running ones with a point-in-time full image consistency --
unthinkable with traditional copying.

-- 
With respect,
Roman

RE: rollback to a snapshot

[Bernd Lentes]

>-----Original Message-----
>From: Roman Mamedov <rm@romanrm.net>
>Sent: Wednesday, June 7, 2023 10:48 PM
>To: Bernd Lentes <bernd.lentes@helmholtz-muenchen.de>
>Cc: remi@georgianit.com; linux-btrfs@vger.kernel.org
>Subject: Re: rollback to a snapshot
>
>On Wed, 7 Jun 2023 20:14:27 +0000
>Bernd Lentes <bernd.lentes@helmholtz-muenchen.de> wrote:
>
>Not all filesystems support reflinks.
>
>Btrfs and XFS support it, but for XFS this has to be enabled at FS creation time
>(IIRC recent mkfs enables it by default). Ext4, ReiserFS and JFS do not.

Is there someone who uses ReiserFS nowadays ?
OCFS2 also knows reflinks.

>For XFS it is a way to have the best of the both worlds: have a filesystem that is
>well regarded for performance, but also gain a degree of the CoW ability that
>Btrfs is liked for, e.g. allowing instant snapshots of VM images, including
>running ones with a point-in-time full image consistency -- unthinkable with
>traditional copying.

Bernd

Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de
Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp, Daniela Sommer (komm.) | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling
Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671