Mounting multiple versions/snapshots/images at the same time (was: Re: Another bcachefs version downgrade bug)

[Martin Steigerwald <martin@lichtvoll.de>]

Hi Carl and Kent,

It is a topic change, so please ping if you like to be dropped from CC.

Thanks for the constructive discussion here. I pick out an aspect I am 
quite interested in.

I am extending this aspect to mounting different snapshots at the same time 
in case you, Carl, did not mean this.

Kent Overstreet - 21.10.24, 03:15:47 MESZ:
> I'll give you an example. More than a year ago I raised the issue that
> 
> > bcachefs does not allow multiple versions/images of a filesystem to be
> > mounted at the same time. I pointed out that there are several
> > professional workflows that require this (forensics, auditing, etc)
> > and non-professional workflows too (retrieving a previous version of a
> > file from an LVM snapshot, etc). I pointed out that other modern
> > filesystems allow it and argued that it is a needed feature. You
> > disagreed that it is that important and have never (to date)
> > implemented the feature. This causes me to have to create special-case
> > workarounds specific to bcachefs which operate differently than what I
> > do for every other filesystem. Despite this extra work and
> > inconvenience for me I have never brought it up again, never nagged
> > you and never demanded that you listen to me and change your mind. I
> > made my pitch, you made your decision and I accept your decision even
> > if I don't like it.
> 
> If I came across as strongly disagreeing on that issue, I apologize.
> 
> On that issue, it's just that it's a really problematic feature for a
> multi device filesystem - we have to have a unique identifier for
> identifying the filesystem, separate from the block device, and if it's
> not actually unique - what do we do?

There are several different variants of this and I bet it is important to 
clarify exactly what is meant here. At least these two come to my mind:

1) Mount a block-for-block clone of the same filesystem another time.

2) Mount a snapshot of one filesystem on one block device to a different 
location in filesystem tree.

Carl, what is the one refering to? If you mean a third thing, please 
elaborate.

I do the second one with BTRFS in combination with the default subvolume 
feature in order to hide away snapshots.

First I create a subvolume where I can create snaphots and one for the 
actual filesystem contents. Then I tell the filesystem to use the snapshot 
for the filesystem contents as default for mounting. And then I additional 
mount the subvolume that contains the snapshots to a different location. 
Something like this:

/dev/nvme/system / btrfs lazytime,compress=zstd,discard=async   0       0

/dev/nvme/system /snap/system  btrfs   subvol=snap   0       0

(in this case BCacheFS on LVM as I wanted to have the flexibility to test 
out different filesystems)

This makes it easy for me to exclude all snapshots from backup operations 
as I can do top level snapshots of the filesystem contents and "hide" them 
away in a subvolume (means sub directory in filesystem tree) of my choice.

I still use rsync for backups as it has stood the test of time. I could 
probably switch to BTRFS send/recieve or a similar functionality in 
BCacheFS. With the added benefit of way better handling of renames.

> Even for single device filesystems it's a problem, because some of our
> userspace interfaces use that same unique identifier (sysfs, debugfs)
> and a single device filesystem can become a multidevice filesystem at
> any time.

I am not sure whether case 2 would already be possible with BCacheFS. If 
not, this would hold me back from switching over completely. Of course I 
can try to rework my setup, but I'd prefer not to. Unless someone can 
convince me of a good technical reason to do it.

> So it's not impossible, but genuinely messy, and it's the kind of thing
> I could see leading to landmines later, which makes me not particularly
> eager to touch it. But if it keeps coming up, I may end up giving it
> more study in the future.

Well it would be important to find a somewhat clean approach about that. 
I'd rather not have a feature than a feature that could be unreliable due 
to it being messy.

I do not know how other filesystems do either 1 or 2 of the above variants. 
All I can say is that with variant 2 on BTRFS I did not see an issue so 
far.

Best,
-- 
Martin