[PATCH 0/2] btrfs: support cloned-device mount capability

[PATCH 0/2] btrfs: support cloned-device mount capability

[Anand Jain]

Guilherme's previous work [1] aimed at the mounting of cloned devices
using a superblock flag SINGLE_DEV during mkfs.
 [1] https://lore.kernel.org/linux-btrfs/20230831001544.3379273-1-gpiccoli@igalia.com/

Building upon this work, here is in-memory only approach. As it mounts
we determine if the same fsid is already mounted if then we generate a
random temp fsid which shall be used the mount, in-memory only not
written to the disk. We distinguish device by devt.

Mount option / superblock flag:
-------------------------------
 These patches show we don't have to limit the single-device / temp_fsid
capability with a mount option or a superblock flag from the btrfs
internals pov. However, if necessary from the user's perspective,
we can add them later on top of this patch. I've prepared a mount option
-o temp_fsid patch, but I'm not included at this time. As most of the
tests was without it.

Compatible with other features that may be affected:
----------------------------------------------------
 Multi device:
    A btrfs filesytem on a single device can be copied using dd and
    mounted simlutaneously. However, a multi device btrfs copied using
    dd and trying to mount simlutaneously is forced to fail:

      mount: /btrfs1: mount(2) system call failed: File exists.

 Send and receive:
    Quick tests shows send and receive between two single devices with
    the same fsid mounted on the _same_ host works!.
    (Also, the receive-mnt can receive from multiple senders as long as
    conflits are managed externally. ;-).)

 Replace: 
     Works fine.

btrfs-progs:
------------
 btrfs-progs needs to be updated to support the commands such as

	btrfs filesystem show

 when devices are not mounted. So the device list is not based on
 the fisd any more.

Testing:
-------
 This patch has been under testing for some time. The challenge is to get
 the fstests to test this reasonably well.

 As of now, this patch runs fine on a large set of fstests test cases
 using a custom-built mkfs.btrfs with the -U option and a new -P option
 to copy the device FSID and UUID from the TEST_DEV to the SCRATCH_DEV
 at the scratch_mkfs time. For example:

  Config file:

     config_fsid=$(btrfs in dump-super $TEST_DEV | grep -E ^fsid | \
							awk '{print $2}')
     config_uuid=$(btrfs in dump-super $TEST_DEV | \
				grep -E ^dev_item.uuid | awk '{print $2}')
     MKFS_OPTIONS="-U $config_fsid -P $config_uuid"

 This configuration option ensures that both TEST_DEV and SCRATCH_DEV will
 have the same FSID and device UUID while still applying test-specific
 scratch mkfs options.

Mkfs.btrfs:
-----------
 mkfs.btrfs needs to be updated to support the -P option for testing only.

   btrfs-progs: allow duplicate fsid for single device
   btrfs-progs: add mkfs -P option for dev_uuid

Anand Jain (2):
  btrfs: add helper function find_fsid_by_disk
  btrfs: support cloned-device mount capability

 fs/btrfs/disk-io.c |  3 +-
 fs/btrfs/volumes.c | 75 +++++++++++++++++++++++++++++++++++++++++++---
 fs/btrfs/volumes.h |  2 ++
 3 files changed, 75 insertions(+), 5 deletions(-)

-- 
2.38.1

[PATCH 1/2] btrfs: add helper function find_fsid_by_disk

[Anand Jain]

In preparation for adding support to mount multiple single-disk
btrfs filesystems with the same FSID, wrap find_fsid() into
find_fsid_by_disk().

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 fs/btrfs/volumes.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 298e5885ed06..39b5bc2521fb 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -553,6 +553,20 @@ static int btrfs_free_stale_devices(dev_t devt, struct btrfs_device *skip_device
 	return ret;
 }
 
+static struct btrfs_fs_devices *find_fsid_by_disk(
+					struct btrfs_super_block *disk_super)
+{
+	struct btrfs_fs_devices *fsid_fs_devices;
+	bool has_metadata_uuid = (btrfs_super_incompat_flags(disk_super) &
+					BTRFS_FEATURE_INCOMPAT_METADATA_UUID);
+
+	/* Find the fs_device by the usual method if found use it */
+	fsid_fs_devices = find_fsid(disk_super->fsid, has_metadata_uuid ?
+					disk_super->metadata_uuid : NULL);
+
+	return fsid_fs_devices;
+}
+
 /*
  * This is only used on mount, and we are protected from competing things
  * messing with our fs_devices by the uuid_mutex, thus we do not need the
@@ -673,10 +687,7 @@ static noinline struct btrfs_device *device_list_add(const char *path,
 		return ERR_PTR(error);
 	}
 
-	if (has_metadata_uuid)
-		fs_devices = find_fsid(disk_super->fsid, disk_super->metadata_uuid);
-	else
-		fs_devices = find_fsid(disk_super->fsid, NULL);
+	fs_devices = find_fsid_by_disk(disk_super);
 
 	if (!fs_devices) {
 		fs_devices = alloc_fs_devices(disk_super->fsid);
-- 
2.38.1

Re: [PATCH 1/2] btrfs: add helper function find_fsid_by_disk

[David Sterba]

On Thu, Sep 28, 2023 at 09:09:46AM +0800, Anand Jain wrote:
> In preparation for adding support to mount multiple single-disk
> btrfs filesystems with the same FSID, wrap find_fsid() into
> find_fsid_by_disk().
> 
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> ---
>  fs/btrfs/volumes.c | 19 +++++++++++++++----
>  1 file changed, 15 insertions(+), 4 deletions(-)
> 
> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> index 298e5885ed06..39b5bc2521fb 100644
> --- a/fs/btrfs/volumes.c
> +++ b/fs/btrfs/volumes.c
> @@ -553,6 +553,20 @@ static int btrfs_free_stale_devices(dev_t devt, struct btrfs_device *skip_device
>  	return ret;
>  }
>  
> +static struct btrfs_fs_devices *find_fsid_by_disk(

A minor thing here, I think we don't want to use "disk" in the
identifiers, it's been converted to 'device' where possible but there
are still some references that could be related to on-disk structures so
these can't be changed. In new API I'd prefer 'device'.

Re: [PATCH 1/2] btrfs: add helper function find_fsid_by_disk

[Anand Jain]

On 02/10/2023 19:45, David Sterba wrote:
> On Thu, Sep 28, 2023 at 09:09:46AM +0800, Anand Jain wrote:
>> In preparation for adding support to mount multiple single-disk
>> btrfs filesystems with the same FSID, wrap find_fsid() into
>> find_fsid_by_disk().
>>
>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>> ---
>>   fs/btrfs/volumes.c | 19 +++++++++++++++----
>>   1 file changed, 15 insertions(+), 4 deletions(-)
>>
>> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
>> index 298e5885ed06..39b5bc2521fb 100644
>> --- a/fs/btrfs/volumes.c
>> +++ b/fs/btrfs/volumes.c
>> @@ -553,6 +553,20 @@ static int btrfs_free_stale_devices(dev_t devt, struct btrfs_device *skip_device
>>   	return ret;
>>   }
>>   
>> +static struct btrfs_fs_devices *find_fsid_by_disk(
> 
> A minor thing here, I think we don't want to use "disk" in the
> identifiers, it's been converted to 'device' where possible but there
> are still some references that could be related to on-disk structures so
> these can't be changed. In new API I'd prefer 'device'.

yeah. It should be device.

Thanks, Anand

[PATCH 2/2] btrfs: support cloned-device mount capability

[Anand Jain]

Guilherme's previous work [1] aimed at the mounting of cloned devices
using a superblock flag SINGLE_DEV during mkfs.
 [1] https://lore.kernel.org/linux-btrfs/20230831001544.3379273-1-gpiccoli@igalia.com/

Building upon this work, here is in memory only approach. As it mounts
we determine if the same fsid is already mounted if then we generate a
random temp fsid which shall be used the mount, in memory only not
written to the disk. We distinguish devices by devt.

Example:
  $ fallocate -l 300m ./disk1.img :0
  $ mkfs.btrfs -f ./disk1.img :0
  $ cp ./disk1.img ./disk2.img :0
  $ cp ./disk1.img ./disk3.img :0
  $ mount -o loop ./disk1.img /btrfs :0
  $ mount -o ./disk2.img /btrfs1 :0
  $ mount -o ./disk3.img /btrfs2 :0

  $ btrfs fi show -m :0
  Label: none  uuid: 4a212b48-1bec-46a5-938a-783c8c1f0b02
	Total devices 1 FS bytes used 144.00KiB
	devid    1 size 300.00MiB used 88.00MiB path /dev/loop0

  Label: none  uuid: adabf2fe-5515-4ad0-95b4-7b1609218c16
	Total devices 1 FS bytes used 144.00KiB
	devid    1 size 300.00MiB used 88.00MiB path /dev/loop1

  Label: none  uuid: 1d77d0df-7d92-439e-adbd-20b9b86fdedb
	Total devices 1 FS bytes used 144.00KiB
	devid    1 size 300.00MiB used 88.00MiB path /dev/loop2

Co-developed-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
---

 fs/btrfs/disk-io.c |  3 ++-
 fs/btrfs/volumes.c | 62 +++++++++++++++++++++++++++++++++++++++++++---
 fs/btrfs/volumes.h |  2 ++
 3 files changed, 63 insertions(+), 4 deletions(-)

diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index a970da7263b3..04f57d8368c8 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -2399,7 +2399,8 @@ int btrfs_validate_super(struct btrfs_fs_info *fs_info,
 		ret = -EINVAL;
 	}
 
-	if (memcmp(fs_info->fs_devices->fsid, sb->fsid, BTRFS_FSID_SIZE) != 0) {
+	if (!fs_info->fs_devices->temp_fsid &&
+	    memcmp(fs_info->fs_devices->fsid, sb->fsid, BTRFS_FSID_SIZE) != 0) {
 		btrfs_err(fs_info,
 		"superblock fsid doesn't match fsid of fs_devices: %pU != %pU",
 			  sb->fsid, fs_info->fs_devices->fsid);
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 39b5bc2521fb..ea4a110d7753 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -554,17 +554,64 @@ static int btrfs_free_stale_devices(dev_t devt, struct btrfs_device *skip_device
 }
 
 static struct btrfs_fs_devices *find_fsid_by_disk(
-					struct btrfs_super_block *disk_super)
+					struct btrfs_super_block *disk_super,
+					dev_t devt, bool *same_fsid_diff_dev)
 {
 	struct btrfs_fs_devices *fsid_fs_devices;
+	struct btrfs_fs_devices *devt_fs_devices;
 	bool has_metadata_uuid = (btrfs_super_incompat_flags(disk_super) &
 					BTRFS_FEATURE_INCOMPAT_METADATA_UUID);
+	bool found_by_devt = false;
 
 	/* Find the fs_device by the usual method if found use it */
 	fsid_fs_devices = find_fsid(disk_super->fsid, has_metadata_uuid ?
 					disk_super->metadata_uuid : NULL);
 
-	return fsid_fs_devices;
+	/* The temp_fsid feature is supported only with single device btrfs */
+	if (btrfs_super_num_devices(disk_super) != 1)
+		return fsid_fs_devices;
+
+	/* Try to find a fs_devices by matching devt */
+	list_for_each_entry(devt_fs_devices, &fs_uuids, fs_list) {
+		struct btrfs_device *device;
+
+		list_for_each_entry(device, &devt_fs_devices->devices,
+				    dev_list) {
+			if (device->devt == devt) {
+				found_by_devt = true;
+				break;
+			}
+		}
+		if (found_by_devt)
+			break;
+	}
+
+	if (found_by_devt) {
+		/* existing device */
+		if (fsid_fs_devices == NULL) {
+			if (devt_fs_devices->opened == 0) {
+				/* stale device */
+				return NULL;
+			} else {
+				/* temp_fsid is mounting a subvol */
+				return devt_fs_devices;
+			}
+		} else {
+			/* regular or temp_fsid device mounting a subvol */
+			return devt_fs_devices;
+		}
+	} else {
+		/* new device */
+		if (fsid_fs_devices == NULL) {
+			return NULL;
+		} else {
+			/* sb::fsid is already used create a new temp_fsid */
+			*same_fsid_diff_dev = true;
+			return NULL;
+		}
+	}
+
+	/* Not reached */
 }
 
 /*
@@ -670,6 +717,7 @@ static noinline struct btrfs_device *device_list_add(const char *path,
 	u64 devid = btrfs_stack_device_id(&disk_super->dev_item);
 	dev_t path_devt;
 	int error;
+	bool same_fsid_diff_dev = false;
 	bool has_metadata_uuid = (btrfs_super_incompat_flags(disk_super) &
 		BTRFS_FEATURE_INCOMPAT_METADATA_UUID);
 
@@ -687,7 +735,8 @@ static noinline struct btrfs_device *device_list_add(const char *path,
 		return ERR_PTR(error);
 	}
 
-	fs_devices = find_fsid_by_disk(disk_super);
+	fs_devices = find_fsid_by_disk(disk_super, path_devt,
+				       &same_fsid_diff_dev);
 
 	if (!fs_devices) {
 		fs_devices = alloc_fs_devices(disk_super->fsid);
@@ -698,6 +747,13 @@ static noinline struct btrfs_device *device_list_add(const char *path,
 		if (IS_ERR(fs_devices))
 			return ERR_CAST(fs_devices);
 
+		if (same_fsid_diff_dev) {
+			generate_random_uuid(fs_devices->fsid);
+			fs_devices->temp_fsid = true;
+			pr_info("BTRFS: device %s using temp fsid %pU\n",
+				path, fs_devices->fsid);
+		}
+
 		mutex_lock(&fs_devices->device_list_mutex);
 		list_add(&fs_devices->fs_list, &fs_uuids);
 
diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
index e485e6a3e52c..5921fdd3dd90 100644
--- a/fs/btrfs/volumes.h
+++ b/fs/btrfs/volumes.h
@@ -364,6 +364,8 @@ struct btrfs_fs_devices {
 	bool discardable;
 	/* The filesystem is a seed filesystem. */
 	bool seeding;
+	/* The mount need to use a randomly generated fsid. */
+	bool temp_fsid;
 
 	struct btrfs_fs_info *fs_info;
 	/* sysfs kobjects */
-- 
2.38.1

Re: [PATCH 2/2] btrfs: support cloned-device mount capability

[David Sterba]

On Thu, Sep 28, 2023 at 09:09:47AM +0800, Anand Jain wrote:
> Guilherme's previous work [1] aimed at the mounting of cloned devices
> using a superblock flag SINGLE_DEV during mkfs.
>  [1] https://lore.kernel.org/linux-btrfs/20230831001544.3379273-1-gpiccoli@igalia.com/
> 
> Building upon this work, here is in memory only approach. As it mounts
> we determine if the same fsid is already mounted if then we generate a
> random temp fsid which shall be used the mount, in memory only not
> written to the disk. We distinguish devices by devt.
> 
> Example:
>   $ fallocate -l 300m ./disk1.img :0
>   $ mkfs.btrfs -f ./disk1.img :0
>   $ cp ./disk1.img ./disk2.img :0
>   $ cp ./disk1.img ./disk3.img :0
>   $ mount -o loop ./disk1.img /btrfs :0
>   $ mount -o ./disk2.img /btrfs1 :0
>   $ mount -o ./disk3.img /btrfs2 :0

I'm confused what the ":0" are supposed to mean, is it some artifact of
your editor?

> 
>   $ btrfs fi show -m :0
>   Label: none  uuid: 4a212b48-1bec-46a5-938a-783c8c1f0b02
> 	Total devices 1 FS bytes used 144.00KiB
> 	devid    1 size 300.00MiB used 88.00MiB path /dev/loop0
> 
>   Label: none  uuid: adabf2fe-5515-4ad0-95b4-7b1609218c16
> 	Total devices 1 FS bytes used 144.00KiB
> 	devid    1 size 300.00MiB used 88.00MiB path /dev/loop1
> 
>   Label: none  uuid: 1d77d0df-7d92-439e-adbd-20b9b86fdedb
> 	Total devices 1 FS bytes used 144.00KiB
> 	devid    1 size 300.00MiB used 88.00MiB path /dev/loop2
> 
> Co-developed-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
> Signed-off-by: Anand Jain <anand.jain@oracle.com>

Re: [PATCH 2/2] btrfs: support cloned-device mount capability

[Anand Jain]

On 02/10/2023 20:57, David Sterba wrote:
> On Thu, Sep 28, 2023 at 09:09:47AM +0800, Anand Jain wrote:
>> Guilherme's previous work [1] aimed at the mounting of cloned devices
>> using a superblock flag SINGLE_DEV during mkfs.

>>   [1] https://lore.kernel.org/linux-btrfs/20230831001544.3379273-1-gpiccoli@igalia.com/

The link should go at SOB with prefix Link:.

>>
>> Building upon this work, here is in memory only approach. As it mounts
>> we determine if the same fsid is already mounted if then we generate a
>> random temp fsid which shall be used the mount, in memory only not
>> written to the disk. We distinguish devices by devt.
>>
>> Example:
>>    $ fallocate -l 300m ./disk1.img :0
>>    $ mkfs.btrfs -f ./disk1.img :0
>>    $ cp ./disk1.img ./disk2.img :0
>>    $ cp ./disk1.img ./disk3.img :0
>>    $ mount -o loop ./disk1.img /btrfs :0
>>    $ mount -o ./disk2.img /btrfs1 :0
>>    $ mount -o ./disk3.img /btrfs2 :0
> 
> I'm confused what the ":0" are supposed to mean, is it some artifact of
> your editor?

Oh, sorry for the confusion. It is the return status of the command.
I have some local scripts to collect the output. Could you pls remove 
":0", or I'll if there is a reroll.

Thanks, Anand

> 
>>
>>    $ btrfs fi show -m :0
>>    Label: none  uuid: 4a212b48-1bec-46a5-938a-783c8c1f0b02
>> 	Total devices 1 FS bytes used 144.00KiB
>> 	devid    1 size 300.00MiB used 88.00MiB path /dev/loop0
>>
>>    Label: none  uuid: adabf2fe-5515-4ad0-95b4-7b1609218c16
>> 	Total devices 1 FS bytes used 144.00KiB
>> 	devid    1 size 300.00MiB used 88.00MiB path /dev/loop1
>>
>>    Label: none  uuid: 1d77d0df-7d92-439e-adbd-20b9b86fdedb
>> 	Total devices 1 FS bytes used 144.00KiB
>> 	devid    1 size 300.00MiB used 88.00MiB path /dev/loop2
>>
>> Co-developed-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
>> Signed-off-by: Anand Jain <anand.jain@oracle.com>

Re: [PATCH 0/2] btrfs: support cloned-device mount capability

[Anand Jain]

   Gentle ping on any comments

Thanks, Anand


On 28/09/2023 09:09, Anand Jain wrote:
> Guilherme's previous work [1] aimed at the mounting of cloned devices
> using a superblock flag SINGLE_DEV during mkfs.
>   [1] https://lore.kernel.org/linux-btrfs/20230831001544.3379273-1-gpiccoli@igalia.com/
> 
> Building upon this work, here is in-memory only approach. As it mounts
> we determine if the same fsid is already mounted if then we generate a
> random temp fsid which shall be used the mount, in-memory only not
> written to the disk. We distinguish device by devt.
> 
> Mount option / superblock flag:
> -------------------------------
>   These patches show we don't have to limit the single-device / temp_fsid
> capability with a mount option or a superblock flag from the btrfs
> internals pov. However, if necessary from the user's perspective,
> we can add them later on top of this patch. I've prepared a mount option
> -o temp_fsid patch, but I'm not included at this time. As most of the
> tests was without it.
> 
> Compatible with other features that may be affected:
> ----------------------------------------------------
>   Multi device:
>      A btrfs filesytem on a single device can be copied using dd and
>      mounted simlutaneously. However, a multi device btrfs copied using
>      dd and trying to mount simlutaneously is forced to fail:
> 
>        mount: /btrfs1: mount(2) system call failed: File exists.
> 
>   Send and receive:
>      Quick tests shows send and receive between two single devices with
>      the same fsid mounted on the _same_ host works!.
>      (Also, the receive-mnt can receive from multiple senders as long as
>      conflits are managed externally. ;-).)
> 
>   Replace:
>       Works fine.
> 
> btrfs-progs:
> ------------
>   btrfs-progs needs to be updated to support the commands such as
> 
> 	btrfs filesystem show
> 
>   when devices are not mounted. So the device list is not based on
>   the fisd any more.
> 
> Testing:
> -------
>   This patch has been under testing for some time. The challenge is to get
>   the fstests to test this reasonably well.
> 
>   As of now, this patch runs fine on a large set of fstests test cases
>   using a custom-built mkfs.btrfs with the -U option and a new -P option
>   to copy the device FSID and UUID from the TEST_DEV to the SCRATCH_DEV
>   at the scratch_mkfs time. For example:
> 
>    Config file:
> 
>       config_fsid=$(btrfs in dump-super $TEST_DEV | grep -E ^fsid | \
> 							awk '{print $2}')
>       config_uuid=$(btrfs in dump-super $TEST_DEV | \
> 				grep -E ^dev_item.uuid | awk '{print $2}')
>       MKFS_OPTIONS="-U $config_fsid -P $config_uuid"
> 
>   This configuration option ensures that both TEST_DEV and SCRATCH_DEV will
>   have the same FSID and device UUID while still applying test-specific
>   scratch mkfs options.
> 
> Mkfs.btrfs:
> -----------
>   mkfs.btrfs needs to be updated to support the -P option for testing only.
> 
>     btrfs-progs: allow duplicate fsid for single device
>     btrfs-progs: add mkfs -P option for dev_uuid
> 
> Anand Jain (2):
>    btrfs: add helper function find_fsid_by_disk
>    btrfs: support cloned-device mount capability
> 
>   fs/btrfs/disk-io.c |  3 +-
>   fs/btrfs/volumes.c | 75 +++++++++++++++++++++++++++++++++++++++++++---
>   fs/btrfs/volumes.h |  2 ++
>   3 files changed, 75 insertions(+), 5 deletions(-)
> 

Re: [PATCH 0/2] btrfs: support cloned-device mount capability

[David Sterba]

On Thu, Sep 28, 2023 at 09:09:45AM +0800, Anand Jain wrote:
> Guilherme's previous work [1] aimed at the mounting of cloned devices
> using a superblock flag SINGLE_DEV during mkfs.
>  [1] https://lore.kernel.org/linux-btrfs/20230831001544.3379273-1-gpiccoli@igalia.com/
> 
> Building upon this work, here is in-memory only approach. As it mounts
> we determine if the same fsid is already mounted if then we generate a
> random temp fsid which shall be used the mount, in-memory only not
> written to the disk. We distinguish device by devt.
> 
> Mount option / superblock flag:
> -------------------------------
>  These patches show we don't have to limit the single-device / temp_fsid
> capability with a mount option or a superblock flag from the btrfs
> internals pov. However, if necessary from the user's perspective,
> we can add them later on top of this patch. I've prepared a mount option
> -o temp_fsid patch, but I'm not included at this time. As most of the
> tests was without it.
> 
> Compatible with other features that may be affected:
> ----------------------------------------------------
>  Multi device:
>     A btrfs filesytem on a single device can be copied using dd and
>     mounted simlutaneously. However, a multi device btrfs copied using
>     dd and trying to mount simlutaneously is forced to fail:
> 
>       mount: /btrfs1: mount(2) system call failed: File exists.
> 
>  Send and receive:
>     Quick tests shows send and receive between two single devices with
>     the same fsid mounted on the _same_ host works!.

Does it depend if the filesystem remains mounted for the whole
time? So if there's an unmount, mount again with a temp-fsid, will
the receive still work?

>     (Also, the receive-mnt can receive from multiple senders as long as
>     conflits are managed externally. ;-).)
> 
>  Replace: 
>      Works fine.
> 
> btrfs-progs:
> ------------
>  btrfs-progs needs to be updated to support the commands such as
> 
> 	btrfs filesystem show
> 
>  when devices are not mounted. So the device list is not based on
>  the fisd any more.
> 
> Testing:
> -------
>  This patch has been under testing for some time. The challenge is to get
>  the fstests to test this reasonably well.
> 
>  As of now, this patch runs fine on a large set of fstests test cases
>  using a custom-built mkfs.btrfs with the -U option and a new -P option
>  to copy the device FSID and UUID from the TEST_DEV to the SCRATCH_DEV
>  at the scratch_mkfs time. For example:
> 
>   Config file:
> 
>      config_fsid=$(btrfs in dump-super $TEST_DEV | grep -E ^fsid | \
> 							awk '{print $2}')
>      config_uuid=$(btrfs in dump-super $TEST_DEV | \
> 				grep -E ^dev_item.uuid | awk '{print $2}')
>      MKFS_OPTIONS="-U $config_fsid -P $config_uuid"
> 
>  This configuration option ensures that both TEST_DEV and SCRATCH_DEV will
>  have the same FSID and device UUID while still applying test-specific
>  scratch mkfs options.
> 
> Mkfs.btrfs:
> -----------
>  mkfs.btrfs needs to be updated to support the -P option for testing only.
> 
>    btrfs-progs: allow duplicate fsid for single device
>    btrfs-progs: add mkfs -P option for dev_uuid
> 
> Anand Jain (2):
>   btrfs: add helper function find_fsid_by_disk
>   btrfs: support cloned-device mount capability

Added to misc-next, thanks.

Re: [PATCH 0/2] btrfs: support cloned-device mount capability

[Anand Jain]

On 02/10/2023 21:00, David Sterba wrote:
> On Thu, Sep 28, 2023 at 09:09:45AM +0800, Anand Jain wrote:
>> Guilherme's previous work [1] aimed at the mounting of cloned devices
>> using a superblock flag SINGLE_DEV during mkfs.
>>   [1] https://lore.kernel.org/linux-btrfs/20230831001544.3379273-1-gpiccoli@igalia.com/
>>
>> Building upon this work, here is in-memory only approach. As it mounts
>> we determine if the same fsid is already mounted if then we generate a
>> random temp fsid which shall be used the mount, in-memory only not
>> written to the disk. We distinguish device by devt.
>>
>> Mount option / superblock flag:
>> -------------------------------
>>   These patches show we don't have to limit the single-device / temp_fsid
>> capability with a mount option or a superblock flag from the btrfs
>> internals pov. However, if necessary from the user's perspective,
>> we can add them later on top of this patch. I've prepared a mount option
>> -o temp_fsid patch, but I'm not included at this time. As most of the
>> tests was without it.
>>
>> Compatible with other features that may be affected:
>> ----------------------------------------------------
>>   Multi device:
>>      A btrfs filesytem on a single device can be copied using dd and
>>      mounted simlutaneously. However, a multi device btrfs copied using
>>      dd and trying to mount simlutaneously is forced to fail:
>>
>>        mount: /btrfs1: mount(2) system call failed: File exists.
>>


>>   Send and receive:
>>      Quick tests shows send and receive between two single devices with
>>      the same fsid mounted on the _same_ host works!.
> 
> Does it depend if the filesystem remains mounted for the whole
> time? So if there's an unmount, mount again with a temp-fsid, will
> the receive still work?


Yes! Send-receive works even after a mount-recycle with the new 
temp-fsid, as shown below.

Cc-ing Filipe for any comments or send-receive scenario that might fail, 
if any?


---------------------
mkfs a test device whose uuid and fsid will be duplicated

  $ mkfs.btrfs -fq /dev/sdc1
  $ blkid /dev/sdc1
/dev/sdc1: UUID="99821bd4-322c-4a71-a88d-b9bb3e56223b" 
UUID_SUB="1881db58-1c2f-4639-bf87-5c0af24433d6" TYPE="btrfs" 
PARTUUID="a0de6580-01"
  $ mount /dev/sdc1 /btrfs


using the above uuid and fsid mkfs two more scratch device

  $ mkfs.btrfs -fq -U 99821bd4-322c-4a71-a88d-b9bb3e56223b -P 
1881db58-1c2f-4639-bf87-5c0af24433d6 /dev/sdc2
  $ mkfs.btrfs -fq -U 99821bd4-322c-4a71-a88d-b9bb3e56223b -P 
1881db58-1c2f-4639-bf87-5c0af24433d6 /dev/sdc3

mount scratch devices; it will mount using temp-fsid

  $ mount /dev/sdc2 /btrfs1
  $ mount /dev/sdc3 /btrfs2
  $ btrfs filesystem show -m
  Label: none  uuid: 99821bd4-322c-4a71-a88d-b9bb3e56223b
	Total devices 1 FS bytes used 144.00KiB
	devid    1 size 10.00GiB used 536.00MiB path /dev/sdc1

  Label: none  uuid: d041437c-d12e-427c-b0c2-e2591b069feb
	Total devices 1 FS bytes used 144.00KiB
	devid    1 size 10.00GiB used 536.00MiB path /dev/sdc2

  Label: none  uuid: 91c7978f-342f-43d5-a88a-d131dd34962e
	Total devices 1 FS bytes used 144.00KiB
	devid    1 size 10.00GiB used 536.00MiB path /dev/sdc3


create first data and send-receive

  $ xfs_io -f -c 'pwrite -S 0x16 0 9000' /btrfs1/foo
  $ btrfs su snap -r /btrfs1 /btrfs1/snap1
  Create a readonly snapshot of '/btrfs1' in '/btrfs1/snap1'
  $ btrfs send /btrfs1/snap1 | btrfs receive /btrfs2
  At subvol /btrfs1/snap1
  At subvol snap1

  $ sha256sum /btrfs1/foo
  e856cd48942364eed9a205c64aa5e737ab52a73ba2800b07de9d4c331f88cb5b 
/btrfs1/foo
  $ sha256sum /btrfs2/snap1/foo
  e856cd48942364eed9a205c64aa5e737ab52a73ba2800b07de9d4c331f88cb5b 
/btrfs2/snap1/foo


mount recycle so that we have new temp-fsid

  $ umount /btrfs2
  $ umount /btrfs1
  $ mount /dev/sdc2 /btrfs1
  $ mount /dev/sdc3 /btrfs2
  $ btrfs filesystem show -m
  Label: none  uuid: 99821bd4-322c-4a71-a88d-b9bb3e56223b
	Total devices 1 FS bytes used 144.00KiB
	devid    1 size 10.00GiB used 536.00MiB path /dev/sdc1

  Label: none  uuid: 34549411-c9cf-4118-8e42-58dbfd5c4964
	Total devices 1 FS bytes used 172.00KiB
	devid    1 size 10.00GiB used 536.00MiB path /dev/sdc2

  Label: none  uuid: a9ec3b45-f809-49ad-bcb2-bd4b65b130d8
	Total devices 1 FS bytes used 172.00KiB
	devid    1 size 10.00GiB used 536.00MiB path /dev/sdc3


modify foo and send-receive

  $ xfs_io -f -c 'pwrite -S 0xdb 0 9000' /btrfs1/foo
  $ btrfs su snap -r /btrfs1 /btrfs1/snap2
  Create a readonly snapshot of '/btrfs1' in '/btrfs1/snap2'
  $ btrfs send -p /btrfs1/snap1 /btrfs1/snap2 | btrfs receive /btrfs2
  At snapshot snap2
  At subvol /btrfs1/snap2

  $ sha256sum /btrfs1/foo
  5a97ea23517b5f1255161345715f5831b59cbcd62f1fd57e40329980faa7dbd8 
/btrfs1/foo
  $ sha256sum /btrfs2/snap2/foo
  5a97ea23517b5f1255161345715f5831b59cbcd62f1fd57e40329980faa7dbd8 
/btrfs2/snap2/foo
-----------------------------------------------------


> 
>>      (Also, the receive-mnt can receive from multiple senders as long as
>>      conflits are managed externally. ;-).)
>>

I mean multiple senders on temp-fsid mount as long as they have the same 
superblock::fsid.


Thanks, Anand



>>   Replace:
>>       Works fine.
>>
>> btrfs-progs:
>> ------------
>>   btrfs-progs needs to be updated to support the commands such as
>>
>> 	btrfs filesystem show
>>
>>   when devices are not mounted. So the device list is not based on
>>   the fisd any more.
>>
>> Testing:
>> -------
>>   This patch has been under testing for some time. The challenge is to get
>>   the fstests to test this reasonably well.
>>
>>   As of now, this patch runs fine on a large set of fstests test cases
>>   using a custom-built mkfs.btrfs with the -U option and a new -P option
>>   to copy the device FSID and UUID from the TEST_DEV to the SCRATCH_DEV
>>   at the scratch_mkfs time. For example:
>>
>>    Config file:
>>
>>       config_fsid=$(btrfs in dump-super $TEST_DEV | grep -E ^fsid | \
>> 							awk '{print $2}')
>>       config_uuid=$(btrfs in dump-super $TEST_DEV | \
>> 				grep -E ^dev_item.uuid | awk '{print $2}')
>>       MKFS_OPTIONS="-U $config_fsid -P $config_uuid"
>>
>>   This configuration option ensures that both TEST_DEV and SCRATCH_DEV will
>>   have the same FSID and device UUID while still applying test-specific
>>   scratch mkfs options.
>>
>> Mkfs.btrfs:
>> -----------
>>   mkfs.btrfs needs to be updated to support the -P option for testing only.
>>
>>     btrfs-progs: allow duplicate fsid for single device
>>     btrfs-progs: add mkfs -P option for dev_uuid
>>
>> Anand Jain (2):
>>    btrfs: add helper function find_fsid_by_disk
>>    btrfs: support cloned-device mount capability
> 
> Added to misc-next, thanks.

Re: [PATCH 0/2] btrfs: support cloned-device mount capability

[Guilherme G. Piccoli]

Hi Anand / David, I was out at a conference and some holidays, so missed
this patch. Is this a replacement of the temp-fsid approach?

So, to clarify a bit the inner workings of this patch: we don't have the
temp-fsid superblock flag anymore? Also, we can mount multiple
partitions holding the same filesystem at the same time, given the
nature of the patch (that generates the random fsid based on devt as per
my superficial understanding) - right? And we don't use the
metadata_uuid field here anymore, i.e., we kinda "lose" the original fsid?

If that approaches is considered better than mine and works fine for the
Steam Deck use case, I'm glad in having that! But I would like at least
to understand why it was preferred over the temp-fsid one, and what are
the differences we can expect (need a flag to mkfs or can use btrfstune
for that, for example).

Thanks in advance,


Guilherme

Re: [PATCH 0/2] btrfs: support cloned-device mount capability

[Anand Jain]

On 10/6/23 15:42, Guilherme G. Piccoli wrote:
> Hi Anand / David, I was out at a conference and some holidays, so missed
> this patch. Is this a replacement of the temp-fsid approach?
> 
> So, to clarify a bit the inner workings of this patch: we don't have the
> temp-fsid superblock flag anymore? 

While btrfs doesn't need this superblock flag internally, we may
consider adding it for improved usability with other Btrfs features.

> Also, we can mount multiple
> partitions holding the same filesystem at the same time, given the
> nature of the patch (that generates the random fsid based on devt as per
> my superficial understanding) - right?

Indeed, devt remains unique to the partition we've utilized for a
similar purpose prior to this patch. Are there any devices lacking
a distinct devt value?


> And we don't use the
> metadata_uuid field here anymore,

Btrfs has always assigned fs_devices::metadata_uuid to either fsid
or metadata_uuid.


> i.e., we kinda "lose" the original fsid?

How? Have you tested to confirm?


> If that approaches is considered better than mine and works fine for the
> Steam Deck use case, 
> I'm glad in having that! 

As you have a use case to verify, can you indeed confirm whether
it works?

> But I would like at least
> to understand why it was preferred over the temp-fsid one, and what are
> the differences we can expect (need a flag to mkfs or can use btrfstune
> for that, for example).

The in-memory disk-super hack in the original patch is essentially a
workaround. This led to the necessity of restricting devices using
metadata_uuid from being used as temp-fsid device. A more appropriate
approach is to enhance device_list_add() to intelligently manage
duplicate disk-super entries by checking devt and permitting them
to mount if unique. This solution deviates from the original patch
and simultaneously addresses the subvol-mount corruption issue
observed in the original implementation.

The additional adjustments [1], such as sysfs interface, the constraints
on device additions, and the limitations on seed devices, are
supplementary patches essential to the comprehensive solution.

   [1] [PATCH 0/4] btrfs: sysfs and unsupported temp-fsid features for 
clones

However, the superblock temp-fsid flag isn't inherently necessary
within btrfs internals. Nevertheless, it can be considered for addition
if it makes the usability of other btrfs features with temp-fsid more 
seamless.

Thanks, Anand

Re: [PATCH 0/2] btrfs: support cloned-device mount capability

[Guilherme G. Piccoli]

On 07/10/2023 10:01, Anand Jain wrote:
> [...]

Hi Anand, thanks for your response and apologies for my delay.


>> Also, we can mount multiple
>> partitions holding the same filesystem at the same time, given the
>> nature of the patch (that generates the random fsid based on devt as per
>> my superficial understanding) - right?
> 
> Indeed, devt remains unique to the partition we've utilized for a
> similar purpose prior to this patch. Are there any devices lacking
> a distinct devt value?
> 

Not that I'm aware of, it was more of a curiosity.


>> i.e., we kinda "lose" the original fsid?
> 
> How? Have you tested to confirm?

Oh no, not literally I meant. When we go with the temp-fsid approach as
you implemented, the kernel doesn't inform the real fsid. But that's not
an issue at all, more of a curiosity...

I just tested misc-next and your approach seems to be working fine!


> 
>> If that approaches is considered better than mine and works fine for the
>> Steam Deck use case, 
>> I'm glad in having that! 
> 
> As you have a use case to verify, can you indeed confirm whether
> it works?

It does work! I'll test more in the Steam Deck, but so far seems to be
addressing fine the use case we have...


> [...] 
>> But I would like at least
>> to understand why it was preferred over the temp-fsid one, and what are
>> the differences we can expect (need a flag to mkfs or can use btrfstune
>> for that, for example).
> 
> The in-memory disk-super hack in the original patch is essentially a
> workaround. This led to the necessity of restricting devices using
> metadata_uuid from being used as temp-fsid device. A more appropriate
> approach is to enhance device_list_add() to intelligently manage
> duplicate disk-super entries by checking devt and permitting them
> to mount if unique. This solution deviates from the original patch
> and simultaneously addresses the subvol-mount corruption issue
> observed in the original implementation.
> 

OK, makes sense Anand.
Thanks,


Guilherme

Re: [PATCH 0/2] btrfs: support cloned-device mount capability

[Anand Jain]

> 
> It does work! I'll test more in the Steam Deck, but so far seems to be
> addressing fine the use case we have...

Thanks for the use-case validation!. Is there a way to turn
your use-case into a test-case?


One remaining challenge is with 'btrfs filesystem show' when
cloned devices are unmounted. Currently, only shows one
cloned device.

We could consider porting kernel changes to btrfs-progs to
display all devices, (perhaps with a random fsid). Please go
ahead if you have some time to work on it, as I won't be able
to look into it for the next two weeks.

Thanks, Anand

Re: [PATCH 0/2] btrfs: support cloned-device mount capability

[Guilherme G. Piccoli]

On 19/10/2023 04:13, Anand Jain wrote:
> [...]
> Thanks for the use-case validation!. Is there a way to turn
> your use-case into a test-case?
> 

The xfstests that was submitted for the incompat TEMP_FSID flag covers
the use case - we just need to rewrite that dropping the test for the
flag and changing that for checking sysfs temp-fsid feature.

I can do that next week if such wait is fine =)
Cheers,


Guilherme