[PATCH 0/5] btrfs: use the super_block as bdev holder

[PATCH 0/5] btrfs: use the super_block as bdev holder

[Johannes Thumshirn]

This is a series I've picked up from Christoph, it changes the
block_device's bdev holder from fs_type to the super block.

Here's the original cover letter:
Hi all,

this series contains the btrfs parts of the "remove get_super" from June
that managed to get lost.

I've dropped all the reviews from back then as the rebase against the new
mount API conversion led to a lot of non-trivial conflicts.

Josef kindly ran it through the CI farm and provided a fixup based on that.

---
Christoph Hellwig (5):
      btrfs: always open the device read-only in btrfs_scan_one_device
      btrfs: call btrfs_close_devices from ->kill_sb
      btrfs: split btrfs_fs_devices.opened
      btrfs: open block devices after superblock creation
      btrfs: use the super_block as holder when mounting file systems

 fs/btrfs/disk-io.c |  4 +--
 fs/btrfs/super.c   | 71 ++++++++++++++++++++++++++++++------------------------
 fs/btrfs/volumes.c | 60 +++++++++++++++++++++++----------------------
 fs/btrfs/volumes.h |  8 +++---
 4 files changed, 78 insertions(+), 65 deletions(-)
---
base-commit: a50d41606b333e4364844987deb1060e7ea6c038
change-id: 20240214-hch-device-open-309ef9c98c62

Best regards,
-- 
Johannes Thumshirn <johannes.thumshirn@wdc.com>

[PATCH 1/5] btrfs: always open the device read-only in btrfs_scan_one_device

[Johannes Thumshirn]

From: Christoph Hellwig <hch@lst.de>

btrfs_scan_one_device opens the block device only to read the super
block.  Instead of passing a blk_mode_t argument to sometimes open
it for writing, just hard code BLK_OPEN_READ as it will never write
to the device or hand the block_device out to someone else.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
 fs/btrfs/super.c   | 9 ++++-----
 fs/btrfs/volumes.c | 4 ++--
 fs/btrfs/volumes.h | 2 +-
 3 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 40ae264fd3ed..b6cadf4f21b8 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -299,10 +299,9 @@ static int btrfs_parse_param(struct fs_context *fc, struct fs_parameter *param)
 		break;
 	case Opt_device: {
 		struct btrfs_device *device;
-		blk_mode_t mode = btrfs_open_mode(fc);
 
 		mutex_lock(&uuid_mutex);
-		device = btrfs_scan_one_device(param->string, mode, false);
+		device = btrfs_scan_one_device(param->string, false);
 		mutex_unlock(&uuid_mutex);
 		if (IS_ERR(device))
 			return PTR_ERR(device);
@@ -1808,7 +1807,7 @@ static int btrfs_get_tree_super(struct fs_context *fc)
 	 * With 'true' passed to btrfs_scan_one_device() (mount time) we expect
 	 * either a valid device or an error.
 	 */
-	device = btrfs_scan_one_device(fc->source, mode, true);
+	device = btrfs_scan_one_device(fc->source, true);
 	ASSERT(device != NULL);
 	if (IS_ERR(device)) {
 		mutex_unlock(&uuid_mutex);
@@ -2210,7 +2209,7 @@ static long btrfs_control_ioctl(struct file *file, unsigned int cmd,
 		 * Scanning outside of mount can return NULL which would turn
 		 * into 0 error code.
 		 */
-		device = btrfs_scan_one_device(vol->name, BLK_OPEN_READ, false);
+		device = btrfs_scan_one_device(vol->name, false);
 		ret = PTR_ERR_OR_ZERO(device);
 		mutex_unlock(&uuid_mutex);
 		break;
@@ -2228,7 +2227,7 @@ static long btrfs_control_ioctl(struct file *file, unsigned int cmd,
 		 * Scanning outside of mount can return NULL which would turn
 		 * into 0 error code.
 		 */
-		device = btrfs_scan_one_device(vol->name, BLK_OPEN_READ, false);
+		device = btrfs_scan_one_device(vol->name, false);
 		if (IS_ERR_OR_NULL(device)) {
 			mutex_unlock(&uuid_mutex);
 			ret = PTR_ERR(device);
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 4ad9eca9b46c..44caf1a48d33 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1308,7 +1308,7 @@ int btrfs_forget_devices(dev_t devt)
  * the device or return an error. Multi-device and seeding devices are registered
  * in both cases.
  */
-struct btrfs_device *btrfs_scan_one_device(const char *path, blk_mode_t flags,
+struct btrfs_device *btrfs_scan_one_device(const char *path,
 					   bool mount_arg_dev)
 {
 	struct btrfs_super_block *disk_super;
@@ -1337,7 +1337,7 @@ struct btrfs_device *btrfs_scan_one_device(const char *path, blk_mode_t flags,
 	 * values temporarily, as the device paths of the fsid are the only
 	 * required information for assembling the volume.
 	 */
-	bdev_handle = bdev_open_by_path(path, flags, NULL, NULL);
+	bdev_handle = bdev_open_by_path(path, BLK_OPEN_READ, NULL, NULL);
 	if (IS_ERR(bdev_handle))
 		return ERR_CAST(bdev_handle);
 
diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
index 21d4de0e3f1f..97c7284e7565 100644
--- a/fs/btrfs/volumes.h
+++ b/fs/btrfs/volumes.h
@@ -655,7 +655,7 @@ struct btrfs_block_group *btrfs_create_chunk(struct btrfs_trans_handle *trans,
 void btrfs_mapping_tree_free(struct btrfs_fs_info *fs_info);
 int btrfs_open_devices(struct btrfs_fs_devices *fs_devices,
 		       blk_mode_t flags, void *holder);
-struct btrfs_device *btrfs_scan_one_device(const char *path, blk_mode_t flags,
+struct btrfs_device *btrfs_scan_one_device(const char *path,
 					   bool mount_arg_dev);
 int btrfs_forget_devices(dev_t devt);
 void btrfs_close_devices(struct btrfs_fs_devices *fs_devices);

-- 
2.43.0

Re: [PATCH 1/5] btrfs: always open the device read-only in btrfs_scan_one_device

[David Sterba]

On Wed, Feb 14, 2024 at 08:42:12AM -0800, Johannes Thumshirn wrote:
> From: Christoph Hellwig <hch@lst.de>
> 
> btrfs_scan_one_device opens the block device only to read the super
> block.  Instead of passing a blk_mode_t argument to sometimes open
> it for writing, just hard code BLK_OPEN_READ as it will never write
> to the device or hand the block_device out to someone else.

Opening for write was not meant to be for writing but also to exclude
other attempted writes.

That it's always for read seems OK, this has changed at some point and
is explained in btrfs_scan_one_device():

1356         /*
1357          * Avoid an exclusive open here, as the systemd-udev may initiate the 
1358          * device scan which may race with the user's mount or mkfs command,  
1359          * resulting in failure.                                              
1360          * Since the device scan is solely for reading purposes, there is no   
1361          * need for an exclusive open. Additionally, the devices are read again
1362          * during the mount process. It is ok to get some inconsistent    
1363          * values temporarily, as the device paths of the fsid are the only
1364          * required information for assembling the volume.
1365          */
1366         bdev_handle = bdev_open_by_path(path, flags, NULL, NULL);

[PATCH 2/5] btrfs: call btrfs_close_devices from ->kill_sb

[Johannes Thumshirn]

From: Christoph Hellwig <hch@lst.de>

blkdev_put must not be called under sb->s_umount to avoid a lock order
reversal with disk->open_mutex once call backs from block devices to
the file system using the holder ops are supported.  Move the call
to btrfs_close_devices into btrfs_free_fs_info so that it is closed
from ->kill_sb (which is also called from the mount failure handling
path unlike ->put_super) as well as when an fs_info is freed because
an existing superblock already exists.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
 fs/btrfs/disk-io.c |  4 ++--
 fs/btrfs/super.c   | 27 ++++++++++++++-------------
 2 files changed, 16 insertions(+), 15 deletions(-)

diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 8ab185182c30..4aa67e2a48f6 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -1266,6 +1266,8 @@ static void free_global_roots(struct btrfs_fs_info *fs_info)
 
 void btrfs_free_fs_info(struct btrfs_fs_info *fs_info)
 {
+	if (fs_info->fs_devices)
+		btrfs_close_devices(fs_info->fs_devices);
 	percpu_counter_destroy(&fs_info->dirty_metadata_bytes);
 	percpu_counter_destroy(&fs_info->delalloc_bytes);
 	percpu_counter_destroy(&fs_info->ordered_bytes);
@@ -3609,7 +3611,6 @@ int __cold open_ctree(struct super_block *sb, struct btrfs_fs_devices *fs_device
 
 	iput(fs_info->btree_inode);
 fail:
-	btrfs_close_devices(fs_info->fs_devices);
 	ASSERT(ret < 0);
 	return ret;
 }
@@ -4389,7 +4390,6 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info)
 	iput(fs_info->btree_inode);
 
 	btrfs_mapping_tree_free(fs_info);
-	btrfs_close_devices(fs_info->fs_devices);
 }
 
 void btrfs_mark_buffer_dirty(struct btrfs_trans_handle *trans,
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index b6cadf4f21b8..51b8fd272b15 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -1822,10 +1822,8 @@ static int btrfs_get_tree_super(struct fs_context *fc)
 	if (ret)
 		return ret;
 
-	if (!(fc->sb_flags & SB_RDONLY) && fs_devices->rw_devices == 0) {
-		ret = -EACCES;
-		goto error;
-	}
+	if (!(fc->sb_flags & SB_RDONLY) && fs_devices->rw_devices == 0)
+		return -EACCES;
 
 	bdev = fs_devices->latest_dev->bdev;
 
@@ -1839,15 +1837,12 @@ static int btrfs_get_tree_super(struct fs_context *fc)
 	 * otherwise it's tied to the lifetime of the super_block.
 	 */
 	sb = sget_fc(fc, btrfs_fc_test_super, set_anon_super_fc);
-	if (IS_ERR(sb)) {
-		ret = PTR_ERR(sb);
-		goto error;
-	}
+	if (IS_ERR(sb))
+		return PTR_ERR(sb);
 
 	set_device_specific_options(fs_info);
 
 	if (sb->s_root) {
-		btrfs_close_devices(fs_devices);
 		if ((fc->sb_flags ^ sb->s_flags) & SB_RDONLY)
 			ret = -EBUSY;
 	} else {
@@ -1866,10 +1861,6 @@ static int btrfs_get_tree_super(struct fs_context *fc)
 
 	fc->root = dget(sb->s_root);
 	return 0;
-
-error:
-	btrfs_close_devices(fs_devices);
-	return ret;
 }
 
 /*
@@ -1962,10 +1953,20 @@ static int btrfs_get_tree_super(struct fs_context *fc)
  */
 static struct vfsmount *btrfs_reconfigure_for_mount(struct fs_context *fc)
 {
+	struct btrfs_fs_info *fs_info = fc->s_fs_info;
 	struct vfsmount *mnt;
 	int ret;
 	const bool ro2rw = !(fc->sb_flags & SB_RDONLY);
 
+	/*
+	 * We got a reference to our fs_devices, so we need to close it here to
+	 * make sure we don't leak our reference on the fs_devices.
+	 */
+	if (fs_info->fs_devices) {
+		btrfs_close_devices(fs_info->fs_devices);
+		fs_info->fs_devices = NULL;
+	}
+
 	/*
 	 * We got an EBUSY because our SB_RDONLY flag didn't match the existing
 	 * super block, so invert our setting here and retry the mount so we

-- 
2.43.0

Re: [PATCH 2/5] btrfs: call btrfs_close_devices from ->kill_sb

[Qu Wenruo]

在 2024/2/15 03:12, Johannes Thumshirn 写道:
> From: Christoph Hellwig <hch@lst.de>
> 
> blkdev_put must not be called under sb->s_umount to avoid a lock order
> reversal with disk->open_mutex once call backs from block devices to
> the file system using the holder ops are supported.  Move the call
> to btrfs_close_devices into btrfs_free_fs_info so that it is closed
> from ->kill_sb (which is also called from the mount failure handling
> path unlike ->put_super) as well as when an fs_info is freed because
> an existing superblock already exists.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
> ---
>   fs/btrfs/disk-io.c |  4 ++--
>   fs/btrfs/super.c   | 27 ++++++++++++++-------------
>   2 files changed, 16 insertions(+), 15 deletions(-)
> 
> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
> index 8ab185182c30..4aa67e2a48f6 100644
> --- a/fs/btrfs/disk-io.c
> +++ b/fs/btrfs/disk-io.c
> @@ -1266,6 +1266,8 @@ static void free_global_roots(struct btrfs_fs_info *fs_info)
>   
>   void btrfs_free_fs_info(struct btrfs_fs_info *fs_info)
>   {
> +	if (fs_info->fs_devices)
> +		btrfs_close_devices(fs_info->fs_devices);
>   	percpu_counter_destroy(&fs_info->dirty_metadata_bytes);
>   	percpu_counter_destroy(&fs_info->delalloc_bytes);
>   	percpu_counter_destroy(&fs_info->ordered_bytes);
> @@ -3609,7 +3611,6 @@ int __cold open_ctree(struct super_block *sb, struct btrfs_fs_devices *fs_device
>   
>   	iput(fs_info->btree_inode);
>   fail:
> -	btrfs_close_devices(fs_info->fs_devices);
>   	ASSERT(ret < 0);
>   	return ret;
>   }
> @@ -4389,7 +4390,6 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info)
>   	iput(fs_info->btree_inode);
>   
>   	btrfs_mapping_tree_free(fs_info);
> -	btrfs_close_devices(fs_info->fs_devices);
>   }
>   
>   void btrfs_mark_buffer_dirty(struct btrfs_trans_handle *trans,
> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
> index b6cadf4f21b8..51b8fd272b15 100644
> --- a/fs/btrfs/super.c
> +++ b/fs/btrfs/super.c
> @@ -1822,10 +1822,8 @@ static int btrfs_get_tree_super(struct fs_context *fc)
>   	if (ret)
>   		return ret;
>   
> -	if (!(fc->sb_flags & SB_RDONLY) && fs_devices->rw_devices == 0) {
> -		ret = -EACCES;
> -		goto error;
> -	}
> +	if (!(fc->sb_flags & SB_RDONLY) && fs_devices->rw_devices == 0)
> +		return -EACCES;
>   
>   	bdev = fs_devices->latest_dev->bdev;
>   
> @@ -1839,15 +1837,12 @@ static int btrfs_get_tree_super(struct fs_context *fc)
>   	 * otherwise it's tied to the lifetime of the super_block.
>   	 */
>   	sb = sget_fc(fc, btrfs_fc_test_super, set_anon_super_fc);
> -	if (IS_ERR(sb)) {
> -		ret = PTR_ERR(sb);
> -		goto error;
> -	}
> +	if (IS_ERR(sb))
> +		return PTR_ERR(sb);
>   
>   	set_device_specific_options(fs_info);
>   
>   	if (sb->s_root) {
> -		btrfs_close_devices(fs_devices);
>   		if ((fc->sb_flags ^ sb->s_flags) & SB_RDONLY)
>   			ret = -EBUSY;
>   	} else {
> @@ -1866,10 +1861,6 @@ static int btrfs_get_tree_super(struct fs_context *fc)
>   
>   	fc->root = dget(sb->s_root);
>   	return 0;
> -
> -error:
> -	btrfs_close_devices(fs_devices);
> -	return ret;
>   }
>   
>   /*
> @@ -1962,10 +1953,20 @@ static int btrfs_get_tree_super(struct fs_context *fc)
>    */
>   static struct vfsmount *btrfs_reconfigure_for_mount(struct fs_context *fc)
>   {
> +	struct btrfs_fs_info *fs_info = fc->s_fs_info;
>   	struct vfsmount *mnt;
>   	int ret;
>   	const bool ro2rw = !(fc->sb_flags & SB_RDONLY);
>   
> +	/*
> +	 * We got a reference to our fs_devices, so we need to close it here to
> +	 * make sure we don't leak our reference on the fs_devices.
> +	 */
> +	if (fs_info->fs_devices) {
> +		btrfs_close_devices(fs_info->fs_devices);
> +		fs_info->fs_devices = NULL;
> +	}
> +

This changed quite some after commit 951a3f59d268 ("btrfs: fix mount 
failure due to remount races") and "btrfs: open code fc_mount() to avoid 
releasing s_umount rw_sempahore" (only in for-next branch).

This part will need some refresh.

Thanks,
Qu>   	/*
>   	 * We got an EBUSY because our SB_RDONLY flag didn't match the existing
>   	 * super block, so invert our setting here and retry the mount so we
> 

[PATCH 3/5] btrfs: split btrfs_fs_devices.opened

[Johannes Thumshirn]

From: Christoph Hellwig <hch@lst.de>

The btrfs_fs_devices.opened member mixes an in use counter for the
fs_devices structure that prevents it from being garbage collected with
a flag if the underlying devices were actually opened.  This not only
makes the code hard to follow, but also prevents btrfs from switching
to opening the block device only after super block creation.  Split it
into an in_use counter and an is_open boolean flag instead.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
 fs/btrfs/volumes.c | 51 +++++++++++++++++++++++++++++----------------------
 fs/btrfs/volumes.h |  6 ++++--
 2 files changed, 33 insertions(+), 24 deletions(-)

diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 44caf1a48d33..f27af155abf0 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -412,7 +412,8 @@ static void free_fs_devices(struct btrfs_fs_devices *fs_devices)
 {
 	struct btrfs_device *device;
 
-	WARN_ON(fs_devices->opened);
+	WARN_ON_ONCE(fs_devices->in_use);
+	WARN_ON_ONCE(fs_devices->is_open);
 	while (!list_empty(&fs_devices->devices)) {
 		device = list_entry(fs_devices->devices.next,
 				    struct btrfs_device, dev_list);
@@ -535,7 +536,7 @@ static int btrfs_free_stale_devices(dev_t devt, struct btrfs_device *skip_device
 				continue;
 			if (devt && devt != device->devt)
 				continue;
-			if (fs_devices->opened) {
+			if (fs_devices->in_use) {
 				if (devt)
 					ret = -EBUSY;
 				break;
@@ -607,7 +608,7 @@ static struct btrfs_fs_devices *find_fsid_by_device(
 	if (found_by_devt) {
 		/* Existing device. */
 		if (fsid_fs_devices == NULL) {
-			if (devt_fs_devices->opened == 0) {
+			if (devt_fs_devices->in_use == 0) {
 				/* Stale device. */
 				return NULL;
 			} else {
@@ -795,7 +796,7 @@ static noinline struct btrfs_device *device_list_add(const char *path,
 	if (!device) {
 		unsigned int nofs_flag;
 
-		if (fs_devices->opened) {
+		if (fs_devices->in_use) {
 			btrfs_err(NULL,
 "device %s belongs to fsid %pU, and the fs is already mounted, scanned by %s (%d)",
 				  path, fs_devices->fsid, current->comm,
@@ -860,7 +861,7 @@ static noinline struct btrfs_device *device_list_add(const char *path,
 		 * tracking a problem where systems fail mount by subvolume id
 		 * when we reject replacement on a mounted FS.
 		 */
-		if (!fs_devices->opened && found_transid < device->generation) {
+		if (!fs_devices->in_use && found_transid < device->generation) {
 			/*
 			 * That is if the FS is _not_ mounted and if you
 			 * are here, that means there is more than one
@@ -921,7 +922,7 @@ static noinline struct btrfs_device *device_list_add(const char *path,
 	 * it back. We need it to pick the disk with largest generation
 	 * (as above).
 	 */
-	if (!fs_devices->opened) {
+	if (!fs_devices->in_use) {
 		device->generation = found_transid;
 		fs_devices->latest_generation = max_t(u64, found_transid,
 						fs_devices->latest_generation);
@@ -1120,15 +1121,19 @@ static void close_fs_devices(struct btrfs_fs_devices *fs_devices)
 
 	lockdep_assert_held(&uuid_mutex);
 
-	if (--fs_devices->opened > 0)
+	if (--fs_devices->in_use > 0)
 		return;
 
+	if (!fs_devices->is_open)
+		goto done;
+
 	list_for_each_entry_safe(device, tmp, &fs_devices->devices, dev_list)
 		btrfs_close_one_device(device);
 
 	WARN_ON(fs_devices->open_devices);
 	WARN_ON(fs_devices->rw_devices);
-	fs_devices->opened = 0;
+	fs_devices->is_open = false;
+done:
 	fs_devices->seeding = false;
 	fs_devices->fs_info = NULL;
 }
@@ -1140,7 +1145,7 @@ void btrfs_close_devices(struct btrfs_fs_devices *fs_devices)
 
 	mutex_lock(&uuid_mutex);
 	close_fs_devices(fs_devices);
-	if (!fs_devices->opened) {
+	if (!fs_devices->in_use) {
 		list_splice_init(&fs_devices->seed_list, &list);
 
 		/*
@@ -1188,7 +1193,7 @@ static int open_fs_devices(struct btrfs_fs_devices *fs_devices,
 	if (fs_devices->open_devices == 0)
 		return -EINVAL;
 
-	fs_devices->opened = 1;
+	fs_devices->is_open = true;
 	fs_devices->latest_dev = latest_dev;
 	fs_devices->total_rw_bytes = 0;
 	fs_devices->chunk_alloc_policy = BTRFS_CHUNK_ALLOC_REGULAR;
@@ -1225,16 +1230,14 @@ int btrfs_open_devices(struct btrfs_fs_devices *fs_devices,
 	 * We also don't need the lock here as this is called during mount and
 	 * exclusion is provided by uuid_mutex
 	 */
-
-	if (fs_devices->opened) {
-		fs_devices->opened++;
-		ret = 0;
-	} else {
+	if (!fs_devices->is_open) {
 		list_sort(NULL, &fs_devices->devices, devid_cmp);
 		ret = open_fs_devices(fs_devices, flags, holder);
+		if (ret)
+			return ret;
 	}
-
-	return ret;
+	fs_devices->in_use++;
+	return 0;
 }
 
 void btrfs_release_disk_super(struct btrfs_super_block *super)
@@ -2201,13 +2204,14 @@ int btrfs_rm_device(struct btrfs_fs_info *fs_info,
 	 * This can happen if cur_devices is the private seed devices list.  We
 	 * cannot call close_fs_devices() here because it expects the uuid_mutex
 	 * to be held, but in fact we don't need that for the private
-	 * seed_devices, we can simply decrement cur_devices->opened and then
+	 * seed_devices, we can simply decrement cur_devices->in_use and then
 	 * remove it from our list and free the fs_devices.
 	 */
 	if (cur_devices->num_devices == 0) {
 		list_del_init(&cur_devices->seed_list);
-		ASSERT(cur_devices->opened == 1);
-		cur_devices->opened--;
+		ASSERT(cur_devices->in_use == 1);
+		cur_devices->in_use--;
+		cur_devices->is_open = false;
 		free_fs_devices(cur_devices);
 	}
 
@@ -2437,7 +2441,8 @@ static struct btrfs_fs_devices *btrfs_init_sprout(struct btrfs_fs_info *fs_info)
 	list_add(&old_devices->fs_list, &fs_uuids);
 
 	memcpy(seed_devices, fs_devices, sizeof(*seed_devices));
-	seed_devices->opened = 1;
+	seed_devices->in_use = 1;
+	seed_devices->is_open = true;
 	INIT_LIST_HEAD(&seed_devices->devices);
 	INIT_LIST_HEAD(&seed_devices->alloc_list);
 	mutex_init(&seed_devices->device_list_mutex);
@@ -7115,7 +7120,8 @@ static struct btrfs_fs_devices *open_seed_devices(struct btrfs_fs_info *fs_info,
 			return fs_devices;
 
 		fs_devices->seeding = true;
-		fs_devices->opened = 1;
+		fs_devices->in_use = 1;
+		fs_devices->is_open = true;
 		return fs_devices;
 	}
 
@@ -7132,6 +7138,7 @@ static struct btrfs_fs_devices *open_seed_devices(struct btrfs_fs_info *fs_info,
 		free_fs_devices(fs_devices);
 		return ERR_PTR(ret);
 	}
+	fs_devices->in_use = 1;
 
 	if (!fs_devices->seeding) {
 		close_fs_devices(fs_devices);
diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
index 97c7284e7565..d6dc41c62998 100644
--- a/fs/btrfs/volumes.h
+++ b/fs/btrfs/volumes.h
@@ -372,8 +372,10 @@ struct btrfs_fs_devices {
 
 	struct list_head seed_list;
 
-	/* Count fs-devices opened. */
-	int opened;
+	/* Count if fs_device is in used. */
+	unsigned int in_use;
+	/* True if the devices were opened. */
+	bool is_open;
 
 	/* Set when we find or add a device that doesn't have the nonrot flag set. */
 	bool rotating;

-- 
2.43.0

[PATCH 4/5] btrfs: open block devices after superblock creation

[Johannes Thumshirn]

From: Christoph Hellwig <hch@lst.de>

Currently btrfs_mount_root opens the block devices before committing to
allocating a super block. That creates problems for restricting the
number of writers to a device, and also leads to a unusual and not very
helpful holder (the fs_type).

Reorganize the code to first check whether the superblock for a
particular fsid does already exist and open the block devices only if it
doesn't, mirroring the recent changes to the VFS mount helpers.  To do
this the increment of the in_use counter moves out of btrfs_open_devices
and into the only caller in btrfs_mount_root so that it happens before
dropping uuid_mutex around the call to sget.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
 fs/btrfs/super.c   | 41 +++++++++++++++++++++++++----------------
 fs/btrfs/volumes.c | 15 +++++----------
 2 files changed, 30 insertions(+), 26 deletions(-)

diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 51b8fd272b15..1fa7d83d02c1 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -1794,7 +1794,6 @@ static int btrfs_get_tree_super(struct fs_context *fc)
 	struct btrfs_fs_info *fs_info = fc->s_fs_info;
 	struct btrfs_fs_context *ctx = fc->fs_private;
 	struct btrfs_fs_devices *fs_devices = NULL;
-	struct block_device *bdev;
 	struct btrfs_device *device;
 	struct super_block *sb;
 	blk_mode_t mode = btrfs_open_mode(fc);
@@ -1817,15 +1816,8 @@ static int btrfs_get_tree_super(struct fs_context *fc)
 	fs_devices = device->fs_devices;
 	fs_info->fs_devices = fs_devices;
 
-	ret = btrfs_open_devices(fs_devices, mode, &btrfs_fs_type);
+	fs_devices->in_use++;
 	mutex_unlock(&uuid_mutex);
-	if (ret)
-		return ret;
-
-	if (!(fc->sb_flags & SB_RDONLY) && fs_devices->rw_devices == 0)
-		return -EACCES;
-
-	bdev = fs_devices->latest_dev->bdev;
 
 	/*
 	 * From now on the error handling is not straightforward.
@@ -1843,24 +1835,41 @@ static int btrfs_get_tree_super(struct fs_context *fc)
 	set_device_specific_options(fs_info);
 
 	if (sb->s_root) {
-		if ((fc->sb_flags ^ sb->s_flags) & SB_RDONLY)
+		if ((fc->sb_flags ^ sb->s_flags) & SB_RDONLY) {
 			ret = -EBUSY;
+			goto error_deactivate;
+		}
 	} else {
-		snprintf(sb->s_id, sizeof(sb->s_id), "%pg", bdev);
+		struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
+
+		mutex_lock(&uuid_mutex);
+		ret = btrfs_open_devices(fs_devices, mode, &btrfs_fs_type);
+		mutex_unlock(&uuid_mutex);
+		if (ret)
+			goto error_deactivate;
+
+		if (!(fc->sb_flags & SB_RDONLY) && !fs_devices->rw_devices) {
+			ret = -EACCES;
+			goto error_deactivate;
+		}
+
+		snprintf(sb->s_id, sizeof(sb->s_id), "%pg",
+			 fs_devices->latest_dev->bdev);
 		shrinker_debugfs_rename(sb->s_shrink, "sb-btrfs:%s", sb->s_id);
 		btrfs_sb(sb)->bdev_holder = &btrfs_fs_type;
 		ret = btrfs_fill_super(sb, fs_devices, NULL);
-	}
-
-	if (ret) {
-		deactivate_locked_super(sb);
-		return ret;
+		if (ret)
+			goto error_deactivate;
 	}
 
 	btrfs_clear_oneshot_options(fs_info);
 
 	fc->root = dget(sb->s_root);
 	return 0;
+
+error_deactivate:
+	deactivate_locked_super(sb);
+	return ret;
 }
 
 /*
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index f27af155abf0..6e82bd7ce501 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1220,8 +1220,6 @@ static int devid_cmp(void *priv, const struct list_head *a,
 int btrfs_open_devices(struct btrfs_fs_devices *fs_devices,
 		       blk_mode_t flags, void *holder)
 {
-	int ret;
-
 	lockdep_assert_held(&uuid_mutex);
 	/*
 	 * The device_list_mutex cannot be taken here in case opening the
@@ -1230,14 +1228,11 @@ int btrfs_open_devices(struct btrfs_fs_devices *fs_devices,
 	 * We also don't need the lock here as this is called during mount and
 	 * exclusion is provided by uuid_mutex
 	 */
-	if (!fs_devices->is_open) {
-		list_sort(NULL, &fs_devices->devices, devid_cmp);
-		ret = open_fs_devices(fs_devices, flags, holder);
-		if (ret)
-			return ret;
-	}
-	fs_devices->in_use++;
-	return 0;
+	ASSERT(fs_devices->in_use);
+	if (fs_devices->is_open)
+		return 0;
+	list_sort(NULL, &fs_devices->devices, devid_cmp);
+	return open_fs_devices(fs_devices, flags, holder);
 }
 
 void btrfs_release_disk_super(struct btrfs_super_block *super)

-- 
2.43.0

Re: [PATCH 4/5] btrfs: open block devices after superblock creation

[Boris Burkov]

On Wed, Feb 14, 2024 at 08:42:15AM -0800, Johannes Thumshirn wrote:
> From: Christoph Hellwig <hch@lst.de>
> 
> Currently btrfs_mount_root opens the block devices before committing to
> allocating a super block. That creates problems for restricting the
> number of writers to a device, and also leads to a unusual and not very
> helpful holder (the fs_type).
> 
> Reorganize the code to first check whether the superblock for a
> particular fsid does already exist and open the block devices only if it
> doesn't, mirroring the recent changes to the VFS mount helpers.  To do
> this the increment of the in_use counter moves out of btrfs_open_devices
> and into the only caller in btrfs_mount_root so that it happens before
> dropping uuid_mutex around the call to sget.

I believe this commit message is now out of date as of
'btrfs: remove old mount API code'
which got rid of btrfs_mount_root.

As far as I can tell, the code itself is updated and fine.

> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
> ---
>  fs/btrfs/super.c   | 41 +++++++++++++++++++++++++----------------
>  fs/btrfs/volumes.c | 15 +++++----------
>  2 files changed, 30 insertions(+), 26 deletions(-)
> 
> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
> index 51b8fd272b15..1fa7d83d02c1 100644
> --- a/fs/btrfs/super.c
> +++ b/fs/btrfs/super.c
> @@ -1794,7 +1794,6 @@ static int btrfs_get_tree_super(struct fs_context *fc)
>  	struct btrfs_fs_info *fs_info = fc->s_fs_info;
>  	struct btrfs_fs_context *ctx = fc->fs_private;
>  	struct btrfs_fs_devices *fs_devices = NULL;
> -	struct block_device *bdev;
>  	struct btrfs_device *device;
>  	struct super_block *sb;
>  	blk_mode_t mode = btrfs_open_mode(fc);
> @@ -1817,15 +1816,8 @@ static int btrfs_get_tree_super(struct fs_context *fc)
>  	fs_devices = device->fs_devices;
>  	fs_info->fs_devices = fs_devices;
>  
> -	ret = btrfs_open_devices(fs_devices, mode, &btrfs_fs_type);
> +	fs_devices->in_use++;
>  	mutex_unlock(&uuid_mutex);
> -	if (ret)
> -		return ret;
> -
> -	if (!(fc->sb_flags & SB_RDONLY) && fs_devices->rw_devices == 0)
> -		return -EACCES;
> -
> -	bdev = fs_devices->latest_dev->bdev;
>  
>  	/*
>  	 * From now on the error handling is not straightforward.
> @@ -1843,24 +1835,41 @@ static int btrfs_get_tree_super(struct fs_context *fc)
>  	set_device_specific_options(fs_info);
>  
>  	if (sb->s_root) {
> -		if ((fc->sb_flags ^ sb->s_flags) & SB_RDONLY)
> +		if ((fc->sb_flags ^ sb->s_flags) & SB_RDONLY) {
>  			ret = -EBUSY;
> +			goto error_deactivate;
> +		}
>  	} else {
> -		snprintf(sb->s_id, sizeof(sb->s_id), "%pg", bdev);
> +		struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
> +
> +		mutex_lock(&uuid_mutex);
> +		ret = btrfs_open_devices(fs_devices, mode, &btrfs_fs_type);
> +		mutex_unlock(&uuid_mutex);
> +		if (ret)
> +			goto error_deactivate;
> +
> +		if (!(fc->sb_flags & SB_RDONLY) && !fs_devices->rw_devices) {
> +			ret = -EACCES;
> +			goto error_deactivate;
> +		}
> +
> +		snprintf(sb->s_id, sizeof(sb->s_id), "%pg",
> +			 fs_devices->latest_dev->bdev);
>  		shrinker_debugfs_rename(sb->s_shrink, "sb-btrfs:%s", sb->s_id);
>  		btrfs_sb(sb)->bdev_holder = &btrfs_fs_type;
>  		ret = btrfs_fill_super(sb, fs_devices, NULL);
> -	}
> -
> -	if (ret) {
> -		deactivate_locked_super(sb);
> -		return ret;
> +		if (ret)
> +			goto error_deactivate;
>  	}
>  
>  	btrfs_clear_oneshot_options(fs_info);
>  
>  	fc->root = dget(sb->s_root);
>  	return 0;
> +
> +error_deactivate:
> +	deactivate_locked_super(sb);
> +	return ret;
>  }
>  
>  /*
> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> index f27af155abf0..6e82bd7ce501 100644
> --- a/fs/btrfs/volumes.c
> +++ b/fs/btrfs/volumes.c
> @@ -1220,8 +1220,6 @@ static int devid_cmp(void *priv, const struct list_head *a,
>  int btrfs_open_devices(struct btrfs_fs_devices *fs_devices,
>  		       blk_mode_t flags, void *holder)
>  {
> -	int ret;
> -
>  	lockdep_assert_held(&uuid_mutex);
>  	/*
>  	 * The device_list_mutex cannot be taken here in case opening the
> @@ -1230,14 +1228,11 @@ int btrfs_open_devices(struct btrfs_fs_devices *fs_devices,
>  	 * We also don't need the lock here as this is called during mount and
>  	 * exclusion is provided by uuid_mutex
>  	 */
> -	if (!fs_devices->is_open) {
> -		list_sort(NULL, &fs_devices->devices, devid_cmp);
> -		ret = open_fs_devices(fs_devices, flags, holder);
> -		if (ret)
> -			return ret;
> -	}
> -	fs_devices->in_use++;
> -	return 0;
> +	ASSERT(fs_devices->in_use);
> +	if (fs_devices->is_open)
> +		return 0;
> +	list_sort(NULL, &fs_devices->devices, devid_cmp);
> +	return open_fs_devices(fs_devices, flags, holder);
>  }
>  
>  void btrfs_release_disk_super(struct btrfs_super_block *super)
> 
> -- 
> 2.43.0
> 

Re: [PATCH 4/5] btrfs: open block devices after superblock creation

[David Sterba]

On Wed, Feb 14, 2024 at 10:58:09AM -0800, Boris Burkov wrote:
> On Wed, Feb 14, 2024 at 08:42:15AM -0800, Johannes Thumshirn wrote:
> > From: Christoph Hellwig <hch@lst.de>
> > 
> > Currently btrfs_mount_root opens the block devices before committing to
> > allocating a super block. That creates problems for restricting the
> > number of writers to a device, and also leads to a unusual and not very
> > helpful holder (the fs_type).
> > 
> > Reorganize the code to first check whether the superblock for a
> > particular fsid does already exist and open the block devices only if it
> > doesn't, mirroring the recent changes to the VFS mount helpers.  To do
> > this the increment of the in_use counter moves out of btrfs_open_devices
> > and into the only caller in btrfs_mount_root so that it happens before
> > dropping uuid_mutex around the call to sget.
> 
> I believe this commit message is now out of date as of
> 'btrfs: remove old mount API code'
> which got rid of btrfs_mount_root.

It's not just that, this patchset was sent before the conversion to new
mount API that changed how devices are scanned (and potentially race
with mount). The changelog should be updated at minimum.

I haven't found any problems so far, the locking around device opening
should serialize any races so the one thread winning will open the super
block and the other will inherit the fs_devices.

[PATCH 5/5] btrfs: use the super_block as holder when mounting file systems

[Johannes Thumshirn]

From: Christoph Hellwig <hch@lst.de>

The file system type is not a very useful holder as it doesn't allow us
to go back to the actual file system instance.  Pass the super_block
instead which is useful when passed back to the file system driver.

This matches what is done for all other block device based file systems.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
---
 fs/btrfs/super.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 1fa7d83d02c1..0c7956e8f21e 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -1843,7 +1843,7 @@ static int btrfs_get_tree_super(struct fs_context *fc)
 		struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
 
 		mutex_lock(&uuid_mutex);
-		ret = btrfs_open_devices(fs_devices, mode, &btrfs_fs_type);
+		ret = btrfs_open_devices(fs_devices, mode, sb);
 		mutex_unlock(&uuid_mutex);
 		if (ret)
 			goto error_deactivate;
@@ -1856,7 +1856,7 @@ static int btrfs_get_tree_super(struct fs_context *fc)
 		snprintf(sb->s_id, sizeof(sb->s_id), "%pg",
 			 fs_devices->latest_dev->bdev);
 		shrinker_debugfs_rename(sb->s_shrink, "sb-btrfs:%s", sb->s_id);
-		btrfs_sb(sb)->bdev_holder = &btrfs_fs_type;
+		btrfs_sb(sb)->bdev_holder = sb;
 		ret = btrfs_fill_super(sb, fs_devices, NULL);
 		if (ret)
 			goto error_deactivate;

-- 
2.43.0

Re: [PATCH 0/5] btrfs: use the super_block as bdev holder

[Boris Burkov]

On Wed, Feb 14, 2024 at 08:42:11AM -0800, Johannes Thumshirn wrote:
> This is a series I've picked up from Christoph, it changes the
> block_device's bdev holder from fs_type to the super block.

Applies and builds on my for-next, and LGTM. A few non-urgent inline
comments in the patches, but assuming this has gone through CI again,

Reviewed-by: Boris Burkov <boris@bur.io>

> 
> Here's the original cover letter:
> Hi all,
> 
> this series contains the btrfs parts of the "remove get_super" from June
> that managed to get lost.
> 
> I've dropped all the reviews from back then as the rebase against the new
> mount API conversion led to a lot of non-trivial conflicts.
> 
> Josef kindly ran it through the CI farm and provided a fixup based on that.
> 
> ---
> Christoph Hellwig (5):
>       btrfs: always open the device read-only in btrfs_scan_one_device
>       btrfs: call btrfs_close_devices from ->kill_sb
>       btrfs: split btrfs_fs_devices.opened
>       btrfs: open block devices after superblock creation
>       btrfs: use the super_block as holder when mounting file systems
> 
>  fs/btrfs/disk-io.c |  4 +--
>  fs/btrfs/super.c   | 71 ++++++++++++++++++++++++++++++------------------------
>  fs/btrfs/volumes.c | 60 +++++++++++++++++++++++----------------------
>  fs/btrfs/volumes.h |  8 +++---
>  4 files changed, 78 insertions(+), 65 deletions(-)
> ---
> base-commit: a50d41606b333e4364844987deb1060e7ea6c038
> change-id: 20240214-hch-device-open-309ef9c98c62
> 
> Best regards,
> -- 
> Johannes Thumshirn <johannes.thumshirn@wdc.com>
> 

Re: [PATCH 0/5] btrfs: use the super_block as bdev holder

[Christoph Hellwig]

On Wed, Feb 14, 2024 at 08:42:11AM -0800, Johannes Thumshirn wrote:
> This is a series I've picked up from Christoph, it changes the
> block_device's bdev holder from fs_type to the super block.

Thanks Johannes,

from a quick look the rebase looks good to me.