Re: [PATCH RFC 00/10] btrfs: new performance-based chunk allocation using device roles

[Ferry Toth <fntoth@gmail.com>]

Hi,

Op 12-05-2025 om 20:07 schreef Anand Jain:
> In host hardware, devices can have different speeds. Generally, faster
> devices come with lesser capacity while slower devices come with larger
> capacity. A typical configuration would expect that:
> 
>   - A filesystem's read/write performance is evenly distributed on average
>   across the entire filesystem. This is not achievable with the current
>   allocation method because chunks are allocated based only on device free
>   space.
> 
>   - Typically, faster devices are assigned to metadata chunk allocations
>   while slower devices are assigned to data chunk allocations.

Finally a new effort in this direction.

> Introducing Device Roles:
> 
>   Here I define 5 device roles in a specific order for metadata and in the
>   reverse order for data: metadata_only, metadata, none, data, data_only.
>   One or more devices may have the same role.
> 
>   The metadata and data roles indicate preference but not exclusivity for
>   that role, whereas data_only and metadata_only are exclusive roles.
> 
> Introducing Role-then-Space allocation method:
> 
>   Metadata allocation can happen on devices with the roles metadata_only,
>   metadata, none, and data in that order. If multiple devices share a role,
>   they are arranged based on device free space.
> 
>   Similarly, data allocation can happen on devices with the roles data_only,
>   data, none, and metadata in that order. If multiple devices share a role,
>   they are arranged based on device free space.

I can see the use case for large pools of disks used in server 
environments where disks get assigned a role.

For desktop use I would like it a lot better with no roles, just a 
performance-based chunk allocation to select between a ssd and a hdd. 
And then used more like a hint to the allocator. Really nothing should 
go wrong if a data or meta-data gets allocated on the wrong / 
sub-optimal disk.

This could then bring back the old hot relocation idea, finally.

> Finding device speed automatically:
> 
>   Measuring device read/write latency for the allocaiton is not good idea,
>   as the historical readings and may be misleading, as they could include
>   iostat data from periods with issues that have since been fixed. Testing
>   to determine relative latency and arranging in ascending order for metadata
>   and descending for data is possible, but is better handled by an external
>   tool that can still set device roles.
> 
> On-Disk Format changes:
> 
>   The following items are defined but are unused on-disk format:
> 
> 	btrfs_dev_item::
> 	 __le64 type; // unused
> 	 __le64 start_offset; // unused
> 	 __le32 dev_group; // unused
> 	 __u8 seek_speed; // unused
> 	 __u8 bandwidth; // unused
> 
>   The device roles is using the dev_item::type 8-bit field to store each
>   device's role.

I think filling the fields with either measured or user entered data 
should be fine, as long as when the disk behavior changes you can 
re-measure or re-enter.

The difference between a ssd and a hdd will be so huge small changes 
will have no real effect.

> Anand Jain (10):
>    btrfs: fix thresh scope in should_alloc_chunk()
>    btrfs: refactor should_alloc_chunk() arg type
>    btrfs: introduce btrfs_split_sysfs_arg() for argument parsing
>    btrfs: introduce device allocation method
>    btrfs: sysfs: show device allocation method
>    btrfs: skip device sorting when only one device is present
>    btrfs: refactor chunk allocation device handling to use list_head
>    btrfs: introduce explicit device roles for block groups
>    btrfs: introduce ROLE_THEN_SPACE device allocation method
>    btrfs: pass device roles through device add ioctl
> 
>   fs/btrfs/block-group.c |  11 +-
>   fs/btrfs/ioctl.c       |  12 +-
>   fs/btrfs/sysfs.c       | 130 ++++++++++++++++++++--
>   fs/btrfs/volumes.c     | 242 +++++++++++++++++++++++++++++++++--------
>   fs/btrfs/volumes.h     |  35 +++++-
>   5 files changed, 366 insertions(+), 64 deletions(-)
>