[PATCH v4 0/6] xfs write streams

[PATCH v4 0/6] xfs write streams

[Kanchan Joshi]

This series introduces a generic interface [1] for write stream management on
files. It enables spatial isolation and concurrency improvments [3] in xfs using
- generic AG-set (patch #4)
- write-stream based AG-set (patch #5)

In LSFMMBPF'26, we discussed write-stream as a mechanism to reduce
the filesystem allocator bottlenecks and improving direct/buffered IO
scalability.

Write streams allow the abstraction provider (fs, block, raid etc.) to
leverage application's intent (file relationships/lifecycle).
- application: sends grouping/isolation intent with a stream id.
- xfs: maps streams to AGs; allocates without interleaving; gains higher
  concurrency due to reduced lock contention.
- hardware: maps streams to underlying allocation unit; reduces device
  internal write amplification, improved life, predictable QoS.

Also:
- Since high-level write stream (in xfs) and logical placement can work
  without the low-level write streams (in block device), series has a general
value beyond hardware that provides spatial isolation.

- For hardware-only spatial isolation, first 3 patches are needed.

- write-stream is different from existing 'filestream' allocator which
  maintains directory-to-AG associations in a global MRU cache. That
requires state managment and memory (and its reclaim). Proposed AG-set
based steering relies on simple, statless/lockless airthmatic that aligns
more with the default allocator heuristics.

[3]
### Performance

1. On regular NVMe

a. Inter-stream concurrency
---------------------------
fio: 4k write, direct IO, 16 jobs, 1 directory, 16 files * 8GiB, iodepth 32
xfs: 16 AGs, 4 write-streams

base: 41 KIOPS
generic AG-set: 93 KIOPS (+126%)
write-stream AG-set: 227 KIOPS (+453%)
here, 16 files are assigned 4 unique write-streams (4 files/stream)

b. Intra-stream concurrency
----------------------------
fio: 4k write, direct IO, 4 jobs, 1 directory, 4 files * 8GiB, iodepth 32
xfs: 16 AGs, 4 write-streams, generic AG-set size = 2, write-stream AG-set size = 4

base: 59 KIOPS
generic AG-set: 94 KIOPS (+59%)
write-stream AG-set: 112 KIOPS (+89%)
here, 4 files are assigned single write-stream

2. On FDP-capable NVMe:
RocksDB YCSB
WAF (base vs write-stream): 35% Reduction


[1]
### Application interface

Four new ioctls:
      FS_IOC_WRITE_STREAM_GET_MAX  query the max streams supported
      FS_IOC_WRITE_STREAM_OPEN     open a stream id, returns a stream fd
      FS_IOC_WRITE_STREAM_SET      attach the stream fd to an open file
      FS_IOC_WRITE_STREAM_GET      query the stream id value set on a file

### Comparison with Write Hints (RWH_WRITE_LIFE_*)

- Semantics: Write Hints describe 'data temperature' (e.g.,
short/long/extreme), implying a lifetime. Write Streams describe 'data
placement' (e.g., Bin 1/Bin 2), implying only separation.

- Scalability: Write Hints are limited to a small, fixed enum (6
values). Write streams are dynamic, provider-dependent values that can
scale much higher (kernel limit: up to 255 due to u8 field).

- Discovery: The existing write-hint interface is advisory and decoupled
  from underlying capabilties; application has no way to probe support
and cannot deterministically know which hints are valid. OTOH, write-streams
provide explicit discovery.

- Usage model: application needs to get a handle (fd) for a write stream
  before being able to use it. This avoids multi-application conflicts.

Note: within the kernel, the separation between two constructs
(write-hint and write-stream) had started from 6.16 itself.

### Changelog

since v3:
https://lore.kernel.org/linux-block/20260616180555.33338-1-joshi.k@samsung.com/
- add fd-based interface to open/set the write stream (Christoph)
- move from single multiplexed ioctl to 4 distinct ioctls (Christoph)
- add mutual exclusion checks against existing write-hint, filestream (Christoph)
- uint16_t for write-stream within iomap and other streamlining (Darrick)

since v2:
https://lore.kernel.org/linux-fsdevel/20260309052944.156054-1-joshi.k@samsung.com/
- xfs default allocator optimization using fixed-size generic AG set (Dave)
- reuse the above to simplify the write-stream AG set handling
- streamline the uapi; Use union for GET_MAX and GET/SET (Darrick)
- uint16_t for write-stream within xfs inode and other cleanups (Darrick)

since v1:
https://lore.kernel.org/linux-fsdevel/20260216052540.217920-1-joshi.k@samsung.com/
- swich from fcntl based to ioctl-based interface (Christian)
- new patch (#4) that makes xfs allocator use the write streams for AG
  selection
- new patch (#5) that introduces software write streams in xfs.

### Interface example

/* FD-based write-stream ioctl */

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <string.h>
#include <errno.h>
#include <linux/types.h>

/* Duplicate the kernel UAPI definitions */
struct fs_write_stream_open {
	uint32_t	stream_id;	/* IN: desired id if OPEN_EXACT set; OUT: assigned id */
	uint32_t	flags;		/* IN: FS_WRITE_STREAM_OPEN_* */
};

#define FS_WRITE_STREAM_OPEN_EXACT      (1 << 0)

#define FS_IOC_WRITE_STREAM_GET_MAX     _IOR('f', 135, __u32)
#define FS_IOC_WRITE_STREAM_OPEN        _IOWR('f', 136, struct fs_write_stream_open)
#define FS_IOC_WRITE_STREAM_SET         _IOW('f', 137, __s32)
#define FS_IOC_WRITE_STREAM_GET         _IOR('f', 138, __u32)

static void usage(const char *prog)
{
	fprintf(stderr, "Usage:\n");
	fprintf(stderr, "  %s <file> max        - get max supported streams\n", prog);
	fprintf(stderr, "  %s <file> get        - get stream id set on file\n", prog);
	fprintf(stderr, "  %s <file> open [id]  - open a stream (EXACT if id given) and bind it to file\n", prog);
	exit(EXIT_FAILURE);
}

int main(int argc, char *argv[])
{
	const char *filepath, *cmd;
	int fd;

	if (argc < 3)
		usage(argv[0]);

	filepath = argv[1];
	cmd = argv[2];

	fd = open(filepath, O_RDWR);
	if (fd < 0) {
		perror("open(file)");
		return EXIT_FAILURE;
	}

	if (!strcmp(cmd, "max")) {
		uint32_t max;

		if (ioctl(fd, FS_IOC_WRITE_STREAM_GET_MAX, &max) < 0) {
			perror("ioctl(GET_MAX)");
			return EXIT_FAILURE;
		}
		printf("Max streams supported: %u\n", max);
	} else if (!strcmp(cmd, "get")) {
		uint32_t stream_id;

		if (ioctl(fd, FS_IOC_WRITE_STREAM_GET, &stream_id) < 0) {
			perror("ioctl(GET)");
			return EXIT_FAILURE;
		}
		printf("Stream id on file: %u\n", stream_id);
	} else if (!strcmp(cmd, "open")) {
		struct fs_write_stream_open wso = { 0 };
		int stream_fd;

		if (argc == 4) {
			wso.flags = FS_WRITE_STREAM_OPEN_EXACT;
			wso.stream_id = atoi(argv[3]);
		}

		/* OPEN can be called through any fd on the target filesystem. */
		stream_fd = ioctl(fd, FS_IOC_WRITE_STREAM_OPEN, &wso);
		if (stream_fd < 0) {
			perror("ioctl(OPEN)");
			return EXIT_FAILURE;
		}
		printf("Opened stream id %u (fd %d)\n", wso.stream_id, stream_fd);

		/* SET takes the stream fd directly as the ioctl argument. */
		if (ioctl(fd, FS_IOC_WRITE_STREAM_SET, (unsigned long)stream_fd) < 0) {
			perror("ioctl(SET)");
			close(stream_fd);
			return EXIT_FAILURE;
		}
		printf("Bound stream %u to %s\n", wso.stream_id, filepath);

		close(stream_fd);
	} else {
		fprintf(stderr, "Unknown command: %s\n", cmd);
		usage(argv[0]);
	}

	close(fd);
	return EXIT_SUCCESS;
}


Anuj Gupta (2):
  fs: add write-stream management ioctls
  xfs: implement write-stream management support

Kanchan Joshi (4):
  iomap: introduce and propagate write_stream
  xfs: generic AG set based steering
  xfs: write stream based AG placement
  xfs: introduce software write streams

 fs/iomap/direct-io.c     |   1 +
 fs/iomap/ioend.c         |   3 +
 fs/xfs/libxfs/xfs_bmap.c |  74 +++++++++++++++++
 fs/xfs/xfs_icache.c      |   1 +
 fs/xfs/xfs_inode.c       | 175 +++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_inode.h       |   8 ++
 fs/xfs/xfs_ioctl.c       |  69 +++++++++++++++
 fs/xfs/xfs_iomap.c       |   1 +
 fs/xfs/xfs_mount.h       |   5 ++
 fs/xfs/xfs_super.c       |  12 +++
 include/linux/iomap.h    |   2 +
 include/uapi/linux/fs.h  |  16 ++++
 12 files changed, 367 insertions(+)

-- 
2.25.1

[PATCH v4 1/6] fs: add write-stream management ioctls

[Kanchan Joshi]

From: Anuj Gupta <anuj20.g@samsung.com>

Wire up the userspace interface for write stream management via four
ioctls:

  FS_IOC_WRITE_STREAM_GET_MAX  query the max streams supported
  FS_IOC_WRITE_STREAM_OPEN     open a stream id, returns a stream fd
  FS_IOC_WRITE_STREAM_SET      attach the stream fd to an open file
  FS_IOC_WRITE_STREAM_GET      query the stream id value set on a file

Application should query the available streams by using
FS_IOC_WRITE_STREAM_GET_MAX. If returned value is N, valid stream id for
the file are 1 to N.

Application calls FS_IOC_WRITE_STREAM_OPEN to get fd for a stream-id.
By default, kernel picks an available stream-id and returns the fd for
it. The flag FS_WRITE_STREAM_OPEN_EXACT can be used to request a
specific stream_id. This is useful if application cares about keeping a
stable stream-id-to-spatial-isolation-bucket mapping across restarts.

Suggested-by: Christoph Hellwig <hch@lst.de>
Co-developed-by: Kanchan Joshi <joshi.k@samsung.com>
Signed-off-by: Anuj Gupta <anuj20.g@samsung.com>
Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
---
 include/uapi/linux/fs.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
index bd87262f2e34..a756d27bdf84 100644
--- a/include/uapi/linux/fs.h
+++ b/include/uapi/linux/fs.h
@@ -345,6 +345,22 @@ struct file_attr {
 /* Get logical block metadata capability details */
 #define FS_IOC_GETLBMD_CAP		_IOWR(0x15, 2, struct logical_block_metadata_cap)
 
+struct fs_write_stream_open {
+	__u32		stream_id;	/* IN: requested id if OPEN_EXACT set; OUT: assigned id */
+	__u32		flags;		/* IN: FS_WRITE_STREAM_OPEN_* */
+};
+
+/*
+ * Flag to ask for specific stream_id, otherwise kernel can return any.
+ * Fails with -EBUSY if that stream_id is already held
+ */
+#define FS_WRITE_STREAM_OPEN_EXACT	(1 << 0)
+
+#define FS_IOC_WRITE_STREAM_GET_MAX	_IOR('f', 135, __u32)
+#define FS_IOC_WRITE_STREAM_OPEN	_IOWR('f', 136, struct fs_write_stream_open)
+#define FS_IOC_WRITE_STREAM_SET		_IOW('f', 137, __s32)
+#define FS_IOC_WRITE_STREAM_GET		_IOR('f', 138, __u32)
+
 /*
  * Inode flags (FS_IOC_GETFLAGS / FS_IOC_SETFLAGS)
  *
-- 
2.25.1

[PATCH v4 2/6] iomap: introduce and propagate write_stream

[Kanchan Joshi]

Add a new write_stream field to struct iomap. Existing hole is used to
place the new field.
Propagate write_stream from iomap to bio in both direct I/O and buffered
writeback paths.

Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
---
 fs/iomap/direct-io.c  | 1 +
 fs/iomap/ioend.c      | 3 +++
 include/linux/iomap.h | 2 ++
 3 files changed, 6 insertions(+)

diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c
index e2cd5f92babe..b7771a25c1c7 100644
--- a/fs/iomap/direct-io.c
+++ b/fs/iomap/direct-io.c
@@ -348,6 +348,7 @@ static ssize_t iomap_dio_bio_iter_one(struct iomap_iter *iter,
 	fscrypt_set_bio_crypt_ctx(bio, iter->inode, pos, GFP_KERNEL);
 	bio->bi_iter.bi_sector = iomap_sector(&iter->iomap, pos);
 	bio->bi_write_hint = iter->inode->i_write_hint;
+	bio->bi_write_stream = iter->iomap.write_stream;
 	bio->bi_ioprio = dio->iocb->ki_ioprio;
 	bio->bi_private = dio;
 	bio->bi_end_io = iomap_dio_bio_end_io;
diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
index 0565328764c1..5ea2c68993cf 100644
--- a/fs/iomap/ioend.c
+++ b/fs/iomap/ioend.c
@@ -165,6 +165,7 @@ static struct iomap_ioend *iomap_alloc_ioend(struct iomap_writepage_ctx *wpc,
 			       GFP_NOFS, &iomap_ioend_bioset);
 	bio->bi_iter.bi_sector = iomap_sector(&wpc->iomap, pos);
 	bio->bi_write_hint = wpc->inode->i_write_hint;
+	bio->bi_write_stream = wpc->iomap.write_stream;
 	wbc_init_bio(wpc->wbc, bio);
 	wpc->nr_folios = 0;
 	return iomap_init_ioend(wpc->inode, bio, pos, ioend_flags);
@@ -188,6 +189,8 @@ static bool iomap_can_add_to_ioend(struct iomap_writepage_ctx *wpc, loff_t pos,
 	if (!(wpc->iomap.flags & IOMAP_F_ANON_WRITE) &&
 	    iomap_sector(&wpc->iomap, pos) != bio_end_sector(&ioend->io_bio))
 		return false;
+	if (wpc->iomap.write_stream != ioend->io_bio.bi_write_stream)
+		return false;
 	/*
 	 * Limit ioend bio chain lengths to minimise IO completion latency. This
 	 * also prevents long tight loops ending page writeback on all the
diff --git a/include/linux/iomap.h b/include/linux/iomap.h
index 56b43d594e6e..4dc71da598f9 100644
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@ -132,6 +132,8 @@ struct iomap {
 	u64			length;	/* length of mapping, bytes */
 	u16			type;	/* type of mapping */
 	u16			flags;	/* flags for mapping */
+	u16			write_stream; /* write stream for I/O */
+	/* 2 bytes padding hole here */
 	struct block_device	*bdev;	/* block device for I/O */
 	struct dax_device	*dax_dev; /* dax_dev for dax operations */
 	void			*inline_data;
-- 
2.25.1

[PATCH v4 3/6] xfs: implement write-stream management support

[Kanchan Joshi]

From: Anuj Gupta <anuj1072538@gmail.com>

Implement FS_IOC_WRITE_STREAM_{GET_MAX,OPEN,SET,GET} handlers.

GET_MAX reports the max write stream ids.
OPEN allocates an fd for the unused write-stream and returns that.
SET binds that fd to an open file.
GET reports the stream-id value set on the file.

To track used write-streams, a per-mount bitmap is kept.
The stream fd's release handler clears the bit when the last reference
drops.

A new i_write_stream field on xfs_inode holds the bound stream id and
is propagated to the iomap during block mapping.

Write streams, filestreams, and write-life-time hints are mutually exclusive;
combining any two of them returns -EINVAL:
  - GET_MAX reports 0 whenever xfs_inode_is_filestream() is true,
    covering both mount-wide filestreams and the per-inode chattr
    flag. Also when the file is on the realtime device.
  - SET refuses to bind a stream to a file that already has a
    write-life-time hint (fcntl F_SET_RW_HINT), is filestream, or is
    on the realtime device.
  - chattr refuses to set the filestream or realtime flag on a file
    that already has a write stream set.

Suggested-by: Christoph Hellwig <hch@lst.de>
Co-developed-by: Kanchan Joshi <joshi.k@samsung.com>
Signed-off-by: Anuj Gupta <anuj20.g@samsung.com>
Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
---
 fs/xfs/xfs_icache.c |   1 +
 fs/xfs/xfs_inode.c  | 155 ++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_inode.h  |   8 +++
 fs/xfs/xfs_ioctl.c  |  69 ++++++++++++++++++++
 fs/xfs/xfs_iomap.c  |   1 +
 fs/xfs/xfs_mount.h  |   3 +
 fs/xfs/xfs_super.c  |  12 ++++
 7 files changed, 249 insertions(+)

diff --git a/fs/xfs/xfs_icache.c b/fs/xfs/xfs_icache.c
index 9d8dd30bd927..7b9dda74122f 100644
--- a/fs/xfs/xfs_icache.c
+++ b/fs/xfs/xfs_icache.c
@@ -129,6 +129,7 @@ xfs_inode_alloc(
 	spin_lock_init(&ip->i_ioend_lock);
 	ip->i_next_unlinked = NULLAGINO;
 	ip->i_prev_unlinked = 0;
+	ip->i_write_stream = 0;
 
 	return ip;
 }
diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
index 15279d22a894..aafc3ffa6e0a 100644
--- a/fs/xfs/xfs_inode.c
+++ b/fs/xfs/xfs_inode.c
@@ -4,6 +4,7 @@
  * All Rights Reserved.
  */
 #include <linux/iversion.h>
+#include <linux/anon_inodes.h>
 
 #include "xfs_platform.h"
 #include "xfs_fs.h"
@@ -47,6 +48,160 @@
 
 struct kmem_cache *xfs_inode_cache;
 
+int
+xfs_inode_max_write_streams(
+	struct xfs_inode	*ip)
+{
+	struct block_device	*bdev;
+	bool			is_filestream, is_realtime;
+
+	xfs_ilock(ip, XFS_ILOCK_SHARED);
+	is_filestream = xfs_inode_is_filestream(ip);
+	is_realtime = XFS_IS_REALTIME_INODE(ip);
+	bdev = xfs_inode_buftarg(ip)->bt_bdev;
+	xfs_iunlock(ip, XFS_ILOCK_SHARED);
+
+	if (!bdev || is_filestream || is_realtime)
+		return 0;
+
+	return bdev_max_write_streams(bdev);
+}
+
+uint16_t
+xfs_inode_get_write_stream(
+	struct xfs_inode	*ip)
+{
+	uint16_t	stream_id;
+
+	xfs_ilock(ip, XFS_ILOCK_SHARED);
+	stream_id = ip->i_write_stream;
+	xfs_iunlock(ip, XFS_ILOCK_SHARED);
+
+	return stream_id;
+}
+
+struct xfs_write_stream {
+	struct xfs_mount	*mp;
+	uint16_t		stream_id;	/* 1-based */
+};
+
+static int
+xfs_write_stream_release(
+	struct inode		*inode,
+	struct file		*file)
+{
+	struct xfs_write_stream	*ws = file->private_data;
+	struct xfs_mount	*mp = ws->mp;
+
+	spin_lock(&mp->m_streams_lock);
+	clear_bit(ws->stream_id - 1, mp->m_streams_in_use);
+	spin_unlock(&mp->m_streams_lock);
+	kfree(ws);
+	return 0;
+}
+
+static const struct file_operations xfs_write_stream_fops = {
+	.release	= xfs_write_stream_release,
+	.llseek		= noop_llseek,
+};
+
+int
+xfs_inode_write_stream_open(
+	struct xfs_inode	*ip,
+	u32			flags,
+	u32			*stream_idp)
+{
+	struct xfs_mount	*mp = ip->i_mount;
+	struct xfs_write_stream	*ws;
+	int			max, slot, fd, ret;
+
+	if (flags & ~FS_WRITE_STREAM_OPEN_EXACT)
+		return -EINVAL;
+
+	max = xfs_inode_max_write_streams(ip);
+	if (!max)
+		return -EOPNOTSUPP;
+	ASSERT(mp->m_streams_in_use);
+
+	ws = kmalloc(sizeof(*ws), GFP_KERNEL);
+	if (!ws)
+		return -ENOMEM;
+
+	spin_lock(&mp->m_streams_lock);
+	if (flags & FS_WRITE_STREAM_OPEN_EXACT) {
+		if (!*stream_idp || *stream_idp > max) {
+			ret = -EINVAL;
+			goto out_unlock;
+		}
+		slot = *stream_idp - 1;
+		if (test_bit(slot, mp->m_streams_in_use)) {
+			ret = -EBUSY;
+			goto out_unlock;
+		}
+	} else {
+		slot = find_first_zero_bit(mp->m_streams_in_use, max);
+		if (slot >= max) {
+			ret = -EBUSY;
+			goto out_unlock;
+		}
+	}
+	set_bit(slot, mp->m_streams_in_use);
+	spin_unlock(&mp->m_streams_lock);
+
+	ws->mp = mp;
+	ws->stream_id = slot + 1;	/* convert to 1-based */
+
+	fd = anon_inode_getfd("[xfs_write_stream]", &xfs_write_stream_fops, ws,
+			      O_RDONLY | O_CLOEXEC);
+	if (fd < 0) {
+		spin_lock(&mp->m_streams_lock);
+		clear_bit(slot, mp->m_streams_in_use);
+		spin_unlock(&mp->m_streams_lock);
+		kfree(ws);
+		return fd;
+	}
+
+	*stream_idp = ws->stream_id;
+	return fd;
+
+out_unlock:
+	spin_unlock(&mp->m_streams_lock);
+	kfree(ws);
+	return ret;
+}
+
+int
+xfs_inode_set_write_stream(
+	struct xfs_inode	*ip,
+	int			stream_fd)
+{
+	CLASS(fd, f)(stream_fd);
+	struct xfs_write_stream	*ws;
+	int			ret = 0;
+
+	if (!fd_file(f))
+		return -EBADF;
+	if (fd_file(f)->f_op != &xfs_write_stream_fops)
+		return -EINVAL;
+
+	ws = fd_file(f)->private_data;
+	if (ws->mp != ip->i_mount)
+		return -EINVAL;
+
+	xfs_ilock(ip, XFS_ILOCK_EXCL);
+
+	if (XFS_IS_REALTIME_INODE(ip) || xfs_inode_is_filestream(ip) ||
+	    VFS_I(ip)->i_write_hint != WRITE_LIFE_NOT_SET) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+	ip->i_write_stream = ws->stream_id;
+out_unlock:
+	xfs_iunlock(ip, XFS_ILOCK_EXCL);
+	return ret;
+}
+
 /*
  * These two are wrapper routines around the xfs_ilock() routine used to
  * centralize some grungy code.  They are used in places that wish to lock the
diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h
index 34c1038ebfcd..6abf82ffbf82 100644
--- a/fs/xfs/xfs_inode.h
+++ b/fs/xfs/xfs_inode.h
@@ -37,6 +37,9 @@ typedef struct xfs_inode {
 	struct xfs_ifork	i_df;		/* data fork */
 	struct xfs_ifork	i_af;		/* attribute fork */
 
+	/* Write stream information */
+	uint16_t		i_write_stream;
+
 	/* Transaction and locking information. */
 	struct xfs_inode_log_item *i_itemp;	/* logging information */
 	struct rw_semaphore	i_lock;		/* inode lock */
@@ -673,4 +676,9 @@ int xfs_icreate_dqalloc(const struct xfs_icreate_args *args,
 		struct xfs_dquot **udqpp, struct xfs_dquot **gdqpp,
 		struct xfs_dquot **pdqpp);
 
+int xfs_inode_max_write_streams(struct xfs_inode *ip);
+uint16_t xfs_inode_get_write_stream(struct xfs_inode *ip);
+int xfs_inode_write_stream_open(struct xfs_inode *ip, u32 flags,
+		u32 *stream_idp);
+int xfs_inode_set_write_stream(struct xfs_inode *ip, int stream_fd);
 #endif	/* __XFS_INODE_H__ */
diff --git a/fs/xfs/xfs_ioctl.c b/fs/xfs/xfs_ioctl.c
index 1b53701bebea..8640e6389768 100644
--- a/fs/xfs/xfs_ioctl.c
+++ b/fs/xfs/xfs_ioctl.c
@@ -557,6 +557,12 @@ xfs_ioctl_setattr_xflags(
 	bool			rtflag = (fa->fsx_xflags & FS_XFLAG_REALTIME);
 	uint64_t		i_flags2;
 
+	if ((fa->fsx_xflags & FS_XFLAG_FILESTREAM) && ip->i_write_stream)
+		return -EINVAL;
+
+	if (rtflag && ip->i_write_stream)
+		return -EINVAL;
+
 	if (rtflag != XFS_IS_REALTIME_INODE(ip)) {
 		/* Can't change realtime flag if any extents are allocated. */
 		if (xfs_inode_has_filedata(ip))
@@ -1200,6 +1206,59 @@ xfs_ioctl_fs_counts(
 	return 0;
 }
 
+static int
+xfs_ioc_write_stream_open(
+	struct file		*filp,
+	void __user		*arg)
+{
+	struct xfs_inode	*ip = XFS_I(file_inode(filp));
+	struct fs_write_stream_open wso;
+	int			fd;
+
+	if (copy_from_user(&wso, arg, sizeof(wso)))
+		return -EFAULT;
+
+	fd = xfs_inode_write_stream_open(ip, wso.flags, &wso.stream_id);
+	if (fd < 0)
+		return fd;
+
+	if (copy_to_user(arg, &wso, sizeof(wso)))
+		return -EFAULT;
+	return fd;
+}
+
+static int
+xfs_ioc_write_stream_set(
+	struct file		*filp,
+	unsigned long		arg)
+{
+	struct xfs_inode	*ip = XFS_I(file_inode(filp));
+
+	if (!(filp->f_mode & FMODE_WRITE))
+		return -EBADF;
+	return xfs_inode_set_write_stream(ip, (int)arg);
+}
+
+static int
+xfs_ioc_write_stream_get(
+	struct xfs_inode	*ip,
+	void __user		*arg)
+{
+	__u32 stream_id = xfs_inode_get_write_stream(ip);
+
+	return put_user(stream_id, (__u32 __user *)arg);
+}
+
+static int
+xfs_ioc_write_stream_get_max(
+	struct xfs_inode	*ip,
+	void __user		*arg)
+{
+	__u32 nr_streams = xfs_inode_max_write_streams(ip);
+
+	return put_user(nr_streams, (__u32 __user *)arg);
+}
+
 /*
  * These long-unused ioctls were removed from the official ioctl API in 5.17,
  * but retain these definitions so that we can log warnings about them.
@@ -1465,6 +1524,16 @@ xfs_file_ioctl(
 		return xfs_ioc_health_monitor(filp, arg);
 	case XFS_IOC_VERIFY_MEDIA:
 		return xfs_ioc_verify_media(filp, arg);
+	case FS_IOC_WRITE_STREAM_OPEN:
+		return xfs_ioc_write_stream_open(filp, (void __user *)arg);
+	case FS_IOC_WRITE_STREAM_SET:
+		return xfs_ioc_write_stream_set(filp, p);
+	case FS_IOC_WRITE_STREAM_GET:
+		return xfs_ioc_write_stream_get(XFS_I(file_inode(filp)),
+						(void __user *)arg);
+	case FS_IOC_WRITE_STREAM_GET_MAX:
+		return xfs_ioc_write_stream_get_max(XFS_I(file_inode(filp)),
+						    (void __user *)arg);
 
 	default:
 		return -ENOTTY;
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index 225c3de88d03..bf423897d916 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -144,6 +144,7 @@ xfs_bmbt_to_iomap(
 	}
 	iomap->offset = XFS_FSB_TO_B(mp, imap->br_startoff);
 	iomap->length = XFS_FSB_TO_B(mp, imap->br_blockcount);
+	iomap->write_stream = ip->i_write_stream;
 	if (mapping_flags & IOMAP_DAX) {
 		iomap->dax_dev = target->bt_daxdev;
 	} else {
diff --git a/fs/xfs/xfs_mount.h b/fs/xfs/xfs_mount.h
index 66a02d1b9ad7..1376963fb7e8 100644
--- a/fs/xfs/xfs_mount.h
+++ b/fs/xfs/xfs_mount.h
@@ -349,6 +349,9 @@ typedef struct xfs_mount {
 
 	/* Index of uuid record in the uuid xarray. */
 	unsigned int		m_uuid_table_index;
+
+	unsigned long		*m_streams_in_use;
+	spinlock_t		m_streams_lock;
 } xfs_mount_t;
 
 #define M_IGEO(mp)		(&(mp)->m_ino_geo)
diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c
index 8531d526fc44..0e2bf6f7b378 100644
--- a/fs/xfs/xfs_super.c
+++ b/fs/xfs/xfs_super.c
@@ -805,6 +805,7 @@ xfs_mount_free(
 #ifdef DEBUG
 	kfree(mp->m_errortag);
 #endif
+	bitmap_free(mp->m_streams_in_use);
 	kfree(mp);
 }
 
@@ -1659,6 +1660,7 @@ xfs_fs_fill_super(
 	struct xfs_mount	*mp = sb->s_fs_info;
 	struct inode		*root;
 	int			flags = 0, error;
+	int			nr_streams;
 
 	mp->m_super = sb;
 
@@ -1708,6 +1710,15 @@ xfs_fs_fill_super(
 	if (error)
 		return error;
 
+	nr_streams = bdev_max_write_streams(mp->m_ddev_targp->bt_bdev);
+	if (nr_streams) {
+		mp->m_streams_in_use = bitmap_zalloc(nr_streams, GFP_KERNEL);
+		if (!mp->m_streams_in_use) {
+			error = -ENOMEM;
+			goto out_shutdown_devices;
+		}
+	}
+
 	if (xfs_debugfs) {
 		mp->m_debugfs = xfs_debugfs_mkdir(mp->m_super->s_id,
 						  xfs_debugfs);
@@ -2249,6 +2260,7 @@ xfs_init_fs_context(
 #endif
 
 	spin_lock_init(&mp->m_sb_lock);
+	spin_lock_init(&mp->m_streams_lock);
 	for (i = 0; i < XG_TYPE_MAX; i++)
 		xa_init(&mp->m_groups[i].xa);
 	mutex_init(&mp->m_growlock);
-- 
2.25.1

[PATCH v4 4/6] xfs: generic AG set based steering

[Kanchan Joshi]

Improve allocator concurrency and reduce interleaving by introducing
fixed sized AG set.
Use low bits of the inode as a hash to select AG within the AG set.
Overall, a file will try to use the same AG (and contiguity is maintained),
but multiple files will be spread across all AGs in the target AG set.

Suggested-by: Dave Chinner <dgc@kernel.org>
Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
Signed-off-by: Anuj Gupta <anuj20.g@samsung.com>
---
 fs/xfs/libxfs/xfs_bmap.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index d64defeda645..fd1a3aa4ad3f 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -3192,6 +3192,36 @@ xfs_bmap_select_minlen(
 	return args->maxlen;
 }
 
+#define	GENERIC_AG_SET_SZ	(2)
+
+static inline xfs_agnumber_t
+xfs_default_ag_set_size(
+	struct xfs_inode	*ip)
+{
+	struct xfs_mount	*mp = ip->i_mount;
+
+	return min_t(xfs_agnumber_t, GENERIC_AG_SET_SZ, mp->m_sb.sb_agcount);
+}
+
+static xfs_agnumber_t
+xfs_ag_to_ag_set(
+	struct xfs_bmalloca	*ap,
+	xfs_agnumber_t		base_agno)
+{
+	struct xfs_inode	*ip = ap->ip;
+	struct xfs_mount	*mp = ip->i_mount;
+	xfs_agnumber_t		set_size;
+
+	/* Apply fanning only for regular file data */
+	if (!(ap->datatype & XFS_ALLOC_USERDATA))
+		return base_agno;
+
+	set_size = xfs_default_ag_set_size(ip);
+	/* Fan out within the AG set using low bits of the inode */
+	return (base_agno + (XFS_INO_TO_AGINO(mp, I_INO(ip)) % set_size)) %
+		mp->m_sb.sb_agcount;
+}
+
 static int
 xfs_bmap_btalloc_select_lengths(
 	struct xfs_bmalloca	*ap,
@@ -3587,8 +3617,16 @@ xfs_bmap_btalloc_best_length(
 {
 	xfs_extlen_t		blen = 0;
 	int			error;
+	xfs_agnumber_t		target_ag, start_ag;
 
 	ap->blkno = XFS_INODE_TO_FSB(ap->ip);
+
+	/* fan out initial AG across the generic AG set */
+	start_ag = XFS_FSB_TO_AGNO(args->mp, ap->blkno);
+	target_ag = xfs_ag_to_ag_set(ap, start_ag);
+	if (target_ag != start_ag)
+		ap->blkno = XFS_AGB_TO_FSB(args->mp, target_ag, 0);
+
 	if (!xfs_bmap_adjacent(ap))
 		ap->eof = false;
 
-- 
2.25.1

[PATCH v4 5/6] xfs: write stream based AG placement

[Kanchan Joshi]

When write stream is set on the file, choose the AG set based on the
write stream value.

Isolating distinct write streams into dedicated allocation groups helps
in reducing the block interleaving of concurrent writers. Keeping these
streams spatially separated reduces AGF lock contention and logical file
fragmentation.

If AGs are fewer than write streams, write streams are distributed into
available AGs in round robin fashion.
If not, available AGs are partitioned into write streams. The write-stream
value is used to derive the AG set, and low bits of the inode is used to
select the AG within the AG set.

While each stream provides the isolation, the intra-stream concurrency
comes from the AG set size.

Example: 8 Allocation Groups, 4 write streams
AG set size = 2 AGs per write stream

   Stream 1 (ID: 1)         Stream 2 (ID: 2)         Streams 3 & 4
 +---------+---------+    +---------+---------+    +-------------
 |   AG0   |   AG1   |    |   AG2   |   AG3   |    |  AG4...AG7
 +---------+---------+    +---------+---------+    +-------------
      ^         ^              ^         ^
      |         |              |         |
      | File B (ino: 101)      | File D (ino: 201)
      | 101 % 2 = 1 -> AG 1    | 201 % 2 = 1 -> AG 3
      |                        |
 File A (ino: 100)        File C (ino: 200)
 100 % 2 = 0 -> AG 0      200 % 2 = 0 -> AG 2

If AGs can not be evenly distributed among streams, the last stream will
absorb the remaining AGs.

Note that there are no hard boundaries; write-stream only provides explicit
routing hint to xfs allocator. We still preserve file contiguity, and the
full space can be utilized even with a single stream.

Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
---
 fs/xfs/libxfs/xfs_bmap.c | 38 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 37 insertions(+), 1 deletion(-)

diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index fd1a3aa4ad3f..872a4abd2315 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -3203,6 +3203,38 @@ xfs_default_ag_set_size(
 	return min_t(xfs_agnumber_t, GENERIC_AG_SET_SZ, mp->m_sb.sb_agcount);
 }
 
+static xfs_agnumber_t
+xfs_inode_write_stream_ag_set(
+	struct xfs_inode	*ip,
+	xfs_agnumber_t		*target_agno)
+{
+	struct xfs_mount	*mp = ip->i_mount;
+	uint32_t		nr_streams = xfs_inode_max_write_streams(ip);
+	uint32_t		stream_id = ip->i_write_stream;
+	uint32_t		nr_ags = mp->m_sb.sb_agcount;
+	xfs_agnumber_t		set_size;
+
+
+	if (!nr_streams)
+		return xfs_default_ag_set_size(ip);
+
+	stream_id -= 1; /* For 0-based math; stream-ids are 1-based */
+	set_size = nr_ags / nr_streams;
+
+	if (set_size) {
+		*target_agno = stream_id * set_size;
+		/* uneven distribution, last stream will cover extra AGs */
+		if (stream_id == nr_streams - 1)
+			set_size = nr_ags - *target_agno;
+	} else {
+		/* for the case when we have fewer AGs than streams */
+		*target_agno = stream_id % nr_ags;
+		set_size = 1;
+	}
+
+	return set_size;
+}
+
 static xfs_agnumber_t
 xfs_ag_to_ag_set(
 	struct xfs_bmalloca	*ap,
@@ -3216,7 +3248,11 @@ xfs_ag_to_ag_set(
 	if (!(ap->datatype & XFS_ALLOC_USERDATA))
 		return base_agno;
 
-	set_size = xfs_default_ag_set_size(ip);
+	if (ip->i_write_stream)
+		set_size = xfs_inode_write_stream_ag_set(ip, &base_agno);
+	else
+		set_size = xfs_default_ag_set_size(ip);
+
 	/* Fan out within the AG set using low bits of the inode */
 	return (base_agno + (XFS_INO_TO_AGINO(mp, I_INO(ip)) % set_size)) %
 		mp->m_sb.sb_agcount;
-- 
2.25.1

[PATCH v4 6/6] xfs: introduce software write streams

[Kanchan Joshi]

Even when the underlying block device does not advertise write streams,
XFS can choose do so, as that enables logical spatial isolation and
dynamic AG-set based concurrency for the standard storage, excluding
rtvolume.

Use AG count based heuristic to derive AG set size and software streams.
Larger filesystem (i.e., more AGs) get wider fanout (i.e., larger AG-set).

XFS_SW_WRITE_STREAMS_MAX bounds the software fallback. The mount-time
write stream bitmap allocation now also falls back to it when the data device
reports no hardware streams.

Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
Signed-off-by: Anuj Gupta <anuj20.g@samsung.com>
---
 fs/xfs/xfs_inode.c | 22 +++++++++++++++++++++-
 fs/xfs/xfs_mount.h |  2 ++
 fs/xfs/xfs_super.c | 12 ++++++------
 3 files changed, 29 insertions(+), 7 deletions(-)

diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
index aafc3ffa6e0a..638bee3e4ac2 100644
--- a/fs/xfs/xfs_inode.c
+++ b/fs/xfs/xfs_inode.c
@@ -53,6 +53,9 @@ xfs_inode_max_write_streams(
 	struct xfs_inode	*ip)
 {
 	struct block_device	*bdev;
+	struct xfs_mount	*mp = ip->i_mount;
+	int nr_streams;
+	xfs_agnumber_t nr_ags, ag_set_size;
 	bool			is_filestream, is_realtime;
 
 	xfs_ilock(ip, XFS_ILOCK_SHARED);
@@ -64,7 +67,24 @@ xfs_inode_max_write_streams(
 	if (!bdev || is_filestream || is_realtime)
 		return 0;
 
-	return bdev_max_write_streams(bdev);
+	nr_streams = bdev_max_write_streams(bdev);
+	if (nr_streams > 0)
+		return nr_streams;
+	/*
+	 * Enable software-only streams if hardware streams are not available.
+	 * This helps to
+	 * - improve isolation; reduce allocation interleaving.
+	 * - improve concurrency using AG-set based steering within and across streams.
+	 */
+	nr_ags = mp->m_sb.sb_agcount;
+	if (nr_ags >= 16)
+		ag_set_size = 4;
+	else if (nr_ags >= 8)
+		ag_set_size = 2;
+	else
+		ag_set_size = 1;
+	nr_streams = nr_ags / ag_set_size;
+	return min_t(uint16_t, nr_streams, XFS_SW_WRITE_STREAMS_MAX);
 }
 
 uint16_t
diff --git a/fs/xfs/xfs_mount.h b/fs/xfs/xfs_mount.h
index 1376963fb7e8..ad51115594d3 100644
--- a/fs/xfs/xfs_mount.h
+++ b/fs/xfs/xfs_mount.h
@@ -45,6 +45,8 @@ enum {
 
 #define XFS_ERR_RETRY_FOREVER	-1
 
+#define XFS_SW_WRITE_STREAMS_MAX	16
+
 /*
  * Although retry_timeout is in jiffies which is normally an unsigned long,
  * we limit the retry timeout to 86400 seconds, or one day.  So even a
diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c
index 0e2bf6f7b378..d698be504c45 100644
--- a/fs/xfs/xfs_super.c
+++ b/fs/xfs/xfs_super.c
@@ -1711,12 +1711,12 @@ xfs_fs_fill_super(
 		return error;
 
 	nr_streams = bdev_max_write_streams(mp->m_ddev_targp->bt_bdev);
-	if (nr_streams) {
-		mp->m_streams_in_use = bitmap_zalloc(nr_streams, GFP_KERNEL);
-		if (!mp->m_streams_in_use) {
-			error = -ENOMEM;
-			goto out_shutdown_devices;
-		}
+	if (!nr_streams)
+		nr_streams = XFS_SW_WRITE_STREAMS_MAX;
+	mp->m_streams_in_use = bitmap_zalloc(nr_streams, GFP_KERNEL);
+	if (!mp->m_streams_in_use) {
+		error = -ENOMEM;
+		goto out_shutdown_devices;
 	}
 
 	if (xfs_debugfs) {
-- 
2.25.1

Re: [PATCH v4 1/6] fs: add write-stream management ioctls

[Darrick J. Wong]

On Fri, Jul 17, 2026 at 06:25:33PM +0530, Kanchan Joshi wrote:
> From: Anuj Gupta <anuj20.g@samsung.com>
> 
> Wire up the userspace interface for write stream management via four
> ioctls:
> 
>   FS_IOC_WRITE_STREAM_GET_MAX  query the max streams supported
>   FS_IOC_WRITE_STREAM_OPEN     open a stream id, returns a stream fd
>   FS_IOC_WRITE_STREAM_SET      attach the stream fd to an open file
>   FS_IOC_WRITE_STREAM_GET      query the stream id value set on a file
> 
> Application should query the available streams by using
> FS_IOC_WRITE_STREAM_GET_MAX. If returned value is N, valid stream id for
> the file are 1 to N.
> 
> Application calls FS_IOC_WRITE_STREAM_OPEN to get fd for a stream-id.
> By default, kernel picks an available stream-id and returns the fd for
> it. The flag FS_WRITE_STREAM_OPEN_EXACT can be used to request a
> specific stream_id. This is useful if application cares about keeping a
> stable stream-id-to-spatial-isolation-bucket mapping across restarts.

Hmm.  If FS_IOC_WRITE_STREAM_OPEN returns an fd that represents an open
stream id, then why does STREAM_SET below take a pointer to a signed
s32?  And why does STREAM_GET take a pointer to a u32?

IOWs it'd be much easier to distinguish these things if the ioctls took
pointers to structs instead of u32/s32 pointers directly.

Also it's a little weird that STREAM_SET associates an open file with a
stream fd, but STREAM_GET returns the stream *id* (not the fd)
associated with an open file.

Can you extract the stream id from the fd that FS_IOC_WRITE_STREAM_OPEN
returns?

--D

> Suggested-by: Christoph Hellwig <hch@lst.de>
> Co-developed-by: Kanchan Joshi <joshi.k@samsung.com>
> Signed-off-by: Anuj Gupta <anuj20.g@samsung.com>
> Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
> ---
>  include/uapi/linux/fs.h | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
> 
> diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
> index bd87262f2e34..a756d27bdf84 100644
> --- a/include/uapi/linux/fs.h
> +++ b/include/uapi/linux/fs.h
> @@ -345,6 +345,22 @@ struct file_attr {
>  /* Get logical block metadata capability details */
>  #define FS_IOC_GETLBMD_CAP		_IOWR(0x15, 2, struct logical_block_metadata_cap)
>  
> +struct fs_write_stream_open {
> +	__u32		stream_id;	/* IN: requested id if OPEN_EXACT set; OUT: assigned id */
> +	__u32		flags;		/* IN: FS_WRITE_STREAM_OPEN_* */
> +};
> +
> +/*
> + * Flag to ask for specific stream_id, otherwise kernel can return any.
> + * Fails with -EBUSY if that stream_id is already held
> + */
> +#define FS_WRITE_STREAM_OPEN_EXACT	(1 << 0)
> +
> +#define FS_IOC_WRITE_STREAM_GET_MAX	_IOR('f', 135, __u32)
> +#define FS_IOC_WRITE_STREAM_OPEN	_IOWR('f', 136, struct fs_write_stream_open)
> +#define FS_IOC_WRITE_STREAM_SET		_IOW('f', 137, __s32)
> +#define FS_IOC_WRITE_STREAM_GET		_IOR('f', 138, __u32)
> +
>  /*
>   * Inode flags (FS_IOC_GETFLAGS / FS_IOC_SETFLAGS)
>   *
> -- 
> 2.25.1
> 
> 

Re: [PATCH v4 3/6] xfs: implement write-stream management support

[Darrick J. Wong]

On Fri, Jul 17, 2026 at 06:25:35PM +0530, Kanchan Joshi wrote:
> From: Anuj Gupta <anuj1072538@gmail.com>
> 
> Implement FS_IOC_WRITE_STREAM_{GET_MAX,OPEN,SET,GET} handlers.
> 
> GET_MAX reports the max write stream ids.
> OPEN allocates an fd for the unused write-stream and returns that.
> SET binds that fd to an open file.
> GET reports the stream-id value set on the file.
> 
> To track used write-streams, a per-mount bitmap is kept.
> The stream fd's release handler clears the bit when the last reference
> drops.
> 
> A new i_write_stream field on xfs_inode holds the bound stream id and
> is propagated to the iomap during block mapping.
> 
> Write streams, filestreams, and write-life-time hints are mutually exclusive;
> combining any two of them returns -EINVAL:
>   - GET_MAX reports 0 whenever xfs_inode_is_filestream() is true,
>     covering both mount-wide filestreams and the per-inode chattr
>     flag. Also when the file is on the realtime device.
>   - SET refuses to bind a stream to a file that already has a
>     write-life-time hint (fcntl F_SET_RW_HINT), is filestream, or is
>     on the realtime device.
>   - chattr refuses to set the filestream or realtime flag on a file
>     that already has a write stream set.

These special "files" that represent stream ids could be generic code
instaed of in xfs.  AFAICT the only thing you need from xfs is a pointer
from struct xfs_inode to struct (xfs_)write_stream, right?

> Suggested-by: Christoph Hellwig <hch@lst.de>
> Co-developed-by: Kanchan Joshi <joshi.k@samsung.com>
> Signed-off-by: Anuj Gupta <anuj20.g@samsung.com>
> Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
> ---
>  fs/xfs/xfs_icache.c |   1 +
>  fs/xfs/xfs_inode.c  | 155 ++++++++++++++++++++++++++++++++++++++++++++
>  fs/xfs/xfs_inode.h  |   8 +++
>  fs/xfs/xfs_ioctl.c  |  69 ++++++++++++++++++++
>  fs/xfs/xfs_iomap.c  |   1 +
>  fs/xfs/xfs_mount.h  |   3 +
>  fs/xfs/xfs_super.c  |  12 ++++
>  7 files changed, 249 insertions(+)
> 
> diff --git a/fs/xfs/xfs_icache.c b/fs/xfs/xfs_icache.c
> index 9d8dd30bd927..7b9dda74122f 100644
> --- a/fs/xfs/xfs_icache.c
> +++ b/fs/xfs/xfs_icache.c
> @@ -129,6 +129,7 @@ xfs_inode_alloc(
>  	spin_lock_init(&ip->i_ioend_lock);
>  	ip->i_next_unlinked = NULLAGINO;
>  	ip->i_prev_unlinked = 0;
> +	ip->i_write_stream = 0;
>  
>  	return ip;
>  }
> diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
> index 15279d22a894..aafc3ffa6e0a 100644
> --- a/fs/xfs/xfs_inode.c
> +++ b/fs/xfs/xfs_inode.c
> @@ -4,6 +4,7 @@
>   * All Rights Reserved.
>   */
>  #include <linux/iversion.h>
> +#include <linux/anon_inodes.h>
>  
>  #include "xfs_platform.h"
>  #include "xfs_fs.h"
> @@ -47,6 +48,160 @@
>  
>  struct kmem_cache *xfs_inode_cache;
>  
> +int
> +xfs_inode_max_write_streams(
> +	struct xfs_inode	*ip)
> +{
> +	struct block_device	*bdev;
> +	bool			is_filestream, is_realtime;
> +
> +	xfs_ilock(ip, XFS_ILOCK_SHARED);
> +	is_filestream = xfs_inode_is_filestream(ip);
> +	is_realtime = XFS_IS_REALTIME_INODE(ip);
> +	bdev = xfs_inode_buftarg(ip)->bt_bdev;
> +	xfs_iunlock(ip, XFS_ILOCK_SHARED);
> +
> +	if (!bdev || is_filestream || is_realtime)

Would be nice if realtime worked, or someone at least adds a comment
about why it isn't (e.g. "we have something more exciting for rt/zoned
filesystems") etc.

--D

> +		return 0;
> +
> +	return bdev_max_write_streams(bdev);
> +}
> +
> +uint16_t
> +xfs_inode_get_write_stream(
> +	struct xfs_inode	*ip)
> +{
> +	uint16_t	stream_id;
> +
> +	xfs_ilock(ip, XFS_ILOCK_SHARED);
> +	stream_id = ip->i_write_stream;
> +	xfs_iunlock(ip, XFS_ILOCK_SHARED);
> +
> +	return stream_id;
> +}
> +
> +struct xfs_write_stream {
> +	struct xfs_mount	*mp;
> +	uint16_t		stream_id;	/* 1-based */
> +};
> +
> +static int
> +xfs_write_stream_release(
> +	struct inode		*inode,
> +	struct file		*file)
> +{
> +	struct xfs_write_stream	*ws = file->private_data;
> +	struct xfs_mount	*mp = ws->mp;
> +
> +	spin_lock(&mp->m_streams_lock);
> +	clear_bit(ws->stream_id - 1, mp->m_streams_in_use);
> +	spin_unlock(&mp->m_streams_lock);
> +	kfree(ws);
> +	return 0;
> +}
> +
> +static const struct file_operations xfs_write_stream_fops = {
> +	.release	= xfs_write_stream_release,
> +	.llseek		= noop_llseek,
> +};
> +
> +int
> +xfs_inode_write_stream_open(
> +	struct xfs_inode	*ip,
> +	u32			flags,
> +	u32			*stream_idp)
> +{
> +	struct xfs_mount	*mp = ip->i_mount;
> +	struct xfs_write_stream	*ws;
> +	int			max, slot, fd, ret;
> +
> +	if (flags & ~FS_WRITE_STREAM_OPEN_EXACT)
> +		return -EINVAL;
> +
> +	max = xfs_inode_max_write_streams(ip);
> +	if (!max)
> +		return -EOPNOTSUPP;
> +	ASSERT(mp->m_streams_in_use);
> +
> +	ws = kmalloc(sizeof(*ws), GFP_KERNEL);
> +	if (!ws)
> +		return -ENOMEM;
> +
> +	spin_lock(&mp->m_streams_lock);
> +	if (flags & FS_WRITE_STREAM_OPEN_EXACT) {
> +		if (!*stream_idp || *stream_idp > max) {
> +			ret = -EINVAL;
> +			goto out_unlock;
> +		}
> +		slot = *stream_idp - 1;
> +		if (test_bit(slot, mp->m_streams_in_use)) {
> +			ret = -EBUSY;
> +			goto out_unlock;
> +		}
> +	} else {
> +		slot = find_first_zero_bit(mp->m_streams_in_use, max);
> +		if (slot >= max) {
> +			ret = -EBUSY;
> +			goto out_unlock;
> +		}
> +	}
> +	set_bit(slot, mp->m_streams_in_use);
> +	spin_unlock(&mp->m_streams_lock);
> +
> +	ws->mp = mp;
> +	ws->stream_id = slot + 1;	/* convert to 1-based */
> +
> +	fd = anon_inode_getfd("[xfs_write_stream]", &xfs_write_stream_fops, ws,
> +			      O_RDONLY | O_CLOEXEC);
> +	if (fd < 0) {
> +		spin_lock(&mp->m_streams_lock);
> +		clear_bit(slot, mp->m_streams_in_use);
> +		spin_unlock(&mp->m_streams_lock);
> +		kfree(ws);
> +		return fd;
> +	}
> +
> +	*stream_idp = ws->stream_id;
> +	return fd;
> +
> +out_unlock:
> +	spin_unlock(&mp->m_streams_lock);
> +	kfree(ws);
> +	return ret;
> +}
> +
> +int
> +xfs_inode_set_write_stream(
> +	struct xfs_inode	*ip,
> +	int			stream_fd)
> +{
> +	CLASS(fd, f)(stream_fd);
> +	struct xfs_write_stream	*ws;
> +	int			ret = 0;
> +
> +	if (!fd_file(f))
> +		return -EBADF;
> +	if (fd_file(f)->f_op != &xfs_write_stream_fops)
> +		return -EINVAL;
> +
> +	ws = fd_file(f)->private_data;
> +	if (ws->mp != ip->i_mount)
> +		return -EINVAL;
> +
> +	xfs_ilock(ip, XFS_ILOCK_EXCL);
> +
> +	if (XFS_IS_REALTIME_INODE(ip) || xfs_inode_is_filestream(ip) ||
> +	    VFS_I(ip)->i_write_hint != WRITE_LIFE_NOT_SET) {
> +		ret = -EINVAL;
> +		goto out_unlock;
> +	}
> +
> +	ip->i_write_stream = ws->stream_id;
> +out_unlock:
> +	xfs_iunlock(ip, XFS_ILOCK_EXCL);
> +	return ret;
> +}
> +
>  /*
>   * These two are wrapper routines around the xfs_ilock() routine used to
>   * centralize some grungy code.  They are used in places that wish to lock the
> diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h
> index 34c1038ebfcd..6abf82ffbf82 100644
> --- a/fs/xfs/xfs_inode.h
> +++ b/fs/xfs/xfs_inode.h
> @@ -37,6 +37,9 @@ typedef struct xfs_inode {
>  	struct xfs_ifork	i_df;		/* data fork */
>  	struct xfs_ifork	i_af;		/* attribute fork */
>  
> +	/* Write stream information */
> +	uint16_t		i_write_stream;
> +
>  	/* Transaction and locking information. */
>  	struct xfs_inode_log_item *i_itemp;	/* logging information */
>  	struct rw_semaphore	i_lock;		/* inode lock */
> @@ -673,4 +676,9 @@ int xfs_icreate_dqalloc(const struct xfs_icreate_args *args,
>  		struct xfs_dquot **udqpp, struct xfs_dquot **gdqpp,
>  		struct xfs_dquot **pdqpp);
>  
> +int xfs_inode_max_write_streams(struct xfs_inode *ip);
> +uint16_t xfs_inode_get_write_stream(struct xfs_inode *ip);
> +int xfs_inode_write_stream_open(struct xfs_inode *ip, u32 flags,
> +		u32 *stream_idp);
> +int xfs_inode_set_write_stream(struct xfs_inode *ip, int stream_fd);
>  #endif	/* __XFS_INODE_H__ */
> diff --git a/fs/xfs/xfs_ioctl.c b/fs/xfs/xfs_ioctl.c
> index 1b53701bebea..8640e6389768 100644
> --- a/fs/xfs/xfs_ioctl.c
> +++ b/fs/xfs/xfs_ioctl.c
> @@ -557,6 +557,12 @@ xfs_ioctl_setattr_xflags(
>  	bool			rtflag = (fa->fsx_xflags & FS_XFLAG_REALTIME);
>  	uint64_t		i_flags2;
>  
> +	if ((fa->fsx_xflags & FS_XFLAG_FILESTREAM) && ip->i_write_stream)
> +		return -EINVAL;
> +
> +	if (rtflag && ip->i_write_stream)
> +		return -EINVAL;
> +
>  	if (rtflag != XFS_IS_REALTIME_INODE(ip)) {
>  		/* Can't change realtime flag if any extents are allocated. */
>  		if (xfs_inode_has_filedata(ip))
> @@ -1200,6 +1206,59 @@ xfs_ioctl_fs_counts(
>  	return 0;
>  }
>  
> +static int
> +xfs_ioc_write_stream_open(
> +	struct file		*filp,
> +	void __user		*arg)
> +{
> +	struct xfs_inode	*ip = XFS_I(file_inode(filp));
> +	struct fs_write_stream_open wso;
> +	int			fd;
> +
> +	if (copy_from_user(&wso, arg, sizeof(wso)))
> +		return -EFAULT;
> +
> +	fd = xfs_inode_write_stream_open(ip, wso.flags, &wso.stream_id);
> +	if (fd < 0)
> +		return fd;
> +
> +	if (copy_to_user(arg, &wso, sizeof(wso)))
> +		return -EFAULT;
> +	return fd;
> +}
> +
> +static int
> +xfs_ioc_write_stream_set(
> +	struct file		*filp,
> +	unsigned long		arg)
> +{
> +	struct xfs_inode	*ip = XFS_I(file_inode(filp));
> +
> +	if (!(filp->f_mode & FMODE_WRITE))
> +		return -EBADF;
> +	return xfs_inode_set_write_stream(ip, (int)arg);
> +}
> +
> +static int
> +xfs_ioc_write_stream_get(
> +	struct xfs_inode	*ip,
> +	void __user		*arg)
> +{
> +	__u32 stream_id = xfs_inode_get_write_stream(ip);
> +
> +	return put_user(stream_id, (__u32 __user *)arg);
> +}
> +
> +static int
> +xfs_ioc_write_stream_get_max(
> +	struct xfs_inode	*ip,
> +	void __user		*arg)
> +{
> +	__u32 nr_streams = xfs_inode_max_write_streams(ip);
> +
> +	return put_user(nr_streams, (__u32 __user *)arg);
> +}
> +
>  /*
>   * These long-unused ioctls were removed from the official ioctl API in 5.17,
>   * but retain these definitions so that we can log warnings about them.
> @@ -1465,6 +1524,16 @@ xfs_file_ioctl(
>  		return xfs_ioc_health_monitor(filp, arg);
>  	case XFS_IOC_VERIFY_MEDIA:
>  		return xfs_ioc_verify_media(filp, arg);
> +	case FS_IOC_WRITE_STREAM_OPEN:
> +		return xfs_ioc_write_stream_open(filp, (void __user *)arg);
> +	case FS_IOC_WRITE_STREAM_SET:
> +		return xfs_ioc_write_stream_set(filp, p);
> +	case FS_IOC_WRITE_STREAM_GET:
> +		return xfs_ioc_write_stream_get(XFS_I(file_inode(filp)),
> +						(void __user *)arg);
> +	case FS_IOC_WRITE_STREAM_GET_MAX:
> +		return xfs_ioc_write_stream_get_max(XFS_I(file_inode(filp)),
> +						    (void __user *)arg);
>  
>  	default:
>  		return -ENOTTY;
> diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
> index 225c3de88d03..bf423897d916 100644
> --- a/fs/xfs/xfs_iomap.c
> +++ b/fs/xfs/xfs_iomap.c
> @@ -144,6 +144,7 @@ xfs_bmbt_to_iomap(
>  	}
>  	iomap->offset = XFS_FSB_TO_B(mp, imap->br_startoff);
>  	iomap->length = XFS_FSB_TO_B(mp, imap->br_blockcount);
> +	iomap->write_stream = ip->i_write_stream;
>  	if (mapping_flags & IOMAP_DAX) {
>  		iomap->dax_dev = target->bt_daxdev;
>  	} else {
> diff --git a/fs/xfs/xfs_mount.h b/fs/xfs/xfs_mount.h
> index 66a02d1b9ad7..1376963fb7e8 100644
> --- a/fs/xfs/xfs_mount.h
> +++ b/fs/xfs/xfs_mount.h
> @@ -349,6 +349,9 @@ typedef struct xfs_mount {
>  
>  	/* Index of uuid record in the uuid xarray. */
>  	unsigned int		m_uuid_table_index;
> +
> +	unsigned long		*m_streams_in_use;
> +	spinlock_t		m_streams_lock;
>  } xfs_mount_t;
>  
>  #define M_IGEO(mp)		(&(mp)->m_ino_geo)
> diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c
> index 8531d526fc44..0e2bf6f7b378 100644
> --- a/fs/xfs/xfs_super.c
> +++ b/fs/xfs/xfs_super.c
> @@ -805,6 +805,7 @@ xfs_mount_free(
>  #ifdef DEBUG
>  	kfree(mp->m_errortag);
>  #endif
> +	bitmap_free(mp->m_streams_in_use);
>  	kfree(mp);
>  }
>  
> @@ -1659,6 +1660,7 @@ xfs_fs_fill_super(
>  	struct xfs_mount	*mp = sb->s_fs_info;
>  	struct inode		*root;
>  	int			flags = 0, error;
> +	int			nr_streams;
>  
>  	mp->m_super = sb;
>  
> @@ -1708,6 +1710,15 @@ xfs_fs_fill_super(
>  	if (error)
>  		return error;
>  
> +	nr_streams = bdev_max_write_streams(mp->m_ddev_targp->bt_bdev);
> +	if (nr_streams) {
> +		mp->m_streams_in_use = bitmap_zalloc(nr_streams, GFP_KERNEL);
> +		if (!mp->m_streams_in_use) {
> +			error = -ENOMEM;
> +			goto out_shutdown_devices;
> +		}
> +	}
> +
>  	if (xfs_debugfs) {
>  		mp->m_debugfs = xfs_debugfs_mkdir(mp->m_super->s_id,
>  						  xfs_debugfs);
> @@ -2249,6 +2260,7 @@ xfs_init_fs_context(
>  #endif
>  
>  	spin_lock_init(&mp->m_sb_lock);
> +	spin_lock_init(&mp->m_streams_lock);
>  	for (i = 0; i < XG_TYPE_MAX; i++)
>  		xa_init(&mp->m_groups[i].xa);
>  	mutex_init(&mp->m_growlock);
> -- 
> 2.25.1
> 
> 

Re: [PATCH v4 4/6] xfs: generic AG set based steering

[Darrick J. Wong]

On Fri, Jul 17, 2026 at 06:25:36PM +0530, Kanchan Joshi wrote:
> Improve allocator concurrency and reduce interleaving by introducing
> fixed sized AG set.
> Use low bits of the inode as a hash to select AG within the AG set.
> Overall, a file will try to use the same AG (and contiguity is maintained),
> but multiple files will be spread across all AGs in the target AG set.
> 
> Suggested-by: Dave Chinner <dgc@kernel.org>
> Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
> Signed-off-by: Anuj Gupta <anuj20.g@samsung.com>
> ---
>  fs/xfs/libxfs/xfs_bmap.c | 38 ++++++++++++++++++++++++++++++++++++++
>  1 file changed, 38 insertions(+)
> 
> diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
> index d64defeda645..fd1a3aa4ad3f 100644
> --- a/fs/xfs/libxfs/xfs_bmap.c
> +++ b/fs/xfs/libxfs/xfs_bmap.c
> @@ -3192,6 +3192,36 @@ xfs_bmap_select_minlen(
>  	return args->maxlen;
>  }
>  
> +#define	GENERIC_AG_SET_SZ	(2)

What does this define?

> +
> +static inline xfs_agnumber_t
> +xfs_default_ag_set_size(
> +	struct xfs_inode	*ip)
> +{
> +	struct xfs_mount	*mp = ip->i_mount;
> +
> +	return min_t(xfs_agnumber_t, GENERIC_AG_SET_SZ, mp->m_sb.sb_agcount);

Because I'm not sure what it means on a single-AG filesystem.

> +}
> +
> +static xfs_agnumber_t
> +xfs_ag_to_ag_set(
> +	struct xfs_bmalloca	*ap,
> +	xfs_agnumber_t		base_agno)
> +{
> +	struct xfs_inode	*ip = ap->ip;
> +	struct xfs_mount	*mp = ip->i_mount;
> +	xfs_agnumber_t		set_size;
> +
> +	/* Apply fanning only for regular file data */
> +	if (!(ap->datatype & XFS_ALLOC_USERDATA))
> +		return base_agno;
> +
> +	set_size = xfs_default_ag_set_size(ip);
> +	/* Fan out within the AG set using low bits of the inode */
> +	return (base_agno + (XFS_INO_TO_AGINO(mp, I_INO(ip)) % set_size)) %
> +		mp->m_sb.sb_agcount;
> +}
> +
>  static int
>  xfs_bmap_btalloc_select_lengths(
>  	struct xfs_bmalloca	*ap,
> @@ -3587,8 +3617,16 @@ xfs_bmap_btalloc_best_length(
>  {
>  	xfs_extlen_t		blen = 0;
>  	int			error;
> +	xfs_agnumber_t		target_ag, start_ag;
>  
>  	ap->blkno = XFS_INODE_TO_FSB(ap->ip);
> +
> +	/* fan out initial AG across the generic AG set */
> +	start_ag = XFS_FSB_TO_AGNO(args->mp, ap->blkno);
> +	target_ag = xfs_ag_to_ag_set(ap, start_ag);
> +	if (target_ag != start_ag)
> +		ap->blkno = XFS_AGB_TO_FSB(args->mp, target_ag, 0);

/me wonders, if xfs_bmap_rtalloc looked at ap->blkno for a hint the way
that the data device allocator does, then would it be trivial to have
write streams on the rt device too?

I guess the tricky part would be figuring out what to do if you ever
want to switch a file between rt and data devices -- presumably you'd
just reset the write stream id to the default, but I guess you could
reject such a switch if the id had been set explicitly.

--D

> +
>  	if (!xfs_bmap_adjacent(ap))
>  		ap->eof = false;
>  
> -- 
> 2.25.1
> 
>