[RFC PATCH 0/5] FDP file I/O

[RFC PATCH 0/5] FDP file I/O

[Kanchan Joshi]

6.16 enables block I/O path for FDP, but file I/O needs further work.
This was covered in LSFMM [1]; among other things, we discussed
application and kernel-driven placement policies.

The series shows one of the ways to enable application-driven
placement, without resorting to per-FS plumbing.
But filesystem can control the number of streams that
applications get to use. This can be useful for fs-specific
policies or for metadata (placement) about which applications have no
knowledge.

Application interface involves three new fcntls:

- F_GET_MAX_WRITE_STREAMS: To query the number of streams that are
available.
- F_SET_WRITE_STREAM: To set a particular stream on a file.
- F_GET_WRITE_STREAM: To know what stream is set on the file.

An example program using these is attached below [2].

[1] https://lwn.net/Articles/1018642/

[2]
#define _GNU_SOURCE
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <sys/socket.h>
#include <errno.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <linux/fcntl.h>

#define BUF_SIZE        (4096)

int write_fd(int fd, int st) {

        char buf[BUF_SIZE];
        int ret, set, get;

        if (fd < 0) {
                printf("[!]invalid fd\n");
                return fd;
        }

        ret = fcntl(fd, F_GET_MAX_WRITE_STREAMS);
        if (ret < 0) {
                printf("F_GET_MAX_WRITE_STREAMS: failed (%s)\n", strerror(errno));
                return ret;
        }
        if (st > ret) {
                printf("error in setting stream, available upto %d\n", ret);
                return -1;
        }

        ret = fcntl(fd, F_SET_WRITE_STREAM, st);
        if (ret < 0) {
                printf("F_SET_WRITE_STREAM: failed (%s)\n", strerror(errno));
                return ret;
        }
        set = st;

        ret = fcntl(fd, F_GET_WRITE_STREAM);
        if (ret < 0) {
                printf("F_GET_WRITE_STREAM: failed (%s)\n", strerror(errno));
                return ret;
        }
        get = ret;

        if (get != set)
                printf("unexpected, set %d but get %d\n", set, get);

        ret = write(fd, buf, BUF_SIZE);
        if (ret < BUF_SIZE) {
                printf("failed, wrote %d bytes (expected %d)\n", ret, BUF_SIZE);
                return ret;
        }
        return 0;
}

int main(int argc, char *argv[])
{
        int ret, regfd;

        /* two file writes, one buffered another direct */
        regfd = open("/mnt/f_buffered", O_CREAT | O_RDWR, 0644);
        ret = write_fd(regfd, 7);
        close(regfd);

        regfd = open("/mnt/f_direct", O_CREAT | O_RDWR| O_DIRECT, 0644);
        ret = write_fd(regfd, 6);
        close(regfd);
        return ret;
}


Kanchan Joshi (5):
  fs: add a new user_write_streams() callback
  fs: add the interface to query user write streams
  fs: add a write stream field to the inode
  fs: propagate write stream
  fs: add set and query write stream

 fs/btrfs/extent_io.c       |  1 +
 fs/buffer.c                | 14 ++++++---
 fs/direct-io.c             |  1 +
 fs/ext4/page-io.c          |  1 +
 fs/fcntl.c                 | 64 ++++++++++++++++++++++++++++++++++++++
 fs/inode.c                 |  1 +
 fs/iomap/direct-io.c       |  1 +
 fs/iomap/ioend.c           |  1 +
 fs/mpage.c                 |  1 +
 include/linux/fs.h         |  8 ++++-
 include/uapi/linux/fcntl.h |  7 +++++
 11 files changed, 94 insertions(+), 6 deletions(-)


base-commit: 66639db858112bf6b0f76677f7517643d586e575
-- 
2.25.1

[RFC PATCH 1/5] fs: add a new user_write_streams() callback

[Kanchan Joshi]

so that filesystem can control number of write streams for user space.

Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
---
 include/linux/fs.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/include/linux/fs.h b/include/linux/fs.h
index 1948b2c828d3..b1d514901bf8 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -2376,6 +2376,11 @@ struct super_operations {
 	 */
 	int (*remove_bdev)(struct super_block *sb, struct block_device *bdev);
 	void (*shutdown)(struct super_block *sb);
+	/*
+	 * Implement this callback if filesystem wants to control the
+	 * number of streams that are available to user space.
+	 */
+	u8 (*user_write_streams)(struct super_block *sb);
 };
 
 /*
-- 
2.25.1

[RFC PATCH 2/5] fs: add the interface to query user write streams

[Kanchan Joshi]

Add new fcntl F_GET_MAX_WRITE_STREAMS.
This returns the numbers of streams that are available for userspace.

And for that, use ->user_write_streams() callback when the involved
filesystem provides it.
In absence of such callback, use 'max_write_streams' queue limit of the
underlying block device.

Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
---
 fs/fcntl.c                 | 31 +++++++++++++++++++++++++++++++
 include/uapi/linux/fcntl.h |  5 +++++
 2 files changed, 36 insertions(+)

diff --git a/fs/fcntl.c b/fs/fcntl.c
index 5598e4d57422..36ca833e9a0b 100644
--- a/fs/fcntl.c
+++ b/fs/fcntl.c
@@ -27,6 +27,7 @@
 #include <linux/compat.h>
 #include <linux/mount.h>
 #include <linux/rw_hint.h>
+#include <linux/blkdev.h>
 
 #include <linux/poll.h>
 #include <asm/siginfo.h>
@@ -394,6 +395,33 @@ static long fcntl_set_rw_hint(struct file *file, unsigned int cmd,
 	return 0;
 }
 
+static u8 vfs_user_write_streams(struct inode *inode)
+{
+	struct super_block *sb;
+
+	if (S_ISBLK(inode->i_mode))
+		return bdev_max_write_streams(I_BDEV(inode));
+
+	sb = inode->i_sb;
+	/* If available, use per-mount/fs policy */
+	if (sb->s_op && sb->s_op->user_write_streams)
+		return sb->s_op->user_write_streams(sb);
+	/* otherwise, fallback to queue limit */
+	if (sb->s_bdev)
+		return bdev_max_write_streams(sb->s_bdev);
+	return 0;
+}
+
+static long fcntl_get_max_write_streams(struct file *file)
+{
+	struct inode *inode = file_inode(file);
+
+	if (S_ISBLK(inode->i_mode))
+		inode = file->f_mapping->host;
+
+	return vfs_user_write_streams(inode);
+}
+
 /* Is the file descriptor a dup of the file? */
 static long f_dupfd_query(int fd, struct file *filp)
 {
@@ -552,6 +580,9 @@ static long do_fcntl(int fd, unsigned int cmd, unsigned long arg,
 	case F_SET_RW_HINT:
 		err = fcntl_set_rw_hint(filp, cmd, arg);
 		break;
+	case F_GET_MAX_WRITE_STREAMS:
+		err = fcntl_get_max_write_streams(filp);
+		break;
 	default:
 		break;
 	}
diff --git a/include/uapi/linux/fcntl.h b/include/uapi/linux/fcntl.h
index f291ab4f94eb..87ec808d0f03 100644
--- a/include/uapi/linux/fcntl.h
+++ b/include/uapi/linux/fcntl.h
@@ -61,6 +61,11 @@
 #define F_GET_FILE_RW_HINT	(F_LINUX_SPECIFIC_BASE + 13)
 #define F_SET_FILE_RW_HINT	(F_LINUX_SPECIFIC_BASE + 14)
 
+/*
+ *  Query available write streams
+ */
+#define F_GET_MAX_WRITE_STREAMS (F_LINUX_SPECIFIC_BASE + 15)
+
 /*
  * Valid hint values for F_{GET,SET}_RW_HINT. 0 is "not set", or can be
  * used to clear any hints previously set.
-- 
2.25.1

[RFC PATCH 3/5] fs: add a write stream field to the inode

[Kanchan Joshi]

Prepare for supporting per-inode write streams.
Part of the existing 32-bit hole is used for the new field.

Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
---
 fs/inode.c         | 1 +
 include/linux/fs.h | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/fs/inode.c b/fs/inode.c
index 01ebdc40021e..bb1a9a043b32 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -250,6 +250,7 @@ int inode_init_always_gfp(struct super_block *sb, struct inode *inode, gfp_t gfp
 	atomic_set(&inode->i_writecount, 0);
 	inode->i_size = 0;
 	inode->i_write_hint = WRITE_LIFE_NOT_SET;
+	inode->i_write_stream = 0;
 	inode->i_blocks = 0;
 	inode->i_bytes = 0;
 	inode->i_generation = 0;
diff --git a/include/linux/fs.h b/include/linux/fs.h
index b1d514901bf8..3c41ce0f641c 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -725,7 +725,8 @@ struct inode {
 
 	/* Misc */
 	u32			i_state;
-	/* 32-bit hole */
+	u8			i_write_stream;
+	/* 24-bit hole */
 	struct rw_semaphore	i_rwsem;
 
 	unsigned long		dirtied_when;	/* jiffies of first dirtying */
-- 
2.25.1

[RFC PATCH 4/5] fs: propagate write stream

[Kanchan Joshi]

bio->bi_write_stream is not set by the filesystem code.
Use inode's write stream value to do that.

Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
---
 fs/btrfs/extent_io.c |  1 +
 fs/buffer.c          | 14 +++++++++-----
 fs/direct-io.c       |  1 +
 fs/ext4/page-io.c    |  1 +
 fs/iomap/direct-io.c |  1 +
 fs/iomap/ioend.c     |  1 +
 fs/mpage.c           |  1 +
 7 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 1dc931c4937f..280fdfcfd855 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -666,6 +666,7 @@ static void alloc_new_bio(struct btrfs_inode *inode,
 			       bio_ctrl->end_io_func, NULL);
 	bbio->bio.bi_iter.bi_sector = disk_bytenr >> SECTOR_SHIFT;
 	bbio->bio.bi_write_hint = inode->vfs_inode.i_write_hint;
+	bbio->bio.bi_write_stream = inode->vfs_inode.i_write_stream;
 	bbio->inode = inode;
 	bbio->file_offset = file_offset;
 	bio_ctrl->bbio = bbio;
diff --git a/fs/buffer.c b/fs/buffer.c
index ead4dc85debd..6b85e9992036 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -55,7 +55,8 @@
 
 static int fsync_buffers_list(spinlock_t *lock, struct list_head *list);
 static void submit_bh_wbc(blk_opf_t opf, struct buffer_head *bh,
-			  enum rw_hint hint, struct writeback_control *wbc);
+			  enum rw_hint hint, u8 write_stream,
+			  struct writeback_control *wbc);
 
 #define BH_ENTRY(list) list_entry((list), struct buffer_head, b_assoc_buffers)
 
@@ -1931,7 +1932,8 @@ int __block_write_full_folio(struct inode *inode, struct folio *folio,
 		struct buffer_head *next = bh->b_this_page;
 		if (buffer_async_write(bh)) {
 			submit_bh_wbc(REQ_OP_WRITE | write_flags, bh,
-				      inode->i_write_hint, wbc);
+				      inode->i_write_hint,
+				      inode->i_write_stream, wbc);
 			nr_underway++;
 		}
 		bh = next;
@@ -1986,7 +1988,8 @@ int __block_write_full_folio(struct inode *inode, struct folio *folio,
 		if (buffer_async_write(bh)) {
 			clear_buffer_dirty(bh);
 			submit_bh_wbc(REQ_OP_WRITE | write_flags, bh,
-				      inode->i_write_hint, wbc);
+				      inode->i_write_hint,
+				      inode->i_write_stream, wbc);
 			nr_underway++;
 		}
 		bh = next;
@@ -2778,7 +2781,7 @@ static void end_bio_bh_io_sync(struct bio *bio)
 }
 
 static void submit_bh_wbc(blk_opf_t opf, struct buffer_head *bh,
-			  enum rw_hint write_hint,
+			  enum rw_hint write_hint, u8 write_stream,
 			  struct writeback_control *wbc)
 {
 	const enum req_op op = opf & REQ_OP_MASK;
@@ -2807,6 +2810,7 @@ static void submit_bh_wbc(blk_opf_t opf, struct buffer_head *bh,
 
 	bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >> 9);
 	bio->bi_write_hint = write_hint;
+	bio->bi_write_stream = write_stream;
 
 	bio_add_folio_nofail(bio, bh->b_folio, bh->b_size, bh_offset(bh));
 
@@ -2826,7 +2830,7 @@ static void submit_bh_wbc(blk_opf_t opf, struct buffer_head *bh,
 
 void submit_bh(blk_opf_t opf, struct buffer_head *bh)
 {
-	submit_bh_wbc(opf, bh, WRITE_LIFE_NOT_SET, NULL);
+	submit_bh_wbc(opf, bh, WRITE_LIFE_NOT_SET, 0, NULL);
 }
 EXPORT_SYMBOL(submit_bh);
 
diff --git a/fs/direct-io.c b/fs/direct-io.c
index 1694ee9a9382..f086d21b5b1c 100644
--- a/fs/direct-io.c
+++ b/fs/direct-io.c
@@ -410,6 +410,7 @@ dio_bio_alloc(struct dio *dio, struct dio_submit *sdio,
 	if (dio->is_pinned)
 		bio_set_flag(bio, BIO_PAGE_PINNED);
 	bio->bi_write_hint = file_inode(dio->iocb->ki_filp)->i_write_hint;
+	bio->bi_write_stream = file_inode(dio->iocb->ki_filp)->i_write_stream;
 
 	sdio->bio = bio;
 	sdio->logical_offset_in_bio = sdio->cur_page_fs_offset;
diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c
index 179e54f3a3b6..573093ecd7d9 100644
--- a/fs/ext4/page-io.c
+++ b/fs/ext4/page-io.c
@@ -447,6 +447,7 @@ static void io_submit_add_bh(struct ext4_io_submit *io,
 	if (io->io_bio == NULL) {
 		io_submit_init_bio(io, bh);
 		io->io_bio->bi_write_hint = inode->i_write_hint;
+		io->io_bio->bi_write_stream = inode->i_write_stream;
 	}
 	if (!bio_add_folio(io->io_bio, io_folio, bh->b_size, bh_offset(bh)))
 		goto submit_and_retry;
diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c
index 6f25d4cfea9f..ba304109da72 100644
--- a/fs/iomap/direct-io.c
+++ b/fs/iomap/direct-io.c
@@ -430,6 +430,7 @@ static int iomap_dio_bio_iter(struct iomap_iter *iter, struct iomap_dio *dio)
 					  GFP_KERNEL);
 		bio->bi_iter.bi_sector = iomap_sector(iomap, pos);
 		bio->bi_write_hint = inode->i_write_hint;
+		bio->bi_write_stream = inode->i_write_stream;
 		bio->bi_ioprio = dio->iocb->ki_ioprio;
 		bio->bi_private = dio;
 		bio->bi_end_io = iomap_dio_bio_end_io;
diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
index b49fa75eab26..652525c93fdd 100644
--- a/fs/iomap/ioend.c
+++ b/fs/iomap/ioend.c
@@ -107,6 +107,7 @@ static struct iomap_ioend *iomap_alloc_ioend(struct iomap_writepage_ctx *wpc,
 			       GFP_NOFS, &iomap_ioend_bioset);
 	bio->bi_iter.bi_sector = iomap_sector(&wpc->iomap, pos);
 	bio->bi_write_hint = wpc->inode->i_write_hint;
+	bio->bi_write_stream = wpc->inode->i_write_stream;
 	wbc_init_bio(wpc->wbc, bio);
 	wpc->nr_folios = 0;
 	return iomap_init_ioend(wpc->inode, bio, pos, ioend_flags);
diff --git a/fs/mpage.c b/fs/mpage.c
index c5fd821fd30e..6a50bbe38adc 100644
--- a/fs/mpage.c
+++ b/fs/mpage.c
@@ -595,6 +595,7 @@ static int mpage_write_folio(struct writeback_control *wbc, struct folio *folio,
 		bio->bi_iter.bi_sector = first_block << (blkbits - 9);
 		wbc_init_bio(wbc, bio);
 		bio->bi_write_hint = inode->i_write_hint;
+		bio->bi_write_stream = inode->i_write_stream;
 	}
 
 	/*
-- 
2.25.1

[RFC PATCH 5/5] fs: add set and query write stream

[Kanchan Joshi]

Add two new fcntls:
F_GET_WRITE_STREAM - to query the write-stream on inode
F_SET_WRITE_STREAM - to set the write-stream on inode

Application should query the available streams by calling
F_GET_MAX_WRITE_STREAMS first.
If returned value is N, applications can choose any value from 1 to N
while setting the stream.
Setting the value 0 is not flagged as an error as that implies no
stream.
But setting a larger value than available streams is rejected.

Signed-off-by: Kanchan Joshi <joshi.k@samsung.com>
---
 fs/fcntl.c                 | 33 +++++++++++++++++++++++++++++++++
 include/uapi/linux/fcntl.h |  2 ++
 2 files changed, 35 insertions(+)

diff --git a/fs/fcntl.c b/fs/fcntl.c
index 36ca833e9a0b..ce89393f8dbf 100644
--- a/fs/fcntl.c
+++ b/fs/fcntl.c
@@ -422,6 +422,33 @@ static long fcntl_get_max_write_streams(struct file *file)
 	return vfs_user_write_streams(inode);
 }
 
+static long fcntl_get_write_stream(struct file *file)
+{
+	struct inode *inode = file_inode(file);
+
+	if (S_ISBLK(inode->i_mode))
+		inode = file->f_mapping->host;
+
+	return inode->i_write_stream;
+}
+
+static long fcntl_set_write_stream(struct file *file, unsigned long arg)
+{
+	struct inode *inode = file_inode(file);
+
+	if (!inode_owner_or_capable(file_mnt_idmap(file), inode))
+		return -EPERM;
+
+	if (S_ISBLK(inode->i_mode))
+		inode = file->f_mapping->host;
+
+	if (arg > vfs_user_write_streams(inode))
+		return -EINVAL;
+
+	WRITE_ONCE(inode->i_write_stream, arg);
+	return 0;
+}
+
 /* Is the file descriptor a dup of the file? */
 static long f_dupfd_query(int fd, struct file *filp)
 {
@@ -583,6 +610,12 @@ static long do_fcntl(int fd, unsigned int cmd, unsigned long arg,
 	case F_GET_MAX_WRITE_STREAMS:
 		err = fcntl_get_max_write_streams(filp);
 		break;
+	case F_GET_WRITE_STREAM:
+		err = fcntl_get_write_stream(filp);
+		break;
+	case F_SET_WRITE_STREAM:
+		err = fcntl_set_write_stream(filp, arg);
+		break;
 	default:
 		break;
 	}
diff --git a/include/uapi/linux/fcntl.h b/include/uapi/linux/fcntl.h
index 87ec808d0f03..dd3c498515ce 100644
--- a/include/uapi/linux/fcntl.h
+++ b/include/uapi/linux/fcntl.h
@@ -65,6 +65,8 @@
  *  Query available write streams
  */
 #define F_GET_MAX_WRITE_STREAMS (F_LINUX_SPECIFIC_BASE + 15)
+#define F_GET_WRITE_STREAM	(F_LINUX_SPECIFIC_BASE + 16)
+#define F_SET_WRITE_STREAM	(F_LINUX_SPECIFIC_BASE + 17)
 
 /*
  * Valid hint values for F_{GET,SET}_RW_HINT. 0 is "not set", or can be
-- 
2.25.1

Re: [RFC PATCH 1/5] fs: add a new user_write_streams() callback

[Christoph Hellwig]

On Tue, Jul 29, 2025 at 08:21:31PM +0530, Kanchan Joshi wrote:
> so that filesystem can control number of write streams for user space.

This feels the wrong way around.  I'd rather implement the actual interface
to get/set the write streams in the file system (maybe using common
helpers) than encode the nunber in a file operation.

Re: [RFC PATCH 2/5] fs: add the interface to query user write streams

[Christoph Hellwig]

On Tue, Jul 29, 2025 at 08:21:32PM +0530, Kanchan Joshi wrote:
> Add new fcntl F_GET_MAX_WRITE_STREAMS.
> This returns the numbers of streams that are available for userspace.
> 
> And for that, use ->user_write_streams() callback when the involved
> filesystem provides it.
> In absence of such callback, use 'max_write_streams' queue limit of the
> underlying block device.

As mentioned in patch 1, I think we'd rather dispath the whole fcntl
to the file system, and then use generic helpers, which will give
more control of the details to the file system.

Re: [RFC PATCH 3/5] fs: add a write stream field to the inode

[Christoph Hellwig]

On Tue, Jul 29, 2025 at 08:21:33PM +0530, Kanchan Joshi wrote:
> Prepare for supporting per-inode write streams.
> Part of the existing 32-bit hole is used for the new field.

Bloating the inode for everyone for this is probably not a good idea.
I.e. you'd probably want this in the file system specific inodes for
file systems that can just do data placement.

Re: [RFC PATCH 4/5] fs: propagate write stream

[Christoph Hellwig]

On Tue, Jul 29, 2025 at 08:21:34PM +0530, Kanchan Joshi wrote:
> bio->bi_write_stream is not set by the filesystem code.
> Use inode's write stream value to do that.

Just passing it through is going to create problems.  i.e. when
the file system does it's own placement or reserves ids.  We'll need
an explicit intercept point between the user write stream and what
does into the bio.