[PATCH v3 0/5] io_uring: extend bvec registration and add mem region lookup

[PATCH v3 0/5] io_uring: extend bvec registration and add mem region lookup

[Joanne Koong]

This series refactors and extends the io_uring registered buffers
infrastructure to allow external subsystems to register pre-existing bvec
arrays directly and obtain a pointer to the registered memory region.

The motivation for the patches in this series is to make fuse zero-copy
possible.

These patches are split out from a previous larger fuse-over-io_uring series
[1]. The remaining fuse patches will be submitted separately and linked to.

Thanks,
Joanne

[1] https://lore.kernel.org/linux-fsdevel/20260116233044.1532965-1-joannelkoong@gmail.com/

Changelog:
v2: https://lore.kernel.org/io-uring/20260324182157.990864-1-joannelkoong@gmail.com/
v2 -> v3:
* drop patch that makes buffer release callback optional
* add patch for renaming/exporting IO_IMU_DEST / IO_IMU_SOURCE

v1: https://lore.kernel.org/io-uring/20260324001007.1144471-1-joannelkoong@gmail.com/
v1 -> v2:
* update io_kernel_buffer_init() to take bitmasked dir directly so callers can
  set both dest and source

Joanne Koong (5):
  io_uring/rsrc: rename
    io_buffer_register_bvec()/io_buffer_unregister_bvec()
  io_uring/rsrc: split io_buffer_register_request() logic
  io_uring/rsrc: add io_buffer_register_bvec()
  io_uring/rsrc: rename and export IO_IMU_DEST / IO_IMU_SOURCE
  io_uring/rsrc: add io_uring_registered_mem_region_get()

 Documentation/block/ublk.rst   |  14 ++--
 drivers/block/ublk_drv.c       |  22 ++---
 include/linux/io_uring/cmd.h   |  48 +++++++++--
 include/linux/io_uring_types.h |   5 ++
 io_uring/io_uring.c            |   2 +-
 io_uring/rsrc.c                | 145 ++++++++++++++++++++++++---------
 io_uring/rsrc.h                |   5 --
 7 files changed, 174 insertions(+), 67 deletions(-)

-- 
2.52.0

[PATCH v3 1/5] io_uring/rsrc: rename io_buffer_register_bvec()/io_buffer_unregister_bvec()

[Joanne Koong]

Currently, io_buffer_register_bvec() takes in a request. In preparation
for supporting kernel-populated buffers in fuse io-uring (which will
need to register bvecs directly, not through a struct request), rename
this to io_buffer_register_request().

A subsequent patch will commandeer the "io_buffer_register_bvec()"
function name to support registering bvecs directly.

Rename io_buffer_unregister_bvec() to a more generic name,
io_buffer_unregister(), as both io_buffer_register_request() and
io_buffer_register_bvec() callers will use it for unregistration.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Caleb Sander Mateos <csander@purestorage.com>
---
 Documentation/block/ublk.rst | 14 +++++++-------
 drivers/block/ublk_drv.c     | 22 +++++++++++-----------
 include/linux/io_uring/cmd.h | 26 ++++++++++++++++++++------
 io_uring/rsrc.c              | 14 +++++++-------
 4 files changed, 45 insertions(+), 31 deletions(-)

diff --git a/Documentation/block/ublk.rst b/Documentation/block/ublk.rst
index 6ad28039663d..f014d1d69019 100644
--- a/Documentation/block/ublk.rst
+++ b/Documentation/block/ublk.rst
@@ -382,17 +382,17 @@ Zero copy
 ---------
 
 ublk zero copy relies on io_uring's fixed kernel buffer, which provides
-two APIs: `io_buffer_register_bvec()` and `io_buffer_unregister_bvec`.
+two APIs: `io_buffer_register_request()` and `io_buffer_unregister`.
 
 ublk adds IO command of `UBLK_IO_REGISTER_IO_BUF` to call
-`io_buffer_register_bvec()` for ublk server to register client request
+`io_buffer_register_request()` for ublk server to register client request
 buffer into io_uring buffer table, then ublk server can submit io_uring
 IOs with the registered buffer index. IO command of `UBLK_IO_UNREGISTER_IO_BUF`
-calls `io_buffer_unregister_bvec()` to unregister the buffer, which is
-guaranteed to be live between calling `io_buffer_register_bvec()` and
-`io_buffer_unregister_bvec()`. Any io_uring operation which supports this
-kind of kernel buffer will grab one reference of the buffer until the
-operation is completed.
+calls `io_buffer_unregister()` to unregister the buffer, which is guaranteed
+to be live between calling `io_buffer_register_request()` and
+`io_buffer_unregister()`. Any io_uring operation which supports this kind of
+kernel buffer will grab one reference of the buffer until the operation is
+completed.
 
 ublk server implementing zero copy or user copy has to be CAP_SYS_ADMIN and
 be trusted, because it is ublk server's responsibility to make sure IO buffer
diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
index 004f367243b6..b9f293261240 100644
--- a/drivers/block/ublk_drv.c
+++ b/drivers/block/ublk_drv.c
@@ -1627,8 +1627,8 @@ ublk_auto_buf_register(const struct ublk_queue *ubq, struct request *req,
 {
 	int ret;
 
-	ret = io_buffer_register_bvec(cmd, req, ublk_io_release,
-				      io->buf.auto_reg.index, issue_flags);
+	ret = io_buffer_register_request(cmd, req, ublk_io_release,
+					 io->buf.auto_reg.index, issue_flags);
 	if (ret) {
 		if (io->buf.auto_reg.flags & UBLK_AUTO_BUF_REG_FALLBACK) {
 			ublk_auto_buf_reg_fallback(ubq, req->tag);
@@ -1868,7 +1868,7 @@ static int __ublk_batch_dispatch(struct ublk_queue *ubq,
 			ublk_io_unlock(io);
 
 			if (index != -1)
-				io_buffer_unregister_bvec(data->cmd, index,
+				io_buffer_unregister(data->cmd, index,
 						data->issue_flags);
 		}
 
@@ -3091,8 +3091,8 @@ static int ublk_register_io_buf(struct io_uring_cmd *cmd,
 	if (!req)
 		return -EINVAL;
 
-	ret = io_buffer_register_bvec(cmd, req, ublk_io_release, index,
-				      issue_flags);
+	ret = io_buffer_register_request(cmd, req, ublk_io_release, index,
+					 issue_flags);
 	if (ret) {
 		ublk_put_req_ref(io, req);
 		return ret;
@@ -3123,8 +3123,8 @@ ublk_daemon_register_io_buf(struct io_uring_cmd *cmd,
 	if (!ublk_dev_support_zero_copy(ub) || !ublk_rq_has_data(req))
 		return -EINVAL;
 
-	ret = io_buffer_register_bvec(cmd, req, ublk_io_release, index,
-				      issue_flags);
+	ret = io_buffer_register_request(cmd, req, ublk_io_release, index,
+					 issue_flags);
 	if (ret)
 		return ret;
 
@@ -3139,7 +3139,7 @@ static int ublk_unregister_io_buf(struct io_uring_cmd *cmd,
 	if (!(ub->dev_info.flags & UBLK_F_SUPPORT_ZERO_COPY))
 		return -EINVAL;
 
-	return io_buffer_unregister_bvec(cmd, index, issue_flags);
+	return io_buffer_unregister(cmd, index, issue_flags);
 }
 
 static int ublk_check_fetch_buf(const struct ublk_device *ub, __u64 buf_addr)
@@ -3280,7 +3280,7 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
 		goto out;
 
 	/*
-	 * io_buffer_unregister_bvec() doesn't access the ubq or io,
+	 * io_buffer_unregister() doesn't access the ubq or io,
 	 * so no need to validate the q_id, tag, or task
 	 */
 	if (_IOC_NR(cmd_op) == UBLK_IO_UNREGISTER_IO_BUF)
@@ -3347,7 +3347,7 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
 		req = ublk_fill_io_cmd(io, cmd);
 		ret = ublk_config_io_buf(ub, io, cmd, addr, &buf_idx);
 		if (buf_idx != UBLK_INVALID_BUF_IDX)
-			io_buffer_unregister_bvec(cmd, buf_idx, issue_flags);
+			io_buffer_unregister(cmd, buf_idx, issue_flags);
 		compl = ublk_need_complete_req(ub, io);
 
 		if (req_op(req) == REQ_OP_ZONE_APPEND)
@@ -3682,7 +3682,7 @@ static int ublk_batch_commit_io(struct ublk_queue *ubq,
 	}
 
 	if (buf_idx != UBLK_INVALID_BUF_IDX)
-		io_buffer_unregister_bvec(data->cmd, buf_idx, data->issue_flags);
+		io_buffer_unregister(data->cmd, buf_idx, data->issue_flags);
 	if (req_op(req) == REQ_OP_ZONE_APPEND)
 		req->__sector = ublk_batch_zone_lba(uc, elem);
 	if (compl)
diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h
index 89e1a80d9f5f..165d436073a4 100644
--- a/include/linux/io_uring/cmd.h
+++ b/include/linux/io_uring/cmd.h
@@ -106,6 +106,12 @@ bool io_uring_is_kmbuf_ring(struct io_uring_cmd *cmd, unsigned int buf_group,
 struct io_br_sel io_ring_buffer_select(struct io_kiocb *req, size_t *len,
 				       struct io_buffer_list *bl,
 				       unsigned int issue_flags);
+
+int io_buffer_register_request(struct io_uring_cmd *cmd, struct request *rq,
+			       void (*release)(void *), unsigned int index,
+			       unsigned int issue_flags);
+int io_buffer_unregister(struct io_uring_cmd *cmd, unsigned int index,
+			 unsigned int issue_flags);
 #else
 static inline int
 io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw,
@@ -184,6 +190,20 @@ static inline struct io_br_sel io_ring_buffer_select(struct io_kiocb *req,
 	};
 	return sel;
 }
+static inline int io_buffer_register_request(struct io_uring_cmd *cmd,
+					     struct request *rq,
+					     void (*release)(void *),
+					     unsigned int index,
+					     unsigned int issue_flags)
+{
+	return -EOPNOTSUPP;
+}
+static inline int io_buffer_unregister(struct io_uring_cmd *cmd,
+				       unsigned int index,
+				       unsigned int issue_flags)
+{
+	return -EOPNOTSUPP;
+}
 #endif
 
 static inline struct io_uring_cmd *io_uring_cmd_from_tw(struct io_tw_req tw_req)
@@ -233,10 +253,4 @@ static inline void io_uring_cmd_done32(struct io_uring_cmd *ioucmd, s32 ret,
 	return __io_uring_cmd_done(ioucmd, ret, res2, issue_flags, true);
 }
 
-int io_buffer_register_bvec(struct io_uring_cmd *cmd, struct request *rq,
-			    void (*release)(void *), unsigned int index,
-			    unsigned int issue_flags);
-int io_buffer_unregister_bvec(struct io_uring_cmd *cmd, unsigned int index,
-			      unsigned int issue_flags);
-
 #endif /* _LINUX_IO_URING_CMD_H */
diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
index 52554ed89b11..7579f6992a25 100644
--- a/io_uring/rsrc.c
+++ b/io_uring/rsrc.c
@@ -924,9 +924,9 @@ int io_sqe_buffers_register(struct io_ring_ctx *ctx, void __user *arg,
 	return ret;
 }
 
-int io_buffer_register_bvec(struct io_uring_cmd *cmd, struct request *rq,
-			    void (*release)(void *), unsigned int index,
-			    unsigned int issue_flags)
+int io_buffer_register_request(struct io_uring_cmd *cmd, struct request *rq,
+			       void (*release)(void *), unsigned int index,
+			       unsigned int issue_flags)
 {
 	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
 	struct io_rsrc_data *data = &ctx->buf_table;
@@ -986,10 +986,10 @@ int io_buffer_register_bvec(struct io_uring_cmd *cmd, struct request *rq,
 	io_ring_submit_unlock(ctx, issue_flags);
 	return ret;
 }
-EXPORT_SYMBOL_GPL(io_buffer_register_bvec);
+EXPORT_SYMBOL_GPL(io_buffer_register_request);
 
-int io_buffer_unregister_bvec(struct io_uring_cmd *cmd, unsigned int index,
-			      unsigned int issue_flags)
+int io_buffer_unregister(struct io_uring_cmd *cmd, unsigned int index,
+			 unsigned int issue_flags)
 {
 	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
 	struct io_rsrc_data *data = &ctx->buf_table;
@@ -1019,7 +1019,7 @@ int io_buffer_unregister_bvec(struct io_uring_cmd *cmd, unsigned int index,
 	io_ring_submit_unlock(ctx, issue_flags);
 	return ret;
 }
-EXPORT_SYMBOL_GPL(io_buffer_unregister_bvec);
+EXPORT_SYMBOL_GPL(io_buffer_unregister);
 
 static int validate_fixed_range(u64 buf_addr, size_t len,
 				const struct io_mapped_ubuf *imu)
-- 
2.52.0

[PATCH v3 2/5] io_uring/rsrc: split io_buffer_register_request() logic

[Joanne Koong]

Split the main initialization logic in io_buffer_register_request() into
a helper function.

This is a preparatory patch for supporting kernel-populated buffers in
fuse io-uring, which will be reusing this logic.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Caleb Sander Mateos <csander@purestorage.com>
---
 io_uring/rsrc.c | 84 ++++++++++++++++++++++++++++++-------------------
 1 file changed, 51 insertions(+), 33 deletions(-)

diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
index 7579f6992a25..01c3619e5f07 100644
--- a/io_uring/rsrc.c
+++ b/io_uring/rsrc.c
@@ -924,64 +924,82 @@ int io_sqe_buffers_register(struct io_ring_ctx *ctx, void __user *arg,
 	return ret;
 }
 
-int io_buffer_register_request(struct io_uring_cmd *cmd, struct request *rq,
-			       void (*release)(void *), unsigned int index,
-			       unsigned int issue_flags)
+static struct io_mapped_ubuf *io_kernel_buffer_init(struct io_ring_ctx *ctx,
+						    unsigned int nr_bvecs,
+						    unsigned int total_bytes,
+						    u8 dir,
+						    void (*release)(void *),
+						    void *priv,
+						    unsigned int index)
 {
-	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
 	struct io_rsrc_data *data = &ctx->buf_table;
-	struct req_iterator rq_iter;
 	struct io_mapped_ubuf *imu;
 	struct io_rsrc_node *node;
-	struct bio_vec bv;
-	unsigned int nr_bvecs = 0;
-	int ret = 0;
 
-	io_ring_submit_lock(ctx, issue_flags);
-	if (index >= data->nr) {
-		ret = -EINVAL;
-		goto unlock;
-	}
+	if (index >= data->nr)
+		return ERR_PTR(-EINVAL);
 	index = array_index_nospec(index, data->nr);
 
-	if (data->nodes[index]) {
-		ret = -EBUSY;
-		goto unlock;
-	}
+	if (data->nodes[index])
+		return ERR_PTR(-EBUSY);
 
 	node = io_rsrc_node_alloc(ctx, IORING_RSRC_BUFFER);
-	if (!node) {
-		ret = -ENOMEM;
-		goto unlock;
-	}
+	if (!node)
+		return ERR_PTR(-ENOMEM);
 
-	/*
-	 * blk_rq_nr_phys_segments() may overestimate the number of bvecs
-	 * but avoids needing to iterate over the bvecs
-	 */
-	imu = io_alloc_imu(ctx, blk_rq_nr_phys_segments(rq));
+	imu = io_alloc_imu(ctx, nr_bvecs);
 	if (!imu) {
 		kfree(node);
-		ret = -ENOMEM;
-		goto unlock;
+		return ERR_PTR(-ENOMEM);
 	}
 
 	imu->ubuf = 0;
-	imu->len = blk_rq_bytes(rq);
+	imu->len = total_bytes;
 	imu->acct_pages = 0;
 	imu->folio_shift = PAGE_SHIFT;
+	imu->nr_bvecs = nr_bvecs;
 	refcount_set(&imu->refs, 1);
 	imu->release = release;
-	imu->priv = rq;
+	imu->priv = priv;
+	imu->dir = dir;
 	imu->flags = IO_REGBUF_F_KBUF;
-	imu->dir = 1 << rq_data_dir(rq);
 
+	node->buf = imu;
+	data->nodes[index] = node;
+
+	return imu;
+}
+
+int io_buffer_register_request(struct io_uring_cmd *cmd, struct request *rq,
+			       void (*release)(void *), unsigned int index,
+			       unsigned int issue_flags)
+{
+	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+	struct req_iterator rq_iter;
+	struct io_mapped_ubuf *imu;
+	struct bio_vec bv;
+	/*
+	 * blk_rq_nr_phys_segments() may overestimate the number of bvecs
+	 * but avoids needing to iterate over the bvecs
+	 */
+	unsigned int nr_bvecs = blk_rq_nr_phys_segments(rq);
+	unsigned int total_bytes = blk_rq_bytes(rq);
+	int ret = 0;
+
+	io_ring_submit_lock(ctx, issue_flags);
+
+	imu = io_kernel_buffer_init(ctx, nr_bvecs, total_bytes,
+				    1 << rq_data_dir(rq), release, rq, index);
+	if (IS_ERR(imu)) {
+		ret = PTR_ERR(imu);
+		goto unlock;
+	}
+
+	nr_bvecs = 0;
 	rq_for_each_bvec(bv, rq, rq_iter)
 		imu->bvec[nr_bvecs++] = bv;
 	imu->nr_bvecs = nr_bvecs;
 
-	node->buf = imu;
-	data->nodes[index] = node;
 unlock:
 	io_ring_submit_unlock(ctx, issue_flags);
 	return ret;
-- 
2.52.0

[PATCH v3 3/5] io_uring/rsrc: add io_buffer_register_bvec()

[Joanne Koong]

Add io_buffer_register_bvec() for registering a bvec array.

This is a preparatory patch for fuse-over-io-uring zero-copy.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Caleb Sander Mateos <csander@purestorage.com>
---
 include/linux/io_uring/cmd.h | 13 +++++++++++++
 io_uring/rsrc.c              | 35 +++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+)

diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h
index 165d436073a4..b4d1d5e8e851 100644
--- a/include/linux/io_uring/cmd.h
+++ b/include/linux/io_uring/cmd.h
@@ -110,6 +110,10 @@ struct io_br_sel io_ring_buffer_select(struct io_kiocb *req, size_t *len,
 int io_buffer_register_request(struct io_uring_cmd *cmd, struct request *rq,
 			       void (*release)(void *), unsigned int index,
 			       unsigned int issue_flags);
+int io_buffer_register_bvec(struct io_uring_cmd *cmd, const struct bio_vec *bvs,
+			    unsigned int nr_bvecs, void (*release)(void *),
+			    void *priv, u8 dir, unsigned int index,
+			    unsigned int issue_flags);
 int io_buffer_unregister(struct io_uring_cmd *cmd, unsigned int index,
 			 unsigned int issue_flags);
 #else
@@ -198,6 +202,15 @@ static inline int io_buffer_register_request(struct io_uring_cmd *cmd,
 {
 	return -EOPNOTSUPP;
 }
+static inline int io_buffer_register_bvec(struct io_uring_cmd *cmd,
+					  const struct bio_vec *bvs,
+					  unsigned int nr_bvecs,
+					  void (*release)(void *), void *priv,
+					  u8 dir, unsigned int index,
+					  unsigned int issue_flags)
+{
+	return -EOPNOTSUPP;
+}
 static inline int io_buffer_unregister(struct io_uring_cmd *cmd,
 				       unsigned int index,
 				       unsigned int issue_flags)
diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
index 01c3619e5f07..039e3b30ff60 100644
--- a/io_uring/rsrc.c
+++ b/io_uring/rsrc.c
@@ -1006,6 +1006,41 @@ int io_buffer_register_request(struct io_uring_cmd *cmd, struct request *rq,
 }
 EXPORT_SYMBOL_GPL(io_buffer_register_request);
 
+/*
+ * bvs is copied internally. caller may free it on return.
+ */
+int io_buffer_register_bvec(struct io_uring_cmd *cmd, const struct bio_vec *bvs,
+			    unsigned int nr_bvecs, void (*release)(void *),
+			    void *priv, u8 dir, unsigned int index,
+			    unsigned int issue_flags)
+{
+	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+	struct io_mapped_ubuf *imu;
+	struct bio_vec *bvec;
+	unsigned int i, total_bytes = 0;
+	int ret = 0;
+
+	for (i = 0; i < nr_bvecs; i++)
+		total_bytes += bvs[i].bv_len;
+
+	io_ring_submit_lock(ctx, issue_flags);
+	imu = io_kernel_buffer_init(ctx, nr_bvecs, total_bytes, dir, release,
+				    priv, index);
+	if (IS_ERR(imu)) {
+		ret = PTR_ERR(imu);
+		goto unlock;
+	}
+
+	bvec = imu->bvec;
+	for (i = 0; i < nr_bvecs; i++)
+		bvec[i] = bvs[i];
+
+unlock:
+	io_ring_submit_unlock(ctx, issue_flags);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(io_buffer_register_bvec);
+
 int io_buffer_unregister(struct io_uring_cmd *cmd, unsigned int index,
 			 unsigned int issue_flags)
 {
-- 
2.52.0

[PATCH v3 4/5] io_uring/rsrc: rename and export IO_IMU_DEST / IO_IMU_SOURCE

[Joanne Koong]

Rename IO_IMU_DEST and IO_IMU_SOURCE to IO_BUF_DEST and IO_BUF_SOURCE
and export it so subsystems may use it.

This is needed by the io_buffer_register_bvec() path for callers who may
need the buffer to be both readable and writable.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
---
 include/linux/io_uring_types.h | 5 +++++
 io_uring/io_uring.c            | 2 +-
 io_uring/rsrc.c                | 2 +-
 io_uring/rsrc.h                | 5 -----
 4 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h
index 328c3c1e2a31..0f41f9351a78 100644
--- a/include/linux/io_uring_types.h
+++ b/include/linux/io_uring_types.h
@@ -44,6 +44,11 @@ enum io_uring_cmd_flags {
 	IO_URING_F_COMPAT		= (1 << 12),
 };
 
+enum {
+	IO_BUF_DEST	= 1 << ITER_DEST,
+	IO_BUF_SOURCE	= 1 << ITER_SOURCE,
+};
+
 struct iou_loop_params;
 
 struct io_wq_work_node {
diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index 16122f877aed..b5debc615657 100644
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -3215,7 +3215,7 @@ static int __init io_uring_init(void)
 	io_uring_optable_init();
 
 	/* imu->dir is u8 */
-	BUILD_BUG_ON((IO_IMU_DEST | IO_IMU_SOURCE) > U8_MAX);
+	BUILD_BUG_ON((IO_BUF_DEST | IO_BUF_SOURCE) > U8_MAX);
 
 	/*
 	 * Allow user copy in the per-command field, which starts after the
diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
index 039e3b30ff60..cf5638406a0c 100644
--- a/io_uring/rsrc.c
+++ b/io_uring/rsrc.c
@@ -820,7 +820,7 @@ static struct io_rsrc_node *io_sqe_buffer_register(struct io_ring_ctx *ctx,
 	imu->release = io_release_ubuf;
 	imu->priv = imu;
 	imu->flags = 0;
-	imu->dir = IO_IMU_DEST | IO_IMU_SOURCE;
+	imu->dir = IO_BUF_DEST | IO_BUF_SOURCE;
 	if (coalesced)
 		imu->folio_shift = data.folio_shift;
 	refcount_set(&imu->refs, 1);
diff --git a/io_uring/rsrc.h b/io_uring/rsrc.h
index cff0f8834c35..8d48195faf9d 100644
--- a/io_uring/rsrc.h
+++ b/io_uring/rsrc.h
@@ -23,11 +23,6 @@ struct io_rsrc_node {
 	};
 };
 
-enum {
-	IO_IMU_DEST	= 1 << ITER_DEST,
-	IO_IMU_SOURCE	= 1 << ITER_SOURCE,
-};
-
 enum {
 	IO_REGBUF_F_KBUF		= 1,
 };
-- 
2.52.0

[PATCH v3 5/5] io_uring/rsrc: add io_uring_registered_mem_region_get()

[Joanne Koong]

Add io_uring_registered_mem_region_get() helper to allow io_uring
command handlers to retrieve the vmapped pointer to the ring's
registered memory region's pages as well as the size of the region. This
provides a way for uring cmd implementations to directly access
pre-registered memory for passing data.

This will be used by fuse for reading/writing header data between the
kernel and the server.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
---
 include/linux/io_uring/cmd.h |  9 +++++++++
 io_uring/rsrc.c              | 18 ++++++++++++++++++
 2 files changed, 27 insertions(+)

diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h
index b4d1d5e8e851..ade0eb807da6 100644
--- a/include/linux/io_uring/cmd.h
+++ b/include/linux/io_uring/cmd.h
@@ -51,6 +51,9 @@ int io_uring_cmd_import_fixed_vec(struct io_uring_cmd *ioucmd,
 				  size_t uvec_segs,
 				  int ddir, struct iov_iter *iter,
 				  unsigned issue_flags);
+void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
+					 unsigned *nr_pages,
+					 unsigned issue_flags);
 
 /*
  * Completes the request, i.e. posts an io_uring CQE and deallocates @ioucmd
@@ -132,6 +135,12 @@ static inline int io_uring_cmd_import_fixed_vec(struct io_uring_cmd *ioucmd,
 {
 	return -EOPNOTSUPP;
 }
+static inline void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
+						       unsigned *nr_pages,
+						       unsigned issue_flags)
+{
+	return NULL;
+}
 static inline void __io_uring_cmd_done(struct io_uring_cmd *cmd, s32 ret,
 		u64 ret2, unsigned issue_flags, bool is_cqe32)
 {
diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
index cf5638406a0c..c706324fd66d 100644
--- a/io_uring/rsrc.c
+++ b/io_uring/rsrc.c
@@ -1182,6 +1182,24 @@ int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter,
 	return io_import_fixed(ddir, iter, node->buf, buf_addr, len);
 }
 
+void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
+					 unsigned *nr_pages,
+					 unsigned issue_flags)
+{
+	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+	void *ptr;
+
+	io_ring_submit_lock(ctx, issue_flags);
+
+	ptr = ctx->param_region.ptr;
+	*nr_pages = ctx->param_region.nr_pages;
+
+	io_ring_submit_unlock(ctx, issue_flags);
+
+	return ptr;
+}
+EXPORT_SYMBOL_GPL(io_uring_registered_mem_region_get);
+
 /* Lock two rings at once. The rings must be different! */
 static void lock_two_rings(struct io_ring_ctx *ctx1, struct io_ring_ctx *ctx2)
 {
-- 
2.52.0

Re: [PATCH v3 5/5] io_uring/rsrc: add io_uring_registered_mem_region_get()

[Jens Axboe]

On 3/24/26 4:14 PM, Joanne Koong wrote:
> diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
> index cf5638406a0c..c706324fd66d 100644
> --- a/io_uring/rsrc.c
> +++ b/io_uring/rsrc.c
> @@ -1182,6 +1182,24 @@ int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter,
>  	return io_import_fixed(ddir, iter, node->buf, buf_addr, len);
>  }
>  
> +void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
> +					 unsigned *nr_pages,
> +					 unsigned issue_flags)
> +{
> +	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
> +	void *ptr;
> +
> +	io_ring_submit_lock(ctx, issue_flags);
> +
> +	ptr = ctx->param_region.ptr;
> +	*nr_pages = ctx->param_region.nr_pages;
> +
> +	io_ring_submit_unlock(ctx, issue_flags);
> +
> +	return ptr;
> +}
> +EXPORT_SYMBOL_GPL(io_uring_registered_mem_region_get);

This looks suspicious, but I actually think it looks suspicious because
you add the submit locking around it. For patterns like that, it makes
the brain go "hmm, what protects this from going invalid the instant
io_ring_submit_unlock() is called??". But this should be stable for the
duration of the ring, hence the locking should not be needed at all?

I'd probably also prefer wrapping this in some kind of return struct, so
that you just return that rather than having one part returned directly
and one stuffed in a pointer to an unsigned for number of pages.

-- 
Jens Axboe

Re: [PATCH v3 5/5] io_uring/rsrc: add io_uring_registered_mem_region_get()

[Joanne Koong]

On Wed, Mar 25, 2026 at 7:56 AM Jens Axboe <axboe@kernel.dk> wrote:
>
> On 3/24/26 4:14 PM, Joanne Koong wrote:
> > diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
> > index cf5638406a0c..c706324fd66d 100644
> > --- a/io_uring/rsrc.c
> > +++ b/io_uring/rsrc.c
> > @@ -1182,6 +1182,24 @@ int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter,
> >       return io_import_fixed(ddir, iter, node->buf, buf_addr, len);
> >  }
> >
> > +void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
> > +                                      unsigned *nr_pages,
> > +                                      unsigned issue_flags)
> > +{
> > +     struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
> > +     void *ptr;
> > +
> > +     io_ring_submit_lock(ctx, issue_flags);
> > +
> > +     ptr = ctx->param_region.ptr;
> > +     *nr_pages = ctx->param_region.nr_pages;
> > +
> > +     io_ring_submit_unlock(ctx, issue_flags);
> > +
> > +     return ptr;
> > +}
> > +EXPORT_SYMBOL_GPL(io_uring_registered_mem_region_get);
>
> This looks suspicious, but I actually think it looks suspicious because
> you add the submit locking around it. For patterns like that, it makes
> the brain go "hmm, what protects this from going invalid the instant
> io_ring_submit_unlock() is called??". But this should be stable for the
> duration of the ring, hence the locking should not be needed at all?

My understanding is that once a memory region is registered to the
ring, it's registered for the ring's lifetime. There's no uapi to
unregister a memory region and my interpretation of the last paragraph
in this thread [1] is that unregistration is not intended to be
added/supported. I think the submit locking is needed in case another
thread is currently registering it so we don't see partially
initialized state between ptr and nr_pages (eg if the caller calls
this from a task work callback).

[1] https://lore.kernel.org/linux-fsdevel/000f7db7-5546-4680-bef2-84ce740ad8fd@gmail.com/

>
> I'd probably also prefer wrapping this in some kind of return struct, so
> that you just return that rather than having one part returned directly
> and one stuffed in a pointer to an unsigned for number of pages.

Sounds good, I will make this change.

Thanks,
Joanne
>
> --
> Jens Axboe

Re: [PATCH v3 5/5] io_uring/rsrc: add io_uring_registered_mem_region_get()

[Jens Axboe]

On 3/25/26 11:24 AM, Joanne Koong wrote:
> On Wed, Mar 25, 2026 at 7:56?AM Jens Axboe <axboe@kernel.dk> wrote:
>>
>> On 3/24/26 4:14 PM, Joanne Koong wrote:
>>> diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
>>> index cf5638406a0c..c706324fd66d 100644
>>> --- a/io_uring/rsrc.c
>>> +++ b/io_uring/rsrc.c
>>> @@ -1182,6 +1182,24 @@ int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter,
>>>       return io_import_fixed(ddir, iter, node->buf, buf_addr, len);
>>>  }
>>>
>>> +void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
>>> +                                      unsigned *nr_pages,
>>> +                                      unsigned issue_flags)
>>> +{
>>> +     struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
>>> +     void *ptr;
>>> +
>>> +     io_ring_submit_lock(ctx, issue_flags);
>>> +
>>> +     ptr = ctx->param_region.ptr;
>>> +     *nr_pages = ctx->param_region.nr_pages;
>>> +
>>> +     io_ring_submit_unlock(ctx, issue_flags);
>>> +
>>> +     return ptr;
>>> +}
>>> +EXPORT_SYMBOL_GPL(io_uring_registered_mem_region_get);
>>
>> This looks suspicious, but I actually think it looks suspicious because
>> you add the submit locking around it. For patterns like that, it makes
>> the brain go "hmm, what protects this from going invalid the instant
>> io_ring_submit_unlock() is called??". But this should be stable for the
>> duration of the ring, hence the locking should not be needed at all?
> 
> My understanding is that once a memory region is registered to the
> ring, it's registered for the ring's lifetime. There's no uapi to
> unregister a memory region and my interpretation of the last paragraph
> in this thread [1] is that unregistration is not intended to be
> added/supported. I think the submit locking is needed in case another
> thread is currently registering it so we don't see partially
> initialized state between ptr and nr_pages (eg if the caller calls
> this from a task work callback).

Yes good point - can you please add a comment to that effect? Both why
it's safe to return this state outside the lock, and also why the lock
is actually required to ensure we see a sane state (either region fully
registered, or not there)?

-- 
Jens Axboe

Re: [PATCH v3 5/5] io_uring/rsrc: add io_uring_registered_mem_region_get()

[Joanne Koong]

On Wed, Mar 25, 2026 at 10:27 AM Jens Axboe <axboe@kernel.dk> wrote:
>
> On 3/25/26 11:24 AM, Joanne Koong wrote:
> > On Wed, Mar 25, 2026 at 7:56?AM Jens Axboe <axboe@kernel.dk> wrote:
> >>
> >> On 3/24/26 4:14 PM, Joanne Koong wrote:
> >>> diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
> >>> index cf5638406a0c..c706324fd66d 100644
> >>> --- a/io_uring/rsrc.c
> >>> +++ b/io_uring/rsrc.c
> >>> @@ -1182,6 +1182,24 @@ int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter,
> >>>       return io_import_fixed(ddir, iter, node->buf, buf_addr, len);
> >>>  }
> >>>
> >>> +void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
> >>> +                                      unsigned *nr_pages,
> >>> +                                      unsigned issue_flags)
> >>> +{
> >>> +     struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
> >>> +     void *ptr;
> >>> +
> >>> +     io_ring_submit_lock(ctx, issue_flags);
> >>> +
> >>> +     ptr = ctx->param_region.ptr;
> >>> +     *nr_pages = ctx->param_region.nr_pages;
> >>> +
> >>> +     io_ring_submit_unlock(ctx, issue_flags);
> >>> +
> >>> +     return ptr;
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(io_uring_registered_mem_region_get);
> >>
> >> This looks suspicious, but I actually think it looks suspicious because
> >> you add the submit locking around it. For patterns like that, it makes
> >> the brain go "hmm, what protects this from going invalid the instant
> >> io_ring_submit_unlock() is called??". But this should be stable for the
> >> duration of the ring, hence the locking should not be needed at all?
> >
> > My understanding is that once a memory region is registered to the
> > ring, it's registered for the ring's lifetime. There's no uapi to
> > unregister a memory region and my interpretation of the last paragraph
> > in this thread [1] is that unregistration is not intended to be
> > added/supported. I think the submit locking is needed in case another
> > thread is currently registering it so we don't see partially
> > initialized state between ptr and nr_pages (eg if the caller calls
> > this from a task work callback).
>
> Yes good point - can you please add a comment to that effect? Both why
> it's safe to return this state outside the lock, and also why the lock
> is actually required to ensure we see a sane state (either region fully
> registered, or not there)?

Good idea, I will add a comment about this to make this more clear,
something like:
/*
 * The submit lock ensures we don't see partially initialized state
 * if another thread is currently registering the region. Once registered,
 * the region is stable for the ring's lifetime (no unregister API exists),
 * so it's safe to access the returned pointer outside the lock.
 */

I will wait a day or two to send out v4 in case additional comments come in.

Thanks,
Joanne
>
> --
> Jens Axboe

Re: [PATCH v3 5/5] io_uring/rsrc: add io_uring_registered_mem_region_get()

[Jens Axboe]

On 3/25/26 1:56 PM, Joanne Koong wrote:
> Good idea, I will add a comment about this to make this more clear,
> something like:
> /*
>  * The submit lock ensures we don't see partially initialized state
>  * if another thread is currently registering the region. Once registered,
>  * the region is stable for the ring's lifetime (no unregister API exists),
>  * so it's safe to access the returned pointer outside the lock.
>  */

Perfect, thanks!

-- 
Jens Axboe

Re: [PATCH v3 0/5] io_uring: extend bvec registration and add mem region lookup

[Joanne Koong]

On Tue, Mar 24, 2026 at 3:15 PM Joanne Koong <joannelkoong@gmail.com> wrote:
>
> This series refactors and extends the io_uring registered buffers
> infrastructure to allow external subsystems to register pre-existing bvec
> arrays directly and obtain a pointer to the registered memory region.
>
> The motivation for the patches in this series is to make fuse zero-copy
> possible.
>
> These patches are split out from a previous larger fuse-over-io_uring series
> [1]. The remaining fuse patches will be submitted separately and linked to.

The fuse patches that build on this series can be found here:
https://lore.kernel.org/linux-fsdevel/20260324224532.3733468-1-joannelkoong@gmail.com/

Thanks,
Joanne