[PATCH 0/2] ublk: run auto buf unregister on same io_ring_ctx with register

[PATCH 0/2] ublk: run auto buf unregister on same io_ring_ctx with register

[Ming Lei]

Hello Jens,

The 1st patch adds helper io_uring_cmd_ctx_handle().

The 2nd one use the added helper to run auto-buf-unreg on the same
io_uring_ctx with the register one.

Thanks,
Ming


Ming Lei (2):
  io_uring: add helper io_uring_cmd_ctx_handle()
  ublk: run auto buf unregister on same io_ring_ctx with register

 drivers/block/ublk_drv.c      | 19 ++++++++++++++++---
 include/linux/io_uring/cmd.h  |  9 +++++++++
 include/uapi/linux/ublk_cmd.h |  6 +++++-
 3 files changed, 30 insertions(+), 4 deletions(-)

-- 
2.47.0

[PATCH 1/2] io_uring: add helper io_uring_cmd_ctx_handle()

[Ming Lei]

Add helper io_uring_cmd_ctx_handle() for driver to track per-context
resource, such as registered kernel io buffer.

Suggested-by: Caleb Sander Mateos <csander@purestorage.com>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 include/linux/io_uring/cmd.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h
index 0634a3de1782..92d523865df8 100644
--- a/include/linux/io_uring/cmd.h
+++ b/include/linux/io_uring/cmd.h
@@ -140,6 +140,15 @@ static inline struct io_uring_cmd_data *io_uring_cmd_get_async_data(struct io_ur
 	return cmd_to_io_kiocb(cmd)->async_data;
 }
 
+/*
+ * Return uring_cmd's context reference as its context handle for driver to
+ * track per-context resource, such as registered kernel IO buffer
+ */
+static inline unsigned long io_uring_cmd_ctx_handle(struct io_uring_cmd *cmd)
+{
+	return (unsigned long)cmd_to_io_kiocb(cmd)->ctx;
+}
+
 int io_buffer_register_bvec(struct io_uring_cmd *cmd, struct request *rq,
 			    void (*release)(void *), unsigned int index,
 			    unsigned int issue_flags);
-- 
2.47.0

Re: [PATCH 1/2] io_uring: add helper io_uring_cmd_ctx_handle()

[Caleb Sander Mateos]

On Thu, May 22, 2025 at 6:51 AM Ming Lei <ming.lei@redhat.com> wrote:
>
> Add helper io_uring_cmd_ctx_handle() for driver to track per-context
> resource, such as registered kernel io buffer.
>
> Suggested-by: Caleb Sander Mateos <csander@purestorage.com>
> Signed-off-by: Ming Lei <ming.lei@redhat.com>
> ---
>  include/linux/io_uring/cmd.h | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h
> index 0634a3de1782..92d523865df8 100644
> --- a/include/linux/io_uring/cmd.h
> +++ b/include/linux/io_uring/cmd.h
> @@ -140,6 +140,15 @@ static inline struct io_uring_cmd_data *io_uring_cmd_get_async_data(struct io_ur
>         return cmd_to_io_kiocb(cmd)->async_data;
>  }
>
> +/*
> + * Return uring_cmd's context reference as its context handle for driver to
> + * track per-context resource, such as registered kernel IO buffer
> + */
> +static inline unsigned long io_uring_cmd_ctx_handle(struct io_uring_cmd *cmd)
> +{
> +       return (unsigned long)cmd_to_io_kiocb(cmd)->ctx;

I would still prefer to return const void *. That would avoid the need
for a cast.
Other than that, this looks good.

Best,
Caleb

> +}
> +
>  int io_buffer_register_bvec(struct io_uring_cmd *cmd, struct request *rq,
>                             void (*release)(void *), unsigned int index,
>                             unsigned int issue_flags);
> --
> 2.47.0
>

[PATCH 2/2] ublk: run auto buf unregister on same io_ring_ctx with register

[Ming Lei]

UBLK_F_AUTO_BUF_REG requires that the buffer registered automatically
is unregistered in same `io_ring_ctx`, so check it explicitly.

Document this requirement for UBLK_F_AUTO_BUF_REG.

Drop WARN_ON_ONCE() which is triggered from userspace code path.

Fixes: 99c1e4eb6a3f ("ublk: register buffer to local io_uring with provided buf index via UBLK_F_AUTO_BUF_REG")
Reported-by: Caleb Sander Mateos <csander@purestorage.com>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 drivers/block/ublk_drv.c      | 19 ++++++++++++++++---
 include/uapi/linux/ublk_cmd.h |  6 +++++-
 2 files changed, 21 insertions(+), 4 deletions(-)

diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
index 180386c750f7..a56e07ee9d4b 100644
--- a/drivers/block/ublk_drv.c
+++ b/drivers/block/ublk_drv.c
@@ -84,6 +84,7 @@ struct ublk_rq_data {
 
 	/* for auto-unregister buffer in case of UBLK_F_AUTO_BUF_REG */
 	u16 buf_index;
+	unsigned long buf_ctx_id;
 };
 
 struct ublk_uring_cmd_pdu {
@@ -1211,6 +1212,8 @@ static bool ublk_auto_buf_reg(struct request *req, struct ublk_io *io,
 	}
 	/* one extra reference is dropped by ublk_io_release */
 	refcount_set(&data->ref, 2);
+
+	data->buf_ctx_id = io_uring_cmd_ctx_handle(io->cmd);
 	/* store buffer index in request payload */
 	data->buf_index = pdu->buf.index;
 	io->flags |= UBLK_IO_FLAG_AUTO_BUF_REG;
@@ -2111,12 +2114,22 @@ static int ublk_commit_and_fetch(const struct ublk_queue *ubq,
 	if (ublk_support_auto_buf_reg(ubq)) {
 		int ret;
 
+		/*
+		 * `UBLK_F_AUTO_BUF_REG` only works iff `UBLK_IO_FETCH_REQ`
+		 * and `UBLK_IO_COMMIT_AND_FETCH_REQ` are issued from same
+		 * `io_ring_ctx`.
+		 *
+		 * If this uring_cmd's io_uring_ctx isn't same with the
+		 * one for registering the buffer, it is ublk server's
+		 * responsibility for unregistering the buffer, otherwise
+		 * this ublk request gets stuck.
+		 */
 		if (io->flags & UBLK_IO_FLAG_AUTO_BUF_REG) {
 			struct ublk_rq_data *data = blk_mq_rq_to_pdu(req);
 
-			WARN_ON_ONCE(io_buffer_unregister_bvec(cmd,
-						data->buf_index,
-						issue_flags));
+			if (data->buf_ctx_id == io_uring_cmd_ctx_handle(cmd))
+				io_buffer_unregister_bvec(cmd, data->buf_index,
+						issue_flags);
 			io->flags &= ~UBLK_IO_FLAG_AUTO_BUF_REG;
 		}
 
diff --git a/include/uapi/linux/ublk_cmd.h b/include/uapi/linux/ublk_cmd.h
index c4b9942697fc..5203963cd08a 100644
--- a/include/uapi/linux/ublk_cmd.h
+++ b/include/uapi/linux/ublk_cmd.h
@@ -226,7 +226,11 @@
  *
  * For using this feature:
  *
- * - ublk server has to create sparse buffer table
+ * - ublk server has to create sparse buffer table on the same `io_ring_ctx`
+ *   for issuing `UBLK_IO_FETCH_REQ` and `UBLK_IO_COMMIT_AND_FETCH_REQ`.
+ *   If uring_cmd isn't issued on same `io_uring_ctx`, it is ublk server's
+ *   responsibility to unregister the buffer by issuing `IO_UNREGISTER_IO_BUF`
+ *   manually, otherwise this ublk request get stuck.
  *
  * - ublk server passes auto buf register data via uring_cmd's sqe->addr,
  *   `struct ublk_auto_buf_reg` is populated from sqe->addr, please see
-- 
2.47.0

Re: [PATCH 2/2] ublk: run auto buf unregister on same io_ring_ctx with register

[Caleb Sander Mateos]

On Thu, May 22, 2025 at 6:51 AM Ming Lei <ming.lei@redhat.com> wrote:
>
> UBLK_F_AUTO_BUF_REG requires that the buffer registered automatically
> is unregistered in same `io_ring_ctx`, so check it explicitly.
>
> Document this requirement for UBLK_F_AUTO_BUF_REG.
>
> Drop WARN_ON_ONCE() which is triggered from userspace code path.
>
> Fixes: 99c1e4eb6a3f ("ublk: register buffer to local io_uring with provided buf index via UBLK_F_AUTO_BUF_REG")
> Reported-by: Caleb Sander Mateos <csander@purestorage.com>
> Signed-off-by: Ming Lei <ming.lei@redhat.com>
> ---
>  drivers/block/ublk_drv.c      | 19 ++++++++++++++++---
>  include/uapi/linux/ublk_cmd.h |  6 +++++-
>  2 files changed, 21 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
> index 180386c750f7..a56e07ee9d4b 100644
> --- a/drivers/block/ublk_drv.c
> +++ b/drivers/block/ublk_drv.c
> @@ -84,6 +84,7 @@ struct ublk_rq_data {
>
>         /* for auto-unregister buffer in case of UBLK_F_AUTO_BUF_REG */
>         u16 buf_index;
> +       unsigned long buf_ctx_id;
>  };
>
>  struct ublk_uring_cmd_pdu {
> @@ -1211,6 +1212,8 @@ static bool ublk_auto_buf_reg(struct request *req, struct ublk_io *io,
>         }
>         /* one extra reference is dropped by ublk_io_release */
>         refcount_set(&data->ref, 2);
> +
> +       data->buf_ctx_id = io_uring_cmd_ctx_handle(io->cmd);
>         /* store buffer index in request payload */
>         data->buf_index = pdu->buf.index;
>         io->flags |= UBLK_IO_FLAG_AUTO_BUF_REG;
> @@ -2111,12 +2114,22 @@ static int ublk_commit_and_fetch(const struct ublk_queue *ubq,
>         if (ublk_support_auto_buf_reg(ubq)) {
>                 int ret;
>
> +               /*
> +                * `UBLK_F_AUTO_BUF_REG` only works iff `UBLK_IO_FETCH_REQ`
> +                * and `UBLK_IO_COMMIT_AND_FETCH_REQ` are issued from same
> +                * `io_ring_ctx`.
> +                *
> +                * If this uring_cmd's io_uring_ctx isn't same with the

nit: "io_ring_ctx"

> +                * one for registering the buffer, it is ublk server's
> +                * responsibility for unregistering the buffer, otherwise
> +                * this ublk request gets stuck.
> +                */
>                 if (io->flags & UBLK_IO_FLAG_AUTO_BUF_REG) {
>                         struct ublk_rq_data *data = blk_mq_rq_to_pdu(req);
>
> -                       WARN_ON_ONCE(io_buffer_unregister_bvec(cmd,
> -                                               data->buf_index,
> -                                               issue_flags));
> +                       if (data->buf_ctx_id == io_uring_cmd_ctx_handle(cmd))
> +                               io_buffer_unregister_bvec(cmd, data->buf_index,
> +                                               issue_flags);
>                         io->flags &= ~UBLK_IO_FLAG_AUTO_BUF_REG;
>                 }
>
> diff --git a/include/uapi/linux/ublk_cmd.h b/include/uapi/linux/ublk_cmd.h
> index c4b9942697fc..5203963cd08a 100644
> --- a/include/uapi/linux/ublk_cmd.h
> +++ b/include/uapi/linux/ublk_cmd.h
> @@ -226,7 +226,11 @@
>   *
>   * For using this feature:
>   *
> - * - ublk server has to create sparse buffer table
> + * - ublk server has to create sparse buffer table on the same `io_ring_ctx`
> + *   for issuing `UBLK_IO_FETCH_REQ` and `UBLK_IO_COMMIT_AND_FETCH_REQ`.
> + *   If uring_cmd isn't issued on same `io_uring_ctx`, it is ublk server's

nit: "io_ring_ctx" here too

> + *   responsibility to unregister the buffer by issuing `IO_UNREGISTER_IO_BUF`
> + *   manually, otherwise this ublk request get stuck.

"get stuck" is a little vague. How about "won't complete"?

Other than that,

Reviewed-by: Caleb Sander Mateos <csander@purestorage.com>

>   *
>   * - ublk server passes auto buf register data via uring_cmd's sqe->addr,
>   *   `struct ublk_auto_buf_reg` is populated from sqe->addr, please see
> --
> 2.47.0
>