[PATCH 0/2] ring buffer fixes

[PATCH 0/2] ring buffer fixes

[Pavel Begunkov]

Pavel Begunkov (2):
  io_uring/kbuf: fix not advancing READV kbuf ring
  io_uring: recycle kbuf recycle on tw requeue

 io_uring/io_uring.c | 1 +
 io_uring/kbuf.h     | 8 ++++++--
 2 files changed, 7 insertions(+), 2 deletions(-)

-- 
2.37.2

[PATCH 1/2] io_uring/kbuf: fix not advancing READV kbuf ring

[Pavel Begunkov]

When we don't recycle a selected ring buffer we should advance the head
of the ring, so don't just skip io_kbuf_recycle() for IORING_OP_READV
but adjust the ring.

Fixes: 934447a603b22 ("io_uring: do not recycle buffer in READV")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
 io_uring/kbuf.h | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/io_uring/kbuf.h b/io_uring/kbuf.h
index d6af208d109f..746fbf31a703 100644
--- a/io_uring/kbuf.h
+++ b/io_uring/kbuf.h
@@ -91,9 +91,13 @@ static inline void io_kbuf_recycle(struct io_kiocb *req, unsigned issue_flags)
 	 * buffer data. However if that buffer is recycled the original request
 	 * data stored in addr is lost. Therefore forbid recycling for now.
 	 */
-	if (req->opcode == IORING_OP_READV)
+	if (req->opcode == IORING_OP_READV) {
+		if ((req->flags & REQ_F_BUFFER_RING) && req->buf_list) {
+			req->buf_list->head++;
+			req->buf_list = NULL;
+		}
 		return;
-
+	}
 	if (req->flags & REQ_F_BUFFER_SELECTED)
 		io_kbuf_recycle_legacy(req, issue_flags);
 	if (req->flags & REQ_F_BUFFER_RING)
-- 
2.37.2

Re: [PATCH 1/2] io_uring/kbuf: fix not advancing READV kbuf ring

[Dylan Yudaken]

On Tue, 2022-09-06 at 17:11 +0100, Pavel Begunkov wrote:
> When we don't recycle a selected ring buffer we should advance the
> head
> of the ring, so don't just skip io_kbuf_recycle() for IORING_OP_READV
> but adjust the ring.
> 
> Fixes: 934447a603b22 ("io_uring: do not recycle buffer in READV")
> Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
> ---
>  io_uring/kbuf.h | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/io_uring/kbuf.h b/io_uring/kbuf.h
> index d6af208d109f..746fbf31a703 100644
> --- a/io_uring/kbuf.h
> +++ b/io_uring/kbuf.h
> @@ -91,9 +91,13 @@ static inline void io_kbuf_recycle(struct io_kiocb
> *req, unsigned issue_flags)
>          * buffer data. However if that buffer is recycled the
> original request
>          * data stored in addr is lost. Therefore forbid recycling
> for now.
>          */
> -       if (req->opcode == IORING_OP_READV)
> +       if (req->opcode == IORING_OP_READV) {
> +               if ((req->flags & REQ_F_BUFFER_RING) && req-
> >buf_list) {
> +                       req->buf_list->head++;
> +                       req->buf_list = NULL;
> +               }
>                 return;
> -
> +       }
>         if (req->flags & REQ_F_BUFFER_SELECTED)
>                 io_kbuf_recycle_legacy(req, issue_flags);
>         if (req->flags & REQ_F_BUFFER_RING)


I do not know a good way to test this, but it reads to me likeit is
correct. It should probably be applied before the patch I sent earlier
today to remove this code so that backporting is easier.

Reviewed-by: Dylan Yudaken <dylany@fb.com>

[PATCH 2/2] io_uring: recycle kbuf recycle on tw requeue

[Pavel Begunkov]

When we queue a request via tw for execution it's not going to be
executed immediately, so when io_queue_async() hits IO_APOLL_READY
and queues a tw but doesn't try to recycle/consume the buffer some other
request may try to use the the buffer.

Fixes: c7fb19428d67 ("io_uring: add support for ring mapped supplied buffers")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
 io_uring/io_uring.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index f9be9b7eb654..b9640ad5069f 100644
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -1728,6 +1728,7 @@ static void io_queue_async(struct io_kiocb *req, int ret)
 
 	switch (io_arm_poll_handler(req, 0)) {
 	case IO_APOLL_READY:
+		io_kbuf_recycle(req, 0);
 		io_req_task_queue(req);
 		break;
 	case IO_APOLL_ABORTED:
-- 
2.37.2

Re: [PATCH 0/2] ring buffer fixes

[Jens Axboe]

On Tue, 6 Sep 2022 17:11:15 +0100, Pavel Begunkov wrote:
> Pavel Begunkov (2):
>   io_uring/kbuf: fix not advancing READV kbuf ring
>   io_uring: recycle kbuf recycle on tw requeue
> 
> io_uring/io_uring.c | 1 +
>  io_uring/kbuf.h     | 8 ++++++--
>  2 files changed, 7 insertions(+), 2 deletions(-)
> 
> [...]

Applied, thanks!

[1/2] io_uring/kbuf: fix not advancing READV kbuf ring
      commit: df6d3422d3eed27afa23df092b3ce147c558d1a8
[2/2] io_uring: recycle kbuf recycle on tw requeue
      commit: 336d28a8f38013a069f2d46e73aaa1880ef17a47

Best regards,
-- 
Jens Axboe