Stable inclusion

Stable inclusion

[Jens Axboe]

Hi,

Can you add the below to 6.1-stable? Thanks!

commit 3181e22fb79910c7071e84a43af93ac89e8a7106
Author: Pavel Begunkov <asml.silence@gmail.com>
Date:   Mon Jan 9 14:46:10 2023 +0000

    io_uring: wake up optimisations

    Commit 3181e22fb79910c7071e84a43af93ac89e8a7106 upstream.
    
    Flush completions is done either from the submit syscall or by the
    task_work, both are in the context of the submitter task, and when it
    goes for a single threaded rings like implied by ->task_complete, there
    won't be any waiters on ->cq_wait but the master task. That means that
    there can be no tasks sleeping on cq_wait while we run
    __io_submit_flush_completions() and so waking up can be skipped.
    
    Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
    Link: https://lore.kernel.org/r/60ad9768ec74435a0ddaa6eec0ffa7729474f69f.1673274244.git.asml.silence@gmail.com
    Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index 4f0ae938b146..0b1361663267 100644
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -582,6 +582,16 @@ static inline void __io_cq_unlock_post(struct io_ring_ctx *ctx)
 	io_cqring_ev_posted(ctx);
 }
 
+static inline void __io_cq_unlock_post_flush(struct io_ring_ctx *ctx)
+	__releases(ctx->completion_lock)
+{
+	io_commit_cqring(ctx);
+	spin_unlock(&ctx->completion_lock);
+	io_commit_cqring_flush(ctx);
+	if (!(ctx->flags & IORING_SETUP_DEFER_TASKRUN))
+		__io_cqring_wake(ctx);
+}
+
 void io_cq_unlock_post(struct io_ring_ctx *ctx)
 {
 	__io_cq_unlock_post(ctx);
@@ -1339,7 +1349,7 @@ static void __io_submit_flush_completions(struct io_ring_ctx *ctx)
 		if (!(req->flags & REQ_F_CQE_SKIP))
 			__io_fill_cqe_req(ctx, req);
 	}
-	__io_cq_unlock_post(ctx);
+	__io_cq_unlock_post_flush(ctx);
 
 	io_free_batch_list(ctx, state->compl_reqs.first);
 	INIT_WQ_LIST(&state->compl_reqs);

-- 
Jens Axboe

Re: Stable inclusion

[Greg Kroah-Hartman]

On Tue, Dec 10, 2024 at 04:30:05PM -0700, Jens Axboe wrote:
> Hi,
> 
> Can you add the below to 6.1-stable? Thanks!
> 

Now queued up, thanks.

greg k-h