Re: [PATCH 4/4] io_uring: optimise compl locking for non-shared rings

[Jens Axboe <axboe@kernel.dk>]

On 3/18/22 7:52 AM, Pavel Begunkov wrote:
> When only one task submits requests, most of CQEs are expected to be
> filled from that task context so we have natural serialisation. That
> would mean that in those cases we don't need spinlocking around CQE
> posting. One downside is that it also mean that io-wq workers can't emit
> CQEs directly but should do it through the original task context using
> task_works. That may hurt latency and performance and might matter much
> to some workloads, but it's not a huge deal in general as io-wq is a
> slow path and there is some additional merit from tw completion
> batching.

Not too worried about io-wq task_work for cq filling, it is the slower
path after all. And I think we can get away with doing notifications as
it's just for CQ filling. If the task is currently waiting in
cqring_wait, then it'll get woken anyway and it will process task work.
If it's in userspace, it doesn't need a notification. That should make
it somewhat lighter than requiring using TIF_NOTIFY_SIGNAL for that.

> The feature should be opted-in by the userspace by setting a new
> IORING_SETUP_PRIVATE_CQ flag. It doesn't work with IOPOLL, and also for
> now only the task that created a ring can submit requests to it.

I know this is a WIP, but why do we need CQ_PRIVATE? And this needs to
work with registered files (and ring fd) as that is probably a bigger
win than skipping the completion_lock if you're not shared anyway.


-- 
Jens Axboe