Re: [PATCH 2/2] io_uring: switch local task_work to a mpscq

[Jens Axboe <axboe@kernel.dk>]

On 6/11/26 11:24 PM, Caleb Sander Mateos wrote:
> On Thu, Jun 11, 2026 at 7:23?PM Jens Axboe <axboe@kernel.dk> wrote:
>>
>> On 6/11/26 7:14 PM, Caleb Sander Mateos wrote:
>>> This is great stuff! I had also observed these hotspots on a ublk
>>> workload. Since incoming ublk requests post task work to the ublk
>>> server's io_urings and completed ublk requests post task work to the
>>> client's io_urings, there is significant cross-CPU contention on the
>>> task work queues.
>>
>> Glad you like it! Once I post v2 tomorrow, perhaps you can try and run
>> some tests with and without and see how it does for you?
> 
> Haven't tested v2 yet, but v1 shows a 4% IOPS improvement on a ublk
> 4-KB read workload. The workload has 8 CPUs (unpaired hypertwins)
> running fio with io_uring submitting I/O to the ublk devices and 32
> ublk server CPUs (paired hypertwins) servicing the requests, achieving
> around 4M IOPS. Both the client and server CPUs look completely busy.

That's a pretty nice improvement! Would be curious to hear what v2 looks
like.

> I can see clear reductions in __io_req_task_work_add() and
> llist_reverse_order() (now gone) on both sets of CPUs, through the
> cache misses popping task work items are now attributed to
> __io_run_local_work() instead.

Right, llist_reverse_order() previously could have had the useful side
effect of priming the cache. Sometimes that could be useful, if the
task_work itself was basically just posting a CQE. Other times, when the
task_work itself does actual work (eg socket recv), then it was just
harmful. For the former case, we could potentially prefetch() next when
popping. Not sure it's worth it though, though we could experiment with
something along those lines.

-- 
Jens Axboe