Re: [PATCH] blk-mq: run the hw queue synchronously for unplugged scheduler inserts

[Kaitao Cheng <kaitao.cheng@linux.dev>]

在 2026/9/30 15:38, Qiliang Yuan 写道:
> When the submitter holds no plug, blk_mq_submit_bio() inserts a request
> into the I/O scheduler and runs the hardware queue asynchronously. Now
> that __submit_bio() no longer opens a plug around every bio, this path
> is taken by all unplugged I/O to devices with an elevator, such as
> io_uring submitting at most two SQEs per io_uring_enter() or synchronous
> O_DIRECT reads and writes.
> 
> Each of these requests queues hctx->run_work on kblockd. kblockd is a
> per-CPU WQ_HIGHPRI workqueue and blk_mq_hctx_next_cpu() returns
> WORK_CPU_UNBOUND for single-queue devices, so the work runs on the
> submitting CPU and the kworker preempts the submitter once per I/O.
> With fio (io_uring, 4k randread, iodepth=16, one job) the submitter
> context switches about once per I/O. A single-queue null_blk device
> with mq-deadline drops from 289K to 193K IOPS, and a single-queue ublk
> device, which defaults to mq-deadline, drops from 265K to 113K IOPS.
> 
> Run the hardware queue from the submitting context for scheduler
> inserts, as flushing the per-bio plug did. Keep the asynchronous run for
> the hctx->dispatch_busy case.
> 
> fio, io_uring, 4k randread, 16 vCPU KVM guest:
> 
>                                                   before     after
>   null_blk, 1 queue, mq-deadline, qd16              193K      289K
>   null_blk, 1 queue, none, qd16                     354K      356K
>   null_blk, 4 queues, none, qd1                     356K      358K
>   ublk (ublksrv null), 1 queue, mq-deadline, qd16   113K      265K
>   ublk (ublksrv null), 4 x 1-queue devices, qd16    290K      852K
> 
> Fixes: 9cbbac29d752 ("block: Remove redundant plug in __submit_bio()")

Commit 9cbbac29d752 does need to be reevaluated.

Our customer ran fio tests with --bs=4K --rw=randwrite --iodepth=1 --numjobs=1
and confirmed that adding bio plugging to __submit_bio() improves performance.

Without bio plugging in __submit_bio():
fio:
__submit_bio                              (17.16%)
  → blk_mq_submit_bio                     (16.80%)
    → blk_mq_run_hw_queue                 (9.97%)
      → blk_mq_delay_run_hw_queue         (9.41%)
        → kblockd_mod_delayed_work_on     (9.13%)
          → mod_delayed_work_on           (9.11%)

kworker:
process_one_work                          (12.48%)
  → blk_mq_run_work_fn                    (11.70%)
    → blk_mq_sched_dispatch_requests      (11.00%)
      → __blk_mq_sched_dispatch_requests  (10.91%)
        → __blk_mq_do_dispatch_sched      (10.26%)
          → blk_mq_dispatch_rq_list       (7.84%)
            → nvme_queue_rq               (5.85%)

With bio plugging in __submit_bio():
__submit_bio                                (26.94%)
  → __blk_flush_plug                        (21.36%)
    → blk_mq_flush_plug_list                (21.24%)
      → blk_mq_flush_plug_list.part.0       (18.38%)
        → blk_mq_run_hw_queue               (15.34%)
          → blk_mq_sched_dispatch_requests  (14.55%)
            → __blk_mq_sched_dispatch_requests (11.27%)
              → blk_mq_dispatch_rq_list      (7.41%)
                → nvme_queue_rq              (5.72%)

Adding bio plugging increased IOPS from 60,000 to 74,000 and reduced latency
from 12.4 µs to 9.5 µs.

Commit 9cbbac29d752 states that bio plugging increases CPU usage. I suspect
that plugging may allow I/O to be dispatched faster, concentrating CPU
activity into a shorter period. This could increase instantaneous CPU
utilization while reducing the total test duration.

> Cc: stable@vger.kernel.org
> Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
> ---
>  block/blk-mq.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index a26a11c73ee3e..54eff3cf45fae 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -3200,8 +3200,16 @@ void blk_mq_submit_bio(struct bio *bio)
>  	}
>  
>  	hctx = rq->mq_hctx;
> -	if ((rq->rq_flags & RQF_USE_SCHED) ||
> -	    (hctx->dispatch_busy && (q->nr_hw_queues == 1 || !is_sync))) {
> +	if (rq->rq_flags & RQF_USE_SCHED) {
> +		/*
> +		 * Run the queue from the submitting context, as flushing a plug
> +		 * does. Punting every unplugged request to kblockd lets the
> +		 * kworker preempt the submitter once per I/O.
> +		 */
> +		blk_mq_insert_request(rq, 0);
> +		blk_mq_run_hw_queue(hctx, false);
> +	} else if (hctx->dispatch_busy &&
> +		   (q->nr_hw_queues == 1 || !is_sync)) {
>  		blk_mq_insert_request(rq, 0);
>  		blk_mq_run_hw_queue(hctx, true);
>  	} else {
> 
> ---
> base-commit: 551c722f40809618230001baccf219193e22fc5a
> change-id: 20260930-bug-blk-mq-noplug-sched-dispatch-4f19360b0bf7
> 
> Best regards,

-- 
Thanks
Kaitao Cheng