The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: Kaitao Cheng <kaitao.cheng@linux.dev>
To: Qiliang Yuan <qiliang.yuan@linux.dev>
Cc: Qiliang Yuan <odys.yuan@gmail.com>, Jens Axboe <axboe@kernel.dk>,
	Ming Lei <tom.leiming@gmail.com>,
	Wen Xiong <wenxiong@linux.ibm.com>,
	yukuai3@huawei.com, linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org, stable@vger.kernel.org,
	zhuzongzhen@sangfor.com.cn
Subject: Re: [PATCH] blk-mq: run the hw queue synchronously for unplugged scheduler inserts
Date: Wed, 18 Nov 2026 17:12:15 +0800	[thread overview]
Message-ID: <ac42e923-06bf-489d-bb9f-2ebec21b20f7@linux.dev> (raw)
In-Reply-To: <20261115022713.391768-1-qiliang.yuan@linux.dev>

在 2026/11/15 10:27, Qiliang Yuan 写道:
> Hi Kaitao,
> 
> On Thu, Oct 29, 2026 at 09:09:19PM +0800, Kaitao Cheng wrote:
>> Our customer ran fio tests with --bs=4K --rw=randwrite --iodepth=1 --numjobs=1
>> and confirmed that adding bio plugging to __submit_bio() improves performance.
>>
>> Without bio plugging in __submit_bio():
>> fio:
>> __submit_bio                              (17.16%)
>>   → blk_mq_submit_bio                     (16.80%)
>>     → blk_mq_run_hw_queue                 (9.97%)
>>       → blk_mq_delay_run_hw_queue         (9.41%)
>>         → kblockd_mod_delayed_work_on     (9.13%)
> [...]
>> With bio plugging in __submit_bio():
>> __submit_bio                                (26.94%)
>>   → __blk_flush_plug                        (21.36%)
>>     → blk_mq_flush_plug_list                (21.24%)
>>       → blk_mq_flush_plug_list.part.0       (18.38%)
>>         → blk_mq_run_hw_queue               (15.34%)
>>           → blk_mq_sched_dispatch_requests  (14.55%)
> [...]
>> Adding bio plugging increased IOPS from 60,000 to 74,000 and reduced latency
>> from 12.4 µs to 9.5 µs.
> 
> Thanks, this is the same pattern on NVMe with an I/O scheduler. Without
> the plug every request is dispatched by kblockd, with it the submitter
> dispatches the request itself through blk_mq_run_hw_queue(). This patch
> leaves __submit_bio() unplugged and runs the queue from the submitter
> for scheduler inserts, which should give the plugged profile without
> the plug.

I previously submitted a similar patch that went further, though the
implementation differs from yours. At the time, it was not intended
to address the plugging issue introduced by 9cbbac29d752.

For details, see:
https://lore.kernel.org/all/20261017105400.95967-1-kaitao.cheng@linux.dev/

> Could you run the same fio job on the unplugged kernel with this patch
> applied? Ming asked earlier whether real devices see a difference, so a
> Tested-by from an NVMe setup would help if it gets back to about 74K
> IOPS.
> 

+Cc: zhuzongzhen

zhuzongzhen helped with the testing back then. I'm not sure whether he
still has the test environment available.

-- 
Thanks
Kaitao Cheng


      reply	other threads:[~2026-11-18  9:12 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-30  7:38 [PATCH] blk-mq: run the hw queue synchronously for unplugged scheduler inserts Qiliang Yuan
2026-10-10  2:35 ` Qiliang Yuan
2026-10-10  3:05   ` Ming Lei
2026-10-15  2:16     ` Qiliang Yuan
2026-10-29 13:09 ` Kaitao Cheng
2026-11-15  2:27   ` Qiliang Yuan
2026-11-18  9:12     ` Kaitao Cheng [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ac42e923-06bf-489d-bb9f-2ebec21b20f7@linux.dev \
    --to=kaitao.cheng@linux.dev \
    --cc=axboe@kernel.dk \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=odys.yuan@gmail.com \
    --cc=qiliang.yuan@linux.dev \
    --cc=stable@vger.kernel.org \
    --cc=tom.leiming@gmail.com \
    --cc=wenxiong@linux.ibm.com \
    --cc=yukuai3@huawei.com \
    --cc=zhuzongzhen@sangfor.com.cn \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox