From: Uday Shankar <ushankar@purestorage.com>
To: Yoav Cohen <yoav@nvidia.com>
Cc: "linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
"ming.lei@redhat.com" <ming.lei@redhat.com>,
"axboe@kernel.dk" <axboe@kernel.dk>,
Guy Eisenberg <geisenberg@nvidia.com>,
Jared Holzman <jholzman@nvidia.com>
Subject: Re: ublk: slow recovery process when io queue depth is low
Date: Thu, 17 Apr 2025 12:12:14 -0600 [thread overview]
Message-ID: <aAFEfqhdRikKfMnu@dev-ushankar.dev.purestorage.com> (raw)
In-Reply-To: <DM4PR12MB6328BA60A2F4C041C3181BC3A9BC2@DM4PR12MB6328.namprd12.prod.outlook.com>
On Thu, Apr 17, 2025 at 10:06:36AM +0000, Yoav Cohen wrote:
> Hi,
>
> I'm running ublk on Ubuntu Kenrel 6.8.0-56-generic.
> I notice that if I'm running an IO commands that causing the IO queues to be full (at least the ublk hw queue) it seems like some of the IO's are done 30 seconds~ after they where submitted.
> Enlarging the IO queues fixed the issue, and I'm pretty sure the 30 seconds magic number may be result of blk_mq_tag_set timeout filed default(see blk_mq_init_allocated_queue where it set to 30 * HZ)
Correct, that's where the 30s comes from.
> Can you guys explain the behaviour?
The behavior you describe was actually just fixed yesterday by [1]; see
that commit message for a more detailed explanation. You might also be
interested in [2], which is a test that recreates exactly the behavior
you describe. Before the fix in [1], the test saw an I/O take ~30s to
complete, but after the fix, the I/O completes quickly.
[1] https://lore.kernel.org/linux-block/20250416035444.99569-6-ming.lei@redhat.com/
[2] https://lore.kernel.org/linux-block/20250416035444.99569-9-ming.lei@redhat.com/
prev parent reply other threads:[~2025-04-17 18:12 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-04-17 10:06 ublk: slow recovery process when io queue depth is low Yoav Cohen
2025-04-17 14:22 ` Jens Axboe
2025-04-17 18:12 ` Uday Shankar [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aAFEfqhdRikKfMnu@dev-ushankar.dev.purestorage.com \
--to=ushankar@purestorage.com \
--cc=axboe@kernel.dk \
--cc=geisenberg@nvidia.com \
--cc=jholzman@nvidia.com \
--cc=linux-block@vger.kernel.org \
--cc=ming.lei@redhat.com \
--cc=yoav@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).