From: Bart Van Assche <bvanassche@acm.org>
To: "Nicholas A. Bellinger" <nab@linux-iscsi.org>
Cc: linux-scsi <linux-scsi@vger.kernel.org>
Subject: Re: scsi-mq prototype
Date: Fri, 29 Nov 2013 15:08:54 +0100 [thread overview]
Message-ID: <52989FF6.4020307@acm.org> (raw)
In-Reply-To: <1373588612.7397.447.camel@haakon3.risingtidesystems.com>
On 07/12/13 02:23, Nicholas A. Bellinger wrote:
> [ ... ] I would like to discuss scsi-mq, a high performance SCSI
> initiator prototype that utilizes blk-mq [ ... ]
(replying to an e-mail of four months ago)
It took a while but I finally found some time to look further into
blk-mq and scsi-mq. Did I see correctly that the scsi-mq prototype
implements the so-called hw queues in the SCSI core and not in the SCSI
LLDs ? Sorry but that approach looks wrong to me. Several high-end
storage adapters already support multiple queues today. E.g. the NVMe
specification supports up to 64.000 I/O queues. With InfiniBand HCA's it
is possible to create multiple queues between initiator and target for
e.g. the iSER and SRP protocols to increase throughput. When there are
multiple queues between initiator and target there is a potential that
SCSI commands get reordered. The SCSI specifications define whether or
not it is allowed to change the submission order of commands. As an
example, if two atomic writes are submitted by the same CPU neither the
block layer nor the SCSI core nor the SCSI LLD nor the SCSI target is
allowed to reorder these. With the scsi-mq prototype present in your
repo the only way for a SCSI LLD to guarantee that SCSI commands for
which the submission order must be preserved do not get reordered is
either to limit the queue depth to one or to use a single queue for
communication with the target. I don't think this is what we want.
In other words, the so-called hw queue concept must be mapped onto
queues managed by the SCSI LLD and not onto queues managed by the SCSI
core. That will allow the block layer and the SCSI core to preserve SCSI
command submission order when necessary by queueing ordered commands on
the same hardware queue.
See also
https://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/log/?h=scsi-mq.
Bart.
next prev parent reply other threads:[~2013-11-29 14:08 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-07-12 0:23 [ATTEND] scsi-mq prototype discussion Nicholas A. Bellinger
2013-07-12 1:02 ` [Ksummit-2013-discuss] " Greg KH
2013-07-12 1:33 ` Nicholas A. Bellinger
2013-07-12 10:52 ` Hannes Reinecke
2013-07-13 6:53 ` James Bottomley
2013-07-16 21:07 ` Nicholas A. Bellinger
2013-07-16 21:15 ` Jens Axboe
2013-07-17 4:52 ` James Bottomley
2013-07-19 14:01 ` Ric Wheeler
2013-07-22 16:34 ` Jens Axboe
2013-07-19 21:22 ` Nicholas A. Bellinger
2013-07-19 21:46 ` James Bottomley
2013-07-19 22:06 ` Nicholas A. Bellinger
2013-07-16 22:19 ` scameron
2013-11-29 14:08 ` Bart Van Assche [this message]
2013-12-09 21:05 ` scsi-mq prototype Nicholas A. Bellinger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=52989FF6.4020307@acm.org \
--to=bvanassche@acm.org \
--cc=linux-scsi@vger.kernel.org \
--cc=nab@linux-iscsi.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).