Re: [PATCH] blk-mq: Fix cpu indexing error in blk_mq_alloc_request_hctx()

[Ming Lei <ming.lei@redhat.com>]

On Thu, Oct 24, 2019 at 11:53:26AM -0700, James Smart wrote:
> On 10/24/2019 6:02 AM, Jens Axboe wrote:
> > On 10/24/19 3:28 AM, Ming Lei wrote:
> > > The normal usage is that user doesn't specify the hctx for allocating
> > > request from, since blk-mq
> > > can figure out which hctx is used for allocation via queue mapping.
> > > Just wondering why NVMe
> > > FC/RDMA can't do that way?
> > 
> > Fully agree, it'd be much nicer if that weird interface could just
> > die.
> > 
> 
> Well.  All depends on what you think a hctx corresponds to, which relates to
> why the caller originally set nr_hw_queues to what it did. I think it is
> reasonable for the caller to say - this must be via this specific context.

Usually MQ is only for handling IO efficiently, and we seldom use
MQ for management purpose, such as, so far all admin queue is SQ.

> 
> To the nvme fabric transports (rdma, fc, tcp) the hctx corresponds to the
> nvme controller queue to issue a request on. In the single case where the

Could you explain a bit about what the purpose of nvme controller is ?

> hctx is specified specifically, it is the 1st command on a new nvme
> controller queue. The command *must* be issued on the queue it is to
> initialize (this is different from pci nvme).  The hctx is specified so the
> correct nvme queue is selected when the command comes down the request path.
> Saying "don't do that" means one of the following: a) snooping every rq on
> the request path to spot initialization ios and move them to the right
> queue; or b) creating a duplicate non-blk-mq request path for this 1
> initialization io. Both of those are ugly.

In nvmf_connect_io_queue(), 'qid' has been encoded into instance of 'struct
nvme_command', that means the 'nvme controller' should know the
specified queue by parsing the command. So still not understand why you
have to submit the command via the specified queue.

Thanks,
Ming