Re: [LSF/MM ATTEND] interest in blk-mq, scsi-mq, dm-cache, dm-thinp, dm-*

[Mike Christie <michaelc@cs.wisc.edu>]

On 01/13/2014 05:36 AM, Hannes Reinecke wrote:
> On 01/10/2014 07:27 PM, Mike Snitzer wrote:
>> I would like to attend to participate in discussions related to topics
>> listed in the subject.  As a maintainer of DM I'd be interested to
>> learn/discuss areas that should become a development focus in the months
>> following LSF.
>>
> +1
> 
> I've been thinking on (re-) implementing multipathing on top of
> blk-mq, and would like to discuss the probability of which.
> There are some design decisions in blk-mq (eg statically allocating
> the number of queues) which do not play well with that.
> 

I think I have been thinking about going a completely different direction.

The thing about dm-multipath is that request based adds the extra queue
locking and that of course is bad. In our testing it is a major perf
issue. We got things like ioscheduling though.

If we went back to bio based multipathing then it turns out that when
scsi also supports multiqueue then it all works pretty nicely. There is
room for improvement in general like with some dm allocations being
numa/cpu aware, but the request_queue locking issues we have go away and
it is very simple code wise.

We could go the route of making request based dm-multipath:

1. aware of underlying multiqueue devices. So just basically keep what
we have more or less but then have dm-multipath make a request that can
be sent to a multiqueue device then call blk_mq_insert_request. This
would all be hidden by nice interfaces that hide if it is multiqueue
underlying device or not.

2. make dm-multipath do multiqueue (so implement map_queue, queue_rq,
etc) and also making it aware of underlying multiqueue devices.

#1 just keeps the existing request spin_lock problem so there is not
much point other than just getting things working.

#2 is a good deal of work and what does it end up buying us over just
making multipath bio based. We lose iosched support. If we are going to
make advanced multiqueue ioschedulers that rely on request structs then
#2 could be useful.