Re: [LSF/MM TOPIC] multiqueue and interrupt assignment

[Hannes Reinecke <hare@suse.de>]

On 02/03/2016 01:57 PM, Sagi Grimberg wrote:
> Hi Bart and Hannes,
> 
>> This topic indeed needs further attention. I also encountered this
>> challenge while adding scsi-mq support to the SRP initiator
>> driver. What
>> I learned while working on the SRP driver is the following:
>> - Although I agree that requests and interrupts should be
>> processed on
>>    the same processor (same physical chip) if the request has been
>>    submitted from the CPU closest to the HBA, I'm not convinced that
>>    processing request completions and interrupts on the same CPU core
>>    yields the best performance. I would appreciate it if there would
>>    remain some freedom in how to assign interrupts to CPU cores.
> 
> This is true not only for this reason. Some block storage transports
> (e.g. srp/iser) share the HBA with the networking stack and possibly
> with user-space workloads in the case of RDMA. This is why I don't see
> how would MSIX assignments can be done anywhere other than user-space.
> 
Oh, I don't doubt that we should allow to have the MSIX assignments
from user-space.
But ATM block-mq assigned a fixed CPU <-> hardware queue mapping,
and all we can do is to fixup things afterwards.
Irrespective on whether it's 'best' for any given hardware.

> However what I think we can do is have blk-mq ask the drivers
> information about the MSIX mappings. This concept was introduced in
> 2011 by Ben Hutchings with the CPU affinity reverse-mapping API [1].
> 
> Perhaps we'd want to have drivers provide blk-mq a struct cpu_rmap when
> assigning the hctx mappings, or possibly per I/O if we want to be
> agnostic to MSIX topology changes. I think this approach would solve
> Hannes is experiencing.
> 
> Thoughts?
> 
Indeed, something like this.
Quite some issues would be solved if we could push a hctx mapping
into blk-mq, instead of having it assign its own made-up one.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html