[LSF/MM TOPIC] multiqueue and interrupt assignment

[Hannes Reinecke <hare@suse.de>]

Hi all,

here's another topic which I've hit during my performance tests:
How should interrupt affinity be handled with blk-multiqueue?

The problem is that the blk-multiqueue assumes a certain
CPU-to-queue mapping, _and_ the 'queue' in blk-mq syntax is actually
a submission/completion queue pair.

To achieve optimal performance one should set the interrupt affinity
for a given (hardware) queue to the matchine (blk-mq) queue.
But typically the interrupt affinity has to be set during HBA setup
ie way before any queues are allocated.
Which means we have three choices:
- outguess the blk-mq algorithm in the driver and set the
  interrupt affinity during HBA setup
- Add some callbacks to coordinate interrupt affinity between
  driver and blk-mq
- Defer it to manual assignment, but inferring the risk of
  a suboptimal performance.

At LSF/MM  I would like to have a discussion on how the interrupt
affinity should be handled for blk-mq, and whether a generic method
is possible or desirable.
Also there is the issue of certain drivers (eg lpfc) which normally
do interrupt affinity themselves, but disable it for multiqueue.
Which results in abysmal performance when comparing single queue
against multiqueue :-(

As a side note, what does blk-mq do if the interrupt affinity is
_deliberately_ set wrong? IE if the completions for one command
arrive on completely the wrong queue? Discard the completion? Move
it to the correct queue?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html