Re: [PATCH v2 10/11] megaraid_sas: Use Block layer API to check SCSI device in-flight IO requests

John Garry
2020-03-03T11:53:35Z

>> And for these low-latency queues, I figure that the issue is not just
>> polling vs interrupt, but indeed how fast the HW queue can consume SQEs.
>> A HW queue may only be able to consume at a limited rate - that's why we
>> segregate.
> Yes, there is no polling in any of HW queues. High IOPs queues have
> interrupt coalescing enabled whereas
> low latency queues does not have interrupt coalescing. megaraid_sas
> driver would choose which set of queues
> among these two has to be used depending on workload. For latency
> oriented workload, driver would use low
> latency queues and for IOPs profile, driver would use High IOPs queues.
>>
>> As an aside, that is actually an issue for blk-mq. For 1 to many HW
>> queue-to-CPU mapping, limiting many CPUs a single queue can limit IOPs
>> since HW queues can only consume at a limited rate.
> We were able to achieve performance target for MegaRAID latest gen
> controller with this model of few set
> of HW queues mapped to local numa CPUs and low latency queues has one
> to one mapping to CPUs.
> This is default behavior of queues segregation in megaraid_sas driver
> to satisfy our IOPs and latency requirements altogether.
> However we have given module parameter- "perf_mode" to tune queues
> behavior. i.e turning on/off interrupt
> coalescing on all HW queues where this one to many queues to CPU
> mapping would not happen.

Hi Sumit,

OK, I have a rough idea of the concept. And again I'd say megaraid sas
may not be a good candidate to expose > 1 HW queues, as we hide HW
queues and don't maintain the symmetry with blk-mq layer.

Indeed, I do not even expect a performance increase in exposing > 1 HW
queues since the driver already uses the reply map + managed interrupts.

The main reason for that change in some drivers - apart from losing the
duplicated ugliness of the reply map - is to leverage the blk-mq feature
to drain a hctx for CPU hotplug [0] - is this something which megaraid
sas is vulnerable to and would benefit from?

Thanks,
John

[0]