Re: [PATCH v2 10/11] megaraid_sas: Use Block layer API to check SCSI device in-flight IO requests

John Garry
2020-03-02T09:51:51Z

>> static inline void
>> megasas_get_msix_index(struct megasas_instance *instance,
>>                 struct scsi_cmnd *scmd,
>>                 struct megasas_cmd_fusion *cmd,
>>                 u8 data_arms)
>> {
>> ...
>>
>> sdev_busy = atomic_read(&hctx->nr_active);
>>
>> if (instance->perf_mode == MR_BALANCED_PERF_MODE &&
>>      sdev_busy > (data_arms * MR_DEVICE_HIGH_IOPS_DEPTH))
>>      cmd->request_desc->SCSIIO.MSIxIndex =
>>              mega_mod64(...),
>>      else if (instance->msix_load_balance)
>>          cmd->request_desc->SCSIIO.MSIxIndex =
>>              (mega_mod64(...),
>>                  instance->msix_vectors));
>>
>> Will this make a difference? I am not sure. Maybe, on this basis,
>> magaraid sas is not a good candidate to change to expose multiple queues.
>>
>> Ignoring that for a moment, since we no longer keep a host busy count,
>> and I figure that we don't want to back to using
>> scsi_device.device_busy, is the judgement of hctx->nr_active ok to use
>> to decide whether to use these performance queues?
>>
> Personally, I wonder if the current implementation of high-IOPs queues
> makes sense with multiqueue. > Thing is, the current high-IOPs queue mechanism of shifting I/O to
> another internal queue doesn't align nicely with the blk-mq architecture.

Right, we should not be hiding HW queues from blk-mq like this. This
breaks the symmetry. Maybe we can move this functionality to blk-mq, but
I doubt that this is a common use case.

> What we _do_ have, though, is a 'poll' queue mechanism, allowing to
> separate out one (or several) queues for polling, to allow for ..
> indeed, high-IOPs.

Any examples or references for this?

> So it would be interesting to figure out if we don't get similar
> performance by using the 'poll' queue implementation from blk-mq instead
> of the current one.

I thought that this driver/or mpt3sas already used a polling mode.

And for these low-latency queues, I figure that the issue is not just
polling vs interrupt, but indeed how fast the HW queue can consume SQEs.
A HW queue may only be able to consume at a limited rate - that's why we
segregate.

As an aside, that is actually an issue for blk-mq. For 1 to many HW
queue-to-CPU mapping, limiting many CPUs a single queue can limit IOPs
since HW queues can only consume at a limited rate.

>
> Which would also have the benefit that we could support the io_uring
> interface natively with megaraid_sas, which I think would be a benefit
> on its own.
>

thanks,
John