Re: SCSI mid layer and high IOPS capable devices

[Bart Van Assche <bvanassche@acm.org>]

On 12/14/12 22:06, scameron@beardog.cce.hp.com wrote:
> [ ... ] how to get the scsi mid layer to provide a wide enough
> highway for requests destined for very low latency devices.

While the SCSI mid-layer is processing an I/O request not only the queue 
lock has to be locked and unlocked several times but also the SCSI host 
lock. The reason that it's unavoidable to lock and unlock the host lock 
is because the SCSI core has been designed for SCSI equipment that has a 
queue depth limit per host (shost->can_queue). For single LUN devices 
that model could be changed in a queue depth limit per LUN. Also, it's 
probably not that hard to modify software SCSI target implementations 
such that these have a queue depth limit per LUN instead of per host. It 
might be interesting to verify whether the following approach helps to 
improve performance of the SCSI mid-layer:
* Make it possible for SCSI LLDs to tell the SCSI core whether there is
   a queue depth limit per host or per LUN.
* Do not update shost->host_busy and shost->target_busy when using the
   QD limit per LUN mode. This change will allow to avoid spin_lock()
   and spin_unlock() calls inside scsi_request_fn(). It will also allow
   to avoid having to take the host lock inside scsi_device_unbusy().
* In queue-depth-limit-per-LUN mode, neither add a SCSI device to the
   starved list if it's busy nor examine the starved list in
   scsi_run_queue().

Bart.