Re: [PATCH v8 00/18] blk-mq/scsi: Provide hostwide shared tags for SCSI HBAs

Hannes Reinecke
2020-09-08T19:41:41Z

On 8/19/20 5:20 PM, John Garry wrote:
> Hi all,
>
> Here is v8 of the patchset.
>
> In this version of the series, we keep the shared sbitmap for driver tags,
> and introduce changes to fix up the tag budgeting across request queues.
> We also have a change to count requests per-hctx for when an elevator is
> enabled, as an optimisation. I also dropped the debugfs changes - more on
> that below.
>
> Some performance figures:
>
> Using 12x SAS SSDs on hisi_sas v3 hw. mq-deadline results are included,
> but it is not always an appropriate scheduler to use.
>
> Tag depth 4000 (default) 260**
>
> Baseline (v5.9-rc1):
> none sched: 2094K IOPS 513K
> mq-deadline sched: 2145K IOPS 1336K
>
> Final, host_tagset=0 in LLDD *, ***:
> none sched: 2120K IOPS 550K
> mq-deadline sched: 2121K IOPS 1309K
>
> Final ***:
> none sched: 2132K IOPS 1185
> mq-deadline sched: 2145K IOPS 2097
>
> * this is relevant as this is the performance in supporting but not
> enabling the feature
> ** depth=260 is relevant as some point where we are regularly waiting for
> tags to be available. Figures were are a bit unstable here.
> *** Included "[PATCH V4] scsi: core: only re-run queue in
> scsi_end_request() if device queue is busy"
>
> A copy of the patches can be found here:
> https://github.com/hisilicon/kernel-dev/tree/private-topic-blk-mq-shared-tags-v8
>
> The hpsa patch depends on:
> https://lore.kernel.org/linux-scsi/20200430131904.5847-1-hare@suse.de/
>
> And the smartpqi patch is not to be accepted.
>
> Comments (and testing) welcome, thanks!
>
> Differences to v7:
> - Add null_blk and scsi_debug support
> - Drop debugfs tags patch - it's too difficult to be the same between
> hostwide and non-hostwide, as discussed:
> https://lore.kernel.org/linux-scsi/1591810159-240929-1-git-send-email-john.garry@huawei.com/T/#mb3eb462d8be40273718505989abd12f8228c15fd
> And from commit 6bf0eb550452 ("sbitmap: Consider cleared bits in
> sbitmap_bitmap_show()"), I guess not many used this anyway...
> - Add elevator per-hctx request count for optimisation
> - Break up "blk-mq: rename blk_mq_update_tag_set_depth()" into 2x patches
> - Pass flags for avoid per-hq queue tags init/free for hostwide tags
> - Add Don's reviewed-tag and tested-by tags to appropiate patches
> - (@Don, please let me know if issue with how I did this)
> - Add "scsi: core: Show nr_hw_queues in sysfs"
> - Rework megaraid SAS patch to have module param (Kashyap)
> - rebase
>
> V7 is here for more info:
> https://lore.kernel.org/linux-scsi/1591810159-240929-1-git-send-email-john.garry@huawei.com/T/#t
>
> Hannes Reinecke (5):
> blk-mq: Rename blk_mq_update_tag_set_depth()
> blk-mq: Free tags in blk_mq_init_tags() upon error
> scsi: Add host and host template flag 'host_tagset'
> hpsa: enable host_tagset and switch to MQ
> smartpqi: enable host tagset
>
> John Garry (10):
> blk-mq: Pass flags for tag init/free
> blk-mq: Use pointers for blk_mq_tags bitmap tags
> blk-mq: Facilitate a shared sbitmap per tagset
> blk-mq: Relocate hctx_may_queue()
> blk-mq: Record nr_active_requests per queue for when using shared
> sbitmap
> blk-mq: Record active_queues_shared_sbitmap per tag_set for when using
> shared sbitmap
> null_blk: Support shared tag bitmap
> scsi: core: Show nr_hw_queues in sysfs
> scsi: hisi_sas: Switch v3 hw to MQ
> scsi: scsi_debug: Support host tagset
>
> Kashyap Desai (2):
> blk-mq, elevator: Count requests per hctx to improve performance
> scsi: megaraid_sas: Added support for shared host tagset for
> cpuhotplug
>
> Ming Lei (1):
> blk-mq: Rename BLK_MQ_F_TAG_SHARED as BLK_MQ_F_TAG_QUEUE_SHARED
>
Now that Jens merged the block bits in his tree, wouldn't it be better
to re-send the SCSI bits only, thereby avoiding a potential merge error
later on?

Cheers,

Hannes
--
Dr. Hannes Reinecke Kernel Storage Architect
hare@suse.de +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), GF: Felix Imendörffer