Re: [PATCH] block: Improve shared tag set performance

[Ming Lei <ming.lei@redhat.com>]

On Sat, Oct 21, 2023 at 09:13:38AM -0700, Bart Van Assche wrote:
> On 10/20/23 18:31, Ming Lei wrote:
> > If two LUNs are attached to same host, one is slow, and another is fast,
> > and the slow LUN can slow down the fast LUN easily without this fairness
> > algorithm.
> > 
> > Your motivation is that "One of these logical units (WLUN) is used
> > to submit control commands, e.g. START STOP UNIT. If any request is
> > submitted to the WLUN, the queue depth is reduced from 31 to 15 or
> > lower for data LUNs." I guess one simple fix is to not account queues
> > of this non-IO LUN as active queues?
> 
> Hi Ming,
> 
> For fast storage devices (e.g. UFS) any time spent in an algorithm for
> fair sharing will reduce IOPS. If there are big differences in the
> request processing latency between different request queues then fair
> sharing is beneficial. Whether or not the fair sharing algorithm is
> improved, how about making it easy to disable fair sharing, e.g. with
> something like the untested patch below? I think that will work better
> than ignoring fair sharing per LUN. UFS devices support multiple logical
> units and with the current fair sharing approach it takes long until
> tags are taken away from an inactive LUN (request queue timeout).
> 
> Thanks,
> 
> Bart.
> 
> 
> diff --git a/block/blk-mq.h b/block/blk-mq.h
> index f75a9ecfebde..b06b161d06de 100644
> --- a/block/blk-mq.h
> +++ b/block/blk-mq.h
> @@ -416,7 +416,8 @@ static inline bool hctx_may_queue(struct blk_mq_hw_ctx
> *hctx,
>  {
>  	unsigned int depth, users;
> 
> -	if (!hctx || !(hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED))
> +	if (!hctx || !(hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED) ||
> +	    hctx->queue->disable_fair_sharing)

Maybe you can propagate this flag into hctx->flags, then
hctx->queue->disable_fair_sharing can be avoided in fast path.

>  		return true;
> 
>  	/*
> diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
> index eef450f25982..63b04cf65887 100644
> --- a/include/linux/blkdev.h
> +++ b/include/linux/blkdev.h
> @@ -523,6 +523,7 @@ struct request_queue {
>  	struct mutex		debugfs_mutex;
> 
>  	bool			mq_sysfs_init_done;
> +	bool			disable_fair_sharing;

You also need to bypass blk_mq_tag_busy() & blk_mq_tag_idle()
in case of disable_fair_sharing which should only be set for
non-IO queues, such as UFS WLUN, and maybe nvme connection queues.


Thanks, 
Ming