Re: [PATCH] blk-mq: Allow complete locally if capacities are different

[Bart Van Assche <bvanassche@acm.org>]

On 8/28/24 7:49 AM, Manish Pandey wrote:
> 'Commit af550e4c9682 ("block/blk-mq: Don't complete locally if
> capacities are different")' enforces to complete the request locally
> only if the submission and completion CPUs have same capacity.
> 
> To have optimal IO load balancing or to avoid contention b/w submission
> path and completion path, user may need to complete IO request of large
> capacity CPU(s) on Small Capacity CPU(s) or vice versa.
> 
> Hence introduce a QUEUE_FLAG_ALLOW_DIFF_CAPACITY blk queue flag to let
> user decide if it wants to complete the request locally or need an IPI
> even if the capacity of the requesting and completion queue is different.
> This gives flexibility to user to choose best CPU for their completion
> to give best performance for their system.

I think that the following is missing from the above description:
- Mentioning that this is for an unusual interrupt routing technology
   (SoC sends the interrupt to another CPU core than what has been
    specified in the smp_affinity mask).
- An explanation why the desired effect cannot be achieved by changing
   rq_affinity into 0.

>   block/blk-mq-debugfs.c |  1 +
>   block/blk-mq.c         |  3 ++-
>   block/blk-sysfs.c      | 12 ++++++++++--
>   include/linux/blkdev.h |  1 +
>   4 files changed, 14 insertions(+), 3 deletions(-)

Since the semantics of a sysfs attribute are modified,
Documentation/ABI/stable/sysfs-block should be updated.

Thanks,

Bart.