[RFC] blk-mq: complete request locallly if not in interrupt context

[RFC] blk-mq: complete request locallly if not in interrupt context

[Xiaoguang Wang]

For block devices that call blk_mq_complete_request() to end request
in process context, it's pointless to redirect the completion to be
done in block soft-irq, also blk_mq_raise_softirq() isnt't a very
light operation, which contains preempt and hard irq disable, wake
up ksoftirqd in non-interrupt context.

I found this issue while I use tcm_loop and tcmu(backstore is file)
to evaluate performance, tcm_loop end request in workqueue.
Without this patch: libaio engine, direct io, randwrite, io size 128k
job0: (groupid=0, jobs=1): err= 0: pid=20876: Thu Jan 27 15:33:45 2022
  write: IOPS=15.7k, BW=1966MiB/s (2062MB/s)(115GiB/60001msec); 0 zone resets
    slat (nsec): min=5675, max=83552, avg=8689.69, stdev=996.96
    clat (usec): min=231, max=99977, avg=498.89, stdev=501.69
     lat (usec): min=291, max=99986, avg=507.70, stdev=501.69

With this patch:
job0: (groupid=0, jobs=1): err= 0: pid=12813: Thu Jan 27 15:50:46 2022
  write: IOPS=16.8k, BW=2101MiB/s (2203MB/s)(123GiB/60001msec); 0 zone resets
    slat (usec): min=5, max=125, avg=14.12, stdev=10.31
    clat (usec): min=306, max=65380, avg=460.78, stdev=506.24
     lat (usec): min=341, max=65389, avg=475.04, stdev=505.27

Which improves throughput and reduces lat.

Signed-off-by: Xiaoguang Wang <xiaoguang.wang@linux.alibaba.com>
---
 block/blk-mq.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 8874a63ae952..9fdffc65d7ba 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -974,6 +974,9 @@ bool blk_mq_complete_request_remote(struct request *rq)
 		return true;
 	}
 
+	if (!in_interrupt())
+		return false;
+
 	if (rq->q->nr_hw_queues == 1) {
 		blk_mq_raise_softirq(rq);
 		return true;
-- 
2.14.4.44.g2045bb6

Re: [RFC] blk-mq: complete request locallly if not in interrupt context

[Christoph Hellwig]

On Thu, Jan 27, 2022 at 05:21:43PM +0800, Xiaoguang Wang wrote:
> For block devices that call blk_mq_complete_request() to end request
> in process context, it's pointless to redirect the completion to be
> done in block soft-irq, also blk_mq_raise_softirq() isnt't a very
> light operation, which contains preempt and hard irq disable, wake
> up ksoftirqd in non-interrupt context.
> 
> I found this issue while I use tcm_loop and tcmu(backstore is file)
> to evaluate performance, tcm_loop end request in workqueue.
> Without this patch: libaio engine, direct io, randwrite, io size 128k
> job0: (groupid=0, jobs=1): err= 0: pid=20876: Thu Jan 27 15:33:45 2022
>   write: IOPS=15.7k, BW=1966MiB/s (2062MB/s)(115GiB/60001msec); 0 zone resets
>     slat (nsec): min=5675, max=83552, avg=8689.69, stdev=996.96
>     clat (usec): min=231, max=99977, avg=498.89, stdev=501.69
>      lat (usec): min=291, max=99986, avg=507.70, stdev=501.69
> 
> With this patch:
> job0: (groupid=0, jobs=1): err= 0: pid=12813: Thu Jan 27 15:50:46 2022
>   write: IOPS=16.8k, BW=2101MiB/s (2203MB/s)(123GiB/60001msec); 0 zone resets
>     slat (usec): min=5, max=125, avg=14.12, stdev=10.31
>     clat (usec): min=306, max=65380, avg=460.78, stdev=506.24
>      lat (usec): min=341, max=65389, avg=475.04, stdev=505.27
> 
> Which improves throughput and reduces lat.

Just use blk_mq_complete_request_direct for callers that know they are
not in interrupt context.