Re: [PATCH v2] blk-mq: release scheduler resource when request complete

[Chengming Zhou <chengming.zhou@linux.dev>]

On 2023/8/17 23:29, Chengming Zhou wrote:
> On 2023/8/17 22:50, Bart Van Assche wrote:
>> On 8/17/23 07:41, kernel test robot wrote:
>>> [  222.622837][ T2216] statistics for priority 1: i 276 m 0 d 276 c 278
>>> [ 222.629307][ T2216] WARNING: CPU: 0 PID: 2216 at block/mq-deadline.c:680 dd_exit_sched (block/mq-deadline.c:680 (discriminator 3))
>>
>> The above information shows that dd_inserted_request() has been called
>> 276 times and also that dd_finish_request() has been called 278 times.
> 
> Thanks much for your help.
> 
> This patch indeed introduced a regression, postflush requests will be completed
> twice, so here dd_finish_request() is more than dd_inserted_request().
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index a8c63bef8ff1..7cd47ffc04ce 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -686,8 +686,10 @@ static void blk_mq_finish_request(struct request *rq)
>  {
>         struct request_queue *q = rq->q;
> 
> -       if (rq->rq_flags & RQF_USE_SCHED)
> +       if (rq->rq_flags & RQF_USE_SCHED) {
>                 q->elevator->type->ops.finish_request(rq);
> +               rq->rq_flags &= ~RQF_USE_SCHED;
> +       }
>  }
> 

I just tried to run LKP and xfstests, firstly failed to run LKP on my server
which seems to miss some dependencies. Then I ran xfstests successfully.

But xfstests generic/704 always pass and no WARN in dmesg. (I don't know why,
maybe my server settings are some different from the test robot.)

So I try to reproduce it manually. Steps:

```
echo mq-deadline > /sys/block/sdb/queue/scheduler

mkfs.ext4 /dev/sdb
mount /dev/sdb /fs/sdb
cd /fs/sdb
stress-ng --symlink 4 --timeout 60

echo none > /sys/block/sdb/queue/scheduler
```

This way the WARNING in mq-deadline can be reproduced easily.

Then retest with the diff, mq-deadline WARNING still happened... So there
are still other requests which have RQF_USE_SCHED flag completed without
being inserted into elevator.

Will use some tracing and look again.

Thanks.

> 
> Clear RQF_USE_SCHED flag here should fix this problem, which should be ok
> since finish_request() is the last callback, this flag isn't needed anymore.
> 
> Jens, should I send this diff as another patch or resend updated v3?
> 
> Thanks.
> 
>> Calling dd_finish_request() more than once per request breaks the code
>> for priority handling since that code checks how many requests are
>> pending per priority level by subtracting the number of completion calls
>> from the number of insertion calls (see also dd_queued()). I think the
>> above output indicates that this patch introduced a regression.
>>
>> Bart.
>>