[PATCH] scsi: mq queue stall during command requeue

[PATCH] scsi: mq queue stall during command requeue

[Hannes Reinecke]

When we're hitting a busy condition in queuecommand() we need to
stop the hardware queue before calling blk_mq_delay_queue(); if
we don't blk_mq_delay_queue() will not do anything and requeue
will never triggered.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/scsi_lib.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index e9e1e14..4b1cbc6 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -1951,8 +1951,10 @@ static int scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
 	switch (ret) {
 	case BLK_MQ_RQ_QUEUE_BUSY:
 		if (atomic_read(&sdev->device_busy) == 0 &&
-		    !scsi_device_blocked(sdev))
+		    !scsi_device_blocked(sdev)) {
+			blk_mq_stop_hw_queue(hctx);
 			blk_mq_delay_queue(hctx, SCSI_QUEUE_DELAY);
+		}
 		break;
 	case BLK_MQ_RQ_QUEUE_ERROR:
 		/*
-- 
1.8.5.6

Re: [PATCH] scsi: mq queue stall during command requeue

[Christoph Hellwig]

On Thu, Jan 19, 2017 at 03:17:45PM +0100, Hannes Reinecke wrote:
> When we're hitting a busy condition in queuecommand() we need to
> stop the hardware queue before calling blk_mq_delay_queue(); if
> we don't blk_mq_delay_queue() will not do anything and requeue
> will never triggered.

I suspect the stop needs to move into blk_mq_delay_queue to make
it more useful.

Re: [PATCH] scsi: mq queue stall during command requeue

[Hannes Reinecke]

On 01/19/2017 03:46 PM, Christoph Hellwig wrote:
> On Thu, Jan 19, 2017 at 03:17:45PM +0100, Hannes Reinecke wrote:
>> When we're hitting a busy condition in queuecommand() we need to
>> stop the hardware queue before calling blk_mq_delay_queue(); if
>> we don't blk_mq_delay_queue() will not do anything and requeue
>> will never triggered.
> 
> I suspect the stop needs to move into blk_mq_delay_queue to make
> it more useful.
> 
Which is what I've suggested to Jens, too.
And he even agreed to it :-)

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

Re: [PATCH] scsi: mq queue stall during command requeue

[Jens Axboe]

On 01/19/2017 06:52 AM, Hannes Reinecke wrote:
> On 01/19/2017 03:46 PM, Christoph Hellwig wrote:
>> On Thu, Jan 19, 2017 at 03:17:45PM +0100, Hannes Reinecke wrote:
>>> When we're hitting a busy condition in queuecommand() we need to
>>> stop the hardware queue before calling blk_mq_delay_queue(); if
>>> we don't blk_mq_delay_queue() will not do anything and requeue
>>> will never triggered.
>>
>> I suspect the stop needs to move into blk_mq_delay_queue to make
>> it more useful.
>>
> Which is what I've suggested to Jens, too.
> And he even agreed to it :-)

Yeah, I made that patch and grepped the tree. The usage in the fc
code in nvme looks weird. It's stopping all potential hardware queues,
yet only delaying one.

if (op->rq) {                                                   
        blk_mq_stop_hw_queues(op->rq->q);                       
        blk_mq_delay_queue(queue->hctx, NVMEFC_QUEUE_DELAY);    
}                                                               
return BLK_MQ_RQ_QUEUE_BUSY;

James/Christoph, what's going on there? I'm going to leave that one
as-is for now, but add the stop to the delay. If the above stop-all is
indeed buggy, then it can later just be removed.

-- 
Jens Axboe

Re: [PATCH] scsi: mq queue stall during command requeue

[Christoph Hellwig]

On Thu, Jan 19, 2017 at 06:58:09AM -0800, Jens Axboe wrote:
> Yeah, I made that patch and grepped the tree. The usage in the fc
> code in nvme looks weird. It's stopping all potential hardware queues,
> yet only delaying one.
> 
> if (op->rq) {                                                   
>         blk_mq_stop_hw_queues(op->rq->q);                       
>         blk_mq_delay_queue(queue->hctx, NVMEFC_QUEUE_DELAY);    
> }                                                               
> return BLK_MQ_RQ_QUEUE_BUSY;
> 
> James/Christoph, what's going on there? I'm going to leave that one
> as-is for now, but add the stop to the delay. If the above stop-all is
> indeed buggy, then it can later just be removed.

It looks broken to me, but I'll need James to confirm.

Re: [PATCH] scsi: mq queue stall during command requeue

[Bart Van Assche]

On Thu, 2017-01-19 at 15:17 +0100, Hannes Reinecke wrote:
> When we're hitting a busy condition in queuecommand() we need to
> stop the hardware queue before calling blk_mq_delay_queue(); if
> we don't blk_mq_delay_queue() will not do anything and requeue
> will never triggered.
> 
> Signed-off-by: Hannes Reinecke <hare@suse.com>
> ---
>  drivers/scsi/scsi_lib.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
> index e9e1e14..4b1cbc6 100644
> --- a/drivers/scsi/scsi_lib.c
> +++ b/drivers/scsi/scsi_lib.c
> @@ -1951,8 +1951,10 @@ static int scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
>  	switch (ret) {
>  	case BLK_MQ_RQ_QUEUE_BUSY:
>  		if (atomic_read(&sdev->device_busy) == 0 &&
> -		    !scsi_device_blocked(sdev))
> +		    !scsi_device_blocked(sdev)) {
> +			blk_mq_stop_hw_queue(hctx);
>  			blk_mq_delay_queue(hctx, SCSI_QUEUE_DELAY);
> +		}
>  		break;
>  	case BLK_MQ_RQ_QUEUE_ERROR:
>  		/*

You may want to add Fixes: 52d7f1b5c2f3 (blk-mq: Avoid that requeueing
starts stopped queues). See also
https://www.spinics.net/lists/linux-block/msg06144.html.

Bart.

Re: [PATCH] scsi: mq queue stall during command requeue

[Hannes Reinecke]

On 01/19/2017 05:11 PM, Bart Van Assche wrote:
> On Thu, 2017-01-19 at 15:17 +0100, Hannes Reinecke wrote:
>> When we're hitting a busy condition in queuecommand() we need to
>> stop the hardware queue before calling blk_mq_delay_queue(); if
>> we don't blk_mq_delay_queue() will not do anything and requeue
>> will never triggered.
>>
>> Signed-off-by: Hannes Reinecke <hare@suse.com>
>> ---
>>  drivers/scsi/scsi_lib.c | 4 +++-
>>  1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
>> index e9e1e14..4b1cbc6 100644
>> --- a/drivers/scsi/scsi_lib.c
>> +++ b/drivers/scsi/scsi_lib.c
>> @@ -1951,8 +1951,10 @@ static int scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
>>  	switch (ret) {
>>  	case BLK_MQ_RQ_QUEUE_BUSY:
>>  		if (atomic_read(&sdev->device_busy) == 0 &&
>> -		    !scsi_device_blocked(sdev))
>> +		    !scsi_device_blocked(sdev)) {
>> +			blk_mq_stop_hw_queue(hctx);
>>  			blk_mq_delay_queue(hctx, SCSI_QUEUE_DELAY);
>> +		}
>>  		break;
>>  	case BLK_MQ_RQ_QUEUE_ERROR:
>>  		/*
> 
> You may want to add Fixes: 52d7f1b5c2f3 (blk-mq: Avoid that requeueing
> starts stopped queues). See also
> https://www.spinics.net/lists/linux-block/msg06144.html.
> 
Yes indeed.

But the powers that be still ponder if it should be fixed in
the SCSI stack or whether it's a generic fix for blk_mq_delay_queue().

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)