Re: [PATCH V3 11/17] SCSI: track pending admin commands

[Ming Lei <ming.lei@redhat.com>]

On Mon, Sep 17, 2018 at 10:46:34AM +0800, jianchao.wang wrote:
> Hi Ming
> 
> On 09/14/2018 07:33 PM, Ming Lei wrote:
> > On Fri, Sep 14, 2018 at 11:40 AM jianchao.wang
> > <jianchao.w.wang@oracle.com> wrote:
> >>
> >> Hi Ming
> >>
> >> On 09/13/2018 08:15 PM, Ming Lei wrote:
> >>>  EXPORT_SYMBOL(__scsi_execute);
> >>> @@ -3246,6 +3251,7 @@ static int scsi_internal_device_block(struct scsi_device *sdev)
> >>>               else
> >>>                       scsi_wait_for_queuecommand(sdev);
> >>>       }
> >>> +     wait_event(sdev->admin_wq, !atomic_read(&sdev->nr_admin_pending));
> >>>       mutex_unlock(&sdev->state_mutex);
> >>>
> >>>       return err;
> >> ...
> >>> diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
> >>> index 3aee9464a7bf..8bcb7ecc0c06 100644
> >>> --- a/drivers/scsi/scsi_sysfs.c
> >>> +++ b/drivers/scsi/scsi_sysfs.c
> >>> @@ -1393,6 +1393,7 @@ void __scsi_remove_device(struct scsi_device *sdev)
> >>>
> >>>       blk_cleanup_queue(sdev->request_queue);
> >>>       cancel_work_sync(&sdev->requeue_work);
> >>> +     wait_event(sdev->admin_wq, !atomic_read(&sdev->nr_admin_pending))
> >>
> >> This nr_admin_pending could drain the ongoing scsi_request_fn/scsi_queue_rq,
> >> but I'm afraid it cannot stop new ones coming in, such as the ones that have passed
> >> the sdev state checking and have not crossed the atomic_inc(&sdev->nr_admin_pending).
> >>
> > 
> > The counter of .nr_admin_pending is introduced for draining queued
> > admin requests to this scsi device.
> > 
> > Actually new requests have been prevented from entering scsi_queue_rq(),
> > please see the two callers of wait_event(sdev->admin_wq,
> > !atomic_read(&sdev->nr_admin_pending)).
> > 
> For example
> 
> _scsi_execute
> ...
>                                           scsi_internal_device_block
>                                             scsi_internal_device_block_nowait
>                                             blk_mq_quiesce_queue
>                                             wait_event(sdev->admin_wq, !atomic_read(&sdev->nr_admin_pending))
>   atomic_inc(&sdev->nr_admin_pending);
> 
>   blk_execute_rq（...）
> 
>   atomic_dec(&sdev->nr_admin_pending);
>   wake_up_all(&sdev->admin_wq);
> 
> Or do you mean the scsi_queue_rq -> scsi_prep_state_check could gate out of ?

I got it, then this issue can be fixed simply by moving atomic_inc/dec(&sdev->nr_admin_pending)
and related wake_up_all(&sdev->admin_wq) into scsi_admin_queue_rq().

Thanks,
Ming