Re: [PATCH V3 0/8] block/scsi: safe SCSI quiescing

Oleksandr Natalenko
2017-09-02T14:47:27Z

Again,

Tested-by: Oleksandr Natalenko <oleksandr@natalenko.name>

On sobota 2. září 2017 15:08:32 CEST Ming Lei wrote:
> Hi,
>
> The current SCSI quiesce isn't safe and easy to trigger I/O deadlock.
>
> Once SCSI device is put into QUIESCE, no new request except for RQF_PREEMPT
> can be dispatched to SCSI successfully, and scsi_device_quiesce() just
> simply waits for completion of I/Os dispatched to SCSI stack. It isn't
> enough at all.
>
> Because new request still can be allocated, but all the allocated
> requests can't be dispatched successfully, so request pool can be
> consumed up easily.
>
> Then request with RQF_PREEMPT can't be allocated, and system may
> hang forever, such as during system suspend or SCSI domain alidation.
>
> Both IO hang inside system suspend[1] or SCSI domain validation
> were reported before.
>
> This patch introduces preempt freez, and tries to solve the issue
> by preempt freezing block queue during SCSI quiesce, and allows
> to allocate request of RQF_PREEMPT when queue is preempt-frozen.
>
> Both SCSI and SCSI_MQ have this IO deadlock issue, this patch fixes
> them all by introducing blk_freeze_queue_preempt() and
> blk_unfreeze_queue_preempt(), also unifying current interfaces for
> freezing queue between block legacy and blk-mq.
>
> Oleksandr has verified that this patchset V2 fixes his I/O hang
> during suspend/resume cycle.
>
> V3:
> - introduce q->preempt_unfreezing to fix one bug of preempt freeze
> - call blk_queue_enter_live() only when queue is preempt frozen
> - cleanup a bit on the implementation of preempt freeze
> - only patch 6 and 7 are changed
>
> V2:
> - drop the 1st patch in V1 because percpu_ref_is_dying() is
> enough as pointed by Tejun
> - introduce preempt version of blk_[freeze|unfreeze]_queue
> - sync between preempt freeze and normal freeze
> - fix warning from percpu-refcount as reported by Oleksandr
>
>
> [1] https://marc.info/?t=150340250100013&r=3&w=2
>
>
>
> Ming Lei (8):
> blk-mq: rename blk_mq_unfreeze_queue as blk_unfreeze_queue
> blk-mq: rename blk_mq_freeze_queue as blk_freeze_queue
> blk-mq: only run hw queues for blk-mq
> blk-mq: rename blk_mq_freeze_queue_wait as blk_freeze_queue_wait
> block: tracking request allocation with q_usage_counter
> block: introduce preempt version of blk_[freeze|unfreeze]_queue
> block: allow to allocate req with REQF_PREEMPT when queue is preempt
> frozen
> SCSI: preempt freeze block queue when SCSI device is put into quiesce
>
> block/bfq-iosched.c | 2 +-
> block/blk-cgroup.c | 8 +--
> block/blk-core.c | 53 ++++++++++++---
> block/blk-mq.c | 170
> +++++++++++++++++++++++++++++++++++++++-------- block/blk-mq.h |
> 1 -
> block/blk.h | 17 +++++
> block/elevator.c | 4 +-
> drivers/block/loop.c | 16 ++---
> drivers/block/rbd.c | 2 +-
> drivers/nvme/host/core.c | 8 +--
> drivers/scsi/scsi_lib.c | 22 +++++-
> include/linux/blk-mq.h | 15 +++--
> include/linux/blkdev.h | 21 +++++-
> 13 files changed, 273 insertions(+), 66 deletions(-)