Re: [PATCH 2/2 v2] nvme-rdma: fix controller reset hang during traffic

[Christoph Hellwig <hch@lst.de>]

Why is this a 2/2 v2, where is the 1/2?

On Mon, Jul 27, 2020 at 05:32:09PM -0700, Sagi Grimberg wrote:
> commit fe35ec58f0d3 ("block: update hctx map when use multiple maps")
> exposed an issue where we may hang trying to wait for queue freeze
> during I/O. We call blk_mq_update_nr_hw_queues which in case of multiple
> queue maps (which we have now for default/read/poll) is attempting to
> freeze the queue. However we never started queue freeze when starting the
> reset, which means that we have inflight pending requests that entered the
> queue that we will not complete once the queue is quiesced.
> 
> So start a freeze before we quiesce the queue, and unfreeze the queue
> after we successfully connected the I/O queues (and make sure to call
> blk_mq_update_nr_hw_queues only after we are sure that the queue was
> already frozen).
> 
> This follows to how the pci driver handles resets.
> 
> Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
> ---
> Changes from v1:
> - fix silly compilation errors
> 
>  drivers/nvme/host/rdma.c | 12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index 5c3848974ccb..44c76ffbb264 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -967,15 +967,20 @@ static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
>  			ret = PTR_ERR(ctrl->ctrl.connect_q);
>  			goto out_free_tag_set;
>  		}
> -	} else {
> -		blk_mq_update_nr_hw_queues(&ctrl->tag_set,
> -			ctrl->ctrl.queue_count - 1);
>  	}
>  
>  	ret = nvme_rdma_start_io_queues(ctrl);
>  	if (ret)
>  		goto out_cleanup_connect_q;
>  
> +	if (!new) {
> +		nvme_start_queues(&ctrl->ctrl);
> +		nvme_wait_freeze(&ctrl->ctrl);
> +		blk_mq_update_nr_hw_queues(ctrl->ctrl.tagset,
> +			ctrl->ctrl.queue_count - 1);
> +		nvme_unfreeze(&ctrl->ctrl);
> +	}
> +
>  	return 0;
>  
>  out_cleanup_connect_q:
> @@ -1008,6 +1013,7 @@ static void nvme_rdma_teardown_io_queues(struct nvme_rdma_ctrl *ctrl,
>  		bool remove)
>  {
>  	if (ctrl->ctrl.queue_count > 1) {
> +		nvme_start_freeze(&ctrl->ctrl);
>  		nvme_stop_queues(&ctrl->ctrl);
>  		nvme_rdma_stop_io_queues(ctrl);
>  		if (ctrl->ctrl.tagset) {
> -- 
> 2.25.1
---end quoted text---

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme