public inbox for linux-scsi@vger.kernel.org
 help / color / mirror / Atom feed
From: Bart Van Assche <bvanassche@acm.org>
To: "Martin K . Petersen" <martin.petersen@oracle.com>
Cc: Jaegeuk Kim <jaegeuk@kernel.org>,
	Adrian Hunter <adrian.hunter@intel.com>,
	linux-scsi@vger.kernel.org, Bart Van Assche <bvanassche@acm.org>,
	"James E.J. Bottomley" <jejb@linux.ibm.com>,
	Bean Huo <beanhuo@micron.com>, Can Guo <cang@codeaurora.org>,
	Avri Altman <avri.altman@wdc.com>,
	Stanley Chu <stanley.chu@mediatek.com>,
	Asutosh Das <asutoshd@codeaurora.org>
Subject: [PATCH v2 13/20] scsi: ufs: Fix a deadlock in the error handler
Date: Fri, 19 Nov 2021 11:57:36 -0800	[thread overview]
Message-ID: <20211119195743.2817-14-bvanassche@acm.org> (raw)
In-Reply-To: <20211119195743.2817-1-bvanassche@acm.org>

The following deadlock has been observed on a test setup:
* All tags allocated.
* The SCSI error handler calls ufshcd_eh_host_reset_handler()
* ufshcd_eh_host_reset_handler() queues work that calls ufshcd_err_handler()
* ufshcd_err_handler() locks up as follows:

Workqueue: ufs_eh_wq_0 ufshcd_err_handler.cfi_jt
Call trace:
 __switch_to+0x298/0x5d8
 __schedule+0x6cc/0xa94
 schedule+0x12c/0x298
 blk_mq_get_tag+0x210/0x480
 __blk_mq_alloc_request+0x1c8/0x284
 blk_get_request+0x74/0x134
 ufshcd_exec_dev_cmd+0x68/0x640
 ufshcd_verify_dev_init+0x68/0x35c
 ufshcd_probe_hba+0x12c/0x1cb8
 ufshcd_host_reset_and_restore+0x88/0x254
 ufshcd_reset_and_restore+0xd0/0x354
 ufshcd_err_handler+0x408/0xc58
 process_one_work+0x24c/0x66c
 worker_thread+0x3e8/0xa4c
 kthread+0x150/0x1b4
 ret_from_fork+0x10/0x30

Fix this lockup by making ufshcd_exec_dev_cmd() allocate a reserved
request.

Signed-off-by: Bart Van Assche <bvanassche@acm.org>
---
 drivers/scsi/ufs/ufshcd.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c
index a241ef6bbc6f..03f4772fc2e2 100644
--- a/drivers/scsi/ufs/ufshcd.c
+++ b/drivers/scsi/ufs/ufshcd.c
@@ -128,8 +128,9 @@ EXPORT_SYMBOL_GPL(ufshcd_dump_regs);
 enum {
 	UFSHCD_MAX_CHANNEL	= 0,
 	UFSHCD_MAX_ID		= 1,
-	UFSHCD_CMD_PER_LUN	= 32,
-	UFSHCD_CAN_QUEUE	= 32,
+	UFSHCD_NUM_RESERVED	= 1,
+	UFSHCD_CMD_PER_LUN	= 32 - UFSHCD_NUM_RESERVED,
+	UFSHCD_CAN_QUEUE	= 32 - UFSHCD_NUM_RESERVED,
 };
 
 static const char *const ufshcd_state_name[] = {
@@ -2941,12 +2942,7 @@ static int ufshcd_exec_dev_cmd(struct ufs_hba *hba,
 
 	down_read(&hba->clk_scaling_lock);
 
-	/*
-	 * Get free slot, sleep if slots are unavailable.
-	 * Even though we use wait_event() which sleeps indefinitely,
-	 * the maximum wait time is bounded by SCSI request timeout.
-	 */
-	scmd = scsi_get_internal_cmd(q, DMA_TO_DEVICE, 0);
+	scmd = scsi_get_internal_cmd(q, DMA_TO_DEVICE, BLK_MQ_REQ_RESERVED);
 	if (IS_ERR(scmd)) {
 		err = PTR_ERR(scmd);
 		goto out_unlock;
@@ -8171,6 +8167,7 @@ static struct scsi_host_template ufshcd_driver_template = {
 	.sg_tablesize		= SG_ALL,
 	.cmd_per_lun		= UFSHCD_CMD_PER_LUN,
 	.can_queue		= UFSHCD_CAN_QUEUE,
+	.reserved_tags		= UFSHCD_NUM_RESERVED,
 	.max_segment_size	= PRDT_DATA_BYTE_COUNT_MAX,
 	.max_host_blocked	= 1,
 	.track_queue_depth	= 1,
@@ -9531,8 +9528,8 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq)
 	/* Configure LRB */
 	ufshcd_host_memory_configure(hba);
 
-	host->can_queue = hba->nutrs;
-	host->cmd_per_lun = hba->nutrs;
+	host->can_queue = hba->nutrs - UFSHCD_NUM_RESERVED;
+	host->cmd_per_lun = hba->nutrs - UFSHCD_NUM_RESERVED;
 	host->max_id = UFSHCD_MAX_ID;
 	host->max_lun = UFS_MAX_LUNS;
 	host->max_channel = UFSHCD_MAX_CHANNEL;

  parent reply	other threads:[~2021-11-19 19:58 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-11-19 19:57 [PATCH v2 00/20] UFS patches for kernel v5.17 Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 01/20] block: Add a flag for internal commands Bart Van Assche
2021-11-22  8:46   ` John Garry
2021-11-22 17:38     ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 02/20] scsi: core: Unexport scsi_track_queue_full() Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 03/20] scsi: core: Fix scsi_device_max_queue_depth() Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 04/20] scsi: core: Fix a race between scsi_done() and scsi_times_out() Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 05/20] scsi: core: Add support for internal commands Bart Van Assche
2021-11-22  8:58   ` John Garry
2021-11-22 17:46     ` Bart Van Assche
2021-11-22 18:08       ` John Garry
2021-11-22 19:04       ` Bart Van Assche
2021-11-23  8:13       ` Hannes Reinecke
2021-11-23 17:46         ` Bart Van Assche
2021-11-23 19:18           ` Bart Van Assche
2021-11-24  6:33             ` Hannes Reinecke
2021-11-19 19:57 ` [PATCH v2 06/20] scsi: core: Add support for reserved tags Bart Van Assche
2021-11-22  8:15   ` John Garry
2021-11-22 17:25     ` Bart Van Assche
2021-11-22 18:13       ` John Garry
2021-11-19 19:57 ` [PATCH v2 07/20] scsi: ufs: Rename a function argument Bart Van Assche
2021-11-22 20:25   ` Bean Huo
2021-11-19 19:57 ` [PATCH v2 08/20] scsi: ufs: Remove is_rpmb_wlun() Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 09/20] scsi: ufs: Remove the sdev_rpmb member Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 10/20] scsi: ufs: Remove dead code Bart Van Assche
2021-11-24 11:11   ` Adrian Hunter
2021-11-29 19:12     ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 11/20] scsi: ufs: Switch to scsi_(get|put)_internal_cmd() Bart Van Assche
2021-11-23 12:20   ` Bean Huo
2021-11-23 17:54     ` Bart Van Assche
2021-11-23 19:41     ` Bart Van Assche
2021-11-24 18:18       ` Bean Huo
2021-11-24 11:02   ` Adrian Hunter
2021-11-24 11:15     ` Adrian Hunter
2021-11-29 19:32     ` Bart Van Assche
2021-11-30  6:41       ` Adrian Hunter
2021-11-30 17:51         ` Bart Van Assche
2021-11-30 19:15           ` Adrian Hunter
2021-11-30 19:21             ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 12/20] scsi: ufs: Rework ufshcd_change_queue_depth() Bart Van Assche
2021-11-19 19:57 ` Bart Van Assche [this message]
2021-11-30  8:54   ` [PATCH v2 13/20] scsi: ufs: Fix a deadlock in the error handler Bean Huo
2021-11-30 17:52     ` Bart Van Assche
2021-11-30 19:32     ` Bart Van Assche
2021-12-01 13:44       ` Bean Huo
2021-12-01 18:31         ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 14/20] scsi: ufs: Introduce ufshcd_release_scsi_cmd() Bart Van Assche
2021-11-24 12:03   ` Adrian Hunter
2021-11-30 18:00     ` Bart Van Assche
2021-11-30 19:02       ` Adrian Hunter
2021-11-30 19:16         ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 15/20] scsi: ufs: Improve SCSI abort handling Bart Van Assche
2021-11-24 12:28   ` Adrian Hunter
2021-11-30  4:13     ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 16/20] scsi: ufs: Fix a kernel crash during shutdown Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 17/20] scsi: ufs: Stop using the clock scaling lock in the error handler Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 18/20] scsi: ufs: Optimize the command queueing code Bart Van Assche
2021-11-22 17:46   ` Asutosh Das (asd)
2021-11-22 18:13     ` Bart Van Assche
2021-11-22 23:02       ` Asutosh Das (asd)
2021-11-22 23:48         ` Bart Van Assche
2021-11-23 18:24           ` Asutosh Das (asd)
2021-12-01 18:33             ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 19/20] scsi: ufs: Implement polling support Bart Van Assche
2021-11-30  8:43   ` Bean Huo
2021-11-30  8:57     ` Avri Altman
2021-11-30  9:15       ` Bean Huo
2021-11-30 14:26     ` Bart Van Assche
2021-11-30 15:40       ` Bean Huo
2021-11-30 17:34         ` Bart Van Assche
2021-11-30 17:37     ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 20/20] scsi: ufs: Fix race conditions related to driver data Bart Van Assche

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211119195743.2817-14-bvanassche@acm.org \
    --to=bvanassche@acm.org \
    --cc=adrian.hunter@intel.com \
    --cc=asutoshd@codeaurora.org \
    --cc=avri.altman@wdc.com \
    --cc=beanhuo@micron.com \
    --cc=cang@codeaurora.org \
    --cc=jaegeuk@kernel.org \
    --cc=jejb@linux.ibm.com \
    --cc=linux-scsi@vger.kernel.org \
    --cc=martin.petersen@oracle.com \
    --cc=stanley.chu@mediatek.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox