[PATCH v2] scsi: libsas: fix HA resume deadlock and hisi_sas disk-wake race

[PATCH v2] scsi: libsas: fix HA resume deadlock and hisi_sas disk-wake race

[Xingui Yang]

Commit fbefe22811c3140 ("scsi: libsas: Don't always drain event workqueue
for HA resume") introduced sas_resume_ha_no_sync() to avoid a deadlock: the
PHYE_RESUME_TIMEOUT handler, running on the HA event workqueue, calls
sas_deform_port() -> sas_destruct_devices(), which removes SCSI devices and
waits for the host to become runtime-active. But the host cannot resume
until sas_resume_ha() -> sas_drain_work() returns, and the drain is blocked
on that very handler.

However skipping the drain reintroduces a race: hisi_sas returns from
resume before all PHY UP work and libsas discovery work finish. The
controller may then autosuspend while disks are still waking up. The disks
issue IO to a suspended controller, the IO fails, and the disks get
disabled.

Fix the deadlock at its source by moving the PHYE_RESUME_TIMEOUT
notification to after sas_drain_work(). By then the host resume is about to
complete, so device removal through device_link no longer blocks on the
resume and the cycle is broken.

With the deadlock gone, restore sas_resume_ha() (the draining variant) in
hisi_sas and remove sas_resume_ha_no_sync().

Fixes: fbefe22811c3140 ("scsi: libsas: Don't always drain event workqueue for HA resume")
Signed-off-by: Xingui Yang <yangxingui@huawei.com>
---
 drivers/scsi/hisi_sas/hisi_sas_v3_hw.c |  9 +-------
 drivers/scsi/libsas/sas_init.c         | 32 ++++++++++++++------------
 include/scsi/libsas.h                  |  1 -
 3 files changed, 18 insertions(+), 24 deletions(-)

diff --git a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
index 0687bdefcd63..c8673ae4e472 100644
--- a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
+++ b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
@@ -5263,14 +5263,7 @@ static int _resume_v3_hw(struct device *device)
 	}
 	phys_init_v3_hw(hisi_hba);
 
-	/*
-	 * If a directly-attached disk is removed during suspend, a deadlock
-	 * may occur, as the PHYE_RESUME_TIMEOUT processing will require the
-	 * hisi_hba->device to be active, which can only happen when resume
-	 * completes. So don't wait for the HA event workqueue to drain upon
-	 * resume.
-	 */
-	sas_resume_ha_no_sync(sha);
+	sas_resume_ha(sha);
 	clear_bit(HISI_SAS_RESETTING_BIT, &hisi_hba->flags);
 
 	dev_warn(dev, "end of resuming controller\n");
diff --git a/drivers/scsi/libsas/sas_init.c b/drivers/scsi/libsas/sas_init.c
index 0bec236f0fb5..624850f1483d 100644
--- a/drivers/scsi/libsas/sas_init.c
+++ b/drivers/scsi/libsas/sas_init.c
@@ -410,7 +410,7 @@ static void sas_resume_insert_broadcast_ha(struct sas_ha_struct *ha)
 	}
 }
 
-static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain)
+static void _sas_resume_ha(struct sas_ha_struct *ha)
 {
 	const unsigned long tmo = msecs_to_jiffies(25000);
 	int i;
@@ -426,6 +426,21 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain)
 		dev_info(ha->dev, "waiting up to 25 seconds for %d phy%s to resume\n",
 			 i, i > 1 ? "s" : "");
 	wait_event_timeout(ha->eh_wait_q, phys_suspended(ha) == 0, tmo);
+
+	/* all phys are back up or timed out, turn on i/o so we can
+	 * flush out disks that did not return
+	 */
+	scsi_unblock_requests(ha->shost);
+	sas_drain_work(ha);
+
+	/* Send PHYE_RESUME_TIMEOUT after sas_drain_work(). The handler
+	 * calls sas_deform_port() -> sas_destruct_devices(), which removes
+	 * SCSI devices and, for LLDDs using device_link() PM sync, waits
+	 * for the host to be runtime-active. Sending it before the drain
+	 * would deadlock: the drain waits for the handler, the handler
+	 * waits for host resume, and host resume waits for the drain to
+	 * finish.
+	 */
 	for (i = 0; i < ha->num_phys; i++) {
 		struct asd_sas_phy *phy = ha->sas_phy[i];
 
@@ -436,12 +451,6 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain)
 		}
 	}
 
-	/* all phys are back up or timed out, turn on i/o so we can
-	 * flush out disks that did not return
-	 */
-	scsi_unblock_requests(ha->shost);
-	if (drain)
-		sas_drain_work(ha);
 	clear_bit(SAS_HA_RESUMING, &ha->state);
 
 	sas_queue_deferred_work(ha);
@@ -453,17 +462,10 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain)
 
 void sas_resume_ha(struct sas_ha_struct *ha)
 {
-	_sas_resume_ha(ha, true);
+	_sas_resume_ha(ha);
 }
 EXPORT_SYMBOL(sas_resume_ha);
 
-/* A no-sync variant, which does not call sas_drain_ha(). */
-void sas_resume_ha_no_sync(struct sas_ha_struct *ha)
-{
-	_sas_resume_ha(ha, false);
-}
-EXPORT_SYMBOL(sas_resume_ha_no_sync);
-
 void sas_suspend_ha(struct sas_ha_struct *ha)
 {
 	int i;
diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h
index 163f23c92b41..36d4cb567837 100644
--- a/include/scsi/libsas.h
+++ b/include/scsi/libsas.h
@@ -680,7 +680,6 @@ extern int sas_register_ha(struct sas_ha_struct *);
 extern int sas_unregister_ha(struct sas_ha_struct *);
 extern void sas_prep_resume_ha(struct sas_ha_struct *sas_ha);
 extern void sas_resume_ha(struct sas_ha_struct *sas_ha);
-extern void sas_resume_ha_no_sync(struct sas_ha_struct *sas_ha);
 extern void sas_suspend_ha(struct sas_ha_struct *sas_ha);
 
 int sas_phy_reset(struct sas_phy *phy, int hard_reset);
-- 
2.43.0

Re: [PATCH v2] scsi: libsas: fix HA resume deadlock and hisi_sas disk-wake race

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Hardware initialization race: scsi_unblock_requests() allows I/O dispatch to uninitialized hardware in _resume_v3_hw().
--

commit 9e820865310a614d1e3c143d85daf43eae7329e2
Author: Xingui Yang <yangxingui@huawei.com>

scsi: libsas: fix HA resume deadlock and hisi_sas disk-wake race

This commit fixes a resume deadlock by moving the PHYE_RESUME_TIMEOUT
notification to occur after sas_drain_work(). It restores the standard
sas_resume_ha() behavior in hisi_sas and removes the no-sync variant.

> diff --git a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
> --- a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
> +++ b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
> @@ -5262,14 +5262,7 @@ static int _resume_v3_hw(struct device *device)
>  
> [ ... ]
>  
>  	scsi_unblock_requests(shost);
>  	clear_bit(HISI_SAS_REJECT_CMD_BIT, &hisi_hba->flags);
>  
>  	sas_prep_resume_ha(sha);
>  	rc = hw_init_v3_hw(hisi_hba);

[Severity: High]
This is a pre-existing issue, but I noticed a potential hardware
initialization race here.

Does unblocking requests before hw_init_v3_hw() completes allow pending I/O
to be dispatched to uninitialized hardware?

During PM resume, _resume_v3_hw() calls scsi_unblock_requests() and clears
HISI_SAS_REJECT_CMD_BIT. Any blocked block-layer requests could immediately
be dispatched to hisi_sas_queue_command().

These requests would be written to the controller hardware before it has
been reset or initialized by hw_init_v3_hw().

Could this lead to hardware corruption, SErrors, or system lockups on
standard suspend/resume cycles if there is pending I/O?

>  	if (rc) {
>  		scsi_remove_host(shost);
>  		return rc;
>  	}
>  	phys_init_v3_hw(hisi_hba);
>  
> -	/*
> -	 * If a directly-attached disk is removed during suspend, a deadlock
> -	 * may occur, as the PHYE_RESUME_TIMEOUT processing will require the
> -	 * hisi_hba->device to be active, which can only happen when resume
> -	 * completes. So don't wait for the HA event workqueue to drain upon
> -	 * resume.
> -	 */
> -	sas_resume_ha_no_sync(sha);
> +	sas_resume_ha(sha);
>  	clear_bit(HISI_SAS_RESETTING_BIT, &hisi_hba->flags);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702033211.1743313-1-yangxingui@huawei.com?part=1

Re: [PATCH v2] scsi: libsas: fix HA resume deadlock and hisi_sas disk-wake race

[yangxingui]

Kindly ping for review...

On 2026/7/2 11:32, Xingui Yang wrote:
> Commit fbefe22811c3140 ("scsi: libsas: Don't always drain event workqueue
> for HA resume") introduced sas_resume_ha_no_sync() to avoid a deadlock: the
> PHYE_RESUME_TIMEOUT handler, running on the HA event workqueue, calls
> sas_deform_port() -> sas_destruct_devices(), which removes SCSI devices and
> waits for the host to become runtime-active. But the host cannot resume
> until sas_resume_ha() -> sas_drain_work() returns, and the drain is blocked
> on that very handler.
> 
> However skipping the drain reintroduces a race: hisi_sas returns from
> resume before all PHY UP work and libsas discovery work finish. The
> controller may then autosuspend while disks are still waking up. The disks
> issue IO to a suspended controller, the IO fails, and the disks get
> disabled.
> 
> Fix the deadlock at its source by moving the PHYE_RESUME_TIMEOUT
> notification to after sas_drain_work(). By then the host resume is about to
> complete, so device removal through device_link no longer blocks on the
> resume and the cycle is broken.
> 
> With the deadlock gone, restore sas_resume_ha() (the draining variant) in
> hisi_sas and remove sas_resume_ha_no_sync().
> 
> Fixes: fbefe22811c3140 ("scsi: libsas: Don't always drain event workqueue for HA resume")
> Signed-off-by: Xingui Yang <yangxingui@huawei.com>
> ---
>   drivers/scsi/hisi_sas/hisi_sas_v3_hw.c |  9 +-------
>   drivers/scsi/libsas/sas_init.c         | 32 ++++++++++++++------------
>   include/scsi/libsas.h                  |  1 -
>   3 files changed, 18 insertions(+), 24 deletions(-)
> 
> diff --git a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
> index 0687bdefcd63..c8673ae4e472 100644
> --- a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
> +++ b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
> @@ -5263,14 +5263,7 @@ static int _resume_v3_hw(struct device *device)
>   	}
>   	phys_init_v3_hw(hisi_hba);
>   
> -	/*
> -	 * If a directly-attached disk is removed during suspend, a deadlock
> -	 * may occur, as the PHYE_RESUME_TIMEOUT processing will require the
> -	 * hisi_hba->device to be active, which can only happen when resume
> -	 * completes. So don't wait for the HA event workqueue to drain upon
> -	 * resume.
> -	 */
> -	sas_resume_ha_no_sync(sha);
> +	sas_resume_ha(sha);
>   	clear_bit(HISI_SAS_RESETTING_BIT, &hisi_hba->flags);
>   
>   	dev_warn(dev, "end of resuming controller\n");
> diff --git a/drivers/scsi/libsas/sas_init.c b/drivers/scsi/libsas/sas_init.c
> index 0bec236f0fb5..624850f1483d 100644
> --- a/drivers/scsi/libsas/sas_init.c
> +++ b/drivers/scsi/libsas/sas_init.c
> @@ -410,7 +410,7 @@ static void sas_resume_insert_broadcast_ha(struct sas_ha_struct *ha)
>   	}
>   }
>   
> -static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain)
> +static void _sas_resume_ha(struct sas_ha_struct *ha)
>   {
>   	const unsigned long tmo = msecs_to_jiffies(25000);
>   	int i;
> @@ -426,6 +426,21 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain)
>   		dev_info(ha->dev, "waiting up to 25 seconds for %d phy%s to resume\n",
>   			 i, i > 1 ? "s" : "");
>   	wait_event_timeout(ha->eh_wait_q, phys_suspended(ha) == 0, tmo);
> +
> +	/* all phys are back up or timed out, turn on i/o so we can
> +	 * flush out disks that did not return
> +	 */
> +	scsi_unblock_requests(ha->shost);
> +	sas_drain_work(ha);
> +
> +	/* Send PHYE_RESUME_TIMEOUT after sas_drain_work(). The handler
> +	 * calls sas_deform_port() -> sas_destruct_devices(), which removes
> +	 * SCSI devices and, for LLDDs using device_link() PM sync, waits
> +	 * for the host to be runtime-active. Sending it before the drain
> +	 * would deadlock: the drain waits for the handler, the handler
> +	 * waits for host resume, and host resume waits for the drain to
> +	 * finish.
> +	 */
>   	for (i = 0; i < ha->num_phys; i++) {
>   		struct asd_sas_phy *phy = ha->sas_phy[i];
>   
> @@ -436,12 +451,6 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain)
>   		}
>   	}
>   
> -	/* all phys are back up or timed out, turn on i/o so we can
> -	 * flush out disks that did not return
> -	 */
> -	scsi_unblock_requests(ha->shost);
> -	if (drain)
> -		sas_drain_work(ha);
>   	clear_bit(SAS_HA_RESUMING, &ha->state);
>   
>   	sas_queue_deferred_work(ha);
> @@ -453,17 +462,10 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain)
>   
>   void sas_resume_ha(struct sas_ha_struct *ha)
>   {
> -	_sas_resume_ha(ha, true);
> +	_sas_resume_ha(ha);
>   }
>   EXPORT_SYMBOL(sas_resume_ha);
>   
> -/* A no-sync variant, which does not call sas_drain_ha(). */
> -void sas_resume_ha_no_sync(struct sas_ha_struct *ha)
> -{
> -	_sas_resume_ha(ha, false);
> -}
> -EXPORT_SYMBOL(sas_resume_ha_no_sync);
> -
>   void sas_suspend_ha(struct sas_ha_struct *ha)
>   {
>   	int i;
> diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h
> index 163f23c92b41..36d4cb567837 100644
> --- a/include/scsi/libsas.h
> +++ b/include/scsi/libsas.h
> @@ -680,7 +680,6 @@ extern int sas_register_ha(struct sas_ha_struct *);
>   extern int sas_unregister_ha(struct sas_ha_struct *);
>   extern void sas_prep_resume_ha(struct sas_ha_struct *sas_ha);
>   extern void sas_resume_ha(struct sas_ha_struct *sas_ha);
> -extern void sas_resume_ha_no_sync(struct sas_ha_struct *sas_ha);
>   extern void sas_suspend_ha(struct sas_ha_struct *sas_ha);
>   
>   int sas_phy_reset(struct sas_phy *phy, int hard_reset);
> 

Re: [PATCH v2] scsi: libsas: fix HA resume deadlock and hisi_sas disk-wake race

[John Garry]

On 13/07/2026 04:10, yangxingui wrote:
> Kindly ping for review...
> 
> On 2026/7/2 11:32, Xingui Yang wrote:
>> Commit fbefe22811c3140 ("scsi: libsas: Don't always drain event workqueue
>> for HA resume") introduced sas_resume_ha_no_sync() to avoid a 
>> deadlock: the
>> PHYE_RESUME_TIMEOUT handler, running on the HA event workqueue, calls
>> sas_deform_port() -> sas_destruct_devices(), which removes SCSI 
>> devices and
>> waits for the host to become runtime-active. But the host cannot resume
>> until sas_resume_ha() -> sas_drain_work() returns, and the drain is 
>> blocked
>> on that very handler.
>>
>> However skipping the drain reintroduces a race: hisi_sas returns from
>> resume before all PHY UP work and libsas discovery work finish. The
>> controller may then autosuspend while disks are still waking up. The 
>> disks
>> issue IO to a suspended controller, the IO fails, and the disks get
>> disabled.
>>
>> Fix the deadlock at its source by moving the PHYE_RESUME_TIMEOUT
>> notification to after sas_drain_work(). By then the host resume is 
>> about to
>> complete, so device removal through device_link no longer blocks on the
>> resume and the cycle is broken.
>>
>> With the deadlock gone, restore sas_resume_ha() (the draining variant) in
>> hisi_sas and remove sas_resume_ha_no_sync().
>>
>> Fixes: fbefe22811c3140 ("scsi: libsas: Don't always drain event 
>> workqueue for HA resume")
>> Signed-off-by: Xingui Yang <yangxingui@huawei.com>

Any idea why this problem has only been discovered after 5 years (from 
fbefe22811c3140 being merged)?

Is there some new test case?

>> ---
>>   drivers/scsi/hisi_sas/hisi_sas_v3_hw.c |  9 +-------
>>   drivers/scsi/libsas/sas_init.c         | 32 ++++++++++++++------------
>>   include/scsi/libsas.h                  |  1 -
>>   3 files changed, 18 insertions(+), 24 deletions(-)
>>
>> diff --git a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c b/drivers/scsi/ 
>> hisi_sas/hisi_sas_v3_hw.c
>> index 0687bdefcd63..c8673ae4e472 100644
>> --- a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
>> +++ b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
>> @@ -5263,14 +5263,7 @@ static int _resume_v3_hw(struct device *device)
>>       }
>>       phys_init_v3_hw(hisi_hba);
>> -    /*
>> -     * If a directly-attached disk is removed during suspend, a deadlock
>> -     * may occur, as the PHYE_RESUME_TIMEOUT processing will require the
>> -     * hisi_hba->device to be active, which can only happen when resume
>> -     * completes. So don't wait for the HA event workqueue to drain upon
>> -     * resume.
>> -     */
>> -    sas_resume_ha_no_sync(sha);
>> +    sas_resume_ha(sha);
>>       clear_bit(HISI_SAS_RESETTING_BIT, &hisi_hba->flags);
>>       dev_warn(dev, "end of resuming controller\n");
>> diff --git a/drivers/scsi/libsas/sas_init.c b/drivers/scsi/libsas/ 
>> sas_init.c
>> index 0bec236f0fb5..624850f1483d 100644
>> --- a/drivers/scsi/libsas/sas_init.c
>> +++ b/drivers/scsi/libsas/sas_init.c
>> @@ -410,7 +410,7 @@ static void sas_resume_insert_broadcast_ha(struct 
>> sas_ha_struct *ha)
>>       }
>>   }
>> -static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain)
>> +static void _sas_resume_ha(struct sas_ha_struct *ha)
>>   {
>>       const unsigned long tmo = msecs_to_jiffies(25000);
>>       int i;
>> @@ -426,6 +426,21 @@ static void _sas_resume_ha(struct sas_ha_struct 
>> *ha, bool drain)
>>           dev_info(ha->dev, "waiting up to 25 seconds for %d phy%s to 
>> resume\n",
>>                i, i > 1 ? "s" : "");
>>       wait_event_timeout(ha->eh_wait_q, phys_suspended(ha) == 0, tmo);
>> +
>> +    /* all phys are back up or timed out, turn on i/o so we can
>> +     * flush out disks that did not return
>> +     */
>> +    scsi_unblock_requests(ha->shost);
>> +    sas_drain_work(ha);
>> +
>> +    /* Send PHYE_RESUME_TIMEOUT after sas_drain_work(). The handler
>> +     * calls sas_deform_port() -> sas_destruct_devices(), which removes
>> +     * SCSI devices and, for LLDDs using device_link() PM sync, waits
>> +     * for the host to be runtime-active. Sending it before the drain
>> +     * would deadlock: the drain waits for the handler, the handler
>> +     * waits for host resume, and host resume waits for the drain to
>> +     * finish.
>> +     */
>>       for (i = 0; i < ha->num_phys; i++) {
>>           struct asd_sas_phy *phy = ha->sas_phy[i];
>> @@ -436,12 +451,6 @@ static void _sas_resume_ha(struct sas_ha_struct 
>> *ha, bool drain)
>>           }
>>       }
>> -    /* all phys are back up or timed out, turn on i/o so we can
>> -     * flush out disks that did not return
>> -     */
>> -    scsi_unblock_requests(ha->shost);
>> -    if (drain)
>> -        sas_drain_work(ha);
>>       clear_bit(SAS_HA_RESUMING, &ha->state);
>>       sas_queue_deferred_work(ha);
>> @@ -453,17 +462,10 @@ static void _sas_resume_ha(struct sas_ha_struct 
>> *ha, bool drain)
>>   void sas_resume_ha(struct sas_ha_struct *ha)
>>   {
>> -    _sas_resume_ha(ha, true);
>> +    _sas_resume_ha(ha);
>>   }
>>   EXPORT_SYMBOL(sas_resume_ha);
>> -/* A no-sync variant, which does not call sas_drain_ha(). */
>> -void sas_resume_ha_no_sync(struct sas_ha_struct *ha)
>> -{
>> -    _sas_resume_ha(ha, false);
>> -}
>> -EXPORT_SYMBOL(sas_resume_ha_no_sync);
>> -
>>   void sas_suspend_ha(struct sas_ha_struct *ha)
>>   {
>>       int i;
>> diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h
>> index 163f23c92b41..36d4cb567837 100644
>> --- a/include/scsi/libsas.h
>> +++ b/include/scsi/libsas.h
>> @@ -680,7 +680,6 @@ extern int sas_register_ha(struct sas_ha_struct *);
>>   extern int sas_unregister_ha(struct sas_ha_struct *);
>>   extern void sas_prep_resume_ha(struct sas_ha_struct *sas_ha);
>>   extern void sas_resume_ha(struct sas_ha_struct *sas_ha);
>> -extern void sas_resume_ha_no_sync(struct sas_ha_struct *sas_ha);
>>   extern void sas_suspend_ha(struct sas_ha_struct *sas_ha);
>>   int sas_phy_reset(struct sas_phy *phy, int hard_reset);
>>
> 

Re: [PATCH v2] scsi: libsas: fix HA resume deadlock and hisi_sas disk-wake race

[yangxingui]

Hi, John

On 2026/7/13 19:26, John Garry wrote:
> On 13/07/2026 04:10, yangxingui wrote:
>> Kindly ping for review...
>>
>> On 2026/7/2 11:32, Xingui Yang wrote:
>>> Commit fbefe22811c3140 ("scsi: libsas: Don't always drain event 
>>> workqueue
>>> for HA resume") introduced sas_resume_ha_no_sync() to avoid a 
>>> deadlock: the
>>> PHYE_RESUME_TIMEOUT handler, running on the HA event workqueue, calls
>>> sas_deform_port() -> sas_destruct_devices(), which removes SCSI 
>>> devices and
>>> waits for the host to become runtime-active. But the host cannot resume
>>> until sas_resume_ha() -> sas_drain_work() returns, and the drain is 
>>> blocked
>>> on that very handler.
>>>
>>> However skipping the drain reintroduces a race: hisi_sas returns from
>>> resume before all PHY UP work and libsas discovery work finish. The
>>> controller may then autosuspend while disks are still waking up. The 
>>> disks
>>> issue IO to a suspended controller, the IO fails, and the disks get
>>> disabled.
>>>
>>> Fix the deadlock at its source by moving the PHYE_RESUME_TIMEOUT
>>> notification to after sas_drain_work(). By then the host resume is 
>>> about to
>>> complete, so device removal through device_link no longer blocks on the
>>> resume and the cycle is broken.
>>>
>>> With the deadlock gone, restore sas_resume_ha() (the draining 
>>> variant) in
>>> hisi_sas and remove sas_resume_ha_no_sync().
>>>
>>> Fixes: fbefe22811c3140 ("scsi: libsas: Don't always drain event 
>>> workqueue for HA resume")
>>> Signed-off-by: Xingui Yang <yangxingui@huawei.com>
> 
> Any idea why this problem has only been discovered after 5 years (from 
> fbefe22811c3140 being merged)?
> 
> Is there some new test case?

Yes, the test environment is connected to more SATA disks. During the 
wake-up process, the controller wakes up first, and then the disks wake 
up one by one. Since the number of SATA disks has increased, and the 
SATA disk enter the EH (Error Handling) process when waking up, the 
wake-up time has become longer than before. It is possible that the 
controller has already entered the next suspend state, while many disks 
are still in the previous wake-up process, leading to a failure in disk 
wake-up and the disks being disabled.

However, we have tested some different kernel versions in between and 
found that the following patch significantly improves this issue, but it 
does not completely resolve it.

bede543d2f8a: ACPI: OSL: Use usleep_range() in acpi_os_sleep()

In addition, if some disks wake up a bit slowly, it may also lead to 
ssubsequent disk wake-up failure.
[  286.932540] hisi_sas_v3_hw 0000:32:04.0: resuming from operating 
state [D0]
[  287.732627] hisi_sas_v3_hw 0000:32:04.0: end of resuming controller
[  293.167893] hisi_sas_v3_hw 0000:32:04.0: entering suspend state
[  341.760789] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[  341.761006] sas: Executing internal abort failed 5000000000000307 (-22)
[  341.761021] hisi_sas_v3_hw 0000:32:04.0: I_T nexus reset: internal 
abort (-22)
[  341.761028] sas: ata4: end_device-3:3: Unable to reset ata device?
[  341.916134] sas: lldd_execute_task returned: -22
[  341.916161] ata4.00: failed to IDENTIFY (I/O error, err_mask=0x40)
[  341.916165] ata4.00: revalidation failed (errno=-5)
[  346.980523] sas: Executing internal abort failed 5000000000000307 (-22)
[  346.980542] hisi_sas_v3_hw 0000:32:04.0: I_T nexus reset: internal 
abort (-22)
[  346.980552] sas: ata4: end_device-3:3: Unable to reset ata device?
[  347.136137] sas: lldd_execute_task returned: -22
[  347.136163] ata4.00: failed to IDENTIFY (I/O error, err_mask=0x40)
[  347.136169] ata4.00: revalidation failed (errno=-5)
[  352.356924] sas: Executing internal abort failed 5000000000000307 (-22)
[  352.357001] hisi_sas_v3_hw 0000:32:04.0: I_T nexus reset: internal 
abort (-22)
[  352.357011] sas: ata4: end_device-3:3: Unable to reset ata device?
[  352.512174] sas: lldd_execute_task returned: -22
[  352.512255] ata4.00: failed to IDENTIFY (I/O error, err_mask=0x40)
[  352.512263] ata4.00: revalidation failed (errno=-5)
[  352.512269] ata4.00: disable device
[  352.512324] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 
tries: 1
[  352.512408] sas: sas_resume_sata: for direct-attached device 
5000000000000307 returned -19


Thanks,
Xingui
.

Re: [PATCH v2] scsi: libsas: fix HA resume deadlock and hisi_sas disk-wake race

[John Garry]

On 14/07/2026 03:15, yangxingui wrote:
>>>> Fixes: fbefe22811c3140 ("scsi: libsas: Don't always drain event 
>>>> workqueue for HA resume")
>>>> Signed-off-by: Xingui Yang <yangxingui@huawei.com>
>> 
>> Any idea why this problem has only been discovered after 5 years (from 
>> fbefe22811c3140 being merged)?
>> 
>> Is there some new test case?

ok, I can check below. But another question is if this applies to only 
hisi_sas driver? I mean, you are changing libsas and the hisi_sas, but 
not other libsas-based drivers - why? They support suspend/resume.

> 
> Yes, the test environment is connected to more SATA disks. During the
> wake-up process, the controller wakes up first, and then the disks wake
> up one by one. Since the number of SATA disks has increased, and the
> SATA disk enter the EH (Error Handling) process when waking up, the
> wake-up time has become longer than before. It is possible that the
> controller has already entered the next suspend state, while many disks
> are still in the previous wake-up process, leading to a failure in disk
> wake-up and the disks being disabled.
> 
> However, we have tested some different kernel versions in between and
> found that the following patch significantly improves this issue, but it
> does not completely resolve it.
> 
> bede543d2f8a: ACPI: OSL: Use usleep_range() in acpi_os_sleep()
> 
> In addition, if some disks wake up a bit slowly, it may also lead to
> ssubsequent disk wake-up failure.
> [  286.932540] hisi_sas_v3_hw 0000:32:04.0: resuming from operating
> state [D0]
> [  287.732627] hisi_sas_v3_hw 0000:32:04.0: end of resuming controller
> [  293.167893] hisi_sas_v3_hw 0000:32:04.0: entering suspend state
> [  341.760789] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
> [  341.761006] sas: Executing internal abort failed 5000000000000307 (-22)
> [  341.761021] hisi_sas_v3_hw 0000:32:04.0: I_T nexus reset: internal
> abort (-22)
> [  341.761028] sas: ata4: end_device-3:3: Unable to reset ata device?
> [  341.916134] sas: lldd_execute_task returned: -22
> [  341.916161] ata4.00: failed to IDENTIFY (I/O error, err_mask=0x40)
> [  341.916165] ata4.00: revalidation failed (errno=-5)
> [  346.980523] sas: Executing internal abort failed 5000000000000307 (-22)
> [  346.980542] hisi_sas_v3_hw 0000:32:04.0: I_T nexus reset: internal
> abort (-22)
> [  346.980552] sas: ata4: end_device-3:3: Unable to reset ata device?
> [  347.136137] sas: lldd_execute_task returned: -22
> [  347.136163] ata4.00: failed to IDENTIFY (I/O error, err_mask=0x40)
> [  347.136169] ata4.00: revalidation failed (errno=-5)
> [  352.356924] sas: Executing internal abort failed 5000000000000307 (-22)
> [  352.357001] hisi_sas_v3_hw 0000:32:04.0: I_T nexus reset: internal
> abort (-22)
> [  352.357011] sas: ata4: end_device-3:3: Unable to reset ata device?
> [  352.512174] sas: lldd_execute_task returned: -22
> [  352.512255] ata4.00: failed to IDENTIFY (I/O error, err_mask=0x40)
> [  352.512263] ata4.00: revalidation failed (errno=-5)
> [  352.512269] ata4.00: disable device
> [  352.512324] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0
> tries: 1
> [  352.512408] sas: sas_resume_sata: for direct-attached device
> 5000000000000307 returned -19

Re: [PATCH v2] scsi: libsas: fix HA resume deadlock and hisi_sas disk-wake race

[yangxingui]

Hi, John

On 2026/7/14 16:26, John Garry wrote:
> On 14/07/2026 03:15, yangxingui wrote:
>>>>> Fixes: fbefe22811c3140 ("scsi: libsas: Don't always drain event 
>>>>> workqueue for HA resume")
>>>>> Signed-off-by: Xingui Yang <yangxingui@huawei.com>
>>>
>>> Any idea why this problem has only been discovered after 5 years 
>>> (from fbefe22811c3140 being merged)?
>>>
>>> Is there some new test case?
> 
> ok, I can check below. But another question is if this applies to only 
> hisi_sas driver? I mean, you are changing libsas and the hisi_sas, but 
> not other libsas-based drivers - why? They support suspend/resume.

There are five libsas consumers (callers of sas_register_ha): hisi_sas,
isci, pm8001, aic94xx, and mvsas. Only hisi_sas needs a driver-side 
change, because it is the only driver that satisfies all three
conditions below simultaneously.

1. Only hisi_sas used sas_resume_ha_no_sync().
isci/init.c:               sas_resume_ha(&ihost->sas_ha);
pm8001/pm8001_init.c:      sas_resume_ha(sha);
hisi_sas/hisi_sas_v3_hw.c: sas_resume_ha_no_sync(sha);  ← only caller
isci and pm8001 have always used the draining variant. aic94xx and
mvsas do not register any PM ops — their pci_driver structs have no
.driver.pm field — so they never enter the resume path.

2. Only hisi_sas establishes a device_link between SCSI devices and
the HBA. Commit 16fd4a7c5917 added device_link_add() in
hisi_sas_v3_hw.c to keep the HBA active while associated SCSI devices
are active. A side effect of this device_link is that
scsi_remove_device() in the PHYE_RESUME_TIMEOUT handler waits for the
HBA to be runtime-active — the direct cause of the original deadlock.
None of the other four drivers uses device_link_add() for PM sync, so
their sas_destruct_devices() path never blocks on host resume.
Original deadlock as follow:
[ 1335.875252] Call trace:
[ 1335.878380]  __switch_to+0x150/0x238
[ 1335.882646]  __schedule+0x388/0x880
[ 1335.886825]  schedule+0x58/0x130
[ 1335.890744]  rpm_resume+0x1e4/0x940
[ 1335.894926]  __pm_runtime_resume+0x68/0x130
[ 1335.899805]  rpm_get_suppliers+0x4c/0x158
[ 1335.904506]  __rpm_callback+0x190/0x248
[ 1335.909034]  rpm_callback+0x40/0x88
[ 1335.913215]  rpm_resume+0x6a8/0x940
[ 1335.917396]  __pm_runtime_resume+0x68/0x130
[ 1335.922271]  device_release_driver_internal+0xd4/0x238
[ 1335.928104]  device_release_driver+0x20/0x38
[ 1335.933065]  bus_remove_device+0xd8/0x158
[ 1335.937768]  device_del+0x168/0x3e0
[ 1335.941944]  __scsi_remove_device+0x120/0x198
[ 1335.946988]  __scsi_remove_target+0xe8/0x1c0
[ 1335.951943]  scsi_remove_target+0x120/0x228
[ 1335.956812]  sas_rphy_remove+0x8c/0x98 [scsi_transport_sas]
[ 1335.963075]  sas_rphy_delete+0x20/0x40 [scsi_transport_sas]
[ 1335.969336]  sas_destruct_devices+0x74/0xb8 [libsas]
[ 1335.974992]  sas_deform_port+0x218/0x270 [libsas]
[ 1335.980387]  sas_phye_resume_timeout+0x2c/0x60 [libsas]
[ 1335.986303]  sas_phy_event_worker+0x38/0x68 [libsas]
[ 1335.991961]  process_one_work+0x174/0x3e0
[ 1335.996656]  worker_thread+0x22c/0x3b0
[ 1336.001091]  kthread+0xec/0x100
[ 1336.004919]  ret_from_fork+0x10/0x20

3. Only hisi_sas supports runtime PM (autosuspend). This is
consistent with your own statement in fbefe22811c3140: "Other drivers
which use libsas don't worry about this as none support runtime
suspend."

Without runtime PM, the controller never autosuspends during disk
wake-up, so the race this patch addresses cannot occur.

Thanks,
Xingui

Re: [PATCH v2] scsi: libsas: fix HA resume deadlock and hisi_sas disk-wake race

[John Garry]

On 02/07/2026 04:32, Xingui Yang wrote:
> Commit fbefe22811c3140 ("scsi: libsas: Don't always drain event workqueue
> for HA resume") introduced sas_resume_ha_no_sync() to avoid a deadlock: the
> PHYE_RESUME_TIMEOUT handler, running on the HA event workqueue, calls
> sas_deform_port() -> sas_destruct_devices(), which removes SCSI devices and
> waits for the host to become runtime-active. But the host cannot resume
> until sas_resume_ha() -> sas_drain_work() returns, and the drain is blocked
> on that very handler.
> 
> However skipping the drain reintroduces a race: hisi_sas returns from
> resume before all PHY UP work and libsas discovery work finish. The
> controller may then autosuspend while disks are still waking up. The disks
> issue IO to a suspended controller, the IO fails, and the disks get
> disabled.
> 
> Fix the deadlock at its source by moving the PHYE_RESUME_TIMEOUT
> notification to after sas_drain_work(). By then the host resume is about to
> complete, so device removal through device_link no longer blocks on the
> resume and the cycle is broken.
> 
> With the deadlock gone, restore sas_resume_ha() (the draining variant) in
> hisi_sas and remove sas_resume_ha_no_sync().
> 
> Fixes: fbefe22811c3140 ("scsi: libsas: Don't always drain event workqueue for HA resume")
> Signed-off-by: Xingui Yang <yangxingui@huawei.com>
> ---
>   drivers/scsi/hisi_sas/hisi_sas_v3_hw.c |  9 +-------
>   drivers/scsi/libsas/sas_init.c         | 32 ++++++++++++++------------
>   include/scsi/libsas.h                  |  1 -
>   3 files changed, 18 insertions(+), 24 deletions(-)
> 
> diff --git a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
> index 0687bdefcd63..c8673ae4e472 100644
> --- a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
> +++ b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
> @@ -5263,14 +5263,7 @@ static int _resume_v3_hw(struct device *device)
>   	}
>   	phys_init_v3_hw(hisi_hba);
>   
> -	/*
> -	 * If a directly-attached disk is removed during suspend, a deadlock
> -	 * may occur, as the PHYE_RESUME_TIMEOUT processing will require the
> -	 * hisi_hba->device to be active, which can only happen when resume
> -	 * completes. So don't wait for the HA event workqueue to drain upon
> -	 * resume.
> -	 */
> -	sas_resume_ha_no_sync(sha);
> +	sas_resume_ha(sha);
>   	clear_bit(HISI_SAS_RESETTING_BIT, &hisi_hba->flags);
>   
>   	dev_warn(dev, "end of resuming controller\n");
> diff --git a/drivers/scsi/libsas/sas_init.c b/drivers/scsi/libsas/sas_init.c
> index 0bec236f0fb5..624850f1483d 100644
> --- a/drivers/scsi/libsas/sas_init.c
> +++ b/drivers/scsi/libsas/sas_init.c
> @@ -410,7 +410,7 @@ static void sas_resume_insert_broadcast_ha(struct sas_ha_struct *ha)
>   	}
>   }
>   
> -static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain)
> +static void _sas_resume_ha(struct sas_ha_struct *ha)
>   {
>   	const unsigned long tmo = msecs_to_jiffies(25000);
>   	int i;
> @@ -426,6 +426,21 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain)
>   		dev_info(ha->dev, "waiting up to 25 seconds for %d phy%s to resume\n",
>   			 i, i > 1 ? "s" : "");
>   	wait_event_timeout(ha->eh_wait_q, phys_suspended(ha) == 0, tmo);
> +
> +	/* all phys are back up or timed out, turn on i/o so we can

That is obviously not the standard formatting for comments. And 
sentences begin with an upper case letter.

> +	 * flush out disks that did not return

. to end a sentence.

Check punctuation in future.

> +	 */
> +	scsi_unblock_requests(ha->shost);
> +	sas_drain_work(ha);
> +
> +	/* Send PHYE_RESUME_TIMEOUT after sas_drain_work(). The handler
> +	 * calls sas_deform_port() -> sas_destruct_devices(), which removes
> +	 * SCSI devices and, for LLDDs using device_link() PM sync, waits
> +	 * for the host to be runtime-active. Sending it before the drain
> +	 * would deadlock: the drain waits for the handler, the handler
> +	 * waits for host resume, and host resume waits for the drain to
> +	 * finish.
> +	 */
>   	for (i = 0; i < ha->num_phys; i++) {
>   		struct asd_sas_phy *phy = ha->sas_phy[i];
>   
> @@ -436,12 +451,6 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain)
>   		}
>   	}
>   
> -	/* all phys are back up or timed out, turn on i/o so we can
> -	 * flush out disks that did not return
> -	 */
> -	scsi_unblock_requests(ha->shost);
> -	if (drain)
> -		sas_drain_work(ha);
>   	clear_bit(SAS_HA_RESUMING, &ha->state);
>   
>   	sas_queue_deferred_work(ha);
> @@ -453,17 +462,10 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain)
>   
>   void sas_resume_ha(struct sas_ha_struct *ha)

what is the purpose of this wrapper now?

>   {
> -	_sas_resume_ha(ha, true);
> +	_sas_resume_ha(ha);
>   }
>   EXPORT_SYMBOL(sas_resume_ha);
>   
> -/* A no-sync variant, which does not call sas_drain_ha(). */
> -void sas_resume_ha_no_sync(struct sas_ha_struct *ha)
> -{
> -	_sas_resume_ha(ha, false);
> -}
> -EXPORT_SYMBOL(sas_resume_ha_no_sync);
> -
>   void sas_suspend_ha(struct sas_ha_struct *ha)
>   {
>   	int i;
> diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h
> index 163f23c92b41..36d4cb567837 100644
> --- a/include/scsi/libsas.h
> +++ b/include/scsi/libsas.h
> @@ -680,7 +680,6 @@ extern int sas_register_ha(struct sas_ha_struct *);
>   extern int sas_unregister_ha(struct sas_ha_struct *);
>   extern void sas_prep_resume_ha(struct sas_ha_struct *sas_ha);
>   extern void sas_resume_ha(struct sas_ha_struct *sas_ha);
> -extern void sas_resume_ha_no_sync(struct sas_ha_struct *sas_ha);
>   extern void sas_suspend_ha(struct sas_ha_struct *sas_ha);
>   
>   int sas_phy_reset(struct sas_phy *phy, int hard_reset);

Re: [PATCH v2] scsi: libsas: fix HA resume deadlock and hisi_sas disk-wake race

[John Garry]

On 14/07/2026 13:06, yangxingui wrote:
> There are five libsas consumers (callers of sas_register_ha): hisi_sas,
> isci, pm8001, aic94xx, and mvsas. Only hisi_sas needs a driver-side
> change, because it is the only driver that satisfies all three
> conditions below simultaneously.

I am saying that other drivers use sas_resume_ha() and you are changing 
the behavior of that function - how can you guarantee that you are not 
causing regression for that those other drivers?