* [PATCH 0/1] Write commands abort hang @ 2026-07-17 14:38 Maurizio Lombardi 2026-07-17 14:38 ` [PATCH 1/1] target: iscsi: Fix hang for aborted WRITE_PENDING commands Maurizio Lombardi 0 siblings, 1 reply; 5+ messages in thread From: Maurizio Lombardi @ 2026-07-17 14:38 UTC (permalink / raw) To: martin.petersen; +Cc: mlombard, michael.christie, target-devel, loberman One of our customers reported that sometimes the LIO driver hangs when performing a LUN reset while I/O is underway. I was able to reproduce the hang by using the following libiscsi patch: https://arkamax.eu/misc/dataout_abort_libiscsi.patch INFO: task kworker/5:0:514619 blocked for more than 46 seconds. __wait_for_common+0x94/0x1b0 target_put_cmd_and_wait+0x6d/0xb0 [target_core_mod] core_tmr_drain_state_list+0x2a3/0x330 [target_core_mod] core_tmr_lun_reset+0x93/0x1a0 [target_core_mod] target_tmr_work+0xce/0x100 [target_core_mod] process_one_work+0x188/0x3b0 worker_thread+0x2ef/0x410 Maurizio Lombardi (1): scsi: target: iscsi: Fix hang for aborted WRITE_PENDING commands drivers/target/iscsi/iscsi_target.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) -- 2.55.0 ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 1/1] target: iscsi: Fix hang for aborted WRITE_PENDING commands 2026-07-17 14:38 [PATCH 0/1] Write commands abort hang Maurizio Lombardi @ 2026-07-17 14:38 ` Maurizio Lombardi 2026-07-17 15:08 ` Laurence Oberman 2026-07-17 16:38 ` Bart Van Assche 0 siblings, 2 replies; 5+ messages in thread From: Maurizio Lombardi @ 2026-07-17 14:38 UTC (permalink / raw) To: martin.petersen; +Cc: mlombard, michael.christie, target-devel, loberman When a LUN_RESET aborts a WRITE command that is in the TRANSPORT_WRITE_PENDING state, the target core sets CMD_T_ABORTED and waits for the frontend to finish processing. If the initiator subsequently sends the remaining dataout PDUs, __iscsit_check_dataout_hdr() catches the payload, stops the dataout timer if the sequence is final and finally dumps the data. However, the iSCSI target doesn't trigger the completion process for these aborted commands. Because of this, the abort path hangs indefinitely in target_put_cmd_and_wait(), leading to a deadlocked target worker thread. Fix this by explicitly calling target_complete_cmd() when the final dataout PDU is received for an aborted WRITE command. target_complete_cmd() detects the CMD_T_ABORTED flag and cleanly routes the command into target_abort_work, allowing the abort completion to successfully unblock. Signed-off-by: Maurizio Lombardi <mlombard@redhat.com> --- drivers/target/iscsi/iscsi_target.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c index 62ada3a52210..124ff269b8e7 100644 --- a/drivers/target/iscsi/iscsi_target.c +++ b/drivers/target/iscsi/iscsi_target.c @@ -1533,8 +1533,10 @@ __iscsit_check_dataout_hdr(struct iscsit_conn *conn, void *buf, */ if (se_cmd->transport_state & CMD_T_ABORTED) { if (hdr->flags & ISCSI_FLAG_CMD_FINAL && - --cmd->outstanding_r2ts < 1) + --cmd->outstanding_r2ts < 1) { iscsit_stop_dataout_timer(cmd); + target_complete_cmd(se_cmd, SAM_STAT_TASK_ABORTED); + } return iscsit_dump_data_payload(conn, payload_length, 1); } -- 2.55.0 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 1/1] target: iscsi: Fix hang for aborted WRITE_PENDING commands 2026-07-17 14:38 ` [PATCH 1/1] target: iscsi: Fix hang for aborted WRITE_PENDING commands Maurizio Lombardi @ 2026-07-17 15:08 ` Laurence Oberman 2026-07-17 16:38 ` Bart Van Assche 1 sibling, 0 replies; 5+ messages in thread From: Laurence Oberman @ 2026-07-17 15:08 UTC (permalink / raw) To: Maurizio Lombardi, martin.petersen Cc: mlombard, michael.christie, target-devel On Fri, 2026-07-17 at 16:38 +0200, Maurizio Lombardi wrote: > When a LUN_RESET aborts a WRITE command that is in the > TRANSPORT_WRITE_PENDING state, the target core sets CMD_T_ABORTED and > waits > for the frontend to finish processing. > > If the initiator subsequently sends the remaining dataout PDUs, > __iscsit_check_dataout_hdr() catches the payload, stops the dataout > timer > if the sequence is final and finally dumps the data. > However, the iSCSI target doesn't trigger the completion process for > these > aborted commands. Because of this, the abort path hangs indefinitely > in > target_put_cmd_and_wait(), leading to a deadlocked target worker > thread. > > Fix this by explicitly calling target_complete_cmd() when the final > dataout > PDU is received for an aborted WRITE command. target_complete_cmd() > detects > the CMD_T_ABORTED flag and cleanly routes the command into > target_abort_work, > allowing the abort completion to successfully unblock. > > Signed-off-by: Maurizio Lombardi <mlombard@redhat.com> > --- > drivers/target/iscsi/iscsi_target.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/drivers/target/iscsi/iscsi_target.c > b/drivers/target/iscsi/iscsi_target.c > index 62ada3a52210..124ff269b8e7 100644 > --- a/drivers/target/iscsi/iscsi_target.c > +++ b/drivers/target/iscsi/iscsi_target.c > @@ -1533,8 +1533,10 @@ __iscsit_check_dataout_hdr(struct iscsit_conn > *conn, void *buf, > */ > if (se_cmd->transport_state & CMD_T_ABORTED) { > if (hdr->flags & ISCSI_FLAG_CMD_FINAL && > - --cmd->outstanding_r2ts < 1) > + --cmd->outstanding_r2ts < 1) { > iscsit_stop_dataout_timer(cmd); > + target_complete_cmd(se_cmd, > SAM_STAT_TASK_ABORTED); > + } > > return iscsit_dump_data_payload(conn, > payload_length, 1); > } This took a long time to solve with many attempts that did not help until Maurizio's solution. This patch was tested fully at a customer to fully resolve the issue for over a week with no negative impacts. Looks good. Reviewed-by: Laurence Oberman <loberman@redhat.com> ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 1/1] target: iscsi: Fix hang for aborted WRITE_PENDING commands 2026-07-17 14:38 ` [PATCH 1/1] target: iscsi: Fix hang for aborted WRITE_PENDING commands Maurizio Lombardi 2026-07-17 15:08 ` Laurence Oberman @ 2026-07-17 16:38 ` Bart Van Assche 2026-07-20 12:41 ` Maurizio Lombardi 1 sibling, 1 reply; 5+ messages in thread From: Bart Van Assche @ 2026-07-17 16:38 UTC (permalink / raw) To: Maurizio Lombardi, martin.petersen Cc: mlombard, michael.christie, target-devel, loberman On 7/17/26 7:38 AM, Maurizio Lombardi wrote: > When a LUN_RESET aborts a WRITE command that is in the > TRANSPORT_WRITE_PENDING state, the target core sets CMD_T_ABORTED and waits > for the frontend to finish processing. > > If the initiator subsequently sends the remaining dataout PDUs, > __iscsit_check_dataout_hdr() catches the payload, stops the dataout timer > if the sequence is final and finally dumps the data. > However, the iSCSI target doesn't trigger the completion process for these > aborted commands. Because of this, the abort path hangs indefinitely in > target_put_cmd_and_wait(), leading to a deadlocked target worker thread. > > Fix this by explicitly calling target_complete_cmd() when the final dataout > PDU is received for an aborted WRITE command. target_complete_cmd() detects > the CMD_T_ABORTED flag and cleanly routes the command into target_abort_work, > allowing the abort completion to successfully unblock. Shouldn't a Fixes: tag be added to this patch? Thanks, Bart. ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 1/1] target: iscsi: Fix hang for aborted WRITE_PENDING commands 2026-07-17 16:38 ` Bart Van Assche @ 2026-07-20 12:41 ` Maurizio Lombardi 0 siblings, 0 replies; 5+ messages in thread From: Maurizio Lombardi @ 2026-07-20 12:41 UTC (permalink / raw) To: Bart Van Assche, Maurizio Lombardi, martin.petersen Cc: mlombard, michael.christie, target-devel, loberman On Fri Jul 17, 2026 at 6:38 PM CEST, Bart Van Assche wrote: > On 7/17/26 7:38 AM, Maurizio Lombardi wrote: >> When a LUN_RESET aborts a WRITE command that is in the >> TRANSPORT_WRITE_PENDING state, the target core sets CMD_T_ABORTED and waits >> for the frontend to finish processing. >> >> If the initiator subsequently sends the remaining dataout PDUs, >> __iscsit_check_dataout_hdr() catches the payload, stops the dataout timer >> if the sequence is final and finally dumps the data. >> However, the iSCSI target doesn't trigger the completion process for these >> aborted commands. Because of this, the abort path hangs indefinitely in >> target_put_cmd_and_wait(), leading to a deadlocked target worker thread. >> >> Fix this by explicitly calling target_complete_cmd() when the final dataout >> PDU is received for an aborted WRITE command. target_complete_cmd() detects >> the CMD_T_ABORTED flag and cleanly routes the command into target_abort_work, >> allowing the abort completion to successfully unblock. > > Shouldn't a Fixes: tag be added to this patch? It's a bit difficult to pinpoint an exact commit. Probably the roots of this bug could be found in aaa00cc93c1d0fd2693a ("scsi: target/core: Fix TAS handling for aborted commands"), you are the author of that commit btw. But at the time (we are talking about 4.x kernels) the abort mechanism was working in a different way, target_abort_work didn't even exists and target_complete_cmd() was quite different... Maurizio ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-07-20 12:47 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-07-17 14:38 [PATCH 0/1] Write commands abort hang Maurizio Lombardi 2026-07-17 14:38 ` [PATCH 1/1] target: iscsi: Fix hang for aborted WRITE_PENDING commands Maurizio Lombardi 2026-07-17 15:08 ` Laurence Oberman 2026-07-17 16:38 ` Bart Van Assche 2026-07-20 12:41 ` Maurizio Lombardi
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox