linux-scsi.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Hannes Reinecke <hare@suse.de>
To: Nic Bellinger <nab@linux-iscsi.org>
Cc: target-devel@vger.kernel.org, linux-scsi@vger.kernel.org,
	Martin Wilck <martin.wilck@suse.com>,
	Hannes Reinecke <hare@suse.de>, Hannes Reinecke <hare@suse.com>
Subject: [PATCH 1/6] target: fix hang in target_wait_for_sess_cmds()
Date: Mon, 22 Aug 2016 10:54:06 +0200	[thread overview]
Message-ID: <1471856051-87398-2-git-send-email-hare@suse.de> (raw)
In-Reply-To: <1471856051-87398-1-git-send-email-hare@suse.de>

When shutting down a session I'm seeing this hung_task message:

INFO: task kworker/u128:0:6 blocked for more than 480 seconds.
      Tainted: G            E   N  4.4.17-default+ #241
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kworker/u128:0  D ffff880237615c00     0     6      2 0x00000000
 Workqueue: fc_rport_eq fc_rport_work [libfc]
 ffff88017edd7c48 ffffffff81c11500 ffff88017edd0180 ffff88017edd8000
 ffff88035b6860a8 ffff88017edd0180 ffff88036461eac0 0000000000000000
 ffff88017edd7c60 ffffffff815db4d5 7fffffffffffffff ffff88017edd7d10
Call Trace:
 [<ffffffff815db4d5>] schedule+0x35/0x80
 [<ffffffff815dde17>] schedule_timeout+0x237/0x2d0
 [<ffffffff815dc6e3>] wait_for_completion+0xa3/0x110
 [<ffffffffa0869fb5>] target_wait_for_sess_cmds+0x45/0x190 [target_core_mod]
 [<ffffffffa03953b4>] ft_close_sess+0x24/0x30 [tcm_fc]
 [<ffffffffa03955ba>] ft_prlo+0x8a/0x95 [tcm_fc]
 [<ffffffffa066fc28>] fc_rport_work+0x3b8/0x7c0 [libfc]
 [<ffffffff810939ee>] process_one_work+0x14e/0x410
 [<ffffffff81094246>] worker_thread+0x116/0x490

Looking at the code it looks as if there is some confusion about
when to call 'wait_for_completion(&cmd->cmd_wait_comp).
The problem is that there are _two_ sections where the code
waits for 'cmd_wait_comp' completion, but the completion
is only called with 'complete', not 'complete_all'.
So we cannot have both waiting for it, doing so will lead
to a stuck wait_for_completion.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/target/target_core_transport.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c
index 6094a6b..2e1a6d8 100644
--- a/drivers/target/target_core_transport.c
+++ b/drivers/target/target_core_transport.c
@@ -2522,7 +2522,7 @@ int transport_generic_free_cmd(struct se_cmd *cmd, int wait_for_tasks)
 	 * the remaining calls to target_put_sess_cmd(), and not the
 	 * callers of this function.
 	 */
-	if (aborted) {
+	if (aborted && !cmd->cmd_wait_set) {
 		pr_debug("Detected CMD_T_ABORTED for ITT: %llu\n", cmd->tag);
 		wait_for_completion(&cmd->cmd_wait_comp);
 		cmd->se_tfo->release_cmd(cmd);
@@ -2642,9 +2642,11 @@ void target_sess_cmd_list_set_waiting(struct se_session *se_sess)
 	list_for_each_entry(se_cmd, &se_sess->sess_wait_list, se_cmd_list) {
 		rc = kref_get_unless_zero(&se_cmd->cmd_kref);
 		if (rc) {
-			se_cmd->cmd_wait_set = 1;
 			spin_lock(&se_cmd->t_state_lock);
-			se_cmd->transport_state |= CMD_T_FABRIC_STOP;
+			if (!(se_cmd->transport_state & CMD_T_FABRIC_STOP)) {
+				se_cmd->cmd_wait_set = 1;
+				se_cmd->transport_state |= CMD_T_FABRIC_STOP;
+			}
 			spin_unlock(&se_cmd->t_state_lock);
 		}
 	}
@@ -2664,24 +2666,26 @@ void target_wait_for_sess_cmds(struct se_session *se_sess)
 
 	list_for_each_entry_safe(se_cmd, tmp_cmd,
 				&se_sess->sess_wait_list, se_cmd_list) {
+		int cmd_wait;
 		pr_debug("Waiting for se_cmd: %p t_state: %d, fabric state:"
 			" %d\n", se_cmd, se_cmd->t_state,
 			se_cmd->se_tfo->get_cmd_state(se_cmd));
 
 		spin_lock_irqsave(&se_cmd->t_state_lock, flags);
 		tas = (se_cmd->transport_state & CMD_T_TAS);
+		cmd_wait = se_cmd->cmd_wait_set;
 		spin_unlock_irqrestore(&se_cmd->t_state_lock, flags);
-
 		if (!target_put_sess_cmd(se_cmd)) {
 			if (tas)
 				target_put_sess_cmd(se_cmd);
 		}
-
+		if (!cmd_wait)
+			continue;
 		wait_for_completion(&se_cmd->cmd_wait_comp);
 		pr_debug("After cmd_wait_comp: se_cmd: %p t_state: %d"
 			" fabric state: %d\n", se_cmd, se_cmd->t_state,
 			se_cmd->se_tfo->get_cmd_state(se_cmd));
-
+		se_cmd->cmd_wait_set = 0;
 		se_cmd->se_tfo->release_cmd(se_cmd);
 	}
 
-- 
1.8.5.6

  reply	other threads:[~2016-08-22  8:54 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-08-22  8:54 [PATCH 0/6] tcm_fc fixes Hannes Reinecke
2016-08-22  8:54 ` Hannes Reinecke [this message]
2016-10-21  8:00   ` [PATCH 1/6] target: fix hang in target_wait_for_sess_cmds() Nicholas A. Bellinger
2016-08-22  8:54 ` [PATCH 2/6] target: fix potential race window in target_sess_cmd_list_waiting() Hannes Reinecke
2016-10-21  8:19   ` Nicholas A. Bellinger
2016-08-22  8:54 ` [PATCH 3/6] target/tcm_fc: print command pointer in debug message Hannes Reinecke
2016-08-22  8:54 ` [PATCH 4/6] target/tcm_fc: return detailed error in ft_sess_create() Hannes Reinecke
2016-08-22  8:54 ` [PATCH 5/6] target/tcm_fc: Update debugging statements to match libfc usage Hannes Reinecke
2016-08-23 18:07   ` Bart Van Assche
2016-08-22  8:54 ` [PATCH 6/6] target/tcm_fc: use CPU affinity for responses Hannes Reinecke
2016-10-21  7:46 ` [PATCH 0/6] tcm_fc fixes Nicholas A. Bellinger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1471856051-87398-2-git-send-email-hare@suse.de \
    --to=hare@suse.de \
    --cc=hare@suse.com \
    --cc=linux-scsi@vger.kernel.org \
    --cc=martin.wilck@suse.com \
    --cc=nab@linux-iscsi.org \
    --cc=target-devel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).