stable.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Gary Guo <ghg@datera.io>,
	Mike Christie <mchristi@redhat.com>,
	Hannes Reinecke <hare@suse.de>,
	Nicholas Bellinger <nab@linux-iscsi.org>
Subject: [PATCH 4.4 04/23] iscsi-target: Fix iscsi_np reset hung task during parallel delete
Date: Mon, 14 Aug 2017 18:18:21 -0700	[thread overview]
Message-ID: <20170815011754.409640676@linuxfoundation.org> (raw)
In-Reply-To: <20170815011754.234902525@linuxfoundation.org>

4.4-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Nicholas Bellinger <nab@linux-iscsi.org>

commit 978d13d60c34818a41fc35962602bdfa5c03f214 upstream.

This patch fixes a bug associated with iscsit_reset_np_thread()
that can occur during parallel configfs rmdir of a single iscsi_np
used across multiple iscsi-target instances, that would result in
hung task(s) similar to below where configfs rmdir process context
was blocked indefinately waiting for iscsi_np->np_restart_comp
to finish:

[ 6726.112076] INFO: task dcp_proxy_node_:15550 blocked for more than 120 seconds.
[ 6726.119440]       Tainted: G        W  O     4.1.26-3321 #2
[ 6726.125045] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 6726.132927] dcp_proxy_node_ D ffff8803f202bc88     0 15550      1 0x00000000
[ 6726.140058]  ffff8803f202bc88 ffff88085c64d960 ffff88083b3b1ad0 ffff88087fffeb08
[ 6726.147593]  ffff8803f202c000 7fffffffffffffff ffff88083f459c28 ffff88083b3b1ad0
[ 6726.155132]  ffff88035373c100 ffff8803f202bca8 ffffffff8168ced2 ffff8803f202bcb8
[ 6726.162667] Call Trace:
[ 6726.165150]  [<ffffffff8168ced2>] schedule+0x32/0x80
[ 6726.170156]  [<ffffffff8168f5b4>] schedule_timeout+0x214/0x290
[ 6726.176030]  [<ffffffff810caef2>] ? __send_signal+0x52/0x4a0
[ 6726.181728]  [<ffffffff8168d7d6>] wait_for_completion+0x96/0x100
[ 6726.187774]  [<ffffffff810e7c80>] ? wake_up_state+0x10/0x10
[ 6726.193395]  [<ffffffffa035d6e2>] iscsit_reset_np_thread+0x62/0xe0 [iscsi_target_mod]
[ 6726.201278]  [<ffffffffa0355d86>] iscsit_tpg_disable_portal_group+0x96/0x190 [iscsi_target_mod]
[ 6726.210033]  [<ffffffffa0363f7f>] lio_target_tpg_store_enable+0x4f/0xc0 [iscsi_target_mod]
[ 6726.218351]  [<ffffffff81260c5a>] configfs_write_file+0xaa/0x110
[ 6726.224392]  [<ffffffff811ea364>] vfs_write+0xa4/0x1b0
[ 6726.229576]  [<ffffffff811eb111>] SyS_write+0x41/0xb0
[ 6726.234659]  [<ffffffff8169042e>] system_call_fastpath+0x12/0x71

It would happen because each iscsit_reset_np_thread() sets state
to ISCSI_NP_THREAD_RESET, sends SIGINT, and then blocks waiting
for completion on iscsi_np->np_restart_comp.

However, if iscsi_np was active processing a login request and
more than a single iscsit_reset_np_thread() caller to the same
iscsi_np was blocked on iscsi_np->np_restart_comp, iscsi_np
kthread process context in __iscsi_target_login_thread() would
flush pending signals and only perform a single completion of
np->np_restart_comp before going back to sleep within transport
specific iscsit_transport->iscsi_accept_np code.

To address this bug, add a iscsi_np->np_reset_count and update
__iscsi_target_login_thread() to keep completing np->np_restart_comp
until ->np_reset_count has reached zero.

Reported-by: Gary Guo <ghg@datera.io>
Tested-by: Gary Guo <ghg@datera.io>
Cc: Mike Christie <mchristi@redhat.com>
Cc: Hannes Reinecke <hare@suse.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 drivers/target/iscsi/iscsi_target.c       |    1 +
 drivers/target/iscsi/iscsi_target_login.c |    7 +++++--
 include/target/iscsi/iscsi_target_core.h  |    1 +
 3 files changed, 7 insertions(+), 2 deletions(-)

--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -418,6 +418,7 @@ int iscsit_reset_np_thread(
 		return 0;
 	}
 	np->np_thread_state = ISCSI_NP_THREAD_RESET;
+	atomic_inc(&np->np_reset_count);
 
 	if (np->np_thread) {
 		spin_unlock_bh(&np->np_thread_lock);
--- a/drivers/target/iscsi/iscsi_target_login.c
+++ b/drivers/target/iscsi/iscsi_target_login.c
@@ -1219,9 +1219,11 @@ static int __iscsi_target_login_thread(s
 	flush_signals(current);
 
 	spin_lock_bh(&np->np_thread_lock);
-	if (np->np_thread_state == ISCSI_NP_THREAD_RESET) {
+	if (atomic_dec_if_positive(&np->np_reset_count) >= 0) {
 		np->np_thread_state = ISCSI_NP_THREAD_ACTIVE;
+		spin_unlock_bh(&np->np_thread_lock);
 		complete(&np->np_restart_comp);
+		return 1;
 	} else if (np->np_thread_state == ISCSI_NP_THREAD_SHUTDOWN) {
 		spin_unlock_bh(&np->np_thread_lock);
 		goto exit;
@@ -1254,7 +1256,8 @@ static int __iscsi_target_login_thread(s
 		goto exit;
 	} else if (rc < 0) {
 		spin_lock_bh(&np->np_thread_lock);
-		if (np->np_thread_state == ISCSI_NP_THREAD_RESET) {
+		if (atomic_dec_if_positive(&np->np_reset_count) >= 0) {
+			np->np_thread_state = ISCSI_NP_THREAD_ACTIVE;
 			spin_unlock_bh(&np->np_thread_lock);
 			complete(&np->np_restart_comp);
 			iscsit_put_transport(conn->conn_transport);
--- a/include/target/iscsi/iscsi_target_core.h
+++ b/include/target/iscsi/iscsi_target_core.h
@@ -784,6 +784,7 @@ struct iscsi_np {
 	int			np_sock_type;
 	enum np_thread_state_table np_thread_state;
 	bool                    enabled;
+	atomic_t		np_reset_count;
 	enum iscsi_timer_flags_table np_login_timer_flags;
 	u32			np_exports;
 	enum np_flags_table	np_flags;

  parent reply	other threads:[~2017-08-15  1:18 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-08-15  1:18 [PATCH 4.4 00/23] 4.4.83-stable review Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 01/23] cpuset: fix a deadlock due to incomplete patching of cpusets_enabled() Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 02/23] mm: ratelimit PFNs busy info message Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 03/23] iscsi-target: fix memory leak in iscsit_setup_text_cmd() Greg Kroah-Hartman
2017-08-15  1:18 ` Greg Kroah-Hartman [this message]
2017-08-15  1:18 ` [PATCH 4.4 05/23] fuse: initialize the flock flag in fuse_file on allocation Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 06/23] nfs/flexfiles: fix leak of nfs4_ff_ds_version arrays Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 07/23] USB: serial: option: add D-Link DWM-222 device ID Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 08/23] USB: serial: cp210x: add support for Qivicon USB ZigBee dongle Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 09/23] USB: serial: pl2303: add new ATEN device id Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 10/23] usb: musb: fix tx fifo flush handling again Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 11/23] USB: hcd: Mark secondary HCD as dead if the primary one died Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 12/23] staging:iio:resolver:ad2s1210 fix negative IIO_ANGL_VEL read Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 13/23] iio: accel: bmc150: Always restore device to normal mode after suspend-resume Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 14/23] iio: light: tsl2563: use correct event code Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 15/23] uas: Add US_FL_IGNORE_RESIDUE for Initio Corporation INIC-3069 Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 16/23] USB: Check for dropped connection before switching to full speed Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 17/23] usb: core: unlink urbs from the tail of the endpoints urb_list Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 18/23] usb: quirks: Add no-lpm quirk for Moshi USB to Ethernet Adapter Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 19/23] usb:xhci:Add quirk for Certain failing HP keyboard on reset after resume Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 20/23] iio: adc: vf610_adc: Fix VALT selection value for REFSEL bits Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 21/23] pnfs/blocklayout: require 64-bit sector_t Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 22/23] pinctrl: sunxi: add a missing function of A10/A20 pinctrl driver Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.4 23/23] pinctrl: samsung: Remove bogus irq_[un]mask from resource management Greg Kroah-Hartman
2017-08-15 10:42 ` [PATCH 4.4 00/23] 4.4.83-stable review Guenter Roeck
2017-08-15 18:08 ` Shuah Khan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170815011754.409640676@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=ghg@datera.io \
    --cc=hare@suse.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mchristi@redhat.com \
    --cc=nab@linux-iscsi.org \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).