All of lore.kernel.org
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Gary Guo <ghg@datera.io>,
	Mike Christie <mchristi@redhat.com>,
	Hannes Reinecke <hare@suse.de>,
	Nicholas Bellinger <nab@linux-iscsi.org>
Subject: [PATCH 4.12 13/65] iscsi-target: Fix iscsi_np reset hung task during parallel delete
Date: Mon, 14 Aug 2017 18:19:04 -0700	[thread overview]
Message-ID: <20170815011942.920672991@linuxfoundation.org> (raw)
In-Reply-To: <20170815011942.395714306@linuxfoundation.org>

4.12-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Nicholas Bellinger <nab@linux-iscsi.org>

commit 978d13d60c34818a41fc35962602bdfa5c03f214 upstream.

This patch fixes a bug associated with iscsit_reset_np_thread()
that can occur during parallel configfs rmdir of a single iscsi_np
used across multiple iscsi-target instances, that would result in
hung task(s) similar to below where configfs rmdir process context
was blocked indefinately waiting for iscsi_np->np_restart_comp
to finish:

[ 6726.112076] INFO: task dcp_proxy_node_:15550 blocked for more than 120 seconds.
[ 6726.119440]       Tainted: G        W  O     4.1.26-3321 #2
[ 6726.125045] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 6726.132927] dcp_proxy_node_ D ffff8803f202bc88     0 15550      1 0x00000000
[ 6726.140058]  ffff8803f202bc88 ffff88085c64d960 ffff88083b3b1ad0 ffff88087fffeb08
[ 6726.147593]  ffff8803f202c000 7fffffffffffffff ffff88083f459c28 ffff88083b3b1ad0
[ 6726.155132]  ffff88035373c100 ffff8803f202bca8 ffffffff8168ced2 ffff8803f202bcb8
[ 6726.162667] Call Trace:
[ 6726.165150]  [<ffffffff8168ced2>] schedule+0x32/0x80
[ 6726.170156]  [<ffffffff8168f5b4>] schedule_timeout+0x214/0x290
[ 6726.176030]  [<ffffffff810caef2>] ? __send_signal+0x52/0x4a0
[ 6726.181728]  [<ffffffff8168d7d6>] wait_for_completion+0x96/0x100
[ 6726.187774]  [<ffffffff810e7c80>] ? wake_up_state+0x10/0x10
[ 6726.193395]  [<ffffffffa035d6e2>] iscsit_reset_np_thread+0x62/0xe0 [iscsi_target_mod]
[ 6726.201278]  [<ffffffffa0355d86>] iscsit_tpg_disable_portal_group+0x96/0x190 [iscsi_target_mod]
[ 6726.210033]  [<ffffffffa0363f7f>] lio_target_tpg_store_enable+0x4f/0xc0 [iscsi_target_mod]
[ 6726.218351]  [<ffffffff81260c5a>] configfs_write_file+0xaa/0x110
[ 6726.224392]  [<ffffffff811ea364>] vfs_write+0xa4/0x1b0
[ 6726.229576]  [<ffffffff811eb111>] SyS_write+0x41/0xb0
[ 6726.234659]  [<ffffffff8169042e>] system_call_fastpath+0x12/0x71

It would happen because each iscsit_reset_np_thread() sets state
to ISCSI_NP_THREAD_RESET, sends SIGINT, and then blocks waiting
for completion on iscsi_np->np_restart_comp.

However, if iscsi_np was active processing a login request and
more than a single iscsit_reset_np_thread() caller to the same
iscsi_np was blocked on iscsi_np->np_restart_comp, iscsi_np
kthread process context in __iscsi_target_login_thread() would
flush pending signals and only perform a single completion of
np->np_restart_comp before going back to sleep within transport
specific iscsit_transport->iscsi_accept_np code.

To address this bug, add a iscsi_np->np_reset_count and update
__iscsi_target_login_thread() to keep completing np->np_restart_comp
until ->np_reset_count has reached zero.

Reported-by: Gary Guo <ghg@datera.io>
Tested-by: Gary Guo <ghg@datera.io>
Cc: Mike Christie <mchristi@redhat.com>
Cc: Hannes Reinecke <hare@suse.de>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 drivers/target/iscsi/iscsi_target.c       |    1 +
 drivers/target/iscsi/iscsi_target_login.c |    7 +++++--
 include/target/iscsi/iscsi_target_core.h  |    1 +
 3 files changed, 7 insertions(+), 2 deletions(-)

--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -418,6 +418,7 @@ int iscsit_reset_np_thread(
 		return 0;
 	}
 	np->np_thread_state = ISCSI_NP_THREAD_RESET;
+	atomic_inc(&np->np_reset_count);
 
 	if (np->np_thread) {
 		spin_unlock_bh(&np->np_thread_lock);
--- a/drivers/target/iscsi/iscsi_target_login.c
+++ b/drivers/target/iscsi/iscsi_target_login.c
@@ -1237,9 +1237,11 @@ static int __iscsi_target_login_thread(s
 	flush_signals(current);
 
 	spin_lock_bh(&np->np_thread_lock);
-	if (np->np_thread_state == ISCSI_NP_THREAD_RESET) {
+	if (atomic_dec_if_positive(&np->np_reset_count) >= 0) {
 		np->np_thread_state = ISCSI_NP_THREAD_ACTIVE;
+		spin_unlock_bh(&np->np_thread_lock);
 		complete(&np->np_restart_comp);
+		return 1;
 	} else if (np->np_thread_state == ISCSI_NP_THREAD_SHUTDOWN) {
 		spin_unlock_bh(&np->np_thread_lock);
 		goto exit;
@@ -1272,7 +1274,8 @@ static int __iscsi_target_login_thread(s
 		goto exit;
 	} else if (rc < 0) {
 		spin_lock_bh(&np->np_thread_lock);
-		if (np->np_thread_state == ISCSI_NP_THREAD_RESET) {
+		if (atomic_dec_if_positive(&np->np_reset_count) >= 0) {
+			np->np_thread_state = ISCSI_NP_THREAD_ACTIVE;
 			spin_unlock_bh(&np->np_thread_lock);
 			complete(&np->np_restart_comp);
 			iscsit_put_transport(conn->conn_transport);
--- a/include/target/iscsi/iscsi_target_core.h
+++ b/include/target/iscsi/iscsi_target_core.h
@@ -787,6 +787,7 @@ struct iscsi_np {
 	int			np_sock_type;
 	enum np_thread_state_table np_thread_state;
 	bool                    enabled;
+	atomic_t		np_reset_count;
 	enum iscsi_timer_flags_table np_login_timer_flags;
 	u32			np_exports;
 	enum np_flags_table	np_flags;

  parent reply	other threads:[~2017-08-15  1:36 UTC|newest]

Thread overview: 80+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-08-15  1:18 [PATCH 4.12 00/65] 4.12.8-stable review Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.12 01/65] mm: ratelimit PFNs busy info message Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.12 02/65] mm: fix list corruptions on shmem shrinklist Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.12 03/65] futex: Remove unnecessary warning from get_futex_key Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.12 04/65] xtensa: fix cache aliasing handling code for WT cache Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.12 05/65] xtensa: mm/cache: add missing EXPORT_SYMBOLs Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.12 06/65] xtensa: dont limit csum_partial export by CONFIG_NET Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.12 07/65] xfs: Fix leak of discard bio Greg Kroah-Hartman
2017-08-15  1:18 ` [PATCH 4.12 08/65] pinctrl: armada-37xx: Fix number of pin in south bridge Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 09/65] mtd: nand: atmel: Fix DT backward compatibility in pmecc.c Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 10/65] mtd: nand: Fix timing setup for NANDs that do not support SET FEATURES Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 11/65] mtd: nand: Declare tBERS, tR and tPROG as u64 to avoid integer overflow Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 12/65] iscsi-target: fix memory leak in iscsit_setup_text_cmd() Greg Kroah-Hartman
2017-08-15  1:19 ` Greg Kroah-Hartman [this message]
2017-08-15  1:19 ` [PATCH 4.12 14/65] usb-storage: fix deadlock involving host lock and scsi_done Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 15/65] target: Fix node_acl demo-mode + uncached dynamic shutdown regression Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 16/65] fuse: initialize the flock flag in fuse_file on allocation Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 17/65] i2c: designware: Some broken DSTDs use 1MiHz instead of 1MHz Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 18/65] nand: fix wrong default oob layout for small pages using soft ecc Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 19/65] mmc: mmc: correct the logic for setting HS400ES signal voltage Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 20/65] nfs/flexfiles: fix leak of nfs4_ff_ds_version arrays Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 21/65] drm/bridge: tc358767: fix probe without attached output node Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 22/65] drm/etnaviv: Fix off-by-one error in reloc checking Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 23/65] drm/i915: Fix out-of-bounds array access in bdw_load_gamma_lut Greg Kroah-Hartman
2017-08-15  1:19   ` Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 24/65] USB: serial: option: add D-Link DWM-222 device ID Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 25/65] USB: serial: cp210x: add support for Qivicon USB ZigBee dongle Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 26/65] USB: serial: pl2303: add new ATEN device id Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 27/65] usb: musb: fix tx fifo flush handling again Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 28/65] USB: hcd: Mark secondary HCD as dead if the primary one died Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 29/65] staging:iio:resolver:ad2s1210 fix negative IIO_ANGL_VEL read Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 30/65] iio: aspeed-adc: wait for initial sequence Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 31/65] iio: accel: st_accel: add SPI-3wire support Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 32/65] iio: accel: bmc150: Always restore device to normal mode after suspend-resume Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 33/65] iio: pressure: st_pressure_core: disable multiread by default for LPS22HB Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 34/65] iio: light: tsl2563: use correct event code Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 35/65] iio: adc: Revert "axp288: Drop bogus AXP288_ADC_TS_PIN_CTRL register modifications" Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 36/65] staging: comedi: comedi_fops: do not call blocking ops when !TASK_RUNNING Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 37/65] uas: Add US_FL_IGNORE_RESIDUE for Initio Corporation INIC-3069 Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 38/65] firmware: fix batched requests - wake all waiters Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 39/65] firmware: fix batched requests - send wake up on failure on direct lookups Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 40/65] firmware: avoid invalid fallback aborts by using killable wait Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 41/65] block: Make blk_mq_delay_kick_requeue_list() rerun the queue at a quiet time Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 42/65] usb: gadget: udc: renesas_usb3: Fix usb_gadget_giveback_request() calling Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 43/65] usb: renesas_usbhs: Fix UGCTRL2 value for R-Car Gen3 Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 44/65] USB: Check for dropped connection before switching to full speed Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 45/65] usb: core: unlink urbs from the tail of the endpoints urb_list Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 46/65] usb: quirks: Add no-lpm quirk for Moshi USB to Ethernet Adapter Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 47/65] usb:xhci:Add quirk for Certain failing HP keyboard on reset after resume Greg Kroah-Hartman
2017-08-18 20:02   ` Ben Hutchings
2017-08-18 22:50     ` Greg Kroah-Hartman
2017-08-18 22:50       ` Greg Kroah-Hartman
2017-08-21  9:55       ` Sandeep Singh
2017-08-15  1:19 ` [PATCH 4.12 48/65] PCI: Protect pci_error_handlers->reset_notify() usage with device_lock() Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 49/65] PCI: Remove __pci_dev_reset() and pci_dev_reset() Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 50/65] PCI: Add pci_reset_function_locked() Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 51/65] xhci: Reset Renesas uPD72020x USB controller for 32-bit DMA issue Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 52/65] iio: adc: vf610_adc: Fix VALT selection value for REFSEL bits Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 53/65] pnfs/blocklayout: require 64-bit sector_t Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 54/65] pinctrl: cherryview: Add Setzer models to the Chromebook DMI quirk Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 55/65] pinctrl: sunxi: add a missing function of A10/A20 pinctrl driver Greg Kroah-Hartman
2017-08-18 20:07   ` Ben Hutchings
2017-08-22  9:11     ` Chen-Yu Tsai
2017-08-22 15:51       ` Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 56/65] pinctrl: intel: merrifield: Correct UART pin lists Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 57/65] pinctrl: uniphier: fix WARN_ON() of pingroups dump on LD11 Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 58/65] pinctrl: uniphier: fix WARN_ON() of pingroups dump on LD20 Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 59/65] pinctrl: samsung: Remove bogus irq_[un]mask from resource management Greg Kroah-Hartman
2017-08-15  1:19   ` Greg Kroah-Hartman
2017-08-15  1:19   ` Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 60/65] pinctrl: meson-gxbb: Add missing GPIODV_18 pin entry Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 61/65] pinctrl: meson-gxl: " Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 62/65] MIPS: DEC: Fix an int-handler.S CPU_DADDI_WORKAROUNDS regression Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 63/65] Revert "MIPS: Dont unnecessarily include kmalloc.h into <asm/cache.h>." Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 64/65] MIPS: Octeon: Fix broken EDAC driver Greg Kroah-Hartman
2017-08-15  1:19 ` [PATCH 4.12 65/65] powerpc: Fix /proc/cpuinfo revision for POWER9 DD2 Greg Kroah-Hartman
2017-08-15 10:45 ` [PATCH 4.12 00/65] 4.12.8-stable review Guenter Roeck
2017-08-15 14:34   ` Greg Kroah-Hartman
2017-08-15 18:09 ` Shuah Khan
2017-08-15 18:10   ` Greg Kroah-Hartman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170815011942.920672991@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=ghg@datera.io \
    --cc=hare@suse.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mchristi@redhat.com \
    --cc=nab@linux-iscsi.org \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.