All of lore.kernel.org
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Sebastian Herbszt <herbszt@gmx.de>,
	James Bottomley <James.Bottomley@HansenPartnership.com>
Subject: [PATCH 3.14 31/70] scsi: fix soft lockup in scsi_remove_target() on module removal
Date: Tue, 23 Feb 2016 19:33:40 -0800	[thread overview]
Message-ID: <20160224033355.108889347@linuxfoundation.org> (raw)
In-Reply-To: <20160224033354.061464831@linuxfoundation.org>

3.14-stable review patch.  If anyone has any objections, please let me know.

------------------

From: James Bottomley <James.Bottomley@HansenPartnership.com>

commit 90a88d6ef88edcfc4f644dddc7eef4ea41bccf8b upstream.

This softlockup is currently happening:

[  444.088002] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kworker/1:1:29]
[  444.088002] Modules linked in: lpfc(-) qla2x00tgt(O) qla2xxx_scst(O) scst_vdisk(O) scsi_transport_fc libcrc32c scst(O) dlm configfs nfsd lockd grace nfs_acl auth_rpcgss sunrpc ed
d snd_pcm_oss snd_mixer_oss snd_seq snd_seq_device dm_mod iTCO_wdt snd_hda_codec_realtek snd_hda_codec_generic gpio_ich iTCO_vendor_support ppdev snd_hda_intel snd_hda_codec snd_hda
_core snd_hwdep tg3 snd_pcm snd_timer libphy lpc_ich parport_pc ptp acpi_cpufreq snd pps_core fjes parport i2c_i801 ehci_pci tpm_tis tpm sr_mod cdrom soundcore floppy hwmon sg 8250_
fintek pcspkr i915 drm_kms_helper uhci_hcd ehci_hcd drm fb_sys_fops sysimgblt sysfillrect syscopyarea i2c_algo_bit usbcore button video usb_common fan ata_generic ata_piix libata th
ermal
[  444.088002] CPU: 1 PID: 29 Comm: kworker/1:1 Tainted: G           O    4.4.0-rc5-2.g1e923a3-default #1
[  444.088002] Hardware name: FUJITSU SIEMENS ESPRIMO E           /D2164-A1, BIOS 5.00 R1.10.2164.A1               05/08/2006
[  444.088002] Workqueue: fc_wq_4 fc_rport_final_delete [scsi_transport_fc]
[  444.088002] task: f6266ec0 ti: f6268000 task.ti: f6268000
[  444.088002] EIP: 0060:[<c07e7044>] EFLAGS: 00000286 CPU: 1
[  444.088002] EIP is at _raw_spin_unlock_irqrestore+0x14/0x20
[  444.088002] EAX: 00000286 EBX: f20d3800 ECX: 00000002 EDX: 00000286
[  444.088002] ESI: f50ba800 EDI: f2146848 EBP: f6269ec8 ESP: f6269ec8
[  444.088002]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[  444.088002] CR0: 8005003b CR2: 08f96600 CR3: 363ae000 CR4: 000006d0
[  444.088002] Stack:
[  444.088002]  f6269eec c066b0f7 00000286 f2146848 f50ba808 f50ba800 f50ba800 f2146a90
[  444.088002]  f2146848 f6269f08 f8f0a4ed f3141000 f2146800 f2146a90 f619fa00 00000040
[  444.088002]  f6269f40 c026cb25 00000001 166c6392 00000061 f6757140 f6136340 00000004
[  444.088002] Call Trace:
[  444.088002]  [<c066b0f7>] scsi_remove_target+0x167/0x1c0
[  444.088002]  [<f8f0a4ed>] fc_rport_final_delete+0x9d/0x1e0 [scsi_transport_fc]
[  444.088002]  [<c026cb25>] process_one_work+0x155/0x3e0
[  444.088002]  [<c026cde7>] worker_thread+0x37/0x490
[  444.088002]  [<c027214b>] kthread+0x9b/0xb0
[  444.088002]  [<c07e72c1>] ret_from_kernel_thread+0x21/0x40

What appears to be happening is that something has pinned the target
so it can't go into STARGET_DEL via final release and the loop in
scsi_remove_target spins endlessly until that happens.

The fix for this soft lockup is to not keep looping over a device that
we've called remove on but which hasn't gone into DEL state.  This
patch will retain a simplistic memory of the last target and not keep
looping over it.

Reported-by: Sebastian Herbszt <herbszt@gmx.de>
Tested-by: Sebastian Herbszt <herbszt@gmx.de>
Fixes: 40998193560dab6c3ce8d25f4fa58a23e252ef38
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 drivers/scsi/scsi_sysfs.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -1123,16 +1123,18 @@ static void __scsi_remove_target(struct
 void scsi_remove_target(struct device *dev)
 {
 	struct Scsi_Host *shost = dev_to_shost(dev->parent);
-	struct scsi_target *starget;
+	struct scsi_target *starget, *last_target = NULL;
 	unsigned long flags;
 
 restart:
 	spin_lock_irqsave(shost->host_lock, flags);
 	list_for_each_entry(starget, &shost->__targets, siblings) {
-		if (starget->state == STARGET_DEL)
+		if (starget->state == STARGET_DEL ||
+		    starget == last_target)
 			continue;
 		if (starget->dev.parent == dev || &starget->dev == dev) {
 			kref_get(&starget->reap_ref);
+			last_target = starget;
 			spin_unlock_irqrestore(shost->host_lock, flags);
 			__scsi_remove_target(starget);
 			scsi_target_reap(starget);

  parent reply	other threads:[~2016-02-24  5:02 UTC|newest]

Thread overview: 75+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-02-24  3:33 [PATCH 3.14 00/70] 3.14.62-stable review Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 02/70] drm/i915: get runtime PM reference around GEM set_caching IOCTL Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 03/70] ALSA: seq: Fix double port list deletion Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 04/70] phy: twl4030-usb: Relase usb phy on unload Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 05/70] wan/x25: Fix use-after-free in x25_asy_open_tty() Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 06/70] staging/speakup: Use tty_ldisc_ref() for paste kworker Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 07/70] pty: fix possible use after free of tty->driver_data Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 08/70] pty: make sure super_block is still valid in final /dev/tty close Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 09/70] serial: 8250_pci: Correct uartclk for xr17v35x expansion chips Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 10/70] AIO: properly check iovec sizes Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 11/70] ext4: fix potential integer overflow Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 13/70] Btrfs: fix hang on extent buffer lock caused by the inode_paths ioctl Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 15/70] perf trace: Fix documentation for -i Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 16/70] ptrace: use fsuid, fsgid, effective creds for fs access checks Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 17/70] tools lib traceevent: Fix output of %llu for 64 bit values read on 32 bit machines Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 18/70] tracing: Fix freak link error caused by branch tracer Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 19/70] klist: fix starting point removed bug in klist iterators Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 20/70] scsi: restart list search after unlock in scsi_remove_target Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 21/70] scsi_sysfs: Fix queue_ramp_up_period return code Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 22/70] iscsi-target: Fix rx_login_comp hang after login failure Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 23/70] Fix a memory leak in scsi_host_dev_release() Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 24/70] SCSI: Fix NULL pointer dereference in runtime PM Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 25/70] Revert "SCSI: Fix NULL pointer dereference in runtime PM" Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 26/70] iscsi-target: Fix potential dead-lock during node acl delete Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 27/70] SCSI: fix crashes in sd and sr runtime PM Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 28/70] drivers/scsi/sg.c: mark VMA as VM_IO to prevent migration Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 29/70] scsi_dh_rdac: always retry MODE SELECT on command lock violation Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 30/70] SCSI: Add Marvell Console to VPD blacklist Greg Kroah-Hartman
2016-02-24  3:33 ` Greg Kroah-Hartman [this message]
2016-02-24  3:33 ` [PATCH 3.14 32/70] iio:ad7793: Fix ad7785 product ID Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 33/70] iio: lpc32xx_adc: fix warnings caused by enabling unprepared clock Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 34/70] iio:ad5064: Make sure ad5064_i2c_write() returns 0 on success Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 35/70] iio: ad5064: Fix ad5629/ad5669 shift Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 36/70] iio: fix some warning messages Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 37/70] iio: adis_buffer: Fix out-of-bounds memory access Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 38/70] iio: dac: mcp4725: set iio name property in sysfs Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 39/70] cifs_dbg() outputs an uninitialized buffer in cifs_readdir() Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 40/70] cifs: fix erroneous return value Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 41/70] nfs: Fix race in __update_open_stateid() Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 42/70] udf: limit the maximum number of indirect extents in a row Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 43/70] udf: Prevent buffer overrun with multi-byte characters Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 44/70] udf: Check output buffer length when converting name to CS0 Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 45/70] ARM: dts: Kirkwood: Fix QNAP TS219 power-off Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 46/70] ARM: 8471/1: need to save/restore arm register(r11) when it is corrupted Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 47/70] ARM: 8519/1: ICST: try other dividends than 1 Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 48/70] ARM: 8517/1: ICST: avoid arithmetic overflow in icst_hz() Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 49/70] KVM: PPC: Fix emulation of H_SET_DABR/X on POWER8 Greg Kroah-Hartman
2016-02-24  3:33 ` [PATCH 3.14 50/70] fuse: break infinite loop in fuse_fill_write_pages() Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 51/70] mm: soft-offline: check return value in second __get_any_page() call Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 52/70] mm: fix mlock accouting Greg Kroah-Hartman
2016-02-25  1:02   ` Luis Henriques
2016-02-25  1:02     ` Luis Henriques
2016-02-25  9:04     ` Michal Hocko
2016-02-25  9:04       ` Michal Hocko
2016-02-25  9:48       ` Jiri Slaby
2016-02-25 19:22       ` Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 53/70] Input: elantech - add Fujitsu Lifebook U745 to force crc_enabled Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 54/70] Input: elantech - mark protocols v2 and v3 as semi-mt Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 56/70] iommu/vt-d: Fix 64-bit accesses to 32-bit DMAR_GSTS_REG Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 57/70] mm/memory_hotplug.c: check for missing sections in test_pages_in_a_zone() Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 58/70] xhci: Fix list corruption in urb dequeue at host removal Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 59/70] m32r: fix m32104ut_defconfig build fail Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 60/70] dma-debug: Fix dma_debug_entry offset calculation Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 61/70] dma-debug: switch check from _text to _stext Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 62/70] scripts/bloat-o-meter: fix python3 syntax error Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 63/70] memcg: only free spare array when readers are done Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 64/70] radix-tree: fix race in gang lookup Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 65/70] radix-tree: fix oops after radix_tree_iter_retry Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 66/70] dump_stack: avoid potential deadlocks Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 67/70] intel_scu_ipcutil: underflow in scu_reg_access() Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 68/70] futex: Drop refcount if requeue_pi() acquired the rtmutex Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 69/70] ip6mr: call del_timer_sync() in ip6mr_free_table() Greg Kroah-Hartman
2016-02-24  3:34 ` [PATCH 3.14 70/70] module: wrapper for symbol name Greg Kroah-Hartman
2016-02-24 18:28 ` [PATCH 3.14 00/70] 3.14.62-stable review Shuah Khan
2016-02-25  5:52 ` Guenter Roeck

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160224033355.108889347@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=James.Bottomley@HansenPartnership.com \
    --cc=herbszt@gmx.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.