stable.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: "Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, "Samuel Cabrero" <scabrero@suse.de>,
	"Aurélien Aptel" <aaptel@suse.de>,
	"Ronnie Sahlberg" <lsahlber@redhat.com>,
	"Steve French" <smfrench@gmail.com>,
	"Sasha Levin" <sashal@kernel.org>
Subject: [PATCH 4.9 47/90] cifs: Check for timeout on Negotiate stage
Date: Mon, 18 May 2020 19:36:25 +0200	[thread overview]
Message-ID: <20200518173500.787954575@linuxfoundation.org> (raw)
In-Reply-To: <20200518173450.930655662@linuxfoundation.org>

From: Samuel Cabrero <scabrero@suse.de>

[ Upstream commit 76e752701a8af4404bbd9c45723f7cbd6e4a251e ]

Some servers seem to accept connections while booting but never send
the SMBNegotiate response neither close the connection, causing all
processes accessing the share hang on uninterruptible sleep state.

This happens when the cifs_demultiplex_thread detects the server is
unresponsive so releases the socket and start trying to reconnect.
At some point, the faulty server will accept the socket and the TCP
status will be set to NeedNegotiate. The first issued command accessing
the share will start the negotiation (pid 5828 below), but the response
will never arrive so other commands will be blocked waiting on the mutex
(pid 55352).

This patch checks for unresponsive servers also on the negotiate stage
releasing the socket and reconnecting if the response is not received
and checking again the tcp state when the mutex is acquired.

PID: 55352  TASK: ffff880fd6cc02c0  CPU: 0   COMMAND: "ls"
 #0 [ffff880fd9add9f0] schedule at ffffffff81467eb9
 #1 [ffff880fd9addb38] __mutex_lock_slowpath at ffffffff81468fe0
 #2 [ffff880fd9addba8] mutex_lock at ffffffff81468b1a
 #3 [ffff880fd9addbc0] cifs_reconnect_tcon at ffffffffa042f905 [cifs]
 #4 [ffff880fd9addc60] smb_init at ffffffffa042faeb [cifs]
 #5 [ffff880fd9addca0] CIFSSMBQPathInfo at ffffffffa04360b5 [cifs]
 ....

Which is waiting a mutex owned by:

PID: 5828   TASK: ffff880fcc55e400  CPU: 0   COMMAND: "xxxx"
 #0 [ffff880fbfdc19b8] schedule at ffffffff81467eb9
 #1 [ffff880fbfdc1b00] wait_for_response at ffffffffa044f96d [cifs]
 #2 [ffff880fbfdc1b60] SendReceive at ffffffffa04505ce [cifs]
 #3 [ffff880fbfdc1bb0] CIFSSMBNegotiate at ffffffffa0438d79 [cifs]
 #4 [ffff880fbfdc1c50] cifs_negotiate_protocol at ffffffffa043b383 [cifs]
 #5 [ffff880fbfdc1c80] cifs_reconnect_tcon at ffffffffa042f911 [cifs]
 #6 [ffff880fbfdc1d20] smb_init at ffffffffa042faeb [cifs]
 #7 [ffff880fbfdc1d60] CIFSSMBQFSInfo at ffffffffa0434eb0 [cifs]
 ....

Signed-off-by: Samuel Cabrero <scabrero@suse.de>
Reviewed-by: Aurélien Aptel <aaptel@suse.de>
Reviewed-by: Ronnie Sahlberg <lsahlber@redhat.com>
Signed-off-by: Steve French <smfrench@gmail.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 fs/cifs/cifssmb.c | 12 ++++++++++++
 fs/cifs/connect.c |  3 ++-
 fs/cifs/smb2pdu.c | 12 ++++++++++++
 3 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/fs/cifs/cifssmb.c b/fs/cifs/cifssmb.c
index 741b83c59a306..568abcd6d0dd3 100644
--- a/fs/cifs/cifssmb.c
+++ b/fs/cifs/cifssmb.c
@@ -184,6 +184,18 @@ cifs_reconnect_tcon(struct cifs_tcon *tcon, int smb_command)
 	 * reconnect the same SMB session
 	 */
 	mutex_lock(&ses->session_mutex);
+
+	/*
+	 * Recheck after acquire mutex. If another thread is negotiating
+	 * and the server never sends an answer the socket will be closed
+	 * and tcpStatus set to reconnect.
+	 */
+	if (server->tcpStatus == CifsNeedReconnect) {
+		rc = -EHOSTDOWN;
+		mutex_unlock(&ses->session_mutex);
+		goto out;
+	}
+
 	rc = cifs_negotiate_protocol(0, ses);
 	if (rc == 0 && ses->need_reconnect)
 		rc = cifs_setup_session(0, ses, nls_codepage);
diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index c018d161735c4..37c8cac86431f 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -561,7 +561,8 @@ server_unresponsive(struct TCP_Server_Info *server)
 	 * 65s kernel_recvmsg times out, and we see that we haven't gotten
 	 *     a response in >60s.
 	 */
-	if (server->tcpStatus == CifsGood &&
+	if ((server->tcpStatus == CifsGood ||
+	    server->tcpStatus == CifsNeedNegotiate) &&
 	    time_after(jiffies, server->lstrp + 2 * server->echo_interval)) {
 		cifs_dbg(VFS, "Server %s has not responded in %lu seconds. Reconnecting...\n",
 			 server->hostname, (2 * server->echo_interval) / HZ);
diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
index e8dc28dbe563e..0a23b6002ff12 100644
--- a/fs/cifs/smb2pdu.c
+++ b/fs/cifs/smb2pdu.c
@@ -246,6 +246,18 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon)
 	 * the same SMB session
 	 */
 	mutex_lock(&tcon->ses->session_mutex);
+
+	/*
+	 * Recheck after acquire mutex. If another thread is negotiating
+	 * and the server never sends an answer the socket will be closed
+	 * and tcpStatus set to reconnect.
+	 */
+	if (server->tcpStatus == CifsNeedReconnect) {
+		rc = -EHOSTDOWN;
+		mutex_unlock(&tcon->ses->session_mutex);
+		goto out;
+	}
+
 	rc = cifs_negotiate_protocol(0, tcon->ses);
 	if (!rc && tcon->ses->need_reconnect) {
 		rc = cifs_setup_session(0, tcon->ses, nls_codepage);
-- 
2.20.1




  parent reply	other threads:[~2020-05-18 18:27 UTC|newest]

Thread overview: 95+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-18 17:35 [PATCH 4.9 00/90] 4.9.224-rc1 review Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 01/90] USB: serial: qcserial: Add DW5816e support Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 02/90] dp83640: reverse arguments to list_add_tail Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 03/90] fq_codel: fix TCA_FQ_CODEL_DROP_BATCH_SIZE sanity checks Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 04/90] net: macsec: preserve ingress frame ordering Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 05/90] net/mlx4_core: Fix use of ENOSPC around mlx4_counter_alloc() Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 06/90] net: usb: qmi_wwan: add support for DW5816e Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 07/90] sch_choke: avoid potential panic in choke_reset() Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 08/90] sch_sfq: validate silly quantum values Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 09/90] bnxt_en: Fix VLAN acceleration handling in bnxt_fix_features() Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 10/90] net/mlx5: Fix forced completion access non initialized command entry Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 11/90] net/mlx5: Fix command entry leak in Internal Error State Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 12/90] bnxt_en: Improve AER slot reset Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 13/90] Revert "ACPI / video: Add force_native quirk for HP Pavilion dv6" Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 14/90] binfmt_elf: move brk out of mmap when doing direct loader exec Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 15/90] USB: uas: add quirk for LaCie 2Big Quadra Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 16/90] USB: serial: garmin_gps: add sanity checking for data length Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 17/90] tracing: Add a vmalloc_sync_mappings() for safe measure Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 18/90] mm/page_alloc: fix watchdog soft lockups during set_zone_contiguous() Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 19/90] batman-adv: fix batadv_nc_random_weight_tq Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 20/90] batman-adv: Fix refcnt leak in batadv_show_throughput_override Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.9 21/90] batman-adv: Fix refcnt leak in batadv_store_throughput_override Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 22/90] batman-adv: Fix refcnt leak in batadv_v_ogm_process Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 23/90] objtool: Fix stack offset tracking for indirect CFAs Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 24/90] scripts/decodecode: fix trapping instruction formatting Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 25/90] binfmt_elf: Do not move brk for INTERP-less ET_EXEC Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 26/90] ext4: add cond_resched() to ext4_protect_reserved_inode Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 27/90] net: ipv6: add net argument to ip6_dst_lookup_flow Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 28/90] net: ipv6_stub: use ip6_dst_lookup_flow instead of ip6_dst_lookup Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 29/90] blktrace: Fix potential deadlock between delete & sysfs ops Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 30/90] blktrace: fix unlocked access to init/start-stop/teardown Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 31/90] blktrace: fix trace mutex deadlock Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 32/90] blktrace: Protect q->blk_trace with RCU Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 33/90] blktrace: fix dereference after null check Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 34/90] ptp: do not explicitly set drvdata in ptp_clock_register() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 35/90] ptp: use is_visible method to hide unused attributes Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 36/90] ptp: create "pins" together with the rest of attributes Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 37/90] chardev: add helper function to register char devs with a struct device Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 38/90] ptp: Fix pass zero to ERR_PTR() in ptp_clock_register Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 39/90] ptp: fix the race between the release of ptp_clock and cdev Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 40/90] ptp: free ptp device pin descriptors properly Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 41/90] shmem: fix possible deadlocks on shmlock_user_lock Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 42/90] net/sonic: Fix a resource leak in an error handling path in jazz_sonic_probe() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 43/90] net: moxa: Fix a potential double free_irq() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 44/90] drop_monitor: work around gcc-10 stringop-overflow warning Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 45/90] scsi: sg: add sg_remove_request in sg_write Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 46/90] spi: spi-dw: Add lock protect dw_spi rx/tx to prevent concurrent calls Greg Kroah-Hartman
2020-05-18 17:36 ` Greg Kroah-Hartman [this message]
2020-05-18 17:36 ` [PATCH 4.9 48/90] cifs: Fix a race condition with cifs_echo_request Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 49/90] dmaengine: pch_dma.c: Avoid data race between probe and irq handler Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 50/90] dmaengine: mmp_tdma: Reset channel error on release Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 51/90] ALSA: hda/hdmi: fix race in monitor detection during probe Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 52/90] drm/qxl: lost qxl_bo_kunmap_atomic_page in qxl_image_init_helper() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 53/90] ipc/util.c: sysvipc_find_ipc() incorrectly updates position index Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 54/90] pinctrl: cherryview: Add missing spinlock usage in chv_gpio_irq_handler Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 55/90] i40iw: Fix error handling in i40iw_manage_arp_cache() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 56/90] netfilter: conntrack: avoid gcc-10 zero-length-bounds warning Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 57/90] IB/mlx4: Test return value of calls to ib_get_cached_pkey Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 58/90] pnp: Use list_for_each_entry() instead of open coding Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 59/90] gcc-10 warnings: fix low-hanging fruit Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 60/90] kbuild: compute false-positive -Wmaybe-uninitialized cases in Kconfig Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 61/90] Stop the ad-hoc games with -Wno-maybe-initialized Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 62/90] net: phy: micrel: Use strlcpy() for ethtool::get_strings Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 63/90] gcc-10: avoid shadowing standard library free() in crypto Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 64/90] gcc-10: disable zero-length-bounds warning for now Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 65/90] gcc-10: disable array-bounds " Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 66/90] gcc-10: disable stringop-overflow " Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 67/90] gcc-10: disable restrict " Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 68/90] net: fix a potential recursive NETDEV_FEAT_CHANGE Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 69/90] netlabel: cope with NULL catmap Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 70/90] Revert "ipv6: add mtu lock check in __ip6_rt_update_pmtu" Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 71/90] net: ipv4: really enforce backoff for redirects Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 72/90] netprio_cgroup: Fix unlimited memory leak of v2 cgroups Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 73/90] ALSA: hda/realtek - Limit int mic boost for Thinkpad T530 Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 74/90] ALSA: rawmidi: Initialize allocated buffers Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 75/90] ALSA: rawmidi: Fix racy buffer resize under concurrent accesses Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 76/90] ALSA: usb-audio: Add control message quirk delay for Kingston HyperX headset Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 77/90] USB: gadget: fix illegal array access in binding with UDC Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 78/90] usb: xhci: Fix NULL pointer dereference when enqueuing trbs from urb sg list Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 79/90] ARM: dts: imx27-phytec-phycard-s-rdk: Fix the I2C1 pinctrl entries Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 80/90] x86: Fix early boot crash on gcc-10, third try Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.9 81/90] exec: Move would_dump into flush_old_exec Greg Kroah-Hartman
2020-05-18 17:37 ` [PATCH 4.9 82/90] usb: gadget: net2272: Fix a memory leak in an error handling path in net2272_plat_probe() Greg Kroah-Hartman
2020-05-18 17:37 ` [PATCH 4.9 83/90] usb: gadget: audio: Fix a missing error return value in audio_bind() Greg Kroah-Hartman
2020-05-18 17:37 ` [PATCH 4.9 84/90] usb: gadget: legacy: fix error return code in gncm_bind() Greg Kroah-Hartman
2020-05-18 17:37 ` [PATCH 4.9 85/90] usb: gadget: legacy: fix error return code in cdc_bind() Greg Kroah-Hartman
2020-05-18 17:37 ` [PATCH 4.9 86/90] Revert "ALSA: hda/realtek: Fix pop noise on ALC225" Greg Kroah-Hartman
2020-05-18 17:37 ` [PATCH 4.9 87/90] ARM: dts: r8a73a4: Add missing CMT1 interrupts Greg Kroah-Hartman
2020-05-18 17:37 ` [PATCH 4.9 88/90] ARM: dts: r8a7740: Add missing extal2 to CPG node Greg Kroah-Hartman
2020-05-18 17:37 ` [PATCH 4.9 89/90] KVM: x86: Fix off-by-one error in kvm_vcpu_ioctl_x86_setup_mce Greg Kroah-Hartman
2020-05-18 17:37 ` [PATCH 4.9 90/90] Makefile: disallow data races on gcc-10 as well Greg Kroah-Hartman
2020-05-19  8:19 ` [PATCH 4.9 00/90] 4.9.224-rc1 review Naresh Kamboju
2020-05-19  8:49 ` Jon Hunter
2020-05-19 15:08 ` shuah
2020-05-19 16:27 ` Guenter Roeck

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200518173500.787954575@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=aaptel@suse.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lsahlber@redhat.com \
    --cc=sashal@kernel.org \
    --cc=scabrero@suse.de \
    --cc=smfrench@gmail.com \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).