From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: "Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, "Samuel Cabrero" <scabrero@suse.de>,
"Aurélien Aptel" <aaptel@suse.de>,
"Ronnie Sahlberg" <lsahlber@redhat.com>,
"Steve French" <smfrench@gmail.com>,
"Sasha Levin" <sashal@kernel.org>
Subject: [PATCH 4.4 41/86] cifs: Check for timeout on Negotiate stage
Date: Mon, 18 May 2020 19:36:12 +0200 [thread overview]
Message-ID: <20200518173458.701109514@linuxfoundation.org> (raw)
In-Reply-To: <20200518173450.254571947@linuxfoundation.org>
From: Samuel Cabrero <scabrero@suse.de>
[ Upstream commit 76e752701a8af4404bbd9c45723f7cbd6e4a251e ]
Some servers seem to accept connections while booting but never send
the SMBNegotiate response neither close the connection, causing all
processes accessing the share hang on uninterruptible sleep state.
This happens when the cifs_demultiplex_thread detects the server is
unresponsive so releases the socket and start trying to reconnect.
At some point, the faulty server will accept the socket and the TCP
status will be set to NeedNegotiate. The first issued command accessing
the share will start the negotiation (pid 5828 below), but the response
will never arrive so other commands will be blocked waiting on the mutex
(pid 55352).
This patch checks for unresponsive servers also on the negotiate stage
releasing the socket and reconnecting if the response is not received
and checking again the tcp state when the mutex is acquired.
PID: 55352 TASK: ffff880fd6cc02c0 CPU: 0 COMMAND: "ls"
#0 [ffff880fd9add9f0] schedule at ffffffff81467eb9
#1 [ffff880fd9addb38] __mutex_lock_slowpath at ffffffff81468fe0
#2 [ffff880fd9addba8] mutex_lock at ffffffff81468b1a
#3 [ffff880fd9addbc0] cifs_reconnect_tcon at ffffffffa042f905 [cifs]
#4 [ffff880fd9addc60] smb_init at ffffffffa042faeb [cifs]
#5 [ffff880fd9addca0] CIFSSMBQPathInfo at ffffffffa04360b5 [cifs]
....
Which is waiting a mutex owned by:
PID: 5828 TASK: ffff880fcc55e400 CPU: 0 COMMAND: "xxxx"
#0 [ffff880fbfdc19b8] schedule at ffffffff81467eb9
#1 [ffff880fbfdc1b00] wait_for_response at ffffffffa044f96d [cifs]
#2 [ffff880fbfdc1b60] SendReceive at ffffffffa04505ce [cifs]
#3 [ffff880fbfdc1bb0] CIFSSMBNegotiate at ffffffffa0438d79 [cifs]
#4 [ffff880fbfdc1c50] cifs_negotiate_protocol at ffffffffa043b383 [cifs]
#5 [ffff880fbfdc1c80] cifs_reconnect_tcon at ffffffffa042f911 [cifs]
#6 [ffff880fbfdc1d20] smb_init at ffffffffa042faeb [cifs]
#7 [ffff880fbfdc1d60] CIFSSMBQFSInfo at ffffffffa0434eb0 [cifs]
....
Signed-off-by: Samuel Cabrero <scabrero@suse.de>
Reviewed-by: Aurélien Aptel <aaptel@suse.de>
Reviewed-by: Ronnie Sahlberg <lsahlber@redhat.com>
Signed-off-by: Steve French <smfrench@gmail.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
fs/cifs/cifssmb.c | 12 ++++++++++++
fs/cifs/connect.c | 3 ++-
fs/cifs/smb2pdu.c | 12 ++++++++++++
3 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/fs/cifs/cifssmb.c b/fs/cifs/cifssmb.c
index b9b8f19dce0e1..fa07f7cb85a51 100644
--- a/fs/cifs/cifssmb.c
+++ b/fs/cifs/cifssmb.c
@@ -184,6 +184,18 @@ cifs_reconnect_tcon(struct cifs_tcon *tcon, int smb_command)
* reconnect the same SMB session
*/
mutex_lock(&ses->session_mutex);
+
+ /*
+ * Recheck after acquire mutex. If another thread is negotiating
+ * and the server never sends an answer the socket will be closed
+ * and tcpStatus set to reconnect.
+ */
+ if (server->tcpStatus == CifsNeedReconnect) {
+ rc = -EHOSTDOWN;
+ mutex_unlock(&ses->session_mutex);
+ goto out;
+ }
+
rc = cifs_negotiate_protocol(0, ses);
if (rc == 0 && ses->need_reconnect)
rc = cifs_setup_session(0, ses, nls_codepage);
diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index c9793ce0d3368..7022750cae2fd 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -558,7 +558,8 @@ server_unresponsive(struct TCP_Server_Info *server)
* 65s kernel_recvmsg times out, and we see that we haven't gotten
* a response in >60s.
*/
- if (server->tcpStatus == CifsGood &&
+ if ((server->tcpStatus == CifsGood ||
+ server->tcpStatus == CifsNeedNegotiate) &&
time_after(jiffies, server->lstrp + 2 * SMB_ECHO_INTERVAL)) {
cifs_dbg(VFS, "Server %s has not responded in %d seconds. Reconnecting...\n",
server->hostname, (2 * SMB_ECHO_INTERVAL) / HZ);
diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
index d4472a4947581..4ffd5e177288e 100644
--- a/fs/cifs/smb2pdu.c
+++ b/fs/cifs/smb2pdu.c
@@ -249,6 +249,18 @@ smb2_reconnect(__le16 smb2_command, struct cifs_tcon *tcon)
* the same SMB session
*/
mutex_lock(&tcon->ses->session_mutex);
+
+ /*
+ * Recheck after acquire mutex. If another thread is negotiating
+ * and the server never sends an answer the socket will be closed
+ * and tcpStatus set to reconnect.
+ */
+ if (server->tcpStatus == CifsNeedReconnect) {
+ rc = -EHOSTDOWN;
+ mutex_unlock(&tcon->ses->session_mutex);
+ goto out;
+ }
+
rc = cifs_negotiate_protocol(0, tcon->ses);
if (!rc && tcon->ses->need_reconnect) {
rc = cifs_setup_session(0, tcon->ses, nls_codepage);
--
2.20.1
next prev parent reply other threads:[~2020-05-18 17:39 UTC|newest]
Thread overview: 95+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-18 17:35 [PATCH 4.4 00/86] 4.4.224-rc1 review Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 01/86] USB: serial: qcserial: Add DW5816e support Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 02/86] Revert "net: phy: Avoid polling PHY with PHY_IGNORE_INTERRUPTS" Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 03/86] dp83640: reverse arguments to list_add_tail Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 04/86] net/mlx4_core: Fix use of ENOSPC around mlx4_counter_alloc() Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 05/86] sch_sfq: validate silly quantum values Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 06/86] sch_choke: avoid potential panic in choke_reset() Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 07/86] Revert "ACPI / video: Add force_native quirk for HP Pavilion dv6" Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 08/86] enic: do not overwrite error code Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 09/86] ipv6: fix cleanup ordering for ip6_mr failure Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 10/86] binfmt_elf: move brk out of mmap when doing direct loader exec Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 11/86] x86/apm: Dont access __preempt_count with zeroed fs Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 12/86] Revert "IB/ipoib: Update broadcast object if PKey value was changed in index 0" Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 13/86] USB: uas: add quirk for LaCie 2Big Quadra Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 14/86] USB: serial: garmin_gps: add sanity checking for data length Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 15/86] batman-adv: fix batadv_nc_random_weight_tq Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 16/86] scripts/decodecode: fix trapping instruction formatting Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 17/86] phy: micrel: Disable auto negotiation on startup Greg Kroah-Hartman
2020-05-19 5:45 ` Henri Rosten
2020-05-19 10:53 ` Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 18/86] phy: micrel: Ensure interrupts are reenabled on resume Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 19/86] binfmt_elf: Do not move brk for INTERP-less ET_EXEC Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 20/86] ext4: add cond_resched() to ext4_protect_reserved_inode Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 21/86] net: ipv6: add net argument to ip6_dst_lookup_flow Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 22/86] net: ipv6_stub: use ip6_dst_lookup_flow instead of ip6_dst_lookup Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 23/86] blktrace: Fix potential deadlock between delete & sysfs ops Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 24/86] blktrace: fix unlocked access to init/start-stop/teardown Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 25/86] blktrace: fix trace mutex deadlock Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 26/86] blktrace: Protect q->blk_trace with RCU Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 27/86] blktrace: fix dereference after null check Greg Kroah-Hartman
2020-05-18 17:35 ` [PATCH 4.4 28/86] ptp: do not explicitly set drvdata in ptp_clock_register() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 29/86] ptp: use is_visible method to hide unused attributes Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 30/86] ptp: create "pins" together with the rest of attributes Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 31/86] chardev: add helper function to register char devs with a struct device Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 32/86] ptp: Fix pass zero to ERR_PTR() in ptp_clock_register Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 33/86] ptp: fix the race between the release of ptp_clock and cdev Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 34/86] ptp: free ptp device pin descriptors properly Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 35/86] net: handle no dst on skb in icmp6_send Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 36/86] net/sonic: Fix a resource leak in an error handling path in jazz_sonic_probe() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 37/86] net: moxa: Fix a potential double free_irq() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 38/86] drop_monitor: work around gcc-10 stringop-overflow warning Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 39/86] scsi: sg: add sg_remove_request in sg_write Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 40/86] spi: spi-dw: Add lock protect dw_spi rx/tx to prevent concurrent calls Greg Kroah-Hartman
2020-05-18 17:36 ` Greg Kroah-Hartman [this message]
2020-05-18 17:36 ` [PATCH 4.4 42/86] cifs: Fix a race condition with cifs_echo_request Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 43/86] dmaengine: pch_dma.c: Avoid data race between probe and irq handler Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 44/86] dmaengine: mmp_tdma: Reset channel error on release Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 45/86] drm/qxl: lost qxl_bo_kunmap_atomic_page in qxl_image_init_helper() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 46/86] ipc/util.c: sysvipc_find_ipc() incorrectly updates position index Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 47/86] net: openvswitch: fix csum updates for MPLS actions Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 48/86] gre: do not keep the GRE header around in collect medata mode Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 49/86] mm/memory_hotplug.c: fix overflow in test_pages_in_a_zone() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 50/86] scsi: qla2xxx: Avoid double completion of abort command Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 51/86] i40e: avoid NVM acquire deadlock during NVM update Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 52/86] net/mlx5: Fix driver load error flow when firmware is stuck Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 53/86] netfilter: conntrack: avoid gcc-10 zero-length-bounds warning Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 54/86] IB/mlx4: Test return value of calls to ib_get_cached_pkey Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 55/86] pnp: Use list_for_each_entry() instead of open coding Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 56/86] gcc-10 warnings: fix low-hanging fruit Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 57/86] kbuild: compute false-positive -Wmaybe-uninitialized cases in Kconfig Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 58/86] Stop the ad-hoc games with -Wno-maybe-initialized Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 59/86] gcc-10: disable zero-length-bounds warning for now Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 60/86] gcc-10: disable array-bounds " Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 61/86] gcc-10: disable stringop-overflow " Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 62/86] gcc-10: disable restrict " Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 63/86] block: defer timeouts to a workqueue Greg Kroah-Hartman
2020-05-19 6:00 ` Henri Rosten
2020-05-19 7:31 ` Greg Kroah-Hartman
2020-05-19 10:53 ` Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 64/86] blk-mq: Allow timeouts to run while queue is freezing Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 65/86] blk-mq: sync the update nr_hw_queues with blk_mq_queue_tag_busy_iter Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 66/86] blk-mq: Allow blocking queue tag iter callbacks Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 67/86] x86/paravirt: Remove the unused irq_enable_sysexit pv op Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 68/86] gcc-10: avoid shadowing standard library free() in crypto Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 69/86] net: fix a potential recursive NETDEV_FEAT_CHANGE Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 70/86] net: ipv4: really enforce backoff for redirects Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 71/86] netlabel: cope with NULL catmap Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 72/86] ALSA: hda/realtek - Limit int mic boost for Thinkpad T530 Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 73/86] ALSA: rawmidi: Fix racy buffer resize under concurrent accesses Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 74/86] ALSA: rawmidi: Initialize allocated buffers Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 75/86] USB: gadget: fix illegal array access in binding with UDC Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 76/86] ARM: dts: imx27-phytec-phycard-s-rdk: Fix the I2C1 pinctrl entries Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 77/86] x86: Fix early boot crash on gcc-10, third try Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 78/86] exec: Move would_dump into flush_old_exec Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 79/86] usb: gadget: net2272: Fix a memory leak in an error handling path in net2272_plat_probe() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 80/86] usb: gadget: audio: Fix a missing error return value in audio_bind() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 81/86] usb: gadget: legacy: fix error return code in gncm_bind() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 82/86] usb: gadget: legacy: fix error return code in cdc_bind() Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 83/86] Revert "ALSA: hda/realtek: Fix pop noise on ALC225" Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 84/86] ARM: dts: r8a7740: Add missing extal2 to CPG node Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 85/86] KVM: x86: Fix off-by-one error in kvm_vcpu_ioctl_x86_setup_mce Greg Kroah-Hartman
2020-05-18 17:36 ` [PATCH 4.4 86/86] Makefile: disallow data races on gcc-10 as well Greg Kroah-Hartman
2020-05-19 8:29 ` [PATCH 4.4 00/86] 4.4.224-rc1 review Naresh Kamboju
2020-05-19 8:49 ` Jon Hunter
2020-05-21 7:47 ` Chris Paterson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200518173458.701109514@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=aaptel@suse.de \
--cc=linux-kernel@vger.kernel.org \
--cc=lsahlber@redhat.com \
--cc=sashal@kernel.org \
--cc=scabrero@suse.de \
--cc=smfrench@gmail.com \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).