From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org,
Nagaraj Arankal <nagaraj.p.arankal@hpe.com>,
Neal Cardwell <ncardwell@google.com>,
Yuchung Cheng <ycheng@google.com>,
Eric Dumazet <edumazet@google.com>,
Paolo Abeni <pabeni@redhat.com>, Sasha Levin <sashal@kernel.org>
Subject: [PATCH 5.10 67/79] tcp: fix early ETIMEDOUT after spurious non-SACK RTO
Date: Tue, 13 Sep 2022 16:05:12 +0200 [thread overview]
Message-ID: <20220913140353.418536205@linuxfoundation.org> (raw)
In-Reply-To: <20220913140350.291927556@linuxfoundation.org>
From: Neal Cardwell <ncardwell@google.com>
[ Upstream commit 686dc2db2a0fdc1d34b424ec2c0a735becd8d62b ]
Fix a bug reported and analyzed by Nagaraj Arankal, where the handling
of a spurious non-SACK RTO could cause a connection to fail to clear
retrans_stamp, causing a later RTO to very prematurely time out the
connection with ETIMEDOUT.
Here is the buggy scenario, expanding upon Nagaraj Arankal's excellent
report:
(*1) Send one data packet on a non-SACK connection
(*2) Because no ACK packet is received, the packet is retransmitted
and we enter CA_Loss; but this retransmission is spurious.
(*3) The ACK for the original data is received. The transmitted packet
is acknowledged. The TCP timestamp is before the retrans_stamp,
so tcp_may_undo() returns true, and tcp_try_undo_loss() returns
true without changing state to Open (because tcp_is_sack() is
false), and tcp_process_loss() returns without calling
tcp_try_undo_recovery(). Normally after undoing a CA_Loss
episode, tcp_fastretrans_alert() would see that the connection
has returned to CA_Open and fall through and call
tcp_try_to_open(), which would set retrans_stamp to 0. However,
for non-SACK connections we hold the connection in CA_Loss, so do
not fall through to call tcp_try_to_open() and do not set
retrans_stamp to 0. So retrans_stamp is (erroneously) still
non-zero.
At this point the first "retransmission event" has passed and
been recovered from. Any future retransmission is a completely
new "event". However, retrans_stamp is erroneously still
set. (And we are still in CA_Loss, which is correct.)
(*4) After 16 minutes (to correspond with tcp_retries2=15), a new data
packet is sent. Note: No data is transmitted between (*3) and
(*4) and we disabled keep alives.
The socket's timeout SHOULD be calculated from this point in
time, but instead it's calculated from the prior "event" 16
minutes ago (step (*2)).
(*5) Because no ACK packet is received, the packet is retransmitted.
(*6) At the time of the 2nd retransmission, the socket returns
ETIMEDOUT, prematurely, because retrans_stamp is (erroneously)
too far in the past (set at the time of (*2)).
This commit fixes this bug by ensuring that we reuse in
tcp_try_undo_loss() the same careful logic for non-SACK connections
that we have in tcp_try_undo_recovery(). To avoid duplicating logic,
we factor out that logic into a new
tcp_is_non_sack_preventing_reopen() helper and call that helper from
both undo functions.
Fixes: da34ac7626b5 ("tcp: only undo on partial ACKs in CA_Loss")
Reported-by: Nagaraj Arankal <nagaraj.p.arankal@hpe.com>
Link: https://lore.kernel.org/all/SJ0PR84MB1847BE6C24D274C46A1B9B0EB27A9@SJ0PR84MB1847.NAMPRD84.PROD.OUTLOOK.COM/
Signed-off-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Yuchung Cheng <ycheng@google.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Link: https://lore.kernel.org/r/20220903121023.866900-1-ncardwell.kernel@gmail.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
net/ipv4/tcp_input.c | 25 ++++++++++++++++++-------
1 file changed, 18 insertions(+), 7 deletions(-)
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index e62500d6fe0d0..4ecd85b1e806c 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -2496,6 +2496,21 @@ static inline bool tcp_may_undo(const struct tcp_sock *tp)
return tp->undo_marker && (!tp->undo_retrans || tcp_packet_delayed(tp));
}
+static bool tcp_is_non_sack_preventing_reopen(struct sock *sk)
+{
+ struct tcp_sock *tp = tcp_sk(sk);
+
+ if (tp->snd_una == tp->high_seq && tcp_is_reno(tp)) {
+ /* Hold old state until something *above* high_seq
+ * is ACKed. For Reno it is MUST to prevent false
+ * fast retransmits (RFC2582). SACK TCP is safe. */
+ if (!tcp_any_retrans_done(sk))
+ tp->retrans_stamp = 0;
+ return true;
+ }
+ return false;
+}
+
/* People celebrate: "We love our President!" */
static bool tcp_try_undo_recovery(struct sock *sk)
{
@@ -2518,14 +2533,8 @@ static bool tcp_try_undo_recovery(struct sock *sk)
} else if (tp->rack.reo_wnd_persist) {
tp->rack.reo_wnd_persist--;
}
- if (tp->snd_una == tp->high_seq && tcp_is_reno(tp)) {
- /* Hold old state until something *above* high_seq
- * is ACKed. For Reno it is MUST to prevent false
- * fast retransmits (RFC2582). SACK TCP is safe. */
- if (!tcp_any_retrans_done(sk))
- tp->retrans_stamp = 0;
+ if (tcp_is_non_sack_preventing_reopen(sk))
return true;
- }
tcp_set_ca_state(sk, TCP_CA_Open);
tp->is_sack_reneg = 0;
return false;
@@ -2561,6 +2570,8 @@ static bool tcp_try_undo_loss(struct sock *sk, bool frto_undo)
NET_INC_STATS(sock_net(sk),
LINUX_MIB_TCPSPURIOUSRTOS);
inet_csk(sk)->icsk_retransmits = 0;
+ if (tcp_is_non_sack_preventing_reopen(sk))
+ return true;
if (frto_undo || tcp_is_sack(tp)) {
tcp_set_ca_state(sk, TCP_CA_Open);
tp->is_sack_reneg = 0;
--
2.35.1
next prev parent reply other threads:[~2022-09-13 14:47 UTC|newest]
Thread overview: 89+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-13 14:04 [PATCH 5.10 00/79] 5.10.143-rc1 review Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 01/79] NFSD: Fix verifier returned in stable WRITEs Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 02/79] xen-blkfront: Cache feature_persistent value before advertisement Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 03/79] tty: n_gsm: initialize more members at gsm_alloc_mux() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 04/79] tty: n_gsm: avoid call of sleeping functions from atomic context Greg Kroah-Hartman
2022-09-14 12:38 ` Pavel Machek
2022-09-13 14:04 ` [PATCH 5.10 05/79] efi: libstub: Disable struct randomization Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 06/79] efi: capsule-loader: Fix use-after-free in efi_capsule_write Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 07/79] wifi: iwlegacy: 4965: corrected fix for potential off-by-one overflow in il4965_rs_fill_link_cmd() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 08/79] net: mvpp2: debugfs: fix memory leak when using debugfs_lookup() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 09/79] fs: only do a memory barrier for the first set_buffer_uptodate() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 10/79] Revert "mm: kmemleak: take a full lowmem check in kmemleak_*_phys()" Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 11/79] scsi: qla2xxx: Disable ATIO interrupt coalesce for quad port ISP27XX Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 12/79] scsi: megaraid_sas: Fix double kfree() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 13/79] drm/gem: Fix GEM handle release errors Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 14/79] drm/amdgpu: Move psp_xgmi_terminate call from amdgpu_xgmi_remove_device to psp_hw_fini Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 15/79] drm/amdgpu: Check num_gfx_rings for gfx v9_0 rb setup Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 16/79] drm/radeon: add a force flush to delay work when radeon Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 17/79] parisc: ccio-dma: Handle kmalloc failure in ccio_init_resources() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 18/79] parisc: Add runtime check to prevent PA2.0 kernels on PA1.x machines Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 19/79] arm64: cacheinfo: Fix incorrect assignment of signed error value to unsigned fw_level Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 20/79] arm64/signal: Raise limit on stack frames Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 21/79] net/core/skbuff: Check the return value of skb_copy_bits() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 22/79] fbdev: chipsfb: Add missing pci_disable_device() in chipsfb_pci_init() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 23/79] drm/amdgpu: mmVM_L2_CNTL3 register not initialized correctly Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 24/79] ALSA: emu10k1: Fix out of bounds access in snd_emu10k1_pcm_channel_alloc() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 25/79] ALSA: aloop: Fix random zeros in capture data when using jiffies timer Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 26/79] ALSA: usb-audio: Fix an out-of-bounds bug in __snd_usb_parse_audio_interface() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 27/79] kprobes: Prohibit probes in gate area Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 28/79] debugfs: add debugfs_lookup_and_remove() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 29/79] nvmet: fix a use-after-free Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 30/79] drm/i915: Implement WaEdpLinkRateDataReload Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 31/79] scsi: mpt3sas: Fix use-after-free warning Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 32/79] scsi: lpfc: Add missing destroy_workqueue() in error path Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 33/79] cgroup: Elide write-locking threadgroup_rwsem when updating csses on an empty subtree Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 34/79] cgroup: Fix threadgroup_rwsem <-> cpus_read_lock() deadlock Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 35/79] cifs: remove useless parameter is_fsctl from SMB2_ioctl() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 36/79] smb3: missing inode locks in punch hole Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 37/79] ARM: dts: imx6qdl-kontron-samx6i: remove duplicated node Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 38/79] regulator: core: Clean up on enable failure Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 39/79] tee: fix compiler warning in tee_shm_register() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 40/79] RDMA/cma: Fix arguments order in net device validation Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 41/79] soc: brcmstb: pm-arm: Fix refcount leak and __iomem leak bugs Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 42/79] RDMA/hns: Fix supported page size Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 43/79] RDMA/hns: Fix wrong fixed value of qp->rq.wqe_shift Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 44/79] ARM: dts: at91: sama5d27_wlsom1: specify proper regulator output ranges Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 45/79] ARM: dts: at91: sama5d2_icp: " Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 46/79] ARM: dts: at91: sama5d27_wlsom1: dont keep ldo2 enabled all the time Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 47/79] ARM: dts: at91: sama5d2_icp: dont keep vdd_other " Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 48/79] netfilter: br_netfilter: Drop dst references before setting Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 49/79] netfilter: nf_tables: clean up hook list when offload flags check fails Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 50/79] netfilter: nf_conntrack_irc: Fix forged IP logic Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 51/79] ALSA: usb-audio: Inform the delayed registration more properly Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 52/79] ALSA: usb-audio: Register card again for iface over delayed_register option Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 53/79] rxrpc: Fix an insufficiently large sglist in rxkad_verify_packet_2() Greg Kroah-Hartman
2022-09-13 14:04 ` [PATCH 5.10 54/79] afs: Use the operation issue time instead of the reply time for callbacks Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 55/79] sch_sfb: Dont assume the skb is still around after enqueueing to child Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 56/79] tipc: fix shift wrapping bug in map_get() Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 57/79] ice: use bitmap_free instead of devm_kfree Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 58/79] i40e: Fix kernel crash during module removal Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 59/79] net: fec: Use a spinlock to guard `fep->ptp_clk_on` Greg Kroah-Hartman
2022-09-13 15:57 ` Marc Kleine-Budde
2022-09-13 14:05 ` [PATCH 5.10 60/79] xen-netback: only remove hotplug-status when the vif is actually destroyed Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 61/79] RDMA/siw: Pass a pointer to virt_to_page() Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 62/79] ipv6: sr: fix out-of-bounds read when setting HMAC data Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 63/79] IB/core: Fix a nested dead lock as part of ODP flow Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 64/79] RDMA/mlx5: Set local port to one when accessing counters Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 65/79] nvme-tcp: fix UAF when detecting digest errors Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 66/79] nvme-tcp: fix regression that causes sporadic requests to time out Greg Kroah-Hartman
2022-09-13 14:05 ` Greg Kroah-Hartman [this message]
2022-09-13 14:05 ` [PATCH 5.10 68/79] sch_sfb: Also store skb len before calling child enqueue Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 69/79] ASoC: mchp-spdiftx: remove references to mchp_i2s_caps Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 70/79] ASoC: mchp-spdiftx: Fix clang -Wbitfield-constant-conversion Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 71/79] MIPS: loongson32: ls1c: Fix hang during startup Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 72/79] swiotlb: avoid potential left shift overflow Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 73/79] iommu/amd: use full 64-bit value in build_completion_wait() Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 74/79] hwmon: (mr75203) fix VM sensor allocation when "intel,vm-map" not defined Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 75/79] hwmon: (mr75203) update pvt->v_num and vm_num to the actual number of used sensors Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 76/79] hwmon: (mr75203) fix voltage equation for negative source input Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 77/79] hwmon: (mr75203) fix multi-channel voltage reading Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 78/79] hwmon: (mr75203) enable polling for all VM channels Greg Kroah-Hartman
2022-09-13 14:05 ` [PATCH 5.10 79/79] arm64: errata: add detection for AMEVCNTR01 incrementing incorrectly Greg Kroah-Hartman
2022-09-14 9:38 ` [PATCH 5.10 00/79] 5.10.143-rc1 review Sudip Mukherjee
2022-09-14 9:51 ` Pavel Machek
2022-09-14 11:04 ` Naresh Kamboju
2022-09-14 20:58 ` Florian Fainelli
2022-09-15 0:13 ` Guenter Roeck
2022-09-15 7:28 ` Rudi Heitbaum
2022-09-17 3:18 ` zhouzhixiu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220913140353.418536205@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=edumazet@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=nagaraj.p.arankal@hpe.com \
--cc=ncardwell@google.com \
--cc=pabeni@redhat.com \
--cc=sashal@kernel.org \
--cc=stable@vger.kernel.org \
--cc=ycheng@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox