Netdev List
 help / color / mirror / Atom feed
From: "Asbjørn Sloth Tønnesen" <ast@fiberby.net>
To: Eric Dumazet <edumazet@google.com>,
	Neal Cardwell <ncardwell@google.com>,
	Kuniyuki Iwashima <kuniyu@google.com>
Cc: "Asbjørn Sloth Tønnesen" <ast@fiberby.net>,
	"David S. Miller" <davem@davemloft.net>,
	"Jakub Kicinski" <kuba@kernel.org>,
	"Paolo Abeni" <pabeni@redhat.com>,
	"Simon Horman" <horms@kernel.org>,
	"Al Viro" <viro@zeniv.linux.org.uk>,
	netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
	"Kristian Nielsen" <knielsen@knielsen-hq.org>,
	stable@vger.kernel.org
Subject: [PATCH net v2] tcp: reset late connection after listening socket close
Date: Mon, 10 Aug 2026 20:56:40 +0000	[thread overview]
Message-ID: <20260810205642.1611338-1-ast@fiberby.net> (raw)

In commit c82199061009 ("task_work: remove fifo ordering guarantee")
Eric removed the ordering guarantee, thereby changing it from a
guaranteed FIFO to currently LIFO ordering, in an effort to reduce
jitter.

This significantly increases the probability for a race to occur
between a TCP handshake and the closing of the listening socket.

In that case the client sees the connection as ESTABLISHED, however in
tcp_v{4,6}_syn_recv_sock() the call to __inet_inherit_port() returns
-ENOENT, and the new connection is dropped silently by put_and_exit.

A client may therefore hang indefinitely on a blocking read() if the
used data communication protocol is initiated by the server, like SMTP
and the reporter[1]'s MariaDB protocol both are.

Had the new connection been processed before the listening socket was
closed, it would have been in the accept queue, and been notified when
inet_csk_listen_stop() was run, and the client would have got a reset.

The call to __inet_inherit_port() returns -ENOENT because
inet_csk(sk)->icsk_bind_hash is NULL, after inet_put_port() has been
called by tcp_set_state(sk, TCP_CLOSE).

This patch adds a check on the return value of the __inet_inherit_port()
call, and jumps to a new label, where it resets the new connection,
before proceeding with the put_and_exit label.

The blamed commit was identified by testing on ancient Debian stable
releases to get a rough scope of where to look, then identifying in
which release between v3.16 and v4.19 it broke, and finally bisecting
v4.2..v4.3 on a fresh Debian then-stable (jessie) VM with tooling from
that era, and confirmed by reverting it from v4.3, v5.10.y and net.

Additionally the race can be reproduced back to v3.6, by backporting
the blamed commit. Beyond v3.6 there are too many conflicts.

Reproducer:
  https://files.fiberby.net/ast/2026/kernel/socket_teardown_test.c

Reported-by: Kristian Nielsen <knielsen@knielsen-hq.org>
Link: https://lore.kernel.org/87sf0ldk41.fsf@urd.knielsen-hq.org # [1]
Fixes: c82199061009 ("task_work: remove fifo ordering guarantee")
Cc: <stable@vger.kernel.org>
Signed-off-by: Asbjørn Sloth Tønnesen <ast@fiberby.net>
---

While the blamed commit properly only made the race observable, then
the commit that made the race possible is less realistic to track down,
and I don't think is worth the time to continue to the search for it
somewhere before v3.6.

I'm not submitting a selftest at this time, as I have only found an
efficient way to often detect the issue, not disprove it, and I would
need more test data from different systems, before I can reliably
disprove it with a low runtime budget, without getting false negatives.

Changelog:
v2:
  - Use return from __inet_inherit_port() to trigger send_reply()
  - Use req->rsk_ops->send_reset.
  - Clarity commit message, and update to reflect the changes.
  (Thanks Kuniyuki)
v1: https://lore.kernel.org/20260807194513.1263310-1-ast@fiberby.net

 net/ipv4/tcp_ipv4.c | 8 +++++++-
 net/ipv6/tcp_ipv6.c | 8 +++++++-
 2 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index b8887cdd66c5..e0fba4579a57 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1690,6 +1690,7 @@ struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
 	int l3index;
 #endif
 	struct ip_options_rcu *inet_opt;
+	int ret;
 
 	if (sk_acceptq_is_full(sk))
 		goto exit_overflow;
@@ -1756,7 +1757,10 @@ struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
 		goto put_and_exit; /* OOM, release back memory */
 #endif
 
-	if (__inet_inherit_port(sk, newsk) < 0)
+	ret = __inet_inherit_port(sk, newsk);
+	if (ret == -ENOENT)
+		goto send_reset_and_exit;
+	else if (ret < 0)
 		goto put_and_exit;
 	*own_req = inet_ehash_nolisten(newsk, req_to_sk(req_unhash),
 				       &found_dup_sk);
@@ -1784,6 +1788,8 @@ struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
 exit:
 	tcp_listendrop(sk);
 	return NULL;
+send_reset_and_exit:
+	req->rsk_ops->send_reset(newsk, skb, SK_RST_REASON_TCP_STATE);
 put_and_exit:
 	newinet->inet_opt = NULL;
 	inet_csk_prepare_forced_close(newsk);
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 9e9155b1b3aa..021c93de2273 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -1400,6 +1400,7 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *
 	int l3index;
 #endif
 	struct flowi6 fl6;
+	int ret;
 
 	if (skb->protocol == htons(ETH_P_IP))
 		return tcp_v4_syn_recv_sock(sk, skb, req, dst,
@@ -1512,7 +1513,10 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *
 		goto put_and_exit; /* OOM */
 #endif
 
-	if (__inet_inherit_port(sk, newsk) < 0)
+	ret = __inet_inherit_port(sk, newsk);
+	if (ret == -ENOENT)
+		goto send_reset_and_exit;
+	else if (ret < 0)
 		goto put_and_exit;
 	*own_req = inet_ehash_nolisten(newsk, req_to_sk(req_unhash),
 				       &found_dup_sk);
@@ -1547,6 +1551,8 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *
 exit:
 	tcp_listendrop(sk);
 	return NULL;
+send_reset_and_exit:
+	req->rsk_ops->send_reset(newsk, skb, SK_RST_REASON_TCP_STATE);
 put_and_exit:
 	inet_csk_prepare_forced_close(newsk);
 	tcp_done(newsk);

base-commit: dd057113ac7ba5bdd2aed3d9405305911152f911
-- 
2.55.0


             reply	other threads:[~2026-08-10 20:57 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-10 20:56 Asbjørn Sloth Tønnesen [this message]
2026-08-11  6:37 ` [PATCH net v2] tcp: reset late connection after listening socket close Kuniyuki Iwashima
2026-08-11  7:44   ` Asbjørn Sloth Tønnesen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260810205642.1611338-1-ast@fiberby.net \
    --to=ast@fiberby.net \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=horms@kernel.org \
    --cc=knielsen@knielsen-hq.org \
    --cc=kuba@kernel.org \
    --cc=kuniyu@google.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=ncardwell@google.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=stable@vger.kernel.org \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox