[PATCH net-next 1/2] tcp: don't set window_clamp from SO_RCVBUF before the socket is connected

[Shashank Mohan Jain <jain.sm@gmail.com>]

Since commit b025461303d8 ("tcp: update window_clamp when SO_RCVBUF is
set"), setsockopt(SO_RCVBUF) stores tp->window_clamp in every socket
state, including TCP_CLOSE and TCP_LISTEN.

tcp_connect_init() and tcp_openreq_init_rwin() treat a non-zero
window_clamp as a user clamp and, in that case, skip the route window
metric (RTAX_WINDOW, "ip route ... window N"). A clamp left behind by an
SO_RCVBUF call made before connect(), or on a listener, therefore
changes the window and window scale advertised in the SYN and SYN-ACK:
the route metric is no longer applied, and even without a route metric
the advertised window and scale come from SO_RCVBUF instead of the
defaults. This affects any application that sets SO_RCVBUF before
connect() or listen(), which is more common than using the route metric.

Example: two netns joined by a veth pair, "ip route ... window 4096" on
both sides, SO_RCVBUF set to 1 MiB before connect():

  before:  SYN win 65535, wscale 5
  after:   SYN win 2920,  wscale 0

wscale 0 means the window scale option is still sent, with a shift of 0.
The "after" values match a kernel with b025461303d8 reverted; with no
route metric the SYN window and scale are the defaults in both cases.

This only concerns the handshake. Since commit a2cbb1603943 ("tcp:
Update window clamping condition") tcp_measure_rcv_mss() recomputes
window_clamp from sk_rcvbuf once the scaling ratio changes, so
TCP_WINDOW_CLAMP does not persist past the first data segments, with or
without this patch.

Only update window_clamp from SO_RCVBUF once the socket is being set up:
skip it in TCP_CLOSE and TCP_LISTEN. When the connection is established
with SOCK_RCVBUF_LOCK set, both functions already clamp window_clamp to
tcp_full_space() (the value tcp_set_rcvbuf() computes), so nothing is
lost.

Fixes: b025461303d8 ("tcp: update window_clamp when SO_RCVBUF is set")
Assisted-by: LLM
Signed-off-by: Shashank Mohan Jain <jain.sm@gmail.com>
---
 net/ipv4/tcp.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index f4c9101e5..e523cb71b 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -1858,6 +1858,14 @@ int tcp_set_rcvlowat(struct sock *sk, int val)
 
 void tcp_set_rcvbuf(struct sock *sk, int val)
 {
+	/* Without a connection there is no clamp to update: it is chosen
+	 * from the route's window metric and the buffer size when the
+	 * connection is set up, see tcp_connect_init() and
+	 * tcp_openreq_init_rwin().
+	 */
+	if ((1 << sk->sk_state) & (TCPF_CLOSE | TCPF_LISTEN))
+		return;
+
 	tcp_set_window_clamp(sk, tcp_win_from_space(sk, val));
 }
 
-- 
2.43.0