Netdev List
 help / color / mirror / Atom feed
* [PATCH net] tcp: fix stale per-CPU tcp_tw_isn leak enabling ISN prediction
@ 2026-05-11 15:15 Eric Dumazet
  2026-05-11 20:23 ` Jakub Kicinski
  2026-05-12  6:18 ` [syzbot ci] " syzbot ci
  0 siblings, 2 replies; 3+ messages in thread
From: Eric Dumazet @ 2026-05-11 15:15 UTC (permalink / raw)
  To: David S . Miller, Jakub Kicinski, Paolo Abeni
  Cc: Simon Horman, Neal Cardwell, Kuniyuki Iwashima, netdev,
	eric.dumazet, Eric Dumazet

Blamed commit moved the TIME_WAIT-derived ISN from the skb control
block to a per-CPU variable, assuming the value would always be consumed
by tcp_conn_request() for the same packet that wrote it. That assumption
is violated by multiple drop paths between the producer
(__this_cpu_write(tcp_tw_isn, isn) in tcp_v{4,6}_rcv()) and the consumer
(tcp_conn_request()):

 - min_ttl / min_hopcount check
 - xfrm policy check
 - tcp_inbound_hash() MD5/AO mismatch
 - tcp_filter() eBPF/SO_ATTACH_FILTER drop
 - th->syn && th->fin discard in tcp_rcv_state_process() TCP_LISTEN
 - psp_sk_rx_policy_check() in tcp_v{4,6}_do_rcv()
 - tcp_checksum_complete() in tcp_v{4,6}_do_rcv()
 - tcp_v{4,6}_cookie_check() returning NULL

When a packet is dropped on any of these paths, tcp_tw_isn is left set.

The next SYN processed on the same CPU then consumes the non zero value in
tcp_conn_request(), receiving a predictable ISN.

We could fix this by clearing tcp_tw_isn at tcp_v{4,6}_do_rcv() start,
at the expense of slower fast path.

This patch clears tcp_tw_isn only when drops are happening.

It also adds two DEBUG_NET_WARN_ON_ONCE() to detect if any drop paths
are missed in the future, as they check the state of tcp_tw_isn at
the beginning of processing a packet that could potentially be a new SYN.

Fixes: 41eecbd712b7 ("tcp: replace TCP_SKB_CB(skb)->tcp_tw_isn with a per-cpu field")
Reported-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
 net/ipv4/tcp_ipv4.c | 3 +++
 net/ipv6/tcp_ipv6.c | 3 +++
 2 files changed, 6 insertions(+)

diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index c0526cc0398049fb34b5de20a1175d54942e80cd..529f10a768183f74b85fcfdbee94d35c88c55122 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1878,6 +1878,7 @@ int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb)
 reset:
 	tcp_v4_send_reset(sk, skb, sk_rst_convert_drop_reason(reason));
 discard:
+	__this_cpu_write(tcp_tw_isn, 0);
 	sk_skb_reason_drop(sk, skb, reason);
 	/* Be careful here. If this function gets more complicated and
 	 * gcc suffers from register pressure on the x86, sk (in %ebx)
@@ -2198,6 +2199,7 @@ int tcp_v4_rcv(struct sk_buff *skb)
 		}
 	}
 
+	DEBUG_NET_WARN_ON_ONCE(__this_cpu_read(tcp_tw_isn));
 process:
 	if (static_branch_unlikely(&ip4_min_ttl)) {
 		/* min_ttl can be changed concurrently from do_ip_setsockopt() */
@@ -2274,6 +2276,7 @@ int tcp_v4_rcv(struct sk_buff *skb)
 	}
 
 discard_it:
+	__this_cpu_write(tcp_tw_isn, 0);
 	SKB_DR_OR(drop_reason, NOT_SPECIFIED);
 	/* Discard frame. */
 	sk_skb_reason_drop(sk, skb, drop_reason);
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index d13d49bfef19457cc5902cb556605a80f4c0ab2c..079f2324a3a777cfe30dd59777c9d3d6d9a6bae0 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -1641,6 +1641,7 @@ int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb)
 reset:
 	tcp_v6_send_reset(sk, skb, sk_rst_convert_drop_reason(reason));
 discard:
+	__this_cpu_write(tcp_tw_isn, 0);
 	if (opt_skb)
 		__kfree_skb(opt_skb);
 	sk_skb_reason_drop(sk, skb, reason);
@@ -1839,6 +1840,7 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb)
 		}
 	}
 
+	DEBUG_NET_WARN_ON_ONCE(__this_cpu_read(tcp_tw_isn));
 process:
 	if (static_branch_unlikely(&ip6_min_hopcount)) {
 		/* min_hopcount can be changed concurrently from do_ipv6_setsockopt() */
@@ -1913,6 +1915,7 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb)
 	}
 
 discard_it:
+	__this_cpu_write(tcp_tw_isn, 0);
 	SKB_DR_OR(drop_reason, NOT_SPECIFIED);
 	sk_skb_reason_drop(sk, skb, drop_reason);
 	return 0;
-- 
2.54.0.563.g4f69b47b94-goog


^ permalink raw reply related	[flat|nested] 3+ messages in thread

* Re: [PATCH net] tcp: fix stale per-CPU tcp_tw_isn leak enabling ISN prediction
  2026-05-11 15:15 [PATCH net] tcp: fix stale per-CPU tcp_tw_isn leak enabling ISN prediction Eric Dumazet
@ 2026-05-11 20:23 ` Jakub Kicinski
  2026-05-12  6:18 ` [syzbot ci] " syzbot ci
  1 sibling, 0 replies; 3+ messages in thread
From: Jakub Kicinski @ 2026-05-11 20:23 UTC (permalink / raw)
  To: Eric Dumazet
  Cc: David S . Miller, Paolo Abeni, Simon Horman, Neal Cardwell,
	Kuniyuki Iwashima, netdev, eric.dumazet

On Mon, 11 May 2026 15:15:59 +0000 Eric Dumazet wrote:
> diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
> index d13d49bfef19457cc5902cb556605a80f4c0ab2c..079f2324a3a777cfe30dd59777c9d3d6d9a6bae0 100644
> --- a/net/ipv6/tcp_ipv6.c
> +++ b/net/ipv6/tcp_ipv6.c
> @@ -1641,6 +1641,7 @@ int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb)
>  reset:
>  	tcp_v6_send_reset(sk, skb, sk_rst_convert_drop_reason(reason));
>  discard:
> +	__this_cpu_write(tcp_tw_isn, 0);
>  	if (opt_skb)
>  		__kfree_skb(opt_skb);
>  	sk_skb_reason_drop(sk, skb, reason);
> @@ -1839,6 +1840,7 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb)
>  		}
>  	}
>  
> +	DEBUG_NET_WARN_ON_ONCE(__this_cpu_read(tcp_tw_isn));
>  process:
>  	if (static_branch_unlikely(&ip6_min_hopcount)) {
>  		/* min_hopcount can be changed concurrently from do_ipv6_setsockopt() */
> @@ -1913,6 +1915,7 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb)
>  	}
>  
>  discard_it:
> +	__this_cpu_write(tcp_tw_isn, 0);
>  	SKB_DR_OR(drop_reason, NOT_SPECIFIED);
>  	sk_skb_reason_drop(sk, skb, drop_reason);
>  	return 0;

CI says:

[  288.637769][ T3492] BUG: using __this_cpu_write() in preemptible [00000000] code: python3/3492
[  288.638116][ T3492] caller is tcp_v6_do_rcv+0x204/0x1cb0
[  288.638240][ T3492] CPU: 1 UID: 0 PID: 3492 Comm: python3 Not tainted 7.1.0-rc2-virtme #1 PREEMPT(full) 
[  288.638245][ T3492] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[  288.638247][ T3492] Call Trace:
[  288.638248][ T3492]  <TASK>
[  288.638250][ T3492]  dump_stack_lvl+0x6f/0xa0
[  288.638257][ T3492]  check_preemption_disabled+0xdb/0xf0
[  288.638261][ T3492]  tcp_v6_do_rcv+0x204/0x1cb0
[  288.638265][ T3492]  __release_sock+0x135/0x3a0
[  288.638272][ T3492]  release_sock+0x19c/0x240
[  288.638275][ T3492]  tcp_sendmsg+0x39/0x50
[  288.638279][ T3492]  __sys_sendto+0x2c9/0x400
[  288.638282][ T3492]  ? __ia32_sys_getpeername+0xd0/0xd0
[  288.638288][ T3492]  ? sock_ioctl+0x3cb/0x5f0
[  288.638294][ T3492]  ? find_held_lock+0x2b/0x80
[  288.638299][ T3492]  ? __lock_release.isra.0+0x6b/0x1a0
[  288.638302][ T3492]  __x64_sys_sendto+0xe4/0x1f0
[  288.638304][ T3492]  ? trace_irq_enable.constprop.0+0x9b/0x180
[  288.638308][ T3492]  ? lockdep_hardirqs_on+0x8c/0x130
[  288.638310][ T3492]  ? do_syscall_64+0x82/0xfc0
[  288.638312][ T3492]  do_syscall_64+0x117/0xfc0
[  288.638314][ T3492]  ? irq_exit_rcu+0x1a/0x30
[  288.638318][ T3492]  entry_SYSCALL_64_after_hwframe+0x4b/0x53
[  288.638321][ T3492] RIP: 0033:0x7f895f8fe08e

^ permalink raw reply	[flat|nested] 3+ messages in thread

* [syzbot ci] Re: tcp: fix stale per-CPU tcp_tw_isn leak enabling ISN prediction
  2026-05-11 15:15 [PATCH net] tcp: fix stale per-CPU tcp_tw_isn leak enabling ISN prediction Eric Dumazet
  2026-05-11 20:23 ` Jakub Kicinski
@ 2026-05-12  6:18 ` syzbot ci
  1 sibling, 0 replies; 3+ messages in thread
From: syzbot ci @ 2026-05-12  6:18 UTC (permalink / raw)
  To: davem, edumazet, eric.dumazet, horms, kuba, kuniyu, ncardwell,
	netdev, pabeni
  Cc: syzbot, syzkaller-bugs

syzbot ci has tested the following series

[v1] tcp: fix stale per-CPU tcp_tw_isn leak enabling ISN prediction
https://lore.kernel.org/all/20260511151559.1916614-1-edumazet@google.com
* [PATCH net] tcp: fix stale per-CPU tcp_tw_isn leak enabling ISN prediction

and found the following issues:
* BUG: using __this_cpu_write() in preemptible code in tcp_v4_do_rcv
* BUG: using __this_cpu_write() in preemptible code in tcp_v6_do_rcv

Full report is available here:
https://ci.syzbot.org/series/45073ca6-4b5b-4cbd-83bf-ae30c470d334

***

BUG: using __this_cpu_write() in preemptible code in tcp_v4_do_rcv

tree:      net
URL:       https://kernel.googlesource.com/pub/scm/linux/kernel/git/netdev/net.git
base:      a450063ef86b9967234ca1f896c0d77400c74f11
arch:      amd64
compiler:  Debian clang version 21.1.8 (++20251221033036+2078da43e25a-1~exp1~20251221153213.50), Debian LLD 21.1.8
config:    https://ci.syzbot.org/builds/d6bfac50-400c-447d-b4f2-f5caa36c16ac/config
syz repro: https://ci.syzbot.org/findings/95b64202-6cf3-4356-94bc-753dbd8443c4/syz_repro

BUG: using __this_cpu_write() in preemptible [00000000] code: syz.0.17/5875
caller is tcp_v4_do_rcv+0xb4a/0x13e0 net/ipv4/tcp_ipv4.c:1881
CPU: 0 UID: 0 PID: 5875 Comm: syz.0.17 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
Call Trace:
 <TASK>
 dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120
 check_preemption_disabled+0xd3/0xe0 lib/smp_processor_id.c:47
 tcp_v4_do_rcv+0xb4a/0x13e0 net/ipv4/tcp_ipv4.c:1881
 sk_backlog_rcv include/net/sock.h:1190 [inline]
 __release_sock+0x265/0x3a0 net/core/sock.c:3216
 release_sock+0x190/0x260 net/core/sock.c:3815
 mptcp_pm_rm_addr_or_subflow+0x472/0x9d0 net/mptcp/pm.c:789
 mptcp_nl_remove_id_zero_address net/mptcp/pm_kernel.c:1140 [inline]
 mptcp_pm_nl_del_addr_doit+0x503/0x1430 net/mptcp/pm_kernel.c:1176
 genl_family_rcv_msg_doit+0x22a/0x330 net/netlink/genetlink.c:1114
 genl_family_rcv_msg net/netlink/genetlink.c:1194 [inline]
 genl_rcv_msg+0x61c/0x7a0 net/netlink/genetlink.c:1209
 netlink_rcv_skb+0x232/0x4b0 net/netlink/af_netlink.c:2550
 genl_rcv+0x28/0x40 net/netlink/genetlink.c:1218
 netlink_unicast_kernel net/netlink/af_netlink.c:1318 [inline]
 netlink_unicast+0x75c/0x8e0 net/netlink/af_netlink.c:1344
 netlink_sendmsg+0x813/0xb40 net/netlink/af_netlink.c:1894
 sock_sendmsg_nosec net/socket.c:787 [inline]
 __sock_sendmsg net/socket.c:802 [inline]
 ____sys_sendmsg+0x972/0x9f0 net/socket.c:2698
 ___sys_sendmsg+0x2a5/0x360 net/socket.c:2752
 __sys_sendmsg net/socket.c:2784 [inline]
 __do_sys_sendmsg net/socket.c:2789 [inline]
 __se_sys_sendmsg net/socket.c:2787 [inline]
 __x64_sys_sendmsg+0x1bd/0x2a0 net/socket.c:2787
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0x15f/0xf80 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f6847f9cdd9
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f6848e95028 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00007f6848215fa0 RCX: 00007f6847f9cdd9
RDX: 0000000024000800 RSI: 0000200000000140 RDI: 0000000000000004
RBP: 00007f6848032d69 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f6848216038 R14: 00007f6848215fa0 R15: 00007ffe50763178
 </TASK>


***

BUG: using __this_cpu_write() in preemptible code in tcp_v6_do_rcv

tree:      net
URL:       https://kernel.googlesource.com/pub/scm/linux/kernel/git/netdev/net.git
base:      a450063ef86b9967234ca1f896c0d77400c74f11
arch:      amd64
compiler:  Debian clang version 21.1.8 (++20251221033036+2078da43e25a-1~exp1~20251221153213.50), Debian LLD 21.1.8
config:    https://ci.syzbot.org/builds/d6bfac50-400c-447d-b4f2-f5caa36c16ac/config
syz repro: https://ci.syzbot.org/findings/f5960777-902c-46ad-b811-9dff0f17d320/syz_repro

BUG: using __this_cpu_write() in preemptible [00000000] code: syz.0.173/6173
caller is tcp_v6_do_rcv+0x10c2/0x1ca0 net/ipv6/tcp_ipv6.c:1644
CPU: 0 UID: 0 PID: 6173 Comm: syz.0.173 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
Call Trace:
 <TASK>
 dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120
 check_preemption_disabled+0xd3/0xe0 lib/smp_processor_id.c:47
 tcp_v6_do_rcv+0x10c2/0x1ca0 net/ipv6/tcp_ipv6.c:1644
 sk_backlog_rcv include/net/sock.h:1190 [inline]
 __release_sock+0x1b8/0x3a0 net/core/sock.c:3216
 __tcp_close+0x832/0xfe0 net/ipv4/tcp.c:3238
 __mptcp_close_ssk+0x52d/0x1180 net/mptcp/protocol.c:2604
 mptcp_destroy_common+0x15c/0x330 net/mptcp/protocol.c:3408
 mptcp_destroy+0x82/0x120 net/mptcp/protocol.c:3616
 __mptcp_destroy_sock+0x156/0x370 net/mptcp/protocol.c:3250
 __mptcp_close+0x8f6/0xd10 net/mptcp/protocol.c:3352
 mptcp_close+0x28/0x1a0 net/mptcp/protocol.c:3367
 inet_release+0x143/0x190 net/ipv4/af_inet.c:442
 __sock_release net/socket.c:722 [inline]
 sock_close+0xc3/0x240 net/socket.c:1514
 __fput+0x44f/0xa60 fs/file_table.c:510
 task_work_run+0x1d9/0x270 kernel/task_work.c:233
 resume_user_mode_work include/linux/resume_user_mode.h:50 [inline]
 __exit_to_user_mode_loop kernel/entry/common.c:67 [inline]
 exit_to_user_mode_loop+0xed/0x480 kernel/entry/common.c:98
 __exit_to_user_mode_prepare include/linux/irq-entry-common.h:207 [inline]
 syscall_exit_to_user_mode_prepare include/linux/irq-entry-common.h:238 [inline]
 syscall_exit_to_user_mode include/linux/entry-common.h:318 [inline]
 do_syscall_64+0x33e/0xf80 arch/x86/entry/syscall_64.c:100
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f69be79cdd9
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffc7668ad18 EFLAGS: 00000246 ORIG_RAX: 00000000000001b4
RAX: 0000000000000000 RBX: 00007ffc7668ae00 RCX: 00007f69be79cdd9
RDX: 0000000000000000 RSI: 000000000000001e RDI: 0000000000000003
RBP: 0000000000012dd1 R08: 0000000000000001 R09: 0000000000000000
R10: 0000001b32b20000 R11: 0000000000000246 R12: 00007ffc7668ae40
R13: 00007f69bea15fac R14: 0000000000012e0a R15: 00007f69bea15fa0
 </TASK>


***

If these findings have caused you to resend the series or submit a
separate fix, please add the following tag to your commit message:
  Tested-by: syzbot@syzkaller.appspotmail.com

---
This report is generated by a bot. It may contain errors.
syzbot ci engineers can be reached at syzkaller@googlegroups.com.

To test a patch for this bug, please reply with `#syz test`
(should be on a separate line).

The patch should be attached to the email.
Note: arguments like custom git repos and branches are not supported.

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2026-05-12  6:18 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-05-11 15:15 [PATCH net] tcp: fix stale per-CPU tcp_tw_isn leak enabling ISN prediction Eric Dumazet
2026-05-11 20:23 ` Jakub Kicinski
2026-05-12  6:18 ` [syzbot ci] " syzbot ci

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox