All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH RFC v2] tipc: fix spinlock recursion in tipc_sk_rcv()
@ 2026-07-29 16:54 syzbot
  2026-08-07  8:54 ` Bartosz Chronowski
  2026-08-07  9:29 ` [PATCH RFC v2] tipc: fix spinlock recursion in tipc_sk_rcv() Bartosz Chronowski
  0 siblings, 2 replies; 11+ messages in thread
From: syzbot @ 2026-07-29 16:54 UTC (permalink / raw)
  To: syzkaller-upstream-moderation; +Cc: immersa.bartosz.chronowski, syzbot

An RCU stall (infinite loop / livelock) can occur due to a spinlock
recursion bug in the TIPC subsystem. Specifically, tipc_sk_rcv() can be
called recursively on the same CPU and try to acquire a socket spinlock
that is already held higher up in the call stack. Because it uses
spin_trylock_bh(), it fails to acquire the lock and enters an infinite
loop.

The issue is triggered by the concurrent execution of sendmsg() and
setsockopt() on a connected TIPC socket pair (fd0 and fd1), which reside on
the same node (loopback).

The sequence of events leading to the livelock is:
1. setsockopt locks fd0 (acquires the socket mutex sk->sk_lock.owned).
While fd0 is locked, sendmsg on fd1 sends messages to fd0. Since fd0 is
owned by the user, tipc_sk_enqueue() adds these messages to fd0's backlog
queue.
2. setsockopt processes SO_RCVBUFFORCE and reduces fd0's sk_rcvbuf to the
minimum value (SOCK_MIN_RCVBUF, 2048).
3. setsockopt finishes and calls release_sock(fd0). Seeing a non-empty
backlog, it calls __release_sock(fd0).
4. __release_sock(fd0) drops fd0's spinlock (sk->sk_lock.slock) and calls
sk_backlog_rcv() -> tipc_sk_backlog_rcv() -> tipc_sk_filter_rcv() to
process the backlog.
5. Because sk_rcvbuf was drastically reduced, the receive queue limit is
exceeded. tipc_sk_filter_rcv(fd0) rejects the message (TIPC_ERR_OVERLOAD)
and queues a reject message to xmitq.
6. tipc_sk_backlog_rcv(fd0) calls tipc_node_distr_xmit(&xmitq) to send the
reject message back to fd1. Since fd1 is on the same node, tipc_node_xmit()
synchronously calls tipc_sk_rcv(fd1).
7. tipc_sk_rcv(fd1) successfully locks fd1's spinlock and calls
tipc_sk_enqueue(fd1) -> tipc_sk_filter_rcv(fd1) ->
tipc_sk_filter_connect(fd1).
8. tipc_sk_filter_connect(fd1) sees that fd1's sk_write_queue is not empty.
It calls tipc_sk_push_backlog(fd1).
9. tipc_sk_push_backlog(fd1) calls tipc_node_xmit() to send the queued
messages to fd0. This synchronously calls tipc_sk_rcv(fd0).
10. tipc_sk_rcv(fd0) successfully locks fd0's spinlock (as __release_sock
dropped it) and calls tipc_sk_enqueue(fd0). Since fd0 is still owned by the
user, it tries to add the message to the backlog. The backlog is full, so
it rejects the message again, sending a new reject message back to fd1.
11. The new reject message is sent to fd1 via tipc_node_xmit() ->
tipc_sk_rcv(fd1). tipc_sk_rcv(fd1) tries to lock fd1's spinlock. However,
fd1's spinlock is already held by the outer tipc_sk_rcv(fd1) at step 7.

In tipc_sk_rcv(), if spin_trylock_bh() fails, it assumes another CPU holds
the lock, so it just continues the loop without dequeuing the message from
inputq. Because the lock is held by the current CPU, it will never be
released, and the while loop spins infinitely, causing the RCU stall:

rcu: INFO: rcu_preempt self-detected stall on CPU
...
Call Trace:
 <TASK>
 __raw_spin_trylock_bh include/linux/spinlock_api_smp.h:207 [inline]
 _raw_spin_trylock_bh+0x60/0x70 kernel/locking/spinlock.c:150
 spin_trylock_bh include/linux/spinlock.h:414 [inline]
 tipc_sk_rcv+0x420/0x2b50 net/tipc/socket.c:2500
 tipc_node_xmit+0x205/0xf10 net/tipc/node.c:1701
 tipc_node_xmit_skb net/tipc/node.c:1766 [inline]
 tipc_node_distr_xmit+0x2cd/0x400 net/tipc/node.c:1781
 tipc_sk_rcv+0x2539/0x2b50 net/tipc/socket.c:2505
 tipc_node_xmit+0x205/0xf10 net/tipc/node.c:1701
 tipc_sk_push_backlog net/tipc/socket.c:1313 [inline]
 tipc_sk_filter_connect net/tipc/socket.c:2259 [inline]
 tipc_sk_filter_rcv+0x17f3/0x3210 net/tipc/socket.c:2368
 tipc_sk_enqueue net/tipc/socket.c:2449 [inline]
 tipc_sk_rcv+0x877/0x2b50 net/tipc/socket.c:2501
 tipc_node_xmit+0x205/0xf10 net/tipc/node.c:1701
 tipc_node_xmit_skb net/tipc/node.c:1766 [inline]
 tipc_node_distr_xmit+0x2cd/0x400 net/tipc/node.c:1781
 tipc_sk_backlog_rcv+0x1ad/0x270 net/tipc/socket.c:2416
 sk_backlog_rcv include/net/sock.h:1190 [inline]
 __release_sock+0x28b/0x390 net/core/sock.c:3261
 release_sock+0x190/0x260 net/core/sock.c:3860
 sockopt_release_sock net/core/sock.c:1163 [inline]
 sk_setsockopt+0x2822/0x2e80 net/core/sock.c:1681
 do_sock_setsockopt+0x11b/0x1b0 net/socket.c:2364
 __sys_setsockopt net/socket.c:2393 [inline]
 __do_sys_setsockopt net/socket.c:2399 [inline]
 __se_sys_setsockopt net/socket.c:2396 [inline]
 __x64_sys_setsockopt+0x13d/0x1b0 net/socket.c:2396
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0x15f/0x560 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
 </TASK>

To fix this, tipc_sk_push_backlog() should not call tipc_node_xmit()
directly for own-node destinations while holding the socket spinlock.
Instead, modify tipc_sk_push_backlog() to take an xmitq parameter. If xmitq
is provided and the destination is on the own node, it splices the write
queue into xmitq, deferring the transmission until after the spinlock is
released. For calls like __tipc_shutdown() that execute under the socket
mutex rather than the spinlock, passing NULL for xmitq allows direct
transmission to continue safely. For off-node destinations, direct
transmission remains safe as it does not lead to synchronous loopback
recursion.

Fixes: c0bceb97db9e ("tipc: add smart nagle feature")
Assisted-by: Gemini:gemini-3.5-flash Gemini:gemini-3.1-pro-preview syzbot
Reported-by: syzbot+10a41dc44eef71aa9450@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=10a41dc44eef71aa9450
Link: https://syzkaller.appspot.com/ai_job?id=e6db6744-c061-4f81-bde8-25993166f061
To: "David S. Miller" <davem@davemloft.net>
To: "Eric Dumazet" <edumazet@google.com>
To: "Jon Maloy" <jmaloy@redhat.com>
To: "Jakub Kicinski" <kuba@kernel.org>
To: <netdev@vger.kernel.org>
To: "Paolo Abeni" <pabeni@redhat.com>
To: <tipc-discussion@lists.sourceforge.net>
To: "Jon Maloy" <jon.maloy@ericsson.com>
Cc: "Simon Horman" <horms@kernel.org>
Cc: <linux-kernel@vger.kernel.org>

---
v2:
- Only defer transmission in tipc_sk_push_backlog() if the destination is on the own node.
- Update the commit description to clarify that deferral is only needed for own-node destinations.

v1:
https://lore.kernel.org/all/01641937-e30d-43dd-ba9c-95b3d2ae5691@mail.kernel.org/T/
---
diff --git a/net/tipc/socket.c b/net/tipc/socket.c
index e564341e0..16d844fc8 100644
--- a/net/tipc/socket.c
+++ b/net/tipc/socket.c
@@ -156,7 +156,8 @@ static int tipc_sk_insert(struct tipc_sock *tsk);
 static void tipc_sk_remove(struct tipc_sock *tsk);
 static int __tipc_sendstream(struct socket *sock, struct msghdr *m, size_t dsz);
 static int __tipc_sendmsg(struct socket *sock, struct msghdr *m, size_t dsz);
-static void tipc_sk_push_backlog(struct tipc_sock *tsk, bool nagle_ack);
+static void tipc_sk_push_backlog(struct tipc_sock *tsk, bool nagle_ack,
+				 struct sk_buff_head *xmitq);
 static int tipc_wait_for_connect(struct socket *sock, long *timeo_p);
 
 static const struct proto_ops packet_ops;
@@ -560,7 +561,7 @@ static void __tipc_shutdown(struct socket *sock, int error)
 					    !tsk_conn_cong(tsk)));
 
 	/* Push out delayed messages if in Nagle mode */
-	tipc_sk_push_backlog(tsk, false);
+	tipc_sk_push_backlog(tsk, false, NULL);
 	/* Remove pending SYN */
 	__skb_queue_purge(&sk->sk_write_queue);
 
@@ -1268,7 +1269,8 @@ void tipc_sk_mcast_rcv(struct net *net, struct sk_buff_head *arrvq,
 /* tipc_sk_push_backlog(): send accumulated buffers in socket write queue
  *                         when socket is in Nagle mode
  */
-static void tipc_sk_push_backlog(struct tipc_sock *tsk, bool nagle_ack)
+static void tipc_sk_push_backlog(struct tipc_sock *tsk, bool nagle_ack,
+				 struct sk_buff_head *xmitq)
 {
 	struct sk_buff_head *txq = &tsk->sk.sk_write_queue;
 	struct sk_buff *skb = skb_peek_tail(txq);
@@ -1310,6 +1312,12 @@ static void tipc_sk_push_backlog(struct tipc_sock *tsk, bool nagle_ack)
 		tsk->pkt_cnt += skb_queue_len(txq);
 	tsk->snt_unacked += tsk->snd_backlog;
 	tsk->snd_backlog = 0;
+
+	if (xmitq && in_own_node(net, dnode)) {
+		skb_queue_splice_tail_init(txq, xmitq);
+		return;
+	}
+
 	rc = tipc_node_xmit(net, txq, dnode, tsk->portid);
 	if (rc == -ELINKCONG)
 		tsk->cong_link_cnt = 1;
@@ -1367,7 +1375,7 @@ static void tipc_sk_conn_proto_rcv(struct tipc_sock *tsk, struct sk_buff *skb,
 			goto exit;
 
 		was_cong = tsk_conn_cong(tsk);
-		tipc_sk_push_backlog(tsk, msg_nagle_ack(hdr));
+		tipc_sk_push_backlog(tsk, msg_nagle_ack(hdr), xmitq);
 		tsk->snt_unacked -= msg_conn_ack(hdr);
 		if (tsk->peer_caps & TIPC_BLOCK_FLOWCTL)
 			tsk->snd_win = msg_adv_win(hdr);
@@ -2165,7 +2173,7 @@ static void tipc_sk_proto_rcv(struct sock *sk,
 		smp_wmb();
 		tsk->cong_link_cnt--;
 		wakeup = true;
-		tipc_sk_push_backlog(tsk, false);
+		tipc_sk_push_backlog(tsk, false, xmitq);
 		break;
 	case GROUP_PROTOCOL:
 		tipc_group_proto_rcv(grp, &wakeup, hdr, inputq, xmitq);
@@ -2256,7 +2264,7 @@ static bool tipc_sk_filter_connect(struct tipc_sock *tsk, struct sk_buff *skb,
 		return false;
 	case TIPC_ESTABLISHED:
 		if (!skb_queue_empty(&sk->sk_write_queue))
-			tipc_sk_push_backlog(tsk, false);
+			tipc_sk_push_backlog(tsk, false, xmitq);
 		/* Accept only connection-based messages sent by peer */
 		if (likely(con_msg && !err && pport == oport &&
 			   pnode == onode)) {


base-commit: 1590cf0329716306e948a8fc29f1d3ee87d3989f
-- 
This is an AI-generated patch subject to moderation.
Reply with '#syz upstream' to Sign-off the patch as a human author
and send it to the upstream kernel mailing lists.
Reply with '#syz reject' to reject it ('#syz unreject' to undo).

See https://goo.gle/syzbot-ai-patches for information about AI-generated patches.
You can comment on the patch as usual, syzbot will try to address
the comments and send a new version of the patch if necessary.
syzbot engineers can be reached at syzkaller@googlegroups.com.

^ permalink raw reply related	[flat|nested] 11+ messages in thread
[parent not found: <ob7cvq6bqedi3rsiyjlcivgfj6mxttkrcrwdciny4brsdpm7yv@zvu3gi5n5cuz>]
* [syzbot] [tipc?] BUG: soft lockup in do_sock_setsockopt
@ 2024-03-22 14:23 syzbot
  2024-03-23  0:02 ` Hillf Danton
  2024-10-03  9:25 ` syzbot
  0 siblings, 2 replies; 11+ messages in thread
From: syzbot @ 2024-03-22 14:23 UTC (permalink / raw)
  To: davem, edumazet, jmaloy, kuba, linux-kernel, netdev, pabeni,
	syzkaller-bugs, tipc-discussion, ying.xue

Hello,

syzbot found the following issue on:

HEAD commit:    707081b61156 Merge branch 'for-next/core', remote-tracking..
git tree:       git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci
console output: https://syzkaller.appspot.com/x/log.txt?x=16d23231180000
kernel config:  https://syzkaller.appspot.com/x/.config?x=caeac3f3565b057a
dashboard link: https://syzkaller.appspot.com/bug?extid=10a41dc44eef71aa9450
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
userspace arch: arm64
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=126fea6e180000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=114e4c81180000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/6cad68bf7532/disk-707081b6.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/1a27e5400778/vmlinux-707081b6.xz
kernel image: https://storage.googleapis.com/syzbot-assets/67dfc53755d0/Image-707081b6.gz.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+10a41dc44eef71aa9450@syzkaller.appspotmail.com

watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [syz-executor178:13146]
Modules linked in:
irq event stamp: 29265101
hardirqs last  enabled at (29265100): [<ffff8000801dae10>] __local_bh_enable_ip+0x224/0x44c kernel/softirq.c:386
hardirqs last disabled at (29265101): [<ffff80008ad66a78>] __el1_irq arch/arm64/kernel/entry-common.c:533 [inline]
hardirqs last disabled at (29265101): [<ffff80008ad66a78>] el1_interrupt+0x24/0x68 arch/arm64/kernel/entry-common.c:551
softirqs last  enabled at (638): [<ffff80008a7fe730>] spin_unlock_bh include/linux/spinlock.h:396 [inline]
softirqs last  enabled at (638): [<ffff80008a7fe730>] tipc_skb_peek_port net/tipc/msg.h:1235 [inline]
softirqs last  enabled at (638): [<ffff80008a7fe730>] tipc_sk_rcv+0x34c/0x1888 net/tipc/socket.c:2494
softirqs last disabled at (640): [<ffff80008a7fe750>] spin_trylock_bh include/linux/spinlock.h:411 [inline]
softirqs last disabled at (640): [<ffff80008a7fe750>] tipc_sk_rcv+0x36c/0x1888 net/tipc/socket.c:2499
CPU: 0 PID: 13146 Comm: syz-executor178 Not tainted 6.8.0-rc7-syzkaller-g707081b61156 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : lock_acquire+0x278/0x71c
lr : lockdep_recursion_finish kernel/locking/lockdep.c:467 [inline]
lr : lock_acquire+0x248/0x71c kernel/locking/lockdep.c:5756
sp : ffff80009ab86080
x29: ffff80009ab86190 x28: dfff800000000000 x27: ffff700013570c1c
x26: ffff0001b3fffdc0 x25: ffff80008ee74ac0 x24: 0000000000000000
x23: 0000000000000000 x22: 0000000000000000 x21: 0000000000000000
x20: ffff80008ebebdc0 x19: ffff0001b3fffdc0 x18: ffff80009ab862e0
x17: 000000000000c55c x16: ffff80008ad6b1c0 x15: 0000000000000001
x14: ffff80008eca0458 x13: dfff800000000000 x12: 00000000af1c601b
x11: 00000000a4e03a31 x10: 0000000000000003 x9 : 0000000000000000
x8 : 00000000000000c0 x7 : ffff80008a80b3f8 x6 : 0000000000000000
x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000000002
x2 : 0000000000000008 x1 : ffff80008aedfba0 x0 : 0000000000000000
Call trace:
 __daif_local_irq_restore arch/arm64/include/asm/irqflags.h:176 [inline]
 arch_local_irq_restore arch/arm64/include/asm/irqflags.h:196 [inline]
 lock_acquire+0x278/0x71c kernel/locking/lockdep.c:5757
 rcu_lock_acquire+0x40/0x4c include/linux/rcupdate.h:298
 rcu_read_lock include/linux/rcupdate.h:750 [inline]
 tipc_sk_lookup+0xc8/0x8b4 net/tipc/socket.c:3003
 tipc_sk_rcv+0x358/0x1888 net/tipc/socket.c:2495
 tipc_node_xmit+0x1b0/0xdb0 net/tipc/node.c:1703
 tipc_node_xmit_skb net/tipc/node.c:1768 [inline]
 tipc_node_distr_xmit+0x28c/0x3a4 net/tipc/node.c:1783
 tipc_sk_rcv+0x1280/0x1888 net/tipc/socket.c:2504
 tipc_node_xmit+0x1b0/0xdb0 net/tipc/node.c:1703
 tipc_sk_push_backlog net/tipc/socket.c:1317 [inline]
 tipc_sk_filter_connect net/tipc/socket.c:2258 [inline]
 tipc_sk_filter_rcv+0x13f8/0x2cac net/tipc/socket.c:2367
 tipc_sk_enqueue net/tipc/socket.c:2448 [inline]
 tipc_sk_rcv+0x6d0/0x1888 net/tipc/socket.c:2500
 tipc_node_xmit+0x1b0/0xdb0 net/tipc/node.c:1703
 tipc_node_xmit_skb net/tipc/node.c:1768 [inline]
 tipc_node_distr_xmit+0x28c/0x3a4 net/tipc/node.c:1783
 tipc_sk_backlog_rcv+0x164/0x214 net/tipc/socket.c:2415
 sk_backlog_rcv include/net/sock.h:1092 [inline]
 __release_sock+0x1a8/0x408 net/core/sock.c:2972
 release_sock+0x68/0x1b8 net/core/sock.c:3538
 sk_setsockopt+0xbdc/0x306c
 sock_setsockopt+0x68/0x80 net/core/sock.c:1548
 do_sock_setsockopt+0x238/0x4e0 net/socket.c:2307
 __sys_setsockopt+0x128/0x1a8 net/socket.c:2334
 __do_sys_setsockopt net/socket.c:2343 [inline]
 __se_sys_setsockopt net/socket.c:2340 [inline]
 __arm64_sys_setsockopt+0xb8/0xd4 net/socket.c:2340
 __invoke_syscall arch/arm64/kernel/syscall.c:34 [inline]
 invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:48
 el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:133
 do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:152
 el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:712
 el0t_64_sync_handler+0x84/0xfc arch/arm64/kernel/entry-common.c:730
 el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:598


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2026-08-11 14:48 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-29 16:54 [PATCH RFC v2] tipc: fix spinlock recursion in tipc_sk_rcv() syzbot
2026-08-07  8:54 ` Bartosz Chronowski
2026-08-07 19:26   ` [syzbot] [tipc?] BUG: soft lockup in do_sock_setsockopt syzbot
2026-08-07  9:29 ` [PATCH RFC v2] tipc: fix spinlock recursion in tipc_sk_rcv() Bartosz Chronowski
     [not found] <ob7cvq6bqedi3rsiyjlcivgfj6mxttkrcrwdciny4brsdpm7yv@zvu3gi5n5cuz>
2026-08-10 14:00 ` [syzbot] [tipc?] BUG: soft lockup in do_sock_setsockopt syzbot
2026-08-11 10:25   ` Bartosz Chronowski
2026-08-11 14:48     ` syzbot
  -- strict thread matches above, loose matches on Subject: below --
2024-03-22 14:23 syzbot
2024-03-23  0:02 ` Hillf Danton
2024-03-23  8:44   ` syzbot
2024-10-03  9:25 ` syzbot

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.