All of lore.kernel.org
 help / color / mirror / Atom feed
* Re: [PATCH RFC v2] tipc: fix spinlock recursion in tipc_sk_rcv()
@ 2026-08-07  8:54 Bartosz Chronowski
  2026-08-07 19:26 ` [syzbot] [tipc?] BUG: soft lockup in do_sock_setsockopt syzbot
  0 siblings, 1 reply; 8+ messages in thread
From: Bartosz Chronowski @ 2026-08-07  8:54 UTC (permalink / raw)
  To: syzbot; +Cc: syzkaller-upstream-moderation, syzbot

#syz test

On Wed, Jul 29, 2026 at 04:54:27PM +0000, syzbot wrote:
> An RCU stall (infinite loop / livelock) can occur due to a spinlock
> recursion bug in the TIPC subsystem. Specifically, tipc_sk_rcv() can be
> called recursively on the same CPU and try to acquire a socket spinlock
> that is already held higher up in the call stack. Because it uses
> spin_trylock_bh(), it fails to acquire the lock and enters an infinite
> loop.
> 
> The issue is triggered by the concurrent execution of sendmsg() and
> setsockopt() on a connected TIPC socket pair (fd0 and fd1), which reside on
> the same node (loopback).
> 
> The sequence of events leading to the livelock is:
> 1. setsockopt locks fd0 (acquires the socket mutex sk->sk_lock.owned).
> While fd0 is locked, sendmsg on fd1 sends messages to fd0. Since fd0 is
> owned by the user, tipc_sk_enqueue() adds these messages to fd0's backlog
> queue.
> 2. setsockopt processes SO_RCVBUFFORCE and reduces fd0's sk_rcvbuf to the
> minimum value (SOCK_MIN_RCVBUF, 2048).
> 3. setsockopt finishes and calls release_sock(fd0). Seeing a non-empty
> backlog, it calls __release_sock(fd0).
> 4. __release_sock(fd0) drops fd0's spinlock (sk->sk_lock.slock) and calls
> sk_backlog_rcv() -> tipc_sk_backlog_rcv() -> tipc_sk_filter_rcv() to
> process the backlog.
> 5. Because sk_rcvbuf was drastically reduced, the receive queue limit is
> exceeded. tipc_sk_filter_rcv(fd0) rejects the message (TIPC_ERR_OVERLOAD)
> and queues a reject message to xmitq.
> 6. tipc_sk_backlog_rcv(fd0) calls tipc_node_distr_xmit(&xmitq) to send the
> reject message back to fd1. Since fd1 is on the same node, tipc_node_xmit()
> synchronously calls tipc_sk_rcv(fd1).
> 7. tipc_sk_rcv(fd1) successfully locks fd1's spinlock and calls
> tipc_sk_enqueue(fd1) -> tipc_sk_filter_rcv(fd1) ->
> tipc_sk_filter_connect(fd1).
> 8. tipc_sk_filter_connect(fd1) sees that fd1's sk_write_queue is not empty.
> It calls tipc_sk_push_backlog(fd1).
> 9. tipc_sk_push_backlog(fd1) calls tipc_node_xmit() to send the queued
> messages to fd0. This synchronously calls tipc_sk_rcv(fd0).
> 10. tipc_sk_rcv(fd0) successfully locks fd0's spinlock (as __release_sock
> dropped it) and calls tipc_sk_enqueue(fd0). Since fd0 is still owned by the
> user, it tries to add the message to the backlog. The backlog is full, so
> it rejects the message again, sending a new reject message back to fd1.
> 11. The new reject message is sent to fd1 via tipc_node_xmit() ->
> tipc_sk_rcv(fd1). tipc_sk_rcv(fd1) tries to lock fd1's spinlock. However,
> fd1's spinlock is already held by the outer tipc_sk_rcv(fd1) at step 7.
> 
> In tipc_sk_rcv(), if spin_trylock_bh() fails, it assumes another CPU holds
> the lock, so it just continues the loop without dequeuing the message from
> inputq. Because the lock is held by the current CPU, it will never be
> released, and the while loop spins infinitely, causing the RCU stall:
> 
> rcu: INFO: rcu_preempt self-detected stall on CPU
> ...
> Call Trace:
>  <TASK>
>  __raw_spin_trylock_bh include/linux/spinlock_api_smp.h:207 [inline]
>  _raw_spin_trylock_bh+0x60/0x70 kernel/locking/spinlock.c:150
>  spin_trylock_bh include/linux/spinlock.h:414 [inline]
>  tipc_sk_rcv+0x420/0x2b50 net/tipc/socket.c:2500
>  tipc_node_xmit+0x205/0xf10 net/tipc/node.c:1701
>  tipc_node_xmit_skb net/tipc/node.c:1766 [inline]
>  tipc_node_distr_xmit+0x2cd/0x400 net/tipc/node.c:1781
>  tipc_sk_rcv+0x2539/0x2b50 net/tipc/socket.c:2505
>  tipc_node_xmit+0x205/0xf10 net/tipc/node.c:1701
>  tipc_sk_push_backlog net/tipc/socket.c:1313 [inline]
>  tipc_sk_filter_connect net/tipc/socket.c:2259 [inline]
>  tipc_sk_filter_rcv+0x17f3/0x3210 net/tipc/socket.c:2368
>  tipc_sk_enqueue net/tipc/socket.c:2449 [inline]
>  tipc_sk_rcv+0x877/0x2b50 net/tipc/socket.c:2501
>  tipc_node_xmit+0x205/0xf10 net/tipc/node.c:1701
>  tipc_node_xmit_skb net/tipc/node.c:1766 [inline]
>  tipc_node_distr_xmit+0x2cd/0x400 net/tipc/node.c:1781
>  tipc_sk_backlog_rcv+0x1ad/0x270 net/tipc/socket.c:2416
>  sk_backlog_rcv include/net/sock.h:1190 [inline]
>  __release_sock+0x28b/0x390 net/core/sock.c:3261
>  release_sock+0x190/0x260 net/core/sock.c:3860
>  sockopt_release_sock net/core/sock.c:1163 [inline]
>  sk_setsockopt+0x2822/0x2e80 net/core/sock.c:1681
>  do_sock_setsockopt+0x11b/0x1b0 net/socket.c:2364
>  __sys_setsockopt net/socket.c:2393 [inline]
>  __do_sys_setsockopt net/socket.c:2399 [inline]
>  __se_sys_setsockopt net/socket.c:2396 [inline]
>  __x64_sys_setsockopt+0x13d/0x1b0 net/socket.c:2396
>  do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
>  do_syscall_64+0x15f/0x560 arch/x86/entry/syscall_64.c:94
>  entry_SYSCALL_64_after_hwframe+0x77/0x7f
>  </TASK>
> 
> To fix this, tipc_sk_push_backlog() should not call tipc_node_xmit()
> directly for own-node destinations while holding the socket spinlock.
> Instead, modify tipc_sk_push_backlog() to take an xmitq parameter. If xmitq
> is provided and the destination is on the own node, it splices the write
> queue into xmitq, deferring the transmission until after the spinlock is
> released. For calls like __tipc_shutdown() that execute under the socket
> mutex rather than the spinlock, passing NULL for xmitq allows direct
> transmission to continue safely. For off-node destinations, direct
> transmission remains safe as it does not lead to synchronous loopback
> recursion.
> 
> Fixes: c0bceb97db9e ("tipc: add smart nagle feature")
> Assisted-by: Gemini:gemini-3.5-flash Gemini:gemini-3.1-pro-preview syzbot
> Reported-by: syzbot+10a41dc44eef71aa9450@syzkaller.appspotmail.com
> Closes: https://syzkaller.appspot.com/bug?extid=10a41dc44eef71aa9450
> Link: https://syzkaller.appspot.com/ai_job?id=e6db6744-c061-4f81-bde8-25993166f061
> To: "David S. Miller" <davem@davemloft.net>
> To: "Eric Dumazet" <edumazet@google.com>
> To: "Jon Maloy" <jmaloy@redhat.com>
> To: "Jakub Kicinski" <kuba@kernel.org>
> To: <netdev@vger.kernel.org>
> To: "Paolo Abeni" <pabeni@redhat.com>
> To: <tipc-discussion@lists.sourceforge.net>
> To: "Jon Maloy" <jon.maloy@ericsson.com>
> Cc: "Simon Horman" <horms@kernel.org>
> Cc: <linux-kernel@vger.kernel.org>
> 
> ---
> v2:
> - Only defer transmission in tipc_sk_push_backlog() if the destination is on the own node.
> - Update the commit description to clarify that deferral is only needed for own-node destinations.
> 
> v1:
> https://lore.kernel.org/all/01641937-e30d-43dd-ba9c-95b3d2ae5691@mail.kernel.org/T/
> ---
> diff --git a/net/tipc/socket.c b/net/tipc/socket.c
> index e564341e0..16d844fc8 100644
> --- a/net/tipc/socket.c
> +++ b/net/tipc/socket.c
> @@ -156,7 +156,8 @@ static int tipc_sk_insert(struct tipc_sock *tsk);
>  static void tipc_sk_remove(struct tipc_sock *tsk);
>  static int __tipc_sendstream(struct socket *sock, struct msghdr *m, size_t dsz);
>  static int __tipc_sendmsg(struct socket *sock, struct msghdr *m, size_t dsz);
> -static void tipc_sk_push_backlog(struct tipc_sock *tsk, bool nagle_ack);
> +static void tipc_sk_push_backlog(struct tipc_sock *tsk, bool nagle_ack,
> +				 struct sk_buff_head *xmitq);
>  static int tipc_wait_for_connect(struct socket *sock, long *timeo_p);
>  
>  static const struct proto_ops packet_ops;
> @@ -560,7 +561,7 @@ static void __tipc_shutdown(struct socket *sock, int error)
>  					    !tsk_conn_cong(tsk)));
>  
>  	/* Push out delayed messages if in Nagle mode */
> -	tipc_sk_push_backlog(tsk, false);
> +	tipc_sk_push_backlog(tsk, false, NULL);
>  	/* Remove pending SYN */
>  	__skb_queue_purge(&sk->sk_write_queue);
>  
> @@ -1268,7 +1269,8 @@ void tipc_sk_mcast_rcv(struct net *net, struct sk_buff_head *arrvq,
>  /* tipc_sk_push_backlog(): send accumulated buffers in socket write queue
>   *                         when socket is in Nagle mode
>   */
> -static void tipc_sk_push_backlog(struct tipc_sock *tsk, bool nagle_ack)
> +static void tipc_sk_push_backlog(struct tipc_sock *tsk, bool nagle_ack,
> +				 struct sk_buff_head *xmitq)
>  {
>  	struct sk_buff_head *txq = &tsk->sk.sk_write_queue;
>  	struct sk_buff *skb = skb_peek_tail(txq);
> @@ -1310,6 +1312,12 @@ static void tipc_sk_push_backlog(struct tipc_sock *tsk, bool nagle_ack)
>  		tsk->pkt_cnt += skb_queue_len(txq);
>  	tsk->snt_unacked += tsk->snd_backlog;
>  	tsk->snd_backlog = 0;
> +
> +	if (xmitq && in_own_node(net, dnode)) {
> +		skb_queue_splice_tail_init(txq, xmitq);
> +		return;
> +	}
> +
>  	rc = tipc_node_xmit(net, txq, dnode, tsk->portid);
>  	if (rc == -ELINKCONG)
>  		tsk->cong_link_cnt = 1;
> @@ -1367,7 +1375,7 @@ static void tipc_sk_conn_proto_rcv(struct tipc_sock *tsk, struct sk_buff *skb,
>  			goto exit;
>  
>  		was_cong = tsk_conn_cong(tsk);
> -		tipc_sk_push_backlog(tsk, msg_nagle_ack(hdr));
> +		tipc_sk_push_backlog(tsk, msg_nagle_ack(hdr), xmitq);
>  		tsk->snt_unacked -= msg_conn_ack(hdr);
>  		if (tsk->peer_caps & TIPC_BLOCK_FLOWCTL)
>  			tsk->snd_win = msg_adv_win(hdr);
> @@ -2165,7 +2173,7 @@ static void tipc_sk_proto_rcv(struct sock *sk,
>  		smp_wmb();
>  		tsk->cong_link_cnt--;
>  		wakeup = true;
> -		tipc_sk_push_backlog(tsk, false);
> +		tipc_sk_push_backlog(tsk, false, xmitq);
>  		break;
>  	case GROUP_PROTOCOL:
>  		tipc_group_proto_rcv(grp, &wakeup, hdr, inputq, xmitq);
> @@ -2256,7 +2264,7 @@ static bool tipc_sk_filter_connect(struct tipc_sock *tsk, struct sk_buff *skb,
>  		return false;
>  	case TIPC_ESTABLISHED:
>  		if (!skb_queue_empty(&sk->sk_write_queue))
> -			tipc_sk_push_backlog(tsk, false);
> +			tipc_sk_push_backlog(tsk, false, xmitq);
>  		/* Accept only connection-based messages sent by peer */
>  		if (likely(con_msg && !err && pport == oport &&
>  			   pnode == onode)) {
> 
> 
> base-commit: 1590cf0329716306e948a8fc29f1d3ee87d3989f
> -- 
> This is an AI-generated patch subject to moderation.
> Reply with '#syz upstream' to Sign-off the patch as a human author
> and send it to the upstream kernel mailing lists.
> Reply with '#syz reject' to reject it ('#syz unreject' to undo).
> 
> See https://goo.gle/syzbot-ai-patches for information about AI-generated patches.
> You can comment on the patch as usual, syzbot will try to address
> the comments and send a new version of the patch if necessary.
> syzbot engineers can be reached at syzkaller@googlegroups.com.

^ permalink raw reply	[flat|nested] 8+ messages in thread
* [syzbot] [tipc?] BUG: soft lockup in do_sock_setsockopt
@ 2024-03-22 14:23 syzbot
  2024-03-23  0:02 ` Hillf Danton
  2024-10-03  9:25 ` syzbot
  0 siblings, 2 replies; 8+ messages in thread
From: syzbot @ 2024-03-22 14:23 UTC (permalink / raw)
  To: davem, edumazet, jmaloy, kuba, linux-kernel, netdev, pabeni,
	syzkaller-bugs, tipc-discussion, ying.xue

Hello,

syzbot found the following issue on:

HEAD commit:    707081b61156 Merge branch 'for-next/core', remote-tracking..
git tree:       git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci
console output: https://syzkaller.appspot.com/x/log.txt?x=16d23231180000
kernel config:  https://syzkaller.appspot.com/x/.config?x=caeac3f3565b057a
dashboard link: https://syzkaller.appspot.com/bug?extid=10a41dc44eef71aa9450
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
userspace arch: arm64
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=126fea6e180000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=114e4c81180000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/6cad68bf7532/disk-707081b6.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/1a27e5400778/vmlinux-707081b6.xz
kernel image: https://storage.googleapis.com/syzbot-assets/67dfc53755d0/Image-707081b6.gz.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+10a41dc44eef71aa9450@syzkaller.appspotmail.com

watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [syz-executor178:13146]
Modules linked in:
irq event stamp: 29265101
hardirqs last  enabled at (29265100): [<ffff8000801dae10>] __local_bh_enable_ip+0x224/0x44c kernel/softirq.c:386
hardirqs last disabled at (29265101): [<ffff80008ad66a78>] __el1_irq arch/arm64/kernel/entry-common.c:533 [inline]
hardirqs last disabled at (29265101): [<ffff80008ad66a78>] el1_interrupt+0x24/0x68 arch/arm64/kernel/entry-common.c:551
softirqs last  enabled at (638): [<ffff80008a7fe730>] spin_unlock_bh include/linux/spinlock.h:396 [inline]
softirqs last  enabled at (638): [<ffff80008a7fe730>] tipc_skb_peek_port net/tipc/msg.h:1235 [inline]
softirqs last  enabled at (638): [<ffff80008a7fe730>] tipc_sk_rcv+0x34c/0x1888 net/tipc/socket.c:2494
softirqs last disabled at (640): [<ffff80008a7fe750>] spin_trylock_bh include/linux/spinlock.h:411 [inline]
softirqs last disabled at (640): [<ffff80008a7fe750>] tipc_sk_rcv+0x36c/0x1888 net/tipc/socket.c:2499
CPU: 0 PID: 13146 Comm: syz-executor178 Not tainted 6.8.0-rc7-syzkaller-g707081b61156 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : lock_acquire+0x278/0x71c
lr : lockdep_recursion_finish kernel/locking/lockdep.c:467 [inline]
lr : lock_acquire+0x248/0x71c kernel/locking/lockdep.c:5756
sp : ffff80009ab86080
x29: ffff80009ab86190 x28: dfff800000000000 x27: ffff700013570c1c
x26: ffff0001b3fffdc0 x25: ffff80008ee74ac0 x24: 0000000000000000
x23: 0000000000000000 x22: 0000000000000000 x21: 0000000000000000
x20: ffff80008ebebdc0 x19: ffff0001b3fffdc0 x18: ffff80009ab862e0
x17: 000000000000c55c x16: ffff80008ad6b1c0 x15: 0000000000000001
x14: ffff80008eca0458 x13: dfff800000000000 x12: 00000000af1c601b
x11: 00000000a4e03a31 x10: 0000000000000003 x9 : 0000000000000000
x8 : 00000000000000c0 x7 : ffff80008a80b3f8 x6 : 0000000000000000
x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000000002
x2 : 0000000000000008 x1 : ffff80008aedfba0 x0 : 0000000000000000
Call trace:
 __daif_local_irq_restore arch/arm64/include/asm/irqflags.h:176 [inline]
 arch_local_irq_restore arch/arm64/include/asm/irqflags.h:196 [inline]
 lock_acquire+0x278/0x71c kernel/locking/lockdep.c:5757
 rcu_lock_acquire+0x40/0x4c include/linux/rcupdate.h:298
 rcu_read_lock include/linux/rcupdate.h:750 [inline]
 tipc_sk_lookup+0xc8/0x8b4 net/tipc/socket.c:3003
 tipc_sk_rcv+0x358/0x1888 net/tipc/socket.c:2495
 tipc_node_xmit+0x1b0/0xdb0 net/tipc/node.c:1703
 tipc_node_xmit_skb net/tipc/node.c:1768 [inline]
 tipc_node_distr_xmit+0x28c/0x3a4 net/tipc/node.c:1783
 tipc_sk_rcv+0x1280/0x1888 net/tipc/socket.c:2504
 tipc_node_xmit+0x1b0/0xdb0 net/tipc/node.c:1703
 tipc_sk_push_backlog net/tipc/socket.c:1317 [inline]
 tipc_sk_filter_connect net/tipc/socket.c:2258 [inline]
 tipc_sk_filter_rcv+0x13f8/0x2cac net/tipc/socket.c:2367
 tipc_sk_enqueue net/tipc/socket.c:2448 [inline]
 tipc_sk_rcv+0x6d0/0x1888 net/tipc/socket.c:2500
 tipc_node_xmit+0x1b0/0xdb0 net/tipc/node.c:1703
 tipc_node_xmit_skb net/tipc/node.c:1768 [inline]
 tipc_node_distr_xmit+0x28c/0x3a4 net/tipc/node.c:1783
 tipc_sk_backlog_rcv+0x164/0x214 net/tipc/socket.c:2415
 sk_backlog_rcv include/net/sock.h:1092 [inline]
 __release_sock+0x1a8/0x408 net/core/sock.c:2972
 release_sock+0x68/0x1b8 net/core/sock.c:3538
 sk_setsockopt+0xbdc/0x306c
 sock_setsockopt+0x68/0x80 net/core/sock.c:1548
 do_sock_setsockopt+0x238/0x4e0 net/socket.c:2307
 __sys_setsockopt+0x128/0x1a8 net/socket.c:2334
 __do_sys_setsockopt net/socket.c:2343 [inline]
 __se_sys_setsockopt net/socket.c:2340 [inline]
 __arm64_sys_setsockopt+0xb8/0xd4 net/socket.c:2340
 __invoke_syscall arch/arm64/kernel/syscall.c:34 [inline]
 invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:48
 el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:133
 do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:152
 el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:712
 el0t_64_sync_handler+0x84/0xfc arch/arm64/kernel/entry-common.c:730
 el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:598


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

^ permalink raw reply	[flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-08-11 14:48 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <ob7cvq6bqedi3rsiyjlcivgfj6mxttkrcrwdciny4brsdpm7yv@zvu3gi5n5cuz>
2026-08-10 14:00 ` [syzbot] [tipc?] BUG: soft lockup in do_sock_setsockopt syzbot
2026-08-11 10:25   ` Bartosz Chronowski
2026-08-11 14:48     ` syzbot
2026-08-07  8:54 [PATCH RFC v2] tipc: fix spinlock recursion in tipc_sk_rcv() Bartosz Chronowski
2026-08-07 19:26 ` [syzbot] [tipc?] BUG: soft lockup in do_sock_setsockopt syzbot
  -- strict thread matches above, loose matches on Subject: below --
2024-03-22 14:23 syzbot
2024-03-23  0:02 ` Hillf Danton
2024-03-23  8:44   ` syzbot
2024-10-03  9:25 ` syzbot

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.