From: Kuniyuki Iwashima <kuniyu@amazon.com>
To: <fw@strlen.de>
Cc: <davem@davemloft.net>, <dsahern@kernel.org>,
<edumazet@google.com>, <kuba@kernel.org>, <kuniyu@amazon.com>,
<linux-kernel@vger.kernel.org>, <netdev@vger.kernel.org>,
<pabeni@redhat.com>,
<syzbot+8ea26396ff85d23a8929@syzkaller.appspotmail.com>,
<syzkaller-bugs@googlegroups.com>
Subject: Re: [syzbot] [net?] WARNING: refcount bug in inet_twsk_kill
Date: Sun, 11 Aug 2024 15:35:30 -0700 [thread overview]
Message-ID: <20240811223530.92827-1-kuniyu@amazon.com> (raw)
In-Reply-To: <20240811133200.GC13736@breakpoint.cc>
From: Florian Westphal <fw@strlen.de>
Date: Sun, 11 Aug 2024 15:32:00 +0200
> > > ------------[ cut here ]------------
> > > refcount_t: decrement hit 0; leaking memory.
> > > WARNING: CPU: 3 PID: 1396 at lib/refcount.c:31 refcount_warn_saturate+0x1ed/0x210 lib/refcount.c:31
> >
> > Eric, this is the weird report I was talking about at netdevconf :)
> >
> > It seems refcount_dec(&tw->tw_dr->tw_refcount) is somehow done earlier
> > than refcount_inc().
> >
> > I started to see the same splat at a very low rate after consuming
> > commit b334b924c9b7 ("net: tcp/dccp: prepare for tw_timer un-pinning").
>
> Could you share a splat you saw?
>
> Was it also for the fallback tcp_hashinfo and/or from error
> unwindinding after a ops->init() failure on netns creation?
>
> I don't think it makes sense following this specific report given
> b334b924c9b7 made changes in this area, but a post-b334b924c9b7
> would be more relevant I think.
Ah, I missed the initial syzbot's report was created before the commit.
Here's splats I saw:
[ Note both splats were mixed in the log as with the syzbot's report,
so two threads were calling tcp_sk_exit_batch() concurrently. ]
---8<---
refcount_t: decrement hit 0; leaking memory.
WARNING: CPU: 0 PID: 1760 at lib/refcount.c:31 refcount_warn_saturate (lib/refcount.c:31 (discriminator 3))
Modules linked in:
CPU: 0 PID: 1760 Comm: syz-executor.4 Not tainted 6.10.0-rc3-00688-g934c29999b57-dirty #20
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org 04/01/2014
RIP: 0010:refcount_warn_saturate (lib/refcount.c:31 (discriminator 3))
Code: 05 cd 8d aa 04 01 e8 48 e4 e9 fe 0f 0b e9 d3 fe ff ff e8 1c 36 1c ff 48 c7 c7 a0 4a 80 85 c6 05 aa 8d aa 04 01 e8 29 e4 e9 fe <0f> 0b e9 b4 fe ff ff 48 89 ef e8 fa 56 69 ff e9 5c fe ff ff 0f 1f
RSP: 0018:ffffc9000118fa78 EFLAGS: 00010286
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffc9000b13b000
RDX: 0000000000040000 RSI: ffffffff8118dedf RDI: 0000000000000001
RBP: ffff88800bb59440 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000000 R11: 000000002d2d2d2d R12: ffff88800bb59440
R13: ffff88810130f3d0 R14: ffff8881013ee820 R15: ffff888102001198
FS: 00007f5252dbd640(0000) GS:ffff88811ae00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000004a6128 CR3: 00000001038b8003 CR4: 0000000000770ef0
PKRU: 55555554
Call Trace:
<TASK>
inet_twsk_kill (./include/linux/refcount.h:336 ./include/linux/refcount.h:351 net/ipv4/inet_timewait_sock.c:70)
inet_twsk_deschedule_put (net/ipv4/inet_timewait_sock.c:266)
inet_twsk_purge (net/ipv4/inet_timewait_sock.c:349)
tcp_twsk_purge (net/ipv4/tcp_minisocks.c:402)
tcp_sk_exit_batch (net/ipv4/tcp_ipv4.c:3512)
ops_exit_list (net/core/net_namespace.c:179)
setup_net (net/core/net_namespace.c:374 (discriminator 3))
copy_net_ns (net/core/net_namespace.c:510)
create_new_namespaces (kernel/nsproxy.c:110)
unshare_nsproxy_namespaces (kernel/nsproxy.c:228 (discriminator 4))
ksys_unshare (kernel/fork.c:3325)
__x64_sys_unshare (kernel/fork.c:3392)
do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130)
RIP: 0033:0x7f5253a29e5d
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 73 9f 1b 00 f7 d8 64 89 01 48
RSP: 002b:00007f5252dbcc88 EFLAGS: 00000246 ORIG_RAX: 0000000000000110
RAX: ffffffffffffffda RBX: 00000000004d4070 RCX: 00007f5253a29e5d
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000040000000
RBP: 00000000004d4070 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000006e R14: 00007f5253a8a530 R15: 0000000000000000
</TASK>
WARNING: CPU: 1 PID: 671 at net/ipv4/tcp_ipv4.c:3514 tcp_sk_exit_batch (net/ipv4/tcp_ipv4.c:3514)
Modules linked in:
CPU: 1 PID: 671 Comm: kworker/u8:15 Not tainted 6.10.0-rc3-00688-g934c29999b57-dirty #20
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org 04/01/2014
Workqueue: netns cleanup_net
RIP: 0010:tcp_sk_exit_batch (net/ipv4/tcp_ipv4.c:3514)
Code: 89 ee e8 9f 1b 28 fd 85 ed 0f 88 b3 00 00 00 e8 92 22 28 fd 31 ff 89 ee e8 89 1b 28 fd 85 ed 0f 84 9d 00 00 00 e8 7c 22 28 fd <0f> 0b e8 75 22 28 fd 48 89 df e8 ed 23 02 00 48 8d 7b 28 48 89 f8
RSP: 0018:ffffc90000ebfc60 EFLAGS: 00010293
RAX: 0000000000000000 RBX: ffff88800bb59140 RCX: ffffffff84230cb7
RDX: ffff888103871180 RSI: ffffffff84230cc4 RDI: 0000000000000005
RBP: 0000000000000002 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000002 R11: 00000000000002bd R12: ffff88800bb59440
R13: dffffc0000000000 R14: ffffc90000ebfd08 R15: fffffbfff0d7fb5c
FS: 0000000000000000(0000) GS:ffff88811af00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f274114ddf8 CR3: 000000010a11c006 CR4: 0000000000770ef0
PKRU: 55555554
Call Trace:
<TASK>
ops_exit_list (net/core/net_namespace.c:179)
cleanup_net (net/core/net_namespace.c:639 (discriminator 3))
process_one_work (kernel/workqueue.c:3236)
worker_thread (kernel/workqueue.c:3306 kernel/workqueue.c:3393)
kthread (kernel/kthread.c:389)
ret_from_fork (arch/x86/kernel/process.c:153)
ret_from_fork_asm (arch/x86/entry/entry_64.S:257)
</TASK>
---8<---
prev parent reply other threads:[~2024-08-11 22:35 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-08-11 1:29 [syzbot] [net?] WARNING: refcount bug in inet_twsk_kill syzbot
2024-08-11 2:29 ` Kuniyuki Iwashima
2024-08-11 5:42 ` Jason Xing
2024-08-11 13:24 ` Florian Westphal
2024-08-11 14:54 ` Florian Westphal
2024-08-11 16:28 ` Florian Westphal
2024-08-11 23:00 ` Kuniyuki Iwashima
2024-08-11 23:08 ` Kuniyuki Iwashima
2024-08-12 0:36 ` Jason Xing
2024-08-12 14:01 ` Florian Westphal
2024-08-12 14:30 ` Jason Xing
2024-08-12 15:03 ` Florian Westphal
2024-08-12 15:49 ` Jason Xing
2024-08-12 20:00 ` Kuniyuki Iwashima
2024-08-12 22:28 ` [PATCH net] tcp: prevent concurrent execution of tcp_sk_exit_batch Florian Westphal
2024-08-12 23:28 ` Kuniyuki Iwashima
2024-08-12 23:52 ` Florian Westphal
2024-08-13 0:01 ` Kuniyuki Iwashima
2024-08-13 2:48 ` Jason Xing
2024-08-15 10:47 ` Paolo Abeni
2024-08-19 15:36 ` Eric Dumazet
2024-08-19 15:50 ` patchwork-bot+netdevbpf
2024-08-11 13:32 ` [syzbot] [net?] WARNING: refcount bug in inet_twsk_kill Florian Westphal
2024-08-11 22:35 ` Kuniyuki Iwashima [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240811223530.92827-1-kuniyu@amazon.com \
--to=kuniyu@amazon.com \
--cc=davem@davemloft.net \
--cc=dsahern@kernel.org \
--cc=edumazet@google.com \
--cc=fw@strlen.de \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=syzbot+8ea26396ff85d23a8929@syzkaller.appspotmail.com \
--cc=syzkaller-bugs@googlegroups.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).