* Re: [syzbot] [net?] WARNING: suspicious RCU usage in dev_activate
2024-07-03 10:07 [syzbot] [net?] WARNING: suspicious RCU usage in dev_activate syzbot
@ 2024-07-03 10:10 ` Eric Dumazet
2024-07-03 11:56 ` Johannes Berg
2024-09-12 19:13 ` syzbot
2024-09-16 3:23 ` syzbot
2 siblings, 1 reply; 5+ messages in thread
From: Eric Dumazet @ 2024-07-03 10:10 UTC (permalink / raw)
To: syzbot, Johannes Berg
Cc: davem, jhs, jiri, kuba, linux-kernel, netdev, pabeni,
syzkaller-bugs, xiyou.wangcong
On Wed, Jul 3, 2024 at 12:07 PM syzbot
<syzbot+2120b9a8f96b3fa90bad@syzkaller.appspotmail.com> wrote:
>
> Hello,
>
> syzbot found the following issue on:
>
> HEAD commit: 1dfe225e9af5 Merge tag 'scsi-fixes' of git://git.kernel.or..
> git tree: upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=14a4f2d1980000
> kernel config: https://syzkaller.appspot.com/x/.config?x=1ace69f521989b1f
> dashboard link: https://syzkaller.appspot.com/bug?extid=2120b9a8f96b3fa90bad
> compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
>
> Unfortunately, I don't have any reproducer for this issue yet.
>
> Downloadable assets:
> disk image: https://storage.googleapis.com/syzbot-assets/3894cf8b5271/disk-1dfe225e.raw.xz
> vmlinux: https://storage.googleapis.com/syzbot-assets/573c202ade8f/vmlinux-1dfe225e.xz
> kernel image: https://storage.googleapis.com/syzbot-assets/a356d869b8f3/bzImage-1dfe225e.xz
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+2120b9a8f96b3fa90bad@syzkaller.appspotmail.com
>
> =============================
> WARNING: suspicious RCU usage
> 6.10.0-rc6-syzkaller-00051-g1dfe225e9af5 #0 Not tainted
> -----------------------------
> net/sched/sch_generic.c:1249 suspicious rcu_dereference_protected() usage!
>
> other info that might help us debug this:
>
>
> rcu_scheduler_active = 2, debug_locks = 1
> 3 locks held by kworker/u8:0/11:
> #0: ffff88801efaa148 ((wq_completion)bond0#9){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3223 [inline]
> #0: ffff88801efaa148 ((wq_completion)bond0#9){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3329
> #1: ffffc90000107d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3224 [inline]
> #1: ffffc90000107d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3329
> #2: ffffffff8e333f20 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:329 [inline]
> #2: ffffffff8e333f20 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:781 [inline]
> #2: ffffffff8e333f20 (rcu_read_lock){....}-{1:2}, at: bond_mii_monitor+0x174/0x3170 drivers/net/bonding/bond_main.c:2824
>
> stack backtrace:
> CPU: 0 PID: 11 Comm: kworker/u8:0 Not tainted 6.10.0-rc6-syzkaller-00051-g1dfe225e9af5 #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/07/2024
> Workqueue: bond0 bond_mii_monitor
> Call Trace:
> <TASK>
> __dump_stack lib/dump_stack.c:88 [inline]
> dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
> lockdep_rcu_suspicious+0x221/0x340 kernel/locking/lockdep.c:6712
> dev_activate+0xf8/0x1240 net/sched/sch_generic.c:1249
> linkwatch_do_dev+0xfb/0x170 net/core/link_watch.c:173
> ethtool_op_get_link+0x15/0x60 net/ethtool/ioctl.c:62
> bond_check_dev_link+0x1f1/0x3f0 drivers/net/bonding/bond_main.c:757
> bond_miimon_inspect drivers/net/bonding/bond_main.c:2604 [inline]
> bond_mii_monitor+0x49a/0x3170 drivers/net/bonding/bond_main.c:2826
> process_one_work kernel/workqueue.c:3248 [inline]
> process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3329
> worker_thread+0x86d/0xd50 kernel/workqueue.c:3409
> kthread+0x
>
>
> ---
> This report is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at syzkaller@googlegroups.com.
>
> syzbot will keep track of this issue. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
>
> If the report is already addressed, let syzbot know by replying with:
> #syz fix: exact-commit-title
>
> If you want to overwrite report's subsystems, reply with:
> #syz set subsystems: new-subsystem
> (See the list of subsystem names on the web dashboard)
>
> If the report is a duplicate of another one, reply with:
> #syz dup: exact-subject-of-another-report
>
> If you want to undo deduplication, reply with:
> #syz undup
I think this came with this patch :
commit facd15dfd69122042502d99ab8c9f888b48ee994
Author: Johannes Berg <johannes.berg@intel.com>
Date: Mon Dec 4 21:47:07 2023 +0100
net: core: synchronize link-watch when carrier is queried
Issue here is that ethtool_op_get_link() could be called from RCU contexts.
Adding linkwatch_sync_dev() in it broke this case.
BTW, this commit also made it difficult to convert "ip link" dumps to
not use RTNL, but rely on RCU instead.
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [syzbot] [net?] WARNING: suspicious RCU usage in dev_activate
2024-07-03 10:10 ` Eric Dumazet
@ 2024-07-03 11:56 ` Johannes Berg
0 siblings, 0 replies; 5+ messages in thread
From: Johannes Berg @ 2024-07-03 11:56 UTC (permalink / raw)
To: Eric Dumazet, syzbot
Cc: davem@davemloft.net, Hadi Salim, Jamal, jiri@resnulli.us,
kuba@kernel.org, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org, pabeni@redhat.com,
syzkaller-bugs@googlegroups.com, xiyou.wangcong@gmail.com
On Wed, 2024-07-03 at 10:10 +0000, Eric Dumazet wrote:
> I think this came with this patch :
>
> commit facd15dfd69122042502d99ab8c9f888b48ee994
> Author: Johannes Berg <johannes.berg@intel.com>
> Date: Mon Dec 4 21:47:07 2023 +0100
>
> net: core: synchronize link-watch when carrier is queried
Yes, that makes sense.
> Issue here is that ethtool_op_get_link() could be called from RCU contexts.
That's ... unexpected, and apparently not just to me. First (but not
only) example I found: usbnet_get_link() calls mii_link_ok() calls the
driver's mdio_read(), i.e. sr_mdio_read() which does a mutex_lock().
And it was always broken? I can't really find anything that introduced
this problem directly - even before 4cb4f97b7e36 you had a read_lock()
there, and the bond slave_dev->ethtool_ops->get_link() call goes back to
the beginning of git history, the usbnet example is just slightly newer
from c41286fd42f3, but even before that there are examples in usbnet
drivers with this problem.
> Adding linkwatch_sync_dev() in it broke this case.
Right, I agree that it made the problem much more wide-spread than it
being driver-dependent as it was before.
Perhaps we should change bond? The original commit that added RCU there
said it even considered RTNL instead.
At the very least I'd say some _big_ documentation is needed there in
ethtool, and probably then __ethtool_get_link() should also
rcu_read_lock() to make this consistent. But like I said, I'm not sure
it isn't bond that's on the wrong side here.
> BTW, this commit also made it difficult to convert "ip link" dumps to
> not use RTNL, but rely on RCU instead.
You could probably sync all of linswatch beforehand, and only acquire
the RTNL if there's work to be done at all?
johannes
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [syzbot] [net?] WARNING: suspicious RCU usage in dev_activate
2024-07-03 10:07 [syzbot] [net?] WARNING: suspicious RCU usage in dev_activate syzbot
2024-07-03 10:10 ` Eric Dumazet
@ 2024-09-12 19:13 ` syzbot
2024-09-16 3:23 ` syzbot
2 siblings, 0 replies; 5+ messages in thread
From: syzbot @ 2024-09-12 19:13 UTC (permalink / raw)
To: davem, edumazet, hdanton, jhs, jiri, johannes.berg, johannes, jv,
kuba, linux-kernel, netdev, pabeni, syzkaller-bugs,
xiyou.wangcong
syzbot has found a reproducer for the following issue on:
HEAD commit: 77f587896757 Merge tag 'arm-fixes-6.11-3' of git://git.ker..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=1781149f980000
kernel config: https://syzkaller.appspot.com/x/.config?x=61d235cb8d15001c
dashboard link: https://syzkaller.appspot.com/bug?extid=2120b9a8f96b3fa90bad
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=128160a9980000
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/3d43e4fb8f51/disk-77f58789.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/3f2facb83939/vmlinux-77f58789.xz
kernel image: https://storage.googleapis.com/syzbot-assets/26e67580882c/bzImage-77f58789.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+2120b9a8f96b3fa90bad@syzkaller.appspotmail.com
=============================
WARNING: suspicious RCU usage
6.11.0-rc7-syzkaller-00039-g77f587896757 #0 Not tainted
-----------------------------
net/sched/sch_generic.c:1250 suspicious rcu_dereference_protected() usage!
other info that might help us debug this:
rcu_scheduler_active = 2, debug_locks = 1
3 locks held by kworker/u8:4/62:
#0: ffff888011c52948 ((wq_completion)bond0#2){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3206 [inline]
#0: ffff888011c52948 ((wq_completion)bond0#2){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3312
#1: ffffc900015d7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3207 [inline]
#1: ffffc900015d7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3312
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:326 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:838 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: bond_mii_monitor+0x174/0x3170 drivers/net/bonding/bond_main.c:2863
stack backtrace:
CPU: 1 UID: 0 PID: 62 Comm: kworker/u8:4 Not tainted 6.11.0-rc7-syzkaller-00039-g77f587896757 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Workqueue: bond0 bond_mii_monitor
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:93 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:119
lockdep_rcu_suspicious+0x221/0x340 kernel/locking/lockdep.c:6724
dev_activate+0xf8/0x1240 net/sched/sch_generic.c:1250
linkwatch_do_dev+0xfb/0x170 net/core/link_watch.c:173
ethtool_op_get_link+0x15/0x60 net/ethtool/ioctl.c:62
bond_check_dev_link+0x1f1/0x3f0 drivers/net/bonding/bond_main.c:799
bond_miimon_inspect drivers/net/bonding/bond_main.c:2643 [inline]
bond_mii_monitor+0x49a/0x3170 drivers/net/bonding/bond_main.c:2865
process_one_work kernel/workqueue.c:3231 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
worker_thread+0x86d/0xd10 kernel/workqueue.c:3389
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
=============================
WARNING: suspicious RCU usage
6.11.0-rc7-syzkaller-00039-g77f587896757 #0 Not tainted
-----------------------------
net/sched/sch_generic.c:1228 suspicious rcu_dereference_protected() usage!
other info that might help us debug this:
rcu_scheduler_active = 2, debug_locks = 1
3 locks held by kworker/u8:4/62:
#0: ffff888011c52948 ((wq_completion)bond0#2){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3206 [inline]
#0: ffff888011c52948 ((wq_completion)bond0#2){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3312
#1: ffffc900015d7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3207 [inline]
#1: ffffc900015d7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3312
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:326 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:838 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: bond_mii_monitor+0x174/0x3170 drivers/net/bonding/bond_main.c:2863
stack backtrace:
CPU: 1 UID: 0 PID: 62 Comm: kworker/u8:4 Not tainted 6.11.0-rc7-syzkaller-00039-g77f587896757 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Workqueue: bond0 bond_mii_monitor
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:93 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:119
lockdep_rcu_suspicious+0x221/0x340 kernel/locking/lockdep.c:6724
transition_one_qdisc+0x8e/0x1c0 net/sched/sch_generic.c:1228
netdev_for_each_tx_queue include/linux/netdevice.h:2513 [inline]
dev_activate+0x838/0x1240 net/sched/sch_generic.c:1258
linkwatch_do_dev+0xfb/0x170 net/core/link_watch.c:173
ethtool_op_get_link+0x15/0x60 net/ethtool/ioctl.c:62
bond_check_dev_link+0x1f1/0x3f0 drivers/net/bonding/bond_main.c:799
bond_miimon_inspect drivers/net/bonding/bond_main.c:2643 [inline]
bond_mii_monitor+0x49a/0x3170 drivers/net/bonding/bond_main.c:2865
process_one_work kernel/workqueue.c:3231 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
worker_thread+0x86d/0xd10 kernel/workqueue.c:3389
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
=============================
WARNING: suspicious RCU usage
6.11.0-rc7-syzkaller-00039-g77f587896757 #0 Not tainted
-----------------------------
include/linux/rtnetlink.h:100 suspicious rcu_dereference_protected() usage!
other info that might help us debug this:
rcu_scheduler_active = 2, debug_locks = 1
3 locks held by kworker/u8:4/62:
#0: ffff888011c52948 ((wq_completion)bond0#2){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3206 [inline]
#0: ffff888011c52948 ((wq_completion)bond0#2){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3312
#1: ffffc900015d7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3207 [inline]
#1: ffffc900015d7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3312
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:326 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:838 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: bond_mii_monitor+0x174/0x3170 drivers/net/bonding/bond_main.c:2863
stack backtrace:
CPU: 0 UID: 0 PID: 62 Comm: kworker/u8:4 Not tainted 6.11.0-rc7-syzkaller-00039-g77f587896757 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Workqueue: bond0 bond_mii_monitor
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:93 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:119
lockdep_rcu_suspicious+0x221/0x340 kernel/locking/lockdep.c:6724
dev_ingress_queue include/linux/rtnetlink.h:100 [inline]
dev_activate+0x925/0x1240 net/sched/sch_generic.c:1259
linkwatch_do_dev+0xfb/0x170 net/core/link_watch.c:173
ethtool_op_get_link+0x15/0x60 net/ethtool/ioctl.c:62
bond_check_dev_link+0x1f1/0x3f0 drivers/net/bonding/bond_main.c:799
bond_miimon_inspect drivers/net/bonding/bond_main.c:2643 [inline]
bond_mii_monitor+0x49a/0x3170 drivers/net/bonding/bond_main.c:2865
process_one_work kernel/workqueue.c:3231 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
worker_thread+0x86d/0xd10 kernel/workqueue.c:3389
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
BUG: sleeping function called from invalid context at kernel/locking/rwsem.c:1525
in_atomic(): 0, irqs_disabled(): 0, non_block: 0, pid: 62, name: kworker/u8:4
preempt_count: 0, expected: 0
RCU nest depth: 1, expected: 0
3 locks held by kworker/u8:4/62:
#0: ffff888011c52948 ((wq_completion)bond0#2){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3206 [inline]
#0: ffff888011c52948 ((wq_completion)bond0#2){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3312
#1: ffffc900015d7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3207 [inline]
#1: ffffc900015d7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3312
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:326 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:838 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: bond_mii_monitor+0x174/0x3170 drivers/net/bonding/bond_main.c:2863
CPU: 0 UID: 0 PID: 62 Comm: kworker/u8:4 Not tainted 6.11.0-rc7-syzkaller-00039-g77f587896757 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Workqueue: bond0 bond_mii_monitor
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:93 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:119
__might_resched+0x5d4/0x780 kernel/sched/core.c:8463
down_read+0x8e/0xa40 kernel/locking/rwsem.c:1525
wireless_nlevent_flush net/wireless/wext-core.c:351 [inline]
wext_netdev_notifier_call+0x1f/0x120 net/wireless/wext-core.c:371
notifier_call_chain+0x19f/0x3e0 kernel/notifier.c:93
netdev_state_change+0x11f/0x1a0 net/core/dev.c:1376
linkwatch_do_dev+0x112/0x170 net/core/link_watch.c:177
ethtool_op_get_link+0x15/0x60 net/ethtool/ioctl.c:62
bond_check_dev_link+0x1f1/0x3f0 drivers/net/bonding/bond_main.c:799
bond_miimon_inspect drivers/net/bonding/bond_main.c:2643 [inline]
bond_mii_monitor+0x49a/0x3170 drivers/net/bonding/bond_main.c:2865
process_one_work kernel/workqueue.c:3231 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
worker_thread+0x86d/0xd10 kernel/workqueue.c:3389
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
=============================
[ BUG: Invalid wait context ]
6.11.0-rc7-syzkaller-00039-g77f587896757 #0 Tainted: G W
-----------------------------
kworker/u8:4/62 is trying to lock:
ffffffff8fc7f690 (net_rwsem){++++}-{3:3}, at: wireless_nlevent_flush net/wireless/wext-core.c:351 [inline]
ffffffff8fc7f690 (net_rwsem){++++}-{3:3}, at: wext_netdev_notifier_call+0x1f/0x120 net/wireless/wext-core.c:371
other info that might help us debug this:
context-{4:4}
3 locks held by kworker/u8:4/62:
#0: ffff888011c52948 ((wq_completion)bond0#2){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3206 [inline]
#0: ffff888011c52948 ((wq_completion)bond0#2){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3312
#1: ffffc900015d7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3207 [inline]
#1: ffffc900015d7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3312
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:326 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:838 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: bond_mii_monitor+0x174/0x3170 drivers/net/bonding/bond_main.c:2863
stack backtrace:
CPU: 0 UID: 0 PID: 62 Comm: kworker/u8:4 Tainted: G W 6.11.0-rc7-syzkaller-00039-g77f587896757 #0
Tainted: [W]=WARN
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Workqueue: bond0 bond_mii_monitor
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:93 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:119
print_lock_invalid_wait_context kernel/locking/lockdep.c:4750 [inline]
check_wait_context kernel/locking/lockdep.c:4820 [inline]
__lock_acquire+0x153b/0x2040 kernel/locking/lockdep.c:5092
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
down_read+0xb1/0xa40 kernel/locking/rwsem.c:1526
wireless_nlevent_flush net/wireless/wext-core.c:351 [inline]
wext_netdev_notifier_call+0x1f/0x120 net/wireless/wext-core.c:371
notifier_call_chain+0x19f/0x3e0 kernel/notifier.c:93
netdev_state_change+0x11f/0x1a0 net/core/dev.c:1376
linkwatch_do_dev+0x112/0x170 net/core/link_watch.c:177
ethtool_op_get_link+0x15/0x60 net/ethtool/ioctl.c:62
bond_check_dev_link+0x1f1/0x3f0 drivers/net/bonding/bond_main.c:799
bond_miimon_inspect drivers/net/bonding/bond_main.c:2643 [inline]
bond_mii_monitor+0x49a/0x3170 drivers/net/bonding/bond_main.c:2865
process_one_work kernel/workqueue.c:3231 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
worker_thread+0x86d/0xd10 kernel/workqueue.c:3389
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
---
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [syzbot] [net?] WARNING: suspicious RCU usage in dev_activate
2024-07-03 10:07 [syzbot] [net?] WARNING: suspicious RCU usage in dev_activate syzbot
2024-07-03 10:10 ` Eric Dumazet
2024-09-12 19:13 ` syzbot
@ 2024-09-16 3:23 ` syzbot
2 siblings, 0 replies; 5+ messages in thread
From: syzbot @ 2024-09-16 3:23 UTC (permalink / raw)
To: davem, edumazet, hdanton, jhs, jiri, johannes.berg, johannes, jv,
kuba, linux-kernel, netdev, pabeni, syzkaller-bugs,
xiyou.wangcong
syzbot has found a reproducer for the following issue on:
HEAD commit: d42f7708e27c Merge tag 'for-linus-6.11' of git://git.kerne..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=1187229f980000
kernel config: https://syzkaller.appspot.com/x/.config?x=61d235cb8d15001c
dashboard link: https://syzkaller.appspot.com/bug?extid=2120b9a8f96b3fa90bad
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=15c828a9980000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1587229f980000
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/92f659e09436/disk-d42f7708.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/1e3b5ab99922/vmlinux-d42f7708.xz
kernel image: https://storage.googleapis.com/syzbot-assets/1c82035e4680/bzImage-d42f7708.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+2120b9a8f96b3fa90bad@syzkaller.appspotmail.com
=============================
WARNING: suspicious RCU usage
6.11.0-rc7-syzkaller-00151-gd42f7708e27c #0 Not tainted
-----------------------------
net/sched/sch_generic.c:1250 suspicious rcu_dereference_protected() usage!
other info that might help us debug this:
rcu_scheduler_active = 2, debug_locks = 1
3 locks held by kworker/u8:14/3029:
#0: ffff88807741a948 ((wq_completion)bond0#3){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3206 [inline]
#0: ffff88807741a948 ((wq_completion)bond0#3){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3312
#1: ffffc9000a2c7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3207 [inline]
#1: ffffc9000a2c7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3312
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:326 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:838 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: bond_mii_monitor+0x174/0x3170 drivers/net/bonding/bond_main.c:2863
stack backtrace:
CPU: 1 UID: 0 PID: 3029 Comm: kworker/u8:14 Not tainted 6.11.0-rc7-syzkaller-00151-gd42f7708e27c #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Workqueue: bond0 bond_mii_monitor
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:93 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:119
lockdep_rcu_suspicious+0x221/0x340 kernel/locking/lockdep.c:6724
dev_activate+0xf8/0x1240 net/sched/sch_generic.c:1250
linkwatch_do_dev+0xfb/0x170 net/core/link_watch.c:173
ethtool_op_get_link+0x15/0x60 net/ethtool/ioctl.c:62
bond_check_dev_link+0x1f1/0x3f0 drivers/net/bonding/bond_main.c:799
bond_miimon_inspect drivers/net/bonding/bond_main.c:2643 [inline]
bond_mii_monitor+0x49a/0x3170 drivers/net/bonding/bond_main.c:2865
process_one_work kernel/workqueue.c:3231 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
worker_thread+0x870/0xd30 kernel/workqueue.c:3393
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
=============================
WARNING: suspicious RCU usage
6.11.0-rc7-syzkaller-00151-gd42f7708e27c #0 Not tainted
-----------------------------
net/sched/sch_generic.c:1228 suspicious rcu_dereference_protected() usage!
other info that might help us debug this:
rcu_scheduler_active = 2, debug_locks = 1
3 locks held by kworker/u8:14/3029:
#0: ffff88807741a948 ((wq_completion)bond0#3){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3206 [inline]
#0: ffff88807741a948 ((wq_completion)bond0#3){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3312
#1: ffffc9000a2c7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3207 [inline]
#1: ffffc9000a2c7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3312
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:326 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:838 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: bond_mii_monitor+0x174/0x3170 drivers/net/bonding/bond_main.c:2863
stack backtrace:
CPU: 0 UID: 0 PID: 3029 Comm: kworker/u8:14 Not tainted 6.11.0-rc7-syzkaller-00151-gd42f7708e27c #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Workqueue: bond0 bond_mii_monitor
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:93 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:119
lockdep_rcu_suspicious+0x221/0x340 kernel/locking/lockdep.c:6724
transition_one_qdisc+0x8e/0x1c0 net/sched/sch_generic.c:1228
netdev_for_each_tx_queue include/linux/netdevice.h:2513 [inline]
dev_activate+0x838/0x1240 net/sched/sch_generic.c:1258
linkwatch_do_dev+0xfb/0x170 net/core/link_watch.c:173
ethtool_op_get_link+0x15/0x60 net/ethtool/ioctl.c:62
bond_check_dev_link+0x1f1/0x3f0 drivers/net/bonding/bond_main.c:799
bond_miimon_inspect drivers/net/bonding/bond_main.c:2643 [inline]
bond_mii_monitor+0x49a/0x3170 drivers/net/bonding/bond_main.c:2865
process_one_work kernel/workqueue.c:3231 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
worker_thread+0x870/0xd30 kernel/workqueue.c:3393
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
=============================
WARNING: suspicious RCU usage
6.11.0-rc7-syzkaller-00151-gd42f7708e27c #0 Not tainted
-----------------------------
include/linux/rtnetlink.h:100 suspicious rcu_dereference_protected() usage!
other info that might help us debug this:
rcu_scheduler_active = 2, debug_locks = 1
3 locks held by kworker/u8:14/3029:
#0: ffff88807741a948 ((wq_completion)bond0#3){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3206 [inline]
#0: ffff88807741a948 ((wq_completion)bond0#3){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3312
#1: ffffc9000a2c7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3207 [inline]
#1: ffffc9000a2c7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3312
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:326 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:838 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: bond_mii_monitor+0x174/0x3170 drivers/net/bonding/bond_main.c:2863
stack backtrace:
CPU: 1 UID: 0 PID: 3029 Comm: kworker/u8:14 Not tainted 6.11.0-rc7-syzkaller-00151-gd42f7708e27c #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Workqueue: bond0 bond_mii_monitor
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:93 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:119
lockdep_rcu_suspicious+0x221/0x340 kernel/locking/lockdep.c:6724
dev_ingress_queue include/linux/rtnetlink.h:100 [inline]
dev_activate+0x925/0x1240 net/sched/sch_generic.c:1259
linkwatch_do_dev+0xfb/0x170 net/core/link_watch.c:173
ethtool_op_get_link+0x15/0x60 net/ethtool/ioctl.c:62
bond_check_dev_link+0x1f1/0x3f0 drivers/net/bonding/bond_main.c:799
bond_miimon_inspect drivers/net/bonding/bond_main.c:2643 [inline]
bond_mii_monitor+0x49a/0x3170 drivers/net/bonding/bond_main.c:2865
process_one_work kernel/workqueue.c:3231 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
worker_thread+0x870/0xd30 kernel/workqueue.c:3393
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
BUG: sleeping function called from invalid context at kernel/locking/rwsem.c:1525
in_atomic(): 0, irqs_disabled(): 0, non_block: 0, pid: 3029, name: kworker/u8:14
preempt_count: 0, expected: 0
RCU nest depth: 1, expected: 0
3 locks held by kworker/u8:14/3029:
#0: ffff88807741a948 ((wq_completion)bond0#3){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3206 [inline]
#0: ffff88807741a948 ((wq_completion)bond0#3){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3312
#1: ffffc9000a2c7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3207 [inline]
#1: ffffc9000a2c7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3312
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:326 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:838 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: bond_mii_monitor+0x174/0x3170 drivers/net/bonding/bond_main.c:2863
CPU: 0 UID: 0 PID: 3029 Comm: kworker/u8:14 Not tainted 6.11.0-rc7-syzkaller-00151-gd42f7708e27c #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Workqueue: bond0 bond_mii_monitor
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:93 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:119
__might_resched+0x5d4/0x780 kernel/sched/core.c:8463
down_read+0x8e/0xa40 kernel/locking/rwsem.c:1525
wireless_nlevent_flush net/wireless/wext-core.c:351 [inline]
wext_netdev_notifier_call+0x1f/0x120 net/wireless/wext-core.c:371
notifier_call_chain+0x19f/0x3e0 kernel/notifier.c:93
netdev_state_change+0x11f/0x1a0 net/core/dev.c:1376
linkwatch_do_dev+0x112/0x170 net/core/link_watch.c:177
ethtool_op_get_link+0x15/0x60 net/ethtool/ioctl.c:62
bond_check_dev_link+0x1f1/0x3f0 drivers/net/bonding/bond_main.c:799
bond_miimon_inspect drivers/net/bonding/bond_main.c:2643 [inline]
bond_mii_monitor+0x49a/0x3170 drivers/net/bonding/bond_main.c:2865
process_one_work kernel/workqueue.c:3231 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
worker_thread+0x870/0xd30 kernel/workqueue.c:3393
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
=============================
[ BUG: Invalid wait context ]
6.11.0-rc7-syzkaller-00151-gd42f7708e27c #0 Tainted: G W
-----------------------------
kworker/u8:14/3029 is trying to lock:
ffffffff8fc7f690 (net_rwsem){++++}-{3:3}, at: wireless_nlevent_flush net/wireless/wext-core.c:351 [inline]
ffffffff8fc7f690 (net_rwsem){++++}-{3:3}, at: wext_netdev_notifier_call+0x1f/0x120 net/wireless/wext-core.c:371
other info that might help us debug this:
context-{4:4}
3 locks held by kworker/u8:14/3029:
#0: ffff88807741a948 ((wq_completion)bond0#3){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3206 [inline]
#0: ffff88807741a948 ((wq_completion)bond0#3){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3312
#1: ffffc9000a2c7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3207 [inline]
#1: ffffc9000a2c7d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3312
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:326 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:838 [inline]
#2: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: bond_mii_monitor+0x174/0x3170 drivers/net/bonding/bond_main.c:2863
stack backtrace:
CPU: 0 UID: 0 PID: 3029 Comm: kworker/u8:14 Tainted: G W 6.11.0-rc7-syzkaller-00151-gd42f7708e27c #0
Tainted: [W]=WARN
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Workqueue: bond0 bond_mii_monitor
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:93 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:119
print_lock_invalid_wait_context kernel/locking/lockdep.c:4750 [inline]
check_wait_context kernel/locking/lockdep.c:4820 [inline]
__lock_acquire+0x153b/0x2040 kernel/locking/lockdep.c:5092
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
down_read+0xb1/0xa40 kernel/locking/rwsem.c:1526
wireless_nlevent_flush net/wireless/wext-core.c:351 [inline]
wext_netdev_notifier_call+0x1f/0x120 net/wireless/wext-core.c:371
notifier_call_chain+0x19f/0x3e0 kernel/notifier.c:93
netdev_state_change+0x11f/0x1a0 net/core/dev.c:1376
linkwatch_do_dev+0x112/0x170 net/core/link_watch.c:177
ethtool_op_get_link+0x15/0x60 net/ethtool/ioctl.c:62
bond_check_dev_link+0x1f1/0x3f0 drivers/net/bonding/bond_main.c:799
bond_miimon_inspect drivers/net/bonding/bond_main.c:2643 [inline]
bond_mii_monitor+0x49a/0x3170 drivers/net/bonding/bond_main.c:2865
process_one_work kernel/workqueue.c:3231 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
worker_thread+0x870/0xd30 kernel/workqueue.c:3393
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
bond0: (slave bridge0): link status definitely down, disabling slave
wlan1: Created IBSS using preconfigured BSSID 50:50:50:50:50:50
wlan1: Creating new IBSS network, BSSID 50:50:50:50:50:50
---
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.
^ permalink raw reply [flat|nested] 5+ messages in thread