* [syzbot] [net?] INFO: rcu detected stall in inet_rtm_newaddr (2) @ 2025-05-06 14:06 syzbot 2025-10-11 7:41 ` [syzbot] [net?] [mm?] " syzbot 0 siblings, 1 reply; 11+ messages in thread From: syzbot @ 2025-05-06 14:06 UTC (permalink / raw) To: davem, dsahern, edumazet, horms, kuba, linux-kernel, netdev, pabeni, syzkaller-bugs Hello, syzbot found the following issue on: HEAD commit: 01f95500a162 Merge tag 'uml-for-linux-6.15-rc6' of git://g.. git tree: upstream console output: https://syzkaller.appspot.com/x/log.txt?x=1347702f980000 kernel config: https://syzkaller.appspot.com/x/.config?x=b39cb28b0a399ed3 dashboard link: https://syzkaller.appspot.com/bug?extid=51cd74c5dfeafd65e488 compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40 syz repro: https://syzkaller.appspot.com/x/repro.syz?x=16cd49b3980000 Downloadable assets: disk image: https://storage.googleapis.com/syzbot-assets/18300d795306/disk-01f95500.raw.xz vmlinux: https://storage.googleapis.com/syzbot-assets/6d358d59c6dc/vmlinux-01f95500.xz kernel image: https://storage.googleapis.com/syzbot-assets/bcdf944974fd/bzImage-01f95500.xz IMPORTANT: if you fix the issue, please add the following tag to the commit: Reported-by: syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: rcu: 0-...!: (1 GPs behind) idle=530c/1/0x4000000000000000 softirq=17280/17282 fqs=3 rcu: (detected by 1, t=10506 jiffies, g=8293, q=1787 ncpus=2) Sending NMI from CPU 1 to CPUs 0: NMI backtrace for cpu 0 CPU: 0 UID: 0 PID: 5923 Comm: syz-executor Not tainted 6.15.0-rc5-syzkaller-00022-g01f95500a162 #0 PREEMPT(full) Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/29/2025 RIP: 0010:pv_queued_spin_unlock arch/x86/include/asm/paravirt.h:577 [inline] RIP: 0010:queued_spin_unlock arch/x86/include/asm/qspinlock.h:57 [inline] RIP: 0010:do_raw_spin_unlock+0x172/0x230 kernel/locking/spinlock_debug.c:142 Code: 48 ba 00 00 00 00 00 fc ff df 48 c1 e8 03 80 3c 10 00 0f 85 ba 00 00 00 48 83 3d 28 ba 89 0c 00 74 4e 48 89 df e8 0e 15 d7 09 <90> 5b 5d 41 5c c3 cc cc cc cc 48 c7 c6 e0 6c 8d 8b 48 89 df e8 45 RSP: 0018:ffffc90000007ce8 EFLAGS: 00000046 RAX: 0000000000000001 RBX: ffffffff9ad4e378 RCX: ffffffff81985ed3 RDX: dffffc0000000000 RSI: 0000000000000004 RDI: ffffffff9ad4e378 RBP: ffffffff9ad4e380 R08: 0000000000000000 R09: fffffbfff35a9c6f R10: ffffffff9ad4e37b R11: ffffffff9ad4e378 R12: ffffffff9ad4e388 R13: ffff88807bc05340 R14: dffffc0000000000 R15: 1ffff92000000fa8 FS: 0000555577c53500(0000) GS:ffff8881249df000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000555562d75808 CR3: 000000005fb8a000 CR4: 00000000003526f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: <IRQ> __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:150 [inline] _raw_spin_unlock_irqrestore+0x22/0x80 kernel/locking/spinlock.c:194 debug_object_activate+0x2ec/0x4c0 lib/debugobjects.c:836 debug_hrtimer_activate kernel/time/hrtimer.c:445 [inline] debug_activate kernel/time/hrtimer.c:484 [inline] enqueue_hrtimer+0x23/0x3b0 kernel/time/hrtimer.c:1088 __run_hrtimer kernel/time/hrtimer.c:1778 [inline] __hrtimer_run_queues+0x8ff/0xad0 kernel/time/hrtimer.c:1825 hrtimer_interrupt+0x397/0x8e0 kernel/time/hrtimer.c:1887 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1038 [inline] __sysvec_apic_timer_interrupt+0x108/0x3f0 arch/x86/kernel/apic/apic.c:1055 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1049 [inline] sysvec_apic_timer_interrupt+0x9f/0xc0 arch/x86/kernel/apic/apic.c:1049 </IRQ> <TASK> asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702 RIP: 0010:unwind_next_frame+0x671/0x20a0 arch/x86/kernel/unwind_orc.c:581 Code: 84 f7 0f 85 a4 15 00 00 83 e0 07 38 c2 40 0f 9e c6 84 d2 0f 95 c0 40 84 c6 0f 85 8d 15 00 00 4c 0f bf 31 4d 01 fe 0f b6 41 05 <83> e0 07 3c 03 0f 84 bd 08 00 00 3c 04 0f 84 f6 07 00 00 3c 02 0f RSP: 0018:ffffc9000477ec48 EFLAGS: 00000292 RAX: 0000000000000002 RBX: 0000000000000001 RCX: ffffffff91275258 RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000001 RBP: ffffc9000477ed00 R08: ffffffff9127525c R09: 0000000000000000 R10: 0000000000000002 R11: 0000000000012081 R12: ffffc9000477ed08 R13: ffffc9000477ecb8 R14: ffffc9000477f040 R15: ffffc9000477f018 arch_stack_walk+0x94/0x100 arch/x86/kernel/stacktrace.c:25 stack_trace_save+0x8e/0xc0 kernel/stacktrace.c:122 kasan_save_stack+0x33/0x60 mm/kasan/common.c:47 kasan_save_track+0x14/0x30 mm/kasan/common.c:68 unpoison_slab_object mm/kasan/common.c:319 [inline] __kasan_slab_alloc+0x89/0x90 mm/kasan/common.c:345 kasan_slab_alloc include/linux/kasan.h:250 [inline] slab_post_alloc_hook mm/slub.c:4147 [inline] slab_alloc_node mm/slub.c:4196 [inline] kmem_cache_alloc_node_noprof+0x1d5/0x3b0 mm/slub.c:4248 __alloc_skb+0x2b2/0x380 net/core/skbuff.c:658 alloc_skb include/linux/skbuff.h:1340 [inline] nlmsg_new include/net/netlink.h:1019 [inline] rtmsg_fib+0x13e/0x520 net/ipv4/fib_semantics.c:552 fib_table_insert+0xbaf/0x1c40 net/ipv4/fib_trie.c:1380 fib_magic+0x4d4/0x5c0 net/ipv4/fib_frontend.c:1133 fib_add_ifaddr+0x3a1/0x580 net/ipv4/fib_frontend.c:1170 fib_inetaddr_event+0x147/0x270 net/ipv4/fib_frontend.c:1469 notifier_call_chain+0xb9/0x410 kernel/notifier.c:85 blocking_notifier_call_chain kernel/notifier.c:380 [inline] blocking_notifier_call_chain+0x69/0xa0 kernel/notifier.c:368 __inet_insert_ifa+0x925/0xcd0 net/ipv4/devinet.c:567 inet_rtm_newaddr+0xd87/0x1540 net/ipv4/devinet.c:1002 rtnetlink_rcv_msg+0x95b/0xe90 net/core/rtnetlink.c:6955 netlink_rcv_skb+0x16a/0x440 net/netlink/af_netlink.c:2534 netlink_unicast_kernel net/netlink/af_netlink.c:1313 [inline] netlink_unicast+0x53a/0x7f0 net/netlink/af_netlink.c:1339 netlink_sendmsg+0x8d1/0xdd0 net/netlink/af_netlink.c:1883 sock_sendmsg_nosec net/socket.c:712 [inline] __sock_sendmsg net/socket.c:727 [inline] __sys_sendto+0x495/0x510 net/socket.c:2180 __do_sys_sendto net/socket.c:2187 [inline] __se_sys_sendto net/socket.c:2183 [inline] __x64_sys_sendto+0xe0/0x1c0 net/socket.c:2183 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0xcd/0x260 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f RIP: 0033:0x7f87d9f907fc Code: 2a 5f 02 00 44 8b 4c 24 2c 4c 8b 44 24 20 89 c5 44 8b 54 24 28 48 8b 54 24 18 b8 2c 00 00 00 48 8b 74 24 10 8b 7c 24 08 0f 05 <48> 3d 00 f0 ff ff 77 34 89 ef 48 89 44 24 08 e8 70 5f 02 00 48 8b RSP: 002b:00007fff248eaf20 EFLAGS: 00000293 ORIG_RAX: 000000000000002c RAX: ffffffffffffffda RBX: 00007f87dace4620 RCX: 00007f87d9f907fc RDX: 0000000000000028 RSI: 00007f87dace4670 RDI: 0000000000000003 RBP: 0000000000000000 R08: 00007fff248eaf74 R09: 000000000000000c R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000003 R13: 0000000000000000 R14: 00007f87dace4670 R15: 0000000000000000 </TASK> rcu: rcu_preempt kthread starved for 10490 jiffies! g8293 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1 rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. rcu: RCU grace-period kthread stack dump: task:rcu_preempt state:R running task stack:27704 pid:16 tgid:16 ppid:2 task_flags:0x208040 flags:0x00004000 Call Trace: <TASK> context_switch kernel/sched/core.c:5382 [inline] __schedule+0x116f/0x5de0 kernel/sched/core.c:6767 __schedule_loop kernel/sched/core.c:6845 [inline] schedule+0xe7/0x3a0 kernel/sched/core.c:6860 schedule_timeout+0x123/0x290 kernel/time/sleep_timeout.c:99 rcu_gp_fqs_loop+0x1ea/0xb00 kernel/rcu/tree.c:2046 rcu_gp_kthread+0x270/0x380 kernel/rcu/tree.c:2248 kthread+0x3c2/0x780 kernel/kthread.c:464 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:153 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 </TASK> rcu: Stack dump where RCU GP kthread last ran: CPU: 1 UID: 101 PID: 5480 Comm: dhcpcd Not tainted 6.15.0-rc5-syzkaller-00022-g01f95500a162 #0 PREEMPT(full) Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/29/2025 RIP: 0010:write_comp_data+0x8b/0x90 kernel/kcov.c:272 Code: 00 00 4a 8d 34 dd 28 00 00 00 48 39 f2 72 1b 48 83 c7 01 48 89 38 4c 89 44 30 e0 4c 89 4c 30 e8 4c 89 54 30 f0 4a 89 4c d8 20 <c3> cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 RSP: 0018:ffffc90002ea77b0 EFLAGS: 00000293 RAX: 0000000000000000 RBX: ffff8880b8441720 RCX: ffffffff81af1bb9 RDX: ffff88802577c880 RSI: 0000000000000000 RDI: 0000000000000005 RBP: 0000000000000003 R08: 0000000000000005 R09: 0000000000000000 R10: 0000000000000001 R11: 0000000000000000 R12: ffffed10170882e5 R13: 0000000000000001 R14: dffffc0000000000 R15: ffff8880b853b040 FS: 00007fe629d36740(0000) GS:ffff888124adf000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000055b10a151068 CR3: 0000000025463000 CR4: 00000000003526f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: <TASK> csd_lock_wait kernel/smp.c:340 [inline] smp_call_function_many_cond+0x4c9/0x1290 kernel/smp.c:885 on_each_cpu_cond_mask+0x40/0x90 kernel/smp.c:1052 __flush_tlb_multi arch/x86/include/asm/paravirt.h:91 [inline] flush_tlb_multi arch/x86/mm/tlb.c:1275 [inline] flush_tlb_mm_range+0x322/0x1780 arch/x86/mm/tlb.c:1365 tlb_flush arch/x86/include/asm/tlb.h:23 [inline] tlb_flush_mmu_tlbonly include/asm-generic/tlb.h:480 [inline] tlb_flush_mmu_tlbonly include/asm-generic/tlb.h:470 [inline] tlb_flush_mmu mm/mmu_gather.c:403 [inline] tlb_finish_mmu+0x3c9/0x7b0 mm/mmu_gather.c:496 vms_clear_ptes+0x55e/0x770 mm/vma.c:1191 vms_complete_munmap_vmas+0x1ca/0x970 mm/vma.c:1233 do_vmi_align_munmap+0x43b/0x7d0 mm/vma.c:1492 __do_sys_brk+0x8d3/0xaa0 mm/mmap.c:176 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0xcd/0x260 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f RIP: 0033:0x7fe629e04c7c Code: 1a 64 c7 03 01 00 00 00 eb 11 64 44 89 23 31 f6 5b 31 ff 5d 41 5c e9 41 ff ff ff 5b 83 c8 ff 5d 41 5c c3 b8 0c 00 00 00 0f 05 <48> 8b 15 d5 61 0d 00 45 31 c0 48 89 02 48 39 c7 76 12 48 8b 05 73 RSP: 002b:00007fffc7ef0748 EFLAGS: 00000206 ORIG_RAX: 000000000000000c RAX: ffffffffffffffda RBX: fffffffffffdf000 RCX: 00007fe629e04c7c RDX: 0000558eaa832990 RSI: 0000558eaa6f2010 RDI: 0000558eaa853000 RBP: 0000000000041670 R08: 0000000000000000 R09: ba568e78e099c46f R10: 00007fffc7ef06a8 R11: 0000000000000206 R12: 0000558eaa874000 R13: 0000558eaa863f20 R14: 00007fe629edbaa0 R15: 00007fffc7ef0978 </TASK> --- This report is generated by a bot. It may contain errors. See https://goo.gl/tpsmEJ for more information about syzbot. syzbot engineers can be reached at syzkaller@googlegroups.com. syzbot will keep track of this issue. See: https://goo.gl/tpsmEJ#status for how to communicate with syzbot. If the report is already addressed, let syzbot know by replying with: #syz fix: exact-commit-title If you want syzbot to run the reproducer, reply with: #syz test: git://repo/address.git branch-or-commit-hash If you attach or paste a git patch, syzbot will apply it before testing. If you want to overwrite report's subsystems, reply with: #syz set subsystems: new-subsystem (See the list of subsystem names on the web dashboard) If the report is a duplicate of another one, reply with: #syz dup: exact-subject-of-another-report If you want to undo deduplication, reply with: #syz undup ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [syzbot] [net?] [mm?] INFO: rcu detected stall in inet_rtm_newaddr (2) 2025-05-06 14:06 [syzbot] [net?] INFO: rcu detected stall in inet_rtm_newaddr (2) syzbot @ 2025-10-11 7:41 ` syzbot 2025-10-11 9:42 ` Eric Dumazet 0 siblings, 1 reply; 11+ messages in thread From: syzbot @ 2025-10-11 7:41 UTC (permalink / raw) To: davem, dsahern, edumazet, hdanton, horms, kuba, linux-kernel, linux-mm, netdev, pabeni, syzkaller-bugs, tglx syzbot has found a reproducer for the following issue on: HEAD commit: 18a7e218cfcd Merge tag 'net-6.18-rc1' of git://git.kernel... git tree: net-next console output: https://syzkaller.appspot.com/x/log.txt?x=12504dcd980000 kernel config: https://syzkaller.appspot.com/x/.config?x=61ab7fa743df0ec1 dashboard link: https://syzkaller.appspot.com/bug?extid=51cd74c5dfeafd65e488 compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 syz repro: https://syzkaller.appspot.com/x/repro.syz?x=14d2a542580000 C reproducer: https://syzkaller.appspot.com/x/repro.c?x=142149e2580000 Downloadable assets: disk image: https://storage.googleapis.com/syzbot-assets/7a01e6dce97e/disk-18a7e218.raw.xz vmlinux: https://storage.googleapis.com/syzbot-assets/5e1b7e41427f/vmlinux-18a7e218.xz kernel image: https://storage.googleapis.com/syzbot-assets/69b558601209/bzImage-18a7e218.xz IMPORTANT: if you fix the issue, please add the following tag to the commit: Reported-by: syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com sched: DL replenish lagged too much rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: rcu: 0-...!: (2 GPs behind) idle=7754/1/0x4000000000000000 softirq=15464/15465 fqs=1 rcu: (detected by 1, t=10502 jiffies, g=11321, q=371 ncpus=2) Sending NMI from CPU 1 to CPUs 0: NMI backtrace for cpu 0 CPU: 0 UID: 0 PID: 5948 Comm: syz-executor Not tainted syzkaller #0 PREEMPT(full) Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/02/2025 RIP: 0010:rb_insert_color_cached include/linux/rbtree.h:113 [inline] RIP: 0010:rb_add_cached include/linux/rbtree.h:183 [inline] RIP: 0010:timerqueue_add+0x1a8/0x200 lib/timerqueue.c:40 Code: e7 31 f6 e8 6a 0c de f6 42 80 3c 2b 00 74 08 4c 89 f7 e8 7b 0a de f6 4d 89 26 4d 8d 7e 08 4c 89 f8 48 c1 e8 03 42 80 3c 28 00 <74> 08 4c 89 ff e8 5e 0a de f6 4d 89 27 4d 85 e4 40 0f 95 c5 eb 07 RSP: 0018:ffffc90000007cf0 EFLAGS: 00000046 RAX: 1ffff110170c4f83 RBX: 1ffff110170c4f82 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88805de72358 RBP: 0000000000000000 R08: ffff88805de72357 R09: 0000000000000000 R10: ffff88805de72340 R11: ffffed100bbce46b R12: ffff88805de72340 R13: dffffc0000000000 R14: ffff8880b8627c10 R15: ffff8880b8627c18 FS: 000055557c657500(0000) GS:ffff888125d0f000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000200000000600 CR3: 000000002ee76000 CR4: 00000000003526f0 Call Trace: <IRQ> __run_hrtimer kernel/time/hrtimer.c:1794 [inline] __hrtimer_run_queues+0x656/0xc60 kernel/time/hrtimer.c:1841 hrtimer_interrupt+0x45b/0xaa0 kernel/time/hrtimer.c:1903 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1041 [inline] __sysvec_apic_timer_interrupt+0x108/0x410 arch/x86/kernel/apic/apic.c:1058 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1052 [inline] sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1052 </IRQ> <TASK> asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702 RIP: 0010:pv_vcpu_is_preempted arch/x86/include/asm/paravirt.h:579 [inline] RIP: 0010:vcpu_is_preempted arch/x86/include/asm/qspinlock.h:63 [inline] RIP: 0010:owner_on_cpu include/linux/sched.h:2282 [inline] RIP: 0010:mutex_spin_on_owner+0x189/0x360 kernel/locking/mutex.c:361 Code: b6 04 30 84 c0 0f 85 59 01 00 00 48 8b 44 24 08 8b 18 48 8b 44 24 48 42 80 3c 30 00 74 0c 48 c7 c7 90 8c fa 8d e8 a7 cd 88 00 <48> 83 3d ff 27 5e 0c 00 0f 84 b9 01 00 00 48 89 df e8 41 e0 d5 ff RSP: 0018:ffffc900034c7428 EFLAGS: 00000246 RAX: 1ffffffff1bf5192 RBX: 0000000000000001 RCX: ffffffff819c6588 RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff8f4df8a0 RBP: 1ffffffff1e9bf14 R08: ffffffff8f4df8a7 R09: 1ffffffff1e9bf14 R10: dffffc0000000000 R11: fffffbfff1e9bf15 R12: ffffffff8f4df8a0 R13: ffffffff8f4df8f0 R14: dffffc0000000000 R15: ffff8880267a9e40 mutex_optimistic_spin kernel/locking/mutex.c:464 [inline] __mutex_lock_common kernel/locking/mutex.c:602 [inline] __mutex_lock+0x311/0x1350 kernel/locking/mutex.c:760 rtnl_net_lock include/linux/rtnetlink.h:130 [inline] inet_rtm_newaddr+0x3b0/0x18b0 net/ipv4/devinet.c:978 rtnetlink_rcv_msg+0x7cf/0xb70 net/core/rtnetlink.c:6954 netlink_rcv_skb+0x205/0x470 net/netlink/af_netlink.c:2552 netlink_unicast_kernel net/netlink/af_netlink.c:1320 [inline] netlink_unicast+0x82f/0x9e0 net/netlink/af_netlink.c:1346 netlink_sendmsg+0x805/0xb30 net/netlink/af_netlink.c:1896 sock_sendmsg_nosec net/socket.c:727 [inline] __sock_sendmsg+0x21c/0x270 net/socket.c:742 __sys_sendto+0x3bd/0x520 net/socket.c:2244 __do_sys_sendto net/socket.c:2251 [inline] __se_sys_sendto net/socket.c:2247 [inline] __x64_sys_sendto+0xde/0x100 net/socket.c:2247 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0xfa/0xfa0 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f RIP: 0033:0x7faade790d5c Code: 2a 5f 02 00 44 8b 4c 24 2c 4c 8b 44 24 20 89 c5 44 8b 54 24 28 48 8b 54 24 18 b8 2c 00 00 00 48 8b 74 24 10 8b 7c 24 08 0f 05 <48> 3d 00 f0 ff ff 77 34 89 ef 48 89 44 24 08 e8 70 5f 02 00 48 8b RSP: 002b:00007ffdd2e3b670 EFLAGS: 00000293 ORIG_RAX: 000000000000002c RAX: ffffffffffffffda RBX: 00007faadf514620 RCX: 00007faade790d5c RDX: 0000000000000028 RSI: 00007faadf514670 RDI: 0000000000000003 RBP: 0000000000000000 R08: 00007ffdd2e3b6c4 R09: 000000000000000c R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000003 R13: 0000000000000000 R14: 00007faadf514670 R15: 0000000000000000 </TASK> rcu: rcu_preempt kthread timer wakeup didn't happen for 10499 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 rcu: Possible timer handling issue on cpu=0 timer-softirq=4286 rcu: rcu_preempt kthread starved for 10500 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. rcu: RCU grace-period kthread stack dump: task:rcu_preempt state:I stack:27224 pid:16 tgid:16 ppid:2 task_flags:0x208040 flags:0x00080000 Call Trace: <TASK> context_switch kernel/sched/core.c:5325 [inline] __schedule+0x1798/0x4cc0 kernel/sched/core.c:6929 __schedule_loop kernel/sched/core.c:7011 [inline] schedule+0x165/0x360 kernel/sched/core.c:7026 schedule_timeout+0x12b/0x270 kernel/time/sleep_timeout.c:99 rcu_gp_fqs_loop+0x301/0x1540 kernel/rcu/tree.c:2083 rcu_gp_kthread+0x99/0x390 kernel/rcu/tree.c:2285 kthread+0x711/0x8a0 kernel/kthread.c:463 ret_from_fork+0x4bc/0x870 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 </TASK> --- If you want syzbot to run the reproducer, reply with: #syz test: git://repo/address.git branch-or-commit-hash If you attach or paste a git patch, syzbot will apply it before testing. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [syzbot] [net?] [mm?] INFO: rcu detected stall in inet_rtm_newaddr (2) 2025-10-11 7:41 ` [syzbot] [net?] [mm?] " syzbot @ 2025-10-11 9:42 ` Eric Dumazet 2025-10-12 15:46 ` Jamal Hadi Salim 0 siblings, 1 reply; 11+ messages in thread From: Eric Dumazet @ 2025-10-11 9:42 UTC (permalink / raw) To: syzbot, Vinicius Costa Gomes, Jamal Hadi Salim Cc: davem, dsahern, hdanton, horms, kuba, linux-kernel, linux-mm, netdev, pabeni, syzkaller-bugs, tglx On Sat, Oct 11, 2025 at 12:41 AM syzbot <syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com> wrote: > > syzbot has found a reproducer for the following issue on: > > HEAD commit: 18a7e218cfcd Merge tag 'net-6.18-rc1' of git://git.kernel... > git tree: net-next > console output: https://syzkaller.appspot.com/x/log.txt?x=12504dcd980000 > kernel config: https://syzkaller.appspot.com/x/.config?x=61ab7fa743df0ec1 > dashboard link: https://syzkaller.appspot.com/bug?extid=51cd74c5dfeafd65e488 > compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=14d2a542580000 > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=142149e2580000 > > Downloadable assets: > disk image: https://storage.googleapis.com/syzbot-assets/7a01e6dce97e/disk-18a7e218.raw.xz > vmlinux: https://storage.googleapis.com/syzbot-assets/5e1b7e41427f/vmlinux-18a7e218.xz > kernel image: https://storage.googleapis.com/syzbot-assets/69b558601209/bzImage-18a7e218.xz > > IMPORTANT: if you fix the issue, please add the following tag to the commit: > Reported-by: syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com > > sched: DL replenish lagged too much > rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: > rcu: 0-...!: (2 GPs behind) idle=7754/1/0x4000000000000000 softirq=15464/15465 fqs=1 > rcu: (detected by 1, t=10502 jiffies, g=11321, q=371 ncpus=2) > Sending NMI from CPU 1 to CPUs 0: > NMI backtrace for cpu 0 > CPU: 0 UID: 0 PID: 5948 Comm: syz-executor Not tainted syzkaller #0 PREEMPT(full) > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/02/2025 > RIP: 0010:rb_insert_color_cached include/linux/rbtree.h:113 [inline] > RIP: 0010:rb_add_cached include/linux/rbtree.h:183 [inline] > RIP: 0010:timerqueue_add+0x1a8/0x200 lib/timerqueue.c:40 > Code: e7 31 f6 e8 6a 0c de f6 42 80 3c 2b 00 74 08 4c 89 f7 e8 7b 0a de f6 4d 89 26 4d 8d 7e 08 4c 89 f8 48 c1 e8 03 42 80 3c 28 00 <74> 08 4c 89 ff e8 5e 0a de f6 4d 89 27 4d 85 e4 40 0f 95 c5 eb 07 > RSP: 0018:ffffc90000007cf0 EFLAGS: 00000046 > RAX: 1ffff110170c4f83 RBX: 1ffff110170c4f82 RCX: 0000000000000000 > RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88805de72358 > RBP: 0000000000000000 R08: ffff88805de72357 R09: 0000000000000000 > R10: ffff88805de72340 R11: ffffed100bbce46b R12: ffff88805de72340 > R13: dffffc0000000000 R14: ffff8880b8627c10 R15: ffff8880b8627c18 > FS: 000055557c657500(0000) GS:ffff888125d0f000(0000) knlGS:0000000000000000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 0000200000000600 CR3: 000000002ee76000 CR4: 00000000003526f0 > Call Trace: > <IRQ> > __run_hrtimer kernel/time/hrtimer.c:1794 [inline] > __hrtimer_run_queues+0x656/0xc60 kernel/time/hrtimer.c:1841 > hrtimer_interrupt+0x45b/0xaa0 kernel/time/hrtimer.c:1903 > local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1041 [inline] > __sysvec_apic_timer_interrupt+0x108/0x410 arch/x86/kernel/apic/apic.c:1058 > instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1052 [inline] > sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1052 > </IRQ> > <TASK> > asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702 > RIP: 0010:pv_vcpu_is_preempted arch/x86/include/asm/paravirt.h:579 [inline] > RIP: 0010:vcpu_is_preempted arch/x86/include/asm/qspinlock.h:63 [inline] > RIP: 0010:owner_on_cpu include/linux/sched.h:2282 [inline] > RIP: 0010:mutex_spin_on_owner+0x189/0x360 kernel/locking/mutex.c:361 > Code: b6 04 30 84 c0 0f 85 59 01 00 00 48 8b 44 24 08 8b 18 48 8b 44 24 48 42 80 3c 30 00 74 0c 48 c7 c7 90 8c fa 8d e8 a7 cd 88 00 <48> 83 3d ff 27 5e 0c 00 0f 84 b9 01 00 00 48 89 df e8 41 e0 d5 ff > RSP: 0018:ffffc900034c7428 EFLAGS: 00000246 > RAX: 1ffffffff1bf5192 RBX: 0000000000000001 RCX: ffffffff819c6588 > RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff8f4df8a0 > RBP: 1ffffffff1e9bf14 R08: ffffffff8f4df8a7 R09: 1ffffffff1e9bf14 > R10: dffffc0000000000 R11: fffffbfff1e9bf15 R12: ffffffff8f4df8a0 > R13: ffffffff8f4df8f0 R14: dffffc0000000000 R15: ffff8880267a9e40 > mutex_optimistic_spin kernel/locking/mutex.c:464 [inline] > __mutex_lock_common kernel/locking/mutex.c:602 [inline] > __mutex_lock+0x311/0x1350 kernel/locking/mutex.c:760 > rtnl_net_lock include/linux/rtnetlink.h:130 [inline] > inet_rtm_newaddr+0x3b0/0x18b0 net/ipv4/devinet.c:978 > rtnetlink_rcv_msg+0x7cf/0xb70 net/core/rtnetlink.c:6954 > netlink_rcv_skb+0x205/0x470 net/netlink/af_netlink.c:2552 > netlink_unicast_kernel net/netlink/af_netlink.c:1320 [inline] > netlink_unicast+0x82f/0x9e0 net/netlink/af_netlink.c:1346 > netlink_sendmsg+0x805/0xb30 net/netlink/af_netlink.c:1896 > sock_sendmsg_nosec net/socket.c:727 [inline] > __sock_sendmsg+0x21c/0x270 net/socket.c:742 > __sys_sendto+0x3bd/0x520 net/socket.c:2244 > __do_sys_sendto net/socket.c:2251 [inline] > __se_sys_sendto net/socket.c:2247 [inline] > __x64_sys_sendto+0xde/0x100 net/socket.c:2247 > do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] > do_syscall_64+0xfa/0xfa0 arch/x86/entry/syscall_64.c:94 > entry_SYSCALL_64_after_hwframe+0x77/0x7f > RIP: 0033:0x7faade790d5c > Code: 2a 5f 02 00 44 8b 4c 24 2c 4c 8b 44 24 20 89 c5 44 8b 54 24 28 48 8b 54 24 18 b8 2c 00 00 00 48 8b 74 24 10 8b 7c 24 08 0f 05 <48> 3d 00 f0 ff ff 77 34 89 ef 48 89 44 24 08 e8 70 5f 02 00 48 8b > RSP: 002b:00007ffdd2e3b670 EFLAGS: 00000293 ORIG_RAX: 000000000000002c > RAX: ffffffffffffffda RBX: 00007faadf514620 RCX: 00007faade790d5c > RDX: 0000000000000028 RSI: 00007faadf514670 RDI: 0000000000000003 > RBP: 0000000000000000 R08: 00007ffdd2e3b6c4 R09: 000000000000000c > R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000003 > R13: 0000000000000000 R14: 00007faadf514670 R15: 0000000000000000 > </TASK> > rcu: rcu_preempt kthread timer wakeup didn't happen for 10499 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 > rcu: Possible timer handling issue on cpu=0 timer-softirq=4286 > rcu: rcu_preempt kthread starved for 10500 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 > rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. > rcu: RCU grace-period kthread stack dump: > task:rcu_preempt state:I stack:27224 pid:16 tgid:16 ppid:2 task_flags:0x208040 flags:0x00080000 > Call Trace: > <TASK> > context_switch kernel/sched/core.c:5325 [inline] > __schedule+0x1798/0x4cc0 kernel/sched/core.c:6929 > __schedule_loop kernel/sched/core.c:7011 [inline] > schedule+0x165/0x360 kernel/sched/core.c:7026 > schedule_timeout+0x12b/0x270 kernel/time/sleep_timeout.c:99 > rcu_gp_fqs_loop+0x301/0x1540 kernel/rcu/tree.c:2083 > rcu_gp_kthread+0x99/0x390 kernel/rcu/tree.c:2285 > kthread+0x711/0x8a0 kernel/kthread.c:463 > ret_from_fork+0x4bc/0x870 arch/x86/kernel/process.c:158 > ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 > </TASK> > > > --- > If you want syzbot to run the reproducer, reply with: > #syz test: git://repo/address.git branch-or-commit-hash > If you attach or paste a git patch, syzbot will apply it before testing. Yet another taprio report. If taprio can not be fixed, perhaps we should remove it from the kernel, or clearly marked as broken. (Then ask syzbot to no longer include it) ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [syzbot] [net?] [mm?] INFO: rcu detected stall in inet_rtm_newaddr (2) 2025-10-11 9:42 ` Eric Dumazet @ 2025-10-12 15:46 ` Jamal Hadi Salim 2025-10-12 15:52 ` Jamal Hadi Salim 2025-10-13 21:51 ` Vinicius Costa Gomes 0 siblings, 2 replies; 11+ messages in thread From: Jamal Hadi Salim @ 2025-10-12 15:46 UTC (permalink / raw) To: Eric Dumazet Cc: syzbot, Vinicius Costa Gomes, davem, dsahern, hdanton, horms, kuba, linux-kernel, linux-mm, netdev, pabeni, syzkaller-bugs, tglx On Sat, Oct 11, 2025 at 5:42 AM Eric Dumazet <edumazet@google.com> wrote: > > On Sat, Oct 11, 2025 at 12:41 AM syzbot > <syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com> wrote: > > > > syzbot has found a reproducer for the following issue on: > > > > HEAD commit: 18a7e218cfcd Merge tag 'net-6.18-rc1' of git://git.kernel... > > git tree: net-next > > console output: https://syzkaller.appspot.com/x/log.txt?x=12504dcd980000 > > kernel config: https://syzkaller.appspot.com/x/.config?x=61ab7fa743df0ec1 > > dashboard link: https://syzkaller.appspot.com/bug?extid=51cd74c5dfeafd65e488 > > compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 > > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=14d2a542580000 > > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=142149e2580000 > > > > Downloadable assets: > > disk image: https://storage.googleapis.com/syzbot-assets/7a01e6dce97e/disk-18a7e218.raw.xz > > vmlinux: https://storage.googleapis.com/syzbot-assets/5e1b7e41427f/vmlinux-18a7e218.xz > > kernel image: https://storage.googleapis.com/syzbot-assets/69b558601209/bzImage-18a7e218.xz > > > > IMPORTANT: if you fix the issue, please add the following tag to the commit: > > Reported-by: syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com > > > > sched: DL replenish lagged too much > > rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: > > rcu: 0-...!: (2 GPs behind) idle=7754/1/0x4000000000000000 softirq=15464/15465 fqs=1 > > rcu: (detected by 1, t=10502 jiffies, g=11321, q=371 ncpus=2) > > Sending NMI from CPU 1 to CPUs 0: > > NMI backtrace for cpu 0 > > CPU: 0 UID: 0 PID: 5948 Comm: syz-executor Not tainted syzkaller #0 PREEMPT(full) > > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/02/2025 > > RIP: 0010:rb_insert_color_cached include/linux/rbtree.h:113 [inline] > > RIP: 0010:rb_add_cached include/linux/rbtree.h:183 [inline] > > RIP: 0010:timerqueue_add+0x1a8/0x200 lib/timerqueue.c:40 > > Code: e7 31 f6 e8 6a 0c de f6 42 80 3c 2b 00 74 08 4c 89 f7 e8 7b 0a de f6 4d 89 26 4d 8d 7e 08 4c 89 f8 48 c1 e8 03 42 80 3c 28 00 <74> 08 4c 89 ff e8 5e 0a de f6 4d 89 27 4d 85 e4 40 0f 95 c5 eb 07 > > RSP: 0018:ffffc90000007cf0 EFLAGS: 00000046 > > RAX: 1ffff110170c4f83 RBX: 1ffff110170c4f82 RCX: 0000000000000000 > > RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88805de72358 > > RBP: 0000000000000000 R08: ffff88805de72357 R09: 0000000000000000 > > R10: ffff88805de72340 R11: ffffed100bbce46b R12: ffff88805de72340 > > R13: dffffc0000000000 R14: ffff8880b8627c10 R15: ffff8880b8627c18 > > FS: 000055557c657500(0000) GS:ffff888125d0f000(0000) knlGS:0000000000000000 > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > CR2: 0000200000000600 CR3: 000000002ee76000 CR4: 00000000003526f0 > > Call Trace: > > <IRQ> > > __run_hrtimer kernel/time/hrtimer.c:1794 [inline] > > __hrtimer_run_queues+0x656/0xc60 kernel/time/hrtimer.c:1841 > > hrtimer_interrupt+0x45b/0xaa0 kernel/time/hrtimer.c:1903 > > local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1041 [inline] > > __sysvec_apic_timer_interrupt+0x108/0x410 arch/x86/kernel/apic/apic.c:1058 > > instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1052 [inline] > > sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1052 > > </IRQ> > > <TASK> > > asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702 > > RIP: 0010:pv_vcpu_is_preempted arch/x86/include/asm/paravirt.h:579 [inline] > > RIP: 0010:vcpu_is_preempted arch/x86/include/asm/qspinlock.h:63 [inline] > > RIP: 0010:owner_on_cpu include/linux/sched.h:2282 [inline] > > RIP: 0010:mutex_spin_on_owner+0x189/0x360 kernel/locking/mutex.c:361 > > Code: b6 04 30 84 c0 0f 85 59 01 00 00 48 8b 44 24 08 8b 18 48 8b 44 24 48 42 80 3c 30 00 74 0c 48 c7 c7 90 8c fa 8d e8 a7 cd 88 00 <48> 83 3d ff 27 5e 0c 00 0f 84 b9 01 00 00 48 89 df e8 41 e0 d5 ff > > RSP: 0018:ffffc900034c7428 EFLAGS: 00000246 > > RAX: 1ffffffff1bf5192 RBX: 0000000000000001 RCX: ffffffff819c6588 > > RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff8f4df8a0 > > RBP: 1ffffffff1e9bf14 R08: ffffffff8f4df8a7 R09: 1ffffffff1e9bf14 > > R10: dffffc0000000000 R11: fffffbfff1e9bf15 R12: ffffffff8f4df8a0 > > R13: ffffffff8f4df8f0 R14: dffffc0000000000 R15: ffff8880267a9e40 > > mutex_optimistic_spin kernel/locking/mutex.c:464 [inline] > > __mutex_lock_common kernel/locking/mutex.c:602 [inline] > > __mutex_lock+0x311/0x1350 kernel/locking/mutex.c:760 > > rtnl_net_lock include/linux/rtnetlink.h:130 [inline] > > inet_rtm_newaddr+0x3b0/0x18b0 net/ipv4/devinet.c:978 > > rtnetlink_rcv_msg+0x7cf/0xb70 net/core/rtnetlink.c:6954 > > netlink_rcv_skb+0x205/0x470 net/netlink/af_netlink.c:2552 > > netlink_unicast_kernel net/netlink/af_netlink.c:1320 [inline] > > netlink_unicast+0x82f/0x9e0 net/netlink/af_netlink.c:1346 > > netlink_sendmsg+0x805/0xb30 net/netlink/af_netlink.c:1896 > > sock_sendmsg_nosec net/socket.c:727 [inline] > > __sock_sendmsg+0x21c/0x270 net/socket.c:742 > > __sys_sendto+0x3bd/0x520 net/socket.c:2244 > > __do_sys_sendto net/socket.c:2251 [inline] > > __se_sys_sendto net/socket.c:2247 [inline] > > __x64_sys_sendto+0xde/0x100 net/socket.c:2247 > > do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] > > do_syscall_64+0xfa/0xfa0 arch/x86/entry/syscall_64.c:94 > > entry_SYSCALL_64_after_hwframe+0x77/0x7f > > RIP: 0033:0x7faade790d5c > > Code: 2a 5f 02 00 44 8b 4c 24 2c 4c 8b 44 24 20 89 c5 44 8b 54 24 28 48 8b 54 24 18 b8 2c 00 00 00 48 8b 74 24 10 8b 7c 24 08 0f 05 <48> 3d 00 f0 ff ff 77 34 89 ef 48 89 44 24 08 e8 70 5f 02 00 48 8b > > RSP: 002b:00007ffdd2e3b670 EFLAGS: 00000293 ORIG_RAX: 000000000000002c > > RAX: ffffffffffffffda RBX: 00007faadf514620 RCX: 00007faade790d5c > > RDX: 0000000000000028 RSI: 00007faadf514670 RDI: 0000000000000003 > > RBP: 0000000000000000 R08: 00007ffdd2e3b6c4 R09: 000000000000000c > > R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000003 > > R13: 0000000000000000 R14: 00007faadf514670 R15: 0000000000000000 > > </TASK> > > rcu: rcu_preempt kthread timer wakeup didn't happen for 10499 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 > > rcu: Possible timer handling issue on cpu=0 timer-softirq=4286 > > rcu: rcu_preempt kthread starved for 10500 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 > > rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. > > rcu: RCU grace-period kthread stack dump: > > task:rcu_preempt state:I stack:27224 pid:16 tgid:16 ppid:2 task_flags:0x208040 flags:0x00080000 > > Call Trace: > > <TASK> > > context_switch kernel/sched/core.c:5325 [inline] > > __schedule+0x1798/0x4cc0 kernel/sched/core.c:6929 > > __schedule_loop kernel/sched/core.c:7011 [inline] > > schedule+0x165/0x360 kernel/sched/core.c:7026 > > schedule_timeout+0x12b/0x270 kernel/time/sleep_timeout.c:99 > > rcu_gp_fqs_loop+0x301/0x1540 kernel/rcu/tree.c:2083 > > rcu_gp_kthread+0x99/0x390 kernel/rcu/tree.c:2285 > > kthread+0x711/0x8a0 kernel/kthread.c:463 > > ret_from_fork+0x4bc/0x870 arch/x86/kernel/process.c:158 > > ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 > > </TASK> > > > > > > --- > > If you want syzbot to run the reproducer, reply with: > > #syz test: git://repo/address.git branch-or-commit-hash > > If you attach or paste a git patch, syzbot will apply it before testing. > > Yet another taprio report. > > If taprio can not be fixed, perhaps we should remove it from the > kernel, or clearly marked as broken. > (Then ask syzbot to no longer include it) Agreed on the challenge with taprio. We need the stakeholders input: Vinicius - are you still working in this space? Vladimir you also seem to have interest (or maybe nxp does) in this? At a minimum, we should mark it as broken unless the stakeholders want to actively fix these issues. Would syzbot still look at it if it was marked broken? cheers, jamal ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [syzbot] [net?] [mm?] INFO: rcu detected stall in inet_rtm_newaddr (2) 2025-10-12 15:46 ` Jamal Hadi Salim @ 2025-10-12 15:52 ` Jamal Hadi Salim 2025-10-14 20:11 ` Vladimir Oltean 2025-10-13 21:51 ` Vinicius Costa Gomes 1 sibling, 1 reply; 11+ messages in thread From: Jamal Hadi Salim @ 2025-10-12 15:52 UTC (permalink / raw) To: Eric Dumazet Cc: syzbot, Vinicius Costa Gomes, davem, dsahern, hdanton, horms, kuba, linux-kernel, linux-mm, netdev, pabeni, syzkaller-bugs, tglx, Vladimir Oltean On Sun, Oct 12, 2025 at 11:46 AM Jamal Hadi Salim <jhs@mojatatu.com> wrote: > > On Sat, Oct 11, 2025 at 5:42 AM Eric Dumazet <edumazet@google.com> wrote: > > > > On Sat, Oct 11, 2025 at 12:41 AM syzbot > > <syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com> wrote: > > > > > > syzbot has found a reproducer for the following issue on: > > > > > > HEAD commit: 18a7e218cfcd Merge tag 'net-6.18-rc1' of git://git.kernel... > > > git tree: net-next > > > console output: https://syzkaller.appspot.com/x/log.txt?x=12504dcd980000 > > > kernel config: https://syzkaller.appspot.com/x/.config?x=61ab7fa743df0ec1 > > > dashboard link: https://syzkaller.appspot.com/bug?extid=51cd74c5dfeafd65e488 > > > compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 > > > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=14d2a542580000 > > > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=142149e2580000 > > > > > > Downloadable assets: > > > disk image: https://storage.googleapis.com/syzbot-assets/7a01e6dce97e/disk-18a7e218.raw.xz > > > vmlinux: https://storage.googleapis.com/syzbot-assets/5e1b7e41427f/vmlinux-18a7e218.xz > > > kernel image: https://storage.googleapis.com/syzbot-assets/69b558601209/bzImage-18a7e218.xz > > > > > > IMPORTANT: if you fix the issue, please add the following tag to the commit: > > > Reported-by: syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com > > > > > > sched: DL replenish lagged too much > > > rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: > > > rcu: 0-...!: (2 GPs behind) idle=7754/1/0x4000000000000000 softirq=15464/15465 fqs=1 > > > rcu: (detected by 1, t=10502 jiffies, g=11321, q=371 ncpus=2) > > > Sending NMI from CPU 1 to CPUs 0: > > > NMI backtrace for cpu 0 > > > CPU: 0 UID: 0 PID: 5948 Comm: syz-executor Not tainted syzkaller #0 PREEMPT(full) > > > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/02/2025 > > > RIP: 0010:rb_insert_color_cached include/linux/rbtree.h:113 [inline] > > > RIP: 0010:rb_add_cached include/linux/rbtree.h:183 [inline] > > > RIP: 0010:timerqueue_add+0x1a8/0x200 lib/timerqueue.c:40 > > > Code: e7 31 f6 e8 6a 0c de f6 42 80 3c 2b 00 74 08 4c 89 f7 e8 7b 0a de f6 4d 89 26 4d 8d 7e 08 4c 89 f8 48 c1 e8 03 42 80 3c 28 00 <74> 08 4c 89 ff e8 5e 0a de f6 4d 89 27 4d 85 e4 40 0f 95 c5 eb 07 > > > RSP: 0018:ffffc90000007cf0 EFLAGS: 00000046 > > > RAX: 1ffff110170c4f83 RBX: 1ffff110170c4f82 RCX: 0000000000000000 > > > RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88805de72358 > > > RBP: 0000000000000000 R08: ffff88805de72357 R09: 0000000000000000 > > > R10: ffff88805de72340 R11: ffffed100bbce46b R12: ffff88805de72340 > > > R13: dffffc0000000000 R14: ffff8880b8627c10 R15: ffff8880b8627c18 > > > FS: 000055557c657500(0000) GS:ffff888125d0f000(0000) knlGS:0000000000000000 > > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > > CR2: 0000200000000600 CR3: 000000002ee76000 CR4: 00000000003526f0 > > > Call Trace: > > > <IRQ> > > > __run_hrtimer kernel/time/hrtimer.c:1794 [inline] > > > __hrtimer_run_queues+0x656/0xc60 kernel/time/hrtimer.c:1841 > > > hrtimer_interrupt+0x45b/0xaa0 kernel/time/hrtimer.c:1903 > > > local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1041 [inline] > > > __sysvec_apic_timer_interrupt+0x108/0x410 arch/x86/kernel/apic/apic.c:1058 > > > instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1052 [inline] > > > sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1052 > > > </IRQ> > > > <TASK> > > > asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702 > > > RIP: 0010:pv_vcpu_is_preempted arch/x86/include/asm/paravirt.h:579 [inline] > > > RIP: 0010:vcpu_is_preempted arch/x86/include/asm/qspinlock.h:63 [inline] > > > RIP: 0010:owner_on_cpu include/linux/sched.h:2282 [inline] > > > RIP: 0010:mutex_spin_on_owner+0x189/0x360 kernel/locking/mutex.c:361 > > > Code: b6 04 30 84 c0 0f 85 59 01 00 00 48 8b 44 24 08 8b 18 48 8b 44 24 48 42 80 3c 30 00 74 0c 48 c7 c7 90 8c fa 8d e8 a7 cd 88 00 <48> 83 3d ff 27 5e 0c 00 0f 84 b9 01 00 00 48 89 df e8 41 e0 d5 ff > > > RSP: 0018:ffffc900034c7428 EFLAGS: 00000246 > > > RAX: 1ffffffff1bf5192 RBX: 0000000000000001 RCX: ffffffff819c6588 > > > RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff8f4df8a0 > > > RBP: 1ffffffff1e9bf14 R08: ffffffff8f4df8a7 R09: 1ffffffff1e9bf14 > > > R10: dffffc0000000000 R11: fffffbfff1e9bf15 R12: ffffffff8f4df8a0 > > > R13: ffffffff8f4df8f0 R14: dffffc0000000000 R15: ffff8880267a9e40 > > > mutex_optimistic_spin kernel/locking/mutex.c:464 [inline] > > > __mutex_lock_common kernel/locking/mutex.c:602 [inline] > > > __mutex_lock+0x311/0x1350 kernel/locking/mutex.c:760 > > > rtnl_net_lock include/linux/rtnetlink.h:130 [inline] > > > inet_rtm_newaddr+0x3b0/0x18b0 net/ipv4/devinet.c:978 > > > rtnetlink_rcv_msg+0x7cf/0xb70 net/core/rtnetlink.c:6954 > > > netlink_rcv_skb+0x205/0x470 net/netlink/af_netlink.c:2552 > > > netlink_unicast_kernel net/netlink/af_netlink.c:1320 [inline] > > > netlink_unicast+0x82f/0x9e0 net/netlink/af_netlink.c:1346 > > > netlink_sendmsg+0x805/0xb30 net/netlink/af_netlink.c:1896 > > > sock_sendmsg_nosec net/socket.c:727 [inline] > > > __sock_sendmsg+0x21c/0x270 net/socket.c:742 > > > __sys_sendto+0x3bd/0x520 net/socket.c:2244 > > > __do_sys_sendto net/socket.c:2251 [inline] > > > __se_sys_sendto net/socket.c:2247 [inline] > > > __x64_sys_sendto+0xde/0x100 net/socket.c:2247 > > > do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] > > > do_syscall_64+0xfa/0xfa0 arch/x86/entry/syscall_64.c:94 > > > entry_SYSCALL_64_after_hwframe+0x77/0x7f > > > RIP: 0033:0x7faade790d5c > > > Code: 2a 5f 02 00 44 8b 4c 24 2c 4c 8b 44 24 20 89 c5 44 8b 54 24 28 48 8b 54 24 18 b8 2c 00 00 00 48 8b 74 24 10 8b 7c 24 08 0f 05 <48> 3d 00 f0 ff ff 77 34 89 ef 48 89 44 24 08 e8 70 5f 02 00 48 8b > > > RSP: 002b:00007ffdd2e3b670 EFLAGS: 00000293 ORIG_RAX: 000000000000002c > > > RAX: ffffffffffffffda RBX: 00007faadf514620 RCX: 00007faade790d5c > > > RDX: 0000000000000028 RSI: 00007faadf514670 RDI: 0000000000000003 > > > RBP: 0000000000000000 R08: 00007ffdd2e3b6c4 R09: 000000000000000c > > > R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000003 > > > R13: 0000000000000000 R14: 00007faadf514670 R15: 0000000000000000 > > > </TASK> > > > rcu: rcu_preempt kthread timer wakeup didn't happen for 10499 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 > > > rcu: Possible timer handling issue on cpu=0 timer-softirq=4286 > > > rcu: rcu_preempt kthread starved for 10500 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 > > > rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. > > > rcu: RCU grace-period kthread stack dump: > > > task:rcu_preempt state:I stack:27224 pid:16 tgid:16 ppid:2 task_flags:0x208040 flags:0x00080000 > > > Call Trace: > > > <TASK> > > > context_switch kernel/sched/core.c:5325 [inline] > > > __schedule+0x1798/0x4cc0 kernel/sched/core.c:6929 > > > __schedule_loop kernel/sched/core.c:7011 [inline] > > > schedule+0x165/0x360 kernel/sched/core.c:7026 > > > schedule_timeout+0x12b/0x270 kernel/time/sleep_timeout.c:99 > > > rcu_gp_fqs_loop+0x301/0x1540 kernel/rcu/tree.c:2083 > > > rcu_gp_kthread+0x99/0x390 kernel/rcu/tree.c:2285 > > > kthread+0x711/0x8a0 kernel/kthread.c:463 > > > ret_from_fork+0x4bc/0x870 arch/x86/kernel/process.c:158 > > > ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 > > > </TASK> > > > > > > > > > --- > > > If you want syzbot to run the reproducer, reply with: > > > #syz test: git://repo/address.git branch-or-commit-hash > > > If you attach or paste a git patch, syzbot will apply it before testing. > > > > Yet another taprio report. > > > > If taprio can not be fixed, perhaps we should remove it from the > > kernel, or clearly marked as broken. > > (Then ask syzbot to no longer include it) > > Agreed on the challenge with taprio. > We need the stakeholders input: Vinicius - are you still working in > this space? Vladimir you also seem to have interest (or maybe nxp > does) in this? + Vladmir.. > At a minimum, we should mark it as broken unless the stakeholders want > to actively fix these issues. > Would syzbot still look at it if it was marked broken? > > cheers, > jamal ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [syzbot] [net?] [mm?] INFO: rcu detected stall in inet_rtm_newaddr (2) 2025-10-12 15:52 ` Jamal Hadi Salim @ 2025-10-14 20:11 ` Vladimir Oltean 2025-10-15 21:50 ` Jamal Hadi Salim 0 siblings, 1 reply; 11+ messages in thread From: Vladimir Oltean @ 2025-10-14 20:11 UTC (permalink / raw) To: Jamal Hadi Salim Cc: Eric Dumazet, syzbot, Vinicius Costa Gomes, davem, dsahern, hdanton, horms, kuba, linux-kernel, linux-mm, netdev, pabeni, syzkaller-bugs, tglx Hi Jamal, On Sun, Oct 12, 2025 at 11:52:54AM -0400, Jamal Hadi Salim wrote: > > > Yet another taprio report. > > > > > > If taprio can not be fixed, perhaps we should remove it from the > > > kernel, or clearly marked as broken. > > > (Then ask syzbot to no longer include it) > > > > Agreed on the challenge with taprio. > > We need the stakeholders input: Vinicius - are you still working in > > this space? Vladimir you also seem to have interest (or maybe nxp > > does) in this? > > + Vladmir.. > > > At a minimum, we should mark it as broken unless the stakeholders want > > to actively fix these issues. > > Would syzbot still look at it if it was marked broken? I still have interest in taprio, but at the moment I can't look at this any sooner than the second half of next week (unless someone else beats me to it). I've added a note not to lose track. What is the situation with syzbot reports? I don't actively monitor them, only if somebody happens to email me. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [syzbot] [net?] [mm?] INFO: rcu detected stall in inet_rtm_newaddr (2) 2025-10-14 20:11 ` Vladimir Oltean @ 2025-10-15 21:50 ` Jamal Hadi Salim 0 siblings, 0 replies; 11+ messages in thread From: Jamal Hadi Salim @ 2025-10-15 21:50 UTC (permalink / raw) To: Vladimir Oltean Cc: Eric Dumazet, syzbot, Vinicius Costa Gomes, davem, dsahern, hdanton, horms, kuba, linux-kernel, linux-mm, netdev, pabeni, syzkaller-bugs, tglx On Tue, Oct 14, 2025 at 4:11 PM Vladimir Oltean <vladimir.oltean@nxp.com> wrote: > > Hi Jamal, > > On Sun, Oct 12, 2025 at 11:52:54AM -0400, Jamal Hadi Salim wrote: > > > > Yet another taprio report. > > > > > > > > If taprio can not be fixed, perhaps we should remove it from the > > > > kernel, or clearly marked as broken. > > > > (Then ask syzbot to no longer include it) > > > > > > Agreed on the challenge with taprio. > > > We need the stakeholders input: Vinicius - are you still working in > > > this space? Vladimir you also seem to have interest (or maybe nxp > > > does) in this? > > > > + Vladmir.. > > > > > At a minimum, we should mark it as broken unless the stakeholders want > > > to actively fix these issues. > > > Would syzbot still look at it if it was marked broken? > > I still have interest in taprio, but at the moment I can't look at this > any sooner than the second half of next week (unless someone else beats > me to it). I've added a note not to lose track. > Thanks! > What is the situation with syzbot reports? I don't actively monitor them, > only if somebody happens to email me. These issues have been lingering forever.. They do get posted on the list. Here are samples: https://lore.kernel.org/netdev/676d25b2.050a0220.2f3838.0464.GAE@google.com/#r https://lore.kernel.org/netdev/67d2a576.050a0220.14e108.0030.GAE@google.com/ https://lore.kernel.org/netdev/67946a0c.050a0220.3ab881.0010.GAE@google.com/ https://lore.kernel.org/netdev/66e96979.050a0220.252d9a.000a.GAE@google.com/ https://lore.kernel.org/netdev/6777334a.050a0220.3a8527.0058.GAE@google.com/#t They all point to the same bisected commit, but am not sure whether they are the same issue To get you on Cc from syzbot - i think we'll have to add you as a maintainer for taprio if you'd be okk with that.. cheers, jamal ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [syzbot] [net?] [mm?] INFO: rcu detected stall in inet_rtm_newaddr (2) 2025-10-12 15:46 ` Jamal Hadi Salim 2025-10-12 15:52 ` Jamal Hadi Salim @ 2025-10-13 21:51 ` Vinicius Costa Gomes 2025-10-15 21:36 ` Jamal Hadi Salim 1 sibling, 1 reply; 11+ messages in thread From: Vinicius Costa Gomes @ 2025-10-13 21:51 UTC (permalink / raw) To: Jamal Hadi Salim, Eric Dumazet Cc: syzbot, davem, dsahern, hdanton, horms, kuba, linux-kernel, linux-mm, netdev, pabeni, syzkaller-bugs, tglx Jamal Hadi Salim <jhs@mojatatu.com> writes: > On Sat, Oct 11, 2025 at 5:42 AM Eric Dumazet <edumazet@google.com> wrote: >> >> On Sat, Oct 11, 2025 at 12:41 AM syzbot >> <syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com> wrote: >> > >> > syzbot has found a reproducer for the following issue on: >> > >> > HEAD commit: 18a7e218cfcd Merge tag 'net-6.18-rc1' of git://git.kernel... >> > git tree: net-next >> > console output: https://syzkaller.appspot.com/x/log.txt?x=12504dcd980000 >> > kernel config: https://syzkaller.appspot.com/x/.config?x=61ab7fa743df0ec1 >> > dashboard link: https://syzkaller.appspot.com/bug?extid=51cd74c5dfeafd65e488 >> > compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 >> > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=14d2a542580000 >> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=142149e2580000 >> > >> > Downloadable assets: >> > disk image: https://storage.googleapis.com/syzbot-assets/7a01e6dce97e/disk-18a7e218.raw.xz >> > vmlinux: https://storage.googleapis.com/syzbot-assets/5e1b7e41427f/vmlinux-18a7e218.xz >> > kernel image: https://storage.googleapis.com/syzbot-assets/69b558601209/bzImage-18a7e218.xz >> > >> > IMPORTANT: if you fix the issue, please add the following tag to the commit: >> > Reported-by: syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com >> > >> > sched: DL replenish lagged too much >> > rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: >> > rcu: 0-...!: (2 GPs behind) idle=7754/1/0x4000000000000000 softirq=15464/15465 fqs=1 >> > rcu: (detected by 1, t=10502 jiffies, g=11321, q=371 ncpus=2) >> > Sending NMI from CPU 1 to CPUs 0: >> > NMI backtrace for cpu 0 >> > CPU: 0 UID: 0 PID: 5948 Comm: syz-executor Not tainted syzkaller #0 PREEMPT(full) >> > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/02/2025 >> > RIP: 0010:rb_insert_color_cached include/linux/rbtree.h:113 [inline] >> > RIP: 0010:rb_add_cached include/linux/rbtree.h:183 [inline] >> > RIP: 0010:timerqueue_add+0x1a8/0x200 lib/timerqueue.c:40 >> > Code: e7 31 f6 e8 6a 0c de f6 42 80 3c 2b 00 74 08 4c 89 f7 e8 7b 0a de f6 4d 89 26 4d 8d 7e 08 4c 89 f8 48 c1 e8 03 42 80 3c 28 00 <74> 08 4c 89 ff e8 5e 0a de f6 4d 89 27 4d 85 e4 40 0f 95 c5 eb 07 >> > RSP: 0018:ffffc90000007cf0 EFLAGS: 00000046 >> > RAX: 1ffff110170c4f83 RBX: 1ffff110170c4f82 RCX: 0000000000000000 >> > RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88805de72358 >> > RBP: 0000000000000000 R08: ffff88805de72357 R09: 0000000000000000 >> > R10: ffff88805de72340 R11: ffffed100bbce46b R12: ffff88805de72340 >> > R13: dffffc0000000000 R14: ffff8880b8627c10 R15: ffff8880b8627c18 >> > FS: 000055557c657500(0000) GS:ffff888125d0f000(0000) knlGS:0000000000000000 >> > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >> > CR2: 0000200000000600 CR3: 000000002ee76000 CR4: 00000000003526f0 >> > Call Trace: >> > <IRQ> >> > __run_hrtimer kernel/time/hrtimer.c:1794 [inline] >> > __hrtimer_run_queues+0x656/0xc60 kernel/time/hrtimer.c:1841 >> > hrtimer_interrupt+0x45b/0xaa0 kernel/time/hrtimer.c:1903 >> > local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1041 [inline] >> > __sysvec_apic_timer_interrupt+0x108/0x410 arch/x86/kernel/apic/apic.c:1058 >> > instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1052 [inline] >> > sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1052 >> > </IRQ> >> > <TASK> >> > asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702 >> > RIP: 0010:pv_vcpu_is_preempted arch/x86/include/asm/paravirt.h:579 [inline] >> > RIP: 0010:vcpu_is_preempted arch/x86/include/asm/qspinlock.h:63 [inline] >> > RIP: 0010:owner_on_cpu include/linux/sched.h:2282 [inline] >> > RIP: 0010:mutex_spin_on_owner+0x189/0x360 kernel/locking/mutex.c:361 >> > Code: b6 04 30 84 c0 0f 85 59 01 00 00 48 8b 44 24 08 8b 18 48 8b 44 24 48 42 80 3c 30 00 74 0c 48 c7 c7 90 8c fa 8d e8 a7 cd 88 00 <48> 83 3d ff 27 5e 0c 00 0f 84 b9 01 00 00 48 89 df e8 41 e0 d5 ff >> > RSP: 0018:ffffc900034c7428 EFLAGS: 00000246 >> > RAX: 1ffffffff1bf5192 RBX: 0000000000000001 RCX: ffffffff819c6588 >> > RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff8f4df8a0 >> > RBP: 1ffffffff1e9bf14 R08: ffffffff8f4df8a7 R09: 1ffffffff1e9bf14 >> > R10: dffffc0000000000 R11: fffffbfff1e9bf15 R12: ffffffff8f4df8a0 >> > R13: ffffffff8f4df8f0 R14: dffffc0000000000 R15: ffff8880267a9e40 >> > mutex_optimistic_spin kernel/locking/mutex.c:464 [inline] >> > __mutex_lock_common kernel/locking/mutex.c:602 [inline] >> > __mutex_lock+0x311/0x1350 kernel/locking/mutex.c:760 >> > rtnl_net_lock include/linux/rtnetlink.h:130 [inline] >> > inet_rtm_newaddr+0x3b0/0x18b0 net/ipv4/devinet.c:978 >> > rtnetlink_rcv_msg+0x7cf/0xb70 net/core/rtnetlink.c:6954 >> > netlink_rcv_skb+0x205/0x470 net/netlink/af_netlink.c:2552 >> > netlink_unicast_kernel net/netlink/af_netlink.c:1320 [inline] >> > netlink_unicast+0x82f/0x9e0 net/netlink/af_netlink.c:1346 >> > netlink_sendmsg+0x805/0xb30 net/netlink/af_netlink.c:1896 >> > sock_sendmsg_nosec net/socket.c:727 [inline] >> > __sock_sendmsg+0x21c/0x270 net/socket.c:742 >> > __sys_sendto+0x3bd/0x520 net/socket.c:2244 >> > __do_sys_sendto net/socket.c:2251 [inline] >> > __se_sys_sendto net/socket.c:2247 [inline] >> > __x64_sys_sendto+0xde/0x100 net/socket.c:2247 >> > do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] >> > do_syscall_64+0xfa/0xfa0 arch/x86/entry/syscall_64.c:94 >> > entry_SYSCALL_64_after_hwframe+0x77/0x7f >> > RIP: 0033:0x7faade790d5c >> > Code: 2a 5f 02 00 44 8b 4c 24 2c 4c 8b 44 24 20 89 c5 44 8b 54 24 28 48 8b 54 24 18 b8 2c 00 00 00 48 8b 74 24 10 8b 7c 24 08 0f 05 <48> 3d 00 f0 ff ff 77 34 89 ef 48 89 44 24 08 e8 70 5f 02 00 48 8b >> > RSP: 002b:00007ffdd2e3b670 EFLAGS: 00000293 ORIG_RAX: 000000000000002c >> > RAX: ffffffffffffffda RBX: 00007faadf514620 RCX: 00007faade790d5c >> > RDX: 0000000000000028 RSI: 00007faadf514670 RDI: 0000000000000003 >> > RBP: 0000000000000000 R08: 00007ffdd2e3b6c4 R09: 000000000000000c >> > R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000003 >> > R13: 0000000000000000 R14: 00007faadf514670 R15: 0000000000000000 >> > </TASK> >> > rcu: rcu_preempt kthread timer wakeup didn't happen for 10499 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 >> > rcu: Possible timer handling issue on cpu=0 timer-softirq=4286 >> > rcu: rcu_preempt kthread starved for 10500 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 >> > rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. >> > rcu: RCU grace-period kthread stack dump: >> > task:rcu_preempt state:I stack:27224 pid:16 tgid:16 ppid:2 task_flags:0x208040 flags:0x00080000 >> > Call Trace: >> > <TASK> >> > context_switch kernel/sched/core.c:5325 [inline] >> > __schedule+0x1798/0x4cc0 kernel/sched/core.c:6929 >> > __schedule_loop kernel/sched/core.c:7011 [inline] >> > schedule+0x165/0x360 kernel/sched/core.c:7026 >> > schedule_timeout+0x12b/0x270 kernel/time/sleep_timeout.c:99 >> > rcu_gp_fqs_loop+0x301/0x1540 kernel/rcu/tree.c:2083 >> > rcu_gp_kthread+0x99/0x390 kernel/rcu/tree.c:2285 >> > kthread+0x711/0x8a0 kernel/kthread.c:463 >> > ret_from_fork+0x4bc/0x870 arch/x86/kernel/process.c:158 >> > ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 >> > </TASK> >> > >> > >> > --- >> > If you want syzbot to run the reproducer, reply with: >> > #syz test: git://repo/address.git branch-or-commit-hash >> > If you attach or paste a git patch, syzbot will apply it before testing. >> >> Yet another taprio report. >> >> If taprio can not be fixed, perhaps we should remove it from the >> kernel, or clearly marked as broken. >> (Then ask syzbot to no longer include it) > > Agreed on the challenge with taprio. > We need the stakeholders input: Vinicius - are you still working in > this space? Vladimir you also seem to have interest (or maybe nxp > does) in this? No, I am not working on this space anymore. I will talk with other Intel folks (and my manager) and see what we can do. But if others that find it useful can help even better. > At a minimum, we should mark it as broken unless the stakeholders want > to actively fix these issues. > Would syzbot still look at it if it was marked broken? > > cheers, > jamal > Cheers, -- Vinicius ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [syzbot] [net?] [mm?] INFO: rcu detected stall in inet_rtm_newaddr (2) 2025-10-13 21:51 ` Vinicius Costa Gomes @ 2025-10-15 21:36 ` Jamal Hadi Salim 2025-10-15 22:30 ` Vinicius Costa Gomes 0 siblings, 1 reply; 11+ messages in thread From: Jamal Hadi Salim @ 2025-10-15 21:36 UTC (permalink / raw) To: Vinicius Costa Gomes Cc: Eric Dumazet, syzbot, davem, dsahern, hdanton, horms, kuba, linux-kernel, linux-mm, netdev, pabeni, syzkaller-bugs, tglx On Mon, Oct 13, 2025 at 5:51 PM Vinicius Costa Gomes <vinicius.gomes@intel.com> wrote: > > Jamal Hadi Salim <jhs@mojatatu.com> writes: > > > On Sat, Oct 11, 2025 at 5:42 AM Eric Dumazet <edumazet@google.com> wrote: > >> > >> On Sat, Oct 11, 2025 at 12:41 AM syzbot > >> <syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com> wrote: > >> > > >> > syzbot has found a reproducer for the following issue on: > >> > > >> > HEAD commit: 18a7e218cfcd Merge tag 'net-6.18-rc1' of git://git.kernel... > >> > git tree: net-next > >> > console output: https://syzkaller.appspot.com/x/log.txt?x=12504dcd980000 > >> > kernel config: https://syzkaller.appspot.com/x/.config?x=61ab7fa743df0ec1 > >> > dashboard link: https://syzkaller.appspot.com/bug?extid=51cd74c5dfeafd65e488 > >> > compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 > >> > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=14d2a542580000 > >> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=142149e2580000 > >> > > >> > Downloadable assets: > >> > disk image: https://storage.googleapis.com/syzbot-assets/7a01e6dce97e/disk-18a7e218.raw.xz > >> > vmlinux: https://storage.googleapis.com/syzbot-assets/5e1b7e41427f/vmlinux-18a7e218.xz > >> > kernel image: https://storage.googleapis.com/syzbot-assets/69b558601209/bzImage-18a7e218.xz > >> > > >> > IMPORTANT: if you fix the issue, please add the following tag to the commit: > >> > Reported-by: syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com > >> > > >> > sched: DL replenish lagged too much > >> > rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: > >> > rcu: 0-...!: (2 GPs behind) idle=7754/1/0x4000000000000000 softirq=15464/15465 fqs=1 > >> > rcu: (detected by 1, t=10502 jiffies, g=11321, q=371 ncpus=2) > >> > Sending NMI from CPU 1 to CPUs 0: > >> > NMI backtrace for cpu 0 > >> > CPU: 0 UID: 0 PID: 5948 Comm: syz-executor Not tainted syzkaller #0 PREEMPT(full) > >> > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/02/2025 > >> > RIP: 0010:rb_insert_color_cached include/linux/rbtree.h:113 [inline] > >> > RIP: 0010:rb_add_cached include/linux/rbtree.h:183 [inline] > >> > RIP: 0010:timerqueue_add+0x1a8/0x200 lib/timerqueue.c:40 > >> > Code: e7 31 f6 e8 6a 0c de f6 42 80 3c 2b 00 74 08 4c 89 f7 e8 7b 0a de f6 4d 89 26 4d 8d 7e 08 4c 89 f8 48 c1 e8 03 42 80 3c 28 00 <74> 08 4c 89 ff e8 5e 0a de f6 4d 89 27 4d 85 e4 40 0f 95 c5 eb 07 > >> > RSP: 0018:ffffc90000007cf0 EFLAGS: 00000046 > >> > RAX: 1ffff110170c4f83 RBX: 1ffff110170c4f82 RCX: 0000000000000000 > >> > RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88805de72358 > >> > RBP: 0000000000000000 R08: ffff88805de72357 R09: 0000000000000000 > >> > R10: ffff88805de72340 R11: ffffed100bbce46b R12: ffff88805de72340 > >> > R13: dffffc0000000000 R14: ffff8880b8627c10 R15: ffff8880b8627c18 > >> > FS: 000055557c657500(0000) GS:ffff888125d0f000(0000) knlGS:0000000000000000 > >> > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > >> > CR2: 0000200000000600 CR3: 000000002ee76000 CR4: 00000000003526f0 > >> > Call Trace: > >> > <IRQ> > >> > __run_hrtimer kernel/time/hrtimer.c:1794 [inline] > >> > __hrtimer_run_queues+0x656/0xc60 kernel/time/hrtimer.c:1841 > >> > hrtimer_interrupt+0x45b/0xaa0 kernel/time/hrtimer.c:1903 > >> > local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1041 [inline] > >> > __sysvec_apic_timer_interrupt+0x108/0x410 arch/x86/kernel/apic/apic.c:1058 > >> > instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1052 [inline] > >> > sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1052 > >> > </IRQ> > >> > <TASK> > >> > asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702 > >> > RIP: 0010:pv_vcpu_is_preempted arch/x86/include/asm/paravirt.h:579 [inline] > >> > RIP: 0010:vcpu_is_preempted arch/x86/include/asm/qspinlock.h:63 [inline] > >> > RIP: 0010:owner_on_cpu include/linux/sched.h:2282 [inline] > >> > RIP: 0010:mutex_spin_on_owner+0x189/0x360 kernel/locking/mutex.c:361 > >> > Code: b6 04 30 84 c0 0f 85 59 01 00 00 48 8b 44 24 08 8b 18 48 8b 44 24 48 42 80 3c 30 00 74 0c 48 c7 c7 90 8c fa 8d e8 a7 cd 88 00 <48> 83 3d ff 27 5e 0c 00 0f 84 b9 01 00 00 48 89 df e8 41 e0 d5 ff > >> > RSP: 0018:ffffc900034c7428 EFLAGS: 00000246 > >> > RAX: 1ffffffff1bf5192 RBX: 0000000000000001 RCX: ffffffff819c6588 > >> > RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff8f4df8a0 > >> > RBP: 1ffffffff1e9bf14 R08: ffffffff8f4df8a7 R09: 1ffffffff1e9bf14 > >> > R10: dffffc0000000000 R11: fffffbfff1e9bf15 R12: ffffffff8f4df8a0 > >> > R13: ffffffff8f4df8f0 R14: dffffc0000000000 R15: ffff8880267a9e40 > >> > mutex_optimistic_spin kernel/locking/mutex.c:464 [inline] > >> > __mutex_lock_common kernel/locking/mutex.c:602 [inline] > >> > __mutex_lock+0x311/0x1350 kernel/locking/mutex.c:760 > >> > rtnl_net_lock include/linux/rtnetlink.h:130 [inline] > >> > inet_rtm_newaddr+0x3b0/0x18b0 net/ipv4/devinet.c:978 > >> > rtnetlink_rcv_msg+0x7cf/0xb70 net/core/rtnetlink.c:6954 > >> > netlink_rcv_skb+0x205/0x470 net/netlink/af_netlink.c:2552 > >> > netlink_unicast_kernel net/netlink/af_netlink.c:1320 [inline] > >> > netlink_unicast+0x82f/0x9e0 net/netlink/af_netlink.c:1346 > >> > netlink_sendmsg+0x805/0xb30 net/netlink/af_netlink.c:1896 > >> > sock_sendmsg_nosec net/socket.c:727 [inline] > >> > __sock_sendmsg+0x21c/0x270 net/socket.c:742 > >> > __sys_sendto+0x3bd/0x520 net/socket.c:2244 > >> > __do_sys_sendto net/socket.c:2251 [inline] > >> > __se_sys_sendto net/socket.c:2247 [inline] > >> > __x64_sys_sendto+0xde/0x100 net/socket.c:2247 > >> > do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] > >> > do_syscall_64+0xfa/0xfa0 arch/x86/entry/syscall_64.c:94 > >> > entry_SYSCALL_64_after_hwframe+0x77/0x7f > >> > RIP: 0033:0x7faade790d5c > >> > Code: 2a 5f 02 00 44 8b 4c 24 2c 4c 8b 44 24 20 89 c5 44 8b 54 24 28 48 8b 54 24 18 b8 2c 00 00 00 48 8b 74 24 10 8b 7c 24 08 0f 05 <48> 3d 00 f0 ff ff 77 34 89 ef 48 89 44 24 08 e8 70 5f 02 00 48 8b > >> > RSP: 002b:00007ffdd2e3b670 EFLAGS: 00000293 ORIG_RAX: 000000000000002c > >> > RAX: ffffffffffffffda RBX: 00007faadf514620 RCX: 00007faade790d5c > >> > RDX: 0000000000000028 RSI: 00007faadf514670 RDI: 0000000000000003 > >> > RBP: 0000000000000000 R08: 00007ffdd2e3b6c4 R09: 000000000000000c > >> > R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000003 > >> > R13: 0000000000000000 R14: 00007faadf514670 R15: 0000000000000000 > >> > </TASK> > >> > rcu: rcu_preempt kthread timer wakeup didn't happen for 10499 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 > >> > rcu: Possible timer handling issue on cpu=0 timer-softirq=4286 > >> > rcu: rcu_preempt kthread starved for 10500 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 > >> > rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. > >> > rcu: RCU grace-period kthread stack dump: > >> > task:rcu_preempt state:I stack:27224 pid:16 tgid:16 ppid:2 task_flags:0x208040 flags:0x00080000 > >> > Call Trace: > >> > <TASK> > >> > context_switch kernel/sched/core.c:5325 [inline] > >> > __schedule+0x1798/0x4cc0 kernel/sched/core.c:6929 > >> > __schedule_loop kernel/sched/core.c:7011 [inline] > >> > schedule+0x165/0x360 kernel/sched/core.c:7026 > >> > schedule_timeout+0x12b/0x270 kernel/time/sleep_timeout.c:99 > >> > rcu_gp_fqs_loop+0x301/0x1540 kernel/rcu/tree.c:2083 > >> > rcu_gp_kthread+0x99/0x390 kernel/rcu/tree.c:2285 > >> > kthread+0x711/0x8a0 kernel/kthread.c:463 > >> > ret_from_fork+0x4bc/0x870 arch/x86/kernel/process.c:158 > >> > ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 > >> > </TASK> > >> > > >> > > >> > --- > >> > If you want syzbot to run the reproducer, reply with: > >> > #syz test: git://repo/address.git branch-or-commit-hash > >> > If you attach or paste a git patch, syzbot will apply it before testing. > >> > >> Yet another taprio report. > >> > >> If taprio can not be fixed, perhaps we should remove it from the > >> kernel, or clearly marked as broken. > >> (Then ask syzbot to no longer include it) > > > > Agreed on the challenge with taprio. > > We need the stakeholders input: Vinicius - are you still working in > > this space? Vladimir you also seem to have interest (or maybe nxp > > does) in this? > > No, I am not working on this space anymore. > > I will talk with other Intel folks (and my manager) and see what we can > do. I assume your customers are still interested in this working? If yes, that would be a good pitch to the manager. In my (extreme) view, another selling point is that there is an ethical obligation to ensure things continue to work as intended. Getting patches in is the easy part. cheers, jamal >But if others that find it useful can help even better. > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [syzbot] [net?] [mm?] INFO: rcu detected stall in inet_rtm_newaddr (2) 2025-10-15 21:36 ` Jamal Hadi Salim @ 2025-10-15 22:30 ` Vinicius Costa Gomes 2025-10-24 0:11 ` Vinicius Costa Gomes 0 siblings, 1 reply; 11+ messages in thread From: Vinicius Costa Gomes @ 2025-10-15 22:30 UTC (permalink / raw) To: Jamal Hadi Salim Cc: Eric Dumazet, syzbot, davem, dsahern, hdanton, horms, kuba, linux-kernel, linux-mm, netdev, pabeni, syzkaller-bugs, tglx Jamal Hadi Salim <jhs@mojatatu.com> writes: > On Mon, Oct 13, 2025 at 5:51 PM Vinicius Costa Gomes > <vinicius.gomes@intel.com> wrote: >> >> Jamal Hadi Salim <jhs@mojatatu.com> writes: >> >> > On Sat, Oct 11, 2025 at 5:42 AM Eric Dumazet <edumazet@google.com> wrote: >> >> >> >> On Sat, Oct 11, 2025 at 12:41 AM syzbot >> >> <syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com> wrote: >> >> > >> >> > syzbot has found a reproducer for the following issue on: >> >> > >> >> > HEAD commit: 18a7e218cfcd Merge tag 'net-6.18-rc1' of git://git.kernel... >> >> > git tree: net-next >> >> > console output: https://syzkaller.appspot.com/x/log.txt?x=12504dcd980000 >> >> > kernel config: https://syzkaller.appspot.com/x/.config?x=61ab7fa743df0ec1 >> >> > dashboard link: https://syzkaller.appspot.com/bug?extid=51cd74c5dfeafd65e488 >> >> > compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 >> >> > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=14d2a542580000 >> >> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=142149e2580000 >> >> > >> >> > Downloadable assets: >> >> > disk image: https://storage.googleapis.com/syzbot-assets/7a01e6dce97e/disk-18a7e218.raw.xz >> >> > vmlinux: https://storage.googleapis.com/syzbot-assets/5e1b7e41427f/vmlinux-18a7e218.xz >> >> > kernel image: https://storage.googleapis.com/syzbot-assets/69b558601209/bzImage-18a7e218.xz >> >> > >> >> > IMPORTANT: if you fix the issue, please add the following tag to the commit: >> >> > Reported-by: syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com >> >> > >> >> > sched: DL replenish lagged too much >> >> > rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: >> >> > rcu: 0-...!: (2 GPs behind) idle=7754/1/0x4000000000000000 softirq=15464/15465 fqs=1 >> >> > rcu: (detected by 1, t=10502 jiffies, g=11321, q=371 ncpus=2) >> >> > Sending NMI from CPU 1 to CPUs 0: >> >> > NMI backtrace for cpu 0 >> >> > CPU: 0 UID: 0 PID: 5948 Comm: syz-executor Not tainted syzkaller #0 PREEMPT(full) >> >> > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/02/2025 >> >> > RIP: 0010:rb_insert_color_cached include/linux/rbtree.h:113 [inline] >> >> > RIP: 0010:rb_add_cached include/linux/rbtree.h:183 [inline] >> >> > RIP: 0010:timerqueue_add+0x1a8/0x200 lib/timerqueue.c:40 >> >> > Code: e7 31 f6 e8 6a 0c de f6 42 80 3c 2b 00 74 08 4c 89 f7 e8 7b 0a de f6 4d 89 26 4d 8d 7e 08 4c 89 f8 48 c1 e8 03 42 80 3c 28 00 <74> 08 4c 89 ff e8 5e 0a de f6 4d 89 27 4d 85 e4 40 0f 95 c5 eb 07 >> >> > RSP: 0018:ffffc90000007cf0 EFLAGS: 00000046 >> >> > RAX: 1ffff110170c4f83 RBX: 1ffff110170c4f82 RCX: 0000000000000000 >> >> > RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88805de72358 >> >> > RBP: 0000000000000000 R08: ffff88805de72357 R09: 0000000000000000 >> >> > R10: ffff88805de72340 R11: ffffed100bbce46b R12: ffff88805de72340 >> >> > R13: dffffc0000000000 R14: ffff8880b8627c10 R15: ffff8880b8627c18 >> >> > FS: 000055557c657500(0000) GS:ffff888125d0f000(0000) knlGS:0000000000000000 >> >> > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >> >> > CR2: 0000200000000600 CR3: 000000002ee76000 CR4: 00000000003526f0 >> >> > Call Trace: >> >> > <IRQ> >> >> > __run_hrtimer kernel/time/hrtimer.c:1794 [inline] >> >> > __hrtimer_run_queues+0x656/0xc60 kernel/time/hrtimer.c:1841 >> >> > hrtimer_interrupt+0x45b/0xaa0 kernel/time/hrtimer.c:1903 >> >> > local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1041 [inline] >> >> > __sysvec_apic_timer_interrupt+0x108/0x410 arch/x86/kernel/apic/apic.c:1058 >> >> > instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1052 [inline] >> >> > sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1052 >> >> > </IRQ> >> >> > <TASK> >> >> > asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702 >> >> > RIP: 0010:pv_vcpu_is_preempted arch/x86/include/asm/paravirt.h:579 [inline] >> >> > RIP: 0010:vcpu_is_preempted arch/x86/include/asm/qspinlock.h:63 [inline] >> >> > RIP: 0010:owner_on_cpu include/linux/sched.h:2282 [inline] >> >> > RIP: 0010:mutex_spin_on_owner+0x189/0x360 kernel/locking/mutex.c:361 >> >> > Code: b6 04 30 84 c0 0f 85 59 01 00 00 48 8b 44 24 08 8b 18 48 8b 44 24 48 42 80 3c 30 00 74 0c 48 c7 c7 90 8c fa 8d e8 a7 cd 88 00 <48> 83 3d ff 27 5e 0c 00 0f 84 b9 01 00 00 48 89 df e8 41 e0 d5 ff >> >> > RSP: 0018:ffffc900034c7428 EFLAGS: 00000246 >> >> > RAX: 1ffffffff1bf5192 RBX: 0000000000000001 RCX: ffffffff819c6588 >> >> > RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff8f4df8a0 >> >> > RBP: 1ffffffff1e9bf14 R08: ffffffff8f4df8a7 R09: 1ffffffff1e9bf14 >> >> > R10: dffffc0000000000 R11: fffffbfff1e9bf15 R12: ffffffff8f4df8a0 >> >> > R13: ffffffff8f4df8f0 R14: dffffc0000000000 R15: ffff8880267a9e40 >> >> > mutex_optimistic_spin kernel/locking/mutex.c:464 [inline] >> >> > __mutex_lock_common kernel/locking/mutex.c:602 [inline] >> >> > __mutex_lock+0x311/0x1350 kernel/locking/mutex.c:760 >> >> > rtnl_net_lock include/linux/rtnetlink.h:130 [inline] >> >> > inet_rtm_newaddr+0x3b0/0x18b0 net/ipv4/devinet.c:978 >> >> > rtnetlink_rcv_msg+0x7cf/0xb70 net/core/rtnetlink.c:6954 >> >> > netlink_rcv_skb+0x205/0x470 net/netlink/af_netlink.c:2552 >> >> > netlink_unicast_kernel net/netlink/af_netlink.c:1320 [inline] >> >> > netlink_unicast+0x82f/0x9e0 net/netlink/af_netlink.c:1346 >> >> > netlink_sendmsg+0x805/0xb30 net/netlink/af_netlink.c:1896 >> >> > sock_sendmsg_nosec net/socket.c:727 [inline] >> >> > __sock_sendmsg+0x21c/0x270 net/socket.c:742 >> >> > __sys_sendto+0x3bd/0x520 net/socket.c:2244 >> >> > __do_sys_sendto net/socket.c:2251 [inline] >> >> > __se_sys_sendto net/socket.c:2247 [inline] >> >> > __x64_sys_sendto+0xde/0x100 net/socket.c:2247 >> >> > do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] >> >> > do_syscall_64+0xfa/0xfa0 arch/x86/entry/syscall_64.c:94 >> >> > entry_SYSCALL_64_after_hwframe+0x77/0x7f >> >> > RIP: 0033:0x7faade790d5c >> >> > Code: 2a 5f 02 00 44 8b 4c 24 2c 4c 8b 44 24 20 89 c5 44 8b 54 24 28 48 8b 54 24 18 b8 2c 00 00 00 48 8b 74 24 10 8b 7c 24 08 0f 05 <48> 3d 00 f0 ff ff 77 34 89 ef 48 89 44 24 08 e8 70 5f 02 00 48 8b >> >> > RSP: 002b:00007ffdd2e3b670 EFLAGS: 00000293 ORIG_RAX: 000000000000002c >> >> > RAX: ffffffffffffffda RBX: 00007faadf514620 RCX: 00007faade790d5c >> >> > RDX: 0000000000000028 RSI: 00007faadf514670 RDI: 0000000000000003 >> >> > RBP: 0000000000000000 R08: 00007ffdd2e3b6c4 R09: 000000000000000c >> >> > R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000003 >> >> > R13: 0000000000000000 R14: 00007faadf514670 R15: 0000000000000000 >> >> > </TASK> >> >> > rcu: rcu_preempt kthread timer wakeup didn't happen for 10499 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 >> >> > rcu: Possible timer handling issue on cpu=0 timer-softirq=4286 >> >> > rcu: rcu_preempt kthread starved for 10500 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 >> >> > rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. >> >> > rcu: RCU grace-period kthread stack dump: >> >> > task:rcu_preempt state:I stack:27224 pid:16 tgid:16 ppid:2 task_flags:0x208040 flags:0x00080000 >> >> > Call Trace: >> >> > <TASK> >> >> > context_switch kernel/sched/core.c:5325 [inline] >> >> > __schedule+0x1798/0x4cc0 kernel/sched/core.c:6929 >> >> > __schedule_loop kernel/sched/core.c:7011 [inline] >> >> > schedule+0x165/0x360 kernel/sched/core.c:7026 >> >> > schedule_timeout+0x12b/0x270 kernel/time/sleep_timeout.c:99 >> >> > rcu_gp_fqs_loop+0x301/0x1540 kernel/rcu/tree.c:2083 >> >> > rcu_gp_kthread+0x99/0x390 kernel/rcu/tree.c:2285 >> >> > kthread+0x711/0x8a0 kernel/kthread.c:463 >> >> > ret_from_fork+0x4bc/0x870 arch/x86/kernel/process.c:158 >> >> > ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 >> >> > </TASK> >> >> > >> >> > >> >> > --- >> >> > If you want syzbot to run the reproducer, reply with: >> >> > #syz test: git://repo/address.git branch-or-commit-hash >> >> > If you attach or paste a git patch, syzbot will apply it before testing. >> >> >> >> Yet another taprio report. >> >> >> >> If taprio can not be fixed, perhaps we should remove it from the >> >> kernel, or clearly marked as broken. >> >> (Then ask syzbot to no longer include it) >> > >> > Agreed on the challenge with taprio. >> > We need the stakeholders input: Vinicius - are you still working in >> > this space? Vladimir you also seem to have interest (or maybe nxp >> > does) in this? >> >> No, I am not working on this space anymore. >> >> I will talk with other Intel folks (and my manager) and see what we can >> do. > > I assume your customers are still interested in this working? If yes, > that would be a good pitch to the manager. I did talk with some people here, and let's just say that I am hearing positive noises. So chances are that I should be able to dedicate some of my "job time" to this area again. > In my (extreme) view, another selling point is that there is an > ethical obligation to ensure things continue to work as intended. > Getting patches in is the easy part. > > cheers, > jamal > >>But if others that find it useful can help even better. >> > Cheers, -- Vinicius ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [syzbot] [net?] [mm?] INFO: rcu detected stall in inet_rtm_newaddr (2) 2025-10-15 22:30 ` Vinicius Costa Gomes @ 2025-10-24 0:11 ` Vinicius Costa Gomes 0 siblings, 0 replies; 11+ messages in thread From: Vinicius Costa Gomes @ 2025-10-24 0:11 UTC (permalink / raw) To: Jamal Hadi Salim Cc: Eric Dumazet, syzbot, davem, dsahern, hdanton, horms, kuba, linux-kernel, linux-mm, netdev, pabeni, syzkaller-bugs, tglx Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: > Jamal Hadi Salim <jhs@mojatatu.com> writes: > >> On Mon, Oct 13, 2025 at 5:51 PM Vinicius Costa Gomes >> <vinicius.gomes@intel.com> wrote: >>> >>> Jamal Hadi Salim <jhs@mojatatu.com> writes: >>> >>> > On Sat, Oct 11, 2025 at 5:42 AM Eric Dumazet <edumazet@google.com> wrote: >>> >> >>> >> On Sat, Oct 11, 2025 at 12:41 AM syzbot >>> >> <syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com> wrote: >>> >> > >>> >> > syzbot has found a reproducer for the following issue on: >>> >> > >>> >> > HEAD commit: 18a7e218cfcd Merge tag 'net-6.18-rc1' of git://git.kernel... >>> >> > git tree: net-next >>> >> > console output: https://syzkaller.appspot.com/x/log.txt?x=12504dcd980000 >>> >> > kernel config: https://syzkaller.appspot.com/x/.config?x=61ab7fa743df0ec1 >>> >> > dashboard link: https://syzkaller.appspot.com/bug?extid=51cd74c5dfeafd65e488 >>> >> > compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 >>> >> > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=14d2a542580000 >>> >> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=142149e2580000 >>> >> > >>> >> > Downloadable assets: >>> >> > disk image: https://storage.googleapis.com/syzbot-assets/7a01e6dce97e/disk-18a7e218.raw.xz >>> >> > vmlinux: https://storage.googleapis.com/syzbot-assets/5e1b7e41427f/vmlinux-18a7e218.xz >>> >> > kernel image: https://storage.googleapis.com/syzbot-assets/69b558601209/bzImage-18a7e218.xz >>> >> > >>> >> > IMPORTANT: if you fix the issue, please add the following tag to the commit: >>> >> > Reported-by: syzbot+51cd74c5dfeafd65e488@syzkaller.appspotmail.com >>> >> > >>> >> > sched: DL replenish lagged too much >>> >> > rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: >>> >> > rcu: 0-...!: (2 GPs behind) idle=7754/1/0x4000000000000000 softirq=15464/15465 fqs=1 >>> >> > rcu: (detected by 1, t=10502 jiffies, g=11321, q=371 ncpus=2) >>> >> > Sending NMI from CPU 1 to CPUs 0: >>> >> > NMI backtrace for cpu 0 >>> >> > CPU: 0 UID: 0 PID: 5948 Comm: syz-executor Not tainted syzkaller #0 PREEMPT(full) >>> >> > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/02/2025 >>> >> > RIP: 0010:rb_insert_color_cached include/linux/rbtree.h:113 [inline] >>> >> > RIP: 0010:rb_add_cached include/linux/rbtree.h:183 [inline] >>> >> > RIP: 0010:timerqueue_add+0x1a8/0x200 lib/timerqueue.c:40 >>> >> > Code: e7 31 f6 e8 6a 0c de f6 42 80 3c 2b 00 74 08 4c 89 f7 e8 7b 0a de f6 4d 89 26 4d 8d 7e 08 4c 89 f8 48 c1 e8 03 42 80 3c 28 00 <74> 08 4c 89 ff e8 5e 0a de f6 4d 89 27 4d 85 e4 40 0f 95 c5 eb 07 >>> >> > RSP: 0018:ffffc90000007cf0 EFLAGS: 00000046 >>> >> > RAX: 1ffff110170c4f83 RBX: 1ffff110170c4f82 RCX: 0000000000000000 >>> >> > RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88805de72358 >>> >> > RBP: 0000000000000000 R08: ffff88805de72357 R09: 0000000000000000 >>> >> > R10: ffff88805de72340 R11: ffffed100bbce46b R12: ffff88805de72340 >>> >> > R13: dffffc0000000000 R14: ffff8880b8627c10 R15: ffff8880b8627c18 >>> >> > FS: 000055557c657500(0000) GS:ffff888125d0f000(0000) knlGS:0000000000000000 >>> >> > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >>> >> > CR2: 0000200000000600 CR3: 000000002ee76000 CR4: 00000000003526f0 >>> >> > Call Trace: >>> >> > <IRQ> >>> >> > __run_hrtimer kernel/time/hrtimer.c:1794 [inline] >>> >> > __hrtimer_run_queues+0x656/0xc60 kernel/time/hrtimer.c:1841 >>> >> > hrtimer_interrupt+0x45b/0xaa0 kernel/time/hrtimer.c:1903 >>> >> > local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1041 [inline] >>> >> > __sysvec_apic_timer_interrupt+0x108/0x410 arch/x86/kernel/apic/apic.c:1058 >>> >> > instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1052 [inline] >>> >> > sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1052 >>> >> > </IRQ> >>> >> > <TASK> >>> >> > asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702 >>> >> > RIP: 0010:pv_vcpu_is_preempted arch/x86/include/asm/paravirt.h:579 [inline] >>> >> > RIP: 0010:vcpu_is_preempted arch/x86/include/asm/qspinlock.h:63 [inline] >>> >> > RIP: 0010:owner_on_cpu include/linux/sched.h:2282 [inline] >>> >> > RIP: 0010:mutex_spin_on_owner+0x189/0x360 kernel/locking/mutex.c:361 >>> >> > Code: b6 04 30 84 c0 0f 85 59 01 00 00 48 8b 44 24 08 8b 18 48 8b 44 24 48 42 80 3c 30 00 74 0c 48 c7 c7 90 8c fa 8d e8 a7 cd 88 00 <48> 83 3d ff 27 5e 0c 00 0f 84 b9 01 00 00 48 89 df e8 41 e0 d5 ff >>> >> > RSP: 0018:ffffc900034c7428 EFLAGS: 00000246 >>> >> > RAX: 1ffffffff1bf5192 RBX: 0000000000000001 RCX: ffffffff819c6588 >>> >> > RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff8f4df8a0 >>> >> > RBP: 1ffffffff1e9bf14 R08: ffffffff8f4df8a7 R09: 1ffffffff1e9bf14 >>> >> > R10: dffffc0000000000 R11: fffffbfff1e9bf15 R12: ffffffff8f4df8a0 >>> >> > R13: ffffffff8f4df8f0 R14: dffffc0000000000 R15: ffff8880267a9e40 >>> >> > mutex_optimistic_spin kernel/locking/mutex.c:464 [inline] >>> >> > __mutex_lock_common kernel/locking/mutex.c:602 [inline] >>> >> > __mutex_lock+0x311/0x1350 kernel/locking/mutex.c:760 >>> >> > rtnl_net_lock include/linux/rtnetlink.h:130 [inline] >>> >> > inet_rtm_newaddr+0x3b0/0x18b0 net/ipv4/devinet.c:978 >>> >> > rtnetlink_rcv_msg+0x7cf/0xb70 net/core/rtnetlink.c:6954 >>> >> > netlink_rcv_skb+0x205/0x470 net/netlink/af_netlink.c:2552 >>> >> > netlink_unicast_kernel net/netlink/af_netlink.c:1320 [inline] >>> >> > netlink_unicast+0x82f/0x9e0 net/netlink/af_netlink.c:1346 >>> >> > netlink_sendmsg+0x805/0xb30 net/netlink/af_netlink.c:1896 >>> >> > sock_sendmsg_nosec net/socket.c:727 [inline] >>> >> > __sock_sendmsg+0x21c/0x270 net/socket.c:742 >>> >> > __sys_sendto+0x3bd/0x520 net/socket.c:2244 >>> >> > __do_sys_sendto net/socket.c:2251 [inline] >>> >> > __se_sys_sendto net/socket.c:2247 [inline] >>> >> > __x64_sys_sendto+0xde/0x100 net/socket.c:2247 >>> >> > do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] >>> >> > do_syscall_64+0xfa/0xfa0 arch/x86/entry/syscall_64.c:94 >>> >> > entry_SYSCALL_64_after_hwframe+0x77/0x7f >>> >> > RIP: 0033:0x7faade790d5c >>> >> > Code: 2a 5f 02 00 44 8b 4c 24 2c 4c 8b 44 24 20 89 c5 44 8b 54 24 28 48 8b 54 24 18 b8 2c 00 00 00 48 8b 74 24 10 8b 7c 24 08 0f 05 <48> 3d 00 f0 ff ff 77 34 89 ef 48 89 44 24 08 e8 70 5f 02 00 48 8b >>> >> > RSP: 002b:00007ffdd2e3b670 EFLAGS: 00000293 ORIG_RAX: 000000000000002c >>> >> > RAX: ffffffffffffffda RBX: 00007faadf514620 RCX: 00007faade790d5c >>> >> > RDX: 0000000000000028 RSI: 00007faadf514670 RDI: 0000000000000003 >>> >> > RBP: 0000000000000000 R08: 00007ffdd2e3b6c4 R09: 000000000000000c >>> >> > R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000003 >>> >> > R13: 0000000000000000 R14: 00007faadf514670 R15: 0000000000000000 >>> >> > </TASK> >>> >> > rcu: rcu_preempt kthread timer wakeup didn't happen for 10499 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 >>> >> > rcu: Possible timer handling issue on cpu=0 timer-softirq=4286 >>> >> > rcu: rcu_preempt kthread starved for 10500 jiffies! g11321 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 >>> >> > rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. >>> >> > rcu: RCU grace-period kthread stack dump: >>> >> > task:rcu_preempt state:I stack:27224 pid:16 tgid:16 ppid:2 task_flags:0x208040 flags:0x00080000 >>> >> > Call Trace: >>> >> > <TASK> >>> >> > context_switch kernel/sched/core.c:5325 [inline] >>> >> > __schedule+0x1798/0x4cc0 kernel/sched/core.c:6929 >>> >> > __schedule_loop kernel/sched/core.c:7011 [inline] >>> >> > schedule+0x165/0x360 kernel/sched/core.c:7026 >>> >> > schedule_timeout+0x12b/0x270 kernel/time/sleep_timeout.c:99 >>> >> > rcu_gp_fqs_loop+0x301/0x1540 kernel/rcu/tree.c:2083 >>> >> > rcu_gp_kthread+0x99/0x390 kernel/rcu/tree.c:2285 >>> >> > kthread+0x711/0x8a0 kernel/kthread.c:463 >>> >> > ret_from_fork+0x4bc/0x870 arch/x86/kernel/process.c:158 >>> >> > ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 >>> >> > </TASK> >>> >> > >>> >> > >>> >> > --- >>> >> > If you want syzbot to run the reproducer, reply with: >>> >> > #syz test: git://repo/address.git branch-or-commit-hash >>> >> > If you attach or paste a git patch, syzbot will apply it before testing. >>> >> >>> >> Yet another taprio report. >>> >> >>> >> If taprio can not be fixed, perhaps we should remove it from the >>> >> kernel, or clearly marked as broken. >>> >> (Then ask syzbot to no longer include it) >>> > >>> > Agreed on the challenge with taprio. >>> > We need the stakeholders input: Vinicius - are you still working in >>> > this space? Vladimir you also seem to have interest (or maybe nxp >>> > does) in this? >>> >>> No, I am not working on this space anymore. >>> >>> I will talk with other Intel folks (and my manager) and see what we can >>> do. >> >> I assume your customers are still interested in this working? If yes, >> that would be a good pitch to the manager. > > I did talk with some people here, and let's just say that I am hearing > positive noises. So chances are that I should be able to dedicate some > of my "job time" to this area again. > Just a quick update, I got to spend some time on this, basically trying to implement the idea here (that I had completely forgotten about): https://lore.kernel.org/all/87jzftpwo2.fsf@intel.com/ Very early code, just to see how the code would look like is here: https://github.com/vcgomes/net-next/tree/taprio-fix-syzkaller-report-clock-adjust In case anyone is interested. >> In my (extreme) view, another selling point is that there is an >> ethical obligation to ensure things continue to work as intended. >> Getting patches in is the easy part. >> >> cheers, >> jamal >> >>>But if others that find it useful can help even better. >>> >> > > > Cheers, > -- > Vinicius > Cheers, -- Vinicius ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2025-10-24 0:10 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-05-06 14:06 [syzbot] [net?] INFO: rcu detected stall in inet_rtm_newaddr (2) syzbot 2025-10-11 7:41 ` [syzbot] [net?] [mm?] " syzbot 2025-10-11 9:42 ` Eric Dumazet 2025-10-12 15:46 ` Jamal Hadi Salim 2025-10-12 15:52 ` Jamal Hadi Salim 2025-10-14 20:11 ` Vladimir Oltean 2025-10-15 21:50 ` Jamal Hadi Salim 2025-10-13 21:51 ` Vinicius Costa Gomes 2025-10-15 21:36 ` Jamal Hadi Salim 2025-10-15 22:30 ` Vinicius Costa Gomes 2025-10-24 0:11 ` Vinicius Costa Gomes
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).