[syzbot] [perf?] [trace?] KCSAN: assert: race in srcu_gp_start_if_needed

[syzbot] [perf?] [trace?] KCSAN: assert: race in srcu_gp_start_if_needed

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit:    42d9e8b7ccdd Merge tag 'powerpc-6.13-1' of git://git.kerne..
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=10a00778580000
kernel config:  https://syzkaller.appspot.com/x/.config?x=3d7fd5be0e73b8b
dashboard link: https://syzkaller.appspot.com/bug?extid=16a19b06125a2963eaee
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/ef231513adc7/disk-42d9e8b7.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/54caaac5960b/vmlinux-42d9e8b7.xz
kernel image: https://storage.googleapis.com/syzbot-assets/85b5a6566143/bzImage-42d9e8b7.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+16a19b06125a2963eaee@syzkaller.appspotmail.com

==================================================================
BUG: KCSAN: assert: race in srcu_get_delay kernel/rcu/srcutree.c:658 [inline]
BUG: KCSAN: assert: race in srcu_funnel_gp_start kernel/rcu/srcutree.c:1089 [inline]
BUG: KCSAN: assert: race in srcu_gp_start_if_needed+0x808/0x9f0 kernel/rcu/srcutree.c:1339

race at unknown origin, with assert no writes to 0xffffffff8676ffc8 of 8 bytes by task 6285 on cpu 0:
 srcu_get_delay kernel/rcu/srcutree.c:658 [inline]
 srcu_funnel_gp_start kernel/rcu/srcutree.c:1089 [inline]
 srcu_gp_start_if_needed+0x808/0x9f0 kernel/rcu/srcutree.c:1339
 __call_srcu kernel/rcu/srcutree.c:1384 [inline]
 __synchronize_srcu+0x114/0x150 kernel/rcu/srcutree.c:1432
 synchronize_srcu+0x1ae/0x1d0
 uprobe_unregister_sync+0x1a/0x20 kernel/events/uprobes.c:1328
 __probe_event_disable kernel/trace/trace_uprobe.c:1138 [inline]
 probe_event_disable+0x1f5/0x240 kernel/trace/trace_uprobe.c:1222
 trace_uprobe_register+0xbb/0x3c0 kernel/trace/trace_uprobe.c:1512
 perf_trace_event_unreg+0x6c/0x1a0 kernel/trace/trace_event_perf.c:162
 perf_uprobe_destroy+0x6a/0xf0 kernel/trace/trace_event_perf.c:345
 _free_event+0x7ca/0xa10 kernel/events/core.c:5357
 put_event kernel/events/core.c:5454 [inline]
 perf_event_release_kernel+0x61a/0x670 kernel/events/core.c:5579
 perf_release+0x1f/0x30 kernel/events/core.c:5589
 __fput+0x17a/0x6d0 fs/file_table.c:450
 ____fput+0x1c/0x30 fs/file_table.c:478
 task_work_run+0x13a/0x1a0 kernel/task_work.c:239
 resume_user_mode_work include/linux/resume_user_mode.h:50 [inline]
 exit_to_user_mode_loop kernel/entry/common.c:114 [inline]
 exit_to_user_mode_prepare include/linux/entry-common.h:329 [inline]
 __syscall_exit_to_user_mode_work kernel/entry/common.c:207 [inline]
 syscall_exit_to_user_mode+0xa8/0x120 kernel/entry/common.c:218
 do_syscall_64+0xd6/0x1c0 arch/x86/entry/common.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

value changed: 0x0000000000000000 -> 0x0000000000000001

Reported by Kernel Concurrency Sanitizer on:
CPU: 0 UID: 0 PID: 6285 Comm: syz.1.1056 Not tainted 6.12.0-syzkaller-08756-g42d9e8b7ccdd #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
==================================================================


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

Re: [syzbot] [perf?] [trace?] KCSAN: assert: race in srcu_gp_start_if_needed

[Marco Elver]

+Cc RCU

On Sun, 24 Nov 2024 at 23:47, syzbot
<syzbot+16a19b06125a2963eaee@syzkaller.appspotmail.com> wrote:
>
> Hello,
>
> syzbot found the following issue on:
>
> HEAD commit:    42d9e8b7ccdd Merge tag 'powerpc-6.13-1' of git://git.kerne..
> git tree:       upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=10a00778580000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=3d7fd5be0e73b8b
> dashboard link: https://syzkaller.appspot.com/bug?extid=16a19b06125a2963eaee
> compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
>
> Unfortunately, I don't have any reproducer for this issue yet.
>
> Downloadable assets:
> disk image: https://storage.googleapis.com/syzbot-assets/ef231513adc7/disk-42d9e8b7.raw.xz
> vmlinux: https://storage.googleapis.com/syzbot-assets/54caaac5960b/vmlinux-42d9e8b7.xz
> kernel image: https://storage.googleapis.com/syzbot-assets/85b5a6566143/bzImage-42d9e8b7.xz
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+16a19b06125a2963eaee@syzkaller.appspotmail.com
>
> ==================================================================
> BUG: KCSAN: assert: race in srcu_get_delay kernel/rcu/srcutree.c:658 [inline]
> BUG: KCSAN: assert: race in srcu_funnel_gp_start kernel/rcu/srcutree.c:1089 [inline]
> BUG: KCSAN: assert: race in srcu_gp_start_if_needed+0x808/0x9f0 kernel/rcu/srcutree.c:1339
>
> race at unknown origin, with assert no writes to 0xffffffff8676ffc8 of 8 bytes by task 6285 on cpu 0:
>  srcu_get_delay kernel/rcu/srcutree.c:658 [inline]
>  srcu_funnel_gp_start kernel/rcu/srcutree.c:1089 [inline]
>  srcu_gp_start_if_needed+0x808/0x9f0 kernel/rcu/srcutree.c:1339
>  __call_srcu kernel/rcu/srcutree.c:1384 [inline]
>  __synchronize_srcu+0x114/0x150 kernel/rcu/srcutree.c:1432
>  synchronize_srcu+0x1ae/0x1d0
>  uprobe_unregister_sync+0x1a/0x20 kernel/events/uprobes.c:1328
>  __probe_event_disable kernel/trace/trace_uprobe.c:1138 [inline]
>  probe_event_disable+0x1f5/0x240 kernel/trace/trace_uprobe.c:1222
>  trace_uprobe_register+0xbb/0x3c0 kernel/trace/trace_uprobe.c:1512
>  perf_trace_event_unreg+0x6c/0x1a0 kernel/trace/trace_event_perf.c:162
>  perf_uprobe_destroy+0x6a/0xf0 kernel/trace/trace_event_perf.c:345
>  _free_event+0x7ca/0xa10 kernel/events/core.c:5357
>  put_event kernel/events/core.c:5454 [inline]
>  perf_event_release_kernel+0x61a/0x670 kernel/events/core.c:5579
>  perf_release+0x1f/0x30 kernel/events/core.c:5589
>  __fput+0x17a/0x6d0 fs/file_table.c:450
>  ____fput+0x1c/0x30 fs/file_table.c:478
>  task_work_run+0x13a/0x1a0 kernel/task_work.c:239
>  resume_user_mode_work include/linux/resume_user_mode.h:50 [inline]
>  exit_to_user_mode_loop kernel/entry/common.c:114 [inline]
>  exit_to_user_mode_prepare include/linux/entry-common.h:329 [inline]
>  __syscall_exit_to_user_mode_work kernel/entry/common.c:207 [inline]
>  syscall_exit_to_user_mode+0xa8/0x120 kernel/entry/common.c:218
>  do_syscall_64+0xd6/0x1c0 arch/x86/entry/common.c:89
>  entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
> value changed: 0x0000000000000000 -> 0x0000000000000001
>
> Reported by Kernel Concurrency Sanitizer on:
> CPU: 0 UID: 0 PID: 6285 Comm: syz.1.1056 Not tainted 6.12.0-syzkaller-08756-g42d9e8b7ccdd #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
> ==================================================================
>
>
> ---
> This report is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at syzkaller@googlegroups.com.
>
> syzbot will keep track of this issue. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
>
> If the report is already addressed, let syzbot know by replying with:
> #syz fix: exact-commit-title
>
> If you want to overwrite report's subsystems, reply with:
> #syz set subsystems: new-subsystem
> (See the list of subsystem names on the web dashboard)
>
> If the report is a duplicate of another one, reply with:
> #syz dup: exact-subject-of-another-report
>
> If you want to undo deduplication, reply with:
> #syz undup
>
> --
> You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-bugs+unsubscribe@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/syzkaller-bugs/6743acf4.050a0220.1cc393.004d.GAE%40google.com.

Re: [syzbot] [perf?] [trace?] KCSAN: assert: race in srcu_gp_start_if_needed

[Paul E. McKenney]

On Sun, Nov 24, 2024 at 11:48:46PM +0100, Marco Elver wrote:
> +Cc RCU
> 
> On Sun, 24 Nov 2024 at 23:47, syzbot
> <syzbot+16a19b06125a2963eaee@syzkaller.appspotmail.com> wrote:
> >
> > Hello,
> >
> > syzbot found the following issue on:
> >
> > HEAD commit:    42d9e8b7ccdd Merge tag 'powerpc-6.13-1' of git://git.kerne..
> > git tree:       upstream
> > console output: https://syzkaller.appspot.com/x/log.txt?x=10a00778580000
> > kernel config:  https://syzkaller.appspot.com/x/.config?x=3d7fd5be0e73b8b
> > dashboard link: https://syzkaller.appspot.com/bug?extid=16a19b06125a2963eaee
> > compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
> >
> > Unfortunately, I don't have any reproducer for this issue yet.
> >
> > Downloadable assets:
> > disk image: https://storage.googleapis.com/syzbot-assets/ef231513adc7/disk-42d9e8b7.raw.xz
> > vmlinux: https://storage.googleapis.com/syzbot-assets/54caaac5960b/vmlinux-42d9e8b7.xz
> > kernel image: https://storage.googleapis.com/syzbot-assets/85b5a6566143/bzImage-42d9e8b7.xz
> >
> > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > Reported-by: syzbot+16a19b06125a2963eaee@syzkaller.appspotmail.com
> >
> > ==================================================================
> > BUG: KCSAN: assert: race in srcu_get_delay kernel/rcu/srcutree.c:658 [inline]
> > BUG: KCSAN: assert: race in srcu_funnel_gp_start kernel/rcu/srcutree.c:1089 [inline]
> > BUG: KCSAN: assert: race in srcu_gp_start_if_needed+0x808/0x9f0 kernel/rcu/srcutree.c:1339

Hmmm...  All of those are from slow paths, so locking looks to be the
best approach.

A very lightly tested prototype patch is shown below (for which feedback
is most welcome), and thank you all for your testing efforts!

							Thanx, Paul

------------------------------------------------------------------------

commit a955c6a7168f7b204784e4ef7e4db9d017043f73
Author: Paul E. McKenney <paulmck@kernel.org>
Date:   Fri Jan 3 17:04:49 2025 -0800

    srcu: Force synchronization for srcu_get_delay()
    
    Currently, srcu_get_delay() can be called concurrently, for example,
    by a CPU that is the first to request a new grace period and the CPU
    processing the current grace period.  Although concurrent access is
    harmless, it unnecessarily expands the state space.  Additionally,
    all calls to srcu_get_delay() are from slow paths.
    
    This commit therefore protects all calls to srcu_get_delay() with
    ssp->srcu_sup->lock, which is already held on the invocation from the
    srcu_funnel_gp_start() function.  While in the area, this commit also
    adds a lockdep_assert_held() to srcu_get_delay() itself.
    
    Reported-by: syzbot+16a19b06125a2963eaee@syzkaller.appspotmail.com
    Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
index 7c7304dee6457..a60acc9cf2f32 100644
--- a/kernel/rcu/srcutree.c
+++ b/kernel/rcu/srcutree.c
@@ -648,6 +648,7 @@ static unsigned long srcu_get_delay(struct srcu_struct *ssp)
 	unsigned long jbase = SRCU_INTERVAL;
 	struct srcu_usage *sup = ssp->srcu_sup;
 
+	lockdep_assert_held(&ACCESS_PRIVATE(ssp->srcu_sup, lock));
 	if (srcu_gp_is_expedited(ssp))
 		jbase = 0;
 	if (rcu_seq_state(READ_ONCE(sup->srcu_gp_seq))) {
@@ -675,9 +676,13 @@ static unsigned long srcu_get_delay(struct srcu_struct *ssp)
 void cleanup_srcu_struct(struct srcu_struct *ssp)
 {
 	int cpu;
+	unsigned long delay;
 	struct srcu_usage *sup = ssp->srcu_sup;
 
-	if (WARN_ON(!srcu_get_delay(ssp)))
+	spin_lock_irq_rcu_node(ssp->srcu_sup);
+	delay = srcu_get_delay(ssp);
+	spin_unlock_irq_rcu_node(ssp->srcu_sup);
+	if (WARN_ON(!delay))
 		return; /* Just leak it! */
 	if (WARN_ON(srcu_readers_active(ssp)))
 		return; /* Just leak it! */
@@ -1100,7 +1105,9 @@ static bool try_check_zero(struct srcu_struct *ssp, int idx, int trycount)
 {
 	unsigned long curdelay;
 
+	spin_lock_irq_rcu_node(ssp->srcu_sup);
 	curdelay = !srcu_get_delay(ssp);
+	spin_unlock_irq_rcu_node(ssp->srcu_sup);
 
 	for (;;) {
 		if (srcu_readers_active_idx_check(ssp, idx))
@@ -1849,7 +1856,9 @@ static void process_srcu(struct work_struct *work)
 	ssp = sup->srcu_ssp;
 
 	srcu_advance_state(ssp);
+	spin_lock_irq_rcu_node(ssp->srcu_sup);
 	curdelay = srcu_get_delay(ssp);
+	spin_unlock_irq_rcu_node(ssp->srcu_sup);
 	if (curdelay) {
 		WRITE_ONCE(sup->reschedule_count, 0);
 	} else {