Re: [Syzkaller & bisect] There is "soft lockup in sys_perf_event_open" BUG in v6.3-rc4 kernel

[Pengfei Xu <pengfei.xu@intel.com>]

Hi Peter Z,

On 2023-03-29 at 11:24:33 +0200, Peter Zijlstra wrote:
> On Wed, Mar 29, 2023 at 04:54:22PM +0800, Pengfei Xu wrote:
> 
> > "
> > 79df45731da68772d2285265864a52c900b8c65f
> > perf/core: Allow ftrace for functions in kernel/event/core.c
> > "
> > After reverted above commit on top of tip tag perf-core-2021-10-31, this issue
> > was gone.
> 
> Glorious recursion... Song, I'm tempted to indeed revert that patch
> again.
> 
Thanks Peter Z!
This bisect script is a different toolset than syzkaller's bisect.
And I'm glad it's helpful.

Thanks!
BR.
-Pengfei(Intel)

> > [   24.618533] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=330 'systemd'
> > [   56.504281] watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [repro:516]
> > [   56.507013] Modules linked in:
> > [   56.507699] irq event stamp: 684720
> > [   56.508318] hardirqs last  enabled at (684719): [<ffffffff8107247c>] __text_poke+0x2ec/0x4e0
> > [   56.509577] hardirqs last disabled at (684720): [<ffffffff82f937a3>] sysvec_apic_timer_interrupt+0x13/0xe0
> > [   56.510993] softirqs last  enabled at (472178): [<ffffffff82fb71a9>] __do_softirq+0x2d9/0x3c3
> > [   56.512283] softirqs last disabled at (472151): [<ffffffff811266f4>] irq_exit_rcu+0xc4/0x100
> > [   56.513543] CPU: 0 PID: 516 Comm: repro Not tainted 6.3.0-rc4-197b6b60ae7b+ #1
> > [   56.514647] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org 04/01/2014
> > [   56.516213] RIP: 0010:debug_lockdep_rcu_enabled+0x0/0x40
> > [   56.517098] Code: c7 c7 a3 d6 96 83 e8 4f 0e 00 00 65 c7 05 e4 c3 08 7d 00 00 00 00 eb ba 66 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 <f3> 0f 1e fa 8b 05 be 65 0d 01 85 c0 74 2b 8b 05 4c cd 0d 01 85 c0
> > [   56.519522] RSP: 0018:ffffc90000fbb260 EFLAGS: 00000246
> > [   56.520439] RAX: 0000000000000000 RBX: ffff888014599aa8 RCX: ffffffff81353c4d
> > [   56.521513] RDX: 0000000000000000 RSI: ffff888013bc4680 RDI: 0000000000000002
> > [   56.522585] RBP: ffffc90000fbb2a8 R08: ffffc90000fbb820 R09: ffffc90000fbb818
> > [   56.523683] R10: ffffc90000fbb7f0 R11: 0000000000000000 R12: ffffc90000fbb2b8
> > [   56.524766] R13: fffffffffffffdff R14: 0000000000000000 R15: ffffffff81353960
> > [   56.525841] FS:  00007fe17f501740(0000) GS:ffff88807dc00000(0000) knlGS:0000000000000000
> > [   56.527044] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [   56.527977] CR2: 00007ffd57376000 CR3: 000000000af0a003 CR4: 0000000000770ef0
> > [   56.529038] PKRU: 55555554
> > [   56.529543] Call Trace:
> > [   56.530040]  <TASK>
> > [   56.530496]  ? arch_ftrace_ops_list_func+0x1b6/0x360
> > [   56.531339]  ? perf_trace_buf_alloc+0x41/0x110
> > [   56.532148]  ? perf_swevent_get_recursion_context+0x4/0xe0
> > [   56.533132]  ? perf_tp_event+0x164/0x880
> > [   56.533853]  ftrace_call+0x5/0x44
> > [   56.534612]  ? write_comp_data+0x2f/0x90
> > [   56.535444]  ? perf_trace_buf_alloc+0x2a/0x110
> > [   56.536301]  ? perf_swevent_get_recursion_context+0x9/0xe0
> > [   56.537201]  ? write_comp_data+0x2f/0x90
> > [   56.537958]  perf_swevent_get_recursion_context+0x9/0xe0
> > [   56.538848]  perf_trace_buf_alloc+0x41/0x110
> > [   56.539644]  ? perf_swevent_get_recursion_context+0x9/0xe0
> > [   56.540564]  ? perf_trace_buf_alloc+0x41/0x110
> > [   56.541415]  ? __pfx_perf_swevent_event+0x10/0x10
> > [   56.542233]  perf_ftrace_function_call+0x28f/0x340
> > [   56.543070]  ? __sanitizer_cov_trace_pc+0x25/0x60
> > [   56.544273]  ? arch_ftrace_ops_list_func+0x2c6/0x360
> > [   56.545328]  ? __pfx_perf_ftrace_function_call+0x10/0x10
> > [   56.546221]  arch_ftrace_ops_list_func+0x2c6/0x360
> > [   56.547029]  ? perf_tp_event+0x164/0x880
> > [   56.547774]  ? __pfx_perf_swevent_event+0x10/0x10
> > [   56.548730]  ftrace_call+0x5/0x44
> > [   56.549407]  ? __sanitizer_cov_trace_pc+0x25/0x60
> > [   56.550239]  ? write_comp_data+0x2f/0x90
> > [   56.551228]  ? perf_swevent_event+0x5/0x170
> > [   56.552001]  ? write_comp_data+0x2f/0x90
> > [   56.552767]  perf_swevent_event+0x5/0x170
> > [   56.553500]  perf_tp_event+0x164/0x880
> > [   56.554219]  ? perf_swevent_event+0x5/0x170
> > [   56.554959]  ? perf_tp_event+0x164/0x880
> > [   56.555959]  ? arch_ftrace_ops_list_func+0x2c6/0x360
> > [   56.556843]  ? write_comp_data+0x2f/0x90
> > [   56.557646]  ? __sanitizer_cov_trace_pc+0x25/0x60
> > [   56.558541]  ? arch_ftrace_ops_list_func+0x207/0x360
> > [   56.559393]  ? perf_ftrace_function_call+0x2d2/0x340
> > [   56.560256]  ? perf_tp_event+0x4/0x880
> > [   56.561026]  ? _raw_spin_lock+0x4/0x50
> > [   56.561723]  ? __get_locked_pte+0x96/0xe0
> > [   56.562493]  ? ftrace_call+0x5/0x44
> > [   56.563335]  ? write_comp_data+0x2f/0x90
> > [   56.564308]  ? _raw_spin_lock+0x4/0x50
> > [   56.564988]  ? __get_locked_pte+0x96/0xe0
> > [   56.565708]  perf_ftrace_function_call+0x2d2/0x340
> > [   56.566516]  ? perf_tp_event+0x9/0x880
> > [   56.567223]  ? perf_ftrace_function_call+0x2d2/0x340
> > [   56.568079]  ? perf_tp_event+0x9/0x880
> > [   56.568741]  ? perf_ftrace_function_call+0x2d2/0x340
> > [   56.570049]  ? 0xffffffffa02010b1
> > [   56.570919]  0xffffffffa02010b1
> > [   56.571678]  ? ftrace_call+0x5/0x44
> > [   56.572420]  ? pmd_page_vaddr+0x2b/0x80
> > [   56.573193]  ? _raw_spin_lock+0x9/0x50
> > [   56.573987]  _raw_spin_lock+0x9/0x50
> > [   56.574672]  __get_locked_pte+0x96/0xe0
> > [   56.575411]  ? _raw_spin_lock+0x9/0x50
> > [   56.576100]  ? __get_locked_pte+0x96/0xe0
> > [   56.576924]  ? __pfx__regmap_raw_write_impl+0x10/0x10
> > [   56.577816]  __text_poke+0xf4/0x4e0
> > [   56.578489]  ? __pfx__regmap_raw_write_impl+0x10/0x10
> > [   56.579399]  ? __pfx_text_poke_memcpy+0x10/0x10
> > [   56.580134]  ? lock_is_held_type+0xe6/0x140
> > [   56.580902]  ? __pfx__regmap_raw_write_impl+0x10/0x10
> > [   56.581822]  text_poke+0x3a/0x60
> > [   56.582538]  text_poke_bp_batch+0x94/0x310
> > [   56.583263]  ? text_poke_bp_batch+0x5/0x310
> > [   56.584081]  ? __pfx_virtblk_update_cache_mode+0x10/0x10
> > [   56.585069]  text_poke_queue+0x93/0xb0
> > [   56.585874]  ftrace_replace_code+0x12a/0x1b0
> > [   56.586789]  ftrace_modify_all_code+0x1b9/0x2a0
> > [   56.587701]  arch_ftrace_update_code+0xd/0x20
> > [   56.588468]  ftrace_startup_enable+0x67/0xa0
> > [   56.589291]  ftrace_startup+0x124/0x200
> > [   56.590082]  register_ftrace_function_nolock+0x43/0x90
> > [   56.590991]  register_ftrace_function+0x1eb/0x280
> > [   56.591873]  ? __sanitizer_cov_trace_switch+0x57/0xa0
> > [   56.592835]  perf_ftrace_event_register+0xcd/0xf0
> > [   56.593735]  perf_trace_event_init+0x98/0x4b0
> > [   56.594639]  perf_trace_init+0xde/0x170
> > [   56.595449]  perf_tp_event_init+0x60/0xa0
> > [   56.596216]  perf_try_init_event+0x88/0x280
> > [   56.597094]  perf_event_alloc+0xe25/0x1c00
> > [   56.597900]  ? perf_event_alloc+0x5/0x1c00
> > [   56.598846]  __do_sys_perf_event_open+0x3b6/0x1910
> > [   56.600490]  __x64_sys_perf_event_open+0x2f/0x40
> > [   56.601330]  do_syscall_64+0x3b/0x90
> > [   56.602067]  entry_SYSCALL_64_after_hwframe+0x72/0xdc
> > [   56.602902] RIP: 0033:0x7fe17f62659d
> > [   56.603610] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d3 08 0d 00 f7 d8 64 89 01 48
> > [   56.606099] RSP: 002b:00007ffd572b7f58 EFLAGS: 00000206 ORIG_RAX: 000000000000012a
> > [   56.607348] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe17f62659d
> > [   56.608429] RDX: 0000000000000000 RSI: 00000000ffffffff RDI: 0000000020000100
> > [   56.609512] RBP: 00007ffd572b7f70 R08: 0000000000000000 R09: 00007ffd572b7f70
> > [   56.610604] R10: 00000000ffffffff R11: 0000000000000206 R12: 0000000000401180
> > [   56.611719] R13: 00007ffd572b8090 R14: 0000000000000000 R15: 0000000000000000
> > [   56.613326]  </TASK>