Re: [PATCH v4 19/23] kprobes: Remove kretprobe hash

Masami Hiramatsu
2020-08-28T18:37:34Z

On Fri, 28 Aug 2020 21:30:06 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> From: Peter Zijlstra <peterz@infradead.org>
>
> The kretprobe hash is mostly superfluous, replace it with a per-task
> variable.
>
> This gets rid of the task hash and it's related locking.
>
> The whole invalidate_rp_inst() is tedious and could go away once we
> drop rp specific ri size.

OK, something wrong with this patch. Now I can reproduce it always,
it takes around 330 second from setting up the kretprobe event on
schedule(). Maybe some timer or timeout value affect?

Anyway, before this patch, I can not reproduce the bug yet. After
applying this patch, I hit the bug always. So something wrong with this.

----
cd /sys/kernel/debug/tracing/

echo r:schedule schedule >> kprobe_events
echo 1 > events/kprobes/enable

sleep 333
----

And show this.

/sys/kernel/debug/tracing # [ 336.718043] ------------[ cut here ]------------
[ 336.719041] kernel BUG at kernel/kprobes.c:1950!
[ 336.720212] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 336.721206] CPU: 0 PID: 85 Comm: kworker/0:2 Not tainted 5.9.0-rc2+ #58
[ 336.722557] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1 04/01/2014
[ 336.724310] Workqueue: 0x0 (events)
[ 336.726997] RIP: 0010:__kretprobe_trampoline_handler+0xe5/0xf0
[ 336.728084] Code: e8 20 04 79 00 65 48 c7 05 ec 4a ec 7e 00 cd 28 82 4c 89 e7 e8 ac fe ff ff 48 85 db 75 97 5b 4c 89 e8 41 5c 41 5d 41 5e 5d c3 <0f> 0b 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 54 49 89 fc 53 48
[ 336.731574] RSP: 0018:ffffc90000307dc8 EFLAGS: 00010246
[ 336.732608] RAX: ffff88807d600000 RBX: 0000000000000000 RCX: 0000000000000000
[ 336.733896] RDX: ffffc90000307ea8 RSI: ffffffff810471e0 RDI: ffff88807c579700
[ 336.735205] RBP: ffffc90000307de8 R08: 0000000000000001 R09: 0000000000000001
[ 336.736506] R10: 0000000000000000 R11: 0000000000000001 R12: ffffc90000307e10
[ 336.737820] R13: ffff88807d62a440 R14: ffffc90000307e10 R15: ffff88807cfb5300
[ 336.739125] FS: 0000000000000000(0000) GS:ffff88807d600000(0000) knlGS:0000000000000000
[ 336.740705] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 336.741773] CR2: 000000000212cd38 CR3: 000000007a732000 CR4: 00000000000006b0
[ 336.742953] Call Trace:
[ 336.743523] trampoline_handler+0x43/0x60
[ 336.744317] kretprobe_trampoline+0x2a/0x50
[ 336.745098] RIP: 0010:kretprobe_trampoline+0x0/0x50
[ 336.745961] Code: c7 39 2e 04 82 e8 a0 f2 0d 00 5d c3 31 f6 e9 79 ff ff ff be 01 00 00 00 e9 6f ff ff ff cc cc cc cc cc cc cc cc cc cc cc cc cc <54> 9c 48 83 ec 18 57 56 52 51 50 41 50 41 51 41 52 41 53 53 55 41
[ 336.749100] RSP: 7c579700:ffffc90000307eb0 EFLAGS: 00000246
[ 336.750077] RAX: ffff88807cfb5300 RBX: ffff88807d62a440 RCX: 0000000000000000
[ 336.751233] RDX: 0000000000000000 RSI: ffffffff818e5334 RDI: ffff88807c579700
[ 336.752391] RBP: ffffc90000307f00 R08: 0000000000000001 R09: 0000000000000001
[ 336.753563] R10: 0000000000000000 R11: 0000000000000001 R12: ffff88807cfb5328
[ 336.754718] R13: ffff88807d62a440 R14: ffffc90000033d58 R15: ffff88807cfb5300
[ 336.756005] ? schedule+0x54/0x100
[ 336.756684] ? process_one_work+0x5c0/0x5c0
[ 336.757504] kthread+0x13c/0x180
[ 336.758102] ? kthread_park+0x90/0x90
[ 336.758931] ret_from_fork+0x22/0x30
[ 336.759719] Modules linked in:
[ 336.760456] ---[ end trace a7f6025840267136 ]---
[ 336.761448] RIP: 0010:__kretprobe_trampoline_handler+0xe5/0xf0
[ 336.762767] Code: e8 20 04 79 00 65 48 c7 05 ec 4a ec 7e 00 cd 28 82 4c 89 e7 e8 ac fe ff ff 48 85 db 75 97 5b 4c 89 e8 41 5c 41 5d 41 5e 5d c3 <0f> 0b 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 54 49 89 fc 53 48
[ 336.766128] RSP: 0018:ffffc90000307dc8 EFLAGS: 00010246
[ 336.767112] RAX: ffff88807d600000 RBX: 0000000000000000 RCX: 0000000000000000
[ 336.768321] RDX: ffffc90000307ea8 RSI: ffffffff810471e0 RDI: ffff88807c579700
[ 336.769539] RBP: ffffc90000307de8 R08: 0000000000000001 R09: 0000000000000001
[ 336.770744] R10: 0000000000000000 R11: 0000000000000001 R12: ffffc90000307e10
[ 336.771947] R13: ffff88807d62a440 R14: ffffc90000307e10 R15: ffff88807cfb5300
[ 336.773201] FS: 0000000000000000(0000) GS:ffff88807d600000(0000) knlGS:0000000000000000
[ 336.774706] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 336.775577] CR2: 000000000212cd38 CR3: 000000007a732000 CR4: 00000000000006b0
[ 336.776699] Kernel panic - not syncing: Fatal exception
[ 336.777829] Kernel Offset: disabled
[ 336.778526] ---[ end Kernel panic - not syncing: Fatal exception ]---

Thank you,

--
Masami Hiramatsu <mhiramat@kernel.org>