From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, Kuniyuki Iwashima <kuniyu@amazon.com>,
Ayushman Dutta <ayudutta@amazon.com>,
"Naveen N. Rao" <naveen.n.rao@linux.ibm.com>,
Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>,
"David S. Miller" <davem@davemloft.net>,
Masami Hiramatsu <mhiramat@kernel.org>,
Wang Nan <wangnan0@huawei.com>,
Kuniyuki Iwashima <kuni1840@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>
Subject: [PATCH 5.15 69/73] kprobes: dont call disarm_kprobe() for disabled kprobes
Date: Fri, 2 Sep 2022 14:19:33 +0200 [thread overview]
Message-ID: <20220902121406.673888685@linuxfoundation.org> (raw)
In-Reply-To: <20220902121404.435662285@linuxfoundation.org>
From: Kuniyuki Iwashima <kuniyu@amazon.com>
commit 9c80e79906b4ca440d09e7f116609262bb747909 upstream.
The assumption in __disable_kprobe() is wrong, and it could try to disarm
an already disarmed kprobe and fire the WARN_ONCE() below. [0] We can
easily reproduce this issue.
1. Write 0 to /sys/kernel/debug/kprobes/enabled.
# echo 0 > /sys/kernel/debug/kprobes/enabled
2. Run execsnoop. At this time, one kprobe is disabled.
# /usr/share/bcc/tools/execsnoop &
[1] 2460
PCOMM PID PPID RET ARGS
# cat /sys/kernel/debug/kprobes/list
ffffffff91345650 r __x64_sys_execve+0x0 [FTRACE]
ffffffff91345650 k __x64_sys_execve+0x0 [DISABLED][FTRACE]
3. Write 1 to /sys/kernel/debug/kprobes/enabled, which changes
kprobes_all_disarmed to false but does not arm the disabled kprobe.
# echo 1 > /sys/kernel/debug/kprobes/enabled
# cat /sys/kernel/debug/kprobes/list
ffffffff91345650 r __x64_sys_execve+0x0 [FTRACE]
ffffffff91345650 k __x64_sys_execve+0x0 [DISABLED][FTRACE]
4. Kill execsnoop, when __disable_kprobe() calls disarm_kprobe() for the
disabled kprobe and hits the WARN_ONCE() in __disarm_kprobe_ftrace().
# fg
/usr/share/bcc/tools/execsnoop
^C
Actually, WARN_ONCE() is fired twice, and __unregister_kprobe_top() misses
some cleanups and leaves the aggregated kprobe in the hash table. Then,
__unregister_trace_kprobe() initialises tk->rp.kp.list and creates an
infinite loop like this.
aggregated kprobe.list -> kprobe.list -.
^ |
'.__.'
In this situation, these commands fall into the infinite loop and result
in RCU stall or soft lockup.
cat /sys/kernel/debug/kprobes/list : show_kprobe_addr() enters into the
infinite loop with RCU.
/usr/share/bcc/tools/execsnoop : warn_kprobe_rereg() holds kprobe_mutex,
and __get_valid_kprobe() is stuck in
the loop.
To avoid the issue, make sure we don't call disarm_kprobe() for disabled
kprobes.
[0]
Failed to disarm kprobe-ftrace at __x64_sys_execve+0x0/0x40 (error -2)
WARNING: CPU: 6 PID: 2460 at kernel/kprobes.c:1130 __disarm_kprobe_ftrace.isra.19 (kernel/kprobes.c:1129)
Modules linked in: ena
CPU: 6 PID: 2460 Comm: execsnoop Not tainted 5.19.0+ #28
Hardware name: Amazon EC2 c5.2xlarge/, BIOS 1.0 10/16/2017
RIP: 0010:__disarm_kprobe_ftrace.isra.19 (kernel/kprobes.c:1129)
Code: 24 8b 02 eb c1 80 3d c4 83 f2 01 00 75 d4 48 8b 75 00 89 c2 48 c7 c7 90 fa 0f 92 89 04 24 c6 05 ab 83 01 e8 e4 94 f0 ff <0f> 0b 8b 04 24 eb b1 89 c6 48 c7 c7 60 fa 0f 92 89 04 24 e8 cc 94
RSP: 0018:ffff9e6ec154bd98 EFLAGS: 00010282
RAX: 0000000000000000 RBX: ffffffff930f7b00 RCX: 0000000000000001
RDX: 0000000080000001 RSI: ffffffff921461c5 RDI: 00000000ffffffff
RBP: ffff89c504286da8 R08: 0000000000000000 R09: c0000000fffeffff
R10: 0000000000000000 R11: ffff9e6ec154bc28 R12: ffff89c502394e40
R13: ffff89c502394c00 R14: ffff9e6ec154bc00 R15: 0000000000000000
FS: 00007fe800398740(0000) GS:ffff89c812d80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c00057f010 CR3: 0000000103b54006 CR4: 00000000007706e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
<TASK>
__disable_kprobe (kernel/kprobes.c:1716)
disable_kprobe (kernel/kprobes.c:2392)
__disable_trace_kprobe (kernel/trace/trace_kprobe.c:340)
disable_trace_kprobe (kernel/trace/trace_kprobe.c:429)
perf_trace_event_unreg.isra.2 (./include/linux/tracepoint.h:93 kernel/trace/trace_event_perf.c:168)
perf_kprobe_destroy (kernel/trace/trace_event_perf.c:295)
_free_event (kernel/events/core.c:4971)
perf_event_release_kernel (kernel/events/core.c:5176)
perf_release (kernel/events/core.c:5186)
__fput (fs/file_table.c:321)
task_work_run (./include/linux/sched.h:2056 (discriminator 1) kernel/task_work.c:179 (discriminator 1))
exit_to_user_mode_prepare (./include/linux/resume_user_mode.h:49 kernel/entry/common.c:169 kernel/entry/common.c:201)
syscall_exit_to_user_mode (./arch/x86/include/asm/jump_label.h:55 ./arch/x86/include/asm/nospec-branch.h:384 ./arch/x86/include/asm/entry-common.h:94 kernel/entry/common.c:133 kernel/entry/common.c:296)
do_syscall_64 (arch/x86/entry/common.c:87)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120)
RIP: 0033:0x7fe7ff210654
Code: 15 79 89 20 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb be 0f 1f 00 8b 05 9a cd 20 00 48 63 ff 85 c0 75 11 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 3a f3 c3 48 83 ec 18 48 89 7c 24 08 e8 34 fc
RSP: 002b:00007ffdbd1d3538 EFLAGS: 00000246 ORIG_RAX: 0000000000000003
RAX: 0000000000000000 RBX: 0000000000000008 RCX: 00007fe7ff210654
RDX: 0000000000000000 RSI: 0000000000002401 RDI: 0000000000000008
RBP: 0000000000000000 R08: 94ae31d6fda838a4 R0900007fe8001c9d30
R10: 00007ffdbd1d34b0 R11: 0000000000000246 R12: 00007ffdbd1d3600
R13: 0000000000000000 R14: fffffffffffffffc R15: 00007ffdbd1d3560
</TASK>
Link: https://lkml.kernel.org/r/20220813020509.90805-1-kuniyu@amazon.com
Fixes: 69d54b916d83 ("kprobes: makes kprobes/enabled works correctly for optimized kprobes.")
Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
Reported-by: Ayushman Dutta <ayudutta@amazon.com>
Cc: "Naveen N. Rao" <naveen.n.rao@linux.ibm.com>
Cc: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Cc: Kuniyuki Iwashima <kuniyu@amazon.com>
Cc: Kuniyuki Iwashima <kuni1840@gmail.com>
Cc: Ayushman Dutta <ayudutta@amazon.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
kernel/kprobes.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
--- a/kernel/kprobes.c
+++ b/kernel/kprobes.c
@@ -1705,11 +1705,12 @@ static struct kprobe *__disable_kprobe(s
/* Try to disarm and disable this/parent probe */
if (p == orig_p || aggr_kprobe_disabled(orig_p)) {
/*
- * If kprobes_all_disarmed is set, orig_p
- * should have already been disarmed, so
- * skip unneed disarming process.
+ * Don't be lazy here. Even if 'kprobes_all_disarmed'
+ * is false, 'orig_p' might not have been armed yet.
+ * Note arm_all_kprobes() __tries__ to arm all kprobes
+ * on the best effort basis.
*/
- if (!kprobes_all_disarmed) {
+ if (!kprobes_all_disarmed && !kprobe_disabled(orig_p)) {
ret = disarm_kprobe(orig_p, true);
if (ret) {
p->flags &= ~KPROBE_FLAG_DISABLED;
next prev parent reply other threads:[~2022-09-02 12:50 UTC|newest]
Thread overview: 82+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-02 12:18 [PATCH 5.15 00/73] 5.15.65-rc1 review Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 01/73] mm: Force TLB flush for PFNMAP mappings before unlink_file_vma() Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 02/73] drm/bridge: Add stubs for devm_drm_of_get_bridge when OF is disabled Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 03/73] ACPI: thermal: drop an always true check Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 04/73] drm/vc4: hdmi: Rework power up Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 05/73] drm/vc4: hdmi: Depends on CONFIG_PM Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 06/73] firmware: tegra: bpmp: Do only aligned access to IPC memory area Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 07/73] crypto: lib - remove unneeded selection of XOR_BLOCKS Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 08/73] Drivers: hv: balloon: Support status report for larger page sizes Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 09/73] mm/hugetlb: avoid corrupting page->mapping in hugetlb_mcopy_atomic_pte Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 10/73] arm64: errata: Add Cortex-A510 to the repeat tlbi list Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 11/73] io_uring: correct fill events helpers types Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 12/73] io_uring: clean cqe filling functions Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 13/73] io_uring: refactor poll update Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 14/73] io_uring: move common poll bits Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 15/73] io_uring: kill poll linking optimisation Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 16/73] io_uring: inline io_poll_complete Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 17/73] io_uring: poll rework Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 18/73] io_uring: Remove unused function req_ref_put Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 19/73] io_uring: remove poll entry from list when canceling all Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 20/73] io_uring: bump poll refs to full 31-bits Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 21/73] io_uring: fail links when poll fails Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 22/73] io_uring: fix wrong arm_poll error handling Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 23/73] io_uring: fix UAF due to missing POLLFREE handling Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 24/73] kbuild: Fix include path in scripts/Makefile.modpost Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 25/73] Bluetooth: L2CAP: Fix build errors in some archs Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 26/73] Revert "PCI/portdrv: Dont disable AER reporting in get_port_device_capability()" Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 27/73] HID: steam: Prevent NULL pointer dereference in steam_{recv,send}_report Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 28/73] udmabuf: Set the DMA mask for the udmabuf device (v2) Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 29/73] media: pvrusb2: fix memory leak in pvr_probe Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 30/73] HID: hidraw: fix memory leak in hidraw_release() Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 31/73] net: fix refcount bug in sk_psock_get (2) Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 32/73] fbdev: fb_pm2fb: Avoid potential divide by zero error Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 33/73] ftrace: Fix NULL pointer dereference in is_ftrace_trampoline when ftrace is dead Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 34/73] bpf: Dont redirect packets with invalid pkt_len Greg Kroah-Hartman
2022-09-02 12:18 ` [PATCH 5.15 35/73] mm/rmap: Fix anon_vma->degree ambiguity leading to double-reuse Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 36/73] ALSA: usb-audio: Add quirk for LH Labs Geek Out HD Audio 1V5 Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 37/73] HID: add Lenovo Yoga C630 battery quirk Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 38/73] HID: AMD_SFH: Add a DMI quirk entry for Chromebooks Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 39/73] HID: asus: ROG NKey: Ignore portion of 0x5a report Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 40/73] HID: thrustmaster: Add sparco wheel and fix array length Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 41/73] drm/i915/gt: Skip TLB invalidations once wedged Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 42/73] mmc: mtk-sd: Clear interrupts when cqe off/disable Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 43/73] mmc: sdhci-of-dwcmshc: add reset call back for rockchip Socs Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 44/73] mmc: sdhci-of-dwcmshc: rename rk3568 to rk35xx Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 45/73] mmc: sdhci-of-dwcmshc: Re-enable support for the BlueField-3 SoC Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 46/73] btrfs: remove root argument from btrfs_unlink_inode() Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 47/73] btrfs: remove no longer needed logic for replaying directory deletes Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 48/73] btrfs: add and use helper for unlinking inode during log replay Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 49/73] btrfs: fix warning during log replay when bumping inode link count Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 50/73] fs/ntfs3: Fix work with fragmented xattr Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 51/73] ASoC: sh: rz-ssi: Improve error handling in rz_ssi_probe() error path Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 52/73] drm/amd/display: Avoid MPC infinite loop Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 53/73] drm/amd/display: Fix HDMI VSIF V3 incorrect issue Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 54/73] drm/amd/display: For stereo keep "FLIP_ANY_FRAME" Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 55/73] drm/amd/display: clear optc underflow before turn off odm clock Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 56/73] ksmbd: return STATUS_BAD_NETWORK_NAME error status if share is not configured Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 57/73] neigh: fix possible DoS due to net iface start/stop loop Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 58/73] s390/hypfs: avoid error message under KVM Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 59/73] ksmbd: dont remove dos attribute xattr on O_TRUNC open Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 60/73] drm/amd/pm: add missing ->fini_microcode interface for Sienna Cichlid Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 61/73] drm/amd/display: Fix pixel clock programming Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 62/73] drm/amdgpu: Increase tlb flush timeout for sriov Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 63/73] drm/amd/display: avoid doing vm_init multiple time Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 64/73] netfilter: conntrack: NF_CONNTRACK_PROCFS should no longer default to y Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 65/73] testing: selftests: nft_flowtable.sh: use random netns names Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 66/73] btrfs: move lockdep class helpers to locking.c Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 67/73] btrfs: fix lockdep splat with reloc root extent buffers Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 68/73] btrfs: tree-checker: check for overlapping extent items Greg Kroah-Hartman
2022-09-02 12:19 ` Greg Kroah-Hartman [this message]
2022-09-02 12:19 ` [PATCH 5.15 70/73] btrfs: fix space cache corruption and potential double allocations Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 71/73] android: binder: fix lockdep check on clearing vma Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 72/73] net/af_packet: check len when min_header_len equals to 0 Greg Kroah-Hartman
2022-09-02 12:19 ` [PATCH 5.15 73/73] net: neigh: dont call kfree_skb() under spin_lock_irqsave() Greg Kroah-Hartman
2022-09-02 16:36 ` [PATCH 5.15 00/73] 5.15.65-rc1 review Jon Hunter
2022-09-02 18:00 ` Florian Fainelli
2022-09-02 22:06 ` Shuah Khan
2022-09-03 0:36 ` Guenter Roeck
2022-09-03 3:33 ` Naresh Kamboju
2022-09-03 4:10 ` Bagas Sanjaya
2022-09-03 7:42 ` Ron Economos
2022-09-03 10:47 ` Sudip Mukherjee
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220902121406.673888685@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=akpm@linux-foundation.org \
--cc=anil.s.keshavamurthy@intel.com \
--cc=ayudutta@amazon.com \
--cc=davem@davemloft.net \
--cc=kuni1840@gmail.com \
--cc=kuniyu@amazon.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mhiramat@kernel.org \
--cc=naveen.n.rao@linux.ibm.com \
--cc=stable@vger.kernel.org \
--cc=wangnan0@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox