* [PATCH v6 0/2] Return EADDRNOTAVAIL when func matches several symbols during kprobe creation
@ 2023-10-20 10:42 Francis Laniel
2023-10-20 10:42 ` [PATCH v6 1/2] tracing/kprobes: Return EADDRNOTAVAIL when func matches several symbols Francis Laniel
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Francis Laniel @ 2023-10-20 10:42 UTC (permalink / raw)
To: linux-trace-kernel; +Cc: Masami Hiramatsu, Francis Laniel
Hi.
In the kernel source code, it exists different functions which share the same
name but which have, of course, different addresses as they can be defined in
different modules:
# Kernel was compiled with CONFIG_NTFS_FS and CONFIG_NTFS3_FS as built-in.
root@vm-amd64:~# grep ntfs_file_write_iter /proc/kallsyms
ffffffff814ce3c0 t __pfx_ntfs_file_write_iter
ffffffff814ce3d0 t ntfs_file_write_iter
ffffffff814fc8a0 t __pfx_ntfs_file_write_iter
ffffffff814fc8b0 t ntfs_file_write_iter
This can be source of troubles when you create a PMU kprobe for such a function,
as it will only install one for the first address (e.g. 0xffffffff814ce3d0 in
the above).
This could lead to some troubles were BPF based tools does not report any event
because the second function is not called:
root@vm-amd64:/mnt# mount | grep /mnt
/foo.img on /mnt type ntfs3 (rw,relatime,uid=0,gid=0,iocharset=utf8)
# ig is a tool which installs a PMU kprobe on ntfs_file_write_iter().
root@vm-amd64:/mnt# ig trace fsslower -m 0 -f ntfs3 --host &> /tmp/foo &
[1] 207
root@vm-amd64:/mnt# dd if=./foo of=./bar count=3
3+0 records in
3+0 records out
1536 bytes (1.5 kB, 1.5 KiB) copied, 0.00543323 s, 283 kB/s
root@vm-amd64:/mnt# fg
ig trace fsslower -m 0 -f ntfs3 --host &> /tmp/foo
^Croot@vm-amd64:/mnt# more /tmp/foo
RUNTIME.CONTAINERNAME RUNTIME.CONTAIN… PID COMM
T BYTES OFFSET LAT FILE
214 dd
R 512 0 766 foo
214 dd
R 512 512 9 foo
214 dd
As you can see in the above, only read events are reported and no write because
the kprobe is installed for the old ntfs_file_write_iter() and not the ntfs3
one.
The same behavior occurs with sysfs kprobe:
root@vm-amd64:/# echo 'p:probe/ntfs_file_write_iter ntfs_file_write_iter' > /sys/kernel/tracing/kprobe_events
root@vm-amd64:/# cat /sys/kernel/tracing/kprobe_events
p:probe/ntfs_file_write_iter ntfs_file_write_iter
root@vm-amd64:/# mount | grep /mnt
/foo.img on /mnt type ntfs3 (rw,relatime,uid=0,gid=0,iocharset=utf8)
root@vm-amd64:/# perf record -e probe:ntfs_file_write_iter &
[1] 210
root@vm-amd64:/# cd /mnt/
root@vm-amd64:/mnt# dd if=./foo of=./bar count=3
3+0 records in
3+0 records out
1536 bytes (1.5 kB, 1.5 KiB) copied, 0.00234793 s, 654 kB/s
root@vm-amd64:/mnt# cd -
/
root@vm-amd64:/# fg
perf record -e probe:ntfs_file_write_iter
^C[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.056 MB perf.data ]
root@vm-amd64:/# perf report
Error:
The perf.data data has no samples!
# To display the perf.data header info, please use --header/--header-only optio>
#
In this contribution, I modified the functions creating sysfs and PMU kprobes to
test if the function name given as argument matches several symbols.
In this case, these functions return EADDRNOTAVAIL to indicate the user to use
addr and offs to remove this ambiguity.
So, when the above BPF tool is run, the following error message is printed:
root@vm-amd64:~# ig trace fsslower -m 0 -f ntfs3 --host &> /tmp/foo &
[1] 228
root@vm-amd64:~# more /tmp/foo
RUNTIME.CONTAINERNAME RUNTIME.CONTAIN… PID COMM
T BYTES OFFSET LAT FILE
Error: running gadget: running gadget: installing tracer: attaching kprobe: crea
ting perf_kprobe PMU (arch-specific fallback for "ntfs_file_write_iter"): token
ntfs_file_write_iter: opening perf event: cannot assign requested address
And the same with sysfs kprobe:
root@vm-amd64:/# echo 'p:probe/ntfs_file_write_iter ntfs_file_write_iter' > /sys/kernel/tracing/kprobe_events
-bash: echo: write error: Cannot assign requested address
Note that, this does not influence perf as it installs kprobes as offset on
_text:
root@vm-amd64:/# perf probe --add ntfs_file_write_iter
Added new events:
probe:ntfs_file_write_iter (on ntfs_file_write_iter)
probe:ntfs_file_write_iter (on ntfs_file_write_iter)
...
root@vm-amd64:/# cat /sys/kernel/tracing/kprobe_events
p:probe/ntfs_file_write_iter _text+5039088
p:probe/ntfs_file_write_iter _text+5228752
Note that, this contribution is the conclusion of a previous RFC which intended
to install a PMU kprobe for all matching symbols [1, 2].
If you see any way to improve this contribution, please share your feedback.
Changes since:
v1:
* Use EADDRNOTAVAIL instead of adding a new error code.
* Correct also this behavior for sysfs kprobe.
v2:
* Count the number of symbols corresponding to function name and return
EADDRNOTAVAIL if higher than 1.
* Return ENOENT if above count is 0, as it would be returned later by while
registering the kprobe.
v3:
* Check symbol does not contain ':' before testing its uniqueness.
* Add a selftest to check this is not possible to install a kprobe for a non
unique symbol.
v5:
* No changes, just add linux-stable as recipient.
v6:
* Rephrase commit message.
* Add "Cc:" to stable.
Francis Laniel (2):
tracing/kprobes: Return EADDRNOTAVAIL when func matches several
symbols
selftests/ftrace: Add new test case which checks non unique symbol
kernel/trace/trace_kprobe.c | 63 +++++++++++++++++++
kernel/trace/trace_probe.h | 1 +
.../test.d/kprobe/kprobe_non_uniq_symbol.tc | 13 ++++
3 files changed, 77 insertions(+)
create mode 100644 tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc
Best regards and thank you in advance.
---
[1]: https://lore.kernel.org/lkml/20230816163517.112518-1-flaniel@linux.microsoft.com/
[2]: https://lore.kernel.org/lkml/20230819101105.b0c104ae4494a7d1f2eea742@kernel.org/
--
2.34.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH v6 1/2] tracing/kprobes: Return EADDRNOTAVAIL when func matches several symbols
2023-10-20 10:42 [PATCH v6 0/2] Return EADDRNOTAVAIL when func matches several symbols during kprobe creation Francis Laniel
@ 2023-10-20 10:42 ` Francis Laniel
2023-10-20 10:42 ` [PATCH v6 2/2] selftests/ftrace: Add new test case which checks non unique symbol Francis Laniel
2023-10-20 12:12 ` [PATCH v6 0/2] Return EADDRNOTAVAIL when func matches several symbols during kprobe creation Masami Hiramatsu
2 siblings, 0 replies; 5+ messages in thread
From: Francis Laniel @ 2023-10-20 10:42 UTC (permalink / raw)
To: linux-trace-kernel
Cc: Masami Hiramatsu, Francis Laniel, stable, Steven Rostedt,
Ananth N Mavinakayanahalli, Frederic Weisbecker, linux-kernel
When a kprobe is attached to a function that's name is not unique (is
static and shares the name with other functions in the kernel), the
kprobe is attached to the first function it finds. This is a bug as the
function that it is attaching to is not necessarily the one that the
user wants to attach to.
Instead of blindly picking a function to attach to what is ambiguous,
error with EADDRNOTAVAIL to let the user know that this function is not
unique, and that the user must use another unique function with an
address offset to get to the function they want to attach to.
Cc: stable@vger.kernel.org
Fixes: 413d37d1eb69 ("tracing: Add kprobe-based event tracer")
Suggested-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Francis Laniel <flaniel@linux.microsoft.com>
Link: https://lore.kernel.org/lkml/20230819101105.b0c104ae4494a7d1f2eea742@kernel.org/
---
kernel/trace/trace_kprobe.c | 63 +++++++++++++++++++++++++++++++++++++
kernel/trace/trace_probe.h | 1 +
2 files changed, 64 insertions(+)
diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
index 3d7a180a8427..a8fef6ab0872 100644
--- a/kernel/trace/trace_kprobe.c
+++ b/kernel/trace/trace_kprobe.c
@@ -705,6 +705,25 @@ static struct notifier_block trace_kprobe_module_nb = {
.priority = 1 /* Invoked after kprobe module callback */
};
+static int count_symbols(void *data, unsigned long unused)
+{
+ unsigned int *count = data;
+
+ (*count)++;
+
+ return 0;
+}
+
+static unsigned int number_of_same_symbols(char *func_name)
+{
+ unsigned int count;
+
+ count = 0;
+ kallsyms_on_each_match_symbol(count_symbols, func_name, &count);
+
+ return count;
+}
+
static int __trace_kprobe_create(int argc, const char *argv[])
{
/*
@@ -836,6 +855,31 @@ static int __trace_kprobe_create(int argc, const char *argv[])
}
}
+ if (symbol && !strchr(symbol, ':')) {
+ unsigned int count;
+
+ count = number_of_same_symbols(symbol);
+ if (count > 1) {
+ /*
+ * Users should use ADDR to remove the ambiguity of
+ * using KSYM only.
+ */
+ trace_probe_log_err(0, NON_UNIQ_SYMBOL);
+ ret = -EADDRNOTAVAIL;
+
+ goto error;
+ } else if (count == 0) {
+ /*
+ * We can return ENOENT earlier than when register the
+ * kprobe.
+ */
+ trace_probe_log_err(0, BAD_PROBE_ADDR);
+ ret = -ENOENT;
+
+ goto error;
+ }
+ }
+
trace_probe_log_set_index(0);
if (event) {
ret = traceprobe_parse_event_name(&event, &group, gbuf,
@@ -1695,6 +1739,7 @@ static int unregister_kprobe_event(struct trace_kprobe *tk)
}
#ifdef CONFIG_PERF_EVENTS
+
/* create a trace_kprobe, but don't add it to global lists */
struct trace_event_call *
create_local_trace_kprobe(char *func, void *addr, unsigned long offs,
@@ -1705,6 +1750,24 @@ create_local_trace_kprobe(char *func, void *addr, unsigned long offs,
int ret;
char *event;
+ if (func) {
+ unsigned int count;
+
+ count = number_of_same_symbols(func);
+ if (count > 1)
+ /*
+ * Users should use addr to remove the ambiguity of
+ * using func only.
+ */
+ return ERR_PTR(-EADDRNOTAVAIL);
+ else if (count == 0)
+ /*
+ * We can return ENOENT earlier than when register the
+ * kprobe.
+ */
+ return ERR_PTR(-ENOENT);
+ }
+
/*
* local trace_kprobes are not added to dyn_event, so they are never
* searched in find_trace_kprobe(). Therefore, there is no concern of
diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h
index 02b432ae7513..850d9ecb6765 100644
--- a/kernel/trace/trace_probe.h
+++ b/kernel/trace/trace_probe.h
@@ -450,6 +450,7 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
C(BAD_MAXACT, "Invalid maxactive number"), \
C(MAXACT_TOO_BIG, "Maxactive is too big"), \
C(BAD_PROBE_ADDR, "Invalid probed address or symbol"), \
+ C(NON_UNIQ_SYMBOL, "The symbol is not unique"), \
C(BAD_RETPROBE, "Retprobe address must be an function entry"), \
C(NO_TRACEPOINT, "Tracepoint is not found"), \
C(BAD_ADDR_SUFFIX, "Invalid probed address suffix"), \
--
2.34.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH v6 2/2] selftests/ftrace: Add new test case which checks non unique symbol
2023-10-20 10:42 [PATCH v6 0/2] Return EADDRNOTAVAIL when func matches several symbols during kprobe creation Francis Laniel
2023-10-20 10:42 ` [PATCH v6 1/2] tracing/kprobes: Return EADDRNOTAVAIL when func matches several symbols Francis Laniel
@ 2023-10-20 10:42 ` Francis Laniel
2023-10-20 12:12 ` [PATCH v6 0/2] Return EADDRNOTAVAIL when func matches several symbols during kprobe creation Masami Hiramatsu
2 siblings, 0 replies; 5+ messages in thread
From: Francis Laniel @ 2023-10-20 10:42 UTC (permalink / raw)
To: linux-trace-kernel
Cc: Masami Hiramatsu, Francis Laniel, stable, Steven Rostedt,
Shuah Khan, linux-kernel, linux-kselftest
If name_show() is non unique, this test will try to install a kprobe on this
function which should fail returning EADDRNOTAVAIL.
On kernel where name_show() is not unique, this test is skipped.
Cc: stable@vger.kernel.org
Signed-off-by: Francis Laniel <flaniel@linux.microsoft.com>
---
.../ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc | 13 +++++++++++++
1 file changed, 13 insertions(+)
create mode 100644 tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc
diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc
new file mode 100644
index 000000000000..bc9514428dba
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc
@@ -0,0 +1,13 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Test failure of registering kprobe on non unique symbol
+# requires: kprobe_events
+
+SYMBOL='name_show'
+
+# We skip this test on kernel where SYMBOL is unique or does not exist.
+if [ "$(grep -c -E "[[:alnum:]]+ t ${SYMBOL}" /proc/kallsyms)" -le '1' ]; then
+ exit_unsupported
+fi
+
+! echo "p:test_non_unique ${SYMBOL}" > kprobe_events
--
2.34.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH v6 0/2] Return EADDRNOTAVAIL when func matches several symbols during kprobe creation
2023-10-20 10:42 [PATCH v6 0/2] Return EADDRNOTAVAIL when func matches several symbols during kprobe creation Francis Laniel
2023-10-20 10:42 ` [PATCH v6 1/2] tracing/kprobes: Return EADDRNOTAVAIL when func matches several symbols Francis Laniel
2023-10-20 10:42 ` [PATCH v6 2/2] selftests/ftrace: Add new test case which checks non unique symbol Francis Laniel
@ 2023-10-20 12:12 ` Masami Hiramatsu
2023-10-20 14:26 ` Francis Laniel
2 siblings, 1 reply; 5+ messages in thread
From: Masami Hiramatsu @ 2023-10-20 12:12 UTC (permalink / raw)
To: Francis Laniel; +Cc: linux-trace-kernel, Masami Hiramatsu
Hi,
Thanks for update the series. The series looks good to me.
Let me pick those in probes/fixes.
Thank you!
On Fri, 20 Oct 2023 13:42:48 +0300
Francis Laniel <flaniel@linux.microsoft.com> wrote:
> Hi.
>
>
> In the kernel source code, it exists different functions which share the same
> name but which have, of course, different addresses as they can be defined in
> different modules:
> # Kernel was compiled with CONFIG_NTFS_FS and CONFIG_NTFS3_FS as built-in.
> root@vm-amd64:~# grep ntfs_file_write_iter /proc/kallsyms
> ffffffff814ce3c0 t __pfx_ntfs_file_write_iter
> ffffffff814ce3d0 t ntfs_file_write_iter
> ffffffff814fc8a0 t __pfx_ntfs_file_write_iter
> ffffffff814fc8b0 t ntfs_file_write_iter
> This can be source of troubles when you create a PMU kprobe for such a function,
> as it will only install one for the first address (e.g. 0xffffffff814ce3d0 in
> the above).
> This could lead to some troubles were BPF based tools does not report any event
> because the second function is not called:
> root@vm-amd64:/mnt# mount | grep /mnt
> /foo.img on /mnt type ntfs3 (rw,relatime,uid=0,gid=0,iocharset=utf8)
> # ig is a tool which installs a PMU kprobe on ntfs_file_write_iter().
> root@vm-amd64:/mnt# ig trace fsslower -m 0 -f ntfs3 --host &> /tmp/foo &
> [1] 207
> root@vm-amd64:/mnt# dd if=./foo of=./bar count=3
> 3+0 records in
> 3+0 records out
> 1536 bytes (1.5 kB, 1.5 KiB) copied, 0.00543323 s, 283 kB/s
> root@vm-amd64:/mnt# fg
> ig trace fsslower -m 0 -f ntfs3 --host &> /tmp/foo
> ^Croot@vm-amd64:/mnt# more /tmp/foo
> RUNTIME.CONTAINERNAME RUNTIME.CONTAIN… PID COMM
> T BYTES OFFSET LAT FILE
> 214 dd
> R 512 0 766 foo
> 214 dd
> R 512 512 9 foo
> 214 dd
> As you can see in the above, only read events are reported and no write because
> the kprobe is installed for the old ntfs_file_write_iter() and not the ntfs3
> one.
> The same behavior occurs with sysfs kprobe:
> root@vm-amd64:/# echo 'p:probe/ntfs_file_write_iter ntfs_file_write_iter' > /sys/kernel/tracing/kprobe_events
> root@vm-amd64:/# cat /sys/kernel/tracing/kprobe_events
> p:probe/ntfs_file_write_iter ntfs_file_write_iter
> root@vm-amd64:/# mount | grep /mnt
> /foo.img on /mnt type ntfs3 (rw,relatime,uid=0,gid=0,iocharset=utf8)
> root@vm-amd64:/# perf record -e probe:ntfs_file_write_iter &
> [1] 210
> root@vm-amd64:/# cd /mnt/
> root@vm-amd64:/mnt# dd if=./foo of=./bar count=3
> 3+0 records in
> 3+0 records out
> 1536 bytes (1.5 kB, 1.5 KiB) copied, 0.00234793 s, 654 kB/s
> root@vm-amd64:/mnt# cd -
> /
> root@vm-amd64:/# fg
> perf record -e probe:ntfs_file_write_iter
> ^C[ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 0.056 MB perf.data ]
>
> root@vm-amd64:/# perf report
> Error:
> The perf.data data has no samples!
> # To display the perf.data header info, please use --header/--header-only optio>
> #
>
> In this contribution, I modified the functions creating sysfs and PMU kprobes to
> test if the function name given as argument matches several symbols.
> In this case, these functions return EADDRNOTAVAIL to indicate the user to use
> addr and offs to remove this ambiguity.
> So, when the above BPF tool is run, the following error message is printed:
> root@vm-amd64:~# ig trace fsslower -m 0 -f ntfs3 --host &> /tmp/foo &
> [1] 228
> root@vm-amd64:~# more /tmp/foo
> RUNTIME.CONTAINERNAME RUNTIME.CONTAIN… PID COMM
> T BYTES OFFSET LAT FILE
> Error: running gadget: running gadget: installing tracer: attaching kprobe: crea
> ting perf_kprobe PMU (arch-specific fallback for "ntfs_file_write_iter"): token
> ntfs_file_write_iter: opening perf event: cannot assign requested address
> And the same with sysfs kprobe:
> root@vm-amd64:/# echo 'p:probe/ntfs_file_write_iter ntfs_file_write_iter' > /sys/kernel/tracing/kprobe_events
> -bash: echo: write error: Cannot assign requested address
> Note that, this does not influence perf as it installs kprobes as offset on
> _text:
> root@vm-amd64:/# perf probe --add ntfs_file_write_iter
> Added new events:
> probe:ntfs_file_write_iter (on ntfs_file_write_iter)
> probe:ntfs_file_write_iter (on ntfs_file_write_iter)
> ...
> root@vm-amd64:/# cat /sys/kernel/tracing/kprobe_events
> p:probe/ntfs_file_write_iter _text+5039088
> p:probe/ntfs_file_write_iter _text+5228752
>
> Note that, this contribution is the conclusion of a previous RFC which intended
> to install a PMU kprobe for all matching symbols [1, 2].
>
> If you see any way to improve this contribution, please share your feedback.
>
> Changes since:
> v1:
> * Use EADDRNOTAVAIL instead of adding a new error code.
> * Correct also this behavior for sysfs kprobe.
> v2:
> * Count the number of symbols corresponding to function name and return
> EADDRNOTAVAIL if higher than 1.
> * Return ENOENT if above count is 0, as it would be returned later by while
> registering the kprobe.
> v3:
> * Check symbol does not contain ':' before testing its uniqueness.
> * Add a selftest to check this is not possible to install a kprobe for a non
> unique symbol.
> v5:
> * No changes, just add linux-stable as recipient.
> v6:
> * Rephrase commit message.
> * Add "Cc:" to stable.
>
> Francis Laniel (2):
> tracing/kprobes: Return EADDRNOTAVAIL when func matches several
> symbols
> selftests/ftrace: Add new test case which checks non unique symbol
>
> kernel/trace/trace_kprobe.c | 63 +++++++++++++++++++
> kernel/trace/trace_probe.h | 1 +
> .../test.d/kprobe/kprobe_non_uniq_symbol.tc | 13 ++++
> 3 files changed, 77 insertions(+)
> create mode 100644 tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc
>
> Best regards and thank you in advance.
> ---
> [1]: https://lore.kernel.org/lkml/20230816163517.112518-1-flaniel@linux.microsoft.com/
> [2]: https://lore.kernel.org/lkml/20230819101105.b0c104ae4494a7d1f2eea742@kernel.org/
> --
> 2.34.1
>
--
Masami Hiramatsu (Google) <mhiramat@kernel.org>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v6 0/2] Return EADDRNOTAVAIL when func matches several symbols during kprobe creation
2023-10-20 12:12 ` [PATCH v6 0/2] Return EADDRNOTAVAIL when func matches several symbols during kprobe creation Masami Hiramatsu
@ 2023-10-20 14:26 ` Francis Laniel
0 siblings, 0 replies; 5+ messages in thread
From: Francis Laniel @ 2023-10-20 14:26 UTC (permalink / raw)
To: Masami Hiramatsu; +Cc: linux-trace-kernel, Masami Hiramatsu
Hi!
Le vendredi 20 octobre 2023, 15:12:39 EEST Masami Hiramatsu a écrit :
> Hi,
>
> Thanks for update the series. The series looks good to me.
> Let me pick those in probes/fixes.
Thank you for picking it and all the good advices during the development
process!
> Thank you!
Best regards.
> On Fri, 20 Oct 2023 13:42:48 +0300
>
> Francis Laniel <flaniel@linux.microsoft.com> wrote:
> > Hi.
> >
> >
> > In the kernel source code, it exists different functions which share the
> > same name but which have, of course, different addresses as they can be
> > defined in different modules:
> > # Kernel was compiled with CONFIG_NTFS_FS and CONFIG_NTFS3_FS as built-in.
> > root@vm-amd64:~# grep ntfs_file_write_iter /proc/kallsyms
> > ffffffff814ce3c0 t __pfx_ntfs_file_write_iter
> > ffffffff814ce3d0 t ntfs_file_write_iter
> > ffffffff814fc8a0 t __pfx_ntfs_file_write_iter
> > ffffffff814fc8b0 t ntfs_file_write_iter
> > This can be source of troubles when you create a PMU kprobe for such a
> > function, as it will only install one for the first address (e.g.
> > 0xffffffff814ce3d0 in the above).
> > This could lead to some troubles were BPF based tools does not report any
> > event because the second function is not called:
> > root@vm-amd64:/mnt# mount | grep /mnt
> > /foo.img on /mnt type ntfs3 (rw,relatime,uid=0,gid=0,iocharset=utf8)
> > # ig is a tool which installs a PMU kprobe on ntfs_file_write_iter().
> > root@vm-amd64:/mnt# ig trace fsslower -m 0 -f ntfs3 --host &> /tmp/foo &
> > [1] 207
> > root@vm-amd64:/mnt# dd if=./foo of=./bar count=3
> > 3+0 records in
> > 3+0 records out
> > 1536 bytes (1.5 kB, 1.5 KiB) copied, 0.00543323 s, 283 kB/s
> > root@vm-amd64:/mnt# fg
> > ig trace fsslower -m 0 -f ntfs3 --host &> /tmp/foo
> > ^Croot@vm-amd64:/mnt# more /tmp/foo
> > RUNTIME.CONTAINERNAME RUNTIME.CONTAIN… PID COMM
> >
> > T BYTES OFFSET LAT FILE
> >
> > 214 dd
> >
> > R 512 0 766 foo
> >
> > 214 dd
> >
> > R 512 512 9 foo
> >
> > 214 dd
> >
> > As you can see in the above, only read events are reported and no write
> > because the kprobe is installed for the old ntfs_file_write_iter() and
> > not the ntfs3 one.
> > The same behavior occurs with sysfs kprobe:
> > root@vm-amd64:/# echo 'p:probe/ntfs_file_write_iter ntfs_file_write_iter'
> > > /sys/kernel/tracing/kprobe_events root@vm-amd64:/# cat
> > /sys/kernel/tracing/kprobe_events
> > p:probe/ntfs_file_write_iter ntfs_file_write_iter
> > root@vm-amd64:/# mount | grep /mnt
> > /foo.img on /mnt type ntfs3 (rw,relatime,uid=0,gid=0,iocharset=utf8)
> > root@vm-amd64:/# perf record -e probe:ntfs_file_write_iter &
> > [1] 210
> > root@vm-amd64:/# cd /mnt/
> > root@vm-amd64:/mnt# dd if=./foo of=./bar count=3
> > 3+0 records in
> > 3+0 records out
> > 1536 bytes (1.5 kB, 1.5 KiB) copied, 0.00234793 s, 654 kB/s
> > root@vm-amd64:/mnt# cd -
> > /
> > root@vm-amd64:/# fg
> > perf record -e probe:ntfs_file_write_iter
> > ^C[ perf record: Woken up 1 times to write data ]
> > [ perf record: Captured and wrote 0.056 MB perf.data ]
> >
> > root@vm-amd64:/# perf report
> > Error:
> > The perf.data data has no samples!
> > # To display the perf.data header info, please use --header/--header-only
> > optio> #
> >
> > In this contribution, I modified the functions creating sysfs and PMU
> > kprobes to test if the function name given as argument matches several
> > symbols. In this case, these functions return EADDRNOTAVAIL to indicate
> > the user to use addr and offs to remove this ambiguity.
> > So, when the above BPF tool is run, the following error message is
> > printed:
> > root@vm-amd64:~# ig trace fsslower -m 0 -f ntfs3 --host &> /tmp/foo &
> > [1] 228
> > root@vm-amd64:~# more /tmp/foo
> > RUNTIME.CONTAINERNAME RUNTIME.CONTAIN… PID COMM
> >
> > T BYTES OFFSET LAT FILE
> >
> > Error: running gadget: running gadget: installing tracer: attaching
> > kprobe: crea ting perf_kprobe PMU (arch-specific fallback for
> > "ntfs_file_write_iter"): token ntfs_file_write_iter: opening perf event:
> > cannot assign requested address And the same with sysfs kprobe:
> > root@vm-amd64:/# echo 'p:probe/ntfs_file_write_iter ntfs_file_write_iter'
> > > /sys/kernel/tracing/kprobe_events -bash: echo: write error: Cannot
> > assign requested address
> > Note that, this does not influence perf as it installs kprobes as offset
> > on
> > _text:
> > root@vm-amd64:/# perf probe --add ntfs_file_write_iter
> >
> > Added new events:
> > probe:ntfs_file_write_iter (on ntfs_file_write_iter)
> > probe:ntfs_file_write_iter (on ntfs_file_write_iter)
> >
> > ...
> > root@vm-amd64:/# cat /sys/kernel/tracing/kprobe_events
> > p:probe/ntfs_file_write_iter _text+5039088
> > p:probe/ntfs_file_write_iter _text+5228752
> >
> > Note that, this contribution is the conclusion of a previous RFC which
> > intended to install a PMU kprobe for all matching symbols [1, 2].
> >
> > If you see any way to improve this contribution, please share your
> > feedback.>
> > Changes since:
> > v1:
> > * Use EADDRNOTAVAIL instead of adding a new error code.
> > * Correct also this behavior for sysfs kprobe.
> >
> > v2:
> > * Count the number of symbols corresponding to function name and return
> > EADDRNOTAVAIL if higher than 1.
> > * Return ENOENT if above count is 0, as it would be returned later by
> > while
> > registering the kprobe.
> >
> > v3:
> > * Check symbol does not contain ':' before testing its uniqueness.
> > * Add a selftest to check this is not possible to install a kprobe for a
> > non unique symbol.
> >
> > v5:
> > * No changes, just add linux-stable as recipient.
> >
> > v6:
> > * Rephrase commit message.
> > * Add "Cc:" to stable.
> >
> > Francis Laniel (2):
> > tracing/kprobes: Return EADDRNOTAVAIL when func matches several
> >
> > symbols
> >
> > selftests/ftrace: Add new test case which checks non unique symbol
> >
> > kernel/trace/trace_kprobe.c | 63 +++++++++++++++++++
> > kernel/trace/trace_probe.h | 1 +
> > .../test.d/kprobe/kprobe_non_uniq_symbol.tc | 13 ++++
> > 3 files changed, 77 insertions(+)
> > create mode 100644
> > tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc>
> > Best regards and thank you in advance.
> > ---
> > [1]:
> > https://lore.kernel.org/lkml/20230816163517.112518-1-flaniel@linux.micros
> > oft.com/ [2]:
> > https://lore.kernel.org/lkml/20230819101105.b0c104ae4494a7d1f2eea742@kern
> > el.org/ --
> > 2.34.1
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-10-20 14:26 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-10-20 10:42 [PATCH v6 0/2] Return EADDRNOTAVAIL when func matches several symbols during kprobe creation Francis Laniel
2023-10-20 10:42 ` [PATCH v6 1/2] tracing/kprobes: Return EADDRNOTAVAIL when func matches several symbols Francis Laniel
2023-10-20 10:42 ` [PATCH v6 2/2] selftests/ftrace: Add new test case which checks non unique symbol Francis Laniel
2023-10-20 12:12 ` [PATCH v6 0/2] Return EADDRNOTAVAIL when func matches several symbols during kprobe creation Masami Hiramatsu
2023-10-20 14:26 ` Francis Laniel
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).