* [PATCH v2 1/3] perf test: Check test suite description properly
@ 2025-07-01 20:10 Namhyung Kim
2025-07-01 20:10 ` [PATCH v2 2/3] perf test: Add libsubcmd help tests Namhyung Kim
` (3 more replies)
0 siblings, 4 replies; 15+ messages in thread
From: Namhyung Kim @ 2025-07-01 20:10 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo, Ian Rogers, Kan Liang
Cc: Jiri Olsa, Adrian Hunter, Peter Zijlstra, Ingo Molnar, LKML,
linux-perf-users
Currently perf test checks the given string with descriptions for both
test suites and cases (subtests). But sometimes it's confusing since
the subtests don't contain the important keyword.
I think it's better to check the suite level and run the whole suite
together if it matches description in the suite.
Before:
$ perf test hwmon
(no output)
After:
$ perf test hwmon
10: Hwmon PMU :
10.1: Basic parsing test : Ok
10.2: Parsing without PMU name : Ok
10.3: Parsing with PMU name : Ok
And keep the existing behavior when it only matches test description only.
$ perf test "Equal cpu map"
39.5: Equal cpu map : Ok
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
tools/perf/tests/builtin-test.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index 80375ca39a37a256..846c9b3a732c9b3a 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -539,6 +539,7 @@ static int __cmd_test(struct test_suite **suites, int argc, const char *argv[],
for (struct test_suite **t = suites; *t; t++, curr_suite++) {
int curr_test_case;
+ bool suite_matched = false;
if (!perf_test__matches(test_description(*t, -1), curr_suite, argc, argv)) {
/*
@@ -556,6 +557,8 @@ static int __cmd_test(struct test_suite **suites, int argc, const char *argv[],
}
if (skip)
continue;
+ } else {
+ suite_matched = true;
}
if (intlist__find(skiplist, curr_suite + 1)) {
@@ -567,10 +570,10 @@ static int __cmd_test(struct test_suite **suites, int argc, const char *argv[],
for (unsigned int run = 0; run < runs_per_test; run++) {
test_suite__for_each_test_case(*t, curr_test_case) {
- if (!perf_test__matches(test_description(*t, curr_test_case),
+ if (!suite_matched &&
+ !perf_test__matches(test_description(*t, curr_test_case),
curr_suite, argc, argv))
continue;
-
err = start_test(*t, curr_suite, curr_test_case,
&child_tests[child_test_num++],
width, pass);
--
2.50.0.727.gbf7dc18ff4-goog
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v2 2/3] perf test: Add libsubcmd help tests 2025-07-01 20:10 [PATCH v2 1/3] perf test: Check test suite description properly Namhyung Kim @ 2025-07-01 20:10 ` Namhyung Kim 2025-07-01 20:10 ` [PATCH v2 3/3] perf tools: Fix use-after-free in help_unknown_cmd() Namhyung Kim ` (2 subsequent siblings) 3 siblings, 0 replies; 15+ messages in thread From: Namhyung Kim @ 2025-07-01 20:10 UTC (permalink / raw) To: Arnaldo Carvalho de Melo, Ian Rogers, Kan Liang Cc: Jiri Olsa, Adrian Hunter, Peter Zijlstra, Ingo Molnar, LKML, linux-perf-users Add a set of tests for subcmd routines. Currently it fails the last one since there's a bug. It'll be fixed by the next commit. $ perf test subcmd 69: libsubcmd help tests : 69.1: Load subcmd names : Ok 69.2: Uniquify subcmd names : Ok 69.3: Exclude duplicate subcmd names : FAILED! Reviewed-by: Ian Rogers <irogers@google.com> Signed-off-by: Namhyung Kim <namhyung@kernel.org> --- tools/perf/tests/Build | 1 + tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/subcmd-help.c | 108 ++++++++++++++++++++++++++++++++ tools/perf/tests/tests.h | 2 + 4 files changed, 112 insertions(+) create mode 100644 tools/perf/tests/subcmd-help.c diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build index d6c35dd0de3bd6ce..3e8394be15aea4fb 100644 --- a/tools/perf/tests/Build +++ b/tools/perf/tests/Build @@ -69,6 +69,7 @@ perf-test-y += symbols.o perf-test-y += util.o perf-test-y += hwmon_pmu.o perf-test-y += tool_pmu.o +perf-test-y += subcmd-help.o ifeq ($(SRCARCH),$(filter $(SRCARCH),x86 arm arm64 powerpc)) perf-test-$(CONFIG_DWARF_UNWIND) += dwarf-unwind.o diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c index 846c9b3a732c9b3a..e242d56523cee171 100644 --- a/tools/perf/tests/builtin-test.c +++ b/tools/perf/tests/builtin-test.c @@ -139,6 +139,7 @@ static struct test_suite *generic_tests[] = { &suite__event_groups, &suite__symbols, &suite__util, + &suite__subcmd_help, NULL, }; diff --git a/tools/perf/tests/subcmd-help.c b/tools/perf/tests/subcmd-help.c new file mode 100644 index 0000000000000000..2280b4c0e5e7083b --- /dev/null +++ b/tools/perf/tests/subcmd-help.c @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "tests.h" +#include <linux/compiler.h> +#include <subcmd/help.h> + +static int test__load_cmdnames(struct test_suite *test __maybe_unused, + int subtest __maybe_unused) +{ + struct cmdnames cmds = {}; + + add_cmdname(&cmds, "aaa", 3); + add_cmdname(&cmds, "foo", 3); + add_cmdname(&cmds, "xyz", 3); + + TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds, "aaa") == 1); + TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds, "bar") == 0); + TEST_ASSERT_VAL("case sensitive", is_in_cmdlist(&cmds, "XYZ") == 0); + + clean_cmdnames(&cmds); + return TEST_OK; +} + +static int test__uniq_cmdnames(struct test_suite *test __maybe_unused, + int subtest __maybe_unused) +{ + struct cmdnames cmds = {}; + + /* uniq() assumes it's sorted */ + add_cmdname(&cmds, "aaa", 3); + add_cmdname(&cmds, "aaa", 3); + add_cmdname(&cmds, "bbb", 3); + + TEST_ASSERT_VAL("invalid original size", cmds.cnt == 3); + /* uniquify command names (to remove second 'aaa') */ + uniq(&cmds); + TEST_ASSERT_VAL("invalid final size", cmds.cnt == 2); + + TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds, "aaa") == 1); + TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds, "bbb") == 1); + TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds, "ccc") == 0); + + clean_cmdnames(&cmds); + return TEST_OK; +} + +static int test__exclude_cmdnames(struct test_suite *test __maybe_unused, + int subtest __maybe_unused) +{ + struct cmdnames cmds1 = {}; + struct cmdnames cmds2 = {}; + + add_cmdname(&cmds1, "aaa", 3); + add_cmdname(&cmds1, "bbb", 3); + add_cmdname(&cmds1, "ccc", 3); + add_cmdname(&cmds1, "ddd", 3); + add_cmdname(&cmds1, "eee", 3); + add_cmdname(&cmds1, "fff", 3); + add_cmdname(&cmds1, "ggg", 3); + add_cmdname(&cmds1, "hhh", 3); + add_cmdname(&cmds1, "iii", 3); + add_cmdname(&cmds1, "jjj", 3); + + add_cmdname(&cmds2, "bbb", 3); + add_cmdname(&cmds2, "eee", 3); + add_cmdname(&cmds2, "jjj", 3); + + TEST_ASSERT_VAL("invalid original size", cmds1.cnt == 10); + TEST_ASSERT_VAL("invalid original size", cmds2.cnt == 3); + + /* remove duplicate command names in cmds1 */ + exclude_cmds(&cmds1, &cmds2); + + TEST_ASSERT_VAL("invalid excluded size", cmds1.cnt == 7); + TEST_ASSERT_VAL("invalid excluded size", cmds2.cnt == 3); + + /* excluded commands should not belong to cmds1 */ + TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "aaa") == 1); + TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds1, "bbb") == 0); + TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "ccc") == 1); + TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "ddd") == 1); + TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds1, "eee") == 0); + TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "fff") == 1); + TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "ggg") == 1); + TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "hhh") == 1); + TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "iii") == 1); + TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds1, "jjj") == 0); + + /* they should be only in cmds2 */ + TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds2, "bbb") == 1); + TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds2, "eee") == 1); + TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds2, "jjj") == 1); + + clean_cmdnames(&cmds1); + clean_cmdnames(&cmds2); + return TEST_OK; +} + +static struct test_case tests__subcmd_help[] = { + TEST_CASE("Load subcmd names", load_cmdnames), + TEST_CASE("Uniquify subcmd names", uniq_cmdnames), + TEST_CASE("Exclude duplicate subcmd names", exclude_cmdnames), + { .name = NULL, } +}; + +struct test_suite suite__subcmd_help = { + .desc = "libsubcmd help tests", + .test_cases = tests__subcmd_help, +}; diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h index 4c128a9594413b32..97e62db8764a0537 100644 --- a/tools/perf/tests/tests.h +++ b/tools/perf/tests/tests.h @@ -3,6 +3,7 @@ #define TESTS_H #include <stdbool.h> +#include "util/debug.h" enum { TEST_OK = 0, @@ -177,6 +178,7 @@ DECLARE_SUITE(sigtrap); DECLARE_SUITE(event_groups); DECLARE_SUITE(symbols); DECLARE_SUITE(util); +DECLARE_SUITE(subcmd_help); /* * PowerPC and S390 do not support creation of instruction breakpoints using the -- 2.50.0.727.gbf7dc18ff4-goog ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v2 3/3] perf tools: Fix use-after-free in help_unknown_cmd() 2025-07-01 20:10 [PATCH v2 1/3] perf test: Check test suite description properly Namhyung Kim 2025-07-01 20:10 ` [PATCH v2 2/3] perf test: Add libsubcmd help tests Namhyung Kim @ 2025-07-01 20:10 ` Namhyung Kim 2025-09-09 9:49 ` perf --help triggers an assertion Guilherme Amadio 2025-07-01 22:38 ` [PATCH v2 1/3] perf test: Check test suite description properly Ian Rogers 2025-07-02 16:43 ` Namhyung Kim 3 siblings, 1 reply; 15+ messages in thread From: Namhyung Kim @ 2025-07-01 20:10 UTC (permalink / raw) To: Arnaldo Carvalho de Melo, Ian Rogers, Kan Liang Cc: Jiri Olsa, Adrian Hunter, Peter Zijlstra, Ingo Molnar, LKML, linux-perf-users Currently perf aborts when it finds an invalid command. I guess it depends on the environment as I have some custom commands in the path. $ perf bad-command perf: 'bad-command' is not a perf-command. See 'perf --help'. Aborted (core dumped) It's because the exclude_cmds() in libsubcmd has a use-after-free when it removes some entries. After copying one to another entry, it keeps the pointer in the both position. And the next copy operation will free the later one but it's the same entry in the previous one. For example, let's say cmds = { A, B, C, D, E } and excludes = { B, E }. ci cj ei cmds-name excludes -----------+-------------------- 0 0 0 | A B : cmp < 0, ci == cj 1 1 0 | B B : cmp == 0 2 1 1 | C E : cmp < 0, ci != cj At this point, it frees cmds->names[1] and cmds->names[1] is assigned to cmds->names[2]. 3 2 1 | D E : cmp < 0, ci != cj Now it frees cmds->names[2] but it's the same as cmds->names[1]. So accessing cmds->names[1] will be invalid. This makes the subcmd tests succeed. $ perf test subcmd 69: libsubcmd help tests : 69.1: Load subcmd names : Ok 69.2: Uniquify subcmd names : Ok 69.3: Exclude duplicate subcmd names : Ok Fixes: 657a3efee43a ("libsubcmd: Avoid SEGV/use-after-free when commands aren't excluded") Reviewed-by: Ian Rogers <irogers@google.com> Signed-off-by: Namhyung Kim <namhyung@kernel.org> --- tools/lib/subcmd/help.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/tools/lib/subcmd/help.c b/tools/lib/subcmd/help.c index 8561b0f01a247690..9ef569492560efd7 100644 --- a/tools/lib/subcmd/help.c +++ b/tools/lib/subcmd/help.c @@ -9,6 +9,7 @@ #include <sys/stat.h> #include <unistd.h> #include <dirent.h> +#include <assert.h> #include "subcmd-util.h" #include "help.h" #include "exec-cmd.h" @@ -82,10 +83,11 @@ void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes) ci++; cj++; } else { - zfree(&cmds->names[cj]); - cmds->names[cj++] = cmds->names[ci++]; + cmds->names[cj++] = cmds->names[ci]; + cmds->names[ci++] = NULL; } } else if (cmp == 0) { + zfree(&cmds->names[ci]); ci++; ei++; } else if (cmp > 0) { @@ -94,12 +96,12 @@ void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes) } if (ci != cj) { while (ci < cmds->cnt) { - zfree(&cmds->names[cj]); - cmds->names[cj++] = cmds->names[ci++]; + cmds->names[cj++] = cmds->names[ci]; + cmds->names[ci++] = NULL; } } for (ci = cj; ci < cmds->cnt; ci++) - zfree(&cmds->names[ci]); + assert(cmds->names[ci] == NULL); cmds->cnt = cj; } -- 2.50.0.727.gbf7dc18ff4-goog ^ permalink raw reply related [flat|nested] 15+ messages in thread
* perf --help triggers an assertion 2025-07-01 20:10 ` [PATCH v2 3/3] perf tools: Fix use-after-free in help_unknown_cmd() Namhyung Kim @ 2025-09-09 9:49 ` Guilherme Amadio 2025-09-09 18:31 ` Ian Rogers 0 siblings, 1 reply; 15+ messages in thread From: Guilherme Amadio @ 2025-09-09 9:49 UTC (permalink / raw) To: namhyung Cc: acme, adrian.hunter, irogers, jolsa, kan.liang, linux-kernel, linux-perf-users, mingo, peterz Hi Namhyung, I was updating perf's package in Gentoo Linux and noticed some problems which were not there before. I tested with the version below and the problem still seems to be there. perf --help triggers an assertion (see below). Looking in the list, it seems related to the patch below: https://lore.kernel.org/linux-perf-users/20250701201027.1171561-3-namhyung@kernel.org/ Cheers, -Guilherme The problem: gentoo perf $ ./perf --help perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. Aborted (core dumped) ./perf --help Some extra information: gentoo perf $ ./perf version perf version 6.17.rc5.gf777d1112ee5 gentoo perf $ ./perf test subcmd 69: libsubcmd help tests : 69.1: Load subcmd names : Ok 69.2: Uniquify subcmd names : Ok 69.3: Exclude duplicate subcmd names : Ok gentoo perf $ ./perf bad-command perf: 'bad-command' is not a perf-command. See 'perf --help'. gentoo perf $ ./perf --help perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. Aborted (core dumped) ./perf --help gentoo perf $ gdb run --args ./perf --help GNU gdb (Gentoo 16.3 vanilla) 16.3 Copyright (C) 2024 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://bugs.gentoo.org/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ./perf... (gdb) run Starting program: /home/amadio/src/linux/tools/perf/perf --help [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib64/libthread_db.so.1". perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. Program received signal SIGABRT, Aborted. __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 44 return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0; (gdb) bt #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 #1 0x00007ffff74c1656 in __pthread_kill_internal (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:89 #2 0x00007ffff74c166d in __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:100 #3 0x00007ffff747509c in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #4 0x00007ffff747637e in __GI_abort () at abort.c:77 #5 0x00007ffff746e023 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x555555974d03 "cmds->names[ci] == NULL", file=file@entry=0x555555974cfc "help.c", line=line@entry=104, function=function@entry=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") at assert.c:118 #6 0x00007ffff746e075 in __assert_fail (assertion=0x555555974d03 "cmds->names[ci] == NULL", file=0x555555974cfc "help.c", line=104, function=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") at assert.c:127 #7 0x0000555555693813 in exclude_cmds (cmds=0x55555614e5e0 <other_cmds>, excludes=0x55555614e5c0 <main_cmds>) at help.c:104 #8 0x0000555555693eb3 in load_command_list (prefix=0x555555954071 "perf-", main_cmds=0x55555614e5c0 <main_cmds>, other_cmds=0x55555614e5e0 <other_cmds>) at help.c:252 #9 0x00005555555e9987 in cmd_help (argc=1, argv=0x7fffffffd1a0) at builtin-help.c:458 #10 0x0000555555685d45 in run_builtin (p=0x555556130de0 <commands+192>, argc=1, argv=0x7fffffffd1a0) at perf.c:349 #11 0x0000555555685fe1 in handle_internal_command (argc=1, argv=0x7fffffffd1a0) at perf.c:401 #12 0x0000555555686142 in run_argv (argcp=0x7fffffffcfac, argv=0x7fffffffcfa0) at perf.c:445 #13 0x0000555555686493 in main (argc=1, argv=0x7fffffffd1a0) at perf.c:553 (gdb) quit ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: perf --help triggers an assertion 2025-09-09 9:49 ` perf --help triggers an assertion Guilherme Amadio @ 2025-09-09 18:31 ` Ian Rogers 2025-09-10 12:52 ` Guilherme Amadio 0 siblings, 1 reply; 15+ messages in thread From: Ian Rogers @ 2025-09-09 18:31 UTC (permalink / raw) To: Guilherme Amadio Cc: namhyung, acme, adrian.hunter, jolsa, kan.liang, linux-kernel, linux-perf-users, mingo, peterz On Tue, Sep 9, 2025 at 2:49 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > Hi Namhyung, > > I was updating perf's package in Gentoo Linux and noticed some problems > which were not there before. I tested with the version below and the problem > still seems to be there. perf --help triggers an assertion (see below). > Looking in the list, it seems related to the patch below: > > https://lore.kernel.org/linux-perf-users/20250701201027.1171561-3-namhyung@kernel.org/ > > Cheers, > -Guilherme > > The problem: > > gentoo perf $ ./perf --help > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > Aborted (core dumped) ./perf --help > > Some extra information: > > gentoo perf $ ./perf version > perf version 6.17.rc5.gf777d1112ee5 > gentoo perf $ ./perf test subcmd > 69: libsubcmd help tests : > 69.1: Load subcmd names : Ok > 69.2: Uniquify subcmd names : Ok > 69.3: Exclude duplicate subcmd names : Ok > gentoo perf $ ./perf bad-command > perf: 'bad-command' is not a perf-command. See 'perf --help'. > gentoo perf $ ./perf --help > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > Aborted (core dumped) ./perf --help > gentoo perf $ gdb run --args ./perf --help > GNU gdb (Gentoo 16.3 vanilla) 16.3 > Copyright (C) 2024 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. > Type "show copying" and "show warranty" for details. > This GDB was configured as "x86_64-pc-linux-gnu". > Type "show configuration" for configuration details. > For bug reporting instructions, please see: > <https://bugs.gentoo.org/>. > Find the GDB manual and other documentation resources online at: > <http://www.gnu.org/software/gdb/documentation/>. > > For help, type "help". > Type "apropos word" to search for commands related to "word"... > Reading symbols from ./perf... > (gdb) run > Starting program: /home/amadio/src/linux/tools/perf/perf --help > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/usr/lib64/libthread_db.so.1". > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > Program received signal SIGABRT, Aborted. > __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 > 44 return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0; > (gdb) bt > #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 > #1 0x00007ffff74c1656 in __pthread_kill_internal (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:89 > #2 0x00007ffff74c166d in __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:100 > #3 0x00007ffff747509c in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 > #4 0x00007ffff747637e in __GI_abort () at abort.c:77 > #5 0x00007ffff746e023 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x555555974d03 "cmds->names[ci] == NULL", > file=file@entry=0x555555974cfc "help.c", line=line@entry=104, function=function@entry=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") > at assert.c:118 > #6 0x00007ffff746e075 in __assert_fail (assertion=0x555555974d03 "cmds->names[ci] == NULL", file=0x555555974cfc "help.c", line=104, > function=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") at assert.c:127 > #7 0x0000555555693813 in exclude_cmds (cmds=0x55555614e5e0 <other_cmds>, excludes=0x55555614e5c0 <main_cmds>) at help.c:104 > #8 0x0000555555693eb3 in load_command_list (prefix=0x555555954071 "perf-", main_cmds=0x55555614e5c0 <main_cmds>, > other_cmds=0x55555614e5e0 <other_cmds>) at help.c:252 > #9 0x00005555555e9987 in cmd_help (argc=1, argv=0x7fffffffd1a0) at builtin-help.c:458 > #10 0x0000555555685d45 in run_builtin (p=0x555556130de0 <commands+192>, argc=1, argv=0x7fffffffd1a0) at perf.c:349 > #11 0x0000555555685fe1 in handle_internal_command (argc=1, argv=0x7fffffffd1a0) at perf.c:401 > #12 0x0000555555686142 in run_argv (argcp=0x7fffffffcfac, argv=0x7fffffffcfa0) at perf.c:445 > #13 0x0000555555686493 in main (argc=1, argv=0x7fffffffd1a0) at perf.c:553 > (gdb) quit Thanks Guilherme, I tried to reproduce the same version with various options: DEBUG=1 -UNDEBUG in EXTRA_CFLAGS, -fsanitize=address. Being in various directories with "perf-" prefixed files. I'm afraid I wasn't able to reproduce. The assert is trying to avoid a memory leak, so non-critical, and I couldn't in a quick inspection eye-ball an issue. Without getting a reproduction I don't think I can make progress with the issue. Thanks, Ian ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: perf --help triggers an assertion 2025-09-09 18:31 ` Ian Rogers @ 2025-09-10 12:52 ` Guilherme Amadio 2025-09-10 15:04 ` Ian Rogers 2025-09-11 20:02 ` Arnaldo Carvalho de Melo 0 siblings, 2 replies; 15+ messages in thread From: Guilherme Amadio @ 2025-09-10 12:52 UTC (permalink / raw) To: Ian Rogers Cc: namhyung, acme, adrian.hunter, jolsa, kan.liang, linux-kernel, linux-perf-users, mingo, peterz On Tue, Sep 09, 2025 at 11:31:51AM -0700, Ian Rogers wrote: > On Tue, Sep 9, 2025 at 2:49 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > > > Hi Namhyung, > > > > I was updating perf's package in Gentoo Linux and noticed some problems > > which were not there before. I tested with the version below and the problem > > still seems to be there. perf --help triggers an assertion (see below). > > Looking in the list, it seems related to the patch below: > > > > https://lore.kernel.org/linux-perf-users/20250701201027.1171561-3-namhyung@kernel.org/ > > > > Cheers, > > -Guilherme > > > > The problem: > > > > gentoo perf $ ./perf --help > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > Aborted (core dumped) ./perf --help > > > > Some extra information: > > > > gentoo perf $ ./perf version > > perf version 6.17.rc5.gf777d1112ee5 > > gentoo perf $ ./perf test subcmd > > 69: libsubcmd help tests : > > 69.1: Load subcmd names : Ok > > 69.2: Uniquify subcmd names : Ok > > 69.3: Exclude duplicate subcmd names : Ok > > gentoo perf $ ./perf bad-command > > perf: 'bad-command' is not a perf-command. See 'perf --help'. > > gentoo perf $ ./perf --help > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > Aborted (core dumped) ./perf --help > > gentoo perf $ gdb run --args ./perf --help > > GNU gdb (Gentoo 16.3 vanilla) 16.3 > > Copyright (C) 2024 Free Software Foundation, Inc. > > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > > This is free software: you are free to change and redistribute it. > > There is NO WARRANTY, to the extent permitted by law. > > Type "show copying" and "show warranty" for details. > > This GDB was configured as "x86_64-pc-linux-gnu". > > Type "show configuration" for configuration details. > > For bug reporting instructions, please see: > > <https://bugs.gentoo.org/>. > > Find the GDB manual and other documentation resources online at: > > <http://www.gnu.org/software/gdb/documentation/>. > > > > For help, type "help". > > Type "apropos word" to search for commands related to "word"... > > Reading symbols from ./perf... > > (gdb) run > > Starting program: /home/amadio/src/linux/tools/perf/perf --help > > [Thread debugging using libthread_db enabled] > > Using host libthread_db library "/usr/lib64/libthread_db.so.1". > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > > > Program received signal SIGABRT, Aborted. > > __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 > > 44 return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0; > > (gdb) bt > > #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 > > #1 0x00007ffff74c1656 in __pthread_kill_internal (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:89 > > #2 0x00007ffff74c166d in __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:100 > > #3 0x00007ffff747509c in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 > > #4 0x00007ffff747637e in __GI_abort () at abort.c:77 > > #5 0x00007ffff746e023 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x555555974d03 "cmds->names[ci] == NULL", > > file=file@entry=0x555555974cfc "help.c", line=line@entry=104, function=function@entry=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") > > at assert.c:118 > > #6 0x00007ffff746e075 in __assert_fail (assertion=0x555555974d03 "cmds->names[ci] == NULL", file=0x555555974cfc "help.c", line=104, > > function=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") at assert.c:127 > > #7 0x0000555555693813 in exclude_cmds (cmds=0x55555614e5e0 <other_cmds>, excludes=0x55555614e5c0 <main_cmds>) at help.c:104 > > #8 0x0000555555693eb3 in load_command_list (prefix=0x555555954071 "perf-", main_cmds=0x55555614e5c0 <main_cmds>, > > other_cmds=0x55555614e5e0 <other_cmds>) at help.c:252 > > #9 0x00005555555e9987 in cmd_help (argc=1, argv=0x7fffffffd1a0) at builtin-help.c:458 > > #10 0x0000555555685d45 in run_builtin (p=0x555556130de0 <commands+192>, argc=1, argv=0x7fffffffd1a0) at perf.c:349 > > #11 0x0000555555685fe1 in handle_internal_command (argc=1, argv=0x7fffffffd1a0) at perf.c:401 > > #12 0x0000555555686142 in run_argv (argcp=0x7fffffffcfac, argv=0x7fffffffcfa0) at perf.c:445 > > #13 0x0000555555686493 in main (argc=1, argv=0x7fffffffd1a0) at perf.c:553 > > (gdb) quit > > Thanks Guilherme, > > I tried to reproduce the same version with various options: DEBUG=1 > -UNDEBUG in EXTRA_CFLAGS, -fsanitize=address. Being in various > directories with "perf-" prefixed files. I'm afraid I wasn't able to > reproduce. The assert is trying to avoid a memory leak, so > non-critical, and I couldn't in a quick inspection eye-ball an issue. > Without getting a reproduction I don't think I can make progress with > the issue. I do not have any special setup on my machine (if you consider Gentoo not special, of course). I just did a git bisect and arrived at commit 9401d1771dad99bfc795dd2ae0c292343fd1f78d, which is the commit I linked above. I used $ make -C tools/perf clean && make -B -j16 -C tools/perf DEBUG=1 WERROR=0 NO_SHELLCHECK=1 then called perf --help each time (but perf help record or equivalent also triggers the assertion). I'm using GCC 14 on a 3950X. The features I have enabled and installed dependencies are below. This patch has been backported to at least 6.12 stable, and I can reproduce this with v6.12.45. gentoo linux $ tools/perf/perf version --build-options perf version 6.16.g45effee4b85a aio: [ on ] # HAVE_AIO_SUPPORT bpf: [ on ] # HAVE_LIBBPF_SUPPORT bpf_skeletons: [ on ] # HAVE_BPF_SKEL debuginfod: [ on ] # HAVE_DEBUGINFOD_SUPPORT dwarf: [ on ] # HAVE_LIBDW_SUPPORT dwarf_getlocations: [ on ] # HAVE_LIBDW_SUPPORT dwarf-unwind: [ on ] # HAVE_DWARF_UNWIND_SUPPORT auxtrace: [ on ] # HAVE_AUXTRACE_SUPPORT libbfd: [ OFF ] # HAVE_LIBBFD_SUPPORT ( tip: Deprecated, license incompatibility, use BUILD_NONDISTRO=1 and install binutils-dev[el] ) libcapstone: [ on ] # HAVE_LIBCAPSTONE_SUPPORT libcrypto: [ on ] # HAVE_LIBCRYPTO_SUPPORT libdw-dwarf-unwind: [ on ] # HAVE_LIBDW_SUPPORT libelf: [ on ] # HAVE_LIBELF_SUPPORT libnuma: [ on ] # HAVE_LIBNUMA_SUPPORT libopencsd: [ OFF ] # HAVE_CSTRACE_SUPPORT libperl: [ on ] # HAVE_LIBPERL_SUPPORT libpfm4: [ on ] # HAVE_LIBPFM libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT libslang: [ on ] # HAVE_SLANG_SUPPORT libtraceevent: [ on ] # HAVE_LIBTRACEEVENT libunwind: [ OFF ] # HAVE_LIBUNWIND_SUPPORT ( tip: Deprecated, use LIBUNWIND=1 and install libunwind-dev[el] to build with it ) lzma: [ on ] # HAVE_LZMA_SUPPORT numa_num_possible_cpus: [ on ] # HAVE_LIBNUMA_SUPPORT zlib: [ on ] # HAVE_ZLIB_SUPPORT zstd: [ on ] # HAVE_ZSTD_SUPPORT And here are the version of all dependencies I have installed: * dependency graph for dev-util/perf-6.16-r1 `-- dev-util/perf-6.16-r1 `-- dev-util/babeltrace-1.5.11 `-- dev-libs/libbpf-1.6.2 `-- dev-util/bpftool-7.6.0 `-- dev-util/pahole-1.30 `-- llvm-core/clang-18.1.8-r6 `-- llvm-core/llvm-18.1.8-r6 `-- llvm-core/clang-19.1.7 `-- llvm-core/llvm-19.1.7 `-- llvm-core/clang-20.1.8 `-- llvm-core/llvm-20.1.8 `-- llvm-core/clang-21.1.0 `-- llvm-core/llvm-21.1.0 `-- sys-libs/libcap-2.76 `-- dev-libs/capstone-5.0.6 `-- dev-libs/openssl-3.5.2 `-- x11-libs/gtk+-2.24.33-r3 `-- virtual/jre-21 `-- dev-libs/libpfm-4.13.0 `-- dev-libs/libtraceevent-1.8.4 `-- dev-libs/libtracefs-1.8.2 `-- app-arch/xz-utils-5.8.1-r1 `-- sys-process/numactl-2.0.19 `-- dev-lang/perl-5.42.0 `-- dev-lang/python-0.3.13.7 `-- dev-lang/python-0.3.14.0_rc2 `-- dev-lang/python-3.11.13_p1 `-- dev-lang/python-3.12.11_p1 `-- dev-lang/python-3.13.9999 `-- dev-lang/python-3.14.0_rc2 `-- sys-libs/slang-2.3.3-r1 `-- dev-debug/systemtap-5.3 `-- dev-util/google-perftools-2.17.2 `-- sys-libs/libunwind-1.8.2-r1 `-- app-arch/zstd-1.5.7-r1 `-- dev-libs/elfutils-0.193 `-- sys-libs/binutils-libs-2.45-r1 `-- sys-libs/zlib-1.3.1-r1 `-- virtual/libcrypt-2-r1 `-- sys-kernel/linux-headers-6.16 `-- virtual/jdk-21 `-- app-arch/tar-1.35 `-- dev-python/setuptools-80.9.0-r1 `-- app-alternatives/yacc-1-r2 `-- app-alternatives/lex-0-r1 `-- sys-apps/which-2.23 `-- virtual/pkgconfig-3 `-- app-text/asciidoc-10.2.1 `-- app-text/sgml-common-0.6.3-r7 `-- app-text/xmlto-0.0.28-r11 `-- sys-process/time-1.9-r1 I hope that this helps with reproducing the problem, but if not, let me know which additional information you'd like to have and I will send it. My running kernel is 6.16.4, and glibc is version 2.42. Best regards, -Guilherme ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: perf --help triggers an assertion 2025-09-10 12:52 ` Guilherme Amadio @ 2025-09-10 15:04 ` Ian Rogers 2025-12-07 19:08 ` Guilherme Amadio 2025-09-11 20:02 ` Arnaldo Carvalho de Melo 1 sibling, 1 reply; 15+ messages in thread From: Ian Rogers @ 2025-09-10 15:04 UTC (permalink / raw) To: Guilherme Amadio Cc: namhyung, acme, adrian.hunter, jolsa, kan.liang, linux-kernel, linux-perf-users, mingo, peterz On Wed, Sep 10, 2025 at 5:52 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > On Tue, Sep 09, 2025 at 11:31:51AM -0700, Ian Rogers wrote: > > On Tue, Sep 9, 2025 at 2:49 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > > > > > Hi Namhyung, > > > > > > I was updating perf's package in Gentoo Linux and noticed some problems > > > which were not there before. I tested with the version below and the problem > > > still seems to be there. perf --help triggers an assertion (see below). > > > Looking in the list, it seems related to the patch below: > > > > > > https://lore.kernel.org/linux-perf-users/20250701201027.1171561-3-namhyung@kernel.org/ > > > > > > Cheers, > > > -Guilherme > > > > > > The problem: > > > > > > gentoo perf $ ./perf --help > > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > > Aborted (core dumped) ./perf --help > > > > > > Some extra information: > > > > > > gentoo perf $ ./perf version > > > perf version 6.17.rc5.gf777d1112ee5 > > > gentoo perf $ ./perf test subcmd > > > 69: libsubcmd help tests : > > > 69.1: Load subcmd names : Ok > > > 69.2: Uniquify subcmd names : Ok > > > 69.3: Exclude duplicate subcmd names : Ok > > > gentoo perf $ ./perf bad-command > > > perf: 'bad-command' is not a perf-command. See 'perf --help'. > > > gentoo perf $ ./perf --help > > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > > Aborted (core dumped) ./perf --help > > > gentoo perf $ gdb run --args ./perf --help > > > GNU gdb (Gentoo 16.3 vanilla) 16.3 > > > Copyright (C) 2024 Free Software Foundation, Inc. > > > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > > > This is free software: you are free to change and redistribute it. > > > There is NO WARRANTY, to the extent permitted by law. > > > Type "show copying" and "show warranty" for details. > > > This GDB was configured as "x86_64-pc-linux-gnu". > > > Type "show configuration" for configuration details. > > > For bug reporting instructions, please see: > > > <https://bugs.gentoo.org/>. > > > Find the GDB manual and other documentation resources online at: > > > <http://www.gnu.org/software/gdb/documentation/>. > > > > > > For help, type "help". > > > Type "apropos word" to search for commands related to "word"... > > > Reading symbols from ./perf... > > > (gdb) run > > > Starting program: /home/amadio/src/linux/tools/perf/perf --help > > > [Thread debugging using libthread_db enabled] > > > Using host libthread_db library "/usr/lib64/libthread_db.so.1". > > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > > > > > Program received signal SIGABRT, Aborted. > > > __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 > > > 44 return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0; > > > (gdb) bt > > > #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 > > > #1 0x00007ffff74c1656 in __pthread_kill_internal (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:89 > > > #2 0x00007ffff74c166d in __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:100 > > > #3 0x00007ffff747509c in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 > > > #4 0x00007ffff747637e in __GI_abort () at abort.c:77 > > > #5 0x00007ffff746e023 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x555555974d03 "cmds->names[ci] == NULL", > > > file=file@entry=0x555555974cfc "help.c", line=line@entry=104, function=function@entry=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") > > > at assert.c:118 > > > #6 0x00007ffff746e075 in __assert_fail (assertion=0x555555974d03 "cmds->names[ci] == NULL", file=0x555555974cfc "help.c", line=104, > > > function=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") at assert.c:127 > > > #7 0x0000555555693813 in exclude_cmds (cmds=0x55555614e5e0 <other_cmds>, excludes=0x55555614e5c0 <main_cmds>) at help.c:104 > > > #8 0x0000555555693eb3 in load_command_list (prefix=0x555555954071 "perf-", main_cmds=0x55555614e5c0 <main_cmds>, > > > other_cmds=0x55555614e5e0 <other_cmds>) at help.c:252 > > > #9 0x00005555555e9987 in cmd_help (argc=1, argv=0x7fffffffd1a0) at builtin-help.c:458 > > > #10 0x0000555555685d45 in run_builtin (p=0x555556130de0 <commands+192>, argc=1, argv=0x7fffffffd1a0) at perf.c:349 > > > #11 0x0000555555685fe1 in handle_internal_command (argc=1, argv=0x7fffffffd1a0) at perf.c:401 > > > #12 0x0000555555686142 in run_argv (argcp=0x7fffffffcfac, argv=0x7fffffffcfa0) at perf.c:445 > > > #13 0x0000555555686493 in main (argc=1, argv=0x7fffffffd1a0) at perf.c:553 > > > (gdb) quit > > > > Thanks Guilherme, > > > > I tried to reproduce the same version with various options: DEBUG=1 > > -UNDEBUG in EXTRA_CFLAGS, -fsanitize=address. Being in various > > directories with "perf-" prefixed files. I'm afraid I wasn't able to > > reproduce. The assert is trying to avoid a memory leak, so > > non-critical, and I couldn't in a quick inspection eye-ball an issue. > > Without getting a reproduction I don't think I can make progress with > > the issue. > > I do not have any special setup on my machine (if you consider Gentoo > not special, of course). I just did a git bisect and arrived at commit > 9401d1771dad99bfc795dd2ae0c292343fd1f78d, which is the commit I linked > above. I used > > $ make -C tools/perf clean && make -B -j16 -C tools/perf DEBUG=1 WERROR=0 NO_SHELLCHECK=1 > > then called perf --help each time (but perf help record or equivalent > also triggers the assertion). I'm using GCC 14 on a 3950X. The features > I have enabled and installed dependencies are below. This patch has been > backported to at least 6.12 stable, and I can reproduce this with > v6.12.45. > > gentoo linux $ tools/perf/perf version --build-options > perf version 6.16.g45effee4b85a > aio: [ on ] # HAVE_AIO_SUPPORT > bpf: [ on ] # HAVE_LIBBPF_SUPPORT > bpf_skeletons: [ on ] # HAVE_BPF_SKEL > debuginfod: [ on ] # HAVE_DEBUGINFOD_SUPPORT > dwarf: [ on ] # HAVE_LIBDW_SUPPORT > dwarf_getlocations: [ on ] # HAVE_LIBDW_SUPPORT > dwarf-unwind: [ on ] # HAVE_DWARF_UNWIND_SUPPORT > auxtrace: [ on ] # HAVE_AUXTRACE_SUPPORT > libbfd: [ OFF ] # HAVE_LIBBFD_SUPPORT ( tip: Deprecated, license incompatibility, use BUILD_NONDISTRO=1 and install binutils-dev[el] ) > libcapstone: [ on ] # HAVE_LIBCAPSTONE_SUPPORT > libcrypto: [ on ] # HAVE_LIBCRYPTO_SUPPORT > libdw-dwarf-unwind: [ on ] # HAVE_LIBDW_SUPPORT > libelf: [ on ] # HAVE_LIBELF_SUPPORT > libnuma: [ on ] # HAVE_LIBNUMA_SUPPORT > libopencsd: [ OFF ] # HAVE_CSTRACE_SUPPORT > libperl: [ on ] # HAVE_LIBPERL_SUPPORT > libpfm4: [ on ] # HAVE_LIBPFM > libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT > libslang: [ on ] # HAVE_SLANG_SUPPORT > libtraceevent: [ on ] # HAVE_LIBTRACEEVENT > libunwind: [ OFF ] # HAVE_LIBUNWIND_SUPPORT ( tip: Deprecated, use LIBUNWIND=1 and install libunwind-dev[el] to build with it ) > lzma: [ on ] # HAVE_LZMA_SUPPORT > numa_num_possible_cpus: [ on ] # HAVE_LIBNUMA_SUPPORT > zlib: [ on ] # HAVE_ZLIB_SUPPORT > zstd: [ on ] # HAVE_ZSTD_SUPPORT > > And here are the version of all dependencies I have installed: > > * dependency graph for dev-util/perf-6.16-r1 > `-- dev-util/perf-6.16-r1 > `-- dev-util/babeltrace-1.5.11 > `-- dev-libs/libbpf-1.6.2 > `-- dev-util/bpftool-7.6.0 > `-- dev-util/pahole-1.30 > `-- llvm-core/clang-18.1.8-r6 > `-- llvm-core/llvm-18.1.8-r6 > `-- llvm-core/clang-19.1.7 > `-- llvm-core/llvm-19.1.7 > `-- llvm-core/clang-20.1.8 > `-- llvm-core/llvm-20.1.8 > `-- llvm-core/clang-21.1.0 > `-- llvm-core/llvm-21.1.0 > `-- sys-libs/libcap-2.76 > `-- dev-libs/capstone-5.0.6 > `-- dev-libs/openssl-3.5.2 > `-- x11-libs/gtk+-2.24.33-r3 > `-- virtual/jre-21 > `-- dev-libs/libpfm-4.13.0 > `-- dev-libs/libtraceevent-1.8.4 > `-- dev-libs/libtracefs-1.8.2 > `-- app-arch/xz-utils-5.8.1-r1 > `-- sys-process/numactl-2.0.19 > `-- dev-lang/perl-5.42.0 > `-- dev-lang/python-0.3.13.7 > `-- dev-lang/python-0.3.14.0_rc2 > `-- dev-lang/python-3.11.13_p1 > `-- dev-lang/python-3.12.11_p1 > `-- dev-lang/python-3.13.9999 > `-- dev-lang/python-3.14.0_rc2 > `-- sys-libs/slang-2.3.3-r1 > `-- dev-debug/systemtap-5.3 > `-- dev-util/google-perftools-2.17.2 > `-- sys-libs/libunwind-1.8.2-r1 > `-- app-arch/zstd-1.5.7-r1 > `-- dev-libs/elfutils-0.193 > `-- sys-libs/binutils-libs-2.45-r1 > `-- sys-libs/zlib-1.3.1-r1 > `-- virtual/libcrypt-2-r1 > `-- sys-kernel/linux-headers-6.16 > `-- virtual/jdk-21 > `-- app-arch/tar-1.35 > `-- dev-python/setuptools-80.9.0-r1 > `-- app-alternatives/yacc-1-r2 > `-- app-alternatives/lex-0-r1 > `-- sys-apps/which-2.23 > `-- virtual/pkgconfig-3 > `-- app-text/asciidoc-10.2.1 > `-- app-text/sgml-common-0.6.3-r7 > `-- app-text/xmlto-0.0.28-r11 > `-- sys-process/time-1.9-r1 > > I hope that this helps with reproducing the problem, but if not, let me > know which additional information you'd like to have and I will send it. > My running kernel is 6.16.4, and glibc is version 2.42. Thanks Guilherme, There is also: https://lore.kernel.org/lkml/CADHxFxSBKuKUWDvki9QDqc5jf1Y+k2EpvOnDsg8O09pZ10n2NA@mail.gmail.com/ which may give me a reproduction. The fix there doesn't look right. I'll try to find time to look into it again. Thanks, Ian ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: perf --help triggers an assertion 2025-09-10 15:04 ` Ian Rogers @ 2025-12-07 19:08 ` Guilherme Amadio 2025-12-07 21:54 ` Ian Rogers 0 siblings, 1 reply; 15+ messages in thread From: Guilherme Amadio @ 2025-12-07 19:08 UTC (permalink / raw) To: Ian Rogers Cc: namhyung, acme, adrian.hunter, jolsa, kan.liang, linux-kernel, linux-perf-users, mingo, peterz Hi Ian, On Wed, Sep 10, 2025 at 08:04:42AM -0700, Ian Rogers wrote: > On Wed, Sep 10, 2025 at 5:52 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > > > On Tue, Sep 09, 2025 at 11:31:51AM -0700, Ian Rogers wrote: > > > On Tue, Sep 9, 2025 at 2:49 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > > > > > > > Hi Namhyung, > > > > > > > > I was updating perf's package in Gentoo Linux and noticed some problems > > > > which were not there before. I tested with the version below and the problem > > > > still seems to be there. perf --help triggers an assertion (see below). > > > > Looking in the list, it seems related to the patch below: > > > > > > > > https://lore.kernel.org/linux-perf-users/20250701201027.1171561-3-namhyung@kernel.org/ > > > > > > > > Cheers, > > > > -Guilherme > > > > > > > > The problem: > > > > > > > > gentoo perf $ ./perf --help > > > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > > > Aborted (core dumped) ./perf --help > > > > > > > > Some extra information: > > > > > > > > gentoo perf $ ./perf version > > > > perf version 6.17.rc5.gf777d1112ee5 > > > > gentoo perf $ ./perf test subcmd > > > > 69: libsubcmd help tests : > > > > 69.1: Load subcmd names : Ok > > > > 69.2: Uniquify subcmd names : Ok > > > > 69.3: Exclude duplicate subcmd names : Ok > > > > gentoo perf $ ./perf bad-command > > > > perf: 'bad-command' is not a perf-command. See 'perf --help'. > > > > gentoo perf $ ./perf --help > > > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > > > Aborted (core dumped) ./perf --help > > > > gentoo perf $ gdb run --args ./perf --help > > > > GNU gdb (Gentoo 16.3 vanilla) 16.3 > > > > Copyright (C) 2024 Free Software Foundation, Inc. > > > > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > > > > This is free software: you are free to change and redistribute it. > > > > There is NO WARRANTY, to the extent permitted by law. > > > > Type "show copying" and "show warranty" for details. > > > > This GDB was configured as "x86_64-pc-linux-gnu". > > > > Type "show configuration" for configuration details. > > > > For bug reporting instructions, please see: > > > > <https://bugs.gentoo.org/>. > > > > Find the GDB manual and other documentation resources online at: > > > > <http://www.gnu.org/software/gdb/documentation/>. > > > > > > > > For help, type "help". > > > > Type "apropos word" to search for commands related to "word"... > > > > Reading symbols from ./perf... > > > > (gdb) run > > > > Starting program: /home/amadio/src/linux/tools/perf/perf --help > > > > [Thread debugging using libthread_db enabled] > > > > Using host libthread_db library "/usr/lib64/libthread_db.so.1". > > > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > > > > > > > Program received signal SIGABRT, Aborted. > > > > __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 > > > > 44 return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0; > > > > (gdb) bt > > > > #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 > > > > #1 0x00007ffff74c1656 in __pthread_kill_internal (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:89 > > > > #2 0x00007ffff74c166d in __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:100 > > > > #3 0x00007ffff747509c in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 > > > > #4 0x00007ffff747637e in __GI_abort () at abort.c:77 > > > > #5 0x00007ffff746e023 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x555555974d03 "cmds->names[ci] == NULL", > > > > file=file@entry=0x555555974cfc "help.c", line=line@entry=104, function=function@entry=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") > > > > at assert.c:118 > > > > #6 0x00007ffff746e075 in __assert_fail (assertion=0x555555974d03 "cmds->names[ci] == NULL", file=0x555555974cfc "help.c", line=104, > > > > function=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") at assert.c:127 > > > > #7 0x0000555555693813 in exclude_cmds (cmds=0x55555614e5e0 <other_cmds>, excludes=0x55555614e5c0 <main_cmds>) at help.c:104 > > > > #8 0x0000555555693eb3 in load_command_list (prefix=0x555555954071 "perf-", main_cmds=0x55555614e5c0 <main_cmds>, > > > > other_cmds=0x55555614e5e0 <other_cmds>) at help.c:252 > > > > #9 0x00005555555e9987 in cmd_help (argc=1, argv=0x7fffffffd1a0) at builtin-help.c:458 > > > > #10 0x0000555555685d45 in run_builtin (p=0x555556130de0 <commands+192>, argc=1, argv=0x7fffffffd1a0) at perf.c:349 > > > > #11 0x0000555555685fe1 in handle_internal_command (argc=1, argv=0x7fffffffd1a0) at perf.c:401 > > > > #12 0x0000555555686142 in run_argv (argcp=0x7fffffffcfac, argv=0x7fffffffcfa0) at perf.c:445 > > > > #13 0x0000555555686493 in main (argc=1, argv=0x7fffffffd1a0) at perf.c:553 > > > > (gdb) quit > > > > > > Thanks Guilherme, > > > > > > I tried to reproduce the same version with various options: DEBUG=1 > > > -UNDEBUG in EXTRA_CFLAGS, -fsanitize=address. Being in various > > > directories with "perf-" prefixed files. I'm afraid I wasn't able to > > > reproduce. The assert is trying to avoid a memory leak, so > > > non-critical, and I couldn't in a quick inspection eye-ball an issue. > > > Without getting a reproduction I don't think I can make progress with > > > the issue. I had a look at this with gdb again while I was updating perf to 6.18 in Gentoo. BTW, I've followed your advice and turned off BUILD_NONDISTRO: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b92c5b8bd89c70c40a84f86013c4ecf321f3132f It looks like the logic to exclude commands sometimes doesn't work as expected, and you reach a line where there is an assertion for cmds->names[ci] == NULL, but it's not actually NULL. The difference with other setups is probably the installation of perf-read-vdso32 into a default path in our packaging. In gdb I saw that when running perf --help, I arrive at exclude_cmds() with the following: Breakpoint 1, exclude_cmds (cmds=cmds@entry=0x55555605bd90 <other_cmds>, excludes=excludes@entry=0x55555605bdb0 <main_cmds>) at help.c:74 74 { (gdb) p *cmds $1 = {alloc = 24, cnt = 1, names = 0x5555568fe180} (gdb) p *excludes $2 = {alloc = 24, cnt = 1, names = 0x5555568fe0c0} (gdb) p *cmds->names[0] $3 = {len = 11, name = 0x5555568db788 "read-vdso32"} (gdb) p *excludes->names[0] $4 = {len = 7, name = 0x555556890198 "archive"} I have /usr/bin/perf and /usr/bin/perf-read-vdso32, as well as this: $ ls /usr/libexec/perf-core/ dlfilters libperf-gtk.so perf-archive perf-iostat scripts tests If I remove /usr/bin/perf-read-vdso32, then perf --help works again: gentoo ~ $ sudo rm /usr/bin/perf-read-vdso32 gentoo ~ $ perf version perf version 6.18 gentoo ~ $ perf --help | head -3 usage: perf [--version] [--help] [OPTIONS] COMMAND [ARGS] gentoo ~ $ touch ~/bin/perf-read-vdso32 gentoo ~ $ chmod +x ~/bin/perf-read-vdso32 gentoo ~ $ perf --help perf: help.c:107: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. Aborted (core dumped) perf --help So, I think that if you don't have perf-read-vdso32 installed by default, you will likely be able to reproduce the problem like shown above. So, the assumption that cmds->names[0] == NULL is not correct, since we have the "read-vdso32" command at position 0, and the excludes list only contained "archive". The patch below is my attempt to fix the issue: diff --git a/tools/lib/subcmd/help.c b/tools/lib/subcmd/help.c index ddaeb4eb3e24..0c4ba87422e5 100644 --- a/tools/lib/subcmd/help.c +++ b/tools/lib/subcmd/help.c @@ -102,10 +102,10 @@ void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes) cmds->names[cj++] = cmds->names[ci]; cmds->names[ci++] = NULL; } + for (ci = cj; ci < cmds->cnt; ci++) + assert(cmds->names[ci] == NULL); + cmds->cnt = cj; } - for (ci = cj; ci < cmds->cnt; ci++) - assert(cmds->names[ci] == NULL); - cmds->cnt = cj; } So, if there are no actual command exclusions, at the end of the while loop we get out with ei > excludes->cnt, and ci == cj, like the example above, in which ci == cj == 0, and ei == 1. So the assumption that cmds->names[0] == NULL is wrong, since there's a valid "read-vdso32" command at position 0, and setting cmds->cnt = cj; is also wrong, since cnt should be 1 and is set to 0. The patch moves the asserts and the update of cmds->cnt inside the if(ci != cj), which indicates something was actually excluded. If cmds was not modified, then the initial value of cmds->cnt was already correct. Please let me know if you'd like me to submit this properly as a patch, but since it's so simple, please feel free to just take this and apply yourself. Best regards, -Guilherme ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: perf --help triggers an assertion 2025-12-07 19:08 ` Guilherme Amadio @ 2025-12-07 21:54 ` Ian Rogers 2025-12-11 14:35 ` Guilherme Amadio 0 siblings, 1 reply; 15+ messages in thread From: Ian Rogers @ 2025-12-07 21:54 UTC (permalink / raw) To: Guilherme Amadio, Jayaramappa, Srilakshmi Cc: namhyung, acme, adrian.hunter, jolsa, linux-kernel, linux-perf-users, mingo, peterz On Sun, Dec 7, 2025 at 11:08 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > Hi Ian, > > On Wed, Sep 10, 2025 at 08:04:42AM -0700, Ian Rogers wrote: > > On Wed, Sep 10, 2025 at 5:52 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > > > > > On Tue, Sep 09, 2025 at 11:31:51AM -0700, Ian Rogers wrote: > > > > On Tue, Sep 9, 2025 at 2:49 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > > > > > > > > > Hi Namhyung, > > > > > > > > > > I was updating perf's package in Gentoo Linux and noticed some problems > > > > > which were not there before. I tested with the version below and the problem > > > > > still seems to be there. perf --help triggers an assertion (see below). > > > > > Looking in the list, it seems related to the patch below: > > > > > > > > > > https://lore.kernel.org/linux-perf-users/20250701201027.1171561-3-namhyung@kernel.org/ > > > > > > > > > > Cheers, > > > > > -Guilherme > > > > > > > > > > The problem: > > > > > > > > > > gentoo perf $ ./perf --help > > > > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > > > > Aborted (core dumped) ./perf --help > > > > > > > > > > Some extra information: > > > > > > > > > > gentoo perf $ ./perf version > > > > > perf version 6.17.rc5.gf777d1112ee5 > > > > > gentoo perf $ ./perf test subcmd > > > > > 69: libsubcmd help tests : > > > > > 69.1: Load subcmd names : Ok > > > > > 69.2: Uniquify subcmd names : Ok > > > > > 69.3: Exclude duplicate subcmd names : Ok > > > > > gentoo perf $ ./perf bad-command > > > > > perf: 'bad-command' is not a perf-command. See 'perf --help'. > > > > > gentoo perf $ ./perf --help > > > > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > > > > Aborted (core dumped) ./perf --help > > > > > gentoo perf $ gdb run --args ./perf --help > > > > > GNU gdb (Gentoo 16.3 vanilla) 16.3 > > > > > Copyright (C) 2024 Free Software Foundation, Inc. > > > > > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > > > > > This is free software: you are free to change and redistribute it. > > > > > There is NO WARRANTY, to the extent permitted by law. > > > > > Type "show copying" and "show warranty" for details. > > > > > This GDB was configured as "x86_64-pc-linux-gnu". > > > > > Type "show configuration" for configuration details. > > > > > For bug reporting instructions, please see: > > > > > <https://bugs.gentoo.org/>. > > > > > Find the GDB manual and other documentation resources online at: > > > > > <http://www.gnu.org/software/gdb/documentation/>. > > > > > > > > > > For help, type "help". > > > > > Type "apropos word" to search for commands related to "word"... > > > > > Reading symbols from ./perf... > > > > > (gdb) run > > > > > Starting program: /home/amadio/src/linux/tools/perf/perf --help > > > > > [Thread debugging using libthread_db enabled] > > > > > Using host libthread_db library "/usr/lib64/libthread_db.so.1". > > > > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > > > > > > > > > Program received signal SIGABRT, Aborted. > > > > > __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 > > > > > 44 return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0; > > > > > (gdb) bt > > > > > #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 > > > > > #1 0x00007ffff74c1656 in __pthread_kill_internal (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:89 > > > > > #2 0x00007ffff74c166d in __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:100 > > > > > #3 0x00007ffff747509c in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 > > > > > #4 0x00007ffff747637e in __GI_abort () at abort.c:77 > > > > > #5 0x00007ffff746e023 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x555555974d03 "cmds->names[ci] == NULL", > > > > > file=file@entry=0x555555974cfc "help.c", line=line@entry=104, function=function@entry=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") > > > > > at assert.c:118 > > > > > #6 0x00007ffff746e075 in __assert_fail (assertion=0x555555974d03 "cmds->names[ci] == NULL", file=0x555555974cfc "help.c", line=104, > > > > > function=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") at assert.c:127 > > > > > #7 0x0000555555693813 in exclude_cmds (cmds=0x55555614e5e0 <other_cmds>, excludes=0x55555614e5c0 <main_cmds>) at help.c:104 > > > > > #8 0x0000555555693eb3 in load_command_list (prefix=0x555555954071 "perf-", main_cmds=0x55555614e5c0 <main_cmds>, > > > > > other_cmds=0x55555614e5e0 <other_cmds>) at help.c:252 > > > > > #9 0x00005555555e9987 in cmd_help (argc=1, argv=0x7fffffffd1a0) at builtin-help.c:458 > > > > > #10 0x0000555555685d45 in run_builtin (p=0x555556130de0 <commands+192>, argc=1, argv=0x7fffffffd1a0) at perf.c:349 > > > > > #11 0x0000555555685fe1 in handle_internal_command (argc=1, argv=0x7fffffffd1a0) at perf.c:401 > > > > > #12 0x0000555555686142 in run_argv (argcp=0x7fffffffcfac, argv=0x7fffffffcfa0) at perf.c:445 > > > > > #13 0x0000555555686493 in main (argc=1, argv=0x7fffffffd1a0) at perf.c:553 > > > > > (gdb) quit > > > > > > > > Thanks Guilherme, > > > > > > > > I tried to reproduce the same version with various options: DEBUG=1 > > > > -UNDEBUG in EXTRA_CFLAGS, -fsanitize=address. Being in various > > > > directories with "perf-" prefixed files. I'm afraid I wasn't able to > > > > reproduce. The assert is trying to avoid a memory leak, so > > > > non-critical, and I couldn't in a quick inspection eye-ball an issue. > > > > Without getting a reproduction I don't think I can make progress with > > > > the issue. > > I had a look at this with gdb again while I was updating perf to 6.18 in Gentoo. > > BTW, I've followed your advice and turned off BUILD_NONDISTRO: > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b92c5b8bd89c70c40a84f86013c4ecf321f3132f > > It looks like the logic to exclude commands sometimes doesn't work as > expected, and you reach a line where there is an assertion for > cmds->names[ci] == NULL, but it's not actually NULL. The difference with > other setups is probably the installation of perf-read-vdso32 into a > default path in our packaging. In gdb I saw that when running perf --help, > I arrive at exclude_cmds() with the following: > > Breakpoint 1, exclude_cmds (cmds=cmds@entry=0x55555605bd90 <other_cmds>, excludes=excludes@entry=0x55555605bdb0 <main_cmds>) at help.c:74 > 74 { > (gdb) p *cmds > $1 = {alloc = 24, cnt = 1, names = 0x5555568fe180} > (gdb) p *excludes > $2 = {alloc = 24, cnt = 1, names = 0x5555568fe0c0} > (gdb) p *cmds->names[0] > $3 = {len = 11, name = 0x5555568db788 "read-vdso32"} > (gdb) p *excludes->names[0] > $4 = {len = 7, name = 0x555556890198 "archive"} > > I have /usr/bin/perf and /usr/bin/perf-read-vdso32, as well as this: > > $ ls /usr/libexec/perf-core/ > dlfilters libperf-gtk.so perf-archive perf-iostat scripts tests > > If I remove /usr/bin/perf-read-vdso32, then perf --help works again: > > gentoo ~ $ sudo rm /usr/bin/perf-read-vdso32 > gentoo ~ $ perf version > perf version 6.18 > gentoo ~ $ perf --help | head -3 > > usage: perf [--version] [--help] [OPTIONS] COMMAND [ARGS] > > gentoo ~ $ touch ~/bin/perf-read-vdso32 > gentoo ~ $ chmod +x ~/bin/perf-read-vdso32 > gentoo ~ $ perf --help > perf: help.c:107: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > Aborted (core dumped) perf --help > > So, I think that if you don't have perf-read-vdso32 installed by default, > you will likely be able to reproduce the problem like shown above. So, > the assumption that cmds->names[0] == NULL is not correct, since we have > the "read-vdso32" command at position 0, and the excludes list only > contained "archive". The patch below is my attempt to fix the issue: > > diff --git a/tools/lib/subcmd/help.c b/tools/lib/subcmd/help.c > index ddaeb4eb3e24..0c4ba87422e5 100644 > --- a/tools/lib/subcmd/help.c > +++ b/tools/lib/subcmd/help.c > @@ -102,10 +102,10 @@ void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes) > cmds->names[cj++] = cmds->names[ci]; > cmds->names[ci++] = NULL; > } > + for (ci = cj; ci < cmds->cnt; ci++) > + assert(cmds->names[ci] == NULL); > + cmds->cnt = cj; > } > - for (ci = cj; ci < cmds->cnt; ci++) > - assert(cmds->names[ci] == NULL); > - cmds->cnt = cj; > } > > So, if there are no actual command exclusions, at the end of the while loop > we get out with ei > excludes->cnt, and ci == cj, like the example above, > in which ci == cj == 0, and ei == 1. So the assumption that > cmds->names[0] == NULL is wrong, since there's a valid "read-vdso32" > command at position 0, and setting cmds->cnt = cj; is also wrong, since > cnt should be 1 and is set to 0. The patch moves the asserts and the > update of cmds->cnt inside the if(ci != cj), which indicates something > was actually excluded. If cmds was not modified, then the initial value > of cmds->cnt was already correct. Please let me know if you'd like me > to submit this properly as a patch, but since it's so simple, please > feel free to just take this and apply yourself. Thanks Guilherme, This code is the gift that keeps on giving in terms of crashes. We did have a recent and related patch sent to the list: https://lore.kernel.org/lkml/20251202213632.2873731-1-sjayaram@akamai.com/ I need some time to get my head around the issue and the fixes. I wandered into this code originally due to address/leak sanitizer issues. Thanks, Ian > Best regards, > -Guilherme ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: perf --help triggers an assertion 2025-12-07 21:54 ` Ian Rogers @ 2025-12-11 14:35 ` Guilherme Amadio 0 siblings, 0 replies; 15+ messages in thread From: Guilherme Amadio @ 2025-12-11 14:35 UTC (permalink / raw) To: Ian Rogers Cc: Jayaramappa, Srilakshmi, namhyung, acme, adrian.hunter, jolsa, linux-kernel, linux-perf-users, mingo, peterz On Sun, Dec 07, 2025 at 01:54:19PM -0800, Ian Rogers wrote: > On Sun, Dec 7, 2025 at 11:08 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > > > Hi Ian, > > > > On Wed, Sep 10, 2025 at 08:04:42AM -0700, Ian Rogers wrote: > > > On Wed, Sep 10, 2025 at 5:52 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > > > > > > > On Tue, Sep 09, 2025 at 11:31:51AM -0700, Ian Rogers wrote: > > > > > On Tue, Sep 9, 2025 at 2:49 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > > > > > > > > > > > Hi Namhyung, > > > > > > > > > > > > I was updating perf's package in Gentoo Linux and noticed some problems > > > > > > which were not there before. I tested with the version below and the problem > > > > > > still seems to be there. perf --help triggers an assertion (see below). > > > > > > Looking in the list, it seems related to the patch below: > > > > > > > > > > > > https://lore.kernel.org/linux-perf-users/20250701201027.1171561-3-namhyung@kernel.org/ > > > > > > > > > > > > Cheers, > > > > > > -Guilherme > > > > > > > > > > > > The problem: > > > > > > > > > > > > gentoo perf $ ./perf --help > > > > > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > > > > > Aborted (core dumped) ./perf --help > > > > > > > > > > > > Some extra information: > > > > > > > > > > > > gentoo perf $ ./perf version > > > > > > perf version 6.17.rc5.gf777d1112ee5 > > > > > > gentoo perf $ ./perf test subcmd > > > > > > 69: libsubcmd help tests : > > > > > > 69.1: Load subcmd names : Ok > > > > > > 69.2: Uniquify subcmd names : Ok > > > > > > 69.3: Exclude duplicate subcmd names : Ok > > > > > > gentoo perf $ ./perf bad-command > > > > > > perf: 'bad-command' is not a perf-command. See 'perf --help'. > > > > > > gentoo perf $ ./perf --help > > > > > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > > > > > Aborted (core dumped) ./perf --help > > > > > > gentoo perf $ gdb run --args ./perf --help > > > > > > GNU gdb (Gentoo 16.3 vanilla) 16.3 > > > > > > Copyright (C) 2024 Free Software Foundation, Inc. > > > > > > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > > > > > > This is free software: you are free to change and redistribute it. > > > > > > There is NO WARRANTY, to the extent permitted by law. > > > > > > Type "show copying" and "show warranty" for details. > > > > > > This GDB was configured as "x86_64-pc-linux-gnu". > > > > > > Type "show configuration" for configuration details. > > > > > > For bug reporting instructions, please see: > > > > > > <https://bugs.gentoo.org/>. > > > > > > Find the GDB manual and other documentation resources online at: > > > > > > <http://www.gnu.org/software/gdb/documentation/>. > > > > > > > > > > > > For help, type "help". > > > > > > Type "apropos word" to search for commands related to "word"... > > > > > > Reading symbols from ./perf... > > > > > > (gdb) run > > > > > > Starting program: /home/amadio/src/linux/tools/perf/perf --help > > > > > > [Thread debugging using libthread_db enabled] > > > > > > Using host libthread_db library "/usr/lib64/libthread_db.so.1". > > > > > > perf: help.c:104: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > > > > > > > > > > > Program received signal SIGABRT, Aborted. > > > > > > __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 > > > > > > 44 return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0; > > > > > > (gdb) bt > > > > > > #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 > > > > > > #1 0x00007ffff74c1656 in __pthread_kill_internal (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:89 > > > > > > #2 0x00007ffff74c166d in __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:100 > > > > > > #3 0x00007ffff747509c in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 > > > > > > #4 0x00007ffff747637e in __GI_abort () at abort.c:77 > > > > > > #5 0x00007ffff746e023 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x555555974d03 "cmds->names[ci] == NULL", > > > > > > file=file@entry=0x555555974cfc "help.c", line=line@entry=104, function=function@entry=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") > > > > > > at assert.c:118 > > > > > > #6 0x00007ffff746e075 in __assert_fail (assertion=0x555555974d03 "cmds->names[ci] == NULL", file=0x555555974cfc "help.c", line=104, > > > > > > function=0x555555974dc8 <__PRETTY_FUNCTION__.0> "exclude_cmds") at assert.c:127 > > > > > > #7 0x0000555555693813 in exclude_cmds (cmds=0x55555614e5e0 <other_cmds>, excludes=0x55555614e5c0 <main_cmds>) at help.c:104 > > > > > > #8 0x0000555555693eb3 in load_command_list (prefix=0x555555954071 "perf-", main_cmds=0x55555614e5c0 <main_cmds>, > > > > > > other_cmds=0x55555614e5e0 <other_cmds>) at help.c:252 > > > > > > #9 0x00005555555e9987 in cmd_help (argc=1, argv=0x7fffffffd1a0) at builtin-help.c:458 > > > > > > #10 0x0000555555685d45 in run_builtin (p=0x555556130de0 <commands+192>, argc=1, argv=0x7fffffffd1a0) at perf.c:349 > > > > > > #11 0x0000555555685fe1 in handle_internal_command (argc=1, argv=0x7fffffffd1a0) at perf.c:401 > > > > > > #12 0x0000555555686142 in run_argv (argcp=0x7fffffffcfac, argv=0x7fffffffcfa0) at perf.c:445 > > > > > > #13 0x0000555555686493 in main (argc=1, argv=0x7fffffffd1a0) at perf.c:553 > > > > > > (gdb) quit > > > > > > > > > > Thanks Guilherme, > > > > > > > > > > I tried to reproduce the same version with various options: DEBUG=1 > > > > > -UNDEBUG in EXTRA_CFLAGS, -fsanitize=address. Being in various > > > > > directories with "perf-" prefixed files. I'm afraid I wasn't able to > > > > > reproduce. The assert is trying to avoid a memory leak, so > > > > > non-critical, and I couldn't in a quick inspection eye-ball an issue. > > > > > Without getting a reproduction I don't think I can make progress with > > > > > the issue. > > > > I had a look at this with gdb again while I was updating perf to 6.18 in Gentoo. > > > > BTW, I've followed your advice and turned off BUILD_NONDISTRO: > > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b92c5b8bd89c70c40a84f86013c4ecf321f3132f > > > > It looks like the logic to exclude commands sometimes doesn't work as > > expected, and you reach a line where there is an assertion for > > cmds->names[ci] == NULL, but it's not actually NULL. The difference with > > other setups is probably the installation of perf-read-vdso32 into a > > default path in our packaging. In gdb I saw that when running perf --help, > > I arrive at exclude_cmds() with the following: > > > > Breakpoint 1, exclude_cmds (cmds=cmds@entry=0x55555605bd90 <other_cmds>, excludes=excludes@entry=0x55555605bdb0 <main_cmds>) at help.c:74 > > 74 { > > (gdb) p *cmds > > $1 = {alloc = 24, cnt = 1, names = 0x5555568fe180} > > (gdb) p *excludes > > $2 = {alloc = 24, cnt = 1, names = 0x5555568fe0c0} > > (gdb) p *cmds->names[0] > > $3 = {len = 11, name = 0x5555568db788 "read-vdso32"} > > (gdb) p *excludes->names[0] > > $4 = {len = 7, name = 0x555556890198 "archive"} > > > > I have /usr/bin/perf and /usr/bin/perf-read-vdso32, as well as this: > > > > $ ls /usr/libexec/perf-core/ > > dlfilters libperf-gtk.so perf-archive perf-iostat scripts tests > > > > If I remove /usr/bin/perf-read-vdso32, then perf --help works again: > > > > gentoo ~ $ sudo rm /usr/bin/perf-read-vdso32 > > gentoo ~ $ perf version > > perf version 6.18 > > gentoo ~ $ perf --help | head -3 > > > > usage: perf [--version] [--help] [OPTIONS] COMMAND [ARGS] > > > > gentoo ~ $ touch ~/bin/perf-read-vdso32 > > gentoo ~ $ chmod +x ~/bin/perf-read-vdso32 > > gentoo ~ $ perf --help > > perf: help.c:107: exclude_cmds: Assertion `cmds->names[ci] == NULL' failed. > > Aborted (core dumped) perf --help > > > > So, I think that if you don't have perf-read-vdso32 installed by default, > > you will likely be able to reproduce the problem like shown above. So, > > the assumption that cmds->names[0] == NULL is not correct, since we have > > the "read-vdso32" command at position 0, and the excludes list only > > contained "archive". The patch below is my attempt to fix the issue: > > > > diff --git a/tools/lib/subcmd/help.c b/tools/lib/subcmd/help.c > > index ddaeb4eb3e24..0c4ba87422e5 100644 > > --- a/tools/lib/subcmd/help.c > > +++ b/tools/lib/subcmd/help.c > > @@ -102,10 +102,10 @@ void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes) > > cmds->names[cj++] = cmds->names[ci]; > > cmds->names[ci++] = NULL; > > } > > + for (ci = cj; ci < cmds->cnt; ci++) > > + assert(cmds->names[ci] == NULL); > > + cmds->cnt = cj; > > } > > - for (ci = cj; ci < cmds->cnt; ci++) > > - assert(cmds->names[ci] == NULL); > > - cmds->cnt = cj; > > } > > > > So, if there are no actual command exclusions, at the end of the while loop > > we get out with ei > excludes->cnt, and ci == cj, like the example above, > > in which ci == cj == 0, and ei == 1. So the assumption that > > cmds->names[0] == NULL is wrong, since there's a valid "read-vdso32" > > command at position 0, and setting cmds->cnt = cj; is also wrong, since > > cnt should be 1 and is set to 0. The patch moves the asserts and the > > update of cmds->cnt inside the if(ci != cj), which indicates something > > was actually excluded. If cmds was not modified, then the initial value > > of cmds->cnt was already correct. Please let me know if you'd like me > > to submit this properly as a patch, but since it's so simple, please > > feel free to just take this and apply yourself. > > Thanks Guilherme, > > This code is the gift that keeps on giving in terms of crashes. We did > have a recent and related patch sent to the list: > https://lore.kernel.org/lkml/20251202213632.2873731-1-sjayaram@akamai.com/ > I need some time to get my head around the issue and the fixes. I > wandered into this code originally due to address/leak sanitizer > issues. Thank you, Ian, I've applied Sri's patch in Gentoo. It seems to be a better fix than what I proposed above, so please feel free to add my Tested-by: to Sri's patch. https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=a77ba80e013b62ca44a3123167aa3ee376e7faeb Best regards, -Guilherme ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: perf --help triggers an assertion 2025-09-10 12:52 ` Guilherme Amadio 2025-09-10 15:04 ` Ian Rogers @ 2025-09-11 20:02 ` Arnaldo Carvalho de Melo 2025-09-11 21:12 ` Namhyung Kim 2025-09-12 11:27 ` Guilherme Amadio 1 sibling, 2 replies; 15+ messages in thread From: Arnaldo Carvalho de Melo @ 2025-09-11 20:02 UTC (permalink / raw) To: Guilherme Amadio Cc: Ian Rogers, namhyung, adrian.hunter, jolsa, kan.liang, linux-kernel, linux-perf-users, mingo, peterz On Wed, Sep 10, 2025 at 02:52:25PM +0200, Guilherme Amadio wrote: > On Tue, Sep 09, 2025 at 11:31:51AM -0700, Ian Rogers wrote: > > On Tue, Sep 9, 2025 at 2:49 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > > I was updating perf's package in Gentoo Linux and noticed some problems > > > which were not there before. I tested with the version below and the problem > > > still seems to be there. perf --help triggers an assertion (see below). > > > Looking in the list, it seems related to the patch below: > > > https://lore.kernel.org/linux-perf-users/20250701201027.1171561-3-namhyung@kernel.org/ <SNIP> > > I tried to reproduce the same version with various options: DEBUG=1 > > -UNDEBUG in EXTRA_CFLAGS, -fsanitize=address. Being in various > > directories with "perf-" prefixed files. I'm afraid I wasn't able to > > reproduce. The assert is trying to avoid a memory leak, so > > non-critical, and I couldn't in a quick inspection eye-ball an issue. > > Without getting a reproduction I don't think I can make progress with > > the issue. > I do not have any special setup on my machine (if you consider Gentoo > not special, of course). I just did a git bisect and arrived at commit > 9401d1771dad99bfc795dd2ae0c292343fd1f78d, which is the commit I linked > above. I used> ⬢ [acme@toolbx perf-tools-next]$ git show 9401d1771dad99bfc795dd2ae0c292343fd1f78d fatal: bad object 9401d1771dad99bfc795dd2ae0c292343fd1f78d ⬢ [acme@toolbx perf-tools-next]$ Looking for the patch title I got to this one: commit 1fdf938168c4d26fa279d4f204768690d1f9c4ae Author: Namhyung Kim <namhyung@kernel.org> Date: Tue Jul 1 13:10:27 2025 -0700 perf tools: Fix use-after-free in help_unknown_cmd() ? - Arnaldo > $ make -C tools/perf clean && make -B -j16 -C tools/perf DEBUG=1 WERROR=0 NO_SHELLCHECK=1 > > then called perf --help each time (but perf help record or equivalent > also triggers the assertion). I'm using GCC 14 on a 3950X. The features > I have enabled and installed dependencies are below. This patch has been > backported to at least 6.12 stable, and I can reproduce this with > v6.12.45. > > gentoo linux $ tools/perf/perf version --build-options > perf version 6.16.g45effee4b85a > aio: [ on ] # HAVE_AIO_SUPPORT > bpf: [ on ] # HAVE_LIBBPF_SUPPORT > bpf_skeletons: [ on ] # HAVE_BPF_SKEL > debuginfod: [ on ] # HAVE_DEBUGINFOD_SUPPORT > dwarf: [ on ] # HAVE_LIBDW_SUPPORT > dwarf_getlocations: [ on ] # HAVE_LIBDW_SUPPORT > dwarf-unwind: [ on ] # HAVE_DWARF_UNWIND_SUPPORT > auxtrace: [ on ] # HAVE_AUXTRACE_SUPPORT > libbfd: [ OFF ] # HAVE_LIBBFD_SUPPORT ( tip: Deprecated, license incompatibility, use BUILD_NONDISTRO=1 and install binutils-dev[el] ) > libcapstone: [ on ] # HAVE_LIBCAPSTONE_SUPPORT > libcrypto: [ on ] # HAVE_LIBCRYPTO_SUPPORT > libdw-dwarf-unwind: [ on ] # HAVE_LIBDW_SUPPORT > libelf: [ on ] # HAVE_LIBELF_SUPPORT > libnuma: [ on ] # HAVE_LIBNUMA_SUPPORT > libopencsd: [ OFF ] # HAVE_CSTRACE_SUPPORT > libperl: [ on ] # HAVE_LIBPERL_SUPPORT > libpfm4: [ on ] # HAVE_LIBPFM > libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT > libslang: [ on ] # HAVE_SLANG_SUPPORT > libtraceevent: [ on ] # HAVE_LIBTRACEEVENT > libunwind: [ OFF ] # HAVE_LIBUNWIND_SUPPORT ( tip: Deprecated, use LIBUNWIND=1 and install libunwind-dev[el] to build with it ) > lzma: [ on ] # HAVE_LZMA_SUPPORT > numa_num_possible_cpus: [ on ] # HAVE_LIBNUMA_SUPPORT > zlib: [ on ] # HAVE_ZLIB_SUPPORT > zstd: [ on ] # HAVE_ZSTD_SUPPORT > > And here are the version of all dependencies I have installed: > > * dependency graph for dev-util/perf-6.16-r1 > `-- dev-util/perf-6.16-r1 > `-- dev-util/babeltrace-1.5.11 > `-- dev-libs/libbpf-1.6.2 > `-- dev-util/bpftool-7.6.0 > `-- dev-util/pahole-1.30 > `-- llvm-core/clang-18.1.8-r6 > `-- llvm-core/llvm-18.1.8-r6 > `-- llvm-core/clang-19.1.7 > `-- llvm-core/llvm-19.1.7 > `-- llvm-core/clang-20.1.8 > `-- llvm-core/llvm-20.1.8 > `-- llvm-core/clang-21.1.0 > `-- llvm-core/llvm-21.1.0 > `-- sys-libs/libcap-2.76 > `-- dev-libs/capstone-5.0.6 > `-- dev-libs/openssl-3.5.2 > `-- x11-libs/gtk+-2.24.33-r3 > `-- virtual/jre-21 > `-- dev-libs/libpfm-4.13.0 > `-- dev-libs/libtraceevent-1.8.4 > `-- dev-libs/libtracefs-1.8.2 > `-- app-arch/xz-utils-5.8.1-r1 > `-- sys-process/numactl-2.0.19 > `-- dev-lang/perl-5.42.0 > `-- dev-lang/python-0.3.13.7 > `-- dev-lang/python-0.3.14.0_rc2 > `-- dev-lang/python-3.11.13_p1 > `-- dev-lang/python-3.12.11_p1 > `-- dev-lang/python-3.13.9999 > `-- dev-lang/python-3.14.0_rc2 > `-- sys-libs/slang-2.3.3-r1 > `-- dev-debug/systemtap-5.3 > `-- dev-util/google-perftools-2.17.2 > `-- sys-libs/libunwind-1.8.2-r1 > `-- app-arch/zstd-1.5.7-r1 > `-- dev-libs/elfutils-0.193 > `-- sys-libs/binutils-libs-2.45-r1 > `-- sys-libs/zlib-1.3.1-r1 > `-- virtual/libcrypt-2-r1 > `-- sys-kernel/linux-headers-6.16 > `-- virtual/jdk-21 > `-- app-arch/tar-1.35 > `-- dev-python/setuptools-80.9.0-r1 > `-- app-alternatives/yacc-1-r2 > `-- app-alternatives/lex-0-r1 > `-- sys-apps/which-2.23 > `-- virtual/pkgconfig-3 > `-- app-text/asciidoc-10.2.1 > `-- app-text/sgml-common-0.6.3-r7 > `-- app-text/xmlto-0.0.28-r11 > `-- sys-process/time-1.9-r1 > > I hope that this helps with reproducing the problem, but if not, let me > know which additional information you'd like to have and I will send it. > My running kernel is 6.16.4, and glibc is version 2.42. > > Best regards, > -Guilherme ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: perf --help triggers an assertion 2025-09-11 20:02 ` Arnaldo Carvalho de Melo @ 2025-09-11 21:12 ` Namhyung Kim 2025-09-12 11:27 ` Guilherme Amadio 1 sibling, 0 replies; 15+ messages in thread From: Namhyung Kim @ 2025-09-11 21:12 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Guilherme Amadio, Ian Rogers, adrian.hunter, jolsa, kan.liang, linux-kernel, linux-perf-users, mingo, peterz On Thu, Sep 11, 2025 at 05:02:52PM -0300, Arnaldo Carvalho de Melo wrote: > On Wed, Sep 10, 2025 at 02:52:25PM +0200, Guilherme Amadio wrote: > > On Tue, Sep 09, 2025 at 11:31:51AM -0700, Ian Rogers wrote: > > > On Tue, Sep 9, 2025 at 2:49 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > > > I was updating perf's package in Gentoo Linux and noticed some problems > > > > which were not there before. I tested with the version below and the problem > > > > still seems to be there. perf --help triggers an assertion (see below). > > > > Looking in the list, it seems related to the patch below: > > > > > https://lore.kernel.org/linux-perf-users/20250701201027.1171561-3-namhyung@kernel.org/ > > <SNIP> > > > > I tried to reproduce the same version with various options: DEBUG=1 > > > -UNDEBUG in EXTRA_CFLAGS, -fsanitize=address. Being in various > > > directories with "perf-" prefixed files. I'm afraid I wasn't able to > > > reproduce. The assert is trying to avoid a memory leak, so > > > non-critical, and I couldn't in a quick inspection eye-ball an issue. > > > Without getting a reproduction I don't think I can make progress with > > > the issue. > > > I do not have any special setup on my machine (if you consider Gentoo > > not special, of course). I just did a git bisect and arrived at commit > > 9401d1771dad99bfc795dd2ae0c292343fd1f78d, which is the commit I linked > > above. I used> > > ⬢ [acme@toolbx perf-tools-next]$ git show 9401d1771dad99bfc795dd2ae0c292343fd1f78d > fatal: bad object 9401d1771dad99bfc795dd2ae0c292343fd1f78d > ⬢ [acme@toolbx perf-tools-next]$ > > Looking for the patch title I got to this one: > > commit 1fdf938168c4d26fa279d4f204768690d1f9c4ae > Author: Namhyung Kim <namhyung@kernel.org> > Date: Tue Jul 1 13:10:27 2025 -0700 > > perf tools: Fix use-after-free in help_unknown_cmd() Sorry for the trouble. The hupu's patch should fix the problem. Thanks, Namhyung ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: perf --help triggers an assertion 2025-09-11 20:02 ` Arnaldo Carvalho de Melo 2025-09-11 21:12 ` Namhyung Kim @ 2025-09-12 11:27 ` Guilherme Amadio 1 sibling, 0 replies; 15+ messages in thread From: Guilherme Amadio @ 2025-09-12 11:27 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Ian Rogers, namhyung, adrian.hunter, jolsa, kan.liang, linux-kernel, linux-perf-users, mingo, peterz On Thu, Sep 11, 2025 at 05:02:52PM -0300, Arnaldo Carvalho de Melo wrote: > On Wed, Sep 10, 2025 at 02:52:25PM +0200, Guilherme Amadio wrote: > > On Tue, Sep 09, 2025 at 11:31:51AM -0700, Ian Rogers wrote: > > > On Tue, Sep 9, 2025 at 2:49 AM Guilherme Amadio <amadio@gentoo.org> wrote: > > > > I was updating perf's package in Gentoo Linux and noticed some problems > > > > which were not there before. I tested with the version below and the problem > > > > still seems to be there. perf --help triggers an assertion (see below). > > > > Looking in the list, it seems related to the patch below: > > > > > https://lore.kernel.org/linux-perf-users/20250701201027.1171561-3-namhyung@kernel.org/ > > <SNIP> > > > > I tried to reproduce the same version with various options: DEBUG=1 > > > -UNDEBUG in EXTRA_CFLAGS, -fsanitize=address. Being in various > > > directories with "perf-" prefixed files. I'm afraid I wasn't able to > > > reproduce. The assert is trying to avoid a memory leak, so > > > non-critical, and I couldn't in a quick inspection eye-ball an issue. > > > Without getting a reproduction I don't think I can make progress with > > > the issue. > > > I do not have any special setup on my machine (if you consider Gentoo > > not special, of course). I just did a git bisect and arrived at commit > > 9401d1771dad99bfc795dd2ae0c292343fd1f78d, which is the commit I linked > > above. I used> > > ⬢ [acme@toolbx perf-tools-next]$ git show 9401d1771dad99bfc795dd2ae0c292343fd1f78d > fatal: bad object 9401d1771dad99bfc795dd2ae0c292343fd1f78d > ⬢ [acme@toolbx perf-tools-next]$ > > Looking for the patch title I got to this one: > > commit 1fdf938168c4d26fa279d4f204768690d1f9c4ae > Author: Namhyung Kim <namhyung@kernel.org> > Date: Tue Jul 1 13:10:27 2025 -0700 > > perf tools: Fix use-after-free in help_unknown_cmd() > > ? Sorry, I used the tag I arrived at with git bisect, which was probably cherry-picked: gentoo linux $ git remote -v amadio git@github.com:amadio/linux (fetch) amadio git@github.com:amadio/linux (push) perf-tools git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools.git (fetch) perf-tools git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools.git (push) perf-tools-next git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git (fetch) perf-tools-next git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git (push) stable git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git (fetch) stable git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git (push) torvalds git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git (fetch) torvalds git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git (push) gentoo linux $ git show 9401d1771dad99bfc795dd2ae0c292343fd1f78d | head commit 9401d1771dad99bfc795dd2ae0c292343fd1f78d Author: Namhyung Kim <namhyung@kernel.org> Date: Tue Jul 1 13:10:27 2025 -0700 perf tools: Fix use-after-free in help_unknown_cmd() [ Upstream commit 1fdf938168c4d26fa279d4f204768690d1f9c4ae ] Currently perf aborts when it finds an invalid command. I guess it depends on the environment as I have some custom commands in the path. gentoo linux $ git tag --contains 9401d1771dad99bfc795dd2ae0c292343fd1f78d v6.16.1 v6.16.2 v6.16.3 v6.16.4 v6.16.5 v6.16.6 v6.16.7 gentoo linux $ git tag --contains 1fdf938168c4d26fa279d4f204768690d1f9c4ae perf-tools-fixes-for-v6.17-2025-08-27 perf-tools-fixes-for-v6.17-2025-09-05 perf-tools-for-v6.17-2025-08-01 v6.17-rc1 v6.17-rc2 v6.17-rc3 v6.17-rc4 v6.17-rc5 Cheers, -Guilherme > > - Arnaldo > > > $ make -C tools/perf clean && make -B -j16 -C tools/perf DEBUG=1 WERROR=0 NO_SHELLCHECK=1 > > > > then called perf --help each time (but perf help record or equivalent > > also triggers the assertion). I'm using GCC 14 on a 3950X. The features > > I have enabled and installed dependencies are below. This patch has been > > backported to at least 6.12 stable, and I can reproduce this with > > v6.12.45. > > > > gentoo linux $ tools/perf/perf version --build-options > > perf version 6.16.g45effee4b85a > > aio: [ on ] # HAVE_AIO_SUPPORT > > bpf: [ on ] # HAVE_LIBBPF_SUPPORT > > bpf_skeletons: [ on ] # HAVE_BPF_SKEL > > debuginfod: [ on ] # HAVE_DEBUGINFOD_SUPPORT > > dwarf: [ on ] # HAVE_LIBDW_SUPPORT > > dwarf_getlocations: [ on ] # HAVE_LIBDW_SUPPORT > > dwarf-unwind: [ on ] # HAVE_DWARF_UNWIND_SUPPORT > > auxtrace: [ on ] # HAVE_AUXTRACE_SUPPORT > > libbfd: [ OFF ] # HAVE_LIBBFD_SUPPORT ( tip: Deprecated, license incompatibility, use BUILD_NONDISTRO=1 and install binutils-dev[el] ) > > libcapstone: [ on ] # HAVE_LIBCAPSTONE_SUPPORT > > libcrypto: [ on ] # HAVE_LIBCRYPTO_SUPPORT > > libdw-dwarf-unwind: [ on ] # HAVE_LIBDW_SUPPORT > > libelf: [ on ] # HAVE_LIBELF_SUPPORT > > libnuma: [ on ] # HAVE_LIBNUMA_SUPPORT > > libopencsd: [ OFF ] # HAVE_CSTRACE_SUPPORT > > libperl: [ on ] # HAVE_LIBPERL_SUPPORT > > libpfm4: [ on ] # HAVE_LIBPFM > > libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT > > libslang: [ on ] # HAVE_SLANG_SUPPORT > > libtraceevent: [ on ] # HAVE_LIBTRACEEVENT > > libunwind: [ OFF ] # HAVE_LIBUNWIND_SUPPORT ( tip: Deprecated, use LIBUNWIND=1 and install libunwind-dev[el] to build with it ) > > lzma: [ on ] # HAVE_LZMA_SUPPORT > > numa_num_possible_cpus: [ on ] # HAVE_LIBNUMA_SUPPORT > > zlib: [ on ] # HAVE_ZLIB_SUPPORT > > zstd: [ on ] # HAVE_ZSTD_SUPPORT > > > > And here are the version of all dependencies I have installed: > > > > * dependency graph for dev-util/perf-6.16-r1 > > `-- dev-util/perf-6.16-r1 > > `-- dev-util/babeltrace-1.5.11 > > `-- dev-libs/libbpf-1.6.2 > > `-- dev-util/bpftool-7.6.0 > > `-- dev-util/pahole-1.30 > > `-- llvm-core/clang-18.1.8-r6 > > `-- llvm-core/llvm-18.1.8-r6 > > `-- llvm-core/clang-19.1.7 > > `-- llvm-core/llvm-19.1.7 > > `-- llvm-core/clang-20.1.8 > > `-- llvm-core/llvm-20.1.8 > > `-- llvm-core/clang-21.1.0 > > `-- llvm-core/llvm-21.1.0 > > `-- sys-libs/libcap-2.76 > > `-- dev-libs/capstone-5.0.6 > > `-- dev-libs/openssl-3.5.2 > > `-- x11-libs/gtk+-2.24.33-r3 > > `-- virtual/jre-21 > > `-- dev-libs/libpfm-4.13.0 > > `-- dev-libs/libtraceevent-1.8.4 > > `-- dev-libs/libtracefs-1.8.2 > > `-- app-arch/xz-utils-5.8.1-r1 > > `-- sys-process/numactl-2.0.19 > > `-- dev-lang/perl-5.42.0 > > `-- dev-lang/python-0.3.13.7 > > `-- dev-lang/python-0.3.14.0_rc2 > > `-- dev-lang/python-3.11.13_p1 > > `-- dev-lang/python-3.12.11_p1 > > `-- dev-lang/python-3.13.9999 > > `-- dev-lang/python-3.14.0_rc2 > > `-- sys-libs/slang-2.3.3-r1 > > `-- dev-debug/systemtap-5.3 > > `-- dev-util/google-perftools-2.17.2 > > `-- sys-libs/libunwind-1.8.2-r1 > > `-- app-arch/zstd-1.5.7-r1 > > `-- dev-libs/elfutils-0.193 > > `-- sys-libs/binutils-libs-2.45-r1 > > `-- sys-libs/zlib-1.3.1-r1 > > `-- virtual/libcrypt-2-r1 > > `-- sys-kernel/linux-headers-6.16 > > `-- virtual/jdk-21 > > `-- app-arch/tar-1.35 > > `-- dev-python/setuptools-80.9.0-r1 > > `-- app-alternatives/yacc-1-r2 > > `-- app-alternatives/lex-0-r1 > > `-- sys-apps/which-2.23 > > `-- virtual/pkgconfig-3 > > `-- app-text/asciidoc-10.2.1 > > `-- app-text/sgml-common-0.6.3-r7 > > `-- app-text/xmlto-0.0.28-r11 > > `-- sys-process/time-1.9-r1 > > > > I hope that this helps with reproducing the problem, but if not, let me > > know which additional information you'd like to have and I will send it. > > My running kernel is 6.16.4, and glibc is version 2.42. > > > > Best regards, > > -Guilherme ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 1/3] perf test: Check test suite description properly 2025-07-01 20:10 [PATCH v2 1/3] perf test: Check test suite description properly Namhyung Kim 2025-07-01 20:10 ` [PATCH v2 2/3] perf test: Add libsubcmd help tests Namhyung Kim 2025-07-01 20:10 ` [PATCH v2 3/3] perf tools: Fix use-after-free in help_unknown_cmd() Namhyung Kim @ 2025-07-01 22:38 ` Ian Rogers 2025-07-02 16:43 ` Namhyung Kim 3 siblings, 0 replies; 15+ messages in thread From: Ian Rogers @ 2025-07-01 22:38 UTC (permalink / raw) To: Namhyung Kim Cc: Arnaldo Carvalho de Melo, Kan Liang, Jiri Olsa, Adrian Hunter, Peter Zijlstra, Ingo Molnar, LKML, linux-perf-users On Tue, Jul 1, 2025 at 1:10 PM Namhyung Kim <namhyung@kernel.org> wrote: > > Currently perf test checks the given string with descriptions for both > test suites and cases (subtests). But sometimes it's confusing since > the subtests don't contain the important keyword. > > I think it's better to check the suite level and run the whole suite > together if it matches description in the suite. > > Before: > $ perf test hwmon > (no output) > > After: > $ perf test hwmon > 10: Hwmon PMU : > 10.1: Basic parsing test : Ok > 10.2: Parsing without PMU name : Ok > 10.3: Parsing with PMU name : Ok > > And keep the existing behavior when it only matches test description only. > > $ perf test "Equal cpu map" > 39.5: Equal cpu map : Ok > > Signed-off-by: Namhyung Kim <namhyung@kernel.org> Reviewed-by: Ian Rogers <irogers@google.com> Thanks, Ian > --- > tools/perf/tests/builtin-test.c | 7 +++++-- > 1 file changed, 5 insertions(+), 2 deletions(-) > > diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c > index 80375ca39a37a256..846c9b3a732c9b3a 100644 > --- a/tools/perf/tests/builtin-test.c > +++ b/tools/perf/tests/builtin-test.c > @@ -539,6 +539,7 @@ static int __cmd_test(struct test_suite **suites, int argc, const char *argv[], > > for (struct test_suite **t = suites; *t; t++, curr_suite++) { > int curr_test_case; > + bool suite_matched = false; > > if (!perf_test__matches(test_description(*t, -1), curr_suite, argc, argv)) { > /* > @@ -556,6 +557,8 @@ static int __cmd_test(struct test_suite **suites, int argc, const char *argv[], > } > if (skip) > continue; > + } else { > + suite_matched = true; > } > > if (intlist__find(skiplist, curr_suite + 1)) { > @@ -567,10 +570,10 @@ static int __cmd_test(struct test_suite **suites, int argc, const char *argv[], > > for (unsigned int run = 0; run < runs_per_test; run++) { > test_suite__for_each_test_case(*t, curr_test_case) { > - if (!perf_test__matches(test_description(*t, curr_test_case), > + if (!suite_matched && > + !perf_test__matches(test_description(*t, curr_test_case), > curr_suite, argc, argv)) > continue; > - > err = start_test(*t, curr_suite, curr_test_case, > &child_tests[child_test_num++], > width, pass); > -- > 2.50.0.727.gbf7dc18ff4-goog > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 1/3] perf test: Check test suite description properly 2025-07-01 20:10 [PATCH v2 1/3] perf test: Check test suite description properly Namhyung Kim ` (2 preceding siblings ...) 2025-07-01 22:38 ` [PATCH v2 1/3] perf test: Check test suite description properly Ian Rogers @ 2025-07-02 16:43 ` Namhyung Kim 3 siblings, 0 replies; 15+ messages in thread From: Namhyung Kim @ 2025-07-02 16:43 UTC (permalink / raw) To: Arnaldo Carvalho de Melo, Ian Rogers, Kan Liang, Namhyung Kim Cc: Jiri Olsa, Adrian Hunter, Peter Zijlstra, Ingo Molnar, LKML, linux-perf-users On Tue, 01 Jul 2025 13:10:25 -0700, Namhyung Kim wrote: > Currently perf test checks the given string with descriptions for both > test suites and cases (subtests). But sometimes it's confusing since > the subtests don't contain the important keyword. > > I think it's better to check the suite level and run the whole suite > together if it matches description in the suite. > > [...] Applied to perf-tools-next, thanks! Best regards, Namhyung ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2025-12-11 14:35 UTC | newest] Thread overview: 15+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-07-01 20:10 [PATCH v2 1/3] perf test: Check test suite description properly Namhyung Kim 2025-07-01 20:10 ` [PATCH v2 2/3] perf test: Add libsubcmd help tests Namhyung Kim 2025-07-01 20:10 ` [PATCH v2 3/3] perf tools: Fix use-after-free in help_unknown_cmd() Namhyung Kim 2025-09-09 9:49 ` perf --help triggers an assertion Guilherme Amadio 2025-09-09 18:31 ` Ian Rogers 2025-09-10 12:52 ` Guilherme Amadio 2025-09-10 15:04 ` Ian Rogers 2025-12-07 19:08 ` Guilherme Amadio 2025-12-07 21:54 ` Ian Rogers 2025-12-11 14:35 ` Guilherme Amadio 2025-09-11 20:02 ` Arnaldo Carvalho de Melo 2025-09-11 21:12 ` Namhyung Kim 2025-09-12 11:27 ` Guilherme Amadio 2025-07-01 22:38 ` [PATCH v2 1/3] perf test: Check test suite description properly Ian Rogers 2025-07-02 16:43 ` Namhyung Kim
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).