* [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions
@ 2026-07-10 5:39 Ian Rogers
2026-07-10 5:39 ` [PATCH v1 2/2] perf cap: If capability is missing still perform root test Ian Rogers
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: Ian Rogers @ 2026-07-10 5:39 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Jiri Olsa, Ian Rogers, Adrian Hunter, James Clark,
Jiawei Sun, linux-kernel, linux-perf-users
Under system-wide (-a) parallel streaming mode (--threads=cpu),
background recording threads can be inundated by a continuous
firehose of hardware samples generated by the OS. In this state,
a background thread's local hit count remains unequal to its
sample count, causing it to bypass the blocking fdarray__poll()
call entirely on each iteration of its recording loop.
Because the termination check relies on the POLLHUP event status
populated specifically by fdarray__poll(), bypassing it prevents
the background thread from ever recognizing that its control pipe
was closed by the main thread. This traps the background thread
in an infinite recording loop, hanging the main thread indefinitely
as it awaits a termination acknowledgment that never arrives.
Ensure teardown completion by adding explicit evlist__disable()
calls in the main thread's cleanup paths at out_child: and
out_child_no_flush:. Additionally, patch fdarray__filter() to
respect the fdarray_flag__nonfilterable flag, preventing it
from incorrectly setting the background thread's control pipe
file descriptor to -1 and clearing its revents mask upon
processing termination POLLHUP signals.
Fixes: f94563fac269 ("perf record: fix poll storm when monitored threads exit")
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/lib/api/fd/array.c | 6 ++++--
tools/perf/builtin-record.c | 2 ++
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/tools/lib/api/fd/array.c b/tools/lib/api/fd/array.c
index ffe8272af59b..16a047f1906e 100644
--- a/tools/lib/api/fd/array.c
+++ b/tools/lib/api/fd/array.c
@@ -115,6 +115,9 @@ int fdarray__filter(struct fdarray *fda, short revents,
return 0;
for (fd = 0; fd < fda->nr; ++fd) {
+ if (fda->priv[fd].flags & fdarray_flag__nonfilterable)
+ continue;
+
if (!fda->entries[fd].events)
continue;
@@ -132,8 +135,7 @@ int fdarray__filter(struct fdarray *fda, short revents,
continue;
}
- if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable))
- ++nr;
+ ++nr;
}
return nr;
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index ebd3ed0c9b3e..d1276382b77a 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -2890,11 +2890,13 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
record__synthesize_workload(rec, true);
out_child:
+ evlist__disable(rec->evlist);
record__stop_threads(rec);
record__mmap_read_all(rec, true);
goto out_free_threads;
out_child_no_flush:
/* mmap read already failed — retrying would just fail again */
+ evlist__disable(rec->evlist);
record__stop_threads(rec);
out_free_threads:
record__free_thread_data(rec);
--
2.55.0.795.g602f6c329a-goog
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH v1 2/2] perf cap: If capability is missing still perform root test
2026-07-10 5:39 [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions Ian Rogers
@ 2026-07-10 5:39 ` Ian Rogers
2026-07-10 22:02 ` [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions Namhyung Kim
2026-07-12 6:56 ` (subset) " Namhyung Kim
2 siblings, 0 replies; 4+ messages in thread
From: Ian Rogers @ 2026-07-10 5:39 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Jiri Olsa, Ian Rogers, Adrian Hunter, James Clark,
Jiawei Sun, linux-kernel, linux-perf-users
Refine the capability checking logic in perf_cap__capable() to in the
event the capability is missing do the fallback used_root
test. Previously this was only performed when the capability syscall
failed.
Fixes: e25ebda78e23 ("perf cap: Tidy up and improve capability testing")
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/util/cap.c | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/tools/perf/util/cap.c b/tools/perf/util/cap.c
index ac6d1d9a523d..a2baecbc3145 100644
--- a/tools/perf/util/cap.c
+++ b/tools/perf/util/cap.c
@@ -20,6 +20,7 @@ bool perf_cap__capable(int cap, bool *used_root)
};
struct __user_cap_data_struct data[MAX_LINUX_CAPABILITY_U32S] = {};
__u32 cap_val;
+ bool has_cap = false;
*used_root = false;
while (syscall(SYS_capget, &header, &data[0]) == -1) {
@@ -37,12 +38,21 @@ bool perf_cap__capable(int cap, bool *used_root)
if (cap >= 32) {
if (header.version == _LINUX_CAPABILITY_VERSION_3) {
cap_val = data[1].effective;
+ has_cap = (cap_val & (1U << (cap & 0x1f))) != 0;
} else {
/* Capability beyond 32 is requested but only 32 are supported. */
- return false;
+ has_cap = false;
}
} else {
cap_val = data[0].effective;
+ has_cap = (cap_val & (1U << (cap & 0x1f))) != 0;
}
- return (cap_val & (1 << (cap & 0x1f))) != 0;
+
+ if (has_cap) {
+ *used_root = false;
+ return true;
+ }
+
+ *used_root = true;
+ return geteuid() == 0;
}
--
2.55.0.795.g602f6c329a-goog
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions
2026-07-10 5:39 [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions Ian Rogers
2026-07-10 5:39 ` [PATCH v1 2/2] perf cap: If capability is missing still perform root test Ian Rogers
@ 2026-07-10 22:02 ` Namhyung Kim
2026-07-12 6:56 ` (subset) " Namhyung Kim
2 siblings, 0 replies; 4+ messages in thread
From: Namhyung Kim @ 2026-07-10 22:02 UTC (permalink / raw)
To: Ian Rogers
Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo, Jiri Olsa,
Adrian Hunter, James Clark, Jiawei Sun, linux-kernel,
linux-perf-users
Hi Ian,
On Thu, Jul 09, 2026 at 10:39:06PM -0700, Ian Rogers wrote:
> Under system-wide (-a) parallel streaming mode (--threads=cpu),
> background recording threads can be inundated by a continuous
> firehose of hardware samples generated by the OS. In this state,
> a background thread's local hit count remains unequal to its
> sample count, causing it to bypass the blocking fdarray__poll()
> call entirely on each iteration of its recording loop.
>
> Because the termination check relies on the POLLHUP event status
> populated specifically by fdarray__poll(), bypassing it prevents
> the background thread from ever recognizing that its control pipe
> was closed by the main thread. This traps the background thread
> in an infinite recording loop, hanging the main thread indefinitely
> as it awaits a termination acknowledgment that never arrives.
>
> Ensure teardown completion by adding explicit evlist__disable()
> calls in the main thread's cleanup paths at out_child: and
> out_child_no_flush:. Additionally, patch fdarray__filter() to
> respect the fdarray_flag__nonfilterable flag, preventing it
> from incorrectly setting the background thread's control pipe
> file descriptor to -1 and clearing its revents mask upon
> processing termination POLLHUP signals.
>
> Fixes: f94563fac269 ("perf record: fix poll storm when monitored threads exit")
> Assisted-by: Antigravity:gemini-3.5-flash
> Signed-off-by: Ian Rogers <irogers@google.com>
I didn't know you already sent the fix so I was doing the same. It
seems your fix is better as it added evlist__disable(). I'll test this
and pick up if it's ok.
Thanks,
Namhyung
> ---
> tools/lib/api/fd/array.c | 6 ++++--
> tools/perf/builtin-record.c | 2 ++
> 2 files changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/tools/lib/api/fd/array.c b/tools/lib/api/fd/array.c
> index ffe8272af59b..16a047f1906e 100644
> --- a/tools/lib/api/fd/array.c
> +++ b/tools/lib/api/fd/array.c
> @@ -115,6 +115,9 @@ int fdarray__filter(struct fdarray *fda, short revents,
> return 0;
>
> for (fd = 0; fd < fda->nr; ++fd) {
> + if (fda->priv[fd].flags & fdarray_flag__nonfilterable)
> + continue;
> +
> if (!fda->entries[fd].events)
> continue;
>
> @@ -132,8 +135,7 @@ int fdarray__filter(struct fdarray *fda, short revents,
> continue;
> }
>
> - if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable))
> - ++nr;
> + ++nr;
> }
>
> return nr;
> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
> index ebd3ed0c9b3e..d1276382b77a 100644
> --- a/tools/perf/builtin-record.c
> +++ b/tools/perf/builtin-record.c
> @@ -2890,11 +2890,13 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
> record__synthesize_workload(rec, true);
>
> out_child:
> + evlist__disable(rec->evlist);
> record__stop_threads(rec);
> record__mmap_read_all(rec, true);
> goto out_free_threads;
> out_child_no_flush:
> /* mmap read already failed — retrying would just fail again */
> + evlist__disable(rec->evlist);
> record__stop_threads(rec);
> out_free_threads:
> record__free_thread_data(rec);
> --
> 2.55.0.795.g602f6c329a-goog
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: (subset) [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions
2026-07-10 5:39 [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions Ian Rogers
2026-07-10 5:39 ` [PATCH v1 2/2] perf cap: If capability is missing still perform root test Ian Rogers
2026-07-10 22:02 ` [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions Namhyung Kim
@ 2026-07-12 6:56 ` Namhyung Kim
2 siblings, 0 replies; 4+ messages in thread
From: Namhyung Kim @ 2026-07-12 6:56 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo, Jiri Olsa,
Adrian Hunter, James Clark, Jiawei Sun, linux-kernel,
linux-perf-users, Ian Rogers
On Thu, 09 Jul 2026 22:39:06 -0700, Ian Rogers wrote:
> Under system-wide (-a) parallel streaming mode (--threads=cpu),
> background recording threads can be inundated by a continuous
> firehose of hardware samples generated by the OS. In this state,
> a background thread's local hit count remains unequal to its
> sample count, causing it to bypass the blocking fdarray__poll()
> call entirely on each iteration of its recording loop.
>
> [...]
Applied to perf-tools-next, thanks!
Best regards,
Namhyung
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-07-12 6:56 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-10 5:39 [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions Ian Rogers
2026-07-10 5:39 ` [PATCH v1 2/2] perf cap: If capability is missing still perform root test Ian Rogers
2026-07-10 22:02 ` [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions Namhyung Kim
2026-07-12 6:56 ` (subset) " Namhyung Kim
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox