From: "Chen, Zide" <zide.chen@intel.com>
To: Namhyung Kim <namhyung@kernel.org>
Cc: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
Peter Zijlstra <peterz@infradead.org>,
Adrian Hunter <adrian.hunter@intel.com>,
Ingo Molnar <mingo@redhat.com>, Jiri Olsa <jolsa@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Ian Rogers <irogers@google.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
thomas.falcon@intel.com, dapeng1.mi@linux.intel.com,
xudong.hao@intel.com
Subject: Re: [PATCH] perf tools: Refactor precise_ip fallback logic
Date: Tue, 11 Nov 2025 11:11:45 -0800 [thread overview]
Message-ID: <42bafd4d-ff6e-4529-95f7-9c4c963fc49c@intel.com> (raw)
In-Reply-To: <aRLq0isFxDJlsHlL@google.com>
On 11/10/2025 11:50 PM, Namhyung Kim wrote:
> On Fri, Nov 07, 2025 at 02:31:23PM -0800, Chen, Zide wrote:
>>
>>
>> On 11/7/2025 1:42 PM, Namhyung Kim wrote:
>>> On Thu, Nov 06, 2025 at 05:23:09PM -0800, Chen, Zide wrote:
>>>>
>>>>
>>>> On 11/6/2025 10:52 AM, Namhyung Kim wrote:
>>>>> On Tue, Nov 04, 2025 at 11:10:44AM -0800, Chen, Zide wrote:
>>>>>>
>>>>>>
>>>>>> On 11/3/2025 7:48 PM, Namhyung Kim wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> Sorry for the delay.
>>>>>>>
>>>>>>> On Mon, Oct 27, 2025 at 11:56:52AM -0700, Chen, Zide wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 10/25/2025 5:42 PM, Namhyung Kim wrote:
>>>>>>>>> On Fri, Oct 24, 2025 at 11:03:17AM -0700, Chen, Zide wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 10/23/2025 7:30 PM, Namhyung Kim wrote:
>>>>>>>>>>> Hello,
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Oct 22, 2025 at 03:08:02PM -0700, Zide Chen wrote:
>>>>>>>>>>>> Commit c33aea446bf555ab ("perf tools: Fix precise_ip fallback logic")
>>>>>>>>>>>> unconditionally called the precise_ip fallback and moved it after the
>>>>>>>>>>>> missing-feature checks so that it could handle EINVAL as well.
>>>>>>>>>>>>
>>>>>>>>>>>> However, this introduced an issue: after disabling missing features,
>>>>>>>>>>>> the event could fail to open, which makes the subsequent precise_ip
>>>>>>>>>>>> fallback useless since it will always fail.
>>>>>>>>>>>>
>>>>>>>>>>>> For example, run the following command on Intel SPR:
>>>>>>>>>>>>
>>>>>>>>>>>> $ perf record -e '{cpu/mem-loads-aux/S,cpu/mem-loads,ldlat=3/PS}' -- ls
>>>>>>>>>>>>
>>>>>>>>>>>> Opening the event "cpu/mem-loads,ldlat=3/PS" returns EINVAL when
>>>>>>>>>>>> precise_ip == 3. It then sets attr.inherit = false, which triggers a
>>>>>>>>>>>
>>>>>>>>>>> I'm curious about this part. Why the kernel set 'inherit = false'? IOW
>>>>>>>>>>> how did the leader event (mem-loads-aux) succeed with inherit = true
>>>>>>>>>>> then?
>>>>>>>>>>
>>>>>>>>>> Initially, the inherit = true for both the group leader
>>>>>>>>>> (cpu/mem-loads-aux/S) and the event in question (cpu/mem-loads,ldlat=3/PS).
>>>>>>>>>>
>>>>>>>>>> When the second event fails with EINVAL, the current logic calls
>>>>>>>>>> evsel__detect_missing_features() first. Since this is a PERF_SAMPLE_READ
>>>>>>>>>> event, the inherit attribute falls back to false, according to the
>>>>>>>>>> fallback order implemented in evsel__detect_missing_features().
>>>>>>>>>
>>>>>>>>> Right, that means the kernel doesn't support PERF_SAMPLE_READ with
>>>>>>>>> inherit = true. How did the first event succeed to open then?
>>>>>>>>
>>>>>>>> The perf tool sets PERF_SAMPLE_TID for Inherit + PERF_SAMPLE_READ
>>>>>>>> events, as implemented in commit 90035d3cd876 ("tools/perf: Allow
>>>>>>>> inherit + PERF_SAMPLE_READ when opening event").
>>>>>>>>
>>>>>>>> Meanwhile, commit 7e8b255650fc ("perf: Support PERF_SAMPLE_READ with
>>>>>>>> inherit") rejects a perf event if has_inherit_and_sample_read(attr) is
>>>>>>>> true and PERF_SAMPLE_TID is not set in attr->sample_type.
>>>>>>>>
>>>>>>>> Therefore, the first event succeeded, while the one opened in
>>>>>>>> evsel__detect_missing_features() which doesn't have PERF_SAMPLE_TID failed.
>>>>>>>
>>>>>>> Why does the first succeed and the second fail? Don't they have the
>>>>>>> same SAMPLE_READ and SAMPLE_TID + inherit flags?
>>>>>>
>>>>>> Sorry, my previous reply wasn’t entirely accurate. The first event
>>>>>> (cpu/mem-loads-aux/S) succeeds because it’s not a precise event
>>>>>> (precise_ip == 0).
>>>>>
>>>>> I'm not sure how it matters. I've tested the same command line on SPR
>>>>> and got this message. It says it failed to open because of inherit and
>>>>> SAMPE_READ. It didn't have precise_ip too.
>>>>>
>>>>> $ perf record -e cpu/mem-loads-aux/S -vv true |& less
>>>>> ...
>>>>> ------------------------------------------------------------
>>>>> perf_event_attr:
>>>>> type 4 (cpu)
>>>>> size 136
>>>>> config 0x8203 (mem-loads-aux)
>>>>> { sample_period, sample_freq } 4000
>>>>> sample_type IP|TID|TIME|READ|ID|PERIOD
>>>>> read_format ID|LOST
>>>>> disabled 1
>>>>> inherit 1
>>>>> mmap 1
>>>>> comm 1
>>>>> freq 1
>>>>> enable_on_exec 1
>>>>> task 1
>>>>> sample_id_all 1
>>>>> mmap2 1
>>>>> comm_exec 1
>>>>> ksymbol 1
>>>>> bpf_event 1
>>>>> ------------------------------------------------------------
>>>>> sys_perf_event_open: pid 1161023 cpu 0 group_fd -1 flags 0x8
>>>>> sys_perf_event_open failed, error -22
>>>>> Using PERF_SAMPLE_READ / :S modifier is not compatible with inherit, falling back to no-inherit.
>>>>> ...
>>>>>
>>>>> And it fell back to no-inherit and succeeded.
>>>>
>>>> On my SPR, with either kernel 6.18.0-rc4 or the older 6.17.0-rc6, my
>>>> test results are different from yours — I didn’t see any EINVAL, and
>>>> there was no fallback. :)
>>>
>>> Yep, your kernel is recent and has the following commit.
>>>
>>> 7e8b255650fcfa1d0 ("perf: Support PERF_SAMPLE_READ with inherit")
>>>
>>> My kernel is 6.6 and it rejects such a combination. I'll test it on
>>> newer kernels later.
>>>
>>>>
>>>> It’s strange, but even so, since there’s no group leader in this case, I
>>>> assume that when it falls back to non-inherit, it should pass the
>>>> following check.
>>>>
>>>> if (task && group_leader &&
>>>> group_leader->attr.inherit != attr.inherit) {
>>>> err = -EINVAL;
>>>> goto err_task;
>>>> }
>>>>
>>>>> I've also found that it
>>>>> worked even with precise_ip = 3.
>>>>>
>>>>> $ perf record -e cpu/mem-loads-aux/PS -vv true |& less
>>>>> ...
>>>>> sys_perf_event_open: pid 1172834 cpu 0 group_fd -1 flags 0x8
>>>>> sys_perf_event_open failed, error -22
>>>>> Using PERF_SAMPLE_READ / :S modifier is not compatible with inherit, falling back to no-inherit.
>>>>> ------------------------------------------------------------
>>>>> perf_event_attr:
>>>>> type 4 (cpu)
>>>>> size 136
>>>>> config 0x8203 (mem-loads-aux)
>>>>> { sample_period, sample_freq } 4000
>>>>> sample_type IP|TID|TIME|READ|ID|PERIOD
>>>>> read_format ID|LOST
>>>>> disabled 1
>>>>> mmap 1
>>>>> comm 1
>>>>> freq 1
>>>>> enable_on_exec 1
>>>>> task 1
>>>>> precise_ip 3 <<<---- here
>>>>> sample_id_all 1
>>>>> mmap2 1
>>>>> comm_exec 1
>>>>> ksymbol 1
>>>>> bpf_event 1
>>>>> ------------------------------------------------------------
>>>>> sys_perf_event_open: pid 1172834 cpu 0 group_fd -1 flags 0x8 = 4
>>>>> ...
>>>>
>>>> Again, on my machine, I didn’t see EINVAL, and no fallback to
>>>> non-inherit. In my test, glc_get_event_constraints() successfully forces
>>>> this event (config == 0x8203) to fixed counter 0, so there’s no issue here.
>>>
>>> That means your missing_features.inherit_sample_read should not be set.
>>> It's strange you have that with the recent kernels.
>>>
>>> Can you run these commands and show the output here?
>>>
>>> $ perf record -e task-clock:S true
>>> $ perf evlist -v
>>
>> On 6.18.0-rc4:
>>
>> $ perf record -e task-clock:S true
>> [ perf record: Woken up 2 times to write data ]
>> [ perf record: Captured and wrote 0.006 MB perf.data ]
>>
>> $ perf evlist -v
>> task-clock:Su: type: 1 (PERF_TYPE_SOFTWARE), size: 136, config: 0x1
>> (PERF_COUNT_SW_TASK_CLOCK), { sample_period, sample_freq }: 4000,
>> sample_type: IP|TID|TIME|READ|ID|PERIOD, read_format: ID|LOST, disabled:
>> 1, inherit: 1, exclude_kernel: 1, exclude_hv: 1, mmap: 1, comm: 1, freq:
>> 1, enable_on_exec: 1, task: 1, sample_id_all: 1, mmap2: 1, comm_exec: 1,
>> ksymbol: 1, bpf_event: 1, build_id: 1
>
> Thanks for sharing this. Yep, it has the inherit bit.
>
> I think there's a bug in the missing feature test. Indeed, it should
> also have PERF_SAMPLE_TID for the test according to the kernel comment.
>
> /*
> * We do not support PERF_SAMPLE_READ on inherited events unless
> * PERF_SAMPLE_TID is also selected, which allows inherited events to
> * collect per-thread samples.
> * See perf_output_read().
> */
> if (has_inherit_and_sample_read(attr) && !(attr->sample_type & PERF_SAMPLE_TID))
> return ERR_PTR(-EINVAL);
It seems that the purpose of the inherit_sample_read fallback is to
remove the inherit attribute when both PERF_SAMPLE_READ and inherit are
present, but PERF_SAMPLE_TID is not. The new change may not be able to
accomplish this?
>
> I'll send a patch soon.
>
> Thanks,
> Namhyung
>
next prev parent reply other threads:[~2025-11-11 19:11 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-10-22 22:08 [PATCH] perf tools: Refactor precise_ip fallback logic Zide Chen
2025-10-23 16:14 ` Ian Rogers
2025-10-23 22:11 ` Chen, Zide
2025-10-24 2:30 ` Namhyung Kim
2025-10-24 18:03 ` Chen, Zide
2025-10-26 0:42 ` Namhyung Kim
2025-10-27 18:56 ` Chen, Zide
2025-11-04 3:48 ` Namhyung Kim
2025-11-04 19:10 ` Chen, Zide
2025-11-06 18:52 ` Namhyung Kim
2025-11-07 1:23 ` Chen, Zide
2025-11-07 21:42 ` Namhyung Kim
2025-11-07 22:31 ` Chen, Zide
2025-11-11 7:50 ` Namhyung Kim
2025-11-11 19:11 ` Chen, Zide [this message]
2025-11-11 19:34 ` Namhyung Kim
2025-11-11 20:01 ` Chen, Zide
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=42bafd4d-ff6e-4529-95f7-9c4c963fc49c@intel.com \
--to=zide.chen@intel.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=dapeng1.mi@linux.intel.com \
--cc=irogers@google.com \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=thomas.falcon@intel.com \
--cc=xudong.hao@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).