* Strange behavior when event installed systemwide multiple times
@ 2022-06-15 13:30 Thomas Richter
2022-06-15 17:12 ` Ian Rogers
0 siblings, 1 reply; 2+ messages in thread
From: Thomas Richter @ 2022-06-15 13:30 UTC (permalink / raw)
To: linux-perf-use.
I have a question regarding the behavior when 2 events are installed
system-wide on the same CPU, for example:
# perf stat -e cpu-cycles -a -C0 -- sleep 5 &
# sleep 2
# perf stat -e instructions -a -C0 -- true
I wonder why at the 2nd perf invocation the event cpu-cycles is deleted
from CPU 0 just to be re-added immediately. I understand this is necessary
for events attached to a process. This is also done when the event is
installed system wide, and is not connected to a process.
Looking at the trace
# tracer: function
#
# _-----=> irqs-off
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / delay
# TASK-PID CPU# |||| TIMESTAMP FUNCTION
# | | | |||| | |
perf-19574 [000] d... 10588.974577: x86_pmu_add <-event_sched_in.isra.130
---> 1st perf runs and 1st event installed
perf-19574 [000] d... 10588.974582: x86_pmu_start <-x86_pmu_enable
perf-19579 [000] d... 10590.974582: x86_pmu_del <-event_sched_out.isra.116
perf-19579 [000] d... 10590.974583: x86_pmu_stop <-x86_pmu_del
---> 2nd perf runs and 1st event is deleted
perf-19579 [000] d... 10590.974585: x86_pmu_add <-event_sched_in.isra.130
---> 1st event cpu-cycles re-added
perf-19579 [000] d... 10590.974586: x86_pmu_add <-event_sched_in.isra.130
perf-19579 [000] d... 10590.974586: x86_pmu_start <-x86_pmu_enable
perf-19579 [000] d... 10590.974587: x86_pmu_start <-x86_pmu_enable
perf-19579 [000] d... 10590.975361: x86_pmu_del <-event_sched_out.isra.116
---> 2nd perf stops and 2nd event removed
perf-19579 [000] d... 10590.975361: x86_pmu_stop <-x86_pmu_del
perf-19574 [000] d... 10593.975959: x86_pmu_del <-event_sched_out.isra.116
perf-19574 [000] d... 10593.975961: x86_pmu_stop <-x86_pmu_del
---> 1st perf stops and 1st event removed.
Is this intended behavior?
Thanks a lot.
--
Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany
--
Vorsitzender des Aufsichtsrats: Gregor Pillen
Geschäftsführung: David Faller
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
^ permalink raw reply [flat|nested] 2+ messages in thread* Re: Strange behavior when event installed systemwide multiple times
2022-06-15 13:30 Strange behavior when event installed systemwide multiple times Thomas Richter
@ 2022-06-15 17:12 ` Ian Rogers
0 siblings, 0 replies; 2+ messages in thread
From: Ian Rogers @ 2022-06-15 17:12 UTC (permalink / raw)
To: Thomas Richter; +Cc: linux-perf-use.
On Wed, Jun 15, 2022 at 6:30 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
>
>
> I have a question regarding the behavior when 2 events are installed
> system-wide on the same CPU, for example:
>
> # perf stat -e cpu-cycles -a -C0 -- sleep 5 &
> # sleep 2
> # perf stat -e instructions -a -C0 -- true
>
> I wonder why at the 2nd perf invocation the event cpu-cycles is deleted
> from CPU 0 just to be re-added immediately. I understand this is necessary
> for events attached to a process. This is also done when the event is
> installed system wide, and is not connected to a process.
>
> Looking at the trace
> # tracer: function
> #
> # _-----=> irqs-off
> # / _----=> need-resched
> # | / _---=> hardirq/softirq
> # || / _--=> preempt-depth
> # ||| / delay
> # TASK-PID CPU# |||| TIMESTAMP FUNCTION
> # | | | |||| | |
> perf-19574 [000] d... 10588.974577: x86_pmu_add <-event_sched_in.isra.130
> ---> 1st perf runs and 1st event installed
> perf-19574 [000] d... 10588.974582: x86_pmu_start <-x86_pmu_enable
> perf-19579 [000] d... 10590.974582: x86_pmu_del <-event_sched_out.isra.116
> perf-19579 [000] d... 10590.974583: x86_pmu_stop <-x86_pmu_del
> ---> 2nd perf runs and 1st event is deleted
> perf-19579 [000] d... 10590.974585: x86_pmu_add <-event_sched_in.isra.130
> ---> 1st event cpu-cycles re-added
> perf-19579 [000] d... 10590.974586: x86_pmu_add <-event_sched_in.isra.130
> perf-19579 [000] d... 10590.974586: x86_pmu_start <-x86_pmu_enable
> perf-19579 [000] d... 10590.974587: x86_pmu_start <-x86_pmu_enable
> perf-19579 [000] d... 10590.975361: x86_pmu_del <-event_sched_out.isra.116
> ---> 2nd perf stops and 2nd event removed
> perf-19579 [000] d... 10590.975361: x86_pmu_stop <-x86_pmu_del
> perf-19574 [000] d... 10593.975959: x86_pmu_del <-event_sched_out.isra.116
> perf-19574 [000] d... 10593.975961: x86_pmu_stop <-x86_pmu_del
> ---> 1st perf stops and 1st event removed.
>
> Is this intended behavior?
It is intended behavior to my knowledge. Every PMU has a context with
per task and per CPU events that may be either pinned or flexible.
Events are scheduled in the order they were opened, when multiplexing
happens we play with that order removing the front event and putting
it at the back. As the scheduling decision is complex the code
schedules everything out and then schedules everything back in, but as
you are pointing out this creates holes in sampling. I believe there
have been patches to make it so we don't do that for say system wide
events, but the change needed to be robust to failing to schedule and
I'm not sure it landed.
Thanks,
Ian
> Thanks a lot.
> --
> Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany
> --
> Vorsitzender des Aufsichtsrats: Gregor Pillen
> Geschäftsführung: David Faller
> Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2022-06-15 17:12 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-06-15 13:30 Strange behavior when event installed systemwide multiple times Thomas Richter
2022-06-15 17:12 ` Ian Rogers
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).