Re: [PATCH v3 0/6] perf auxtrace: Support AUX pause and resume with BPF

[Arnaldo Carvalho de Melo <acme@kernel.org>]

On Fri, Aug 08, 2025 at 12:47:34PM +0100, Leo Yan wrote:
> On Tue, Aug 05, 2025 at 10:16:29PM +0300, Adrian Hunter wrote:
> > On 30/07/2025 21:26, Leo Yan wrote:
> > > On Mon, Jul 28, 2025 at 08:02:51PM +0300, Adrian Hunter wrote:
> > >> On 25/07/2025 12:59, Leo Yan wrote:
> > >>> This series extends Perf for fine-grained tracing by using BPF program
> > >>> to pause and resume AUX tracing. The BPF program can be attached to
> > >>> tracepoints (including ftrace tracepoints and dynamic tracepoints, like
> > >>> kprobe, kretprobe, uprobe and uretprobe).

> > >> Using eBPF to pause/resume AUX tracing seems like a great idea.

> > >> AFAICT with this patch set, there is just support for pause/resume
> > >> much like what could be done directly without eBPF, so I wonder if you
> > >> could share a bit more on how you see this evolving, and what your
> > >> future plans are?

> > > IIUC, here you mean the tool can use `perf probe` to firstly create
> > > probes, then enable tracepoints as PMU event for AUX pause and resume.

> > Yes, like:

> > $ sudo perf probe 'do_sys_openat2 how->flags how->mode'
> > Added new event:
> >   probe:do_sys_openat2 (on do_sys_openat2 with flags=how->flags mode=how->mode)

> > You can now use it in all perf tools, such as:

> >         perf record -e probe:do_sys_openat2 -aR sleep 1

> > $ sudo perf probe do_sys_openat2%return
> > Added new event:
> >   probe:do_sys_openat2__return (on do_sys_openat2%return)

> > You can now use it in all perf tools, such as:

> >         perf record -e probe:do_sys_openat2__return -aR sleep 1

> > $ sudo perf record --kcore -e intel_pt/aux-action=start-paused/k -e probe:do_sys_openat2/aux-action=resume/ --filter='flags==0x98800' -e probe:do_sys_openat2__return/aux-action=pause/ -- ls

> Thanks a lot for sharing the commands. I was able to replicate them
> using CoreSight.

> Given that we can achieve the same result without using BPF, I am not
> sure how useful this series is. It may give us a base for exploring
> profiling that combines AUX trace and BPF, but I am fine with holding
> on until we have clear requirements for it.

> I would get suggestion from you and maintainers before proceeding
> further.

Maybe retrofit this for starting stopping profiling non HW tracing
sections?

We have now:

⬢ [acme@toolbx perf-tools-next]$ perf record -h switch

 Usage: perf record [<options>] [<command>]
    or: perf record [<options>] -- <command> [<options>]

        --switch-events   Record context switch events
        --switch-max-files <n>
                          Limit number of switch output generated files
        --switch-output[=<signal or size[BKMG] or time[smhd]>]
                          Switch output when receiving SIGUSR2 (signal) or cross a size or time threshold
        --switch-output-event <switch output event>
                          switch output event selector. use 'perf list' to list available events

⬢ [acme@toolbx perf-tools-next]$

That will dump a snapshot when some event takes place, but that is done
with a sideband thread, from 'man perf-record':

--switch-output-event::
Events that will cause the switch of the perf.data file, auto-selecting
--switch-output=signal, the results are similar as internally the side band
thread will also send a SIGUSR2 to the main one.

Uses the same syntax as --event, it will just not be recorded, serving only to
switch the perf.data file as soon as the --switch-output event is processed by
a separate sideband thread.

This sideband thread is also used to other purposes, like processing the
PERF_RECORD_BPF_EVENT records as they happen, asking the kernel for extra BPF
information, etc.

----------------------

And in perf-report we have:

----

--switch-on EVENT_NAME::
        Only consider events after this event is found.

        This may be interesting to measure a workload only after some initialization
        phase is over, i.e. insert a perf probe at that point and then using this
        option with that probe.

--switch-off EVENT_NAME::
        Stop considering events after this event is found.

--show-on-off-events::
        Show the --switch-on/off events too. This has no effect in 'perf report' now
        but probably we'll make the default not to show the switch-on/off events
        on the --group mode and if there is only one event besides the off/on ones,
        go straight to the histogram browser, just like 'perf report' with no events
        explicitly specified does.

----

If we had it in 'perf record' then we would have reduced perf.data
files.

I.e. we would have something like '-e {cycles,instructions}/action=start-paused/k' -e probe:do_sys_openat2/action=resume/ --filter='flags==0x98800' -e probe:do_sys_openat2__return/action=pause/

- Arnaldo