public inbox for linux-s390@vger.kernel.org
 help / color / mirror / Atom feed
From: Ilya Leoshkevich <iii@linux.ibm.com>
To: Thomas Richter <tmricht@linux.ibm.com>,
	linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org,
	linux-perf-users@vger.kernel.org, acme@kernel.org,
	namhyung@kernel.org, andriin@fb.com, irogers@google.com,
	bpf@vger.kernel.org
Cc: agordeev@linux.ibm.com, gor@linux.ibm.com,
	sumanthk@linux.ibm.com, hca@linux.ibm.com, japo@linux.ibm.com,
	Jiri Olsa <jolsa@redhat.com>
Subject: Re: [PATCH v2] perf/s390: Regression: Move uid filtering to BPF filters
Date: Thu, 31 Jul 2025 10:38:27 +0200	[thread overview]
Message-ID: <6018b52aec24000a751165f816dbd4522be8d06d.camel@linux.ibm.com> (raw)
In-Reply-To: <20250728144340.711196-1-tmricht@linux.ibm.com>

On Mon, 2025-07-28 at 16:43 +0200, Thomas Richter wrote:
> V1 --> V2: Added Jiri Olsa's suggestion and introduced
>            member bpf_perf_event_opts::no_ioctl_enable.
> 
> On linux-next
> commit b4c658d4d63d61 ("perf target: Remove uid from target")
> introduces a regression on s390. In fact the regression exists
> on all platforms when the event supports auxiliary data gathering.
> 
> Command
>    # ./perf record -u 0 -aB --synth=no -- ./perf test -w thloop
>    [ perf record: Woken up 1 times to write data ]
>    [ perf record: Captured and wrote 0.011 MB perf.data ]
>    # ./perf report --stats | grep SAMPLE
>    #
> 
> does not generate samples in the perf.data file.
> On x86 command
>   # sudo perf record -e intel_pt// -u 0 ls
> is broken too.
> 
> Looking at the sequence of calls in 'perf record' reveals this
> behavior:
> 1. The event 'cycles' is created and enabled:
>    record__open()
>    +-> evlist__apply_filters()
>        +-> perf_bpf_filter__prepare()
> 	   +-> bpf_program.attach_perf_event()
> 	       +-> bpf_program.attach_perf_event_opts()
> 	           +-> __GI___ioctl(..., PERF_EVENT_IOC_ENABLE, ...)
>    The event 'cycles' is enabled and active now. However the event's
>    ring-buffer to store the samples generated by hardware is not
>    allocated yet. This happens now after enabling the event:
> 
> 2. The event's fd is mmap() to create the ring buffer:
>    record__open()
>    +-> record__mmap()
>        +-> record__mmap_evlist()
> 	   +-> evlist__mmap_ex()
> 	       +-> perf_evlist__mmap_ops()
> 	           +-> mmap_per_cpu()
> 	               +-> mmap_per_evsel()
> 	                   +-> mmap__mmap()
> 	                       +-> perf_mmap__mmap()
> 	                           +-> mmap()
> 
>    This allocates the ring-buffer for the event 'cycles'.  With
> mmap()
>    the kernel creates the ring buffer:
> 
>    perf_mmap(): kernel function to create the event's ring
>    |            buffer to save the sampled data.
>    |
>    +-> ring_buffer_attach(): Allocates memory for ring buffer.
>        |        The PMU has auxiliary data setup function. The
>        |        has_aux(event) condition is true and the PMU's
>        |        stop() is called to stop sampling. It is not
>        |        restarted:
>        |        if (has_aux(event))
>        |                perf_event_stop(event, 0);
>        |
>        +-> cpumsf_pmu_stop():
> 
>    Hardware sampling is stopped. No samples are generated and saved
>    anymore.
> 
> 3. After the event 'cycles' has been mapped, the event is enabled a
>    second time in:
>    __cmd_record()
>    +-> evlist__enable()
>        +-> __evlist__enable()
> 	   +-> evsel__enable_cpu()
> 	       +-> perf_evsel__enable_cpu()
> 	           +-> perf_evsel__run_ioctl()
> 	               +-> perf_evsel__ioctl()
> 	                   +-> __GI___ioctl(.,
> PERF_EVENT_IOC_ENABLE, .)
>    The second
>       ioctl(fd, PERF_EVENT_IOC_ENABLE, 0);
>    is just a NOP in this case. The first invocation in (1.) sets the
>    event::state to PERF_EVENT_STATE_ACTIVE. The kernel functions
>    perf_ioctl()
>    +-> _perf_ioctl()
>        +-> _perf_event_enable()
>            +-> __perf_event_enable() returns immediately because
> 	              event::state is already set to
> 		      PERF_EVENT_STATE_ACTIVE.
> 
> This happens on s390, because the event 'cycles' offers the
> possibility
> to save auxilary data. The PMU call backs setup_aux() and
> free_aux() are defined. Without both call back functions,
> cpumsf_pmu_stop() is not invoked and sampling continues.
> 
> To remedy this, remove the first invocation of
>    ioctl(..., PERF_EVENT_IOC_ENABLE, ...).
> in step (1.) Create the event in step (1.) and enable it in step (3.)
> after the ring buffer has been mapped.
> Make the change backward compatible and introduce a new structure
> member bpf_perf_event_opts::no_ioctl_enable. It defaults to false and
> only
> bpf_program__attach_perf_event() sets it to true. This way only
> perf tool invocation do not enable the sampling event.
> 
> Output after:
>  # ./perf record -aB --synth=no -u 0 -- ./perf test -w thloop 2
>  [ perf record: Woken up 3 times to write data ]
>  [ perf record: Captured and wrote 0.876 MB perf.data ]
>  # ./perf  report --stats | grep SAMPLE
>               SAMPLE events:      16200  (99.5%)
>               SAMPLE events:      16200
>  #
> 
> The software event succeeded before and after the patch:
>  # ./perf record -e cpu-clock -aB --synth=no -u 0 -- \
> 					  ./perf test -w thloop 2
>  [ perf record: Woken up 7 times to write data ]
>  [ perf record: Captured and wrote 2.870 MB perf.data ]
>  # ./perf  report --stats | grep SAMPLE
>               SAMPLE events:      53506  (99.8%)
>               SAMPLE events:      53506
>  #
> 
> Fixes: 63f2f5ee856ba ("libbpf: add ability to attach/detach BPF
> program to perf event")
> To: Andrii Nakryiko <andriin@fb.com>
> To: Ian Rogers <irogers@google.com>
> To: Ilya Leoshkevich <iii@linux.ibm.com>
> Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
> Suggested-by: Jiri Olsa <jolsa@redhat.com>
> ---
>  tools/lib/bpf/libbpf.c | 19 +++++++++++++------
>  tools/lib/bpf/libbpf.h |  3 ++-
>  2 files changed, 15 insertions(+), 7 deletions(-)

What do you think about rather calling the new field ioctl_enable?
So that we don't get double negations in the API users and
implementation - they are sometimes unnecessarily confusing.

I also think enablement should be the default in
bpf_program__attach_perf_event(), and perf should now call
bpf_program__attach_perf_event_opts() instead.

Based on your request in v1, I can offer to take over the patch and
send a v3 with the changes I suggested above.

  reply	other threads:[~2025-07-31  8:38 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-07-28 14:43 [PATCH v2] perf/s390: Regression: Move uid filtering to BPF filters Thomas Richter
2025-07-31  8:38 ` Ilya Leoshkevich [this message]
2025-07-31  9:31   ` Thomas Richter

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6018b52aec24000a751165f816dbd4522be8d06d.camel@linux.ibm.com \
    --to=iii@linux.ibm.com \
    --cc=acme@kernel.org \
    --cc=agordeev@linux.ibm.com \
    --cc=andriin@fb.com \
    --cc=bpf@vger.kernel.org \
    --cc=gor@linux.ibm.com \
    --cc=hca@linux.ibm.com \
    --cc=irogers@google.com \
    --cc=japo@linux.ibm.com \
    --cc=jolsa@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=linux-s390@vger.kernel.org \
    --cc=namhyung@kernel.org \
    --cc=sumanthk@linux.ibm.com \
    --cc=tmricht@linux.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox