* [PATCH v5 1/9] arm64: pmu: Add function implementation to update event index in userpage
2021-01-14 2:05 [PATCH v5 0/9] libperf and arm64 userspace counter access support Rob Herring
@ 2021-01-14 2:05 ` Rob Herring
2021-01-14 2:05 ` [PATCH v5 2/9] arm64: perf: Enable PMU counter direct access for perf event Rob Herring
` (8 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Rob Herring @ 2021-01-14 2:05 UTC (permalink / raw)
To: Will Deacon, Catalin Marinas, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Jiri Olsa, Mark Rutland
Cc: Ian Rogers, Alexander Shishkin, linux-kernel,
honnappa.nagarahalli, Raphael Gault, Jonathan Cameron,
Namhyung Kim, Itaru Kitayama, linux-arm-kernel
From: Raphael Gault <raphael.gault@arm.com>
In order to be able to access the counter directly for userspace,
we need to provide the index of the counter using the userpage.
We thus need to override the event_idx function to retrieve and
convert the perf_event index to armv8 hardware index.
Since the arm_pmu driver can be used by any implementation, even
if not armv8, two components play a role into making sure the
behaviour is correct and consistent with the PMU capabilities:
* the ARMPMU_EL0_RD_CNTR flag which denotes the capability to access
counter from userspace.
* the event_idx call back, which is implemented and initialized by
the PMU implementation: if no callback is provided, the default
behaviour applies, returning 0 as index value.
Signed-off-by: Raphael Gault <raphael.gault@arm.com>
Signed-off-by: Rob Herring <robh@kernel.org>
---
arch/arm64/kernel/perf_event.c | 21 +++++++++++++++++++++
include/linux/perf/arm_pmu.h | 2 ++
2 files changed, 23 insertions(+)
diff --git a/arch/arm64/kernel/perf_event.c b/arch/arm64/kernel/perf_event.c
index 38bb07eff872..21f6f4cdd05f 100644
--- a/arch/arm64/kernel/perf_event.c
+++ b/arch/arm64/kernel/perf_event.c
@@ -873,6 +873,22 @@ static void armv8pmu_clear_event_idx(struct pmu_hw_events *cpuc,
clear_bit(idx - 1, cpuc->used_mask);
}
+static int armv8pmu_access_event_idx(struct perf_event *event)
+{
+ if (!(event->hw.flags & ARMPMU_EL0_RD_CNTR))
+ return 0;
+
+ /*
+ * We remap the cycle counter index to 32 to
+ * match the offset applied to the rest of
+ * the counter indices.
+ */
+ if (event->hw.idx == ARMV8_IDX_CYCLE_COUNTER)
+ return 32;
+
+ return event->hw.idx;
+}
+
/*
* Add an event filter to a given event.
*/
@@ -969,6 +985,9 @@ static int __armv8_pmuv3_map_event(struct perf_event *event,
if (armv8pmu_event_is_64bit(event))
event->hw.flags |= ARMPMU_EVT_64BIT;
+ if (!armv8pmu_event_is_chained(event))
+ event->hw.flags |= ARMPMU_EL0_RD_CNTR;
+
/* Only expose micro/arch events supported by this PMU */
if ((hw_event_id > 0) && (hw_event_id < ARMV8_PMUV3_MAX_COMMON_EVENTS)
&& test_bit(hw_event_id, armpmu->pmceid_bitmap)) {
@@ -1100,6 +1119,8 @@ static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name,
cpu_pmu->set_event_filter = armv8pmu_set_event_filter;
cpu_pmu->filter_match = armv8pmu_filter_match;
+ cpu_pmu->pmu.event_idx = armv8pmu_access_event_idx;
+
cpu_pmu->name = name;
cpu_pmu->map_event = map_event;
cpu_pmu->attr_groups[ARMPMU_ATTR_GROUP_EVENTS] = events ?
diff --git a/include/linux/perf/arm_pmu.h b/include/linux/perf/arm_pmu.h
index bf7966776c55..bb69d14eaa82 100644
--- a/include/linux/perf/arm_pmu.h
+++ b/include/linux/perf/arm_pmu.h
@@ -26,6 +26,8 @@
*/
/* Event uses a 64bit counter */
#define ARMPMU_EVT_64BIT 1
+/* Allow access to hardware counter from userspace */
+#define ARMPMU_EL0_RD_CNTR 2
#define HW_OP_UNSUPPORTED 0xFFFF
#define C(_x) PERF_COUNT_HW_CACHE_##_x
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH v5 2/9] arm64: perf: Enable PMU counter direct access for perf event
2021-01-14 2:05 [PATCH v5 0/9] libperf and arm64 userspace counter access support Rob Herring
2021-01-14 2:05 ` [PATCH v5 1/9] arm64: pmu: Add function implementation to update event index in userpage Rob Herring
@ 2021-01-14 2:05 ` Rob Herring
2021-01-14 2:05 ` [PATCH v5 3/9] tools/include: Add an initial math64.h Rob Herring
` (7 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Rob Herring @ 2021-01-14 2:05 UTC (permalink / raw)
To: Will Deacon, Catalin Marinas, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Jiri Olsa, Mark Rutland
Cc: Ian Rogers, Alexander Shishkin, linux-kernel,
honnappa.nagarahalli, Raphael Gault, Jonathan Cameron,
Namhyung Kim, Itaru Kitayama, linux-arm-kernel
From: Raphael Gault <raphael.gault@arm.com>
Keep track of event opened with direct access to the hardware counters
and modify permissions while they are open.
The strategy used here is the same which x86 uses: every time an event
is mapped, the permissions are set if required. The atomic field added
in the mm_context helps keep track of the different event opened and
de-activate the permissions when all are unmapped.
We also need to update the permissions in the context switch code so
that tasks keep the right permissions.
Signed-off-by: Raphael Gault <raphael.gault@arm.com>
Signed-off-by: Rob Herring <robh@kernel.org>
---
I'm not completely sure if using current->active_mm in an IPI is okay?
It seems to work in my testing.
Peter Z says (event->oncpu == smp_processor_id()) in the user page
update is always true, but my testing says otherwise[1].
v5:
- Only set cap_user_rdpmc if event is on current cpu
- Limit enabling/disabling access to CPUs associated with the PMU
(supported_cpus) and with the mm_struct matching current->active_mm.
v2:
- Move mapped/unmapped into arm64 code. Fixes arm32.
- Rebase on cap_user_time_short changes
Changes from Raphael's v4:
- Drop homogeneous check
- Disable access for chained counters
- Set pmc_width in user page
[1] https://lore.kernel.org/lkml/CAL_JsqK+eKef5NaVnBfARCjRE3MYhfBfe54F9YHKbsTnWqLmLw@mail.gmail.com/
---
arch/arm64/include/asm/mmu.h | 5 +++
arch/arm64/include/asm/mmu_context.h | 2 ++
arch/arm64/include/asm/perf_event.h | 14 +++++++++
arch/arm64/kernel/perf_event.c | 47 ++++++++++++++++++++++++++++
4 files changed, 68 insertions(+)
diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h
index 75beffe2ee8a..ee08447455da 100644
--- a/arch/arm64/include/asm/mmu.h
+++ b/arch/arm64/include/asm/mmu.h
@@ -18,6 +18,11 @@
typedef struct {
atomic64_t id;
+ /*
+ * non-zero if userspace have access to hardware
+ * counters directly.
+ */
+ atomic_t pmu_direct_access;
#ifdef CONFIG_COMPAT
void *sigpage;
#endif
diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h
index 0b3079fd28eb..25aee6ab4127 100644
--- a/arch/arm64/include/asm/mmu_context.h
+++ b/arch/arm64/include/asm/mmu_context.h
@@ -21,6 +21,7 @@
#include <asm/proc-fns.h>
#include <asm-generic/mm_hooks.h>
#include <asm/cputype.h>
+#include <asm/perf_event.h>
#include <asm/sysreg.h>
#include <asm/tlbflush.h>
@@ -231,6 +232,7 @@ static inline void __switch_mm(struct mm_struct *next)
}
check_and_switch_context(next);
+ perf_switch_user_access(next);
}
static inline void
diff --git a/arch/arm64/include/asm/perf_event.h b/arch/arm64/include/asm/perf_event.h
index 60731f602d3e..112f3f63b79e 100644
--- a/arch/arm64/include/asm/perf_event.h
+++ b/arch/arm64/include/asm/perf_event.h
@@ -8,6 +8,7 @@
#include <asm/stack_pointer.h>
#include <asm/ptrace.h>
+#include <linux/mm_types.h>
#define ARMV8_PMU_MAX_COUNTERS 32
#define ARMV8_PMU_COUNTER_MASK (ARMV8_PMU_MAX_COUNTERS - 1)
@@ -254,4 +255,17 @@ extern unsigned long perf_misc_flags(struct pt_regs *regs);
(regs)->pstate = PSR_MODE_EL1h; \
}
+static inline void perf_switch_user_access(struct mm_struct *mm)
+{
+ if (!IS_ENABLED(CONFIG_PERF_EVENTS))
+ return;
+
+ if (atomic_read(&mm->context.pmu_direct_access)) {
+ write_sysreg(ARMV8_PMU_USERENR_ER|ARMV8_PMU_USERENR_CR,
+ pmuserenr_el0);
+ } else {
+ write_sysreg(0, pmuserenr_el0);
+ }
+}
+
#endif
diff --git a/arch/arm64/kernel/perf_event.c b/arch/arm64/kernel/perf_event.c
index 21f6f4cdd05f..38b1cdc7bbe2 100644
--- a/arch/arm64/kernel/perf_event.c
+++ b/arch/arm64/kernel/perf_event.c
@@ -889,6 +889,46 @@ static int armv8pmu_access_event_idx(struct perf_event *event)
return event->hw.idx;
}
+static void refresh_pmuserenr(void *mm)
+{
+ if (mm == current->active_mm)
+ perf_switch_user_access(mm);
+}
+
+static void armv8pmu_event_mapped(struct perf_event *event, struct mm_struct *mm)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
+
+ if (!(event->hw.flags & ARMPMU_EL0_RD_CNTR))
+ return;
+
+ /*
+ * This function relies on not being called concurrently in two
+ * tasks in the same mm. Otherwise one task could observe
+ * pmu_direct_access > 1 and return all the way back to
+ * userspace with user access disabled while another task is still
+ * doing on_each_cpu_mask() to enable user access.
+ *
+ * For now, this can't happen because all callers hold mmap_lock
+ * for write. If this changes, we'll need a different solution.
+ */
+ lockdep_assert_held_write(&mm->mmap_lock);
+
+ if (atomic_inc_return(&mm->context.pmu_direct_access) == 1)
+ on_each_cpu_mask(&armpmu->supported_cpus, refresh_pmuserenr, mm, 1);
+}
+
+static void armv8pmu_event_unmapped(struct perf_event *event, struct mm_struct *mm)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
+
+ if (!(event->hw.flags & ARMPMU_EL0_RD_CNTR))
+ return;
+
+ if (atomic_dec_and_test(&mm->context.pmu_direct_access))
+ on_each_cpu_mask(&armpmu->supported_cpus, refresh_pmuserenr, mm, 1);
+}
+
/*
* Add an event filter to a given event.
*/
@@ -1120,6 +1160,8 @@ static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name,
cpu_pmu->filter_match = armv8pmu_filter_match;
cpu_pmu->pmu.event_idx = armv8pmu_access_event_idx;
+ cpu_pmu->pmu.event_mapped = armv8pmu_event_mapped;
+ cpu_pmu->pmu.event_unmapped = armv8pmu_event_unmapped;
cpu_pmu->name = name;
cpu_pmu->map_event = map_event;
@@ -1299,6 +1341,11 @@ void arch_perf_update_userpage(struct perf_event *event,
userpg->cap_user_time = 0;
userpg->cap_user_time_zero = 0;
userpg->cap_user_time_short = 0;
+ userpg->cap_user_rdpmc = !!(event->hw.flags & ARMPMU_EL0_RD_CNTR) &&
+ (event->oncpu == smp_processor_id());
+
+ if (userpg->cap_user_rdpmc)
+ userpg->pmc_width = armv8pmu_event_is_64bit(event) ? 64 : 32;
do {
rd = sched_clock_read_begin(&seq);
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH v5 3/9] tools/include: Add an initial math64.h
2021-01-14 2:05 [PATCH v5 0/9] libperf and arm64 userspace counter access support Rob Herring
2021-01-14 2:05 ` [PATCH v5 1/9] arm64: pmu: Add function implementation to update event index in userpage Rob Herring
2021-01-14 2:05 ` [PATCH v5 2/9] arm64: perf: Enable PMU counter direct access for perf event Rob Herring
@ 2021-01-14 2:05 ` Rob Herring
2021-01-14 2:06 ` [PATCH v5 4/9] libperf: Add evsel mmap support Rob Herring
` (6 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Rob Herring @ 2021-01-14 2:05 UTC (permalink / raw)
To: Will Deacon, Catalin Marinas, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Jiri Olsa, Mark Rutland
Cc: Ian Rogers, Alexander Shishkin, linux-kernel,
honnappa.nagarahalli, Raphael Gault, Jonathan Cameron,
Namhyung Kim, Itaru Kitayama, linux-arm-kernel
Add an initial math64.h similar to linux/math64.h with functions
mul_u64_u64_div64() and mul_u64_u32_shr(). This isn't a direct copy of
include/linux/math64.h as that doesn't define mul_u64_u64_div64().
Implementation was written by Peter Zilkstra based on linux/math64.h
and div64.h[1]. The original implementation was not optimal on arm64 as
__int128 division is not optimal with a call out to __udivti3, so I
dropped the __int128 variant of mul_u64_u64_div64().
[1] https://lore.kernel.org/lkml/20200322101848.GF2452@worktop.programming.kicks-ass.net/
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Rob Herring <robh@kernel.org>
---
tools/include/linux/math64.h | 75 ++++++++++++++++++++++++++++++++++++
1 file changed, 75 insertions(+)
create mode 100644 tools/include/linux/math64.h
diff --git a/tools/include/linux/math64.h b/tools/include/linux/math64.h
new file mode 100644
index 000000000000..4ad45d5943dc
--- /dev/null
+++ b/tools/include/linux/math64.h
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_MATH64_H
+#define _LINUX_MATH64_H
+
+#include <linux/types.h>
+
+#ifdef __x86_64__
+static inline u64 mul_u64_u64_div64(u64 a, u64 b, u64 c)
+{
+ u64 q;
+
+ asm ("mulq %2; divq %3" : "=a" (q)
+ : "a" (a), "rm" (b), "rm" (c)
+ : "rdx");
+
+ return q;
+}
+#define mul_u64_u64_div64 mul_u64_u64_div64
+#endif
+
+#ifdef __SIZEOF_INT128__
+static inline u64 mul_u64_u32_shr(u64 a, u32 b, unsigned int shift)
+{
+ return (u64)(((unsigned __int128)a * b) >> shift);
+}
+
+#else
+
+#ifdef __i386__
+static inline u64 mul_u32_u32(u32 a, u32 b)
+{
+ u32 high, low;
+
+ asm ("mull %[b]" : "=a" (low), "=d" (high)
+ : [a] "a" (a), [b] "rm" (b) );
+
+ return low | ((u64)high) << 32;
+}
+#else
+static inline u64 mul_u32_u32(u32 a, u32 b)
+{
+ return (u64)a * b;
+}
+#endif
+
+static inline u64 mul_u64_u32_shr(u64 a, u32 b, unsigned int shift)
+{
+ u32 ah, al;
+ u64 ret;
+
+ al = a;
+ ah = a >> 32;
+
+ ret = mul_u32_u32(al, b) >> shift;
+ if (ah)
+ ret += mul_u32_u32(ah, b) << (32 - shift);
+
+ return ret;
+}
+
+#endif /* __SIZEOF_INT128__ */
+
+#ifndef mul_u64_u64_div64
+static inline u64 mul_u64_u64_div64(u64 a, u64 b, u64 c)
+{
+ u64 quot, rem;
+
+ quot = a / c;
+ rem = a % c;
+
+ return quot * b + (rem * b) / c;
+}
+#endif
+
+#endif /* _LINUX_MATH64_H */
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH v5 4/9] libperf: Add evsel mmap support
2021-01-14 2:05 [PATCH v5 0/9] libperf and arm64 userspace counter access support Rob Herring
` (2 preceding siblings ...)
2021-01-14 2:05 ` [PATCH v5 3/9] tools/include: Add an initial math64.h Rob Herring
@ 2021-01-14 2:06 ` Rob Herring
2021-01-23 22:42 ` Jiri Olsa
2021-01-14 2:06 ` [PATCH v5 5/9] libperf: tests: Add support for verbose printing Rob Herring
` (5 subsequent siblings)
9 siblings, 1 reply; 12+ messages in thread
From: Rob Herring @ 2021-01-14 2:06 UTC (permalink / raw)
To: Will Deacon, Catalin Marinas, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Jiri Olsa, Mark Rutland
Cc: Ian Rogers, Alexander Shishkin, linux-kernel,
honnappa.nagarahalli, Raphael Gault, Jonathan Cameron,
Namhyung Kim, Itaru Kitayama, linux-arm-kernel
In order to support usersapce access, an event must be mmapped. While
there's already mmap support for evlist, the usecase is a bit different
than the self monitoring with userspace access. So let's add a new
perf_evsel__mmap() function to mmap an evsel. This allows implementing
userspace access as a fastpath for perf_evsel__read().
The mmapped address is returned by perf_evsel__mmap_base() which
primarily for users/tests to check if userspace access is enabled.
Signed-off-by: Rob Herring <robh@kernel.org>
---
v5:
- Create an mmap for every underlying event opened. Due to this, we
need a different way to get the mmap ptr, so perf_evsel__mmap_base()
is introduced.
v4:
- Change perf_evsel__mmap size to pages instead of bytes
v3:
- New patch split out from user access patch
---
tools/lib/perf/Documentation/libperf.txt | 2 +
tools/lib/perf/evsel.c | 47 +++++++++++++++++++++++-
tools/lib/perf/include/internal/evsel.h | 2 +
tools/lib/perf/include/perf/evsel.h | 2 +
tools/lib/perf/libperf.map | 2 +
5 files changed, 53 insertions(+), 2 deletions(-)
diff --git a/tools/lib/perf/Documentation/libperf.txt b/tools/lib/perf/Documentation/libperf.txt
index 0c74c30ed23a..a2c73df191ca 100644
--- a/tools/lib/perf/Documentation/libperf.txt
+++ b/tools/lib/perf/Documentation/libperf.txt
@@ -136,6 +136,8 @@ SYNOPSIS
struct perf_thread_map *threads);
void perf_evsel__close(struct perf_evsel *evsel);
void perf_evsel__close_cpu(struct perf_evsel *evsel, int cpu);
+ int perf_evsel__mmap(struct perf_evsel *evsel, int pages);
+ void *perf_evsel__mmap_base(struct perf_evsel *evsel, int cpu, int thread);
int perf_evsel__read(struct perf_evsel *evsel, int cpu, int thread,
struct perf_counts_values *count);
int perf_evsel__enable(struct perf_evsel *evsel);
diff --git a/tools/lib/perf/evsel.c b/tools/lib/perf/evsel.c
index 4dc06289f4c7..0b5bdf4badae 100644
--- a/tools/lib/perf/evsel.c
+++ b/tools/lib/perf/evsel.c
@@ -11,10 +11,12 @@
#include <stdlib.h>
#include <internal/xyarray.h>
#include <internal/cpumap.h>
+#include <internal/mmap.h>
#include <internal/threadmap.h>
#include <internal/lib.h>
#include <linux/string.h>
#include <sys/ioctl.h>
+#include <sys/mman.h>
void perf_evsel__init(struct perf_evsel *evsel, struct perf_event_attr *attr)
{
@@ -37,11 +39,17 @@ void perf_evsel__delete(struct perf_evsel *evsel)
free(evsel);
}
-#define FD(e, x, y) (*(int *) xyarray__entry(e->fd, x, y))
+struct evsel_fd {
+ int fd;
+ struct perf_mmap mmap;
+};
+
+#define FD(e, x, y) ((struct evsel_fd *) xyarray__entry(e->fd, x, y))->fd
+#define MMAP(e, x, y) (&(((struct evsel_fd *) xyarray__entry(e->fd, x, y))->mmap))
int perf_evsel__alloc_fd(struct perf_evsel *evsel, int ncpus, int nthreads)
{
- evsel->fd = xyarray__new(ncpus, nthreads, sizeof(int));
+ evsel->fd = xyarray__new(ncpus, nthreads, sizeof(struct evsel_fd));
if (evsel->fd) {
int cpu, thread;
@@ -156,6 +164,41 @@ void perf_evsel__close_cpu(struct perf_evsel *evsel, int cpu)
perf_evsel__close_fd_cpu(evsel, cpu);
}
+int perf_evsel__mmap(struct perf_evsel *evsel, int pages)
+{
+ int ret, cpu, thread;
+ struct perf_mmap_param mp = {
+ .prot = PROT_READ | PROT_WRITE,
+ .mask = (pages * page_size) - 1,
+ };
+
+ for (cpu = 0; cpu < xyarray__max_x(evsel->fd); cpu++) {
+ for (thread = 0; thread < xyarray__max_y(evsel->fd); thread++) {
+ int fd = FD(evsel, cpu, thread);
+ struct perf_mmap *map = MMAP(evsel, cpu, thread);
+
+ if (fd < 0)
+ continue;
+
+ perf_mmap__init(map, NULL, false, NULL);
+
+ ret = perf_mmap__mmap(map, &mp, fd, cpu);
+ if (ret)
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+void *perf_evsel__mmap_base(struct perf_evsel *evsel, int cpu, int thread)
+{
+ if (FD(evsel, cpu, thread) < 0)
+ return NULL;
+
+ return MMAP(evsel, cpu, thread)->base;
+}
+
int perf_evsel__read_size(struct perf_evsel *evsel)
{
u64 read_format = evsel->attr.read_format;
diff --git a/tools/lib/perf/include/internal/evsel.h b/tools/lib/perf/include/internal/evsel.h
index 1ffd083b235e..a7985dbb68ff 100644
--- a/tools/lib/perf/include/internal/evsel.h
+++ b/tools/lib/perf/include/internal/evsel.h
@@ -9,6 +9,7 @@
struct perf_cpu_map;
struct perf_thread_map;
+struct perf_mmap;
struct xyarray;
/*
@@ -40,6 +41,7 @@ struct perf_evsel {
struct perf_cpu_map *cpus;
struct perf_cpu_map *own_cpus;
struct perf_thread_map *threads;
+ struct perf_mmap *mmap;
struct xyarray *fd;
struct xyarray *sample_id;
u64 *id;
diff --git a/tools/lib/perf/include/perf/evsel.h b/tools/lib/perf/include/perf/evsel.h
index c82ec39a4ad0..9f5265f2f39f 100644
--- a/tools/lib/perf/include/perf/evsel.h
+++ b/tools/lib/perf/include/perf/evsel.h
@@ -27,6 +27,8 @@ LIBPERF_API int perf_evsel__open(struct perf_evsel *evsel, struct perf_cpu_map *
struct perf_thread_map *threads);
LIBPERF_API void perf_evsel__close(struct perf_evsel *evsel);
LIBPERF_API void perf_evsel__close_cpu(struct perf_evsel *evsel, int cpu);
+LIBPERF_API int perf_evsel__mmap(struct perf_evsel *evsel, int pages);
+LIBPERF_API void *perf_evsel__mmap_base(struct perf_evsel *evsel, int cpu, int thread);
LIBPERF_API int perf_evsel__read(struct perf_evsel *evsel, int cpu, int thread,
struct perf_counts_values *count);
LIBPERF_API int perf_evsel__enable(struct perf_evsel *evsel);
diff --git a/tools/lib/perf/libperf.map b/tools/lib/perf/libperf.map
index 7be1af8a546c..0b993de15830 100644
--- a/tools/lib/perf/libperf.map
+++ b/tools/lib/perf/libperf.map
@@ -23,6 +23,8 @@ LIBPERF_0.0.1 {
perf_evsel__disable;
perf_evsel__open;
perf_evsel__close;
+ perf_evsel__mmap;
+ perf_evsel__mmap_base;
perf_evsel__read;
perf_evsel__cpus;
perf_evsel__threads;
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH v5 4/9] libperf: Add evsel mmap support
2021-01-14 2:06 ` [PATCH v5 4/9] libperf: Add evsel mmap support Rob Herring
@ 2021-01-23 22:42 ` Jiri Olsa
0 siblings, 0 replies; 12+ messages in thread
From: Jiri Olsa @ 2021-01-23 22:42 UTC (permalink / raw)
To: Rob Herring
Cc: Mark Rutland, Ian Rogers, Peter Zijlstra, Catalin Marinas,
linux-kernel, Arnaldo Carvalho de Melo, Alexander Shishkin,
Raphael Gault, Ingo Molnar, honnappa.nagarahalli,
Jonathan Cameron, Namhyung Kim, Itaru Kitayama, Will Deacon,
linux-arm-kernel
On Wed, Jan 13, 2021 at 08:06:00PM -0600, Rob Herring wrote:
> In order to support usersapce access, an event must be mmapped. While
> there's already mmap support for evlist, the usecase is a bit different
> than the self monitoring with userspace access. So let's add a new
> perf_evsel__mmap() function to mmap an evsel. This allows implementing
> userspace access as a fastpath for perf_evsel__read().
>
> The mmapped address is returned by perf_evsel__mmap_base() which
> primarily for users/tests to check if userspace access is enabled.
>
> Signed-off-by: Rob Herring <robh@kernel.org>
> ---
> v5:
> - Create an mmap for every underlying event opened. Due to this, we
> need a different way to get the mmap ptr, so perf_evsel__mmap_base()
> is introduced.
> v4:
> - Change perf_evsel__mmap size to pages instead of bytes
> v3:
> - New patch split out from user access patch
> ---
> tools/lib/perf/Documentation/libperf.txt | 2 +
> tools/lib/perf/evsel.c | 47 +++++++++++++++++++++++-
> tools/lib/perf/include/internal/evsel.h | 2 +
> tools/lib/perf/include/perf/evsel.h | 2 +
> tools/lib/perf/libperf.map | 2 +
> 5 files changed, 53 insertions(+), 2 deletions(-)
>
> diff --git a/tools/lib/perf/Documentation/libperf.txt b/tools/lib/perf/Documentation/libperf.txt
> index 0c74c30ed23a..a2c73df191ca 100644
> --- a/tools/lib/perf/Documentation/libperf.txt
> +++ b/tools/lib/perf/Documentation/libperf.txt
> @@ -136,6 +136,8 @@ SYNOPSIS
> struct perf_thread_map *threads);
> void perf_evsel__close(struct perf_evsel *evsel);
> void perf_evsel__close_cpu(struct perf_evsel *evsel, int cpu);
> + int perf_evsel__mmap(struct perf_evsel *evsel, int pages);
> + void *perf_evsel__mmap_base(struct perf_evsel *evsel, int cpu, int thread);
> int perf_evsel__read(struct perf_evsel *evsel, int cpu, int thread,
> struct perf_counts_values *count);
> int perf_evsel__enable(struct perf_evsel *evsel);
> diff --git a/tools/lib/perf/evsel.c b/tools/lib/perf/evsel.c
> index 4dc06289f4c7..0b5bdf4badae 100644
> --- a/tools/lib/perf/evsel.c
> +++ b/tools/lib/perf/evsel.c
> @@ -11,10 +11,12 @@
> #include <stdlib.h>
> #include <internal/xyarray.h>
> #include <internal/cpumap.h>
> +#include <internal/mmap.h>
> #include <internal/threadmap.h>
> #include <internal/lib.h>
> #include <linux/string.h>
> #include <sys/ioctl.h>
> +#include <sys/mman.h>
>
> void perf_evsel__init(struct perf_evsel *evsel, struct perf_event_attr *attr)
> {
> @@ -37,11 +39,17 @@ void perf_evsel__delete(struct perf_evsel *evsel)
> free(evsel);
> }
>
> -#define FD(e, x, y) (*(int *) xyarray__entry(e->fd, x, y))
> +struct evsel_fd {
> + int fd;
> + struct perf_mmap mmap;
> +};
nice shortcut ;-) but 'struct perf_mmap' is too big for that
I think it's better to add new 'evsel::mmap' xyarray to hold it,
add perf_evsel__alloc_mmap to allocate it and call it from
perf_evsel__mmap the same way as we callperf_evsel__alloc_fd
from perf_evsel__open
thanks,
jirka
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v5 5/9] libperf: tests: Add support for verbose printing
2021-01-14 2:05 [PATCH v5 0/9] libperf and arm64 userspace counter access support Rob Herring
` (3 preceding siblings ...)
2021-01-14 2:06 ` [PATCH v5 4/9] libperf: Add evsel mmap support Rob Herring
@ 2021-01-14 2:06 ` Rob Herring
2021-01-14 2:06 ` [PATCH v5 6/9] libperf: Add support for user space counter access Rob Herring
` (4 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Rob Herring @ 2021-01-14 2:06 UTC (permalink / raw)
To: Will Deacon, Catalin Marinas, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Jiri Olsa, Mark Rutland
Cc: Ian Rogers, Alexander Shishkin, linux-kernel,
honnappa.nagarahalli, Raphael Gault, Jonathan Cameron,
Namhyung Kim, Itaru Kitayama, linux-arm-kernel
Add __T_VERBOSE() so tests can add verbose output. The verbose output is
enabled with the '-v' command line option.
Signed-off-by: Rob Herring <robh@kernel.org>
---
v5:
- Pass verbose flag to static tests
- Fix getopt loop with unsigned char (arm64)
v3:
- New patch
---
tools/lib/perf/include/internal/tests.h | 32 +++++++++++++++++++++++++
tools/lib/perf/tests/Makefile | 6 +++--
2 files changed, 36 insertions(+), 2 deletions(-)
diff --git a/tools/lib/perf/include/internal/tests.h b/tools/lib/perf/include/internal/tests.h
index 2093e8868a67..29425c2dabe1 100644
--- a/tools/lib/perf/include/internal/tests.h
+++ b/tools/lib/perf/include/internal/tests.h
@@ -3,11 +3,32 @@
#define __LIBPERF_INTERNAL_TESTS_H
#include <stdio.h>
+#include <unistd.h>
int tests_failed;
+int tests_verbose;
+
+static inline int get_verbose(char **argv, int argc)
+{
+ int c;
+ int verbose = 0;
+
+ while ((c = getopt(argc, argv, "v")) != -1) {
+ switch (c)
+ {
+ case 'v':
+ verbose = 1;
+ break;
+ default:
+ break;
+ }
+ }
+ return verbose;
+}
#define __T_START \
do { \
+ tests_verbose = get_verbose(argv, argc); \
fprintf(stdout, "- running %s...", __FILE__); \
fflush(NULL); \
tests_failed = 0; \
@@ -30,4 +51,15 @@ do {
} \
} while (0)
+#define __T_VERBOSE(...) \
+do { \
+ if (tests_verbose) { \
+ if (tests_verbose == 1) { \
+ fputc('\n', stderr); \
+ tests_verbose++; \
+ } \
+ fprintf(stderr, ##__VA_ARGS__); \
+ } \
+} while (0)
+
#endif /* __LIBPERF_INTERNAL_TESTS_H */
diff --git a/tools/lib/perf/tests/Makefile b/tools/lib/perf/tests/Makefile
index 96841775feaf..b536cc9a26dd 100644
--- a/tools/lib/perf/tests/Makefile
+++ b/tools/lib/perf/tests/Makefile
@@ -5,6 +5,8 @@ TESTS = test-cpumap test-threadmap test-evlist test-evsel
TESTS_SO := $(addsuffix -so,$(TESTS))
TESTS_A := $(addsuffix -a,$(TESTS))
+TEST_ARGS := $(if $(V),-v)
+
# Set compile option CFLAGS
ifdef EXTRA_CFLAGS
CFLAGS := $(EXTRA_CFLAGS)
@@ -28,9 +30,9 @@ all: $(TESTS_A) $(TESTS_SO)
run:
@echo "running static:"
- @for i in $(TESTS_A); do ./$$i; done
+ @for i in $(TESTS_A); do ./$$i $(TEST_ARGS); done
@echo "running dynamic:"
- @for i in $(TESTS_SO); do LD_LIBRARY_PATH=../ ./$$i; done
+ @for i in $(TESTS_SO); do LD_LIBRARY_PATH=../ ./$$i $(TEST_ARGS); done
clean:
$(call QUIET_CLEAN, tests)$(RM) $(TESTS_A) $(TESTS_SO)
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH v5 6/9] libperf: Add support for user space counter access
2021-01-14 2:05 [PATCH v5 0/9] libperf and arm64 userspace counter access support Rob Herring
` (4 preceding siblings ...)
2021-01-14 2:06 ` [PATCH v5 5/9] libperf: tests: Add support for verbose printing Rob Herring
@ 2021-01-14 2:06 ` Rob Herring
2021-01-14 2:06 ` [PATCH v5 7/9] libperf: Add arm64 support to perf_mmap__read_self() Rob Herring
` (3 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Rob Herring @ 2021-01-14 2:06 UTC (permalink / raw)
To: Will Deacon, Catalin Marinas, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Jiri Olsa, Mark Rutland
Cc: Ian Rogers, Alexander Shishkin, linux-kernel,
honnappa.nagarahalli, Raphael Gault, Jonathan Cameron,
Namhyung Kim, Itaru Kitayama, linux-arm-kernel
x86 and arm64 can both support direct access of event counters in
userspace. The access sequence is less than trivial and currently exists
in perf test code (tools/perf/arch/x86/tests/rdpmc.c) with copies in
projects such as PAPI and libpfm4.
In order to support usersapce access, an event must be mmapped first
with perf_evsel__mmap(). Then subsequent calls to perf_evsel__read()
will use the fast path (assuming the arch supports it).
Signed-off-by: Rob Herring <robh@kernel.org>
---
v5:
- Make raw count s64 instead of u64 so that counter width shifting
works
- Adapt to mmap changes
v4:
- Update perf_evsel__mmap size to pages
v3:
- Split out perf_evsel__mmap() to separate patch
squash! libperf: Add support for user space counter access
---
tools/lib/perf/evsel.c | 3 +
tools/lib/perf/include/internal/mmap.h | 3 +
tools/lib/perf/mmap.c | 88 ++++++++++++++++++++++++++
tools/lib/perf/tests/test-evsel.c | 65 +++++++++++++++++++
4 files changed, 159 insertions(+)
diff --git a/tools/lib/perf/evsel.c b/tools/lib/perf/evsel.c
index 0b5bdf4badae..c0ecac77c85e 100644
--- a/tools/lib/perf/evsel.c
+++ b/tools/lib/perf/evsel.c
@@ -234,6 +234,9 @@ int perf_evsel__read(struct perf_evsel *evsel, int cpu, int thread,
if (FD(evsel, cpu, thread) < 0)
return -EINVAL;
+ if (!perf_mmap__read_self(MMAP(evsel, cpu, thread), count))
+ return 0;
+
if (readn(FD(evsel, cpu, thread), count->values, size) <= 0)
return -errno;
diff --git a/tools/lib/perf/include/internal/mmap.h b/tools/lib/perf/include/internal/mmap.h
index be7556e0a2b2..5e3422f40ed5 100644
--- a/tools/lib/perf/include/internal/mmap.h
+++ b/tools/lib/perf/include/internal/mmap.h
@@ -11,6 +11,7 @@
#define PERF_SAMPLE_MAX_SIZE (1 << 16)
struct perf_mmap;
+struct perf_counts_values;
typedef void (*libperf_unmap_cb_t)(struct perf_mmap *map);
@@ -52,4 +53,6 @@ void perf_mmap__put(struct perf_mmap *map);
u64 perf_mmap__read_head(struct perf_mmap *map);
+int perf_mmap__read_self(struct perf_mmap *map, struct perf_counts_values *count);
+
#endif /* __LIBPERF_INTERNAL_MMAP_H */
diff --git a/tools/lib/perf/mmap.c b/tools/lib/perf/mmap.c
index 79d5ed6c38cc..915469f00cf4 100644
--- a/tools/lib/perf/mmap.c
+++ b/tools/lib/perf/mmap.c
@@ -8,9 +8,11 @@
#include <linux/perf_event.h>
#include <perf/mmap.h>
#include <perf/event.h>
+#include <perf/evsel.h>
#include <internal/mmap.h>
#include <internal/lib.h>
#include <linux/kernel.h>
+#include <linux/math64.h>
#include "internal.h"
void perf_mmap__init(struct perf_mmap *map, struct perf_mmap *prev,
@@ -273,3 +275,89 @@ union perf_event *perf_mmap__read_event(struct perf_mmap *map)
return event;
}
+
+#if defined(__i386__) || defined(__x86_64__)
+static u64 read_perf_counter(unsigned int counter)
+{
+ unsigned int low, high;
+
+ asm volatile("rdpmc" : "=a" (low), "=d" (high) : "c" (counter));
+
+ return low | ((u64)high) << 32;
+}
+
+static u64 read_timestamp(void)
+{
+ unsigned int low, high;
+
+ asm volatile("rdtsc" : "=a" (low), "=d" (high));
+
+ return low | ((u64)high) << 32;
+}
+#else
+static u64 read_perf_counter(unsigned int counter) { return 0; }
+static u64 read_timestamp(void) { return 0; }
+#endif
+
+int perf_mmap__read_self(struct perf_mmap *map, struct perf_counts_values *count)
+{
+ struct perf_event_mmap_page *pc = map->base;
+ u32 seq, idx, time_mult = 0, time_shift = 0;
+ u64 cnt, cyc = 0, time_offset = 0, time_cycles = 0, time_mask = ~0ULL;
+
+ if (!pc || !pc->cap_user_rdpmc)
+ return -1;
+
+ do {
+ seq = READ_ONCE(pc->lock);
+ barrier();
+
+ count->ena = READ_ONCE(pc->time_enabled);
+ count->run = READ_ONCE(pc->time_running);
+
+ if (pc->cap_user_time && count->ena != count->run) {
+ cyc = read_timestamp();
+ time_mult = READ_ONCE(pc->time_mult);
+ time_shift = READ_ONCE(pc->time_shift);
+ time_offset = READ_ONCE(pc->time_offset);
+
+ if (pc->cap_user_time_short) {
+ time_cycles = READ_ONCE(pc->time_cycles);
+ time_mask = READ_ONCE(pc->time_mask);
+ }
+ }
+
+ idx = READ_ONCE(pc->index);
+ cnt = READ_ONCE(pc->offset);
+ if (pc->cap_user_rdpmc && idx) {
+ s64 evcnt = read_perf_counter(idx - 1);
+ u16 width = READ_ONCE(pc->pmc_width);
+
+ evcnt <<= 64 - width;
+ evcnt >>= 64 - width;
+ cnt += evcnt;
+ } else
+ return -1;
+
+ barrier();
+ } while (READ_ONCE(pc->lock) != seq);
+
+ if (count->ena != count->run) {
+ u64 delta;
+
+ /* Adjust for cap_usr_time_short, a nop if not */
+ cyc = time_cycles + ((cyc - time_cycles) & time_mask);
+
+ delta = time_offset + mul_u64_u32_shr(cyc, time_mult, time_shift);
+
+ count->ena += delta;
+ if (idx)
+ count->run += delta;
+
+ cnt = mul_u64_u64_div64(cnt, count->ena, count->run);
+ }
+
+ count->val = cnt;
+
+ return 0;
+}
diff --git a/tools/lib/perf/tests/test-evsel.c b/tools/lib/perf/tests/test-evsel.c
index 135722ac965b..95380ce7e76c 100644
--- a/tools/lib/perf/tests/test-evsel.c
+++ b/tools/lib/perf/tests/test-evsel.c
@@ -120,6 +120,69 @@ static int test_stat_thread_enable(void)
return 0;
}
+static int test_stat_user_read(int event)
+{
+ struct perf_counts_values counts = { .val = 0 };
+ struct perf_thread_map *threads;
+ struct perf_evsel *evsel;
+ struct perf_event_mmap_page *pc;
+ struct perf_event_attr attr = {
+ .type = PERF_TYPE_HARDWARE,
+ .config = event,
+ };
+ int err, i;
+
+ threads = perf_thread_map__new_dummy();
+ __T("failed to create threads", threads);
+
+ perf_thread_map__set_pid(threads, 0, 0);
+
+ evsel = perf_evsel__new(&attr);
+ __T("failed to create evsel", evsel);
+
+ err = perf_evsel__open(evsel, NULL, threads);
+ __T("failed to open evsel", err == 0);
+
+ err = perf_evsel__mmap(evsel, 0);
+ __T("failed to mmap evsel", err == 0);
+
+ pc = perf_evsel__mmap_base(evsel, 0, 0);
+
+#if defined(__i386__) || defined(__x86_64__) || defined(__aarch64__)
+ __T("userspace counter access not supported", pc->cap_user_rdpmc);
+ __T("userspace counter access not enabled", pc->index);
+ __T("userspace counter width not set", pc->pmc_width >= 32);
+#endif
+
+ perf_evsel__read(evsel, 0, 0, &counts);
+ __T("failed to read value for evsel", counts.val != 0);
+
+ for (i = 0; i < 5; i++) {
+ volatile int count = 0x10000 << i;
+ __u64 start, end, last = 0;
+
+ __T_VERBOSE("\tloop = %u, ", count);
+
+ perf_evsel__read(evsel, 0, 0, &counts);
+ start = counts.val;
+
+ while (count--) ;
+
+ perf_evsel__read(evsel, 0, 0, &counts);
+ end = counts.val;
+
+ __T("invalid counter data", (end - start) > last);
+ last = end - start;
+ __T_VERBOSE("count = %llu\n", end - start);
+ }
+
+ perf_evsel__close(evsel);
+ perf_evsel__delete(evsel);
+
+ perf_thread_map__put(threads);
+ return 0;
+}
+
int main(int argc, char **argv)
{
__T_START;
@@ -129,6 +192,8 @@ int main(int argc, char **argv)
test_stat_cpu();
test_stat_thread();
test_stat_thread_enable();
+ test_stat_user_read(PERF_COUNT_HW_INSTRUCTIONS);
+ test_stat_user_read(PERF_COUNT_HW_CPU_CYCLES);
__T_END;
return 0;
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH v5 7/9] libperf: Add arm64 support to perf_mmap__read_self()
2021-01-14 2:05 [PATCH v5 0/9] libperf and arm64 userspace counter access support Rob Herring
` (5 preceding siblings ...)
2021-01-14 2:06 ` [PATCH v5 6/9] libperf: Add support for user space counter access Rob Herring
@ 2021-01-14 2:06 ` Rob Herring
2021-01-14 2:06 ` [PATCH v5 8/9] perf: arm64: Add test for userspace counter access on heterogeneous systems Rob Herring
` (2 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Rob Herring @ 2021-01-14 2:06 UTC (permalink / raw)
To: Will Deacon, Catalin Marinas, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Jiri Olsa, Mark Rutland
Cc: Ian Rogers, Alexander Shishkin, linux-kernel,
honnappa.nagarahalli, Raphael Gault, Jonathan Cameron,
Namhyung Kim, Itaru Kitayama, linux-arm-kernel
Add the arm64 variants for read_perf_counter() and read_timestamp().
Unfortunately the counter number is encoded into the instruction, so the
code is a bit verbose to enumerate all possible counters.
Signed-off-by: Rob Herring <robh@kernel.org>
---
tools/lib/perf/mmap.c | 98 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 98 insertions(+)
diff --git a/tools/lib/perf/mmap.c b/tools/lib/perf/mmap.c
index 915469f00cf4..216e6c6212a2 100644
--- a/tools/lib/perf/mmap.c
+++ b/tools/lib/perf/mmap.c
@@ -13,6 +13,7 @@
#include <internal/lib.h>
#include <linux/kernel.h>
#include <linux/math64.h>
+#include <linux/stringify.h>
#include "internal.h"
void perf_mmap__init(struct perf_mmap *map, struct perf_mmap *prev,
@@ -294,6 +295,103 @@ static u64 read_timestamp(void)
return low | ((u64)high) << 32;
}
+#elif defined(__aarch64__)
+#define read_sysreg(r) ({ \
+ u64 __val; \
+ asm volatile("mrs %0, " __stringify(r) : "=r" (__val)); \
+ __val; \
+})
+
+static u64 read_pmccntr(void)
+{
+ return read_sysreg(pmccntr_el0);
+}
+
+#define PMEVCNTR_READ(idx) \
+ static u64 read_pmevcntr_##idx(void) { \
+ return read_sysreg(pmevcntr##idx##_el0); \
+ }
+
+PMEVCNTR_READ(0);
+PMEVCNTR_READ(1);
+PMEVCNTR_READ(2);
+PMEVCNTR_READ(3);
+PMEVCNTR_READ(4);
+PMEVCNTR_READ(5);
+PMEVCNTR_READ(6);
+PMEVCNTR_READ(7);
+PMEVCNTR_READ(8);
+PMEVCNTR_READ(9);
+PMEVCNTR_READ(10);
+PMEVCNTR_READ(11);
+PMEVCNTR_READ(12);
+PMEVCNTR_READ(13);
+PMEVCNTR_READ(14);
+PMEVCNTR_READ(15);
+PMEVCNTR_READ(16);
+PMEVCNTR_READ(17);
+PMEVCNTR_READ(18);
+PMEVCNTR_READ(19);
+PMEVCNTR_READ(20);
+PMEVCNTR_READ(21);
+PMEVCNTR_READ(22);
+PMEVCNTR_READ(23);
+PMEVCNTR_READ(24);
+PMEVCNTR_READ(25);
+PMEVCNTR_READ(26);
+PMEVCNTR_READ(27);
+PMEVCNTR_READ(28);
+PMEVCNTR_READ(29);
+PMEVCNTR_READ(30);
+
+/*
+ * Read a value direct from PMEVCNTR<idx>
+ */
+static u64 read_perf_counter(unsigned int counter)
+{
+ static u64 (* const read_f[])(void) = {
+ read_pmevcntr_0,
+ read_pmevcntr_1,
+ read_pmevcntr_2,
+ read_pmevcntr_3,
+ read_pmevcntr_4,
+ read_pmevcntr_5,
+ read_pmevcntr_6,
+ read_pmevcntr_7,
+ read_pmevcntr_8,
+ read_pmevcntr_9,
+ read_pmevcntr_10,
+ read_pmevcntr_11,
+ read_pmevcntr_13,
+ read_pmevcntr_12,
+ read_pmevcntr_14,
+ read_pmevcntr_15,
+ read_pmevcntr_16,
+ read_pmevcntr_17,
+ read_pmevcntr_18,
+ read_pmevcntr_19,
+ read_pmevcntr_20,
+ read_pmevcntr_21,
+ read_pmevcntr_22,
+ read_pmevcntr_23,
+ read_pmevcntr_24,
+ read_pmevcntr_25,
+ read_pmevcntr_26,
+ read_pmevcntr_27,
+ read_pmevcntr_28,
+ read_pmevcntr_29,
+ read_pmevcntr_30,
+ read_pmccntr
+ };
+
+ if (counter < ARRAY_SIZE(read_f))
+ return (read_f[counter])();
+
+ return 0;
+}
+
+static u64 read_timestamp(void) { return read_sysreg(cntvct_el0); }
+
#else
static u64 read_perf_counter(unsigned int counter) { return 0; }
static u64 read_timestamp(void) { return 0; }
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH v5 8/9] perf: arm64: Add test for userspace counter access on heterogeneous systems
2021-01-14 2:05 [PATCH v5 0/9] libperf and arm64 userspace counter access support Rob Herring
` (6 preceding siblings ...)
2021-01-14 2:06 ` [PATCH v5 7/9] libperf: Add arm64 support to perf_mmap__read_self() Rob Herring
@ 2021-01-14 2:06 ` Rob Herring
2021-01-14 2:06 ` [PATCH v5 9/9] Documentation: arm64: Document PMU counters access from userspace Rob Herring
2021-02-04 15:18 ` [PATCH v5 0/9] libperf and arm64 userspace counter access support Rob Herring
9 siblings, 0 replies; 12+ messages in thread
From: Rob Herring @ 2021-01-14 2:06 UTC (permalink / raw)
To: Will Deacon, Catalin Marinas, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Jiri Olsa, Mark Rutland
Cc: Ian Rogers, Alexander Shishkin, linux-kernel,
honnappa.nagarahalli, Raphael Gault, Jonathan Cameron,
Namhyung Kim, Itaru Kitayama, linux-arm-kernel
Userspace counter access only works on heterogeneous systems with some
restrictions. The userspace process must be pinned to a homogeneous
subset of CPUs and must open the corresponding PMU for those CPUs. This
commit adds a test implementing these requirements.
Signed-off-by: Rob Herring <robh@kernel.org>
---
v5:
- Adapt to libperf mmap API changes
v4:
- Update perf_evsel__mmap params
v2:
- Drop all but heterogeneous test as others covered by libperf tests
- Rework to use libperf
---
tools/perf/arch/arm64/include/arch-tests.h | 7 +
tools/perf/arch/arm64/tests/Build | 1 +
tools/perf/arch/arm64/tests/arch-tests.c | 4 +
tools/perf/arch/arm64/tests/user-events.c | 170 +++++++++++++++++++++
4 files changed, 182 insertions(+)
create mode 100644 tools/perf/arch/arm64/tests/user-events.c
diff --git a/tools/perf/arch/arm64/include/arch-tests.h b/tools/perf/arch/arm64/include/arch-tests.h
index 90ec4c8cb880..380ad34a3f09 100644
--- a/tools/perf/arch/arm64/include/arch-tests.h
+++ b/tools/perf/arch/arm64/include/arch-tests.h
@@ -2,11 +2,18 @@
#ifndef ARCH_TESTS_H
#define ARCH_TESTS_H
+#include <linux/compiler.h>
+
#ifdef HAVE_DWARF_UNWIND_SUPPORT
struct thread;
struct perf_sample;
+int test__arch_unwind_sample(struct perf_sample *sample,
+ struct thread *thread);
#endif
extern struct test arch_tests[];
+int test__rd_pinned(struct test __maybe_unused *test,
+ int __maybe_unused subtest);
+
#endif
diff --git a/tools/perf/arch/arm64/tests/Build b/tools/perf/arch/arm64/tests/Build
index a61c06bdb757..3f9a20c17fc6 100644
--- a/tools/perf/arch/arm64/tests/Build
+++ b/tools/perf/arch/arm64/tests/Build
@@ -1,4 +1,5 @@
perf-y += regs_load.o
perf-$(CONFIG_DWARF_UNWIND) += dwarf-unwind.o
+perf-y += user-events.o
perf-y += arch-tests.o
diff --git a/tools/perf/arch/arm64/tests/arch-tests.c b/tools/perf/arch/arm64/tests/arch-tests.c
index 5b1543c98022..80ce7bd3c16d 100644
--- a/tools/perf/arch/arm64/tests/arch-tests.c
+++ b/tools/perf/arch/arm64/tests/arch-tests.c
@@ -10,6 +10,10 @@ struct test arch_tests[] = {
.func = test__dwarf_unwind,
},
#endif
+ {
+ .desc = "Pinned CPU user counter access",
+ .func = test__rd_pinned,
+ },
{
.func = NULL,
},
diff --git a/tools/perf/arch/arm64/tests/user-events.c b/tools/perf/arch/arm64/tests/user-events.c
new file mode 100644
index 000000000000..379d3f688cd7
--- /dev/null
+++ b/tools/perf/arch/arm64/tests/user-events.c
@@ -0,0 +1,170 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <unistd.h>
+#include <sched.h>
+#include <cpumap.h>
+
+#include <perf/core.h>
+#include <perf/threadmap.h>
+#include <perf/evsel.h>
+
+#include "pmu.h"
+#include "debug.h"
+#include "tests/tests.h"
+#include "arch-tests.h"
+
+static int run_test(struct perf_evsel *evsel)
+{
+ int n;
+ volatile int tmp = 0;
+ u64 delta, i, loops = 1000;
+ struct perf_counts_values counts = { .val = 0 };
+
+ for (n = 0; n < 6; n++) {
+ u64 stamp, now;
+
+ perf_evsel__read(evsel, 0, 0, &counts);
+ stamp = counts.val;
+
+ for (i = 0; i < loops; i++)
+ tmp++;
+
+ perf_evsel__read(evsel, 0, 0, &counts);
+ now = counts.val;
+ loops *= 10;
+
+ delta = now - stamp;
+ pr_debug("%14d: %14llu\n", n, (long long)delta);
+
+ if (!delta)
+ break;
+ }
+ return delta ? 0 : -1;
+}
+
+static struct perf_pmu *pmu_for_cpu(int cpu)
+{
+ int acpu, idx;
+ struct perf_pmu *pmu = NULL;
+
+ while ((pmu = perf_pmu__scan(pmu)) != NULL) {
+ if (pmu->is_uncore)
+ continue;
+ perf_cpu_map__for_each_cpu(acpu, idx, pmu->cpus)
+ if (acpu == cpu)
+ return pmu;
+ }
+ return NULL;
+}
+
+static bool pmu_is_homogeneous(void)
+{
+ int core_cnt = 0;
+ struct perf_pmu *pmu = NULL;
+
+ while ((pmu = perf_pmu__scan(pmu)) != NULL) {
+ if (!pmu->is_uncore && !perf_cpu_map__empty(pmu->cpus))
+ core_cnt++;
+ }
+ return core_cnt == 1;
+}
+
+static int libperf_print(enum libperf_print_level level,
+ const char *fmt, va_list ap)
+{
+ (void)level;
+ return vfprintf(stderr, fmt, ap);
+}
+
+static struct perf_evsel *perf_init(struct perf_event_attr *attr)
+{
+ int err;
+ struct perf_thread_map *threads;
+ struct perf_evsel *evsel;
+
+ libperf_init(libperf_print);
+
+ threads = perf_thread_map__new_dummy();
+ if (!threads) {
+ pr_err("failed to create threads\n");
+ return NULL;
+ }
+
+ perf_thread_map__set_pid(threads, 0, 0);
+
+ evsel = perf_evsel__new(attr);
+ if (!evsel) {
+ pr_err("failed to create evsel\n");
+ goto out_thread;
+ }
+
+ err = perf_evsel__open(evsel, NULL, threads);
+ if (err) {
+ pr_err("failed to open evsel\n");
+ goto out_open;
+ }
+
+ if (perf_evsel__mmap(evsel, 0)) {
+ pr_err("failed to mmap evsel\n");
+ goto out_mmap;
+ }
+
+ return evsel;
+
+out_mmap:
+ perf_evsel__close(evsel);
+out_open:
+ perf_evsel__delete(evsel);
+out_thread:
+ perf_thread_map__put(threads);
+ return NULL;
+}
+
+int test__rd_pinned(struct test __maybe_unused *test,
+ int __maybe_unused subtest)
+{
+ int cpu, cputmp, ret = -1;
+ struct perf_evsel *evsel;
+ struct perf_event_attr attr = {
+ .config = 0x8, /* Instruction count */
+ .config1 = 0, /* 32-bit counter */
+ .exclude_kernel = 1,
+ };
+ cpu_set_t cpu_set;
+ struct perf_pmu *pmu;
+
+ if (pmu_is_homogeneous())
+ return TEST_SKIP;
+
+ cpu = sched_getcpu();
+ pmu = pmu_for_cpu(cpu);
+ if (!pmu)
+ return -1;
+ attr.type = pmu->type;
+
+ CPU_ZERO(&cpu_set);
+ perf_cpu_map__for_each_cpu(cpu, cputmp, pmu->cpus)
+ CPU_SET(cpu, &cpu_set);
+ if (sched_setaffinity(0, sizeof(cpu_set), &cpu_set) < 0)
+ pr_err("Could not set affinity\n");
+
+ evsel = perf_init(&attr);
+ if (!evsel)
+ return -1;
+
+ perf_cpu_map__for_each_cpu(cpu, cputmp, pmu->cpus) {
+ CPU_ZERO(&cpu_set);
+ CPU_SET(cpu, &cpu_set);
+ if (sched_setaffinity(0, sizeof(cpu_set), &cpu_set) < 0)
+ pr_err("Could not set affinity\n");
+
+ pr_debug("Running on CPU %d\n", cpu);
+
+ ret = run_test(evsel);
+ if (ret)
+ break;
+ }
+
+ perf_evsel__close(evsel);
+ perf_evsel__delete(evsel);
+ return ret;
+}
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH v5 9/9] Documentation: arm64: Document PMU counters access from userspace
2021-01-14 2:05 [PATCH v5 0/9] libperf and arm64 userspace counter access support Rob Herring
` (7 preceding siblings ...)
2021-01-14 2:06 ` [PATCH v5 8/9] perf: arm64: Add test for userspace counter access on heterogeneous systems Rob Herring
@ 2021-01-14 2:06 ` Rob Herring
2021-02-04 15:18 ` [PATCH v5 0/9] libperf and arm64 userspace counter access support Rob Herring
9 siblings, 0 replies; 12+ messages in thread
From: Rob Herring @ 2021-01-14 2:06 UTC (permalink / raw)
To: Will Deacon, Catalin Marinas, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Jiri Olsa, Mark Rutland
Cc: Ian Rogers, Alexander Shishkin, linux-kernel,
honnappa.nagarahalli, Raphael Gault, Jonathan Cameron,
Namhyung Kim, Itaru Kitayama, linux-arm-kernel
From: Raphael Gault <raphael.gault@arm.com>
Add a documentation file to describe the access to the pmu hardware
counters from userspace
Signed-off-by: Raphael Gault <raphael.gault@arm.com>
Signed-off-by: Rob Herring <robh@kernel.org>
---
v2:
- Update links to test examples
Changes from Raphael's v4:
- Convert to rSt
- Update chained event status
- Add section for heterogeneous systems
---
Documentation/arm64/index.rst | 1 +
.../arm64/perf_counter_user_access.rst | 56 +++++++++++++++++++
2 files changed, 57 insertions(+)
create mode 100644 Documentation/arm64/perf_counter_user_access.rst
diff --git a/Documentation/arm64/index.rst b/Documentation/arm64/index.rst
index 97d65ba12a35..eb7b1cabbf08 100644
--- a/Documentation/arm64/index.rst
+++ b/Documentation/arm64/index.rst
@@ -18,6 +18,7 @@ ARM64 Architecture
memory
memory-tagging-extension
perf
+ perf_counter_user_access
pointer-authentication
silicon-errata
sve
diff --git a/Documentation/arm64/perf_counter_user_access.rst b/Documentation/arm64/perf_counter_user_access.rst
new file mode 100644
index 000000000000..e49e141f10cc
--- /dev/null
+++ b/Documentation/arm64/perf_counter_user_access.rst
@@ -0,0 +1,56 @@
+=============================================
+Access to PMU hardware counter from userspace
+=============================================
+
+Overview
+--------
+The perf userspace tool relies on the PMU to monitor events. It offers an
+abstraction layer over the hardware counters since the underlying
+implementation is cpu-dependent.
+Arm64 allows userspace tools to have access to the registers storing the
+hardware counters' values directly.
+
+This targets specifically self-monitoring tasks in order to reduce the overhead
+by directly accessing the registers without having to go through the kernel.
+
+How-to
+------
+The focus is set on the armv8 pmuv3 which makes sure that the access to the pmu
+registers is enabled and that the userspace has access to the relevant
+information in order to use them.
+
+In order to have access to the hardware counter it is necessary to open the event
+using the perf tool interface: the sys_perf_event_open syscall returns a fd which
+can subsequently be used with the mmap syscall in order to retrieve a page of
+memory containing information about the event.
+The PMU driver uses this page to expose to the user the hardware counter's
+index and other necessary data. Using this index enables the user to access the
+PMU registers using the `mrs` instruction.
+
+The userspace access is supported in libperf using the perf_evsel__mmap()
+and perf_evsel__read() functions. See `tools/lib/perf/tests/test-evsel.c`_ for
+an example.
+
+About heterogeneous systems
+---------------------------
+On heterogeneous systems such as big.LITTLE, userspace PMU counter access can
+only be enabled when the tasks are pinned to a homogeneous subset of cores and
+the corresponding PMU instance is opened by specifying the 'type' attribute.
+The use of generic event types is not supported in this case.
+
+Have a look at `tools/perf/arch/arm64/tests/user-events.c`_ for an example. It
+can be run using the perf tool to check that the access to the registers works
+correctly from userspace:
+
+.. code-block:: sh
+
+ perf test -v user
+
+About chained events
+--------------------
+Chained events are not supported in userspace. If a 64-bit counter is requested,
+userspace access will only be enabled if the underlying counter is 64-bit.
+
+.. Links
+.. _tools/perf/arch/arm64/tests/user-events.c:
+ https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/tools/perf/arch/arm64/tests/user-events.c
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH v5 0/9] libperf and arm64 userspace counter access support
2021-01-14 2:05 [PATCH v5 0/9] libperf and arm64 userspace counter access support Rob Herring
` (8 preceding siblings ...)
2021-01-14 2:06 ` [PATCH v5 9/9] Documentation: arm64: Document PMU counters access from userspace Rob Herring
@ 2021-02-04 15:18 ` Rob Herring
9 siblings, 0 replies; 12+ messages in thread
From: Rob Herring @ 2021-02-04 15:18 UTC (permalink / raw)
To: Will Deacon, Catalin Marinas, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Jiri Olsa, Mark Rutland
Cc: Ian Rogers, Alexander Shishkin, linux-kernel,
honnappa.nagarahalli, Raphael Gault, Jonathan Cameron,
Namhyung Kim, Itaru Kitayama, linux-arm-kernel
On Wed, Jan 13, 2021 at 08:05:56PM -0600, Rob Herring wrote:
> Hi all,
>
> Another version of libperf and arm64 userspace counter access support.
> This originally resurrected Raphael's series[1] to enable userspace counter
> access on arm64. My previous versions are here[2][3][4][5]. A git branch is
> here[6].
Ping.
Will, Mark, comments on the Arm bits?
>
> Changes in v5:
> - Limit enabling/disabling access to CPUs associated with the PMU
> (supported_cpus) and with the mm_struct matching current->active_mm.
> The x86 method of using mm_cpumask doesn't work for arm64 as it is not
> updated.
> - Only set cap_user_rdpmc if event is on current cpu. See patch 2.
> - Create an mmap for every event in an evsel. This results in some changes
> to the libperf mmap API from the last version.
> - Rebase to v5.11-rc2
>
> Changes in v4:
> - Dropped 'arm64: pmu: Add hook to handle pmu-related undefined instructions'.
> The onus is on userspace to pin itself to a homogeneous subset of CPUs
> and avoid any aborts on heterogeneous systems, so the hook is not needed.
> - Make perf_evsel__mmap() take pages rather than bytes for size
> - Fix building arm64 heterogeneous test.
>
> Changes in v3:
> - Dropped removing x86 rdpmc test until libperf tests can run via 'perf test'
> - Added verbose prints for tests
> - Split adding perf_evsel__mmap() to separate patch
>
>
> The following changes to the arm64 support have been made compared to
> Raphael's last version:
>
> The major change is support for heterogeneous systems with some
> restrictions. Specifically, userspace must pin itself to like CPUs, open
> a specific PMU by type, and use h/w specific events. The tests have been
> reworked to demonstrate this.
>
> Chained events are not supported. The problem with supporting chained
> events was there's no way to distinguish between a chained event and a
> native 64-bit counter. We could add some flag, but do self monitoring
> processes really need that? Native 64-bit counters are supported if the
> PMU h/w has support. As there's already an explicit ABI to request 64-bit
> counters, userspace can request 64-bit counters and if user
> access is not enabled, then it must retry with 32-bit counters.
>
> Prior versions broke the build on arm32 (surprisingly never caught by
> 0-day). As a result, event_mapped and event_unmapped implementations have
> been moved into the arm64 code.
>
> There was a bug in that pmc_width was not set in the user page. The tests
> now check for this.
>
> The documentation has been converted to rST. I've added sections on
> chained events and heterogeneous.
>
> The tests have been expanded to test the cycle counter access.
>
> Rob
>
> [1] https://lore.kernel.org/r/20190822144220.27860-1-raphael.gault@arm.com/
> [2] https://lore.kernel.org/r/20200707205333.624938-1-robh@kernel.org/
> [3] https://lore.kernel.org/r/20200828205614.3391252-1-robh@kernel.org/
> [4] https://lore.kernel.org/r/20200911215118.2887710-1-robh@kernel.org/
> [5] https://lore.kernel.org/r/20201001140116.651970-1-robh@kernel.org/
> [6] git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git user-perf-event-v5
>
>
> Raphael Gault (3):
> arm64: pmu: Add function implementation to update event index in
> userpage
> arm64: perf: Enable PMU counter direct access for perf event
> Documentation: arm64: Document PMU counters access from userspace
>
> Rob Herring (6):
> tools/include: Add an initial math64.h
> libperf: Add evsel mmap support
> libperf: tests: Add support for verbose printing
> libperf: Add support for user space counter access
> libperf: Add arm64 support to perf_mmap__read_self()
> perf: arm64: Add test for userspace counter access on heterogeneous
> systems
>
> Documentation/arm64/index.rst | 1 +
> .../arm64/perf_counter_user_access.rst | 56 ++++++
> arch/arm64/include/asm/mmu.h | 5 +
> arch/arm64/include/asm/mmu_context.h | 2 +
> arch/arm64/include/asm/perf_event.h | 14 ++
> arch/arm64/kernel/perf_event.c | 68 +++++++
> include/linux/perf/arm_pmu.h | 2 +
> tools/include/linux/math64.h | 75 +++++++
> tools/lib/perf/Documentation/libperf.txt | 2 +
> tools/lib/perf/evsel.c | 50 ++++-
> tools/lib/perf/include/internal/evsel.h | 2 +
> tools/lib/perf/include/internal/mmap.h | 3 +
> tools/lib/perf/include/internal/tests.h | 32 +++
> tools/lib/perf/include/perf/evsel.h | 2 +
> tools/lib/perf/libperf.map | 2 +
> tools/lib/perf/mmap.c | 186 ++++++++++++++++++
> tools/lib/perf/tests/Makefile | 6 +-
> tools/lib/perf/tests/test-evsel.c | 65 ++++++
> tools/perf/arch/arm64/include/arch-tests.h | 7 +
> tools/perf/arch/arm64/tests/Build | 1 +
> tools/perf/arch/arm64/tests/arch-tests.c | 4 +
> tools/perf/arch/arm64/tests/user-events.c | 170 ++++++++++++++++
> 22 files changed, 751 insertions(+), 4 deletions(-)
> create mode 100644 Documentation/arm64/perf_counter_user_access.rst
> create mode 100644 tools/include/linux/math64.h
> create mode 100644 tools/perf/arch/arm64/tests/user-events.c
>
> --
> 2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread