From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752941AbdKWPJw (ORCPT ); Thu, 23 Nov 2017 10:09:52 -0500 Received: from mail.kernel.org ([198.145.29.99]:53772 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752889AbdKWPJv (ORCPT ); Thu, 23 Nov 2017 10:09:51 -0500 DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6AE7E218A6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=acme@kernel.org Date: Thu, 23 Nov 2017 12:09:48 -0300 From: Arnaldo Carvalho de Melo To: James Yang Cc: Kim Phillips , Peter Zijlstra , Ingo Molnar , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Thomas Gleixner , Darren Hart , Colin Ian King , linux-kernel@vger.kernel.org Subject: Re: [PATCH 1/3] perf bench futex: benchmark only online CPUs Message-ID: <20171123150948.GD8789@kernel.org> References: <20171122182528.3a5ac33fa0563b9e25271659@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20171122182528.3a5ac33fa0563b9e25271659@arm.com> X-Url: http://acmel.wordpress.com User-Agent: Mutt/1.9.1 (2017-09-22) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Em Wed, Nov 22, 2017 at 06:25:28PM -0600, Kim Phillips escreveu: > From: James Yang > > The "perf bench futex" benchmarks have a problem when not all CPUs in > the system are online: perf assumes the CPUs that are online are > contiguously numbered and assigns processor affinity to the threads by > modulo striping. When the online CPUs are not contiguously numbered, > perf errors out with: > > $ echo 0 | sudo tee /sys/devices/system/cpu/cpu3/online > 0 > $ ./oldperf bench futex all > perf: pthread_create: Operation not permitted > Run summary [PID 14934]: 7 threads, each operating on 1024 [private] futexes for 10 secs. > > $ > > This patch makes perf not assume all CPUs configured are online, and > adds a mapping table to stripe affinity across the CPUs that are > online. So, have you looked at tools/perf/util/cpumap.c? I think you can use: int i; struct cpu_map *cpus = cpu_map__new(NULL); for (i = 0; i < cpus->nr; ++i) { int cpu = cpus->map[i]; ... } No? - Arnaldo > Signed-off-by: James Yang > Signed-off-by: Kim Phillips > --- > tools/perf/bench/Build | 1 + > tools/perf/bench/cpu-online-map.c | 21 +++++++++++++++++++++ > tools/perf/bench/cpu-online-map.h | 6 ++++++ > tools/perf/bench/futex-hash.c | 14 ++++++++++++-- > tools/perf/bench/futex-lock-pi.c | 15 ++++++++++++--- > tools/perf/bench/futex-requeue.c | 14 +++++++++++--- > tools/perf/bench/futex-wake-parallel.c | 17 ++++++++++++++--- > tools/perf/bench/futex-wake.c | 15 ++++++++++++--- > 8 files changed, 89 insertions(+), 14 deletions(-) > create mode 100644 tools/perf/bench/cpu-online-map.c > create mode 100644 tools/perf/bench/cpu-online-map.h > > diff --git a/tools/perf/bench/Build b/tools/perf/bench/Build > index 60bf11943047..bc79d3577ace 100644 > --- a/tools/perf/bench/Build > +++ b/tools/perf/bench/Build > @@ -6,6 +6,7 @@ perf-y += futex-wake.o > perf-y += futex-wake-parallel.o > perf-y += futex-requeue.o > perf-y += futex-lock-pi.o > +perf-y += cpu-online-map.o > > perf-$(CONFIG_X86_64) += mem-memcpy-x86-64-asm.o > perf-$(CONFIG_X86_64) += mem-memset-x86-64-asm.o > diff --git a/tools/perf/bench/cpu-online-map.c b/tools/perf/bench/cpu-online-map.c > new file mode 100644 > index 000000000000..4157d2220bf4 > --- /dev/null > +++ b/tools/perf/bench/cpu-online-map.c > @@ -0,0 +1,21 @@ > +#include > +#include > +#include > +#include > +#include "cpu-online-map.h" > + > +void compute_cpu_online_map(int *cpu_online_map) > +{ > + long ncpus_conf = sysconf(_SC_NPROCESSORS_CONF); > + cpu_set_t set; > + size_t j = 0; > + > + CPU_ZERO(&set); > + > + if (sched_getaffinity(0, sizeof(set), &set)) > + err(EXIT_FAILURE, "sched_getaffinity"); > + > + for (int i = 0; i < ncpus_conf; i++) > + if (CPU_ISSET(i, &set)) > + cpu_online_map[j++] = i; > +} > diff --git a/tools/perf/bench/cpu-online-map.h b/tools/perf/bench/cpu-online-map.h > new file mode 100644 > index 000000000000..0620a674c6d8 > --- /dev/null > +++ b/tools/perf/bench/cpu-online-map.h > @@ -0,0 +1,6 @@ > +#ifndef CPU_ONLINE_MAP_H > +#define CPU_ONLINE_MAP_H > + > +void compute_cpu_online_map(int *cpu_online_map); > + > +#endif > diff --git a/tools/perf/bench/futex-hash.c b/tools/perf/bench/futex-hash.c > index 58ae6ed8f38b..81314eea1cf5 100644 > --- a/tools/perf/bench/futex-hash.c > +++ b/tools/perf/bench/futex-hash.c > @@ -24,6 +24,7 @@ > #include > #include "bench.h" > #include "futex.h" > +#include "cpu-online-map.h" > > #include > #include > @@ -120,9 +121,11 @@ int bench_futex_hash(int argc, const char **argv) > int ret = 0; > cpu_set_t cpu; > struct sigaction act; > - unsigned int i, ncpus; > + unsigned int i; > + long ncpus; > pthread_attr_t thread_attr; > struct worker *worker = NULL; > + int *cpu_map; > > argc = parse_options(argc, argv, options, bench_futex_hash_usage, 0); > if (argc) { > @@ -132,6 +135,12 @@ int bench_futex_hash(int argc, const char **argv) > > ncpus = sysconf(_SC_NPROCESSORS_ONLN); > > + cpu_map = calloc(ncpus, sizeof(int)); > + if (!cpu_map) > + goto errmem; > + > + compute_cpu_online_map(cpu_map); > + > sigfillset(&act.sa_mask); > act.sa_sigaction = toggle_done; > sigaction(SIGINT, &act, NULL); > @@ -164,7 +173,7 @@ int bench_futex_hash(int argc, const char **argv) > goto errmem; > > CPU_ZERO(&cpu); > - CPU_SET(i % ncpus, &cpu); > + CPU_SET(cpu_map[i % ncpus], &cpu); > > ret = pthread_attr_setaffinity_np(&thread_attr, sizeof(cpu_set_t), &cpu); > if (ret) > @@ -217,6 +226,7 @@ int bench_futex_hash(int argc, const char **argv) > print_summary(); > > free(worker); > + free(cpu_map); > return ret; > errmem: > err(EXIT_FAILURE, "calloc"); > diff --git a/tools/perf/bench/futex-lock-pi.c b/tools/perf/bench/futex-lock-pi.c > index 08653ae8a8c4..42ab351b2c50 100644 > --- a/tools/perf/bench/futex-lock-pi.c > +++ b/tools/perf/bench/futex-lock-pi.c > @@ -15,6 +15,7 @@ > #include > #include "bench.h" > #include "futex.h" > +#include "cpu-online-map.h" > > #include > #include > @@ -113,7 +114,8 @@ static void *workerfn(void *arg) > return NULL; > } > > -static void create_threads(struct worker *w, pthread_attr_t thread_attr) > +static void create_threads(struct worker *w, pthread_attr_t thread_attr, > + int *cpu_map) > { > cpu_set_t cpu; > unsigned int i; > @@ -131,7 +133,7 @@ static void create_threads(struct worker *w, pthread_attr_t thread_attr) > worker[i].futex = &global_futex; > > CPU_ZERO(&cpu); > - CPU_SET(i % ncpus, &cpu); > + CPU_SET(cpu_map[i % ncpus], &cpu); > > if (pthread_attr_setaffinity_np(&thread_attr, sizeof(cpu_set_t), &cpu)) > err(EXIT_FAILURE, "pthread_attr_setaffinity_np"); > @@ -147,12 +149,18 @@ int bench_futex_lock_pi(int argc, const char **argv) > unsigned int i; > struct sigaction act; > pthread_attr_t thread_attr; > + int *cpu_map; > > argc = parse_options(argc, argv, options, bench_futex_lock_pi_usage, 0); > if (argc) > goto err; > > ncpus = sysconf(_SC_NPROCESSORS_ONLN); > + cpu_map = calloc(ncpus, sizeof(int)); > + if (!cpu_map) > + err(EXIT_FAILURE, "calloc"); > + > + compute_cpu_online_map(cpu_map); > > sigfillset(&act.sa_mask); > act.sa_sigaction = toggle_done; > @@ -180,7 +188,7 @@ int bench_futex_lock_pi(int argc, const char **argv) > pthread_attr_init(&thread_attr); > gettimeofday(&start, NULL); > > - create_threads(worker, thread_attr); > + create_threads(worker, thread_attr, cpu_map); > pthread_attr_destroy(&thread_attr); > > pthread_mutex_lock(&thread_lock); > @@ -218,6 +226,7 @@ int bench_futex_lock_pi(int argc, const char **argv) > print_summary(); > > free(worker); > + free(cpu_map); > return ret; > err: > usage_with_options(bench_futex_lock_pi_usage, options); > diff --git a/tools/perf/bench/futex-requeue.c b/tools/perf/bench/futex-requeue.c > index 1058c194608a..366961b901c6 100644 > --- a/tools/perf/bench/futex-requeue.c > +++ b/tools/perf/bench/futex-requeue.c > @@ -22,6 +22,7 @@ > #include > #include "bench.h" > #include "futex.h" > +#include "cpu-online-map.h" > > #include > #include > @@ -83,7 +84,7 @@ static void *workerfn(void *arg __maybe_unused) > } > > static void block_threads(pthread_t *w, > - pthread_attr_t thread_attr) > + pthread_attr_t thread_attr, int *cpu_map) > { > cpu_set_t cpu; > unsigned int i; > @@ -93,7 +94,7 @@ static void block_threads(pthread_t *w, > /* create and block all threads */ > for (i = 0; i < nthreads; i++) { > CPU_ZERO(&cpu); > - CPU_SET(i % ncpus, &cpu); > + CPU_SET(cpu_map[i % ncpus], &cpu); > > if (pthread_attr_setaffinity_np(&thread_attr, sizeof(cpu_set_t), &cpu)) > err(EXIT_FAILURE, "pthread_attr_setaffinity_np"); > @@ -116,12 +117,18 @@ int bench_futex_requeue(int argc, const char **argv) > unsigned int i, j; > struct sigaction act; > pthread_attr_t thread_attr; > + int *cpu_map; > > argc = parse_options(argc, argv, options, bench_futex_requeue_usage, 0); > if (argc) > goto err; > > ncpus = sysconf(_SC_NPROCESSORS_ONLN); > + cpu_map = calloc(ncpus, sizeof(int)); > + if (!cpu_map) > + err(EXIT_FAILURE, "calloc"); > + > + compute_cpu_online_map(cpu_map); > > sigfillset(&act.sa_mask); > act.sa_sigaction = toggle_done; > @@ -156,7 +163,7 @@ int bench_futex_requeue(int argc, const char **argv) > struct timeval start, end, runtime; > > /* create, launch & block all threads */ > - block_threads(worker, thread_attr); > + block_threads(worker, thread_attr, cpu_map); > > /* make sure all threads are already blocked */ > pthread_mutex_lock(&thread_lock); > @@ -210,6 +217,7 @@ int bench_futex_requeue(int argc, const char **argv) > print_summary(); > > free(worker); > + free(cpu_map); > return ret; > err: > usage_with_options(bench_futex_requeue_usage, options); > diff --git a/tools/perf/bench/futex-wake-parallel.c b/tools/perf/bench/futex-wake-parallel.c > index b4732dad9f89..5617bcd17e55 100644 > --- a/tools/perf/bench/futex-wake-parallel.c > +++ b/tools/perf/bench/futex-wake-parallel.c > @@ -21,6 +21,7 @@ > #include > #include "bench.h" > #include "futex.h" > +#include "cpu-online-map.h" > > #include > #include > @@ -119,7 +120,8 @@ static void *blocked_workerfn(void *arg __maybe_unused) > return NULL; > } > > -static void block_threads(pthread_t *w, pthread_attr_t thread_attr) > +static void block_threads(pthread_t *w, pthread_attr_t thread_attr, > + int *cpu_map) > { > cpu_set_t cpu; > unsigned int i; > @@ -129,7 +131,7 @@ static void block_threads(pthread_t *w, pthread_attr_t thread_attr) > /* create and block all threads */ > for (i = 0; i < nblocked_threads; i++) { > CPU_ZERO(&cpu); > - CPU_SET(i % ncpus, &cpu); > + CPU_SET(cpu_map[i % ncpus], &cpu); > > if (pthread_attr_setaffinity_np(&thread_attr, sizeof(cpu_set_t), &cpu)) > err(EXIT_FAILURE, "pthread_attr_setaffinity_np"); > @@ -205,6 +207,7 @@ int bench_futex_wake_parallel(int argc, const char **argv) > struct sigaction act; > pthread_attr_t thread_attr; > struct thread_data *waking_worker; > + int *cpu_map; > > argc = parse_options(argc, argv, options, > bench_futex_wake_parallel_usage, 0); > @@ -218,6 +221,13 @@ int bench_futex_wake_parallel(int argc, const char **argv) > sigaction(SIGINT, &act, NULL); > > ncpus = sysconf(_SC_NPROCESSORS_ONLN); > + > + cpu_map = calloc(ncpus, sizeof(int)); > + if (!cpu_map) > + err(EXIT_FAILURE, "calloc"); > + > + compute_cpu_online_map(cpu_map); > + > if (!nblocked_threads) > nblocked_threads = ncpus; > > @@ -259,7 +269,7 @@ int bench_futex_wake_parallel(int argc, const char **argv) > err(EXIT_FAILURE, "calloc"); > > /* create, launch & block all threads */ > - block_threads(blocked_worker, thread_attr); > + block_threads(blocked_worker, thread_attr, cpu_map); > > /* make sure all threads are already blocked */ > pthread_mutex_lock(&thread_lock); > @@ -295,5 +305,6 @@ int bench_futex_wake_parallel(int argc, const char **argv) > print_summary(); > > free(blocked_worker); > + free(cpu_map); > return ret; > } > diff --git a/tools/perf/bench/futex-wake.c b/tools/perf/bench/futex-wake.c > index 8c5c0b6b5c97..c0c71a769d2f 100644 > --- a/tools/perf/bench/futex-wake.c > +++ b/tools/perf/bench/futex-wake.c > @@ -22,6 +22,7 @@ > #include > #include "bench.h" > #include "futex.h" > +#include "cpu-online-map.h" > > #include > #include > @@ -89,7 +90,7 @@ static void print_summary(void) > } > > static void block_threads(pthread_t *w, > - pthread_attr_t thread_attr) > + pthread_attr_t thread_attr, int *cpu_map) > { > cpu_set_t cpu; > unsigned int i; > @@ -99,7 +100,7 @@ static void block_threads(pthread_t *w, > /* create and block all threads */ > for (i = 0; i < nthreads; i++) { > CPU_ZERO(&cpu); > - CPU_SET(i % ncpus, &cpu); > + CPU_SET(cpu_map[i % ncpus], &cpu); > > if (pthread_attr_setaffinity_np(&thread_attr, sizeof(cpu_set_t), &cpu)) > err(EXIT_FAILURE, "pthread_attr_setaffinity_np"); > @@ -122,6 +123,7 @@ int bench_futex_wake(int argc, const char **argv) > unsigned int i, j; > struct sigaction act; > pthread_attr_t thread_attr; > + int *cpu_map; > > argc = parse_options(argc, argv, options, bench_futex_wake_usage, 0); > if (argc) { > @@ -131,6 +133,12 @@ int bench_futex_wake(int argc, const char **argv) > > ncpus = sysconf(_SC_NPROCESSORS_ONLN); > > + cpu_map = calloc(ncpus, sizeof(int)); > + if (! cpu_map) > + err(EXIT_FAILURE, "calloc"); > + > + compute_cpu_online_map(cpu_map); > + > sigfillset(&act.sa_mask); > act.sa_sigaction = toggle_done; > sigaction(SIGINT, &act, NULL); > @@ -161,7 +169,7 @@ int bench_futex_wake(int argc, const char **argv) > struct timeval start, end, runtime; > > /* create, launch & block all threads */ > - block_threads(worker, thread_attr); > + block_threads(worker, thread_attr, cpu_map); > > /* make sure all threads are already blocked */ > pthread_mutex_lock(&thread_lock); > @@ -204,5 +212,6 @@ int bench_futex_wake(int argc, const char **argv) > print_summary(); > > free(worker); > + free(cpu_map); > return ret; > } > -- > 2.15.0