From: Vadim Fedorenko <vadfed@meta.com>
To: Borislav Petkov <bp@alien8.de>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Andrii Nakryiko <andrii@kernel.org>,
Eduard Zingerman <eddyz87@gmail.com>,
Thomas Gleixner <tglx@linutronix.de>,
Yonghong Song <yonghong.song@linux.dev>,
Vadim Fedorenko <vadim.fedorenko@linux.dev>,
Mykola Lysenko <mykolal@fb.com>
Cc: <x86@kernel.org>, <bpf@vger.kernel.org>,
Vadim Fedorenko <vadfed@meta.com>,
Martin KaFai Lau <martin.lau@linux.dev>
Subject: [PATCH bpf-next v7 0/4] bpf: add cpu cycles kfuncss
Date: Mon, 18 Nov 2024 10:52:41 -0800 [thread overview]
Message-ID: <20241118185245.1065000-1-vadfed@meta.com> (raw)
This patchset adds 2 kfuncs to provide a way to precisely measure the
time spent running some code. The first patch provides a way to get cpu
cycles counter which is used to feed CLOCK_MONOTONIC_RAW. On x86
architecture it is effectively rdtsc_ordered() function while on other
architectures it falls back to __arch_get_hw_counter(). The second patch
adds a kfunc to convert cpu cycles to nanoseconds using shift/mult
constants discovered by kernel. JIT version is done for x86 for now, on
other architectures it falls back to slightly simplified version of
vdso_calc_ns.
Selftests are also added to check whether the JIT implementation is
correct and to show the simplest usage example.
Change log:
v6 -> v7:
* change boot_cpu_has() to cpu_feature_enabled() (Borislav)
* return constant clock_mode in __arch_get_hw_counter() call
v5 -> v6:
* added cover letter
* add comment about dropping S64_MAX manipulation in jitted
implementation of rdtsc_oredered (Alexey)
* add comment about using 'lfence;rdtsc' variant (Alexey)
* change the check in fixup_kfunc_call() (Eduard)
* make __arch_get_hw_counter() call more aligned with vDSO
implementation (Yonghong)
v4 -> v5:
* use #if instead of #ifdef with IS_ENABLED
v3 -> v4:
* change name of the helper to bpf_get_cpu_cycles (Andrii)
* Hide the helper behind CONFIG_GENERIC_GETTIMEOFDAY to avoid exposing
it on architectures which do not have vDSO functions and data
* reduce the scope of check of inlined functions in verifier to only 2,
which are actually inlined.
* change helper name to bpf_cpu_cycles_to_ns.
* hide it behind CONFIG_GENERIC_GETTIMEOFDAY to avoid exposing on
unsupported architectures.
v2 -> v3:
* change name of the helper to bpf_get_cpu_cycles_counter to
* explicitly mention what counter it provides (Andrii)
* move kfunc definition to bpf.h to use it in JIT.
* introduce another kfunc to convert cycles into nanoseconds as
* more meaningful time units for generic tracing use case (Andrii)
v1 -> v2:
* Fix incorrect function return value type to u64
* Introduce bpf_jit_inlines_kfunc_call() and use it in
mark_fastcall_pattern_for_call() to avoid clobbering in case
of running programs with no JIT (Eduard)
* Avoid rewriting instruction and check function pointer directly
in JIT (Alexei)
* Change includes to fix compile issues on non x86 architectures
Vadim Fedorenko (4):
bpf: add bpf_get_cpu_cycles kfunc
bpf: add bpf_cpu_cycles_to_ns helper
selftests/bpf: add selftest to check rdtsc jit
selftests/bpf: add usage example for cpu cycles kfuncs
arch/x86/net/bpf_jit_comp.c | 61 ++++++++++
arch/x86/net/bpf_jit_comp32.c | 33 ++++++
include/linux/bpf.h | 6 +
include/linux/filter.h | 1 +
kernel/bpf/core.c | 11 ++
kernel/bpf/helpers.c | 39 +++++++
kernel/bpf/verifier.c | 41 ++++++-
.../bpf/prog_tests/test_cpu_cycles.c | 35 ++++++
.../selftests/bpf/prog_tests/verifier.c | 2 +
.../selftests/bpf/progs/test_cpu_cycles.c | 25 +++++
.../selftests/bpf/progs/verifier_cpu_cycles.c | 104 ++++++++++++++++++
11 files changed, 352 insertions(+), 6 deletions(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/test_cpu_cycles.c
create mode 100644 tools/testing/selftests/bpf/progs/test_cpu_cycles.c
create mode 100644 tools/testing/selftests/bpf/progs/verifier_cpu_cycles.c
--
2.43.5
next reply other threads:[~2024-11-18 18:53 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-11-18 18:52 Vadim Fedorenko [this message]
2024-11-18 18:52 ` [PATCH bpf-next v7 1/4] bpf: add bpf_get_cpu_cycles kfunc Vadim Fedorenko
2024-11-19 11:18 ` Peter Zijlstra
2024-11-19 14:29 ` Vadim Fedorenko
2024-11-19 16:17 ` Peter Zijlstra
2024-11-19 18:03 ` Vadim Fedorenko
2024-11-19 19:16 ` Andrii Nakryiko
2024-11-19 19:27 ` Vadim Fedorenko
2024-11-18 18:52 ` [PATCH bpf-next v7 2/4] bpf: add bpf_cpu_cycles_to_ns helper Vadim Fedorenko
2024-11-19 11:28 ` Peter Zijlstra
2024-11-19 14:38 ` Vadim Fedorenko
2024-11-20 8:49 ` Peter Zijlstra
2024-11-20 13:39 ` Vadim Fedorenko
2024-11-18 18:52 ` [PATCH bpf-next v7 3/4] selftests/bpf: add selftest to check rdtsc jit Vadim Fedorenko
2024-11-18 18:52 ` [PATCH bpf-next v7 4/4] selftests/bpf: add usage example for cpu cycles kfuncs Vadim Fedorenko
2024-11-19 11:47 ` Peter Zijlstra
2024-11-19 14:45 ` Vadim Fedorenko
2024-11-20 8:51 ` Peter Zijlstra
2024-11-20 17:19 ` Vadim Fedorenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20241118185245.1065000-1-vadfed@meta.com \
--to=vadfed@meta.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bp@alien8.de \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=eddyz87@gmail.com \
--cc=martin.lau@linux.dev \
--cc=mykolal@fb.com \
--cc=tglx@linutronix.de \
--cc=vadim.fedorenko@linux.dev \
--cc=x86@kernel.org \
--cc=yonghong.song@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox