From: "Masami Hiramatsu (Google)" <mhiramat@kernel.org>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>,
Steven Rostedt <rostedt@goodmis.org>,
Florent Revest <revest@chromium.org>
Cc: linux-trace-kernel@vger.kernel.org,
LKML <linux-kernel@vger.kernel.org>,
Martin KaFai Lau <martin.lau@linux.dev>,
bpf <bpf@vger.kernel.org>, Alexei Starovoitov <ast@kernel.org>,
Jiri Olsa <jolsa@kernel.org>,
Alan Maguire <alan.maguire@oracle.com>,
Mark Rutland <mark.rutland@arm.com>,
linux-arch@vger.kernel.org,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>,
Huacai Chen <chenhuacai@kernel.org>,
WANG Xuerui <kernel@xen0n.name>,
Paul Walmsley <paul.walmsley@sifive.com>,
Palmer Dabbelt <palmer@dabbelt.com>,
Albert Ou <aou@eecs.berkeley.edu>,
Heiko Carstens <hca@linux.ibm.com>,
Vasily Gorbik <gor@linux.ibm.com>,
Alexander Gordeev <agordeev@linux.ibm.com>,
Christian Borntraeger <borntraeger@linux.ibm.com>,
Sven Schnelle <svens@linux.ibm.com>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
Dave Hansen <dave.hansen@linux.intel.com>,
x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
Arnd Bergmann <arnd@arndb.de>,
Masami Hiramatsu <mhiramat@kernel.org>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Subject: [PATCH v18 12/17] fprobe: Add fprobe_header encoding feature
Date: Sat, 26 Oct 2024 13:37:59 +0900 [thread overview]
Message-ID: <172991747946.443985.11014834036464028393.stgit@devnote2> (raw)
In-Reply-To: <172991731968.443985.4558065903004844780.stgit@devnote2>
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Fprobe store its data structure address and size on the fgraph return stack
by __fprobe_header. But most 64bit architecture can combine those to
one unsigned long value because 4 MSB in the kernel address are the same.
With this encoding, fprobe can consume less space on ret_stack.
This introduces asm/fprobe.h to define arch dependent encode/decode
macros. Note that since fprobe depends on CONFIG_HAVE_FUNCTION_GRAPH_FREGS,
currently only arm64, loongarch, riscv, s390 and x86 are supported.
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Huacai Chen <chenhuacai@kernel.org>
Cc: WANG Xuerui <kernel@xen0n.name>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: x86@kernel.org
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
arch/arm64/include/asm/fprobe.h | 7 +++++++
arch/loongarch/include/asm/fprobe.h | 5 +++++
arch/riscv/include/asm/fprobe.h | 9 +++++++++
arch/s390/include/asm/fprobe.h | 10 ++++++++++
arch/x86/include/asm/fprobe.h | 9 +++++++++
include/asm-generic/fprobe.h | 33 +++++++++++++++++++++++++++++++++
kernel/trace/fprobe.c | 29 +++++++++++++++++++++++++++++
7 files changed, 102 insertions(+)
create mode 100644 arch/arm64/include/asm/fprobe.h
create mode 100644 arch/loongarch/include/asm/fprobe.h
create mode 100644 arch/riscv/include/asm/fprobe.h
create mode 100644 arch/s390/include/asm/fprobe.h
create mode 100644 arch/x86/include/asm/fprobe.h
create mode 100644 include/asm-generic/fprobe.h
diff --git a/arch/arm64/include/asm/fprobe.h b/arch/arm64/include/asm/fprobe.h
new file mode 100644
index 000000000000..bbf254db878d
--- /dev/null
+++ b/arch/arm64/include/asm/fprobe.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_ARM64_FPROBE_H
+#define _ASM_ARM64_FPROBE_H
+
+#include <asm-generic/fprobe.h>
+
+#endif /* _ASM_ARM64_FPROBE_H */
\ No newline at end of file
diff --git a/arch/loongarch/include/asm/fprobe.h b/arch/loongarch/include/asm/fprobe.h
new file mode 100644
index 000000000000..68156a66873c
--- /dev/null
+++ b/arch/loongarch/include/asm/fprobe.h
@@ -0,0 +1,5 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LOONGARCH_FPROBE_H
+#define _ASM_LOONGARCH_FPROBE_H
+
+#endif /* _ASM_LOONGARCH_FPROBE_H */
\ No newline at end of file
diff --git a/arch/riscv/include/asm/fprobe.h b/arch/riscv/include/asm/fprobe.h
new file mode 100644
index 000000000000..51fc2ef3eda1
--- /dev/null
+++ b/arch/riscv/include/asm/fprobe.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_RISCV_FPROBE_H
+#define _ASM_RISCV_FPROBE_H
+
+#ifdef CONFIG_64BIT
+#include <asm-generic/fprobe.h>
+#endif
+
+#endif /* _ASM_RISCV_FPROBE_H */
\ No newline at end of file
diff --git a/arch/s390/include/asm/fprobe.h b/arch/s390/include/asm/fprobe.h
new file mode 100644
index 000000000000..84b94ba6e3a4
--- /dev/null
+++ b/arch/s390/include/asm/fprobe.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_S390_FPROBE_H
+#define _ASM_S390_FPROBE_H
+
+#include <asm-generic/fprobe.h>
+
+#undef FPROBE_HEADER_MSB_PATTERN
+#define FPROBE_HEADER_MSB_PATTERN 0
+
+#endif /* _ASM_S390_FPROBE_H */
\ No newline at end of file
diff --git a/arch/x86/include/asm/fprobe.h b/arch/x86/include/asm/fprobe.h
new file mode 100644
index 000000000000..c863518bef90
--- /dev/null
+++ b/arch/x86/include/asm/fprobe.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_X86_FPROBE_H
+#define _ASM_X86_FPROBE_H
+
+#ifdef CONFIG_64BIT
+#include <asm-generic/fprobe.h>
+#endif
+
+#endif /* _ASM_X86_FPROBE_H */
\ No newline at end of file
diff --git a/include/asm-generic/fprobe.h b/include/asm-generic/fprobe.h
new file mode 100644
index 000000000000..00f480748a1d
--- /dev/null
+++ b/include/asm-generic/fprobe.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * linux/include/asm-generic/fprobe.h
+ */
+#ifndef __ASM_GENERIC_FPROBE_H__
+#define __ASM_GENERIC_FPROBE_H__
+
+#include <linux/bits.h>
+
+#define ARCH_HAVE_ENCODE_FPROBE_HEADER
+
+#define FPROBE_HEADER_MSB_SIZE_SHIFT (BITS_PER_LONG - FPROBE_DATA_SIZE_BITS)
+#define FPROBE_HEADER_MSB_MASK \
+ GENMASK(FPROBE_HEADER_MSB_SIZE_SHIFT - 1, 0)
+#define FPROBE_HEADER_MSB_PATTERN \
+ GENMASK(BITS_PER_LONG - 1, FPROBE_HEADER_MSB_SIZE_SHIFT)
+
+#define arch_fprobe_header_encodable(fp) \
+ (((unsigned long)(fp) & ~FPROBE_HEADER_MSB_MASK) == \
+ FPROBE_HEADER_MSB_PATTERN)
+
+#define arch_encode_fprobe_header(fp, size) \
+ ((unsigned long)(fp) & FPROBE_HEADER_MSB_MASK) | \
+ ((unsigned long)(size) << FPROBE_HEADER_MSB_SIZE_SHIFT)
+
+#define arch_decode_fprobe_header_size(val) \
+ ((unsigned long)(val) >> FPROBE_HEADER_MSB_SIZE_SHIFT)
+
+#define arch_decode_fprobe_header_fp(val) \
+ (struct fprobe *)(((unsigned long)(val) & FPROBE_HEADER_MSB_MASK) | \
+ FPROBE_HEADER_MSB_PATTERN)
+
+#endif /* __ASM_GENERIC_FPROBE_H__ */
\ No newline at end of file
diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c
index ed9c1d79426a..949963f1f17a 100644
--- a/kernel/trace/fprobe.c
+++ b/kernel/trace/fprobe.c
@@ -13,6 +13,8 @@
#include <linux/slab.h>
#include <linux/sort.h>
+#include <asm/fprobe.h>
+
#include "trace.h"
#define FPROBE_IP_HASH_BITS 8
@@ -143,6 +145,31 @@ static int del_fprobe_hash(struct fprobe *fp)
return 0;
}
+#ifdef ARCH_HAVE_ENCODE_FPROBE_HEADER
+
+/* The arch should encode fprobe_header info into one unsigned long */
+#define FPROBE_HEADER_SIZE_IN_LONG 1
+
+static inline bool write_fprobe_header(unsigned long *stack,
+ struct fprobe *fp, unsigned int size_words)
+{
+ if (WARN_ON_ONCE(size_words > MAX_FPROBE_DATA_SIZE_WORD ||
+ !arch_fprobe_header_encodable(fp)))
+ return false;
+
+ *stack = arch_encode_fprobe_header(fp, size_words);
+ return true;
+}
+
+static inline void read_fprobe_header(unsigned long *stack,
+ struct fprobe **fp, unsigned int *size_words)
+{
+ *fp = arch_decode_fprobe_header_fp(*stack);
+ *size_words = arch_decode_fprobe_header_size(*stack);
+}
+
+#else
+
/* Generic fprobe_header */
struct __fprobe_header {
struct fprobe *fp;
@@ -173,6 +200,8 @@ static inline void read_fprobe_header(unsigned long *stack,
*size_words = fph->size_words;
}
+#endif
+
/*
* fprobe shadow stack management:
* Since fprobe shares a single fgraph_ops, it needs to share the stack entry
next prev parent reply other threads:[~2024-10-26 4:38 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-10-26 4:35 [PATCH v18 00/17] tracing: fprobe: function_graph: Multi-function graph and fprobe on fgraph Masami Hiramatsu (Google)
2024-10-26 4:35 ` [PATCH v18 01/17] fgraph: Pass ftrace_regs to entryfunc Masami Hiramatsu (Google)
2024-10-28 15:25 ` Will Deacon
2024-10-31 1:18 ` Masami Hiramatsu
2024-10-31 19:53 ` Steven Rostedt
2024-11-01 1:51 ` Masami Hiramatsu
2024-11-01 10:50 ` Steven Rostedt
2024-11-09 14:24 ` Masami Hiramatsu
2024-11-09 14:47 ` Masami Hiramatsu
2024-10-26 4:35 ` [PATCH v18 02/17] fgraph: Replace fgraph_ret_regs with ftrace_regs Masami Hiramatsu (Google)
2024-10-28 15:26 ` Will Deacon
2024-10-26 4:36 ` [PATCH v18 03/17] fgraph: Pass ftrace_regs to retfunc Masami Hiramatsu (Google)
2024-10-26 4:36 ` [PATCH v18 04/17] fprobe: Use ftrace_regs in fprobe entry handler Masami Hiramatsu (Google)
2024-10-26 4:36 ` [PATCH v18 05/17] fprobe: Use ftrace_regs in fprobe exit handler Masami Hiramatsu (Google)
2024-10-26 4:36 ` [PATCH v18 06/17] tracing: Add ftrace_partial_regs() for converting ftrace_regs to pt_regs Masami Hiramatsu (Google)
2024-10-28 15:27 ` Will Deacon
2024-10-26 4:36 ` [PATCH v18 07/17] tracing: Add ftrace_fill_perf_regs() for perf event Masami Hiramatsu (Google)
2024-10-26 4:37 ` [PATCH v18 08/17] tracing/fprobe: Enable fprobe events with CONFIG_DYNAMIC_FTRACE_WITH_ARGS Masami Hiramatsu (Google)
2024-10-26 4:37 ` [PATCH v18 09/17] bpf: Enable kprobe_multi feature if CONFIG_FPROBE is enabled Masami Hiramatsu (Google)
2024-10-26 4:37 ` [PATCH v18 10/17] ftrace: Add CONFIG_HAVE_FTRACE_GRAPH_FUNC Masami Hiramatsu (Google)
2024-10-26 4:37 ` [PATCH v18 11/17] fprobe: Rewrite fprobe on function-graph tracer Masami Hiramatsu (Google)
2024-11-01 19:28 ` Steven Rostedt
2024-11-09 14:55 ` Masami Hiramatsu
2024-10-26 4:37 ` Masami Hiramatsu (Google) [this message]
2024-11-01 14:22 ` [PATCH v18 12/17] fprobe: Add fprobe_header encoding feature Steven Rostedt
2024-11-09 15:10 ` Masami Hiramatsu
2024-11-12 14:44 ` Steven Rostedt
2024-11-12 23:43 ` Masami Hiramatsu
2024-10-26 4:38 ` [PATCH v18 13/17] tracing/fprobe: Remove nr_maxactive from fprobe Masami Hiramatsu (Google)
2024-10-26 4:38 ` [PATCH v18 14/17] selftests: ftrace: Remove obsolate maxactive syntax check Masami Hiramatsu (Google)
2024-10-26 4:38 ` [PATCH v18 15/17] selftests/ftrace: Add a test case for repeating register/unregister fprobe Masami Hiramatsu (Google)
2024-10-26 4:38 ` [PATCH v18 16/17] Documentation: probes: Update fprobe on function-graph tracer Masami Hiramatsu (Google)
2024-11-01 14:14 ` Steven Rostedt
2024-11-12 23:48 ` Masami Hiramatsu
2024-10-26 4:38 ` [PATCH v18 17/17] bpf: Add get_entry_ip() for arm64 Masami Hiramatsu (Google)
2024-10-28 8:27 ` [PATCH 0/1] s390: enable HAVE_FTRACE_GRAPH_FUNC Sven Schnelle
2024-10-31 1:20 ` Masami Hiramatsu
2024-10-28 8:27 ` [PATCH 1/1] s390/tracing: Enable HAVE_FTRACE_GRAPH_FUNC Sven Schnelle
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=172991747946.443985.11014834036464028393.stgit@devnote2 \
--to=mhiramat@kernel.org \
--cc=agordeev@linux.ibm.com \
--cc=alan.maguire@oracle.com \
--cc=alexei.starovoitov@gmail.com \
--cc=aou@eecs.berkeley.edu \
--cc=arnd@arndb.de \
--cc=ast@kernel.org \
--cc=borntraeger@linux.ibm.com \
--cc=bp@alien8.de \
--cc=bpf@vger.kernel.org \
--cc=catalin.marinas@arm.com \
--cc=chenhuacai@kernel.org \
--cc=dave.hansen@linux.intel.com \
--cc=gor@linux.ibm.com \
--cc=hca@linux.ibm.com \
--cc=hpa@zytor.com \
--cc=jolsa@kernel.org \
--cc=kernel@xen0n.name \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=martin.lau@linux.dev \
--cc=mathieu.desnoyers@efficios.com \
--cc=mingo@redhat.com \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
--cc=revest@chromium.org \
--cc=rostedt@goodmis.org \
--cc=svens@linux.ibm.com \
--cc=tglx@linutronix.de \
--cc=will@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox